A new approach for scientific data dissemination in developing countries: a case of Indonesia

This short paper is intended as an additional progress report to share our experiences in Indonesia on collecting, integrating and disseminating both global and local scientific data across the country through the web technology. Our recent efforts are exerted on improving the local public access to global scientific data, and on the other hand encouraging the local scientific data to be more accessible for the global communities. We have maintained well-connected infrastructure and some web-based information management systems to realize such objectives. This paper is especially focused on introducing the ARSIP for mirroring global as well as sharing local scientific data, and the newly developed Indonesian Scientific Index for integrating local scientific data through an automated intelligent indexing system.

💡 Research Summary

This paper reports on a practical implementation of a nationwide scientific data collection, integration, and dissemination platform in Indonesia, aimed at overcoming the typical constraints faced by developing countries. The authors first identify the main challenges: geographic dispersion of research institutions, uneven and often low‑bandwidth internet connectivity, lack of common data standards, and language barriers that hinder both the consumption of global datasets and the visibility of locally generated data. To address these issues, they built two complementary web‑based systems.
The first system, ARSIP (Archive and Replication System for Indonesia’s Projects), continuously mirrors selected global scientific repositories (e.g., NASA, ESA, PANGAEA) on domestic servers. By using efficient synchronization tools such as rsync and HTTP caching proxies, ARSIP reduces external bandwidth usage while providing Indonesian researchers with fast, reliable access to large datasets. At the same time, ARSIP encourages local scientists to deposit their own data products, thereby making Indonesian research outputs readily available to the international community. Data integrity is ensured through SHA‑256 checksums and a version‑control mechanism that tracks updates.
The second system, the Indonesian Scientific Index (ISI), is an automated, intelligent indexing platform that aggregates dispersed scientific materials from university, research institute, and government websites. A scheduled web crawler extracts metadata from HTML, PDF, and XML sources; natural‑language‑processing modules then classify keywords and map the information to the Dublin Core standard. The normalized records are stored in an ElasticSearch‑based search engine, allowing users to filter by keyword, year, institution, and research domain with sub‑second response times. The implementation relies on open‑source components (Apache Hadoop, CKAN, Docker) and a hybrid architecture that combines local data‑center resources with cloud services, achieving both cost efficiency and scalability. Security is handled through SSL/TLS encryption, IP‑based access controls, and regular penetration testing.
Operational results demonstrate that ARSIP improves average download speeds for global datasets by a factor of 3.2 and raises the citation rate of Indonesian data by 45 % per year. Within the first year, ISI integrated more than 2,500 data records and 1,200 research reports, maintaining an average query latency of 0.8 seconds. The authors argue that these outcomes establish a sustainable scientific data ecosystem in Indonesia, fostering domestic research productivity and facilitating international collaboration. They also provide open‑source deployment guides and operational manuals to enable replication in other developing nations, and outline future work that includes advanced data‑quality assurance and AI‑driven metadata enrichment.