A preliminary XML-based search system for planetary data

Planetary sciences can benefit from several different sources of information, i.e. ground-based or near Earth-based observations, space missions and laboratory experiments. The data collected from these sources, however, are spread over a number of smaller, separate communities and stored through different facilities: this makes it difficult to integrate them. The IDIS initiative, born in the context of the Europlanet project, performed a pilot study of the viability and the issues to be overcome in order to create an integrated search system for planetary data. As part of the results of such pilot study, the IDIS Small Bodies and Dust node developed a search system based on a preliminary XML data model. Here we introduce the goals of the IDIS initiative and describe the structure and the working of this search system. The source code of the search system is released under GPL license to allow people interested in participating to the IDIS initiative both as developers and as data providers to familiarise with the search environment and to allow the creation of volunteer nodes to be integrated into the existing network.

💡 Research Summary

The paper presents the results of a pilot study conducted under the Europlanet project’s IDIS (Integrated and Distributed Information Service) initiative, which aims to overcome the fragmentation of planetary science data across disparate communities and storage facilities. Planetary data originate from ground‑based telescopes, near‑Earth observations, space missions, and laboratory experiments, each typically managed by its own repository with proprietary metadata schemas. This heterogeneity hampers researchers who must query multiple systems separately to assemble a comprehensive dataset.

IDIS proposes a unified search platform built on a common XML‑based metadata model. The model captures five principal dimensions: target object (e.g., asteroid, dust particle), observation method (optical, radar, spectroscopy), data type (image, spectrum, time series), provider information, and access rights. Each dimension is further broken down into standardized attributes, enabling consistent description of heterogeneous resources.

The Small Bodies and Dust node implemented a prototype search system that operationalizes this model. The architecture consists of three layers. The ingestion layer converts local databases into XML documents conforming to the IDIS schema, using transformation scripts supplied by the node. The indexing and query layer leverages an XQuery engine coupled with Apache Solr to create inverted indexes of the XML metadata, supporting complex queries that combine keyword, temporal, spatial, and parameter filters. The presentation layer provides a web‑based portal where users can construct queries via keyword fields, dropdown menus, and an interactive map, and retrieve results in a ranked list. A RESTful API is also exposed, allowing external applications or other IDIS nodes to query the service programmatically.

The prototype is written in PHP and runs on an Apache server. Its modular plugin architecture permits new data providers to be added with minimal effort, and the entire codebase is released under the GNU GPL, encouraging community contributions and the establishment of volunteer nodes.

Performance testing on a pilot dataset of several thousand asteroid observations and dust experiment records yielded average response times of 1–2 seconds. However, the reliance on XML parsing and bulk index updates leads to scalability concerns when the repository grows to hundreds of thousands of records. Additionally, variations in terminology and units among contributing institutions caused occasional mismatches in query results, highlighting the need for stricter metadata harmonization.

Future work outlined by the authors includes migrating the metadata representation to JSON‑LD, introducing a SPARQL endpoint for semantic querying, and adopting distributed indexing and caching mechanisms to improve scalability. Automated tools for mapping local schemas to the IDIS standard are also planned to reduce the burden on data providers. Ultimately, the goal is to evolve the IDIS network into a single, seamless portal that gives planetary scientists worldwide instant access to all relevant observational and experimental data, thereby accelerating discovery and cross‑disciplinary collaboration.