A Practical Framework of Key Performance Indicators for Multi-Robot Lunar and Planetary Field Tests

Robotic prospecting for critical resources on the Moon, such as ilmenite, rare earth elements, and water ice, requires robust exploration methods given the diverse terrain and harsh environmental conditions. Although numerous analog field trials address these goals, comparing their results remains challenging because of differences in robot platforms and experimental setups. These missions typically assess performance using selected, scenario-specific engineering metrics that fail to establish a clear link between field performance and science-driven objectives. In this paper, we address this gap by deriving a structured framework of KPI from three realistic multi-robot lunar scenarios reflecting scientific objectives and operational constraints. Our framework emphasizes scenario-dependent priorities in efficiency, robustness, and precision, and is explicitly designed for practical applicability in field deployments. We validated the framework in a multi-robot field test and found it practical and easy to apply for efficiency- and robustness-related KPI, whereas precision-oriented KPI require reliable ground-truth data that is not always feasible to obtain in outdoor analog environments. Overall, we propose this framework as a common evaluation standard enabling consistent, goal-oriented comparison of multi-robot field trials and supporting systematic development of robotic systems for future planetary exploration.

💡 Research Summary

The paper addresses a critical gap in lunar and planetary field testing: the lack of a unified, science‑driven set of performance metrics for multi‑robot missions. While numerous analog field trials have demonstrated the technical feasibility of heterogeneous robot teams—combining wheeled, legged, and aerial platforms—their results are typically evaluated with a narrow set of engineering‑centric metrics inherited from competition scoring rubrics. These metrics do not directly reflect the scientific objectives that motivate resource prospecting on the Moon, such as mapping ilmenite, rare‑earth‑element (REE) rich ejecta, or permanently shadowed water‑ice deposits.

To bridge this gap, the authors first construct three realistic lunar mission scenarios grounded in ESA’s Space Resource Strategy. Scenario 1 focuses on ilmenite in volcanic plains, requiring vertical profiling of TiO₂/FeO proxies via impact craters and horizontal validation with in‑situ spectrometers. Scenario 2 targets REE‑rich KREEP material on the Aristarchus plateau, emphasizing the need to resolve both the horizontal distribution of thorium‑related gamma signatures and the vertical accessibility of the deposits. Scenario 3 addresses water‑ice characterization in polar permanently shadowed regions (PSRs), where thermophysical properties, ice form, and environmental stability must be quantified for future ISRU.

Each scenario follows a three‑phase operational concept: Phase 0 (pre‑mission orbital data analysis), Phase 1 (wide‑area scouting by “scout” robots equipped with non‑contact sensors such as spectral cameras, gamma‑ray, and neutron spectrometers), and Phase 2 (targeted in‑situ measurements by “scientist” robots carrying instruments like APXS, LIBS, XRF, or laser‑induced mass spectrometers). This structure allows the mission to iteratively refine a prospecting grid based on increasingly detailed data.

From these scenarios the authors derive a structured KPI framework organized around three priority axes: efficiency, robustness, and precision. Efficiency KPIs quantify how quickly and with how much energy a robot team can cover the prescribed area, including metrics such as area‑per‑hour, energy‑per‑square‑meter, map resolution, and map‑to‑orbital‑data correlation. Robustness KPIs capture the system’s ability to tolerate failures and environmental challenges, measuring obstacle‑avoidance success rate, fault‑recovery count, communication loss frequency, and inter‑robot collaboration success. Precision KPIs assess scientific fidelity: positional accuracy of sampling points, spectral similarity to ground‑truth laboratory analyses, and the degree of agreement between in‑situ measurements and orbital proxies. The authors note that precision‑oriented KPIs demand reliable ground‑truth data, which is often difficult to obtain in outdoor analog environments, limiting their practical applicability.

A field validation was conducted using a heterogeneous robot team in a terrestrial analog of the ilmenite scenario. The team successfully collected efficiency and robustness data automatically through onboard logging and post‑mission analysis, confirming that these KPIs are straightforward to compute in real‑time. However, precision KPIs proved more challenging: aligning field spectroscopic measurements with orbital datasets required extensive calibration and ground‑truth sampling that was not feasible within the test’s time constraints. This result underscores the need for dedicated ground‑truth campaigns when precision metrics are essential.

A key contribution of the framework is its scenario‑dependent weighting scheme. For missions where scientific return hinges on accurate compositional mapping (e.g., REE or water‑ice), precision KPIs receive higher weight, whereas missions constrained by harsh terrain or communication limitations (e.g., polar ice) prioritize robustness. This flexibility enables mission designers to tailor the KPI set to the specific risk profile and scientific goals of a given exploration campaign.

The paper also emphasizes measurability and practical deployment. Energy efficiency is expressed as power per unit area rather than absolute consumption, allowing comparison across missions of different scales. Robustness incorporates team‑level metrics such as collaborative task success, reflecting the growing importance of heterogeneous swarms. By grounding KPI definitions in observable, sensor‑derived quantities, the framework avoids reliance on abstract or hard‑to‑measure concepts that have limited prior use.

In conclusion, the authors present a comprehensive, science‑aligned KPI framework that can serve as a common evaluation standard for multi‑robot lunar and planetary field tests. The framework’s modular design, scenario‑specific priority weighting, and focus on measurable quantities make it suitable for cross‑project benchmarking, competition scoring, and iterative system development. Adoption of this framework could accelerate the maturation of autonomous multi‑robot exploration strategies, ultimately supporting the systematic development of robotic systems for future sustainable presence on the Moon and beyond.