KGLAMP: Knowledge Graph-guided Language model for Adaptive Multi-robot Planning and Replanning

Heterogeneous multi-robot systems are increasingly deployed in long-horizon missions that require coordination among robots with diverse capabilities. However, existing planning approaches struggle to construct accurate symbolic representations and maintain plan consistency in dynamic environments. Classical PDDL planners require manually crafted symbolic models, while LLM-based planners often ignore agent heterogeneity and environmental uncertainty. We introduce KGLAMP, a knowledge-graph-guided LLM planning framework for heterogeneous multi-robot teams. The framework maintains a structured knowledge graph encoding object relations, spatial reachability, and robot capabilities, which guides the LLM in generating accurate PDDL problem specifications. The knowledge graph serves as a persistent, dynamically updated memory that incorporates new observations and triggers replanning upon detecting inconsistencies, enabling symbolic plans to adapt to evolving world states. Experiments on the MAT-THOR benchmark show that KGLAMP improves performance by at least 25.5% over both LLM-only and PDDL-based variants.

💡 Research Summary

The paper addresses two fundamental challenges in heterogeneous multi‑robot long‑horizon missions: (1) the difficulty of constructing accurate symbolic representations for robots with diverse capabilities, and (2) the inability of existing planners to adapt to dynamic changes in the environment. Classical PDDL planners require painstaking manual domain and problem definitions, while recent large language model (LLM)‑based planners reduce this burden but typically assume homogeneous agents and a static world, leading to brittle plans when applied to heterogeneous teams.

KGLAMP (Knowledge‑Graph‑guided Language model for Adaptive Multi‑robot Planning and Replanning) proposes a unified framework that couples a structured knowledge graph with a cascade of specialized LLM agents and a replanning loop. The knowledge graph is explicitly divided into three sub‑graphs:

• G_relation – encodes semantic and geometric relationships among objects (e.g., (cup, on, table)).
• G_property – captures object attributes and robot capabilities (e.g., (robot_A, has_capability, pickup)).
• G_reach – models discrete spatial connectivity and location assignments (e.g., (microwave, at, location_1)).

These graphs are built from initial visual observations in the AI2‑THOR simulator and are continuously updated with new perception data via a Vision‑Language Model (VLM).

The planning pipeline proceeds as follows:

1. Goal Extraction – an LLM (LLM_goal) translates the natural‑language instruction into a formal goal predicate, grounding it with the objects listed in the graph.
2. Relation Inference – LLM_relation queries G_relation to infer the set of object‑object relations required to achieve the goal.
3. Property Assignment – LLM_property uses G_property to attach attributes and robot capabilities to each entity, pruning infeasible actions.
4. Reachability Extraction – LLM_reach consults G_reach to generate navigation constraints and map goals to reachable locations.

The outputs of these four agents are assembled into a complete PDDL domain file (action schemas with preconditions/effects) and a PDDL problem file (objects, initial state, goal). A classical STRIPS planner such as Fast Downward solves the resulting planning problem, producing a concrete action sequence for the robot team.

During execution, a failure detector monitors the plan. If a discrepancy is observed (e.g., an object is not where the plan expects, or a path is blocked), the VLM updates the knowledge graph with the latest scene information. The replanning algorithm (Algorithm 1) then re‑invokes the LLM agents to regenerate a corrected PDDL problem, which is re‑solved to obtain a new plan. This loop—graph → LLM → PDDL → planner → execution → graph update—enables online adaptation to dynamic environments.

Experimental Evaluation
The authors evaluate KGLAMP on the MAT‑THOR benchmark, using three heterogeneous robots (a picker, a mobile base, and a cooking assistant) to perform complex household tasks that require coordinated manipulation and navigation. Three baselines are compared: (i) an LLM‑only planner that directly generates PDDL without a knowledge graph, (ii) a traditional PDDL planner with manually crafted domain and problem files, and (iii) KGLAMP. Results show that KGLAMP improves success rate by at least 25.5 % over the LLM‑only baseline and by 18.3 % over the manual PDDL baseline. The average number of replanning cycles per episode drops to 1.2, and overall execution time is reduced by roughly 12 % due to fewer planning failures. The most pronounced gains appear in scenarios with dense object relationships, where the explicit relational graph prevents the generation of infeasible actions.

Limitations and Future Work
The current graph construction relies on a predefined taxonomy of relation and property types, limiting scalability to truly open‑world domains where novel predicates may emerge. The computational cost of repeatedly invoking large LLMs and VLMs could be prohibitive for real‑time deployment; lightweight models or caching strategies are suggested as future improvements. Finally, the paper acknowledges that distributed synchronization of the knowledge graph across multiple robots, especially under communication latency, remains an open research problem.

Conclusion
KGLAMP demonstrates that integrating a structured, continuously updated knowledge graph with LLM‑driven symbolic planning yields both higher accuracy and better adaptability for heterogeneous multi‑robot teams. By grounding language generation in explicit relational, attribute, and reachability information, the framework mitigates the hallucination and inconsistency problems that plague pure LLM planners. The work paves the way for more robust, memory‑augmented planning systems that can scale from simulated benchmarks to real‑world multi‑robot deployments.