RecoWorld: Building Simulated Environments for Agentic Recommender Systems

We present RecoWorld, a blueprint for building simulated environments tailored to agentic recommender systems. Such environments give agents a proper training space where they can learn from errors without impacting real users. RecoWorld distinguishes itself with a dual-view architecture: a simulated user and an agentic recommender engage in multi-turn interactions aimed at maximizing user retention. The user simulator reviews recommended items, updates its mindset, and when sensing potential user disengagement, generates reflective instructions. The agentic recommender adapts its recommendations by incorporating these user instructions and reasoning traces, creating a dynamic feedback loop that actively engages users. This process leverages the exceptional reasoning capabilities of modern LLMs. We explore diverse content representations within the simulator, including text-based, multimodal, and semantic ID modeling, and discuss how multi-turn RL enables the recommender to refine its strategies through iterative interactions. RecoWorld also supports multi-agent simulations, allowing creators to simulate the responses of targeted user populations. It marks an important first step toward recommender systems where users and agents collaboratively shape personalized information streams. We envision new interaction paradigms where “user instructs, recommender responds,” jointly optimizing user retention and engagement.

💡 Research Summary

**
RecoWorld is a blueprint for building simulated environments specifically designed for agentic recommender systems—systems that act as autonomous agents capable of reasoning, planning, and following explicit user instructions. The paper argues that traditional offline metrics (Recall@N, NDCG) and online A/B tests are insufficient for evaluating such agents because they either suffer from exposure bias or provide slow feedback loops that risk real users. RecoWorld addresses this gap by introducing a dual‑view architecture composed of (1) a user simulator powered by large language models (LLMs) and (2) an agentic recommender that can ingest instructions, perform multi‑turn reasoning, and adapt its recommendation list dynamically.

User Simulator
The simulator mimics individual or group user behavior across multiple modalities (clicks, skips, watch time, shares, etc.). When it detects a risk of disengagement—e.g., a user is about to leave—it generates natural‑language instructions such as “show me more interesting content.” These instructions can be explicit (textual commands) or implicit (derived from behavioral patterns like a preference for long‑form videos). The simulator also supports diverse user profiles (age, location, interests) and can instantiate multi‑agent scenarios, enabling researchers to test how a recommender performs across heterogeneous populations.

Agentic Recommender
The recommender incorporates LLM‑based reasoning modules that interpret user instructions, combine them with historical interaction data, demographic context, and content metadata, and then re‑configure the downstream retrieval‑ranking pipeline. The system produces a new list of items, possibly accompanied by clarifying questions, and the cycle repeats. Each turn yields a trajectory of actions and observations; from these, RecoWorld extracts reward signals such as total session time, number of clicks, and dropout risk. These “pseudo‑rewards” are fed into reinforcement‑learning (RL) algorithms or offline policy‑optimization methods, allowing the agent to learn policies that balance immediate relevance (high NDCG) with long‑term user retention.

Technical Contributions

1. Gym‑like API – RecoWorld offers an OpenAI‑Gym‑style interface, making it easy to plug in different agents or simulators and to benchmark them under identical conditions.
2. Multi‑turn Interaction Loop – The environment records full interaction trajectories, enabling trajectory‑level evaluation and self‑critique. An LLM‑based judge scores each trajectory against predefined rubrics, retaining only successful ones for training.
3. Content Representation Flexibility – The simulator supports text‑only, multimodal (image, video, audio), and semantic‑ID representations, allowing the agent to reason over heterogeneous content spaces.
4. Instruction‑Following Evaluation – By measuring how well the agent satisfies user‑generated instructions, RecoWorld provides a direct metric for the emerging capability of “instruction‑following recommenders.”
5. Community Leaderboard – A shared leaderboard encourages practitioners to submit their agents, fostering a collaborative ecosystem and enabling fair comparison across a spectrum of tasks from simple to complex.

Use Cases Highlighted

• Evaluating instruction‑following ability: Simulated users issue natural‑language preferences; the agent’s success is measured without real‑world exposure.
• Creator strategy testing: Content producers can experiment with posting frequency, topic shifts, or controversial material in a risk‑free sandbox.
• Exploration for new or marginal users: By simulating groups with similar features, the environment supplies pseudo‑rewards that guide contextual bandits toward safe exploration.
• Leaderboard for agentic RecSys: Standardizes evaluation across diverse internal recommender systems, supporting curriculum learning and agent‑agent collaboration.

Findings and Hypotheses
The authors hypothesize three regimes: (a) high NDCG + high retention → strong exploitation; (b) high NDCG + low retention → over‑specialization or repetitive content; (c) low NDCG + high retention → effective exploration delivering novel, engaging items. Preliminary experiments suggest that the simulated trajectories correlate with these patterns, indicating that RecoWorld can surface nuanced trade‑offs invisible to offline metrics.

Limitations and Future Directions

• Simulation fidelity: LLM‑generated instructions and behaviors may not capture the full complexity of real humans, risking a simulation‑to‑reality gap.
• Hallucination and bias: LLMs can produce inaccurate or biased feedback, which could mislead the learning agent.
• Reward design: Over‑emphasizing long‑term retention might degrade immediate user satisfaction; multi‑objective reward shaping is needed.
  Future work includes domain adaptation techniques to bridge the simulated‑real gap, richer multi‑objective reward formulations, and hybrid online‑offline training pipelines that gradually incorporate real user data.

Conclusion
RecoWorld represents the first environment that enables instruction‑following, multi‑turn, agentic recommender systems to be trained and evaluated safely before deployment. By coupling LLM‑driven user simulation with a flexible RL‑compatible interface, it opens a pathway for rapid experimentation, community benchmarking, and the development of next‑generation recommender agents that can reason, adapt, and collaborate with users to maximize long‑term engagement.