SAGEO Arena: A Realistic Environment for Evaluating Search-Augmented Generative Engine Optimization

Search-Augmented Generative Engines (SAGE) have emerged as a new paradigm for information access, bridging web-scale retrieval with generative capabilities to deliver synthesized answers. This shift has fundamentally reshaped how web content gains exposure online, giving rise to Search-Augmented Generative Engine Optimization (SAGEO), the practice of optimizing web documents to improve their visibility in AI-generated responses. Despite growing interest, no evaluation environment currently supports comprehensive investigation of SAGEO. Specifically, existing benchmarks lack end-to-end visibility evaluation of optimization strategies, operating on pre-determined candidate documents that abstract away retrieval and reranking preceding generation. Moreover, existing benchmarks discard structural information (e.g., schema markup) present in real web documents, overlooking the rich signals that search systems actively leverage in practice. Motivated by these gaps, we introduce SAGEO Arena, a realistic and reproducible environment for stage-level SAGEO analysis. Our objective is to jointly target search-oriented optimization (SEO) and generation-centric optimization (GEO). To achieve this, we integrate a full generative search pipeline over a large-scale corpus of web documents with rich structural information. Our findings reveal that existing approaches remain largely impractical under realistic conditions and often degrade performance in retrieval and reranking. We also find that structural information helps mitigate these limitations, and that effective SAGEO requires tailoring optimization to each pipeline stage. Overall, our benchmark paves the way for realistic SAGEO evaluation and optimization beyond simplified settings.

💡 Research Summary

The paper addresses a critical gap in the emerging field of Search‑Augmented Generative Engine Optimization (SAGEO). While Search‑Augmented Generative Engines (SAGE) combine large‑scale retrieval with large language model (LLM) generation to produce synthesized answers, there has been no realistic benchmark that evaluates how web documents can be optimized for visibility across the entire generative search pipeline. Existing benchmarks such as GEO‑Bench, AutoGEO, and C‑SEO Bench operate on a fixed set of pre‑selected candidate documents and ignore structural signals (titles, meta descriptions, headings, schema markup) that real‑world search systems heavily rely on.

To fill this void, the authors introduce SAGEO Arena, a reproducible environment that (1) integrates a full retrieval‑augmented generation pipeline (Retriever → Reranker → Generator) and (2) provides a large‑scale corpus of 170 k web pages from nine diverse domains, each enriched with structured metadata. The corpus is built by sampling 300 queries per domain (2,700 queries total), retrieving up to 100 results per query via the Google Custom Search API, and crawling each URL to extract both body text and the aforementioned structural fields. After cleaning, the final collection contains 171,003 documents, with an average of 63 candidates per query.

The benchmark’s evaluation protocol proceeds as follows: for a given query q, the pipeline is run to obtain a generated answer A_q that includes inline citations. A target document d_tgt that appears in the generation stage is selected as a baseline. The document is then subjected to various optimization strategies (e.g., keyword insertion, meta‑tag tweaking, schema augmentation), re‑indexed, and the pipeline is re‑executed with the same query. Visibility is measured at three stages: (1) retrieval (whether d_tgt appears in the top‑k set), (2) reranking (its rank after the cross‑encoder), and (3) generation (whether it is cited). The authors report Hit Rate (fraction of queries where the target is cited) and Rank Change (difference in rank across stages) as primary metrics.

Empirical results reveal two pivotal insights. First, approaches that focus solely on body‑text manipulation—common in prior GEO work—often degrade retrieval performance, causing the optimized document to fall out of the candidate pool and never reach the generator. This demonstrates that the retrieval stage is highly sensitive to structural cues rather than raw textual fluency. Second, optimizing structural signals (titles, meta descriptions, headings, and especially schema/JSON‑LD markup) substantially improves performance across all stages. Structural information acts as the primary hook for the retriever, while enriched body content remains important for reranking and for being selected as a citation during generation.

Motivated by these findings, the authors propose Stage‑aware SAGEO, a method that tailors optimization to the distinct priorities of each pipeline component. In the retrieval stage, the method emphasizes schema and meta‑data enhancements; in reranking, it balances keyword relevance with semantic richness; and in generation, it introduces citation‑friendly phrasing and explicit source markers. Across multiple baselines, Stage‑aware SAGEO achieves the highest Hit Rate and the most favorable Rank Change, confirming that a one‑size‑fits‑all optimization is insufficient.

The paper also conducts domain‑level analyses, showing that the relative importance of structural versus textual signals varies across topics (e.g., medical queries rely more on schema, while technical queries benefit from detailed headings). Additionally, the authors examine the interaction between optimization and different retriever/reranker models, confirming that the observed trends hold under both dense vector retrievers and sparse BM25 baselines.

Limitations are acknowledged. The current LLM generator is prompt‑sensitive, so optimization gains may fluctuate with different prompting strategies. The quality and completeness of schema markup in the wild are uneven, potentially limiting the generalizability of structural‑only optimizations. Moreover, the benchmark focuses on textual and markup signals, leaving multimodal content (images, video) and user interaction data unexplored.

Future work suggested includes extending the corpus with multimodal annotations, integrating user feedback loops for dynamic optimization, and evaluating SAGEO strategies on commercial generative search services (e.g., Bing Copilot, Google Search).

In summary, SAGEO Arena provides the first end‑to‑end, structurally‑rich benchmark for evaluating search‑augmented generative engine optimization. By demonstrating the critical role of structured web signals and the necessity of stage‑specific optimization, the work lays a solid foundation for both academic research and practical SEO/GEO practices in the era of AI‑generated search answers.