Optimizing Retrieval Components for a Shared Backbone via Component-Wise Multi-Stage Training

Recent advances in embedding-based retrieval have enabled dense retrievers to serve as core infrastructure in many industrial systems, where a single retrieval backbone is often shared across multiple downstream applications. In such settings, retrieval quality directly constrains system performance and extensibility, while coupling model selection, deployment, and rollback decisions across applications. In this paper, we present empirical findings and a system-level solution for optimizing retrieval components deployed as a shared backbone in production legal retrieval systems. We adopt a multi-stage optimization framework for dense retrievers and rerankers, and show that different retrieval components exhibit stage-dependent trade-offs. These observations motivate a component-wise, mixed-stage configuration rather than relying on a single uniformly optimal checkpoint. The resulting backbone is validated through end-to-end evaluation and deployed as a shared retrieval service supporting multiple industrial applications.

💡 Research Summary

The paper addresses the practical problem of optimizing a single dense‑retrieval backbone that is shared across multiple downstream legal‑AI applications. In such a setting, the quality of the retrieval layer directly limits the performance, stability, and extensibility of all dependent services, making it infeasible to maintain separate models for each task due to deployment complexity and operational cost.
To tackle this, the authors propose a component‑wise multi‑stage training framework inspired by curriculum learning. The same model family (Qwen3‑Embedding‑4B for dense retrieval and Qwen3‑Reranker‑4B for re‑ranking) is refined through three successive stages, each emphasizing a different supervision signal:

• Stage 1 – Large‑scale alignment: Weak supervision from heterogeneous legal sources (statutes, case law, synthetic queries) is used to learn broad semantic alignment. This stage maximizes coverage and improves Recall@K, providing a solid foundation for candidate generation.

• Stage 2 – Hard‑sample refinement: Automatically mined hard negatives—instances where the current model fails—are added to the training set. The model learns to discriminate fine‑grained relevance differences, which translates into higher MRR and nDCG for the re‑ranking component.

• Stage 3 – Robustness calibration: High‑quality queries and dynamically refreshed challenging negatives are employed to enhance robustness under ambiguous or adversarial inputs. While this stage further raises overall recall, it can slightly degrade fine‑grained ranking precision, illustrating a trade‑off between robustness and exact relevance discrimination.

Experiments are conducted on two Chinese legal QA benchmarks: CSAID (complex, context‑rich queries) and STARD (standardized statute retrieval). Results show a clear progression for the embedding models: Stage 3 consistently achieves the highest recall across all K values, often reaching the same recall with 30 % fewer candidates compared to earlier stages. For the re‑ranker, Stage 2 yields the best MRR and nDCG on CSAID, while Stage 3 marginally outperforms on STARD, confirming that the optimal training stage differs between components.

Guided by these findings, the authors adopt a mixed‑stage configuration for end‑to‑end evaluation: the Stage 3 embedding model combined with the Stage 2 re‑ranker. This hybrid pipeline outperforms the baseline (un‑fine‑tuned Qwen3 models) on both benchmarks, improving recall, MRR, and nDCG simultaneously.

System‑level analysis of recall‑budget curves demonstrates practical benefits: later‑stage embeddings allow latency‑sensitive services to reduce the number of retrieved candidates without sacrificing coverage, thereby lowering downstream re‑ranking cost and overall response time. Conversely, latency‑tolerant scenarios can afford larger candidate pools to exploit the full power of the re‑ranker.

A lightweight online A/B test with 389 real user queries shows the new backbone is preferred in 54.6 % of non‑tie cases, with an average latency increase of only 0.10 seconds (≈6.7 %). This confirms that the performance gains translate into real‑world user satisfaction while keeping overhead minimal.

In summary, the study demonstrates that stage‑dependent training does not yield a single universally optimal checkpoint; instead, different retrieval components reach their sweet spots at different stages. By selecting the best checkpoint for each component and combining them, a shared retrieval backbone can be both high‑performing and operationally efficient, offering a scalable solution for industrial legal‑AI systems.