The rapid evolution of Large Language Models (LLMs) is predicated on the quality and diversity of post-training datasets. However, a critical dichotomy persists: while models are rigorously benchmarked, the data fueling them remains a black box--characterized by opaque composition, uncertain provenance, and a lack of systematic evaluation. This opacity hinders reproducibility and obscures the causal link between data characteristics and model behaviors. To bridge this gap, we introduce OpenDataArena (ODA), a holistic and open platform designed to benchmark the intrinsic value of post-training data. ODA establishes a comprehensive ecosystem comprising four key pillars: (i) a unified training-evaluation pipeline that ensures fair, open comparisons across diverse models (e.g., Llama, Qwen) and domains; (ii) a multi-dimensional scoring framework that profiles data quality along tens of distinct axes; (iii) an interactive data lineage explorer to visualize dataset genealogy and dissect component sources; and (iv) a fully open-source toolkit for training, evaluation, and scoring to foster data research. Extensive experiments on ODA--covering over 120 training datasets across multiple domains on 22 benchmarks, validated by more than 600 training runs and 40 million processed data points--reveal non-trivial insights. Our analysis uncovers the inherent trade-offs between data complexity and task performance, identifies redundancy in popular benchmarks through lineage tracing, and maps the genealogical relationships across datasets. We release all results, tools, and configurations to democratize access to high-quality data evaluation. Rather than merely expanding a leaderboard, ODA envisions a shift from trial-and-error data curation to a principled science of Data-Centric AI, paving the way for rigorous studies on data mixing laws and the strategic composition of foundation models.
💡 Deep Analysis
📄 Full Content
OpenDataArena: A Fair and Open Arena for Benchmarking
Post-Training Dataset Value
OpenDataArena Team1
1Shanghai Artificial Intelligence Laboratory, OpenDataLab
The rapid evolution of Large Language Models (LLMs) is predicated on the quality and
diversity of post-training datasets. However, a critical dichotomy persists: while models
are rigorously benchmarked, the data fueling them remains a “black box”—characterized by
opaque composition, uncertain provenance, and a lack of systematic evaluation. This opacity
hinders reproducibility and obscures the causal link between data characteristics and model
behaviors. To bridge this gap, we introduce OpenDataArena (ODA), a holistic and open
platform designed to benchmark the intrinsic value of post-training data. ODA establishes
a comprehensive ecosystem comprising four key pillars: (i) a unified training–evaluation
pipeline that ensures fair, open comparisons across diverse models (e.g., Llama, Qwen) and
domains; (ii) a multi-dimensional scoring framework that profiles data quality along tens
of distinct axes; (iii) an interactive data lineage explorer to visualize dataset genealogy and
dissect component sources; and (iv) a fully open-source toolkit for training, evaluation, and
scoring to foster data research. Extensive experiments on ODA—covering over 120 training
datasets across multiple domains on 22 benchmarks, validated by more than 600 training runs
and 40 million processed data points—reveal non-trivial insights. Our analysis uncovers the
inherent trade-offs between data complexity and task performance, identifies redundancy in
popular benchmarks through lineage tracing, and maps the “genealogical” relationships across
datasets. We release all results, tools, and configurations to democratize access to high-quality
data evaluation. Rather than merely expanding a leaderboard, ODA envisions a shift from
trial-and-error data curation to a principled science of Data-Centric AI, paving the way for
rigorous studies on data mixing laws and the strategic composition of foundation models.
Date: December 17, 2025
Correspondence: Lijun Wu, wulijun@pjlab.org.cn
Project Page: https://opendataarena.github.io/
Toolkit: https://github.com/OpenDataArena/OpenDataArena-Tool
HuggingFace: https://huggingface.co/OpenDataArena/datasets
1
Introduction
The rapid evolution of Large Language Models (LLMs), such as the GPT series [6, 2, 24], Qwen
series [4, 60, 59] and Llama series [53, 54, 19], has marked a paradigm shift in Artificial Intelligence
(AI), demonstrating remarkable capabilities in understanding, generation, and reasoning. While
much of the community’s focus has been on architectural innovations [36] and scaling laws [26], a
critical determinant of these models’ ultimate performance and alignment lies in the post-training
phase. This stage, encompassing Supervised Fine-Tuning (SFT) and alignment processes [42], relies
heavily on curated datasets to sculpt a base model’s behavior, imbuing it with the ability to follow
instructions, engage in dialog, and adhere to human values. The quality, diversity, and composition
of this post-training data are therefore not just influential but are arguably the key ingredients that
1
arXiv:2512.14051v1 [cs.AI] 16 Dec 2025
OpenDataArena: A Fair and Open Arena for Benchmarking Post-Training Dataset Value
Figure 1: Overview of the OpenDataArena framework. We provide four integral components: a
Data Value Leaderboard for standardized benchmarking, a Multi-dimension Data Scorer for granular
quality assessment, a Data Analysis Platform for lineage and composition tracing, and an Open-source
Evaluation Toolkit to ensure reproducibility.
transform a powerful predictive engine into a helpful and reliable AI assistant [49, 18, 50, 52, 8].
Despite its pivotal role, the landscape of post-training datasets is fraught with opacity and lacks a stan-
dardized evaluation protocol. The creation and selection of datasets is often an ad-hoc process, leading
to a proliferation of resources with varying quality, such as those generated through distillation from
proprietary models like Alpaca [50] or crowd-sourcing efforts like Dolly [13]. While some studies have
argued for the power of small, high-quality datasets [67], and others have begun to analyze the factors
that make data effective for alignment [37], the community still lacks a systematic and fair methodology
to evaluate dataset quality and its downstream impact. This opacity hinders scientific progress by
making it difficult to reproduce results, understand the source of performance gains, and efficiently al-
locate resources for data curation. The fundamental question of “what constitutes a ‘good’ dataset?”
remains largely unanswered in a quantifiable and generalizable way.
To bridge this gap, we present OpenDataArena (ODA), a fair, open, and transparent platform designed
to systematically benchmark the value of post-training datasets. Our primary contributions through
ODA are fourfold (as sho
