Density-Aware Farthest Point Sampling

We focus on training machine learning regression models in scenarios where the availability of labeled training data is limited due to computational constraints or high labeling costs. Thus, selecting suitable training sets from unlabeled data is essential for balancing performance and efficiency. For the selection of the training data, we focus on passive and model-agnostic sampling methods that only consider the data feature representations. We derive an upper bound for the expected prediction error of Lipschitz continuous regression models that linearly depends on the weighted fill distance of the training set: a quantity we can estimate simply by considering the data features. We introduce ‘‘Density-Aware Farthest Point Sampling’’ (DA-FPS), a novel sampling method. We prove that DA-FPS provides approximate minimizers for a data-driven estimation of the weighted fill distance, thereby aiming at minimizing our derived bound. We conduct experiments using two regression models across three datasets. The results demonstrate that DA-FPS significantly reduces the mean absolute prediction error compared to other sampling strategies.

💡 Research Summary

The paper addresses a practical problem that arises in many scientific and engineering domains: how to select a small, highly informative training set for regression when labeling is extremely costly (e.g., quantum‑chemical simulations for molecular property prediction). While active learning methods can iteratively query the most informative points, they require repeated model training and are unsuitable when the labeling budget is limited to a single batch. The authors therefore focus on passive, model‑agnostic sampling strategies that rely solely on the feature representations of the unlabeled pool.

The core theoretical contribution is an upper bound on the expected prediction error of any Lipschitz‑continuous regression model trained on a selected subset (L). By introducing a novel quantity called the weighted fill distance, \