Astroinformatics of galaxies and quasars: a new general method for photometric redshifts estimation

With the availability of the huge amounts of data produced by current and future large multi-band photometric surveys, photometric redshifts have become a crucial tool for extragalactic astronomy and cosmology. In this paper we present a novel method, called Weak Gated Experts (WGE), which allows to derive photometric redshifts through a combination of data mining techniques. \noindent The WGE, like many other machine learning techniques, is based on the exploitation of a spectroscopic knowledge base composed by sources for which a spectroscopic value of the redshift is available. This method achieves a variance \sigma^2(\Delta z)=2.3x10^{-4} (\sigma^2(\Delta z) =0.08), where \Delta z = z_{phot} - z_{spec}) for the reconstruction of the photometric redshifts for the optical galaxies from the SDSS and for the optical quasars respectively, while the Root Mean Square (RMS) of the \Delta z variable distributions for the two experiments is respectively equal to 0.021 and 0.35. The WGE provides also a mechanism for the estimation of the accuracy of each photometric redshift. We also present and discuss the catalogs obtained for the optical SDSS galaxies, for the optical candidate quasars extracted from the DR7 SDSS photometric dataset {The sample of SDSS sources on which the accuracy of the reconstruction has been assessed is composed of bright sources, for a subset of which spectroscopic redshifts have been measured.}, and for optical SDSS candidate quasars observed by GALEX in the UV range. The WGE method exploits the new technological paradigm provided by the Virtual Observatory and the emerging field of Astroinformatics.

💡 Research Summary

The paper introduces a novel machine‑learning framework called Weak Gated Experts (WGE) for estimating photometric redshifts from the massive multi‑band data streams produced by modern and upcoming astronomical surveys. WGE combines unsupervised clustering with supervised regression in a hierarchical “gating” architecture. First, the full feature space (optical colours, magnitudes, and, for quasars, UV fluxes) is partitioned into a set of clusters using a data‑driven clustering algorithm. Within each cluster a dedicated regression model (e.g., linear regression, multilayer perceptron, or random forest) is trained on objects that have spectroscopic redshifts, forming a local expert that captures the specific colour‑redshift mapping of that region. When a new object is presented, a gating function determines the most appropriate cluster(s) and combines the predictions of the corresponding experts, typically via a weighted average. This local‑expert strategy mitigates the global non‑linearity and degeneracy problems that plague single‑model approaches, especially for quasars whose spectra exhibit strong emission‑line shifts across filters.

The authors build their knowledge base (KB) from the Sloan Digital Sky Survey Data Release 7 (SDSS‑DR7). For galaxies they use ∼120 000 spectroscopic sources with ugriz photometry; for quasars they augment the optical data with GALEX ultraviolet measurements, yielding ∼80 000 training objects. The experiments follow a classic supervised learning pipeline: training, validation (to avoid over‑fitting), and testing on an independent set. Performance is quantified by the variance σ²(Δz) and the root‑mean‑square error (RMS) of Δz = z_phot − z_spec. For galaxies the method achieves σ²(Δz)=2.3 × 10⁻⁴ and RMS = 0.021, comparable to or slightly better than established techniques such as polynomial fitting, neural networks, or support‑vector machines. For quasars the results are σ²(Δz)=0.08 and RMS = 0.35, demonstrating that even in the more challenging high‑redshift regime the average error remains within acceptable limits.

A distinctive feature of WGE is its built‑in error‑estimation module. By tracking the residual variance of each local regression model, the algorithm assigns an individual photometric‑redshift uncertainty (σ_phot) to every prediction. These uncertainties are then used to flag potential catastrophic outliers—objects whose Δz deviates dramatically from the spectroscopic value. In the test samples roughly 5 % of sources are marked as outliers, most of which lie near cluster boundaries or suffer from incomplete spectroscopic coverage.

The authors argue that WGE is well suited to the “astroinformatics” paradigm: clustering can be parallelized across compute nodes, making the method scalable to the billions of sources expected from surveys like LSST or Euclid. The local‑expert design reduces over‑fitting risk and allows heterogeneous regression techniques to be employed where they are most effective. Moreover, the per‑object error estimates provide immediate feedback for follow‑up spectroscopic campaigns, enabling efficient allocation of telescope time.

Nevertheless, the study has limitations. The KB is biased toward bright, low‑redshift galaxies and relatively luminous quasars, so the generalization to faint objects or very high‑redshift (z > 2) quasars remains untested. Hyper‑parameter choices—such as the number of clusters, the gating function form, and the specific regression algorithm per cluster—are shown to affect performance, yet the paper does not present an automated tuning strategy or a systematic sensitivity analysis. Comparisons with other state‑of‑the‑art methods are limited to basic RMS and variance metrics; computational cost, memory footprint, and processing speed are not discussed, leaving open the question of practical efficiency in production pipelines. Finally, the method does not address temporal variability (e.g., variable AGN) or multi‑epoch data, which are increasingly important in time‑domain surveys.

In summary, Weak Gated Experts offers an innovative, modular approach to photometric redshift estimation that blends clustering‑driven locality with supervised regression, delivers per‑object uncertainty estimates, and includes an outlier‑detection mechanism. Its design aligns with the needs of large‑scale survey science and the emerging field of astroinformatics. Future work should expand the training set to include fainter and higher‑redshift sources, develop automated hyper‑parameter optimization, and benchmark the method against contemporary deep‑learning frameworks to fully assess its competitiveness for next‑generation astronomical data challenges.