Reference-Free 3D Reconstruction of Brain Dissection Slabs via Learned Atlas Coordinates

Correlation of neuropathology with MRI has the potential to transfer microscopic signatures of pathology to in vivo scans. There is increasing interest in building these correlations from 3D reconstructed stacks of slab photographs, which are routinely taken during dissection at brain banks. These photographs bypass the need for ex vivo MRI, which is not widely accessible. However, existing methods either require a corresponding 3D reference (e.g., an ex vivo MRI scans, or a brain surface acquired with a structured light scanner) or a full stack of brain slabs, which severely limits applicability. Here we propose RefFree, a 3D reconstruction method for dissection photographs that does not require an external reference. RefFree coherently reconstructs a 3D volume for an arbitrary set of slabs (including a single slab) using predicted 3D coordinates in the standard atlas space (MNI) as guidance. To support RefFree’s pipeline, we train an atlas coordinate prediction network that estimates the coordinate map from a 2D photograph, using synthetic photographs generated from digitally sliced 3D MRI data with randomized appearance for enhanced generalization. As a by-product, RefFree can propagate information (e.g., anatomical labels) from atlas space to one single photograph even without reconstruction. Experiments on simulated and real data show that, when all slabs are available, RefFree achieves performance comparable to existing classical methods but at substantially higher speed. Moreover, RefFree yields accurate reconstruction and registration for partial stacks or even a single slab. Our code is available at https://github.com/lintian-a/reffree.

💡 Research Summary

This paper presents “RefFree,” a novel learning-based framework for 3D reconstruction of brain dissection slab photographs without requiring any external, specimen-specific 3D reference data (e.g., ex vivo MRI or surface scans). The core challenge addressed is correlating neuropathology findings with in vivo MRI, where slab photographs serve as a crucial intermediary. Existing methods are limited as they necessitate either a full stack of slabs or a 3D reference for guidance, restricting their applicability to retrospective brain bank datasets where such data is often incomplete or unavailable.

RefFree innovates by reformulating the 3D reconstruction problem as a direct prediction of canonical 3D atlas coordinates. The method operates in two stages. First, a 2D convolutional neural network (CNN) independently processes each input slab photograph, along with its normalized position along the anterior-posterior axis, to predict a dense map of 3D coordinates in the standard MNI (Montreal Neurological Institute) atlas space for every pixel. Second, given these predicted coordinate maps from one or multiple available slabs, an optimal set of global affine transformations that jointly map all slabs into the coherent MNI space is estimated analytically via least-squares fitting. This design inherently supports flexible input scenarios: a full stack, a partial stack, or even a single slab.

A significant contribution is the development of a sophisticated synthetic data engine to train the coordinate prediction network, as paired real slab photos with ground-truth MNI coordinates are nonexistent. The engine generates anatomically plausible slab images by digitally slicing 3D MRI-derived segmentation maps and their corresponding non-linearly registered MNI coordinate fields. It incorporates extensive domain randomization to simulate real-world variability: randomized 3D affine transformations (for anatomical pose and cutting angle), simulated slab-specific non-rigid deformations, random cropping (for framing variability), and randomized appearance via Gaussian mixture models conditioned on anatomical labels (for tissue contrast and lighting). This enables robust generalization to real autopsy photographs without any manual annotation.

Comprehensive experiments validate RefFree’s performance. On a synthetic benchmark derived from 11 public MRI cohorts, the method demonstrates accurate coordinate prediction. Evaluation on two real post-mortem datasets (comprising 149 single-hemisphere and 26 bi-hemispheric stacks) shows that when a full slab stack is available, RefFree achieves reconstruction fidelity comparable to state-of-the-art, optimization-based reference-guided methods but at a substantially higher computational speed. Crucially, RefFree maintains accurate reconstruction and registration performance with partial stacks or single slabs, a scenario where prior methods fail. An additional practical byproduct is the ability to project anatomical segmentations from the MNI atlas onto any single photograph in near real-time using the predicted coordinate map, enabling immediate anatomical interpretation.

In summary, RefFree provides a unified, efficient, and reference-free solution for 3D reconstruction from brain dissection photographs, dramatically increasing the feasibility of large-scale neuroimaging-pathology correlation studies on existing brain bank resources, even those with incomplete data.