Forget Superresolution, Sample Adaptively (when Path Tracing)

Real-time path tracing increasingly operates under extremely low sampling budgets, often below one sample per pixel, as rendering complexity, resolution, and frame-rate requirements continue to rise. While super-resolution is widely used in production, it uniformly sacrifices spatial detail and cannot exploit variations in noise, reconstruction difficulty, and perceptual importance across the image. Adaptive sampling offers a compelling alternative, but existing end-to-end approaches rely on approximations that break down in sparse regimes. We introduce an end-to-end adaptive sampling and denoising pipeline explicitly designed for the sub-1-spp regime. Our method uses a stochastic formulation of sample placement that enables gradient estimation despite discrete sampling decisions, allowing stable training of a neural sampler at low sampling budgets. To better align optimization with human perception, we propose a tonemapping-aware training pipeline that integrates differentiable filmic operators and a state-of-the-art perceptual loss, preventing oversampling of regions with low visual impact. In addition, we introduce a gather-based pyramidal denoising filter and a learnable generalization of albedo demodulation tailored to sparse sampling. Our results show consistent improvements over uniform sparse sampling, with notably better reconstruction of perceptually critical details such as specular highlights and shadow boundaries, and demonstrate that adaptive sampling remains effective even at minimal budgets.

💡 Research Summary

The paper tackles the emerging challenge of real‑time path tracing under extreme sampling constraints—often less than one sample per pixel (spp). While super‑resolution (SR) techniques are widely adopted in production pipelines, they uniformly upscale a low‑resolution render and therefore cannot exploit spatial variations in Monte‑Carlo noise, reconstruction difficulty, or perceptual importance. Adaptive sampling promises to allocate the limited budget where it matters most, but existing end‑to‑end methods rely on gradient approximations that break down when the sample count is very low.

To overcome this, the authors propose a fully differentiable adaptive sampling and denoising pipeline designed explicitly for the sub‑1 spp regime. The key technical innovations are:

1. Stochastic Sample Discretization – The sampler network predicts a continuous per‑pixel density map. Instead of deterministic rounding, the fractional densities are interpreted as probabilities and sampled via Bernoulli trials. This yields an unbiased estimator of the expected image and allows back‑propagation of gradients even when many pixels receive zero samples. The total budget is split into a small uniform portion (1/8) for numerical stability and a larger adaptive portion (7/8).

2. Tone‑Mapping‑Aware Perceptual Training – Recognizing that most losses are defined in linear HDR space, the authors embed a family of differentiable parametric tone‑mapping operators into the training loop. The tone‑mapped output is then compared to the reference using the state‑of‑the‑art perceptual metric MILO. This aligns optimization with human visual sensitivity, preventing wasteful sampling in dark or over‑exposed regions where large linear errors are invisible.

3. Sparse‑Friendly Denoiser – Rather than predicting pixel colors directly, the denoiser predicts weights for a pyramidal gather filter. The multi‑scale gather structure aggregates information from coarser levels, making the filter robust when the majority of pixels lack samples. Additionally, a learnable albedo demodulation module generalizes traditional albedo‑based texture preservation to cases where explicit albedo buffers are unavailable.

4. Shared Latent Feature Space – The sampler and denoiser share intermediate feature maps, enabling a cooperative loop: the sampler learns to place samples where the denoiser’s reconstruction error is high, while the denoiser adapts its filter parameters to the sampler’s allocation pattern. The tone‑mapped output is fed back to the sampler for the next frame, forming a perceptual feedback loop that can be replaced at inference time with any engine‑specific tone mapper.

The authors evaluate the method on a variety of scenes at 0.25 spp (equivalent to 50 % SR). Compared against NVIDIA’s DLSS‑4 Ray Reconstruction, the NPPD method, and recent adaptive sampling baselines, their approach achieves higher PSNR, SSIM, and lower LPIPS. Qualitatively, it preserves fine details such as specular highlights on metallic surfaces, sharp shadow boundaries, and disoccluded geometry that other methods blur or miss entirely. Moreover, increasing the resolution while keeping the same per‑pixel budget further improves quality, demonstrating that the learned sampling strategy scales with resolution.

The pipeline requires a differentiable tone mapper only during training; at runtime any standard tone mapper can be used, making integration into existing game engines straightforward. Code and a synthetic dataset will be released upon acceptance, providing the community with a practical baseline for ultra‑low‑budget real‑time path tracing. In summary, the paper delivers a robust, perception‑driven, end‑to‑end solution that makes adaptive sampling viable even when the renderer can afford fewer than one sample per pixel, effectively challenging the dominance of super‑resolution in modern real‑time graphics.