Monte Carlo Rendering to Diffusion Curves with Differential BEM

We present a method for generating vector graphics, in the form of diffusion curves, directly from noisy samples produced by a Monte Carlo renderer. While generating raster images from 3D geometry via Monte Carlo raytracing is commonplace, there is no corresponding practical approach for robustly and directly extracting editable vector images with shading information from 3D geometry. To fill this gap, we formulate the problem as a stochastic optimization problem over the space of geometries and colors of diffusion curve handles, and solve it with the Levenberg-Marquardt algorithm. At the core of our method is a novel differential boundary element method (BEM) framework that reconstructs colors from diffusion curve handles and computes gradients with respect to their parameters, requiring the expensive matrix factorization only once at the beginning of the optimization. Unlike triangulation-based techniques that require a clean domain decomposition, our method is robust to geometrically challenging scenarios, such as intersecting diffusion curves, and to color noise in the target image, enabling the direct use of noisy Monte Carlo samples without requiring a converged, error-free input image. We demonstrate the robustness and broad applicability of our approach across several test cases. Finally, we highlight several open questions raised by our work, which spans both theory and applications.

💡 Research Summary

The paper introduces a novel pipeline that converts noisy Monte Carlo rendering samples directly into vector graphics represented by diffusion curves, bypassing the traditional raster‑to‑vector workflow. Diffusion curves encode color information as double‑sided Dirichlet boundary conditions on a set of 2‑D curves (handles); the interior color field is obtained by solving the Laplace equation with these boundaries. The authors formulate the inverse problem: given a target image supplied only as Monte Carlo color samples, find the geometry and colors of a sparse set of diffusion‑curve handles that minimise a reconstruction loss.

To solve this, they adopt a stochastic optimisation framework based on the Levenberg–Marquardt (LM) algorithm. LM requires the Jacobian of the reconstructed color field with respect to the handle parameters (positions, orientations, and side colors) and an approximation of the Hessian. Computing these derivatives efficiently is the core technical contribution: a “differential boundary element method” (Differential BEM). Traditional BEM rewrites the Laplace PDE as a boundary integral equation (BIE) and solves for the interior field by discretising only the boundary. The differential version extends this by analytically differentiating the BIE with respect to the handle parameters, yielding closed‑form expressions for ∂u/∂θ. This avoids repeated dense matrix factorizations; the system matrix, which depends only on the fixed domain and the discretisation of the handles, is factorised once at the start of optimisation. Subsequent LM iterations update only the right‑hand side, allowing rapid recomputation of the solution and its gradients.

The method explicitly models double‑sided boundaries using two jump quantities: a color jump w_d (the difference between the two side colors) and a normal‑derivative jump w_c (difference of normal derivatives of the side colors). These appear in the BIE as source terms and enable the representation of sharp color discontinuities as well as smooth gradient changes across a handle. A compatibility condition (zero net flux of normal derivatives) is enforced to guarantee a well‑posed BIE.

Monte Carlo noise is handled directly: the loss and its gradients are estimated by averaging over a modest number of noisy samples. Because the BEM solution is deterministic, the variance of the gradient estimate is dominated by the Monte Carlo sampling error, which can be reduced by increasing the sample count without altering the underlying optimisation machinery. Consequently, the pipeline works with partially converged renders, eliminating the need for a high‑quality raster reference.

Experimental results cover a variety of 2‑D and 3‑D scenes, including intersecting curves, complex shading, and soft shadows. Starting from a dense, possibly intersecting set of random handles, the optimisation jointly refines geometry, colors, and prunes unnecessary handles. The reconstruction error (RMSE) decreases steadily, and the final diffusion‑curve images are visually indistinguishable from the ground‑truth Monte Carlo renders. The authors demonstrate that the one‑time matrix factorisation dominates the runtime; subsequent LM iterations scale linearly with the number of handles, enabling near‑interactive performance for moderate‑size problems.

Key contributions are: (1) a differential BEM formulation that provides exact sensitivities of the Laplace solution with respect to handle parameters, (2) integration of this formulation with LM to jointly optimise geometry and color in a single stochastic optimisation loop, and (3) a robust pipeline that accepts noisy Monte Carlo samples as direct input, removing the rasterisation bottleneck and showing resilience to intersecting handles and high‑frequency noise.

The paper also outlines future directions: extending the approach to high‑frequency detail by incorporating adaptive handle refinement, applying the method to curved surfaces in 3‑D (e.g., diffusion curves on meshes), accelerating the initial matrix factorisation and Monte Carlo sampling on GPUs for real‑time editing, and generalising the framework to other elliptic PDEs (Poisson, Helmholtz) for broader vector‑graphics applications. By bridging physically‑based rendering, numerical PDE solvers, and vector‑graphics optimisation, this work opens a new avenue for directly generating editable, resolution‑independent vector art from stochastic renderers.