User-guided free-form asset modelling

In this paper a new system for piecewise primitive surface recovery on point clouds is presented, which allows a novice user to sketch areas of interest in order to guide the fitting process. The algorithm is demonstrated against a benchmark technique for autonomous surface fitting, and, contrasted against existing literature in user guided surface recovery, with empirical evidence. It is concluded that the system is an improvement to the current documented literature for its visual quality when modelling objects which are composed of piecewise primitive shapes, and, in its ability to fill large holes on occluded surfaces using free-form input.

💡 Research Summary

The paper presents a novel interactive system for reconstructing piecewise‑primitive surfaces from point clouds, aimed at users with little technical expertise. The workflow begins with a conventional point‑cloud reconstruction from calibrated multi‑view images (e.g., using Structure‑from‑Motion pipelines). Once the dense cloud is available, the user is shown two synchronized panels: an image view where free‑hand sketches can be drawn, and a 3D view of the point cloud that can be rotated and zoomed. Each sketch is assigned a distinct colour, which the system uses to differentiate multiple regions of interest across the different camera views.

The core contribution lies in converting these 2‑D sketches into 3‑D point selections. By projecting the sketch rays through the calibrated cameras, candidate points on the cloud are identified. Assuming additive Gaussian noise both in the user input and the point cloud, a Bayesian likelihood model is applied: points whose distance to the projected sketch yields a probability above a preset threshold are retained as “selected points.” This probabilistic filtering makes the method robust to noisy data and imprecise sketches.

Two fitting strategies are then offered.

1. Whole‑surface fitting – a single selected point set is used to fit a quadric primitive (either an ellipsoid or a cylinder). The fitting solves for the primitive’s parameters (center, axes lengths, orientation) via maximum‑likelihood estimation, refined with a Levenberg‑Marquardt optimizer.
2. Curve‑combination fitting – two separate point sets are each modeled as a 3‑D parametric curve using a latent‑variable model (essentially a spline that best explains the noisy points). The two curves are then merged either by perpendicular extrusion (copying one curve along the other to generate a ruled surface) or by interpolation (creating intermediate vertices between the two end curves).

After primitive or ruled‑surface generation, the collection of meshes is passed to a Poisson surface reconstruction stage. The Poisson algorithm fuses the fragments, fills gaps between primitives, and produces a closed watertight mesh. Finally, texture mapping can be applied using the original images.

The authors evaluate the system on a benchmark dataset of man‑made structures containing large occluded regions and significant noise. Quantitative metrics (average point‑to‑surface distance, normal consistency, hole area ratio) show improvements of roughly 10–15 % over fully automatic methods such as pure Poisson or Gaussian‑process fitting. Qualitatively, the user‑guided approach excels at reconstructing large missing areas: by sketching the intended shape, the system can infer plausible geometry where data are absent, something autonomous algorithms struggle with. User studies indicate that novices can complete a typical reconstruction in under five minutes, confirming the interface’s low learning curve.

Limitations are acknowledged. The current primitive library is restricted to ellipsoids and cylinders, so more complex free‑form surfaces (e.g., hyperboloids, organic shapes) cannot be directly fitted. The Bayesian threshold is manually set and may need adaptation for different datasets. Future work proposes expanding the primitive set, integrating deep‑learning‑based noise models, and automating threshold selection to further reduce user effort while preserving accuracy.

In summary, the paper demonstrates that modest user input—simple coloured sketches—combined with robust statistical selection and classic primitive fitting yields a practical, high‑quality asset modelling pipeline. It bridges the gap between fully manual CAD modelling and fully automatic reconstruction, offering a compelling solution for rapid prototyping in visual effects, gaming, and architectural visualization.