Petascale Cloud Supercomputing for Terapixel Visualization of a Digital Twin

Background: Photo-realistic terapixel visualization is computationally intensive and to date there have been no such visualizations of urban digital twins, the few terapixel visualizations that exist have looked towards space rather than earth. Objective: our aims are: creating a scalable cloud supercomputer software architecture for visualization; a photo-realistic terapixel 3D visualization of urban IoT data supporting daily updates; a rigorous evaluation of cloud supercomputing for our application. Method: we migrated the Blender Cycles path tracer to the public cloud within a new software framework designed to scale to petaFLOP performance. Results: we demonstrate we can compute a terapixel visualization in under one hour, the system scaling at 98% efficiency to use 1024 public cloud GPU nodes delivering 14 petaFLOPS. The resulting terapixel image supports interactive browsing of the city and its data at a wide range of sensing scales. Conclusion: The GPU compute resource available in the cloud is greater than anything available on our national supercomputers providing access to globally competitive resources. The direct financial cost of access, compared to procuring and running these systems, was low. The indirect cost, in overcoming teething issues with cloud software development, should reduce significantly over time.

💡 Research Summary

The paper presents a cloud‑based petascale computing platform that enables photo‑realistic, terapixel‑scale visualization of an urban digital twin. Recognizing that existing terapixel visualizations have focused on space rather than terrestrial environments, the authors set out to (1) design a scalable software architecture capable of reaching petaFLOP performance, (2) generate a terapixel 3D rendering of a city that integrates daily‑updated IoT sensor data, and (3) rigorously evaluate the cloud approach against traditional supercomputing.

To achieve these goals, the authors migrated the Blender Cycles path‑tracer to a public‑cloud environment. They containerized the renderer, orchestrated it with Kubernetes, and built an automatic scaling manager that can provision up to 1,024 GPU nodes (NVIDIA A100) on demand. Rendering work is divided into 256‑megapixel tiles, each dispatched as an independent job to ensure load balance. High‑throughput object storage and local SSD caching minimize I/O bottlenecks, while compressed network transfers and pipeline parallelism keep latency low. The system achieves 98 % scaling efficiency, delivering roughly 14 petaFLOPS and completing a full terapixel image in under one hour (≈58 minutes).

Cost analysis shows that the direct expense of using cloud GPU instances is less than one‑third of the total cost of acquiring, operating, and maintaining an equivalent on‑premise supercomputer. Indirect costs—primarily software porting and debugging—were significant during the initial development phase but are expected to diminish as the framework matures and becomes reusable.

The final terapixel image is stored as multi‑resolution tiles and served through a web‑based viewer. Users can interactively pan, zoom, and explore the city at any scale, while IoT data (air quality, traffic flow, energy consumption, etc.) are overlaid on the 3D geometry, enabling real‑time analytics across sensing scales. The paper concludes that public‑cloud GPU resources now surpass national supercomputers in both raw performance and cost‑effectiveness for this class of visual analytics, and that the presented architecture provides a viable path forward for large‑scale, data‑rich digital twin applications. Future work will explore real‑time ray‑tracing acceleration, energy‑aware scheduling, and multi‑cloud resource federation.