HeatMat: Simulation of City Material Impact on Urban Heat Island Effect

The Urban Heat Island (UHI) effect, defined as a significant increase in temperature in urban environments compared to surrounding areas, is difficult to study in real cities using sensor data (satellites or in-situ stations) due to their coarse spatial and temporal resolution. Among the factors contributing to this effect are the properties of urban materials, which differ from those in rural areas. To analyze their individual impact and to test new material configurations, a high-resolution simulation at the city scale is required. Estimating the current materials used in a city, including those on building facades, is also challenging. We propose HeatMat, an approach to analyze at high resolution the individual impact of urban materials on the UHI effect in a real city, relying only on open data. We estimate building materials using street-view images and a pre-trained vision-language model (VLM) to supplement existing OpenStreetMap data, which describes the 2D geometry and features of buildings. We further encode this information into a set of 2D maps that represent the city’s vertical structure and material characteristics. These maps serve as inputs for our 2.5D simulator, which models coupled heat transfers and enables random-access surface temperature estimation at multiple resolutions, reaching an x20 speedup compared to an equivalent simulation in 3D.

💡 Research Summary

HeatMat addresses the pressing need for high‑resolution, city‑scale analysis of how building materials influence the Urban Heat Island (UHI) effect. Traditional observational data (satellite thermal imagery, sparse weather stations) lack the spatial and temporal granularity required to isolate material‑level impacts, while existing physics‑based simulators such as ENVI‑met or Stardis demand a full 3‑D digital twin with detailed material annotations—a labor‑intensive prerequisite for whole‑city studies.
The proposed pipeline circumvents these limitations by relying exclusively on openly available data: OpenStreetMap (OSM) for 2‑D building footprints and a large collection of geotagged street‑view panoramas (e.g., Mapillary’s Metropolis dataset). For each building the system selects the most informative view based on proximity, frontal orientation, and image coverage, reducing the number of images to be processed by roughly an order of magnitude.
From the chosen 360° panoramas the method extracts rectified left/right perspective images, isolates façade regions using Grounded‑SAM with the prompt “facade”, and then feeds the cropped patches to a pre‑trained vision‑language model (VLM), specifically LLaVA‑1.6. A carefully crafted textual prompt forces the VLM to output a JSON structure describing the material type and proportion for each façade element (ground‑level walls, doors, windows, frames, upper‑level cladding, shutters, etc.). A manually labelled validation set of 50 façades shows IoU and accuracy above 85 %, and ablation studies confirm the contribution of the segmentation step and the choice of VLM.
The extracted metadata are rasterised into a suite of 18 2‑D textures that together constitute a “2.5‑D” city representation. These textures encode building identifiers, height, signed distance fields (SDF), roof and ground material IDs, and per‑pixel façade material composition maps. All textures are stored in 16‑bit RGBA format to minimise GPU bandwidth and enable random‑access sampling within a pixel shader.
The core simulation engine is a Monte‑Carlo (MC) random‑walk solver implemented as a GPU pixel shader. Building on prior work that couples radiative, convective, and conductive heat transfers in a MC framework, HeatMat adapts the algorithm to operate on the 2.5‑D texture stack rather than a volumetric mesh. For each photon or heat packet the shader samples the height and material textures to compute direct and indirect solar irradiance, long‑wave radiation, convection with ambient air, and conduction through walls. The MC walk proceeds until energy is either emitted to the sky or absorbed by the ground, producing a surface temperature field that can be queried at arbitrary resolutions.
Performance benchmarks demonstrate a ~20× speedup compared with an equivalent 3‑D MC implementation, while maintaining temperature errors below 0.5 °C relative to ENVI‑met and Stardis. The authors validate conductive transfer by cross‑checking a finite‑difference implementation and by comparing simulated land‑surface temperature (LST) maps against satellite observations, achieving statistically significant correlations. Sensitivity experiments illustrate that substituting high‑albedo, low‑conductivity façade materials can reduce city‑wide mean surface temperature by up to 0.8 °C, highlighting the practical relevance for urban planners.
Limitations include reliance on street‑view coverage (areas with sparse imagery yield incomplete material maps) and the inherent uncertainty of VLM‑generated material proportions. The authors propose future extensions that fuse LiDAR or multispectral satellite data to fill coverage gaps, employ ensemble VLMs or post‑hoc verification to reduce prediction bias, and integrate a full atmospheric model to capture wind, humidity, and precipitation effects.
In summary, HeatMat delivers a novel, open‑data‑driven workflow that automatically extracts façade material statistics, encodes a city into a compact 2.5‑D representation, and runs a GPU‑accelerated Monte‑Carlo heat‑transfer simulation at city scale. It offers a practical tool for rapid, physics‑based evaluation of material‑level mitigation strategies against the UHI phenomenon, bridging the gap between high‑fidelity urban climate modeling and the data‑availability constraints that have historically limited such analyses.