Machine Learning Detection of Road Surface Conditions: A Generalizable Model using Traffic Cameras and Weather Data

Transportation agencies make critical operational decisions during hazardous weather events, including assessment of road conditions and resource allocation. In this study, machine learning models are developed to provide additional support for the New York State Department of Transportation (NYSDOT) by automatically classifying current road conditions across the state. Convolutional neural networks and random forests are trained on NYSDOT roadside camera images and weather data to predict road surface conditions. This task draws critically on a robust hand-labeled dataset of ~22,000 camera images containing six road surface conditions: severe snow, snow, wet, dry, poor visibility, or obstructed. Model generalizability is prioritized to meet the operational needs of the NYSDOT decision makers, including integration of operational datasets and use of representative and realistic images. The weather-related road surface condition model in this study achieves an accuracy of 81.5% on completely unseen cameras. With operational deployment, this model has the potential to improve spatial and temporal awareness of road surface conditions, which can strengthen decision-making for operations, roadway maintenance, and traveler safety, particularly during winter weather events.

💡 Research Summary

This paper presents a machine‑learning pipeline designed to automatically classify road‑surface conditions across the state of New York, supporting the New York State Department of Transportation (NYSDOT) in winter‑weather operations. The authors assembled a hand‑labeled dataset of 21,653 images drawn from 45 distinct traffic‑camera sites out of NYSDOT’s roughly 2,400‑camera network. Each image is annotated with one of six classes—severe snow, snow, wet, dry, poor visibility, or obstructed—using a systematic Quantitative Content Analysis protocol that involved a detailed codebook, independent labeling by six annotators, and reliability checks. Both high‑quality (well‑lit) and low‑quality (dim, distant) images are deliberately included to ensure the model can handle the full range of real‑world camera conditions.

In addition to visual data, the authors integrate short‑range meteorological forecasts from the High‑Resolution Rapid Refresh (HRRR) model. Six HRRR variables (2‑m temperature, 2‑m relative humidity, 10‑m wind speed, 2‑hour accumulated snow depth, 2‑hour total precipitation, and total cloud cover) are matched to each image based on timestamp and camera location, providing a fine‑scale (≈3 km) weather context that is more detailed than the state mesonet network.

The classification system follows a two‑stage architecture. Stage 1 employs a convolutional neural network (CNN) to extract probabilistic predictions for the five weather‑related surface classes (excluding “obstructed”). Stage 2 combines these CNN probabilities with the six HRRR variables and feeds them into a random‑forest (RF) classifier that outputs the final surface‑condition label. A separate CNN performs binary obstruction detection, allowing the system to flag images that are visually blocked (e.g., by foliage or construction) regardless of weather. By fusing image‑derived features with forecast data, the authors report a 12.1 percentage‑point boost in overall accuracy compared with an image‑only baseline.

A central focus of the work is model generalizability. To mimic operational deployment, the dataset is split using a “site‑specific” strategy: all images from a given camera belong to the same fold, guaranteeing that training, validation, and test sets contain mutually exclusive camera sites. This prevents leakage of site‑specific visual cues and provides a realistic estimate of performance on unseen cameras. Under this regime, the full system achieves 81.5 % accuracy on completely new camera sites, with class‑wise precision and recall analyses showing balanced performance despite class imbalance (e.g., fewer obstructed samples).

The study emphasizes a co‑design process with NYSDOT stakeholders. Throughout data collection, labeling, model selection, and output formatting, the authors incorporated feedback from agency engineers and decision‑makers, aligning the tool with operational needs such as five‑minute update intervals and compatibility with existing GIS dashboards. This collaborative approach is highlighted as a key factor in building a trustworthy, deployable system.

Compared with prior literature, the paper advances the state of the art in several ways: (1) it handles six surface‑condition categories, including poor visibility and obstruction, whereas many earlier works limited themselves to binary snow/no‑snow or wet/dry distinctions; (2) it deliberately includes low‑quality camera imagery, reflecting the heterogeneous nature of NYSDOT’s installed hardware; (3) it validates performance on entirely unseen camera locations, addressing a common shortcoming of same‑camera cross‑validation; and (4) it demonstrates the additive value of short‑range weather forecasts to visual classification.

Limitations are acknowledged. The exact CNN architecture and hyper‑parameter settings are not fully disclosed, which may hinder reproducibility. Reliance on a 2‑hour forecast window could reduce robustness during rapid weather changes, and the model has so far been trained only on New York data, so transfer to other regions may require domain adaptation. Additionally, the obstruction detector is trained solely on visual cues without leveraging auxiliary sensors, potentially missing subtle occlusions.

Future work outlined includes: (a) deploying the pipeline as a real‑time streaming service integrated with NYSDOT’s traffic‑management platform; (b) exploring temporal models (e.g., LSTM or transformer‑based architectures) that jointly ingest sequences of images and forecasts; (c) extending the approach to other states or national networks using the same HRRR data; (d) incorporating model‑explainability techniques (e.g., Grad‑CAM, SHAP) to provide transparent rationale for each prediction; and (e) evaluating the impact of the system on operational metrics such as snow‑plow dispatch efficiency and accident reduction.

Overall, the paper delivers a comprehensive, operationally focused solution that blends computer vision, meteorology, and stakeholder co‑design to produce a generalizable, accurate road‑surface condition classifier ready for real‑world deployment.