Estimating the Impact of COVID-19 on Travel Demand in Houston Area Using Deep Learning and Satellite Imagery

E S T I M A T I N G T H E I M P A C T O F C OV I D - 1 9 O N T R A V E L D E M A N D I N H O U S T O N A R E A U S I N G D E E P L E A R N I N G A N D S A T E L L I T E I M A G E RY Alekhya Pachika 1 , Lu Gao, Ph.D. 2 , Lingguang Song, Ph.D. 3 Civil and En vironmental Engineering, Univ ersity of Houston apachika@uh.edu, lgao5@central.uh.edu, lsong5@central.uh.edu Pan Lu, Ph.D. 4 4 Department of T ransportation, Logistics and Finance, North Dakota State Uni versity pan.lu@ndsu.edu Xingju W ang, Ph.D. 5 5 School of T raffic and T ransportation, Shijiazhuang Tiedao Uni versity wangxingju@stdu.edu.cn A B S T R A C T Considering recent advances in remote sensing satellite systems and computer vision algorithms, many satellite sensing platforms and sensors ha ve been used to monitor the condition and usage of transportation infrastructure systems. The lev el of detail that can be detected increases significantly with the increase of ground sample distance (GSD), which is around 15 cm–30 cm for high-resolution satellite images. In this study , we analyzed data acquired from high-resolution satellite imagery to provide insights, predicti ve signals, and trends for tra vel demand estimation. More specifically , we estimate the impact of CO VID-19 in the metropolitan area of Houston using satellite imagery from Google Earth Engine datasets. W e de veloped a car -counting model through Detectron2 and Faster R-CNN to monitor the presence of cars within dif ferent locations (i.e., univ ersity , shopping mall, community plaza, restaurant, supermarket) before and during CO VID-19. The results show that the number of cars detected at these selected locations reduced on a verage 30% in 2020 compared with the pre vious year 2019. The results also show that satellite imagery pro vides rich information for trav el demand and economic acti vity estimation. T ogether with adv anced computer vision and deep learning algorithms, it can generate reliable and accurate information for transportation agency decision makers. K eywords satellite imagery · trav el demand · CO VID-19 · deep learning · Detectron2 · vehicle counting 1 Introduction Satellite image analysis uses images tak en from an artificial satellite and analyzes them for v arious purposes such as meteorology , landscape, regional planning, agricultural studies, geology (geomorphology), forestry , urban studies, geography , en vironmental research, cartography , aerology , climatology , oceanography , military , intelligence, and warfare [1, 2]. The adv ancement of satellites and image-analysis methods has become more sophisticated in the past decade and has led to many applications [3 – 5]. Recent years hav e seen an upsur ge in interest in high-resolution synthetic aperture radar (SAR) data and innov ativ e data processing methods. Thanks to these dev elopments, it has now become possible to track issues in relati vely small targets and their e xtent. For this reason, satellite imagery analysis has been extensi vely used for applications in ci vil engineering, including monitoring the condition of infrastructure facilities, detecting damage, ev aluating impacts of disasters, and monitoring the usage of infrastructure facilities [6, 7]. The research community has discussed the possibility of applying satellite imagery for pa vement management in the past decade. For example, Haider et al. [8] suggested that a satellite-based system of pavement monitoring could improv e highway maintenance and reduce the number of vehicle-based inspections. Faghri and Ozden [9] re viewed and summarized the fields that use satellite imagery . This study examines ho w ef fectiv e satellite imagery is in pa vement management, analyzing historical data and identifying deformations and deformation velocity for highways, rail ways, and pav ement roughness. Li et al. [10] studied the financial aspects of the technology and its components based on a cost-benefit analysis of ongoing pa vement monitoring acti vities. More recently , due to the fast de velopment of deep learning-based image analysis tools, researchers hav e in vestigated the ef fectiv eness of satellite imagery in monitoring pav ement surface conditions [11 – 19]. Brewer et al. [20] summarized the use of remotely sensed images to determine road quality with con volutional neural networks. As inputs, they used high-resolution satellite imagery to assemble information about road quality and achie ved 80% accurac y . Bashar and T orres-Machi [21] summarized that satellite imagery can be used as a cost-ef fecti ve and rapid means for e v aluating the condition of roads. An assessment to find pav ement distresses was conducted using spectral and texture information deriv ed from 30-cm panchromatic and 1.2-m optical imagery . Based on the spectral analysis of the multispectral images, it was found that the smoother the roads, the brighter the pav ement across the whole spectrum. The spectral index approach may be useful for identifying individual distresses; howe ver , the opposing behavior of pixel brightness v alues limits its application to analyzing the condition of a pav ement when multiple distresses are present. Satellite imagery has also been applied to monitor building damage. For example, W ang et al. [22] used SAR and optical images from the W enchuan earthquak e to determine a damaged building’ s characteristics. Pan and T ang [23] studied the interrelationships between grade of building damage and variations in backscatter intensity in the case of the W enchuan earthquake. Three categories of damage were identified: se verely damaged, moderately damaged, and some what damaged. Dell’Acqua and Polli [24] analyzed the dif ferences in texture statistics based solely on post-ev ent COSMO/SkyMed data to assess damage based only on radar reflecti vity patterns affected by damage based on block-av eraged measurements. Cossu et al. [25] summarized how SAR images of various resolutions can be used to examine damage and te xtural characteristics. In this study , the authors used a completely div erse training sample from different spatial resolutions in satellite images to classify b uilding damage more accurately . Satellite imagery analysis through deep learning models is increasingly being used for monitoring b uilding damage in recent studies. For e xample, Ji et al. [26] used a con volutional neural network to identify collapsed buildings in the Haiti earthquake with an o verall accuracy of 78.6%. Nex et al. [27] stated that deep learning techniques ha ve improved traditional image analysis approaches, allo wing them to accurately identify visible structural damage in b uildings. Their work in vestigated the performance of a CNN for detecting visible structural damage. It is e vident from the experiments presented that many factors impact the quality of the results and that it is nearly impossible to predict the exact behavior of the network with a dataset in adv ance, especially when the dataset includes dif ferent geographical regions and dif ferent building types compared to the ones used as training samples. Since satellite imagery has dev eloped dramatically in recent years, studies have concentrated more on how satellite imagery can be used for monitoring the usage of transportation infrastructure systems [28 – 43]. For example, Hoppe et al. [44] summarized that monitoring long-term transport infrastructure with InSAR technology is appealing because of the wide a vailability of radar satellites and the rapid de velopment of digitized signal processing techniques. Chen et al. [45 – 47] used multitemporal planet satellite images and de veloped a v ehicle detection method to determine how mobility has changed in numerous cities around the world because of the CO VID-19 pandemic. 2 Estimating CO VID-19 Impact on T ra vel Demand There is a gro wing interest in the analysis of satellite imagery in the domain of monitoring usage of infrastructure facilities. When governments need to monitor economic activities in cities, the y can act fast and efficiently through satellite imagery analysis. The recurrence of hurricanes and the CO VID-19 pandemic has caused crises with impacts on local economic sectors. The CO VID-19 pandemic has also created substantial disruptions across multiple dimensions of transportation systems and infrastructure decision-making. In this case study , we propose an automatic approach for analyzing the impact on economic acti vities via satellite images. In this application, we apply the Detectron2 object detection pipeline [48] together with the Faster R-CNN family of detectors [49] to the task of car detection in satellite imagery . The car detection model was trained on a v ery large and di verse satellite image dataset with around 30 cm resolution. The model was able to achie ve 90% accurac y on av erage. This journal manuscript further e xtends our earlier conference inv estigation of CO VID-19-related travel demand changes in Houston using satellite imagery and deep learning. 2 2.1 Car Detection Model Computer vision techniques such as object detection allow us to identify and locate items in images or videos. The precise location of objects in a scene can be determined through this kind of identification and localization, while the objects can be counted and accurately categorized. Similar deep learning approaches have also been applied to traffic-state e v aluation and prediction in transportation systems. In this case study , we trained our vehicle counting model using the Cars Overhead with Context (COWC) dataset, which consists of annotated cars from satellite imagery [50]. The data consists of around 33,000 unique cars from six different image locations: T oronto, Canada; Selwyn, Ne w Zealand; Potsdam and V aihingen, Germany; and Columb us and Utah, United States. The Columbus and V aihingen datasets are in grayscale, which are not used for the training in this case study . The other datasets are 3-band RGB images. The COWC imagery has a resolution of around 15 cm ground sample distance (GSD). W e split the dataset into 70% training and 30% testing. The bounding boxes of 16 sample CO WC images are displayed in Figure 1. W e trained the Detectron2 model for 9,000 epochs, which takes about 1 hour on a Google Colab Pro account. Figure 1: Sixteen CO WC images and bounding boxes. 3 2.2 Dataset: High-Resolution Dataset over Houston In this section, we collected multiple images ov er Houston using Google Earth Pro and the National Oceanic and Atmospheric Administration (NO AA) website. W e downloaded images from eight different parking spaces in Houston. In total, we collected 127 images ov er the past few years. T able 1: Data collected from 8 places in Houston. No. Place Area (km 2 ) 1 Evelyn Rubenstein Je wish Community Center (JCC) of Houston 0.04 2 Univ ersity of Houston (UH) Parking Lot 16B, 16C, and 16F 0.04 3 Southwest corner of the intersection of Chimney Rock Rd and S Braeswood Blvd 0.02 4 Braeswood Square 0.06 5 South of the intersection of Hillcroft A ve and S Braeswood Blvd 0.07 6 Meyerland Plaza 0.21 7 Houston Chinatown 0.40 8 Katy Mills 0.51 The selection of these locations in Houston allows us to analyze dif ferent economic sectors. For example, locations 1, 3, 4, and 5 can be used to assess the impacts of flooding and CO VID-19 on community commercial centers. Location 2 can be used to analyze the effect of CO VID-19 on univ ersity students’ attendance. Locations 6 and 8 can be used to measure economic acti vity at major commercial malls. Location 7 can be used to measure the impacts of disasters on local restaurants. Figure 2: Locations selected for this case study . The following figures sho w the comparison between the periods before and during CO VID-19. W e can visually observe a reduction in the total number of cars before and during the pandemic. 4 Figure 3: Parking located at JCC. Figure 4: Parking located at UH. Figure 5: Parking located at Braeswood Square. 5 Figure 6: Parking located at Hillcroft and S Braeswood. Figure 7: Parking located at Meyerland Plaza. Figure 8: Parking located at Chinato wn. 6 Figure 9: Parking located at Katy Mills. By counting the number of vehicles in a certain region during the past decade, we were able to measure changes in economic acti vity . W e cropped each collected satellite image into tiles of size 256 × 256 and passed them through the Detectron2 pipeline. Overlapped bounding box es were discarded when combining tiles together to remov e repeated vehicles in consecutiv e tiles. The results sho w that the number of cars parked at these selected locations reduced on a verage 30% in 2020 compared with the pre vious year 2019. These findings also suggest that image-deriv ed transportation indicators can complement broader infrastructure and socio-economic assessment framew orks. 3 Conclusion This research re viewed pre vious studies on satellite image analysis and its application in infrastructure management, including monitoring pav ement conditions, disaster management, and damage assessment. As a result, the follo wing conclusions hav e been summarized: • Satellite image analysis provides cost-ef fectiv e approaches for continuously monitoring infrastructure assets that cover lar ge areas. As a complement to traditional methods and practices, satellite-based infrastructure monitoring is useful. • The studies have demonstrated that conv olutional neural network (CNN) related deep learning models are promising in estimating road quality from remotely sensed imagery . Further research is needed to better understand their applications in pav ement quality estimation. W ith additional research, this method could be valuable in estimating pa vement quality . • The application of satellite imagery methods to sinkhole detection, slope stability monitoring, and b uild- ing damage monitoring has been proven ef fectiv e. The ability of satellite imagery methods to measure surface displacements at the millimeter scale over a large landmass creates the potential for network-le vel implementation. • The increased av ailability of radar satellites, combined with the rapid progress in digital signal processing, is useful for long-term performance monitoring of infrastructure systems. • The images used in the case study were taken from Google Earth Pro, which only provides a few images each year . This may add uncertainty in estimating the number of v ehicles. For future studies, more images from commercial satellite vendors are needed to overcome this limitation. Another approach to reduce the uncertainty is to estimate yearly av erage vehicle counts. • The economic impact can be better estimated if other data sources such as surv eys, intervie ws, employment, and prices are integrated together with satellite image analysis. References [1] Dav e Donaldson and Adam Storeygard. The vie w from above: Applications of satellite data in economics. J ournal of Economic P erspectives , 30(4):171–198, 2016. [2] Charlotta Mellander, José Lobo, Ke vin Stolarick, and Zara Matheson. Night-time light data: A good proxy measure for economic activity? PLOS ONE , 10(10):e0139779, 2015. 7 [3] Noel Gorelick, Matt Hancher , Mike Dixon, Simon Ilyushchenko, David Thau, and Rebecca Moore. Google earth engine: Planetary-scale geospatial analysis for ev eryone. Remote Sensing of En vir onment , 202:18–27, 2017. [4] Annalisa Sheehan, Andrew Beddows, David C. Green, and Sean Beevers. City scale traffic monitoring using worldvie w satellite imagery and deep learning: A case study of barcelona. Remote Sensing , 15(24):5709, 2023. [5] Hashir T anv eer , Timo Balz, Francesca Cigna, and Deodato T apete. Monitoring 2011–2020 traffic patterns in wuhan (china) with cosmo-skymed sar , amidst the 7th cism military world games and covid-19 outbreak. Remote Sensing , 12(10):1636, 2020. [6] Prathyush Kumar Reddy Lebaku, Lu Gao, P an Lu, and Jingran Sun. Deep learning for pav ement condition ev aluation using satellite imagery . Infrastructur es , 9(9):155, 2024. [7] Xingju W ang, Y anting Li, Rongqun Zhang, Jiayu Liu, and Lu Gao. Comprehensi ve network-lev el urban road asset valuation method integrating physical and social values. Journal of T ransportation Engineering, P art A: Systems , 150(7):04024024, 2024. [8] S. W . Haider, G. Y . Baladi, K. Chatti, and C. M. Dean. Effect of frequency of pa vement condition data collection on performance prediction. T ransportation Researc h Recor d: Journal of the T ransportation Researc h Boar d , 2225, 2010. [9] A. Faghri and A. Ozden. Satellite assessment and monitoring for pavement management. T echnical report, Center for Advanced Infrastructure and T ransportation, State University of Ne w Jersey , Piscataway , NJ, USA, 2015. [10] M. Li, A. F aghri, A. Ozden, and Y . Y ue. Economic feasibility study for pavement monitoring using synthetic aperture radar -based satellite remote sensing: Cost-benefit analysis. T ransportation Resear ch Recor d , 2645(1), 2017. [11] Ke Li, Gang W an, Gong Cheng, Liqiu Meng, and Junwei Han. Object detection in optical remote sensing images: A surve y and a new benchmark. ISPRS Journal of Photo grammetry and Remote Sensing , 159:296–307, 2020. [12] Gong Cheng and Junwei Han. A survey on object detection in optical remote sensing images. ISPRS Journal of Photogrammetry and Remote Sensing , 117:11–28, 2016. [13] Thorsten Hoeser and Claudia Kuenzer . Object detection and image segmentation with deep learning on earth observation data: A revie w-part i: Evolution and recent trends. Remote Sensing , 12(10):1667, 2020. [14] Zheng Li, Y ongcheng W ang, Ning Zhang, Y uxi Zhang, Zhikang Zhao, Dongdong Xu, Guangli Ben, and Y unxiao Gao. Deep learning-based object detection techniques for remote sensing images: A survey . Remote Sensing , 14(10):2385, 2022. [15] Y ating Gu, Y antian W ang, and Y ansheng Li. A survey on deep learning-driv en remote sensing image scene under- standing: Scene classification, scene retrie val and scene-guided object detection. Applied Sciences , 9(10):2110, 2019. [16] Li Liu, W anli Ouyang, Xiaogang W ang, Paul Fieguth, Jie Chen, Xinwang Liu, and Matti Pietikäinen. Deep learning for generic object detection: A surve y . International Journal of Computer V ision , 128(2):261–318, 2019. [17] Licheng Jiao, Fan Zhang, Fang Liu, Shuyuan Y ang, Lingling Li, Zhixi Feng, and Rong Qu. A survey of deep learning-based object detection. IEEE Access , 7:128837–128868, 2019. [18] Y ang Long, Gui-Song Xia, Shengyang Li, W en Y ang, Michael Y ing Y ang, Xiao Xiang Zhu, Liangpei Zhang, and Deren Li. On creating benchmark dataset for aerial image interpretation: Re views, guidances, and million-aid. IEEE Journal of Selected T opics in Applied Earth Observations and Remote Sensing , 14:4205–4230, 2021. [19] Jiayu Liu, Xingju W ang, Y anting Li, Xuejian Kang, and Lu Gao. Method of ev aluating and predicting traffic state of highway network based on deep learning. Journal of Advanced T ransportation , 2021(1):8878494, 2021. [20] E. Bre wer, J. Lin, P . Kemper , J. Hennin, and D. Runfola. Predicting road quality using high-resolution satellite imagery: A transfer learning approach. PLOS ONE , 16(7):e0253370, 2021. [21] M. Z. Bashar and C. T orres-Machi. Exploring the capabilities of optical satellite imagery in ev aluating pavement condition. In Construction Researc h Congr ess 2022 , pages 108–115, 2022. [22] H. W ang and Y . Q. Jin. Statistical analysis to assess building damage in the 2008 wenchuan earthquake from multi-temporal sar images. In 2nd Asian P acific Conference on Synthetic Apertur e Radar , pages 121–123, 2009. [23] G. Pan and D. L. T ang. Damage information deriv ed from multi-sensor data of the wenchuan earthquake of may 2008. International Journal of Remote Sensing , 31:3509–3519, 2010. [24] F . Dell’Acqua and D. A. Polli. Post-e vent only vhr radar satellite data for automated damage assessment: A study on cosmo/skymed and the 2010 haiti earthquake. Photogrammetric Engineering and Remote Sensing , 77:1037–1043, 2011. 8 [25] R. Cossu, F . Dell’Acqua, D. A. Polli, and G. Rogolino. Sar-based seismic damage assessment in urban areas: Scaling down the resolution, scaling up computational performance. IEEE Journal of Selected T opics in Applied Earth Observations and Remote Sensing , 5(4):1110–1117, 2012. [26] Min Ji, Lanfa Liu, and Manfred Buchroithner . Identifying collapsed buildings using postearthquake satellite imagery and con volutional neural networks: A case study of the 2010 haiti earthquake. Remote Sensing , 10:1689, 2018. [27] F . Ne x, D. Duarte, F . G. T onolo, and N. K erle. Structural building damage detection with deep learning: Assessment of a state-of-the-art cnn in operational conditions. Remote Sensing , 11(23):2765, 2019. [28] G. Sharma, C. J. Merry , P . Goel, and M. McCord. V ehicle detection in 1-m resolution satellite and airborne imagery . International Journal of Remote Sensing , 27(4):779–797, 2006. [29] Xiaoying Jin and Curt H. Da vis. V ehicle detection from high-resolution satellite imagery using morphological shared-weight neural networks. Image and V ision Computing , 25(9):1422–1431, 2007. [30] Huan W ei, Guoqing Zhou, Zezhong Zheng, Xiaowen Li, Y alan Liu, Y ing Zhang, Shang Li, and T ao Y ue. V ehicle detection from parking lot aerial images. In 2013 IEEE International Geoscience and Remote Sensing Symposium - IGARSS , pages 4002–4005, 2013. [31] Paul Kidwell and K ofi Boakye. Object detection in lo w resolution ov erhead imagery . In 2015 IEEE W inter Applications and Computer V ision W orkshops , pages 21–27, 2015. [32] W esam Sakla, Goran K onjev od, and T . Nathan Mundhenk. Deep multi-modal vehicle detection in aerial isr imagery . In 2017 IEEE W inter Conference on Applications of Computer V ision (W A CV) , pages 916–923, 2017. [33] Nicolas Audebert, Bertrand Le Saux, and Sébastien Lefèvre. Segment-before-detect: V ehicle detection and classification through semantic segmentation of aerial images. Remote Sensing , 9(4):368, 2017. [34] Oliv er Acatay , Lars Sommer , Arne Schumann, and Jurgen Be yerer . Comprehensive e valuation of deep learning based detection methods for vehicle detection in aerial imagery . In 2018 15th IEEE International Conference on Advanced V ideo and Signal Based Surveillance (A VSS) , pages 1–6, 2018. [35] J. Alex Hurt, Da vid Huangal, Curt H. Davis, and Grant J. Scott. A comparison of deep learning vehicle group detection in satellite imagery . In 2019 IEEE International Conference on Big Data (Big Data) , pages 5419–5427, 2019. [36] T anguy Ophof f, Stev en Puttemans, V asileios Kalogirou, Jean-Philippe Robin, and T oon Goedemé. V ehicle and vessel detection on satellite imagery: A comparativ e study on single-shot detectors. Remote Sensing , 12(7):1217, 2020. [37] Sebastien Drouyer . V ehsat: a large-scale dataset for vehicle detection in satellite images. In IGARSS 2020 - 2020 IEEE International Geoscience and Remote Sensing Symposium , pages 268–271, 2020. [38] Jianxiang Li, Zili Zhang, Y an T ian, Y iping Xu, Y ihong W en, and Shicheng W ang. T arget-guided feature super - resolution for vehicle detection in remote sensing images. IEEE Geoscience and Remote Sensing Letters , 19:1–5, 2022. [39] Y inong Guo, Chen W u, Bo Du, and Liangpei Zhang. Density map-based vehicle counting in remote sensing images with limited resolution. ISPRS Journal of Photogr ammetry and Remote Sensing , 189:201–217, 2022. [40] T essa Berry , Nicholas Dronen, Brett Jackson, and Ian Endres. P arking lot instance segmentation from satellite imagery through associativ e embeddings. In Pr oceedings of the 27th ACM SIGSP A TIAL International Conference on Advances in Geographic Information Systems , pages 528–531, 2019. [41] Hao Chen, W en Chen, Jing Zhao, Xueqi Y in, and Y e Zhang. Hierarchical detection from parking lot to vehicle in large-area remote sensing images based on visual saliency and angle estimation. In IGARSS 2019 - 2019 IEEE International Geoscience and Remote Sensing Symposium , pages 1232–1235, 2019. [42] D. S. V ohra, Pradeep Kumar Garg, and Sanjay Ghosh. Real-time vehicle detection for traffic monitoring by applying a deep learning algorithm over images acquired from satellite and drone. International Journal of Intelligent Unmanned Systems , 11(4):441–452, 2022. [43] Mahmoud Ragab, Hesham A. Abdushkour , Adil O. Khadidos, Abdulrhman M. Alshareef, Khaled H. Alyoubi, and Alaa O. Khadidos. Improved deep learning-based v ehicle detection for urban applications using remote sensing imagery . Remote Sensing , 15(19):4747, 2023. [44] E. Hoppe, B. Bruckno, E. Campbell, S. Acton, A. V accari, M. Stuecheli, and J. Morgan. T ransportation infrastructure monitoring using satellite remote sensing. Materials and Infrastructur es , 1(5):185–198, 2016. [45] Y . Chen, R. Qin, G. Zhang, and H. Albanw an. Spatial-temporal analysis of traffic patterns during the covid-19 epidemic by vehicle detection using planet remote-sensing satellite images. Remote Sensing , 13(2):208, 2021. 9 [46] Lu Gao, Pan Lu, Fengxiang Qiao, Joshua Qiang Li, Y unpeng Zhang, and Y ihao Ren. Ev aluating the impact of covid- 19 on transportation infrastructure funding in the united states. In International Conference on T ransportation and Development 2022 , pages 134–142, 2022. [47] Alekhya Pachika, Lu Gao, Lingguang Song, Pan Lu, and Xingju W ang. Estimating the impact of covid-19 on trav el demand in the houston area using deep learning and satellite imagery . In International Conference on T ransportation and Development 2023 , pages 437–444, 2023. [48] Y uxin W u, Alexander Kirillov , Francisco Massa, W an-Y en Lo, and Ross Girshick. Detectron2. https://github. com/facebookresearch/detectron2 , 2019. [49] Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. Faster r -cnn: T owards real-time object detection with region proposal networks. In Advances in Neural Information Pr ocessing Systems 28 (NeurIPS 2015) , 2015. [50] T . N. Mundhenk, G. K onjev od, W . A. Sakla, and K. Boakye. A lar ge contextual dataset for classification, detection and counting of cars with deep learning. In Eur opean Confer ence on Computer V ision , pages 785–800, Cham, 2016. Springer . 10