Accelerated Evaluation of Automated Vehicles Safety in Lane Change Scenarios Based on Importance Sampling Techniques

📝 Abstract

Automated vehicles (AVs) must be evaluated thoroughly before their release and deployment. A widely-used evaluation approach is the Naturalistic-Field Operational Test (N-FOT), which tests prototype vehicles directly on the public roads. Due to the low exposure to safety-critical scenarios, N-FOTs are time-consuming and expensive to conduct. In this paper, we propose an accelerated evaluation approach for AVs. The results can be used to generate motions of the primary other vehicles to accelerate the verification of AVs in simulations and controlled experiments. Frontal collision due to unsafe cut-ins is the target crash type of this paper. Human-controlled vehicles making unsafe lane changes are modeled as the primary disturbance to AVs based on data collected by the University of Michigan Safety Pilot Model Deployment Program. The cut-in scenarios are generated based on skewed statistics of collected human driver behaviors, which generate risky testing scenarios while preserving the statistical information so that the safety benefits of AVs in non-accelerated cases can be accurately estimated. The Cross Entropy method is used to recursively search for the optimal skewing parameters. The frequencies of occurrence of conflicts, crashes and injuries are estimated for a modeled automated vehicle, and the achieved accelerated rate is around 2,000 to 20,000. In other words, in the accelerated simulations, driving for 1,000 miles will expose the AV with challenging scenarios that will take about 2 to 20 million miles of real-world driving to encounter. This technique thus has the potential to reduce greatly the development and validation time for AVs.

💡 Analysis

Automated vehicles (AVs) must be evaluated thoroughly before their release and deployment. A widely-used evaluation approach is the Naturalistic-Field Operational Test (N-FOT), which tests prototype vehicles directly on the public roads. Due to the low exposure to safety-critical scenarios, N-FOTs are time-consuming and expensive to conduct. In this paper, we propose an accelerated evaluation approach for AVs. The results can be used to generate motions of the primary other vehicles to accelerate the verification of AVs in simulations and controlled experiments. Frontal collision due to unsafe cut-ins is the target crash type of this paper. Human-controlled vehicles making unsafe lane changes are modeled as the primary disturbance to AVs based on data collected by the University of Michigan Safety Pilot Model Deployment Program. The cut-in scenarios are generated based on skewed statistics of collected human driver behaviors, which generate risky testing scenarios while preserving the statistical information so that the safety benefits of AVs in non-accelerated cases can be accurately estimated. The Cross Entropy method is used to recursively search for the optimal skewing parameters. The frequencies of occurrence of conflicts, crashes and injuries are estimated for a modeled automated vehicle, and the achieved accelerated rate is around 2,000 to 20,000. In other words, in the accelerated simulations, driving for 1,000 miles will expose the AV with challenging scenarios that will take about 2 to 20 million miles of real-world driving to encounter. This technique thus has the potential to reduce greatly the development and validation time for AVs.

📄 Content

 Abstract—Automated vehicles (AVs) must be evaluated thoroughly before their release and deployment. A widely-used evaluation approach is the Naturalistic-Field Operational Test (N-FOT), which tests prototype vehicles directly on the public roads. Due to the low exposure to safety-critical scenarios, N-FOTs are time-consuming and expensive to conduct. In this paper, we propose an accelerated evaluation approach for AVs. The results can be used to generate motions of the primary other vehicles to accelerate the verification of AVs in simulations and controlled experiments. Frontal collision due to unsafe cut-ins is the target crash type of this paper. Human-controlled vehicles making unsafe lane changes are modeled as the primary disturbance to AVs based on data collected by the University of Michigan Safety Pilot Model Deployment Program. The cut-in scenarios are generated based on skewed statistics of collected human driver behaviors, which generate risky testing scenarios while preserving the statistical information so that the safety benefits of AVs in non-accelerated cases can be accurately estimated. The Cross Entropy method is used to recursively search for the optimal skewing parameters. The frequencies of occurrence of conflicts, crashes and injuries are estimated for a modeled automated vehicle, and the achieved accelerated rate is around 2,000 to 20,000. In other words, in the accelerated simulations, driving for 1,000 miles will expose the AV with challenging scenarios that will take about 2 to 20 million miles of real-world driving to encounter. This technique thus has the potential to reduce greatly the development and validation time for AVs.

Manuscript received on XXXXXX. This work was supported by the U.S. National Institute for Occupational Safety Health (NIOSH) under Grant F031433. Ding Zhao and Huei Peng are with the Mechanical Engineering Department, University of Michigan, Ann Arbor, MI 48109 USA (e-mail: zhaoding@umich.edu and hpeng@umich.edu). Henry Lam is with the Department of Industrial and Operations Engineering, University of Michigan, Ann Arbor, MI 48109 USA (e-mail: khlam@umich.edu). Shan Bao, David J. LeBlanc and Kazutoshi Nobukawa are with the University of Michigan Transportation Research Institute, Ann Arbor, MI 481099 USA (e-mail: shanbao@umich.edu, leblanc@umich.edu and knobukaw@umich.edu). Christopher S. Pan is with the Division of Safety Research, NIOSH, CDC Morgantown, WV 26505 USA (e-mail: syp4@cdc.gov). Color versions of one or more of the figures in this paper are available online at http://ieeexplore.ieee.org . Digital Object Identifier XXXXXX Index Terms—Automated vehicles, importance sampling, crash avoidance, active safety systems, lane change, AEB

I. INTRODUCTION utomated vehicle (AV) technologies are actively studied by many companies because of their potential to save fuel, reduce crashes, ease traffic congestion, and provide better mobility, especially to those who cannot drive [1]. Currently, almost all major automakers have research and development programs on AVs. By 2030, it is estimated that the sales of AVs may reach $87 billion dollars [2]. National Highway Traffic Safety Administration defines five levels of AV automation [3]. AVs are quickly being developed from level 0 automation, which conducts no driving tasks and up, possibly all the way to level 4 automation, which monitors the driving environment performs all dynamic driving duties.
As the level of automation goes up, AVs need to deal with many uncertainties/disturbances in the real world, including imperfect human driver behaviors. AVs are projected to penetrate the market gradually and will co-exist with human-controlled vehicles (HVs) for decades [4]. During this transitional period, AVs will interact primarily with HVs. It is estimated that 70-90% of motor vehicle crashes are due to human errors [5], [6]. However, AVs can have their own crash modes. A practical and effective evaluation of the safety performance of AVs should consider their interactions with HVs.
Fig. 1. Summary of evaluation approaches for automated vehicles Accelerated Evaluation of Automated Vehicles Safety in Lane Change Scenarios based on Importance Sampling Techniques
Ding Zhao, Henry Lam, Huei Peng, Shan Bao, David J. LeBlanc, Kazutoshi Nobukawa, and Christopher S. Pan A

Approaches for AV evaluation can be summarized into four categories as shown in Fig. 1. One approach to studying the interactions between AVs and HVs is through Naturalistic Field Operational Tests (N-FOT) [7]. In an N-FOT, data is collected from a number of equipped vehicles driven under naturalistic conditions over an extended period of time [8]. Several N-FOT projects [9]–[16] have been conducted in the U.S. and Europe. Conducting an N-FOT to evaluate an AV function typically involves non-intrusive conditions, i.e., the test drivers were to

This content is AI-processed based on ArXiv data.