Background
Robust perception algorithms are a vital component in autonomous systems such as self-driving vehicles. Although deep learning (DL) and convolutional neural networks (CNNs) have advanced real-time object recognition capabilities, testing of these systems for correctness has lacked a generalized, reason-based approach. Currently, perception testing for self-driving vehicles typically involves either (a) measuring recognition error from a pre-recorded dataset, or (b) performing driving tests with a backup human driver and analyzing disengagement events, that is, when the autonomous vehicle returns control back to the human driver.
Incorrect classification of an object (e.g., as a pedestrian, cyclist, or vehicle) has been a main cause of system disengagement and was cited as factor in the fatal Uber accident in 2018. Therefore, introducing a reason-based framework for monitoring perception algorithm outputs can enhance autonomous vehicle safety.
Invention Description
Researchers at Arizona State University and University of Southern California have developed a new framework for evaluating the degree to which a perception algorithm adheres to a set of spatio-temporal relations. This method is an extension of Timed Propositional Temporal Logic (TPTL) and allows analysis of object labels across different image frames. By monitoring specified requirements and when they are violated, low-quality perception can be promptly located within the dataset even in the absence of ground truth comparisons. For example, perception data can be checked for the following condition: If a cyclist is labeled in a frame with >70% probability, then the label’s probability does not fall below 60% in the next 5 frames. Such a platform would integrate precise and real-life requirements in learning-based models and can also be used in real-time systems to detect perception failures.
Potential Applications
• Autonomous vehicles
• Spatio-temporal object identification
Benefits and Advantages
• Self-Contained – Checks for perception quality without the need for ground truth datasets
• Multi-Dimensional – Allows perception analysis to incorporate information from both space and time domains
• Versatile – Can be used as a debugging tool or as a real-time monitor in autonomous vehicles
Homepage of Professor Georgios Fainekos
Homepage of Professor Heni Ben Amor