Fraunhofer Institute for Experimental Software Engineering IESEOur tool-based solution AutoTestReduction explores the automated generation of logical scenarios for vehicle validation, leveraging vehicle recordings as a foundational data source. We address the challenge of determining the relevance of external influences, such as other vehicles, by employing auto-encoders that utilize unsupervised learning to automatically identify key features. These features are then clustered to generate logical scenarios. This approach enhances traditional engineered scenarios with data-driven insights and enables early-stage validation of vehicle functions through fully automated, AI-based scenario generation.
Automating Vehicle Validation with AutoTestReduction
AutoTestReduction enables different use cases for validating autonomous vehicles:
AI Automated Testing of Autonomous Vehicles
In cooperation with RevoAI GmbH and Fraunhofer IESE
Screenshot of FastLoop+AutoTestReduction for the scenario-based analysis and test execution.
The tool allows insight details for analysis as well as parallel execution for regression testing.
It identifies critical executions by various simulation runs. The clustering of scenarios defines different cause-effect relations. The replay and visualization complement test reports.
Problem: Testing autonomous vehicles requires covering all critical scenarios using both engineering methods and natural driving data, following ISO 34501-34504 standards.
Solution: RevoAI GmbH and Fraunhofer IESE developed FastLoop+AutoTestReduction, an AI-based tool for fully automated scenario generation and execution. Key features include scenario definition, unsupervised learning with auto-encoders, parameterized testing, and reinforcement-based test optimization.
Benefit: This enables efficient, AI-driven virtual validation of autonomous vehicles, ensuring safety through optimized testing.
Runtime Scenario Identification for Autonomous Vehicles
Clustering of traffic scenarios based on unsupervised learning with an auto-encoder. If new records are determined to be outliers and not to belonging to existing clusters, then a new relevant case has been identified.
Problem: Autonomous vehicles face unforeseen scenarios in real-world environments, requiring field observations to identify new or relevant situations. Limited storage and bandwidth make it essential to filter runtime data to determine whether a scenario is known or an edge case.
Solution: Fraunhofer IESE uses auto-encoders for engineering-agnostic detection of new driving scenarios, requiring no feature engineering. This AI-driven approach identifies cause-effect relationships, going beyond simple anomaly detection.
Benefit: Manufacturers can fulfill product monitoring obligations and automatically detect unexpected scenarios during vehicle runtime without needing prior system knowledge.
Street design comparison for Operation Design Domain (ODD) compliance
Clustering of traffic scenarios based on unsupervised learning with an auto-encoder for two intersections. the clusters that cannot be matched from one intersection to the other determine the missing or additional behavior interaction possibilities. In case of clusters would match both intersections exhibit the same traffic participant behavior interactions.
Problem: Defining an Operational Design Domain (ODD) for autonomous vehicles requires ensuring that real-world street designs and traffic behaviors align with the abstract designs defined during ODD development. The complexity of interactions in different street designs makes it challenging to compare scenarios effectively.
Solution: Fraunhofer IESE uses auto-encoders to automatically identify possible behavior interactions within different street designs. This unsupervised, engineering-agnostic approach compares interactions across street designs, highlighting both overlaps and differences.
Benefit: Autonomous vehicle manufacturers can ensure real-world environments match their ODD, supporting safety validation. Street designers can optimize designs by analyzing behavior interactions and potential causes of unexpected behavior.