Fraunhofer Institute for Experimental Software Engineering IESEInnovative security solutions
How Fraunhofer IESE and Robert Bosch GmbH make complex safety-critical applications automatically testable using Linux as the operating system platform.
Last modified:
Bosch GmbH Success Story
Bosch Success Story at a glace
What it's all about:
Fraunhofer IESE supports Bosch in developing an approach for testing safety-critical applications
The challenge:
Evaluating the impact of operating system errors on safety-critical applications under Linux.
The result:
Chaos engineering-based test framework for safety-critical applications
Your advantages:
Fraunhofer IESE is your competent partner for the development of innovative safety solutions
What it's all about: Fraunhofer IESE supports Bosch in developing an approach for testing safety-critical applications
In certain industries, such as the automotive industry, the safety of the developed software is of the utmost importance and often associated with high costs. One approach to reducing software development costs is to use Linux as operating system platform. Linux as a platform, also allows to apply virtualization solutions such as Docker. These make it possible to run several applications on one computing platform and isolate their contexts from one another. Another advantage of Linux as a platform is the possibility of using existing open-source applications as a basis for own developments and extensions.
Despite the possibility of inspecting the source code of Linux (kernel and runtime libraries), analyzing it for errors and fixing them, requires a detailed analysis and is very difficult due to the complexity. This applies in particular to errors and their effects on the behavior and output of safety-critical applications caused by errors within the hardware or the operating system (e.g. the kernel). Due to the scope, an efficient and automated methodology is required to identify possible critical errors of the computing platform affecting the safety-critical application. These identified errors must be systematically tracked and the safety-critical application must be safe-guarded accordingly and additional safety mechanisms must be planned and integrated, respectively. The following figure illustrates the components of the developed framework and their interaction. In this diagram, the “application under test” represents the safety-critical application to be safe-guarded, which (in this case) runs within a virtual environment with Linux as operating system.
The challenge: Evaluating the impact of operating system errors on safety-critical applications under Linux.
Software developed by open-source communities is a popular alternative to self-developed solutions. This saves costs compared to in-house development and enables shorter update intervals in which bugs are fixed or new features are added. Despite the often-large communities, most applications have only been functionally tested and not tested with focus on safety. Using these applications in a safety-critical context is usually not part of the development of classic open-source applications. Hence, especially the aspect that the functionality and robustness can also be limited by hardware or operating system errors is not or not sufficiently addressed.
The challenge in the project was to create an efficient way of not only testing the functionality of (virtualized) applications, but also to test the effects of errors at the interface between the application and the Linux operating system and then enabling an analysis.
The support: Fraunhofer IESE supports Robert Bosch GmbH in the design, architecture and development of a framework for efficient fault injection of safety-critical applications
Fraunhofer IESE supported Robert Bosch GmbH with its experience in the validation and application development in order to realize a framework to test safety-critical applications semi-automatically through fault injection and enables subsequent analysis of fault effects. First, the requirements were analyzed and prioritized together with Bosch. The state of the art and existing best practices in the field of chaos engineering were evaluated and a customized concept for the test framework has been created. The developed framework enables the automated execution of a test sequence in which the behavior of the application runs is stored and afterwards an automated detection of deviations is performed.
Based on the analyzed requirements, the test framework was realized by Fraunhofer IESE, aiming for a high degree of configurability to make the framework as generic as possible and to support different application types in terms of testing. At the same time, the architecture was designed to be open and modular to ensure extensibility. The test results can be provided in various formats to enable detailed analysis. Additionally, the results of the test runs can be analyzed to classify the faulty behaviour of the application. The developed test framework has been successfully evaluated using a sample application together with Robert Bosch GmbH.
This is what Peter Schneider, Project Manager of Central Research at Robert Bosch GmbH, says:
“A test framework focused on virtualization and automation is an important tool for us to efficiently develop future software validation concepts. In collaboration with IESE, we succeeded in developing our ideas into an executable prototype in just a few weeks, which supports a high degree of test automation and also meets our requirements in terms of modularity and expandability.”
Peter Schneider, Project Manager of Central Research at Robert Bosch GmbH
The result: Chaos engineering-based test framework for safety-critical applications
The result of this collaboration is a test framework for safety-critical (and optionally virtualized) applications under Linux, which enables a tester to automatically perform a large number of test runs with different fault injections. The automatic classification of behavioral deviations of the application enables an efficient identification of relevant errors and reduces the number of tests that have to be investigated manually with respect to safety impacts.