Testing complex autonomous systems

Cooperative automated or autonomous vehicles are terms used for intelligent cars, buses and trucks that communicate with each other and the traffic system infrastructure to navigate and drive more or less automated, by themselves. Designing software-intensive embedded systems for these intelligent vehicles is challenging. In addition to fulfilling complex functional requirements, the system must be safe under all operating conditions, even in the presence of faults.

”The key to achieving this is by simulating and testing the system enough, including possible faults that can be expected, to be confident that it reaches an acceptable level of performance with preserved safety”, says Benjamin Vedder that recently published a doctoral thesis on the subject.

Benjamin Vedder is a PhD student within Halmstad University Embedded and Intelligent Systems Industrial Graduate School (EISIGS). The research has been done in collaboration between Halmstad University and RISE research institute.

Benjamin Vedder, what testing methods have you developed in your research?
”We utilize techniques from the areas of fault injection and property-based testing, and apply them on simulations, hardware-in-the-loop tests as well as on tests with full hardware. The work is rather hands-on, and we have developed a custom quadcopter system as well as self-driving model vehicles in the process of evaluating our approach.”

What answers did you get, on the research questions you had?
”In short, we show practical and feasible methods to automatically test functional and safety aspects of complex autonomous systems. In the process of doing so, we encountered many challenges and proposed methods to deal with them. One example of such a challenge is how to safely let a computer control a moving model vehicle with randomly generated trajectories for it in real-time. The answer to this challenge includes constraining how the random trajectories are generated, and making sure that the test scenario gives computation and communication enough headroom to finish without letting the vehicle under test run out of commands on how to drive next.”

Are any results especially surprising to you?
“The repeatability of the results in the last paper was a positive surprise to me. We compared two positioning systems on our self-driving model vehicle while driving on automatically generated trajectories. Sometimes we got position deviations that were difficult to explain from studying the results of one experiment. By implementing methods to re-run any experiment accurately several times, we could see that these deviations occurred in almost exactly the same manner every time the experiment was run. This helped us conclude that the deviations were caused by objects in the test environment that blocked signal paths. Figuring this out while manually driving the model vehicle would have been significantly more challenging.”

How can your research benefit the society?
“This research can benefit society by making it easier to develop and test safe autonomous systems. Autonomous systems can solve many problems and increase the quality of life for all people.”

What are you plans for the future?
“I'm currently working in several projects that involve autonomous drones and vehicles, and I will continue working in them for now. I like building hardware and developing software, and I enjoy sharing the results as well as source code from my work with others. The self-driving model vehicle platform that I developed during the thesis is available as open-source software and hardware, and I will continue developing and sharing it for the current and upcoming projects.”

Text: Louise Wandel och Benjamin Vedder
Photo: Magnus Jonsson
Top illustration: ISTOCK

Read more