Testing times ahead for autonomous vehicles

The evolution of technology today has moved autonomous vehicles closer to reality than ever before. With the help of technological advances, engineers can now look to prevent the occurrence of more than 10 million road accidents across the world on an annual basis. To make this vision a reality, OEMs and Tier 1 suppliers have begun applying cutting-edge technology to solve some of the biggest problems in computing, engineering, software developing, and algorithm design today.

This move towards resolving pressing industrial problems, however, has created an extremely challenging scenario for test engineers. They must now deal with conducting tests using unknown regulations, technology, architectures, and even unknown algorithms, each of which have been created by neural networks instead of manual coding by software engineers. Additionally, safety malfunctions in new technology, such as crashes of self-driving vehicles, bring greater scrutiny and doubt upon these technologies. Consumers need to be consistently guaranteed of the feasibility and safety of autonomous technology before such vehicles achieve commercial success. The onus of giving this guarantee now falls on technology developers and testers (a terrifying prospect for vehicle developers) as they work towards meeting government, legal and customer standards of safety, with ‘operator error’ no longer being considered a viable excuse.

Testing against a time crunch
One of the most prominent technologies that will help autonomous vehicles become a reality is machine learning. This technology will enable engineers to design algorithms that can successfully negotiate the multitude of scenarios a vehicle could possibly encounter by combining them with data on human reactions to create neural networks.

The algorithms that emerge from these networks, however, tend to be difficult to predict as engineers do not possess a fundamental understanding of the code that can be used to generate test scenarios. In addition to this, any change in isolated sections of the code require testing to be recommenced completely as engineers cannot adapt to changing data sets, thus elongating the testing timeline.

With automotive companies consistently shortening testing timelines in a bid to release products to market quickly, test engineers have now turned to machine learning themselves to define the test cases that will give them maximum test coverage in the limited testing time available and are even using it to identify sections of code that are most likely to contain errors.

Test organisations can maximise this effectively by pairing machine learning with software-defined test racks that can be controlled and configured remotely. This enables teams around the world to access the same test resources and run testing 24 hours a day, seven days a week. As an example, Cummins worked with National Instruments and their alliance partner Wineman Technology to build up a software-defined hardware-in-the-loop test architecture that allowed them increase test system usage by 20 percent.

Competing vehicle architectures
Just as technology changes, design too must change; design engineers today have been tasked with keeping themselves up-to-date with changing system architectures to include new and different sensor types. Aspects such as centralised versus distributed processing architectures, 5G radio technology and cloud computing have fundamentally altered bandwidth requirements to process the massive data streams coming off sensor systems. All these changes, hence, call for flexibility on the part of test engineers to deal with potential future requirements such as the addition of different types of sensors such as camera, radio or LIDAR sensors.

Choosing a test architecture based on commercial off-the-shelf platforms like PXI, is one way to handle these changing vehicle topologies. With a platform-based system, the test vendor integrates new technology and I/O as they become available to the market. These new capabilities can then easily be integrated into existing system designs.

Testing beyond industry and regulatory norms
Regulation is another area that test engineers must now be cognizant of. While most governments have left autonomous vehicle testing largely unregulated in the interest of industry investment, the productisation of this technology now requires regulation to ensure consumer safety and build consumer confidence.

This is a boon for automotive companies and allows for the rapid technology development we are currently seeing. The ISO 26262 functional safety standards represent the beginning of this endeavour. Other measures include the setting of an autonomous vehicle ethical standard by Germany that governs the decisions self-driving cars should make. Organisations now are poised to develop their own in-house test standards as technology and their expertise develops.

Despite the expectation of impending standards, the test regulation space is still relatively unknown. Test engineers need to be prepared to adapt test procedures and documentation as these standards continue to be written and evolve. They can start hedging against this unknown future today by investing in an open and adaptable test infrastructure.

Collaboration drives innovation
The rise of this new technology now demands that automotive companies move beyond their own boundaries to source experts in radar, machine vision, motor controls, battery chemistries, big data analysis, and machine learning. Collaboration with OEMs, integrators, governments, and test vendors is the current way forward as evidenced by competing companies like Lyft and Waymo in the US creating partnerships to build upon the strengths of each company and stay ahead of their competition.

Collaboration is even happening among test vendors who serve the automotive market. The ADAS IIT consortium, for example, was created by four companies with specialities in radar test, V2X testing, hardware-in-the-loop methodologies, and data management. The companies recognised that their skill sets complemented each other and teamed up to provide a more holistic solution to test engineers.

The road ahead
The move to this new world of collaborations will also bring its own set of challenges. The testing conducted by suppliers will have to be scrutinised by OEMs to ensure precision. OEMs will also have to deal with brand image issues and quality perception despite the failures being at the suppliers’ end. These challenges, however, do not diminish the importance of the progress that has been made in the 11 years since the completion of the DARPA urban challenge in 2007.

Bill Gates said, “We always overestimate the change that will occur in the next two years and underestimate the change that will occur in the next ten.” Organisations today are hopeful that these challenges are surmountable and that, one day, roadway accidents will become mere distant memories.

(This column was first published in the December 1, 2018 issue of Autocar Professional)