Imagine driving down a country road at night. It’s dark, raining and road construction cones block the lane ahead. Driving through such a scene, or even driving on big-city streets crowded with pedestrians, takes a mixture of awareness, caution, split-second decision-making and good judgment — often all at once.
Sandia is working with industry and academia to understand how all that experience and reflex can be entered into a computer to achieve what once existed only in science fiction — on-vehicle computation that enables highly automated driving in demanding environments. Given the limited amount of energy available in electric vehicles, one of the biggest hurdles is efficiency for the computers that will control automated vehicles.
“We see an opening to make progress in this direction for commercial automated vehicles,” said John Aidun, former manager of the team. “But beyond vehicles, it’s plain to see that there are multiple Sandia mission needs for autonomous or other smart systems that demand improved energy efficiency of size, weight and power-constrained, embedded computing.”
The need in automated vehicles is to ensure computers within vehicles are capable enough without relying upon the power of cloud-based computing, which would delay responses to driving conditions with the time needed for information to travel from the vehicle to the cloud and back.
“The car has to provide everything, because at first there will be little external computational infrastructure available to assist in automated driving,” explained team member Lennie Klebanoff. “However, there is almost nothing known about how much computation is needed and how that will be provided. Most assume the computer technology will somehow ‘be there.’ It may not be. At this point, we are not trying to solve problems. Rather, we are trying to figure out what the important research and development challenges are to enabling high-performance computing for highly automated driving.”
A working group of academic, government and commercial partners have joined the Sandia team in this effort, including engineers from the University of Michigan, Carnegie Mellon University, Arm, Hewlett Packard Enterprise, Intel Corp. and the U.S. Council for Automotive Research.
“The automotive environment makes the computational problem harder and even more fun to work on,” Lennie said. “Computer technology in cars has to survive shock and vibration, wide variations in temperature, satisfy size and weight restrictions on commercial-vehicle technology, not consume all the power available to the vehicle, all with safety being the most critical item. That’s where Sandia’s expertise and experience in engineering solutions in complex technical systems are crucial.”
The group organized a virtual workshop in May 2021 that brought together about 50 people from the automotive and microelectronics industries and research institutions. The workshop participants provided feedback about the research and development problems identified by the team, divided into four areas seen as critical to energy-efficient computing in automated vehicles, starting with the computer chips themselves. This feedback helped inform a research and development roadmap, meant to guide future research.
“The projected on-board computing needs for highly automated vehicles are unlikely to be met with currently available commercial computing technology,” John said.
The workshop participants agreed that advances must be made not only in the raw chip capability, like density of transistors, but also in computer architectures, as well as in the algorithms that simulate human decision-making and, finally, the sensors feeding information into the computing system about the real-world driving environment, including rain, darkness and traffic cones.
There was general appreciation that each participant is necessary to develop computer chips, system architecture and algorithms. Integrated sensors should be co-optimized to advance the energy efficiency of automated vehicle computing.
Sensors that can gather as much data as human perception does require a lot of calculating power, so one question for energy-efficient computing looms large: How smart does a sensor have to be?
“Do we need color capabilities or will black and white be enough?” Lennie asked. “How much spatial resolution is needed? To improve energy efficiency, we need to reduce the demand on computation just as much as we need to increase the computational supply that new chip technology can provide.”
Sensors and the other three technical areas — chips, system architecture and algorithms — all need to be considered when trying to improve computational energy efficiency.
These issues are not just theoretical nor isolated to computing for automated cars.
“Any research and development roadmap must be cognizant of the timescales for not only commercially creating the technology, but also, the timescale for implementing it,” Lennie said. “We described not only the research and development timeline, but also the associated timelines of chip commercialization and the timeline for implementation of the technology by the automobile manufacturers.” The work of the team can be seen on the Sandia Transportation Energy website.