Road changes such as lane shifts, new signs and speed-limit modifications can be confusing to drivers, both human and mechanical.
A human driver can quickly perceive and understand new or temporary changes to road conditions. A new project at UC Merced aims to deliver that same swift processing power to autonomous cars.
“There’s a gap between what can run in the vehicle at the power that the vehicle has available and the speed the vehicle needs,” said computer science and engineering professor Ross Greer. “There’s a gap between model performance in the lab and in the real world using real vehicles.”
Thanks to a grant from Santa Clara-based tech company NVDIA, researchers are looking into ways to narrow that gap. The artificial intelligence pioneer has selected the project “Edge-Deployed Multimodal Safety Reasoning for Autonomous Vehicles” for inclusion in its Academic Grant Program.
According to the proposal, construction zones and dynamic speed limits are rarely reflected on digital maps and are often communicated through signs, leading to missed or delayed responses and elevated crash risk. While recent AI advances support autonomous driving in complex scenarios, many systems remain limited to perception or recognition tasks and do not directly translate these insights into real-time control or safety responses.
Greer’s project aimsto leverage NVIDIA's technology to adapt driverless cars to better process what is happening on the roads, with explicit attention to uncertainty introduced by temporary map changes.
“NVIDIA has offered access to state-of-the-art hardware that will help our team translate research into vision-language foundation models from the lab to the road,” Greer said of NVDIA. The new effort builds on an earlier research project he had to bring new AI material into real-world vehicles.
“We can build these high-parameter AI systems that can run on a giant computer that is sitting in my office, but to get it to run in a smaller system in a car is a challenge,” he said.
Greer’s team won access to different types of embedded hardware that can run in a vehicle on low power, and a heavy graphics processing unit to train the models and make them smaller in the lab.
“We are always looking for a balance between model performance and electric power cost, and these questions become much more pronounced when there is a direct relationship between model performance and human safety,” Greer said. His team will compare three different types of NVDIA edge hardware, ranging from a large processor that costs $3,500 to the smallest one that costs $250.
“With memory and timing constraints, AI performance off these different devices varies,” Greer said. “But there are also additional power considerations, such as vehicle range. One device will consume the battery charge much more quickly than others.”
Working on this project with NVDIA is a great opportunity for Greer and those in his lab.
“The fact that UC Merced gets to contribute to this new frontier in autonomous driving with the support of NVIDIA is really exciting.”
