"The next big milestone we are heading for now is proving self-driving is possible in traffic," said professor Sebastian Thrun, director of Stanford’s Artificial Intelligence Laboratory and the university’s racing-team leader. "Our goal at Stanford is to be able, within the next two years, to drive from downtown San Francisco to downtown Los Angeles with 100 percent autonomy–without any human intervention whatsoever."

The journey will take seven hours, Thrun said, and "will involve all types of traffic conditions, from urban driving to congested highways to very long traverses of interstates. We feel that this stretch of road is rich enough to show off our evolving self-driving capabilities."

In Thrun’s view, the 2005 Grand Challenge "has answered the question of whether you can drive autonomously at moderate speeds in an empty desert, which is relevant in itself but bears little resemblance to real driving. So the next question is, can you drive at higher speeds in real traffic?"

Semiconductor designers, automotive engineers, and software and middleware experts all say the answer is yes, and that cars that drive themselves will be a market reality at some point in the not-too-distant future. "It is only a matter of time until consumers have self-driving cars," said Thrun.

BMW’s ‘active steering’ technology adjusts the gear ratios based on how fast you are driving.

Sensors for such vehicles already exist, and semiconductors are coming along fast. "We see the autonomous vehicle as one of the key growth areas for automotive electronics over the next few years," said Peter Schulmeyer, director of strategy and marketing for transportation products at Freescale Semiconductor Inc. "The endpoint is the self-driving car, but what we see today is, step-by-step, the automobile is getting more and more autonomy–meaning the ability to act on its own without the driver or passenger activating any devices."

Stanford’s Thrun predicts that full autonomy–not just convoy lanes on the freeway–is at least 30 years away. But between then and now will come many milestones, such as autonomous military convoys and a whole raft of convenience and safety features that will slowly bestow various degrees of autonomy onto commercial and consumer vehicles.

"Advances in automotive electronics are spurring the bulk of developments in car design," said Mike Williams, principal analyst for Gartner Dataquest’s worldwide semiconductor group, who said that the worldwide automotive-semiconductor market surpassed $16.7 billion in 2005. "The safety-driven electronics innovations today will empower autonomous-vehicle capabilities tomorrow," Williams said.

Freescale’s Schulmeyer sees collision avoidance as a passing goal on the way to full autonomy, with all new innovations in automobiles pointing to increased automation. The path, he said, will progress "from simple convenience items like autonomous windshield wipers to safety systems like antilock brake [ABS] systems, to the next step, which is vehicle stability systems and adaptive cruise control. These will be the first systems that attempt to avoid collisions, rather than just minimize the damage, like pretensors and airbags."

Schulmeyer sees three distinct stages ahead for autonomous safety systems: First, radar will be proven out on adaptive cruise controls; next will come more-active safety systems, such as emergency brakes that apply maximum braking to minimize damage when an "inevitable collision" is detected; and finally, full collision-avoidance sys- tems will steer around upcoming obstacles to prevent collisions from ever happening.

Proven feasible
Since the 1960s, researchers have vigorously explored all sorts of schemes for self-driving cars–from Stanley-style total autonomy, where the vehicle’s software makes all the driving choices, to central control, in which a "master" traffic computer sends out signals to coordinate "slave" vehicles on an automated freeway. Many of these technologies have been proven feasible in real highway field tests, even as the electronics footprint has shrunk from trunk-filling to chip-size. Today the technology for automated highways could easily be retrofitted into the dashboards of modern commercial and consumer vehicles. So why don’t cars drive themselves already?

One reason is that vehicles need to be able to "drive by wire," meaning that all mechanical linkages, from accelerator to transmission to brakes and steering, have to be controlled by computer-activated electric servos. For instance, the computer directly controls antilock braking systems, and steering-by-wire is being installed in many cars today. BMW’s "active steering" technology, for example, uses a computer to adjust the angle of the steering wheel independently of the angle of the car’s front wheels–say, to compensate for side winds.

Planes like Boeing’s X-45 Unmanned Combat Air Vehicle point the way.

BMW, Daimler-Chrysler, General Motors, Honda, Mercedes, Toyota and Volkswagen all plan to use drive-by-wire capa- bilities in their vehicles for reasons of convenience and safety–but not for full auton- omy. "People don’t want a car that takes over the navigation, steering, brakes and just takes you to your destination as a passenger," said analyst Williams. "They don’t want cars that drive themselves, because that takes all the fun out of driving."

Nevertheless, "I would argue that even someone who loves to drive will have to admit there are times when a self-driving car would be welcome," said Stanford’s Thrun, himself the owner of a sports car, "because I love to drive."

If you could switch on the self-driving mode, he said, "you could become more productive–you could read or sleep or answer your e-mail, or even watch a movie." And such a car could keep the elderly, who might otherwise have to give up their driver’s licenses, independent longer.

According to Thrun, self-parking is safe enough to attempt in fully autonomous modes today. In fact, several automobile manufacturers have already demonstrated fully autonomous parallel parking. "Even though self-parking is in a static environment, it has the same requirements for complete computer control [over the steering, brakes, transmission and accelerator]," Thrun said. "We also have adaptive cruise controls, lane departure systems, ABS–all of which technically put the computer in control of the car."

Other safety features that today aim at enhancing a driver’s awareness may be used for full autonomy tomorrow. For instance, today a heads-up night-vision system can project on the windshield a "high-beam" view of the road ahead from infrared sensors that cut through fog. Tomorrow, a self-driving vehicle could use those sensors to recognize objects that match up with the GPS-based map.

"There is a lot of attention given today to a system that advances the awareness of people–systems that give you data from sensors that let you perceive in a certain way," Thrun said. "For instance, night-vision systems send out an infrared light deep into the scene [and] can be on high beams all the time–because the eye cannot see infrared. But with a camera that gathers infrared light, you can see the whole scene illuminated, greatly assisting people to see in the dark."

He also pointed to "radar emergency-warning systems that advise you to slow down because they see an obstacle through the fog ahead," as well as "a whole set of new driver-assistance systems" that sort out obstacle types. "That’s what will really make driving safer for everybody," he said.

Freescale’s Schulmeyer believes the transition to fully self-driving vehicles will proceed from fail-safe to fault-tolerant systems, in the pattern of the aeronautics industry. "Today, fail-safe systems have the option of just disengaging when they detect an error, such as when the airbag or ABS brake systems turn themselves off and light up a dashboard indicator when their self-testing fails," said Schulmeyer. "But fault-tolerant systems, on the other hand, have no safe position."

If a fault is detected in steering, for example, "you can’t just turn the steering off like you can an airbag when it fails self-test," he said. "For safety and reliability, when going from fail-safe to fault-tolerant, you need fault-tolerant communications between subsystems, you need redundancy of critical components and your software’s stability has to meet very critical safety standards."

To encourage automobile makers to create fault-tolerant communications systems, Freescale Semiconductor has created a consortium behind its free real-time software, called FlexRay (currently used by Audi, BMW, Bosch, DaimlerChrysler, Ford, Freescale, General Motors, Philips, Siemens and Volkswagen).

Likewise, Real-Time Innovations (RTI) Inc. is offering what it calls a "middleware" solution called Network Data Distribution System (NDDS) that handles real-time communications among an automobile’s many microcontrollers.

"NDDS middleware mediates between sensors and high-level software used for recognition. It distributes data from sensors to algorithms under predictable real-time constraints," said Rajive Joshi, a principal engineer at RTI.

As the glue that connects data to processing nodes, NDDS does symmetric peer-to-peer data distribution, eliminating the need for a central server–all nodes are identical as far as NDDS is concerned.

"NDDS feeds each algorithm the sensor data it requires, when it requires it, regardless of how many sensors there are or how many algorithms are accessing that same sensor data," said Joshi. "Sensors are ‘publishers’ of data streams and algorithms are ’subscribers’ to data streams, and if sensors are redundant, then when data becomes unavailable from one, NDDS substitutes data from another sensor in real-time without interruption. And even if no new published sensor data is available for a real-time subscriber, then NDDS notifies the subscriber of that eventuality, so its algorithms can substitute interpolated data for what’s missing."

In addition to sensor fusion, NDDS moves data navigation and actuation systems too. Joshi said that last year, the Department of Defense standardized on NDDS as middleware for its autonomous vehicles. NDDS is also installed in autonomous submarines used for servicing, installation, exploration, salvage and recovery operations under the sea. In addition, RTI’s middleware is installed on Robonaut, a humanoid robot designed by the Robot Systems Technology Branch at NASA’s Johnson Space Center in a collaborative effort with Darpa.

RTI offers versions of NDDS middleware for ARM, STMicroelectronics and Infineon microcontrollers. Separately, ARM offers its own software development tools, which the company claims enable integration of processing and communication among tasks handling body, chassis, safety and power train systems.

"ARM cores are running in about 65 percent of the world’s ABS and chassis control systems," said Wayne Lyons, director of embedded systems at ARM Ltd. "Choosing ARM’s cores allows the automobile designers to consolidate their architectures, moving from 20 or more that they use today, down to a handful."

Instead of maintaining software development tools for all these architectures, said Lyons, "you can standardize on our tools and concentrate on successive generations of new innovations," eliminating "all the problems of compiling from scratch for each new model."