Blind driver Mark Riccobono behind the wheel at the Daytona International Speedway. (Steven Mackay / Virginia Tech)
Last Saturday, a blind driver dodged cardboard boxes thrown in front of him while driving a modified Ford Hybrid Escape around the Daytona International Speedway.
He had only seconds to react to the obstacles.
"If we just put boxes on the track, people might think we planned the route," said Dennis Hong, whose robotics and mechanisms lab at Virginia Tech modified the cars.
Instead, Hong's team threw boxes from a van so they bounced around. “That shows everyone that their position is random, and that the drivers are really driving,” said Hong.
In addition to avoiding boxes and taking the raceway's turns, the driver, Mark Riccobono, also passed the van.
Fortunately, Riccobono and a second blind driver, Anil Lewis, had done it before.
"The other day, when we got to drive by ourselves, it surpassed any perception of thrilling," Riccobono wrote in an e-mail after a mid-January test run at Virginia International Speedway. Riccobono, who lost his sight at the age of five, is executive director of the National Federation of the Blind's research arm, the Jernigan Institute.
"It's scary and exciting," said Lewis, 46, who lost his sight 21 years ago and thought he would never drive again.
The demonstration at Daytona took place before blind supporters and fans gathered for the Rolex 24 sports car race as part of the National Federation of the Blind's Blind Driver Challenge to develop a car blind people can drive independently.
"We're trying to change people's minds about what blind people can do, and driving is going to change minds," said Lewis, the Federation's communications director. "This is taking it to whole different level."
Riccobono reached speeds of more than 25 mph and did not hit a single box.
Lewis was not worried about a small accident or two. "If it goes off too perfectly, people won't believe its credibility," Lewis said. After all, he explained, the car is a research project and people should expect some failures. Besides, "sighted" drivers have traffic accidents too.
Hong had other goals. "I want it to be perfect," he said. "This is a controversial project. I'm getting hundreds of e-mails from people saying, 'You won't believe how much hope this brings to us.' "But I'm also getting hate mail, saying 'Are you insane?'"
Hong understands. In fact, he had doubts until blind drivers tested a prototype.
Development
The National Federation of the Blind announced the Blind Driver Challenge in 2004. Only Virginia Tech responded, and it did not sign up until 2006.
At the time, Hong's lab was developing a driverless car for the DARPA Urban Challenge. Sponsored by the Defense Advanced Research Projects Agency, it called for robot cars to navigate through 60 miles of traffic, lights, stop signs, and obstacles at an old military base. Virginia Tech's upstart team surprised everyone by claiming the $500,000 third prize.
Hong thought he could use a similar approach to build a car for the blind. That was not the Federation's goal.
"They wanted the blind person to make active decisions, and actually drive and control the vehicle," Hong said.
Yet the two vehicles had much in common. Both use similar sensors. A laser light detection and ranging (LIDAR) system identified cars and other obstacles in the road. In addition, two forward-pointing cameras monitored the road as well as lights and stop signs. A GPS system located the car on a map, while an inertial measurement unit tracked the car's speed and direction in case it lost GPS contact.
The vehicles' computers automatically gathered all the sensory information and blended it together to create a model of the car's rapidly changing environment.
In the driverless car, the computer assessed the model, picked out the route, and used the drive-by-wire system to drive the car.
The Blind Driver Challenge vehicle was different. Instead of telling the car how to drive, it had to communicate this information to the driver, who then had to respond accordingly.
Hong's team faced two initial hurdles. First came money. Autonomous cars are expensive, but the Federation offered Hong only $5,000 to get started. He put together a team of 12 undergraduates and they bought a used gasoline-powered dune buggy on eBay for $2,000.
They bought whatever equipment was not donated. The LIDAR made by Hokuyo Automatic Co. was the most expensive component, and ordinarily cost about $8,000.
"Hokuyo originally donated it for another robotics project, but we used it on this," Hong related. "I was afraid to tell them, but when they saw what we did, they became big fans."
Driving By Touch
The second problem was more profound. How could Virginia Tech convey information fast enough to a driver who cannot see? After all, most computers communicate with humans through visual displays. For the blind, that wasn't possible.
Hong started by letting the computer pick the best route and communicate driving information -- fast, slow, right, left, stop -- to the driver.
For the dune buggy, the students built a vest from a massage chair vibrator. Different massage patterns told the driver when to speed up, slow down, or stop. In the Ford Escape, the expanded team of engineers built the vibrator, now branded SpeedStrip, into the driver's seat.
"We're experimenting with straight up-and-down and zigzag patterns, still figuring out which ones are most effective," Hong explained.
The dune buggy's original steering system used a steering wheel that made clicking noises when it turned. The car's computer told the driver how many clicks to turn the wheel. It was awkward, and forced blind drivers to listen to the computer rather than use their hearing to assess their environment.
The Ford Escape, by contrast, uses DriveGrip, a glove with a small vibrating motor on the knuckle of each finger. The more motors that vibrate, the sharper the driver needs to turn.
Finding the right vibration pattern proved a challenge. Hong considered which hand to signal for which turn; whether the motors should vibrate all at once or sequentially; and whether they should turn off suddenly or gradually as the driver completed the turn.
He also had to face the human-in-the-loop problem. An autonomous vehicle is easy for a computer to control because it responds to commands the same way every time.
Humans are another story. Different drivers may interpret the signal to turn hard differently. Even the same driver may respond differently on different days.
This showed up during testing. The computer would define the "road" as an 18 foot wide path on the 30 foot wide racetrack. "If I steered out of that lane, the car would shut down," Lewis recounted. This often happened when cornering.
"Sighted people get a chance to correct their errors," Lewis said. "We asked for the same chance, and when we got it. We got better at staying on the road."
In this case, the solution to the human-in-the-loop was to modify the car's controls so that the driver could learn to adapt to its signals.
The blind drivers and a blind engineer on the Virginia Tech team also suggested cutting off the ends of the DriveGrip gloves, since blind people need touch to sense their environment. They also collaborated on finding the best vibration patterns.
Lewis and Riccobono practiced for a total of seven days during the months before Daytona. Lewis says the Blind Driver Challenge will help blind people stay on the forefront of technology.
"We were worried that the knobs and buttons we used on appliances and phones were going to touch screens. We wanted to make sure there are non-visual interfaces out there for us," Lewis said.
Those interfaces are coming. They are what Hong calls "informational" interfaces. On the Escape, they communicate what is going on outside the car.
One is AirPix, a pad with pressurized air flowing through a grid of pinholes. It works like an air hockey table, but AirPix controls each pinhole's air pulses to form "pictures" that a blind person can view with his or her fingers, like Braille. A similar technology uses a 3-dimensional rubber membrane that changes shape to show road conditions.
The technology is too new to use at Daytona. Yet one day, similar technologies may not only help the blind to drive, but perhaps to navigate streets, use portable computer devices, or view notes from an electronic blackboard.
Get a daily look at what’s developing in science and technology throughout the world.