Google's New Self-Driving Car Is About to Hit the Streets

MOUNTAIN VIEW, Calif.—The rooftop of the new Google X building is a parking lot, but today it's also kind of a circus. There's even a tent. Chris Urmson, head of the self-driving car program, is discussing the autonomous prototype that Google built from scratch. Sergey Brin is giving interviews in French. Members of the local community of all ages and races and backgrounds have assembled. A blind woman who says she's never been in the driver's seat of a car is positively gleeful. A handful of press are here, as is ubiquitous transportation guru Gabe Klein, the former D.C. and Chicago DOT chief, chatting with some of them on behalf of NACTO about what it all means for cities, while wearing sunglasses.

And then there's the tiny electric prototype itself. It's running a preset loop with preset obstacles: pausing for a temporary construction sign, slowing for an oblivious pedestrian on his cell phone, trailing a cyclist a safe distance, stopping for a car that cuts it off, parking itself for another go. It would be a mundane route if not for the car's lack of a steering wheel, or gas pedal, or brake. Rides are taken two-at-a-time—vis-a-vis the Ark or the seating capacity, depending on your perspective.

Here's the news: Google is announcing that the prototype, which up to now has been developed in private, will soon be making its way onto the streets of Mountain View. With Jaime Waydo now on the self-driving car team, after 12 years at NASA working on the Mars rovers, it doesn't seem entirely inappropriate to paraphrase Neil Armstrong. It's a small step for the prototype and while it's not quite a giant leap for driverless technology—the car will be fitted with a steering wheel, gas, and brake during public road testing, as per state law—it's easy to imagine how this type of car will one day change everything about how people move around cities.

"I know it sounds cliché, but safety is job one," says Brin (not in French). "I think as far as this project goes, once we sort of deal with safety, we want to really think about what's the transportation experience we're providing. And not having to have a driver gives a lot of new possibilities in terms of what people can do with their time, in terms of how to organize these vehicles, where they should be at different times, how to make really efficient use of space on the roads—and so forth. It's a world of possibilities."

A German reporter, who has been hammering on the industry angle all day like a good countryman, asks what message Google is sending to auto makers.

"Ultimately I think self-driving technology should be on all vehicles," he says. "We don't attempt to be making huge number of vehicles. We really want to partner with German, Asian, or American automakers to bring self-driving to all the vehicles in the world."

Small car. Slightly less small plans.

The prototype is super small and loaded with sensors.

The self-driving prototype does not have an official name. It does have nicknames. Chris Urmson's kids call it the "koala." Someone at some point during the day floated the name Phyllis, which oddly fits. The self-driving team calls it just Prototype. (They call the fleet of Lexuses equipped with driverless technology—which will continue to operate on public streets—Lexus.) Google's marketing team is hesitant to christen it because they know any name will stick forever.

Photos were not allowed from inside Prototype. To be honest there's not much to see. Most of the interior consists of two seats separated by a central console. The console has two cup holders and some basic buttons: window controls, door locks, seat warmers. There's a big black button for GO and a red one for an emergency stop. The windshield is flanked by two monitors on either side and a camera at the top. Below it is a display panel that shows passengers where the car is located. Below that is a little space for cargo (today: two Google tote bags). That's about it. The color scheme is light blue and beige.

The exterior looks a little like a computer mouse with a very hunched back. It's white with light grey doors and a light grey hatch and otherwise unadorned except for a little Google stamp on one side and a battery door on the other. (Right now the car is electric but Urmson says it's "resource agnostic," capable of adapting to future power modes.) There's a flexible windshield meant to protect pedestrians. There are numerous sensors: most notably the little black gumdrop on top, which looks a little like a siren, and a vertical camera between the headlights, which looks a little like a nose. The headlights themselves, up close, seem to have freckles.

Jaime Waydo sorts the car's sensors into three types. There are lasers, which give the car high-resolution data about the world, tracking things both right beside it and up to two football fields away several times a second, 360 degrees (a human driver can only see 120 degrees). Radar tells the car how fast vehicles, pedestrians, cyclists, and various other obstacles are moving. The cameras pick up shapes and colors, good for detecting the likes of traffic lights and construction cones.

Less visible from outside or inside the car are the various safety precautions built into it. Prototype remains by design the prototypical defensive driver: it never travels in a blind spot, waits a full 1.5 seconds after a light turns green to proceed (privy to research about the high prevalence of accidents that occur just after signal changes), and generally responds to uncertainty with the most cautious option. Since there are no manual controls—meaning humans can't intervene in a pinch—Prototype was made with essential redundancies: the brake system, steering, and electronic and computing systems all have backups.

"If you want to get to the point where we can take the person out of the immediate control of the vehicle," says Chris Urmson, "then you have to put much more technology into the vehicle in the first place."

This version also has a number of limitations.

Asked why Google is releasing Prototype onto Mountain View streets now—in several different ways, by several different reporters—Urmson gave a consistently vague answer: it's ready to operate safely and reliably. He did offer a tasty peek at where the car might go next. Mountain View has great weather and a flat landscape. So to test the car in rain, the team is looking someplace that isn't drought-ridden California, and to test it on hills, it's looking someplace just north.

"You can imagine you'll see some of our vehicles out in San Francisco in the not-too-distant future driving out there," he says.

Prototype is currently in its fifth iteration. Right now there are roughly 25 of them, with plans to expand the fleet to a cool 100. Since Google is not a car manufacturer, Prototype has been assembled in partnership with Roush in Detroit. It’s been tested for endurance with what Waydo describes as "this giant spider that grabs the car by the wheels and it literally is shaking and bouncing and jouncing and jerking and wobbling the car all over the place"—a form of torture that calls to mind The Machine from The Princess Bride. It's gone through that wringer 18 hours a day for weeks on end.

On the streets of Mountain View, at least for now, a steering wheel and gas and brake pedals will be added to Prototype, in compliance with state law. The California DMV's "testing" regulations for autonomous vehicles require them to have manual controls. That stipulation may be removed when the state announces its "operating" regulations, which remain incomplete despite an end-of-2014 deadline, but in the meantime members of Google's driving team will be present inside the car, just as they've been in the self-driving Lexus for years.

Prototype (like New York City streets) has also been capped at 25 miles an hour. That qualifies it as a "low-speed vehicle" under federal regulations. This status will limit the car only to roads with speed limits of 35 miles per hour or less, says Urmson. It also means Prototype doesn't have to abide all the standards of conventional cars; it doesn't have airbags, for instance, and when asked if the car has been crash-tested, Urmson said only that it met the company's own safety measures.

The 25-mph speed limit obviously precludes the car from going on highways. That's fine with Urmson, who views the current version of the car "as a first learning step" to getting full self-driving vehicles on the road in the future. (Brin later clarified that the car is "quite capable" of going faster once deemed ready by developers: "We definitely wish to do full-speed vehicles," he said.) The low-speed trial is also the latest sign that Google sees this technology as especially impactful for cities and local communities, and has shifted the bulk of its efforts to that challenge.

"We've done an awful lot of testing on highways with the previous generation of vehicles and we'll continue to do that," says Urmson. "Right now we're pretty much focused on neighborhood driving and city driving."

Google's cars have learned a lot in the past year ...

Dmitri Dolgov, software lead for the self-driving team, boils the car's decision-making process down to four parts. The first is understanding where it is in the world, which it does with customized maps—loads more intricate than Google Maps—that prepopulate road curvature, lane widths, traffic signals, and so on. The car can't go anywhere without these maps, but Dolgov says the team is getting better at building them "high-quality, very fast, and at scale." The second part is knowing what's around the car, which is where all the sensors come in.

It's the car's final two steps—predicting what will happen next, and determining what it should do in response—that make city driving "orders of magnitude more difficult" for the Google car than highway driving. During our first ride on Mountain View streets in the Lexus car last year, it handled remarkably well but still struggled with something as common as a construction cone, at one point stopping helplessly before a line of them. Since then the car's situational learning and ability to classify objects has improved greatly, says Dolgov.

Here's a few things the car's been working on:

• It can distinguish the blinking lights of a police car, which are fine to maneuver around when off to the side, from the blinking lights and temporary stop sign of a school bus, which mean don't proceed.
• It's trying to learn casual gestures like the "let's go" wave of a traffic cop.
• It understands the basic parameters of a bike lane and also knows that a cyclist riding against the flow of traffic is likely to stick to the edge of the road (as opposed to darting out in front of it).
• It's recognizing similar patterns in joggers. Months ago, the car might have seen a jogger heading toward it and stopped until the runner passed; now it realizes the person will likely continue to hug the curb. Joggers that exit a perpendicular sidewalk and turn, with no intention to dash into the street, are another point of focus.
• It's beginning to recognize the subtle distinction between a left turn and a U-turn, which is a little wider and slower.
• It knows a bird will (probably) fly out of the way.
• It knows that occasionally people make terrible decisions—turning right from the far left lane, for instance, or running a red light—and is on alert for them. In one recent case, a cyclist entered the intersection after a light turned green; the SUV beside the Google car nearly hit the rider, but the Lexus waited patiently because it knew what was happening.

These instances, while rare, are not outside the imagination. But the car must be ready for events that fall into that category, too, especially in city settings. Dolgov described one circumstance that (correctly) puzzled the car into submission recently: it turned out to be a person in a wheelchair chasing a duck with a broomstick. In circles. In the middle of the street.

"This is not something we likely would have anticipated," he says. "And this is not something you can very accurately model."

… much of it at a private new training facility.

Google needed a way to test these rare cases before bringing Prototype onto public streets. So for the past year or so it's been running a structured testing program on a closed course at the old Castle Air Force Base in Atwater, California, just northwest of Merced. There a team led by Stephanie Villegas has been putting the car through various obstacles it needs to be able to handle but wouldn't reliably encounter in the everyday world.

"They happen very, very rarely if you're not specifically looking for them," says Dolgov. "And this is what these guys do: They go out looking for them, and then they put the car in those situations."

The testing generally addresses three types of incidents: those that require a very quick response time, those that result from breaking a traffic law, and those that are just plain weird. They have a dummy called Bob the Pedestrian, and they've been known to toss a beach ball in front of the car, or let loose a stack of papers on a windy day, or even set a canvas bag in the road and have someone pop out. One of the biggest focuses of the last six month has been teaching the car to react to pedestrians or cyclists that suddenly emerge from behind an obstruction.

Each trial incident can come with countless adjustments to its details and degree. So if they're testing how the car responds to a cyclist that appears out of nowhere, they might begin with a typical person on a bike 50 meters away. Then they might bring the cyclist closer and closer to determine a potential stress point in the car's response. And they might dress the person in a flowing black cape, ala Frank Costanza's lawyer, to make sure the car can still classify the bike. And then they might test the same scenario at night or on different road surfaces, just to be sure.

"If a cyclist cuts us off once every 1,000 miles [on the road], we can test that 100 times in a day," says structured team member Cam Crary. "And the cyclist can be wearing different things and coming from different angles and coming out from behind different objects."

Engineers can also tweak the parameters of a situation in a virtual program, enabling them to run even more trials than the structured testing team can reenact. That marriage of live and virtual testing—combined with roughly 10,000 miles of on-road driving every week—mean the Prototype is learning at an astonishing rate. And keep in mind that everything one car learns, the entire fleet learns, whether Lexus or Prototype, because they all share a brain. Urmson says each car has a computational power equivalent to five high-end laptops.

There is one thing the car still hasn't learned yet: honking. "We're exploring," says Dolgov.

Urmson on self-driving accidents, and the potential for cities.

Since Sergey Brin and everyone else at Google describe safety as Job One for the self-driving car, he had to acknowledge the recent Associated Press report that the vehicle has been involved in 11 accidents over the past 6 years. While the news was framed as a scare piece that questions the car's safety, Chris Urmson takes it as a compliment. "We've had 11 accidents over the last 1.7 million miles of total testing we've done," he says. "This is something we're pretty incredibly proud of."

In a blog post at Medium, Urmson wrote that "not once was the self-driving car the cause of the accident." Reiterating that position, Urmson says the car was rear-ended 7 times ("nothing you can do") and side-swiped twice ("again, only so much you can do"); in the post he adds the car was hit by a driver who ran a stop sign. Urmson also says the eleventh accident occurred when the car was not in self-driving mode.

Google reported the incidents to the California DMV, as required by autonomous vehicle regulations, though he says given their lack of severity they're "not things a normal driver would have to report." (The DMV has not released the reports because they contain confidential information.) In conjunction with the prototype's launch on public streets, the self-driving team plans to put up a website that serves as a forum for community feedback, releases monthly progress statistics, and mentions any incidents reported to the DMV.

Toward the end of the rooftop event, Urmson chatted a bit with CityLab about what Prototype, and the driverless technology more broadly, might mean specifically for urban environments. One idea he outlined is familiarly tantalizing to those who've been thinking about the transformative potential of self-driving cars in cities: namely, that they might operate as a shared robotaxi fleet.

"You can imagine if these vehicles could work, and they're in the community, that they're going to find their way in, either shared between individuals or shared by an entity that's owning a fleet of them," he says. "It may be that eventually things like metro transit authorities start buying these instead of buses. … If you can get these vehicles and they're inexpensive and you don't have to pay the same amount of up-front capital because they're smaller and you don't have to pay to operate them, you can have a much better transit service."

But overhauling a bus fleet, let alone an existing transit system, takes time, and cities don't always move quickly. Given that Urmson seems committed to a five-year timeline for getting a fully autonomous car on the road—one that aligns with his oldest son, Ethan, turning 16—I asked if it's not too soon for cities to start thinking seriously about how to incorporate the technology onto their streets. He said if he were in their shoes he might want to "see more evidence first," but that as the person providing that evidence, he's sure it'll arrive.

"It's on us and on the other folks developing the technology to show it works," he says. "It is one of these things where there is a big mental step that happens that day when these vehicles are out, in a community, and there's nobody in them some of the time. When that day happens, there's starting to be a proof point: Ok, now we really need to take this seriously, and start to plan for it. But we've got a little bit of work to do."