The joint compilation | relaxation, xiu-qin li, meng-hua zhang, lee angel, Gao Yu, Brian Chan, haohappy, dan-dan zhang, Liu Cong

Ai-drive reports

Recently, Waymo (formerly Google’s self-driving project) submitted a 43-page safety report to the U.S. Department of Transportation detailing how Waymo equips and trains its self-driving cars to avoid common and unexpected situations while driving. The report is Waymo’s most complete first look at its self-driving technology.

According to new wisdom, Waymo may launch self-driving taxi service in November this year at the earliest. Waymo’s report is not only a summary of its eight years of development, but also a sign of its confidence in itself and its new technology as autonomous driving moves toward commercialization. Perhaps we are about to usher in a new world of autonomous driving, and this report is a starting point.

Here is the full Chinese version of Waymo’s report, compiled by Lei Feng’s authors Zhang Chi, Li Xiuqin, Zhang Menghua, Li Angel, Gao Yu, Brian Chan, Chen Hao, Zhang Dandan and Liu Cong.

The following is the full report:

Self-driving cars could improve road safety and provide millions of people with new ways to move. Whether it’s commuting to work, taking your kids to school, or saving lives, fully autonomous vehicles have enormous potential — because they can change people’s lives for the better.

Safety is a core mission at Waymo, and it’s the goal we set up Google’s Self-driving project eight years ago.

Every year, 1.2 million people around the world die in traffic accidents. In the United States, the number of such accident-related deaths is increasing. A common phenomenon is that 94 percent of traffic accidents are caused by human error. We believe Waymo’s technology could save thousands of lives lost in traffic accidents every year.

Our commitment to security is reflected in everything we do, from Google’s culture to how the technology is designed and tested. In the safety report on Waymo’s self-driving technology, we detail what Waymo does: safety.

This overview of safety systems highlights the data miles that Waymo’s autonomous vehicles have accumulated over 3.5 million measured miles, as well as the important lessons learned during tens of millions of miles of simulated driving.

Waymo’s safety report also influenced the federal policy framework issued by the U.S. Transportation Agency, Autonomous Driving Systems 2.0: Thinking from a Safety Perspective (Automated Driving Systems 2.0: A Vision for Safety ). The transportation agency’s framework lists 12 safety design elements and encourages companies to test and deploy their own autonomous driving systems to address the areas.

In this report, we will outline the processes associated with each safety design element and how they underpin the development, testing, and deployment of autonomous vehicles as a whole.

Fully autonomous vehicles will only be accepted by the public if they are safe. That’s why we’ve been working on safety. All in all, our self-driving cars will achieve safer transportation and more diverse mobility to better serve the needs of the masses.

The report is divided into five parts:

One, system safety procedures: safety design

How do Waymo’s self-driving cars work

Iii. Test and verification methods: To ensure the safety and effectiveness of the vehicle

Safe interaction with the crowd

Five, the summary

Waymo’s system security program – security design

As the first company to do autonomous driving on public roads, Waymo had to do it all itself.

In the early days of Waymo, we established our system security program, a security philosophy that has been embedded in our technology testing and development processes. This project is comprehensive and robust, we call it: safe design.

Safe design means that we think about security from the ground up and incorporate it at every system level and at every stage of development, from design to testing and verification. It is a multi-pronged approach, built across industries including aerospace, automotive best practices and defense systems.

Based on these practices, we have conducted robust tests on the various components of autonomous vehicles to ensure that all subsystems can operate safely when integrated as a complete self-actuated system.

This approach also helps verify that the vehicle is safely on the road as a fully autonomous vehicle. At the same time, we can also learn about any changes or failures in system components, subsystems, or other aspects, as well as the changes caused by the overall autopilot system.

This process has inspired many of Waymo’s critical safety features, including redundant critical safety systems that enable vehicles to safely stop in the event of a technical failure, along with the use of multiple sensors and a widely implemented testing program that allows us to make rapid technical improvements.

Waymo’s system security solution

Our system safety program covers five different safety areas: behavioral safety, functional safety, collision safety, operational safety and non-collision safety. Each area requires a combination of testing methods that allow us to verify the safety of fully autonomous vehicles.

Conduct safety

Behavioral safety refers to the driving decisions and behaviors of vehicles on the road. Just like human drivers, autonomous vehicles obey the rules of the road and must navigate safely in all situations — both expected and unexpected.

Waymo employs functional analysis, simulation tools, and road driving to fully understand the challenges presented in the design areas of our business, and to develop safety requirements and a multi-pronged testing and validation process.

Functional safety

Functional safety is designed to ensure that our vehicles operate safely, even if there are system defects or failures. This means setting up backup systems and redundancy mechanisms to deal with unexpected situations.

For example, all of our autonomous vehicles are equipped with a second computer that takes over immediately if the main computer fails, allowing the vehicle to stop safely (i.e., minimum risk conditions). Every one of our cars has backup steering and braking, and a lot of other redundant features throughout the system.

Crash safety

Crash safety, or crashworthiness, refers to a vehicle’s ability to protect its occupants through a variety of measures, from the structural design to protect occupants to features such as seat restraints and airbags to mitigate injury or prevent death.

Crash safety is defined by the US Federal Motor Vehicle Safety Standard (FMVSS), published by the US National Highway Traffic Safety Administration (NHTSA). Automakers must prove that their base car models meet FMVSS requirements.

The operational safety

Operational safety refers to the interaction between our vehicles and passengers. With operational safety, we can ensure that consumers have a safe and comfortable experience in autonomous vehicles.

Our approach to building safe products is informed by risk analysis, existing safety standards, extensive testing and best practices in a variety of industries. For example, through our early ride project (further described in Section 4), we have developed and tested a user interface that enables riders to clearly indicate their destination, direct the vehicle to the side of the road, and contact Waymo.

Non-collision safety

We physically secure people who may interact with vehicles. For example, hazards from electronic systems or sensors can cause injury to occupants, vehicle technicians, drivers, first responders or bystanders.

Security process

To reduce potential internal risks, security must be emphasized in the design and then validated to demonstrate that security risks have been reduced to identifiable levels.

Our approach starts with identifying hazardous scenarios and mitigation measures for potential risks. These measures can take many forms, such as software or hardware requirements, design recommendations, program controls, or additional analysis recommendations.

We use a variety of risk assessment methods such as pre-hazard analysis, fault tree, design failure mode and consequence analysis (DFMEA). This continuous process is closely related to ongoing engineering and testing activities and safety engineering analysis.

The risk analysis process helps us identify the requirements for the architecture, subsystems, and components of the autonomous driving system. These safety requirements have evolved from a range of subsystem and systems analysis techniques, various systems engineering processes, and federal and state laws and regulations. The analysis also supports Waymo’s evolving behavioral security testing needs and how the system detects and handles failures.

Waymo has conducted extensive testing on public roads, closed loops, and simulated driving environments. We use the information gleaned from this test, as well as studies of national crash data and natural driving, to provide additional analysis of potential hazards.

The combination of tools from these plays an important role in Waymo’s understanding of system readiness. Based on this understanding, we can fully analyze and evaluate system safety before fully autonomous driving can be carried out on public roads.

How do Waymo self-driving cars work?

Autopilot system

Unlike today’s cars, such as adaptive cruise control or lane-keeping systems, drivers are constantly monitoring them. Waymo’s autonomous driving system is based on human participation. Waymo’s self-driving system consists of software and hardware that, when integrated into the car, perform all of the driving functions.

In autonomous driving lingo, Waymo’s autonomous driving system can perform the entire dynamic driving task in a specific geographic area and under certain conditions, without the human driver having to provide the controls.

SAE International defines this technology as Level 4 for autonomous driving systems, which means that in the event of any system failure, our technology provides the vehicle with the ability to safely stop with minimal safety risk.

Unlike Level 1, Level 2, and Level 3, which are lower levels of autonomous driving, Level 4 gives the vehicle the ability to safely stop in the event of any system failure without the need for a human driver to take over.

Fully autonomous driving: Keeping humans as passengers at all times

Advanced driver assistance is one of the first technologies the Waymo team is exploring. In 2012, we developed and tested Level 3 autonomous driving, which allows vehicles to drive themselves on single-lane highways, but the process still requires a human driver to take over. In our internal tests, we also found that humans rely too much on the technology and don’t monitor the road carefully.

As driver assistance technology becomes more advanced, humans are often asked to switch from “passenger” to “driver” in a matter of seconds, but these scenarios are rarely used in more challenging or complex situations. The more tasks the vehicle takes on, the more complex this transition becomes.

Avoiding the problems associated with this switching process is part of the reason Waymo is developing fully autonomous cars. Our technology will focus on all driving, allowing humans to remain passengers in the car.

Target and event detection response: vehicle sensors

To address the complex needs of autonomous driving, Waymo has developed a series of sensors that allow autonomous vehicles to monitor 360 degrees, day and night, with a field of vision the size of three football fields.

The multi-layer sensor suite works seamlessly together to create a 3D image of the entire field of vision and display both dynamic and static objects, including pedestrians, bicycles, passing vehicles, traffic lights, buildings and other road features.

LiDAR system

Working day and night, LiDAR (Light Detection and Ranging) outputs millions of 360° laser pulses per second, measuring the time it takes to bounce off the surface and back to the vehicle.

Waymo’s system includes three types of lidar developed internally: short-range lidar that allows vehicles to observe and monitor continuously; High resolution medium range lidar; A new generation of powerful long-distance lidar with a line-of-sight area up to three football fields.

Visual (camera) system

Our visual system consists of a camera that looks at the world like a human, with a 360° field of vision, whereas humans only have a 120° field of vision. Because Waymo’s high-resolution vision system detects color, it can help the system spot traffic lights, construction areas, school buses and gray lights for emergency vehicles. Waymo’s vision system consists of multiple sets of high-resolution cameras that work well over long distances, in daylight and in low brightness.

Radar system

In general, radar uses wavelengths to sense objects and motion. These wavelengths can travel around objects such as raindrops, allowing radar to work in rain, fog and snow. Waymo’s radar system has a continuous 360° field of vision that tracks the speed of passing vehicles in front, behind and on either side of the vehicle.

Supplementary sensor

Waymo also has some additional sensors, including an audio detection system that can hear the sirens of police cars and emergency vehicles hundreds of feet away, and GPS, which can supplement a vehicle’s knowledge of its geographic location.

Waymo self-driving software

The self-driving software is the “brain” of the vehicle. It makes sense of the information from the sensors, and the “brain” can use that information to help the vehicle make the best driving decisions.

Waymo has spent eight years building and perfecting the software, using machine learning and other advanced engineering techniques. After years of careful design and testing, Waymo has reaped billions of miles of simulated driving and more than 3.5 million miles of on-the-road driving experience.

At the same time, our system also has a deep contextual understanding of the world, which is a key differentiating part of Level 4 autonomous driving technology.

Waymo’s self-driving software doesn’t just detect the presence of other objects, but really understand what that object is, how it might behave and how it might affect the behavior of our vehicles on the road. That’s how Waymo’s vehicles can drive safely in fully autonomous mode.

Since our software is made up of different parts, Waymo will detail three important components here: perception, behavior prediction, and planning.

perception

Awareness is part of the Waymo software’s ability to detect and classify road objects, as well as estimate speed, heading and acceleration. Our self-driving software can take countless details from Waymo’s sensors and turn them into a real-time view.

Perception helps the vehicle distinguish between pedestrians, cyclists, motorcyclists, vehicles and other objects, among other things. It can also distinguish the colors of static objects such as transmitting signals. For these objects, perception allows our system to semantically know what’s going on with the cars around it — whether the traffic light is green, whether the car is lit, whether the lane is blocked.

Behavior prediction

Through behavior prediction, our software can model, predict, and understand the intent of each object of the road. Because Waymo has millions of miles of experience, our vehicles have been modeled with a high degree of precision in terms of how different road objects are likely to behave.

For example, our software learned that pedestrians, cyclists, motorcyclists might look similar but behave very differently. Pedestrians can be slower than cyclists or motorcyclists, but both can swerve.

planning

Our planning software takes into account all the information gathered from both the perception and behavior prediction programs and maps the vehicle’s path. In our experience, the best drivers tend to be defensive. That’s why we train defensive driving behaviors. Such as moving away from other drivers’ blind spots and giving extra space to cyclists and pedestrians.

Waymo’s planning software prioritizes these steps. For example, if the autopilot software thinks the lane ahead is closed due to construction and predicts that the bike in the lane will move, the planning software can make a decision to slow down or make room for the cyclist in advance.

Based on the road experience, we also improve the driving experience to ensure that the passengers in the vehicle are smooth and comfortable on the road. It is also natural and predictable for other road users.

Design operation range: to ensure that the vehicle can operate safely under specific conditions

Operational Design Domains, or ODD, are the conditions under which autonomous systems can operate safely. Waymo’s scope includes geographic location, road type, speed range, weather, time of day, and national and local traffic laws and regulations.

In fact, the ODD for autonomous driving may be very limited. For example, a one-way fixed route on a low-speed public street or a private site such as a business park in a temperate climate during the day. Waymo, however, aims to be able to navigate city streets in a wide geographic area and under a variety of conditions. Our vehicles are already capable of driving in severe weather, such as moderate rain, during the day and at night.

Waymo’s systems are also designed not to operate outside the scope of an unapproved design operation. For example, passengers cannot choose destinations outside of our approved geographical location, and our software does not create routes outside the “geo-fenced” area.

Similarly, our vehicles are designed to automatically detect sudden changes that may affect safe driving within their ODD, such as snowstorms, so that the vehicle can safely stop in time (i.e., reach minimum risk conditions) until driving conditions improve.

Waymo’s vehicles are also subject to federal, state and local laws within its geographic area. Any variation in these requirements, as required by law, is considered to be a safety requirement in our system, including the associated speed limits, traffic instructions and signals.

Before our vehicles enter a new area, our team learns about any unique road rules or driving habits one by one, updates the software in time, and allows the vehicles to respond safely. California and Texas, for example, have different rules about how to make right turns in bike lanes.

In the meantime, Waymo’s ODD will continue to evolve. Our ultimate goal is to develop fully autonomous driving technology that will allow humans to get from point A to point B at any time, anywhere, and under any circumstances.

As we continue to grow and validate the capabilities of our systems, we will continue to expand the scope of our design operations and bring our technology to more people.

Minimum risk condition: Ensure that the vehicle can transition to a safe stop

For vehicles with low level of autonomous driving, when the road environment is too complicated to handle the vehicle or the vehicle itself fails, human drivers need to perform the control of the vehicle.

For a fully autonomous car, Waymo’s technology must be robust enough to handle these situations on its own.

If our autonomous vehicle cannot continue a planned journey, it must be able to make a safe stop, known as a “minimum risk state” or “rollback.”

This may include the following situations: the automatic driving system senses a fault, the vehicle has a collision, environmental conditions change, which may affect the safety of driving within the set design operating environment, etc.

Waymo’s system is designed to automatically detect each of these scenarios. In addition, our system runs thousands of times per second, checking the system and finding system errors. Waymo is also equipped with a series of redundant designs for critical systems such as sensor systems, computing systems and braking systems.

Our vehicle response varies depending on factors such as road type, current traffic conditions, and severity of technical failure. Based on these factors, the system will determine an appropriate response action to ensure the safety of the vehicle and its passengers, including pulling over or stopping safely.

Vehicle redundancy – Safety first automated driving system

Backup computing system

The standby computing system always runs in the background with the intention of controlling the vehicle to perform a safe stop when it detects a primary computing system failure.

Stand-by braking system

If the main braking system fails, we have a full backup braking system that takes effect immediately. When a fault occurs, both the primary braking system and the standby braking system allow the vehicle to perform a safe stop.

Backup steering system

The standby steering system has independent controllers and independent power supplies to perform redundant steering controls. For the main steering system and the standby steering system, when one of them fails, the other can execute the steering operation of the vehicle.

Backup power system

Provide two independent power supplies for each critical drive system. These independent power supplies ensure that our vehicle’s critical drive components remain available in the event of a single power failure or circuit interruption.

Backup collision detection and collision avoidance systems

Multiple collision detection and collision avoidance systems continuously scan objects in front and behind vehicles, including pedestrians, bicycles and other vehicles. In rare cases, these backup systems can control the vehicle to slow down or stop when the primary system is undetected or unresponsive to objects in the path.

Redundant inertial measurement system: For vehicle positioning

Redundant inertial measurement systems help the vehicle accurately track its trajectory. The main inertial measurement system and the redundant inertial measurement system cross-check each other, and the other system performs the vehicle location when one detects a fault.

Data recording and post-accident behavior

Waymo’s self-driving technology will never stop advancing. Waymo has a robust system to collect and analyze data generated by existing vehicles on the road. Any useful lessons we learn from one car are shared across the fleet.

Waymo’s systems can detect that a collision has occurred and will automatically notify Waymo’s back office operations center, where our trained experts can initiate post-collision response procedures, including communicating with law enforcement and first responders and sending personnel to the scene. We also have passenger support specialists in our operations center who can communicate directly with passengers through the on-board sound system.

After a crash, we can analyze all available data, including video and other sensor data, to assess what might have caused this accident. In the meantime, we can make any appropriate software changes and upgrade every car in the fleet accordingly. Any problems affecting the safety of the vehicle will be fixed and safety tests will be conducted before the vehicle is upgraded.

Cyber security of autonomous vehicles

Waymo has developed a robust process for identifying, prioritizing, and mitigating cyber security threats. Our security practices are based on Google’s security processes and follow the guidelines issued by NHTSA and Auto-ISAC.

To further enhance cybersecurity, Waymo has also joined Auto-ISAC, an industry action plan aimed at increasing cybersecurity awareness and collaboration in the global automotive industry.

How to build a map for self-driving cars to use

Before self-driving vehicles hit the road, our mapping team first uses sensors from test vehicles to create highly detailed 3D maps. These maps are different from basic satellite images or online maps.

Instead, Waymo’s maps give cars a deep understanding of the physical environment: the type of roads, their distances, dimensions and other geomorphic features.

We use this data and add highlighted information, including traffic control information, such as the length of crosswalks, the location of traffic lights, relevant signs, etc.

Waymo’s system focuses on dynamic parts of the environment, such as other road users, through maps installed on cars. Our system cross-references real-time sensor data with on-board 3D maps to detect road changes.

If a change in the road is detected (such as congestion at an intersection due to a collision ahead), our cars can replan the path within the designed operating environment and inform the back office so that other vehicles can avoid driving in the area.

In this case, the map not only adds to our software as a reference point, but also provides important information feedback to the system. These detailed custom maps provide a comprehensive understanding of each driving position. Coupled with our deep understanding of the system, Waymo ensures that vehicles are only used in the environment they are designed to operate in.

Waymo’s approach to security

1. Build verifiable software and systems

2. Encrypt and verify the communication

3. Build redundant security measures for critical systems

4. Restrict communication between critical systems

5. Provide timely software updates

6. Model and prioritize security threats

We conducted a comprehensive review of all potential safety access points for autonomous driving systems, both inside and outside the physical vehicle, and took steps to limit the number and functionality of those access points.

This starts with working with our automotive partners to identify and mitigate vulnerabilities in the base vehicle. We take known threats into account in our software design and vehicle design processes to ensure that our systems and vehicles are designed to counter these threats.

New software releases go through a process of peer review and validation. Our risk analysis and risk assessment processes are designed to identify and mitigate these risks, including those related to cyber security. In Waymo’s design, safety is crucial, such as steering, braking, and control isolation from outside communications.

We also consider the security of wireless communications. Waymo’s vehicles don’t rely on a fixed connection to remain secure. While on the road, all vehicles’ communications with Waymo (i.e., redundant connections) are encrypted, including those of Waymo operations support personnel and passengers. The vehicle can communicate with our operations center to gather more information about road conditions, while our vehicle performs driving tasks in real time.

These protections help protect against those who have physical access to autonomous vehicles, including passengers and malicious people nearby. We have different mechanisms to observe abnormal behavior and internal mechanisms to analyze these events.

If we become aware of an attempt to compromise vehicle safety, Waymo will trigger company-level incident response procedures, including assessment, containment, recovery and remediation.

Iii. Test and verification methods: to ensure the performance and safety of the vehicle

Waymo’s technology has been extensively tested, including open road, closed road and simulation, which makes every part of the system effective, reliable and safe when operating.

Waymo’s self-driving cars are made up of three main subsystems, all of which have been rigorously tested:

  • The vehicle itself, certified by OEM;

  • Internal hardware, including sensors and computers;

  • Autonomous driving software for making driving decisions.

These subsystems are combined into fully autonomous vehicles for further testing and verification. Testing the hardware and software as a whole ensures that the autonomous vehicle meets all the safety requirements we have set for the system.

Vehicle safety

Waymo’s current self-driving cars are retrofitted from a 2017 Chrysler Pacifica hybrid Minivan with self-driving systems. The modified vehicles are certified by Fiat Chrysler (FCA) to meet all applicable federal Motor Vehicle Safety standards (FMVSS).

Autopilot hardware test

In a technical collaboration between FCA and Waymo, Waymo’s autopilot system, including sensors and hardware, was integrated with a modified Pacifica Minivan supplied by FCA.

To make sure the integration worked, Waymo ran thousands of additional tests on top of the FCA’s. These tests include private test scenarios, lab and simulation scenarios that assess every safety feature of the vehicle, such as brakes, steering, and physical controls such as locks, headlights, and doors.

Through testing, we can ensure the safety of the vehicle in manual mode, autonomous mode (with the test driver behind the wheel) and fully autonomous mode (with no one in the vehicle). In general, the purpose of the test is to make sure that the vehicle can operate safely with autopilot.

Autopilot software testing

Like hardware, self-driving software follows the principle of “Safety by Design.” Waymo rigorously tests every component of its software, including perception, behavior prediction and planning, and overall software.

Our technology will continue to learn and improve. Every update to the software goes through a rigorous release process. Each update will go through simulation tests, closed road tests and public road tests.

Simulation test

In the simulation, we rigorously test any modifications and updates before deploying them into the vehicle. We also identified the most challenging situations the vehicle encountered on public roads and digitized them into virtual scenarios for the autonomous driving software to practice in simulations.

Closed road test

The new software will be rolled out to a handful of cars first so experienced drivers can test it in private venues. We can use different versions of the software on different vehicles to test new or specific features.

Real world testing

Once we were sure the software worked as expected, we started testing it on public roads. It will be careful at first to push software updates to the entire fleet after the cars can safely and consistently follow their intended routes. The more distance you drive on public roads, the more you can monitor and evaluate the performance of your software.

As we drive more miles, we will further improve the driving experience and update the software. This continuous feedback process builds confidence in the system and enables the vehicle to achieve SAE Level 4 autonomous driving.

Simulators: Virtual worlds help vehicles learn advanced real-world driving skills

Waymo’s simulator can play back real-world driving data with each new version of the software, and build new real-world virtual scenarios for testing with our software.

Every day, as many as 25,000 virtual Waymo vehicles drive up to eight million miles in simulators, consolidating existing skills and testing new business skills to help the vehicles safely drive in the real world.

One example: a yellow left-turn light flashes at the corner of South Longmore and Southwest In Mesa, Arizona. This type of intersection can be tricky for both humans and driverless cars, as drivers have to find gaps in traffic after entering a five-direction intersection. Turning left too early can be dangerous, turning too late can block traffic.

Simulators allow us to practice this single situation many times to master a skill.

How the simulator works

Step 1: Start with the visual world

* We can recreate a very detailed virtual reality version of the Eastern Valley

We built virtual intersections with the same dimensions, lane lines, kerbs, and traffic lights with a powerful suite of custom sensors.

In the simulator, instead of a single highway, we can focus on the most challenging intersections, such as left turns with flashing yellow lights, errant drivers and bicycles that don’t follow the rules.

Step two: Drive, drive, drive

In the simulator, we can have different vehicles in the fleet go through the same intersection many times under the same driving conditions. As shown above, we’re simulating an intersection where the self-driving car is passing.

Our software can practice this scenario thousands of times, as the bright yellow left turn is digitized in the virtual world. Whenever we update the software, we can test changes to the same intersection under a variety of driving conditions.

This is why we are able to move forward naturally at the intersection with the yellow light and insert ourselves into the complex traffic. In addition, in the simulation we could practice this new skill at every intersection we encountered with a flashing yellow light in order to iterate the software more quickly.

Step 3: Create a lot of change

* Through the “blur” process we can change the speed, trajectory and position of various objects on these virtual streets

Next, we can explore countless possibilities with this tricky left turn.

Through a process called “blurring,” we vary the speed of traffic and the timing of traffic lights to ensure the car can still find a safe distance. By adding simulations of pedestrians, motorcycle lane changes and even joggers crossing the road, the scene can become busier and more complex to see how they affect our driverless cars.

Step 4: Validate and iterate

* To make the scene more complex, we can add cars, pedestrians, and cyclists that are not present in the original scene

Today, our driverless cars have learned how to confidently turn left at an intersection with a flashing yellow light.

This new skill becomes part of our permanent knowledge base and is shared with every car in the fleet. In turn, we will use real-world driving and our own enclosed test site to validate the experience in the simulator, and then the cycle starts again from the first step.

Ability to drive normally

Fully autonomous vehicles must be able to handle all the everyday driving tasks expected of human drivers within the same business design domain.

This means that autopilot systems need to demonstrate that they have sufficient skill, or “behavioural capability,” which is necessary for the desired location and operating conditions.

The U.S. Department of Transportation (DOT) has recommended that Level 3, Level 4, and Levle 5 (SAE) autonomous driving be able to demonstrate at least 28 core competencies that easily meet the findings of a California partner, THE Institute for Advanced Transportation Technology (PATH), at the University of California, Berkeley’s Institute of Transportation Studies. DOT also encouraged companies to “design, test, and validate autonomous driving systems, taking into account all known behavioral capabilities.”

Waymo’s security plan expands its 28 core competencies in breadth and depth, and we test a wide range of scenarios in complexity to ensure our systems can safely handle real-world challenges. In addition, we have identified additional categories to expand on the initial 28 core competencies.

For each capability, the Waymo team created a variety of individual tests on closed scenarios and emulators. For example, to test Waymo’s ability to turn left, we built dozens of real-life situations and tested whether our vehicles could react properly. We brought challenging variables into ordinary road exercises, including traffic flow on multiple routes and blocking the view of vehicles with big trucks.

We also use simulators to create hundreds of different variations in each scenario. Through the virtual test, we can also create a whole new scenario where a car turns left so that we can further test this skill.

As we expand the business design area, the number of core competencies may increase (for example, to operate in North America, our systems must be able to safely travel in snow), and the number of tests in each category may expand to include more complex and unique scenarios.

While this scenario test can demonstrate the core driving skills of our software, these abilities need to be translated into the real world. Therefore, this is just a starting point. Next, we need to test the hardware and software of our vehicle with the results of simulation verification, and then seamlessly integrate the software and hardware so that it can drive on public roads and show the capability of Waymo self-driving cars in real traffic conditions.

Conduct field tests in enclosed facilities

Waymo designed and built a privately owned, closed-end testing facility on 91 acres in California specifically for testing needs.

Nicknamed the Castle, the privately-owned facility resembles a sim city and includes everything from motorways and suburban driveways to railway crossings. Our team uses these facilities to validate new software before updating it to our fleet on the road.

We can also create challenging or rare road conditions on these sites so that our vehicles can gain experience in special situations.

We were able to run thousands of “structured tests” on closed roads, recreating specific scenarios for learning and testing. To help our simulator get the material, we set up over 20,000 simulation scenarios in Castle.

Each scenario recreates a driving situation we want to practice, such as a impatient driver making a quick lane change or a person suddenly getting out of a parked car that might be seen only once in a while on a public road.

We’ve recreated situations where people come out of roadside tents or portable toilets, and skateboarders lie down on their boards and throw piles of paper in front of the sensors.

This “structured testing” is key to accelerating technological progress and ensuring vehicle safety in everyday driving and challenging driving conditions.

     

Full self-driving car test

After testing the underlying vehicle, autonomous driving systems and software, we tested the entire autonomous vehicle, including crashworthiness tests on closed roads, reliability and durability tests, and road tests with the driver behind the wheel.

Open road test

Waymo has a comprehensive road-testing program that has been updated and improved over the past eight years.

This is an important part of proving our technology, finding new challenging solutions, and constantly developing new skills. Road tests require highly trained drivers in the vehicles to ensure safety.

Our test drivers have extensive professional training in understanding the entire driving system and how to monitor the car safely on open roads. They also took defensive driving courses. With this training, the driver can monitor the system condition when testing the vehicle on the open road and take control of the vehicle when needed.

In road tests, tens of thousands of miles driven each week are used to evaluate software. We monitor the system to ensure that it effectively displays the characteristics of vehicle behavior. Then we look for solutions that can build on this feature to facilitate more stable driving.

Real-world testing provides a continuous feedback loop that helps us keep the system updated. Engineers make these changes by monitoring real-world solutions, tweaking software and updating driving systems.

Repeated testing and validation on the open road helped us to test our capabilities more safely while expanding the vehicle’s operational design.

Real scene experience

Over the past eight years, Waymo has tested in four US states and driven itself in more than 20 cities — from sunny Phoenix to rainy Kirkland — amassing more than 3.5m miles of self-driving data in the process. Every time we visit a new place, we can gain experience in different road environments, street scenes and driving habits.

For example, in Phoenix, we were able to test how sensors and software respond to desert conditions, including extreme temperatures and dust. In addition, we learned how to make new vehicles more durable, such as watering trucks in the middle of the road from 3 MPH to 45 MPH. Austin has horizontal traffic lights, while Kirkland offers a more humid experience.

In new cities, we encounter people every day who are not used to seeing self-driving cars on the road. We get a lot of fresh perspectives and perspectives from these different people — the willingness of people to use autonomous vehicles, how they think about autonomous vehicles and so on, and these people can provide a lot of guidance for us to develop and update autonomous driving technology.

Autopilot in extreme temperatures

Our autonomous vehicles also need to operate reliably and safely in extreme cold and very hot temperatures. Waymo engineers developed the hardware and software for autonomous driving, creating a suite of systems that can operate reliably in extreme environments.

High temperature poses a challenge to all modern technology. Electronic devices, such as mobile phones, can overheat and shut down when used in the hot sun. However, our autopilot systems must also work safely in hot conditions.

Our vehicle has a special cooling system that allows these electronics to operate at very high temperatures, such as when the engine is running at maximum power and the system is using maximum energy. Waymo engineers conducted extensive experiments in a wind tunnel that mimicked almost every weather condition, including the highest temperature ever recorded on Earth.

In addition to wind tunnel testing, we also tested our autonomous vehicles in three of the hottest places in the United States: Las Vegas, Davis Dan, and Death Canyon.

Davis Dan sits on the arizona-Nevada border and has a long, steep desert road to drive under the hot sun. Las Vegas allows us to test our vehicles under the hot sun in endless stop-and-go city traffic jams. Death Valley has the highest officially recorded temperature on Earth, 134 ° F (57 ° C).

During the test, we closely monitored the system temperature, recording more than 200 different temperature points per second to verify that our self-developed sensor suite and algorithm worked properly.

Test vehicle anti-collision ability

In addition to testing core capabilities, our engineers also performed crashworthiness tests in various scenarios (Appendix 8 documents Waymo crashworthiness test scenarios).

Waymo has completed thousands of crash-avoidance tests on private test roads. Each test was conducted under different driving conditions, allowing us to analyze the car’s response. We use simulators to test these scenarios and improve overall software capabilities.

We learned what kind of crash to test from a lot of data, including our analysis, such as NHTSA’s fatal crash database, and our extensive experience with autonomous vehicles to expand to NHTSA’s 37 pre-crash scenarios. We also tested other scenarios, such as other road users causing potentially dangerous situations, such as vehicles veering out of lanes, large vehicles cutting into target lanes, motorcycles entering the road, pedestrians jaywalking, etc.

In 2015, NHTSA released data on the most common pre-crash scenarios. For example, 84 percent of collision scenarios are covered by just four crash catalogs: a back-end collision, a vehicle turning or crossing an intersection, a vehicle going off the curb, and a vehicle changing lanes. Therefore, preventing or mitigating such collisions is an important goal of our test program.

Hardware reliability and durability testing

Autonomous vehicles, like conventional vehicles, must operate reliably. This means that the vehicle and every component must function in extreme environments and throughout its life cycle.

Waymo engineers designed unique stress tests. Using our knowledge of fatigue physics to accelerate environmental stress on the vehicle and its components, we compressed years of real-world use into days or weeks of testing.

We exposed the parts to ultraviolet radiation, bombarded them with powerful water guns, submerged the car in a bucket of near-zero temperature, slowly corroded, rocked and powerfully vibrated in a room filled with salt spray, heated and chilled in a humid space for weeks.

We analyze any type of failure and improve the design to increase component reliability, monitor the health of each sensor and the vehicle itself, so that potential hazards can be identified and addressed before problems arise.

Safe interaction with passengers

Our vehicles are designed to drive themselves, so the user interface is designed with the passenger in mind, not the driver.

That’s why we developed special in-car content and user interfaces. This helps riders understand what Waymo’s self-driving vehicles are doing on the road, letting riders know things like choosing a destination, pulling over, and contacting Waymo support lines if needed.

In addition to providing passengers with safe and intuitive daily travel solutions, Waymo has developed processes for responding to emergencies. For example, our vehicles can not only detect collisions and respond to emergency vehicles on the road, but we have trained with the legal department and first aid personnel, the agencies that might have contact with our vehicles.

Finally, the potential of autonomous driving can only be fully realized through increased public awareness and acceptance. In October, Waymo launched “Let’s Talk Self-driving” — the world’s largest public education campaign about autonomous driving.

Through partnerships with national and regional safety groups and senior groups, we hope to engage the public and educate them about how this technology works and the enormous benefits behind autonomous driving technology.

Waymo’s early test driver program

We wanted to understand how a self-driving car could meet People’s Daily transportation needs, whether as a private vehicle or a shared vehicle, or make mass transportation cheaper. That’s why, in April, we launched the Early Rider program in Phoenix, the first public trial of Waymo self-driving cars.

Our test riders come from a wide range of people, from families with teenagers to young workers. They use our vehicles for their daily needs, from commuting to work to taking their kids to soccer practice. It is vital that early adopters learn how to use the vehicle, and our research team works closely with users to provide them with project information on how to use the vehicle and how to provide feedback.

For the last 100 years, when vehicles were designed, they always assumed a human driver. The test ride tells us a lot about how people want to interact with the vehicle and the experience of being a passenger rather than a driver. Their experiences helped us create a more intuitive and convenient in-car experience.

Driving experience

Waymo’s user experience is driven by four main principles: giving riders the information they need for a seamless ride; Helping passengers anticipate what will happen; Proactively communicate vehicle feedback on road conditions; Help passengers and vehicles coexist safely.

We want passengers to know what information the vehicle is receiving and why they are acting accordingly. Each vehicle provides passengers with useful visual and auditory information throughout their journey, helping them understand the actions the vehicle and other road users are taking. In Waymo self-driving cars, a display provides visual driving information, such as the destination, the current speed, and the route the vehicle chooses. A voice system notifies passengers.

We’ve already provided a variety of ways for riders to interact with the vehicle, such as pressing a physical button, moving an App and talking to a Waymo driving support expert.

Make Waymo vehicles easier to use

According to

The Waymo passenger display screen displays important traffic information, such as destination and arrival time. It also shows static road elements, such as traffic lights and stop signs, as well as dynamic elements in the environment, such as vehicles, bicycles, and pedestrians.

In this way, the passenger can understand that the vehicle is sensing and responding to these elements, thus gaining more confidence in the vehicle’s capabilities.

Start button

At any time, users can use the App or a button in the car to start their journey.

Pull over

The vehicle has a pull over button, and when the button is pressed, the vehicle will find a nearby safe place to park.

Mobile App

Participants in Waymo’s early user program, using a mobile App in a Wayo vehicle, can initiate destination requests. The App also allows users to give feedback and contact Waymo for support.

Driving Support Group

Waymo created a user support group to answer questions from early adopters. The experts can be contacted through a mobile App or a button in the car.

Accessibility: Providing opportunities for people who cannot drive today

We believe Waymo’s technology has the potential to improve security and mobility for people around the world. From the beginning, Waymo has listened to, collaborated with, and worked with the disabled community.

We continue to learn about the unique needs of different drivers. At the same time, these requirements as we understand them will constitute new features that will allow people to gain experience that they have long had to rely on others.

We also know that we cannot accomplish our goals alone. Waymo is committed to working with partners to build vehicle platforms and solutions that can serve more people.

Accessibility features under development

A barrier-free mobile App

We are building a mobile application that is intuitive and convenient. Designed for use with Android TalkBack, iOS VoiceOver and other accessibility services.

Voice prompts and tools

Visually impaired drivers need us to locate the vehicle at the starting point. We are exploring specific “pathfinding” features, including ways in which drivers can have their vehicles provide voice to help guide them. Additional voice prompts can be opened in the mobile App and used in the car to keep drivers informed of their journey.

Braille label

Our autonomous vehicles have drive buttons with braille labels that allow visually impaired drivers to start the vehicle, pull over, or call an operator who can provide them with more assistance and information. These buttons can also be used in mobile apps.

Video display

At each stage of the journey, deaf and hard of hearing drivers can use on-screen video prompts to keep track of the vehicle.

Barrier-free driving assistance

We use the driving help in the form of dialogue, through the video display or audio in the car to enable all drivers to obtain all driving capabilities.

 

Emergency situations and interactions with law enforcement officers and first responders

Wammo’s autonomous vehicles can interact with law enforcement officers and first responders. Using our customized sensor suite, including an audio detection system, our software can identify nearby fire trucks, detect their flashing lights, and hear sirens from hundreds of feet away. Audio sensors can identify the direction in which alarms may be coming from, improving vehicle safety and responsiveness. Once an emergency is detected, our vehicles can respond in time, pull to the side of the road or stop immediately.

Waymo also provides government information for each city we tested and provides a range of traffic management mechanisms. In some cities, Waymo also conducts on-site training to help police and other emergency workers identify our vehicles in an emergency.

We plan to continue this kind of on-site training, and we plan to expand it to make our vehicles smarter.

With the Chandler, Ariz., Police Department

We’ve partnered with the Chandler, Ariz., Police department and fire Department to conduct emergency vehicle testing of our self-driving minivans. Local police cars, motorcycles, ambulances, fire trucks and unmarked special operations vehicles track, pass and guide our vehicles with a powerful suite of sensors, including remote audio detection systems. Our sensors can collect data on a variety of speeds, distances and angles — building a visual and audio library that will help our vehicles respond safely when they encounter emergency vehicles on the road.

Five, the summary

Together, Waymo has focused on one thing for eight years: making fully autonomous driving a reality. We adhere to safety design, and our culture places safety, and open communication over safety, at the core. All of us at Waymo want to achieve this goal and make autonomous driving safe and available to everyone.

This report summarizes our efforts to ensure the safe deployment of fully autonomous vehicles. We are excited about the potential for this new technology to improve road safety and provide new travel options. 【 the 】

Welcome to follow @xinzhijia in Toutiao, Tiantan Express, UC Headlines, a bit of information, Sina Weibo, Sohu, netease.

Recommended reading:

Interview with Waymo’S CEO: What is the self-driving leader hiding?

Scan code to sign up for a course to become an autonomous driving engineer

For more information about the course, click “Read the original article”.