Toyota and its Partners Are Building Maps for Autonomous Vehicles Using Satellite Images & Dash Cam Footage
Although many drivers feel Google Maps for navigation is indispensable, the maps and technology that human drivers rely on for turn-by-turn directions are simply not accurate enough for a self-driving car to navigate with. Autonomous vehicles require a much higher level of accuracy and detail for localization and navigation.
Instead, developers of autonomous vehicles rely on lane-level, high-definition or "HD maps", that contain much more data, including the location of trees, curbs and road signs, as well as where the painting lane markers are on the road are.
To build these HD maps and continuously keep them updated is a more complicated and time-consuming process, so automakers like Toyota Motor Corp are exploring ways to create HD maps more efficiently and for less cost and the results of a recent collaboration look promising.
The advanced research arm of Toyota, Toyota Research Institute-Advanced Development (TRI-AD), which is tasked with robotics and AI development for the automaker, has successfully completed a proof of concepts ("PoC") for HD map building using satellite images with a relative accuracy of less than 50 centimeters, which is enough accuracy to support autonomous driving.
TRI-AD worked with Maxar Technologies (Maxar) on the project along with Japanese IT services provider NTT Data. Maxar is a Colorado-based company specializing in Earth observation, orbiting satellites, radar and other services. Maxar and TRI-AD collaborated on the PoC and built automated HD maps for automated vehicles using the highest-resolution commercial satellite imagery.
The companies have been collecting the satellite images since April 2019.
Once a satellite image is captured, the map data is extracted. This is done by analyzing, correcting and removing "non-map image pixels" such as vehicles, shadows cast by large buildings, and occlusions due to the inclination of buildings in satellite imagery. What left is a cleaner image of the road which can be used to build HD maps.
The PoC project was able to achieve a relative accuracy of 25 centimeters for key navigation features, representing a significant improvement in accuracy. The maps created were verified to be useful enough for autonomous vehicles to navigate with.
The original satellite image.
The same image with all of the vehicles removed.
TRI-AD is Also Working with CARMERA
TRI_AD is also working with New York City and Seattle-based company CARMERA. The company develops and maintains fully regenerative, production HD maps using crowdsourcing.
CARMERA collects street images from its fleet partners and drivers that agree to share their dash cam data with the company. The data from the street images collected from the vehicles is then used to make HD maps for autonomous vehicles. CARMERA is backed by the venture capital investment arm of Alphabet Inc. Google Ventures.
For the PoC using just dash cam footage, TRI-AD and CARMERA conducted a similar camera-based HD mapping initiative in Tokyo to detect and place key road features―such as lane markings, traffic signals and signs.
While this method is not as accurate as using satellite images, it achieved a relative accuracy of 40 centimeters for key navigation features, which is still a major advance in camera-only detection. TRI-AD found that both methods are effective for HD map building for autonomous vehicles.
CARMERA builds maps using data from inexpensive dash cams. (Photo: CARMERA)
One of the issues with this method of creating maps is being able to keep the maps updated. An autonomous vehicle using a HD map for navigation requires it to be up-to-date at all times. However, simple things like a temporary road closure must be reflected in the map then pushed out to all of the vehicles that will use it for navigation.
This is a painstaking process, which requires a vehicle to be manually driven over the area to be remapped to record the new changes. In the future, this may be able to be done using satellite images.
These methods to create HD maps takes much less time, reducing the cost of mapping the world's roads for autonomous vehicles.
TRI-AD also collaborated with location data and technology platform HERE Technologies to increase accuracy. HERE specializes in HD map building. The company is owned by a consortium of German automakers, BMW, Mercedes Benz and Audi.
HERE assisted TRI-AD by correcting any positional errors in the vehicle data collected. Afterwards, HERE automatically generated HD maps including the lane level information required for automated driving.
In addition to all of the other partners on the various projects, TRI-AD partnered with TomTom, a leader in navigation and mapping services. The companies showed that urban roads, including lane markings, could be successfully created or updated in near real-time using TomTom's cloud-based transactional mapmaking platform, just with data collected from a vehicle's sensors.
Today just 1% of the world's road and lane network is covered by HD maps. That figure is assuming that HD maps available are for the entire highway road network in the US, China, Germany, and Japan.
The roads in the four countries makes up less than 1% of the world's total road network, so there is still much mapping to be done. However, with the research being done by Toyota and its partner companies, mapping roads for autonomous vehicles might be much easier in the future.
- Air Taxi Startup Lilium Raises $240 Million in Fundraising Round Led By Tencent
- Self-Driving Truck Startup TuSimple Announced Partnership with Auto Supplier ZF for the Production of Autonomous Trucks
- Baidu is Building a 5G-Powered, 20 Square Kilometer Test Site for Connected & Autonomous Vehicles in Chongqing, China
- Production of the Electric Performance Polestar 2 Begins in China
- Chinese Automaker Geely is Delivering Keys by Drone for New Vehicle Purchases
- Former Head of Google’s Self-Driving Car Project Pleads Guilty to Stealing Trade Secrets
- Uber Shares Surge 37% After its CEO Says the Ride-Hailing Giant Has $10 Billion in Cash to Weather the Coronavirus Crisis
- Chinese EV Startup XPeng Motors Granted Permit to Test its New P7 Sedan on Roads in the U.S.