In order to make sharp turns along the fastest “track” without losing control, the driver must brake, turn and accelerate in a precise chronological order. The process depends on the limit of friction, which is controlled by known physical laws, which means that the self driving car can learn to finish as fast as possible (as some people have already done). But when autonomous drivers have to share space with other vehicles, this becomes a more difficult problem. Now, scientists have basically solved this challenge by training an artificial intelligence program to surpass human competitors in the surreal racing game Gran Turismo sport. These findings may provide new ways for researchers of self driving cars to make the technology play a role in the real world.
AI has conquered human players in some video games, such as StarCraft II and dota 2. Peter Wurman, director of Sony AI America and co-author of the new study, said Gran Turismo was very different from other games. The study was published this week in the journal Nature. “In most games, the environment defines rules and protects users from each other,” he explained. “But in the race, the cars are very close to each other, and AI agents must learn and use a very exquisite etiquette. In order to win the race, they must respect their opponents, but they must also maintain their driving route and ensure that they don’t just give way.”
Sony AI researchers used a technology called deep reinforcement learning to teach their programs. They reward certain behaviors of AI, such as staying on the track, maintaining control of vehicles and respecting race etiquette. Then they relax their plans and try different ways to achieve these goals. Sony’s AI team has trained several different versions of AI, called Gran Turismo Sophie (GT Sophie), each of which is dedicated to driving a specific type of car on a specific track. Then, the researchers compared the project with the champion of human tourism. In the first test conducted in July last year, humans got the highest total team score. In the second operation in October 2021, artificial intelligence broke through. It defeated the human enemy in the form of individual and team, and obtained the fastest lap speed.
Human players seem to have accepted their failure, and some people like to compete with AI. Erica Kato Marcus, director of strategy and cooperation at Sony AI, said: “some of the things we hear from drivers are that they also learn something new from Sophie’s actions.”. Emily Jones was the world finalist of the FIA certified 2020 Grand Prix and later competed with GT Sophie. “Ai uses a very complicated circuit, and I may only do it once. But it’s too difficult and I’ll never try it in a race,” she said. Although Jones said she felt a little powerless to compete with AI, she described the experience as impressive.
“Like many sports, racing is about getting as close to the perfect lap as possible, but you’ll never really get there,” Jones said. “It’s crazy for Sophie to see the perfect lap. There’s no way to run faster.”
Sony team is further developing artificial intelligence. “We trained an agent for each rail combination, a version of GT Sophie,” Wurman said. “One thing we’re thinking about is: can we train a single strategy to run on any car on any track in the game?” In terms of business, Sony AI also cooperates with the developers of Sony interactive entertainment subsidiary polyphony digital Gran Turismo, which may include a version of GT Sophie in the future update of the game. It is not a challenge for researchers to adjust their performance, but it is not an insurmountable challenge for researchers to do so.
Because Gran Turismo provides a real approximation of specific cars and specific tracks, as well as the unique physical parameters controlling each track, this study may also have applications outside of video games. Brooke Chan, a software engineer at artificial intelligence research firm openai, is a co-author of the openai five project, which beat humans on dota 2. “What I think is interesting is that the difference between it and dota game is that it is in a physics based environment,” he said. “It has not yet appeared in the real world, but it can still simulate the characteristics of the real world, so we are training artificial intelligence to better understand the physical world.” (Chan was not involved in the GT Sophie study.)
J. Christian Gerdes, a professor of mechanical engineering at Stanford University who was not involved in the new study, said: “Gran Turismo is a very good simulator. It is gamified in some ways, but it does faithfully represent many differences between different cars and different tracks.”. “In my opinion, this is the closest thing to anyone who has published a paper saying that AI can keep pace with humans in a racing environment.”
However, not everyone fully agrees. Steven Shladover said, “in the real world, you have to deal with things, such as cyclists, pedestrians, animals, things falling from trucks and things falling on the road. You have to be able to avoid these things, bad weather, vehicle failure, etc.”, A research engineer in the California advanced transportation technology partnership (California path) program at the University of California Transportation Research Institute. He was also not involved in nature. “None of these things appear in the game world.”
But Gerdes said that the success of GT Sophy is still useful because it subverts certain assumptions about the programming mode of autopilot. Automated vehicles can make decisions based on physical laws or artificial intelligence training. “If you look at the literature and, to some extent, what people do on the road, motion planners will tend to be based on optimized physics, and the perception and prediction part will be artificial intelligence,” Gerdes said. However, for GT Sophie, the motion planning of AI (e.g. deciding how to approach the corner with the highest limit of its performance without causing collision) is based on the AI aspect of formula. “I think the lesson that automated car developers should learn is that there is a data point, and maybe some of our preconceived ideas need to be revisited, that is, some parts of this problem are best done in physics,” he said. “AI may also be able to play there.”
Gedders also said that the achievements of GT Sophie can provide reference for other areas of human interaction with automation systems. He pointed out that in Gran Turismo, AI must balance the problem of realizing the fastest route around the track with the problem of smooth interaction with often unpredictable humans. “If we really have an artificial intelligence system that can make some complex decisions in that environment, it may not only apply to automatic driving,” gedders said, “But it also applies to interactions such as robot assisted surgery or machines that help families. If you have a task in which a person works with a robot to move something, in some ways, it’s much more complex than what the robot tries to do on its own.”