Google’s DeepMind researchers have reached a significant achievement in robotics by successfully training 20-inch tall humanoid robots to engage in one-on-one soccer matches. Their study, published in Science Robotics, outlines how they utilized deep reinforcement learning (RL) to impart complex locomotion and gameplay skills to the robots.
The commercially accessible Robotis OP3 robots mastered the abilities to run, kick, block, recover from falls, and score goals autonomously, without any manual programming. Instead, AI agents guiding the robots acquired these skills through trial and error in simulated environments, guided by a reward system.
How the Robot works
The robotic soccer system operates as follows: First, researchers trained separate neural networks, termed “skill policies,” for fundamental movements like walking, kicking, and getting up. Each skill was acquired in a specialized environment rewarding the robot for mastering that particular ability. Next, through policy distillation, the individual skill policies were combined into a unified master policy network capable of activating the appropriate skill as per the situation.
Subsequently, the researchers refined the master policy through self-play, where the robot engaged in simulated matches against previous versions of itself, leading to continuous enhancements in strategy and gameplay. To ready the policy for real-world deployment, the simulated training environment was randomized in terms of factors such as friction and robot mass distribution, enhancing the policy’s robustness to physical variations.
Finally, following exclusive training in simulation, the finalized policy was transferred to real OP3 robots, which then participated in physical soccer matches without requiring additional fine-tuning. The outcomes are remarkable – with dynamic and agile movements, the robots spin to change direction and coordinate their limbs to kick and balance simultaneously.
DeepMind Excited about their Achievement
DeepMind describes their achievement in the paper, highlighting the agent’s robust and dynamic movement skills, anticipation of ball movements, and ability to block opponent shots. Compared to a conventional rules-based policy programmed specifically for the OP3, DeepMind’s RL approach yielded significantly superior performance.
The AI-trained robots exhibited a 181% increase in walking speed, a 302% increase in turning speed, a 63% quicker recovery from falls, and a 34% harder kick of the ball. Coupled with DeepMind’s advancements in AI-optimized football coaching in collaboration with Liverpool FC, sports may be transitioning towards a more heavily digitized era. It may only be a matter of time before we witness a Robot League featuring custom robots competing in high-octane sports matches.
See also: SAG-AFTRA And Record Labels Unite To Safeguard Artists From AI
