Allen School professor Abhishek Gupta, who directs the Washington Embodied Intelligence and Robotics Development (WEIRD) Lab, is interested in developing ways to help robots learn new skills with minimal human help and engineering. Gupta joined the Allen School faculty in 2022, and already he has introduced research that has shaped the future of robotics.
His contributions to the field earned him the IEEE Robotics & Automation Society (RAS) Early Academic Career Award in Robotics and Automation where the organization recognized him “for pioneering contributions to real world robotic reinforcement learning.”
“It is an honor to receive this award, which will support our group’s ongoing research into robot learning methods that are deployable and improvable in high-impact, human-centric environments,” Gupta said.
Gupta’s research has focused on developing methods that can make it practical for robots to improve safely and reliably through reinforcement learning. However, applying reinforcement learning to real-world robotics presents challenges, from safety, to reward specification, to efficiency; as such, its success has been limited to controlled settings or simulation. To address these challenges, Gupta established that robotic systems learning in the real world need to be able to determine the success measure from its own sensory input, and then reset the environment without human help so it can retry solving a task and learn from a small set of real-world interactions. He subsequently demonstrated what that process looks like via projects focused on dexterous, multi-fingered hands, fine manipulation tasks and teaching robots to grasp different objects through expert demonstrations.
His work was one of the first to propose solutions to the unavailability of automatic resets — one of the most fundamental, yet often overlooked, struggles in implementing robot learning in the real world. He developed a formalism and set of benchmark tasks that help robots navigate a continual, non-episodic world without assuming access to an oracle reset mechanism. His later work addressed the reset-free learning problem as actually being a multi-task learning problem, where a robot performing some tasks then resets others. These systems and algorithms set the stage for the next generation of deployment systems that will not just remain static, but improve autonomously on the job through multi-task reset-free data collection. Gupta has also built a range of reinforcement learning libraries and tooling to make real-world learning accessible to a broader range of developers.
Building off of that research, Gupta has been investigating how leveraging alternative sources of data such as generative models, simulation and videos can help scale up robotic learning. He and his collaborators were one of the first to develop GenAug, short for generative augmentation, which uses a diffusion model to synthetically modify robotic images for improved generalization. This system tackles the issue of the lack of in-domain robotics data through pre-trained generative models.
Gupta has also introduced a new method for robotic learning in simulation by using a real world to simulation and then back to the real world approach. Using small amounts of real-world data, researchers can construct a simulation of the deployment area that the robot can interact with and learn from. Simulations also have helped the design of effective policies that, when deployed in the real world, could help robots perform tasks with many variations and disturbances. For example, robots using this framework can efficiently put away dishes in a dish rack where they have to account for different dish shapes and configurations. Through a set of algorithmic ideas, Gupta and his collaborators were able to directly transfer behaviors from simulations to reality, and then efficiently finetune those behaviors using small amounts of real-world experience. More recently, Gupta and his students have developed techniques for learning unified prediction and control models from raw-video experience, allowing for the use of large internet-scale datasets in robot learning.
In addition to pioneering real-world reinforcement learning, Gupta has developed methods for unsupervised and self-supervised reinforcement learning. By combining the best of both worlds from model-based and model-free reinforcement learning, he introduced a simple and effective self-supervised reinforcement learning technique to make successor representations more practical using deep reinforcement learning methods. These representations help predict how likely a robot is to visit different states in the future. He and his collaborators were also among the first to develop a method that enables robots to learn useful skills without a reward function. This work has prompted a subcommunity of research on unsupervised reinforcement learning and skill discovery for both robotics and machine learning.
“Abhishek’s work has been consistently creative, innovative and practical, making a significant impact on the current and future state of robotic reinforcement learning,” Allen School professor Dieter Fox said. “He’s been a wonderful person to collaborate with, and we are very excited to have him in the Allen School. Abhishek’s work is laying the foundation for the next generation of robot learning, and he is poised to become one of the key leaders in our field.”
Prior to earning this year’s RAS Early Academic Career Award, Gupta has received a 2024 Amazon Science Hub Research Award and was named a 2023 Toyota Research Institute Young Faculty Investigator.
Read more about the RAS Early Academic Career Award.