Kuo-Hao Zeng, a Ph.D. student working with Allen School professor Ali Farhadi, has been named a 2021 J.P. Morgan Ph.D. Fellow for his research focused on the problem of forecasting for decision making in artificial intelligence (AI). While Zeng’s work is in the context of visual forecasting and action, his algorithms and findings can be extended well beyond pixels, for example, they can facilitate policy learning for robotics applications.
Zeng aims to use his Fellowship to help AI predict future outcomes from current observations and past experiences, just as humans are able to start doing at infancy. Equipping artificial agents in robotics, finance and medicine with such capabilities can assist in important — even potentially life-saving — work. For instance, these agents can assist warning systems for autonomous vehicles in assessing long-term risk by forecasting future situations for real-time response, or act as decision making agents for trading stock where the agent has to anticipate the future trend before developing a strategy. In the medical field, they can help where a model needs to predict the tumor growth to assist the doctor in making a diagnosis.
Zeng develops deep learning models for visual forecasting and integrates these models into embodied agents for long-term decision-making tasks in interactive settings. In one paper, published at the 2020 Conference on Computer Vision and Pattern Recognition (CVPR), Zeng addressed the problem of visual reaction by teaching a virtual robot to forecast the future trajectory of objects and plan accordingly to interact with them. By playing catch with a drone in a near-photo-realistic simulated environment, Zeng adjusted the robot’s actions based on visual feedback and seeing its environment. From that he introduced a model that integrates forecasting and planning.
Zeng’s subsequent work has focused on a novel problem in robot investigation — training a robotic agent using reinforcement learning to unblock paths towards a target. So far he has achieved good results with agents performing actions and those actions changing the environment around them in a specific way. By explicitly learning the expected outcome of actions using his proposed neural physics model, Zeng showed that agents can now plan in more complicated environments. The paper describing his findings, “Pushing it out the way: Interactive Visual Navigation,” will appear at CVPR 2021.
“I am incredibly impressed with the quality of research Kuo-Hao conducts,” Farhadi said. “He is an outstanding researcher with a strong publication track record compared to his peers at his stage. He has worked on very challenging problems and has done outstandingly well so far. He is on his way to be a star researcher at the boundary of decision making and forecasting in real world and complicated situations.”
So far, Zeng has advanced visual reasoning for robot learning through eight papers published at top AI and computer vision conferences, with another one currently under submission.
Congratulations, Kuo-Hao!