Members of Team Sounding Board onstage at AWS re:Invent 2017
A team of students and faculty from the Allen School and the University of Washington Department of Electrical Engineering has won the inaugural Alexa Prize from Amazon for Sounding Board, a socialbot that interacts with users through engaging and informative conversation. Sounding Board was chosen from among three finalists as part of a worldwide university competition designed to advance the state of the art in conversational artificial intelligence.
The Sounding Board team was led by EE Ph.D. student Hao Fang and EE professor Mari Ostendorf in collaboration with EE Ph.D. student Hao Cheng, Allen School Ph.D. students Elizabeth Clark, Ari Holtzman, and Maarten Sap, and professors Yejin Choi and Noah Smith of the Allen School’s Natural Language Processing (NLP) research group. Their goal was to develop an AI conversation agent capable of engaging in coherent conversation with humans about popular topics and current events. The UW team members produced a socialbot with an average conversation duration of 10 minutes and 22 seconds and earned an average score of 3.17 out of 5 from the company’s panel of independent judges.
As Fang, the team leader, explained, “The philosophy behind developing Sounding Board is bringing a variety of relevant content into a natural conversation. Ultimately, we hope Sounding Board can become a conversational gateway to online information that users enjoy talking with.”
Teams participating in the Alexa Prize competition built their socialbots using the Alexa Skills Kit and AWS cloud. For many on the Sounding Board team, the competition offered an opportunity to try something completely new.
“The students started from scratch, with no experience building a dialog system or working with Alexa skills,” noted Ostendorf, who served as the lead faculty advisor. “But together they brought a breadth of perspectives on language processing and a passion for understanding both the technical and human factors challenges of conversational AI.”
As the competition progressed, the team received continuous, real-world feedback on Sounding Board from millions of Amazon customers who interacted with the competing socialbots anonymously through Alexa.
“This competition was particularly interesting compared to other research competitions in that it let us see how people interacted with our system, outside of the lab,” noted Smith. “It’s pretty unusual for academics to get to observe that kind of real-world interaction.”
That interaction also produced some interesting challenges that the team had to overcome.
“One of the biggest challenges we faced when designing Sound Board was the diversity of people we interacted with,” said Clark. “We needed to handle a wide range of topics, conversational styles, and user personalities and interests.”
Sounding Board is both user-driven and content-driven. The system aims to understand users’ comments in multiple dimensions — such as directives, sentiment, and personality — to best serve their interests, while having interesting and timely content to talk about. Sounding Board actively harvests online content and leverages a knowledge graph to provide connections between related topics which can be used to steer the conversation. But as researchers found out, the process is a two-way street.
“While developing Sounding Board, we observed that users try to accommodate our system almost as much as it attempts to accommodate them,” explained Holtzman. “This led us into an investigation of what really makes conversations work with a resource few scientists ever get to touch: a live, diverse, interactive human data stream.”
According to Sap, the he and his colleagues were able to use this knowledge to provide a more personalized experience to users — a development that sets their socialbot apart.
“Sounding Board is unique in its ability to understand what type of person the user is, and is able to adjust parts of the conversation based on who it thinks the user is,” he said.
Rohit Prasad, vice president and head scientist for Alexa Machine Learning, and Ashwin Ram, senior manager of AI science for Alexa Machine Learning, announced Sounding Board as the winner of the Alexa Prize, which comes with a check for $500,000, earlier today at the AWS re:Invent 2017 conference underway in Las Vegas, Nevada. The runner-up was Alquist from Czech Technical University in Prague, and third place went to What’s Up Bot from Heriot-Watt University in Edinburgh, Scotland. Sounding Board was the only North American competitor to make it to the final round. A total of more than 100 teams from 22 countries applied to the inaugural competition.
According to Choi, this is only the beginning.
“We envision that conversational AI will be integral at the interface between humans and machines, and the Alexa prize makes an important step toward that vision,” she said. “It has been an exciting journey to build Sounding Board, and we look forward to working on crucial research challenges that we have identified along the way!”
In a blog post about the award, Amazon’s Ram announced that applications for the 2018 Alexa Prize competition will open on December 4.
“It’s been a rewarding initial year — for us and for students and faculty — as we work together to create the future of conversational AI,” he said.
Read the Amazon announcement here, the UW News release here, and a terrific behind-the-scenes look at how the competition unfolded courtesy of Wired here. Visit the Sounding Board website here.
Way to go, team!