A team of UMass CICS / CIIR researchers was among 10 teams selected to compete in this year’s prestigious Alexa Prize TaskBot Challenge. According to Amazon, the year-long competition is “the first conversational AI challenge to incorporate multimodal (voice and vision) customer experiences.” The CIIR team has received $250,000 of funding from Amazon for their participation.
Selected from over 125 universities worldwide, the CIIR team is led by Hamed Zamani, CICS Assistant Professor and CIIR Associate Director as the Faculty Advisor and principle investigator (PI) of the team. In addition to Zamani’s students, the CIIR team also consists of students who are advised by Bruce Croft, CICS Distinguished Professor Emeritus and CIIR Former Director, and Mohit Iyyer, CICS Assistant Professor. CIIR doctoral student Helia Hashemi, advised by Croft, will lead the students’ effort of CIIR participation in this competition.
The other selected universities for this year’s Alexa Prize Challenge include Carnegie Mellon University, National Taiwan University (NTU), NOVA School of Science and Technology (Portugal), Ohio State University, Texas A&M University, University College London, University of California Santa Barbara, University of Glasgow (with CIIR PhD alum Dr. Jeff Dalton as the team’s faculty advisor), and the University of Pennsylvania. The winning team will receive a $500,000 prize.
The UMass CICS/CIIR team’s focus will be on the IR and NLP aspects of the challenge on cooking and do-it-yourself (DIY) domains. Their work consists of an information need understanding pipeline for identifying the correct cooking recipe or DIY guideline, and a pipeline for walking the user through the steps in the selected guideline until the task is successfully finished.
During the competition, the researchers will work to develop an interactive retrieval model that can handle various types of complex queries; to design a novel multimodal result presentation model that highlights the unique characteristics of each retrieved item to ease the decision making process for the user, and to train models that can make mixed initiative interactions with the users to make sure they are smoothly doing the task.
“We are excited to take on this TaskBot Challenge”, said Zamani, “As a world-leading research group on conversational search, we have recently developed models for mixed-initiative conversational search and invested on pushing the boundaries in multi-modal conversational search systems. This challenge is a unique opportunity for us to expand our development and evaluate our systems within the Alexa platform by interacting with real users.”