Award Number: 2039449
Award Title: NSF EAGER: Dynamic Contextual Explanation of Search Results (Defuddle)
Duration: 09/01/2020 - 02/28/2022
Principal Investigator: James Allan, PI
allan@cs.umass.edu
Center for Intelligent Information Retrieval (CIIR)
Manning College of Information and Computer Sciences
140 Governors Drive
University of Massachusetts Amherst
Amherst, MA 01003-9264
Project Abstract/Goals
This research project aims to investigate and develop Defuddle, an approach and a system that analyzes documents at the top of a search engine’s ranked list to find human-readable explanations for why documents were retrieved for this query and, unlike existing technology, for how the documents relate to each other. The resulting advances in result explanation will make it easier for people to make sense of what happens when they search the web or any other collection of text documents. Given that search is among the most common online activities, the reduced frustration and time savings will be substantial: the impact on e-commerce, information gathering, education, and the other myriad uses of search will be far-reaching. The Defuddle project will train one graduate student and one postdoctoral researcher. It will also be incorporated into coursework at both the undergraduate and graduate courses.
Defuddle provides explanations of each search result item in the context of the query and in light of the other top-ranked items. It treats the search engine as a “black box” so that the techniques can be applied broadly and independent of any particular engine. Defuddle focuses on designing and training appropriate neural models for explaining search results, without using any predefined explanation templates or explicit query facets. Defuddle lays the foundation for other search-related tasks that will greatly benefit from explanations: providing diversified search results such that the initially shown results cover the full range of query aspects; improving conversational search by elucidating why a user might prefer one answer over another; and particularly importantly, providing a mechanism to identify biased results, either alerting a user to the bias or possibly adjusting the ranked list to compensate. As permitted by human subjects review, data sets and software developed in the Defuddle project will be made freely available at or before the conclusion of the project.
Research Challenges
The problems this Defuddle proposal seeks to address are complex and difficult. Query- and user- contextual document explanation is a long-standing goal of the information retrieval research community, largely addressed in current systems by heuristic keyword-matching algorithms that do not illuminate how documents relate to each other.
Broader Impacts
If the Defuddle project is successful, it will make it easier for people to make sense of the results of searching on the web or in any other collection of text documents. Given that search is among the most common online activities on and off the web, the reduced frustration and time savings will be substantial.
Publications
Sarwar, S., Moraes, F., Jiang, J. and Allan, J., "Utility of Missing Concepts in Query Biased Summarization," in the Proceedings of The 44th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 21), Online, July 11-15, 2021, pp. 2056-2060.
Chowdhury, T., Rahimi, N. and Allan, J., "Equi-explanation Maps: Concise and Informative Global Summary Explanations," to appear in the Proceedings of the ACM FAccT* conference, Seoul, South Korea on June 21-24 2022.
Yu, P., Rahimi, N. and Allan, J., "Towards Explainable Search Results: A Listwise Explanation Generator," to appear in Proceedings of The 45th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2022), Madrid, Spain, July 11-15, 2022
Point of Contact: allan@cs.umass.edu
This material is based upon work supported in part by the Center for Intelligent Information Retrieval (CIIR) and in part by the National Science Foundation under Grant No. 2039449. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.