Speaker: Arman Cohan, Yale University
Title: Piecing the Puzzle: Language Models for Multi-Document Contexts
Abstract: Large language models have shown significant capabilities in a variety of NLP tasks. Despite several advances, their abilities for processing multi-document tasks remain less explored. In this talk, I will first discuss our earlier work focusing on the extension of long-context language models for tasks that necessitate cross-document understanding, such as multi-document summarization. Subsequently, I will delve into our work on enhancing these models to manage both short and long-form generation across multiple documents. This work proposes a novel pre-training approach to improve the language models' ability to understand and integrate cross-document information. I will then present our work on extreme multi-document scenarios and the role of retrieval, providing further insights into these tasks as well as novel cross document applications in the scholarly documents. Finally I'll end by discussing challenges in evaluation and the need for new benchmarks. This talk describes joint work with Avi Cacialuru, Wen Xiao, John Giorgi, Yilun Zhao, Benjamin Newman, and several other collaborators.
Bio: Arman Cohan is an Assistant Professor of Computer Science at Yale University and a faculty Research Scientist at the Allen Institute for AI (AI2). His research spans various problems at the intersection of Natural Language Processing and Machine Learning, including Language Modeling, Representation Learning, Generation, and their applications to specialized domains include science. His research has been recognized with multiple awards, including a best paper award at EMNLP, an outstanding paper award at EACL, and an honorable mention at COLING. Prior to joining Yale, he was a Research Scientist at AI2 and an Affiliate Assistant Professor at University of Washington.
Date: Friday, April 19, 2024 - 1:30 - 2:30 PM EDT (North American Eastern Daylight Saving Time) via Zoom. On campus attendees will gather in CS 151 to view the presentation.
Zoom Link: Subscribe to mailing list (details at https://ciir.cs.umass.edu/talks) for Zoom Link/Passcode notifications; or click here for Zoom link and reach out to Hamed Zamani or Alex Taubman for the passcode.