I have recently joined the Department of Computer & Information Sciences at the University of Delaware! Go to my home page at UDel.

Publications

• 2008
  • Evaluation Over Thousands of Queries
    Ben Carterette, Virgil Pavlu, Evangelos Kanoulas, James Allan, and Javed A. Aslam. SIGIR 2008.
  • Evaluation Measures for Preference Judgments
    Ben Carterette & Paul N. Bennett. SIGIR 2008.
  • Here or There: Preference Judgments for Relevance
    Ben Carterette, Paul N. Bennett, D. Maxwell Chickering, and Susan T. Dumais. ECIR 2008.
• 2007
  • Million Query Track 2007 Overview
    James Allan, Ben Carterette, Javed A. Aslam, Virgil Pavlu, Blagovest Dachev, and Evangelos Kanoulas. TREC Notebook 2007.
  • Evaluating Search Engines by Modeling the Relationship Between Relevance and Clicks
    Ben Carterette & Rosie Jones. NIPS 2007.
  • Hypothesis Testing with Incomplete Relevance Judgments
    Ben Carterette & Mark D. Smucker. CIKM 2007.
  • Semiautomatic Evaluation of Retrieval Systems Using Document Similarity
    Ben Carterette & James Allan. CIKM 2007.
  • A Comparison of Statistical Significance Tests for Information Retrieval Evaluation
    Mark D. Smucker, James Allan, and Ben Carterette. CIKM 2007.
  • Robust Test Collections for Retrieval Evaluation
    Ben Carterette. SIGIR 2007.
  • Research Methodology in Studies of Assessor Effort for Information Retrieval Evaluation
    Ben Carterette & James Allan. RIAO 2007.
• 2006
  • Minimal Test Collections for Retrieval Evaluation
    Ben Carterette, James Allan, and Ramesh Sitaraman. SIGIR 2006.
    Winner of the Best Paper Award at SIGIR 2006!
  • Learning a Ranking from Pairwise Preferences
    Ben Carterette & Desislava Petkova. SIGIR 2006.
  • N Semantic Classes Are Harder Than Two
    Ben Carterette, Rosie Jones, Wiley Greiner, and Cory Barr. ACL/COLING 2006.
• 2005
  • Incremental Test Collections
    Ben Carterette & James Allan. CIKM 2005.
  • When Will Information Retrieval be Good Enough? User Effectiveness as a Function of Retrieval Accuracy
    James Allan, Ben Carterette, and Joshua Lewis. SIGIR 2005.
  • Comparing Inverted Files and Signature Files for Searching a Large Lexicon
    Ben Carterette & Fazli Can. Information Process and Management 41(3):613-633 (2005). (abstract)
• 2003
  • Ranking Document Retrieval Systems with BLEU
    Ben Carterette, James Allan, and W. Bruce Croft. CIIR Tech Report.

Code

• mtc-eval: an implementation of the evaluation model described in Carterette (2007) and Carterette, Allan, & Sitaraman (2006).
• R package for the likelihood maximization described in Carterette & Allan (2007).
  To install, start R and type
  install.packages("preferences", repos="http://ciir.cs.umass.edu/~carteret")
  To get started, see help(maximize). More documentation coming soon...

Experience

• Research Assistant, CIIR, UMass Amherst - Sep '03 to present
  Advisor: Prof. James Allan
• Intern, Microsoft Research - Jun '07 to Aug '07
  Supervisor: Dr. Susan Dumais
• Intern, Yahoo! Research - Jun '06 to Aug '06
  Supervisor: Dr. Rosie Jones
• Intern, Yahoo! Research - Jun '05 to Aug '05
  Supervisor: Dr. Rosie Jones
• Research Assistant, Miami University - '98-'99, '02-'03
  Advisor: Prof. Fazli Can

Professional Activities

• PC Member for SIGIR '07, ECIR '08, SIGIR '08, EVIA '08.
• Reviewer for TOIS, JASIST, IR, IP&M.
• Referee for InfoScale '07, InfoScale '08.
• Referee for ADVIS '04 and '06.
• Member of ACM and ACM SIGIR.

Other

• My Erdös Number is no greater than 3 (and almost certainly not less!):
  • I wrote "Minimal Test Collections for Retrieval Evaluation" with James Allan and Ramesh Sitaraman;
  • Ramesh Sitaraman wrote "Augmented Ring Networks" with William Aiello, Sandeep Bhatt, Fan Chung, and Arnold Rosenberg;
  • Fan Chung wrote "On the Decomposition of Graphs into Complete Bipartite Subgraphs" with Paul Erdös.

---