Manmatha's Home Page

I am interested in the areas of Computer Vision, Information Retrieval and also at their intersection in Image and Video Retrieval. I am also interested in Document Analysis and Recognition including the recognition and retrieval of printed material and the recognition and retrieval of handwritten manuscripts.

I am currently a Principal Scientist at Amazon and an Adjunct Professor in the Department of Computer Science.

Till 2016, I used to be a Research Associate Professor in the Department of Computer Science and lead the Multi-media Indexing and Retrieval (MIR) group at the Center for Intelligent Information Retrieval (CIIR). The group's aim was to index non-textual sources of information by either converting them to ASCII text and using a search engine or by directly indexing the information's content.

My previous work while at UMass focused on:

1. Automatically Annotating and Retrieving Images. Along with Victor Lavrenko and my former students Jiwoon Jeon and Shaolei Feng I investigated a number of models in this area. These included both discrete and continuous relevance models (CMRM, CRM, MBRM and NCRM), a maximum entropy model with various students including Venkatesh Narasimha Murthy. Here is the first paper which started it all
   
2. Indexing and Retrieving Handwritten Manuscripts. We focused in particular on George Washington's manuscripts. I came up with the idea of word spotting for handwritten documents. This work was primarily done with Toni Rath my former student and Victor Lavrenko. We used an approach based on relevance models which allowed us to use ASCII queries. We also used approach called word spotting (using word image matching) - the idea being to create automatic indices. Jamie Rothfeder, Nitin Srimal and I also investigated scale space techniques for segmenting handwritten manuscripts. Along with my former student Shaolei Feng and Prof. Nicholas Howe at Smith College I also investigated some recognition models for such manuscripts. Go to the publication list below to check out papes. to check out papers.

   We created a demonstration handwriting retrieval system based on relevance models for 1000 pages (8 GB of data) of George Washington's manuscripts based on text queries. This is the first automatic (does not use manual annotations) retrieval system for historical manuscripts. Unfortunately, this link is down.
   

3. Alignment Techniques for Printed and Handwritten Documents. I have investigated a number of techniques for aligning handwritten document images to transcripts to automatically generate groundtruth. This includes work with Micah Kornfield and James Allan using dynamic time warping and with Jamie Rothfeder and Toni Rath using HMM's. Along with Shaolei Feng, I investigated an automatic technique to align OCR output for printed books and their electronic versions on Gutenberg using HMM's
4. Searching Printed Indian Language Documents. Along with Prof. C. V. Jawahar at IIIT Hyderabad and Anand Kumar I investigated techniques to search printed documents in Indian languages for which OCR systems are not readily available. This work uses locality sensitive hashing for fast search in a book.

Other previous work includes:

1. Meta Search (or combining the outputs of multiple search engines). This work is based on modeling the score distributions of relevant documents as Gaussians and those of non-relevant documents as exponentials. A mixture model can be solved using Expectation-Maximization to recover the parameters of these distributions when relevance is unknown.
2. I have also worked on image matching under deformations (affine, similarity), image retrieval using color and appearance and text detection in images and on the scale space segmentation of handwritten manuscripts.

More on my research.

Here is a list of papers that I have been involved with or associated with.

This is my picture.

I was a co-founder and technical advisor to SnapTell a mobile image search company which was acquired by Amazon. Some of its technology can still be seen in the Amazon mobile app.

I used to write stories. Here are two samples The Shadow and Marshall Teddy if you have time to kill.

R. Manmatha
manmatha at cs.umass.edu

Quote for the day.
From Ignazio Silone's Fontemara
At the head of everything is God, the Lord of Heaven.
Everyone knows that.
Then comes Prince Torlonia, lord of the earth.
Then come Prince Torlonia's guards.
Then come Prince Torlonia's guards' dogs.
Then, nothing at all.
Then, nothing at all.
Then, nothing at all.
Then come the peasants. And that's all.

Click here to go to the CIIR multi-media indexing and retrieval homepage - another old page.