my picture

Nitin Madnani

[Here's how to pronounce my name]

Mailing address
Nitin Madnani
666 Rosedale Rd., MS 13-R
Princeton, NJ 08541
Phone
(O) +1-609-734-5898
E-mail
(first initial+last name)@ets.org
Research Interests
Automatic Paraphrase Generation & Recognition
Statistical Machine Translation
Machine Learning
Automatic Text Summarization
Information Visualization
Computer Science Education
Blog
Colorful Green Ideas
Website
DesiLinguist

Complete Curriculum Vitae

I am currently an Associate Research Scientist with the Text, Language and Computation research group at the Educational Testing Service.

In my earlier life (which ended in May 2010), I was a Ph.D. student in the Department of Computer Science at University of Maryland, College Park. I was also a graduate research assistant in the Computational Linguistics and Information Processing Laboratory at the Institute for Advanced Computer Studies, where I worked with my advisor, Bonnie Dorr.

In general, my research has been focused on building systems—with underlying computational models of language—that process written text so as to enhance our experience with those texts. As an example, in my dissertation, I demonstrated the existence of a concrete symbiotic semantic relationship between systems that translate text and those that paraphrase text; I then exploited this relationship to build better translation and paraphrasing systems. Besides machine translation and paraphrase generation, I have also worked on automatic summarization and information retrieval systems.

I am also particularly interested in information and data visualization and over the last few years, I have worked on some interesting projects such as visualizing poetry for humanity scholars and interactive scoring for statistical machine translation systems.

I have also had a great time teaching computer science during my days as a graduate student and I am hoping to be involved with more of that in the future.

Education

Experience

Publications

2012
Generating Targeted Paraphrases for Improved Translation.
ACM Transactions on Intelligent Systems and Technology (Special Issue on Paraphrasing).
Nitin Madnani and Bonnie Dorr.
Discriminative Edit Models for Paraphrase Scoring.
Proceedings of the 6th International Workshop on Semantic Evaluation (SemEval).
Michael Heilman and Nitin Madnani.
[pdf] [bib]
Exploring Grammatical Error Correction with Not-So-Crummy Machine Translation.
Proceedings of the 7th Workshop on Innovative Use of Natural Language Processing for Building Educational Applications (BEA).
Nitin Madnani, Joel Tetreault and Martin Chodorow.
[pdf] [bib]
Re-examining Machine Translation Metrics for Paraphrase Identification.
Proceedings of the 12th Annual NAACL-HLT Conference.
Nitin Madnani, Joel Tetreault and Martin Chodorow.
[pdf] [bib] [zip]
Identifying High Level Organizational Elements in Argumentative Discourse.
Proceedings of the 12th Annual NAACL-HLT Conference.
Nitin Madnani, Michael Heilman, Joel Tetreault and Martin Chodorow.
[pdf] [bib]
Building Subjectivity Lexicon(s) from Scratch for Essay Data.
Proceedings of the 13th International Conference on Intelligent Text Processing and Computational Linguistics (CICLing).
Beata Beigman Klebanov, Jill Burstein, Nitin Madnani and Adam Faulkner.
[pdf] [bib]
2011
iBLEU: Interactively Scoring and Debugging Statistical Machine Translation Systems.
Proceedings of the Fifth IEEE International Conference on Semantic Computing (Demos).
Nitin Madnani.
[pdf] [bib] [website]
E-rating Machine Translation.
Proceedings of the Sixth Workshop on Statistical Machine Translation (WMT'11).
Kristen Parton, Joel Tetreault, Nitin Madnani, Martin Chodorow.
[pdf] [bib]
They Can Help: Using Crowdsourcing to Improve the Evaluation of Grammatical Error Detection Systems.
Proceedings of the 49th Annual ACL Conference (Short Papers).
Nitin Madnani, Joel Tetreault, Martin Chodorow and Alla Rozovskaya.
[Note: The code and data used for this paper are available for download. See linked tarball. ]
[pdf] [bib] [tgz]
The Web is not a PERSON, Berners-Lee is not an ORGANIZATION, and African-Americans are not LOCATIONS: An Analysis of the Performance of Named-Entity Recognition.
Proceedings of the ACL 2011 Workshop on Multiword Expressions.
Bob Krovetz, Paul Deane and Nitin Madnani.
[pdf] [bib]
Machine Translation Evaluation and Optimization.
Handbook of Natural Language Processing and Machine Translation.
Joseph Olive, John McCary, and Caitlin Christianson (editors). Yaser Al-Onaisan, Bonnie Dorr, Doug Jones, Jeremy Kahn, Seth Kulick, Alon Lavie, Gregor Leusch, Nitin Madnani, Chris Manning, Arne Mauser, Alok Parlikar, Mark Przybocki, Rich Schwartz, Matt Snover, Stephan Vogel and Clare Voss.
2010
Putting the User in the Loop: Interactive Maximal Marginal Relevance for Query-Focused Summarization.
Proceedings of the 11th Annual NAACL-HLT Conference (Short Papers) .
Jimmy Lin, Nitin Madnani and Bonnie J. Dorr.
[pdf] [bib]
Measuring Transitivity using Untrained Annotators.
Proceedings of the NAACL Workshop on Creating Speech and Language Data With Amazon’s Mechanical Turk.
Nitin Madnani, Jordan Boyd-Graber and Philip Resnik.
[pdf] [bib] [zip]
Generating Phrasal & Sentential Paraphrases: A Survey of Data-Driven Methods.
Computational Linguistics, 36(3).
Nitin Madnani and Bonnie Dorr.
[pdf] [bib]
The Circle of Meaning: From Translation to Paraphrasing and Back.
PhD Dissertation. Department of Computer Science. University of Maryland College Park.
[pdf] [bib]
The Python and The Elephant: Large Scale Natural Language Processing with NLTK and Dumbo.
Proceedings of the Eighth Annual Python Conference.
Nitin Madnani and Jimmy Lin.
[Note: The word associations generated for this talk are available for download. See linked zip file. ]
[video] [bib] [zip]
TER-Plus: Paraphrase, Semantic, and Alignment Enhancements to Translation Edit Rate.
Machine Translation, 23(2-3):117-127.
Matthew Snover, Nitin Madnani, Bonnie Dorr and Richard Schwartz.
[html] [bib]
2009
Fluency, Adequacy, or HTER? Exploring Different Human Judgments with a Tunable MT Metric.
Proceedings of the Fourth Workshop on Statistical Machine Translation (WMT'09).
Matthew Snover, Nitin Madnani, Bonnie Dorr and Richard Schwartz.
[pdf] [bib]
Querying and Serving N-gram Language Models with Python.
The Python Papers. Volume 4, Issue 2.
Nitin Madnani.
[pdf] [bib]
Source Code: Querying and Serving N-gram Language Models with Python.
The Python Papers Source Codes. Volume 1.
Nitin Madnani.
[pdf] [bib]
2008
Applying Automatically Generated Semantic Knowledge: A Case Study in Machine Translation.
Proceedings of the Symposium on Semantic Knowledge Discovery, Organization and Use .
Nitin Madnani, Philip Resnik, Bonnie Dorr and Richard Schwartz.
[pdf] [bib] [poster]
Are Multiple Reference Translations Necessary? Investigating the Value of Paraphrased Reference Translations in Parameter Optimization.
Proceedings of the Eighth Conference of the Association for Machine Translation in the Americas (AMTA).
Nitin Madnani, Philip Resnik, Bonnie Dorr and Richard Schwartz.
[pdf] [bib]
Combining Open-Source with Research to Re-engineer a Hands-on Introductory NLP Course.
Proceedings of the Third ACL Workshop on Issues in Teaching Computational Linguistics (TeachCL-08).
Nitin Madnani and Bonnie Dorr.
[pdf] [bib]
Multiple Alternative Sentence Compressions and Word-Pair Antonymy for Automatic Text Summarization and Recognizing Textual Entailment
Proceedings of the Text Analysis Conference (TAC) .
Saif Mohammad, Bonnie J. Dorr, Melissa Egan, Nitin Madnani, David Zajic, and Jimmy Lin.
[pdf] [bib]
TERp: A System Description.
Proceedings of the First NIST Metrics for Machine Translation Challenge (MetricsMATR).
Matt Snover, Nitin Madnani, Bonnie Dorr and Richard Schwartz.
[pdf] [bib]
2007
Using Paraphrases for Parameter Tuning in Statistical Machine Translation.
Proceedings of the Second Workshop on Statistical Machine Translation (WMT'07).
Nitin Madnani, Necip Fazil Ayan, Philip Resnik, Bonnie Dorr.
[pdf] [bib]
Measuring Variability in Sentence Ordering for News Summarization.
Proceedings of the 11th European Workshop on Natural Language Generation (ENLG).
Nitin Madnani, Rebecca Passonneau, John Conroy, Necip Fazil Ayan, Bonnie Dorr, Judith Klavans, Dianne O'Leary and Judith Schlesinger.
[pdf] [bib]
Getting Started on Natural Language Processing with Python.
ACM Crossroads, Volume 13, Issue 4.
Nitin Madnani.
[Note: The PDF version has been completely revised since the official ACM version to keep up to date with the changes made to the software used in the article. ]
[pdf] [html] [bib]
TREC 2007 ciQA Task: University of Maryland.
Proceedings of the Sixteenth Text Retrieval Conference (TREC).
Nitin Madnani, Jimmy Lin, and Bonnie Dorr.
[pdf] [bib]
Multiple Alternative Sentence Compressions for Automatic Text Summarization.
Proceedings of the Document Understanding Conference (DUC) at HLT/NAACL.
Nitin Madnani, David Zajic, Bonnie Dorr, Necip Fazil Ayan and Jimmy Lin.
[pdf] [bib]
2003-2005
The Hiero Machine Translation System: Extensions, Evaluation, and Analysis.
Proceedings of HLT/EMNLP.
David Chiang, Adam Lopez, Nitin Madnani, Christof Monz, Philip Resnik and Michael Subotin.
[pdf] [bib]
Rapid Porting of DUSTer to Hindi.
ACM Transactions on Asian Language Information Processing, Volume 2, Issue 2.
Bonnie J. Dorr, Necip Fazil Ayan, Nizar Habash, Nitin Madnani, and Rebecca Hwa.
[pdf] [bib]

Software

Python & Perl wrappers for SRILMs: Wrappers that will allow you to read and query an SRI language model directly in your Python and Perl code. [tgz]
(Note: I also have working Python and Perl wrappers for the IRSTLM toolkit but I am too lazy to put them up here. Drop me a note if you are interested and I will send them to you.)
Interactive BLEU Scoring Tool: A visual and interactive environment for scoring output of automatic machine translation systems. It's written to run entirely in the browser and utilizes the latest web technologies to allow interactive qualitative examination of MT output. [link]
clusterinfo: A Python script that displays current usage of a PBS-based cluster in a more condensed and easier-to-read format. [tgz]
LM Server: A Python-based XML-RPC server for an SRILM language model. Allows multiple clients to query the same language model that's loaded in memory in server mode. [tgz]
UMIACS Word Alignment Interface: A Java-based tool for creating and viewing word alignments between language pairs. It has been widely used across the community to create aligments for many language pairs including Welsh-English, Swahili-English, Czech-English and Chinese-English. [link]
Patch for compiling the Charniak Parser on 64-bit Linux Systems: I used to have a patch that would allow the original Charniak Parser to compile on 64-bit Linux systems. However, the better solution is to get the latest version of the codebase from here. You can always download the patch if you need it. [tgz]
[Important: Refer to the included README for additional instructions before applying the patch. This patch was developed against the Charniak Parser downloaded from Brown University FTP servers (filename: reranking-parserAug06.tar.gz, md5sum: da9d473b399a03dda2bc1cc248b5ce9d)]
TER-plus (TERp): TERp is an automatic evaluation metric for Machine Translation, which takes as input a set of reference translations, and a set of machine translation output for that same data. TERp utilizes automatically generated paraphrases, stemming, synonyms, relaxed shifting constraints and other improvements. [link]
(Note: Work done in collaboration with Matt Snover, who is the main developer of TERp.)

Working Papers

A Pythonic Exploration of Vector-Space Methods for Semantic Similarity (Magazine Article).
In Preparation.

Unpublished Manuscripts

Active Learning for Mention Detection: A Comparison of Sentence Selection Strategies.
(Available as arXiv:0911.1965v1 from the arXiv Computing Research Repository (CoRR))
Emily: A Tool for Visual Poetry Analysis. [pdf]

Selected Invited Talks

Using Statistical Machine Translation to Improve Statistical Machine Translation. 2011. Yahoo! Data Sciences Seminar at Rutgers University. [pdf]
The Circle of Meaning: From Translation to Paraphrasing and Back. 2010. NLP Seminar at CUNY Graduate Center, New York.
Using Paraphrases for Parameter Tuning in Statistical Machine Translation. 2007. Annual Technical Presentation at the Technical Meeting for Global Autonomous Language Exploitation.

Posters

Exploring Emily Dickinson Letters. June 2005. HCIL 22nd Annual Symposium and Open House, University of Maryland. Catherine Plaisant, Nitin Madnani, Matt Kirschenbaum, Martha Nell Smith, Tanya Clement, Greg Lord. [pdf]
Portable Divergence Unraveling - The Case of Hindi. 2004. Research Review Day, University of Maryland. Nitin Madnani, Necip Fazil Ayan, Bonnie Dorr, Nizar Habash, Christof Monz. [pdf]

Miscellanea

Expectation Maximization. 2004.
Advanced NLP Seminar, University of Maryland. [slides]
Decoding in Statistical Machine Translation. 2006.
StatMT Reading Group, University of Maryland. [slides]
A timeline of inter-annotator agreement measures in Computational Linguistics based on Inter-Coder Agreement for Computational Linguistics by Ron Artstein and Massimo Poesio.
Linguistics seminar on Corpus-based Social Science, University of Maryland. [pdf]

Service