Data

Some of the resources below are copyrighted (marked with a '+') and require a password to access. To use these resources, you must be a member of the LDC or have purchased a license for the relevant resource. If you are a student at the University of Illinois at Urbana-Champaign and need to use these resources, you may be covered by an existing license; check with your professor and then email us to get access.
IT IS ILLEGAL TO SHARE A COPYRIGHTED LDC RESOURCE WITH PEOPLE OR ORGANIZATIONS WHO DO NOT HAVE EITHER AN LDC MEMBERSHIP OR A LICENSE TO USE THE COPYRIGHTED RESOURCE.

For large corpora, the links below take you to a directory structure; to download the data, you can use wget.

Corpora and Preprocessed Data for a number of NLP tasks

• +POS tagging
• +Gazetteers for NER
• +Noun Phrase Identification
• +Prepositional Phrase Attachment
• +Subject/Verb Identification
• +Verb/Object Identification
• +Chunking
• Context Sensitive Spelling Correction
• Data for Entity and Relation Recognition Experiments
• Data for Information Extraction
• Data for the named-entity coreference task
• Data for the semantic role-labeling task
• Data for NE extraction from temporally aligned corpora
• Opinion Recognition and Multi-Perspective Question Answering (MPQA) Data
• +Lin and Pantel's Clustering By Committe data
• +Chinese Learner English Corpus (by kind permission of Professor Yang Huizhong): Gui, Shicun and Huizhong Yang (eds.). 2003. Zhongguo Xuexizhe Yingyu Yuliaohu. (Chinese Learner English Corpus). Shanghai Waiyu Jiaoyu Chubanshe. (In Chinese). Here is a page that talks about the CLEC corpus.

Corpora for Question Answering Task

• TREC 8 questions
• TREC 9 questions
• TREC 10 questions
• TREC document library
• Question Classification
• Named Entity Recognition (with HTML tags)
• TREC 11 Answers
• TREC 11 Answers (long)

Semantic Entailment Corpora

The entailment corpora from the three PASCAL Recognizing Textual Entailment challenges are provided here in a column format that encodes a range of annotation of the original text. The original corpora can be accessed from the NIST Text Analysis Conference RTE track web site. corpus. This file explains the column format. The PARC sentence pairs were provided separately by Xerox PARC.

• PARC sentence pairs: plain text text+info column
• RTE_1 dev
  RTE_1 test
  
• RTE_2 dev
  RTE_2 test
  
• RTE_3 dev
  RTE_3 test
  

FRACAS Entailment(-like) Data

For a more extensive set of examples testing the kinds of phenomena modeled in the PARC dataset, take a look at the FRACAS dataset provided by Bill MacCartney of the Stanford University NLP Group.

REMEDIA Story Comprehension Corpus

• Text files
• SRL-annotated files

Comma Resolution Data

The corpus and annotation guidelines developed for (V. Srikumar, R. Roichert, M. Sammons, A. Rappoport, and D. Roth, "Extraction of Entailed Semantic Relations Through Syntax-Based Comma Resolution", Proc. of the Annual Meeting of the ACL (2008)) can be downloaded for research use via the link below.

comma resolution data.

If you use this data, please cite the work referenced above.

Image Data

UIUC Image Database for Car Detection.

This data was used in the research described in the paper, "Learning to Detect Objects in Images via a Sparse, Part-Based Representation". The software used in this research can also be downloaded here. If you use this data or the code provided, please cite the above work.

Emotion corpus:

http://lrc.cornell.edu/swedish/dataset/affectdata/index.html

Clean wikipedia data:

http://nlp.cs.nyu.edu/wikipedia-data/

Shared Corpora Collection

If you are UIUC faculty or working on a UIUC-supported project, you can access a number of copyright-protected corpora that are used by UIUC researchers. You can obtain the username and password by emailing mssammon curly-a illinois period edu.

Other Corpora

Note that these will soon be merged into the shared corpora directory linked to above.

• +British National Corpus documents (xml) | documentation
• +Penn's Treebank Corpora
• +MUC-7 Corpora
• +ACE-2 Corpora [download] (LDC page)
• +ACE 2004 Multilingual Training Corpus [download]
• +ACE 2005 Multilingual Training Corpus (LDC page)
• +TDT2 English Text Only Corpus
• +Reuters 2003 corpora for NER
• +BBN Coref corpus (LDC page)
• +Joseph Turian's embedding of words in 50 dimensional space
• +TimeBank 1.2 (LDC page)
• +Google N-Gram Data (LDC page)
• +OntoNotes v1.0 (LDC page)
• +ACQUAINT English News Text (LDC page)
• +TRECWeb 2002 Data (CSIRO TRECWeb page)
• +Three verb class models. These models allow to stem, lemmatize verbs, and to abstract them into verb groups. These models were used in SRL and other applications.

Other Resources

• Some incomplete entity lists
• +A polarity lexicon of factive verbs used in the research described in a paper by Nairn, Condoravdi, and Karttunen
• Network Store of Papers
• +Lin and Pantel's DIRT rules

Dumps of code and data which was used in experiments

• Code and data for the semi-supervised learning with constraints project
• Code and data for the Dataless classification project