Data

Some of the resources below are copyrighted (marked with a '+') and require a password to access. To use these resources, you must be a member of the LDC or have purchased a license for the relevant resource. If you are a student at the University of Illinois at Urbana-Champaign and need to use these resources, you may be covered by an existing license; check with your professor and then email us to get access.
IT IS ILLEGAL TO SHARE A COPYRIGHTED LDC RESOURCE WITH PEOPLE OR ORGANIZATIONS WHO DO NOT HAVE EITHER AN LDC MEMBERSHIP OR A LICENSE TO USE THE COPYRIGHTED RESOURCE.

Corpora and Preprocessed Data for a number of NLP tasks

• +POS tagging
• +Noun Phrase Identification
• +Prepositional Phrase Attachment
• +Subject/Verb Identification
• +Verb/Object Identification
• +Chunking
• Context Sensitive Spelling Correction
• Data for Entity and Relation Recognition Experiments
• Data for Information Extraction
• Data for the named-entity coreference task
• Data for the semantic role-labeling task
• Data for NE extraction from temporally aligned corpora
• Opinion Recognition and Multi-Perspective Question Answering (MPQA) Data

Corpora for Question Answering Task

• TREC 8 questions
• TREC 9 questions
• TREC 10 questions
• TREC document library
• Question Classification
• Named Entity Recognition (with HTML tags)
• TREC 11 Answers
• TREC 11 Answers (long)

Semantic Entailment Corpora

The entailment corpora here are either in plain text, xml or column format. xml files contain the truth value of each sentence pair. Plain text files are accompanied by a .info file that contains the truth value in the same order as the sentence pairs in the associated corpus. This file explains the column format.

• PARC sentence pairs: plain text text+info column
• PASCAL RTE 1 sentence pairs (2005):
  
  • development corpus: xml column
    
  • test corpus: xml column
• PASCAL RTE 2 sentence pairs (2006):
  
  • development corpus: xml preprocessed (by RTE organizers) column
    
  • test corpus: xml column
    
• PASCAL RTE 3 sentence pairs (2007): (please note that the column format versions contain an additional final column, as compared to previous years' annotations, representing full parse information)
  
  • development corpus: xml column
    
  • test corpus: xml column
    

For a more extensive set of examples testing the kinds of phenomena modeled in the PARC dataset, take a look at the FRACAS dataset provided by Bill MacCartney of the Stanford University NLP Group.

Comma Resolution Data

The corpus and annotation guidelines developed for (V. Srikumar, R. Roichert, M. Sammons, A. Rappoport, and D. Roth, "Extraction of Entailed Semantic Relations Through Syntax-Based Comma Resolution", Proc. of the Annual Meeting of the ACL (2008)) can be downloaded here.

If you use this data, please cite the work referenced above.

Image Data

• UIUC Image Database for Car Detection

Other Corpora

• +Penn's Treebank Corpora
• +MUC-7 Corpora
• +ACE-2 Corpora [download] (LDC page)
• +ACE 2004 Multilingual Training Corpus [download]
• +ACE 2005 Multilingual Training Corpus (LDC page)
• +TDT2 English Text Only Corpus
• +Reuters 2003 corpora for NER
• +BBN Coref corpus (LDC page)
• +TimeBank 1.2 (LDC page)
• +Google N-Gram Data (LDC page)
• +OntoNotes v1.0 (LDC page)
• +ACQUAINT English News Text (LDC page)
• +TRECWeb 2002 Data (CSIRO TRECWeb page)

Other Resources

• Some incomplete entity lists
• +A polarity lexicon of factive verbs used in the research described in a paper by Nairn, Condoravdi, and Karttunen
• Network Store of Papers
• +Lin and Pantel's DIRT rules