CCG: Projects

Themes | Projects | Funding | Past Projects

Unsupervised and Semi-Supervised Learning

Overview:

Supervised learning strategies are costly in terms of resources. However, one can often reduce costs--make use of only a small amount of labeled data along with a large pool of unlabeled examples--by exploiting regularities present in the data and, possibly, domain specific information. We investigate semi-supervised and unsupervised learning methods to minimize the need for supervision in a variety of learning protocols and multiple NLP problems.

Details:

Examples of learning protocols and problems studied include:

We propose a general mathematical and algorithmic framework for unsupervised rank aggregation which can be used to combine rankings over objects. See the Learning to Rank project.
We exploit temporal alignment between two sides of the bilingual corpus to derive a nearly unsupervised learning algorithm for trasliteration and automatic discovery of Named Entities in the resource poor language. See the Transliteration project.
We develop an unsupervised, bootstrapped learning approach to context sensitive lexical paraphrasing. See a recent paper.
We suggest a method for incorporating domain knowledge in the context of Constrained Conditional Models. Domain knowledge is encoded in the form of task specific constraints into semi-supervised learning algorithms. See a recent paper.

Relevant Publications:

J. Eisenstein, J. Clarke, D. Goldwasser, and D. Roth, Reading to Learn: Constructing Features from Semantic Abstracts. Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP-2009) (2009) pp. xx--yy
A. Klementiev, D. Roth, K. Small, and I. Titov, Unsupervised Rank Aggregation with Domain-Specific Expertise. Proc. of the International Joint Conference on Artificial Intelligence (IJCAI) (2009)
A. Klementiev and D. Roth, Named Entity Transliteration and Discovery in Multilingual Corpora. Learning Machine Translation (2008)
A. Klementiev, D. Roth, and K. Small, A Framework for Unsupervised Rank Aggregation. Proc. of the SIGIR Workshop on Learning to Rank for Information Retrieval (2008) pp. 32-39
M. Chang, L. Ratinov, and D. Roth, Constraints as Prior Knowledge. ICML Workshop on Prior Knowledge for Text and Language Processing (2008) pp. 32-39
A. Klementiev, D. Roth, and K. Small, Unsupervised Rank Aggregation with Distance-Based Models. Proc. of the International Conference on Machine Learning (ICML) (2008)
M. Chang, L. Ratinov, and D. Roth, Guiding Semi-Supervision with Constraint-Driven Learning. Proc. of the Annual Meeting of the ACL (2007) pp. 280--287
M. Connor and D. Roth, Context Sensitive Paraphrasing with a Single Unsupervised Classifier. Proc. of the European Conference on Machine Learning (ECML) (2007)
A. Klementiev, D. Roth, and K. Small, An Unsupervised Learning Algorithm for Rank Aggregation. Proc. of the European Conference on Machine Learning (ECML) (2007)
A. Klementiev and D. Roth, Weakly Supervised Named Entity Transliteration and Discovery from Multilingual Comparable Corpora. Proc. of the Annual Meeting of the ACL (2006) pp. USS,TL,ADAPT
A. Klementiev and D. Roth, Named Entity Transliteration and Discovery from Multilingual Comparable Corpora. Proc. of the Annual Meeting of the North American Association of Computational Linguistics (NAACL) (2006) pp. 82--88
S. Hanneke and D. Roth, Iterative Labeling for Semi-Supervised Learning. (2004)