Lecture: Tuesday/Thursday 12:30 - 1:45, 1105 Siebel Center
Professor Dan Roth
Office Hours: Thursday 2-4 pm
Phone: (217) 244-7068
E-mail: danr at uiuc dot edu
The study of the computational
processes underlying comprehension and generation of natural language is an
important scientific and engineering task. Instilling machines with abilities
that allow them interact to intelligently with humans depends on increasing
levels of natural language comprehension (and generation), although shallow
levels of understanding can already successfully support challenging
applications such as intelligent information extraction, automatic translation,
summarization and others. At the same time, understanding how humans process language is a challenging and important
scientific endeavor that may also shed some light on many language engineering
Realizing that natural language understanding tasks are knowledge intensive, in the sense that they depend heavily on ``real-world knowledge'', earlier lines of research in these directions relied heavily on manual knowledge preparation -- "world" knowledge as well as linguistics knowledge. It is generally accepted today that a learning component must have a central role in supporting natural language related tasks; learning processes have multiple roles, from knowledge acquisition to supporting non-brittleness when facing previously unseen situations. A significant amount of work has been devoted in the last few years to developing statistics based and learning methods for these tasks, with considerable success.
This course will introduce some of the central
learning frameworks and techniques that have emerged in this field and found
applications in several areas in text and speech processing: from information
retrieval and extraction, through speech recognition to translation and text
comprehension. We will present the main theoretical paradigms used in the field
-- learning theoretic, probabilistic, and information theoretic -- the
relations between them, and the main algorithmic techniques developed within
The course will be reading intensive as well as project intensive. We will introduce some of the fundamental ideas and discuss those by going through relevant papers and working on relevant projects.
This will be done in the context of thinking, and attempting to address, the problem that will serve as the final project -- story comprehension.
Previous knowledge of NLP or AI is not required. Background in algorithms (CS225 or equivalent) and theory of computation(CS273 or equivalent) is required.
I will not follow a text book. Relevant papers and notes will be available from the course home page. The following texts contain some of the material that will be covered and some portions of them will be recommended reading.
This is an advanced course. I view my role as guiding you through the material and helping you in your first steps as an researcher. I expect that your participation in class, reading assignments and experimental assignments will reflect independence, mathematical rigor and critical thinking.