Machine Learning and Natural Language

Spring 2009

Course Plan and Lecture Notes

Note: lecture notes for future dates are preliminary and will be changed

    0. Tutorial on Machine Learning Tools in NLP (May be given at a later time)

    I.    Introduction

  1. Introduction to the Class [PPT] [PDF] (01/21)
  2. Reading:

    Additional Recommended Reading:

  3. Introduction to Linguistics and its role in Natural Language Processing [PPT] [PDF] (1/25)
    Additional Notes: Martha Palmer's slides (2004)
  4. Reading:

    Additional Recommended Reading:

  5. Natural Language and Statistics (01/28)
  6. Reading:

    Additional Recommended Reading:

    II.   Mathematical Preliminaries

  7. Mathematical Preliminaries and Computational Paradigms (1/30; 2/4)
  8. Reading:

    Additional Recommended Reading:


    III.    Statistics: Representation-less Approaches

  9. Statistical Estimation: Ngrams and Backoff Models (02/06)
  10. Reading:

    Additional Recommended Reading:

    IV.    Learning Classifiers

  11. Introduction to Classification(lecture 02/11, 02/13, 02/18)
  12. Reading:

    Additional Recommended Reading:

    Student presentations:


  13. Discriminatory Approaches(02/20, 02/25) Additional Notes: On-Line; Additional Notes: SVM
  14. Reading:

    Additional Recommended Reading:

    Student Presentations


  15. Probabilistic Classifiers: NB, MaxEnt (Logistic Regression), Theoretical and Empirical Comparison of Logistic Regression and NB [PDF] [PPT] (02/27, 03/04)
  16. Reading:

    Additional Recommended Reading:

    Student Presentations:



  17. Preparation for Term Project: Dependency Parsing and Semantic Role Labeling
  18. (03/05)

    Reading:

  19. Weak Supervision in NLP: Weakly Supervised Tranliteration (guest lecture: Alex Klementiev) (03/11)
  20. Active Learning (guest lecture: Kevin Small) (03/13)
  21. Student Presentation:

  22. No lectures on Mar, 17 and Mar, 20: Work on the 1st phase of the term project
  23. V.    MultiClass and Structured Prediction

  24. MultiClass and Structured Prediction: Sequence Labeling Problems(lecture Apr, 1-3)
  25. Reading:

    Additional Recommended Reading:


  26. Parsing and Other Problems with Complex Structured Output, Reranking
  27. (Apr, 8)

    Reading:

    Additional Recommended Reading:

    Student Presentations:


  28. Local vs. Joint Approaches to Learning, Inference with Constraints(April, 10-15)
  29. Reading:

    Additional Recommended Reading:

    Student presentation:


  30. Features and Kernels (Additional Notes: Tree Kernels (by Mike Collins)) (April, 17)
  31. Reading:

    Additional Recommended Reading:

    Student presenations:

  32. Semi-Supervised Methods: Generative Models, Self-Training, Co-training and other techniques EM (Additional Notes: EM for HMMs (Notes by Mike Collins)) (April, 21-24)
  33. Reading:

    Additional Recommended Reading:


    V.    Clustering

  34. Statistical Similarity and Clustering (additional material, no lecture)
  35. Reading:

    Additional Recommended Reading:

    Student Presentations


    V.    Latent Variable Models -- Inducing Features and Inducing Topics


  36. Topic Models, Approximate Learning and Inference
  37. (April, 24-29)

    Reading:

    Additional Reading:


  38. Latent Variable Models in Natural Language Processing
  39. (May, 1)

    Reading:

    Additional Reading:

    Student Presentations:

  40. Domain Adaptation (May, 6) - guest lecture by Ming-Wei Chang
  41. Reading:

    Additional Recommended Reading:

    Student Presentations:

  42. Advanced Topics:
  43. Reading: