Natural Language Processing

Fall 2018 — CMSC 473/673

Announcements
Who, What, When, and Where

Check out the syllabus (to be released soon) for all this information, including policies on academic honesty, accomodations, and late assignments.

Meeting Times
Sherman Hall, 015
Monday & Wednesday, 1pm - 2:15pm
Instructor
Frank Ferraro
ferraro [at] umbc [dot] edu
ITE 358
Monday 2:15 - 3pm
Tuesday 11:00 - 11:30
by appointment
TA
Caroline Kery
ckery1 [at] umbc [dot] edu
Tuesday 2:00 - 3:30
Thursday 1:00 - 2:30
by appointment ITE 349
Topics
The topics covered will include
  • probability, classification, and the efficacy of simple counting methods
  • language modeling (n-gram models, smoothing heuristics, maxent/log-linear models, and distributed/vector-valued representations)
  • sequences of latent variables (e.g., hidden Markov models, some basic machine translation alignment)
  • trees and graphs, as applied to syntax and semantics
  • some discourse-related applications (coreference resolution, textual entailment), and
  • special and current topics (e.g., fairness and ethics in NLP).
Goals
After taking this course, you will
  • be introduced to some of the core problems and solutions of NLP;
  • learn different ways that success and progress can be measured in NLP;
  • be exposed to how these problems relate to those in statistics, machine learning, and linguistics;
  • have experience implementing a number of NLP programs;
  • read and analyze research papers;
  • practice your (written) communication skills.
Schedule

The following schedule of topics is subject to change.

Legend:

Date Topic Suggested Reading Assignment Out Assignment Due
Wednesday, 8/29 Intro: what is NLP? 2SLP: Ch 1

[473/673] Assignment 1
[673] Graduate Paper

Wednesday, 9/5 Probability Concepts & Language Modeling 3SLP: Ch 2.2
3SLP: Ch 4
2SLP: Ch 4
[473/673] Course Project
Monday, 9/10 3SLP: Ch 4
[473/673] Assignment 2 [473/673] Assignment 1
Wednesday, 9/12
Monday, 9/17-Monday, 9/24 Intro to ML: the Noisy Channel Model, & Classification 3SLP: Ch 4
Wednesday, 9/26 Naïve Bayes, Maximum Entropy (Log-linear) Models, and Neural Language Models 3SLP: Ch 7
Ferraro and Eisner (2013)
Monday, 10/1 Assignment 2
Wednesday, 10/3 [473/673] Assignment 3
Monday, 10/8
Wednesday, 10/10 Distributed Representations 3SLP: Ch 6
Friday, 10/12 [473/673] Project Proposal
Monday, 10/15
Wednesday, 10/17 [673] Graduate Paper
Monday, 10/22 Intro to Latent Sequences & Expectation Maximization
Wednesday, 10/24
Friday, 10/26 [473/673] Assignment 3
Monday, 10/29 Intro to Hidden Markov Models/Midterm Review
Wednesday, 10/31 Midterm
Monday, 11/5 HMMs Continued
  • Task: Part of speech
  • Forward Algorithm
  • Viterbi Algorithm
  • Backward Algorithm
  • EM/Baum Welch
  • Semi-supervised learning)
Wednesday, 11/7 [473/673] Assignment 4
Monday, 11/12
Wednesday, 11/14 [673] Graduate Paper Reviews
Friday, 11/16 [473/673] Project Update
Monday, 11/19
Wednesday, 11/21 Overview of other Latent Models:
  • Machine Translation Alignment
  • State-based Sequence Models: MEMMs and CRFs
  • Recurrent Neural Networks
Monday, 11/26 Overview of Syntax 3SLP: Ch 10, Ch 11
[473/673] Assignment 5 [473/673] Assignment 4
Wednesday, 11/28
December, 12/3 Semantics: 3SLP: Ch 17, Ch 18
Wednesday, 12/5 [673] Final Graduate Paper
Monday, 12/10 [473/673] Assignment 5
Wednesday, 12/19 [473/673] Course Project