Natural Language Processing

Fall 2021 — CMSC 473/673

Announcements
Who, What, When, and Where

Check out the syllabus for all this information, including policies on academic honesty, accomodations, and late assignments.

Meeting Times
ILSB 201
Monday & Wednesday, 1pm - 2:15pm
Instructor
Frank Ferraro
ferraro [at] umbc [dot] edu
ITE 358/remote
Monday 2:15 - 3pm
Thursday 1:00 - 1:30
by appointment
TA
Ryan Barron
ryanb4 [at] umbc [dot] edu
ITE 334/remote
Monday 4:30 - 5:30pm
Wednesday 4:30 - 5:30
by appointment
Topics
The topics covered will include
  • probability, classification, and the efficacy of simple counting methods
  • language modeling (n-gram models, smoothing heuristics, maxent/log-linear models, and distributed/vector-valued representations)
  • sequences of latent variables (e.g., hidden Markov models, some basic machine translation alignment)
  • trees and graphs, as applied to syntax and semantics
  • some discourse-related applications (coreference resolution, textual entailment), and
  • special and current topics (e.g., fairness and ethics in NLP).
  • modern, neural approaches to NLP, such as recurrent neural networks and transformers (e.g., BERT or GPT-2).
Goals
After taking this course, you will
  • be introduced to some of the core problems and solutions of NLP;
  • learn different ways that success and progress can be measured in NLP;
  • be exposed to how these problems relate to those in statistics, machine learning, and linguistics;
  • have experience implementing a number of NLP programs;
  • read and analyze research papers;
  • practice your (written) communication skills.
Schedule

The following schedule of topics is subject to change.

Legend:

Date Topic Suggested Reading Assignment Out Assignment Due
Wednesday, 9/1
  1. Intro/administrivia
Eisenstein: Ch. 1 (Optional: 1.2.2)

[473/673] Assignment 1

Wednesday, 9/8
  1. What is NLP?
  2. Probability Review
  • What is NLP?: Eisenstein: Ch. 1 (Optional 1.2.2)
  • Probability: Eisenstein: Appendix A
Monday, 9/13 None required, though this will cover approximately the intro of every chapter in either Eisenstein or SLP.
Monday, 9/20
  1. Intro to NLP Research and Research Community
Wednesday, 9/22
  1. (Lossy) Overview of Different Types of NLP Tasks
Wednesday, 9/29
  1. Machine Learning: Methodology and Evaluation
  • Eisenstein: Ch 4 (optional: 4.4.3, 4.4.4, and 4.5, unless planning to do doctoral work)
  • 3SLP: intro of Ch 4, 4.7, 4.8, and 4.10
Wednesday, 10/06
  1. Classification with Maxent Models
  • Eisenstein: Ch 2.5
  • 3SLP: Ch. 5
Wednesday, 10/20
  1. Distributed Representations
  • 3SLP: Ch. 6
  • Eisenstein: Ch. 14, with exceptions listed on Discord
Monday, 11/1
  1. (Generative) Language Modeling
  • Eisenstein: Ch. 6 (except 6.2.2-6.2.4, 6.3.1)
Wednedsay 11/10
  1. Recurrent Neural Language Modeling
  • Eisenstein: Ch. 6 (except 6.2.2-6.2.4, 6.3.1)
Monday 11/29
  1. General Language Modeling: Attention and Transformers