Natural Language Processing

Fall 2020 — CMSC 473/673

11/27 [473/673] Assignment 5 is available. It's due 12/08, by 11:59 PM.
11/09 [473/673] The take-home midterm is out. See Piazza note 142. It's due 11/13, by 11:59 PM; late-days are not allowed.
10/19 [473/673] Assignment 4 is available. It's due 10/30, by 11:59 PM.
10/08 [473/673] There is a classwide extension for A3. It is due 10/16 by 11:59 PM.
10/01 [473/673] The project prompt is available. There are multiple milestones and due dates.
9/30 [473/673] Assignment 3 is available. It's due 10/12, by 11:59 PM.
9/21 [673] The Graduate Assessment prompt is available. There are multiple milestones and due dates.
9/14 [473/673] Assignment 2 is available. It's due 9/28, by 11:59 PM.
8/31 [473/673] Assignment 1 is available. It's due 9/9, by 11:59 PM.
8/27 [473/673] Assignments and coursework can be submitted through the submission site, available at https://www.csee.umbc.edu/courses/undergraduate/473/f20/submit. You must be logged in with your UMBC ID.
8/27 [473/673] The syllabus is available.
8/27 [473/673] The Piazza discussion and announcement board is available.

Check out the syllabus for all this information, including policies on academic honesty, accomodations, and late assignments.

Meeting Times: Synchronous remote, recorded lectures (check Piazza for the links)
Monday & Wednesday, 1pm - 2:15pm
Instructor: Frank Ferraro
ferraro [at] umbc [dot] edu
ITE 358/remote
Monday 2:15 - 3pm
Thursday 11:00 - 11:30
by appointment
TA: Charu Sharma
charus2 [at] umbc [dot] edu
Office hours TBD
by appointment

Topics

The topics covered will include

probability, classification, and the efficacy of simple counting methods
language modeling (n-gram models, smoothing heuristics, maxent/log-linear models, and distributed/vector-valued representations)
sequences of latent variables (e.g., hidden Markov models, some basic machine translation alignment)
trees and graphs, as applied to syntax and semantics
some discourse-related applications (coreference resolution, textual entailment), and
special and current topics (e.g., fairness and ethics in NLP).

Goals

After taking this course, you will

be introduced to some of the core problems and solutions of NLP;
learn different ways that success and progress can be measured in NLP;
be exposed to how these problems relate to those in statistics, machine learning, and linguistics;
have experience implementing a number of NLP programs;
read and analyze research papers;
practice your (written) communication skills.

The following schedule of topics is subject to change.

Legend:

2SLP: 2nd Edition of Jurafsky and Martin's Speech and Language Processing.
3SLP: draft 3rd Edition of Jurafsky and Martin's Speech and Language Processing. The authors have made the chapters, as they're rewritten, available: https://web.stanford.edu/~jurafsky/slp3/.

Date	Topic	Suggested Reading	Assignment Out	Assignment Due
Monday, 8/31	Intro/administrivia What is NLP?	2SLP: Ch. 1: [473] up through Ch 1.4 [673] entire chapter	[473/673] Assignment 1	—
Wednesday, 9/2	Probability Review	[473/673] Hal Daume III's "Math for Machine Learning" primer	—	—
Wednesday, 9/9	NLP Task Overview: part-of-speech, dependency parsing, textual entailment, and featurization	Cambria and White (2014) [473] Sections 1, 3, 4, 7, 8 [673] Entire paper	—	Assignment 1
Monday, 9/14			[473/673] Assignment 2	—
Wednesday, 9/16	Intro to ML: the Noisy Channel Model, Classification, & Evaluation	[473/673] 3SLP: Ch 4.0, 4.1, 4.7, 4.8	—	—
Monday, 9/21 [Whiteboard]		[473/673] 3SLP: Ch 4.0, 4.1, 4.7, 4.8	[673] Graduate Assessment	—
Wednesday, 9/23	catch-up	—	—	—
Monday, 9/28 [Whiteboard]	Count-based Language Modeling	[473/673] 3SLP Ch 3 (2SLP Ch 4)	—	Assignment 2
Wednesday, 9/30 [Whiteboard]	Count-based Language Modeling	[473/673] 3SLP Ch 3 (2SLP Ch 4)	[473/673] Assignment 3	—
Monday, 10/05 [Whiteboard]	Maximum-entropy Models (Log-linear models) and Neural Language Models	3SLP: Ch 5 Ferraro and Eisner (2013)	—	[673] Grad Assessment Milestone 1
Wednesday, 10/07 [Whiteboard]			—	—
Monday, 10/12 [Whiteboard]			—	—
Wednesday, 10/14 [Whiteboard]			—	—
Monday, 10/19 [Whiteboard]	Distributed Representations	3SLP: Ch 6	[473/673] Assignment 4	—
Wednesday, 10/21		3SLP: Ch 6	—	—
Monday, 10/26 [Whiteboard]		[473/673] Mikolov et al. (NeurIPS, 2013) [673] Levy and Goldberg (NeurIPS, 2014)	—	—
Wednesday, 10/28 [Whiteboard]	Recurrent Neural Models: Language Models, and Sequence Prediction and Generation	3SLP: Ch 9	—	—
Monday, 11/02 [Whiteboard]			—	—
Wednesday 11/04 [Whiteboard]			—	—
Monday, 11/09 [Whiteboard]	(Even More) Language Modeling: Multi-Task Learning, Attention, and Transformer-based Language Models	Wolf et al. (2020): 🤗 Transformers: State-of-the-Art Natural Language Processing	Take-home midterm out: due 11/13	—
Wednesday, 11/11 [Whiteboard] (and starting HMMs; see 11/16)			—	—
Monday, 11/16 [Whiteboard]	Hidden Markov Models	3SLP Ch 8 (Part-of-Speech Tagging) 3SLP Appendix A (Hidden Markov Models)	—	—
Wednesday, 11/18 [Whiteboard]			—	—
Monday, 11/23 [Whiteboard]			—	—
Wednesday, 11/25 [Whiteboard]	Probabilistic Context Free Grammars, and CKY-based Algorithms	3SLP Ch 12 (Constituency Grammars) 3SLP Ch 13 (Constituency Parsing: CKY)	—	—
Monday, 11/30 [Whiteboard]			—	—
Wednesday, 12/02 [Whiteboard]	Syntax to Semantics: Dependency Grammars/Parsing, Open IE, and Semantic Represenations/SRL Course Recap	3SLP Ch 15 (Dependency Parsing) WordNet: 3SLP Ch 19.3 (WordNet) Semantic Role Labeling: 3SLP Ch 20 (Semantic Role Labeling)	—	—
Monday, 12/07			—	—