CMSC 691 --- Graphical and Statistical Models of Learning

4/22 Assignment 2 is available. It's due 5/1 by 11:59 PM.
3/6 Assignment 1 is available. It's due 3/27 by 11:59 PM.
1/27 The weekly journal entry handout is available. It's due (roughly) every Monday by 11:59 AM (see handout for exceptions).
1/25 Assignments, the writing log, and projects can be submitted through the submission site, available at https://www.csee.umbc.edu/~ferraro/teaching/691-s20/submit. You must be logged in with your UMBC ID.
1/25 The syllabus is available.
1/25 See the Piazza discussion board for class announcements and Q&A.

Check out the syllabus for all this information, including policies on academic honesty, accomodations, and late assignments.

Meeting Times: ITE 229
Monday & Wednesday, 2:30pm - 3:45pm
Instructor: Frank Ferraro
ferraro [at] umbc [dot] edu
ITE 358
Monday 3:45 - 4:30
Tuesday 11:00 - 11:30
by appointment

Topics

The topics covered will include forming probabilistic and neural machine learning models and appropriate inference techniques. The covered techniques will include maximum likelihood estimation, advanced approaches to empirical risk minimization, expectation maximization, belief propagation for structured inference, and approximate probabilistic inference techniques such as Laplace’s method, Monte Carlo sampling approaches, and variational inference.

Goals

After taking this course, you will

be introduced to advanced statistical estimation and modeling in ML;
understand how these approaches can be used to develop semi-supervised and unsupervised ML algorithms;
have experience implementing a number of ML programs;
read and analyze research papers;
practice your (written) communication skills.

The following schedule of topics is subject to change.

Legend:

ESL: Elements of Statistical Learning: https://web.stanford.edu/~hastie/ElemStatLearn/printings/ESLII_print12.pdf
ITILA: Information Theory, Inference and Learning Algorithms: http://www.inference.phy.cam.ac.uk/itprnn/book.pdf
UML: Understanding Machine Learning: From Theory to Algorithms: http://www.cs.huji.ac.il/~shais/UnderstandingMachineLearning/
CIML: A Course in Machine Learning: http://ciml.info/

Date	Topic	Main Reading: Read All	Advanced Reading: Optionally Read Some	Assignment Out	Assignment Due
Monday, 1/27	Introduction: why probabilistic & statistical ML?	ITILA Ch 2 (probability, entropy, MLE) ITILA Ch 23 (common/useful probability distributions)	—	—	—
Wednesday, 1/29	Probability distributions, entropy, and maximum likelihood estimation	CIML Ch 2	UML Ch 2 UML Ch 14 ITILA Ch 36	—	—
Monday, 2/3				—	—
Wednesday, 2/5				—	—
Monday, 2/10	No class: AAAI	—	—	—	—
Wednesday, 2/12	Guest lecture: Dr. Cynthia Matuszek	—	—	—	—
Monday, 2/3	Decision Theory and Loss Functions	CIML Ch 2	Introduction to Bayesian Decision Theory . by Angela Yu	—	—
Wednesday, 2/19	Decision Theory and Loss Functions	CIML Ch 2	Introduction to Bayesian Decision Theory . by Angela Yu	—	—
Monday, 2/24	Latent MLE/MAP Intro to EM	CIML Ch 9	A Gentle Tutorial of the EM Algorithmand its Application to ParameterEstimation for Gaussian Mixture andHidden Markov Models . by Jeff Bilmes	—	—
Wednesday, 2/26	Latent MLE/MAP Intro to EM	CIML Ch 9		—	—
Monday, 3/9	EM in HMMs	A Gentle Tutorial of the EM Algorithmand its Application to ParameterEstimation for Gaussian Mixture andHidden Markov Models . by Jeff Bilmes	—	—	—
Wednesday, 3/11				—	—
Monday, 3/23 [whiteboard]				—	—
Wednesday, 3/25	Probabilistic Graphical Models	Ch 2 ("Conditional Independence and Fatorization") of An Introduction to Probabilistic Graphical Models . by Michael Jordan	—	—	—
Monday, 3/30 [whiteboard]	Exponential Family Distributions	A Compendium of Conjugate Priors by Daniel Fink	—	—	—
Wednesday, 4/1 [whiteboard]	Variational Inference	ITILA Ch 33	—	—	—
Monday, 4/13	Approximate Inference via Sampling	ITILA Ch 29	—	—	—
Monday, 4/20 [whiteboard]	Belief Propagation	ITILA Ch 26	ITILA Ch 25 (inference in trellis-structured graphs)	—	—
Wednesday, 4/29	Proximal SVI	Khan, Babanezhad, Lin, Schmidt and Sugiyama (2016, UAI)	—	—	—
Monday, 5/4 [whiteboard]	Variational autoencoders (see whiteboard)	Autoencoding Variational Bayes by Kingma and Welling (2013 ICLR)	—	—	—

Graphical and Statistical Models of Learning

Spring 2020 — CMSC 691