Introduction to Machine Learning

Spring 2022 — CMSC 678

Announcements
Who, What, When, and Where

Check out the syllabus for all this information, including policies on academic honesty, accomodations, and late assignments.

Meeting Times
ILSB 116A
Monday & Wednesday, 2:30pm - 3:45pm
Instructor
Frank Ferraro
ferraro [at] umbc [dot] edu
ITE 358 / remote
Monday 12:45 - 1:30
by appointment
TA
Abhiram Janagama
PI25537 [at] umbc [dot] edu
TBD
Topics
The topics covered will include, but are not limited to:
  • perceptrons
  • regression (linear, logistic, and non-linear)
  • spectral, clustering, and dimensionality reduction techniques
  • support vector machines and kernel methods
  • neural networks, including deep learning, recurrent neural networks, and convolutional neural networks
  • Bayesian networks and probabilistic graphical models
  • clustering
  • evaluation methodologies and experiment design.
Goals
After taking this course, you will
  • be introduced to some of the core problems and solutions of ML;
  • learn different ways that success and progress can be measured in ML;
  • be exposed to how these problems relate to those in statistics, artificial intelligence, and specialized areas of ML (such as natural language processing and computer vision);
  • have experience implementing a number of ML programs;
  • read and analyze research papers;
  • practice your (written) communication skills.
Schedule

The following schedule of topics is subject to change.

Legend:

The following schedule is approximate, and subject to change. Slides, prior to the day of the class, may not be fully updated and also subject to change.
Date Topic Main Reading: Read All Advanced Reading: Optionally Read Some Assignment Out Assignment Due
Monday, 1/31
  1. Introduction / Administrivia
ESL Ch 1 ESL Ch 2 A1: Math & Programming Review
Wednesday, 2/2
  1. What is learning?
  • CIML Ch 2
  • UML Ch 2
  • UML Ch 14
  • ITILA Ch 36
Monday, 2/7
Wednesday, 2/9
  1. Probability, loss functions, and decision theory
  • if in need of a probability refresh: ITILA Ch 2
  • UML Ch 2
  • UML Ch 14
  • ITILA Ch 36
due: A1 (2/11)
Monday, 2/14
  1. Linear regression, classification, and perceptrons
  • CIML Ch 7 (linear models)
  • CIML Ch 4 (perceptrons)
  • ESL Ch 3
  • UML Ch 9.2
Wednesday, 2/16
Monday, 2/21 A2
Wednesday, 2/22
  1. Beyond binary classification
  2. Experimental Setup, and Evaluation
  • CIML Ch 9.5-9.7
  • ESL Ch 4.4
  • UML Ch 9.3
  • ITILA Ch 39, 41.1-41.3
Monday, 2/28
Wednesday, 3/2
  1. Logistic regression/maxent models
Logistic Regression:
  • CIML Ch 9.5-9.7
  • ESL Ch 4.4
  • UML Ch 9.3
  • ITILA Ch 39, 41.1-41.3
Monday, 3/7
  1. Neural networks, backpropagation, & autodifferentiation
Neural nets:
  • CIML Ch 10
  • Goodfellow et al. (2016), Ch 6 (Deep Feedforward Networks)

Goodfellow et al. (2016), Ch 11 (Practical Methodology)
Wednesday, 3/9
Monday, 3/14
  1. Recurrent & convolutional neural networks
Goodfellow et al. (2016), Ch 11 (Practical Methodology)
  • Goodfellow et al. (2016), Ch 9 (CNNs)
  • Goodfellow et al. (2016), Ch 10 (RNNs)
Wednesday, 3/16
Monday, 3/28
Wednesday, 3/30 Exam 1
Monday, 4/4
  1. Overview of (Deep) Reinforcement Learning
Poole and Mackwork: Ch 12.1 (intro to RL), Ch 12.4 (Q-learning), Ch 12.5 (exploration vs. exploitation), Ch 12.9 (RL with features: SARSA, leading to deep Q-learning), Exam 1 (due 4/3 at 11:59 PM)
Wednesday, 4/6; Monday 4/11
  1. Dimensionality Reduction: Linear Discriminant Analysis & Principal Component Analysis
CIML Ch 15.2
  • ESL Ch 4.3
  • ESL Ch 14.5-14.10
  • UML Ch 23, 24.3
Wednesday 4/13
  1. Clustering
CIML Ch 15.1
  • ESL Ch 13, 14.3
  • UML Ch 22
  • ITILA Ch 20
Monday, 4/18
  1. Kernel methods & Support vector machines
CIML Ch 11 (Kernel + SVM), 7.7 (SVM)
  • ESL Ch 6 (Kernel)
  • UML Ch 16 (Kernel)
  • ESL Ch 12 (SVM)
  • UML Ch 15 (SVM)
Monday, 4/25
  1. Expectation Maximization & Probabilistic Modeling
CIML Ch 16
  • ESL Ch 8.5
  • UML Ch 24.0-24.1 (Maximum likelihood)
  • UML Ch 24.4 (EM)
  • ITILA Ch 20
Wednesday, 4/27
Monday 5/2 HMM Example for EM (Mostly board work), and start of PGM slides (see next class)
Wednesday 5/4--Monday 5/16
  1. Graphical Models
  • ITILA Ch 16 (Message Passing, e.g., for forward-backward)
  • CIML Ch 9
    • ITILA Ch 25 and 26 (~16 pages)
    • ESL Ch 17 (excluding 17.3.2, 17.4.2-end; ~17 pages)
    • ESL Ch 17 (excluding 17.3.2, 17.4.2-end; ~17 pages)
    Monday 5/16 Continuation of PGMs, and
    1. Summary/overview of the course