Date |
Topic |
Reading |
Items due |
week 1 (Mon: 08/26)
|
Introduction to Machine Learning
Lecture Slides 1
Notebook#0 (released)
|
|
|
week 1 (Wed: 08/28)
|
Google Colab and Python Review
Day01 Notes: Python Refresher
Lecture Slides 2
|
Reading: |
Notebook#0 (due on 08/28) |
Week 2 (Wed: 09/02)
|
Labor Day (university holiday; classes do not meet)
|
|
|
week 2 (Wed: 09/04)
|
Pandas Tutorial
Day02 Notes: Pandas Tutorial
Lecture Slides 3
|
Reading: |
|
week 3 (Mon: 09/09)
|
Practicing with Pandas (Give it a try first, and I'll provide a solution)
Day03 Notes: Pandas Practice
Lecture Slides 4
Notebook#1 (released on 09/08)
|
Reading: |
|
week 3 (Wed: 09/11)
|
k-Nearest Neighbor (kNN)
Day04 Notes: k-Nearest Neighbor
Lecture Slides 5
|
|
|
week 4 (Mon: 09/16)
|
kNN code
Day05 Notes: kNN code
Lecture Slides 6
Notebook#2 (released on 09/16)
|
Reading:
The relationship ML and other fields
isna()
notna()
any()
value_counts()
dropna()
fillna()
|
Notebook#1 (due on 09/16)
|
week 4 (Wed: 09/18)
|
Missing Data and Normalization
Day06 Notes: Missing Data and Normalization
Lecture Slides 7
|
Reading:
sklearn: standard scalar
|
|
week 5 (Mon: 09/23)
|
Weighted k-NN
Day07 Notes: Normalization and Weighted k-NN
Lecture Slides 8
Quiz#1 (released on Blackboard 09/23)
|
Reading:
Pandas DataFrame: sample
|
Notebook#2 (due on 09/24)
Extended deadline
|
Week 5 (Wed: 09/25)
|
Graph Plot and Evaluation Metrics Discussion
Day08 Notes: Graph Plot
Lecture Slides 9
|
Reading:
matplotlib: Pyplot
sklearn.metrics: accuracy_score
sklearn.metrics: confusion_matrix
sklearn.metrics: mean_absolute_error
sklearn.metrics: mean_squared_error
|
|
week 6 (Mon: 09/30)
|
Evaluation Metrics on kNN Regression Coding Activity
Introduction to Decision Trees
Lecture Slides 10
Day09 Notes: Evaluation Metrics and Testing
Notebook#3 (released on 09/29)
|
Reading:
|
Quiz#1 (due on 09/30)
|
week 6 (Wed: 10/02)
|
Decision trees and Entropy Calculation
Paper-based in-class Activity on Decision Trees
Lecture Slides 11
Day10 Notes: Decision Trees
|
Reading:
sklearn: train and test split
sklearn: confusion matrix
sklearn: standard scalar
|
|
week 7 (Mon: 10/07)
|
Introduction to Scikit Learn
Lecture Slides 12a
Day11 Notes: introduction to sklearn
|
Reading:
sklearn: Decision Tree classifier
sklearn: k-Nearest-Neighbor Classifier
sklearn: evaluation metrics
|
|
week 7 (Wed: 10/09)
|
Scikit Learn Practice
Lecture Slides 12b
kNN, weighted-kNN, Decision Tree using Scikit Learn
Day12 Notes: Sklearn Practice
Notebook#4 (released on 10/09)
|
Reading:
sklearn: Decision Tree classifier
sklearn: k-Nearest-Neighbor Classifier
|
Notebook#3 (extended: due on 10/09) |
week 8 (Mon: 10/14)
|
Fall Break (classes do not meet)
|
|
|
week 8 (Wed: 10/16)
|
Random Forest
Day13 Notes: Random Forest
Lecture Slides 13
|
Reading:
sklearn: Random Forest Classifier
Random Forests: Leo Breiman and Adele Cutler
Application of Random Forest: A Computer Vision research paper ICCV'2007
sklearn: Feature Selection: SelectKBest
sklearn: Principle Component Analysis (PCA)
|
|
week 9 (Mon: 10/21)
|
Day14 Notes: Dimensionality Reduction Techniques
Principle Component Analysis (PCA)
Lecture Slides 14
Project#1 (released on Blackboard 10/21)
Day14 Notes: Project#1
|
Reading:
sklearn: k-Nearest-Neighbor Classifier
sklearn: k-Nearest-Neighbor Regressor
sklearn: Decision Tree Classifier
sklearn: Decision Tree Regressor
sklearn: Random Forest Classifier
sklearn: Random Forest Regressor
|
Notebook#4 (due on 10/21)
|
week 9 (Wed: 10/23)
|
Linear Classifiers
Perceptron
Perceptron Update Algorithm
Lecture Slides 15
|
Reading:
The Perceptron: A Perceiving and Recognizing Automaton (Rosenblatt - 1957)
|
|
week 10 (Mon: 10/28)
|
Neuron Model and General Weight Learning Algorithm
Lecture Slides 16
In-class activity (linear model, perceptron)
Quiz#2 (released on Blackboard 10/28)
Day16 Notes: Perceptron Code (optional)
|
Reading:
|
|
week 10 (Wed: 10/30)
|
Virtual session (classes do not meet in-person)
Dr. Reza is traveling to China as part of a Drake University delegation;
Optimization: Gradient Descent, Stochastic Gradient Descent (SGD)
Lecture Slides 17 Day17 Notes: Loss Surface Visualization
Day17 Notes: Stochastic Gradient Descent (code)
|
Reading:
|
|
week 11 (Mon: 11/04)
|
Virtual session (classes do not meet in-person)
Dr. Reza is traveling to China as part of a Drake University delegation;
|
Reading:
|
Project #1 (due by 11/04) |
week 11 (Wed: 11/06)
|
Introduction to Neural Networks
Multilayer Perceptron (MLP)
Lecture Slides 18
|
Reading:
Multilayer Perceptron (MLP)
The Backpropgation Algorithm (Rumelhart-1985)
|
Quiz#2 (due by 11/06)
|
Week 12 (Mon: 11/11)
|
PyTorch Basics
Lecture Slides 19
Day19 Notes: PyTorch Basics in-class activity
Day19 Notes: Building a very simple MLP using PyTorch
|
Reading:
PyTorch
PyTorch: matmul()
PyTorch: transpose()
|
|
Week 12 (Wed: 11/13)
|
Multilayer Perceptron (MLP) - (Modular PyTorch Implementation)
Lecture Slides 20
Day 20 Notes: PyTorch's Useful Functions for MLP
Day 20 Notes: Building Modular MLP with PyTorch
|
Reading:
PyTorch: nn.Linear()
PyTorch: nn.Sigmoid()
PyTorch: nn.ReLU()
PyTorch: nn.Softmax()
PyTorch: nn.Flatten()
|
|
Week 13 (Mon: 11/18)
|
Building and Training MLP using PyTorch
Lecture Slides 21a
Day 21 Notes: Training MLP with PyTorch
Convolutional Neural Network (CNN)
Lecture Slides 21b: Intro to CNN
|
Reading:
PyTorch: nn.CrossEntropyLoss()
PyTorch: nn.MSELoss()
PyTorch: optim.SGD()
PyTorch: optim.ADAM()
PyTorch: optim.RMSprop()
PyTorch: torchvision.datasets
|
|
Week 13 (Wed: 11/20)
|
Convolutional Neural Network (CNN)
Lecture Slides 22 [pdf] [powerpoint]
Day 22 Notes: Training CNN for image classification
Notebook #5 (Released)
|
Reading:
Earliest CNN: LeNet (LeCunn-1988)
PyTorch: nn.Conv2d()
PyTorch: nn.Flatten()
PyTorch: nn.MaxPool2d()
PyTorch: nn.AvgPool2d()
|
|
Week 14 (Mon: 11/25)
|
Fine-tuning vs. Training
Fine-tuning CNNs
Lecture Slides 23
Day 23 Notes
|
Reading:
Popular CNN: AlexNet (Krizevsky-2012)
Popular CNN: VGGNet (Simonyan-2015)
Popular CNN: ResNet (He-2015)
|
|
Week 14 (Wed: 11/27)
|
Thanksgiving break (classes do not meet)
|
|
|
Week 15 (Mon: 12/02)
|
Fine-tuning CNNs (continued)
Project #2 (released)
Recurrent Neural Network (RNN)
Lecture Slides 24a
Transformer
Lecture Slides 24b
|
Reading:
Transformer: Attention is All You Need (2017)
The Illustrated Transformer
Harvard University NLP: The Annotated
LSTM is dead. Long Live Transformers! - Paul Dirac talk
Transformer by Hand - Anna Rahn 5 min video tutorial
|
Notebook#5 (due on 12/06)
extended deadline
|
Week 15 (Wed: 12/04)
|
Large Language Model (LLM)
LLM Application for NLP Tasks using PyTorch
Lecture Slides 25
Day 25 Notes: LLM for sentiment classification
Day 25 Notes: LLM for paraphrase detection
Day 25 Notes: LLM for question answering
Quiz#3 (released on Blackboard 12/04)
Course Evaluation
|
Reading:
GPT-1 (100M parameters) - Radford et al. 2018 (OpenAI)
BERT (300M parameters) - Devlin et al. 2018
GPT-2 (1.5B parameters) - Radford et al. 2019 (OpenAI)
Megatron-LM (8.0B parameters) - Shoeybi et al. 2019 (NVidia)
T5 (11.0B parameters) - Raffel et al. 2020 (Google)
T-NLG (17.0B parameters) - Microsoft Corporation 2020 (Microsoft)
GPT-3 (175.0B parameters) - Brown et al. 2020 (OpenAI)
ChatGPT (1.5B parameters) - OpenAI 2022 (OpenAI)
Sparks of Artificial General Intelligence: Early experiments with GPT-4 - Microsoft Research '23
Dissociating language and thought in large language models - Mahowald, Tenenbaum et al.'24
OpenAI-o1 - 2024 (OpenAI)
Is OpenAI-o1 reasoning? (ML Street Talk'24)
|
|
Week 16 (Mon: 12/09)
|
No final exam!
|
Reading:
|
Quiz#3 (due on 12/13 Friday)
Project #2 (due by 12/14 Saturday)
|