CS 167: Machine Learning
Fall 2023
Instructor: Md Alimoor Reza
Assistant Professor of Computer Science
Department of Mathematics and Computer Science
Drake University

Meeting time (Section 10790): Tues (09:30am-10:45pm) and Thurs (09:30am-10:45pm)
Class room: Science Connector Building # 301
Meeting time (Section 8363): Tues (11:00am-12:15pm) and Thurs (11:00am-12:15pm)
Class room: Harvey Ingham # 0004
Office hours (Collier Scripps#323): Mon (3:00pm-5:00pm) + Tues & Wed (3:30pm-5:00pm) or by appointment


Schedule
A tentative schedule below (subject to change as we progress).


Date Topic Reading Items due
week 1 (Tue: 08/29)

  Introduction to Machine Learning
  Day00 Notes: Introduction
  Lecture Slides 1
  N0 (released)

 
 
week 1 (Thu: 08/31)

  Google Colab and Python Review
  Day01 Notes: Python Refresher
  Lecture Slides 2
 
  Reading:
  N0 (due on 08/31)
week 2 (Tue: 09/05)

  Pandas Tutorial
  Day02 Notes: Pandas Tutorial
  Lecture Slides 3
  Notebook#1 (released)
 
  Reading:
 
week 2 (Thu: 09/07)

  Practicing with Pandas
  Day03 Notes: Pandas Practice
  Lecture Slides 4
 
  Reading:
 
week 3 (Tue: 09/12)

  k-Nearest Neighbor (kNN)
  Day04 Notes: k-Nearest Neighbor
  Lecture Slides 5
  Notebook#2 (released)

 
 
  Notebook#1 (due on 09/12)
week 3 (Thu: 09/14)

  kNN code, Missing Data and Normalization
  Day05 Notes: kNN code, Missing Data and Normalization
  Lecture Slides 6
 
  Reading:
  The relationship ML and other fields
  isna()
  notna()
  any()
  value_counts()   dropna()   fillna()
 
week 4 (Tue: 09/19)

  Normalization and Weighted k Nearest Neighbor (kNN)
  Day06 Notes: Normalization and Weighted k-Nearest Neighbor
  Lecture Slides 7
  Q1 (released on Blackboard)

 
 
  Reading:
  sklearn: standard scalar
  Notebook#2 (due on 09/20)
week 4 (Thu: 09/21)

  Weighted k-NN, Graph Plot, and Metrics
  Day07 Notes: Weighted k-NN, Graph, and Metrics
  Lecture Slides 8
  Notebook#3 (released)

 

  Reading:
  Pandas DataFrame: sample
  matplotlib: Pyplot
 
 
week 5 (Tue: 09/26)

  Evaluation Metrics and Introduction to Decision Trees
  Day08 Notes: Metrics and Testing
  Lecture Slides 9
 

  Reading:
  sklearn.metrics: accuracy_score
  sklearn.metrics: confusion_matrix
  sklearn.metrics: mean_absolute_error
  sklearn.metrics: mean_squared_error
 

  Q1 (due on 09/26)
 
week 5 (Thu: 09/28)

  Decision Trees
  Day09 Notes: Decision Trees
  Lecture Slides 10
 
  Reading:
 

 
 
week 6 (Tue: 10/03)

  Introduction to Scikit Learn
  Watch the video lectures on Blackboard
  Day10 Notes: introduction to sklearn
  Notebook#4 (released)

 
  Reading:
  sklearn: train and test split
  sklearn: Decision Tree classifier
  sklearn: k-Nearest-Neighbor Classifier
  sklearn: evaluation metrics
  sklearn: confusion matrix
  sklearn: standard scalar
 
  Notebook#3 (due on 10/03)
week 6 (Thu: 10/05)

  Scikit Learn Practice
  Day11 Notes: sklearn practice
 

  Reading:
 
 
week 7 (Tue: 10/10)

  Random Forests
  Day12 Notes: Random Forests
  Lecture Slides 13

  Reading:
  sklearn: Random Forest Classifier
  Random Forests: Leo Breiman and Adele Cutler
  Application of Random Forest: A Computer Vision research paper ICCV'2007
 
  Notebook#4 (due on 10/12)
week 7 (Thu: 10/12)

  Project #1
  Day13 Notes: Project #1
  Project #1 (released)

 
  Reading:
  sklearn: k-Nearest-Neighbor Classifier
  sklearn: k-Nearest-Neighbor Regressor
  sklearn: Decision Tree Classifier
  sklearn: Decision Tree Regressor
  sklearn: Random Forest Classifier
  sklearn: Random Forest Regressor
 
 
week 8 (Tue: 10/17)

  Fall Break (no class)
 
 
week 8 (Thu: 10/19)

  Project # 1 (discussion)
  Dimensionality Reduction (feature selection and feature extraction)
  Day13 Notes: Dimensionality Reduction
  Lecture Slides 15

 
  sklearn: Feature Selection: SelectKBest
  sklearn: Principle Component Analysis (PCA)
 
 
week 9 (Tue: 10/24)

  Principle Component Analysis (PCA)
  Support Vector Machine (SVM)
  Lecture Slides 16

  Reading:
 
 
week 9 (Thu: 10/26)

  Support Vector Machine (SVM) (continued)
  Lecture Slides 17
  In-class activity (vector-vector dot product)
  Q2 (released)


  Reading:
 
  Project #1 (due by 10/26)
 
week 10 (Tue: 10/31)

  Day16 Notes: SVM code
  Introduction to Neural Networks -- Perceptrons
  Lecture Slides 18
  In-class activity (linear model, perceptron)
  Notebook #5


  Reading:
 
 
week 10 (Thu: 11/02)

  Perceptron Update Rule
  Day18 Notes: Perceptron Code
  Lecture Slides 19

  Reading:
  The Perceptron: A Perceiving and Recognizing Automaton (Rosenblatt - 1957)
 
  Q2 (due on 11/03)
week 11 (Tue: 11/07)

  Optimization: Gradient Descent, Stochastic Gradient Descent (SGD)
  Lecture Slides 20
  Day19 Notes: Loss Surface Visualization + Gradient Descent
  Day19 Notes: Stochastic Gradient Descent (code)

  Reading:
 
  Notebook #5 (due on 11/10)
 
week 11 (Thu: 11/09)

  Multilayer Perceptron (MLP)
  PyTorch Basics
  Lecture Slides 21
  PyTorch Basics in-class activity

  Reading:
  Multilayer Perceptron (MLP)
  The Backpropgation Algorithm (Rumelhart-1985)
  PyTorch
  PyTorch: matmul()
  PyTorch: transpose()
 
week 12 (Tue: 11/14)

  Multilayer Perceptron (MLP) - (PyTorch Implementation)
  Day 22 Notes

  Reading:
  PyTorch: nn.Linear()
  PyTorch: nn.Sigmoid()
  PyTorch: nn.ReLU()
  PyTorch: nn.Softmax()
  PyTorch: nn.Flatten()
  PyTorch: nn.CrossEntropyLoss()
  PyTorch: nn.MSELoss()
  PyTorch: optim.SGD()
  PyTorch: optim.ADAM()
  PyTorch: optim.RMSprop()
  PyTorch: torchvision.datasets
 
 
 
week 12 (Thu: 11/16)

  Convolutional Neural Networks (CNN)
  Lecture Slides 23

  Reading:
  Popular CNN: LeNet (LeCunn-1988)
  Popular CNN: AlexNet (Krizevsky-2012)
  Popular CNN: VGGNet (Simonyan-2015)
  Popular CNN: ResNet (He-2015)
 
 
week 13 (Tue: 11/21)

  Convolutional Neural Network (CNN) - (PyTorch Implementation)
  Day 24 Notes
  Notebook #6

  Reading:
  PyTorch: nn.Conv2d()
  PyTorch: nn.Flatten()
  PyTorch: nn.MaxPool2d()
  PyTorch: nn.AvgPool2d()

 
week 13 (Thu: 11/23)

  Thanksgiving break! (no class)
 
  Reading:
 
 
week 14 (Tue: 11/28)


  Recurrent Neural Network (RNN)
  RNN for Natural Language Processing (NLP)
  Lecture Slides 25
  Project #2

  Reading:
  RNN: Serial Order (Jordan-1986)   Finding Structure in Time (Elman-1990)
  LSTM: Long Short-Term Memory (Hochreiter-1997)
  GRU: Gated Recurrent Unit (Hochreiter-2014)
  The Unreasonable Effectiveness of Recurrent Neural Networks (RNNs)
 
 

week 14 (Thu: 11/30)

  Transformer
  Lecture Slides 26

  Reading:
  Transformer: Attention is All You Need (2017)
  The Illustrated Transformer
  Harvard University NLP: The Annotated
  LSTM is dead. Long Live Transformers! - Paul Dirac talk
 
  Notebook #6 (due on 11/29)
week 15 (Tue: 12/05)

  Large Language Model (LLM)
  Lecture Slides 27
  Project #2 work time
  Q3


  Reading:
  GPT-1 (100M parameters) - Radford et al. 2018 (OpenAI)
  BERT (300M parameters) - Devlin et al. 2018
  GPT-2 (1.5B parameters) - Radford et al. 2019 (OpenAI)
  Megatron-LM (8.0B parameters) - Shoeybi et al. 2019 (NVidia)
  T5 (11.0B parameters) - Raffel et al. 2020 (Google)
  T-NLG (17.0B parameters) - Microsoft Corporation 2020 (Microsoft)
  GPT-3 (175.0B parameters) - Brown et al. 2020 (OpenAI)
  ChatGPT (1.5B parameters) - OpenAI 2022 (OpenAI)
 

week 15 (Thu: 12/07)

  Vision and Language Model (CLIP, BLIP)
  Lecture Slides 28
  Course evaluation

 
  Reading:
  Interacting with CLIP
 

 
Week 16 (Tue: 12/12)

  No final exam!
 
 
  Reading:
 
  Q3 (due on 12/12)
  Project #2 (due by 12/12)