Schedule
A tentative schedule below (subject to change as we progress).
Date Topic Reading Items due week 1 (Tue: 08/25)
Introduction to Computer Vision
Lecture Slides 1
Course logistics
Introduction to computer vision
week 1 (Thur: 08/27)
Image Filtering: cross-correlation
Lecture Slides 2
In-class activity #1 (image filtering cross-correlation)
Pillow concepts
Pillow tutorial
Image.open(path)
Image.save(path)
Image.new('L', (width, height), color=0)
Image.getpixel( (x,y) )
Image.putpixel( (x,y), value)
ImageDraw( img )
week 2 (Tue: 09/01)
Image Filtering: convolution
Lecture Slides 3
Notebook#1 (released on 08/31)
week 2 (Thur: 09/03)
Image-based Recognition
Image Classification and Neural Networks (NN)
Types of NN: Multilayer Perceptron (MLP)
Lecture Slides 4
Day04 Notes: PyTorch MLP
PyTorch Basics (review this before attempting activity#2)
PyTorch: nn.Linear()
PyTorch: nn.Sigmoid()
PyTorch: nn.Tanh()
PyTorch: nn.ReLU()
week 3 (Tue: 09/07)
Labor Day (university holiday; classes do not meet)
Notebook#1 (due on 09/07) week 3 (Thur: 09/08)
Image-based Recognition
Image Classification and Neural Networks (NN)
Types of NN: Convolution Neural Network (CNN)
Lecture Slides 5
Day06 Notes: PyTorch CNN
Notebook #2 (released on 09/08)
PyTorch: nn.Conv2d()
PyTorch: nn.MaxPool2d()
PyTorch: nn.AvgPool2d()
week 4 (Tue: 09/15)
Popular Image Classification Networks (AlexNet, VGG, ResNet)
Dissecting Popular Pre-trained CNN models
Lecture Slides 6
Day07 Notes: AlexNet dissection
Day07 Notes: VGG-16 dissection
Day07 Notes: ResNet dissection
The Shattered Gradients Problem: If resnets are the answer, then what is the question?
week 4 (Thur: 09/17)
Fine-tuning a Pre-trained CNN model (AlexNet, VGG, ResNet)
Lecture Slides 7
Day08 Notes: Fine-tuning CNNs
Notebook#3 (Released on 09/16)
Notebook#2 (due on 09/16)
Week 5 (Tue: 09/22)
Vision Transformer (ViT)
Other ViT variants (SwinTransformer, MaxViT)
Lecture Slides 8
Day09 Notes: Vision Transformer (ViT) Dissection
The Hardware Lottery - Sara Hooker
Transformer (designed to win the hardware lottery)
Vision Transformer (ViT): Transformers for image recognition at scale
week 5 (Thur: 09/24)
Object Detection: Classical and Deep Learing-Based
Lecture Slides 9a: Classical Detectors
Lecture Slides 9b: Deep learning-based Detectors (Faster R-CNN, Mask R-CNN)
week 6 (Tue: 09/29)
Inference with Popular Object Detectors (Faster R-CNN, Mask R-CNN)
Lecture Slides 10
Day11 Notes: Inference with Object Detector with Faster R-CNN
Quiz#1 (released on Blackboard 09/30)
Notebook#3 (due on 09/28)
week 6 (Thur: 10/01)
Lecture Slides 11
Fine-tuning Popular Object Detectors (Faster R-CNN, Mask R-CNN)
Day12 Notes: Fine-tuning Object Detectors (Detectron/Faster R-CNN)
Notebook#4: Fine-tuning Detectors: Faster & Mask R-CNN
week 7 (Tue: 10/06)
Image Segmentation
Classical Segmentation (Felzenswalb, SLIC, clustering)
Lecture Slides 12 part1
Lecture Slides 12 part2
Day13 Notes: Classical Image Segmentation Codes
week 7 (Thur: 10/08)
Vision Foundation Model for Image Segmentation: Segment Anything (SAM)
Lecture Slides 13 [pptx]
Day14 Notes: Inference with SAM: Generating Object Masks with SAM
Quiz#2 (due on 10/07)
week 8 (Tue: 10/13)
Fall Break (classes do not meet)
week 8 (Thur: 10/15)
Semantic Segmentaiton
Popular Semantic Segmentation Models (U-Net, FCN, SegNet, PSPNet)
Lecture Slides 14
Notebook #5: Semantic Segmentation with UNet
Notebook#4 (due on 10/14) week 9 (Tue: 10/20)
Deep Neural Network Feature Space Visualization: tSNE
Application to Image Retrieval
Lecture Slides 15
Day17 Notes: Deep Features from Popular Networks (VGG, ResNet, ViT)
week 9 (Thur: 10/22)
Probability Basics
Discrete Probability Distribution
Lecture Slides 16 part1
Day18 Notes: Discrte probability distribution code
Continous Probability Distribution
Gaussian Distributions
Lecture Slides 16 part2
Day18 Notes: In-class activity#5: Continous probability distribution
week 10 (Tue: 10/27)
Image Generation and Deep Generative Models (Part#1)
Generative Adversarial Network (GAN)
Lecture Slides 17
AI Art
Notebook#5 (due on 10/26)
week 10 (Thur: 10/29)
Maximum Likelihood (ML) Estimate
Lecture Slides 18 part1
Day20 Notes: In-class activity#6: Maximum likelihood estimate for Gaussian distribution
Quiz#2 (released on 10/28)
Tutorial on variational autoencoder by Carl Doersch
Generating Counterfactual Images: C2C-VAE - CBREIS'22
Generative model: implicit reparameterization
Wasserstein Distance
Week 11 (Tue: 11/03)
Image Generation and Deep Generative Models (Part#2)
Variational Auto Encoder (VAE)
Lecture Slides 19
Day21 Notes: In-class activity#7: VAE code for digit image generation using MNIST
Week 11 (Thur: 11/05)
Image Generation and Deep Generative Models (Part#3)
Diffusion Model
Lecture Slidees 20
Day22 Notes: In-class activity#8: Image generation from text using Stable Diffusion
Diffusion models are evolutionary algorithms - arxiv'24
Quiz#2 (due on 11/04)
Week 12 (Tue: 11/10)
Image Transformation
Linear transformation
Lecture Slidees 21
In-class activity#8: Rotation, Scaling, Shearing transformation
Week 12 (Thur: 11/12)
Week 13 (Tue: 11/17)
Homogenous coordinates
Affine and projective transformation
Lecture Slides 22a
Camera Projection
Modeling projection
Activity: projection practice
Generalizing the model
Adding a lens
Lecture Slides 22b
From Images to 3D Models-ACM Communication'2002
Week 13 (Thur: 11/19)
Classical 3D Reconstruction (Part#1)
Tueocular Depth Estimation: 3D Reconstruction from Single Images
Stereo Matching: 3D Reconstruction from Two Images
Multiview Stereo: 3D Reconstruction from Multiple Images
Lecture Slides
Final Project Presentation on 12/11
Week 14 (Tue: 11/24)
Deep Learning-based 3D Reconstruction (Part#2)
Monocular Depth Estimation Demo
Stereo Matching Demo
Multiview Stereo Demo
Lecture Slides
week 14 (Thur: 11/26)
Thanksgiving Break (classes do not meet)
Week 15 (Tue: 12/01)
Large Multimodal Model (LMM)
LMM Application using PyTorch
Lecture Slides 27
Reading:
Dissociating language and thought in large language models-2024
Is OpenAI-o1 reasoning? (ML Street Talk'24)
World Model (Jurgen Schmidhuber@NeurIPS'18)
LoRA: Low-Rank Adaptation of Large Language Models (ICLR'22)
Week 15 (Thur: 12/03)
Final project group presentation
Location: Collier-Scripps#335
Quiz#3 (released on 12/03)
Course Evaluation
Week 16 (Tueday: 12/07)
No final exam!
Reading:
Q3 (due on 12/07)
Final Project Code +
Presentation Slides (due by 12/11)