Schedule
A tentative schedule below (subject to change as we progress).
Date Topic Reading Items due week 1 (Mon: 08/24)
Introduction to Computer Vision
Lecture Slides 1
Course logistics
Introduction to computer vision
week 1 (Wed: 08/26)
Image Filtering: cross-correlation
Lecture Slides 2
In-class activity #1 (image filtering cross-correlation)
Pillow concepts
Pillow tutorial
Image.open(path)
Image.save(path)
Image.new('L', (width, height), color=0)
Image.getpixel( (x,y) )
Image.putpixel( (x,y), value)
ImageDraw( img )
week 2 (Mon: 08/31)
Image Filtering: convolution
Lecture Slides 3
Notebook#1 (released on 08/31)
week 2 (Wed: 09/02)
Image-based Recognition
Image Classification and Neural Networks (NN)
Types of NN: Multilayer Perceptron (MLP)
Lecture Slides 4
Day04 Notes: PyTorch MLP
PyTorch Basics (review this before attempting activity#2)
PyTorch: nn.Linear()
PyTorch: nn.Sigmoid()
PyTorch: nn.Tanh()
PyTorch: nn.ReLU()
week 3 (Mon: 09/07)
Labor Day (university holiday; classes do not meet)
Notebook#1 (due on 09/07) week 3 (Wed: 09/09)
Image-based Recognition
Image Classification and Neural Networks (NN)
Types of NN: Convolution Neural Network (CNN)
Lecture Slides 5
Day06 Notes: PyTorch CNN
Notebook #2 (released on 09/09)
PyTorch: nn.Conv2d()
PyTorch: nn.MaxPool2d()
PyTorch: nn.AvgPool2d()
week 4 (Mon: 09/14)
PyTorch Implementation
In-class activity #3 (PyTorch CNN)
week 4 (Wed: 09/16)
Popular Image Classification Networks (AlexNet, VGG, ResNet)
Dissecting Popular Pre-trained CNN models
Lecture Slides 8
AlexNet dissection
VGG-16 dissection
ResNet dissection
Notebook#2 (due on 09/16)
Week 5 (Mon: 09/21)
Fine-tuning a Pre-trained CNN model
Lecture Slides 9
Fine-tuning CNNs
Notebook#3 (Fine-tuning popular CNNs)
week 5 (Wed: 09/23)
Vision Transformer (ViT)
Other ViT variants (Swin Transformer Tiny)
ConvNext, Inflated-ConvNext Tiny
Lecture Slides 10
Vision Transformer (ViT) Dissection
The Hardware Lottery - Sara Hooker
Transformer (designed to win the hardware lottery)
Vision Transformer (ViT): Transformers for image recognition at scale
week 6 (Mon: 09/28)
Object Detection: Classical and Deep Learing-Based
Lecture Slides 11a: Classical Detectors
Lecture Slides 11b: Deep learning-based Detectors (Faster R-CNN, Mask R-CNN)
Notebook#3 (due on 10/02)
week 6 (Wed: 09/30)
Inference with Popular Object Detectors (Faster R-CNN, Mask R-CNN)
Lecture Slides 12
Inference with Object Detector with Faster R-CNN
Quiz#1 (released on Blackboard 09/30)
week 7 (Mon: 10/05)
Lecture Slides 13
Fine-tuning Popular Object Detectors (Faster R-CNN, Mask R-CNN)
Fine-tuning Object Detector: Detectron/Faster R-CNN
Notebook#4 (Fine-tuning Detectors: Faster & Mask R-CNN)
week 7 (Wed: 10/07)
Image Segmentation
Classical Segmentation (Felzenswalb, SLIC, clustering)
Lecture Slides 13 part1
Lecture Slides 13 part2
Classical Image Segmentation Codes
Quiz#2 (due on 10/07)
week 8 (Mon: 10/12)
Fall Break (classes do not meet)
week 8 (Wed: 10/14)
Vision Foundation Model for Image Segmentation: Segment Anything (SAM)
Lecture Slides 14 [pptx]
Inference with SAM: Automatically generating object masks with Segment Anything (SAM)
Notebook#4 (due on 10/14) week 9 (Mon: 10/19)
Semantic Segmentaiton
Popular Semantic Segmentation Models (U-Net, FCN, SegNet, PSPNet)
Lecture Slides 15
Notebook #5 (Semantic Segmentation Inference and Training with UNet)
week 9 (Wed: 10/21)
Probability Basics
Discrete Probability Distribution
Lecture Slides 16 part1
Discrte probability distribution code
Continous Probability Distribution
Gaussian Distributions
Lecture Slides 16 part2
In-class activity#5: Continous probability distribution
week 10 (Mon: 10/26)
Image Generation and Deep Generative Models: Generative Adversarial Network (GAN)
Lecture Slides 17
AI Art
week 10 (Wed: 10/28)
Maximum Likelihood (ML) Estimate
Lecture Slides 18 part1
In-class activity#6: Maximum likelihood estimate for Gaussian distribution
Image Generation and Deep Generative Models: Variational Auto Encoder (VAE)
Lecture Slides 18 part2
Quiz#2 (released on 10/28)
Tutorial on variational autoencoder by Carl Doersch
Generating Counterfactual Images: C2C-VAE - CBREIS'22
Generative model: implicit reparameterization
Wasserstein Distance
Week 11 (Mon: 11/02)
Image Generation and Deep Generative Models: Variational Auto Encoder (VAE)
Lecture Slides 18 part2 (continued)
In-class activity#7: VAE code for digit image generation using MNIST dataset
Notebook#5 (due on 11/02)
Week 11 (Wed: 11/04)
Image Generation and Deep Generative Models: Diffusion Model
Lecture Slidees 19
In-class activity#8: Generating images from text prompt using Stable Diffusion Model
Diffusion models are evolutionary algorithms - arxiv'24
Quiz#2 (due on 11/04)
Week 12 (Mon: 11/09)
Image Transformation
Linear transformation
Lecture Slidees 20
In-class activity#8: Rotation, Scaling, Shearing transformation
Week 12 (Wed: 11/11)
Week 13 (Mon: 11/16)
Homogenous coordinates
Affine and projective transformation
Lecture Slides 21 part1
Camera Projection
Modeling projection
Activity: projection practice
Generalizing the model
Adding a lens
Lecture Slides 21 part2
From Images to 3D Models-ACM Communication'2002
Week 13 (Wed: 11/18)
Classical 3D Reconstruction (Part#1)
Monocular Depth Estimation: 3D Reconstruction from Single Images
Stereo Matching: 3D Reconstruction from Two Images
Multiview Stereo: 3D Reconstruction from Multiple Images
Lecture Slides
Final Project Presentation on 12/11
Week 14 (Mon: 11/23)
Deep Learning-based 3D Reconstruction (Part#2)
Monocular Depth Estimation Demo
Stereo Matching Demo
Multiview Stereo Demo
Lecture Slides
week 14 (Wed: 11/25)
Thanksgiving Break (classes do not meet)
Week 15 (Mon: 11/30)
Large Multimodal Model (LMM)
LMM Application using PyTorch
Lecture Slides 27
Reading:
Dissociating language and thought in large language models-2024
Is OpenAI-o1 reasoning? (ML Street Talk'24)
World Model (Jurgen Schmidhuber@NeurIPS'18)
LoRA: Low-Rank Adaptation of Large Language Models (ICLR'22)
Week 15 (Wed: 12/02)
Final project group presentation
Location: Collier-Scripps#335
Quiz#3 (released on 12/02)
Course Evaluation
Week 16 (Monday: 12/07)
No final exam!
Reading:
Q3 (due on 12/07)
Final Project Code +
Presentation Slides (due by 12/11)