EECS 6322 Deep Learning Deep Learning in Computer Vision

with Prof. Kosta Derpanis (York University)

Course description. Computer Vision is broadly defined as the study of recovering useful properties of the world from one or more images.  In recent years, Deep Learning has emerged as a powerful tool for addressing computer vision tasks.  This course will cover a range of foundational topics at the intersection of Deep Learning and Computer Vision.

Homepage www.eecs.yorku.ca/~kosta

Prerequisites. University-level courses on linear algebra, multivariable calculus, and probability

Lectures.

# TOPIC SLIDES
PDF    MOV
0
Introduction to Computer Vision PDF MOV
1
Ethics, Privacy and Security in Machine Learning
PDF MOV
2
Machine Learning Crash Course PDF MOV

NEURAL NETWORK FOUNDATIONS

3
Multilayer Perceptron
PDF MOV
4
Convolutional Network (ConvNet)
PDF MOV
5
Backprop
PDF MOV
6
Optimization
PDF MOV
7
Introduction to PyTorch (version 1.0+)
PDF MOV
9
Training Networks
PDF MOV
10
Weight Initialization
PDF MOV
11
Transfer Learning PDF MOV
12
Teacher-Student Learning
PDF MOV

SPATIAL MODELS

13 Object Recognition Architectures PDF MOV
14
Object Detection PDF MOV
15
Metric Learning
PDF MOV
16
Pixel Labeling Tasks PDF MOV
17
Optical Flow
PDF MOV
18
Segmentation Aware Filtering
PDF MOV

VISUALIZATION
19
Understanding ConvNets
PDF MOV
20
Texture Synthesis
PDF MOV
21
Style Transfer
PDF MOV

SEQUENCE MODELS
22
Recurrent Neural Network (RNN)
PDF MOV
23
Long Short-Term Memory (LSTM)
PDF MOV
24
Bidirectional RNN
PDF MOV
25
Transformers
PDF MOV
26
Vision and Language
PDF MOV
27
Action Recognition
PDF MOV

SELF-SUPERVISED LEARNING
27
Look Ma No Labels: Learning Without Labels
PDF MOV

GEOMETRIC DEEP LEARNING
28
Graph Neural Networks
PDF MOV

GENERATIVE MODELS
29
Generative Models Introduction
PDF MOV
30
PixelNN (PixelRNN and PixelCNN) PDF MOV
31
Variational Autoencoder (VAE) PDF MOV
32
Invertible Density Models - Normalizing Flows PDF MOV
33
Generative Adversarial Network (GAN) PDF MOV
34
Diffusion Models
PDF MOV
35
Flow Matching
PDF MOV

ADVERSARIAL EXAMPLES
36
Adversarial Examples
PDF MOV

Useful textbooks.
     Ian Goodfellow, Yoshua Bengio and Aaron Courville, Deep Learning (available for free or purchase)
     Simon Prince, Understanding Deep Learning (available for free or purchase)
     Francois Fleuret, Deep Learning (available for free or purchase)
     Marc Peter Deisenroth, A Aldo Faisal, and Cheng Soon Ong, Mathematics for Machine Learning, (available for free or purchase)

Useful resources.
    
Terence Parr and Jeremy Howard, The Matrix Calculus You Need for Deep Learning, arXiv
     Justin Johnson, Python NumPy tutorial
     Basic Linear algebra review
     Zico Kolter, Linear algebra review and reference
     Kaare Brandt Petersen and Michael Syskind Pedersen, The Matrix Cookbook

Related courses.
     York University EECS4422/5323: Introduction to Computer Vision with Kosta Derpanis
     University of Michigan EECS 498.008/598.008: Deep Learning foir Computer Vision with Justin Johnson
     Stanford CS231N:
Convolutional Neural Networks for Visual Recognition with Fei Fei Li, Justin Johnson and Serena Young
     EPFL EE-559: Deep Learning
with Francois Fleuret
 

Acknowledgements. While a great effort has been made to assemble an original set of lecture slides, the essence of the presentation of many of the slides rely on material prepared by the following people: Andrej Karpathy, Justin Johnson, Serena Young, Fei Fei Li, Francois Fleuret, Graham Taylor, Carl Doersch.