EECS 6322 Deep Learning Deep Learning in Computer Vision

with Prof. Kosta Derpanis (York University)

Course description. Computer Vision is broadly defined as the study of recovering useful properties of the world from one or more images.  In recent years, Deep Learning has emerged as a powerful tool for addressing computer vision tasks.  This course will cover a range of foundational topics at the intersection of Deep Learning and Computer Vision.

Homepage www.eecs.yorku.ca/~kosta

Prerequisites. University-level courses on linear algebra, multivariable calculus, and probability

Lectures.

# TOPIC SLIDES
PDF    MOV
0
Introduction to Computer Vision PDF MOV
1
Ethics, Privacy and Security in Machine Learning
PDF MOV
2
Machine Learning Crash Course PDF MOV

NEURAL NETWORK FOUNDATIONS

3
Multilayer Perceptron
PDF MOV
4
Convolutional Network (ConvNet)
PDF MOV
5
Backprop
PDF MOV
6
Optimization
PDF MOV
7
Introduction to PyTorch (version 1.0)
PDF MOV

SPATIAL MODELS
8
Object Recognition Architectures PDF MOV
9
Training Networks
PDF MOV
10
Weight Initialization
PDF MOV
11
Transfer Learning PDF MOV
12
Object Detection PDF MOV
13
Metric Learning
PDF MOV
14
Pixel Labeling Tasks PDF MOV
15
Optical Flow
PDF MOV
16
Segmentation Aware Filtering
PDF MOV

VISUALIZATION
17
Understanding ConvNets
PDF MOV
18
Texture Synthesis
PDF MOV
19
Style Transfer
PDF MOV

SEQUENCE MODELS
20
Recurrent Neural Network (RNN)
PDF MOV
21
Long Short-Term Memory (LSTM)
PDF MOV
22
Bidirectional RNN
PDF MOV
23
Transformers
PDF MOV
24
Vision and Language
PDF MOV
25
Action Recognition
PDF MOV

SELF-SUPERVISED LEARNING
26
Look Ma No Labels: Learning Without Labels
PDF MOV

GEOMETRIC DEEP LEARNING
27
Graph Neural Networks
PDF MOV

GENERATIVE MODELS
28
Generative Models Introduction
PDF MOV
29
PixelNN (PixelRNN and PixelCNN) PDF MOV
30
Variational Autoencoder (VAE) PDF MOV
31
Invertible Density Models - Normalizing Flows PDF MOV
32
Generative Adversarial Network (GAN) PDF MOV
33
Diffusion Models
PDF MOV

ADVERSARIAL EXAMPLES
34
Adversarial Examples
PDF MOV

Useful textbooks.
     Ian Goodfellow, Yoshua Bengio and Aaron Courville, Deep Learning (available for free or purchase)
     Simon Prince, Understanding Deep Learning (available for free or purchase)
     Francois Fleuret, Deep Learning (available for free or purchase)
     Marc Peter Deisenroth, A Aldo Faisal, and Cheng Soon Ong, Mathematics for Machine Learning, (available for free or purchase)

Useful resources.
    
Terence Parr and Jeremy Howard, The Matrix Calculus You Need for Deep Learning, arXiv
     Justin Johnson, Python NumPy tutorial
     Basic Linear algebra review
     Zico Kolter, Linear algebra review and reference
     Kaare Brandt Petersen and Michael Syskind Pedersen, The Matrix Cookbook

Related courses.
     York University EECS4422/5323: Introduction to Computer Vision with Kosta Derpanis
     University of Michigan EECS 498.008/598.008: Deep Learning foir Computer Vision with Justin Johnson
     Stanford CS231N:
Convolutional Neural Networks for Visual Recognition with Fei Fei Li, Justin Johnson and Serena Young
     EPFL EE-559: Deep Learning
with Francois Fleuret
 

Acknowledgements. While a great effort has been made to assemble an original set of lecture slides, the essence of the presentation of many of the slides rely on material prepared by the following people: Andrej Karpathy, Justin Johnson, Serena Young, Fei Fei Li, Francois Fleuret, Graham Taylor, Carl Doersch.