| Event Type | Date | Description | Course Materials |
|---|---|---|---|
| Lecture | Tuesday, Aug. 25 | Intro to Deep Learning, historical context. |
[slides] [python/numpy tutorial] [jupyter tutorial] |
| Lecture | Thursday, Aug. 27 | Image classification and the data-driven approach k-nearest neighbor Linear classification |
[slides] [image classification notes] [linear classification notes] |
| Lecture | Tuesday, Sept. 1 | Loss functions |
[slides] |
| Lecture | Thursday, Sept. 3 | Optimization: Stochastic Gradient Descent and Backpropagation |
[slides] [optimization notes] |
| Optional Discussion | Friday, Sept. 4 | (8:30-9:30am) Slicing and broadcasting in Python |
[slicing and broadcasting ipynb] |
| Lecture | Tuesday, Sept. 8 | Backpropagation & Neural Networks I |
[slides] [backprop notes] [Efficient BackProp] (optional) related: [1], [2], [3] (optional) |
| Lecture | Thursday, Sept. 10 |
Neural Networks II Higher-level representations, image features Vector, Matrix, and Tensor Derivatives |
[slides] handout 1: Vector, Matrix, and Tensor Derivatives handout 2: Derivatives, Backpropagation, and Vectorization Deep Learning [Nature] (optional) |
| Optional Discussion | Friday, Sept. 11 | (8:30-9:30am) Vector, Matrix, Tensor Derivatives and Backpropagation | [notes] |
| Lecture | Tuesday, Sept. 15 |
Neural Networks III Training Neural Networks I Activation Functions |
[slides] [Neural Nets notes 1] tips/tricks: [1], [2] (optional) |
| Lecture | Thursday, Sept. 17 |
Training Neural Networks II: weight initialization, batch normalization |
[slides] [Neural Nets notes 2] [Batch Norm] Copula Normalization (optional) |
| Lecture | Tuesday, Sept. 22 |
Training Neural Networks III: babysitting the learning process, hyperparameter optimization |
[slides] [Bengio 2012] (optional) |
| Lecture | Thursday, Sept. 24 |
Training Neural Network IV: model ensembles, dropout |
[slides] [Neural Nets notes 3] LeNet (optional) |
| Optional Discussion | Friday, Sept. 25 | (8:30-9:30am) A closer look at the maths inside batch normalization | [notes] |
| Lecture | Tuesday, Sept. 29 |
Training Neural Network V: parameter updates |
[slides] |
| Lecture | Thursday, Oct. 1 |
Convolutional Neural Networks: convolution layer, pooling layer, fully connected layer |
[slides] |
| Optional Discussion | Friday, Oct. 2 | (8:30-9:30am) Convolutional neural networks | [slides] |
| Lecture | Tuesday, Oct. 6 |
Convolutional Neural Networks: (cont.) convolution layer, pooling layer, fully connected layer |
|
| Lecture | Thursday, Oct. 8 | ConvNets for spatial localization, Object detection |
[slides] FCN (optional) mAP (optional) |
| Lecture | Tuesday, Oct. 13 | ConvNets for spatial localization, Object detection (cont.) | |
| Lecture | Thursday, Oct. 15 |
Understanding and visualizing Convolutional Neural Networks |
[slides] a tool to visualize convolutions |
| Lecture | Tuesday, Oct. 20 | Understanding and visualizing Convolutional Neural Networks (cont.) | |
| Lecture | Thursday, Oct. 22 | Creating Adversarial Examples |
[slides] |
| Lecture | Tuesday, Oct. 27 | Finish Generative Adversarial Networks, Start RNNs |
[slides] |
| Lecture | Thursday, Oct. 29 | Finish RNNs. |
[slides] |
| Lecture | Tuesday, Nov. 3 | Do word embeddings, ElMO? Attention and Self-Attention in NLP, Transformers |
[slides] |
| Lecture | Thursday, Nov. 5 | Exam review and instructions for exam |
[slides] [review sheet] |
| Optional Discussion | Friday, Nov. 6 | (8:30-9:30am) Midterm Review | |
| Exam | Tuesday, Nov. 10 | Midterm Exam (no class) | |
| Lecture | Thursday, Nov. 12 | Face recognition and the dangers of AI technology. | |
| Lecture | Tuesday, Nov. 17 | Final project presentation instructions and highlights of recent DNN research. | |
| Final Presentation | Thursday, Nov. 19 |
Final presentation Final report due (Last day of classes) |