Dive into Deep Learning in 1 Day

Last updated：|today|

Information

• Speaker： Alex Smola

Overview

Did you ever want to find out about deep learning but didn't have time to spend months? New to machine learning? Do you want to build image classifiers, NLP apps, train on many GPUs or even on many machines? If you're an engineer or data scientist, this course is for you. This is about the equivalent of a Coursera course, all packed into one day. The course consists of four segments of 90 minutes each.

1. Deep Learning Basics
2. Convolutional Neural Networks for computer vision
3. Best practices (GPUs, Parallelization, Fine Tuning, Transfer Learning)
4. Recurrent Neural Networks for natural language (RNN, LSTM)

Prerequisites

You should have some basic knowledge of Linear Algebra, Calculus, Probability, and Python (here's another book to learn Python). Moreover, you should have some experience with Jupyter notebooks, or with SageMaker notebooks. To run things on (multiple) GPUs you need access to a GPU server, such as the P2, G3, or P3 instances.

Syllabus

• This course relies heavily on the Dive into Deep Learning book. There's a lot more detail in the book (notebooks, examples, math, applications).
• The crash course will get you started. For more information also see other courses and tutorials based on the book.
• All notebooks below are availabe at d2l-ai/1day-notebooks, which contains instructions how to setup the running environments.

Time	Topics
9:00---10:00	Part 1: Deep learning basic
10:00---11:00	Part 2: Convolutional neural networks
11:00---12:00	Part 3: Performance
12:00---1:00	Part 4: Recurrent neural networks

Part 1: Deep Learning Basic

Slides: [keynote], [pdf]

Notebooks:

1. Data Manipulation with Numpy [ipynb] [slides]
2. Automatic Differentiation [ipynb] [slides]
3. Linear Regression [ipynb] [slides]
4. Image Classification Data (Fashion-MNIST) [ipynb] [slides]
5. Softmax Regression [ipynb] [slides]
6. Multilayer Perceptrons [ipynb] [slides]

Part 2: Convolutional neural networks

Slides: [keynote], [pdf]

Notebooks:

1. GPUs [ipynb] [slides]
2. Convolutions [ipynb] [slides]
3. Pooling [ipynb] [slides]
4. Convolutional Neural Networks (LeNet) [ipynb] [slides]
5. Deep Convolutional Neural Networks (AlexNet) [ipynb] [slides]
6. Inception Networks (GoogLeNet) [ipynb] [slides]
7. Residual Networks (ResNet) [ipynb] [slides]

Part 3: Performance

Slides: [keynote], [pdf]

Notebooks:

1. A Hybrid of Imperative and Symbolic Programming [ipynb] [slides]
2. Multi-GPU Computation Implementation from Scratch [ipynb] [slides]
3. Concise Implementation of Multi-GPU Computation [ipynb] [slides]
4. Fine Tuning [ipynb] [slides]

Part 4: Recurrent neural networks

Slides: [keynote], [pdf]

Notebooks:

1. Text Preprocessing [ipynb] [slides]
2. Concise Implementation of Recurrent Neural Networks [ipynb] [slides]
3. Long Short Term Memory (LSTM) [ipynb] [slides]