06_cnn

Convolutional Neural Network (CNN)

• Week 6 CNN Slides

Session A: What is CNN?

Objectives:

• Understand when and why you might train your own model from scratch versus use a pre-trained model or transfer learning.
• Learn about the Google “Quick, Draw!” dataset.
• Understand how ato work with image data for training your own model.

Quick, Draw! Data

• Data and Documentation
• Video tutorial: Replaying Drawings with node server
• Video tutorial: Replaying Drawings with Google Web API
• Preparing Data as Images for Doodle Classifer Part 1
• Preparing Data as Images for Doodle Classifer Part 2

Examples

• Displaying MNIST
• Displaying Quick, Draw! 28x28 images
• Animating Quick, Draw! paths

Creative Quick, Draw! projects

• Letter collages by Deborah Schmidt
• Face tracking experiment by Neil Mendoza
• Faces of Humanity by Tortue
• Scribbling Speech by Xinyue Yang
• How do you draw a circle?

Other Related Projects:

• Machine Learning for Visualization by Ian Johnson
• MegaPixels: Faces curated by Tactical Tech, design and development by Adam Harvey
• Watch What Neural Networks See by Gene Kogan
• Recognizing Human Facial Expressions With Machine Learning by Angelica Perez

Session B: Doodle Classification

Objectives

• Learn to train an image classifier (no convolutional layers) with ml5.js.
• Learn the distinction between different types of layers of a neural network, specifically “What is a convolutional layer?”
• Learn to feed the input of a graphics canvas into a machine learning model.

Video Tutorials

• What is Convolutional Neural Network Part 1
• What is Convolutional Neural Network Part 2

Convolutional Neural Nets

• Original 1998 "LetNet5" paper: "Gradient-Based Learning Applied to Document Recognition" by Y. Lecun, L. Bottou, Y. Bengio, P. Haffner
• Interactive Node-Link Visualizations of Convolutional Neural Networks
• How computers got shockingly good at recognizing images by Timothy B. Lee
• Image Kernels Explained Visually by Victor Powell
• A visual and intuitive understanding of deep learning, CNNs (0:00 - 9:40) by Octavio Good

Examples

• p5.js Convolution demo
• p5.js Convolution demo -- max pooling
• Training a model with ml5.neuralNetwork() and Google Quick, Draw! images
• Classifying Drawings with ml5's DoodleNet (model trained by @yining1023)

Assignment 6 Due Sunday October 18

Reading

• An Intuitive Explanation of Convolutional Neural Networks by Ujjwal Karn
• Exploring and Visualizing an Open Global Dataset by Google Research

Reflection

• After playing Quick, Draw in class, reviewing the Quick, Draw, and reading the above Exploring and Visualizing an Open Global Dataset by Google Research, consider the following question. How can visualization help diagnose "data for inclusion" and "identify concrete ways that anyone can improve the variety of data, as well as check for potential biases."

Coding Exercise

• Build off of one of the following code examples (or invent your own) to develop you own creative use of Quick, Draw data.
  • Rendering Quick, Draw drawings
  • Animating Quick, Draw! paths
  • Classifying Drawings with ml5's DoodleNet: output in DOM element -- Can you make this one work with webcam input instead of canvas?
  • Classifying Drawings with ml5's DoodleNet: output in separate canvas

Complete a blog post with thoughts on the Quick, Draw dataset and your code exercise. Link from the homework wiki.