Lab_CNN_Santas.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Using Convolutional Neural Networks to analyze Santa Data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1.  First things first: Importing images"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# import time\n",
    "import matplotlib.pyplot as plt\n",
    "import scipy\n",
    "from PIL import Image\n",
    "from scipy import ndimage\n",
    "import numpy as np\n",
    "\n",
    "np.random.seed(123)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os, shutil\n",
    "directory = 'data/'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(len(os.listdir('data/santa')))\n",
    "print(len(os.listdir('data/not_santa')))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from keras.preprocessing.image import ImageDataGenerator, array_to_img, img_to_array, load_img\n",
    "\n",
    "# get all the data in the directory split/test (180 images), and reshape them\n",
    "data_all = ImageDataGenerator(rescale=1./255).flow_from_directory( \n",
    "        directory, \n",
    "        target_size=(64, 64), \n",
    "        batch_size = 924, \n",
    "        seed = 123) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "images, labels = next(data_all)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(np.shape(images))\n",
    "print(np.shape(labels))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.1  Looking at the images"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "np.shape(images[1])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt\n",
    "plt.imshow(images[4])\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "r_img = images[4][:,:,0]\n",
    "g_img = images[4][:,:,1]\n",
    "b_img = images[4][:,:,2]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "np.shape(b_img)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plt.imshow(b_img, cmap=plt.cm.binary)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.2 Looking at the labels"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The first columns represents \"no santa\", the second one represents \"santa\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "labels"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. Building a densely connected network"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.1 reshaping the images"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "X = images.reshape(images.shape[0], -1)\n",
    "y = np.reshape(labels[:,0], (922,1))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "np.shape(X)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "np.shape(y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.2 Creating a train, test and validation set"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "X_model, X_test, y_model, y_test = train_test_split(X, y, test_size=0.20, random_state=123)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "np.shape(X_model)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "np.shape(X_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "X_train, X_val, y_train, y_val = train_test_split(X_model, y_model, test_size=0.20, random_state=123)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "np.shape(X_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "np.shape(y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "np.shape(y_val)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- Our training set contrains 589 images\n",
    "- Our test set contains 185 images\n",
    "- Our validation set contains 148 images"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, let's build our baseline network. You'll build two dense hidden layers, the first one with 128 hidden nodes, and the second one with 64 modes. Use the rectified linear unit as an activation function for both. Remember that the input shape is 64 x 64 x 3 for both layers. The last layer of the network should has a sigmoid activation function (because this is a binary classification problem), and 1 output node.\n",
    "\n",
    "Let's train the model for 60 epochs in batches of 50."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from keras import models\n",
    "from keras import layers\n",
    "from numpy.random import seed\n",
    "seed(123)\n",
    "from tensorflow import set_random_seed\n",
    "set_random_seed(123)\n",
    "\n",
    "dense = models.Sequential()\n",
    "dense.add(layers.Dense(128, activation='relu', input_shape=(12288,))) #2 hidden layers\n",
    "dense.add(layers.Dense(64, activation='relu'))\n",
    "dense.add(layers.Dense(1, activation='sigmoid'))\n",
    "\n",
    "dense.compile(optimizer='sgd',\n",
    "              loss='binary_crossentropy',\n",
    "              metrics=['accuracy'])\n",
    "\n",
    "dense_fit = dense.fit(X_train,\n",
    "                    y_train,\n",
    "                    epochs=60,\n",
    "                    batch_size=50,\n",
    "                    validation_data=(X_val, y_val))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "hist_dense = dense_fit.history\n",
    "loss_values = hist_dense['loss']\n",
    "val_loss_values = hist_dense['val_loss']\n",
    "\n",
    "epochs = range(1, len(loss_values) + 1)\n",
    "plt.plot(epochs, loss_values, 'g.', label='Training loss')\n",
    "plt.plot(epochs, val_loss_values, 'g', label='Validation loss')\n",
    "\n",
    "plt.title('Training and validation loss')\n",
    "plt.xlabel('Epochs')\n",
    "plt.ylabel('Loss')\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "acc_values = hist_dense['acc'] \n",
    "val_acc_values = hist_dense['val_acc']\n",
    "\n",
    "plt.plot(epochs, acc_values, 'r.', label='Training acc')\n",
    "plt.plot(epochs, val_acc_values, 'r', label='Validation acc')\n",
    "plt.title('Training and validation accuracy')\n",
    "plt.xlabel('Epochs')\n",
    "plt.ylabel('Accuracy')\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from keras import models\n",
    "from keras import layers\n",
    "from numpy.random import seed\n",
    "seed(123)\n",
    "from tensorflow import set_random_seed\n",
    "set_random_seed(123)\n",
    "\n",
    "dense = models.Sequential()\n",
    "dense.add(layers.Dense(128, activation='relu', input_shape=(12288,))) #2 hidden layers\n",
    "dense.add(layers.Dense(64, activation='relu'))\n",
    "dense.add(layers.Dense(1, activation='sigmoid'))\n",
    "\n",
    "dense.compile(optimizer='sgd',\n",
    "              loss='binary_crossentropy',\n",
    "              metrics=['accuracy'])\n",
    "\n",
    "dense_fit = dense.fit(X_train,\n",
    "                    y_train,\n",
    "                    epochs=30,\n",
    "                    batch_size=50,\n",
    "                    validation_data=(X_val, y_val))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "hist_dense = dense_fit.history\n",
    "loss_values = hist_dense['loss']\n",
    "val_loss_values = hist_dense['val_loss']\n",
    "acc_values = hist_dense['acc'] \n",
    "val_acc_values = hist_dense['val_acc']\n",
    "\n",
    "epochs = range(1, len(loss_values) + 1)\n",
    "\n",
    "plt.figure(figsize=(15,4))\n",
    "plt.subplot(121)\n",
    "plt.plot(epochs, loss_values, 'g.', label='Training loss')\n",
    "plt.plot(epochs, val_loss_values, 'g', label='Validation loss')\n",
    "\n",
    "plt.title('Training and validation loss')\n",
    "plt.xlabel('Epochs')\n",
    "plt.ylabel('Loss')\n",
    "plt.legend()\n",
    "\n",
    "plt.subplot(122)\n",
    "plt.plot(epochs, acc_values, 'r.', label='Training acc')\n",
    "plt.plot(epochs, val_acc_values, 'r', label='Validation acc')\n",
    "plt.title('Training and validation accuracy')\n",
    "plt.xlabel('Epochs')\n",
    "plt.ylabel('Accuracy')\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "results_train = dense.evaluate(X_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "results_test = dense.evaluate(X_test, y_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "results_train"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "results_test"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. Convolutional neural network"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "X_model, X_test, y_model, y_test = train_test_split(images, y, test_size=0.20, random_state=123)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "X_train, X_val, y_train, y_val = train_test_split(X_model, y_model, test_size=0.20, random_state=123)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from numpy.random import seed\n",
    "seed(123)\n",
    "from tensorflow import set_random_seed\n",
    "set_random_seed(123)\n",
    "\n",
    "model = models.Sequential()\n",
    "model.add(layers.Conv2D(64, (3, 3), activation='relu', input_shape=(64, 64,  3)))\n",
    "model.add(layers.MaxPooling2D((2, 2)))\n",
    "model.add(layers.Conv2D(32, (3, 3), activation='relu'))\n",
    "model.add(layers.MaxPooling2D((2, 2)))\n",
    "\n",
    "model.add(layers.Flatten())\n",
    "model.add(layers.Dense(32, activation='relu'))\n",
    "model.add(layers.Dense(1, activation='sigmoid'))\n",
    "\n",
    "model.compile(loss='binary_crossentropy',\n",
    "              optimizer=\"sgd\",\n",
    "              metrics=['acc'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.summary()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# filter is 3*3*3, 64 or them and bias term for each filter\n",
    "3*3*3*64+64"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# filter for second one\n",
    "32*3*3*64+32"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "history = model.fit(X_train,\n",
    "                    y_train,\n",
    "                    epochs=60,\n",
    "                    batch_size=50,\n",
    "                    validation_data=(X_val, y_val))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "hist_cnn = history.history\n",
    "loss_values = hist_cnn['loss']\n",
    "val_loss_values = hist_cnn['val_loss']\n",
    "\n",
    "epochs = range(1, len(loss_values) + 1)\n",
    "plt.plot(epochs, loss_values, 'g.', label='Training loss')\n",
    "plt.plot(epochs, val_loss_values, 'g', label='Validation loss')\n",
    "\n",
    "plt.title('Training and validation loss')\n",
    "plt.xlabel('Epochs')\n",
    "plt.ylabel('Loss')\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "acc_values = hist_cnn['acc'] \n",
    "val_acc_values = hist_cnn['val_acc']\n",
    "\n",
    "plt.plot(epochs, acc_values, 'r.', label='Training acc')\n",
    "plt.plot(epochs, val_acc_values, 'r', label='Validation acc')\n",
    "plt.title('Training and validation accuracy')\n",
    "plt.xlabel('Epochs')\n",
    "plt.ylabel('Accuracy')\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "results_train = model.evaluate(X_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "results_test = model.evaluate(X_test, y_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "results_train"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "results_test"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4. Drop-out regularization"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from numpy.random import seed\n",
    "seed(123)\n",
    "from tensorflow import set_random_seed\n",
    "set_random_seed(123)\n",
    "\n",
    "dropout = models.Sequential()\n",
    "dropout.add(layers.Conv2D(64, (3, 3), activation='relu', input_shape=(64, 64,  3)))\n",
    "dropout.add(layers.MaxPooling2D((2, 2)))\n",
    "dropout.add(layers.Conv2D(32, (3, 3), activation='relu'))\n",
    "dropout.add(layers.MaxPooling2D((2, 2)))\n",
    "\n",
    "dropout.add(layers.Flatten())\n",
    "dropout.add(layers.Dropout(0.5))\n",
    "dropout.add(layers.Dense(32, activation='relu'))\n",
    "dropout.add(layers.Dense(1, activation='sigmoid'))\n",
    "\n",
    "dropout.compile(loss='binary_crossentropy',\n",
    "              optimizer=\"sgd\",\n",
    "              metrics=['acc'])\n",
    "drop_model = dropout.fit(X_train,\n",
    "                    y_train,\n",
    "                    epochs=40,\n",
    "                    batch_size=50,\n",
    "                    validation_data=(X_val, y_val))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "hist_cnn_drop = drop_model.history\n",
    "loss_values = hist_cnn_drop['loss']\n",
    "val_loss_values = hist_cnn_drop['val_loss']\n",
    "\n",
    "epochs = range(1, len(loss_values) + 1)\n",
    "plt.plot(epochs, loss_values, 'g.', label='Training loss')\n",
    "plt.plot(epochs, val_loss_values, 'g', label='Validation loss')\n",
    "\n",
    "plt.title('Training and validation loss')\n",
    "plt.xlabel('Epochs')\n",
    "plt.ylabel('Loss')\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "acc_values = hist_cnn_drop['acc'] \n",
    "val_acc_values = hist_cnn_drop['val_acc']\n",
    "\n",
    "plt.plot(epochs, acc_values, 'r.', label='Training acc')\n",
    "plt.plot(epochs, val_acc_values, 'r', label='Validation acc')\n",
    "plt.title('Training and validation accuracy')\n",
    "plt.xlabel('Epochs')\n",
    "plt.ylabel('Accuracy')\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "results_train = dropout.evaluate(X_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "results_test = dropout.evaluate(X_test, y_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "results_train"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "results_test"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5. Try tweaking the model, the possibilities are endless!\n",
    "- add/remove layers\n",
    "- In some layers, change `padding = valid` and `padding = same`\n",
    "- change activation functions\n",
    "- change optimizer\n",
    "- change batch size\n",
    "- change patch dimensions (from 3 x 3 to 5 x 5)\n",
    "- If you have more time, try running everything on the bigger training set! Do results improve?"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}