tf_data_with_tf_keras.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "tf.data with tf.keras.ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/Aayushktyagi/DeepLearning_Resources/blob/master/tf_data_with_tf_keras.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "07mAPLQ8yqc9",
        "colab_type": "text"
      },
      "source": [
        "The purpose of this notebook is two-fold:\n",
        "\n",
        "- Show the usage of `tf.data.Dataset.from_generator` along with Keras `ImageDataGenerator` for Keras models.\n",
        "- Compare the performance between `tf.data.Dataset.from_generator` and `ImageDtaGenerator`. \n",
        "\n",
        "A huge thanks to **Picsou Balthazar** for helping me out in this. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "PHp2tBzeAZT3",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Install TensorFlow 2.0 (GPU)\n",
        "!pip install tensorflow-gpu"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "UQ53B3_fA6CY",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Import the packages\n",
        "# Import the packages for DL stuff\n",
        "from tensorflow.keras.preprocessing.image import ImageDataGenerator\n",
        "from tensorflow.keras.applications import VGG16\n",
        "from tensorflow.keras.layers import Dropout\n",
        "from tensorflow.keras.layers import Flatten\n",
        "from tensorflow.keras.layers import Dense\n",
        "from tensorflow.keras.layers import Input\n",
        "from tensorflow.keras.models import Model\n",
        "from tensorflow.keras.optimizers import SGD\n",
        "from imutils import paths\n",
        "import matplotlib.pyplot as plt\n",
        "import tensorflow as tf\n",
        "import pandas as pd\n",
        "import numpy as np\n",
        "import time"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ArwTIOe7uBay",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "36539805-9f39-48e4-c6f6-62b6c2bc0805"
      },
      "source": [
        "# verify if the right version was installed\n",
        "tf.__version__"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "'2.0.0'"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 2
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "A6Qij-Z5BELC",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Get the flowers' dataset\n",
        "flowers = tf.keras.utils.get_file(\n",
        "    'flower_photos',\n",
        "    'https://storage.googleapis.com/download.tensorflow.org/example_images/flower_photos.tgz',\n",
        "    untar=True)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "d_6Lc1Yaze9d",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "c27cc451-953c-484b-a86e-c6aa33ca1cd8"
      },
      "source": [
        "!ls {flowers}"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "daisy  dandelion  LICENSE.txt  roses  sunflowers  tulips\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hJVAZAqxxzmP",
        "colab_type": "text"
      },
      "source": [
        "Know more about the dataset here: https://www.tensorflow.org/tutorials/load_data/images#load_using_tfdata. Following is the glimpse of the dataset.\n",
        "\n",
        "![](https://www.tensorflow.org/tutorials/load_data/images_files/output_suh6Sjv68rY3_0.png)"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "RvcluHUqBJ26",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Initialize the data augmentation object and set its mean to the\n",
        "# mean of the ImageNet dataset\n",
        "img_gen = tf.keras.preprocessing.image.ImageDataGenerator(rotation_range=20)\n",
        "mean = np.array([123.68, 116.779, 103.939], dtype=\"float32\")\n",
        "img_gen.mean = mean"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "qb8tvXmvBSjL",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "5165ce96-cbae-483f-be17-3c346d6e5f7b"
      },
      "source": [
        "# Wrap the generator with tf.data\n",
        "ds = tf.data.Dataset.from_generator(\n",
        "    lambda: img_gen.flow_from_directory(flowers,\n",
        "            class_mode=\"categorical\",\n",
        "            target_size=(224, 224),\n",
        "            color_mode=\"rgb\",\n",
        "            shuffle=True),\n",
        "    output_types=(tf.float32, tf.float32),\n",
        "    output_shapes = ([None,224,224,3],[None,5])\n",
        ")\n",
        "\n",
        "ds"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<DatasetV1Adapter shapes: ((None, 224, 224, 3), (None, 5)), types: (tf.float32, tf.float32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 11
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "3gsOo6vSBWNd",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 51
        },
        "outputId": "9359cfa1-afa3-4d2b-c2f4-2635f5039bf7"
      },
      "source": [
        "# Verify the shapes yielded by the data generator\n",
        "train_gen = img_gen.flow_from_directory(flowers,\n",
        "            class_mode=\"categorical\",\n",
        "            target_size=(224, 224),\n",
        "            color_mode=\"rgb\",\n",
        "            shuffle=True)\n",
        "images, labels = next(train_gen)\n",
        "images.shape, labels.shape"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Found 3670 images belonging to 5 classes.\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "((32, 224, 224, 3), (32, 5))"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 7
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "PHlIxEt_BZE_",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# A helper script for which would initialize and compile \n",
        "# our model\n",
        "def get_training_model():\n",
        "    baseModel = VGG16(weights=\"imagenet\", include_top=False,\n",
        "        input_tensor=Input(shape=(224, 224, 3)))\n",
        "\n",
        "    headModel = baseModel.output\n",
        "    headModel = Flatten(name=\"flatten\")(headModel)\n",
        "    headModel = Dense(512, activation=\"relu\")(headModel)\n",
        "    headModel = Dropout(0.5)(headModel)\n",
        "    headModel = Dense(5, activation=\"softmax\")(headModel)\n",
        "\n",
        "    model = Model(inputs=baseModel.input, outputs=headModel)\n",
        "\n",
        "    for layer in baseModel.layers:\n",
        "        layer.trainable = False\n",
        "\n",
        "    opt = SGD(lr=1e-4, momentum=0.9)\n",
        "    model.compile(loss=\"categorical_crossentropy\", optimizer=opt,\n",
        "        metrics=[\"accuracy\"])\n",
        "    return model"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ERLsOthaBj76",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Get the total number of images present in the\n",
        "# root dataset directory\n",
        "total_data = len(list(paths.list_images(flowers)))"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "M32wuUHvBwlU",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 238
        },
        "outputId": "c733a950-7a59-4c32-8cdf-57b54f13ec42"
      },
      "source": [
        "# Kickstart model training with tf.data\n",
        "model = get_training_model()\n",
        "start = time.time()\n",
        "model.fit(ds, \n",
        "         steps_per_epoch=total_data//32,\n",
        "         epochs=5)\n",
        "print(\"It took {} seconds\".format(time.time() - start))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Train for 114 steps\n",
            "Epoch 1/5\n",
            "Found 3670 images belonging to 5 classes.\n",
            "114/114 [==============================] - 80s 704ms/step - loss: 5.2687 - accuracy: 0.6757\n",
            "Epoch 2/5\n",
            "114/114 [==============================] - 82s 715ms/step - loss: 0.9821 - accuracy: 0.7883\n",
            "Epoch 3/5\n",
            "114/114 [==============================] - 79s 695ms/step - loss: 0.6616 - accuracy: 0.8219\n",
            "Epoch 4/5\n",
            "114/114 [==============================] - 79s 695ms/step - loss: 0.4871 - accuracy: 0.8535\n",
            "Epoch 5/5\n",
            "114/114 [==============================] - 80s 699ms/step - loss: 0.3722 - accuracy: 0.8821\n",
            "It took 399.8591330051422 seconds\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "yfDEGi7Zwvkr",
        "colab_type": "text"
      },
      "source": [
        "It took **399.85 seconds**. Let's now see how `ImageDataGenerator` performs. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "9dMLGo9Vw0LM",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 204
        },
        "outputId": "87ed4320-5d07-41ea-b6d5-42c4cb22dde8"
      },
      "source": [
        "# Kickstart model training with ImageDataGenerator\n",
        "model = get_training_model()\n",
        "start = time.time()\n",
        "model.fit_generator(train_gen, \n",
        "                   steps_per_epoch=total_data//32,\n",
        "                   epochs=5)\n",
        "print(\"It took {} seconds\".format(time.time() - start))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Epoch 1/5\n",
            "114/114 [==============================] - 144s 1s/step - loss: 5.5306 - accuracy: 0.5962\n",
            "Epoch 2/5\n",
            "114/114 [==============================] - 135s 1s/step - loss: 3.3431 - accuracy: 0.7526\n",
            "Epoch 3/5\n",
            "114/114 [==============================] - 135s 1s/step - loss: 2.7142 - accuracy: 0.7974\n",
            "Epoch 4/5\n",
            "114/114 [==============================] - 135s 1s/step - loss: 2.4073 - accuracy: 0.8120\n",
            "Epoch 5/5\n",
            "114/114 [==============================] - 135s 1s/step - loss: 2.0779 - accuracy: 0.8378\n",
            "It took 685.5545630455017 seconds\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "IlvOBeUOyQZl",
        "colab_type": "text"
      },
      "source": [
        "Takes **685.55 seconds**. "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pv6aB4K23Wnv",
        "colab_type": "text"
      },
      "source": [
        "Links to `tf.data` guides: \n",
        "- https://www.tensorflow.org/guide/data\n",
        "- https://www.tensorflow.org/guide/data_performance"
      ]
    }
  ]
}