Data Augmentation.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Data Augmentation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**About the data:** <br>\n",
    "The dataset contains 2 folders: yes and no which contains 253 Brain MRI Images. The folder yes contains 155 Brain MRI Images that are tumorous and the folder no contains 98 Brain MRI Images that are non-tumorous. You can find [here](https://www.kaggle.com/navoneel/brain-mri-images-for-brain-tumor-detection)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Since this is a small dataset, I used data augmentation in order to create more images."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Also, we could solve the data imbalance issue (since 61% of the data belongs to the tumorous class) using data augmentation."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Import Necessary Modules"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Using TensorFlow backend.\n"
     ]
    }
   ],
   "source": [
    "import tensorflow as tf\n",
    "from keras.preprocessing.image import ImageDataGenerator\n",
    "import cv2\n",
    "import imutils\n",
    "import matplotlib.pyplot as plt\n",
    "from os import listdir\n",
    "import time    \n",
    "\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Nicely formatted time string\n",
    "def hms_string(sec_elapsed):\n",
    "    h = int(sec_elapsed / (60 * 60))\n",
    "    m = int((sec_elapsed % (60 * 60)) / 60)\n",
    "    s = sec_elapsed % 60\n",
    "    return f\"{h}:{m}:{round(s,1)}\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "def augment_data(file_dir, n_generated_samples, save_to_dir):\n",
    "    \"\"\"\n",
    "    Arguments:\n",
    "        file_dir: A string representing the directory where images that we want to augment are found.\n",
    "        n_generated_samples: A string representing the number of generated samples using the given image.\n",
    "        save_to_dir: A string representing the directory in which the generated images will be saved.\n",
    "    \"\"\"\n",
    "    \n",
    "    #from keras.preprocessing.image import ImageDataGenerator\n",
    "    #from os import listdir\n",
    "    \n",
    "    data_gen = ImageDataGenerator(rotation_range=10, \n",
    "                                  width_shift_range=0.1, \n",
    "                                  height_shift_range=0.1, \n",
    "                                  shear_range=0.1, \n",
    "                                  brightness_range=(0.3, 1.0),\n",
    "                                  horizontal_flip=True, \n",
    "                                  vertical_flip=True, \n",
    "                                  fill_mode='nearest'\n",
    "                                 )\n",
    "\n",
    "    \n",
    "    for filename in listdir(file_dir):\n",
    "        # load the image\n",
    "        image = cv2.imread(file_dir + '\\\\' + filename)\n",
    "        # reshape the image\n",
    "        image = image.reshape((1,)+image.shape)\n",
    "        # prefix of the names for the generated sampels.\n",
    "        save_prefix = 'aug_' + filename[:-4]\n",
    "        # generate 'n_generated_samples' sample images\n",
    "        i=0\n",
    "        for batch in data_gen.flow(x=image, batch_size=1, save_to_dir=save_to_dir, \n",
    "                                           save_prefix=save_prefix, save_format='jpg'):\n",
    "            i += 1\n",
    "            if i > n_generated_samples:\n",
    "                break"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Remember that 61% of the data (155 images) are tumorous. And, 39% of the data (98 images) are non-tumorous.<br>\n",
    "So, in order to balance the data we can generate 9 new images for every image that belongs to 'no' class and 6 images for every image that belongs the 'yes' class.<br>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Elapsed time: 0:2:16.2\n"
     ]
    }
   ],
   "source": [
    "start_time = time.time()\n",
    "\n",
    "augmented_data_path = 'augmented data/'\n",
    "\n",
    "# augment data for the examples with label equal to 'yes' representing tumurous examples\n",
    "augment_data(file_dir=yes_path, n_generated_samples=6, save_to_dir=augmented_data_path+'yes')\n",
    "# augment data for the examples with label equal to 'no' representing non-tumurous examples\n",
    "augment_data(file_dir=no_path, n_generated_samples=9, save_to_dir=augmented_data_path+'no')\n",
    "\n",
    "end_time = time.time()\n",
    "execution_time = (end_time - start_time)\n",
    "print(f\"Elapsed time: {hms_string(execution_time)}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's see how many tumorous and non-tumorous examples after performing data augmentation:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "def data_summary(main_path):\n",
    "    \n",
    "    yes_path = main_path+'yes'\n",
    "    no_path = main_path+'no'\n",
    "        \n",
    "    # number of files (images) that are in the the folder named 'yes' that represent tumorous (positive) examples\n",
    "    m_pos = len(listdir(yes_path))\n",
    "    # number of files (images) that are in the the folder named 'no' that represent non-tumorous (negative) examples\n",
    "    m_neg = len(listdir(no_path))\n",
    "    # number of all examples\n",
    "    m = (m_pos+m_neg)\n",
    "    \n",
    "    pos_prec = (m_pos* 100.0)/ m\n",
    "    neg_prec = (m_neg* 100.0)/ m\n",
    "    \n",
    "    print(f\"Number of examples: {m}\")\n",
    "    print(f\"Percentage of positive examples: {pos_prec}%, number of pos examples: {m_pos}\") \n",
    "    print(f\"Percentage of negative examples: {neg_prec}%, number of neg examples: {m_neg}\") "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of examples: 2065\n",
      "Percentage of positive examples: 52.54237288135593%, number of pos examples: 1085\n",
      "Percentage of negative examples: 47.45762711864407%, number of neg examples: 980\n"
     ]
    }
   ],
   "source": [
    "data_summary(augmented_data_path)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "That's it for this notebook. Now, we can use the augmented data to train our convolutional neural network."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}