Clasificacion_insectos_Grupo3.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Clasificacion_insectos_Grupo3.ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "toc_visible": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "a75aG3xBlMvY"
      },
      "source": [
        "##**Clasificación de imágenes de insectos**\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "LEiaI54xlodB"
      },
      "source": [
        "Armamos tres modelos que clasifican cuatro clases de imágenes: abejas, avispas, otros insectos y otros (animales, comida, personas)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3KZqC3CPWM6P"
      },
      "source": [
        "Tres modelos: \n",
        "\n",
        "Secuencial con data augmentation, dropout y regularización\n",
        "\n",
        "VGG19 con Imagenet\n",
        "\n",
        "ResNet50 con Imagenet"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_zoHKxn0lrjh"
      },
      "source": [
        "Dejamos una clase de otros objetos que no sean insectos para chequear si los modelos diferencian insectos que no son abejas ni avispas de otros objetos"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "MPqWwymjfc7n"
      },
      "source": [
        "%%bash\n",
        "mkdir ~/.kaggle\n",
        "mv kaggle.json ~/.kaggle/\n",
        "chmod 600 ~/.kaggle/kaggle.json\n",
        "cat ~/.kaggle/kaggle.json #credenciales de kaggle generadas con la API"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "agSdDdeCfyLP"
      },
      "source": [
        "!kaggle datasets download -d jerzydziewierz/bee-vs-wasp"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "kjVMvDeWhJQL"
      },
      "source": [
        "!unzip bee-vs-wasp.zip "
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "61R7G0EuhXV4"
      },
      "source": [
        "from pathlib import Path\n",
        "import pandas as pd\n",
        "import numpy as np\n",
        "import random\n",
        "import tensorflow as tf\n",
        "from tensorflow.keras.preprocessing.image import ImageDataGenerator\n",
        "from keras_preprocessing.image import ImageDataGenerator\n",
        "from keras.layers import Dense, Activation, Flatten, Dropout, BatchNormalization\n",
        "from keras.layers import Conv2D, MaxPooling2D\n",
        "from keras import regularizers, optimizers"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "o33t_hbxhemo"
      },
      "source": [
        "labels = pd.read_csv(\"kaggle_bee_vs_wasp/labels.csv\", dtype=str)\n",
        "labels\n",
        "#el dataset con los paths y las etiquetas de cada clase"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "L-cY8IfCfU9c"
      },
      "source": [
        "duplicados = labels[labels.duplicated()]\n",
        "print(duplicados) #Chequeamos no tener valores duplicados. No tenemos"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "1lC1DGnVinc2"
      },
      "source": [
        "import os\n",
        "labels.path = labels[\"path\"].str.replace(\"\\\\\",\"/\")\n",
        "labels.describe()\n",
        "#reemplazamos la doble barra en el nombre de archivos del dataset de etiquetas porque genera problemas para que flow from dataframe detecte el path"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "BvIxWwp5ByNF"
      },
      "source": [
        "labels.shape #contamos con 11421 datos"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "q_9gckP8BaK7"
      },
      "source": [
        "#visualizamos qué cantidad tenemos de cada clase\n",
        "import seaborn as sns\n",
        "import matplotlib.pyplot as plt\n",
        "\n",
        "plt.figure(figsize=(10,6))\n",
        "sns.set(style='white',context='notebook')\n",
        "sns.countplot(labels['label'],color=(0.09, 0.26, 0.42, 0));\n",
        "plt.title('clases'.title(),fontsize=15);\n",
        "plt.ylabel('count'.title(), fontsize=8)\n",
        "plt.xlabel('seccion'.title(), fontsize=8)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "cqG4vmHcOEnf"
      },
      "source": [
        "labels['label'].value_counts(1)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "uN-2h9aVa3Um"
      },
      "source": [
        "from sklearn.model_selection import train_test_split\n",
        "\n",
        "X=labels.drop(columns=['label','id'])\n",
        "y=labels[\"label\"]\n",
        "\n",
        "train, val = train_test_split(labels, test_size=.2, random_state=42)\n",
        "val, test = train_test_split(val, test_size=.5, random_state=42)\n",
        "print('train:', train.shape, '\\nval:', val.shape, '\\ntest',  test.shape)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3geGSsJzs7pT"
      },
      "source": [
        "## Probamos un modelo secuencial con data augmentation, regularización y dropout"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "-UcaV3B3v47L"
      },
      "source": [
        "#chequeamos valores que tenemos en train\n",
        "labels[\"label\"].value_counts()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "wG2rKoV0QZwh"
      },
      "source": [
        "#generador de imagenes nuevas\n",
        "directory_path=None\n",
        "batch_size=64\n",
        "generador_train = ImageDataGenerator(\n",
        "    rescale=1./255,\n",
        "    rotation_range=40,\n",
        "    width_shift_range=0.2,\n",
        "    height_shift_range=0.2,\n",
        "    shear_range=0.2,\n",
        "    zoom_range=0.2,\n",
        "    horizontal_flip=True,)\n",
        "\n",
        "val_datagen = ImageDataGenerator(rescale=1./255)\n",
        "\n",
        "train_generator = generador_train.flow_from_dataframe(\n",
        "            train,\n",
        "            directory=\"kaggle_bee_vs_wasp/\",\n",
        "            x_col=\"path\",\n",
        "            y_col=\"label\",\n",
        "            has_ext=False,\n",
        "            subset=\"training\",\n",
        "            batch_size=64,\n",
        "            seed=42,\n",
        "            clases=None,\n",
        "            shuffle=True,\n",
        "            class_mode=\"categorical\",\n",
        "            target_size=(150, 150),\n",
        "            color_mode=\"rgb\")\n",
        "\n",
        "validation_generator = val_datagen.flow_from_dataframe(\n",
        "            val,\n",
        "            directory=\"kaggle_bee_vs_wasp/\",\n",
        "            x_col=\"path\",\n",
        "            y_col=\"label\",\n",
        "            has_ext=False,\n",
        "            batch_size=64,\n",
        "            seed=42,\n",
        "            classes=None,\n",
        "            shuffle=False,\n",
        "            class_mode=\"categorical\",\n",
        "            target_size=(150, 150),\n",
        "            color_mode=\"rgb\") \n",
        "\n",
        "test_datagen=ImageDataGenerator(rescale=1./255.)\n",
        "test_g = test_datagen.flow_from_dataframe(test,\n",
        "            directory=\"kaggle_bee_vs_wasp/\",\n",
        "            x_col=\"path\",\n",
        "            y_col='label',\n",
        "            has_ext=False,\n",
        "            target_size=(150, 150),\n",
        "            class_mode=\"categorical\",\n",
        "            classes=None,\n",
        "            batch_size=64,\n",
        "            color_mode=\"rgb\",\n",
        "            seed=42,)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "LuzWbDoccIbZ"
      },
      "source": [
        "classes = list(train_generator.class_indices.keys())\n",
        "classes\n",
        "w = 10\n",
        "h = 10\n",
        "fig = plt.figure(figsize=(10, 15))\n",
        "columns = 8\n",
        "rows = 4\n",
        "ax = []\n",
        "x,y = train_generator.next()\n",
        "for i in range(columns*rows):\n",
        "    image = x[i]\n",
        "    label = classes[list(y[i]).index(1)]\n",
        "    ax.append( fig.add_subplot(rows, columns, i+1) )\n",
        "    ax[-1].set_title(label)\n",
        "    plt.imshow(image)\n",
        "plt.show()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "-DyGydTN1Z0F"
      },
      "source": [
        "classes"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Ko4gy7hFUnOn"
      },
      "source": [
        "from tensorflow.keras.callbacks import EarlyStopping, ReduceLROnPlateau, ModelCheckpoint\n",
        "from tensorflow.keras.layers import Dense, Dropout\n",
        "from tensorflow.keras import regularizers\n",
        "from google.colab import files\n",
        "from keras.models import load_model\n",
        "checkpoint_model3 = ModelCheckpoint('from_scratch/weights.{epoch:02d}-{val_loss:.2f}.h5', monitor='val_accuracy', verbose=0,\n",
        "                             save_best_only=False, save_weights_only=False, mode='auto')\n",
        "\n",
        "#files.download('model3.h5')\n",
        "\n",
        "reduce_lr=ReduceLROnPlateau(monitor='val_loss',patience=3,verbose=1)\n",
        "early_stopping=EarlyStopping(monitor='val_loss',min_delta=0.001,patience=4,restore_best_weights=True,verbose=1)\n",
        "\n",
        "callbacks=[reduce_lr, early_stopping, checkpoint_model3]\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "cKqfMt5_65C1"
      },
      "source": [
        "#Probamos una red con data augmentation + regularización y dropout en la última capa\n",
        "from tensorflow.keras.layers import Dense, Dropout\n",
        "from tensorflow.keras import regularizers\n",
        "from keras import layers\n",
        "from keras import models\n",
        "from keras import optimizers\n",
        "from tensorflow.keras.models import Sequential\n",
        "from tensorflow.keras.wrappers.scikit_learn import KerasClassifier\n",
        "from sklearn.model_selection import train_test_split\n",
        "from sklearn.metrics import accuracy_score\n",
        "\n",
        "dropout_rate=0.3\n",
        "model3 = models.Sequential()\n",
        "model3.add(layers.Conv2D(32, (3, 3), activation='relu',\n",
        "                        input_shape=(150, 150, 3)))\n",
        "model3.add(layers.MaxPooling2D((2, 2)))\n",
        "\n",
        "model3.add(layers.Conv2D(64, (3, 3), activation='relu'))\n",
        "model3.add(layers.MaxPooling2D((2, 2)))\n",
        "\n",
        "model3.add(layers.Conv2D(128, (3, 3), activation='relu'))\n",
        "model3.add(layers.MaxPooling2D((2, 2)))\n",
        "\n",
        "model3.add(layers.Conv2D(128, (3, 3), activation='relu'))\n",
        "model3.add(layers.MaxPooling2D((2, 2)))\n",
        "\n",
        "model3.add(layers.Flatten())\n",
        "model3.add(layers.Dense(512, activation='relu',kernel_regularizer=regularizers.l1(0.001)))\n",
        "model3.add(Dropout(dropout_rate))\n",
        "model3.add(layers.Dense(4, activation='softmax'))\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Tf81Xjbd8T1S"
      },
      "source": [
        "model3.summary()\n",
        "model3.save('model3.h5')"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "_9FyqdW88ZKg"
      },
      "source": [
        "model3.compile(loss='categorical_crossentropy',\n",
        "              optimizer='adam',\n",
        "              metrics=['accuracy'])"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "qUdgm7gwdd3C"
      },
      "source": [
        "#Antes de fitear vamos a equilibrar las clases \n",
        "from sklearn import preprocessing\n",
        "lb_classes = preprocessing.LabelBinarizer()\n",
        "\n",
        "# Ajustamos a la variable labels\n",
        "lb_classes.fit(labels['label'])\n",
        "lb_classes.classes_"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "VX7Yl0pIekRd"
      },
      "source": [
        "# Encodeamos las labels\n",
        "labels_ohe = pd.DataFrame(lb_classes.transform(labels['label']), columns=lb_classes.classes_, index=labels['id'])\n",
        "labels_ohe.head()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "offZp0EEeJOg"
      },
      "source": [
        "class_weight = {k:v for k,v in zip(range(len(lb_classes.classes_)),\n",
        "                                   (1/labels_ohe.iloc[train.index].sum()) * 100)}\n",
        "\n",
        "class_weight"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "dXNjFBYsfKti"
      },
      "source": [
        "# Visualizamos los pesos definidos\n",
        "((1/labels_ohe.iloc[train.index].sum()) * 100).plot(kind='bar', color=[\"#0e2a47\"]);"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Csn_P4QK8lqu"
      },
      "source": [
        "history3 = model3.fit(\n",
        "      train_generator,\n",
        "      steps_per_epoch=train_generator.n//train_generator.batch_size, #train / batch size. \n",
        "      epochs=20,\n",
        "      validation_data=validation_generator,\n",
        "      validation_steps=validation_generator.n//validation_generator.batch_size,\n",
        "      class_weight=class_weight,\n",
        "      callbacks=callbacks)\n",
        "#fiteamos el modelo con data augmentation"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "jZKRFZwcwt7Q"
      },
      "source": [
        "from keras.utils.vis_utils import plot_model\n",
        "plot_model(model3, show_shapes=True)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "EqSmUqX3WL_2"
      },
      "source": [
        "from keras.models import load_model\n",
        "model3.save('model3.h5')\n",
        "#model3 = load_model('model3.h5')"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "5diklsW2IPnV"
      },
      "source": [
        "acc3=history3.history['accuracy']\n",
        "val_loss3=history3.history['val_loss']\n",
        "loss3=history3.history['loss']\n",
        "val_acc3=history3.history['val_accuracy']\n",
        "epochs3 = range(1, len(acc3) + 1)\n",
        "\n",
        "plt.figure(figsize=(20,8))\n",
        "plt.title('Modelo de base')\n",
        "plt.plot(epochs3,loss3)\n",
        "plt.plot(epochs3,val_loss3)\n",
        "plt.xticks(ticks=epochs3)\n",
        "plt.ylabel('Loss')\n",
        "plt.xlabel('Epochs')\n",
        "plt.legend(['Training loss','Validation Loss'])\n",
        "\n",
        "plt.figure(figsize=(20,8))\n",
        "plt.plot(epochs3,acc3)\n",
        "plt.plot(epochs3,val_acc3)\n",
        "plt.xticks(ticks=epochs3)\n",
        "plt.xlabel('Epochs')\n",
        "plt.ylabel('Accuracy')\n",
        "plt.legend(['Training Accuracy','Validation Accuracy']);"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "MfRelA_G1KCb"
      },
      "source": [
        "model3.evaluate(test_g)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5tF5Wd39sPTi"
      },
      "source": [
        "##Probamos con VGG19"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "BLzxoxYlsVaM"
      },
      "source": [
        "from keras.applications import VGG19\n",
        "\n",
        "conv_base = VGG19(weights='imagenet',\n",
        "                  include_top=False,\n",
        "                  input_shape=(150, 150, 3))\n",
        "\n",
        "\n",
        "conv_base.summary()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "YhCneFhVsa7h"
      },
      "source": [
        "#Creamos train y test para vgg19 con el preprocesamiento que ya existe\n",
        "from tensorflow.keras.applications.vgg16 import preprocess_input\n",
        "\n",
        "train_datagen_vgg = ImageDataGenerator(\n",
        "    preprocessing_function=preprocess_input,\n",
        "    horizontal_flip=True\n",
        ")\n",
        "\n",
        "val_datagen_vgg = ImageDataGenerator(preprocessing_function=preprocess_input)\n",
        "\n",
        "batch_size = 20\n",
        "\n",
        "       # Train \n",
        "train_vgg = train_datagen_vgg.flow_from_dataframe(train,\n",
        "            directory=\"kaggle_bee_vs_wasp/\",\n",
        "            x_col=\"path\",\n",
        "            y_col=\"label\",\n",
        "            has_ext=False,\n",
        "            subset=\"training\",\n",
        "            batch_size=batch_size,\n",
        "            classes=None,\n",
        "            seed=42,\n",
        "            shuffle=True,\n",
        "            class_mode=\"categorical\",\n",
        "            target_size=(150, 150))\n",
        "\n",
        "\n",
        "        # Validacion\n",
        "val_vgg = val_datagen_vgg.flow_from_dataframe(val,\n",
        "            directory=\"kaggle_bee_vs_wasp/\",\n",
        "            x_col=\"path\",\n",
        "            y_col=\"label\",\n",
        "            has_ext=False,\n",
        "            batch_size=batch_size,\n",
        "            seed=42,\n",
        "            classes=None,\n",
        "            shuffle=True,\n",
        "            class_mode=\"categorical\",\n",
        "            target_size=(150, 150))    \n",
        "\n",
        "        # Test\n",
        "test_datagen_vgg=ImageDataGenerator(preprocessing_function=preprocess_input)\n",
        "test_vgg = test_datagen_vgg.flow_from_dataframe(test,\n",
        "            directory=\"kaggle_bee_vs_wasp/\",\n",
        "            x_col=\"path\",\n",
        "            y_col='label',\n",
        "            has_ext=False,\n",
        "            target_size=(150, 150),\n",
        "            class_mode=\"categorical\",\n",
        "            classes=None,\n",
        "            batch_size=batch_size,\n",
        "            color_mode=\"rgb\",\n",
        "            seed=42,\n",
        "            shuffle=False)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Iw71kSsn0y7L"
      },
      "source": [
        "from tensorflow.keras.layers import Input, ZeroPadding2D, Conv2D, BatchNormalization, Activation, MaxPooling2D, GlobalAveragePooling2D, Add, Dense,Flatten\n",
        "from tensorflow.keras.applications import ResNet50\n",
        "from tensorflow.keras.models import Model\n",
        "from tensorflow.keras.initializers import he_normal\n",
        "\n",
        "x_input_vgg = Input(shape=(150, 150, 3))\n",
        "# Instanciamos la base convolucional de VGG19 pre-entrenada en ImageNet\n",
        "vgg19_pretrained = VGG19(\n",
        "    include_top=False,\n",
        "    weights='imagenet',\n",
        "    pooling='avg',\n",
        "    input_shape=((150, 150, 3)),\n",
        ")\n",
        "\n",
        "# Freezamos los pesos de la base convolucional\n",
        "vgg19_pretrained.trainable = False"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "8xe0RfKm1V75"
      },
      "source": [
        "# La base convolucional que procesa los datos de entrada\n",
        "d = vgg19_pretrained(x_input_vgg)\n",
        "d = Dropout(0.5)(d)\n",
        "# La capa que procesa la salida\n",
        "d = Dense(4, activation='softmax', name='ResNet_Bee_Wasp')(d)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "1LEvkGMquOHN"
      },
      "source": [
        "# Instanciamos el modelo\n",
        "vgg19_pretrained = Model(inputs=x_input_vgg, outputs=d, name='VGG19')"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "WItzhQPp1lPw"
      },
      "source": [
        "# Observamos el summary\n",
        "vgg19_pretrained.summary()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "JeZJQ_mWsa4a"
      },
      "source": [
        "from tensorflow.keras.utils import plot_model\n",
        "plot_model(vgg19_pretrained, show_shapes=True)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "2hr9EWyesa0r"
      },
      "source": [
        "reduce_lr = ReduceLROnPlateau(monitor='val_loss', factor=0.1, patience=2, verbose=1)\n",
        "early_stopping = EarlyStopping(monitor='val_loss', min_delta=0.01, patience=5, restore_best_weights=True, verbose=1)\n",
        "\n",
        "callbacks_list2=[reduce_lr,early_stopping]"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "yN4kbGozul7L"
      },
      "source": [
        "vgg19_pretrained.compile(loss='categorical_crossentropy',optimizer=optimizers.RMSprop(lr=1e-4),\n",
        "              metrics=['acc'])"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "lbCnDAaAurPl"
      },
      "source": [
        "history2 = vgg19_pretrained.fit(train_vgg,\n",
        "      steps_per_epoch=train_vgg.n//train_vgg.batch_size, #train / batch size. \n",
        "      epochs=30,\n",
        "      validation_data=val_vgg,\n",
        "      validation_steps=val_vgg.n//val_vgg.batch_size,\n",
        "      callbacks=callbacks_list2)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "bCIJLllbvX6_"
      },
      "source": [
        "acc = history2.history['acc']\n",
        "val_acc = history2.history['val_acc']\n",
        "loss = history2.history['loss']\n",
        "val_loss = history2.history['val_loss']\n",
        "epochs = range(1, len(acc) + 1)\n",
        "import matplotlib.pyplot as plt\n",
        "\n",
        "plt.figure(figsize=(12,8))\n",
        "plt.plot(epochs, acc, label='Training acc')\n",
        "plt.plot(epochs, val_acc, label='Validation acc')\n",
        "plt.title('Training and validation accuracy')\n",
        "plt.legend()\n",
        "\n",
        "plt.figure(figsize=(12,8))\n",
        "plt.plot(epochs, loss, label='Training loss')\n",
        "plt.plot(epochs, val_loss, label='Validation loss')\n",
        "plt.title('Training and validation loss')\n",
        "plt.legend()\n",
        "plt.show()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "QL4alpaLyfJE",
        "outputId": "09c6ac35-5bd3-4aea-9b7a-8a7d25488d81"
      },
      "source": [
        "vgg19_pretrained.evaluate(test_vgg)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "58/58 [==============================] - 6s 97ms/step - loss: 0.4337 - acc: 0.8959\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[0.43369823694229126, 0.8958880305290222]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 31
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "agG0fUE9roE5"
      },
      "source": [
        "##Usamos ResNet50 con imagenet"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "DBMurba_X0_j"
      },
      "source": [
        "#ResNet50\n",
        "from keras import layers\n",
        "from keras import models\n",
        "from keras import utils\n",
        "from keras import optimizers\n",
        "from tensorflow.keras.wrappers.scikit_learn import KerasClassifier\n",
        "from tensorflow.keras.layers import Dense, Dropout\n",
        "from tensorflow.keras import regularizers\n",
        "from keras.applications import VGG19\n",
        "from keras.preprocessing import image\n",
        "from tensorflow.keras.applications import ResNet50\n",
        "from tensorflow.keras.models import Model\n",
        "from tensorflow.keras.layers import Input, ZeroPadding2D, Conv2D, BatchNormalization, Activation, MaxPooling2D, GlobalAveragePooling2D, Add, Dense,Flatten\n",
        "from tensorflow.keras.initializers import he_normal"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "_1jt2mwth1D9"
      },
      "source": [
        "x_input = Input(shape=(224, 224, 3)) #definimos el input"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "a2RJ_2e3hyJ_"
      },
      "source": [
        "# Instanciamos la base convolucional de ResNet50 pre-entrenada en ImageNet\n",
        "resnet50_pretrained = ResNet50(\n",
        "    include_top=False,\n",
        "    weights='imagenet',\n",
        "    pooling='avg',\n",
        "    input_shape=((224, 224, 3)),\n",
        ")"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "n-_DdH0kWo6N"
      },
      "source": [
        "for layer in resnet50_pretrained.layers:\n",
        "    print(layer.name)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "kU9Y_ajehv8J"
      },
      "source": [
        "# Freezamos los pesos de la base convolucional\n",
        "resnet50_pretrained.trainable = False"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "GrjUFITBcxYc"
      },
      "source": [
        "x = resnet50_pretrained(x_input)\n",
        "x = Dropout(0.3)(x)\n",
        "x = Dense(4, activation='softmax', name='ResNet_Bee_Wasp')(x)\n",
        "\n",
        "# Instanciamos el modelo\n",
        "resnet50_pretrained = Model(inputs=x_input, outputs=x, name='ResNet50')"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "t7EXdH6Nc7H6"
      },
      "source": [
        "# Observamos el summary\n",
        "resnet50_pretrained.summary()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ekQ6QTkWc79l"
      },
      "source": [
        "plot_model(resnet50_pretrained, show_shapes=True)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Gb1QgtBmc8IU"
      },
      "source": [
        "from tensorflow.keras.applications.resnet50 import preprocess_input\n",
        "\n",
        "train_datagen_resnet = ImageDataGenerator(\n",
        "    # Incluimos el preprocesamiento propio de ResNet50 en nuestro generador\n",
        "    preprocessing_function=preprocess_input,\n",
        "    horizontal_flip=True\n",
        ")\n",
        "\n",
        "val_datagen_resnet = ImageDataGenerator(preprocessing_function=preprocess_input)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Eg8ddVIIc77G"
      },
      "source": [
        "#Creamos diferentes variables para generadores de train, valudacion test para cada modelo\n",
        "\n",
        "train_generator50 = train_datagen_resnet.flow_from_dataframe(train,\n",
        "            directory=\"kaggle_bee_vs_wasp/\",\n",
        "            x_col=\"path\",\n",
        "            y_col=\"label\",\n",
        "            has_ext=False,\n",
        "            subset=\"training\",\n",
        "            batch_size=32,\n",
        "            seed=42,\n",
        "            clases=None,\n",
        "            shuffle=True,\n",
        "            class_mode=\"categorical\",\n",
        "            target_size=(224, 224),\n",
        "            color_mode=\"rgb\")\n",
        "\n",
        "val_generator50 = val_datagen_resnet.flow_from_dataframe(val,\n",
        "            directory=\"kaggle_bee_vs_wasp/\",\n",
        "            x_col=\"path\",\n",
        "            y_col=\"label\",\n",
        "            has_ext=False,\n",
        "            batch_size=32,\n",
        "            seed=42,\n",
        "            classes=None,\n",
        "            shuffle=False,\n",
        "            class_mode=\"categorical\",\n",
        "            target_size=(224, 224),\n",
        "            color_mode=\"rgb\") \n",
        "\n",
        "test_datagen_resnet=ImageDataGenerator(preprocessing_function=preprocess_input)\n",
        "test_generator50 = test_datagen_resnet.flow_from_dataframe(test,\n",
        "            directory=\"kaggle_bee_vs_wasp/\",\n",
        "            x_col=\"path\",\n",
        "            y_col='label',\n",
        "            has_ext=False,\n",
        "            target_size=(224, 224),\n",
        "            class_mode=\"categorical\",\n",
        "            classes=None,\n",
        "            batch_size=32,\n",
        "            color_mode=\"rgb\",\n",
        "            seed=42,\n",
        "            shuffle=False)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Ea90kYTQdElD"
      },
      "source": [
        "resnet50_pretrained.compile(optimizer='adam',\n",
        "                            loss='categorical_crossentropy',\n",
        "                            metrics=['acc'])"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "7qX-cXl0dGoU"
      },
      "source": [
        "# Instanciamos early_stopping\n",
        "early_stopping = EarlyStopping(monitor='val_loss', min_delta=0.001, patience=10, restore_best_weights=True, verbose=1)\n",
        "\n",
        "# Instanciamos chekpoint\n",
        "checkpoint = ModelCheckpoint('pretrained/weights.{epoch:02d}-{val_loss:.2f}.h5', monitor='val_loss', verbose=0,\n",
        "                             save_best_only=True, save_weights_only=False, mode='auto')\n",
        "\n",
        "es = [early_stopping]"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "m-OW6Am5dIMc"
      },
      "source": [
        "# Entrenamos la capa densa de nuestro modelo pre-entrenado\n",
        "history_resnet = resnet50_pretrained.fit(train_generator50,\n",
        "                                  steps_per_epoch=train_generator50.n//train_generator50.batch_size,\n",
        "                                  epochs=30,\n",
        "                                  validation_data=val_generator50,\n",
        "                                  validation_steps=val_generator50.n//val_generator50.batch_size,\n",
        "                                  callbacks=es)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "l-rSiIm7dIJc"
      },
      "source": [
        "acc_r= history_resnet.history['acc']\n",
        "val_acc_r = history_resnet.history['val_acc']\n",
        "loss_r = history_resnet.history['loss']\n",
        "val_loss_r = history_resnet.history['val_loss']\n",
        "epochs_r = range(1, len(acc_r) + 1)\n",
        "import matplotlib.pyplot as plt\n",
        "\n",
        "plt.figure(figsize=(12,8))\n",
        "plt.plot(epochs_r, acc_r, label='Training acc')\n",
        "plt.plot(epochs_r, val_acc_r, label='Validation acc')\n",
        "plt.title('Training and validation accuracy')\n",
        "plt.legend()\n",
        "\n",
        "plt.figure(figsize=(12,8))\n",
        "plt.plot(epochs_r, loss_r, label='Training loss')\n",
        "plt.plot(epochs_r, val_loss_r, label='Validation loss')\n",
        "plt.title('Training and validation loss')\n",
        "plt.legend()\n",
        "plt.show()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "5_UOoqkKdIGw"
      },
      "source": [
        "resnet50_pretrained.evaluate(test_generator50)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "_n2A1m-ufRL4"
      },
      "source": [
        "# Predecimos\n",
        "val_pred_resnet = resnet50_pretrained.predict(val_generator50)\n",
        "\n",
        "# Obtenemos la matriz de etiquetas reales de validación\n",
        "val_labels_resnet = lb_classes.transform(val['label'])\n",
        "\n",
        "# Evaluamos las shapes de los arrays\n",
        "val_pred_resnet.shape, val_labels_resnet.shape"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ytuQpIs2weVQ"
      },
      "source": [
        "#generar best_threshold\n",
        "threshold = np.arange(0.1, 0.9, 0.05)\n",
        "\n",
        "# Definimos listas vacíos donde volcamos los resultados\n",
        "acc_ = []\n",
        "accs = []\n",
        "best_threshold = np.zeros(val_pred_resnet.shape[1])\n",
        "\n",
        "# Iteramos sobre las distintas clases\n",
        "for i in range(len(best_threshold)):\n",
        "    # Indexamos las probabilidades predichas por clase\n",
        "    y_prob = np.array(val_pred_resnet[:, i])\n",
        "    \n",
        "    # Iteramos sobre los posibles umbrales\n",
        "    for j in threshold:\n",
        "        # Binarizamos las predicciones de acuerdo al umbral\n",
        "        y_pred = [1 if prob >= j else 0 for prob in y_prob]\n",
        "        y_pred = np.array(y_pred)\n",
        "        # Calculamos accuracy\n",
        "        acc = accuracy_score(val_labels_resnet[:, i], y_pred)\n",
        "        acc_.append(acc)\n",
        "    \n",
        "    # Volcamos los resultados en las listas generales\n",
        "    acc_  = np.array(acc_)\n",
        "    index = np.where(acc_==acc_.max())\n",
        "    accs.append(acc_.max())\n",
        "    best_threshold[i] = threshold[index[0][0]]\n",
        "    # Vaciamos la lista de para volver a iterar\n",
        "    acc_ = []"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "HbBfO7OtxIN4"
      },
      "source": [
        "accs"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "jtf4-otLxI2t"
      },
      "source": [
        "best_threshold"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "U2mMAKaSxV9j"
      },
      "source": [
        "y_pred_resnet = np.array([[1 if val_pred_resnet[i, j] >= best_threshold[j] else 0 for j in range(val_labels_resnet.shape[1])] for i in range(len(val_labels_resnet))])"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "J1nGQw52ACAS"
      },
      "source": [
        "from sklearn.metrics import accuracy_score, precision_score, recall_score, f1_score, confusion_matrix\n",
        "\n",
        "# Definimos listas vacías para ir volcando las métricas por clase\n",
        "accuracy_per_class = []\n",
        "precision_per_class = []\n",
        "recall_per_class = []\n",
        "f1_per_class = []\n",
        "\n",
        "# Definimos un listado de etiquetas de clase\n",
        "classes = list(lb_classes.classes_)\n",
        "\n",
        "for i in range(val_labels_resnet.shape[1]):\n",
        "    # Computamos las métricas de evaluación por clase y hacemos un append a la lista correspondiente\n",
        "    accuracy_per_class.append(accuracy_score(val_labels_resnet[:, i], (val_pred_resnet[:, i] > 0.5).astype(int)).round(2))\n",
        "    precision_per_class.append(precision_score(val_labels_resnet[:,  i], (val_pred_resnet[:, i] > 0.5).astype(int)).round(2))\n",
        "    recall_per_class.append(recall_score(val_labels_resnet[:,  i], (val_pred_resnet[:, i] > 0.5).astype(int)).round(2))\n",
        "    f1_per_class.append(f1_score(val_labels_resnet[:,  i], (val_pred_resnet[:, i] > 0.5).astype(int)).round(2))\n",
        "    \n",
        "    print(classes[i])\n",
        "    print('Accuracy:', accuracy_per_class[i], 'Precision:', precision_per_class[i],\n",
        "          'Recall:', recall_per_class[i], 'F1:', f1_per_class[i])\n",
        "from sklearn.metrics import confusion_matrix\n",
        "# Convert predictions classes to one hot vectors \n",
        "Y_pred_classes = np.argmax(val_pred_resnet,axis = 1) \n",
        "# Convert validation observations to one hot vectors\n",
        "Y_true = np.argmax(val_labels_resnet,axis = 1) \n",
        "# compute the confusion matrix\n",
        "cm = confusion_matrix(Y_true, Y_pred_classes) \n",
        "ax= plt.subplot()\n",
        "sns.heatmap(cm, annot=True, ax = ax); #annot=True to annotate cells\n",
        "\n",
        "# labels, title and ticks\n",
        "ax.set_xlabel('Predicted labels');ax.set_ylabel('True labels'); \n",
        "ax.set_title('Confusion Matrix'); \n",
        "ax.xaxis.set_ticklabels(classes); ax.yaxis.set_ticklabels(classes);\n",
        "  #matrices de confusión de cada clase sobre el set de validación\n",
        "  #Lo que mejor clasifica el modelo son otros objetos que nos son insectos\n",
        "  #de la clase insectos clasifica más falsos negativos que falsos positivos"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "-8CybPxNfA0E"
      },
      "source": [
        "# Predecimos\n",
        "\n",
        "test_pred_resnet = resnet50_pretrained.predict(test_generator50)\n",
        "\n",
        "# Tomamos los umbrales óptimos obtenidos en validación para mapear probabilidades a etiquetas de clase\n",
        "y_pred_resnet = np.array([[1 if test_pred_resnet[i, j] >= best_threshold[j] else 0 for j in range(test_pred_resnet.shape[1])] for i in range(len(test_pred_resnet))])\n",
        "\n",
        "# Creamos un DF de y_pred\n",
        "y_pred_resnet = pd.DataFrame(y_pred_resnet, index=test.index)\n",
        "\n",
        "# Evaluamos las shapes de los arrays resultantes\n",
        "y_pred_resnet.shape\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "fS9qZZ4VdOYx"
      },
      "source": [
        "import time\n",
        "from IPython.display import clear_output\n",
        "from tensorflow.keras.preprocessing.image import load_img, img_to_array\n",
        "data_dir= \"/content/kaggle_bee_vs_wasp\"\n",
        "\n",
        "bichos = list()\n",
        "n = 5\n",
        "fig, axes = plt.subplots(1, n, figsize=(25,10))\n",
        "\n",
        "for i in range(n):\n",
        "    random_bichos = np.random.choice(test['path'])\n",
        "    while random_bichos in bichos:\n",
        "        random_bichos = np.random.choice(test['path'])\n",
        "    bichos.append(random_bichos)\n",
        "\n",
        "    # Imagen original\n",
        "    random_bichos_path = os.path.join(data_dir, random_bichos)\n",
        "    test_image = load_img(random_bichos_path, target_size=(224, 224))\n",
        "    test_image = img_to_array(test_image)\n",
        " \n",
        "    # Predicción\n",
        "    prediction = y_pred_resnet.loc[test.loc[test['path'] == random_bichos].index[0]]\n",
        "    prediction_probability=np.amax(prediction)\n",
        "    predicted_labels = (prediction > 0.50).astype(int)\n",
        "    # Etiquetas\n",
        "    labels = lb_classes.classes_[predicted_labels > 0]\n",
        "\n",
        "    # Visualizamos la imagen y su predicción\n",
        "    title = \"{}\\n Etiquetas reales = {}\\nEtiquetas predichas = {}\\nProbabilidad = {}%\" \\\n",
        "                .format(test.loc[test['path'] == random_bichos, 'path'].values[0],\n",
        "                        test.loc[test['path'] == random_bichos, 'label'].values[0],\n",
        "                        labels, prediction_probability*100 if prediction_probability>0.5 else (1-prediction_probability)*100)\n",
        "\n",
        "    axes[i].imshow(plt.imread(os.path.join(data_dir, random_bichos)))\n",
        "    axes[i].set_title(title)\n",
        "    axes[i].axis('off')\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "hj8WqfxshV8d"
      },
      "source": [
        "#creamos variables para almacenar test y train acc de cada modelo\n",
        "train_model3=model3.evaluate(train_generator)\n",
        "test_model3=model3.evaluate(test_g)\n",
        "\n",
        "result_train_vgg=vgg19_pretrained.evaluate(train_vgg)\n",
        "result_test_vgg=vgg19_pretrained.evaluate(test_vgg)\n",
        "\n",
        "train_resnet=resnet50_pretrained.evaluate(train_generator50)\n",
        "test_resnet=resnet50_pretrained.evaluate(test_generator50)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "AM4i9df5hMzp"
      },
      "source": [
        "#Creamos un DataFrame para mostrar los scores de cada modelo\n",
        "df_Results=pd.DataFrame(columns=['Model','Train Loss','Test Loss'])\n",
        "\n",
        "df_Results.loc[1,'Model']='Modelo 1'\n",
        "df_Results.loc[1,'Train Acc']=train_model3[0]\n",
        "df_Results.loc[1,'Test Acc']=test_model3[0]\n",
        "\n",
        "df_Results.loc[2,'Model']='VGG19'\n",
        "df_Results.loc[2,'Train Acc']=result_train_vgg[0]\n",
        "df_Results.loc[2,'Test Acc']=result_test_vgg[0]\n",
        "\n",
        "df_Results.loc[3,'Model']='Resnet 50'\n",
        "df_Results.loc[3,'Train Acc']=train_resnet[0]\n",
        "df_Results.loc[3,'Test Acc']=test_resnet[0]\n",
        "\n",
        "df_Results"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "vvUv76orB5Pp"
      },
      "source": [
        "#Creamos un DataFrame para mostrar los scores de cada modelo\n",
        "df_Results=pd.DataFrame(columns=['Model','Train Acc','Test Acc'])\n",
        "\n",
        "df_Results.loc[1,'Model']='Modelo 1'\n",
        "df_Results.loc[1,'Train Loss']=train_model3[1]\n",
        "df_Results.loc[1,'Test Loss']=test_model3[1]\n",
        "\n",
        "df_Results.loc[2,'Model']='VGG19'\n",
        "df_Results.loc[2,'Train Loss']=result_train_vgg[1]\n",
        "df_Results.loc[2,'Test Loss']=result_test_vgg[1]\n",
        "\n",
        "df_Results.loc[3,'Model']='Resnet 50'\n",
        "df_Results.loc[3,'Train Loss']=train_resnet[1]\n",
        "df_Results.loc[3,'Test Loss']=test_resnet[1]\n",
        "\n",
        "df_Results"
      ],
      "execution_count": null,
      "outputs": []
    }
  ]
}