Neural_Network_from_scratch.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_SLSg7j1_Xrh"
      },
      "source": [
        "<h2 style=\"color:green\" align=\"center\">Implement Neural Network (or Logistic Regression) From Scratch</h2>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5-3SwJFX_Xrk"
      },
      "source": [
        "<h4 style=\"color:blue\">Predicting if a person would buy life insurnace based on his age using logistic regression</h4>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "tmQJOKB8_Xrk"
      },
      "source": [
        "Above is a binary logistic regression problem as there are only two possible outcomes (i.e. if person buys insurance or he/she doesn't)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "GE9v57lg_Xrl"
      },
      "outputs": [],
      "source": [
        "import numpy as np\n",
        "import tensorflow as tf\n",
        "from tensorflow import keras\n",
        "import pandas as pd\n",
        "from matplotlib import pyplot as plt\n",
        "%matplotlib inline"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "sjazIwJL_Xrl",
        "outputId": "2d213e90-adeb-48bb-9f13-15bafe9ca6f9"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>age</th>\n",
              "      <th>affordibility</th>\n",
              "      <th>bought_insurance</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>22</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>25</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>47</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>52</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>46</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "   age  affordibility  bought_insurance\n",
              "0   22              1                 0\n",
              "1   25              0                 0\n",
              "2   47              1                 1\n",
              "3   52              0                 0\n",
              "4   46              1                 1"
            ]
          },
          "execution_count": 2,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "df = pd.read_csv(\"insurance_data.csv\")\n",
        "df.head()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2BC2aTiE_Xrm"
      },
      "source": [
        "**Split train and test set**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "z8iVl7OC_Xrn"
      },
      "outputs": [],
      "source": [
        "from sklearn.model_selection import train_test_split\n",
        "X_train, X_test, y_train, y_test = train_test_split(df[['age','affordibility']],df.bought_insurance,test_size=0.2, random_state=25)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "U25PXMqu_Xrn"
      },
      "source": [
        "**Preprocessing: Scale the data so that both age and affordibility are in same scaling range**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "D1Bt6gMw_Xrn"
      },
      "outputs": [],
      "source": [
        "X_train_scaled = X_train.copy()\n",
        "X_train_scaled['age'] = X_train_scaled['age'] / 100\n",
        "\n",
        "X_test_scaled = X_test.copy()\n",
        "X_test_scaled['age'] = X_test_scaled['age'] / 100"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xHzQxkTs_Xrn"
      },
      "source": [
        "**Model Building: First build a model in keras/tensorflow and see what weights and bias values it comes up with. We will than try to reproduce same weights and bias in our plain python implementation of gradient descent. Below is the architecture of our simple neural network**"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WOTvG5Tz_Xrn"
      },
      "source": [
        "<img src=\"nn.jpg\" height=800 width=800/>"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "vpNMowOh_Xro",
        "outputId": "6ede0833-46e4-4193-f4f7-6f956ab9415b"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Epoch 1/5000\n",
            "1/1 [==============================] - 0s 0s/step - loss: 0.7113 - accuracy: 0.5000\n",
            "Epoch 2/5000\n",
            "1/1 [==============================] - 0s 15ms/step - loss: 0.7110 - accuracy: 0.5000\n",
            "Epoch 3/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.7106 - accuracy: 0.5000\n",
            "Epoch 4/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.7102 - accuracy: 0.5000\n",
            "Epoch 5/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.7098 - accuracy: 0.5000\n",
            "Epoch 6/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.7094 - accuracy: 0.5000\n",
            "Epoch 7/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.7091 - accuracy: 0.5000\n",
            "Epoch 8/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.7087 - accuracy: 0.5000\n",
            "Epoch 9/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.7083 - accuracy: 0.5000\n",
            "Epoch 10/5000\n",
            "1/1 [==============================] - 0s 0s/step - loss: 0.7079 - accuracy: 0.5000\n",
            "Epoch 11/5000\n",
            "1/1 [==============================] - 0s 3ms/step - loss: 0.7076 - accuracy: 0.5000\n",
            "Epoch 12/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.7072 - accuracy: 0.5000\n",
            "Epoch 13/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.7068 - accuracy: 0.5000\n",
            "Epoch 14/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.7065 - accuracy: 0.5000\n",
            "Epoch 15/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.7061 - accuracy: 0.5000\n",
            "Epoch 16/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.7057 - accuracy: 0.5000\n",
            "Epoch 17/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.7054 - accuracy: 0.5000\n",
            "Epoch 18/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.7050 - accuracy: 0.5000\n",
            "Epoch 19/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.7046 - accuracy: 0.5000\n",
            "Epoch 20/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.7043 - accuracy: 0.5000\n",
            "Epoch 21/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.7039 - accuracy: 0.5000\n",
            "Epoch 22/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.7035 - accuracy: 0.5000\n",
            "Epoch 23/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.7032 - accuracy: 0.5000\n",
            "Epoch 24/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.7028 - accuracy: 0.5000\n",
            "Epoch 25/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.7025 - accuracy: 0.5000\n",
            "Epoch 26/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.7021 - accuracy: 0.5000\n",
            "Epoch 27/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.7017 - accuracy: 0.5000\n",
            "Epoch 28/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.7014 - accuracy: 0.5000\n",
            "Epoch 29/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.7010 - accuracy: 0.5000\n",
            "Epoch 30/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.7007 - accuracy: 0.5000\n",
            "Epoch 31/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.7003 - accuracy: 0.5000\n",
            "Epoch 32/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.7000 - accuracy: 0.5000\n",
            "Epoch 33/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6996 - accuracy: 0.5000\n",
            "Epoch 34/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6993 - accuracy: 0.5000\n",
            "Epoch 35/5000\n",
            "1/1 [==============================] - 0s 3ms/step - loss: 0.6989 - accuracy: 0.5000\n",
            "Epoch 36/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6986 - accuracy: 0.5000\n",
            "Epoch 37/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6982 - accuracy: 0.5000\n",
            "Epoch 38/5000\n",
            "1/1 [==============================] - 0s 3ms/step - loss: 0.6979 - accuracy: 0.5000\n",
            "Epoch 39/5000\n",
            "1/1 [==============================] - 0s 0s/step - loss: 0.6975 - accuracy: 0.5000\n",
            "Epoch 40/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6972 - accuracy: 0.5000\n",
            "Epoch 41/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6969 - accuracy: 0.5000\n",
            "Epoch 42/5000\n",
            "1/1 [==============================] - 0s 998us/step - loss: 0.6965 - accuracy: 0.5000\n",
            "Epoch 43/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6962 - accuracy: 0.5000\n",
            "Epoch 44/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6958 - accuracy: 0.5000\n",
            "Epoch 45/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6955 - accuracy: 0.5000\n",
            "Epoch 46/5000\n",
            "1/1 [==============================] - 0s 0s/step - loss: 0.6952 - accuracy: 0.5000\n",
            "Epoch 47/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6948 - accuracy: 0.5000\n",
            "Epoch 48/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6945 - accuracy: 0.5000\n",
            "Epoch 49/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6942 - accuracy: 0.5000\n",
            "Epoch 50/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6938 - accuracy: 0.5000\n",
            "Epoch 51/5000\n",
            "1/1 [==============================] - 0s 0s/step - loss: 0.6935 - accuracy: 0.5000\n",
            "Epoch 52/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6932 - accuracy: 0.5000\n",
            "Epoch 53/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6928 - accuracy: 0.5000\n",
            "Epoch 54/5000\n",
            "1/1 [==============================] - 0s 3ms/step - loss: 0.6925 - accuracy: 0.5000\n",
            "Epoch 55/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6922 - accuracy: 0.5000\n",
            "Epoch 56/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6918 - accuracy: 0.5000\n",
            "Epoch 57/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6915 - accuracy: 0.5000\n",
            "Epoch 58/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6912 - accuracy: 0.5000\n",
            "Epoch 59/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6909 - accuracy: 0.5000\n",
            "Epoch 60/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6906 - accuracy: 0.5000\n",
            "Epoch 61/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6902 - accuracy: 0.5000\n",
            "Epoch 62/5000\n",
            "1/1 [==============================] - 0s 1000us/step - loss: 0.6899 - accuracy: 0.5000\n",
            "Epoch 63/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6896 - accuracy: 0.5000\n",
            "Epoch 64/5000\n",
            "1/1 [==============================] - 0s 3ms/step - loss: 0.6893 - accuracy: 0.5000\n",
            "Epoch 65/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6890 - accuracy: 0.5000\n",
            "Epoch 66/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6886 - accuracy: 0.5000\n",
            "Epoch 67/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6883 - accuracy: 0.5000\n",
            "Epoch 68/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6880 - accuracy: 0.5000\n",
            "Epoch 69/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6877 - accuracy: 0.5000\n",
            "Epoch 70/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6874 - accuracy: 0.5000\n",
            "Epoch 71/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6871 - accuracy: 0.5000\n",
            "Epoch 72/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6868 - accuracy: 0.5000\n",
            "Epoch 73/5000\n",
            "1/1 [==============================] - 0s 1000us/step - loss: 0.6865 - accuracy: 0.5000\n",
            "Epoch 74/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6862 - accuracy: 0.5000\n",
            "Epoch 75/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6859 - accuracy: 0.5000\n",
            "Epoch 76/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6856 - accuracy: 0.5000\n",
            "Epoch 77/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6853 - accuracy: 0.5000\n",
            "Epoch 78/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6849 - accuracy: 0.5000\n",
            "Epoch 79/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6846 - accuracy: 0.5000\n",
            "Epoch 80/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6843 - accuracy: 0.5000\n",
            "Epoch 81/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6840 - accuracy: 0.5000\n",
            "Epoch 82/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6838 - accuracy: 0.5000\n",
            "Epoch 83/5000\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6835 - accuracy: 0.5000\n",
            "Epoch 84/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6832 - accuracy: 0.5000\n",
            "Epoch 85/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6829 - accuracy: 0.5000\n",
            "Epoch 86/5000\n",
            "1/1 [==============================] - 0s 999us/step - loss: 0.6826 - accuracy: 0.5000\n",
            "Epoch 87/5000\n",
            "1/1 [==============================] - 0s 0s/step - loss: 0.6823 - accuracy: 0.5000\n",
            "Epoch 88/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6820 - accuracy: 0.5000\n",
            "Epoch 89/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6817 - accuracy: 0.5000\n",
            "Epoch 90/5000\n",
            "1/1 [==============================] - 0s 3ms/step - loss: 0.6814 - accuracy: 0.5000\n",
            "Epoch 91/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6811 - accuracy: 0.5000\n",
            "Epoch 92/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6808 - accuracy: 0.5000\n",
            "Epoch 93/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6805 - accuracy: 0.5000\n",
            "Epoch 94/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6803 - accuracy: 0.5000\n",
            "Epoch 95/5000\n",
            "1/1 [==============================] - 0s 1000us/step - loss: 0.6800 - accuracy: 0.5000\n",
            "Epoch 96/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6797 - accuracy: 0.5000\n",
            "Epoch 97/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6794 - accuracy: 0.5000\n",
            "Epoch 98/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6791 - accuracy: 0.5000\n",
            "Epoch 99/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6788 - accuracy: 0.5000\n",
            "Epoch 100/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6786 - accuracy: 0.5000\n",
            "Epoch 101/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6783 - accuracy: 0.5000\n",
            "Epoch 102/5000\n",
            "1/1 [==============================] - 0s 1000us/step - loss: 0.6780 - accuracy: 0.5000\n",
            "Epoch 103/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6777 - accuracy: 0.5000\n",
            "Epoch 104/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6775 - accuracy: 0.5000\n",
            "Epoch 105/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6772 - accuracy: 0.5000\n",
            "Epoch 106/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6769 - accuracy: 0.5000\n",
            "Epoch 107/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6766 - accuracy: 0.5000\n",
            "Epoch 108/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6764 - accuracy: 0.5000\n",
            "Epoch 109/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6761 - accuracy: 0.5000\n",
            "Epoch 110/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6758 - accuracy: 0.5000\n",
            "Epoch 111/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6756 - accuracy: 0.5000\n",
            "Epoch 112/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6753 - accuracy: 0.5000\n",
            "Epoch 113/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6750 - accuracy: 0.5000\n",
            "Epoch 114/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6748 - accuracy: 0.5000\n",
            "Epoch 115/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6745 - accuracy: 0.5000\n",
            "Epoch 116/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6742 - accuracy: 0.5000\n",
            "Epoch 117/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6740 - accuracy: 0.5000\n",
            "Epoch 118/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6737 - accuracy: 0.5000\n",
            "Epoch 119/5000\n",
            "1/1 [==============================] - 0s 999us/step - loss: 0.6735 - accuracy: 0.5000\n",
            "Epoch 120/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6732 - accuracy: 0.5000\n",
            "Epoch 121/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6730 - accuracy: 0.5000\n",
            "Epoch 122/5000\n",
            "1/1 [==============================] - 0s 4ms/step - loss: 0.6727 - accuracy: 0.5000\n",
            "Epoch 123/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6724 - accuracy: 0.5000\n",
            "Epoch 124/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6722 - accuracy: 0.5000\n",
            "Epoch 125/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6719 - accuracy: 0.5000\n",
            "Epoch 126/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6717 - accuracy: 0.5000\n",
            "Epoch 127/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6714 - accuracy: 0.5000\n",
            "Epoch 128/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6712 - accuracy: 0.5000\n",
            "Epoch 129/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6709 - accuracy: 0.5000\n",
            "Epoch 130/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6707 - accuracy: 0.5000\n",
            "Epoch 131/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6704 - accuracy: 0.5000\n",
            "Epoch 132/5000\n",
            "1/1 [==============================] - 0s 3ms/step - loss: 0.6702 - accuracy: 0.5000\n",
            "Epoch 133/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6699 - accuracy: 0.5000\n",
            "Epoch 134/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6697 - accuracy: 0.5000\n",
            "Epoch 135/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6695 - accuracy: 0.5000\n",
            "Epoch 136/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6692 - accuracy: 0.5000\n",
            "Epoch 137/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6690 - accuracy: 0.5000\n",
            "Epoch 138/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6687 - accuracy: 0.5000\n",
            "Epoch 139/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6685 - accuracy: 0.5000\n",
            "Epoch 140/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6683 - accuracy: 0.5000\n",
            "Epoch 141/5000\n",
            "1/1 [==============================] - 0s 3ms/step - loss: 0.6680 - accuracy: 0.5000\n",
            "Epoch 142/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6678 - accuracy: 0.5000\n",
            "Epoch 143/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6675 - accuracy: 0.5000\n",
            "Epoch 144/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6673 - accuracy: 0.5000\n",
            "Epoch 145/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6671 - accuracy: 0.5000\n",
            "Epoch 146/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6668 - accuracy: 0.5000\n",
            "Epoch 147/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6666 - accuracy: 0.5000\n",
            "Epoch 148/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6664 - accuracy: 0.5000\n",
            "Epoch 149/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6661 - accuracy: 0.5000\n",
            "Epoch 150/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6659 - accuracy: 0.5000\n",
            "Epoch 151/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6657 - accuracy: 0.5000\n",
            "Epoch 152/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6655 - accuracy: 0.5000\n",
            "Epoch 153/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6652 - accuracy: 0.5000\n",
            "Epoch 154/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6650 - accuracy: 0.5000\n",
            "Epoch 155/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6648 - accuracy: 0.5000\n",
            "Epoch 156/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6646 - accuracy: 0.5000\n",
            "Epoch 157/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6643 - accuracy: 0.5000\n",
            "Epoch 158/5000\n",
            "1/1 [==============================] - 0s 3ms/step - loss: 0.6641 - accuracy: 0.5000\n",
            "Epoch 159/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6639 - accuracy: 0.5000\n",
            "Epoch 160/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6637 - accuracy: 0.5000\n",
            "Epoch 161/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6635 - accuracy: 0.5000\n",
            "Epoch 162/5000\n",
            "1/1 [==============================] - 0s 3ms/step - loss: 0.6632 - accuracy: 0.5000\n",
            "Epoch 163/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6630 - accuracy: 0.5000\n",
            "Epoch 164/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6628 - accuracy: 0.5000\n",
            "Epoch 165/5000\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6626 - accuracy: 0.5000\n",
            "Epoch 166/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6624 - accuracy: 0.5000\n",
            "Epoch 167/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6622 - accuracy: 0.5000\n",
            "Epoch 168/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6620 - accuracy: 0.5000\n",
            "Epoch 169/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6617 - accuracy: 0.5000\n",
            "Epoch 170/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6615 - accuracy: 0.5000\n",
            "Epoch 171/5000\n",
            "1/1 [==============================] - 0s 1000us/step - loss: 0.6613 - accuracy: 0.5000\n",
            "Epoch 172/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6611 - accuracy: 0.5000\n",
            "Epoch 173/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6609 - accuracy: 0.5455\n",
            "Epoch 174/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6607 - accuracy: 0.5455\n",
            "Epoch 175/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6605 - accuracy: 0.5455\n",
            "Epoch 176/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6603 - accuracy: 0.5455\n",
            "Epoch 177/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6601 - accuracy: 0.5455\n",
            "Epoch 178/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6599 - accuracy: 0.5455\n",
            "Epoch 179/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6597 - accuracy: 0.5455\n",
            "Epoch 180/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6595 - accuracy: 0.5455\n",
            "Epoch 181/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6593 - accuracy: 0.5455\n",
            "Epoch 182/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6591 - accuracy: 0.5455\n",
            "Epoch 183/5000\n",
            "1/1 [==============================] - 0s 999us/step - loss: 0.6589 - accuracy: 0.5455\n",
            "Epoch 184/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6587 - accuracy: 0.5455\n",
            "Epoch 185/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6585 - accuracy: 0.5455\n",
            "Epoch 186/5000\n",
            "1/1 [==============================] - 0s 3ms/step - loss: 0.6583 - accuracy: 0.5455\n",
            "Epoch 187/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6581 - accuracy: 0.5455\n",
            "Epoch 188/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6579 - accuracy: 0.5455\n",
            "Epoch 189/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6577 - accuracy: 0.5455\n",
            "Epoch 190/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6575 - accuracy: 0.5455\n",
            "Epoch 191/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6573 - accuracy: 0.5455\n",
            "Epoch 192/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6571 - accuracy: 0.5455\n",
            "Epoch 193/5000\n",
            "1/1 [==============================] - 0s 0s/step - loss: 0.6569 - accuracy: 0.5455\n",
            "Epoch 194/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6568 - accuracy: 0.5455\n",
            "Epoch 195/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6566 - accuracy: 0.5455\n",
            "Epoch 196/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6564 - accuracy: 0.5455\n",
            "Epoch 197/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6562 - accuracy: 0.5455\n",
            "Epoch 198/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6560 - accuracy: 0.5455\n",
            "Epoch 199/5000\n",
            "1/1 [==============================] - 0s 3ms/step - loss: 0.6558 - accuracy: 0.5455\n",
            "Epoch 200/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6556 - accuracy: 0.5455\n",
            "Epoch 201/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6555 - accuracy: 0.5455\n",
            "Epoch 202/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6553 - accuracy: 0.5455\n",
            "Epoch 203/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6551 - accuracy: 0.5455\n",
            "Epoch 204/5000\n",
            "1/1 [==============================] - 0s 0s/step - loss: 0.6549 - accuracy: 0.5455\n",
            "Epoch 205/5000\n",
            "1/1 [==============================] - 0s 0s/step - loss: 0.6547 - accuracy: 0.5455\n",
            "Epoch 206/5000\n",
            "1/1 [==============================] - 0s 995us/step - loss: 0.6545 - accuracy: 0.5455\n",
            "Epoch 207/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6544 - accuracy: 0.5455\n",
            "Epoch 208/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6542 - accuracy: 0.5455\n",
            "Epoch 209/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6540 - accuracy: 0.5455\n",
            "Epoch 210/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6538 - accuracy: 0.5455\n",
            "Epoch 211/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6537 - accuracy: 0.5455\n",
            "Epoch 212/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6535 - accuracy: 0.5455\n",
            "Epoch 213/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6533 - accuracy: 0.5455\n",
            "Epoch 214/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6531 - accuracy: 0.5455\n",
            "Epoch 215/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6530 - accuracy: 0.5455\n",
            "Epoch 216/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6528 - accuracy: 0.5455\n",
            "Epoch 217/5000\n",
            "1/1 [==============================] - 0s 4ms/step - loss: 0.6526 - accuracy: 0.5455\n",
            "Epoch 218/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6525 - accuracy: 0.5455\n",
            "Epoch 219/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6523 - accuracy: 0.5455\n",
            "Epoch 220/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6521 - accuracy: 0.5455\n",
            "Epoch 221/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6519 - accuracy: 0.5455\n",
            "Epoch 222/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6518 - accuracy: 0.5455\n",
            "Epoch 223/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6516 - accuracy: 0.5455\n",
            "Epoch 224/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6515 - accuracy: 0.5455\n",
            "Epoch 225/5000\n",
            "1/1 [==============================] - 0s 308us/step - loss: 0.6513 - accuracy: 0.5455\n",
            "Epoch 226/5000\n",
            "1/1 [==============================] - 0s 3ms/step - loss: 0.6511 - accuracy: 0.5455\n",
            "Epoch 227/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6510 - accuracy: 0.5455\n",
            "Epoch 228/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6508 - accuracy: 0.5455\n",
            "Epoch 229/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6506 - accuracy: 0.5455\n",
            "Epoch 230/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6505 - accuracy: 0.5455\n",
            "Epoch 231/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6503 - accuracy: 0.5455\n",
            "Epoch 232/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6502 - accuracy: 0.5455\n",
            "Epoch 233/5000\n",
            "1/1 [==============================] - 0s 0s/step - loss: 0.6500 - accuracy: 0.5455\n",
            "Epoch 234/5000\n",
            "1/1 [==============================] - 0s 931us/step - loss: 0.6498 - accuracy: 0.5455\n",
            "Epoch 235/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6497 - accuracy: 0.5455\n",
            "Epoch 236/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6495 - accuracy: 0.5455\n",
            "Epoch 237/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6494 - accuracy: 0.5455\n",
            "Epoch 238/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6492 - accuracy: 0.5455\n",
            "Epoch 239/5000\n",
            "1/1 [==============================] - 0s 3ms/step - loss: 0.6491 - accuracy: 0.5455\n",
            "Epoch 240/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6489 - accuracy: 0.5455\n",
            "Epoch 241/5000\n",
            "1/1 [==============================] - 0s 0s/step - loss: 0.6488 - accuracy: 0.5455\n",
            "Epoch 242/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6486 - accuracy: 0.5909\n",
            "Epoch 243/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6485 - accuracy: 0.5909\n",
            "Epoch 244/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6483 - accuracy: 0.5909\n",
            "Epoch 245/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6482 - accuracy: 0.5909\n",
            "Epoch 246/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6480 - accuracy: 0.5909\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Epoch 247/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6479 - accuracy: 0.5909\n",
            "Epoch 248/5000\n",
            "1/1 [==============================] - 0s 3ms/step - loss: 0.6477 - accuracy: 0.5909\n",
            "Epoch 249/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6476 - accuracy: 0.5909\n",
            "Epoch 250/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6474 - accuracy: 0.5909\n",
            "Epoch 251/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6473 - accuracy: 0.5909\n",
            "Epoch 252/5000\n",
            "1/1 [==============================] - 0s 5ms/step - loss: 0.6471 - accuracy: 0.5909\n",
            "Epoch 253/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6470 - accuracy: 0.5909\n",
            "Epoch 254/5000\n",
            "1/1 [==============================] - 0s 0s/step - loss: 0.6468 - accuracy: 0.5909\n",
            "Epoch 255/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6467 - accuracy: 0.5909\n",
            "Epoch 256/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6465 - accuracy: 0.5909\n",
            "Epoch 257/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6464 - accuracy: 0.5909\n",
            "Epoch 258/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6463 - accuracy: 0.5909\n",
            "Epoch 259/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6461 - accuracy: 0.6364\n",
            "Epoch 260/5000\n",
            "1/1 [==============================] - 0s 1000us/step - loss: 0.6460 - accuracy: 0.6364\n",
            "Epoch 261/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6458 - accuracy: 0.6364\n",
            "Epoch 262/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6457 - accuracy: 0.6364\n",
            "Epoch 263/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6456 - accuracy: 0.6364\n",
            "Epoch 264/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6454 - accuracy: 0.6364\n",
            "Epoch 265/5000\n",
            "1/1 [==============================] - ETA: 0s - loss: 0.6453 - accuracy: 0.63 - 0s 3ms/step - loss: 0.6453 - accuracy: 0.6364\n",
            "Epoch 266/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6452 - accuracy: 0.6364\n",
            "Epoch 267/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6450 - accuracy: 0.6364\n",
            "Epoch 268/5000\n",
            "1/1 [==============================] - 0s 997us/step - loss: 0.6449 - accuracy: 0.6364\n",
            "Epoch 269/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6448 - accuracy: 0.6364\n",
            "Epoch 270/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6446 - accuracy: 0.6364\n",
            "Epoch 271/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6445 - accuracy: 0.6364\n",
            "Epoch 272/5000\n",
            "1/1 [==============================] - 0s 751us/step - loss: 0.6444 - accuracy: 0.6364\n",
            "Epoch 273/5000\n",
            "1/1 [==============================] - 0s 0s/step - loss: 0.6442 - accuracy: 0.6364\n",
            "Epoch 274/5000\n",
            "1/1 [==============================] - 0s 996us/step - loss: 0.6441 - accuracy: 0.6364\n",
            "Epoch 275/5000\n",
            "1/1 [==============================] - 0s 998us/step - loss: 0.6440 - accuracy: 0.6364\n",
            "Epoch 276/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6438 - accuracy: 0.6364\n",
            "Epoch 277/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6437 - accuracy: 0.6364\n",
            "Epoch 278/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6436 - accuracy: 0.6364\n",
            "Epoch 279/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6435 - accuracy: 0.6364\n",
            "Epoch 280/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6433 - accuracy: 0.6364\n",
            "Epoch 281/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6432 - accuracy: 0.6364\n",
            "Epoch 282/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6431 - accuracy: 0.6364\n",
            "Epoch 283/5000\n",
            "1/1 [==============================] - 0s 1000us/step - loss: 0.6430 - accuracy: 0.6364\n",
            "Epoch 284/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6428 - accuracy: 0.6364\n",
            "Epoch 285/5000\n",
            "1/1 [==============================] - 0s 0s/step - loss: 0.6427 - accuracy: 0.6364\n",
            "Epoch 286/5000\n",
            "1/1 [==============================] - 0s 0s/step - loss: 0.6426 - accuracy: 0.6364\n",
            "Epoch 287/5000\n",
            "1/1 [==============================] - 0s 0s/step - loss: 0.6425 - accuracy: 0.6364\n",
            "Epoch 288/5000\n",
            "1/1 [==============================] - 0s 1000us/step - loss: 0.6423 - accuracy: 0.6364\n",
            "Epoch 289/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6422 - accuracy: 0.6364\n",
            "Epoch 290/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6421 - accuracy: 0.6364\n",
            "Epoch 291/5000\n",
            "1/1 [==============================] - 0s 0s/step - loss: 0.6420 - accuracy: 0.6364\n",
            "Epoch 292/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6419 - accuracy: 0.6364\n",
            "Epoch 293/5000\n",
            "1/1 [==============================] - 0s 3ms/step - loss: 0.6417 - accuracy: 0.6364\n",
            "Epoch 294/5000\n",
            "1/1 [==============================] - 0s 1000us/step - loss: 0.6416 - accuracy: 0.6364\n",
            "Epoch 295/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6415 - accuracy: 0.6364\n",
            "Epoch 296/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6414 - accuracy: 0.6364\n",
            "Epoch 297/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6413 - accuracy: 0.6364\n",
            "Epoch 298/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6411 - accuracy: 0.6364\n",
            "Epoch 299/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6410 - accuracy: 0.6364\n",
            "Epoch 300/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6409 - accuracy: 0.6364\n",
            "Epoch 301/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6408 - accuracy: 0.6364\n",
            "Epoch 302/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6407 - accuracy: 0.6364\n",
            "Epoch 303/5000\n",
            "1/1 [==============================] - 0s 1000us/step - loss: 0.6406 - accuracy: 0.6364\n",
            "Epoch 304/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6405 - accuracy: 0.6364\n",
            "Epoch 305/5000\n",
            "1/1 [==============================] - 0s 975us/step - loss: 0.6403 - accuracy: 0.6364\n",
            "Epoch 306/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6402 - accuracy: 0.6364\n",
            "Epoch 307/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6401 - accuracy: 0.6364\n",
            "Epoch 308/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6400 - accuracy: 0.6364\n",
            "Epoch 309/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6399 - accuracy: 0.6364\n",
            "Epoch 310/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6398 - accuracy: 0.6364\n",
            "Epoch 311/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6397 - accuracy: 0.6364\n",
            "Epoch 312/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6396 - accuracy: 0.6364\n",
            "Epoch 313/5000\n",
            "1/1 [==============================] - 0s 4ms/step - loss: 0.6395 - accuracy: 0.6364\n",
            "Epoch 314/5000\n",
            "1/1 [==============================] - 0s 0s/step - loss: 0.6394 - accuracy: 0.6364\n",
            "Epoch 315/5000\n",
            "1/1 [==============================] - 0s 911us/step - loss: 0.6392 - accuracy: 0.6364\n",
            "Epoch 316/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6391 - accuracy: 0.6364\n",
            "Epoch 317/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6390 - accuracy: 0.6364\n",
            "Epoch 318/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6389 - accuracy: 0.6364\n",
            "Epoch 319/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6388 - accuracy: 0.6364\n",
            "Epoch 320/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6387 - accuracy: 0.6364\n",
            "Epoch 321/5000\n",
            "1/1 [==============================] - 0s 4ms/step - loss: 0.6386 - accuracy: 0.6364\n",
            "Epoch 322/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6385 - accuracy: 0.6364\n",
            "Epoch 323/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6384 - accuracy: 0.6364\n",
            "Epoch 324/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6383 - accuracy: 0.6364\n",
            "Epoch 325/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6382 - accuracy: 0.6364\n",
            "Epoch 326/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6381 - accuracy: 0.6364\n",
            "Epoch 327/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6380 - accuracy: 0.6364\n",
            "Epoch 328/5000\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6379 - accuracy: 0.6364\n",
            "Epoch 329/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6378 - accuracy: 0.6364\n",
            "Epoch 330/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6377 - accuracy: 0.6364\n",
            "Epoch 331/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6376 - accuracy: 0.6364\n",
            "Epoch 332/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6375 - accuracy: 0.6364\n",
            "Epoch 333/5000\n",
            "1/1 [==============================] - 0s 0s/step - loss: 0.6374 - accuracy: 0.6364\n",
            "Epoch 334/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6373 - accuracy: 0.6364\n",
            "Epoch 335/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6372 - accuracy: 0.6364\n",
            "Epoch 336/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6371 - accuracy: 0.6364\n",
            "Epoch 337/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6370 - accuracy: 0.6364\n",
            "Epoch 338/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6369 - accuracy: 0.6364\n",
            "Epoch 339/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6368 - accuracy: 0.6364\n",
            "Epoch 340/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6367 - accuracy: 0.6364\n",
            "Epoch 341/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6366 - accuracy: 0.6364\n",
            "Epoch 342/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6365 - accuracy: 0.6364\n",
            "Epoch 343/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6364 - accuracy: 0.6364\n",
            "Epoch 344/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6363 - accuracy: 0.6364\n",
            "Epoch 345/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6362 - accuracy: 0.6364\n",
            "Epoch 346/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6361 - accuracy: 0.6364\n",
            "Epoch 347/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6361 - accuracy: 0.6364\n",
            "Epoch 348/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6360 - accuracy: 0.6364\n",
            "Epoch 349/5000\n",
            "1/1 [==============================] - 0s 3ms/step - loss: 0.6359 - accuracy: 0.6364\n",
            "Epoch 350/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6358 - accuracy: 0.6364\n",
            "Epoch 351/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6357 - accuracy: 0.6364\n",
            "Epoch 352/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6356 - accuracy: 0.6364\n",
            "Epoch 353/5000\n",
            "1/1 [==============================] - 0s 4ms/step - loss: 0.6355 - accuracy: 0.6364\n",
            "Epoch 354/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6354 - accuracy: 0.6364\n",
            "Epoch 355/5000\n",
            "1/1 [==============================] - 0s 1000us/step - loss: 0.6353 - accuracy: 0.6364\n",
            "Epoch 356/5000\n",
            "1/1 [==============================] - 0s 1000us/step - loss: 0.6352 - accuracy: 0.6364\n",
            "Epoch 357/5000\n",
            "1/1 [==============================] - 0s 999us/step - loss: 0.6352 - accuracy: 0.6364\n",
            "Epoch 358/5000\n",
            "1/1 [==============================] - 0s 4ms/step - loss: 0.6351 - accuracy: 0.6364\n",
            "Epoch 359/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6350 - accuracy: 0.6364\n",
            "Epoch 360/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6349 - accuracy: 0.6364\n",
            "Epoch 361/5000\n",
            "1/1 [==============================] - 0s 0s/step - loss: 0.6348 - accuracy: 0.6364\n",
            "Epoch 362/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6347 - accuracy: 0.6364\n",
            "Epoch 363/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6346 - accuracy: 0.6364\n",
            "Epoch 364/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6345 - accuracy: 0.6364\n",
            "Epoch 365/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6345 - accuracy: 0.6364\n",
            "Epoch 366/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6344 - accuracy: 0.6364\n",
            "Epoch 367/5000\n",
            "1/1 [==============================] - 0s 1000us/step - loss: 0.6343 - accuracy: 0.6364\n",
            "Epoch 368/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6342 - accuracy: 0.6364\n",
            "Epoch 369/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6341 - accuracy: 0.6364\n",
            "Epoch 370/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6340 - accuracy: 0.6364\n",
            "Epoch 371/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6339 - accuracy: 0.6364\n",
            "Epoch 372/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6339 - accuracy: 0.6364\n",
            "Epoch 373/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6338 - accuracy: 0.6364\n",
            "Epoch 374/5000\n",
            "1/1 [==============================] - 0s 0s/step - loss: 0.6337 - accuracy: 0.6364\n",
            "Epoch 375/5000\n",
            "1/1 [==============================] - 0s 0s/step - loss: 0.6336 - accuracy: 0.6364\n",
            "Epoch 376/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6335 - accuracy: 0.6364\n",
            "Epoch 377/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6335 - accuracy: 0.6364\n",
            "Epoch 378/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6334 - accuracy: 0.6364\n",
            "Epoch 379/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6333 - accuracy: 0.6364\n",
            "Epoch 380/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6332 - accuracy: 0.6364\n",
            "Epoch 381/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6331 - accuracy: 0.6364\n",
            "Epoch 382/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6331 - accuracy: 0.6364\n",
            "Epoch 383/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6330 - accuracy: 0.6364\n",
            "Epoch 384/5000\n",
            "1/1 [==============================] - 0s 999us/step - loss: 0.6329 - accuracy: 0.6364\n",
            "Epoch 385/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6328 - accuracy: 0.6364\n",
            "Epoch 386/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6327 - accuracy: 0.6364\n",
            "Epoch 387/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6327 - accuracy: 0.6364\n",
            "Epoch 388/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6326 - accuracy: 0.6364\n",
            "Epoch 389/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6325 - accuracy: 0.6364\n",
            "Epoch 390/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6324 - accuracy: 0.6364\n",
            "Epoch 391/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6323 - accuracy: 0.6364\n",
            "Epoch 392/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6323 - accuracy: 0.6364\n",
            "Epoch 393/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6322 - accuracy: 0.6364\n",
            "Epoch 394/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6321 - accuracy: 0.6364\n",
            "Epoch 395/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6320 - accuracy: 0.6364\n",
            "Epoch 396/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6320 - accuracy: 0.6364\n",
            "Epoch 397/5000\n",
            "1/1 [==============================] - 0s 0s/step - loss: 0.6319 - accuracy: 0.6364\n",
            "Epoch 398/5000\n",
            "1/1 [==============================] - 0s 0s/step - loss: 0.6318 - accuracy: 0.6364\n",
            "Epoch 399/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6317 - accuracy: 0.6364\n",
            "Epoch 400/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6317 - accuracy: 0.6364\n",
            "Epoch 401/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6316 - accuracy: 0.6364\n",
            "Epoch 402/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6315 - accuracy: 0.6364\n",
            "Epoch 403/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6314 - accuracy: 0.6364\n",
            "Epoch 404/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6314 - accuracy: 0.6364\n",
            "Epoch 405/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6313 - accuracy: 0.6364\n",
            "Epoch 406/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6312 - accuracy: 0.6364\n",
            "Epoch 407/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6312 - accuracy: 0.6364\n",
            "Epoch 408/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6311 - accuracy: 0.6364\n",
            "Epoch 409/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6310 - accuracy: 0.6364\n",
            "Epoch 410/5000\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6309 - accuracy: 0.6364\n",
            "Epoch 411/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6309 - accuracy: 0.6364\n",
            "Epoch 412/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6308 - accuracy: 0.6364\n",
            "Epoch 413/5000\n",
            "1/1 [==============================] - 0s 0s/step - loss: 0.6307 - accuracy: 0.6364\n",
            "Epoch 414/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6307 - accuracy: 0.6364\n",
            "Epoch 415/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6306 - accuracy: 0.6364\n",
            "Epoch 416/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6305 - accuracy: 0.6364\n",
            "Epoch 417/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6305 - accuracy: 0.6364\n",
            "Epoch 418/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6304 - accuracy: 0.6364\n",
            "Epoch 419/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6303 - accuracy: 0.6364\n",
            "Epoch 420/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6302 - accuracy: 0.6364\n",
            "Epoch 421/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6302 - accuracy: 0.6364\n",
            "Epoch 422/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6301 - accuracy: 0.6364\n",
            "Epoch 423/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6300 - accuracy: 0.6364\n",
            "Epoch 424/5000\n",
            "1/1 [==============================] - 0s 3ms/step - loss: 0.6300 - accuracy: 0.6364\n",
            "Epoch 425/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6299 - accuracy: 0.6364\n",
            "Epoch 426/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6298 - accuracy: 0.6364\n",
            "Epoch 427/5000\n",
            "1/1 [==============================] - 0s 4ms/step - loss: 0.6298 - accuracy: 0.6364\n",
            "Epoch 428/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6297 - accuracy: 0.6364\n",
            "Epoch 429/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6296 - accuracy: 0.6364\n",
            "Epoch 430/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6296 - accuracy: 0.6364\n",
            "Epoch 431/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6295 - accuracy: 0.6364\n",
            "Epoch 432/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6294 - accuracy: 0.6364\n",
            "Epoch 433/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6294 - accuracy: 0.6364\n",
            "Epoch 434/5000\n",
            "1/1 [==============================] - 0s 3ms/step - loss: 0.6293 - accuracy: 0.6364\n",
            "Epoch 435/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6292 - accuracy: 0.6364\n",
            "Epoch 436/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6292 - accuracy: 0.6364\n",
            "Epoch 437/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6291 - accuracy: 0.6364\n",
            "Epoch 438/5000\n",
            "1/1 [==============================] - 0s 3ms/step - loss: 0.6290 - accuracy: 0.6364\n",
            "Epoch 439/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6290 - accuracy: 0.6364\n",
            "Epoch 440/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6289 - accuracy: 0.6364\n",
            "Epoch 441/5000\n",
            "1/1 [==============================] - 0s 3ms/step - loss: 0.6288 - accuracy: 0.6364\n",
            "Epoch 442/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6288 - accuracy: 0.6364\n",
            "Epoch 443/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6287 - accuracy: 0.6364\n",
            "Epoch 444/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6287 - accuracy: 0.6364\n",
            "Epoch 445/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6286 - accuracy: 0.6364\n",
            "Epoch 446/5000\n",
            "1/1 [==============================] - 0s 0s/step - loss: 0.6285 - accuracy: 0.6364\n",
            "Epoch 447/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6285 - accuracy: 0.6364\n",
            "Epoch 448/5000\n",
            "1/1 [==============================] - 0s 3ms/step - loss: 0.6284 - accuracy: 0.6364\n",
            "Epoch 449/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6283 - accuracy: 0.6364\n",
            "Epoch 450/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6283 - accuracy: 0.6364\n",
            "Epoch 451/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6282 - accuracy: 0.6364\n",
            "Epoch 452/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6282 - accuracy: 0.6364\n",
            "Epoch 453/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6281 - accuracy: 0.6364\n",
            "Epoch 454/5000\n",
            "1/1 [==============================] - 0s 3ms/step - loss: 0.6280 - accuracy: 0.6364\n",
            "Epoch 455/5000\n",
            "1/1 [==============================] - 0s 1000us/step - loss: 0.6280 - accuracy: 0.6364\n",
            "Epoch 456/5000\n",
            "1/1 [==============================] - 0s 1000us/step - loss: 0.6279 - accuracy: 0.6364\n",
            "Epoch 457/5000\n",
            "1/1 [==============================] - 0s 1000us/step - loss: 0.6278 - accuracy: 0.6364\n",
            "Epoch 458/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6278 - accuracy: 0.6364\n",
            "Epoch 459/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6277 - accuracy: 0.6364\n",
            "Epoch 460/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6277 - accuracy: 0.6364\n",
            "Epoch 461/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6276 - accuracy: 0.6364\n",
            "Epoch 462/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6275 - accuracy: 0.6364\n",
            "Epoch 463/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6275 - accuracy: 0.6364\n",
            "Epoch 464/5000\n",
            "1/1 [==============================] - 0s 999us/step - loss: 0.6274 - accuracy: 0.6364\n",
            "Epoch 465/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6274 - accuracy: 0.6364\n",
            "Epoch 466/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6273 - accuracy: 0.6364\n",
            "Epoch 467/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6272 - accuracy: 0.6364\n",
            "Epoch 468/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6272 - accuracy: 0.6364\n",
            "Epoch 469/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6271 - accuracy: 0.6364\n",
            "Epoch 470/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6271 - accuracy: 0.6364\n",
            "Epoch 471/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6270 - accuracy: 0.6364\n",
            "Epoch 472/5000\n",
            "1/1 [==============================] - 0s 1000us/step - loss: 0.6269 - accuracy: 0.6364\n",
            "Epoch 473/5000\n",
            "1/1 [==============================] - 0s 1000us/step - loss: 0.6269 - accuracy: 0.6364\n",
            "Epoch 474/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6268 - accuracy: 0.6364\n",
            "Epoch 475/5000\n",
            "1/1 [==============================] - 0s 0s/step - loss: 0.6268 - accuracy: 0.6364\n",
            "Epoch 476/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6267 - accuracy: 0.6364\n",
            "Epoch 477/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6267 - accuracy: 0.6364\n",
            "Epoch 478/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6266 - accuracy: 0.6364\n",
            "Epoch 479/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6265 - accuracy: 0.6364\n",
            "Epoch 480/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6265 - accuracy: 0.6364\n",
            "Epoch 481/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6264 - accuracy: 0.6364\n",
            "Epoch 482/5000\n",
            "1/1 [==============================] - 0s 504us/step - loss: 0.6264 - accuracy: 0.6364\n",
            "Epoch 483/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6263 - accuracy: 0.6364\n",
            "Epoch 484/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6262 - accuracy: 0.6364\n",
            "Epoch 485/5000\n",
            "1/1 [==============================] - 0s 999us/step - loss: 0.6262 - accuracy: 0.6364\n",
            "Epoch 486/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6261 - accuracy: 0.6364\n",
            "Epoch 487/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6261 - accuracy: 0.6364\n",
            "Epoch 488/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6260 - accuracy: 0.6364\n",
            "Epoch 489/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6260 - accuracy: 0.6364\n",
            "Epoch 490/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6259 - accuracy: 0.6364\n",
            "Epoch 491/5000\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6259 - accuracy: 0.6364\n",
            "Epoch 492/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6258 - accuracy: 0.6364\n",
            "Epoch 493/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6257 - accuracy: 0.6364\n",
            "Epoch 494/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6257 - accuracy: 0.6364\n",
            "Epoch 495/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6256 - accuracy: 0.6364\n",
            "Epoch 496/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6256 - accuracy: 0.6364\n",
            "Epoch 497/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6255 - accuracy: 0.6364\n",
            "Epoch 498/5000\n",
            "1/1 [==============================] - 0s 3ms/step - loss: 0.6255 - accuracy: 0.6364\n",
            "Epoch 499/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6254 - accuracy: 0.6364\n",
            "Epoch 500/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6253 - accuracy: 0.6364\n",
            "Epoch 501/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6253 - accuracy: 0.6364\n",
            "Epoch 502/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6252 - accuracy: 0.6364\n",
            "Epoch 503/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6252 - accuracy: 0.6364\n",
            "Epoch 504/5000\n",
            "1/1 [==============================] - 0s 1000us/step - loss: 0.6251 - accuracy: 0.6364\n",
            "Epoch 505/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6251 - accuracy: 0.6364\n",
            "Epoch 506/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6250 - accuracy: 0.6364\n",
            "Epoch 507/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6250 - accuracy: 0.6364\n",
            "Epoch 508/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6249 - accuracy: 0.6364\n",
            "Epoch 509/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6249 - accuracy: 0.6364\n",
            "Epoch 510/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6248 - accuracy: 0.6364\n",
            "Epoch 511/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6247 - accuracy: 0.6364\n",
            "Epoch 512/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6247 - accuracy: 0.6364\n",
            "Epoch 513/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6246 - accuracy: 0.6364\n",
            "Epoch 514/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6246 - accuracy: 0.6364\n",
            "Epoch 515/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6245 - accuracy: 0.6364\n",
            "Epoch 516/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6245 - accuracy: 0.6364\n",
            "Epoch 517/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6244 - accuracy: 0.6364\n",
            "Epoch 518/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6244 - accuracy: 0.6364\n",
            "Epoch 519/5000\n",
            "1/1 [==============================] - 0s 3ms/step - loss: 0.6243 - accuracy: 0.6818\n",
            "Epoch 520/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6243 - accuracy: 0.6818\n",
            "Epoch 521/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6242 - accuracy: 0.6818\n",
            "Epoch 522/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6242 - accuracy: 0.6818\n",
            "Epoch 523/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6241 - accuracy: 0.6818\n",
            "Epoch 524/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6241 - accuracy: 0.6818\n",
            "Epoch 525/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6240 - accuracy: 0.6818\n",
            "Epoch 526/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6239 - accuracy: 0.6818\n",
            "Epoch 527/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6239 - accuracy: 0.6818\n",
            "Epoch 528/5000\n",
            "1/1 [==============================] - 0s 0s/step - loss: 0.6238 - accuracy: 0.6818\n",
            "Epoch 529/5000\n",
            "1/1 [==============================] - 0s 1000us/step - loss: 0.6238 - accuracy: 0.6818\n",
            "Epoch 530/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6237 - accuracy: 0.6818\n",
            "Epoch 531/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6237 - accuracy: 0.6818\n",
            "Epoch 532/5000\n",
            "1/1 [==============================] - 0s 3ms/step - loss: 0.6236 - accuracy: 0.6818\n",
            "Epoch 533/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6236 - accuracy: 0.6818\n",
            "Epoch 534/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6235 - accuracy: 0.6818\n",
            "Epoch 535/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6235 - accuracy: 0.6818\n",
            "Epoch 536/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6234 - accuracy: 0.6818\n",
            "Epoch 537/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6234 - accuracy: 0.6818\n",
            "Epoch 538/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6233 - accuracy: 0.6818\n",
            "Epoch 539/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6233 - accuracy: 0.6818\n",
            "Epoch 540/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6232 - accuracy: 0.6818\n",
            "Epoch 541/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6232 - accuracy: 0.6818\n",
            "Epoch 542/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6231 - accuracy: 0.6818\n",
            "Epoch 543/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6231 - accuracy: 0.6818\n",
            "Epoch 544/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6230 - accuracy: 0.6818\n",
            "Epoch 545/5000\n",
            "1/1 [==============================] - 0s 0s/step - loss: 0.6230 - accuracy: 0.6818\n",
            "Epoch 546/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6229 - accuracy: 0.6818\n",
            "Epoch 547/5000\n",
            "1/1 [==============================] - 0s 1000us/step - loss: 0.6229 - accuracy: 0.6818\n",
            "Epoch 548/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6228 - accuracy: 0.6818\n",
            "Epoch 549/5000\n",
            "1/1 [==============================] - 0s 3ms/step - loss: 0.6227 - accuracy: 0.6818\n",
            "Epoch 550/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6227 - accuracy: 0.6818\n",
            "Epoch 551/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6226 - accuracy: 0.6818\n",
            "Epoch 552/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6226 - accuracy: 0.6818\n",
            "Epoch 553/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6225 - accuracy: 0.6818\n",
            "Epoch 554/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6225 - accuracy: 0.6818\n",
            "Epoch 555/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6224 - accuracy: 0.6818\n",
            "Epoch 556/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6224 - accuracy: 0.6818\n",
            "Epoch 557/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6223 - accuracy: 0.6818\n",
            "Epoch 558/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6223 - accuracy: 0.6818\n",
            "Epoch 559/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6222 - accuracy: 0.6818\n",
            "Epoch 560/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6222 - accuracy: 0.6818\n",
            "Epoch 561/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6221 - accuracy: 0.6818\n",
            "Epoch 562/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6221 - accuracy: 0.6818\n",
            "Epoch 563/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6220 - accuracy: 0.6818\n",
            "Epoch 564/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6220 - accuracy: 0.6818\n",
            "Epoch 565/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6219 - accuracy: 0.6818\n",
            "Epoch 566/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6219 - accuracy: 0.6818\n",
            "Epoch 567/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6218 - accuracy: 0.6818\n",
            "Epoch 568/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6218 - accuracy: 0.6818\n",
            "Epoch 569/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6217 - accuracy: 0.6818\n",
            "Epoch 570/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6217 - accuracy: 0.6818\n",
            "Epoch 571/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6216 - accuracy: 0.6818\n",
            "Epoch 572/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6216 - accuracy: 0.6818\n",
            "Epoch 573/5000\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "1/1 [==============================] - 0s 0s/step - loss: 0.6215 - accuracy: 0.6818\n",
            "Epoch 574/5000\n",
            "1/1 [==============================] - 0s 0s/step - loss: 0.6215 - accuracy: 0.6818\n",
            "Epoch 575/5000\n",
            "1/1 [==============================] - 0s 0s/step - loss: 0.6214 - accuracy: 0.6818\n",
            "Epoch 576/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6214 - accuracy: 0.6818\n",
            "Epoch 577/5000\n",
            "1/1 [==============================] - 0s 1000us/step - loss: 0.6213 - accuracy: 0.6818\n",
            "Epoch 578/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6213 - accuracy: 0.6818\n",
            "Epoch 579/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6212 - accuracy: 0.6818\n",
            "Epoch 580/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6212 - accuracy: 0.6818\n",
            "Epoch 581/5000\n",
            "1/1 [==============================] - 0s 3ms/step - loss: 0.6211 - accuracy: 0.6818\n",
            "Epoch 582/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6211 - accuracy: 0.6818\n",
            "Epoch 583/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6210 - accuracy: 0.6818\n",
            "Epoch 584/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6210 - accuracy: 0.6818\n",
            "Epoch 585/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6209 - accuracy: 0.6818\n",
            "Epoch 586/5000\n",
            "1/1 [==============================] - 0s 3ms/step - loss: 0.6209 - accuracy: 0.7273\n",
            "Epoch 587/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6208 - accuracy: 0.7273\n",
            "Epoch 588/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6208 - accuracy: 0.7273\n",
            "Epoch 589/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6207 - accuracy: 0.7273\n",
            "Epoch 590/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6207 - accuracy: 0.7273\n",
            "Epoch 591/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6206 - accuracy: 0.7273\n",
            "Epoch 592/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6206 - accuracy: 0.7273\n",
            "Epoch 593/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6205 - accuracy: 0.7273\n",
            "Epoch 594/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6205 - accuracy: 0.7273\n",
            "Epoch 595/5000\n",
            "1/1 [==============================] - 0s 3ms/step - loss: 0.6204 - accuracy: 0.7273\n",
            "Epoch 596/5000\n",
            "1/1 [==============================] - 0s 0s/step - loss: 0.6204 - accuracy: 0.7273\n",
            "Epoch 597/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6203 - accuracy: 0.7273\n",
            "Epoch 598/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6203 - accuracy: 0.7273\n",
            "Epoch 599/5000\n",
            "1/1 [==============================] - 0s 0s/step - loss: 0.6202 - accuracy: 0.7273\n",
            "Epoch 600/5000\n",
            "1/1 [==============================] - 0s 3ms/step - loss: 0.6202 - accuracy: 0.7273\n",
            "Epoch 601/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6201 - accuracy: 0.7273\n",
            "Epoch 602/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6201 - accuracy: 0.7273\n",
            "Epoch 603/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6200 - accuracy: 0.7273\n",
            "Epoch 604/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6200 - accuracy: 0.7273\n",
            "Epoch 605/5000\n",
            "1/1 [==============================] - 0s 3ms/step - loss: 0.6200 - accuracy: 0.7273\n",
            "Epoch 606/5000\n",
            "1/1 [==============================] - 0s 999us/step - loss: 0.6199 - accuracy: 0.7273\n",
            "Epoch 607/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6199 - accuracy: 0.7273\n",
            "Epoch 608/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6198 - accuracy: 0.7273\n",
            "Epoch 609/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6198 - accuracy: 0.7273\n",
            "Epoch 610/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6197 - accuracy: 0.7273\n",
            "Epoch 611/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6197 - accuracy: 0.7273\n",
            "Epoch 612/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6196 - accuracy: 0.7273\n",
            "Epoch 613/5000\n",
            "1/1 [==============================] - 0s 3ms/step - loss: 0.6196 - accuracy: 0.7273\n",
            "Epoch 614/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6195 - accuracy: 0.7273\n",
            "Epoch 615/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6195 - accuracy: 0.7273\n",
            "Epoch 616/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6194 - accuracy: 0.7273\n",
            "Epoch 617/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6194 - accuracy: 0.7273\n",
            "Epoch 618/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6193 - accuracy: 0.7273\n",
            "Epoch 619/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6193 - accuracy: 0.7273\n",
            "Epoch 620/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6192 - accuracy: 0.7273\n",
            "Epoch 621/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6192 - accuracy: 0.7273\n",
            "Epoch 622/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6191 - accuracy: 0.7273\n",
            "Epoch 623/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6191 - accuracy: 0.7273\n",
            "Epoch 624/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6190 - accuracy: 0.7273\n",
            "Epoch 625/5000\n",
            "1/1 [==============================] - 0s 999us/step - loss: 0.6190 - accuracy: 0.7273\n",
            "Epoch 626/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6189 - accuracy: 0.7273\n",
            "Epoch 627/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6189 - accuracy: 0.7273\n",
            "Epoch 628/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6188 - accuracy: 0.7273\n",
            "Epoch 629/5000\n",
            "1/1 [==============================] - 0s 3ms/step - loss: 0.6188 - accuracy: 0.7273\n",
            "Epoch 630/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6187 - accuracy: 0.7273\n",
            "Epoch 631/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6187 - accuracy: 0.7273\n",
            "Epoch 632/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6186 - accuracy: 0.7273\n",
            "Epoch 633/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6186 - accuracy: 0.7273\n",
            "Epoch 634/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6185 - accuracy: 0.7273\n",
            "Epoch 635/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6185 - accuracy: 0.7273\n",
            "Epoch 636/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6184 - accuracy: 0.7273\n",
            "Epoch 637/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6184 - accuracy: 0.7273\n",
            "Epoch 638/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6183 - accuracy: 0.7273\n",
            "Epoch 639/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6183 - accuracy: 0.7273\n",
            "Epoch 640/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6182 - accuracy: 0.7273\n",
            "Epoch 641/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6182 - accuracy: 0.7273\n",
            "Epoch 642/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6181 - accuracy: 0.7273\n",
            "Epoch 643/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6181 - accuracy: 0.7273\n",
            "Epoch 644/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6181 - accuracy: 0.7273\n",
            "Epoch 645/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6180 - accuracy: 0.7273\n",
            "Epoch 646/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6180 - accuracy: 0.7273\n",
            "Epoch 647/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6179 - accuracy: 0.7273\n",
            "Epoch 648/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6179 - accuracy: 0.7273\n",
            "Epoch 649/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6178 - accuracy: 0.7273\n",
            "Epoch 650/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6178 - accuracy: 0.7273\n",
            "Epoch 651/5000\n",
            "1/1 [==============================] - 0s 995us/step - loss: 0.6177 - accuracy: 0.7273\n",
            "Epoch 652/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6177 - accuracy: 0.7273\n",
            "Epoch 653/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6176 - accuracy: 0.7273\n",
            "Epoch 654/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6176 - accuracy: 0.7273\n",
            "Epoch 655/5000\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6175 - accuracy: 0.7273\n",
            "Epoch 656/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6175 - accuracy: 0.7273\n",
            "Epoch 657/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6174 - accuracy: 0.7273\n",
            "Epoch 658/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6174 - accuracy: 0.7273\n",
            "Epoch 659/5000\n",
            "1/1 [==============================] - 0s 1000us/step - loss: 0.6173 - accuracy: 0.7273\n",
            "Epoch 660/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6173 - accuracy: 0.7273\n",
            "Epoch 661/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6172 - accuracy: 0.7273\n",
            "Epoch 662/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6172 - accuracy: 0.7273\n",
            "Epoch 663/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6171 - accuracy: 0.7273\n",
            "Epoch 664/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6171 - accuracy: 0.7273\n",
            "Epoch 665/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6170 - accuracy: 0.7273\n",
            "Epoch 666/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6170 - accuracy: 0.7273\n",
            "Epoch 667/5000\n",
            "1/1 [==============================] - 0s 0s/step - loss: 0.6169 - accuracy: 0.7273\n",
            "Epoch 668/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6169 - accuracy: 0.7273\n",
            "Epoch 669/5000\n",
            "1/1 [==============================] - 0s 1000us/step - loss: 0.6168 - accuracy: 0.7273\n",
            "Epoch 670/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6168 - accuracy: 0.7273\n",
            "Epoch 671/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6167 - accuracy: 0.7273\n",
            "Epoch 672/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6167 - accuracy: 0.7273\n",
            "Epoch 673/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6166 - accuracy: 0.7273\n",
            "Epoch 674/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6166 - accuracy: 0.7273\n",
            "Epoch 675/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6166 - accuracy: 0.7273\n",
            "Epoch 676/5000\n",
            "1/1 [==============================] - 0s 0s/step - loss: 0.6165 - accuracy: 0.7273\n",
            "Epoch 677/5000\n",
            "1/1 [==============================] - 0s 0s/step - loss: 0.6165 - accuracy: 0.7273\n",
            "Epoch 678/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6164 - accuracy: 0.7273\n",
            "Epoch 679/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6164 - accuracy: 0.7273\n",
            "Epoch 680/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6163 - accuracy: 0.7273\n",
            "Epoch 681/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6163 - accuracy: 0.7273\n",
            "Epoch 682/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6162 - accuracy: 0.7273\n",
            "Epoch 683/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6162 - accuracy: 0.7273\n",
            "Epoch 684/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6161 - accuracy: 0.7273\n",
            "Epoch 685/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6161 - accuracy: 0.7273\n",
            "Epoch 686/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6160 - accuracy: 0.7273\n",
            "Epoch 687/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6160 - accuracy: 0.7273\n",
            "Epoch 688/5000\n",
            "1/1 [==============================] - 0s 3ms/step - loss: 0.6159 - accuracy: 0.7273\n",
            "Epoch 689/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6159 - accuracy: 0.7273\n",
            "Epoch 690/5000\n",
            "1/1 [==============================] - 0s 1000us/step - loss: 0.6158 - accuracy: 0.7273\n",
            "Epoch 691/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6158 - accuracy: 0.7273\n",
            "Epoch 692/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6157 - accuracy: 0.7273\n",
            "Epoch 693/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6157 - accuracy: 0.7273\n",
            "Epoch 694/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6156 - accuracy: 0.7273\n",
            "Epoch 695/5000\n",
            "1/1 [==============================] - 0s 999us/step - loss: 0.6156 - accuracy: 0.7273\n",
            "Epoch 696/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6155 - accuracy: 0.7273\n",
            "Epoch 697/5000\n",
            "1/1 [==============================] - 0s 2ms/step - loss: 0.6155 - accuracy: 0.7273\n",
            "Epoch 698/5000\n",
            "1/1 [==============================] - 0s 4ms/step - loss: 0.6154 - accuracy: 0.7273\n",
            "Epoch 699/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6154 - accuracy: 0.7273\n",
            "Epoch 700/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6153 - accuracy: 0.7273\n",
            "Epoch 701/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6153 - accuracy: 0.7273\n",
            "Epoch 702/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6152 - accuracy: 0.7273\n",
            "Epoch 703/5000\n",
            "1/1 [==============================] - 0s 3ms/step - loss: 0.6152 - accuracy: 0.7273\n",
            "Epoch 704/5000\n",
            "1/1 [==============================] - 0s 1ms/step - loss: 0.6152 - accuracy: 0.7273\n",
            "Epoch 705/5000\n",
            "1/1 [==============================] - 0s 1000us/step - loss: 0.6151 - accuracy: 0.7273\n",
            "Epoch 706/5000\n",
            "1/1 [==============================] - 0s 0s/step - loss: 0.6151 - accuracy: 0.7273\n",
            "Epoch 707/5000\n",
            "1/1 [==============================] - ETA: 0s - loss: 0.6150 - accuracy: 0.7273"
          ]
        }
      ],
      "source": [
        "model = keras.Sequential([\n",
        "    keras.layers.Dense(1, input_shape=(2,), activation='sigmoid', kernel_initializer='ones', bias_initializer='zeros')\n",
        "])\n",
        "\n",
        "model.compile(optimizer='adam',\n",
        "              loss='binary_crossentropy',\n",
        "              metrics=['accuracy'])\n",
        "\n",
        "model.fit(X_train_scaled, y_train, epochs=5000)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5E98N-lK_Xro"
      },
      "source": [
        "**Evaluate the model on test set**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "scrolled": false,
        "id": "qqpM3BSU_Xro",
        "outputId": "1d3ed34e-6134-4a32-84c9-b11507026b6d"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "1/1 [==============================] - 0s 1ms/step - loss: 0.3550 - accuracy: 1.0000\n"
          ]
        },
        {
          "data": {
            "text/plain": [
              "[0.35497748851776123, 1.0]"
            ]
          },
          "execution_count": 6,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "model.evaluate(X_test_scaled,y_test)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "J-Eyr5yz_Xro",
        "outputId": "2587a1a7-333d-4aac-f898-7b18a80eb1e1"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "array([[0.70548487],\n",
              "       [0.3556955 ],\n",
              "       [0.16827849],\n",
              "       [0.47801173],\n",
              "       [0.7260697 ],\n",
              "       [0.8294984 ]], dtype=float32)"
            ]
          },
          "execution_count": 7,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "model.predict(X_test_scaled)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "scrolled": true,
        "id": "CrSE4Ywf_Xro",
        "outputId": "d427a2fb-2dba-4667-8cf5-7cc213feb367"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "2     1\n",
              "10    0\n",
              "21    0\n",
              "11    0\n",
              "14    1\n",
              "9     1\n",
              "Name: bought_insurance, dtype: int64"
            ]
          },
          "execution_count": 8,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "y_test"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "m1WbGoUN_Xrp"
      },
      "source": [
        "**Now get the value of weights and bias from the model**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "_uyK118R_Xrp"
      },
      "outputs": [],
      "source": [
        "coef, intercept = model.get_weights()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "scrolled": false,
        "id": "eaZZUu-M_Xrp",
        "outputId": "ce2d2edd-4eb6-4da4-ac52-37284cf8b982"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "(array([[5.060867 ],\n",
              "        [1.4086502]], dtype=float32),\n",
              " array([-2.9137027], dtype=float32))"
            ]
          },
          "execution_count": 10,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "coef, intercept"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "A9N8DKD-_Xrp"
      },
      "source": [
        "**This means w1=5.060867, w2=1.4086502, bias =-2.9137027**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "sJ08G7Ly_Xrp",
        "outputId": "45196845-9009-499f-977a-14e75ebe0c93"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "0.9999999847700205"
            ]
          },
          "execution_count": 11,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "def sigmoid(x):\n",
        "        import math\n",
        "        return 1 / (1 + math.exp(-x))\n",
        "sigmoid(18)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "z_BxZTZE_Xrq",
        "outputId": "b3da7edf-84c9-4f8d-8bcd-a1a090bfcdc6"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>age</th>\n",
              "      <th>affordibility</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>47</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>10</th>\n",
              "      <td>18</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>21</th>\n",
              "      <td>26</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>11</th>\n",
              "      <td>28</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>14</th>\n",
              "      <td>49</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>9</th>\n",
              "      <td>61</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "    age  affordibility\n",
              "2    47              1\n",
              "10   18              1\n",
              "21   26              0\n",
              "11   28              1\n",
              "14   49              1\n",
              "9    61              1"
            ]
          },
          "execution_count": 12,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "X_test"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "gx0eIuWA_Xrq"
      },
      "source": [
        "**Instead of model.predict, write our own prediction function that uses w1,w2 and bias**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "xVFHqKgH_Xrq",
        "outputId": "5fa9144d-a642-4389-a282-b9e1acbec09a"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "0.7054848693136117"
            ]
          },
          "execution_count": 13,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "def prediction_function(age, affordibility):\n",
        "    weighted_sum = coef[0]*age + coef[1]*affordibility + intercept\n",
        "    return sigmoid(weighted_sum)\n",
        "\n",
        "prediction_function(.47, 1)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "scrolled": false,
        "id": "idSjFsLW_Xrq",
        "outputId": "c3d6d251-b4fa-40c8-d737-3ce750f00cf5"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "0.35569549781937626"
            ]
          },
          "execution_count": 14,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "prediction_function(.18, 1)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "O0J9tYat_Xrq"
      },
      "source": [
        "**Now we start implementing gradient descent in plain python. Again the goal is to come up with same w1, w2 and bias that keras model calculated. We want to show how keras/tensorflow would have computed these values internally using gradient descent**"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Cvw6MDfo_Xrq"
      },
      "source": [
        "**First write couple of helper routines such as sigmoid and log_loss**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "TZtoYlHG_Xrq",
        "outputId": "6b948807-bab2-4bfa-d4c1-6a9505eb5258"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "array([0.99999386, 0.5       , 0.73105858])"
            ]
          },
          "execution_count": 15,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "def sigmoid_numpy(X):\n",
        "   return 1/(1+np.exp(-X))\n",
        "\n",
        "sigmoid_numpy(np.array([12,0,1]))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "6i0xg166_Xrq"
      },
      "outputs": [],
      "source": [
        "def log_loss(y_true, y_predicted):\n",
        "    epsilon = 1e-15\n",
        "    y_predicted_new = [max(i,epsilon) for i in y_predicted]\n",
        "    y_predicted_new = [min(i,1-epsilon) for i in y_predicted_new]\n",
        "    y_predicted_new = np.array(y_predicted_new)\n",
        "    return -np.mean(y_true*np.log(y_predicted_new)+(1-y_true)*np.log(1-y_predicted_new))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_YEfXij2_Xrr"
      },
      "source": [
        "**All right now comes the time to implement our own custom neural network class !! yay !!!**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "aklwAEJJ_Xrr"
      },
      "outputs": [],
      "source": [
        "class myNN:\n",
        "    def __init__(self):\n",
        "        self.w1 = 1\n",
        "        self.w2 = 1\n",
        "        self.bias = 0\n",
        "\n",
        "    def fit(self, X, y, epochs, loss_thresold):\n",
        "        self.w1, self.w2, self.bias = self.gradient_descent(X['age'],X['affordibility'],y, epochs, loss_thresold)\n",
        "        print(f\"Final weights and bias: w1: {self.w1}, w2: {self.w2}, bias: {self.bias}\")\n",
        "\n",
        "    def predict(self, X_test):\n",
        "        weighted_sum = self.w1*X_test['age'] + self.w2*X_test['affordibility'] + self.bias\n",
        "        return sigmoid_numpy(weighted_sum)\n",
        "\n",
        "    def gradient_descent(self, age,affordability, y_true, epochs, loss_thresold):\n",
        "        w1 = w2 = 1\n",
        "        bias = 0\n",
        "        rate = 0.5\n",
        "        n = len(age)\n",
        "        for i in range(epochs):\n",
        "            weighted_sum = w1 * age + w2 * affordability + bias\n",
        "            y_predicted = sigmoid_numpy(weighted_sum)\n",
        "            loss = log_loss(y_true, y_predicted)\n",
        "\n",
        "            w1d = (1/n)*np.dot(np.transpose(age),(y_predicted-y_true))\n",
        "            w2d = (1/n)*np.dot(np.transpose(affordability),(y_predicted-y_true))\n",
        "\n",
        "            bias_d = np.mean(y_predicted-y_true)\n",
        "            w1 = w1 - rate * w1d\n",
        "            w2 = w2 - rate * w2d\n",
        "            bias = bias - rate * bias_d\n",
        "\n",
        "            if i%50==0:\n",
        "                print (f'Epoch:{i}, w1:{w1}, w2:{w2}, bias:{bias}, loss:{loss}')\n",
        "\n",
        "            if loss<=loss_thresold:\n",
        "                print (f'Epoch:{i}, w1:{w1}, w2:{w2}, bias:{bias}, loss:{loss}')\n",
        "                break\n",
        "\n",
        "        return w1, w2, bias"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "scrolled": true,
        "id": "c9u6tFWB_Xrr",
        "outputId": "b65aa5a5-a661-4a4b-9ccf-9a14a699f881"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Epoch:0, w1:0.974907633470177, w2:0.948348125394529, bias:-0.11341867736368583, loss:0.7113403233723417\n",
            "Epoch:50, w1:1.503319554173139, w2:1.108384790367645, bias:-1.2319047301235464, loss:0.5675865113475955\n",
            "Epoch:100, w1:2.200713131760032, w2:1.2941584023238903, bias:-1.6607009122062801, loss:0.5390680417774752\n",
            "Epoch:150, w1:2.8495727769689085, w2:1.3696895491572745, bias:-1.986105845859897, loss:0.5176462164249294\n",
            "Epoch:200, w1:3.443016970881803, w2:1.4042218624465033, bias:-2.2571369883752723, loss:0.5005011269691375\n",
            "Epoch:250, w1:3.982450494649576, w2:1.4239127329321233, bias:-2.494377365971801, loss:0.48654089537617085\n",
            "Epoch:300, w1:4.472179522095915, w2:1.438787986553552, bias:-2.707387811922373, loss:0.4750814640632793\n",
            "Epoch:350, w1:4.917245868007634, w2:1.4525660781176122, bias:-2.901176333556766, loss:0.46561475306999006\n",
            "Epoch:366, w1:5.051047623653049, w2:1.4569794548473887, bias:-2.9596534546250037, loss:0.46293944095888917\n",
            "Final weights and bias: w1: 5.051047623653049, w2: 1.4569794548473887, bias: -2.9596534546250037\n"
          ]
        }
      ],
      "source": [
        "customModel = myNN()\n",
        "customModel.fit(X_train_scaled, y_train, epochs=8000, loss_thresold=0.4631)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "scrolled": true,
        "id": "HsXGGBSa_Xrr",
        "outputId": "cd3afb50-a961-4864-db90-371b9ac7a1a9"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "(array([[5.060867 ],\n",
              "        [1.4086502]], dtype=float32),\n",
              " array([-2.9137027], dtype=float32))"
            ]
          },
          "execution_count": 47,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "coef, intercept"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "jpgML8e9_Xrr"
      },
      "source": [
        "**This shows that in the end we were able to come up with same value of w1,w2 and bias using a plain python implementation of gradient descent function**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "scrolled": true,
        "id": "0UKL1TWC_Xrr",
        "outputId": "3063849b-99b9-4196-fcee-7fc4f4854ad5"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>age</th>\n",
              "      <th>affordibility</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>0.47</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>10</th>\n",
              "      <td>0.18</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>21</th>\n",
              "      <td>0.26</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>11</th>\n",
              "      <td>0.28</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>14</th>\n",
              "      <td>0.49</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>9</th>\n",
              "      <td>0.61</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "     age  affordibility\n",
              "2   0.47              1\n",
              "10  0.18              1\n",
              "21  0.26              0\n",
              "11  0.28              1\n",
              "14  0.49              1\n",
              "9   0.61              1"
            ]
          },
          "execution_count": 24,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "X_test_scaled"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "oTNzg6sh_Xrr"
      },
      "source": [
        "**(1) Predict using custom model**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "scrolled": false,
        "id": "AMv2lS4D_Xrr",
        "outputId": "2bc7a36f-dbb6-4d35-8afa-b7b3ba0a9973"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "2     0.705020\n",
              "10    0.355836\n",
              "21    0.161599\n",
              "11    0.477919\n",
              "14    0.725586\n",
              "9     0.828987\n",
              "dtype: float64"
            ]
          },
          "execution_count": 25,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "customModel.predict(X_test_scaled)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xLlad7v3_Xrr"
      },
      "source": [
        "**(2) Predict using tensorflow model**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Iknx-mG7_Xrs",
        "outputId": "661e9de4-bcba-4235-a150-acbb303a6fd7"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "array([[0.70548487],\n",
              "       [0.3556955 ],\n",
              "       [0.16827849],\n",
              "       [0.47801173],\n",
              "       [0.7260697 ],\n",
              "       [0.8294984 ]], dtype=float32)"
            ]
          },
          "execution_count": 26,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "model.predict(X_test_scaled)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "32Ns7ubg_Xrv"
      },
      "source": [
        "**Above you can compare predictions from our own custom model and tensoflow model. You will notice that predictions are almost same**"
      ]
    }
  ],
  "metadata": {
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.8.5"
    },
    "colab": {
      "provenance": []
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}