Multivariate_Linear_Regression.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Multivariate Linear Regression\n",
    "Today we will learn how to implement multivariate linear regression. The task is to predict the price of a house based on the size of the house in square feet and the number of bedrooms in the house. so, here we have 2 independent variables and 1 dependent variable."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Reading the dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "import os\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Size</th>\n",
       "      <th>Bedrooms</th>\n",
       "      <th>Price</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2104</td>\n",
       "      <td>3</td>\n",
       "      <td>399900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1600</td>\n",
       "      <td>3</td>\n",
       "      <td>329900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2400</td>\n",
       "      <td>3</td>\n",
       "      <td>369000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1416</td>\n",
       "      <td>2</td>\n",
       "      <td>232000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>3000</td>\n",
       "      <td>4</td>\n",
       "      <td>539900</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Size  Bedrooms   Price\n",
       "0  2104         3  399900\n",
       "1  1600         3  329900\n",
       "2  2400         3  369000\n",
       "3  1416         2  232000\n",
       "4  3000         4  539900"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.read_csv('data\\ex1data2.txt', header = None, names = ['Size', 'Bedrooms', 'Price'])\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Since the scale of the values of the variables is vastly different, there is a need to perform 'Feature Normalization'. Otherwise, the weightage of values in size variable will be much more than the values in bedrooms variable supressing the contributions of the bedrooms feature.\n",
    "\n",
    "In feature normalization, we adjust the scale of the features by subtracting the mean from each value of the feature and then dividing by the standard deviation or the range of the values in the feature."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = (df - df.mean())/df.std()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Size</th>\n",
       "      <th>Bedrooms</th>\n",
       "      <th>Price</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.130010</td>\n",
       "      <td>-0.223675</td>\n",
       "      <td>0.475747</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>-0.504190</td>\n",
       "      <td>-0.223675</td>\n",
       "      <td>-0.084074</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.502476</td>\n",
       "      <td>-0.223675</td>\n",
       "      <td>0.228626</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>-0.735723</td>\n",
       "      <td>-1.537767</td>\n",
       "      <td>-0.867025</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1.257476</td>\n",
       "      <td>1.090417</td>\n",
       "      <td>1.595389</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       Size  Bedrooms     Price\n",
       "0  0.130010 -0.223675  0.475747\n",
       "1 -0.504190 -0.223675 -0.084074\n",
       "2  0.502476 -0.223675  0.228626\n",
       "3 -0.735723 -1.537767 -0.867025\n",
       "4  1.257476  1.090417  1.595389"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Adding ones to the dataframe"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Ones</th>\n",
       "      <th>Size</th>\n",
       "      <th>Bedrooms</th>\n",
       "      <th>Price</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>0.130010</td>\n",
       "      <td>-0.223675</td>\n",
       "      <td>0.475747</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>-0.504190</td>\n",
       "      <td>-0.223675</td>\n",
       "      <td>-0.084074</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>0.502476</td>\n",
       "      <td>-0.223675</td>\n",
       "      <td>0.228626</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>-0.735723</td>\n",
       "      <td>-1.537767</td>\n",
       "      <td>-0.867025</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1</td>\n",
       "      <td>1.257476</td>\n",
       "      <td>1.090417</td>\n",
       "      <td>1.595389</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Ones      Size  Bedrooms     Price\n",
       "0     1  0.130010 -0.223675  0.475747\n",
       "1     1 -0.504190 -0.223675 -0.084074\n",
       "2     1  0.502476 -0.223675  0.228626\n",
       "3     1 -0.735723 -1.537767 -0.867025\n",
       "4     1  1.257476  1.090417  1.595389"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.insert(0, 'Ones', 1)\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Setting X (training data) and y (target variable)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "cols = df.shape[1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "X1 = df.iloc[:, 0:cols-1]\n",
    "y1 = df.iloc[:, cols-1:cols]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Convert to matrices and initialize theta"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "X1 = np.matrix(X1.values)\n",
    "y1 = np.matrix(y1.values)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "theta = np.matrix(np.array([0,0,0]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "def computeCost(X, y, theta):\n",
    "    inner = np.power(((X * theta.T) - y), 2)\n",
    "    return np.sum(inner) / (2 * len(X))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "def gradientDescent(X, y, theta, alpha, iters):\n",
    "    temp = np.matrix(np.zeros(theta.shape))\n",
    "    parameters = int(theta.ravel().shape[1])\n",
    "    cost = np.zeros(iters)\n",
    "    \n",
    "    for i in range(iters):\n",
    "        error = (X * theta.T) - y\n",
    "        \n",
    "        for j in range(parameters):\n",
    "            term  = np.multiply(error, X[:,j])\n",
    "            temp[0,j] = theta[0,j] - ((alpha/len(X)) * np.sum(term))\n",
    "            \n",
    "        theta = temp\n",
    "        cost[i] = computeCost(X1, y1, theta)\n",
    "        \n",
    "    return theta, cost"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Initialize variables for learning rate and iterations"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "alpha = 0.01  \n",
    "iters = 1000"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "# perform linear regression on the dataset\n",
    "g, cost = gradientDescent(X1, y1, theta, alpha, iters)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.13070336960771892"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# get the cost(error) of the model\n",
    "computeCost(X1, y1, g)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "###  Plotting the training progress to check that the error is decreasing with each iteration of gradient descent."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0.5,1,'Error vs Training Epoch')"
      ]
     },
     "execution_count": 77,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAtoAAAHwCAYAAACYMcj+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvAOZPmwAAIABJREFUeJzt3Xu8HWV97/HPLzcCJJCEbBIJCQkQhQiUSwQEQVtBsSKgxcNFClZbaitF0aNirdpDa6VqUascrW3R04IgotioyEVE8AYkSOQqGsItBEiAkBACIZff+WPWJis7O9l7Z6/Zsy6f9+s1r7Vm5plZv8W44jdPnnkmMhNJkiRJjTWs6gIkSZKkdmTQliRJkkpg0JYkSZJKYNCWJEmSSmDQliRJkkpg0JYkSZJKYNCWpA4XEcMjYmVETGtk21YQERdHxN9XXYek9mTQltQxIuLBiHi+FhS7ly9XXddA9ah/fY/v9I6Bni8z12XmmMx8uJFtByoi/jEi1vT4fk82+nMkaaiMqLoASRpib8nMH/fVKCJGZObavrYN9ByNkJlj6j7jQeDPt/SdyqqjJJdk5jurLkKSGsEebUkCIuKdEfGLiPh8RDwN/P1mtg2LiL+LiIciYklE/FdE7Fg7x/SIyIh4d0Q8DPykl8+5NyKOrVsfERFPRsSBETG6NpThqYh4JiLmRsSkrfgu/xgR34qISyPiWeC0iHh1RNxcO+9jEfGvETGyroaMiOm19Ytr+38UEc9GxK8iYsZA29b2vykifhcRyyPiS7X/nu/ciu/U/bl/ExEP1P6bnR8Rw2r7h0XEJ+quyzciYoe644+sff/lEfFIRPxp3eknbK5+SRoMg7YkbXAIsBDYGfjUZra9s7b8IbA7MAboOfzktcDewBt7+YxLgVPq1t8IPJmZvwbOAHYEpgI7Ae8Bnt/K7/JW4Ju1830LWAu8D5gIHA4cA/zlFo4/Ffg4MAF4GPiHgbaNiJ2By4EP1T73AeDgrfw+3Y4HDgRmAycCp9e2/zlwGvA6YA9gPPDFWh0zgB8CF1D8dz0AuLOv+iVpsAzakjrN92q9ut3LX9TtW5yZX8rMtZn5/Ga2vQO4IDMXZuZK4KPAyRFRPxTv7zPzubpz1PsmcFxEbFdbP7W2DWANRRDcszYW+rbMXLGV3/Pnmfn9zFyfmc9n5tzMvKX2PRYCX6P4C8HmXJGZ8zJzDXAJsP9WtD0WmJ+Z/1Pb93mgrzHXp/a4Ptf12H9+Zi7LzAeBf2XDX1reAXwuMx/IzGeBv62daxhFAL86My+vff8nM3P+Vn5XSeo3x2hL6jQnbGE88yP92LYL8FDd+kMUf5bWD/Ho7TwAZOaCiLgXeEtEfB84jqKHFeC/KXqzL4uIccDFwMdqAXCgNqohIvYC/gU4CNiuVvMtWzj+8br3qyh67gfadpf6OjIzI2JRH3V/s48x2vXf66HaZ3R/Vs/rMgroovhvev8WzjmQ7ypJ/WaPtiRtkP3YthjYrW59GsWwjCf6OE+97uEjxwP3ZOYCgMxck5n/JzNnAYdR9AifvvnTbFHPGv4NuIuit3wH4BNAbOW5++sxYNfulYgIYMogzzm17v00iusBvV+XF4GlFOF8j0F+riQNmEFbkgbmUuCciJgREWOAfwK+NcBZPS4D3gD8FRuGjRARfxgR+0bEcGAFxVCSdQ2qeyywHHguIvZmy+OzG+UHwIER8Zba0Jr3UfQwD8aHI2JcFPN4n00x/hyK6/KB2g2pYynG01+amesp/mXgmIj4k9pNlRMj4g8GWYck9cmgLanTfD82nqf5ygEefxHFEI+bKG7uewH4m4GcIDMfA35F0Wv9rbpdk4ErKEL2vcCNFCGxET5IcbPlsxS929/acvPBy8wngJMobkJ8iqJX+XZg9RYOe0eP67MyInaq2/99YH7tPFcC36ht/3eK7/QziptXn6UI9mTmA8BbgI8ATwO/BvZtxHeUpC2JzL7+hVOSpMGr9dQvBk7MzJ8N8NgRFD38M2o3QkpS07NHW5JUmog4JiJ2jIhtKKbQWwvcWnFZkjQkDNqSpDK9hmIox5MUc3efkJlbGjoiSW3DoSOSJElSCezRliRJkkpg0JYkSZJK0DZPhpw4cWJOnz696jIkSZLU5m677bYnM7PP5wK0TdCePn068+bNq7oMSZIktbmIeKg/7Rw6IkmSJJXAoC1JkiSVwKAtSZIklcCgLUmSJJXAoC1JkiSVwKAtSZIklcCgLUmSJJXAoC1JkiSVwKAtSZIklcCgLUmSJJXAoC1JkiSVwKAtSZIklcCgLUmSJJXAoC1JkiSVwKAtSZIklcCgLUmSJJXAoD1Yzz0HK1ZUXYUkSZKajEF7sGbNgrPPrroKSZIkNRmD9mCNGwfLllVdhSRJkpqMQXuwxo83aEuSJGkTBu3BMmhLkiSpFwbtwRo/Hp55puoqJEmS1GQM2oNlj7YkSZJ6YdAerPHjiyn+1qypuhJJkiQ1EYP2YI0bV7zaqy1JkqQ6Bu3BGj++eDVoS5IkqY5Be7C6g7Y3REqSJKmOQXuw7NGWJElSLwzag2XQliRJUi8M2oPlzZCSJEnqhUF7sOzRliRJUi8M2oO1zTaw7bbeDClJkqSNGLQbwadDSpIkqQeDdiMYtCVJktSDQbsRxo0zaEuSJGkjBu1GsEdbkiRJPRi0G2H8eG+GlCRJ0kYM2o1gj7YkSZJ6MGg3wvjxsHw5rFtXdSWSJElqEgbtRuh+OuTy5dXWIUmSpKZh0G4Enw4pSZKkHgzajdAdtL0hUpIkSTUG7UawR1uSJEk9GLQbwaAtSZKkHgzajdB9M6RBW5IkSTUG7UawR1uSJEk9GLQbYbvtYORIg7YkSZJeYtBuhAgfwy5JkqSNlBq0I+KYiLgvIhZExLlbaHdiRGREzK6tT4+I5yNifm35apl1NoSPYZckSVKdEWWdOCKGAxcCRwOLgLkRMScz7+nRbixwNnBLj1Pcn5n7l1Vfw40bZ9CWJEnSS8rs0T4YWJCZCzPzReAy4Phe2v0D8BnghRJrKZ892pIkSapTZtCeAjxSt76otu0lEXEAMDUzf9DL8TMi4vaIuDEijiixzsYwaEuSJKlOaUNHgOhlW760M2IY8Hngnb20ewyYlplPRcRBwPci4pWZuWKjD4g4EzgTYNq0aY2qe+t4M6QkSZLqlNmjvQiYWre+K7C4bn0ssA/w04h4EDgUmBMRszNzdWY+BZCZtwH3Ay/v+QGZ+bXMnJ2Zs7u6ukr6Gv3UHbQz+24rSZKktldm0J4LzIyIGRExCjgZmNO9MzOXZ+bEzJyemdOBm4HjMnNeRHTVbqYkInYHZgILS6x18MaNg3Xr4Nlnq65EkiRJTaC0oJ2Za4GzgGuAe4HLM/PuiDgvIo7r4/AjgTsi4jfAFcB7MvPpsmptiJ12Kl6fbu4yJUmSNDTKHKNNZl4FXNVj2yc20/Z1de+/A3ynzNoabsKE4vWpp2D69EpLkSRJUvV8MmSjdPdoP/VUtXVIkiSpKRi0G8WgLUmSpDoG7UYxaEuSJKmOQbtR6sdoS5IkqeMZtBtlxAjYcUeDtiRJkgCDdmNNmOD0fpIkSQIM2o210072aEuSJAkwaDeWQVuSJEk1Bu1GMmhLkiSpxqDdSAZtSZIk1Ri0G2mnnWD5cli7tupKJEmSVDGDdiN1P7TGmUckSZI6nkG7kbofWmPQliRJ6ngG7UbyMeySJEmqMWg3kkFbkiRJNQbtRjJoS5Ikqcag3UgGbUmSJNUYtBtp7FgYMcKgLUmSJIN2Q0X40BpJkiQBBu3GmzDB6f0kSZJk0G44e7QlSZKEQbvxDNqSJEnCoN14Bm1JkiRh0G687qCdWXUlkiRJqpBBu9F22glWr4ZVq6quRJIkSRUyaDfahAnFq8NHJEmSOppBu9G6nw7pFH+SJEkdzaDdaD6GXZIkSRi0G8+gLUmSJAzajWfQliRJEgbtxjNoS5IkCYN2440aBWPHwpNPVl2JJEmSKmTQLkNXFyxdWnUVkiRJqpBBuwwGbUmSpI5n0C5DVxcsWVJ1FZIkSaqQQbsM9mhLkiR1PIN2GXbeuQjamVVXIkmSpIoYtMvQ1QVr1sCKFVVXIkmSpIoYtMvQ1VW8OnxEkiSpYxm0y2DQliRJ6ngG7TIYtCVJkjqeQbsMBm1JkqSOZ9AuQ3fQdi5tSZKkjmXQLsN228H229ujLUmS1MEM2mXxoTWSJEkdzaBdFoO2JElSRzNol8WgLUmS1NEM2mUxaEuSJHW0UoN2RBwTEfdFxIKIOHcL7U6MiIyI2XXbPlo77r6IeGOZdZaiO2hnVl2JJEmSKlBa0I6I4cCFwJuAWcApETGrl3ZjgbOBW+q2zQJOBl4JHAP839r5WkdXF7zwAjz3XNWVSJIkqQJl9mgfDCzIzIWZ+SJwGXB8L+3+AfgM8ELdtuOByzJzdWY+ACyona917Lxz8epc2pIkSR2pzKA9BXikbn1RbdtLIuIAYGpm/mCgxzY9nw4pSZLU0coM2tHLtpcGLEfEMODzwAcHemzdOc6MiHkRMW9pswVag7YkSVJHKzNoLwKm1q3vCiyuWx8L7AP8NCIeBA4F5tRuiOzrWAAy82uZOTszZ3d1B9tmYdCWJEnqaGUG7bnAzIiYERGjKG5unNO9MzOXZ+bEzJyemdOBm4HjMnNerd3JEbFNRMwAZgK3llhr4xm0JUmSOtqIsk6cmWsj4izgGmA4cFFm3h0R5wHzMnPOFo69OyIuB+4B1gLvzcx1ZdVaiu23h9GjDdqSJEkdqrSgDZCZVwFX9dj2ic20fV2P9U8BnyqtuLJF+NAaSZKkDuaTIcu0884GbUmSpA5l0C5TV5fzaEuSJHUog3aZHDoiSZLUsQzaZXLoiCRJUscyaJdp0iRYtQpWrqy6EkmSJA0xg3aZJk8uXh9/vNo6JEmSNOQM2mWaNKl4NWhLkiR1HIN2mbp7tJ94oto6JEmSNOQM2mWyR1uSJKljGbTLNHEiDBtm0JYkSepABu0yDR9eTPHn0BFJkqSOY9Au26RJ9mhLkiR1IIN22SZPNmhLkiR1IIN22SZPduiIJElSBzJol6176Ehm1ZVIkiRpCBm0yzZ5Mrz4IjzzTNWVSJIkaQgZtMvWPZe2w0ckSZI6ikG7bN1Ph/SGSEmSpI5i0C6bj2GXJEnqSAbtsvkYdkmSpI5k0C7b+PEwcqRBW5IkqcMYtMs2bFjRq+3QEUmSpI5i0B4KPoZdkiSp4xi0h4KPYZckSeo4Bu2h4NARSZKkjmPQHgqTJxdBe/36qiuRJEnSEDFoD4XJk2HdOnjqqaorkSRJ0hAxaA8FH8MuSZLUcQzaQ8HHsEuSJHUcg/ZQMGhLkiR1HIP2UHjZy4rXxx6rtg5JkiQNGYP2UBg7FsaMgUcfrboSSZIkDRGD9lCZMgUWL666CkmSJA0Rg/ZQ2WUXe7QlSZI6iEF7qNijLUmS1FEM2kNll12KoJ1ZdSWSJEkaAgbtobLLLvDiiz4dUpIkqUMYtIfKlCnFq8NHJEmSOoJBe6jsskvx6g2RkiRJHcGgPVTs0ZYkSeooBu2h0v10SHu0JUmSOoJBe6iMGgVdXfZoS5IkdQiD9lDqnuJPkiRJbc+gPZR8OqQkSVLHMGgPJZ8OKUmS1DEM2kNpl13giSdgzZqqK5EkSVLJDNpDacqU4hHsTzxRdSWSJEkqWalBOyKOiYj7ImJBRJzby/73RMSdETE/In4eEbNq26dHxPO17fMj4qtl1jlkfGiNJElSxxhR1okjYjhwIXA0sAiYGxFzMvOeumbfzMyv1tofB1wAHFPbd39m7l9WfZXoDtqO05YkSWp7ZfZoHwwsyMyFmfkicBlwfH2DzFxRt7o9kCXWUz2fDilJktQxygzaU4BH6tYX1bZtJCLeGxH3A58Bzq7bNSMibo+IGyPiiBLrHDpdXTBihENHJEmSOkCZQTt62bZJj3VmXpiZewAfAf6utvkxYFpmHgB8APhmROywyQdEnBkR8yJi3tKlSxtYekmGDSsexW6PtiRJUtsrM2gvAqbWre8KbClhXgacAJCZqzPzqdr724D7gZf3PCAzv5aZszNzdldXV8MKL5UPrZEkSeoIZQbtucDMiJgREaOAk4E59Q0iYmbd6puB39e2d9VupiQidgdmAgtLrHXoTJli0JYkSeoApc06kplrI+Is4BpgOHBRZt4dEecB8zJzDnBWRBwFrAGWAWfUDj8SOC8i1gLrgPdk5tNl1Tqkpk6Fa68t5tOO3kbXSJIkqR2UFrQBMvMq4Koe2z5R9/59mznuO8B3yqytMlOnwsqV8MwzMH581dVIkiSpJD4ZcqhNm1a8PvLIlttJkiSppRm0h9rU2v2hDz9cbR2SJEkqlUF7qNmjLUmS1BEM2kNt8mQYOdIebUmSpDZn0B5qw4YVU/zZoy1JktTWDNpVmDbNHm1JkqQ2Z9CuwtSp9mhLkiS1OYN2FaZNg0WLYN26qiuRJElSSQzaVZg6FdauhSeeqLoSSZIklcSgXYXuKf4cpy1JktS2DNpV6H5ojeO0JUmS2pZBuwr2aEuSJLU9g3YVdtwRxoyxR1uSJKmNGbSrEOFc2pIkSW3OoF0V59KWJElqawbtqtijLUmS1NYM2lWZOhWWLIEXXqi6EkmSJJXAoF2V7plHFi2qtg5JkiSVwqBdle65tB0+IkmS1JYM2lWZPr14ffDBKquQJElSSQzaVZk6FYYNgwceqLoSSZIklcCgXZWRI4uwbdCWJElqSwbtKs2YYdCWJElqUwbtKhm0JUmS2pZBu0ozZsBjj8Hzz1ddiSRJkhrMoF2lGTOK14ceqrYOSZIkNZxBu0rdQdvhI5IkSW3HoF0lg7YkSVLbMmhXafJk2GYbg7YkSVIbMmhXadiw4gmRBm1JkqS2Y9CumlP8SZIktSWDdtUM2pIkSW3JoF21GTNg2TJYvrzqSiRJktRABu2qOfOIJElSWzJoV2369OLVoC1JktRWDNpVs0dbkiSpLRm0qzZhAowda9CWJElqM/0K2hHx3/3Zpq0QAbvvDgsXVl2JJEmSGqi/PdqvrF+JiOHAQY0vp0PtuScsWFB1FZIkSWqgLQbtiPhoRDwL7BcRK2rLs8AS4H+GpMJOMHNmMXRk7dqqK5EkSVKDbDFoZ+anM3Ms8NnM3KG2jM3MnTLzo0NUY/vbc09YswYefrjqSiRJktQg/R068oOI2B4gIk6LiAsiYrcS6+osM2cWr7//fbV1SJIkqWH6G7S/AqyKiD8APgw8BPxXaVV1GoO2JElS2+lv0F6bmQkcD3wxM78IjC2vrA4zeTJsv703REqSJLWREf1s92xEfBT4U+CI2qwjI8srq8NEFOO07dGWJElqG/3t0T4JWA28KzMfB6YAny2tqk40c6ZBW5IkqY30K2jXwvUlwI4RcSzwQmY6RruRnOJPkiSprfT3yZD/C7gVeDvwv4BbIuLEMgvrODNnFiH7oYeqrkSSJEkN0N+hIx8DXpWZZ2Tm6cDBwMf7OigijomI+yJiQUSc28v+90TEnRExPyJ+HhGz6vZ9tHbcfRHxxv5+oZblzCOSJEltpb9Be1hmLqlbf6qvY2s3TF4IvAmYBZxSH6RrvpmZ+2bm/sBngAtqx84CTqZ49PsxwP+tna997bln8WrQliRJagv9nXXk6oi4Bri0tn4ScFUfxxwMLMjMhQARcRnF9ID3dDfIzBV17bcHsvb+eOCyzFwNPBARC2rn+1U/6209kybBmDFO8SdJktQmthi0I2JPYFJmfigi3ga8BgiKwHtJH+eeAjxSt74IOKSXz3gv8AFgFPBHdcfe3OPYKX18XmuLcOYRSZKkNtLX0JEvAM8CZOZ3M/MDmXkORW/2F/o4NnrZlptsyLwwM/cAPgL83UCOjYgzI2JeRMxbunRpH+W0AIO2JElS2+graE/PzDt6bszMecD0Po5dBEytW98VWLyF9pcBJwzk2Mz8WmbOzszZXV1dfZTTAvbcEx58ENasqboSSZIkDVJfQXv0FvZt28exc4GZETEjIkZR3Nw4p75BRMysW30z0N2dOwc4OSK2iYgZwEyK6QXb28tfXkzx9+CDVVciSZKkQeoraM+NiL/ouTEi3g3ctqUDM3MtcBZwDXAvcHlm3h0R50XEcbVmZ0XE3RExn2Kc9hm1Y+8GLqe4cfJq4L2ZuW4A36s17bVX8XrvvdXWIUmSpEGLzE2GPm/YGTEJuBJ4kQ3BejbFjYtvrT0xsinMnj07582bV3UZg7N8OYwbB+efDx/5SNXVSJIkqRcRcVtmzu6r3RZnHcnMJ4DDIuIPgX1qm3+YmT9pQI3qaccd4WUvs0dbkiSpDfRrHu3MvAG4oeRaBLD33gZtSZKkNtDfJ0NqqHQH7S0M6ZEkSVLzM2g3m733hmefhcVbmglRkiRJzc6g3Wz23rt4dfiIJElSSzNoN5vuoP3b31ZbhyRJkgbFoN1sJk8uZh+xR1uSJKmlGbSbTYQzj0iSJLUBg3Yz2msvg7YkSVKLM2g3o733hscfh2eeqboSSZIkbSWDdjNy5hFJkqSWZ9BuRgZtSZKklmfQbkYzZsA228A991RdiSRJkraSQbsZDR9e9GrfdVfVlUiSJGkrGbSb1X77wZ13Vl2FJEmStpJBu1ntuy8sXgxPPVV1JZIkSdoKBu1mtd9+xau92pIkSS3JoN2s9t23eL3jjmrrkCRJ0lYxaDeryZNh4kSDtiRJUosyaDerCG+IlCRJamEG7Wa2777FFH/r11ddiSRJkgbIoN3M9tsPVq2ChQurrkSSJEkDZNBuZt4QKUmS1LIM2s3sla8sxmo7TluSJKnlGLSb2XbbwZ572qMtSZLUggzazc6ZRyRJklqSQbvZ7bcfLFgAK1dWXYkkSZIGwKDd7A48EDLhN7+puhJJkiQNgEG72R14YPF6223V1iFJkqQBMWg3u112KR7H/utfV12JJEmSBsCg3QoOOsgebUmSpBZj0G4FBx4I99xTPCVSkiRJLcGg3QoOOgjWr3c+bUmSpBZi0G4F3hApSZLUcgzarWDXXaGryxsiJUmSWohBuxVEeEOkJElSizFot4oDD4S774YXXqi6EkmSJPWDQbtVHHQQrF0Ld95ZdSWSJEnqB4N2q/CGSEmSpJZi0G4Vu+1W3BB5yy1VVyJJkqR+MGi3igh49avh5purrkSSJEn9YNBuJYceCr/9LSxbVnUlkiRJ6oNBu5Ucemjxeuut1dYhSZKkPhm0W8ns2TBsGPzqV1VXIkmSpD4YtFvJ2LGwzz6O05YkSWoBBu1Wc+ihxcwj69dXXYkkSZK2wKDdag49FJ55Bn73u6orkSRJ0hYYtFtN9w2RjtOWJElqaqUG7Yg4JiLui4gFEXFuL/s/EBH3RMQdEXF9ROxWt29dRMyvLXPKrLOlvOIVsOOOjtOWJElqciPKOnFEDAcuBI4GFgFzI2JOZt5T1+x2YHZmroqIvwI+A5xU2/d8Zu5fVn0ta9gwOOQQe7QlSZKaXJk92gcDCzJzYWa+CFwGHF/fIDNvyMxVtdWbgV1LrKd9HH443HWXD66RJElqYmUG7SnAI3Xri2rbNufdwI/q1kdHxLyIuDkiTujtgIg4s9Zm3tKlSwdfcas48kjIhF/8oupKJEmStBllBu3oZVv22jDiNGA28Nm6zdMyczZwKvCFiNhjk5Nlfi0zZ2fm7K6urkbU3BoOOQRGjoSbbqq6EkmSJG1GmUF7ETC1bn1XYHHPRhFxFPAx4LjMXN29PTMX114XAj8FDiix1tay7bZw8MEGbUmSpCZWZtCeC8yMiBkRMQo4Gdho9pCIOAD4N4qQvaRu+/iI2Kb2fiJwOFB/E6Ve+1q47TZYubLqSiRJktSL0oJ2Zq4FzgKuAe4FLs/MuyPivIg4rtbss8AY4Ns9pvHbG5gXEb8BbgDO7zFbiY48EtaudZo/SZKkJlXa9H4AmXkVcFWPbZ+oe3/UZo77JbBvmbW1vMMOK6b6u/FGOKrX/4ySJEmqkE+GbFVjx8KBBzpOW5IkqUkZtFvZa18Lt9wCL7xQdSWSJEnqwaDdyo48ElavhltvrboSSZIk9WDQbmVHHFGM077++qorkSRJUg8G7VY2fjzMng3XXVd1JZIkSerBoN3qjj66GDqyfHnVlUiSJKmOQbvVHX00rFsHP/1p1ZVIkiSpjkG71b361bD99g4fkSRJajIG7VY3alQxzZ9BW5IkqakYtNvB0UfD734HDz9cdSWSJEmqMWi3g6OPLl7t1ZYkSWoaBu12MGsW7LKLQVuSJKmJGLTbQQS88Y1w7bWwdm3V1UiSJAmDdvs49lhYtgx++cuqK5EkSRIG7fZx9NEwciT84AdVVyJJkiQM2u1j7Fh43esM2pIkSU3CoN1Ojj0W7r0X7r+/6kokSZI6nkG7nbz5zcXrD39YbR2SJEkyaLeVPfaAvfeG73+/6kokSZI6nkG73Rx7LNx4I6xYUXUlkiRJHc2g3W7e8hZYswZ+9KOqK5EkSepoBu12c9hhMHkyXHFF1ZVIkiR1NIN2uxk+HN72NrjqKnjuuaqrkSRJ6lgG7XZ04omwahVcfXXVlUiSJHUsg3Y7OuII6Opy+IgkSVKFDNrtaMQIeOtbi6dEPv981dVIkiR1JIN2uzrxRFi5Eq69tupKJEmSOpJBu1297nUwYQJcfnnVlUiSJHUkg3a7GjkS3v52+N73ip5tSZIkDSmDdjs77bRi9pErr6y6EkmSpI5j0G5nhx8O06fDxRdXXYkkSVLHMWi3s4iiV/vHP4bHHqu6GkmSpI5i0G53p50G69fDpZdWXYkkSVJHMWi3u1e8Al71Kvjv/666EkmSpI5i0O4Ep58O8+cXiyRJkoaEQbsTnHoqbLMN/Pu/V12JJElSxzBod4IJE4o5tS++uJjuT5IkSaUzaHeKv/gLWLHCJ0VKkiQNEYN2pzjiiOLGSIePSJIkDQmDdqeIgDPPhF/+Eu6+u+pqJEmS2p5Bu5OcfjqMGgVf+UrVlUiSJLU9g3YnmTgRTjkFvvENeOaZqquRJElqawbtTvO+98Fzz8F//mfVlUiSJLU1g3anOeAAOPJI+NKXYO3aqquRJElqWwbtTvT+98NDD8GcOVVXIkmS1LYM2p393ey7AAAVf0lEQVTouONg+nT4/OerrkSSJKltGbQ70fDhcPbZ8POfF9P9SZIkqeFKDdoRcUxE3BcRCyLi3F72fyAi7omIOyLi+ojYrW7fGRHx+9pyRpl1dqQzz4SddoJ/+qeqK5EkSWpLpQXtiBgOXAi8CZgFnBIRs3o0ux2YnZn7AVcAn6kdOwH4JHAIcDDwyYgYX1atHWn77Yux2j/8IcyfX3U1kiRJbafMHu2DgQWZuTAzXwQuA46vb5CZN2TmqtrqzcCutfdvBK7LzKczcxlwHXBMibV2pve+F8aOhU9/uupKJEmS2k6ZQXsK8Ejd+qLats15N/CjrTxWW2P8ePjrv4Zvfxvuu6/qaiRJktpKmUE7etmWvTaMOA2YDXx2IMdGxJkRMS8i5i1dunSrC+1o55wDo0fDeedVXYkkSVJbKTNoLwKm1q3vCizu2SgijgI+BhyXmasHcmxmfi0zZ2fm7K6uroYV3lEmTSqeFnnppXDHHVVXI0mS1DbKDNpzgZkRMSMiRgEnAxs9ISUiDgD+jSJkL6nbdQ3whogYX7sJ8g21bSrDhz4EO+wAH/941ZVIkiS1jdKCdmauBc6iCMj3Apdn5t0RcV5EHFdr9llgDPDtiJgfEXNqxz4N/ANFWJ8LnFfbpjJMmFCE7Tlz4JZbqq5GkiSpLURmr8OmW87s2bNz3rx5VZfRulauhN13h332geuvh+htmLwkSZIi4rbMnN1XO58MqcKYMfCJT8ANN8D3v191NZIkSS3PoK0N/vIvYe+94YMfhNWr+24vSZKkzTJoa4ORI+GCC2DBAvjyl6uuRpIkqaUZtLWxY46BN72pmFd7yZK+20uSJKlXBm1t6l/+BZ5/vhhCIkmSpK1i0Nam9t4bPvxhuPhi+PGPq65GkiSpJRm01buPfQz23BPe856id1uSJEkDYtBW77bdFr76Vbj/fvjHf6y6GkmSpJZj0Nbmvf71cPrp8JnPwB13VF2NJElSSzFoa8v+5V+KR7Sfdppza0uSJA2AQVtbNnEiXHQR3HknfPzjVVcjSZLUMgza6tub31w8NfJzn4Mbb6y6GkmSpJZg0Fb/fO5zsMcexZjtZ56puhpJkqSmZ9BW/4wZU8yr/eij8O53Q2bVFUmSJDU1g7b675BD4J//Gb77XbjggqqrkSRJamoGbQ3MBz4Ab3sbfOQjcNNNVVcjSZLUtAzaGpgI+PrXYffd4aST4LHHqq5IkiSpKRm0NXA77FAMH1mxAk44AVatqroiSZKkpmPQ1tbZZx/45jdh7lw44wxYv77qiiRJkpqKQVtb7/jji2n/rrgC/u7vqq5GkiSpqYyougC1uHPOgd/9Dj79aZg+Hc48s+qKJEmSmoJBW4MTAV/6Ejz8MLznPbDjjsVNkpIkSR3OoSMavJEji+Ejr3kNnHYaXHVV1RVJkiRVzqCtxthuO/j+92G//eBP/gR++tOqK5IkSaqUQVuNs+OOcPXVxRzbf/zH8OMfV12RJElSZQzaaqyuLrjhBthzTzj2WPjRj6quSJIkqRIGbTXezjsXYXvWrOKBNldeWXVFkiRJQ86grXLstBNcfz0ceGAxZvvCC6uuSJIkaUgZtFWe8eOLsP2Wt8BZZ8GHP+wTJCVJUscwaKtc220H3/0u/NVfwWc/C+94B7zwQtVVSZIklc6grfINH14MHTn/fLjsMjjiCHjkkaqrkiRJKpVBW0MjAj7yEfje9+C+++Cgg5xrW5IktTWDtobW8cfDrbcWN0sedRRccAFkVl2VJElSwxm0NfT22gtuuQWOOw4++MFivu0lS6quSpIkqaEM2qrGDjvAd74DX/5yMTPJfvvBtddWXZUkSVLDGLRVnQh473th7lyYOBHe+EY4+2xYubLqyiRJkgbNoK3q7btvEbbPPrvo4d5nH7juuqqrkiRJGhSDtprDttvCF78IP/sZjB4Nb3gDvOtdsGxZ1ZVJkiRtFYO2msvhh8P8+fDRj8J//Vdx4+RFF/lESUmS1HIM2mo+o0fDP/0TzJsHM2fCu98NhxwCv/pV1ZVJkiT1m0FbzWv//YuhJJdcAosXw2GHwemnw0MPVV2ZJElSnwzaam4RcOqpxdMkzz0XLr8cXv5yeP/7nXtbkiQ1NYO2WsOYMfDpT8Pvfw9/+qfwpS/BHnvAJz8JzzxTdXWSJEmbMGirtUydCv/xH3D33XDMMXDeebDbbvC3f2sPtyRJaioGbbWmvfaCb38bbr+9CNznn18E7rPPhocfrro6SZIkg7Za3P77w7e+BffeC6ecAl/5SjGk5JRT4OabIbPqCiVJUocyaKs9vOIVxXzb998Pf/M3cNVV8OpXF9MCXnwxrF5ddYWSJKnDGLTVXqZNgwsugEcfhQsvhBUripsnp02DD3+4mL1EkiRpCJQatCPimIi4LyIWRMS5vew/MiJ+HRFrI+LEHvvWRcT82jKnzDrVhsaMgb/+a7jnHrjmmqJ3+4ILirHdr3kNfP3r8NxzVVcpSZLaWGlBOyKGAxcCbwJmAadExKwezR4G3gl8s5dTPJ+Z+9eW48qqU21u2DB4wxvge9+DRYvgn/8Zli6Fd70LJk+GP/uzIoivXVt1pZIkqc2U2aN9MLAgMxdm5ovAZcDx9Q0y88HMvANYX2IdUmHy5GL4yG9/Wzxx8u1vh+9+t5i1ZJddih7wn/0M1vs/R0mSNHhlBu0pwCN164tq2/prdETMi4ibI+KE3hpExJm1NvOWLl06mFrVSSKK4SMXXQRPPAFXXgl/+IfwjW/AkUcW0wS+//3wk5/AmjVVVytJklpUmUE7etk2kLnWpmXmbOBU4AsRsccmJ8v8WmbOzszZXV1dW1unOtno0XDCCcUUgUuWwCWXwAEHwFe/Cq9/Pey8M5x2WvHo9xUrqq5WkiS1kDKD9iJgat36rsDi/h6cmYtrrwuBnwIHNLI4aRNjxsCpp8KcOfDUU8WwkuOPh6uvhpNOgokTi2EmX/hCcZOlc3RLkqQtKDNozwVmRsSMiBgFnAz0a/aQiBgfEdvU3k8EDgfuKa1Sqaftt4e3vrUYTvLEE3DTTfC+98EDD8A558ArX1k8Dv5d74LLLoMnn6y6YkmS1GQiS+yVi4g/Br4ADAcuysxPRcR5wLzMnBMRrwKuBMYDLwCPZ+YrI+Iw4N8obpIcBnwhM/9zS581e/bsnDdvXmnfRXrJgw/CddcVy49/DMuWFeO+998fXvvaYpz3EUcUPeCSJKntRMRttSHOW25XZtAeSgZtVWLdOrjtNrj2Wrj++uKx7y+8UOybNasI3d3LlIHcCyxJkpqVQVuqwurVRfC+6aZi+fnP4dlni31Tp8LBBxePhT/kEDjooGKIiiRJaikGbakZrF0Ld9xRhO5bb4VbboGFC4t9w4bBPvsUofvgg+HAA4te8NGjq61ZkiRtkUFbalZLl8LcuUXovuWWIoAvW1bsGzGieEz8/vtvvOy0U7U1S5Kklxi0pVaRCfffD/Pnb7w8+uiGNrvuCvvtV/R4dy977w077FBd3ZIkdaj+Bu0RQ1GMpC2IgD33LJYTT9ywfelS+M1vitB9++1w113FLCcvvrihza67bhy+99oLZs6ESZOK80qSpMrYoy21krVri7m87723eGhO93LvvbBq1YZ222+/IbzvuWcRvrvfv+xlxfhwSZK0VezRltrRiBFFaJ45E447bsP29evh4Yfht78thqEsWFAsd91VPOlyzZoNbbfdFvbYA3bbrfdl0iSDuCRJDWDQltrBsGEwfXqx9LRuHTzyyIbw3b08+CD84hfwzDMbtx81CqZNK5bu8D1lCuyyy4Zl4kTDuCRJfTBoS+1u+PANIfyoozbdv2JF0Rv+0EObLldfDY89tukxI0cWQ1Dqw3f9MmkS7LxzEchH+MeMJKkz+f+AUqfbYYdiPu999ul9/4svwuOPw+LFxfLooxveL15cDFf5yU827RnvNmFCEbq7uorXzb2fOBHGjy9CvCRJbcCgLWnL6oeSbMmqVUXv9+LFsGTJxsvSpcXrPffAjTfCU08V0xr2ZuzYIpxPmFAE7+73m9s2fjyMG1fcAOpwFklSEzFoS2qM7bYrbrLcY4++265dW4Tt7gC+ZAk8+WTx4J6nn96wLFsGd9+9Yb3+ps6eIoqQvsMOsOOOxWv30nO957axY4ug3r1su63TI0qSBs2gLWnojRhRjOOeNKn/x2TCc89tGsaffroYZ75iBSxfvuH9ihVF24ce2rC+cmX/Piti4+A90GX06GLZdtstvx850kAvSW3MoC2pNUTAmDHFMnXq1p1j3Tp49tmNw3j38txzmy4rV2667cknN22zfv3W1TNsWP9D+ahR5S8jRmy8DB/uXwQkaRAM2pI6x/DhxXjuceMad85MWL16Q/BetQpeeKFYnn9+cO+fe64YYvP888WwmRdf3HTZ0nCaRhg+fEPwHjly0zDen6Wv44YPL/7S0YyvEcUybFjf7/vbroz3/oVIakoGbUkajIgNvc877TT0n5+5+RA+kGX16qLHf+3aTZc1a3rf3teyZk1x3r7arF9ffPaWXrf2Xw06SX/CeH0o789rWW3LPn+ZtfS0pb/kDMUxVX/+UB2zue0f+lDvU9c2CYO2JLWyiA1DP9pZZrH0Fcgb+bpu3YbPzSy2N+J9I8+1NZ/d/d+zr9ey2pZ9/jJr6Wlz24fqmDI+v7f9zfw9y/5XvUEyaEuSml99D60ktQj/xJIkSZJKYNCWJEmSSmDQliRJkkpg0JYkSZJKYNCWJEmSSmDQliRJkkpg0JYkSZJKYNCWJEmSSmDQliRJkkpg0JYkSZJKYNCWJEmSSmDQliRJkkpg0JYkSZJKYNCWJEmSSmDQliRJkkpg0JYkSZJKYNCWJEmSSmDQliRJkkoQmVl1DQ0REUuBhyr6+InAkxV9toaO17kzeJ3bn9e4M3idO0NV13m3zOzqq1HbBO0qRcS8zJxddR0ql9e5M3id25/XuDN4nTtDs19nh45IkiRJJTBoS5IkSSUwaDfG16ouQEPC69wZvM7tz2vcGbzOnaGpr7NjtCVJkqQS2KMtSZIklcCgPQgRcUxE3BcRCyLi3Krr0daLiKkRcUNE3BsRd0fE+2rbJ0TEdRHx+9rr+Nr2iIh/rV37OyLiwGq/gQYiIoZHxO0R8YPa+oyIuKV2nb8VEaNq27eprS+o7Z9eZd3qv4gYFxFXRMRva7/rV/t7bj8RcU7tz+y7IuLSiBjt77n1RcRFEbEkIu6q2zbg329EnFFr//uIOKOK72LQ3koRMRy4EHgTMAs4JSJmVVuVBmEt8MHM3Bs4FHhv7XqeC1yfmTOB62vrUFz3mbXlTOArQ1+yBuF9wL116/8MfL52nZcB765tfzewLDP3BD5fa6fW8EXg6szcC/gDiuvt77mNRMQU4GxgdmbuAwwHTsbfczv4BnBMj20D+v1GxATgk8AhwMHAJ7vD+VAyaG+9g4EFmbkwM18ELgOOr7gmbaXMfCwzf117/yzF/ylPobim/6/W7P8BJ9TeHw/8VxZuBsZFxMuGuGxthYjYFXgz8B+19QD+CLii1qTnde6+/lcAr6+1VxOLiB2AI4H/BMjMFzPzGfw9t6MRwLYRMQLYDngMf88tLzNvAp7usXmgv983Atdl5tOZuQy4jk3De+kM2ltvCvBI3fqi2ja1uNo/Jx4A3AJMyszHoAjjwM61Zl7/1vUF4MPA+tr6TsAzmbm2tl5/LV+6zrX9y2vt1dx2B5YCX68NEfqPiNgef89tJTMfBT4HPEwRsJcDt+HvuV0N9PfbFL9rg/bW6+1vwU7h0uIiYgzwHeD9mbliS0172eb1b3IRcSywJDNvq9/cS9Psxz41rxHAgcBXMvMA4Dk2/DNzb7zOLag2DOB4YAawC7A9xTCCnvw9t7fNXdemuN4G7a23CJhat74rsLiiWtQAETGSImRfkpnfrW1+ovufkGuvS2rbvf6t6XDguIh4kGK41x9R9HCPq/3TM2x8LV+6zrX9O7LpP2eq+SwCFmXmLbX1KyiCt7/n9nIU8EBmLs3MNcB3gcPw99yuBvr7bYrftUF7680FZtbubh5FcQPGnIpr0laqjdP7T+DezLygbtccoPtO5TOA/6nbfnrtbudDgeXd/6Sl5pWZH83MXTNzOsVv9ieZ+Q7gBuDEWrOe17n7+p9Ya28PWJPLzMeBRyLiFbVNrwfuwd9zu3kYODQitqv9Gd59nf09t6eB/n6vAd4QEeNr//rxhtq2IeUDawYhIv6YojdsOHBRZn6q4pK0lSLiNcDPgDvZMHb3bynGaV8OTKP4Q/3tmfl07Q/1L1PcWLEK+LPMnDfkhWurRcTrgP+dmcdGxO4UPdwTgNuB0zJzdUSMBv6bYsz+08DJmbmwqprVfxGxP8UNr6OAhcCfUXQu+XtuIxHxf4CTKGaOuh34c4pxuP6eW1hEXAq8DpgIPEExe8j3GODvNyLeRfH/5QCfysyvD+X3AIO2JEmSVAqHjkiSJEklMGhLkiRJJTBoS5IkSSUwaEuSJEklMGhLkiRJJTBoS1KTi4iVtdfpEXFqg8/9tz3Wf9nI80tSJzNoS1LrmA4MKGhHxPA+mmwUtDPzsAHWJEnaDIO2JLWO84EjImJ+RJwTEcMj4rMRMTci7oiIv4TiYTwRcUNEfJPiIUxExPci4raIuDsizqxtOx/Ytna+S2rbunvPo3buuyLizog4qe7cP42IKyLitxFxSe2BEUTE+RFxT62Wzw35fx1JajIjqi5AktRv51J7miVALTAvz8xXRcQ2wC8i4tpa24OBfTLzgdr6u2pPUdsWmBsR38nMcyPirMzcv5fPehuwP/AHFE9nmxsRN9X2HQC8ElgM/AI4PCLuAd4K7JWZGRHjGv7tJanF2KMtSa3rDcDpETEfuAXYCZhZ23drXcgGODsifgPcDEyta7c5rwEuzcx1mfkEcCPwqrpzL8rM9cB8iiEtK4AXgP+IiLdRPApZkjqaQVuSWlcAf5OZ+9eWGZnZ3aP93EuNIl4HHAW8OjP/ALgdGN2Pc2/O6rr364ARmbmWohf9O8AJwNUD+iaS1IYM2pLUOp4FxtatXwP8VUSMBIiIl0fE9r0ctyOwLDNXRcRewKF1+9Z0H9/DTcBJtXHgXcCRwK2bKywixgA7ZuZVwPsphp1IUkdzjLYktY47gLW1ISDfAL5IMWzj17UbEpdS9Cb3dDXwnoi4A7iPYvhIt68Bd0TErzPzHXXbrwReDfwGSODDmfl4Laj3ZizwPxExmqI3/Jyt+4qS1D4iM6uuQZIkSWo7Dh2RJEmSSmDQliRJkkpg0JYkSZJKYNCWJEmSSmDQliRJkkpg0JYkSZJKYNCWJEmSSmDQliRJkkrw/wGhFK+CF0q1QwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x20d530bcb70>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig, ax = plt.subplots(figsize = (12, 8))\n",
    "ax.plot(np.arange(iters), cost, 'r')\n",
    "ax.set_xlabel('Iterations')\n",
    "ax.set_ylabel('Cost')\n",
    "ax.set_title('Error vs Training Epoch')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### The cost (error) of the solution decreases with each training epoch. This shows that gradient descent worked properly and our algorithm worked."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}