class_04_add.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h1><center> Class_04_add</center></h1>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#  Perceptron\n",
    "\n",
    "* Deep Learning from Scratch : https://www.ebook3000.co/deep-learning-from-scratch/\n",
    "\n",
    "* Perceptron is an algorithm developed by Frank Rosenblatt in 1957. **Perceptron is an algorithm that is the origin of neural networks(deep learning)**. Learning the structure of the perceptron is an important idea to advance into neural networks and deep learning.  \n",
    "\n",
    "* The most important basic unit of a neural network is the perceptron.  A perceptron is the smallest neural network unit that uses an input value and an activation function to pass an output value to the next.    \n",
    "    \n",
    "<img src = './images/neuron.png' width=400 height=100> \n",
    "\n",
    "<img src = './images/fig 1_1.png' width=410 height=100>  \n",
    "\n",
    "\n",
    "\n",
    "## What is perceptron\n",
    "\n",
    "* Perceptron receives **multiple signals** as inputs and outputs **one signal**. It is good to imagine that there is a flow like a electric current or a river. Like a electric current or a river, perceptron signal makes flow and forwards information. But perceptron signal can have two values : 'flow(1) / not flow(0)'.  \n",
    "\n",
    "\n",
    "\n",
    "\n",
    "[Figure 1]\n",
    "<img src = './images/fig 2-1.png' width=200 height=100>\n",
    "\n",
    "[Figure 1] is an example of perceptron that receives two signals as input. x1 and x2 are input signals, y is output signal, w1 and w2 are weights. Circle is called neuron or node. When the input signal is sent to a neuron, each unique weight is multiplied(w1x1, w2x2). Outputs 1 only when the sum of the signals sent from the neurons exceeds the specified limit(it is also referred to as 'neuron activate'). The limit is called the threshold and it is represented by a theta character.\n",
    "\n",
    "The above expression can be expressed as [Expression 1]\n",
    "\n",
    "[Expression 1]\n",
    "<img src = './images/e 2.1.png' width=200 height=100>\n",
    "\n",
    "Perceptrons give unique weights to each of the multiple input signals. The weights act as factors that control the effect each signal has on the outcome. In other words, the larger the weight, the more important the signal is.\n",
    "\n",
    "Substituting -b for theta in the above expression gives: \n",
    "    \n",
    "<img src = './images/e 2.2.png' width=200 height=100>\n",
    "\n",
    "b is the bias and w is the weight\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Simple logic gate\n",
    "### AND, NAND, OR gate\n",
    "\n",
    ">   ### AND      : Both 1 => 1\n",
    ">   ### NAND :  One or Two 1 =>1  (Both 1 => 0)\n",
    ">   ### OR        : One or Two 1 =>1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Implementing Perceptron"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 123,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "import math\n",
    "import time\n",
    "import numpy as np\n",
    "import tensorflow as tf\n",
    "# weight and bias functions\n",
    "\n",
    "def AND(x1, x2):\n",
    "    x = np.array([x1, x2]) # input\n",
    "    w = np.array([0.5, 0.5]) # weight\n",
    "    b = -0.7  #bias                  \n",
    "    tmp = np.sum(w * x) + b\n",
    "    if tmp <= 0:\n",
    "        return 0\n",
    "    else:\n",
    "        return 1\n",
    "\n",
    "\n",
    "def NAND(x1, x2):\n",
    "    x = np.array([x1, x2])\n",
    "    w = np.array([-0.5, -0.5])  # Only the w and b are different from AND.\n",
    "    b = 0.7\n",
    "    tmp = np.sum(w * x) + b\n",
    "    if tmp <= 0:\n",
    "        return 0\n",
    "    else:\n",
    "        return 1\n",
    "\n",
    "\n",
    "def OR(x1, x2):\n",
    "    x = np.array([x1, x2])\n",
    "    w = np.array([0.5, 0.5])  # nly the w and b are different from AND.\n",
    "    b = -0.2\n",
    "    tmp = np.sum(w * x) + b\n",
    "    if tmp <= 0:\n",
    "        return 0\n",
    "    else:\n",
    "        return 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 124,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "AND\n",
      "0\n",
      "0\n",
      "0\n",
      "1\n"
     ]
    }
   ],
   "source": [
    "print(\"AND\")\n",
    "print(AND(0, 0))  # 0\n",
    "print(AND(0, 1))  # 0\n",
    "print(AND(1, 0))  # 0\n",
    "print(AND(1, 1))  # 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 125,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "NAND\n",
      "1\n",
      "1\n",
      "1\n",
      "0\n"
     ]
    }
   ],
   "source": [
    "print(\"NAND\")\n",
    "print(NAND(0, 0))  # 1\n",
    "print(NAND(0, 1))  # 1\n",
    "print(NAND(1, 0))  # 1\n",
    "print(NAND(1, 1))  # 0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 126,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "OR\n",
      "0\n",
      "1\n",
      "1\n",
      "1\n"
     ]
    }
   ],
   "source": [
    "print(\"OR\")\n",
    "print(OR(0, 0))  # 0\n",
    "print(OR(0, 1))  # 1\n",
    "print(OR(1, 0))  # 1\n",
    "print(OR(1, 1))  # 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Limitations of Perceptron"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### XOR     : only when different values (Only one 1 => 1)  \n",
    "   >### Exclusive OR: Reject values that are equal to each other \n",
    "    (배타적 논리합 : 서로 같은 값을 거부함)\n",
    "    >### XOR is a two-layer perceptron. It has been proven that any function can be expressed by using a two-layer perceptron (= nonlinear signoid function) as an activation function.\n",
    "> ## Multilayer Perceptron (MLP)\n",
    "<img src = './images/fig 8.png' width=400 height=200>  \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### The XOR gate cannot be implemented with a single-layer (linear) perceptron.  \n",
    ">####  = A single-layer perceptron cannot separate nonlinear regions.  \n",
    ">#### Can be implemented through multi-layer perceptron.\n",
    "\n",
    "#### Combining Existing Gates\n",
    "\n",
    ">### XOR(x1, x2) = AND(NAND(x1, x2), OR(x1, x2))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 127,
   "metadata": {},
   "outputs": [],
   "source": [
    "# XOR gate\n",
    "\n",
    "def XOR(x1, x2):\n",
    "    s1 = NAND(x1, x2)\n",
    "    s2 = OR(x1, x2)\n",
    "    y = AND(s1, s2)\n",
    "    return y"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 128,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "XOR\n",
      "0\n",
      "1\n",
      "1\n",
      "0\n"
     ]
    }
   ],
   "source": [
    "print(\"XOR\")\n",
    "print(XOR(0, 0))  # 0\n",
    "print(XOR(0, 1))  # 1\n",
    "print(XOR(1, 0))  # 1\n",
    "print(XOR(1, 1))  # 0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src = './images/xor.png' width=400 height=200>\n",
    "<img src = './images/xor_2.png' width=400 height=200>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# XOR logic with two Dense layers"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 129,
   "metadata": {},
   "outputs": [],
   "source": [
    "from __future__ import absolute_import, division, print_function, unicode_literals\n",
    "\n",
    "try:\n",
    "  %tensorflow_version 2.x\n",
    "except Exception:\n",
    "  pass"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 130,
   "metadata": {},
   "outputs": [],
   "source": [
    "import tensorflow as tf\n",
    "from tensorflow.keras.layers import Dense\n",
    "from tensorflow.keras.models import Sequential\n",
    "# random seed for always same results\n",
    "tf.random.set_seed(678)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 131,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 132,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2.4.1\n"
     ]
    }
   ],
   "source": [
    "print(tf.__version__)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Inputs and Labels"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 133,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_ = np.array([[0.,0.],[0.,1.],[1.,0.],[1.,1.]])\n",
    "y_ = np.array([0.,1.,1.,0.])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Two dense Layers"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 134,
   "metadata": {},
   "outputs": [],
   "source": [
    "model = Sequential()\n",
    "\n",
    "# first dense layer\n",
    "model.add(Dense(units=2,activation='sigmoid',input_dim=2))\n",
    "# second dense layer\n",
    "model.add(Dense(units=1,activation='sigmoid'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 135,
   "metadata": {},
   "outputs": [],
   "source": [
    "# loss function and optimization\n",
    "model.compile(loss='binary_crossentropy',optimizer='sgd',metrics=['accuracy'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 136,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"sequential_5\"\n",
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "dense_11 (Dense)             (None, 2)                 6         \n",
      "_________________________________________________________________\n",
      "dense_12 (Dense)             (None, 1)                 3         \n",
      "=================================================================\n",
      "Total params: 9\n",
      "Trainable params: 9\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n",
      "None\n"
     ]
    }
   ],
   "source": [
    "# model summary\n",
    "print(model.summary())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Train"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 137,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tensorflow.python.keras.callbacks.History at 0x19c7f8b2a20>"
      ]
     },
     "execution_count": 137,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Train (takes about 3 minutes from Colab)\n",
    "model.fit(X_,y_,epochs=50000,batch_size=4,verbose=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Test"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[0, 0], [0, 1] outputs are less than 0.5,  \n",
    "[0, 1], [1, 0] outputs are greater than 0.5"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 138,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[0.06947979]\n",
      " [0.86743605]\n",
      " [0.8677261 ]\n",
      " [0.2499043 ]]\n"
     ]
    }
   ],
   "source": [
    "print(model.predict(X_,batch_size=4))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Printing First Dense layer's weights and bias"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> first node's weights are 3.5567355, 3.549393, bias is -5.4060693 \n",
    "> second node's weights are 5.79164, 5.7427516, bias is -2.3299"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 139,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "first layer weights:  [[-2.7477374 -5.467265 ]\n",
      " [-2.765913  -5.5713253]]\n",
      "first layer bias:  [3.8581967 1.7341115]\n"
     ]
    }
   ],
   "source": [
    "print(\"first layer weights: \",model.layers[0].get_weights()[0])\n",
    "print(\"first layer bias: \",model.layers[0].get_weights()[1])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Print Second Dense layer's weights and bias"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    ">weights are -7.8013854, 7.392064, bias is-3.3388186"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 140,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "second layer weights:  [[ 5.3055277]\n",
      " [-6.872516 ]]\n",
      "second layer bias:  [-1.9493619]\n"
     ]
    }
   ],
   "source": [
    "print(\"second layer weights: \",model.layers[1].get_weights()[0])\n",
    "print(\"second layer bias: \",model.layers[1].get_weights()[1])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# calculate yourself and verify same result with TF dense layers"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 141,
   "metadata": {},
   "outputs": [],
   "source": [
    "import math\n",
    "\n",
    "def sigmoid(x):\n",
    "  return 1 / (1 + math.exp(-x))\n",
    "\n",
    "def get_output(x):\n",
    "    layer0 = model.layers[0]\n",
    "    # first dense layer, first node output\n",
    "    layer0_weights, layer0_bias = layer0.get_weights()\n",
    "    layer0_node0_weights = np.transpose(layer0_weights)[0]\n",
    "    layer0_node0_bias = layer0_bias[0]\n",
    "    layer0_node0_output = sigmoid( np.dot( x, layer0_node0_weights ) + layer0_node0_bias )\n",
    "    # second dense layer, second node output\n",
    "    layer0_node1_weights = np.transpose(layer0_weights)[1]\n",
    "    layer0_node1_bias = layer0_bias[1]\n",
    "    layer0_node1_output = sigmoid( np.dot( x, layer0_node1_weights ) + layer0_node1_bias )\n",
    "    # second layer output\n",
    "    layer1 = model.layers[1]\n",
    "    layer1_weights, layer1_bias = layer1.get_weights()\n",
    "    layer1_output = sigmoid( np.dot( [layer0_node0_output, layer0_node1_output], layer1_weights ) + layer1_bias )\n",
    "\n",
    "    print(layer1_output)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 142,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.06947978261626493\n"
     ]
    }
   ],
   "source": [
    "get_output([0,0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 143,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.8674360204719486\n"
     ]
    }
   ],
   "source": [
    "get_output([0,1])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 144,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.8677261423759945\n"
     ]
    }
   ],
   "source": [
    "get_output([1,0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 145,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.24990429967153485\n"
     ]
    }
   ],
   "source": [
    "get_output([1,1])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# From perceptron to neural network\n",
    "\n",
    "Neural networks have much in common with the perceptrons described in the above. \n",
    "We describe the structure of neural networks, focusing on differences from perceptron.\n",
    "\n",
    "### Examples of neural networks\n",
    "\n",
    "If the neural network is shown as a picture, it becomes like Figure 3. Here, the leftmost line is the input layer, the rightmost line is the output layer, and the middle line is the hidden layer. The Neuron of the hidden layer (unlike the input layer and output layer) is not visible to the human eye. So it is 'Hidden'. In this book, we will say 0-layer, 1-layer, 2-layer in the direction of output layer from input layer. In Figure 3-1, the 0-layer is the input layer, the 1-layer is the hidden layer, and the 2-layer is the output layer.\n",
    "\n",
    "[Figure 3] Examples of neural networks\n",
    "<img src = './images/fig 3-15-1.png' width=400 height=200>\n",
    "\n",
    "Add bias and activation function\n",
    ">A function that converts the sum of input signals into an output signal is generally called an activation function (_h(x)_).\n",
    "<img src = './images/fig 3-4.png' width=200 height=100>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Activation Functions\n",
    "\n",
    "<img src = './images/fig 9.png' width=800 height=200>  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Gradient Loss Problem and Activation Function\n",
    "\n",
    "\n",
    "#### The multilayer perceptron met error backpropagation and became a neural network, and the neural network solves the XOR problem lightly, but the results are not as good as expected.\n",
    "\n",
    "#### Why?\n",
    "\n",
    "<img src = './images/fig 4.png' width=400 height=200>  \n",
    "\n",
    " * Backpropagation is a way to go backward one by one from the output layer and modify the weights of each layer  \n",
    " * Modifying the weights requires the derivative, i.e. the gradient  \n",
    " * However, as the layers increase, **the vanishing gradient** problem begins to occur, in which the gradient becomes 0 in the middle.\n",
    "\n",
    "<img src = './images/fig 5.png' width=400 height=200>  \n",
    "\n",
    "[sigmoid function]\n",
    "<img src = './images/fig 6.png' width=400 height=200>  \n",
    "* To solve this, we started replacing the activation function with a number of non-sigmoid functions.\n",
    "\n",
    "<img src = './images/fig 7.png' width=600 height=300>  \n",
    "\n",
    "* ReLU, proposed by Professor Jeffrey Hinton of the University of Toronto, is emerging as an alternative to sigmoid and is currently the most widely used activation function. Relu treats all values as 0 when x is less than 0, and uses x as it is for values greater than 0. Using this method, the derivative is 1 as long as x is greater than 0.  \n",
    "* Therefore, even if it is multiplied through several hidden layers, it can remain without disappearing until the first layer: the development of deep learning is accelerating.  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Implementing a 3-layer neural network"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src = './images/fig 3-15-2.png' width=500 height=300>  \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 146,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0.31682708 0.69627909]\n"
     ]
    }
   ],
   "source": [
    "\n",
    "def sigmoid(x):\n",
    "    return 1 / (1 + np.exp(-x))\n",
    "\n",
    "def identity_function(x):\n",
    "    return x\n",
    "\n",
    "def init_network():\n",
    "    network = {}\n",
    "    network['W1'] = np.array([[0.1, 0.3, 0.5], [0.2, 0.4, 0.6]])\n",
    "    network['b1'] = np.array([0.1, 0.2, 0.3])\n",
    "    network['W2'] = np.array([[0.1, 0.4], [0.2, 0.5], [0.3, 0.6]])\n",
    "    network['b2'] = np.array([0.1, 0.2])\n",
    "    network['W3'] = np.array([[0.1, 0.3], [0.2, 0.4]])\n",
    "    network['b3'] = np.array([0.1, 0.2])\n",
    "    \n",
    "    return network\n",
    "\n",
    "def forward(network, x):\n",
    "    W1, W2, W3 = network['W1'], network['W2'], network['W3']\n",
    "    b1, b2, b3 = network['b1'], network['b2'], network['b3']\n",
    "    \n",
    "    a1 = np.dot(x, W1) + b1\n",
    "    z1 = sigmoid(a1)\n",
    "    a2 = np.dot(z1, W2) + b2\n",
    "    z2 = sigmoid(a2)\n",
    "    a3 = np.dot(z2, W3) + b3\n",
    "    y = identity_function(a3)\n",
    "    \n",
    "    return y\n",
    "\n",
    "network = init_network()\n",
    "x = np.array([1.0, 0.5])\n",
    "y = forward(network, x)\n",
    "print(y) # [ 0.31682708 0.69627909 ]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Linear Regression _Ex1_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 149,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Requirement already satisfied: xlrd in c:\\users\\owner\\anaconda\\lib\\site-packages (1.0.0)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING: Ignoring invalid distribution -umpy (c:\\users\\owner\\anaconda\\lib\\site-packages)\n",
      "WARNING: Ignoring invalid distribution -rotobuf (c:\\users\\owner\\anaconda\\lib\\site-packages)\n",
      "WARNING: Ignoring invalid distribution -illow (c:\\users\\owner\\anaconda\\lib\\site-packages)\n",
      "WARNING: Ignoring invalid distribution -cipy (c:\\users\\owner\\anaconda\\lib\\site-packages)\n",
      "WARNING: Ignoring invalid distribution -umpy (c:\\users\\owner\\anaconda\\lib\\site-packages)\n",
      "WARNING: Ignoring invalid distribution -rotobuf (c:\\users\\owner\\anaconda\\lib\\site-packages)\n",
      "WARNING: Ignoring invalid distribution -illow (c:\\users\\owner\\anaconda\\lib\\site-packages)\n",
      "WARNING: Ignoring invalid distribution -cipy (c:\\users\\owner\\anaconda\\lib\\site-packages)\n",
      "WARNING: Ignoring invalid distribution -umpy (c:\\users\\owner\\anaconda\\lib\\site-packages)\n",
      "WARNING: Ignoring invalid distribution -rotobuf (c:\\users\\owner\\anaconda\\lib\\site-packages)\n",
      "WARNING: Ignoring invalid distribution -illow (c:\\users\\owner\\anaconda\\lib\\site-packages)\n",
      "WARNING: Ignoring invalid distribution -cipy (c:\\users\\owner\\anaconda\\lib\\site-packages)\n",
      "WARNING: Ignoring invalid distribution -umpy (c:\\users\\owner\\anaconda\\lib\\site-packages)\n",
      "WARNING: Ignoring invalid distribution -rotobuf (c:\\users\\owner\\anaconda\\lib\\site-packages)\n",
      "WARNING: Ignoring invalid distribution -illow (c:\\users\\owner\\anaconda\\lib\\site-packages)\n",
      "WARNING: Ignoring invalid distribution -cipy (c:\\users\\owner\\anaconda\\lib\\site-packages)\n",
      "WARNING: Ignoring invalid distribution -umpy (c:\\users\\owner\\anaconda\\lib\\site-packages)\n",
      "WARNING: Ignoring invalid distribution -rotobuf (c:\\users\\owner\\anaconda\\lib\\site-packages)\n",
      "WARNING: Ignoring invalid distribution -illow (c:\\users\\owner\\anaconda\\lib\\site-packages)\n",
      "WARNING: Ignoring invalid distribution -cipy (c:\\users\\owner\\anaconda\\lib\\site-packages)\n",
      "WARNING: Ignoring invalid distribution -umpy (c:\\users\\owner\\anaconda\\lib\\site-packages)\n",
      "WARNING: Ignoring invalid distribution -rotobuf (c:\\users\\owner\\anaconda\\lib\\site-packages)\n",
      "WARNING: Ignoring invalid distribution -illow (c:\\users\\owner\\anaconda\\lib\\site-packages)\n",
      "WARNING: Ignoring invalid distribution -cipy (c:\\users\\owner\\anaconda\\lib\\site-packages)\n"
     ]
    }
   ],
   "source": [
    "!pip install xlrd"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\Owner\\Anaconda\\lib\\site-packages\\numpy\\_distributor_init.py:32: UserWarning: loaded more than 1 DLL from .libs:\n",
      "C:\\Users\\Owner\\Anaconda\\lib\\site-packages\\numpy\\.libs\\libopenblas.PYQHXLVVQ7VESDPUVUADXEVJOBGHJPAY.gfortran-win_amd64.dll\n",
      "C:\\Users\\Owner\\Anaconda\\lib\\site-packages\\numpy\\.libs\\libopenblas.WCDJNK7YVMPZQ2ME2ZZHJJRJ3JIKNDB7.gfortran-win_amd64.dll\n",
      "  stacklevel=1)\n"
     ]
    }
   ],
   "source": [
    "from sklearn.linear_model import LinearRegression\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(20,)"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "seoul_born_death_mrg = pd.read_csv('./data/05 seoul_born_death_mrg_div_df.csv',  encoding='cp949') # utf-8 makes error (hangul)\n",
    "\n",
    "seoul_born_death_mrg\n",
    "\n",
    "seoul_pop = pd.read_excel('./data/02_1. Population_in_Seoul(2002_2021).xlsx',  encoding='cp949') # utf-8 makes error (hangul)\n",
    "seoul_pop\n",
    "\n",
    "sel_pop_tot = seoul_pop.iloc[1:2,[4,10,16,22,28,34,40,46,52,58,64,70,76,82,88,94,100,106,112,118]]\n",
    "sel_pop_tot.iloc[0].shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "seoul_born_death_inc = seoul_born_death_mrg['# of born'] - seoul_born_death_mrg['# of death']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Year</th>\n",
       "      <th># of marriges</th>\n",
       "      <th># of divorces</th>\n",
       "      <th># of death</th>\n",
       "      <th># of born</th>\n",
       "      <th># inc</th>\n",
       "      <th>tot_pop</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2002</td>\n",
       "      <td>72696</td>\n",
       "      <td>29351</td>\n",
       "      <td>38925</td>\n",
       "      <td>102008</td>\n",
       "      <td>63083</td>\n",
       "      <td>10207295</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2003</td>\n",
       "      <td>72750</td>\n",
       "      <td>32499</td>\n",
       "      <td>38068</td>\n",
       "      <td>101227</td>\n",
       "      <td>63159</td>\n",
       "      <td>10174086</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2004</td>\n",
       "      <td>71553</td>\n",
       "      <td>26994</td>\n",
       "      <td>38274</td>\n",
       "      <td>99828</td>\n",
       "      <td>61554</td>\n",
       "      <td>10173162</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2005</td>\n",
       "      <td>71286</td>\n",
       "      <td>25300</td>\n",
       "      <td>38751</td>\n",
       "      <td>90468</td>\n",
       "      <td>51717</td>\n",
       "      <td>10167344</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2006</td>\n",
       "      <td>73924</td>\n",
       "      <td>24354</td>\n",
       "      <td>38519</td>\n",
       "      <td>93922</td>\n",
       "      <td>55403</td>\n",
       "      <td>10181166</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>2007</td>\n",
       "      <td>76399</td>\n",
       "      <td>24615</td>\n",
       "      <td>39016</td>\n",
       "      <td>101249</td>\n",
       "      <td>62233</td>\n",
       "      <td>10192710</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>2008</td>\n",
       "      <td>71753</td>\n",
       "      <td>23319</td>\n",
       "      <td>38298</td>\n",
       "      <td>94736</td>\n",
       "      <td>56438</td>\n",
       "      <td>10200827</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>2009</td>\n",
       "      <td>68841</td>\n",
       "      <td>23920</td>\n",
       "      <td>38790</td>\n",
       "      <td>89594</td>\n",
       "      <td>50804</td>\n",
       "      <td>10208302</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>2010</td>\n",
       "      <td>70466</td>\n",
       "      <td>21768</td>\n",
       "      <td>40130</td>\n",
       "      <td>93266</td>\n",
       "      <td>53136</td>\n",
       "      <td>10312545</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>2011</td>\n",
       "      <td>71519</td>\n",
       "      <td>20617</td>\n",
       "      <td>40320</td>\n",
       "      <td>91526</td>\n",
       "      <td>51206</td>\n",
       "      <td>10249679</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>2012</td>\n",
       "      <td>71695</td>\n",
       "      <td>20177</td>\n",
       "      <td>41514</td>\n",
       "      <td>93914</td>\n",
       "      <td>52400</td>\n",
       "      <td>10195318</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>2013</td>\n",
       "      <td>68819</td>\n",
       "      <td>20126</td>\n",
       "      <td>42063</td>\n",
       "      <td>84066</td>\n",
       "      <td>42003</td>\n",
       "      <td>10143645</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>2014</td>\n",
       "      <td>64823</td>\n",
       "      <td>19477</td>\n",
       "      <td>42153</td>\n",
       "      <td>83711</td>\n",
       "      <td>41558</td>\n",
       "      <td>10103233</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>2015</td>\n",
       "      <td>64193</td>\n",
       "      <td>18176</td>\n",
       "      <td>43053</td>\n",
       "      <td>83005</td>\n",
       "      <td>39952</td>\n",
       "      <td>10022181</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>2016</td>\n",
       "      <td>57643</td>\n",
       "      <td>17777</td>\n",
       "      <td>43540</td>\n",
       "      <td>75536</td>\n",
       "      <td>31996</td>\n",
       "      <td>9930616</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>2017</td>\n",
       "      <td>53776</td>\n",
       "      <td>17083</td>\n",
       "      <td>42932</td>\n",
       "      <td>65389</td>\n",
       "      <td>22457</td>\n",
       "      <td>9857426</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>2018</td>\n",
       "      <td>52005</td>\n",
       "      <td>16940</td>\n",
       "      <td>45219</td>\n",
       "      <td>58074</td>\n",
       "      <td>12855</td>\n",
       "      <td>9765623</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>2019</td>\n",
       "      <td>48261</td>\n",
       "      <td>16972</td>\n",
       "      <td>43829</td>\n",
       "      <td>53673</td>\n",
       "      <td>9844</td>\n",
       "      <td>9729107</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>2020</td>\n",
       "      <td>44746</td>\n",
       "      <td>16282</td>\n",
       "      <td>45522</td>\n",
       "      <td>47445</td>\n",
       "      <td>1923</td>\n",
       "      <td>9668465</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>2021</td>\n",
       "      <td>37012</td>\n",
       "      <td>14967</td>\n",
       "      <td>48798</td>\n",
       "      <td>45531</td>\n",
       "      <td>-3267</td>\n",
       "      <td>9509458</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    Year  # of marriges  # of divorces  # of death  # of born  # inc   tot_pop\n",
       "0   2002          72696          29351       38925     102008  63083  10207295\n",
       "1   2003          72750          32499       38068     101227  63159  10174086\n",
       "2   2004          71553          26994       38274      99828  61554  10173162\n",
       "3   2005          71286          25300       38751      90468  51717  10167344\n",
       "4   2006          73924          24354       38519      93922  55403  10181166\n",
       "5   2007          76399          24615       39016     101249  62233  10192710\n",
       "6   2008          71753          23319       38298      94736  56438  10200827\n",
       "7   2009          68841          23920       38790      89594  50804  10208302\n",
       "8   2010          70466          21768       40130      93266  53136  10312545\n",
       "9   2011          71519          20617       40320      91526  51206  10249679\n",
       "10  2012          71695          20177       41514      93914  52400  10195318\n",
       "11  2013          68819          20126       42063      84066  42003  10143645\n",
       "12  2014          64823          19477       42153      83711  41558  10103233\n",
       "13  2015          64193          18176       43053      83005  39952  10022181\n",
       "14  2016          57643          17777       43540      75536  31996   9930616\n",
       "15  2017          53776          17083       42932      65389  22457   9857426\n",
       "16  2018          52005          16940       45219      58074  12855   9765623\n",
       "17  2019          48261          16972       43829      53673   9844   9729107\n",
       "18  2020          44746          16282       45522      47445   1923   9668465\n",
       "19  2021          37012          14967       48798      45531  -3267   9509458"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "seoul_born_death_mrg['# inc'] = seoul_born_death_inc\n",
    "seoul_born_death_mrg['tot_pop'] = list(sel_pop_tot.iloc[0])\n",
    "\n",
    "seoul_born_death_mrg"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "x=seoul_born_death_mrg['Year']\n",
    "y=seoul_born_death_mrg['# of born']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY0AAAD8CAYAAACLrvgBAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAGYFJREFUeJzt3X+Q3PV93/Hny0LYh2t8kpEIOkGFW0WuY2IktiBMx4PrWCcxGetC4hT3h84M7TXEbu3ORBNpmqk6kBZcdZqGJKVVTIqUcWwTIoRSgy+qsNuZFhFWCBCYKBKODXdS0MXiBClXR6jv/rGfFavT7t1397t339vb12NmZ7/73s/3833vV6t77/fz/ex+FRGYmZll8a6iEzAzs87homFmZpm5aJiZWWYuGmZmlpmLhpmZZeaiYWZmmblomJlZZi4aZmaWmYuGmZlldlHRCbTbZZddFitWrCg6DTOzjnLw4MG/iIgl07Wbd0VjxYoVlMvlotMwM+sokn6QpZ2Hp8zMLDMXDTMzy8xFw8zMMnPRMDOzzFw0zMwss3k3e6oVew6Nsn34CMfHJ1jW28Pm/lUMrO4rOi0zszmn64vGnkOjbN19mIkzZwEYHZ9g6+7DAC4cZmaTdP3w1PbhI+cKRtXEmbNsHz5SUEZmZnNX1x9pHB+faCpeT7cPb3X76zfrJl1fNJb19jBap0As6+3JtH63D291++s36zZdPzy1uX8VPQsXnBfrWbiAzf2rMq3f7cNb3f76zbpN1x9pVD8Ntzq80o7hrU7W7a/frNtMWzQk/Q7w08DJiPhIii0GvgGsAL4P/HxEvC5JwK8DtwBvAZ+LiGfSOoPAr6RufzUidqb4dcCDQA/wGPDFiIhG28j9iusYWN3X8lBK3uGtTtftr9+s22QZnnoQWD8ptgXYHxErgf3pMcAGYGW6DQH3w7kisw24Abge2CZpUVrn/tS2ut76abYxp+Qd3oLKeYGb7n2Cq7d8k5vufYI9h0bbneaMacfrN7POMW3RiIj/CZyaFN4I7EzLO4GBmviuqDgA9Eq6AugH9kXEqXS0sA9Yn567NCKejIgAdk3qq9425pSB1X3cc+s19PX2IKCvt4d7br0m85FL9UTy6PgEwTsnkjulcOR9/WbWWVo9p3F5RJwAiIgTkpameB/wak27kRSbKj5SJz7VNi4gaYjK0QpXXXVViy+pdXmGt6Y6kdwpf3jzvH4z6yztnj2lOrFoId6UiNgREaWIKC1ZMu2Fp+YUn0g2s07SatF4LQ0tke5PpvgIcGVNu+XA8Wniy+vEp9rGvNLohLFPJJvZXNRq0dgLDKblQeDRmvgmVawFTqchpmFgnaRF6QT4OmA4PfempLVp5tWmSX3V28a84hPJZtZJsky5/RpwM3CZpBEqs6DuBR6SdAfwCvCZ1PwxKtNtj1GZcns7QEScknQ38HRqd1dEVE+u38k7U24fTzem2Ma8kvd7ImZms0mVSUvzR6lUinK5XHQaZmYdRdLBiChN167rf0bEzMyyc9EwM7PMXDTMzCwzFw0zM8vMRcPMzDJz0TAzs8xcNMzMLDMXDTMzy8xFw8zMMnPRMDOzzLr+GuFWvD2HRv3bW2YdwkXDClW9cmH1QlTVKxcCLhxmc5CHp6xQU1250MzmHh9pWKHDQ75yoYfnrLO4aHS5ooeHlvX2MFqnQHTLlQuL3v9mzfLwVJcrenhoLly5cM+hUW669wmu3vJNbrr3CfYcGp21bRe9/82a5SONLlf08FDRVy4s+pN+0fvfrFkuGl1uLgwPDazuK2woZqpP+h6eM7uQh6e63FwYHipS0Z/0u33/W+fxkUaXK3p4qGhFf9Lv9v1vnUcRUXQObVUqlaJcLhedhnWIyec0oPJJ/55br/Efbusqkg5GRGm6dj7SsK7Wjk/6/p6FdRMXDet6eU7EFz37ymy2+US4WQ7+noV1GxcNsxyKnn1lNts8PDUPeEy9OEXPvjKbbbmONCR9UdILkl6U9KUUWyxpn6Sj6X5RikvSfZKOSXpe0pqafgZT+6OSBmvi10k6nNa5T5Ly5DsfVcfUR8cnCN4ZU5/Nn8LoZv6ehXWblouGpI8A/wS4Hvgo8NOSVgJbgP0RsRLYnx4DbABWptsQcH/qZzGwDbgh9bWtWmhSm6Ga9da3mu985TH1Yg2s7uOeW6+hr7cHAX29PZ6ua/NanuGpvwUciIi3ACT9D+BngI3AzanNTuA7wC+n+K6ofDHkgKReSVektvsi4lTqZx+wXtJ3gEsj4skU3wUMAI/nyHne8Zh68cNzRf4MitlsyzM89QLwcUkfkHQJcAtwJXB5RJwASPdLU/s+4NWa9UdSbKr4SJ34BSQNSSpLKo+NjeV4SZ2n0dh5t4ype3jObHa1XDQi4iXgy8A+4FvAc8DbU6xS73xEtBCvl8uOiChFRGnJkiVT5j3fdPuYuofnzGZXrhPhEfFARKyJiI8Dp4CjwGtp2Il0fzI1H6FyJFK1HDg+TXx5nbjV6PYxdQ/Pmc2uXFNuJS2NiJOSrgJuBW4ErgYGgXvT/aOp+V7gC5K+TuWk9+mIOCFpGPi3NSe/1wFbI+KUpDclrQWeAjYBv5En3/mqm8fUPeXVbHbl/XLfH0j6LvCHwOcj4nUqxeJTko4Cn0qPAR4DvgccA34b+EWAdAL8buDpdLurelIcuBP4SlrnZXwS3Cbp9uE5s9nmX7m1jlf07Cmz+cC/cmtdo5uH58xmm397yszMMnPRMDOzzFw0zMwsMxcNMzPLzEXDzMwyc9EwM7PMXDTMzCwzFw0zM8vMRcPMzDJz0TAzs8xcNMzMLDP/9pRZh/MPNtpsctEw62DVy91Wr15YvdwtkLlwuOhYMzw8ZdbB8l7u1tdYt2a5aJh1sLyXu/U11q1ZHp4y62B5L3fbjmuse3iru/hIw6yD5b3cbaPikrXoeHir+7homHWwgdV93HPrNfT19iCgr7eHe269JvMn/bxFx8Nb3cfDU2YdLs/lbqvrtTq81I7hLessLhpmXS5P0cl7TsU6j4enzKxleYe3rPP4SMPMWpZ3eMs6j4uGmeWSZ3jLOo+Hp8zMLDMXDTMzyyxX0ZD0LyS9KOkFSV+T9B5JV0t6StJRSd+QdHFq++70+Fh6fkVNP1tT/Iik/pr4+hQ7JmlLnlzNzCy/louGpD7gnwOliPgIsAC4Dfgy8GsRsRJ4HbgjrXIH8HpE/E3g11I7JH04rfcTwHrgP0laIGkB8FvABuDDwGdTWzMzK0je4amLgB5JFwGXACeAvws8nJ7fCQyk5Y3pMen5T0pSin89In4UEX8GHAOuT7djEfG9iPgr4OuprZmZFaTlohERo8C/B16hUixOAweB8Yh4OzUbAarTKvqAV9O6b6f2H6iNT1qnUfwCkoYklSWVx8bGWn1JZmY2jTzDU4uofPK/GlgGvJfKUNJkUV2lwXPNxi8MRuyIiFJElJYsWTJd6mZm1qI8w1M/BfxZRIxFxBlgN/AxoDcNVwEsB46n5RHgSoD0/PuBU7XxSes0ipuZWUHyFI1XgLWSLknnJj4JfBf4NvBzqc0g8Gha3psek55/IiIixW9Ls6uuBlYCfww8DaxMs7EupnKyfG+OfM3MLKeWvxEeEU9Jehh4BngbOATsAL4JfF3Sr6bYA2mVB4DflXSMyhHGbamfFyU9RKXgvA18PiLOAkj6AjBMZWbW70TEi63ma2Zm+anyYX/+KJVKUS6Xi07DzKyjSDoYEaXp2vkb4WZmlpmLhpmZZeaiYWZmmblomJlZZi4aZmaWmYuGmZll5qJhZmaZ+XKvZlaoPYdGfY3xDuKiYWaF2XNolK27DzNx5iwAo+MTbN19GMCFY47y8JSZFWb78JFzBaNq4sxZtg8fKSgjm46LhpkV5vj4RFNxK56LhpkVZllvT1NxK56LhpkVZnP/KnoWLjgv1rNwAZv7VxWUkU3HJ8LNrDDVk92ePdU5XDTMrFADq/tcJDqIh6fMzCwzFw0zM8vMRcPMzDJz0TAzs8xcNMzMLDMXDTMzy8xFw8zMMnPRMDOzzFw0zMwsMxcNMzPLzEXDzMwya7loSFol6dma2xuSviRpsaR9ko6m+0WpvSTdJ+mYpOclranpazC1PyppsCZ+naTDaZ37JCnfyzUzszxaLhoRcSQiro2Ia4HrgLeAR4AtwP6IWAnsT48BNgAr020IuB9A0mJgG3ADcD2wrVpoUpuhmvXWt5qvmZnl167hqU8CL0fED4CNwM4U3wkMpOWNwK6oOAD0SroC6Af2RcSpiHgd2AesT89dGhFPRkQAu2r6MjOzArSraNwGfC0tXx4RJwDS/dIU7wNerVlnJMWmio/UiV9A0pCksqTy2NhYzpdiZmaN5C4aki4GPg38/nRN68SihfiFwYgdEVGKiNKSJUumScPMzFrVjiONDcAzEfFaevxaGloi3Z9M8RHgypr1lgPHp4kvrxM3M7OCtKNofJZ3hqYA9gLVGVCDwKM18U1pFtVa4HQavhoG1klalE6ArwOG03NvSlqbZk1tqunLzMwKkOtyr5IuAT4F/NOa8L3AQ5LuAF4BPpPijwG3AMeozLS6HSAiTkm6G3g6tbsrIk6l5TuBB4Ee4PF0MzOzgqgyMWn+KJVKUS6Xi07DzKyjSDoYEaXp2vkb4WZmlpmLhpmZZeaiYWZmmblomJlZZi4aZmaWmYuGmZll5qJhZmaZ5fpyn5lZ0fYcGmX78BGOj0+wrLeHzf2rGFhd97dNrQ1cNMysY+05NMrW3YeZOHMWgNHxCbbuPgzgwjFDPDxlZh1r+/CRcwWjauLMWbYPHykoo/nPRcPMOtbx8Ymm4pafi4aZdaxlvT1NxS0/Fw0z61ib+1fRs3DBebGehQvY3L+qoIzmP58IN7OOVT3Z7dlTs8dFw8w62sDqPheJWeThKTMzy8xFw8zMMnPRMDOzzFw0zMwsMxcNMzPLzEXDzMwyc9EwM7PMXDTMzCwzFw0zM8vMRcPMzDJz0TAzs8xyFQ1JvZIelvQnkl6SdKOkxZL2STqa7heltpJ0n6Rjkp6XtKamn8HU/qikwZr4dZIOp3Xuk6Q8+ZqZWT55jzR+HfhWRHwI+CjwErAF2B8RK4H96THABmBlug0B9wNIWgxsA24Arge2VQtNajNUs976nPmamVkOLRcNSZcCHwceAIiIv4qIcWAjsDM12wkMpOWNwK6oOAD0SroC6Af2RcSpiHgd2AesT89dGhFPRkQAu2r6MjOzAuQ50vggMAb8V0mHJH1F0nuByyPiBEC6X5ra9wGv1qw/kmJTxUfqxC8gaUhSWVJ5bGwsx0syM7Op5CkaFwFrgPsjYjXwf3hnKKqeeucjooX4hcGIHRFRiojSkiVLps7azMxalqdojAAjEfFUevwwlSLyWhpaIt2frGl/Zc36y4Hj08SX14mbmVlBWi4aEfHnwKuSqhfj/STwXWAvUJ0BNQg8mpb3ApvSLKq1wOk0fDUMrJO0KJ0AXwcMp+felLQ2zZraVNOXmZkVIO/lXv8Z8FVJFwPfA26nUogeknQH8ArwmdT2MeAW4BjwVmpLRJySdDfwdGp3V0ScSst3Ag8CPcDj6WZm1jZ7Do36GuNNUGVi0vxRKpWiXC4XnYaZdYA9h0bZuvswE2fOnov1LFzAPbde03WFQ9LBiChN187fCDezrrV9+Mh5BQNg4sxZtg8fKSijuc9Fw8y61vHxiabi5qJhZl1sWW9PU3Fz0TCzLra5fxU9CxecF+tZuIDN/asarGF5Z0+ZmXWs6sluz57KzkXDzLrawOo+F4kmeHjKzMwyc9EwM7PMXDTMzCwzFw0zM8vMRcPMzDJz0TAzs8xcNMzMLDN/T8PMLIdu+2l1Fw0zsxZN/mn10fEJtu4+DDBvC4eHp8zMWtSNP63uomFm1qJu/Gl1Fw0zsxZ140+ru2iYmbWoG39a3SfCzcxa1I0/re6iYWaWQ7f9tLqHp8zMLDMXDTMzy8xFw8zMMnPRMDOzzFw0zMwss1xFQ9L3JR2W9KykcootlrRP0tF0vyjFJek+ScckPS9pTU0/g6n9UUmDNfHrUv/H0rrKk6+ZmeXTjiONT0TEtRFRSo+3APsjYiWwPz0G2ACsTLch4H6oFBlgG3ADcD2wrVpoUpuhmvXWtyFfMzNr0UwMT20EdqblncBATXxXVBwAeiVdAfQD+yLiVES8DuwD1qfnLo2IJyMigF01fZmZWQHyFo0A/kjSQUlDKXZ5RJwASPdLU7wPeLVm3ZEUmyo+Uid+AUlDksqSymNjYzlfkpmZNZL3G+E3RcRxSUuBfZL+ZIq29c5HRAvxC4MRO4AdAKVSqW4bMzPLL9eRRkQcT/cngUeonJN4LQ0tke5PpuYjwJU1qy8Hjk8TX14nbmZmBWm5aEh6r6T3VZeBdcALwF6gOgNqEHg0Le8FNqVZVGuB02n4ahhYJ2lROgG+DhhOz70paW2aNbWppi8zMytAnuGpy4FH0izYi4Dfi4hvSXoaeEjSHcArwGdS+8eAW4BjwFvA7QARcUrS3cDTqd1dEXEqLd8JPAj0AI+nm5mZFUSViUnzR6lUinK5XHQaZmYdRdLBmq9ONOSfRjczK9CeQ6MddT0OFw0zs4LsOTTK1t2HmThzFoDR8Qm27j4MMGcLh397ysysINuHj5wrGFUTZ86yffhIQRlNz0XDzKwgx8cnmorPBS4aZmYFWdbb01R8LnDRMDMryOb+VfQsXHBerGfhAjb3ryooo+n5RLiZWUGqJ7s9e8rMzDIZWN2Xq0jM9pRdFw0zsw5VxJRdn9MwM+tQRUzZddEwM+tQRUzZddEwM+tQRUzZddEwM+tQRUzZ9YlwM7MOVcSUXRcNM7MOlnfKbrM8PGVmZpm5aJiZWWYuGmZmlpmLhpmZZeaiYWZmmSkiis6hrSSNAT+Yoe4vA/5ihvpuB+eXj/PLx/nlU3R+fz0ilkzXaN4VjZkkqRwRpaLzaMT55eP88nF++cz1/Ko8PGVmZpm5aJiZWWYuGs3ZUXQC03B++Ti/fJxfPnM9P8DnNMzMrAk+0jAzs8y6pmhIulLStyW9JOlFSV9M8cWS9kk6mu4Xpbgk3SfpmKTnJa1J8WslPZn6eF7S32uwvc9JGpP0bLr949nILz13tma7exts792SvpHWf0rSilnaf5+oye1ZSf9X0kDe/ddijh9K/5Y/kvRLk/paL+lIyn9L3n3Yrtwa9VNnezdLOl2z//7VLO+/70s6nLZdbrC9hu/hGd6Hqya9B9+Q9KW8+7CF/P5Bet3PS/rfkj5a01db339tFRFdcQOuANak5fcBfwp8GPh3wJYU3wJ8OS3fAjwOCFgLPJXiPw6sTMvLgBNAb53tfQ74zdnOLz33lxm294vAf07LtwHfmK38avpcDJwCLsm7/1rMcSnwt4F/A/xSTT8LgJeBDwIXA88BH86zD9uYW91+6mzvZuC/FbH/0nPfBy6bZnvTvkdmKr9J/9Z/TuU7Crn2YQv5fQxYlJY38M7fmLa//9p5m/ENzNUb8CjwKeAIcEXNP/qRtPxfgM/WtD/XblI/z5GKyKT452jyj1678iNb0RgGbkzLF1H5UpFmc/8BQ8BXG/Sfa/9lybGm3b/m/D/MNwLDNY+3AlvbuQ9bza1RP3XiN9Nk0WhnjmQrGpn+j83kPgTWAf+rwXO59mHW/FJ8ETA6W++/PLeuGZ6qlQ7jVgNPAZdHxAmAdL80NesDXq1ZbSTFavu5nsongZcbbOpn06Hnw5KunMX83iOpLOmA6gz9TF4/It4GTgMfmKX8qm4DvjbFplraf03k2EiW3M9r18w+zJlbo37quVHSc5Iel/QTWfttU44B/JGkg5KGGrTJup9nIr+q6d6DLe3DFvK7g8pRF8zw+y+vrisakv4a8AfAlyLijama1omdm2om6Qrgd4HbI+L/1Wn7h8CKiPhJ4L8DO2cxv6ui8s3Svw/8R0l/o8n1Zzq/6v67hsqnpXpa2n9N5tiwizqxevum6X3Yhtyy9vMMlSGXjwK/AexpY99Z3BQRa6gMu3xe0sfrbapOrJ3vwen6uRj4NPD7DZq0tA+bzU/SJ6gUjV+uhuo0a8v7rx26qmhIWkjlH/OrEbE7hV9Lf8Cqf8hOpvgIUPvpdjlwPLW7FPgm8CsRcaDetiLihxHxo/Twt4HrZiu/iKjefw/4DpVPPJOdW1/SRcD7qZxfmPH8kp8HHomIM/W21cr+ayHHRqbL/YJ2WfZhm3Jr1M95IuKNiPjLtPwYsFDSZS323XSONe/Bk8AjwPV1mmXdz23PL9kAPBMRrzV4DU3vw2bzk/STwFeAjRHxwxSekfdfu3RN0ZAk4AHgpYj4DzVP7QUG0/IglXHIanyTKtYCpyPiRPp08giwKyIafUKpvjmqPg28NEv5LZL07tTnZcBNwHfrbLK2358Dnog0ODqT+dWs91mmGBZodv+1mGMjTwMrJV2d/r1vS31Mlnkftiu3KfqZ3O7HUtvqMOq7gB82at/mHN8r6X3VZSrnDV6o03S698iM5FdjuvdgU/uw2fwkXQXsBv5RRPxpTfu2v//aaqZPmsyVG/B3qBy6PQ88m263UBkD3A8cTfeLU3sBv0XlfMVhoJTi/xA4U9PHs8C16bm7gE+n5XuAF6mcKP828KFZyu9j6fFz6f6Omm3U5vceKoflx4A/Bj44G/ml51YAo8C7Jm2j5f3XYo4/RuXT2hvAeFq+ND13C5XZLy8D/zLvPmxXbo36Sev8AvALafkLNfvvAPCx2dp/VGb9PJduL07af7U5NnyPzMK/7yVUCsD7J22j5X3YQn5fAV6vaVuu6aut77923vyNcDMzy6xrhqfMzCw/Fw0zM8vMRcPMzDJz0TAzs8xcNMzMLDMXDTMzy8xFw8zMMnPRMDOzzP4/FlSp4ToLSXUAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(x, y, 'o')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "LinearRegression(copy_X=True, fit_intercept=True, n_jobs=None,\n",
       "         normalize=False)"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "line_fitter = LinearRegression()\n",
    "line_fitter.fit(x.values.reshape(-1,1), y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([29790.11578947])"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "line_fitter.predict([[2030]])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([-2844.23157895])"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "line_fitter.coef_ # slope"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "5803580.221052634"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "line_fitter.intercept_ # y bias 절편"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY0AAAD8CAYAAACLrvgBAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3Xl8VNX9//HXhwASqBhWC0HEhborSwQEd5TNBcR9AxFEFFFrpWKtta21YvWn34IgUlDBHRERF4ysosgWFkFFBLUqSwVlU0HZzu+Pe9IOcUIms90k834+HvOYyZlz7/3MJeQz95xzzzHnHCIiIrGoFHYAIiJSfihpiIhIzJQ0REQkZkoaIiISMyUNERGJmZKGiIjETElDRERipqQhIiIxU9IQEZGYVQ47gGSrW7eua9KkSdhhiIiUKwsXLvzWOVevpHoVLmk0adKEgoKCsMMQESlXzOzLWOqpeUpERGKmpCEiIjFT0hARkZiVmDTM7AkzW29mH0aUXWxmH5nZHjPLK1L/TjNbZWYrzKxjRHknX7bKzAZFlB9iZvPMbKWZvWhmVX35fv7nVf79Jsn4wCIiEr9YrjSeAjoVKfsQ6A7Miiw0s6OBy4Bj/DbDzSzLzLKAYUBn4Gjgcl8X4AHgEedcU2AT0NuX9wY2OecOBx7x9UREJEQlJg3n3CxgY5Gy5c65FVGqdwVecM797Jz7AlgFtPKPVc65z51zO4AXgK5mZsCZwHi//RigW8S+xvjX44H2vr6IiIQk2X0aucDXET+v9mXFldcBNjvndhUp32tf/v0tvv4vmFlfMysws4INGzYk6aOIiEhRyU4a0a4EXBzl+9rXLwudG+mcy3PO5dWrV+K9KdF9PhPmjoBdO+LbXkQkAyQ7aawGDor4uRGwdh/l3wI5Zla5SPle+/LvH0CRZrKkWv4avHUHDGsFH70CWjtdROQXkp00JgGX+ZFPhwBNgfnAAqCpHylVlaCzfJJzzgEzgIv89j2BVyP21dO/vgiY7uunRpeH4MqXoUo2vHQNjDoLvpyTssOJiJRHsQy5fR6YAxxhZqvNrLeZXWBmq4GTgDfMLB/AOfcRMA74GHgL6O+c2+37JG4C8oHlwDhfF+AO4DYzW0XQZzHal48G6vjy24D/DtNNCTNoehb0ew+6DoOta+DJTvDClfDtypQeWkSkvLBUfnkPQ15enkvK3FM7tsHc4fDe/8HObdDyGjh9EPyqfuL7FhEpY8xsoXMur6R6uiO8OFWrw6m3w82LIe9aWDQGhjSHd/4BO34MOzoRkVAoaZTkV/XgnIfgxnlw2Bkw4z4Y0gIWjoE9u8OOTkQkrZQ0YlX3cLj0Gbg2H3Iaw2s3w2Pt4NO3NdJKRDKGkkZpNW4Dvd+GS8bC7p/huYthzHmwdnHYkYmIpJySRjzM4Oiu0H8+dH4Q1n8MI0+Hl/vAppjWMRERKZeUNBKRVQVa9w06y0++LbhB8NE8yL8Ltm8KOzoRkaRT0kiGagfAWffAgEVw3MUwZxj8sxm8PxR2/Rx2dCIiSaOkkUwH5EK34cENgrkt4e0/Blcey8bDnj1hRycikjAljVT49bFw9QS4+pXgKuTl3jDqTPji3bAjExFJiJJGKh12JvSdBd1GwA8bYMy58NylsP6TsCMTEYmLkkaqVaoEzS6HAQVw1p/hy/fhsZNg0s3w/X/Cjk5EpFSUNNKlSjac/Fu4eQm0uh6WPBdMSzLj7/DzD2FHJyISEyWNdKtRBzoPhpvmQ9MO8M4DQfJYMBp27yp5exGREClphKX2oXDJGOgzDeocBm/cFjRbffKGpiURkTJLSSNsjfKg12S49NkgWbxwBTx1DqxeGHZkIiK/oKRRFpjBUefCjXPgnP8H334aDNF9qRds/CLs6ERE/qtyyVUqvomL1/Bg/grWbt5Ow5xsBnY8gm7Nc9MfSFYVOLEPHH8pzB4Ccx4NpiZpdR2cOhCq105/TCIiETL+SmPi4jXcOWEZazZvxwFrNm/nzgnLmLh4TXhB7bc/nHlXMC1Js8th3ohgWpL3/g92/hReXCKS8TI+aTyYv4LtO/deTGn7zt08mL8ipIgi1GwA5w+FfrOhcWuYek8wLckHL2haEhEJRcY3T63dvL1U5dGkvHnrwKPhypfg83dgyt3wyvXBpIgd7oVDT0/eceJUZpr3RCTlMv5Ko2FOdqnKi0pr89ahp8F1M6H7KNi+GcZ2hWcuhG8+Sv6xYlQmm/dEJGUyPmkM7HgE2VWy9irLrpLFwI5HxLR92pu3KlWC4y+GmxZAh7/B6gUw4mR4tT9sXZuaY+5DmW7eE5GkKzFpmNkTZrbezD6MKKttZlPMbKV/ruXLzcyGmNkqM1tqZi0itunp6680s54R5S3NbJnfZoiZ2b6OkWzdmudyf/fjyM3JxoDcnGzu735czM0ryWjeikuVatB2QDAtSZsbYek4GNICpt0LP21N7bEjhPb5RSQUsVxpPAV0KlI2CJjmnGsKTPM/A3QGmvpHX+AxCBIAcA/QGmgF3BORBB7zdQu361TCMZKuW/NcZg86ky8Gn8PsQWeWqj0+0eYtCJp42g2eziGD3qDd4Omla9qpXhs63hdceRx5Drz7UDAtyfx/we6dse8nTsn4/CJSfpSYNJxzs4CNRYq7AmP86zFAt4jysS4wF8gxswZAR2CKc26jc24TMAXo5N+r6Zyb45xzwNgi+4p2jDIl0eatpPUJ1GoCF42G62ZA/aPgzdthWGv4eFJKpyVJ9POLSPkSb5/Ggc65dQD+ub4vzwW+jqi32pftq3x1lPJ9HeMXzKyvmRWYWcGGDRvi/EjxSbR5K+l9ArktoOdrcPmLUKkyjLsanugEX8+Pb38lSPTzi0j5kuwhtxalzMVRXirOuZHASIC8vLy0z/bXrXlu3H8kU9InYAZHdILDz4IlzwTTr48+G446P1jTo85h8e87ikQ+v4iUL/FeaXzjm5bwz+t9+WrgoIh6jYC1JZQ3ilK+r2NUKCntE8iqDC2vCe4sP/0PsGoaDGsFb/4efvw28f2LSMaJN2lMAgpHQPUEXo0o7+FHUbUBtvimpXygg5nV8h3gHYB8/973ZtbGj5rqUWRf0Y5RoaSlT2C/X8Hpd8DNi6FFD1gwKpiWZNZDsGNb8o4jIhWeuRI6Sc3seeB0oC7wDcEoqInAOKAx8BVwsXNuo//D/yjBCKhtQC/nXIHfz7XAH/xu73POPenL8whGaGUDk4EBzjlnZnWiHaOkD5SXl+cKCgpi/fxlQtrvqN6wAqb+GVa8Cfs3hDP/CCdcBpWyStxURComM1vonMsrsV5JSaO8KY9JIzT/nh1MS7JmIRx4LJz9l6AfREQyTqxJI+PvCM9oTdoFKwde9CTs+CGYkmRsN1i3NOzIRKSMUtLIdGZwbHfoPx86DYZ1S+DxU+GVfrD565K3F5GMoqQhgcr7QZsbgmlJ2t0MH06AoS1hyj3B5IgiIihpSFHZOXD2X2HAQjjmApj9z2BakrmPwa4dYUcnIiFT0pDocg6C7o/D9e/Ar4+DtwbBsBODK5AKNnhCRGKnpCH71uAE6PEqXPkyVKkB43vBqPbw5fthRyYiIVDSkJKZQdOzoN+70HUYbF0HT3aG56+ADZ+GHZ2IpJGShsSuUhY0vyro7zjzbvhiFgxvA6//Fn6okLO8iEgRurlP4vfjt/DOA1DwBFSuBm1vhrY3QdUapdqN1hgXCZ9u7pPUq1EXujwIN86Dw86AmX8PRlotfAp274ppF1pjXKR8UdKQxFYOBKh7OFz6DFz7NuQcDK/dAiPawYq3ShxppTXGk3D+RdJISSPDJfWbfuPW0PttuOTpYKnZ5y+FMefBmkXFbpLpa4zrSkvKGyWNDJf0b/pmcPT50H8edHkI1n8M/zoDxveGTV/+onpZWGM8zG/6utKS8kZJI8Ol7Jt+VhVodV0wLckpv4NPXodH8yD/Ltj2vxnuw15jPOxv+pl+pSXlj5JGhkv5N/1qNaH9n4LVA4+7BOYMCzrL3x8Ku34OfY3xsL/pl4UrLZHSUNLIcGn7pn9ALnQbBv3eg9yW8PYfgyuPpS/R7YQGzB50Jl8MPofZg85M63DbsL/ph32lJVJaShoZLu3f9H99LFw9Aa5+BaodABP6BH0eX8xKzfFKEPY3/bCvtERKSzf3SXj27IGlL8L0v8HW1dC0Y7B6YP2j0hZCYZ9GZBNVdpUs/eGWjBPrzX2V0xGMSFSVKkGzy+GYbjBvBLz7MDzWNpiq5Iy7YP9fpzyEwsSQyB3puqNdMomuNKTs+PE7mPUgLBgVjL5qOyB47Ld/2JEVS1cqUlFoGhEpf2rUgc6D4ab58JuOwbxWQ1rAgtExT0uSbmGPvhJJNyWNCqDCTUNR+1C4+CnoMw3qHAZv3BbMpvvJG2VuAaiwR1+JpFtCScPMbjGzD83sIzO71ZfVNrMpZrbSP9fy5WZmQ8xslZktNbMWEfvp6euvNLOeEeUtzWyZ32aImVki8VZEYd+cllKN8qDXZLjsueDnF66AJ7vA6rLT/Bj26CuRdIs7aZjZscB1QCvgBOBcM2sKDAKmOeeaAtP8zwCdgab+0Rd4zO+nNnAP0Nrv657CROPr9I3YrlO88VZUFb55xAyOPAdunAPnPAzfrQxWDnzpGtj4edjR6T4LyTiJjJ46CpjrnNsGYGbvABcAXYHTfZ0xwEzgDl8+1gU973PNLMfMGvi6U5xzG/1+pgCdzGwmUNM5N8eXjwW6AZMTiLnCyZjmkawqcGJvOP6S4G7y94fC8tfhxD68Uftq/j7jm1BGLyVj9JVIeZJI0vgQuM/M6gDbgS5AAXCgc24dgHNunZnV9/Vzga8jtl/ty/ZVvjpK+S+YWV+CKxIaN26cwEcqfxrmZLMmSoKosM0j++0PZ/wBWvaCmffj5j3OKW4M5+7qylN0ZM1muHPCMoC0Jg4lCckUcTdPOeeWAw8AU4C3gA+AfQ1xidYf4eIojxbLSOdcnnMur169evuMu6LJ2OaRmg3g/CFcXfUR5u85gjurPM/0/X5H90qz+GnnzorTPCdSxiTUEe6cG+2ca+GcOxXYCKwEvvHNTvjnwsWjVwMHRWzeCFhbQnmjKOUSIdOnoZi9tR59dg7k8h138Z2rycNVR/B61bs4ZOv8sEMTqZASurnPzOo759abWWPgbeAk4A/Ad865wWY2CKjtnPu9mZ0D3ETQjNUaGOKca+U7whcChaOpFgEtnXMbzWwBMACYB7wJDHXOvbmvmHRzX2ZpN3j6f5vnjD2cV2kuAyu/yEGVNsBh7eHsvwbzXYnIPqXr5r6Xzexj4DWgv3NuEzAYONvMVgJn+58h+KP/ObAK+BdwI4DvAL8XWOAffy3sFAduAEb5bT5DneBSRGTznKMSk/a05Rz3CB8eMxDWFMCIk2Fif9hSAYYgi5QBmkZEyr1i537athHeexjmPQ6WBSfdCO1uDdb4EJG9xHqloaQhFd+mL2H6vbDsJaheB04bBHm9gmG8IgJo7imR/6l1MFw4CvrOhPpHw+SBMKw1fPxqmZuWRKSsU9KQzNGwOfR8Da4YF1xljOsBT3SEr+aFHZlIuaGkIZnFLJhBt99sOG9I0HT1RAd48Sr4dlXY0YmUeUoakpmyKkPLnnDzomDBp89mwPDW8OZA+PHbsKMTKbOUNCSzVa0Bp/0ebl4MLXoGa3f8sxnMegh2bAs7OpEyR0lDBOBX9eHch+HGuXDIqcFoq6EtYfEzsGd3yduLZAglDZFI9X4Dlz8XrONRsyG82h9GnAIrp2qklQhKGiLRHdwW+kyFi56EnT/CsxfC091g3dKwIxMJlZKGSHHM4Nju0H8BdBoM6z6Ax0+FCdfD5q9L3l6kAkpkPQ2RzFC5KrS5AU64HN57BOY+Bh+9Am36wcm3QXZOqOEVO42KSApoGhGR0tr8Ncy4Dz54AbJrBaOv8noHySXNCteIj1zyN7tKVqmmx1fSEdA0IiKpk3MQXDACrn8HGhwPbw2CYSfChxPS3lme6BrxhUlnzebtOGDN5u3cOWEZExdrVmCJTklDJF4NToCrJ8JVL0OVGjC+F4xqD1++n7YQEl0jPtGkA0HiaTd4OocMeoN2g6cr4VRwShoiiTCDw8+Cfu9C1+GwdR082RmevwI2fJrywxe3Fnysa8QnmnR0pZJ5lDREkqFSFjS/EgYshDPvhi9mwfA28Ppv4Yf1JW8fp0TXiE806STjSkXKFyUNkWSqWh1OvR1uWQIn9oZFY2FIc5j5AOz4MemHS3SN+ESTTqJXKlL+aMitSCrUqAtdHoRW18O0v8DMv0PBE3DGH6DZlcGEiUnSrXlu3KOdCreLd/RUw5zs/67RXrRcKiYNuRVJh6/mwZS74et5UO9IOOsvwRTtZmFHlpBkDPmVskFDbkXKksat4dp8uORp2L0Tnr8UxpwHaxaFHVlCEm0ek/JHVxoi6bZ7Jyx8CmYOhm3fwrEXQfs/BcvSioREVxoiZVVWFWh1XbCGxym3wydvwKN5kH8XbNsYdnQi+5RQ0jCz35rZR2b2oZk9b2bVzOwQM5tnZivN7EUzq+rr7ud/XuXfbxKxnzt9+Qoz6xhR3smXrTKzQYnEKlLmVKsJ7e8OhukefwnMGRaMtHp/KOz6OezoRKKKO2mYWS5wM5DnnDsWyAIuAx4AHnHONQU2Ab39Jr2BTc65w4FHfD3M7Gi/3TFAJ2C4mWWZWRYwDOgMHA1c7uuKVCwH5ELXYXDDbGiUB2//MbjyWPoS7NkTdnQie0m0eaoykG1mlYHqwDrgTGC8f38M0M2/7up/xr/f3szMl7/gnPvZOfcFsApo5R+rnHOfO+d2AC/4uiIV04HHBFOSXD0Rqh0AE/rAv84IbhQUKSPiThrOuTXAQ8BXBMliC7AQ2Oyc2+WrrQYKh1HkAl/7bXf5+nUiy4tsU1z5L5hZXzMrMLOCDRs2xPuRRMqGw86AvrPggsfhx2+DUVbPXgLrl4cdmUhCzVO1CL75HwI0BGoQNCUVVTg8K9qAdBdH+S8LnRvpnMtzzuXVq1evpNBFyr5KleCEy4L+jrP+Al/NhcfawqQBwfxWIiFJpHnqLOAL59wG59xOYALQFsjxzVUAjYC1/vVq4CAA//4BwMbI8iLbFFcukjmqVIOTbw2mJWndD5Y8D0NbwPT74Ofvw45OMlAiSeMroI2ZVfd9E+2Bj4EZwEW+Tk/gVf96kv8Z//50F9wkMgm4zI+uOgRoCswHFgBN/WisqgSd5ZMSiFek/KpeGzrdDzfNh990gln/CEZaLRgV3PchkiaJ9GnMI+jQXgQs8/saCdwB3GZmqwj6LEb7TUYDdXz5bcAgv5+PgHEECectoL9zbrfv97gJyAeWA+N8XZHMVftQuPhJ6DMN6jSFN34Hw08K7vWoYDfqStmkO8JFyivnYMVkmHoPfPspNG4LHe4Nhu2KlJLuCBep6MzgyC5wwxw49xH4bmWwcuBL18DGz8OOTiooJQ2R8i6rMuRdG0xLctod8Gk+PNoKJg+CH78LOzqpYJQ0RCqK/fYP1usYsAiaXQHzHw86y997BHZqUSRJDiUNkYqmZgM4fwjc8D4cfBJM/TMMzQuG62paEkmQkoZIRVX/KLjiRej5WrCS4MR+MPJU+GxG2JFJOaakIVLRHXIqXDcDLhwNP22Bp7vBMxfCfz4MOzIgWP2v3eDpHDLoDdoNns7ExWvCDkn2QUlDJBNUqgTHXQQ3FUCHv8HqBTDiZJjYH7aGN9FC4XKxazZvxwFrNm/nzgnLlDjKMCUNkUxSeT9oOwBuXgIn9Ydl42BIC5j2V/hpa9rDeTB/xV7riwNs37mbB/NXpD0WiY2Shkgmql4bOt4HNy2Ao86Fd/9fMNJq3si0TkuydnP0UV3FlUv4lDREMlmtJnDhKOg7M+g4nzwQhrWGjyelZVqShjnZpSqX8ClpiAg0bB6MsrpiXLCG+bir4YmO8NW8lB52YMcjyK6StVdZdpUsBnY8IqXHlfgpaYhIwAx+0xH6zYbzhsCmL+GJDvDiVfDdZyk5ZLfmudzf/Thyc7IxIDcnm/u7H0e35lHXW5MyQBMWikh0O36EOcNg9j9h10/BVCWn3RHc8yEVjiYsFJHEVK0Bp/0+mNOqRU9YMBr+2QxmPQQ7toUdnYRESUNE9u1X9eHch+HGucGNgtPvhaEtYfEzsGd3ydtLhaKkISKxqfcbuPw56DUZajaEV/vDiFNg5VQtAJVBlDREpHQObgt9psLFT8HObfDshTC2K6z7IOzIJA2UNESk9MzgmAug/3zo9AD8Zxk8fhpMuB42fx12dJJCShoiEr/KVaFNv6CzvN0t8PHEoL9jyp9g++awo5MUUNIQkcRl58DZfwkmRDy2O8weAkOawZzhsGtH2NFJEilpiEjy5BwEF4yA62dBg2aQfycMOxE+fFmd5RWEkoaIJF+D46HHRLjqZahSA8ZfC6Paw5fvhx2ZJCjupGFmR5jZkojHVjO71cxqm9kUM1vpn2v5+mZmQ8xslZktNbMWEfvq6euvNLOeEeUtzWyZ32aImVliH1dE0urws6Dfu9B1OGxdB092hucvhw2fhh2ZxCnupOGcW+Gca+acawa0BLYBrwCDgGnOuabANP8zQGegqX/0BR4DMLPawD1Aa6AVcE9hovF1+kZs1yneeEUkJJWyoPmVMGAhtP8TfPEuDG8Dr/8Wvv8m7OiklJLVPNUe+Mw59yXQFRjjy8cA3fzrrsBYF5gL5JhZA6AjMMU5t9E5twmYAnTy79V0zs1xwQRZYyP2JSLlTdXqcMrv4JYlcGIfWDQ2WMNj5gPBPFdSLiQraVwGPO9fH+icWwfgn+v78lwgcgD3al+2r/LVUcp/wcz6mlmBmRVs2LAhwY8iIilVoy50+Udwj8fh7WHm34PksfAp2L0r7OikBAknDTOrCpwPvFRS1ShlLo7yXxY6N9I5l+ecy6tXr14JYYhImVDnMLj0aeg9JVgM6rVbYEQ7WPGWRlqVYcm40ugMLHLOFTZOfuOblvDP6335auCgiO0aAWtLKG8UpVxEKpKDWsG1+XDJ08FSs89fCmPOgzWLwo5MokhG0ric/zVNAUwCCkdA9QRejSjv4UdRtQG2+OarfKCDmdXyHeAdgHz/3vdm1saPmuoRsS8RqUjM4Ojzof886PIQrF8O/zoDxveGTf8OOzqJkNAiTGZWnaA/4lDn3BZfVgcYBzQGvgIuds5t9H/4HyUYAbUN6OWcK/DbXAv8we/2Pufck748D3gKyAYmAwNcCQFrESaRCuCnrcHiT3OGgdsNrfoGnejVa4cdWYUV6yJMWrlPRMqurWthxn2w+FmoVhNOuT1IIFWqhR1ZhaOV+0Sk/KvZELoOgxtmQ6NWMOVuePREWPoS7NkTdnQZSUlDRMq+A4+Bq8ZDj1eDyREn9An6PL6YFXZkGUdJQ0TKj0NPh77vwAUjYdt3wSirZy8JOs4lLZQ0RKR8qVQJTrg0mIb97L/CV3PhsbYwaUAwv5WklJKGiJRPVaoFCz/dsgRa94Mlz8PQFjD9Pvj5+7Cjq7CUNESkXJu4YjvtlpzNqdv/wdTdLWDWP4JpSRaMCm4WlKRS0hCRcmvi4jXcOWEZazZv5yt3IH223cglu+/j22qN4Y3fwfCTYPnrmpYkiZQ0RKTcejB/Bdt37t6rbP7OQ+j6w11w2fPBneYvXhms4/H1gpCirFiUNESk3Fq7eXv08i0/wZFd4IY5cO4j8N1nMPosGNcTNn6e5igrFiUNESm3GuZk77s8qzLkXQs3L4LTBsHKt+HRVjD5DvjxuzRGWnEoaYhIuTWw4xFkV8naqyy7ShYDOx6xd8X99ocz7oSbF0OzK2D+SBjSDN57BHZGv1qR6JQ0RKTc6tY8l/u7H0duTjYG5OZkc3/34+jWPOp6bbD/r+H8IUGz1cFtYeqfYWheMFx3z+7o28heNGGhiGSuL96Ft/8I65bAgcfB2X8JVhPMQJqwUESkJIecAtfNgAtHw89b4Jnu8PQF8J9lYUdWZilpiEhmq1QJjrsomJakw33BioEjToFXboAta8KOrsxR0hARAai8H7S9KZiWpO1N8OH4YFqSqX+Bn7aEHV2ZoaQhIhIpuxZ0+Ftw5XHUefDew8G0JPNGwq4dYUcXOiUNEZFoah0MF46CvjOh/tEweSAMbw0fv5rR05IoaYiI7EvD5tDzNbjiJcjaD8b1gNEd4Kt5YUcWCiUNEZGSmMFvOkC/9+D8obD5K3iiA7x4FXy7Kuzo0kpJQ0QkVlmVoUWPYFqSM+6Cz2YETVZv3A4/bAg7urRIKGmYWY6ZjTezT8xsuZmdZGa1zWyKma30z7V8XTOzIWa2ysyWmlmLiP309PVXmlnPiPKWZrbMbzPEzCyReEVEkqJqDTjt98G0JC16QsETQWf5rAdhx7awo0upRK80/gm85Zw7EjgBWA4MAqY555oC0/zPAJ2Bpv7RF3gMwMxqA/cArYFWwD2FicbX6RuxXacE4xURSZ5f1YdzH4b+8+DQ02D634JhuouerrDTksSdNMysJnAqMBrAObfDObcZ6AqM8dXGAN38667AWBeYC+SYWQOgIzDFObfRObcJmAJ08u/VdM7NccFcJ2Mj9iUiUnbUbQqXPQu9JkPNXJh0U3CD4MqpFW6kVSJXGocCG4AnzWyxmY0ysxrAgc65dQD+ub6vnwt8HbH9al+2r/LVUcpFRMqmg9tCn6lw8VOwcxs8eyGM7Qprl4QdWdIkkjQqAy2Ax5xzzYEf+V9TVDTR+iNcHOW/3LFZXzMrMLOCDRsyozNKRMooMzjmAug/Hzo9EMxjNfI0mNA3GHVVziWSNFYDq51zhYOVxxMkkW980xL+eX1E/YMitm8ErC2hvFGU8l9wzo10zuU55/Lq1auXwEcSEUmSylWhTb+gs7zdrfDRxGAa9rfvhu2bw44ubnEnDefcf4CvzaxwtZP2wMfAJKBwBFRP4FX/ehLQw4+iagNs8c1X+UAHM6vlO8A7APn+ve/NrI0fNdUjYl8iIuVDdk4w5fqAhXBsd3h/aLAA1JzhsOvnsKMrtYTW0zCzZsAooCrwOdCLIBGNAxoDXwEXO+c2+j/8jxKMgNoG9HLOFfj9XAv8we/2Pufck75dMT25AAAKmUlEQVQ8D3gKyAYmAwNcCQFrPQ0RKY2Ji9fwYP4K1m7eTsOcbAZ2PKL4RZySYd1SmPIn+HwG5BwMZ90Dx3QPmrVCFOt6GlqESUQy1sTFa7hzwjK27/zf8NjsKln7Xv0vWVZNhbf/BOs/goYtgkkSm7RL7TH3QYswiYiU4MH8FXslDIDtO3fzYP6K1B/88LOg37vQdTh8/x94qgs8dxlsSMOxE6CkISIZa+3m7aUqT7pKWdD8yqC/48y74d/vwfCT4LVb4ftv0hNDKSlpiEjGapiTXarylKlaHU69PVgA6sQ+sPjpYFqSmYPh5x/SG0sJlDREJGMN7HgE2VWy9irLrpLFwI5HFLNFitWoC13+EdzjcXh7mHl/MC1JwZOwe1c4MRWhpCEiGatb81zu734cuTnZGJCbk52eTvCS1DkMLn0aek+BWk3g9VvhsbawYnLo05Jo9JSISFnmHCx/Dab+GTZ+BgefDB3uhdwWJW5aGho9JSJSEZjB0ecHM+l2eQg2fAL/OgPGXwub/p32cJQ0RETKg6wq0Oq6YFqSU26HT96ER0+E/Ltg28a0haGkISJSnlSrCe3vDlYPPP4SmDMsmJZk9hDY+VPKD6+kISJSHtVsCF2HwQ2zoVErmHI3fPpWyg9bOeVHEBGR1DnwGLhqPHw1DxqdmPLDKWmIiFQEjVun5TBKGiIiCUj7LLkhU9IQEYlT0Vly12zezp0TlgFU2MShjnARkTiFOktuSJQ0RETiFPosuSFQ0hARiVOZmSU3jZQ0RETiVOZmyU0DdYSLiMSpsLNbo6dERCQm3ZrnVugkUZSap0REJGZKGiIiErOEkoaZ/dvMlpnZEjMr8GW1zWyKma30z7V8uZnZEDNbZWZLzaxFxH56+vorzaxnRHlLv/9VfltLJF4REUlMMq40znDONYtY8WkQMM051xSY5n8G6Aw09Y++wGMQJBngHqA10Aq4pzDR+Dp9I7brlIR4RUQkTqlonuoKjPGvxwDdIsrHusBcIMfMGgAdgSnOuY3OuU3AFKCTf6+mc26OC9akHRuxLxERCUGiScMBb5vZQjPr68sOdM6tA/DP9X15LvB1xLarfdm+yldHKRcRkZAkOuS2nXNurZnVB6aY2Sf7qButP8LFUf7LHQcJqy9A48aN9x2xiIjELaErDefcWv+8HniFoE/iG9+0hH9e76uvBg6K2LwRsLaE8kZRyqPFMdI5l+ecy6tXr14iH0lERPYh7qRhZjXMbP/C10AH4ENgElA4Aqon8Kp/PQno4UdRtQG2+OarfKCDmdXyHeAdgHz/3vdm1saPmuoRsS8REQlBIs1TBwKv+FGwlYHnnHNvmdkCYJyZ9Qa+Ai729d8EugCrgG1ALwDn3EYzuxdY4Ov91Tm30b++AXgKyAYm+4eIiITEgoFJFUdeXp4rKCgIOwwRkXLFzBZG3DpRLM09JSISsvK0ZKyShohIiMrbkrGae0pEJETlbclYJQ0RkRCVtyVjlTREREJU3paMVdIQEQlReVsyVh3hIiIhSsaSsekcfaWkISISskSWjE336Cs1T4mIlGPpHn2lpCEiUo6le/SVkoaISDmW7tFXShoiIuVYukdfqSNcRKQcS8boq9JQ0hARKecSGX1VWmqeEhGRmClpiIhIzJQ0REQkZkoaIiISMyUNERGJWYVbI9zMNgBfpmj3dYFvU7TvZFB8iVF8iVF8iQszxoOdc/VKqlThkkYqmVlBLAuvh0XxJUbxJUbxJa48xKjmKRERiZmShoiIxExJo3RGhh1ACRRfYhRfYhRf4sp8jOrTEBGRmOlKQ0REYpYxScPMDjKzGWa23Mw+MrNbfHltM5tiZiv9cy1fbmY2xMxWmdlSM2vhy5uZ2Ry/j6Vmdmkxx7vGzDaY2RL/6JOuGP17uyOOPamY4+1nZi/67eeZWZM0ncMzImJbYmY/mVm3RM9hHPEd6f8tfzaz24vsq5OZrfCxDwrp/EWNr7j9RDne6Wa2JeL8/SmN5+/fZrbMH7egmOMV+/ub4vN3RJHfv61mdmsI5+9K/7mXmtn7ZnZCxL6S/vuXNM65jHgADYAW/vX+wKfA0cA/gEG+fBDwgH/dBZgMGNAGmOfLfwM09a8bAuuAnCjHuwZ4NIwY/Xs/xHC8G4ER/vVlwIvpii9in7WBjUD1RM9hHPHVB04E7gNuj9hPFvAZcChQFfgAODqE81dcfFH3E+V4pwOvp/v8+ff+DdQt4Xgl/n6kKr4i/9b/IbhHId3nry1Qy7/uzP/+xqTk9y9Zj5QfoKw+gFeBs4EVQIOIf/QV/vXjwOUR9f9br8h+PsAnkSLl11DKpJHMGIktaeQDJ/nXlQluKrJ0nkOgL/BsMftP6ByWFF9EvT+z9x/lk4D8iJ/vBO5M9/krLr7i9hOl/HRK8UcvmfERW9KI6f9YKs8f0AGYXcx7aTl/vrwWsCadv3/xPjKmeSqSv4xrDswDDnTOrQPwz/V9tVzg64jNVvuyyP20Ivgm8Fkxh7rQX3qON7OD0hxjNTMrMLO5FqXpp+j2zrldwBagTpriK3QZ8Pw+DhXXOYwxvuLEEvde9VJ0/kq7n2hOMrMPzGyymR0T537jic8Bb5vZQjPrW0ydWM9zKuIrVNLvX7rOX2+Cqy5Iw+9fIjIuaZjZr4CXgVudc1v3VTVK2X+HmplZA+BpoJdzbk+Uuq8BTZxzxwNTgTFpjrGxC+4svQL4PzM7rJTbpzq+wnN4HME3pmjiOoeliK/YXUQpi3ZeUn3+Et3PIoJmlxOAocDEJO03Fu2ccy0Iml36m9mp0Q4VpSyd568qcD7wUjFV0nL+zOwMgqRxR2FRlGpJ+/1LVEYlDTOrQvCP+axzboIv/sb/8Sr8I7bel68GIr/ZNgLW+no1gTeAPzrn5kY7lnPuO+fcz/7HfwEt0xmjc67w+XNgJsG3nqL+u72ZVQYOIOhfSHl83iXAK865ndGOFc85LGV8xSkp7l/US9H5K+1+9uKc2+qc+8G/fhOoYmZ10xFfxO/feuAVoFWUarGe56TH53UGFjnnvinmM6T8/JnZ8cAooKtz7jtfnLLfv2TImKRhZgaMBpY75x6OeGsS0NO/7knQDllY3sMCbYAtzrl1/tvJK8BY51xx31AKfzkKnQ8sT2OMtcxsP7/PukA74OMoh4zc70XAdOcbSFMZX8R2l7OPpoHSnsM44ivOAqCpmR3i/70v8/soKtXnr7T7KVrv175uYVNqJeC7fdRPVnw1zGz/wtcE/QYfRqla0u9HSuKLUNLvX0rPn5k1BiYAVzvnPo2on5Lfv6RJdadJWXkAJxNcui0FlvhHF4I2wGnASv9c29c3YBhBf8UyIM+XXwXsjNjHEqCZf++vwPn+9f3ARwQd5TOAI9MYY1v/8wf+uXfEMSJjrEZwab4KmA8cmo74/HtNgDVApSLHiPscxhHfrwm+rW0FNvvXNf17XQhGv3wG3BXS+YsaX3H78dv0A/r51zdFnL+5QNs0xXeoP+YH/viR5y8yvmJ/P9Lw71udIAEcUOQY6Tx/o4BNEXULIvaV9N+/ZD10R7iIiMQsY5qnREQkcUoaIiISMyUNERGJmZKGiIjETElDRERipqQhIiIxU9IQEZGYKWmIiEjM/j9uqr93IQYVbgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(x, y, 'o')\n",
    "plt.plot(x,line_fitter.predict(x.values.reshape(-1,1)))\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAEDCAYAAADZUdTgAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3Xl8FeW9x/HPj90FEQ0uBRENFheMBihLUFFURKrUtQii4IagiGi9rdZ7tbXXertKYyyISgVBtK5FqqBFQCUsBpAAWjShFiMiIIKiIttz/3gmNIGEnHCWOSfzfb9e8+KcmTkzvzM5/M6cZ575PeacQ0REoqFe2AGIiEjqKOmLiESIkr6ISIQo6YuIRIiSvohIhCjpi4hESNomfTMbZ2ZrzWxZDOs+aGbvBtMHZrYxFTGKiGQaS9d++mZ2BrAZmOCca1+L190C5Drnrk1acCIiGSptz/Sdc28CGyrOM7NsM5tmZgvN7C0zO76Kl/YHJqckSBGRDNMg7ABqaSww1Dn3oZl1Af4M9CxfaGZHA8cAb4QUn4hIWsuYpG9mBwJ5wLNmVj678W6rXQE855zbkcrYREQyRcYkfXxT1Ebn3Kl7WecK4OYUxSMiknHStk1/d865L4F/mdnlAOadUr7czNoBzYG5IYUoIpL20jbpm9lkfAJvZ2ZlZnYdcCVwnZktAZYDP6rwkv7A0y5duyOJiKSBtO2yKSIiiZe2Z/oiIpJ4aXchNysry7Vp0ybsMEREMsrChQvXO+da1LRe2iX9Nm3aUFRUFHYYIiIZxcz+Hct6at4REYkQJX0RkQhR0hcRiRAlfRGRCFHSFxGJECV9EZEIUdIXEYmQupP0nYPpd8Mni8KOREQkbdWdpL9hJSyaAI+eBeMvhJIZ/otARER2qTtJ/9BsuG05nHsfrPsAJl4Cj5wOS5+DHdvDjk5EJC3UnaQP0OQg6H4rjCyGvg/Bti3w/HXwUAdY8Chs/SbsCEVEQlW3kn65Bo2hw9Vw8wLoNwkOPAxeuQNGtYdZv4FvNtS8DRGROqhuJv1y9erBCRfAda/DNa9Cy04w69fw4Enw6p2w8eOwIxQRSam0q7KZFGZwdJ6fPnsPCvPhnUdhwVg4+TLfJHT4SWFHKSKSdHX7TL8qh58IF4+BEe9Clxvh/akwOg8mXQ4fva0ePxlozOxSCkvXV5pXWLqeMbNLQ4pIJH1FL+mXO/go6P0A3LYMzvpv37//iR/CY+fA+y/Dzp1hRygxymnVjOFPLd6V+AtL1zP8qcXktGoWcmQi6Sftxsjt1KmTC2UQlW3fwuKJMLcAvvgIDm0LeSPglCv8hWFJa+WJfmCX1kycv4qCAbnkZWeFHZZIypjZQudcp5rWi+6Z/u4a7gedb4DhC+GycdBwf3h5BIzKgbdHwZZNYUcoe5GXncXALq3Jf6OEgV1aK+GLVENJf3f1G0D7S+HGN+GqF+Gw4+Ef98KD7eH1e+CrNWFHKFUoLF3PxPmrGNGzLRPnr9qjjV9EPCX96phBdk+4+m8wZBa0PRsKH4JRJ8OUW2D9h2FHKIHypp2CAbnc3qsdBQNyK7Xxi8h/qE2/Njas9Il/8STYsRWO/yGcdhu0qrEZTZJozOxSclo1q9SkU1i6nuKyTQztkR1iZCKpE2ubvpL+vti8FuY/4vv6b9kER5/m+/ofd67/hSAikmK6kJtMBx4GZ/+PL/B23q/hi3/BU5fD6O6w5BnYsS3sCEVEqqSkH4/GTaHbzf5Gr4tGg9sBLw6B/FyYNxq2fh12hCIilSjpJ0KDRnDqABg2F/o/A81awbQ7fY2fmb+Gr3VBUUTSg5J+ItWrB+16w7XT4NrXoHU3mP0b393z73f4m75EREIUjYJrYWjdBVpPhnUrYE4+LHwCisbBSRf7i75H5oQdoYhEkM70k61FO7joYbh1CXS7CT6Y5kf0evJiWDlbBd5EJKWU9FOlWUvo9b++x8/Z98CaZTChrx/Td/mLsHNH2BGKSAQo6afafgfD6T+BkUvhgqCmz7ODoaCTb/7ZtiXsCCNHpZklSpT0w9KwCXS6BoYXweXjoUkzmHqbH9Lxzd/Dt1+EHWFkqDSzRInuyE0XzsFHb/mKnqUzoNGB0HEwdL3JNw1JUqk0s2Q63ZGbaczgmDPgqhfgxreg3fn+Bq8/nQIv3QRr/xl2hHWaSjNLVCjpp6Mjc+DSx2DEIt8EtOwF+HMXeOoKWDUv7OjqJJVmlqhQ0k9nzdtAn9/5IR173Akfz4Nx58Hj58GKVzWkY4KoNLNEidr0M8nWr2HRk35Ix00fQ4vj/ZCOJ1/uS0HIPlFpZqkLVFq5Ltuxzfftf3sUrF0OB7X0F3w7DvJF4EQkcnQhty6r3xByfgzD5sCVz8Ehx8Jrd/sCbzPu8/X+RUSqoKSfycz8wC2Dp8L1M3zvn7f+6Au8vTwSPs+Mm4t0c5RI6tSJpK+kgR+ysd9Ef7PXKVfAu5P8Xb5/HQSrF4cd3V7p5iiR1Kkx6ZvZODNba2bLqlluZpZvZiVmVmxmHYL5p5rZXDNbHszvl+jgyylpVJDVFvrm+zIP3W+F0jdg7Jkw/kIomZGUAm/xfunmZWft6jHzx9dW7OpJo77yIokXy5n+E0DvvSw/HzgumIYAo4P53wBXO+dOCl4/yswO3vdQq6ekUYWmR8A5v/AF3s69D9Z9ABMv8RU+lz4HO7YnbFeJ+NLVzVEiqVFj0nfOvQls2MsqPwImOG8ecLCZHemc+8A592GwjdXAWqBFIoKuipJGNZoc5M/4RxZD3wJf0O356+ChDrDgUdj6TVqcqWfyzVFqXpRMkog2/ZbAxxWelwXzdjGzzkAjoMr/BWY2xMyKzKxo3bp1+xREJieNlGjQGDpcBTcvgH6T/ODur9wBo9rTZ8ME7p70Zmhn6pl+c5SaFyWTxNRP38zaAFOdc+2rWPZ34AHn3NvB8xnAT51zC4PnRwKzgEHBL4G92pd++hWTRl521h7PpQrOwaq5vq//h9PZUX8/ntnZk69zhzB6ybZ9OlPf14JldeHmKBVsk7DF2k8f51yNE9AGWFbNskeA/hWerwCODB4fBCwCLo9lP845Onbs6Gpr9KwSN6dkXaV5c0rWudGzSmq9rUwU9/tfs9y5F25023/R3G2752C3vKCfc2uWxbz/OSXrXO59r+2KYffnUfGH6f90R/9sqvvD9H+GHYpEEFDkYsixiWjemQJcHfTi6Qpscs59amaNgBfx7f3PJmA/1RraI3uPs6q87KyMOUuMV9zNC4efSGHO/3KhFbDke/1os24mjM6DSZfDR2/X2OOnuGxTpTPb8jb+4rJNcb2vTKLmRckYNX0rAJOBT4Ft+Pb664ChwNBguQEP49vrlwKdgvkDg9e8W2E6tab97cuZfrziPVNOh18a5WfXf5j+z1qfZe9+Zj5/+Yeu4BdD3NZft3Hu3oOcG9vTufemOLdjR7LCz2j6pSPpgBjP9GNqcknlFEbSj/c/bbyvT9SXxr42L1S3/7Ezljk3f6xzD57sk39+B+eKnnBu25Zabb+uS4cvfZFYk74KrgXivRAXz+sTcSE6qRcSd2yH9//mL/quKYYDj4Cuw3yt/ybqoSKSDlRlcx/88bUV5L9Rwoiebbm9V7uUvj7sL42YOAcrZ8KcP8HKWdD4IOh0rf8CaHpE4vYjIrWmKpu1FO+FuHhfH08/95RdSDWD7J5w9d9gyCxoezYU5sOok2HKLbC+JLH7E5GE05k+8Z8pp33zTDJtWAmFBb7A2/bv4Pgfwmm3+QJwIpIyat6phXhvDor39XXi5rLN62D+GHjnMdiyEY4+DU4bCW3P8b8QRCSplPQzSF24I3WX776CRRNg7sPw5Sdw2Em+9k/7S/zgL7KHOvX3l9CoTT+D1Kmbyxo3hW43w4h34aIx4HbCi0MgPxfmjfHj/Eolqt0jqaQzfUmunTvhw9dgzihf62e/5tB5CHS+EQ44NOzo0kbGXtORtKEzfUkP9epBu95w7TS49jVonQezf+PH833lv+CLf4cdYVpQaXBJFSV9SZ3WXaD/U768c/tLoegvvtnn+ethzdKwowuVavdIqijpS+q1aAcXPQy3LoFuN8GKV2HMafDkJbBydlKGdExnmT6egGQWtelL+L7dCEWP+wu9X6+F73XwPX5OuBDq1Q87uqRT7x1JBHXZlMyzbQssmezv8t2wEg7Jhrxb4JT+0LBJ2NGJpDVdyJXM07CJL+I2vAguH+/H95060pd5eOsP/heBiMRFSV/ST736cNJFcMNMGPQyHHEyzLjP9/iZfjd8uTrsCNOKBmaX2lDSl/RlBsecAVe9ADe+Be3Oh3mjYVQOvHQTrFsRdoRpQTd3SW2oTV8yyxcf+RIPi56E7d9Cuz7QfaTvDhphurlL1KYvdVPzNtDnd3Dbcuhxp7/Ld1wvePw83/Vz586wIwyFbu6SWCnpS2Y64FA46y6f/M//rW/nn3wFjO4G7z4F27eGHWFK6eYuiZWSvmS2RgdAlxthxCK45FGo1wBeGgb5p/o6/999FXaESaebu6Q21KYvdYtzUDLDF3j76C0/hu8ProcuQ+HAw8KOLil0c5eAbs4SgbKFPvm//zLUbwS5V/qbvQ45NuzIRBJOF3JFWnWEfk/6m71OuQIWT4SHOsKzg2H14rCjEwmFkr7UfVltoW8+jFzqa/qUzICxZ8L4vlD6RuQKvEm0KelLdDQ9As75he/xc+59/uauJy+GR86Apc/Bju1hRyiSdEr6Ej1NDvJn/COLoW8BbPsWnr8OCjrCgkf9c5E6SklfoqtBY+hwlR/Upd8kOKAFvHIHPNgeZv8WvtkQdoQiCaekL1KvHpxwAVz3OlzzKrTsCDPv98l/2l2w8eOwIxRJmAZhByCSNszg6Dw/fbYc5uTDgrF+an+ZbxI6/MSwoxSJi870Rapy+ElwySMw4l3oPMT39R/dDSb9GD6aox4/krGU9EX25uCjoPcDcNsyOOtu+KQInugDj58L70+NbIE3yVxK+iKx2P8Q6PFTGLkM+vweNq+FZ66EhzvDogmw/buwIxSJiZK+SG002h863wC3LIJLH/dDPE65xQ/s8vYo2PJl2BGK7JWSvsi+qN8ATr7Mj+h11YvQoh38414/pOPr98JXa8KOMGYabjFalPRF4mEG2T1h0BQ/pm92TyjM94O5T7kF1peEHWGNNNxitKjKpkiifV4Kcwtg8STYsdXfA9D9Nl8ALk1puMXMpyqbImE5NBsueND3+Dn9dvjXm/BYT3jiAvjw9bTs7qnhFqNDSV8kWQ48DM6+xxd463W//wUw6TIYcxoU/xV2bAs7wl003GJ01Jj0zWycma01s2XVLDczyzezEjMrNrMOFZYNMrMPg2lQIgMXyRiNm0LecLh1CVw0GnZuhxdugPwOMG8MbP061PA03GK0xHKm/wTQey/LzweOC6YhwGgAMzsEuBfoAnQG7jWz5vEEK5LRGjSCUwfAsLnQ/2k46Hsw7We+xs/MB+Drz0MJq7hsU6U2/LzsLAoG5FJctimUeCS5YrqQa2ZtgKnOufZVLHsEmOWcmxw8XwGcWT45526sar3q6EKuRMqqeTDnT7DiFWiwn6/62W04ND867Mgkw8R6ITcRBddaAhXLEJYF86qbvwczG4L/lUDr1q0TEJJIhmjd1U/rVvgCb0V/gXceh/aX+AJvR5wcdoRSxyTiQq5VMc/tZf6eM50b65zr5Jzr1KJFiwSEJJJhWrSDix727f5dh8GKV/0F3ycv8b1/0rDHj2SmRCT9MuCoCs9bAav3Ml9EqtOsJZx3v+/uefY9sGYpjL8QHu0Jy1+CnTvCjlAyXCKS/hTg6qAXT1dgk3PuU2A60MvMmgcXcHsF80SkJvs1h9N/4gdzv+BB2LIRnh0EBT/wTUDbtoQdoWSoWLpsTgbmAu3MrMzMrjOzoWY2NFjlFWAlUAI8CtwE4JzbAPwKeCeY7gvmiUisGjaBTtfC8CK4fLwf33fqSF/m4a0/wLcbw45QMozKMIhkEud8G/+cP0HpDGh0IHQcDN1u9l1AJbJUhkGkLjKDY3vAVS/4Cp/f7w3z/uxLO790s+8FJLIXSvoimerIHLjscRixGDpdA8ue94O6TO4Pq+aHHZ2kKSV9kUzXvA30+Z3v8dPjZ7BqLozrBeN6w4ppGtJRKlHSF6krDsiCs37uC7z1/g1sKoPJ/WB0Hrz7FGzfGnaEkgaU9EXqmkYHQNehvtnn4rFg9eClYZB/KhQWwHdfhR2hhEhJX6Suqt8QTukHw+bAlc9B82Pgtbv9kI4zfgWb14UdoYRASV+krjOD486Fa/4O18+AY87wffxHtYept8GGlWFHKCmkpC8SJa06Qb+JMPwdyPkxLJ4ID3WEZwfD6sVhRycpoKQvEkVZx0Hfh3yZh7wRUDIDxp4J4/tC6Ru1KvA2ZnbpHgOuFJauZ8zs0gQHLYmgpC8SZU2PgHN/6bt7nvNLf3PXkxfDI2fA0udgx/YaN5HTqlmlkbbKR+LKadUs2dHLPlAZBhH5j+3fQfEzvrb/5x/6ewC6DYfcgdBwv2pfVp7oB3ZpzcT5qyqNxCWpoTIMIlJ7DRpDh6vh5gW+7X//LHjlDj+k4+zfwjdV10zMy85iYJfW5L9RwsAurZXw05iSvojsqV49OOFCuP4fMPgVaNkBZt7vk/+0u2Djx5VWLyxdz8T5qxjRsy0T56/SoOppLBHDJYpIXWUGbbr76bPlvtlnwVg/tb8Muo+gcPPhDH9q8a4mna7Zh1Z6LulFbfoiUjsbP/aVPReOh21f8+9DTuOrTjfTvtv5/ksCf+ZfXLaJoT2yQw42OmJt01fSF5F9880GeOcxmD8GvvkcWv3AD+be7oe+eUhSShdyRSS59j8EevwURi6DPr+HzWvhmYG+vPOiCb4nkKQdJX0RiU+j/aHzDXDLIrj0cd+1c8otfmCXt0fBlk1hRygVKOmLSGLUbwAnXwY3vglXvQgt2sE/7vU9fl6/F75aE3aEgpK+iCSaGWT3hEFTYMgsaHs2FOb7wdynjID1JWFHGGlK+iKSPN/LhcufgOFF/q7eJU9DQSff9l+2MOzoIklJX0SS79BsuOBBX+Pn9J/Av96Ex3rCExfAh6/XqsCbxEdJX0RS58DD4Oz/8UM6nvdrX8t/0mUw5jQo/ivs2BZ2hHWekr6IpF7jptDtZhjxLlw0GnZuhxdugPwOMG8MbP067AjrLCV9EQlPg0Zw6gAYNhf6Pw3NWsK0n/kePzMfgK8/DzvCOkdJX0RCNWZ2KYX/2gDtzodrp8G1r7Hh0A4w+//8eL6v/Bd88e+ww6wzlPRFJFR7DMKyLZtzPh3K4gunQftLoegvkJ8Lz18Pa5aGHG3mU+0dEQndXgdh2fRJUODtCdi6GbLPhtNGQpvTdxV4E9XeEZEMstdBWJq1hPPu9909z77Hn+2PvxAe7QnLX4KdO8ILPAMp6YtI6GIahGW/5r6P/8ilvs//lo3w7CB/s1fRONi2JfWBZyAlfREJVXnTTsGAXG7v1Y6CAbmV2vj30LAJdLrW3+V7+Xho0gym3ubLPLz1B/h2Y2rfQIZRm76IhGrM7FJyWjWr1KRTq0FYnPN3+M4ZBaVvQKMDoeNgfx/AQd9LXuBpRoOoiEj0fFoMc/4Ey18Aqw85/aD7CF/xs47ThVwRiZ4jc+Cyx2HEYuh0DSx73g/qMrk/rJofdnRpQUlfROqe5m2gz+98j58eP4NVc2FcLxjXG1ZMg507w44wNEr6IlJ3HZAFZ/3cF3jr/RvYVAaT+8HoPHj3Kdi+NewIU05JX0TqvkYHQNehvtnn4rFg9eClYZB/Ksx9GL7bHHaEKaOkLyLRUb8hnNIPhs2BK5+D5sfA9J/7Gj8zfgWb14UdYdLFlPTNrLeZrTCzEjO7s4rlR5vZDDMrNrNZZtaqwrLfmtlyM3vfzPLNdN+0iITMDI47F675O1w/A4453ffxH9Uept7u6/zXUTUmfTOrDzwMnA+cCPQ3sxN3W+33wATnXA5wH/BA8No8oDuQA7QHfgD0SFj0IiLxatUJ+k2E4e9Azo9h8ZPwUEd4djCsXhx2dAkXy5l+Z6DEObfSObcVeBr40W7rnAjMCB7PrLDcAU2ARkBjoCHwWbxBi4gkXNZx0PchX+YhbwSUzICxZ8L4vv6mrzS7p2lfxZL0WwIfV3heFsyraAlwafD4YqCpmR3qnJuL/xL4NJimO+fejy9kEZEkanoEnPtL393znF/CuhXw5MXwyBmw9DnYsT3sCOMSS9Kvqg1+96+8O4AeZrYY33zzCbDdzNoCJwCt8F8UPc3sjD12YDbEzIrMrGjdurp/IUVEMkCTZr6E88hi/wtg27fw/HVQ0BEWPOqfZ6BYkn4ZcFSF562A1RVXcM6tds5d4pzLBe4O5m3Cn/XPc85tds5tBl4Fuu6+A+fcWOdcJ+dcpxYtWuzjWxERSYIGjaHD1XDzAt/2v38WvHKHH9Jx9m/hmw1hR1grsST9d4DjzOwYM2sEXAFMqbiCmWWZWfm27gLGBY9X4X8BNDCzhvhfAWreEZHMU68enHAhXP8PGPwKtOwAM+/3yX/aXbDx45q3kQZqTPrOue3AcGA6PmH/1Tm33MzuM7O+wWpnAivM7APgcOD+YP5zQCmwFN/uv8Q593Ji34KISAqZQZvucOWzMKzQfxEsGOtv9HrhRvjsvbAj3CtV2RQRidfGj/2dvYsmwLav4bjzoPutcHReyoZ0VJVNEYmEMbNL9xhwpbB0PWNml6YuiIOPgvP/z/f4Oetu+KQInugDj58L709NqwJvSvoiktFyWjWrNNJW+UhcOa2apT6Y/Q+BHj+Fkcugz+9h81p45kpf3nnRBNj+Xepj2o2ad0Qk45Un+oFdWjNx/ioKBuRWHlw9LDu2w3sv+VG91iyFpkdC12HQ8RpoclBCd6XmHRGJjLzsLAZ2aU3+GyUM7NI6PRI+QP0GcPJlcONbcNWLkPV9eP0eX+Dt9XvhqzUpD0lJX0QyXmHpeibOX8WInm2ZOH9V9YOqh8UMsnvCoClww0z/uDDfD+Y+ZQSsL0lZKEr6IpLRypt2CgbkcnuvdhQMyK3Uxp92WnaAH4+H4UWQOxCWPA0FneCZgVC2MOm7V9IXkYxWXLapUht+XnYWBQNyKS7bFHJkNTg0Gy540Pf4Of12+Neb8PKtSS/spgu5IiLp4Luv4MvV0KLdPr1cF3JFRGKQFv38ARo33eeEXxtK+iISaWnVzz8FGoQdgIhImMqvAaRlP/8k0Jm+iERe2vbzTwIlfRGJvLTv559ASvoiEmkZ188/Tkr6IhJpGdvPfx+pn76ISB2gfvoiIrIHJX0RkQhR0hcRiRAlfRGRCFHSFxGJECV9EZEIUdIXEYkQJX0RkTikTWnmGCnpi4jEIdNKM6u0sohIHDKtNLPO9EVE4pRJpZmV9EVE4pRJpZmV9EVE4pBppZmV9EVE4pBppZlVWllEpA5QaWUREdmDkr6ISIQo6YuIRIiSvohIhCjpi4hEiJK+iEiEKOmLiERITEnfzHqb2QozKzGzO6tYfrSZzTCzYjObZWatKixrbWavmdn7ZvaembVJXPgiIlIbNSZ9M6sPPAycD5wI9DezE3db7ffABOdcDnAf8ECFZROA3znnTgA6A2sTEbiIiNReLGf6nYES59xK59xW4GngR7utcyIwI3g8s3x58OXQwDn3OoBzbrNz7puERC4iIrUWS9JvCXxc4XlZMK+iJcClweOLgaZmdijwfWCjmb1gZovN7HfBLwcREQlBLEnfqpi3e8GeO4AeZrYY6AF8AmzHD9JyerD8B8CxwOA9dmA2xMyKzKxo3bp1sUcvIiK1EkvSLwOOqvC8FbC64grOudXOuUucc7nA3cG8TcFrFwdNQ9uBl4AOu+/AOTfWOdfJOdepRYsW+/hWRESkJrEk/XeA48zsGDNrBFwBTKm4gpllmVn5tu4CxlV4bXMzK8/kPYH34g9bRET2RY1JPzhDHw5MB94H/uqcW25m95lZ32C1M4EVZvYBcDhwf/DaHfimnRlmthTfVPRowt+FiIjERPX0RUTqANXTFxGRPSjpi4hEiJK+iEiEKOmLiESIkr6ISIQo6YuIRIiSvohIhCjpi4hEiJK+iEiEKOmLiESIkr6ISIQo6YuIRIiSvohIhCjpi4iEaMzsUgpL11eaV1i6njGzS5OyPyV9EZEQ5bRqxvCnFu9K/IWl6xn+1GJyWjVLyv4aJGWrIiISk7zsLAoG5DL8qcUM7NKaifNXUTAgl7zsrKTsT2f6IiIhy8vOYmCX1uS/UcLALq2TlvBBSV9EJHSFpeuZOH8VI3q2ZeL8VXu08SeSkr6ISIjK2/ALBuRye692u5p6kpX4lfRFREJUXLapUht+eRt/cdmmpOxPA6OLiNQBGhhdRET2oKQvIhIhSvoiIhGipC8iEiFK+iIiEZJ2vXfMbB3w7yTuIgtI3p0P8VN88VF88VF88QkzvqOdcy1qWintkn6ymVlRLN2awqL44qP44qP44pPu8YGad0REIkVJX0QkQqKY9MeGHUANFF98FF98FF980j2+6LXpi4hEWRTP9EVEIktJX0QkSpxzGTMBRwEzgfeB5cCtwfxDgNeBD4N/mwfzDcgHSoBioEMw/1RgbrCNYqBfNfsbDKwD3g2m61MRX7BsR4X9Tqlmf42BZ4LXzwfapOj4nVUhtneBLcBFIRy/44O/43fAHbttqzewIoj9zpCOX5XxVbedKvZ3JrCpwvG7J4XH7yNgabDfomr2V+3nN8nHr91un78vgZEhHL8rg/ddDBQCpyTz85eoKek7SGiwcCT/STxNgQ+AE4Hflh9Y4E7gN8HjPsCrwYezKzA/mP994Ljg8feAT4GDq9jfYKAg1fEFyzbHsL+bgDHB4yuAZ1IVX4VtHgJsAPYP4fgdBvwAuJ/KSaE+UAocCzQClgAnhnD8qouvyu1Usb8zgampPn7Bso+ArBr2V+PnI1nx7fa3XoO/MSnVxy+P/3wBnM9/8ktSPn+JmpK+g6QGD38DzsV/ox5Z4Q+3Inj8CNC/wvq71tttO0sIvgR2mz+YWiStRMZHbEl/OtAteNwAfyegpfL4AUOASdVsP6nHr8J6v6ByUu1nVKfLAAADn0lEQVQGTK/w/C7grlQfv+riq247Vcw/k1okrUTGR2xJP6b/X8k8fkAvYE41y1Jy/IL5zYFPUvn529cpY9v0zawNkIv/WXS4c+5TgODfw4LVWgIfV3hZWTCv4nY647+NS6vZ1aVmVmxmz5nZUSmMr4mZFZnZPDO7qJrd7Hq9c247/qfsoSmKr9wVwOS97CqZx686scRdab0kHb/abqcq3cxsiZm9amYn7eN29yU+B7xmZgvNbEg168R6nJMRX7maPn+pOn7X4X/1QAo+f/HIyKRvZgcCz+Pb8b7c26pVzHMVtnMk8CRwjXNuZxXrvoxvZ8sB/gGMT2F8rZ2/nXsAMMrMsmv5+mTHV378TsafsVQl2cev2k1UMa+q45Ls4xfvdhbhmy1OAR4CXkrQdmPR3TnXAd9scbOZnVHVrqqYl8rj1wjoCzxbzSopOX5mdhY+6f+sfFYVqyXs8xevjEv6ZtYQ/weZ5Jx7IZj9WZCAyhPR2mB+Gf7iTLlWwOpgvYOAvwP/7ZybV9W+nHOfO+e+C54+CnRMVXzOufJ/VwKz8Gcdu9v1ejNrADTDt68nPb7Aj4EXnXPbqtpXCo5fdWqKe4/1knT8arudSpxzXzrnNgePXwEamllWKuKr8PlbC7wIdK5itViPc8LjC5wPLHLOfVbNe0j68TOzHOAx4EfOuc+D2Un7/CVCRiV9MzPgceB959wfKyyaAgwKHg/Ct8WVz7/avK7AJufcp8EZwovABOdcdWcJ5X/gcn3xV/VTEV9zM2scbDML6A68V8UuK273MuANFzQQJjO+Cq/rz15+Wqfg+FXnHeA4Mzsm+FtfEWxjd8k+frXdzu7rHRGsW94MWQ/4fC/rJyq+A8ysafljfLv5sipWrenzkZT4Kqjp85fU42dmrYEXgKuccx9UWD8pn7+ESfZFg0ROwGn4nz/F/KcbVh98O9gMfJeqGcAhwfoGPIxvr18KdArmDwS2Ubnb16nBsvuAvsHjB/Bdt5bgu3Idn6L48oLnS4J/r6uwj4rxNcH/tC0BFgDHpiK+YFkb4BOg3m77SOXxOwJ/tvQlsDF4fFCwrA++90UpcHdIx6/K+KrbTvCaocDQ4PHwCsdvHpCXoviODfa5JNh/xeNXMb5qPx8p+Pvuj0/gzXbbRyqP32PAFxXWLaqwrYR//hI1qQyDiEiEZFTzjoiIxEdJX0QkQpT0RUQiRElfRCRClPRFRCJESV9EJEKU9EVEIuT/ARhTWR/xWM95AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "x=seoul_born_death_mrg['Year']\n",
    "y2=seoul_born_death_mrg['tot_pop']\n",
    "line_fitter = LinearRegression()\n",
    "line_fitter.fit(x.values.reshape(-1,1), y2)\n",
    "plt.plot(x, y2, 'x')\n",
    "plt.plot(x,line_fitter.predict(x.values.reshape(-1,1)))\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAEDCAYAAADZUdTgAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3Xt8FNX5x/HPwx0EuQUVCUEMiFpFwQgYrSBai9pKlWqBYtXaH7WVarX2Vy9tVarirdYLVqSWWqR468XS/lRU5KYBJIhEEMEEFCIoIBJRVAg8vz9mApuwSTawm93Nft+v177YOXN25jmJPjs5c+Ycc3dERCQzNEp2ACIiUn+U9EVEMoiSvohIBlHSFxHJIEr6IiIZRElfRCSDpGzSN7NJZrbBzJbGUPcPZvZm+FppZlvqI0YRkXRjqTpO38xOBT4DJrv7MXX43M+APu7+w4QFJyKSplL2St/d5wCbI8vMLNfMXjCzRWY218yOjPLREcAT9RKkiEiaaZLsAOpoInC5u79rZv2BPwKDK3aaWTegO/BKkuITEUlpaZP0zaw1kA88Y2YVxc2rVBsO/N3dd9ZnbCIi6SJtkj5BV9QWdz++hjrDgSvqKR4RkbSTsn36Vbn7p8BqM7sAwALHVew3s15Ae2BekkIUEUl5KZv0zewJggTey8xKzewy4PvAZWa2BFgGDI34yAjgSU/V4UgiIikgZYdsiohI/KXslb6IiMRfyt3IzcrK8sMOOyzZYYiIpJVFixZtcvdOtdVLuaR/2GGHUVhYmOwwRETSipm9H0s9de+IiGQQJX0RkQyipC8ikkGU9EVEMoiSvohIBlHSF5G0N2F2CQUlmyqVFZRsYsLskiRFlLqU9EUk7fXObsuYqYt3J/6Ckk2MmbqY3tltkxxZ6km5cfoiInWVn5vF+JF9GDN1MaP65zBlwRrGj+xDfm5WskNLObVe6de2Vm042+UDZlZsZkVm1jcs7xaucPWmmS0zs8vjHbyISIX83CxG9c/hgVeKGdU/Rwm/GrF07zwGDKlh/1lAz/A1Gng4LF8P5Ifz3/cHrjOzQ/c9VBGR6hWUbGLKgjVcObgHUxas2auPXwK1Jv1oa9VWMZRg8XJ39/lAOzPr7O7b3f2rsE7zWM4lIrIvKvrwx4/swzVn9trd1aPEv7d4JOIuwNqI7dKwDDPramZF4f473X1dtAOY2WgzKzSzwo0bN8YhJBHJJEWlZZX68Cv6+ItKy5IcWeqJx41ci1LmAO6+Fugddus8a2Z/d/eP9qrsPpFg0XPy8vI0wb+I1MnlA3P3KsvPzVK/fhTxuNIvBbpGbGcDla7owyv8ZcDX43A+ERHZR/FI+tOAH4SjeAYAZe6+3syyzawlgJm1B04GVsThfCIiso9q7d4J16odBGSZWSlwE9AUwN0nAM8BZwPFwDbg0vCjRwG/NzMn6AK6x93fincDREQahLIP4NN10PXEhJ6m1qTv7iNq2e/AFVHKXwJ673toIiIZYNtmePUP8PpEaNcNrlgAFu1WaXzoiVwRkWTYvg0WPAyv3g9ffQrHDYfTbkhowgclfRGR+vXVVhiXvWf7iCFw+m/h4K/Vy+mV9EVE6sPOcvhdx8plg26AQb+q1zCU9EVEEm3iIFi3eM9264PhFysS3pUTjZK+iKS0CbNL6J3dttKDVgUlmygqLYv6UFZK+c9VsOixymU3fgRNWyQlHNB8OCKS4tJyrvz5E+DmtpUT/rXFcHNZUhM+6EpfRFJcWs2Vv3I6TL2wctlP58NBRyUnniiU9EUk5UXOlX/l4B6pl/A/fAsmnFK5bNQ/ocfpyYmnBkr6IpLyqs6VPyC3Y2ok/q0fwu97VS4751448bLkxBMDJX0RSWmRc+Xn52YxILdjpe2k2L4Nbu9cuaz/5XDWncmJpw6U9EUkpdU0V369J/1du+COHNi+dU9ZTj788Pn6jWM/WDB1TurIy8vzwsLCZIchIlLZX8+F1bP3bDdtBdd/AI1SYxCkmS1y97za6ulKX0SkJtNvhHnjK5fdsA6aHZCcePaTkr6ISDSLHgserop0zXI48NCkhBMvqfF3iYhklAmzS/ZatLygZBMTZpckKaIIq2YFD1ZFJvwfzwkerEpAwq/vn4WSvojUu5R8ynbjyiDZTx66p2z4E0Gy73xcwk5b3z+LWm/kmtkk4FvABnc/Jsp+A+4nWD1rG3CJu79hZscDDwMHAjuB29z9qdoC0o1ckcxQkdzq4ynbGufvyWsHdx9e+QNn3gr5P0tILNHE42cR643cWK70HwOG1LD/LKBn+BpNkOgh+AL4gbt/Lfz8fWbWLobziUgGiHzKdlT/nLgl/GjdJY0bwWWPFVa6mr76b69z+cy+lRN+n1HBlX09JnxI3M8imlqTvrvPATbXUGUoMNkD84F2ZtbZ3Ve6+7vhMdYBG4BO8QhaRNJf1adsqybqfRWtu+ThWau45syejJm6mHunv8Phj/djwa6IlWAP6Q03bYGhD8UlhppE+1L609wS/jR3ddx/FtHEY/ROF2BtxHZpWLa+osDM+gHNgKh3JsxsNMFfCeTk5MQhJBFJZYl8yramCdoGLb6GnvNmVf7Abz6GxvU3kLHiS6kipj/NLeH2/3uHG845kv/5em7CnziOx43caKsA7L5RYGadgceBS919V7QDuPtEd89z97xOnfTHgEhDV9NTtvGwV3fJmj/BzW3puXnW7joLvvdm0JVTjwm/IraKL6V7X1zBvS++uzvhR+6P18+iqni0thToGrGdDawDMLMDgf8Dfh12/YiIRF38JD83Ky5XthNml9C4EUxZsIY/HlvM2QUjK1e4qoiCzQeEV9NtkjJ/T9VZQysSfuT+RMUVj6Q/DRhjZk8C/YEyd19vZs2AfxH09z8Th/OIiNSqcSNY+cIE3mg6Ed7dUz6hxwQuHxX04+e3J2nz90R+KVX04bdp2YSdu6J/GcZbrUnfzJ4ABgFZZlYK3AQ0BXD3CcBzBMM1iwlG7FwafvRC4FSgo5ldEpZd4u5vxjF+EZE91i7kf2acEWaowHX2c3IH/wCqdC4n6mq6tuUdGzeiUh9+m5ZNdm/Xh1qTvruPqGW/A1dEKZ8CTNn30EQkk+zXWrgfl8CDfSsVbWzVgxM3j43afZJIVW/URt60Bti5C24450genrWKrV+UM2XBGm4450h2Rr3jGX+ae0dEUkJtyTKqaPPaAwUXlTBm6mKuHJxT74uu1La8Y8UX2NYvyqvt008kJX0RSQl1WgvXHW6J8qznTVsoWPVx0hddqW15x2SuBKakLyIpI6a1cG+OMifNdWuhxYFAaiy6UlNST/ZKYFpERURSRo1z0ERL9j+dDwcdVb9B1qJqUq+6vV/3LmoQ69w7SvoikhKqS5bTjppB9tKHK1ce9mc49rvJCbQWiUrqtVHSF5FaJStBxRTLsmfhmYsrVzrxR3DO7+s1rnSh5RJFpFb7NGImQXZ/yXy4FCacXCXQ4XD+I/UeU0OkpC+Sweo0YibRvvoMxnWpXNbmUPjF8vqPpQFT0hfJcDGNmEmkXTvh1oNgV3nl8psTM+FYptNyiSIZLlHz2sdk7r0wtsOehN+yA/z2EyX8BNKVvkgGS9qY8fVL4OWboeSVPWU3fghNWybunAIo6YtktHp/kOnjEph5Gyz9B7RsD2feFozIadoi/ueSqDRkU0QSb+tHMOcuWPQYNG4GA34KJ18JLaI8cCX7REM2RST5viyD1x6A+X+Enduh78Uw8H+hzSHJjixjKemLSPzt+BIWPgpzfw9fbIZjhsFpN0LH+n3gS/ZW6+gdM5tkZhvMbGk1+83MHjCzYjMrMrO+EfteMLMtZvbfeAYtIvtnwuySvUbpFJRsYsLskv078K6dsPhv8OAJ8OKNcOjxMHo2fHeSEn6KiGXI5mPAkBr2nwX0DF+jgchJMu4GLtrX4EQkMSqexK1I/BWjeHpn72Mfuzu88xw8fDL8+6fQ+iD4wTS46F9B4peUEcvKWXPM7LAaqgwlWAfXgflm1s7MOrv7enefYWaD4hOqiMRLXJ/Efb8gGH65dgF07AEXToajzgWzuMct+y8effpdgLUR26Vh2fo4HFtEEmS/n8T9aBm8fAu8Ox3adIZv3w/Hj4LGulWYyuLx24n2dV6ncaBmNpqga4icnJw4hCQitdnn1Zs+eR9m3g5FTwULl5xxM/T7MTRrleiQJQ7ikfRLga4R29nAurocwN0nAhMhGKcfh5hEpAb79CTu55tgzj3BqJxGjYNx9qdcHTxkJWkjHkl/GjDGzJ4E+gNl7q6uHZEUVqcncb/aCvMegoIHYccX0GcUDPwVtO0S5ciS6mpN+mb2BDAIyDKzUuAmoCmAu08AngPOBoqBbcClEZ+dCxwJtA4/e5m7T49zG0SkjqItkJKfm1U54Zdvh0V/gdl3wbZNwc3Zwb+BTkfUY6QSb7GM3hlRy34Hrqhm39f3MS4RSZZdu2Dp3+GVW2HL+3DY1+GMWyD7hGRHJnGg2+wiEnCHd1+CGbfAR0vhkN4w6p+QO1jDLxsQJX0RgbUL4eWb4P3XoH33YOHxr50PjbTkRkOjpC+SyTaugBlj4Z3/wgEHwdn3BJOiNWmW7MgkQZT0RTJRWSnMGgdvToWmB8Bpv4YBP4HmrZMdmSSYkr5IJtm2GV69FxZMBDyY1/6Ua+CAjsmOTOqJkr5IJtj+Ocx/OJjb/qtP4bgRcNr10E5PwGcaJX2RhmznDnhjMsy+Ez77CHqdHYy1P/joZEcmSaKkL9IQ7doFbz8bjLXfXAI5JwWzX+YMSHZkkmRK+iINTckrweyX69+Eg46GEU/BEd/UWHsBlPRFGo4P3gjmtV89G9rmwHmPwLEXBJOjiYSU9EXS3aZieOV3QXdOq44w5A7I+yE0aZ7syCQFKemLpKtP1wc3aN+YDE1aBDNfnjQmmOM+igmzS+id3bbSpGoFJZsoKi2LOgGbNEx6xlok3XyxJejGeaAPLJ4CJ14GV70Jp91QbcKHBKyLK2lJV/oi6WLHF/D6RJh7L3y5JeivP+1G6NA9po/HdV1cSVtK+iKpbmc5LJkKs+6ATz+AHt+A038LnXvX+VD7vS6upD0lfZFU5Q7L/xPcpN20ErrkwfkT4bBT9vmQ+7wurjQYtfbpm9kkM9tgZkur2W9m9oCZFZtZkZn1jdh3sZm9G74ujmfgIg3a6rnw6Bnw9EXB9vemwI9e3u+EX7EO7jVn9trd1VPRxy+ZIZYbuY8BQ2rYfxbQM3yNBh4GMLMOBEsr9gf6ATeZmVZQFqnJ+iKYMgz++i3Yuh7OHQ8/mQdHfXu/H66qaV1cyRyxLJc4x8wOq6HKUGByuGzifDNrZ2adCdbVfcndNwOY2UsEXx5P7G/QIg3O5tUw8zZ46xlo0Q6+8Tvo9z/QtGXcThHTurjS4MWjT78LsDZiuzQsq658L2Y2muCvBHJyNOufZJDPNsCcu6HwL9CoSTDN8clXQct2yY5MGqh4JP1of3N6DeV7F7pPBCYC5OXlRa0j0qB8+SkUPAjzHoLyL+GEi4OHq9ockuzIpIGLR9IvBbpGbGcD68LyQVXKZ8XhfCLpq/wrWPhnmHsPbPsYvnZeMNVxRz0RK/UjHkl/GjDGzJ4kuGlb5u7rzWw6cHvEzdszgevjcD6R9LNrJxQ9DTNvh7I1cPggOP0m6NK3tk+KxFWtSd/MniC4Ys8ys1KCETlNAdx9AvAccDZQDGwDLg33bTaz3wELw0ONrbipK5Ix3GHldJhxC2x4GzofD+c+ALmnJTsyyVCxjN4ZUct+B66oZt8kYNK+hSaS5tbMD+bIWTMPOuTCBY/BUUOhkaa8kuTRE7ki8bZhebCIycrnofUh8K0/QJ+LoHHTZEcmoqQvEjdb1sDMcbDkCWh+YDA/Tv+fQLNWyY5MZDclfZH99fnHwWichY8CBvljgvH2rTokOzKRvSjpi+yrrz6D+X+E1x6AHZ/D8d+HQddD26jPIIqkBCV9kboq3w5v/BVm3wWfb4AjvxV05XTqlezIRGqlpC8Sq127YNk/g6mOP3kPup0Cw6dC1xOTHZlIzJT0RWrjDiUzghE5HxbBwcfC9/8OPc7Y75kvReqbkr5ITUoLg7H2782Fdt3g/EfhmGEaay9pS0lfJJqNK+GVscHKVQd0grPuhhMugSbNolafMLuE3tltK01TXFCyiaLSsqhTGoskiy5XRCKVfQDTfgZ/7A8lM2HQDXDlm9B/dLUJH6B3dttKq1BVrFLVO7ttfUUuEhNd6YsAbNsMr/4BXp8Ivgv6/RhOvRYOiG2BkYpVqMZMXcyo/jlMWbCm0ipVIqlCSV8y2/ZtsGACvHZfMMf9ccODsfbtu9X5UPm5WYzqn8MDrxRz5eAeSviSkpT0JTPt3AGLp8DsO4O1aI8YEoy1P/hr+3zIgpJNTFmwhisH92DKgjUMyO2oxC8pR0lfMos7vP3vYKz9x8XQtT989y/Q7aT9OmxFH35Fl86A3I6VtkVShZK+ZI5Vs4Lhl+sWQ6ejYMSTwRV+HMbaF5WWVUrwFX38RaVlSvqSUiyYDj915OXleWFhYbLDkIZk3ZtBsl81E9p2hdNugN7fg0aNkx2ZSNyY2SJ3z6utXkxDNs1siJmtMLNiM7suyv5uZjbDzIrMbJaZZUfsu9PMloav79WtGSL74eMSeOZSmDgQ1i+Bb94OYwrh+JFK+JKxYlkusTHwEPANgsXOF5rZNHd/O6LaPcBkd/+rmQ0GxgEXmdk5QF/geKA5MNvMnnf3T+PdEJHdtn4Y3KB9YzI0bgan/hLyfwYtNGZeJJY+/X5AsbuvAggXQB8KRCb9o4Grw/czgWcjyme7ezlQbmZLgCHA03GIXaSyL8vgtfth/sOwczuccGmQ8NscnOzIRFJGLN07XYC1EdulYVmkJcCw8P15QBsz6xiWn2VmrcwsCzgN6Fr1BGY22swKzaxw48aNdW2DZLodX0LBg3D/cTD399DrbBizEM65RwlfpIpYrvSjDW2oevf3WmC8mV0CzAE+AMrd/UUzOxEoADYC84DyvQ7mPhGYCMGN3Jijl8y2szxYmnDWOPj0A8g9Hc64CTofl+zIRFJWLEm/lMpX59nAusgK7r4OOB/AzFoDw9y9LNx3G3BbuG8q8O7+hy0ZzR3e+T+YMRY2rYAuJ8B5E6D7qcmOTCTlxZL0FwI9zaw7wRX8cGBkZIWw62azu+8CrgcmheWNgXbu/rGZ9QZ6Ay/GMX7JNO+9Fgy/LH0dOvaECyfDUedqXnuRGNWa9N293MzGANOBxsAkd19mZmOBQnefBgwCxpmZE3TvXBF+vCkw14L/IT8FRoU3dUXq5sOlMOMWePdFaHMofPuBYE3axnq+UKQu9HCWpLZP3oOZt0PR09DiQDjlGuj/Y2jaMtmRiaSUWB/O0mWSpKbPNsKcu6FwUvAg1clXwSk/h5btkx2ZSFpT0pd6VesKU19thYLxMG887PgC+oyCQdfBgYcmMWqRhkMrZ0m9qm6FqeM6t4D5E+D+42H2HdDjdLhiAZz7gBK+SBzpSl/qVdUVpqbOX81TJ62l53PXwJY1wbDLM24OhmGKSNwp6Uu9y8/NYlS/rrw16xmeb/cvOr1WDIf0hlH3Qe5gDb8USSAlfal3b82bzqB5N3FNs+Ws3XYIK065j16DL4ZG6m0USTT9Xyb1Z8NyNj86jGOnX8gxLTbC2fdQOnImI+ZlU7B6c7KjE8kIutKXxNuyNpgfZ8kTHNCoFe8ffw3dzvoFNG/NScD4kc20wpRIPVHSl8TZtjmY9fL1PwEOA35K81OuodsBHStVy8/NUsIXqSdK+hJ/2z+H+X+E1x6A7Z/BcSNg0PXQbq9ZtUWkninpS/zs3AGLHoPZd8HnG6DXOXD6b+Cgo5IdmYiElPRl/+3aBcv+Ca/cCp+shpx8+N4UyOmf7MhEpAolfdl37lDySjDV8YdFcNDXYOTT0PNMjbUXSVFK+rJvShfByzfBe3OhbQ6c9wgce0EwOZqIpCwlfambTe8GK1YtnwatOsKQOyDvh9CkebIjE5EYxJT0zWwIcD/BIiqPuvsdVfZ3I1gtqxOwmWCxlNJw313AOQQPgr0EXOWpNom/1O7TdTDrDlg8JZjLfuB1kD8GmrdJdmQiUge1Jv1wycOHgG8QrJe70MymufvbEdXuASa7+1/NbDAwDrjIzPKBkwmWSQR4FRgIzIpfEyShvvgEXr0PFkyAXTvhxB/Bqb+E1p2SHZmI7INYrvT7AcXuvgrAzJ4EhgKRSf9o4Orw/Uzg2fC9Ay2AZoARLJ/40f6HLQm34wtY8Ai8ei98+Sn0vjAYa9+he7IjE5H9EEvS7wKsjdguBaqOxVsCDCPoAjoPaGNmHd19npnNBNYTJP3x7r686gnMbDQwGiAnJ6fOjZA42lkOb04JunK2rg9G4pz+Wzjk2GRHJiJxEMuEa9HG3lXtk78WGGhmiwm6bz4Ays2sB3AUkE3w5THYzE7d62DuE909z93zOnVSt0FSuMPb/4Y/DoD/XAVts+GS5+D7zyjhizQgsVzplwKRz89nA+siK7j7OuB8ADNrDQxz97LwCn6+u38W7nseGADMiUPsEi+rZgdj7de9AVm94Ht/gyPP0Vh7kQYoliv9hUBPM+tuZs2A4cC0yApmlmVmFce6nmAkD8Aagr8AmphZU4K/Avbq3pEkWb8EHj8PJp8Ln30E546HnxTAUd9SwhdpoGq90nf3cjMbA0wnGLI5yd2XmdlYoNDdpwGDgHFm5gRX8VeEH/87MBh4i6BL6AV3/0/8myF1snlVMGXC0n9Ay/Zw5q3BqJymLZMdmYgkmKXakPm8vDwvLCxMdhgN09aPYM5dwaRojZrCST+F/CuhZbtkRyYi+8nMFrl7Xm319ERuAzNhdgm9s9tWmp9+wfLVNJk/nhPWTYXyr+CEi2Hgr6DNIUmMVESSQUm/gemd3ZYxUxczfmQf8nNas/qF+zli0YO0Zyt87XwY/GvomJvsMEUkSZT0G5j83CzGD+/N9Cn3ckzTf9B9+4ds6XwKfPtWOLRPssMTkSRT0m9I3GHF8+TPGEu+L2fJl4ez8pjxXHDhRcmOTERSRCxDNiUdvD8PJn0TnhzBF199yS8b/YIZJz/BuBWHUFCyKdnRiUiK0JV+uvtoWTDV8coXoPUhlPS/lRELe3Df908kPzeLAT2y9vTxa/FxkYynpJ+uPnkfZt4ORU9B8wPh9Jug/+W8NG89931/z+id/Nwsxo/sQ1FpmZK+iGicftr5fBPMuQcK/wzWCPqNhlOuhlYdkh2ZiCSRxuk3NF99BvMegoIHYcfn0GdUsJBJ2y7JjkxE0oiSfhJEe4CqoGQTRaVlXD6wyhj68u3BE7Rz7oLPN8JR34bBv4VOR9Rv0CLSIGj0ThJUPEBVMaqmoGQTY6Yupnd22z2Vdu2CoqdhfB48/0vodCT8aAZ8b4oSvojsM13pJ0HFzdUxUxczqn8OUxas2TO6xh2KX4aXb4GP3grmsv/+P6DH6Zr5UkT2m5J+kuTnZjGqfw4PvFLMlYN7BAl/7cJgXvv3X4X2h8GwPwdTJzTSH2QiEh9K+klSULKJKQvWcOXgHrw2v4CL1/6ajmtfhAM6wdn3QN+LoUmzZIcpIg2Mkn4SVPThTxx6MHmrH+Rq/xufr23OmuOuJufsa6F562SHKCINlJJ+EqxYvYb/9JpOl2mPg+/C+v+Et7tewhubGnO5Er6IJFBMSd/MhgD3E6yc9ai731FlfzeCJRI7AZuBUe5eamanAX+IqHokMNzdn41H8Gln+zZY8DCXLrwfvvoUjhsBp10P7XLoB/RLdnwi0uDVmvTNrDHwEPANgkXSF5rZNHd/O6LaPcBkd/+rmQ0GxgEXuftM4PjwOB2AYuDFOLch9e3cAYsfh1l3wmcfwhFnwem/hYOPTnZkIpJhYrnS7wcUu/sqADN7EhgKRCb9o4Grw/czgWhX8t8Fnnf3bfsebppxh7efhRm/g80l0HUAXPAYdDsp2ZGJSIaKZSxgF2BtxHZpWBZpCTAsfH8e0MbMOlapMxx4ItoJzGy0mRWaWeHGjRtjCCkNlMyEiYPgmUugSXMY8RT88AUlfBFJqliSfrQngqrO0nYtMNDMFgMDgQ+A8t0HMOsMHAtMj3YCd5/o7nnuntepU6eYAk9Z6xbD5KHw+Hdg22b4zgS4/FXoNUQPV4lI0sXSvVMKdI3YzgbWRVZw93XA+QBm1hoY5u5lEVUuBP7l7jv2L9wU9nEJvPI7WPYvaNkBvjkOTrwsuMoXEUkRsST9hUBPM+tOcAU/HBgZWcHMsoDN7r4LuJ5gJE+kEWF5w7P1Q5h1B7wxGZq0gFP/F/J/Bi0OTHZkIiJ7qTXpu3u5mY0h6JppDExy92VmNhYodPdpwCBgnJk5MAe4ouLzZnYYwV8Ks+MefTJ9sQVeux/mPwy7yoOr+lN/Ca0PSnZkIiLVyqhFVOo0pXF1dnwJr0+Eub+HL7fAsRfAaTdCh+4JiVlEJBaxLqKSUTN5xTSlcXV2lsMbj8ODfeGl30B2Hvx4Lgx7VAlfRNJGRk3DUOOUxtVxh3f+G4y137QCuuTBeY9A96/XX+AiInGSUUkfqpnSuDrvvRpMdVy6ELKOCBYwOfJbGnopImkr45J+5JTGUxasYUBux70T/4dvBYuYFL8EbQ6Fcx+E40ZC44z7cYlIA5NRWayiD7+iS2dAbsdK22xeDTNvh7eegRZt4Rtjod9oaNoy2aGLiMRFRiX9otKySn34FX38765aRf6KO6HwL9CoCZzyczj559CyXZIjFhGJr4xK+nsNy/zyU/LXTCT/9fFQ/iX0/QEM/BUc2Dk5AYqIJFhGJf3dyr+Cwkkw527Y9jEc/R0Y/BvI6pHsyEREEiqzkv6unUF//czbYMsa6D4QzrgJupyQ7MhEROpFZiR9d1g5HWaMhQ3LoPNx8O37IXdwsiMTEalXDT/pr1kAL98Ea+ZBh8Phu38JunMaZdTDyCIiQENO+huWB1f2K56D1gfDOfcGN2obN012ZCIiSdPwkv6WNTBzHCx5Apq3CW7QDvgJNDsg2ZGJiCRdw0n6X3wCs++GhX8CDE66Ar7+C2jVIdmRiYikjIaT9HeWw+IpcOyFMOguBwEpAAAJIUlEQVQ6aNe19s+IiGSYmO5mmtkQM1thZsVmdl2U/d3MbIaZFZnZLDPLjtiXY2YvmtlyM3s7XFQl/lp3gp8XwXceUsIXEalGrUnfzBoDDwFnAUcDI8zs6CrV7gEmu3tvYCwwLmLfZOBudz8K6AdsiEfgUWnaBBGRGsVypd8PKHb3Ve6+HXgSGFqlztHAjPD9zIr94ZdDE3d/CcDdP3P3bXGJXERE6iyWpN8FWBuxXRqWRVoCDAvfnwe0MbOOwBHAFjP7p5ktNrO7w78cREQkCWJJ+tFWDKm6sO61wEAzWwwMBD4AygluFH893H8icDhwyV4nMBttZoVmVrhx48bYoxcRkTqJJemXApF3RrOBdZEV3H2du5/v7n2AG8OysvCzi8OuoXLgWaBv1RO4+0R3z3P3vE6dOu1jU0REpDaxJP2FQE8z625mzYDhwLTICmaWZWYVx7oemBTx2fZmVpHJBwNv73/YIiKyL2pN+uEV+hhgOrAceNrdl5nZWDM7N6w2CFhhZiuBg4Hbws/uJOjamWFmbxF0Ff0p7q0QEZGYmHvV7vnkysvL88LCwmSHISKSVsxskbvn1VZPU02KiGQQJX0RkQyipC8ikkGU9EVEMoiSvohIBlHSFxHJIEr6IiIZRElfRCSDKOmLiGQQJX0RkQyipC8ikkGU9EVEMoiSvohIBlHSFxHJIA0i6U+YXUJByaZKZQUlm5gwuyRJEYmIpKYGkfR7Z7dlzNTFuxN/QckmxkxdTO/stkmOTEQktcSU9M1siJmtMLNiM7suyv5uZjbDzIrMbJaZZUfs22lmb4avaVU/Gw/5uVmMH9mHMVMXc++LKxgzdTHjR/YhPzcrEacTEUlbtSZ9M2sMPAScBRwNjDCzo6tUuweY7O69gbHAuIh9X7j78eHrXBIkPzeLUf1zeOCVYkb1z1HCFxGJIpYr/X5AsbuvcvftwJPA0Cp1jgZmhO9nRtmfcAUlm5iyYA1XDu7BlAVr9urjFxGR2JJ+F2BtxHZpWBZpCTAsfH8e0MbMOobbLcys0Mzmm9l3op3AzEaHdQo3btxYh/ADFX3440f24Zoze+3u6lHiFxGpLJakb1HKqq6mfi0w0MwWAwOBD4DycF9OuFjvSOA+M8vd62DuE909z93zOnXqFHv0oaLSskp9+BV9/EWlZXU+lohIQ9YkhjqlQNeI7WxgXWQFd18HnA9gZq2BYe5eFrEPd19lZrOAPkBcx1JePnCv7xHyc7PUry8iUkUsV/oLgZ5m1t3MmgHDgUqjcMwsy8wqjnU9MCksb29mzSvqACcDb8creBERqZtak767lwNjgOnAcuBpd19mZmPNrGI0ziBghZmtBA4GbgvLjwIKzWwJwQ3eO9xdSV9EJEnMvWr3fHLl5eV5YWFhssMQEUkrZrYovH9aowbxRK6IiMRGSV9EJIOkXPeOmW0E3k92HDXIAhriAwBqV/poiG0CtWt/dXP3Wse8p1zST3VmVhhLv1m6UbvSR0NsE6hd9UXdOyIiGURJX0Qkgyjp193EZAeQIGpX+miIbQK1q16oT19EJIPoSl9EJIMo6YuIZJCMTfpm9p6ZvRUu41gYlnUws5fM7N3w3/ZhuZnZA+FykUVm1jfiOBeH9d81s4sjyk8Ij18cfjbaFNWJaFc7M/u7mb1jZsvN7KR0b5eZ9YpYcvNNM/vUzH7eANp1tZktM7OlZvaEmbUIJzZcEMb3VDjJIWbWPNwuDvcfFnGc68PyFWb2zYjyGpc5TWC7rgrbtMzMfh6WpeXvyswmmdkGM1saUZbwtlR3jrhw94x8Ae8BWVXK7gKuC99fB9wZvj8beJ5gbYEBwIKwvAOwKvy3ffi+fbjvdeCk8DPPA2fVU7v+CvwofN8MaNcQ2hXRvsbAh0C3dG4XwUJEq4GW4fbTwCXhv8PDsgnAT8L3PwUmhO+HA0+F748mWMSoOdCdYNryxuGrBDg8/O9gCXB0Pfx+jgGWAq0Ipm5/GeiZrr8r4FSgL7A0oizhbanuHHFpU6L/I0jVF9GT/gqgc/i+M7AifP8IMKJqPWAE8EhE+SNhWWfgnYjySvUS2KYDw0RiDaldVdpyJvBaureLPSvSdSBIjv8Fvknw5GaTsM5JwPTw/XTgpPB9k7CeEUxlfn3EcaeHn9v92bC8Ur0EtusC4NGI7d8A/5vmv6vDqJz0E96W6s4Rj1fGdu8QrP71opktMrPRYdnB7r4eIPz3oLC8uiUjayovjVKeaIcDG4G/mNliM3vUzA4g/dsVaTjwRPg+bdvl7h8A9wBrgPVAGbAI2OLBdOZV49gde7i/DOhI3duaaEuBU82so5m1Irj67Uoa/66iqI+2VHeO/ZbJSf9kd+8LnAVcYWan1lC3uiUj61qeaE0I/hR92N37AJ8T/GlYnXRpFwBh//a5wDO1VY1SllLtCvtohxJ0yRwKHEDw32J1caR8mwDcfTlwJ/AS8AJBt1J5DR9Ji3bFKC3akrFJ3/cs47gB+BfQD/jIzDoDhP9uCKtXt2RkTeXZUcoTrRQodfcF4fbfCb4E0r1dFc4C3nD3j8LtdG7XGcBqd9/o7juAfwL5QDszq1jGNDKO3bGH+9sCm6l7WxPO3f/s7n3d/dQwxndJ799VVfXRlurOsd8yMumb2QFm1qbiPUE/8VKCZSAr7qxfDPw7fD8N+EF4d34AUBb+yTUdONOCZSHbh8eZHu7bamYDwrvxP4g4VsK4+4fAWjPrFRadTrA8ZVq3K8II9nTtVMSfru1aAwwws1bhOSt+VzOB71bTpoq2fhd4xYMO32nAcAtG93QnuGn6OjEsc5ooZnZQ+G8OwdrZT5Dev6uq6qMt1Z1j/yX6xk4qvgj6vpeEr2XAjWF5R2AGwZXJDKBDWG7AQwSjId4C8iKO9UOgOHxdGlGeR/BFUgKMp8rN1QS27XigECgCniUYLdAQ2tUK+BhoG1GW1u0CbgHeCc/7OMEInMMJknYxQTdW87Bui3C7ONx/eMRxbgzjXkHESBaC/vSV4b4b6/H/r7kEX2BLgNPT+XdF8IW1HthBcGV+WX20pbpzxOOlaRhERDJIRnbviIhkKiV9EZEMoqQvIpJBlPRFRDKIkr6ISAZR0hcRySBK+iIiGeT/AfZlJvuaJs1cAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "x3=seoul_born_death_mrg['# of born']\n",
    "y3=seoul_born_death_mrg['tot_pop']\n",
    "line_fitter.fit(x3.values.reshape(-1,1), y3)\n",
    "plt.plot(x3.values.reshape(-1,1), y3, 'x')\n",
    "plt.plot(x3,line_fitter.predict(x3.values.reshape(-1,1)))\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Linear Regression _Ex 2_\n",
    "<a href=\"http://www.data4sci.com/\">www.data4sci.com</a>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "ename": "OSError",
     "evalue": "'./class_04_d4sci.style' not found in the style library and input is not a valid URL or path; see `style.available` for list of available styles",
     "output_type": "error",
     "traceback": [
      "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[1;31mFileNotFoundError\u001b[0m                         Traceback (most recent call last)",
      "\u001b[1;32m~\\Anaconda\\lib\\site-packages\\matplotlib\\style\\core.py\u001b[0m in \u001b[0;36muse\u001b[1;34m(style)\u001b[0m\n\u001b[0;32m    110\u001b[0m             \u001b[1;32mtry\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 111\u001b[1;33m                 \u001b[0mrc\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mrc_params_from_file\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mstyle\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0muse_default_template\u001b[0m\u001b[1;33m=\u001b[0m\u001b[1;32mFalse\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m    112\u001b[0m                 \u001b[0m_apply_style\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mrc\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32m~\\Anaconda\\lib\\site-packages\\matplotlib\\__init__.py\u001b[0m in \u001b[0;36mrc_params_from_file\u001b[1;34m(fname, fail_on_error, use_default_template)\u001b[0m\n\u001b[0;32m   1069\u001b[0m     \"\"\"\n\u001b[1;32m-> 1070\u001b[1;33m     \u001b[0mconfig_from_file\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0m_rc_params_in_file\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mfname\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mfail_on_error\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m   1071\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32m~\\Anaconda\\lib\\site-packages\\matplotlib\\__init__.py\u001b[0m in \u001b[0;36m_rc_params_in_file\u001b[1;34m(fname, fail_on_error)\u001b[0m\n\u001b[0;32m    985\u001b[0m     \u001b[0mrc_temp\u001b[0m \u001b[1;33m=\u001b[0m \u001b[1;33m{\u001b[0m\u001b[1;33m}\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 986\u001b[1;33m     \u001b[1;32mwith\u001b[0m \u001b[0m_open_file_or_url\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mfname\u001b[0m\u001b[1;33m)\u001b[0m \u001b[1;32mas\u001b[0m \u001b[0mfd\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m    987\u001b[0m         \u001b[1;32mtry\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32m~\\Anaconda\\lib\\contextlib.py\u001b[0m in \u001b[0;36m__enter__\u001b[1;34m(self)\u001b[0m\n\u001b[0;32m    111\u001b[0m         \u001b[1;32mtry\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 112\u001b[1;33m             \u001b[1;32mreturn\u001b[0m \u001b[0mnext\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mgen\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m    113\u001b[0m         \u001b[1;32mexcept\u001b[0m \u001b[0mStopIteration\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32m~\\Anaconda\\lib\\site-packages\\matplotlib\\__init__.py\u001b[0m in \u001b[0;36m_open_file_or_url\u001b[1;34m(fname)\u001b[0m\n\u001b[0;32m    970\u001b[0m             \u001b[0mencoding\u001b[0m \u001b[1;33m=\u001b[0m \u001b[1;34m\"utf-8\"\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 971\u001b[1;33m         \u001b[1;32mwith\u001b[0m \u001b[0mopen\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mfname\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mencoding\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mencoding\u001b[0m\u001b[1;33m)\u001b[0m \u001b[1;32mas\u001b[0m \u001b[0mf\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m    972\u001b[0m             \u001b[1;32myield\u001b[0m \u001b[0mf\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;31mFileNotFoundError\u001b[0m: [Errno 2] No such file or directory: './class_04_d4sci.style'",
      "\nDuring handling of the above exception, another exception occurred:\n",
      "\u001b[1;31mOSError\u001b[0m                                   Traceback (most recent call last)",
      "\u001b[1;32m<ipython-input-14-c3bf6019326d>\u001b[0m in \u001b[0;36m<module>\u001b[1;34m\u001b[0m\n\u001b[0;32m      5\u001b[0m \u001b[0mget_ipython\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mrun_line_magic\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;34m'matplotlib'\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;34m'inline'\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m      6\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m----> 7\u001b[1;33m \u001b[0mplt\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mstyle\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0muse\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;34m'./class_04_d4sci.style'\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[1;32m~\\Anaconda\\lib\\site-packages\\matplotlib\\style\\core.py\u001b[0m in \u001b[0;36muse\u001b[1;34m(style)\u001b[0m\n\u001b[0;32m    115\u001b[0m                     \u001b[1;34m\"{!r} not found in the style library and input is not a \"\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    116\u001b[0m                     \u001b[1;34m\"valid URL or path; see `style.available` for list of \"\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 117\u001b[1;33m                     \"available styles\".format(style))\n\u001b[0m\u001b[0;32m    118\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    119\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;31mOSError\u001b[0m: './class_04_d4sci.style' not found in the style library and input is not a valid URL or path; see `style.available` for list of available styles"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "import matplotlib\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "%matplotlib inline\n",
    "\n",
    "plt.style.use('./class_04_d4sci.style')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Load dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[10.    8.04]\n",
      " [ 8.    6.95]\n",
      " [13.    7.58]\n",
      " [ 9.    8.81]\n",
      " [11.    8.33]\n",
      " [14.    9.96]\n",
      " [ 6.    7.24]\n",
      " [ 4.    4.26]\n",
      " [12.   10.84]\n",
      " [ 7.    4.82]\n",
      " [ 5.    5.68]]\n"
     ]
    }
   ],
   "source": [
    "data = np.array(np.loadtxt(\"./data/Anscombe1.dat\"))\n",
    "\n",
    "print(data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "X = data[:, 0].reshape(-1, 1)\n",
    "y = data[:, 1].reshape(-1, 1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[10.],\n",
       "       [ 8.],\n",
       "       [13.],\n",
       "       [ 9.],\n",
       "       [11.],\n",
       "       [14.],\n",
       "       [ 6.],\n",
       "       [ 4.],\n",
       "       [12.],\n",
       "       [ 7.],\n",
       "       [ 5.]])"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 8.04],\n",
       "       [ 6.95],\n",
       "       [ 7.58],\n",
       "       [ 8.81],\n",
       "       [ 8.33],\n",
       "       [ 9.96],\n",
       "       [ 7.24],\n",
       "       [ 4.26],\n",
       "       [10.84],\n",
       "       [ 4.82],\n",
       "       [ 5.68]])"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0x1d590d73860>"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEKCAYAAAAfGVI8AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAHZ1JREFUeJzt3XtwXGeZ5/Hv0xfJshxLvsEwlmTHLghx2ZYttx0DtRFswgzsugJDgSrE2bIN+FLFwpCqYRaK1NT+kZ3Nkil2h5raeJyAnIUkizYLHnZdmyWVrXFqiwRbNlIi7LDBg9EFkhjHKFhWubtPP/uHW8TxRWq1uvt09/l9qly69enzHFvuX59z3vd5zd0REZHoioVdgIiIhEtBICIScQoCEZGIUxCIiEScgkBEJOIUBCIiEacgEBGJOAWBiEjEKQhERCIuEXYBhVi6dKmvXLky7DJERGrK8ePHf+vuy2Z6XE0EwcqVK+nv7w+7DBGRmmJmvyrkcWULAjP7NrANeN3d1+a/9yng3wK3AlvcXa/uIjItd+fE8HkGRsaZuJSluTHBhvYWujoWYWZhl1cXynlGcBD4O+C/XPG9IeATwN+Xcb8iUgcyQY6+YyPsP3KacxNpskGOTOAk40YiHmNJcwP7ulfTs7mdZFy3O+eibEHg7s+Z2cqrvncKUIqLyLQmLmXZ2XuUobE3mcwEb/tZOnDSQcDF9CQPHD7JoYExDu7aQnNjTVzprkpV+zdnZnuAPQAdHR0hVyMilZIJcuzsPcrg6DjpbG7axyZj8JEVcfoHh1i2oCGybzLnzZtHW1sbyWSyqO2rNgjc/QBwACCVSmnRBJGI6Ds2wtDYmzOGAMAXblvEhlXvoqF5Ie9sbWLJgsYbPrZe7zW4O+fOnWN0dJSbb765qOeo2iAQkehxd/YfOX3N5aAbWdGaJDH/Jhw4e+ESi5uvPSuo93sNZsaSJUs4e/Zs0c+hIBCRqnFi+DznJtIFP96wP7zwZwPnYjp4272CqNxrmOsZTdniz8yeBJ4HbjGzUTP7rJn9mZmNAu8DDpvZ/y7X/kWk9gyMjJMNZr4kdD0OTKbferG/8l7DTGcYk5kcg6Pj7Ow9SqbI/deysgWBu3/a3d/l7kl3b3P3b7n7D/KfN7r7O939T8u1fxGpPROXsmSC4m4JujvBFWuwz+ZeA0A6m2NobJy+/pGi9l+oz33uc5w8eXLaxxw6dGjGx5RS7V0QE5G61dyYIBkv7jKHmRHPXyKZ7b2GKZOZHPuPnMa9fONTHn30UdasWTPtYxQEIhJZG9pbSBR5w9aApoY4MPt7DVc6dyHNieHzBT/+zJkzvPe972XHjh2sX7+eT37yk1y8eJFnn32WjRs3sm7dOj7zmc9w6dIlAD74wQ/+oWXOggUL+NrXvkZnZydbt27ltdde48c//jE//OEP+fKXv8yGDRs4ffo03/zmN1mzZg3r16/n7rvvLuq4pqMgEJGq0dWxiCXNDUVtm4gb8/NBMJd7DdnAGRwZn9U2P//5z9mzZw8vvvgiCxcu5Bvf+AY7d+7ke9/7Hi+99BLZbJaHH374mu0mJibYunUrg4OD3H777TzyyCO8//3v56677uKhhx5iYGCA1atX8+CDD/LTn/6UF198kf379xd1XNNREIhI1TAz9nWvpikZn9V2MTOWLWj8w+iZudxryAQ5Ji5lZ7VNe3s7H/jABwC49957efbZZ7n55pt5z3veA8COHTt47rnnrtmuoaGBbdu2AbBp0ybOnDlz3edfv34927dv57vf/S6JROlHNSkIRKSq9GxuZ+3yhTQkCnt5ipnRlIyz6Iozibnca0jGY7MeQlrs8M1kMvmHbePxONns9QPo8OHDfP7zn+f48eNs2rTpho8rloJARKpKMh7j4K4tdLa10JSc/iUqZtCUjLNyaTOxK16M53KvIRE3OttbZrXN8PAwzz//PABPPvkkd955J2fOnOEXv/gFAN/5znfo7u4u+Pluuukmfv/73wOQy+UYGRnhQx/6EF//+tf53e9+x4ULF2ZV30wUBCJSdZobEzyxeyv3b1tD++Im5jfEaYjHMKAhHmN+Q5yOxfNpaUpy87Jm4rG3vyOfy72GpQsa6epYNKttbr31Vh577DHWr1/PG2+8wX333Udvby+f+tSnWLduHbFYjH379hX8fHfffTcPPfQQGzdu5JVXXuHee+9l3bp1bNy4kfvuu4/W1tbZHta0rJzDpEollUq5FqYRiaapHkGDV/QI6mxvpaujlZdffplbb731uts9/sKveODwqVkNIW1Kxrh/2xq237ai4G3OnDnDtm3bGBoaKnibcjh16tQ1fxdmdtzdUzNtW3tzqUUkUsyMTSsWs2nF4llt17O5nUMDYwV1MQVoTMRYt7yVnlR7saXWLF0aEpG6NJt7DU3JGJ1trfTu2jzrxnMrV64M/WxgrnRGICI1zd1vOGpn6l5DX3++++iFNNnAyQQ5kvEYibixdEEje7tX0ZOqze6jwJxnQisIRKRmzZs3j3PnzrFkyZIbhkEyHmP7bSu4Z0vHDe811MN6BPPmzSv6ORQEIlKz2traGB0dLbgX/3zgfUumvsrCxVd5+eVXy1VexUytUFYsBYGI1KxkMln0qlzyltq8ICYiIiWjIBARiTgFgYhIxCkIREQiTjeLRUSqzFRbjYErhrpuaG+hq2NRWYa6KghERKpEJsjRdyw/+W0iTTbIkQmcZNxIxGMsaW5gX/dqejaXdvKbgkBEpApMXMqys/coQ2NvXtMoLx046SDgYnqSBw6f5NDAGAd3bZn1ugk3UrZ7BGb2bTN73cyGrvjeYjN7xsxeyX+cXa9XEZE6lAly7Ow9yuDo+IzdUiczOQZHx9nZe5RMkctxXq2cN4sPAh+56ntfAZ5193cDz+a/FhGJtL5jIwyNvVlQl1SAdDbH0Ng4ff0jJdl/2YLA3Z8D3rjq2x8DHst//hjw8XLtX0SkFrg7+4+cntW6CXD5zGD/kdNzbjgHlR8++k53/w1A/uM7bvRAM9tjZv1m1l9oHxERkVpzYvg85ybSRW177kKaE8Pn51xD1c4jcPcD7p5y99SyZcvCLkdEpCwGRsbJFnmtPxs4gyPjc66h0kHwmpm9CyD/8fUK719EpKpMXMqSCYq7vJMJckxcys65hkoHwQ+BHfnPdwD/UOH9i4hUlebGBMl4cZPEkvFYSYaQlnP46JPA88AtZjZqZp8FHgQ+bGavAB/Ofy0iElkb2ltIFDk5LBE3Ottb5lxD2SaUufunb/CjO8q1TxGRWtPVsYglzQ1cTE/OetulCxrp6pj7dKyqvVksIhIFZsa+7tU0JeOz2q4pGWNv96qS9B5SEIiIhKxncztrly+kIVHYS3JjIsa65a30pNpLsn8FgYhIyJLxGAd3baGzrYWm5PQvy03JGJ1trfTu2lyyxnNqOiciUgWaGxM8sXsrff357qMX0mQDJxPkSMZjJOLG0gWN7O1eRU9K3UdFROpSMh5j+20ruGdLByeGzzN4xXoEne2tdHW0aj0CEZEoMDM2rVjMphWLK7I/3SMQEYk4BYGISMQpCEREIk5BICIScQoCEZGIUxCIiEScgkBEJOIUBCIiEacgEBGJOAWBiEjEKQhERCJOQSAiEnEKAhGRiFMQiIhEnIJARCTiQgkCM/tzMxsys5+Z2ZfCqEFERC6r+MI0ZrYW2A1sAdLA02Z22N1fqXQtIsVwd04Mn2fgitWjNrS30NWxqCyrR4mUWxgrlN0KvODuFwHM7AjwZ8DXQ6hFpGCZIEffsfx6shNpskGOTOAk40YiHmNJcwP7ulfTs7m068mKlFsYQTAE/DszWwJMAv8C6A+hDpGCTVzKsrP3KENjbzKZCd72s3TgpIOAi+lJHjh8kkMDYxzctYXmRq0EK7Wh4m9b3P0U8B+AZ4CngUEge/XjzGyPmfWbWf/Zs2crXKXIWzJBjp29RxkcHb8mBK42mckxODrOzt6jZIJchSoUmZtQzl/d/Vvu3uXutwNvANfcH3D3A+6ecvfUsmXLKl+kSF7fsRGGxt4knS3shT2dzTE0Nk5f/0iZKxMpjbBGDb0j/7ED+ATwZBh1iMzE3dl/5PSMZwJXm8zk2H/kNO5epspESiesO1r/3cxOAv8D+Ly7nw+pDpFpnRg+z7mJdFHbnruQ5sSwfrWl+oVyN8vd/1kY+xWZrYGRcbJFXuvPBs7gyDibViwucVUipaUxbiLTmLiUJRMUd3knE+SYuHTNOAiRqqMgEJlGc2OCZLy4SWLJeExDSKUmKAhEprGhvYVEkZPDEnGjs72lxBWJlJ6CQGQaXR2LWNLcUNS2Sxc00tWxqMQViZSegkBkGmbGvu7VNCXjs9quKRljb/cq9R6SmqAgEJlBz+Z21i5fSEOisP8ujYkY65a30pNqL3NlIqWhIBCZQTIe4+CuLXS2tdCUnP6/TFMyRmdbK727NqvxnNQMDWkQKUBzY4Indm+lrz/fffRCmmzgZIIcyXiMRNxYuqCRvd2r6EnVX/dRtd6ubwoCkQIl4zG237aCe7Z0cGL4PINXvCh2trfS1dFady+Kar0dDVYLvVBSqZT396tTtUglTdd6+0pNyRhrl7eo9XYVMrPj7p6a6XGKcBG5hlpvR4uCQESuodbb0aIgEJG3Uevt6FEQiMjbqPV29CgIRORtStF6W2qLbvGL1JBKjOdX6+3oURCI1IBKjuefar2dLiIM1Hq7NulfTKTKTTeePx046SDgYnqSBw6f5NDA2JzH80+13k4Hs7tZDGq9Xat0j0CkioUxnl+tt6NHQSBSxcIYz6/W29GjIBCpUmGO51fr7WgJJQjM7D4z+5mZDZnZk2Y2L4w6RKpZmOP51Xo7Wir+r2Zmy4EvAil3XwvEgbsrXYdItQt7PP9U6+37t62hfXET8xviNMRjGNAQjzG/IU7H4vncv20Nj+++TaOFalhY/3IJoMnMMsB84Nch1SFStaphPH8UW29HUcWDwN3HzOxvgGFgEviRu/+o0nWIVLtqGs9vZmxasZhNKxaX7DmleoRxaWgR8DHgZuCPgWYzu/c6j9tjZv1m1n/27NlKlykSuqnx/MXQeH6ZjTDu7NwJ/NLdz7p7Bvg+8P6rH+TuB9w95e6pZcuWVbxIkbBpPL9UShhBMAxsNbP5dvni4h3AqRDqEKlqGs8vlVLxIHD3nwBPASeAl/I1HKh0HSK1QOP5pRK0ZrFIlXur19A4k5kbDydtSl4Ogd5dmzWUswQq0em13Apds1i/LSJVbmo8f19/vvvohTTZwMkEOZLxGIm4sXRBI3u7V9GTmnv30airZKfXaqEzApEaMvUuVeP5y2O6Tq9XakrGWLu8Zc6dXsut0DMCBYGICJfPBO555AUGR8cLavLXkIjR2dbCE7u3Vu2ZQaFBUJ3Vi4hUWBidXquFgkBEIi/MTq/VQEEgIpEXZqfXajBjEJjZv863hRARqUthd3oNWyFnBH8EHDOzPjP7iGlogojUmWro9BqmGYPA3e8H3g18C9gJvGJmf21mq8tcm4hIRUx1ei1GqTu9hqGg6t3dzexV4FUgCywCnjKzZ9z9L8tZoBSmHmZBioRlqtNrOpjdzWKoj06vMwaBmX0R2AH8FngU+LK7Z8wsBrwCKAhCFMVZkCKlNtXp9WJ6ctbb1kOn10JeGZYCn3D3P3X3/5ZvHY2754BtZa1OpjVxKcs9j7zAA4dPMXJ+kovpgHTgOJAOnIvpgJHzkzxw+CT3PPJCzV/HFCmXqHd6LeQewV+5+69u8DO1jw5JJsixs/cog6PjM459nszkGBwdZ2fvUTJFjowQqXdR7vSqawU1KsqzIEXKIRmPcXDXFjrbWmhKTv/S2JSM0dl2udNrPVxyrf0jiKCoz4IUKZepTq/3b1tD++Im5jfEaYjHMKAhHmN+Q5yOxfO5f9saHt99W82PFppSH0cRMaWYBalFyEWuLxmPsf22FdyzpSMynV4VBDWoFLMgFQQi0zMzNq1YHIn/K7o0VIOiPgtSREpLZwQ1aGoWZLqIMKiHWZBTNIlOpDTq4xUhYqI+C1KT6ERKS0FQg6I8C3K6pQTTgZMOAi6mL0+iOzQwVvVLCYpUg4q/XTKzW8xs4Io/b5rZlypdRy2L6ixITaITKY+KB4G7/9zdN7j7BmATcBH4QaXrqHVRnAWpSXQi5RH2BdQ7gNM3amEhNxa1WZCaRCdSPmG/KtwNPBlyDTUrSrMgo76UoEg5hfbKYGYNwF3AV2/w8z3AHoCOjo4KVlZbojILUpPoRMonzLeIHwVOuPtr1/uhux8ADgCkUimd18+g3mdBahKdSPmEeWno0+iykBQo6ksJipRTKEFgZvOBDwPfD2P/UnumJtEVox4m0YmUUyhB4O4X3X2Ju4+HsX+pPVOT6IpR65PoRMot7FFDIgWJ6iQ6kUpQEEjNiOIkOpFKUBBIzYjaJDqRStFQCqkpU5Po+vrz3UcvpMkGTibIkYzHSMSNpQsa2du9ip6Uuo+KFEJBIDUnKpPoRCpFQSA1q94n0YlUis6bRUQiTkEgIhJxCgIRkYhTEIiIRJyCQEQk4hQEIiIRpyAQEYk4BYGISMQpCEREIk5BICIScQoCEZGIUxCIiEScgkBEJOIUBCIiEacgEBGJuFCCwMxazewpM3vZzE6Z2fvCqENERMJbmOZvgafd/ZNm1gDMD6kOEZHIq3gQmNlC4HZgJ4C7p4F0pesQEZHLwrg0tAo4C/Sa2U/N7FEzaw6hDhERIZwgSABdwMPuvhGYAL5y9YPMbI+Z9ZtZ/9mzZytdo4hIZIRxj2AUGHX3n+S/forrBIG7HwAOAKRSKa9ceXPj7pwYPs/AyDgTl7I0NybY0N5CV8cizCzs8kRErlHxIHD3V81sxMxucfefA3cAJytdR6llghx9x0bYf+Q05ybSZIMcmcBJxo1EPMaS5gb2da+mZ3M7ybhG7YpI9Qhr1NAXgMfzI4b+CdgVUh0lMXEpy87eowyNvclkJnjbz9KBkw4CLqYneeDwSQ4NjHFw1xaaG8P6qxcRebtQ3pq6+4C7p9x9vbt/3N3Ph1FHKWSCHDt7jzI4On5NCFxtMpNjcHScnb1HyQS5ClUoIjI9XaOYo75jIwyNvUk6W9gLezqbY2hsnL7+kTJXJiJSGAXBHLg7+4+cnvFM4GqTmRz7j5zGvWbugYtIHVMQzMGJ4fOcmyhuLty5C2lODNfsFTERqSMKgjkYGBknW+S1/mzgDI6Ml7giEZHZUxDMwcSlLJmguMs7mSDHxKVsiSsSEZk9BcEcNDcmSMaLmySWjMc0hFREqoKCYA42tLeQKHJyWCJudLa3lLgiEZHZUxDMQVfHIpY0NxS17dIFjXR1LCpxRSIis6cgmAMzY1/3apqS8Vlt15SMsbd7lXoPiUhVUBDMUc/mdtYuX0hDorC/ysZEjHXLW+lJtZe5MhGRwigI5igZj3Fw1xY621poSk7/19mUjNHZ1krvrs1qPCciVUPDVkqguTHBE7u30tef7z56IU02cDJBjmQ8RiJuLF3QyN7uVfSk1H1URKqLgqBEkvEY229bwT1bOjgxfJ7BK9Yj6GxvpaujVfcERKQqKQhKzMzYtGIxm1YsDrsUEZGC6BqFiEjEKQhERCJOQSAiEnEKAhGRiFMQiIhEnIJARCTiFAQiIhEXyjwCMzsD/B4IgKy7p8KoQ0REwp1Q9iF3/22I+5cScHdODJ9n4IqZ1BvaW+jqWKSZ1CI1QjOLpSiZIEffsXxvpYk02SBHJnCScSMRj7GkuYF93avp2azeSiLVLqwgcOBHZubA37v7gZDqkCJMXMqys/coQ2NvMpkJ3vazdOCkg4CL6UkeOHySQwNjHNy1RctyilSxsN6qfcDdu4CPAp83s9uvfoCZ7TGzfjPrP3v2bOUrlOvKBDl29h5lcHT8mhC42mQmx+DoODt7j5IJchWqUERmK5QgcPdf5z++DvwA2HKdxxxw95S7p5YtW1bpEuUG+o6NMDT2JulsYS/s6WyOobFx+vpHylyZiBSr4kFgZs1mdtPU58CfAEOVrkNmz93Zf+T0jGcCV5vM5Nh/5DTuXqbKRGQuwjgjeCfwf81sEDgKHHb3p0OoQ2bpxPB5zk2ki9r23IU0J4bPl7giESmFit/Bc/d/AjorvV+Zu4GRcbJFXuvPBs7gyLjWaRCpQhrXJwWbuJQlExR3eScT5Ji4lC1xRSJSCgoCKVhzY4JkvLhJYsl4TENIRaqUgkAKtqG9hUSRk8MScaOzvaXEFYlIKSgIpGBdHYtY0txQ1LZLFzTS1bGoxBWJSCkoCKRgZsa+7tU0JeOz2q4pGWNv9yr1HhKpUgoCmZWeze2sXb6QhkRhvzqNiRjrlrfSk2ovc2UiUiwFgcxKMh7j4K4tdLa10JSc/tenKRmjs62V3l2b1XhOpIppGIfMWnNjgid2b6WvP9999EKabOBkghzJeIxE3Fi6oJG93avoSan7qEi1UxBIUZLxGNtvW8E9Wzo4MXyewSvWI+hsb6Wro1X3BERqRF0GgRZLqRwzY9OKxZoxLFLD6ioItFiKiMjs1U0QaLEUEZHi1MXbYi2WIiJSvLoIAi2WIiJSvJoPAi2WIiIyNzUfBFosRURkbmo+CEqxWIqISJTVfBBosRQRkbmp+SDQYikiInNT80GgxVJEROam5oNAi6WIiMxNaEFgZnEz+6mZ/c85Po8WSxERmYMwzwj+HDhViifSYikiIsULJQjMrA34l8CjpXg+LZYiIlK8sIbM/CfgL4GbSvWEWixFRKQ4FQ8CM9sGvO7ux83sg9M8bg+wB6Cjo6Og59ZiKSIis2eV7rVjZv8e+FdAFpgHLAS+7+733mibVCrl/f39FapQRKQ+mNlxd0/N+Lgwm67lzwj+wt23zfC4s8CvitzNUuC3RW5bq3TM0aBjjoa5HPMKd18204NqYlptIQdyI2bWX0gi1hMdczTomKOhEsccahC4+z8C/xhmDSIiUaehMyIiEReFIDgQdgEh0DFHg445Gsp+zKHeLBYRkfBF4YxARESmUddBUKrGdrXEzFrN7Ckze9nMTpnZ+8KuqZzM7D4z+5mZDZnZk2Y2L+yaysHMvm1mr5vZ0BXfW2xmz5jZK/mPddNK9wbH+1D+9/pFM/uBmbWGWWOpXe+Yr/jZX5iZm9nScuy7roOAEja2qyF/Czzt7u8FOqnj4zez5cAXgZS7rwXiwN3hVlU2B4GPXPW9rwDPuvu7gWfzX9eLg1x7vM8Aa919PfD/gK9WuqgyO8i1x4yZtQMfBobLteO6DYJSN7arBWa2ELgd+BaAu6fd/XfhVlV2CaDJzBLAfODXIddTFu7+HPDGVd/+GPBY/vPHgI9XtKgyut7xuvuP3H1qbdkXgLaKF1ZGN/g3BviPXO7NVrYbunUbBLzV2K64le1r0yrgLNCbvyT2qJk1h11Uubj7GPA3XH6n9Btg3N1/FG5VFfVOd/8NQP7jO0Kup5I+A/yvsIsoNzO7Cxhz98Fy7qcug+DKxnZh11JhCaALeNjdNwIT1NflgrfJXxP/GHAz8MdAs5ndsGeV1Acz+xqXe5U9HnYt5WRm84GvAX9V7n3VZRAAHwDuMrMzwH8F/rmZfTfckipiFBh195/kv36Ky8FQr+4EfunuZ909A3wfeH/INVXSa2b2LoD8x9dDrqfszGwHsA3Y7vU/9n01l9/kDOZfy9qAE2b2R6XeUV0Ggbt/1d3b3H0ll28e/p/pupvWC3d/FRgxs1vy37oDOBliSeU2DGw1s/l2ub/4HdTxzfHr+CGwI//5DuAfQqyl7MzsI8C/Ae5y94th11Nu7v6Su7/D3VfmX8tGga78//OSqssgiLgvAI+b2YvABuCvQ66nbPJnPk8BJ4CXuPz7XJczT83sSeB54BYzGzWzzwIPAh82s1e4PKrkwTBrLKUbHO/fcXkxq2fMbMDM9odaZInd4Jgrs+/6P7sSEZHp6IxARCTiFAQiIhGnIBARiTgFgYhIxCkIREQiTkEgMktm1m5mvzSzxfmvF+W/XhF2bSLFUBCIzJK7jwAP89a4/QeBA+7+q/CqEime5hGIFMHMksBx4NvAbmCju6fDrUqkOImwCxCpRe6eMbMvA08Df6IQkFqmS0Mixfsol9tfrw27EJG5UBCIFMHMNnC5v89W4L6pLqAitUhBIDJL+U6nDwNfcvdh4CEuL5AjUpMUBCKztxsYdvdn8l//Z+C9ZtYdYk0iRdOoIRGRiNMZgYhIxCkIREQiTkEgIhJxCgIRkYhTEIiIRJyCQEQk4hQEIiIRpyAQEYm4/w+oAqO38CZ/jQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(X, y, '.', markersize= 30, label='points')\n",
    "plt.xlabel('X')\n",
    "plt.ylabel('y')\n",
    "plt.legend()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Get matrix dimensions and add the bias column"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(11, 1)"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "# np.ones((M, 1)),X"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "M, N = X.shape\n",
    "X = np.concatenate((np.ones((M, 1)), X), axis=1) #Add x0 for the bias/intercept"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[ 1. 10.]\n",
      " [ 1.  8.]\n",
      " [ 1. 13.]\n",
      " [ 1.  9.]\n",
      " [ 1. 11.]\n",
      " [ 1. 14.]\n",
      " [ 1.  6.]\n",
      " [ 1.  4.]\n",
      " [ 1. 12.]\n",
      " [ 1.  7.]\n",
      " [ 1.  5.]]\n"
     ]
    }
   ],
   "source": [
    "print(X)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Set the training parameters and initialize the weight matrix"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "alpha = 0.01\n",
    "epsilon = 0.12\n",
    "\n",
    "weights = 2*np.random.rand(N+1, 1)*epsilon - epsilon\n",
    "count = 0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Training Procedure"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "100 0.9967653786551831 0.0008092079653841955 [0.40377468 0.75717688]\n",
      "200 0.924128457519429 0.0006508561694966719 [0.67162423 0.73065455]\n",
      "300 0.8657056660038159 0.0005234918234771069 [0.91184103 0.7068684 ]\n",
      "400 0.8187154781612214 0.00042105107409418085 [1.1272758  0.68553615]\n",
      "500 0.7809206776013702 0.0003386566877363606 [1.3204852  0.66640465]\n",
      "600 0.7505218429735144 0.00027238584391586773 [1.49376211 0.64924685]\n",
      "700 0.7260716769971559 0.00021908336865217315 [1.64916286 0.63385915]\n",
      "800 0.7064061002212734 0.00017621151572955185 [1.78853166 0.62005892]\n",
      "900 0.6905888288660569 0.00014172914387255986 [1.91352245 0.60768239]\n",
      "1000 0.677866798082226 0.00011399453741578469 [2.02561853 0.59658269]\n",
      "1100 0.6676343083164525 9.168724374875037e-05 [2.12615019 0.58662809]\n",
      "1200 0.6594041877809986 7.37452061907451e-05 [2.21631049 0.57770047]\n",
      "1300 0.6527845980644617 5.9314199159166137e-05 [2.29716937 0.56969386]\n",
      "1400 0.6474603785920349 4.770715824953253e-05 [2.36968643 0.56251325]\n",
      "1500 0.6431780417396833 3.8371468897513417e-05 [2.43472225 0.55607344]\n",
      "1600 0.6397337042152857 3.086265624341333e-05 [2.49304864 0.55029799]\n",
      "1700 0.6369633801163171 2.4823223549685913e-05 [2.54535777 0.54511836]\n",
      "1800 0.6347351735137514 1.9965631685825436e-05 [2.59227042 0.5404731 ]\n",
      "1900 0.6329429988488575 1.6058609302760374e-05 [2.63434331 0.53630706]\n",
      "2000 0.6315015301691626 1.2916141937657777e-05 [2.67207575 0.53257081]\n",
      "2100 0.6303421387352564 1.0388615814371782e-05 [2.70591551 0.52922001]\n",
      "2200 0.6294096255867228 8.35569468482511e-06 [2.73626418 0.5262149 ]\n",
      "2300 0.6286595935037202 6.720590587705466e-06 [2.76348192 0.52351982]\n",
      "2400 0.6280563332425454 5.405455747120591e-06 [2.78789174 0.52110277]\n",
      "2500 0.6275711234082464 4.347676212379525e-06 [2.80978332 0.51893507]\n",
      "2600 0.6271808630207284 3.496890795151586e-06 [2.82941644 0.51699101]\n",
      "2700 0.6268669716704302 2.8125933576594875e-06 [2.84702411 0.51524751]\n",
      "2800 0.6266145048996715 2.262204299996462e-06 [2.86281529 0.51368387]\n",
      "2900 0.6264114426927108 1.8195194415593363e-06 [2.87697736 0.51228155]\n",
      "3000 0.6262481171993323 1.4634624287968023e-06 [2.88967839 0.5110239 ]\n",
      "3100 0.6261167524457111 1.1770812831102262e-06 [2.90106913 0.509896  ]\n",
      "3175 0.6260353923067838 9.997124750249498e-07 [2.90883414 0.50912711]\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "Text(0, 0.5, 'Error')"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEKCAYAAAAIO8L1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAF5xJREFUeJzt3XlwHOWZx/HfM6PTliUfkg8sg23sgJ1NuByWEJIlgRCTlMMmJBy5WEKF2myoTf7IQSq1u0lqa2uT1Ka2qFABs5CwW1lIAmQDFJtAQcKxHMYGG8zpA4OFL9myZSzLljR69o9uyWO5Wy3bGvWM+vspT0332z0zz7TG+qn7fbvH3F0AAAyVS7sAAEB5IiAAAJEICABAJAICABCJgAAARCIgAACRCAgAQCQCAgAQiYAAAESqSruA49Hc3Oxz585NuwwAqCirVq3a6e4tSetVdEDMnTtXK1euTLsMAKgoZvbmSNbjEBMAIBIBAQCIREAAACIREACASAQEACASAQEAiERAAAAiZTIgnly/Uz998LW0ywCAspbJgHj6jQ7d8Mj6tMsAgLKWyYAAACQjIAAAkTIZEJZ2AQBQATIZEACAZAQEACBSpgPC3dMuAQDKViYDwuiEAIBEmQwIAECyigwIM1tmZss7OzuP63k4wgQA8SoyINz9Pne/tqmp6Zgebwx0BYBEFRkQAIDSy3RAcIQJAOJlMiAYxQQAyTIZEACAZAQEACBSpgOCM6kBIF4mA4IuCABIlsmAAAAky3RAcIAJAOJlMiAY5goAyTIZEACAZAQEACBSpgOCUa4AEC+TAWF0QgBAokwGBAAgWaYDwhnoCgCxMh0QAIB4BAQAIFKmA4JRTAAQL5MBwSAmAEiWyYAAACQjIAAAkQgIAECkTAaE8ZVBAJAokwEBAEiW6YBgmCsAxMtkQDDMFQCSZTIgAADJMh0QXKwPAOJlMiA4wgQAyTIZEACAZAQEACBS2QSEmc03s1vN7K6xek2GuQJAvJIGhJndZmY7zGztkPalZvaama03s+slyd03uvs1pazn0OuPxasAQGUr9R7ELyUtLW4ws7ykGyVdLGmxpCvNbHGJ6wAAHKWSBoS7PyapY0jz2ZLWh3sMPZLulHRJKeuIwxEmAIiXRh/EbEmbi+bbJM02s2lmdpOkM8zsu3EPNrNrzWylma1sb28/pgK4WB8AJKtK4TWjfju7u++S9LdJD3b35ZKWS9KSJUvYCQCAEkljD6JN0pyi+VZJW1KoAwAwjDQC4llJC81snpnVSLpC0r0p1CFnnCsAxCr1MNc7JD0l6RQzazOza9y9T9J1kv4o6RVJv3H3l0pZx5F1jeWrAUBlKmkfhLtfGdP+gKQHSvnaAIDjUzZnUh8NM1tmZss7OzuP63k4wAQA8SoyINz9Pne/tqmpKe1SAGDcqsiAAACUXqYDgkFMABAvkwFhDGMCgESZDAgAQLKKDIjRGsUEAIhXkQExaqOY6IMAgFgVGRDHix4IAEiWyYAAACTLdEA4x5gAIFYmA4JRrgCQLJMBAQBIlumA4ExqAIhXkQFxvOdBcIQJAJJVZEBwNVcAKL2KDAgAQOllOiDoggCAeJkMCK7mCgDJMhkQAIBkmQ4IZ5wrAMTKZEBwhAkAklVkQPB9EABQehUZEJwHAQClV5EBMVrogQCAeJkMCLogACBZJgMCAJAs0wHBKFcAiJfNgGCcKwAkymZAAAASZTog+E5qAIiXyYDgABMAJKvIgOBMagAovYoMCM6kBoDSq8iAGDV0QQBArEwGBKNcASBZYkCYWd7MfjIWxQAAykdiQLh7QdJZNg6/p5MjTAAQr2qE6z0v6fdm9ltJXQON7n5PSaoqMWOgKwAkGmlATJW0S9JHitpcUkUGBAAg2YgCwt2vLnUhaeBifQAQb0SjmMys1cx+Z2Y7zGy7md1tZq2lLq5Uxl9vCgCMvpEOc/2FpHslnSBptqT7wjYAwDg10oBocfdfuHtfePulpJYS1gUASNlIA2KnmX0hPCcib2ZfUNBpnYrRuhYTV3MFgHgjDYgvS7pM0jZJWyV9JmxLxfFei4kuCABIljiKyczyki5190+OQT0AgDIx0jOpLxmDWsYcw1wBIN5IT5T7PzP7maRf6/AzqZ8rSVUlxjBXAEg20oA4N7z/YVGb6/AzqwEA48hI+iBykn7u7r8Zg3rGFEeYACDeSPog+iVdNwa1jBku1gcAyUY6zPUhM/ummc0xs6kDt5JWBgBI1Uj7IAbOefhaUZtLmj+65QAAysVIr+Y6r9SFpMEZ5woAsYY9xGRm3y6a/uyQZf9SqqJKji4IAEiU1AdxRdH0d4csWzrKtQAAykhSQFjMdNR8xeEIEwDESwoIj5mOmq8YFZ9sADAGkjqpTzOzvQp+p9aH0wrn60paGQAgVcMGhLvnx6qQo2FmyyQtW7BgQdqlAMC4NdIT5crK8X4fBAAgWUUGxPEyLucKAIkyGRAAgGSZDgiGuQJAvEwGBAeYACBZJgMCAJAs0wHhlXuuHwCUXCYDgkFMAJAskwEBAEhGQAAAImU6IBjmCgDxMhkQ9EEAQLJMBgQAIFmmA4IjTAAQL5MBYZxLDQCJMhkQAIBkmQ4IZxgTAMTKZEAwigkAkmUyIAAAyQgIAECkTAcEPRAAEK8iA8LMlpnZ8s7OzrRLAYBxqyIDwt3vc/drm5qa0i4FAMatigyI0cIoVwCIl8mAMMa5AkCiTAYEACAZAQEAiJTxgKATAgDiZDIg6IEAgGSZDAgAQLJMBwTDXAEgXiYDglGuAJAskwEBAEiW6YDgCBMAxMtkQPCd1ACQLJMBAQBIRkAAACJlOiAY5goA8TIZEAxzBYBkmQwIAECyTAeEM9AVAGJlMiA4wgQAyTIZEACAZJkOCEYxAUC8TAYEo5gAIFkmAwIAkIyAAABEynRA0AcBAPEyGhB0QgBAkowGBAAgSaYDgjOpASBeJgOCYa4AkCyTAQEASJbpgHjurT1plwAAZSuTAXHmiVN0QlOd/uF/1urym5/SUxt2pV0SAJSdsgkIM5toZreb2S1m9vlSvlbLpFo98s3z9f1li/XGzi5decvTuuzmp/Tkhp1yTo4AAEklDggzu83MdpjZ2iHtS83sNTNbb2bXh82flnSXu39F0idLWZck1VXn9TcfmKfHvv1h/eCT79abu7r0uVue0eXLn9bj69oJCgCZV+o9iF9KWlrcYGZ5STdKuljSYklXmtliSa2SNoerFUpc16C66ryuOneuHv3WoaD44q0r9IkbntDvV7+t3kL/WJUCAGWlpAHh7o9J6hjSfLak9e6+0d17JN0p6RJJbQpCYti6zOxaM1tpZivb29tHrdaBoHjs2x/Wjy99r3oK/fr6nat1/k/+rNueeENdB/tG7bUAoBKk0QcxW4f2FKQgGGZLukfSpWb2c0n3xT3Y3Ze7+xJ3X9LS0jLqxdVW5XXZ++bowW98SP/xpSU6YXKdfnj/yzr3Xx/Rj/7wqtp27x/11wSAclSVwmtGnabm7t4l6eqxLiZOLme6cPEMXbh4hp57a7eWP7pRNz+6QTc/ukEXLJqhq94/Vx9YME3GWXcAxqk0AqJN0pyi+VZJW1KoY8TOPHGKbvriWWrbvV///cxbuvPZzXro5e2a3zJRXzrnJH36rFY11lWnXSYAjCor9WgdM5sr6X53/4twvkrS65IukPS2pGclfc7dXzra516yZImvXLly9IodoYN9BT3w4lbd/uSbWr15j+qr87r4PTN12ZI5+st5U9mrAFDWzGyVuy9JWq+kexBmdoek8yU1m1mbpH9y91vN7DpJf5SUl3TbsYRDmmqr8vrUGa361BmteqFtj+5YsVn3r9mie557WydOnaDPntWqS89q1QmT69MuFQCOWcn3IErBzJZJWrZgwYKvrFu3Lu1yJEndPQX94aWt+u3KNj25YZfMpPMWNOuS02fronfP4BAUgLIx0j2IigyIAWkdYkqyuWO/7lrVprufa1Pb7m7V5HM6/5QWLTvtBF2waLom1KTR9QMAAQKiDLi7nt+8R/ev2ar7X9iiHe8cVH11XhcunqFPvGemPvSuFsICwJgjIMpMod/17KYO3bdmix54cat27+9VTVVO5y1o1kcXz9AFi6Zr+qS6tMsEkAEERBnrK/RrxaYOPfTydj308na17e6WmXT6nMm6cNEMffiU6Vo0axKjoQCUBAFRIdxdr257ZzAsXny7U5LU3FCrDy5s1gcXNuu8hc3sXQAYNQREhdq+94AeX7dTj69r1xPrdmpXV48kadGsRn1wYbPOmT9VZ500VU31jIoCcGzGdUCU4zDXUujvd728de9gYKzctFs9hX6ZSYtmNurseVN19rypet/cqWqZVJt2uQAqxLgOiAHjcQ9iOAd6C3r+rT16dlOHVrzRoVVv7lZ3b3Bl9PktE3XmiVN02pzJOq21SafObFRNVdl8HxSAMlIWZ1JjdNVV5/X+k6fp/SdPkyT1Fvq19u1OrXijQ89u6tCfX9uhu1a1SZJqqnJaPKtRp8+ZrNPmNOk9sydrXvNE5XN0fAMYGfYgxhF319t7urVmc6fWtO3R6s179GJb5+BeRl11TqfMmKRTZzbq1FmTtGhWoxbNbFTTBPozgCzhEBMkBUNq17fv04ttnXp12zt6ddtevbL1HXWEnd+SdEJTnd41c5JObmkIbxM1v6VBzQ01DLUFxiEOMUGSVJXPBXsMMxsH29xd7e8c1Mtb9+rVbe/ola179fr2fXp64y4d6D30FauNdVU6eXqD5jc36OTpE3Xi1AmaM2WCTpw6QZMnVBMewDhHQGSQmWl6Y52mN9bp/FOmD7b397u2dHZrQ3uXNrbv04b2fdqwo0tPrG/X3c+1HfYcDbVVap1SrzmDoRFMz2qq18ymOk0hQICKV5EBUTTMNe1SxpVcztQ6ZYJap0zQX73r8K9z3XewT5s79mtzx3691bFfbbu7tbljvzbt7NLj69oP2/OQgk7ymY11mtlUd8T9jMY6tTTUalpDjSbWVuRHEMgE+iBw3NxdO/f1aPPu/drWeUBbOw9o+94D2tYZ3vYGt56+/iMeW1ed07SJtWpuqNG0hlpNmxjcB/M1mjqxVpPrqzV5QrWa6qs1qa6akVjAcaIPAmPGzNQyqXbYk/XcXXv29w6Gx859B7Wrq0e79h3Urn092tnVo+17D+jlLXu1q+ugegvRf7iYSZNqq9QUBkZTfbUm19eosf7QfENdlRpq85pYU6WG2ipNDG/BdNCeI2SARAQExoSZacrEGk2ZWKPFJzQOu667a++BviA8unrUub9Xnd292tMd3O8N7/fs71Fnd6+2de5VZ3ef9nb3qqdw5F5KlAk1+SNCo74mr7qqvOqqc6qvyau2Kq+66rzqqw+11VXlVVudC9vyhz2mtiqv6ipTdT6nmqqcavI5Vedz7PGgYhEQKDtmNrg3ML8lef0B7q7u3oL2HexT18GCug72hdN9kW1dPX3aV9TW0dWjA70FHejtV3dvIZwuxO7NjFQ+Z6rOB8FRWxWExkCIVOdzqsnb4PShdlM+l1NVzpTP2ZD7nPI5Hbk8f2h5Vc6UO+JxpqpcEFjBLdjWOTPlTMqZycL7nCUvz9nA8kNtZkFfVvH6+cH1TZbTYc936GcumSy81+AABxtYxoCHVBAQGDfMTBNqqoIvYZo0es9b6Hcd6C0UhUb/YHh0FwVKb1+/egr96i30q2dgus+D+bCtt2h5b8GPaO862KeeQvCY/n5XX7+r0O/q6+8P712FwqH2ggf3WTGQEwMhMhggOrSguC1qfQ1tiwklFQWWFB9ig8uKMmywniF1F7182G6R7YpZv/gx31l6qj66eIZKiYAAEuRzNtiPUY7c/VB4RIRKYWh7Ibjv94Fb8Bz9rmC+v2jaXT44rXC+ePnAY139/Tps/ULx8v7ixx9aXjxGxsN5H5yXXEVt4coetayoLfyngQE47oc/puipgtcsni96zqCt6DkSXnPwfQz52Rz2szpsWfJjjoj+oobGutJ/HsvzEw9gxMxMVXlTVT7tSjDeVOTlPs1smZkt7+zsTLsUABi3KjIg3P0+d7+2qakp7VIAYNyqyIAAAJQeAQEAiERAAAAiERAAgEgEBAAgEgEBAIhU0Zf7NrN2SW8e48ObJe0cxXLGGvWni/rTRf3H5yR3T7zSWUUHxPEws5UjuR56uaL+dFF/uqh/bHCICQAQiYAAAETKckAsT7uA40T96aL+dFH/GMhsHwQAYHhZ3oMAAAwjkwFhZkvN7DUzW29m16ddTxwz22RmL5rZajNbGbZNNbOHzGxdeD8lbDczuyF8Ty+Y2Zkp1Hubme0ws7VFbUddr5ldFa6/zsyuSrn+75vZ2+HPYLWZfbxo2XfD+l8zs48VtY/558vM5pjZn8zsFTN7ycy+HrZXxPYfpv5K2f51ZrbCzNaE9f8gbJ9nZs+E2/LXZlYTtteG8+vD5XOT3lcqPPyGqKzcJOUlbZA0X1KNpDWSFqddV0ytmyQ1D2n7saTrw+nrJf0onP64pP9V8A2F50h6JoV6PyTpTElrj7VeSVMlbQzvp4TTU1Ks//uSvhmx7uLws1MraV74mcqn9fmSNEvSmeH0JEmvhzVWxPYfpv5K2f4mqSGcrpb0TLhdfyPpirD9JklfDaf/TtJN4fQVkn493Psai89/1C2LexBnS1rv7hvdvUfSnZIuSbmmo3GJpNvD6dsl/XVR+3964GlJk81s1lgW5u6PSeoY0ny09X5M0kPu3uHuuyU9JGlp6auPrT/OJZLudPeD7v6GpPUKPlupfL7cfau7PxdOvyPpFUmzVSHbf5j645Tb9nd33xfOVoc3l/QRSXeF7UO3/8DP5S5JF5iZKf59pSKLATFb0uai+TYN/0FMk0t60MxWmdm1YdsMd98qBf+pJE0P28v1fR1tveX4Pq4LD8PcNnCIRmVcf3i44gwFf8VW3PYfUr9UIdvfzPJmtlrSDgXBukHSHnfvi6hlsM5weaekaSqD7V8siwFhEW3lOpTrA+5+pqSLJX3NzD40zLqV9L6k+HrL7X38XNLJkk6XtFXSv4XtZVm/mTVIulvSN9x973CrRrSVY/0Vs/3dveDup0tqVfBX/6Jhaim7+qNkMSDaJM0pmm+VtCWlWobl7lvC+x2SfqfgQ7d94NBReL8jXL1c39fR1ltW78Pdt4f/8fsl3aJDu/tlV7+ZVSv45ford78nbK6Y7R9VfyVt/wHuvkfSnxX0QUw2s6qIWgbrDJc3KTi8mXr9xbIYEM9KWhiOLqhR0EF0b8o1HcHMJprZpIFpSRdJWqug1oGRJVdJ+n04fa+kL4WjU86R1DlwaCFlR1vvHyVdZGZTwsMJF4VtqRjSj/MpBT8DKaj/inA0yjxJCyWtUEqfr/D49a2SXnH3nxYtqojtH1d/BW3/FjObHE7XS7pQQT/KnyR9Jlxt6PYf+Ll8RtIjHvRSx72vdKTVO57mTcEIjtcVHCP8Xtr1xNQ4X8FohjWSXhqoU8FxyoclrQvvp4btJunG8D29KGlJCjXfoeAwQK+Cv4SuOZZ6JX1ZQefceklXp1z/f4X1vaDgP++sovW/F9b/mqSL0/x8STpPwaGIFyStDm8fr5TtP0z9lbL93yvp+bDOtZL+MWyfr+AX/HpJv5VUG7bXhfPrw+Xzk95XGjfOpAYARMriISYAwAgQEACASAQEACASAQEAiERAAAAiERDAMMysUHQl0dWjeXVQM5trRVeOBcpNVfIqQKZ1e3D5BCBz2IMAjoEF39Xxo/A7AFaY2YKw/SQzezi8uNzDZnZi2D7DzH4Xfl/AGjM7N3yqvJndEn6HwIPhWbhAWSAggOHVDznEdHnRsr3ufrakn0n697DtZwouo/1eSb+SdEPYfoOkR939NAXfOfFS2L5Q0o3u/m5JeyRdWuL3A4wYZ1IDwzCzfe7eENG+SdJH3H1jeJG5be4+zcx2KrgcRG/YvtXdm82sXVKrux8seo65Cr57YWE4/x1J1e7+z6V/Z0Ay9iCAY+cx03HrRDlYNF0Q/YIoIwQEcOwuL7p/Kpx+UsEVRCXp85KeCKcflvRVafCLZRrHqkjgWPHXCjC8+vBbwgb8wd0HhrrWmtkzCv7QujJs+3tJt5nZtyS1S7o6bP+6pOVmdo2CPYWvKrhyLFC26IMAjkHYB7HE3XemXQtQKhxiAgBEYg8CABCJPQgAQCQCAgAQiYAAAEQiIAAAkQgIAEAkAgIAEOn/AQYufbejdfrBAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "oldJ = 0\n",
    "err = 1\n",
    "\n",
    "Js = []\n",
    "\n",
    "while err > 1e-6:\n",
    "    Hs = np.dot(X, weights)\n",
    "    deltas = alpha/M*np.dot(X.T, (Hs-y))\n",
    "\n",
    "    count += 1\n",
    "    weights -= deltas\n",
    "\n",
    "    J = np.sum(np.power(Hs-y, 2.))/(2*M)\n",
    "    Js.append(J)\n",
    "    err = np.abs(oldJ-J)\n",
    "    oldJ = J\n",
    "    \n",
    "    if count % 100 == 0:\n",
    "        print(count, J, err, weights.flatten())\n",
    "\n",
    "print(count, J, err, weights.flatten())\n",
    "\n",
    "fig, ax = plt.subplots(1)\n",
    "ax.semilogy(Js)\n",
    "ax.set_xlabel('Epoch')\n",
    "ax.set_ylabel('Error')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([2.90883414, 0.50912711])"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "weights.flatten()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0x1d590ee5390>"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEKCAYAAAAfGVI8AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3Xl8VPW9//HXdyaThQBhE7ckIFCVNRBCQK3igtYFafUqKngvoLLcWrXe677VtmitWlrtIsUlaBVbcMHey69WxYq2LuwoBXsVxSzKYsQgIWS27++PBMiQSTKZzMzJzLyfj4cPmO+cmfmMJPOec873+znGWouIiKQvl9MFiIiIsxQEIiJpTkEgIpLmFAQiImlOQSAikuYUBCIiaU5BICKS5hQEIiJpTkEgIpLmMpwuIBJ9+vSx/fv3d7oMEZGksmbNmi+ttYe1tV1SBEH//v1ZvXq102WIiCQVY8xnkWwXtyAwxjwBTAR2WGuHNY5dDNwNDAZKrbX6dBeRVllrWVu+i/UVNdTW+8nNymBkQR7FhT0xxjhdXkqI5x7BQuA3wFNNxjYCFwK/j+PrikgK8AWCLF5VwfwVW6iu9eIPBPEFLB63IcPtonduJnPGD2TymAI8bp3u7Ii4BYG19k1jTP9DxjYDSnERaVVtvZ/pZSvZWLWbOl8g5D5vwOINBNjrrWPusk0sXV/Fwhml5GYlxZHuTqnT/p8zxswCZgEUFhY2u9/n81FZWcm+ffsSXZo4IDs7m/z8fDwej9OlSJz5AkGml61kQ2UNXn+w1W3rfEE2VNYwvWwli2aO055BlDptEFhrFwALAEpKSppdNKGyspJu3brRv3//NvcwdIwxuVlrqa6uprKykmOOOcbpciTOFq+qYGPV7jZDYD+vP8jGqhoWr65g6th+ca4uNXXaIGjLvn372gwBHWNMDcYYevfuzc6dO50uReLMWsv8FVuaHQ5qS50vyPwVW5hSWqgvd1FI2iCA1s816BhjatEvd3pYW76L6lpvVI+t3uNlbfkuRvfrFeOqUl/cvgYbY54F3gGOM8ZUGmOuNMZcYIypBE4Alhlj/hqP1256jLGtbxZNjzH6ApHtiopIfKyvqMEf5e+hP2DZUFET44rSQ9yCwFp7mbX2SGutx1qbb6193Fr7YuPfs6y1h1trvxOP1+7IMcZ4uuqqq9i0aVOr2yxdurTNbURSVW29H18guuuo+wJBauv9Ma4oPaTcgfGOHmO0Nrofwkg89thjDBkypNVtFASSznKzMvC4ozsM6HG7dHg3SikXBLE4xhiprVu3cvzxxzNt2jRGjBjBRRddxN69e1m+fDmjRo1i+PDhXHHFFdTX1wNw6qmnHmiV0bVrV26//XaKiooYN24c27dv5+233+bPf/4zN954IyNHjmTLli08/PDDDBkyhBEjRnDppZdG9b5EksXIgjwyopy4keE2FBXkxbii9JByQZDoY4z/+te/mDVrFu+//z7du3dn3rx5TJ8+nT/96U988MEH+P1+HnnkkWaPq62tZdy4cWzYsIFTTjmFRx99lBNPPJFJkybxwAMPsH79egYOHMh9993HunXreP/995k/f35U70skWRQX9qR3bmZUj+3TNYviwp4xrig9pFwQJPoYY0FBASeddBIAl19+OcuXL+eYY47h2GOPBWDatGm8+eabzR6XmZnJxIkTARg9ejRbt24N+/wjRoxg6tSpPP3002RkaLdXUpsxhjnjB5LjcbfrcTkeF7PHD9DssiilXBAk+hhjtD94Ho/nwGPdbjd+f/gAWrZsGVdffTVr1qxh9OjRLW4nkiomjylg2NHdycyI7OMpK8PF8KN7MLmkIM6Vpa6UC4JEH2MsLy/nnXfeAeDZZ59lwoQJbN26lY8//hiAP/zhD4wfPz7i5+vWrRvffPMNAMFgkIqKCk477TTuv/9+vv76a/bs2dOu+kSSjcftYuGMUory88jxtP67nONxUZTfg7IZY7QotANS7v9coo8xDh48mCeffJIRI0bw1Vdfcf3111NWVsbFF1/M8OHDcblczJkzJ+Lnu/TSS3nggQcYNWoUH330EZdffjnDhw9n1KhRXH/99fTo0aO9b0sk6eRmZbBo5jjumDiEgl45dMl0k+l2YYBMt4sumW4Ke3XhjolDeGbmWM0W6iATz+mSsVJSUmIPvTDN5s2bGTx4cNjtn3n3M+Yu29yuKaQ5Hhd3TBzSrl4lW7duZeLEiWzcuDHix0j0Wvs3l9S1v1fYhia9wooKelBc2EPnBNpgjFljrS1pa7uUjNHJYwpYur4qou6FoGOMIp2ZMYbR/XqpdUQcpdyhIUjcMcb+/ftrb0BEkl5K7hHAwWOMi1c3dh/d48UfsPgCQTxuFxluQ5+uWcweP4DJJeo+KiLpK2WDABr2DKaO7ceU0kIdYxQRaUFKB8F+OsYoItIyHQ8REUlzCoJOZN68eQcazJ1xxhl89tlnYbf705/+xIgRIxg6dCg33XTTgfE333yT4uJiMjIyeO6552JSU319PZdccgmDBg1i7NixLbbC6N+/P8OHD2fkyJGUlBycrbZkyRKGDh2Ky+Xi0CnAItI5KAg6kVGjRrF69Wref/99LrroopAP+f2qq6u58cYbWb58Of/85z/Zvn07y5cvB6CwsJCFCxcyZcqUiF7vjTfeYPr06a1u8/jjj9OzZ08+/vhjrr/+em6++eYWt/3b3/7G+vXrQz7whw0bxgsvvMApp5wSUU0ikngKgijdeeedPPTQQwdu33777Tz88MMdes7TTjuNLl26ADBu3DgqKyubbfPJJ59w7LHHcthhhwEwYcIEnn/+eaDhW/mIESNwuWL3z/rSSy8xbdo0AC666CKWL1/erms2DB48mOOOO67VbV588UUmTJiAtZYvvviCY489lm3btnWobhGJXGqcLP7LLbDtg9g+5xHD4Zz7Wrz7yiuv5MILL+S6664jGAzyxz/+kZUrVzbb7uSTTz7QO6ipBx98kAkTJrT4/I8//jjnnHNOs/FBgwbx4YcfsnXrVvLz81m6dCleb3TXX4hEVVUVBQUNC+0yMjLIy8ujurqaPn36hGxnjOGss87CGMPs2bOZNWtWxK9xwQUX8Pzzz/Pb3/6Wl19+mR//+MccccQRMX0fItKy1AgCB/Tv35/evXuzbt06tm/fzqhRo+jdu3ez7d566612P/fTTz/N6tWrWbFiRbP7evbsySOPPMIll1yCy+XixBNP5JNPPmnX848dO5b6+nr27NnDV199xciRIwH4+c9/zne+E3r10HDf/sNNuf3HP/7BUUcdxY4dOzjzzDM5/vjj23U46Ne//jXDhg1j3LhxXHbZZe16PyKpZn9bjfVNpryPLMijuLBnXKa8p0YQtPLNPZ6uuuoqFi5cyLZt27jiiivCbtPePYLXXnuNe+65hxUrVpCVlRX2Oc8//3zOP/98ABYsWIDb3b7e7e+99x7QcI5g4cKFLFy4sMVt8/PzqaioID8/H7/fT01NDb16NZ+Ge9RRRwHQt29fLrjgAlauXNmuIKiqqsLlcrF9+3aCwWBMD2+JJAtfIMjiVY2LYGu9+ANBfAGLx23IcLvonZvJnPEDmTwmtotgUyMIHHLBBRdw11134fP5WLRoUdht2rNHsG7dOmbPns3LL79M3759W9xux44d9O3bl127dvG73/2OxYsXt7v2SE2aNIknn3ySE044geeee47TTz+92TeS2tpagsEg3bp1o7a2lldeeYW77ror4tfw+/3MmDGDRYsW8dRTTzFv3jxuuOGGWL8VkU6ttt7P9LKVbKza3axhpjdg8QYC7PXWMXfZJpaur2LhjNKYdV2N29cuY8wTxpgdxpiNTcZ6GWNeNcZ81PhnUl9XLjMzk9NOO43Jkye3+1t5ODfeeCN79uzh4osvZuTIkUyaNOnAffsP3wBcd911DBkyhJNOOolbbrnlwNXQVq1aRX5+PkuWLGH27NkMHTq0wzVdeeWVVFdXM2jQIObNm8d99zXsfX3++eece+65AGzfvp1vf/vbFBUVUVpaynnnncfZZ58NNJwIzs/P55133uG8885rdugJ4N577+Xkk0/m5JNPZt68eTz22GNs3ry5w7WLJAtfIMj0spVsqKxps2tynS/IhsoappetxBflZXkPFbc21MaYU4A9wFPW2mGNY/cDX1lr7zPG3AL0tNa2PB+xUXvbUCdKMBikuLiYJUuW8K1vfcvRWtJBZ/g3F4mHcK3ze7KbE1yb+EuwFBvmO3skrfMjbUMdtz0Ca+2bwFeHDH8XeLLx708C34vX68fbpk2bGDRoEGeccYZCQESiZq1l/ootB0KgB9/wdtYPWJc9h99lPkyB2Rn2cXW+IPNXbGnXdO6WJPocweHW2i8ArLVfGGNaPBBujJkFzIKGhVKdzZAhQ9o9W0dE5FBry3dRXesljz0sy7qNfPPlgft+5ruMcnt4i4+t3uNlbfmuDvdR67Qni621C4AF0HBoqIVt1D00TSTDlfREorHpkwr+an5AQfbBb/73+y7hd4HvtvlYf8CyoaIm6YJguzHmyMa9gSOBHdE+UXZ2NtXV1fTu3VthkOKstVRXV5Odne10KSKxU/c1PHoa//7VJ9D4EfaAbzK/DUR+xNwXCFJb7+9wKYkOgj8D04D7Gv98Kdonys/Pp7Kykp07wx8/k9SSnZ1Nfn6+02WIdNy+Gnj0dKj++MDQQ4GL+KXvwnY/lcftiskU0rgFgTHmWeBUoI8xphL4EQ0BsNgYcyVQDlwc7fN7PB6OOeaYWJQqIhJ/+2rgsQnw5f8dHBt/C2sGzOb3j68EWp82Gk6G21BUkNfh0uIWBNbalvoEnBGv1xQR6XT27YbHz4KdTdbGnHITnHYbGEOxtfTOzWSvt67dT92naxbFhR1fjtVpTxaLiCS1+m/gibNh+8aDYyf/N5x+JzQ5r2mMYc74gc3WEbQlx+Ni9vgBMTlHqiAQEYml+j1QdnZoR+STfggT7g4JgKYmjylg6foqNlTW4PW3vVo4K8PF8KN7MLmkICYlKwhERGLBWwtl58IX6w+OnXgtnPmTFgNgP4/bxcIZpY29hmqo87UcBjmehhAomzEmZo3nFAQiIh3hrYWFE+HztQfHTvgBnDW3zQBoKjcrg0Uzx7F4dWP30T1e/AGLLxDE43aR4Tb06ZrF7PEDmFyi7qMiIs7z7oWnJkHlqoNj474P37m3XQHQlMftYurYfkwpLWRt+S42NLkeQVFBD4oLe+h6BCIijvPVwVPfg4p3D46VzoZzfh51ABzKGMPofr06vGI4UgoCEZFI+OrgDxdC+dsHx8bMhHMfiFkAOEVBICLSGt8+eOYi2NrkIlMlV8B585I+APZTEIiIhLNvN9x3yPTM4mkw8VeQYpdSVRCIiDRV/w38LExfq7t2pVwA7KcgEBGBhoVgPzu6+fid1eBO7Y/K1H53IiJt8dbCvUc1H7/zS3B7El+PAxQEIpKevHvh3iObj9+xEzIyE1+PgxQEIpJefHVwzxHNx9MwAPZTEIhIevDtg3vCXP/3jh2QkZX4ejoRBYGIpDZ/Pczt23z89u3g0eVPQUEgIqnK74W5hzUfv30beHISX08npiAQkdTSUgDc9gVkdkl8PUlAQSAiqSHgg5/2aT5+2+eQmZv4epKIgkBEklvADz/t3Xz81irI6pr4epKQgkBEklOLAVAJWd0SX08ScyQIjDHXATMBAzxqrf2VE3WISBIKBuAnYfr031IB2d0TX08KSHgQGGOG0RACpYAXeNkYs8xa+1GiaxGJhrWWteW7WN/k6lEjC/IoLuwZl6tHSaMWA6AcsvMSX08KcWKPYDDwrrV2L4AxZgVwAXC/A7WIRMwXCLJ4VeP1ZGu9+ANBfAGLx23IcLvonZvJnPEDmTwmtteTTXstBcDNWyGnZ8LLSUVOBMFG4B5jTG+gDjgXWO1AHSIRq633M71sJRurdlPnC4Tc5w1YvIEAe711zF22iaXrq1g4o5TcLJ2C65BgEH4S5oP+pk+hS2Iu4ZguEv6Taq3dbIz5OfAqsAfYAPgP3c4YMwuYBVBYWJjQGkWa8gWCTC9byYbKGrz+YKvb1vmCbKisYXrZShbNHKc9g2i0FADX/xPywlwnQDrMkZ9Sa+3j1tpia+0pwFdAs/MD1toF1toSa23JYYeFWRwikiCLV1WwsWp3myGwn9cfZGNVDYtXV8S5shRjLdyd1zwEfvgB3F2jEIgjp2YN9bXW7jDGFAIXAic4UYdIW6y1zF+xpdnhoLbU+YLMX7GFKaWFOoHcFmvhxz2aj1+3AXr2T3g56cipg5jPN54j8AFXW2t3OVSHSKvWlu+iutYb1WOr93hZW76L0f10PDuslgLg2nXQa0Di60ljjgSBtfZkJ15XpL3WV9TgD0R2SOhQ/oBlQ0WNguBQLQXAD9ZAn0GJr0e0slikNbX1fnwBG9VjfYEgtfXN5kGkr5YC4OpVcNixia9HDlAQiLQiNysDj9vgjSIMPG6XppDud3eYBV/ffxf6Dk58LdKMfkpFWjGyII8MtwtvoH0niwEy3IaigjRf8RouAP7zbTh8aOJrkRYpCERaUVzYk965mez11rX7sX26ZlFcmKYrX8MFwOy34MgRia9F2qQgEGmFMYY54wcyd9nmdk0hzfG4mD1+QPpNHQ0XALPegKNGJboSaQcFgUgbJo8pYOn6qohWFgNkZbgYfnQPJpcUJKC6TiJcAFz1OuSPTnwt0m4KApE2eNwuFs4obew1VEOdr+UwyPE0hEDZjDHp0V4iXABc+SoUlCa+FomagkAkArlZGSyaOY7Fqxu7j+7x4g9YfIEgHreLDLehT9csZo8fwOSS1Os+emjr7WvfKmm+0RV/hcJxiS9OOkxBIBIhj9vF1LH9mFJayNryXWxocj2CooIeFBf2SLlzAoe23t7kuqTZNq+WPsGp37kg5cIvnSgIRNrJGMPofr1SfsVw09bbm92XNGtReZn3dt4JDiXnHRfDyt9V6+0kpn81EWlmf+vtJdvOAXfofZd7b+XvweEHbqv1dvJTEIhIM56f9mTJIWPTvDezIlgUdvumrbenju0X/wIlphQEInJQmFlAV3hv4PVgcZsPVevt5KUgEJGwATDT+1+8GgwzO6gVar2dnBQEIuksTAC8NuIXfH/NUXiD7W+0p9bbyUlBIJJEDp3Pn5uVwciCPIoLe7bvcEy4hWAXlcGwC9m0/CN8gf+Lqj613k5OCgKRJHDofH5/IIgvYPG4DRluF71zM5kzfiCTx7SxmC1cAFz4GIy4+MBNtd5OP/oXE+nkms7nP7TxnTdg8QYC7PXWMXfZJpaurwo/nz9cAHxvPoy8rNmwWm+nHwWBSCe2fz5/JA3vws7nDxcA3/0tjLq8xedR6+30oyAQ6cQWr6pgY9XuiLqewsH5/J6fhvkwPv8hGD29zedQ6+30oyAQ6aSstcxfsaVdH8YfZ11OhjkkNM59EEpntuu11Xo7vTiyFtwYc70x5p/GmI3GmGeNMdlO1CHSma0t30V1rTeibT/MmsbW7CkhIVA+9kdwd027QwAOtt4uys8jx9P6x0SOx0VRfhq13k5BCf9XM8YcDVwLlFhrh9HQyeTSRNch0tmtr6jBH2j92/j7WVeyNXsK2cZ3YOynvqkc6/sjr3W7oEOvv7/19h0Th1DQK4cumW4y3S4MkOl20SXTTWGvLtwxcQjPzByr2UJJzKl/uQwgxxjjA7oAnztUh0inVVvvx9fCFM6t2VOajf3Mdxm/D5wPgCE28/nTsfV2Okp4EFhrq4wxDwLlQB3wirX2lUTXIdLZhZvPHy4A7vdN5neB74WMxXo+f7q03k5XCQ8CY0xP4LvAMcDXwBJjzOXW2qcP2W4WMAugsLAw0WWKOK7pfP5wAbDQfxZ3+6eHfazm80t7OHFoaALwqbV2J4Ax5gXgRCAkCKy1C4AFACUlJe1f4iiS5IoLe/K+6zIyskPPEzzjP4Pb/Ve2+ljN55f2cCIIyoFxxpguNBwaOgNY7UAdIp3XPUdifHtDfkGX+E/hRv+cNh+q+fzSXk6cI3jPGPMcsBbwA+to/OYvkvbuK4R9NSFDb2afzlV7Zmk+v8SNI7OGrLU/An7kxGuLdEoPDILanaFjQy+Aixcyut5PUdlKNlbVUOdrOQxyPA0hoPn8sRGzTq9JQBN/RZw0byjsrgwdO34iXPrMgZv75/MvXt3YfXSPF3/A4gsE8bhdZLgNfbpmMXv8ACaXtNF9VNoUs06vScRY2/nPw5aUlNjVq3UaQVLIQyNh16ehY9/6Dkxd3OrD9n9L1Xz++Git02tTOR4Xw47OC9/ptRMxxqyx1rZ5mTkFgUgi/aYUvvxX6NiA0+A/ljpTjxzgCwSZ8ui7EfdXysxwUZSfd7DTaycUaRB03igTSSXzvw3bPggd638yTP9fZ+qRZqLt9Lp4dQVTx/aLc3XxpSAQiadHT4eqNaFj+aVw1avO1CNhRdPpFRquATF/xRamlBYm9aE5BYFIPDxxNpS/Ezp25EiYvcKZeqRV7en0eqjqPV7Wlu9K6vYbbQaBMeYHwDPW2l0JqEckuT05CT495MO+71D4/tvO1CMRiaTTa0v8AcuGiprUDgLgCGCVMWYt8ATwV5sMZ5hFEunpf4OPXwsd6z0IrlkTfnvpVFrr9NoWXyA2nV6d1GYQWGvvMMbcCZwFzAB+Y4xZDDxurd0S7wJFOrVFl8L//SV0rEch/PCD8NtLpxSu02ukYt3p1QkRVW+ttcaYbcA2GtpC9ASeM8a8aq29KZ4FSmTSaRVkp/DmA/D63NCxbkfCf3/oTD3SIU07vbZXKnR6jeQcwbXANOBL4DHgRmutzxjjAj4CFAQOSsdVkI76+y/htbtDx3J6wc2fht1ckkNxYU9652ay11vX7semQqfXSPYI+gAXWms/azporQ0aYybGpyyJRGurIL0BizcQYK+3jrnLNrF0fVWnXwXZqb39a3jljtCxwefDJU+H316SijGGOeMHMnfZ5nZNIU2VTq9tfkW01t51aAg0uW9z7EuSSPgCQaaXrWRDZU2bP7h1viAbKmuYXrYSX5QzI9LWu4/A3XmhIXDcuQ0XhVcIpJTJYwoYdnR3MjMi23NOpU6vOlaQpDqyClIisPLRhgB4+ZaDY4PObAiAy551ri6JG4/bxcIZpRTl55Hjaf2jMcfjoig/dTq96jhBEkr3VZBxtepxWPZfoWMDToX/eMmJaiTB0rXTq4IgCaX7Ksi4WPMk/M+1oWP9vg0zljlTjzjG43YxdWw/ppQWpk2nVwVBEkr3VZAxte5peOnq0LGCcXDlX52pRzoNYwyj+/VKi98VBUESSvdVkDGx4Y/w4uzQsaOKYdbfnKlHxEEKgiSU7qsg94tqEd37S+CFq0LHDh8O//n3+Bcs0kmlxidCmkn3VZBRLaLb+AI8NyP0iQ4bDFe/m/g3INLJKAiSUDqvgmzvIro/nLCN7BcPCQA1gxMJkfC5T8aY44wx65v8t9sY88NE15HM9q+CzPG42/W4ZF8F2Z5FdCcH3mPJtnNCQ6BHYcM6AIWASIiE7xFYa/8FjAQwxriBKuDFRNeR7CaPKWDp+qqIr6+aCqsgI1lEd4ZrDY9n/iJkbG9WX7rc+lG8yxNJWk6vhjgD2NJSCwtpWbqtgmxrEd2prnVszZ4SEgI7bXf671vEd9y/R5fQEGmZ0+cILgW0Xj9K6bQKsqVFdN8ylbyaFdoAt8Z2oaj+sQO3tYhOpHWOBYExJhOYBNzawv2zgFkAhYWFCawsuaTLKshDF9ENNFUsz7oxZJu9Nosh9WXNHqtFdCKtc3KP4BxgrbV2e7g7rbULgAUAJSUl2q9vQ6qvgty/iG6A+ZzXs24Iue8P/gnc6b+ixcdqEZ1I65wMgsvQYSGJ0JGBKj7NnhIytsh/Grf5Z7b52FRaRCcSD478dhhjugBnArPb2lbSXPUW+HUxFzcZWuwfz03+yH90UmERnUg8ORIE1tq9QG8nXluSxFefwsMjQ4b+4j6V/6yd1e6nSvZFdCLxpv1l6Vx2fQYPjQgdGz4Z/u1Rvnr3M3LS9FKCIvGkIJDO4ety+NXw0LGhF8LFB2cBpeMiOpFEUBCIs2oq4ZdDQ8cGT4JL/tBs0/2L6Bp6DdVQ52s5DHI8DSGQzIvoRBJFQSDOqKmCXw4JHTt+Ilz6TKsPS6dFdCKJoiCQxNr9Bcw7PnTs2LNhyp8ifop0WUQnkigKAkmMb7bBL44LHRs0AS5/PuqnTPVFdCKJoiCQ+PpmO/zi2NCxAafCf7zkRDUiEoaCQOJjz054cFDoWP+TYfr/OlOPiLRIQSCxVfslPDAwdKzwRLjiL87UIyJtUhBIbNRWwwMDQsfyS+HKV0AnbkU6NQWBdMzer+D+Y0LHjiqGma8rAESShIJAohMuAI4YAbPfVACIJBkFgbRP3dfw836hY4cPgzl/VwCIJCkFgURmXw3cd8iV4g47Hr7/rgJAJMkpCKR1+3bDfYc0bes9CH6wWgEgkiIUBBJe/Tfws/zQsZ794Zp14FL/HpFUoiCQUPV74GdHh47lFcB17ysARFKUgkAaeGvh3qNCx7odCddvUgCIpDgFQboLFwC5feG/PwSX25maRCShFATpyrsX7j0ydCynF9z4sQJAJM0oCNKNrw7uOSJ0LCsPbv5UASCSphwJAmNMD+AxYBhggSuste84UUva8O2Dew4PHfPkwi3l4Nb3AZF05tQnwEPAy9bai4wxmUAXh+pIff56mNs3dMydBbd9rgAQEcCBIDDGdAdOAaYDWGu9gDfRdaS8cAFg3HDHdnB7nKlJRDolJ74SDgB2AmXGmCJgDXCdtbbWgVpSj98Lcw9rPn7HTsjITHw9ItLpOREEGUAxcI219j1jzEPALcCdTTcyxswCZgEUFhY2exI5RMAHP+3TfFwBICJtcCIIKoFKa+17jbefoyEIQlhrFwALAEpKSmziyusYay1ry3exvqKG2no/uVkZjCzIo7iwJyYevXkCvoZZQEF/6PgdOyAjK/avJyIpJ+FBYK3dZoypMMYcZ639F3AGsCnRdcSaLxBk8aoK5q/YQnWtF38giC9g8bgNGW4XvXMzmTN+IJPHFOBxx2ClbsD8e/gUAAAL5klEQVTf0ArCvy90/Pbt4Mnu+POLSNpwatrINcAzjTOGPgFmOFRHTNTW+5letpKNVbup8wVC7vMGLN5AgL3eOuYu28TS9VUsnFFKblaU/+sD/obrAXj3hI7fvg08OVG+AxFJZ44EgbV2PVDixGvHmi8QZHrZSjZU1uD1B1vdts4XZENlDdPLVrJo5rj27RkEAw1XBNtXEzquABCRDtJE8g5avKqCjVW72wyB/bz+IBurali8uoKpY/u1/YBgAB78FuytDh2/7QvI1PILEek4BUEHWGuZv2JLs8NBbanzBZm/YgtTSgtbPoEcDMIvjoPaHaHjt30OmblRViwi0pyCoAPWlu+iuja6tXDVe7ysLd/F6H69Qu8IBuGXQ+Gbz0PHb62CrK5RVioi0jIFQQesr6jBH4jskNCh/AHLhoqag0EQDMJDRVBTHrrhrZWQ1a2DlYqItExB0AG19X58geiWOPgCQWrr/WAtPDwKdn0ausEtFZDdPQZVioi0TkHQAblZGXjcBm8UYeBxG/59zUXw1tbQO24ph+y82BQoIhIBBUEHjCzII8Ptwhtoz8liy18zb+Y4VyXsbTJ882eQ0yPWJYqItElB0AHFhT3pnZvJXm9dBFtb/l/mbQxxfRY6fPNWyOkZj/JERCKiq5J3gDGGOeMHkuNp7cpelv/JvI2t2VNDQmDJGW/B3TUKARFxnPYIOmjymAKWrq8Ks7LYsjTzLka6toRsP9b/KP3yC3jmxGGJLVREpAXaI+ggj9vFwhmlFOXnkeNxAZaHPb9ma/bUkBAYtW8+gwN/pF9+AWUzxsSm8ZyISAxojyAGcrMyWDRzHJufvZURH88PuW+c7/fsdvegT68sbhg/gMklMeo+KiISIwqCWHjj53jeuJcRjTf3dT+G50Y9ya5gF2ZlZVBU0IPiwh7xuR6BiEgHKQg64s0H4PW5B2/3GgAz/0Z2Tg8ud64qEZF2URBE461fwPKfHLzdsz/MekMzgEQkKSkI2uPvv4LXfnTwdl4hzF4BXXq1/BgRkU5OQRCJfzwMr9558Hb3o2HO3xUAIpISFASteee38NfbDt7udiTM+Qfk9nauJhGRGFMQhPPuI/DyLQdv5/aF778DuX2cq0lEJE4UBE29twD+cuPB2116w/ffg66HOVeTiEicKQgAVj4K/++Gg7dzesLVK6FrX+dqEhFJEEeCwBizFfgGCAB+a22JE3Ww6nFY9l8Hb2fnwdWroNvhjpQjIuIEJ/cITrPWfunIK69ZCP9z3cHbmd3gB6ug+5GOlJPMrLWsLd/F+ooaauv95GZlMLIgj+LCnlpJLZIk0uvQ0Nqn4M/XHLzt6QLXrIHuRzlXU5LyBYIsXlXB/BVbqK714g8E8QUsHrchw+2id24mc8YPZPIY9VYS6eycCgILvGKMscDvrbUL4vpq656Gl64+eDsjG65ZC3lHx/VlU1VtvZ/pZSvZWLWbOl/o1dm8AYs3EGCvt465yzaxdH0VC2eUkpuVXt85RJKJU7+dJ1lrPzfG9AVeNcZ8aK19s+kGxphZwCyAwsLC6F6legv8uvjgbXcmXLsO8vKjrTvt+QJBppetDHP9hebqfEE2VNYwvWwli2aO056BSCflyG+mtfbzxj93AC8CpWG2WWCtLbHWlhx2WJTTN7e83vCnccEPP4A7dyoEOmjxqgo2Vu1uMwT28/qDbKyqYfHqijhXJiLRSngQGGNyjTHd9v8dOAvYGJcXK53ZcDnIH+2CHlHuVcgB1lrmr9jS7HBQW+p8Qeav2IK1Nk6ViUhHOLFHcDjwd2PMBmAlsMxa+7IDdUg7rS3fRXWtN6rHVu/xsrZ8V4wrEpFYSPg5AmvtJ0BRol9XOm59RQ3+QGSHhA7lD1g2VNQwup8a9Yl0Njp7JxGrrffjC0R3eMcXCFJb749xRSISCwoCiVhuVgYed3SLxDxul6aQinRSCgKJ2MiCPDKinAKa4TYUFeTFuCIRiQUFgUSsuLAnvXMzo3psn65ZFBfqUp4inZGCQCJmjGHO+IHkeNztelyOx8Xs8QPUe0ikk1IQSLtMHlPAsKO7k5kR2Y9OVoaL4Uf3YHJJQZwrE5FoKQikXTxuFwtnlFKUn0eOp/UfnxyPi6L8HpTNGKP2EiKdmKZxSLvlZmWwaOY4Fq9u7D66x4s/YPEFgnjcLjLchj5ds5g9fgCTS9R9VKSzUxBIVDxuF1PH9mNKaSFry3exocn1CIoKelBc2EPnBESSREoGgS6WkjjGGEb366UVwyJJLKWCQBdLERFpv5QJAl0sRUQkOinxtbjpxVLaapHc9GIpvigbqImIpJKUCAJdLEVEJHpJHwS6WIqISMckfRDoYikiIh2T9EEQi4uliIiks6QPAl0sRUSkY5I+CHSxFBGRjkn6INDFUkREOibpg0AXSxER6RjHgsAY4zbGrDPG/G8Hn0cXSxER6QAn9wiuAzbH4ol0sRQRkeg5EgTGmHzgPOCxWDyfLpYiIhI9p6bM/Aq4CegWqyfUxVJERKKT8CAwxkwEdlhr1xhjTm1lu1nALIDCwsKInlsXSxERaT+T6F47xpifAf8O+IFsoDvwgrX28pYeU1JSYlevXp2gCkVEUoMxZo21tqTN7Zxsuta4R3CDtXZiG9vtBD6L8mX6AF9G+dhkpfecHvSe00NH3nM/a+1hbW2UFMtqI3kjLTHGrI4kEVOJ3nN60HtOD4l4z44GgbX2DeANJ2sQEUl3mjojIpLm0iEIFjhdgAP0ntOD3nN6iPt7dvRksYiIOC8d9ghERKQVKR0EsWpsl0yMMT2MMc8ZYz40xmw2xpzgdE3xZIy53hjzT2PMRmPMs8aYbKdrigdjzBPGmB3GmI1NxnoZY141xnzU+GfKtNJt4f0+0Phz/b4x5kVjTA8na4y1cO+5yX03GGOsMaZPPF47pYOAGDa2SyIPAS9ba48Hikjh92+MORq4Fiix1g4D3MClzlYVNwuBsw8ZuwVYbq39FrC88XaqWEjz9/sqMMxaOwL4P+DWRBcVZwtp/p4xxhQAZwLl8XrhlA2CWDe2SwbGmO7AKcDjANZar7X2a2erirsMIMcYkwF0AT53uJ64sNa+CXx1yPB3gScb//4k8L2EFhVH4d6vtfYVa+3+a8u+C+QnvLA4auHfGOCXNPRmi9sJ3ZQNAg42tovuyvbJaQCwEyhrPCT2mDEm1+mi4sVaWwU8SMM3pS+AGmvtK85WlVCHW2u/AGj8s6/D9STSFcBfnC4i3owxk4Aqa+2GeL5OSgZB08Z2TteSYBlAMfCItXYUUEtqHS4I0XhM/LvAMcBRQK4xpsWeVZIajDG309Cr7Bmna4knY0wX4Hbgrni/VkoGAXASMMkYsxX4I3C6MeZpZ0tKiEqg0lr7XuPt52gIhlQ1AfjUWrvTWusDXgBOdLimRNpujDkSoPHPHQ7XE3fGmGnARGCqTf257wNp+JKzofGzLB9Ya4w5ItYvlJJBYK291Vqbb63tT8PJw9db626aKqy124AKY8xxjUNnAJscLCneyoFxxpgupqG/+Bmk8MnxMP4MTGv8+zTgJQdriTtjzNnAzcAka+1ep+uJN2vtB9bavtba/o2fZZVAcePveUylZBCkuWuAZ4wx7wMjgXsdriduGvd8ngPWAh/Q8POckitPjTHPAu8AxxljKo0xVwL3AWcaYz6iYVbJfU7WGEstvN/f0HAxq1eNMeuNMfMdLTLGWnjPiXnt1N+7EhGR1miPQEQkzSkIRETSnIJARCTNKQhERNKcgkBEJM0pCETayRhTYIz51BjTq/F2z8bb/ZyuTSQaCgKRdrLWVgCPcHDe/n3AAmvtZ85VJRI9rSMQiYIxxgOsAZ4AZgKjrLVeZ6sSiU6G0wWIJCNrrc8YcyPwMnCWQkCSmQ4NiUTvHBraXw9zuhCRjlAQiETBGDOShv4+44Dr93cBFUlGCgKRdmrsdPoI8ENrbTnwAA0XyBFJSgoCkfabCZRba19tvP074HhjzHgHaxKJmmYNiYikOe0RiIikOQWBiEiaUxCIiKQ5BYGISJpTEIiIpDkFgYhImlMQiIikOQWBiEia+/9mPApqfsgHgAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(X.T[1], y, '.', markersize=30, label='points')\n",
    "plt.plot(X.T[1], np.dot(X, weights.flatten()), '-',\\\n",
    "         label='y = %2.2f + %2.2f x' % tuple(weights.flatten()))\n",
    "plt.xlabel('X')\n",
    "plt.ylabel('y')\n",
    "plt.legend()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "weights"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Multiple Linear Regression"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Year</th>\n",
       "      <th># of marriges</th>\n",
       "      <th># of divorces</th>\n",
       "      <th># of death</th>\n",
       "      <th># of born</th>\n",
       "      <th># inc</th>\n",
       "      <th>tot_pop</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2002</td>\n",
       "      <td>72696</td>\n",
       "      <td>29351</td>\n",
       "      <td>38925</td>\n",
       "      <td>102008</td>\n",
       "      <td>63083</td>\n",
       "      <td>10207295</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2003</td>\n",
       "      <td>72750</td>\n",
       "      <td>32499</td>\n",
       "      <td>38068</td>\n",
       "      <td>101227</td>\n",
       "      <td>63159</td>\n",
       "      <td>10174086</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2004</td>\n",
       "      <td>71553</td>\n",
       "      <td>26994</td>\n",
       "      <td>38274</td>\n",
       "      <td>99828</td>\n",
       "      <td>61554</td>\n",
       "      <td>10173162</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2005</td>\n",
       "      <td>71286</td>\n",
       "      <td>25300</td>\n",
       "      <td>38751</td>\n",
       "      <td>90468</td>\n",
       "      <td>51717</td>\n",
       "      <td>10167344</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2006</td>\n",
       "      <td>73924</td>\n",
       "      <td>24354</td>\n",
       "      <td>38519</td>\n",
       "      <td>93922</td>\n",
       "      <td>55403</td>\n",
       "      <td>10181166</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>2007</td>\n",
       "      <td>76399</td>\n",
       "      <td>24615</td>\n",
       "      <td>39016</td>\n",
       "      <td>101249</td>\n",
       "      <td>62233</td>\n",
       "      <td>10192710</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>2008</td>\n",
       "      <td>71753</td>\n",
       "      <td>23319</td>\n",
       "      <td>38298</td>\n",
       "      <td>94736</td>\n",
       "      <td>56438</td>\n",
       "      <td>10200827</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>2009</td>\n",
       "      <td>68841</td>\n",
       "      <td>23920</td>\n",
       "      <td>38790</td>\n",
       "      <td>89594</td>\n",
       "      <td>50804</td>\n",
       "      <td>10208302</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>2010</td>\n",
       "      <td>70466</td>\n",
       "      <td>21768</td>\n",
       "      <td>40130</td>\n",
       "      <td>93266</td>\n",
       "      <td>53136</td>\n",
       "      <td>10312545</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>2011</td>\n",
       "      <td>71519</td>\n",
       "      <td>20617</td>\n",
       "      <td>40320</td>\n",
       "      <td>91526</td>\n",
       "      <td>51206</td>\n",
       "      <td>10249679</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>2012</td>\n",
       "      <td>71695</td>\n",
       "      <td>20177</td>\n",
       "      <td>41514</td>\n",
       "      <td>93914</td>\n",
       "      <td>52400</td>\n",
       "      <td>10195318</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>2013</td>\n",
       "      <td>68819</td>\n",
       "      <td>20126</td>\n",
       "      <td>42063</td>\n",
       "      <td>84066</td>\n",
       "      <td>42003</td>\n",
       "      <td>10143645</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>2014</td>\n",
       "      <td>64823</td>\n",
       "      <td>19477</td>\n",
       "      <td>42153</td>\n",
       "      <td>83711</td>\n",
       "      <td>41558</td>\n",
       "      <td>10103233</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>2015</td>\n",
       "      <td>64193</td>\n",
       "      <td>18176</td>\n",
       "      <td>43053</td>\n",
       "      <td>83005</td>\n",
       "      <td>39952</td>\n",
       "      <td>10022181</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>2016</td>\n",
       "      <td>57643</td>\n",
       "      <td>17777</td>\n",
       "      <td>43540</td>\n",
       "      <td>75536</td>\n",
       "      <td>31996</td>\n",
       "      <td>9930616</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>2017</td>\n",
       "      <td>53776</td>\n",
       "      <td>17083</td>\n",
       "      <td>42932</td>\n",
       "      <td>65389</td>\n",
       "      <td>22457</td>\n",
       "      <td>9857426</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>2018</td>\n",
       "      <td>52005</td>\n",
       "      <td>16940</td>\n",
       "      <td>45219</td>\n",
       "      <td>58074</td>\n",
       "      <td>12855</td>\n",
       "      <td>9765623</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>2019</td>\n",
       "      <td>48261</td>\n",
       "      <td>16972</td>\n",
       "      <td>43829</td>\n",
       "      <td>53673</td>\n",
       "      <td>9844</td>\n",
       "      <td>9729107</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>2020</td>\n",
       "      <td>44746</td>\n",
       "      <td>16282</td>\n",
       "      <td>45522</td>\n",
       "      <td>47445</td>\n",
       "      <td>1923</td>\n",
       "      <td>9668465</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>2021</td>\n",
       "      <td>37012</td>\n",
       "      <td>14967</td>\n",
       "      <td>48798</td>\n",
       "      <td>45531</td>\n",
       "      <td>-3267</td>\n",
       "      <td>9509458</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    Year  # of marriges  # of divorces  # of death  # of born  # inc   tot_pop\n",
       "0   2002          72696          29351       38925     102008  63083  10207295\n",
       "1   2003          72750          32499       38068     101227  63159  10174086\n",
       "2   2004          71553          26994       38274      99828  61554  10173162\n",
       "3   2005          71286          25300       38751      90468  51717  10167344\n",
       "4   2006          73924          24354       38519      93922  55403  10181166\n",
       "5   2007          76399          24615       39016     101249  62233  10192710\n",
       "6   2008          71753          23319       38298      94736  56438  10200827\n",
       "7   2009          68841          23920       38790      89594  50804  10208302\n",
       "8   2010          70466          21768       40130      93266  53136  10312545\n",
       "9   2011          71519          20617       40320      91526  51206  10249679\n",
       "10  2012          71695          20177       41514      93914  52400  10195318\n",
       "11  2013          68819          20126       42063      84066  42003  10143645\n",
       "12  2014          64823          19477       42153      83711  41558  10103233\n",
       "13  2015          64193          18176       43053      83005  39952  10022181\n",
       "14  2016          57643          17777       43540      75536  31996   9930616\n",
       "15  2017          53776          17083       42932      65389  22457   9857426\n",
       "16  2018          52005          16940       45219      58074  12855   9765623\n",
       "17  2019          48261          16972       43829      53673   9844   9729107\n",
       "18  2020          44746          16282       45522      47445   1923   9668465\n",
       "19  2021          37012          14967       48798      45531  -3267   9509458"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# seoul_born_death_mrg['tot_pop'] = list(sel_pop_tot.iloc[0])\n",
    "seoul_born_death_mrg"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "multi_x = seoul_born_death_mrg[['# of born','# of marriges']]\n",
    "multi_y = seoul_born_death_mrg[['tot_pop']]\n",
    "\n",
    "multi_x_train, multi_x_test, multi_y_train, multi_y_test = train_test_split(multi_x, multi_y, train_size=0.8, test_size=0.2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "16"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(multi_x_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "LinearRegression(copy_X=True, fit_intercept=True, n_jobs=None,\n",
       "         normalize=False)"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mlr = LinearRegression()\n",
    "mlr.fit(multi_x_train, multi_y_train) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.7849032007323704"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "r_sq = mlr.score(multi_x_test, multi_y_test) # Performance Indicators of Regression Models\n",
    "r_sq"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.9612113219710109"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "r_sq_t = mlr.score(multi_x_train, multi_y_train) # Performance Indicators of Regression Models\n",
    "r_sq_t"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 1.5194918 , 18.01167898]])"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mlr.coef_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([8776954.04746057])"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mlr.intercept_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [],
   "source": [
    "multi_y_predict = mlr.predict(multi_x_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "No handles with labels found to put in legend.\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAEWCAYAAACXGLsWAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3XuYHFWZx/Hvj2RgArmSjNwCSQiIomKEcJENyioowQuiuBBXCReXBVFXXEVdLyx4YXVRV1gVWcVwB2Fll+XOsgQIEmAQgqACCQEZEsKYkISEjEng3T/qTOh0ero6Sdd0z8zv8zz9TPc5dXmnpqbfOqeqTikiMDMzq2aLRgdgZmbNz8nCzMxyOVmYmVkuJwszM8vlZGFmZrmcLMzMLJeThZn1W5I+KWlmo+PoD5wsGkDS05JWSxpTVv6wpJA0Pn2eKemTZdMcLKmj5HNI2q1smq9LWpFeXZJeKfk8R9LgsvV8S9KaVL9U0j2S9k91Pf6zSZqVlr+i5HVtD9PmLee49P6QFNuPyqaZLenjJct6pWy9KyS9rsJy/yxpy7LyS9P2XyFpiaRbJb2+UmxVts9+JfWHSHq1Qjz7lkxzWPp7viRpsaSHJJ0uaat6rEPSWyTdJunF9GqX9N5UJ0lfS/vdCkkdki6rtP3T520lnS9pkaSXJT0i6diybdIhaaGkrUvKTpb0v1W240mSHk/b4HlJ10vapqfpiyZpt7SvdW/L+ZJOb1Q8zc7JonHmA9O6P0h6CzCkHguOiG9GxNCIGAp8Gri7+3NEvLWH2S5L078OuA/4zxpXd3LJsodGxJF1+BVeAk6QtHOVae4uW+/QiHihu1LSRODtwCDgfRXm/076fXcCXgD+Iyem7u3TBtwNXF1W/6cK8TyQYjkGuAq4BNglIkaT/e3HATtu7jokCbgeuJHs77c9cBqwIs13AnAM8K60/H2BmZV+yZS8bgfGAvsBI4EvA9+X9Nmyybck279ySXo3cCbwNxExDHgTcE0t8xat5H/lE8CZkg5pdEzNyMmicS4BSo/WpgMXNyiWdSJiNXARsJOkkQ0KYwlwKfCNzVjGdGAW2Xae3tNEEbEK+BUwqZaFRsQa4HJgF0mj8qaXtAXwA+AbEfGLiHgxLeePEXFqRMzf3HUA2wG7AP8REWsi4i8RcXdE3JPq9wVujoin0vIXRkRPyfE4YAeyL/VnImJ1RNwIfA74lqShJdN+Dzhd0vAaYtwXuCci5qQYFkfEjIhYCSCpVdIPJD2bWjQ/kdTaPXNqtcxNrbL/krRDKt9N0nrDUJS3lGoVEbOAPwJvTsuZklpoyyTdr9TaLlnHt0vqr63xb9VnOVk0zmxguKQ3ShoEHE32BdlQ6cjyOODpiFjawFC+BRyjsi62WqQj7U8Al6XX4Srr8iuZdijZUf7cGpe9FVmS7wSW1zDLnmRfvrW21DZlHS8ATwGXSTpCZd1xZPva8ZK+IGmftL/15FDghoh4uaz8GmAosH9J2X3Ab4DP1xDjbOB9ks6QdGB391uJc4AJwF7A7sB44KsAkt4DnAUcRdYSXED2d62b1FV3EPBG4KG0v9wAfB8YDZwL3FiWEI5Nrx0BAT+sZ0zNpt8lC0kXSnpB0qM1TPtDZecJHpb0hKTe/nLsbl0cSnZE81wvr7/Ux9Lv/yzwFqDW7qSfKOtj736dUY9gIuI5sq6hM3uYZErZeh8vqXsn2ZfKNRFxH/AnSrr8ki+n3/clsi/AHlsfSff2eTlNe1REvFJSv0tZPEvTF2J3knq+e0JJ16T6lyWVxrVJ64iIV4GDyfafHwILJd2RuuKIiBlkLYOpwF3AC5K+0MPvOQZYWF4YEX8ha/GVJ92vA5+TNLqH5XXPP5Psy35f4Cbgz5L+VdIWqfX1SeBzEfFiRCwHzibrOgP4W+DnEfFwRHSRdYu9U9LYauusVdrmS4ALgH+MiDuBDwCPRcQVEbE2Ii4lS8ilXZoXRcTvU+voG2QHN6pHTM2o3yULYAZwWC0TRsRpETEpIiYB5wG/LjKwCi4BPkZ2JF+pC2ot0FJW1gKsKSCWyyNiZES8LiIOiYiHa5zvU2m+7ldPX+6b4mzg/ZLeXKFuVtl69yipmw7cFBFL0ufL2TAZ/EtEjCQ7ml1NdjRbzeVp+u2Bx4G3ldX/qSyekekLdnGq36F7wog4Ki3rEbJzKpu7DiLi2Yj4VETsmn6nNWT/C93rvCQi3k12DuJU4Ox0HqHcn0tj7ZYS37apfp3UrXQLkHtiOCJuiIj3A6OADwN/Bxyfft+tgDndSZDsHEx3C2lH4JmS5SwHXiQ7INhsaTuOiog3RsSPK60zeaZsnc+W1XVvo36p3yWLiLiL7ChhHUkTJd0s6UFJd0t6Q4VZpwFX9EqQSUQ8Q3ai+3AqJ6o/kTXHS01gw524X4qITrIkflat8yi7uuYo4N3Krrh5HvgMsI+kN1VYx9NkJ4PPq9A10lNMf0/Wf79dDSH9nuxI/cO1/g6bsI7y+f8E/ITU915WtyYirgQeq1QP/C9Zd9HWZeVHASuB+yvM8w3gFLIv/VriezUibiM7yf5mYBFZwt6jJAmOiIgRaZYFZBcDACBpGFnCeS7FRFm8NcWRY711Jruwfut/57K67tZXv9TvkkUPLgA+ExH7AF8g+0daR9I4si/h/2tAbCeSXaWyskLdVWR9zfulPtXXk32xXVk23ZbpBGH3q1qf9KbYomz5rfmz1G0555B1seQd+Xf7MNk/7RvITlpPIuuHvpf1LyhYJyJuIjs/8MlK9RWmf4zsiqGeunJKp30F+CJwlqQTJY0s+Vu21WMdksakcwG7pmW3kR2xz071J0g6XNKw1O3zPmAPKn/xX0T25X2VpHGSWiRNJeve+kZEvFQh1sfJzsl8pkqMR0r6G0mjUowHAAcBs9M2+jnwb5LaUv1YZecqIDuIO1HSXimhn012NVwHWffe88DHJQ2SdBIbfslviuuBN0k6Wtml5h8DdiO74qzbsZLekA5QzgR+FdF/n/nQ75OFshOYBwJXS3oY+BkbNrOPIevffqV8/qJFxLyIaO+h7hay/tlfAsvIdtSLyJJfqceAVSWv4+sc5kFly19VUne+1r/uv9IXUC3LqSidZD+HDZv3B2nDew7eRtbd9IuI6IiI57tfwL+TvlB6WNU5wJdUdk9GFf8KnKLXTpzvUiGeD6Xf4TKylut0oIOsK+dKsoOWal2fta7jL8BE4A6yy2V/l36ekOZbDnyNrNvkReA7wEkRcW/5CtPVYe8i+wJ+IM37PeBLEVHtBO6ZZCfAe7IUOJnsQoLlZPvxdyLiqlT/j2Qt5vvJ9vVbSQcIEXEzWevyWrJW2i5k5zFIX85/B/wT2XbdjezE+2ZJrbsPAl8i60o8DXh/SdcmZN3Il6aYBpGdF+q31B8TobKbza6PiDcru6zv8YjYoB+2ZPqHgFMj4je9FKKZ9WGSZpGddJ/R6Fh6S79vWaSTYfMlfRTWXSK37sY0SXuQ9X9ucJRlZmaZfpcsJF1B9sW/h7IhCU4ka7KeKGkOWZfNESWzTAOu7M99jWZmm6tfdkOZmVl99buWhZmZ1d/gRgdQL2PGjInx48c3Ogwzsz7lwQcf/HNE9HgZd7d+kyzGjx9Pe3vFK1DNzKwHkmq6ydfdUGZmlsvJwszMcjlZmJlZrn5zzsKsmaxZs4aOjg66uro2qGttbWXs2LG0tJQPKGzWvJwszArQ0dHBsGHDGD9+PCp5xEFEsHjxYjo6OpgwYUIDIzTbOE4WZgXo6uraIFEASGL06NF0dnY2KDLra+Z3rmDWvMUsWtbFdiNamTJxNBPaqo3ZWAyfszArSHmiyCs3Kze/cwVXtXewsmst249oZWXXWq5q72B+54pej8XJwsysSc2at5iRQ1oYPqSFLSSGD2lh5JAWZs1bnD9znRWWLJTzLOw0+uu5kuZKekTS3ql8nLIn2j0s6TFJJxcVo5lZM1u0rIuhreufLRjaOphFyza8cKJoRbYsZlD9WdhTyR5usjtwEvDTVL4QODA9F3t/4MuSdiwwTrNC9DRIpwfvtFptN6KVFV1r1ytb0bWW7UZs6sMqN11hyaLSs7DLHAFcHJnZwEhJO0TE6u6H0JM9AN1dZdbntLa2snjx4g0SQ/fVUK2tvf/Pbn3PlImjWbpqDctXreHVCJavWsPSVWuYMnF0r8fSyKuhdiJ7zGO3jlS2UNLOwA1kj0j8YkQsqLSA9LzdkwB22WWXYqM12whjx46lo6Oj4lVP3fdZmOWZ0DaUoyePZda8xTyfroZ675u2a8jVUI1MFpUuCQmAiHgW2Ct1P/2XpGsiYtEGE0dcQHoe9eTJk922t6bR0tLi+yisLia0DW1IcijXyC6eDmDnks9jgfVaEKlF8RhwUC/GZWZmZRqZLK4Djk1XRR0ALIuIhZLGShoCIGkU8FfA4w2M08xswCusGyo9C/tgYIykDuAMoAUgIs4HbgQOB+YCLwPHp1nfCHxfUpB1VZ0TEb8rKk4zM8tXWLKIiGk59QGcWqH8NmCvouIyM7ON58tSzcwsl5OFmZnlcrIwM7NcThZmZpbLycLMzHI5WZiZWS4nCzMzy+VkYWZmuZwszMwsl5OFmZnlcrIwM7NcThZmZpbLycLMzHI5WZiZWS4nCzMzy+VkYWZmuQp7+JGZ2UA3v3MFs+YtZtGyLrYb0cqUiaOZ0Da00WFtErcszMwKML9zBVe1d7Cyay3bj2hlZddarmrvYH7nikaHtkmcLMzMCjBr3mJGDmlh+JAWtpAYPqSFkUNamDVvcaND2yROFmZmBVi0rIuhrev39A9tHcyiZV0NimjzOFmYmRVguxGtrOhau17Ziq61bDeitUERbR4nCzOzAkyZOJqlq9awfNUaXo1g+ao1LF21hikTRzc6tE3iZGFmVoAJbUM5evJYtmkdzPPLutimdTBHTx7bZ6+G8qWzZmYFmdA2tM8mh3JuWZiZWS4nCzMzy+VkYWZmuZwszMwsl09wm5n1Ub059pRbFmZmfVBvjz3lZGFm1gf19thTThZmZn1Qb4895WRhZtYH9fbYU04WZmZ9UG+PPVVYspB0oaQXJD3aQ70knStprqRHJO2dyidJulfSY6n86KJiNDPrq3p77KkiL52dAfw7cHEP9VOB3dNrf+Cn6efLwLER8aSkHYEHJd0SEUsLjNXMrM/pzbGnCksWEXGXpPFVJjkCuDgiApgtaaSkHSLiiZJlLJD0AtAGOFmYmTVII89Z7AQ8W/K5I5WtI2k/YEtgXqUFSDpJUruk9s7OzsICNTMb6BqZLFShLNZVSjsAlwDHR8SrlRYQERdExOSImNzW1lZQmGZm1shk0QHsXPJ5LLAAQNJw4AbgaxExuwGxmZlZiUYmi+uAY9NVUQcAyyJioaQtgWvJzmdc3cD4zMwsKewEt6QrgIOBMZI6gDOAFoCIOB+4ETgcmEt2BdTxada/Ad4BjJZ0XCo7LiIeLipWMzOrrsiroabl1AdwaoXyS4FLi4rLzMw2nu/gNjOzXE4WZmaWy8nCzMxyOVmYmVmu3GQh6a9qKTMzs/6rlpbFeTWWmZlZP9XjpbOS3g4cCLRJ+nxJ1XBgUNGBmZlZ86h2n8WWwNA0zbCS8uXAUUUGZWZmzaXHZBERdwJ3SpoREc9IGpYVx4reC8/MzJpBLXdwD5P0ELAtgKQ/A9MjouIT8MzMrP+p5QT3BcDnI2JcRIwD/jGVmZnZAFFLstgmIu7o/hARM4FtCovIzMyaTi3dUE9J+jrZg4gAPg7MLy4kMzNrNrW0LE4gewb2r9NrDHBcgTGZmVmTqaVlcUhEfLa0QNJHAT+YyMxsgKilZfGVGsvMzKyfqnYH91SyJ9ntJOnckqrhwNqiAzMzs+ZRrRtqAdAOfBB4sKT8JeC0IoMyM7PmUu0O7jnAHEmXR8SaXozJzMyaTO45CycKMzPzw4/MzCxXj8lC0iXp5z/0XjhmZtaMqrUs9pE0DjhB0ihJ25a+eitAMzNrvGpXQ50P3AzsSnY1lErqIpWbmdkA0GPLIiLOjYg3AhdGxK4RMaHk5URhZjaA5A73ERGnSHorcFAquisiHik2LDMzaya5V0NJ+ixwGfC69LpM0meKDszMzJpHLQMJfhLYPyJWAkj6LnAvcF6RgZmZWfOo5T4LAa+UfH6F9U92m5lZP1dLy+KXwH2Srk2fPwT8oriQzMys2dRygvsHkmYCU8haFMdHxENFB2ZmZs2jlpYFEfFb4LcFx2JmZk3KY0OZmVkuJwszM8tVy30WYzZlwZIulPSCpEd7qJekcyXNlfSIpL1L6m6WtFTS9ZuybjMzq69qo852191aUrYxI9DOAA6rUj8V2D29TgJ+WlL3r8AnNmJdZmZWoGotizsl3QxsL+kwSTsC02tdcETcBSypMskRwMWRmQ2MlLRDmvd2sse3mplZE6g2kOBBwDHAKmA/4Fzg9ZKulHRKHda9E/BsyeeOVFYzSSdJapfU3tnZWYeQzMyskmrdULcCpwGvAudFxFHAk8Dp1Oeov9Jd4LExC4iICyJickRMbmtrq0NIZmZWSbVuqA8BdwFDgYsl3Q+MAz4C/LEO6+4Adi75PBZYUIflmplZnVXrhno5nTt4PiI+EBH7Ac+RdR0dW4d1Xwccm66KOgBYFhEL67BcMzOrs1ru4P5IyftZEXENcE3eTJKuAA4GxkjqAM4AWgAi4nzgRuBwYC7wMnB8ybx3A28AhqZ5T4yIW2r5hczMrP5qGRvqqZL3NZ/YjohpOfUBnNpD3UGVys3MrDF8B7eZmeVysjAzs1xOFmZmlqvHcxaSPl9txoj4Qf3DMTOzZlTtBPew9HMPYF+yS10BPkB2/4WZmQ0QPSaLiDgT1t3JvXdEvJQ+/zNwda9EZ2ZmTaGWcxa7AKtLPq8GxhcSjZmZNaVabsq7BLhf0rVkYzcdCVxcaFRmZtZUarkp79uSbgK6b5Q7PiIeKjYsMzNrJrVeOrs1sDwifgR0SJpQYExmZtZkanms6hnAl4CvpKIW4NIigzIzs+ZSS8viSOCDwEqAiFjAa5fVmpnZAFBLslidBv0LAEnbFBuSmZk1m1qSxa8k/YzsGdl/B/wv8PNiwzIzs2ZSy9VQ50g6FFhOdjf3NyLitsIjMzOzppGbLCR9NyK+BNxWoczMzAaAWrqhDq1QNrXegZiZWfOqNursKcCngImSHimpGgb8pujAzMyseVTrhrocuAk4G/hySflLEbGk0KjMzKyp9NgNFRHLIuJp4EfAkoh4JiKeAdZI2r+3AjQzs8ar5ZzFT4EVJZ9XpjIzMxsgakkWSjflARARr1LbaLVmZtZP1JIsnpL0WUkt6fUPwFNFB2ZmZs2jlmRxMnAg8BzQAewPnFRkUGZm1lxquYP7BeCYXojFzMyaVLX7LE6PiO9JOo80iGCpiPhsoZGZmVnTqNay+EP62d4bgZiZWfPqMVlExP+knxf1XjhmZtaMqnVD/Q8Vup+6RcQHC4nIzMyaTrVuqHPSzw8D2/Pao1SnAU8XGJOZmTWZat1QdwJI+mZEvKOk6n8k3VV4ZGZm1jRquc+iTdKu3R8kTQDaigvJzMyaTS3DdpwGzJTUfdf2eODvC4vIzMyaTm7LIiJuBnYH/iG99oiIW/Lmk3ShpBckPdpDvSSdK2mupEck7V1SN13Sk+k1vfZfx8zMipCbLCRtDXwR+HREzAF2kfT+GpY9AzisSv1UsiS0O9nwIT9N69sWOINsWJH9gDMkjaphfWZmVpBazln8ElgNvD197gC+lTdTRNwFVHtI0hHAxZGZDYyUtAPwXuC2iFgSES+SPfu7WtIxM7OC1ZIsJkbE94A1ABGxClAd1r0T8GzJ545U1lO5mZk1SC3JYrWkIaQb9CRNBP5Sh3VXSjhRpXzDBUgnSWqX1N7Z2VmHkMzMrJJaksUZwM3AzpIuA24HTq/DujuAnUs+jwUWVCnfQERcEBGTI2JyW5uv5jUzK0rVZCFJwB/J7uI+DrgCmBwRM+uw7uuAY9NVUQcAyyJiIXAL8B5Jo9KJ7fekMjMza5Cq91lEREj6r4jYB7hhYxYs6QrgYGCMpA6yFkpLWu75wI3A4cBc4GXg+FS3RNI3gQfSos6KiGonys3MrGC13JQ3W9K+EfFA/qSviYhpOfUBnNpD3YXAhRuzPjMzK04tyeKvgZMlPQ2sJDsBHRGxV5GBmZlZ86glWUwtPAozM2tq1Z5n0QqcDOwG/A74RUSs7a3AzMyseVS7GuoiYDJZopgKfL9XIjIzs6ZTrRtqz4h4C4CkXwD3905IZmbWbKq1LNZ0v3H3k5nZwFatZfFWScvTewFD0ufuq6GGFx6dmZk1hWqPVR3Um4GYmVnzqmVsKDMzG+CcLMzMLJeThZmZ5XKyMDOzXE4WZmaWy8nCzMxyOVmYmVkuJwszM8vlZGFmZrmcLMzMLJeThZmZ5XKyMDOzXE4WZmaWy8nCzMxyOVmYmVkuJwszM8vlZGFmZrmcLMzMLJeThZmZ5XKyMDOzXE4WZmaWy8nCzMxyOVmYmVkuJwszM8vlZGFmZrmcLMzMLFehyULSYZIelzRX0pcr1I+TdLukRyTNlDS2pO67kh5Nr6OLjNPMzKorLFlIGgT8GJgK7AlMk7Rn2WTnABdHxF7AWcDZad73AXsDk4D9gS9KGl5UrGZmVl2RLYv9gLkR8VRErAauBI4om2ZP4Pb0/o6S+j2BOyNibUSsBOYAhxUYq5mZVVFkstgJeLbkc0cqKzUH+Eh6fyQwTNLoVD5V0taSxgB/DexcvgJJJ0lql9Te2dlZ91/AzMwyRSYLVSiLss9fAN4p6SHgncBzwNqIuBW4EfgNcAVwL7B2g4VFXBARkyNicltbW12DNzOz1xSZLDpYvzUwFlhQOkFELIiID0fE24CvprJl6ee3I2JSRBxKlnieLDBWMzOroshk8QCwu6QJkrYEjgGuK51A0hhJ3TF8BbgwlQ9K3VFI2gvYC7i1wFjNzKyKwUUtOCLWSvo0cAswCLgwIh6TdBbQHhHXAQcDZ0sK4C7g1DR7C3C3JIDlwMcjYoNuKDMz6x2KKD+N0DdNnjw52tvbGx2GmVmfIunBiJicN53v4DYzs1xOFmZmlsvJwszMcjlZmJlZLicLMzPL5WRhZma5nCzMzCyXk4WZmeVysjAzs1xOFmZmlsvJwszMcjlZmJlZLicLMzPL5WRhZma5nCzMzCxXYQ8/6ivmd65g1rzFLFrWxXYjWpkycTQT2oY2Oiwzs6YyoFsW8ztXcFV7Byu71rL9iFZWdq3lqvYO5neuaHRoZmZNZUAni1nzFjNySAvDh7SwhcTwIS2MHNLCrHmLGx2amVlTGdDJYtGyLoa2rt8TN7R1MIuWdTUoIjOz5jSgk8V2I1pZ0bV2vbIVXWvZbkRrgyIyM2tOAzpZTJk4mqWr1rB81RpejWD5qjUsXbWGKRNHNzo0M7OmMqCTxYS2oRw9eSzbtA7m+WVdbNM6mKMnj/XVUGZmZQb8pbMT2oY6OZiZ5RjQLQszM6uNk4WZmeVysjAzs1xOFmZmlsvJwszMcikiGh1DXUjqBJ7JmWwM8OdeCKde+lq80PdidrzF62sxD7R4x0VEW95E/SZZ1EJSe0RMbnQctepr8ULfi9nxFq+vxex4K3M3lJmZ5XKyMDOzXAMtWVzQ6AA2Ul+LF/pezI63eH0tZsdbwYA6Z2FmZptmoLUszMxsEzhZmJlZrj6bLCQdJulxSXMlfblC/ThJt0t6RNJMSWNL6r4n6TFJf5B0riSl8plpmQ+n1+uaKObvSno0vY4uKZ8g6T5JT0q6StKWTR7vDEnzS7bxpDrGe6GkFyQ92kO90t97bop575K66WkbPilpekn5PpJ+l+ZZt680cbxF78ObE/PNkpZKur5sniL34SLibbp9WNIkSfcq+157pJDviIjocy9gEDAP2BXYEpgD7Fk2zdXA9PT+XcAl6f2BwD1pGYOAe4GDU91MYHITxvw+4DayIeW3AdqB4anuV8Ax6f35wClNHu8M4KiCtvE7gL2BR3uoPxy4CRBwAHBfKt8WeCr9HJXej0p19wNvT/PcBExt8ngL24c3J+ZU927gA8D1ZfMUsg8XGG8z7sOvB3ZP73cEFgIj67l9+2rLYj9gbkQ8FRGrgSuBI8qm2RO4Pb2/o6Q+gFayL8CtgBZgUeERb17MewJ3RsTaiFhJ9sV9WDrKfRdwTZruIuBDzRpvneLqUUTcBSypMskRwMWRmQ2MlLQD8F7gtohYEhEvkiW6w1Ld8Ii4N7L/tIup3/ate7z1iquazYiZiLgdeKl04oL34brHW7RNjTcinoiIJ9MyFgAvAG313L59NVnsBDxb8rkjlZWaA3wkvT8SGCZpdETcS/bFtjC9bomIP5TM98vUtPx6PbscNifmVD5V0taSxgB/DewMjAaWRsTaKstspni7fTs1lX8oaas6xVuLnn6nauUdFcp7y8bG262ofbgWtew3pYrch2uxsfF2a7Z9eB1J+5EdDM+jjtu3ryaLSv8A5dcAfwF4p6SHgHcCzwFrJe0GvBEYS7bR3iXpHWmev42ItwAHpdcnmiHmiLgVuBH4DXAFWdfZ2hqX2UzxAnwFeAOwL1k3ypfqFG8tevqdNra8t2xKXEXuw7XY2G3WrNu4mmbch7PKrFV0CXB8RLyaN/3G6KvJooP1j1THAgtKJ4iIBRHx4Yh4G/DVVLaM7Ah4dkSsiIgVZP1/B6T659LPl4DLybpimiFmIuLbETEpIg4l2wGeJBs8bKSkwT0ts8niJSIWpib0X4BfUt9tnKen36la+dgK5b1lY+Mteh+uRe5+U6bIfbgWGxtvs+7DSBoO3AB8LXVRQR23b19NFg8Au6ez/FsCxwDXlU4gaYyk7t/vK8CF6f2fyI6GB0tqITsi/kP6PCbN2wK8H6h4RUJvxyxpUOreQdJewF7Arakf/Q7gqDTPdOC/mzXe9HmH9FNkfaf13MZ5rgOOTVeUHAAsi4iFwC3AeySNkjQKeA9Z9+RC4CVJB6R4j6V+27fu8fbCPrw5MVdU8D5ci42KF5pzH07/o9eSnc+4unvium7fTTkr3gwvsqsCniDrl/tqKjsL+GB6fxTZ0ewTwM+BrVL5IOAyyBinAAAD30lEQVRnwB+A3wM/SOXbAA8CjwCPAT8CBjVJzK0p1t8Ds4FJJcvcleyKnblkVydt1eTx/h/wO7J/sEuBoXWM9wqy81BryI7ATgROBk5O9QJ+nH6f31Fy1RBwQtqGc8ma8N3lk1Os84B/J4160Izx9tI+vDkx3w10AqvSvO/thX24iHibbh8GPp7mebjkName29fDfZiZWa6+2g1lZma9yMnCzMxyOVmYmVkuJwszM8vlZGFm1kcpZ+DBsml/qNcGP3xC0tKNWZeThQ14ko6UFJLeUMO0x0nacTPWdbDKRjE12wwzqHFcsIg4LbIbZScB5wG/3pgVOVmYwTRgFtmNh3mOIxvV06zhosLAg5ImKhte/UFJd/dwEDSN7J6OmjlZ2IAmaSjwV2Q3Px1TVne6smdZzJH0L5KOIrtJ77LUlB8i6emSu6YnS5qZ3u8n6TeSHko/98iJ4zhJ/53+yR+XdEZJ3ef12rNBPpfKxkv6o6SLlA1od42kreu5bazPugD4TETsQzZ+209KKyWNAyaQ3VxYs8H5k5j1ax8Cbo6IJyQtkbR3RPxW0tRUt39EvCxp24hYIunTwBcioh1APQ/q+kfgHRGxVtIhwHd4bYTenuwHvBl4GXhA0g1kg74dD+xPdvfufZLuBF4E9gBOjIh7JF0IfAo4Z5O3hPV56eDnQODqkn2zfFTcY4BrIuKVjVm2k4UNdNOAf0vvr0yffwscAvwyIl4GiIhqzxioZARwkaTdyb7wW2qY57aIWAwg6dfAlDTvtZE9F6S7/CCyMYKejYh70ryXAp/FyWKg24JsSPJqT+87Bjh1UxZsNiClwQ7fBfxc0tPAF4Gj0wBxorahnNfy2v9Ra0n5N4E7IuLNZE9bay2fsYLy9fU0HHm16W0Ai4jlwHxJH4V1j2F9a3d96g4dRfbYgI3iZGED2VFko3SOi4jxEbEzMJ/siP5W4ITu8wCStk3zvAQMK1nG08A+6X1pN9MIsud7QHZSvBaHStpW0hCyLrB7gLuADyl7kNQ2ZEPs352m30XS29P77pP0NoBI6n5ezB6SOiSdCPwtcKKkOWQDSpY+4XIacGVswqCA7oaygWwa8C9lZf8JfCwiTpE0CWiXtJrsYU7/RHap4vmSVpE9m/tM4BeS/gm4r2Q53yPrhvo8tZ9InEX24JrdgMtLzovMIBs1FODnEfGQpPFkIydPl/QzstF/f1rjeqyfiIhpPVRVvJw2Iv55U9flUWfNmoCk48iGm/50jdOPB65P3VxmhXM3lJmZ5XLLwszMcrllYWZmuZwszMwsl5OFmZnlcrIwM7NcThZmZpbr/wGUExMc+TLjHAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.scatter(multi_y_test, multi_y_predict, alpha=0.4)\n",
    "plt.xlabel(\"Actual pop\")\n",
    "plt.ylabel(\"Predicted # of tot\")\n",
    "plt.title(\"MULTIPLE LINEAR REGRESSION Seoul Pop \")\n",
    "plt.legend(loc= 'upper center')\n",
    "\n",
    "plt.show()\n",
    "# plt.plot(x,line_fitter.predict(x.values.reshape(-1,1)))\n",
    "# plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAEDCAYAAADZUdTgAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAG5tJREFUeJzt3XFwXWeZ3/HvL5YSKVYsG1koITK28WYBL01DejEBTJJlW4jpDi6E3djd3UCWjmfbpNPtTNpJhnaYusNkWbKz3cymBJe6NOyMY6ALdVvApN6wwVsCud7gEBOcyHHAihPnroNl5EhEIk//uK/C9c2VdWUf6dyr8/vM3NE57/vec57XSp579N73nFcRgZmZFcN5eQdgZmbzx0nfzKxAnPTNzArESd/MrECc9M3MCsRJ38ysQFo26UvaLul5SY810fZPJX0/vZ6QdGI+YjQzazdq1Xn6kq4GRoF7I+Its3jfvwTeGhG/P2fBmZm1qZa90o+IB4EXasskrZH0DUn7JH1b0psavHUzsGNegjQzazMdeQcwS9uAP4iIJyW9HfjPwHumKiWtBFYDf5VTfGZmLa1tkr6kHuCdwJckTRVfUNdsE/DliPjFfMZmZtYu2ibpUx2KOhERV5yhzSbg5nmKx8ys7bTsmH69iDgJHJb0WwCq+vtT9ZLeCCwDvpNTiGZmLa9lk76kHVQT+BslDUv6GPA7wMck7QcOABtr3rIZuC9adTqSmVkLaNkpm2Zmlr2WvdI3M7PstdwXucuXL49Vq1blHYaZWVvZt2/f30VE/0ztWi7pr1q1inK5nHcYZmZtRdKPm2nn4R0zswJx0jczKxAnfTOzAnHSNzMrECd9M7MCabnZO2ZmZ+NwZZS9h45zbGScgd4u1q/pY3V/T95htRxf6ZtZ2ztcGWVneZhT45Nc3NvFqfFJdpaHOVwZzTu0luOkb2Ztb++h4yzt7mRJdyfnSSzp7mRpdyd7Dx3PO7SWM2PSn2mt2vS0y7skDUl6VNKVqXxlWuHq+5IOSPqDrIM3MwM4NjJOT9fpo9U9XR0cGxnPKaLW1cyV/ueB685QvwG4LL22AJ9J5c8C70zPv387cJuk1519qGZmjQ30djE6Pnla2ej4JAO9XTlF1LpmTPqN1qqts5Hq4uUREQ8BSyVdEhEvRcTPU5sLmjmXmdnZWL+mjxNjE5wcm+DlCE6OTXBibIL1a/ryDq3lZJGILwWO1OwPpzIkrZD0aKr/VEQcbXQASVsklSWVK5VKBiGZWZGs7u/hhtIgi7s6eG5knMVdHdxQGvTsnQaymLKpBmUBEBFHgMvTsM5XJX05Io69qnHENqqLnlMqlfyAfzObtdX9PU7yTcjiSn8YWFGzPwicdkWfrvAPAO/O4HxmZnaWskj6u4Ab0yyeq4CRiHhW0qCkbgBJy4B3AQczOJ+ZmZ2lGYd30lq11wLLJQ0DnwA6ASLiHuBrwPuBIeBF4Kb01jcDfyIpqA4B3RkRP8i6A2Zm1rwZk35EbJ6hPoCbG5TfD1x+9qGZmVnWPI3SzKxAnPTNzArESd/MrECc9M3MCsTP0zezludn5WfHV/pm1tL8rPxsOembWUvzs/Kz5aRvZi3Nz8rPlpO+mbU0Pys/W076ZtbS/Kz8bDnpm1lL87Pys+Upm2bW8vys/Oz4St/MrECc9M3MCsTDO2aWC99l+0vz+W/hK30zm3e+y/aX5vvfYsakL2m7pOclPTZNvSTdJWlI0qOSrkzlV0j6jqQDqfyGrIM3s/aUx122hyujfOGhH3Pn7oN84aEft8wHzHz/WzQzvPN54M+Be6ep3wBcll5vBz6Tfr4I3BgRT0p6HbBP0u6IOHHOUZtZWzs2Ms7FdTdX9aQpmVmoHy5Ztaybv3nqBZZ2d3JxutlrZ3m4JaZ+zvW/Rb1mlkt8UNKqMzTZCNyblk18SNJSSZdExBM1xzgq6XmgH3DSNyu4qbtsl3R3vlKW1V22U8MltQl+297DvHngolfON/Vz76HjuST92g+lp4+f4qWJX7CqJo65vOM4izH9S4EjNfvDqewVktYB5wOHGh1A0hZJZUnlSqWSQUhm1srm8i7bRsMlky+/zLMnT79yzuv5PfVj+K/r7eJ7P/4pT1dG5+WO4yySvhqUxSuV0iXAF4CbIuLlRgeIiG0RUYqIUn9/fwYhmVkrm8u7bBs9oG15zwUcP/XSaWV5Pb+n/kNp1fIe1q1axtGT4/Nyx3EWUzaHgRU1+4PAUQBJS4D/A/y7iHgog3OZ2QIxV3fZDvR28ZPjp3juZz/n5NgES7o7ubDzPH4qcXJsgp6uDkbHJzkxNsH7fm0g8/PPpNEY/uv7FnN+xyJufd8b5/z8WST9XcAtku6j+gXuSEQ8K+l84CtUx/u/lMF5zMxmtGpZN/9j33C6mu7g5IsT/GRsgt952yDjL8Nz6cvd9/3aQC7j+Y0+lC6+6AJW9C2el/PPmPQl7QCuBZZLGgY+AXQCRMQ9wNeA9wNDVGfs3JTe+tvA1UCfpI+mso9GxPczjN/M7DRP/3SMdSuX8dzPfs7PxqtJ9Vdf28P4y/B7V62clxjOdLNVww+l4y/y7l9ZPi+xNTN7Z/MM9QHc3KD8L4C/OPvQzKxosrgz9djIOK9fvvi02TAvR8zZFMh6jWYP1U4Pne5D6emfjvHueYjPj2Ews5YwU7Js1lxOB21G7Re18OrpoXl/KPkxDGbWErK6MzXvRVdmWt4x75XAnPTNrCVktRZu3ouuzJTU8/5Q8vCOmbWELIdl8lx0Zf2aPnaWhwEaTg+d+lDae+h4LjOJnPTNrCXMlCzbRTNJPc8PJSd9s4Jrlefa530FnKVWXt7RSd+swLKaMZOVVk6WC4W/yDUrsDyea2/5ctI3K7CsZsxY+3DSNyuwvOeM2/xz0jcrsLznjNv8c9I3K7C8b2Sy+efZO2YF5xkzxeIrfTOzAnHSNzMrkGYWUdkO/CbwfES8pUG9gD+jupDKi1QXSvnbVPcN4Cpgb0T8ZpaBm9m5aZU7cW1+NXOl/3ngujPUbwAuS68twGdq6j4N/N7ZBmdmc2PqTtxT45Nc3NvFqXQn7uHKaN6h2RybMelHxIPAC2dospHqOriRFj9fKumS9N49wM8yidTMMuM7cYsrizH9S4EjNfvDqczMWpTvxC2uLJK+GpTFrA4gbZFUllSuVCoZhGRmZ+I7cYsri6Q/DKyo2R8Ejs7mABGxLSJKEVHq7+/PICQzOxPfiVtcWST9XcCNqroKGImIZzM4rpnNEd+JW1zNTNncAVwLLJc0DHwC6ASIiHuAr1GdrjlEdcrmTTXv/TbwJqAnvfdjEbE74z6Y2VnwnbjFNGPSj4jNM9QHcPM0de8+y7jMzGwO+I5cM7MCcdI3MysQJ30zswJx0jczKxAnfTOzAnHSNzMrECd9M7MCcdI3MysQJ30zswJx0jczKxAnfTOzApnx2TtmtnB4XVzzlb5ZQXhdXAMnfbPC8Lq4Bk76ZoXhdXENnPTNCsPr4ho0kfQlbZf0vKTHpqmXpLskDUl6VNKVNXUfkfRken0ky8DNbHa8Lq5Bc1f6nweuO0P9BuCy9NoCfAZA0muoLq34dmAd8AlJy84lWDM7e14X16C55RIflLTqDE02AvemZRMfkrRU0iVU19W9PyJeAJB0P9UPjx3nGrSZnR2vi2tZjOlfChyp2R9OZdOVv4qkLZLKksqVSiWDkMzMrJEskr4alMUZyl9dGLEtIkoRUerv788gJDMzaySLpD8MrKjZHwSOnqHczMxykkXS3wXcmGbxXAWMRMSzwG7gvZKWpS9w35vKzMwsJzN+kStpB9UvZZdLGqY6I6cTICLuAb4GvB8YAl4Ebkp1L0j6j8DD6VBbp77UNTOzfDQze2fzDPUB3DxN3XZg+9mFZmZmWfMduWZmBeKkb2ZWIE76ZmYF4qRvZlYgTvpmZgXipG9mViBO+mZmBeKkb2ZWIDPenGVmzTlcGWXvoeMcGxlnoLeL9Wv6/Bhjazm+0jfLwOHKKDvLw5wan+Ti3i5OjU+yszzM4cpo3qGZncZJ3ywDew8dZ2l3J0u6OzlPYkl3J0u7O9l76HjeoZmdxknfLAPHRsbp6Tp9tLSnq4NjI+M5RWTWmJO+WQYGersYHZ88rWx0fJKB3q6cIjJrzEnfLAPr1/RxYmyCk2MTvBzBybEJToxNsH5NX96hmZ3GSd8sA6v7e7ihNMjirg6eGxlncVcHN5QGPXvHWo6nbJplZHV/j5O8tbymrvQlXSfpoKQhSbc1qF8paY+kRyV9S9JgTd2nJD2WXjdkGbyZmc3OjElf0iLgbmADsBbYLGltXbM7gXsj4nJgK3BHeu8/Bq4ErgDeDvwbSUuyC9/MzGajmSv9dcBQRDwVES8B9wEb69qsBfak7Qdq6tcCfx0RkxFxCtgPXHfuYZuZ2dloJulfChyp2R9OZbX2A9en7Q8CF0nqS+UbJF0oaTnw68CK+hNI2iKpLKlcqVRm2wczM2tSM0lfDcqibv9W4BpJjwDXAM8AkxHxTeBrwP8DdgDfASbr3ktEbIuIUkSU+vv7ZxO/mZnNQjNJf5jTr84HgaO1DSLiaER8KCLeCnw8lY2kn5+MiCsi4h9R/QB5MpPIzcxs1ppJ+g8Dl0laLel8YBOwq7aBpOWSpo51O7A9lS9KwzxIuhy4HPhmVsGbmdnszDhPPyImJd0C7AYWAdsj4oCkrUA5InYB1wJ3SArgQeDm9PZO4NuSAE4CvxsRrxreMTOz+aGI+uH5fJVKpSiXy3mHYWbWViTti4jSTO38GAYzswLxYxhs3nmFKbP8+Erf5pVXmDLLl5O+zSuvMGWWLyd9m1deYcosX076Nq+8wpRZvpz0bV55hSmzfDnp27zyClNm+fKUTZt3XmHKLD++0jczKxAnfTOzAnHSNzMrECd9M7MCcdI3MysQJ30zswJpKulLuk7SQUlDkm5rUL9S0h5Jj0r6lqTBmro/lnRA0uOS7lJaUcXMzObfjElf0iLgbmADsBbYLGltXbM7gXsj4nJgK3BHeu87gXdRXSbxLcDbqC6cbmZmOWjmSn8dMBQRT0XES8B9wMa6NmuBPWn7gZr6ALqA84ELqC6feOxcgzYzs7PTTNK/FDhSsz+cymrtB65P2x8ELpLUFxHfofoh8Gx67Y6Ix+tPIGmLpLKkcqVSmW0fzMysSc0k/UZj8PUL694KXCPpEarDN88Ak5J+BXgzMEj1g+I9kq5+1cEitkVEKSJK/f39s+qAmZk1r5ln7wwDK2r2B4GjtQ0i4ijwIQBJPcD1ETEiaQvwUESMprqvA1cBD2YQu5mZzVIzV/oPA5dJWi3pfGATsKu2gaTlkqaOdTuwPW3/hOpfAB2SOqn+FfCq4R0zM5sfMyb9iJgEbgF2U03YX4yIA5K2SvpAanYtcFDSE8AA8MlU/mXgEPADquP++yPif2XbBTMza5Yi6ofn81UqlaJcLucdhplZW5G0LyJKM7Xz8/QXoMOVUfYeOs6xkXEGertYv6bPz683M8CPYVhwDldG2Vke5tT4JBf3dnFqfJKd5WEOV0bzDs3MWoCT/gKz99BxlnZ3sqS7k/MklnR3srS7k72Hjucdmpm1ACf9BebYyDg9XaeP2vV0dXBsZDyniMyslTjpLzADvV2Mjk+eVjY6PslAb1dOEZlZK3HSX2DWr+njxNgEJ8cmeDmCk2MTnBibYP2avrxDM7MW4KS/wKzu7+GG0iCLuzp4bmScxV0d3FAa9OwdMwM8ZXNBWt3f4yRvZg35St/MrEB8pZ8T30BlZnnwlX4OfAOVmeXFST8HvoHKzPLipJ8D30BlZnlx0s+Bb6Ays7w46efAN1CZWV6c9HPgG6jMLC9NTdmUdB3wZ8Ai4HMR8Ud19SupLpHYD7wA/G5EDEv6deBPa5q+CdgUEV/NIvh25huozCwPM17pS1oE3A1sANYCmyWtrWt2J3BvRFwObAXuAIiIByLiioi4AngP8CLwzQzjNzOzWWhmeGcdMBQRT0XES8B9wMa6NmuBPWn7gQb1AB8Gvh4RL55tsGZmdm6aSfqXAkdq9odTWa39wPVp+4PARZLqv5XcBOxodAJJWySVJZUrlUoTIZmZ2dloJumrQVn9auq3AtdIegS4BngGeGVOoqRLgL8H7G50gojYFhGliCj19/c3FbiZmc1eM1/kDgMravYHgaO1DSLiKPAhAEk9wPURMVLT5LeBr0TExLmFa2Zm56KZK/2HgcskrZZ0PtVhml21DSQtlzR1rNupzuSptZlphnbMzGz+zJj0I2ISuIXq0MzjwBcj4oCkrZI+kJpdCxyU9AQwAHxy6v2SVlH9S+GvM43czMxmTRH1w/P5KpVKUS6X5+z4fqSxmS1EkvZFRGmmdoW6I9ePNDazoitU0vcjjc2s6AqV9P1IYzMrukIlfT/S2MyKrlBJ3480NrOiK1TS9yONzazomnq08kLiRxqbWZEV6krfzKzonPTNzArESd/MrECc9M3MCsRJ38ysQJz0zcwKxEnfzKxAnPTNzAqkqaQv6TpJByUNSbqtQf1KSXskPSrpW5IGa+peL+mbkh6X9MO0qIqZmeVgxqQvaRFwN7ABWAtslrS2rtmdwL0RcTmwFbijpu5e4NMR8WZgHfB8FoGbmdnsNXOlvw4YioinIuIl4D5gY12btcCetP3AVH36cOiIiPsBImI0Il7MJHIzM5u1ZpL+pcCRmv3hVFZrP3B92v4gcJGkPuBXgROS/lLSI5I+nf5yMDOzHDST9NWgrH5h3VuBayQ9AlwDPANMUn2g27tT/duANwAffdUJpC2SypLKlUql+ejNzGxWmkn6w8CKmv1B4Ghtg4g4GhEfioi3Ah9PZSPpvY+koaFJ4KvAlfUniIhtEVGKiFJ/f/9ZdsXMzGbSTNJ/GLhM0mpJ5wObgF21DSQtlzR1rNuB7TXvXSZpKpO/B/jhuYdtZmZnY8akn67QbwF2A48DX4yIA5K2SvpAanYtcFDSE8AA8Mn03l9QHdrZI+kHVIeK/kvmvTAzs6Yoon54Pl+lUinK5XLeYZiZtRVJ+yKiNFM735FrZlYgTvpmZgXipG9mViBO+mZmBeKkb2ZWIE76ZmYF4qRvZlYgTvpmZgXipG9mViBO+mZmBeKkb2ZWIE76ZmYF4qRvZlYgTvpmZgXSkXcAWTlcGWXvoeMcGxlnoLeL9Wv6WN3fk3dYZmYtZUFc6R+ujLKzPMyp8Uku7u3i1PgkO8vDHK6M5h2amVlLaSrpS7pO0kFJQ5Jua1C/UtIeSY9K+pakwZq6X0j6fnrtqn9vFvYeOs7S7k6WdHdynsSS7k6Wdney99DxuTidmVnbmjHpS1oE3A1sANYCmyWtrWt2J3BvRFwObAXuqKkbi4gr0usDzIFjI+P0dJ0+UtXT1cGxkfG5OJ2ZWdtq5kp/HTAUEU9FxEvAfcDGujZrgT1p+4EG9XNqoLeL0fHJ08pGxycZ6O2azzDMzFpeM0n/UuBIzf5wKqu1H7g+bX8QuEhSX9rvklSW9JCkf9LoBJK2pDblSqUyi/Cr1q/p48TYBCfHJng5gpNjE5wYm2D9mr6Z32xmViDNJH01KKtfTf1W4BpJjwDXAM8AU5fer0+L9f5T4D9JWvOqg0Vsi4hSRJT6+/ubjz5Z3d/DDaVBFnd18NzIOIu7OrihNOjZO2ZmdZqZsjkMrKjZHwSO1jaIiKPAhwAk9QDXR8RITR0R8ZSkbwFvBQ6dc+R1Vvf3OMmbmc2gmSv9h4HLJK2WdD6wCThtFo6k5ZKmjnU7sD2VL5N0wVQb4F3AD7MK3szMZmfGpB8Rk8AtwG7gceCLEXFA0lZJU7NxrgUOSnoCGAA+mcrfDJQl7af6Be8fRYSTvplZThRRPzyfr1KpFOVyOe8wzMzaiqR96fvTM1oQd+SamVlznPTNzAqk5YZ3JFWAH+cdxxksB/4u7yDmgPvVPhZin8D9OlcrI2LGOe8tl/RbnaRyM+Nm7cb9ah8LsU/gfs0XD++YmRWIk76ZWYE46c/etrwDmCPuV/tYiH0C92teeEzfzKxAfKVvZlYgTvpmZgVS2KQv6WlJP0jLOJZT2Wsk3S/pyfRzWSqXpLvScpGPSrqy5jgfSe2flPSRmvJ/kI4/lN7b6BHVc9GvpZK+LOlHkh6X9I5275ekN9Ysufl9SScl/eEC6Ne/lnRA0mOSdkjqSg82/G6Kb2d6yCGSLkj7Q6l+Vc1xbk/lByW9r6b8jMuczmG//lXq0wFJf5jK2vJ3JWm7pOclPVZTNud9me4cmYiIQr6Ap4HldWV/DNyWtm8DPpW23w98neraAlcB303lrwGeSj+Xpe1lqe57wDvSe74ObJinfv134J+l7fOBpQuhXzX9WwQ8B6xs535RXYjoMNCd9r8IfDT93JTK7gH+edr+F8A9aXsTsDNtr6W6iNEFwGqqjy1flF6HgDek/w72A2vn4ffzFuAx4EKqj27/v8Bl7fq7Aq4GrgQeqymb875Md45M+jTX/xG06ovGSf8gcEnavgQ4mLY/C2yubwdsBj5bU/7ZVHYJ8KOa8tPazWGflqREooXUr7q+vBf4m3bvF79cke41VJPj/wbeR/XOzY7U5h3A7rS9G3hH2u5I7UT1Uea31xx3d3rfK+9N5ae1m8N+/RbwuZr9fw/82zb/Xa3i9KQ/532Z7hxZvAo7vEN19a9vStonaUsqG4iIZwHSz9em8umWjDxT+XCD8rn2BqAC/DdJj0j6nKTFtH+/am0CdqTttu1XRDwD3An8BHgWGAH2ASei+jjz+jheiT3VjwB9zL6vc+0x4GpJfZIupHr1u4I2/l01MB99me4c56zISf9dEXElsAG4WdLVZ2g73ZKRsy2fax1U/xT9TES8FThF9U/D6bRLvwBI49sfAL40U9MGZS3VrzRGu5HqkMzrgMVU/1ucLo6W7xNARDwOfAq4H/gG1WGlyTO8pS361aS26Ethk378chnH54GvAOuAY5IuAUg/n0/Np1sy8kzlgw3K59owMBwR3037X6b6IdDu/ZqyAfjbiDiW9tu5X/8QOBwRlYiYAP4SeCewVNLUMqa1cbwSe6rvBV5g9n2dcxHxXyPiyoi4OsX4JO39u6o3H32Z7hznrJBJX9JiSRdNbVMdJ36M6jKQU9+sfwT4n2l7F3Bj+nb+KmAk/cm1G3ivqstCLkvH2Z3qfibpqvRt/I01x5ozEfEccETSG1PRb1BdnrKt+1VjM78c2pmKv1379RPgKkkXpnNO/a4eAD48TZ+m+vph4K+iOuC7C9ik6uye1VS/NP0eTSxzOlckvTb9fD3VtbN30N6/q3rz0ZfpznHu5vqLnVZ8UR373p9eB4CPp/I+YA/VK5M9wGtSuYC7qc6G+AFQqjnW7wND6XVTTXmJ6gfJIeDPqftydQ77dgVQBh4Fvkp1tsBC6NeFwHGgt6asrfsF/AfgR+m8X6A6A+cNVJP2ENVhrAtS2660P5Tq31BznI+nuA9SM5OF6nj6E6nu4/P4/9e3qX6A7Qd+o51/V1Q/sJ4FJqhemX9sPvoy3TmyePkxDGZmBVLI4R0zs6Jy0jczKxAnfTOzAnHSNzMrECd9M7MCcdI3MysQJ30zswL5/35uIdIj7eBaAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.scatter(seoul_born_death_mrg[['# of born']], seoul_born_death_mrg[['tot_pop']], alpha=0.4)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAEDCAYAAADZUdTgAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAHH9JREFUeJzt3X9wXeWd3/H3B1u2FAv/iK01rOU1xssmcVJq2BvCJubHstONTTJ4E7aL3W7zY9Nx00Bn2g5tYdJpUncY8oOd7DJLSZ3UmzqZgkmmobRNljBeWOJZSLgOxsGhBtlO1sJgLqaWI2NhKf72j/vIXF+udK+kK90jnc9r5o7PeZ7nnvvVI/mro+c85zyKCMzMLB/Oa3UAZmY2dZz0zcxyxEnfzCxHnPTNzHLESd/MLEec9M3MciSzSV/SNkmvSHq2gbZfkbQnvZ6XdHwqYjQzm26U1Xn6kq4G+oHtEfGeMbzvXwCXRcSfTFpwZmbTVGbP9CPiceC1yjJJqyT9laTdkn4o6Z013roJuG9KgjQzm2ZmtzqAMdoKfDoiXpD0PuA/A9cNV0paAawE/rpF8ZmZZdq0SfqSOoH3A9+WNFw8t6rZRuA7EfGrqYzNzGy6mDZJn/JQ1PGIWDNKm43AzVMUj5nZtJPZMf1qEXECOCTpHwKo7O8P10t6B7AIeKJFIZqZZV5mk76k+ygn8HdI6pX0KeAfA5+S9AywD9hQ8ZZNwP2R1elIZmYZkNkpm2Zm1nyZPdM3M7Pmy9yF3CVLlsRFF13U6jDMzKaV3bt3vxoRXfXaZS7pX3TRRRSLxVaHYWY2rUj6RSPtPLxjZpYjTvpmZjnipG9mliNO+mZmOeKkb2aWI5mbvWNmNl6HSv3sOnCMo30DLF3QztpVi1nZ1dnqsDLFZ/pmNiMcKvWzo9jLyYEhLljQzsmBIXYUezlU6m91aJnipG9mM8KuA8dY2NHG/I42zpOY39HGwo42dh041urQMqVu0q+3Vm162uXdknok7ZV0eSpfkVa42iNpn6RPNzt4M7NhR/sG6Gw/d8S6s302R/sGWhRRNjVypv8NYN0o9euBS9JrM3BvKn8JeH96/v37gNsk/fr4QzUzG9nSBe30DwydU9Y/MMTSBe0tiiib6ib9WmvVVtlAefHyiIgngYWSLoyI0xHxRmozt5HPMjMbr7WrFnP81CAnTg1yJoITpwY5fmqQtasWtzq0TGlGIl4GHK7Y701lSFouaW+q/2JEHKl1AEmbJRUlFUulUhNCMrO8WdnVyU2Fbua1z+blvgHmtc/mpkK3Z+9UacaUTdUoC4CIOAxcmoZ1HpT0nYg4+pbGEVspL3pOoVDwA/7NbFxWdnU6ydfRjDP9XmB5xX43cM4ZfTrD3wdc1YTPMzOzcWpG0n8I+FiaxXMl0BcRL0nqltQBIGkR8AFgfxM+z8zMxqnu8E5aq/ZaYImkXuBzQBtARHwV+B5wPdADvA58Mr31XcCfSgrKQ0B3RcRPm/0FmJlZ4+om/YjYVKc+gJtrlD8CXDr+0MzMrNk8jdLMLEec9M3McsRJ38wsR5z0zcxyxM/TN7Pc8PP2faZvZjnh5+2XOembWS74eftlTvpmlgt+3n6Zk76Z5YKft1/mpG9mueDn7Zc56ZtZLvh5+2WesmlmueHn7ftM38wsV5z0zcxyxMM7ZpZZrbiDdqbfteszfTPLpFbcQZuHu3brJn1J2yS9IunZEeol6W5JPZL2Sro8la+R9ISkfan8pmYHb2Yz12TeQXuo1M83n/wFdz28n28++YuzST0Pd+02MrzzDeAvgO0j1K8HLkmv9wH3pn9fBz4WES9I+nVgt6SHI+L4hKM2sxnvaN8AF1TdONWZpltOxKFSP//l8YO8dvI0p4fO8PzRX7LvxT7+2dUXT9pnZkkjyyU+LumiUZpsALanZROflLRQ0oUR8XzFMY5IegXoApz0zayu4Tto53e0nS1rxh20D+55kZ+/epIFb5vDgo42BobO8PNXT/Lgnhcn7TPrmcrrCM0Y018GHK7Y701lZ0m6ApgDHKh1AEmbJRUlFUulUhNCMrPpbrLuoN1z+Djz39ZGR9ssJNHRNov5b2tjz+HjLblrd6qvIzQj6atGWZytlC4Evgl8MiLO1DpARGyNiEJEFLq6upoQkplNd5N3B61QVJVEubwVd+1O9XWEZkzZ7AWWV+x3A0cAJM0H/g/w7yPiySZ8lpnlyGTcQXvZ8gU8cfA1JDG37TzeGDzD8VOD/M7Fbx/TZzZrSGaqryM040z/IeBjaRbPlUBfRLwkaQ7wXcrj/d9uwueYmU3YhjXLuLhrHmcIjr9+mjMEF3fNY8OaN0elR5rdU1nfrCGZqX76ZyNTNu8DngDeIalX0qckfVrSp1OT7wEHgR7ga8BnUvkfAVcDn5C0J73WNP9LMDMbm187fy5vDP4KEKsvOJ/NV1189iy9kYTezCGZqb6O0MjsnU116gO4uUb5t4BvjT80M7PmOVTq53/ueZHHX3iVJZ1zuWz5QtrbZnP81OA57SoTOnD2310Hjp39xdDMIZnh6wi7Dhzj5TRU9MF3L5206wh+DIOZzXjDZ+8HX+2nq3MuSPzk745TWLHo7Bn6WBJ6s6d2TuXTP/0YBjOb8YbP3k8PnaF9ziw62mbRMWc2PaWTb1kysZEx9um8IIuTvpnNeMPr487vaOONwfLM8fbZ5/HLgcFxJfTpvCCLh3fMbMYbPnv/za55FH9efijAGYK2Wedx/NQgH3z30rNtGx1jn64Lsjjpm9mMt3bVYnYUe1nY0cZvr1jAvpd+yasn3uCq31rCH6xZNmMSeiOc9M0sk5r5PJrKs/f+viGuuqRrxj0nv1FO+maWOcOzbRZ2tHFBGprZUeyd0Lj5TD57HwtfyDWzzMnDc+1bxUnfzDJneLZNpeqplTY+TvpmljlT/TyaPHHSN7PMmc43P2Wdk76ZZc50vvkp6zx7x8wyybNtJofP9M3McsRJ38wsR+oO70jaBnwYeCUi3lOjXsCfA9cDrwOfiIifpLq/Aq4EdkXEh5sZuJlNjWbeGWut18iZ/jeAdaPUrwcuSa/NwL0VdV8G/sl4gzOz1mrmsoCWDXWTfkQ8Drw2SpMNlNfBjbT4+UJJF6b37gR+2ZRIzWzK+c7YmacZY/rLgMMV+72pzMymOd8ZO/M0I+mrRlmM6QDSZklFScVSqdSEkMysGXxn7MzTjKTfCyyv2O8GjozlABGxNSIKEVHo6upqQkhm1gy+M3bmaUbSfwj4mMquBPoi4qUmHNfMWsx3xs48jUzZvA+4FlgiqRf4HNAGEBFfBb5HebpmD+Upm5+seO8PgXcCnem9n4qIh5v8NZjZJPKdsTNL3aQfEZvq1Adw8wh1V40zLjMzmwS+I9fMLEec9M3McsRJ38wsR5z0zcxyxEnfzCxHnPTNzHLESd/MLEec9M3McsRJ38wsR5z0zcxyxEnfzCxH6j57x8zyw+vhznw+0zczwOvh5oWTvpkBXg83L5z0zQzwerh54aRvZoDXw82Luklf0jZJr0h6doR6SbpbUo+kvZIur6j7uKQX0uvjzQzczJrL6+HmQyNn+t8A1o1Svx64JL02A/cCSHo75aUV3wdcAXxO0qKJBGtmk8fr4eZDI8slPi7polGabAC2p2UTn5S0UNKFlNfVfSQiXgOQ9AjlXx73TTRoM5scXg935mvGmP4y4HDFfm8qG6n8LSRtllSUVCyVSk0IyczMamlG0leNshil/K2FEVsjohARha6uriaEZGZmtTQj6fcCyyv2u4Ejo5SbmVmLNCPpPwR8LM3iuRLoi4iXgIeB35e0KF3A/f1UZmZmLVL3Qq6k+yhflF0iqZfyjJw2gIj4KvA94HqgB3gd+GSqe03SfwKeSofaMnxR18zMWqOR2Tub6tQHcPMIdduAbeMLzczMms135JqZ5YiTvplZjjjpm5nliJO+mVmOOOmbmeWIk76ZWY446ZuZ5YiTvplZjtS9OcvMRneo1M+uA8c42jfA0gXtrF212I8ntszymb7ZBBwq9bOj2MvJgSEuWNDOyYEhdhR7OVTqb3VoZjU56ZtNwK4Dx1jY0cb8jjbOk5jf0cbCjjZ2HTjW6tDManLSN5uAo30DdLafO0ra2T6bo30DLYrIbHRO+mYTsHRBO/0DQ+eU9Q8MsXRBe4siMhudk77ZBKxdtZjjpwY5cWqQMxGcODXI8VODrF21uNWhmdXkpG82ASu7Ormp0M289tm83DfAvPbZ3FTo9uwdyyxP2TSboJVdnU7yNm00dKYvaZ2k/ZJ6JN1Wo36FpJ2S9kp6TFJ3Rd0XJT2bXjc1M3gzMxubuklf0izgHmA9sBrYJGl1VbO7gO0RcSmwBbgzvfdDwOXAGuB9wL+RNL954ZuZ2Vg0cqZ/BdATEQcj4jRwP7Chqs1qYGfafrSifjXwNxExFBEngWeAdRMP28zMxqORpL8MOFyx35vKKj0D3Ji2PwKcL2lxKl8v6W2SlgC/Cyyv/gBJmyUVJRVLpdJYvwYzM2tQI0lfNcqiav9W4BpJTwPXAC8CQxHxA+B7wN8C9wFPAENV7yUitkZEISIKXV1dY4nfzMzGoJGk38u5Z+fdwJHKBhFxJCI+GhGXAZ9NZX3p3zsiYk1E/APKv0BeaErkZmY2Zo0k/aeASyStlDQH2Ag8VNlA0hJJw8e6HdiWymelYR4kXQpcCvygWcGbmdnY1J2nHxFDkm4BHgZmAdsiYp+kLUAxIh4CrgXulBTA48DN6e1twA8lAZwA/jgi3jK8Y2ZmU0MR1cPzrVUoFKJYLLY6DDOzaUXS7ogo1GvnxzCYmeWIH8NgM4ZXsDKrz2f6NiN4BSuzxjjp24zgFazMGuOkbzOCV7Aya4yTvs0IXsHKrDFO+jYjeAUrs8Y46duM4BWszBrjKZs2Y3gFK7P6fKZvZpYjTvpmZjnipG9mliNO+mZmOeKkb2aWI076ZmY50lDSl7RO0n5JPZJuq1G/QtJOSXslPSapu6LuS5L2SXpO0t1KK6qYmdnUq5v0Jc0C7gHWA6uBTZJWVzW7C9geEZcCW4A703vfD3yA8jKJ7wHeS3nhdDMza4FGzvSvAHoi4mBEnAbuBzZUtVkN7Ezbj1bUB9AOzAHmUl4+8ehEgzYzs/FpJOkvAw5X7PemskrPADem7Y8A50taHBFPUP4l8FJ6PRwRz1V/gKTNkoqSiqVSaaxfg5mZNaiRpF9rDL56Yd1bgWskPU15+OZFYEjSbwLvArop/6K4TtLVbzlYxNaIKEREoaura0xfgJmZNa6RZ+/0Assr9ruBI5UNIuII8FEASZ3AjRHRJ2kz8GRE9Ke67wNXAo83IXYzMxujRs70nwIukbRS0hxgI/BQZQNJSyQNH+t2YFva/jvKfwHMltRG+a+AtwzvmJnZ1Kib9CNiCLgFeJhywn4gIvZJ2iLphtTsWmC/pOeBpcAdqfw7wAHgp5TH/Z+JiP/V3C/BzMwapYjq4fnWKhQKUSwWWx2Gmdm0Iml3RBTqtfPz9C2TDpX62XXgGEf7Bli6oJ21qxb7WflmTeDHMFjmHCr1s6PYy8mBIS5Y0M7JgSF2FHs5VOpvdWhm056TvmXOrgPHWNjRxvyONs6TmN/RxsKONnYdONbq0MymPSd9y5yjfQN0tp878tjZPpujfQMtishs5nDSt8xZuqCd/oGhc8r6B4ZYuqC9RRGZzRxO+pY5a1ct5vipQU6cGuRMBCdODXL81CBrVy1udWhm056TvmXOyq5Obip0M699Ni/3DTCvfTY3Fbo9e8esCTxl0zJpZVenk7zZJPCZvplZjvhM3+ryjVJmM4fP9G1UvlHKbGZx0rdR+UYps5nFSd9G5RulzGYWJ30blW+UMptZnPRtVL5RymxmcdK3UflGKbOZpaEpm5LWAX8OzAK+HhFfqKpfQXmJxC7gNeCPI6JX0u8CX6lo+k5gY0Q82IzgbWr4RimzmaPumb6kWcA9wHpgNbBJ0uqqZncB2yPiUmALcCdARDwaEWsiYg1wHfA68IMmxm9mZmPQyPDOFUBPRByMiNPA/cCGqjargZ1p+9Ea9QB/CHw/Il4fb7BmZjYxjST9ZcDhiv3eVFbpGeDGtP0R4HxJ1Vf6NgL31foASZslFSUVS6VSAyGZmdl4NJL0VaOsejX1W4FrJD0NXAO8CJyd5yfpQuDvAQ/X+oCI2BoRhYgodHV1NRS4mZmNXSMXcnuB5RX73cCRygYRcQT4KICkTuDGiOiraPJHwHcjYnBi4ZqZ2UQ0cqb/FHCJpJWS5lAepnmosoGkJZKGj3U75Zk8lTYxwtCOmZlNnbpJPyKGgFsoD808BzwQEfskbZF0Q2p2LbBf0vPAUuCO4fdLuojyXwp/09TIzcxszBRRPTzfWoVCIYrFYqvDyDQ/6tjMqknaHRGFeu18R+4040cdm9lEOOlPM37UsZlNhJP+NONHHZvZRDjpTzN+1LGZTYST/jTjRx2b2UQ46U8zftSxmU1EQ49Wtmzxo47NbLx8pm9mliNO+mZmOeKkb2aWI076ZmY54qRvZpYjTvpmZjnipG9mliNO+mZmOdJQ0pe0TtJ+ST2SbqtRv0LSTkl7JT0mqbui7jck/UDSc5J+lhZVMTOzFqib9CXNAu4B1gOrgU2SVlc1uwvYHhGXAluAOyvqtgNfjoh3AVcArzQjcDMzG7tGzvSvAHoi4mBEnAbuBzZUtVkN7Ezbjw7Xp18OsyPiEYCI6I+I15sSuZmZjVkjSX8ZcLhivzeVVXoGuDFtfwQ4X9Ji4LeA45L+h6SnJX05/eVgZmYt0EjSV42y6oV1bwWukfQ0cA3wIjBE+YFuV6X69wIXA594ywdImyUVJRVLpVLj0ZuZ2Zg0kvR7geUV+93AkcoGEXEkIj4aEZcBn01lfem9T6ehoSHgQeDy6g+IiK0RUYiIQldX1zi/FDMzq6eRpP8UcImklZLmABuBhyobSFoiafhYtwPbKt67SNJwJr8O+NnEwzYzs/Gom/TTGfotwMPAc8ADEbFP0hZJN6Rm1wL7JT0PLAXuSO/9FeWhnZ2Sfkp5qOhrTf8qzMysIYqoHp5vrUKhEMVisdVhmJlNK5J2R0ShXjvfkWtmliNO+mZmOeKkb2aWI076ZmY54qRvZpYjTvpmZjnipG9mliNO+mZmOeKkb2aWI076ZmY54qRvZpYjTvpmZjnipG9mliNO+mZmOTK71QE0y6FSP7sOHONo3wBLF7SzdtViVnZ1tjosM7NMmRFn+odK/ewo9nJyYIgLFrRzcmCIHcVeDpX6Wx2amVmmNJT0Ja2TtF9Sj6TbatSvkLRT0l5Jj0nqrqj7laQ96fVQ9XubYdeBYyzsaGN+RxvnSczvaGNhRxu7DhybjI8zM5u26iZ9SbOAe4D1wGpgk6TVVc3uArZHxKXAFuDOirpTEbEmvW5gEhztG6Cz/dyRqs722RztG5iMjzMzm7YaOdO/AuiJiIMRcRq4H9hQ1WY1sDNtP1qjflItXdBO/8DQOWX9A0MsXdA+lWGYmWVeI0l/GXC4Yr83lVV6BrgxbX8EOF/S4rTfLqko6UlJf1DrAyRtTm2KpVJpDOGXrV21mOOnBjlxapAzEZw4NcjxU4OsXbW4/pvNzHKkkaSvGmXVq6nfClwj6WngGuBFYPjU+zfSYr3/CPgzSavecrCIrRFRiIhCV1dX49EnK7s6uanQzbz22bzcN8C89tncVOj27B0zsyqNTNnsBZZX7HcDRyobRMQR4KMAkjqBGyOir6KOiDgo6THgMuDAhCOvsrKr00nezKyORs70nwIukbRS0hxgI3DOLBxJSyQNH+t2YFsqXyRp7nAb4APAz5oVvJmZjU3dpB8RQ8AtwMPAc8ADEbFP0hZJw7NxrgX2S3oeWArckcrfBRQlPUP5Au8XIsJJ38ysRRRRPTzfWoVCIYrFYqvDMDObViTtTtdPRzUj7sg1M7PGOOmbmeVI5oZ3JP0S2N/qOEawBHi11UHUkNW4wLGNR1bjAsc2HlMV14qIqDvnPYtP2dzfyLhUK0gqZjG2rMYFjm08shoXOLbxyFpcHt4xM8sRJ30zsxzJYtLf2uoARpHV2LIaFzi28chqXODYxiNTcWXuQq6ZmU2eLJ7pm5nZJHHSNzPLk4ho+gtoB35M+Tn7+4D/mMq/ARwC9qTXmlQu4G6gB9gLXF5xrI8DL6TXxyvKfxv4aXrP3aShqjHEOAt4GvjfaX8l8KP0OTuAOal8btrvSfUXVRzj9lS+H/hgRfm6VNYD3DbBuDLRZ8DP03v3AMVU9nbgkfQ5jwCLMhTb5yk/4nu4364f7/dtpJ+NBuJaCHwH+L+Un1v1Oxnqs1qxZaHP3lHx+XuAE8C/bHW/jRJXy/tsrK+mH7DiG9GZttvSF3Il5QT2hzXaXw98P73vSuBH8WZSOZj+XZS2h7/ZP04/qErvXT/GGP818N95M7k+AGxM218F/nna/gzw1bS9EdiRtldT/qU2N32zDlBO2LPS9sXAnNRm9QTiykSfUU6sS6rKvjT8QwvcBnwxQ7F9Hri1Rtsxf99G+tloIK7/BvzTtD2HcqLNSp/Viq3lfVb1ubOAl4EVWem3GnFlqs8aeU3K8E6U9afdtvQa7YrxBspr7EZEPAkslHQh8EHgkYh4LSL+H+Xf8OtS3fyIeCLKPbQdqLkqVy1p4fYPAV9P+wKuo3zmA+X/EMPH25D2SfW/l9pvAO6PiDci4hDl39pX0Njykg3FVceU9tkoMQz3TXWftTq20WJu+PtW52djRJLmA1cD/xUgIk5HxHEy0GejxDaSKemzGn4POBARvyAD/TZCXCNpVZ/VNWlj+pJmSdoDvEK583+Uqu6QtFfSV4aftc/ISzKOVt5bo7xRfwb8W+BM2l8MHI/yY6Srj3c2hlTfl9qPNebxxDUsC30WwA8k7Za0OZUtjYiXANK/v5ah2ABuSf22TdKiccY22s/GaC4GSsBfSnpa0tclzSMbfTZSbNDaPqu2EbgvbWeh32rFBdnqs7omLelHxK8iYg3llbaukPQeymNc7wTeS/nPrn+Xmo+0JONYy+uS9GHglYjYXVk8yvGmJLYR4oIM9FnygYi4HFgP3Czp6lHaZiG2e4FVwBrgJeBPpzi22cDlwL0RcRlwkvKwxEimss9Giq3VfXZWWrDpBuDb9ZpOZWw14spMnzVq0mfvpD8bHwPWRcRL6c+wN4C/pPynDoy8JONo5d01yhvxAeAGST+n/KfVdZTPsBdKGn4WUeXxzsaQ6hcAr40j5jHHJelbGekz4s1lL18BvpviOJr+XCb9+0pWYouIo+nE4wzwNcbfb68y8s/GaHqB3oq/cL9DOdFmoc9qxpaBPqu0HvhJRBxN+1not7fElbE+a0xMwoUCoAtYmLY7gB8CHwYuTGWinGi/kPY/xLkXY34cb16MOUT5QsyitP32VPdUajt8Meb6ccR5LW9eMP02515E+UzavplzL+Q+kLbfzbkXag5SvkgzO22v5M0LNe+eQFwt7zNgHnB+xfbfUp6B8GXOvbj2pQzFdmFFm39FeXx1XN+3kX42Gojth8A70vbnU3+1vM9Gia3lfVbx+fcDn6zYz0q/VceVmT5r+GuYlIPCpZSnHe4FngX+Qyr/a8pTpZ4FvsWbM3wE3EP5qvZPgULFsf6E8kWQnqrOLqTjHAD+gjFO2UzHuJY3k+vFlK/q96TOn5vK29N+T6q/uOL9n02fv5+KGQCUZxQ8n+o+O8G4Wt5nqW+e4c0puJ9N5YuBnZSnmO2s+E+Vhdi+mT57L+U1nSv/c47p+zbSz0YDsa0BiimGByknn5b32SixtbzP0nvfBhwDFlSUtbzfRogrE302lpcfw2BmliO+I9fMLEec9M3McsRJ38wsR5z0zcxyxEnfzCxHnPTNzHLESd/MLEf+PytVhbytJMb9AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.scatter(seoul_born_death_mrg[['# of marriges']], seoul_born_death_mrg[['tot_pop']], alpha=0.4)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "ename": "SyntaxError",
     "evalue": "invalid syntax (<ipython-input-40-94b124c2bc4c>, line 1)",
     "output_type": "error",
     "traceback": [
      "\u001b[1;36m  File \u001b[1;32m\"<ipython-input-40-94b124c2bc4c>\"\u001b[1;36m, line \u001b[1;32m1\u001b[0m\n\u001b[1;33m    <img src = './images/fig 10.png' width=800 height=200>\u001b[0m\n\u001b[1;37m    ^\u001b[0m\n\u001b[1;31mSyntaxError\u001b[0m\u001b[1;31m:\u001b[0m invalid syntax\n"
     ]
    }
   ],
   "source": [
    "<img src = './images/fig 10.png' width=800 height=200>  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Logistic Regression (Classification) _Ex 1_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "#-*- coding: utf-8 -*-\n",
    "import tensorflow as tf\n",
    "import numpy as np"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "# x,y random data values\n",
    "data = [[2, 0], [4, 0], [6, 0], [8, 1], [10, 1], [12, 1], [14, 1]]\n",
    "x_data = [x_row[0] for x_row in data]\n",
    "y_data = [y_row[1] for y_row in data]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[2, 4, 6, 8, 10, 12, 14]"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x_data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Arbitrarily set the values of a and b\n",
    "a = tf.Variable(tf.random.normal([1], dtype=tf.float64, seed=0))\n",
    "b = tf.Variable(tf.random.normal([1], dtype=tf.float64, seed=0))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Establish an equation for the y sigmoid function\n",
    "y = 1/(1 + np.e**(a * x_data + b))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "# function to find the loss\n",
    "loss = -tf.reduce_mean(np.array(y_data) * tf.math.log(y) + (1 - np.array(y_data)) * tf.math.log(1 - y))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "# learning_rate\n",
    "learning_rate=0.5"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Find the value that minimizes the loss\n",
    "import tensorflow.compat.v1 as tf\n",
    "tf.disable_v2_behavior()\n",
    "gradient_decent = tf.train.GradientDescentOptimizer(learning_rate).minimize(loss)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch: 0, loss = 1.2676, 기울기 a = 0.1849, 바이어스 b = -0.4334\n",
      "Epoch: 6000, loss = 0.0152, 기울기 a = -2.9211, 바이어스 b = 20.2982\n",
      "Epoch: 12000, loss = 0.0081, 기울기 a = -3.5637, 바이어스 b = 24.8010\n",
      "Epoch: 18000, loss = 0.0055, 기울기 a = -3.9557, 바이어스 b = 27.5463\n",
      "Epoch: 24000, loss = 0.0041, 기울기 a = -4.2380, 바이어스 b = 29.5231\n",
      "Epoch: 30000, loss = 0.0033, 기울기 a = -4.4586, 바이어스 b = 31.0675\n",
      "Epoch: 36000, loss = 0.0028, 기울기 a = -4.6396, 바이어스 b = 32.3346\n",
      "Epoch: 42000, loss = 0.0024, 기울기 a = -4.7930, 바이어스 b = 33.4086\n",
      "Epoch: 48000, loss = 0.0021, 기울기 a = -4.9261, 바이어스 b = 34.3406\n",
      "Epoch: 54000, loss = 0.0019, 기울기 a = -5.0436, 바이어스 b = 35.1636\n",
      "Epoch: 60000, loss = 0.0017, 기울기 a = -5.1489, 바이어스 b = 35.9005\n"
     ]
    }
   ],
   "source": [
    "# training\n",
    "with tf.Session() as sess:\n",
    "    sess.run(tf.global_variables_initializer())\n",
    "\n",
    "    for i in range(60001):\n",
    "        sess.run(gradient_decent)\n",
    "        if i % 6000 == 0:\n",
    "            print(\"Epoch: %.f, loss = %.4f, 기울기 a = %.4f, 바이어스 b = %.4f\" % (i, sess.run(loss), sess.run(a), sess.run(b)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Logistic Regression _Ex 2_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\Owner\\Anaconda\\lib\\site-packages\\numpy\\_distributor_init.py:32: UserWarning: loaded more than 1 DLL from .libs:\n",
      "C:\\Users\\Owner\\Anaconda\\lib\\site-packages\\numpy\\.libs\\libopenblas.PYQHXLVVQ7VESDPUVUADXEVJOBGHJPAY.gfortran-win_amd64.dll\n",
      "C:\\Users\\Owner\\Anaconda\\lib\\site-packages\\numpy\\.libs\\libopenblas.WCDJNK7YVMPZQ2ME2ZZHJJRJ3JIKNDB7.gfortran-win_amd64.dll\n",
      "  stacklevel=1)\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "import seaborn as sns\n",
    "\n",
    "%matplotlib inline\n",
    "\n",
    "plt.style.use('./class_04_plot.style')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# '''\n",
    "# lines.linewidth: 4\n",
    "# lines.solid_capstyle: butt\n",
    "\n",
    "# legend.fancybox: true\n",
    "\n",
    "# axes.prop_cycle: cycler('color', ['51a7f9', 'cf51f9', '70bf41', 'f39019', 'f9e351', 'f9517b', '6d904f', '8b8b8b','810f7c']) \n",
    "\n",
    "# axes.labelsize: large\n",
    "# axes.axisbelow: true\n",
    "# axes.grid: true\n",
    "# axes.edgecolor: f0f0f0\n",
    "# axes.linewidth: 3.0\n",
    "# axes.titlesize: x-large\n",
    "\n",
    "# patch.edgecolor: f0f0f0\n",
    "# patch.linewidth: 0.5\n",
    "\n",
    "# svg.fonttype: path\n",
    "\n",
    "# grid.linestyle: -\n",
    "# grid.linewidth: 1.0\n",
    "\n",
    "# xtick.major.size: 0\n",
    "# xtick.minor.size: 0\n",
    "# ytick.major.size: 0\n",
    "# ytick.minor.size: 0\n",
    "\n",
    "# font.size: 24.0\n",
    "\n",
    "# savefig.edgecolor: f0f0f0\n",
    "# savefig.facecolor: f0f0f0\n",
    "\n",
    "# figure.subplot.left: 0.08\n",
    "# figure.subplot.right: 0.95\n",
    "# figure.subplot.bottom: 0.07\n",
    "# figure.figsize: 12.8, 8.8\n",
    "# figure.autolayout: True\n",
    "# figure.dpi: 300\n",
    "# '''"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Load the dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "data = pd.read_csv('./data/iris.csv', delimiter=',', header=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Quick visualization of the data. Always a good way to make sure we have what we expect"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>sepal_length</th>\n",
       "      <th>sepal_width</th>\n",
       "      <th>petal_length</th>\n",
       "      <th>petal_width</th>\n",
       "      <th>species</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>5.1</td>\n",
       "      <td>3.5</td>\n",
       "      <td>1.4</td>\n",
       "      <td>0.2</td>\n",
       "      <td>setosa</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>4.9</td>\n",
       "      <td>3.0</td>\n",
       "      <td>1.4</td>\n",
       "      <td>0.2</td>\n",
       "      <td>setosa</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>4.7</td>\n",
       "      <td>3.2</td>\n",
       "      <td>1.3</td>\n",
       "      <td>0.2</td>\n",
       "      <td>setosa</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4.6</td>\n",
       "      <td>3.1</td>\n",
       "      <td>1.5</td>\n",
       "      <td>0.2</td>\n",
       "      <td>setosa</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5.0</td>\n",
       "      <td>3.6</td>\n",
       "      <td>1.4</td>\n",
       "      <td>0.2</td>\n",
       "      <td>setosa</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>5.4</td>\n",
       "      <td>3.9</td>\n",
       "      <td>1.7</td>\n",
       "      <td>0.4</td>\n",
       "      <td>setosa</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>4.6</td>\n",
       "      <td>3.4</td>\n",
       "      <td>1.4</td>\n",
       "      <td>0.3</td>\n",
       "      <td>setosa</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>5.0</td>\n",
       "      <td>3.4</td>\n",
       "      <td>1.5</td>\n",
       "      <td>0.2</td>\n",
       "      <td>setosa</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>4.4</td>\n",
       "      <td>2.9</td>\n",
       "      <td>1.4</td>\n",
       "      <td>0.2</td>\n",
       "      <td>setosa</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>4.9</td>\n",
       "      <td>3.1</td>\n",
       "      <td>1.5</td>\n",
       "      <td>0.1</td>\n",
       "      <td>setosa</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>5.4</td>\n",
       "      <td>3.7</td>\n",
       "      <td>1.5</td>\n",
       "      <td>0.2</td>\n",
       "      <td>setosa</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>4.8</td>\n",
       "      <td>3.4</td>\n",
       "      <td>1.6</td>\n",
       "      <td>0.2</td>\n",
       "      <td>setosa</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>4.8</td>\n",
       "      <td>3.0</td>\n",
       "      <td>1.4</td>\n",
       "      <td>0.1</td>\n",
       "      <td>setosa</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>4.3</td>\n",
       "      <td>3.0</td>\n",
       "      <td>1.1</td>\n",
       "      <td>0.1</td>\n",
       "      <td>setosa</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>5.8</td>\n",
       "      <td>4.0</td>\n",
       "      <td>1.2</td>\n",
       "      <td>0.2</td>\n",
       "      <td>setosa</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>5.7</td>\n",
       "      <td>4.4</td>\n",
       "      <td>1.5</td>\n",
       "      <td>0.4</td>\n",
       "      <td>setosa</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>5.4</td>\n",
       "      <td>3.9</td>\n",
       "      <td>1.3</td>\n",
       "      <td>0.4</td>\n",
       "      <td>setosa</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>5.1</td>\n",
       "      <td>3.5</td>\n",
       "      <td>1.4</td>\n",
       "      <td>0.3</td>\n",
       "      <td>setosa</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>5.7</td>\n",
       "      <td>3.8</td>\n",
       "      <td>1.7</td>\n",
       "      <td>0.3</td>\n",
       "      <td>setosa</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>5.1</td>\n",
       "      <td>3.8</td>\n",
       "      <td>1.5</td>\n",
       "      <td>0.3</td>\n",
       "      <td>setosa</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>5.4</td>\n",
       "      <td>3.4</td>\n",
       "      <td>1.7</td>\n",
       "      <td>0.2</td>\n",
       "      <td>setosa</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>5.1</td>\n",
       "      <td>3.7</td>\n",
       "      <td>1.5</td>\n",
       "      <td>0.4</td>\n",
       "      <td>setosa</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>4.6</td>\n",
       "      <td>3.6</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.2</td>\n",
       "      <td>setosa</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>5.1</td>\n",
       "      <td>3.3</td>\n",
       "      <td>1.7</td>\n",
       "      <td>0.5</td>\n",
       "      <td>setosa</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>4.8</td>\n",
       "      <td>3.4</td>\n",
       "      <td>1.9</td>\n",
       "      <td>0.2</td>\n",
       "      <td>setosa</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>5.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>1.6</td>\n",
       "      <td>0.2</td>\n",
       "      <td>setosa</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>5.0</td>\n",
       "      <td>3.4</td>\n",
       "      <td>1.6</td>\n",
       "      <td>0.4</td>\n",
       "      <td>setosa</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>5.2</td>\n",
       "      <td>3.5</td>\n",
       "      <td>1.5</td>\n",
       "      <td>0.2</td>\n",
       "      <td>setosa</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>5.2</td>\n",
       "      <td>3.4</td>\n",
       "      <td>1.4</td>\n",
       "      <td>0.2</td>\n",
       "      <td>setosa</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>4.7</td>\n",
       "      <td>3.2</td>\n",
       "      <td>1.6</td>\n",
       "      <td>0.2</td>\n",
       "      <td>setosa</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>120</th>\n",
       "      <td>6.9</td>\n",
       "      <td>3.2</td>\n",
       "      <td>5.7</td>\n",
       "      <td>2.3</td>\n",
       "      <td>virginica</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>121</th>\n",
       "      <td>5.6</td>\n",
       "      <td>2.8</td>\n",
       "      <td>4.9</td>\n",
       "      <td>2.0</td>\n",
       "      <td>virginica</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>122</th>\n",
       "      <td>7.7</td>\n",
       "      <td>2.8</td>\n",
       "      <td>6.7</td>\n",
       "      <td>2.0</td>\n",
       "      <td>virginica</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>123</th>\n",
       "      <td>6.3</td>\n",
       "      <td>2.7</td>\n",
       "      <td>4.9</td>\n",
       "      <td>1.8</td>\n",
       "      <td>virginica</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>124</th>\n",
       "      <td>6.7</td>\n",
       "      <td>3.3</td>\n",
       "      <td>5.7</td>\n",
       "      <td>2.1</td>\n",
       "      <td>virginica</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>125</th>\n",
       "      <td>7.2</td>\n",
       "      <td>3.2</td>\n",
       "      <td>6.0</td>\n",
       "      <td>1.8</td>\n",
       "      <td>virginica</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>126</th>\n",
       "      <td>6.2</td>\n",
       "      <td>2.8</td>\n",
       "      <td>4.8</td>\n",
       "      <td>1.8</td>\n",
       "      <td>virginica</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>127</th>\n",
       "      <td>6.1</td>\n",
       "      <td>3.0</td>\n",
       "      <td>4.9</td>\n",
       "      <td>1.8</td>\n",
       "      <td>virginica</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>128</th>\n",
       "      <td>6.4</td>\n",
       "      <td>2.8</td>\n",
       "      <td>5.6</td>\n",
       "      <td>2.1</td>\n",
       "      <td>virginica</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>129</th>\n",
       "      <td>7.2</td>\n",
       "      <td>3.0</td>\n",
       "      <td>5.8</td>\n",
       "      <td>1.6</td>\n",
       "      <td>virginica</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>130</th>\n",
       "      <td>7.4</td>\n",
       "      <td>2.8</td>\n",
       "      <td>6.1</td>\n",
       "      <td>1.9</td>\n",
       "      <td>virginica</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>131</th>\n",
       "      <td>7.9</td>\n",
       "      <td>3.8</td>\n",
       "      <td>6.4</td>\n",
       "      <td>2.0</td>\n",
       "      <td>virginica</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>132</th>\n",
       "      <td>6.4</td>\n",
       "      <td>2.8</td>\n",
       "      <td>5.6</td>\n",
       "      <td>2.2</td>\n",
       "      <td>virginica</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>133</th>\n",
       "      <td>6.3</td>\n",
       "      <td>2.8</td>\n",
       "      <td>5.1</td>\n",
       "      <td>1.5</td>\n",
       "      <td>virginica</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>134</th>\n",
       "      <td>6.1</td>\n",
       "      <td>2.6</td>\n",
       "      <td>5.6</td>\n",
       "      <td>1.4</td>\n",
       "      <td>virginica</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>135</th>\n",
       "      <td>7.7</td>\n",
       "      <td>3.0</td>\n",
       "      <td>6.1</td>\n",
       "      <td>2.3</td>\n",
       "      <td>virginica</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>136</th>\n",
       "      <td>6.3</td>\n",
       "      <td>3.4</td>\n",
       "      <td>5.6</td>\n",
       "      <td>2.4</td>\n",
       "      <td>virginica</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>137</th>\n",
       "      <td>6.4</td>\n",
       "      <td>3.1</td>\n",
       "      <td>5.5</td>\n",
       "      <td>1.8</td>\n",
       "      <td>virginica</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>138</th>\n",
       "      <td>6.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>4.8</td>\n",
       "      <td>1.8</td>\n",
       "      <td>virginica</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>139</th>\n",
       "      <td>6.9</td>\n",
       "      <td>3.1</td>\n",
       "      <td>5.4</td>\n",
       "      <td>2.1</td>\n",
       "      <td>virginica</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>140</th>\n",
       "      <td>6.7</td>\n",
       "      <td>3.1</td>\n",
       "      <td>5.6</td>\n",
       "      <td>2.4</td>\n",
       "      <td>virginica</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>141</th>\n",
       "      <td>6.9</td>\n",
       "      <td>3.1</td>\n",
       "      <td>5.1</td>\n",
       "      <td>2.3</td>\n",
       "      <td>virginica</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>142</th>\n",
       "      <td>5.8</td>\n",
       "      <td>2.7</td>\n",
       "      <td>5.1</td>\n",
       "      <td>1.9</td>\n",
       "      <td>virginica</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>143</th>\n",
       "      <td>6.8</td>\n",
       "      <td>3.2</td>\n",
       "      <td>5.9</td>\n",
       "      <td>2.3</td>\n",
       "      <td>virginica</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>144</th>\n",
       "      <td>6.7</td>\n",
       "      <td>3.3</td>\n",
       "      <td>5.7</td>\n",
       "      <td>2.5</td>\n",
       "      <td>virginica</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>145</th>\n",
       "      <td>6.7</td>\n",
       "      <td>3.0</td>\n",
       "      <td>5.2</td>\n",
       "      <td>2.3</td>\n",
       "      <td>virginica</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>146</th>\n",
       "      <td>6.3</td>\n",
       "      <td>2.5</td>\n",
       "      <td>5.0</td>\n",
       "      <td>1.9</td>\n",
       "      <td>virginica</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>147</th>\n",
       "      <td>6.5</td>\n",
       "      <td>3.0</td>\n",
       "      <td>5.2</td>\n",
       "      <td>2.0</td>\n",
       "      <td>virginica</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>148</th>\n",
       "      <td>6.2</td>\n",
       "      <td>3.4</td>\n",
       "      <td>5.4</td>\n",
       "      <td>2.3</td>\n",
       "      <td>virginica</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>149</th>\n",
       "      <td>5.9</td>\n",
       "      <td>3.0</td>\n",
       "      <td>5.1</td>\n",
       "      <td>1.8</td>\n",
       "      <td>virginica</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>150 rows × 5 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "     sepal_length  sepal_width  petal_length  petal_width    species\n",
       "0             5.1          3.5           1.4          0.2     setosa\n",
       "1             4.9          3.0           1.4          0.2     setosa\n",
       "2             4.7          3.2           1.3          0.2     setosa\n",
       "3             4.6          3.1           1.5          0.2     setosa\n",
       "4             5.0          3.6           1.4          0.2     setosa\n",
       "5             5.4          3.9           1.7          0.4     setosa\n",
       "6             4.6          3.4           1.4          0.3     setosa\n",
       "7             5.0          3.4           1.5          0.2     setosa\n",
       "8             4.4          2.9           1.4          0.2     setosa\n",
       "9             4.9          3.1           1.5          0.1     setosa\n",
       "10            5.4          3.7           1.5          0.2     setosa\n",
       "11            4.8          3.4           1.6          0.2     setosa\n",
       "12            4.8          3.0           1.4          0.1     setosa\n",
       "13            4.3          3.0           1.1          0.1     setosa\n",
       "14            5.8          4.0           1.2          0.2     setosa\n",
       "15            5.7          4.4           1.5          0.4     setosa\n",
       "16            5.4          3.9           1.3          0.4     setosa\n",
       "17            5.1          3.5           1.4          0.3     setosa\n",
       "18            5.7          3.8           1.7          0.3     setosa\n",
       "19            5.1          3.8           1.5          0.3     setosa\n",
       "20            5.4          3.4           1.7          0.2     setosa\n",
       "21            5.1          3.7           1.5          0.4     setosa\n",
       "22            4.6          3.6           1.0          0.2     setosa\n",
       "23            5.1          3.3           1.7          0.5     setosa\n",
       "24            4.8          3.4           1.9          0.2     setosa\n",
       "25            5.0          3.0           1.6          0.2     setosa\n",
       "26            5.0          3.4           1.6          0.4     setosa\n",
       "27            5.2          3.5           1.5          0.2     setosa\n",
       "28            5.2          3.4           1.4          0.2     setosa\n",
       "29            4.7          3.2           1.6          0.2     setosa\n",
       "..            ...          ...           ...          ...        ...\n",
       "120           6.9          3.2           5.7          2.3  virginica\n",
       "121           5.6          2.8           4.9          2.0  virginica\n",
       "122           7.7          2.8           6.7          2.0  virginica\n",
       "123           6.3          2.7           4.9          1.8  virginica\n",
       "124           6.7          3.3           5.7          2.1  virginica\n",
       "125           7.2          3.2           6.0          1.8  virginica\n",
       "126           6.2          2.8           4.8          1.8  virginica\n",
       "127           6.1          3.0           4.9          1.8  virginica\n",
       "128           6.4          2.8           5.6          2.1  virginica\n",
       "129           7.2          3.0           5.8          1.6  virginica\n",
       "130           7.4          2.8           6.1          1.9  virginica\n",
       "131           7.9          3.8           6.4          2.0  virginica\n",
       "132           6.4          2.8           5.6          2.2  virginica\n",
       "133           6.3          2.8           5.1          1.5  virginica\n",
       "134           6.1          2.6           5.6          1.4  virginica\n",
       "135           7.7          3.0           6.1          2.3  virginica\n",
       "136           6.3          3.4           5.6          2.4  virginica\n",
       "137           6.4          3.1           5.5          1.8  virginica\n",
       "138           6.0          3.0           4.8          1.8  virginica\n",
       "139           6.9          3.1           5.4          2.1  virginica\n",
       "140           6.7          3.1           5.6          2.4  virginica\n",
       "141           6.9          3.1           5.1          2.3  virginica\n",
       "142           5.8          2.7           5.1          1.9  virginica\n",
       "143           6.8          3.2           5.9          2.3  virginica\n",
       "144           6.7          3.3           5.7          2.5  virginica\n",
       "145           6.7          3.0           5.2          2.3  virginica\n",
       "146           6.3          2.5           5.0          1.9  virginica\n",
       "147           6.5          3.0           5.2          2.0  virginica\n",
       "148           6.2          3.4           5.4          2.3  virginica\n",
       "149           5.9          3.0           5.1          1.8  virginica\n",
       "\n",
       "[150 rows x 5 columns]"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Set the labels. We will try to predict the 'virginica' label"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "data['y'] = 0\n",
    "data.loc[data['species'] == 'virginica', 'y'] = 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Extract the data as arrays"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>sepal_length</th>\n",
       "      <th>sepal_width</th>\n",
       "      <th>petal_length</th>\n",
       "      <th>petal_width</th>\n",
       "      <th>species</th>\n",
       "      <th>y</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>145</th>\n",
       "      <td>6.7</td>\n",
       "      <td>3.0</td>\n",
       "      <td>5.2</td>\n",
       "      <td>2.3</td>\n",
       "      <td>virginica</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>146</th>\n",
       "      <td>6.3</td>\n",
       "      <td>2.5</td>\n",
       "      <td>5.0</td>\n",
       "      <td>1.9</td>\n",
       "      <td>virginica</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>147</th>\n",
       "      <td>6.5</td>\n",
       "      <td>3.0</td>\n",
       "      <td>5.2</td>\n",
       "      <td>2.0</td>\n",
       "      <td>virginica</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>148</th>\n",
       "      <td>6.2</td>\n",
       "      <td>3.4</td>\n",
       "      <td>5.4</td>\n",
       "      <td>2.3</td>\n",
       "      <td>virginica</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>149</th>\n",
       "      <td>5.9</td>\n",
       "      <td>3.0</td>\n",
       "      <td>5.1</td>\n",
       "      <td>1.8</td>\n",
       "      <td>virginica</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     sepal_length  sepal_width  petal_length  petal_width    species  y\n",
       "145           6.7          3.0           5.2          2.3  virginica  1\n",
       "146           6.3          2.5           5.0          1.9  virginica  1\n",
       "147           6.5          3.0           5.2          2.0  virginica  1\n",
       "148           6.2          3.4           5.4          2.3  virginica  1\n",
       "149           5.9          3.0           5.1          1.8  virginica  1"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.tail()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "X = data[['sepal_length','sepal_width','petal_length','petal_width']].values\n",
    "y = data['y'].values.reshape(-1,1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Utility functions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Logistic function"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "def logistic(z):\n",
    "    return 1./(1+np.exp(-z))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Get matrix dimensions and add the bias column"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "M, N = X.shape\n",
    "X = np.concatenate((np.ones((M, 1)), X), axis=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Now we have a few more features"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[1.  5.1 3.5 1.4 0.2]\n",
      " [1.  4.9 3.  1.4 0.2]\n",
      " [1.  4.7 3.2 1.3 0.2]\n",
      " [1.  4.6 3.1 1.5 0.2]\n",
      " [1.  5.  3.6 1.4 0.2]\n",
      " [1.  5.4 3.9 1.7 0.4]\n",
      " [1.  4.6 3.4 1.4 0.3]\n",
      " [1.  5.  3.4 1.5 0.2]\n",
      " [1.  4.4 2.9 1.4 0.2]\n",
      " [1.  4.9 3.1 1.5 0.1]\n",
      " [1.  5.4 3.7 1.5 0.2]\n",
      " [1.  4.8 3.4 1.6 0.2]\n",
      " [1.  4.8 3.  1.4 0.1]\n",
      " [1.  4.3 3.  1.1 0.1]\n",
      " [1.  5.8 4.  1.2 0.2]\n",
      " [1.  5.7 4.4 1.5 0.4]\n",
      " [1.  5.4 3.9 1.3 0.4]\n",
      " [1.  5.1 3.5 1.4 0.3]\n",
      " [1.  5.7 3.8 1.7 0.3]\n",
      " [1.  5.1 3.8 1.5 0.3]\n",
      " [1.  5.4 3.4 1.7 0.2]\n",
      " [1.  5.1 3.7 1.5 0.4]\n",
      " [1.  4.6 3.6 1.  0.2]\n",
      " [1.  5.1 3.3 1.7 0.5]\n",
      " [1.  4.8 3.4 1.9 0.2]\n",
      " [1.  5.  3.  1.6 0.2]\n",
      " [1.  5.  3.4 1.6 0.4]\n",
      " [1.  5.2 3.5 1.5 0.2]\n",
      " [1.  5.2 3.4 1.4 0.2]\n",
      " [1.  4.7 3.2 1.6 0.2]\n",
      " [1.  4.8 3.1 1.6 0.2]\n",
      " [1.  5.4 3.4 1.5 0.4]\n",
      " [1.  5.2 4.1 1.5 0.1]\n",
      " [1.  5.5 4.2 1.4 0.2]\n",
      " [1.  4.9 3.1 1.5 0.1]\n",
      " [1.  5.  3.2 1.2 0.2]\n",
      " [1.  5.5 3.5 1.3 0.2]\n",
      " [1.  4.9 3.1 1.5 0.1]\n",
      " [1.  4.4 3.  1.3 0.2]\n",
      " [1.  5.1 3.4 1.5 0.2]\n",
      " [1.  5.  3.5 1.3 0.3]\n",
      " [1.  4.5 2.3 1.3 0.3]\n",
      " [1.  4.4 3.2 1.3 0.2]\n",
      " [1.  5.  3.5 1.6 0.6]\n",
      " [1.  5.1 3.8 1.9 0.4]\n",
      " [1.  4.8 3.  1.4 0.3]\n",
      " [1.  5.1 3.8 1.6 0.2]\n",
      " [1.  4.6 3.2 1.4 0.2]\n",
      " [1.  5.3 3.7 1.5 0.2]\n",
      " [1.  5.  3.3 1.4 0.2]\n",
      " [1.  7.  3.2 4.7 1.4]\n",
      " [1.  6.4 3.2 4.5 1.5]\n",
      " [1.  6.9 3.1 4.9 1.5]\n",
      " [1.  5.5 2.3 4.  1.3]\n",
      " [1.  6.5 2.8 4.6 1.5]\n",
      " [1.  5.7 2.8 4.5 1.3]\n",
      " [1.  6.3 3.3 4.7 1.6]\n",
      " [1.  4.9 2.4 3.3 1. ]\n",
      " [1.  6.6 2.9 4.6 1.3]\n",
      " [1.  5.2 2.7 3.9 1.4]\n",
      " [1.  5.  2.  3.5 1. ]\n",
      " [1.  5.9 3.  4.2 1.5]\n",
      " [1.  6.  2.2 4.  1. ]\n",
      " [1.  6.1 2.9 4.7 1.4]\n",
      " [1.  5.6 2.9 3.6 1.3]\n",
      " [1.  6.7 3.1 4.4 1.4]\n",
      " [1.  5.6 3.  4.5 1.5]\n",
      " [1.  5.8 2.7 4.1 1. ]\n",
      " [1.  6.2 2.2 4.5 1.5]\n",
      " [1.  5.6 2.5 3.9 1.1]\n",
      " [1.  5.9 3.2 4.8 1.8]\n",
      " [1.  6.1 2.8 4.  1.3]\n",
      " [1.  6.3 2.5 4.9 1.5]\n",
      " [1.  6.1 2.8 4.7 1.2]\n",
      " [1.  6.4 2.9 4.3 1.3]\n",
      " [1.  6.6 3.  4.4 1.4]\n",
      " [1.  6.8 2.8 4.8 1.4]\n",
      " [1.  6.7 3.  5.  1.7]\n",
      " [1.  6.  2.9 4.5 1.5]\n",
      " [1.  5.7 2.6 3.5 1. ]\n",
      " [1.  5.5 2.4 3.8 1.1]\n",
      " [1.  5.5 2.4 3.7 1. ]\n",
      " [1.  5.8 2.7 3.9 1.2]\n",
      " [1.  6.  2.7 5.1 1.6]\n",
      " [1.  5.4 3.  4.5 1.5]\n",
      " [1.  6.  3.4 4.5 1.6]\n",
      " [1.  6.7 3.1 4.7 1.5]\n",
      " [1.  6.3 2.3 4.4 1.3]\n",
      " [1.  5.6 3.  4.1 1.3]\n",
      " [1.  5.5 2.5 4.  1.3]\n",
      " [1.  5.5 2.6 4.4 1.2]\n",
      " [1.  6.1 3.  4.6 1.4]\n",
      " [1.  5.8 2.6 4.  1.2]\n",
      " [1.  5.  2.3 3.3 1. ]\n",
      " [1.  5.6 2.7 4.2 1.3]\n",
      " [1.  5.7 3.  4.2 1.2]\n",
      " [1.  5.7 2.9 4.2 1.3]\n",
      " [1.  6.2 2.9 4.3 1.3]\n",
      " [1.  5.1 2.5 3.  1.1]\n",
      " [1.  5.7 2.8 4.1 1.3]\n",
      " [1.  6.3 3.3 6.  2.5]\n",
      " [1.  5.8 2.7 5.1 1.9]\n",
      " [1.  7.1 3.  5.9 2.1]\n",
      " [1.  6.3 2.9 5.6 1.8]\n",
      " [1.  6.5 3.  5.8 2.2]\n",
      " [1.  7.6 3.  6.6 2.1]\n",
      " [1.  4.9 2.5 4.5 1.7]\n",
      " [1.  7.3 2.9 6.3 1.8]\n",
      " [1.  6.7 2.5 5.8 1.8]\n",
      " [1.  7.2 3.6 6.1 2.5]\n",
      " [1.  6.5 3.2 5.1 2. ]\n",
      " [1.  6.4 2.7 5.3 1.9]\n",
      " [1.  6.8 3.  5.5 2.1]\n",
      " [1.  5.7 2.5 5.  2. ]\n",
      " [1.  5.8 2.8 5.1 2.4]\n",
      " [1.  6.4 3.2 5.3 2.3]\n",
      " [1.  6.5 3.  5.5 1.8]\n",
      " [1.  7.7 3.8 6.7 2.2]\n",
      " [1.  7.7 2.6 6.9 2.3]\n",
      " [1.  6.  2.2 5.  1.5]\n",
      " [1.  6.9 3.2 5.7 2.3]\n",
      " [1.  5.6 2.8 4.9 2. ]\n",
      " [1.  7.7 2.8 6.7 2. ]\n",
      " [1.  6.3 2.7 4.9 1.8]\n",
      " [1.  6.7 3.3 5.7 2.1]\n",
      " [1.  7.2 3.2 6.  1.8]\n",
      " [1.  6.2 2.8 4.8 1.8]\n",
      " [1.  6.1 3.  4.9 1.8]\n",
      " [1.  6.4 2.8 5.6 2.1]\n",
      " [1.  7.2 3.  5.8 1.6]\n",
      " [1.  7.4 2.8 6.1 1.9]\n",
      " [1.  7.9 3.8 6.4 2. ]\n",
      " [1.  6.4 2.8 5.6 2.2]\n",
      " [1.  6.3 2.8 5.1 1.5]\n",
      " [1.  6.1 2.6 5.6 1.4]\n",
      " [1.  7.7 3.  6.1 2.3]\n",
      " [1.  6.3 3.4 5.6 2.4]\n",
      " [1.  6.4 3.1 5.5 1.8]\n",
      " [1.  6.  3.  4.8 1.8]\n",
      " [1.  6.9 3.1 5.4 2.1]\n",
      " [1.  6.7 3.1 5.6 2.4]\n",
      " [1.  6.9 3.1 5.1 2.3]\n",
      " [1.  5.8 2.7 5.1 1.9]\n",
      " [1.  6.8 3.2 5.9 2.3]\n",
      " [1.  6.7 3.3 5.7 2.5]\n",
      " [1.  6.7 3.  5.2 2.3]\n",
      " [1.  6.3 2.5 5.  1.9]\n",
      " [1.  6.5 3.  5.2 2. ]\n",
      " [1.  6.2 3.4 5.4 2.3]\n",
      " [1.  5.9 3.  5.1 1.8]]\n"
     ]
    }
   ],
   "source": [
    "print(X)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Set the training parameters and initialize the weight matrix"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "alpha = 0.5\n",
    "epsilon = 0.12"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "weights = 2*np.random.rand(N+1, 1)*epsilon - epsilon"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Training procedure"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "10 [[0.85967953]] [[2.59346825]] [-0.38069311 -1.31346633 -1.24547118  0.73171913  0.60903529]\n",
      "20 [[2.19045594]] [[0.68055068]] [-0.52316232 -1.47724486 -1.55174862  1.5906725   1.15411386]\n",
      "30 [[2.10072195]] [[1.32128281]] [-0.65683789 -1.71067617 -1.78845817  2.15120887  1.56628452]\n",
      "40 [[1.89205729]] [[1.14641659]] [-0.78758932 -2.00289062 -1.99809281  2.52745284  1.896139  ]\n",
      "50 [[1.71473272]] [[0.96960292]] [-0.91458165 -2.28799971 -2.19613546  2.88136966  2.21331864]\n",
      "60 [[1.53905317]] [[0.79514638]] [-1.03856947 -2.56431069 -2.38759473  3.22600491  2.52367729]\n",
      "70 [[1.35980632]] [[0.62846873]] [-1.1584644  -2.82733699 -2.57100338  3.56180841  2.82657404]\n",
      "80 [[1.17416308]] [[0.47453581]] [-1.27237343 -3.07044955 -2.74278449  3.88548904  3.11884577]\n",
      "90 [[0.97492751]] [[0.33480951]] [-1.37750758 -3.28335138 -2.89724977  4.19176968  3.39507959]\n",
      "100 [[0.73611773]] [[0.2025702]] [-1.46827    -3.4432504  -3.02223997  4.47411939  3.64590757]\n",
      "110 [[0.35653213]] [[0.044936]] [-1.52415421 -3.46092247 -3.07071173  4.73685735  3.85242363]\n",
      "120 [[0.09601013]] [[0.00196533]] [-1.52829853 -3.32204903 -3.02064862  4.91097704  3.95869774]\n",
      "122 [[0.09308819]] [[0.00087729]] [-1.53094918 -3.31451857 -3.01867013  4.92213024  3.96882772]\n"
     ]
    }
   ],
   "source": [
    "oldJ = 0\n",
    "err = 1\n",
    "\n",
    "Js = []\n",
    "count = 0\n",
    "\n",
    "while err > 1e-3:\n",
    "    Zs = np.dot(X, weights)\n",
    "    Hs = logistic(Zs) # phi(z)\n",
    "    \n",
    "    deltas = alpha/M*np.dot(X.T, (Hs-y))\n",
    "    weights -= deltas\n",
    "\n",
    "    count += 1\n",
    "\n",
    "    # Cross Entropy\n",
    "    J = -1/M*(np.dot(y.T, np.log(Hs)) + np.dot(1-y.T, np.log(1-Hs)))\n",
    "    \n",
    "    Js.append(float(J))\n",
    "    err = np.abs(oldJ-J)\n",
    "    oldJ = J\n",
    "    \n",
    "    if count % 10 == 0:\n",
    "        print(count, J, err, weights.flatten())\n",
    "\n",
    "print(count, J, err, weights.flatten())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Solution exploration"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\Owner\\Anaconda\\lib\\site-packages\\matplotlib\\figure.py:2369: UserWarning: This figure includes Axes that are not compatible with tight_layout, so results might be incorrect.\n",
      "  warnings.warn(\"This figure includes Axes that are not compatible \"\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAAEYCAYAAAAJeGK1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3Xt8FNXd+PHPd3Y3G+4IYkABCQpVJBBvUC/FeMenLWgtoH280NbWtsbWar1U/altlbbax9Yaqzw+WmtLRbASaKu1glK0aFQUXARRQBBEIiUkCiSb7M75/TF7z26yCclest/367Wv3Zk5M3POzO5+58ycOSPGGJRSSqlcY2U7A0oppVQyGqCUUkrlJA1QSimlcpIGKKWUUjnJne0MdERDQ4O26FBKqR5mwIABkmy81qCUUkrlJA1QSimlcpMxJm9e9fX1JvzqLtXV1d227EzRMuQGLUNu0DLkhsQyJPyfJ/3P1xqUUkqpnKQBSimlVE7Kq1Z8qRhjqKurw7btA17WiBEj2LVrVxfkKns6UwbLshg0aBAiSRvTKNXj7N9s8/6tAbDBAGJB6Y0uahfajLrOjecg57fQUmfYcm+Aw77p4qNHgpH3Ude5wcDmX7Zg7wf/Dhh5rcWuJTYiQumNblr2GN6/NUDxoTD6Fg+NW202/ywIHigaBM07oWgotOwBBEpvclG7wI4ue04LjdthhH02n44KxuVt/2abjf8vQNEwGHmVm20PBSLrDee9I8LljC17V6Q9ED0iQNXV1dGnTx+Ki4sPeFkej4eBAwd2Qa6ypzNlaGpqoq6ujsGDB3dTrpTKLVvvC2CaosMG2PKrIPZ+2LkwyIhvO3+POxcG2bfesPU3AfzbibzvXBgEA43vR5fx4X02BJyl7VwYZO87NqYJGjc76fe8FIqGzU5wgug7wJZ7ouvHOPMBCK5Wedt6XwC7EZo2h/K0LbrecN47IlzOdObvSNoD0SNO8dm23SXBqZAVFxd3SQ1UqXywf7Md+kOPZ+8DDNS9YNOyx9BSZ6h70Qkq/m3Eve9eZrN7WcJvJhD9uHtp/Dp2P2c7y29DeP27l9nUxSxbkLi8fbomGLfs2M91y5y8p8tXa/PD+c3sfsGOrPua+c34alv/H3QkbVfISIASkUdF5BMRWZtiuojIb0Vko4i8LSLHZSJfSqnCtPW+QJvTje3UEnYuDGJS/fcGiAtISafHLbQDGQyASbHsYBDWzgnGLS72c7AF/vzLlrSChq/WZs6KAONWQSAYWnUQjlkFc1YE4pbRkbRdJVOn+B4DqoDHU0w/DxgTek0GHgy9K6VUl0pVe4oTcGoHEvqcVHf2a9PGsq0g9ApC7JWf2M8WMG4j/PrvTfxw+x8o+896aGyE5uboq6UFAgGqLpiLx11C+QfgDsUXtw3lW2DFOKiq3sbc300D26bqqr/i8R6aOm1NgLnTirp0M2QkQBljVojIqDaSTAceN8YY4FURGSgiw4wxH2cifwdi9+7dnHnmmQDs3LkTl8vFkCFD2LJlC4ceeijr1q3Lcg6VUrHaqz1FBLo3BnUnlw0nrXdTNfIs5t7+vZTpKrdeyuvn/h3BEzdeDJzua2HSPy6D9W87ae+fnTrteph0XdeHEzEZemBhKED9zRgzPsm0vwG/MMa8HBpeBtxojHkjNl1sX3zLly+PjB8xYgSjR4/ulnx3xC9+8Qv69OnD1VdfzYcffsisWbN45ZVXunQdgUAAt7t7jis2b97Mtm3tHVoqld9G/mEqQs9vrep3G/qsPYfx61e0me7N8xuwpPU1fNs0cVz1gLTStniC7PjaPzuUv4qKisjnVH3x5UorvmSZazNyTp8+PfJ5165d8a3WurqpdJpBvLi4mF69ejFw4EDq6+sBuP7661m5ciWHHXYYixcvplevXmzatImrrrqKXbt20bt3bx5++GGOOuootm7dyje+8Q127drFkCFD+P3vf8/IkSOZPXs2gwYN4q233qK8vJy//e1vrFy5kiFDhmDbNmPHjuXVV1/l4IMPBqC+vr5TLRFHjBjBccflxuW/xYsXx+3jfKRlyA2JZVj/QnP7p/gg+q+Ug9Uog2kzyNoY/P5XmdxOcPKNO43VpS4mfBg9bQcQsODtkS6Kjp4SCXBtpfUd7mLS57/M+JLUzRoS90NDQ0M7pcydALUdGBEzPBzYkaW8dJn333+fJ554gocffpiZM2fyl7/8hUsuuYRvf/vbPPTQQ4wZM4aamhq+973v8cILL1BZWclll13G5ZdfzqOPPsr3v/99qqurAXjvvfdYunQpLpeLgQMHMm/ePK655hqWLl3KxIkTI8FJKdW2w3/g5r0fpXGaz+3EKNPS7VnqhLYPwoOW8NiXTmTQ6X9ivPwHiovB64WiIufd7QaPh6q9p9K4z0PZh/HzG4EXyzzUnPx35o7ZBCJUvX8EjZ+mSHs01OTrNag0LAEqRWQ+TuOIhny4/tSe0tJSysvLATj++OPZsmULe/fuZeXKlcyYMSOSzu/3A/DKK6/w9NNPA3DppZdyww03RNLMmDEDl8sFwDe+8Q2mT5/ONddcw6OPPsrXv/71TBVJqbzXe7SFdwRt16LcMPhMK9KUOmlDCaFDtatw0rZCiwlNt0PvkmRawIJmF/Rqia/khT/bwOpSqO/t5v6DZ7QZNCpDLfNWl8KxocYPAQtWj4JAX6j8Qm8omeikLWkn7eSuDycZCVAi8gRQARwsItuB28G50maMeQh4BvgvYCOwH+gR/7herzfy2eVy0djYiG3bDBw4kNWrV7c7f2yvDn369Il8HjFiBCUlJbzwwgvU1NQwb968rs24Uj1ce7UosWDoDJdz39GLdvI45MaJDCkWExBwmWjgCJD+H27QcuZzJ2m5bQSeOgkuTXH2Lmg5reo8VvtBo6zE4uYpbh4NBjjxQ8AGtwveOR5unuKmLOaUXUfSdpWM3AdljLnYGDPMGOMxxgw3xjxijHkoFJwIdW57lTHmCGNMWWLjiE6ssNOv+j17Wo/vQv3796e0tJSFCxeGsmpYs2YNACeffDLz588HYN68eZx66qkpl3PFFVdwySWXMHPmzEjNSimVnnAtKpHVBxAYdIaF5yDBM0gYdLoTLbwjiHsffKbl1LJiOT0UYYC3RsMn/Z3rRQbD6iOg0ROdnuzV6InWgN4qTTFtFGwZCv/pH532SczndUeCeyDcelp6QaOsxOLXFxUx+AwrUq7fXFSUdN6OpO0KPaIniXwzb948HnnkESZOnMgxxxzD4sWLAfjtb3/L73//eyZMmMAf//hH7rvvvpTLmDZtGnv37tXTe0p10uE/cCPFIEVAEUgxjPqRiz5Hi1N7Chk6wxl3+DXuuPehM1wMneGi1xjwHgYIjPyBxUeDYPsgpxZTPRn8bmHHQGHFOPjLSU4Q8Qvs7uN8/k9v55Sd3wULT4IPhzjzvjQOPh4IQTc0uwwLT4JtQ2DYTOHpi4o45TY3rl7QazR84XY3vcdAn7HC12708PsLOh40wuWMLXtXpD0QGWtm3hVim5kPGBBt/hhu9dYVOtsCLtPeeOMNfvjDH/LSSy+1mtbZMnTldjxQPbH1WD7SMqTPV2tTVRPguGHCc5vCV5E6Z3a5xbSjoqfneuJ+iG3Fl+vNzFUH/OIXv+DBBx/Ua09K5YhwN0D+ICzfFODQHZvYMWxMh2558Vjpn5YrFBqg8tBNN93ETTfdlO1sKKVC7l3pBCcAP64OBSe3Bf9PA1NSGqCUUqqTfLU2964MsLc5YUI7wanYBU1BOPcI4coTPW2mLWQaoJRSqhOWvBvgsdUd78HbJfDjbmqW3dPoFlJKqQ7y1dppByeXODUlAfp44LYKDU7p0hqUUkp1UFVNmj2iA0EDb35s+MtFXdsNUCEouDDuq7W5ckn0CZCJw7nk5JNP7tS0ttxxxx386le/6myWlFLAF8em/9eZTo8OKrmCClDhpqC79jtPgFy0Phg3nGtBauXKla3GBYPBlNOUUt0v7tpTGveR9vagp/Q6qaC2WlVNTFPQICxYG4wb7ki1PdGf/vQnJk2aRHl5OVdeeSXBYJC+ffty4403cvzxx3PWWWfx2muvUVFRwejRo1myZAkAjz32GNOnT2fq1Kl87nOf4yc/+UlkmX379gWcZ1+dfvrpfO1rX6OsrCxuGsDdd99NWVkZEydOjDQ/f/jhhznxxBOZOHEiF154Ifv37+902ZRSjlbXntppred1wbUna+2pswoqQFVOduON6ZkjHJzA+SJd3clq+Pr163nyySf597//zerVq3G5XMybN499+/ZRUVHBqlWr6NevH7feeivPP/88ixYt4rbbbovM/9prrzFv3jxWr17NwoULeeON1l0Rvvbaa9x1112tntD77LPPUl1dTU1NDWvWrIn0gP6Vr3yF119/nTVr1nD00UfzyCOPdKpsSqmojh7EdlcnqoWioEJ7WYnFzPGuuJoTOMFp1ngX40ssQs8Z7JBly5axatUqTjzxRAAaGxs55JBDKCoqYurUqc66y8rwer14PB7KysrYsmVLZP6zzz6bwYMHA05gefnllznhhBPi1jFp0iRKS0tbrXvp0qV8/etfp3fv3gAMGjSI+vp61q5dy6233kp9fT179+7l3HPP7XjBlFIRvlqb5qBzTakljasBs8stDU4HqKAClK/WbhWcwKlJPbk2yJGDhOHe5PO2xRjD5Zdfzs9//vO48b/61a8ij8ywLCvy+A3LsggEokdiknCaIHEY4h+3kbjuZOlnz55NdXU1EydO5LHHHmP58uUdKpNSKiq2KyML23nmRRun9xL70lOdU1DhPfYaFNDqdN/9nbwGdeaZZ/LUU0/xySefAFBXV8fWrVvTnv/555+nrq6OxsZGqqurOeWUU9Ke95xzzuHRRx+NXGOqq6sD4LPPPmPYsGG0tLRon31KHaDY/w4bq83gNKQ3Gpy6SEEFqNhrUOHTerHDnW0KOm7cOO68807OOeccJkyYwNlnn83HH6f/QOBTTz2VSy+9lPLyci688MJWp/faMnXqVKZNm8YJJ5xAeXl5pAn5z372MyZPnszZZ5/NUUcd1eEyKaWiEq9ft5dWdY2C2pLhJ0JW1QS4erKb8SUWRwwSqmoCVE52LmZ25hoUwKxZs5g1a1bcuL1790Y+33HHHSmnHXLIIVRVVbVaZjhNRUUFFRUVKedP7Dy2vr6e7373u3z3u99ttczEfCil2ldWYjHzGIv5qxpp8USvA1h2EHG5CIZam+t1p65VUAEKnC/a3GlFKYeVUiqRr9ZmwdstccEJwLZceAT6FjnNyTU4da2CC1C5Zvbs2cyePTvb2VBKpeCrtfnJ8gA2Mef4jA3iBKMWG4pcejNud9AtqpRSKYRb79mGaK8RxoSCkzN8INevVds0QCmlVApxLX9FQjWncAs+wRK9Gbc7FexWbakzvH9rCy172u9LSylVmJzWezH/ERL9y/S64A59dEa3Ktgtu3NhkH3rDTsXBttPrJQqSGUlFjP3voHXvy9uvNdlIr3PqO5TkFu3pc5Q96INBupesLu8FtXeIy2qq6tb9amnlMo9vp1BFrjG4/fG9+TiDwpPrg2yNseegNDTFGSA2rkwiAn3lm+T8VqUBiilcl+49Z6/qFd0pIkGpAPpfUalp+ACVKT2FP5eBbqmFnXXXXfxuc99jrPOOosNGzYAyR95sXLlSpYsWcL1119PeXk5mzZt0kdjKJVjIq33kCSt9xzaeq/7FVyAiq09hR1oLWrVqlXMnz+ft956i6effprXX38dSP7Ii5NPPplp06Zxzz33sHr1ao444gh9NIZSOabt1nto670MKbitu2dFTO0pLAB7/tX5c8kvvfQSF1xwAb1796Z///5MmzYNgLVr1/KFL3yBsrIy5s2bxzvvvJN0/nTTKaUyo3KyG6/EPphQW+9lQ8Ft4YOmWK37z3DDQacd2KZI9ciLqqoqfD4ft99+O01NTUnnTTedUiozykosZm5/PknrPbT1XgYV3FYeOsMVezAEOAdHQ2ek2VVxElOmTGHRokU0Njby2Wef8de//hVI/ciLfv368dlnn0WG9dEYSuUW384gC4ZMSdJ6D229l0EFF6A8g4RBp8fUotww6AwLz0Gpn+/SnuOOO45Zs2ZFHpfxhS98AUj9yIuLLrqIe+65h2OPPZZNmzbpozGUyjFVLzfGtd7z+qMNl7T1XuYUZBOUoTNc1L1oYzjw2lPYLbfcwi233NJqfLJHXpxyyilxzcxTPRpDKZUdlRv+zJzhM/B7++D172PWuqd58oRL8Qe19V4mFVwNCmJqUXLgtSelVA9jDGWP3cPNd1/AkF1bueWX53N+mZebp7gZ0ltb72VSwR4GDJ3hommb6ZLak1KqB3n7bdi4kTI2Mvf7Y6GoCL68hLIB+uy4TMvYYYCITBWRDSKyUURuSjJ9pIi8KCJvicjbIvJf3ZkfzyBhzJ0erT0ppeI99VT88DnnwIAB2clLgctIgBIRF/AAcB4wDrhYRMYlJLsVWGCMORa4CPhdusu3LEubZh+gpqYmLEtPWyjFc8/FD194YXbyoTJ2im8SsNEYsxlAROYD04HYDukM0D/0eQCwI92FDxo0iLq6urim2521bds2RowYccDLyabOlMGyLAYNGtRNOVIqT+zZA2+8ET/uvPOykxeFGNP9z0MSka8CU40xV4SGLwUmG2MqY9IMA/4JHAT0Ac4yxqyKXU5DQ0Mks8uXL+/2fCulCsuwV15h0i9/GRluOPxwlt93XxZz1HNVVFREPg8YMCD5tRZjTLe/gBnA/8UMXwrcn5DmWuC60OeTcGpXVmya+vp6E351l+rq6m5bdqZoGXKDliE3dKgM3/mOMU63sM7r2mu7L2Md0BP3Q8L/edLYkamLDtuB2HNOw2l9Cu+bwAIAY8wrQDFwcEZyp5RSAEuXxg+fdVZ28qGAzLXiex0YIyKlIlKE0whiSUKaD4EzAUTkaJwAtStD+VNKFbotW2DjxuiwxwOhXmFUdmQkQBljAkAl8BywHqe13jsi8lMRmRZKdh3wLRFZAzwBzDYmAxfIlFIFz1drc+XK/vjGneYMjzuNKx/YiG9f7yznrLBl7EZdY8wzwDMJ426L+bwOOCVT+VFKKYg+nNBv9WXODYuY+Ze7WHDhLfi9fZizIqA9R2SRbnWlVEGLfTih39snEpzA6Ri2SjuGzRoNUEqpgpb4cMLYR2x4XXC1dgybNRqglFIFrazEYuaelfpwwhykW14pVdB8tTYL+p6gDyfMQRqglFIFrerVAH53cWQ4tialDyfMLg1QSqmCVjng/UhQ8vr3Meuf9+ENPYVHH06YXRqglFIFa8m7AX7y8RFULP8jQ3ZtpWL54/zpizdSMUr04YQ5QLe8UqogLXk3wGOrbWyxeO6cKxmxfR3PnfMdbMvFc5sMXxxraXDKMt36SqmC9IfVMY0fRHizfCpItFPtx9do44hs0wCllCo4vlobK/EBDxI/4vKJ+veYbboHlFIFp6omQLCNnj49Fnz5KG0ckW0aoJRSBadyshtX8kfkAdBiw1/f1ebl2aYBSilVcMpKrDZrUAB/0GtQWacBSilVcJa8GwDajlCX6TWorNM9oJQqOE4LvphzfEkePTdNr0FlnQYopVRBSacF3+xy/WvMBboXlFIFpb0WfMUurT3lCg1QSqmC8sWxbf/ttdhoD+Y5QgOUUqqg/P29toNP0GgP5rlCA5RSqqAcN0ySNooIc4n2YJ4rNEAppQqGr9bmuU2mVaOIWH2L0E5ic4TuBaVUwbh3Zfun7i44Wv8Wc4XuCaWUitHeNSqVORqglFIF49qT3Vht9CDhsfT6Uy7RAKWUKihiUteQ/nuCPqQwl+ieUEoVjLtfDhAUV8rpi9br6b1cogFKKVUQfLU2+1oSRrbR3FxlnwYopVRBqEq8+da0bm5+7cl6/SmXaIBSShWEysluvBJzCi9JB7F6/Sm36N5QShUMtx3A09wUN85jwWUTXdpBbA7SAKWU6vF8tTZ3/ivAPimipag4blqLDU/4gtpBbA7SAKWU6vGqagK0tBF/WmztIDYXaYBSSvV4lZPdeNr4t9MbdHOTBiilVI9XVmJx0bA9eJob48Z7LOjjgVtPc2sDiRyUsT0iIlNFZIOIbBSRm1KkmSki60TkHRH5c6byppTq2Xy1Ngs+6kdLUa+48S02BGxI3be5yqaMBCgRcQEPAOcB44CLRWRcQpoxwI+BU4wxxwDXZCJvSqmer6omgJ/oKTyvf1/ksz+o159yVaZqUJOAjcaYzcaYZmA+MD0hzbeAB4wxewCMMZ9kKG9KqR6ucrIbb4vTvNzr38esp+7Ea5yg5HXp9adcJSYDXX2IyFeBqcaYK0LDlwKTjTGVMWmqgfeAUwAXcIcx5h+xy2loaIhkdvny5d2eb6VUD2EMI+/6Xx667D6ufvAKxq9fwR/mPs3S/mdwStHbDHPVZTuHBaeioiLyecCAAcnPshpjuv0FzAD+L2b4UuD+hDR/AxYBHqAU2A4MjE1TX19vwq/uUl1d3W3LzhQtQ27QMuSG6upqYz780BincyPn1bevMcFgtrOWth6zH2Ik/J8njR2ZOsW3HRgRMzwc2JEkzWJjTIsx5gNgAzAmQ/lTSvVkq1fHD5eVgaWt9nJdpvbQ68AYESkVkSLgImBJQppq4HQAETkYGAtszlD+lFI92Zo18cPl5dnJh+qQjAQoY0wAqASeA9YDC4wx74jIT0VkWijZc8BuEVkHvAhcb4zZnYn8KaV6uMQa1MSJ2cmH6pCMNV0xxjwDPJMw7raYzwa4NvRSSqmuozWovKQnYZVSPZq7sRE2boyOEIHx47OXIZU2DVBKqR6t39at8SPGjoU+fbKTGdUhGqCUUj3agM0Jba30+lPe0ACllOrRBmzZEj9Crz/lDQ1QSqkerX9igNIaVN7QAKWU6rmCwdYBSmtQeUMDlFKq59q4EXdzc3T44INh2LDs5Ud1iAYopVTPlXiDbnm508xc5YUOBSgR6Ssiw0Wkb3dlSCmlukziDbp6/SmvtBugRGS8iNwvIpuBBuBDoEFENolIlYiUdXsulVKqg3y1NleO/h6+cac5w+NO48rjb8FXa2c5ZypdbQYoEXkC+DPwMXAJcDBQFHq/FPgImCci87s5n0oplTZfrc2cFQF29T2EOTcsYtGXr2PODYvYJX2YsyKgQSpPtNcX35+NMX9NMn4PsDL0+rmIfKnLc6aUUp1UVRPAH3Q++719WHDhLfi9Tu8R/qAzfe60oizmUKWjzRpUiuCULN3fuiY7Sil14Conu/ESjAyHgxM4j3i/Wh/xnhfSbiQhIkER+ZOIeBPGf9r12VJKqc4rK7GYWf8KXv++uPFeF8wa72J8iTZgzgcd2UtNofeXReTQmPHaZlMplVN8tTYL+p4YV3MC5/Tek2uDrNVrUHmhIwEqaIy5BHgKeE1ETgqNN12fLaWU6ryqmgB+d/RkT2xNyh+E+2sC2ciW6qAO13ONMb8EvgUsEpEruj5LSil1YCqPM5Gg5PXvY9ZTd+K1nGNpr8u5RqVyX0cCVORUnjHmWeA04DpAH6yilMopZf9Zz813X8CQXVu55Zfnc/6ahdx8mochveHmKW7K9BpUXujIYcTY2AFjzAYRmQQc17VZUkqpA7RmDWXr/sXc74f+tr70JcpKLG1anmfau1H3kPBnY8zHidONMZ8ZY/4lIiXdkTmllOqUxD74tIujvNRePfdFEfmdiJwkInFpRcQSkc+LyO+AZd2XRaWU6qDEPvj0ERt5qb0AdSywDvhf4DMR8YnIShHxAXuBh4C16Gk+pVQO8NXaXLmkGd9+59K4b9xpXPnb9/CNnpTlnKnOaPMalDGmGagCqkRkJDAeGIjT1dHbxpiPuj+LSinVvnD/e/4gzPnu48w85K5IF0dzNsHNh9naOCLPdGRv7QSGA6cCF+P0wfe4iDzeLTlTSqkOSKf/PZVfOhKg/gBcA3wGbEp4KaVUVlVOduN1RYe1/73815E9NhUoNcbUd1dmlFKqs8pKLGaOd7Fg1f64XiQ8QT+zJvbW/vfyUEf22IeAt91USimVBb5amwVrg3HBCaDF5dX+9/JUmzUoETkjZvBxYLGI3AfUxqYzxrzQDXlTSqm0xV6DAqeLo9hrUPfrM6DyTnun+B5JMm5OwrABRndNdpRSqnMqJ7uZs9yP37gi/e89OeM2/EW9tP+9PNVeM/PSTGVEKaUORFmJxc17n6dq/9Fc/eAVjF+/giOG9+Hu067jhin9tIl5HtJDCqVUj1H25nPM/d306PChHr7aazllJdPbmEvlKj2kUEr1HNoHX4+iAUop1TMEg60D1LHHZicvqktogFJK9Qzvvw/790eHDz4Yhg/PXn7UActYgBKRqSKyQUQ2ishNbaT7qogYETkhU3lTSvUAb74ZP3zccSCSPK3KCxkJUCLiAh4AzgPGAReLyLgk6foB3wdqMpEvpVQP8tZb8cN6ei/vZaoGNQnYaIzZHOohfT6QrFnNz4C7gaYM5Usp1VMkq0GpvCbGmO5fichXganGmCtCw5cCk40xlTFpjgVuNcZcKCLLgR8ZY96IXU5DQ0Mks8uXL+/2fCul8oQxnHfJJRTt2xcZtfTBB9k3bFgWM6XaUlFREfk8YMCA5OdijTHd/gJmAP8XM3wpcH/MsAUsB0aFhpcDJyQup76+3oRf3aW6urrblp0pWobcoGXIoA8+MAair379jAkGjTF5VIY29MQyJPyfJ40dmTrFtx0YETM8HNgRM9wP52GIy0VkC/B5YIk2lFBKpSXx9N6xx4KljZTzXab24OvAGBEpFZEi4CJgSXiiMabBGHOwMWaUMWYU8CowzSSc4lNKqaT0+lOPlJEAZYwJAJXAc8B6YIEx5h0R+amITMtEHpRSPZi24OuRMtYXnzHmGeCZhHG3pUhbkYk8KaV6CK1B9Uh6klYplbd8tTZXLmrEN+hzzvC407jy/vcjwyq/aYBSSuUlX63NnBUBdvldzLlhEYu+fB1zbljEroNHMuffBp8+QTfvaYBSSuWl2Cfo+r19WHDhLXFP0K2qCWQxd6oraIBSSuWlysluvK7ocDg4AXhdcLU+QTfvaYBSSuWlshKLmeNdeJs5gnIJAAAYw0lEQVQb48Z7xWbWeBfj9Qm6eU/3oFIqL/lqbRb4AviLesWN9xuLJ9cGWavXoPKeBiilVF6qqgngt6NduHn90X74/EG4X69B5T0NUEqpvFQ52Y3XbgGc4DTrqTvxBv3OsMuZrvKb7kGlVN4q9u+l2O/nut9eihGhWGyKvXDtyW7K9BpU3tMApZTKO+F7oPy9DsJr7eP9I090mplbvfDqmb0eQw8xlFJ5R++BKgwaoJRSeadyshuviQYhvQeqZ9IApZTKO2UlFjO3/iOu5R44wUnvgeo5dC8qpfKOr9ZmwWFnxtWcwDm9p/dA9RwaoJRSeafq1Rb8nugNunoPVM+kAUoplXcq+78XCUpe/z5mLXsg0i+f3gPVc2iAUkrlFV+tTdVHw7j4yTsYsmsrFz95O89WfIOLyyyG9Iabp+g9UD2F7kWlVN6IPAPK3Z8nZt3B1Ofn8sSsn7Cr98E84bOpnKzBqSfRPamUyht6/1Nh0QCllMoblZPdeC0TGdb7n3o2DVBKqbxRVmIx0/We3v9UIHRvKqXyhq/WZoG/VO9/KhAaoJRSeaOqJoDfKooM6/1PPZsGKKVU3qic0BJ//9NTd0auSen9Tz2PBiilVF7w1dpU1QTi73/6YiUXT3Dp/U89lO5NpVTOi9z/ZHrF3/808DC9/6kH0z2qlMp5ev9TYdIApZTKeXr/U2HSAKWUynllJRYz3e/r/U8FRveqUirn+WptFjSO0vufCowGKKVUzquqCeB36f1PhUYDlFIq51WOa9T7nwqQBiilVE7z1dpUvWHi73/60tV6/1MByNheFZGpIrJBRDaKyE1Jpl8rIutE5G0RWSYih2cqb0qp3BS5/4ne8fc/DThU738qABnZsyLiAh4AzgPGAReLyLiEZG8BJxhjJgBPAXdnIm9Kqdyl9z8VtkwdekwCNhpjNhtjmoH5wPTYBMaYF40x+0ODrwLDM5Q3pVSO+uJYC9D7nwpVpgLUYcC2mOHtoXGpfBN4tltzpJTKab5amyd8NiBgTKvpU0ZZev9TDycmyY7v8pWIzADONcZcERq+FJhkjLk6SdpLgErgNGOMP3ZaQ0NDJLPLly/v1jwrpbLrqcYK9tGrjRSGc4teY6irLmN5Ul2noqIi8nnAgAGSLE2m6sfbgRExw8OBHYmJROQs4BaSBKdE06dPb2typy1evLjblp0pWobcoGU4MKNrbW5/sQVI+t8FCG+6JzN3WlGK6Q7dD7khsQwNDQ3tzpOp+vHrwBgRKRWRIuAiYElsAhE5FpgLTDPGfJKhfCmlclRZicXZ9uakp/dA738qBBnZu8aYgIhUAs8BLuBRY8w7IvJT4A1jzBLgHqAvsFBEAD40xkzLRP6UUrnFV2tz78oA+80IsJLXoC4us7SJeQ+XscMPY8wzwDMJ426L+XxWpvKilMpd4Xuf/EHAlfr03d/fs5l2VObypTJPDz+UUjkl9t6nVPT0XmHQAKWUyimVk914XcmneSwY4NXujQqF7mGlVE4pK7GYeYyFt3l/3HgvQS4uc/H7C4o0OBUI3ctKqZyy5N0Af3w7iL+od9x4Py599lOB0QCllMoZvlqbx1aHeo8Ii2lmrs9+KiwaoJRSOePelQnBxxiQ+Gbm2jiicGiAUkrlBF+tzf6WhJEJwWl2ud77VEh0TyulckJVTYCWNi4v9fHAtKO09lRINEAppXJC4qM1EjUH0QYSBUYDlFIq65a8G2jdOCJBi60NJAqNBiilVFZFW+61zWNpA4lCowFKKZVVd7+cXq2otwdtIFFgdG8rpbLGV2uzL7HlXhJeF1x7staeCo0GKKVU1lSleU1JH61RmHSPK6WypnKyG08a/0J/f09b7xUiDVBKqawpK7Fw2e2f49PGEYVJ97pSKuN8tTZVNQG+eCQ0GXdbrcvpo40jCpYGKKVURvlqbe78l9NrxGNrEvraS9L33g2n6t9UodLDEqVURt27MqZLI5G43sq17z0VSw9NlFIZFUhs75AQlPp4nHueKifrU3MLnQYopVRG+Gpt7l0ZoKmNluUWcOOpbsZrYFJogFJKZYCv1mbOigD+YNvptDG5iqWHKUqpbnfvyvaDU9j/JD60UBUsDVBKqW7lq7XZ25ztXKh8pKf4lFLdxldrc8eLgTae8hTPJdrnnorSb4JSqltEn/GUvr5FelOuitJvglKqy6X7jKdY2mO5SqTfBqVUlwk3Jf/Un/48LnFqTteerPc9qXgaoJRSB8xXa/PzFQGaYlvqJem2KJliN/z+gqLuy5zKWxqglFKd5qu1ufvlQPKHDqYRnADcWmlSKWiAUkp1WJuBqQM8ll53UqnpN0Mp1a4l7wb4w2obAxS7oDl44L0+CHDraXrdSaVWcAGqpc6w5d4Ao65z4zkovVMQ+zfbbLwtwJF3uuk9Kr0fU3g9h33TxbaHAogIpTe2XmdLnWHzL1sQEYZf6WL73CAGw+gbPXFpP10dZPNPgyBQVAIt9eA9FEZe5Wb73CB2k6HpE+h1KHiO6cf7t7a0KmNs2Ru32mz+WdBZ1p5Qh9ICvQ6DEaFlBpsMLbtg1A0udj5hE9xn8O+A0be76D/B1WpbdmbbdmYelZ7Y7+BHjwTb3MYtdYZ37mph514Y9m0Ls8DmwTJobJmK64pm3DbMdsGLx8Csfzvfl9394a8nwhdXgRWKVpaBQZ8BAnV9IOCGvx8PX34dBu+FJSfAtDegrh8c9j2LonsCrPkYxtzlxtNf2DynhcYdUFwC4gIDWG7ntxH+HQ2/0ilPuFyHfbP17yb2e+Xa7437PcT+5pL9Jju6ffW7231cd9xxR7bzkDa/339H+HNxcXGnlrHj8SANNQbbDwOObx1sNmzYwFFHHRU3buMdLQR2w951NkPOc3VoPXvX2TR9AC27wW5uvc4djwf59DVDy27Yu95JG0iSdsP1AUzodEpwLxCAwJ6YeRqi44pqB2J/5G1Vxtiy75xvY5pDywpGX7HLDDaACUDDG4bmnRD8zFlOw+uGkq+4Wm3L9rZtW9spcZ5k+yHfZKsMvlqb215oYdBzNqyGnattgltgc63NnLogowYKJX0lkvam51v49Embkk3Qdz/UrjH0/hikGQ7fJRy1A/o1Qf9GGLsDigPgNs64kf+BYfXO535N0NcPLuO8+vqdeUb+B4Y2gNuGz30ERTb0bwLvFkPTViDg/Laad8Gnbxjne9zgfBcDe4j7bYQ/N2525gm/J/5uYr9XjRsEa8OAuO9p+DeX7DeZrs583zurJ/4e/P5oU8/i4uKfJJsnY3VrEZkqIhtEZKOI3JRkuldEngxNrxGRUV25fl+tzQ/nN7P7BRsM7F5mc838Zny1qU9ULHk3wPceaqZpmzPctA2++1AzS95N3VdYeD27ljnrCc8LUPu8zVV/dtbpq7W5al4ztc9H1x+bduc/nbRzVjRz3W+aCe5Lvr7YecKKGvqBgV1LnTIueTcQV/ad/7BTLi/ZMhPTBvfBj+9u5pOldmQ933soWuZ0tm1n9odqX7hT1sbdUPy6cxrN84nzXvwG7N8Nc1YEnO/gR83cubyFYD0cs9FJI8Ahnzp/DOUfQPnm6HgBerXEDx/yafxwsldsGo+JPjzXH/M982+D3UtT7/vEtJiE95Ddy2z2f2BT92L0e9X3/eFgoO4FZ9ruF6LrqVtm07In3X4uolrqTGQddS90bhmqfRk5xSciLuAB4GxgO/C6iCwxxqyLSfZNYI8x5kgRuQj4JTCrK9Yf/tGesQoCQYMbIRA0HLNKmGMaufnTf1DW8jEApW+/Ddu2saR4PI/1OYkrXxWcEw3O+/mvCnMHBOHll5nWtDZ+PZ5hzOk/lTPe9GDbBivhOdZWEMrfgp+Kc+RwzmpX5NRIIpftpH1WDD96paOnD5z0tu2U8TET5Lw3JVL2rtjpX6kBI852sW3D9FclUuZU2zYsdjul2h+lq56DbUmib1cxXfCHkriMhOHRPh9s2RIdn+w99pVsnDFg2/GvYDD6HghE3wMBqk76Cf5egzljXeunqIuBKevg2eOhauFmMIaWIYdz1jqnxpPIZbf5JPaul2Znsm0KwNbfBDB2dFiMcxxubGcaMceXJgA7FwYZ8e2O/Sp2LgxG1mHszi1DtS9TW3QSsNEYsxlAROYD04HYADUduCP0+SmgSkTEmAP/J6mqCeDZ6xwRum3nJ+e2hfItsGKch6rmCcz9/lcBmBCa5/E/7aOkXkJHf848gjM8pF54vP8kpn3r9Pj1/PY9PE2euPXE/sDDR6UrxrkQnKPT2CpssrRbDpbIUWsyycaHx4XLuGq0pMxTOstMNuw2ICa6LWO3U6ptG5ZsO7Wa5+H4efJRWRbWWfnv97jv+9WUf9Abd+gPNPp9gPItUHPkfq7+3TcxIvz6B4sp39wr6fcw2emV9r4byWQ0yCXUqDDR7yWBhGmh6XXLbIbOMB26blr3oh0NdAGnFtWRZaj0ZOoU32FA7Fdje2hc0jTGmADQAAzuipVXTnZTsT75EeXpvhaufvCKVvNcNu/HnP9q8tj4lRrD5X+6sfV6HvoWFb6WNn+QLhtOWxtkytpA0qPWxLTnv3Zg8VkMXPCqyeyfBG1v21Tbqa15VHrK1v2Ly559G0lxXCe24bJn1jB+/QrK1v2L2c+swdUVtck8Fq5FpSu29hRZht2xZaj0SBdUUNpficgM4FxjzBWh4UuBScaYq2PSvBNKsz00vCmUZnc4TUNDQySzy5cv71AeDv3zOXhaWjdwsE0Tx1UPSDrPWxc0RY++YhgMxy5K3kjjzfMbsKTtBhy2aQJoN114Xcny0BFdsYzOaGvbptpObc2j2ucbdxr7j3kObyD1/va7DX3WnoPtsmg8+h9tpi0UtqeFbV9bmlbaEX8+C6vFc0DLUFBRURH5PGDAgKRfwkyd4tsOjIgZHg7sSJFmu4i4gQFAXaoFTp8+Pe2V+2ptXv93gAmbiZz2AAhY8PYoD0U338v4hk0AbP7gA0aXlrLk0Cl4N8Ahn8XXvAzwSX/4613z+fKOl+LXM+BIVjd5mLA1fj2xbODNI4rAwHEftF2FtYGghE6npV3aeAELdveBwftS56kzwlflUg0n27ZhqbZT7Dy9Vy9ldGlp12U4mTR7OujQMmKGN23ezBGjR8ePT/YefiUOi4BlOa/wZ5crOs7tdoZdLuezx0NV369ywlvCsR8k398BC94+XHjjv/8JIpzwEhy/qcB7jRYYcqaX49L8T9m2M8DuZXbctSzcHVtGRy1evLhD/3m5KLEMDQ0N7c6TqQD1OjBGREqBj4CLgK8lpFkCXA68AnwVeKErrj+Bcw2q8Wgo2xw/3gi8eIyLmsFXMXea0xeYb/FiRk+fzuNPNjPkIPj2862X9/RkYfegC/jyzfFtOKqWNNO4G8q2ps5L0IJ/HeM0n5i4xblvpK201ScKX61Jr5zJGIFFnxe+uazzy+jsehO3bViq7RQ7z39NGs3oPP9Brl28mCMyXIbKWpv7AgHKP0g+3QjUlMEPPu/UAH69J+BcCy3gs3zihqEz0rt9BJy0dS/acc+4Eqtjy1DpyciBU+iaUiXwHLAeWGCMeUdEfioi00LJHgEGi8hG4FqgVVP0zqqc7KalL6wudY4gwXlfPQoCfZ3piS6baFF7kFNbCn8Rw7WnXQc509NZT+yX2MaZ1tQbGnvD6tHxd+MnS7v+cGj0xE8jxTyJ48Jl/OSg1nlq6/8ocVqy4YDEb8tP+qe/bTuzP1R6ykosfnCumw1jwIT+LyPfXxdsOBJ+cK7Te0NZicUPp7pZNyb199Ck/ajBHCHgHUH00FtiyuAOTZP49IPOtDrUuMEzSBh0uhVdhxsGndGxZaj0ZKxmb4x5xhgz1hhzhDHmrtC424wxS0Kfm4wxM4wxRxpjJoVb/HWFshKLm6e4WXc8uEM/WrcL3jkebp6SvKuVaUe5mV1usfjz8eOrPw+zyy2mHdX6TzR2PVaSLWu7YPWxcFuFm9sq3KwuBzvFHghaTtoTDoW/nNTREjs/SMtyyji73Iore+q7uNL39OToWSnLgsWfj5a5vW3bmf2h0ldWYjHreg9WaNtGWuW5YNYNnrjtW1Zi8bUbPFgxl1Qif7MeMJLB+9K6ogLihsOvcSNWdNiE7uUQy5kWe96oo7WnsKEzXJF1aO2p+xTMP0FZicWvLypi8BkWCAw+0+I3FxW1+Wc47Sg3v/tOEcWhq2fFI+DB7xQlDU6J6xlyprOe4pgrbyVnWzzwtaLI0esD/11EydnR9cemHXqOk/bmKUX8zzVFuPokX1/sPGHNA5wLZ0POcso47Sh3XNmHTrVSLi/ZMhPTuvrAz28o4pCzrMh6fvedaJnT2bad2R8qfZGjfInWGlId5XsGibMfQsLpB59psXfs9ri0VsJ3wZvk+5coLo07+XjvCBh8Vup9n5g2tlyx0wafadG71IqUffCZFnvHbI+Uv3epFVfWjtaewmK3r9aeuk/B/RsMneGiz9HSoSOew3/gxuoNh/8w/VNP4fUcfo2bXmOg99jk6xw6wxWZfvg1bnqPFXqNbX1ENuq66OFw0VCQYigeTWSe4pFAMfQaDbunvJ20jLFlH3WdK7osL1AEeJ35w8v0jgSrF4z6kcsZPsxZ/6jrXUm3ZWe2bWfmUemJ/Q62t40Tv4fh9J9O2ESvMc73otdY57sgxUBR/Hel12hn2DsS8DrfqeKR0eUVlzrfpZE/sJDi6Ly9RoP0cn5bQ2e46DUaKIbiw500xaOjy0jMX+x74u8m9nv16YRNrb6nbf0mO7p99bvbjYwxefOqr6834Vd3qa6u7rZlZ4qWITdoGXKDliE3JJYh4f886X9+wdWglFJK5YeM3KjbVWJv1FVKKdUzpLpRV2tQSimlcpIGKKWUUjkpr07xKaWUKhxag1JKKZWTNEAppZTKSRqgQkTkHhF5V0TeFpFFIjIwNH6UiDSKyOrQ66Fs5zWVVGUITfuxiGwUkQ0icm4289kWEZkhIu+IiC0iJ8SMz6f9kLQMoWl5sR9iicgdIvJRzLb/r2znKV0iMjW0rTeKSJf175lJIrJFRHyhbf9GtvOTDhF5VEQ+EZG1MeMGicjzIvJ+6P2g9pajASrqeWC8MWYC8B7w45hpm4wx5aHXd7KTvbQkLYOIjMPpQf4YYCrwOxHJ1dvf1wJfAVYkmZYv+yFpGfJsPyT6dcy2fybbmUlHaNs+AJwHjAMuDu2DfHR6aNuf0H7SnPAYznc81k3AMmPMGGAZaXQIrgEqxBjzT+P0ug7wKs4zq/JKG2WYDsw3xviNMR8AG4FJ2chje4wx640xG7KdjwPRRhnyZj/0EJOAjcaYzcaYZmA+zj5Q3cwYs4LWz/ObDvwh9PkPwPntLUcDVHLfAJ6NGS4VkbdE5F8i8oVsZaqDYstwGLAtZtr20Lh8k4/7IVY+74fK0KnjR9M5NZMj8nl7xzLAP0VklYh8O9uZOQAlxpiPAULvh7Q3Q0E9eEdElgJDk0y6xRizOJTmFpwnUswLTfsYGGmM2S0ixwPVInKMMebTjGQ6QSfLkOwu7azdX5BOGZLIu/2QbLYk43LiPo+2ygM8CPwMJ68/A/4H5wAo1+Xs9u6gU4wxO0TkEOB5EXk3VEPp8QoqQBljzmpruohcDnwJONOEbhAzxvgBf+jzKhHZBIwFsnKxsjNlwDlyjH3owXBgR/fksH3tlSHFPHm1H1LIqf0QK93yiMjDwN+6OTtdJWe3d0cYY3aE3j8RkUU4py7zMUDVisgwY8zHIjIM+KS9GfQUX4iITAVuBKYZY/bHjB8SvpAtIqOBMUCXPUyxK6UqA7AEuEhEvCJSilOG17KRx87Kp/3QhrzcD6E/k7ALcBqB5IPXgTEiUioiRTgNVJZkOU8dIiJ9RKRf+DNwDvmz/RMtAS4Pfb4cSHWmIaKgalDtqAK8OFVogFdDLcWmAD8VkQAQBL5jjEm8+JcrkpbBGPOOiCwA1uGc+rvKGBPMYj5TEpELgPuBIcDfRWS1MeZc8mg/pCpDPu2HBHeLSDnO6bEtwJXZzU56jDEBEakEnsN5Xu+jxph3spytjioBFoV+z27gz8aYf2Q3S+0TkSeACuBgEdkO3A78AlggIt8EPgRmtLsc7epIKaVULtJTfEoppXKSBiillFI5SQOUUkqpnKQBSimlVE7SAKWUUionaYBSSimVkzRAKaWUykkaoJRSSuUkDVBK5QgRmSUie2NefhFZnu18KZUtGqCUyhHGmCeNMX2NMX2BQ3H6Gnwiy9lSKmu0qyOlcoyIWDgda24zxnw32/lRKlu0BqVU7rkL6Ad8P9sZUSqbtDdzpXKIiFwEXAycaIxpyXZ+lMomPcWnVI4QkWOBfwJnG2NWZzs/SmWbnuJTKndMBw4CXo5pyfdstjOlVLZoDUoppVRO0hqUUkqpnKQBSimlVE7SAKWUUionaYBSSimVkzRAKaWUykkaoJRSSuUkDVBKKaVykgYopZRSOen/A+VqDkK4lDG2AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "def plot_boundary(Zs, Hs, y):\n",
    "    data_fit = np.concatenate((Zs, Hs), axis=1)\n",
    "    data_fit.sort(axis = 0)\n",
    "\n",
    "    z = np.linspace(Zs.min(), Zs.max(), 100)\n",
    "\n",
    "    plt.plot(z, logistic(z), 'r-', label='Theory')\n",
    "    plt.plot(Zs, Hs, 'X', label='empirical', markersize=8)\n",
    "    plt.plot(Zs, y, 'd', label = 'data', markersize=8)\n",
    "    plt.xlabel('z')\n",
    "    plt.ylabel('h(z)')\n",
    "    #plt.title('Logistic Regression')\n",
    "    plt.legend()\n",
    "\n",
    "plot_boundary(Zs, Hs, y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Utility function to plot labeled points"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "def plot_points(data, features, weights, label='y'):\n",
    "    plt.plot(data[features[0]][data[label]==0], \n",
    "             data[features[1]][data[label]==0], '*', label='y=0', markersize=8)\n",
    "    \n",
    "    plt.plot(data[features[0]][data[label]==1], \n",
    "             data[features[1]][data[label]==1], 'd', label='y=1', markersize=8)\n",
    "\n",
    "    plt.xlabel(features[0])\n",
    "    plt.ylabel(features[1])\n",
    "    plt.legend()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\Owner\\Anaconda\\lib\\site-packages\\matplotlib\\figure.py:2369: UserWarning: This figure includes Axes that are not compatible with tight_layout, so results might be incorrect.\n",
      "  warnings.warn(\"This figure includes Axes that are not compatible \"\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAAEYCAYAAAAJeGK1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3X+cXGV99//XZ2ZnCURYkkAIyJKQhCJB+WUQeqNk+WX42nIvRAlq/dG78AVri+jXfutdlfS2j6p4P77lLhS/Fm4RU5VSoNjYWkEMLhFbwq+ihUQUEmCTmADZsIRgsrszn/uPmdmd3czunDNn5pwzs+/n47GPzM7M2euas5v5zHVdn/O5zN0RERFJm0zSHRAREalGAUpERFJJAUpERFJJAUpERFKpI+kOAAwODipTQ0Rkmuvq6rLK7zWCEhGRVFKAEhGRVErFFF+lrq4u1qxZQ29vb9JdSTWdo6np/NSmc1SbztHUGnF+BgcHJ31MIygREUmlWAKUmR1vZk9WfL1mZp+Mo20REWlNsUzxufszwCkAZpYFtgLfDXAcAwMDFAqFJvcwGZlMhtmzZ2NmtZ8sIjKJ4QHn+etHWPDpDnKz2uf9JIk1qPOA59z9hVpPHBgYYObMmcyYMSOGbsVv7969DAwMMGfOnKS7IiItbPtdefZsdLbflaf7ytSlFtTN4i4Wa2bfAJ5w95vK91VeB9XX1zf63O7ubhYuXBhr/+K2adMm+vv7k+6GiLSo7BsHcNQ9y8jksxSyeba+t4/CgUNJdyuwnp6e0dsTr4OKNUCZWSewDTjR3XeU768MUJVZfC+//DKHH354bP1LQr2vUdlFU9P5qU3nqLZWOEf9N4+wc20BRoAOmHN+JrZRVKOz+JK+UPf/ojh62lHzmSIiMqXhAWfgx6XgBDACAw8UGN7VHsV54g5QHwD+vpkN7BlyrvvJMHuGkvsFbd68mTPOOIPjjjuOyy67jKGh1hlui0jr2H5XHp+QQ+aF4v3tILYAZWYHARcA9zSznUe3Fnhkq/PYtuQy/z7zmc/wqU99il/96lfMmjWLW2+9NbG+iEj72rWuYvRUNgK7HmyPzOfYApS7v+Huc9x98suGG2Dt5uIvZu2m6L+ga6+9lhtuuGH0+8997nPceOONUx7j7jzwwAO8733vA+CjH/0o//RP/xS5LyISr+EB54h7z4h1umx4wPnV54cDtznr7Mz+udgdMGtZe9RgaIt8xJV3DjFSikflFbanX3JW3FGcWuvIwJ0rO0P/3Msvv5wVK1ZwzTXXUCgUuOOOO3jggQc45ZRTqj7/9ttvZ+7cuRx66KF0dBRP7dFHH83WrVtDty0iydp+V54DdsyKNXU7bLr4vEuzDPy4QGU4s0zx/nbQFgHqtotzPLatwH3PFti0yxnKQy4LC2cZyxdnWHpUfZ8mFixYwJw5c/iP//gPduzYwamnnsr8+fN58sknJz3m5Zdf3u8+XYgr0lrKyQeGMfBAgXmXetMvgB1NeHACt5mbbcw+JzMui2/2uZm2uVi3LQLUzE5j2YIs7nDL48XFwYzB8sUZli2I9kniiiuu4Jvf/Cbbt2/nD/7gD9i9ezfvete7qj739ttv54QTTuDVV19lZGSEjo4OtmzZwlFHHRWpDyISr8rkg3LSQbNHUfW2WTmKaqfRE7RZsdi1mwvsG4FcBvaNNGYd6pJLLuHee+/l0UcfZfny5Rx88ME8+eSTVb+WLFmCmXHOOedw9913A7B69erUX0chImOipm6HXUeK2mZ5FIW11+gJ2ixA5TLFUdN1F3SwfHGGjga8us7OTs455xxWrlxJNhvsk8lXvvIVrr/+ehYvXszOnTu5/PLLo3dERGIRNXW7ch0prjbnXZpl5gnWVqMnaJMpvrJVPbnR21cubUzsLRQKPPzww9x1112Bj1m4cCGPPPJIQ9oXkXhNlbrdfeXUx9azjhS1TSiOoo77y1ztJ7aYthpBNdqGDRtYvHgx5513Hscdd1zS3RGRGERJ3a62jtTsNttZW42gGm3JkiVs2rQp6W6ISIzqTd2ebB0pyCiq3dPF6zW9w7OIyASjSQflj+8BU7ejrCPV22a7U4ASEZlg3qVZrPTuGHQkE7XsUD1ttru2C1D1pHiKiFTKzTYO/S+G4xx6lgUayURdR2rndPF6tV2AqifFU0QkqsoRUFnYkVC7povXq60C1MQUz6RGUTfddBOLFy/GzHjllVcS6YOI1G94wHn13xzDePWnHu6C2QjrSOV0cY2eitoqQNWb4tloZ511Fj/60Y+YP39+Iu2LSDT1vpdoHamx2iZANWNnyXq22wA49dRTWbBgQd3tisiYuNeVVXYoPdomQDVjZ8nLL7+c1atXA4xut9Hb28spp5xS9WvDhg1RXoKIVBH3urLKDqVH21yoG7VUSDX1bLchIo1Tb+mgKFR2KD3aJkDNOrtiT5SyBpQKCbvdxpIlSyK1JyJjktj2olnvJRJe2wSoZpUKueSSS1i1ahXDw8PcfvvtZLNZjaBEYhCldFAUKjuUHm3zkaBZpULq2W7jxhtv5Oijj2bLli2cdNJJXHHFFZH6IDIdNWNdOQiVHUqPtglQ0JwUz/J2G2H2dPrEJz7Bli1bGBkZYdu2bXz961+P3A+R6SZq6aAolC6eDm0VoBqd4qntNkSS04gtKOpNUS+/lzjeEqOnJEq8DQ84R9x7RlPbbKsABY1N8Sxvt/FXf/VXDeiZiITRiNJBUVLU512aZd8Ru1pi9JREibftd+U5YMesprbZdgFKpUJE2kPUtaCopc9ys40dF65P/XtJEiXeym0a1tQ2Ux2gMpkMe/fuTbobTbN3714ymVT/CkQSFWUtKC2lz5otidcZV5upTjOfPXs2AwMD7N69O+muNEUmk2H27NlJd0MktcrbXuzqC77tBSSXoh63JF5nnG2mOkCZGXPmzEm6GyLSYqZKUW/2hb5xSuJ1xtmm5pdEJLXK214Agbe9gGRT1OOUxOuMs83YApSZHWpmd5vZL8xso5n9dlxti0hrqnetoxEp6vWKM+U7idc56+zipTzjWHPajHMEdQNwr7u/BTgZ2Bhj2yLSYqJse9GIFPV6xZnyncTrnHNBBib+ChzmvLtFA5SZHQKcDdwK4O5D7v5qHG2LSGuKUuooqXJFcad8l5NIKoVJJqnHzvsLVUdQO3/Y+Ck+c2/+MNTMTgFuATZQHD09Dlzj7nsABgcHRzvR19fX9P6ISPp1334+meH9t60o5Ibp/+CPah6ffeMAjrpnGZl8lkI2z9b39lE4cKgZXR016+ElHPyro7FCFs/k2X3cFnad2dx94ub85G3M3PRmDMNxXl+0lYF3/mfT2ov6e5mop6dn9HZXV9e40BdXgFoKPAyc5e7rzewG4DV3vxbGB6iuri7WrFlDb29v0/vVynSOpqbzU1s952h4wHn++hEWfLqj6aOR/ptHqm57Mef8TOBssf6bR9j5wwJzlgc/plKYczQ84Gz4+DBeEQOtE5Z8LXjhgLDntxFthtWI30ulwcHB0dsTA1Rca1BbgC3uvr70/d3AaTG1LSIN0mrrK3HubtuI6uthz28SFd/jXPeKJUC5+3ag38yOL911HsXpPhFpEa24vhJn6bOo6df1nN8k0szjXN+LM4vvauA7ZvZz4BTgSzG2LSIRTZfSQfWKmvJdz/lNKp0+ru1IYgtQ7v6kuy9195Pc/WJ33xVX2yISTZSU7yhtli/SLQtzsW7cokx91Xt+k0qnL49unXAlqMJSJQkRqSmJtY6kdtStV+V+dECofenqfa1R2mwFClAiUlO7l9RplHEXsYa4eDXKa623zSjKo1vDmjqqVYASaVHToqROQuWK6rXz/vEBJejFq1Fe67gLZ+u4YPaNTQV+/qEh3ng++HFxrUem9zctIlNqtZTvVmgzitF1pArNLs1UmfkH1JVh+cINIxTegBf+18QhXI02Y1iPVIASaUFJpHzHXTooiTI+UWy/K49P+Kzg+eClmep5rVHX6d7YVGBff/H2vn4CjaLiXBtUgBJpQUmkfMeVWtyqdq0rwMRfQz7d63Qv3DD+4CCjqLbcbkNEGiOJlG8YnzEWV+HVVkoz7zqj+vnoOjNYyaJ6XmuUtavK0VNZkFFUnGuDClAiLSbJ9OtWKx3UKup9rVHW6SaOnkbvrzGKartSRyLSOEmmX7dS6aC4Da6vPtoZfLh5JYuirA1OHD3Vur8RbYalACXSYlox/boes86u/nrS+jqj7DQb5Xda79rgAd3h7m9Em2Gl8zctIpNqtfTrekVZ00lClJ1mo/xOKzMAw2Q5zr+m+tYY8z9Ve8uM8ijK8aauRypAibSYpHaLjdvW26qvv2y9NZ1rUFF2mk3id3rQwsx+o6UDuuGgBcHCwrxLs+w7YldTPxgpQIm0oOmQ8l3vGklSdq0rVB1BBV0zq/d3WpkBGDbLceIoKsjoqSw329hx4fqmBlEFKJEWFHfKdxKirJGUxV4Oqs41KKj/d1p5gXDQC4PLKkdRYUZPcUlXb0QksDhTvpMQZY2kLM5yUFHWoMrC/k5Hr4krv7x8+Gvi5l/TQeagcOc1LgpQIi0qzpTvJERdI4m7HFSUNaiysL/TKOWVyg5amOGkb3embvQEClAikmJR1kjiLgcVdQ2q7jZjLq8UJwUoEUmtetdIGlEOanjAOeLeMwIfk8T1aa2Wih+WApSIpFo9aySNKJO0/a48B+yYFfiY6XJ9WpwUoEQk1epZI4laJqk8AjMs8MgriWuZopRXagUKUCLSdqJOt9W7fhX39WmtVg4qrPZ4FSISStj1lSTVcy1TlOm2KOtXcV+fNueC6m/hYVLb47xWLCwFKJFpKOz6SpLquZZpNFBUCBowoq5fxXl9WiNS2+O8ViwsBSiRaaae9ZWyPUPOdT8ZZs9QPJ+2o1zLNDHDLWhmW9T1q9i3JImQ2h73tWJhKUCJTDNRSuM8urXAI1udx7bFc51NlGuZJhabDVpkNmrJojhF7Wvc14qFlb4zLiJNE7U0ztrNxXeztZuaH6CirAXVu505NKZkUVyi9LUR14o1W/rOuIg0TT2lcVbeOcSKO4pfG14qvnk9/ZKP3rfyzqFAbYedHoyyFlTvduZQWtepdn+IdZ24RFmDasS1Ys0WW4Ays+fN7D/N7EkzeyyudkVkTD2lcW67OMc1Z2Z5y2FGrrTun8vCWw4zrjkzy20X5wK1HXZ6MMpaUJStOnatq/7z01g+KMoaVNS1tjjEPYI6x91PcfelMbcrItRXGmdmp7FsQZZ3L8qQKT0tY7B8cYZlC7LM7AyWDBB2ejDKtUydR4W7v1IrlQ+Kco6SKM0UVvrqq4tIKq3dXGDfCOQysG+kGGiWLZg6lXrlnUOMlOJR+e29PD0I0JGBO1d2Vj123qVZBn5cGDdACHot04HzjaFt+08lHnhs+oJMFFHOUZRj42Lu8SyImdlmYBfFAenN7n5L+bHBwcHRTvT19cXSH5HpqPv288kM7z8lV8gN0//BH0157P37lnKwvcFvdfTzy5FudvtBXHDA1LP1Q95Bf34uz4wcw4AfQp4sWfLMttc4vuNFurMv0WmTrwvN6TuZmS8ciWE4zp75v2Znz8+a+jqjHJuEWQ8v4eBfHY0Vsngmz+7jtrDrzA1NP7ZRenp6Rm93dXWN+wQRZ4A6yt23mdlc4H7gandfB+MDVFdXF2vWrKG3tzeWfrUqnaOp6fxU13/zCDt/NGEdKlvMBuu+snkTKn2b89zyeJ69IzCjA65amq05+gLY8MdDDG0b+77zKFhyU/URV6Uor7P/5hF23rf/NOScC5t7juo1POBs+PgwPgTWCUu+FvwarCjHQmP+nw0ODo7enhigYptsdPdtpX9fAr4LvCOutkWkaN6lWWxCXLBs86d1qk0P1vLGpsK44AQwtC1YqniU19mI8kFxilJeKe7STGHFcsbNbKaZHVy+DbwbeCqOtkXaVdi07RV3DHHZD4d5pBtGSu9DIwaPdMNl9w2Prgs1Qy5TTKq47oIOli/O0BHgnSdKqvjoG285HmWDvwE3onxQ3KKUV4qzNFNYcY1XjwC+a2blNm9393tjalukLVWmbQeZLrv5og7u2ZDn0WHnlM2Ag2fg0ZNg+SJjxZLmvUGt6hlb07lyabDPxVFSxaEiCSAfbpQ4Vep295XB2o5bubxS3Mc2W10jKDPLVH7Ver67b3L3k0tfJ7r7F+tpV0TGhE3bPnxmhqtOz3Hq8caTx0IBeHIBvP0txlWn5zh8ZrqmsHJzJ7n/iIDH1zl91Qrp19NF4BGUmZ0GfBU4CZhRvpviZ430jQ1F2lCUtO2yn/Y7mSVw+GvOuiUGLzpXnd7ETtdh5Z1DXLUHZjN+ts2B7a/DF+4cqvk6oThq2tvvoaavWiH9eroI85FgNfBjYCmwsPR1bOlfkWktrirflVUdyus4HZlwVR2yBl1z4fvnDnLoXEYvvg0iztc5e0/VpSBm7yFw9Yp6KosnsTOuVBcmQM0HPufuG939hcqvZnVOpFXEVeW7sqpDoRQjCh6uqsPqFZ28d0mWnX4o7zsxy+oVtUciZXG+zhndVZeCmNFN4OoV9Yp7Z1ypLkyA+i7F7DsRmSDOKt/l9vKld++8h2+33v7G+TrnX1N9BWL+p5qf21UeRTmu0VOCpvxNm9m3GPsQcwDFTLyHgO2Vz3P3jzSneyLp1Yj1oLAmSwV/qqLde95fvc16+5vE6wQ4aGGG3bPh4IGxxe7ds+GgBfEkK8y7NMvWJ19h3qUBszKk4Wp9FHl2wvfx1sAQSbHbLs7x2LYC9z1bYNMuZyhfrPK9cJaxfHGGpUcFeyPdM+T8zfoRrj6jo+bUVTlVfN0Lzm8qLgc6sAPOnj91qni9/W3U66zH0s928Ms/Kb5QA5Z+Pr5KDrnZxo4L15ObpYokSZnyt+3uXyjfNrN57r594nPMbF4zOiaSduX1IHe45fFiTZ3KKt9BhbmeqZgqnqHgw9y/aWyF5uz5xVTxoP392qPF/rrX7m+jXufwgPP89SMs+HRH4CmzgxZmOKC7eO3TAd3xjZ4kHcL8tn85yf0aVcm0Vk8Zn4nHQ7jjfto/Pn3goReDZ9Wt3VxguNTUcCF4u1Ff5/a78uzZ6KE3xJt/TQeZg+JZe5J0CfMb3+8jj5kdQvF6P5Fpq1zG54JFGe5/rsD212sHi0asB1XaM1xcn5pqPWiyY8vrV7XWknIZOPdYY8ceOGIm7PzNlC9xnNHtxb24rfi8Sz3UKOqkbzd+jUvSr2aAMrN+iuuTB5rZixMengP8fTM6JtIq6injU7mu8+yAM1KAbAYWzw6/HtQZcD2o8thfvDIWRN9yWLC1pFU9Ofo251m7Ps95ZwarRl5Wub14eVvxNFYGl3QJ8hfyIYof8v4V+HDF/Q7scPdnmtExkXZWua7zy1eKU16FQnPXgyqP/cUrY9NsYdaSKqcjgx4zOnoqJ3WMhB9FyfRUM0C5+4MAZnaYu7/R/C6JtL9q020F4IaH89zwcL7mdFs9u9tOlqJebhOqp6hHTTOvHD2VaRQlQYT56/jvpWrkE+0DtgD3uvuOhvRKpMWESRWHsem2rz+e543h4nSEAQfl4Iq3Z2tOt+Uyxcy9J37tnHakMbivdh//fJlx43pn1979H5s1Az5xRvV+R00z37WuYvRUNpLu6uCSDmGy+H4L+AxwDrC49O9ngFOBPwQ2mdmFDe+hSAsIWwKoPN12+WnZcTX1rnh7NlDJolU9OTqzsHuouENt5TrYZE4+MsetF3cya8b4+2fNgFsv7uTkI6v/jMrySuW6fZXTirX6qurgUq8wfyEZ4P3u/i53/6C7vwtYCeTd/Uzg48B1zeikSNpFKR1UT8o3jKWah0kxBypGUD7h+6nVm2ZeWdeuTPXtJIgwU3zLgQ9MuO9fgG+Vbn8buKkRnRJpBVHWZupN+a62jlROMS+brNRRJQPmZ7bxQuHN+xVknUw96fQwVtdu59rSVJ+qg0tAYQLUcxSn8iqD0MdK9wMcBuxpUL9EUi/K2kzlsc8NOMOFYgBYVCPNPEqpo7JyAFuz5uf09h4b+PXWk05fVrnHkkZPElSYAHUFcI+ZfQbYCrwZyAMrSo8fD1zb2O6JpFejUr6/urOUZh6g7FCUUkdJGh1F/bCg0ZMEFvhjkLs/ARwH/B7wv0r/Hle6H3df5+7/uym9FEmxKCWA6t02I0qpo6TMuzTLzBNMoycJLNRFCO4+DKwzG1vyNLOM+8SrHESmj3pKAEXZNgOKu+J2HwIrT8xy59N5Xg2Y6FC2Z8h5YN9pnD/kTd/8r6y8u61IUIEDlJmdBnwVOAkoJ6qWt2nRRyKZtuopARR1LalyF9yz5of/7/fo1gL9hSMCVVAXSUqYlc7VwI+BpcDC0texpX9FprWwaebFtaQc7zxm/OilvJZ0+MzmXiMU9w7AIvUIM8U3H/icu6d/slskBlHSzCeb4rvvOee+52pP8cXdX5EkhAlQ3wXeDdzXpL6INERc6ytR0szLU3z3Pbf/573li4Kli8fZX5EkhAlQM4DvmtlDwLiddd39Iw3tlUgEca2vREkznyxdfPmi5qWLN2pnXJG4hPnItAH4CvBTihfnVn6JpEbc6ytR0syTSBcv9zdLvq6dcUXiEngE5e5faGZHRKJIcn2lnhJAk61BVZYtavQaVFm5v7n+hxjuPjtwySKRuIW6DsrMLgDeD8x194vMbClwiLs/EPD4LPAYsNXdfzd0b0UmkeT6yqqeXOjtNhpRsihKfwHWbN1N71LtxyTpFfh/rZldDXwN+BVwdunu3wB/GaK9a4CNIZ4vEkjllhDlPFP34FtCRBV2u42k08xFWkGY/wWfBM539+sobv4J8AuKNfhqMrOjgd8Bvh6qhyIhRNm+Imq7EL69VixZJBKXMOP7g4H+0u3y/6IcUH0yfX9/Dfxp6eeINFS921dMFGaqrhnXQYVZgwo7rSjSaizodbdmdjfwH+7+RTMbcPfZZvanwCnu/sEax/4u8B53/7iZ9QB/UrkGNTg4ONqJvr6+Ol6GTHdD3kF/fi7PjBzDgB9CnixZ8sy21zi+40W6sy/RaRP3Hd/fcyNH8dDwybwz9zMWdWwL3OZO76JAhgwF5thgzTZ35WfyyPAStvtsiuGtXDXMmWcDvCO3gVnZqXevCdNXkbTq6ekZvd3V1TXuk1aYAHUk8M8U9316M7AJeA24yN231zj2y8CHKW5XNgM4BLjH3T8E4wNUV1cXa9asobe3N1C/piudo+r6Nue55fE8e0eKW6FftTRYbbyyax8Y5umXnLfONf7i3GDXI/VtznPT+jwFinPmVwesxwdw7dohnn557Pu3zYUvnBsse6+evlbS31BtOkdTa8T5GRwcHL09MUCFSTP/tZmdDrwDOIbidN8jQSqZu/ufAX8GUDGC+lDQtkWCGn+NT5a1m2pfrFvvVF21acUCcMPDeW54OB9oWnHjK+O/rwxWjeyrSCsKu92GA+tLXyKpU881PtV2t+0IsLtt+bivP57nN8OMjqAOzMEVb88GSm03YGYOVpyQ4Z6NBfbWmIVUuSKZTqYMUGbWz1hCxKTc/ZigDbp7H9AX9PkiYdRzjU9lCaCvDeRH769VAmhi6aC9I9DZUQxOQaf47rpsbLRzyZJwfVW5Iml3tf4HaxpOpo1qKepB3vSrlTpqdrBIok2RuE0ZoNz9wTA/zMy+7+6/E61LItGErWYeNUW9nlJHUSXRpkjcGl3n5F0N/nkioYWtZl5tXaczxLpOeVoR4Mql8awBJdGmSNz0ly1tJ2xVh8oySZnSgKtyXUcXwYokQ5UipS00Iv1a6zoi6aIAJW2hEenXSa3rqGSRSHWNDlD63yWJaET6dVLrOpWV0DViExnT6P+FX2rwzxMJpRV3i417B2CRVlHrQt2/CPJD3H1V6d8vN6JTIvVqld1iVbJIpLZaU3zdsfRCpEFW9eR4eU+Ba55dyg0nZFK78Z9KFonUVutC3f8WV0dEGuXup/PsZQb3bMhz1enpfKNXySKR2kL/7zWzg83sWDNbWP5qRsdE6lXepbYVdqetltouIkWBs/jMbAnwHeBkigVkyzusAegjnySq2g61lbvTQu0dapOgkkUikwuTZv7/Az8GzgE2AwuALwP/1vhuiYRz80Ud3LMhz7oXnN9UbFlxYAecPd9YsSSdn6FUskhkcmEC1MnABe4+bGbm7oNm9v8CTwHfbk73RII5fGaGq07PUPBh7t80Ngo5e75x1enhd5sVkeSF+ci2Fyj/T3/FzI4pHT+n4b0SqVN5/ak8+9wK61AiUl2YAPUTYGXp9t3AD4AHgQca3SmRemUNjnwT5BjmyDcxWvxVRFpP4Ck+d19Z8e1nKU7tHQysbnSnROq1ekUnX3tkmPtf7+SkIzS9J9LKAo+gzOxPyrfdveDu33b3rwEfa0rPROrUSmnmIjK5MEkSq4D/r8r9nweub0x3ROrTqmnmIjK5mgHKzM4t3cya2TmMr1i+ENjdjI6JhNGqaeYiMrkgI6hbS//OAL5Rcb8DO4CrG90pkbCUZi7SfmoGKHc/FsDM/s7dP9L8LonUb3yaufHQi85VpyfZIxGpV+AkCXf/iJnlzOxdZnYZgJnNNLOZzeueSDhZg+5D4Ozck3QfojRzkVYWphbf24DvAfuAo4F/AJYBHwUua0rvREJavaKYCLFmzXY++R4lRYi0sjAX6n4NWOXubwGGS/c9CLyz4b0SEZFpL0yAOpGxmnsO4O57gAMb3SkREZEwAep54O2Vd5jZO4BnG9khkSTtGXKu+8kwe4Z0ka9I0sIEqGuB75vZF4ADzOzPKNbk+3ytA81shpk9YmY/M7OnSz9DJHUe3Vrgka3OY9u0caBI0sJk8f0LcCFwOMV9oY4BLnH3HwY4fB9wrrufDJwCXGhmZ9bRX5GmWru5GJi0s61I8sJk8XUClwDvBo4CtlLcduNpd9871bHu7sDrpW9zpS/NoUgqrLxziJFSPCpnpT/9ko+WSerIwJ0rlREoEjcrxo4ATzS7FTge+CLwAsUR1GeBZ939DwIcnwUeBxYDX3X3z5QfGxwcHO1EX19fiO6LRDfkHfTn5/LMyDEM+CHkyZIlz2x7jeM7XqQ7+xJhKzwGAAAPxklEQVSdNlL7B4lIaD09PaO3u7q6xl25GCZA7QQWufurFffNphigZgftjJkdCnwXuNrdn4LxAaqrq4s1a9bQ29sb9EdOSzpHU6vn/PRtznPL43n2jsCMDrhqaZZlC9q3hp/+hmrTOZpaI87P4ODg6O2JASpMksR24KAJ9x0I/DpMZ0oBro/iepZIaqzdXGDfCOQysG9E61AiSQuz3ca3gHvN7G+ALUA38EfA31VUPMfd99th18wOB4bd/VUzOxA4H/hKpJ6LNFguA8sXZ7hgUYb7nyuw/XUtk4okKUyAuqr072cn3P8xxjYtdIpbcEx0JLC6tA6VAe4sZQWKpMaqnrGq51cuDTO5ICLNEGbL92PrbcTdfw6cWu/xIiIy/ehjooiIpJIClIiIpJIClIiIpJIClIiIpJIClIiIpJIClIiIpJIClIiIpJIClIiIpJIClIiIpJIClIiIpJIClIiIpJIClIiIpJIClIiIpJIClIiIpJIClIiIpJIClIiIpJIClIiIpJIClIiIpJIClIiIpJIClIiIpJIClIiIpJIClIiIpJIClIiIpJIClIiIpJIClIiIpJIClIiIpJIClIiIpFIsAcrMus3sx2a20cyeNrNr4mhXRERaV0dM7YwAn3b3J8zsYOBxM7vf3TfE1L6IiLSYWEZQ7v5rd3+idHs3sBF4cxxti4hIazJ3j7dBswXAOuCt7v4awODg4Ggn+vr6Yu0PwJB38NDQSbyz8+d02kjbtikikjY9PT2jt7u6uqzysbim+AAwszcB/wh8shycJurt7WXNmjX09vbG1q++zXn61+eZe8qFLFuQbYk24z5HrUbnpzado9p0jqbWiPMzODg46WOxZfGZWY5icPqOu98TV7tBrN1cKP67qdDWbYqItJJYRlBmZsCtwEZ3vz6ONmtZeecQI6XYUB5TPv2Ss+KOIQA6MnDnys6Wb1NEpFXFNcV3FvBh4D/N7MnSfZ9193+Nqf393HZxjse2Fbjv2QKbdjlDechlYeEsY/niDEuPavzgMok2RURaVSwByt0fYmzQkAozO41lC7K4wy2P5wHIGCxfnGnaOlQSbYqItKpp/5F97eYC+0Ygl4F9I/GsCSXRpohIq5n2ASqXgXOPNY4/rPhvR8gzsmfIue4nw+wZCp6u34g2H9h3Wqg2RURazbQPUKt6cpw4N8NTL8Fbj8iwqicX6vhHtxZ4ZKvz2Lbgo6BGtNlfOCJUmyIirWbaByiIlvJd77FJtCki0kpivVA3TaKkfNd7bBJtioi0qmkboCpTvn/5iuMU3/iPP6x2yne96eKVxz034AwXioFl0ezmtVlpz5DzN+tHuPqMDmZ2piqpUkRkP9N2iq+c8v3uRRnKE2UFxlK+p3oDrzw2U3paZbr4ZMdWHlepmW1Wqme9TEQkKdM2QJWV13NGvw+xrlNvuvjazQWGS08dLtTXZpZ86BR1rV2JSCtpqym+Z14e4bNrC3zpvAzHHz71Syuv3Uz0VMW6zj3vn3pNp5wuvmMPHDETdv5m6v5VriNVazPIOlIuUxw15fofYrj7bLa/PnWqudauRKRVtVWAuumRAg589dECN75n6uf++TLjxvXOrr37PzZrBnzijNpTZqt6cvRtzrN2fZ7zzszWrAZRuY70TMh1r8o2AdZs3U3v0tq/PpVXEpFW1VbvTlt3F//dUnUjj/FOPjLHrRd3MmvG+PtnzYBbL+7k5CODXZsUZtqsch2pPO5xwq0jhdWItSsRkSS0/Ahqsqm6yvurTdVNdtyuvdSc4qt32myyNm94OM8ND+enbDOqautlqv8nImnW8iOoL5+X4ehDqj929CHFx6v5zFlw0CSDpINyxccnc9vFOa45M8tbDjOypQiVNXjLYcY1Z2a57eLqP/ivL8zytrnVf+bb5hYfb5by2tV1F3SwfHEmdHklEZG4tfwI6vjDO7jxPXD194dGp/igGJxufM/ko5Ezujs5oxs+9I9DvDE8dv9BOfj2e6cexVRWJX/mleLIJ++1q5Ifc2iWL5yb5dq1Qzz98tj9b5sLXzi3uYkKleWUrlyq6CQi6dc271SVwQmCrUMB44JTte+nsnZzYdxaUtD07Y2vjP++MliJiEhRy4+gJkvdBqZM3Z5sPajyscnWg6KmqBswMwcrTshwz8YCe0cmfaqIyLTV8gGqWhp1Z4A06j98O9zyRHFqbqKswZWnTd7mX1+Y5dYn8jz1ElQebsBb58Llp029lnTXZZ2jZYf+9qKcMulERKpo+Sm+etOoLziuk7su6xxNcijLWjGAXHDc5COg4lpSJ0sOH3//W0trScccWjvZQWWHRESm1vIBqqzeskMTR1DVRlSTibKWpLJDIiJTa/kpvrJyGvUFizLc/1yhZgmgZqxBFXzqY1V2SEQkuLYJUGHTqL98XoavPlqomu139CHwR6dP/jNuvqiDezbkWfeC85uKBIcDO+Ds+caKJdWn+FR2SEQkuGn7jli8fqqTNx88/v7y9VNTFZs9fGaGq07P8c5jxi9gnT3fuOr0HIfPrH5aVXZIRCS4aRugVtwxxIo7hqpeP1V+rJaf9o+fRnzoxWALWPWul4mITCdtFaD2DDnX/WSYPUO1A8XNF3WwfFH1EcvyRcbNF9We/cwadB8Cn/7tLN2HMDoqqkVlh0REamubNSgYn7pdqxBqcZouQ8GHuX/TWEBbvqg4TRfE6hVjCQ1nzQ9eR09lh0REamurd8d6UrfrnaYTEZHmavkRVNTU7fI03coTs9z5dJ5Xq2xgKCIi8YslQJnZN4DfBV5y97c28mdHTd1evWKs7NCXzlfZIRGRtIhriu+bwIXN+MGNSN1W2SERkfSJJUC5+zpgoJltREndVtkhEZH0afk1qLKwpY5UdkhEJN3MPZ6sNTNbAPxLtTWowcHB0U709fXF0p8h76A/P5dnRo5hwA8hT5YseWbbaxzf8SLd2ZfoNG3UJCLSTD09PaO3u7q6xq3JpG4E1dvby5o1a+jt7Y2lvb7NeW55PE9+BHIdWT649DCWLTgilrajiPMctSKdn9p0jmrTOZpaI87P4ODgpI+11XVQ9VDZIRGRdIolQJnZ3wP/DhxvZlvM7PI42g1CZYdERNIplik+d/9AHO3UQ2WHRETSSe/IIiKSSgpQIiKSSgpQIiKSSgpQIiKSSrFdqDuVygt1RURkepp4oa5GUCIikkoKUCIikkqpmOITERGZSCMoERFJJQUoERFJpUQDlJldaGbPmNmzZvbfqzx+gJn9Q+nx9aUtO6aNAOfn983sZTN7svR1RRL9TIqZfcPMXjKzpyZ53MzsxtL5+7mZnRZ3H5MW4Bz1mNlgxd/Qqrj7mCQz6zazH5vZRjN72syuqfKcaf13FPAcNefvyN0T+QKywHPAQqAT+BmwZMJzPg78ben2+4F/SKq/KT0/vw/clHRfEzxHZwOnAU9N8vh7gB9Q3JPyTGB90n1O4TnqobhPW+J9Tej8HAmcVrp9MPDLKv/PpvXfUcBz1JS/oyRHUO8AnnX3Te4+BNwBTNxYpBdYXbp9N3CemRnTQ5DzM625+zpgYIqn9AJ/50UPA4ea2ZHx9C4dApyjac3df+3uT5Ru7wY2Am+e8LRp/XcU8Bw1RZIB6s1Af8X3W9j/RY8+x91HgEFgTiy9S16Q8wPw3tK0w91m1h1P11pG0HM43f22mf3MzH5gZicm3ZmklJYQTgXWT3hIf0clU5wjaMLfUZIBqtpIaGLOe5DntKsgr/2fgQXufhLwI8ZGm1I0nf9+gnoCmO/uJwN/A/xTwv1JhJm9CfhH4JPu/trEh6scMu3+jmqco6b8HSUZoLYAlZ/4jwa2TfYcM+sAupg+0xU1z4+773T3faVv/zfw9pj61iqC/I1Na+7+mru/Xrr9r0DOzA5LuFuxMrMcxTfe77j7PVWeMu3/jmqdo2b9HSUZoB4FjjOzY82sk2ISxPcmPOd7wEdLt98HPOClFblpoOb5mTAP/l8pzg3LmO8BHyllYZ0JDLr7r5PuVJqY2bzyuq6ZvYPie8LOZHsVn9JrvxXY6O7XT/K0af13FOQcNevvKJYddatx9xEz+2PgPooZa99w96fN7C+Ax9z9exRPyrfM7FmKI6f3J9XfuAU8P58ws/8KjFA8P7+fWIcTYGZ/TzF76DAz2wL8OZADcPe/Bf6VYgbWs8AbwH9LpqfJCXCO3gf8oZmNAL8B3j+NPgQCnAV8GPhPM3uydN9ngWNAf0clQc5RU/6OVOpIRERSSZUkREQklRSgREQklRSgREQklRSgREQklRSgREQklRSgRGJgZt80s79s0M/6PTP74RSP901V2b6RfRFpJgUokTqY2fNmdn4Sbbv7d9z93UGeW9qS5aFm90mkGRSgREQklRSgZForjYT+zMw2mNkuM7vNzGaUHvvd0uZrr5rZv5nZSaX7v0XxKvp/NrPXzexPS/ffZWbbSxu3rQtb0dnMHjSz95Zuv9PM3MzeU/r+/PJV/BNHRWZ2gZn9otTuTZSKm5rZCcDfUqwy/bqZvVrR3Cwz+76Z7bbiZqCL6juDIs2jACUCvwcsBxYBvwV8vrRr6jeAqyhu8XIz8D0zO8DdPwy8CFzk7m9y9/9Z+jk/AI4D5lKs7vydkP14kGJZIihuNLgJWFbx/YMTDygV5PxH4PPAYRQ3uTwLwN03Ah8D/r3Uz0MrDv0A8AVgFsUSPl8M2VeRplOAEinuStzv7gMU36g/APzfwM3uvt7d8+6+GthHcUfVqtz9G+6+u1Rh/n8AJ5tZV4h+PMj4gPTliu+XUSVAUawRt8Hd73b3YeCvge0B2rrH3R8p7bP2HeCUEP0UiYUClMj4zeheAI4C5gOfLk3vvVqaHusuPbYfM8ua2XVm9pyZvQY8X3oozJYD/w78lpkdQTFg/B3QXRolvQNYV+WYoyr7XyrQ2V/leRNVBrE3gDeF6KdILBKrZi6SIpV7/RxDca+ffuCL7j7Z1NfEKssfpLg1+PkUg1MXsIvqm91V/4Hub5jZ48A1wFPuPmRm/wb8P8Bz7v5KlcN+Xdn/0pYHla9H1aClZWkEJQJ/ZGZHm9lsitsI/APFDSA/ZmZnlPYBmmlmv2NmB5eO2QEsrPgZB1OcAtwJHAR8qc6+PAj8MWPTeX0Tvp/o+8CJZraitKnnJ4B5FY/vAI4u7Skm0lIUoETgduCHFJMSNgF/6e6PUVyHuoniSOhZxu+39WWKyRSvmtmfUJyOewHYCmwAHq6zLw9SDHbrJvl+nNKo6lLgOorB8TjgpxVPeQB4GthuZtVGYCKppf2gZFozs+eBK9z9R0n3RUTG0whKRERSSQFKJEZm9tnSRbMTv36QdN9E0kZTfCIikkoaQYmISCopQImISCopQImISCopQImISCopQImISCr9H03/R4ALUbjtAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_points(data, ['petal_width', 'petal_length'], weights)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The separation bounday is not perfect here either"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\Owner\\Anaconda\\lib\\site-packages\\matplotlib\\figure.py:2369: UserWarning: This figure includes Axes that are not compatible with tight_layout, so results might be incorrect.\n",
      "  warnings.warn(\"This figure includes Axes that are not compatible \"\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAAEYCAYAAAAJeGK1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3X2YnHV59//3ObOzPASyJIEQMSEhhIYH5ckgUTSEZ7TaIEqk3qh3Cz9o/RWjt96lSuWovavi8Wu5C9IfxUop1VINCI23bUUILA/WhBKgFoIUSIAkkADZZIFAsrsz5/3HzGxmN7s718PMdc0183kdxx47OzvXXt/57jVzzvfp/Jq7IyIi0mpyaRdARERkLApQIiLSkhSgRESkJSlAiYhIS+pKuwAA/f39mqkhItJBenp6rN5j1IISEZGWpAAlIiItqSW6+Gr19PREPnbFihUsWbKkgaVpL6qfial+xqe6mZjqZ2LV+unv7w91nFpQIiLSkhIJUGY238wer/l63cy+kMS5RUQkmxLp4nP3p4HjAcwsD2wC7kzi3BLPYJ/z/DVDzPlSF4UpdSfdiEhA7k5fXx+lUintojRFLpdj6tSpmEV/30hjDOoM4Dl3fyGFc0tIm28rsuMpZ/NtRWZd2nJDliKZ1dfXx6RJk9h7773TLkpT7Ny5k76+PqZNmxb5b1jSyWLN7G+BR939+up9teugent7Ey2PjC//1l4ccsep5Ip5Svkimz7eS2mfgbSLJdIWZs2axdy5c9MuRlOtW7eODRs2DP+8ePHi4dtB1kElGqDMrBt4CTjG3bdU768NUJrF1zxh62fDjUNsXVmCIaALpp2Za+tWlK6f8aluJhalfl599VUOOuigJpWoNVSf41iz+Fpxoe6HKLeettR9pKRqsM/pu68SnACGoO/eEoPblPRDRJKRdID6beAfEz6nRLD5tiI+auzWS+X7RSQdOwacqx8cZMdAeh8U169fz8knn8wRRxzBJz/5SQYGmtftn1iAMrN9gbOAO5I6p0S37YGa1lPVEGy7vz1nHIlkwb9vKvHwJueRl9J7HV5xxRV88Ytf5JlnnmHKlCncdNNNTTtXYgHK3d9y92nuHm4psaRiyqLcnnM8u2DKqc2/ZAb7nGf+eFDdiRJau187K9eXA9PKdfED1Ne+9jWuvfba4Z+vvPJKrrvuugmPcXfuvfdePvGJTwDw2c9+ln/6p3+KXZbxtO+It8Qy44I8ffeVqH2ZW658f7NpartE1Y7XztLlAwxV4lF1VsGTrzjn/7DctdaVg+VLu0P/3Ysvvpjzzz+fZcuWUSqV+OEPf8i9997L8ccfP+bjb731VqZPn84BBxxAV1e5bmfOnMmmTZtCnzuo9vgPSsMVphpTT8uNmMU39fRc0xfrDk/O8PKkjBkXuBYISyDteu3cfF6BR14qcdezJdZtcwaKUMjD3CnGOfNyLDgkWq/GnDlzmDZtGo899hhbtmzhhBNOYPbs2Tz++OPjHvPqq6/ucV+chbj1KEDJuGpbUUm2nqqTM6qTMtrlk7A0V7teO5O6jVPn5HGH764pT1LKGZwzL8epc+K9Ji+55BL+7u/+js2bN/O7v/u7vPHGG3zwgx8c87G33norRx11FNu3b2doaIiuri42btzIIYccEqsME8n+f0+aZrgV9fNSsq2nUVPbk/gkPNjnHPyzkxlc1B6fujtNmtdOUlauL7FrCAo52DVUHoeKG6A+9rGPcdVVVzE4OMitt95KPp+fsAUFcNppp3H77bdz4YUXcssttzR1fZyymcuEZlyQZ9JRlnjrqSqpqe2bbyuy15YpmkafUZ2wLKKQK7earj6ri3Pm5ehqwLt3d3c3p512GkuXLiWfD/Ya//a3v80111zDvHnz2Lp1KxdffHH8goxDLSiZUGGqccSfFRI510RT22dd2rzzVj99G9Z2n7o7RVrXTpKuWrz7dXjpgsa0LUqlEqtWreK2224LfMzcuXN5+OGHG3L+etSCkpaR1tT2scYuJFvSXBaRVWvXrmXevHmcccYZHHHEEWkXZ0xqQUnLSGNqeyeMXXSCNJdFZNXRRx/NunXr0i7GhPTxQlpGdVLG8MemBKa2d8LYRSdI49qR5lOAkpYy44I8Vrkqk/gErJRO7SPpa0eaTwFKWsrwJ2FL5hOwxi7aR2GqccD7y9fLAadYW7ae2j2V02h6FUrLSXJqe+2n7ip9+pZWVZvKqRMoQEnLqU5tT+ITsMYu2sdgn7P938oti+2/8LZrZYxO5ZTW87v++uuZN28eZsZrr73W1HMpQEnH09hFe2j35QKt8vxOOeUU7rnnHmbPnt30cylASduI2j9fbUU5rtZTRrX7DtDNeH5RttsAOOGEE5gzZ07k84ahACVtI07//IwL8uw6eJtaTxnV7ssFmvH8Lr74Ym655RaA4e02lixZwvHHHz/m19q1a+M8hUi0UFfaQtytFgpTjS3nrqYwpXmJL6V52j3VUTOeX5TtNpKmACVtoV23WpBgpiyq2busqo2WCzTr+YXdbuPoo4+Odb6w9AqWzFO6Imn3VEfNen5RtttIUnt8vJCO1u7jD1Jfuy8XaNbzi7LdxnXXXcfMmTPZuHEjxx57LJdcckmsMkxEAUoyT+mKBNp/uUAznl91u40wezp9/vOfZ+PGjQwNDfHSSy/xve99L3Y5xqMAJS0n7HTxRqQrGt5Rt02mJbeKJFPzJJ0mK2mNfn5Z2G5DAUpaTtjp4o1IV6QddZsj6dQ8SabJSkMjn191u42/+Iu/aEDJmkMBSlpKlHQucfvnR++oq1ZUY6SRmifJNFlpaPfnN5oClLSUqOlc4vTPt0oKmXajep1YLpdj586daRejaXbu3EkuFy/EaJq5tIw408WrraitPy9Faj0lPUV9sM95/poh5nypK/R54hybFE39r2/q1Kn09fXxxhtvpF2UpsjlckydOjXW31CAkpYx0XTxIItuZ1yQZ+cGj9x6inLOqGrHZsKeJ86xSUmrXrPEzJg2bVraxWhp6uKTlhF3uniU/vk0pqjHGZtplS0X6tHUf2mExAKUmR1gZreb2a/N7Ckze19S55ZsSGN32zTOufm2Il4ZjvFiuLGZrIzraKdiaYQkr5ZrgZ+5+5HAccBTCZ5bMiCN3W2TPudwC6gaV4rBW0JZ2lJCOxVLIyQSoMxsMrAIuAnA3QfcfXsS55bsSCNdTWGqccD7R/79A06xpp2ztvVUFbQVlaWUTu2eekiSYe7N//RlZscD3wXWUm49rQGWufsOgP7+/uFC9Pb2Nr080rryb+3FIXecSq6Yp5QvsunjvZT2GWjqOac+9G72e+6dGIbjvHn4Rvo+8ERTzjXr1jPJDRb2uL9UGGTDp+5p2rFpSON/Ka1t8eLFw7d7enrqflpJKkAtAFYBp7j7ajO7Fnjd3b8GIwNUT09P5POsWLGCJUu0n89YBvucNVdu4T3fPLjlP8VuuHGIrT8vMe2cXNNnfA32OWs/N4jXvG9aNxx9Q3MWQ75w3SDbevd8zU05zZh9+Z7Bp9aGG4fG3HJh2pnNr6eor60k/5dp0nvPxKr109/fP3xfkACV1BjURmCju6+u/Hw7cGJC5xaylconyXQ1Weo2y+K4TrunHpLmSiRAuftmYIOZza/cdQbl7j5JQNZS+SSZziXp6dD9q8eu+/5VzU/plIZOS80jjZXkLL7LgX8ws18BxwPfTPDcHS0rU5PTMGXR2C+BZk2HnrIoB6MbE/ng52v3LSVEaiUWoNz9cXdf4O7Huvt57r4tqXN3sixNTU5Dz8ljf7LvWdicT/wzLshjo2KK5YMHmsJUY/J7yrcnL0AtE2lrWjXX5rI0xpKGTTePXQ+bbmpO/dTu6QNE2tvn7edHfhdpVwpQbU4pZya2a0O4+xth2lk5qDZgHaadHfxl+Na6EgMvl28PvARvPa//o7QvBag2FzflTJI7ola9ta7Ery4aCP3mG6Wse80Kd38jzrn17tKIFtTWnwd/ni9cO/LTxgv/e/Snj/Gl8b8UiUMBqs3FnZqc9I6oUH4TLr0V7s0XopV19rKx1+bM/mKwNTthz1mb7BUIlfT1rXWlPVp2uzYEb0Wl8b8UiUMBqs3FmZqcRubs2jfhMG++Ucu679zcHq2lvWbBvnPqvzSinDPOmODo1tPw/QECeVayoIvUUoDqAFGnJqcxPT1qF1acso5uRYVpPYU9Z5wxwTjjZVpqIFmkANUBqq0ox8O3nhKcnh61CytuWfedm6P7YHCc7oNDtp5CnjPOmGDU8TItNZCsUoDqEDMuyLPr4G2RWk9Vzf7kHbULqxFlraakDPqWHfWcccYEo46XaamBZJUCVIcoTDW2nLs68HqbNKanR+3CilvWt9aVGHwFDGNwS7Bxr6jnjDMmGHW8TEsNJKsUoGRMaeyIGrULK25Zo4x7TVlUs9i2yoKdM066oijjZdrdVrJKV6iMKY3M2VG7sOKUNeq414jFtlUBF93WZpMIm0WithUVdLZhFrOgi4AClIwjjczZUbuw4pQ16rjXiMW2VSEW3cbZhmL2si5y+wafbdiI9EoiaVCAknGlkTk76pTvqGWNNe41Rgsq6LhOnG0o9p2b49gfdAdqPVXFSa8kkhZdpTKuOF1REC21TpQurDhljTXuFXEMCpJPOxQnvZJIWhSgZEJxuqKiptYJ24VVFaWsUce94oxBQbJph+KkVxJJkwKUTChqV1Sc1DpRurCiljXquFecMaik0w5pHZRklQKUNEWWUutEGfeKMwaVdN1oHZRklQKUNFzWUuvsOzdHoZLqqBAw1VHUtUVp1I3WQUlW6QqVhstil9Lw/IGAvYNR1xalUTdaByVZpQAlDZe1LqW31pUY2FJOdTSwOViqo6hrr9KomzTWtIk0ggKUNFzWupSibvERZe1VWnWTxpo2kbha8x1DMi1ul1LULd+jiLNLbWGqccD7y62QA06xQC2StLrb4q5pE0mDApQ03PCbYY0wb4pRt3yPIs4utVGkmXYozpo2kTQoQElT9Jw88g23Z2GwN+CoW75HFWeX2sE+Z/u/lWffbf+FB56Jl1baoTjplUTSoAAVQ9LpauJ4a12JWbeemUi3GcCmm4u178FsuinYLLWo40FRRU11BNHXMyntkEgwClAxJJmuJq4Xrh3CBrsS6TartoJq3oMDtYbijAdF9c7fGbu7650XT9wNFnU9k9IOiQSnABVR0ulq4tgdMKzpb/hLlw/w4J8OjZVkgQf/ZIilywfGPTbp8SCA/tVj/9/6V038/4y6nimLa8RE0pJYgDKz583sP83scTN7JKnzNkuWUvkk2W1283kFpr8+Zpo6pr9e/v144owHRbXtgbGDdb11SVHXM2VtjZhImpJuQZ3m7se7+4KEz9tQWUrlk3S32aRuY+9ZY6apY+9Z5d+PJ854UFRR1yUlfZxIJ9KrIoIsddOk0W0WdQuLqMdFtXT5AFd2lxgc9b8cdPhqoTRhd2TU9UxKOyQSnLkn86nfzNYD2yh/mL7R3b9b/V1/f/9wIXp7exMpTxyzbj2T3OCeXVWlwiAbPnVPCiUa36G3nIvt0eFWToz64md/1rTz9vz4LHrezGMYjrN9vyKvf/zuuse9484PUnh90vBxg5Pf5OWPPdSUMg54FxuK0znkF0cy//nu4XM+PWeAl075NbPyr9Bt4wfyKauOZr9nZpIr5Snlirx5xEa2LVxb97xTVh3N/s/MxEp5PFfkjYDHiWTd4sWLh2/39PTUXe+QZIA6xN1fMrPpwN3A5e7+AIwMUD09PZHPsWLFCpYsWRK7rPVsuHGIrStHjSV0wbQzc8y6tDmf9qN6atnAmGM4e82Co67tbtp51/9/A/T/cvfPPe+Hw75c/3xRj4tj9TcH6H6kPE7mwMBJcPJX6p9zsM954vcGsSHwArzrr4OtMRrsc9Z+bhAfAOuGo29o/bVJSb22skr1M7Fq/fT39w/fFyRAJdbF5+4vVb6/AtwJvDepczdalrppku42g/Ib8OtrRt73+iMEmoId5bg4BvucrsdGTonPPxrsnIWpxrojoQSsm0/gIKO0QyLBJBKgzGySme1fvQ2cDTyRxLmbISvZoZcuH+Cih4fYMnnEshu2TIaLVk085TuOWFOwRz3Ei80d29t8W7EcYWpYnbIuXT7A+T8sf/3kMHjxIFhxGMP3BanXyefl6Duk/F1ExpbUq+Ng4CEz+w/gYeCf3b15AyAJyEJ26JvPK7BsYZ7Hzxl5/+PnwrKF+QmnfMcRawr26LhQbO4U7G0PlMiPaizl6+yMW63XIw80du0Hf38aDOwHRx5ogev1sbec60+Bx99qvZmfIq0iUoAys1ztV73Hu/s6dz+u8nWMu38jynlbSRa6aSZ1G6fOyXPywjyvVVpRr/XAwoV5Tp2Tn3DKdxxRp1KPzt83fH/APH5RRDlntV7PPjxHrvKwnME583KB63Xl+nIAXLlO659ExhM4QJnZiWb2SzPbAQxWvoYq3ztSVrJDr1xf4s6TYVfB+fF7m/+mGGWMbunyAe5b72Oun7pvnTetOzKOletL7BqCQg52DdWv19quwbWvlJ/pk694qK5BkU4SpgV1C3AfsACYW/k6rPK9I2UlO3QhB8eflOOpC3/BiSfl6Gpyx26UMbqbzytw3OaxM1Act3niDBRVOwacqx8cZMdA8G6zqKmOoFyvpx9mzD/QOP0wq1uvtV2DhUqsLuTDdQ1CtOcpkkVh3qpmA1e6+1Pu/kLtV7MKJ41x1eICly7oYmruDS5d0MVVi5sz9lQr7BjdpG7jwMU5fNTDPA8HnpYL1G3275tKPLzJeeSl4C3EKYtyMLpo+WCZHa5aXOCY6TmeeMV518G5uvVa2zVYXd3hHq5rEKI9T5EsChOg7qQ8+06krihjdDMuyI+eUEeR4BNQoozrzLggj43685Zv7jlXrt+dvWKwFL7LVeNX0ikmXAxjZt9n9wzlvYA7zewhYHPt49z9M80pnmTZjAvy7Nzggd7sly4fYKgEH5oDJ6yDLochg8fmwDfvGqQrB8uX7rl4tnoc7O4erI7rAOMeV1UNpFvvqcwgzNcPqFHPWXtcrScqx05U1rjPUySL6q3WfHbUz8rHIoFVx+iCuPm8Ao+8VOL+fYoc/zxQBM/Byx+EZcfmWXDI2I396nF3PVti3TZnoFge15k7xThnXm7c42rNuCBP370lvBis9RT1nGMd1x2wrI14niJZM2GAcvevV2+b2Qx33zz6MWY2oxkFk8Ya7HMO/tnJDC7ylpzUUR2fcYdfHVbkhGfhV4fBqceWp8QHOe6Gfy8voqod1wmiMNXYeRIU/g0GT6qfEaL2nN9dUz5n7TTzRh8X91iRrArzseu/xrlfraoM2Hxbkb22TGnJjOu1Vq4vcf9R5ewMvUcFH2eJO65zz/zyOe/5jXBlDTPNPO5xcY8VyZowCdn2+FhpZpPZI1FM59gx4Hxn9RCXn9zVtEWvjVDdv8ow+u4tMeOC1mtFjRif2aecnQHqj880clxnzWlgOwg+fpUrt2DOOjzH3c+V2PxmsGnf1enpW3bAwZNg69uBDot1TsjO9SpSVTdAmdkGyhMl9jGzF0f9ehrwj80oWBbUTvdt5W6WsXb/bbWs67VjLM9sdYoOeYMjpgUf1wlz3Ohj/2ur4w5mMD/AscCIaeWXLgjeGXHV4gK964usXF3kjIUTd2E26pyQnetVpCrIu9RFlD9g/gvw6Zr7Hdji7k83o2BZUDvdt1Vf8OPt/ttqrajaMZanXyt3Q5YCjCVFPW70sb8OeWxcaVw7WbheRWrVDVDufj+AmR3o7m81v0itLWvTfSfKLN5KraixuuocuHZVkWtXFUN18QU5DnZ35Y1WPRbgjgsb979M49rJ2vUqUivMO9QfmY35iXsXsBH4mbtvaUipWlgjpvsmORYwUWbxWZfWPz6pslbr9Xtrirw9VG7J5Az26YJL3lN/mnnY4wBu/GgXd6wtcvdzPmIgNQecdbhx/tH1Wxlh6ieNqeKani5ZFubq/A3gCuA0YF7l+xXACcDvA+vM7NyGl7DFNCKTdZKpaqJmFq9KqqzVer34xDzdlbjQnS8HmYnqNepxAAdNynHZSQXOmDvyMWcdblx2UoGDJtWvozD104hrJ6w0zinSKGECVA640N0/6O6fcvcPAkuBorsvBD4HXN2MQraiuFOFIZkpwnF3/006rU4aU7d/sWHkTLiHXgw+My5qqqOkp4prerpkUZguvnOA3x5130+B71du/wC4vhGFyoKw033TGgsYTuWzstLVFyCzeJrjFnGmboc9brwp6jsGCTy1PVJ6pRhTxaNK45wicYUJUM9R7sqrDUK/V7kf4EBgR4PK1fLCTvdNcyxgxgV5+u4r4QRrPaVZ1jhTt1/dUeKKu4f49lldgbrnoqYeils/caaKR5XGOUXiCnOlXgJ82cw2mNmqyvqo/wlcXPn9fOBrjS5gu0hzLKDainI8UGbxrI5b3P5kke074Y61wbJlRH2eWa0fkawJHKDc/VHgCOC/Af+78v2Iyv24+wPu/jdNKWUbSWssYMYFeXYdvC3U7r9ZG7eojiWFGUOC3c8zTzGxcS8RqS/UQhh3HwQeMNs97G5mOffRK21kPGmNBRSmGlvOXU1hypLgx8QsaxJT1Mday1QdQ6qqt5apkINFs2HVC0MsnJ2nf1ewc2tcR6S5AgcoMzsR+CvgWGDv6t2U10VqWXpAWRoLiFvWJFLrVNcyPfCC83bNeq99umDR7GBrma5aXOCGhwfZxV7s3WUse1+wLUKy9L8UyaIwr6pbgPuABcDcytdhle8ie0hiinp1LdMHDh3ZQls0O/haJojePSgizROmi282cKW76xUs40p6ivp4U8Xves6567mJs5k3ontQRJonTAvqTuDsZhVE2sPN5xVYtjDPkQcahUrvWiEPRx5oLFuY5+bz6nef7Rhwrn5wkB0D9T8LVc+XGzXElTPqnu/Gj3ZxzuHGPqM+pu3TBeccbtz40dbJVSjSicK8AvcG7jSzh4ARO+u6+2caWirJrEbscBtm7Gr0TrM7h2DvLrhsQf1tLMrdgzlKPsjd63YHw2r3oIikK0wLai3wbeAXlBfn1n6JjBBnh9uk0wfFSXUkIs0TuAXl7l9vZkGkPUTd4TaN9EFRUx2JSDJCdbKb2VnAhcB0d/+omS0AJrv7vQGPzwOPAJvc/SOhS9tioq7zSWPr7R0Dzr27TuTMAW9qWbOUPihqWUUkGYFfgWZ2OXAD8AywqHL328CfhTjfMuCpEI9vaVG3okhyu43ac24oHdz0sjYifVB1nmjt2JW2ohDpPGE+In4BONPdr4bh/d1+TTkHX11mNhP4TeB7oUrYwqKu80l6C4s454xzXNRtM6KOXUUVNdWRiDRXmC6+/YENldvVDv4CMPa+2Xv6S+APK38ns6KOlWRpu+9GlLWaPujRl+HEd1A3fVDUsatGqI5fFTY8xOCsRUpZJNIiLOi6WzO7HXjM3b9hZn3uPtXM/hA43t0/VefYjwAfdvfPmdli4Mu1Y1D9/f3Dhejt7Y3wNJIz4F1sKE7n6aFD6fPJFMmTp8hUe535XS8yK/8K3TZ6j/XoxzWqrFu9hxI5cpSYZv2JlPWXA8fwX8VD+Y38C7yve23gsiZVPyKSrMWLFw/f7unpqduHHiZAvQP4P5T3fXonsA54Hfiou2+uc+y3gE9T3jJvb2AycIe7XwQjA1RPT0+g8oxlxYoVLFkSPBlqHL3ri6HX3cQ5Lm5Zr19dpES5T/fyhcmU9aIfD/DWIEwqwPc/Hqz1k0b9VCV5/WSN6mZiqp+JVeunv79/+L4gASrMNPOXzewk4L3AoZS7+x4Oksnc3b8CfAWgpgV1UdBzt6KxxliCvJFGPS6KsbrNSsC1q4pcu6pYt9ssSlnjpg9Ksn5EpLWF3W7DgdWVr46W5NbktcJM+65Oo/7emiJvDzLcgtqnAJe8J193GnWUssbNLq4tLESkasIAVdk1t+47hLsfGvSE7t4L9AZ9fKuKszV5lOOq4qYB6u4qB6cgrZIoZY2bPkhbWIhIVb0WVKa74dpR7bTvoF1fI6dR5xPpNhsrfdBlJzX1lCLSZiYMUO5+f5g/Zmb/7O6/Ga9IMtpYY0lPREgDlOQ06rzBrMmw9Jg8y58ssn1n008pIm2m0fsJfLDBf08YmZLnma1O0csB4Ihp4dIArdj0BksWJLOFxC3n7w6Yp8zWJAcRCU8b3mRA7VjS06+Vt7AohdzCQkQkaxSgMmCsLj4n+HRxEZEs0jSpDKjuGjupwIikppMK9XeN7SRhduIVkdbX6ACl9M9NUO3iu/jEPN2VHr3u/O7p4sq6XZZGlngRaZ5GB6hvNvjvSY04u8Z2gjSyxItI89RbqPunQf6Iu19V+f6tRhRKxqYsC3tKI0u8iCSj3iSJWYmUogGi7hibJWlkWUhj998w4u7EKyKtq95C3d9JqiBx1e4Yq6nXjRMmtVIaRqdzgpG74opIdoX+eGlm+5vZYWY2t/rVjIKFpfGH5shKvWp8TqT9BF4HZWZHA/8AHEd5GY6xO5FsKh9VNf7QHFmsV43PibSfMAt1/3/gPuA0YD0wB/gW8G+NL1YwGn9ojizWq7Kgi7SfMK/k44Ar3H075Z14+4H/CfyvppQsgOr4w9mH50YsYK2OP7TioH4WqF5FpBWECVA7gerH1NfM7NDK8dMaXqqQRm4nofGHRtG4joikKUyAehBYWrl9O/CvwP3AvY0uVFjV8YcP7/VLzpmXo6vFe3jipORJMp1PtV6vPqsrE/UqIu0l8BiUuy+t+fGrwBPA/sAtjS5UWGlsJxFHnKnbSU771riOiKQp8LuOmX25etvdS+7+A3e/Afi9ppSsjcWZup2Vad8iInGFaW5cBfz5GPf/MXBNY4rTvuJM3c7itG8RkbjqBigzO71yM29mpzEyY/lc4I1mFKzdxJm6ncVp3yIicQVpQd1U+b438Lc19zuwBbi80YVqR3FS8iidj4h0oroByt0PAzCzv3f3zzS/SO1trKnbQYNMnGNFRLImzCy+z5hZAVgIHOLuPzKzSZXf7WhWAdtNnJQ8SucjIp0kTC6+dwM/AXYBM4EfAacCnwU+2ZTStaFoJp6AAAASq0lEQVQ4U7c17VtEOkmYd7kbgKvc/UhgsHLf/cAHGl4qERHpeGEC1DHADyq3HYa79vZpdKFERETCBKjngffU3mFm7wWebWSBpDmqOw4nkSJJRKQRwgSorwH/bGZfB/Yys69Qzsn3x/UONLO9zexhM/sPM3uy8jckQbU7DouIZEHgAOXuPwXOBQ6ivC/UocDH3P3nAQ7fBZzu7scBxwPnmtnCCOWViJQiSUSyJswsvm7gY8DZwCHAJsrbbjzp7jsnOtbdHXiz8mOh8qW+piZTiiQRyTIrx44ADzS7CZgPfAN4gXIL6qvAs+7+uwGOzwNrgHnAX7n7FdXf9ff3Dxeit7c3RPFlIgPexYbidJ4eOpQ+n0yRPHmKTLXXmd/1IrPyr9BtQ2kXU0Q6xOLFi4dv9/T01N35NEyA2gocXtlRt3rfVMoBamrQAprZAcCdwOXu/gSMDFA9PT1B/9QeVqxYwZIlSyIf36561xf57poiO4dg7y64bEFeGSjGoOtnfKqbial+Jlatn/7+/uH7ggSoMJMkNgP7jrpvH+DlEH+DSoDrpTyeJQnQjsMikkVhttv4PvAzM/sOsBGYBfy/wN/XZDzH3ffYYdfMDgIG3X27me0DnAl8O1bJJbBqiqTChocYnLVIKZJEJBPCBKjLKt+/Our+32P3poVOeQuO0d4B3FIZh8oByyuzAiUBWdtxWEQEwiWLPSzqSdz9V8AJUY8XEZHOo4yjIiLSkhSgRESkJSlAiYhIS1KAEhGRlqQAJSIiLUkBSkREWpIClIiItCQFKBERaUkKUCIi0pIUoEREpCUpQImISEtSgBIRkZakACUiIi1JAUpERFqSApSIiLQkBSgREWlJClAiItKSFKBERKQlKUCJiEhLUoASEZGWpAAlIiItSQFKRERakgKUiIi0JAUoERFpSQpQIiLSkhSgRESkJSlAiYhIS0okQJnZLDO7z8yeMrMnzWxZEucVEZHs6kroPEPAl9z9UTPbH1hjZne7+9qEzi8iIhmTSAvK3V9290crt98AngLemcS5RUQkm8zdkz2h2RzgAeBd7v46QH9//3Ahent7Ey2PSKcY8C4eGjiWD3T/im4bSrs4TdEJzzHLFi9ePHy7p6fH6j0+qS4+AMxsP+DHwBeqwWm0JUuWRP77K1asiHV8u1P9TKzd66d3fZENq4tMP/5cTp2TD3VsVuomznOMIyv1k5Zq/fT394c6LrFZfGZWoByc/sHd70jqvCJStnJ9qfx9XSnlkjRPJzzHTpJIC8rMDLgJeMrdr0ninCICS5cPMFR5r672pzz5inP+DwcA6MrB8qXd6RSuQTrhOXaqpLr4TgE+DfynmT1eue+r7v4vCZ1fpCPdfF6BR14qcdezJdZtcwaKUMjD3CnGOfNyLDgk+0shO+E5dqpEApS7P8TuDzcikpBJ3capc/K4w3fXFAHIGZwzL5foGE0zdcJz7FT6aCHSAVauL7FrCAo52DXUnmM0nfAcO40ClEjG7Bhwrn5wkB0DwZeIFHJw+mHG/APL37va8JUf9zlGqVdprja8TEXa279vKvHwJueRl4K3EK5aXOCY6TmeeAXedXCOqxYXmljCdMR9jlHqVZpLAUokY6JOpe6EKdhxnmMn1E/WJLpQV0SiiTqVuhOmYMd5jp1QP1mmACVCefzh3l0ncuaAM6k7+ITTHQPOd1YPcfnJXU09LupU6trjnutzBkvlN93Dpwafgh21bpIS5zlqinprU+2LUB5/2FA6OPT4Q9Rxi7DHVadSn314jlwlRtROpR4vcNQeV6qM/Ze8/nGjyxqlbpJS+xxrBXmOUetVkqEAJULy4zpxjosylXrl+hLFSoAqerjzZmVsZuX6EoOVIg6Wwj9HTVFvPerik47ViHGdqicSGg8q5Mqf7s86PMfdz5XY/ObEU6Krf3u02vLecWHrjV29uL3Il39e5M/PznPoARMvth3r/wG7n2Mz6rURonYPpyGtsipAScdqxLjOr1/b/UZ25IHBj4s63lE7dfrSBfUff+NHu7hjbZEHXnDertl9Yp8uWDTbOP/osd/8a8v6bJ8zVIJ8DuaFGLuK42/WFBkqwU2PFvn66RMHqLHK2hWyrGHrtRFqu3lbPeNFWmVVF590rEaM69QKc1xS4x0HTcpx2UkFPnDoyL+9aLZx2UkFDpo09lvAiLGrSuukVEpubOap18rfn3y1/mPTLmtUWek6hfTKqhaUdLzq+EOeIruG8qxcN/GnxPG6za5dVeTaVeVccGN1m40+X+14R7M+lY5X1ruec+56LlgXX1WJ3c+xGV18Y5W15CPvb5WyRpV212kYrVBWtaCkbURNVVPIwftnld/U3j+LuilyvnVGjpmTx/7dzMnl39c7XzkljzU9Jc9fnpvn3dPH/t27p5d/P5abzyuwbGGeSYXdb04GTCrAsoV5bj4vWJaGKGUd3d6xEGWtVmUuQlmTUC3rkQcahcrTKeTL3cPNqte0yxqHApS0jahTvq9aXGD7TnDyvL6Luily5h/UxXUf7uad+4+8f+ZkuO7D3cw/aOKOid0pebzpKXkOPSDP10/v5piDRt7/7unw9dO7x52AUO02u/jE/IjuyEvekw/VbRalrEePKuu7QpS1u1L13V3hy5qERnXzJpGWqRWm4CtASduI008eZsyjatMbI3/e+HrwY5NOyVN9flVBn2ec6enV4yG5smZlqnjcsiY5JpRmvWoMSjItTj951DGPWl0G58wz7nrWGarT25JGSp7xxqBqn+dYzzHq9PQ4Za2qdiWef1SOO54qsXNo3IeOkMZU8aiilDWtMaE069Xc0/8n9vf3Dxeip6cn8t9ZsWIFS5YsaUiZmimtNQVZqZ8wdgz4HlO3u0dN3R6vjl/cXuSmR4s88QrUvgqMcrfSxSfWX4MTtazVlDyFUSl5xitr1Of56o5S3WnmY83ki3pcnLKmLc7rMonXVlbrFXbXT39///B9PT09dQurLr4UKK1/48TpJ4865tGIsoZNOxT1eUadZh71uDhlTVurvy6zWq9xKEClIEvrH7IiTj951DGPqOKmHYryPH+xYWRPyUMvBus5iXocZGtMCLLzusxavcahMaiEtMKagnYWt0+/VnV8ptH/kzjjOlVRxwPyBrMmw9Jj8ix/ssj2ndHKumNw4rGrRpQ1SY14XSad7T0L9dooGoNKSCv0H7dy/aQh6f9JnHGdpGWprHE04hroXV/kutVFli3Mt3zKorRoDKrFdWL/catL+n8SZ1wnaVkqaxyNuAay0jWYReriS1iSaW4kmLCpjqIar0uxmnaoFbt5d49BOWA89KJz2Ulplqg5wr4u1WWfDAWohHVS/3FWVFMPPfH8dt4150C2vt2c81Szbn9ndXF4Fh+UP7FffnK+JXdvrY5dHfb246zf54S6Y1dZFfZ1qZ14k6EAlbA00vrLxK5aXKB3fZGV66dx4cG5prVoq91J7vDdNUV2DsHeXXDZgtYdu7jl/HIrYMWKzXzhw+3bIgj7uhz9v4SRXYPSGHqHFKFzUsdIY43sHtb/stHUgpKO1YmpY6Sxqv/LwoaHGJy1SP/LBkskQJnZ3wIfAV5x93clcU6RetIaR+ikbt4sbWseRfV/uWLTGyxZoM/7jZbUq+PvgHMTOpdIIJr633ytnj5IWlsiAcrdHwD6kjiXSFgaR2gerRGSONQmlY6ncYTG0hohaZTEUh2Z2Rzgp2ONQdWmOurt7U2kPCLSHAPexYbidJ4eOpQ+n0yRPHmKTLXXmd/1IrPyr9BtATd5krayePHi4dtBUh21XAsqTq445ZqbmOpnYqqf8UWpm971Rb67pkhxCApdeT614EBOnXNwk0qYLl07ExsrF18Q7T2FSERSo/VeElciAcrM/hH4JTDfzDaa2cVJnFdE0lMd27v6rC7OmZejSx+HJaREuvjc/beTOI+ItI5OWu8lzaGrRkREWpIClIiItCQFKBERaUkKUCIi0pISW6g7kdqFuiIi0v6CLNRVC0pERFqSApSIiLSklujiExERGU0tKBERaUkKUCIi0pIyF6DMbJaZ3WdmT5nZk2a2bIzHmJldZ2bPmtmvzOzENMqahoD1s9jM+s3s8crXVWmUNWlmtreZPWxm/1Gpm6+P8Zi9zOxHlWtndWWbmI4QsH7+u5m9WnPtXJJGWdNkZnkze8zMfjrG7zr2+oG6dRP62mm57TYCGAK+5O6Pmtn+wBozu9vd19Y85kPAEZWvk4EbKt87QZD6AXjQ3T+SQvnStAs43d3fNLMC8JCZ/au7r6p5zMXANnefZ2YXAt8GPplGYVMQpH4AfuTuf5BC+VrFMuApYPIYv+vk6wcmrhsIee1krgXl7i+7+6OV229Qrox3jnrYEuDvvWwVcICZvSPhoqYiYP10pMr18Gblx0Lla/QsoSXALZXbtwNnmFnd9RrtIGD9dDQzmwn8JvC9cR7SsddPgLoJLXMBqlal+XwCsHrUr94JbKj5eSMd+CY9Qf0AvK/SlfOvZnZMogVLUaUL4nHgFeBudx/32nH3IaAfmJZsKdMToH4APl7pOr/dzGYlXMS0/SXwh8B4m1t18vVTr24g5LWT2QBlZvsBPwa+4O6vj/71GId01CfBOvXzKDDb3Y8DvgP8U9LlS4u7F939eGAm8F4ze9eoh3T0tROgfv4PMMfdjwXuYXdroe2Z2UeAV9x9zUQPG+O+tr9+AtZN6GsnkwGq0j/+Y+Af3P2OMR6yEaiNzjOBl5IoWyuoVz/u/nq1K8fd/wUomNmBCRczVe6+HegFzh31q+Frx8y6gB6gL9HCtYDx6sfdt7r7rsqPfwO8J+GipekU4LfM7Hngh8DpZvaDUY/p1Ounbt1EuXYyF6Aq/bk3AU+5+zXjPOwnwGcqs/kWAv3u/nJihUxRkPoxsxnVfnEzey/l62BrcqVMh5kdZGYHVG7vA5wJ/HrUw34CfLZy+xPAvd4hq9mD1M+osdzfojzG2RHc/SvuPtPd5wAXUr42Lhr1sI68foLUTZRrJ4uz+E4BPg38Z6WvHOCrwKEA7v7XwL8AHwaeBd4CfieFcqYlSP18Avh9MxsC3gYu7IQXEfAO4BYzy1MOysvd/adm9qfAI+7+E8rB/ftm9izlT74XplfcxAWpn8+b2W9Rni3aB/z31ErbInT9jC/utaNURyIi0pIy18UnIiKdQQFKRERakgKUiIi0JAUoERFpSQpQIiLSkhSgRJrMzNzM5jXob/2rmX12nN/NqZxr3OUjjSyLSLNlcR2USMdy9w8FfayZ9QI/cPeGJe8USZJaUCIi0pIUoKTjmNkVZrbJzN4ws6fN7Awzy5nZH5nZc2a21cyWm9nUyuOrXWeXmtlLZvaymX2p5u+918x+aWbbK7+73sy6Q5TnsMqxucrP3zOzV2p+/wMz+0Lldm91o7dK5vE/N7PXzGwd5a0Oqsd8A/ggcL2ZvWlm19ec8kwze8bMtpnZX3XKdhCSPQpQ0lHMbD7wB8BJ7r4/cA7wPPB54DzgVOAQYBvwV6MOP43yJphnA39kZmdW7i8CXwQOBN4HnAF8LmiZ3H098DrlrVGgHFjeNLOjKj8vAu4f49D/B/hI5bgFlFNYVf/mlcCDwB+4+36jNon7CHAScBywtFIHIi1HAUo6TRHYCzjazAru/ry7PwdcBlzp7hsrGZf/BPjEqAkHX3f3He7+n8DNwG8DuPsad1/l7kPu/jxwI+VAF8b9wKlmNqPy8+2Vnw+jvDvpf4xxzFLgL919g7v3Ad8KeK6r3X27u78I3AccH7KsIonQJAnpKO7+bKW77E+AY8zsLuB/ALOBO82sdrO1InBwzc+1m2C+ALwbwMx+A7iGcitmX8qvq4n2xRnL/ZQzPG8EHqC81cWngZ3Ag+4+1iZwh4xRpiA219x+C9gvZFlFEqEWlHQcd7/V3T9AOSg58G3Kb/QfcvcDar72dvdNNYfW7jF2KLv3GLuB8rYUR7j7ZMrZ48OO69xPuWtvceX2Q5Qz05/K2N17AC+PUaZaygQtmaYAJR3FzOab2elmthfl1snblFtKfw18w8xmVx53kJktGXX418xsXzM7hvIWLj+q3L8/5TGkN83sSOD3w5bL3Z+plOUi4IHKLshbgI8zfoBaTnkLg5lmNgX4o1G/3wLMDVsWkVahACWdZi/gauA1yl1d0ym3eK6lvNncz83sDWAVcPKoY++nvMfYSuDP3f3nlfu/DHwKeIPyTqE/Ipr7ga2VsaHqzwY8Ns7j/wa4i/L41KPA6N2Tr6U8jrbNzK6LWCaR1Gg/KJE6zGwOsB4ouPtQuqUR6RxqQYmISEtSgBJJiJk9WVk0O/rrv6VdNpFWpC4+ERFpSWpBiYhIS1KAEhGRlqQAJSIiLUkBSkREWpIClIiItKT/C3zj5AGyCOVpAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_points(data, ['sepal_width', 'petal_length'], weights)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\Owner\\Anaconda\\lib\\site-packages\\matplotlib\\figure.py:2369: UserWarning: This figure includes Axes that are not compatible with tight_layout, so results might be incorrect.\n",
      "  warnings.warn(\"This figure includes Axes that are not compatible \"\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAAEYCAYAAAAJeGK1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3XmcFNW9///Xp7tnhtVBEBBlZBMEZNMoGE0MLiCYBK4rSkSJMZpcMdH7jUvUnyHmakzM9SZXTGK8Eo3BuEUBDYq4cE2CLBpRUEQHBEFWB2ZknZnuPr8/epnunp6Z7mF6m3k/H49+TFfVqapzqmr606fq9DnmnENERCTfeHKdARERkWQUoEREJC8pQImISF5SgBIRkbzky3UG0lFVVaUWHSIirUxpaaklm68alIiI5CUFKBERyU/OuYJ5VVZWusgrU+bOnZuxbWeLypAfVIb8oDLkh8QyJHyeJ/3MVw1KRETykgKUiIjkpYJqxdcQ5xy7du0iGAwe8rbKysrYuXNnC+QqdzJVBo/HQ9euXTFL2uBGpKBs3lPO/e/8AOeCOAcej5crh93Joo2PcfnQOzispCsAVdUVPPbBzzjv2Ot4rvx+/u3YGcwtn8XlQ+/A4fjj6js44N/HjgObuHzI7fzf5mcA48phd/JFzS5mrfwh3duV8d0Rd7Nl33r+8N7NeCmiS7vufH7wM7qVHMUXNRWYeeL273D876pb2b7vUwKdgwzZVcbLG/8UzdvmPeXhbffmksE38fRH90X3G8l7OiLljC17S6Q9FK0iQO3atYuOHTvSrl27Q95WUVERXbp0aYFc5U6mynDw4EF27dpFt27dWnzbItk2Z83d1AQP1s0IwqPv/5SDgX28vPExLhz0QwAWbXiM9VWr+POau9i+fyNz1tzN9v0beXnjY+AcG/esidnmzwngB+DljY+xrvJdqgMH2LzvI17e+Bj/2v4qDoefGj4/+BkAFdVbQiu7+P3jHJv3fhxaZvDI+zPj8jZnzd3hbX/Mn9fcxbb9G6L7jeQ9HZFyprJ+OmkPRau4xRcMBlskOEnj2rVr1yK1VJFc27ynPPqBHutAYC8Ox/JtL/JF9S6qqitYtv0lHI5t+zfE/V22dQFLty6IWz8SnACWbvlb3D7+ueV5DgT2NpqvyP6XbV3A0m0x27b4vH246624bce+XxbOe6pWbQ9yzfNbWbotVM6lW1/kmue3sWp7/f/1dNK2hKwEKDObbWY7zGx1A8vNzP7HzMrN7D0zOzEb+RKRtmnOmrsbXR50QV7e+BiLNjyGc8k/fP3OHxeQEtVflvqHuN/5Cbjk264NBnngnTuJHYgi9n1toJZbXn8kpaCxanuQu9/w4/c8jj/85dMfDOL3PM7db/jjtpFO2paSrVt8jwCzgD81sHwiMDD8GgP8LvxXRKRFNVR7ihVwfpaFa0cNBQrIZMc2jW3bj9ezl9hHwfHvHUUlL3P361O49aP5DP98DRw4ADU1da/aWvD7mXXeg/hLfXTqsAgzf3h9P506vEzVvqnMmlvNg7+dBMEgs659Hn/XkobTLuvKg5OKW/QoZCVAOefeMLO+jSSZDPzJOeeApWbWxcx6Oee2ZiN/h6KiooKzzjoLgG3btuH1eunevTsbNmzgqKOO4oMPPshxDkUkVlO1pwi/81OozYGMWjp0fpJZh3+TB3/y7w2mm7FxGvf/fCKWULszgnRt9yjX/WYBrHkvlPb+6Q2n7fw41514fcuXw2VpwMJwgHrBOTcsybIXgHucc/8IT78K3Oyceys2XWxffIsXL47OLysro3///hnJdzruueceOnbsyHXXXcenn37KlClTePPNN1t0H36/H58vd21b1q9fz6ZNm3K2f5FDtfiw31CwkScNwWB7vjtjL8PWvNFouh+9dBaBkvq357zVHn414dWU0lqwhK/t+V5a+Rs7dmz0fUN98eVLK75kmWs0ck6ePDn6fufOnfGt1lq6GXSKQbxdu3a0b9+eLl26UFlZCcCNN97IkiVLOProo5k3bx7t27dn3bp1XHvttezcuZMOHTrw0EMPMXjwYDZu3MiVV17Jzp076d69O3/84x855phjmD59Ol27duWdd95h1KhRvPDCCyxZsoTu3bsTDAYZNGgQS5cu5YgjjgCgsrIyYy0Ry8rKOPHEzD8inDdvXtw5LkQqQ35ILMOHy+c1eYsvxDDAZfRWXvM41/jHnHNG7/c6MGzNgoYTAauGfo0vqsfRofjV6G270Po+vqg+i9VDaqMBrrG0+6rHMeCUbzKsZ8PNGhLPQ1VVVVPFzJsAtRkoi5nuDWzJUV5azMcff8xf/vIXHnroIS6++GL++te/ctlll3H11Vfz+9//noEDB7Js2TL+/d//nddee40ZM2Zw+eWXc8UVVzB79mx+8IMfMHfuXAA++ugjXnnlFbxeL126dGHOnDlcf/31vPLKK4wcOTIanESkcd8aciv/9fbVTabzWejj0e9qM52lFuco4l9H/prV//1vDLPPoV07KCmB4uLQX58PioqYtfcr7D64j/adX4+rJTg87Dp4BfffeS0PDlwHZsz6eAC7D36RPO2eqdy/zF+Yz6BSMB+YYWZPEGocUVUIz5+a0q9fP0aNGgXAl770JTZs2MDevXtZsmQJF110UTRddXU1AG+++SbPPvssANOmTeOmm26Kprnooovwer0AXHnllUyePJnrr7+e2bNn8+1vfztbRRIpeL07H8uRHfo2Wovymo8xvc4F51i6bUEDDSVC9atURW7ENF7zCS13LrTt2LR1y3wEgu3iGkrE1qicM/buH0813bm//0WNBo0Z24Pc/UY79u4fR+cOCzHz45yPvfvHU2RdmfFVH/QcGUrbM8jdbxQ1nHZMy4eTbDUz/wvwJnCcmW02s++Y2ffMLHLTcgGwHigHHgIafqpXQEpKSqLvvV4vfr+fYDBIly5dWLlyZfS1Zs2apOvH9tjQsWPH6PuysjJ69uzJa6+9xrJly5g4cWLmCiHSCn1ryK2NLveYh/F9pjGu7zTMkn9M+syHt5Hv+M754puCp/Fx6/DRUP3B4eHzyobz7yiiat9Uijw0GTSG9/Rw6+k+fMGp+Dyh/Pk8HnzBqdx6uo/hMbfs0knbUrISoJxzlzrnejnnipxzvZ1zDzvnfu+c+314uXPOXeucG+CcG57YOKIZO2z2q3L37vrzW9Bhhx1Gv379ePrpp8NZdbz77rsAnHrqqTzxxBMAzJkzh6985SsNbueqq67isssu4+KLL47WrEQkNZFaVKL23k4YxugjJ3JYSVdKS7oxpucEDOPIDn3j/o7pdS6n9Do3bv1IwHIO9uyfQI3/mOjHyN795xIIdmr0Iyi03Ni7/xz27D+ngWXjqa49EX/gmOi+avx9oh9VtdXj6VzUldu/llrQGN7Tw4Pf7MUpR4bKeUqviTz4zSOTrptO2pbQKnqSKDRz5szh4YcfZuTIkRx//PHMmzcPgP/5n//hj3/8IyNGjOCxxx7jN7/5TYPbmDRpEnv37tXtPZFm+taQWyn2tKPIivFRTLGnPVcc/xP6lw5nfJ9p0XTj+k6jf+lwLhtyG/1Lh/OtIbdG04zrO40+nYfQo/0xgPGtIT/mYM1xVNcOpmrfVCqqbsK59lTXHkvVvql8XvljwHCumFr/UQBU1/bCuRKca8fOytuorj2eqn1T+WLfVGpqj8UssuxWqmuPZ0Lfy3j2kmJuGXM7Jd72lHUayG2n3E7fw4bQp/NQ7jljOn88rzjtoBEpZ2zZWyLtochaM/OWENvMvLS0NDo/0uqtJWSyBVxLeuutt7jhhhv4+9//Xm9ZJsvQkse6Ma2x9VghUhlSt2p7kFnL/JzYy1i4LsihtGWfPsrDpMF1t+da43mIbcWX783MJQ333HMPv/vd75gzZ06usyIi1HUDVB2Axev8HLVlHVt6DUzrJy9FHlK+LddWKEAVoFtuuYVbbrkl19kQkbD7loSCE0A13rSCk88D/58CU1IKUCIizbRqe5D7lvjZW5OwoIng1M4LBwNwzgDjmpOLMpfBAqcAJSLSDPM/9PPIyvR78PYa/DhDzbJbGx0hEZE0rdoeTDk4eS1UUzKgYxHcMVbBKVWqQYmIpGnWsobHgUoUcPCvrY6/XtKy3QC1BW0ujK/aHuSa+TXRwbUSp/PNqaee2qxljZk5cya/+tWv6s1/4403OPHEE/H5fDzzzDPN2rZIW/D1Qal/dKbSo4Mk16YCVKQp6M79cPcbfp5bE4ibzscgtWTJknrzAoFAg8sOxTHHHMMjjzzC1KlTW3S7Iq1J3LOnFH5H2qEI3dJrpjZ11GYti2kKGoCnVgfiptOptif685//zOjRoxk1ahTXXHNNNIh06tSJm2++mS996UucffbZLF++nLFjx9K/f3/mz58PwCOPPMLkyZOZMGECxx13HD/96U+j2+3UqRMQGv/qjDPOYOrUqQwfPjxuGcAvf/lLhg8fzsiRI5k5cyYADz30ECeffDIjR47kggsuYP/+/Y2WoW/fvowYMQKPp01dFiIpq/fsqYnWeiVe+I9TVXtqrjb1STRjjI+SmG7rIsEJQhfSdc2shq9Zs4Ynn3ySf/7zn6xcuRKv1xv9Ee2+ffsYO3Ysb7/9Np07d+b2229n0aJFPPfcc9xxxx3RbSxfvpw5c+awcuVKnn76ad56q353hMuXL+euu+6qN0rviy++yNy5c1m2bBnvvvsuP/zhDwE4//zzWbFiBe+++y5Dhgzh4Ycfblb5RCQk3S+xmepEta1oU6F9eE8PFw/zxtWcIBScpgzzMqynh/A4g2l59dVXefvttzn55JMBOHDgAD169ACguLiYCRMmhPY/fDglJSUUFRUxfPhwNmzYEN3GuHHj6NatGxAKLP/4xz846aST4vYzevRo+vXrV2//r7zyCt/+9rfp0KEDAIcffjgAq1ev5vbbb6eyspK9e/dyzjnnpF84EQFCtaeaQOiZUm0KTwOmj/IoOB2iNhWgVm0P1gtOEKpJPbk6wLFdjd4lyddtjHOOK664gp///Of1lhUVFUWHzfB4PNEhODweD35/3bcxS7hVkDgN8UNuJO4/Wfrp06czd+5cRo4cySOPPMLixYtTLpOI1IntyshDEJw1ensvsS89aZ42Fd5jn0EB9W733d/MZ1BnnXUWzzzzDDt27ABg165dbNy4Ma1tLFq0iF27dnHgwAHmzp3LaaedlvK648ePZ/bs2dFnTLt37wZgz5499OrVi9raWvXbJ3IIYj87gngaDU7dO6Dg1ELaVICKfQYVua0XO93cpqBDhw7lP//zPxk/fjwjRoxg3LhxbN2a3oDAX/nKV5g2bRqjRo3iggsuqHd7rzETJkxg0qRJnHTSSYwaNYr7778fgJ/97GeMGTOGcePGMXjw4Ca3s2LFCnr37s3TTz/NNddcw/HHH59WGURaq8Tn102llZbRpo5kZETIWcv8XDfGx7CeHgZ0NWYt8zNjTOhhZnOeQQFMmTKFKVOm1Ju/d+/e6PtI67pky3r06MGsWbMaXH/s2LGMHTu2wfVjO5CtDBfi+9//Pt///vfrbTMxHxEnn3wymzdvTrpMpC0b3tPDxcd7eOLtA9QW1T0H8AQDmNdLINzaXM+dWlabClAQHhFyUnGD0yIiiVZtD/LUe7VxwQkg6PFSZNCpONScXMGpZbW5AJWPpk+fzvTp03OdDRFJYtX2ID9d7CdIzD0+FwQLBaPaIBR79WPcTNARFRFpQKT1XtBR12uEc+HgFJo+lOfX0jgFKBGRBsS1/DUL15wiLfgMj+nHuJmkoyoi0oBQ672Y/vas7iOzxAszNXRGRunIiog0YHhPDxfvfYuS6n1x80u8Ltr7jGROm7pxuvrKGvyNNCP3dYHe92UvP6k49dRTG+y1vLFljZk5cyadOnXiRz/6Udz86upqLr/8ct5++226devGk08+Sd++fZuTbZFWYdW2AE95h1Fd3D5ufnXAor3PKEhlTps6so0Fp1SW50I2h9t4+OGHOfzwwykvL+eGG27g5ptvbtHtixSSSOu9uODk6jrhO5TeZyQ1bSpAZVJrGG5j3rx5XHHFFQBceOGFvPrqq7gUxrsRaW2irfewJK33QtR6L/MUoFpAaxlu47PPPqOsrAwAn89HaWkpFRUVh3RsRApR4633UOu9LFH4bwGtZbiNZLWlZL2ki7R2M8b4uHtxDdUuHIASak63ne7Ts6csUIBqAa1luI3evXuzadMmevfujd/vp6qqiq5duza6jkhrNLynh4s3L+KpI75CdUnd/13s2HGSeTrKLaC1DLcxadIkHn30UQCeeeYZzjzzTNWgpE1atS3AU91PjwtOUDd23OrtKYxYKIdMNagWEDvcRjAYpKioiAceeIA+ffqkvI3IcBvl5eVMnTo17eE2Vq5cyUknnURxcTFnnnkm9913X3S4jT59+jB8+HD27NnT6Ha+853vMG3aNI499li6du3KE088kXIeRFqTWf84ENd6r6R6P9UloVvokdZ76mQ689pUgPJ1abwpua9L87fdGobbaNeuHU8//XTSZSJtyYy1j3N374uoLulISfU+pnzwLE+eNI3qgFrvZVObOsrDZjf9jaeysvGm2CLSyjnH8Efu5dbix5n1vYe47ndXMezW7zIgPJZcZOw4ybw2FaDylYbbEMkj770H5eUMp5wHfzAIiovhm/MZXqqx47Ita18DzGyCma01s3IzuyXJ8mPM7HUze8fM3jOzc7OVNxGRqGeeiZ8ePx5KS3OTlzYuKwHKzLzAA8BEYChwqZkNTUh2O/CUc+4E4BLgt9nIm4hInIUL46cvuCA3+ZCs1aBGA+XOufXOuRrgCWByQhoHHBZ+XwpsyWSGqqormPXO9XxRvSuTuxGRQrJ7NyT24jJxYm7yIlg2+lozswuBCc65q8LT04AxzrkZMWl6AS8DhwMdgbOdc2/Hbqeqqiqa2dgfnZaVldG/f/+08vS3TQ/ydsXLfKnbOXy97Oq0y9RWrV+/nk2bNuU6GyIZ0evNNxn9i19Ep6v69GHxb36Twxy1XrGtkktLS5P+4DJbjSSS7TwxMl4KPOKc+y8z+zLwmJkNc84l/UXc5Ml1FbCdO3fSpUvqbcSrqitYuft1HI53d7/GNwddxWEloR4TKisr09pWMg0NZxExd+5cBg0axNChiXc5W0ZLlKEhZWVlnHjiiRnZdqx58+bFneNCpDLkh7TK8NJLcZOlF1yQF+VvjeehqqqqyXWydYtvM1AWM92b+rfwvgM8BeCcexNoBxyRicws2vAYkbgXdEFe3vhYJnbToLlz59br8FVE8sArr8RPn312bvIhQPYC1ApgoJn1M7NiQo0g5iek+RQ4C8DMhhAKUDtbOiNV1RUs2/4SARfqBy/g/Czf9uIhP4u66667OO644zj77LNZu3YtkHy4iyVLljB//nxuvPFGRo0axbp169IeFkNEMmDDBigvr5suKoKvfjVn2ZEsBSjnnB+YASwE1hBqrfe+md1pZpPCyf4f8F0zexf4CzDdZeABWWztKeJQa1Fvv/02TzzxBO+88w7PPvssK1asAJIPd3HqqacyadIk7r33XlauXMmAAQPSHhZDRFrWqu1BrllyGKuGfi00PfRrXPNAOav2dchxztq2rP1Q1zm3AFiQMO+OmPcfAKn3kNpMb+14JVp7igg4P29tX8SFg37YrG3+/e9/57zzzosOdzFpUijmpjrcRbrDYohIy4kMTljt6cTdNz3HxX+9i6cuuI3qko7c/YZf4z7lUJs76if1OBuvxcdlr/k4qee4Q9puQ8NdzJo1i1WrVvGTn/yEgwcPJl031XQi0vJiByesLukYDU4Q6hh2loZ1z5k2F6DG9Z2GWXyxPeZhfJ9pzd7m6aefznPPPceBAwfYs2cPzz//PNDwcBedO3eO61k83WExRKTlzBjjo8Tqbvsnjv90nTqGzZk2F6BKS7oxpueEaC3Kaz5GHzkx2sy8OU488USmTJnCqFGjuOCCC/hq+MFqZLiLcePGMXjw4Gj6Sy65hHvvvZcTTjiBdevWNZhORDJveE8PF+9eQkn1vrj5Gpww99rkV4NxfaexbPtL4A699hRx2223cdttt9Wbn2y4i9NOOy2umXlDw2KISOat2h7kqU4nUe1rFzc/MjjhsV1NQSpH2uRRj9SiDDvk2pOIFLZZS/1xwSm2JhUZnFByo00GKAjVovqXDm+R2pOIFK4ZpR9Hg1JJ9T6mvPwbSryhZRqcMLfabIAqLenGjBN+rdqTSBs2/0M/P906gLGLH6P7zo2MXfwn/vz1mxnb1+jeATUxz7FWceQ9Ho+aZmfBwYMH8XhaxSUjwvwP/TyyMkjQPCwcfw1lmz9g4fjvEfR4WbjO8fVBHgWnHGsVddeuXbuya9euuKbbzbVp0ybKysqaTpjHMlUGj8dD166qcUrr8OjKmB5lzPjXqAkQ83vGP70bZJIa1eZUqwhQZka3bt1aZFtLlizJSm/dmdQayiCSSau2B/EYBGI7U0v4sf0VI1V7yjWdARFpc2Yt88cHpwRFHvjm4Fbx/b2gKUCJSJszY4wPb9Ih8kJqg/D8h2penmv6iiAibY792M9tlQ0v31sCv7Yg39QzqJxqkwFq9erV9QYMHDp0KMOGDWtwnR07dsQNMw/QvXt3zjjjjEb39dRTT9Wbd/HFFze6zuuvv87OnfFDYY0dO5YePXo0uE5imZ566qlWVyZofecJ8r9MO3bsKOgy1dTU1Fuvd/FpdKB7g+scHLiGi20tsavlU5ny/dprKbrFJyIieUkBSkRE8pIClIiI5CUFKBERyUvmXCM/BsgzVVVV0cyWlpZmZB/z5s1j8uTJGdl2tqgM+UFlyA+JZVh5fk1K6416tjhTWUpbazwPVVVV0felpaVJG/2rBiUibcbqK1MLTpIfFKBEpM3wN/Lbp1i+LpnNh6RGAUpEJMGw2flze68tU4ASEZG8pAAlIiJ5SQFKRCSGnj/lDwUoEWkTUm3Bp+dP+UMBSkTahFRb8En+UIASEZG8pAAlIhKm50/5RQFKRFo9PX8qTApQItLq6flTYVKAEhGRvKQAJSIieUkBSkQENZDIR1kLUGY2wczWmlm5md3SQJqLzewDM3vfzB7PVt5ERNRAIv/4srETM/MCDwDjgM3ACjOb75z7ICbNQODHwGnOud1m1iMbeRMRkfyUrRrUaKDcObfeOVcDPAEkDg/5XeAB59xuAOfcjizlTURE8lBWhnw3swuBCc65q8LT04AxzrkZMWnmAh8BpwFeYKZz7qXY7cQO+b548eKM51tEWofeT56J92BJg8sD7arZPOW1LOZIxo4dG33f0JDvOOcy/gIuAv43ZnoacH9CmheA54AioB+hW4FdYtNUVla6yCtT5s6dm7FtZ4vKkB9Uhvwwd+5c5z791Dmoe3Xq5FwgkOuspazVnIcYCZ/nSWNHtm7xbQbKYqZ7A1uSpJnnnKt1zn0CrAUGZil/ItKarVwZPz18OHjUiDnfZesMrQAGmlk/MysGLgHmJ6SZC5wBYGZHAIOA9VnKn4i0Zu++Gz89alRu8iFpyUqAcs75gRnAQmAN8JRz7n0zu9PMJoWTLQQqzOwD4HXgRudcRTbyJyKtXGINauTI3ORD0pKVZuYAzrkFwIKEeXfEvHfAf4RfIiItRzWogqSbsCLSqvkOHIDy8roZZjBsWO4yJClTgBKRVq3zxo3xMwYNgo4dc5MZSYsClIi0aqXrE9pa6flTwVCAEpFWrXTDhvgZev5UMBSgRKRVOywxQKkGVTAUoESk9QoE6gco1aAKhgKUiLRe5eX4amrqpo84Anr1yl1+JC0KUCLSeiX+QHfUqFAzcykIaQUoM+tkZr3NrFOmMiQi0mISf6Cr508FpcmeJMxsGHAN8HWgD2CAM7MNwIvAg865VZnMpIhIOlZfWYO/EmAmnDezbsF64PwafF00gm4haLQGZWZ/AR4HtgKXAUcAxeG/04DPgDlm9kSG8ykikrJQcGr+cskPTdWgHnfOPZ9k/m5gSfj1czP7RovnTERE2rRGa1ANBKdk6V5omeyIiIiEpNxIwswCZvZnMytJmP9Fy2dLRETaunRa8R0M//2HmR0VM19tNkVEpMWlE6ACzrnLgGeA5Wb25fB81/LZEhGRti7tAQudc78ws/eA58zs9gzkSUREJK0AFb2V55x70cy+BswFNLCKiOQVX0c//n0Nf7z5umQxM9Js6QSoQbETzrm1ZjYaOLFlsyQicmiGnf00TJ9eN+Mb34DnU2qULHmkqR/q9oi8d85tTVzunNvjnPs/M+uZicyJiDRLYh986uKoIDXVSOJ1M/utmX3ZzOLSmpnHzE4xs98Cr2YuiyIiaUrsg09DbBSkpm7xnQBcDfwB6G9m64E9QGdgAPBReNn1mcykiEhT6vrfA7q8BOfFLHwcfAtq1P9egWk0QDnnaoBZwCwzOwYYBnQh1NXRe865zzKfRRGRpqn/vdYnnUYS24AJwCigE3CphcdVcc5d3vJZExGRtiydAPUoMBJ4HtiemeyIiIiEpBOgJgD9nHOqKIuISMal09XRp0BJk6lERERaQKM1KDM7M2byT8A8M/sNCbf4nHOvZSBvIiLShjV1i+/hJPPuTph2QP+WyY6IiEhIU83M+2UrIyIih8LXpfGm5Op/r/Ck3Zu5iEg+Gja7GK69Fn7727qZd97JvBEjmDx5cu4yJs2WTiMJEZH8pj74WhUFKBFpHQKB+gHqhBNykxdpEQpQItI6fPwx7N9fN33EEdC7d+7yI4csawHKzCaY2VozKzezWxpJd6GZOTM7KVt5E5FW4F//ip8+8UQwS55WCkJWApSZeYEHgInAUEL9+A1Nkq4z8ANgWTbyJSKtyDvvxE/r9l7By1YNajRQ7pxbH+4h/QkgWbOanwG/BA5mKV8i0lokq0FJQTPnXOZ3YnYhMME5d1V4ehowxjk3IybNCcDtzrkLzGwx8CPn3Fux26mqqopmdvHixRnPt4gUCOeYeNllFO/bF531yu9+x75evXKYKWnM2LFjo+9LS0uT34t1zmX8BVwE/G/M9DTg/phpD7AY6BueXgyclLidyspKF3llyty5czO27WxRGfKDypBNfw97AAAWrElEQVRFn3ziHNS9Ond2LhBwzhVQGRrRGsuQ8HmeNHZk6xbfZqAsZro3sCVmujOhwRAXm9kG4BRgvhpKiEhKEm/vnXACeNRIudBl6wyuAAaaWT8zKwYuAeZHFjrnqpxzRzjn+jrn+gJLgUku4RafiEhSev7UKmUlQDnn/MAMYCGwBnjKOfe+md1pZpOykQcRacXUgq9VylpffM65BcCChHl3NJB2bDbyJCKthGpQrZI6ixWRgrT6ypq63su/vDF+4R3g61IT6kBWCpaeIopIQWpsaI1Ulkv+U4ASEZG8pAAlIiJ5SQFKRETykgKUiIjkJQUoERHJSwpQIiKSlxSgRKTgrL6ypsk0vi5ZyIhklAKUiBScVH7jpB/pFj4FKBERyUsKUCIikpcUoEREJC8pQImISF5SgBIRkbykACUiInlJ40GJSMGIGwOqEfoNVOugACUiBSOV4DTqWf3+qbXQLT4REclLClAiIpKXFKBERCQvKUCJiEheUoASEZG8pAAlIiJ5Sc3MRSTv6fdPbZMClIjkPf3+qW3SLT4REclLClAiIpKXFKBERCQvKUCJiEheUoASEZG8pAAlIiJ5Sc3MRSRv6fdPbVvWalBmNsHM1ppZuZndkmT5f5jZB2b2npm9amZ9spU3EclPqf7+adhs/QaqNcpKgDIzL/AAMBEYClxqZkMTkr0DnOScGwE8A/wyG3kTEZH8lK0a1Gig3Dm33jlXAzwBTI5N4Jx73Tm3Pzy5FOidpbyJSB5afWVNrrMgOZatAHU0sClmenN4XkO+A7yY0RyJSF5L5faetG7mnMv8TswuAs5xzl0Vnp4GjHbOXZck7WXADOBrzrnq2GVVVVXRzC5evDijeRaR3Orz6MSU0m28Qt9lC9HYsWOj70tLSy1Zmmy14tsMlMVM9wa2JCYys7OB20gSnBJNnjy5scXNNm/evIxtO1tUhvygMhyalY+mdouvqfzpPOSHxDJUVVU1uU62bvGtAAaaWT8zKwYuAebHJjCzE4AHgUnOuR1ZypeIiOSprNSgnHN+M5sBLAS8wGzn3PtmdifwlnNuPnAv0Al42swAPnXOTcpG/kQkf6T62yfQ759au6z9UNc5twBYkDDvjpj3Z2crLyKSv9JpHKHfP7Vu6upIRETykgKUiBQk3d5r/dQXn4gUHA3v3jaoBiUieUO9R0gsBSgRyRvqPUJiKUCJSF5Q7UkSKUCJSF7Qb58kkQKUiBQU/fap7VCAEpGc0+09SUYBSkRyKp2ujaRtUYASkZxScJKGKECJSM6ke2tPDSTaFgUoEcmZdGtPaiDRtihAiUhBUO2p7VGAEpGCoNpT26MAJSIieUm9mYtIVjWnWblu77VNqkGJSFY1p1m5bu+1TQpQIpLXVHtquxSgRCRrmvO7J9We2i49gxKRjEv3uZNGzBVQDUpEskDdGUlzKECJSEapp3JpLgUoEcko1Z6kufQMSkQyQsNoyKFSDUpEWtyhBCc1K5cIBSgRaXGHUnNSs3KJ0C0+EWkRLXFLT7UniaUAJSKHpKUCk2pOkkgBSkSapSUbQSg4STIKUCKSFrXOk2xRgBKRRq28oAZc5rav507SkDYXoKqqK3jsg59x+dA7OKyka0rrbN5TzgMrb2DGqF9zdOcBae3n346dwTMf/TdgXDnsznr7rKqu4I+r7wCMiwbdwNMf3QfAlcN+Fpf2w11v8eB7NwFGt3a92FOzix7ty7hk8E08/dF91PgPUlG9lR4dyujpOZlZ71xfr4yxZd+ybz1/eO9murbrxRfVFTgHZtCz4zFMOe5Gnvnov6n2H2B3zQ6+ffxPefGT2Rzw72Xngc18b8S9DOp6Yr1j2Zxj25x1JDWRY3vesdfxXPn9jR7j5dduZdFpd2JmnPLadSz/2u/42ku30p3R/O2CHxL0BvAEvJzw5hW8/s07CTrH4ZVlnPrKDbx55v0EPX4AglbLni7bAOhc1YsifztOee06/nnWfew5fAtfXnQ9S8b9mtLKozn1lRt455pfs3P/Zq7b8z90Ku7C/666lR37NtGtQy+8+HA4fJ5iLhx0ffT/6MJB1zO3fFa0XOcde129/5vY66ra9sX9P8T+zyX7n0z3+OrazRzvzJkzc52HlFVXV8+MvG/Xrl2ztvHCuj/w3ud/pyZYzdBup9RbvnbtWgYPHhw373fv/ojKmp2sr1rFV46enNZ+1let4rN95VTV7KQ2yT5fWPcHVlX8k6rw9kNpP6+X9r63voffhbqMOeDfQ8D5+aJ2V3Sdvf7K0LyaXVR5P2Nn7cZ6ZYwt+0ufPEKtq+GAfw9BAtHXFzW7+CRum7W8//mbfH5wC/v8XwDw/udLOKvPpfWOZVPHNp3zkew8FJpclGH1lTVseTTAticDvFTxBz4u/Sfvr3qXrZ6P2fq3/fh++SW2PRng84UBdswLRNOuOOlhNg14k/2dPmf7Ue9T0bMcv6+a7b1XsWnAmxzouIv9nSvY3G8ZtSX7cd4ABzruYvtR77O7xzoOdNzFgY67ONixCucN4LwBDnaojG6vsvsnBL1+Ng1YSqCohgMdd7Gj7H12eNcTcH7WV61i98HtvL/rTQL42VtbyRe1u9hTu4uqmp3RazLyfvPej1hf9V70b+L/Tex19cme1Wzyr467TiP/c8n+J1PVnOu9uVrj/0N1dXX0fbt27X6abJ2s1aDMbALwG8AL/K9z7p6E5SXAn4AvARXAFOfchpbYd+Se+f4OFSyd/hLO51i68UWOvnMK7fd3TdqCKHJbo6JbOdumbgCDbfs28Np3P6RrRX8wGPXX4ib3s21faF2ApZ/W7RNC6d6c/mL0LMSmfXPjgmjaz45+mwPn740uixW7TsR+767Q/mLKGJunf258AXyBpNtLts0D/r31pp/70SssnVB3LLveO56lU1I7tqmcD1L7HiAJIs+G9neooHzoIjBHVbeNYFA+9GVGrphK+/1d454h7e9QwcdDX46e40j6j4cuxJmLO/c17eKvhUjaxsSmCXr90feVh2+Mptm2fwM7D2xucBvb9m+o9z7xL8CyrQv4cq9vsGz7Szgcy7YuIFAcxOFYvu1FTun1dZZue7Eu/bYXGd9nWto1oKrqiug+ljdzG9K0rAQoM/MCDwDjgM3ACjOb75z7ICbZd4DdzrljzewS4BfAlJbYf+Sf8b3Rj+MIAuAsyLsnP84p/zcjtHzWLAD6vfcebNoE7moA/n7OL+O29cY5v+DfHn8wdE8+vE7dfq6ut59YAU9tdJ+RdJFbI4mCXn807Rvn/rxZ5Y4tY1yefIFmbS/WPybUHRdnQd4455dNHtuIZMcpcZ1+f/tb6DxkimuBhyqJ20iY7r9qFWzYUDc/2d/YV7J5zkEwGP8KBOr++v11f/1+IHSsk12Dscc4VkPXYSiYZPDhU4KAS/6/kA6/8/PnNXfhXDA6HXmAFnRB5qy5O24/gWAtL298jAsH/TCt/Sza8Fh0H0EXbNY2pGnZqkGNBsqdc+sBzOwJQt+RYwPUZGBm+P0zwCwzM+da4pOk7htl0Be+V+7zx32j5LrrABgRWeG8q6noVk5Vt0/rviFa6Nvgrm7rQ7Wo8DrErJO4n7hvlx5H+fGhfTpc6FurJ6Z4CWk/HrqQHpuH1fvWGifZ/Mi31XAZB62e2HCeUtlmkunYb8JBnz/+W3IDxzYqyXFKXGfEQw81kcn8NzwXOz1vVoPXYL3zQkztKdl16Enyr9fUtZGMNfA+I1xcjQrqaoAB509YRqiWlWYNKFJ7igS6gPOrFpUh2erq6Ggg9uvw5vC8pGmcc36gCujWUhlo7BtlQxJrTxFvnPOLtPYTK1KLaqz2FBH0+vnnOb9qNE1TEms32dLUsW3O+ZDUNHYNJh7jVK7D1i5Si0pVbO0pIlKLkpaVrRpUsu9NiV/PUkkTNW/evJR33oeJrD/u9bpvlGFBn5/1g1+rd8sjIq72FJPLqm4bk6YHku4njsexfvBr0feN8jiC5j+kb52JtZsW08Q36aaObXPOh6SmsWsw8RivP+71pq/DVs7hWPrZixSt6ZtS+qWHvUTA4o9vwPnT2kZzpPOZl69iyzB27Ngm02crQG0GymKmewNbGkiz2cx8QCmwq6ENTp6c+lP0lY/W0H/tGXw8dGHcP67H76P/h2eGJq69FoD1n3xC/379YAuUVhxTP0g5KK3oE7dO1BaS7idO0EL7NMfaYQsa/3AIGp6gN+52Wro8fh+dK49mT5fPGg+c6XLUOy6x08mObVQDxyl2nfXnnhs6D5lkLRC1E7cRM71u/XoG9O8fPz/Z38grcdoMPJ7QK/Le662b5/OFpr3e0PuiInip8Wsw7rwQStvkddjKGcYpR09k8hmpfabUrv2EpdsWxD3L8pqPU45KfRvpmjdvXlqfefkosQxVVVVNrpOtALUCGGhm/YDPgEuAqQlp5gNXAG8CFwKvtdTzJ4ARy6eGWjXFMOdh5IpwNsIP8lfNm0f/yZPh/Bq+uvAmXpha/9v86Qtvjlsn6vyapPuJ5Q0WxT2DCnpqG0zrCfg47eX/x9/PvafBNE0x5+H0hTexYMoNzd5Gc/ebeGyjGjhOseusuvrq0HkoYKvnzWNAtsvwUuPXYNx5IfR/0dR12Np5PUWM7zMt5fTj+k5j2faX4u7veMyT1jYkNVl5BhV+pjQDWAisAZ5yzr1vZnea2aRwsoeBbmZWDvwHcEtL5qHD/m4c+8E4PP5QTPb4fRz7wfjow+JkulUcS2nFMXUXYrj21LWif8r7ibtJGTSOfT+0zw77uzHwg/EQjPkGnpB24Afn0H/dWIoPdmr4Zmey+eF5kTJ2rRhQP0+Nhf7EZUmmPQFf3LEsreiT1rFtzvmQpvm6NHwNxh5jX5e6tA1eh0HLaA8SmWEc2aEvXvNFpyNl8JqPIzv0JbaqbxhjjpyYVuOG0pJujOk5IboPr/kYneY2JDVZGw/KObfAOTfIOTfAOXdXeN4dzrn54fcHnXMXOeeOdc6NjrT4awmRrlRGLJ+KhYsc+00yaVcr4Wv4qwtvipsdrT0luTuUbD+xIrWniBHLp+IJJq/EegK+aNrTF/w4aZqmxJYxLk9+b7O2F+srL90UdyxPX3hTyse2WedDUjJsdjGjni3m0mum4ykOn+/wteot9jD1e9MZ9Wwxw2YXx6X1emOuw3B6n9dHNoeMqwsqzeczH5cNuQ0zT3TaCF3vHvPwrSG3xu0n3dpTxLi+06L7UO0pc9rEgIWRf8RT/9yLU8omhO4595nIl/98ZPSfNdGov4bWOeuhoRzZsS8AR3bsy5kPDWbUs8X1fqTb0H4i6wKcckzdPiPpvlw2Mbo8Nu2X+5wbTfv1+79Me1+npGWLXSeiQ6BrvTLG5um0Pt9ocHvJtpmYtr2vE+f96uy4Y3nmQ0NSPrbNOR+Snsi3fAvXKAxr8Ft+aUk3Tjky5joMpx/T61x61Rwfl7a9N/5aCNVIGhebxhvzVCF2/pEd+nLKkeemtI1I/mL/RozpdS5Hdx4QLfuYXudyZM3QaPl7dz42rqzp1p4iYo+vak+Z0+a6Ojq680A27fmQ8wf+gBJf+3rLk3Up0vew43lnx2tcNfzulC/EyH4uPu7/sWVvOV1KenDhoOvr7fPozgP5pGo1XUp6cOngm/hsbzldSrrXS3t054G8vX0Rob74jiIQrKVXh35MHfJjPttbTgdvZ2qCB+nVsR99d3+N9j2sXhljy96/ywj+tf0VurU7Cn+gBg8+vObjqE79uXTwLWzZW057bydqXQ3Tj5/Jzv2bKfG2Z79/D1cNv5tu7XvVO5ZNHdt0zkdr7Nol2yLHdspxP2L7/g2NnpfY6/CSwTeyff9Gzh/4Az5fexDvEQfoXHw4h5f05MJBN/Dezr/jwctRHQcwdcgtfLa3nM5Fh3NYcTc6+A6jOnAAr/no0f4YurU/iksG38inX6yhJniQSwffxIe7V9CrQ38uHXIzm/Z8SLX/AN8d/nOGdBtDeeVKDtbup0fHMroUd6dzcVe6tjuSSwbfGP0/iuQvUq4px/2o3v9N7HX1+dqDFPfwx12nkbIm+59M9/imc703V66vpZbQnK6OcM4VzKuystJFXpkyd+7cjG07W1SG/KAy5AeVIT8kliHh8zzpZ36buMUnIiKFx1zLteTOuKqqqsLJrIiIpKS0tDTpjxJVgxIRkbykACUiInmpoG7xiYhI26EalIiI5CUFKBERyUsKUGFmdq+ZfWhm75nZc2bWJTy/r5kdMLOV4dfvc53XhjRUhvCyH5tZuZmtNbNzcpnPxpjZRWb2vpkFzeykmPmFdB6SliG8rCDOQywzm2lmn8Uc+4a7fMgzZjYhfKzLzaxF+/fMFjPbYGarwsf+rVznJxVmNtvMdpjZ6ph5Xc1skZl9HP57eFPbUYCqswgY5pwbAXwExHaAt845Nyr8+l5uspeSpGUws6GEepA/HpgA/NbMDr1DvsxYDZwPvJFkWaGch6RlKLDzkOi/Y479glxnJhXhY/sAMBEYClwaPgeF6IzwsT+p6aR54RFC13isW4BXnXMDgVdJoUNwBagw59zLzkUHeFlKaMyqgtJIGSYDTzjnqp1znwDlwOhc5LEpzrk1zrm1uc7HoWikDAVzHlqJ0UC5c269c64GeILQOZAMc869Qf3x/CYDj4bfPwr8W1PbUYBK7krgxZjpfmb2jpn9n5l9NVeZSlNsGY4GNsUs2xyeV2gK8TzEKuTzMCN863h2Krdm8kQhH+9YDnjZzN42s6tznZlD0NM5txUg/LdHUytka8DCvGBmrwBHJll0m3NuXjjNbYAfmBNethU4xjlXYWZfAuaa2fHOuS+ykukEzSxDsl9p5+z3BamUIYmCOw/JVksyLy9+59FYeYDfAT8jlNefAf9F6AtQvsvb452m05xzW8ysB7DIzD4M11BavTYVoJxzZze23MyuAL4BnOXCPxBzzlUD1eH3b5vZOmAQkJOHlc0pA6FvjmUxyXoDWzKTw6Y1VYYG1imo89CAvDoPsVItj5k9BLyQ4ey0lLw93ulwzm0J/91hZs8RunVZiAFqu5n1cs5tNbNewI6mVtAtvjAzmwDcDExyzu2Pmd898iDbzPoDA4EWG0yxJTVUBmA+cImZlZhZP0JlWJ6LPDZXIZ2HRhTkeQh/mEScR6gRSCFYAQw0s35mVkyogcr8HOcpLWbW0cw6R94D4ymc459oPnBF+P0VQEN3GqLaVA2qCbOAEkJVaICl4ZZipwN3mpkfCADfc84lPvzLF0nL4Jx738yeAj4gdOvvWudcIIf5bJCZnQfcD3QH/mZmK51z51BA56GhMhTSeUjwSzMbRej22AbgmtxmJzXOOb+ZzQAWAl5gtnPu/RxnK109gefC/88+4HHn3Eu5zVLTzOwvwFjgCDPbDPwEuAd4ysy+A3wKXNTkdtTVkYiI5CPd4hMRkbykACUiInlJAUpERPKSApSIiOQlBSgREclLClAiIpKXFKBERCQvKUCJiEheUoASyRNmNsXM9sa8qs1sca7zJZIrClAiecI596RzrpNzrhNwFKG+Bv+S42yJ5Iy6OhLJM2bmIdSx5ibn3PdznR+RXFENSiT/3AV0Bn6Q64yI5JJ6MxfJI2Z2CXApcLJzrjbX+RHJJd3iE8kTZnYC8DIwzjm3Mtf5Eck13eITyR+TgcOBf8S05Hsx15kSyRXVoEREJC+pBiUiInlJAUpERPKSApSIiOQlBSgREclLClAiIpKXFKBERCQvKUCJiEheUoASEZG89P8D7wH2/LqfNQkAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "def plot_boundary_threshold(Zs, Hs, y, threshold=0.5):\n",
    "    data_fit = np.concatenate((Zs, Hs), axis=1)\n",
    "    data_fit.sort(axis = 0)\n",
    "\n",
    "    z = np.linspace(Zs.min(), Zs.max(), 100)\n",
    "\n",
    "    plt.plot(z, logistic(z), 'r-', label='Theory')\n",
    "    plt.plot(Zs[Hs>threshold], Hs[Hs>threshold], 'X', label='empirical 1', markersize=8)\n",
    "    plt.plot(Zs[Hs<=threshold], Hs[Hs<=threshold], 's', label='empirical 0', markersize=8)\n",
    "\n",
    "    plt.plot(Zs, y, 'd', label = 'data', markersize=8)\n",
    "    plt.xlabel('z')\n",
    "    plt.ylabel('h(z)')    \n",
    "    plt.plot([Zs.min(), Zs.max()], [threshold, threshold],  lw=5, linestyle=':', color='darkgray')\n",
    "    plt.legend()\n",
    "\n",
    "plot_boundary_threshold(Zs, Hs, y, 0.7)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## No Assignment !!! \n",
    "\n",
    "### Do it Yourself !!!\n",
    ">Use your data to experiment with regression by choosing one of three regression types.  \n",
    "> It's all up to You!!  \n",
    "\n",
    "\n",
    "### And Try running the following code and put comments (explanations) on each line of code and write a description of the output\n",
    "\n",
    "> It's all up to You!!  Not required!!!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### MNIST "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0.00237792 0.95932017 0.00646385 0.00237792 0.00237792 0.00237792\n",
      " 0.00237792 0.01757056 0.00237792 0.00237792]\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "\n",
    "# softmax \n",
    "def softmax(x):\n",
    "    return np.exp(x) / np.sum(np.exp(x), axis=0)\n",
    "scores = [1, 7, 2, 1, 1, 1, 1, 3, 1, 1]\n",
    "print(softmax(scores))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "from __future__ import absolute_import, division, print_function, unicode_literals\n",
    "\n",
    "try:\n",
    "  %tensorflow_version 2.x\n",
    "except Exception:\n",
    "  pass"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\Owner\\miniconda3\\envs\\jeff_heaton_t81_park\\lib\\site-packages\\tensorboard\\compat\\tensorflow_stub\\dtypes.py:541: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n",
      "  _np_qint8 = np.dtype([(\"qint8\", np.int8, 1)])\n",
      "C:\\Users\\Owner\\miniconda3\\envs\\jeff_heaton_t81_park\\lib\\site-packages\\tensorboard\\compat\\tensorflow_stub\\dtypes.py:542: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n",
      "  _np_quint8 = np.dtype([(\"quint8\", np.uint8, 1)])\n",
      "C:\\Users\\Owner\\miniconda3\\envs\\jeff_heaton_t81_park\\lib\\site-packages\\tensorboard\\compat\\tensorflow_stub\\dtypes.py:543: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n",
      "  _np_qint16 = np.dtype([(\"qint16\", np.int16, 1)])\n",
      "C:\\Users\\Owner\\miniconda3\\envs\\jeff_heaton_t81_park\\lib\\site-packages\\tensorboard\\compat\\tensorflow_stub\\dtypes.py:544: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n",
      "  _np_quint16 = np.dtype([(\"quint16\", np.uint16, 1)])\n",
      "C:\\Users\\Owner\\miniconda3\\envs\\jeff_heaton_t81_park\\lib\\site-packages\\tensorboard\\compat\\tensorflow_stub\\dtypes.py:545: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n",
      "  _np_qint32 = np.dtype([(\"qint32\", np.int32, 1)])\n",
      "C:\\Users\\Owner\\miniconda3\\envs\\jeff_heaton_t81_park\\lib\\site-packages\\tensorboard\\compat\\tensorflow_stub\\dtypes.py:550: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n",
      "  np_resource = np.dtype([(\"resource\", np.ubyte, 1)])\n"
     ]
    }
   ],
   "source": [
    "import tensorflow as tf\n",
    "tf.random.set_seed(1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import tensorflow as tf\n",
    "from tensorflow.keras.models import Sequential\n",
    "from tensorflow.keras.layers import Flatten\n",
    "from tensorflow.keras.layers import Dense\n",
    "from tensorflow.keras.layers import Activation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Get MNIST Data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "(x_train, y_train), (x_test, y_test) = tf.keras.datasets.mnist.load_data()\n",
    "x_train = x_train.astype('float32')\n",
    "x_test = x_test.astype('float32')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "gray_scale = 255\n",
    "x_train /= gray_scale\n",
    "x_test /= gray_scale"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[[0., 0., 0., ..., 0., 0., 0.],\n",
       "        [0., 0., 0., ..., 0., 0., 0.],\n",
       "        [0., 0., 0., ..., 0., 0., 0.],\n",
       "        ...,\n",
       "        [0., 0., 0., ..., 0., 0., 0.],\n",
       "        [0., 0., 0., ..., 0., 0., 0.],\n",
       "        [0., 0., 0., ..., 0., 0., 0.]],\n",
       "\n",
       "       [[0., 0., 0., ..., 0., 0., 0.],\n",
       "        [0., 0., 0., ..., 0., 0., 0.],\n",
       "        [0., 0., 0., ..., 0., 0., 0.],\n",
       "        ...,\n",
       "        [0., 0., 0., ..., 0., 0., 0.],\n",
       "        [0., 0., 0., ..., 0., 0., 0.],\n",
       "        [0., 0., 0., ..., 0., 0., 0.]],\n",
       "\n",
       "       [[0., 0., 0., ..., 0., 0., 0.],\n",
       "        [0., 0., 0., ..., 0., 0., 0.],\n",
       "        [0., 0., 0., ..., 0., 0., 0.],\n",
       "        ...,\n",
       "        [0., 0., 0., ..., 0., 0., 0.],\n",
       "        [0., 0., 0., ..., 0., 0., 0.],\n",
       "        [0., 0., 0., ..., 0., 0., 0.]],\n",
       "\n",
       "       ...,\n",
       "\n",
       "       [[0., 0., 0., ..., 0., 0., 0.],\n",
       "        [0., 0., 0., ..., 0., 0., 0.],\n",
       "        [0., 0., 0., ..., 0., 0., 0.],\n",
       "        ...,\n",
       "        [0., 0., 0., ..., 0., 0., 0.],\n",
       "        [0., 0., 0., ..., 0., 0., 0.],\n",
       "        [0., 0., 0., ..., 0., 0., 0.]],\n",
       "\n",
       "       [[0., 0., 0., ..., 0., 0., 0.],\n",
       "        [0., 0., 0., ..., 0., 0., 0.],\n",
       "        [0., 0., 0., ..., 0., 0., 0.],\n",
       "        ...,\n",
       "        [0., 0., 0., ..., 0., 0., 0.],\n",
       "        [0., 0., 0., ..., 0., 0., 0.],\n",
       "        [0., 0., 0., ..., 0., 0., 0.]],\n",
       "\n",
       "       [[0., 0., 0., ..., 0., 0., 0.],\n",
       "        [0., 0., 0., ..., 0., 0., 0.],\n",
       "        [0., 0., 0., ..., 0., 0., 0.],\n",
       "        ...,\n",
       "        [0., 0., 0., ..., 0., 0., 0.],\n",
       "        [0., 0., 0., ..., 0., 0., 0.],\n",
       "        [0., 0., 0., ..., 0., 0., 0.]]], dtype=float32)"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x_test"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# MLP Model for classifiation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "model = Sequential([\n",
    "    Flatten(input_shape=(28, 28)),    # reshape 28 row x 28 column data to Flatten (28*28 rows)\n",
    "    Dense(256, activation='sigmoid'), # dense layer 1\n",
    "    Dense(128, activation='sigmoid'), # dense layer 2\n",
    "    Dense(10, activation='softmax'),  # dense layer 3\n",
    "])\n",
    "\n",
    "model.compile(optimizer='adam',\n",
    "              loss='sparse_categorical_crossentropy',\n",
    "              metrics=['accuracy'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Train"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/10\n",
      "24/24 [==============================] - 1s 40ms/step - loss: 2.2103 - accuracy: 0.2821 - val_loss: 1.6765 - val_accuracy: 0.6772\n",
      "Epoch 2/10\n",
      "24/24 [==============================] - 0s 18ms/step - loss: 1.4961 - accuracy: 0.6937 - val_loss: 0.9877 - val_accuracy: 0.8015\n",
      "Epoch 3/10\n",
      "24/24 [==============================] - 0s 18ms/step - loss: 0.9065 - accuracy: 0.8072 - val_loss: 0.6429 - val_accuracy: 0.8677\n",
      "Epoch 4/10\n",
      "24/24 [==============================] - 0s 17ms/step - loss: 0.6213 - accuracy: 0.8629 - val_loss: 0.4766 - val_accuracy: 0.8901\n",
      "Epoch 5/10\n",
      "24/24 [==============================] - 0s 17ms/step - loss: 0.4816 - accuracy: 0.8846 - val_loss: 0.3923 - val_accuracy: 0.9030\n",
      "Epoch 6/10\n",
      "24/24 [==============================] - 0s 17ms/step - loss: 0.4052 - accuracy: 0.8976 - val_loss: 0.3438 - val_accuracy: 0.9107\n",
      "Epoch 7/10\n",
      "24/24 [==============================] - 0s 17ms/step - loss: 0.3545 - accuracy: 0.9056 - val_loss: 0.3120 - val_accuracy: 0.9163\n",
      "Epoch 8/10\n",
      "24/24 [==============================] - 0s 18ms/step - loss: 0.3203 - accuracy: 0.9130 - val_loss: 0.2903 - val_accuracy: 0.9215\n",
      "Epoch 9/10\n",
      "24/24 [==============================] - 0s 18ms/step - loss: 0.3051 - accuracy: 0.9151 - val_loss: 0.2714 - val_accuracy: 0.9253\n",
      "Epoch 10/10\n",
      "24/24 [==============================] - 0s 18ms/step - loss: 0.2806 - accuracy: 0.9214 - val_loss: 0.2568 - val_accuracy: 0.9292\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<tensorflow.python.keras.callbacks.History at 0x14c820894e0>"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.fit(x_train, y_train, epochs=10, batch_size=2000, validation_split = 0.2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Test"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "test loss, test acc: [0.26460137963294983, 0.9257000088691711]\n"
     ]
    }
   ],
   "source": [
    "results = model.evaluate(x_test,  y_test, verbose = 0)\n",
    "\n",
    "print('test loss, test acc:', results)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3.6 (tensorflow)",
   "language": "python",
   "name": "jeff_heaton_t81_park"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}