Lesson15-Python进阶ML-MultiCollinearity In Linear Regression多重共线性及VIF.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 第十五课 MultiCollinearity In Linear Regression多重共线性及VIF \n",
    "\n",
    "\n",
    "Python零基础学习：https://www.ixigua.com/home/77346806707?utm_source=xiguastudio\n",
    "\n",
    "源文件下载链接: https://pan.baidu.com/s/1yuNlG6u9_C31fzhzbzqASA 提取码: ebrh\n",
    "\n",
    "Multicollinearity refers to a situation in which two or more explanatory variables in a multiple regression model are highly linearly related. We have perfect multicollinearity if, for example as in the equation above, the correlation between two independent variables is equal to 1 or −1.\n",
    "\n",
    "Multicollinearity occurs when independent variables in a regression model are correlated. This correlation is a problem because independent variables should be independent. If the degree of correlation between variables is high enough, it can cause problems when you fit the model and interpret the results\n",
    "\n",
    "Variance Inflation Factor (VIF)\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 252,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Import necessary libraries \n",
    "import pandas as pd \n",
    "import numpy as np \n",
    "import matplotlib.pyplot as plt \n",
    "import seaborn as sns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 253,
   "metadata": {},
   "outputs": [],
   "source": [
    "df=pd.read_csv('Lesson15-Salary_Data.csv')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 254,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>YearsExperience</th>\n",
       "      <th>Salary</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>24</td>\n",
       "      <td>1.1</td>\n",
       "      <td>39343</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>25</td>\n",
       "      <td>1.3</td>\n",
       "      <td>46205</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0</td>\n",
       "      <td>24</td>\n",
       "      <td>1.5</td>\n",
       "      <td>37731</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>25</td>\n",
       "      <td>2.0</td>\n",
       "      <td>43525</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0</td>\n",
       "      <td>25</td>\n",
       "      <td>2.2</td>\n",
       "      <td>39891</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>1</td>\n",
       "      <td>26</td>\n",
       "      <td>2.9</td>\n",
       "      <td>56642</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>0</td>\n",
       "      <td>27</td>\n",
       "      <td>3.0</td>\n",
       "      <td>60150</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>1</td>\n",
       "      <td>26</td>\n",
       "      <td>3.2</td>\n",
       "      <td>54445</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>1</td>\n",
       "      <td>27</td>\n",
       "      <td>3.2</td>\n",
       "      <td>64445</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>1</td>\n",
       "      <td>27</td>\n",
       "      <td>3.7</td>\n",
       "      <td>57189</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>0</td>\n",
       "      <td>28</td>\n",
       "      <td>3.9</td>\n",
       "      <td>63218</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>0</td>\n",
       "      <td>28</td>\n",
       "      <td>4.0</td>\n",
       "      <td>55794</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>0</td>\n",
       "      <td>28</td>\n",
       "      <td>4.0</td>\n",
       "      <td>56957</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>1</td>\n",
       "      <td>29</td>\n",
       "      <td>4.1</td>\n",
       "      <td>57081</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>1</td>\n",
       "      <td>29</td>\n",
       "      <td>4.5</td>\n",
       "      <td>61111</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>1</td>\n",
       "      <td>30</td>\n",
       "      <td>4.9</td>\n",
       "      <td>67938</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>1</td>\n",
       "      <td>31</td>\n",
       "      <td>5.1</td>\n",
       "      <td>66029</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>0</td>\n",
       "      <td>32</td>\n",
       "      <td>5.3</td>\n",
       "      <td>83088</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>0</td>\n",
       "      <td>32</td>\n",
       "      <td>5.9</td>\n",
       "      <td>81363</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>0</td>\n",
       "      <td>33</td>\n",
       "      <td>6.0</td>\n",
       "      <td>93940</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>0</td>\n",
       "      <td>34</td>\n",
       "      <td>6.8</td>\n",
       "      <td>91738</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>0</td>\n",
       "      <td>35</td>\n",
       "      <td>7.1</td>\n",
       "      <td>98273</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>1</td>\n",
       "      <td>36</td>\n",
       "      <td>7.9</td>\n",
       "      <td>101302</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>1</td>\n",
       "      <td>36</td>\n",
       "      <td>8.2</td>\n",
       "      <td>113812</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>1</td>\n",
       "      <td>38</td>\n",
       "      <td>8.7</td>\n",
       "      <td>109431</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>1</td>\n",
       "      <td>38</td>\n",
       "      <td>9.0</td>\n",
       "      <td>105582</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>1</td>\n",
       "      <td>39</td>\n",
       "      <td>9.5</td>\n",
       "      <td>116969</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>1</td>\n",
       "      <td>39</td>\n",
       "      <td>9.6</td>\n",
       "      <td>112635</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>0</td>\n",
       "      <td>40</td>\n",
       "      <td>10.3</td>\n",
       "      <td>122391</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>0</td>\n",
       "      <td>41</td>\n",
       "      <td>10.5</td>\n",
       "      <td>121872</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    Sex  Age  YearsExperience  Salary\n",
       "0     0   24              1.1   39343\n",
       "1     1   25              1.3   46205\n",
       "2     0   24              1.5   37731\n",
       "3     1   25              2.0   43525\n",
       "4     0   25              2.2   39891\n",
       "5     1   26              2.9   56642\n",
       "6     0   27              3.0   60150\n",
       "7     1   26              3.2   54445\n",
       "8     1   27              3.2   64445\n",
       "9     1   27              3.7   57189\n",
       "10    0   28              3.9   63218\n",
       "11    0   28              4.0   55794\n",
       "12    0   28              4.0   56957\n",
       "13    1   29              4.1   57081\n",
       "14    1   29              4.5   61111\n",
       "15    1   30              4.9   67938\n",
       "16    1   31              5.1   66029\n",
       "17    0   32              5.3   83088\n",
       "18    0   32              5.9   81363\n",
       "19    0   33              6.0   93940\n",
       "20    0   34              6.8   91738\n",
       "21    0   35              7.1   98273\n",
       "22    1   36              7.9  101302\n",
       "23    1   36              8.2  113812\n",
       "24    1   38              8.7  109431\n",
       "25    1   38              9.0  105582\n",
       "26    1   39              9.5  116969\n",
       "27    1   39              9.6  112635\n",
       "28    0   40             10.3  122391\n",
       "29    0   41             10.5  121872"
      ]
     },
     "execution_count": 254,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 255,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>YearsExperience</th>\n",
       "      <th>Salary</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>24</td>\n",
       "      <td>1.1</td>\n",
       "      <td>39343</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>25</td>\n",
       "      <td>1.3</td>\n",
       "      <td>46205</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0</td>\n",
       "      <td>24</td>\n",
       "      <td>1.5</td>\n",
       "      <td>37731</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>25</td>\n",
       "      <td>2.0</td>\n",
       "      <td>43525</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0</td>\n",
       "      <td>25</td>\n",
       "      <td>2.2</td>\n",
       "      <td>39891</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Sex  Age  YearsExperience  Salary\n",
       "0    0   24              1.1   39343\n",
       "1    1   25              1.3   46205\n",
       "2    0   24              1.5   37731\n",
       "3    1   25              2.0   43525\n",
       "4    0   25              2.2   39891"
      ]
     },
     "execution_count": 255,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 256,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>YearsExperience</th>\n",
       "      <th>Salary</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Sex</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.050028</td>\n",
       "      <td>0.066731</td>\n",
       "      <td>0.045346</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Age</th>\n",
       "      <td>0.050028</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.994212</td>\n",
       "      <td>0.982714</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>YearsExperience</th>\n",
       "      <td>0.066731</td>\n",
       "      <td>0.994212</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.978242</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Salary</th>\n",
       "      <td>0.045346</td>\n",
       "      <td>0.982714</td>\n",
       "      <td>0.978242</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                      Sex       Age  YearsExperience    Salary\n",
       "Sex              1.000000  0.050028         0.066731  0.045346\n",
       "Age              0.050028  1.000000         0.994212  0.982714\n",
       "YearsExperience  0.066731  0.994212         1.000000  0.978242\n",
       "Salary           0.045346  0.982714         0.978242  1.000000"
      ]
     },
     "execution_count": 256,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.corr()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 257,
   "metadata": {},
   "outputs": [],
   "source": [
    "X=X1=df.iloc[:,:-1]\n",
    "y=df.iloc[:,-1]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 第一种方式：使用.corr()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 258,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>YearsExperience</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Sex</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.050028</td>\n",
       "      <td>0.066731</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Age</th>\n",
       "      <td>0.050028</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.994212</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>YearsExperience</th>\n",
       "      <td>0.066731</td>\n",
       "      <td>0.994212</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                      Sex       Age  YearsExperience\n",
       "Sex              1.000000  0.050028         0.066731\n",
       "Age              0.050028  1.000000         0.994212\n",
       "YearsExperience  0.066731  0.994212         1.000000"
      ]
     },
     "execution_count": 258,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X.corr()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 第二种方式：使用pairplot"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 259,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<seaborn.axisgrid.PairGrid at 0x1895ccab348>"
      ]
     },
     "execution_count": 259,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAhUAAAIVCAYAAABm5A1+AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAABKQ0lEQVR4nO3de5wcdZ3v/9dnZhInJJkhhMlkSDIE3KgwkYTsiIqXRVBOdFHiLYiruP7YDZ6fCrvsniO78vPK/s5e3Byvq0RlFXWVuGhAlkU4XLwsiEQMISMiEAOETCaTiJMLDMmkP+ePrp70TLpnqqaruqu638/Hox/TXbf+dPW3qj9TVd9PmbsjIiIiUqmmWgcgIiIi9UFJhYiIiMRCSYWIiIjEQkmFiIiIxEJJhYiIiMRCSYWIiIjEInNJxcqVKx3QQ4+kH6GoPepRpUcoao96VOlRVuaSit27d9c6BJFRao+SJmqPUmuZSypEREQknRJLKszsGjPbZWZbyow3M/usmT1qZpvNbEVSsYiIiEjyWhJc9teAzwPXlhn/emBJ8Hgp8MXgr0jdGxnJ0dc/RP/QMF3tM+jpaqOlpSm28ZW+P0Au52zbc4CBvcN0trWyeO5Mmpos9Pg0qHQ9hVlGHO9RLVG+s6l8v6XmyeW87Pop9x6VDO9qb+VwDnbtS2+7hKM/e/ecY3ji6WdCb29hpn/idwcY2PscBw6OcOJxMznp+OTXRWJJhbv/2MwWTzDJ+cC1nr/5yM/M7Fgz63L3/qRiEkmDkZEcGx54iis3bGH4UI7WaU1ctWopq5YtoKWlqeLxlb4/5HdIt/Tt5PL1m0anWbt6OSt75o/uxCcanwaVrqcwy4jjPaolync2le+31DxXv3sFg/sOllw/TU1W8j3OPaWTWx8amNLwOcdM56KXn8hnbn8kte0Sjl5XJ86dwQfPXjJmPU20vYWZ/o6HB3hkYH/V10UtW/0C4Mmi19uDYSJ1ra9/aHRnADB8KMeVG7bQ1z8Uy/hK3x9g254DozuwwjSXr9/Etj0HQo1Pg0rXU5hlxPEe1RLlO5vK91tqnn3PHi67fsq9R1//0JSHv2XFwtEf0bBx18L4z37eaQuOWk8TbW9hpt+8fagm66KWSUWpVKlkVxUzW2NmG81s4+DgYMJhSZosWNSNmSXyWLCoe0oxVdoe+4eGRzf0guFDOXYODccyvtL3BxjYW3qaXfuGQ41Pg0rXU5hlxPEelQrbHqN8Z1P5fkvNc+C5kbLrp9x7lFunYYabkfp2CUevq8ninsr0Oa/NukjymorJbAcWFb1eCOwoNaG7rwPWAfT29k7YR1bqy47tT3LB1XcnsuzrLjlzSvNV2h672mfQOq1pzAbfOq2J+e2tsYyv9P0BOttaS04zb3ZrqPFpUOl6CrOMON6jUmHbY5TvbCrfb6l5Zra2lF0/s1unlRxXbp2GHZ72dgnl12/U7W2i6ZutNuuilkcqbgQuCnqBvAwY0vUU0gh6utq4atVSWqflN7/CeeaervZYxlf6/gCL585k7erlY6ZZu3o5i+fODDU+DSpdT2GWEcd7VEuU72wq32+peWa3NpddP+Xeo6erbcrDr//Fdi47Z0mq2yUcva5+8MBTR62niba3MNO/eGF7TdaF5a+TTGDBZt8GzgKOBwaAjwLTANz9S2Zm5HuHrASeAd7r7hsnW25vb69v3DjpZFInzCzRIxUTtP9QVzJNtT0WegzsHBpmfnsrPV3tJXsUTHV8pe8PR64237VvmHmzy1+NXm58GlS6nsIsI473CCGW9hjlO5vK91tqnkLvj1Lrp9x7VDJ8flu+98fg/vS2Szj6sxd6c4Td3sJMX+j98czBEbrj7f1RdiGJJRVJUVLRWOo1qRCJSO1R0qRse0xXnycRERHJLCUVIiIiEgslFSIiIhILJRUiIiISCyUVIiIiEgslFSIiIhILJRUiIiISCyUVIiIiEgslFSIiIhILJRUiIiISCyUVIiIiEgslFSIiIhILJRUiIiISCyUVIiIiEou6SioWLOrGzBJ7LFjUXeuPKCIiklottQ4gTju2P8kFV9+d2PKvu+TMxJYtIiKSdXV1pEJERERqR0mFiIiIxEJJhYiIiMRCSYWIiIjEItGkwsxWmtnDZvaomV1RYny7mf3AzB4wsz4ze2+S8YiIiEhyEksqzKwZ+ALweuBU4EIzO3XcZO8HfuXuy4CzgH82s+lJxSQiIiLJSfJIxRnAo+6+1d0PAt8Bzh83jQOzzcyAWcDvgJEEYxIREZGEJJlULACeLHq9PRhW7PPAKcAO4EHgMnfPjV+Qma0xs41mtnFwcDCpeEVCUXuUNFF7lDRJMqmwEsN83Ov/BmwCTgCWA583s7ajZnJf5+697t7b0dERd5wikag9SpqoPUqaJJlUbAcWFb1eSP6IRLH3At/zvEeB3wIvSjAmERERSUiSScV9wBIzOym4+PIdwI3jpnkCOAfAzDqBFwJbE4xJREREEpLYvT/cfcTMPgD8EGgGrnH3PjN7XzD+S8Anga+Z2YPkT5d8yN13JxWTiIiIJCfRG4q5+83AzeOGfano+Q7g3CRjEBERkepQRU0RERGJhZIKERERiYWSChEREYmFkgoRERGJhZIKERERiYWSChEREYmFkgoRERGJhZIKERERiYWSChEREYmFkgoRERGJhZIKERERiYWSChEREYmFkgoRERGJhZIKERERiYWSChEREYmFkgoRERGJhZIKERERiYWSChEREYmFkgoRERGJRaJJhZmtNLOHzexRM7uizDRnmdkmM+szsx8lGY+IiIgkpyWpBZtZM/AF4HXAduA+M7vR3X9VNM2xwL8AK939CTObl1Q8IiIikqwkj1ScATzq7lvd/SDwHeD8cdO8E/ieuz8B4O67EoxHREREEpRkUrEAeLLo9fZgWLEXAHPM7C4z+4WZXZRgPCIiIpKgxE5/AFZimJd4/z8EzgFmAPeY2c/c/TdjFmS2BlgD0N3dnUCoIuGpPUqaqD1KmiR5pGI7sKjo9UJgR4lpbnH3A+6+G/gxsGz8gtx9nbv3untvR0dHYgGLhKH2KGmi9ihpkmRScR+wxMxOMrPpwDuAG8dNcwPwKjNrMbNjgJcCDyUYk4iIiCQksdMf7j5iZh8Afgg0A9e4e5+ZvS8Y/yV3f8jMbgE2AzngK+6+JamYREREJDlJXlOBu98M3Dxu2JfGvf4n4J+SjENERESSp4qaIiIiEgslFSIiIhILJRUiIiISi1BJhZldPO51s5l9NJmQREREJIvCHqk4x8xuNrMuM1sK/AyYnWBcIiIikjGhen+4+zvN7ALgQeAZ4EJ3/69EIxMREZFMCXv6YwlwGXA9sA14d1CsSkRERAQIf/rjB8BH3P0S4I+AR8hXzBQREREBwhe/OsPd9wK4uwP/bGbjS26LiIhIAwt7pGKGmX01KKmNmZ0KvDq5sERERCRrwiYVXyN/D4+u4PVvgL9IIB4RERHJqLBJxfHuvp78Tb9w9xHgcGJRiYiISOaETSoOmNlcwAHM7GXAUGJRiYiISOaEvVDzcuBG4Plm9l9AB/C2xKISERGRzJnwSIWZvcTM5rv7/eS7kv4t8BxwK7C9CvGJiIhIRkx2+uNq4GDw/Ezgw8AXgKeBdQnGJSIiIhkz2emPZnf/XfD8AmCdu18PXG9mmxKNTERERDJlsiMVzWZWSDzOAe4oGhf2egwRERFpAJMlBt8GfmRmu4FngZ8AmNkfoN4fIiIiUmTCpMLd/87Mbidf9OrWoEQ35I9wfDDp4ERERCQ7Jq1T4e4/c/fvu/uBomG/CXqETMjMVprZw2b2qJldMcF0LzGzw2ambqoiIiIZFbb4VWRm1ky+p8jrgVOBC4N7hpSa7h/IlwEXERGRjEosqQDOAB51963ufhD4DnB+iek+CFwP7EowFhEREUlYkknFAuDJotfbg2GjzGwB8GbgSxMtyMzWmNlGM9s4ODgYe6AiUag9SpqoPUqaJJlUWIlhPu71p4EPufuENydz93Xu3uvuvR0dHXHFJzIlao+SJmqPkiZJ1prYDiwqer0Q2DFuml7gO2YGcDzwBjMbcfcNCcYlIiIiCUgyqbgPWGJmJwFPAe8A3lk8gbufVHhuZl8DblJCISIikk2JJRXuPmJmHyDfq6MZuMbd+8zsfcH4Ca+jEBERkWxJtNS2u98M3DxuWMlkwt3/NMlYREREJFlJXqgpIiIiDURJhYiIiMRCSYWIiIjEQkmFiIiIxEJJhYiIiMRCSYWIiIjEQkmFiIiIxEJJhYiIiMRCSYWIiIjEQkmFiIiIxEJJhYiIiMRCSYWIiIjEQkmFiIiIxEJJhYiIiMRCSYWIiIjEQkmFiIiIxEJJhYiIiMRCSYWIiIjEQkmFiIiIxEJJhYiIiMQi0aTCzFaa2cNm9qiZXVFi/J+Y2ebgcbeZLUsyHhEREUlOYkmFmTUDXwBeD5wKXGhmp46b7LfAH7n7acAngXVJxSMiIiLJSvJIxRnAo+6+1d0PAt8Bzi+ewN3vdveng5c/AxYmGI+IiIgkKMmkYgHwZNHr7cGwci4G/rPUCDNbY2YbzWzj4OBgjCGKRKf2KGmi9ihpkmRSYSWGeckJzV5DPqn4UKnx7r7O3XvdvbejoyPGEEWiU3uUNFF7lDRpSXDZ24FFRa8XAjvGT2RmpwFfAV7v7nsSjEdEREQSlOSRivuAJWZ2kplNB94B3Fg8gZl1A98D3u3uv0kwFhEREUlYYkcq3H3EzD4A/BBoBq5x9z4ze18w/kvAR4C5wL+YGcCIu/cmFZOIiIgkJ8nTH7j7zcDN44Z9qej5nwF/lmQMIiIiUh2qqCkiIiKxUFIhIiIisVBSISIiIrFQUiEiIiKxUFIhIiIisVBSISIiIrFQUiEiIiKxUFIhIiIisVBSISIiIrFQUiEiIiKxUFIhIiIisVBSISIiIrFQUiEiIiKxUFIhIiIisVBSISIiIrFQUiEiIiKxUFIhIiIisVBSISIiIrFQUiEiIiKxSDSpMLOVZvawmT1qZleUGG9m9tlg/GYzW5FkPCIiIpKclqQWbGbNwBeA1wHbgfvM7EZ3/1XRZK8HlgSPlwJfDP6KZNbISI6+/iH6h4bpap9BT1cbLS1j8/dcztm25wADe4fpbGtl8dyZNDVZ6PeYbP6DBw+zeccQO/cO09XWyotPaGf69ObYPmMYk62HMOtgsmkqXY/VUOvPUG75Ud43aoxT+UyFeYaePcjhHOze/1zZ7afSzz5+mid+d4CBvc9x4OAIJx43k5OOj/75gNS3xXKKP09XeyuHc7Br39Q+R2JJBXAG8Ki7bwUws+8A5wPFScX5wLXu7sDPzOxYM+ty9/4E4xJJzMhIjg0PPMWVG7YwfChH67Qmrlq1lFXLFozuGHM555a+nVy+ftPoNGtXL2dlz/xQG+9k8x88eJgNm3fwkRuOxPCJ85ey6rQTqpZYTLYewqyDyaapdD1WQ60/Q7nln3tKJ7c+NBDqfaPGOJXPVJjnmp8+xltXdPPxm/rKbj+VfvbxbeyOhwd4ZGA/n7n9kSl/vs+/83QOjniq22I5xZ9nzjHTuejlJ4ZeF6UkefpjAfBk0evtwbCo04hkRl//0OgPKcDwoRxXbthCX//Q6DTb9hwY3fkUprl8/Sa27TkQ6j0mm3/zjqHRhKIw/iM3bGHzjqGyy4zbZOshzDqYbJpK12M11PozlFt+X/9Q6PeNGuNUPlNhnovOPHk0oSjMO377qfSzj29jm7cPjf6ITvXzbd4efn2mTfHnecuKhZHWRSlJJhWl0hqfwjSY2Roz22hmGwcHB2MJTmSqJmqP/UPDoxtkwfChHDuHhkdfD+wtPc2ufcOEMdn8O8uMH9gbbvlxmGw9hFkHk01T6Xqshmp8honaY7nll/t+Sr1v1Bin8pkK8zz73Mik209YYdtYzqn480VdRpoUfx6zyj9HkknFdmBR0euFwI4pTIO7r3P3Xnfv7ejoiD1QkSgmao9d7TNonTZ2s2qd1sT89tbR151trSWnmTe7lTAmm7+rzPjOtnDLj8Nk6yHMOphsmkrXYzVU4zNM1B7LLb/c91PqfaPGOJXPVJjnmOe1TLr9hBW2jTUbFX++qMtIk/Gfp9LPkWRScR+wxMxOMrPpwDuAG8dNcyNwUdAL5GXAkK6nkCzr6WrjqlVLRzfMwjnhnq720WkWz53J2tXLx0yzdvXy0Yu9JjPZ/C8+oZ1PnD82hk+cv5TTTmgvu8y4TbYewqyDyaapdD1WQ60/Q7nl93S1hX7fqDFO5TMV5vn63Vv56Hk9E24/lX728W3sxQvbueycJRV9vhcvbE99Wyyn+PNc/4vtkdZFKZa/RjIZZvYG4NNAM3CNu/+dmb0PwN2/ZGYGfB5YCTwDvNfdN060zN7eXt+4sfQkZsYFV98d4ycY67pLziTJ9SVHS/I7neT7DHVVUqn2WOj1sHNomPntrfR0tZft/bFr3zDzZk+990e5+Qu9PwpXop9Ww94f5dZDmHUw2TSVrsdqiOkzTLk9llt+lHUXdT1P5XspzLP32YOMjPb+KL39hBW2jRV6fzxzcITuCL0/ipcLpL4tllP8eea35Xt/DO6fWntMNKlIgpKKxpLFpEIkAWqPkiZl26MqaoqIiEgslFSIiIhILDJ3+sPMBoHHy4w+HthdxXAmolhKy0osu9195WQLmKQ9VhpDWijGeFQSY7XaY1RpXe9pjKueYirbHjOXVEzEzDa6e2+t4wDFUo5iSV8Mk1GM8chCjFGl9TOlMa5GiUmnP0RERCQWSipEREQkFvWWVKyrdQBFFEtpimWsNMQwGcUYjyzEGFVaP1Ma42qImOrqmgoRERGpnXo7UiEiIiI1oqRCREREYpG5pGLlypVO/vboeuiR5CMUtUc9qvQIRe1Rjyo9yspcUrF7d9pqh0gjU3uUNFF7lFrLXFIhIiIi6aSkQkRERGLRUusA4lC4F/zA3mE627J1H3sREZG0qPT3NPNJRS7n3NK3k8vXb2L4UI7WaU2sXb2clT3zlViIiIiEFMfvaeZPf2zbc2B0BQAMH8px+fpNbNtzoMaRiYiIZEccv6eZTyoG9g6ProCC4UM5du0brlFEIiIi2RPH72nmk4rOtlZap439GK3Tmpg3u7VGEYlUx4JF3ZhZIo8Fi7pr/fFEpMri+D3N/DUVi+fOZO3q5UedA1o8d2atQxNJ1I7tT3LB1XcnsuzrLjkzkeWKSHrF8Xua+aSiqclY2TOfF136KnbtG2bebPX+EBERiSqO39PMJxWQXxEnd8zi5I5ZtQ5FREQksyr9Pa3qNRVm1mxmvzSzm4LXx5nZbWb2SPB3TjXjERERkfhU+0LNy4CHil5fAdzu7kuA24PXkeVyztbB/dzz2G62Du4nl5vwficiIiISiPM3tGqnP8xsIfDHwN8BlweDzwfOCp5/HbgL+FCU5ar4lYiIyNTE/RtazSMVnwb+J1DcCbbT3fsBgr/zoi5Uxa9ERESmJu7f0KokFWZ2HrDL3X8xxfnXmNlGM9s4ODg4ZpyKX0m1TdQeRapN7VEqEfdvaLWOVLwCeJOZbQO+A5xtZt8EBsysCyD4u6vUzO6+zt173b23o6NjzDgVv5Jqm6g9ilSb2qNUIu7f0KokFe7+N+6+0N0XA+8A7nD3dwE3Au8JJnsPcEPUZReKdRRWiopfiYiIhBP3b2it61T8PbDezC4GngDeHnUBKn4lIiIyNXH/hlY9qXD3u8j38sDd9wDnVLpMFb8SERGZmjh/Q2t9pEKk6kZGcvT1D9E/NExX+wx6utpoacn8vfVEREblcs4TvzvAwN7nOHBwhBOPm8lJxyd/FF9JhTSUkZEcGx54iis3bBntk33VqqWsWrZAiYWI1IVczrnj4QEeGdjPZ25/pKo1nLQXlYbS1z80mlBAvuvUlRu20Nc/VOPIRETisW3PATZvHxpNKKB6NZyUVEhD6R8q3Sd755DqmohIfRjYO0zOqUkNJyUV0lC62meU7JM9v111TUSkPnS2tdJs1KSGk5IKaSg9XW1ctWrpmD7ZV61aSk9Xe+zvtWBRN2aW2ENEpJTFc2fy4oXtXHbOkqrXcNKFmtJQWlqaWLVsAUvmzWLn0DDz21vp6WpP5CLNHduf5IKr7459uQXXXXJmYssWkexqajLOfmEnf9AxixXdc3jm4Ajd6v0hkoyWliaWLZrDskW1jkREJBlNTcbi42ex+Pjq1m/S6Q8RERGJhY5USMNR8SsRyZpcztm25wADe4fpbGule84xPPH0M6Ov03J7CiUV0lBU/EpEsiaXc27p28nl6zcxfCjHiXNn8MGzl4zZj1WjsFUY2otKQ1HxKxHJmm17DowmFADnnbbgqP1YNQpbhaGkQhqKil+JSNYM7B273zKrTWGrMJRUSENR8SsRyZrOttaS+63xr5MubBWGkgppKNUsfiUiEofFc2eydvXy0f3WDx546qj9WDUKW4WhCzWloVSz+JWISByamoyVPfN50aWvYte+YebNzvf+WNE9Z/S1en+I1IiKX4lI1jQ1GSd3zOLkjiPFrMa/TgP9eyYiIiKxUFIhIiIisVBSISIiIrFQUiEiIiKxUFIhIiIisVBSISIiIrFQUiEiIiKxUFIhIiIisVDxK2k4IyM5+vqH6B8apqt9Bj1dbaqoKSJVlcs52/YcYGDvMJ1t+QqZTzz9zOjrtFTIjEpJhTSUkZEcGx54avS2wYV7f6xatkCJhYhURS7n3NK3c/R25ifOncEHz14yZr+0dvVyVvbMz1xiob2oNJS+/qHRDRfytwu+csMW+vqHahyZiDSKbXsOjCYUAOedtuCo/dLl6zexbc+BWoY5JUoqpKH0Dw2PbrgFw4dy7BwarlFEItJoBvaO3Q+ZUXK/tGtf9vZLSiqkoXS1zxi9XXBB67Qm5re31igiEWk0nW2tJfdD41/Pm529/ZKSCmkoPV1tXLVq6egGXLimoqervcaRiUijWDx3JmtXLx/dD/3ggaeO2i+tXb2cxXNn1jLMKdGFmtJQWlqaWLVsAUvmzWLn0DDz21vp6WrXRZoiUjVNTcbKnvm86NJXsWvfMPNm53t/rOieM/pavT9EMqKlpYlli+awbFGtIxGRRtXUZJzcMYuTO2aNDhv/Oouq8u+ZmbWa2c/N7AEz6zOzjwfDP2ZmT5nZpuDxhmrEIyIiIvGr1pGK54Cz3X2/mU0Dfmpm/xmM+9/u/qlKFj6+iEhWDxuJiEg26XcorypJhbs7sD94OS14eBzLHl9EJMtFQ0REJHv0O3RE1a5OM7NmM9sE7AJuc/d7g1EfMLPNZnaNmc2JutzxRUSyXDRERESyR79DR1QtqXD3w+6+HFgInGFmS4EvAs8HlgP9wD+XmtfM1pjZRjPbODg4OGbc+CIikN2iIZINE7VHkWpTe6w9/Q4dUfV+dO7+e+AuYKW7DwTJRg74MnBGmXnWuXuvu/d2dHSMGVeuiEgWi4ZINkzUHkWqTe2x9vQ7dES1en90mNmxwfMZwGuBX5tZV9Fkbwa2RF32+CIiWS4aIiIi2aPfoSOq1fujC/i6mTWTT2TWu/tNZvYNM1tO/qLNbcAlURdcqohIo151KyIi1affoSOq1ftjM3B6ieHvjmP5pYqIiIiIVIt+h/LqoqKm+geLiEi16DenvMwnFeofLCIi1aLfnIll/i5K6h8sIiLVot+ciWU+qVD/YBERqRb95kws80mF+geLiEi16DdnYplPKtQ/WEREqkW/ORPL/IWa6h8sIiLVot+ciWU+qQD1DxYRkerRb055mT/9ISIiIukQOakws04z+6qZ/Wfw+lQzuzj+0MLL5Zytg/u557HdbB3cTy7ntQxHREQyRL8h8ZnK6Y+vAf8KfDh4/RvgOuCrMcUUiQqRiIjIVOk3JF5TOf1xvLuvB3IA7j4CHI41qghUiERERKZKvyHxmkpSccDM5pK/syhm9jJgKNaoIlAhEhERmSr9hsRrKqc/LgduBJ5vZv8FdABvizWqCAqFSIobhQqRiIhIGPoNiVfkIxXufj/wR8CZwCVAT3Br85pQIRIREZkq/YbEK/KRCjN7y7hBLzCzIeBBd98VT1jhqRCJiIhMlX5D4jWV0x8XAy8H7gxenwX8jHxy8Ql3/0ZMsYWmQiQiIjJV+g2Jz1SSihxwirsPQL5uBfBF4KXAj4GqJxUjIzn6+ofoHxqmq30GPV1ttLSorpeISCPL5Zxtew4wsHeYzrbSRyDCTCPhTSWpWFxIKAK7gBe4++/M7FBMcYU2MpJjwwNPceWGLaN9jK9atZRVyxYosRARaVBh6k+oRkX8pvKr+xMzu8nM3mNm7wFuAH5sZjOB38caXQh9/UOjCQXkuwJduWELff016+Uqkn1NLZhZIo8Fi7oTDX3Bou7EYm+Z3prZ9dJowtSfUI2K+E3lSMX7gbcArwxe/xzocvcDwGviCiys/qHSfYx3Dg2zbFG1oxGpE7kRLrj67kQWfd0lZyay3IId259MNPasrpdGM1H9icK1E2GmkWim0qXUgceAQ8CbgXOAh2KOK7Su9hmjXYEKWqc1Mb9dfYxFRBpVof5EsfH1J8JMI9GETirM7AVm9hEzewj4PPAkYO7+Gnf/fGIRTqKnq42rVi0d08f4qlVL6elqr1VIIiJSY2HqT6hGRfyinP74NfAT4I3u/iiAmf1lIlFF0NLSxKplC1gybxY7h4aZ395KT1e7LtIUEWlgYepPqEZF/KIkFW8F3gHcaWa3AN8BUrHmW1qaWLZojq6hEBGRUWHqT6hGRbxCJxXu/n3g+0Evj1XAXwKdZvZF4PvufmsyIYrES/3SReqXtu/aitz7I+jl8S3gW2Z2HPB24ApASYWknvqli9Qvbd+1V9GFB+7+O3e/2t3PjisgkSSpX7pI/dL2XXu6mlEaykT90kUk27R9156SCmko6pcuUr+0fdeekgppKOqXLlK/tH3X3lTKdItklvqli9Qvbd+1p6RCGo76pYvUL23ftVWV0x9m1mpmPzezB8ysz8w+Hgw/zsxuM7NHgr9zqhGPiIiIxK9a11Q8B5zt7suA5cBKM3sZ+foWt7v7EuD24LVIonI5Z+vgfu55bDdbB/eTy3mtQxKRCmibTo+qnP4I7my6P3g5LXg4cD5wVjD868BdwIeqEZM0JhXHEakv2qbTpWq9P8ys2cw2AbuA29z9XqDT3fsBgr/zqhWPNCYVxxGpL9qm06VqSYW7H3b35cBC4AwzWxp2XjNbY2YbzWzj4OBgYjFK/YujOI7ao6RJo7dHFbxKl6rXqXD335M/zbESGDCzLoDg764y86xz91537+3o6KhWqFKH4iiOo/ZYoaYWzCyxR6Np9PaoglfpUpVrKsysAzjk7r83sxnAa4F/AG4E3gP8ffD3hmrEI42rUBxn/PlXFcepotwIF1x9d2KLv+6SMxNbtqSPtul0qVadii7g62bWTP7oyHp3v8nM7gHWm9nFwBPk73gqkhgVxxGpL9qm06VavT82A6eXGL4HOKcaMYgUqDiOSH3RNp0eqqgpDWdkJEdf/xD9Q8N0tc+gp6uNlhbdBkckLXI5Z9ueAwzsHaaz7ciRh3LDJT2UVEhDGRnJseGBp7hyw5bR869XrVrKqmULlFiIpEC5uhPnntLJrQ8NqB5FymkvKg2lr39oNKGAfNezKzdsoa9/qMaRiQiUrzvR1z+kehQZoKRCGkr/UOk+7TuH1KddJA3K1Z0ot+2qHkW6KKmQhtLVPqNkn/b57erTLpIG5epOlNt2VY8iXZRUSEPp6WrjqlVLR3dOhWsqerraaxyZiMCRuhPF2+ja1cvp6WorOVz1KNJFF2pKQ2lpaWLVsgUsmTeLnUPDzG9vpaerXRdpiqTERHUnVI8i/ZRUSMNpaWli2aI5LFtU60hEpJRydSdUjyL96iKpOHjwMJt3DLFz7zBdba28+IR2pk9vrnVYIiISUqEGxdCzBzmcg937n1MdmQzKfFJx8OBhNmzewUduOFJ34BPnL2XVaScosRARyYBCbYprfvoYb13Rzcdv6lMdmYzK/Le0ecfQaEIB+S5GH7lhC5t3qO6AiEgWFGpTXHTmyaMJBaiOTBZlPqnYWaZP88Be9V0WEcmCQm2KZ58bUR2ZjMt8UtFVpk9zZ5v6LouIZEGhNsUxz2tRHZmMy3xS8eIT2vnE+WPrDnzi/KWcdoLqDoiIZEGhNsXX797KR8/rUR2ZDMv8hZrTpzez6rQTOPn4maN3rjtNvT9ERDJjtAbF/NnsffYg37z4pUHvD9WRyZrMJxWQTyx6Fx9X6zBERGSKCjUoJNuU/omIiEgs6uJIRaFoSuH0h0q3ykRGRnL09Q/RPzSs4joiCSjeJ3e1t3I4B7v2Hb1/1r67/mQ+qSgUTbl8/abRYilrVy9nZc98NU45yshIjg0PPMWVG7aouI5IAor3yXOOmc5FLz+Rz9z+yFH7Z0D77jqU+b1ooWhKcbGUy9dvYtueAzWOTNKor39oNKEAFdcRiVvxPvktKxaOJhQwdv+sfXd9ynxSMVCm+NWufSqWIkfrHyrdXlRcRyQexftkM8run7Xvrk+ZTyo6yxS/mjdbxVLkaF3tM1RcRyRB4/fJ5fbP2nfXp8wnFYWiKcXFUtauXs7iuTNrHJmkUU9XG1etWqriOiIJKd4nX/+L7Vx2zpKS+2ftu+tT5i/UHC2acumr2LVvmHmzdQWxlNfS0sSqZQtYMm8WO4eGma/iOiKxGr9Pnt/Wyrmnzmdw/9H7Z+2760/mkwo4UjRFhVMkjJaWJpYtmsOyRbWORKQ+ldonP3/e0ftn7bvrT10kFSJRqE6FSDJK1Z0AVIuigSipkIaiOhUiyShVM+jz7zydgyOuWhQNRHtRaSiqUyGSjFJ1JzZvH1ItigajpEIaiupUiCSjVN2JnJevUyH1SUmFNBTVqRBJRqm6E81Wvk6F1CclFdJQVKdCJBml6k68eGG7alE0GF2oKQ1FdSpEklGuZhCgWhQNREmFNBzVqRBJRrm6E6pF0Tiq8u+ZmS0yszvN7CEz6zOzy4LhHzOzp8xsU/B4QzXiERERkfhV60jFCPBX7n6/mc0GfmFmtwXj/re7f6qShZcquKLDa1KOil9Jo4qyryyetqu9lcM52LVP+1iZWFWSCnfvB/qD5/vM7CFgQRzLLlVwRcVVpBwVv5JGFWVfWTztnGOmc9HLT+Qztz+ifaxMqup7UTNbDJwO3BsM+oCZbTaza8xsTtTllSq4ouIqUo6KX0mjirKvLJ72LSsWjiYUk80nUtWkwsxmAdcDf+Hue4EvAs8HlpM/kvHPZeZbY2YbzWzj4ODgmHGlCq6ouIqUE0fxq4nao0i1hW2PUfaVxdOaqYCVhFe1pMLMppFPKL7l7t8DcPcBdz/s7jngy8AZpeZ193Xu3uvuvR0dHWPGlSq4ouIqUk4cxa8mao8i1Ra2PUbZV46fVvtYCatavT8M+CrwkLuvLRreVTTZm4EtUZddquCKiqtIOSp+JY0qyr6yeNrrf7Gdy85Zon2shFKt3h+vAN4NPGhmm4JhfwtcaGbLAQe2AZdEXXC5giu6gEhKUfEraVRR9pXjp53f1sq5p85ncL/2sTKxavX++ClQqgXeHMfyyxVcESlFxa+kUUXZV5aa9vnztI+VidVFRU3VHRARKa1cbQrV95EkZD6pUN0BEZHSytWmOPeUTm59aED1fSR2mf/VVd0BEZHSytWm6OsfUn0fSUTmk4o46g6IiNSjcrUpyu03VXtCKpX5pCKOugMiIvWoXG2KcvtN1Z6QSmU+qVDdARGR0srVpujpalN9H0lE5i/UVN0BEZHSJqpNofo+koTMJxWgugMiIuWUq02h+j6SBP07LyIiIrFQUiEiIiKxMHevdQyRmNkg8HiZ0ccDu6sYzkQUS2lZiWW3u6+cbAGTtMdKY0gLxRiPSmKsVnuMKq3rPY1x1VNMZdtj5pKKiZjZRnfvrXUcoFjKUSzpi2EyijEeWYgxqrR+pjTG1Sgx6fSHiIiIxEJJhYiIiMSi3pKKdbUOoIhiKU2xjJWGGCajGOORhRijSutnSmNcDRFTXV1TISIiIrVTb0cqREREpEaUVIiIiEgslFSIiIhILDKXVKxcudIBPfRI+hGK2qMeVXqEovaoR5UeZWUuqdi9O20FyaSRqT1Kmqg9Sq1lLqkQERGRdFJSISIiIrFoqXUAccjlnG17DjCwd5jOtlYWz51JU5PVOiwRkUlp/yX1JPNJRS7n3NK3k8vXb2L4UI7WaU2sXb2clT3ztWGKSKpp/yX1JvOnP7btOTC6QQIMH8px+fpNbNtzoMaRiYhMTPsvqTeZTyoG9g6PbpAFw4dy7No3XKOIRESOyOWcrYP7ueex3Wwd3E8ud6RHXj3tvxYs6sbMEnksWNRd648nIWX+9Me82a20Tmsas2G2TmuiY1ZrDaMSEZn89EZnW+n917zZ2dt/7dj+JBdcfXciy77ukjMTWa7EL/NHKpqb4LJzltA6Lf9RWqc1cdk5S2jO/CcTkayb7PTG4rkzWbt6+Zj919rVy1k8d2bNYhapROaPVPQPDXPtPY9z8StPxgzc4dp7Huf07mNZfPysWocnIg1sotMbJ3fMoqnJWNkznxdd+ip27Rtm3mz1/pBsq1pSYWbXAOcBu9x9aTDsOOA6YDGwDVjt7k9HWW5nWytPP3OQL9z56OiwrB4+FJH6Eub0RlOTcXLHLE7u0D9Bkn3VPEnwNWDluGFXALe7+xLg9uB1JDp8KCJppf2TNJqqHalw9x+b2eJxg88Hzgqefx24C/hQlOXq8KGIpJX2T9Joan1NRae79wO4e7+ZzSs1kZmtAdYAdHcf3bVIhw+lmiZrjyLFkt4/qT1KmmSij4S7r3P3Xnfv7ejoqHU40uDUHiVN1B4lTWqdVAyYWRdA8HdXjeMRERGRKap1UnEj8J7g+XuAG2oYi4iIiFSgml1Kv03+oszjzWw78FHg74H1ZnYx8ATw9mrFI41Ld4WUOKk9iRxRzd4fF5YZdU61YhDRXSElTmpPImPV+vSHSFX9dnfpssm/3a27Qkp0ak8iYympkIby+O8OlCyb/MTv9CMg0ak9iYylpEIayszpLaPVDQtapzVxzPRal2yRLFJ7EhlLSYU0lM6255W8q21n2/NqHJlkkdqTyFhKp6WhdB83kyWds1jz6pPJOTQZLOmcRfdxuheDRKf2JDKWkgppKE1Nxtkv7OTk42fpXgxSMbUnkbGUVEjD0b1iJE5qTyJHKKmQhqNiRRKF2otIeEoqpKGoWJFEofYiEo16f0hD2bandLGibXtUV0COpvYiEo2SCmkoA3uHSxYr2rVvuEYRSZqpvYhEo6RCGkpnW2vJYkXzZrfWKCJJM7UXkWiUVEhD6Z5zDFetWjqmWNFVq5bSPeeYGkcmaaT2IhKNLtSUhvLE08/wuTse4eJXnowZuMPn7niEFd1z1CVQjqL2IhKNkgppKAN7h3l8z7N84c5HxwzftW9YPxJyFLUXkWh0+kMais6RSxRqLyLRKKmQhrJ47kzWrl4+5hz52tXLWTxX92qQo6m9iESj0x/SUJqajJU983nRpa/SvRpkUmovItEoqZCGo3s1SBRqLyLhRU4qzOxEYIm7/x8zmwG0uPu++EMTSYbu5dA49F2LVFekpMLM/hxYAxwHPB9YCHwJOCf+0ETip3s5NA591yLVF/VCzfcDrwD2Arj7I8C8SgIws780sz4z22Jm3zYzXVYtidG9HBqHvmuR6ouaVDzn7gcLL8ysBfCpvrmZLQAuBXrdfSnQDLxjqssTmYzu5dA49F2LVF/UpOJHZva3wAwzex3wXeAHFcbQEiyvBTgG2FHh8kTKUt2BxqHvWqT6oiYVVwCDwIPAJcDNwJVTfXN3fwr4FPAE0A8MufutU12eyGR0L4fGoe9apPqi9v6YAVzj7l8GMLPmYNgzU3lzM5sDnA+cBPwe+K6ZvcvdvzluujXkLxClu7t7Km8lAsRzLwe1x2xolPt2qD1KmkRNKm4HXgvsD17PAG4Fzpzi+78W+K27DwKY2feCZY1JKtx9HbAOoLe3d8rXcIjEcS8HtcdsaJT7dqg9SppEPf3R6u6FhILgeSXHEp8AXmZmx5iZke+a+lAFyxOZkM6zNw591yLVFzWpOGBmKwovzOwPgWen+ubufi/w78D95K/TaCLIuEWSoHs5NA591yLVF/X0x1+Qv+6h0EOjC7igkgDc/aPARytZhkhYupdD49B3LVJ9kZIKd7/PzF4EvBAw4NfufiiRyEQSons5NA591yLVNZUbir0EWBzMe7qZ4e7XxhqViIiIZE7Ue398g/w9PzYBh4PBDiipEBERaXBRj1T0Aqe6u7otiYiIyBhRe39sAeYnEYiIiIhkW9QjFccDvzKznwPPFQa6+5tijUpEREQyJ2pS8bEkgqhULuds23OAgb3DdLap25hIo9C2L5IuUbuU/sjMTgSWuPv/MbNjyN+uvGZyOeeWvp1cvn4Tw4dyowVuVvbM185FpI5p2xdJn0jXVJjZn5OvgHl1MGgBsCHmmCLZtufA6E4FYPhQjsvXb2LbngO1DEtEEqZtXyR9ol6o+X7gFcBeAHd/BJgXd1BRDOwdHt2pFAwfyrFr33CNIhKRatC2L5I+UZOK59z9YOGFmbWQr1NRM7ppkEj9yuWcrYP7ueex3Wwd3E8ud2R3o21fJH2iJhU/MrO/BWaY2euA7wI/iD+s8LrnHMNVq5aOuWnQVauW0j2nkpunikitFa6ZeMNnf8KFX76XN3z2J9zSt3M0sdANw0TSJ2rvjyuAi8nfUfQS4GbgK3EHFcUTTz/D5+54hItfeTJm4A6fu+MRVnTPUb1/kQwrd83Eiy59FSd3zNINw0RSKGrvjxzw5eCRCgN7h3l8z7N84c5HxwzftW9YSYVIhk10zURh29YNw0TSJVRSYWbr3X21mT1IiWso3P202CMLqXBetXjno/OqItmnbVske8JeU3FZ8Pc84I0lHjWj86oi9Unbtkj2hDpS4e79ZtYMfNXdX5twTJHovKpIfdK2LZI9oa+pcPfDZvaMmbW7+1CSQUWl86oi9Unbtki2RO39MQw8aGa3AaNl69z90lijEhERkcyJmlT8R/AQERERGSNql9Kvm9kMoNvdH04oJhEREcmgqDcUeyOwCbgleL3czG5MIC4RERHJmKhluj8GnAH8HsDdNwEnVRqEmR1rZv9uZr82s4fM7OVR5p/o/gAikk7abkXqT9RrKkbcfchsTJeuOPYEnwFucfe3mdl0IPSNOwr3ByiU8y30ZV/ZM19dz0RSStutSH2KeqRii5m9E2g2syVm9jng7koCMLM24NXAVwHc/aC7/z7s/OXuD7Btz4FJ5hSRJE10JELbbfUtWNSNmSX2kOpL8jtdsKh7SjFFPVLxQeDDwHPAt4EfAp+c0jsfcTIwCPyrmS0DfgFc5u6jexczWwOsAejuHvtBy90fYGCv7v0hyZioPUreZEciwtzXQ8IJ2x53bH+SC66u6H/ACV13yZmJLVtKS/I7ner3GelIhbs/4+4fBs4BXuPuH3b34Sm98xEtwArgi+5+Ovn6F1eMe9917t7r7r0dHR1jZj5mestoGd+C1mlNHDO9ucKwREqbqD1K3mRHIgr39Sim+3pMjdqjpEnU3h8vCW4qtpl8EawHzOwPK4xhO7Dd3e8NXv87+SQjlIOHD3Pp2UvG3B/g0rOXcOhwbpI5RSQpEx2JAN3XQ6ReRT398VXg/3X3nwCY2SuBfwWmfJdSd99pZk+a2QuD2hfnAL8KO//cmc/juo1PcPErT8YM3OG6jU+wcun8qYYkIhWa7A6juq+HSH2KmlTsKyQUAO7+UzPbF0McHwS+FfT82Aq8N+yMi+fO5EMrTznq3K3+4xGpncKRiIm2S93XQ6T+RE0qfm5mV5O/SNOBC4C7zGwFgLvfP5UggnoXvVOZV//xiKSPtkuRxhQ1qVge/P3ouOFnkk8yzq40oKnQfzwi6aPtUqTxRL33x2uSCkRERESyLWrvj2+YWXvR6xPN7Pb4w4pG5X5F0kfbpUjjiXr646fAvWZ2ObAA+B/AX8UeVQQq9yuSPtouRRpT1OJXVwN/BtwAfAJ4tbv/IInAwlK5X5H00XYp0piinv54N3ANcBHwNeDmoLR2zUxWZEdkPB2Wj8dE61HbpUhjinr6463AK919F/BtM/s+8HWO9AqpusmK7IgU02H5eEy2HrVdijSmqKc/VgUJReH1z4EzYo8qApX7lSh0WD4ek61HbZcijSnUkQozW+/uq4Pn/+DuHyoafRNwbhLBhaEiOxKF7o4ZXi7nbNtzgIG9w3S2jd2uJluP2i5FGlPY0x9Lip6/DihOKmp+WzwV2ZGw5s0ufVi+Y5YOyxeL4/SGtkuJTVMLZskkpM3TnsfhQ88lsmyAExYu4qknn0hs+WkTNqmY6Eo2XeUmmdHcBJeds4TP3P7I6I/lZecsoTnSicD6V+70xosufRUnd8wKdW8PkdjkRrjg6rsTWfR1l5yZ2LILy28kYZOKY8zsdPLXYMwInlvwmJFUcCJx6x8a5tp7Hh9zV9tr73mc07uPZfHx+o+6QKc3RGQqwiYV/cDa4PnOoueF1yKZ0NnWytPPHOQLdz46Oky9Eo6m0xsiMhWhDvq6+2smeiQdpEhc1CshHK0nEZmKSHUqzOztwC3uvs/MrgRWAJ90918mEp1IzJqajHNP6eS6NS+jf2iYrvZWerraG+6w/UQ9O0C9qkRkaqIWv/r/3P27ZvZK4L8BnwK+BLw09shEEpDLObc+NNDQxa/CFgDT6Q0RiSrqNe+Hg79/DHzR3W8ApscbkkhyVPxK60BEkhM1qXjKzK4GVpO/78fzprAMkZrRPSm0DkQkOVETgtXAD4GV7v574Djytz8XyYRCr4Zijdb7Q+tARJISOqkwsybg5+7+PXd/BMDd+9391sSiE4mZejVoHYhIckJfqOnuOTN7wMy63b1xao5KXVHvD60DEUlO1N4fXUCfmf0cGL2qy93fVEkQZtYMbASecvfzKlmWyETU+0PrQESSEzWp+HgiUcBlwENA21RmHhnJ0dc/FPzXNYOerjZaWnT9qBxtsnta1IuJtolGWQciUn2Rkgp3/1HcAZjZQvJdVP8OuDzq/CMjOTY88BRXbtgy+l/XVauWsmrZAiUWcpRGuPX5ZNtEI6wDEamNSL+6ZvYyM7vPzPab2UEzO2xmeyuM4dPA/wRyk0xXUt+OodGdJ+R3jldu2ELfjqEKw5J61Ag9HybbJhphHYhIbUT9V/7zwIXAI+TvTvpnwbApMbPzgF3u/otJpltjZhvNbOPg4OCYcU8NPVvyv66nhp6dalhSx+Lo+TBRe0yDybYJ9f6oL2lvj9JYol5Tgbs/ambN7n4Y+Fczq+RG9K8A3mRmbwBagTYz+6a7v2vce64D1gH09vZ68bjjZz2v5N0Uj5/5vArCknoVR8+HidpjtUx0747Jtgnd16O+pKE9ygSaWjBrnG0ralLxjJlNBzaZ2T+SvyX6lP+9cfe/Af4GwMzOAv56fEIxmc625/HRN/bw8R/0jZ4//ugbe+hsV1IhR6uHng+T3bsjzDah+3qIVEluhAuuruR/7/Kuu+TMRJZbiahJxbvJnzL5APCXwCLgrXEHFcWiOTM54dgDfOptyzhwcISZ01uYPaOZRXN0KFeOVg89Hyb7DNomRKRWovb+eNzMZgBd7h5r91J3vwu4K+p8TU3Gq/5gHtv2HNChXJlUPfR8mOwzaJsQkVqJlFSY2RvJ3+58OnCSmS0HPlFp8atK6VCuhFXo+TD+eoMs9XwI8xm0TYhILUTt/fEx4Azg9wDuvglYHGdAIkmqh54P9fAZRKQ+Rb2mYsTdhxrpSlapL/XQ86EePoOI1KdQSYWZ3Qy8H9hiZu8Ems1sCXApkMxlrSIJqYdTA/XwGUSk/oQ9/fE14IfANmAp8Bzwb8AQ+ft2iGTGyEiOB558mlu29PPAk79nZGRKxVwTlcs5Wwf3c89ju9k6uJ9cziONFxGphVBHKtx9vZn9B/ARYCXwDaCwF3s/sDaZ8ETilYV7xUxWh2Ky8SIitRJlL3qI/O3OnwfMKnrMTiAukUT09Ze5L0Z/eu4VU64OxbY9B0KNFxGplbDXVKwkfzTiRmCFuz+TaFQRTVSyWKRY/1DpGg87h4ZZtqhGQY0zWR2Keqi1ISL1KWzvjw8Db3f3viSDmQodCpYoutpnlKzxML89PXUqJqtDMW926fEds9LzGUSkMYU6/eHur0pjQgE6FCzRnNI5m0+cv3RMjYdPnL+UUzrbahzZEd1zjuGqVWNjvGrVUrrnHANAcxNcds6SMeMvO2cJzem4JEREGljku5SmjQ4FSxTbh55l/X2P849vW8azB0eYMb2Fa+/eSu+Jc1LTXp54+hk+d8cjXPzKkzEDd/jcHY+wojsfY//QMNfe8/iY8dfe8zindx/L4uPT8RlEpDFlPqmoh7LLUj0De4fZ+PgQGx//5ZjhaUpCB/YO8/ieZ/nCnY+OGV6IsbOtlaefOThmvNq8iKRB5g+YqmSxRFFIQoul7Qd5shjV5kUkrTJ/pEIliyWKwg/y+At70/SDPFmMavMiklaZTypAJYslvCz8IIeJUW1eRNKoLpIKkSiy8IOchRhFRMbL/DUVIiIikg5KKkRERCQWSipEREQkFnVxTYXu/SFZozYrIvUo80mF7v0hWaM2KyL1KvOnP3TvD8katVkRqVc1TyrMbJGZ3WlmD5lZn5ldFmX+ie79IVJKLudsHdzPPY/tZuvgfnI5r+r7q82KSL1Kw+mPEeCv3P1+M5sN/MLMbnP3X4WZWff+kCjScOpBty4XkXpV8yMV7t7v7vcHz/cBDwELws6v+yBIFGk49aBbl4tIvUrDkYpRZrYYOB24d9zwNcAagO7u7jHzZKHssqTHRKcewlavnKg9hqFbl0ucKm2PInFKTVJhZrOA64G/cPe9xePcfR2wDqC3t/eoE+AqaSxhxXG6bLL2GCYG3bpc4lJpexSJUyoOuJrZNPIJxbfc/Xu1jkfqVxpOl6UhBhGRJNT8SIWZGfBV4CF3X1vreKS+peF0WRpiEBFJQs2TCuAVwLuBB81sUzDsb9395tqFJPUsDafL0hCDiEjcap5UuPtPgYr+RVPJY0kTtUcRaVQ1TyoqlYa6AyIFao8i0shScaFmJdJQd0CkQO1RRBpZ5pMKlTyWqJIs0x22Pda6VLiISBIyf/pDZboliqRPT4RpjzpFIiL1KvNHKtTnX6JI+vREmPaoUyQiUq8yf6Siqck495ROrlvzMvqHhulqb6Wnq13/8UlJcZTpnkiY9ph0DCIitZL5pCKXc259aECHkiWUpE+XhWmPOmUnIvUq86c/dChZokj6dFmY9qhTdiJSrzJ/pEKHkiWKpEtkh2mPKtMtIvUq80mFDiVLVEmWyA7bHlWmW0TqUeZPf+hQsqSJ2qOINLLMH6nQoWRJE7VHEWlkmU8qQIeSJV3UHkWkUWX+9IeIiIikg5IKERERiYW5Z+tGRmY2CDxeZvTxwO4qhjMRxVJaVmLZ7e4rJ1vAJO2x0hjSQjHGo5IYq9Ueo0rrek9jXPUUU9n2mLmkYiJmttHde2sdByiWchRL+mKYjGKMRxZijCqtnymNcTVKTDr9ISIiIrFQUiEiIiKxqLekYl2tAyiiWEpTLGOlIYbJKMZ4ZCHGqNL6mdIYV0PEVFfXVIiIiEjt1NuRChEREamRzCQVZrbSzB42s0fN7IoS483MPhuM32xmK8LOm0AsfxLEsNnM7jazZUXjtpnZg2a2ycw2ViGWs8xsKHi/TWb2kbDzJhDL/yiKY4uZHTaz44Jxsa0XM7vGzHaZ2ZYy46vWVoqWu8jM7jSzh8ysz8wuC4Z/zMyeKlovb4jrPacQY6uZ/dzMHghi/Hgw/Dgzu83MHgn+zklhjKlZj0WxNpvZL83spuB1atZjFOXa7rhpyu5nEoxrwn3GRNt5gjG9sGgdbDKzvWb2F+OmSXxdldoHhm1/Fe8D3T31D6AZeAw4GZgOPACcOm6aNwD/CRjwMuDesPMmEMuZwJzg+esLsQSvtwHHV3G9nAXcNJV5445l3PRvBO5IaL28GlgBbCkzviptZdx7dgErguezgd8ApwIfA/46jveIIUYDZgXPpwH3BuvnH4ErguFXAP+QwhhTsx6LYr0c+LfC9pem9Rjxc5Rsu+OmKbmfSTiuCfcZ5bbzKsbXDOwETqz2uiq1DwzT/uLYB2blSMUZwKPuvtXdDwLfAc4fN835wLWe9zPgWDPrCjlvrLG4+93u/nTw8mfAwgrer6JYEpo3juVdCHy7gvcry91/DPxugkmq1VaKY+p39/uD5/uAh4AFcSw7LsH62B+8nBY8nPw6+How/OvAqupHlzdBjKliZguBPwa+UjQ4Nesxiiy03TLKbefVcg7wmLtXsxgZUHYfGKb9VbwPzEpSsQB4suj1do5u1OWmCTNv3LEUu5h8tlzgwK1m9gszW1NBHFFieXlwuPg/zawn4rxxx4KZHQOsBK4vGhzneplMtdpKSWa2GDid/H/ZAB8IDs9eU+tD4sEh+03ALuA2d78X6HT3fsj/wADzahhiuRghResR+DTwP4Fc0bBUrcepKNF2i5XazyRpsn1GVbbnCbyD8v84VXtdQbj2V/E6y0pSUeq+0eP/Oyk3TZh5444lP6HZa8gnFR8qGvwKd19B/rTI+83s1QnHcj/5w2/LgM8BGyLMG3csBW8E/svdizPpONfLZKrVVo5+Y7NZ5JOpv3D3vcAXgecDy4F+4J/jfL+o3P2wuy8nf3TtDDNbWst4SikTY2rWo5mdB+xy91/UKoYklGi7xcrtZ5I02T4j8e25HDObDrwJ+G6J0bVYV2FVvM6yklRsBxYVvV4I7Ag5TZh5444FMzuN/KHP8919T2G4u+8I/u4Cvk/+cFNisbj73sLhYne/GZhmZseH/RxxxlLkqAw+5vUymWq1lTHMbBr5nfK33P17AO4+EPxI5oAvk+znDs3dfw/cRf6I0kDhsHHwd1ftIjuiOMaUrcdXAG8ys23kDx+fbWbfJKXrMYxSbbfYBPuZxITYZyS6PU/i9cD97j4wfkQt1lUgTPureJ1lJam4D1hiZicFGeA7gBvHTXMjcFFwxe/LgKHgEE+YeWONxcy6ge8B73b33xQNn2lmswvPgXOBkj0UYoxlvplZ8PwM8t/5njDzxh1LEEM78EfADUXD4l4vk6lWWxkVfAdfBR5y97VFw4vP8b6ZZD/3hMysw8yODZ7PAF4L/Jr8OnhPMNl7KPruqq1cjGlaj+7+N+6+0N0Xk29Dd7j7u0jReoyiXNsdN025/UxSMYXZZ5Tbzquh7DVj1V5XRcK0v8r3gVGu6qzlg/yVvL8hf2Xqh4Nh7wPeFzw34AvB+AeB3onmTTiWrwBPA5uCx8Zg+Mnkr6Z9AOirUiwfCN7rAfIXjZ5Zq/USvP5T4Dvj5ot1vZDfmPuBQ+Qz74tr1VaKlvtK8ocRNxe1izcA3whi2Ex+4+2q5nY1LsbTgF8GsWwBPhIMnwvcDjwS/D0uhTGmZj2Oi/csjvT+SM16jPgZyrXdUPuZhGIquc8Iu50nHNsx5JOE9qJhVV1XZfaBJdsfcAJwc9G8Fe0DVVFTREREYpGV0x8iIiKSckoqREREJBZKKkRERCQWSipEREQkFkoqREREJBZKKuqEmX3Y8ncQ3Gz5O9+9tNYxSeMyszebmZvZi2odi9ROUCPip2b2+qJhq83slgTe6y7L312zcPfPf4/7Pca93wlJv0cWqUtpHTCzlwNrgbPc/bmgOtt0DyrOiVSbma0nf3fL2939YzUOR2ooKKP+XfL3DGkmX+dipbs/NoVlNbv74TLj7iJ/p9qjboMeNzNrcfeRpN8ni3Skoj50Abvd/TkAd9/t7jvM7A/N7EeWv+HOD82sy8zag2z+hQBm9m0z+/OaRi91xfL3iHgF+YI77wiGNZnZvwRH024ys5vN7G3BuKPaaQ3Dl5i5+xbgB+TvgfRR4JvAh83sPjP7pZmdD/mblZnZT8zs/uBxZjD8LDO708z+DXgwqKb5H5a/IdcWM7tgovc3sxvM7KLg+SVm9q3g+V1m9mkzuztYzhnB8JmWvynd+Pj+1My+a2Y/IH8js8VmtiUY12xm/xTMs9nMLimK/S4z+3cz+7WZfauomuZLgvd+wMx+bmazyy0nU6pRYUyPZB/ALPLZ/2+AfyFfBnsacDfQEUxzAXBN8Px1wD3kd/i31Dp+PerrAbwL+Grw/G5gBfA24Gby/8jMJ19x9m0TtVM96ucBzAQeJl/Z8n8B7wqGHxvst2aSr0TZGgxfwpFKxGcBB4CTgtdvBb5ctOz24O9dwXtsCh7/FAzvBB4FXhW813FF0385eP5qYEvw/P8vE9+fkq9OWZh/cdE8a4Arg+fPAzYCJwWxD5G/h0ZTsN99JTAd2Aq8JJinDWgpt5xaf39RHi1I5rn7fjP7Q/IbzWuA64CrgKXAbUFi3Ey+bCvufpuZvZ18CdtlNQla6tmF5G/9Dfkbal1IPnn4rudv+LXTzO4Mxr+QMu1U6oe7HzCz64D9wGrgjWb218HoVqCb/I2rPm9my4HDwAuKFvFzd/9t8PxB4FNm9g/kS6D/pGi6P/Fxpz/cfcDMPgLcCbzZx94d+dvBND82szbL31fmXPI3hBsfH8Bt4+YvOBc4rXD0DWgnnxgdDGLfDmBmm8gnI0NAv7vfF7z/3mB8ueUUPnvqKamoE54/z3gXcJeZPQi8H+hz95ePn9bMmoBTgGeB48hn3yIVM7O5wNnAUjNz8kmCk7+LZMlZKNNOpe7kgocBb3X3h4tHmtnHgAHy/+g0AcNFow8Unrj7b4J/ot4A/C8zu9XdPzHJe7+Y/P04Thg3fPxFhT5BfC8tjmMcAz7o7j8cN89ZwHNFgw6T/921Eu9ddjlZomsq6oCZvdDMlhQNWg48BHQEF3FiZtPMrCcY/5fB+AuBayx/W2OROLwNuNbdT3T3xe6+iPx/WbuBtwbXVnSSPywM+cPV5dqp1KcfAh8surbg9GB4O/n/3nPAu8knpEcxsxOAZ9z9m8CnyJ9eKyu4VuL15C8U/WszO6lo9AXBNK8kfxfToQnim+wz/ffCvtTMXmD5u6eW82vgBDN7STD9bDNrmcJyUkdHKurDLOBzwaG7EfLnD9cA64DPWv524y3Ap83sEPBnwBnuvs/MfgxcSf4CKpFKXQj8/bhh15M/Mrad/J1FfwPcS34nfjA41DumnZK/i6PUp0+S/443Bz/c24DzyF8Pdn1wavZOyh8VeDHwT2aWI38Xzv9eNO5bZvZs8Hw38MfAl4H3ev7i9b8i/4/U2cE0T5vZ3eSvafh/JolvIl8hf1rj/mCeQWBVuYmDdn8B+f32DPJHjV8bdTlppC6lIlIVZjYruP5nLvBz4BXuvrPWcUljsip2QW0kOlIhItVyU3A0bTrwSSUUIvVHRypEREQkFrpQU0RERGKhpEJERERioaRCREREYqGkQkRERGKhpEJERERioaRCREREYvF/AaoShkE2+MeYAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 540x540 with 12 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "sns.pairplot(X)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 第三种方式：使用VIF（必须使用）"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 136,
   "metadata": {},
   "outputs": [],
   "source": [
    "#pip install statsmodels\n",
    "#conda install -c conda-forge statsmodels"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 226,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Import library for VIF (方差扩大因子)\n",
    "from statsmodels.stats.outliers_influence import variance_inflation_factor\n",
    "\n",
    "def vif(X):\n",
    "\n",
    "    # Calculating VIF\n",
    "    vif = pd.DataFrame()\n",
    "    vif[\"variables\"] = X.columns\n",
    "    vif[\"VIF\"] = [variance_inflation_factor(X.values, i) for i in range(X.shape[1])]\n",
    "\n",
    "    return(vif)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 260,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>YearsExperience</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>24</td>\n",
       "      <td>1.1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>25</td>\n",
       "      <td>1.3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0</td>\n",
       "      <td>24</td>\n",
       "      <td>1.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>25</td>\n",
       "      <td>2.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0</td>\n",
       "      <td>25</td>\n",
       "      <td>2.2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>1</td>\n",
       "      <td>26</td>\n",
       "      <td>2.9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>0</td>\n",
       "      <td>27</td>\n",
       "      <td>3.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>1</td>\n",
       "      <td>26</td>\n",
       "      <td>3.2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>1</td>\n",
       "      <td>27</td>\n",
       "      <td>3.2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>1</td>\n",
       "      <td>27</td>\n",
       "      <td>3.7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>0</td>\n",
       "      <td>28</td>\n",
       "      <td>3.9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>0</td>\n",
       "      <td>28</td>\n",
       "      <td>4.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>0</td>\n",
       "      <td>28</td>\n",
       "      <td>4.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>1</td>\n",
       "      <td>29</td>\n",
       "      <td>4.1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>1</td>\n",
       "      <td>29</td>\n",
       "      <td>4.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>1</td>\n",
       "      <td>30</td>\n",
       "      <td>4.9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>1</td>\n",
       "      <td>31</td>\n",
       "      <td>5.1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>0</td>\n",
       "      <td>32</td>\n",
       "      <td>5.3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>0</td>\n",
       "      <td>32</td>\n",
       "      <td>5.9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>0</td>\n",
       "      <td>33</td>\n",
       "      <td>6.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>0</td>\n",
       "      <td>34</td>\n",
       "      <td>6.8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>0</td>\n",
       "      <td>35</td>\n",
       "      <td>7.1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>1</td>\n",
       "      <td>36</td>\n",
       "      <td>7.9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>1</td>\n",
       "      <td>36</td>\n",
       "      <td>8.2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>1</td>\n",
       "      <td>38</td>\n",
       "      <td>8.7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>1</td>\n",
       "      <td>38</td>\n",
       "      <td>9.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>1</td>\n",
       "      <td>39</td>\n",
       "      <td>9.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>1</td>\n",
       "      <td>39</td>\n",
       "      <td>9.6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>0</td>\n",
       "      <td>40</td>\n",
       "      <td>10.3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>0</td>\n",
       "      <td>41</td>\n",
       "      <td>10.5</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    Sex  Age  YearsExperience\n",
       "0     0   24              1.1\n",
       "1     1   25              1.3\n",
       "2     0   24              1.5\n",
       "3     1   25              2.0\n",
       "4     0   25              2.2\n",
       "5     1   26              2.9\n",
       "6     0   27              3.0\n",
       "7     1   26              3.2\n",
       "8     1   27              3.2\n",
       "9     1   27              3.7\n",
       "10    0   28              3.9\n",
       "11    0   28              4.0\n",
       "12    0   28              4.0\n",
       "13    1   29              4.1\n",
       "14    1   29              4.5\n",
       "15    1   30              4.9\n",
       "16    1   31              5.1\n",
       "17    0   32              5.3\n",
       "18    0   32              5.9\n",
       "19    0   33              6.0\n",
       "20    0   34              6.8\n",
       "21    0   35              7.1\n",
       "22    1   36              7.9\n",
       "23    1   36              8.2\n",
       "24    1   38              8.7\n",
       "25    1   38              9.0\n",
       "26    1   39              9.5\n",
       "27    1   39              9.6\n",
       "28    0   40             10.3\n",
       "29    0   41             10.5"
      ]
     },
     "execution_count": 260,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 262,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2.1349098210011803"
      ]
     },
     "execution_count": 262,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "variance_inflation_factor(X.values, 0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 263,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "12.138592602889707"
      ]
     },
     "execution_count": 263,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "variance_inflation_factor(X.values, 1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 264,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "10.367631468765076"
      ]
     },
     "execution_count": 264,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "variance_inflation_factor(X.values, 2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 265,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Index(['Sex', 'Age', 'YearsExperience'], dtype='object')"
      ]
     },
     "execution_count": 265,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X.columns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 266,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 0. , 24. ,  1.1],\n",
       "       [ 1. , 25. ,  1.3],\n",
       "       [ 0. , 24. ,  1.5],\n",
       "       [ 1. , 25. ,  2. ],\n",
       "       [ 0. , 25. ,  2.2],\n",
       "       [ 1. , 26. ,  2.9],\n",
       "       [ 0. , 27. ,  3. ],\n",
       "       [ 1. , 26. ,  3.2],\n",
       "       [ 1. , 27. ,  3.2],\n",
       "       [ 1. , 27. ,  3.7],\n",
       "       [ 0. , 28. ,  3.9],\n",
       "       [ 0. , 28. ,  4. ],\n",
       "       [ 0. , 28. ,  4. ],\n",
       "       [ 1. , 29. ,  4.1],\n",
       "       [ 1. , 29. ,  4.5],\n",
       "       [ 1. , 30. ,  4.9],\n",
       "       [ 1. , 31. ,  5.1],\n",
       "       [ 0. , 32. ,  5.3],\n",
       "       [ 0. , 32. ,  5.9],\n",
       "       [ 0. , 33. ,  6. ],\n",
       "       [ 0. , 34. ,  6.8],\n",
       "       [ 0. , 35. ,  7.1],\n",
       "       [ 1. , 36. ,  7.9],\n",
       "       [ 1. , 36. ,  8.2],\n",
       "       [ 1. , 38. ,  8.7],\n",
       "       [ 1. , 38. ,  9. ],\n",
       "       [ 1. , 39. ,  9.5],\n",
       "       [ 1. , 39. ,  9.6],\n",
       "       [ 0. , 40. , 10.3],\n",
       "       [ 0. , 41. , 10.5]])"
      ]
     },
     "execution_count": 266,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X.values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 267,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3"
      ]
     },
     "execution_count": 267,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X.shape[1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 268,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "range(0, 3)"
      ]
     },
     "execution_count": 268,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "range(X.shape[1])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 269,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>variables</th>\n",
       "      <th>VIF</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Sex</td>\n",
       "      <td>2.134910</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Age</td>\n",
       "      <td>12.138593</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>YearsExperience</td>\n",
       "      <td>10.367631</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         variables        VIF\n",
       "0              Sex   2.134910\n",
       "1              Age  12.138593\n",
       "2  YearsExperience  10.367631"
      ]
     },
     "execution_count": 269,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "vif(X)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "VIF starts at 1 and has no upper limit\n",
    "\n",
    "VIF = 1, no correlation between the independent variable and the other variables\n",
    "\n",
    "VIF exceeding 5 or 10 indicates high multicollinearity between this independent variable and the others\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 270,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>YearsExperience</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>24</td>\n",
       "      <td>1.1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>25</td>\n",
       "      <td>1.3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0</td>\n",
       "      <td>24</td>\n",
       "      <td>1.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>25</td>\n",
       "      <td>2.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0</td>\n",
       "      <td>25</td>\n",
       "      <td>2.2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>1</td>\n",
       "      <td>26</td>\n",
       "      <td>2.9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>0</td>\n",
       "      <td>27</td>\n",
       "      <td>3.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>1</td>\n",
       "      <td>26</td>\n",
       "      <td>3.2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>1</td>\n",
       "      <td>27</td>\n",
       "      <td>3.2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>1</td>\n",
       "      <td>27</td>\n",
       "      <td>3.7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>0</td>\n",
       "      <td>28</td>\n",
       "      <td>3.9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>0</td>\n",
       "      <td>28</td>\n",
       "      <td>4.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>0</td>\n",
       "      <td>28</td>\n",
       "      <td>4.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>1</td>\n",
       "      <td>29</td>\n",
       "      <td>4.1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>1</td>\n",
       "      <td>29</td>\n",
       "      <td>4.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>1</td>\n",
       "      <td>30</td>\n",
       "      <td>4.9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>1</td>\n",
       "      <td>31</td>\n",
       "      <td>5.1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>0</td>\n",
       "      <td>32</td>\n",
       "      <td>5.3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>0</td>\n",
       "      <td>32</td>\n",
       "      <td>5.9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>0</td>\n",
       "      <td>33</td>\n",
       "      <td>6.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>0</td>\n",
       "      <td>34</td>\n",
       "      <td>6.8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>0</td>\n",
       "      <td>35</td>\n",
       "      <td>7.1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>1</td>\n",
       "      <td>36</td>\n",
       "      <td>7.9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>1</td>\n",
       "      <td>36</td>\n",
       "      <td>8.2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>1</td>\n",
       "      <td>38</td>\n",
       "      <td>8.7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>1</td>\n",
       "      <td>38</td>\n",
       "      <td>9.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>1</td>\n",
       "      <td>39</td>\n",
       "      <td>9.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>1</td>\n",
       "      <td>39</td>\n",
       "      <td>9.6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>0</td>\n",
       "      <td>40</td>\n",
       "      <td>10.3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>0</td>\n",
       "      <td>41</td>\n",
       "      <td>10.5</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    Sex  Age  YearsExperience\n",
       "0     0   24              1.1\n",
       "1     1   25              1.3\n",
       "2     0   24              1.5\n",
       "3     1   25              2.0\n",
       "4     0   25              2.2\n",
       "5     1   26              2.9\n",
       "6     0   27              3.0\n",
       "7     1   26              3.2\n",
       "8     1   27              3.2\n",
       "9     1   27              3.7\n",
       "10    0   28              3.9\n",
       "11    0   28              4.0\n",
       "12    0   28              4.0\n",
       "13    1   29              4.1\n",
       "14    1   29              4.5\n",
       "15    1   30              4.9\n",
       "16    1   31              5.1\n",
       "17    0   32              5.3\n",
       "18    0   32              5.9\n",
       "19    0   33              6.0\n",
       "20    0   34              6.8\n",
       "21    0   35              7.1\n",
       "22    1   36              7.9\n",
       "23    1   36              8.2\n",
       "24    1   38              8.7\n",
       "25    1   38              9.0\n",
       "26    1   39              9.5\n",
       "27    1   39              9.6\n",
       "28    0   40             10.3\n",
       "29    0   41             10.5"
      ]
     },
     "execution_count": 270,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 271,
   "metadata": {},
   "outputs": [],
   "source": [
    "X=X.drop('Age',axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 272,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Sex</th>\n",
       "      <th>YearsExperience</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>1.1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>1.3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0</td>\n",
       "      <td>1.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>2.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0</td>\n",
       "      <td>2.2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>1</td>\n",
       "      <td>2.9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>0</td>\n",
       "      <td>3.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>1</td>\n",
       "      <td>3.2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>1</td>\n",
       "      <td>3.2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>1</td>\n",
       "      <td>3.7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>0</td>\n",
       "      <td>3.9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>0</td>\n",
       "      <td>4.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>0</td>\n",
       "      <td>4.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>1</td>\n",
       "      <td>4.1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>1</td>\n",
       "      <td>4.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>1</td>\n",
       "      <td>4.9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>1</td>\n",
       "      <td>5.1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>0</td>\n",
       "      <td>5.3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>0</td>\n",
       "      <td>5.9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>0</td>\n",
       "      <td>6.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>0</td>\n",
       "      <td>6.8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>0</td>\n",
       "      <td>7.1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>1</td>\n",
       "      <td>7.9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>1</td>\n",
       "      <td>8.2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>1</td>\n",
       "      <td>8.7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>1</td>\n",
       "      <td>9.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>1</td>\n",
       "      <td>9.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>1</td>\n",
       "      <td>9.6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>0</td>\n",
       "      <td>10.3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>0</td>\n",
       "      <td>10.5</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    Sex  YearsExperience\n",
       "0     0              1.1\n",
       "1     1              1.3\n",
       "2     0              1.5\n",
       "3     1              2.0\n",
       "4     0              2.2\n",
       "5     1              2.9\n",
       "6     0              3.0\n",
       "7     1              3.2\n",
       "8     1              3.2\n",
       "9     1              3.7\n",
       "10    0              3.9\n",
       "11    0              4.0\n",
       "12    0              4.0\n",
       "13    1              4.1\n",
       "14    1              4.5\n",
       "15    1              4.9\n",
       "16    1              5.1\n",
       "17    0              5.3\n",
       "18    0              5.9\n",
       "19    0              6.0\n",
       "20    0              6.8\n",
       "21    0              7.1\n",
       "22    1              7.9\n",
       "23    1              8.2\n",
       "24    1              8.7\n",
       "25    1              9.0\n",
       "26    1              9.5\n",
       "27    1              9.6\n",
       "28    0             10.3\n",
       "29    0             10.5"
      ]
     },
     "execution_count": 272,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 273,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>variables</th>\n",
       "      <th>VIF</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Sex</td>\n",
       "      <td>1.804751</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>YearsExperience</td>\n",
       "      <td>1.804751</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         variables       VIF\n",
       "0              Sex  1.804751\n",
       "1  YearsExperience  1.804751"
      ]
     },
     "execution_count": 273,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "vif(X)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 第四种方式：使用statsmodels.regression.linear_model.OLS Ordinary Least Squares最小二乘法(又称最小平方法)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 274,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>YearsExperience</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>24</td>\n",
       "      <td>1.1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>25</td>\n",
       "      <td>1.3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0</td>\n",
       "      <td>24</td>\n",
       "      <td>1.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>25</td>\n",
       "      <td>2.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0</td>\n",
       "      <td>25</td>\n",
       "      <td>2.2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>1</td>\n",
       "      <td>26</td>\n",
       "      <td>2.9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>0</td>\n",
       "      <td>27</td>\n",
       "      <td>3.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>1</td>\n",
       "      <td>26</td>\n",
       "      <td>3.2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>1</td>\n",
       "      <td>27</td>\n",
       "      <td>3.2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>1</td>\n",
       "      <td>27</td>\n",
       "      <td>3.7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>0</td>\n",
       "      <td>28</td>\n",
       "      <td>3.9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>0</td>\n",
       "      <td>28</td>\n",
       "      <td>4.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>0</td>\n",
       "      <td>28</td>\n",
       "      <td>4.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>1</td>\n",
       "      <td>29</td>\n",
       "      <td>4.1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>1</td>\n",
       "      <td>29</td>\n",
       "      <td>4.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>1</td>\n",
       "      <td>30</td>\n",
       "      <td>4.9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>1</td>\n",
       "      <td>31</td>\n",
       "      <td>5.1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>0</td>\n",
       "      <td>32</td>\n",
       "      <td>5.3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>0</td>\n",
       "      <td>32</td>\n",
       "      <td>5.9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>0</td>\n",
       "      <td>33</td>\n",
       "      <td>6.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>0</td>\n",
       "      <td>34</td>\n",
       "      <td>6.8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>0</td>\n",
       "      <td>35</td>\n",
       "      <td>7.1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>1</td>\n",
       "      <td>36</td>\n",
       "      <td>7.9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>1</td>\n",
       "      <td>36</td>\n",
       "      <td>8.2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>1</td>\n",
       "      <td>38</td>\n",
       "      <td>8.7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>1</td>\n",
       "      <td>38</td>\n",
       "      <td>9.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>1</td>\n",
       "      <td>39</td>\n",
       "      <td>9.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>1</td>\n",
       "      <td>39</td>\n",
       "      <td>9.6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>0</td>\n",
       "      <td>40</td>\n",
       "      <td>10.3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>0</td>\n",
       "      <td>41</td>\n",
       "      <td>10.5</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    Sex  Age  YearsExperience\n",
       "0     0   24              1.1\n",
       "1     1   25              1.3\n",
       "2     0   24              1.5\n",
       "3     1   25              2.0\n",
       "4     0   25              2.2\n",
       "5     1   26              2.9\n",
       "6     0   27              3.0\n",
       "7     1   26              3.2\n",
       "8     1   27              3.2\n",
       "9     1   27              3.7\n",
       "10    0   28              3.9\n",
       "11    0   28              4.0\n",
       "12    0   28              4.0\n",
       "13    1   29              4.1\n",
       "14    1   29              4.5\n",
       "15    1   30              4.9\n",
       "16    1   31              5.1\n",
       "17    0   32              5.3\n",
       "18    0   32              5.9\n",
       "19    0   33              6.0\n",
       "20    0   34              6.8\n",
       "21    0   35              7.1\n",
       "22    1   36              7.9\n",
       "23    1   36              8.2\n",
       "24    1   38              8.7\n",
       "25    1   38              9.0\n",
       "26    1   39              9.5\n",
       "27    1   39              9.6\n",
       "28    0   40             10.3\n",
       "29    0   41             10.5"
      ]
     },
     "execution_count": 274,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 275,
   "metadata": {},
   "outputs": [],
   "source": [
    "X2=sm.add_constant(X1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 276,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>const</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>YearsExperience</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1.0</td>\n",
       "      <td>0</td>\n",
       "      <td>24</td>\n",
       "      <td>1.1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1</td>\n",
       "      <td>25</td>\n",
       "      <td>1.3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1.0</td>\n",
       "      <td>0</td>\n",
       "      <td>24</td>\n",
       "      <td>1.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1</td>\n",
       "      <td>25</td>\n",
       "      <td>2.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1.0</td>\n",
       "      <td>0</td>\n",
       "      <td>25</td>\n",
       "      <td>2.2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1</td>\n",
       "      <td>26</td>\n",
       "      <td>2.9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>1.0</td>\n",
       "      <td>0</td>\n",
       "      <td>27</td>\n",
       "      <td>3.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1</td>\n",
       "      <td>26</td>\n",
       "      <td>3.2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1</td>\n",
       "      <td>27</td>\n",
       "      <td>3.2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1</td>\n",
       "      <td>27</td>\n",
       "      <td>3.7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>1.0</td>\n",
       "      <td>0</td>\n",
       "      <td>28</td>\n",
       "      <td>3.9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>1.0</td>\n",
       "      <td>0</td>\n",
       "      <td>28</td>\n",
       "      <td>4.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>1.0</td>\n",
       "      <td>0</td>\n",
       "      <td>28</td>\n",
       "      <td>4.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1</td>\n",
       "      <td>29</td>\n",
       "      <td>4.1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1</td>\n",
       "      <td>29</td>\n",
       "      <td>4.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1</td>\n",
       "      <td>30</td>\n",
       "      <td>4.9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1</td>\n",
       "      <td>31</td>\n",
       "      <td>5.1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>1.0</td>\n",
       "      <td>0</td>\n",
       "      <td>32</td>\n",
       "      <td>5.3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>1.0</td>\n",
       "      <td>0</td>\n",
       "      <td>32</td>\n",
       "      <td>5.9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>1.0</td>\n",
       "      <td>0</td>\n",
       "      <td>33</td>\n",
       "      <td>6.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>1.0</td>\n",
       "      <td>0</td>\n",
       "      <td>34</td>\n",
       "      <td>6.8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>1.0</td>\n",
       "      <td>0</td>\n",
       "      <td>35</td>\n",
       "      <td>7.1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1</td>\n",
       "      <td>36</td>\n",
       "      <td>7.9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1</td>\n",
       "      <td>36</td>\n",
       "      <td>8.2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1</td>\n",
       "      <td>38</td>\n",
       "      <td>8.7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1</td>\n",
       "      <td>38</td>\n",
       "      <td>9.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1</td>\n",
       "      <td>39</td>\n",
       "      <td>9.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1</td>\n",
       "      <td>39</td>\n",
       "      <td>9.6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>1.0</td>\n",
       "      <td>0</td>\n",
       "      <td>40</td>\n",
       "      <td>10.3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>1.0</td>\n",
       "      <td>0</td>\n",
       "      <td>41</td>\n",
       "      <td>10.5</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    const  Sex  Age  YearsExperience\n",
       "0     1.0    0   24              1.1\n",
       "1     1.0    1   25              1.3\n",
       "2     1.0    0   24              1.5\n",
       "3     1.0    1   25              2.0\n",
       "4     1.0    0   25              2.2\n",
       "5     1.0    1   26              2.9\n",
       "6     1.0    0   27              3.0\n",
       "7     1.0    1   26              3.2\n",
       "8     1.0    1   27              3.2\n",
       "9     1.0    1   27              3.7\n",
       "10    1.0    0   28              3.9\n",
       "11    1.0    0   28              4.0\n",
       "12    1.0    0   28              4.0\n",
       "13    1.0    1   29              4.1\n",
       "14    1.0    1   29              4.5\n",
       "15    1.0    1   30              4.9\n",
       "16    1.0    1   31              5.1\n",
       "17    1.0    0   32              5.3\n",
       "18    1.0    0   32              5.9\n",
       "19    1.0    0   33              6.0\n",
       "20    1.0    0   34              6.8\n",
       "21    1.0    0   35              7.1\n",
       "22    1.0    1   36              7.9\n",
       "23    1.0    1   36              8.2\n",
       "24    1.0    1   38              8.7\n",
       "25    1.0    1   38              9.0\n",
       "26    1.0    1   39              9.5\n",
       "27    1.0    1   39              9.6\n",
       "28    1.0    0   40             10.3\n",
       "29    1.0    0   41             10.5"
      ]
     },
     "execution_count": 276,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 277,
   "metadata": {},
   "outputs": [],
   "source": [
    "model=sm.OLS(y,X2).fit() #Ordinary Least Squares最小二乘法(又称最小平方法)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 278,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"simpletable\">\n",
       "<caption>OLS Regression Results</caption>\n",
       "<tr>\n",
       "  <th>Dep. Variable:</th>         <td>Salary</td>      <th>  R-squared:         </th> <td>   0.966</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Model:</th>                   <td>OLS</td>       <th>  Adj. R-squared:    </th> <td>   0.962</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Method:</th>             <td>Least Squares</td>  <th>  F-statistic:       </th> <td>   245.4</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Date:</th>             <td>Tue, 17 Nov 2020</td> <th>  Prob (F-statistic):</th> <td>3.49e-19</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Time:</th>                 <td>21:35:09</td>     <th>  Log-Likelihood:    </th> <td> -297.95</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>No. Observations:</th>      <td>    30</td>      <th>  AIC:               </th> <td>   603.9</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Df Residuals:</th>          <td>    26</td>      <th>  BIC:               </th> <td>   609.5</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Df Model:</th>              <td>     3</td>      <th>                     </th>     <td> </td>   \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Covariance Type:</th>      <td>nonrobust</td>    <th>                     </th>     <td> </td>   \n",
       "</tr>\n",
       "</table>\n",
       "<table class=\"simpletable\">\n",
       "<tr>\n",
       "         <td></td>            <th>coef</th>     <th>std err</th>      <th>t</th>      <th>P>|t|</th>  <th>[0.025</th>    <th>0.975]</th>  \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>const</th>           <td>-6.833e+04</td> <td> 3.72e+04</td> <td>   -1.838</td> <td> 0.078</td> <td>-1.45e+05</td> <td> 8094.672</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Sex</th>             <td> -311.4461</td> <td> 1984.473</td> <td>   -0.157</td> <td> 0.877</td> <td>-4390.590</td> <td> 3767.697</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Age</th>             <td> 4463.4461</td> <td> 1750.353</td> <td>    2.550</td> <td> 0.017</td> <td>  865.544</td> <td> 8061.348</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>YearsExperience</th> <td> 1098.8909</td> <td> 3298.652</td> <td>    0.333</td> <td> 0.742</td> <td>-5681.586</td> <td> 7879.368</td>\n",
       "</tr>\n",
       "</table>\n",
       "<table class=\"simpletable\">\n",
       "<tr>\n",
       "  <th>Omnibus:</th>       <td> 1.673</td> <th>  Durbin-Watson:     </th> <td>   1.559</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Prob(Omnibus):</th> <td> 0.433</td> <th>  Jarque-Bera (JB):  </th> <td>   1.020</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Skew:</th>          <td> 0.451</td> <th>  Prob(JB):          </th> <td>   0.601</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Kurtosis:</th>      <td> 3.058</td> <th>  Cond. No.          </th> <td>1.23e+03</td>\n",
       "</tr>\n",
       "</table><br/><br/>Notes:<br/>[1] Standard Errors assume that the covariance matrix of the errors is correctly specified.<br/>[2] The condition number is large, 1.23e+03. This might indicate that there are<br/>strong multicollinearity or other numerical problems."
      ],
      "text/plain": [
       "<class 'statsmodels.iolib.summary.Summary'>\n",
       "\"\"\"\n",
       "                            OLS Regression Results                            \n",
       "==============================================================================\n",
       "Dep. Variable:                 Salary   R-squared:                       0.966\n",
       "Model:                            OLS   Adj. R-squared:                  0.962\n",
       "Method:                 Least Squares   F-statistic:                     245.4\n",
       "Date:                Tue, 17 Nov 2020   Prob (F-statistic):           3.49e-19\n",
       "Time:                        21:35:09   Log-Likelihood:                -297.95\n",
       "No. Observations:                  30   AIC:                             603.9\n",
       "Df Residuals:                      26   BIC:                             609.5\n",
       "Df Model:                           3                                         \n",
       "Covariance Type:            nonrobust                                         \n",
       "===================================================================================\n",
       "                      coef    std err          t      P>|t|      [0.025      0.975]\n",
       "-----------------------------------------------------------------------------------\n",
       "const           -6.833e+04   3.72e+04     -1.838      0.078   -1.45e+05    8094.672\n",
       "Sex              -311.4461   1984.473     -0.157      0.877   -4390.590    3767.697\n",
       "Age              4463.4461   1750.353      2.550      0.017     865.544    8061.348\n",
       "YearsExperience  1098.8909   3298.652      0.333      0.742   -5681.586    7879.368\n",
       "==============================================================================\n",
       "Omnibus:                        1.673   Durbin-Watson:                   1.559\n",
       "Prob(Omnibus):                  0.433   Jarque-Bera (JB):                1.020\n",
       "Skew:                           0.451   Prob(JB):                        0.601\n",
       "Kurtosis:                       3.058   Cond. No.                     1.23e+03\n",
       "==============================================================================\n",
       "\n",
       "Notes:\n",
       "[1] Standard Errors assume that the covariance matrix of the errors is correctly specified.\n",
       "[2] The condition number is large, 1.23e+03. This might indicate that there are\n",
       "strong multicollinearity or other numerical problems.\n",
       "\"\"\""
      ]
     },
     "execution_count": 278,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.summary()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 280,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.datasets import load_boston"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 281,
   "metadata": {},
   "outputs": [],
   "source": [
    "boston=load_boston()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 282,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'data': array([[6.3200e-03, 1.8000e+01, 2.3100e+00, ..., 1.5300e+01, 3.9690e+02,\n",
       "         4.9800e+00],\n",
       "        [2.7310e-02, 0.0000e+00, 7.0700e+00, ..., 1.7800e+01, 3.9690e+02,\n",
       "         9.1400e+00],\n",
       "        [2.7290e-02, 0.0000e+00, 7.0700e+00, ..., 1.7800e+01, 3.9283e+02,\n",
       "         4.0300e+00],\n",
       "        ...,\n",
       "        [6.0760e-02, 0.0000e+00, 1.1930e+01, ..., 2.1000e+01, 3.9690e+02,\n",
       "         5.6400e+00],\n",
       "        [1.0959e-01, 0.0000e+00, 1.1930e+01, ..., 2.1000e+01, 3.9345e+02,\n",
       "         6.4800e+00],\n",
       "        [4.7410e-02, 0.0000e+00, 1.1930e+01, ..., 2.1000e+01, 3.9690e+02,\n",
       "         7.8800e+00]]),\n",
       " 'target': array([24. , 21.6, 34.7, 33.4, 36.2, 28.7, 22.9, 27.1, 16.5, 18.9, 15. ,\n",
       "        18.9, 21.7, 20.4, 18.2, 19.9, 23.1, 17.5, 20.2, 18.2, 13.6, 19.6,\n",
       "        15.2, 14.5, 15.6, 13.9, 16.6, 14.8, 18.4, 21. , 12.7, 14.5, 13.2,\n",
       "        13.1, 13.5, 18.9, 20. , 21. , 24.7, 30.8, 34.9, 26.6, 25.3, 24.7,\n",
       "        21.2, 19.3, 20. , 16.6, 14.4, 19.4, 19.7, 20.5, 25. , 23.4, 18.9,\n",
       "        35.4, 24.7, 31.6, 23.3, 19.6, 18.7, 16. , 22.2, 25. , 33. , 23.5,\n",
       "        19.4, 22. , 17.4, 20.9, 24.2, 21.7, 22.8, 23.4, 24.1, 21.4, 20. ,\n",
       "        20.8, 21.2, 20.3, 28. , 23.9, 24.8, 22.9, 23.9, 26.6, 22.5, 22.2,\n",
       "        23.6, 28.7, 22.6, 22. , 22.9, 25. , 20.6, 28.4, 21.4, 38.7, 43.8,\n",
       "        33.2, 27.5, 26.5, 18.6, 19.3, 20.1, 19.5, 19.5, 20.4, 19.8, 19.4,\n",
       "        21.7, 22.8, 18.8, 18.7, 18.5, 18.3, 21.2, 19.2, 20.4, 19.3, 22. ,\n",
       "        20.3, 20.5, 17.3, 18.8, 21.4, 15.7, 16.2, 18. , 14.3, 19.2, 19.6,\n",
       "        23. , 18.4, 15.6, 18.1, 17.4, 17.1, 13.3, 17.8, 14. , 14.4, 13.4,\n",
       "        15.6, 11.8, 13.8, 15.6, 14.6, 17.8, 15.4, 21.5, 19.6, 15.3, 19.4,\n",
       "        17. , 15.6, 13.1, 41.3, 24.3, 23.3, 27. , 50. , 50. , 50. , 22.7,\n",
       "        25. , 50. , 23.8, 23.8, 22.3, 17.4, 19.1, 23.1, 23.6, 22.6, 29.4,\n",
       "        23.2, 24.6, 29.9, 37.2, 39.8, 36.2, 37.9, 32.5, 26.4, 29.6, 50. ,\n",
       "        32. , 29.8, 34.9, 37. , 30.5, 36.4, 31.1, 29.1, 50. , 33.3, 30.3,\n",
       "        34.6, 34.9, 32.9, 24.1, 42.3, 48.5, 50. , 22.6, 24.4, 22.5, 24.4,\n",
       "        20. , 21.7, 19.3, 22.4, 28.1, 23.7, 25. , 23.3, 28.7, 21.5, 23. ,\n",
       "        26.7, 21.7, 27.5, 30.1, 44.8, 50. , 37.6, 31.6, 46.7, 31.5, 24.3,\n",
       "        31.7, 41.7, 48.3, 29. , 24. , 25.1, 31.5, 23.7, 23.3, 22. , 20.1,\n",
       "        22.2, 23.7, 17.6, 18.5, 24.3, 20.5, 24.5, 26.2, 24.4, 24.8, 29.6,\n",
       "        42.8, 21.9, 20.9, 44. , 50. , 36. , 30.1, 33.8, 43.1, 48.8, 31. ,\n",
       "        36.5, 22.8, 30.7, 50. , 43.5, 20.7, 21.1, 25.2, 24.4, 35.2, 32.4,\n",
       "        32. , 33.2, 33.1, 29.1, 35.1, 45.4, 35.4, 46. , 50. , 32.2, 22. ,\n",
       "        20.1, 23.2, 22.3, 24.8, 28.5, 37.3, 27.9, 23.9, 21.7, 28.6, 27.1,\n",
       "        20.3, 22.5, 29. , 24.8, 22. , 26.4, 33.1, 36.1, 28.4, 33.4, 28.2,\n",
       "        22.8, 20.3, 16.1, 22.1, 19.4, 21.6, 23.8, 16.2, 17.8, 19.8, 23.1,\n",
       "        21. , 23.8, 23.1, 20.4, 18.5, 25. , 24.6, 23. , 22.2, 19.3, 22.6,\n",
       "        19.8, 17.1, 19.4, 22.2, 20.7, 21.1, 19.5, 18.5, 20.6, 19. , 18.7,\n",
       "        32.7, 16.5, 23.9, 31.2, 17.5, 17.2, 23.1, 24.5, 26.6, 22.9, 24.1,\n",
       "        18.6, 30.1, 18.2, 20.6, 17.8, 21.7, 22.7, 22.6, 25. , 19.9, 20.8,\n",
       "        16.8, 21.9, 27.5, 21.9, 23.1, 50. , 50. , 50. , 50. , 50. , 13.8,\n",
       "        13.8, 15. , 13.9, 13.3, 13.1, 10.2, 10.4, 10.9, 11.3, 12.3,  8.8,\n",
       "         7.2, 10.5,  7.4, 10.2, 11.5, 15.1, 23.2,  9.7, 13.8, 12.7, 13.1,\n",
       "        12.5,  8.5,  5. ,  6.3,  5.6,  7.2, 12.1,  8.3,  8.5,  5. , 11.9,\n",
       "        27.9, 17.2, 27.5, 15. , 17.2, 17.9, 16.3,  7. ,  7.2,  7.5, 10.4,\n",
       "         8.8,  8.4, 16.7, 14.2, 20.8, 13.4, 11.7,  8.3, 10.2, 10.9, 11. ,\n",
       "         9.5, 14.5, 14.1, 16.1, 14.3, 11.7, 13.4,  9.6,  8.7,  8.4, 12.8,\n",
       "        10.5, 17.1, 18.4, 15.4, 10.8, 11.8, 14.9, 12.6, 14.1, 13. , 13.4,\n",
       "        15.2, 16.1, 17.8, 14.9, 14.1, 12.7, 13.5, 14.9, 20. , 16.4, 17.7,\n",
       "        19.5, 20.2, 21.4, 19.9, 19. , 19.1, 19.1, 20.1, 19.9, 19.6, 23.2,\n",
       "        29.8, 13.8, 13.3, 16.7, 12. , 14.6, 21.4, 23. , 23.7, 25. , 21.8,\n",
       "        20.6, 21.2, 19.1, 20.6, 15.2,  7. ,  8.1, 13.6, 20.1, 21.8, 24.5,\n",
       "        23.1, 19.7, 18.3, 21.2, 17.5, 16.8, 22.4, 20.6, 23.9, 22. , 11.9]),\n",
       " 'feature_names': array(['CRIM', 'ZN', 'INDUS', 'CHAS', 'NOX', 'RM', 'AGE', 'DIS', 'RAD',\n",
       "        'TAX', 'PTRATIO', 'B', 'LSTAT'], dtype='<U7'),\n",
       " 'DESCR': \".. _boston_dataset:\\n\\nBoston house prices dataset\\n---------------------------\\n\\n**Data Set Characteristics:**  \\n\\n    :Number of Instances: 506 \\n\\n    :Number of Attributes: 13 numeric/categorical predictive. Median Value (attribute 14) is usually the target.\\n\\n    :Attribute Information (in order):\\n        - CRIM     per capita crime rate by town\\n        - ZN       proportion of residential land zoned for lots over 25,000 sq.ft.\\n        - INDUS    proportion of non-retail business acres per town\\n        - CHAS     Charles River dummy variable (= 1 if tract bounds river; 0 otherwise)\\n        - NOX      nitric oxides concentration (parts per 10 million)\\n        - RM       average number of rooms per dwelling\\n        - AGE      proportion of owner-occupied units built prior to 1940\\n        - DIS      weighted distances to five Boston employment centres\\n        - RAD      index of accessibility to radial highways\\n        - TAX      full-value property-tax rate per $10,000\\n        - PTRATIO  pupil-teacher ratio by town\\n        - B        1000(Bk - 0.63)^2 where Bk is the proportion of blacks by town\\n        - LSTAT    % lower status of the population\\n        - MEDV     Median value of owner-occupied homes in $1000's\\n\\n    :Missing Attribute Values: None\\n\\n    :Creator: Harrison, D. and Rubinfeld, D.L.\\n\\nThis is a copy of UCI ML housing dataset.\\nhttps://archive.ics.uci.edu/ml/machine-learning-databases/housing/\\n\\n\\nThis dataset was taken from the StatLib library which is maintained at Carnegie Mellon University.\\n\\nThe Boston house-price data of Harrison, D. and Rubinfeld, D.L. 'Hedonic\\nprices and the demand for clean air', J. Environ. Economics & Management,\\nvol.5, 81-102, 1978.   Used in Belsley, Kuh & Welsch, 'Regression diagnostics\\n...', Wiley, 1980.   N.B. Various transformations are used in the table on\\npages 244-261 of the latter.\\n\\nThe Boston house-price data has been used in many machine learning papers that address regression\\nproblems.   \\n     \\n.. topic:: References\\n\\n   - Belsley, Kuh & Welsch, 'Regression diagnostics: Identifying Influential Data and Sources of Collinearity', Wiley, 1980. 244-261.\\n   - Quinlan,R. (1993). Combining Instance-Based and Model-Based Learning. In Proceedings on the Tenth International Conference of Machine Learning, 236-243, University of Massachusetts, Amherst. Morgan Kaufmann.\\n\",\n",
       " 'filename': 'C:\\\\Users\\\\raymond\\\\anaconda3\\\\lib\\\\site-packages\\\\sklearn\\\\datasets\\\\data\\\\boston_house_prices.csv'}"
      ]
     },
     "execution_count": 282,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "boston"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Bunch is a subclass of the Dict class and supports all the methods as dict does. In addition, it allows you to use the keys as attributes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 283,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "sklearn.utils.Bunch"
      ]
     },
     "execution_count": 283,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "type(boston)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 284,
   "metadata": {},
   "outputs": [],
   "source": [
    "X=pd.DataFrame(boston['data'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 285,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "      <th>3</th>\n",
       "      <th>4</th>\n",
       "      <th>5</th>\n",
       "      <th>6</th>\n",
       "      <th>7</th>\n",
       "      <th>8</th>\n",
       "      <th>9</th>\n",
       "      <th>10</th>\n",
       "      <th>11</th>\n",
       "      <th>12</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.00632</td>\n",
       "      <td>18.0</td>\n",
       "      <td>2.31</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.538</td>\n",
       "      <td>6.575</td>\n",
       "      <td>65.2</td>\n",
       "      <td>4.0900</td>\n",
       "      <td>1.0</td>\n",
       "      <td>296.0</td>\n",
       "      <td>15.3</td>\n",
       "      <td>396.90</td>\n",
       "      <td>4.98</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.02731</td>\n",
       "      <td>0.0</td>\n",
       "      <td>7.07</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.469</td>\n",
       "      <td>6.421</td>\n",
       "      <td>78.9</td>\n",
       "      <td>4.9671</td>\n",
       "      <td>2.0</td>\n",
       "      <td>242.0</td>\n",
       "      <td>17.8</td>\n",
       "      <td>396.90</td>\n",
       "      <td>9.14</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.02729</td>\n",
       "      <td>0.0</td>\n",
       "      <td>7.07</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.469</td>\n",
       "      <td>7.185</td>\n",
       "      <td>61.1</td>\n",
       "      <td>4.9671</td>\n",
       "      <td>2.0</td>\n",
       "      <td>242.0</td>\n",
       "      <td>17.8</td>\n",
       "      <td>392.83</td>\n",
       "      <td>4.03</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0.03237</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.18</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.458</td>\n",
       "      <td>6.998</td>\n",
       "      <td>45.8</td>\n",
       "      <td>6.0622</td>\n",
       "      <td>3.0</td>\n",
       "      <td>222.0</td>\n",
       "      <td>18.7</td>\n",
       "      <td>394.63</td>\n",
       "      <td>2.94</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0.06905</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.18</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.458</td>\n",
       "      <td>7.147</td>\n",
       "      <td>54.2</td>\n",
       "      <td>6.0622</td>\n",
       "      <td>3.0</td>\n",
       "      <td>222.0</td>\n",
       "      <td>18.7</td>\n",
       "      <td>396.90</td>\n",
       "      <td>5.33</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>501</th>\n",
       "      <td>0.06263</td>\n",
       "      <td>0.0</td>\n",
       "      <td>11.93</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.573</td>\n",
       "      <td>6.593</td>\n",
       "      <td>69.1</td>\n",
       "      <td>2.4786</td>\n",
       "      <td>1.0</td>\n",
       "      <td>273.0</td>\n",
       "      <td>21.0</td>\n",
       "      <td>391.99</td>\n",
       "      <td>9.67</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>502</th>\n",
       "      <td>0.04527</td>\n",
       "      <td>0.0</td>\n",
       "      <td>11.93</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.573</td>\n",
       "      <td>6.120</td>\n",
       "      <td>76.7</td>\n",
       "      <td>2.2875</td>\n",
       "      <td>1.0</td>\n",
       "      <td>273.0</td>\n",
       "      <td>21.0</td>\n",
       "      <td>396.90</td>\n",
       "      <td>9.08</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>503</th>\n",
       "      <td>0.06076</td>\n",
       "      <td>0.0</td>\n",
       "      <td>11.93</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.573</td>\n",
       "      <td>6.976</td>\n",
       "      <td>91.0</td>\n",
       "      <td>2.1675</td>\n",
       "      <td>1.0</td>\n",
       "      <td>273.0</td>\n",
       "      <td>21.0</td>\n",
       "      <td>396.90</td>\n",
       "      <td>5.64</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>504</th>\n",
       "      <td>0.10959</td>\n",
       "      <td>0.0</td>\n",
       "      <td>11.93</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.573</td>\n",
       "      <td>6.794</td>\n",
       "      <td>89.3</td>\n",
       "      <td>2.3889</td>\n",
       "      <td>1.0</td>\n",
       "      <td>273.0</td>\n",
       "      <td>21.0</td>\n",
       "      <td>393.45</td>\n",
       "      <td>6.48</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>505</th>\n",
       "      <td>0.04741</td>\n",
       "      <td>0.0</td>\n",
       "      <td>11.93</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.573</td>\n",
       "      <td>6.030</td>\n",
       "      <td>80.8</td>\n",
       "      <td>2.5050</td>\n",
       "      <td>1.0</td>\n",
       "      <td>273.0</td>\n",
       "      <td>21.0</td>\n",
       "      <td>396.90</td>\n",
       "      <td>7.88</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>506 rows × 13 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "          0     1      2    3      4      5     6       7    8      9     10  \\\n",
       "0    0.00632  18.0   2.31  0.0  0.538  6.575  65.2  4.0900  1.0  296.0  15.3   \n",
       "1    0.02731   0.0   7.07  0.0  0.469  6.421  78.9  4.9671  2.0  242.0  17.8   \n",
       "2    0.02729   0.0   7.07  0.0  0.469  7.185  61.1  4.9671  2.0  242.0  17.8   \n",
       "3    0.03237   0.0   2.18  0.0  0.458  6.998  45.8  6.0622  3.0  222.0  18.7   \n",
       "4    0.06905   0.0   2.18  0.0  0.458  7.147  54.2  6.0622  3.0  222.0  18.7   \n",
       "..       ...   ...    ...  ...    ...    ...   ...     ...  ...    ...   ...   \n",
       "501  0.06263   0.0  11.93  0.0  0.573  6.593  69.1  2.4786  1.0  273.0  21.0   \n",
       "502  0.04527   0.0  11.93  0.0  0.573  6.120  76.7  2.2875  1.0  273.0  21.0   \n",
       "503  0.06076   0.0  11.93  0.0  0.573  6.976  91.0  2.1675  1.0  273.0  21.0   \n",
       "504  0.10959   0.0  11.93  0.0  0.573  6.794  89.3  2.3889  1.0  273.0  21.0   \n",
       "505  0.04741   0.0  11.93  0.0  0.573  6.030  80.8  2.5050  1.0  273.0  21.0   \n",
       "\n",
       "         11    12  \n",
       "0    396.90  4.98  \n",
       "1    396.90  9.14  \n",
       "2    392.83  4.03  \n",
       "3    394.63  2.94  \n",
       "4    396.90  5.33  \n",
       "..      ...   ...  \n",
       "501  391.99  9.67  \n",
       "502  396.90  9.08  \n",
       "503  396.90  5.64  \n",
       "504  393.45  6.48  \n",
       "505  396.90  7.88  \n",
       "\n",
       "[506 rows x 13 columns]"
      ]
     },
     "execution_count": 285,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 286,
   "metadata": {},
   "outputs": [],
   "source": [
    "X.columns=boston['feature_names']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 287,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>CRIM</th>\n",
       "      <th>ZN</th>\n",
       "      <th>INDUS</th>\n",
       "      <th>CHAS</th>\n",
       "      <th>NOX</th>\n",
       "      <th>RM</th>\n",
       "      <th>AGE</th>\n",
       "      <th>DIS</th>\n",
       "      <th>RAD</th>\n",
       "      <th>TAX</th>\n",
       "      <th>PTRATIO</th>\n",
       "      <th>B</th>\n",
       "      <th>LSTAT</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.00632</td>\n",
       "      <td>18.0</td>\n",
       "      <td>2.31</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.538</td>\n",
       "      <td>6.575</td>\n",
       "      <td>65.2</td>\n",
       "      <td>4.0900</td>\n",
       "      <td>1.0</td>\n",
       "      <td>296.0</td>\n",
       "      <td>15.3</td>\n",
       "      <td>396.90</td>\n",
       "      <td>4.98</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.02731</td>\n",
       "      <td>0.0</td>\n",
       "      <td>7.07</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.469</td>\n",
       "      <td>6.421</td>\n",
       "      <td>78.9</td>\n",
       "      <td>4.9671</td>\n",
       "      <td>2.0</td>\n",
       "      <td>242.0</td>\n",
       "      <td>17.8</td>\n",
       "      <td>396.90</td>\n",
       "      <td>9.14</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.02729</td>\n",
       "      <td>0.0</td>\n",
       "      <td>7.07</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.469</td>\n",
       "      <td>7.185</td>\n",
       "      <td>61.1</td>\n",
       "      <td>4.9671</td>\n",
       "      <td>2.0</td>\n",
       "      <td>242.0</td>\n",
       "      <td>17.8</td>\n",
       "      <td>392.83</td>\n",
       "      <td>4.03</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0.03237</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.18</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.458</td>\n",
       "      <td>6.998</td>\n",
       "      <td>45.8</td>\n",
       "      <td>6.0622</td>\n",
       "      <td>3.0</td>\n",
       "      <td>222.0</td>\n",
       "      <td>18.7</td>\n",
       "      <td>394.63</td>\n",
       "      <td>2.94</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0.06905</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.18</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.458</td>\n",
       "      <td>7.147</td>\n",
       "      <td>54.2</td>\n",
       "      <td>6.0622</td>\n",
       "      <td>3.0</td>\n",
       "      <td>222.0</td>\n",
       "      <td>18.7</td>\n",
       "      <td>396.90</td>\n",
       "      <td>5.33</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>501</th>\n",
       "      <td>0.06263</td>\n",
       "      <td>0.0</td>\n",
       "      <td>11.93</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.573</td>\n",
       "      <td>6.593</td>\n",
       "      <td>69.1</td>\n",
       "      <td>2.4786</td>\n",
       "      <td>1.0</td>\n",
       "      <td>273.0</td>\n",
       "      <td>21.0</td>\n",
       "      <td>391.99</td>\n",
       "      <td>9.67</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>502</th>\n",
       "      <td>0.04527</td>\n",
       "      <td>0.0</td>\n",
       "      <td>11.93</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.573</td>\n",
       "      <td>6.120</td>\n",
       "      <td>76.7</td>\n",
       "      <td>2.2875</td>\n",
       "      <td>1.0</td>\n",
       "      <td>273.0</td>\n",
       "      <td>21.0</td>\n",
       "      <td>396.90</td>\n",
       "      <td>9.08</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>503</th>\n",
       "      <td>0.06076</td>\n",
       "      <td>0.0</td>\n",
       "      <td>11.93</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.573</td>\n",
       "      <td>6.976</td>\n",
       "      <td>91.0</td>\n",
       "      <td>2.1675</td>\n",
       "      <td>1.0</td>\n",
       "      <td>273.0</td>\n",
       "      <td>21.0</td>\n",
       "      <td>396.90</td>\n",
       "      <td>5.64</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>504</th>\n",
       "      <td>0.10959</td>\n",
       "      <td>0.0</td>\n",
       "      <td>11.93</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.573</td>\n",
       "      <td>6.794</td>\n",
       "      <td>89.3</td>\n",
       "      <td>2.3889</td>\n",
       "      <td>1.0</td>\n",
       "      <td>273.0</td>\n",
       "      <td>21.0</td>\n",
       "      <td>393.45</td>\n",
       "      <td>6.48</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>505</th>\n",
       "      <td>0.04741</td>\n",
       "      <td>0.0</td>\n",
       "      <td>11.93</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.573</td>\n",
       "      <td>6.030</td>\n",
       "      <td>80.8</td>\n",
       "      <td>2.5050</td>\n",
       "      <td>1.0</td>\n",
       "      <td>273.0</td>\n",
       "      <td>21.0</td>\n",
       "      <td>396.90</td>\n",
       "      <td>7.88</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>506 rows × 13 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "        CRIM    ZN  INDUS  CHAS    NOX     RM   AGE     DIS  RAD    TAX  \\\n",
       "0    0.00632  18.0   2.31   0.0  0.538  6.575  65.2  4.0900  1.0  296.0   \n",
       "1    0.02731   0.0   7.07   0.0  0.469  6.421  78.9  4.9671  2.0  242.0   \n",
       "2    0.02729   0.0   7.07   0.0  0.469  7.185  61.1  4.9671  2.0  242.0   \n",
       "3    0.03237   0.0   2.18   0.0  0.458  6.998  45.8  6.0622  3.0  222.0   \n",
       "4    0.06905   0.0   2.18   0.0  0.458  7.147  54.2  6.0622  3.0  222.0   \n",
       "..       ...   ...    ...   ...    ...    ...   ...     ...  ...    ...   \n",
       "501  0.06263   0.0  11.93   0.0  0.573  6.593  69.1  2.4786  1.0  273.0   \n",
       "502  0.04527   0.0  11.93   0.0  0.573  6.120  76.7  2.2875  1.0  273.0   \n",
       "503  0.06076   0.0  11.93   0.0  0.573  6.976  91.0  2.1675  1.0  273.0   \n",
       "504  0.10959   0.0  11.93   0.0  0.573  6.794  89.3  2.3889  1.0  273.0   \n",
       "505  0.04741   0.0  11.93   0.0  0.573  6.030  80.8  2.5050  1.0  273.0   \n",
       "\n",
       "     PTRATIO       B  LSTAT  \n",
       "0       15.3  396.90   4.98  \n",
       "1       17.8  396.90   9.14  \n",
       "2       17.8  392.83   4.03  \n",
       "3       18.7  394.63   2.94  \n",
       "4       18.7  396.90   5.33  \n",
       "..       ...     ...    ...  \n",
       "501     21.0  391.99   9.67  \n",
       "502     21.0  396.90   9.08  \n",
       "503     21.0  396.90   5.64  \n",
       "504     21.0  393.45   6.48  \n",
       "505     21.0  396.90   7.88  \n",
       "\n",
       "[506 rows x 13 columns]"
      ]
     },
     "execution_count": 287,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 288,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>CRIM</th>\n",
       "      <th>ZN</th>\n",
       "      <th>INDUS</th>\n",
       "      <th>CHAS</th>\n",
       "      <th>NOX</th>\n",
       "      <th>RM</th>\n",
       "      <th>AGE</th>\n",
       "      <th>DIS</th>\n",
       "      <th>RAD</th>\n",
       "      <th>TAX</th>\n",
       "      <th>PTRATIO</th>\n",
       "      <th>B</th>\n",
       "      <th>LSTAT</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>CRIM</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>-0.200469</td>\n",
       "      <td>0.406583</td>\n",
       "      <td>-0.055892</td>\n",
       "      <td>0.420972</td>\n",
       "      <td>-0.219247</td>\n",
       "      <td>0.352734</td>\n",
       "      <td>-0.379670</td>\n",
       "      <td>0.625505</td>\n",
       "      <td>0.582764</td>\n",
       "      <td>0.289946</td>\n",
       "      <td>-0.385064</td>\n",
       "      <td>0.455621</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ZN</th>\n",
       "      <td>-0.200469</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>-0.533828</td>\n",
       "      <td>-0.042697</td>\n",
       "      <td>-0.516604</td>\n",
       "      <td>0.311991</td>\n",
       "      <td>-0.569537</td>\n",
       "      <td>0.664408</td>\n",
       "      <td>-0.311948</td>\n",
       "      <td>-0.314563</td>\n",
       "      <td>-0.391679</td>\n",
       "      <td>0.175520</td>\n",
       "      <td>-0.412995</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>INDUS</th>\n",
       "      <td>0.406583</td>\n",
       "      <td>-0.533828</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.062938</td>\n",
       "      <td>0.763651</td>\n",
       "      <td>-0.391676</td>\n",
       "      <td>0.644779</td>\n",
       "      <td>-0.708027</td>\n",
       "      <td>0.595129</td>\n",
       "      <td>0.720760</td>\n",
       "      <td>0.383248</td>\n",
       "      <td>-0.356977</td>\n",
       "      <td>0.603800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>CHAS</th>\n",
       "      <td>-0.055892</td>\n",
       "      <td>-0.042697</td>\n",
       "      <td>0.062938</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.091203</td>\n",
       "      <td>0.091251</td>\n",
       "      <td>0.086518</td>\n",
       "      <td>-0.099176</td>\n",
       "      <td>-0.007368</td>\n",
       "      <td>-0.035587</td>\n",
       "      <td>-0.121515</td>\n",
       "      <td>0.048788</td>\n",
       "      <td>-0.053929</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>NOX</th>\n",
       "      <td>0.420972</td>\n",
       "      <td>-0.516604</td>\n",
       "      <td>0.763651</td>\n",
       "      <td>0.091203</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>-0.302188</td>\n",
       "      <td>0.731470</td>\n",
       "      <td>-0.769230</td>\n",
       "      <td>0.611441</td>\n",
       "      <td>0.668023</td>\n",
       "      <td>0.188933</td>\n",
       "      <td>-0.380051</td>\n",
       "      <td>0.590879</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>RM</th>\n",
       "      <td>-0.219247</td>\n",
       "      <td>0.311991</td>\n",
       "      <td>-0.391676</td>\n",
       "      <td>0.091251</td>\n",
       "      <td>-0.302188</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>-0.240265</td>\n",
       "      <td>0.205246</td>\n",
       "      <td>-0.209847</td>\n",
       "      <td>-0.292048</td>\n",
       "      <td>-0.355501</td>\n",
       "      <td>0.128069</td>\n",
       "      <td>-0.613808</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>AGE</th>\n",
       "      <td>0.352734</td>\n",
       "      <td>-0.569537</td>\n",
       "      <td>0.644779</td>\n",
       "      <td>0.086518</td>\n",
       "      <td>0.731470</td>\n",
       "      <td>-0.240265</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>-0.747881</td>\n",
       "      <td>0.456022</td>\n",
       "      <td>0.506456</td>\n",
       "      <td>0.261515</td>\n",
       "      <td>-0.273534</td>\n",
       "      <td>0.602339</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>DIS</th>\n",
       "      <td>-0.379670</td>\n",
       "      <td>0.664408</td>\n",
       "      <td>-0.708027</td>\n",
       "      <td>-0.099176</td>\n",
       "      <td>-0.769230</td>\n",
       "      <td>0.205246</td>\n",
       "      <td>-0.747881</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>-0.494588</td>\n",
       "      <td>-0.534432</td>\n",
       "      <td>-0.232471</td>\n",
       "      <td>0.291512</td>\n",
       "      <td>-0.496996</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>RAD</th>\n",
       "      <td>0.625505</td>\n",
       "      <td>-0.311948</td>\n",
       "      <td>0.595129</td>\n",
       "      <td>-0.007368</td>\n",
       "      <td>0.611441</td>\n",
       "      <td>-0.209847</td>\n",
       "      <td>0.456022</td>\n",
       "      <td>-0.494588</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.910228</td>\n",
       "      <td>0.464741</td>\n",
       "      <td>-0.444413</td>\n",
       "      <td>0.488676</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>TAX</th>\n",
       "      <td>0.582764</td>\n",
       "      <td>-0.314563</td>\n",
       "      <td>0.720760</td>\n",
       "      <td>-0.035587</td>\n",
       "      <td>0.668023</td>\n",
       "      <td>-0.292048</td>\n",
       "      <td>0.506456</td>\n",
       "      <td>-0.534432</td>\n",
       "      <td>0.910228</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.460853</td>\n",
       "      <td>-0.441808</td>\n",
       "      <td>0.543993</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>PTRATIO</th>\n",
       "      <td>0.289946</td>\n",
       "      <td>-0.391679</td>\n",
       "      <td>0.383248</td>\n",
       "      <td>-0.121515</td>\n",
       "      <td>0.188933</td>\n",
       "      <td>-0.355501</td>\n",
       "      <td>0.261515</td>\n",
       "      <td>-0.232471</td>\n",
       "      <td>0.464741</td>\n",
       "      <td>0.460853</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>-0.177383</td>\n",
       "      <td>0.374044</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>B</th>\n",
       "      <td>-0.385064</td>\n",
       "      <td>0.175520</td>\n",
       "      <td>-0.356977</td>\n",
       "      <td>0.048788</td>\n",
       "      <td>-0.380051</td>\n",
       "      <td>0.128069</td>\n",
       "      <td>-0.273534</td>\n",
       "      <td>0.291512</td>\n",
       "      <td>-0.444413</td>\n",
       "      <td>-0.441808</td>\n",
       "      <td>-0.177383</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>-0.366087</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>LSTAT</th>\n",
       "      <td>0.455621</td>\n",
       "      <td>-0.412995</td>\n",
       "      <td>0.603800</td>\n",
       "      <td>-0.053929</td>\n",
       "      <td>0.590879</td>\n",
       "      <td>-0.613808</td>\n",
       "      <td>0.602339</td>\n",
       "      <td>-0.496996</td>\n",
       "      <td>0.488676</td>\n",
       "      <td>0.543993</td>\n",
       "      <td>0.374044</td>\n",
       "      <td>-0.366087</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "             CRIM        ZN     INDUS      CHAS       NOX        RM       AGE  \\\n",
       "CRIM     1.000000 -0.200469  0.406583 -0.055892  0.420972 -0.219247  0.352734   \n",
       "ZN      -0.200469  1.000000 -0.533828 -0.042697 -0.516604  0.311991 -0.569537   \n",
       "INDUS    0.406583 -0.533828  1.000000  0.062938  0.763651 -0.391676  0.644779   \n",
       "CHAS    -0.055892 -0.042697  0.062938  1.000000  0.091203  0.091251  0.086518   \n",
       "NOX      0.420972 -0.516604  0.763651  0.091203  1.000000 -0.302188  0.731470   \n",
       "RM      -0.219247  0.311991 -0.391676  0.091251 -0.302188  1.000000 -0.240265   \n",
       "AGE      0.352734 -0.569537  0.644779  0.086518  0.731470 -0.240265  1.000000   \n",
       "DIS     -0.379670  0.664408 -0.708027 -0.099176 -0.769230  0.205246 -0.747881   \n",
       "RAD      0.625505 -0.311948  0.595129 -0.007368  0.611441 -0.209847  0.456022   \n",
       "TAX      0.582764 -0.314563  0.720760 -0.035587  0.668023 -0.292048  0.506456   \n",
       "PTRATIO  0.289946 -0.391679  0.383248 -0.121515  0.188933 -0.355501  0.261515   \n",
       "B       -0.385064  0.175520 -0.356977  0.048788 -0.380051  0.128069 -0.273534   \n",
       "LSTAT    0.455621 -0.412995  0.603800 -0.053929  0.590879 -0.613808  0.602339   \n",
       "\n",
       "              DIS       RAD       TAX   PTRATIO         B     LSTAT  \n",
       "CRIM    -0.379670  0.625505  0.582764  0.289946 -0.385064  0.455621  \n",
       "ZN       0.664408 -0.311948 -0.314563 -0.391679  0.175520 -0.412995  \n",
       "INDUS   -0.708027  0.595129  0.720760  0.383248 -0.356977  0.603800  \n",
       "CHAS    -0.099176 -0.007368 -0.035587 -0.121515  0.048788 -0.053929  \n",
       "NOX     -0.769230  0.611441  0.668023  0.188933 -0.380051  0.590879  \n",
       "RM       0.205246 -0.209847 -0.292048 -0.355501  0.128069 -0.613808  \n",
       "AGE     -0.747881  0.456022  0.506456  0.261515 -0.273534  0.602339  \n",
       "DIS      1.000000 -0.494588 -0.534432 -0.232471  0.291512 -0.496996  \n",
       "RAD     -0.494588  1.000000  0.910228  0.464741 -0.444413  0.488676  \n",
       "TAX     -0.534432  0.910228  1.000000  0.460853 -0.441808  0.543993  \n",
       "PTRATIO -0.232471  0.464741  0.460853  1.000000 -0.177383  0.374044  \n",
       "B        0.291512 -0.444413 -0.441808 -0.177383  1.000000 -0.366087  \n",
       "LSTAT   -0.496996  0.488676  0.543993  0.374044 -0.366087  1.000000  "
      ]
     },
     "execution_count": 288,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X.corr()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 第五种方式：sns.heatmap"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 289,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<AxesSubplot:>"
      ]
     },
     "execution_count": 289,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAk4AAAHWCAYAAACBhAZGAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAAA2S0lEQVR4nO3deZxkZXn3/89XBASBEFEBARk1KBrUEUdM4oYicQkRiRuDC2NiRiKIQUExPCYkRoUYg/IDJRNkM4mIj4mOSkQjoiZoYIRBNkE2cVhEwKg8oix9/f6oM23RU91dU1Pdp2bq857XeU2dc+4656q1r7ruu+5KVSFJkqTZPajtACRJktYXJk6SJEl9MnGSJEnqk4mTJElSn0ycJEmS+mTiJEmS1CcTJ0mStN5JckqS25JcNs3+JDk+yTVJvptk92Gc18RJkiStj04DXjzD/pcAuzTLUuBjwzipiZMkSVrvVNU3gDtnaLIvcEZ1fBvYOsn263peEydJkrQh2gH4Ydf6qmbbOnnwuh5glN17+3Uj8Xsy73v6e9oOYdJhL/hR2yEA8Plztms7BABe9ge3tR3CpGO/8vC2QwDgHU+7qe0QAHjzRVu3HcKk09+9oO0QOn7xi7YjAODny7/fdgiTDr5+q7ZDAODUfSfaDmHSlsd/IfN5vrn4W7vJIx73Zjrda6stq6pla3mYXvfDOse6QSdOkiRp/dMkSWubKE21Ctipa31H4OZ1PKaJkyRJWgcT97cdwXSWA4ckORN4JvDTqrplXQ9q4iRJktY7ST4J7Ak8PMkq4K+AjQGq6iTgbOClwDXAL4A3DuO8Jk6SJGlw1c74rqpaPMv+Ag4e9nn9Vp0kSVKfrDhJkqTBTYzONwrng4mTJEkaWLXUVdcWu+okSZL6ZMVJkiQNbsy66qw4SZIk9cmKkyRJGtyYjXEycZIkSYMb3ZnD58S8dNUl2S7JmUmuTXJFkrOTPD7J3UlWNtvOSLJx037PJF9oLi9JUkn26jrefs22V85H/JIkSTAPiVOSAP8OnFdVj6uqJwF/AWwLXFtVC4En0/nxvVdPc5hLge4ZQvcHLpmzoCVJUn9qYvjLCJuPitPzgXub340BoKpWAj/sWr8fuADYYZpjfBPYI8nGSbYAfgtYOVcBS5Ik9TIfY5x2A74zU4MkD6Hzy8Vvm6ZJAf8JvAj4DTq/ePyYIcYoSZIG4XQE8+pxSVYCdwA3VtV3Z2h7Jp0uuv2BT07XKMnSJCuSrDj5jGmbSZKkIaiaGPoyyuaj4nQ5MN0g7muramGS7YHzkrysqpb3alhVFyTZDbi7qq7uDJ3q2W4ZsAzg3tuvq3UPX5IkqWM+Kk7nApsm+dPVG5I8A9h59XpV3QIcCbx7lmO9m87AckmSNAomJoa/jLA5T5yqqoD9gL2b6QguB44Gbp7S9LPA5kmeM8Ox/qOqvjZXsUqSJM1kXibArKqb6T3VwG5dbQp4ate+85rtpwGn9TjmkiGGKEmSBjHiY5KGre3B4ZIkSesNf3JFkiQNbsx+csXESZIkDc6uOkmSJPVixUmSJA1uxKcPGDYrTpIkSX2y4iRJkgY3ZmOcTJwkSdLg7KqTJElSL1acJEnSwKrGax4nK06SJEl92qArTu97+nvaDgGAo77z3rZDmPS7Tz6w7RAAeEeq7RAAeOEXf9V2CJO++Ns3tR0CAM85/5dthwDAig8+qe0QJh36l9e0HQIAxWi8bu6srdoOYdL7Nrmv7RAAOOCzo1OH+Pzx83xCB4dLkiT1ycHhkiRJ6sWKkyRJGtyYddVZcZIkSeqTFSdJkjS4ifGajsDESZIkDc6uOkmSJPVixUmSJA3O6QgkSZLUixUnSZI0OMc4SZIkqRcTJ0mSNLiJieEvfUjy4iRXJbkmyZE99v9Gks8nuSTJ5UneOIybO1KJU5L9kqycskwk+bMkleStXW1PSLKkxXAlSVILiVOSjYATgZcATwIWJ5n6y+AHA1dU1VOBPYEPJdlkXW/uSCVOVfXvVbVw9QJ8FPgmcA5wG/C2YdxoSZK0XtsDuKaqrquqe4AzgX2ntClgyyQBtgDuBO5b1xOP7ODwJI8H/hL4PToJ3o+B/wYOBP6pxdAkSVKjqpWZw3cAfti1vgp45pQ2JwDLgZuBLYHXVK37SPaRqjitlmRj4F+Bw6vqxq5dxwDvaEp0kiRpA5RkaZIVXcvSqU16XK2mrL8IWAk8ClgInJBkq3WNbVQrTu8FLq+qM7s3VtX1SS4ADpjuis2duxRgn4ftwdO3+K05DVSSpLE2BxNgVtUyYNkMTVYBO3Wt70instTtjcAxVVXANUmuB3YFLliX2Eau4pRkT+AVwCHTNHk/8C6mib2qllXVoqpaZNIkSdIcq4nhL7O7ENglyWOasc/70+mW63YjsBdAkm2BJwDXrevNHanEKclvAqcCb6iqn/dqU1XfA64A9pnP2CRJ0mioqvvoFFjOAa4Ezqqqy5MclOSgptl7gd9LcinwVeBdVXX7up571LrqDgIeCXysMwh+0ientHsfcPF8BSVJkqbR0m/VVdXZwNlTtp3Udflm4PeHfd6RSpyq6gPAB6bZfWxXu0sYsWqZJEna8I1U4iRJktYzY/ZbdSZOkiRpcC111bXF7i5JkqQ+WXGSJEmDG7OuOitOkiRJfbLiJEmSBucYJ0mSJPVixUmSJA1uzCpOJk6SJGlwDg6XJElSL1acJEnS4Oyq23Ac9oIftR0CAL/75APbDmHSty49ve0QAPjv335X2yEA8PUzXtV2CJNeu/TLbYcAwIr3P6PtEADY9s8+1XYIk259/9B/J3QgefSCtkMA4FdnndN2CJPecv7WbYcAwKeO2L7tEDRPNujESZIkzbExG+Nk4iRJkgY3Zl11Dg6XJEnqkxUnSZI0uDHrqrPiJEmS1CcrTpIkaXBjNsbJxEmSJA1uzBInu+okSZL6ZMVJkiQNrqrtCOaVFSdJkqQ+WXGSJEmDc4yTJEmSerHiJEmSBmfFaTBJ7mr+X5Ckkry1a98JSZY0l09Lcn2SS5JcneSMJDtMPU7X+pIkJzSXn5DkvCQrk1yZZNmw4pckSQOoieEvI2yuuupuA96WZJNp9h9RVU8FngBcDHxthrbdjgeOq6qFVfVE4P8bTriSJEmzm6vE6cfAV4EDZ2pUHccBtwIv6eO42wOruq5/6boEKUmS1tHExPCXETaXg8OPAd6RZKM+2l4E7NpHu+OAc5P8R5LDkmy9LgFKkiStjTlLnKrqeuAC4IA+mme2wzXHPBV4IvBpYE/g20k2fcCBkqVJViRZcdpVN6113JIkaS1UDX8ZYXM9HcH7gXf1cZ6nAVc2l++eMt7pYcDtq1eq6uaqOqWq9gXuA3brPlBVLauqRVW1aMkTdkCSJM0hu+qGp6q+B1wB7NNrfzoOpTN26UvN5q8Dr2v2bwa8Gvhas/7iJBs3l7cDtgEsK0mSpHkxH/M4vY/ON+e6fTDJe4DNgW8Dz6+qe5p9bwP+sUmoApxRVd9o9v0+8JEkv2zWj6iqW+c2fEmSNK0RrxAN29ASp6raovn/Brq6z6rqEroqW1W1ZJbj3MQ0Faqqejvw9nWPVpIkae05c7gkSRrciE9YOWwmTpIkaWA1Mdrfghs2f+RXkiSpT1acJEnS4MZscLgVJ0mSpD5ZcZIkSYMbs8HhVpwkSdJ6p5kU+6ok1yQ5cpo2eyZZmeTyJF8fxnmtOEmSpMG18K26JBsBJwJ7A6uAC5Msr6orutpsDXwUeHFV3ZjkkcM4t4mTJEkaXDuDw/cArqmq6wCSnAnsS+dn3lY7APi3qroRoKpuG8aJ7aqTJEkjJcnSJCu6lqVTmuwA/LBrfVWzrdvjgd9Mcl6S7yR5wzBis+IkSZIGNwcVp6paBiyboUl6XW3K+oOBpwN7AZsB30ry7aq6el1iS9WGO+PnPz/qdSNx4zZa47Fsz/b33zN7o3nwrMuPbTsEAC588hFthzByqnq9H82/Szd+SNshTNru3tH41tCvMhqPzW0PHo04AJ50z71thwDALRtt3HYIk1538z/P6wP0i48cNPQ/cpu/7aQZb0OS3wWOrqoXNevvBqiqD3S1ORJ4SFUd3ax/HPhSVX16XWKzq06SJA2uavjL7C4EdknymCSbAPsDy6e0+RzwnCQPTrI58EzgynW9uXbVSZKkwbUwOLyq7ktyCHAOsBFwSlVdnuSgZv9JVXVlki8B3wUmgJOr6rJ1PbeJkyRJWu9U1dnA2VO2nTRl/YPAB4d5XhMnSZI0uBbmcWqTY5wkSZL6ZMVJkiQNbsx+q87ESZIkDc6uOkmSJPVixUmSJA2s2vmtutZYcZIkSeqTFSdJkjQ4xzhJkiSplzlLnJJsl+TMJNcmuSLJ2Uken+SyKe2OTnJ41/qDk9ye5ANT2u2T5OIklzTHe/NcxS5JkvpUE8NfRticdNUlCfDvwOlVtX+zbSGwbR9X/33gKuDVSf6iqirJxsAyYI+qWpVkU2DBXMQuSZLWgl11Q/F84N7u34ypqpXAD/u47mLgI8CNwO8027akk+Td0RzrV1V11TADliRJms1cDQ7fDfjONPsel2Rl1/p2wN8DJNkM2At4M7A1nSTqW1V1Z5LlwA+SfBX4AvDJqhGv50mStKFzOoI5d21VLVy9AN2/ZLwP8LWq+gXwGWC/JBsBVNWb6CRVFwCHA6f0OniSpUlWJFlx7i++P5e3Q5IkjZm5SpwuB54+wPUWAy9McgOditU2dLr9AKiqS6vqOGBv4BW9DlBVy6pqUVUtesHmuwwQgiRJ6ttEDX8ZYXOVOJ0LbJrkT1dvSPIMYOfprpBkK+DZwKOrakFVLQAOBhYn2SLJnl3NFwI/GH7YkiRprYzZt+rmJHGqqgL2A/ZupiO4HDgauHmGq/0RcG5V/apr2+eAlwEbAe9MclUzPuqvgSVzELokSdK05mzm8Kq6GXh1j127TWl3dNfqaVP23Qk8oll96RDDkyRJwzDiXWvD5szhkiRJffK36iRJ0sBqzKYjMHGSJEmDs6tOkiRJvVhxkiRJg7PiJEmSpF6sOEmSpMGN+ISVw2bFSZIkqU9WnCRJ0uDGbIyTiZMkSRpYjVniZFedJElSnzboitPL/uC2tkMA4IVf/NXsjebJ1894VdshAHDhk49oOwQAnnHpB9sOYdIhi97VdggAfPj4Z7YdAgDPe+VxbYcw6WfH7dd2CABkyy3bDgGAG9+/su0QJv2fB23SdggAnH7Y1m2H0B4rTpIkSeplg644SZKkOeZv1UmSJPXJrjpJkiT1YsVJkiQNzoqTJEmSerHiJEmSBlY1XhUnEydJkjQ4u+okSZLUixUnSZI0OCtOkiRJoy3Ji5NcleSaJEfO0O4ZSe5P8sphnNeKkyRJGli1UHFKshFwIrA3sAq4MMnyqrqiR7tjgXOGdW4rTpIkaX2zB3BNVV1XVfcAZwL79mj3VuAzwG3DOvG8J05JKsmHutYPT3J01/rSJN9rlguSPLvZ/vYkH+9q99okX5zX4CVJ0gNN1NCXJhdY0bUsnXLWHYAfdq2varZNSrIDsB9w0jBvbhtddb8C/ijJB6rq9u4dSfYB3gw8u6puT7I78NkkewDHAyuSPAu4HPhbYK95jl2SJHWbg9/4raplwLIZmqTX1aasfxh4V1Xdn/RqPpg2uuruo3NnHNZj37uAI1YnVFV1EXA6cHBV3Qe8hU6f5t8Bp1TVdfMTsiRJGiGrgJ261ncEbp7SZhFwZpIbgFcCH03y8nU9cVtjnE4EXpvkN6Zs/23gO1O2rWi2U1XnA1cCL6STPK2hu7x36vdWDTdqSZL0ADVRQ1/6cCGwS5LHJNkE2B9Y/oC4qh5TVQuqagHwf4G3VNVn1/X2tpI4VdXPgDOAQ/toHpryW5It6GSQGwOPmObYy6pqUVUteuOuOw4pYkmSNCqaXqhD6Hxb7krgrKq6PMlBSQ6ay3O3OR3Bh4GLgFO7tl0BPB04t2vb7s12gL8G/hn4EXAc8Ko5j1KSJE2vpQkwq+ps4Owp23oOBK+qJcM6b2vTEVTVncBZwJ90bf474Ngk2wAkWQgsodMv+WTgD+jMx7AM2DnJ3vMZsyRJmmJiDpYR1vYEmB+iU2oDoKqWN18fPD9JAT8HXgfcCnwaOKyqfgmQ5C3AGUkWNnM4SJIkzal5T5yqaouuyz8CNp+y/2PAx3pc9dlT2q0AnjQXMUqSpP60MXN4m5w5XJIkqU9td9VJkqT12YiPSRo2EydJkjQwu+okSZLUkxUnSZI0uDHrqrPiJEmS1CcrTpIkaWA1ZhUnEydJkjS4MUuc7KqTJEnqkxUnSZI0sHHrqrPiJEmS1KdUbbgTVx214ICRuHF/vstNbYcw6c1X/0bbIQBw+D0btx0CAKdvOjqfHU5YcWzbIQBw1KKj2g4BgDdvcWfbIUw69a5t2g4BgDu5r+0QAPjfurftECYdPjEa5Y5X3n1j2yFMuu72izOf57v9Rc8b+t/ah5/z9Xm9DWtjdP5qSJIkjTjHOEmSpIGN2xgnEydJkjSwcUuc7KqTJEnqkxUnSZI0MCtOkiRJ6smKkyRJGlyN7MwBc8LESZIkDcyuOkmSJPVkxUmSJA2sJsarq86KkyRJUp+sOEmSpIE5xqklSe5PsjLJZUk+n2TrZvuCJJXkvV1tH57k3iQntBawJEmiKkNfRtnIJE7A3VW1sKp2A+4EDu7adx2wT9f6q4DL5zM4SZKkUUqcun0L2KFr/W7gyiSLmvXXAGfNe1SSJOkBamL4yygbucQpyUbAXsDyKbvOBPZPsiNwP3DzfMcmSZLG2yglTpslWQncATwM+MqU/V8C9gYWA5+a7iBJliZZkWTFxT+/Zq5ilSRJdKYjGPYyykYpcbq7qhYCOwOb8MAxTlTVPcB3gHcAn5nuIFW1rKoWVdWip235W3MYriRJGjcjNx1BVf00yaHA55J8bMruDwFfr6o7ktHOSCVJGgdVbUcwv0YucQKoqouTXALsD3yza/vl+G06SZJGxqh3rQ3byCROVbXFlPU/7FrdrUf704DT5jYqSZKkXxuZxEmSJK1/xq3iNEqDwyVJkkaaFSdJkjQwB4dLkiT1ya46SZKkEZfkxUmuSnJNkiN77H9tku82y/lJnjqM81pxkiRJA6ua/4pT8/NsJ9L5RZFVwIVJllfVFV3NrgeeV1U/SfISYBnwzHU9txUnSZK0vtkDuKaqrmt+WeRMYN/uBlV1flX9pFn9NrDjME5sxUmSJA2sJlo57Q7AD7vWVzFzNelPgP8YxolNnCRJ0sAm5qCrLslSYGnXpmVVtay7SY+r9fx+X5Ln00mcnj2M2EycJEnSSGmSpGUzNFkF7NS1viNw89RGSZ4CnAy8pKruGEZsJk6SJGlgbQwOBy4EdknyGOAmOr9te0B3gySPBv4NeH1VXT2sE2/QidM7nnZT2yEA8Jzzf9l2CJNWvP8ZbYcAwEVH3dB2CAB8+Ph1/oLF0By16Ki2QwDgfSve13YIAGyx4/PaDmHS7UuH8i3mdfagLTdrOwQA/vfcn7YdwqRjb3lE2yEAcNlf/W7bIYyVqrovySHAOcBGwClVdXmSg5r9JwF/CWwDfDQJwH1VtWhdz71BJ06SJGlutTUBZlWdDZw9ZdtJXZffBLxp2Od1OgJJkqQ+WXGSJEkD87fqJEmS+uRv1UmSJKknK06SJGlgczEB5iiz4iRJktQnK06SJGlgLU2A2RoTJ0mSNLBx+1adXXWSJEl9suIkSZIG5uBwSZIk9WTFSZIkDWzcBoe3VnFKsl+SSrJr17Y9kpyX5PtJLkryxSRPbvYdneSmJCu7lq3bil+SJHUGhw97GWVtVpwWA/8F7A8cnWRb4CzggKo6HyDJs4HHAZc21zmuqv6+jWAlSZJaSZySbAE8C3g+sBw4GjgEOH110gRQVf/VRnySJKk/Dg6fHy8HvlRVVwN3Jtkd+G3golmud1hXN93XejVIsjTJiiQrTr/hluFGLUmSxlpbXXWLgQ83l89s1h8gyf8AWwFfrqq3NZtn7aqrqmXAMoA793veiPeUSpK0fhu3weHznjgl2QZ4AbBbkgI2Ago4Hdgd+BxAVT0zySuBfeY7RkmSpF7a6Kp7JXBGVe1cVQuqaifgeuDLwJIkv9fVdvMW4pMkSX2aqAx9GWVtdNUtBo6Zsu0zwAHAa4Bjk+wA3AbcDvxNV7vDkryua/3lVXXDHMYqSZJmMG5jYuY9caqqPXtsO75r9XnTXO9oOt++kyRJaoUzh0uSpIGNetfasPlbdZIkSX2y4iRJkgbmdASSJEl9mmg7gHlmV50kSVKfrDhJkqSBFePVVWfFSZIkqU9WnCRJ0sAmxmwGTBMnSZI0sAm76iRJktSLFSdJkjSwcRscvkEnTm++aOu2QwBgxQef1HYIk7b9s0+1HQIAf/ew32s7BACe98rj2g5h0lWP363tEADYYseePxc57+5a9fW2Q5i051Pf1HYIANw98b9thwDA3pvu1HYIk45acEvbIQCw9zE/bjuESf/91rYj2LBt0ImTJEmaW06AKUmSpJ6sOEmSpIE5xkmSJKlPdtVJkiSpJytOkiRpYFacJEmS1JOJkyRJGliRoS/9SPLiJFcluSbJkT32J8nxzf7vJtl9GLfXrjpJkjSwiRa+VJdkI+BEYG9gFXBhkuVVdUVXs5cAuzTLM4GPNf+vEytOkiRpfbMHcE1VXVdV9wBnAvtOabMvcEZ1fBvYOsn263piK06SJGlgE+3M47QD8MOu9VWsWU3q1WYHYJ1+p8eKkyRJGilJliZZ0bUsndqkx9VqgDZrbSQqTknuBy4FNgbuA04HPlxVE0n2BA6vqn2SbAt8HNipaXtDVb20naglSdI6ZyK9jlm1DFg2Q5NVdHKB1XYEbh6gzVobicQJuLuqFgIkeSTwr8BvAH81pd3fAF+pqo80bZ8yn0FKkqQHamkepwuBXZI8BrgJ2B84YEqb5cAhSc6k043306pap246GMGuuqq6DVhK58ZOLbNtTyeDXN32u/MZmyRJal9V3QccApwDXAmcVVWXJzkoyUFNs7OB64BrgH8C3jKMc49KxekBquq6JA8CHjll14nAp5IcAvwncGpVrXPZTZIkDWZijRrH/Kiqs+kkR93bTuq6XMDBwz7vyFWcuqzxSFTVOcBj6WSOuwIXJ3nEA67UNaDsurtumJdAJUnSeBjJxCnJY4H7gdum7quqO6vqX6vq9XT6OJ87Zf+yqlpUVYseu8WCeYlXkqRxVXOwjLKRS5yaCtJJwAlNma173wuSbN5c3hJ4HHDj/EcpSZLG0aiMcdosyUp+PR3BJ4B/6NHu6cAJSe6jk/SdXFUXzluUkiTpAVr6Vl1rRiJxqqqNZth3HnBec/mDwAfnJypJkjSbNn6rrk0j11UnSZI0qkai4iRJktZPLf1WXWusOEmSJPXJipMkSRrYqE8fMGwmTpIkaWAODpckSVJPVpwkSdLAxm0eJytOkiRJfbLiJEmSBubgcEmSpD45OFySJEk9bdAVp9PfvaDtEAA49C+vaTuESbe+//fbDgGAL3/gZ22HAMDPjtuv7RAmvf+Dd7YdAgC3L31q2yEAsOdT39R2CJPOu+TktkMAYOLOm9sOAYB7Pnps2yFMesVnHtJ2CACc87rN2w6hNQ4OlyRJUk8bdMVJkiTNLStOkiRJ6smKkyRJGliN2bfqTJwkSdLA7KqTJElST1acJEnSwKw4SZIkqScrTpIkaWD+Vp0kSVKf/K06SZIk9WTFSZIkDczB4ZIkSepp3hOnJPcnWZnksiSfT7L1lP2XJPnklG2nJbm+2Xd1kjOS7DCvgUuSpDVMzMEyytqoON1dVQurajfgTuDg1TuSPLGJ6blJHjrlekdU1VOBJwAXA19Lssl8BS1JktZUc7CMsra76r4FdFeODgA+AXwZeFmvK1THccCtwEvmPEJJkqRGa4lTko2AvYDlXZtfA3wK+CSweJZDXATs2uO4S5OsSLLilG9eNqxwJUlSDxMZ/jLK2kicNkuyErgDeBjwFYAkzwB+XFU/AL4K7J7kN2c4Ts+7tqqWVdWiqlr0x8/ZbbiRS5KksdbaGCdgZ2ATfj3GaTGwa5IbgGuBrYBXzHCcpwFXzl2YkiRpNg4OnydV9VPgUODwJJsCrwKeUlULqmoBsC89uuvScSiwPfCleQxZkiSNuVYHh1fVxcAlwKuBm6rqpq7d3wCelGT7Zv2DSS4BrgaeATy/qu6Z14AlSdIDjNu36uZ95vCq2mLK+h82Fz8xZfv9dKpKAEvmPjJJkrS2JkY+1RmutqcjkCRJWm+YOEmSpIGN4uDwJA9L8pUk32/+X+Nb+kl2SvK1JFcmuTzJ2/o5tomTJEna0BwJfLWqdqEzxdGRPdrcB7yjqp4I/A5wcJInzXZgEydJkjSwER0cvi9wenP5dODla8RddUtVXdRc/jmdKY5m/R3ceR8cLkmSNhwjOu/StlV1C3QSpCSPnKlxkgV05of8n9kObOIkSZJGSpKlwNKuTcuqatmUNv8JbNfj6ket5bm2AD4D/HlV/Wy29iZOkiRpYHPx23JNkrRsljYvnG5fkh8l2b6pNm0P3DZNu43pJE3/UlX/1k9sjnGSJEkbmuXAgc3lA4HPTW2QJMDHgSur6h/6PbCJkyRJGtgENfRlCI4B9k7yfWDvZp0kj0pydtPmWcDrgRckWdksL53twHbVSZKkgY3ivOFVdQewV4/tNwMvbS7/F7DWHY0bduL0i1+0HQEAHzl0aw49/idthwFAHr2g7RAA+FUubTsEAD7/tz/hZcc+uu0wALizdxf8vHvQlpu1HQIAX16yGc855Za2wwBg4s6b2w4BgAc97FFthwDAQ/7PR/jl3xzadhgAZO3/7s2JV/zr3fzbQY9oOwzNgw07cRoRo5I0aU2jkjRpTaOSNGlNo5I0jZJxTppGdDqCOeMYJ0mSpD5ZcZIkSQMb0mDu9YaJkyRJGth4pU121UmSJPXNipMkSRqYg8MlSZLUkxUnSZI0sHEbHG7FSZIkqU9WnCRJ0sDGq95k4iRJktaBg8MlSZLUkxUnSZI0sBqzzrpWKk5JtkmyslluTXJT1/q2Se5N8uau9lsmuTbJLs36xkkuTfLMNuKXJEnjqZXEqaruqKqFVbUQOAk4rmv9FcC3gcVd7X8OvBs4sdl0OHB+Vf3PvAYuSZIeYGIOllE2imOcFgPvAHZMssPqjVV1FjCR5J3AQXQSKUmS1KIJaujLKBupxCnJTsB2VXUBcBbwmilN/hw4FvjbqrpznsOTJEljbqQSJ2B/OgkTwJl0ddc1XgzcAuw23QGSLE2yIsmKU771vbmJUpIkAZ15nIa9jLJRS5wWA0uS3AAsB57aNSD8UcChwB7AS5M8pdcBqmpZVS2qqkV//Lu7zlPYkiRpHIxM4pTkCcBDq2qHqlpQVQuAD9CpQgEcB7y/qlYBbwdOTJJ2opUkSeAYpzYtBv59yrbPAIuT7A08Gvg4QFV9HvgJ8IZ5jVCSJD3AuH2rrvUJMKvq6Bn2fRd4UrP6lSn7XjaHYUmSJK2h9cRJkiStv5w5XJIkST1ZcZIkSQMb9TFJw2bFSZIkqU9WnCRJ0sDGbYyTiZMkSRqYXXWSJEnqyYqTJEka2ESNV1edFSdJkqQ+WXGSJEkDG696k4mTJElaB6P+o7zDtkEnTj9f/v22QwDgztqq7RAm/eqsc9oOAYDbHrxD2yEAcOP7V7YdwqT/rYe0HQIA/3vuT9sOAYC9N92p7RAm3fPRY9sOoWNiNP5APeQvj287hEnP/dx72g4BgHu/+8O2Q9A82aATJ0mSNLfGbR4nB4dLkiT1yYqTJEka2LhNgGniJEmSBjZug8PtqpMkSeqTiZMkSRpYzcG/dZXkYUm+kuT7zf+/OUPbjZJcnOQL/RzbxEmSJG1ojgS+WlW7AF9t1qfzNuDKfg9s4iRJkgY2MQfLEOwLnN5cPh14ea9GSXYE/gA4ud8DmzhJkqQNzbZVdQtA8/8jp2n3YeCdrEW+5rfqJEnSwKqG/626JEuBpV2bllXVsilt/hPYrsfVj+rzHPsAt1XVd5Ls2W9sJk6SJGlgczEdQZMkLZulzQun25fkR0m2r6pbkmwP3Naj2bOAlyV5KfAQYKsk/1xVr5vpvHbVSZKkDc1y4MDm8oHA56Y2qKp3V9WOVbUA2B84d7akCfpInJLcn2RlksuSfDrJDs36yiS3Jrmpa32TKe0/n2TrKce7JMknm8tv7LruPUkubS4fk2RJkhO6rrc0yfea5YIkz54tdkmSNLdGdHD4McDeSb4P7N2sk+RRSc5elwP301V3d1UtbE74L8BrutaPBu6qqr9f3ThJd/vTgYOB9zXrT6STrD03yUOr6lTg1GbfDcDzq+r2Zn1J1zH3Ad4MPLuqbk+yO/DZJHtU1a2D3nhJkrThqao7gL16bL8ZeGmP7ecB5/Vz7LXtqvsm8Ftr0f5bwA5d6wcAnwC+DLxsLY7zLuCI1UlVVV1E5+uFB6/FMSRJ0pCN4gSYc6nvxCnJg4GXAJf22X4jOtne8q7NrwE+BXwSWNx/mPw28J0p21Y02yVJUksmqKEvo6yfxGmzJCvpJCo3Ah/vs/0dwMOArwAkeQbw46r6AZ1ZPHefaQr0PgTWvHebsVArkqz4xM03r8PhJUmSHqifxOnuqlrYLG+tqnv6aQ/sDGzCr7vTFgO7NmOZrgW2Al7RZ5xXAE+fsm33ZvsDVNWyqlpUVYte/6hH9Xl4SZI0iKoa+jLK5mw6gqr6KXAocHiSTYFXAU+pqgXNV//2pf/uur8Djk2yDUCShcAS4KNDDluSJGlaczoBZlVdnOQS4NXATVV1U9fubwBPWj1B1SzHWZ5kB+D8JAX8HHjdbNeTJElza0jTB6w3Zk2cqmqLGfYdPVv7qvrD5uInpmy/H9i+a33BlP2nAad1rX8M+Nhs8UqSpPkz6t+CGzZnDpckSeqTv1UnSZIGNurTBwybFSdJkqQ+WXGSJEkDG/XpA4bNipMkSVKfrDhJkqSBjdsYJxMnSZI0MKcjkCRJUk9WnCRJ0sAmHBwuSZKkXqw4SZKkgY1XvWkDT5wOvn6rtkMA4H2b3Nd2CJPecv7WbYcAwJ/cc2/bIQDwfx60SdshTHr3xGj8VOaxtzyi7RAAOGrB6PyG9ys+85C2QwAgpO0QAHju597TdgiT3vmd97YdAgBvePrb2w5h0ifn+Xzj9q06u+okSZL6tEFXnCRJ0tyy4iRJkqSerDhJkqSBjdtv1Zk4SZKkgdlVJ0mSpJ6sOEmSpIH5W3WSJEnqyYqTJEka2LgNDrfiJEmS1CcrTpIkaWDj9q06EydJkjQwu+okSZLU03pVcUpyP3ApEOB+4JCqOr/dqCRJGl921Y22u6tqIUCSFwEfAJ7XakSSJGlsrG+JU7etgJ+0HYQkSeNs3CbAXN8Sp82SrAQeAmwPvKDdcCRJGm8TDg4faXdX1cKq2hV4MXBGknQ3SLI0yYokK6676wftRClJkjZI61viNKmqvgU8HHjElO3LqmpRVS167BY7txOcJEljoubg3yhbbxOnJLsCGwF3tB2LJEkaD+vrGCfoTElwYFXd32I8kiSNtXEb47ReJU5VtVHbMUiSpF8bxa61JA8DPgUsAG4AXl1Va3wTP8nWwMnAbkABf9wMBZrWettVJ0mSNI0jga9W1S7AV5v1Xj4CfKn50tlTgStnO/B6VXGSJEmjZUS76vYF9mwunw6cB7yru0GSrYDnAksAquoe4J7ZDmzFSZIkbWi2rapbAJr/H9mjzWOBHwOnJrk4yclJHjrbgU2cJEnSwOZiOoLuORmbZenU8yb5zySX9Vj27TP0BwO7Ax+rqqcB/4/pu/QecCVJkqSRUVXLgGWztHnhdPuS/CjJ9lV1S5Ltgdt6NFsFrKqq/2nW/y99JE5WnCRJ0sAmqoa+DMFy4MDm8oHA56Y2qKpbgR8meUKzaS/gitkObMVJkiQNbBSnIwCOAc5K8ifAjcCrAJI8Cji5ql7atHsr8C9JNgGuA94424FNnCRJ0galqu6gU0Gauv1m4KVd6yuBRWtzbBMnSZI0sKqJtkOYV45xkiRJ6pMVJ0mSNLCJ0RzjNGc26MTp1H1Ho3x4wGdHp7D3qSO2bzsEAP7tAz9rOwQATj9s67ZDmLTb317QdggAXPZXv9t2CADsfcyP2w5h0jmv27ztEADIVlu0HQIA9373h22HMOkNT3972yEAcMZ3/qHtEFpTozlz+JwZnb/okiRJI26DrjhJkqS5NW5ddVacJEmS+mTFSZIkDWzcxjiZOEmSpIEN6SdS1ht21UmSJPXJipMkSRrYiP5W3Zyx4iRJktQnK06SJGlg4zY43IqTJElSn6w4SZKkgY3bBJgmTpIkaWB21a2jJHf12PaEJOclWZnkyiTLkryoWV+Z5K4kVzWXz2ius1+SSrJrs/4/zf4bk/y467oLhn0bJEmSepmvitPxwHFV9TmAJE+uqkuBc5r184DDq2pF13UWA/8F7A8cXVXPbNouARZV1SHzFLskSZqGE2DOje2BVatXmqRpWkm2AJ4F/AmdxEmSJKl185U4HQecm+Q/khyWZOtZ2r8c+FJVXQ3cmWT3uQ5QkiStvaoa+jLK5iVxqqpTgScCnwb2BL6dZNMZrrIYOLO5fGaz3pckS5OsSLLi1MtuHDBiSZLUjwlq6Msom7dv1VXVzcApwClJLgN2A74ztV2SbYAXALslKWAjoJK8s/pIQ6tqGbAM4OeH7jPa974kSVqvzEvFKcmLk2zcXN4O2Aa4aZrmrwTOqKqdq2pBVe0EXA88ez5ilSRJ/Ru3rrq5qDhtnmRV1/o/ADsCH0nyy2bbEVV16zTXXwwcM2XbZ4ADgG8ONVJJkqS1MPTEqaqmq2K9fYbr7Nnrcte247sunwacNmh8kiRpeMZtOgJnDpckSQOrER/MPWz+yK8kSVKfrDhJkqSBjVtXnRUnSZKkPllxkiRJAxv16QOGzYqTJElSn6w4SZKkgY3bt+pMnCRJ0sDsqpMkSVJPVpwkSdLArDhJkiSpJytOkiRpYONVb4KMW4ltEEmWVtUy4xitOGB0YjGONY1KLMaxplGJxTjWNEqxqDe76vqztO0AGsaxplGJxTjWNCqxGMeaRiUW41jTKMWiHkycJEmS+mTiJEmS1CcTp/6MSn+zcaxpVGIxjjWNSizGsaZRicU41jRKsagHB4dLkiT1yYqTJElSn8YycUqyXZIzk1yb5IokZyd5fJK7k6xstp2RZOOm/Z5JvtBcXpKkkuzVdbz9mm2vHEJs+zUxdC8TSf6sOcdbu9qekGTJgOe5q/l/wUzHTXJakuuTXJLk6uZ+2WHqcbrWlyQ5obn8hCTnNbfhyiQzlqBneFwum9Lu6CSHd60/OMntST4wpd0+SS5uYr8iyZsHuJ8qyYe61g9PcnTX+tIk32uWC5I8u9n+9iQf72r32iRfXNvzzxDX/c39elmSzyfZutm++vF8b1fbhye5d/XjMqTzr37O79q1bY/m8f5+kouSfDHJk5t9Rye5acrzeushxrP6/ri8ebzfnuRBzb7u1++2Sb7Q9Zw4e8jnf8Dj0bX/kiSfnLJtxtfWgHFs03X/3jrlPt+2eR68uav9ls3rbZdmfeMklyZ5Zh+389NJdpjhfJuszf2S5I1d172niWNlkmPS9b7StO35uhvg/lod3yXNc/b3BjnONMe+q8e2Nd4Tk7yo63bfleSq5vIZzXUe8FpL8j/N/huT/LjruguGFbtmUVVjtQABvgUc1LVtIfAc4LJmfSPgXOC1zfqewBeay0uA7wInd13/U8BK4JVzEO9S4OvAY4EfAdcAmzT7TgCWDHjcu5r/F8x0XOC01berue8OA67uanvXlOMuAU5oLp8D7Nu178nr8rh0bT8aOLxr/aXAfwPX8uvu542Bm4Edm/VNgScMcD/9ErgeeHizfjhwdHN5H+A7Xft2B24EtqMzuexK4FnA1s0xHjvE58VdXZdPB47qejyvBS7u2v9nTSwnDPH8ZwHf7LovtgVuAH6vq82zgZf3eszm4HXSfX88EvhP4K+b9T359ev3H4G3dbV9ylw+Hs36E4FLgZuAh3Ztn/G1NYSYpr5O3tI8ZudNafdq4MvN5XcD/9jn7fwX4O3TnW/Q+6XZd8Pq11WzvoRfv69M+7pbx8ftRcDX5+I52bVtxvdE4Dxg0ZRtD3it9bpPXOZ3GceK0/OBe6vqpNUbqmol8MOu9fuBC4DpPv19E9ij+XS2BfBbdP4wDVWSxwN/CbwemAB+DHwVOHDIp+rruNVxHHAr8JI+jrs9sKrr+pfO0HbWx2UGi4GP0Hnz/J1m25Z0kpc7mmP9qqqu6uNYU91HZ7DmYT32vQs4oqpub85xEZ0/DgdX1X10/lCdCPwdcEpVXTfA+fvxLR74XL0buDLJomb9NXTefIeiec4/C/gTYP9m8yHA6VV1/up2VfVfVfXZYZ23X1V1G50PHIckyZTdU5+T352DEKY+HgcAnwC+DLys1xUGeG0NYjHwDmDH7spWVZ0FTCR5J3AQneSpH9+k897Xr7W+X6Yx7etuLY7Ry1bAT9bxGLNZm/fE6V5ratk4Jk670fm0Mq0kDwGeCXxpmiZF5xPti4B9geXDDLCJYWPgX+l8gruxa9cxwDuSbDTkU67NcS8Cdp21FRwHnJvkP5IcNrVMP8VMj8vjusrRK+m8uQOQZDNgL+ALwCfp/HGgqu6k87j8IMkn0+kqG/T5fiLw2iS/MWX7b/eIeUWznSaJuBJ4IZ3kaeiax2sv1nwOngnsn2RH4H461bdheTnwpaq6Grgzye50bvNFs1zvsK7H8WtDjGcNTZL6IDrVp24nAh9P8rUkRyV51DDPO83j8Ro6VenJ5+cM+n1trW1cO9GpyFxAJ4l+zZQmfw4cC/xt89qZ7XgPppPgzfiHv6v9ut4v3WZ83a2lzZrn4/eAk4H3znaFdbQ274nQ+7Wmlo1j4jSTxzV/mO8Abpzl0+iZdD4B7E/nhT9s7wUur6ozuzdW1fV0qmEHDPNka3ncqZ/i1zhcc8xT6ZTjP02nu+TbSTYdILxrq2rh6gU4qWvfPsDXquoXwGeA/VYnf1X1Jjpv1hfQ6WI7ZYBzU1U/A84ADu2jeWhuf/NpcRGdbsNHDHLuGWzW9Vx9GPCVKfu/BOxN5w/Sp4Z87sV0nv80/6/xR68Zh3Flko90bT6u63F8/pBj6mWN52lVnUOn2/uf6CQoFycZxmPT8/FI8gzgx1X1AzpV3d2T/ObaxDwk+/PrqmOvx+zFwC10PsDMZPXtXEGnwvvxmZsP7X6ZzeTrbi3d3Twfd6VzH5zRo0o5NAO8J876WtP8G8fE6XLg6dPsu7b5w/xbwO8kmbZ83Hxy241OP/vVwwwwyZ7AK+h0f/Tyfjrl6mE/fv0e92l0KikAdyfZpGvfw4DbV69U1c1VdUpV7Uun22u6N+aZHpeZLAZemOQGOp9Ct6HT7bf6/Jc2XSB707lPB/VhOuXyh3Ztu4I1Y9692Q7w18A/A++j80lzmO5unqs7A5swpZuiqu6hc3+8g05CORRJtgFeAJzc3OdH0KkcXE7ntq8+/zOB9wBTq3TzIslj6VTabpu6r6rurKp/rarXAxcCzx3CKad7PBYDuzb31bV0uoNmeh52v7aGaTGwpIljOfDU/HpA+KPofCjYA3hpkqfMcJy7u5LftzbPs5kM637pNtvrbiBV9S3g4Qz/Q87U8/T1njjda20uEzv1ZxwTp3OBTZP86eoNzaefnVevV9UtwJHM3tf/buAvhhlc86nrVOANVfXzXm2q6nt03iT2Gea5ZztuOg6l00+/uhvz68Drmv2b0Rlo+rVm/cX59TcTt6OT1Nw0zelnfVx6xLMVnQHIj66qBVW1gM4b8+IkWzQJ6GoLgR9Md6zZNN0XZ9FJnlb7O+DY5g2OJAvpDNj8aDrfJvsDOt0fy4Cdk+w96PlniOundP7oHb76vu7yIeBdVXXHEE/5SuCMqtq5uc93ojPw/ct0/jB3fytp8yGet29NBekkOgNna8q+FyTZvLm8JfA4OpWToZjyeGwKvIrOAPTVz8996V2h6/XaGookT6Az+HqHrjg+wK/HzBwHvL+qVgFvB04c9h/nQe+XaUz7uluXGNP51tpGNOMi58JavidO91ob6BuEGp4Htx3AfKuqSrIf8OEkR9L51tQNdPr4u30WODrJc2Y41n/MQYgH0RmX8bEp711TuwPfB1w8B+fvddwPJnkPnT+E3wae3/VJ823APzZv+qHzQv9Gs+/3gY8k+WWzfkRV3drrpGvxuHT7I+DcqvpV17bP0XljfTvwziT/SGew9P+j8+a6Lj5EVxWwqpanM8j2/CQF/JxOEnkrnVL8YVX1S4Akb6HTDbCwj0/pa6WqLk5yCZ0/hN/s2n45nUrQMC2mMx6u22fodPG+hs4ftB3oVHpuB/6mq91hSV7Xtf7yqrphSHGt7hLamM6n+E8A/9Cj3dOBE5LcR+eD48lVdeGQYgAe8Hi8Gripqrr/MH4DeFKS7Zv1mV5bw7IY+Pcp2z4DnJnk28CjabrcqurzzYeXN9AZcD00/d4vzQfXmY7T83U32/Wmsfp5A533rwOr8+WgYdg8yaqu9X8AdqTP90Rmfq19c83mmi/OHC5JktSnceyqkyRJGoiJkyRJUp9MnCRJkvpk4iRJktQnEydJkqQ+mThJkiT1ycRJkiSpTyZOkiRJffr/AZRCXQgxwHftAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 720x576 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.figure(figsize = (10,8))\n",
    "sns.heatmap(X.corr())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 290,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>variables</th>\n",
       "      <th>VIF</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>CRIM</td>\n",
       "      <td>2.100373</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>ZN</td>\n",
       "      <td>2.844013</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>INDUS</td>\n",
       "      <td>14.485758</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>CHAS</td>\n",
       "      <td>1.152952</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>NOX</td>\n",
       "      <td>73.894947</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>RM</td>\n",
       "      <td>77.948283</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>AGE</td>\n",
       "      <td>21.386850</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>DIS</td>\n",
       "      <td>14.699652</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>RAD</td>\n",
       "      <td>15.167725</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>TAX</td>\n",
       "      <td>61.227274</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>PTRATIO</td>\n",
       "      <td>85.029547</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>B</td>\n",
       "      <td>20.104943</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>LSTAT</td>\n",
       "      <td>11.102025</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   variables        VIF\n",
       "0       CRIM   2.100373\n",
       "1         ZN   2.844013\n",
       "2      INDUS  14.485758\n",
       "3       CHAS   1.152952\n",
       "4        NOX  73.894947\n",
       "5         RM  77.948283\n",
       "6        AGE  21.386850\n",
       "7        DIS  14.699652\n",
       "8        RAD  15.167725\n",
       "9        TAX  61.227274\n",
       "10   PTRATIO  85.029547\n",
       "11         B  20.104943\n",
       "12     LSTAT  11.102025"
      ]
     },
     "execution_count": 290,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "vif(X)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 101,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 291,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>variables</th>\n",
       "      <th>VIF</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>CRIM</td>\n",
       "      <td>2.099345</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>ZN</td>\n",
       "      <td>2.451624</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>INDUS</td>\n",
       "      <td>14.275283</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>CHAS</td>\n",
       "      <td>1.142167</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>NOX</td>\n",
       "      <td>73.894171</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>RM</td>\n",
       "      <td>60.598846</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>AGE</td>\n",
       "      <td>21.361234</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>DIS</td>\n",
       "      <td>12.221605</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>RAD</td>\n",
       "      <td>15.159162</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>TAX</td>\n",
       "      <td>59.301541</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>B</td>\n",
       "      <td>18.614751</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>LSTAT</td>\n",
       "      <td>10.138324</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   variables        VIF\n",
       "0       CRIM   2.099345\n",
       "1         ZN   2.451624\n",
       "2      INDUS  14.275283\n",
       "3       CHAS   1.142167\n",
       "4        NOX  73.894171\n",
       "5         RM  60.598846\n",
       "6        AGE  21.361234\n",
       "7        DIS  12.221605\n",
       "8        RAD  15.159162\n",
       "9        TAX  59.301541\n",
       "10         B  18.614751\n",
       "11     LSTAT  10.138324"
      ]
     },
     "execution_count": 291,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "vif(X.drop('PTRATIO',axis=1))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 292,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>variables</th>\n",
       "      <th>VIF</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>CRIM</td>\n",
       "      <td>2.100373</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>ZN</td>\n",
       "      <td>2.844013</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>INDUS</td>\n",
       "      <td>14.485758</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>CHAS</td>\n",
       "      <td>1.152952</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>NOX</td>\n",
       "      <td>73.894947</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>RM</td>\n",
       "      <td>77.948283</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>AGE</td>\n",
       "      <td>21.386850</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>DIS</td>\n",
       "      <td>14.699652</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>RAD</td>\n",
       "      <td>15.167725</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>TAX</td>\n",
       "      <td>61.227274</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>PTRATIO</td>\n",
       "      <td>85.029547</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>B</td>\n",
       "      <td>20.104943</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>LSTAT</td>\n",
       "      <td>11.102025</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   variables        VIF\n",
       "0       CRIM   2.100373\n",
       "1         ZN   2.844013\n",
       "2      INDUS  14.485758\n",
       "3       CHAS   1.152952\n",
       "4        NOX  73.894947\n",
       "5         RM  77.948283\n",
       "6        AGE  21.386850\n",
       "7        DIS  14.699652\n",
       "8        RAD  15.167725\n",
       "9        TAX  61.227274\n",
       "10   PTRATIO  85.029547\n",
       "11         B  20.104943\n",
       "12     LSTAT  11.102025"
      ]
     },
     "execution_count": 292,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "vif(X)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 293,
   "metadata": {},
   "outputs": [],
   "source": [
    "X=X.drop('PTRATIO',axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 294,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>variables</th>\n",
       "      <th>VIF</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>CRIM</td>\n",
       "      <td>2.099345</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>ZN</td>\n",
       "      <td>2.451624</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>INDUS</td>\n",
       "      <td>14.275283</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>CHAS</td>\n",
       "      <td>1.142167</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>NOX</td>\n",
       "      <td>73.894171</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>RM</td>\n",
       "      <td>60.598846</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>AGE</td>\n",
       "      <td>21.361234</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>DIS</td>\n",
       "      <td>12.221605</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>RAD</td>\n",
       "      <td>15.159162</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>TAX</td>\n",
       "      <td>59.301541</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>B</td>\n",
       "      <td>18.614751</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>LSTAT</td>\n",
       "      <td>10.138324</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   variables        VIF\n",
       "0       CRIM   2.099345\n",
       "1         ZN   2.451624\n",
       "2      INDUS  14.275283\n",
       "3       CHAS   1.142167\n",
       "4        NOX  73.894171\n",
       "5         RM  60.598846\n",
       "6        AGE  21.361234\n",
       "7        DIS  12.221605\n",
       "8        RAD  15.159162\n",
       "9        TAX  59.301541\n",
       "10         B  18.614751\n",
       "11     LSTAT  10.138324"
      ]
     },
     "execution_count": 294,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "vif(X)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 198,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>variables</th>\n",
       "      <th>VIF</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>CRIM</td>\n",
       "      <td>2.097537</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>ZN</td>\n",
       "      <td>2.449661</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>INDUS</td>\n",
       "      <td>13.150904</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>CHAS</td>\n",
       "      <td>1.138277</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>RM</td>\n",
       "      <td>41.406746</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>AGE</td>\n",
       "      <td>19.889623</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>DIS</td>\n",
       "      <td>12.032952</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>RAD</td>\n",
       "      <td>15.155012</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>TAX</td>\n",
       "      <td>57.720347</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>B</td>\n",
       "      <td>18.396072</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>LSTAT</td>\n",
       "      <td>9.207840</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   variables        VIF\n",
       "0       CRIM   2.097537\n",
       "1         ZN   2.449661\n",
       "2      INDUS  13.150904\n",
       "3       CHAS   1.138277\n",
       "4         RM  41.406746\n",
       "5        AGE  19.889623\n",
       "6        DIS  12.032952\n",
       "7        RAD  15.155012\n",
       "8        TAX  57.720347\n",
       "9          B  18.396072\n",
       "10     LSTAT   9.207840"
      ]
     },
     "execution_count": 198,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "vif(X.drop('NOX',axis=1))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 295,
   "metadata": {},
   "outputs": [],
   "source": [
    "X=X.drop('NOX',axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 296,
   "metadata": {},
   "outputs": [],
   "source": [
    "X=X.drop('RM',axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 297,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>variables</th>\n",
       "      <th>VIF</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>CRIM</td>\n",
       "      <td>2.095375</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>ZN</td>\n",
       "      <td>2.432618</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>INDUS</td>\n",
       "      <td>13.149180</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>CHAS</td>\n",
       "      <td>1.131756</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>AGE</td>\n",
       "      <td>14.795487</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>DIS</td>\n",
       "      <td>9.234022</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>RAD</td>\n",
       "      <td>15.045607</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>TAX</td>\n",
       "      <td>54.461654</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>B</td>\n",
       "      <td>14.233397</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>LSTAT</td>\n",
       "      <td>8.396362</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  variables        VIF\n",
       "0      CRIM   2.095375\n",
       "1        ZN   2.432618\n",
       "2     INDUS  13.149180\n",
       "3      CHAS   1.131756\n",
       "4       AGE  14.795487\n",
       "5       DIS   9.234022\n",
       "6       RAD  15.045607\n",
       "7       TAX  54.461654\n",
       "8         B  14.233397\n",
       "9     LSTAT   8.396362"
      ]
     },
     "execution_count": 297,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "vif(X)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 299,
   "metadata": {},
   "outputs": [],
   "source": [
    "X=X.drop('TAX',axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 300,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>variables</th>\n",
       "      <th>VIF</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>CRIM</td>\n",
       "      <td>2.095211</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>ZN</td>\n",
       "      <td>2.313889</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>INDUS</td>\n",
       "      <td>8.205465</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>CHAS</td>\n",
       "      <td>1.106350</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>DIS</td>\n",
       "      <td>8.209371</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>RAD</td>\n",
       "      <td>4.689836</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>B</td>\n",
       "      <td>10.074224</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>LSTAT</td>\n",
       "      <td>6.856889</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  variables        VIF\n",
       "0      CRIM   2.095211\n",
       "1        ZN   2.313889\n",
       "2     INDUS   8.205465\n",
       "3      CHAS   1.106350\n",
       "4       DIS   8.209371\n",
       "5       RAD   4.689836\n",
       "6         B  10.074224\n",
       "7     LSTAT   6.856889"
      ]
     },
     "execution_count": 300,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "vif(X)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 301,
   "metadata": {},
   "outputs": [],
   "source": [
    "X=X.drop('B',axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 302,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>variables</th>\n",
       "      <th>VIF</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>CRIM</td>\n",
       "      <td>2.067135</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>ZN</td>\n",
       "      <td>2.299351</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>INDUS</td>\n",
       "      <td>6.900077</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>CHAS</td>\n",
       "      <td>1.086574</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>DIS</td>\n",
       "      <td>3.968908</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>RAD</td>\n",
       "      <td>4.678181</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>LSTAT</td>\n",
       "      <td>6.726973</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  variables       VIF\n",
       "0      CRIM  2.067135\n",
       "1        ZN  2.299351\n",
       "2     INDUS  6.900077\n",
       "3      CHAS  1.086574\n",
       "4       DIS  3.968908\n",
       "5       RAD  4.678181\n",
       "6     LSTAT  6.726973"
      ]
     },
     "execution_count": 302,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "vif(X)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 303,
   "metadata": {},
   "outputs": [],
   "source": [
    "#X=X.drop('B',axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 304,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>variables</th>\n",
       "      <th>VIF</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>CRIM</td>\n",
       "      <td>2.067135</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>ZN</td>\n",
       "      <td>2.299351</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>INDUS</td>\n",
       "      <td>6.900077</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>CHAS</td>\n",
       "      <td>1.086574</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>DIS</td>\n",
       "      <td>3.968908</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>RAD</td>\n",
       "      <td>4.678181</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>LSTAT</td>\n",
       "      <td>6.726973</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  variables       VIF\n",
       "0      CRIM  2.067135\n",
       "1        ZN  2.299351\n",
       "2     INDUS  6.900077\n",
       "3      CHAS  1.086574\n",
       "4       DIS  3.968908\n",
       "5       RAD  4.678181\n",
       "6     LSTAT  6.726973"
      ]
     },
     "execution_count": 304,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "vif(X)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#VIF should be below 4. That is analogous to having R^2 value less than .75\n",
    "\n",
    "variance inflation factor\n",
    "\n",
    "Interpreting the Variance Inflation Factor\n",
    "Variance inflation factors range from 1 upwards. The numerical value for VIF tells you (in decimal form) what percentage the variance (i.e. the standard error squared) is inflated for each coefficient. For example, a VIF of 1.9 tells you that the variance of a particular coefficient is 90% bigger than what you would expect if there was no multicollinearity — if there was no correlation with other predictors.\n",
    "A rule of thumb for interpreting the variance inflation factor:\n",
    "\n",
    "1 = not correlated.\n",
    "Between 1 and 5 = moderately correlated.\n",
    "Greater than 5 = highly correlated."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 305,
   "metadata": {},
   "outputs": [],
   "source": [
    "X=X.drop('LSTAT',axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 306,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>variables</th>\n",
       "      <th>VIF</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>CRIM</td>\n",
       "      <td>1.977574</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>ZN</td>\n",
       "      <td>2.249163</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>INDUS</td>\n",
       "      <td>4.357839</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>CHAS</td>\n",
       "      <td>1.082837</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>DIS</td>\n",
       "      <td>3.367351</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>RAD</td>\n",
       "      <td>4.632245</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  variables       VIF\n",
       "0      CRIM  1.977574\n",
       "1        ZN  2.249163\n",
       "2     INDUS  4.357839\n",
       "3      CHAS  1.082837\n",
       "4       DIS  3.367351\n",
       "5       RAD  4.632245"
      ]
     },
     "execution_count": 306,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "vif(X)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 第六种方式 Understanding Data -- the most important"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 217,
   "metadata": {},
   "outputs": [],
   "source": [
    "#such as cloumn C,F"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}