ToBigs_2wk_Regression3.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "fu39oBW0RVn5"
   },
   "source": [
    "# [과제 3] 로지스틱 회귀분석\n",
    "### - sklearn 패키지를 사용해 로지스틱 회귀분석을 진행해주세요.\n",
    "### - 성능지표를 계산하고 이에 대해 해석해주세요.\n",
    "### - 성능 개선을 시도해주세요. (어떠한 성능지표를 기준으로 개선을 시도했는지, 그 이유도 함께 적어주세요.)\n",
    "### - 주석으로 설명 및 근거 자세하게 달아주시면 감사하겠습니다. :)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "8rN2SWezRVn_"
   },
   "source": [
    "## Data "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Y7SYKNvQRVn_"
   },
   "source": [
    "출처 : https://www.kaggle.com/mlg-ulb/creditcardfraud\n",
    "\n",
    "\n",
    "* V1 ~ V28 : 비식별화 된 개인정보 \n",
    "* **Class** : Target 변수  \n",
    "  - 1 : fraudulent transactions (사기)\n",
    "  - 0 : otherwise "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "id": "Uvjw2fTCRVoA"
   },
   "outputs": [],
   "source": [
    "# 필요한 라이브러리들 import\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import seaborn as sns\n",
    "import matplotlib.pyplot as plt\n",
    "import matplotlib as mpl\n",
    "import warnings\n",
    "warnings.filterwarnings(action='ignore')\n",
    "%matplotlib inline\n",
    "\n",
    "plt.style.use('seaborn') # matplotlib 테마\n",
    "plt.rc('font', family = \"Malgun Gothic\") # 그래프에서 한글 깨지는 현상 방지"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "id": "znQit70ZRVoA"
   },
   "outputs": [],
   "source": [
    "data = pd.read_csv(\"2wk_Regression_assignment3_creditcard.csv\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 220
    },
    "id": "v98OeXW5RVoB",
    "outputId": "42afeddc-07e6-4224-95ee-08b455f72475"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>V1</th>\n",
       "      <th>V2</th>\n",
       "      <th>V3</th>\n",
       "      <th>V4</th>\n",
       "      <th>V5</th>\n",
       "      <th>V6</th>\n",
       "      <th>V7</th>\n",
       "      <th>V8</th>\n",
       "      <th>V9</th>\n",
       "      <th>V10</th>\n",
       "      <th>...</th>\n",
       "      <th>V20</th>\n",
       "      <th>V21</th>\n",
       "      <th>V22</th>\n",
       "      <th>V23</th>\n",
       "      <th>V24</th>\n",
       "      <th>V25</th>\n",
       "      <th>V26</th>\n",
       "      <th>V27</th>\n",
       "      <th>V28</th>\n",
       "      <th>Class</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>-1.848212</td>\n",
       "      <td>2.384900</td>\n",
       "      <td>0.379573</td>\n",
       "      <td>1.048381</td>\n",
       "      <td>-0.845070</td>\n",
       "      <td>2.537837</td>\n",
       "      <td>-4.542983</td>\n",
       "      <td>-10.201458</td>\n",
       "      <td>-1.504967</td>\n",
       "      <td>-2.234167</td>\n",
       "      <td>...</td>\n",
       "      <td>2.585817</td>\n",
       "      <td>-5.291690</td>\n",
       "      <td>0.859364</td>\n",
       "      <td>0.423231</td>\n",
       "      <td>-0.506985</td>\n",
       "      <td>1.020052</td>\n",
       "      <td>-0.627751</td>\n",
       "      <td>-0.017753</td>\n",
       "      <td>0.280982</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2.071805</td>\n",
       "      <td>-0.477943</td>\n",
       "      <td>-1.444444</td>\n",
       "      <td>-0.548657</td>\n",
       "      <td>0.010036</td>\n",
       "      <td>-0.582242</td>\n",
       "      <td>-0.042878</td>\n",
       "      <td>-0.247160</td>\n",
       "      <td>1.171923</td>\n",
       "      <td>-0.342382</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.077306</td>\n",
       "      <td>0.042858</td>\n",
       "      <td>0.390125</td>\n",
       "      <td>0.041569</td>\n",
       "      <td>0.598427</td>\n",
       "      <td>0.098803</td>\n",
       "      <td>0.979686</td>\n",
       "      <td>-0.093244</td>\n",
       "      <td>-0.065615</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>-2.985294</td>\n",
       "      <td>-2.747472</td>\n",
       "      <td>1.194068</td>\n",
       "      <td>-0.003036</td>\n",
       "      <td>-1.151041</td>\n",
       "      <td>-0.263559</td>\n",
       "      <td>0.553500</td>\n",
       "      <td>0.635600</td>\n",
       "      <td>0.438545</td>\n",
       "      <td>-1.806488</td>\n",
       "      <td>...</td>\n",
       "      <td>1.345776</td>\n",
       "      <td>0.373760</td>\n",
       "      <td>-0.385777</td>\n",
       "      <td>1.197596</td>\n",
       "      <td>0.407229</td>\n",
       "      <td>0.008013</td>\n",
       "      <td>0.762362</td>\n",
       "      <td>-0.299024</td>\n",
       "      <td>-0.303929</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>-1.479452</td>\n",
       "      <td>1.542874</td>\n",
       "      <td>0.290895</td>\n",
       "      <td>0.838142</td>\n",
       "      <td>-0.529290</td>\n",
       "      <td>-0.717661</td>\n",
       "      <td>0.484516</td>\n",
       "      <td>0.545092</td>\n",
       "      <td>-0.780767</td>\n",
       "      <td>0.324804</td>\n",
       "      <td>...</td>\n",
       "      <td>0.038397</td>\n",
       "      <td>0.116771</td>\n",
       "      <td>0.405560</td>\n",
       "      <td>-0.116453</td>\n",
       "      <td>0.541275</td>\n",
       "      <td>-0.216665</td>\n",
       "      <td>-0.415578</td>\n",
       "      <td>0.027126</td>\n",
       "      <td>-0.150347</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>-0.281976</td>\n",
       "      <td>-0.309699</td>\n",
       "      <td>-2.162299</td>\n",
       "      <td>-0.851514</td>\n",
       "      <td>0.106167</td>\n",
       "      <td>-1.483888</td>\n",
       "      <td>1.930994</td>\n",
       "      <td>-0.843049</td>\n",
       "      <td>-1.249272</td>\n",
       "      <td>1.079608</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.875516</td>\n",
       "      <td>-0.004199</td>\n",
       "      <td>1.015108</td>\n",
       "      <td>-0.026748</td>\n",
       "      <td>0.077115</td>\n",
       "      <td>-1.468822</td>\n",
       "      <td>0.751700</td>\n",
       "      <td>0.496732</td>\n",
       "      <td>0.331001</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 29 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "         V1        V2        V3        V4        V5        V6        V7  \\\n",
       "0 -1.848212  2.384900  0.379573  1.048381 -0.845070  2.537837 -4.542983   \n",
       "1  2.071805 -0.477943 -1.444444 -0.548657  0.010036 -0.582242 -0.042878   \n",
       "2 -2.985294 -2.747472  1.194068 -0.003036 -1.151041 -0.263559  0.553500   \n",
       "3 -1.479452  1.542874  0.290895  0.838142 -0.529290 -0.717661  0.484516   \n",
       "4 -0.281976 -0.309699 -2.162299 -0.851514  0.106167 -1.483888  1.930994   \n",
       "\n",
       "          V8        V9       V10  ...       V20       V21       V22       V23  \\\n",
       "0 -10.201458 -1.504967 -2.234167  ...  2.585817 -5.291690  0.859364  0.423231   \n",
       "1  -0.247160  1.171923 -0.342382  ... -0.077306  0.042858  0.390125  0.041569   \n",
       "2   0.635600  0.438545 -1.806488  ...  1.345776  0.373760 -0.385777  1.197596   \n",
       "3   0.545092 -0.780767  0.324804  ...  0.038397  0.116771  0.405560 -0.116453   \n",
       "4  -0.843049 -1.249272  1.079608  ... -0.875516 -0.004199  1.015108 -0.026748   \n",
       "\n",
       "        V24       V25       V26       V27       V28  Class  \n",
       "0 -0.506985  1.020052 -0.627751 -0.017753  0.280982      0  \n",
       "1  0.598427  0.098803  0.979686 -0.093244 -0.065615      0  \n",
       "2  0.407229  0.008013  0.762362 -0.299024 -0.303929      0  \n",
       "3  0.541275 -0.216665 -0.415578  0.027126 -0.150347      0  \n",
       "4  0.077115 -1.468822  0.751700  0.496732  0.331001      0  \n",
       "\n",
       "[5 rows x 29 columns]"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 28678 entries, 0 to 28677\n",
      "Data columns (total 29 columns):\n",
      " #   Column  Non-Null Count  Dtype  \n",
      "---  ------  --------------  -----  \n",
      " 0   V1      28678 non-null  float64\n",
      " 1   V2      28678 non-null  float64\n",
      " 2   V3      28678 non-null  float64\n",
      " 3   V4      28678 non-null  float64\n",
      " 4   V5      28678 non-null  float64\n",
      " 5   V6      28678 non-null  float64\n",
      " 6   V7      28678 non-null  float64\n",
      " 7   V8      28678 non-null  float64\n",
      " 8   V9      28678 non-null  float64\n",
      " 9   V10     28678 non-null  float64\n",
      " 10  V11     28678 non-null  float64\n",
      " 11  V12     28678 non-null  float64\n",
      " 12  V13     28678 non-null  float64\n",
      " 13  V14     28678 non-null  float64\n",
      " 14  V15     28678 non-null  float64\n",
      " 15  V16     28678 non-null  float64\n",
      " 16  V17     28678 non-null  float64\n",
      " 17  V18     28678 non-null  float64\n",
      " 18  V19     28678 non-null  float64\n",
      " 19  V20     28678 non-null  float64\n",
      " 20  V21     28678 non-null  float64\n",
      " 21  V22     28678 non-null  float64\n",
      " 22  V23     28678 non-null  float64\n",
      " 23  V24     28678 non-null  float64\n",
      " 24  V25     28678 non-null  float64\n",
      " 25  V26     28678 non-null  float64\n",
      " 26  V27     28678 non-null  float64\n",
      " 27  V28     28678 non-null  float64\n",
      " 28  Class   28678 non-null  int64  \n",
      "dtypes: float64(28), int64(1)\n",
      "memory usage: 6.3 MB\n"
     ]
    }
   ],
   "source": [
    "data.info()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "일단 먼저 target 함수인 Class에 대해서 수치형 변수를 카테고리형 변수로 바꿔준다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "data[\"Class\"] = data[\"Class\"].astype(\"category\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0        0\n",
       "1        0\n",
       "2        0\n",
       "3        0\n",
       "4        0\n",
       "        ..\n",
       "28673    0\n",
       "28674    0\n",
       "28675    0\n",
       "28676    0\n",
       "28677    0\n",
       "Name: Class, Length: 28678, dtype: category\n",
       "Categories (2, int64): [0, 1]"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data[\"Class\"] # target 변수 확인하기"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 데이터 나누기\n",
    "features = data.iloc[:,:-1] \n",
    "target = data[\"Class\"] "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>V1</th>\n",
       "      <th>V2</th>\n",
       "      <th>V3</th>\n",
       "      <th>V4</th>\n",
       "      <th>V5</th>\n",
       "      <th>V6</th>\n",
       "      <th>V7</th>\n",
       "      <th>V8</th>\n",
       "      <th>V9</th>\n",
       "      <th>V10</th>\n",
       "      <th>...</th>\n",
       "      <th>V19</th>\n",
       "      <th>V20</th>\n",
       "      <th>V21</th>\n",
       "      <th>V22</th>\n",
       "      <th>V23</th>\n",
       "      <th>V24</th>\n",
       "      <th>V25</th>\n",
       "      <th>V26</th>\n",
       "      <th>V27</th>\n",
       "      <th>V28</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>28678.000000</td>\n",
       "      <td>28678.000000</td>\n",
       "      <td>28678.000000</td>\n",
       "      <td>28678.000000</td>\n",
       "      <td>28678.000000</td>\n",
       "      <td>28678.000000</td>\n",
       "      <td>28678.000000</td>\n",
       "      <td>28678.000000</td>\n",
       "      <td>28678.000000</td>\n",
       "      <td>28678.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>28678.000000</td>\n",
       "      <td>28678.000000</td>\n",
       "      <td>28678.000000</td>\n",
       "      <td>28678.000000</td>\n",
       "      <td>28678.000000</td>\n",
       "      <td>28678.000000</td>\n",
       "      <td>28678.000000</td>\n",
       "      <td>28678.000000</td>\n",
       "      <td>28678.000000</td>\n",
       "      <td>28678.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>-0.015438</td>\n",
       "      <td>0.053653</td>\n",
       "      <td>-0.046031</td>\n",
       "      <td>0.037348</td>\n",
       "      <td>-0.033724</td>\n",
       "      <td>-0.003299</td>\n",
       "      <td>-0.051054</td>\n",
       "      <td>0.006064</td>\n",
       "      <td>-0.018530</td>\n",
       "      <td>-0.041149</td>\n",
       "      <td>...</td>\n",
       "      <td>0.007183</td>\n",
       "      <td>0.002633</td>\n",
       "      <td>0.010289</td>\n",
       "      <td>-0.000656</td>\n",
       "      <td>-0.004800</td>\n",
       "      <td>-0.000897</td>\n",
       "      <td>-0.001989</td>\n",
       "      <td>-0.000765</td>\n",
       "      <td>0.000948</td>\n",
       "      <td>0.001535</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>2.031529</td>\n",
       "      <td>1.616186</td>\n",
       "      <td>1.758169</td>\n",
       "      <td>1.482109</td>\n",
       "      <td>1.486998</td>\n",
       "      <td>1.339259</td>\n",
       "      <td>1.454827</td>\n",
       "      <td>1.364342</td>\n",
       "      <td>1.134065</td>\n",
       "      <td>1.252593</td>\n",
       "      <td>...</td>\n",
       "      <td>0.827420</td>\n",
       "      <td>0.720307</td>\n",
       "      <td>0.847152</td>\n",
       "      <td>0.739469</td>\n",
       "      <td>0.593663</td>\n",
       "      <td>0.603349</td>\n",
       "      <td>0.517968</td>\n",
       "      <td>0.483852</td>\n",
       "      <td>0.397075</td>\n",
       "      <td>0.296736</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>-30.552380</td>\n",
       "      <td>-42.172688</td>\n",
       "      <td>-31.103685</td>\n",
       "      <td>-5.560118</td>\n",
       "      <td>-42.147898</td>\n",
       "      <td>-21.929312</td>\n",
       "      <td>-41.506796</td>\n",
       "      <td>-39.267378</td>\n",
       "      <td>-13.434066</td>\n",
       "      <td>-24.403185</td>\n",
       "      <td>...</td>\n",
       "      <td>-4.619034</td>\n",
       "      <td>-21.387122</td>\n",
       "      <td>-21.453736</td>\n",
       "      <td>-8.887017</td>\n",
       "      <td>-36.666000</td>\n",
       "      <td>-2.718024</td>\n",
       "      <td>-6.712624</td>\n",
       "      <td>-2.241620</td>\n",
       "      <td>-7.418878</td>\n",
       "      <td>-9.617915</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>-0.916927</td>\n",
       "      <td>-0.575381</td>\n",
       "      <td>-0.899872</td>\n",
       "      <td>-0.843321</td>\n",
       "      <td>-0.714901</td>\n",
       "      <td>-0.763757</td>\n",
       "      <td>-0.568146</td>\n",
       "      <td>-0.206103</td>\n",
       "      <td>-0.661909</td>\n",
       "      <td>-0.543450</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.459073</td>\n",
       "      <td>-0.209678</td>\n",
       "      <td>-0.225520</td>\n",
       "      <td>-0.539244</td>\n",
       "      <td>-0.160583</td>\n",
       "      <td>-0.356047</td>\n",
       "      <td>-0.318619</td>\n",
       "      <td>-0.327343</td>\n",
       "      <td>-0.070558</td>\n",
       "      <td>-0.052189</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.020050</td>\n",
       "      <td>0.075358</td>\n",
       "      <td>0.180610</td>\n",
       "      <td>-0.008844</td>\n",
       "      <td>-0.060040</td>\n",
       "      <td>-0.271363</td>\n",
       "      <td>0.036107</td>\n",
       "      <td>0.022463</td>\n",
       "      <td>-0.055095</td>\n",
       "      <td>-0.097390</td>\n",
       "      <td>...</td>\n",
       "      <td>0.002111</td>\n",
       "      <td>-0.062792</td>\n",
       "      <td>-0.028778</td>\n",
       "      <td>0.007302</td>\n",
       "      <td>-0.011199</td>\n",
       "      <td>0.040006</td>\n",
       "      <td>0.019770</td>\n",
       "      <td>-0.056260</td>\n",
       "      <td>0.002049</td>\n",
       "      <td>0.011075</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>1.317461</td>\n",
       "      <td>0.806957</td>\n",
       "      <td>1.029928</td>\n",
       "      <td>0.771958</td>\n",
       "      <td>0.613328</td>\n",
       "      <td>0.397269</td>\n",
       "      <td>0.559409</td>\n",
       "      <td>0.329606</td>\n",
       "      <td>0.605704</td>\n",
       "      <td>0.460681</td>\n",
       "      <td>...</td>\n",
       "      <td>0.467781</td>\n",
       "      <td>0.131199</td>\n",
       "      <td>0.184312</td>\n",
       "      <td>0.526358</td>\n",
       "      <td>0.146835</td>\n",
       "      <td>0.437146</td>\n",
       "      <td>0.352717</td>\n",
       "      <td>0.240713</td>\n",
       "      <td>0.091637</td>\n",
       "      <td>0.078911</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>2.399484</td>\n",
       "      <td>21.467203</td>\n",
       "      <td>4.069865</td>\n",
       "      <td>11.927512</td>\n",
       "      <td>32.911462</td>\n",
       "      <td>22.529298</td>\n",
       "      <td>36.677268</td>\n",
       "      <td>20.007208</td>\n",
       "      <td>8.113152</td>\n",
       "      <td>15.236028</td>\n",
       "      <td>...</td>\n",
       "      <td>5.228342</td>\n",
       "      <td>26.237391</td>\n",
       "      <td>27.202839</td>\n",
       "      <td>8.361985</td>\n",
       "      <td>9.637187</td>\n",
       "      <td>3.948061</td>\n",
       "      <td>2.510401</td>\n",
       "      <td>3.122747</td>\n",
       "      <td>11.135740</td>\n",
       "      <td>14.929133</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>8 rows × 28 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                 V1            V2            V3            V4            V5  \\\n",
       "count  28678.000000  28678.000000  28678.000000  28678.000000  28678.000000   \n",
       "mean      -0.015438      0.053653     -0.046031      0.037348     -0.033724   \n",
       "std        2.031529      1.616186      1.758169      1.482109      1.486998   \n",
       "min      -30.552380    -42.172688    -31.103685     -5.560118    -42.147898   \n",
       "25%       -0.916927     -0.575381     -0.899872     -0.843321     -0.714901   \n",
       "50%        0.020050      0.075358      0.180610     -0.008844     -0.060040   \n",
       "75%        1.317461      0.806957      1.029928      0.771958      0.613328   \n",
       "max        2.399484     21.467203      4.069865     11.927512     32.911462   \n",
       "\n",
       "                 V6            V7            V8            V9           V10  \\\n",
       "count  28678.000000  28678.000000  28678.000000  28678.000000  28678.000000   \n",
       "mean      -0.003299     -0.051054      0.006064     -0.018530     -0.041149   \n",
       "std        1.339259      1.454827      1.364342      1.134065      1.252593   \n",
       "min      -21.929312    -41.506796    -39.267378    -13.434066    -24.403185   \n",
       "25%       -0.763757     -0.568146     -0.206103     -0.661909     -0.543450   \n",
       "50%       -0.271363      0.036107      0.022463     -0.055095     -0.097390   \n",
       "75%        0.397269      0.559409      0.329606      0.605704      0.460681   \n",
       "max       22.529298     36.677268     20.007208      8.113152     15.236028   \n",
       "\n",
       "       ...           V19           V20           V21           V22  \\\n",
       "count  ...  28678.000000  28678.000000  28678.000000  28678.000000   \n",
       "mean   ...      0.007183      0.002633      0.010289     -0.000656   \n",
       "std    ...      0.827420      0.720307      0.847152      0.739469   \n",
       "min    ...     -4.619034    -21.387122    -21.453736     -8.887017   \n",
       "25%    ...     -0.459073     -0.209678     -0.225520     -0.539244   \n",
       "50%    ...      0.002111     -0.062792     -0.028778      0.007302   \n",
       "75%    ...      0.467781      0.131199      0.184312      0.526358   \n",
       "max    ...      5.228342     26.237391     27.202839      8.361985   \n",
       "\n",
       "                V23           V24           V25           V26           V27  \\\n",
       "count  28678.000000  28678.000000  28678.000000  28678.000000  28678.000000   \n",
       "mean      -0.004800     -0.000897     -0.001989     -0.000765      0.000948   \n",
       "std        0.593663      0.603349      0.517968      0.483852      0.397075   \n",
       "min      -36.666000     -2.718024     -6.712624     -2.241620     -7.418878   \n",
       "25%       -0.160583     -0.356047     -0.318619     -0.327343     -0.070558   \n",
       "50%       -0.011199      0.040006      0.019770     -0.056260      0.002049   \n",
       "75%        0.146835      0.437146      0.352717      0.240713      0.091637   \n",
       "max        9.637187      3.948061      2.510401      3.122747     11.135740   \n",
       "\n",
       "                V28  \n",
       "count  28678.000000  \n",
       "mean       0.001535  \n",
       "std        0.296736  \n",
       "min       -9.617915  \n",
       "25%       -0.052189  \n",
       "50%        0.011075  \n",
       "75%        0.078911  \n",
       "max       14.929133  \n",
       "\n",
       "[8 rows x 28 columns]"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "features.describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "전체적으로 feature들의 통계량을 살펴봤을 때, 각 변수 별로 데이터 분포나 범위가 상이하고 또한 이상치도 많이 보이기 때문에 뒤에서 회귀모델에 조금 더 적합한 MinMaxScaler를 이용하여 Scaling을 진행할 예정이다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Class\n",
       "0    28432\n",
       "1      246\n",
       "Name: count, dtype: int64"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "target.value_counts() # y 값 데이터 수 파악"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "약 115:1 비율로 굉장히 불균형적인 데이터 형태로 볼 수 있다. 따라서 0으로만 계속 예측을 진행했을 때에도 `정확도(Accuracy)`는 높게 측정될 수 있다. 또한 우리는 신용카드 사기를 검출에 대해서 보는 것이기 때문에 평가지표 중 `정확도(Accuracy)`를 가지고 판단하기에는 어려움이 있을 것 같고, `재현율(Recall)` 혹은 `F1 score`를 가지고 판단할 예정이다."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### train data // test data 나누기"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "#로지스틱 회귀 import\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.metrics import *\n",
    "\n",
    "# train, test 분할\n",
    "X_train, X_test, y_train, y_test = train_test_split(features, target, test_size = 0.2, random_state = 42, stratify = target)\n",
    "# stratify를 이용하여 train data 와 test data에 균일하게 class를 분포하고자 한다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(22942, 28) (5736, 28) (22942,) (5736,)\n"
     ]
    }
   ],
   "source": [
    "print(X_train.shape, X_test.shape, y_train.shape, y_test.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "데이터가 잘 나눠졌음을 확인할 수 있다."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 앞서 X 변수들의 통계량을 다시 한 번 살펴봤을 때, 스케일링이 필요함을 확인했으므로 MinMaxSCaler를 적용시켜서 졍규화를 진행하고자 한다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>V1</th>\n",
       "      <th>V2</th>\n",
       "      <th>V3</th>\n",
       "      <th>V4</th>\n",
       "      <th>V5</th>\n",
       "      <th>V6</th>\n",
       "      <th>V7</th>\n",
       "      <th>V8</th>\n",
       "      <th>V9</th>\n",
       "      <th>V10</th>\n",
       "      <th>...</th>\n",
       "      <th>V19</th>\n",
       "      <th>V20</th>\n",
       "      <th>V21</th>\n",
       "      <th>V22</th>\n",
       "      <th>V23</th>\n",
       "      <th>V24</th>\n",
       "      <th>V25</th>\n",
       "      <th>V26</th>\n",
       "      <th>V27</th>\n",
       "      <th>V28</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>28678.000000</td>\n",
       "      <td>28678.000000</td>\n",
       "      <td>28678.000000</td>\n",
       "      <td>28678.000000</td>\n",
       "      <td>28678.000000</td>\n",
       "      <td>28678.000000</td>\n",
       "      <td>28678.000000</td>\n",
       "      <td>28678.000000</td>\n",
       "      <td>28678.000000</td>\n",
       "      <td>28678.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>28678.000000</td>\n",
       "      <td>28678.000000</td>\n",
       "      <td>28678.000000</td>\n",
       "      <td>28678.000000</td>\n",
       "      <td>28678.000000</td>\n",
       "      <td>28678.000000</td>\n",
       "      <td>28678.000000</td>\n",
       "      <td>28678.000000</td>\n",
       "      <td>28678.000000</td>\n",
       "      <td>28678.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>-0.015438</td>\n",
       "      <td>0.053653</td>\n",
       "      <td>-0.046031</td>\n",
       "      <td>0.037348</td>\n",
       "      <td>-0.033724</td>\n",
       "      <td>-0.003299</td>\n",
       "      <td>-0.051054</td>\n",
       "      <td>0.006064</td>\n",
       "      <td>-0.018530</td>\n",
       "      <td>-0.041149</td>\n",
       "      <td>...</td>\n",
       "      <td>0.007183</td>\n",
       "      <td>0.002633</td>\n",
       "      <td>0.010289</td>\n",
       "      <td>-0.000656</td>\n",
       "      <td>-0.004800</td>\n",
       "      <td>-0.000897</td>\n",
       "      <td>-0.001989</td>\n",
       "      <td>-0.000765</td>\n",
       "      <td>0.000948</td>\n",
       "      <td>0.001535</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>2.031529</td>\n",
       "      <td>1.616186</td>\n",
       "      <td>1.758169</td>\n",
       "      <td>1.482109</td>\n",
       "      <td>1.486998</td>\n",
       "      <td>1.339259</td>\n",
       "      <td>1.454827</td>\n",
       "      <td>1.364342</td>\n",
       "      <td>1.134065</td>\n",
       "      <td>1.252593</td>\n",
       "      <td>...</td>\n",
       "      <td>0.827420</td>\n",
       "      <td>0.720307</td>\n",
       "      <td>0.847152</td>\n",
       "      <td>0.739469</td>\n",
       "      <td>0.593663</td>\n",
       "      <td>0.603349</td>\n",
       "      <td>0.517968</td>\n",
       "      <td>0.483852</td>\n",
       "      <td>0.397075</td>\n",
       "      <td>0.296736</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>-30.552380</td>\n",
       "      <td>-42.172688</td>\n",
       "      <td>-31.103685</td>\n",
       "      <td>-5.560118</td>\n",
       "      <td>-42.147898</td>\n",
       "      <td>-21.929312</td>\n",
       "      <td>-41.506796</td>\n",
       "      <td>-39.267378</td>\n",
       "      <td>-13.434066</td>\n",
       "      <td>-24.403185</td>\n",
       "      <td>...</td>\n",
       "      <td>-4.619034</td>\n",
       "      <td>-21.387122</td>\n",
       "      <td>-21.453736</td>\n",
       "      <td>-8.887017</td>\n",
       "      <td>-36.666000</td>\n",
       "      <td>-2.718024</td>\n",
       "      <td>-6.712624</td>\n",
       "      <td>-2.241620</td>\n",
       "      <td>-7.418878</td>\n",
       "      <td>-9.617915</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>-0.916927</td>\n",
       "      <td>-0.575381</td>\n",
       "      <td>-0.899872</td>\n",
       "      <td>-0.843321</td>\n",
       "      <td>-0.714901</td>\n",
       "      <td>-0.763757</td>\n",
       "      <td>-0.568146</td>\n",
       "      <td>-0.206103</td>\n",
       "      <td>-0.661909</td>\n",
       "      <td>-0.543450</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.459073</td>\n",
       "      <td>-0.209678</td>\n",
       "      <td>-0.225520</td>\n",
       "      <td>-0.539244</td>\n",
       "      <td>-0.160583</td>\n",
       "      <td>-0.356047</td>\n",
       "      <td>-0.318619</td>\n",
       "      <td>-0.327343</td>\n",
       "      <td>-0.070558</td>\n",
       "      <td>-0.052189</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.020050</td>\n",
       "      <td>0.075358</td>\n",
       "      <td>0.180610</td>\n",
       "      <td>-0.008844</td>\n",
       "      <td>-0.060040</td>\n",
       "      <td>-0.271363</td>\n",
       "      <td>0.036107</td>\n",
       "      <td>0.022463</td>\n",
       "      <td>-0.055095</td>\n",
       "      <td>-0.097390</td>\n",
       "      <td>...</td>\n",
       "      <td>0.002111</td>\n",
       "      <td>-0.062792</td>\n",
       "      <td>-0.028778</td>\n",
       "      <td>0.007302</td>\n",
       "      <td>-0.011199</td>\n",
       "      <td>0.040006</td>\n",
       "      <td>0.019770</td>\n",
       "      <td>-0.056260</td>\n",
       "      <td>0.002049</td>\n",
       "      <td>0.011075</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>1.317461</td>\n",
       "      <td>0.806957</td>\n",
       "      <td>1.029928</td>\n",
       "      <td>0.771958</td>\n",
       "      <td>0.613328</td>\n",
       "      <td>0.397269</td>\n",
       "      <td>0.559409</td>\n",
       "      <td>0.329606</td>\n",
       "      <td>0.605704</td>\n",
       "      <td>0.460681</td>\n",
       "      <td>...</td>\n",
       "      <td>0.467781</td>\n",
       "      <td>0.131199</td>\n",
       "      <td>0.184312</td>\n",
       "      <td>0.526358</td>\n",
       "      <td>0.146835</td>\n",
       "      <td>0.437146</td>\n",
       "      <td>0.352717</td>\n",
       "      <td>0.240713</td>\n",
       "      <td>0.091637</td>\n",
       "      <td>0.078911</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>2.399484</td>\n",
       "      <td>21.467203</td>\n",
       "      <td>4.069865</td>\n",
       "      <td>11.927512</td>\n",
       "      <td>32.911462</td>\n",
       "      <td>22.529298</td>\n",
       "      <td>36.677268</td>\n",
       "      <td>20.007208</td>\n",
       "      <td>8.113152</td>\n",
       "      <td>15.236028</td>\n",
       "      <td>...</td>\n",
       "      <td>5.228342</td>\n",
       "      <td>26.237391</td>\n",
       "      <td>27.202839</td>\n",
       "      <td>8.361985</td>\n",
       "      <td>9.637187</td>\n",
       "      <td>3.948061</td>\n",
       "      <td>2.510401</td>\n",
       "      <td>3.122747</td>\n",
       "      <td>11.135740</td>\n",
       "      <td>14.929133</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>8 rows × 28 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                 V1            V2            V3            V4            V5  \\\n",
       "count  28678.000000  28678.000000  28678.000000  28678.000000  28678.000000   \n",
       "mean      -0.015438      0.053653     -0.046031      0.037348     -0.033724   \n",
       "std        2.031529      1.616186      1.758169      1.482109      1.486998   \n",
       "min      -30.552380    -42.172688    -31.103685     -5.560118    -42.147898   \n",
       "25%       -0.916927     -0.575381     -0.899872     -0.843321     -0.714901   \n",
       "50%        0.020050      0.075358      0.180610     -0.008844     -0.060040   \n",
       "75%        1.317461      0.806957      1.029928      0.771958      0.613328   \n",
       "max        2.399484     21.467203      4.069865     11.927512     32.911462   \n",
       "\n",
       "                 V6            V7            V8            V9           V10  \\\n",
       "count  28678.000000  28678.000000  28678.000000  28678.000000  28678.000000   \n",
       "mean      -0.003299     -0.051054      0.006064     -0.018530     -0.041149   \n",
       "std        1.339259      1.454827      1.364342      1.134065      1.252593   \n",
       "min      -21.929312    -41.506796    -39.267378    -13.434066    -24.403185   \n",
       "25%       -0.763757     -0.568146     -0.206103     -0.661909     -0.543450   \n",
       "50%       -0.271363      0.036107      0.022463     -0.055095     -0.097390   \n",
       "75%        0.397269      0.559409      0.329606      0.605704      0.460681   \n",
       "max       22.529298     36.677268     20.007208      8.113152     15.236028   \n",
       "\n",
       "       ...           V19           V20           V21           V22  \\\n",
       "count  ...  28678.000000  28678.000000  28678.000000  28678.000000   \n",
       "mean   ...      0.007183      0.002633      0.010289     -0.000656   \n",
       "std    ...      0.827420      0.720307      0.847152      0.739469   \n",
       "min    ...     -4.619034    -21.387122    -21.453736     -8.887017   \n",
       "25%    ...     -0.459073     -0.209678     -0.225520     -0.539244   \n",
       "50%    ...      0.002111     -0.062792     -0.028778      0.007302   \n",
       "75%    ...      0.467781      0.131199      0.184312      0.526358   \n",
       "max    ...      5.228342     26.237391     27.202839      8.361985   \n",
       "\n",
       "                V23           V24           V25           V26           V27  \\\n",
       "count  28678.000000  28678.000000  28678.000000  28678.000000  28678.000000   \n",
       "mean      -0.004800     -0.000897     -0.001989     -0.000765      0.000948   \n",
       "std        0.593663      0.603349      0.517968      0.483852      0.397075   \n",
       "min      -36.666000     -2.718024     -6.712624     -2.241620     -7.418878   \n",
       "25%       -0.160583     -0.356047     -0.318619     -0.327343     -0.070558   \n",
       "50%       -0.011199      0.040006      0.019770     -0.056260      0.002049   \n",
       "75%        0.146835      0.437146      0.352717      0.240713      0.091637   \n",
       "max        9.637187      3.948061      2.510401      3.122747     11.135740   \n",
       "\n",
       "                V28  \n",
       "count  28678.000000  \n",
       "mean       0.001535  \n",
       "std        0.296736  \n",
       "min       -9.617915  \n",
       "25%       -0.052189  \n",
       "50%        0.011075  \n",
       "75%        0.078911  \n",
       "max       14.929133  \n",
       "\n",
       "[8 rows x 28 columns]"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "features.describe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.preprocessing import MinMaxScaler\n",
    "\n",
    "scaler = MinMaxScaler()\n",
    "\n",
    "X_train = pd.DataFrame(scaler.fit_transform(X_train), columns = features.columns)\n",
    "X_test = pd.DataFrame(scaler.fit_transform(X_test), columns = features.columns)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>V1</th>\n",
       "      <th>V2</th>\n",
       "      <th>V3</th>\n",
       "      <th>V4</th>\n",
       "      <th>V5</th>\n",
       "      <th>V6</th>\n",
       "      <th>V7</th>\n",
       "      <th>V8</th>\n",
       "      <th>V9</th>\n",
       "      <th>V10</th>\n",
       "      <th>...</th>\n",
       "      <th>V19</th>\n",
       "      <th>V20</th>\n",
       "      <th>V21</th>\n",
       "      <th>V22</th>\n",
       "      <th>V23</th>\n",
       "      <th>V24</th>\n",
       "      <th>V25</th>\n",
       "      <th>V26</th>\n",
       "      <th>V27</th>\n",
       "      <th>V28</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>22942.000000</td>\n",
       "      <td>22942.000000</td>\n",
       "      <td>22942.000000</td>\n",
       "      <td>22942.000000</td>\n",
       "      <td>22942.000000</td>\n",
       "      <td>22942.000000</td>\n",
       "      <td>22942.000000</td>\n",
       "      <td>22942.000000</td>\n",
       "      <td>22942.000000</td>\n",
       "      <td>22942.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>22942.000000</td>\n",
       "      <td>22942.000000</td>\n",
       "      <td>22942.000000</td>\n",
       "      <td>22942.000000</td>\n",
       "      <td>22942.000000</td>\n",
       "      <td>22942.000000</td>\n",
       "      <td>22942.000000</td>\n",
       "      <td>22942.000000</td>\n",
       "      <td>22942.000000</td>\n",
       "      <td>22942.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.926541</td>\n",
       "      <td>0.639517</td>\n",
       "      <td>0.882866</td>\n",
       "      <td>0.320483</td>\n",
       "      <td>0.561021</td>\n",
       "      <td>0.493161</td>\n",
       "      <td>0.530282</td>\n",
       "      <td>0.662537</td>\n",
       "      <td>0.622577</td>\n",
       "      <td>0.614722</td>\n",
       "      <td>...</td>\n",
       "      <td>0.457297</td>\n",
       "      <td>0.474950</td>\n",
       "      <td>0.441076</td>\n",
       "      <td>0.514945</td>\n",
       "      <td>0.791743</td>\n",
       "      <td>0.407366</td>\n",
       "      <td>0.727433</td>\n",
       "      <td>0.417707</td>\n",
       "      <td>0.399902</td>\n",
       "      <td>0.391835</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.062362</td>\n",
       "      <td>0.027395</td>\n",
       "      <td>0.050275</td>\n",
       "      <td>0.084667</td>\n",
       "      <td>0.020138</td>\n",
       "      <td>0.030504</td>\n",
       "      <td>0.018796</td>\n",
       "      <td>0.023300</td>\n",
       "      <td>0.052581</td>\n",
       "      <td>0.031697</td>\n",
       "      <td>...</td>\n",
       "      <td>0.085944</td>\n",
       "      <td>0.016195</td>\n",
       "      <td>0.017085</td>\n",
       "      <td>0.042762</td>\n",
       "      <td>0.012978</td>\n",
       "      <td>0.090524</td>\n",
       "      <td>0.056167</td>\n",
       "      <td>0.089889</td>\n",
       "      <td>0.021823</td>\n",
       "      <td>0.012261</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.899391</td>\n",
       "      <td>0.628892</td>\n",
       "      <td>0.858895</td>\n",
       "      <td>0.270149</td>\n",
       "      <td>0.552030</td>\n",
       "      <td>0.476010</td>\n",
       "      <td>0.523594</td>\n",
       "      <td>0.658961</td>\n",
       "      <td>0.592634</td>\n",
       "      <td>0.601957</td>\n",
       "      <td>...</td>\n",
       "      <td>0.409075</td>\n",
       "      <td>0.470209</td>\n",
       "      <td>0.436257</td>\n",
       "      <td>0.483774</td>\n",
       "      <td>0.788415</td>\n",
       "      <td>0.354539</td>\n",
       "      <td>0.692969</td>\n",
       "      <td>0.356775</td>\n",
       "      <td>0.395999</td>\n",
       "      <td>0.389663</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.927690</td>\n",
       "      <td>0.639934</td>\n",
       "      <td>0.889353</td>\n",
       "      <td>0.317587</td>\n",
       "      <td>0.560789</td>\n",
       "      <td>0.487051</td>\n",
       "      <td>0.531404</td>\n",
       "      <td>0.662801</td>\n",
       "      <td>0.620845</td>\n",
       "      <td>0.613223</td>\n",
       "      <td>...</td>\n",
       "      <td>0.457009</td>\n",
       "      <td>0.473505</td>\n",
       "      <td>0.440295</td>\n",
       "      <td>0.515304</td>\n",
       "      <td>0.791641</td>\n",
       "      <td>0.413554</td>\n",
       "      <td>0.729692</td>\n",
       "      <td>0.407690</td>\n",
       "      <td>0.399940</td>\n",
       "      <td>0.392247</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>0.967217</td>\n",
       "      <td>0.652287</td>\n",
       "      <td>0.913567</td>\n",
       "      <td>0.362298</td>\n",
       "      <td>0.569751</td>\n",
       "      <td>0.502185</td>\n",
       "      <td>0.538133</td>\n",
       "      <td>0.668009</td>\n",
       "      <td>0.651235</td>\n",
       "      <td>0.627343</td>\n",
       "      <td>...</td>\n",
       "      <td>0.505209</td>\n",
       "      <td>0.477776</td>\n",
       "      <td>0.444684</td>\n",
       "      <td>0.545424</td>\n",
       "      <td>0.795040</td>\n",
       "      <td>0.472585</td>\n",
       "      <td>0.765856</td>\n",
       "      <td>0.462544</td>\n",
       "      <td>0.404784</td>\n",
       "      <td>0.395029</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>8 rows × 28 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                 V1            V2            V3            V4            V5  \\\n",
       "count  22942.000000  22942.000000  22942.000000  22942.000000  22942.000000   \n",
       "mean       0.926541      0.639517      0.882866      0.320483      0.561021   \n",
       "std        0.062362      0.027395      0.050275      0.084667      0.020138   \n",
       "min        0.000000      0.000000      0.000000      0.000000      0.000000   \n",
       "25%        0.899391      0.628892      0.858895      0.270149      0.552030   \n",
       "50%        0.927690      0.639934      0.889353      0.317587      0.560789   \n",
       "75%        0.967217      0.652287      0.913567      0.362298      0.569751   \n",
       "max        1.000000      1.000000      1.000000      1.000000      1.000000   \n",
       "\n",
       "                 V6            V7            V8            V9           V10  \\\n",
       "count  22942.000000  22942.000000  22942.000000  22942.000000  22942.000000   \n",
       "mean       0.493161      0.530282      0.662537      0.622577      0.614722   \n",
       "std        0.030504      0.018796      0.023300      0.052581      0.031697   \n",
       "min        0.000000      0.000000      0.000000      0.000000      0.000000   \n",
       "25%        0.476010      0.523594      0.658961      0.592634      0.601957   \n",
       "50%        0.487051      0.531404      0.662801      0.620845      0.613223   \n",
       "75%        0.502185      0.538133      0.668009      0.651235      0.627343   \n",
       "max        1.000000      1.000000      1.000000      1.000000      1.000000   \n",
       "\n",
       "       ...           V19           V20           V21           V22  \\\n",
       "count  ...  22942.000000  22942.000000  22942.000000  22942.000000   \n",
       "mean   ...      0.457297      0.474950      0.441076      0.514945   \n",
       "std    ...      0.085944      0.016195      0.017085      0.042762   \n",
       "min    ...      0.000000      0.000000      0.000000      0.000000   \n",
       "25%    ...      0.409075      0.470209      0.436257      0.483774   \n",
       "50%    ...      0.457009      0.473505      0.440295      0.515304   \n",
       "75%    ...      0.505209      0.477776      0.444684      0.545424   \n",
       "max    ...      1.000000      1.000000      1.000000      1.000000   \n",
       "\n",
       "                V23           V24           V25           V26           V27  \\\n",
       "count  22942.000000  22942.000000  22942.000000  22942.000000  22942.000000   \n",
       "mean       0.791743      0.407366      0.727433      0.417707      0.399902   \n",
       "std        0.012978      0.090524      0.056167      0.089889      0.021823   \n",
       "min        0.000000      0.000000      0.000000      0.000000      0.000000   \n",
       "25%        0.788415      0.354539      0.692969      0.356775      0.395999   \n",
       "50%        0.791641      0.413554      0.729692      0.407690      0.399940   \n",
       "75%        0.795040      0.472585      0.765856      0.462544      0.404784   \n",
       "max        1.000000      1.000000      1.000000      1.000000      1.000000   \n",
       "\n",
       "                V28  \n",
       "count  22942.000000  \n",
       "mean       0.391835  \n",
       "std        0.012261  \n",
       "min        0.000000  \n",
       "25%        0.389663  \n",
       "50%        0.392247  \n",
       "75%        0.395029  \n",
       "max        1.000000  \n",
       "\n",
       "[8 rows x 28 columns]"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_train.describe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>V1</th>\n",
       "      <th>V2</th>\n",
       "      <th>V3</th>\n",
       "      <th>V4</th>\n",
       "      <th>V5</th>\n",
       "      <th>V6</th>\n",
       "      <th>V7</th>\n",
       "      <th>V8</th>\n",
       "      <th>V9</th>\n",
       "      <th>V10</th>\n",
       "      <th>...</th>\n",
       "      <th>V19</th>\n",
       "      <th>V20</th>\n",
       "      <th>V21</th>\n",
       "      <th>V22</th>\n",
       "      <th>V23</th>\n",
       "      <th>V24</th>\n",
       "      <th>V25</th>\n",
       "      <th>V26</th>\n",
       "      <th>V27</th>\n",
       "      <th>V28</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>5736.000000</td>\n",
       "      <td>5736.000000</td>\n",
       "      <td>5736.000000</td>\n",
       "      <td>5736.000000</td>\n",
       "      <td>5736.000000</td>\n",
       "      <td>5736.000000</td>\n",
       "      <td>5736.000000</td>\n",
       "      <td>5736.000000</td>\n",
       "      <td>5736.000000</td>\n",
       "      <td>5736.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>5736.000000</td>\n",
       "      <td>5736.000000</td>\n",
       "      <td>5736.000000</td>\n",
       "      <td>5736.000000</td>\n",
       "      <td>5736.000000</td>\n",
       "      <td>5736.000000</td>\n",
       "      <td>5736.000000</td>\n",
       "      <td>5736.000000</td>\n",
       "      <td>5736.000000</td>\n",
       "      <td>5736.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.918028</td>\n",
       "      <td>0.719928</td>\n",
       "      <td>0.882839</td>\n",
       "      <td>0.273614</td>\n",
       "      <td>0.557526</td>\n",
       "      <td>0.598071</td>\n",
       "      <td>0.663657</td>\n",
       "      <td>0.795093</td>\n",
       "      <td>0.607623</td>\n",
       "      <td>0.697335</td>\n",
       "      <td>...</td>\n",
       "      <td>0.556424</td>\n",
       "      <td>0.310243</td>\n",
       "      <td>0.286869</td>\n",
       "      <td>0.650453</td>\n",
       "      <td>0.681945</td>\n",
       "      <td>0.421339</td>\n",
       "      <td>0.646583</td>\n",
       "      <td>0.341459</td>\n",
       "      <td>0.606529</td>\n",
       "      <td>0.287662</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.067122</td>\n",
       "      <td>0.026758</td>\n",
       "      <td>0.050277</td>\n",
       "      <td>0.091344</td>\n",
       "      <td>0.043181</td>\n",
       "      <td>0.081175</td>\n",
       "      <td>0.029728</td>\n",
       "      <td>0.027558</td>\n",
       "      <td>0.073228</td>\n",
       "      <td>0.038986</td>\n",
       "      <td>...</td>\n",
       "      <td>0.099799</td>\n",
       "      <td>0.017953</td>\n",
       "      <td>0.023817</td>\n",
       "      <td>0.054560</td>\n",
       "      <td>0.024052</td>\n",
       "      <td>0.097677</td>\n",
       "      <td>0.073552</td>\n",
       "      <td>0.106209</td>\n",
       "      <td>0.030401</td>\n",
       "      <td>0.020476</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.885770</td>\n",
       "      <td>0.709279</td>\n",
       "      <td>0.856729</td>\n",
       "      <td>0.219510</td>\n",
       "      <td>0.535580</td>\n",
       "      <td>0.549810</td>\n",
       "      <td>0.653136</td>\n",
       "      <td>0.790618</td>\n",
       "      <td>0.566744</td>\n",
       "      <td>0.681802</td>\n",
       "      <td>...</td>\n",
       "      <td>0.499500</td>\n",
       "      <td>0.304790</td>\n",
       "      <td>0.280519</td>\n",
       "      <td>0.610908</td>\n",
       "      <td>0.675028</td>\n",
       "      <td>0.361388</td>\n",
       "      <td>0.602614</td>\n",
       "      <td>0.270937</td>\n",
       "      <td>0.600877</td>\n",
       "      <td>0.283568</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.918946</td>\n",
       "      <td>0.720087</td>\n",
       "      <td>0.889299</td>\n",
       "      <td>0.271963</td>\n",
       "      <td>0.555683</td>\n",
       "      <td>0.581719</td>\n",
       "      <td>0.665477</td>\n",
       "      <td>0.795514</td>\n",
       "      <td>0.605417</td>\n",
       "      <td>0.696008</td>\n",
       "      <td>...</td>\n",
       "      <td>0.555218</td>\n",
       "      <td>0.308493</td>\n",
       "      <td>0.285696</td>\n",
       "      <td>0.651480</td>\n",
       "      <td>0.681370</td>\n",
       "      <td>0.427909</td>\n",
       "      <td>0.650225</td>\n",
       "      <td>0.328326</td>\n",
       "      <td>0.606746</td>\n",
       "      <td>0.288183</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>0.963184</td>\n",
       "      <td>0.732411</td>\n",
       "      <td>0.913896</td>\n",
       "      <td>0.319302</td>\n",
       "      <td>0.576642</td>\n",
       "      <td>0.623575</td>\n",
       "      <td>0.676420</td>\n",
       "      <td>0.801894</td>\n",
       "      <td>0.649352</td>\n",
       "      <td>0.713267</td>\n",
       "      <td>...</td>\n",
       "      <td>0.611677</td>\n",
       "      <td>0.313730</td>\n",
       "      <td>0.291278</td>\n",
       "      <td>0.688997</td>\n",
       "      <td>0.688210</td>\n",
       "      <td>0.494296</td>\n",
       "      <td>0.696832</td>\n",
       "      <td>0.394754</td>\n",
       "      <td>0.614130</td>\n",
       "      <td>0.293040</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>8 rows × 28 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                V1           V2           V3           V4           V5  \\\n",
       "count  5736.000000  5736.000000  5736.000000  5736.000000  5736.000000   \n",
       "mean      0.918028     0.719928     0.882839     0.273614     0.557526   \n",
       "std       0.067122     0.026758     0.050277     0.091344     0.043181   \n",
       "min       0.000000     0.000000     0.000000     0.000000     0.000000   \n",
       "25%       0.885770     0.709279     0.856729     0.219510     0.535580   \n",
       "50%       0.918946     0.720087     0.889299     0.271963     0.555683   \n",
       "75%       0.963184     0.732411     0.913896     0.319302     0.576642   \n",
       "max       1.000000     1.000000     1.000000     1.000000     1.000000   \n",
       "\n",
       "                V6           V7           V8           V9          V10  ...  \\\n",
       "count  5736.000000  5736.000000  5736.000000  5736.000000  5736.000000  ...   \n",
       "mean      0.598071     0.663657     0.795093     0.607623     0.697335  ...   \n",
       "std       0.081175     0.029728     0.027558     0.073228     0.038986  ...   \n",
       "min       0.000000     0.000000     0.000000     0.000000     0.000000  ...   \n",
       "25%       0.549810     0.653136     0.790618     0.566744     0.681802  ...   \n",
       "50%       0.581719     0.665477     0.795514     0.605417     0.696008  ...   \n",
       "75%       0.623575     0.676420     0.801894     0.649352     0.713267  ...   \n",
       "max       1.000000     1.000000     1.000000     1.000000     1.000000  ...   \n",
       "\n",
       "               V19          V20          V21          V22          V23  \\\n",
       "count  5736.000000  5736.000000  5736.000000  5736.000000  5736.000000   \n",
       "mean      0.556424     0.310243     0.286869     0.650453     0.681945   \n",
       "std       0.099799     0.017953     0.023817     0.054560     0.024052   \n",
       "min       0.000000     0.000000     0.000000     0.000000     0.000000   \n",
       "25%       0.499500     0.304790     0.280519     0.610908     0.675028   \n",
       "50%       0.555218     0.308493     0.285696     0.651480     0.681370   \n",
       "75%       0.611677     0.313730     0.291278     0.688997     0.688210   \n",
       "max       1.000000     1.000000     1.000000     1.000000     1.000000   \n",
       "\n",
       "               V24          V25          V26          V27          V28  \n",
       "count  5736.000000  5736.000000  5736.000000  5736.000000  5736.000000  \n",
       "mean      0.421339     0.646583     0.341459     0.606529     0.287662  \n",
       "std       0.097677     0.073552     0.106209     0.030401     0.020476  \n",
       "min       0.000000     0.000000     0.000000     0.000000     0.000000  \n",
       "25%       0.361388     0.602614     0.270937     0.600877     0.283568  \n",
       "50%       0.427909     0.650225     0.328326     0.606746     0.288183  \n",
       "75%       0.494296     0.696832     0.394754     0.614130     0.293040  \n",
       "max       1.000000     1.000000     1.000000     1.000000     1.000000  \n",
       "\n",
       "[8 rows x 28 columns]"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_test.describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Scaling이 잘 이뤄진 것을 볼 수 있기 때문에 다음부터 로지스틱 회귀분석을 진행하고자 한다."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 로지스틱 회귀분석 진행"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.linear_model import LogisticRegression\n",
    "\n",
    "model = LogisticRegression() # 로지스틱 회귀분석을 통한 모델 생성\n",
    "model.fit(X_train, y_train) # 학습\n",
    "\n",
    "y_pred = model.predict(X_test) # 에측값 설정"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0, 0, 0, ..., 0, 0, 0], dtype=int64)"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y_pred"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "pred_proba = model.predict_proba(X_test)[:, -1].reshape(-1,1)\n",
    "# AUC를 구하기 위해서 확률 따로 설정"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 성능 평가 지표 (Confusion Matrix)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.metrics import *"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       1.00      1.00      1.00      5706\n",
      "           1       0.61      1.00      0.76        30\n",
      "\n",
      "    accuracy                           1.00      5736\n",
      "   macro avg       0.81      1.00      0.88      5736\n",
      "weighted avg       1.00      1.00      1.00      5736\n",
      "\n"
     ]
    }
   ],
   "source": [
    "print(classification_report(y_pred, y_test))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "위 자료를 살펴봤을 때 모델 성능이 굉장히 좋은 것으로 확인할 수 있다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "TN, FP, FN, TP = confusion_matrix(y_test, y_pred).ravel()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Accuracy : 예측한 값과 실제 값이 같은 경우"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.9966875871687587\n"
     ]
    }
   ],
   "source": [
    "accuracy = (TP + TN) / (TP + TN + FP + FN)\n",
    "print(accuracy)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "사기를 사기라고 맞게 분류한 경우 + 사기 X를 사기 X라고 맞게 분류한 경우"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Precision(정밀도) : Positive(True) 라고 예측한 것 중에서 실제 값도 True인 경우"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1.0\n"
     ]
    }
   ],
   "source": [
    "precision = TP / (TP + FP)\n",
    "print(precision)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "잘못 예측한 값은 하나도 없다."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Recall(재현율) : 실제로 True인 것들 중에서 모델이 Positive(True)라고 잘 예측한 경우"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.6122448979591837\n"
     ]
    }
   ],
   "source": [
    "recall = TP / (TP + FN)\n",
    "print(recall)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "재현율은 다른 지표들에 비해 다소 낮은 것을 확인할 수 있다."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* F1 Score : Precison(정밀도)과 Recall의 조화 평균"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.759493670886076\n"
     ]
    }
   ],
   "source": [
    "f1_score = (2*(precision*recall))/(precision + recall)\n",
    "print(f1_score)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Specificity(특이도) : 실제 사기 x 인 사람 중에서 올바르게 분류된 비율 (negative로 판단한 것 중에서 실제 negative인 비율)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1.0\n"
     ]
    }
   ],
   "source": [
    "specificity = TN / (TN + FP)\n",
    "print(specificity)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### ROC 커브 "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "auc : 0.9768214653542092\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAArEAAAHnCAYAAACv5QVaAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8pXeV/AAAACXBIWXMAAA9hAAAPYQGoP6dpAABrm0lEQVR4nO3dd1hTVwMG8DcJew8Bt3WvWtxbFPeoOOpqrbWOOnCg1ll31YpWcaE42trhaGu1WIt1gtoq4kKlDsSNo4Kyl5Dkfn/4mZqKEELCzYX39zzf85WbS/LKMfhyOPdcmSAIAoiIiIiIJEQudgAiIiIiooJiiSUiIiIiyWGJJSIiIiLJYYklIiIiIslhiSUiIiIiyWGJJSIiIiLJYYklIiIiIslhiSUiIiIiyTETO0BRi49PLbLXkstlcHGxRUJCOtRq3lNCijiG0sbxkz6OofRxDKVNrPFzc7PP9xzOxBqRXC6DTCaDXC4TOwrpiWMobRw/6eMYSh/HUNpMefxYYomIiIhIclhiiYiIiEhyWGKJiIiISHJYYomIiIhIclhiiYiIiEhyWGKJiIiISHJYYomIiIhIclhiiYiIiEhyWGKJiIiISHJYYomIiIhIclhiiYiIiEhyWGKJiIiISHLMxA5ARMWPIAi4EZuEhJQs3HmcCrUgoLSLDdo3LAe5vOA/O6vVaoReeIgniZnwcLbW+3kMSRAEXLv7DH+ceYD0zBxULeeIQe2rQqFQiJbJFL9OhlTc/3xUdPL7u1RS/q4JgoDo+4m4ei8Rz5KyIJPJ4OpohdpvOaNmBScAwPV7ici5mwhzGVC1rANkMpm4oV8hEwRBEDsE8OILuXfvXuzcuRM//fRTrudcvXoVCxYsQHx8PKytrTF79my0atWqQK8TH59qiLg6MTOTw9nZFomJ6VAq1UX2umQ4HMOCOx8dj11hNxGXlPnaYwq5DJ2bVEB/72o6P9+usJs4dDYWKvW/36p0fR5jjd/56Hhs2XcF2bk8Z4PqpTDhvXcM9lq6KszXyZS9HMONv1zEgYj7xe7PVxKY2vfR/N4rxfW99F/no+Ox7WA0kjOyc33c2tIM5mZypKT/+7i7kzX6e1dDo5puRs/n5maf7zkmUWJPnDiBL7/8EllZWVAoFDhw4MBr56SlpaF79+7w9/dHy5YtcebMGfj6+uKPP/6Am5vuX0yWWCoIjmHBnI+Ox4bgKOT3XaVbs4o6/WOwK+wm/oi4r/fzGGP8zkfHY/2vUXmeU9RFtrBfJ1NmZibH3pN3sTvs5hvPkfKfryQwpe+j+b1XKpexx53Hb+4JxeXvmi7fx95EJgN8e9czepHVpcSaxHKCzMxMTJ06FVZWVpg/f36u54SEhKBevXpo2bIlAKBp06Zo3Lgx9u/fj6FDhxZlXCLKhSAI2BV2M98CCwAHz8SifnXXPH89p1arcfDMm/+x0eV5zBRy2Kc8R2pqFpSqwv/jKQgCfjh4Ld/zImOe4kZsQpEsLTDE18mUyWXAr8dv5XmOlP98JYGh34f60uW9kleBBYrH3zVBELD9cHQhPh/YdewmGtYoJfrSApMosV26dAEAREREvPGcixcvomHDhlrHPD09cf369QK9llwug1xeNF90hUKu9f8kPRxD3V2/l5jrEoLcqAUBS7dFFvo1DfU8xuC//aLYETRM+etkCMX9z0emg3/XXohLzMTtxymoWdFZ1BwmUWJ1ERcXh2bNmmkdc3V1xaVLlwr0PC4utkX+k4ODg3WRvh4ZHscwfzl3E8WOQERERSRHLYOzs62oGSRTYlUqVa7HClpIExLSi3Qm1sHBGikpmVCJ+CsU0h/HUHfmBXxbdWtWEU3reLzx8YirT3Agj7VrujyPXC6Dra0l0tOfQ60u/PL/e/+k4ts/dPvtT7Pa7ujavFKhXzM/hvg6mbIz157gj9PF989XEhj6fagvXd8r+ZH63zVdvo8lPbkJR/eqeXYsc7mAxMR0Q8fT0KUgS6bEOjo6IjFRe6YnISGhQBd1AYBaLRT5m0ilUou+mJ0Kh2OYv6plHeDuZK3TkgKFXIb32lbJc11ZBTdbHP7PFcIFfR5DX1BS0d0O+/66jWepuV/N+6qR79YukjWxhvg6mbLKZexx6Ezx/fOVBKZyYZcu75X8FIe/axXd7RBy6i7ik7NeeyzneQauHv8GsX8fQYPuU1Culleuz+HubI0qZRxE/3dRMqNQt25dREZqr0OJjIxE/fr1xQlEVIK93Fsw4uoTRN9PhCAIkMlk6O9dDbr8cqRzkwr5/iMgl8vRuUmFQj+PIclkMgzqWDPf8xpUL1Vk+8Wa4tfJkORyOXq3rZrnOVL+81HR0eW9UrlM3lfEF4e/azKZDAPaV3/t+NPYKJz4wQ+xfx8BAMRE7IIgvF5SZTKgf7tqol/UBUhoJtbHxwdbtmxBeHg4WrRogePHj+P27dvo2rWr2NGISpTc9oF9de/Ark0r4uCZWKhz2aagoHstvjzPlPZsbFTTDeP61DOpfWJN8etkSB+/WxdZWTncJ5YKTZf3SknYJ/bl97Fth6KRkJSK6ye34c6FfZrHy9ZsgyZdx8LOzgrJr+4T62yN/u2KZp9YXZjEPrEvRUREYP78+Zp9Yvfu3YuoqCjMmTMHAPDnn39iyZIlSE5ORqVKlbBw4ULUrJn/rMiruE8sFQTHUFte+8DKZEDXphVx4Mz9N26zNbZXXTSpXfC1ZC/vnhOXlAkPZxt4Nyir02yIMcfv5R27Dpx5gIysHFQp64iBJnLHroJ+nUzZq2OYna0sdn++ksAUv4/m914pju+l3Fy4cA6jxnyC+3dfbGNnbeuAIaNnY8gHg1Dj/3fsuvUoBUpBBnO5gCpliu6OXZK52UFRYomlguAY/ksQBMzadDrPNa9yGZDXcjN3Z2ssHdW8yL4Jcvykj2MofRxD07Rt23eYNm2S5sL5Tp26ICBgHTw8SmudJ9b46VJii9+PFURkFDdik/K9aCu/6yXiEjMR8yDZgKmIiEgfTZo0g5mZGWxt7bBqVSC2bfv5tQJr6iSzJpaIxJWUlv8V+bo9z3ODPA8REelOpVJBrVbD3NwcAFCzZi1s2LAFnp4NULGi8bcDNAbOxBKRTpzsLAz0PJYGeR4iItLN7du30KtXNyxbtkTreM+evSVbYAGWWCLSUY0KTnB3yvvOZYp8biTi7myN6uUdDRmLiIjeQBAEbN36Fdq3b4UzZ04jMHA1Ll0qPrfNZYklKkK57a9aFJRKJdbvicK8ryOwfk8UlEplgZ8jv31gZbIXeyjm9bip7C1IRFTcPXr0EAMG9MaMGVOQkZEBMzMzTJ/+GerWrSd2NIPhmliiIpLf/qrGsui7s7jz+N9dOR7Ep2PUinhULmOPuUObFOi5GtV0w1ul7bWe76W3Stujv3c1VCnriF3HbiIu8ZU/p4ntLUhEVFwJgoBdu37EZ59NR0rKiwtpa9eug8DATahXz1PkdIbFLbaMiNuKSJ+hxjC//VV9e9czSsH7b4H9r4IW2XW7LyMy5ukbH3+50b8gCLgRm4Tk9Gw42VmienlHUWZg+R6UPo6h9HEMi05CwjNMmTIR+/e/uHGBTCbDuHF+mDFjNiwt9bsewZS32OJMLJGRCYKAXWE333gDAEEAdhy5AUc7c4MWPaVSmWeBBYA7j1MRff8ZzMzy/1agUqnyLLAAEBnzFCqVCgqFAjUrOhcoLxERFY5CocDFixcAAG+9VRnr1m1Cs2bNRU5lPCyxREamy/6qianP8cUPF4ookbZlOy4Z9Pl+DruF9zvWMOhzEhFR/hwdnbB69Xrs378P8+Ytgp2dndiRjIoXdhEZmaH2V5WKxwkZYkcgIioRjh8Pw5QpE7QuEm7Xrj2WL19V7AsswJlYIqPTdX/VIV1qoKJH/muAdPXLsZuIvp//3bFqVXLEe22r5XvewTP3ce56fL7nlXGx0SkfERHpJz09HYsWzcM332wBADRo0AhDhnwsbigRsMQSGdnL/VXzWlLg7myNdvXLGXRN7KcDPDFqxYl8z5vS31OnNbGje9bBuevH8z1vgHdVnfIREVHBnTkTgQkTRuPOndsAgFKlSsHNzV3kVOLgcgLSmVh7nL6kVqtx5Fwsth++gSPnYqFWS+MqV132VzXG/qlmZmaoXCbvmd3KZex1KrDAiwsGGlQvlec5DaqXgkKh0DkjERHp5vnz51i0aD58fLpoCmz37j1x/HgEunbtLnI6cXAmlnQi1h6nL+0Ku4lDZ2OhUv9bnH8KvYnOTSqgv3f+vwoXW6OabujatCIOnomF+pXyr5DL0LlJBaN9DecObfLGbbb02Sd2wnvvvHGbrZfbaxERkWFFRV3G+PGjcO3aVQCAg4MjvvhiOfr3H1SibyDDfWKNqLjsjSfWHqcv7Qq7iT8i7r/x8W7NKhqtyEp9n9iXlEolNv12DXFJGfBwtsGonrV1noHNjUqlwk+ht/BPYgbKuNhggHdVk5yBLS7vwZKMYyh9HMPCe+89H/z55zEAQNu23li9ej3KlStfJK/NfWJJssTa4/QltVqNg2feXGAB4OCZWNSv7gq53PCrY8wUctinPEdqahaUKv3evIIgYMeRG3l+DXcdu4mGNUoZ7SdqMzMzjOtruFsNKhQKfNCJ22gRERWFlSvXoHv3jpg6dSaGDRtZomdfX8USS3ky9T1OAUAtCFi6LVK01zeEuMRMxDxIRo0KTmJHISIiEanVauzcuQ29e78HW1tbAC9uXHD+/N+wtrYWOZ1p4YVdlKeStsepmJLSnosdgYiIRBQbex/9+vlg8uTxWLx4vtZjLLCv40ws5UmsPU5fOnvtCQ6dfZDveZ2blkeTWh4Gf30zhRz29laFWk5w/0kqfjh4I9/znOz0u681ERFJmyAI2LHjB8ydOwtpaS+u3TlzJgJZWVmwsrISOZ3pYomlPIm1x+lLlUvb4+j5h1q7EvyXQi7DgHbVjLMm1gAL2quUccDBiNh8v4bVyzvqG5OIiCTqyZN/8OmnE3Ho0AEAgFwuh5/fFHz66UxYWOg2kVRScTmBxBl771ax9jh9SS6Xo3OTCnme07lJBaMUWEMR+2tIRESmae/ePfDyaqYpsNWqVUdIyGHMmjWPBVYHnImVsKLau1WsPU5ferl91n/3iX35+lLZJ9a3dz3sOnYTcYmvjJezNfq3K5q9domIyHTs3LkNfn6+mo9HjRqLzz6bDxsb3rpbV9wn1oiMubdaUe47KvYepy+p1WqEXniIuKRMeDjbwLtBWaPPwBp6DAVBwI3YJCSnZ8PJzhLVyztyBtaIuD+l9HEMpY9jmLv09HR4e7eESqXCmjUb0Lq1l9iRcsV9YsmginLvVlPY4/QluVyOjo3zXlpg6mQyGWpWdBY7BhERFbG0tFSkp6fDw6M0AMDW1hbbtv2MMmXKwN7eQeR00sQSK0Gmtncr9zglIiJ6s/Dwk5gwYSzKly+PPXt+1/wWsUaNmiInkzbTvRqG3sgU927lHqdERETaMjMzMXfuLPTu3R3379/FqVN/4fDhg2LHKjY4EytBRbl3K/c4JSIiKriLFy9g/PjRuHEjGgDg5OQEf/+V6Ny5q8jJig+WWAkqyr1buccpERGR7nJychAQsByrV6+ASqUCAHTo0AmrVgWidOkyIqcrXricQIKKct9R7nFKRESkm4SEZ+jWrQNWrlwGlUoFW1s7rFy5Fjt2/MICawQssRJmbf76RLpcBnRtWtHg+8T69q4Hd2ft+za7O1sX2fZaREREps7Z2QWurq4AgBYtWuHYsVMYMuRjTvQYCZcTSND56His/zUq18fUAnAg4j6qlHU0eJFtWKMU9zglIiJ6hUqlgkKhAPDit5erV6/Hvn3BGDlyjEnfTbI44FdXYgRBwM+hMXmfgxd7txrjFrQ1KzqjaW0P1KjgxAJLREQlliAI+Pbbr9GuXQukpCRrjpcpUxajRvmywBYBfoUl5kZsEuKTs/I97+XerURERGRYjx8/wqBBfTF9+mRER1/H/PmzxY5UIrHESkxB9ojl3q1ERESGIwgCdu36EV5ezREWdhQAUKtWbQwbNlLkZCUT18RKjK57xL44l3u3EhERGcLTp08xbdokhIT8BuDFEjtf34mYMWM2rKysRE5XMrHESkyNCk5wc7TKd0kB924lIiIyjD/+CMGnn07E06fxAIBKld7CunWb0Lx5C5GTlWxcTmDiBEFA9P1ERFx9guj7iRAEAa3qlc7zc2Tg3q1ERESGEhERrimwQ4eOQFjYKRZYE8CZWBN2Pjoeu8Juat0tSy57sY3Wm5gpZBjt8zb3biUiIjKQmTPn4OrVvzFmzHi0b99R7Dj0f5yJNVHno+OxITjqtdu95lVgAUCpEnDq78dGTEZERFR8paenY86cGYiJuaE5ZmVlhZ9/DmaBNTGciTVBgiBgV9hN6LvNa2TMU63Nl4mIiCh/Z89GYPz40bhz5zbOnTuD338/DDMzViVTxZlYE3QjNum1GdiC+jnsloHSEBERFW/Pnz/H4sUL0LNnF9y5cxsA4O5eGhkZ6SIno7zwxwsTVJC9YN/kcUKGAZIQEREVb1FRlzF+/Ghcu3YFAGBv74AlS5Zh4MAPeIG0iWOJNUEF2Qv2Tcq42BggCRERUfGkVCqxbt0qrFjhj5ycHABAmzbtsGbNepQvX0HkdKQLllgjEgQBf996itjHybC3NkeNCk46/VRXo4IT3J2sC7WkYIB3Vb0/l4iIqLg7evQwli5dBACwtrbGvHmLMGzYSMjlXGkpFSyxRnI+Oh67jt1EXOK/RdTdyRr9vavlu/2VTCZDf+9q2PBrFPS5tqtB9VK8qIuIiCgPnTt3RffuPREX9wSBgRtRpUo1sSNRAbHEGsHL7bH+u7tAXFImNgRHwbd3PZ32cdW3wE547x09PpOIiKj4io29j4cPH2puUiCTybBuXRBsbGw58SNRLLEGlt/2WIIA7DhyA4525m9cWiAIAnYcuZHrYy/ZWCngXb8s/knIwvMcJcq42mKAd1W+EYmIiF4hCAJ+/HE7Zs+eAWtra5w4EQFXV1cALy7iIuliiTUwXbbHSkx9ji9+uFCo18nIUqFeVTe8186pUM9DRERUXD158gRTp07EwYN/AAAyMtLx55/H0Lv3e6LmIsPg6mUDM8T2WLq/1vMiey0iIiIp2bcvGG3bNtMU2KpVq+H33w+xwBYjnIk1MF23xxrSpQYqetjn+tj9J6n44WDeywlevJZlgbIREREVd4mJCZg1axr27NmlOfbJJ2Mwe/YC2Nhw+8nihCXWwHTZHsvd2Rrt6pd745rYKmUccDAiNt/nqF7esdB5iYiIiovMzEx07OiF2Nj7AIDy5StgzZoNaNOmrcjJyBi4nMDAXm6P9abtYGUyoH+7annuF2uI5yAiIipprK2tMWjQYADA++9/iGPHTrHAFmMssUbQqKYb6lcrletj9auV0ml7rUY13eDbux7cna21jrs7W+u8RRcREVFx988/j7U+njRpKn755TesWbMBDg78jWVxxuUERrAr7CYiY57m+lhkzFPsCruJ/t75b6rcqKYbGtYohRuxSUhOz4aTnSWql3fkDCwREZV4WVlZ+OKLz7F16xbs338U9eq92CPd3NwcXl7txA1HRYIzsQamVqtx6GxsnuccOhsLtVqt0/PJZDLUrOiMprU9dL5tLRERUXF28eIFdOzYBhs3BuL58+eYM2eG2JFIBCyxBhZ64SFU6rzvtaVSCwiLfFREiYiIiIqHnJwcLF/+Bbp164AbN6IBAO3bd8SmTd+InIzEwOUEBvYkMe8bHfx7XoaRkxARERUf0dHXMX78aFy6FAkAsLGxxeeff4EhQz7mbylLKJZYA/P4z4VYbz6Pe9URERHp4vvvt2L27Ol4/vzFTX5atGiFNWs24K23KoucjMTE5QQG1r5hOSjkef9EqJDL4N2gbBElIiIikjZX11J4/vw5LC0tsXDhF/j11xAWWOJMrKHJ5XJ0blIBf0Tcf+M5nZtUgFzOnx+IiIhyIwiC1hKBHj16YtasuejevSdq1qwlYjIyJWxSRtDfuxoql8n9lrKVy9jrtL0WERFRSfT48SO8//57+PXXX7SOT548jQWWtLDEGsGusJu48zg118fuPE7FrrCbRZyIiIjItAmCgN27f4aXV3OEhh7BjBlTXruRAdGrWGINzND7xBIRERV3z549w8iRQzF27EgkJydBJpPh/feHwMnJWexoZMK4JtbACrJPbIdG5YsoFRERkWk6ePAPTJkyAfHxcQCAihXfwrp1QWjRopXIycjUcSbWwLhPLBERUf5SU1Pg5+eLIUMGagrskCHDcOzYSRZY0glnYg2M+8QSERHl79GjR9izZxcAwMOjNFavDkSHDp1FTkVSYhIzsVlZWZg7dy68vb3h5eWF5cuXQxBe/5X8kSNH0KNHD7Rr1w79+vXDuXPnREibN+4TS0RElL+aNWth5sy56Nu3H06cOM0CSwVmEiXW398farUahw8fRkhICCIiIrBt2zatc2JjYzF9+nT4+/vj2LFjmDx5Mnx9fZGamvsuAGJ5uU9sXrhPLBERlTRnz57Bli1BWsd8fSdg48Zv4OzsIlIqkjLRm1R6ejqCg4Mxbdo0mJmZwd7eHqNHj8bu3bu1zrtx4wbeeust1KtXDwDQqlUrWFlZ4e7duyKkzhv3iSUiInohOzsbs2fPRrduHTF37iycOROheezVGxoQFZToa2KvXLmC8uXLw8nJSXPM09MTMTExUKlUUCgUAIDGjRsjISEBJ0+eRKtWrfD777/D0dERNWvWLNDryeUyyPP5dX9h/XQ0Js99Yncfv4WBHaobNQMZhkIh1/p/khaOn/RxDKXtypW/MXbsJ/j77ygAgJ2dPZ49i4OZGcdTKkz5PSh6iY2Pj4erq6vWMRcXFyiVSqSmpmrKraOjI6ZPn47hw4fDxsYGOTk52L59OywsLAr0ei4utkb9yU+tVuNAHrecBYADEfcxqu87XFIgIQ4Oul2wR6aJ4yd9HENpUSqV+PLLLzF//nzk5OQAALy9vbF161ZUqlRJ5HSkD1N8D4peYpVK5WsXcb28EcCrZfPy5ctYtWoVgoODUatWLYSHh2PChAnYsWMHypfXfb/VhIR0o87EHjpzX6d9Yn85Eo1OTSoaLQcZhkIhh4ODNVJSMqFS8QYVUsPxkz6OofTcunUTY8eOwrlzZwAA1tbWWLZsGYYMGQ5BABIT00VOSAUh1nvQ2dk233NEL7FOTk5ITEzUOpaQkABLS0vY2/+7rvS7777D4MGDUbt2bQBAy5Yt0alTJ+zatQuTJ0/W+fXUagHqfEpmYTx+ptv+r4+fZUCp5DdkqVCp1BwvCeP4SR/HUBpUKhUGDeqHW7de3F69UaPGCAragiZN6iMxMZ1jKGGm+B4U/ffZderUwZ07d5CcnKw5FhkZCU9PT61ft+fk5GjWx75kZmaG7OzsIsuqC+4TS0REJZVCocAXX3wJCwsLfPbZPOzbdwjVqvEaEDIO0Uusm5sb2rRpg4CAACiVSiQkJCAoKAhDhw7VOq9r167Ytm0bHj16BAC4du0agoOD0alTJzFivxH3iSUiopJCEASEhh7RWhbo7d0BZ89exqRJU2FmJvovfKkYE73EAsCSJUsQFxeH1q1bo1+/fhg4cCA6duyIvXv3YvHixQCA7t27Y+TIkRg5ciS8vb0xc+ZMLFq0CA0bNhQ5vTbuE0tERCVBXFwchg59H4MG9cX27d9rPVamDCdqyPhkQm63xirG4uONf3OE89Hx2LLvCrJzWTvSoHopTHjvHaNnIMMwM5PD2dmWa7kkiuMnfRxD07Rv315Mnz4Jz549AwC88059HDp0LNcJGo6htIk1fm5uue+3/ypOBxrY+eh4rP81KtcCCwAXY57ifHR8EaciIiIqvKSkRIwdOxIjRgzRFNgRI0Zh794/+BtGKnJcrGJAgiDg59CYvM8BsOvYTTSsUYp3KiEiIskIDT2CyZPH4/HjF9emlCtXHqtXr0fbtt4iJ6OSij82GdCN2CTEJ2fle15cYiZiHiTnex4REZEp2LIlCIMG9dUU2IEDP8Dx4+EssCQqllgDSkrTfbuvpLTnRkxCRERkOB07doGNjS1KlXLDd9/txLp1G+Hg4Ch2LCrhuJzAgJzsdL8FrpOdpRGTEBER6S8rKwuCIMDa+sXe55UrV8HWrdtQr54nSpUqJXI6ohc4E2tANSo4wc3RKt/z3J2tUb08f4IlIiLTc/nyRXTu3Baffz5X67i3dwcWWDIpLLEGJJPJMKB93ncmkQHo364aL+oiIiKTkpOTgxUr/NG1a3tcv34NX3+9GRcvXhA7FtEbcTmBgTWq6YYG1UshMubpa49Zmssx8t26aFTTTYRkREREubtxIxrjx4/CxYuRAAAbG1ssWLAYnp4NRE5G9GaciTWwXWE3cy2wAPA8R43bj7grARERmQa1Wo2goEB06NBaU2CbNWuBsLCT+PjjEfytIZk0llgDUqvVOHQ2Ns9zDp2NhVrNO5YQEZG4njz5B3369MD8+Z/h+fPnsLCwwPz5ixEcvB+VK1cROx5RvlhiDSj0wkOo1HnfxVelFhAW+aiIEhEREeXO1tYOjx49BPDitrFHjvyJceMmQqFQiJyMSDdcE2tATxIzdTwvw8hJiIiI8mZnZ4d16zbi+PEwTJkyHebm5mJHIioQzsQakIeztY7n2Rg5CRER0b8EQcCePbswbNiHWkvamjdviRkzZrPAkiSxxBpQ+4bloJDnvQheIZfBu0HZIkpEREQl3bNnz/DJJx9jzJgRCAn5DV99tVHsSEQGwRJrQHK5HJ2bVMjznM5NKkAu55ediIiM7+DBP+Dl1Qy//fYrAKBixUqoV89T5FREhsE2ZWD9vauhW7OKkP9nWxKFXIZuzSqiv3c1kZIREVFJkZqaAj8/XwwZMhDx8XEAgCFDhuHYsVNo0aKVyOmIDIMXdhlBf+9qqF/dFUu3vdhzr1uzinivbRXOwBIRkdH99dcJTJw4Fg8evNjy0cOjNFavDkSHDp1FTkZkWGxVRvJqYW1ax4MFloiIisS6das0BbZv3344ceI0CywVS2xWRERExciqVYF4663K+Oqr77Bx4zdwdnYROxKRUbDEEhERSVR2djbWrl2F1NQUzbGyZcvh1Knz8PHpI2IyIuPjmlgiIiIJunr1CsaPH42//76M27dvYvXq9ZrHzMz4zzsVf5yJJSIikhCVSoW1a1ehUycv/P33ZQDA/fv38Pz5c5GTERUt/qhGREQkEbdv38KECWNw9mwEAMDKygpz5y7EiBGjeQExlTgssURERCZOrVZj69avsGjRPGRkZAAAGjZshMDAzahWrbrI6YjEwRJLRERk4n744VvMmjUVAGBubo6pU2diwoTJXPtKJRr/9hMREZm4gQM/wFdfbYRcLkdg4GbUq/eO2JGIRMcSS0REZGLi4+ORkZGOSpXeAvBi7ev27bvg4VEalpaW4oYjMhFcBU5ERGRCQkL2oW3bZhg9ehiUSqXmeMWKlVhgiV7BEktERGQCkpOT4Ov7CYYNG4ynT5/iwoXzOHEiTOxYRCaLJZaIiEhkYWFH4eXVHL/88hOAF3fd2rVrL9q37yRyMiLTxRJLREQkkrS0NEyfPhkDB/bB48ePALy4iOv48XC0bestcjoi08YLu4iIiETw9OlTdO/eAXfv3gEAlCpVCitWrEX37u+KnIxIGjgTS0REJAJXV1fUrl0XANC9e08cPx7BAktUAJyJJSIiKiKZmZmwtrYGAMhkMqxYsQY+Pr3Rt29/yGQykdMRSQtnYomIiIxMqVQiIGA5WrZshISEZ5rjbm5ueO+9ASywRHpgiSUiIjKimJgb6NGjI/z9F+PhwweYN+8zsSMRFQsssUREREagVquxadN6dOjQGpGRFwAATZo0w5Qp00VORlQ8cE0sERGRgd2/fw8TJ47FqVN/AQAsLCwwY8Yc+PpOgEKhEDkdUfHAEktERGRAO3duw2efTUd6ehoA4O2330Fg4CbUqVNX5GRExQuXExARERlQbOx9pKenQaFQYMqU6ThwIJQFlsgIOBNLRERkQJMnT8Pt2zcxapQvGjZsLHYcomKLM7FERER6Skh4hrFjR+Lvv6M0x8zNzbFx4zcssERGxplYIiIiPRw+fACTJ09AXNwTXLt2FQcPhsHS0lLsWEQlBmdiiYiICiA1NQWTJ4/H4MEDEBf3BADQoEFDKJVKkZMRlSyciSUiItLRyZN/YuLEsYiNvQ8AcHf3wKpV69CpU1eRkxGVPJyJJSIiykdmZibmzp2JPn16aAps7959ceLEaRZYIpFwJpaIiCgfJ0+ewKZNGwAAzs7OWLYsAL17vydyKqKSjSWWiIgoHx07dkG/fgORnJyEgIB18PAoLXYkohKPJZaIiOg/rl27isePH6F9+46aYwEB62BpaQmZTCZiMiJ6iWtiiYiI/k+lUmHdutXo1MkLY8eOwJMn/2ges7KyYoElMiF6l9izZ89izpw5GDFihObYlStXkJKSYpBgRERERen27Vvo1asbFi2ah+zsbGRkZODChfNixyKiN9CrxO7duxcTJ06Eo6MjTp06pTkeGhqKgIAAg4UjIiIyNkEQsHXrV2jfvhXOnDkN4MW+r0eP/oVu3XqInI6I3kSvErtlyxYEBgZi2rRpEARBc7x37944ceKEwcIREREZ06NHDzFgQG/MmDEFGRkZMDMzw4wZsxEScgQ1atQUOx4R5UGvC7tiY2Px9ttvA4DW+iBnZ2c8ffrUMMmIiIiMKCMjA506tUV8fBwAoFat2ggM3IR33qkvbjAi0oleM7GVK1fGhQsXAEBrJjY0NBSlS3PbESIiMn02Njbw9Z0ImUyG8eMn4fDhEyywRBKi10zslClTMH36dEyZMgUymQzh4eE4d+4ctm7dipkzZxo6IxERkUHExNxA9eo1NB+PGTMObdp4sbwSSZBeM7FeXl5YuXIlgoODYWVlBT8/P5w8eRLLly/HgAEDDJ2RiIioUJKTkzB+/Gi0adMUZ89GaI4rFAoWWCKJ0vtmB02bNkXTpk0NmYWIiMjgjh8Pg5+fLx49eggAWLp0Efbs+V3kVERUWHrNxNauXTvX40lJSWjTpk2hAhERERlCeno6Zs78FP3799IU2P79B+Gbb34QORkRGYLOM7EqlQqpqakQBAGCICA5OVnroi61Wo3Tp08jKyvLKEGlxvzJY3S4chQ2zzPh6nwbcp/uUJcpK3YsIqIS4cyZCEyYMBp37twGALi6uuLLL9fg3Xd9RE5GRIaic4n9559/0KFDB8hkMshkMjRv3lzrcUEQYG5ujk8//dTgIaXE7OIF2KxeiQaH/kAjpfLFwWOAsHA6sjt3Q8akT6Gs31DUjERExdn69WuxaNE8qNVqAEDXrj2wYsUauLu7i5yMiAxJ5xJbrlw5hIWFQaVSoWPHjjh69KjW43K5HK6urrCwsDB4SKmwCNkHhzHDIXv+/LXHZEolLPfvg8XRQ0jZtBXZ3d8VISERUfFXu3YdqNVq2Ns74IsvlmPAgPe19jQnouKhQBd2lSlTBgAwfvx4lCtXziiBpMrs4oU3FthXyZ4/h8PoYUjad5AzskREBqBUKqFQKDRFtX37jvD3X4kuXbqhXLnyIqcjImPR68Ku8ePHGzqH5NmsXplvgX1J9vw5bNYEGDkREVHxd/NmDN59txN27NC+WGv48E9YYImKOb1vO7tw4UJcunQJaWlprz1+7dq1QgeTEvnjR7A4uL9An2NxcD/kjx/xYi8iIj2o1Wp89dVGLF68AFlZWYiOjka7du1ZXIlKEL1mYufOnQtra2usXbsW1tbW+Omnn7BkyRLUqFED69evN3RGk2d+PAwylapAnyNTKmF+PMxIiYiIiq/Y2Pvo188Hc+bMRFZWFiwsLDB58jSULl1G7GhEVIT0mom9dOkSTpw4AXt7e5QtWxalS5fGO++8g4oVKyIwMBDt27cv0PNlZWVhyZIl+Ouvv6BSqfDuu+9i2rRpry3EFwQB3377LX788Uc8f/4c5ubm2L9/P8zNzfX5YxiMPDVFr8+TpaUaOAkRUfElCAJ27tyGOXNmIu3/3z/ffvsdBAZuQp06dUVOR0RFTa8Sa2dnh/T0dNjb26Ny5cq4desW3N3d8fbbb+PixYsFfj5/f3+o1WocPnwYmZmZ+Pjjj7Ft2zYMGTJE67ygoCCEh4djx44dcHV1xZMnT6BQKPT5IxiU2t5Br88T7OwNnISIqHh68uQJ/PzG4dChAwBe3C7Wz28KpkyZUaJ3xSEqyfRaTtC0aVMcOnQIANC8eXN89913ePjwIbZt2wYPD48CPVd6ejqCg4Mxbdo0mJmZwd7eHqNHj8bu3bu1zktISMCWLVuwfPlyuLq6AgA8PDwgl+v1RzConLbeEMwK9vOAYGaGnLbeRkpERFS8pKen4a+/TgAAqlWrjpCQw5g5cy4LLFEJptdM7OTJk3H9+nUAQP/+/REaGooOHTrA1tYWK1asKNBzXblyBeXLl4eTk5PmmKenJ2JiYqBSqTQzrWFhYWjYsKFmmy99yeUyyOUG3i+wQnnkdO0Gi9/36fwpOV27Q16hvH4/RVCRUSjkWv9P0sLxk76XY1e9enUsWvQFbty4gXnzFsLa2lrkZKQrvg+lzZTHTya8eu/YQkhJSYGNjQ3MCjgjGRISgp9//hnfffed5lhOTg7efvttREREaMrt0qVLkZmZCQD466+/YG9vj2HDhqF3794Fej1BEIyz6fW5c0Dr1oAu22xZWQF//gk0bmz4HERExcD+/ftx/vx5zJ07V+woRGSi9JqJzY2DgwNiY2OxZMkSbNy4UefPUyqV+G+PfnmrwFfLZnp6Oo4dO4bly5dj4cKFuH79OoYPH46yZcuiadOmOr9eQkK64WdiAaBqbZh/9S1sR36c536xgqUl0rdsRU7V2kBiuuFzkEEpFHI4OFgjJSUTKpVa7DhUQBw/6UlNTcXcubPw/fffAgDq12+Mnj27cQwljO9DaRNr/JydbfM9p0Alds+ePQgLC4ONjQ0++ugj1K374mrQrKwsBAUFYevWrWjWrFmBQjo5OSExMVHrWEJCAiwtLWFv/++FT87OzmjTpg1atmwJAKhduzZ8fHwQGhpaoBKrVgtQqw0y+fwaZZceyNl3EDZrAmB+cD/kSqXmMcHMDNlduiPDb8qLO3Up+UaWEpVKDSXHTLI4ftJw6tRfmDhxLO7fvwcAcHNzR3Z2DgCOYXHAMZQ2Uxw/nUvs999/jw0bNmDAgAFIS0vDiBEjsHPnTty9excLFiyAg4MD1q9fjzZt2hQoQJ06dXDnzh0kJyfD0dERABAZGQlPT0+ti7aqVq2Ke/fuaX2uXC43uUX9yvoNkbJ1G+5HXseJgG2wyc5A5w514dCzG29sQESUi8zMTHzxxefYvHmD5jdzvXr1xbJlK+Hu7iZyOiIyVTqX2J07d2LVqlVo0aIFAKBx48aYPXs2bt68CT8/P7z//vt67RTg5uaGNm3aICAgAHPnzkVKSgqCgoLg5+endV7Xrl2xYsUKnDp1Ci1btsStW7ewb98+fPXVVwV+zaKQ41EGoXVf7JfbqFcT2HlwOy0iov+6ePECxo8fjRs3ogG8+O3csmUB6NOnn8jJiMjU6VxiHzx4gEaNGmk+bt++PaZMmYJNmzahbdu2hQqxZMkSzJ49G61bt4aNjQ2GDx+Ojh07Yu/evYiKisKcOXNgZWWFdevWYeHChUhISICLiwuWLFmCWrVqFeq1iYhIHCqVCmPHjsStWzcBAB06dMKqVYG88xYR6UTnEqtUKrV+dW9lZQWZTAYvL69Ch3BxcUFQUNBrx3v16oVevXppPm7QoAGCg4ML/XpERCQ+hUKBVasCMXjwACxcuASDB39knN1jiKhY0rnEvrzl66s7Cbw89l/Dhg0zSDgiIio+VCoVgoN3o0+ffprlZ82bt0Rk5BU4ODiKnI6IpEbnEtukSRMcPXr0tWOhoaFax2QyGUssERFpuXPnNiZOHIuIiHAkJDzDJ5+M1TzGAktE+tC5xP7www/GzEFERMWQIAj47rtvsGDBHGRkvNgb+/fff8OIEaNN4rbhRCRdBrvZARER0aseP36ESZPGISzsxW/xzMzMMGXKdPj5fcoCS0SFxhJLREQGJQgCdu/+GbNmTUNychIAoGbNWggM3ARPzwbihiOiYoM/ChMRkUGtXRsAX99PkJycBJlMhnHj/HD48AkWWCIyKJZYIiIyqH79BsLBwRGVKr2FvXv/wPz5i2BlZSV2LCIqZricgIiICiUlJRlyuRx2di/uTFiuXHns3PkLateuCzs7O5HTEVFxxZlYIiLS24kTx9C2bQvMmTNT63iTJs1YYInIqPQusWfPnsWcOXMwYsQIzbErV64gJSXFIMGIiMh0ZWRkYNasqejXzwcPHz7Ajh0/4PLli2LHIqISRK8Su3fvXkycOBGOjo44deqU5nhoaCgCAgIMFo6IiEzP2bMRaN++Fb7+ejMAwNXVFV9//QPeeae+uMGIqETRq8Ru2bIFgYGBmDZtmtZtaHv37o0TJ04YLBwREZmO58+fY/HiBejZswtu374FAOjatQeOH49Az569RE5HRCWNXhd2xcbG4u233wbw4jazLzk7O+Pp06eGSUZERCYjNvY+hgwZhKtX/wYA2Ns7YMmSZRg48AOtfweIiIqKXjOxlStXxoULFwBAayY2NDQUpUuXNkwyIiIyGW5u7lAqcwAAbdq0w/Hj4Rg0aDALLBGJRq+Z2ClTpmD69OmYMmUKZDIZwsPDce7cOWzduhUzZ87M/wmIiMjkCYKgKalWVlYIDNyE8+fPYtiwT3jbWCISnV7fhby8vLBy5UoEBwfDysoKfn5+OHnyJJYvX44BAwYYOiMRERUhtVqNLVuC8N57PaFUKjXH69dviBEjRrPAEpFJ0Gsm9t69e2jatCmaNm1q6DxERCSi2Nj7mDRpHP788zgAIDBwNSZNmipyKiKi1+n143SPHj0wZMgQ/Pbbb8jOzjZ0JiIiKmKCIGDnzm1o27aFpsDWqfM2OnXqKnIyIqLc6VViT5w4gY4dO+L7779H69at8fnnn+PatWuGzkZEREXgyZMnGDr0ffj5+SItLRVyuRyTJk3FoUPHULfu22LHIyLKlV7LCVxcXDB06FAMHToUt2/fxr59+zBx4kTY29tjwIABGDRokKFzEhGREezbF4xp0yYhISEBAFC1ajWsW7cRjRtzuRgRmbZCr86vUqUKxo0bh5kzZ8LR0RGLFi0yRC4iIjIyQRDwyy8/awrsyJGjcfToXyywRCQJes3EAi+uXg0PD0dISAiOHj0KFxcX9OnTB0uXLjVkPiIiMhKZTIYVK9bg0aOHmDt3Iby82okdiYhIZ3qV2AULFuDw4cNQKpXo1q0bNm/eDE9PT0NnIyIiA0pLS8WKFcswceJkuLi4AgDc3Nxw6NAx3rSAiCRHrxL7zz//YO7cuWjfvj0sLCwMnYmIiAzs9OlTGD9+DO7fv4uHDx9gy5ZvNY+xwBKRFOlVYjdu3GjoHEREZARZWVlYunQRNm4M1NwmXK1W4/nz57C0tBQ5HRGR/nQqsVOnTsWKFSs0H3/00Ud5nv/9998XLhURERXaxYsXMH78aNy4EQ0AcHR0gr//CvTt25+zr0QkeTqV2IYNG2p93LZtW34DJCIyUTk5OVi16kusWvUlVCoVAKB9+45YtSoQZcqUFTkdEZFh6FRiP/jgA62PR4wYYZQwRERUeD/88C1WrPAHANjY2OLzz7/AkCEfc/KBiIoVvfaJ7dChQ67Hk5KSeKMDIiKRDRnyMd55pz6aN2+JY8dO4aOPhrHAElGxo9eFXQ8fPsz1+L179xAdHV2oQEREVDB3795BdnY2atSoCQAwNzfHzp274eLiAoVCIXI6IiLj0LnE3r17F3369IFMJoNMJnttnawgCMjKysLQoUMNHpKIiF4nCAK+/34r5s+fjbfeqoyDB8M0Ow64ubmJnI6IyLh0LrFvvfUWtmzZApVKhaFDh762zZZcLoeHhwcqVKhg8JBERKTt8eNHmDx5PEJDjwAAoqOvISIinHfdIqISo0DLCRo3bgwA6NOnD5o25b21iYiKmiAI2LNnF2bNmoqkpCQAQM2atRAYuAmeng3EDUdEVIT0WhO7dOlSQ+cgIqJ8PHv2DNOnT8a+fcEAXtxpa8yY8Zg1ay6srKzEDUdEVMT0utnBkCFD8rzSlTc7ICIyrPj4eLRr1wLx8XEAgIoV38K6dUFo0aKVyMmIiMSh180OvL29jRKGiIhy5+bmBi+vdti9+2cMGTIMCxcuhp2dvdixiIhEo9fNDoYPH26UMERE9K/ExAQ4O7toPl669Ev07z8I7dt3FDEVEZFp0OtmB9euXUNsbKzm49jYWPj7+2P79u0GC0ZEVFJlZGRg9uzpaNmyEZ48+Udz3MnJmQWWiOj/9CqxEydOREJCAgAgOzsbw4cPR0xMDHbv3o3AwECDBiQiKknOnz+LDh1aY8uWjXj27BkWLJgjdiQiIpOkV4l98uQJateuDQDYt28fHB0d8fXXXyMoKAh79uwxaECpEgRB89/3/knV+piI6L+ys7PxxRefo0ePTrh16yYAoEuXbliwYInIyYiITJNeJbZMmTJ48OABBEHAd999p7lLl729PZ49e2bQgFJ0PjoeQcFXNB9/+8d1zNp0Guej40VMRUSm6sqVv9GlizdWr14BtVoNOzt7rF0bhO+//xEeHh5ixyMiMkl6ldiPPvoIQ4cORZ8+fSAIAnr06AEAiIqKKvHfcM9Hx2NDcBQSU59rHY9LysSG4CgWWSLSEhQUiM6d2+LKlSgAQOvWXjh+PByDBg3OcytDIqKSTq+bHQwePBhly5bF/fv30bNnT8jlL7pwRkYGpk+fbtCAUiIIAnaF3cSbVg4IArDr2E00rFGK/zgREQBApVIhJycH1tbWmDt3IYYPH6X5nkpERG+mV4kFct8rtqTvH3sjNglxSZl5nhOXmImYB8moUcGpaEIRkUkbO3Y8Hjy4j08+GYOqVauLHYeISDL0/nE/PDwcQ4YMQfPmzdGwYUMMHToU586dM2Q2yUlKy9bxvOf5n0RExc6DB7EYOLAPzp8/qzmmUCjg77+SBZaIqID0KrH79++Hr68vGjdujJUrV2L9+vVo1KgRfH19ERERYeiMkuFkZ6HjeZZGTkJEpkQQBPz443a0bdsCYWFHMX78aGRkZIgdi4hI0vRaTrBx40YsXLgQPj4+mmMtWrRA5cqVsXr1auzcudNgAaWkRgUnuDtZ57mkwN3ZGtXLOxZhKiISU1xcHKZO9cOBAyEAALlcjnff7QWFQiFyMiIiadNrJvbu3bto3779a8e9vb1x9erVQoeSKplMhv7e1fCma7ZkMqB/u2q8qIuohNi3by/atm2mKbBVqlTFvn0HMXv2fFha8jcyRESFoVeJtbe3R2Ji4mvHExMTYWGh26/Ui6tGNd3g27senO21/4Fyd7aGb+96aFTTTaRkRFRUkpISMXbsSIwYMUSzd/aIEaNw9OhfaNKkmcjpiIiKB71KbNu2bbFs2TJkZ/97IVNOTg6+/PJLeHl5GSycVDWq6YaxvetqPv64Wy0sHdWcBZaohIiMvIDdu38GAJQrVx6//PIbli5dAVtbW5GTEREVH3qtiZ06dSo++ugjdO7cGY0aNYJcLsf58+dhbm6OH374wdAZJenVJQOVSttzCQFRCeLt3QFDhgxDTk42Fi/2h4MD18ETERmaXiXWxcUFe/bswf79+xEVFQWlUonRo0fDx8cH1tbWhs5IRGTSTp8OR3x8HHr27KU5tnx5AC/eIiIyogKV2Li4OBw8eBCPHz9GjRo10L17d/Tu3dtI0YiITFtWVhb8/RcjKGgdbG3tUL9+A1SoUBEAWGCJiIxM5zWx169fR8+ePfHrr7/i0aNH2LRpE/r374/U1FRj5iMiMkmXL19E585tsWHDWgiCAIVCgVu3boodi4ioxNC5xK5atQqDBg3Cnj17sHr1auzfvx9169ZFQECAMfNJliAImv++90+q1sdEJF05OTlYscIfXbu2x/Xr1wC8WAN74sRptGv3+taDRERkHDqX2KioKHz44Yeaj2UyGcaOHYvDhw8bJZiUnY+OR1DwFc3H3/5xHbM2ncb56HgRUxFRYUVHX0ePHh2xfPkXUCqVsLGxxZdfrsaPP+5BmTJlxY5HRFSi6FxiExMT4eamvUVU+fLlkZCQYPBQUnY+Oh4bgqOQmPpc63hcUiY2BEexyBJJVHp6Onr16oqLFyMBAM2atUBY2EkMHTqcu48QEYmgQBd2Xb169bVfiwuC8NrxunXr/vdTSwRBELAr7CbetHJAEIBdx26iYY1S/EePSGJsbW0xffpszJs3C7NmzcOYMeN48RYRkYh0LrGCIKBv3765PvbqcZlMhmvXrhU+mQTdiE1CXFJmnufEJWYi5kEyalRwKppQRKQXQRBw7twZrTtsDRs2Et7eHVC5chURkxEREVCAEnv9+nVj5igWktKy8z8JQFLa8/xPIiLR/PPPY0yZMgFHjhzCnj2/o3XrF3cilMlkLLBERCZCr9vOUu6c7Cx0PM/SyEmISF+//voLvLya4ciRQwCAwMDV4gYiIqJcscQaUI0KTnB3yvuOZe7O1qhenregJDI1z549wyeffIzRo4cjKSkJMpkMo0ePw9at28WORkREuWCJNSCZTIb+3tXwpmu2ZDKgf7tqvKiLyMQcOvQHvLyaYe/ePQCAihUr4ddfQ7Bo0VLeSpuIyESxxBpYo5pu8O1dD8722ksG3J2t4du7HhrVdHvDZxKRGPz9F+PDDwciPj4OADBkyMc4duwUWrZsLXIyIiLKC0usETSq6Yaxvf/dZuzjbrWwdFRzFlgiE9S2rTdkMhk8PEpjx45dWLlyLezs7MWORURE+SjQPrGku1eXDFQqbc8lBEQmIjMzE5aWlpDLX/wM36JFK2zYsAXt23eEs7OLyOmIiEhXes/EJicn47fffsPGjRsLHSIrKwtz586Ft7c3vLy8sHz58tduqvCqjIwMNG/eHJs3by70axNRyXH+/Fl06NAa33yj/b3jvfcGsMASEUmMXiU2KioKXbp0wQ8//IA1a9Zojk+bNk2vYunv7w+1Wo3Dhw8jJCQEERER2LZt2xvP3759O1JSUvSJTkQlUHZ2NpYu/Rw9enTCzZsxWLRoPh48iBU7FhERFYJeJdbf3x9Tp07Frl27tGZMR4wYgd9++61Az5Weno7g4GBMmzYNZmZmsLe3x+jRo7F79+5cz3/y5Al++eUXdOjQQZ/oRFTCREVFoVOndli1agXUajXs7OyxdOkKlCtXXuxoRERUCHqtib169Sq++uorAP9Z+1mpEh4+fFig57py5QrKly8PJycnzTFPT0/ExMRApVK9dm/yL774AmPGjEFERIQ+0SGXyyCXG399qpni358P5HIZzMx4DZ0UKf4/jgoFx09qVCoV1qwJwBdfLEJOTg4AoHXrNggM3IiKFSuJnI50xfeg9HEMpc2Ux0+vEuvs7IyHDx+iWrVqWsevXr0KW1vbAj1XfHw8XF1dtY65uLhAqVQiNTVVq9zu27cPSUlJ6N27t94l1sXFtkgusrJP+ffWsra2lnB2LtjXhUyLgwP3CpWSO3fuYPDgwQgPDwcAWFlZYdmyZRg/frzmgi6SFr4HpY9jKG2mOH56ldgPP/wQc+bMwYoVKwAASqUSFy5cwLx589CjR48CPZdSqXztIi61Wg1Ae5Y3NjYWq1atwvbt2wtVQhMS0otkJjY1NUvz3+npz5GYmG701yTDUyjkcHCwRkpKJlQqtdhxSEdpadm4cuUqAKBp06bYsGEzqlSphuTkTJGTUUHxPSh9HENpE2v8dJn806vEDh8+HFlZWfDx8YEgCPD09IRcLsegQYPw6aefFui5nJyckJiYqHUsISEBlpaWsLd/sVdjVlYWJkyYgKlTp6JMmTL6RNZQqwWo1W/e+cBQlK8MtFotQKnkG1fKVCo1x1BCSpcuC3//FXj4MBYLFsxFaupzjp/E8T0ofRxDaTPF8dN7n1hfX1+MGDECN27cgLm5OSpVqqTX7Rnr1KmDO3fuIDk5GY6OjgCAyMhITTEGgPDwcNy+fRvz5s3DvHnzALwotnK5HOHh4di6dau+fwwikjhBEPDzzztx5crf+PzzLzTH+/UbCDMzOczMzAA8f/MTEBGRJBXqZgeWlpaoV69eoQK4ubmhTZs2CAgIwNy5c5GSkoKgoCD4+flpzvH29sbly5e1Pm/mzJmoUqUKRo0aVajXJyLpio+Px9Spfvjjj98BAG3aeKFTp64ipyIioqKgV4kdM2ZMno8X9AYIS5YswezZs9G6dWvY2Nhg+PDh6NixI/bu3YuoqCjMmTNHn5hEVIyFhOzDtGl+ePr0KQDgrbcqw8nJWeRURERUVPQqsW+//bbWxzk5OYiOjsalS5cwcODAAj+fi4sLgoKCXjveq1cv9OrVK9fP8ff3L/DrEJH0JScnYdasafjll580x4YNG4l58xYVeHcUIiKSLr1K7Pjx43M9vnPnTty6datQgYiI3iQs7CgmTRqHx48fAQDKli2H1avXo1279iInIyKiombQDRN9fHywd+9eQz4lERGAF9vxzZs3S1NgBwx4H8ePh7PAEhGVUAYtsdeuXYOlpaUhn5KICABgZmaGdes2onTpMvj22x0IDNwER0cnsWMREZFI9FpO8N8trVQqFR4+fIiQkBC89957BglGRCXb8+fP8cMPW/HxxyP/v00WUL9+Q5w9e5k/LBMRkX4lNjQ0VOtjuVyOUqVKYfr06ejTp49BghFRyRUVdQnjx4/GtWtXkZ6eDj+/f2+iwgJLRESAniX266+/hrm5eaFu/0pE9F9KpRJr1wZgxQp/KJVKAMCZM6chCAK/3xARkZYCr4kVBAH169dHSkqKMfIQUQkVE3MDPXp0hL//YiiVStjY2GDZsgBs2/YzCywREb2mwCVWJpOhbt26ePjwoTHyEFEJo1arsXnzBnTo0BqRkRcAAE2aNENo6EkMGzaSBZaIiHKl13KC0aNHY9GiRejWrRvq1KkDa2trrcfr1q1rkHBEVPx9+eVSrFy5DABgYWGBGTPmwNd3AhQKhcjJiIjIlBXqZgeRkZGvPSaTyXDt2rXCpSKiEmPYsE/w7bdfoUyZcggM3IQ6dfhDMBER5U/nEtuhQwccPXoUAHD9+nWjBSKi4u3Jk39gYWEBZ2cXAIC7uzv27AlB1arVYGFhIXI6IiKSCp3XxHINLBEV1t69e+Dl1QwzZkzROl67dh0WWCIiKhCdSywvriAifSUkPMPo0cPwyScfIzExEcHBexAVdVnsWEREJGE6LycQBAEfffSRTud+//33egciouLlyJGDmDRpPOLingAAKlSoiLVrg1Cv3jsiJyMiIikr0IVd7dq1M1IMIipu0tJSMW/eZ9i27TvNscGDP8Lnn38Be3sHEZMREVFxoHOJlclkGD58uDGzEFExcefObfTv3wv3798DALi7e2DVqnXo1KmryMmIiKi40HlNrCAIxsxBRMVIuXLl4ejoBADo3bsvTpw4zQJLREQGpfNMbJ8+fYyZg4gkLicnB+bm5gBe3LRg/frNuH79Knr3fk/kZEREVBzpPBO7dOlSY+YgIonKzs6Gv/9idO/eEdnZ2ZrjtWrVZoElIiKj0bnEEhH917VrV9GtWwcEBCzHpUuRWLXqS7EjERFRCcESS0QFplKpsG7danTq5IWoqEsAgJYtW+P99z8UORkREZUUBdpii4jo9u1bmDhxLM6cOQ0AsLKywuzZ8/HJJ2Mhl/PnYiIiKhossUSkE0EQ8O23X2PhwjnIyMgAADRo0BCBgZtRvXoNkdMREVFJw2kTItKJTCZDREQ4MjIyYGZmhpkz5yAk5AgLLBERiYIzsUSkM3//FUhMTMCcOQtQr56n2HGIiKgE40wsEeXq6dOnmDhxLJ48eaI55uTkjJ9++pUFloiIRMeZWCJ6zf79v2Pq1Il4+vQpEhKe4YcffoJMJhM7FhERkQZLLBFpJCcnYfbsGfj5552aY+XKlUdOTg4sLCxETEZERKSNJZaIAADHjoVi0qRxePToIQCgTJmyWL16Pby9O4icjIiI6HVcE0tUwqWnp2PmzE8xYEBvTYHt338Qjh8PZ4ElIiKTxZlYohLup5924JtvtgAAXF1d8eWXa/Duuz4ipyIiIsobSyxRCTd06HDs3v0zXF1LYcWKNXB3dxc7EhERUb5YYolKmKioyxAENd55pz4AQKFQ4Mcfd8POzp47EBARkWRwTSxRCaFUKhEQsBxdurTDmDEjkJmZqXnM3t6BBZaIiCSFJZaoBLh5MwbvvtsJ/v6LoVQq8ejRQ1y6FCl2LCIiIr2xxBIVY2q1Gps3b0D79q1w4cJ5AECTJs0QGvoXmjdvKXI6IiIi/XFNLFExFRt7H35+vvjrrxMAAAsLC0yfPhvjxk2EQqEQOR0REVHhsMQSFUNPnjxBu3YtkZqaAgCoW7ce1q/fjDp16oqcjIiIyDC4nICoGPLw8EDv3n0hl8sxefJUHDwYxgJLRETFCmdiiYqJBw9iUb58Bc3HCxcuweDBH6Fhw8YipiIiIjIOzsQSSVxiYgJGjx4GL6/miI29rzluZ2fPAktERMUWSyyRhB05chBeXs3x66+7kZaWiqVLF4kdiYiIqEiwxBJJUFpaKj79dCI++KA/njz5BwDwwQdDsGzZSpGTERERFQ2uiSWSmPDwk5gwYSzu378LAHBzc0dAwDp06dJN3GBERERFiDOxRBLi778IvXt31xRYH58+OHEiggWWiIhKHM7EEkmIm5sHBEGAk5MT/P1Xok+ffpDJZGLHIiIiKnIssUQmTK1WQy7/9xcmw4aNxNOn8Rg6dDhKly4jYjIiIiJxcTkBkYm6du0qunTxxsmTf2qOyeVyzJgxmwWWiIhKPJZYIhOjUqkQGLgGnTp54dKlSEycOBZpaalixyIiIjIpXE5AZELu3LmNiRPHIiIiHABgaWmJTz4ZAxsbW5GTERERmRaWWCITIAgCvvvuGyxYMAcZGekAgPr1GyAwcDNq1KgpcjoiIiLTwxJLJLLHjx9h0qRxCAs7CgAwMzPDlCnT4ef3KczNzUVOR0REZJpYYolEdvfuHRw7FgoAqFmzFgIDN8HTs4HIqYiIiEwbSyyRyFq0aIWxYycAAGbOnAMrKyuRExEREZk+lliiInbgwH4kJSVi0KDBmmPz5y/iTQuIiIgKgCWWqIikpCRjzpyZ+PHH7bC2tkaTJk1RtWp1AGCBJSIiKiDuE0tUBE6cOIa2bVvgxx+3AwAcHZ3w9OkzkVMRERFJF0sskRFlZGRg1qyp6NfPBw8fPgAAvPfeAJw4cRrNmjUXOR0REZF0cTkBkZGcPRuBCRPG4PbtWwAAFxcXfPnlGvTs2UvkZERERNLHEktkBGlpafjwwwFITEwEAHTt2h0rVqyFu7u7yMmIiIiKBy4nIDICOzs7LFz4BeztHbB2bRC++24nCywREZEBscQSGYBSqcSRIwe1jg0c+AHCwy9g0KDB3H2AiIjIwFhiiQrp1q0Y9OzZBR980B+HDx/QHJfJZJx9JSIiMhKWWCI9qdVqfPXVRrRv3xrnz58FAGzf/oPIqYiIiEoGXthFpIcHD2Lh5+eLP/88DgAwNzfHjBmzMW6cn8jJiIiISgaWWKICEAQBP/20A7Nnz0BqagoAoE6dt7F+/WbUrfu2yOmIiIhKDpNYTpCVlYW5c+fC29sbXl5eWL58OQRB0DonJycHgYGB6NmzJ9q2bYsPPvgA165dEykxlVTz5n2GiRPHIjU1BXK5HJMmTcWhQ8dYYImIiIqYSZRYf39/qNVqHD58GCEhIYiIiMC2bdu0zrl79y5UKhV++uknHD9+HD4+PhgzZgxycnJESk0lUd++/aBQKFClSlX8/vshfPbZPFhYWIgdi4iIqMQRvcSmp6cjODgY06ZNg5mZGezt7TF69Gjs3r1b67zq1avDz88PNjY2AIBBgwYhMzMT9+7dEyM2lRDJyUlQqVSajxs0aIQffvgRoaEn0bhxUxGTERERlWyir4m9cuUKypcvDycnJ80xT09PxMTEQKVSQaFQ5Pp5mZmZyMzMhL29fYFeTy6XQS43/p6dZop/fz6Qy2UwMxP95wUqoCNHDsPPzxcTJ06Er++/F2x17dpNxFRUEIr/vw8VCr7/pIpjKH0cQ2kz5fETvcTGx8fD1dVV65iLiwuUSiVSU1O1yu2rVq1ahaZNm8LDw6NAr+fiYlskG8/bpzzX/LetrSWcnW2N/ppkGGlpaZg6dSo2bdoEAJg3bx4GDx6MChUqiJyM9OXgYC12BCokjqH0cQylzRTHT/QSq1QqX7uIS61WA0CuZTMjIwMLFy7EjRs38PXXXxf49RIS0otkJjY1NUvz3+npz5GYmG7016TCCw8/iXHjxuDu3TsAAHd3d2zZsgWOjqU4hhKkUMjh4GCNlJRMqFRqseOQHjiG0scxlDaxxk+XyT/RS6yTkxMSExO1jiUkJMDS0vK1pQL379/HmDFj0KBBA+zYsQPW1gX/qUCtFqBWC/mfWEjKVwZarRagVPKNa8qysrLg778YQUHrND9U9ezZGwEBq1GtWiUkJqZzDCVMpVJz/CSOYyh9HENpM8XxE73E1qlTB3fu3EFycjIcHR0BAJGRkfD09IRc/u/6i5SUFAwdOhRjx47FgAEDxIpLxdCNG9EYMWIIoqOvA3jxg5W//0r06dMP5ua5r8kmIiIicYm+StfNzQ1t2rRBQEAAlEolEhISEBQUhKFDh2qdd+DAAVSpUoUFlgzOyckZ8fFxAID27TvixIkI9O3bv0jWThMREZF+RJ+JBYAlS5Zg9uzZaN26NWxsbDB8+HB07NgRe/fuRVRUFObMmYO7d+8iMjIS7du31/rcMWPGsNhSobi7u2PlynV49uwphgz5mOWViIhIAmTCf6+qKubi41OL5HVuPUrGku/PAwDmD2uCSh4F2wqMjEOlUmHTpg2Ijr6GNWs25Hu+mZkczs62XBMrURw/6eMYSh/HUNrEGj83t/x7k0nMxBIVhbt372DixLE4ffoUAKBDh07w8ekjcioiIiLSh+hrYomMTRAEfPfdN2jXrqWmwL7zTn3UqFFL5GRERESkL87EUrH2+PEjTJ48HqGhRwAACoUCkydPw+TJ02Bubi5yOiIiItIXSywVS4IgYM+eXZg5cyqSk5MAADVr1kJg4CZ4ejYQNxwREREVGpcTULGkUqmwfv1aJCcnQSaTYezYCTh8+AQLLBERUTHBEkvFkpmZGQIDN6F69RoIDt6PhQuXwMrKSuxYREREZCAssVQspKamYPnyL5Cdna05VqdOXfz55xm0aNFKxGRERERkDFwTS5L355/H4efniwcPYqFWqzBz5lzNY6/eupiIiIiKD5ZYkqyMjAwsXjwfX321SXMsNjYWgiDwrltERETFHEssSdK5c2cwYcIY3Lp1EwDg4uKC5ctX8eYFREREJQRLLElKdnY2Vqzwx9q1AVCrX9z+rkuXblixYi08PDxETkdERERFhSWWJOXLL5dizZqVAAA7O3ssWbIMgwYN5vIBIiKiEoZXvZCk+PpOgIdHabRu7YXjx8Px/vsfssASERGVQJyJJZN2+/ZN2Nk5wN3dHQDg7OyCkJDDKF++AnceICIiKsHYAsgkqdVqfP31Jnh7t8Knn06AIAiaxypWrMQCS0REVMKxCZDJefAgFv3798asWdOQmZmJ0NAjuHbtqtixiIiIyISwxJLJEAQBP/64HW3btsCffx4DANSuXRcHDx5DnTp1xYxGREREJoZrYskkxMXFYerUiThwYD+AF3famjBhMqZOnQlLS0uR0xEREZGpYYkl0cXE3ICPTxc8e/YMAFClSlWsW7cRTZo0EzkZERERmSouJyDRValSFZUrVwUAjBgxCkeP/sUCS0RERHniTCyJIj09Hba2tgAAhUKBwMCNePDgAby82okbjIiIiCSBM7FUpNLS0jBt2mR07twWmZmZmuNVqlRjgSUiIiKdscRSkTl9Ohze3i3x3XdfIybmBlat+lLsSERERCRRXE5ARpeVlYVly5Zgw4a1mpsW9Ojhg1GjfEVORkRERFLFEktGdfnyRYwfPxrXr18DADg6OmHp0i/x3nsDIJPJRE5HREREUsUSS0YhCAICApZj5cplUCqVAABv7w5YtSoQZcuWEzkdERERSR1LLBmFTCbDo0cPoVQqYWNjiwULFmPo0OGcfSUiIiKDYIklo1m4cAnS09MwY8YcVK5cRew4REREVIxwdwIyiHv37mLIkIF48CBWc8zOzh4bN37DAktEREQGx5lYKhRBELBt23eYN+8zpKenISMjA7t27YVczp+PiIiIyHhYYklv//zzGFOmTMCRI4cAvLjzVtOmzaFWq1liiYiIyKhYYkkvv/76C2bMmIKkpCQAQPXqNRAYuAkNGjQSNxgRERGVCCyxVCAJCc8wY8an2Lt3D4AXuxCMGuWLzz6bB2tra5HTERERUUnBEksF8vvvv2kKbMWKlbB2bRBatmwtcioiIiIqaVhiqUA+/HAofvstGJUqVcLChUtgZ2cvdiQiIiIqgVhiKU9//XUC5uYWaNasOQBALpdjx45dsLCwEDkZERERlWS8hJxylZmZiTlzZqBv33cxbtwnSEtL1TzGAktERERiY4ml11y4cA4dOrTG5s1BAIDU1BRER18XORURERHRv1hiSSM7Oxv+/ovQo0cn3LwZAwDo1KkLTpyIQKNGTUROR0RERPQvroklAMC1a1cxbtwo/P33ZQAvbhm7eLE/3n//Q8hkMpHTEREREWljiSU8efIPunRph6ysLABAq1ZtsGbNBlSsWEnkZERERES543ICgodHaQwdOgJWVlZYsmQZdu/exwJLREREJo0ltgQSBAHXrl3VOvbZZ/MQFnYSn3wyFnI5/1oQERGRaWNbKWEePnyAAQN6o2tXb9y6FaM5bm1tjapVq4uYjIiIiEh3LLElhCAI+OmnHWjbtgWOHw9DZmYm1qwJEDsWERERkV54YVcJEB8fj6lT/fDHH78DeHHXrXHj/DB9+mciJyMiIiLSD0tsMRcSsg/Tpvnh6dOnAIC33qqMwMDNaNq0mcjJiIiIiPTH5QTF2KxZUzFs2GBNgR02bCTCwk6xwBIREZHkcSa2GPP0bAAAKFOmLFavXg9v7w4iJyIiIiIyDJbYYiQ7OxsWFhaajwcO/AApKckYOPADODo6iReMiIiIyMC4nKCYiIg4jTZtmuLIkYOaYzKZDKNG+bLAEhERUbHDEitxz58/x+efz4OPTxfcuXMbkydPQGpqitixiIiIiIyKywkkLCrqEsaPH625+5aDgyPmzfscdnb2IicjIiIiMi6WWAlSKpVYuzYAK1b4Q6lUAgDatvXGmjUbULZsOZHTERERERkfS6zE3LwZg3HjPkFk5AUAgI2NDebPX4yPPx4BmUwmcjoiIiKiosESKzHJyUm4dOkiAKBJk2ZYt24jqlSpKm4oIiIioiLGEisxjRo1wbRps2BhYQlf3wlQKBRiRyIiIiIqciyxJkwQBGzf/j0yMzPwySdjNcc//XSGiKmIiIiIxMcSaySCIGj++94/qajoblegNatPnvyDKVMm4PDhg7CwsECrVl6oU6euMaISERERSQ73iTWC89HxCAq+ovn42z+uY9am0zgfHa/T5wcH74aXVzMcPvzixgWVKr0FlUpplKxEREREUsQSa2Dno+OxITgKianPtY7HJWViQ3BUnkU2IeEZRo36GKNGDUNiYiIAYPRoXxw58ifq1fM0am4iIiIiKeFyAgMSBAG7wm7ilZUE/3kc2HXsJhrWKPXa0oIjRw5i0qTxiIt7AgCoUKEi1q4NQqtWbYwdm4iIiEhyOBNrQDdikxCXlJnnOXGJmYh5kKx1LC0tFePHj9YU2MGDP8KxY6dYYImIiIjegCXWgJLSsnU8T3upgZ2dPZYvXwV3dw9s3/4zVq0KhL29gzEiEhERERULLLEG5GRnodN51mYCfv55p9YxH58+OH36Ajp16mqMaERERETFCtfEGlCNCk5wd7LOc0mBLP0+xnw8HTExN2BtbY2ePXtrHrOzsy+ClERERETSZxIzsVlZWZg7dy68vb3h5eWF5cuXa+2z+tLVq1cxYMAAeHt7o3v37jh58qQIad9MJpOhv3c15LYbrFqVg+iT2xHy1WTExNwAABw8+EfRBiQiIiIqJkyixPr7+0OtVuPw4cMICQlBREQEtm3bpnVOWloaxowZg0mTJiEsLAwLFiyAn58f4uN123u1KP23fqc8vYe/ds5ATMQuqFUq2NraISBgHdat2yhKPiIiIiKpE73EpqenIzg4GNOmTYOZmRns7e0xevRo7N69W+u8kJAQ1KtXDy1btgQANG3aFI0bN8b+/fvFiJ2rl1tsaT5Wq3Dz7B78tf1TpMTdBgCUfusdHDt2Ch9+OLRAd/AiIiIion+Jvib2ypUrKF++PJycnDTHPD09ERMTA5VKBYVCAQC4ePEiGjZsqPW5np6euH79eoFeTy6XQS43Tnm8fi9Raz1s1NFNuB916MXrKixQq82HqNzgXSjNnWFmJvrPD6QDhUKu9f8kLRw/6eMYSh/HUNpMefxEL7Hx8fFwdXXVOubi4gKlUonU1FRNuY2Li0OzZs20znN1dcWlS5cK9HouLrZGmwHNuZuo9fFb9bsj9kooHNwqo37XibB3rfDiPLUMzs62RslAxuHgYC12BCoEjp/0cQylj2MobaY4fqKXWKVS+dpFXGq1GgC0yqZKpXrtc1UqVYELaUJCutFmYs3/87QObm+hRf9FcCpTA3K54t/z5AISE9ONkoEMS6GQw8HBGikpmVCp1GLHoQLi+Ekfx1D6OIbSJtb46TLZJ3qJdXJyQmKi9gxmQkICLC0tYW//75ZTjo6OuZ7n5uZWoNdTqwWo1W+4L2whVS3r8NoWWy7lamud4+5sjSplHKBU8o0sJSqVmmMmYRw/6eMYSh/HUNpMcfxEX+BQp04d3LlzB8nJ/96KNTIyEp6enpDL/41Xt25dREZGan1uZGQk6tevX1RR86XZYusNE70yGdC/XTVe0EVERERUSKKXWDc3N7Rp0wYBAQFQKpVISEhAUFAQhg4dqnWej48PwsPDER4eDgA4fvw4bt++ja5dTesOV41qusG3dz24O2uvHXF3toZv73poVLNgM8dERERE9DrRlxMAwJIlSzB79my0bt0aNjY2GD58ODp27Ii9e/ciKioKc+bMQenSpREQEICFCxciOTkZlSpVQlBQEGxsbMSO/5pGNd3QsEYp3HqUAqUgg7lcQJUyDpyBJSIiIjIQmZDbrbGKsfj41CJ7LTMzOZydbZGYmG5y60hINxxDaeP4SR/HUPo4htIm1vi5udnne47oywmIiIiIiAqKJZaIiIiIJIclloiIiIgkhyWWiIiIiCSHJZaIiIiIJIclloiIiIgkhyWWiIiIiCSHJZaIiIiIJIclloiIiIgkhyWWiIiIiCSHJZaIiIiIJIclloiIiIgkhyWWiIiIiCRHJgiCIHYIIiIiIqKC4EwsEREREUkOSywRERERSQ5LLBERERFJDkssEREREUkOSywRERERSQ5LLBERERFJDkssEREREUkOSywRERERSQ5LLBERERFJDktsIWVlZWHu3Lnw9vaGl5cXli9fjtxugnb16lUMGDAA3t7e6N69O06ePClCWsqNLmOYk5ODwMBA9OzZE23btsUHH3yAa9euiZSY/kvX9+FLGRkZaN68OTZv3lyEKSkvuo6hIAjYunUrunTpgnbt2qFTp07IyckRITG9StfxO3LkCHr06IF27dqhX79+OHfunAhp6U0EQUBwcDAGDhz4xnNMqc+wxBaSv78/1Go1Dh8+jJCQEERERGDbtm1a56SlpWHMmDGYNGkSwsLCsGDBAvj5+SE+Pl6k1PQqXcbw7t27UKlU+Omnn3D8+HH4+PhgzJgx/MfTROgyhq/avn07UlJSijAh5UfXMQwKCkJoaCh27NiBY8eOYdu2bVAoFCIkplfpMn6xsbGYPn06/P39cezYMUyePBm+vr5ITU0VKTW96sSJE/Dx8cH69euRnJyc6zkm12cE0ltaWprg6ekpJCYmao4dPHhQ6NWrl9Z5P/74o+Dr66t1bPTo0cK3335bBCkpL7qOYW6aNGkixMTEGC8c6aSgY/jPP/8InTt3FsaPHy9s2rSpaEJSnnQdw2fPngn169cXHj16VLQBKU+6jt+RI0eEPn36aB1r06aNcPny5SJISfk5cOCAcOzYMeH06dNCly5dcj3H1PoMZ2IL4cqVKyhfvjycnJw0xzw9PRETEwOVSqU5dvHiRTRs2FDrcz09PXH9+vWiikpvoOsY/ldmZiYyMzNhb29fBCkpLwUdwy+++AJjxoyBra1tEaakvOg6hmFhYWjYsCHKlCkjQkp6E13Hr3HjxkhISND8+vn333+Ho6MjatasWdSRKRddunRB27Zt8zzH1PoMS2whxMfHw9XVVeuYi4sLlEql1q9H4uLiXjvP1dUViYmJRZKT3kzXMfyvVatWoWnTpvDw8DB2RMpHQcZw3759SEpKQu/evYswIeVH1zG8ceMGypUrh3nz5qF9+/bo1asXgoODizgt/Zeu4+fo6Ijp06dj+PDhaNCgAWbOnInFixfDwsKiqCOTnkytz7DEFoJSqXxt4bparQYAyGQyzbHcZoNUKpXWOSQOXcfwpYyMDMyYMQNnz57Fl19+WSQZKW+6jmFsbCxWrVoFf39/vvdMjK5jmJ6ejtDQUHTt2hVHjx6Fv78/li1bhjNnzhRpXtKm6/hdvnwZq1atQnBwMC5cuIDNmzdjwoQJePDgQZHmJf2ZWp9hiS0EJyen1376SEhIgKWlpdavmR0dHXM9z83NrUhy0pvpOoYAcP/+ffTr1w9mZmbYsWMHXFxcijIqvYEuY5iVlYUJEyZg6tSp/FW0CdL1fejs7Iw2bdqgZcuWkMlkqF27Nnx8fBAaGlrUkekVuo7fd999h8GDB6N27dqQyWRo2bIlOnXqhF27dhV1ZNKTqfUZlthCqFOnDu7cuaN1FV9kZCQ8PT0hl//7pa1bty4iIyO1PjcyMhL169cvqqj0BrqOYUpKCoYOHYqPP/4YS5YsgbW1tRhxKRe6jGF4eDhu376NefPmoXHjxmjcuDF+//13BAYGYtiwYWJFp//T9X1YtWpVpKena32uXC6HpaVlkWWl1+k6fjk5Oa/tJGFmZobs7Owiy0qFY2p9hiW2ENzc3NCmTRsEBARAqVQiISEBQUFBGDp0qNZ5Pj4+CA8PR3h4OADg+PHjuH37Nrp27SpGbHqFrmN44MABVKlSBQMGDBApKb2JLmPo7e2Ny5cv49y5c5r/vfvuuxg/fjy2bt0qYnoCdH8fdu3aFRcuXMCpU6cAALdu3cK+ffvQrVs3MWLT/xVk/LZt24ZHjx4BAK5du4bg4GB06tRJjNikB1PrM2aivGoxsmTJEsyePRutW7eGjY0Nhg8fjo4dO2Lv3r2IiorCnDlzULp0aQQEBGDhwoVITk5GpUqVEBQUBBsbG7HjE3Qbw7t37yIyMhLt27fX+twxY8aw2JoAXcaQTJsuY2hlZYV169Zh4cKFSEhIgIuLC5YsWYJatWqJHb/E02X8unfvjvT0dIwcORKZmZlwcHDAokWLXrvanUyLKfcZmfDf1dhERERERCaOywmIiIiISHJYYomIiIhIclhiiYiIiEhyWGKJiIiISHJYYomIiIhIclhiiYiIiEhyWGKJiIiISHJYYomIiIhIclhiiajEGzJkCL7++muxY+jlypUr6NatW573n9flHCIiqeFtZ4moWFi3bh0CAwNfO37lyhWYmYn7rW7IkCE4c+YMAMDGxgZVqlTBqFGj0KVLl0I/d926dfHHH39oHXv69CmsrKxgZ2f3xnMM7dWvv0wmg4eHBzp27IjJkydrcugiJSUFSqUSLi4uxopKRMUEZ2KJqNho164doqOjtf4ndoF9afr06YiOjsaxY8fw8ccfY8aMGTh+/LhRXqtNmza4c+eOUZ47Ly+//leuXMGWLVtw+fJlfP755wV6jr59+2oKPxFRXlhiiYiKkKOjI3r27IlBgwbh559/NsprqNVqozyvrhQKBWrUqIGJEyciLCysQJ8rdnYikg6WWCIq9q5du4YxY8agRYsWaNSoEcaNG4dnz5698fzt27ejS5cuqFevHjp06IArV65oHjtx4gR69eqFd955B927d0dwcLBemSpUqIDHjx8DAJRKJdavX4+OHTvi7bffRrt27bBmzRoolUrN+eHh4Rg4cCA8PT3RokUL7Nq1CwAQFRWFmjVrAgD27Nmj+e9+/fqhZs2aePDggdY5X3/9Nbp16/Zanp49e+Knn34CANy9excjR46Ep6cnvL29sWLFCr3W06alpcHGxkbzcVZWFpYtW4aOHTvC09MTPXv2xJ9//gkAiIiIQM2aNfHw4UP4+fmhZs2aiIiIMGgeIipeWGKJqNi7dOkSOnfujL179+LAgQN4+vQp1qxZk+u5YWFhCAoKwvr163H27FmsWLEC9vb2AIAzZ87gs88+w7Rp0xAREYHPP/8cS5cu1ZStgrh27RqqV68OAJg/fz5CQkKwfPlyREREYMWKFfj999+xbt06AEBCQgLGjBmDUaNG4cyZM9ixYweqVq362nP27dsX0dHRAIBffvkF0dHRKF++vNY5PXv2xL1793D9+nXNsevXr+PBgwfo0aMHEhIS8NFHH6Fdu3Y4efIkvv32W5w5cwabNm3S+c+mVCpx8eJFrFq1CqNGjdIcT0xMRFZWFjZu3IhTp06hZ8+e+PTTT5GRkYFmzZohOjoa5cqVw5o1axAdHY1mzZoZJA8RFU8ssURUbBw7dgw1a9bU/M/X1xcAMGjQIPTt2xfu7u5wc3NDv379cPHixVyf4969e3BxcUGVKlVgZWWFBg0aoGLFigCAwMBATJ48Ga1bt4a1tTUaN26M9957D3v37tU547Nnz/D999/j0KFDGDVqFB48eIDdu3cjICAADRs2hK2tLRo3bozPP/8c33zzDbKysvDkyRNkZ2fD09MTlpaWqFy5Mho2bKjX18jd3R2tWrVCSEiI5tjevXvRtWtX2NnZYceOHWjRogU+/PBD2NnZoVKlShg3bpxOf8aXX/+6deti3LhxmD9/PgYPHqx5vEyZMpg/fz6qVasGW1tbDBs2DKmpqbh79+4bn7MweYioeDONKx6IiAygXbt2uc7QpaWlYefOnTh37hzu3r2Lf/75B66urrk+h4+PD3755Rf06tULQ4cOhY+PDywsLAAAly9fRkREBD777DOtz2nRokW+2ZYvX47ly5fD1tYWTZs2xfbt21G1alUcOHAApUqVQq1atbTOb9SoEXJycnDv3j3UqlULXbt2hY+PDz744AN88MEHhbp6v0+fPli5ciU+/fRTqNVqhISEYPXq1Zo/4/Hjx19bJqFQKPJ93pdf/5s3b2LChAn466+/0KpVK61zjhw5giNHjuDGjRt4+PAh1Go1UlNT3/ichclDRMUbSywRFWs5OTl4//33UbduXYwcORKVK1fGmTNnsGLFilzPd3FxQXBwMEJCQvD9998jKCgIW7duRcWKFWFubo7169e/Vsx0MX36dIwYMeK14yqVCjKZLNfPkclkyM7Ohkwmw6pVq3DmzBl8//336NChAwICAuDt7V3gHADQoUMHzJs3D5cuXUJGRgbs7e01M7vm5uYYM2YMJk+erNdzA0C1atUQEBCA/v37491330XdunUBAJs3b8aePXswadIk+Pr6okyZMmjSpEmez2WIPERUPHE5AREVazExMbh16xYWL16MJk2aoFSpUrh161aen2NmZoZevXrhl19+QaVKlbBt2zYAQNWqVXH+/HmD5qtduzbi4+MRExOjdfzChQuwtLREtWrVNMeaNm2KwMBAjBo16o1reoEX5VcQhDc+bmlpie7du2P//v347bff0K9fP81jhvoz1q5dG3379sWiRYs0WX7//Xd88skn6Nq1KypWrIgnT54gMzPzteyv7lBgjK85ERUPLLFEVKy5urpCEAT88ccfyMzMRFhYWJ47CoSGhuL06dNIT09HXFwcEhMTUaFCBQDAiBEjsHXrVuzduxdpaWmIjY3F2rVrERUVpXe+KlWqoHv37pg8eTIuXryIjIwMnD9/HvPnz8eoUaNgbW2Nmzdv4rfffkNCQgLS0tLw4MEDTabceHh4ICIiAmlpaW+8ir937974888/ERERgV69emmODx48GH///TcCAgLw9OlTPHv2DL/++qtmN4SC8PPzQ0xMjGb9qpubG8LCwpCSkoL79+9j8eLFMDc3fy37uXPnkJmZifT0dIPmIaLihSWWiIo1Dw8PfP7551i5ciWaN2+OPXv2YOzYsW88X61WY+7cuWjevDkGDRqEdu3aaS5O6tSpE2bMmIHAwEA0b94cH330EZ4+fZpnodSFv78/2rZtiwkTJqBp06b47LPPMGTIEM2FaWZmZvj222/h7e2Njh074vnz51i4cOEbn2/atGn45ptv0L59eyQnJ+d6TsOGDaFSqVCvXj2t9bUeHh7YsmULTp8+jXbt2qFnz544ePAg3nnnnQL/uVxdXTF69GisWLECaWlpmD17Np49e4bWrVtjzJgxGDp0KKysrLQ+Z8KECQgNDUXLli1x584dg+YhouJFJuT1OyciIiIiIhPEmVgiIiIikhyWWCIiIiKSHJZYIiIiIpIcllgiIiIikhyWWCIiIiKSHJZYIiIiIpIcllgiIiIikhyWWCIiIiKSHJZYIiIiIpIcllgiIiIikhyWWCIiIiKSHJZYIiIiIpKc/wEcSFz0byWd+gAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 800x550 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from sklearn.metrics import roc_curve\n",
    "fpr, tpr, thresholds = roc_curve(y_test, model.decision_function(X_test))\n",
    "# 진짜 양성 비율, 거짓 양성 비율, 임계값 계산\n",
    "\n",
    "# ROC 커브 생성\n",
    "plt.plot(fpr, tpr, 'o-', label = \"LogisticRegression\") # 점과 선으로 그래프 그리기\n",
    "plt.plot([0,1],[0,1], \"k--\") # 대각선으로 서로를 잇는 직선\n",
    "plt.plot([1-specificity],[recall],'ro', ms= 10) # 임의의 특정 지점 cut-off value\n",
    "plt.xlabel(\"False Positive Rate\")\n",
    "plt.ylabel(\"True Positive Rate\")\n",
    "\n",
    "#AUC 값\n",
    "from sklearn.metrics import auc\n",
    "auc = auc(fpr, tpr)\n",
    "\n",
    "print(\"auc :\", auc)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "auc 값은 0.97 정도로 높은 편이지만, cut-off value(경계값) 값이 optimal 하다고 보기엔 힘들다.\n",
    "\n",
    "또한 이 데이터는 0이 굉장히 많은 불균형한 데이터이다. 이는 사기라고 예측되는 일이 많지 않기 때문에 cut off 자체를 낮게 잡아서 예측을 하면, 조금 더 optimal한 값을 얻을 수 있다."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 성능 향상 - Find Optimal Cut Off Value"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "대부분 통상적으로 최적의 cut off 값을 \n",
    "민감도(sensitivity) + 특이도(Specificity) - 1 = k(가장 큰)로 설정한다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [],
   "source": [
    "optimal_index = np.argmax(tpr - fpr)\n",
    "optimal_threshold = thresholds[optimal_index]\n",
    "\n",
    "optimal_pred = model.decision_function(X_test) > optimal_threshold"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 성능 향상에 대한 새로운 평가 지표 (Confusion Matrix)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [],
   "source": [
    "TN, FP, FN, TP = confusion_matrix(y_test, optimal_pred).ravel()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Accuracy(정확도) : 예측이 실제와 잘 맞은 비율"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.953626220362622\n"
     ]
    }
   ],
   "source": [
    "accuracy = (TP + TN) / (TP + TN + FP + FN)\n",
    "print(accuracy)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Precision(정밀도) : Positive(True) 라고 예측한 것 중에서 실제 값도 True인 경우"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.14426229508196722\n"
     ]
    }
   ],
   "source": [
    "precision = TP / (TP + FP)\n",
    "print(precision)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Recall(재현율) : 실제로 True인 것들 중에서 모델이 Positive(True)라고 잘 예측한 경우"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.8979591836734694\n"
     ]
    }
   ],
   "source": [
    "recall = TP / (TP + FN)\n",
    "print(recall)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* F1 Score : Precison(정밀도)과 Recall의 조화 평균\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.24858757062146894\n"
     ]
    }
   ],
   "source": [
    "f1_score = (2*(precision*recall))/(precision + recall)\n",
    "print(f1_score)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Specificity(특이도) : 실제 사기 x 인 사람 중에서 올바르게 분류된 비율 (negative로 판단한 것 중에서 실제 negative인 비율)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.9541058554598206\n"
     ]
    }
   ],
   "source": [
    "specificity = TN / (TN + FP)\n",
    "print(specificity)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### ROC 커브"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAArEAAAHnCAYAAACv5QVaAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8pXeV/AAAACXBIWXMAAA9hAAAPYQGoP6dpAABrWklEQVR4nO3dd1hTZ8MG8DsJew8Bt3XiqMW9RcCtFUddrbXWUQcO1DrrqlUrjuLC4mhrh6Ot1WKt1glqq4gLldeBuCqOCspeQpLz/eFnaipCCAknB+7fdb3XW04OyQ2P0ZuH5zxHJgiCACIiIiIiCZGLHYCIiIiIqKhYYomIiIhIclhiiYiIiEhyWGKJiIiISHJYYomIiIhIclhiiYiIiEhyWGKJiIiISHJYYomIiIhIcszEDlDSEhPTS+y15HIZXFxskZSUCbWa95SQIo6htHH8pI9jKH0cQ2kTa/zc3OwLPYczsUYkl8sgk8kgl8vEjkJ64hhKG8dP+jiG0scxlDZTHj+WWCIiIiKSHJZYIiIiIpIcllgiIiIikhyWWCIiIiKSHJZYIiIiIpIcllgiIiIikhyWWCIiIiKSHJZYIiIiIpIcllgiIiIikhyWWCIiIiKSHJZYIiIiIpIcllgiIiIikhwzsQMQUekjCAJuxKcgKS0Hdx6lQy0IKO9iA78mlSCXF/1nZ7VajfALD/A4ORseztZ6P48hCYKAa3ef4o8z95GZnYealRwx2K8mFAqFaJlM8ftkSKX966OSU9ifpbLyZ00QBMTeS8bVv5PxNCUHMpkMro5WqPeGMzyrOAEArv+djLy7yTCXATUrOkAmk4kb+iUyQRAEsUMAz7+Re/bswY4dO/DTTz/le87Vq1fx6aefIjExEdbW1pgzZw7atm1bpNdJTEw3RFydmJnJ4exsi+TkTCiV6hJ7XTIcjmHRnY9NxM6Im0hIyX7lMYVchi7Nq2CAby2dn29nxE0cOhsPlfrfv6p0fR5jjd/52ERs3nsFufk8Z+Pa5TDxnbcM9lq6Ks73yZS9GMMNv1zEgah7pe7rKwtM7e/Rwt4rpfW99F/nYxOx9WAsUrNy833c2tIM5mZypGX++7i7kzUG+NZCU083o+dzc7Mv9ByTKLEnTpzAihUrkJOTA4VCgQMHDrxyTkZGBnr06IGgoCC0adMGZ86cQUBAAP744w+4uen+zWSJpaLgGBbN+dhEfBkWg8L+VunesqpO/xjsjLiJP6Lu6f08xhi/87GJWP9rTIHnlHSRLe73yZSZmcmx5+Rd7Iq4+dpzpPz1lQWm9PdoYe+V6hXscefR63tCafmzpsvfY68jkwEBfRoavcjqUmJNYjlBdnY2pk2bBisrKyxYsCDfc/bt24eGDRuiTZs2AIAWLVqgWbNm2L9/P4YNG1aScYkoH4IgYGfEzUILLAAcPBOPRrVdC/z1nFqtxsEzr//HRpfnMVPIYZ/2DOnpOVCqiv+PpyAI+OHgtULPi457ghvxSSWytMAQ3ydTJpcBvx6/VeA5Uv76ygJDvw/1pct7paACC5SOP2uCIGDb4dhifD6w89hNNKlTTvSlBSZRYrt27QoAiIqKeu05Fy9eRJMmTbSOeXl54fr160V6LblcBrm8ZL7pCoVc6/9JejiGurv+d3K+SwjyoxYELN0aXezXNNTzGEPQtotiR9Aw5e+TIZT2r49MB/+sPZeQnI3bj9LgWdVZ1BwmUWJ1kZCQgJYtW2odc3V1xaVLl4r0PC4utiX+k4ODg3WJvh4ZHsewcHl3k8WOQEREJSRPLYOzs62oGSRTYlUqVb7HilpIk5IyS3Qm1sHBGmlp2VCJ+CsU0h/HUHfmRXxbdW9ZFS3qe7z28airj3GggLVrujyPXC6Dra0lMjOfQa0u/vL/v/9Jx7d/6Pbbn5b13NGtVbViv2ZhDPF9MmVnrj3GH6dL79dXFhj6fagvXd8rhZH6nzVd/h5LeXwTju41C+xY5nIBycmZho6noUtBlkyJdXR0RHKy9kxPUlJSkS7qAgC1WijxN5FKpRZ9MTsVD8ewcDUrOsDdyVqnJQUKuQzvdKhR4LqyKm62OPyfK4SL+jyGvqCkqrsd9v51G0/T87+a92Wj3q5XImtiDfF9MmXVK9jj0JnS+/WVBaZyYZcu75XClIY/a1Xd7bDv1F0kpua88ljesyxcPf4N4v93BI17TEWlut75Poe7szVqVHAQ/d9FyYxCgwYNEB2tvQ4lOjoajRo1EicQURn2Ym/BqKuPEXsvGYIgQCaTYYBvLejyy5EuzasU+o+AXC5Hl+ZViv08hiSTyTC4k2eh5zWuXa7E9os1xe+TIcnlcvTpULPAc6T89VHJ0eW9Ur1CwVfEl4Y/azKZDAP9ar9y/El8DE78EIj4/x0BAMRF7YQgvFpSZTJggE8t0S/qAiQ0E+vv74/NmzcjMjISrVu3xvHjx3H79m1069ZN7GhEZUp++8C+vHdgtxZVcfBMPNT5bFNQ1L0WX5xnSns2NvV0w/i+DU1qn1hT/D4Z0odvN0BOTh73iaVi0+W9Uhb2iX3x99jWQ7FISknH9ZNbcefCXs3jFT3bo3m3cbCzs0Lqy/vEOltjgE/J7BOrC5PYJ/aFqKgoLFiwQLNP7J49exATE4O5c+cCAP78808sWbIEqampqFatGhYuXAhPz8JnRV7GfWKpKDiG2graB1YmA7q1qIoDZ+69dputcb0boHm9oq8le3H3nISUbHg428C3cUWdZkOMOX4v7th14Mx9ZOXkoUZFRwwykTt2FfX7ZMpeHsPcXGWp+/rKAlP8e7Sw90ppfC/l58KFcxg99iPcu/t8GztrWwcMHTMHQ98bjDr/f8euWw/ToBRkMJcLqFGh5O7YJZmbHZQkllgqCo7hvwRBwOyNpwtc8yqXAQUtN3N3tsbS0a1K7C9Bjp/0cQylj2NomrZu/Q7Tp0/WXDjfuXNXBAevg4dHea3zxBo/XUps6fuxgoiM4kZ8SqEXbRV2vURCcjbi7qcaMBUREemjefOWMDMzg62tHVatCsHWrT+/UmBNnWTWxBKRuFIyCr8iX7fneWaQ5yEiIt2pVCqo1WqYm5sDADw96+LLLzfDy6sxqlY1/naAxsCZWCLSiZOdhYGex9Igz0NERLq5ffsWevfujmXLlmgd79Wrj2QLLMASS0Q6qlPFCe5OBd+5TFHIjUTcna1Ru7KjIWMREdFrCIKALVu+gp9fW5w5cxohIatx6VLpuW0uSyxRCcpvf9WSoFQqsX53DOZ/HYX1u2OgVCqL/ByF7QMrkz3fQ7Ggx01lb0EiotLu4cMHGDiwD2bOnIqsrCyYmZlhxoxP0KBBQ7GjGQzXxBKVkML2VzWWRd+dxZ1H/+7KcT8xE6NXJqJ6BXvMG9a8SM/V1NMNb5S313q+F94ob48BvrVQo6Ijdh67iYTkl75OE9tbkIiotBIEATt3/ohPPpmBtLTnF9LWq1cfISEb0bChl8jpDItbbBkRtxWRPkONYWH7qwb0aWiUgvffAvtfRS2y63ZdRnTck9c+/mKjf0EQcCM+BamZuXCys0Ttyo6izMDyPSh9HEPp4xiWnKSkp5g6dRL2739+4wKZTIbx4wMxc+YcWFrqdz2CKW+xxZlYIiMTBAE7I26+9gYAggBsP3IDjnbmBi16SqWywAILAHcepSP23lOYmRX+V4FKpSqwwAJAdNwTqFQqKBQKeFZ1LlJeIiIqHoVCgYsXLwAA3nijOtat24iWLVuJnMp4WGKJjEyX/VWT05/h8x8ulFAibcu2XzLo8/0ccQvvdqpj0OckIqLCOTo6YfXq9di/fy/mz18EOzs7sSMZFS/sIjIyQ+2vKhWPkrLEjkBEVCYcPx6BqVMnal0k7OPjh+XLV5X6AgtwJpbI6HTdX3Vo1zqo6lH4GiBd/XLsJmLvFX53rLrVHPFOh1qFnnfwzD2cu55Y6HkVXGx0ykdERPrJzMzEokXz8c03mwEAjRs3xdChH4obSgQssURG9mJ/1YKWFLg7W8OnUSWDron9eKAXRq88Ueh5Uwd46bQmdkyv+jh3/Xih5w30ralTPiIiKrozZ6IwceIY3LlzGwBQrlw5uLm5i5xKHFxOQDoTa4/TF9RqNY6ci8e2wzdw5Fw81GppXOWqy/6qxtg/1czMDNUrFDyzW72CvU4FFnh+wUDj2uUKPKdx7XJQKBQ6ZyQiIt08e/YMixYtgL9/V02B7dGjF44fj0K3bj1ETicOzsSSTsTa4/SFnRE3cehsPFTqf4vzT+E30aV5FQzwLfxX4WJr6umGbi2q4uCZeKhfKv8KuQxdmlcx2vdw3rDmr91mS599Yie+89Zrt9l6sb0WEREZVkzMZUyYMBrXrl0FADg4OOLzz5djwIDBZfoGMtwn1ohKy954Yu1x+sLOiJv4I+reax/v3rKq0Yqs1PeJfUGpVGLjb9eQkJIFD2cbjO5VT+cZ2PyoVCr8FH4L/yRnoYKLDQb61jTJGdjS8h4syziG0scxLL533vHHn38eAwB06OCL1avXo1KlyiXy2twnliRLrD1OX1Cr1Th45vUFFgAOnolHo9qukMsNvzrGTCGHfdozpKfnQKnS780rCAK2H7lR4Pdw57GbaFKnnNF+ojYzM8P4foa71aBCocB7nbmNFhFRSfjiizXo0aMTpk2bheHDR5Xp2deXscRSgUx9j1MAUAsClm6NFu31DSEhORtx91NRp4qT2FGIiEhEarUaO3ZsRZ8+78DW1hbA8xsXnD//P1hbW4uczrTwwi4qUFnb41RMKRnPxI5AREQiio+/h/79/TFlygQsXrxA6zEW2FdxJpYKJNYepy+cvfYYh87eL/S8Li0qo3ldD4O/vplCDnt7q2ItJ7j3OB0/HLxR6HlOdvrd15qIiKRNEARs3/4D5s2bjYyM59funDkThZycHFhZWYmcznSxxFKBxNrj9IXq5e1x9PwDrV0J/kshl2GgTy3jrIk1wIL2GhUccDAqvtDvYe3KjvrGJCIiiXr8+B98/PEkHDp0AAAgl8sRGDgVH388CxYWuk0klVVcTiBxxt67Vaw9Tl+Qy+Xo0rxKged0aV7FKAXWUMT+HhIRkWnas2c3vL1bagpsrVq1sW/fYcyePZ8FVgeciZWwktq7Vaw9Tl94sX3Wf/eJffH6UtknNqBPQ+w8dhMJyS+Nl7M1BviUzF67RERkOnbs2IrAwADNx6NHj8MnnyyAjQ1v3a0r7hNrRMbcW60k9x0Ve4/TF9RqNcIvPEBCSjY8nG3g27ii0WdgDT2GgiDgRnwKUjNz4WRnidqVHTkDa0Tcn1L6OIbSxzHMX2ZmJnx920ClUmHNmi/Rrp232JHyxX1iyaBKcu9WU9jj9AW5XI5OzQpeWmDqZDIZPKs6ix2DiIhKWEZGOjIzM+HhUR4AYGtri61bf0aFChVgb+8gcjppYomVIFPbu5V7nBIREb1eZORJTJw4DpUrV8bu3b9rfotYp46nyMmkzXSvhqHXMsW9W7nHKRERkbbs7GzMmzcbffr0wL17d3Hq1F84fPig2LFKDc7EStDLe7e6pD9F43sXYfMsG1mW1oiu2ghJ9q4ADLN3K/c4JSIiKrqLFy9gwoQxuHEjFgDg5OSEoKAv0KVLN5GTlR4ssRJUp4oTmmfGo9PRbWhx+yzM1CrNY0q5AmdqNMeRTkPg08i32OtUuccpERGR7vLy8hAcvByrV6+ESvX83+eOHTtj1aoQlC9fQeR0pQtLrARZ7v8dc7Z8DEXuq8sKzNQqtLl5Gi3vXUBGK1fk9ni7WK/1Yo/TgnYn4B6nREREQFLSUwwc2BeXL18EANja2uGzzz7H++8P47+TRsA1sRJjdvECHMaOyLfAvkyRmwuHMcNhdrH4F3e92OPU3Vn7vs3uztYltr0WERGRqXN2doGr6/Mlfa1bt8WxY6cwdOiHLLBGwplYibFZ/QVkz3S7iEr27Bls1gQjbcvWYr9uU083NKlTjnucEhERvUSlUkGhUAB4/tvL1avXY+/eMIwaNdak7yZZGvC7KyHyRw9hcXB/kT7H4uB+yB89NMjrv9jjtEU9D9Sp4sQCS0REZZYgCPj226/h49MaaWmpmuMVKlTE6NEBLLAlgN9hCTE/HgGZSlX4iS+RKZUwPx5hpERERERlz6NHDzF4cD/MmDEFsbHXsWDBHLEjlUkssRIiT0/T6/NkGSV3q10iIqLSShAE7Nz5I7y9WyEi4igAoG7dehg+fJTIycomromVELWet6UT7Iq3VywREVFZ9+TJE0yfPhn79v0G4PkSu4CASZg5cw6srKxETlc2scRKSF4HXwhmZpAplTp/jmBmhrwOvkZMRUREVLr98cc+fPzxJDx5kggAqFbtDaxbtxGtWrUWOVnZxuUEJk4QBMTeS0bU1ce4lmeNZ0W800du1x5QV6hopHRERESlX1RUpKbADhs2EhERp1hgTQBnYk3Y+dhE7Iy4qXW3rDrOPliqOAgLVV6hny9YWSErcKoRExIREZV+s2bNxdWr/8PYsRPg59dJ7Dj0/zgTa6LOxybiy7CYV273esOjFlb0/Bi5CvMCP1+wtETahm+gbNTEmDGJiIhKlczMTMydOxNxcTc0x6ysrPDzz2EssCaGJdYECYKAnRE3873NKwCcrtUKMwctxalaraCUK7QeU8oVOFWrFZ6G/VHsW84SERGVJWfPRsHPry02bQrFxIljoCzCNShU8ricwATdiE95ZQb2v26Wr4Wl/rPgkv4Uje5dgk1uFrIsbHCxqheS7F3ROdkB75ZQXiIiIil79uwZVqxYipCQ1VCr1QAAd/fyyMrKhIODo8jp6HVYYk1QSkauzucm2bsivIHfK8cfJWUZMhIREVGpFBNzGRMmjMG1a1cAAPb2DliyZBkGDXqPd6Y0cSyxJsjJzqLYz1HBxcYASYiIiEonpVKJdetWYeXKIOTlPb9Yun17H6xZsx6VK1cROR3pgiXWiARBwP9uPUH8o1TYW5ujThUnnX6qq1PFCe5O1oUuKSjIQN+aen8uERFRaXf06GEsXboIAGBtbY358xdh+PBRkMt5uZBUsMQayfnYROw8dhMJyf8WUXcnawzwrYWmnm4Ffq5MJsMA31r48tcYvObargI1rl0OCoWi8BOJiIjKqC5duqFHj15ISHiMkJANqFGjltiRqIhYYo3gxfZY/91dICElG1+GxSCgT8NCiywAvQvsxHfe0uMziYiISq/4+Ht48OCB5iYFMpkM69aFwsbGlhM/EsUSa2CFbY8lCMD2IzfgaGf+2qUFgiBg+5Eb+T72go2VAr6NKuKfpBw8y1OigqstBvrW5BuRiIjoJYIg4Mcft2HOnJmwtrbGiRNRcHV1BfD8Ii6SLpZYA9Nle6zk9Gf4/IcLxXqdrBwVGtZ0wzs+TsV6HiIiotLq8ePHmDZtEg4e/AMAkJWViT//PIY+fd4RNRcZBlcvG1hRtscq/ms9K7HXIiIikpK9e8PQoUNLTYGtWbMWfv/9EAtsKcKZWAPTdXusoV3roKqHfb6P3Xucjh8OFryc4PlrWRYpGxERUWmXnJyE2bOnY/funZpjH300FnPmfAobG24/WZqwxBqYLttjuTtbw6dRpdeuia1RwQEHo+ILfY7alXkXESIioheys7PRqZM34uPvAQAqV66CNWu+RPv2HURORsbA5QQG9mJ7rNdtByuTAQN8ahW4X6whnoOIiKissba2xuDBQwAA7777Po4dO8UCW4qxxBpBU083NKpVLt/HGtUqp9P2Wk093RDQpyHcna21jrs7W+u8RRcREVFp988/j7Q+njx5Gn755TesWfMlHBz4G8vSjMsJjGBnxE1Exz3J97HouCfYGXETA3wL31S5qacbmtQphxvxKUjNzIWTnSVqV3bkDCwREZV5OTk5+Pzzz7Bly2bs338UDRs+3yPd3Nwc3t4+4oajEsGZWANTq9U4dDa+wHMOnY2HWq3W6flkMhk8qzqjRT0PnW9bS0REVJpdvHgBnTq1x4YNIXj27Bnmzp0pdiQSAUusgYVfeACVuuB7banUAiKiH5ZQIiIiotIhLy8Py5d/ju7dO+LGjVgAgJ9fJ2zc+I3IyUgMXE5gYI+TC77Rwb/nZRk5CRERUekRG3sdEyaMwaVL0QAAGxtbfPbZ5xg69EP+lrKMYok1MI//XIj1+vO4Vx0REZEuvv9+C+bMmYFnz57f5Kd167ZYs+ZLvPFGdZGTkZi4nMDA/JpUgkJe8E+ECrkMvo0rllAiIiIiaXN1LYdnz57B0tISCxd+jl9/3ccCS5yJNTS5XI4uzavgj6h7rz2nS/MqkMv58wMREVF+BEHQWiLQs2cvzJ49Dz169IKnZ10Rk5EpYZMyggG+tVC9Qv63lK1ewV6n7bWIiIjKokePHuLdd9/Br7/+onV8ypTpLLCkhSXWCHZG3MSdR+n5PnbnUTp2Rtws4URERESmTRAE7Nr1M7y9WyE8/Ahmzpz6yo0MiF7GEmtght4nloiIqLR7+vQpRo0ahnHjRiE1NQUymQzvvjsUTk7OYkcjE8Y1sQZWlH1iOzatXEKpiIiITNPBg39g6tSJSExMAABUrfoG1q0LRevWbUVORqaOM7EGxn1iiYiICpeenobAwAAMHTpIU2CHDh2OY8dOssCSTjgTa2DcJ5aIiKhwDx8+xO7dOwEAHh7lsXp1CDp27CJyKpISk5iJzcnJwbx58+Dr6wtvb28sX74cgvDqr+SPHDmCnj17wsfHB/3798e5c+dESFsw7hNLRERUOE/Pupg1ax769euPEydOs8BSkZlEiQ0KCoJarcbhw4exb98+REVFYevWrVrnxMfHY8aMGQgKCsKxY8cwZcoUBAQEID09/10AxPJin9iCcJ9YIiIqa86ePYPNm0O1jgUETMSGDd/A2dlFpFQkZaI3qczMTISFhWH69OkwMzODvb09xowZg127dmmdd+PGDbzxxhto2LAhAKBt27awsrLC3bt3RUhdMO4TS0RE9Fxubi7mzJmD7t07Yd682ThzJkrz2Ms3NCAqKtHXxF65cgWVK1eGk5OT5piXlxfi4uKgUqmgUCgAAM2aNUNSUhJOnjyJtm3b4vfff4ejoyM8PT2L9HpyuQzyQn7dX1w/HY0rcJ/YXcdvYVDH2kbNQIahUMi1/p+kheMnfRxDabty5X8YN+4j/O9/MQAAOzt7PH2aADMzjqdUmPJ7UPQSm5iYCFdXV61jLi4uUCqVSE9P15RbR0dHzJgxAyNGjICNjQ3y8vKwbds2WFhYFOn1XFxsjfqTn1qtxoECbjkLAAei7mF0v7e4pEBCHBx0u2CPTBPHT/o4htKiVCqxYsUKLFiwAHl5eQAAX19fbNmyBdWqVRM5HenDFN+DopdYpVL5ykVcL24E8HLZvHz5MlatWoWwsDDUrVsXkZGRmDhxIrZv347KlXXfbzUpKdOoM7GHztzTaZ/YX47EonPzqkbLQYahUMjh4GCNtLRsqFS8QYXUcPykj2MoPbdu3cS4caNx7twZAIC1tTWWLVuGoUNHQBCA5ORMkRNSUYj1HnR2ti30HNFLrJOTE5KTk7WOJSUlwdLSEvb2/64r/e677zBkyBDUq1cPANCmTRt07twZO3fuxJQpU3R+PbVagLqQklkcj57qtv/ro6dZUCr5F7JUqFRqjpeEcfykj2MoDSqVCoMH98etW89vr960aTOEhm5G8+aNkJycyTGUMFN8D4r+++z69evjzp07SE1N1RyLjo6Gl5eX1q/b8/LyNOtjXzAzM0Nubm6JZdUF94klIqKySqFQ4PPPV8DCwgKffDIfe/ceQq1avAaEjEP0Euvm5ob27dsjODgYSqUSSUlJCA0NxbBhw7TO69atG7Zu3YqHDx8CAK5du4awsDB07txZjNivxX1iiYiorBAEAeHhR7SWBfr6dsTZs5cxefI0mJmJ/gtfKsVEL7EAsGTJEiQkJKBdu3bo378/Bg0ahE6dOmHPnj1YvHgxAKBHjx4YNWoURo0aBV9fX8yaNQuLFi1CkyZNRE6vjfvEEhFRWZCQkIBhw97F4MH9sG3b91qPVajAiRoyPpmQ362xSrHEROPfHOF8bCI2772C3HzWjjSuXQ4T33nL6BnIMMzM5HB2tuVaLoni+Ekfx9A07d27BzNmTMbTp08BAG+91QiHDh3Ld4KGYyhtYo2fm1v+++2/jNOBBnY+NhHrf43Jt8ACwMW4Jzgfm1jCqYiIiIovJSUZ48aNwsiRQzUFduTI0diz5w/+hpFKHBerGJAgCPg5PK7gcwDsPHYTTeqU451KiIhIMsLDj2DKlAl49Oj5tSmVKlXG6tXr0aGDr8jJqKzij00GdCM+BYmpOYWel5Ccjbj7qYWeR0REZAo2bw7F4MH9NAV20KD3cPx4JAssiYol1oBSMnTf7isl45kRkxARERlOp05dYWNji3Ll3PDddzuwbt0GODg4ih2LyjguJzAgJzvdb4HrZGdpxCRERET6y8nJgSAIsLZ+vvd59eo1sGXLVjRs6IVy5cqJnI7oOc7EGlCdKk5wc7Qq9Dx3Z2vUrsyfYImIyPRcvnwRXbp0wGefzdM67uvbkQWWTApLrAHJZDIM9Cv4ziQyAAN8avGiLiIiMil5eXlYuTII3br54fr1a/j66024ePGC2LGIXovLCQysqacbGtcuh+i4J688Zmkux6i3G6Cpp5sIyYiIiPJ340YsJkwYjYsXowEANja2+PTTxfDyaixyMqLX40ysge2MuJlvgQWAZ3lq3H7IXQmIiMg0qNVqhIaGoGPHdpoC27Jla0REnMSHH47kbw3JpLHEGpBarcahs/EFnnPobDzUat6xhIiIxPX48T/o27cnFiz4BM+ePYOFhQUWLFiMsLD9qF69htjxiArFEmtA4RceQKUu+C6+KrWAiOiHJZSIiIgof7a2dnj48AGA57eNPXLkT4wfPwkKhULkZES64ZpYA3qcnK3jeVlGTkJERFQwOzs7rFu3AcePR2Dq1BkwNzcXOxJRkXAm1oA8nK11PM/GyEmIiIj+JQgCdu/eieHD39da0taqVRvMnDmHBZYkiSXWgPyaVIJCXvAieIVcBt/GFUsoERERlXVPnz7FRx99iLFjR2Lfvt/w1VcbxI5EZBAssQYkl8vRpXmVAs/p0rwK5HJ+24mIyPgOHvwD3t4t8dtvvwIAqlathoYNvURORWQYbFMGNsC3Frq3rAr5f7YlUchl6N6yKgb41hIpGRERlRXp6WkIDAzA0KGDkJiYAAAYOnQ4jh07hdat24qcjsgweGGXEQzwrYVGtV2xdOvzPfe6t6yKdzrU4AwsEREZ3V9/ncCkSeNw//7zLR89PMpj9eoQdOzYReRkRIbFVmUkLxfWFvU9WGCJiKhErFu3SlNg+/XrjxMnTrPAUqnEZkVERFSKrFoVgjfeqI6vvvoOGzZ8A2dnF7EjERkFSywREZFE5ebmYu3aVUhPT9Mcq1ixEk6dOg9//74iJiMyPq6JJSIikqCrV69gwoQx+N//LuP27ZtYvXq95jEzM/7zTqUfZ2KJiIgkRKVSYe3aVejc2Rv/+99lAMC9e3/j2bNnIicjKln8UY2IiEgibt++hYkTx+Ls2SgAgJWVFebNW4iRI8fwAmIqc1hiiYiITJxarcaWLV9h0aL5yMrKAgA0adIUISGbUKtWbZHTEYmDJZaIiMjE/fDDt5g9exoAwNzcHNOmzcLEiVO49pXKNP7pJyIiMnGDBr2Hr77aALlcjpCQTWjY8C2xIxGJjiWWiIjIxCQmJiIrKxPVqr0B4Pna123bdsLDozwsLS3FDUdkIrgKnIiIyITs27cXHTq0xJgxw6FUKjXHq1atxgJL9BKWWCIiIhOQmpqCgICPMHz4EDx58gQXLpzHiRMRYsciMlkssURERCKLiDgKb+9W+OWXnwA8v+vWzp174OfXWeRkRKaLJZaIiEgkGRkZmDFjCgYN6otHjx4CeH4R1/HjkejQwVfkdESmjRd2ERERieDJkyfo0aMj7t69AwAoV64cVq5cix493hY5GZE0cCaWiIhIBK6urqhXrwEAoEePXjh+PIoFlqgIOBNLRERUQrKzs2FtbQ0AkMlkWLlyDfz9+6BfvwGQyWQipyOSFs7EEhERGZlSqURw8HK0adMUSUlPNcfd3NzwzjsDWWCJ9MASS0REZERxcTfQs2cnBAUtxoMH9zF//idiRyIqFVhiiYiIjECtVmPjxvXo2LEdoqMvAACaN2+JqVNniJyMqHTgmlgiIiIDu3fvb0yaNA6nTv0FALCwsMDMmXMREDARCoVC5HREpQNLLBERkQHt2LEVn3wyA5mZGQCAN998CyEhG1G/fgORkxGVLlxOQEREZEDx8feQmZkBhUKBqVNn4MCBcBZYIiPgTCwREZEBTZkyHbdv38To0QFo0qSZ2HGISi3OxBIREekpKekpxo0bhf/9L0ZzzNzcHBs2fMMCS2RknIklIiLSw+HDBzBlykQkJDzGtWtXcfBgBCwtLcWORVRmcCaWiIioCNLT0zBlygQMGTIQCQmPAQCNGzeBUqkUORlR2cKZWCIiIh2dPPknJk0ah/j4ewAAd3cPrFq1Dp07dxM5GVHZw5lYIiKiQmRnZ2PevFno27enpsD26dMPJ06cZoElEglnYomIiApx8uQJbNz4JQDA2dkZy5YFo0+fd0RORVS2scQSEREVolOnrujffxBSU1MQHLwOHh7lxY5EVOaxxBIREf3HtWtX8ejRQ/j5ddIcCw5eB0tLS8hkMhGTEdELXBNLRET0/1QqFdatW43Onb0xbtxIPH78j+YxKysrFlgiE6J3iT179izmzp2LkSNHao5duXIFaWlpBglGRERUkm7fvoXevbtj0aL5yM3NRVZWFi5cOC92LCJ6Db1K7J49ezBp0iQ4Ojri1KlTmuPh4eEIDg42WDgiIiJjEwQBW7Z8BT+/tjhz5jSA5/u+Hj36F7p37ylyOiJ6Hb1K7ObNmxESEoLp06dDEATN8T59+uDEiRMGC0dERGRMDx8+wMCBfTBz5lRkZWXBzMwMM2fOwb59R1CnjqfY8YioAHpd2BUfH48333wTALTWBzk7O+PJkyeGSUZERGREWVlZ6Ny5AxITEwAAdevWQ0jIRrz1ViNxgxGRTvSaia1evTouXLgAAFozseHh4ShfntuOEBGR6bOxsUFAwCTIZDJMmDAZhw+fYIElkhC9ZmKnTp2KGTNmYOrUqZDJZIiMjMS5c+ewZcsWzJo1y9AZiYiIDCIu7gZq166j+Xjs2PFo396b5ZVIgvSaifX29sYXX3yBsLAwWFlZITAwECdPnsTy5csxcOBAQ2ckIiIqltTUFEyYMAbt27fA2bNRmuMKhYIFlkii9L7ZQYsWLdCiRQtDZiEiIjK448cjEBgYgIcPHwAAli5dhN27fxc5FREVl14zsfXq1cv3eEpKCtq3b1+sQERERIaQmZmJWbM+xoABvTUFdsCAwfjmmx9ETkZEhqDzTKxKpUJ6ejoEQYAgCEhNTdW6qEutVuP06dPIyckxSlAiIiJdnTkThYkTx+DOndsAAFdXV6xYsQZvv+0vcjIiMhSdS+w///yDjh07QiaTQSaToVWrVlqPC4IAc3NzfPzxxwYPSUREpKv169di0aL5UKvVAIBu3Xpi5co1cHd3FzkZERmSziW2UqVKiIiIgEqlQqdOnXD06FGtx+VyOVxdXWFhYWHwkERERLqqV68+1Go17O0d8PnnyzFw4Ltae5oTUelQpAu7KlSoAACYMGECKlWqZJRARERERaFUKqFQKDRF1c+vE4KCvkDXrt1RqVJlkdMRkbHodWHXhAkTDJ2DiIioyG7ejMPbb3fG9u3aF2uNGPERCyxRKaf3bWcXLlyIS5cuISMj45XHr127VuxgREREr6NWq/HVVxuwePGnyMnJQWxsLHx8/FhcicoQvWZi582bB2tra6xduxbW1tb46aefsGTJEtSpUwfr1683dEYiIiKN+Ph76N/fH3PnzkJOTg4sLCwwZcp0lC9fQexoRFSC9JqJvXTpEk6cOAF7e3tUrFgR5cuXx1tvvYWqVasiJCQEfn5+RXq+nJwcLFmyBH/99RdUKhXefvttTJ8+/ZWF+IIg4Ntvv8WPP/6IZ8+ewdzcHPv374e5ubk+XwYREUmIIAjYsWMr5s6dhYyMdADAm2++hZCQjahfv4HI6YiopOlVYu3s7JCZmQl7e3tUr14dt27dgru7O958801cvHixyM8XFBQEtVqNw4cPIzs7Gx9++CG2bt2KoUOHap0XGhqKyMhIbN++Ha6urnj8+DEUCoU+XwIREUnI48ePERg4HocOHQDw/HaxgYFTMXXqTO6KQ1RG6bWcoEWLFjh06BAAoFWrVvjuu+/w4MEDbN26FR4eHkV6rszMTISFhWH69OkwMzODvb09xowZg127dmmdl5SUhM2bN2P58uVwdXUFAHh4eEAu1+tLICIiCcnMzMBff50AANSqVRv79h3GrFnzWGCJyjC9ZmKnTJmC69evAwAGDBiA8PBwdOzYEba2tli5cmWRnuvKlSuoXLkynJycNMe8vLwQFxcHlUqlmWmNiIhAkyZNNNt86Usul0EuN/5+gWaKf8u1XC6DmRnLthQp/n8cFQqOnxRx/KTvxdjVrl0bixZ9jhs3bmD+/IWwtrYWORnpiu9DaTPl8dOrxFauXBmVKz+/AtTCwgJff/010tLSYGNjAzOzoj1lYmKiZmb1BRcXFyiVSqSnp2vK7Y0bN1CpUiXMnz8ff/31F+zt7TF8+HD06dOnSK/n4mJbIpte26c90/y3ra0lnJ1tjf6aZDwODvwHU8o4ftKzf/9+nD9/HvPmzQPwfAynTg0UORUVB9+H0maK46dXic2Pg4MD4uPjsWTJEmzYsEHnz1MqlRAEQevYi1sFvlw2MzMzcezYMSxfvhwLFy7E9evXMWLECFSsWBEtWrTQ+fWSkjJLZCY2PT1H89+Zmc+QnJxp9Nckw1Mo5HBwsEZaWjZUKrXYcaiIOH7Sk56ejnnzZuP7778FADRq1Ay9enXnGEoY34fSJtb46TL5V6QSu3v3bkRERMDGxgYffPABGjR4fjVoTk4OQkNDsWXLFrRs2bJIIZ2cnJCcnKx1LCkpCZaWlrC3t9ccc3Z2Rvv27dGmTRsAQL169eDv74/w8PAilVi1WoBaLRR+YjEpXxpotVqAUsk3rpSpVGqOoYRx/KTh1Km/MGnSONy79zcAwM3NHbm5eQA4hqUBx1DaTHH8dF7g8P3332P58uWoXr06bG1tMXLkSNy5cwcRERHo2rUrwsPDsX79emzevLlIAerXr487d+4gNTVVcyw6OhpeXl5aF23VrFkTmZnas5lyuRyWlpZFej0iIjIt2dnZmDdvNvr27akpsL1798Off0bBz6+jyOmIyFTpPBO7Y8cOrFq1Cq1btwYANGvWDHPmzMHNmzcRGBiId999V6+dAtzc3NC+fXsEBwdj3rx5SEtLQ2hoKAIDtdc+devWDStXrsSpU6fQpk0b3Lp1C3v37sVXX31V5NckIiLTcPHiBUyYMAY3bsQCeP7buWXLgtG3b3+RkxGRqdO5xN6/fx9NmzbVfOzn54epU6di48aN6NChQ7FCLFmyBHPmzEG7du1gY2ODESNGoFOnTtizZw9iYmIwd+5cWFlZYd26dVi4cCGSkpLg4uKCJUuWoG7dusV6bSIiEodKpcK4caNw69ZNAEDHjp2xalUI77xFRDrRucQqlUqt/fisrKwgk8ng7e1d7BAuLi4IDQ195Xjv3r3Ru3dvzceNGzdGWFhYsV+PiIjEp1AosGpVCIYMGYiFC5dgyJAPSmT3GCIqHXQusS9u+fryTgIvjv3X8OHDDRKOiIhKD5VKhbCwXejbt79m+VmrVm0QHX0FDg6OIqcjIqnRucQ2b94cR48efeVYeHi41jGZTMYSS0REWu7cuY1Jk8YhKioSSUlP8dFH4zSPscASkT50LrE//PCDMXMQEVEpJAgCvvvuG3z66VxkZT3fYeb333/DyJFjeNtwIioWg93sgIiI6GWPHj3E5MnjERHx/Ld4ZmZmmDp1BgIDP2aBJaJiY4klIiKDEgQBu3b9jNmzpyM1NQUA4OlZFyEhG+Hl1VjccERUavBHYSIiMqi1a4MREPARUlNTIJPJMH58IA4fPsECS0QGxRJLREQG1b//IDg4OKJatTewZ88fWLBgEaysrMSORUSlDJcTEBFRsaSlpUIul8POzh4AUKlSZezY8Qvq1WsAOzs7kdMRUWnFmVgiItLbiRPH0KFDa8ydO0vrePPmLVlgicio9C6xZ8+exdy5czFy5EjNsStXriAtLc0gwYiIyHRlZWVh9uxp6N/fHw8e3Mf27T/g8uWLYsciojJErxK7Z88eTJo0CY6Ojjh16pTmeHh4OIKDgw0WjoiITM/Zs1Hw82uLr7/eBABwdXXF11//gLfeaiRuMCIqU/QqsZs3b0ZISAimT5+udRvaPn364MSJEwYLR0REpuPZs2dYvPhT9OrVFbdv3wIAdOvWE8ePR6FXr94ipyOiskavC7vi4+Px5ptvAnh+m9kXnJ2d8eTJE8MkIyIikxEffw9Dhw7G1av/AwDY2ztgyZJlGDToPa1/B4iISopeM7HVq1fHhQsXAEBrJjY8PBzly5c3TDIiIjIZbm7uUCrzAADt2/vg+PFIDB48hAWWiESj10zs1KlTMWPGDEydOhUymQyRkZE4d+4ctmzZglmzZhX+BEREZPIEQdCUVCsrK4SEbMT582cxfPhHvG0sEYlOr7+FvL298cUXXyAsLAxWVlYIDAzEyZMnsXz5cgwcONDQGYmIqASp1Wps3hyKd97pBaVSqTneqFETjBw5hgWWiEyCXjOxf//9N1q0aIEWLVoYOg8REYkoPv4eJk8ejz//PA4ACAlZjcmTp4mciojoVXr9ON2zZ08MHToUv/32G3Jzcw2diYiISpggCNixYys6dGitKbD167+Jzp27iZyMiCh/epXYEydOoFOnTvj+++/Rrl07fPbZZ7h27ZqhsxERUQl4/Pgxhg17F4GBAcjISIdcLsfkydNw6NAxNGjwptjxiIjypddyAhcXFwwbNgzDhg3D7du3sXfvXkyaNAn29vYYOHAgBg8ebOicRERkBHv3hmH69MlISkoCANSsWQvr1m1As2ZcLkZEpq3Yq/Nr1KiB8ePHY9asWXB0dMSiRYsMkYuIiIxMEAT88svPmgI7atQYHD36FwssEUmCXjOxwPOrVyMjI7Fv3z4cPXoULi4u6Nu3L5YuXWrIfEREZCQymQwrV67Bw4cPMG/eQnh7+4gdiYhIZ3qV2E8//RSHDx+GUqlE9+7dsWnTJnh5eRk6GxERGVBGRjpWrlyGSZOmwMXFFQDg5uaGQ4eO8aYFRCQ5epXYf/75B/PmzYOfnx8sLCwMnYmIiAzs9OlTmDBhLO7du4sHD+5j8+ZvNY+xwBKRFOlVYjds2GDoHEREZAQ5OTlYunQRNmwI0dwmXK1W49mzZ7C0tBQ5HRGR/nQqsdOmTcPKlSs1H3/wwQcFnv/9998XLxURERXbxYsXMGHCGNy4EQsAcHR0QlDQSvTrN4Czr0QkeTqV2CZNmmh93KFDB/4FSERkovLy8rBq1QqsWrUCKpUKAODn1wmrVoWgQoWKIqcjIjIMnUrse++9p/XxyJEjjRKGiIiK74cfvsXKlUEAABsbW3z22ecYOvRDTj4QUami1z6xHTt2zPd4SkoKb3RARCSyoUM/xFtvNUKrVm1w7NgpfPDBcBZYIip19Lqw68GDB/ke//vvvxEbG1usQEREVDR3795Bbm4u6tTxBACYm5tjx45dcHFxgUKhEDkdEZFx6Fxi7969i759+0Imk0Emk72yTlYQBOTk5GDYsGEGD0lERK8SBAHff78FCxbMwRtvVMfBgxGaHQfc3NxETkdEZFw6l9g33ngDmzdvhkqlwrBhw17ZZksul8PDwwNVqlQxeEgiItL26NFDTJkyAeHhRwAAsbHXEBUVybtuEVGZUaTlBM2aNQMA9O3bFy1a8N7aREQlTRAE7N69E7NnT0NKSgoAwNOzLkJCNsLLq7G44YiISpBea2KXLl1q6BxERFSIp0+fYsaMKdi7NwzA8zttjR07AbNnz4OVlZW44YiISpheNzsYOnRogVe68mYHRESGlZiYCB+f1khMTAAAVK36BtatC0Xr1m1FTkZEJA69bnbg6+trlDBERJQ/Nzc3eHv7YNeunzF06HAsXLgYdnb2YsciIhKNXjc7GDFihFHCEBHRv5KTk+Ds7KL5eOnSFRgwYDD8/DqJmIqIyDTodbODa9euIT4+XvNxfHw8goKCsG3bNoMFIyIqq7KysjBnzgy0adMUjx//oznu5OTMAktE9P/0KrGTJk1CUlISACA3NxcjRoxAXFwcdu3ahZCQEIMGJCIqS86fP4uOHdth8+YNePr0KT79dK7YkYiITJJeJfbx48eoV68eAGDv3r1wdHTE119/jdDQUOzevdugAaVKEATNf//9T7rWx0RE/5Wbm4vPP/8MPXt2xq1bNwEAXbt2x6efLhE5GRGRadKrxFaoUAH379+HIAj47rvvNHfpsre3x9OnTw0aUIrOxyYiNOyK5uNv/7iO2RtP43xsooipiMhUXbnyP3Tt6ovVq1dCrVbDzs4ea9eG4vvvf4SHh4fY8YiITJJeJfaDDz7AsGHD0LdvXwiCgJ49ewIAYmJiyvxfuOdjE/FlWAyS059pHU9IycaXYTEsskSkJTQ0BF26dMCVKzEAgHbtvHH8eCQGDx5S4FaGRERlnV43OxgyZAgqVqyIe/fuoVevXpDLn3fhrKwszJgxw6ABpUQQBOyMuInXrRwQBGDnsZtoUqcc/3EiIgCASqVCXl4erK2tMW/eQowYMVrzdyoREb2eXiUWyH+v2LK+f+yN+BQkpGQXeE5Ccjbi7qeiThWnkglFRCZt3LgJuH//Hj76aCxq1qwtdhwiIsnQ+8f9yMhIDB06FK1atUKTJk0wbNgwnDt3zpDZJCclI1fH854VfhIRlTr378dj0KC+OH/+rOaYQqFAUNAXLLBEREWkV4ndv38/AgIC0KxZM3zxxRdYv349mjZtioCAAERFRRk6o2Q42VnoeJ6lkZMQkSkRBAE//rgNHTq0RkTEUUyYMAZZWVlixyIikjS9lhNs2LABCxcuhL+/v+ZY69atUb16daxevRo7duwwWEApqVPFCe5O1gUuKXB3tkbtyo4lmIqIxJSQkIBp0wJx4MA+AIBcLsfbb/eGQqEQORkRkbTpNRN79+5d+Pn5vXLc19cXV69eLXYoqZLJZBjgWwuvu2ZLJgMG+NTiRV1EZcTevXvQoUNLTYGtUaMm9u49iDlzFsDSkr+RISIqDr1KrL29PZKTk185npycDAsL3X6lXlo19XRDQJ+GcLbX/gfK3dkaAX0aoqmnm0jJiKikpKQkY9y4URg5cqhm7+yRI0fj6NG/0Lx5S5HTERGVDnqV2A4dOmDZsmXIzf33Qqa8vDysWLEC3t7eBgsnVU093TCuTwPNxx92r4ulo1uxwBKVEdHRF7Br188AgEqVKuOXX37D0qUrYWtrK3IyIqLSQ681sdOmTcMHH3yALl26oGnTppDL5Th//jzMzc3xww8/GDqjJL28ZKBaeXsuISAqQ3x9O2Lo0OHIy8vF4sVBcHDgOngiIkPTq8S6uLhg9+7d2L9/P2JiYqBUKjFmzBj4+/vD2tra0BmJiEza6dORSExMQK9evTXHli8P5sVbRERGVKQSm5CQgIMHD+LRo0eoU6cOevTogT59+hgpGhGRacvJyUFQ0GKEhq6Dra0dGjVqjCpVqgIACywRkZHpvCb2+vXr6NWrF3799Vc8fPgQGzduxIABA5Cenm7MfEREJuny5Yvo0qUDvvxyLQRBgEKhwK1bN8WORURUZuhcYletWoXBgwdj9+7dWL16Nfbv348GDRogODjYmPkkSxAEzX///U+61sdEJF15eXlYuTII3br54fr1awCer4E9ceI0fHxe3XqQiIiMQ+cSGxMTg/fff1/zsUwmw7hx43D48GGjBJOy87GJCA27ovn42z+uY/bG0zgfmyhiKiIqrtjY6+jZsxOWL/8cSqUSNja2WLFiNX78cTcqVKgodjwiojJF5xKbnJwMNzftLaIqV66MpKQkg4eSsvOxifgyLAbJ6c+0jiekZOPLsBgWWSKJyszMRO/e3XDxYjQAoGXL1oiIOIlhw0Zw9xEiIhEU6cKuq1evvvJrcUEQXjneoEGD/35qmSAIAnZG3MTrVg4IArDz2E00qVOO/+gRSYytrS1mzJiD+fNnY/bs+Rg7djwv3iIiEpHOJVYQBPTr1y/fx14+LpPJcO3ateInk6Ab8SlISMku8JyE5GzE3U9FnSpOJROKiPQiCALOnTujdYet4cNHwde3I6pXryFiMiIiAopQYq9fv27MHKVCSkZu4ScBSMl4VvhJRCSaf/55hKlTJ+LIkUPYvft3tGv3/E6EMpmMBZaIyEToddtZyp+TnYWO51kaOQkR6evXX3+Bt3dLHDlyCAAQErJa3EBERJQvllgDqlPFCe5OBd+xzN3ZGrUr8xaURKbm6dOn+OijDzFmzAikpKRAJpNhzJjx2LJlm9jRiIgoHyyxBiSTyTDAtxZed82WTAYM8KnFi7qITMyhQ3/A27sl9uzZDQCoWrUafv11HxYtWspbaRMRmSiWWANr6umGgD4N4WyvvWTA3dkaAX0aoqmn22s+k4jEEBS0GO+/PwiJiQkAgKFDP8SxY6fQpk07kZMREVFBWGKNoKmnG8b1+XebsQ+718XS0a1YYIlMUIcOvpDJZPDwKI/t23fiiy/Wws7OXuxYRERUiCLtE0u6e3nJQLXy9lxCQGQisrOzYWlpCbn8+c/wrVu3xZdfboafXyc4O7uInI6IiHSl90xsamoqfvvtN2zYsKHYIXJycjBv3jz4+vrC29sby5cvf+WmCi/LyspCq1atsGnTpmK/NhGVHefPn0XHju3wzTfaf3e8885AFlgiIonRq8TGxMSga9eu+OGHH7BmzRrN8enTp+tVLIOCgqBWq3H48GHs27cPUVFR2Lp162vP37ZtG9LS0vSJTkRlUG5uLpYu/Qw9e3bGzZtxWLRoAe7fjxc7FhERFYNeJTYoKAjTpk3Dzp07tWZMR44cid9++61Iz5WZmYmwsDBMnz4dZmZmsLe3x5gxY7Br1658z3/8+DF++eUXdOzYUZ/oRFTGxMTEoHNnH6xatRJqtRp2dvZYunQlKlWqLHY0IiIqBr3WxF69ehVfffUVgP+s/axWDQ8ePCjSc125cgWVK1eGk5OT5piXlxfi4uKgUqleuTf5559/jrFjxyIqKkqf6JDLZZDLjb8+1Uzx788HcrkMZma8hk6KFP8/jgoFx09qVCoV1qwJxuefL0JeXh4AoF279ggJ2YCqVauJnI50xfeg9HEMpc2Ux0+vEuvs7IwHDx6gVq1aWsevXr0KW1vbIj1XYmIiXF1dtY65uLhAqVQiPT1dq9zu3bsXKSkp6NOnj94l1sXFtkQusrJP+/fWsra2lnB2Ltr3hUyLgwP3CpWSO3fuYMiQIYiMjAQAWFlZYdmyZZgwYYLmgi6SFr4HpY9jKG2mOH56ldj3338fc+fOxcqVKwEASqUSFy5cwPz589GzZ88iPZdSqXzlIi61Wg1Ae5Y3Pj4eq1atwrZt24pVQpOSMktkJjY9PUfz35mZz5CcnGn01yTDUyjkcHCwRlpaNlQqtdhxSEcZGbm4cuUqAKBFixb48stNqFGjFlJTs0VORkXF96D0cQylTazx02XyT68SO2LECOTk5MDf3x+CIMDLywtyuRyDBw/Gxx9/XKTncnJyQnJystaxpKQkWFpawt7++V6NOTk5mDhxIqZNm4YKFSroE1lDrRagVr9+5wNDUb400Gq1AKWSb1wpU6nUHEMJKV++IoKCVuLBg3h8+uk8pKc/4/hJHN+D0scxlDZTHD+994kNCAjAyJEjcePGDZibm6NatWp63Z6xfv36uHPnDlJTU+Ho6AgAiI6O1hRjAIiMjMTt27cxf/58zJ8/H8DzYiuXyxEZGYktW7bo+2UQkcQJgoCff96BK1f+h88++1xzvH//QTAzk8PMzAzAs9c/ARERSVKxbnZgaWmJhg0bFiuAm5sb2rdvj+DgYMybNw9paWkIDQ1FYGCg5hxfX19cvnxZ6/NmzZqFGjVqYPTo0cV6fSKSrsTEREybFog//vgdANC+vTc6d+4mcioiIioJepXYsWPHFvh4UW+AsGTJEsyZMwft2rWDjY0NRowYgU6dOmHPnj2IiYnB3Llz9YlJRKXYvn17MX16IJ48eQIAeOON6nBychY5FRERlRS9Suybb76p9XFeXh5iY2Nx6dIlDBo0qMjP5+LigtDQ0FeO9+7dG7179873c4KCgor8OkQkfampKZg9ezp++eUnzbHhw0dh/vxFRd4dhYiIpEuvEjthwoR8j+/YsQO3bt0qViAioteJiDiKyZPH49GjhwCAihUrYfXq9fDx8RM5GRERlTSDbpjo7++PPXv2GPIpiYgAPN+Ob/782ZoCO3Dguzh+PJIFloiojDJoib127RosLS0N+ZRERAAAMzMzrFu3AeXLV8C3325HSMhGODo6iR2LiIhEotdygv9uaaVSqfDgwQPs27cP77zzjkGCEVHZ9uzZM/zwwxZ8+OGo/98mC2jUqAnOnr3MH5aJiEi/EhseHq71sVwuR7ly5TBjxgz07dvXIMGIqOyKibmECRPG4Nq1q8jMzERg4L83UWGBJSIiQM8S+/XXX8Pc3LxYt38lIvovpVKJtWuDsXJlEJRKJQDgzJnTEASBf98QEZGWIq+JFQQBjRo1QlpamjHyEFEZFRd3Az17dkJQ0GIolUrY2Nhg2bJgbN36MwssERG9osglViaToUGDBnjw4IEx8hBRGaNWq7Fp05fo2LEdoqMvAACaN2+J8PCTGD58FAssERHlS6/lBGPGjMGiRYvQvXt31K9fH9bW1lqPN2jQwCDhiKj0W7FiKb74YhkAwMLCAjNnzkVAwEQoFAqRkxERkSkr1s0OoqOjX3lMJpPh2rVrxUtFRGXG8OEf4dtvv0KFCpUQErIR9evzh2AiIiqcziW2Y8eOOHr0KADg+vXrRgtERKXb48f/wMLCAs7OLgAAd3d37N69DzVr1oKFhYXI6YiISCp0XhPLNbBEVFx79uyGt3dLzJw5Vet4vXr1WWCJiKhIdC6xvLiCiPSVlPQUY8YMx0cffYjk5GSEhe1GTMxlsWMREZGE6bycQBAEfPDBBzqd+/333+sdiIhKlyNHDmLy5AlISHgMAKhSpSrWrg1Fw4ZviZyMiIikrEgXdvn4+BgpBhGVNhkZ6Zg//xNs3fqd5tiQIR/gs88+h729g4jJiIioNNC5xMpkMowYMcKYWYiolLhz5zYGDOiNe/f+BgC4u3tg1ap16Ny5m8jJiIiotNB5TawgCMbMQUSlSKVKleHo6AQA6NOnH06cOM0CS0REBqXzTGzfvn2NmYOIJC4vLw/m5uYAnt+0YP36Tbh+/Sr69HlH5GRERFQa6TwTu3TpUmPmICKJys3NRVDQYvTo0Qm5ubma43Xr1mOBJSIio9G5xBIR/de1a1fRvXtHBAcvx6VL0Vi1aoXYkYiIqIxgiSWiIlOpVFi3bjU6d/ZGTMwlAECbNu3w7rvvi5yMiIjKiiJtsUVEdPv2LUyaNA5nzpwGAFhZWWHOnAX46KNxkMv5czEREZUMllgi0okgCPj226+xcOFcZGVlAQAaN26CkJBNqF27jsjpiIiorOG0CRHpRCaTISoqEllZWTAzM8OsWXOxb98RFlgiIhIFZ2KJSGdBQSuRnJyEuXM/RcOGXmLHISKiMowzsUSUrydPnmDSpHF4/Pix5piTkzN++ulXFlgiIhIdZ2KJ6BX79/+OadMm4cmTJ0hKeooffvgJMplM7FhEREQaLLFEpJGamoI5c2bi5593aI5VqlQZeXl5sLCwEDEZERGRNpZYIgIAHDsWjsmTx+PhwwcAgAoVKmL16vXw9e0ocjIiIqJXcU0sURmXmZmJWbM+xsCBfTQFdsCAwTh+PJIFloiITBZnYonKuJ9+2o5vvtkMAHB1dcWKFWvw9tv+IqciIiIqGEssURk3bNgI7Nr1M1xdy2HlyjVwd3cXOxIREVGhWGKJypiYmMsQBDXeeqsRAEChUODHH3fBzs6eOxAQEZFkcE0sURmhVCoRHLwcXbv6YOzYkcjOztY8Zm/vwAJLRESSwhJLVAbcvBmHt9/ujKCgxVAqlXj48AEuXYoWOxYREZHeWGKJSjG1Wo1Nm76En19bXLhwHgDQvHlLhIf/hVat2oicjoiISH9cE0tUSsXH30NgYAD++usEAMDCwgIzZszB+PGToFAoRE5HRERUPCyxRKXQ48eP4ePTBunpaQCABg0aYv36Tahfv4HIyYiIiAyDywmISiEPDw/06dMPcrkcU6ZMw8GDESywRERUqnAmlqiUuH8/HpUrV9F8vHDhEgwZ8gGaNGkmYioiIiLj4EwskcQlJydhzJjh8PZuhfj4e5rjdnb2LLBERFRqscQSSdiRIwfh7d0Kv/66CxkZ6Vi6dJHYkYiIiEoESyyRBGVkpOPjjyfhvfcG4PHjfwAA7703FMuWfSFyMiIiopLBNbFEEhMZeRITJ47DvXt3AQBubu4IDl6Hrl27ixuMiIioBHEmlkhCgoIWoU+fHpoC6+/fFydORLHAEhFRmcOZWCIJcXPzgCAIcHJyQlDQF+jbtz9kMpnYsYiIiEocSyyRCVOr1ZDL//2FyfDho/DkSSKGDRuB8uUriJiMiIhIXFxOQGSirl27iq5dfXHy5J+aY3K5HDNnzmGBJSKiMo8llsjEqFQqhISsQefO3rh0KRqTJo1DRka62LGIiIhMCpcTEJmQO3duY9KkcYiKigQAWFpa4qOPxsLGxlbkZERERKaFJZbIBAiCgO+++waffjoXWVmZAIBGjRojJGQT6tTxFDkdERGR6WGJJRLZo0cPMXnyeEREHAUAmJmZYerUGQgM/Bjm5uYipyMiIjJNLLFEIrt79w6OHQsHAHh61kVIyEZ4eTUWORUREZFpY4klElnr1m0xbtxEAMCsWXNhZWUlciIiIiLTxxJLVMIOHNiPlJRkDB48RHNswYJFvGkBERFREbDEEpWQtLRUzJ07Cz/+uA3W1tZo3rwFatasDQAssEREREXEfWKJSsCJE8fQoUNr/PjjNgCAo6MTnjx5KnIqIiIi6WKJJTKirKwszJ49Df37++PBg/sAgHfeGYgTJ06jZctWIqcjIiKSLi4nIDKSs2ejMHHiWNy+fQsA4OLighUr1qBXr94iJyMiIpI+llgiI8jIyMD77w9EcnIyAKBbtx5YuXIt3N3dRU5GRERUOnA5AZER2NnZYeHCz2Fv74C1a0Px3Xc7WGCJiIgMiCWWyACUSiWOHDmodWzQoPcQGXkBgwcP4e4DREREBsYSS1RMt27FoVevrnjvvQE4fPiA5rhMJuPsKxERkZGwxBLpSa1W46uvNsDPrx3Onz8LANi27QeRUxEREZUNvLCLSA/378cjMDAAf/55HABgbm6OmTPnYPz4QJGTERERlQ0ssURFIAgCfvppO+bMmYn09DQAQP36b2L9+k1o0OBNkdMRERGVHSaxnCAnJwfz5s2Dr68vvL29sXz5cgiCoHVOXl4eQkJC0KtXL3To0AHvvfcerl27JlJiKqvmz/8EkyaNQ3p6GuRyOSZPnoZDh46xwBIREZUwkyixQUFBUKvVOHz4MPbt24eoqChs3bpV65y7d+9CpVLhp59+wvHjx+Hv74+xY8ciLy9PpNRUFvXr1x8KhQI1atTE778fwiefzIeFhYXYsYiIiMoc0UtsZmYmwsLCMH36dJiZmcHe3h5jxozBrl27tM6rXbs2AgMDYWNjAwAYPHgwsrOz8ffff4sRm8qI1NQUqFQqzceNGzfFDz/8iPDwk2jWrIWIyYiIiMo20dfEXrlyBZUrV4aTk5PmmJeXF+Li4qBSqaBQKPL9vOzsbGRnZ8Pe3r5IryeXyyCXG3/PTjPFvz8fyOUymJmJ/vMCFdGRI4cRGBiASZMmISDg3wu2unXrLmIqKgrF/78PFQq+/6SKYyh9HENpM+XxE73EJiYmwtXVVeuYi4sLlEol0tPTtcrty1atWoUWLVrAw8OjSK/n4mJbIhvP26c90/y3ra0lnJ1tjf6aZBgZGRmYNm0aNm7cCACYP38+hgwZgipVqoicjPTl4GAtdgQqJo6h9HEMpc0Ux0/0EqtUKl+5iEutVgNAvmUzKysLCxcuxI0bN/D1118X+fWSkjJLZCY2PT1H89+Zmc+QnJxp9Nek4ouMPInx48fi7t07AAB3d3ds3rwZjo7lOIYSpFDI4eBgjbS0bKhUarHjkB44htLHMZQ2scZPl8k/0Uusk5MTkpOTtY4lJSXB0tLylaUC9+7dw9ixY9G4cWNs374d1tZF/6lArRagVguFn1hMypcGWq0WoFTyjWvKcnJyEBS0GKGh6zQ/VPXq1QfBwatRq1Y1JCdncgwlTKVSc/wkjmMofRxDaTPF8RO9xNavXx937txBamoqHB0dAQDR0dHw8vKCXP7v+ou0tDQMGzYM48aNw8CBA8WKS6XQjRuxGDlyKGJjrwN4/oNVUNAX6Nu3P8zN81+TTUREROISfZWum5sb2rdvj+DgYCiVSiQlJSE0NBTDhg3TOu/AgQOoUaMGCywZnJOTMxITEwAAfn6dcOJEFPr1G1Aia6eJiIhIP6LPxALAkiVLMGfOHLRr1w42NjYYMWIEOnXqhD179iAmJgZz587F3bt3ER0dDT8/P63PHTt2LIstFYu7uzu++GIdnj59gqFDP2R5JSIikgCZ8N+rqkq5xMT0EnmdWw9TseT78wCABcObo5pH0bYCI+NQqVTYuPFLxMZew5o1XxZ6vpmZHM7OtlwTK1EcP+njGEofx1DaxBo/N7fCe5NJzMQSlYS7d+9g0qRxOH36FACgY8fO8PfvK3IqIiIi0ofoa2KJjE0QBHz33Tfw8WmjKbBvvdUIderUFTkZERER6YszsVSqPXr0EFOmTEB4+BEAgEKhwJQp0zFlynSYm5uLnI6IiIj0xRJLpZIgCNi9eydmzZqG1NQUAICnZ12EhGyEl1djccMRERFRsXE5AZVKKpUK69evRWpqCmQyGcaNm4jDh0+wwBIREZUSLLFUKpmZmSEkZCNq166DsLD9WLhwCaysrMSORURERAbCEkulQnp6GpYv/xy5ubmaY/XrN8Cff55B69ZtRUxGRERExsA1sSR5f/55HIGBAbh/Px5qtQqzZs3TPPbyrYuJiIio9GCJJcnKysrC4sUL8NVXGzXH4uPjIQgC77pFRERUyrHEkiSdO3cGEyeOxa1bNwEALi4uWL58FW9eQEREVEawxJKk5ObmYuXKIKxdGwy1+vnt77p27Y6VK9fCw8ND5HRERERUUlhiSVJWrFiKNWu+AADY2dljyZJlGDx4CJcPEBERlTG86oUkJSBgIjw8yqNdO28cPx6Jd999nwWWiIioDOJMLJm027dvws7OAe7u7gAAZ2cX7Nt3GJUrV+HOA0RERGUYWwCZJLVaja+/3ghf37b4+OOJEARB81jVqtVYYImIiMo4NgEyOffvx2PAgD6YPXs6srOzER5+BNeuXRU7FhEREZkQllgyGYIg4Mcft6FDh9b4889jAIB69Rrg4MFjqF+/gZjRiIiIyMRwTSyZhISEBEybNgkHDuwH8PxOWxMnTsG0abNgaWkpcjoiIiIyNSyxJLq4uBvw9++Kp0+fAgBq1KiJdes2oHnzliInIyIiIlPF5QQkuho1aqJ69ZoAgJEjR+Po0b9YYImIiKhAnIklUWRmZsLW1hYAoFAoEBKyAffv34e3t4+4wYiIiEgSOBNLJSojIwPTp09Bly4dkJ2drTleo0YtFlgiIiLSGUsslZjTpyPh69sG3333NeLibmDVqhViRyIiIiKJ4nICMrqcnBwsW7YEX365VnPTgp49/TF6dIDIyYiIiEiqWGLJqC5fvogJE8bg+vVrAABHRycsXboC77wzEDKZTOR0REREJFUssWQUgiAgOHg5vvhiGZRKJQDA17cjVq0KQcWKlUROR0RERFLHEktGIZPJ8PDhAyiVStjY2OLTTxdj2LARnH0lIiIig2CJJaNZuHAJMjMzMHPmXFSvXkPsOERERFSKcHcCMoi//76LoUMH4f79eM0xOzt7bNjwDQssERERGRxnYqlYBEHA1q3fYf78T5CZmYGsrCzs3LkHcjl/PiIiIiLjYYklvf3zzyNMnToRR44cAvD8zlstWrSCWq1miSUiIiKjYoklvfz66y+YOXMqUlJSAAC1a9dBSMhGNG7cVNxgREREVCawxFKRJCU9xcyZH2PPnt0Anu9CMHp0AD75ZD6sra1FTkdERERlBUssFcnvv/+mKbBVq1bD2rWhaNOmncipiIiIqKxhiaUief/9YfjttzBUq1YNCxcugZ2dvdiRiIiIqAxiiaUC/fXXCZibW6Bly1YAALlcju3bd8LCwkLkZERERFSW8RJyyld2djbmzp2Jfv3exvjxHyEjI13zGAssERERiY0lll5x4cI5dOzYDps2hQIA0tPTEBt7XeRURERERP9iiSWN3NxcBAUtQs+enXHzZhwAoHPnrjhxIgpNmzYXOR0RERHRv7gmlgAA165dxfjxo/G//10G8PyWsYsXB+Hdd9+HTCYTOR0RERGRNpZYwuPH/6BrVx/k5OQAANq2bY81a75E1arVRE5GRERElD8uJyB4eJTHsGEjYWVlhSVLlmHXrr0ssERERGTSWGLLIEEQcO3aVa1jn3wyHxERJ/HRR+Mgl/OPBREREZk2tpUy5sGD+xg4sA+6dfPFrVtxmuPW1taoWbO2iMmIiIiIdMcSW0YIgoCfftqODh1a4/jxCGRnZ2PNmmCxYxERERHphRd2lQGJiYmYNi0Qf/zxO4Dnd90aPz4QM2Z8InIyIiIiIv2wxJZy+/btxfTpgXjy5AkA4I03qiMkZBNatGgpcjIiIiIi/XE5QSk2e/Y0DB8+RFNghw8fhYiIUyywREREJHmciS3FvLwaAwAqVKiI1avXw9e3o8iJiIiIiAyDJbYUyc3NhYWFhebjQYPeQ1paKgYNeg+Ojk7iBSMiIiIyMC4nKCWiok6jffsWOHLkoOaYTCbD6NEBLLBERERU6rDEStyzZ8/w2Wfz4e/fFXfu3MaUKRORnp4mdiwiIiIio+JyAgmLibmECRPGaO6+5eDgiPnzP4Odnb3IyYiIiIiMiyVWgpRKJdauDcbKlUFQKpUAgA4dfLFmzZeoWLGSyOmIiIiIjI8lVmJu3ozD+PEfITr6AgDAxsYGCxYsxocfjoRMJhM5HREREVHJYImVmNTUFFy6dBEA0Lx5S6xbtwE1atQUNxQRERFRCWOJlZimTZtj+vTZsLCwREDARCgUCrEjEREREZU4llgTJggCtm37HtnZWfjoo3Ga4x9/PFPEVERERETiY4k1EkEQNP/99z/pqOpuV6Q1q48f/4OpUyfi8OGDsLCwQNu23qhfv4ExohIRERFJDveJNYLzsYkIDbui+fjbP65j9sbTOB+bqNPnh4Xtgrd3Sxw+/PzGBdWqvQGVSmmUrERERERSxBJrYOdjE/FlWAyS059pHU9IycaXYTEFFtmkpKcYPfpDjB49HMnJyQCAMWMCcOTIn2jY0MuouYmIiIikhMsJDEgQBOyMuImXVhL853Fg57GbaFKn3CtLC44cOYjJkycgIeExAKBKlapYuzYUbdu2N3ZsIiIiIsnhTKwB3YhPQUJKdoHnJCRnI+5+qtaxjIx0TJgwRlNghwz5AMeOnWKBJSIiInoNllgDSsnI1fE87aUGdnb2WL58FdzdPbBt289YtSoE9vYOxohIREREVCqwxBqQk52FTudZmwn4+ecdWsf8/fvi9OkL6Ny5mzGiEREREZUqXBNrQHWqOMHdybrAJQWyzHsY++EMxMXdgLW1NXr16qN5zM7OvgRSEhEREUmfSczE5uTkYN68efD19YW3tzeWL1+utc/qC1evXsXAgQPh6+uLHj164OTJkyKkfT2ZTIYBvrWQ326walUeYk9uw76vpiAu7gYA4ODBP0o2IBEREVEpYRIlNigoCGq1GocPH8a+ffsQFRWFrVu3ap2TkZGBsWPHYvLkyYiIiMCnn36KwMBAJCbqtvdqSfpv/U578jf+2jETcVE7oVapYGtrh+DgdVi3boMo+YiIiIikTvQSm5mZibCwMEyfPh1mZmawt7fHmDFjsGvXLq3z9u3bh4YNG6JNmzYAgBYtWqBZs2bYv3+/GLHz9WKLLc3HahVunt2Nv7Z9jLSE2wCA8m+8hWPHTuH994cV6Q5eRERERPQv0dfEXrlyBZUrV4aTk5PmmJeXF+Li4qBSqaBQKAAAFy9eRJMmTbQ+18vLC9evXy/S68nlMsjlximP1/9O1loPG3N0I+7FHHr+ugoL1G3/Pqo3fhtKc2eYmYn+8wPpQKGQa/0/SQvHT/o4htLHMZQ2Ux4/0UtsYmIiXF1dtY65uLhAqVQiPT1dU24TEhLQsmVLrfNcXV1x6dKlIr2ei4ut0WZA8+4ma338RqMeiL8SDge36mjUbRLsXas8P08tg7OzrVEykHE4OFiLHYGKgeMnfRxD6eMYSpspjp/oJVapVL5yEZdarQYArbKpUqle+VyVSlXkQpqUlGm0mVjz/zytg9sbaD1gEZwq1IFcrvj3PLmA5ORMo2Qgw1Io5HBwsEZaWjZUKrXYcaiIOH7SxzGUPo6htIk1frpM9oleYp2cnJCcrD2DmZSUBEtLS9jb/7vllKOjY77nubm5Fen11GoBavVr7gtbTDUrOryyxZZLpXpa57g7W6NGBQcolXwjS4lKpeaYSRjHT/o4htLHMZQ2Uxw/0Rc41K9fH3fu3EFq6r+3Yo2OjoaXlxfk8n/jNWjQANHR0VqfGx0djUaNGpVU1EJptth6zUSvTAYM8KnFC7qIiIiIikn0Euvm5ob27dsjODgYSqUSSUlJCA0NxbBhw7TO8/f3R2RkJCIjIwEAx48fx+3bt9Gtm2nd4aqppxsC+jSEu7P22hF3Z2sE9GmIpp5FmzkmIiIioleJvpwAAJYsWYI5c+agXbt2sLGxwYgRI9CpUyfs2bMHMTExmDt3LsqXL4/g4GAsXLgQqampqFatGkJDQ2FjYyN2/Fc09XRDkzrlcOthGpSCDOZyATUqOHAGloiIiMhAZEJ+t8YqxRIT00vstczM5HB2tkVycqbJrSMh3XAMpY3jJ30cQ+njGEqbWOPn5mZf6DmiLycgIiIiIioqllgiIiIikhyWWCIiIiKSHJZYIiIiIpIcllgiIiIikhyWWCIiIiKSHJZYIiIiIpIcllgiIiIikhyWWCIiIiKSHJZYIiIiIpIcllgiIiIikhyWWCIiIiKSHJZYIiIiIpIcmSAIgtghiIiIiIiKgjOxRERERCQ5LLFEREREJDkssUREREQkOSyxRERERCQ5LLFEREREJDkssUREREQkOSyxRERERCQ5LLFEREREJDkssUREREQkOSyxxZSTk4N58+bB19cX3t7eWL58OfK7CdrVq1cxcOBA+Pr6okePHjh58qQIaSk/uoxhXl4eQkJC0KtXL3To0AHvvfcerl27JlJi+i9d34cvZGVloVWrVti0aVMJpqSC6DqGgiBgy5Yt6Nq1K3x8fNC5c2fk5eWJkJhepuv4HTlyBD179oSPjw/69++Pc+fOiZCWXkcQBISFhWHQoEGvPceU+gxLbDEFBQVBrVbj8OHD2LdvH6KiorB161atczIyMjB27FhMnjwZERER+PTTTxEYGIjExESRUtPLdBnDu3fvQqVS4aeffsLx48fh7++PsWPH8h9PE6HLGL5s27ZtSEtLK8GEVBhdxzA0NBTh4eHYvn07jh07hq1bt0KhUIiQmF6my/jFx8djxowZCAoKwrFjxzBlyhQEBAQgPT1dpNT0shMnTsDf3x/r169HampqvueYXJ8RSG8ZGRmCl5eXkJycrDl28OBBoXfv3lrn/fjjj0JAQIDWsTFjxgjffvttCaSkgug6hvlp3ry5EBcXZ7xwpJOijuE///wjdOnSRZgwYYKwcePGkglJBdJ1DJ8+fSo0atRIePjwYckGpALpOn5HjhwR+vbtq3Wsffv2wuXLl0sgJRXmwIEDwrFjx4TTp08LXbt2zfccU+sznIkthitXrqBy5cpwcnLSHPPy8kJcXBxUKpXm2MWLF9GkSROtz/Xy8sL169dLKiq9hq5j+F/Z2dnIzs6Gvb19CaSkghR1DD///HOMHTsWtra2JZiSCqLrGEZERKBJkyaoUKGCCCnpdXQdv2bNmiEpKUnz6+fff/8djo6O8PT0LOnIlI+uXbuiQ4cOBZ5jan2GJbYYEhMT4erqqnXMxcUFSqVS69cjCQkJr5zn6uqK5OTkEslJr6frGP7XqlWr0KJFC3h4eBg7IhWiKGO4d+9epKSkoE+fPiWYkAqj6xjeuHEDlSpVwvz58+Hn54fevXsjLCyshNPSf+k6fo6OjpgxYwZGjBiBxo0bY9asWVi8eDEsLCxKOjLpydT6DEtsMSiVylcWrqvVagCATCbTHMtvNkilUmmdQ+LQdQxfyMrKwsyZM3H27FmsWLGiRDJSwXQdw/j4eKxatQpBQUF875kYXccwMzMT4eHh6NatG44ePYqgoCAsW7YMZ86cKdG8pE3X8bt8+TJWrVqFsLAwXLhwAZs2bcLEiRNx//79Es1L+jO1PsMSWwxOTk6v/PSRlJQES0tLrV8zOzo65nuem5tbieSk19N1DAHg3r176N+/P8zMzLB9+3a4uLiUZFR6DV3GMCcnBxMnTsS0adP4q2gTpOv70NnZGe3bt0ebNm0gk8lQr149+Pv7Izw8vKQj00t0Hb/vvvsOQ4YMQb169SCTydCmTRt07twZO3fuLOnIpCdT6zMsscVQv3593LlzR+sqvujoaHh5eUEu//db26BBA0RHR2t9bnR0NBo1alRSUek1dB3DtLQ0DBs2DB9++CGWLFkCa2trMeJSPnQZw8jISNy+fRvz589Hs2bN0KxZM/z+++8ICQnB8OHDxYpO/0/X92HNmjWRmZmp9blyuRyWlpYllpVepev45eXlvbKThJmZGXJzc0ssKxWPqfUZlthicHNzQ/v27REcHAylUomkpCSEhoZi2LBhWuf5+/sjMjISkZGRAIDjx4/j9u3b6Natmxix6SW6juGBAwdQo0YNDBw4UKSk9Dq6jKGvry8uX76Mc+fOaf739ttvY8KECdiyZYuI6QnQ/X3YrVs3XLhwAadOnQIA3Lp1C3v37kX37t3FiE3/ryjjt3XrVjx8+BAAcO3aNYSFhaFz585ixCY9mFqfMRPlVUuRJUuWYM6cOWjXrh1sbGwwYsQIdOrUCXv27EFMTAzmzp2L8uXLIzg4GAsXLkRqaiqqVauG0NBQ2NjYiB2foNsY3r17F9HR0fDz89P63LFjx7LYmgBdxpBMmy5jaGVlhXXr1mHhwoVISkqCi4sLlixZgrp164odv8zTZfx69OiBzMxMjBo1CtnZ2XBwcMCiRYteudqdTIsp9xmZ8N/V2EREREREJo7LCYiIiIhIclhiiYiIiEhyWGKJiIiISHJYYomIiIhIclhiiYiIiEhyWGKJiIiISHJYYomIiIhIclhiiYiIiEhyWGKJqMwbOnQovv76a7Fj6OXKlSvo3r17gfef1+UcIiKp4W1niahUWLduHUJCQl45fuXKFZiZiftX3dChQ3HmzBkAgI2NDWrUqIHRo0eja9euxX7uBg0a4I8//tA69uTJE1hZWcHOzu615xjay99/mUwGDw8PdOrUCVOmTNHk0EVaWhqUSiVcXFyMFZWISgnOxBJRqeHj44PY2Fit/4ldYF+YMWMGYmNjcezYMXz44YeYOXMmjh8/bpTXat++Pe7cuWOU5y7Ii+//lStXsHnzZly+fBmfffZZkZ6jX79+msJPRFQQllgiohLk6OiIXr16YfDgwfj555+N8hpqtdooz6srhUKBOnXqYNKkSYiIiCjS54qdnYikgyWWiEq9a9euYezYsWjdujWaNm2K8ePH4+nTp689f9u2bejatSsaNmyIjh074sqVK5rHTpw4gd69e+Ott95Cjx49EBYWplemKlWq4NGjRwAApVKJ9evXo1OnTnjzzTfh4+ODNWvWQKlUas6PjIzEoEGD4OXlhdatW2Pnzp0AgJiYGHh6egIAdu/erfnv/v37w9PTE/fv39c65+uvv0b37t1fydOrVy/89NNPAIC7d+9i1KhR8PLygq+vL1auXKnXetqMjAzY2NhoPs7JycGyZcvQqVMneHl5oVevXvjzzz8BAFFRUfD09MSDBw8QGBgIT09PREVFGTQPEZUuLLFEVOpdunQJXbp0wZ49e3DgwAE8efIEa9asyffciIgIhIaGYv369Th79ixWrlwJe3t7AMCZM2fwySefYPr06YiKisJnn32GpUuXaspWUVy7dg21a9cGACxYsAD79u3D8uXLERUVhZUrV+L333/HunXrAABJSUkYO3YsRo8ejTNnzmD79u2oWbPmK8/Zr18/xMbGAgB++eUXxMbGonLlylrn9OrVC3///TeuX7+uOXb9+nXcv38fPXv2RFJSEj744AP4+Pjg5MmT+Pbbb3HmzBls3LhR569NqVTi4sWLWLVqFUaPHq05npycjJycHGzYsAGnTp1Cr1698PHHHyMrKwstW7ZEbGwsKlWqhDVr1iA2NhYtW7Y0SB4iKp1YYomo1Dh27Bg8PT01/wsICAAADB48GP369YO7uzvc3NzQv39/XLx4Md/n+Pvvv+Hi4oIaNWrAysoKjRs3RtWqVQEAISEhmDJlCtq1awdra2s0a9YM77zzDvbs2aNzxqdPn+L777/HoUOHMHr0aNy/fx+7du1CcHAwmjRpAltbWzRr1gyfffYZvvnmG+Tk5ODx48fIzc2Fl5cXLC0tUb16dTRp0kSv75G7uzvatm2Lffv2aY7t2bMH3bp1g52dHbZv347WrVvj/fffh52dHapVq4bx48fr9DW++P43aNAA48ePx4IFCzBkyBDN4xUqVMCCBQtQq1Yt2NraYvjw4UhPT8fdu3df+5zFyUNEpZtpXPFARGQAPj4++c7QZWRkYMeOHTh37hzu3r2Lf/75B66urvk+h7+/P3755Rf07t0bw4YNg7+/PywsLAAAly9fRlRUFD755BOtz2ndunWh2ZYvX47ly5fD1tYWLVq0wLZt21CzZk0cOHAA5cqVQ926dbXOb9q0KfLy8vD333+jbt266NatG/z9/fHee+/hvffeK9bV+3379sUXX3yBjz/+GGq1Gvv27cPq1as1X+Px48dfWSahUCgKfd4X3/+bN29i4sSJ+Ouvv9C2bVutc44cOYIjR47gxo0bePDgAdRqNdLT01/7nMXJQ0SlG0ssEZVqeXl5ePfdd9GgQQOMGjUK1atXx5kzZ7By5cp8z3dxcUFYWBj27duH77//HqGhodiyZQuqVq0Kc3NzrF+//pViposZM2Zg5MiRrxxXqVSQyWT5fo5MJkNubi5kMhlWrVqFM2fO4Pvvv0fHjh0RHBwMX1/fIucAgI4dO2L+/Pm4dOkSsrKyYG9vr5nZNTc3x9ixYzFlyhS9nhsAatWqheDgYAwYMABvv/02GjRoAADYtGkTdu/ejcmTJyMgIAAVKlRA8+bNC3wuQ+QhotKJywmIqFSLi4vDrVu3sHjxYjRv3hzlypXDrVu3CvwcMzMz9O7dG7/88guqVauGrVu3AgBq1qyJ8+fPGzRfvXr1kJiYiLi4OK3jFy5cgKWlJWrVqqU51qJFC4SEhGD06NGvXdMLPC+/giC89nFLS0v06NED+/fvx2+//Yb+/ftrHjPU11ivXj3069cPixYt0mT5/fff8dFHH6Fbt26oWrUqHj9+jOzs7Feyv7xDgTG+50RUOrDEElGp5urqCkEQ8McffyA7OxsREREF7igQHh6O06dPIzMzEwkJCUhOTkaVKlUAACNHjsSWLVuwZ88eZGRkID4+HmvXrkVMTIze+WrUqIEePXpgypQpuHjxIrKysnD+/HksWLAAo0ePhrW1NW7evInffvsNSUlJyMjIwP379zWZ8uPh4YGoqChkZGS89ir+Pn364M8//0RUVBR69+6tOT5kyBD873//Q3BwMJ48eYKnT5/i119/1eyGUBSBgYGIi4vTrF91c3NDREQE0tLScO/ePSxevBjm5uavZD937hyys7ORmZlp0DxEVLqwxBJRqebh4YHPPvsMX3zxBVq1aoXdu3dj3Lhxrz1frVZj3rx5aNWqFQYPHgwfHx/NxUmdO3fGzJkzERISglatWuGDDz7AkydPCiyUuggKCkKHDh0wceJEtGjRAp988gmGDh2quTDNzMwM3377LXx9fdGpUyc8e/YMCxcufO3zTZ8+Hd988w38/PyQmpqa7zlNmjSBSqVCw4YNtdbXenh4YPPmzTh9+jR8fHzQq1cvHDx4EG+99VaRvy5XV1eMGTMGK1euREZGBubMmYOnT5+iXbt2GDt2LIYNGwYrKyutz5k4cSLCw8PRpk0b3Llzx6B5iKh0kQkF/c6JiIiIiMgEcSaWiIiIiCSHJZaIiIiIJIclloiIiIgkhyWWiIiIiCSHJZaIiIiIJIclloiIiIgkhyWWiIiIiCSHJZaIiIiIJIclloiIiIgkhyWWiIiIiCSHJZaIiIiIJIclloiIiIgk5/8AiWIaGxCNTwYAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 800x550 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.9768214653542092\n"
     ]
    }
   ],
   "source": [
    "from sklearn.metrics import roc_curve\n",
    "fpr, tpr, thresholds = roc_curve(y_test, model.decision_function(X_test))\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "plt.plot(fpr, tpr, 'o-', label = \"LogisticRegression\") # 점과 선으로 그래프 그리기\n",
    "plt.plot([0,1],[0,1], \"k--\") # 대각선으로 서로를 잇는 직선\n",
    "plt.plot([1-specificity],[recall],'ro', ms= 10) # 임의의 특정 지점 cut-off value\n",
    "plt.xlabel(\"False Positive Rate\")\n",
    "plt.ylabel(\"True Positive Rate\")\n",
    "plt.show()\n",
    "\n",
    "from sklearn.metrics import auc\n",
    "auc = auc(fpr, tpr)\n",
    "print(auc)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 성능 향상 전/후에 대한 분석"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- ROC 커브 그래프를 살펴봤을 때 개선 전 후 cut off가 더 optimal하게 향상된 것을 확인할 수 있다."
   ]
  },
  {
   "attachments": {
    "image.png": {
     "image/png": "iVBORw0KGgoAAAANSUhEUgAAAqAAAACxCAYAAAAbKQ/SAAAAAXNSR0IArs4c6QAAAARnQU1BAACxjwv8YQUAAAAJcEhZcwAAFiUAABYlAUlSJPAAADrJSURBVHhe7Z29buM6E4Z5viY3YmARBNjGXa4gwBbeLt0C6tKlWbhMkdJIky6dge3SrYsAvoJ0bhYwFgv4RlLlG/5JlERRHIlSZOd9AJ7NsSmaGpLD4XAo/ff29vYuAAAAAAAAGIn/mX8BAAAAAAAYBRigAAAAAABgVGCAAgAAAACAUYEBCgAAAAAARgUGKAAAAAAAGBUYoAAAAAAAYFRggAIAAAAAgFGBAQoAAAAAAEYFBigAAAAAABgVGKAAAAAAAGBUYIACAAAAAIBRgQEKAAAAAABGBQYoAAAAAAAYFRigAAAAAABgVGCAAgAAAACAUYEBCj4hW3FzdibObrbm/yfK4VFcUj0vHw/mAx9D3csHymh7I87ot6fePOCzAb3RztAyOojHSyr/8pH+AscODFBwQmjlF1a8DKIU+REykoG3vaGJQk5GremGWg6AjwJ6IwosDEFiYIACMAEOj5cVo4zSxVLs6Lvd8qL+3RF4AK6e3sTbm5s2IqPP56t95fMncaUvAQAwOEW9kXs5q3VX6UIs9c2JC+/3lOAdPRpggAIwAWa3rxWjrCW93oqZufbY2P3F9ABACk5Tb8zE7aun7rHpiHXjZwMGKDgdDv/EH/pn9/zirIB9q+mFWJtvT4b1ori/SA/An38jG4Lb31ruf/7BQwGmA/SGTlPVG+BkgQEKTobtg956ktszD3mckm81rbeCJ4uJtaql0ASRbYr7a/EAbH/rabQ84Q6P/V36YfGS/7A5tOCmxclN82DCQG+Y+5uo3rAxtd57U+lSnFq47WcBBig4DUhJ3ZN+zDZ7sZrLhX26gy3eWKqm1Cv+yHhdFn/Eau9OfDptvuq4p16HAKycMppKacL90aa5O3hIvNDkKO1K3T47sfxhy7oST5X7fNtMepoHpwT0RhwfojfM4vRiKb5u6vdl034lxPJC/hYOMx4bMEDBCUAK+MdS7Gg1/3QlPRdyMlmLRV+FNLsVrx6FF0x94o+2DyrAPtu8iltPIVdPZpK876HQ5QEFKaenJ2XoyUkyeFqX4SFphn5XW5+6fX6txDxmEgNgUKA34vgYvbG9kSEPmdhQGU+BU4oqDpas0DnlvodOOSpggIIjR66SL8RSrMQ+11LuZHKCj0Px0epxkHIihT535HRFk4l2HwzoPfD8rpygYyYxAAYDekMxWb0BPgMwQMHxomKejHKsrbLlZEKr72MydK5+ak/Fwh/TtL3RjyDJ7jwehZDHISQn463ZZHrSTfqMv9DvyknMtE2/7UcAmEBvFExRbxiunmTMbXv56lFU6tFTmbjzuYDBZIEBCo4TGZhutnWDWzzS0CFF+RqpmOIfnh6R2FrZTn5fTUxTOS3+0ERA95I7bGKQkwjJST17MyAn+8xOVtkhYtrHeFKwHQ9GA3ojjo/SGyVMfDjpiD+L+n3ZRLaniX3F84SPjf+o87ybvwEAR4fZIpMT6jCzwPiYyS/bDDWxAfDZOUG9AY4OeEDByeF9O4hNJ7fda7wEzElkdBmp7bxhtuoASAH0RjufS0ZgaGCAgtPBGDnFlkw92UeS8La5pLegomzdNKDiDW/tdXj+3WAyAuBIgd5oZyS98bGhDGBsYICC08CN7XrzP45EomKWZNzhehFxwMC+DWUh/tTeX16kXPEmnFCsp2Eh5P34f/dtfy2eOc+/G0RGABwx0BvtjKg3bExpc9KPlBLyYJT3eychtGDywAAFJ8Hh5VnsxFysfkYondmtuJPPU255o8fh8YdY7qhMWvGHDiPkijfVYZrtjbhY7lQMZFCJqpOo5qRoxGp/CBkNgjkAgvkDDA30xgnpDXB0wAAFwMtBvDzvaKV9Lb41zyEFsy/iK/2z+9tf7R7+yTdTz8V5zO+KK/GdFL5Y/47zZgAABgR6A4BYYICCk2D27ZpU704si5c5N3PQr5WbX39rfgwLffPtei6X8s57ywMc/gml/uO0f5DZFzUlibg5aSvUK5rn54F70aSXEQDHDfQG9Ab4OGCAgtNAbittaEm/lm/2aA6yV0HuF/q1cm3P+Jvd/hLqveUX4beicMqM4upJ6FuhckNbZDI2Sz5KRb6uLvRMQ8sAMgLgqIHegN4AHwaeAwpODhmIL2OhvHjfftKGeWae+b8ancqMQR5m0G8x8SPjzJoPBYRIL6MW5ClaeZCBhXwPNB4uDcYBeqOd0fVGDXNv8hWqg/8WGBoYoAAAAAA4AmCAnhIwQAEAAAAAwKggBhQAAAAAAIwKDFAAAAAAADAqMEABAAAAAMCowAAFAAAAAACjAgMUAAAAAACMCgxQAAAAAAAwKjBAAQAAAADAqMAABQCAT4R6ZWLglYpVuPkBACAGGKAgLeo9w3LCKlLotcRs5CsdS+W3TIzc/AY96Trp8lGEfyYyf60+lGoCkm/7qOTxpdJvtFwzen0cnD4Reje2C+RJqek3erEVv+W7IefX4lvUa2S4+VuYjH7o0r7y1ZrVvDf0aTux/bmWL7G+4pbPrg+rfSHP2PqcKjBAQTLke4LPLpai+qbg9SLe8AihBm/tfeI7sbzwKzlufom6B1IMsa8t5+T314dYL6iMOMWbkqHrY2Xj6xNNQJ7Dcni8V+8mz+7iXmPIzR9iavqBhzRYfe91X4tFwIiI7s/GcKvnS6SvuOV3qA+vfSFPTbg+pw4MUJAGGmA/llr1ZJs38fZm0iZTn+2WP3qt9KRi0IM3ExtbNqX9aq6+Xy/KEzw3v2J7Iy7MPcxX+/walXzvHebkJ/ncy/rMV2Lv5qOkRbQWv/MKzcTtazlPKRmZzq+/1evkKV+lkesj5W9lQx0il3sQyFOnUH16cRAvzyRfquPPK/NREG7+ABPTDzmx7auYi9W+nE9Xn4yIB0/pnP48+yauqarVfEY8Yn3v8cANWT43f6f2hTyD9fkE9DZA7YrBTa2rWWro6jXBFUBMfpvHU5CtY/krswVjXevV32io0CD3S4NXro6avSR2qyKRu7719/gcXp5JbegB9uROVldPZhJoUCpRmIlQKawn4RY/u301g9id4Ln5JSRjNYNppfh6W59+yjDzH/5q+XiMiqvvWgv9+RfTuPZ3M3HXWscAA9dndntHnxhjoNQhmoA8NYnq42P7IOT8Gm3YcvMHmJZ+6IJcNLyKapNc/VzRr/pg9mdVfj1fXv7ur56ncoYun5ef376Qp6Q5/+eglwEqtzzsisFlt7xoMOCM0acausx64TOuuPm78eLbulkvaoblYPc7uxV3ISW5/U3fENldbcB2Yftgt0nW4j6NRZtPANe1QLGteLAy+/Ov4yA7iL+qiK/ii+f+6xM8Nz9hZDxf/YqTMTe/Ybd8IIm4UB9RrjOqra+yFeyW6Hz1szTRdmW4+lyJp7eyMRAE8lSkrk+BrUesYcvNH2Jq+mEEOvbnGrMvdFcehi6/CW/+oduX+FTy/Dz09oBWXcpvbxtSWcT6vmYgHh5/qBW17KhyleFeZ13RLtz8ndgtxVLp2U1Rvil89/xSGzBD3a9VkmuPBbpVpwCoit/TTEnFKjOVl8VMAJWDCtpb7MT5dF7lzcS5f1lcYvfXls7NT3X994f+KxWo0IsG46lWyROAzs1PQhd6N1DGOFkPOE3ylxe6j1D/a3cUWmWeoN0mVh/IU5KwPlUOL0LZCNn3OMOWmz/I1PRDKrbiRsU8kn6vxCiw+3MTh39CliTm53SXBUOX34g3f6r2hTw/HWQMvadOZFy9k0TfyehyPt+8k4nl+bwpMfNvMspL+bNN7bv9aq6+yzbu5/t3mjA8n7vfZe+b0uf+lPZ+q79pPp+v3velz6eUqnW096JTtuHJ05e0jClV23e/eqe5p/Zd5/yNqVx3bv7gdZ4+60u2H9MiyPN90Z+rqd6/izRcfcqpLS/kyZMnN+l6xeoifv5wmp5+6Nq+to2KxOiXpRRzr0Udq3Uaunx/asrfvX0hT07+00u9PaDq5KG7WqDk2XEuLP3YbWRu/s5kou5Y1PEab54txMHul37pJ/VG6UEpOUHN1kCKU6hjUFr1mgD/ds9PO7nXdi1PFDvyd05dzs8LCXHzF5QPMZCSyD1b/nAFXv6Zz1Wz/k1r/za6e8fkKdSmmOaPqE8YyDO595PKVpso0bqImz+eqeiHEKH2reN6vH1w9UnB9ibGez50+QUx+fu3L+T5mehhgMrtpYjHFVhskH6kEmDnH5yB75eYfbtWStQ9Eae3331G8gTZLcVCjSg5pmjw5icHbYxWD2a34tWGO7hIJUfL1xrc/AoZKlFddNBi5Jee3OohGZz8uv+oGGKjmF3lFnr0iKQ9NtAumirJ3itNzGWlPnR9ugB5po/9LMqODeHh5o9mSvqB3b4aeajJzb/XDUzZfe3L1ScF0tGh5hoyTpoP8Q1dfkFU/g7tC3nG5T9Vuhug5oQkSbCyYgjHZ3LjcNLH7XRkjPs1j2qgni5e1GXaEzHEpJQWNwZLy6c0ppLFuciDLWXZSyUn8vidaunc/Dvhba5ZU6B4fP48HlgqnFwx60lQK93QKeAe3rGrp7x/uocwPqw+QSDPIbyfqmwyxOIepcTNH8NU9YOHhvZtQhpQtn3rfZerTzTxxsnQ5Wva86drX8jzc9HZANVBu1JPRRpHs3O1iojbjiK4+Ruxyr0fg9+vgiYsdRx+J56lBaq23yMV54cyE1/MCM021RUkyc4+ouO6/+Nc6tgJM/7NLr78duvUO+lYBfr1S15/bv6DcQH4vEozK7yGU6KpvGPuKewp1McF8kwrzxx7ujd27HHzR3FM+qEg5qkFTXD7syXWOBm6fEtc/uHb93PJ8xNBq0RvcGhbolWKDs4tHY5xg4/rAey0sjTf1QN6N1nP/HmwufO5G4BOqRzo2xwY7Utj3K9ORUD3Sl4feYCBlXK5xN17VHJkXZKzPRwWOtAQk8eT8jap/mZDCuZvqr/TxqXPmfmb+0IRiO5va1seTzY2Nf3u2PWxsm88YAN5dqpPOPF0HD8/Ix2BfpCpuR0bUl63Sn6uPmntJ5U0dPmp6sNtO8jzU6XOBqjbYP7k63BFY6bP7zRwJWWZ7tT+DhSpaEa5X526KE5OKpRs2hO3br2rqfk+Ku3WODibZekvm5s/XH9fvVj5O/Uf5zdalRazrw1dn9byKVXKgDzr3/dKpk7RY5ybn5lC7Tsl/eBtq9ww8qWWtvUlTt+3qeT8GLb81PWptQHkWU+V+nyG1D0GVAZ973VwboGO/yBhm/+vomN0yACqkW3qb0Xg5ZfxVjbo32LiUb6b/+3DKPersYeRZPmpzwFI0j8HVKMCyms3KwO7QychbdiBhnPwgSZKFecVu5PRlt9ff9lWb97tElZ+dUii2j8NcjvmzdMf8tfbkQw7BuTpe/aU/UH1CQF5pkW/cCI+hIebn8vU9YOksX0bCOXn6hMuQ5fPpVv7loE8Pxf/SSvU/A2mgnx152KtBmP7K8EAAKCCfN2ufARRbKwZNz8AAPSk93NAQWKM8UnrrgFOxAIAPgPWmxnrWeXmBwCAvsADOgVyo9Mity3it4EAAAAAAI4JeEAnh4wrhfEJAAAAgNMFHlAAAAAAADAq8IACAAAAAIBRgQEKAAAAAABGBQYoAAAAAAAYFRigAAAAAABgVGCAKrbi5uxMnFG62ZqPkjJ0+QAAAAAAxwMMUOLweC/sUzjXv9NbiEOXDwAAsWxv5GL4UjwezActcPMDAEAMMECJ2e2dsG+A5bxrOJahyz965GsAjYf4MvUs55RtE7zQ4POyFb/lanh+LeJe+c7N38IY47GTPjmIx0tbpxhjOy6/Nt7dFGfI1667fKRfrMMtn10f+ZKUqPyuPDypof7s/lCrD6W2DsTpD9H3WybcXi2ysckjo8Hb94OBAaq4Ek9vb+KN0jCvQR66/OPk8HipB4l8B7X5LCWqfE/Z68UAhi4AR4DdjcnubkWMPcnNH2Lo8dhHnxwef4gl46LW/MboKb3gTrETy4tmI8veQ/26CtzyO9RHGTO1C8L158DtD/76EOsFyeyGlkpluP2hy/1GtxeXEdp3CsAABR+CHLgXVoNnG7FfzfXfqaAB+cOUn2208a/SRvuid8sfk14ZApCeg3h5pjExX4m4V75z8wcYeDz20ie2bvNMZDGXxeSffRPX9N18tS/ulZK5XbG+93gEtzf5PVSve3utLAC45TPzS3lqY0a+ma/Ib+W6XtQNPgX1lb2TP0/V+nP7A+W/l/XxlK8vWQs3uo3bHzrdb3R7zcTtq/NdNZl7nl9/K64ZuH2nQncD1Liq9UqlOGSjk69zGje0dTNXXd3NSwxl2bt5W635mhvdd43fLd5atqc+/tV7x/I9dfeWz5Z/D/J7TleuDkswg30At/Dh5VmteuWALBV/9WSUCq0MH5JKCYBps31QXrvSRBeCmz/A0OOxuz4hPf1DesjmYvXrpzg3nzYTm18bHa+Vdypf/VzRlcTur54Hc0iHKwuIyt3Xr6vDLZ+T3yw8VF2ehCvN2e2r1+Djwu4Ph786v6cvXn3XVtaff8Ud8PpDl/vltlcTtpxM3JXKGLJ9p0N/D+jfBzJOFmqbpmAtFgFj5cXn6l4v6oaWNLIaXPR+g9UYfdWyifWifyyEXCX56rNbXjQYoTyathhk+Y0xNE3yb8rfge2Dvee1uE9wnxoZllAe7OkoFMp1LXBtKx7syvjPv0kOSgDSQ7pRuZCqE10T3PwhxhiP3fSJ3Uqfr36JmNvk5q8x+yK+mj9LbH8rHd65XEtT+U148x/EX9UkX8UXT118Bh+P7v1ht3yo2BW2n1JtS5Xl9IcO95uovWyIy3z1M66uSdp3OvQ2QHfrNRknehWg3b57ob3WDcbKbimWSq9tClex8RPvnl+cDlesDFyX+Nvbhj4h1vc1g7KIy3Hro5N1RRfoFYP9vs1F7w6aatn+a5nlk7Gtbc9K+Xu7glkKn4OgLn8jn92zePGIvwv5KirJZDQGRqFUDk6oBYRrrE90VQhAcg4vQqmv7HvcRMfNH2Si49FuA9NcFOXB4ub3cfgn/sh/5+clT97hn/xUGmSivnPGcSY0lN+IN/9MnLdNh8Tub9fW6tAfrn7mdsWCZKL9T9LhdKHnfGoTluO7BP9+07SXNbYZ82qS9p0OCWJApYH46qwCyPC6MwZlQwdVMR9ub7Gdy+lwemUgDavqKoZWNsoo24nnkoVlG1Ne49ZHc/VU/ywV0k3f3QWv2aqjplI2lXrObsWrsZ79cRzV+70SesG2E531QxVZB2XcDuWxHBodomDjdbKNXSQB8DnQuxikKyKDObn5eUxhPJLxorbSaf6Ksly4+X3YMuieK4e6DsoikwdGjEHlsluKi6jwp+by/TTln4kvym1Gxl51p1GGY3l26XJUXcsGWWvYWVR/0A4d60hSu6BnhfHZL4yLf78p2ovt/UzWvtOhvwHqWyFffadhSni3VDJRfxKR9RYWBk7RwOXOrJLZBi8ZuNbSz+4GMjRn4puM8rV1SrjFLQegtj99siGsPH0eAs/9zmKWc5+E0qraBLD30lUAHB1Gv0TrRm7+eKYyHu1uWbaJW1Rz8/vY3sR466o7fi07ig5x5ReE8ue7Xmt5wrw+90rm53Gdozlkjt8fvHPb+nersddG9/vt2l5872fK9p0KCTygQ3AQysPNwQYpRw6KLkhPpw0XKK302pd4cTS6yeO2CEAFaqOFGoFyDJJyyE8m2pgfAE6f/FFKkc8g5uaPZirj0dlKj5qYufk9yPh+HVEW8tb5dvxm4vaXNo7KIWpl4sovaM2vdr1MOJeLNBLr8WxEOeQsT4XLsu4JZfWHg9rqVl7S/CR8Yewt+j7vkn2/ku7txfV+Jm/fiTCMAWq9kZ2xLnHZwE5nriaPYLvHpURy9ZT/fh7XKVdNKTyijTFQNmZmmnEc08M12PUKtdRVJh4XA0A6jKeFJtK43XRu/himNR7tCeyat8tu6dpdLvpMHi7l5q8Sbww0hE3NwgdJhjNOiudX54mMRO0d8h0g8kDzpbXfikM8/P6Qn++Qdc4NVW30ek/Nd4J7v93aKx9jdO8x3s9TNT4lgxig+SMWejy+Q7vaq3GeAWbnatWRwh0fi/KI2lVYr0M/dkA2PNrCnLgTX790lufnwi5g5Bisb5ml6J8AHAX2tG5sX+fmj+LzjsdYY8BuLXtPlluDzKP/xzdO7AKF/2as4pQ6vz/okDzK7/HKz2xhgzzVxH+/XdtLwvF+nrLxKUlvgG7tw1kjV0gNzL5dK4My+hFHs1uhzz5Jd3w9AHh70/cxTDJQ2hdYbLcL/I9wiMPGl8pFdqWeThD0h73GU9ZBrfhjAuFHIH9WanOb2pgeGX9U2vpJ1D8BmD6HD3z0Uple4zFivHPQjoOKp0slu6Vb7LzJw6Xc/Bq9ZRxrDLjzXXmrmuYdE4dY1v+88vn567jxmrGHWnKDiPqUW/2u/aH+QHjbZ4nEDprQ/fLby2K9n23zz/jt+yHQoHnvlDbZO13emOarfeWa/TsNVvoue9+UPm9O+9W8Vm6R5u806CvXbN7JBvXklamSf796pw7kyVek8j2EyvbcL7t8Kx9/qpVv5Z9typ9TsnLLNuXPu6ZNFqhH1xQhH9+91eTkzaNTqP+kkg0S0mSTGWPRY5abn5m6jcfI8d5Zn7jJ/pZvbvGl5vzhucuk+ep9H3tNpe7c8vn1aZ7v/G3FmHtNYvWH1vblz+9lmXLvl9detWta+uLw7TuNNMAWvI7p6PtYIolaedrnYEahYzh8McO1xxuxsY9/qqJXv/3v141ncUlVfnfyE4IDeEb4kJzMY74kIa9w6dBYjpbnsSwQAeiKfZRSrKefm59Lt/EYP96PHb985Nz1NgmPFi1MlJeXUxV9jX/uZfUHdUjIepwrSI9fw2/0oe1+2e1lD7PJexzk8WbHx3/SCjV/85DbItLfe0zuXgAA+AzIsBm5FRirn7n5AQCgJxN9DBMAAICuTOvB8wAAUAceUAAAAAAAMCrwgAIAAAAAgFHp7gEFAAAAAACgA/CAAgAAAACAUYEBCgAAAAAARgUGKAAAAAAAGBUYoAAAAAAAYFRggAIAAAAAgFH5xAaofnn/2Vk53WzN1zW4+QEAYHpsb6TuuhSPB/NBC9z8AAAQAzygnw35yr0RjGg9aTnp8pFM+AacOl22zHK1clsmRm7+AnfBEXNNZH75AodSfSiFGqCW31e2f3FUS742iCpf0vIbTe0bXb4Doz/w5LkVN9W8Zzf0aZUe8pw8W/F7Tf/Mr0XcK9+5+VsYWv9gfB3B+KoTni+6yr97fcq/2S7T6PmuS3spePU5Fj6xAToTt69v4u1Np/1qbj5vgpt/ehweL8WZfN+z+X/LehGhiCJRv0GDRL4kqw2b11enGkZp1svdieWFZxLj5q9wePwhlq2VKojJr5SUTzDrBcmhrhj9+ePqH8PUymf1B4InT6nAF6Keey0WJ6TQ2zg83isZZHe3pNHa4eYPMbT+wfiSHNf44swXPPqN91j9z6l/n/7AnY+OBXhAPwtkkP0wPTjbFIb02yZTn+2WP/pPwrS6uzC/MV/ti9+Q6bU8gcmBa/PK17m2GvSzb+KaslTLNdUX6/vKipOb38XKap6JLGadEZOf8txL3TNfib1Tn6JOa/HbUUJSPlpXZWLj5LVyWi/cCaC8OKolc9Pz6295G/DKd/DUXyVP+3LKZ/cHpjw1c7Ha+/LSJPDgZubL8zg4iJdn2U9XIu6V79z8AYbWPxhfKh3H+DJEzxddxyOzPhbbV9v0P3O+69QfJLH1OUJ6GKDGJWxdzVXXcpNJT8Jkb8HU3NbN18iGruZN5d0bjVxGgU7J5PDyrFa9cqCUXt1/9WQGQcugbGUrbtQI04P+9basCqrMbu9oKJrBWKpQE1oJVcu9+rmiXyR2fysGJTe/hfr1D+khoPv49VOcm0+bicx/+Kvl7zFarr5rBfrnn62RmfiVLJ+EK53Z7WtgAvBh2yUTd7ksUpbvg18+uz+w5CmR/eFVVLtl3h+i8cnzSNg+KC9KtOHMzR9gcP2D8aU4nvHFmy+aaRqPXcd7rP7n1L9Pf+DOR8dFEg/oi8+1vF7UDT9pSDZswfgNVmPkVssm1ou6G126uPNVnsNueRGOA5oY2wcro7W4T2I8FwPguhbItRUPVmZ//lHOjmx/U23pF1a/aoPez5V4eisPxk7Mvoiv5s8oWvLbrY7Y++Dm3y0fSOIu1MeVq0GIr19sAQfxVzXJV5F/5OCfAPzYLdT56qcj63Tl++lSfrf+ECfPdPjleQxYucQaztz8IUbQPwaMryMZX+z5wk/q8Ritz1n1794fuPPLsdHfAN0txVK2RLYp3M/apBe75xcSvaVYqbgu6Le3DX1CrO9rBmUR96BXGcU11pVep+YKD5Q/VYpVWiovixkAlYME2lvsxMk0egXbOfz7Q/+VE4zQiwbXC90UkJ2Cwz8hf1nMz+O8NKH8dquD+nLUipyT/+qn0LstMgbJevBJmV9e6D5OZRSOiZk4Dy/TFbu/bVK1k3vdO5Cm/CaGLp9gybMJ0klqQUz6JWqPuUmeR8DhRSgbMPseN1Fz8wcZXv9gfNWZ8vhKM19wx2PLeGfoc179O7YXdz46QpJ4QFVMj9sbbed1FIpeqUhDsroKo5XZXhpcO/H84jaA7VzyGo8r/cn3mc8VfiXMAuN4mN2KV2U8J/AQeqGBSAPFeouzzd4om+4c1AyzE8sLo6BcaJFykTCcoIAUotqeoHuIOiQRym+/owVSu2YluPl1SIBdOCmv/1mhzMvbYjPxRblpSflXPfcyPMOzI+Cj2TvQo3zVlmWFWy0iVf3DcORZUA7RkcaPXBDXdYmP4/V+0ohXuyqxhjY/P4/0+gfjy3Ak4yvFfBEzHuPHO0+f8+rfpb2488txksAAzcT3mnx053UNqKLBbGdwktmWL60ArKcqu4uaHCzqpFml/CTj8UQoeR1MgHna/i0HuPU+y2Qnl1ThBAXbm0IhxtxDKL/1tmebOKOfm98y8y2F179ryjb3gqsTp05/dkJY5uehgRH2DvQvv8AXQpOy/BCx8myGJgWqV32Sr3LE3k+qu3qUUrQu5eaPZ2j9g/F1bOOr63zRdTz669NVn8fWn9te3etzXCTxgLZzEMpjzcEGQUcPIloxXMLYDEIrs4UatLJj02DJT+rZGJW++DzctBj5pQdfOSSjH3Khodo6sBp3CeanVajd6oiaDLn5Fbp/Kq9PfrK0UFa1R4MoL7gJH3GR1zbFnzi0egfY5dtFZSUVLpKyUu9Z/3aY8jTIoH+3/sUp1PCjWY7Z+2nrntU9BV64+aMZVP9gfCmOanx1ny9ix2NUfTrpcwmj/pz26lyf42MkA9S6oGWDFZ2hljzSjo5jMSc2qdUqK5LmeNHPgxuDouVTEjU3jrKRnfA214x5UKiFpMYnYU/oSiVfWp3aLSf61nru5cE6bn5JHs8s65BPvHrSaT4FLA8OlPuyvFav5nwHOiyx3oGu5TtcPeXjqx5En6D8BrrJs46coGz+ZlVz3N5PVXea6OJ207n5Yxhe/2B8dSy/gXHGV9f5ovt49NWniz7XcOsf117d63N8jGSAUpsoDVSN8wwwO6cmISLd/ToomJrxCD0Uw2MXAFKf1F36+SNSPI/ciMVu1XhPdtoJ5uuXzuVbUhufY6FDUKgaHq/SzDZO1ClgayA0v5mmn7euvfwm4k7Fdi/fJZ082+knzw/GntaNHdvc/FEMr38wvizHMb76zBdTGI/p5rs07XWsjGeAfrtWBqV8JFKU1T67FXdq5Sfd/fWA5O2NfwugdvKeVgnKADkmZGCyWvGkO7hjY1BkPFFpKyd/mG5gxZw/h9Uvc4nbvuX4Gnvy0K/M4nFCLKKMyfj81W2aItktp8JzLw+5cfO71B84TPU0jzVpU1hu/FzzoSuj0ELt2UBc+WVyA1/4YsHLdCm/jT7yVOT9v6n+3eX58VhZxHqKuPnjGVr/WDC+jmN8dZ8veo5HT3266PMU811Te/WZX44Ouon3bmn/TsJ4J9G9b7zf19N+Naf88hpfmr+TUCvXbN7JBvXklamSf796p7bx5LOJm1+8z1f77vl7pE2WvkyZQvLPNv5rinY2Kdt48ugUbN/qdRHydK8J9x2T5qv3fcf8/mTv3dc3fSmQn9s/A32/ua2c+w60k07c8hljsSW/t3xmf2DLc5N58tjU3L7x8pxgMjKK1iHc/MwUGpO99Q/GV56OZXwFdXSDfKPl33G8l1NY//Pq362/lRN3Ppp+Gs0DKlGWvXrkUiw6ZsIXw5ltKo9SkEG+tbJ1vBF1FPP/x0F+Yi6xJ0LJvyZMvZpqdhDOxK12RStCqzp/+bKtaLX2gdvfk0AFodsVbAXpoY14FBAZBmrV2yhKG7wu27RDAF9r+R70NXGPMepSfiMJ5CkJ1r+nPD8a+yilWE8RNz+XQfUPxtfRjS/2fJFgPHLk2Ubf+S5pex0p/5EApBUOAADgVJBhPHIrsCX8JIebHwAAejKqBxQAAMDwTOvB8wAAUAceUAAAAAAAMCrwgAIAAAAAgFGBAQoAAAAAAEYFBigAAAAAABgVGKAAAAAAAGBUYIACAAAAAIBRgQEKAAAAAABGBQYoAAAAAAAYlYQG6EE8Xp6JszM3XYrHg/kaAADAh7O94elmbn4AAIghkQG6FTdnF0K9pjUG+do3Y6ReQqudFk7b2nSzNd91xre48aTLR8ppabmmlNdhe1PJGzvxur8XuIZVvhxXbl6ZbujTdrTR4CSObGzyyYjbvrX7pRS6gJvfEL5fh6j695CPIbo+o7MVv9f0z/xaxL3ynZu/hUH0Q0FN7oOPX0reG0ilfyjVyu/SPzn16dH/O7Zv7Hjp1L5OneLtjcj+YIge7731YeL+/MEkMUAPj/dC6igxX4n925t6wb5O5Zf+Hx4vtUDkO4fNZ+B0UO3radv14jgWGkqJLFRPdtiJ5UW7Ej08/mhdgPHKlwpkocdVibVYBBSKHWO1n0kAt33990usF1THuiHNzS/h3O8Y/XNI+afA6urs7lbE2JPc/CEGlb8xMupyH3r8EoH+yWHo8oemS/tGj5cO7WvL9tWpjZj+IOGM9zT6MF1/ngJpDNC/8k7nYvWrWUnJhrqwEsk2Yr+a67/BaUAK4odp32zjLEI2mfpst/zRYxU2E7evTpnVZH5jfv2t3v9qiyKTXst9VfZPPdYzsXHy2X66XgQmAHvv80xkDd26W/k0pvZFXpn0rZISevDUhlbLdozNV/vSdeX77SBPbvtS/nt5vx7560vW4rd7C9z8kuj7JVj179jfOPX5EA7i5ZnqRzKOe+U7N38Abv/hMvsmrmkoVeVuihfr+waPlMTWLTB+O/VPiSe/Sp7+GV/+0PpwBP0g4YwXZvv2sjdi+oOEqX84/Wfo+WgqjHYIaXZ7R6I0wnzqq80CkPCrWwCtq2vqSG5+mRpXGJ68/vKNC9y64qvXNf2Ap/5tqx02+W+kW1UfXp7VKlMOxFLzXj2ZQdNgNPVmK27USM3EnetuZ2EmWmXwPQm3+rPb1/AEI9v5h1xhywXYT3FuPi3TpXw5CZR3ECRXP1dUig8rB220vnaWhV+e7PY9/NX5PZPg1Xc9a/z554wbbn7m/abrn039LZX8B2T7oLwiXsPEBzd/gOH1gzaaqnLPx8vur9bDNWLGL8Hun0ySld/UP1ORSD+wxwuvfbvbG5H9gVt/VvsOPR9Nh24GaMVIUu0gO9hF8ZlKJcvpSjy9lYWZGrlq8Lnbd8uLsJGob6DEelHf5mxyocvyG2M+iBffdetFvU7SSG3YwgjGiTDZPtjfWIv7NuM8imLAXNcCxbbiwawSaYQ1yqgrdotwvvrZo28dhHLii6/ii0ePhCYAu9UxX/2qGYsF3cuPZvvbyCFUj3b88uzevrvlA+VwoTGnXAEkDY8wovOz7jdd/2zsb4nkPxxWjrGGCTd/iI/TD2L2hUZdM3Hjt4Dbn7n0LT+NPmwmmX5INV4a27ebvRHdHzrWP659h56PpsNoHtDhKQaBXJG4Lm69AqtTxEnUr7Gu/RwyDrUNWcm7tyuwpfAu4Onzpbwu2xTXmMJ3zy/FgKRuaVeWrsv97W1DnxDr+35bVA6FFy3VKtkMmMpBBbUgOHPiGBu9EF2xyq3vfczEub+LlNj9rdTebnVQ24ZXwB3Lr0F9RC1QqA9W9kQP//7Qf+UEIJzgc5MCi6MyTfLs0L5XP4UedjJm1XrxSdlemsOKJLOyp4SXn3e/qfpnc39LI/8BObwIpR6z73GTMjd/kI/SD8Thn5AtI+bnNc9T/PgluP2ZS5LyU+nDJtLph2TjJdS+XBj9gV1/VvsOPR9NCDJy3vsmsqfeqTHeyTDzfu9LZBTSNeJ9vtp7v+en/Ts1MKMem3cy7KLz63sU79nG8/0mU9/RsuN9n39u6+O7xn6XvZOxqT7T8mioy371Tv0xoaxSJyPL/P6tbO391+83RQr3oUL+1eRrQ9u+NHjL3xnZ17/z3ZP9rN6O/PJ1svdYJL8M8/IbU7vsm+XZvX299fLcp02x+Xn3m6Z/hvpbCvkPmXT94nU0N384pZE/P9lyfWPe95v2s+b7ju2f7m9Xk3cOMSm+/HpKqQ99KaV+SDNeintqu4ewbGSq1zHUH7rWP7Z983zV7xLNR1NJJ+QBnYlvMkrZhgK0raLsyim7i3BVm0eR0Mrvu28VevWdviG8K3jfNTbIu9gi0Ae5PGEMMplt+XYP2cdTWvWagOtenoFGuq/2fSENuVdYnUisy14yPy9+x3rPs03cNg+3/GbcFbSPqgd9n6+8w+EWcfLktu/Mt5Rf/6Zf88PNz73f7v0ztr91lf+QGP0Vpesk3PzxjKcf6C5umr2H3PFr4ffPOqGQqu7lp9WHdYbRD7K8ruMl1L5cuvYHbv1j23fo+WgqnJABSo17+5pvb8ut7wvbaL7BZYOCoyZ9Q6ObP85l3sxBKI/+sUMyXyiNIAcCDcb8JKDZoklIe6xTw0lO2z9oYJe6xexWvNpwBxepRO01FmerI1rxccp3UH3aqT+t5NXnvhhlKqwWtK7k8Esrs3LIR5mo2DFW+8rtpTN9StRMRK5yrj9KiptfwrzfHv0zLrauu/yHxNY9866e63DzRzOifpDx+jqiaVM/hNJl/LL7J1P/sMsvk1wfVkivHyTdx0uwfbl06g8STv2Z7Tv0fDQRTsoAVVw95YPLTtZqFdHgEWV5FRtjlMwA6xyHMhNfVCS17NBGMfjSZHuXa4DrFWGpqinjdBTdV/uyf9jxWw/iloHrFZmTEtWrAxnvY9SpOfGp+pW7Os1fxlB4sssHzeLKDyENUt2vd6LedX2fEbPwQYx2efLbN4+vlhNEPhHpSdDW3z0Vy82vib3fvv0ztr91lf+QmLrTxBX3KCVu/hjG1Q9txkmX8dutf3po0D/9yh9KH1rS6wdNt/GS1Pgkuuvz+Pp3a9+h56OP5/QMUAftPTKrjN2zeHFlPztXq5S47Q07wBoefWBOxImvXyoDLB7tmt+J51IljwVrQMvxVV0RFgMkxeNcJHHeqHbiTq3aCbkcYJ+ONOXbrR3vJGIngIb+GeM94bavDimh/B4v2swW5pyK5ebn3W+//hnT3/rIf1Dsad3YscfNH8V4+iG1cWLh9s8YXP3Tp/yh9eEQ+qHreBmqfblw65+u/ww9H40MWdXe4FBOohUUO9CVrH4VSNscFMxNMvDZF/hrA6Lr3+l6N31Xvh9b39p9OkHBWSkQ2hcUHEhOOelk0pDy34qsW0xqkoM9oBXqHzF58mTbk9ffbAq1eTUVbR4bqM8L+uaXTymXVaX+jf2wOBDg/41IeTLbt1nOVkaUnCB6bn72/Xbunz3l0yr/IRNTB7HzM1Jn+VOK0g8N/YSVbBn132H3z4bUVE738iP7Z0Nq/l2bhtEP/PHSr3272RvN/YFb/xT9Z+j56CPSeAao02CNqbPikKloeF/yd7zQNdX7cTqKJ9XLt/mbBnY9uR2sntJ1omIwpDV2Q/VvHjAVucYOwta+wmnbcP64wS5TaMAzy88Vty/5+0Kw/zTIK16ezPZtHe+Ve+Dmp8S9X1b9q9f0lE/M9cmTkWn0GOfmZ6Yu8o/VD0HZ25Sf0m5KgfHL7p9M/dOh/8sU3z+5+lCnVP3f177BNqv8Hrt9W+VJqfWeQvqcV/+U/ad5rFRTuP5TSCe0BX8lnuwzOUvouEr/c7F0jAUZZDWyTfUtNG68hkuofB4qZMB7D2lJ/xxQTekQWI6WT/NuCcn1rrjGt0WRY4OtZZkdA9RocqVFV/0NQz503lDd+9Gl/FD9/fKXffnNv13FlCerfVUQvQ2yryC3z6r3wM1PcO+X3T+TyCcg/4HRL5yIiy2WcPNzYctfwdAPQ9Khf/poHL9dyh9aHw6pH4ipjRcurPon6D9Dz0cfwX/SCjV/AwAAOAXIeLiUj2yRk1vMjMXNDwAAPTnpQ0gAAPAZsd7MWM8YNz8AAPQFHlAAAAAAADAq8IACAAAAAIBRgQEKAAAAAABGBQYoAAAAAAAYFRigAAAAAABgVGCAAgAAAACAUYEBCgAAAAAARgUGqGIrbs7OxBmlm635KCnHXj4AAAAAQDpggBKHx3uxNn+vf8dacAfxeKmNviJdiseD+dqhW/nxDF0+AOB02N406yof3PwAABADDFBidnsn7Btd4941LD2OF0K9JjcCfvk8WOVvb7Sx/JlcpfI1g2aRcJl6FnXKtgleaDBdtuK3XK3Or0XcK9+5+VsYerxY/ZanJsPZ50Bw0uUj5QjhXt9inHP0T7R8Ota/Jh9K3h/glj+0PFPdb0tbRctHoxdnborvCzbF9P/a77TK8ziAAaq4Ek9vb9Ev+s89jvOV2JvrdHoVt14lzSufz9DlHyeHx0s9WOU7rs1nKVHle8peLwYwdAFIgNVd2d2tiLEnuflDDD1e1CS9UJrZYSeWF+kXhYfHH60OCK7++Rj5EOsF1fOGlhofR4w8uXD7A0s+xpCsZ28uv0v72j7kq9YpAAO0A4e/sgvNxepXf6UMhkEO3Aur0bKN2K/m+u9UkAL6YcrPNs4iZKN90bvlj/BKGIDROYiXZ+qztHCOe+U7N3+AgceLHO96ks7ExpZNyY779aLBwKo5EUx6Deh2ey/zTGQNaoWtf7rKJ7b+VP69lI8nv/6JtfBGb3HlM5A8cyLLZ/cHrnxm38Q1FTVf7T15Kfd9xUPZpX23N3kfqv5OUJ5HRH8DlARbdSl7rXnj2tbfFYdmdAqsvjzlt65mPW70+jV+l366lXKP8lPX3y3PLqXUqq58bd5uucyb2sW2X8t2Qyytv8dHhyUY5TOAW/jw8qxWslIxlIq/ejJKjlbCD6nuBoAEbB+Ul2l+/S1u8uLmDzDseDGGsnQK7J+EW/zs9jVsYLEhvftDerGkA+KnODefVuHqn8H1yeGvLt/TllfftRH0518KZc4lTp48OvQHtnxm4vb1TbxWtjyvfq7oV4ndX6pFAb99aY5Vc7W8h/rvnAq9DFDlHva4lHfLi+Ytg78PZGws1LZOwVosfMaHNJwaXNbBuBWPv3q96GEsVYxgXbx2tbsGXP+4yoHqz2V2K+5CSnv7W7dfdtcQcsBj+2DbeC3uk92kDEsoK590FAruuhYYtxUPZtVKGqukhAD4OEi3KBdPJu6iBi03f4ihx8tBqE0p8VV88VQ1pYFlt4rnq18tuo+jf8bTJ7vlQ2Wete1M0vMJb2Di5cmhe3/oLZ/ZF/rVKh3a18yxaeUyPXoYoO4qw3ENU7Jubh+79ZqMDfeavdDZq8aHXQGUXehvbxv6hFjf1wyyIo6kXifrGi/QKxj7fajO3eCXP1j9aZWVl2ULyjbFZya5qyw7SH2n6rfqVAIVkehAVb5qTDLZjYFRcJWDGWpB5i6uKqtgAD6Mw4tQ6jr7HmcUcfMHGXq8zMR5hPre/e05Gg9mG5V0Z1qP1Aj65OpnPs8uzuxOmXR4mMO0dE8Rjtq0DCbPDv0hlXwO/8Qf+e/8nGph4bfv4Z8sRRqsor7TeSIHkCSDxIBKN3dzh5IGpXtYhwwp7W4rdQgd/C4NseoqklaWe2mw7MTzi9sMdiUhr6kfBrp6ajogFMHsVrw6hpq24epGYr+t3gHr34Wr78bQ/11ZEdpTsQniwiy5fIfyWA6NDkmw8TrZxi6qAJgGepeBdEvkoOXm55F6vMzEF+V2IuOhugsld688O0o5u6W4cCd3Sv6NLDJG1FYxzV+DW2oM+UTXXzssrP9B7SLaJ7lIZ0TTPUWXbxhanlHld+kPHeVTwt4TXRI8tNfevvqcidxl9TxtR8kgXbjaR9LDAJ2JbzIKVwmJGivWKvetqK2x47igiwYodzaVzLZ8aQVjVx6JtoVHZ3L1vxI/1aiobMObrYEUp2JPgdIq1gSwDz4/AcDCLBqjdQs3fzxDjZd8F6Ua2+6EcM3P427GF+Jld6eyzbCL5BTyaQ5Ro1nb5xqsORnChMr3MbQ8feV37Q995LO9afeW8tu3ugPctGN8nPTygEpPZ76l665MGJ3Tz0EoDzQHG0QcqWQmxwTrP/t2rQaxe6JPb79nYoDHmR4f1OcX+SqWlEN+MtHGIAHw8eSPUooctNz80Qw5XtQuignPcpGTvJ2jSpRDmPJUuMAKz5qzVTzo4pIlH0b9FXI72XjdjOHjGjOL2oFSbvlDy5NZH3Z/4MqnjHyEk44YDHuTef1f7oZWDXSSwy9tXO+eX+jK46b/FrwTX5jHIVJnYMUpWO9fjnWhe7a53eRp6N5xPh/MpOpvHjVBPV3oaAftGZmvfg7qBZg+boyRXqGWuqI3DgiAj8CE9tCkGrebzs0fw1jjpXgecp5oktfeDN8BEA80n1n7xB5SsSeY1bzmetPyl5EUO3WNh2+DJJSPp/6S/HyBNJByw0cbdXrepnuIOWXfUH4jQ8szWJ/4/tBHPu3GZ9f23QmvOTDzHXQ6TpLGgCqPqF015EZLO/kjCpxHIGhXeDXOM8DsXK0KuNsJk2GS9acBqOJzTTuo7fdIRX7S2AWS1Dn1LSRffwbgQ7CnaWP7Ijd/FB85XqxBzX+T03inwoeRj1t/HdJG5Xu82jP748xT9lz5DC3PuPL9/aGrfKI8nx3a14YCeI18a7B+/TLAeBmXHgaoDKT1BcJad7L/EQg1tvZhq2XDxm7/Bh/p5OI8Osj3SKftzYiPMerCWPXPDd36UwS8mPhc6e5/lO7PIWJUZWC4WgFPJLA6f25qs8xtjJGMPypt/TT0ZwDG5/CBj14q02u8RIxHH268XWzMem5QkAysLaIdKxVPmkp2i7bYqet6mjuVPvHV36X+QH7b5kSEQdNWfpWh5cmpT0x/iJeP3rJXvx00PjXc9nXtn1J+qt2NiWNNHiLzEVAjv3dLm3cyTN6pCG+ar/bl/JvMm8+mWn5K+9Xcm1en+Tt10so1oTpV8u9X79TAnnxF8tVJpk0mv/f9vpM6lT9G/ffvNMgZ+cvtkG3q3/dNWp7hOrBThHxIcXiurcjHm0enUP8cQk5ISKxkxkD0mOLmZ6Zu4yV2PDbrTn/ZDF3bmGzdPPk76B+efJj1b61P9RqufAaWZ8L6ePsDUz5h28Sk+ep97/wGt/8HfyMwLx1T6uEBtY9DqqJXL/ErQR0T4cuvVkre32hCx3zYmBCXbDPyY4w6MUb9dVyL7zeasKsx2VZDLLryE4tU/sc/B9SGHWhCq8zSIbwc3f9bFsQADI59lFKsJ56bn0u38RI/HquQIa28aJyxqK/5mLkihT5prL86lGM9jBWkBy/ynrnyGVqenPJ13gZZJpJPCG77+vPL6ry1elyPhf+kFWr+Hha5jSL91RHuajAxTNvJAdx1iwkAMCIyrEVu1cXqW25+AADoSdJDSOAEsQuHSXgnAQAxTOvB8wAAUAceUFAnNzotcptguG0UAAAAAHwu4AEFLcgYXRifAAAAAEjHeB5QAAAAAAAACHhAAQAAAADAqMAABQAAAAAAowIDFAAAAAAAjAoMUAAAAAAAMCowQAEAAAAAwKjAAAUAAAAAAKMCAxQAAAAAAIwKDFAAAAAAADAqMEABAAAAAMCowAAFAAAAAACjAgMUAAAAAACMCgxQAAAAAAAwIkL8H90RTQ81O5ywAAAAAElFTkSuQmCC"
    }
   },
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![image.png](attachment:image.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "평가 지표들을 살펴보면, 오히려 대부분의 성능이 떨어진 것 같다.\n",
    "\n",
    "* accuracy and specificity <br>\n",
    "정확도와 특이도는 살짝 떨어졌지만, 그럼에도 높은 수치를 보이고 있다.\n",
    "\n",
    "* precision and f1 score <br>\n",
    "정밀도와 f1 score는 정말 큰 폭으로 성능이 하락된 것을 확인할 수 있다.\n",
    "\n",
    "* recall <br>\n",
    "재현율은 전과 대비 나름 큰 폭으로 성능이 향상된 것을 확인할 수 있다."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "대부분의 데이터 형태로 봤을 때는 `정확도`, `정밀도`, `재현율` 등 모두 다 중요한 지표이다. 그러나 분석을 진행한 본 데이터 셋은 `카드 사기` 이기 때문에 정상 거래를 사기라고 예측한 것보다 사기 거래를 사기라고 예측하지 못한( 정상 거래라고 생각) 것이 더 큰 손실이 있을 것이다."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`따라서 본 카드 사기 데이터는 정밀도 보다 재현율이 조금 더 중요한 지표라고 볼 수 있기 때문에 재현율 측면에서 성능 향상은 큰 폭으로 개선되었기 때문에 성능 향상이 잘 이루어졌다고 볼 수 있다.`"
   ]
  }
 ],
 "metadata": {
  "colab": {
   "collapsed_sections": [],
   "name": "Regression_과제3",
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}