mimic-iv-demo.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "055e570d",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "# Introduction to the MIMIC-IV Database and Machine Learning with scikit-learn\n",
    "\n",
    "**Author: Meghan R. Hutch**\n",
    "\n",
    "**Date: January 30th, 2025**"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "816c9875-a894-4b15-a77e-46e726715880",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "### **About Me**\n",
    "\n",
    "* Current Preceptor of Data Science at the University of Chicago\n",
    "* Past PhD Candidate the Luo Lab \n",
    "* Graduated summer 2024 from DGP, Biomedical Informatics Concentration\n",
    "* Dissertation focused on using EHR data to develop predictive models for patients with neurological illness"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "70f28c72-1ad4-407d-980a-4036d1d68d20",
   "metadata": {
    "jp-MarkdownHeadingCollapsed": true,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "### **Today's lecture will be divided into two parts:**\n",
    "\n",
    "* Introduction to working with the MIMIC-IV dataset\n",
    "\n",
    "* Introduction to machine learning with scikit learn\n",
    "\n",
    "\n",
    "We will discuss both of these in the context of a framework for conducting clinical research\n",
    "\n",
    "**Inspired and adapted from Dr. Garrett Eickelberg's [workshop on MIMIC-III and scikit learn](https://github.com/geickelb/MIMIC-III_to_Model/tree/master)*"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "084c22f1",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "# Framework for Conducting Clinical Research \n",
    "\n",
    "---\n",
    "\n",
    "1. Research Question Specification (hypothesis)\n",
    "<br></br>\n",
    "2. Cohort Specification\n",
    "<br></br>\n",
    "3. Data Extraction\n",
    "<br></br>\n",
    "4. Data Pre-processing (cleaning!)\n",
    "<br></br>\n",
    "5. Model Training\n",
    "<br></br>\n",
    "5. Model Evaluation\n",
    "<br></br>\n",
    "\n",
    "**Today, we will largely focus on steps 4-6**"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1529e06d",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "### Introducing the MIMIC-IV Database"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a7f7ce6b-f64e-4cda-bd4c-424edb9b4e80",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "source": [
    "* The Medical Information Mart for Intensive Care (MIMIC)-IV is a large dataset curated to help support studies on intensive care unit (ICU) patients. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "624083fd-dfde-44e9-a0b5-331a4825a459",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "source": [
    "* The MIMIC-IV dataset (since April 2024) contains structured EHR data from >360,000 patients who were admitted to Beth Israel Deaconess Medical Center in Boston, MA between 2008-2022."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0ad515fd-7d4e-4e26-b6d9-9238647ee573",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "source": [
    "* Today, we will use a previously curated [MIMIC-IV demo-dataset](https://physionet.org/content/mimic-iv-demo/2.2/) to introduce how EHR data is computationally structured and how it can be used for research purposes."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6d761e39-ce03-4964-9b26-0aa020e4be30",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "source": [
    "* Notably, this demo-dataset contains the data from a random subset of 100 hospitalized patients. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "94552f0a-61f0-4337-9f2d-9a702e48d931",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "source": [
    "* This dataset was curated for the purposes of workshops and tutorials. Thus, having appropriate CITI training is not needed. Although deidentified, access to the full dataset requires a physionet account and [CITI training](https://physionet.org/about/citi-course/)."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4ef1578d",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "# Data Extraction\n",
    "\n",
    "- The full demo dataset has been downloaded to the classroom's quest allocation.\n",
    "- The data used for this lecture are available in the relevant [GitHub Repo](https://github.com/meghutch/mimic-ml-demo)\n",
    "- We will further load this data into our notebook\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1e9e4be5",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "### Load in MIMIC-IV Demo\n",
    "\n",
    "First, we will import some basic packages"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "23991120",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "# import our packages\n",
    "import os\n",
    "\n",
    "import pandas as pd \n",
    "import numpy as np\n",
    "import seaborn as sns\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "abc9914d",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "# set our working directory - this notebook assume you are in the classroom Quest folder ('/projects/e30766/mimic-iv-demo/2.2') where the MIMIC-IV demo dataset is located\n",
    "#os.chdir('/projects/e30766/mimic-iv-demo/2.2')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "a758127d-ca7c-4637-9ca2-661ddda4eaee",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# set relative working directory assuming you downloaded this lecture from GitHub:\n",
    "os.chdir('data/mimic-iv-clinical-database-demo-2.2')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "edb5b820",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'C:\\\\Users\\\\meghan_hutch\\\\Documents\\\\Projects\\\\mimic-ml-demo\\\\data\\\\mimic-iv-clinical-database-demo-2.2'"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# make sure you are in the working directory: \n",
    "os.getcwd()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "49af8727-79e7-40a9-9031-1adf291cf848",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "Next, we will load the list of subject identifiers (`subject_id`) for the demo cohort"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "5064c9af",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "# load data\n",
    "subject_id = pd.read_csv('demo_subject_id.csv')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "254b15fe",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>subject_id</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>10000032</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>10001217</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>10001725</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>10002428</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>10002495</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   subject_id\n",
       "0    10000032\n",
       "1    10001217\n",
       "2    10001725\n",
       "3    10002428\n",
       "4    10002495"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# view the first 5 rows of data\n",
    "subject_id.head(5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "6c0131df",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "100"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# check how many unique subjects\n",
    "subject_id['subject_id'].nunique()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d0941b1f",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "**Overview of MIMIC-IV Tables**\n",
    "\n",
    "For this demo, we will explore data found in the following two MIMIC-IV modules:\n",
    "\n",
    "\n",
    "### [Hosp](https://mimic.mit.edu/docs/iv/modules/hosp/)\n",
    "\n",
    "> \"The Hosp module provides all data acquired from the hospital wide electronic health record. Information covered includes patient and admission information, laboratory measurements, microbiology, medication administration, and billed diagnoses.\"\n",
    "\n",
    "### [ICU](https://mimic.mit.edu/docs/iv/modules/icu/)\n",
    "\n",
    "> \"The ICU module contains information collected from the clinical information system used within the ICU. Documented data includes intravenous administrations, ventilator settings, and other charted items.\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dacfe198-ae72-4b56-88f2-7e644e628882",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "**Notes:**\n",
    "\n",
    "* The [GitHub repo](https://github.com/meghutch/mimic-ml-demo/tree/main) for this lecture only contains the data used for this workshop (due to GitHub storage limits). However, all data tables within the `hosp` and `icu` modules have been downloaded to Quest: `/projects/e30766/mimic-iv-demo/2.2`\n",
    "\n",
    "* The full MIMIC-IV dataset also contains additional modules for data from emergency department (ED), chest X-rays (CXR), and clinical notes (Note)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "902f72ef-429e-4bf2-bb0b-e479c9668462",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "Next, let's look at the `patient` table found within the `hosp` module.\n",
    "\n",
    "The `patient` table contains each patient's unique `subject_id` and some simple demographic data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "7e68d33d",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "patients = pd.read_csv('hosp/patients.csv.gz')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "6b015ec8-eb6b-461e-824d-fba012cad7d6",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>subject_id</th>\n",
       "      <th>gender</th>\n",
       "      <th>anchor_age</th>\n",
       "      <th>anchor_year</th>\n",
       "      <th>anchor_year_group</th>\n",
       "      <th>dod</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>10014729</td>\n",
       "      <td>F</td>\n",
       "      <td>21</td>\n",
       "      <td>2125</td>\n",
       "      <td>2011 - 2013</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>10003400</td>\n",
       "      <td>F</td>\n",
       "      <td>72</td>\n",
       "      <td>2134</td>\n",
       "      <td>2011 - 2013</td>\n",
       "      <td>2137-09-02</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>10002428</td>\n",
       "      <td>F</td>\n",
       "      <td>80</td>\n",
       "      <td>2155</td>\n",
       "      <td>2011 - 2013</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>10032725</td>\n",
       "      <td>F</td>\n",
       "      <td>38</td>\n",
       "      <td>2143</td>\n",
       "      <td>2011 - 2013</td>\n",
       "      <td>2143-03-30</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>10027445</td>\n",
       "      <td>F</td>\n",
       "      <td>48</td>\n",
       "      <td>2142</td>\n",
       "      <td>2011 - 2013</td>\n",
       "      <td>2146-02-09</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   subject_id gender  anchor_age  anchor_year anchor_year_group         dod\n",
       "0    10014729      F          21         2125       2011 - 2013         NaN\n",
       "1    10003400      F          72         2134       2011 - 2013  2137-09-02\n",
       "2    10002428      F          80         2155       2011 - 2013         NaN\n",
       "3    10032725      F          38         2143       2011 - 2013  2143-03-30\n",
       "4    10027445      F          48         2142       2011 - 2013  2146-02-09"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# review the first 5 rows\n",
    "patients.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7fc4f80e-6c13-4856-b862-0e9f3c232998",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "What do these columns mean? \n",
    "\n",
    "For this, we can review the data dictionary: https://mimic.mit.edu/docs/iv/modules/hosp/patients/"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "af7401f6-e814-44a0-8192-eb3894ad42f6",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>subject_id</th>\n",
       "      <th>gender</th>\n",
       "      <th>anchor_age</th>\n",
       "      <th>anchor_year</th>\n",
       "      <th>anchor_year_group</th>\n",
       "      <th>dod</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>10014729</td>\n",
       "      <td>F</td>\n",
       "      <td>21</td>\n",
       "      <td>2125</td>\n",
       "      <td>2011 - 2013</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>10003400</td>\n",
       "      <td>F</td>\n",
       "      <td>72</td>\n",
       "      <td>2134</td>\n",
       "      <td>2011 - 2013</td>\n",
       "      <td>2137-09-02</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>10002428</td>\n",
       "      <td>F</td>\n",
       "      <td>80</td>\n",
       "      <td>2155</td>\n",
       "      <td>2011 - 2013</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>10032725</td>\n",
       "      <td>F</td>\n",
       "      <td>38</td>\n",
       "      <td>2143</td>\n",
       "      <td>2011 - 2013</td>\n",
       "      <td>2143-03-30</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>10027445</td>\n",
       "      <td>F</td>\n",
       "      <td>48</td>\n",
       "      <td>2142</td>\n",
       "      <td>2011 - 2013</td>\n",
       "      <td>2146-02-09</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   subject_id gender  anchor_age  anchor_year anchor_year_group         dod\n",
       "0    10014729      F          21         2125       2011 - 2013         NaN\n",
       "1    10003400      F          72         2134       2011 - 2013  2137-09-02\n",
       "2    10002428      F          80         2155       2011 - 2013         NaN\n",
       "3    10032725      F          38         2143       2011 - 2013  2143-03-30\n",
       "4    10027445      F          48         2142       2011 - 2013  2146-02-09"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# review the first 5 rows\n",
    "patients.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "22091455",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "Important note about MIMIC's pre-processing of [dates](https://mimic.mit.edu/docs/iv/modules/hosp/patients/#anchor_age-anchor_year-anchor_year_group):\n",
    "\n",
    "* Dates are randomly shifted consistently for all patients\n",
    "\n",
    "* `anchor_year` - shifted year for the patient\n",
    "\n",
    "* `anchor_year_group` -  is a range of years - the patient’s `anchor_year` actually occurred during this range\n",
    "\n",
    "* `anchor_age` - the patient’s age in the `anchor_year` (year of admission). \n",
    "\n",
    "**Examples**: \n",
    "\n",
    "A patient has an `anchor_year` of 2153, `anchor_year_group` of 2008 - 2010, and an `anchor_age` of 60.\n",
    "\n",
    "* The year 2153 for the patient corresponds to 2008, 2009, or 2010.\n",
    "\n",
    "* The patient was 60 in the shifted year of 2153, i.e. they were 60 in 2008, 2009, or 2010.\n",
    "\n",
    "* A patient admission in 2154 will occur in 2009-2011, an admission in 2155 will occur in 2010-2012, and so on.\n",
    "\n",
    "* Note: Due to HIPAA patient confidentiality concerns, if a patient’s `anchor_age` is over 89 in the `anchor_year` then their `anchor_age` is set to 91, regardless of how old they actually were."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "9140b75f-61c8-4c4f-8180-99a79b21b5c7",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>subject_id</th>\n",
       "      <th>gender</th>\n",
       "      <th>anchor_age</th>\n",
       "      <th>anchor_year</th>\n",
       "      <th>anchor_year_group</th>\n",
       "      <th>dod</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>10014729</td>\n",
       "      <td>F</td>\n",
       "      <td>21</td>\n",
       "      <td>2125</td>\n",
       "      <td>2011 - 2013</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>10003400</td>\n",
       "      <td>F</td>\n",
       "      <td>72</td>\n",
       "      <td>2134</td>\n",
       "      <td>2011 - 2013</td>\n",
       "      <td>2137-09-02</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>10002428</td>\n",
       "      <td>F</td>\n",
       "      <td>80</td>\n",
       "      <td>2155</td>\n",
       "      <td>2011 - 2013</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>10032725</td>\n",
       "      <td>F</td>\n",
       "      <td>38</td>\n",
       "      <td>2143</td>\n",
       "      <td>2011 - 2013</td>\n",
       "      <td>2143-03-30</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>10027445</td>\n",
       "      <td>F</td>\n",
       "      <td>48</td>\n",
       "      <td>2142</td>\n",
       "      <td>2011 - 2013</td>\n",
       "      <td>2146-02-09</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   subject_id gender  anchor_age  anchor_year anchor_year_group         dod\n",
       "0    10014729      F          21         2125       2011 - 2013         NaN\n",
       "1    10003400      F          72         2134       2011 - 2013  2137-09-02\n",
       "2    10002428      F          80         2155       2011 - 2013         NaN\n",
       "3    10032725      F          38         2143       2011 - 2013  2143-03-30\n",
       "4    10027445      F          48         2142       2011 - 2013  2146-02-09"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# review the first 5 rows\n",
    "patients.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "92cb7891",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "100"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# review the number of unique values in each dataset\n",
    "patients['subject_id'].nunique()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "bc78ad6f",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "100"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(patients)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "eeaf294e",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "**Let's load in several additional tables from MIMIC:**\n",
    "\n",
    "Here, we will load a few of the tables we will use to demonstate the workflow of aquiring, pre-processing, and analyzing data from the MIMIC database."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "1f4a6a73",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "# load several tables from the hosp module\n",
    "admissions = pd.read_csv('hosp/admissions.csv.gz') # info about hospital stay\n",
    "omr = pd.read_csv('hosp/omr.csv.gz') # misc info from EHR\n",
    "labevents = pd.read_csv('hosp/labevents.csv.gz') # laboratory measurements\n",
    "d_labitems = pd.read_csv('hosp/d_labitems.csv.gz') # dictionary of laboratory values\n",
    "diagnoses = pd.read_csv('hosp/diagnoses_icd.csv.gz') # icd codes assigned to patient for billing\n",
    "d_icd_diagnoses = pd.read_csv('hosp/d_icd_diagnoses.csv.gz') # dictionary of icd variable\n",
    "\n",
    "# load several tables from the icu module\n",
    "chartevents = pd.read_csv('icu/chartevents.csv.gz') # info charted during patient's hospital stay\n",
    "d_items = pd.read_csv('icu/d_items.csv.gz') # dictionary of ICU event variables  "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "10c13328",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "### omr table\n",
    "\n",
    "> \"The Online Medical Record (OMR) table contains miscellaneous information from the EHR.\"\n",
    "\n",
    "[omr documentation](https://mimic.mit.edu/docs/iv/modules/hosp/omr) of columns:\n",
    "\n",
    "* `chartdate` - the date on which the observation was recorded\n",
    "* `seq_num`- a monotonically increasing integer which uniquely distinguishes results of the same type recorded on the same day. For example, if two blood pressure measurements occur on the same day, seq_num orders them chronologically.\n",
    "* `result_name` - human interpretable description of the observation\n",
    "* `result_value` - the value associated with the observation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "8f9b6c6e",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>subject_id</th>\n",
       "      <th>chartdate</th>\n",
       "      <th>seq_num</th>\n",
       "      <th>result_name</th>\n",
       "      <th>result_value</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>10011398</td>\n",
       "      <td>2146-12-01</td>\n",
       "      <td>1</td>\n",
       "      <td>Height (Inches)</td>\n",
       "      <td>63</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>10011398</td>\n",
       "      <td>2147-01-22</td>\n",
       "      <td>1</td>\n",
       "      <td>Weight (Lbs)</td>\n",
       "      <td>127</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>10011398</td>\n",
       "      <td>2146-12-01</td>\n",
       "      <td>1</td>\n",
       "      <td>Weight (Lbs)</td>\n",
       "      <td>135</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>10011398</td>\n",
       "      <td>2147-07-24</td>\n",
       "      <td>1</td>\n",
       "      <td>Weight (Lbs)</td>\n",
       "      <td>136</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>10011398</td>\n",
       "      <td>2147-03-26</td>\n",
       "      <td>1</td>\n",
       "      <td>Weight (Lbs)</td>\n",
       "      <td>136</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   subject_id   chartdate  seq_num      result_name result_value\n",
       "0    10011398  2146-12-01        1  Height (Inches)           63\n",
       "1    10011398  2147-01-22        1     Weight (Lbs)          127\n",
       "2    10011398  2146-12-01        1     Weight (Lbs)          135\n",
       "3    10011398  2147-07-24        1     Weight (Lbs)          136\n",
       "4    10011398  2147-03-26        1     Weight (Lbs)          136"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# review the first 5 rows\n",
    "omr.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "e6e5dc8f",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>result_name</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Height (Inches)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Weight (Lbs)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>BMI (kg/m2)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>Blood Pressure</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2206</th>\n",
       "      <td>Blood Pressure Standing (1 min)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2208</th>\n",
       "      <td>Blood Pressure Sitting</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2222</th>\n",
       "      <td>Blood Pressure Lying</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2590</th>\n",
       "      <td>Blood Pressure Standing (3 mins)</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                           result_name\n",
       "0                      Height (Inches)\n",
       "1                         Weight (Lbs)\n",
       "8                          BMI (kg/m2)\n",
       "14                      Blood Pressure\n",
       "2206   Blood Pressure Standing (1 min)\n",
       "2208            Blood Pressure Sitting\n",
       "2222              Blood Pressure Lying\n",
       "2590  Blood Pressure Standing (3 mins)"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# review unique result_names (observations)\n",
    "# here we can see the unique variables/measurements contained in this table\n",
    "omr[['result_name']].drop_duplicates()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1b308bff-5eb6-4bd1-80fb-a6345b79bad2",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "# Data Pre-processing (Data Cleaning)\n",
    "\n",
    "### What does it mean to clean data?\n",
    "\n",
    "Simply, data cleaning is the act of preparing data for analysis. In a clinical context, we also want to make sure that the data we are using is clinically relevant and accurate, especially in regard to the specific clinical question or problem we are trying to solve.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c73f4330",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "### How do we \"clean\" data:\n",
    "\n",
    "* Ensure data are formatted into the right data types (numeric, character, date, etc)\n",
    "\n",
    "* Review summary statistics (how many observations, how many patients have each observation, mean/median/sd/variance of values, proportion of missing data)\n",
    "\n",
    "* Review the distribution of data\n",
    "\n",
    "* Are there possible data entry problems or unit conversions needed?\n",
    "\n",
    "* How should we handle missing data?\n",
    "\n",
    "* Should patients have repeated measurements (e.g. multiple white blood cell counts on the same day)? If so, how should we handle repeated observations?"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "afd17f08-494b-4db5-9f92-ae857d7ae393",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "source": [
    "> “Data science, he says, involves multiple “very small decisions” — data cleaning and filtering steps, for instance, which are crucially important, but difficult to document. And journal page limits preclude exposition. But by blending code, data and text in a single document, researchers can show just how their results were generated.” - Ben Marwick; [Perkel, JM](https://www.nature.com/articles/d41586-022-00563-z)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "766a63a1",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "### Example of Data Cleaning (Height)\n",
    "\n",
    "**Let's focus on the variables `Height` and `Weight`**\n",
    "\n",
    "We will filter the `omr` table to only contains results for height and weight. During data cleaning, each variable will be saved in its own dataframe for ease of pre-processing"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "2cfb1122",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "# create separate dataframes for each variable\n",
    "height = omr[omr['result_name']==\"Height (Inches)\"].copy() # copy tells Python to keep our new df 'height' independent from the original df 'omr' it was sourced from\n",
    "\n",
    "weight = omr[omr['result_name']==\"Weight (Lbs)\"].copy()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6db2b8ac",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "**Examine Height (inches)**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "d578015b",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>subject_id</th>\n",
       "      <th>chartdate</th>\n",
       "      <th>seq_num</th>\n",
       "      <th>result_name</th>\n",
       "      <th>result_value</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>10011398</td>\n",
       "      <td>2146-12-01</td>\n",
       "      <td>1</td>\n",
       "      <td>Height (Inches)</td>\n",
       "      <td>63</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>10011398</td>\n",
       "      <td>2147-01-22</td>\n",
       "      <td>1</td>\n",
       "      <td>Height (Inches)</td>\n",
       "      <td>61.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>10014729</td>\n",
       "      <td>2125-02-22</td>\n",
       "      <td>1</td>\n",
       "      <td>Height (Inches)</td>\n",
       "      <td>65</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>10014729</td>\n",
       "      <td>2125-04-05</td>\n",
       "      <td>1</td>\n",
       "      <td>Height (Inches)</td>\n",
       "      <td>66</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>10014729</td>\n",
       "      <td>2126-01-24</td>\n",
       "      <td>1</td>\n",
       "      <td>Height (Inches)</td>\n",
       "      <td>66</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    subject_id   chartdate  seq_num      result_name result_value\n",
       "0     10011398  2146-12-01        1  Height (Inches)           63\n",
       "13    10011398  2147-01-22        1  Height (Inches)         61.5\n",
       "20    10014729  2125-02-22        1  Height (Inches)           65\n",
       "21    10014729  2125-04-05        1  Height (Inches)           66\n",
       "22    10014729  2126-01-24        1  Height (Inches)           66"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "height.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6f573ee9-a122-4b21-94a4-374022a9a296",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "source": [
    "Let's check the number of observations"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "569094e8-a04b-456b-9e4e-5e9357c0c9e2",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of height measurements: 378\n",
      "Number of unique patients with height measurements: 61\n"
     ]
    }
   ],
   "source": [
    "print('Number of height measurements:', len(height))\n",
    "print('Number of unique patients with height measurements:', height['subject_id'].nunique())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b7f7bae6",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "source": [
    "It looks like `result_value` is a continuous variable. Let's make sure that Python also has recognized it as such."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "98a6482b",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "subject_id       int64\n",
       "chartdate       object\n",
       "seq_num          int64\n",
       "result_name     object\n",
       "result_value    object\n",
       "dtype: object"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# check variable data types\n",
    "height.dtypes"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6d9b1b48",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "source": [
    "It looks like Python read `result_value` as a object (or character) variable. Let's convert to numeric. This is an important step to make sure any downstream functions (calculating the mean or median, plotting the distribution, etc) work correctly."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "80ac6731",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "height['result_value'] = pd.to_numeric(height['result_value'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "f73725a7",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "subject_id        int64\n",
       "chartdate        object\n",
       "seq_num           int64\n",
       "result_name      object\n",
       "result_value    float64\n",
       "dtype: object"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# check variable data types again to make sure our transformation and code worked\n",
    "# float64 indicates a continuous variable, so this looks good!\n",
    "height.dtypes"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "20d9236e",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "### Using Summary Statistics and Visualization to help Evaluate Data\n",
    "\n",
    "Next, let's calculate summary statistics to get a better sense of our variable"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "f8c506f3",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "count    378.000000\n",
       "mean      67.105952\n",
       "std        5.117295\n",
       "min        5.000000\n",
       "25%       63.500000\n",
       "50%       67.000000\n",
       "75%       71.000000\n",
       "max       73.000000\n",
       "Name: result_value, dtype: float64"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "height['result_value'].describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b9a8974f",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "**Visualize distribution of values**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "8dee820d",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<seaborn.axisgrid.FacetGrid at 0x23b871b2660>"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAeoAAAHpCAYAAABN+X+UAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8hTgPZAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAtt0lEQVR4nO3df1RVdb7/8ddR8AgGKKTncCYUKrJUUtOuo3YTU/Ga2ni5NzPN7Fozlj+StF+MZYx3BWVXopHJrt5GTTOnNZOOM1MhlmLFVIKRPyrtB/7I4YSjBEdRMNjfP1qdbyfAH3jgfI4+H2vttdifz2dv3u9h4uU+Zx+2zbIsSwAAwEhtAl0AAABoGkENAIDBCGoAAAxGUAMAYDCCGgAAgxHUAAAYjKAGAMBgBLUky7JUVVUlPlIOADANQS3J4/EoKipKHo8n0KUAAOCDoAYAwGAENQAABiOoAQAwGEENAIDBCGoAAAxGUAMAYDCCGgAAgxHUAAAYjKAGAMBgBDUAAAYjqAEAMBhBDQCAwQhqAAAMRlADAGAwghoAAIMR1AAAGIygBgDAYAQ1AAAGCwl0AQAABJLH41F1dfVZrQ0PD1dEREQLV+QroFfUW7du1dixY+VyuWSz2bR+/fom106bNk02m005OTk+4zU1NZo1a5YuvfRSdejQQbfccou+/vrrli0cAHBB8Hg86hafIKfTeVZbt/gEeTyeVq0xoFfUx48fV+/evfVf//Vf+o//+I8m161fv14ffPCBXC5Xg7m0tDT95S9/0dq1axUTE6O5c+dqzJgxKi4uVtu2bVuyfABAkKuurlbF0SMaOX+V2kd0Ou3ak54K5S2YrOrq6la9qg5oUI8aNUqjRo067ZpDhw5p5syZysvL0+jRo33mKisr9eKLL2rVqlUaPny4JGn16tWKi4vTpk2bNHLkyEbPWVNTo5qaGu9+VVXVeXYCAAhm7SM6qX1kdKDLaJTRN5PV19dr8uTJeuihh9SzZ88G88XFxTp16pRSUlK8Yy6XS7169VJhYWGT583KylJUVJR3i4uLa5H6AQA4X0YH9dNPP62QkBDdf//9jc673W61a9dOnTr5vlzhcDjkdrubPG96eroqKyu928GDB/1aNwAA/mLsXd/FxcV67rnntH37dtlstnM61rKs0x5jt9tlt9vPt0QAAFqcsVfU77zzjsrLy9W1a1eFhIQoJCRE+/fv19y5cxUfHy9Jcjqdqq2tVUVFhc+x5eXlcjgcAagaAAD/MjaoJ0+erB07dqikpMS7uVwuPfTQQ8rLy5Mk9evXT6GhocrPz/ceV1ZWpl27dmnQoEGBKh0AAL8J6Evfx44d0xdffOHdLy0tVUlJiaKjo9W1a1fFxMT4rA8NDZXT6VT37t0lSVFRUbr77rs1d+5cxcTEKDo6Wg8++KCSkpK8d4EDABDMAhrURUVFGjp0qHd/zpw5kqQpU6ZoxYoVZ3WOZ599ViEhIRo/frxOnDihYcOGacWKFXyGGgBwQQhoUCcnJ8uyrLNev2/fvgZj7du31+LFi7V48WI/VgYAgBmMfY8aAAAQ1AAAGI2gBgDAYAQ1AAAGI6gBADAYQQ0AgMEIagAADEZQAwBgMIIaAACDEdQAABiMoAYAwGAENQAABiOoAQAwGEENAIDBCGoAAAxGUAMAYDCCGgAAgxHUAAAYjKAGAMBgBDUAAAYjqAEAMBhBDQCAwQhqAAAMRlADAGAwghoAAIMR1AAAGIygBgDAYAQ1AAAGI6gBADAYQQ0AgMEIagAADEZQAwBgMIIaAACDEdQAABiMoAYAwGAENQAABiOoAQAwGEENAIDBCGoAAAxGUAMAYDCCGgAAgxHUAAAYjKAGAMBgBDUAAAYjqAEAMBhBDQCAwQIa1Fu3btXYsWPlcrlks9m0fv1679ypU6f0yCOPKCkpSR06dJDL5dKdd96pf/zjHz7nqKmp0axZs3TppZeqQ4cOuuWWW/T111+3cicAALSMgAb18ePH1bt3b+Xm5jaYq66u1vbt2/X4449r+/bteu2117R3717dcsstPuvS0tK0bt06rV27Vu+++66OHTumMWPGqK6urrXaAACgxYQE8puPGjVKo0aNanQuKipK+fn5PmOLFy/Wv/zLv+jAgQPq2rWrKisr9eKLL2rVqlUaPny4JGn16tWKi4vTpk2bNHLkyBbvAQCAlhRU71FXVlbKZrOpY8eOkqTi4mKdOnVKKSkp3jUul0u9evVSYWFhk+epqalRVVWVzwYAgImCJqhPnjypRx99VBMnTlRkZKQkye12q127durUqZPPWofDIbfb3eS5srKyFBUV5d3i4uJatHYAAJorKIL61KlTmjBhgurr6/X888+fcb1lWbLZbE3Op6enq7Ky0rsdPHjQn+UCAOA3xgf1qVOnNH78eJWWlio/P997NS1JTqdTtbW1qqio8DmmvLxcDoejyXPa7XZFRkb6bAAAmMjooP4hpD///HNt2rRJMTExPvP9+vVTaGioz01nZWVl2rVrlwYNGtTa5QIA4HcBvev72LFj+uKLL7z7paWlKikpUXR0tFwul/7zP/9T27dv11//+lfV1dV533eOjo5Wu3btFBUVpbvvvltz585VTEyMoqOj9eCDDyopKcl7FzgAAMEsoEFdVFSkoUOHevfnzJkjSZoyZYoyMjK0YcMGSVKfPn18jtu8ebOSk5MlSc8++6xCQkI0fvx4nThxQsOGDdOKFSvUtm3bVukBAICWFNCgTk5OlmVZTc6fbu4H7du31+LFi7V48WJ/lgYAgBGMfo8aAICLHUENAIDBCGoAAAxGUAMAYDCCGgAAgxHUAAAYjKAGAMBgBDUAAAYjqAEAMBhBDQCAwQhqAAAMRlADAGAwghoAAIMR1AAAGIygBgDAYAQ1AAAGI6gBADAYQQ0AgMEIagAADEZQAwBgMIIaAACDEdQAABiMoAYAwGAENQAABiOoAQAwGEENAIDBCGoAAAxGUAMAYDCCGgAAgxHUAAAYjKAGAMBgBDUAAAYjqAEAMBhBDQCAwQhqAAAMRlADAGAwghoAAIMR1AAAGIygBgDAYAQ1AAAGI6gBADAYQQ0AgMEIagAADEZQAwBgMIIaAACDEdQAABiMoAYAwGABDeqtW7dq7NixcrlcstlsWr9+vc+8ZVnKyMiQy+VSWFiYkpOTtXv3bp81NTU1mjVrli699FJ16NBBt9xyi77++utW7AIAgJYT0KA+fvy4evfurdzc3EbnFy5cqOzsbOXm5mrbtm1yOp0aMWKEPB6Pd01aWprWrVuntWvX6t1339WxY8c0ZswY1dXVtVYbAAC0mJBAfvNRo0Zp1KhRjc5ZlqWcnBzNmzdPqampkqSVK1fK4XBozZo1mjZtmiorK/Xiiy9q1apVGj58uCRp9erViouL06ZNmzRy5MhGz11TU6OamhrvflVVlZ87AwDAP4x9j7q0tFRut1spKSneMbvdriFDhqiwsFCSVFxcrFOnTvmscblc6tWrl3dNY7KyshQVFeXd4uLiWq4RAADOg7FB7Xa7JUkOh8Nn3OFweOfcbrfatWunTp06NbmmMenp6aqsrPRuBw8e9HP1AAD4R0Bf+j4bNpvNZ9+yrAZjP3WmNXa7XXa73S/1AQDQkoy9onY6nZLU4Mq4vLzce5XtdDpVW1urioqKJtcAABDMjA3qhIQEOZ1O5efne8dqa2tVUFCgQYMGSZL69eun0NBQnzVlZWXatWuXdw0AAMEsoC99Hzt2TF988YV3v7S0VCUlJYqOjlbXrl2VlpamzMxMJSYmKjExUZmZmQoPD9fEiRMlSVFRUbr77rs1d+5cxcTEKDo6Wg8++KCSkpK8d4EDABDMAhrURUVFGjp0qHd/zpw5kqQpU6ZoxYoVevjhh3XixAlNnz5dFRUVGjBggDZu3KiIiAjvMc8++6xCQkI0fvx4nThxQsOGDdOKFSvUtm3bVu8HAAB/s1mWZQW6iECrqqpSVFSUKisrFRkZGehyAACt5JtvvpHT6dQvnvmr2kdGn3btyaqj+vNDY+R2u1v1Pijj7/oGAAQHj8ej6urqs14fHh7u8wopGkdQAwDOm8fjUbf4BFUcPXLWx3SKjtH+faWE9RkQ1ACA81ZdXa2Ko0c0cv4qtY/odMb1Jz0VylswWdXV1QT1GRDUAAC/aR/R6Yzv9eLcGPs5agAAQFADAGA0ghoAAIMR1AAAGIygBgDAYAQ1AAAGI6gBADAYQQ0AgMEIagAADEZQAwBgMIIaAACDEdQAABiMoAYAwGAENQAABiOoAQAwGEENAIDBCGoAAAxGUAMAYDCCGgAAgxHUAAAYjKAGAMBgBDUAAAYjqAEAMBhBDQCAwQhqAAAMRlADAGAwghoAAIMR1AAAGIygBgDAYAQ1AAAGI6gBADAYQQ0AgMEIagAADEZQAwBgMIIaAACDEdQAABiMoAYAwGAENQAABiOoAQAwGEENAIDBCGoAAAxGUAMAYDCCGgAAgxkd1N99950ee+wxJSQkKCwsTJdffrkWLFig+vp67xrLspSRkSGXy6WwsDAlJydr9+7dAawaAAD/MTqon376ab3wwgvKzc3Vp59+qoULF+qZZ57R4sWLvWsWLlyo7Oxs5ebmatu2bXI6nRoxYoQ8Hk8AKwcAwD9CAl3A6fz973/XL37xC40ePVqSFB8fr1deeUVFRUWSvr+azsnJ0bx585SamipJWrlypRwOh9asWaNp06Y1et6amhrV1NR496uqqlq4EwAAmqdZV9SXX365jhw50mD822+/1eWXX37eRf3ghhtu0FtvvaW9e/dKkj7++GO9++67uvnmmyVJpaWlcrvdSklJ8R5jt9s1ZMgQFRYWNnnerKwsRUVFebe4uDi/1QwAgD8164p63759qqurazBeU1OjQ4cOnXdRP3jkkUdUWVmpq6++Wm3btlVdXZ2efPJJ3X777ZIkt9stSXI4HD7HORwO7d+/v8nzpqena86cOd79qqoqwhoAYKRzCuoNGzZ4v87Ly1NUVJR3v66uTm+99Zbi4+P9Vtwf/vAHrV69WmvWrFHPnj1VUlKitLQ0uVwuTZkyxbvOZrP5HGdZVoOxH7Pb7bLb7X6rEwCAlnJOQT1u3DhJ3wfjj4NSkkJDQxUfH69Fixb5rbiHHnpIjz76qCZMmCBJSkpK0v79+5WVlaUpU6bI6XRK+v7KOjY21ntceXl5g6tsAACC0Tm9R11fX6/6+np17dpV5eXl3v36+nrV1NRoz549GjNmjN+Kq66uVps2viW2bdvW+/GshIQEOZ1O5efne+dra2tVUFCgQYMG+a0OAAACpVnvUZeWlvq7jkaNHTtWTz75pLp27aqePXvqo48+UnZ2tqZOnSrp+yv7tLQ0ZWZmKjExUYmJicrMzFR4eLgmTpzYKjUCANCSmv3xrLfeektvvfWW98r6x37/+9+fd2GStHjxYj3++OOaPn26ysvL5XK5NG3aNM2fP9+75uGHH9aJEyc0ffp0VVRUaMCAAdq4caMiIiL8UgMAAIHUrKD+zW9+owULFqh///6KjY097Y1b5yMiIkI5OTnKyclpco3NZlNGRoYyMjJapAYAAAKpWUH9wgsvaMWKFZo8ebK/6wEAAD/SrD94Ultby81aAAC0gmYF9T333KM1a9b4uxYAAPATzXrp++TJk1q6dKk2bdqka6+9VqGhoT7z2dnZfikOAICLXbOCeseOHerTp48kadeuXT5zLXVjGQAAF6NmBfXmzZv9XQcAAGiE0c+jBgDgYtesK+qhQ4ee9iXut99+u9kFAQCA/69ZQf3D+9M/OHXqlEpKSrRr164GD+sAAADN16ygfvbZZxsdz8jI0LFjx86rIAAA8P/59T3qO+64w29/5xsAAPg5qP/+97+rffv2/jwlAAAXtWa99J2amuqzb1mWysrKVFRUpMcff9wvhQEAgGYGdVRUlM9+mzZt1L17dy1YsEApKSl+KQwAADQzqJcvX+7vOgAAQCOaFdQ/KC4u1qeffiqbzaYePXqob9++/qoLAAComUFdXl6uCRMmaMuWLerYsaMsy1JlZaWGDh2qtWvXqnPnzv6uEwCAi1Kz7vqeNWuWqqqqtHv3bh09elQVFRXatWuXqqqqdP/99/u7RgAALlrNuqJ+8803tWnTJl1zzTXesR49euh3v/sdN5MBAOBHzbqirq+vb/AMakkKDQ1VfX39eRcFAAC+16ygvummmzR79mz94x//8I4dOnRIDzzwgIYNG+a34gAAuNg1K6hzc3Pl8XgUHx+vK664QldeeaUSEhLk8Xi0ePFif9cIAMBFq1nvUcfFxWn79u3Kz8/XZ599Jsuy1KNHDw0fPtzf9QEAcFE7pyvqt99+Wz169FBVVZUkacSIEZo1a5buv/9+XX/99erZs6feeeedFikUAICL0TkFdU5Ojn75y18qMjKywVxUVJSmTZum7OxsvxUHAMDF7pyC+uOPP9a//du/NTmfkpKi4uLi8y4KAAB875yC+ptvvmn0Y1k/CAkJ0eHDh8+7KAAA8L1zCuqf/exn2rlzZ5PzO3bsUGxs7HkXBQAAvndOQX3zzTdr/vz5OnnyZIO5EydO6IknntCYMWP8VhwAABe7c/p41mOPPabXXntNV111lWbOnKnu3bvLZrPp008/1e9+9zvV1dVp3rx5LVUrAAAXnXMKaofDocLCQt13331KT0+XZVmSJJvNppEjR+r555+Xw+FokUIBALgYnfMfPOnWrZtef/11VVRU6IsvvpBlWUpMTFSnTp1aoj4AAC5qzfrLZJLUqVMnXX/99f6sBQAA/ESz/tY3AABoHQQ1AAAGI6gBADAYQQ0AgMEIagAADEZQAwBgMIIaAACDEdQAABiMoAYAwGAENQAABiOoAQAwGEENAIDBCGoAAAxmfFAfOnRId9xxh2JiYhQeHq4+ffqouLjYO29ZljIyMuRyuRQWFqbk5GTt3r07gBUDAOA/Rgd1RUWFBg8erNDQUL3xxhv65JNPtGjRInXs2NG7ZuHChcrOzlZubq62bdsmp9OpESNGyOPxBK5wAAD8pNnPo24NTz/9tOLi4rR8+XLvWHx8vPdry7KUk5OjefPmKTU1VZK0cuVKORwOrVmzRtOmTWvtkgEA8Cujr6g3bNig/v3769Zbb1WXLl3Ut29fLVu2zDtfWloqt9utlJQU75jdbteQIUNUWFjY5HlrampUVVXlswEAYCKjg/qrr77SkiVLlJiYqLy8PN177726//779dJLL0mS3G63JMnhcPgc53A4vHONycrKUlRUlHeLi4truSYAADgPRgd1fX29rrvuOmVmZqpv376aNm2afvnLX2rJkiU+62w2m8++ZVkNxn4sPT1dlZWV3u3gwYMtUj8AAOfL6KCOjY1Vjx49fMauueYaHThwQJLkdDolqcHVc3l5eYOr7B+z2+2KjIz02QAAMJHRQT148GDt2bPHZ2zv3r3q1q2bJCkhIUFOp1P5+fne+draWhUUFGjQoEGtWisAAC3B6Lu+H3jgAQ0aNEiZmZkaP368PvzwQy1dulRLly6V9P1L3mlpacrMzFRiYqISExOVmZmp8PBwTZw4McDVAwBw/owO6uuvv17r1q1Tenq6FixYoISEBOXk5GjSpEneNQ8//LBOnDih6dOnq6KiQgMGDNDGjRsVERERwMoBAPAPo4NaksaMGaMxY8Y0OW+z2ZSRkaGMjIzWKwoAgFZi9HvUAABc7AhqAAAMRlADAGAwghoAAIMR1AAAGIygBgDAYAQ1AAAGI6gBADAYQQ0AgMEIagAADEZQAwBgMIIaAACDEdQAABiMoAYAwGAENQAABiOoAQAwGEENAIDBCGoAAAxGUAMAYDCCGgAAgxHUAAAYjKAGAMBgBDUAAAYjqAEAMBhBDQCAwQhqAAAMRlADAGAwghoAAIMR1AAAGIygBgDAYAQ1AAAGI6gBADAYQQ0AgMEIagAADEZQAwBgsJBAFwAAuHgdPnz4rNaFh4crIiKihasxE0ENAGh1p05WS7Y2SkpKOqv1naJjtH9f6UUZ1gQ1AKDV1Z2qkax6DUtfrkuiO5927UlPhfIWTFZ1dTVBDQBAa2p3SUe1j4wOdBlG42YyAAAMRlADAGAwghoAAIMR1AAAGIygBgDAYAQ1AAAGI6gBADAYQQ0AgMGCKqizsrJks9mUlpbmHbMsSxkZGXK5XAoLC1NycrJ2794duCIBAPCjoAnqbdu2aenSpbr22mt9xhcuXKjs7Gzl5uZq27ZtcjqdGjFihDweT4AqBQDAf4IiqI8dO6ZJkyZp2bJl6tSpk3fcsizl5ORo3rx5Sk1NVa9evbRy5UpVV1drzZo1TZ6vpqZGVVVVPhsAACYKiqCeMWOGRo8ereHDh/uMl5aWyu12KyUlxTtmt9s1ZMgQFRYWNnm+rKwsRUVFebe4uLgWqx0AgPNhfFCvXbtW27dvV1ZWVoM5t9stSXI4HD7jDofDO9eY9PR0VVZWereDBw/6t2gAAPzE6KdnHTx4ULNnz9bGjRvVvn37JtfZbDaffcuyGoz9mN1ul91u91udAAC0FKOvqIuLi1VeXq5+/fopJCREISEhKigo0G9/+1uFhIR4r6R/evVcXl7e4CobAIBgZHRQDxs2TDt37lRJSYl369+/vyZNmqSSkhJdfvnlcjqdys/P9x5TW1urgoICDRo0KICVAwDgH0a/9B0REaFevXr5jHXo0EExMTHe8bS0NGVmZioxMVGJiYnKzMxUeHi4Jk6cGIiSAQDwK6OD+mw8/PDDOnHihKZPn66KigoNGDBAGzduVERERKBLAwDgvAVdUG/ZssVn32azKSMjQxkZGQGpBwCAlmT0e9QAAFzsCGoAAAxGUAMAYLCge48aANB6PB6Pqqurz7ju8OHDrVDNxYmgBgA0yuPxqFt8giqOHjnrY+rq6lqwoosTQQ0AaFR1dbUqjh7RyPmr1D6i02nXVpbt05bsmaqrq2+l6i4eBDUA4LTaR3RS+8jo06456alopWouPtxMBgCAwQhqAAAMRlADAGAwghoAAIMR1AAAGIygBgDAYAQ1AAAGI6gBADAYQQ0AgMEIagAADMafEAUAXFDO9olfUnA89YugBgBcMJrzxC/J7Kd+EdQAgAvGuTzxSwqOp34R1ACAC87ZPPFLCo6nfnEzGQAABiOoAQAwGEENAIDBCGoAAAxGUAMAYDCCGgAAgxHUAAAYjKAGAMBgBDUAAAYjqAEAMBhBDQCAwQhqAAAMRlADAGAwghoAAIMR1AAAGIygBgDAYAQ1AAAGI6gBADAYQQ0AgMEIagAADEZQAwBgMIIaAACDEdQAABiMoAYAwGAENQAABjM6qLOysnT99dcrIiJCXbp00bhx47Rnzx6fNZZlKSMjQy6XS2FhYUpOTtbu3bsDVDEAAP5ldFAXFBRoxowZev/995Wfn6/vvvtOKSkpOn78uHfNwoULlZ2drdzcXG3btk1Op1MjRoyQx+MJYOUAAPhHSKALOJ0333zTZ3/58uXq0qWLiouLdeONN8qyLOXk5GjevHlKTU2VJK1cuVIOh0Nr1qzRtGnTAlE2AAB+Y/QV9U9VVlZKkqKjoyVJpaWlcrvdSklJ8a6x2+0aMmSICgsLmzxPTU2NqqqqfDYAAEwUNEFtWZbmzJmjG264Qb169ZIkud1uSZLD4fBZ63A4vHONycrKUlRUlHeLi4trucIBADgPQRPUM2fO1I4dO/TKK680mLPZbD77lmU1GPux9PR0VVZWereDBw/6vV4AAPzB6PeofzBr1ixt2LBBW7du1WWXXeYddzqdkr6/so6NjfWOl5eXN7jK/jG73S673d5yBQMA/O7w4cN+WRNsjA5qy7I0a9YsrVu3Tlu2bFFCQoLPfEJCgpxOp/Lz89W3b19JUm1trQoKCvT0008HomQAgJ+dOlkt2dooKSnprI+pq6trwYpal9FBPWPGDK1Zs0Z//vOfFRER4X3fOSoqSmFhYbLZbEpLS1NmZqYSExOVmJiozMxMhYeHa+LEiQGuHgDgD3WnaiSrXsPSl+uS6M6nXVtZtk9bsmeqrq6+lapreUYH9ZIlSyRJycnJPuPLly/XXXfdJUl6+OGHdeLECU2fPl0VFRUaMGCANm7cqIiIiFauFgDQktpd0lHtI6NPu+akp6KVqmk9Rge1ZVlnXGOz2ZSRkaGMjIyWLwgAgFYWNHd9AwBwMSKoAQAwGEENAIDBCGoAAAxGUAMAYDCCGgAAgxHUAAAYjKAGAMBgBDUAAAYjqAEAMBhBDQCAwQhqAAAMRlADAGAwghoAAIMR1AAAGIygBgDAYAQ1AAAGI6gBADAYQQ0AgMEIagAADEZQAwBgMIIaAACDEdQAABiMoAYAwGAENQAABiOoAQAwGEENAIDBCGoAAAxGUAMAYLCQQBcAAGg9Ho9H1dXVZ7X28OHDLVwNzgZBDQAXCY/Ho27xCao4euScjqurq2uhinA2CGoAuEhUV1er4ugRjZy/Su0jOp1xfWXZPm3Jnqm6uvpWqA5NIagB4CLTPqKT2kdGn3HdSU9FK1SDM+FmMgAADEZQAwBgMIIaAACDEdQAABiMoAYAwGAENQAABiOoAQAwGEENAIDBCGoAAAxGUAMAYDCCGgAAgxHUAAAYjKAGAMBgBDUAAAa7YB5z+fzzz+uZZ55RWVmZevbsqZycHP3rv/5rQGrxeDyqrq4+q7Xh4eGKiIho4YoAXMjO9nfO4cOHW6Ea+NsFEdR/+MMflJaWpueff16DBw/W//7v/2rUqFH65JNP1LVr11atxePxqFt8giqOHjmr9Z2iY7R/XylhDaBZzvV3jiTV1dW1YEXwtwsiqLOzs3X33XfrnnvukSTl5OQoLy9PS5YsUVZWVqvWUl1drYqjRzRy/iq1j+h02rUnPRXKWzBZ1dXVBDWAZjmX3zmVZfu0JXum6urqW6k6+EPQB3Vtba2Ki4v16KOP+oynpKSosLCw0WNqampUU1Pj3a+srJQkVVVVnXc9Ho9HkvRdzQmdCrWfdu13NSckSaWlpd7jzsSyLNlstoCuDdZzm1JHS57blDpa8tym1GHKuf/5z39KOrffOdVHymSrP3XGcx8/4j7r9eeytiXP3ZJ11Bz7VtL3v+fDwsLOeO6zERERceafsxXkDh06ZEmy3nvvPZ/xJ5980rrqqqsaPeaJJ56wJLGxsbGxsQV0q6ysPGPOBf0V9Q9++i8S6zT/Gk1PT9ecOXO8+/X19Tp69KhiYmLO6V+8gVZVVaW4uDgdPHhQkZGRgS7H7y7k/ugteF3I/V3IvUlm9nc2b3sGfVBfeumlatu2rdxut894eXm5HA5Ho8fY7XbZ7b4vEXXs2LGlSmxxkZGRxvyfriVcyP3RW/C6kPu7kHuTgq+/oP8cdbt27dSvXz/l5+f7jOfn52vQoEEBqgoAAP8I+itqSZozZ44mT56s/v37a+DAgVq6dKkOHDige++9N9ClAQBwXi6IoL7tttt05MgRLViwQGVlZerVq5def/11devWLdCltSi73a4nnniiwcv4F4oLuT96C14Xcn8Xcm9S8PZnsyzLCnQRAACgcUH/HjUAABcyghoAAIMR1AAAGIygBgDAYAR1ENi6davGjh0rl8slm82m9evX+8xblqWMjAy5XC6FhYUpOTlZu3fvDkyx5ygrK0vXX3+9IiIi1KVLF40bN0579uzxWROs/S1ZskTXXnut948rDBw4UG+88YZ3Plj7akxWVpZsNpvS0tK8Y8HcX0ZGhmw2m8/mdDq988HcmyQdOnRId9xxh2JiYhQeHq4+ffqouLjYOx/M/cXHxzf42dlsNs2YMUNScPZGUAeB48ePq3fv3srNzW10fuHChcrOzlZubq62bdsmp9OpESNGnPWDPgKpoKBAM2bM0Pvvv6/8/Hx99913SklJ0fHjx71rgrW/yy67TE899ZSKiopUVFSkm266Sb/4xS+8vxSCta+f2rZtm5YuXaprr73WZzzY++vZs6fKysq8286dO71zwdxbRUWFBg8erNDQUL3xxhv65JNPtGjRIp+/zhjM/W3bts3n5/bDH8O69dZbJQVpb+fxPAwEgCRr3bp13v36+nrL6XRaTz31lHfs5MmTVlRUlPXCCy8EoMLzU15ebkmyCgoKLMu68Prr1KmT9X//938XTF8ej8dKTEy08vPzrSFDhlizZ8+2LCv4f25PPPGE1bt370bngr23Rx55xLrhhhuanA/2/n5q9uzZ1hVXXGHV19cHbW9cUQe50tJSud1upaSkeMfsdruGDBnS5GM+TfbDI0ejo6MlXTj91dXVae3atTp+/LgGDhx4wfQ1Y8YMjR49WsOHD/cZvxD6+/zzz+VyuZSQkKAJEyboq6++khT8vW3YsEH9+/fXrbfeqi5duqhv375atmyZdz7Y+/ux2tparV69WlOnTpXNZgva3gjqIPfDw0h++gASh8PR4EElprMsS3PmzNENN9ygXr16SQr+/nbu3KlLLrlEdrtd9957r9atW6cePXoEfV+StHbtWm3fvl1ZWVkN5oK9vwEDBuill15SXl6eli1bJrfbrUGDBunIkSNB39tXX32lJUuWKDExUXl5ebr33nt1//3366WXXpIU/D+7H1u/fr2+/fZb3XXXXZKCt7cL4k+I4twe82mqmTNnaseOHXr33XcbzAVrf927d1dJSYm+/fZb/elPf9KUKVNUUFDgnQ/Wvg4ePKjZs2dr48aNat++fZPrgrW/UaNGeb9OSkrSwIEDdcUVV2jlypX6+c9/Lil4e6uvr1f//v2VmZkpSerbt692796tJUuW6M477/SuC9b+fuzFF1/UqFGj5HK5fMaDrTeuqIPcD3einstjPk00a9YsbdiwQZs3b9Zll13mHQ/2/tq1a6crr7xS/fv3V1ZWlnr37q3nnnsu6PsqLi5WeXm5+vXrp5CQEIWEhKigoEC//e1vFRIS4u0hWPv7qQ4dOigpKUmff/550P/sYmNj1aNHD5+xa665RgcOHJAU/P/N/WD//v3atGmT7rnnHu9YsPZGUAe5hIQEOZ1On8d81tbWqqCgICge82lZlmbOnKnXXntNb7/9thISEnzmg72/n7IsSzU1NUHf17Bhw7Rz506VlJR4t/79+2vSpEkqKSnR5ZdfHtT9/VRNTY0+/fRTxcbGBv3PbvDgwQ0+Arl3717vQ4yCvb8fLF++XF26dNHo0aO9Y0HbW6DuYsPZ83g81kcffWR99NFHliQrOzvb+uijj6z9+/dblmVZTz31lBUVFWW99tpr1s6dO63bb7/dio2NtaqqqgJc+Zndd999VlRUlLVlyxarrKzMu1VXV3vXBGt/6enp1tatW63S0lJrx44d1q9//WurTZs21saNGy3LCt6+mvLju74tK7j7mzt3rrVlyxbrq6++st5//31rzJgxVkREhLVv3z7LsoK7tw8//NAKCQmxnnzySevzzz+3Xn75ZSs8PNxavXq1d00w92dZllVXV2d17drVeuSRRxrMBWNvBHUQ2Lx5syWpwTZlyhTLsr7/OMUTTzxhOZ1Oy263WzfeeKO1c+fOwBZ9lhrrS5K1fPly75pg7W/q1KlWt27drHbt2lmdO3e2hg0b5g1pywrevpry06AO5v5uu+02KzY21goNDbVcLpeVmppq7d692zsfzL1ZlmX95S9/sXr16mXZ7Xbr6quvtpYuXeozH+z95eXlWZKsPXv2NJgLxt54zCUAAAbjPWoAAAxGUAMAYDCCGgAAgxHUAAAYjKAGAMBgBDUAAAYjqAEAMBhBDQCAwQhqAD7i4+OVk5MT0BruuusujRs3LqA1AKYgqAGcls1m0/r16wNdBnDRIqiBIFJbWxvoEgC0MoIaMFhycrJmzpypOXPm6NJLL9WIESP0ySef6Oabb9Yll1wih8OhyZMn65///Kf3mD/+8Y9KSkpSWFiYYmJiNHz4cB0/ftx7vrS0NJ/vMW7cON11112Nfv/4+HhJ0r//+7/LZrN595uyZ88e2Ww2ffbZZz7j2dnZio+Pl2VZqqur0913362EhASFhYWpe/fueu6550573sZeju/Tp48yMjK8+5WVlfrVr36lLl26KDIyUjfddJM+/vjj054XCAYENWC4lStXKiQkRO+9956eeuopDRkyRH369FFRUZHefPNNffPNNxo/frwkqaysTLfffrumTp2qTz/9VFu2bFFqaqqa++ydbdu2Sfr+2b5lZWXe/aZ0795d/fr108svv+wzvmbNGk2cOFE2m0319fW67LLL9Oqrr+qTTz7R/Pnz9etf/1qvvvpqs2qUvn/O9+jRo+V2u/X666+ruLhY1113nYYNG6ajR482+7yACUICXQCA07vyyiu1cOFCSdL8+fN13XXXKTMz0zv/+9//XnFxcdq7d6+OHTum7777TqmpqerWrZskKSkpqdnfu3PnzpKkjh07yul0ntUxkyZNUm5urv77v/9bkrR3714VFxfrpZdekiSFhobqN7/5jXd9QkKCCgsL9eqrr3r/wXGuNm/erJ07d6q8vFx2u12S9D//8z9av369/vjHP+pXv/pVs84LmIArasBw/fv3935dXFyszZs365JLLvFuV199tSTpyy+/VO/evTVs2DAlJSXp1ltv1bJly1RRUdGq9U6YMEH79+/X+++/L0l6+eWX1adPH/Xo0cO75oUXXlD//v3VuXNnXXLJJVq2bJkOHDjQ7O9ZXFysY8eOKSYmxud/m9LSUn355Zfn3RMQSFxRA4br0KGD9+v6+nqNHTtWTz/9dIN1sbGxatu2rfLz81VYWKiNGzdq8eLFmjdvnj744AMlJCSoTZs2DV4GP3XqlF/rjY2N1dChQ7VmzRr9/Oc/1yuvvKJp06Z551999VU98MADWrRokQYOHKiIiAg988wz+uCDD5o855nqrq+vV2xsrLZs2dLg2I4dO553T0AgEdRAELnuuuv0pz/9SfHx8QoJafw/X5vNpsGDB2vw4MGaP3++unXrpnXr1mnOnDnq3LmzysrKvGvr6uq0a9cuDR06tMnvGRoaqrq6unOqc9KkSXrkkUd0++2368svv9SECRO8c++8844GDRqk6dOne8fOdNX707qrqqpUWlrq3b/uuuvkdrsVEhJyxhvegGDDS99AEJkxY4aOHj2q22+/XR9++KG++uorbdy4UVOnTlVdXZ0++OADZWZmqqioSAcOHNBrr72mw4cP65prrpEk3XTTTfrb3/6mv/3tb/rss880ffp0ffvtt6f9nvHx8XrrrbfkdrvP+mX01NRUVVVV6b777tPQoUP1s5/9zDt35ZVXqqioSHl5edq7d68ef/zxM96kdtNNN2nVqlV65513tGvXLk2ZMkVt27b1zg8fPlwDBw7UuHHjlJeXp3379qmwsFCPPfaYioqKzqpmwFQENRBEXC6X3nvvPdXV1WnkyJHq1auXZs+eraioKLVp00aRkZHaunWrbr75Zl111VV67LHHtGjRIo0aNUqSNHXqVE2ZMkV33nmnhgwZooSEhNNeTUvSokWLlJ+fr7i4OPXt2/es6oyMjNTYsWP18ccfa9KkST5z9957r1JTU3XbbbdpwIABOnLkiM/VdWPS09N14403asyYMbr55ps1btw4XXHFFd55m82m119/XTfeeKOmTp2qq666ShMmTNC+ffvkcDjOqmbAVDaruZ/bAAAALY4ragAADEZQAzgnPXv29PkI1I+3n/6hEwDnj5e+AZyT/fv3N/mRLofDoYiIiFauCLiwEdQAABiMl74BADAYQQ0AgMEIagAADEZQAwBgMIIaAACDEdQAABiMoAYAwGD/D6Y4BWRzardaAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 500x500 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "sns.displot(height['result_value'])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4bccaf75",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "source": [
    "**Potential red flag:** Is there really a patient who might be 5 inches tall? This is a clear case I'd like to investigate more. Perhaps this is a data entry issue or a newborn?"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fef95b9c-a55b-434a-8aae-3fb059f38377",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "**Further explore data to check for inconsistencies**"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c807f9cc-4925-4501-a28d-2c99f980693d",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "source": [
    "First, we can sort the height values from lowest to highest in order to retrieve the `subject_id` of the patient with a height of 5 inches\n",
    "\n",
    "I can also see if there are any other patients with unusually low heights"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "401f9519-8d9f-4b4a-a053-b6517cda6482",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>subject_id</th>\n",
       "      <th>chartdate</th>\n",
       "      <th>seq_num</th>\n",
       "      <th>result_name</th>\n",
       "      <th>result_value</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1844</th>\n",
       "      <td>10012853</td>\n",
       "      <td>2175-04-05</td>\n",
       "      <td>1</td>\n",
       "      <td>Height (Inches)</td>\n",
       "      <td>5.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2883</th>\n",
       "      <td>10002428</td>\n",
       "      <td>2154-10-18</td>\n",
       "      <td>1</td>\n",
       "      <td>Height (Inches)</td>\n",
       "      <td>58.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2884</th>\n",
       "      <td>10002428</td>\n",
       "      <td>2158-09-13</td>\n",
       "      <td>1</td>\n",
       "      <td>Height (Inches)</td>\n",
       "      <td>59.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2885</th>\n",
       "      <td>10002428</td>\n",
       "      <td>2155-08-12</td>\n",
       "      <td>1</td>\n",
       "      <td>Height (Inches)</td>\n",
       "      <td>59.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1262</th>\n",
       "      <td>10037928</td>\n",
       "      <td>2175-12-06</td>\n",
       "      <td>1</td>\n",
       "      <td>Height (Inches)</td>\n",
       "      <td>60.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>776</th>\n",
       "      <td>10014354</td>\n",
       "      <td>2150-05-01</td>\n",
       "      <td>1</td>\n",
       "      <td>Height (Inches)</td>\n",
       "      <td>73.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>775</th>\n",
       "      <td>10014354</td>\n",
       "      <td>2147-06-04</td>\n",
       "      <td>1</td>\n",
       "      <td>Height (Inches)</td>\n",
       "      <td>73.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>777</th>\n",
       "      <td>10014354</td>\n",
       "      <td>2148-12-07</td>\n",
       "      <td>1</td>\n",
       "      <td>Height (Inches)</td>\n",
       "      <td>73.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2951</th>\n",
       "      <td>10004733</td>\n",
       "      <td>2175-12-10</td>\n",
       "      <td>1</td>\n",
       "      <td>Height (Inches)</td>\n",
       "      <td>73.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>48</th>\n",
       "      <td>10007818</td>\n",
       "      <td>2146-06-10</td>\n",
       "      <td>1</td>\n",
       "      <td>Height (Inches)</td>\n",
       "      <td>73.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>378 rows × 5 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "      subject_id   chartdate  seq_num      result_name  result_value\n",
       "1844    10012853  2175-04-05        1  Height (Inches)           5.0\n",
       "2883    10002428  2154-10-18        1  Height (Inches)          58.0\n",
       "2884    10002428  2158-09-13        1  Height (Inches)          59.0\n",
       "2885    10002428  2155-08-12        1  Height (Inches)          59.0\n",
       "1262    10037928  2175-12-06        1  Height (Inches)          60.0\n",
       "...          ...         ...      ...              ...           ...\n",
       "776     10014354  2150-05-01        1  Height (Inches)          73.0\n",
       "775     10014354  2147-06-04        1  Height (Inches)          73.0\n",
       "777     10014354  2148-12-07        1  Height (Inches)          73.0\n",
       "2951    10004733  2175-12-10        1  Height (Inches)          73.0\n",
       "48      10007818  2146-06-10        1  Height (Inches)          73.0\n",
       "\n",
       "[378 rows x 5 columns]"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "height.sort_values(['result_value'])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6a86cf85-0390-4eb4-818f-55e79d9c341e",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "source": [
    "Let's see if this patient has other recorded height measurements"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "a7fd8904",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>subject_id</th>\n",
       "      <th>chartdate</th>\n",
       "      <th>seq_num</th>\n",
       "      <th>result_name</th>\n",
       "      <th>result_value</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1844</th>\n",
       "      <td>10012853</td>\n",
       "      <td>2175-04-05</td>\n",
       "      <td>1</td>\n",
       "      <td>Height (Inches)</td>\n",
       "      <td>5.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1846</th>\n",
       "      <td>10012853</td>\n",
       "      <td>2176-06-07</td>\n",
       "      <td>1</td>\n",
       "      <td>Height (Inches)</td>\n",
       "      <td>64.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1847</th>\n",
       "      <td>10012853</td>\n",
       "      <td>2177-01-05</td>\n",
       "      <td>1</td>\n",
       "      <td>Height (Inches)</td>\n",
       "      <td>64.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1845</th>\n",
       "      <td>10012853</td>\n",
       "      <td>2178-11-04</td>\n",
       "      <td>1</td>\n",
       "      <td>Height (Inches)</td>\n",
       "      <td>64.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      subject_id   chartdate  seq_num      result_name  result_value\n",
       "1844    10012853  2175-04-05        1  Height (Inches)           5.0\n",
       "1846    10012853  2176-06-07        1  Height (Inches)          64.0\n",
       "1847    10012853  2177-01-05        1  Height (Inches)          64.0\n",
       "1845    10012853  2178-11-04        1  Height (Inches)          64.0"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# let's see if this patient has other recorded height measurements\n",
    "height[height['subject_id']==10012853].sort_values('chartdate')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "059cd240",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "**Investigate patients with multiple observations**"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "43f821b3-272e-4075-a00d-e8c035fc07d2",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "source": [
    "For each unique patient, we will calculate the standard deviation of the patient's height measurements\n",
    "\n",
    "*Note*: `ddof = 0` indicates that std for patients with one measurement will be displayed as 0 rather than NaN"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "a258a3aa",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\meghan_hutch\\AppData\\Local\\Temp\\ipykernel_34512\\368505687.py:1: FutureWarning: The provided callable <function std at 0x0000023B82357600> is currently using SeriesGroupBy.std. In a future version of pandas, the provided callable will be used directly. To keep current behavior pass the string \"std\" instead.\n",
      "  height['std'] = height.groupby('subject_id')['result_value'].transform(np.std, ddof=0)\n"
     ]
    }
   ],
   "source": [
    "height['std'] = height.groupby('subject_id')['result_value'].transform(np.std, ddof=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "53e4b0d9",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>subject_id</th>\n",
       "      <th>std</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1845</th>\n",
       "      <td>10012853</td>\n",
       "      <td>25.547749</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1096</th>\n",
       "      <td>10005909</td>\n",
       "      <td>2.724312</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>461</th>\n",
       "      <td>10019917</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2719</th>\n",
       "      <td>10022880</td>\n",
       "      <td>0.942809</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>299</th>\n",
       "      <td>10001725</td>\n",
       "      <td>0.862706</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>10011398</td>\n",
       "      <td>0.750000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1263</th>\n",
       "      <td>10037928</td>\n",
       "      <td>0.745356</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2733</th>\n",
       "      <td>10004457</td>\n",
       "      <td>0.731247</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2885</th>\n",
       "      <td>10002428</td>\n",
       "      <td>0.707107</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>534</th>\n",
       "      <td>10005348</td>\n",
       "      <td>0.596212</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      subject_id        std\n",
       "1845    10012853  25.547749\n",
       "1096    10005909   2.724312\n",
       "461     10019917   1.000000\n",
       "2719    10022880   0.942809\n",
       "299     10001725   0.862706\n",
       "13      10011398   0.750000\n",
       "1263    10037928   0.745356\n",
       "2733    10004457   0.731247\n",
       "2885    10002428   0.707107\n",
       "534     10005348   0.596212"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "height.sort_values('std', ascending = False)[['subject_id', 'std']].drop_duplicates().head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "571be392-c5cf-427f-a3e6-4888c9480fd9",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "source": [
    "We can clearly see the one 'concerning' patient has a much higher standard deviation of their height measurements compared to other patients.\n",
    "\n",
    "Let's take a look at patient with the second highest standard deviation "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "id": "1fe650c0",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>subject_id</th>\n",
       "      <th>chartdate</th>\n",
       "      <th>seq_num</th>\n",
       "      <th>result_name</th>\n",
       "      <th>result_value</th>\n",
       "      <th>std</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1094</th>\n",
       "      <td>10005909</td>\n",
       "      <td>2144-10-29</td>\n",
       "      <td>1</td>\n",
       "      <td>Height (Inches)</td>\n",
       "      <td>65.0</td>\n",
       "      <td>2.724312</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1102</th>\n",
       "      <td>10005909</td>\n",
       "      <td>2144-11-15</td>\n",
       "      <td>1</td>\n",
       "      <td>Height (Inches)</td>\n",
       "      <td>72.5</td>\n",
       "      <td>2.724312</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1095</th>\n",
       "      <td>10005909</td>\n",
       "      <td>2144-12-10</td>\n",
       "      <td>1</td>\n",
       "      <td>Height (Inches)</td>\n",
       "      <td>70.0</td>\n",
       "      <td>2.724312</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1096</th>\n",
       "      <td>10005909</td>\n",
       "      <td>2145-01-03</td>\n",
       "      <td>1</td>\n",
       "      <td>Height (Inches)</td>\n",
       "      <td>70.0</td>\n",
       "      <td>2.724312</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      subject_id   chartdate  seq_num      result_name  result_value       std\n",
       "1094    10005909  2144-10-29        1  Height (Inches)          65.0  2.724312\n",
       "1102    10005909  2144-11-15        1  Height (Inches)          72.5  2.724312\n",
       "1095    10005909  2144-12-10        1  Height (Inches)          70.0  2.724312\n",
       "1096    10005909  2145-01-03        1  Height (Inches)          70.0  2.724312"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "height[height['subject_id']==10005909].sort_values('chartdate')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "49d26544-8aed-4284-b1ca-b5e646ad4d96",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "**For patients with multiple observations, let's calculate the median**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "id": "0544a024-3b30-4fa3-b13e-29b511783f26",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "# create a new variable with the median height value for each patient\n",
    "height['result_value_median'] = height.groupby('subject_id')['result_value'].transform('median')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2221a6f4-7766-40d6-9b63-b4c5e43565e9",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "**Save the new dataset**\n",
    "\n",
    "This next step ensures that we will now only have one unique row per patient"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "id": "a29609a8-630c-4273-8b2c-bb5ce3a558c6",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "# assuming we don't need to link by date\n",
    "height_clean = height[['subject_id', 'result_name', 'result_value_median']].drop_duplicates()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "52d977b8-0180-4697-84bf-1231e9d62eba",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "source": [
    "Let's review the distribution again"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "id": "121fa636-c409-4ce7-8f69-55a63318232a",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<seaborn.axisgrid.FacetGrid at 0x23b88a9ac10>"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAeoAAAHpCAYAAABN+X+UAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8hTgPZAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAoRElEQVR4nO3de3zNd57H8fchHFQSFRVJm0i0dVdRjCpTtMUqWmO3daej7Wpdiux2NXUJ9kF6WRnzYNSwy3RX9fLYxdhtS1N1a7UqUYoGpSFGkzFRk+MSEcl3/+jDmYZETiI5v+9JXs/H4zwe/f1+5+R8vlLn5Xdy+bmMMUYAAMBKtZweAAAAlI5QAwBgMUINAIDFCDUAABYj1AAAWIxQAwBgMUINAIDFqn2ojTHyeDzix8UBAIGo2of6/PnzCg0N1fnz550eBQCAcqv2oQYAIJARagAALEaoAQCwGKEGAMBihBoAAIsRagAALEaoAQCwGKEGAMBihBoAAIsRagAALEaoAQCwGKEGAMBihBoAAIsRagAALEaoAQCwGKEGAMBihBoAAIsRagAALBbk9AAAbk1mZqZycnKcHqNKNGnSRNHR0U6PATiKUAMBLDMzU61bt1Fe3iWnR6kS9es30OHD6cQaNRqhBgJYTk6O8vIuqdv4RIVExDg9TqXyZJ3Q7lXzlJOTQ6hRoxFqoBoIiYhR4+hWTo8BoArwzWQAAFiMUAMAYDFCDQCAxQg1AAAWI9QAAFiMUAMAYDFCDQCAxQg1AAAWI9QAAFiMUAMAYDFCDQCAxQg1AAAWI9QAAFiMUAMAYDFCDQCAxQg1AAAWI9QAAFiMUAMAYDFCDQCAxQg1AAAWI9QAAFiMUAMAYDFCDQCAxQg1AAAWI9QAAFjM0VDv2LFDgwcPVmRkpFwulzZs2OA9VlBQoBkzZqhDhw667bbbFBkZqbFjx+qHH35wbmAAAPzM0VBfvHhRHTt21NKlS284dunSJe3du1ezZ8/W3r17tW7dOh09elSPP/64A5MCAOCMICeffMCAARowYECJx0JDQ5WSklJs35IlS/SLX/xCmZmZio6O9seIAAA4ytFQl1dubq5cLpcaNWpU6n3y8/OVn5/v3fZ4PH6YDEBVSU9Pd3qEKtGkSZNqe8KRmZmpnJwcp8eoEk583gIm1JcvX9bLL7+skSNHKiQkpNT7JSUlad68eX6cDEBVyMs9K8ml0aNHOz1Klahfv4EOH06vdrHOzMxU69ZtlJd3yelRqoQTn7eACHVBQYGGDx+uoqIiLVu27Kb3TUhIUHx8vHfb4/EoKiqqqkcEUMkKLp2XZBQ3cobuiG3t9DiVypN1QrtXzVNOTk61C3VOTo7y8i6p2/hEhUTEOD1OpXLq82Z9qAsKCvTUU08pIyNDn3766U3PpiXJ7XbL7Xb7aToAVa1h02g1jm7l9Bgop5CIGD5vlcTqUF+L9HfffaetW7cqLCzM6ZEAAPArR0N94cIFHTt2zLudkZGhffv2qXHjxoqMjNQ//MM/aO/evfq///s/FRYWKjs7W5LUuHFj1a1b16mxAQDwG0dDnZqaqj59+ni3r31tedy4cZo7d642btwoSYqLiyv2uK1bt6p3797+GhMAAMc4GurevXvLGFPq8ZsdAwCgJuB3fQMAYDFCDQCAxQg1AAAWI9QAAFiMUAMAYDFCDQCAxQg1AAAWI9QAAFiMUAMAYDFCDQCAxQg1AAAWI9QAAFiMUAMAYDFCDQCAxQg1AAAWI9QAAFiMUAMAYDFCDQCAxQg1AAAWI9QAAFiMUAMAYDFCDQCAxQg1AAAWI9QAAFiMUAMAYDFCDQCAxQg1AAAWI9QAAFiMUAMAYDFCDQCAxQg1AAAWI9QAAFiMUAMAYDFCDQCAxQg1AAAWI9QAAFiMUAMAYDFCDQCAxQg1AAAWI9QAAFiMUAMAYDFCDQCAxQg1AAAWI9QAAFiMUAMAYDFCDQCAxQg1AAAWI9QAAFiMUAMAYDFCDQCAxQg1AAAWI9QAAFjM0VDv2LFDgwcPVmRkpFwulzZs2FDsuDFGc+fOVWRkpOrXr6/evXvr0KFDzgwLAIADHA31xYsX1bFjRy1durTE46+//rqSk5O1dOlS7dmzR82aNVPfvn11/vx5P08KAIAzgpx88gEDBmjAgAElHjPGaPHixZo5c6aGDh0qSXrrrbcUHh6utWvXasKECSU+Lj8/X/n5+d5tj8dT+YMDAOAn1n6NOiMjQ9nZ2erXr593n9vtVq9evbRr165SH5eUlKTQ0FDvLSoqyh/jAgBQJawNdXZ2tiQpPDy82P7w8HDvsZIkJCQoNzfXezt16lSVzgkAQFVy9K1vX7hcrmLbxpgb9v2c2+2W2+2u6rEAAPALa8+omzVrJkk3nD2fOXPmhrNsAACqK2tDHRsbq2bNmiklJcW778qVK9q+fbsefPBBBycDAMB/HH3r+8KFCzp27Jh3OyMjQ/v27VPjxo0VHR2tadOmaeHChbr33nt17733auHChWrQoIFGjhzp4NQAAPiPo6FOTU1Vnz59vNvx8fGSpHHjxukPf/iD/uVf/kV5eXmaOHGizp07p27duunjjz9WcHCwUyMDAOBXjoa6d+/eMsaUetzlcmnu3LmaO3eu/4YCAMAi1n6NGgAAEGoAAKxGqAEAsBihBgDAYoQaAACLEWoAACxGqAEAsBihBgDAYoQaAACLEWoAACxGqAEAsBihBgDAYoQaAACLEWoAACxGqAEAsBihBgDAYoQaAACLEWoAACxGqAEAsBihBgDAYoQaAACLEWoAACxGqAEAsBihBgDAYoQaAACLEWoAACxGqAEAsBihBgDAYoQaAACLEWoAACxGqAEAsBihBgDAYoQaAACLEWoAACxGqAEAsBihBgDAYoQaAACLEWoAACxGqAEAsBihBgDAYoQaAACLEWoAACxGqAEAsBihBgDAYoQaAACLEWoAACxGqAEAsBihBgDAYoQaAACLEWoAACxGqAEAsJjVob569apmzZql2NhY1a9fXy1atND8+fNVVFTk9GgAAPhFkNMD3Mxrr72m5cuX66233lK7du2UmpqqX//61woNDdXUqVOdHg8AgCpndai/+OILPfHEExo4cKAkKSYmRu+8845SU1MdngwAAP+wOtQ9e/bU8uXLdfToUbVs2VL79+/XZ599psWLF5f6mPz8fOXn53u3PR6PHyatHjIzM5WTk+P0GFWiSZMmio6OdnoMoJj09HSnR6h01XFNTrM61DNmzFBubq5at26t2rVrq7CwUAsWLNCIESNKfUxSUpLmzZvnxymrh8zMTLVu3UZ5eZecHqVK1K/fQIcPpxNrWCEv96wkl0aPHu30KFWmIP+K0yNUG1aH+r333tOaNWu0du1atWvXTvv27dO0adMUGRmpcePGlfiYhIQExcfHe7c9Ho+ioqL8NXLAysnJUV7eJXUbn6iQiBinx6lUnqwT2r1qnnJycgg1rFBw6bwko7iRM3RHbGunx6lUWQe+0MGNK3T16lWnR6k2rA71Sy+9pJdfflnDhw+XJHXo0EEnT55UUlJSqaF2u91yu93+HLNaCYmIUePoVk6PAdQIDZtGV7u/b56sE06PUO1Y/eNZly5dUq1axUesXbs2P54FAKgxrD6jHjx4sBYsWKDo6Gi1a9dOX3/9tZKTkzV+/HinRwMAwC+sDvWSJUs0e/ZsTZw4UWfOnFFkZKQmTJigOXPmOD0aAAB+YXWog4ODtXjx4pv+OBYAANWZ1V+jBgCgpiPUAABYjFADAGAxQg0AgMUINQAAFiPUAABYjFADAGAxQg0AgMUINQAAFiPUAABYjFADAGAxQg0AgMUqFOoWLVro7NmzN+z/61//qhYtWtzyUAAA4CcVCvWJEydUWFh4w/78/HydPn36locCAAA/KddlLjdu3Oj9782bNys0NNS7XVhYqC1btigmJqbShgMAoKYrV6iHDBkiSXK5XBo3blyxY3Xq1FFMTIwWLVpUacMBAFDTlSvURUVFkqTY2Fjt2bNHTZo0qZKhAADAT8oV6msyMjIqew4AAFCCCoVakrZs2aItW7bozJkz3jPta1atWnXLgwEAgAqGet68eZo/f766dOmiiIgIuVyuyp4LAACogqFevny5/vCHP2jMmDGVPQ8AAPiZCv0c9ZUrV/Tggw9W9iwAAOA6FQr1s88+q7Vr11b2LAAA4DoVeuv78uXLWrFihT755BPdd999qlOnTrHjycnJlTIcAAA1XYVC/c033yguLk6SdPDgwWLH+MYyAAAqT4VCvXXr1sqeAwAAlIDLXAIAYLEKnVH36dPnpm9xf/rppxUeCAAA/E2FQn3t69PXFBQUaN++fTp48OANF+sAAAAVV6FQ/+Y3vylx/9y5c3XhwoVbGggAAPxNpX6NevTo0fyebwAAKlGlhvqLL75QvXr1KvNDAgBQo1Xore+hQ4cW2zbGKCsrS6mpqZo9e3alDAYAACoY6tDQ0GLbtWrVUqtWrTR//nz169evUgYDAAAVDPXq1asrew4AAFCCCoX6mrS0NKWnp8vlcqlt27bq1KlTZc0FAABUwVCfOXNGw4cP17Zt29SoUSMZY5Sbm6s+ffro3Xff1R133FHZcwIAUCNV6Lu+p0yZIo/Ho0OHDunHH3/UuXPndPDgQXk8Hr344ouVPSMAADVWhc6oN23apE8++URt2rTx7mvbtq1+97vf8c1kAABUogqdURcVFd1wDWpJqlOnjoqKim55KAAA8JMKhfrhhx/W1KlT9cMPP3j3nT59WtOnT9cjjzxSacMBAFDTVSjUS5cu1fnz5xUTE6O7775b99xzj2JjY3X+/HktWbKksmcEAKDGqtDXqKOiorR3716lpKTo8OHDMsaobdu2evTRRyt7PgAAarRynVF/+umnatu2rTwejySpb9++mjJlil588UV17dpV7dq1086dO6tkUAAAaqJyhXrx4sV67rnnFBIScsOx0NBQTZgwQcnJyZU2HAAANV25Qr1//3793d/9XanH+/Xrp7S0tFseCgAA/KRcof7zn/9c4o9lXRMUFKS//OUvtzwUAAD4SblCfeedd+rAgQOlHv/mm28UERFxy0MBAICflCvUjz32mObMmaPLly/fcCwvL0+JiYkaNGhQpQ0HAEBNV64fz5o1a5bWrVunli1bavLkyWrVqpVcLpfS09P1u9/9ToWFhZo5c2ZVzQoAQI1TrlCHh4dr165deuGFF5SQkCBjjCTJ5XKpf//+WrZsmcLDw6tkUAAAaqJy/8KT5s2b68MPP9S5c+d07NgxGWN077336vbbb6+K+QAAqNEq9CtEJen2229X165d9Ytf/KJKI3369GmNHj1aYWFhatCggeLi4vgRMABAjVGhXyHqL+fOnVOPHj3Up08fffTRR2ratKmOHz+uRo0aOT0aAAB+YXWoX3vtNUVFRWn16tXefTExMTd9TH5+vvLz873b137dKZCenu70CJWuOq4JQHFWh3rjxo3q37+/nnzySW3fvl133nmnJk6cqOeee67UxyQlJWnevHl+nBK2y8s9K8ml0aNHOz1KlSnIv+L0CACqiNWh/v777/Xmm28qPj5er7zyir766iu9+OKLcrvdGjt2bImPSUhIUHx8vHfb4/EoKirKXyPDQgWXzksyihs5Q3fEtnZ6nEqVdeALHdy4QlevXnV6FABVxOpQFxUVqUuXLlq4cKEkqVOnTjp06JDefPPNUkPtdrvldrv9OSYCRMOm0Woc3crpMSqVJ+uE0yMAqGIV/q5vf4iIiFDbtm2L7WvTpo0yMzMdmggAAP+yOtQ9evTQkSNHiu07evSomjdv7tBEAAD4l9Whnj59ur788kstXLhQx44d09q1a7VixQpNmjTJ6dEAAPALq0PdtWtXrV+/Xu+8847at2+vf/3Xf9XixYs1atQop0cDAMAvrP5mMkkaNGgQV+QCANRYVp9RAwBQ0xFqAAAsRqgBALAYoQYAwGKEGgAAixFqAAAsRqgBALAYoQYAwGKEGgAAixFqAAAsRqgBALAYoQYAwGKEGgAAixFqAAAsRqgBALAYoQYAwGKEGgAAixFqAAAsRqgBALAYoQYAwGKEGgAAixFqAAAsRqgBALAYoQYAwGKEGgAAixFqAAAsRqgBALAYoQYAwGKEGgAAixFqAAAsRqgBALAYoQYAwGKEGgAAixFqAAAsRqgBALAYoQYAwGKEGgAAixFqAAAsRqgBALAYoQYAwGKEGgAAixFqAAAsRqgBALAYoQYAwGKEGgAAixFqAAAsRqgBALAYoQYAwGKEGgAAixFqAAAsRqgBALBYQIU6KSlJLpdL06ZNc3oUAAD8ImBCvWfPHq1YsUL33Xef06MAAOA3ARHqCxcuaNSoUVq5cqVuv/12p8cBAMBvAiLUkyZN0sCBA/Xoo4+Wed/8/Hx5PJ5iNwAAAlWQ0wOU5d1339XevXu1Z88en+6flJSkefPmVfFUAAD4h9Vn1KdOndLUqVO1Zs0a1atXz6fHJCQkKDc313s7depUFU8JAEDVsfqMOi0tTWfOnFHnzp29+woLC7Vjxw4tXbpU+fn5ql27drHHuN1uud1uf48KAECVsDrUjzzyiA4cOFBs369//Wu1bt1aM2bMuCHSAABUN1aHOjg4WO3bty+277bbblNYWNgN+wEAqI6s/ho1AAA1ndVn1CXZtm2b0yMAAOA3nFEDAGAxQg0AgMUINQAAFiPUAABYjFADAGAxQg0AgMUINQAAFiPUAABYjFADAGAxQg0AgMUINQAAFiPUAABYjFADAGAxQg0AgMUINQAAFiPUAABYjFADAGAxQg0AgMUINQAAFiPUAABYjFADAGAxQg0AgMUINQAAFiPUAABYjFADAGAxQg0AgMUINQAAFiPUAABYjFADAGAxQg0AgMUINQAAFiPUAABYjFADAGAxQg0AgMUINQAAFiPUAABYjFADAGAxQg0AgMUINQAAFiPUAABYjFADAGAxQg0AgMUINQAAFiPUAABYjFADAGAxQg0AgMUINQAAFiPUAABYjFADAGAxQg0AgMUINQAAFiPUAABYzOpQJyUlqWvXrgoODlbTpk01ZMgQHTlyxOmxAADwG6tDvX37dk2aNElffvmlUlJSdPXqVfXr108XL150ejQAAPwiyOkBbmbTpk3FtlevXq2mTZsqLS1NDz30UImPyc/PV35+vnfb4/FU6kyZmZnKycmp1I9pg/T0dKdHAACUwOpQXy83N1eS1Lhx41Lvk5SUpHnz5lXJ82dmZqp16zbKy7tUJR/fBgX5V5weAQDwMwETamOM4uPj1bNnT7Vv377U+yUkJCg+Pt677fF4FBUVVSkz5OTkKC/vkrqNT1RIREylfExbZB34Qgc3rtDVq1edHgUA8DMBE+rJkyfrm2++0WeffXbT+7ndbrnd7iqdJSQiRo2jW1Xpc/ibJ+uE0yMAAEoQEKGeMmWKNm7cqB07duiuu+5yehwAAPzG6lAbYzRlyhStX79e27ZtU2xsrNMjAQDgV1aHetKkSVq7dq3++Mc/Kjg4WNnZ2ZKk0NBQ1a9f3+HpAACoelb/HPWbb76p3Nxc9e7dWxEREd7be++95/RoAAD4hdVn1MYYp0cAAMBRVp9RAwBQ0xFqAAAsRqgBALAYoQYAwGKEGgAAixFqAAAsRqgBALAYoQYAwGKEGgAAixFqAAAsRqgBALAYoQYAwGKEGgAAixFqAAAsRqgBALAYoQYAwGKEGgAAixFqAAAsRqgBALAYoQYAwGKEGgAAixFqAAAsRqgBALAYoQYAwGKEGgAAixFqAAAsRqgBALAYoQYAwGKEGgAAixFqAAAsRqgBALAYoQYAwGKEGgAAixFqAAAsRqgBALAYoQYAwGKEGgAAixFqAAAsRqgBALAYoQYAwGKEGgAAixFqAAAsRqgBALAYoQYAwGKEGgAAixFqAAAsRqgBALAYoQYAwGKEGgAAixFqAAAsRqgBALBYQIR62bJlio2NVb169dS5c2ft3LnT6ZEAAPAL60P93nvvadq0aZo5c6a+/vpr/fKXv9SAAQOUmZnp9GgAAFQ560OdnJysZ555Rs8++6zatGmjxYsXKyoqSm+++abTowEAUOWCnB7gZq5cuaK0tDS9/PLLxfb369dPu3btKvEx+fn5ys/P927n5uZKkjwezy3Pc+HCBUnSjyeP6Gp+3i1/PJt4sk5KknJPf6c6QS6Hp6lcrC0wsbbAVK3Xlv3TO7kXLlyolKZIUnBwsFyuMv6cjMVOnz5tJJnPP/+82P4FCxaYli1blviYxMREI4kbN27cuHGz/pabm1tmC60+o77m+n9tGGNK/RdIQkKC4uPjvdtFRUX68ccfFRYWVva/Wizj8XgUFRWlU6dOKSQkxOlxKhVrC0zVdW3VdV0Sa7NdcHBwmfexOtRNmjRR7dq1lZ2dXWz/mTNnFB4eXuJj3G633G53sX2NGjWqqhH9IiQkJGD/JywLawtM1XVt1XVdEmsLZFZ/M1ndunXVuXNnpaSkFNufkpKiBx980KGpAADwH6vPqCUpPj5eY8aMUZcuXdS9e3etWLFCmZmZev75550eDQCAKmd9qIcNG6azZ89q/vz5ysrKUvv27fXhhx+qefPmTo9W5dxutxITE294K786YG2BqbqurbquS2Jt1YHLGGOcHgIAAJTM6q9RAwBQ0xFqAAAsRqgBALAYoQYAwGKE2gKnT5/W6NGjFRYWpgYNGiguLk5paWne48YYzZ07V5GRkapfv7569+6tQ4cOOTix7262toKCAs2YMUMdOnTQbbfdpsjISI0dO1Y//PCDw1P7pqzP289NmDBBLpdLixcv9u+QFeTL2tLT0/X4448rNDRUwcHBeuCBBwLiqnZlre3ChQuaPHmy7rrrLtWvX19t2rQJiIsAxcTEyOVy3XCbNGmSpMB+HbnZ2gL9dcQXhNph586dU48ePVSnTh199NFH+vbbb7Vo0aJiv03t9ddfV3JyspYuXao9e/aoWbNm6tu3r86fP+/c4D4oa22XLl3S3r17NXv2bO3du1fr1q3T0aNH9fjjjzs7uA98+bxds2HDBu3evVuRkZH+H7QCfFnb8ePH1bNnT7Vu3Vrbtm3T/v37NXv2bNWrV8+5wX3gy9qmT5+uTZs2ac2aNUpPT9f06dM1ZcoU/fGPf3RucB/s2bNHWVlZ3tu1XxT15JNPSgrc1xHp5msL5NcRn93qhTNwa2bMmGF69uxZ6vGioiLTrFkz8+qrr3r3Xb582YSGhprly5f7Y8QKK2ttJfnqq6+MJHPy5Mkqmqpy+Lq2P/3pT+bOO+80Bw8eNM2bNze/+c1vqn64W+TL2oYNG2ZGjx7tp4kqjy9ra9eunZk/f36xfffff7+ZNWtWVY5W6aZOnWruvvtuU1RUFNCvIyX5+dpKEiivI77ijNphGzduVJcuXfTkk0+qadOm6tSpk1auXOk9npGRoezsbPXr18+7z+12q1evXqVe6tMWZa2tJLm5uXK5XNb/fnZf1lZUVKQxY8bopZdeUrt27RyatPzKWltRUZE++OADtWzZUv3791fTpk3VrVs3bdiwwbmhfeTL561nz57auHGjTp8+LWOMtm7dqqNHj6p///4OTV1+V65c0Zo1azR+/Hi5XK6Afh253vVrK0mgvI74zOl/KdR0brfbuN1uk5CQYPbu3WuWL19u6tWrZ9566y1jjDGff/65kWROnz5d7HHPPfec6devnxMj+6ystV0vLy/PdO7c2YwaNcrPk5afL2tbuHCh6du3r/df/YFyRl3W2rKysowk06BBA5OcnGy+/vprk5SUZFwul9m2bZvD09+cL5+3/Px8M3bsWCPJBAUFmbp165r//M//dHDq8nvvvfdM7dq1va8bgfw6cr3r13a9QHod8RWhdlidOnVM9+7di+2bMmWKeeCBB4wxf/sL9sMPPxS7z7PPPmv69+/vtzkroqy1/dyVK1fME088YTp16uTT9VmdVtbaUlNTTXh4eLEXk0AJdVlru3ad+BEjRhS7z+DBg83w4cP9NmdF+PL/5BtvvGFatmxpNm7caPbv32+WLFliGjZsaFJSUvw9boX169fPDBo0yLsdyK8j17t+bT8XaK8jvuKtb4dFRESobdu2xfa1adPG+92zzZo1k6RyXerTFmWt7ZqCggI99dRTysjIUEpKSkBcrq6ste3cuVNnzpxRdHS0goKCFBQUpJMnT+qf/umfFBMT48DEvitrbU2aNFFQUJBPn1vblLW2vLw8vfLKK0pOTtbgwYN13333afLkyRo2bJj+7d/+zYmRy+3kyZP65JNP9Oyzz3r3BfLryM+VtLZrAvF1xFeE2mE9evTQkSNHiu07evSo96IjsbGxatasWbFLfV65ckXbt2+3/lKfZa1N+ttfru+++06ffPKJwsLC/D1mhZS1tjFjxuibb77Rvn37vLfIyEi99NJL2rx5sxMj+6ystdWtW1ddu3Yt83Nro7LWVlBQoIKCAtWqVfylsXbt2ioqKvLbnLdi9erVatq0qQYOHOjdF8ivIz9X0tqkwH0d8ZnTp/Q13VdffWWCgoLMggULzHfffWfefvtt06BBA7NmzRrvfV599VUTGhpq1q1bZw4cOGBGjBhhIiIijMfjcXDyspW1toKCAvP444+bu+66y+zbt89kZWV5b/n5+Q5Pf3O+fN6uFyhvffuytnXr1pk6deqYFStWmO+++84sWbLE1K5d2+zcudPBycvmy9p69epl2rVrZ7Zu3Wq+//57s3r1alOvXj2zbNkyByf3TWFhoYmOjjYzZsy44Vigvo5cU9raAvl1xFeE2gL/+7//a9q3b2/cbrdp3bq1WbFiRbHjRUVFJjEx0TRr1sy43W7z0EMPmQMHDjg0bfncbG0ZGRlGUom3rVu3Oje0j8r6vF0vUEJtjG9r+4//+A9zzz33mHr16pmOHTuaDRs2ODBp+ZW1tqysLPP000+byMhIU69ePdOqVSuzaNGiUn8UyCabN282ksyRI0duOBbIryPGlL62QH8d8QWXuQQAwGJ8jRoAAIsRagAALEaoAQCwGKEGAMBihBoAAIsRagAALEaoAQCwGKEGAMBihBqwRExMjBYvXuzoDE8//bSGDBni6AxVZe7cuYqLi/NuV+e1onoJcnoAACVzuVxav349Makiv/3tb8UvZkQgINSAD65cuaK6des6PQYqUWhoqNMjAD7hrW+gBL1799bkyZMVHx+vJk2aqG/fvvr222/12GOPqWHDhgoPD9eYMWOUk5Pjfcx///d/q0OHDqpfv77CwsL06KOP6uLFi96PN23atGLPMWTIED399NMlPv+1a1b/6le/ksvlKvMa1keOHJHL5dLhw4eL7U9OTlZMTIyMMSosLNQzzzyj2NhY1a9fX61atdJvf/vbm37ckt6Oj4uL09y5c73bubm5+sd//Ec1bdpUISEhevjhh7V///6bftxrrr0dvWrVKkVHR6thw4Z64YUXVFhYqNdff13NmjVT06ZNtWDBgmKP8+U5X331VYWHhys4OFjPPPOMLl++XOz49W99b9q0ST179lSjRo0UFhamQYMG6fjx497jJ06ckMvl0rp169SnTx81aNBAHTt21BdffOHTWoGKItRAKd566y0FBQXp888/16uvvqpevXopLi5Oqamp2rRpk/785z/rqaeekiRlZWVpxIgRGj9+vNLT07Vt2zYNHTq0wm+t7tmzR9JP19/NysrybpemVatW6ty5s95+++1i+9euXauRI0fK5XKpqKhId911l95//319++23mjNnjl555RW9//77FZpRkowxGjhwoLKzs/Xhhx8qLS1N999/vx555BH9+OOPPn2M48eP66OPPtKmTZv0zjvvaNWqVRo4cKD+9Kc/afv27Xrttdc0a9Ysffnllz4/5/vvv6/ExEQtWLBAqampioiI0LJly246x8WLFxUfH689e/Zoy5YtqlWrln71q1/dcB3qmTNn6p//+Z+1b98+tWzZUiNGjNDVq1cr8KcH+MjBK3cB1urVq5eJi4vzbs+ePdv069ev2H1OnTrlvexeWlqakWROnDhR6sebOnVqsX1PPPGEGTdunHf7+stgSjLr16/3eebk5GTTokUL7/aRI0eMJHPo0KFSHzNx4kTz93//997tcePGmSeeeKLUmYwxpmPHjiYxMdEYY8yWLVtMSEiIuXz5crH73H333eb3v/99mTMnJiaaBg0aFLsmcv/+/U1MTIwpLCz07mvVqpVJSkry+Tm7d+9unn/++WLHu3XrZjp27FjqWq935swZI8l7Kchrl1P893//d+99Dh06ZCSZ9PT0MtcKVBRn1EApunTp4v3vtLQ0bd26VQ0bNvTeWrduLemnM8KOHTvqkUceUYcOHfTkk09q5cqVOnfunF/nHT58uE6ePOk983z77bcVFxentm3beu+zfPlydenSRXfccYcaNmyolStXKjMzs8LPmZaWpgsXLigsLKzYn01GRkaxt41vJiYmRsHBwd7t8PBwtW3bVrVq1Sq278yZMz4/Z3p6urp3717sea7fvt7x48c1cuRItWjRQiEhIYqNjZWkG/587rvvPu9/R0RESJJ3NqAq8M1kQCluu+02738XFRVp8ODBeu211264X0REhGrXrq2UlBTt2rVLH3/8sZYsWaKZM2dq9+7dio2NVa1atW54G7ygoKBS542IiFCfPn20du1aPfDAA3rnnXc0YcIE7/H3339f06dP16JFi9S9e3cFBwfrjTfe0O7du0v9mGXNXVRUpIiICG3btu2GxzZq1MinuevUqVNs2+Vylbjv2lvQlfGcJRk8eLCioqK0cuVKRUZGqqioSO3bt9eVK1dKndflcnlnAqoKoQZ8cP/99+t//ud/FBMTo6Cgkv/auFwu9ejRQz169NCcOXPUvHlzrV+/XvHx8brjjjuUlZXlvW9hYaEOHjyoPn36lPqcderUUWFhYbnmHDVqlGbMmKERI0bo+PHjGj58uPfYzp079eCDD2rixInefWWd9V4/t8fjUUZGhnf7/vvvV3Z2toKCgsr8hrfK4stztmnTRl9++aXGjh3r3XftnYaSnD17Vunp6fr973+vX/7yl5Kkzz77rFLnBiqKt74BH0yaNEk//vijRowYoa+++krff/+9Pv74Y40fP16FhYXavXu3Fi5cqNTUVGVmZmrdunX6y1/+ojZt2kiSHn74YX3wwQf64IMPdPjwYU2cOFF//etfb/qcMTEx2rJli7Kzs31+G33o0KHyeDx64YUX1KdPH915553eY/fcc49SU1O1efNmHT16VLNnzy7zm9Qefvhh/dd//Zd27typgwcPaty4capdu7b3+KOPPqru3btryJAh2rx5s06cOKFdu3Zp1qxZSk1N9Wnm8vLlOadOnapVq1Zp1apVOnr0qBITE3Xo0KFSP+btt9+usLAwrVixQseOHdOnn36q+Pj4KpkfKC9CDfggMjJSn3/+uQoLC9W/f3+1b99eU6dOVWhoqGrVqqWQkBDt2LFDjz32mFq2bKlZs2Zp0aJFGjBggCRp/PjxGjdunMaOHatevXopNjb2pmfTkrRo0SKlpKQoKipKnTp18mnOkJAQDR48WPv379eoUaOKHXv++ec1dOhQDRs2TN26ddPZs2eLnV2XJCEhQQ899JAGDRqkxx57TEOGDNHdd9/tPe5yufThhx/qoYce0vjx49WyZUsNHz5cJ06cUHh4uE8zl5cvzzls2DDNmTNHM2bMUOfOnXXy5Em98MILpX7MWrVq6d1331VaWprat2+v6dOn64033qiS+YHycpnrvwAFAACswRk1AAAWI9RAgGjXrl2xH0f6+e36X3Rii0CcGbANb30DAeLkyZOl/kjXtV+VaZtAnBmwDaEGAMBivPUNAIDFCDUAABYj1AAAWIxQAwBgMUINAIDFCDUAABYj1AAAWOz/AYfwEChA8f3LAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 500x500 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "sns.displot(height_clean['result_value_median'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "id": "e16f27d2-a946-43fa-93e7-3fb0fa2d59ed",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# the number of unique subject_id should match our original height dataframe\n",
    "height_clean['subject_id'].nunique() == height['subject_id'].nunique()\n",
    "\n",
    "# important to check to make sure we didn't accidently lose information"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "792882fd",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "### Another Example of Data Cleaning (Weight)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "id": "fdbdf0a6",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>subject_id</th>\n",
       "      <th>chartdate</th>\n",
       "      <th>seq_num</th>\n",
       "      <th>result_name</th>\n",
       "      <th>result_value</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>10011398</td>\n",
       "      <td>2147-01-22</td>\n",
       "      <td>1</td>\n",
       "      <td>Weight (Lbs)</td>\n",
       "      <td>127</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>10011398</td>\n",
       "      <td>2146-12-01</td>\n",
       "      <td>1</td>\n",
       "      <td>Weight (Lbs)</td>\n",
       "      <td>135</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>10011398</td>\n",
       "      <td>2147-07-24</td>\n",
       "      <td>1</td>\n",
       "      <td>Weight (Lbs)</td>\n",
       "      <td>136</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>10011398</td>\n",
       "      <td>2147-03-26</td>\n",
       "      <td>1</td>\n",
       "      <td>Weight (Lbs)</td>\n",
       "      <td>136</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>10011398</td>\n",
       "      <td>2147-06-25</td>\n",
       "      <td>1</td>\n",
       "      <td>Weight (Lbs)</td>\n",
       "      <td>139</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   subject_id   chartdate  seq_num   result_name result_value\n",
       "1    10011398  2147-01-22        1  Weight (Lbs)          127\n",
       "2    10011398  2146-12-01        1  Weight (Lbs)          135\n",
       "3    10011398  2147-07-24        1  Weight (Lbs)          136\n",
       "4    10011398  2147-03-26        1  Weight (Lbs)          136\n",
       "5    10011398  2147-06-25        1  Weight (Lbs)          139"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "weight.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f31c3d57",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "source": [
    "**Review variable types**\n",
    "\n",
    "It looks like `result_value` is a continuous variable. Let's make sure that Python also has recognized it as such."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "id": "39db656c",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "subject_id       int64\n",
       "chartdate       object\n",
       "seq_num          int64\n",
       "result_name     object\n",
       "result_value    object\n",
       "dtype: object"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# check variable data types\n",
    "weight.dtypes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "id": "63c76423",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "weight['result_value'] = pd.to_numeric(weight['result_value'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "id": "33eb7d22",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "subject_id        int64\n",
       "chartdate        object\n",
       "seq_num           int64\n",
       "result_name      object\n",
       "result_value    float64\n",
       "dtype: object"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# check variable data types\n",
    "weight.dtypes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "id": "ed8487aa",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "count    941.000000\n",
       "mean     173.202763\n",
       "std       44.009522\n",
       "min       88.000000\n",
       "25%      147.300000\n",
       "50%      169.310000\n",
       "75%      185.000000\n",
       "max      296.000000\n",
       "Name: result_value, dtype: float64"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "weight['result_value'].describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2094cd6b",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "**Distribution of weight values**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "id": "a68a72ef",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<seaborn.axisgrid.FacetGrid at 0x23b88b5f250>"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAeoAAAHpCAYAAABN+X+UAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8hTgPZAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAu2klEQVR4nO3de3BU9f3/8ddCwhowCRBgN9HcrAG5CQoWQSv3WBSUL/MtItTiQC2KoBGvKV5Wv18TwRLjEC/FrwWqUuq0gM5XKwTlIqZYCCDXSLUxQZpt+l1jlkDMjfP7g3F/riSYbHazn5DnY+bM5Jzz+Zx975mz88q52yzLsgQAAIzUKdwFAACAphHUAAAYjKAGAMBgBDUAAAYjqAEAMBhBDQCAwQhqAAAMRlBLsixLXq9X3FIOADANQS3pxIkTio2N1YkTJ8JdCgAAfghqAAAMRlADAGAwghoAAIMR1AAAGIygBgDAYAQ1AAAGI6gBADAYQQ0AgMEIagAADEZQAwBgMIIaAACDEdQAABiMoAYAwGAENQAABiOoAQAwGEENAIDBCGoAAAxGUAMAYLCIcBcAoGmlpaXyeDwt6hMXF6ekpKQQVQSgrRHUgKFKS0t1Wf8Bqj51skX9orp2U9GRw4Q1cJ4gqAFDeTweVZ86qavnuhTjTG5WH6+7RDtfdcnj8RDUwHmCoAYMF+NMVo+kfuEuA0CYcDEZAAAGI6gBADAYQQ0AgMEIagAADEZQAwBgMIIaAACDEdQAABiMoAYAwGAENQAABiOoAQAwGEENAIDBCGoAAAxGUAMAYDCCGgAAgxHUAAAYjKAGAMBgBDUAAAYjqAEAMFhYg3r79u2aMmWKEhISZLPZtGHDhibbzps3TzabTbm5uX7Ta2pqtHDhQvXq1UvdunXTTTfdpC+//DK0hQMA0EbCGtQnT57UkCFDlJeXd852GzZs0Mcff6yEhISz5mVkZGj9+vVau3atduzYoaqqKk2ePFkNDQ2hKhsAgDYTEc4PnzRpkiZNmnTONsePH9eCBQu0ceNG3XjjjX7zKisr9eqrr+q1117ThAkTJEmvv/66EhMTtXnzZl1//fUhqx0AgLZg9Dnq06dP67bbbtODDz6ogQMHnjW/sLBQdXV1Sk9P901LSEjQoEGDVFBQ0ORya2pq5PV6/QYAAExkdFAvWbJEERERuueeexqd73a71aVLF/Xo0cNvusPhkNvtbnK52dnZio2N9Q2JiYlBrRsAgGAxNqgLCwv1/PPPa9WqVbLZbC3qa1nWOftkZmaqsrLSNxw7dqy15QIAEBLGBvWHH36o8vJyJSUlKSIiQhERESopKdH999+vlJQUSZLT6VRtba0qKir8+paXl8vhcDS5bLvdrpiYGL8BAAATGRvUt912m/bv3699+/b5hoSEBD344IPauHGjJGnYsGGKjIxUfn6+r19ZWZkOHjyoUaNGhat0AACCJqxXfVdVVemzzz7zjRcXF2vfvn3q2bOnkpKSFBcX59c+MjJSTqdT/fr1kyTFxsZq7ty5uv/++xUXF6eePXvqgQce0ODBg31XgQMA0J6FNah3796tsWPH+sYXLVokSZo9e7ZWrVrVrGU899xzioiI0PTp01VdXa3x48dr1apV6ty5cyhKBgCgTYU1qMeMGSPLsprd/osvvjhr2gUXXKDly5dr+fLlQawMAAAzGHuOGgAAENQAABiNoAYAwGAENQAABiOoAQAwGEENAIDBCGoAAAxGUAMAYDCCGgAAgxHUAAAYjKAGAMBgBDUAAAYjqAEAMBhBDQCAwQhqAAAMRlADAGAwghoAAIMR1AAAGIygBgDAYAQ1AAAGI6gBADAYQQ0AgMEIagAADEZQAwBgMIIaAACDEdQAABiMoAYAwGAENQAABiOoAQAwGEENAIDBCGoAAAxGUAMAYDCCGgAAgxHUAAAYjKAGAMBgBDUAAAYjqAEAMBhBDQCAwQhqAAAMRlADAGAwghoAAIMR1AAAGIygBgDAYAQ1AAAGC2tQb9++XVOmTFFCQoJsNps2bNjgm1dXV6eHH35YgwcPVrdu3ZSQkKBf/OIX+uc//+m3jJqaGi1cuFC9evVSt27ddNNNN+nLL79s428CAEBohDWoT548qSFDhigvL++seadOndKePXv02GOPac+ePVq3bp2OHj2qm266ya9dRkaG1q9fr7Vr12rHjh2qqqrS5MmT1dDQ0FZfAwCAkIkI54dPmjRJkyZNanRebGys8vPz/aYtX75cP/7xj1VaWqqkpCRVVlbq1Vdf1WuvvaYJEyZIkl5//XUlJiZq8+bNuv766xtddk1NjWpqanzjXq83SN8IAIDgalfnqCsrK2Wz2dS9e3dJUmFhoerq6pSenu5rk5CQoEGDBqmgoKDJ5WRnZys2NtY3JCYmhrp0AAAC0m6C+ptvvtEjjzyimTNnKiYmRpLkdrvVpUsX9ejRw6+tw+GQ2+1uclmZmZmqrKz0DceOHQtp7QAABCqsh76bq66uTjNmzNDp06f14osv/mB7y7Jks9manG+322W324NZIgAAIWH8HnVdXZ2mT5+u4uJi5efn+/amJcnpdKq2tlYVFRV+fcrLy+VwONq6VAAAgs7ooP42pP/+979r8+bNiouL85s/bNgwRUZG+l10VlZWpoMHD2rUqFFtXS4AAEEX1kPfVVVV+uyzz3zjxcXF2rdvn3r27KmEhAT953/+p/bs2aP//d//VUNDg++8c8+ePdWlSxfFxsZq7ty5uv/++xUXF6eePXvqgQce0ODBg31XgQMA0J6FNah3796tsWPH+sYXLVokSZo9e7ZcLpfefvttSdLQoUP9+m3ZskVjxoyRJD333HOKiIjQ9OnTVV1drfHjx2vVqlXq3Llzm3wHAABCKaxBPWbMGFmW1eT8c8371gUXXKDly5dr+fLlwSwNAAAjGH2OGgCAjo6gBgDAYAQ1AAAGI6gBADAYQQ0AgMEIagAADEZQAwBgMIIaAACDEdQAABiMoAYAwGAENQAABiOoAQAwGEENAIDBCGoAAAxGUAMAYDCCGgAAgxHUAAAYjKAGAMBgBDUAAAYjqAEAMBhBDQCAwQhqAAAMRlADAGAwghoAAIMR1AAAGIygBgDAYAQ1AAAGI6gBADAYQQ0AgMEIagAADEZQAwBgMIIaAACDRYS7AKCjKC0tlcfjaXb7oqKiEFYDoL0gqIE2UFpaqsv6D1D1qZMt7ltbWxeCigC0FwQ10AY8Ho+qT53U1XNdinEmN6tP2cGdOvDWb1VfXx/i6gCYjKAG2lCMM1k9kvo1q63XXRLiagC0B1xMBgCAwQhqAAAMRlADAGAwghoAAIMR1AAAGIygBgDAYAQ1AAAGC2tQb9++XVOmTFFCQoJsNps2bNjgN9+yLLlcLiUkJCgqKkpjxozRoUOH/NrU1NRo4cKF6tWrl7p166abbrpJX375ZRt+CwAAQiesQX3y5EkNGTJEeXl5jc5funSpcnJylJeXp127dsnpdGrixIk6ceKEr01GRobWr1+vtWvXaseOHaqqqtLkyZPV0NDQVl8DAICQCeuTySZNmqRJkyY1Os+yLOXm5mrx4sWaNm2aJGn16tVyOBxas2aN5s2bp8rKSr366qt67bXXNGHCBEnS66+/rsTERG3evFnXX399m30XAABCwdhz1MXFxXK73UpPT/dNs9vtGj16tAoKCiRJhYWFqqur82uTkJCgQYMG+do0pqamRl6v128AAMBExga12+2WJDkcDr/pDofDN8/tdqtLly7q0aNHk20ak52drdjYWN+QmJgY5OoBAAgOY4P6WzabzW/csqyzpn3fD7XJzMxUZWWlbzh27FhQagUAINiMDWqn0ylJZ+0Zl5eX+/aynU6namtrVVFR0WSbxtjtdsXExPgNAACYyNigTk1NldPpVH5+vm9abW2ttm3bplGjRkmShg0bpsjISL82ZWVlOnjwoK8NAADtWViv+q6qqtJnn33mGy8uLta+ffvUs2dPJSUlKSMjQ1lZWUpLS1NaWpqysrLUtWtXzZw5U5IUGxuruXPn6v7771dcXJx69uypBx54QIMHD/ZdBQ4AQHsW1qDevXu3xo4d6xtftGiRJGn27NlatWqVHnroIVVXV2v+/PmqqKjQiBEjtGnTJkVHR/v6PPfcc4qIiND06dNVXV2t8ePHa9WqVercuXObfx8AAIItrEE9ZswYWZbV5HybzSaXyyWXy9VkmwsuuEDLly/X8uXLQ1AhAADhZew5agAAQFADAGA0ghoAAIMR1AAAGIygBgDAYAQ1AAAGI6gBADAYQQ0AgMEIagAADEZQAwBgMIIaAACDEdQAABiMoAYAwGAENQAABiOoAQAwGEENAIDBCGoAAAxGUAMAYDCCGgAAgxHUAAAYjKAGAMBgBDUAAAYjqAEAMBhBDQCAwQhqAAAMRlADAGAwghoAAINFhLsAIJxKS0vl8Xha1CcuLk5JSUkhqggA/BHU6LBKS0t1Wf8Bqj51skX9orp2U9GRw4Q1gDZBUKPD8ng8qj51UlfPdSnGmdysPl53iXa+6pLH4yGoAbQJghodXowzWT2S+oW7DABoFBeTAQBgMIIaAACDEdQAABiMoAYAwGABBfUll1zS6L2nX3/9tS655JJWFwUAAM4IKKi/+OILNTQ0nDW9pqZGx48fb3VRAADgjBbdnvX222/7/t64caNiY2N94w0NDXr//feVkpIStOIAAOjoWhTUU6dOlSTZbDbNnj3bb15kZKRSUlK0bNmyoBUHAEBH16KgPn36tCQpNTVVu3btUq9evUJSFAAAOCOgJ5MVFxcHuw4AANCIgB8h+v777+v9999XeXm5b0/7W7/73e9aXRgAAAgwqJ988kk99dRTGj58uOLj42Wz2YJdFwAAUIBB/fLLL2vVqlW67bbbgl0PAAD4joDuo66trdWoUaOCXctZ6uvr9eijjyo1NVVRUVG65JJL9NRTT/kdarcsSy6XSwkJCYqKitKYMWN06NChkNcGAEBbCCiof/nLX2rNmjXBruUsS5Ys0csvv6y8vDwdOXJES5cu1bPPPqvly5f72ixdulQ5OTnKy8vTrl275HQ6NXHiRJ04cSLk9QEAEGoBHfr+5ptvtGLFCm3evFmXX365IiMj/ebn5OQEpbi//vWvuvnmm3XjjTdKklJSUvSHP/xBu3fvlnRmbzo3N1eLFy/WtGnTJEmrV6+Ww+HQmjVrNG/evEaXW1NTo5qaGt+41+sNSr0AAARbQHvU+/fv19ChQ9WpUycdPHhQe/fu9Q379u0LWnHXXnut3n//fR09elSS9Mknn2jHjh264YYbJJ25Tcztdis9Pd3Xx263a/To0SooKGhyudnZ2YqNjfUNiYmJQasZAIBgCmiPesuWLcGuo1EPP/ywKisrddlll6lz585qaGjQ008/rVtvvVWS5Ha7JUkOh8Ovn8PhUElJSZPLzczM1KJFi3zjXq+XsAYAGCng+6jbwh//+Ee9/vrrWrNmjQYOHKh9+/YpIyNDCQkJfo8w/f7tYZZlnfOWMbvdLrvdHrK6AQAIloCCeuzYsecMwg8++CDggr7rwQcf1COPPKIZM2ZIkgYPHqySkhJlZ2dr9uzZcjqdks7sWcfHx/v6lZeXn7WXDQBAexTQOeqhQ4dqyJAhvmHAgAGqra3Vnj17NHjw4KAVd+rUKXXq5F9i586d/Z457nQ6lZ+f75tfW1urbdu2tcntYwAAhFpAe9TPPfdco9NdLpeqqqpaVdB3TZkyRU8//bSSkpI0cOBA7d27Vzk5OZozZ46kM4e8MzIylJWVpbS0NKWlpSkrK0tdu3bVzJkzg1YHAADhEtRz1D//+c/14x//WL/5zW+Csrzly5frscce0/z581VeXq6EhATNmzdPjz/+uK/NQw89pOrqas2fP18VFRUaMWKENm3apOjo6KDUAABAOAU1qP/617/qggsuCNryoqOjlZubq9zc3Cbb2Gw2uVwuuVyuoH0uAACmCCiov324yLcsy1JZWZl2796txx57LCiFAQhcUVFRi/vExcUpKSkpBNUAaI2Agjo2NtZvvFOnTurXr5+eeuopv4ePAGhb1ZUeyWYL6BqNqK7dVHTkMGENGCagoF65cmWw6wAQBHXVVZJlaeisTPVJ6dvsfl53iXa+6pLH4yGoAcO06hx1YWGhjhw5IpvNpgEDBuiKK64IVl0AWuHCPonqkdQv3GUACIKAgrq8vFwzZszQ1q1b1b17d1mWpcrKSo0dO1Zr165V7969g10nAAAdUkAPPFm4cKG8Xq8OHTqkr776ShUVFTp48KC8Xq/uueeeYNcIAECHFdAe9XvvvafNmzerf//+vmkDBgzQCy+8wMVkAAAEUUB71KdPnz7rHdSSFBkZ6Xu8JwAAaL2AgnrcuHG699579c9//tM37fjx47rvvvs0fvz4oBUHAEBHF1BQ5+Xl6cSJE0pJSdGPfvQjXXrppUpNTdWJEye0fPnyYNcIAECHFdA56sTERO3Zs0f5+fkqKiqSZVkaMGCAJkyYEOz6AADo0Fq0R/3BBx9owIAB8nq9kqSJEydq4cKFuueee3TVVVdp4MCB+vDDD0NSKAAAHVGLgjo3N1d33HGHYmJizpoXGxurefPmKScnJ2jFAQDQ0bUoqD/55BP99Kc/bXJ+enq6CgsLW10UAAA4o0VB/a9//avR27K+FRERoX//+9+tLgoAAJzRoqC+6KKLdODAgSbn79+/X/Hx8a0uCgAAnNGioL7hhhv0+OOP65tvvjlrXnV1tZ544glNnjw5aMUBANDRtej2rEcffVTr1q1T3759tWDBAvXr1082m01HjhzRCy+8oIaGBi1evDhUtQIA0OG0KKgdDocKCgp01113KTMzU5ZlSZJsNpuuv/56vfjii3I4HCEpFACAjqjFDzxJTk7Wu+++q4qKCn322WeyLEtpaWnq0aNHKOoDAKBDC+jJZJLUo0cPXXXVVcGsBQAAfE9Az/oGAABtg6AGAMBgBDUAAAYjqAEAMBhBDQCAwQhqAAAMRlADAGAwghoAAIMR1AAAGIygBgDAYAQ1AAAGI6gBADAYQQ0AgMEIagAADEZQAwBgMIIaAACDEdQAABiMoAYAwGAENQAABiOoAQAwGEENAIDBCGoAAAxmfFAfP35cP//5zxUXF6euXbtq6NChKiws9M23LEsul0sJCQmKiorSmDFjdOjQoTBWDABA8Bgd1BUVFbrmmmsUGRmpv/zlLzp8+LCWLVum7t27+9osXbpUOTk5ysvL065du+R0OjVx4kSdOHEifIUDABAkEeEu4FyWLFmixMRErVy50jctJSXF97dlWcrNzdXixYs1bdo0SdLq1avlcDi0Zs0azZs3r61LBgAgqIzeo3777bc1fPhw/exnP1OfPn10xRVX6JVXXvHNLy4ultvtVnp6um+a3W7X6NGjVVBQ0ORya2pq5PV6/QYAAExkdFD/4x//0EsvvaS0tDRt3LhRd955p+655x79/ve/lyS53W5JksPh8OvncDh88xqTnZ2t2NhY35CYmBi6LwEAQCsYHdSnT5/WlVdeqaysLF1xxRWaN2+e7rjjDr300kt+7Ww2m9+4ZVlnTfuuzMxMVVZW+oZjx46FpH4AAFrL6KCOj4/XgAED/Kb1799fpaWlkiSn0ylJZ+09l5eXn7WX/V12u10xMTF+AwAAJjI6qK+55hp9+umnftOOHj2q5ORkSVJqaqqcTqfy8/N982tra7Vt2zaNGjWqTWsFACAUjL7q+7777tOoUaOUlZWl6dOn629/+5tWrFihFStWSDpzyDsjI0NZWVlKS0tTWlqasrKy1LVrV82cOTPM1QMA0HpGB/VVV12l9evXKzMzU0899ZRSU1OVm5urWbNm+do89NBDqq6u1vz581VRUaERI0Zo06ZNio6ODmPlAAAEh9FBLUmTJ0/W5MmTm5xvs9nkcrnkcrnarigAANqI0eeoAQDo6AhqAAAMRlADAGAw489RwwylpaXyeDwt7hcXF6ekpKQQVOQvkPqKiopCVA0ABA9BjR9UWlqqy/oPUPWpky3uG9W1m4qOHA5pWLemPkmqra0LckUAEDwENX6Qx+NR9amTunquSzHO5Gb387pLtPNVlzweT0iDOtD6yg7u1IG3fqv6+vqQ1QYArUVQo9linMnqkdQv3GU0qaX1ed0lIawGAIKDi8kAADAYQQ0AgMEIagAADEZQAwBgMIIaAACDEdQAABiMoAYAwGAENQAABiOoAQAwGEENAIDBCGoAAAxGUAMAYDCCGgAAgxHUAAAYjKAGAMBgBDUAAAYjqAEAMBhBDQCAwQhqAAAMRlADAGCwiHAXAMAcRUVFLWofFxenpKSkEFUDQCKoAUiqrvRINptmzpzZon5RXbup6MhhwhoIIYIagOqqqyTL0tBZmeqT0rdZfbzuEu181SWPx0NQAyFEUAPwubBPonok9Qt3GQC+g4vJAAAwGEENAIDBOPQNBKClV0e3tD0AfIugBlog0Kujv1VbWxfkigCc7whqoAUCuTpaksoO7tSBt36r+vr6EFYH4HxEUAMBaOnV0V53SQirAXA+42IyAAAMRlADAGAwghoAAIMR1AAAGIygBgDAYAQ1AAAGI6gBADBYuwrq7Oxs2Ww2ZWRk+KZZliWXy6WEhARFRUVpzJgxOnToUPiKBAAgiNpNUO/atUsrVqzQ5Zdf7jd96dKlysnJUV5ennbt2iWn06mJEyfqxIkTYaoUAIDgaRdBXVVVpVmzZumVV15Rjx49fNMty1Jubq4WL16sadOmadCgQVq9erVOnTqlNWvWhLFiAACCo10E9d13360bb7xREyZM8JteXFwst9ut9PR03zS73a7Ro0eroKCgyeXV1NTI6/X6DQAAmMj4Z32vXbtWe/bs0a5du86a53a7JUkOh8NvusPhUElJ089Wzs7O1pNPPhncQgEACAGj96iPHTume++9V6+//rouuOCCJtvZbDa/ccuyzpr2XZmZmaqsrPQNx44dC1rNAAAEk9F71IWFhSovL9ewYcN80xoaGrR9+3bl5eXp008/lXRmzzo+Pt7Xpry8/Ky97O+y2+2y2+2hKxwAgCAxeo96/PjxOnDggPbt2+cbhg8frlmzZmnfvn265JJL5HQ6lZ+f7+tTW1urbdu2adSoUWGsHACA4DB6jzo6OlqDBg3ym9atWzfFxcX5pmdkZCgrK0tpaWlKS0tTVlaWunbtqpkzZ4ajZAAAgsrooG6Ohx56SNXV1Zo/f74qKio0YsQIbdq0SdHR0eEuDQCAVmt3Qb1161a/cZvNJpfLJZfLFZZ6AAAIJaPPUQMA0NER1AAAGKzdHfpG+1NUVNSi9t98880575tv7fIBoD0hqBEy1ZUeyWZr+RX4NptkWS3+vNrauhb3AQDTEdQImbrqKsmyNHRWpvqk9G1Wn7KDO3Xgrd8G1Ke+vr415QKAkQhqhNyFfRLVI6lfs9p63SUB9wGA8xEXkwEAYDCCGgAAgxHUAAAYjKAGAMBgBDUAAAYjqAEAMBhBDQCAwQhqAAAMRlADAGAwghoAAIMR1AAAGIygBgDAYAQ1AAAGI6gBADAYQQ0AgMEIagAADEZQAwBgMIIaAACDEdQAABiMoAYAwGAENQAABosIdwEA2reioqIW94mLi1NSUlIIqgHOPwQ1gIBUV3okm00zZ85scd+ort1UdOQwYQ00A0ENICB11VWSZWnorEz1Senb7H5ed4l2vuqSx+MhqIFmIKhDoLS0VB6Pp0V9OBSI9urCPonqkdQv3GUA5y2COshKS0t1Wf8Bqj51skX9OBQIAGgMQR1kHo9H1adO6uq5LsU4k5vVh0OBAICmENQhEuNM5nAgAKDVuI8aAACDEdQAABiMoAYAwGAENQAABuNiMgBAk3guRPgR1ACARvFcCDMQ1ACARvFcCDMQ1ACAc+K5EOHFxWQAABiMPWoAQNDxnvLgMTqos7OztW7dOhUVFSkqKkqjRo3SkiVL1K/f/z8EY1mWnnzySa1YsUIVFRUaMWKEXnjhBQ0cODCMlQNAx8R7yoPP6KDetm2b7r77bl111VWqr6/X4sWLlZ6ersOHD6tbt26SpKVLlyonJ0erVq1S37599d///d+aOHGiPv30U0VHR4f5GwBAx8J7yoPP6KB+7733/MZXrlypPn36qLCwUNddd50sy1Jubq4WL16sadOmSZJWr14th8OhNWvWaN68eY0ut6amRjU1Nb5xr9cbui/RAi09VMRhIrRnbO/nN95THjxGB/X3VVZWSpJ69uwpSSouLpbb7VZ6erqvjd1u1+jRo1VQUNBkUGdnZ+vJJ58MfcHNFOihIg4ToT1iewdapt0EtWVZWrRoka699loNGjRIkuR2uyVJDofDr63D4VBJSUmTy8rMzNSiRYt8416vV4mJiSGounkCOVTEYSK0V2zvQMu0m6BesGCB9u/frx07dpw1z2az+Y1blnXWtO+y2+2y2+1Br7G1OFSEjoTtvW0F8ijQQK7cRvC1i6BeuHCh3n77bW3fvl0XX3yxb7rT6ZR0Zs86Pj7eN728vPysvWwA6KgCfRTot2pr64JcEVrC6KC2LEsLFy7U+vXrtXXrVqWmpvrNT01NldPpVH5+vq644gpJUm1trbZt26YlS5aEo2QAME4gjwKVpLKDO3Xgrd+qvr4+hNXhhxgd1HfffbfWrFmjt956S9HR0b5z0rGxsYqKipLNZlNGRoaysrKUlpamtLQ0ZWVlqWvXrgHdwwcA57OWPgrU6276Wp9Q4W6Asxkd1C+99JIkacyYMX7TV65cqdtvv12S9NBDD6m6ulrz58/3PfBk06ZN3EMNAO0IdwM0zeigtizrB9vYbDa5XC65XK7QFwQACAnuBmia0UENAOhYuBvgbLw9CwAAgxHUAAAYjKAGAMBgBDUAAAbjYjIA7Qb32KIjIqgBGI97bNGREdQAjMc9tujICGoA7Qb32KIj4mIyAAAMxh41AHxPIO9u5sI1hApBDQDfEei7m7lwDaFCUAPAdwTy7mYuXEMoEdQA0IiWvrsZ7Ucgpzak8J3eIKgBAB1GoKc2pPCd3iCoAQAdRiCnNqTwnt4gqAEAHU57OrXBfdQAABiMoAYAwGAENQAABiOoAQAwGBeTAQDatZa8p7yl7zQ3AUENAGiXAn1PuSTV1taFoKLQIKgBAO1SIO8pLzu4Uwfe+q3q6+tDXF3wENQAgHatJe8p97pLQlxN8HExGQAABiOoAQAwGEENAIDBCGoAAAzGxWQAzmstvW+2Pd5ni/MbQQ3gvNSae2yl9nWfLc5vBDWA81Ig99hK7fM+W5zfCOoOqLS0VB6Pp9ntORSI9qwl99hKbX+fbUt/j5IUFxenpKSkEFUE0xDUHUxpaaku6z9A1adOtrgvhwKB4Ar09xjVtZuKjhwmrDsIgrqD8Xg8qj51UlfPdSnGmdysPhwKBEIjkN+j112ina+65PF4COoOgqDuoGKcyef1I/eAcAj0CvOW/B7R8RDUANBKXGGOUCKoAaCVuMIcoURQA0CQmH6FOdonHiEKAIDB2KNu53g8ItAxteS3zO++fSOo2ykuXgE6ptb89vndt08EdTvFxStAxxTIb5/ffft23gT1iy++qGeffVZlZWUaOHCgcnNz9ZOf/CTcZYUcF68AHVNLfvv87tu38+Jisj/+8Y/KyMjQ4sWLtXfvXv3kJz/RpEmTVFpaGu7SAABolfMiqHNycjR37lz98pe/VP/+/ZWbm6vExES99NJL4S4NAIBWafeHvmtra1VYWKhHHnnEb3p6eroKCgoa7VNTU6OamhrfeGVlpSTJ6/W2up6qqipJ0leln6q+prpZfbxlZw5LVR7/TF06N+9zAulzvn6W6fW15WeZXl9bfpbp9bXlZ5leX1t+VsD1/evMEdqqqqqgZIUkRUdHy2az/XBDq507fvy4Jcn66KOP/KY//fTTVt++fRvt88QTT1iSGBgYGBgYwjZUVlY2K+fa/R71t77/X4llWU3+p5KZmalFixb5xk+fPq2vvvpKcXFxzfvvpo15vV4lJibq2LFjiomJCXc55zXWddtgPbcN1nPbCHQ9R0dHN6tduw/qXr16qXPnznK73X7Ty8vL5XA4Gu1jt9tlt9v9pnXv3j1UJQZNTEwMP7Y2wrpuG6zntsF6bhuhWs/t/mKyLl26aNiwYcrPz/ebnp+fr1GjRoWpKgAAgqPd71FL0qJFi3Tbbbdp+PDhGjlypFasWKHS0lLdeeed4S4NAIBWOS+C+pZbbpHH49FTTz2lsrIyDRo0SO+++66Sk5PDXVpQ2O12PfHEE2cdrkfwsa7bBuu5bbCe20ao17PNsiwrJEsGAACt1u7PUQMAcD4jqAEAMBhBDQCAwQhqAAAMRlCH0fbt2zVlyhQlJCTIZrNpw4YNfvMty5LL5VJCQoKioqI0ZswYHTp0yK9NTU2NFi5cqF69eqlbt2666aab9OWXX7bhtzDfD63n22+/XTabzW+4+uqr/dqwns8tOztbV111laKjo9WnTx9NnTpVn376qV8btufgaM66ZptuvZdeekmXX3657yEmI0eO1F/+8hff/LbcngnqMDp58qSGDBmivLy8RucvXbpUOTk5ysvL065du+R0OjVx4kSdOHHC1yYjI0Pr16/X2rVrtWPHDlVVVWny5MlqaGhoq69hvB9az5L005/+VGVlZb7h3Xff9ZvPej63bdu26e6779bOnTuVn5+v+vp6paen6+TJk742bM/B0Zx1LbFNt9bFF1+sZ555Rrt379bu3bs1btw43Xzzzb4wbtPtuRXvw0AQSbLWr1/vGz99+rTldDqtZ555xjftm2++sWJjY62XX37ZsizL+vrrr63IyEhr7dq1vjbHjx+3OnXqZL333nttVnt78v31bFmWNXv2bOvmm29usg/rueXKy8stSda2bdssy2J7DqXvr2vLYpsOlR49elj/8z//0+bbM3vUhiouLpbb7VZ6erpvmt1u1+jRo32v7ywsLFRdXZ1fm4SEBA0aNKjJV3yicVu3blWfPn3Ut29f3XHHHSovL/fNYz233Levju3Zs6cktudQ+v66/hbbdPA0NDRo7dq1OnnypEaOHNnm2zNBbahvXzLy/ReLOBwO3zy3260uXbqoR48eTbbBD5s0aZLeeOMNffDBB1q2bJl27dqlcePG+d5ZznpuGcuytGjRIl177bUaNGiQJLbnUGlsXUts08Fy4MABXXjhhbLb7brzzju1fv16DRgwoM235/PiEaLns5a8vrMlbfD/3XLLLb6/Bw0apOHDhys5OVnvvPOOpk2b1mQ/1nPjFixYoP3792vHjh1nzWN7Dq6m1jXbdHD069dP+/bt09dff60///nPmj17trZt2+ab31bbM3vUhnI6nZJ0ztd3Op1O1dbWqqKiosk2aLn4+HglJyfr73//uyTWc0ssXLhQb7/9trZs2aKLL77YN53tOfiaWteNYZsOTJcuXXTppZdq+PDhys7O1pAhQ/T888+3+fZMUBsqNTVVTqfT7/WdtbW12rZtm+/1ncOGDVNkZKRfm7KyMh08eJBXfLaCx+PRsWPHFB8fL4n13ByWZWnBggVat26dPvjgA6WmpvrNZ3sOnh9a141hmw4Oy7JUU1PT9ttzwJe/odVOnDhh7d2719q7d68lycrJybH27t1rlZSUWJZlWc8884wVGxtrrVu3zjpw4IB16623WvHx8ZbX6/Ut484777Quvvhia/PmzdaePXuscePGWUOGDLHq6+vD9bWMc671fOLECev++++3CgoKrOLiYmvLli3WyJEjrYsuuoj13AJ33XWXFRsba23dutUqKyvzDadOnfK1YXsOjh9a12zTwZGZmWlt377dKi4utvbv32/9+te/tjp16mRt2rTJsqy23Z4J6jDasmWLJemsYfbs2ZZlnbml5YknnrCcTqdlt9ut6667zjpw4IDfMqqrq60FCxZYPXv2tKKioqzJkydbpaWlYfg25jrXej516pSVnp5u9e7d24qMjLSSkpKs2bNnn7UOWc/n1tj6lWStXLnS14btOTh+aF2zTQfHnDlzrOTkZKtLly5W7969rfHjx/tC2rLadnvmNZcAABiMc9QAABiMoAYAwGAENQAABiOoAQAwGEENAIDBCGoAAAxGUAMAYDCCGgAAgxHUAHxSUlKUm5sb1hpuv/12TZ06Naw1ACYhqAE0yWazacOGDeEuA+jQCGqgnaitrQ13CQDCgKAGDDVmzBgtWLBAixYtUq9evTRx4kQdPnxYN9xwgy688EI5HA7ddttt+r//+z9fnz/96U8aPHiwoqKiFBcXpwkTJujkyZO+5WVkZPh9xtSpU3X77bc3+vkpKSmSpP/4j/+QzWbzjTfl008/lc1mU1FRkd/0nJwcpaSkyLIsNTQ0aO7cuUpNTVVUVJT69eun559//pzLbexw/NChQ+VyuXzjlZWV+tWvfqU+ffooJiZG48aN0yeffHLO5QLtBUENGGz16tWKiIjQRx99pGeeeUajR4/W0KFDtXv3br333nv617/+penTp0s6867bW2+9VXPmzNGRI0e0detWTZs2TYG+d2fXrl2SpJUrV6qsrMw33pR+/fpp2LBheuONN/ymr1mzRjNnzpTNZtPp06d18cUX680339Thw4f1+OOP69e//rXefPPNgGqUzrwj+MYbb5Tb7da7776rwsJCXXnllRo/fry++uqrgJcLmCIi3AUAaNqll16qpUuXSpIef/xxXXnllcrKyvLN/93vfqfExEQdPXpUVVVVqq+v17Rp05ScnCxJGjx4cMCf3bt3b0lS9+7d5XQ6m9Vn1qxZysvL03/9139Jko4eParCwkL9/ve/lyRFRkbqySef9LVPTU1VQUGB3nzzTd8/HC21ZcsWHThwQOXl5bLb7ZKk3/zmN9qwYYP+9Kc/6Ve/+lVAywVMwR41YLDhw4f7/i4sLNSWLVt04YUX+obLLrtMkvT5559ryJAhGj9+vAYPHqyf/exneuWVV1RRUdGm9c6YMUMlJSXauXOnJOmNN97Q0KFDNWDAAF+bl19+WcOHD1fv3r114YUX6pVXXlFpaWnAn1lYWKiqqirFxcX5rZvi4mJ9/vnnrf5OQLixRw0YrFu3br6/T58+rSlTpmjJkiVntYuPj1fnzp2Vn5+vgoICbdq0ScuXL9fixYv18ccfKzU1VZ06dTrrMHhdXV1Q642Pj9fYsWO1Zs0aXX311frDH/6gefPm+ea/+eabuu+++7Rs2TKNHDlS0dHRevbZZ/Xxxx83ucwfqvv06dOKj4/X1q1bz+rbvXv3Vn8nINwIaqCduPLKK/XnP/9ZKSkpioho/Kdrs9l0zTXX6JprrtHjjz+u5ORkrV+/XosWLVLv3r1VVlbma9vQ0KCDBw9q7NixTX5mZGSkGhoaWlTnrFmz9PDDD+vWW2/V559/rhkzZvjmffjhhxo1apTmz5/vm/ZDe73fr9vr9aq4uNg3fuWVV8rtdisiIuIHL3gD2iMOfQPtxN13362vvvpKt956q/72t7/pH//4hzZt2qQ5c+aooaFBH3/8sbKysrR7926VlpZq3bp1+ve//63+/ftLksaNG6d33nlH77zzjoqKijR//nx9/fXX5/zMlJQUvf/++3K73c0+jD5t2jR5vV7dddddGjt2rC666CLfvEsvvVS7d+/Wxo0bdfToUT322GM/eJHauHHj9Nprr+nDDz/UwYMHNXv2bHXu3Nk3f8KECRo5cqSmTp2qjRs36osvvlBBQYEeffRR7d69u1k1AyYjqIF2IiEhQR999JEaGhp0/fXXa9CgQbr33nsVGxurTp06KSYmRtu3b9cNN9ygvn376tFHH9WyZcs0adIkSdKcOXM0e/Zs/eIXv9Do0aOVmpp6zr1pSVq2bJny8/OVmJioK664oll1xsTEaMqUKfrkk080a9Ysv3l33nmnpk2bpltuuUUjRoyQx+Px27tuTGZmpq677jpNnjxZN9xwg6ZOnaof/ehHvvk2m03vvvuurrvuOs2ZM0d9+/bVjBkz9MUXX8jhcDSrZsBkNivQezcAAEDIsUcNAIDBCGoAzTZw4EC/W6C+O3z/QScAgoND3wCaraSkpMlbuhwOh6Kjo9u4IuD8R1ADAGAwDn0DAGAwghoAAIMR1AAAGIygBgDAYAQ1AAAGI6gBADAYQQ0AgMH+H/Qd8pSTmKeWAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 500x500 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "sns.displot(weight['result_value'])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fb5e64d7",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "**Investigate the standard deviation in the measurements of patients with multiple observations**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "id": "b9b44176",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\meghan_hutch\\AppData\\Local\\Temp\\ipykernel_34512\\2567597466.py:1: FutureWarning: The provided callable <function std at 0x0000023B82357600> is currently using SeriesGroupBy.std. In a future version of pandas, the provided callable will be used directly. To keep current behavior pass the string \"std\" instead.\n",
      "  weight['std'] = weight.groupby('subject_id')['result_value'].transform(np.std, ddof=0)# ddof = 0 indicates that std for patients with one measurement will be displayed as 0 rather than NaN\n"
     ]
    }
   ],
   "source": [
    "weight['std'] = weight.groupby('subject_id')['result_value'].transform(np.std, ddof=0)# ddof = 0 indicates that std for patients with one measurement will be displayed as 0 rather than NaN"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "id": "e5ef71ba",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>subject_id</th>\n",
       "      <th>std</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>177</th>\n",
       "      <td>10019385</td>\n",
       "      <td>58.385000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>644</th>\n",
       "      <td>10021487</td>\n",
       "      <td>22.758934</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2122</th>\n",
       "      <td>10019003</td>\n",
       "      <td>19.732249</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1176</th>\n",
       "      <td>10039708</td>\n",
       "      <td>19.442869</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2705</th>\n",
       "      <td>10040025</td>\n",
       "      <td>16.104849</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      subject_id        std\n",
       "177     10019385  58.385000\n",
       "644     10021487  22.758934\n",
       "2122    10019003  19.732249\n",
       "1176    10039708  19.442869\n",
       "2705    10040025  16.104849"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "weight.sort_values('std', ascending = False)[['subject_id', 'std']].drop_duplicates().head(5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "id": "d7493dee",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>subject_id</th>\n",
       "      <th>chartdate</th>\n",
       "      <th>seq_num</th>\n",
       "      <th>result_name</th>\n",
       "      <th>result_value</th>\n",
       "      <th>std</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>179</th>\n",
       "      <td>10019385</td>\n",
       "      <td>2180-02-15</td>\n",
       "      <td>1</td>\n",
       "      <td>Weight (Lbs)</td>\n",
       "      <td>214.07</td>\n",
       "      <td>58.385</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>177</th>\n",
       "      <td>10019385</td>\n",
       "      <td>2180-03-04</td>\n",
       "      <td>1</td>\n",
       "      <td>Weight (Lbs)</td>\n",
       "      <td>97.30</td>\n",
       "      <td>58.385</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     subject_id   chartdate  seq_num   result_name  result_value     std\n",
       "179    10019385  2180-02-15        1  Weight (Lbs)        214.07  58.385\n",
       "177    10019385  2180-03-04        1  Weight (Lbs)         97.30  58.385"
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "weight[weight['subject_id']==10019385].sort_values('chartdate') # did this patient lose > 110 pounds in < 1 month?"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7b53af5d",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "source": [
    "**Potential Unit Issue?**\n",
    "\n",
    "If 1 kg = 2.20462 lbs:\n",
    "\n",
    "97.30 kg * 2.20462 = 214.5 lbs (this is almost exactly what was measured in the first observation!)\n",
    "\n",
    "It's also possible that the patient was actually 97.3 lbs and the 214.07 was mistakenly measured in kgs (~471 pounds) - this seems a bit less likely, due to our distribution (max value is 296) - but may still be possible."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "id": "850d04c1-6b5e-4326-8380-3d5e60948594",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "count    941.000000\n",
       "mean     173.202763\n",
       "std       44.009522\n",
       "min       88.000000\n",
       "25%      147.300000\n",
       "50%      169.310000\n",
       "75%      185.000000\n",
       "max      296.000000\n",
       "Name: result_value, dtype: float64"
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "weight['result_value'].describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "99b21d88-0fc6-4c0c-baaa-20c812c3bfe4",
   "metadata": {},
   "source": [
    "Review a second patient with high standard deviation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "id": "5f9e92a0",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>subject_id</th>\n",
       "      <th>chartdate</th>\n",
       "      <th>seq_num</th>\n",
       "      <th>result_name</th>\n",
       "      <th>result_value</th>\n",
       "      <th>std</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>644</th>\n",
       "      <td>10021487</td>\n",
       "      <td>2117-01-14</td>\n",
       "      <td>1</td>\n",
       "      <td>Weight (Lbs)</td>\n",
       "      <td>264.0</td>\n",
       "      <td>22.758934</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>643</th>\n",
       "      <td>10021487</td>\n",
       "      <td>2117-01-19</td>\n",
       "      <td>1</td>\n",
       "      <td>Weight (Lbs)</td>\n",
       "      <td>254.0</td>\n",
       "      <td>22.758934</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>641</th>\n",
       "      <td>10021487</td>\n",
       "      <td>2117-02-10</td>\n",
       "      <td>1</td>\n",
       "      <td>Weight (Lbs)</td>\n",
       "      <td>227.0</td>\n",
       "      <td>22.758934</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>638</th>\n",
       "      <td>10021487</td>\n",
       "      <td>2117-05-05</td>\n",
       "      <td>1</td>\n",
       "      <td>Weight (Lbs)</td>\n",
       "      <td>204.0</td>\n",
       "      <td>22.758934</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>635</th>\n",
       "      <td>10021487</td>\n",
       "      <td>2117-06-16</td>\n",
       "      <td>1</td>\n",
       "      <td>Weight (Lbs)</td>\n",
       "      <td>193.0</td>\n",
       "      <td>22.758934</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>656</th>\n",
       "      <td>10021487</td>\n",
       "      <td>2117-07-07</td>\n",
       "      <td>1</td>\n",
       "      <td>Weight (Lbs)</td>\n",
       "      <td>196.6</td>\n",
       "      <td>22.758934</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>636</th>\n",
       "      <td>10021487</td>\n",
       "      <td>2117-08-18</td>\n",
       "      <td>1</td>\n",
       "      <td>Weight (Lbs)</td>\n",
       "      <td>199.0</td>\n",
       "      <td>22.758934</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>637</th>\n",
       "      <td>10021487</td>\n",
       "      <td>2117-08-18</td>\n",
       "      <td>2</td>\n",
       "      <td>Weight (Lbs)</td>\n",
       "      <td>199.0</td>\n",
       "      <td>22.758934</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>639</th>\n",
       "      <td>10021487</td>\n",
       "      <td>2117-09-09</td>\n",
       "      <td>1</td>\n",
       "      <td>Weight (Lbs)</td>\n",
       "      <td>208.0</td>\n",
       "      <td>22.758934</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>657</th>\n",
       "      <td>10021487</td>\n",
       "      <td>2117-12-03</td>\n",
       "      <td>1</td>\n",
       "      <td>Weight (Lbs)</td>\n",
       "      <td>210.1</td>\n",
       "      <td>22.758934</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>640</th>\n",
       "      <td>10021487</td>\n",
       "      <td>2118-01-24</td>\n",
       "      <td>1</td>\n",
       "      <td>Weight (Lbs)</td>\n",
       "      <td>212.0</td>\n",
       "      <td>22.758934</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>642</th>\n",
       "      <td>10021487</td>\n",
       "      <td>2118-08-01</td>\n",
       "      <td>1</td>\n",
       "      <td>Weight (Lbs)</td>\n",
       "      <td>241.0</td>\n",
       "      <td>22.758934</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     subject_id   chartdate  seq_num   result_name  result_value        std\n",
       "644    10021487  2117-01-14        1  Weight (Lbs)         264.0  22.758934\n",
       "643    10021487  2117-01-19        1  Weight (Lbs)         254.0  22.758934\n",
       "641    10021487  2117-02-10        1  Weight (Lbs)         227.0  22.758934\n",
       "638    10021487  2117-05-05        1  Weight (Lbs)         204.0  22.758934\n",
       "635    10021487  2117-06-16        1  Weight (Lbs)         193.0  22.758934\n",
       "656    10021487  2117-07-07        1  Weight (Lbs)         196.6  22.758934\n",
       "636    10021487  2117-08-18        1  Weight (Lbs)         199.0  22.758934\n",
       "637    10021487  2117-08-18        2  Weight (Lbs)         199.0  22.758934\n",
       "639    10021487  2117-09-09        1  Weight (Lbs)         208.0  22.758934\n",
       "657    10021487  2117-12-03        1  Weight (Lbs)         210.1  22.758934\n",
       "640    10021487  2118-01-24        1  Weight (Lbs)         212.0  22.758934\n",
       "642    10021487  2118-08-01        1  Weight (Lbs)         241.0  22.758934"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "weight[weight['subject_id']==10021487].sort_values('chartdate')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0f7ea010",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "### Notes:\n",
    "\n",
    "In the first case, my intuition is that the discrepancy in weight measurements is due to a unit conversions issue\n",
    "\n",
    "In the second case, the patient has many weight measurements over the a 1.5 year period. The measurements themselves don't vary too widely. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5634f4ca",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "**Important Notes:** \n",
    "\n",
    "* The best way to handle these types of inconsistencies will largely depend on the exact variable, question of interest, and domain knowledge. \n",
    "<br> </br>\n",
    "* Seek consultation from mentors and domain experts! \n",
    "<br> </br>\n",
    "* In this case, I'm not considering the date or specific admission. Some patients have multiple admissions. \n",
    "<br> </br>\n",
    "* Additionally, depending how old a patient is, or even gender, height or weight may be expected to change more dramatically overtime. In those cases, we might want to consider these demographic factors before just taking the median."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "86b7e80c",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "#### Missing Data\n",
    "\n",
    "This topic could be it's own class. \n",
    "\n",
    "Rule of thumb: Talk to your domain experts/collaborators!\n",
    "\n",
    "Some common ways to handle missing data:\n",
    "\n",
    "* Remove those patients - need to make sure you have a valid reason for this!\n",
    "* Impute mean or median\n",
    "* Forward or backward filling of the first or most recent value \n",
    "* multiple imputation by chained equations (MICE) \n",
    "\n",
    "In the above cases, I used the median to impute. This may or may not be the best method depending on your use cases. Here, I wanted to keep it simple!"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "79ecd716",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "### Evaluating Unit Conversions\n",
    "\n",
    "Another clinical variable that often needs cleaning are laboratory test measurements.\n",
    "\n",
    "In MIMIC we can use the `labevents` and `d_labitems` tables to work with laboratory data. \n",
    "\n",
    "Importantly, `d_labitems` links to the `labevents` table. \n",
    "\n",
    "The `d_labitems` provides the definitions (or names) or the specific labs via the `itemid`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "id": "56cbe6cb",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>labevent_id</th>\n",
       "      <th>subject_id</th>\n",
       "      <th>hadm_id</th>\n",
       "      <th>specimen_id</th>\n",
       "      <th>itemid</th>\n",
       "      <th>order_provider_id</th>\n",
       "      <th>charttime</th>\n",
       "      <th>storetime</th>\n",
       "      <th>value</th>\n",
       "      <th>valuenum</th>\n",
       "      <th>valueuom</th>\n",
       "      <th>ref_range_lower</th>\n",
       "      <th>ref_range_upper</th>\n",
       "      <th>flag</th>\n",
       "      <th>priority</th>\n",
       "      <th>comments</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>172061</td>\n",
       "      <td>10014354</td>\n",
       "      <td>29600294.0</td>\n",
       "      <td>1808066</td>\n",
       "      <td>51277</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2148-08-16 00:00:00</td>\n",
       "      <td>2148-08-16 01:30:00</td>\n",
       "      <td>15.4</td>\n",
       "      <td>15.40</td>\n",
       "      <td>%</td>\n",
       "      <td>10.5</td>\n",
       "      <td>15.5</td>\n",
       "      <td>NaN</td>\n",
       "      <td>ROUTINE</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>172062</td>\n",
       "      <td>10014354</td>\n",
       "      <td>29600294.0</td>\n",
       "      <td>1808066</td>\n",
       "      <td>51279</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2148-08-16 00:00:00</td>\n",
       "      <td>2148-08-16 01:30:00</td>\n",
       "      <td>3.35</td>\n",
       "      <td>3.35</td>\n",
       "      <td>m/uL</td>\n",
       "      <td>4.6</td>\n",
       "      <td>6.1</td>\n",
       "      <td>abnormal</td>\n",
       "      <td>ROUTINE</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>172068</td>\n",
       "      <td>10014354</td>\n",
       "      <td>29600294.0</td>\n",
       "      <td>1808066</td>\n",
       "      <td>52172</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2148-08-16 00:00:00</td>\n",
       "      <td>2148-08-16 01:30:00</td>\n",
       "      <td>49.7</td>\n",
       "      <td>49.70</td>\n",
       "      <td>fL</td>\n",
       "      <td>35.1</td>\n",
       "      <td>46.3</td>\n",
       "      <td>abnormal</td>\n",
       "      <td>ROUTINE</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>172063</td>\n",
       "      <td>10014354</td>\n",
       "      <td>29600294.0</td>\n",
       "      <td>1808066</td>\n",
       "      <td>51301</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2148-08-16 00:00:00</td>\n",
       "      <td>2148-08-16 01:30:00</td>\n",
       "      <td>20.3</td>\n",
       "      <td>20.30</td>\n",
       "      <td>K/uL</td>\n",
       "      <td>4.0</td>\n",
       "      <td>10.0</td>\n",
       "      <td>abnormal</td>\n",
       "      <td>ROUTINE</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>172050</td>\n",
       "      <td>10014354</td>\n",
       "      <td>29600294.0</td>\n",
       "      <td>1808066</td>\n",
       "      <td>51249</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2148-08-16 00:00:00</td>\n",
       "      <td>2148-08-16 01:30:00</td>\n",
       "      <td>31.1</td>\n",
       "      <td>31.10</td>\n",
       "      <td>g/dL</td>\n",
       "      <td>32.0</td>\n",
       "      <td>37.0</td>\n",
       "      <td>abnormal</td>\n",
       "      <td>ROUTINE</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   labevent_id  subject_id     hadm_id  specimen_id  itemid order_provider_id  \\\n",
       "0       172061    10014354  29600294.0      1808066   51277               NaN   \n",
       "1       172062    10014354  29600294.0      1808066   51279               NaN   \n",
       "2       172068    10014354  29600294.0      1808066   52172               NaN   \n",
       "3       172063    10014354  29600294.0      1808066   51301               NaN   \n",
       "4       172050    10014354  29600294.0      1808066   51249               NaN   \n",
       "\n",
       "             charttime            storetime value  valuenum valueuom  \\\n",
       "0  2148-08-16 00:00:00  2148-08-16 01:30:00  15.4     15.40        %   \n",
       "1  2148-08-16 00:00:00  2148-08-16 01:30:00  3.35      3.35     m/uL   \n",
       "2  2148-08-16 00:00:00  2148-08-16 01:30:00  49.7     49.70       fL   \n",
       "3  2148-08-16 00:00:00  2148-08-16 01:30:00  20.3     20.30     K/uL   \n",
       "4  2148-08-16 00:00:00  2148-08-16 01:30:00  31.1     31.10     g/dL   \n",
       "\n",
       "   ref_range_lower  ref_range_upper      flag priority comments  \n",
       "0             10.5             15.5       NaN  ROUTINE      NaN  \n",
       "1              4.6              6.1  abnormal  ROUTINE      NaN  \n",
       "2             35.1             46.3  abnormal  ROUTINE      NaN  \n",
       "3              4.0             10.0  abnormal  ROUTINE      NaN  \n",
       "4             32.0             37.0  abnormal  ROUTINE      NaN  "
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# let's first look at the labevents table\n",
    "labevents.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "id": "cc8cdc50",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>itemid</th>\n",
       "      <th>label</th>\n",
       "      <th>fluid</th>\n",
       "      <th>category</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>50808</td>\n",
       "      <td>Free Calcium</td>\n",
       "      <td>Blood</td>\n",
       "      <td>Blood Gas</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>50826</td>\n",
       "      <td>Tidal Volume</td>\n",
       "      <td>Blood</td>\n",
       "      <td>Blood Gas</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>50813</td>\n",
       "      <td>Lactate</td>\n",
       "      <td>Blood</td>\n",
       "      <td>Blood Gas</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>52029</td>\n",
       "      <td>% Ionized Calcium</td>\n",
       "      <td>Blood</td>\n",
       "      <td>Blood Gas</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>50801</td>\n",
       "      <td>Alveolar-arterial Gradient</td>\n",
       "      <td>Blood</td>\n",
       "      <td>Blood Gas</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   itemid                       label  fluid   category\n",
       "0   50808                Free Calcium  Blood  Blood Gas\n",
       "1   50826                Tidal Volume  Blood  Blood Gas\n",
       "2   50813                     Lactate  Blood  Blood Gas\n",
       "3   52029           % Ionized Calcium  Blood  Blood Gas\n",
       "4   50801  Alveolar-arterial Gradient  Blood  Blood Gas"
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# let's first look at the d_labitems table\n",
    "d_labitems.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "62d5c5e5",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "### Example Data Cleaning (Laboratory Tests)\n",
    "\n",
    "1. We can merge `d_labitems` and `labevents` by shared identifiers (we'll review this in the next section on merging tables)\n",
    "\n",
    "2. We can search for specific labs of interest via the `label` column\n",
    "\n",
    "3. We can also filter with specific `itemid` numbers if we know which correspond to labs of interest"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9be92774",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "First, let's try and see whether we can find the `itemid` for potassium"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "id": "061fbc9c",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>itemid</th>\n",
       "      <th>label</th>\n",
       "      <th>fluid</th>\n",
       "      <th>category</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>52452</td>\n",
       "      <td>Potassium, Whole Blood</td>\n",
       "      <td>Blood</td>\n",
       "      <td>Blood Gas</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>39</th>\n",
       "      <td>50822</td>\n",
       "      <td>Potassium, Whole Blood</td>\n",
       "      <td>Blood</td>\n",
       "      <td>Blood Gas</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>56</th>\n",
       "      <td>52046</td>\n",
       "      <td>Potassium, Urine</td>\n",
       "      <td>Urine</td>\n",
       "      <td>Blood Gas</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>59</th>\n",
       "      <td>50833</td>\n",
       "      <td>Potassium</td>\n",
       "      <td>Other Body Fluid</td>\n",
       "      <td>Blood Gas</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>218</th>\n",
       "      <td>52610</td>\n",
       "      <td>Potassium</td>\n",
       "      <td>Blood</td>\n",
       "      <td>Chemistry</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>308</th>\n",
       "      <td>50971</td>\n",
       "      <td>Potassium</td>\n",
       "      <td>Blood</td>\n",
       "      <td>Chemistry</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>490</th>\n",
       "      <td>51064</td>\n",
       "      <td>Potassium, Stool</td>\n",
       "      <td>Stool</td>\n",
       "      <td>Chemistry</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>520</th>\n",
       "      <td>51097</td>\n",
       "      <td>Potassium, Urine</td>\n",
       "      <td>Urine</td>\n",
       "      <td>Chemistry</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>636</th>\n",
       "      <td>50847</td>\n",
       "      <td>Potassium, Ascites</td>\n",
       "      <td>Ascites</td>\n",
       "      <td>Chemistry</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>684</th>\n",
       "      <td>51057</td>\n",
       "      <td>Potassium, Pleural</td>\n",
       "      <td>Pleural</td>\n",
       "      <td>Chemistry</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>708</th>\n",
       "      <td>51822</td>\n",
       "      <td>Potassium, Joint Fluid</td>\n",
       "      <td>Joint Fluid</td>\n",
       "      <td>Chemistry</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>788</th>\n",
       "      <td>51041</td>\n",
       "      <td>Potassium, Body Fluid</td>\n",
       "      <td>Other Body Fluid</td>\n",
       "      <td>Chemistry</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>811</th>\n",
       "      <td>51800</td>\n",
       "      <td>Potassium, CSF</td>\n",
       "      <td>Cerebrospinal Fluid</td>\n",
       "      <td>Chemistry</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     itemid                   label                fluid   category\n",
       "20    52452  Potassium, Whole Blood                Blood  Blood Gas\n",
       "39    50822  Potassium, Whole Blood                Blood  Blood Gas\n",
       "56    52046        Potassium, Urine                Urine  Blood Gas\n",
       "59    50833               Potassium     Other Body Fluid  Blood Gas\n",
       "218   52610               Potassium                Blood  Chemistry\n",
       "308   50971               Potassium                Blood  Chemistry\n",
       "490   51064        Potassium, Stool                Stool  Chemistry\n",
       "520   51097        Potassium, Urine                Urine  Chemistry\n",
       "636   50847      Potassium, Ascites              Ascites  Chemistry\n",
       "684   51057      Potassium, Pleural              Pleural  Chemistry\n",
       "708   51822  Potassium, Joint Fluid          Joint Fluid  Chemistry\n",
       "788   51041   Potassium, Body Fluid     Other Body Fluid  Chemistry\n",
       "811   51800          Potassium, CSF  Cerebrospinal Fluid  Chemistry"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "d_labitems[d_labitems['label'].str.contains('potassium', case = False, na=False)]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "id": "1c15eb0d",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "labevent_id          3022\n",
       "subject_id            100\n",
       "hadm_id               250\n",
       "specimen_id          3022\n",
       "itemid                  1\n",
       "order_provider_id      93\n",
       "charttime            3018\n",
       "storetime            3021\n",
       "value                  31\n",
       "valuenum               51\n",
       "valueuom                1\n",
       "ref_range_lower         2\n",
       "ref_range_upper         2\n",
       "flag                    1\n",
       "priority                2\n",
       "comments               11\n",
       "dtype: int64"
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "potassium = labevents[labevents['itemid']==50971]\n",
    "potassium.nunique()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "id": "e1eb659c",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>labevent_id</th>\n",
       "      <th>subject_id</th>\n",
       "      <th>hadm_id</th>\n",
       "      <th>specimen_id</th>\n",
       "      <th>itemid</th>\n",
       "      <th>order_provider_id</th>\n",
       "      <th>charttime</th>\n",
       "      <th>storetime</th>\n",
       "      <th>value</th>\n",
       "      <th>valuenum</th>\n",
       "      <th>valueuom</th>\n",
       "      <th>ref_range_lower</th>\n",
       "      <th>ref_range_upper</th>\n",
       "      <th>flag</th>\n",
       "      <th>priority</th>\n",
       "      <th>comments</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>172088</td>\n",
       "      <td>10014354</td>\n",
       "      <td>29600294.0</td>\n",
       "      <td>43195817</td>\n",
       "      <td>50971</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2148-08-16 00:00:00</td>\n",
       "      <td>2148-08-16 01:47:00</td>\n",
       "      <td>5.2</td>\n",
       "      <td>5.2</td>\n",
       "      <td>mEq/L</td>\n",
       "      <td>3.3</td>\n",
       "      <td>5.1</td>\n",
       "      <td>abnormal</td>\n",
       "      <td>ROUTINE</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>81</th>\n",
       "      <td>172597</td>\n",
       "      <td>10014354</td>\n",
       "      <td>26486158.0</td>\n",
       "      <td>78241917</td>\n",
       "      <td>50971</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2148-08-24 00:00:00</td>\n",
       "      <td>2148-08-24 01:34:00</td>\n",
       "      <td>4.4</td>\n",
       "      <td>4.4</td>\n",
       "      <td>mEq/L</td>\n",
       "      <td>3.3</td>\n",
       "      <td>5.1</td>\n",
       "      <td>NaN</td>\n",
       "      <td>ROUTINE</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>109</th>\n",
       "      <td>408883</td>\n",
       "      <td>10035631</td>\n",
       "      <td>29462354.0</td>\n",
       "      <td>7964528</td>\n",
       "      <td>50971</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2112-09-24 00:00:00</td>\n",
       "      <td>2112-09-24 01:17:00</td>\n",
       "      <td>4.2</td>\n",
       "      <td>4.2</td>\n",
       "      <td>mEq/L</td>\n",
       "      <td>3.3</td>\n",
       "      <td>5.1</td>\n",
       "      <td>NaN</td>\n",
       "      <td>ROUTINE</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>155</th>\n",
       "      <td>336034</td>\n",
       "      <td>10029484</td>\n",
       "      <td>NaN</td>\n",
       "      <td>79775396</td>\n",
       "      <td>50971</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2160-11-08 00:00:00</td>\n",
       "      <td>2160-11-08 00:33:00</td>\n",
       "      <td>4.5</td>\n",
       "      <td>4.5</td>\n",
       "      <td>mEq/L</td>\n",
       "      <td>3.3</td>\n",
       "      <td>5.1</td>\n",
       "      <td>NaN</td>\n",
       "      <td>STAT</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>173</th>\n",
       "      <td>411486</td>\n",
       "      <td>10035631</td>\n",
       "      <td>20385771.0</td>\n",
       "      <td>88649212</td>\n",
       "      <td>50971</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2112-12-16 00:00:00</td>\n",
       "      <td>2112-12-16 02:03:00</td>\n",
       "      <td>3.8</td>\n",
       "      <td>3.8</td>\n",
       "      <td>mEq/L</td>\n",
       "      <td>3.3</td>\n",
       "      <td>5.1</td>\n",
       "      <td>NaN</td>\n",
       "      <td>ROUTINE</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>107595</th>\n",
       "      <td>418158</td>\n",
       "      <td>10035631</td>\n",
       "      <td>29276678.0</td>\n",
       "      <td>2091476</td>\n",
       "      <td>50971</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2116-03-05 18:59:00</td>\n",
       "      <td>2116-03-05 19:39:00</td>\n",
       "      <td>3.7</td>\n",
       "      <td>3.7</td>\n",
       "      <td>mEq/L</td>\n",
       "      <td>3.3</td>\n",
       "      <td>5.1</td>\n",
       "      <td>NaN</td>\n",
       "      <td>STAT</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>107605</th>\n",
       "      <td>408782</td>\n",
       "      <td>10035631</td>\n",
       "      <td>29462354.0</td>\n",
       "      <td>48589180</td>\n",
       "      <td>50971</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2112-09-21 22:59:00</td>\n",
       "      <td>2112-09-22 01:11:00</td>\n",
       "      <td>3.8</td>\n",
       "      <td>3.8</td>\n",
       "      <td>mEq/L</td>\n",
       "      <td>3.3</td>\n",
       "      <td>5.1</td>\n",
       "      <td>NaN</td>\n",
       "      <td>ROUTINE</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>107638</th>\n",
       "      <td>255520</td>\n",
       "      <td>10021487</td>\n",
       "      <td>28998349.0</td>\n",
       "      <td>52829826</td>\n",
       "      <td>50971</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2116-12-22 04:59:00</td>\n",
       "      <td>2116-12-22 06:08:00</td>\n",
       "      <td>4.2</td>\n",
       "      <td>4.2</td>\n",
       "      <td>mEq/L</td>\n",
       "      <td>3.3</td>\n",
       "      <td>5.1</td>\n",
       "      <td>NaN</td>\n",
       "      <td>STAT</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>107663</th>\n",
       "      <td>473242</td>\n",
       "      <td>10040025</td>\n",
       "      <td>27876215.0</td>\n",
       "      <td>29795328</td>\n",
       "      <td>50971</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2147-11-14 04:59:00</td>\n",
       "      <td>2147-11-14 06:18:00</td>\n",
       "      <td>3.9</td>\n",
       "      <td>3.9</td>\n",
       "      <td>mEq/L</td>\n",
       "      <td>3.3</td>\n",
       "      <td>5.1</td>\n",
       "      <td>NaN</td>\n",
       "      <td>STAT</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>107703</th>\n",
       "      <td>565</td>\n",
       "      <td>10000032</td>\n",
       "      <td>25742920.0</td>\n",
       "      <td>6101561</td>\n",
       "      <td>50971</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2180-08-07 00:59:00</td>\n",
       "      <td>2180-08-07 02:16:00</td>\n",
       "      <td>___</td>\n",
       "      <td>5.3</td>\n",
       "      <td>mEq/L</td>\n",
       "      <td>3.3</td>\n",
       "      <td>5.1</td>\n",
       "      <td>abnormal</td>\n",
       "      <td>STAT</td>\n",
       "      <td>VERIFIED BY ALTERNATE METHODOLOGY.</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>3022 rows × 16 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "        labevent_id  subject_id     hadm_id  specimen_id  itemid  \\\n",
       "21           172088    10014354  29600294.0     43195817   50971   \n",
       "81           172597    10014354  26486158.0     78241917   50971   \n",
       "109          408883    10035631  29462354.0      7964528   50971   \n",
       "155          336034    10029484         NaN     79775396   50971   \n",
       "173          411486    10035631  20385771.0     88649212   50971   \n",
       "...             ...         ...         ...          ...     ...   \n",
       "107595       418158    10035631  29276678.0      2091476   50971   \n",
       "107605       408782    10035631  29462354.0     48589180   50971   \n",
       "107638       255520    10021487  28998349.0     52829826   50971   \n",
       "107663       473242    10040025  27876215.0     29795328   50971   \n",
       "107703          565    10000032  25742920.0      6101561   50971   \n",
       "\n",
       "       order_provider_id            charttime            storetime value  \\\n",
       "21                   NaN  2148-08-16 00:00:00  2148-08-16 01:47:00   5.2   \n",
       "81                   NaN  2148-08-24 00:00:00  2148-08-24 01:34:00   4.4   \n",
       "109                  NaN  2112-09-24 00:00:00  2112-09-24 01:17:00   4.2   \n",
       "155                  NaN  2160-11-08 00:00:00  2160-11-08 00:33:00   4.5   \n",
       "173                  NaN  2112-12-16 00:00:00  2112-12-16 02:03:00   3.8   \n",
       "...                  ...                  ...                  ...   ...   \n",
       "107595               NaN  2116-03-05 18:59:00  2116-03-05 19:39:00   3.7   \n",
       "107605               NaN  2112-09-21 22:59:00  2112-09-22 01:11:00   3.8   \n",
       "107638               NaN  2116-12-22 04:59:00  2116-12-22 06:08:00   4.2   \n",
       "107663               NaN  2147-11-14 04:59:00  2147-11-14 06:18:00   3.9   \n",
       "107703               NaN  2180-08-07 00:59:00  2180-08-07 02:16:00   ___   \n",
       "\n",
       "        valuenum valueuom  ref_range_lower  ref_range_upper      flag  \\\n",
       "21           5.2    mEq/L              3.3              5.1  abnormal   \n",
       "81           4.4    mEq/L              3.3              5.1       NaN   \n",
       "109          4.2    mEq/L              3.3              5.1       NaN   \n",
       "155          4.5    mEq/L              3.3              5.1       NaN   \n",
       "173          3.8    mEq/L              3.3              5.1       NaN   \n",
       "...          ...      ...              ...              ...       ...   \n",
       "107595       3.7    mEq/L              3.3              5.1       NaN   \n",
       "107605       3.8    mEq/L              3.3              5.1       NaN   \n",
       "107638       4.2    mEq/L              3.3              5.1       NaN   \n",
       "107663       3.9    mEq/L              3.3              5.1       NaN   \n",
       "107703       5.3    mEq/L              3.3              5.1  abnormal   \n",
       "\n",
       "       priority                            comments  \n",
       "21      ROUTINE                                 NaN  \n",
       "81      ROUTINE                                 NaN  \n",
       "109     ROUTINE                                 NaN  \n",
       "155        STAT                                 NaN  \n",
       "173     ROUTINE                                 NaN  \n",
       "...         ...                                 ...  \n",
       "107595     STAT                                 NaN  \n",
       "107605  ROUTINE                                 NaN  \n",
       "107638     STAT                                 NaN  \n",
       "107663     STAT                                 NaN  \n",
       "107703     STAT  VERIFIED BY ALTERNATE METHODOLOGY.  \n",
       "\n",
       "[3022 rows x 16 columns]"
      ]
     },
     "execution_count": 49,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "potassium"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "id": "0771e36a",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<seaborn.axisgrid.FacetGrid at 0x23b88b5efd0>"
      ]
     },
     "execution_count": 50,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAeoAAAHpCAYAAABN+X+UAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8hTgPZAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAoHUlEQVR4nO3df3RU9Z3/8dcYQggxBEIgPyBAqAGFACpwhPgDSCA0SKylFRH1ID+qXQEJSFGKu2RZGipbgXNIxaYqIIh4divWXdQQfipluw1UJOGHgELFNGk2mp9IJyG53z/8MnWYiZpwk/tJ8nycc89xPnMzeU+tPLkzd+64LMuyBAAAjHSd0wMAAICGEWoAAAxGqAEAMBihBgDAYIQaAACDEWoAAAxGqAEAMBihlmRZliorK8VHygEApiHUkqqqqhQWFqaqqiqnRwEAwAuhBgDAYIQaAACDEWoAAAxGqAEAMBihBgDAYIQaAACDEWoAAAxGqAEAMBihBgDAYIQaAACDEWoAAAxGqAEAMBihBgDAYIQaAACDEWoAAAxGqAEAMBihBgDAYIQaAACDEWoAAAzWwekBgOY0Z95CFZaW+6z3iuiqF7PWtvxAANBIhBptWmFpucKSHvVd35vtwDQA0Hi89A0AgMEINQAABiPUAAAYjFADAGAwQg0AgMEINQAABuPjWWgz/H1m+uSp0xqV5Lvv8YJ8pU6b6bXGZ6sBmIhQo83w95lpd366331rrQCffflsNQAT8dI3AAAGI9QAABiMUAMAYDBCDQCAwQg1AAAGI9QAABiMUAMAYDBCDQCAwQg1AAAG48pkMJq/y4JyqU8A7QmhhtH8XRaUS30CaE946RsAAIMRagAADEaoAQAwGKEGAMBghBoAAINx1jdaneMF+UqdNtNn/eSp0xqV5MBAANCMCDVanVorwOcjW5Lkzk9v+WEAoJnx0jcAAAYj1AAAGIxQAwBgMEINAIDBCDUAAAYj1AAAGIxQAwBgMEdDvWrVKo0cOVKhoaHq2bOn7r33Xn300Ude+1iWpYyMDMXExCg4OFhjx47V8ePHvfZxu92aP3++IiIiFBISonvuuUefffZZSz4VAACahaOhPnDggObOnas//vGPys3N1eXLl5WSkqKLFy969lm9erXWrFmjrKws5eXlKSoqShMmTFBVVZVnn/T0dO3YsUPbt2/XwYMHVV1drcmTJ6uurs6JpwUAgG0cvTLZu+++63V748aN6tmzp44cOaK77rpLlmVp3bp1WrZsmaZMmSJJ2rx5syIjI7Vt2zY99thjqqio0EsvvaQtW7Zo/PjxkqStW7cqNjZWu3fv1sSJE1v8eQEAYBej3qOuqKiQJIWHh0uSzp07p+LiYqWkpHj2CQoK0pgxY3To0CFJ0pEjR1RbW+u1T0xMjBISEjz7XM3tdquystJrAwDARMaE2rIsLVq0SHfccYcSEhIkScXFxZKkyMhIr30jIyM99xUXF6tjx47q1q1bg/tcbdWqVQoLC/NssbGxdj8dAABsYUyo582bp2PHjum1117zuc/lcnndtizLZ+1q37TP0qVLVVFR4dkuXLjQ9MEBAGhGRoR6/vz5euutt7Rv3z717t3bsx4VFSVJPkfGJSUlnqPsqKgo1dTUqKysrMF9rhYUFKQuXbp4bQAAmMjRUFuWpXnz5umNN97Q3r17FRcX53V/XFycoqKilJub61mrqanRgQMHlJiYKEkaPny4AgMDvfYpKipSQUGBZx8AAForR8/6njt3rrZt26bf//73Cg0N9Rw5h4WFKTg4WC6XS+np6crMzFR8fLzi4+OVmZmpzp07a/r06Z59Z8+erSeffFLdu3dXeHi4Fi9erCFDhnjOAgcAoLVyNNQbNmyQJI0dO9ZrfePGjXrkkUckSUuWLNGlS5f0+OOPq6ysTLfddpt27dql0NBQz/5r165Vhw4dNHXqVF26dEnJycnatGmTAgICWuqpoA04XpCv1GkzfdZ7RXTVi1lrHZgIABwOtWVZ37qPy+VSRkaGMjIyGtynU6dOWr9+vdavX2/jdGhvaq0AhSU96rNeuDfbgWkA4CtGnEwGAAD8I9QAABiMUAMAYDBCDQCAwQg1AAAGI9QAABiMUAMAYDBCDQCAwQg1AAAGI9QAABiMUAMAYDBCDQCAwQg1AAAGI9QAABiMUAMAYDBCDQCAwQg1AAAGI9QAABiMUAMAYDBCDQCAwQg1AAAGI9QAABiMUAMAYDBCDQCAwQg1AAAGI9QAABiMUAMAYDBCDQCAwQg1AAAGI9QAABiMUAMAYDBCDQCAwQg1AAAGI9QAABiMUAMAYDBCDQCAwQg1AAAG6+D0AIAkzZm3UIWl5T7rJ0+d1qiklp8HAExBqGGEwtJyhSU96rPuzk9v+WEAwCCEGvgWxwvylTptps96r4iuejFrrQMTAWhPCDXwLWqtAL9H+4V7sx2YBkB7w8lkAAAYjFADAGAwQg0AgMEINQAABiPUAAAYjFADAGAwQg0AgMEINQAABiPUAAAYjFADAGAwQg0AgMEINQAABiPUAAAYjFADAGAwQg0AgMEINQAABiPUAAAYjFADAGAwQg0AgMEINQAABiPUAAAYjFADAGAwQg0AgMEINQAABiPUAAAYjFADAGAwQg0AgMEINQAABiPUAAAYjFADAGAwQg0AgMEINQAABiPUAAAYjFADAGAwQg0AgMEINQAABiPUAAAYjFADAGAwQg0AgMEINQAABnM01O+9957S0tIUExMjl8ulN9980+v+Rx55RC6Xy2sbNWqU1z5ut1vz589XRESEQkJCdM899+izzz5rwWcBAEDzcTTUFy9e1LBhw5SVldXgPt///vdVVFTk2d5++22v+9PT07Vjxw5t375dBw8eVHV1tSZPnqy6urrmHh8AgGbXwclfnpqaqtTU1G/cJygoSFFRUX7vq6io0EsvvaQtW7Zo/PjxkqStW7cqNjZWu3fv1sSJE22fGQCAlmT8e9T79+9Xz549NWDAAP3kJz9RSUmJ574jR46otrZWKSkpnrWYmBglJCTo0KFDDT6m2+1WZWWl1wYAgImMDnVqaqpeffVV7d27V88995zy8vKUlJQkt9stSSouLlbHjh3VrVs3r5+LjIxUcXFxg4+7atUqhYWFebbY2NhmfR4AADSVoy99f5v777/f888JCQkaMWKE+vbtq507d2rKlCkN/pxlWXK5XA3ev3TpUi1atMhzu7KyklgDAIxk9BH11aKjo9W3b1+dOXNGkhQVFaWamhqVlZV57VdSUqLIyMgGHycoKEhdunTx2gAAMFGrCvXnn3+uCxcuKDo6WpI0fPhwBQYGKjc317NPUVGRCgoKlJiY6NSYAADYxtGXvqurq3X27FnP7XPnzuno0aMKDw9XeHi4MjIy9KMf/UjR0dE6f/68fv7znysiIkI//OEPJUlhYWGaPXu2nnzySXXv3l3h4eFavHixhgwZ4jkLHACA1szRUB8+fFjjxo3z3L7yvvGMGTO0YcMG5efn65VXXlF5ebmio6M1btw4vf766woNDfX8zNq1a9WhQwdNnTpVly5dUnJysjZt2qSAgIAWfz4AANjN0VCPHTtWlmU1eH9OTs63PkanTp20fv16rV+/3s7RAAAwQqt6jxoAgPaGUAMAYDBCDQCAwQg1AAAGI9QAABiMUAMAYDBCDQCAwQg1AAAGI9QAABiMUAMAYDBCDQCAwRy91jfQ1syZt1CFpeU+670iuurFrLUtPxCAVo9QAzYqLC1XWNKjvut7sx2YBkBbwEvfAAAYjFADAGAwXvpGi/P3Pu7JU6c1KsmZeQDAZIQaLc7f+7ju/HRnhgEAw/HSNwAABiPUAAAYjFADAGAwQg0AgMEINQAABiPUAAAYjFADAGAwQg0AgMEINQAABiPUAAAYjFADAGAwQg0AgMEINQAABuPbs4AWcLwgX6nTZnqt9Yroqhez1jo0EYDWglADLaDWCvD5as/CvdkOTQOgNeGlbwAADEaoAQAwGKEGAMBghBoAAIMRagAADEaoAQAwWJNC3b9/f33++ec+6+Xl5erfv/81DwUAAL7SpM9Rnz9/XnV1dT7rbrdbhYWF1zwU2oY58xaqsLTcZ/3kqdMaldTy8wBAa9SoUL/11luef87JyVFYWJjndl1dnfbs2aN+/frZNhxat8LScp+LfEiSOz+95YcBgFaqUaG+9957JUkul0szZszwui8wMFD9+vXTc889Z9twAAC0d40KdX19vSQpLi5OeXl5ioiIaJahAADAV5r0HvW5c+fsngMAAPjR5C/l2LNnj/bs2aOSkhLPkfYVL7/88jUPBgAAmhjqf/3Xf9WKFSs0YsQIRUdHy+Vy2T0XAABQE0P9wgsvaNOmTXr44YftngcAAHxNky54UlNTo8TERLtnAQAAV2lSqOfMmaNt27bZPQsAALhKk176/vvf/67s7Gzt3r1bQ4cOVWBgoNf9a9assWU4AADauyaF+tixY7r55pslSQUFBV73cWIZAAD2aVKo9+3bZ/ccAADAD77mEgAAgzXpiHrcuHHf+BL33r17mzwQAAD4hyaF+sr701fU1tbq6NGjKigo8PmyDgAA0HRNCvXatWv9rmdkZKi6uvqaBgIAAP9g63vUDz30ENf5BgDARraG+n/+53/UqVMnOx8SAIB2rUkvfU+ZMsXrtmVZKioq0uHDh/XP//zPtgwGAACaGOqwsDCv29ddd50GDhyoFStWKCUlxZbBAABAE0O9ceNGu+cAAAB+NCnUVxw5ckQnT56Uy+XSoEGDdMstt9g1FwAAUBNDXVJSomnTpmn//v3q2rWrLMtSRUWFxo0bp+3bt6tHjx52zwkAQLvUpLO+58+fr8rKSh0/flxffPGFysrKVFBQoMrKSj3xxBN2zwgAQLvVpCPqd999V7t379ZNN93kWRs0aJB+/etfczIZAAA2atIRdX19vc93UEtSYGCg6uvrr3koAADwlSaFOikpSQsWLNBf//pXz1phYaEWLlyo5ORk24YDAKC9a1Kos7KyVFVVpX79+ul73/uebrjhBsXFxamqqkrr16+3e0YAANqtJr1HHRsbqz//+c/Kzc3VqVOnZFmWBg0apPHjx9s9HwAA7Vqjjqj37t2rQYMGqbKyUpI0YcIEzZ8/X0888YRGjhypwYMH6/3332+WQQEAaI8adUS9bt06/eQnP1GXLl187gsLC9Njjz2mNWvW6M4777RtQKCtOl6Qr9RpM33We0V01YtZ/r9KFkD706hQf/jhh3r22WcbvD8lJUW/+tWvrnkooD2otQIUlvSoz3rh3mwHpgFgqka99P23v/3N78eyrujQoYP+7//+75qHAgAAX2lUqHv16qX8/PwG7z927Jiio6OveSgAAPCVRoV60qRJ+pd/+Rf9/e9/97nv0qVLWr58uSZPnmzbcAAAtHeNeo/6mWee0RtvvKEBAwZo3rx5GjhwoFwul06ePKlf//rXqqur07Jly5prVgAA2p1GhToyMlKHDh3SP/3TP2np0qWyLEuS5HK5NHHiRD3//POKjIxslkEBAGiPGn3Bk759++rtt99WWVmZzp49K8uyFB8fr27dujXHfAAAtGtNujKZJHXr1k0jR460cxYAAHCVJl3rGwAAtAxCDQCAwRwN9Xvvvae0tDTFxMTI5XLpzTff9LrfsixlZGQoJiZGwcHBGjt2rI4fP+61j9vt1vz58xUREaGQkBDdc889+uyzz1rwWQAA0HwcDfXFixc1bNgwZWVl+b1/9erVWrNmjbKyspSXl6eoqChNmDBBVVVVnn3S09O1Y8cObd++XQcPHlR1dbUmT56surq6lnoaAAA0myafTGaH1NRUpaam+r3PsiytW7dOy5Yt05QpUyRJmzdvVmRkpLZt26bHHntMFRUVeumll7RlyxbPV2xu3bpVsbGx2r17tyZOnNhizwUAgOZg7HvU586dU3FxsVJSUjxrQUFBGjNmjA4dOiRJOnLkiGpra732iYmJUUJCgmcff9xutyorK702AABMZGyoi4uLJcnnAiqRkZGe+4qLi9WxY0efz3B/fR9/Vq1apbCwMM8WGxtr8/QAANjD2FBf4XK5vG5bluWzdrVv22fp0qWqqKjwbBcuXLBlVgAA7GZsqKOioiTJ58i4pKTEc5QdFRWlmpoalZWVNbiPP0FBQerSpYvXBgCAiYwNdVxcnKKiopSbm+tZq6mp0YEDB5SYmChJGj58uAIDA732KSoqUkFBgWcfAABaM0fP+q6urtbZs2c9t8+dO6ejR48qPDxcffr0UXp6ujIzMxUfH6/4+HhlZmaqc+fOmj59uiQpLCxMs2fP1pNPPqnu3bsrPDxcixcv1pAhQzxngQMA0Jo5GurDhw9r3LhxntuLFi2SJM2YMUObNm3SkiVLdOnSJT3++OMqKyvTbbfdpl27dik0NNTzM2vXrlWHDh00depUXbp0ScnJydq0aZMCAgJa/PkAAGA3R0M9duxYz1dl+uNyuZSRkaGMjIwG9+nUqZPWr1+v9evXN8OEAAA4y9j3qAEAAKEGAMBohBoAAIMRagAADEaoAQAwGKEGAMBghBoAAIMRagAADEaoAQAwGKEGAMBghBoAAIM5eq1vtB1z5i1UYWm519rJU6c1KsmZeQCgrSDUsEVhabnCkh71WnPnpzszDAC0Ibz0DQCAwQg1AAAGI9QAABiMUAMAYDBCDQCAwQg1AAAG4+NZgGGOF+QrddpMr7VeEV31YtZahyYC4CRCDRim1grw+Ux64d5sh6YB4DRe+gYAwGCEGgAAgxFqAAAMRqgBADAYoQYAwGCEGgAAgxFqAAAMRqgBADAYoQYAwGCEGgAAgxFqAAAMRqgBADAYoQYAwGCEGgAAgxFqAAAMRqgBADBYB6cHAPDtjhfkK3XaTJ/1XhFd9WLWWgcmAtBSCDXQCtRaAQpLetRnvXBvtgPTAGhJhBpoxfwdaXOUDbQthBpoxfwdaXOUDbQtnEwGAIDBCDUAAAYj1AAAGIxQAwBgMEINAIDBCDUAAAYj1AAAGIxQAwBgMEINAIDBuDIZGmXOvIUqLC33WT956rRGJbX8PADQ1hFqNEphabnfL4dw56e3/DAA0A7w0jcAAAYj1AAAGIxQAwBgMEINAIDBCDUAAAYj1AAAGIxQAwBgMEINAIDBCDUAAAYj1AAAGIxQAwBgMEINAIDBCDUAAAYj1AAAGIxQAwBgMEINAIDBCDUAAAYj1AAAGIxQAwBgMEINAIDBCDUAAAYj1AAAGIxQAwBgMEINAIDBCDUAAAYj1AAAGIxQAwBgMEINAIDBCDUAAAYzOtQZGRlyuVxeW1RUlOd+y7KUkZGhmJgYBQcHa+zYsTp+/LiDEwMAYC+jQy1JgwcPVlFRkWfLz8/33Ld69WqtWbNGWVlZysvLU1RUlCZMmKCqqioHJwYAwD7Gh7pDhw6KiorybD169JD01dH0unXrtGzZMk2ZMkUJCQnavHmzvvzyS23bts3hqQEAsIfxoT5z5oxiYmIUFxenadOm6ZNPPpEknTt3TsXFxUpJSfHsGxQUpDFjxujQoUPf+Jhut1uVlZVeGwAAJjI61LfddpteeeUV5eTk6Le//a2Ki4uVmJiozz//XMXFxZKkyMhIr5+JjIz03NeQVatWKSwszLPFxsY223MAAOBaGB3q1NRU/ehHP9KQIUM0fvx47dy5U5K0efNmzz4ul8vrZyzL8lm72tKlS1VRUeHZLly4YP/wAADYwOhQXy0kJERDhgzRmTNnPGd/X330XFJS4nOUfbWgoCB16dLFawMAwEStKtRut1snT55UdHS04uLiFBUVpdzcXM/9NTU1OnDggBITEx2cEgAA+3RweoBvsnjxYqWlpalPnz4qKSnRypUrVVlZqRkzZsjlcik9PV2ZmZmKj49XfHy8MjMz1blzZ02fPt3p0QEAsIXRof7ss8/0wAMPqLS0VD169NCoUaP0xz/+UX379pUkLVmyRJcuXdLjjz+usrIy3Xbbbdq1a5dCQ0MdnhxwzvGCfKVOm+mz3iuiq17MWuvARACuhdGh3r59+zfe73K5lJGRoYyMjJYZCGgFaq0AhSU96rNeuDfbgWkAXKtW9R41AADtDaEGAMBghBoAAIMRagAADEaoAQAwGKEGAMBghBoAAIMZ/TlqAPbxdyEULoICmI9QA+2EvwuhcBEUwHy89A0AgMEINQAABiPUAAAYjFADAGAwQg0AgMEINQAABiPUAAAYjFADAGAwLngCtGP+rlYmccUywCSEGmjH/F2tTOKKZYBJeOkbAACDEWoAAAxGqAEAMBihBgDAYIQaAACDEWoAAAxGqAEAMBifo4bmzFuowtJyn3UuegEAziPUUGFpORe9AABDEWo0yN/lJU+eOq1RSQ4NBADtEKFGg/xdXtKdn+7MMADQTnEyGQAABiPUAAAYjFADAGAwQg0AgMEINQAABiPUAAAYjFADAGAwPkcNwIe/i91wSVnAGYQagA9/F7vhkrKAM3jpGwAAgxFqAAAMRqgBADAYoQYAwGCEGgAAg3HWN4DvxN9HtiQ+tgU0N0IN4Dvx95EtiY9tAc2Nl74BADAYoQYAwGC89A3gmnC5UaB5EWoA14TLjQLNi1C3I3PmLVRhabnP+slTpzUqqeXnAQB8O0LdjhSWlvs9a9edn97ywwAAvhNOJgMAwGCEGgAAgxFqAAAMRqgBADAYoQYAwGCEGgAAgxFqAAAMRqgBADAYoQYAwGCEGgAAgxFqAAAMRqgBADAYoQYAwGCEGgAAg/E1lwBsd7wgX6nTZvqs94roqhez1jowEdB6EWoAtqu1Avx+93nh3mwHpgFaN0LdRs2Zt1CFpeVeaydPndaoJGfmASSOtIGmINRtVGFpuc8RjTs/3ZlhgP+PI22g8TiZDAAAgxFqAAAMxkvfABzn773rxrxv7e+cjMY+BmAqQg3Acf7eu27M+9b+zslo7GMApiLUrYi/o4bzZ06rX/wAn305wxsA2gZC3Yr4O2ooz0/3eyTBGd4A0DZwMhkAAAbjiBqAkey4OEpjTlLz99YSJ6PBBIQagJHsuDhKY05S8/fWEiejwQSEGgCaEUfquFaEGgCaEUfquFZtJtTPP/+8/v3f/11FRUUaPHiw1q1bpzvvvNPpsQDYzN/7zo35OGJD7323x480crTfOrSJUL/++utKT0/X888/r9tvv12/+c1vlJqaqhMnTqhPnz5OjwfARv7ed27MxxEbeu+7PX6kkaP91qFNhHrNmjWaPXu25syZI0lat26dcnJytGHDBq1atarF57nWv6U2dDnE9vg3fqC1aMx/t3ac0d6Yy6baMZu/iyu19NF3S78CYMorDq0+1DU1NTpy5Iiefvppr/WUlBQdOnTI78+43W653W7P7YqKCklSZWWlLTP9pahEXe7y/j/6X97b+J0f39/PS9Klo0+r9tJFr7X6ujqftYbWG7OvHY/RlveVpMu1NT7/Ti/X1jg+W2vb1+TZ/P07bkhj/rt1X5Y6j37Q9zFs+HPC32PYMdsXR5/WoKvWGzOvHa71z1YTf19oaKhcLtc372S1coWFhZYk6w9/+IPX+i9+8QtrwIABfn9m+fLlliQ2NjY2NjZHt4qKim/tXKs/or7i6r+RWJbV4N9Sli5dqkWLFnlu19fX6y9/+YtuvvlmXbhwQV26dGnWWZ1UWVmp2NhYnmcbwfNsW3iebct3eZ6hoaHf+jitPtQREREKCAhQcXGx13pJSYkiIyP9/kxQUJCCgoK81q677qurqXbp0qVN/x/nCp5n28LzbFt4nm3LtT7PVn+t744dO2r48OHKzc31Ws/NzVViYqJDUwEAYI9Wf0QtSYsWLdLDDz+sESNGaPTo0crOztann36qn/70p06PBgDANWkTob7//vv1+eefa8WKFSoqKlJCQoLefvtt9e3b9zs/RlBQkJYvX+7zknhbw/NsW3iebQvPs22x63m6LMuybJoJAADYrNW/Rw0AQFtGqAEAMBihBgDAYIQaAACDtftQr1q1SiNHjlRoaKh69uype++9Vx999JHTY9luw4YNGjp0qOeD96NHj9Y777zj9FjNatWqVXK5XEpPT3d6FNtlZGTI5XJ5bVFRUU6P1SwKCwv10EMPqXv37urcubNuvvlmHTlyxOmxbNWvXz+ff58ul0tz5851ejTbXL58Wc8884zi4uIUHBys/v37a8WKFaqvr3d6NNtVVVUpPT1dffv2VXBwsBITE5WXl9fkx2sTH8+6FgcOHNDcuXM1cuRIXb58WcuWLVNKSopOnDihkJAQp8ezTe/evfXLX/5SN9xwgyRp8+bN+sEPfqAPPvhAgwcPdng6++Xl5Sk7O1tDhw51epRmM3jwYO3evdtzOyAgwMFpmkdZWZluv/12jRs3Tu+884569uypjz/+WF27dnV6NFvl5eWprq7Oc7ugoEATJkzQfffd5+BU9nr22Wf1wgsvaPPmzRo8eLAOHz6smTNnKiwsTAsWLHB6PFvNmTNHBQUF2rJli2JiYrR161aNHz9eJ06cUK9evRr/gNf+tRhtS0lJiSXJOnDggNOjNLtu3bpZL774otNj2K6qqsqKj4+3cnNzrTFjxlgLFixweiTbLV++3Bo2bJjTYzS7p556yrrjjjucHqPFLViwwPre975n1dfXOz2Kbe6++25r1qxZXmtTpkyxHnroIYcmah5ffvmlFRAQYP33f/+31/qwYcOsZcuWNekx2/1L31e78pWX4eHhDk/SfOrq6rR9+3ZdvHhRo0ePdnoc282dO1d33323xo8f7/QozerMmTOKiYlRXFycpk2bpk8++cTpkWz31ltvacSIEbrvvvvUs2dP3XLLLfrtb3/r9FjNqqamRlu3btWsWbO+/esPW5E77rhDe/bs0enTpyVJH374oQ4ePKhJkyY5PJm9Ll++rLq6OnXq1MlrPTg4WAcPHmzag9rxN4i2or6+3kpLS2uzf4M/duyYFRISYgUEBFhhYWHWzp07nR7Jdq+99pqVkJBgXbp0ybIsq80eUb/99tvWf/7nf1rHjh3zvHIQGRlplZaWOj2arYKCgqygoCBr6dKl1p///GfrhRdesDp16mRt3rzZ6dGazeuvv24FBARYhYWFTo9iq/r6euvpp5+2XC6X1aFDB8vlclmZmZlOj9UsRo8ebY0ZM8YqLCy0Ll++bG3ZssVyuVwNfvXytyHUX/P4449bffv2tS5cuOD0KM3C7XZbZ86csfLy8qynn37aioiIsI4fP+70WLb59NNPrZ49e1pHjx71rLXVUF+turraioyMtJ577jmnR7FVYGCgNXr0aK+1+fPnW6NGjXJoouaXkpJiTZ482ekxbPfaa69ZvXv3tl577TXr2LFj1iuvvGKFh4dbmzZtcno02509e9a66667LElWQECANXLkSOvBBx+0brrppiY9HqH+/+bNm2f17t3b+uSTT5wepcUkJydbjz76qNNj2GbHjh2e/zCubJIsl8tlBQQEWJcvX3Z6xGY1fvx466c//anTY9iqT58+1uzZs73Wnn/+eSsmJsahiZrX+fPnreuuu8568803nR7Fdr1797aysrK81v7t3/7NGjhwoEMTNb/q6mrrr3/9q2VZljV16lRr0qRJTXqcdn/Wt2VZmj9/vnbs2KH9+/crLi7O6ZFajGVZcrvdTo9hm+TkZOXn53utzZw5UzfeeKOeeuqpNnlW9BVut1snT57UnXfe6fQotrr99tt9Pi55+vTpRn3hTmuyceNG9ezZU3fffbfTo9juyy+/1HXXeZ8WFRAQ0CY/nnVFSEiIQkJCVFZWppycHK1evbpJj9PuQz137lxt27ZNv//97xUaGqri4mJJUlhYmIKDgx2ezj4///nPlZqaqtjYWFVVVWn79u3av3+/3n33XadHs01oaKgSEhK81kJCQtS9e3ef9dZu8eLFSktLU58+fVRSUqKVK1eqsrJSM2bMcHo0Wy1cuFCJiYnKzMzU1KlT9ac//UnZ2dnKzs52ejTb1dfXa+PGjZoxY4Y6dGh7fzSnpaXpF7/4hfr06aPBgwfrgw8+0Jo1azRr1iynR7NdTk6OLMvSwIEDdfbsWf3sZz/TwIEDNXPmzKY9oG3H+K2UJL/bxo0bnR7NVrNmzbL69u1rdezY0erRo4eVnJxs7dq1y+mxml1bfY/6/vvvt6Kjo63AwEArJibGmjJlSps63+Dr/uu//stKSEiwgoKCrBtvvNHKzs52eqRmkZOTY0myPvroI6dHaRaVlZXWggULrD59+lidOnWy+vfvby1btsxyu91Oj2a7119/3erfv7/VsWNHKyoqypo7d65VXl7e5Mfjay4BADAYn6MGAMBghBoAAIMRagAADEaoAQAwGKEGAMBghBoAAIMRagAADEaoAQAwGKEG2rF+/fpp3bp1To8B4BsQagAADEaoAQAwGKEGWqnf/OY36tWrl8/XBN5zzz2aMWOGPv74Y/3gBz9QZGSkrr/+eo0cOVK7d+9u8PHOnz8vl8ulo0ePetbKy8vlcrm0f/9+z9qJEyc0adIkXX/99YqMjNTDDz+s0tJSz/1jx47VE088oSVLlig8PFxRUVHKyMho1O/Zv3+/XC6XcnJydMsttyg4OFhJSUkqKSnRO++8o5tuukldunTRAw88oC+//LJJ//sBrQWhBlqp++67T6Wlpdq3b59n7cr33j744IOqrq7WpEmTtHv3bn3wwQeaOHGi0tLS9Omnnzb5dxYVFWnMmDG6+eabdfjwYb377rv629/+pqlTp3rtt3nzZoWEhOh///d/tXr1aq1YsUK5ubmN/n0ZGRnKysrSoUOHdOHCBU2dOlXr1q3Ttm3btHPnTuXm5mr9+vVNfj5Aa9D2vvQUaCfCw8P1/e9/X9u2bVNycrIk6T/+4z8UHh6u5ORkBQQEaNiwYZ79V65cqR07duitt97SvHnzmvQ7N2zYoFtvvVWZmZmetZdfflmxsbE6ffq0BgwYIEkaOnSoli9fLkmKj49XVlaW9uzZowkTJjTq961cuVK33367JGn27NlaunSpPv74Y/Xv31+S9OMf/1j79u3TU0891aTnA7QGHFEDrdiDDz6o3/3ud3K73ZKkV199VdOmTVNAQIAuXryoJUuWaNCgQeratauuv/56nTp16pqOqI8cOaJ9+/bp+uuv92w33nijJOnjjz/27Dd06FCvn4uOjlZJSUmjf9/XHycyMlKdO3f2RPrKWlMeF2hNOKIGWrG0tDTV19dr586dGjlypN5//32tWbNGkvSzn/1MOTk5+tWvfqUbbrhBwcHB+vGPf6yamhq/j3XddV/9vf3rX1FfW1vrtU99fb3S0tL07LPP+vx8dHS0558DAwO97nO5XJ730r/L7/H3OC6X6xsfF2irCDXQigUHB2vKlCl69dVXdfbsWQ0YMEDDhw+XJL3//vt65JFH9MMf/lCSVF1drfPnzzf4WD169JD01fvQt9xyiyR5nfAlSbfeeqt+97vfqV+/furQoWl/fHyX3wPgH3jpG2jlHnzwQe3cuVMvv/yyHnroIc/6DTfcoDfeeENHjx7Vhx9+qOnTp3/j0WdwcLBGjRqlX/7ylzpx4oTee+89PfPMM177zJ07V1988YUeeOAB/elPf9Inn3yiXbt2adasWaqrq/tO836X3wPgHwg10MolJSUpPDxcH330kaZPn+5ZX7t2rbp166bExESlpaVp4sSJuvXWW7/xsV5++WXV1tZqxIgRWrBggVauXOl1f0xMjP7whz+orq5OEydOVEJCghYsWKCwsDDPS9rfxbf9HgD/4LK+/kYRAAAwCkfUAAAYjFADAGAwQg0AgMEINQAABiPUAAAYjFADAGAwQg0AgMEINQAABiPUAAAYjFADAGAwQg0AgMH+H8s44pWFHrH8AAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 500x500 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "sns.displot(potassium['valuenum'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "id": "199c0143",
   "metadata": {
    "scrolled": true,
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "valueuom\n",
       "mEq/L       3022\n",
       "Name: count, dtype: int64"
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# let's check the distinct units of potassium\n",
    "potassium[['valueuom']].value_counts() # only unit is Milliequivalents per liter "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4e236f58",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "**Let's evaluate another lab test, one which has more than one unit specification**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "id": "318115e4",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>itemid</th>\n",
       "      <th>label</th>\n",
       "      <th>fluid</th>\n",
       "      <th>category</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>979</th>\n",
       "      <td>51249</td>\n",
       "      <td>MCHC</td>\n",
       "      <td>Blood</td>\n",
       "      <td>Hematology</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     itemid label  fluid    category\n",
       "979   51249  MCHC  Blood  Hematology"
      ]
     },
     "execution_count": 52,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "d_labitems[d_labitems['itemid']==51249]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "id": "e57a9eba",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "valueuom\n",
       "g/dL    1555\n",
       "%       1205\n",
       "Name: count, dtype: int64"
      ]
     },
     "execution_count": 53,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# *Mean Corpuscular Hemoglobin Concentration\n",
    "MCHC = labevents[(labevents['itemid']==51249)]\n",
    "MCHC['valueuom'].value_counts()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "id": "0d1a4a9c",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "lab_perc = MCHC[MCHC['valueuom']=='%']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "id": "e215fdf6",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "lab_gdl = MCHC[MCHC['valueuom']=='g/dL']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "id": "eccde471",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<seaborn.axisgrid.FacetGrid at 0x23b88d3e710>"
      ]
     },
     "execution_count": 56,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAeoAAAHqCAYAAADLbQ06AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8hTgPZAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAySUlEQVR4nO3de3QUZZ7G8aclIQTIhYDppjVclIByVYFBo8tFIAHloqyDCCKOzCyz3IwoKIuOwXUSZMeQEQSHWRUEI3P2CIiiQFAIOplRCEZElJtREMj0MpNO0lySkNT+wdJjC4F06KTfkO/nnDrHfuut6l/XMDxU1Vv12izLsgQAAIx0VbALAAAAVSOoAQAwGEENAIDBCGoAAAxGUAMAYDCCGgAAgxHUAAAYjKAGAMBgBDUAAAYjqCVZlqXi4mLxkjYAgGmCGtTbtm3T8OHD5XQ6ZbPZtHbt2vP6fP311xoxYoSioqIUERGhW2+9VYcOHfKuLy0t1bRp09SqVSs1a9ZMI0aM0A8//OBXHSUlJYqKilJJScnl/iQAAAIqqEF94sQJ9ejRQ4sWLbrg+oMHD+qOO+7QDTfcoK1bt+qLL77QM888oyZNmnj7JCcna82aNVq1apU++eQTeTweDRs2TBUVFXX1MwAAqDU2UyblsNlsWrNmje655x5v25gxYxQaGqoVK1ZccJuioiJdffXVWrFihe6//35J0tGjRxUXF6f3339fSUlJ1fru4uJiRUVFqaioSJGRkZf9WwAACBRj71FXVlZq/fr16tixo5KSkhQbG6s+ffr4XB7Pzc1VeXm5EhMTvW1Op1Ndu3ZVTk5OlfsuLS1VcXGxzwIAgImMDWqXyyWPx6N58+ZpyJAh2rRpk+69916NGjVK2dnZkqSCggI1btxYLVq08NnWbreroKCgyn2npaUpKirKu8TFxdXqbwEAoKaMDerKykpJ0siRI/XYY4/ppptu0lNPPaVhw4bplVdeuei2lmXJZrNVuX727NkqKiryLocPHw5o7QAABIqxQd2qVSuFhISoc+fOPu033nijd9S3w+FQWVmZCgsLffq4XC7Z7fYq9x0WFqbIyEifBQAAExkb1I0bN1bv3r21d+9en/Z9+/apbdu2kqSePXsqNDRUWVlZ3vXHjh3T7t27lZCQUKf1AgBQG0KC+eUej0cHDhzwfs7Pz1deXp5iYmLUpk0bzZw5U/fff7/69u2rAQMGaMOGDXr33Xe1detWSVJUVJQmTpyoxx9/XC1btlRMTIyeeOIJdevWTYMGDQrSrwIAIHCC+njW1q1bNWDAgPPaJ0yYoGXLlkmSXnvtNaWlpemHH35Qp06dNHfuXI0cOdLb9/Tp05o5c6YyMzN16tQpDRw4UIsXL/ZrgBiPZwEATGXMc9TBRFADAExl7D1qAABAUAMAYDSCGgAAgxHUAAAYjKAGAMBgBDUAAAYjqAEAMBhBDQCAwYL6ClEAqA6XyyW32+3XNtHR0YqNja2dgoA6RFADMJrL5dL1HeLlKSn2a7vmEZE6eGA/YY16j6AGYDS32y1PSbH6Tk9X81bOam3jOX5U216aIbfbTVCj3iOoAdQLzVs5FWFvE+wygDrHYDIAAAxGUAMAYDCCGgAAgxHUAAAYjKAGAMBgBDUAAAYjqAEAMBhBDQCAwQhqAAAMRlADAGAwghoAAIMR1AAAGIygBgDAYAQ1AAAGI6gBADAYQQ0AgMEIagAADEZQAwBgMIIaAACDEdQAABiMoAYAwGAENQAABiOoAQAwGEENAIDBCGoAAAxGUAMAYDCCGgAAgxHUAAAYjKAGAMBgBDUAAAYLalBv27ZNw4cPl9PplM1m09q1a6vsO2nSJNlsNmVkZPi0l5aWatq0aWrVqpWaNWumESNG6IcffqjdwgEAqCNBDeoTJ06oR48eWrRo0UX7rV27Vp9++qmcTud565KTk7VmzRqtWrVKn3zyiTwej4YNG6aKioraKhsAgDoTEswvHzp0qIYOHXrRPkeOHNHUqVO1ceNG3X333T7rioqK9Oqrr2rFihUaNGiQJGnlypWKi4vT5s2blZSUVGu1AwBQF4y+R11ZWanx48dr5syZ6tKly3nrc3NzVV5ersTERG+b0+lU165dlZOTU+V+S0tLVVxc7LMAAGAio4P6hRdeUEhIiKZPn37B9QUFBWrcuLFatGjh026321VQUFDlftPS0hQVFeVd4uLiAlo3AACBYmxQ5+bm6ve//72WLVsmm83m17aWZV10m9mzZ6uoqMi7HD58+HLLBQCgVhgb1B9//LFcLpfatGmjkJAQhYSE6Pvvv9fjjz+udu3aSZIcDofKyspUWFjos63L5ZLdbq9y32FhYYqMjPRZAAAwkbFBPX78eO3atUt5eXnexel0aubMmdq4caMkqWfPngoNDVVWVpZ3u2PHjmn37t1KSEgIVukAAARMUEd9ezweHThwwPs5Pz9feXl5iomJUZs2bdSyZUuf/qGhoXI4HOrUqZMkKSoqShMnTtTjjz+uli1bKiYmRk888YS6devmHQUOAEB9FtSg3rFjhwYMGOD9PGPGDEnShAkTtGzZsmrtY8GCBQoJCdHo0aN16tQpDRw4UMuWLVOjRo1qo2QAAOpUUIO6f//+siyr2v2/++6789qaNGmihQsXauHChQGsDAAAMxh7jxoAABDUAAAYjaAGAMBgBDUAAAYjqAEAMBhBDQCAwQhqAAAMRlADAGCwoL7wBEDtcrlccrvd1e4fHR2t2NjY2isIgN8IauAK5XK5dH2HeHlKiqu9TfOISB08sJ+wBgxCUANXKLfbLU9JsfpOT1fzVs5L9vccP6ptL82Q2+0mqAGDENTAFa55K6ci7G2CXQaAGmIwGQAABiOoAQAwGEENAIDBCGoAAAxGUAMAYDCCGgAAgxHUAAAYjKAGAMBgBDUAAAYjqAEAMBhBDQCAwQhqAAAMRlADAGAwghoAAIMR1AAAGIygBgDAYAQ1AAAGI6gBADAYQQ0AgMEIagAADEZQAwBgMIIaAACDEdQAABiMoAYAwGAENQAABgsJdgEAGhaXyyW3213t/vn5+bVXDFAPENQA6ozL5dL1HeLlKSn2e9vy8jO1UBFgPoIaQJ1xu93ylBSr7/R0NW/lrNY2rv152rFinirOENRomAhqAHWueSunIuxtqtXXc/xoLVcDmC2og8m2bdum4cOHy+l0ymazae3atd515eXlevLJJ9WtWzc1a9ZMTqdTDz30kI4e9f0/bWlpqaZNm6ZWrVqpWbNmGjFihH744Yc6/iUAANSOoJ5RnzhxQj169NAvfvEL/eu//qvPupMnT2rnzp165pln1KNHDxUWFio5OVkjRozQjh07vP2Sk5P17rvvatWqVWrZsqUef/xxDRs2TLm5uWrUqFFd/ySgwfFncFhdDwzz5/uio6MVGxtbi9UANRPUoB46dKiGDh16wXVRUVHKysryaVu4cKF+9rOf6dChQ2rTpo2Kior06quvasWKFRo0aJAkaeXKlYqLi9PmzZuVlJRU678BaMhqOjistgeGlXqKJJtNQ4YMqfY2zSMidfDAfsIaxqlX96iLiopks9kUHR0tScrNzVV5ebkSExO9fZxOp7p27aqcnByCGqhl/g4Oq6uBYeWnT0iWpVsnzVPMNe0u2d9z/Ki2vTRDbreboIZx6k1Qnz59Wk899ZTGjh2ryMhISVJBQYEaN26sFi1a+PS12+0qKCiocl+lpaUqLS31fi4u9v9REQD/VN3BYXU9MKxpjKPag9YAU9WLN5OVl5drzJgxqqys1OLFiy/Z37Is2Wy2KtenpaUpKirKu8TFxQWyXAAAAsb4oC4vL9fo0aOVn5+vrKws79m0JDkcDpWVlamwsNBnG5fLJbvdXuU+Z8+eraKiIu9y+PDhWqsfAIDLYXRQnwvp/fv3a/PmzWrZsqXP+p49eyo0NNRn0NmxY8e0e/duJSQkVLnfsLAwRUZG+iwAAJgoqPeoPR6PDhw44P2cn5+vvLw8xcTEyOl06r777tPOnTv13nvvqaKiwnvfOSYmRo0bN1ZUVJQmTpyoxx9/XC1btlRMTIyeeOIJdevWzTsKHACA+iyoQb1jxw4NGDDA+3nGjBmSpAkTJiglJUXr1q2TJN10000+223ZskX9+/eXJC1YsEAhISEaPXq0Tp06pYEDB2rZsmU8Qw3UkD/PHjNhBlD7ghrU/fv3l2VZVa6/2LpzmjRpooULF2rhwoWBLA1ocGry7PE5TJgB1J5683gWgNrl77PHEhNmAHWBoAbgw59nj5kwA6h9Ro/6BgCgoeOMGqgn/Jn8QmKgF3ClIKiBeqCmk19IDPQC6juCGqgH/J38QmKgF3ClIKiBeqS6k19IDPQCrhQMJgMAwGAENQAABiOoAQAwGEENAIDBCGoAAAxGUAMAYDCCGgAAgxHUAAAYjKAGAMBgBDUAAAYjqAEAMBhBDQCAwQhqAAAMRlADAGAwghoAAIMR1AAAGIygBgDAYAQ1AAAGI6gBADAYQQ0AgMEIagAADEZQAwBgMIIaAACDEdQAABiMoAYAwGAENQAABiOoAQAwGEENAIDBCGoAAAxGUAMAYDCCGgAAgxHUAAAYjKAGAMBgBDUAAAYjqAEAMFhQg3rbtm0aPny4nE6nbDab1q5d67PesiylpKTI6XQqPDxc/fv311dffeXTp7S0VNOmTVOrVq3UrFkzjRgxQj/88EMd/goAAGpPUIP6xIkT6tGjhxYtWnTB9fPnz1d6eroWLVqk7du3y+FwaPDgwSopKfH2SU5O1po1a7Rq1Sp98skn8ng8GjZsmCoqKurqZwAAUGtCgvnlQ4cO1dChQy+4zrIsZWRkaM6cORo1apQkafny5bLb7crMzNSkSZNUVFSkV199VStWrNCgQYMkSStXrlRcXJw2b96spKSkOvstAADUBmPvUefn56ugoECJiYnetrCwMPXr1085OTmSpNzcXJWXl/v0cTqd6tq1q7fPhZSWlqq4uNhnAQDARMYGdUFBgSTJbrf7tNvtdu+6goICNW7cWC1atKiyz4WkpaUpKirKu8TFxQW4egAAAsPYoD7HZrP5fLYs67y2n7pUn9mzZ6uoqMi7HD58OCC1AgAQaMYGtcPhkKTzzoxdLpf3LNvhcKisrEyFhYVV9rmQsLAwRUZG+iwAAJjI2KBu3769HA6HsrKyvG1lZWXKzs5WQkKCJKlnz54KDQ316XPs2DHt3r3b2wcAgPosqKO+PR6PDhw44P2cn5+vvLw8xcTEqE2bNkpOTlZqaqri4+MVHx+v1NRUNW3aVGPHjpUkRUVFaeLEiXr88cfVsmVLxcTE6IknnlC3bt28o8ABAKjPghrUO3bs0IABA7yfZ8yYIUmaMGGCli1bplmzZunUqVOaPHmyCgsL1adPH23atEkRERHebRYsWKCQkBCNHj1ap06d0sCBA7Vs2TI1atSozn8PAACBFtSg7t+/vyzLqnK9zWZTSkqKUlJSquzTpEkTLVy4UAsXLqyFCgEACC5j71EDAACCGgAAoxHUAAAYjKAGAMBgBDUAAAYjqAEAMBhBDQCAwQhqAAAMRlADAGAwghoAAIMR1AAAGIygBgDAYAQ1AAAGI6gBADAYQQ0AgMEIagAADEZQAwBgMIIaAACDEdQAABgsJNgFAIAp8vPz/eofHR2t2NjYWqoGOIugBtDglXqKJJtNQ4YM8Wu75hGROnhgP2GNWkVQA2jwyk+fkCxLt06ap5hr2lVrG8/xo9r20gy53W6CGrWKoAaA/9c0xqEIe5tglwH4YDAZAAAGI6gBADAYQQ0AgMFqFNTXXXed/v73v5/X7na7dd111112UQAA4KwaBfV3332nioqK89pLS0t15MiRyy4KAACc5deo73Xr1nn/e+PGjYqKivJ+rqio0Icffqh27doFrDgAABo6v4L6nnvukSTZbDZNmDDBZ11oaKjatWunF198MWDFAQDQ0PkV1JWVlZKk9u3ba/v27WrVqlWtFAUAAM6q0QtP/H0fLgAAqJkav5nsww8/1IcffiiXy+U90z7ntddeu+zCAABADYN67ty5eu6559SrVy+1bt1aNpst0HUBAADVMKhfeeUVLVu2TOPHjw90PQAA4Edq9Bx1WVmZEhISAl0LAAD4iRoF9S9/+UtlZmYGuhYAAPATNbr0ffr0aS1dulSbN29W9+7dFRoa6rM+PT09IMUBANDQ1Siod+3apZtuukmStHv3bp91DCwDACBwahTUW7ZsCXQdAADgApjmEgAAg9XojHrAgAEXvcT90Ucf1bggAADwTzUK6nP3p88pLy9XXl6edu/efd5kHZfjzJkzSklJ0ZtvvqmCggK1bt1aDz/8sJ5++mldddXZiwGWZWnu3LlaunSpCgsL1adPH7388svq0qVLwOoAaoPL5ZLb7a5WX17bCzRcNQrqBQsWXLA9JSVFHo/nsgr6sRdeeEGvvPKKli9fri5dumjHjh36xS9+oaioKD366KOSpPnz5ys9PV3Lli1Tx44d9fzzz2vw4MHau3evIiIiAlYLEEgul0vXd4iXp6TYr+3Ky8/UUkUATFXjd31fyIMPPqif/exn+t3vfheQ/f3lL3/RyJEjdffdd0uS2rVrp7feeks7duyQdPZsOiMjQ3PmzNGoUaMkScuXL5fdbldmZqYmTZoUkDqAQHO73fKUFKvv9HQ1b+W8ZH/X/jztWDFPFWcIaqChCehgsr/85S9q0qRJwPZ3xx136MMPP9S+ffskSV988YU++eQT3XXXXZLOXg4sKChQYmKid5uwsDD169dPOTk5Ve63tLRUxcXFPgsQDM1bORVhb3PJpWmL2GCXCiBIanRGfe7s9RzLsnTs2DHt2LFDzzzzTEAKk6Qnn3xSRUVFuuGGG9SoUSNVVFTot7/9rR544AFJUkFBgSTJbrf7bGe32/X9999Xud+0tDTNnTs3YHUCAFBbahTUUVFRPp+vuuoqderUSc8995zP2e3l+tOf/qSVK1cqMzNTXbp0UV5enpKTk+V0On0Grf10BLplWRcdlT579mzNmDHD+7m4uFhxcXEBqxsAgECpUVC//vrrga7jgmbOnKmnnnpKY8aMkSR169ZN33//vdLS0jRhwgQ5HA5J8o4IP8flcp13lv1jYWFhCgsLq93iAQAIgMu6R52bm6uVK1fqzTff1Oeffx6omrxOnjzpfQzrnEaNGqmyslKS1L59ezkcDmVlZXnXl5WVKTs7m9m9AABXhBqdUbtcLo0ZM0Zbt25VdHS0LMtSUVGRBgwYoFWrVunqq68OSHHDhw/Xb3/7W7Vp00ZdunTR559/rvT0dD3yyCOSzl7yTk5OVmpqquLj4xUfH6/U1FQ1bdpUY8eODUgNAAAEU42Cetq0aSouLtZXX32lG2+8UZK0Z88eTZgwQdOnT9dbb70VkOIWLlyoZ555RpMnT5bL5ZLT6dSkSZP0m9/8xttn1qxZOnXqlCZPnux94cmmTZt4hhoAcEWoUVBv2LBBmzdv9oa0JHXu3Fkvv/xyQAeTRUREKCMjQxkZGVX2sdlsSklJUUpKSsC+FwAAU9ToHnVlZeV5c1BLUmhoqPf+MQAAuHw1Cuo777xTjz76qI4ePeptO3LkiB577DENHDgwYMUBANDQ1SioFy1apJKSErVr107XX3+9OnTooPbt26ukpEQLFy4MdI0AADRYNbpHHRcXp507dyorK0vffPONLMtS586dNWjQoEDXBwBAg+bXGfVHH32kzp07e9+NPXjwYE2bNk3Tp09X79691aVLF3388ce1UigAAA2RX0GdkZGhX/3qV4qMjDxvXVRUlCZNmqT09PSAFQcAQEPnV1B/8cUXGjJkSJXrExMTlZube9lFAQCAs/wK6r/97W8XfCzrnJCQEP3v//7vZRcFAADO8iuor7nmGn355ZdVrt+1a5fP5BgAAODy+BXUd911l37zm9/o9OnT5607deqUnn32WQ0bNixgxQEA0ND59XjW008/rdWrV6tjx46aOnWqOnXqJJvNpq+//lovv/yyKioqNGfOnNqqFQCABsevoLbb7crJydG///u/a/bs2bIsS9LZ920nJSVp8eLFF50HGgAA+MfvF560bdtW77//vgoLC3XgwAFZlqX4+Hi1aNGiNuoDAKBBq9GbySSpRYsW6t27dyBrAQAAP1Gjd30DAIC6QVADAGAwghoAAIMR1AAAGIygBgDAYAQ1AAAGI6gBADAYQQ0AgMEIagAADEZQAwBgMIIaAACDEdQAABiMoAYAwGAENQAABiOoAQAwGEENAIDBCGoAAAxGUAMAYLCQYBcAAPVZfn5+tftGR0crNja2FqvBlYigBoAaKPUUSTabhgwZUu1tmkdE6uCB/YQ1/EJQA0ANlJ8+IVmWbp00TzHXtLtkf8/xo9r20gy53W6CGn4hqAHgMjSNcSjC3ibYZeAKxmAyAAAMRlADAGAwghoAAIMR1AAAGIygBgDAYMYH9ZEjR/Tggw+qZcuWatq0qW666Sbl5uZ611uWpZSUFDmdToWHh6t///766quvglgxAACBY3RQFxYW6vbbb1doaKg++OAD7dmzRy+++KKio6O9febPn6/09HQtWrRI27dvl8Ph0ODBg1VSUhK8wgEACBCjn6N+4YUXFBcXp9dff93b1q5dO+9/W5aljIwMzZkzR6NGjZIkLV++XHa7XZmZmZo0aVJdlwwAQEAZfUa9bt069erVSz//+c8VGxurm2++WX/84x+96/Pz81VQUKDExERvW1hYmPr166ecnJxglAwAQEAZHdTffvutlixZovj4eG3cuFG//vWvNX36dL3xxhuSpIKCAkmS3W732c5ut3vXXUhpaamKi4t9FgAATGT0pe/Kykr16tVLqampkqSbb75ZX331lZYsWaKHHnrI289ms/lsZ1nWeW0/lpaWprlz59ZO0QAABJDRZ9StW7dW586dfdpuvPFGHTp0SJLkcDgk6byzZ5fLdd5Z9o/Nnj1bRUVF3uXw4cMBrhwAgMAwOqhvv/127d2716dt3759atu2rSSpffv2cjgcysrK8q4vKytTdna2EhISqtxvWFiYIiMjfRYAAExk9KXvxx57TAkJCUpNTdXo0aP12WefaenSpVq6dKmks5e8k5OTlZqaqvj4eMXHxys1NVVNmzbV2LFjg1w9AACXz+ig7t27t9asWaPZs2frueeeU/v27ZWRkaFx48Z5+8yaNUunTp3S5MmTVVhYqD59+mjTpk2KiIgIYuUAAASG0UEtScOGDdOwYcOqXG+z2ZSSkqKUlJS6KwoAgDpi9D1qAAAaOoIaAACDEdQAABjM+HvUwOVyuVxyu91+bRMdHa3Y2NjaKQgA/EBQ44rmcrl0fYd4eUr8e01s84hIHTywn7AGEHQENa5obrdbnpJi9Z2eruatnNXaxnP8qLa9NENut5ugBhB0BDUahOatnIqwtwl2GQDgNwaTAQBgMIIaAACDEdQAABiMoAYAwGAMJgOAOpSfn+9Xf57pB0ENAHWg1FMk2WwaMmSIX9vxTD8IagCoA+WnT0iWpVsnzVPMNe2qtQ3P9EMiqAGgTjWNcfBMP/zCYDIAAAzGGTUQAP5O/OHvgCIADRdBDVymmk78IUnl5WdqoSIAVxKCGrhMNZn4w7U/TztWzFPFGYIawMUR1ECA+DPxh+f40VquBsCVgsFkAAAYjKAGAMBgBDUAAAYjqAEAMBhBDQCAwQhqAAAMRlADAGAwghoAAIMR1AAAGIygBgDAYAQ1AAAGI6gBADAYk3IAVajunNHMLQ2gNhHUwE+Ueookm01DhgzxazvmlgZQGwhq4CfKT5+QLEu3TpqnmGvaXbI/c0sDqE0ENVCFpjGOas0vzdzSAGoTg8kAADAYQQ0AgMEIagAADEZQAwBgMIIaAACDEdQAABisXgV1WlqabDabkpOTvW2WZSklJUVOp1Ph4eHq37+/vvrqq+AVCQBAANWboN6+fbuWLl2q7t27+7TPnz9f6enpWrRokbZv3y6Hw6HBgwerpKQkSJUCABA49SKoPR6Pxo0bpz/+8Y9q0aKFt92yLGVkZGjOnDkaNWqUunbtquXLl+vkyZPKzMwMYsUAAARGvQjqKVOm6O6779agQYN82vPz81VQUKDExERvW1hYmPr166ecnJwq91daWqri4mKfBQAAExn/CtFVq1Zp586d2r59+3nrCgoKJEl2u92n3W636/vvv69yn2lpaZo7d25gCwUAoBYYfUZ9+PBhPfroo1q5cqWaNGlSZT+bzebz2bKs89p+bPbs2SoqKvIuhw8fDljNAAAEktFn1Lm5uXK5XOrZs6e3raKiQtu2bdOiRYu0d+9eSWfPrFu3bu3t43K5zjvL/rGwsDCFhYXVXuEAAASI0WfUAwcO1Jdffqm8vDzv0qtXL40bN055eXm67rrr5HA4lJWV5d2mrKxM2dnZSkhICGLlAAAEhtFn1BEREeratatPW7NmzdSyZUtve3JyslJTUxUfH6/4+HilpqaqadOmGjt2bDBKBgAgoIwO6uqYNWuWTp06pcmTJ6uwsFB9+vTRpk2bFBEREezSAAC4bPUuqLdu3erz2WazKSUlRSkpKUGpBwCA2mT0PWoAABo6ghoAAIMR1AAAGIygBgDAYAQ1AAAGI6gBADAYQQ0AgMEIagAADFbvXngCAA1Nfn5+tftGR0crNja2FqtBXSOoAcBQpZ4iyWbTkCFDqr1N84hIHTywn7C+ghDUAGCo8tMnJMvSrZPmKeaadpfs7zl+VNtemiG3201QX0EIagAwXNMYhyLsbYJdBoKEwWQAABiMoAYAwGAENQAABiOoAQAwGEENAIDBCGoAAAxGUAMAYDCCGgAAgxHUAAAYjDeTAcAVxp9JPCSpvLxcoaGh1e7PxB91i6AGgCtETSbxkCTZrpKsymp3Z+KPukVQA8AVwt9JPCTJtT9PO1bMY+IPgxHUAHCF8WcSD8/xo35vg7rFYDIAAAxGUAMAYDCCGgAAgxHUAAAYjKAGAMBgBDUAAAYjqAEAMBhBDQCAwQhqAAAMRlADAGAwghoAAIMR1AAAGIygBgDAYAQ1AAAGI6gBADAY81EjYFwul9xut1/bREdHM/k8AFwEQY2AcLlcur5DvDwlxX5t1zwiUgcP7CesAaAKRgd1WlqaVq9erW+++Ubh4eFKSEjQCy+8oE6dOnn7WJaluXPnaunSpSosLFSfPn308ssvq0uXLkGsvOFxu93ylBSr7/R0NW/lrNY2nuNHte2lGXK73X4FtT9n7vn5+dXeLwCYyOigzs7O1pQpU9S7d2+dOXNGc+bMUWJiovbs2aNmzZpJkubPn6/09HQtW7ZMHTt21PPPP6/Bgwdr7969ioiICPIvaHiat3Iqwt6m1vZf0zP38vIztVQRANQuo4N6w4YNPp9ff/11xcbGKjc3V3379pVlWcrIyNCcOXM0atQoSdLy5ctlt9uVmZmpSZMmBaNs1CJ/z9xd+/O0Y8U8VZwhqAHUT0YH9U8VFRVJkmJiYiSdvaxZUFCgxMREb5+wsDD169dPOTk5VQZ1aWmpSktLvZ+Li/07O0PwVffM3XP8aB1UAwC1p948nmVZlmbMmKE77rhDXbt2lSQVFBRIkux2u09fu93uXXchaWlpioqK8i5xcXG1VzgAAJeh3gT11KlTtWvXLr311lvnrbPZbD6fLcs6r+3HZs+eraKiIu9y+PDhgNcLAEAg1ItL39OmTdO6deu0bds2XXvttd52h8Mh6eyZdevWrb3tLpfrvLPsHwsLC1NYWFjtFQwAQIAYfUZtWZamTp2q1atX66OPPlL79u191rdv314Oh0NZWVnetrKyMmVnZyshIaGuywUAIOCMPqOeMmWKMjMz9c477ygiIsJ73zkqKkrh4eGy2WxKTk5Wamqq4uPjFR8fr9TUVDVt2lRjx44NcvUAAFw+o4N6yZIlkqT+/fv7tL/++ut6+OGHJUmzZs3SqVOnNHnyZO8LTzZt2sQz1ACAK4LRQW1Z1iX72Gw2paSkKCUlpfYLAgCgjhl9jxoAgIaOoAYAwGAENQAABiOoAQAwGEENAIDBjB71jeBi3mcACD6CGhfEvM8AYAaCGhfEvM8AYAaCGhdVF/M++3PZnEvsABoaghpBU+opkmw2DRkyxO9tucQOoKEgqBE05adPSJalWyfNU8w17aq1DZfYATP4e3UrOjpasbGxtVTNlY2gRtA1jXFU6/K6dHmX2AFcvppeCWseEamDB/YT1jVAUAMAqq0mV8I8x49q20sz5Ha7CeoaIKgBAH7z50oYLg9vJgMAwGAENQAABiOoAQAwGEENAIDBCGoAAAxGUAMAYDCCGgAAg/EcNQCgTvDa0ZohqAEAtYrXjl4eghoAUKt47ejlIagBAHWC147WDIPJAAAwGGfUBnC5XHK73X5twyALAGgYCOogc7lcur5DvDwlxX5txyALAGgYCOogc7vd8pQUq+/0dDVv5azWNgyyAICGg6A2RPNWTr8HWfjzTCKXygGgfiKo66GaPJPIpXIAqJ8I6nrI32cSuVQOAPUXQV2P8UwigCsdt/gIagCAgbjF908EdS3w57lof19Sfzn8+a66rAsAfopbfP9EUAdYTZ+LLi8/U0sV1fyF+FLt1gUAl8ItPoI64Px9Ltq1P087VsxTxZnaC8SavBC/LuoCAFwaQV1LqvtctOf40Tqo5ix//mVal3UBAKrGpBwAABiMM2oAwBXD34Gw9eGRLoIaAFDv1XTQbH14pOuKCerFixfrv/7rv3Ts2DF16dJFGRkZ+pd/+ZdglwUAqAM1GTRbXx7puiKC+k9/+pOSk5O1ePFi3X777frDH/6goUOHas+ePWrTpmEP6weAhqQmj3OZ/vazKyKo09PTNXHiRP3yl7+UJGVkZGjjxo1asmSJ0tLSglwdAMBE9eXtZ/U+qMvKypSbm6unnnrKpz0xMVE5OTlBqgoAYLr68vazeh/Ux48fV0VFhex2u0+73W5XQUHBBbcpLS1VaWmp93NRUZEkqbjYv7eJXYjH45EkuY98q/LTJy/Zv+Rvh89+d8F3amSrrNZ3+LvNlfIdptbFd5j1HabWxXeYW9eZstPV+vv6TOlpSWf/ng9EXkhSRESEbDbbxTtZ9dyRI0csSVZOTo5P+/PPP2916tTpgts8++yzliQWFhYWFpagLkVFRZfMuXp/Rt2qVSs1atTovLNnl8t13ln2ObNnz9aMGTO8nysrK/WPf/xDLVu2vPS/bAxSXFysuLg4HT58WJGRkcEup97guPmPY1YzHDf/NbRjFhERcck+9T6oGzdurJ49eyorK0v33nuvtz0rK0sjR4684DZhYWEKCwvzaYuOjq7NMmtVZGRkg/gDHWgcN/9xzGqG4+Y/jtk/1fuglqQZM2Zo/Pjx6tWrl2677TYtXbpUhw4d0q9//etglwYAwGW5IoL6/vvv19///nc999xzOnbsmLp27ar3339fbdu2DXZpAABclisiqCVp8uTJmjx5crDLqFNhYWF69tlnz7uMj4vjuPmPY1YzHDf/cczOZ7Msywp2EQAA4MKY5hIAAIMR1AAAGIygBgDAYAQ1AAAGI6gNl5aWpt69eysiIkKxsbG65557tHfvXp8+Ho9HU6dO1bXXXqvw8HDdeOONWrJkSZAqNsOSJUvUvXt370sTbrvtNn3wwQfe9ZZlKSUlRU6nU+Hh4erfv7+++uqrIFYcfBc7ZuXl5XryySfVrVs3NWvWTE6nUw899JCOHj0a5KqD71J/1n5s0qRJstlsysjIqNsiDVSd4/b1119rxIgRioqKUkREhG699VYdOnQoSBUHD0FtuOzsbE2ZMkV//etflZWVpTNnzigxMVEnTpzw9nnssce0YcMGrVy5Ul9//bUee+wxTZs2Te+8804QKw+ua6+9VvPmzdOOHTu0Y8cO3XnnnRo5cqQ3jOfPn6/09HQtWrRI27dvl8Ph0ODBg1VSUhLkyoPnYsfs5MmT2rlzp5555hnt3LlTq1ev1r59+zRixIhglx10l/qzds7atWv16aefyul0BqlSs1zquB08eFB33HGHbrjhBm3dulVffPGFnnnmGTVp0iTIlQfB5U+LgbrkcrksSVZ2dra3rUuXLtZzzz3n0++WW26xnn766bouz2gtWrSw/vu//9uqrKy0HA6HNW/ePO+606dPW1FRUdYrr7wSxArNc+6YXchnn31mSbK+//77Oq7KfD89bj/88IN1zTXXWLt377batm1rLViwIHjFGezHx+3++++3HnzwwSBXZAbOqOuZc1NyxsTEeNvuuOMOrVu3TkeOHJFlWdqyZYv27dunpKSkYJVplIqKCq1atUonTpzQbbfdpvz8fBUUFCgxMdHbJywsTP369WMO8//302N2IUVFRbLZbPX6PfmBdqHjVllZqfHjx2vmzJnq0qVLkCs000+PW2VlpdavX6+OHTsqKSlJsbGx6tOnj9auXRvsUoMj2P9SQPVVVlZaw4cPt+644w6f9tLSUuuhhx6yJFkhISFW48aNrTfeeCNIVZpj165dVrNmzaxGjRpZUVFR1vr16y3Lsqw///nPliTryJEjPv1/9atfWYmJicEo1RhVHbOfOnXqlNWzZ09r3LhxdVyhmS523FJTU63BgwdblZWVlmVZnFH/SFXH7dixY5Ykq2nTplZ6err1+eefW2lpaZbNZrO2bt0a5Krr3hXzCtGGYOrUqdq1a5c++eQTn/aXXnpJf/3rX7Vu3Tq1bdtW27Zt0+TJk9W6dWsNGjQoSNUGX6dOnZSXlye32623335bEyZMUHZ2tnf9T6c0tSyrXk1zWhuqOmadO3f29ikvL9eYMWNUWVmpxYsXB7Fac1R13E6dOqXf//732rlzZ4P/s3UhVR23c1dpRo4cqccee0ySdNNNNyknJ0evvPKK+vXrF8SqgyDY/1JA9UydOtW69tprrW+//dan/eTJk1ZoaKj13nvv+bRPnDjRSkpKqssSjTdw4EDr3/7t36yDBw9akqydO3f6rB8xYoT10EMPBak6M507ZueUlZVZ99xzj9W9e3fr+PHjQazMbOeO24IFCyybzWY1atTIu0iyrrrqKqtt27bBLtM4545baWmpFRISYv3nf/6nz/pZs2ZZCQkJQaoueLhHbTjLsjR16lStXr1aH330kdq3b++zvry8XOXl5brqKt//KRs1aqTKysq6LNV4lmWptLRU7du3l8PhUFZWlnddWVmZsrOzlZCQEMQKzXPumEln/6yNHj1a+/fv1+bNm9WyZcsgV2euc8dt/Pjx2rVrl/Ly8ryL0+nUzJkztXHjxmCXaZxzx61x48bq3bv3eY+i7tu3r0HOisilb8NNmTJFmZmZeueddxQREaGCggJJUlRUlMLDwxUZGal+/fpp5syZCg8PV9u2bZWdna033nhD6enpQa4+eP7jP/5DQ4cOVVxcnEpKSrRq1Spt3bpVGzZskM1mU3JyslJTUxUfH6/4+HilpqaqadOmGjt2bLBLD5qLHbMzZ87ovvvu086dO/Xee++poqLC+2cxJiZGjRs3DnL1wXOx49ayZcvz/kETGhoqh8OhTp06BaliM1zsuEnSzJkzdf/996tv374aMGCANmzYoHfffVdbt24NbuHBENTzeVySpAsur7/+urfPsWPHrIcffthyOp1WkyZNrE6dOlkvvviid/BKQ/TII49Ybdu2tRo3bmxdffXV1sCBA61NmzZ511dWVlrPPvus5XA4rLCwMKtv377Wl19+GcSKg+9ixyw/P7/KP4tbtmwJbuFBdqk/az/FYLKzqnPcXn31VatDhw5WkyZNrB49elhr164NUrXBxTSXAAAYjHvUAAAYjKAGAMBgBDUAAAYjqAEAMBhBDQCAwQhqAAAMRlADAGAwghpowNq1a6eMjIxglwHgIghqAAAMRlADAGAwghqop/7whz/ommuuOW+WtBEjRmjChAk6ePCgRo4cKbvdrubNm6t3797avHlzlfv77rvvZLPZlJeX521zu92y2Ww+EyHs2bNHd911l5o3by673a7x48fr+PHj3vX9+/fX9OnTNWvWLMXExMjhcCglJcWv79m6datsNps2btyom2++WeHh4brzzjvlcrn0wQcf6MYbb1RkZKQeeOABnTx5skbHD6gvCGqgnvr5z3+u48ePa8uWLd62wsJCbdy4UePGjZPH49Fdd92lzZs36/PPP1dSUpKGDx+uQ4cO1fg7jx07pn79+ummm27Sjh07tGHDBv3tb3/T6NGjffotX75czZo106effqr58+frueee85lWtLpSUlK0aNEi5eTk6PDhwxo9erQyMjKUmZmp9evXKysrSwsXLqzx7wHqA6a5BOqpmJgYDRkyRJmZmRo4cKAk6X/+538UExOjgQMHqlGjRurRo4e3//PPP681a9Zo3bp1mjp1ao2+c8mSJbrllluUmprqbXvttdcUFxenffv2qWPHjpKk7t2769lnn5UkxcfHa9GiRfrwww81ePBgv77v+eef1+233y5JmjhxombPnq2DBw/quuuukyTdd9992rJli5588ska/R6gPuCMGqjHxo0bp7ffflulpaWSpDfffFNjxoxRo0aNdOLECc2aNUudO3dWdHS0mjdvrm+++eayzqhzc3O1ZcsWNW/e3LvccMMNkqSDBw96+3Xv3t1nu9atW8vlcvn9fT/ej91uV9OmTb0hfa6tJvsF6hPOqIF6bPjw4aqsrNT69evVu3dvffzxx0pPT5ckzZw5Uxs3btTvfvc7dejQQeHh4brvvvtUVlZ2wX1dddXZf7f/eObb8vJynz6VlZUaPny4XnjhhfO2b926tfe/Q0NDfdbZbDbvvfTqfM+F9mOz2S66X+BKRVAD9Vh4eLhGjRqlN998UwcOHFDHjh3Vs2dPSdLHH3+shx9+WPfee68kyePx6LvvvqtyX1dffbWks/ehb775ZknyGfAlSbfccovefvtttWvXTiEhNfvrozrfA+CfuPQN1HPjxo3T+vXr9dprr+nBBx/0tnfo0EGrV69WXl6evvjiC40dO/aiZ5/h4eG69dZbNW/ePO3Zs0fbtm3T008/7dNnypQp+sc//qEHHnhAn332mb799ltt2rRJjzzyiCoqKqpVb3W+B8A/EdRAPXfnnXcqJiZGe/fu1dixY73tCxYsUIsWLZSQkKDhw4crKSlJt9xyy0X39dprr6m8vFy9evXSo48+queff95nvdPp1J///GdVVFQoKSlJXbt21aOPPqqoqCjvJe3quNT3APgnm/XjG0UAAMAonFEDAGAwghoAAIMR1AAAGIygBgDAYAQ1AAAGI6gBADAYQQ0AgMEIagAADEZQAwBgMIIaAACDEdQAABiMoAYAwGD/B1GVFnXffAEsAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 500x500 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "sns.displot(lab_gdl['valuenum'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "id": "2e8b71aa",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<seaborn.axisgrid.FacetGrid at 0x23b88e0f250>"
      ]
     },
     "execution_count": 57,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAeoAAAHqCAYAAADLbQ06AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8hTgPZAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAtrklEQVR4nO3df3RU9Z3/8deQQAhJGBOQhIEIQQMIiCJQKrIlioT6A6SsRQVRi7pUREWtSIrWSG2CWEPOhorSVaS1rJw9ostKFYICauOuEEAEBQSjAiGk1jG/CJOYfL5/+M2UMQlkkpnMJ8nzcc49x7n385l539v0vrg/Pw5jjBEAALBSp1AXAAAAGkdQAwBgMYIaAACLEdQAAFiMoAYAwGIENQAAFiOoAQCwGEENAIDFCGpJxhiVlpaKd78AAGxDUEsqKyuT0+lUWVlZqEsBAMAHQQ0AgMUIagAALEZQAwBgMYIaAACLEdQAAFiMoAYAwGIENQAAFiOoAQCwGEENAIDFCGoAACxGUAMAYDGCGgAAixHUAABYjKAGAMBiBDUAABYjqAEAsBhBDQCAxQhqAAAsFh7qAgCgJQoLC+V2u/3qExsbK5fLFaSKgMAiqAG0WYWFhRo4aLAqysv86hcVHaODB/YT1mgTCGoAbZbb7VZFeZnG3b1E0b36NqlPefFRvb9iodxuN0GNNoGgBtDmRffqK6drQKjLAIKCm8kAALAYQQ0AgMUIagAALEZQAwBgMYIaAACLEdQAAFgspEH97rvvavLkyXK5XHI4HHr99de9y6qrq/XII4/ooosuUlRUlFwul2699VYVFhb6fIfH49G9996rnj17KioqSlOmTNHRo0dbeU0AAAiOkAZ1RUWFLr74Yi1fvrzespMnT2rnzp167LHHtHPnTq1bt04HDx7UlClTfNrNnz9fr732ml555RW9//77Ki8v13XXXaeamprWWg0AAIImpC88ufrqq3X11Vc3uMzpdCo3N9dnXk5Ojn70ox/pq6++0nnnnaeSkhK98MIL+vOf/6yrrrpKkvTyyy8rMTFRmzdv1qRJk4K+DgAABFObukZdUlIih8Ohc845R5KUn5+v6upqpaametu4XC4NGzZMeXl5jX6Px+NRaWmpzwQAgI3aTFCfOnVKCxcu1IwZM9S9e3dJUlFRkbp06aLY2FiftvHx8SoqKmr0uzIzM+V0Or1TYmJiUGsHAKC52kRQV1dX66abblJtba2effbZs7Y3xsjhcDS6PC0tTSUlJd7pyJEjgSwXAICAsT6oq6urNX36dBUUFCg3N9d7NC1JCQkJqqqqqjcWbXFxseLj4xv9zoiICHXv3t1nAgDARlYHdV1If/bZZ9q8ebN69Ojhs3zkyJHq3Lmzz01nx48f1969ezV27NjWLhcAgIAL6V3f5eXlOnTokPdzQUGBdu/erbi4OLlcLt1www3auXOn3njjDdXU1HivO8fFxalLly5yOp2644479NBDD6lHjx6Ki4vTr371K1100UXeu8ABAGjLQhrUO3bs0BVXXOH9/OCDD0qSbrvtNqWnp2v9+vWSpEsuucSn35YtW5SSkiJJWrZsmcLDwzV9+nRVVlZqwoQJeumllxQWFtYq6wAAQDCFNKhTUlJkjGl0+ZmW1enatatycnKUk5MTyNIAALCC1deoAQDo6AhqAAAsRlADAGAxghoAAIsR1AAAWIygBgDAYgQ1AAAWI6gBALAYQQ0AgMUIagAALEZQAwBgMYIaAACLEdQAAFiMoAYAwGIENQAAFgvpeNQA0FYUFhbK7Xb71Sc2NlYulytIFaGjIKgB4CwKCws1cNBgVZSX+dUvKjpGBw/sJ6zRIgQ1AJyF2+1WRXmZxt29RNG9+japT3nxUb2/YqHcbjdBjRYhqAGgiaJ79ZXTNSDUZaCD4WYyAAAsRlADAGAxghoAAIsR1AAAWIygBgDAYgQ1AAAWI6gBALAYQQ0AgMUIagAALEZQAwBgMYIaAACLEdQAAFiMoAYAwGIENQAAFiOoAQCwGEENAIDFCGoAACxGUAMAYDGCGgAAi4WHugAACIVDhw4FpS0QaAQ1gA7lVJlbcjg0depUv/tWV1UHviDgLAhqAB1KdWWFZIxGzX5CPRPPb1KfE/vztWvtMn1XUxPk6oD6CGoAHVJUzz5yugY0qW1Z8dEgVwM0jqAGYI3CwkK53e4mt+faMToCghqAFQoLCzVw0GBVlJf53Zdrx2jPCGoAVnC73aooL9O4u5coulffJvXh2jE6AoIagFWie/Xl2jFwGl54AgCAxQhqAAAsRlADAGAxghoAAIsR1AAAWIygBgDAYgQ1AAAWI6gBALBYSIP63Xff1eTJk+VyueRwOPT666/7LDfGKD09XS6XS5GRkUpJSdG+fft82ng8Ht17773q2bOnoqKiNGXKFB09yksQAADtQ0jfTFZRUaGLL75Yv/jFL/Sv//qv9ZYvXbpUWVlZeumllzRw4EA9+eSTmjhxog4cOKCYmBhJ0vz58/U///M/euWVV9SjRw899NBDuu6665Sfn6+wsLDWXiUA/x8DbACBEdKgvvrqq3X11Vc3uMwYo+zsbC1atEjTpk2TJK1evVrx8fFas2aN5syZo5KSEr3wwgv685//rKuuukqS9PLLLysxMVGbN2/WpEmTWm1dAPwTA2wAgWPtu74LCgpUVFSk1NRU77yIiAiNHz9eeXl5mjNnjvLz81VdXe3TxuVyadiwYcrLyyOogRBhgA0gcKwN6qKiIklSfHy8z/z4+Hh9+eWX3jZdunRRbGxsvTZ1/Rvi8Xjk8Xi8n0tLSwNVNoDTMMAG0HLW3/XtcDh8Phtj6s37obO1yczMlNPp9E6JiYkBqRUAgECzNqgTEhIkqd6RcXFxsfcoOyEhQVVVVfVuWDm9TUPS0tJUUlLinY4cORLg6gEACAxrgzopKUkJCQnKzc31zquqqtK2bds0duxYSdLIkSPVuXNnnzbHjx/X3r17vW0aEhERoe7du/tMAADYKKTXqMvLy30eySgoKNDu3bsVFxen8847T/Pnz1dGRoaSk5OVnJysjIwMdevWTTNmzJAkOZ1O3XHHHXrooYfUo0cPxcXF6Ve/+pUuuugi713gAAC0ZSEN6h07duiKK67wfn7wwQclSbfddpteeuklLViwQJWVlZo7d67cbrfGjBmjTZs2eZ+hlqRly5YpPDxc06dPV2VlpSZMmKCXXnqJZ6gBAO1CSIM6JSVFxphGlzscDqWnpys9Pb3RNl27dlVOTo5ycnKCUCEAAKFl7TVqAABAUAMAYDWCGgAAixHUAABYjKAGAMBiBDUAABYjqAEAsBhBDQCAxQhqAAAsZu141ADsUVhYWG+UujM5/R3+AFqGoAZwRoWFhRo4aLAqysv87ltdVR2EioCOhaAGcEZut1sV5WUad/cSRffq26Q+J/bna9faZfqupibI1QHtH0ENoEmie/WV0zWgSW3Lio8GuRqg4+BmMgAALEZQAwBgMYIaAACLEdQAAFiMoAYAwGIENQAAFiOoAQCwGEENAIDFCGoAACxGUAMAYDGCGgAAixHUAABYjKAGAMBiBDUAABYjqAEAsBhBDQCAxQhqAAAsRlADAGAxghoAAIsR1AAAWIygBgDAYgQ1AAAWI6gBALAYQQ0AgMUIagAALEZQAwBgMYIaAACLEdQAAFiMoAYAwGIENQAAFiOoAQCwGEENAIDFCGoAACxGUAMAYDGCGgAAixHUAABYjKAGAMBiBDUAABYjqAEAsBhBDQCAxQhqAAAsRlADAGAxq4P6u+++06OPPqqkpCRFRkZqwIABWrx4sWpra71tjDFKT0+Xy+VSZGSkUlJStG/fvhBWDQBA4Fgd1E899ZSee+45LV++XJ9++qmWLl2qp59+Wjk5Od42S5cuVVZWlpYvX67t27crISFBEydOVFlZWQgrBwAgMKwO6g8++EDXX3+9rr32WvXv31833HCDUlNTtWPHDknfH01nZ2dr0aJFmjZtmoYNG6bVq1fr5MmTWrNmTYirBwCg5awO6nHjxuntt9/WwYMHJUkfffSR3n//fV1zzTWSpIKCAhUVFSk1NdXbJyIiQuPHj1deXl5IagYAIJDCQ13AmTzyyCMqKSnR4MGDFRYWppqaGv3ud7/TzTffLEkqKiqSJMXHx/v0i4+P15dfftno93o8Hnk8Hu/n0tLSIFQPAEDLWX1EvXbtWr388stas2aNdu7cqdWrV+v3v/+9Vq9e7dPO4XD4fDbG1Jt3uszMTDmdTu+UmJgYlPoBAGgpq4P64Ycf1sKFC3XTTTfpoosu0qxZs/TAAw8oMzNTkpSQkCDpn0fWdYqLi+sdZZ8uLS1NJSUl3unIkSPBWwkAAFrA6qA+efKkOnXyLTEsLMz7eFZSUpISEhKUm5vrXV5VVaVt27Zp7NixjX5vRESEunfv7jMBAGAjq69RT548Wb/73e903nnnaejQodq1a5eysrI0e/ZsSd+f8p4/f74yMjKUnJys5ORkZWRkqFu3bpoxY0aIqwcAoOWsDuqcnBw99thjmjt3roqLi+VyuTRnzhz95je/8bZZsGCBKisrNXfuXLndbo0ZM0abNm1STExMCCsHACAwrA7qmJgYZWdnKzs7u9E2DodD6enpSk9Pb7W6AFsUFhbK7Xb71Sc2NlYulytIFQEINKuDGkDjCgsLNXDQYFWU+/cWvqjoGB08sJ+wBtoIghpoo9xutyrKyzTu7iWK7tW3SX3Ki4/q/RUL5Xa7CWqgjSCogTYuuldfOV0DQl0GgCCx+vEsAAA6OoIaAACLEdQAAFiMoAYAwGIENQAAFiOoAQCwGEENAIDFCGoAACxGUAMAYDHeTAYAQXTo0CG/2jNoCn6IoAaAIDhV5pYcDk2dOtWvfgyagh8iqAEgCKorKyRjNGr2E+qZeH6T+jBoChpCUANAEEX17MOgKWgRbiYDAMBiBDUAABYjqAEAsBhBDQCAxQhqAAAsRlADAGAxghoAAIsR1AAAWKxZQT1gwAD94x//qDf/22+/1YABPNgPAECgNOvNZF988YVqamrqzfd4PDp27FiLiwKAjoyBPHA6v4J6/fr13v/euHGjnE6n93NNTY3efvtt9e/fP2DFAUBHwkAeaIhfQV33x+NwOHTbbbf5LOvcubP69++vZ555JmDFAUBHwkAeaIhfQV1bWytJSkpK0vbt29WzZ8+gFAUAHRkDeeB0zbpGXVBQEOg6AABAA5o9zOXbb7+tt99+W8XFxd4j7TovvvhiiwsDAADNDOonnnhCixcv1qhRo9S7d285HI5A1wUAANTMoH7uuef00ksvadasWYGuBwAAnKZZLzypqqrS2LFjA10LAAD4gWYF9Z133qk1a9YEuhYAAPADzTr1ferUKa1cuVKbN2/W8OHD1blzZ5/lWVlZASkOAICOrllBvWfPHl1yySWSpL179/os48YyAAACp1lBvWXLlkDXAQAAGtDs56gBtF3+DPrg7wARAAKrWUF9xRVXnPEU9zvvvNPsggAET3MHfZCk6qrqwBcE4KyaFdR116frVFdXa/fu3dq7d2+9wToA2KM5gz6c2J+vXWuX6bsGhrYFEHzNCuply5Y1OD89PV3l5eUtKghA8Pkz6ENZ8dEgVwPgTJr1HHVjbrnlFt7zDQBAAAU0qD/44AN17do1kF8JAECH1qxT39OmTfP5bIzR8ePHtWPHDj322GMBKQwAADQzqJ1Op8/nTp06adCgQVq8eLFSU1MDUhgAAGhmUK9atSrQdQAAgAa06IUn+fn5+vTTT+VwODRkyBCNGDEiUHUBAAA1M6iLi4t10003aevWrTrnnHNkjFFJSYmuuOIKvfLKKzr33HMDXScAAB1Ss+76vvfee1VaWqp9+/bpm2++kdvt1t69e1VaWqr77rsv0DUCANBhNeuI+q233tLmzZt14YUXeucNGTJEf/jDH7iZDACAAGrWEXVtbW29MaglqXPnzqqtrW1xUQAA4HvNCuorr7xS999/vwoLC73zjh07pgceeEATJkwIWHEAAHR0zQrq5cuXq6ysTP3799f555+vCy64QElJSSorK1NOTk6gawQAoMNq1jXqxMRE7dy5U7m5udq/f7+MMRoyZIiuuuqqQNcHAECH5tcR9TvvvKMhQ4aotLRUkjRx4kTde++9uu+++zR69GgNHTpU7733XkALPHbsmG655Rb16NFD3bp10yWXXKL8/HzvcmOM0tPT5XK5FBkZqZSUFO3bty+gNQAAECp+BXV2drbuuusude/evd4yp9OpOXPmKCsrK2DFud1uXX755ercubPefPNNffLJJ3rmmWd0zjnneNssXbpUWVlZWr58ubZv366EhARNnDhRZWVlAasDAIBQ8SuoP/roI/30pz9tdHlqaqrP0W5LPfXUU0pMTNSqVav0ox/9SP3799eECRN0/vnfD3hvjFF2drYWLVqkadOmadiwYVq9erVOnjypNWvWBKwOAABCxa+gPnHiRIOPZdUJDw/X3//+9xYXVWf9+vUaNWqUfv7zn6tXr14aMWKE/vjHP3qXFxQUqKioyOfZ7YiICI0fP155eXmNfq/H41FpaanPBACAjfwK6j59+ujjjz9udPmePXvUu3fvFhdV5/PPP9eKFSuUnJysjRs36pe//KXuu+8+/elPf5IkFRUVSZLi4+N9+sXHx3uXNSQzM1NOp9M7JSYmBqxmAAACya+gvuaaa/Sb3/xGp06dqressrJSjz/+uK677rqAFVdbW6tLL71UGRkZGjFihObMmaO77rpLK1as8GnncDh8Phtj6s07XVpamkpKSrzTkSNHAlYzAACB5NfjWY8++qjWrVungQMHat68eRo0aJAcDoc+/fRT/eEPf1BNTY0WLVoUsOJ69+6tIUOG+My78MIL9eqrr0qSEhISJH1/ZH36kXxxcXG9o+zTRUREKCIiImB1AgAQLH4FdXx8vPLy8nT33XcrLS1NxhhJ3x/RTpo0Sc8+++wZA9Jfl19+uQ4cOOAz7+DBg+rXr58kKSkpSQkJCcrNzfUOsVlVVaVt27bpqaeeClgdAACEit8vPOnXr5/++te/yu1269ChQzLGKDk5WbGxsQEv7oEHHtDYsWOVkZGh6dOn68MPP9TKlSu1cuVKSd//A2H+/PnKyMhQcnKykpOTlZGRoW7dumnGjBkBrwcAgNbWrDeTSVJsbKxGjx4dyFrqGT16tF577TWlpaVp8eLFSkpKUnZ2tmbOnOlts2DBAlVWVmru3Llyu90aM2aMNm3apJiYmKDWBgBAa2h2ULeW66677ow3qDkcDqWnpys9Pb31igIAoJU0a1AOAADQOghqAAAsRlADAGAxghoAAIsR1AAAWIygBgDAYgQ1AAAWI6gBALAYQQ0AgMUIagAALEZQAwBgMYIaAACLEdQAAFiMoAYAwGIENQAAFiOoAQCwGEENAIDFCGoAACwWHuoCAPxTYWGh3G53k9oeOnQoyNUAsAFBDViisLBQAwcNVkV5mV/9qquqg1QRABsQ1IAl3G63KsrLNO7uJYru1fes7U/sz9eutcv0XU1NK1QHIFQIasAy0b36yukacNZ2ZcVHW6EaAKHGzWQAAFiMoAYAwGIENQAAFuMaNQC0A/4+rhcbGyuXyxWkahBIBDUAtGGnytySw6GpU6f61S8qOkYHD+wnrNsAghoA2rDqygrJGI2a/YR6Jp7fpD7lxUf1/oqFcrvdBHUbQFADQDsQ1bNPkx7rQ9vDzWQAAFiMoAYAwGIENQAAFiOoAQCwGEENAIDFCGoAACzG41lAkBQWFsrtdje5vb9vlgLQMRDUQBAUFhZq4KDBqigv87tvdVV1ECoC0FYR1EAQuN1uVZSXadzdSxTdq2+T+pzYn69da5fpu5qaIFcHoC0hqIEgiu7Vt8lviyorPhrkagC0RdxMBgCAxQhqAAAsRlADAGAxghoAAIsR1AAAWIygBgDAYgQ1AAAWI6gBALAYQQ0AgMUIagAALEZQAwBgMYIaAACLEdQAAFiMoAYAwGIENQAAFmtTQZ2ZmSmHw6H58+d75xljlJ6eLpfLpcjISKWkpGjfvn2hKxIAgABqM0G9fft2rVy5UsOHD/eZv3TpUmVlZWn58uXavn27EhISNHHiRJWVlYWoUgAAAqdNBHV5eblmzpypP/7xj4qNjfXON8YoOztbixYt0rRp0zRs2DCtXr1aJ0+e1Jo1a0JYMQAAgdEmgvqee+7Rtddeq6uuuspnfkFBgYqKipSamuqdFxERofHjxysvL6+1ywQAIODCQ13A2bzyyivauXOntm/fXm9ZUVGRJCk+Pt5nfnx8vL788stGv9Pj8cjj8Xg/l5aWBqhaAAACy+oj6iNHjuj+++/Xyy+/rK5duzbazuFw+Hw2xtSbd7rMzEw5nU7vlJiYGLCaAQAIJKuDOj8/X8XFxRo5cqTCw8MVHh6ubdu26d///d8VHh7uPZKuO7KuU1xcXO8o+3RpaWkqKSnxTkeOHAnqegAA0FxWn/qeMGGCPv74Y595v/jFLzR48GA98sgjGjBggBISEpSbm6sRI0ZIkqqqqrRt2zY99dRTjX5vRESEIiIiglo7AACBYHVQx8TEaNiwYT7zoqKi1KNHD+/8+fPnKyMjQ8nJyUpOTlZGRoa6deumGTNmhKJkAAACyuqgbooFCxaosrJSc+fOldvt1pgxY7Rp0ybFxMSEujQAAFqszQX11q1bfT47HA6lp6crPT09JPUAABBMVt9MBgBAR0dQAwBgMYIaAACLEdQAAFiMoAYAwGIENQAAFiOoAQCwGEENAIDFCGoAACzW5t5MBgAIjEOHDvnVPjY2Vi6XK0jVoDEENQB0MKfK3JLDoalTp/rVLyo6RgcP7CesWxlBDQAdTHVlhWSMRs1+Qj0Tz29Sn/Lio3p/xUK53W6CupUR1ADQQUX17COna0Coy8BZcDMZAAAWI6gBALAYQQ0AgMUIagAALEZQAwBgMYIaAACLEdQAAFiMoAYAwGIENQAAFiOoAQCwGEENAIDFCGoAACxGUAMAYDGCGgAAixHUAABYjKAGAMBiBDUAABYjqAEAsBhBDQCAxQhqAAAsRlADAGAxghoAAIsR1AAAWIygBgDAYgQ1AAAWI6gBALAYQQ0AgMUIagAALEZQAwBgMYIaAACLEdQAAFiMoAYAwGIENQAAFiOoAQCwGEENAIDFCGoAACxGUAMAYDGCGgAAixHUAABYjKAGAMBiBDUAABYjqAEAsJjVQZ2ZmanRo0crJiZGvXr10tSpU3XgwAGfNsYYpaeny+VyKTIyUikpKdq3b1+IKgYAILCsDupt27bpnnvu0f/+7/8qNzdX3333nVJTU1VRUeFts3TpUmVlZWn58uXavn27EhISNHHiRJWVlYWwcgAAAiM81AWcyVtvveXzedWqVerVq5fy8/P1k5/8RMYYZWdna9GiRZo2bZokafXq1YqPj9eaNWs0Z86cUJQNAEDAWH1E/UMlJSWSpLi4OElSQUGBioqKlJqa6m0TERGh8ePHKy8vr9Hv8Xg8Ki0t9ZkAALBRmwlqY4wefPBBjRs3TsOGDZMkFRUVSZLi4+N92sbHx3uXNSQzM1NOp9M7JSYmBq9wAABaoM0E9bx587Rnzx7953/+Z71lDofD57Mxpt6806WlpamkpMQ7HTlyJOD1AgAQCFZfo65z7733av369Xr33XfVt29f7/yEhARJ3x9Z9+7d2zu/uLi43lH26SIiIhQRERG8ggEACBCrj6iNMZo3b57WrVund955R0lJST7Lk5KSlJCQoNzcXO+8qqoqbdu2TWPHjm3tcgEACDirj6jvuecerVmzRv/93/+tmJgY73Vnp9OpyMhIORwOzZ8/XxkZGUpOTlZycrIyMjLUrVs3zZgxI8TVAwDQclYH9YoVKyRJKSkpPvNXrVql22+/XZK0YMECVVZWau7cuXK73RozZow2bdqkmJiYVq4WAIDAszqojTFnbeNwOJSenq709PTgFwQAQCuz+ho1AAAdHUENAIDFCGoAACxGUAMAYDGCGgAAi1l91zcAwC6HDh3yq31sbKxcLleQqukYCGoAwFmdKnNLDoemTp3qV7+o6BgdPLCfsG4BghoAcFbVlRWSMRo1+wn1TDy/SX3Ki4/q/RUL5Xa7CeoWIKgBAE0W1bOPnK4BoS6jQyGo0eEUFhbK7Xb71cfj8fg14pq/1/EAoDEENTqUwsJCDRw0WBXlZf51dHSSTK3fv1ddVe13HwA4HUGNDsXtdquivEzj7l6i6F59z95B0on9+dq1dplf1+bq+nxXU9OScgGAoEbHFN2rb5Ovs5UVH5Xk37W5uj4A0FK88AQAAIsR1AAAWIygBgDAYgQ1AAAWI6gBALAYQQ0AgMUIagAALEZQAwBgMYIaAACLEdQAAFiMoAYAwGIENQAAFmNQDgCAVZozZnxsbKxcLleQKgotghoAYI3mjhkfFR2jgwf2t8uwJqgBANZozpjx5cVH9f6KhXK73QQ1AAD+OnTokN9t/Rkzvr0jqAEAQXGqzC05HJo6darffaurqgNfUBtFUAMAgqK6skIyRqNmP6Geiec3qc+J/fnatXaZvqupCXJ1bQdBDQAIqqiefZp8Grus+GiQq2l7CGq0af4+xuHPtTIAsAFBjTaruY9xSFz/AtB2ENRos5rzGAfXvwC0NQQ12jx/HuPg+heAtoZ3fQMAYDGCGgAAixHUAABYjKAGAMBiBDUAABYjqAEAsBhBDQCAxQhqAAAsRlADAGAx3kwGazDABgDUR1DDCgywAQANI6hhBQbYAICGEdQIiuaexmaADQDwRVAj4DiNDQCBQ1Aj4DiNDQCBQ1AHgb+nfSUpNjZWLpcrSBX9U2vWxmlsAK3J3ydBPB6PIiIi/OrTWvvq0xHUAdbc075R0TE6eGB/UP8AbK4NAJrrVJlbcjg0depU/zo6Okmm1q8uodgfEtQB1pzTvuXFR/X+ioVyu91B/R/f5toAoLmqKyskYzRq9hPqmXh+k/rUXW7zp0+o9oftJqifffZZPf300zp+/LiGDh2q7Oxs/cu//EvI6vHntG9ra05t/pxS4kUkAEIhqmcfvy+3+dMnVNpFUK9du1bz58/Xs88+q8svv1zPP/+8rr76an3yySc677zzQl1em9bsU0riDm4ACIR2EdRZWVm64447dOedd0qSsrOztXHjRq1YsUKZmZkhrq5ta8kpJe7gBoCWa/NBXVVVpfz8fC1cuNBnfmpqqvLy8kJUVfvTnFNKAICWa/NB/fXXX6umpkbx8fE+8+Pj41VUVNRgH4/HI4/H4/1cUlIiSSotLW1xPeXl5ZKkb48e0neek03r8/dCSdKePXu8/YPh888/97u2shNfSZJKCw+rc6em3R1Jn9bpY2td9KFPe+1Tt68uLy8PSF5IUkxMjBwOx5kbmTbu2LFjRpLJy8vzmf/kk0+aQYMGNdjn8ccfN5KYmJiYmJhCOpWUlJw159r8EXXPnj0VFhZW7+i5uLi43lF2nbS0ND344IPez7W1tfrmm2/Uo0ePs//LRt8feScmJurIkSPq3r17y1agjero24D179jrL7ENWP/ArH9MTMxZ27T5oO7SpYtGjhyp3Nxc/exnP/POz83N1fXXX99gn4iIiHpvoznnnHP8/u3u3bt3yD/Q03X0bcD6d+z1l9gGrH/w17/NB7UkPfjgg5o1a5ZGjRqlyy67TCtXrtRXX32lX/7yl6EuDQCAFmkXQX3jjTfqH//4hxYvXqzjx49r2LBh+utf/6p+/fqFujQAAFqkXQS1JM2dO1dz585tld+KiIjQ448/7vfL3NuTjr4NWP+Ovf4S24D1b731dxhjTNB/BQAANEunUBcAAAAaR1ADAGAxghoAAIsR1I3IzMzU6NGjFRMTo169emnq1Kk6cOBAvXaffvqppkyZIqfTqZiYGP34xz/WV199FYKKA68p26C8vFzz5s1T3759FRkZqQsvvFArVqwIUcWBtWLFCg0fPtz7nORll12mN99807vcGKP09HS5XC5FRkYqJSVF+/btC2HFgXembVBdXa1HHnlEF110kaKiouRyuXTrrbeqsLAwxFUHztn+Bk43Z84cORwOZWdnt26RQdSU9W/P+0Dp7NugVfaBLX+JZ/s0adIks2rVKrN3716ze/duc+2115rzzjvPlJeXe9scOnTIxMXFmYcfftjs3LnTHD582LzxxhvmxIkTIaw8cJqyDe68805z/vnnmy1btpiCggLz/PPPm7CwMPP666+HsPLAWL9+vdmwYYM5cOCAOXDggPn1r39tOnfubPbu3WuMMWbJkiUmJibGvPrqq+bjjz82N954o+ndu7cpLS0NceWBc6Zt8O2335qrrrrKrF271uzfv9988MEHZsyYMWbkyJGhLjtgzvY3UOe1114zF198sXG5XGbZsmWhKTYIzrb+7X0faMzZt0Fr7AMJ6iYqLi42ksy2bdu882688UZzyy23hLCq1tXQNhg6dKhZvHixT7tLL73UPProo61dXquIjY01//Ef/2Fqa2tNQkKCWbJkiXfZqVOnjNPpNM8991wIKwy+um3QkA8//NBIMl9++WUrV9V6frj+R48eNX369DF79+41/fr1a1dB3ZDT17+j7QPrnL4NWmMfyKnvJqobYSsuLk7S9+8H37BhgwYOHKhJkyapV69eGjNmjF5//fUQVhlcP9wGkjRu3DitX79ex44dkzFGW7Zs0cGDBzVp0qRQlRkUNTU1euWVV1RRUaHLLrtMBQUFKioqUmpqqrdNRESExo8f326HV/3hNmhISUmJHA5Hs17Ja7uG1r+2tlazZs3Sww8/rKFDh4a4wuD64fp3xH1gQ38DrbIPDFjkt2O1tbVm8uTJZty4cd55x48fN5JMt27dTFZWltm1a5fJzMw0DofDbN26NYTVBkdD28AYYzwej7n11luNJBMeHm66dOli/vSnP4WoysDbs2ePiYqKMmFhYcbpdJoNGzYYY4z529/+ZiSZY8eO+bS/6667TGpqaihKDZrGtsEPVVZWmpEjR5qZM2e2coXBdab1z8jIMBMnTjS1tbXGGNMuj6gbW/+OtA88099Aa+wDCeommDt3runXr585cuSId17d8Jo333yzT9vJkyebm266qbVLDLqGtoExxjz99NNm4MCBZv369eajjz4yOTk5Jjo62uTm5oao0sDyeDzms88+M9u3bzcLFy40PXv2NPv27fMGdWFhoU/7O++800yaNClE1QZHY9vgdFVVVeb66683I0aMaNKwfW1JY+u/Y8cOEx8f7/OPtfYY1I2tf0faB57p/wOtsQ8kqM9i3rx5pm/fvubzzz/3me/xeEx4eLj57W9/6zN/wYIFZuzYsa1ZYtA1tg1OnjxpOnfubN544w2f+XfccUe7C6s6EyZMMP/2b/9mDh8+bCSZnTt3+iyfMmWKufXWW0NUXeuo2wZ1qqqqzNSpU83w4cPN119/HcLKWkfd+i9btsw4HA4TFhbmnSSZTp06mX79+oW6zKCpW/+OtA/8obpt0Fr7QK5RN8IYo3nz5mndunV65513lJSU5LO8S5cuGj16dL3HlQ4ePNhuBgM52zaorq5WdXW1OnXy/TMKCwtTbW1ta5baaowx8ng8SkpKUkJCgnJzc73LqqqqtG3bNo0dOzaEFQZf3TaQvv8bmD59uj777DNt3rxZPXr0CHF1wVe3/rNmzdKePXu0e/du7+RyufTwww9r48aNoS4zaOrWvyPsAxtTtw1abR8YsMhvZ+6++27jdDrN1q1bzfHjx73TyZMnvW3WrVtnOnfubFauXGk+++wzk5OTY8LCwsx7770XwsoDpynbYPz48Wbo0KFmy5Yt5vPPPzerVq0yXbt2Nc8++2wIKw+MtLQ08+6775qCggKzZ88e8+tf/9p06tTJbNq0yRjz/eNZTqfTrFu3znz88cfm5ptvbnePZ51pG1RXV5spU6aYvn37mt27d/v8jXg8nlCXHhBn+xv4ofZ26vts69/e94HGnH0btMY+kKBuhKQGp1WrVvm0e+GFF8wFF1xgunbtai6++OJ28fxwnaZsg+PHj5vbb7/duFwu07VrVzNo0CDzzDPPeG+uactmz55t+vXrZ7p06WLOPfdcM2HCBJ8ddG1trXn88cdNQkKCiYiIMD/5yU/Mxx9/HMKKA+9M26CgoKDRv5EtW7aEtvAAOdvfwA+1t6Buyvq3532gMWffBq2xD2T0LAAALMY1agAALEZQAwBgMYIaAACLEdQAAFiMoAYAwGIENQAAFiOoAQCwGEENAIDFCGqgA+vfv7+ys7NDXQaAMyCoAQCwGEENAIDFCGqgjXr++efVp0+fesPpTZkyRbfddpsOHz6s66+/XvHx8YqOjtbo0aO1efPmRr/viy++kMPh0O7du73zvv32WzkcDm3dutU775NPPtE111yj6OhoxcfHa9asWfr666+9y1NSUnTfffdpwYIFiouLU0JCgtLT0/36na1bt8rhcGjjxo0aMWKEIiMjdeWVV6q4uFhvvvmmLrzwQnXv3l0333yzTp482aztB7QVBDXQRv385z/X119/rS1btnjnud1ubdy4UTNnzlR5ebmuueYabd68Wbt27dKkSZM0efJkffXVV83+zePHj2v8+PG65JJLtGPHDr311ls6ceKEpk+f7tNu9erVioqK0v/93/9p6dKlWrx4sc/Y3U2Vnp6u5cuXKy8vT0eOHNH06dOVnZ2tNWvWaMOGDcrNzVVOTk6z1wdoC8JDXQCA5omLi9NPf/pTrVmzRhMmTJAk/dd//Zfi4uI0YcIEhYWF6eKLL/a2f/LJJ/Xaa69p/fr1mjdvXrN+c8WKFbr00kuVkZHhnffiiy8qMTFRBw8e1MCBAyVJw4cP1+OPPy5JSk5O1vLly/X2229r4sSJfv3ek08+qcsvv1ySdMcddygtLU2HDx/WgAEDJEk33HCDtmzZokceeaRZ6wO0BRxRA23YzJkz9eqrr8rj8UiS/vKXv+imm25SWFiYKioqtGDBAg0ZMkTnnHOOoqOjtX///hYdUefn52vLli2Kjo72ToMHD5YkHT582Ntu+PDhPv169+6t4uJiv3/v9O+Jj49Xt27dvCFdN6853wu0JRxRA23Y5MmTVVtbqw0bNmj06NF67733lJWVJUl6+OGHtXHjRv3+97/XBRdcoMjISN1www2qqqpq8Ls6dfr+3+2nD1FfXV3t06a2tlaTJ0/WU089Va9/7969vf/duXNnn2UOh8N7Lb0pv9PQ9zgcjjN+L9BeEdRAGxYZGalp06bpL3/5iw4dOqSBAwdq5MiRkqT33ntPt99+u372s59JksrLy/XFF180+l3nnnuupO+vQ48YMUKSfG74kqRLL71Ur776qvr376/w8ObtPpryOwD+iVPfQBs3c+ZMbdiwQS+++KJuueUW7/wLLrhA69at0+7du/XRRx9pxowZZzz6jIyM1I9//GMtWbJEn3zyid599109+uijPm3uueceffPNN7r55pv14Ycf6vPPP9emTZs0e/Zs1dTUNKnepvwOgH8iqIE27sorr1RcXJwOHDigGTNmeOcvW7ZMsbGxGjt2rCZPnqxJkybp0ksvPeN3vfjii6qurtaoUaN0//3368knn/RZ7nK59Le//U01NTWaNGmShg0bpvvvv19Op9N7SrspzvY7AP7JYU6/UAQAAKzCETUAABYjqAEAsBhBDQCAxQhqAAAsRlADAGAxghoAAIsR1AAAWIygBgDAYgQ1AAAWI6gBALAYQQ0AgMUIagAALPb/AE3eLUTCAgSXAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 500x500 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "sns.displot(lab_perc['valuenum'])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bb6564f7-c0a9-4a1d-947f-1a285a074730",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "### Cleaning EHR Data - Common pitfalls :\n",
    "\n",
    "- missing data\n",
    "- inaccurate data\n",
    "- data inconsistencies\n",
    "- misunderstanding data and where/how it was generated"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "59e44546-16d6-45bf-9f5e-cc985349d88d",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "## Working with large EHR datasets\n",
    "\n",
    "#### Merging tables \n",
    "\n",
    "Tables can be merged (e.g. linked) together through shared identifiers. The MIMIC official [documentation](https://mimic.mit.edu/docs/iv/) is a great reference for learning more about the associations between tables"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6a735950-9ed8-4cf2-9686-91a2ff1fd327",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "First, let's take a look at the diagnosis data that we have for patients"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "id": "df6234c1-2e9d-4975-b5e6-c2108cbf347f",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>subject_id</th>\n",
       "      <th>hadm_id</th>\n",
       "      <th>seq_num</th>\n",
       "      <th>icd_code</th>\n",
       "      <th>icd_version</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>10035185</td>\n",
       "      <td>22580999</td>\n",
       "      <td>3</td>\n",
       "      <td>4139</td>\n",
       "      <td>9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>10035185</td>\n",
       "      <td>22580999</td>\n",
       "      <td>10</td>\n",
       "      <td>V707</td>\n",
       "      <td>9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>10035185</td>\n",
       "      <td>22580999</td>\n",
       "      <td>1</td>\n",
       "      <td>41401</td>\n",
       "      <td>9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>10035185</td>\n",
       "      <td>22580999</td>\n",
       "      <td>9</td>\n",
       "      <td>3899</td>\n",
       "      <td>9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>10035185</td>\n",
       "      <td>22580999</td>\n",
       "      <td>11</td>\n",
       "      <td>V8532</td>\n",
       "      <td>9</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   subject_id   hadm_id  seq_num icd_code  icd_version\n",
       "0    10035185  22580999        3     4139            9\n",
       "1    10035185  22580999       10     V707            9\n",
       "2    10035185  22580999        1    41401            9\n",
       "3    10035185  22580999        9     3899            9\n",
       "4    10035185  22580999       11    V8532            9"
      ]
     },
     "execution_count": 58,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "diagnoses.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "750f3f43-1861-49b4-94c4-45f471f38366",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "source": [
    "In order to obtain more detailed information about each ICD code, we can use the the `d_icd_diagnoses` which contains a longer description of each icd code."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "id": "8a6b2f8e-19a7-4af2-8a06-760e099db849",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>icd_code</th>\n",
       "      <th>icd_version</th>\n",
       "      <th>long_title</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0090</td>\n",
       "      <td>9</td>\n",
       "      <td>Infectious colitis, enteritis, and gastroenter...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>01160</td>\n",
       "      <td>9</td>\n",
       "      <td>Tuberculous pneumonia [any form], unspecified</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>01186</td>\n",
       "      <td>9</td>\n",
       "      <td>Other specified pulmonary tuberculosis, tuberc...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>01200</td>\n",
       "      <td>9</td>\n",
       "      <td>Tuberculous pleurisy, unspecified</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>01236</td>\n",
       "      <td>9</td>\n",
       "      <td>Tuberculous laryngitis, tubercle bacilli not f...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>109770</th>\n",
       "      <td>Z88</td>\n",
       "      <td>10</td>\n",
       "      <td>Allergy status to drugs, medicaments and biolo...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>109771</th>\n",
       "      <td>Z89012</td>\n",
       "      <td>10</td>\n",
       "      <td>Acquired absence of left thumb</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>109772</th>\n",
       "      <td>Z90410</td>\n",
       "      <td>10</td>\n",
       "      <td>Acquired total absence of pancreas</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>109773</th>\n",
       "      <td>Z948</td>\n",
       "      <td>10</td>\n",
       "      <td>Other transplanted organ and tissue status</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>109774</th>\n",
       "      <td>Z96</td>\n",
       "      <td>10</td>\n",
       "      <td>Presence of other functional implants</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>109775 rows × 3 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "       icd_code  icd_version  \\\n",
       "0          0090            9   \n",
       "1         01160            9   \n",
       "2         01186            9   \n",
       "3         01200            9   \n",
       "4         01236            9   \n",
       "...         ...          ...   \n",
       "109770      Z88           10   \n",
       "109771   Z89012           10   \n",
       "109772   Z90410           10   \n",
       "109773     Z948           10   \n",
       "109774      Z96           10   \n",
       "\n",
       "                                               long_title  \n",
       "0       Infectious colitis, enteritis, and gastroenter...  \n",
       "1           Tuberculous pneumonia [any form], unspecified  \n",
       "2       Other specified pulmonary tuberculosis, tuberc...  \n",
       "3                       Tuberculous pleurisy, unspecified  \n",
       "4       Tuberculous laryngitis, tubercle bacilli not f...  \n",
       "...                                                   ...  \n",
       "109770  Allergy status to drugs, medicaments and biolo...  \n",
       "109771                     Acquired absence of left thumb  \n",
       "109772                 Acquired total absence of pancreas  \n",
       "109773         Other transplanted organ and tissue status  \n",
       "109774              Presence of other functional implants  \n",
       "\n",
       "[109775 rows x 3 columns]"
      ]
     },
     "execution_count": 59,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "d_icd_diagnoses"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9025bcb3-68b3-4a90-b0db-50437e4cc07e",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "source": [
    "This table can be linked to the `diagnoses` dataframe using the `icd_code` and `icd_version` columns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "id": "13daa5ce-8e0c-40a5-9c15-d8abed7d4712",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>subject_id</th>\n",
       "      <th>hadm_id</th>\n",
       "      <th>seq_num</th>\n",
       "      <th>icd_code</th>\n",
       "      <th>icd_version</th>\n",
       "      <th>long_title</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>10035185</td>\n",
       "      <td>22580999</td>\n",
       "      <td>3</td>\n",
       "      <td>4139</td>\n",
       "      <td>9</td>\n",
       "      <td>Other and unspecified angina pectoris</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>10035185</td>\n",
       "      <td>22580999</td>\n",
       "      <td>10</td>\n",
       "      <td>V707</td>\n",
       "      <td>9</td>\n",
       "      <td>Examination of participant in clinical trial</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>10035185</td>\n",
       "      <td>22580999</td>\n",
       "      <td>1</td>\n",
       "      <td>41401</td>\n",
       "      <td>9</td>\n",
       "      <td>Coronary atherosclerosis of native coronary ar...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>10035185</td>\n",
       "      <td>22580999</td>\n",
       "      <td>9</td>\n",
       "      <td>3899</td>\n",
       "      <td>9</td>\n",
       "      <td>Unspecified hearing loss</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>10035185</td>\n",
       "      <td>22580999</td>\n",
       "      <td>11</td>\n",
       "      <td>V8532</td>\n",
       "      <td>9</td>\n",
       "      <td>Body Mass Index 32.0-32.9, adult</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4501</th>\n",
       "      <td>10004733</td>\n",
       "      <td>27411876</td>\n",
       "      <td>19</td>\n",
       "      <td>3129</td>\n",
       "      <td>9</td>\n",
       "      <td>Unspecified disturbance of conduct</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4502</th>\n",
       "      <td>10004733</td>\n",
       "      <td>27411876</td>\n",
       "      <td>30</td>\n",
       "      <td>30000</td>\n",
       "      <td>9</td>\n",
       "      <td>Anxiety state, unspecified</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4503</th>\n",
       "      <td>10004733</td>\n",
       "      <td>27411876</td>\n",
       "      <td>26</td>\n",
       "      <td>4739</td>\n",
       "      <td>9</td>\n",
       "      <td>Unspecified sinusitis (chronic)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4504</th>\n",
       "      <td>10004733</td>\n",
       "      <td>27411876</td>\n",
       "      <td>22</td>\n",
       "      <td>56210</td>\n",
       "      <td>9</td>\n",
       "      <td>Diverticulosis of colon (without mention of he...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4505</th>\n",
       "      <td>10004733</td>\n",
       "      <td>27411876</td>\n",
       "      <td>29</td>\n",
       "      <td>30390</td>\n",
       "      <td>9</td>\n",
       "      <td>Other and unspecified alcohol dependence, unsp...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>4506 rows × 6 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "      subject_id   hadm_id  seq_num icd_code  icd_version  \\\n",
       "0       10035185  22580999        3     4139            9   \n",
       "1       10035185  22580999       10     V707            9   \n",
       "2       10035185  22580999        1    41401            9   \n",
       "3       10035185  22580999        9     3899            9   \n",
       "4       10035185  22580999       11    V8532            9   \n",
       "...          ...       ...      ...      ...          ...   \n",
       "4501    10004733  27411876       19     3129            9   \n",
       "4502    10004733  27411876       30    30000            9   \n",
       "4503    10004733  27411876       26     4739            9   \n",
       "4504    10004733  27411876       22    56210            9   \n",
       "4505    10004733  27411876       29    30390            9   \n",
       "\n",
       "                                             long_title  \n",
       "0                 Other and unspecified angina pectoris  \n",
       "1          Examination of participant in clinical trial  \n",
       "2     Coronary atherosclerosis of native coronary ar...  \n",
       "3                              Unspecified hearing loss  \n",
       "4                      Body Mass Index 32.0-32.9, adult  \n",
       "...                                                 ...  \n",
       "4501                 Unspecified disturbance of conduct  \n",
       "4502                         Anxiety state, unspecified  \n",
       "4503                    Unspecified sinusitis (chronic)  \n",
       "4504  Diverticulosis of colon (without mention of he...  \n",
       "4505  Other and unspecified alcohol dependence, unsp...  \n",
       "\n",
       "[4506 rows x 6 columns]"
      ]
     },
     "execution_count": 60,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "diagnoses_with_code = pd.merge(diagnoses, \n",
    "                               d_icd_diagnoses,\n",
    "                               on = ['icd_code', 'icd_version'],\n",
    "                               how = 'inner')\n",
    "\n",
    "diagnoses_with_code"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ae078234-705f-4dec-812d-f1b5c367617c",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "Let's count the most frequent diagnoses:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "id": "4975455b-cdc3-4298-b5bc-a49c2665332b",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "long_title                                 \n",
       "Unspecified essential hypertension             68\n",
       "Hyperlipidemia, unspecified                    57\n",
       "Acute kidney failure, unspecified              56\n",
       "Other and unspecified hyperlipidemia           55\n",
       "Hypothyroidism, unspecified                    47\n",
       "Obesity, unspecified                           43\n",
       "Anemia, unspecified                            41\n",
       "Long term (current) use of insulin             37\n",
       "Urinary tract infection, site not specified    36\n",
       "Personal history of nicotine dependence        35\n",
       "Name: count, dtype: int64"
      ]
     },
     "execution_count": 61,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "diagnoses_with_code[['long_title']].value_counts().head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1f6252a4-9da1-4071-b067-bbbd11b92193",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "source": [
    "Similarly, we can merge the diagnostic data with our `admissions` dataframe in order to aquire more information about the hospital stays of these patients"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "id": "67c3670c-e783-4991-a602-d81e32fa7464",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "admissions_with_icd = pd.merge(admissions,\n",
    "                               diagnoses_with_code,\n",
    "                               on = ['subject_id', 'hadm_id'],\n",
    "                               how = 'inner')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "id": "2085d049-348f-49bb-9d1d-1843c34a1d77",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>subject_id</th>\n",
       "      <th>hadm_id</th>\n",
       "      <th>admittime</th>\n",
       "      <th>dischtime</th>\n",
       "      <th>deathtime</th>\n",
       "      <th>admission_type</th>\n",
       "      <th>admit_provider_id</th>\n",
       "      <th>admission_location</th>\n",
       "      <th>discharge_location</th>\n",
       "      <th>insurance</th>\n",
       "      <th>language</th>\n",
       "      <th>marital_status</th>\n",
       "      <th>race</th>\n",
       "      <th>edregtime</th>\n",
       "      <th>edouttime</th>\n",
       "      <th>hospital_expire_flag</th>\n",
       "      <th>seq_num</th>\n",
       "      <th>icd_code</th>\n",
       "      <th>icd_version</th>\n",
       "      <th>long_title</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>10004235</td>\n",
       "      <td>24181354</td>\n",
       "      <td>2196-02-24 14:38:00</td>\n",
       "      <td>2196-03-04 14:02:00</td>\n",
       "      <td>NaN</td>\n",
       "      <td>URGENT</td>\n",
       "      <td>P03YMR</td>\n",
       "      <td>TRANSFER FROM HOSPITAL</td>\n",
       "      <td>SKILLED NURSING FACILITY</td>\n",
       "      <td>Medicaid</td>\n",
       "      <td>ENGLISH</td>\n",
       "      <td>SINGLE</td>\n",
       "      <td>BLACK/CAPE VERDEAN</td>\n",
       "      <td>2196-02-24 12:15:00</td>\n",
       "      <td>2196-02-24 17:07:00</td>\n",
       "      <td>0</td>\n",
       "      <td>15</td>\n",
       "      <td>42731</td>\n",
       "      <td>9</td>\n",
       "      <td>Atrial fibrillation</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>10004235</td>\n",
       "      <td>24181354</td>\n",
       "      <td>2196-02-24 14:38:00</td>\n",
       "      <td>2196-03-04 14:02:00</td>\n",
       "      <td>NaN</td>\n",
       "      <td>URGENT</td>\n",
       "      <td>P03YMR</td>\n",
       "      <td>TRANSFER FROM HOSPITAL</td>\n",
       "      <td>SKILLED NURSING FACILITY</td>\n",
       "      <td>Medicaid</td>\n",
       "      <td>ENGLISH</td>\n",
       "      <td>SINGLE</td>\n",
       "      <td>BLACK/CAPE VERDEAN</td>\n",
       "      <td>2196-02-24 12:15:00</td>\n",
       "      <td>2196-02-24 17:07:00</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "      <td>51881</td>\n",
       "      <td>9</td>\n",
       "      <td>Acute respiratory failure</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>10004235</td>\n",
       "      <td>24181354</td>\n",
       "      <td>2196-02-24 14:38:00</td>\n",
       "      <td>2196-03-04 14:02:00</td>\n",
       "      <td>NaN</td>\n",
       "      <td>URGENT</td>\n",
       "      <td>P03YMR</td>\n",
       "      <td>TRANSFER FROM HOSPITAL</td>\n",
       "      <td>SKILLED NURSING FACILITY</td>\n",
       "      <td>Medicaid</td>\n",
       "      <td>ENGLISH</td>\n",
       "      <td>SINGLE</td>\n",
       "      <td>BLACK/CAPE VERDEAN</td>\n",
       "      <td>2196-02-24 12:15:00</td>\n",
       "      <td>2196-02-24 17:07:00</td>\n",
       "      <td>0</td>\n",
       "      <td>12</td>\n",
       "      <td>75169</td>\n",
       "      <td>9</td>\n",
       "      <td>Other anomalies of gallbladder, bile ducts, an...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>10004235</td>\n",
       "      <td>24181354</td>\n",
       "      <td>2196-02-24 14:38:00</td>\n",
       "      <td>2196-03-04 14:02:00</td>\n",
       "      <td>NaN</td>\n",
       "      <td>URGENT</td>\n",
       "      <td>P03YMR</td>\n",
       "      <td>TRANSFER FROM HOSPITAL</td>\n",
       "      <td>SKILLED NURSING FACILITY</td>\n",
       "      <td>Medicaid</td>\n",
       "      <td>ENGLISH</td>\n",
       "      <td>SINGLE</td>\n",
       "      <td>BLACK/CAPE VERDEAN</td>\n",
       "      <td>2196-02-24 12:15:00</td>\n",
       "      <td>2196-02-24 17:07:00</td>\n",
       "      <td>0</td>\n",
       "      <td>13</td>\n",
       "      <td>4254</td>\n",
       "      <td>9</td>\n",
       "      <td>Other primary cardiomyopathies</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>10004235</td>\n",
       "      <td>24181354</td>\n",
       "      <td>2196-02-24 14:38:00</td>\n",
       "      <td>2196-03-04 14:02:00</td>\n",
       "      <td>NaN</td>\n",
       "      <td>URGENT</td>\n",
       "      <td>P03YMR</td>\n",
       "      <td>TRANSFER FROM HOSPITAL</td>\n",
       "      <td>SKILLED NURSING FACILITY</td>\n",
       "      <td>Medicaid</td>\n",
       "      <td>ENGLISH</td>\n",
       "      <td>SINGLE</td>\n",
       "      <td>BLACK/CAPE VERDEAN</td>\n",
       "      <td>2196-02-24 12:15:00</td>\n",
       "      <td>2196-02-24 17:07:00</td>\n",
       "      <td>0</td>\n",
       "      <td>21</td>\n",
       "      <td>2749</td>\n",
       "      <td>9</td>\n",
       "      <td>Gout, unspecified</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   subject_id   hadm_id            admittime            dischtime deathtime  \\\n",
       "0    10004235  24181354  2196-02-24 14:38:00  2196-03-04 14:02:00       NaN   \n",
       "1    10004235  24181354  2196-02-24 14:38:00  2196-03-04 14:02:00       NaN   \n",
       "2    10004235  24181354  2196-02-24 14:38:00  2196-03-04 14:02:00       NaN   \n",
       "3    10004235  24181354  2196-02-24 14:38:00  2196-03-04 14:02:00       NaN   \n",
       "4    10004235  24181354  2196-02-24 14:38:00  2196-03-04 14:02:00       NaN   \n",
       "\n",
       "  admission_type admit_provider_id      admission_location  \\\n",
       "0         URGENT            P03YMR  TRANSFER FROM HOSPITAL   \n",
       "1         URGENT            P03YMR  TRANSFER FROM HOSPITAL   \n",
       "2         URGENT            P03YMR  TRANSFER FROM HOSPITAL   \n",
       "3         URGENT            P03YMR  TRANSFER FROM HOSPITAL   \n",
       "4         URGENT            P03YMR  TRANSFER FROM HOSPITAL   \n",
       "\n",
       "         discharge_location insurance language marital_status  \\\n",
       "0  SKILLED NURSING FACILITY  Medicaid  ENGLISH         SINGLE   \n",
       "1  SKILLED NURSING FACILITY  Medicaid  ENGLISH         SINGLE   \n",
       "2  SKILLED NURSING FACILITY  Medicaid  ENGLISH         SINGLE   \n",
       "3  SKILLED NURSING FACILITY  Medicaid  ENGLISH         SINGLE   \n",
       "4  SKILLED NURSING FACILITY  Medicaid  ENGLISH         SINGLE   \n",
       "\n",
       "                 race            edregtime            edouttime  \\\n",
       "0  BLACK/CAPE VERDEAN  2196-02-24 12:15:00  2196-02-24 17:07:00   \n",
       "1  BLACK/CAPE VERDEAN  2196-02-24 12:15:00  2196-02-24 17:07:00   \n",
       "2  BLACK/CAPE VERDEAN  2196-02-24 12:15:00  2196-02-24 17:07:00   \n",
       "3  BLACK/CAPE VERDEAN  2196-02-24 12:15:00  2196-02-24 17:07:00   \n",
       "4  BLACK/CAPE VERDEAN  2196-02-24 12:15:00  2196-02-24 17:07:00   \n",
       "\n",
       "   hospital_expire_flag  seq_num icd_code  icd_version  \\\n",
       "0                     0       15    42731            9   \n",
       "1                     0        5    51881            9   \n",
       "2                     0       12    75169            9   \n",
       "3                     0       13     4254            9   \n",
       "4                     0       21     2749            9   \n",
       "\n",
       "                                          long_title  \n",
       "0                                Atrial fibrillation  \n",
       "1                          Acute respiratory failure  \n",
       "2  Other anomalies of gallbladder, bile ducts, an...  \n",
       "3                     Other primary cardiomyopathies  \n",
       "4                                  Gout, unspecified  "
      ]
     },
     "execution_count": 63,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "admissions_with_icd.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "eb2d0680-3514-4d6b-a76e-5a0a4629ab51",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "source": [
    "What if we are interested in a specific disease, but did not know the associated ICD code?"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d6dcc8e5-4ad0-4926-9468-279b1081e348",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "### Filtering datasets (cohort specification)\n",
    "\n",
    "Let's say we are only interested in patients with sepsis. We could filter our dataset using keyword matching or specific ICD codes (if we have specific codes we are interested in)\n",
    "\n",
    "*Note: ICD codes are used for insurance billing purposes and have their own limitations. Sepsis, a critical illness that can lead to multiorgan failure, is also a highly difficult disease to diagnose and detect. ICD codes may not be the more sensitive for detecting true sepsis patients - this example is just for demonstration purposes since sepsis a common disease of interest with the mimic dataset. A google scholar search for \"mimic data sepsis\" returns [>186,000 results!](https://scholar.google.com/scholar?hl=en&as_sdt=0%2C14&q=mimic+data+sepsis&btnG=)*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "id": "107c92b8-2a49-4efd-9da2-efe244bbb7ec",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "sepsis_cohort = admissions_with_icd[admissions_with_icd['long_title'].str.contains('sepsis', case = False)]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "id": "d16a2661-a8fe-4f0e-b822-0011f21f9d58",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>subject_id</th>\n",
       "      <th>hadm_id</th>\n",
       "      <th>admittime</th>\n",
       "      <th>dischtime</th>\n",
       "      <th>deathtime</th>\n",
       "      <th>admission_type</th>\n",
       "      <th>admit_provider_id</th>\n",
       "      <th>admission_location</th>\n",
       "      <th>discharge_location</th>\n",
       "      <th>insurance</th>\n",
       "      <th>language</th>\n",
       "      <th>marital_status</th>\n",
       "      <th>race</th>\n",
       "      <th>edregtime</th>\n",
       "      <th>edouttime</th>\n",
       "      <th>hospital_expire_flag</th>\n",
       "      <th>seq_num</th>\n",
       "      <th>icd_code</th>\n",
       "      <th>icd_version</th>\n",
       "      <th>long_title</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>10004235</td>\n",
       "      <td>24181354</td>\n",
       "      <td>2196-02-24 14:38:00</td>\n",
       "      <td>2196-03-04 14:02:00</td>\n",
       "      <td>NaN</td>\n",
       "      <td>URGENT</td>\n",
       "      <td>P03YMR</td>\n",
       "      <td>TRANSFER FROM HOSPITAL</td>\n",
       "      <td>SKILLED NURSING FACILITY</td>\n",
       "      <td>Medicaid</td>\n",
       "      <td>ENGLISH</td>\n",
       "      <td>SINGLE</td>\n",
       "      <td>BLACK/CAPE VERDEAN</td>\n",
       "      <td>2196-02-24 12:15:00</td>\n",
       "      <td>2196-02-24 17:07:00</td>\n",
       "      <td>0</td>\n",
       "      <td>14</td>\n",
       "      <td>99592</td>\n",
       "      <td>9</td>\n",
       "      <td>Severe sepsis</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>242</th>\n",
       "      <td>10020740</td>\n",
       "      <td>23831430</td>\n",
       "      <td>2150-03-11 15:34:00</td>\n",
       "      <td>2150-04-25 13:50:00</td>\n",
       "      <td>NaN</td>\n",
       "      <td>URGENT</td>\n",
       "      <td>P7554I</td>\n",
       "      <td>TRANSFER FROM HOSPITAL</td>\n",
       "      <td>SKILLED NURSING FACILITY</td>\n",
       "      <td>Other</td>\n",
       "      <td>ENGLISH</td>\n",
       "      <td>SINGLE</td>\n",
       "      <td>WHITE</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>12</td>\n",
       "      <td>99591</td>\n",
       "      <td>9</td>\n",
       "      <td>Sepsis</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>332</th>\n",
       "      <td>10020786</td>\n",
       "      <td>23488445</td>\n",
       "      <td>2189-06-09 12:45:00</td>\n",
       "      <td>2189-06-13 17:20:00</td>\n",
       "      <td>NaN</td>\n",
       "      <td>URGENT</td>\n",
       "      <td>P031HZ</td>\n",
       "      <td>TRANSFER FROM HOSPITAL</td>\n",
       "      <td>HOME</td>\n",
       "      <td>Medicare</td>\n",
       "      <td>ENGLISH</td>\n",
       "      <td>WIDOWED</td>\n",
       "      <td>WHITE</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>R6521</td>\n",
       "      <td>10</td>\n",
       "      <td>Severe sepsis with septic shock</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>341</th>\n",
       "      <td>10020786</td>\n",
       "      <td>23488445</td>\n",
       "      <td>2189-06-09 12:45:00</td>\n",
       "      <td>2189-06-13 17:20:00</td>\n",
       "      <td>NaN</td>\n",
       "      <td>URGENT</td>\n",
       "      <td>P031HZ</td>\n",
       "      <td>TRANSFER FROM HOSPITAL</td>\n",
       "      <td>HOME</td>\n",
       "      <td>Medicare</td>\n",
       "      <td>ENGLISH</td>\n",
       "      <td>WIDOWED</td>\n",
       "      <td>WHITE</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>A419</td>\n",
       "      <td>10</td>\n",
       "      <td>Sepsis, unspecified organism</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>451</th>\n",
       "      <td>10003400</td>\n",
       "      <td>23559586</td>\n",
       "      <td>2137-08-04 00:07:00</td>\n",
       "      <td>2137-09-02 17:05:00</td>\n",
       "      <td>2137-09-02 17:05:00</td>\n",
       "      <td>URGENT</td>\n",
       "      <td>P99U21</td>\n",
       "      <td>TRANSFER FROM HOSPITAL</td>\n",
       "      <td>DIED</td>\n",
       "      <td>Medicare</td>\n",
       "      <td>ENGLISH</td>\n",
       "      <td>MARRIED</td>\n",
       "      <td>BLACK/AFRICAN AMERICAN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>R6521</td>\n",
       "      <td>10</td>\n",
       "      <td>Severe sepsis with septic shock</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     subject_id   hadm_id            admittime            dischtime  \\\n",
       "22     10004235  24181354  2196-02-24 14:38:00  2196-03-04 14:02:00   \n",
       "242    10020740  23831430  2150-03-11 15:34:00  2150-04-25 13:50:00   \n",
       "332    10020786  23488445  2189-06-09 12:45:00  2189-06-13 17:20:00   \n",
       "341    10020786  23488445  2189-06-09 12:45:00  2189-06-13 17:20:00   \n",
       "451    10003400  23559586  2137-08-04 00:07:00  2137-09-02 17:05:00   \n",
       "\n",
       "               deathtime admission_type admit_provider_id  \\\n",
       "22                   NaN         URGENT            P03YMR   \n",
       "242                  NaN         URGENT            P7554I   \n",
       "332                  NaN         URGENT            P031HZ   \n",
       "341                  NaN         URGENT            P031HZ   \n",
       "451  2137-09-02 17:05:00         URGENT            P99U21   \n",
       "\n",
       "         admission_location        discharge_location insurance language  \\\n",
       "22   TRANSFER FROM HOSPITAL  SKILLED NURSING FACILITY  Medicaid  ENGLISH   \n",
       "242  TRANSFER FROM HOSPITAL  SKILLED NURSING FACILITY     Other  ENGLISH   \n",
       "332  TRANSFER FROM HOSPITAL                      HOME  Medicare  ENGLISH   \n",
       "341  TRANSFER FROM HOSPITAL                      HOME  Medicare  ENGLISH   \n",
       "451  TRANSFER FROM HOSPITAL                      DIED  Medicare  ENGLISH   \n",
       "\n",
       "    marital_status                    race            edregtime  \\\n",
       "22          SINGLE      BLACK/CAPE VERDEAN  2196-02-24 12:15:00   \n",
       "242         SINGLE                   WHITE                  NaN   \n",
       "332        WIDOWED                   WHITE                  NaN   \n",
       "341        WIDOWED                   WHITE                  NaN   \n",
       "451        MARRIED  BLACK/AFRICAN AMERICAN                  NaN   \n",
       "\n",
       "               edouttime  hospital_expire_flag  seq_num icd_code  icd_version  \\\n",
       "22   2196-02-24 17:07:00                     0       14    99592            9   \n",
       "242                  NaN                     0       12    99591            9   \n",
       "332                  NaN                     0        2    R6521           10   \n",
       "341                  NaN                     0        1     A419           10   \n",
       "451                  NaN                     1        2    R6521           10   \n",
       "\n",
       "                          long_title  \n",
       "22                     Severe sepsis  \n",
       "242                           Sepsis  \n",
       "332  Severe sepsis with septic shock  \n",
       "341     Sepsis, unspecified organism  \n",
       "451  Severe sepsis with septic shock  "
      ]
     },
     "execution_count": 65,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sepsis_cohort.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "id": "f9fd6a46-f45d-49cd-bce6-fa2a6329bf53",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "subject_id              17\n",
       "hadm_id                 24\n",
       "admittime               24\n",
       "dischtime               24\n",
       "deathtime                5\n",
       "admission_type           4\n",
       "admit_provider_id       22\n",
       "admission_location       5\n",
       "discharge_location       7\n",
       "insurance                3\n",
       "language                 1\n",
       "marital_status           4\n",
       "race                     4\n",
       "edregtime               18\n",
       "edouttime               18\n",
       "hospital_expire_flag     2\n",
       "seq_num                 17\n",
       "icd_code                 9\n",
       "icd_version              2\n",
       "long_title               9\n",
       "dtype: int64"
      ]
     },
     "execution_count": 66,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sepsis_cohort.nunique()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "id": "cdabb213-784b-4078-bcfb-136e3a182751",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>icd_code</th>\n",
       "      <th>icd_version</th>\n",
       "      <th>long_title</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>99592</td>\n",
       "      <td>9</td>\n",
       "      <td>Severe sepsis</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>242</th>\n",
       "      <td>99591</td>\n",
       "      <td>9</td>\n",
       "      <td>Sepsis</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>332</th>\n",
       "      <td>R6521</td>\n",
       "      <td>10</td>\n",
       "      <td>Severe sepsis with septic shock</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>341</th>\n",
       "      <td>A419</td>\n",
       "      <td>10</td>\n",
       "      <td>Sepsis, unspecified organism</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1417</th>\n",
       "      <td>R6520</td>\n",
       "      <td>10</td>\n",
       "      <td>Severe sepsis without septic shock</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2720</th>\n",
       "      <td>A4151</td>\n",
       "      <td>10</td>\n",
       "      <td>Sepsis due to Escherichia coli [E. coli]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3388</th>\n",
       "      <td>A4102</td>\n",
       "      <td>10</td>\n",
       "      <td>Sepsis due to Methicillin resistant Staphyloco...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3407</th>\n",
       "      <td>A408</td>\n",
       "      <td>10</td>\n",
       "      <td>Other streptococcal sepsis</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3678</th>\n",
       "      <td>A4189</td>\n",
       "      <td>10</td>\n",
       "      <td>Other specified sepsis</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     icd_code  icd_version                                         long_title\n",
       "22      99592            9                                      Severe sepsis\n",
       "242     99591            9                                             Sepsis\n",
       "332     R6521           10                    Severe sepsis with septic shock\n",
       "341      A419           10                       Sepsis, unspecified organism\n",
       "1417    R6520           10                 Severe sepsis without septic shock\n",
       "2720    A4151           10           Sepsis due to Escherichia coli [E. coli]\n",
       "3388    A4102           10  Sepsis due to Methicillin resistant Staphyloco...\n",
       "3407     A408           10                         Other streptococcal sepsis\n",
       "3678    A4189           10                             Other specified sepsis"
      ]
     },
     "execution_count": 67,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sepsis_cohort[['icd_code', 'icd_version', 'long_title']].drop_duplicates()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "id": "0bb09369-2089-405d-91bb-f0cddb8a577a",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>subject_id</th>\n",
       "      <th>hadm_id</th>\n",
       "      <th>admittime</th>\n",
       "      <th>dischtime</th>\n",
       "      <th>deathtime</th>\n",
       "      <th>admission_type</th>\n",
       "      <th>admit_provider_id</th>\n",
       "      <th>admission_location</th>\n",
       "      <th>discharge_location</th>\n",
       "      <th>insurance</th>\n",
       "      <th>language</th>\n",
       "      <th>marital_status</th>\n",
       "      <th>race</th>\n",
       "      <th>edregtime</th>\n",
       "      <th>edouttime</th>\n",
       "      <th>hospital_expire_flag</th>\n",
       "      <th>seq_num</th>\n",
       "      <th>icd_code</th>\n",
       "      <th>icd_version</th>\n",
       "      <th>long_title</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>10004235</td>\n",
       "      <td>24181354</td>\n",
       "      <td>2196-02-24 14:38:00</td>\n",
       "      <td>2196-03-04 14:02:00</td>\n",
       "      <td>NaN</td>\n",
       "      <td>URGENT</td>\n",
       "      <td>P03YMR</td>\n",
       "      <td>TRANSFER FROM HOSPITAL</td>\n",
       "      <td>SKILLED NURSING FACILITY</td>\n",
       "      <td>Medicaid</td>\n",
       "      <td>ENGLISH</td>\n",
       "      <td>SINGLE</td>\n",
       "      <td>BLACK/CAPE VERDEAN</td>\n",
       "      <td>2196-02-24 12:15:00</td>\n",
       "      <td>2196-02-24 17:07:00</td>\n",
       "      <td>0</td>\n",
       "      <td>14</td>\n",
       "      <td>99592</td>\n",
       "      <td>9</td>\n",
       "      <td>Severe sepsis</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>593</th>\n",
       "      <td>10037975</td>\n",
       "      <td>27617929</td>\n",
       "      <td>2185-01-17 19:11:00</td>\n",
       "      <td>2185-01-22 14:25:00</td>\n",
       "      <td>2185-01-22 14:25:00</td>\n",
       "      <td>URGENT</td>\n",
       "      <td>P17MK7</td>\n",
       "      <td>TRANSFER FROM HOSPITAL</td>\n",
       "      <td>DIED</td>\n",
       "      <td>Medicare</td>\n",
       "      <td>ENGLISH</td>\n",
       "      <td>MARRIED</td>\n",
       "      <td>UNKNOWN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1</td>\n",
       "      <td>10</td>\n",
       "      <td>99592</td>\n",
       "      <td>9</td>\n",
       "      <td>Severe sepsis</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>986</th>\n",
       "      <td>10018081</td>\n",
       "      <td>21027282</td>\n",
       "      <td>2133-12-18 16:58:00</td>\n",
       "      <td>2134-01-12 11:00:00</td>\n",
       "      <td>NaN</td>\n",
       "      <td>EW EMER.</td>\n",
       "      <td>P19SWB</td>\n",
       "      <td>EMERGENCY ROOM</td>\n",
       "      <td>SKILLED NURSING FACILITY</td>\n",
       "      <td>Medicare</td>\n",
       "      <td>ENGLISH</td>\n",
       "      <td>MARRIED</td>\n",
       "      <td>WHITE</td>\n",
       "      <td>2133-12-18 12:09:00</td>\n",
       "      <td>2133-12-18 17:10:00</td>\n",
       "      <td>0</td>\n",
       "      <td>27</td>\n",
       "      <td>99592</td>\n",
       "      <td>9</td>\n",
       "      <td>Severe sepsis</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1160</th>\n",
       "      <td>10020944</td>\n",
       "      <td>29974575</td>\n",
       "      <td>2131-02-27 15:34:00</td>\n",
       "      <td>2131-03-13 17:01:00</td>\n",
       "      <td>NaN</td>\n",
       "      <td>EW EMER.</td>\n",
       "      <td>P27588</td>\n",
       "      <td>EMERGENCY ROOM</td>\n",
       "      <td>SKILLED NURSING FACILITY</td>\n",
       "      <td>Medicare</td>\n",
       "      <td>ENGLISH</td>\n",
       "      <td>NaN</td>\n",
       "      <td>UNKNOWN</td>\n",
       "      <td>2131-02-27 13:16:00</td>\n",
       "      <td>2131-02-27 16:40:00</td>\n",
       "      <td>0</td>\n",
       "      <td>20</td>\n",
       "      <td>99592</td>\n",
       "      <td>9</td>\n",
       "      <td>Severe sepsis</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1516</th>\n",
       "      <td>10007928</td>\n",
       "      <td>20338077</td>\n",
       "      <td>2129-04-05 22:56:00</td>\n",
       "      <td>2129-04-11 17:25:00</td>\n",
       "      <td>NaN</td>\n",
       "      <td>EW EMER.</td>\n",
       "      <td>P825SO</td>\n",
       "      <td>EMERGENCY ROOM</td>\n",
       "      <td>HOME</td>\n",
       "      <td>Other</td>\n",
       "      <td>ENGLISH</td>\n",
       "      <td>DIVORCED</td>\n",
       "      <td>WHITE</td>\n",
       "      <td>2129-04-05 21:18:00</td>\n",
       "      <td>2129-04-06 00:25:00</td>\n",
       "      <td>0</td>\n",
       "      <td>6</td>\n",
       "      <td>99592</td>\n",
       "      <td>9</td>\n",
       "      <td>Severe sepsis</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2440</th>\n",
       "      <td>10009049</td>\n",
       "      <td>22995465</td>\n",
       "      <td>2174-05-26 08:21:00</td>\n",
       "      <td>2174-05-31 14:15:00</td>\n",
       "      <td>NaN</td>\n",
       "      <td>EW EMER.</td>\n",
       "      <td>P3529J</td>\n",
       "      <td>EMERGENCY ROOM</td>\n",
       "      <td>HOME</td>\n",
       "      <td>Other</td>\n",
       "      <td>ENGLISH</td>\n",
       "      <td>MARRIED</td>\n",
       "      <td>WHITE</td>\n",
       "      <td>2174-05-26 04:20:00</td>\n",
       "      <td>2174-05-26 09:18:00</td>\n",
       "      <td>0</td>\n",
       "      <td>7</td>\n",
       "      <td>99592</td>\n",
       "      <td>9</td>\n",
       "      <td>Severe sepsis</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2681</th>\n",
       "      <td>10002428</td>\n",
       "      <td>28662225</td>\n",
       "      <td>2156-04-12 14:16:00</td>\n",
       "      <td>2156-04-29 16:26:00</td>\n",
       "      <td>NaN</td>\n",
       "      <td>EW EMER.</td>\n",
       "      <td>P64TOH</td>\n",
       "      <td>EMERGENCY ROOM</td>\n",
       "      <td>SKILLED NURSING FACILITY</td>\n",
       "      <td>Medicare</td>\n",
       "      <td>ENGLISH</td>\n",
       "      <td>WIDOWED</td>\n",
       "      <td>WHITE</td>\n",
       "      <td>2156-04-12 09:56:00</td>\n",
       "      <td>2156-04-12 17:11:00</td>\n",
       "      <td>0</td>\n",
       "      <td>15</td>\n",
       "      <td>99592</td>\n",
       "      <td>9</td>\n",
       "      <td>Severe sepsis</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2810</th>\n",
       "      <td>10031757</td>\n",
       "      <td>28477280</td>\n",
       "      <td>2137-10-12 22:43:00</td>\n",
       "      <td>2137-10-24 17:30:00</td>\n",
       "      <td>NaN</td>\n",
       "      <td>DIRECT EMER.</td>\n",
       "      <td>P94RT6</td>\n",
       "      <td>CLINIC REFERRAL</td>\n",
       "      <td>HOSPICE</td>\n",
       "      <td>Other</td>\n",
       "      <td>ENGLISH</td>\n",
       "      <td>DIVORCED</td>\n",
       "      <td>WHITE</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>9</td>\n",
       "      <td>99592</td>\n",
       "      <td>9</td>\n",
       "      <td>Severe sepsis</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      subject_id   hadm_id            admittime            dischtime  \\\n",
       "22      10004235  24181354  2196-02-24 14:38:00  2196-03-04 14:02:00   \n",
       "593     10037975  27617929  2185-01-17 19:11:00  2185-01-22 14:25:00   \n",
       "986     10018081  21027282  2133-12-18 16:58:00  2134-01-12 11:00:00   \n",
       "1160    10020944  29974575  2131-02-27 15:34:00  2131-03-13 17:01:00   \n",
       "1516    10007928  20338077  2129-04-05 22:56:00  2129-04-11 17:25:00   \n",
       "2440    10009049  22995465  2174-05-26 08:21:00  2174-05-31 14:15:00   \n",
       "2681    10002428  28662225  2156-04-12 14:16:00  2156-04-29 16:26:00   \n",
       "2810    10031757  28477280  2137-10-12 22:43:00  2137-10-24 17:30:00   \n",
       "\n",
       "                deathtime admission_type admit_provider_id  \\\n",
       "22                    NaN         URGENT            P03YMR   \n",
       "593   2185-01-22 14:25:00         URGENT            P17MK7   \n",
       "986                   NaN       EW EMER.            P19SWB   \n",
       "1160                  NaN       EW EMER.            P27588   \n",
       "1516                  NaN       EW EMER.            P825SO   \n",
       "2440                  NaN       EW EMER.            P3529J   \n",
       "2681                  NaN       EW EMER.            P64TOH   \n",
       "2810                  NaN   DIRECT EMER.            P94RT6   \n",
       "\n",
       "          admission_location        discharge_location insurance language  \\\n",
       "22    TRANSFER FROM HOSPITAL  SKILLED NURSING FACILITY  Medicaid  ENGLISH   \n",
       "593   TRANSFER FROM HOSPITAL                      DIED  Medicare  ENGLISH   \n",
       "986           EMERGENCY ROOM  SKILLED NURSING FACILITY  Medicare  ENGLISH   \n",
       "1160          EMERGENCY ROOM  SKILLED NURSING FACILITY  Medicare  ENGLISH   \n",
       "1516          EMERGENCY ROOM                      HOME     Other  ENGLISH   \n",
       "2440          EMERGENCY ROOM                      HOME     Other  ENGLISH   \n",
       "2681          EMERGENCY ROOM  SKILLED NURSING FACILITY  Medicare  ENGLISH   \n",
       "2810         CLINIC REFERRAL                   HOSPICE     Other  ENGLISH   \n",
       "\n",
       "     marital_status                race            edregtime  \\\n",
       "22           SINGLE  BLACK/CAPE VERDEAN  2196-02-24 12:15:00   \n",
       "593         MARRIED             UNKNOWN                  NaN   \n",
       "986         MARRIED               WHITE  2133-12-18 12:09:00   \n",
       "1160            NaN             UNKNOWN  2131-02-27 13:16:00   \n",
       "1516       DIVORCED               WHITE  2129-04-05 21:18:00   \n",
       "2440        MARRIED               WHITE  2174-05-26 04:20:00   \n",
       "2681        WIDOWED               WHITE  2156-04-12 09:56:00   \n",
       "2810       DIVORCED               WHITE                  NaN   \n",
       "\n",
       "                edouttime  hospital_expire_flag  seq_num icd_code  \\\n",
       "22    2196-02-24 17:07:00                     0       14    99592   \n",
       "593                   NaN                     1       10    99592   \n",
       "986   2133-12-18 17:10:00                     0       27    99592   \n",
       "1160  2131-02-27 16:40:00                     0       20    99592   \n",
       "1516  2129-04-06 00:25:00                     0        6    99592   \n",
       "2440  2174-05-26 09:18:00                     0        7    99592   \n",
       "2681  2156-04-12 17:11:00                     0       15    99592   \n",
       "2810                  NaN                     0        9    99592   \n",
       "\n",
       "      icd_version     long_title  \n",
       "22              9  Severe sepsis  \n",
       "593             9  Severe sepsis  \n",
       "986             9  Severe sepsis  \n",
       "1160            9  Severe sepsis  \n",
       "1516            9  Severe sepsis  \n",
       "2440            9  Severe sepsis  \n",
       "2681            9  Severe sepsis  \n",
       "2810            9  Severe sepsis  "
      ]
     },
     "execution_count": 68,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sepsis_cohort_icd = admissions_with_icd[admissions_with_icd['icd_code']=='99592']\n",
    "sepsis_cohort_icd"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3bcb99c9-5fc2-44ff-b84f-d23500180304",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "# Introduction to Machine Learning with scikit-learn"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "462dba09",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "## Case Study: Prediction of ICU Mortality with Supervised Learning\n",
    "\n",
    "For this demonstration, we will use a previously prepared dataset from an earlier version of MIMIC. This data comes from a [community challenge](https://physionet.org/content/challenge-2012/1.0.0/) focused on predicting mortality with the MIMIC dataset.\n",
    "\n",
    "We will use the previously pre-processed dataset from one of the challenge winners [Alistair Johnson](https://github.com/alistairewj/challenge2012) - so we won't do much of our data cleaning this time (though it is still good practice!). Johnson also includes his [pre-processing notebook](https://github.com/alistairewj/challenge2012/blob/master/prepare-data.ipynb) which can provide another example of how to proceed with cleaning clinical data."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a1abe321-e4a3-4451-b0c5-3b343abae845",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "First, we will load in our machine learning functions from scikit-learn"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "id": "1fdff6f8",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "# load in our machine learning functions from scikit-learn\n",
    "from sklearn.metrics import confusion_matrix, ConfusionMatrixDisplay, classification_report, roc_curve, roc_auc_score, precision_score, recall_score, accuracy_score, auc, average_precision_score, precision_recall_curve, precision_recall_fscore_support, f1_score, log_loss\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "from sklearn.ensemble import RandomForestClassifier\n",
    "from sklearn.model_selection import KFold, GridSearchCV,RandomizedSearchCV,cross_validate\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "from sklearn.impute import SimpleImputer\n",
    "from sklearn.pipeline import Pipeline\n",
    "from sklearn.compose import ColumnTransformer"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f38d7cdd-c67c-4d03-8e91-9b646a6b0335",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "In this dataset, we will train a machine learning model to predict mortality using 4,000 unique patient records"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "id": "b609a367",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>recordid</th>\n",
       "      <th>SAPS-I</th>\n",
       "      <th>SOFA</th>\n",
       "      <th>Length_of_stay</th>\n",
       "      <th>Survival</th>\n",
       "      <th>In-hospital_death</th>\n",
       "      <th>Age</th>\n",
       "      <th>Gender</th>\n",
       "      <th>Height</th>\n",
       "      <th>Weight</th>\n",
       "      <th>CCU</th>\n",
       "      <th>CSRU</th>\n",
       "      <th>SICU</th>\n",
       "      <th>DiasABP_first</th>\n",
       "      <th>GCS_first</th>\n",
       "      <th>Glucose_first</th>\n",
       "      <th>HR_first</th>\n",
       "      <th>MAP_first</th>\n",
       "      <th>NIDiasABP_first</th>\n",
       "      <th>NIMAP_first</th>\n",
       "      <th>NISysABP_first</th>\n",
       "      <th>RespRate_first</th>\n",
       "      <th>SaO2_first</th>\n",
       "      <th>Temp_first</th>\n",
       "      <th>DiasABP_last</th>\n",
       "      <th>GCS_last</th>\n",
       "      <th>Glucose_last</th>\n",
       "      <th>HR_last</th>\n",
       "      <th>MAP_last</th>\n",
       "      <th>NIDiasABP_last</th>\n",
       "      <th>NIMAP_last</th>\n",
       "      <th>NISysABP_last</th>\n",
       "      <th>RespRate_last</th>\n",
       "      <th>SaO2_last</th>\n",
       "      <th>Temp_last</th>\n",
       "      <th>DiasABP_lowest</th>\n",
       "      <th>GCS_lowest</th>\n",
       "      <th>Glucose_lowest</th>\n",
       "      <th>HR_lowest</th>\n",
       "      <th>MAP_lowest</th>\n",
       "      <th>NIDiasABP_lowest</th>\n",
       "      <th>NIMAP_lowest</th>\n",
       "      <th>NISysABP_lowest</th>\n",
       "      <th>RespRate_lowest</th>\n",
       "      <th>SaO2_lowest</th>\n",
       "      <th>Temp_lowest</th>\n",
       "      <th>DiasABP_highest</th>\n",
       "      <th>GCS_highest</th>\n",
       "      <th>Glucose_highest</th>\n",
       "      <th>HR_highest</th>\n",
       "      <th>MAP_highest</th>\n",
       "      <th>NIDiasABP_highest</th>\n",
       "      <th>NIMAP_highest</th>\n",
       "      <th>NISysABP_highest</th>\n",
       "      <th>RespRate_highest</th>\n",
       "      <th>SaO2_highest</th>\n",
       "      <th>Temp_highest</th>\n",
       "      <th>DiasABP_median</th>\n",
       "      <th>GCS_median</th>\n",
       "      <th>Glucose_median</th>\n",
       "      <th>HR_median</th>\n",
       "      <th>MAP_median</th>\n",
       "      <th>NIDiasABP_median</th>\n",
       "      <th>NIMAP_median</th>\n",
       "      <th>NISysABP_median</th>\n",
       "      <th>RespRate_median</th>\n",
       "      <th>SaO2_median</th>\n",
       "      <th>Temp_median</th>\n",
       "      <th>ALP_first</th>\n",
       "      <th>ALT_first</th>\n",
       "      <th>AST_first</th>\n",
       "      <th>Albumin_first</th>\n",
       "      <th>BUN_first</th>\n",
       "      <th>Bilirubin_first</th>\n",
       "      <th>Cholesterol_first</th>\n",
       "      <th>Creatinine_first</th>\n",
       "      <th>FiO2_first</th>\n",
       "      <th>HCO3_first</th>\n",
       "      <th>HCT_first</th>\n",
       "      <th>K_first</th>\n",
       "      <th>Lactate_first</th>\n",
       "      <th>Mg_first</th>\n",
       "      <th>Na_first</th>\n",
       "      <th>PaCO2_first</th>\n",
       "      <th>PaO2_first</th>\n",
       "      <th>Platelets_first</th>\n",
       "      <th>SysABP_first</th>\n",
       "      <th>TroponinI_first</th>\n",
       "      <th>TroponinT_first</th>\n",
       "      <th>WBC_first</th>\n",
       "      <th>Weight_first</th>\n",
       "      <th>pH_first</th>\n",
       "      <th>ALP_last</th>\n",
       "      <th>ALT_last</th>\n",
       "      <th>AST_last</th>\n",
       "      <th>Albumin_last</th>\n",
       "      <th>BUN_last</th>\n",
       "      <th>Bilirubin_last</th>\n",
       "      <th>Cholesterol_last</th>\n",
       "      <th>Creatinine_last</th>\n",
       "      <th>FiO2_last</th>\n",
       "      <th>HCO3_last</th>\n",
       "      <th>HCT_last</th>\n",
       "      <th>K_last</th>\n",
       "      <th>Lactate_last</th>\n",
       "      <th>Mg_last</th>\n",
       "      <th>Na_last</th>\n",
       "      <th>PaCO2_last</th>\n",
       "      <th>PaO2_last</th>\n",
       "      <th>Platelets_last</th>\n",
       "      <th>SysABP_last</th>\n",
       "      <th>TroponinI_last</th>\n",
       "      <th>TroponinT_last</th>\n",
       "      <th>WBC_last</th>\n",
       "      <th>Weight_last</th>\n",
       "      <th>pH_last</th>\n",
       "      <th>MechVentStartTime</th>\n",
       "      <th>MechVentDuration</th>\n",
       "      <th>MechVentLast8Hour</th>\n",
       "      <th>UrineOutputSum</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>132539</td>\n",
       "      <td>6</td>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "      <td>-1</td>\n",
       "      <td>0</td>\n",
       "      <td>54.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>NaN</td>\n",
       "      <td>15.0</td>\n",
       "      <td>205.0</td>\n",
       "      <td>73.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>65.0</td>\n",
       "      <td>92.33</td>\n",
       "      <td>147.0</td>\n",
       "      <td>19.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>35.1</td>\n",
       "      <td>NaN</td>\n",
       "      <td>15.0</td>\n",
       "      <td>115.0</td>\n",
       "      <td>86.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>55.0</td>\n",
       "      <td>79.33</td>\n",
       "      <td>128.0</td>\n",
       "      <td>23.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>37.8</td>\n",
       "      <td>NaN</td>\n",
       "      <td>14.0</td>\n",
       "      <td>115.0</td>\n",
       "      <td>58.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>39.0</td>\n",
       "      <td>58.67</td>\n",
       "      <td>96.0</td>\n",
       "      <td>12.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>35.1</td>\n",
       "      <td>NaN</td>\n",
       "      <td>15.0</td>\n",
       "      <td>205.0</td>\n",
       "      <td>86.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>67.0</td>\n",
       "      <td>92.33</td>\n",
       "      <td>157.0</td>\n",
       "      <td>24.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>38.2</td>\n",
       "      <td>NaN</td>\n",
       "      <td>15.0</td>\n",
       "      <td>160.0</td>\n",
       "      <td>73.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>49.0</td>\n",
       "      <td>70.000</td>\n",
       "      <td>110.0</td>\n",
       "      <td>18.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>37.70</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>13.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.8</td>\n",
       "      <td>NaN</td>\n",
       "      <td>26.0</td>\n",
       "      <td>33.7</td>\n",
       "      <td>4.4</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.5</td>\n",
       "      <td>137.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>221.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>11.2</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>8.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.7</td>\n",
       "      <td>NaN</td>\n",
       "      <td>28.0</td>\n",
       "      <td>30.3</td>\n",
       "      <td>4.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.9</td>\n",
       "      <td>136.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>185.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>9.4</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>132540</td>\n",
       "      <td>16</td>\n",
       "      <td>8</td>\n",
       "      <td>8</td>\n",
       "      <td>-1</td>\n",
       "      <td>0</td>\n",
       "      <td>76.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>175.3</td>\n",
       "      <td>76.0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>67.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>105.0</td>\n",
       "      <td>88.0</td>\n",
       "      <td>79.0</td>\n",
       "      <td>38.0</td>\n",
       "      <td>49.33</td>\n",
       "      <td>72.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>99.0</td>\n",
       "      <td>35.2</td>\n",
       "      <td>51.0</td>\n",
       "      <td>15.0</td>\n",
       "      <td>146.0</td>\n",
       "      <td>65.0</td>\n",
       "      <td>69.0</td>\n",
       "      <td>49.0</td>\n",
       "      <td>68.33</td>\n",
       "      <td>107.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>97.0</td>\n",
       "      <td>37.1</td>\n",
       "      <td>32.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>105.0</td>\n",
       "      <td>65.0</td>\n",
       "      <td>43.0</td>\n",
       "      <td>38.0</td>\n",
       "      <td>49.33</td>\n",
       "      <td>72.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>93.0</td>\n",
       "      <td>34.5</td>\n",
       "      <td>81.0</td>\n",
       "      <td>15.0</td>\n",
       "      <td>146.0</td>\n",
       "      <td>90.0</td>\n",
       "      <td>100.0</td>\n",
       "      <td>67.0</td>\n",
       "      <td>88.33</td>\n",
       "      <td>131.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>99.0</td>\n",
       "      <td>37.9</td>\n",
       "      <td>59.0</td>\n",
       "      <td>15.0</td>\n",
       "      <td>125.5</td>\n",
       "      <td>80.0</td>\n",
       "      <td>79.0</td>\n",
       "      <td>59.0</td>\n",
       "      <td>78.165</td>\n",
       "      <td>115.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>97.0</td>\n",
       "      <td>37.45</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>16.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.8</td>\n",
       "      <td>1.0</td>\n",
       "      <td>21.0</td>\n",
       "      <td>24.7</td>\n",
       "      <td>4.3</td>\n",
       "      <td>NaN</td>\n",
       "      <td>3.1</td>\n",
       "      <td>139.0</td>\n",
       "      <td>34.0</td>\n",
       "      <td>344.0</td>\n",
       "      <td>164.0</td>\n",
       "      <td>105.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>7.4</td>\n",
       "      <td>80.6</td>\n",
       "      <td>7.45</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>21.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.3</td>\n",
       "      <td>0.4</td>\n",
       "      <td>24.0</td>\n",
       "      <td>29.4</td>\n",
       "      <td>3.5</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.9</td>\n",
       "      <td>135.0</td>\n",
       "      <td>46.0</td>\n",
       "      <td>140.0</td>\n",
       "      <td>135.0</td>\n",
       "      <td>103.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>13.3</td>\n",
       "      <td>81.6</td>\n",
       "      <td>7.37</td>\n",
       "      <td>71.0</td>\n",
       "      <td>360.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>5.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>132541</td>\n",
       "      <td>21</td>\n",
       "      <td>11</td>\n",
       "      <td>19</td>\n",
       "      <td>-1</td>\n",
       "      <td>0</td>\n",
       "      <td>44.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>56.7</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>81.0</td>\n",
       "      <td>7.0</td>\n",
       "      <td>141.0</td>\n",
       "      <td>100.0</td>\n",
       "      <td>107.0</td>\n",
       "      <td>84.0</td>\n",
       "      <td>100.30</td>\n",
       "      <td>133.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>95.0</td>\n",
       "      <td>37.8</td>\n",
       "      <td>70.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>143.0</td>\n",
       "      <td>71.0</td>\n",
       "      <td>92.0</td>\n",
       "      <td>84.0</td>\n",
       "      <td>103.00</td>\n",
       "      <td>141.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>95.0</td>\n",
       "      <td>37.2</td>\n",
       "      <td>52.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>119.0</td>\n",
       "      <td>57.0</td>\n",
       "      <td>72.0</td>\n",
       "      <td>66.0</td>\n",
       "      <td>83.33</td>\n",
       "      <td>111.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>95.0</td>\n",
       "      <td>36.7</td>\n",
       "      <td>81.0</td>\n",
       "      <td>8.0</td>\n",
       "      <td>143.0</td>\n",
       "      <td>113.0</td>\n",
       "      <td>107.0</td>\n",
       "      <td>95.0</td>\n",
       "      <td>110.00</td>\n",
       "      <td>150.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>95.0</td>\n",
       "      <td>39.0</td>\n",
       "      <td>67.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>141.0</td>\n",
       "      <td>85.5</td>\n",
       "      <td>90.0</td>\n",
       "      <td>79.0</td>\n",
       "      <td>97.670</td>\n",
       "      <td>134.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>95.0</td>\n",
       "      <td>37.85</td>\n",
       "      <td>127.0</td>\n",
       "      <td>91.0</td>\n",
       "      <td>235.0</td>\n",
       "      <td>2.7</td>\n",
       "      <td>8.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.4</td>\n",
       "      <td>1.0</td>\n",
       "      <td>24.0</td>\n",
       "      <td>28.5</td>\n",
       "      <td>3.3</td>\n",
       "      <td>1.3</td>\n",
       "      <td>1.9</td>\n",
       "      <td>137.0</td>\n",
       "      <td>37.0</td>\n",
       "      <td>65.0</td>\n",
       "      <td>72.0</td>\n",
       "      <td>148.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>4.2</td>\n",
       "      <td>56.7</td>\n",
       "      <td>7.51</td>\n",
       "      <td>105.0</td>\n",
       "      <td>75.0</td>\n",
       "      <td>164.0</td>\n",
       "      <td>2.3</td>\n",
       "      <td>3.0</td>\n",
       "      <td>2.8</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.3</td>\n",
       "      <td>0.4</td>\n",
       "      <td>25.0</td>\n",
       "      <td>29.4</td>\n",
       "      <td>3.7</td>\n",
       "      <td>0.9</td>\n",
       "      <td>1.7</td>\n",
       "      <td>138.0</td>\n",
       "      <td>37.0</td>\n",
       "      <td>173.0</td>\n",
       "      <td>113.0</td>\n",
       "      <td>126.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>6.2</td>\n",
       "      <td>56.7</td>\n",
       "      <td>7.47</td>\n",
       "      <td>617.0</td>\n",
       "      <td>2160.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>14.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>132543</td>\n",
       "      <td>7</td>\n",
       "      <td>1</td>\n",
       "      <td>9</td>\n",
       "      <td>575</td>\n",
       "      <td>0</td>\n",
       "      <td>68.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>180.3</td>\n",
       "      <td>84.6</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>15.0</td>\n",
       "      <td>129.0</td>\n",
       "      <td>79.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>63.0</td>\n",
       "      <td>86.67</td>\n",
       "      <td>134.0</td>\n",
       "      <td>19.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>36.3</td>\n",
       "      <td>NaN</td>\n",
       "      <td>15.0</td>\n",
       "      <td>117.0</td>\n",
       "      <td>79.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>81.0</td>\n",
       "      <td>98.67</td>\n",
       "      <td>134.0</td>\n",
       "      <td>22.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>37.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>14.0</td>\n",
       "      <td>106.0</td>\n",
       "      <td>57.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>45.0</td>\n",
       "      <td>68.33</td>\n",
       "      <td>102.0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>35.1</td>\n",
       "      <td>NaN</td>\n",
       "      <td>15.0</td>\n",
       "      <td>129.0</td>\n",
       "      <td>88.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>81.0</td>\n",
       "      <td>100.70</td>\n",
       "      <td>140.0</td>\n",
       "      <td>24.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>37.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>15.0</td>\n",
       "      <td>117.0</td>\n",
       "      <td>74.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>64.0</td>\n",
       "      <td>83.670</td>\n",
       "      <td>120.0</td>\n",
       "      <td>16.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>36.40</td>\n",
       "      <td>105.0</td>\n",
       "      <td>12.0</td>\n",
       "      <td>15.0</td>\n",
       "      <td>4.4</td>\n",
       "      <td>23.0</td>\n",
       "      <td>0.2</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.9</td>\n",
       "      <td>NaN</td>\n",
       "      <td>28.0</td>\n",
       "      <td>41.3</td>\n",
       "      <td>4.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2.1</td>\n",
       "      <td>140.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>391.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>11.5</td>\n",
       "      <td>84.6</td>\n",
       "      <td>NaN</td>\n",
       "      <td>105.0</td>\n",
       "      <td>12.0</td>\n",
       "      <td>15.0</td>\n",
       "      <td>4.4</td>\n",
       "      <td>10.0</td>\n",
       "      <td>0.2</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.7</td>\n",
       "      <td>NaN</td>\n",
       "      <td>28.0</td>\n",
       "      <td>36.3</td>\n",
       "      <td>3.8</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.9</td>\n",
       "      <td>137.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>284.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>7.9</td>\n",
       "      <td>84.6</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>132545</td>\n",
       "      <td>17</td>\n",
       "      <td>2</td>\n",
       "      <td>4</td>\n",
       "      <td>918</td>\n",
       "      <td>0</td>\n",
       "      <td>88.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>15.0</td>\n",
       "      <td>113.0</td>\n",
       "      <td>93.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>41.0</td>\n",
       "      <td>75.33</td>\n",
       "      <td>144.0</td>\n",
       "      <td>24.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>37.8</td>\n",
       "      <td>NaN</td>\n",
       "      <td>15.0</td>\n",
       "      <td>92.0</td>\n",
       "      <td>68.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>42.0</td>\n",
       "      <td>73.00</td>\n",
       "      <td>135.0</td>\n",
       "      <td>16.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>36.7</td>\n",
       "      <td>NaN</td>\n",
       "      <td>15.0</td>\n",
       "      <td>92.0</td>\n",
       "      <td>65.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>26.0</td>\n",
       "      <td>52.33</td>\n",
       "      <td>105.0</td>\n",
       "      <td>13.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>35.8</td>\n",
       "      <td>NaN</td>\n",
       "      <td>15.0</td>\n",
       "      <td>113.0</td>\n",
       "      <td>94.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>96.0</td>\n",
       "      <td>105.70</td>\n",
       "      <td>157.0</td>\n",
       "      <td>27.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>37.8</td>\n",
       "      <td>NaN</td>\n",
       "      <td>15.0</td>\n",
       "      <td>102.5</td>\n",
       "      <td>73.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>42.0</td>\n",
       "      <td>75.330</td>\n",
       "      <td>131.0</td>\n",
       "      <td>19.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>37.00</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>3.3</td>\n",
       "      <td>45.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>18.0</td>\n",
       "      <td>22.6</td>\n",
       "      <td>6.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.5</td>\n",
       "      <td>140.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>109.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>3.8</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>3.3</td>\n",
       "      <td>25.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>20.0</td>\n",
       "      <td>30.9</td>\n",
       "      <td>4.1</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.6</td>\n",
       "      <td>139.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>97.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>4.8</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3995</th>\n",
       "      <td>142665</td>\n",
       "      <td>19</td>\n",
       "      <td>7</td>\n",
       "      <td>10</td>\n",
       "      <td>336</td>\n",
       "      <td>0</td>\n",
       "      <td>70.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>87.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>50.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>121.0</td>\n",
       "      <td>83.0</td>\n",
       "      <td>66.0</td>\n",
       "      <td>47.0</td>\n",
       "      <td>61.33</td>\n",
       "      <td>90.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>35.5</td>\n",
       "      <td>79.0</td>\n",
       "      <td>15.0</td>\n",
       "      <td>106.0</td>\n",
       "      <td>89.0</td>\n",
       "      <td>109.0</td>\n",
       "      <td>87.0</td>\n",
       "      <td>104.00</td>\n",
       "      <td>138.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>36.9</td>\n",
       "      <td>45.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>106.0</td>\n",
       "      <td>83.0</td>\n",
       "      <td>57.0</td>\n",
       "      <td>40.0</td>\n",
       "      <td>60.00</td>\n",
       "      <td>88.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>35.5</td>\n",
       "      <td>82.0</td>\n",
       "      <td>15.0</td>\n",
       "      <td>136.0</td>\n",
       "      <td>113.0</td>\n",
       "      <td>112.0</td>\n",
       "      <td>87.0</td>\n",
       "      <td>104.00</td>\n",
       "      <td>138.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>37.8</td>\n",
       "      <td>68.0</td>\n",
       "      <td>14.0</td>\n",
       "      <td>121.0</td>\n",
       "      <td>99.5</td>\n",
       "      <td>86.5</td>\n",
       "      <td>62.5</td>\n",
       "      <td>78.000</td>\n",
       "      <td>109.5</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>37.10</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>14.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.8</td>\n",
       "      <td>0.5</td>\n",
       "      <td>21.0</td>\n",
       "      <td>27.8</td>\n",
       "      <td>2.4</td>\n",
       "      <td>2.3</td>\n",
       "      <td>1.0</td>\n",
       "      <td>138.0</td>\n",
       "      <td>40.0</td>\n",
       "      <td>205.0</td>\n",
       "      <td>198.0</td>\n",
       "      <td>108.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>11.5</td>\n",
       "      <td>NaN</td>\n",
       "      <td>7.42</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>18.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.5</td>\n",
       "      <td>22.0</td>\n",
       "      <td>30.3</td>\n",
       "      <td>3.9</td>\n",
       "      <td>1.1</td>\n",
       "      <td>2.0</td>\n",
       "      <td>136.0</td>\n",
       "      <td>42.0</td>\n",
       "      <td>123.0</td>\n",
       "      <td>218.0</td>\n",
       "      <td>152.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>17.5</td>\n",
       "      <td>NaN</td>\n",
       "      <td>7.39</td>\n",
       "      <td>23.0</td>\n",
       "      <td>1200.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>6.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3996</th>\n",
       "      <td>142667</td>\n",
       "      <td>8</td>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "      <td>-1</td>\n",
       "      <td>0</td>\n",
       "      <td>25.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>166.4</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>15.0</td>\n",
       "      <td>96.0</td>\n",
       "      <td>81.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>52.0</td>\n",
       "      <td>75.33</td>\n",
       "      <td>122.0</td>\n",
       "      <td>18.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>36.8</td>\n",
       "      <td>NaN</td>\n",
       "      <td>15.0</td>\n",
       "      <td>88.0</td>\n",
       "      <td>80.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>60.0</td>\n",
       "      <td>84.33</td>\n",
       "      <td>133.0</td>\n",
       "      <td>18.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>36.8</td>\n",
       "      <td>NaN</td>\n",
       "      <td>15.0</td>\n",
       "      <td>65.0</td>\n",
       "      <td>59.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>38.0</td>\n",
       "      <td>61.67</td>\n",
       "      <td>109.0</td>\n",
       "      <td>14.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>35.5</td>\n",
       "      <td>NaN</td>\n",
       "      <td>15.0</td>\n",
       "      <td>251.0</td>\n",
       "      <td>106.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>78.0</td>\n",
       "      <td>100.70</td>\n",
       "      <td>151.0</td>\n",
       "      <td>22.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>37.4</td>\n",
       "      <td>NaN</td>\n",
       "      <td>15.0</td>\n",
       "      <td>96.0</td>\n",
       "      <td>79.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>58.5</td>\n",
       "      <td>81.835</td>\n",
       "      <td>125.5</td>\n",
       "      <td>18.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>36.70</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>5.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>117.0</td>\n",
       "      <td>0.9</td>\n",
       "      <td>NaN</td>\n",
       "      <td>20.0</td>\n",
       "      <td>30.3</td>\n",
       "      <td>3.6</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.3</td>\n",
       "      <td>141.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>155.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>6.5</td>\n",
       "      <td>166.4</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>7.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>117.0</td>\n",
       "      <td>0.7</td>\n",
       "      <td>NaN</td>\n",
       "      <td>28.0</td>\n",
       "      <td>31.9</td>\n",
       "      <td>3.4</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.5</td>\n",
       "      <td>142.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>118.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>3.0</td>\n",
       "      <td>166.4</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3997</th>\n",
       "      <td>142670</td>\n",
       "      <td>8</td>\n",
       "      <td>5</td>\n",
       "      <td>11</td>\n",
       "      <td>-1</td>\n",
       "      <td>0</td>\n",
       "      <td>44.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>109.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>67.0</td>\n",
       "      <td>8.0</td>\n",
       "      <td>99.0</td>\n",
       "      <td>86.0</td>\n",
       "      <td>81.0</td>\n",
       "      <td>52.0</td>\n",
       "      <td>67.67</td>\n",
       "      <td>99.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>37.8</td>\n",
       "      <td>65.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>132.0</td>\n",
       "      <td>86.0</td>\n",
       "      <td>79.0</td>\n",
       "      <td>61.0</td>\n",
       "      <td>77.00</td>\n",
       "      <td>109.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>38.4</td>\n",
       "      <td>63.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>97.0</td>\n",
       "      <td>80.0</td>\n",
       "      <td>79.0</td>\n",
       "      <td>40.0</td>\n",
       "      <td>60.33</td>\n",
       "      <td>94.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>37.1</td>\n",
       "      <td>88.0</td>\n",
       "      <td>8.0</td>\n",
       "      <td>132.0</td>\n",
       "      <td>143.0</td>\n",
       "      <td>103.0</td>\n",
       "      <td>90.0</td>\n",
       "      <td>108.30</td>\n",
       "      <td>156.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>38.4</td>\n",
       "      <td>74.0</td>\n",
       "      <td>7.0</td>\n",
       "      <td>99.0</td>\n",
       "      <td>94.5</td>\n",
       "      <td>89.5</td>\n",
       "      <td>60.5</td>\n",
       "      <td>78.835</td>\n",
       "      <td>117.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>37.80</td>\n",
       "      <td>51.0</td>\n",
       "      <td>20.0</td>\n",
       "      <td>20.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>10.0</td>\n",
       "      <td>0.5</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.2</td>\n",
       "      <td>0.6</td>\n",
       "      <td>23.0</td>\n",
       "      <td>37.9</td>\n",
       "      <td>4.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2.5</td>\n",
       "      <td>140.0</td>\n",
       "      <td>38.0</td>\n",
       "      <td>212.0</td>\n",
       "      <td>220.0</td>\n",
       "      <td>114.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>11.8</td>\n",
       "      <td>109.0</td>\n",
       "      <td>7.39</td>\n",
       "      <td>51.0</td>\n",
       "      <td>20.0</td>\n",
       "      <td>20.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>6.0</td>\n",
       "      <td>0.5</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.4</td>\n",
       "      <td>25.0</td>\n",
       "      <td>37.8</td>\n",
       "      <td>3.9</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2.2</td>\n",
       "      <td>142.0</td>\n",
       "      <td>44.0</td>\n",
       "      <td>127.0</td>\n",
       "      <td>229.0</td>\n",
       "      <td>113.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>10.6</td>\n",
       "      <td>109.0</td>\n",
       "      <td>7.41</td>\n",
       "      <td>80.0</td>\n",
       "      <td>2580.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>15.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3998</th>\n",
       "      <td>142671</td>\n",
       "      <td>22</td>\n",
       "      <td>10</td>\n",
       "      <td>8</td>\n",
       "      <td>7</td>\n",
       "      <td>1</td>\n",
       "      <td>37.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>87.4</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>88.0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>125.0</td>\n",
       "      <td>74.0</td>\n",
       "      <td>107.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>37.8</td>\n",
       "      <td>82.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>118.0</td>\n",
       "      <td>82.0</td>\n",
       "      <td>98.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>37.4</td>\n",
       "      <td>59.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>118.0</td>\n",
       "      <td>74.0</td>\n",
       "      <td>71.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>37.4</td>\n",
       "      <td>113.0</td>\n",
       "      <td>7.0</td>\n",
       "      <td>140.0</td>\n",
       "      <td>101.0</td>\n",
       "      <td>143.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>39.4</td>\n",
       "      <td>95.0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>130.0</td>\n",
       "      <td>88.0</td>\n",
       "      <td>120.5</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>38.70</td>\n",
       "      <td>176.0</td>\n",
       "      <td>2364.0</td>\n",
       "      <td>2038.0</td>\n",
       "      <td>3.1</td>\n",
       "      <td>65.0</td>\n",
       "      <td>0.9</td>\n",
       "      <td>NaN</td>\n",
       "      <td>7.6</td>\n",
       "      <td>1.0</td>\n",
       "      <td>31.0</td>\n",
       "      <td>28.8</td>\n",
       "      <td>4.1</td>\n",
       "      <td>1.9</td>\n",
       "      <td>2.7</td>\n",
       "      <td>144.0</td>\n",
       "      <td>38.0</td>\n",
       "      <td>114.0</td>\n",
       "      <td>288.0</td>\n",
       "      <td>158.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>14.7</td>\n",
       "      <td>87.4</td>\n",
       "      <td>7.52</td>\n",
       "      <td>158.0</td>\n",
       "      <td>1513.0</td>\n",
       "      <td>1277.0</td>\n",
       "      <td>3.1</td>\n",
       "      <td>114.0</td>\n",
       "      <td>0.6</td>\n",
       "      <td>NaN</td>\n",
       "      <td>11.7</td>\n",
       "      <td>0.5</td>\n",
       "      <td>21.0</td>\n",
       "      <td>27.1</td>\n",
       "      <td>5.0</td>\n",
       "      <td>1.9</td>\n",
       "      <td>3.1</td>\n",
       "      <td>144.0</td>\n",
       "      <td>44.0</td>\n",
       "      <td>190.0</td>\n",
       "      <td>314.0</td>\n",
       "      <td>145.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>11.5</td>\n",
       "      <td>87.4</td>\n",
       "      <td>7.34</td>\n",
       "      <td>82.0</td>\n",
       "      <td>2585.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>15.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3999</th>\n",
       "      <td>142673</td>\n",
       "      <td>25</td>\n",
       "      <td>11</td>\n",
       "      <td>7</td>\n",
       "      <td>-1</td>\n",
       "      <td>0</td>\n",
       "      <td>78.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>157.5</td>\n",
       "      <td>70.7</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>62.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>126.0</td>\n",
       "      <td>88.0</td>\n",
       "      <td>81.0</td>\n",
       "      <td>39.0</td>\n",
       "      <td>52.00</td>\n",
       "      <td>91.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>95.0</td>\n",
       "      <td>34.6</td>\n",
       "      <td>59.0</td>\n",
       "      <td>14.0</td>\n",
       "      <td>126.0</td>\n",
       "      <td>84.0</td>\n",
       "      <td>86.0</td>\n",
       "      <td>66.0</td>\n",
       "      <td>82.00</td>\n",
       "      <td>129.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>98.0</td>\n",
       "      <td>37.3</td>\n",
       "      <td>29.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>126.0</td>\n",
       "      <td>59.0</td>\n",
       "      <td>31.0</td>\n",
       "      <td>39.0</td>\n",
       "      <td>52.00</td>\n",
       "      <td>91.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>95.0</td>\n",
       "      <td>34.6</td>\n",
       "      <td>153.0</td>\n",
       "      <td>15.0</td>\n",
       "      <td>126.0</td>\n",
       "      <td>177.0</td>\n",
       "      <td>191.0</td>\n",
       "      <td>87.0</td>\n",
       "      <td>92.00</td>\n",
       "      <td>143.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>99.0</td>\n",
       "      <td>37.9</td>\n",
       "      <td>54.5</td>\n",
       "      <td>10.0</td>\n",
       "      <td>126.0</td>\n",
       "      <td>87.0</td>\n",
       "      <td>73.0</td>\n",
       "      <td>59.0</td>\n",
       "      <td>76.000</td>\n",
       "      <td>122.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>98.0</td>\n",
       "      <td>36.30</td>\n",
       "      <td>46.0</td>\n",
       "      <td>28.0</td>\n",
       "      <td>153.0</td>\n",
       "      <td>2.2</td>\n",
       "      <td>22.0</td>\n",
       "      <td>0.7</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>24.0</td>\n",
       "      <td>22.3</td>\n",
       "      <td>4.9</td>\n",
       "      <td>1.0</td>\n",
       "      <td>2.1</td>\n",
       "      <td>144.0</td>\n",
       "      <td>45.0</td>\n",
       "      <td>285.0</td>\n",
       "      <td>71.0</td>\n",
       "      <td>117.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>4.8</td>\n",
       "      <td>88.8</td>\n",
       "      <td>7.37</td>\n",
       "      <td>42.0</td>\n",
       "      <td>9.0</td>\n",
       "      <td>99.0</td>\n",
       "      <td>2.2</td>\n",
       "      <td>24.0</td>\n",
       "      <td>0.5</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.5</td>\n",
       "      <td>0.5</td>\n",
       "      <td>19.0</td>\n",
       "      <td>30.7</td>\n",
       "      <td>4.9</td>\n",
       "      <td>1.8</td>\n",
       "      <td>2.1</td>\n",
       "      <td>145.0</td>\n",
       "      <td>40.0</td>\n",
       "      <td>122.0</td>\n",
       "      <td>82.0</td>\n",
       "      <td>129.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>11.0</td>\n",
       "      <td>87.3</td>\n",
       "      <td>7.31</td>\n",
       "      <td>216.0</td>\n",
       "      <td>2295.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>19.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>4000 rows × 120 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "      recordid  SAPS-I  SOFA  Length_of_stay  Survival  In-hospital_death  \\\n",
       "0       132539       6     1               5        -1                  0   \n",
       "1       132540      16     8               8        -1                  0   \n",
       "2       132541      21    11              19        -1                  0   \n",
       "3       132543       7     1               9       575                  0   \n",
       "4       132545      17     2               4       918                  0   \n",
       "...        ...     ...   ...             ...       ...                ...   \n",
       "3995    142665      19     7              10       336                  0   \n",
       "3996    142667       8     2               3        -1                  0   \n",
       "3997    142670       8     5              11        -1                  0   \n",
       "3998    142671      22    10               8         7                  1   \n",
       "3999    142673      25    11               7        -1                  0   \n",
       "\n",
       "       Age  Gender  Height  Weight  CCU  CSRU  SICU  DiasABP_first  GCS_first  \\\n",
       "0     54.0     0.0     NaN     NaN    0     0     1            NaN       15.0   \n",
       "1     76.0     1.0   175.3    76.0    0     1     0           67.0        3.0   \n",
       "2     44.0     0.0     NaN    56.7    0     0     0           81.0        7.0   \n",
       "3     68.0     1.0   180.3    84.6    0     0     0            NaN       15.0   \n",
       "4     88.0     0.0     NaN     NaN    0     0     0            NaN       15.0   \n",
       "...    ...     ...     ...     ...  ...   ...   ...            ...        ...   \n",
       "3995  70.0     0.0     NaN    87.0    0     0     1           50.0        3.0   \n",
       "3996  25.0     1.0     NaN   166.4    0     0     0            NaN       15.0   \n",
       "3997  44.0     1.0     NaN   109.0    0     0     0           67.0        8.0   \n",
       "3998  37.0     1.0     NaN    87.4    0     0     0           88.0        6.0   \n",
       "3999  78.0     0.0   157.5    70.7    0     0     1           62.0        3.0   \n",
       "\n",
       "      Glucose_first  HR_first  MAP_first  NIDiasABP_first  NIMAP_first  \\\n",
       "0             205.0      73.0        NaN             65.0        92.33   \n",
       "1             105.0      88.0       79.0             38.0        49.33   \n",
       "2             141.0     100.0      107.0             84.0       100.30   \n",
       "3             129.0      79.0        NaN             63.0        86.67   \n",
       "4             113.0      93.0        NaN             41.0        75.33   \n",
       "...             ...       ...        ...              ...          ...   \n",
       "3995          121.0      83.0       66.0             47.0        61.33   \n",
       "3996           96.0      81.0        NaN             52.0        75.33   \n",
       "3997           99.0      86.0       81.0             52.0        67.67   \n",
       "3998          125.0      74.0      107.0              NaN          NaN   \n",
       "3999          126.0      88.0       81.0             39.0        52.00   \n",
       "\n",
       "      NISysABP_first  RespRate_first  SaO2_first  Temp_first  DiasABP_last  \\\n",
       "0              147.0            19.0         NaN        35.1           NaN   \n",
       "1               72.0             NaN        99.0        35.2          51.0   \n",
       "2              133.0             NaN        95.0        37.8          70.0   \n",
       "3              134.0            19.0         NaN        36.3           NaN   \n",
       "4              144.0            24.0         NaN        37.8           NaN   \n",
       "...              ...             ...         ...         ...           ...   \n",
       "3995            90.0             NaN         NaN        35.5          79.0   \n",
       "3996           122.0            18.0         NaN        36.8           NaN   \n",
       "3997            99.0             NaN         NaN        37.8          65.0   \n",
       "3998             NaN             NaN         NaN        37.8          82.0   \n",
       "3999            91.0             NaN        95.0        34.6          59.0   \n",
       "\n",
       "      GCS_last  Glucose_last  HR_last  MAP_last  NIDiasABP_last  NIMAP_last  \\\n",
       "0         15.0         115.0     86.0       NaN            55.0       79.33   \n",
       "1         15.0         146.0     65.0      69.0            49.0       68.33   \n",
       "2          5.0         143.0     71.0      92.0            84.0      103.00   \n",
       "3         15.0         117.0     79.0       NaN            81.0       98.67   \n",
       "4         15.0          92.0     68.0       NaN            42.0       73.00   \n",
       "...        ...           ...      ...       ...             ...         ...   \n",
       "3995      15.0         106.0     89.0     109.0            87.0      104.00   \n",
       "3996      15.0          88.0     80.0       NaN            60.0       84.33   \n",
       "3997       5.0         132.0     86.0      79.0            61.0       77.00   \n",
       "3998       3.0         118.0     82.0      98.0             NaN         NaN   \n",
       "3999      14.0         126.0     84.0      86.0            66.0       82.00   \n",
       "\n",
       "      NISysABP_last  RespRate_last  SaO2_last  Temp_last  DiasABP_lowest  \\\n",
       "0             128.0           23.0        NaN       37.8             NaN   \n",
       "1             107.0            NaN       97.0       37.1            32.0   \n",
       "2             141.0            NaN       95.0       37.2            52.0   \n",
       "3             134.0           22.0        NaN       37.0             NaN   \n",
       "4             135.0           16.0        NaN       36.7             NaN   \n",
       "...             ...            ...        ...        ...             ...   \n",
       "3995          138.0            NaN        NaN       36.9            45.0   \n",
       "3996          133.0           18.0        NaN       36.8             NaN   \n",
       "3997          109.0            NaN        NaN       38.4            63.0   \n",
       "3998            NaN            NaN        NaN       37.4            59.0   \n",
       "3999          129.0            NaN       98.0       37.3            29.0   \n",
       "\n",
       "      GCS_lowest  Glucose_lowest  HR_lowest  MAP_lowest  NIDiasABP_lowest  \\\n",
       "0           14.0           115.0       58.0         NaN              39.0   \n",
       "1            3.0           105.0       65.0        43.0              38.0   \n",
       "2            5.0           119.0       57.0        72.0              66.0   \n",
       "3           14.0           106.0       57.0         NaN              45.0   \n",
       "4           15.0            92.0       65.0         NaN              26.0   \n",
       "...          ...             ...        ...         ...               ...   \n",
       "3995         3.0           106.0       83.0        57.0              40.0   \n",
       "3996        15.0            65.0       59.0         NaN              38.0   \n",
       "3997         3.0            97.0       80.0        79.0              40.0   \n",
       "3998         3.0           118.0       74.0        71.0               NaN   \n",
       "3999         3.0           126.0       59.0        31.0              39.0   \n",
       "\n",
       "      NIMAP_lowest  NISysABP_lowest  RespRate_lowest  SaO2_lowest  \\\n",
       "0            58.67             96.0             12.0          NaN   \n",
       "1            49.33             72.0              NaN         93.0   \n",
       "2            83.33            111.0              NaN         95.0   \n",
       "3            68.33            102.0              6.0          NaN   \n",
       "4            52.33            105.0             13.0          NaN   \n",
       "...            ...              ...              ...          ...   \n",
       "3995         60.00             88.0              NaN          NaN   \n",
       "3996         61.67            109.0             14.0          NaN   \n",
       "3997         60.33             94.0              NaN          NaN   \n",
       "3998           NaN              NaN              NaN          NaN   \n",
       "3999         52.00             91.0              NaN         95.0   \n",
       "\n",
       "      Temp_lowest  DiasABP_highest  GCS_highest  Glucose_highest  HR_highest  \\\n",
       "0            35.1              NaN         15.0            205.0        86.0   \n",
       "1            34.5             81.0         15.0            146.0        90.0   \n",
       "2            36.7             81.0          8.0            143.0       113.0   \n",
       "3            35.1              NaN         15.0            129.0        88.0   \n",
       "4            35.8              NaN         15.0            113.0        94.0   \n",
       "...           ...              ...          ...              ...         ...   \n",
       "3995         35.5             82.0         15.0            136.0       113.0   \n",
       "3996         35.5              NaN         15.0            251.0       106.0   \n",
       "3997         37.1             88.0          8.0            132.0       143.0   \n",
       "3998         37.4            113.0          7.0            140.0       101.0   \n",
       "3999         34.6            153.0         15.0            126.0       177.0   \n",
       "\n",
       "      MAP_highest  NIDiasABP_highest  NIMAP_highest  NISysABP_highest  \\\n",
       "0             NaN               67.0          92.33             157.0   \n",
       "1           100.0               67.0          88.33             131.0   \n",
       "2           107.0               95.0         110.00             150.0   \n",
       "3             NaN               81.0         100.70             140.0   \n",
       "4             NaN               96.0         105.70             157.0   \n",
       "...           ...                ...            ...               ...   \n",
       "3995        112.0               87.0         104.00             138.0   \n",
       "3996          NaN               78.0         100.70             151.0   \n",
       "3997        103.0               90.0         108.30             156.0   \n",
       "3998        143.0                NaN            NaN               NaN   \n",
       "3999        191.0               87.0          92.00             143.0   \n",
       "\n",
       "      RespRate_highest  SaO2_highest  Temp_highest  DiasABP_median  \\\n",
       "0                 24.0           NaN          38.2             NaN   \n",
       "1                  NaN          99.0          37.9            59.0   \n",
       "2                  NaN          95.0          39.0            67.0   \n",
       "3                 24.0           NaN          37.0             NaN   \n",
       "4                 27.0           NaN          37.8             NaN   \n",
       "...                ...           ...           ...             ...   \n",
       "3995               NaN           NaN          37.8            68.0   \n",
       "3996              22.0           NaN          37.4             NaN   \n",
       "3997               NaN           NaN          38.4            74.0   \n",
       "3998               NaN           NaN          39.4            95.0   \n",
       "3999               NaN          99.0          37.9            54.5   \n",
       "\n",
       "      GCS_median  Glucose_median  HR_median  MAP_median  NIDiasABP_median  \\\n",
       "0           15.0           160.0       73.0         NaN              49.0   \n",
       "1           15.0           125.5       80.0        79.0              59.0   \n",
       "2            5.0           141.0       85.5        90.0              79.0   \n",
       "3           15.0           117.0       74.0         NaN              64.0   \n",
       "4           15.0           102.5       73.0         NaN              42.0   \n",
       "...          ...             ...        ...         ...               ...   \n",
       "3995        14.0           121.0       99.5        86.5              62.5   \n",
       "3996        15.0            96.0       79.0         NaN              58.5   \n",
       "3997         7.0            99.0       94.5        89.5              60.5   \n",
       "3998         6.0           130.0       88.0       120.5               NaN   \n",
       "3999        10.0           126.0       87.0        73.0              59.0   \n",
       "\n",
       "      NIMAP_median  NISysABP_median  RespRate_median  SaO2_median  \\\n",
       "0           70.000            110.0             18.0          NaN   \n",
       "1           78.165            115.0              NaN         97.0   \n",
       "2           97.670            134.0              NaN         95.0   \n",
       "3           83.670            120.0             16.0          NaN   \n",
       "4           75.330            131.0             19.0          NaN   \n",
       "...            ...              ...              ...          ...   \n",
       "3995        78.000            109.5              NaN          NaN   \n",
       "3996        81.835            125.5             18.0          NaN   \n",
       "3997        78.835            117.0              NaN          NaN   \n",
       "3998           NaN              NaN              NaN          NaN   \n",
       "3999        76.000            122.0              NaN         98.0   \n",
       "\n",
       "      Temp_median  ALP_first  ALT_first  AST_first  Albumin_first  BUN_first  \\\n",
       "0           37.70        NaN        NaN        NaN            NaN       13.0   \n",
       "1           37.45        NaN        NaN        NaN            NaN       16.0   \n",
       "2           37.85      127.0       91.0      235.0            2.7        8.0   \n",
       "3           36.40      105.0       12.0       15.0            4.4       23.0   \n",
       "4           37.00        NaN        NaN        NaN            3.3       45.0   \n",
       "...           ...        ...        ...        ...            ...        ...   \n",
       "3995        37.10        NaN        NaN        NaN            NaN       14.0   \n",
       "3996        36.70        NaN        NaN        NaN            NaN        5.0   \n",
       "3997        37.80       51.0       20.0       20.0            NaN       10.0   \n",
       "3998        38.70      176.0     2364.0     2038.0            3.1       65.0   \n",
       "3999        36.30       46.0       28.0      153.0            2.2       22.0   \n",
       "\n",
       "      Bilirubin_first  Cholesterol_first  Creatinine_first  FiO2_first  \\\n",
       "0                 NaN                NaN               0.8         NaN   \n",
       "1                 NaN                NaN               0.8         1.0   \n",
       "2                 3.0                NaN               0.4         1.0   \n",
       "3                 0.2                NaN               0.9         NaN   \n",
       "4                 NaN                NaN               1.0         NaN   \n",
       "...               ...                ...               ...         ...   \n",
       "3995              NaN                NaN               0.8         0.5   \n",
       "3996              NaN              117.0               0.9         NaN   \n",
       "3997              0.5                NaN               1.2         0.6   \n",
       "3998              0.9                NaN               7.6         1.0   \n",
       "3999              0.7                NaN               1.0         1.0   \n",
       "\n",
       "      HCO3_first  HCT_first  K_first  Lactate_first  Mg_first  Na_first  \\\n",
       "0           26.0       33.7      4.4            NaN       1.5     137.0   \n",
       "1           21.0       24.7      4.3            NaN       3.1     139.0   \n",
       "2           24.0       28.5      3.3            1.3       1.9     137.0   \n",
       "3           28.0       41.3      4.0            NaN       2.1     140.0   \n",
       "4           18.0       22.6      6.0            NaN       1.5     140.0   \n",
       "...          ...        ...      ...            ...       ...       ...   \n",
       "3995        21.0       27.8      2.4            2.3       1.0     138.0   \n",
       "3996        20.0       30.3      3.6            NaN       1.3     141.0   \n",
       "3997        23.0       37.9      4.0            NaN       2.5     140.0   \n",
       "3998        31.0       28.8      4.1            1.9       2.7     144.0   \n",
       "3999        24.0       22.3      4.9            1.0       2.1     144.0   \n",
       "\n",
       "      PaCO2_first  PaO2_first  Platelets_first  SysABP_first  TroponinI_first  \\\n",
       "0             NaN         NaN            221.0           NaN              NaN   \n",
       "1            34.0       344.0            164.0         105.0              NaN   \n",
       "2            37.0        65.0             72.0         148.0              NaN   \n",
       "3             NaN         NaN            391.0           NaN              NaN   \n",
       "4             NaN         NaN            109.0           NaN              NaN   \n",
       "...           ...         ...              ...           ...              ...   \n",
       "3995         40.0       205.0            198.0         108.0              NaN   \n",
       "3996          NaN         NaN            155.0           NaN              NaN   \n",
       "3997         38.0       212.0            220.0         114.0              NaN   \n",
       "3998         38.0       114.0            288.0         158.0              NaN   \n",
       "3999         45.0       285.0             71.0         117.0              NaN   \n",
       "\n",
       "      TroponinT_first  WBC_first  Weight_first  pH_first  ALP_last  ALT_last  \\\n",
       "0                 NaN       11.2           NaN       NaN       NaN       NaN   \n",
       "1                 NaN        7.4          80.6      7.45       NaN       NaN   \n",
       "2                 NaN        4.2          56.7      7.51     105.0      75.0   \n",
       "3                 NaN       11.5          84.6       NaN     105.0      12.0   \n",
       "4                 NaN        3.8           NaN       NaN       NaN       NaN   \n",
       "...               ...        ...           ...       ...       ...       ...   \n",
       "3995              NaN       11.5           NaN      7.42       NaN       NaN   \n",
       "3996              NaN        6.5         166.4       NaN       NaN       NaN   \n",
       "3997              NaN       11.8         109.0      7.39      51.0      20.0   \n",
       "3998              NaN       14.7          87.4      7.52     158.0    1513.0   \n",
       "3999              NaN        4.8          88.8      7.37      42.0       9.0   \n",
       "\n",
       "      AST_last  Albumin_last  BUN_last  Bilirubin_last  Cholesterol_last  \\\n",
       "0          NaN           NaN       8.0             NaN               NaN   \n",
       "1          NaN           NaN      21.0             NaN               NaN   \n",
       "2        164.0           2.3       3.0             2.8               NaN   \n",
       "3         15.0           4.4      10.0             0.2               NaN   \n",
       "4          NaN           3.3      25.0             NaN               NaN   \n",
       "...        ...           ...       ...             ...               ...   \n",
       "3995       NaN           NaN      18.0             NaN               NaN   \n",
       "3996       NaN           NaN       7.0             NaN             117.0   \n",
       "3997      20.0           NaN       6.0             0.5               NaN   \n",
       "3998    1277.0           3.1     114.0             0.6               NaN   \n",
       "3999      99.0           2.2      24.0             0.5               NaN   \n",
       "\n",
       "      Creatinine_last  FiO2_last  HCO3_last  HCT_last  K_last  Lactate_last  \\\n",
       "0                 0.7        NaN       28.0      30.3     4.0           NaN   \n",
       "1                 1.3        0.4       24.0      29.4     3.5           NaN   \n",
       "2                 0.3        0.4       25.0      29.4     3.7           0.9   \n",
       "3                 0.7        NaN       28.0      36.3     3.8           NaN   \n",
       "4                 1.0        NaN       20.0      30.9     4.1           NaN   \n",
       "...               ...        ...        ...       ...     ...           ...   \n",
       "3995              1.0        0.5       22.0      30.3     3.9           1.1   \n",
       "3996              0.7        NaN       28.0      31.9     3.4           NaN   \n",
       "3997              1.0        0.4       25.0      37.8     3.9           NaN   \n",
       "3998             11.7        0.5       21.0      27.1     5.0           1.9   \n",
       "3999              1.5        0.5       19.0      30.7     4.9           1.8   \n",
       "\n",
       "      Mg_last  Na_last  PaCO2_last  PaO2_last  Platelets_last  SysABP_last  \\\n",
       "0         1.9    136.0         NaN        NaN           185.0          NaN   \n",
       "1         1.9    135.0        46.0      140.0           135.0        103.0   \n",
       "2         1.7    138.0        37.0      173.0           113.0        126.0   \n",
       "3         1.9    137.0         NaN        NaN           284.0          NaN   \n",
       "4         1.6    139.0         NaN        NaN            97.0          NaN   \n",
       "...       ...      ...         ...        ...             ...          ...   \n",
       "3995      2.0    136.0        42.0      123.0           218.0        152.0   \n",
       "3996      1.5    142.0         NaN        NaN           118.0          NaN   \n",
       "3997      2.2    142.0        44.0      127.0           229.0        113.0   \n",
       "3998      3.1    144.0        44.0      190.0           314.0        145.0   \n",
       "3999      2.1    145.0        40.0      122.0            82.0        129.0   \n",
       "\n",
       "      TroponinI_last  TroponinT_last  WBC_last  Weight_last  pH_last  \\\n",
       "0                NaN             NaN       9.4          NaN      NaN   \n",
       "1                NaN             NaN      13.3         81.6     7.37   \n",
       "2                NaN             NaN       6.2         56.7     7.47   \n",
       "3                NaN             NaN       7.9         84.6      NaN   \n",
       "4                NaN             NaN       4.8          NaN      NaN   \n",
       "...              ...             ...       ...          ...      ...   \n",
       "3995             NaN             NaN      17.5          NaN     7.39   \n",
       "3996             NaN             NaN       3.0        166.4      NaN   \n",
       "3997             NaN             NaN      10.6        109.0     7.41   \n",
       "3998             NaN             NaN      11.5         87.4     7.34   \n",
       "3999             NaN             NaN      11.0         87.3     7.31   \n",
       "\n",
       "      MechVentStartTime  MechVentDuration  MechVentLast8Hour  UrineOutputSum  \n",
       "0                   NaN               NaN                NaN             NaN  \n",
       "1                  71.0             360.0                0.0             5.0  \n",
       "2                 617.0            2160.0                1.0            14.0  \n",
       "3                   NaN               NaN                NaN             NaN  \n",
       "4                   NaN               NaN                NaN             NaN  \n",
       "...                 ...               ...                ...             ...  \n",
       "3995               23.0            1200.0                0.0             6.0  \n",
       "3996                NaN               NaN                NaN             NaN  \n",
       "3997               80.0            2580.0                1.0            15.0  \n",
       "3998               82.0            2585.0                1.0            15.0  \n",
       "3999              216.0            2295.0                1.0            19.0  \n",
       "\n",
       "[4000 rows x 120 columns]"
      ]
     },
     "execution_count": 70,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.set_option('display.max_columns', None) \n",
    "\n",
    "# load in pre-processed challenge dataset\n",
    "train = pd.read_csv('../challenge2012/data/PhysionetChallenge2012-set-a.csv')\n",
    "train"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "71062d7d-5892-49c1-98ee-00d0536e5342",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "source": [
    "This data has already been pre-processed but it is good practice to still examine our data"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "051bd80e-895e-406e-9fca-ed78b189c088",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "First, let's examine the variables with the highest number of missing values "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "id": "f5f160ce-72de-40e9-b7ae-241927a1dcf6",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "TroponinI_last       3795\n",
       "TroponinI_first      3795\n",
       "Cholesterol_first    3695\n",
       "Cholesterol_last     3695\n",
       "TroponinT_last       3137\n",
       "TroponinT_first      3137\n",
       "RespRate_first       2899\n",
       "RespRate_highest     2899\n",
       "RespRate_lowest      2899\n",
       "RespRate_median      2899\n",
       "dtype: int64"
      ]
     },
     "execution_count": 71,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train.isnull().sum().sort_values(ascending = False)[:10]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ca0fd5b9-5687-458d-a621-febe30abbf45",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "source": [
    "Let's create a separate dataframe to tally the number of missing values and plot the proportion of missing values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "id": "616ce71e-19c3-494b-82d4-2aea66c885e6",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<Axes: >"
      ]
     },
     "execution_count": 72,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAh8AAAGdCAYAAACyzRGfAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8hTgPZAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAfB0lEQVR4nO3df2xV9f3H8dctvVxabHGK9rZSWdHiL4ZzMColDuZsDRp/jCwzqzFodGLQjY64CmObt5sryrIGHcqiccgfdhidOpMh9CaLFe3cCkrGilOnFUGopIq0Ury9tp/vH6b3u9Jaetp734d7fT4S/rinp6fvvlvGc6e93oBzzgkAAMBIlt8DAACALxfiAwAAmCI+AACAKeIDAACYIj4AAIAp4gMAAJgiPgAAgCniAwAAmMr2e4Bj9fX1af/+/crLy1MgEPB7HAAAMALOOXV1damoqEhZWcPf2zjh4mP//v0qLi72ewwAADAKe/fu1ZQpU4Y954SLj7y8PEmfD5+fn5/Ua8fjcTU2NqqyslLBYDCp18bw2L1/2L1/2L1/2L29zs5OFRcXJ/4dH84JFx/9P2rJz89PSXzk5uYqPz+fb0Zj7N4/7N4/7N4/7N4/I/mVCX7hFAAAmCI+AACAKeIDAACYIj4AAIAp4gMAAJgiPgAAgCniAwAAmCI+AACAKeIDAACYIj4AAIApT/ERiUQUCAQG/AmHw4m3O+cUiURUVFSknJwcLViwQK2trUkfGgAApC/Pdz4uuOACHThwIPFn165dibetWbNG9fX1WrdunVpaWhQOh1VRUaGurq6kDg0AANKX5/jIzs5WOBxO/DnttNMkfX7XY+3atVq1apUWLVqkGTNmaOPGjeru7lZDQ0PSBwcAAOnJ86vavvXWWyoqKlIoFFJZWZnq6uo0bdo0tbW1qb29XZWVlYlzQ6GQ5s+fr+bmZi1ZsmTI68ViMcViscTjzs5OSZ+/ImE8Hvc63rD6r5fs6+L42L1/2L1/2L1/2L09L7sOOOfcSE9+/vnn1d3drenTp+uDDz7QPffco//85z9qbW3VG2+8oXnz5un9999XUVFR4n1uvfVW7dmzR1u3bh3ympFIRLW1tYOONzQ0KDc3d8SfCAAA8E93d7eqqqp0+PBh5efnD3uup/g41pEjR3TWWWeppqZGF198sebNm6f9+/ersLAwcc4Pf/hD7d27V1u2bBnyGkPd+SguLlZHR8dxh/cqHo8rGo3qF9uzFOsLJPXaqfTvyOV+jzBm/buvqKhQMBj0e5wvFXbvH3bvH3Zvr7OzU5MnTx5RfHj+scv/mjhxor72ta/prbfe0rXXXitJam9vHxAfBw8eVEFBwRdeIxQKKRQKDToeDAZT9g0T6wso1ps+8ZFJf3FS+XXF8Ni9f9i9f9i9HS97HtN/5yMWi+n1119XYWGhSkpKFA6HFY1GE2/v6elRU1OTysvLx/JhAABABvF05+POO+/UVVddpTPPPFMHDx7UPffco87OTi1evFiBQEDV1dWqq6tTaWmpSktLVVdXp9zcXFVVVaVqfgAAkGY8xce+ffv0gx/8QB0dHTrttNN08cUX65VXXtHUqVMlSTU1NTp69KiWLl2qQ4cOqaysTI2NjcrLy0vJ8AAAIP14io9NmzYN+/ZAIKBIJKJIJDKWmQAAQAbjtV0AAIAp4gMAAJgiPgAAgCniAwAAmCI+AACAKeIDAACYIj4AAIAp4gMAAJgiPgAAgCniAwAAmCI+AACAKeIDAACYIj4AAIAp4gMAAJgiPgAAgCniAwAAmCI+AACAKeIDAACYIj4AAIAp4gMAAJgiPgAAgCniAwAAmCI+AACAKeIDAACYIj4AAIAp4gMAAJgiPgAAgCniAwAAmCI+AACAKeIDAACYIj4AAIAp4gMAAJgiPgAAgCniAwAAmCI+AACAKeIDAACYIj4AAIAp4gMAAJgiPgAAgCniAwAAmCI+AACAKeIDAACYIj4AAIAp4gMAAJgiPgAAgCniAwAAmCI+AACAKeIDAACYIj4AAIAp4gMAAJgiPgAAgCniAwAAmCI+AACAKeIDAACYIj4AAIAp4gMAAJgiPgAAgCniAwAAmCI+AACAqTHFx+rVqxUIBFRdXZ045pxTJBJRUVGRcnJytGDBArW2to51TgAAkCFGHR8tLS16+OGHNXPmzAHH16xZo/r6eq1bt04tLS0Kh8OqqKhQV1fXmIcFAADpb1Tx8cknn+j666/XI488oq985SuJ4845rV27VqtWrdKiRYs0Y8YMbdy4Ud3d3WpoaEja0AAAIH2NKj5uv/12XXnllbrssssGHG9ra1N7e7sqKysTx0KhkObPn6/m5uaxTQoAADJCttd32LRpk1599VW1tLQMelt7e7skqaCgYMDxgoIC7dmzZ8jrxWIxxWKxxOPOzk5JUjweVzwe9zresPqvF8pySb1uqiV7D37o/xwy4XNJN+zeP+zeP+zenpdde4qPvXv3atmyZWpsbNSECRO+8LxAIDDgsXNu0LF+q1evVm1t7aDjjY2Nys3N9TLeiP16dl9Krpsqmzdv9nuEpIlGo36P8KXF7v3D7v3D7u10d3eP+NyAc27EtwGeffZZffe739W4ceMSx3p7exUIBJSVlaU33nhDZ599tl599VVddNFFiXOuueYanXzyydq4ceOgaw5156O4uFgdHR3Kz88f8ScyEvF4XNFoVL/YnqVY39AxdCL6d+Ryv0cYs/7dV1RUKBgM+j3Olwq79w+79w+7t9fZ2anJkyfr8OHDx/3329Odj+985zvatWvXgGM33XSTzj33XN11112aNm2awuGwotFoIj56enrU1NSk++67b8hrhkIhhUKhQceDwWDKvmFifQHFetMnPjLpL04qv64YHrv3D7v3D7u342XPnuIjLy9PM2bMGHBs4sSJOvXUUxPHq6urVVdXp9LSUpWWlqqurk65ubmqqqry8qEAAECG8vwLp8dTU1Ojo0ePaunSpTp06JDKysrU2NiovLy8ZH8oAACQhsYcHy+88MKAx4FAQJFIRJFIZKyXBgAAGYjXdgEAAKaIDwAAYIr4AAAApogPAABgivgAAACmiA8AAGCK+AAAAKaIDwAAYIr4AAAApogPAABgivgAAACmiA8AAGCK+AAAAKaIDwAAYIr4AAAApogPAABgivgAAACmiA8AAGCK+AAAAKaIDwAAYIr4AAAApogPAABgivgAAACmiA8AAGCK+AAAAKaIDwAAYIr4AAAApogPAABgivgAAACmiA8AAGCK+AAAAKaIDwAAYIr4AAAApogPAABgivgAAACmiA8AAGCK+AAAAKaIDwAAYIr4AAAApogPAABgivgAAACmiA8AAGCK+AAAAKaIDwAAYIr4AAAApogPAABgivgAAACmiA8AAGCK+AAAAKaIDwAAYIr4AAAApogPAABgivgAAACmiA8AAGCK+AAAAKaIDwAAYIr4AAAApogPAABgivgAAACmiA8AAGDKU3ysX79eM2fOVH5+vvLz8zV37lw9//zzibc75xSJRFRUVKScnBwtWLBAra2tSR8aAACkL0/xMWXKFN17773avn27tm/frksvvVTXXHNNIjDWrFmj+vp6rVu3Ti0tLQqHw6qoqFBXV1dKhgcAAOnHU3xcddVVuuKKKzR9+nRNnz5dv/nNb3TSSSfplVdekXNOa9eu1apVq7Ro0SLNmDFDGzduVHd3txoaGlI1PwAASDPZo33H3t5ePfnkkzpy5Ijmzp2rtrY2tbe3q7KyMnFOKBTS/Pnz1dzcrCVLlgx5nVgsplgslnjc2dkpSYrH44rH46Mdb0j91wtluaReN9WSvQc/9H8OmfC5pBt27x927x92b8/Lrj3Hx65duzR37lx9+umnOumkk/TMM8/o/PPPV3NzsySpoKBgwPkFBQXas2fPF15v9erVqq2tHXS8sbFRubm5XscbkV/P7kvJdVNl8+bNfo+QNNFo1O8RvrTYvX/YvX/YvZ3u7u4Rn+s5Ps455xzt3LlTH3/8sf785z9r8eLFampqSrw9EAgMON85N+jY/1q5cqWWL1+eeNzZ2ani4mJVVlYqPz/f63jDisfjikaj+sX2LMX6vnimE82/I5f7PcKY9e++oqJCwWDQ73G+VNi9f9i9f9i9vf6fXIyE5/gYP368zj77bEnS7Nmz1dLSovvvv1933XWXJKm9vV2FhYWJ8w8ePDjobsj/CoVCCoVCg44Hg8GUfcPE+gKK9aZPfGTSX5xUfl0xPHbvH3bvH3Zvx8uex/zf+XDOKRaLqaSkROFweMAtrp6eHjU1Nam8vHysHwYAAGQIT3c+fvazn2nhwoUqLi5WV1eXNm3apBdeeEFbtmxRIBBQdXW16urqVFpaqtLSUtXV1Sk3N1dVVVWpmh8AAKQZT/HxwQcf6IYbbtCBAwc0adIkzZw5U1u2bFFFRYUkqaamRkePHtXSpUt16NAhlZWVqbGxUXl5eSkZHgAApB9P8fHoo48O+/ZAIKBIJKJIJDKWmQAAQAbjtV0AAIAp4gMAAJgiPgAAgCniAwAAmCI+AACAKeIDAACYIj4AAIAp4gMAAJgiPgAAgCniAwAAmCI+AACAKeIDAACYIj4AAIAp4gMAAJgiPgAAgCniAwAAmCI+AACAKeIDAACYIj4AAIAp4gMAAJgiPgAAgCniAwAAmCI+AACAKeIDAACYIj4AAIAp4gMAAJgiPgAAgCniAwAAmCI+AACAKeIDAACYIj4AAIAp4gMAAJgiPgAAgCniAwAAmCI+AACAKeIDAACYIj4AAIAp4gMAAJgiPgAAgCniAwAAmCI+AACAKeIDAACYIj4AAIAp4gMAAJgiPgAAgCniAwAAmCI+AACAKeIDAACYIj4AAIAp4gMAAJgiPgAAgCniAwAAmCI+AACAKeIDAACYIj4AAIAp4gMAAJgiPgAAgCniAwAAmCI+AACAKU/xsXr1an3zm99UXl6eTj/9dF177bV64403BpzjnFMkElFRUZFycnK0YMECtba2JnVoAACQvjzFR1NTk26//Xa98sorikaj+uyzz1RZWakjR44kzlmzZo3q6+u1bt06tbS0KBwOq6KiQl1dXUkfHgAApJ9sLydv2bJlwOMNGzbo9NNP144dO/Stb31LzjmtXbtWq1at0qJFiyRJGzduVEFBgRoaGrRkyZLkTQ4AANKSp/g41uHDhyVJp5xyiiSpra1N7e3tqqysTJwTCoU0f/58NTc3DxkfsVhMsVgs8bizs1OSFI/HFY/HxzLeIP3XC2W5pF431ZK9Bz/0fw6Z8LmkG3bvH3bvH3Zvz8uuA865Uf1L7JzTNddco0OHDmnbtm2SpObmZs2bN0/vv/++ioqKEufeeuut2rNnj7Zu3TroOpFIRLW1tYOONzQ0KDc3dzSjAQAAY93d3aqqqtLhw4eVn58/7LmjvvNxxx136F//+pdeeumlQW8LBAIDHjvnBh3rt3LlSi1fvjzxuLOzU8XFxaqsrDzu8F7F43FFo1H9YnuWYn1Dz3Mi+nfkcr9HGLP+3VdUVCgYDPo9zpBmRAbH8YluJN8b6bD7TMXu/cPu7fX/5GIkRhUfP/rRj/Tcc8/pxRdf1JQpUxLHw+GwJKm9vV2FhYWJ4wcPHlRBQcGQ1wqFQgqFQoOOB4PBlH3DxPoCivWmT3xk0l+cVH5dxyqdvif6ednlibz7TMfu/cPu7XjZs6dnuzjndMcdd+jpp5/W3/72N5WUlAx4e0lJicLhsKLRaOJYT0+PmpqaVF5e7uVDAQCADOXpzsftt9+uhoYG/eUvf1FeXp7a29slSZMmTVJOTo4CgYCqq6tVV1en0tJSlZaWqq6uTrm5uaqqqkrJJwAAANKLp/hYv369JGnBggUDjm/YsEE33nijJKmmpkZHjx7V0qVLdejQIZWVlamxsVF5eXlJGRgAAKQ3T/ExkifGBAIBRSIRRSKR0c4EAAAyGK/tAgAATBEfAADAFPEBAABMER8AAMAU8QEAAEwRHwAAwBTxAQAATBEfAADAFPEBAABMER8AAMAU8QEAAEwRHwAAwBTxAQAATBEfAADAFPEBAABMER8AAMAU8QEAAEwRHwAAwBTxAQAATBEfAADAFPEBAABMER8AAMAU8QEAAEwRHwAAwBTxAQAATBEfAADAFPEBAABMER8AAMAU8QEAAExl+z0Aju+rK/7q9wievXvvlX6PAAA4QXHnAwAAmCI+AACAKeIDAACYIj4AAIAp4gMAAJgiPgAAgCniAwAAmCI+AACAKeIDAACYIj4AAIAp4gMAAJgiPgAAgCniAwAAmCI+AACAKeIDAACYIj4AAIAp4gMAAJgiPgAAgCniAwAAmCI+AACAKeIDAACYIj4AAIAp4gMAAJgiPgAAgCniAwAAmCI+AACAKeIDAACYIj4AAIAp4gMAAJgiPgAAgCnP8fHiiy/qqquuUlFRkQKBgJ599tkBb3fOKRKJqKioSDk5OVqwYIFaW1uTNS8AAEhznuPjyJEjuvDCC7Vu3boh375mzRrV19dr3bp1amlpUTgcVkVFhbq6usY8LAAASH/ZXt9h4cKFWrhw4ZBvc85p7dq1WrVqlRYtWiRJ2rhxowoKCtTQ0KAlS5aMbVoAAJD2PMfHcNra2tTe3q7KysrEsVAopPnz56u5uXnI+IjFYorFYonHnZ2dkqR4PK54PJ7M8RLXC2W5pF4Xgx37tet/nOyvaTKFxqXf98VI9pkOu89U7N4/7N6el10nNT7a29slSQUFBQOOFxQUaM+ePUO+z+rVq1VbWzvoeGNjo3Jzc5M5XsKvZ/el5Lr4f5s3bx7yeDQaNZ5k5NbM8XsC775oz0M5kXef6di9f9i9ne7u7hGfm9T46BcIBAY8ds4NOtZv5cqVWr58eeJxZ2eniouLVVlZqfz8/KTOFY/HFY1G9YvtWYr1DT0PkuPfkcsHPO7ffUVFhYLBoE9TDW9GZKvfI3h27J6Hkg67z1Ts3j/s3l7/Ty5GIqnxEQ6HJX1+B6SwsDBx/ODBg4PuhvQLhUIKhUKDjgeDwZR9w8T6Aor1Eh+p9EVfu1R+XccqHb8nvOzyRN59pmP3/mH3drzsOan/nY+SkhKFw+EBt7l6enrU1NSk8vLyZH4oAACQpjzf+fjkk0/03//+N/G4ra1NO3fu1CmnnKIzzzxT1dXVqqurU2lpqUpLS1VXV6fc3FxVVVUldXAAAJCePMfH9u3b9e1vfzvxuP/3NRYvXqzHHntMNTU1Onr0qJYuXapDhw6prKxMjY2NysvLS97UAAAgbXmOjwULFsi5L35KYiAQUCQSUSQSGctcAAAgQ/HaLgAAwBTxAQAATBEfAADAFPEBAABMER8AAMAU8QEAAEwRHwAAwFRKXlgOADLNV1f81fP7hMY5rZnz+YsW+vHaQe/ee6X5xwRGgjsfAADAFPEBAABMER8AAMAU8QEAAEwRHwAAwBTxAQAATPFUWwCmRvOUVQCZhTsfAADAFPEBAABMER8AAMAU8QEAAEwRHwAAwBTxAQAATPFUW6TEsU+n9PvVPQEAJw7ufAAAAFPEBwAAMEV8AAAAU8QHAAAwRXwAAABTxAcAADDFU20BIEOl4ysIv3vvlX6PAAPc+QAAAKaIDwAAYIr4AAAApogPAABgivgAAACmiA8AAGCKp9oCADAGPKXZO+58AAAAU8QHAAAwRXwAAABTxAcAADBFfAAAAFPEBwAAMMVTbYE0NpKn+IXGOa2ZI82IbFWsN2AwFQAMjzsfAADAFPEBAABMER8AAMAU8QEAAEwRHwAAwBTxAQAATPFUWwDACSNZrxDLU8xPbNz5AAAApogPAABgivgAAACmiA8AAGCK+AAAAKaIDwAAYIr4AAAApogPAABgivgAAACmUhYfDz30kEpKSjRhwgTNmjVL27ZtS9WHAgAAaSQl8fHEE0+ourpaq1at0muvvaZLLrlECxcu1HvvvZeKDwcAANJISuKjvr5eN998s2655Radd955Wrt2rYqLi7V+/fpUfDgAAJBGkv7Ccj09PdqxY4dWrFgx4HhlZaWam5sHnR+LxRSLxRKPDx8+LEn66KOPFI/HkzpbPB5Xd3e3suNZ6u3jhYYsZfc5dXf3sXsfsHv/sHv/sPvhffjhh0m/ZldXlyTJOXfcc5MeHx0dHert7VVBQcGA4wUFBWpvbx90/urVq1VbWzvoeElJSbJHg8+q/B7gS4zd+4fd+4fdf7HJv0vdtbu6ujRp0qRhz0l6fPQLBAaWpnNu0DFJWrlypZYvX5543NfXp48++kinnnrqkOePRWdnp4qLi7V3717l5+cn9doYHrv3D7v3D7v3D7u355xTV1eXioqKjntu0uNj8uTJGjdu3KC7HAcPHhx0N0SSQqGQQqHQgGMnn3xysscaID8/n29Gn7B7/7B7/7B7/7B7W8e749Ev6b9wOn78eM2aNUvRaHTA8Wg0qvLy8mR/OAAAkGZS8mOX5cuX64YbbtDs2bM1d+5cPfzww3rvvfd02223peLDAQCANJKS+Ljuuuv04Ycf6le/+pUOHDigGTNmaPPmzZo6dWoqPtyIhUIh3X333YN+zIPUY/f+Yff+Yff+YfcntoAbyXNiAAAAkoTXdgEAAKaIDwAAYIr4AAAApogPAABgKuPi46GHHlJJSYkmTJigWbNmadu2bcOe39TUpFmzZmnChAmaNm2a/vCHPxhNmnm87P7pp59WRUWFTjvtNOXn52vu3LnaunWr4bSZxev3fb+XX35Z2dnZ+vrXv57aATOY193HYjGtWrVKU6dOVSgU0llnnaU//vGPRtNmFq+7f/zxx3XhhRcqNzdXhYWFuummm1LyGicYAZdBNm3a5ILBoHvkkUfc7t273bJly9zEiRPdnj17hjz/nXfecbm5uW7ZsmVu9+7d7pFHHnHBYNA99dRTxpOnP6+7X7ZsmbvvvvvcP//5T/fmm2+6lStXumAw6F599VXjydOf1933+/jjj920adNcZWWlu/DCC22GzTCj2f3VV1/tysrKXDQadW1tbe4f//iHe/nllw2nzgxed79t2zaXlZXl7r//fvfOO++4bdu2uQsuuMBde+21xpPDOecyKj7mzJnjbrvttgHHzj33XLdixYohz6+pqXHnnnvugGNLlixxF198ccpmzFRedz+U888/39XW1iZ7tIw32t1fd9117uc//7m7++67iY9R8rr7559/3k2aNMl9+OGHFuNlNK+7/+1vf+umTZs24NgDDzzgpkyZkrIZ8cUy5scuPT092rFjhyorKwccr6ysVHNz85Dv8/e//33Q+Zdffrm2b9+ueDyeslkzzWh2f6y+vj51dXXplFNOScWIGWu0u9+wYYPefvtt3X333akeMWONZvfPPfecZs+erTVr1uiMM87Q9OnTdeedd+ro0aMWI2eM0ey+vLxc+/bt0+bNm+Wc0wcffKCnnnpKV155pcXIOEbKXtXWWkdHh3p7ewe9eF1BQcGgF7nr197ePuT5n332mTo6OlRYWJiyeTPJaHZ/rN/97nc6cuSIvv/976dixIw1mt2/9dZbWrFihbZt26bs7Iz5nwBzo9n9O++8o5deekkTJkzQM888o46ODi1dulQfffQRv/fhwWh2X15erscff1zXXXedPv30U3322We6+uqr9fvf/95iZBwjY+589AsEAgMeO+cGHTve+UMdx/F53X2/P/3pT4pEInriiSd0+umnp2q8jDbS3ff29qqqqkq1tbWaPn261XgZzcv3fV9fnwKBgB5//HHNmTNHV1xxherr6/XYY49x92MUvOx+9+7d+vGPf6xf/vKX2rFjh7Zs2aK2tjZec8wnGfN/eyZPnqxx48YNqt6DBw8OquN+4XB4yPOzs7N16qmnpmzWTDOa3fd74okndPPNN+vJJ5/UZZddlsoxM5LX3Xd1dWn79u167bXXdMcdd0j6/B9E55yys7PV2NioSy+91GT2dDea7/vCwkKdccYZA152/LzzzpNzTvv27VNpaWlKZ84Uo9n96tWrNW/ePP30pz+VJM2cOVMTJ07UJZdconvuuYc73cYy5s7H+PHjNWvWLEWj0QHHo9GoysvLh3yfuXPnDjq/sbFRs2fPVjAYTNmsmWY0u5c+v+Nx4403qqGhgZ+7jpLX3efn52vXrl3auXNn4s9tt92mc845Rzt37lRZWZnV6GlvNN/38+bN0/79+/XJJ58kjr355pvKysrSlClTUjpvJhnN7ru7u5WVNfCfvHHjxkn6/zveMOTXb7qmQv9Trx599FG3e/duV11d7SZOnOjeffdd55xzK1ascDfccEPi/P6n2v7kJz9xu3fvdo8++ihPtR0lr7tvaGhw2dnZ7sEHH3QHDhxI/Pn444/9+hTSltfdH4tnu4ye1913dXW5KVOmuO9973uutbXVNTU1udLSUnfLLbf49SmkLa+737Bhg8vOznYPPfSQe/vtt91LL73kZs+e7ebMmePXp/ClllHx4ZxzDz74oJs6daobP368+8Y3vuGampoSb1u8eLGbP3/+gPNfeOEFd9FFF7nx48e7r371q279+vXGE2cOL7ufP3++kzToz+LFi+0HzwBev+//F/ExNl53//rrr7vLLrvM5eTkuClTprjly5e77u5u46kzg9fdP/DAA+788893OTk5rrCw0F1//fVu3759xlPDOecCznG/CQAA2MmY3/kAAADpgfgAAACmiA8AAGCK+AAAAKaIDwAAYIr4AAAApogPAABgivgAAACmiA8AAGCK+AAAAKaIDwAAYIr4AAAApv4PZwKyQA4ub28AAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# calculate the proportion of missingness\n",
    "num_missing = train.isnull().sum()\n",
    "num_missing =  num_missing/len(train)\n",
    "\n",
    "num_missing.hist()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d111f5a1-1bf0-4ed8-bd4a-628a7400bbfd",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "**Determine which features to keep in our model**\n",
    "\n",
    "We can see that there are many variables that are largely incomplete. For our purposes, let's first only keep those variables which have <= 25% missing"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "id": "4143bfb4-3edd-44a1-abaf-cc5b1110cd18",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "71"
      ]
     },
     "execution_count": 73,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "vars_low_missing_rate = num_missing[num_missing <= 0.25].index.tolist()\n",
    "\n",
    "len(vars_low_missing_rate)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2a6ae36f-2ae0-47aa-ba39-0d5820c5628a",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "source": [
    "Using the list of columns with a low missing rate, we can subset our `train` data to only include these columns "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "id": "9b0ff507-362d-4613-b313-7c19d4855182",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>recordid</th>\n",
       "      <th>SAPS-I</th>\n",
       "      <th>SOFA</th>\n",
       "      <th>Length_of_stay</th>\n",
       "      <th>Survival</th>\n",
       "      <th>In-hospital_death</th>\n",
       "      <th>Age</th>\n",
       "      <th>Gender</th>\n",
       "      <th>Weight</th>\n",
       "      <th>CCU</th>\n",
       "      <th>CSRU</th>\n",
       "      <th>SICU</th>\n",
       "      <th>GCS_first</th>\n",
       "      <th>Glucose_first</th>\n",
       "      <th>HR_first</th>\n",
       "      <th>NIDiasABP_first</th>\n",
       "      <th>NIMAP_first</th>\n",
       "      <th>NISysABP_first</th>\n",
       "      <th>Temp_first</th>\n",
       "      <th>GCS_last</th>\n",
       "      <th>Glucose_last</th>\n",
       "      <th>HR_last</th>\n",
       "      <th>NIDiasABP_last</th>\n",
       "      <th>NIMAP_last</th>\n",
       "      <th>NISysABP_last</th>\n",
       "      <th>Temp_last</th>\n",
       "      <th>GCS_lowest</th>\n",
       "      <th>Glucose_lowest</th>\n",
       "      <th>HR_lowest</th>\n",
       "      <th>NIDiasABP_lowest</th>\n",
       "      <th>NIMAP_lowest</th>\n",
       "      <th>NISysABP_lowest</th>\n",
       "      <th>Temp_lowest</th>\n",
       "      <th>GCS_highest</th>\n",
       "      <th>Glucose_highest</th>\n",
       "      <th>HR_highest</th>\n",
       "      <th>NIDiasABP_highest</th>\n",
       "      <th>NIMAP_highest</th>\n",
       "      <th>NISysABP_highest</th>\n",
       "      <th>Temp_highest</th>\n",
       "      <th>GCS_median</th>\n",
       "      <th>Glucose_median</th>\n",
       "      <th>HR_median</th>\n",
       "      <th>NIDiasABP_median</th>\n",
       "      <th>NIMAP_median</th>\n",
       "      <th>NISysABP_median</th>\n",
       "      <th>Temp_median</th>\n",
       "      <th>BUN_first</th>\n",
       "      <th>Creatinine_first</th>\n",
       "      <th>HCO3_first</th>\n",
       "      <th>HCT_first</th>\n",
       "      <th>K_first</th>\n",
       "      <th>Mg_first</th>\n",
       "      <th>Na_first</th>\n",
       "      <th>PaCO2_first</th>\n",
       "      <th>PaO2_first</th>\n",
       "      <th>Platelets_first</th>\n",
       "      <th>WBC_first</th>\n",
       "      <th>pH_first</th>\n",
       "      <th>BUN_last</th>\n",
       "      <th>Creatinine_last</th>\n",
       "      <th>HCO3_last</th>\n",
       "      <th>HCT_last</th>\n",
       "      <th>K_last</th>\n",
       "      <th>Mg_last</th>\n",
       "      <th>Na_last</th>\n",
       "      <th>PaCO2_last</th>\n",
       "      <th>PaO2_last</th>\n",
       "      <th>Platelets_last</th>\n",
       "      <th>WBC_last</th>\n",
       "      <th>pH_last</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>132539</td>\n",
       "      <td>6</td>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "      <td>-1</td>\n",
       "      <td>0</td>\n",
       "      <td>54.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>15.0</td>\n",
       "      <td>205.0</td>\n",
       "      <td>73.0</td>\n",
       "      <td>65.0</td>\n",
       "      <td>92.33</td>\n",
       "      <td>147.0</td>\n",
       "      <td>35.1</td>\n",
       "      <td>15.0</td>\n",
       "      <td>115.0</td>\n",
       "      <td>86.0</td>\n",
       "      <td>55.0</td>\n",
       "      <td>79.33</td>\n",
       "      <td>128.0</td>\n",
       "      <td>37.8</td>\n",
       "      <td>14.0</td>\n",
       "      <td>115.0</td>\n",
       "      <td>58.0</td>\n",
       "      <td>39.0</td>\n",
       "      <td>58.67</td>\n",
       "      <td>96.0</td>\n",
       "      <td>35.1</td>\n",
       "      <td>15.0</td>\n",
       "      <td>205.0</td>\n",
       "      <td>86.0</td>\n",
       "      <td>67.0</td>\n",
       "      <td>92.33</td>\n",
       "      <td>157.0</td>\n",
       "      <td>38.2</td>\n",
       "      <td>15.0</td>\n",
       "      <td>160.0</td>\n",
       "      <td>73.0</td>\n",
       "      <td>49.0</td>\n",
       "      <td>70.000</td>\n",
       "      <td>110.0</td>\n",
       "      <td>37.70</td>\n",
       "      <td>13.0</td>\n",
       "      <td>0.8</td>\n",
       "      <td>26.0</td>\n",
       "      <td>33.7</td>\n",
       "      <td>4.4</td>\n",
       "      <td>1.5</td>\n",
       "      <td>137.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>221.0</td>\n",
       "      <td>11.2</td>\n",
       "      <td>NaN</td>\n",
       "      <td>8.0</td>\n",
       "      <td>0.7</td>\n",
       "      <td>28.0</td>\n",
       "      <td>30.3</td>\n",
       "      <td>4.0</td>\n",
       "      <td>1.9</td>\n",
       "      <td>136.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>185.0</td>\n",
       "      <td>9.4</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>132540</td>\n",
       "      <td>16</td>\n",
       "      <td>8</td>\n",
       "      <td>8</td>\n",
       "      <td>-1</td>\n",
       "      <td>0</td>\n",
       "      <td>76.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>76.0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>105.0</td>\n",
       "      <td>88.0</td>\n",
       "      <td>38.0</td>\n",
       "      <td>49.33</td>\n",
       "      <td>72.0</td>\n",
       "      <td>35.2</td>\n",
       "      <td>15.0</td>\n",
       "      <td>146.0</td>\n",
       "      <td>65.0</td>\n",
       "      <td>49.0</td>\n",
       "      <td>68.33</td>\n",
       "      <td>107.0</td>\n",
       "      <td>37.1</td>\n",
       "      <td>3.0</td>\n",
       "      <td>105.0</td>\n",
       "      <td>65.0</td>\n",
       "      <td>38.0</td>\n",
       "      <td>49.33</td>\n",
       "      <td>72.0</td>\n",
       "      <td>34.5</td>\n",
       "      <td>15.0</td>\n",
       "      <td>146.0</td>\n",
       "      <td>90.0</td>\n",
       "      <td>67.0</td>\n",
       "      <td>88.33</td>\n",
       "      <td>131.0</td>\n",
       "      <td>37.9</td>\n",
       "      <td>15.0</td>\n",
       "      <td>125.5</td>\n",
       "      <td>80.0</td>\n",
       "      <td>59.0</td>\n",
       "      <td>78.165</td>\n",
       "      <td>115.0</td>\n",
       "      <td>37.45</td>\n",
       "      <td>16.0</td>\n",
       "      <td>0.8</td>\n",
       "      <td>21.0</td>\n",
       "      <td>24.7</td>\n",
       "      <td>4.3</td>\n",
       "      <td>3.1</td>\n",
       "      <td>139.0</td>\n",
       "      <td>34.0</td>\n",
       "      <td>344.0</td>\n",
       "      <td>164.0</td>\n",
       "      <td>7.4</td>\n",
       "      <td>7.45</td>\n",
       "      <td>21.0</td>\n",
       "      <td>1.3</td>\n",
       "      <td>24.0</td>\n",
       "      <td>29.4</td>\n",
       "      <td>3.5</td>\n",
       "      <td>1.9</td>\n",
       "      <td>135.0</td>\n",
       "      <td>46.0</td>\n",
       "      <td>140.0</td>\n",
       "      <td>135.0</td>\n",
       "      <td>13.3</td>\n",
       "      <td>7.37</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>132541</td>\n",
       "      <td>21</td>\n",
       "      <td>11</td>\n",
       "      <td>19</td>\n",
       "      <td>-1</td>\n",
       "      <td>0</td>\n",
       "      <td>44.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>56.7</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7.0</td>\n",
       "      <td>141.0</td>\n",
       "      <td>100.0</td>\n",
       "      <td>84.0</td>\n",
       "      <td>100.30</td>\n",
       "      <td>133.0</td>\n",
       "      <td>37.8</td>\n",
       "      <td>5.0</td>\n",
       "      <td>143.0</td>\n",
       "      <td>71.0</td>\n",
       "      <td>84.0</td>\n",
       "      <td>103.00</td>\n",
       "      <td>141.0</td>\n",
       "      <td>37.2</td>\n",
       "      <td>5.0</td>\n",
       "      <td>119.0</td>\n",
       "      <td>57.0</td>\n",
       "      <td>66.0</td>\n",
       "      <td>83.33</td>\n",
       "      <td>111.0</td>\n",
       "      <td>36.7</td>\n",
       "      <td>8.0</td>\n",
       "      <td>143.0</td>\n",
       "      <td>113.0</td>\n",
       "      <td>95.0</td>\n",
       "      <td>110.00</td>\n",
       "      <td>150.0</td>\n",
       "      <td>39.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>141.0</td>\n",
       "      <td>85.5</td>\n",
       "      <td>79.0</td>\n",
       "      <td>97.670</td>\n",
       "      <td>134.0</td>\n",
       "      <td>37.85</td>\n",
       "      <td>8.0</td>\n",
       "      <td>0.4</td>\n",
       "      <td>24.0</td>\n",
       "      <td>28.5</td>\n",
       "      <td>3.3</td>\n",
       "      <td>1.9</td>\n",
       "      <td>137.0</td>\n",
       "      <td>37.0</td>\n",
       "      <td>65.0</td>\n",
       "      <td>72.0</td>\n",
       "      <td>4.2</td>\n",
       "      <td>7.51</td>\n",
       "      <td>3.0</td>\n",
       "      <td>0.3</td>\n",
       "      <td>25.0</td>\n",
       "      <td>29.4</td>\n",
       "      <td>3.7</td>\n",
       "      <td>1.7</td>\n",
       "      <td>138.0</td>\n",
       "      <td>37.0</td>\n",
       "      <td>173.0</td>\n",
       "      <td>113.0</td>\n",
       "      <td>6.2</td>\n",
       "      <td>7.47</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>132543</td>\n",
       "      <td>7</td>\n",
       "      <td>1</td>\n",
       "      <td>9</td>\n",
       "      <td>575</td>\n",
       "      <td>0</td>\n",
       "      <td>68.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>84.6</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>15.0</td>\n",
       "      <td>129.0</td>\n",
       "      <td>79.0</td>\n",
       "      <td>63.0</td>\n",
       "      <td>86.67</td>\n",
       "      <td>134.0</td>\n",
       "      <td>36.3</td>\n",
       "      <td>15.0</td>\n",
       "      <td>117.0</td>\n",
       "      <td>79.0</td>\n",
       "      <td>81.0</td>\n",
       "      <td>98.67</td>\n",
       "      <td>134.0</td>\n",
       "      <td>37.0</td>\n",
       "      <td>14.0</td>\n",
       "      <td>106.0</td>\n",
       "      <td>57.0</td>\n",
       "      <td>45.0</td>\n",
       "      <td>68.33</td>\n",
       "      <td>102.0</td>\n",
       "      <td>35.1</td>\n",
       "      <td>15.0</td>\n",
       "      <td>129.0</td>\n",
       "      <td>88.0</td>\n",
       "      <td>81.0</td>\n",
       "      <td>100.70</td>\n",
       "      <td>140.0</td>\n",
       "      <td>37.0</td>\n",
       "      <td>15.0</td>\n",
       "      <td>117.0</td>\n",
       "      <td>74.0</td>\n",
       "      <td>64.0</td>\n",
       "      <td>83.670</td>\n",
       "      <td>120.0</td>\n",
       "      <td>36.40</td>\n",
       "      <td>23.0</td>\n",
       "      <td>0.9</td>\n",
       "      <td>28.0</td>\n",
       "      <td>41.3</td>\n",
       "      <td>4.0</td>\n",
       "      <td>2.1</td>\n",
       "      <td>140.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>391.0</td>\n",
       "      <td>11.5</td>\n",
       "      <td>NaN</td>\n",
       "      <td>10.0</td>\n",
       "      <td>0.7</td>\n",
       "      <td>28.0</td>\n",
       "      <td>36.3</td>\n",
       "      <td>3.8</td>\n",
       "      <td>1.9</td>\n",
       "      <td>137.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>284.0</td>\n",
       "      <td>7.9</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>132545</td>\n",
       "      <td>17</td>\n",
       "      <td>2</td>\n",
       "      <td>4</td>\n",
       "      <td>918</td>\n",
       "      <td>0</td>\n",
       "      <td>88.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>15.0</td>\n",
       "      <td>113.0</td>\n",
       "      <td>93.0</td>\n",
       "      <td>41.0</td>\n",
       "      <td>75.33</td>\n",
       "      <td>144.0</td>\n",
       "      <td>37.8</td>\n",
       "      <td>15.0</td>\n",
       "      <td>92.0</td>\n",
       "      <td>68.0</td>\n",
       "      <td>42.0</td>\n",
       "      <td>73.00</td>\n",
       "      <td>135.0</td>\n",
       "      <td>36.7</td>\n",
       "      <td>15.0</td>\n",
       "      <td>92.0</td>\n",
       "      <td>65.0</td>\n",
       "      <td>26.0</td>\n",
       "      <td>52.33</td>\n",
       "      <td>105.0</td>\n",
       "      <td>35.8</td>\n",
       "      <td>15.0</td>\n",
       "      <td>113.0</td>\n",
       "      <td>94.0</td>\n",
       "      <td>96.0</td>\n",
       "      <td>105.70</td>\n",
       "      <td>157.0</td>\n",
       "      <td>37.8</td>\n",
       "      <td>15.0</td>\n",
       "      <td>102.5</td>\n",
       "      <td>73.0</td>\n",
       "      <td>42.0</td>\n",
       "      <td>75.330</td>\n",
       "      <td>131.0</td>\n",
       "      <td>37.00</td>\n",
       "      <td>45.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>18.0</td>\n",
       "      <td>22.6</td>\n",
       "      <td>6.0</td>\n",
       "      <td>1.5</td>\n",
       "      <td>140.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>109.0</td>\n",
       "      <td>3.8</td>\n",
       "      <td>NaN</td>\n",
       "      <td>25.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>20.0</td>\n",
       "      <td>30.9</td>\n",
       "      <td>4.1</td>\n",
       "      <td>1.6</td>\n",
       "      <td>139.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>97.0</td>\n",
       "      <td>4.8</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   recordid  SAPS-I  SOFA  Length_of_stay  Survival  In-hospital_death   Age  \\\n",
       "0    132539       6     1               5        -1                  0  54.0   \n",
       "1    132540      16     8               8        -1                  0  76.0   \n",
       "2    132541      21    11              19        -1                  0  44.0   \n",
       "3    132543       7     1               9       575                  0  68.0   \n",
       "4    132545      17     2               4       918                  0  88.0   \n",
       "\n",
       "   Gender  Weight  CCU  CSRU  SICU  GCS_first  Glucose_first  HR_first  \\\n",
       "0     0.0     NaN    0     0     1       15.0          205.0      73.0   \n",
       "1     1.0    76.0    0     1     0        3.0          105.0      88.0   \n",
       "2     0.0    56.7    0     0     0        7.0          141.0     100.0   \n",
       "3     1.0    84.6    0     0     0       15.0          129.0      79.0   \n",
       "4     0.0     NaN    0     0     0       15.0          113.0      93.0   \n",
       "\n",
       "   NIDiasABP_first  NIMAP_first  NISysABP_first  Temp_first  GCS_last  \\\n",
       "0             65.0        92.33           147.0        35.1      15.0   \n",
       "1             38.0        49.33            72.0        35.2      15.0   \n",
       "2             84.0       100.30           133.0        37.8       5.0   \n",
       "3             63.0        86.67           134.0        36.3      15.0   \n",
       "4             41.0        75.33           144.0        37.8      15.0   \n",
       "\n",
       "   Glucose_last  HR_last  NIDiasABP_last  NIMAP_last  NISysABP_last  \\\n",
       "0         115.0     86.0            55.0       79.33          128.0   \n",
       "1         146.0     65.0            49.0       68.33          107.0   \n",
       "2         143.0     71.0            84.0      103.00          141.0   \n",
       "3         117.0     79.0            81.0       98.67          134.0   \n",
       "4          92.0     68.0            42.0       73.00          135.0   \n",
       "\n",
       "   Temp_last  GCS_lowest  Glucose_lowest  HR_lowest  NIDiasABP_lowest  \\\n",
       "0       37.8        14.0           115.0       58.0              39.0   \n",
       "1       37.1         3.0           105.0       65.0              38.0   \n",
       "2       37.2         5.0           119.0       57.0              66.0   \n",
       "3       37.0        14.0           106.0       57.0              45.0   \n",
       "4       36.7        15.0            92.0       65.0              26.0   \n",
       "\n",
       "   NIMAP_lowest  NISysABP_lowest  Temp_lowest  GCS_highest  Glucose_highest  \\\n",
       "0         58.67             96.0         35.1         15.0            205.0   \n",
       "1         49.33             72.0         34.5         15.0            146.0   \n",
       "2         83.33            111.0         36.7          8.0            143.0   \n",
       "3         68.33            102.0         35.1         15.0            129.0   \n",
       "4         52.33            105.0         35.8         15.0            113.0   \n",
       "\n",
       "   HR_highest  NIDiasABP_highest  NIMAP_highest  NISysABP_highest  \\\n",
       "0        86.0               67.0          92.33             157.0   \n",
       "1        90.0               67.0          88.33             131.0   \n",
       "2       113.0               95.0         110.00             150.0   \n",
       "3        88.0               81.0         100.70             140.0   \n",
       "4        94.0               96.0         105.70             157.0   \n",
       "\n",
       "   Temp_highest  GCS_median  Glucose_median  HR_median  NIDiasABP_median  \\\n",
       "0          38.2        15.0           160.0       73.0              49.0   \n",
       "1          37.9        15.0           125.5       80.0              59.0   \n",
       "2          39.0         5.0           141.0       85.5              79.0   \n",
       "3          37.0        15.0           117.0       74.0              64.0   \n",
       "4          37.8        15.0           102.5       73.0              42.0   \n",
       "\n",
       "   NIMAP_median  NISysABP_median  Temp_median  BUN_first  Creatinine_first  \\\n",
       "0        70.000            110.0        37.70       13.0               0.8   \n",
       "1        78.165            115.0        37.45       16.0               0.8   \n",
       "2        97.670            134.0        37.85        8.0               0.4   \n",
       "3        83.670            120.0        36.40       23.0               0.9   \n",
       "4        75.330            131.0        37.00       45.0               1.0   \n",
       "\n",
       "   HCO3_first  HCT_first  K_first  Mg_first  Na_first  PaCO2_first  \\\n",
       "0        26.0       33.7      4.4       1.5     137.0          NaN   \n",
       "1        21.0       24.7      4.3       3.1     139.0         34.0   \n",
       "2        24.0       28.5      3.3       1.9     137.0         37.0   \n",
       "3        28.0       41.3      4.0       2.1     140.0          NaN   \n",
       "4        18.0       22.6      6.0       1.5     140.0          NaN   \n",
       "\n",
       "   PaO2_first  Platelets_first  WBC_first  pH_first  BUN_last  \\\n",
       "0         NaN            221.0       11.2       NaN       8.0   \n",
       "1       344.0            164.0        7.4      7.45      21.0   \n",
       "2        65.0             72.0        4.2      7.51       3.0   \n",
       "3         NaN            391.0       11.5       NaN      10.0   \n",
       "4         NaN            109.0        3.8       NaN      25.0   \n",
       "\n",
       "   Creatinine_last  HCO3_last  HCT_last  K_last  Mg_last  Na_last  PaCO2_last  \\\n",
       "0              0.7       28.0      30.3     4.0      1.9    136.0         NaN   \n",
       "1              1.3       24.0      29.4     3.5      1.9    135.0        46.0   \n",
       "2              0.3       25.0      29.4     3.7      1.7    138.0        37.0   \n",
       "3              0.7       28.0      36.3     3.8      1.9    137.0         NaN   \n",
       "4              1.0       20.0      30.9     4.1      1.6    139.0         NaN   \n",
       "\n",
       "   PaO2_last  Platelets_last  WBC_last  pH_last  \n",
       "0        NaN           185.0       9.4      NaN  \n",
       "1      140.0           135.0      13.3     7.37  \n",
       "2      173.0           113.0       6.2     7.47  \n",
       "3        NaN           284.0       7.9      NaN  \n",
       "4        NaN            97.0       4.8      NaN  "
      ]
     },
     "execution_count": 74,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# keep columns with a low missing rate\n",
    "X_vars = train[vars_low_missing_rate]\n",
    "\n",
    "X_vars.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "905a8e64-8a4e-4a33-9d50-9f346f141c0e",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "We can further reduce the number of variables by only choosing the first lab or vital measurements"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "id": "faf9a2ea-9ca4-4204-9aa6-77fe2c243195",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Index(['GCS_first', 'Glucose_first', 'HR_first', 'NIDiasABP_first',\n",
       "       'NIMAP_first', 'NISysABP_first', 'Temp_first', 'BUN_first',\n",
       "       'Creatinine_first', 'HCO3_first', 'HCT_first', 'K_first', 'Mg_first',\n",
       "       'Na_first', 'PaCO2_first', 'PaO2_first', 'Platelets_first', 'WBC_first',\n",
       "       'pH_first'],\n",
       "      dtype='object')"
      ]
     },
     "execution_count": 75,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_first = X_vars.filter(regex='_first$')# keeps columns with the word 'first'\n",
    "X_first.columns"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c7a9db5f-5cd5-48b5-ac76-6e6826da78af",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "source": [
    "We should also consider keeping demographic (`Age`, `Gender`) and baseline information like `Weight` or intensive care unit admitted (`CCU`, `CSRU`, `SICU`) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "id": "57f35692-871e-4435-9138-6646c268ec10",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "X_baseline = X_vars[['Age', 'Gender', 'Weight', 'CCU', 'CSRU', 'SICU']]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d9ec144b-d669-4961-8d45-10f4b6387259",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "source": [
    "Combine maintained set of features"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "id": "d267bc9b-2ece-4a22-9afb-1e63a81a7cb1",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "X_train = pd.merge(X_baseline, \n",
    "                   X_first,\n",
    "                   left_index=True, \n",
    "                   right_index=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "id": "2b0994d2-82ad-4d5c-8178-d775ec809ceb",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Age</th>\n",
       "      <th>Gender</th>\n",
       "      <th>Weight</th>\n",
       "      <th>CCU</th>\n",
       "      <th>CSRU</th>\n",
       "      <th>SICU</th>\n",
       "      <th>GCS_first</th>\n",
       "      <th>Glucose_first</th>\n",
       "      <th>HR_first</th>\n",
       "      <th>NIDiasABP_first</th>\n",
       "      <th>NIMAP_first</th>\n",
       "      <th>NISysABP_first</th>\n",
       "      <th>Temp_first</th>\n",
       "      <th>BUN_first</th>\n",
       "      <th>Creatinine_first</th>\n",
       "      <th>HCO3_first</th>\n",
       "      <th>HCT_first</th>\n",
       "      <th>K_first</th>\n",
       "      <th>Mg_first</th>\n",
       "      <th>Na_first</th>\n",
       "      <th>PaCO2_first</th>\n",
       "      <th>PaO2_first</th>\n",
       "      <th>Platelets_first</th>\n",
       "      <th>WBC_first</th>\n",
       "      <th>pH_first</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>54.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>15.0</td>\n",
       "      <td>205.0</td>\n",
       "      <td>73.0</td>\n",
       "      <td>65.0</td>\n",
       "      <td>92.33</td>\n",
       "      <td>147.0</td>\n",
       "      <td>35.1</td>\n",
       "      <td>13.0</td>\n",
       "      <td>0.8</td>\n",
       "      <td>26.0</td>\n",
       "      <td>33.7</td>\n",
       "      <td>4.4</td>\n",
       "      <td>1.5</td>\n",
       "      <td>137.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>221.0</td>\n",
       "      <td>11.2</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>76.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>76.0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>105.0</td>\n",
       "      <td>88.0</td>\n",
       "      <td>38.0</td>\n",
       "      <td>49.33</td>\n",
       "      <td>72.0</td>\n",
       "      <td>35.2</td>\n",
       "      <td>16.0</td>\n",
       "      <td>0.8</td>\n",
       "      <td>21.0</td>\n",
       "      <td>24.7</td>\n",
       "      <td>4.3</td>\n",
       "      <td>3.1</td>\n",
       "      <td>139.0</td>\n",
       "      <td>34.0</td>\n",
       "      <td>344.0</td>\n",
       "      <td>164.0</td>\n",
       "      <td>7.4</td>\n",
       "      <td>7.45</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>44.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>56.7</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7.0</td>\n",
       "      <td>141.0</td>\n",
       "      <td>100.0</td>\n",
       "      <td>84.0</td>\n",
       "      <td>100.30</td>\n",
       "      <td>133.0</td>\n",
       "      <td>37.8</td>\n",
       "      <td>8.0</td>\n",
       "      <td>0.4</td>\n",
       "      <td>24.0</td>\n",
       "      <td>28.5</td>\n",
       "      <td>3.3</td>\n",
       "      <td>1.9</td>\n",
       "      <td>137.0</td>\n",
       "      <td>37.0</td>\n",
       "      <td>65.0</td>\n",
       "      <td>72.0</td>\n",
       "      <td>4.2</td>\n",
       "      <td>7.51</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>68.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>84.6</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>15.0</td>\n",
       "      <td>129.0</td>\n",
       "      <td>79.0</td>\n",
       "      <td>63.0</td>\n",
       "      <td>86.67</td>\n",
       "      <td>134.0</td>\n",
       "      <td>36.3</td>\n",
       "      <td>23.0</td>\n",
       "      <td>0.9</td>\n",
       "      <td>28.0</td>\n",
       "      <td>41.3</td>\n",
       "      <td>4.0</td>\n",
       "      <td>2.1</td>\n",
       "      <td>140.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>391.0</td>\n",
       "      <td>11.5</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>88.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>15.0</td>\n",
       "      <td>113.0</td>\n",
       "      <td>93.0</td>\n",
       "      <td>41.0</td>\n",
       "      <td>75.33</td>\n",
       "      <td>144.0</td>\n",
       "      <td>37.8</td>\n",
       "      <td>45.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>18.0</td>\n",
       "      <td>22.6</td>\n",
       "      <td>6.0</td>\n",
       "      <td>1.5</td>\n",
       "      <td>140.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>109.0</td>\n",
       "      <td>3.8</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    Age  Gender  Weight  CCU  CSRU  SICU  GCS_first  Glucose_first  HR_first  \\\n",
       "0  54.0     0.0     NaN    0     0     1       15.0          205.0      73.0   \n",
       "1  76.0     1.0    76.0    0     1     0        3.0          105.0      88.0   \n",
       "2  44.0     0.0    56.7    0     0     0        7.0          141.0     100.0   \n",
       "3  68.0     1.0    84.6    0     0     0       15.0          129.0      79.0   \n",
       "4  88.0     0.0     NaN    0     0     0       15.0          113.0      93.0   \n",
       "\n",
       "   NIDiasABP_first  NIMAP_first  NISysABP_first  Temp_first  BUN_first  \\\n",
       "0             65.0        92.33           147.0        35.1       13.0   \n",
       "1             38.0        49.33            72.0        35.2       16.0   \n",
       "2             84.0       100.30           133.0        37.8        8.0   \n",
       "3             63.0        86.67           134.0        36.3       23.0   \n",
       "4             41.0        75.33           144.0        37.8       45.0   \n",
       "\n",
       "   Creatinine_first  HCO3_first  HCT_first  K_first  Mg_first  Na_first  \\\n",
       "0               0.8        26.0       33.7      4.4       1.5     137.0   \n",
       "1               0.8        21.0       24.7      4.3       3.1     139.0   \n",
       "2               0.4        24.0       28.5      3.3       1.9     137.0   \n",
       "3               0.9        28.0       41.3      4.0       2.1     140.0   \n",
       "4               1.0        18.0       22.6      6.0       1.5     140.0   \n",
       "\n",
       "   PaCO2_first  PaO2_first  Platelets_first  WBC_first  pH_first  \n",
       "0          NaN         NaN            221.0       11.2       NaN  \n",
       "1         34.0       344.0            164.0        7.4      7.45  \n",
       "2         37.0        65.0             72.0        4.2      7.51  \n",
       "3          NaN         NaN            391.0       11.5       NaN  \n",
       "4          NaN         NaN            109.0        3.8       NaN  "
      ]
     },
     "execution_count": 78,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_train.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "09e05e04-0f65-4208-9009-81cfd005488a",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "source": [
    "Review number of missing variables once more on the 'cleaner' dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "id": "1ebd21bd-f595-4681-899e-a915403c8e87",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Age                   0\n",
       "CCU                   0\n",
       "SICU                  0\n",
       "CSRU                  0\n",
       "Gender                3\n",
       "HR_first             63\n",
       "Creatinine_first     64\n",
       "GCS_first            64\n",
       "BUN_first            64\n",
       "Temp_first           64\n",
       "HCT_first            64\n",
       "Platelets_first      68\n",
       "Na_first             75\n",
       "HCO3_first           76\n",
       "WBC_first            92\n",
       "K_first              96\n",
       "Mg_first            103\n",
       "Glucose_first       113\n",
       "Weight              331\n",
       "NISysABP_first      515\n",
       "NIDiasABP_first     518\n",
       "NIMAP_first         520\n",
       "pH_first            962\n",
       "PaO2_first          977\n",
       "PaCO2_first         977\n",
       "dtype: int64"
      ]
     },
     "execution_count": 79,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_train.isnull().sum().sort_values()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 80,
   "id": "d5f19121-5458-4851-a1ff-bcafff99e838",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "# we will drop the 3 patients without a gender\n",
    "X_train = X_train.dropna(subset=['Gender'])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "58140154-65bc-46cb-8ca3-9ab8b9247ca0",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "Create `y` dataframe that will only contain our outcome of interest (`in-hospital_death`):\n",
    "\n",
    "- When `in-hospital_death`=1 - indicates the patient died in the hospital\n",
    "- When `in-hospital_death`=0 - indicates the patient did not die in the hospital"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 81,
   "id": "d802a332-6e23-4e7d-942d-055de5fdad33",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0       0\n",
       "1       0\n",
       "2       0\n",
       "3       0\n",
       "4       0\n",
       "       ..\n",
       "3995    0\n",
       "3996    0\n",
       "3997    0\n",
       "3998    1\n",
       "3999    0\n",
       "Name: In-hospital_death, Length: 3997, dtype: int64"
      ]
     },
     "execution_count": 81,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# this step takes our `in-hospital_death` column from the original `train` data and combines it with the cleaned up X_train dataframe\n",
    "# this allows us to remove the 3 patients that we dropped who were missing gender\n",
    "y = pd.merge(train['In-hospital_death'], X_train,\n",
    "             left_index=True, \n",
    "             right_index=True)\n",
    "\n",
    "# select outcome of interest\n",
    "y = y['In-hospital_death']\n",
    "\n",
    "# rename y df\n",
    "y_train = y.copy()\n",
    "\n",
    "y_train"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "813b5a83-69e8-45bc-be32-6f3f87a1b6cd",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "source": [
    "Tally the number of patients meeting each outcome"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 82,
   "id": "be2f6f1e-b2d2-4a2c-9b27-c4450998ea92",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "In-hospital_death\n",
       "0    3444\n",
       "1     553\n",
       "Name: count, dtype: int64"
      ]
     },
     "execution_count": 82,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y_train.value_counts()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c400d017-90b5-42d6-8784-a40a8fa090b0",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "### **Training a [Lasso Logistic Regression](https://scikit-learn.org/dev/modules/linear_model.html#binary-case) Model**\n",
    "\n",
    "Now that are data is clean and we have `X_train` and `y_train` dataframes, we can be start getting ready to train models.\n",
    "\n",
    "We will focus on training a Lasso Logistic regression classifier to predict hospital survival or death.\n",
    "\n",
    "**Lasso** = Least Absolute Shrinkage and Selection Operator \n",
    "\n",
    "<img src=\"lecture_images/logistic_regression.png\" alt=\"title\" width=\"400\" align=\"left\">\n",
    "\n",
    "[Image Source](https://www.javatpoint.com/logistic-regression-in-machine-learning)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f6933686-d748-4f3d-a16a-f65475db6ad8",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "source": [
    "In logistic regression, we are trying to predict the probability of $y=1$ given a set of features $X$. \n",
    "\n",
    "In our case, $y$ represents `in-hospital_death`. \n",
    "\n",
    "Thus, `in-hospital__death` = 1 is equivalent to $y=1$ that the patient died in the hospital"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5f5e6c7a-aae9-41a1-829a-bd1975f83af0",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "source": [
    "For each data point $i$ (or each patient observation):\n",
    "\n",
    "$P(y_i=1|X_i)$\n",
    "\n",
    "In other words, logistic regression allows us to predict the probability that a patient has our outcome of interest (`in-hospital_death`=1) **given** the specific set of $X$ features of that patient."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4ce82bc3-bae3-4157-88c7-0407e86d339e",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "**Other Notes**:\n",
    "\n",
    "* Logistic Regeression is used for classification, thus need a binary outcome variable (e.g. survival or death)\n",
    "* Good baseline model (simple!)\n",
    "* L1 (Lasso) vs L2 (Ridge) regularization - a penalty to avoid overfitting - essentially, shrinking the coefficients of non-informative variables\n",
    "* [scikit-learn documentation](https://scikit-learn.org/dev/modules/generated/sklearn.linear_model.LogisticRegression.html#logisticregression) shows us which parameters we can set when constructing a logistic regression model. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4f6c4f9a-c8ed-4550-a55c-3e3403437dbc",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "### **Setting up a Machine Learning Pipeline**\n",
    "\n",
    "Before, we can start training our model, there are some additional machine learning pre-processing steps we have to take. \n",
    "\n",
    "Fortuntately, scikit-learn has a lot of useful functions - such as `Pipeline()` - to help streamline this process. \n",
    "\n",
    "The code below sets up the pipeline. Essentially, we want our pipeline to do the following:\n",
    "\n",
    "1. Impute any missing values (via the median)\n",
    "2. Standardize numeric variables (mean 0 and standard deviation of 1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 83,
   "id": "4cfaa998-9f07-433b-a42d-5f5d51f37625",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "# specify numeric features to standardize and impute \n",
    "numeric_features = ['Age', 'Weight', 'GCS_first', 'Glucose_first', 'HR_first', 'NIDiasABP_first', 'NIMAP_first',\n",
    "                    'NISysABP_first', 'Temp_first', 'BUN_first', 'Creatinine_first', 'HCO3_first', 'HCT_first', 'K_first', \n",
    "                    'Mg_first', 'Na_first', 'PaCO2_first', 'PaO2_first', 'Platelets_first', 'WBC_first', 'pH_first']\n",
    "\n",
    "# specify non-numeric features\n",
    "non_numeric_features = ['Gender', 'CCU', 'CSRU', 'SICU']\n",
    "\n",
    "# construct pipeline for preprocessing data \n",
    "numeric_transformer = Pipeline(steps=[\n",
    "    ('imputer', SimpleImputer(strategy='median')),\n",
    "    ('scaler', StandardScaler())])\n",
    "\n",
    "non_numeric_transformer = 'passthrough'  # Pass non-numeric features unchanged\n",
    "\n",
    "preprocessor = ColumnTransformer(\n",
    "    transformers=[\n",
    "        ('num', numeric_transformer, numeric_features),\n",
    "        ('non_num', non_numeric_transformer, non_numeric_features)\n",
    "    ]\n",
    ")\n",
    "\n",
    "# Initialize lasso (penalty = 'l1') logistic regression model\n",
    "# class_weight = 'balanced' since the classes of our model (death vs no death) is heavily imbalanced\n",
    "lr = LogisticRegression(solver='liblinear', random_state = 12345, penalty = 'l1', class_weight = 'balanced')\n",
    "\n",
    "# Create a pipeline with preprocessing and a model\n",
    "model_pipeline = Pipeline(steps=[\n",
    "    ('preprocessor', preprocessor),\n",
    "    ('classifier', lr)  # Replace with your model of choice\n",
    "])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "abf367e9-db7d-4ef5-ba25-2e3c8334b7a7",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "source": [
    "These pipeline steps make it possible to prepare our data for model training and tuning. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "621e355a-e220-41f8-8204-dff7c9e2cf8a",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "#### What is model tuning?"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b611fcf7-d5cd-414b-a408-92b817a7e011",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "source": [
    "Machine learning models have different types of hyperparamters which can be changed to try and optimize model performance. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2618540b-8bef-42a9-bea4-212fb85d4d93",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "source": [
    "We won't discuss too much here, but essentially, we often are training a model to minimize an objective function (also referred to as a loss or cost function depending on the context). "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "08837f15-6c93-4b1b-963e-a6eeb363ed23",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "source": [
    "Tuning these parameters is what can help minimize the model's objective function and improve classification performance."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "944aad93-466c-43f1-aaa0-054870ec3edd",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "Additionally, tuning is often performed using **cross-fold validation**. A technique where a dataset is divived into $k$ folds (usually k=5 or k=10). \n",
    "\n",
    "Thus, if $k=5$, we randomly divide the model into 5 equal folds:\n",
    "\n",
    "<img src=\"lecture_images/k_fold_cross_validation.png\" alt=\"title\" width=\"400\" align=\"left\">\n",
    "\n",
    "* 4 folds are used for training with different sets of hyperparameters \n",
    "* The remaining held-out fold is used how the model does on data it hasn't seen before\n",
    "* This process is repeated until each fold is used as the held-out test set once\n",
    "* The optimal hyperparaters are chosen based on the average performance across folds "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8fa44501-b8e8-4b0f-95a4-05ab29faae61",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "#### How do we choose and evaluate the different hyperparameters?"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c08d3c24-5d2a-4177-9db1-399b1d290412",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "source": [
    "In this demonstration, we will evaluate hyperparameters using the scikit-learn function `GridSearch` - this allows us to pre-specify all of the specific hyperparameters we want to evaluate in a grid."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c7ff7144-4eec-45d9-9b59-04bcb9b7abb0",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "source": [
    " During Cross-fold validation, we test all possible hyperparameter combinations in this grid to determine those which provide the best performance across folds"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "063f58dd-2820-4f7a-9c06-e68a3f50903a",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "### Cross-validation and GridSearch\n",
    "\n",
    "Let's set up `GridSearchCV` framework to train and tune our model with 5-fold Cross Validation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 84,
   "id": "83a249bb-ea9d-4bb7-8cbb-2b14eec808c5",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Define parameter grid for model tuning\n",
    "param_grid = {\n",
    "    'classifier__tol': [1e-5, 1e-4, 1e-3], # model training stops once the optimization convergence criteria falls below a certain threshold\n",
    "    'classifier__C': [0.001, 0.01, 0.1, 1, 10, 100]   # Range of regularization strengths (inverse of regularization strength)\n",
    "}\n",
    "\n",
    "# Set up GridSearchCV\n",
    "lr_grid_search = GridSearchCV(estimator=model_pipeline, # specific our pipeline as our estimator \n",
    "                              param_grid=param_grid, # specify the hyperparameter parameter grid\n",
    "                              cv=5, # how many folds to evaluate with cross-fold validation\n",
    "                              scoring='roc_auc', # this specifies that we are going to use the AUC as a metric to evaluate how well our model does across folds\n",
    "                              return_train_score=True) # whether to return the training scores"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "60f0998a-394c-450a-9585-569f3e13e6a3",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "Now that we've specified our pipeline and implemented it into our gridsearch/cross validation, we can train (or fit!) our model to our training data. The code will automatically create our folds for us!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 85,
   "id": "9139bacc-7622-4d35-bc01-78361b541a75",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<style>#sk-container-id-1 {\n",
       "  /* Definition of color scheme common for light and dark mode */\n",
       "  --sklearn-color-text: #000;\n",
       "  --sklearn-color-text-muted: #666;\n",
       "  --sklearn-color-line: gray;\n",
       "  /* Definition of color scheme for unfitted estimators */\n",
       "  --sklearn-color-unfitted-level-0: #fff5e6;\n",
       "  --sklearn-color-unfitted-level-1: #f6e4d2;\n",
       "  --sklearn-color-unfitted-level-2: #ffe0b3;\n",
       "  --sklearn-color-unfitted-level-3: chocolate;\n",
       "  /* Definition of color scheme for fitted estimators */\n",
       "  --sklearn-color-fitted-level-0: #f0f8ff;\n",
       "  --sklearn-color-fitted-level-1: #d4ebff;\n",
       "  --sklearn-color-fitted-level-2: #b3dbfd;\n",
       "  --sklearn-color-fitted-level-3: cornflowerblue;\n",
       "\n",
       "  /* Specific color for light theme */\n",
       "  --sklearn-color-text-on-default-background: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, black)));\n",
       "  --sklearn-color-background: var(--sg-background-color, var(--theme-background, var(--jp-layout-color0, white)));\n",
       "  --sklearn-color-border-box: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, black)));\n",
       "  --sklearn-color-icon: #696969;\n",
       "\n",
       "  @media (prefers-color-scheme: dark) {\n",
       "    /* Redefinition of color scheme for dark theme */\n",
       "    --sklearn-color-text-on-default-background: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, white)));\n",
       "    --sklearn-color-background: var(--sg-background-color, var(--theme-background, var(--jp-layout-color0, #111)));\n",
       "    --sklearn-color-border-box: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, white)));\n",
       "    --sklearn-color-icon: #878787;\n",
       "  }\n",
       "}\n",
       "\n",
       "#sk-container-id-1 {\n",
       "  color: var(--sklearn-color-text);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 pre {\n",
       "  padding: 0;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 input.sk-hidden--visually {\n",
       "  border: 0;\n",
       "  clip: rect(1px 1px 1px 1px);\n",
       "  clip: rect(1px, 1px, 1px, 1px);\n",
       "  height: 1px;\n",
       "  margin: -1px;\n",
       "  overflow: hidden;\n",
       "  padding: 0;\n",
       "  position: absolute;\n",
       "  width: 1px;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-dashed-wrapped {\n",
       "  border: 1px dashed var(--sklearn-color-line);\n",
       "  margin: 0 0.4em 0.5em 0.4em;\n",
       "  box-sizing: border-box;\n",
       "  padding-bottom: 0.4em;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-container {\n",
       "  /* jupyter's `normalize.less` sets `[hidden] { display: none; }`\n",
       "     but bootstrap.min.css set `[hidden] { display: none !important; }`\n",
       "     so we also need the `!important` here to be able to override the\n",
       "     default hidden behavior on the sphinx rendered scikit-learn.org.\n",
       "     See: https://github.com/scikit-learn/scikit-learn/issues/21755 */\n",
       "  display: inline-block !important;\n",
       "  position: relative;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-text-repr-fallback {\n",
       "  display: none;\n",
       "}\n",
       "\n",
       "div.sk-parallel-item,\n",
       "div.sk-serial,\n",
       "div.sk-item {\n",
       "  /* draw centered vertical line to link estimators */\n",
       "  background-image: linear-gradient(var(--sklearn-color-text-on-default-background), var(--sklearn-color-text-on-default-background));\n",
       "  background-size: 2px 100%;\n",
       "  background-repeat: no-repeat;\n",
       "  background-position: center center;\n",
       "}\n",
       "\n",
       "/* Parallel-specific style estimator block */\n",
       "\n",
       "#sk-container-id-1 div.sk-parallel-item::after {\n",
       "  content: \"\";\n",
       "  width: 100%;\n",
       "  border-bottom: 2px solid var(--sklearn-color-text-on-default-background);\n",
       "  flex-grow: 1;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-parallel {\n",
       "  display: flex;\n",
       "  align-items: stretch;\n",
       "  justify-content: center;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "  position: relative;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-parallel-item {\n",
       "  display: flex;\n",
       "  flex-direction: column;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-parallel-item:first-child::after {\n",
       "  align-self: flex-end;\n",
       "  width: 50%;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-parallel-item:last-child::after {\n",
       "  align-self: flex-start;\n",
       "  width: 50%;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-parallel-item:only-child::after {\n",
       "  width: 0;\n",
       "}\n",
       "\n",
       "/* Serial-specific style estimator block */\n",
       "\n",
       "#sk-container-id-1 div.sk-serial {\n",
       "  display: flex;\n",
       "  flex-direction: column;\n",
       "  align-items: center;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "  padding-right: 1em;\n",
       "  padding-left: 1em;\n",
       "}\n",
       "\n",
       "\n",
       "/* Toggleable style: style used for estimator/Pipeline/ColumnTransformer box that is\n",
       "clickable and can be expanded/collapsed.\n",
       "- Pipeline and ColumnTransformer use this feature and define the default style\n",
       "- Estimators will overwrite some part of the style using the `sk-estimator` class\n",
       "*/\n",
       "\n",
       "/* Pipeline and ColumnTransformer style (default) */\n",
       "\n",
       "#sk-container-id-1 div.sk-toggleable {\n",
       "  /* Default theme specific background. It is overwritten whether we have a\n",
       "  specific estimator or a Pipeline/ColumnTransformer */\n",
       "  background-color: var(--sklearn-color-background);\n",
       "}\n",
       "\n",
       "/* Toggleable label */\n",
       "#sk-container-id-1 label.sk-toggleable__label {\n",
       "  cursor: pointer;\n",
       "  display: flex;\n",
       "  width: 100%;\n",
       "  margin-bottom: 0;\n",
       "  padding: 0.5em;\n",
       "  box-sizing: border-box;\n",
       "  text-align: center;\n",
       "  align-items: start;\n",
       "  justify-content: space-between;\n",
       "  gap: 0.5em;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 label.sk-toggleable__label .caption {\n",
       "  font-size: 0.6rem;\n",
       "  font-weight: lighter;\n",
       "  color: var(--sklearn-color-text-muted);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 label.sk-toggleable__label-arrow:before {\n",
       "  /* Arrow on the left of the label */\n",
       "  content: \"▸\";\n",
       "  float: left;\n",
       "  margin-right: 0.25em;\n",
       "  color: var(--sklearn-color-icon);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 label.sk-toggleable__label-arrow:hover:before {\n",
       "  color: var(--sklearn-color-text);\n",
       "}\n",
       "\n",
       "/* Toggleable content - dropdown */\n",
       "\n",
       "#sk-container-id-1 div.sk-toggleable__content {\n",
       "  max-height: 0;\n",
       "  max-width: 0;\n",
       "  overflow: hidden;\n",
       "  text-align: left;\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-toggleable__content.fitted {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-toggleable__content pre {\n",
       "  margin: 0.2em;\n",
       "  border-radius: 0.25em;\n",
       "  color: var(--sklearn-color-text);\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-toggleable__content.fitted pre {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 input.sk-toggleable__control:checked~div.sk-toggleable__content {\n",
       "  /* Expand drop-down */\n",
       "  max-height: 200px;\n",
       "  max-width: 100%;\n",
       "  overflow: auto;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {\n",
       "  content: \"▾\";\n",
       "}\n",
       "\n",
       "/* Pipeline/ColumnTransformer-specific style */\n",
       "\n",
       "#sk-container-id-1 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
       "  color: var(--sklearn-color-text);\n",
       "  background-color: var(--sklearn-color-unfitted-level-2);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-label.fitted input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
       "  background-color: var(--sklearn-color-fitted-level-2);\n",
       "}\n",
       "\n",
       "/* Estimator-specific style */\n",
       "\n",
       "/* Colorize estimator box */\n",
       "#sk-container-id-1 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-2);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-estimator.fitted input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-2);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-label label.sk-toggleable__label,\n",
       "#sk-container-id-1 div.sk-label label {\n",
       "  /* The background is the default theme color */\n",
       "  color: var(--sklearn-color-text-on-default-background);\n",
       "}\n",
       "\n",
       "/* On hover, darken the color of the background */\n",
       "#sk-container-id-1 div.sk-label:hover label.sk-toggleable__label {\n",
       "  color: var(--sklearn-color-text);\n",
       "  background-color: var(--sklearn-color-unfitted-level-2);\n",
       "}\n",
       "\n",
       "/* Label box, darken color on hover, fitted */\n",
       "#sk-container-id-1 div.sk-label.fitted:hover label.sk-toggleable__label.fitted {\n",
       "  color: var(--sklearn-color-text);\n",
       "  background-color: var(--sklearn-color-fitted-level-2);\n",
       "}\n",
       "\n",
       "/* Estimator label */\n",
       "\n",
       "#sk-container-id-1 div.sk-label label {\n",
       "  font-family: monospace;\n",
       "  font-weight: bold;\n",
       "  display: inline-block;\n",
       "  line-height: 1.2em;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-label-container {\n",
       "  text-align: center;\n",
       "}\n",
       "\n",
       "/* Estimator-specific */\n",
       "#sk-container-id-1 div.sk-estimator {\n",
       "  font-family: monospace;\n",
       "  border: 1px dotted var(--sklearn-color-border-box);\n",
       "  border-radius: 0.25em;\n",
       "  box-sizing: border-box;\n",
       "  margin-bottom: 0.5em;\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-estimator.fitted {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-0);\n",
       "}\n",
       "\n",
       "/* on hover */\n",
       "#sk-container-id-1 div.sk-estimator:hover {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-2);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-estimator.fitted:hover {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-2);\n",
       "}\n",
       "\n",
       "/* Specification for estimator info (e.g. \"i\" and \"?\") */\n",
       "\n",
       "/* Common style for \"i\" and \"?\" */\n",
       "\n",
       ".sk-estimator-doc-link,\n",
       "a:link.sk-estimator-doc-link,\n",
       "a:visited.sk-estimator-doc-link {\n",
       "  float: right;\n",
       "  font-size: smaller;\n",
       "  line-height: 1em;\n",
       "  font-family: monospace;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "  border-radius: 1em;\n",
       "  height: 1em;\n",
       "  width: 1em;\n",
       "  text-decoration: none !important;\n",
       "  margin-left: 0.5em;\n",
       "  text-align: center;\n",
       "  /* unfitted */\n",
       "  border: var(--sklearn-color-unfitted-level-1) 1pt solid;\n",
       "  color: var(--sklearn-color-unfitted-level-1);\n",
       "}\n",
       "\n",
       ".sk-estimator-doc-link.fitted,\n",
       "a:link.sk-estimator-doc-link.fitted,\n",
       "a:visited.sk-estimator-doc-link.fitted {\n",
       "  /* fitted */\n",
       "  border: var(--sklearn-color-fitted-level-1) 1pt solid;\n",
       "  color: var(--sklearn-color-fitted-level-1);\n",
       "}\n",
       "\n",
       "/* On hover */\n",
       "div.sk-estimator:hover .sk-estimator-doc-link:hover,\n",
       ".sk-estimator-doc-link:hover,\n",
       "div.sk-label-container:hover .sk-estimator-doc-link:hover,\n",
       ".sk-estimator-doc-link:hover {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-3);\n",
       "  color: var(--sklearn-color-background);\n",
       "  text-decoration: none;\n",
       "}\n",
       "\n",
       "div.sk-estimator.fitted:hover .sk-estimator-doc-link.fitted:hover,\n",
       ".sk-estimator-doc-link.fitted:hover,\n",
       "div.sk-label-container:hover .sk-estimator-doc-link.fitted:hover,\n",
       ".sk-estimator-doc-link.fitted:hover {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-3);\n",
       "  color: var(--sklearn-color-background);\n",
       "  text-decoration: none;\n",
       "}\n",
       "\n",
       "/* Span, style for the box shown on hovering the info icon */\n",
       ".sk-estimator-doc-link span {\n",
       "  display: none;\n",
       "  z-index: 9999;\n",
       "  position: relative;\n",
       "  font-weight: normal;\n",
       "  right: .2ex;\n",
       "  padding: .5ex;\n",
       "  margin: .5ex;\n",
       "  width: min-content;\n",
       "  min-width: 20ex;\n",
       "  max-width: 50ex;\n",
       "  color: var(--sklearn-color-text);\n",
       "  box-shadow: 2pt 2pt 4pt #999;\n",
       "  /* unfitted */\n",
       "  background: var(--sklearn-color-unfitted-level-0);\n",
       "  border: .5pt solid var(--sklearn-color-unfitted-level-3);\n",
       "}\n",
       "\n",
       ".sk-estimator-doc-link.fitted span {\n",
       "  /* fitted */\n",
       "  background: var(--sklearn-color-fitted-level-0);\n",
       "  border: var(--sklearn-color-fitted-level-3);\n",
       "}\n",
       "\n",
       ".sk-estimator-doc-link:hover span {\n",
       "  display: block;\n",
       "}\n",
       "\n",
       "/* \"?\"-specific style due to the `<a>` HTML tag */\n",
       "\n",
       "#sk-container-id-1 a.estimator_doc_link {\n",
       "  float: right;\n",
       "  font-size: 1rem;\n",
       "  line-height: 1em;\n",
       "  font-family: monospace;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "  border-radius: 1rem;\n",
       "  height: 1rem;\n",
       "  width: 1rem;\n",
       "  text-decoration: none;\n",
       "  /* unfitted */\n",
       "  color: var(--sklearn-color-unfitted-level-1);\n",
       "  border: var(--sklearn-color-unfitted-level-1) 1pt solid;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 a.estimator_doc_link.fitted {\n",
       "  /* fitted */\n",
       "  border: var(--sklearn-color-fitted-level-1) 1pt solid;\n",
       "  color: var(--sklearn-color-fitted-level-1);\n",
       "}\n",
       "\n",
       "/* On hover */\n",
       "#sk-container-id-1 a.estimator_doc_link:hover {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-3);\n",
       "  color: var(--sklearn-color-background);\n",
       "  text-decoration: none;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 a.estimator_doc_link.fitted:hover {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-3);\n",
       "}\n",
       "</style><div id=\"sk-container-id-1\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>GridSearchCV(cv=5,\n",
       "             estimator=Pipeline(steps=[(&#x27;preprocessor&#x27;,\n",
       "                                        ColumnTransformer(transformers=[(&#x27;num&#x27;,\n",
       "                                                                         Pipeline(steps=[(&#x27;imputer&#x27;,\n",
       "                                                                                          SimpleImputer(strategy=&#x27;median&#x27;)),\n",
       "                                                                                         (&#x27;scaler&#x27;,\n",
       "                                                                                          StandardScaler())]),\n",
       "                                                                         [&#x27;Age&#x27;,\n",
       "                                                                          &#x27;Weight&#x27;,\n",
       "                                                                          &#x27;GCS_first&#x27;,\n",
       "                                                                          &#x27;Glucose_first&#x27;,\n",
       "                                                                          &#x27;HR_first&#x27;,\n",
       "                                                                          &#x27;NIDiasABP_first&#x27;,\n",
       "                                                                          &#x27;NIMAP_first&#x27;,\n",
       "                                                                          &#x27;NISysABP_first&#x27;,\n",
       "                                                                          &#x27;Temp_first&#x27;,\n",
       "                                                                          &#x27;BUN_first&#x27;,\n",
       "                                                                          &#x27;Creatinine_first&#x27;,\n",
       "                                                                          &#x27;HCO3_first&#x27;...\n",
       "                                                                          &#x27;Platelets_first&#x27;,\n",
       "                                                                          &#x27;WBC_first&#x27;,\n",
       "                                                                          &#x27;pH_first&#x27;]),\n",
       "                                                                        (&#x27;non_num&#x27;,\n",
       "                                                                         &#x27;passthrough&#x27;,\n",
       "                                                                         [&#x27;Gender&#x27;,\n",
       "                                                                          &#x27;CCU&#x27;,\n",
       "                                                                          &#x27;CSRU&#x27;,\n",
       "                                                                          &#x27;SICU&#x27;])])),\n",
       "                                       (&#x27;classifier&#x27;,\n",
       "                                        LogisticRegression(class_weight=&#x27;balanced&#x27;,\n",
       "                                                           penalty=&#x27;l1&#x27;,\n",
       "                                                           random_state=12345,\n",
       "                                                           solver=&#x27;liblinear&#x27;))]),\n",
       "             param_grid={&#x27;classifier__C&#x27;: [0.001, 0.01, 0.1, 1, 10, 100],\n",
       "                         &#x27;classifier__tol&#x27;: [1e-05, 0.0001, 0.001]},\n",
       "             return_train_score=True, scoring=&#x27;roc_auc&#x27;)</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item sk-dashed-wrapped\"><div class=\"sk-label-container\"><div class=\"sk-label fitted sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-1\" type=\"checkbox\" ><label for=\"sk-estimator-id-1\" class=\"sk-toggleable__label fitted sk-toggleable__label-arrow\"><div><div>GridSearchCV</div></div><div><a class=\"sk-estimator-doc-link fitted\" rel=\"noreferrer\" target=\"_blank\" href=\"https://scikit-learn.org/1.6/modules/generated/sklearn.model_selection.GridSearchCV.html\">?<span>Documentation for GridSearchCV</span></a><span class=\"sk-estimator-doc-link fitted\">i<span>Fitted</span></span></div></label><div class=\"sk-toggleable__content fitted\"><pre>GridSearchCV(cv=5,\n",
       "             estimator=Pipeline(steps=[(&#x27;preprocessor&#x27;,\n",
       "                                        ColumnTransformer(transformers=[(&#x27;num&#x27;,\n",
       "                                                                         Pipeline(steps=[(&#x27;imputer&#x27;,\n",
       "                                                                                          SimpleImputer(strategy=&#x27;median&#x27;)),\n",
       "                                                                                         (&#x27;scaler&#x27;,\n",
       "                                                                                          StandardScaler())]),\n",
       "                                                                         [&#x27;Age&#x27;,\n",
       "                                                                          &#x27;Weight&#x27;,\n",
       "                                                                          &#x27;GCS_first&#x27;,\n",
       "                                                                          &#x27;Glucose_first&#x27;,\n",
       "                                                                          &#x27;HR_first&#x27;,\n",
       "                                                                          &#x27;NIDiasABP_first&#x27;,\n",
       "                                                                          &#x27;NIMAP_first&#x27;,\n",
       "                                                                          &#x27;NISysABP_first&#x27;,\n",
       "                                                                          &#x27;Temp_first&#x27;,\n",
       "                                                                          &#x27;BUN_first&#x27;,\n",
       "                                                                          &#x27;Creatinine_first&#x27;,\n",
       "                                                                          &#x27;HCO3_first&#x27;...\n",
       "                                                                          &#x27;Platelets_first&#x27;,\n",
       "                                                                          &#x27;WBC_first&#x27;,\n",
       "                                                                          &#x27;pH_first&#x27;]),\n",
       "                                                                        (&#x27;non_num&#x27;,\n",
       "                                                                         &#x27;passthrough&#x27;,\n",
       "                                                                         [&#x27;Gender&#x27;,\n",
       "                                                                          &#x27;CCU&#x27;,\n",
       "                                                                          &#x27;CSRU&#x27;,\n",
       "                                                                          &#x27;SICU&#x27;])])),\n",
       "                                       (&#x27;classifier&#x27;,\n",
       "                                        LogisticRegression(class_weight=&#x27;balanced&#x27;,\n",
       "                                                           penalty=&#x27;l1&#x27;,\n",
       "                                                           random_state=12345,\n",
       "                                                           solver=&#x27;liblinear&#x27;))]),\n",
       "             param_grid={&#x27;classifier__C&#x27;: [0.001, 0.01, 0.1, 1, 10, 100],\n",
       "                         &#x27;classifier__tol&#x27;: [1e-05, 0.0001, 0.001]},\n",
       "             return_train_score=True, scoring=&#x27;roc_auc&#x27;)</pre></div> </div></div><div class=\"sk-parallel\"><div class=\"sk-parallel-item\"><div class=\"sk-item\"><div class=\"sk-label-container\"><div class=\"sk-label fitted sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-2\" type=\"checkbox\" ><label for=\"sk-estimator-id-2\" class=\"sk-toggleable__label fitted sk-toggleable__label-arrow\"><div><div>best_estimator_: Pipeline</div></div></label><div class=\"sk-toggleable__content fitted\"><pre>Pipeline(steps=[(&#x27;preprocessor&#x27;,\n",
       "                 ColumnTransformer(transformers=[(&#x27;num&#x27;,\n",
       "                                                  Pipeline(steps=[(&#x27;imputer&#x27;,\n",
       "                                                                   SimpleImputer(strategy=&#x27;median&#x27;)),\n",
       "                                                                  (&#x27;scaler&#x27;,\n",
       "                                                                   StandardScaler())]),\n",
       "                                                  [&#x27;Age&#x27;, &#x27;Weight&#x27;, &#x27;GCS_first&#x27;,\n",
       "                                                   &#x27;Glucose_first&#x27;, &#x27;HR_first&#x27;,\n",
       "                                                   &#x27;NIDiasABP_first&#x27;,\n",
       "                                                   &#x27;NIMAP_first&#x27;,\n",
       "                                                   &#x27;NISysABP_first&#x27;,\n",
       "                                                   &#x27;Temp_first&#x27;, &#x27;BUN_first&#x27;,\n",
       "                                                   &#x27;Creatinine_first&#x27;,\n",
       "                                                   &#x27;HCO3_first&#x27;, &#x27;HCT_first&#x27;,\n",
       "                                                   &#x27;K_first&#x27;, &#x27;Mg_first&#x27;,\n",
       "                                                   &#x27;Na_first&#x27;, &#x27;PaCO2_first&#x27;,\n",
       "                                                   &#x27;PaO2_first&#x27;,\n",
       "                                                   &#x27;Platelets_first&#x27;,\n",
       "                                                   &#x27;WBC_first&#x27;, &#x27;pH_first&#x27;]),\n",
       "                                                 (&#x27;non_num&#x27;, &#x27;passthrough&#x27;,\n",
       "                                                  [&#x27;Gender&#x27;, &#x27;CCU&#x27;, &#x27;CSRU&#x27;,\n",
       "                                                   &#x27;SICU&#x27;])])),\n",
       "                (&#x27;classifier&#x27;,\n",
       "                 LogisticRegression(C=1, class_weight=&#x27;balanced&#x27;, penalty=&#x27;l1&#x27;,\n",
       "                                    random_state=12345, solver=&#x27;liblinear&#x27;,\n",
       "                                    tol=0.001))])</pre></div> </div></div><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-serial\"><div class=\"sk-item sk-dashed-wrapped\"><div class=\"sk-label-container\"><div class=\"sk-label fitted sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-3\" type=\"checkbox\" ><label for=\"sk-estimator-id-3\" class=\"sk-toggleable__label fitted sk-toggleable__label-arrow\"><div><div>preprocessor: ColumnTransformer</div></div><div><a class=\"sk-estimator-doc-link fitted\" rel=\"noreferrer\" target=\"_blank\" href=\"https://scikit-learn.org/1.6/modules/generated/sklearn.compose.ColumnTransformer.html\">?<span>Documentation for preprocessor: ColumnTransformer</span></a></div></label><div class=\"sk-toggleable__content fitted\"><pre>ColumnTransformer(transformers=[(&#x27;num&#x27;,\n",
       "                                 Pipeline(steps=[(&#x27;imputer&#x27;,\n",
       "                                                  SimpleImputer(strategy=&#x27;median&#x27;)),\n",
       "                                                 (&#x27;scaler&#x27;, StandardScaler())]),\n",
       "                                 [&#x27;Age&#x27;, &#x27;Weight&#x27;, &#x27;GCS_first&#x27;, &#x27;Glucose_first&#x27;,\n",
       "                                  &#x27;HR_first&#x27;, &#x27;NIDiasABP_first&#x27;, &#x27;NIMAP_first&#x27;,\n",
       "                                  &#x27;NISysABP_first&#x27;, &#x27;Temp_first&#x27;, &#x27;BUN_first&#x27;,\n",
       "                                  &#x27;Creatinine_first&#x27;, &#x27;HCO3_first&#x27;, &#x27;HCT_first&#x27;,\n",
       "                                  &#x27;K_first&#x27;, &#x27;Mg_first&#x27;, &#x27;Na_first&#x27;,\n",
       "                                  &#x27;PaCO2_first&#x27;, &#x27;PaO2_first&#x27;,\n",
       "                                  &#x27;Platelets_first&#x27;, &#x27;WBC_first&#x27;, &#x27;pH_first&#x27;]),\n",
       "                                (&#x27;non_num&#x27;, &#x27;passthrough&#x27;,\n",
       "                                 [&#x27;Gender&#x27;, &#x27;CCU&#x27;, &#x27;CSRU&#x27;, &#x27;SICU&#x27;])])</pre></div> </div></div><div class=\"sk-parallel\"><div class=\"sk-parallel-item\"><div class=\"sk-item\"><div class=\"sk-label-container\"><div class=\"sk-label fitted sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-4\" type=\"checkbox\" ><label for=\"sk-estimator-id-4\" class=\"sk-toggleable__label fitted sk-toggleable__label-arrow\"><div><div>num</div></div></label><div class=\"sk-toggleable__content fitted\"><pre>[&#x27;Age&#x27;, &#x27;Weight&#x27;, &#x27;GCS_first&#x27;, &#x27;Glucose_first&#x27;, &#x27;HR_first&#x27;, &#x27;NIDiasABP_first&#x27;, &#x27;NIMAP_first&#x27;, &#x27;NISysABP_first&#x27;, &#x27;Temp_first&#x27;, &#x27;BUN_first&#x27;, &#x27;Creatinine_first&#x27;, &#x27;HCO3_first&#x27;, &#x27;HCT_first&#x27;, &#x27;K_first&#x27;, &#x27;Mg_first&#x27;, &#x27;Na_first&#x27;, &#x27;PaCO2_first&#x27;, &#x27;PaO2_first&#x27;, &#x27;Platelets_first&#x27;, &#x27;WBC_first&#x27;, &#x27;pH_first&#x27;]</pre></div> </div></div><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-estimator fitted sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-5\" type=\"checkbox\" ><label for=\"sk-estimator-id-5\" class=\"sk-toggleable__label fitted sk-toggleable__label-arrow\"><div><div>SimpleImputer</div></div><div><a class=\"sk-estimator-doc-link fitted\" rel=\"noreferrer\" target=\"_blank\" href=\"https://scikit-learn.org/1.6/modules/generated/sklearn.impute.SimpleImputer.html\">?<span>Documentation for SimpleImputer</span></a></div></label><div class=\"sk-toggleable__content fitted\"><pre>SimpleImputer(strategy=&#x27;median&#x27;)</pre></div> </div></div><div class=\"sk-item\"><div class=\"sk-estimator fitted sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-6\" type=\"checkbox\" ><label for=\"sk-estimator-id-6\" class=\"sk-toggleable__label fitted sk-toggleable__label-arrow\"><div><div>StandardScaler</div></div><div><a class=\"sk-estimator-doc-link fitted\" rel=\"noreferrer\" target=\"_blank\" href=\"https://scikit-learn.org/1.6/modules/generated/sklearn.preprocessing.StandardScaler.html\">?<span>Documentation for StandardScaler</span></a></div></label><div class=\"sk-toggleable__content fitted\"><pre>StandardScaler()</pre></div> </div></div></div></div></div></div></div><div class=\"sk-parallel-item\"><div class=\"sk-item\"><div class=\"sk-label-container\"><div class=\"sk-label fitted sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-7\" type=\"checkbox\" ><label for=\"sk-estimator-id-7\" class=\"sk-toggleable__label fitted sk-toggleable__label-arrow\"><div><div>non_num</div></div></label><div class=\"sk-toggleable__content fitted\"><pre>[&#x27;Gender&#x27;, &#x27;CCU&#x27;, &#x27;CSRU&#x27;, &#x27;SICU&#x27;]</pre></div> </div></div><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-estimator fitted sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-8\" type=\"checkbox\" ><label for=\"sk-estimator-id-8\" class=\"sk-toggleable__label fitted sk-toggleable__label-arrow\"><div><div>passthrough</div></div></label><div class=\"sk-toggleable__content fitted\"><pre>passthrough</pre></div> </div></div></div></div></div></div></div><div class=\"sk-item\"><div class=\"sk-estimator fitted sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-9\" type=\"checkbox\" ><label for=\"sk-estimator-id-9\" class=\"sk-toggleable__label fitted sk-toggleable__label-arrow\"><div><div>LogisticRegression</div></div><div><a class=\"sk-estimator-doc-link fitted\" rel=\"noreferrer\" target=\"_blank\" href=\"https://scikit-learn.org/1.6/modules/generated/sklearn.linear_model.LogisticRegression.html\">?<span>Documentation for LogisticRegression</span></a></div></label><div class=\"sk-toggleable__content fitted\"><pre>LogisticRegression(C=1, class_weight=&#x27;balanced&#x27;, penalty=&#x27;l1&#x27;,\n",
       "                   random_state=12345, solver=&#x27;liblinear&#x27;, tol=0.001)</pre></div> </div></div></div></div></div></div></div></div></div></div></div>"
      ],
      "text/plain": [
       "GridSearchCV(cv=5,\n",
       "             estimator=Pipeline(steps=[('preprocessor',\n",
       "                                        ColumnTransformer(transformers=[('num',\n",
       "                                                                         Pipeline(steps=[('imputer',\n",
       "                                                                                          SimpleImputer(strategy='median')),\n",
       "                                                                                         ('scaler',\n",
       "                                                                                          StandardScaler())]),\n",
       "                                                                         ['Age',\n",
       "                                                                          'Weight',\n",
       "                                                                          'GCS_first',\n",
       "                                                                          'Glucose_first',\n",
       "                                                                          'HR_first',\n",
       "                                                                          'NIDiasABP_first',\n",
       "                                                                          'NIMAP_first',\n",
       "                                                                          'NISysABP_first',\n",
       "                                                                          'Temp_first',\n",
       "                                                                          'BUN_first',\n",
       "                                                                          'Creatinine_first',\n",
       "                                                                          'HCO3_first'...\n",
       "                                                                          'Platelets_first',\n",
       "                                                                          'WBC_first',\n",
       "                                                                          'pH_first']),\n",
       "                                                                        ('non_num',\n",
       "                                                                         'passthrough',\n",
       "                                                                         ['Gender',\n",
       "                                                                          'CCU',\n",
       "                                                                          'CSRU',\n",
       "                                                                          'SICU'])])),\n",
       "                                       ('classifier',\n",
       "                                        LogisticRegression(class_weight='balanced',\n",
       "                                                           penalty='l1',\n",
       "                                                           random_state=12345,\n",
       "                                                           solver='liblinear'))]),\n",
       "             param_grid={'classifier__C': [0.001, 0.01, 0.1, 1, 10, 100],\n",
       "                         'classifier__tol': [1e-05, 0.0001, 0.001]},\n",
       "             return_train_score=True, scoring='roc_auc')"
      ]
     },
     "execution_count": 85,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Fit the random search object to the data\n",
    "fit_lr_grid_search = lr_grid_search.fit(X_train, y_train)\n",
    "fit_lr_grid_search"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c644a28a-d0c1-4e24-b964-13659b239c17",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "#### So what is going on when we fit this model?\n",
    "\n",
    "Our pipeline ensures the following:\n",
    "\n",
    "1. Our data is divided into 5-folds\n",
    "2. We train the model on 4-folds\n",
    "3. These 4-folds are pre-processed (numeric data standardized and imputed as needed)\n",
    "4. GridSearch tests each combination of hyperparamters (e.g., `tol=1e-4` and `C=0.001`; `tol=1e-4` and `C=0.01` *[....] and so forth across the specified grid*)\n",
    "5. For each set of training folds and combination of hyperparameters, we evaluate performance (AUC) on the held-out fold\n",
    "6. This process repeats until all 5-folds and hyperparameter combos have been evaluted\n",
    "7. The set of hyperparameters with the highest average *AUC will be chosen as our best model \n",
    "\n",
    "*We'll discuss more, but the AUC (Area Under the Curve) refers to the area under the Receiver Operating Characteristic (ROC) curve. This metric quantifies the average classification performance of a model (ranging from 0 to 1) by evaluating its ability to distinguish between true positives and false positives across various thresholds."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e37420e2-b616-40f8-ab38-318729b835bb",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "#### **Concept Check:** \n",
    "\n",
    "Why do we perform the pre-processing step on 4-folds at a time? Can we standardize and impute all of the data prior to starting the training/tuning steps? Why or why not?"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "02f9295b-e90c-4f91-a128-d55f5d64119d",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "### Model Evaluation on Training set\n",
    "\n",
    "We can determine what the best hyperparameters and the average AUC is from the fitted object `fit_lr_grid_search`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 86,
   "id": "1be341d0-42d2-457b-88ba-7628e10bf11f",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Best performing hyperparameters: {'classifier__C': 1, 'classifier__tol': 0.001}\n",
      "Average AUC: 0.7388\n"
     ]
    }
   ],
   "source": [
    "# Best parameters and score\n",
    "print(f\"Best performing hyperparameters: {fit_lr_grid_search.best_params_}\")\n",
    "print(f\"Average AUC: {fit_lr_grid_search.best_score_.round(4)}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c20dab9f-11e4-4448-a0c5-cfae6a2d8c73",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "source": [
    "We can also see the results from all of the models evaluated during the tuning process"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 87,
   "id": "0f483431-60d2-4405-ab4d-2ae88b899419",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>mean_fit_time</th>\n",
       "      <th>std_fit_time</th>\n",
       "      <th>mean_score_time</th>\n",
       "      <th>std_score_time</th>\n",
       "      <th>param_classifier__C</th>\n",
       "      <th>param_classifier__tol</th>\n",
       "      <th>params</th>\n",
       "      <th>split0_test_score</th>\n",
       "      <th>split1_test_score</th>\n",
       "      <th>split2_test_score</th>\n",
       "      <th>split3_test_score</th>\n",
       "      <th>split4_test_score</th>\n",
       "      <th>mean_test_score</th>\n",
       "      <th>std_test_score</th>\n",
       "      <th>rank_test_score</th>\n",
       "      <th>split0_train_score</th>\n",
       "      <th>split1_train_score</th>\n",
       "      <th>split2_train_score</th>\n",
       "      <th>split3_train_score</th>\n",
       "      <th>split4_train_score</th>\n",
       "      <th>mean_train_score</th>\n",
       "      <th>std_train_score</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.015744</td>\n",
       "      <td>0.002281</td>\n",
       "      <td>0.004711</td>\n",
       "      <td>0.000561</td>\n",
       "      <td>0.001</td>\n",
       "      <td>0.00001</td>\n",
       "      <td>{'classifier__C': 0.001, 'classifier__tol': 1e...</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>16</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.014128</td>\n",
       "      <td>0.000950</td>\n",
       "      <td>0.004585</td>\n",
       "      <td>0.000646</td>\n",
       "      <td>0.001</td>\n",
       "      <td>0.00010</td>\n",
       "      <td>{'classifier__C': 0.001, 'classifier__tol': 0....</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>16</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.018422</td>\n",
       "      <td>0.004155</td>\n",
       "      <td>0.006555</td>\n",
       "      <td>0.001995</td>\n",
       "      <td>0.001</td>\n",
       "      <td>0.00100</td>\n",
       "      <td>{'classifier__C': 0.001, 'classifier__tol': 0....</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>16</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0.014009</td>\n",
       "      <td>0.000462</td>\n",
       "      <td>0.003830</td>\n",
       "      <td>0.000119</td>\n",
       "      <td>0.010</td>\n",
       "      <td>0.00001</td>\n",
       "      <td>{'classifier__C': 0.01, 'classifier__tol': 1e-05}</td>\n",
       "      <td>0.724160</td>\n",
       "      <td>0.770957</td>\n",
       "      <td>0.707653</td>\n",
       "      <td>0.668083</td>\n",
       "      <td>0.674667</td>\n",
       "      <td>0.709104</td>\n",
       "      <td>0.037211</td>\n",
       "      <td>13</td>\n",
       "      <td>0.723917</td>\n",
       "      <td>0.712969</td>\n",
       "      <td>0.725333</td>\n",
       "      <td>0.719736</td>\n",
       "      <td>0.734565</td>\n",
       "      <td>0.723304</td>\n",
       "      <td>0.007083</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0.015100</td>\n",
       "      <td>0.001532</td>\n",
       "      <td>0.004223</td>\n",
       "      <td>0.000592</td>\n",
       "      <td>0.010</td>\n",
       "      <td>0.00010</td>\n",
       "      <td>{'classifier__C': 0.01, 'classifier__tol': 0.0...</td>\n",
       "      <td>0.724160</td>\n",
       "      <td>0.770957</td>\n",
       "      <td>0.707640</td>\n",
       "      <td>0.668083</td>\n",
       "      <td>0.674667</td>\n",
       "      <td>0.709101</td>\n",
       "      <td>0.037211</td>\n",
       "      <td>14</td>\n",
       "      <td>0.723913</td>\n",
       "      <td>0.712969</td>\n",
       "      <td>0.725332</td>\n",
       "      <td>0.719735</td>\n",
       "      <td>0.734565</td>\n",
       "      <td>0.723303</td>\n",
       "      <td>0.007083</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>0.013665</td>\n",
       "      <td>0.000553</td>\n",
       "      <td>0.004021</td>\n",
       "      <td>0.000463</td>\n",
       "      <td>0.010</td>\n",
       "      <td>0.00100</td>\n",
       "      <td>{'classifier__C': 0.01, 'classifier__tol': 0.001}</td>\n",
       "      <td>0.724160</td>\n",
       "      <td>0.770944</td>\n",
       "      <td>0.707600</td>\n",
       "      <td>0.668083</td>\n",
       "      <td>0.674628</td>\n",
       "      <td>0.709083</td>\n",
       "      <td>0.037214</td>\n",
       "      <td>15</td>\n",
       "      <td>0.723908</td>\n",
       "      <td>0.712976</td>\n",
       "      <td>0.725331</td>\n",
       "      <td>0.719734</td>\n",
       "      <td>0.734574</td>\n",
       "      <td>0.723305</td>\n",
       "      <td>0.007084</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>0.021751</td>\n",
       "      <td>0.003328</td>\n",
       "      <td>0.004551</td>\n",
       "      <td>0.001044</td>\n",
       "      <td>0.100</td>\n",
       "      <td>0.00001</td>\n",
       "      <td>{'classifier__C': 0.1, 'classifier__tol': 1e-05}</td>\n",
       "      <td>0.739641</td>\n",
       "      <td>0.777521</td>\n",
       "      <td>0.730175</td>\n",
       "      <td>0.743660</td>\n",
       "      <td>0.701406</td>\n",
       "      <td>0.738481</td>\n",
       "      <td>0.024478</td>\n",
       "      <td>11</td>\n",
       "      <td>0.754453</td>\n",
       "      <td>0.745228</td>\n",
       "      <td>0.756436</td>\n",
       "      <td>0.750137</td>\n",
       "      <td>0.764688</td>\n",
       "      <td>0.754189</td>\n",
       "      <td>0.006512</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>0.020036</td>\n",
       "      <td>0.001942</td>\n",
       "      <td>0.005058</td>\n",
       "      <td>0.000808</td>\n",
       "      <td>0.100</td>\n",
       "      <td>0.00010</td>\n",
       "      <td>{'classifier__C': 0.1, 'classifier__tol': 0.0001}</td>\n",
       "      <td>0.739641</td>\n",
       "      <td>0.777521</td>\n",
       "      <td>0.730189</td>\n",
       "      <td>0.743660</td>\n",
       "      <td>0.701367</td>\n",
       "      <td>0.738475</td>\n",
       "      <td>0.024489</td>\n",
       "      <td>12</td>\n",
       "      <td>0.754454</td>\n",
       "      <td>0.745226</td>\n",
       "      <td>0.756437</td>\n",
       "      <td>0.750136</td>\n",
       "      <td>0.764687</td>\n",
       "      <td>0.754188</td>\n",
       "      <td>0.006513</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>0.017170</td>\n",
       "      <td>0.001602</td>\n",
       "      <td>0.004493</td>\n",
       "      <td>0.000576</td>\n",
       "      <td>0.100</td>\n",
       "      <td>0.00100</td>\n",
       "      <td>{'classifier__C': 0.1, 'classifier__tol': 0.001}</td>\n",
       "      <td>0.739667</td>\n",
       "      <td>0.777534</td>\n",
       "      <td>0.730228</td>\n",
       "      <td>0.743660</td>\n",
       "      <td>0.701341</td>\n",
       "      <td>0.738486</td>\n",
       "      <td>0.024498</td>\n",
       "      <td>10</td>\n",
       "      <td>0.754462</td>\n",
       "      <td>0.745207</td>\n",
       "      <td>0.756420</td>\n",
       "      <td>0.750135</td>\n",
       "      <td>0.764676</td>\n",
       "      <td>0.754180</td>\n",
       "      <td>0.006513</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>0.050443</td>\n",
       "      <td>0.001875</td>\n",
       "      <td>0.005762</td>\n",
       "      <td>0.001275</td>\n",
       "      <td>1.000</td>\n",
       "      <td>0.00001</td>\n",
       "      <td>{'classifier__C': 1, 'classifier__tol': 1e-05}</td>\n",
       "      <td>0.735875</td>\n",
       "      <td>0.774932</td>\n",
       "      <td>0.725294</td>\n",
       "      <td>0.754440</td>\n",
       "      <td>0.703515</td>\n",
       "      <td>0.738811</td>\n",
       "      <td>0.024438</td>\n",
       "      <td>2</td>\n",
       "      <td>0.757279</td>\n",
       "      <td>0.748484</td>\n",
       "      <td>0.759709</td>\n",
       "      <td>0.751647</td>\n",
       "      <td>0.766390</td>\n",
       "      <td>0.756702</td>\n",
       "      <td>0.006266</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>0.041580</td>\n",
       "      <td>0.003886</td>\n",
       "      <td>0.004438</td>\n",
       "      <td>0.000360</td>\n",
       "      <td>1.000</td>\n",
       "      <td>0.00010</td>\n",
       "      <td>{'classifier__C': 1, 'classifier__tol': 0.0001}</td>\n",
       "      <td>0.735888</td>\n",
       "      <td>0.774906</td>\n",
       "      <td>0.725294</td>\n",
       "      <td>0.754427</td>\n",
       "      <td>0.703488</td>\n",
       "      <td>0.738800</td>\n",
       "      <td>0.024436</td>\n",
       "      <td>3</td>\n",
       "      <td>0.757282</td>\n",
       "      <td>0.748484</td>\n",
       "      <td>0.759710</td>\n",
       "      <td>0.751648</td>\n",
       "      <td>0.766391</td>\n",
       "      <td>0.756703</td>\n",
       "      <td>0.006266</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>0.037155</td>\n",
       "      <td>0.008572</td>\n",
       "      <td>0.005531</td>\n",
       "      <td>0.000977</td>\n",
       "      <td>1.000</td>\n",
       "      <td>0.00100</td>\n",
       "      <td>{'classifier__C': 1, 'classifier__tol': 0.001}</td>\n",
       "      <td>0.735888</td>\n",
       "      <td>0.774932</td>\n",
       "      <td>0.725518</td>\n",
       "      <td>0.754374</td>\n",
       "      <td>0.703462</td>\n",
       "      <td>0.738835</td>\n",
       "      <td>0.024420</td>\n",
       "      <td>1</td>\n",
       "      <td>0.757285</td>\n",
       "      <td>0.748491</td>\n",
       "      <td>0.759731</td>\n",
       "      <td>0.751645</td>\n",
       "      <td>0.766384</td>\n",
       "      <td>0.756707</td>\n",
       "      <td>0.006265</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>0.051984</td>\n",
       "      <td>0.004365</td>\n",
       "      <td>0.004999</td>\n",
       "      <td>0.000500</td>\n",
       "      <td>10.000</td>\n",
       "      <td>0.00001</td>\n",
       "      <td>{'classifier__C': 10, 'classifier__tol': 1e-05}</td>\n",
       "      <td>0.735195</td>\n",
       "      <td>0.774657</td>\n",
       "      <td>0.724396</td>\n",
       "      <td>0.756313</td>\n",
       "      <td>0.703200</td>\n",
       "      <td>0.738753</td>\n",
       "      <td>0.024820</td>\n",
       "      <td>8</td>\n",
       "      <td>0.757276</td>\n",
       "      <td>0.748470</td>\n",
       "      <td>0.759731</td>\n",
       "      <td>0.751725</td>\n",
       "      <td>0.766333</td>\n",
       "      <td>0.756707</td>\n",
       "      <td>0.006242</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>0.042413</td>\n",
       "      <td>0.002736</td>\n",
       "      <td>0.004668</td>\n",
       "      <td>0.000728</td>\n",
       "      <td>10.000</td>\n",
       "      <td>0.00010</td>\n",
       "      <td>{'classifier__C': 10, 'classifier__tol': 0.0001}</td>\n",
       "      <td>0.735169</td>\n",
       "      <td>0.774657</td>\n",
       "      <td>0.724396</td>\n",
       "      <td>0.756313</td>\n",
       "      <td>0.703213</td>\n",
       "      <td>0.738750</td>\n",
       "      <td>0.024817</td>\n",
       "      <td>9</td>\n",
       "      <td>0.757277</td>\n",
       "      <td>0.748477</td>\n",
       "      <td>0.759731</td>\n",
       "      <td>0.751725</td>\n",
       "      <td>0.766335</td>\n",
       "      <td>0.756709</td>\n",
       "      <td>0.006240</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>0.031830</td>\n",
       "      <td>0.002272</td>\n",
       "      <td>0.004320</td>\n",
       "      <td>0.000432</td>\n",
       "      <td>10.000</td>\n",
       "      <td>0.00100</td>\n",
       "      <td>{'classifier__C': 10, 'classifier__tol': 0.001}</td>\n",
       "      <td>0.735326</td>\n",
       "      <td>0.774657</td>\n",
       "      <td>0.724383</td>\n",
       "      <td>0.756340</td>\n",
       "      <td>0.703213</td>\n",
       "      <td>0.738784</td>\n",
       "      <td>0.024818</td>\n",
       "      <td>5</td>\n",
       "      <td>0.757280</td>\n",
       "      <td>0.748493</td>\n",
       "      <td>0.759733</td>\n",
       "      <td>0.751725</td>\n",
       "      <td>0.766349</td>\n",
       "      <td>0.756716</td>\n",
       "      <td>0.006241</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>0.051903</td>\n",
       "      <td>0.002438</td>\n",
       "      <td>0.005079</td>\n",
       "      <td>0.000620</td>\n",
       "      <td>100.000</td>\n",
       "      <td>0.00001</td>\n",
       "      <td>{'classifier__C': 100, 'classifier__tol': 1e-05}</td>\n",
       "      <td>0.735143</td>\n",
       "      <td>0.774696</td>\n",
       "      <td>0.724304</td>\n",
       "      <td>0.756366</td>\n",
       "      <td>0.703344</td>\n",
       "      <td>0.738771</td>\n",
       "      <td>0.024810</td>\n",
       "      <td>6</td>\n",
       "      <td>0.757270</td>\n",
       "      <td>0.748495</td>\n",
       "      <td>0.759715</td>\n",
       "      <td>0.751758</td>\n",
       "      <td>0.766342</td>\n",
       "      <td>0.756716</td>\n",
       "      <td>0.006231</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>0.040404</td>\n",
       "      <td>0.001907</td>\n",
       "      <td>0.004438</td>\n",
       "      <td>0.000293</td>\n",
       "      <td>100.000</td>\n",
       "      <td>0.00010</td>\n",
       "      <td>{'classifier__C': 100, 'classifier__tol': 0.0001}</td>\n",
       "      <td>0.735156</td>\n",
       "      <td>0.774709</td>\n",
       "      <td>0.724291</td>\n",
       "      <td>0.756340</td>\n",
       "      <td>0.703344</td>\n",
       "      <td>0.738768</td>\n",
       "      <td>0.024811</td>\n",
       "      <td>7</td>\n",
       "      <td>0.757270</td>\n",
       "      <td>0.748495</td>\n",
       "      <td>0.759715</td>\n",
       "      <td>0.751756</td>\n",
       "      <td>0.766339</td>\n",
       "      <td>0.756715</td>\n",
       "      <td>0.006230</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>0.035361</td>\n",
       "      <td>0.005802</td>\n",
       "      <td>0.005307</td>\n",
       "      <td>0.001120</td>\n",
       "      <td>100.000</td>\n",
       "      <td>0.00100</td>\n",
       "      <td>{'classifier__C': 100, 'classifier__tol': 0.001}</td>\n",
       "      <td>0.735195</td>\n",
       "      <td>0.774749</td>\n",
       "      <td>0.724357</td>\n",
       "      <td>0.756353</td>\n",
       "      <td>0.703305</td>\n",
       "      <td>0.738792</td>\n",
       "      <td>0.024827</td>\n",
       "      <td>4</td>\n",
       "      <td>0.757281</td>\n",
       "      <td>0.748504</td>\n",
       "      <td>0.759720</td>\n",
       "      <td>0.751742</td>\n",
       "      <td>0.766334</td>\n",
       "      <td>0.756716</td>\n",
       "      <td>0.006229</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    mean_fit_time  std_fit_time  mean_score_time  std_score_time  \\\n",
       "0        0.015744      0.002281         0.004711        0.000561   \n",
       "1        0.014128      0.000950         0.004585        0.000646   \n",
       "2        0.018422      0.004155         0.006555        0.001995   \n",
       "3        0.014009      0.000462         0.003830        0.000119   \n",
       "4        0.015100      0.001532         0.004223        0.000592   \n",
       "5        0.013665      0.000553         0.004021        0.000463   \n",
       "6        0.021751      0.003328         0.004551        0.001044   \n",
       "7        0.020036      0.001942         0.005058        0.000808   \n",
       "8        0.017170      0.001602         0.004493        0.000576   \n",
       "9        0.050443      0.001875         0.005762        0.001275   \n",
       "10       0.041580      0.003886         0.004438        0.000360   \n",
       "11       0.037155      0.008572         0.005531        0.000977   \n",
       "12       0.051984      0.004365         0.004999        0.000500   \n",
       "13       0.042413      0.002736         0.004668        0.000728   \n",
       "14       0.031830      0.002272         0.004320        0.000432   \n",
       "15       0.051903      0.002438         0.005079        0.000620   \n",
       "16       0.040404      0.001907         0.004438        0.000293   \n",
       "17       0.035361      0.005802         0.005307        0.001120   \n",
       "\n",
       "    param_classifier__C  param_classifier__tol  \\\n",
       "0                 0.001                0.00001   \n",
       "1                 0.001                0.00010   \n",
       "2                 0.001                0.00100   \n",
       "3                 0.010                0.00001   \n",
       "4                 0.010                0.00010   \n",
       "5                 0.010                0.00100   \n",
       "6                 0.100                0.00001   \n",
       "7                 0.100                0.00010   \n",
       "8                 0.100                0.00100   \n",
       "9                 1.000                0.00001   \n",
       "10                1.000                0.00010   \n",
       "11                1.000                0.00100   \n",
       "12               10.000                0.00001   \n",
       "13               10.000                0.00010   \n",
       "14               10.000                0.00100   \n",
       "15              100.000                0.00001   \n",
       "16              100.000                0.00010   \n",
       "17              100.000                0.00100   \n",
       "\n",
       "                                               params  split0_test_score  \\\n",
       "0   {'classifier__C': 0.001, 'classifier__tol': 1e...           0.500000   \n",
       "1   {'classifier__C': 0.001, 'classifier__tol': 0....           0.500000   \n",
       "2   {'classifier__C': 0.001, 'classifier__tol': 0....           0.500000   \n",
       "3   {'classifier__C': 0.01, 'classifier__tol': 1e-05}           0.724160   \n",
       "4   {'classifier__C': 0.01, 'classifier__tol': 0.0...           0.724160   \n",
       "5   {'classifier__C': 0.01, 'classifier__tol': 0.001}           0.724160   \n",
       "6    {'classifier__C': 0.1, 'classifier__tol': 1e-05}           0.739641   \n",
       "7   {'classifier__C': 0.1, 'classifier__tol': 0.0001}           0.739641   \n",
       "8    {'classifier__C': 0.1, 'classifier__tol': 0.001}           0.739667   \n",
       "9      {'classifier__C': 1, 'classifier__tol': 1e-05}           0.735875   \n",
       "10    {'classifier__C': 1, 'classifier__tol': 0.0001}           0.735888   \n",
       "11     {'classifier__C': 1, 'classifier__tol': 0.001}           0.735888   \n",
       "12    {'classifier__C': 10, 'classifier__tol': 1e-05}           0.735195   \n",
       "13   {'classifier__C': 10, 'classifier__tol': 0.0001}           0.735169   \n",
       "14    {'classifier__C': 10, 'classifier__tol': 0.001}           0.735326   \n",
       "15   {'classifier__C': 100, 'classifier__tol': 1e-05}           0.735143   \n",
       "16  {'classifier__C': 100, 'classifier__tol': 0.0001}           0.735156   \n",
       "17   {'classifier__C': 100, 'classifier__tol': 0.001}           0.735195   \n",
       "\n",
       "    split1_test_score  split2_test_score  split3_test_score  \\\n",
       "0            0.500000           0.500000           0.500000   \n",
       "1            0.500000           0.500000           0.500000   \n",
       "2            0.500000           0.500000           0.500000   \n",
       "3            0.770957           0.707653           0.668083   \n",
       "4            0.770957           0.707640           0.668083   \n",
       "5            0.770944           0.707600           0.668083   \n",
       "6            0.777521           0.730175           0.743660   \n",
       "7            0.777521           0.730189           0.743660   \n",
       "8            0.777534           0.730228           0.743660   \n",
       "9            0.774932           0.725294           0.754440   \n",
       "10           0.774906           0.725294           0.754427   \n",
       "11           0.774932           0.725518           0.754374   \n",
       "12           0.774657           0.724396           0.756313   \n",
       "13           0.774657           0.724396           0.756313   \n",
       "14           0.774657           0.724383           0.756340   \n",
       "15           0.774696           0.724304           0.756366   \n",
       "16           0.774709           0.724291           0.756340   \n",
       "17           0.774749           0.724357           0.756353   \n",
       "\n",
       "    split4_test_score  mean_test_score  std_test_score  rank_test_score  \\\n",
       "0            0.500000         0.500000        0.000000               16   \n",
       "1            0.500000         0.500000        0.000000               16   \n",
       "2            0.500000         0.500000        0.000000               16   \n",
       "3            0.674667         0.709104        0.037211               13   \n",
       "4            0.674667         0.709101        0.037211               14   \n",
       "5            0.674628         0.709083        0.037214               15   \n",
       "6            0.701406         0.738481        0.024478               11   \n",
       "7            0.701367         0.738475        0.024489               12   \n",
       "8            0.701341         0.738486        0.024498               10   \n",
       "9            0.703515         0.738811        0.024438                2   \n",
       "10           0.703488         0.738800        0.024436                3   \n",
       "11           0.703462         0.738835        0.024420                1   \n",
       "12           0.703200         0.738753        0.024820                8   \n",
       "13           0.703213         0.738750        0.024817                9   \n",
       "14           0.703213         0.738784        0.024818                5   \n",
       "15           0.703344         0.738771        0.024810                6   \n",
       "16           0.703344         0.738768        0.024811                7   \n",
       "17           0.703305         0.738792        0.024827                4   \n",
       "\n",
       "    split0_train_score  split1_train_score  split2_train_score  \\\n",
       "0             0.500000            0.500000            0.500000   \n",
       "1             0.500000            0.500000            0.500000   \n",
       "2             0.500000            0.500000            0.500000   \n",
       "3             0.723917            0.712969            0.725333   \n",
       "4             0.723913            0.712969            0.725332   \n",
       "5             0.723908            0.712976            0.725331   \n",
       "6             0.754453            0.745228            0.756436   \n",
       "7             0.754454            0.745226            0.756437   \n",
       "8             0.754462            0.745207            0.756420   \n",
       "9             0.757279            0.748484            0.759709   \n",
       "10            0.757282            0.748484            0.759710   \n",
       "11            0.757285            0.748491            0.759731   \n",
       "12            0.757276            0.748470            0.759731   \n",
       "13            0.757277            0.748477            0.759731   \n",
       "14            0.757280            0.748493            0.759733   \n",
       "15            0.757270            0.748495            0.759715   \n",
       "16            0.757270            0.748495            0.759715   \n",
       "17            0.757281            0.748504            0.759720   \n",
       "\n",
       "    split3_train_score  split4_train_score  mean_train_score  std_train_score  \n",
       "0             0.500000            0.500000          0.500000         0.000000  \n",
       "1             0.500000            0.500000          0.500000         0.000000  \n",
       "2             0.500000            0.500000          0.500000         0.000000  \n",
       "3             0.719736            0.734565          0.723304         0.007083  \n",
       "4             0.719735            0.734565          0.723303         0.007083  \n",
       "5             0.719734            0.734574          0.723305         0.007084  \n",
       "6             0.750137            0.764688          0.754189         0.006512  \n",
       "7             0.750136            0.764687          0.754188         0.006513  \n",
       "8             0.750135            0.764676          0.754180         0.006513  \n",
       "9             0.751647            0.766390          0.756702         0.006266  \n",
       "10            0.751648            0.766391          0.756703         0.006266  \n",
       "11            0.751645            0.766384          0.756707         0.006265  \n",
       "12            0.751725            0.766333          0.756707         0.006242  \n",
       "13            0.751725            0.766335          0.756709         0.006240  \n",
       "14            0.751725            0.766349          0.756716         0.006241  \n",
       "15            0.751758            0.766342          0.756716         0.006231  \n",
       "16            0.751756            0.766339          0.756715         0.006230  \n",
       "17            0.751742            0.766334          0.756716         0.006229  "
      ]
     },
     "execution_count": 87,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Access the cv_results_ dictionary\n",
    "cv_results = fit_lr_grid_search.cv_results_\n",
    "\n",
    "# Create a DataFrame from cv_results_\n",
    "results_df = pd.DataFrame(cv_results)\n",
    "\n",
    "results_df"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4b5a7948-eafe-4221-8109-dc3d78c9a767",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "source": [
    "Let's just look at the best performing model (based on AUC)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 88,
   "id": "a3bbbc8f-41e1-4c4c-bc65-f9e16758c520",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>mean_fit_time</th>\n",
       "      <th>std_fit_time</th>\n",
       "      <th>mean_score_time</th>\n",
       "      <th>std_score_time</th>\n",
       "      <th>param_classifier__C</th>\n",
       "      <th>param_classifier__tol</th>\n",
       "      <th>params</th>\n",
       "      <th>split0_test_score</th>\n",
       "      <th>split1_test_score</th>\n",
       "      <th>split2_test_score</th>\n",
       "      <th>split3_test_score</th>\n",
       "      <th>split4_test_score</th>\n",
       "      <th>mean_test_score</th>\n",
       "      <th>std_test_score</th>\n",
       "      <th>rank_test_score</th>\n",
       "      <th>split0_train_score</th>\n",
       "      <th>split1_train_score</th>\n",
       "      <th>split2_train_score</th>\n",
       "      <th>split3_train_score</th>\n",
       "      <th>split4_train_score</th>\n",
       "      <th>mean_train_score</th>\n",
       "      <th>std_train_score</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>0.037155</td>\n",
       "      <td>0.008572</td>\n",
       "      <td>0.005531</td>\n",
       "      <td>0.000977</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.001</td>\n",
       "      <td>{'classifier__C': 1, 'classifier__tol': 0.001}</td>\n",
       "      <td>0.735888</td>\n",
       "      <td>0.774932</td>\n",
       "      <td>0.725518</td>\n",
       "      <td>0.754374</td>\n",
       "      <td>0.703462</td>\n",
       "      <td>0.738835</td>\n",
       "      <td>0.02442</td>\n",
       "      <td>1</td>\n",
       "      <td>0.757285</td>\n",
       "      <td>0.748491</td>\n",
       "      <td>0.759731</td>\n",
       "      <td>0.751645</td>\n",
       "      <td>0.766384</td>\n",
       "      <td>0.756707</td>\n",
       "      <td>0.006265</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    mean_fit_time  std_fit_time  mean_score_time  std_score_time  \\\n",
       "11       0.037155      0.008572         0.005531        0.000977   \n",
       "\n",
       "    param_classifier__C  param_classifier__tol  \\\n",
       "11                  1.0                  0.001   \n",
       "\n",
       "                                            params  split0_test_score  \\\n",
       "11  {'classifier__C': 1, 'classifier__tol': 0.001}           0.735888   \n",
       "\n",
       "    split1_test_score  split2_test_score  split3_test_score  \\\n",
       "11           0.774932           0.725518           0.754374   \n",
       "\n",
       "    split4_test_score  mean_test_score  std_test_score  rank_test_score  \\\n",
       "11           0.703462         0.738835         0.02442                1   \n",
       "\n",
       "    split0_train_score  split1_train_score  split2_train_score  \\\n",
       "11            0.757285            0.748491            0.759731   \n",
       "\n",
       "    split3_train_score  split4_train_score  mean_train_score  std_train_score  \n",
       "11            0.751645            0.766384          0.756707         0.006265  "
      ]
     },
     "execution_count": 88,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Extract the row corresponding to the best index\n",
    "best_row_df = results_df.loc[[fit_lr_grid_search.best_index_]]  # Use double brackets to keep it as a DataFrame\n",
    "best_row_df"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a18789f9-28f6-44cc-87c2-9dbd98bfcb91",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "## Training Evaluation\n",
    "\n",
    "There are many ways to evaluate the performance of our training. The below modules will review a few!"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8160fad7-1d60-4204-9fef-b61e6c5989d4",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "### Model Coefficients (important predictors)\n",
    "\n",
    "Default coefficients are represented as log-odds\n",
    "\n",
    "We can exponentiate the log-odds to get the odds ratio:\n",
    "\n",
    "$\\text{Odds Ratio} = e^{\\text{coef}}$\n",
    "\n",
    "**To intepret:**\n",
    "* A feature with an odds ratio > 1 - are features in which the odds of death increase when that feature increases\n",
    "* A feature with an odds ratio < 1 - are features in which the odds of death decrease when that feature increases\n",
    "* A feature with an odds ratio = 1 - are features which have no effect on the outcome (death)\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 89,
   "id": "5fbf8fd2-1d58-439f-aaeb-0b2885a72975",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Feature</th>\n",
       "      <th>Coefficient</th>\n",
       "      <th>abs(coef)</th>\n",
       "      <th>Odds Ratio</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>WBC_first</td>\n",
       "      <td>-1.561197</td>\n",
       "      <td>1.561197</td>\n",
       "      <td>0.209885</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Platelets_first</td>\n",
       "      <td>-0.452355</td>\n",
       "      <td>0.452355</td>\n",
       "      <td>0.636128</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Weight</td>\n",
       "      <td>-0.370489</td>\n",
       "      <td>0.370489</td>\n",
       "      <td>0.690397</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>SICU</td>\n",
       "      <td>-0.369363</td>\n",
       "      <td>0.369363</td>\n",
       "      <td>0.691174</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Glucose_first</td>\n",
       "      <td>-0.258278</td>\n",
       "      <td>0.258278</td>\n",
       "      <td>0.772380</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>HCT_first</td>\n",
       "      <td>-0.208594</td>\n",
       "      <td>0.208594</td>\n",
       "      <td>0.811725</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>PaCO2_first</td>\n",
       "      <td>-0.204994</td>\n",
       "      <td>0.204994</td>\n",
       "      <td>0.814652</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>K_first</td>\n",
       "      <td>-0.172407</td>\n",
       "      <td>0.172407</td>\n",
       "      <td>0.841636</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>HCO3_first</td>\n",
       "      <td>-0.150314</td>\n",
       "      <td>0.150314</td>\n",
       "      <td>0.860438</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>Mg_first</td>\n",
       "      <td>-0.149042</td>\n",
       "      <td>0.149042</td>\n",
       "      <td>0.861533</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "           Feature  Coefficient  abs(coef)  Odds Ratio\n",
       "0        WBC_first    -1.561197   1.561197    0.209885\n",
       "1  Platelets_first    -0.452355   0.452355    0.636128\n",
       "2           Weight    -0.370489   0.370489    0.690397\n",
       "3             SICU    -0.369363   0.369363    0.691174\n",
       "4    Glucose_first    -0.258278   0.258278    0.772380\n",
       "5        HCT_first    -0.208594   0.208594    0.811725\n",
       "6      PaCO2_first    -0.204994   0.204994    0.814652\n",
       "7          K_first    -0.172407   0.172407    0.841636\n",
       "8       HCO3_first    -0.150314   0.150314    0.860438\n",
       "9         Mg_first    -0.149042   0.149042    0.861533"
      ]
     },
     "execution_count": 89,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Access the best estimator\n",
    "best_lasso = lr_grid_search.best_estimator_\n",
    "\n",
    "# Extract coefficients from the Lasso model\n",
    "lasso_coefficients = best_lasso.named_steps['classifier'].coef_.flatten().tolist()\n",
    "\n",
    "# If needed, map coefficients to feature names\n",
    "feature_names = X_train.columns.tolist()  # Replace with your list of feature names\n",
    "coefficients_df = pd.DataFrame({\n",
    "    'Feature': feature_names,\n",
    "    'Coefficient': lasso_coefficients\n",
    "})\n",
    "\n",
    "coefficients_df['abs(coef)'] = coefficients_df['Coefficient'].abs()\n",
    "coefficients_df['Odds Ratio'] = np.exp(coefficients_df['Coefficient'])\n",
    "\n",
    "# Display the coefficients\n",
    "coefficients_df.sort_values(by='Odds Ratio', ascending=True).reset_index(drop=True).head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2a9f7243-2709-4e83-a6c2-0e42cd317bc9",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "### Confusion Matrix\n",
    "\n",
    "A confusion matrix helps us calculate our true and false positive rates. This in turn, helps us calculate important metrics such as: \n",
    "\n",
    "* sensitivity - TP / (TP + FN)\n",
    "* specificity - TN / (TN + FP) \n",
    "* positive predictive value (precision) - TP / (TP + FP)\n",
    "* negative predictive value - TN / (TN + FN)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 90,
   "id": "fa8338b4-0cd0-4e80-a25e-da6159cf8166",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "classification report:\n",
      "                precision    recall  f1-score   support\n",
      "\n",
      "           0      0.935     0.681     0.788      3444\n",
      "           1      0.262     0.705     0.382       553\n",
      "\n",
      "    accuracy                          0.685      3997\n",
      "   macro avg      0.599     0.693     0.585      3997\n",
      "weighted avg      0.842     0.685     0.732      3997\n",
      "\n",
      "confusion matrix:\n",
      " \n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<sklearn.metrics._plot.confusion_matrix.ConfusionMatrixDisplay at 0x23b8b1d1940>"
      ]
     },
     "execution_count": 90,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAgQAAAGwCAYAAADWsX1oAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8hTgPZAAAACXBIWXMAAA9hAAAPYQGoP6dpAABHh0lEQVR4nO3de1xUZf4H8M9wGy4yowMOA4qI5q3AGxhgN0xDKbxkpS4uq4VY621dtVpzTdtWyXZTS1dzzRVTXO23m9aWS+E9U7yAWChLmqigjKDCIPdh5vz+IE+N4MAwMyCcz/v1el7rnPOcc74zy6v5zvd5nnNkgiAIICIiIklzaO0AiIiIqPUxISAiIiImBERERMSEgIiIiMCEgIiIiMCEgIiIiMCEgIiIiAA4tXYA1jAajbh27Ro8PT0hk8laOxwiIrKQIAi4ffs2/Pz84OBgv9+oVVVVqKmpsfo8Li4ucHV1tUFE9582nRBcu3YN/v7+rR0GERFZKS8vD127drXLuauqqhAY0AHaQoPV59JoNMjNzW2XSUGbTgg8PT0BAJczukPRgaMf1D4N2TSttUMgshtjdRUurv6T+N9ze6ipqYG20IDL6d2h8Gz+d0XpbSMCQi6hpqaGCcH95s4wgaKDg1X/JxPdzxzl7e8/PER3a4lh3w6eMnTwbP51jGjfQ9NtOiEgIiJqKoNghMGKp/cYBKPtgrkPMSEgIiJJMEKAEc3PCKw5ti1gnZ2IiIhYISAiImkwwghriv7WHX3/Y0JARESSYBAEGITml/2tObYt4JABERERsUJARETSwEmF5jEhICIiSTBCgIEJwT1xyICIiIhYISAiImngkIF5rBAQEZEk3FllYE2zRGJiIoYMGQJPT0+o1WqMGzcOOTk54n69Xo/XX38dwcHB8PDwgJ+fH37zm9/g2rVrJueJjIyETCYzaZMmTTLpU1xcjLi4OCiVSiiVSsTFxaGkpMSieJkQEBER2cGhQ4cwc+ZMpKWlITU1FbW1tYiKikJ5eTkAoKKiAhkZGVi8eDEyMjLw6aef4ocffsCYMWPqnSshIQEFBQVi27Bhg8n+2NhYZGZmIiUlBSkpKcjMzERcXJxF8XLIgIiIJMH4U7PmeEukpKSYvN68eTPUajXS09Px+OOPQ6lUIjU11aTPmjVr8PDDD+PKlSvo1q2buN3d3R0ajabB62RnZyMlJQVpaWkICwsDAGzcuBERERHIyclBnz59mhQvKwRERCQJhp9WGVjTAKC0tNSkVVdXN+n6Op0OAKBSqcz2kclk6Nixo8n25ORkeHt746GHHsKCBQtw+/Ztcd+xY8egVCrFZAAAwsPDoVQqcfTo0aZ+PKwQEBGRNBgEWPm0w7r/9ff3N9m+ZMkSLF261OyxgiBg3rx5ePTRRxEUFNRgn6qqKvzhD39AbGwsFAqFuH3y5MkIDAyERqNBVlYWFi5ciDNnzojVBa1WC7VaXe98arUaWq22ye+PCQEREZEF8vLyTL6w5XJ5o8fMmjUL3333HY4cOdLgfr1ej0mTJsFoNGLdunUm+xISEsR/BwUFoVevXggNDUVGRgYGDx4MAJDJZPXOKQhCg9vvhQkBERFJgq3mECgUCpOEoDGzZ8/G559/jsOHD6Nr16719uv1ekyYMAG5ubnYv39/o+cePHgwnJ2dcf78eQwePBgajQbXr1+v16+oqAg+Pj5NjpNzCIiISBKMkMFgRTOi6b+2gbpf6LNmzcKnn36K/fv3IzAwsF6fO8nA+fPnsXfvXnh5eTV63rNnz0Kv18PX1xcAEBERAZ1OhxMnToh9jh8/Dp1Oh6FDhzY5XlYIiIiI7GDmzJnYvn07PvvsM3h6eorj+UqlEm5ubqitrcXzzz+PjIwMfPHFFzAYDGIflUoFFxcX/Pjjj0hOTsbTTz8Nb29vnDt3DvPnz8egQYPwyCOPAAD69euHUaNGISEhQVyOOH36dMTExDR5hQHACgEREUmEUbC+WWL9+vXQ6XSIjIyEr6+v2Hbu3AkAyM/Px+eff478/HwMHDjQpM+d1QEuLi7Yt28fRo4ciT59+mDOnDmIiorC3r174ejoKF4rOTkZwcHBiIqKQlRUFPr374+tW7daFC8rBEREJAl3Sv/WHG8JoZE7G3bv3r3RPv7+/jh06FCj11KpVNi2bZtF8d2NFQIiIiJihYCIiKShpSsEbQ0TAiIikgSjIINRaP6XujXHtgUcMiAiIiJWCIiISBo4ZGAeEwIiIpIEAxxgsKIwbrBhLPcjJgRERCQJgpVzCATOISAiIqL2jhUCIiKSBM4hMI8JARERSYJBcIBBsGIOgYW3Lm5rOGRARERErBAQEZE0GCGD0YrfwUa07xIBEwIiIpIEziEwj0MGRERExAoBERFJg/WTCjlkQERE1ObVzSGw4uFGHDIgIiKi9o4VAiIikgSjlc8y4CoDIiKidoBzCMxjQkBERJJghAPvQ2AG5xAQERERKwRERCQNBkEGgxWPMLbm2LaACQEREUmCwcpJhQYOGRAREVF7xwoBERFJglFwgNGKVQZGrjIgIiJq+zhkYB6HDIiIiIgVAiIikgYjrFspYLRdKPclJgRERCQJ1t+YqH0X1dv3uyMiIqImYYWAiIgkwfpnGbTv39Dt+90RERH9xAiZ1c0SiYmJGDJkCDw9PaFWqzFu3Djk5OSY9BEEAUuXLoWfnx/c3NwQGRmJs2fPmvSprq7G7Nmz4e3tDQ8PD4wZMwb5+fkmfYqLixEXFwelUgmlUom4uDiUlJRYFC8TAiIikoQ7FQJrmiUOHTqEmTNnIi0tDampqaitrUVUVBTKy8vFPu+++y5WrlyJtWvX4uTJk9BoNHjqqadw+/Ztsc/cuXOxa9cu7NixA0eOHEFZWRliYmJgMBjEPrGxscjMzERKSgpSUlKQmZmJuLg4i+LlkAEREZEdpKSkmLzevHkz1Go10tPT8fjjj0MQBKxevRqLFi3C+PHjAQBbtmyBj48Ptm/fjpdffhk6nQ6bNm3C1q1bMWLECADAtm3b4O/vj71792LkyJHIzs5GSkoK0tLSEBYWBgDYuHEjIiIikJOTgz59+jQpXlYIiIhIEu7cmMiaBgClpaUmrbq6uknX1+l0AACVSgUAyM3NhVarRVRUlNhHLpfjiSeewNGjRwEA6enp0Ov1Jn38/PwQFBQk9jl27BiUSqWYDABAeHg4lEql2KcpmBAQEZEkGAWZ1Q0A/P39xbF6pVKJxMTERq8tCALmzZuHRx99FEFBQQAArVYLAPDx8THp6+PjI+7TarVwcXFBp06dzPZRq9X1rqlWq8U+TcEhAyIiIgvk5eVBoVCIr+VyeaPHzJo1C9999x2OHDlSb59MZjpZURCEetvudnefhvo35Ty/xAoBERFJgtHK4YI7NyZSKBQmrbGEYPbs2fj8889x4MABdO3aVdyu0WgAoN6v+MLCQrFqoNFoUFNTg+LiYrN9rl+/Xu+6RUVF9aoP5jAhICIiSbjztENrmiUEQcCsWbPw6aefYv/+/QgMDDTZHxgYCI1Gg9TUVHFbTU0NDh06hKFDhwIAQkJC4OzsbNKnoKAAWVlZYp+IiAjodDqcOHFC7HP8+HHodDqxT1NwyICIiMgOZs6cie3bt+Ozzz6Dp6enWAlQKpVwc3ODTCbD3LlzsXz5cvTq1Qu9evXC8uXL4e7ujtjYWLFvfHw85s+fDy8vL6hUKixYsADBwcHiqoN+/fph1KhRSEhIwIYNGwAA06dPR0xMTJNXGABMCIiISCIMkMFg4c2F7j7eEuvXrwcAREZGmmzfvHkzpk6dCgB47bXXUFlZiRkzZqC4uBhhYWH4+uuv4enpKfZftWoVnJycMGHCBFRWVmL48OFISkqCo6Oj2Cc5ORlz5swRVyOMGTMGa9eutShemSAIbfYBz6WlpVAqlSj+oQcUnhz9oPbpwfUzWjsEIrsxVFfhwoo3oNPpTCbq2dKd74q3jo+Aa4fm/w6uKqvFkrC9do21NfFblIiIiDhkQERE0mCA5WX/u49vz5gQEBGRJDRnpcDdx7dnTAiIiEgS+Phj89r3uyMiIqImYYWAiIgkQYAMRivmEAhWHNsWMCEgIiJJ4JCBee373REREVGTsEJARESS8MtHGDf3+PaMCQEREUnCnacWWnN8e9a+3x0RERE1CSsEREQkCRwyMI8JARERSYIRDjBaURi35ti2oH2/OyIiImoSVgiIiEgSDIIMBivK/tYc2xYwISAiIkngHALzmBAQEZEkCFY+7VDgnQqJiIiovWOFgIiIJMEAGQxWPKDImmPbAiYEREQkCUbBunkARsGGwdyHOGRARERErBBIzY41any7pyPyLsjh4mrEg6EViF90Df4PVIt9tv5Vg4OfdUTRNWc4uwh4ILgSL/6hAH0HV9Q7nyAAf/x1D5w6oMCSTbkYGq0DAJw52gGvPf9AgzF8sCcHfQZW2ucNkuSF+F7DSwMz8VDnIqg9KjD7v6Ow71LgL3oImBl6Ci88eA4KeTW+u+6DP3/zGC4Uq8Qe/godXo04hsG+BXBxNODIlW5YduRR3Kx0BwAM8buKLWM/b/D6E/71HLKK1PZ8i9RMRisnFVpzbFvAhEBivjvWAaOn3kDvgRUw1AJJK3zxxq96YuOh/8HV3QgA6NKjCjOX5cM3oAbVVQ7Y9ffOWPirnth89Bw6ehlMzrdrY2fIGqjAPRhajn9mZpls2/KuL05/0wG9BzAZIPtxd9Yj56YXdv2vLz4Y9VW9/fEDMzFlwBm8sf9JXNIp8crgDHw0+j94+p+/QoXeBW5OemyM+QI5N73w4udjAABzHj6Bv0X/F7/6dDwEyJCp1eDxpCkm55398AlEdM1HVlHnFnmfZDkjZDBaMQ/AmmPbglZPd9atW4fAwEC4uroiJCQE33zzTWuH1K4t334RURNvoXufKvR8qArzV11B4VUXnP/OTezz5PgSDH68DL4BNejepwrTl15FxW1H5J5zMznXj2dd8e8NnTFv5ZV613F2EaBS14pN0akWaV8rMHLSrQYTCCJb+eZKAD44EYa9uT0a2CvgN/2/w4b0EOzN7YELt7ywcP+TcHWqRUyv8wCAQRotunjexhv7n8T5W144f8sLi/Y/if4+hQjvchUAoDc64kalu9hKquUY1v0SPv1fX6Cdf2lQ+9WqCcHOnTsxd+5cLFq0CKdPn8Zjjz2G6OhoXLlS/wuG7KO81BEA4NnR0OB+fY0Me7Z5wUNhQI8Hf/5lX1UhwzszumPmsnyo1LWNXufY10qU3nLCUxNu2SZwombo6nkbnT0qcDS/q7hNb3TEqWt+GKjRAgBcHA0QANQYHMU+1QZHGIwyDPYtaPC8w7pfQifXKuz+X1+7xk/WuXOnQmtae9aqCcHKlSsRHx+PadOmoV+/fli9ejX8/f2xfv361gxLMgQB+PvSLnjo4TJ071tlsi8tVYGxDwRjdGB/7NrYGYk7LkD5i+GCDUu74MHQcgwdVdqka331Ty+ERN6Guovepu+ByBLe7nXzYG5UuJtsv1HpBm+3uoT3zHUfVOqdMT/iGFyd9HBz0mNBxDE4Ogjo7F5/Hg0APNf3f/g2zx/a8g72fQNklTtzCKxp7Vmrvbuamhqkp6cjKirKZHtUVBSOHj3a4DHV1dUoLS01adR8f3ujC3Kz3bBw3eV6+wY+UoZ1qTlY9fl5hEbexrKXu6PkRt2Uk2NfKZD5rSde+dPVJl2n6Joz0g96YuSvbto0fqLmunv1mOwX24qr3PD7r6MQGXAZp6Z9hOPxm+DpUoOzRd4N/kL08SjDI/55+DerA9TGtdqkwhs3bsBgMMDHx8dku4+PD7RabYPHJCYm4q233mqJ8Nq9vy3qgmNfK/Hergvo7Ff/V7uruxFdAmvQJbAG/UIq8OIj/ZDyTxUmzS5E5reeKLjkgvF9g02OeTuhO4LCyvGXf18w2f71ThU8O9UiIkpn1/dE1Jg7lYHO7hW4UeEhbvdyq8TNyp/nyBzN98eo7ZPR0bUSBqMDbtfIcXhKEq7e9qx3zmf7/g8l1XIcuNTd7vGTdYyw8lkG7Xx+SKuvMpDdNcNMEIR62+5YuHAh5s2bJ74uLS2Fv7+/XeNrbwShLhk4mqLEX/51AZpuNU0+Tl9dV1CaOOs6omNNf+2//GRfvLz0KsKjSusd9/VOFUY8XwwnZ9u8B6Lmyr/tiaJyd0R0zUf2jbrVAM4OBoT6XcPKtPB6/Uuq6pKEsC75ULlVYn+9L30Bz/b9Hz7P6YNao2O94+n+Ili5ykBgQmAf3t7ecHR0rFcNKCwsrFc1uEMul0Mul7dEeO3W2je64sCuTli6+SLcOhhxq7DuT8DD0wC5m4CqCgdsf98HEVE6qHz0KL3lhC+2eONGgTMeG10CAOLKgbupu+jrJRiZRzpAe0WOUbEcLqCW4e6kRzflz9WoLopS9PW6AV21HAVlnvj4u/6YPjgDl3VKXNYpMX1wBqpqnfDF+V7iMc/2+R9+LOmI4ko3DPS5joWPHsHHZwbgUkknk2uFd7kKf8Vt/Du7X4u9P2o+Pu3QvFZLCFxcXBASEoLU1FQ8++yz4vbU1FSMHTu2tcJq977Y4g0AePW5Xibb56+6gqiJt+DgICD/ghxv/193lN5ygmcnA3oPqMB7u86je5+qhk5pVso/vfBgaBm69apuvDORDTykLjS5adAfHqmbk7Trf32w6MCT2JQ5EK5OtXjzsW/qbkxUqMa0L2JQoXcRj+nesQS/D0+DUl6Nq7c9sSE9BFu+61/vWuP7ZSOjQIOLdyUKRG2RTBCEVrs7886dOxEXF4cPP/wQERER+Pvf/46NGzfi7NmzCAgIaPT40tJSKJVKFP/QAwrP9j37k6TrwfUzWjsEIrsxVFfhwoo3oNPpoFAo7HKNO98Vz6a+CGcPl8YPuAd9eQ12PbXZrrG2plb9Fp04cSJWr16NP/3pTxg4cCAOHz6MPXv2NCkZICIissSdIQNrmiUOHz6M0aNHw8/PDzKZDLt37zbZL5PJGmx/+ctfxD6RkZH19k+aNMnkPMXFxYiLi4NSqYRSqURcXBxKSkos/nxa/Wf1jBkzcOnSJVRXVyM9PR2PP/54a4dERERktfLycgwYMABr165tcH9BQYFJ+8c//gGZTIbnnnvOpF9CQoJJvw0bNpjsj42NRWZmJlJSUpCSkoLMzEzExcVZHG+rrzIgIiJqCbZ6lsHd98C514T36OhoREdH3/N8Go3G5PVnn32GYcOGoUcP09tuu7u71+t7R3Z2NlJSUpCWloawsDAAwMaNGxEREYGcnBz06dOn8Tf2k1avEBAREbUEWw0Z+Pv7i+V5pVKJxMREq2O7fv06vvzyS8THx9fbl5ycDG9vbzz00ENYsGABbt++Le47duwYlEqlmAwAQHh4OJRK5T1v8ncvrBAQERFZIC8vz2RSoS2Ww2/ZsgWenp4YP368yfbJkycjMDAQGo0GWVlZWLhwIc6cOYPU1FQAgFarhVpd/3HbarX6njf5uxcmBEREJAm2ug+BQqGw+SqDf/zjH5g8eTJcXV1NtickJIj/DgoKQq9evRAaGoqMjAwMHjwYQP0b/AHmb/J3LxwyICIiSWjpVQZN9c033yAnJwfTpk1rtO/gwYPh7OyM8+frHtet0Whw/fr1ev2KiorueZO/e2FCQERE1Io2bdqEkJAQDBgwoNG+Z8+ehV6vh6+vLwAgIiICOp0OJ06cEPscP34cOp0OQ4cOtSgODhkQEZEktPSti8vKynDhws8Pe8vNzUVmZiZUKhW6desGoG7Fwv/93//hvffeq3f8jz/+iOTkZDz99NPw9vbGuXPnMH/+fAwaNAiPPPIIAKBfv34YNWoUEhISxOWI06dPR0xMjEUrDABWCIiISCIE/Lz0sDnN0tv6njp1CoMGDcKgQYMAAPPmzcOgQYPw5ptvin127NgBQRDwq1/9qt7xLi4u2LdvH0aOHIk+ffpgzpw5iIqKwt69e+Ho+PPDtJKTkxEcHIyoqChERUWhf//+2Lp1q8WfDysEREQkCS1dIYiMjERjTweYPn06pk+f3uA+f39/HDp0qNHrqFQqbNu2zaLYGsIKAREREbFCQERE0sDHH5vHhICIiCSBCYF5HDIgIiIiVgiIiEgaWCEwjwkBERFJgiDIIFjxpW7NsW0BhwyIiIiIFQIiIpKGOzcYsub49owJARERSQLnEJjHIQMiIiJihYCIiKSBkwrNY0JARESSwCED85gQEBGRJLBCYB7nEBARERErBEREJA2ClUMG7b1CwISAiIgkQQAgCNYd355xyICIiIhYISAiImkwQgYZ71R4T0wIiIhIErjKwDwOGRARERErBEREJA1GQQYZb0x0T0wIiIhIEgTBylUG7XyZAYcMiIiIiBUCIiKSBk4qNI8JARERSQITAvOYEBARkSRwUqF5nENARERErBAQEZE0cJWBeUwIiIhIEuoSAmvmENgwmPsQhwyIiIiIFQIiIpIGrjIwjxUCIiKSBMEGzRKHDx/G6NGj4efnB5lMht27d5vsnzp1KmQymUkLDw836VNdXY3Zs2fD29sbHh4eGDNmDPLz8036FBcXIy4uDkqlEkqlEnFxcSgpKbEwWiYEREREdlFeXo4BAwZg7dq19+wzatQoFBQUiG3Pnj0m++fOnYtdu3Zhx44dOHLkCMrKyhATEwODwSD2iY2NRWZmJlJSUpCSkoLMzEzExcVZHC+HDIiISBJsNWRQWlpqsl0ul0Mul9frHx0djejoaLPnlMvl0Gg0De7T6XTYtGkTtm7dihEjRgAAtm3bBn9/f+zduxcjR45EdnY2UlJSkJaWhrCwMADAxo0bERERgZycHPTp06fJ748VAiIikgYbjRn4+/uL5XmlUonExMRmh3Tw4EGo1Wr07t0bCQkJKCwsFPelp6dDr9cjKipK3Obn54egoCAcPXoUAHDs2DEolUoxGQCA8PBwKJVKsU9TsUJARETSYGWFAD8dm5eXB4VCIW5uqDrQFNHR0XjhhRcQEBCA3NxcLF68GE8++STS09Mhl8uh1Wrh4uKCTp06mRzn4+MDrVYLANBqtVCr1fXOrVarxT5NxYSAiIjIAgqFwiQhaK6JEyeK/w4KCkJoaCgCAgLw5ZdfYvz48fc8ThAEyGQ/Jza//Pe9+jQFhwyIiEgS7typ0JpmT76+vggICMD58+cBABqNBjU1NSguLjbpV1hYCB8fH7HP9evX652rqKhI7NNUTAiIiEgS7kwqtKbZ082bN5GXlwdfX18AQEhICJydnZGamir2KSgoQFZWFoYOHQoAiIiIgE6nw4kTJ8Q+x48fh06nE/s0FYcMiIiI7KCsrAwXLlwQX+fm5iIzMxMqlQoqlQpLly7Fc889B19fX1y6dAlvvPEGvL298eyzzwIAlEol4uPjMX/+fHh5eUGlUmHBggUIDg4WVx3069cPo0aNQkJCAjZs2AAAmD59OmJiYixaYQAwISAiIqkQZOLEwGYfb4FTp05h2LBh4ut58+YBAKZMmYL169fj+++/x8cff4ySkhL4+vpi2LBh2LlzJzw9PcVjVq1aBScnJ0yYMAGVlZUYPnw4kpKS4OjoKPZJTk7GnDlzxNUIY8aMMXvvg3thQkBERJLQ0k87jIyMhGDmoK+++qrRc7i6umLNmjVYs2bNPfuoVCps27bNsuAawDkERERExAoBERFJRHMeSHD38e0YEwIiIpIEPu3QvCYlBB988EGTTzhnzpxmB0NERESto0kJwapVq5p0MplMxoSAiIjuX+287G+NJiUEubm59o6DiIjIrjhkYF6zVxnU1NQgJycHtbW1toyHiIjIPmz0tMP2yuKEoKKiAvHx8XB3d8dDDz2EK1euAKibO/DOO+/YPEAiIiKyP4sTgoULF+LMmTM4ePAgXF1dxe0jRozAzp07bRocERGR7chs0Novi5cd7t69Gzt37kR4eLjJoxUffPBB/PjjjzYNjoiIyGZ4HwKzLK4QFBUVQa1W19teXl5u8bOXiYiI6P5gcUIwZMgQfPnll+LrO0nAxo0bERERYbvIiIiIbImTCs2yeMggMTERo0aNwrlz51BbW4v3338fZ8+exbFjx3Do0CF7xEhERGS9Fn7aYVtjcYVg6NCh+Pbbb1FRUYGePXvi66+/ho+PD44dO4aQkBB7xEhERER21qxnGQQHB2PLli22joWIiMhuWvrxx21NsxICg8GAXbt2ITs7GzKZDP369cPYsWPh5MRnJRER0X2KqwzMsvgbPCsrC2PHjoVWq0WfPn0AAD/88AM6d+6Mzz//HMHBwTYPkoiIiOzL4jkE06ZNw0MPPYT8/HxkZGQgIyMDeXl56N+/P6ZPn26PGImIiKx3Z1KhNa0ds7hCcObMGZw6dQqdOnUSt3Xq1AnLli3DkCFDbBocERGRrciEumbN8e2ZxRWCPn364Pr16/W2FxYW4oEHHrBJUERERDbH+xCY1aSEoLS0VGzLly/HnDlz8K9//Qv5+fnIz8/Hv/71L8ydOxcrVqywd7xERERkB00aMujYsaPJbYkFQcCECRPEbcJPazFGjx4Ng8FghzCJiIisxBsTmdWkhODAgQP2joOIiMi+uOzQrCYlBE888YS94yAiIqJW1Ow7CVVUVODKlSuoqakx2d6/f3+rgyIiIrI5VgjMsjghKCoqwosvvoj//ve/De7nHAIiIrovMSEwy+Jlh3PnzkVxcTHS0tLg5uaGlJQUbNmyBb169cLnn39ujxiJiIjIziyuEOzfvx+fffYZhgwZAgcHBwQEBOCpp56CQqFAYmIinnnmGXvESUREZB2uMjDL4gpBeXk51Go1AEClUqGoqAhA3RMQMzIybBsdERGRjdy5U6E1rT1r1p0Kc3JyAAADBw7Ehg0bcPXqVXz44Yfw9fW1eYBERERkfxYPGcydOxcFBQUAgCVLlmDkyJFITk6Gi4sLkpKSbB0fERGRbXBSoVkWVwgmT56MqVOnAgAGDRqES5cu4eTJk8jLy8PEiRNtHR8REVGbdPjwYYwePRp+fn6QyWTYvXu3uE+v1+P1119HcHAwPDw84Ofnh9/85je4du2ayTkiIyMhk8lM2qRJk0z6FBcXIy4uDkqlEkqlEnFxcSgpKbE4XosTgru5u7tj8ODB8Pb2tvZUREREdiODlXMILLxeeXk5BgwYgLVr19bbV1FRgYyMDCxevBgZGRn49NNP8cMPP2DMmDH1+iYkJKCgoEBsGzZsMNkfGxuLzMxMpKSkICUlBZmZmYiLi7Mw2iYOGcybN6/JJ1y5cqXFQRAREbU30dHRiI6ObnCfUqlEamqqybY1a9bg4YcfxpUrV9CtWzdxu7u7OzQaTYPnyc7ORkpKCtLS0hAWFgYA2LhxIyIiIpCTk4M+ffo0Od4mJQSnT59u0sl++QCklvRs72A4yZxb5dpE9tb9gYLWDoHIbmoN1bjQUhez0bLD0tJSk81yuRxyudyayAAAOp0OMpkMHTt2NNmenJyMbdu2wcfHB9HR0ViyZAk8PT0BAMeOHYNSqRSTAQAIDw+HUqnE0aNHbZ8Q8OFGRETU5tloUqG/v7/J5iVLlmDp0qVWnBioqqrCH/7wB8TGxkKhUIjbJ0+ejMDAQGg0GmRlZWHhwoU4c+aMWF3QarXirQB+Sa1WQ6vVWhRDs59lQEREJEV5eXkmX9rWVgf0ej0mTZoEo9GIdevWmexLSEgQ/x0UFIRevXohNDQUGRkZGDx4MICGq/OCIFhctbd6UiEREVGbINigAVAoFCbNmoRAr9djwoQJyM3NRWpqqkmi0ZDBgwfD2dkZ58+fBwBoNBpcv369Xr+ioiL4+PhYFAsTAiIikoT77U6Fd5KB8+fPY+/evfDy8mr0mLNnz0Kv14s3AoyIiIBOp8OJEyfEPsePH4dOp8PQoUMtiodDBkRERHZQVlaGCxd+njKZm5uLzMxMqFQq+Pn54fnnn0dGRga++OILGAwGccxfpVLBxcUFP/74I5KTk/H000/D29sb586dw/z58zFo0CA88sgjAIB+/fph1KhRSEhIEJcjTp8+HTExMRZNKASYEBARkVS08J0KT506hWHDhomv7yzhnzJlCpYuXSo+IXjgwIEmxx04cACRkZFwcXHBvn378P7776OsrAz+/v545plnsGTJEjg6Oor9k5OTMWfOHERFRQEAxowZ0+C9DxrTrIRg69at+PDDD5Gbm4tjx44hICAAq1evRmBgIMaOHducUxIREdlXCycEkZGREIR7H2RuH1C3muHQoUONXkelUmHbtm2WBdcAi+cQrF+/HvPmzcPTTz+NkpISGAwGAEDHjh2xevVqqwMiIiKilmdxQrBmzRps3LgRixYtMilZhIaG4vvvv7dpcERERLZyv00qvN9YPGSQm5uLQYMG1dsul8tRXl5uk6CIiIhszkZ3KmyvLK4QBAYGIjMzs972//73v3jwwQdtERMREZHt2eg+BO2VxRWCV199FTNnzkRVVRUEQcCJEyfwz3/+E4mJifjoo4/sESMRERHZmcUJwYsvvoja2lq89tprqKioQGxsLLp06YL333+/3jOaiYiI7hfWzgPgHIIGJCQkICEhATdu3IDRaGzwwQpERET3lRZedtjWWHVjIm9vb1vFQURERK3I4oQgMDDQ7BOULl68aFVAREREdmHt0kFWCEzNnTvX5LVer8fp06eRkpKCV1991VZxERER2RaHDMyyOCH43e9+1+D2v/3tbzh16pTVAREREVHLs9njj6Ojo/Hvf//bVqcjIiKyLd6HwCybPe3wX//6F1Qqla1OR0REZFNcdmiexQnBoEGDTCYVCoIArVaLoqIirFu3zqbBERERUcuwOCEYN26cyWsHBwd07twZkZGR6Nu3r63iIiIiohZkUUJQW1uL7t27Y+TIkdBoNPaKiYiIyPa4ysAsiyYVOjk54be//S2qq6vtFQ8REZFd8PHH5lm8yiAsLAynT5+2RyxERETUSiyeQzBjxgzMnz8f+fn5CAkJgYeHh8n+/v372yw4IiIim2rnv/Kt0eSE4KWXXsLq1asxceJEAMCcOXPEfTKZDIIgQCaTwWAw2D5KIiIia3EOgVlNTgi2bNmCd955B7m5ufaMh4iIiFpBkxMCQahLjQICAuwWDBERkb3wxkTmWTSHwNxTDomIiO5rHDIwy6KEoHfv3o0mBbdu3bIqICIiImp5FiUEb731FpRKpb1iISIishsOGZhnUUIwadIkqNVqe8VCRERkPxwyMKvJNybi/AEiIqL2y+JVBkRERG0SKwRmNTkhMBqN9oyDiIjIrjiHwDyLb11MRETUJrFCYJbFDzciIiKi9ocJARERSYNgg2aBw4cPY/To0fDz84NMJsPu3btNwxEELF26FH5+fnBzc0NkZCTOnj1r0qe6uhqzZ8+Gt7c3PDw8MGbMGOTn55v0KS4uRlxcHJRKJZRKJeLi4lBSUmJZsGBCQEREEnFnDoE1zRLl5eUYMGAA1q5d2+D+d999FytXrsTatWtx8uRJaDQaPPXUU7h9+7bYZ+7cudi1axd27NiBI0eOoKysDDExMSYPEoyNjUVmZiZSUlKQkpKCzMxMxMXFWfz5cA4BERGRHURHRyM6OrrBfYIgYPXq1Vi0aBHGjx8PoO4hgj4+Pti+fTtefvll6HQ6bNq0CVu3bsWIESMAANu2bYO/vz/27t2LkSNHIjs7GykpKUhLS0NYWBgAYOPGjYiIiEBOTg769OnT5HhZISAiImmw0ZBBaWmpSauurrY4lNzcXGi1WkRFRYnb5HI5nnjiCRw9ehQAkJ6eDr1eb9LHz88PQUFBYp9jx45BqVSKyQAAhIeHQ6lUin2aigkBERFJgq2GDPz9/cXxeqVSicTERItj0Wq1AAAfHx+T7T4+PuI+rVYLFxcXdOrUyWyfhu4grFarxT5NxSEDIiIiC+Tl5UGhUIiv5XJ5s891912ABUFo9M7Ad/dpqH9TznM3VgiIiEgabDRkoFAoTFpzEgKNRgMA9X7FFxYWilUDjUaDmpoaFBcXm+1z/fr1eucvKiqqV31oDBMCIiKShhZedmhOYGAgNBoNUlNTxW01NTU4dOgQhg4dCgAICQmBs7OzSZ+CggJkZWWJfSIiIqDT6XDixAmxz/Hjx6HT6cQ+TcUhAyIiIjsoKyvDhQsXxNe5ubnIzMyESqVCt27dMHfuXCxfvhy9evVCr169sHz5cri7uyM2NhYAoFQqER8fj/nz58PLywsqlQoLFixAcHCwuOqgX79+GDVqFBISErBhwwYAwPTp0xETE2PRCgOACQEREUmE7KdmzfGWOHXqFIYNGya+njdvHgBgypQpSEpKwmuvvYbKykrMmDEDxcXFCAsLw9dffw1PT0/xmFWrVsHJyQkTJkxAZWUlhg8fjqSkJDg6Oop9kpOTMWfOHHE1wpgxY+557wOz709ow48xLC0thVKpRCTGwknm3NrhENmF4wOBrR0Ckd3UGqqx7+IH0Ol0JhP1bOnOd8WDv10OR7lrs89jqK7CufVv2DXW1sQKARERSQKfdmgeJxUSERERKwRERCQRfPyxWUwIiIhIOtr5l7o1OGRARERErBAQEZE0cFKheUwIiIhIGjiHwCwOGRARERErBEREJA0cMjCPCQEREUkDhwzM4pABERERsUJARETSwCED85gQEBGRNHDIwCwmBEREJA1MCMziHAIiIiJihYCIiKSBcwjMY0JARETSwCEDszhkQERERKwQEBGRNMgEATKh+T/zrTm2LWBCQERE0sAhA7M4ZEBERESsEBARkTRwlYF5TAiIiEgaOGRgFocMiIiIiBUCIiKSBg4ZmMeEgIiIpIFDBmYxISAiIklghcA8ziEgIiIiVgiIiEgiOGRgFhMCIiKSjPZe9rcGhwyIiIiIFQIiIpIIQahr1hzfjrFCQEREknBnlYE1zRLdu3eHTCar12bOnAkAmDp1ar194eHhJueorq7G7Nmz4e3tDQ8PD4wZMwb5+fm2+khMMCEgIiKyg5MnT6KgoEBsqampAIAXXnhB7DNq1CiTPnv27DE5x9y5c7Fr1y7s2LEDR44cQVlZGWJiYmAwGGweL4cMiIhIGmy0yqC0tNRks1wuh1wur9e9c+fOJq/feecd9OzZE0888YTJsRqNpsHL6XQ6bNq0CVu3bsWIESMAANu2bYO/vz/27t2LkSNHWvFm6mOFgIiIJEFmtL4BgL+/P5RKpdgSExMbvXZNTQ22bduGl156CTKZTNx+8OBBqNVq9O7dGwkJCSgsLBT3paenQ6/XIyoqStzm5+eHoKAgHD161HYfzE9YISAiIrJAXl4eFAqF+Lqh6sDddu/ejZKSEkydOlXcFh0djRdeeAEBAQHIzc3F4sWL8eSTTyI9PR1yuRxarRYuLi7o1KmTybl8fHyg1Wpt9n7uYEJACAorwwszitAruAJemlosfak7jqUoTfr4P1CF+D8WoH94GWQOwOUcVyx7JQBFV10AAHNW5GHQY2Xw8tGjssIB2ac8sGmZL/IuuLbGWyISPT02F8+My4WPpgIAcDnXE//c0henjvsAADp2qsKLr5zF4CFF8OigR9YZL3z4fn9cy+8gnsPJ2YBpM7LwxPCrkMsNyMzojL+tHICbRW6t8p6omWw0ZKBQKEwSgqbYtGkToqOj4efnJ26bOHGi+O+goCCEhoYiICAAX375JcaPH3/vMATBpMpgKxwyILi6G3HxrCv+tqhLg/t9A6qxcvcF5F2Q49Xne+K3I3pj+2of1FT9/Ad5/jt3vPd7fyQ80ReLYnsAMmD5Py/CwaF9L9Oh+9+NIlds3vAgfpcQid8lROJMRmcsXp6Gbt1LAQhYvOw4fP0q8Kc3wjA7PhKF192xfOW3kLvWiud4efb3GPpYAVa8FYoFsx6Dm1stlr5zjH/fbUxLrzK44/Lly9i7dy+mTZtmtp+vry8CAgJw/vx5AIBGo0FNTQ2Ki4tN+hUWFsLHx6d5wZjRqgnB4cOHMXr0aPj5+UEmk2H37t2tGY5knTqgwJZ3ffHtfzs2uH/qH7Q4sV+BTX/2w49Z7tBekePEPgV0N53FPv9N9kLW8Q64nu+CC9+7Y8sKDdRd9PDxr2mhd0HUsBNHfXEqTYOr+R1wNb8DPv7oQVRVOqHvQ7fQpWs5+gUVY+17A3D+f51wNc8T61YOgKtbLSKH1y3tcvfQI+qZy/hoXRAy09W4eL4j/vJ2CLr3KMXAkMJGrk73lTv3IbCmNcPmzZuhVqvxzDPPmO138+ZN5OXlwdfXFwAQEhICZ2dncXUCABQUFCArKwtDhw5tVizmtGpCUF5ejgEDBmDt2rWtGQaZIZMJeHh4Ka5elGPZ9h+x87uzeP+L84gYpbvnMXI3A6Im3kLBZRcUXXO+Zz+ilubgIODxJ/Ph6mpAdpYKzi51S7dqahzFPkajDLW1Dniw/00AQK8+JXB2FpBxQi32uXXTDZdzFegXdKtl3wC1OUajEZs3b8aUKVPg5PTzKH1ZWRkWLFiAY8eO4dKlSzh48CBGjx4Nb29vPPvsswAApVKJ+Ph4zJ8/H/v27cPp06fx61//GsHBweKqA1tq1TkE0dHRiI6ObnL/6upqVFdXi6/vXvpBttfRuxbuHYyYOKsQSSs02LTMD6HDSvHmR5fw2vM98X3az+OsMVNuYNofC+DmYcSV83IsnNQDtXqOSlHr695Dh/fWHYaLixGVlY54+48PI++yAo6ORlwvcMOL089izV8HoqrKCc9OvACVVzVUXnX/remkqoK+xgFlZS4m5ywplqOTV3VDl6P7VGs8/njv3r24cuUKXnrpJZPtjo6O+P777/Hxxx+jpKQEvr6+GDZsGHbu3AlPT0+x36pVq+Dk5IQJEyagsrISw4cPR1JSEhwdHe++lNXa1KTCxMREvPXWW60dhqTIfvo+P/aVArs21q2pvXjWDQ+GVuCZ39w0SQj2f9oJGYc9oVLr8fxvi7Bow2X8fuwD0FczKaDWlX/FE7Pih6FDBz0eeeIa5r+RgddmP4q8ywosWxyG372egU/27IGhVobT6Z1xMq3x8VkZ0O6fftfutMLTDqOioiA0MNTg5uaGr776qtHjXV1dsWbNGqxZs8byi1uoTf2XeuHChdDpdGLLy8tr7ZDavdJbjqjVA5d/MF0tkHdeDnUX0/kBFbcdcS1XjqzjHfDnhAD4P1CNR6LvPbRA1FJqax1QcLUDzud0QtLfH8LFC0qMfeEiAODCDx0xO/5JPB/9DCY/OwpvvjoUCkUNrhe4AwCKb7nC2cWIDh1M/96VnapRfKvx5WZEbUWbSgjkcrm43KM5yz7IcrV6B/xwxh1de5qWRrv0qEZhvss9jvqJTICzC39C0f1HJhPg7Gx669eKcmeU6uTw61qGB/oU49iRuold53M6Qq+XYdCQIrFvJ68qBASWIjtL1aJxk3Vaa5VBW9GmhgzIPlzdDfAL/PnXj8a/Bj0eqsTtEkcUXXXB/61T440PLyMrzQNnjnZA6LDbCH+qFK8+37Ouf7dqPDGmBOmHPKG75QRvjR4TZhaiptIBJ/Z53uuyRC1iSsI5nDquRlGhG9zda/H4k1cRPPAG3ny1bpb2o5FXoStxQdF1d3TvWYqXZ3+HtCO+OH2ybhJhRbkzvv4yANNmZqFU54Lbt50xbUYWLl1UIDNdbe7SdL/h0w7NYkJA6D2gEn/594/i61feugYA+HpnJ7z3+244mqLEB3/ogkmzCvHbt68i/6Icbyd0x9kTdfMHaqodEBRWjmcTbqCD0oCSG074Ps0Dvx/7gMnSRKLW0FFVhQWL0qHyqkZ5uRNyf1TizVeH4vSpui9zlVcVEmZloWOnKhTfdMW+r/zxzy19Tc7x97XBMBhkWPjWCbjIjTiT7o2VieEwGm1/cxii1iITGprt0ELKyspw4cIFAMCgQYOwcuVKDBs2DCqVCt26dWv0+NLSUiiVSkRiLJxk/OKh9snxgcDWDoHIbmoN1dh38QPodDq7DQPf+a6IiP4TnJybf/fUWn0Vjv33TbvG2ppatUJw6tQpDBs2THw9b948AMCUKVOQlJTUSlEREVG71AqrDNqSVk0IIiMjG1yOQURERC2LcwiIiEgSWuPGRG0JEwIiIpIGo1DXrDm+HWNCQERE0sA5BGa1qRsTERERkX2wQkBERJIgg5VzCGwWyf2JCQEREUkD71RoFocMiIiIiBUCIiKSBi47NI8JARERSQNXGZjFIQMiIiJihYCIiKRBJgiQWTEx0Jpj2wImBEREJA3Gn5o1x7djHDIgIiIiVgiIiEgaOGRgHhMCIiKSBq4yMIsJARERSQPvVGgW5xAQERERKwRERCQNvFOheUwIiIhIGjhkYBaHDIiIiIgVAiIikgaZsa5Zc3x7xoSAiIikgUMGZnHIgIiIiFghICIiieCNicxihYCIiCThzq2LrWmWWLp0KWQymUnTaDTifkEQsHTpUvj5+cHNzQ2RkZE4e/asyTmqq6sxe/ZseHt7w8PDA2PGjEF+fr5NPo+7MSEgIiKyk4ceeggFBQVi+/7778V97777LlauXIm1a9fi5MmT0Gg0eOqpp3D79m2xz9y5c7Fr1y7s2LEDR44cQVlZGWJiYmAwGGweK4cMiIhIGlphUqGTk5NJVeDnUwlYvXo1Fi1ahPHjxwMAtmzZAh8fH2zfvh0vv/wydDodNm3ahK1bt2LEiBEAgG3btsHf3x979+7FyJEjm/9eGsAKARERSYMAwGhF+ykfKC0tNWnV1dX3vOT58+fh5+eHwMBATJo0CRcvXgQA5ObmQqvVIioqSuwrl8vxxBNP4OjRowCA9PR06PV6kz5+fn4ICgoS+9gSEwIiIpIEW80h8Pf3h1KpFFtiYmKD1wsLC8PHH3+Mr776Chs3boRWq8XQoUNx8+ZNaLVaAICPj4/JMT4+PuI+rVYLFxcXdOrU6Z59bIlDBkRERBbIy8uDQqEQX8vl8gb7RUdHi/8ODg5GREQEevbsiS1btiA8PBwAIJPJTI4RBKHetrs1pU9zsEJARETSIODneQTNanWnUSgUJu1eCcHdPDw8EBwcjPPnz4vzCu7+pV9YWChWDTQaDWpqalBcXHzPPrbEhICIiKTBqmTAygmJqFtCmJ2dDV9fXwQGBkKj0SA1NVXcX1NTg0OHDmHo0KEAgJCQEDg7O5v0KSgoQFZWltjHljhkQEREZAcLFizA6NGj0a1bNxQWFuLPf/4zSktLMWXKFMhkMsydOxfLly9Hr1690KtXLyxfvhzu7u6IjY0FACiVSsTHx2P+/Pnw8vKCSqXCggULEBwcLK46sCUmBEREJA1GANYMvVv4cKP8/Hz86le/wo0bN9C5c2eEh4cjLS0NAQEBAIDXXnsNlZWVmDFjBoqLixEWFoavv/4anp6e4jlWrVoFJycnTJgwAZWVlRg+fDiSkpLg6OhoxRtpmEwQ2u7TGkpLS6FUKhGJsXCSObd2OER24fhAYGuHQGQ3tYZq7Lv4AXQ6nclEPVu6810xPOg1ODk2bby/IbWGauzLeteusbYmziEgIiIiDhkQEZFE8PHHZjEhICIiaWBCYBaHDIiIiIgVAiIikghWCMxiQkBERNLQwssO2xomBEREJAm/fEBRc49vzziHgIiIiFghICIiieAcArOYEBARkTQYBUBmxZe6sX0nBBwyICIiIlYIiIhIIjhkYBYTAiIikggrEwK074SAQwZERETECgEREUkEhwzMYkJARETSYBRgVdmfqwyIiIiovWOFgIiIpEEw1jVrjm/HmBAQEZE0cA6BWUwIiIhIGjiHwCzOISAiIiJWCIiISCI4ZGAWEwIiIpIGAVYmBDaL5L7EIQMiIiJihYCIiCSCQwZmMSEgIiJpMBoBWHEvAWP7vg8BhwyIiIiIFQIiIpIIDhmYxYSAiIikgQmBWRwyICIiIlYIiIhIInjrYrNYISAiIkkQBKPVzRKJiYkYMmQIPD09oVarMW7cOOTk5Jj0mTp1KmQymUkLDw836VNdXY3Zs2fD29sbHh4eGDNmDPLz863+PO7GhICIiKRBEOp+5Te3WTiH4NChQ5g5cybS0tKQmpqK2tpaREVFoby83KTfqFGjUFBQILY9e/aY7J87dy527dqFHTt24MiRIygrK0NMTAwMBoPVH8kvcciAiIjIDlJSUkxeb968GWq1Gunp6Xj88cfF7XK5HBqNpsFz6HQ6bNq0CVu3bsWIESMAANu2bYO/vz/27t2LkSNH2ixeVgiIiEga7qwysKYBKC0tNWnV1dVNurxOpwMAqFQqk+0HDx6EWq1G7969kZCQgMLCQnFfeno69Ho9oqKixG1+fn4ICgrC0aNHrf1ETDAhICIiaTAarW8A/P39oVQqxZaYmNjopQVBwLx58/Doo48iKChI3B4dHY3k5GTs378f7733Hk6ePIknn3xSTDK0Wi1cXFzQqVMnk/P5+PhAq9Xa8MPhkAEREZFF8vLyoFAoxNdyubzRY2bNmoXvvvsOR44cMdk+ceJE8d9BQUEIDQ1FQEAAvvzyS4wfP/6e5xMEATKZrBnR3xsrBEREJA02GjJQKBQmrbGEYPbs2fj8889x4MABdO3a1WxfX19fBAQE4Pz58wAAjUaDmpoaFBcXm/QrLCyEj4+PFR9GfUwIiIhIEgSj0epm0fUEAbNmzcKnn36K/fv3IzAwsNFjbt68iby8PPj6+gIAQkJC4OzsjNTUVLFPQUEBsrKyMHToUMs+gEZwyICIiMgOZs6cie3bt+Ozzz6Dp6enOOavVCrh5uaGsrIyLF26FM899xx8fX1x6dIlvPHGG/D29sazzz4r9o2Pj8f8+fPh5eUFlUqFBQsWIDg4WFx1YCtMCIiISBoEK+9UaOF9CNavXw8AiIyMNNm+efNmTJ06FY6Ojvj+++/x8ccfo6SkBL6+vhg2bBh27twJT09Psf+qVavg5OSECRMmoLKyEsOHD0dSUhIcHR2b/14awISAiIikwSgAspZLCIRG+ru5ueGrr75q9Dyurq5Ys2YN1qxZY9H1LcU5BERERMQKARERSYQgALBsYmD949svJgRERCQJglGAYMWQQWNDAG0dEwIiIpIGwQjrKgRWHNsGcA4BERERsUJARETSwCED85gQEBGRNHDIwKw2nRDcydZqobfqXhNE9zPB0LRHqxK1RbXGur/vlvj1be13RS30tgvmPtSmE4Lbt28DAI5gTytHQmRHF1s7ACL7u337NpRKpV3O7eLiAo1GgyNa678rNBoNXFxcbBDV/UcmtOFBEaPRiGvXrsHT09Pmj4GkhpWWlsLf37/e4z+J2gP+fbc8QRBw+/Zt+Pn5wcHBfvPcq6qqUFNTY/V5XFxc4OrqaoOI7j9tukLg4ODQ6KMkyT7uPPaTqD3i33fLsldl4JdcXV3b7Re5rXDZIRERETEhICIiIiYEZCG5XI4lS5ZALpe3dihENse/b5KyNj2pkIiIiGyDFQIiIiJiQkBERERMCIiIiAhMCIiIiAhMCMgC69atQ2BgIFxdXRESEoJvvvmmtUMisonDhw9j9OjR8PPzg0wmw+7du1s7JKIWx4SAmmTnzp2YO3cuFi1ahNOnT+Oxxx5DdHQ0rly50tqhEVmtvLwcAwYMwNq1a1s7FKJWw2WH1CRhYWEYPHgw1q9fL27r168fxo0bh8TExFaMjMi2ZDIZdu3ahXHjxrV2KEQtihUCalRNTQ3S09MRFRVlsj0qKgpHjx5tpaiIiMiWmBBQo27cuAGDwQAfHx+T7T4+PtBqta0UFRER2RITAmqyux8xLQgCHztNRNROMCGgRnl7e8PR0bFeNaCwsLBe1YCIiNomJgTUKBcXF4SEhCA1NdVke2pqKoYOHdpKURERkS05tXYA1DbMmzcPcXFxCA0NRUREBP7+97/jypUreOWVV1o7NCKrlZWV4cKFC+Lr3NxcZGZmQqVSoVu3bq0YGVHL4bJDarJ169bh3XffRUFBAYKCgrBq1So8/vjjrR0WkdUOHjyIYcOG1ds+ZcoUJCUltXxARK2ACQERERFxDgERERExISAiIiIwISAiIiIwISAiIiIwISAiIiIwISAiIiIwISAiIiIwISAiIiIwISCy2tKlSzFw4EDx9dSpUzFu3LgWj+PSpUuQyWTIzMy8Z5/u3btj9erVTT5nUlISOnbsaHVsMpkMu3fvtvo8RGQ/TAioXZo6dSpkMhlkMhmcnZ3Ro0cPLFiwAOXl5Xa/9vvvv9/k29025UuciKgl8OFG1G6NGjUKmzdvhl6vxzfffINp06ahvLwc69evr9dXr9fD2dnZJtdVKpU2OQ8RUUtihYDaLblcDo1GA39/f8TGxmLy5Mli2fpOmf8f//gHevToAblcDkEQoNPpMH36dKjVaigUCjz55JM4c+aMyXnfeecd+Pj4wNPTE/Hx8aiqqjLZf/eQgdFoxIoVK/DAAw9ALpejW7duWLZsGQAgMDAQADBo0CDIZDJERkaKx23evBn9+vWDq6sr+vbti3Xr1plc58SJExg0aBBcXV0RGhqK06dPW/wZrVy5EsHBwfDw8IC/vz9mzJiBsrKyev12796N3r17w9XVFU899RTy8vJM9v/nP/9BSEgIXF1d0aNHD7z11luora21OB4iaj1MCEgy3NzcoNfrxdcXLlzAJ598gn//+99iyf6ZZ56BVqvFnj17kJ6ejsGDB2P48OG4desWAOCTTz7BkiVLsGzZMpw6dQq+vr71vqjvtnDhQqxYsQKLFy/GuXPnsH37dvj4+ACo+1IHgL1796KgoACffvopAGDjxo1YtGgRli1bhuzsbCxfvhyLFy/Gli1bAADl5eWIiYlBnz59kJ6ejqVLl2LBggUWfyYODg744IMPkJWVhS1btmD//v147bXXTPpUVFRg2bJl2LJlC7799luUlpZi0qRJ4v6vvvoKv/71rzFnzhycO3cOGzZsQFJSkpj0EFEbIRC1Q1OmTBHGjh0rvj5+/Ljg5eUlTJgwQRAEQViyZIng7OwsFBYWin327dsnKBQKoaqqyuRcPXv2FDZs2CAIgiBEREQIr7zyisn+sLAwYcCAAQ1eu7S0VJDL5cLGjRsbjDM3N1cAIJw+fdpku7+/v7B9+3aTbW+//bYQEREhCIIgbNiwQVCpVEJ5ebm4f/369Q2e65cCAgKEVatW3XP/J598Inh5eYmvN2/eLAAQ0tLSxG3Z2dkCAOH48eOCIAjCY489JixfvtzkPFu3bhV8fX3F1wCEXbt23fO6RNT6OIeA2q0vvvgCHTp0QG1tLfR6PcaOHYs1a9aI+wMCAtC5c2fxdXp6OsrKyuDl5WVynsrKSvz4448AgOzsbLzyyism+yMiInDgwIEGY8jOzkZ1dTWGDx/e5LiLioqQl5eH+Ph4JCQkiNtra2vF+QnZ2dkYMGAA3N3dTeKw1IEDB7B8+XKcO3cOpaWlqK2tRVVVFcrLy+Hh4QEAcHJyQmhoqHhM37590bFjR2RnZ+Phhx9Geno6Tp48aVIRMBgMqKqqQkVFhUmMRHT/YkJA7dawYcOwfv16ODs7w8/Pr96kwTtfeHcYjUb4+vri4MGD9c7V3KV3bm5uFh9jNBoB1A0bhIWFmexzdHQEAAiC0Kx4funy5ct4+umn8corr+Dtt9+GSqXCkSNHEB8fbzK0AtQtG7zbnW1GoxFvvfUWxo8fX6+Pq6ur1XESUctgQkDtloeHBx544IEm9x88eDC0Wi2cnJzQvXv3Bvv069cPaWlp+M1vfiNuS0tLu+c5e/XqBTc3N+zbtw/Tpk2rt9/FxQVA3S/qO3x8fNClSxdcvHgRkydPbvC8Dz74ILZu3YrKykox6TAXR0NOnTqF2tpavPfee3BwqJtO9Mknn9TrV1tbi1OnTuHhhx8GAOTk5KCkpAR9+/YFUPe55eTkWPRZE9H9hwkB0U9GjBiBiIgIjBs3DitWrECfPn1w7do17NmzB+PGjUNoaCh+97vfYcqUKQgNDcWjjz6K5ORknD17Fj169GjwnK6urnj99dfx2muvwcXFBY888giKiopw9uxZxMfHQ61Ww83NDSkpKejatStcXV2hVCqxdOlSzJkzBwqFAtHR0aiursapU6dQXFyMefPmITY2FosWLUJ8fDz++Mc/4tKlS/jrX/9q0fvt2bMnamtrsWbNGowePRrffvstPvzww3r9nJ2dMXv2bHzwwQdwdnbGrFmzEB4eLiYIb775JmJiYuDv748XXngBDg4O+O677/D999/jz3/+s+X/RxBRq+AqA6KfyGQy7NmzB48//jheeukl9O7dG5MmTcKlS5fEVQETJ07Em2++iddffx0hISG4fPkyfvvb35o97+LFizF//ny8+eab6NevHyZOnIjCwkIAdePzH3zwATZs2AA/Pz+MHTsWADBt2jR89NFHSEpKQnBwMJ544gkkJSWJyxQ7dOiA//znPzh37hwGDRqERYsWYcWKFRa934EDB2LlypVYsWIFgoKCkJycjMTExHr93N3d8frrryM2NhYRERFwc3PDjh07xP0jR47EF198gdTUVAwZMgTh4eFYuXIlAgICLIqHiFqXTLDFYCQRERG1aawQEBERERMCIiIiYkJAREREYEJAREREYEJAREREYEJAREREYEJAREREYEJAREREYEJAREREYEJAREREYEJAREREAP4fdp5r61BAp6IAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 640x480 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# predicted classes using default 0.5 threshold\n",
    "y_hat = fit_lr_grid_search.predict(X_train) \n",
    "\n",
    "print(\"classification report:\\n \", classification_report(y_train, y_hat, digits=3))\n",
    "print(\"confusion matrix:\\n \")\n",
    "\n",
    "# Create the confusion matrix\n",
    "cm = confusion_matrix(y_train, y_hat)\n",
    "ConfusionMatrixDisplay(confusion_matrix=cm).plot()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9e45e045-8594-46ba-8a5a-e22878751443",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "### Receiver Operating Characteristic Curve\n",
    "\n",
    "A ROC plots the true positive rate (sensitivity) vs false positive rate (1-specificity) at different decision thresholds. This allows us to calculate the area under the curve (AUC) to estimate model performance. An AUC of 0.5 indicates a model that performs as well as a coin flip (50/50)."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4cb73c5f-ce83-46d4-a0b3-48091b12d4c8",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "source": [
    "The use of the `GridSearchCV()` function, automatically refits the best trained/tuned model using all of our data.\n",
    "\n",
    "Thus, we can use the fitted object to evaluate some performance metrics"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 91,
   "id": "7109aa7d-c3f3-477b-aa7f-50bc3b5c00f1",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "# calculate the predicted probabilities on all of the training data\n",
    "y_proba = fit_lr_grid_search.predict_proba(X_train)[:,1] \n",
    "\n",
    "# calculate the area under the curve (AUC)\n",
    "auc = roc_auc_score(y_train, y_proba)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 92,
   "id": "1fc0bbd8-4dbf-4920-83ac-560466ed4752",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0.5, 0, 'False Positive Rate')"
      ]
     },
     "execution_count": 92,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAkIAAAHFCAYAAAAe+pb9AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8hTgPZAAAACXBIWXMAAA9hAAAPYQGoP6dpAABlB0lEQVR4nO3deXiMV/sH8O9kj5BYE0kQlDRBFUmtRXkRtJS2GkttRSmKpKVUK3jb+lVbr52WEDSIWFptUanWThFRa0sJsSRVW0Iisp3fH6czMskkZpKZPLN8P9c1V/OceZ6Ze/qEuZ1zn3NUQggBIiIiIhtkp3QAREREREphIkREREQ2i4kQERER2SwmQkRERGSzmAgRERGRzWIiRERERDaLiRARERHZLCZCREREZLOYCBEREZHNYiJERDpFRUVBpVJpHg4ODvD29kbfvn1x4cIFnddkZ2djyZIlaNWqFTw8PODq6orAwEBMnjwZt2/f1nlNXl4e1qxZg06dOqFq1apwdHSEp6cnXnrpJXz//ffIy8t7Yqz37t1D1apVsX79ely+fFkr7uIely9fLs3/IgwZMgS1a9cu0bWRkZHw9fVFenp6qWIgotJRcYsNItIlKioKQ4cOxcqVKxEQEIDMzEwcOHAAn3zyCSpUqIA//vgDlSpV0pyfkZGB7t27Y//+/Xjrrbfw0ksvwdXVFYcOHcIXX3yB8uXLIy4uDk8//bTmmszMTPTq1Qs7d+5E37590bt3b1SvXh3//PMPduzYgdWrVyMmJgYvv/xysbGGhYVh165d+P3335GVlYWEhASt50ePHo3U1FRER0drtTdt2hTOzs4l/n908eJFpKWloWnTpgZfm5OTgwYNGqBfv36YMWNGiWMgolISREQ6rFy5UgAQR48e1WqfMWOGACBWrFih1f7WW28JAGL9+vWFXuvPP/8UHh4eomHDhiInJ0fT/vbbbwsAYtWqVTpjOH/+vPj999+LjfP27dvC1dVVLF26tMhz2rdvLxo2bFjs6wghREZGxhPPMaYvvvhCeHh4iPT09DJ9XyJ6jENjRGSQ4OBgAMDff/+taUtJScGKFSsQEhKC0NDQQtf4+/vj/fffx5kzZ/Dtt99qrlm+fDlCQkIwaNAgne9Vv359NG7cuNh4oqKikJOTo/N9i1O7dm289NJL2Lx5M5o2bQoXFxdNz8yiRYvQrl07eHp6ws3NDc888wxmz56N7OxsrdfQNTSmUqkwduxYrFmzBoGBgShXrhyeffZZ/PDDD4ViGDBgANLS0rB+/XqDYici42EiREQGSUxMBCCTG7Vff/0VOTk56NWrV5HXqZ+Li4vTXJOdnV3sNfr48ccf0bRpU1SsWNHga48fP46JEydi3Lhx2LFjB1599VUAcsirf//+WLNmDX744QcMGzYMn3/+OUaOHKl3TAsXLsTMmTOxadMmVK5cGb1798alS5e0zqtevToCAgLw448/Ghw7ERmHg9IBEJF5y83NRU5OjqZG6OOPP0a7du3Qs2dPzTlJSUkAgDp16hT5Ourn1Ofqc40+Dh8+XGSP0pPcvHkTZ8+e1UrqAGDOnDman/Py8tC2bVtUqVIFQ4cOxZdffqlVG6XLw4cP8fPPP6NChQoAgGbNmsHHxwcbNmzA5MmTtc5t1qwZfv755xLFT0Slx0SIiIrVsmVLrePAwEB89913cHAo2V8fKpXKGGEBkLPFMjIy4OnpWaLrGzduXCgJAoCEhARERETgwIEDuHPnjtZz58+fR4sWLYp93Q4dOmiSIADw8vKCp6cnrly5UuhcT09P3Lx5Ezk5OSX+f0pEJcehMSIq1urVq3H06FH88ssvGDlyJM6dO4d+/fppnVOrVi0Aj4fNdFE/V7NmTb2veZKHDx8CAFxcXEp0vbe3d6G2pKQktG3bFtevX8e8efOwb98+HD16FIsWLdJ6z+JUqVKlUJuzs7POa11cXCCEQGZmZgk+ARGVFv/5QUTFCgwM1BRId+jQAbm5uVi+fDk2btyI1157TdPu4OCAb7/9FqNGjdL5Ouoi6c6dO2uucXR0LPaaJ1EnHAV7bfSlq3fq22+/RXp6OjZv3gw/Pz9N+4kTJ0r0Hk9y584dODs7o3z58iZ5fSIqHnuEiMggs2fPRqVKlTBt2jTNYofVq1fHm2++iZ9++gkxMTGFrjl//jw+++wzNGzYUFMcXb16dQwfPhw//fQTVq9erfO9Ll68iJMnTxYZi5OTE+rWrYuLFy+W/oP9S50c5V9fSAiBZcuWGe098rt06RIaNGhgktcmoidjIkREBqlUqRKmTJmCc+fOYe3atZr2OXPmoH379njjjTcwZswY7NixA7/++itmzZqFVq1aoUKFCti0aRPs7e21rgkJCcGQIUMwYMAAbNy4Efv27cOWLVswevRoNGrU6IlDZy+88AIOHz5stM/XuXNnODk5oV+/fti+fTu2bNmCkJAQ3L1712jvoZaXl4cjR46gQ4cORn9tItIPEyEiMtg777yDWrVqYebMmcjNzQUAuLm5IS4uDvPmzUN8fDz69OmDbt26YdWqVRg+fDhOnDihtao0IOtjfvzxR0RFRSElJQUjR45Ex44dMXLkSFy+fBkrVqxAjx49io1lwIABSE5OxtGjR43y2QICArBp0ybcvXsXr7zyCt555x00adIE8+fPN8rr57d7926kpqZiwIABRn9tItIPt9ggIovXuHFjtGnTBkuWLFE6FIMMHDgQly5dwoEDB5QOhchmMREiIou3Y8cO9O7dGxcuXECNGjWUDkcvFy9eRGBgIH755Rc8//zzSodDZLM4NEZEFq9r1674/PPPSzUVv6wlJSVh4cKFTIKIFMYeISIiIrJZivYI7d27Fz169ICPjw9UKpVmnZHi7NmzB0FBQXBxcUHdunWxdOlS0wdKREREVknRRCg9PR3PPvssFi5cqNf5iYmJ6N69O9q2bYuEhAR88MEHGDduHDZt2mTiSImIiMgamc3QmEqlwpYtW4rdifr999/H1q1bce7cOU3bqFGj8Pvvv+PQoUNlECURERFZE4vaYuPQoUPo0qWLVltISAgiIyORnZ0NR0fHQtc8evQIjx490hzn5eXhzp07qFKlilE3fyQiIiLTEULg/v378PHxgZ2d8Qa0LCoRSklJgZeXl1abl5cXcnJycOvWLZ0bKM6aNQszZswoqxCJiIjIhK5evWrUZTIsKhECCm+SqB7ZK6p3Z8qUKQgPD9ccp6amolatWrh69Src3d1NFygREZEN27IF+OQT4MEDeZycrP+16n6NNpm7cM6xMe7YV0NeXhr+/rsmKlSoYNQ4LSoRql69OlJSUrTabt68CQcHB80u1AU5OztrbZ6o5u7uzkSIiIioFGJjgWnTgPv3Cz93/XrR1/n66m6vUAH473+B117OBj76CPjsM6BLF2D7dqQ9ADw8iu74KCmLSoRatWqF77//Xqtt586dCA4O1lkfRERERKYzbRrwxx9PPk+d+GgSndeKOfnqVeCFvsDBg/K4Xj0gJ6fUsRZF0UTowYMH+OuvvzTHiYmJOHHiBCpXroxatWphypQpuH79OlavXg1AzhBbuHAhwsPDMWLECBw6dAiRkZFYt26dUh+BiIjI4hTXk2MI9XCXnd3j4az89Ep88vvhB2DwYODOHcDdHVi+HOjTRz6XmVm6YIugaCJ07NgxdOjQQXOsruUZPHgwoqKikJycjKSkJM3zderUwbZt2xAWFoZFixbBx8cH8+fPx6uvvlrmsRMRERmLsRITfRU3bFUS/v5AvpVtDJedDUyZAnz5pTwOCgJiYoCnnjJKfMUxm3WEykpaWho8PDyQmprKGiEiIjIpfRMcYycmhiiqXkdfBvf66HL/PtCsGfDXX8D48bI2qEB9r6m+vy2qRoiIiMgcFZXwlCTBKW1ioi+jJDDGUqECsGEDcOUKUMzCyqbAHiEiIqJSCgx8ctHwkxIcs0pMTO3RI2DSJDn0NW6cXpewR4iIiMgMxcY+ToJ0FQ3bVIKjj4sXgdBQID5eDn+99hrg46NYOEyEiIiI/lWSouX8w1+lLhq2drGxwPDhQFoaUKUKsGqVokkQwESIiIgIgPyOfv310r3Gf/9rnFisTmYmEB4OLFkij9u0AdavB4y4VUZJMREiIiKrZUgPT8HCZkOKljn8VYycHKBdO+DoUXk8ZQowcybgYB4piHlEQUREZAL6rnxcUGwskxqjcXCQ/zMvXwbWrAFCQpSOSAtnjRERkcV6Uo9PcjKQl1f0yscFsWfHSDIygJs3gdq15XFeHnDrFuDpWeKX5KwxIiKiAvTt8WERcxk6d04WW+XmyuEwNzeZiZYiCTIlJkJERGSW9KnvedJeV8DjXh4qA6tWAaNHyx4hLy85Vb5xY6WjKhYTISIiMkuG1Pewx0dh6enAmDEyEQKA//wH+OYboHp1ZePSAxMhIiIyO09apDA/9vgo7PRpORR27py8WdOnAx98ANjbKx2ZXpgIERFRmdJnyIuLFFqQ99+XN8jHB1i7FmjfXumIDMJEiIiIykxJFi1kb4+ZW74cmDgR+N//gGrVlI7GYEyEiIjIpPL3ABmyaCGnspupEyeA7dvlwoiAHLf85htFQyoNJkJERGRUBYe+CiY/+c9jkmNBhACWLgXCwuTu8YGBQK9eSkdVakyEiIjIYMXV+RSV+ACyB4g9PRYoNRUYMULeeAB46SWgbVtlYzISJkJERKQ3dQKk77R29dAXkx8LduwYEBoKXLokt8v47DPZK6RSKR2ZUTARIiKiQorq8dHV26OrzoeJj5VYtkyuD5SdDfj5ATExQIsWSkdlVEyEiIgIQPFFzboEBDDZsXrVqskkqFcvYMUKoFIlpSMyOiZCREQ2pCS1PQV7fNjbY+XS0+X+YIBMgHbvBtq1s5qhsIKYCBER2QhD1vBhUbMNEgKYMwf48kvgyBGgRg3ZbmELJBqKiRARkZV40orN+qzhw+THRt2+DQwZAvzwgzxeuRL46CNFQyorTISIiCycoTO51Ncw2SEAwIEDQN++wLVrgLOzXCF61CiloyozTISIiCyIrl4ffWdyAezxoXzy8oDZs4EPPwRyc4H69YENG4AmTZSOrEwxESIisiBP6vnhTC7S24IFj7fJ6N9frhpdoYKyMSmAiRARkRkqqt4nOVn+185ObvGkxp4eMtiIEUB0NDByJPDmm1Y7K+xJmAgREZnYk4qYdXnSOj7+/sC5c6WLi2xMbi6wbp3s/bGzA8qVAw4flj/bMCZCRERGUNK9t/RR1Do+RHpLSQHeeAPYtUsWRU+eLNttPAkCmAgREZVKSffe0geHu8godu0CBgwA/v5b9gIZ8ktoA5gIEREZ6ElbUXB9HjILubnAjBnAxx/LxRIbNZKzwgIDlY7MrDARIiLSgz77cHHGFpmNGzdkLdCePfJ4+HBg3jzZI0RamAgRERWg71o9ALeiIDP199/AoUNA+fLAV1/JpIh0YiJERFTAk2p+mPyQ2WvaFFizRi6O6O+vdDRmjYkQEdksrtVDVuPqVblX2GefAcHBsk3fHXZtHBMhIrJJ+uzEzrV6yCL8+CMwaBBw5w7w1ltAfLzNLo5YEkyEiMgmFOz9edJO7Fyrh8xedrbcIuPLL+VxUBAQE8MkyEBMhIjIJhRX98Od2MniXL4sd4z/7Td5PG6c3EDV2VnRsCwRl5QkIqsWGyuXTTl/Xh7b2cneH19fOd2dSRBZnPPnZTH0b78BFSsCmzfLqfFMgkqEPUJEZLV01QGx7ocsXr16QKtWwO3bciisdm2lI7JoTISIyGpNm6Z9rF7wkMjiXLoEeHkBbm6yW3PtWrk4opOT0pFZPA6NEZHVKTgcpm47d47DYGSBYmPlUNg77zxuq1iRSZCRsEeIiKxCcVtgBAQwASILlJkJhIcDS5bI4z//BDIyuE2GkTERIiKLZcj+X0QW5cIFWeB24oQ8njwZmDkTcHRUNCxrxESIiCyKPskPt8Agi7ZunVwY8cEDoGpVuVVG165KR2W1mAgRkUUpaj0gJj9kFVJTgfHjZRLUrp0sii642icZFYulicgiFLUekHotoGvXWAxNVsDDQ/YAffghsGsXk6AywB4hIjJr6qGwgr1AXA+IrMbq1bI7s3dveRwSIh9UJpgIEZFZedKeYAALoMlKpKcDY8cCUVGyJ+i554AaNZSOyuYwESIis1LcnmDqBIjDX2TxTp+Ws8LOnZPjvO++C3h7Kx2VTWKNEBGZBX32BGMNEFk8IYDISNn7c+6cTH5++QX46CPA3l7p6GwSe4SISFGsASKbkZsLDB4MREfL45AQWR/k6alsXDaOiRARlbknrQXEGiCySvb2QOXK8r8ffwxMmiS7PklRTISIyKQKFj8DT14FmsNfZDWEkEXR5cvL488/BwYOlENjZBaYCBGRSRVX/AxwIUSyYqmpcoXolBS5JpCDA+DszCTIzDARIiKTUPcE5S9+zj8phskPWbX4eCA0FLh4USZAhw8Dzz+vdFSkAxMhIjKa4mp/WPxMNkEIYOFC4L33gKwswM8PWL8eaNlS6cioCEyEiMgoYmPlsii6sPiZbMLdu8CwYcCWLfK4Vy9gxQqgUiVFw6LisVydiEpFvf5PwSSI6/+QzRk4UCZBjo7AvHnA5s1MgiwAe4SIyCD6bIERG8vEh2zQZ58BV64AK1cCwcFKR0N6Yo8QERlEPQvs+vXCSZC6B4hJENmEO3dkr49aw4bA778zCbIw7BEiIr3Fxj6eCp9/FhhngJHNOXgQ6NsXuHED2LMHaNNGtnOBRIvDRIiIdHrSQoicBUY2KS9PLoo4darcMqN+fcDNTemoqBQUT10XL16MOnXqwMXFBUFBQdi3b1+x50dHR+PZZ59FuXLl4O3tjaFDh+L27dtlFC2RbVDPAMs/BFZwGIyzwMjm/PMP8NJLwOTJMgnq10+uF9SkidKRUSkomgjFxMRgwoQJmDp1KhISEtC2bVt069YNSUlJOs/fv38/Bg0ahGHDhuHMmTOIjY3F0aNHMXz48DKOnMg6FTcDrOBO8BwGI5uyd69MeLZvB1xcgGXL5OapFSooHRmVkkoIIZR68xYtWqBZs2ZYsmSJpi0wMBC9evXCrFmzCp3/xRdfYMmSJbh48aKmbcGCBZg9ezauXr2q13umpaXBw8MDqampcHd3L/2HILIigYGFt8Ng0kMEYMECYNw4+S+BDRuAZ55ROiKbY6rvb8V6hLKyshAfH48uXbpotXfp0gUHDx7UeU3r1q1x7do1bNu2DUII/P3339i4cSNefPHFIt/n0aNHSEtL03oQ0WPqXqAaNbS3w2DPD9m8/P0EY8fKZOjoUSZBVkaxROjWrVvIzc2Fl5eXVruXlxdSUlJ0XtO6dWtER0cjNDQUTk5OqF69OipWrIgFCxYU+T6zZs2Ch4eH5lGzZk2jfg4iS5d/OnxenmxTF0IzCSKb9csvQLt2gPofzyqVTIbUu8iT1VC8WFqlUmkdCyEKtamdPXsW48aNw7Rp0xAfH48dO3YgMTERo0aNKvL1p0yZgtTUVM1D3yE0Imun7gnK3wukrgFiITTZrNxcICIC6NQJ2L8f+PhjpSMiE1Ns+nzVqlVhb29fqPfn5s2bhXqJ1GbNmoU2bdpg4sSJAIDGjRvDzc0Nbdu2xccffwzv/Ftb/8vZ2RnOzs7G/wBEFkzXvmCcDk8278YNYMAAYPdueTxsGDB9upIRURlQrEfIyckJQUFBiIuL02qPi4tD69atdV6TkZEBuwKLVdnb2wOQPUlEVLT8tUAFkyD2ApHN27lTzgrbvVuuC/TNN8Dy5UC5ckpHRiam6IKK4eHhGDhwIIKDg9GqVSt8/fXXSEpK0gx1TZkyBdevX8fq1asBAD169MCIESOwZMkShISEIDk5GRMmTEDz5s3h4+Oj5EchMnvqWqCCWBBNNm/NGmDQIPnzs8/KWWH+/srGRGVG0UQoNDQUt2/fxsyZM5GcnIxGjRph27Zt8PPzAwAkJydrrSk0ZMgQ3L9/HwsXLsS7776LihUromPHjvjss8+U+ghEZk+9QnT+WiBvb26LQaTRtSvg4wP07AnMmQO4uiodEZUhRdcRUgLXESJbU3BtoIAA1gIR4eRJoHHjx8e3bwNVqigXDz2R1a0jRESmpWtWGGuByOZlZwMTJ8ohsG++edzOJMhmcdNVIivEWWFEOly5IneMP3xYHp8+rWw8ZBbYI0RkRYraK4w9QWTzvvtOzgo7fBjw8AA2bQL+7/+UjorMAHuEiKyIrplhnBVGNi0rC5g0CZg3Tx4/9xwQEwPUqaNsXGQ22CNEZMHyrw3EvcKIdDh06HESFB4uV4tmEkT5sEeIyELpqgNSYz0Q0b/atwc++URulNqjh9LRkBliIkRkoaZN0z729ZX/Va8PRGSTMjOBDz8Exox53PPzwQfKxkRmjYkQkYW6f//xzxwCIwJw4QIQGgokJAAHDsiHHStAqHj8DSGyQLGxwPXr8mdfXyZBRFi3DmjWTCZBVavKHeSZBJEe+FtCZIHyD4tVqKBcHESKe/gQeOstoH9/4MEDoF074MQJuW0GkR6YCBFZkIKrRQOsByIbdu0a0KIFsGwZoFLJ2qBdux4XzBHpgTVCRBak4DpBAQEcFiMbVrUq4OAAeHoC0dFAp05KR0QWiIkQkYWIjX2cBNnZySny7A0im5ORATg7A/b2gIuLXCHaxQXw9lY6MrJQHBojsgAF1wxSrxPE3iCyKWfOyJWhZ8583FanDpMgKhUmQkRmTtfCiewJIpsiBLBihUyCzp4FIiO1148gKgUmQkRmqqgNVLlmENmUBw+AgQOBYcPkDLEuXYDjxzldkoyGiRCRmeIGqmTzfv8dCAqShdB2dnKrjO3bZXE0kZGwWJrIDBVVGM0kiGzGgwdAx47AnTtyOvy6dUDbtkpHRVaIiRCRGYmNLdwTxA1UySaVLw98/rmcFbZqlZwqT2QCTISIzERRu8mzMJpsxvHjQE4O0Ly5PB46VD5UKmXjIqvGGiEiBakLomvUKJwEBQSwJohshBDAwoVAq1byF/7OHdmuUjEJIpNjjxCRgnQVRANMgMiG3LsnZ4Rt3iyPmzVj8kNlij1CRAopWBDt68teILIxR44ATZvKJMjREZg7F9iyBahUSenIyIawR4hIAUWtFE1kE4SQSc/77wPZ2XJ16JgYuWAiURljjxBRGeNK0UQA9u6VSdCrr8oiaSZBpBD2CBGVsWnTtI85FEY2Q4jHBdArVgAvvQS8+SZrgkhR7BEiKiPqGWLnz2u3MQkiq5eXB8yeDQweLJMhQNYBDRvGJIgUxx4hojKgazgsIIBJENmAf/6RCdD27fJ44ECgc2dlYyLKhz1CRCZU1MapAQGsCyIbsG8f0KSJTIJcXICvvgI6dVI6KiIt7BEiMiFunEo2KS8PmDVL/gHIywOefhrYsAFo3FjpyIgKYY8QkQkUrAeys+MaQWRDhg4FPvxQJkEDBwLHjjEJIrPFHiEiE+DGqWTThg6ViyQuWAAMGaJ0NETFYo8QkREV1RPEeiCyarm5wMmTj49feAG4coVJEFkE9ggRGYmumWHsCSKrl5wM9O8vh7/i4+UvPQBUrqxsXER6Yo8QkZEUXCiRPUFk9XbuBJ59Fti9W64PlH+RLCILwR4hIiPIv4Gq+phF0WS1cnKAiAg5M0wIWQgdEyOzfyILw0SIyAjy9wZxoUSyateuyaGwffvk8ciRwP/+B7i6KhsXUQkxESIqpYK9QRwOI6u2bJlMgipUAL7+GujbV+mIiEqFiRBRKbE3iGzKhx/KAulJk4B69ZSOhqjUWCxNVEr37z/+mb1BZHWSkoAxY4DsbHns6Ch7gpgEkZVgjxCRkfj6sjeIrMzWrXItoLt35W7xH3+sdERERsceISIi0paVBYSFAS+/LJOg554Dhg1TOioik2AiRFQC6hWka9SQ5RJEViMxEXj+eWDuXHkcFgbs3w/UqaNoWESmUqKhsZycHOzevRsXL15E//79UaFCBdy4cQPu7u4oX768sWMkMhuxsbp3lAfkJBoiixYXB/TpA6SmyqGwqCigZ0+loyIyKYMToStXrqBr165ISkrCo0eP0LlzZ1SoUAGzZ89GZmYmli5daoo4iRSjTn7u3weuXy/8vK+vTIJYKE0Wr3ZtuW9Yq1bA+vVArVpKR0RkcgYPjY0fPx7BwcG4e/cuXPMtoNW7d2/s2rXLqMERmQN1D1DBJCggQCZJ167J/cRYKE0WKTX18c/16wN79sgHkyCyEQYnQvv378eHH34IJycnrXY/Pz9c1/XPZSILln+xRDs72fujToCY/JDFW79e9gL9+uvjtmbN5BR5Ihth8NBYXl4ecnNzC7Vfu3YNFVgkQVYm/2KJ3EmerMbDh8CECXI9IABYsgTo0EHRkIiUYnCPUOfOnTFXPZsAgEqlwoMHDxAREYHu3bsbMzYixahnheXfTJs1QGQV/vwTaNlSJkEqFTB1KrB2rdJRESlGJYQQhlxw48YNdOjQAfb29rhw4QKCg4Nx4cIFVK1aFXv37oWnp6epYjWKtLQ0eHh4IDU1Fe7u7kqHQ2YoNhZ4/XXttoAA9gaRFfjmG2DUKCA9HahWDYiOBjp3VjoqIr2Y6vvb4KExHx8fnDhxAuvXr0d8fDzy8vIwbNgwDBgwQKt4msgSFZUEsTeILN6ePcDAgfLnDh1kEuTtrWxMRGbA4B6hvXv3onXr1nBw0M6hcnJycPDgQbRr186oARobe4SoOIGB2msExcayIJqshBDA4MFA3brARx8B9vZKR0RkEFN9fxucCNnb2yM5ObnQENjt27fh6emps5DanDARouLUqPF4mjyTILJoQgDr1gFduwKVKz9uU6mUjYuohEz1/W1wsbQQAiodf5Bu374NNzc3owRFpITY2MdJEDdQJYv24IHs/RkwAHjzTZkAAUyCiHTQu0bolVdeASBniQ0ZMgTOzs6a53Jzc3Hy5Em0bt3a+BESlZH8U+W5EgRZrJMnZaHbn3/Kxa+aN2dPEFEx9E6EPDw8AMgeoQoVKmgVRjs5OaFly5YYMWKE8SMkKgP5F04EWBxNFkgIYNkyYPx4IDNTdmuuWwe0bat0ZERmTe9EaOXKlQCA2rVr47333uMwGFmV/L1BAQEcFiMLk5YGjBwpV4oGgG7dgNWrgapVlY2LyAIYXCMUERHBJIisBhdOJKuQkwMcPChngs2eDfzwA5MgIj0ZvI4QAGzcuBEbNmxAUlISsrKytJ47fvy4UQIjMrWi1gxibxBZhPwF0JUry19o9c7xRKQ3g3uE5s+fj6FDh8LT0xMJCQlo3rw5qlSpgkuXLqFbt26miJHIqNS9QFw4kSzWvXtAnz5AZOTjtubNmQQRlYDB6wgFBAQgIiIC/fr1Q4UKFfD777+jbt26mDZtGu7cuYOFCxeaKlaj4DpCtk1XL5C6nT1BZBGOHgVCQ4HERMDdHbhyBahYUemoiEzObNYRSkpK0kyTd3V1xf379wEAAwcOxLp164wWGJEp5C+KBmQvEJMgsghCAHPnAm3ayCSodm0gLo5JEFEpGZwIVa9eHbdv3wYA+Pn54fDhwwCAxMREGNi5RFSmCk6Rj42VG6kyCSKzd+cO0KsXEBYGZGcDr7wCJCTI4TAiKhWDE6GOHTvi+++/BwAMGzYMYWFh6Ny5M0JDQ9G7d2+DA1i8eDHq1KkDFxcXBAUFYd++fcWe/+jRI0ydOhV+fn5wdnbGU089hRUrVhj8vmRbCg6JsSiaLEZGBhAcDGzdCjg5AQsXAhs3sieIyEgMnjX29ddfIy8vDwAwatQoVK5cGfv370ePHj0watQog14rJiYGEyZMwOLFi9GmTRt89dVX6NatG86ePYtatWrpvOb111/H33//jcjISNSrVw83b95ETk6OoR+DbERsrBwOy98TBLAomixIuXLAoEHAN98AGzYAzZopHRGRVTG4WLo4169fh6+vr97nt2jRAs2aNcOSJUs0bYGBgejVqxdmzZpV6PwdO3agb9++uHTpEiqrNxE0EIulbUvB3eQB1gSRBbh1S+4XVru2PM7NlT1D3PuFbJjZFEvrkpKSgnfeeQf16tXT+5qsrCzEx8ejS5cuWu1dunTBwYMHdV6zdetWBAcHY/bs2fD19YW/vz/ee+89PHz4sMj3efToEdLS0rQeZP0KLpRoZ8fCaLIQ+/YBTZoAvXvLrTIAuVAikyAik9A7Ebp37x4GDBiAatWqwcfHB/Pnz0deXh6mTZuGunXr4vDhwwbV6ty6dQu5ubnw8vLSavfy8kJKSorOay5duoT9+/fj9OnT2LJlC+bOnYuNGzdizJgxRb7PrFmz4OHhoXnUrFlT7xjJcqmHw/4dxYW/Pwujyczl5QGffgp06ABcvy57gIr4u5CIjEfvGqEPPvgAe/fuxeDBg7Fjxw6EhYVhx44dyMzMxPbt29G+ffsSBaAqsCOyEKJQm1peXh5UKhWio6M1m8DOmTMHr732GhYtWqS1EazalClTEB4erjlOS0tjMmQD/l3VAXZ2MgliTRCZtZs3gTfekNPhAfnzkiVA+fLKxkVkA/ROhH788UesXLkSnTp1wujRo1GvXj34+/tj7ty5JXrjqlWrwt7evlDvz82bNwv1Eql5e3vD19dXkwQBsqZICIFr166hfv36ha5xdnaGs7NziWIky6IujL5/H0hOlm3e3rIniMhs/for0L+/7P1xdQUWLQKGDJFbZxCRyek9NHbjxg00aNAAAFC3bl24uLhg+PDhJX5jJycnBAUFIU79L6B/xcXFaRZsLKhNmza4ceMGHjx4oGk7f/487OzsUKNGjRLHQpZPPT3+jz/kqIJ6SIxlFWTWhAA++kgmQQ0ayFWjhw5lEkRUhvROhPLy8uDo6Kg5tre3L/Uu9OHh4Vi+fDlWrFiBc+fOISwsDElJSZpp+FOmTMGgQYM05/fv3x9VqlTB0KFDcfbsWezduxcTJ07Em2++qXNYjGxHwRWjfX25dxhZAJUKiI4Gxo4FjhwBGjZUOiIim6P30JgQAkOGDNEMM2VmZmLUqFGFkqHNmzfr/eahoaG4ffs2Zs6cieTkZDRq1Ajbtm2Dn58fACA5ORlJSUma88uXL4+4uDi88847CA4ORpUqVfD666/j448/1vs9yTqpa4IAzgwjM/fzz7LnZ8oUeeznByxYoGxMRDZM73WEhg4dqtcLrly5slQBmRrXEbI++VeN9vUFrl1TNh4inXJygOnT5cwwIWRC9J//KB0VkcUw1fe33j1C5p7gkO3KPyzGmiAyS9evy4LovXvl8ciRQBG1kERUtoyyoCKREgoumgiwJojM0PbtcoHEvXtlpr5uHbB0qZwhRkSKYyJEFkedAKlnialniHEjVTI7M2cC3bvLLTOaNgXi44G+fZWOiojyMXjTVSIlFdxFXo0zxMgsqbcdGjMG+OILwMVF2XiIqBAmQmRRCk6TVydA7Akis3HvHlCxovy5f3+5tHlwsJIREVExODRGFkFXPVBsLPcPIzOSlQWEh8u1gG7efNzOJIjIrJUoEVqzZg3atGkDHx8fXLlyBQAwd+5cfPfdd0YNjoj1QGQREhOBtm2B//0PuHED2LpV6YiISE8GJ0JLlixBeHg4unfvjnv37iE3NxcAULFixRLvO0akS/5tM/JjPRCZlc2bZSH0kSNApUrAd98Bpdh+iIjKlsGJ0IIFC7Bs2TJMnToV9vb2mvbg4GCcOnXKqMGR7dJVFB0QwOEwMiOPHgHvvAO8+iqQmgq0bAkkJAA9eyodGREZwOBEKDExEU2bNi3U7uzsjPT0dKMERbZJPQxWo0bhJIgJEJmdjz8GFi6UP0+cKNcJ+nd7ICKyHAYnQnXq1MGJEycKtW/fvl2zOz1RSUyb9nj3+Py4dxiZpYkTZS/QDz8As2cD+TalJiLLYfD0+YkTJ2LMmDHIzMyEEAJHjhzBunXrMGvWLCxfvtwUMZINiI19XAtkZwd4e8tFeDk1nsxGZiawahXw1lty13h3d+DgQfkzEVksgxOhoUOHIicnB5MmTUJGRgb69+8PX19fzJs3D325YiqVUP71gfz95TAYkdn48085XnvypKwNGjdOtjMJIrJ4JZo+P2LECFy5cgU3b95ESkoKrl69imHDhhk7NrIB3C+MzF50NBAUJJMgT0/5C0tEVsPgRGjGjBm4ePEiAKBq1arw9PQ0elBkG/JPj+f6QGR2MjLkNPg33gDS04EOHYATJ4DOnZWOjIiMyOBEaNOmTfD390fLli2xcOFC/PPPP6aIi6xcUdPj2RtEZuHsWaB5cyAyUg5/RUQAcXGyeI2IrIrBidDJkydx8uRJdOzYEXPmzIGvry+6d++OtWvXIiMjwxQxkhUquGcYp8eTWbl3T3ZVVq8O/PwzMH06kG/dNCKyHiohhCjNCxw4cABr165FbGwsMjMzkZaWZqzYTCItLQ0eHh5ITU2Fu7u70uHYrBo1Hk+T5/R4MgtCaBc/x8YC7doBXl7KxUREGqb6/i71pqtubm5wdXWFk5MTsrOzjRET2RBfXyZBZAZOnZKbo54+/bitTx8mQUQ2oESJUGJiIj755BM0aNAAwcHBOH78OKZPn46UlBRjx0dWKDa28KKJRIoQAli2TNYDHT8OhIUpHRERlTGD1xFq1aoVjhw5gmeeeQZDhw7VrCNEpK/89UEVKigXB9m4tDRg5Ehg/Xp53K0bsHq1sjERUZkzOBHq0KEDli9fjoYNG5oiHrJy+VeQBjhLjBSSkCCnLf71lyyC/vRT4L335LLmRGRTSl0sbWlYLK2cglPmAwK4gjQp4MgRoG1bICsLqFlT9gi1bq10VET0BKb6/tarRyg8PBz//e9/4ebmhvDw8GLPnTNnjlECI+sRG/t4Q9X82BtEiggKAlq1knuFRUUBlSsrHRERKUivRCghIUEzIywhIcGkAZH10ZUEcco8lanffweefhpwcZFDYVu3ygI17hVGZPP0SoR+/fVXnT8T6eP+fflfOzu5oSp3lKcyIwQwfz4wcaLcNX7hQtnOYXEi+pfBlYFvvvkm7qu/2fJJT0/Hm2++aZSgyDp5e3P1aCpDd+8Cr7wCTJgAZGcDKSlATo7SURGRmTE4EVq1ahUePnxYqP3hw4dYzamnVADXDCJFHD4MNG0KfPst4OQke4JiYwEHgyfKEpGV0/tvhbS0NAghIITA/fv34eLionkuNzcX27Zt4070pKXgLDGuGUQml5cHzJkDTJkie3+eegrYsAFo1kzpyIjITOmdCFWsWBEqlQoqlQr+/v6FnlepVJgxY4ZRgyPLpWt3ec4SI5NLSQE++UQmQaGhwNdfsx6IiIqldyL066+/QgiBjh07YtOmTaicb8qpk5MT/Pz84OPjY5IgyfLo2l2etUFkcj4+ckp8SoosjuasMCJ6Ar0Tofbt2wOQ+4zVqlULKv4FQ8XIX0/PJIhMJi8P+OwzoEkTuUUGALz8sqIhEZFl0SsROnnyJBo1agQ7Ozukpqbi1KlTRZ7buHFjowVHlil/gTR3lyeTuXkTGDgQ2LkTqFIF+PNP+V8iIgPolQg1adIEKSkp8PT0RJMmTaBSqaBrZw6VSoXc3FyjB0mWhZuqksnt3g307w8kJwOursDs2VwhmohKRK9EKDExEdWqVdP8TFSc/MNiLJAmo8rNlcXQM2bIYbEGDeSsMG4CTUQlpFci5Ofnp/NnovzUe4olJ8tjDouRUWVmAi++CPzyizweOhRYsABwc1M2LiKyaCVaUPHHH3/UHE+aNAkVK1ZE69atceXKFaMGR5ZDPV3+jz/kP9QBDouRkbm4ALVry8Rn9WpgxQomQURUagYnQp9++ilcXV0BAIcOHcLChQsxe/ZsVK1aFWFhYUYPkMyfrjWDAgI4LEZGkJMDpKY+Pl6wADh+XBZJExEZgcHrzV+9ehX16tUDAHz77bd47bXX8NZbb6FNmzZ44YUXjB0fWQCuGUQmcf26LIh2dQW2bZO79pYrJ3fuJSIyEoN7hMqXL4/bt28DAHbu3IlOnToBAFxcXHTuQUbWLTZWDoflP2YSRKW2Y4dcG2jvXuDAAblbLxGRCRjcI9S5c2cMHz4cTZs2xfnz5/Hiiy8CAM6cOYPatWsbOz4yc/l7gwICmARRKWVnAx99JBdJBOTGqTExQP36ysZFRFbL4B6hRYsWoVWrVvjnn3+wadMmVPl3AbP4+Hj069fP6AGSeYqNBQIDgfPnH7exJohK5epV4IUXHidBY8YABw8yCSIik1IJXSsjWrG0tDR4eHggNTUV7tyMsUSKKo7m6AWVmBBAq1bAb7/JTVIjI9m9SERaTPX9bfDQGADcu3cPkZGROHfuHFQqFQIDAzFs2DB4eHgYLTAyT5whRiahUgFLlgDjxgGrVgF16yodERHZCIN7hI4dO4aQkBC4urqiefPmEELg2LFjePjwIXbu3IlmzZqZKlajYI9Q6QQGsjiajOTyZeDYMe1fICG4YzwR6WSq72+DE6G2bduiXr16WLZsGRwcZIdSTk4Ohg8fjkuXLmHv3r1GC84UmAiVXMHeICZBVGJbtgBvvglkZMg6oKAgpSMiIjNnNkNjx44d00qCAMDBwQGTJk1CcHCw0QIj81IwCeIMMSqRR4+AiRPlwogA0LIlULWqsjERkU0zeNaYu7s7kpKSCrVfvXoVFbingtUquGgia4LIYBcvAm3aPE6CJk6U6wRx/0IiUpDBiVBoaCiGDRuGmJgYXL16FdeuXcP69esxfPhwTp+3Yvl3lOeQGBksNhZo1gyIjweqVAF++AGYPRtwdFQ6MiKycQYPjX3xxRdQqVQYNGgQcnJyAACOjo54++238X//939GD5DMC3eUpxK5eBFISwOefx5Ytw6oUUPpiIiIAJRiHaGMjAxcvHgRQgjUq1cP5cqVM3ZsJsFiacPlrw/y9QWuXVM2HrIQ+WeA5eXJafEDBwIOJVq1g4hsnKm+v/UeGsvIyMCYMWPg6+sLT09PDB8+HN7e3mjcuLHFJEFUMvnrg1gGRnqJjpYLJKany2M7O2DoUCZBRGR29E6EIiIiEBUVhRdffBF9+/ZFXFwc3n77bVPGRmag4KaqLJKmYmVkAMOHA2+8IVeJXrxY6YiIiIql9z/PNm/ejMjISPTt2xcA8MYbb6BNmzbIzc2Fvb29yQIk5XDKPBnk3Dn5C3P6tBwSmzYNCA9XOioiomLp3SN09epVtG3bVnPcvHlzODg44MaNGyYJjJTHKfOkt1WrgOBgmQRVrw78/DMwfTrAfyQRkZnTOxHKzc2Fk5OTVpuDg4Nm5hhZH06ZJ7188QUwZIgcFuvUCThxAujYUemoiIj0ovfQmBACQ4YMgbOzs6YtMzMTo0aNgpubm6Zt8+bNxo2QFMcp81Ssfv2AL78Exo4FJk9mLxARWRS9E6HBgwcXanvjjTeMGgyZh9hYOSyWnKx0JGSWhAAOH5azwgCZKZ8/zymFRGSR9E6EVq5caco4yIxMm6Y9U4zfb6Rx/z4wahSwdi2waRPwyiuynb8kRGShuKgHack/Xd7ODvD3Z5E0/evECTkr7MIFOfx1/brSERERlRoTIdIoOF3e31/OiCYbJwSwdCkQFiZ3j69ZE1i/HmjdWunIiIhKjYkQASicBAHsCSIAqanAiBHyFwQAevQAVq6UG6cSEVkBg3efJ+tUcM0gTpcnAMDevfKXwcEBmDMH+O47JkFEZFUUT4QWL16MOnXqwMXFBUFBQdi3b59e1x04cAAODg5o0qSJaQO0crGxQGCgnPSTv41JEAGQPUAffwwcOCCHxtSbqBIRWYkSJUJr1qxBmzZt4OPjgytXrgAA5s6di++++86g14mJicGECRMwdepUJCQkoG3btujWrRuSkpKKvS41NRWDBg3Cf/7zn5KET/moZ4jl5cljbqNh4+7eBYYN0y6EnjoVaN5cuZiIiEzI4ERoyZIlCA8PR/fu3XHv3j3k5uYCACpWrIi5c+ca9Fpz5szBsGHDMHz4cAQGBmLu3LmoWbMmlixZUux1I0eORP/+/dFKvY4JlUjBGWIBAawLsmm//QY0bQqsWCF3iicisgEGJ0ILFizAsmXLMHXqVK3NVoODg3Hq1Cm9XycrKwvx8fHo0qWLVnuXLl1w8ODBIq9buXIlLl68iIiICL3e59GjR0hLS9N6UNEzxNgbZIOEkCtDP/88cOUK8NRTwKxZSkdFRFQmDE6EEhMT0bRp00Ltzs7OSE9P1/t1bt26hdzcXHh5eWm1e3l5ISUlRec1Fy5cwOTJkxEdHQ0HB/0mvM2aNQseHh6aR82aNfWO0Vpxhhhp3L4N9OwJvPcekJMjfzHi44GgIKUjIyIqEwYnQnXq1MGJEycKtW/fvh0NGjQwOABVgeJLIUShNkBu+tq/f3/MmDED/v7+er/+lClTkJqaqnlcvXrV4BitDWeIEQDZBdikCfDDD4Czs1wraP16wMND6ciIiMqMwesITZw4EWPGjEFmZiaEEDhy5AjWrVuHWbNmYfny5Xq/TtWqVWFvb1+o9+fmzZuFeokA4P79+zh27BgSEhIwduxYAEBeXh6EEHBwcMDOnTvRUceO187Ozlobxdq6/HVB6mMmQTaqVi3A3V2Oi27YADz7rNIRERGVOYMToaFDhyInJweTJk1CRkYG+vfvD19fX8ybNw99+/bV+3WcnJwQFBSEuLg49O7dW9MeFxeHl19+udD57u7uhWqQFi9ejF9++QUbN25EnTp1DP0oNqfgkBhniNmgO3eAihVldbybm+wNqlqVe4URkc0q0crSI0aMwIgRI3Dr1i3k5eXB09OzRG8eHh6OgQMHIjg4GK1atcLXX3+NpKQkjBo1CoAc1rp+/TpWr14NOzs7NGrUSOt6T09PuLi4FGonberd5PP3BAGsC7I5e/YA/foBEyYAkybJNv4DgohsXKm22KhatWqp3jw0NBS3b9/GzJkzkZycjEaNGmHbtm3w8/MDACQnJz9xTSF6Ml1JEIfEbEhuLvDpp8D06XLBqOhouTiio6PSkRERKU4lhBCGXFCnTh2dxcxqly5dKnVQppSWlgYPDw+kpqbC3d1d6XBMSt0TdP68/P7Lv5s8kyAbkZICvPEGsGuXPB4yBFi4UA6LERFZEFN9fxvcIzRhwgSt4+zsbCQkJGDHjh2YOHGiseKiUtI1RZ67yduYXbuAAQOAv/8GypUDliwBBg1SOioiIrNicCI0fvx4ne2LFi3CsWPHSh0QGUfBKfJcNdrG/P038NJLQGYm0KiRzIwDApSOiojI7Bht09Vu3bph06ZNxno5KqX79x//HBvLVaNtjpcXMHs2MGIEcOQIkyAioiKUqlg6v40bN6Jy5crGejkqAXVN0P37QHKybPP1ZQJkM376CfD0lPuFAcDYsdwtnojoCQxOhJo2bapVLC2EQEpKCv755x8sXrzYqMGR/nTVBAFcHsYm5OQAH30E/N//yX3Cjh+XCyUyCSIieiKDE6FevXppHdvZ2aFatWp44YUXEMDud8UUrAny9ZVJEOuCrNzVq3JtoAMH5HFICODkpGxMREQWxKBEKCcnB7Vr10ZISAiqV69uqpjIQNw2w0b9+KOcBXbnjuwBWr4c6NNH6aiIiCyKQcXSDg4OePvtt/Ho0SNTxUMlkL83iNtm2ICcHGDiRDkr7M4dIDgYSEhgEkREVAIGzxpr0aIFEhISTBELlVD+GWIcCrMBdnaAet+98eOB/fuBunWVjYmIyEIZXCM0evRovPvuu7h27RqCgoLgVmCF2saNGxstODIMZ4hZOfXy4HZ2wOrVwG+/AT16KB0VEZFF03uLjTfffBNz585FxYoVC7+ISgUhBFQqFXJzc40do1FZ2xYb+WeL+foC164pGw+ZwKNHcigsI0PWARER2SBTfX/rnQjZ29sjOTkZDx8+LPY89Yap5sraEqHAwMeF0gEB3ELD6ly8CISGAvHx8vj48cfrBBER2RDF9xpT50vmnujYkoKzxVgfZGViY4Hhw4G0NKByZTkcxiSIiMioDCqWLm7XeSp7nC1mpTIzgdGj5ZhnWhrQpg1w4gTw4otKR0ZEZHUMKpb29/d/YjJ0586dUgVE+uNsMSvVsycQFyd/njIFmDkTcDDabjhERJSPQX+7zpgxAx4eHqaKhQwQGwtcvy5/5mwxKxMWBvz+uxwKCwlROhoiIqtmUCLUt29feHp6mioWMkD+YTHuJ2bhMjKAs2flwogA0K0bcOkSUGBpCiIiMj69a4RYH2Q+WCRtRc6dA1q0ADp3Bi5fftzOJIiIqEzonQjpOcueygCLpK3EqlWyF+j0acDZGUhOVjoiIiKbo/fQWF5eninjIAOwSNrCpacDY8bIRAgA/vMf4JtvAG5kTERU5gzea4zMB4ukLdDp08Bzz8kkyM5OZrI//cQkiIhIIZyTa2HyzxYjC7R8uawL8vEB1q4F2rdXOiIiIpvGRMjCcLaYhfu//5P/nToVqFZN2ViIiIhDY5aEs8Us0IkTwLBhgHozYhcXYO5cJkFERGaCiZAF4WwxCyIEsGQJ0LIlsGIF8OWXSkdEREQ6cGjMQrA3yIKkpgJvvQVs2CCPX3pJ9goREZHZYY+QhWBvkIWIjweaNZNJkIOD7AnauhWoUkXpyIiISAf2CFkIrh1kAdauBYYOBbKyAD8/ICZGrhpNRERmiz1CFoAbrFqIxo0Be3ugd28gIYFJEBGRBWCPkAXglHkzdvMmoN6IuFEj4NgxIDAQ4N58REQWgT1CFoDDYmYoL0/W/9SuDRw69Li9QQMmQUREFoSJkJnjsJgZun0b6NkTeO894OFDWQtEREQWiUNjZo7DYmbmwAGgb1/g2jW5Y/zcucDIkUpHRUREJcQeITPHYTEzkZcnt8do314mQfXrA4cPA6NGcSiMiMiCMRGyEBwWU9i33wJTpsitMvr3l+sFNWmidFRERFRKHBozY9xp3oz07i0ToA4d5CrR7AUiIrIKTITMGOuDFJSbCyxaBAwZAri7y8QnOlrpqIiIyMg4NGbGWB+kkJQUICQEGD9eFkILoXRERERkIuwRsgCsDypDu3YBAwYAf/8NlCsHdO3KYTAiIivGHiEzFBsrFydOTlY6EhuSmwtERACdO8skqFEj4OhRYPBgpSMjIiITYo+QmYmNBV5/XbuN9UEmlpIi1wbas0ceDx8OzJsne4SIiMiqsUfITKh7gQomQQEBrA8yOTs74Px5oHx5WRC9bBmTICIiG8EeITMxbRrwxx/abbGxrA0ymbw8mQABctPUTZuAKlUAf39l4yIiojLFHiEzoZ4hZmcne4GYBJnQ1atAu3bA2rWP21q1YhJERGSD2CNkBvIvnOjtDZw7p2w8Vu2HH2QB9J07wOXLwKuvyj3DiIjIJrFHyAxw4cQykJUld4vv0UMmQUFBsjiaSRARkU1jj5AZ4MKJJnb5spwV9ttv8njcOGD2bCZBRETEHiElFVwviAsnmsDt27L357ffgIoVgc2b5dR4JkFERAT2CCmq4EwxDouZQJUqcpPUPXuAmBigdm2lIyIiIjPCREghsbGPkyA7OzlhicNiRnLpEuDgANSqJY8/+UTuF+bkpGxcRERkdpgIKSR/gbS/P2eKGc3GjbIHqEEDYO9ewNFRPoiIiHRgjZBCWCBtZJmZwOjRQJ8+QFqa7GZLTVU6KiIiMnNMhBTGAmkjuHBBLoi4ZIk8njwZ2L0bqFpV0bCIiMj8cWhMAfkXUKRSWrcOeOst4MEDmfisWQN07ap0VEREZCGYCCmACygaSU6OXA/owYPHW2b4+iodFRERWRAmQgpgfZCRODgAGzbIBGjqVHlMRERkANYIlbH8w2KsDyqB1auBzz57fFy/PhARwSSIiIhKhN8eZSg2Fnj99cfHHBYzQHo6MHYsEBUFqFRAx47Ac88pHRUREVk4JkJlKH9tEMBhMb2dPi0zyHPn5LT46dOBZs2UjoqIiKwAE6EylL82KDaWw2JPJASwYgXwzjvAw4eAt7esB3rhBaUjIyIiK8FEqIywNqgERo4Eli2TP4eEyPogT09lYyIiIqvCYmkTU+8wz9qgEmjeHLC3B2bNArZtYxJERERGxx4hEyu4wzzA2qAiCQHcvAl4ecnjYcOA558HAgKUjYuIiKyW4j1CixcvRp06deDi4oKgoCDs27evyHM3b96Mzp07o1q1anB3d0erVq3w008/lWG0hlPXBdnZye9z1gYVIS0N6NtX9gLdvSvbVComQUREZFKKJkIxMTGYMGECpk6dioSEBLRt2xbdunVDUlKSzvP37t2Lzp07Y9u2bYiPj0eHDh3Qo0cPJCQklHHkT6YeEktOlsfe3nLSE5MgHeLj5SywDRuAGzeAYpJhIiIiY1IJIYRSb96iRQs0a9YMS9SbZQIIDAxEr169MGvWLL1eo2HDhggNDcW0gnPTi5CWlgYPDw+kpqbC3d29RHE/ScH1ggDZsXHunEneznIJASxcCLz3HpCVBfj5AevXAy1bKh0ZERGZGVN9fyvWI5SVlYX4+Hh06dJFq71Lly44ePCgXq+Rl5eH+/fvo3LlykWe8+jRI6SlpWk9TKmoJIh1QQXcvQu8+iowbpxMgnr1AhISmAQREVGZUiwRunXrFnJzc+GlLoz9l5eXF1JSUvR6jS+//BLp6el4vWDmkc+sWbPg4eGhedSsWbNUcT9JwY6p2FgOien0wQfAli2AoyMwbx6weTNQqZLSURERkY1RvFhapVJpHQshCrXpsm7dOkyfPh0xMTHwLGZa9ZQpU5Camqp5XL16tdQxF4eLJurpk0+ADh2Agwdlr5Ae95yIiMjYFJs+X7VqVdjb2xfq/bl582ahXqKCYmJiMGzYMMTGxqJTp07Fnuvs7AxnZ+dSx2soLppYwJ07ckHE8eNl0lO5MvDLL0pHRURENk6xHiEnJycEBQUhLi5Oqz0uLg6tW7cu8rp169ZhyJAhWLt2LV588UVTh2mQ/KtHUz4HDwJNmgBhYUBkpNLREBERaSi6oGJ4eDgGDhyI4OBgtGrVCl9//TWSkpIwatQoAHJY6/r161i9ejUAmQQNGjQI8+bNQ8uWLTW9Sa6urvDw8FDsc6jlrw/i6tEA8vKAzz8Hpk4FcnOB+vWB4GCloyIiItJQNBEKDQ3F7du3MXPmTCQnJ6NRo0bYtm0b/Pz8AADJyclaawp99dVXyMnJwZgxYzBmzBhN++DBgxEVFVXW4ReSvz7I5meJ/fMPMHgwsH27PO7XD/jqK2aIRERkVhRdR0gJplqHIP+0eV9f4No1o7205dm/HwgNlYsjurgACxbI7TJYEE1ERCVkqu9v7jVmJBwWyyc7Wy6pHRAgV4t+5hmlIyIiItKJiZCR2PywWG6u3CkekNPiN28GOnUCypdXNi4iIqJiKL6OkDXIP1vMJqfN79olN1a7cOFxW69eTIKIiMjsMREyApsdFsvNBSIigM6dZRIUEaF0RERERAbh0JgR2OSw2I0bwIABwO7d8njYMGD+fEVDIiIiMhQToVKyyWGxn34CBg6UU+Td3OS0+AEDlI6KiIjIYBwaK6HYWFkWk3+/V5sYFtu+HejaVSZBzz4LHD/OJIiIiCwWe4RKaNo04I8/tNtsYlisUyegZUu5ZcacOYCrq9IRERERlRgToRKIjX2cBNnZAf7+Mgmy2mGxPXuA1q0BR0f5+OUXJkBERGQVODRWAvlnifn7A+fOWWkSlJ0NTJoEvPAC8OGHj9uZBBERkZVgj1AJ2MQssStXgL59gcOH5fGjR4AQ3CaDiIisChOhUrDaWWLffgsMHQrcuwd4eAArVgCvvKJ0VEREREbHoTED5Z8ub3WysoAJE4DevWUS1Lw5kJDAJIiIiKwWEyEDWfUq0levAsuXy5/Dw4F9+4A6dZSNiYiIyIQ4NGaA/LPFACusD3rqKWDlSsDFBejRQ+loiIiITI49QgbI3xsUEGAF9UGZmcA77zzeJgMA+vRhEkRERDaDPUIGsKrZYhcuAKGhsgZo82bgr784LZ6IiGwOe4RKwOJni61fDzRrJpOgqlVlXRCTICIiskFMhPRkFbPFHj4ERo4E+vUDHjwA2rYFTpwAunVTOjIiIiJFcGhMTxY/W+zePaBdO+DUKbko4tSpQEQE4MBfASIisl38FtSDVcwW8/AAGjYE/v4b+OYboHNnpSMiIiJSnEoIIZQOoiylpaXBw8MDqampcHd31+uawMDHiVBAgNxbzCKkpwM5OTIJAoC0NNnm7a1sXERERAYqyfe3PlgjpAeLnC125oxcGXrIELlHGAC4uzMJIiIiyoeJkAEsYraYEHJvsOeeA86eBX77Dbh2TemoiIiIzBIToSewqNliDx4AAwcCw4bJGWJdushZYTVrKh0ZERGRWWIi9AQWM1vs99+BoCAgOhqwtwc+/RTYvh3w9FQ6MiIiIrPFWWPFsJjZYrm5wOuvA+fPy/G79euB559XOioiIiKzxx6hYljM3mL29nKz1JdflkNhTIKIiIj0wkSoGGY9W+z4cdllpda6NfDtt3LLDCIiItILE6Ei5C+SNqvZYkIACxcCrVoBgwfLafJERERUIqwRKoJZFknfuydnhG3eLI979uS6QERERKXAHqEimN2w2JEjQNOmMglydATmzpVDYZUrKx0ZERGRxWIiVEBsrNxSIzlZHpvFsNi8ebIA+vJloE4d4MABYPx4uXkqERERlRgToQKmTZNT5vPy5LFZDIvduQNkZwOvviqLpJ97TumIiIiIrAJrhApQD4nZ2QH+/goOi+XkAA7/3p5p04BnnpGJEHuBiEhhQgjk5OQgNzdX6VDIyjg6OsLe3r5M35OJUBG8vRXaZT4vD/jiC1kLtGcP4Ows1wlSfHyOiAjIyspCcnIyMjIylA6FrJBKpUKNGjVQvnz5MntPJkLm5J9/5JT47dvl8bp1cvd4IiIzkJeXh8TERNjb28PHxwdOTk5QsZeajEQIgX/++QfXrl1D/fr1y6xniIlQPopusLp3L9CvH3DjBuDiAsyfL5MiIiIzkZWVhby8PNSsWRPlypVTOhyyQtWqVcPly5eRnZ1dZokQi6X/FRsrt+tSK7Mi6bw84JNPgA4dZBL09NPAb78BI0awHoiIzJKdHb86yDSU6GHkb/O/8i+gCJRhkfSkScCHH8qEaOBA4NgxoHHjMnpzIiIi22bziZB63aDz57Xbyqw2eexYwMcHWLECWLUKKMMCMSIiIltn04mQejgs/7pBJt9lPjcX+Pnnx8e1awMXLwJDh3IojIhIAU8//TScnJxwXUeRaO3atTF37txC7XPnzkXt2rW12tLS0jB16lQEBATAxcUF1atXR6dOnbB582YIIUwUPXDq1Cm0b98erq6u8PX1xcyZM4t9v927d0OlUul8HD16VHOerueXLl2qef7y5cs6z9mxY4fW++3ZswdBQUFwcXFB3bp1tV7DHNh0sXTB4bCAABMPiSUnAwMGAL/+CmzbBnTrJttdXEz4pkREVJT9+/cjMzMTffr0QVRUFKZOnVqi17l37x6ef/55pKam4uOPP8Zzzz0HBwcH7NmzB5MmTULHjh1RsWJF4wYPmXx17twZHTp0wNGjR3H+/HkMGTIEbm5uePfdd3Ve07p1aySrt0/410cffYSff/4ZwcHBWu0rV65E165dNcceHh6FXu/nn39Gw4YNNceV8239lJiYiO7du2PEiBH45ptvcODAAYwePRrVqlXDq6++WqLPbGw2nQjl30/M5MNhcXHAG28AN28Cbm7ab05ERIqIjIxE//790b59e4wZMwYffPBBiQp2P/jgA1y+fBnnz5+Hj4+Ppt3f3x/9+vWDi4n+wRsdHY3MzExERUXB2dkZjRo1wvnz5zFnzhyEh4fr/CxOTk6oXr265jg7Oxtbt27F2LFjC51fsWJFrXN1qVKlSpHnLF26FLVq1dL0qgUGBuLYsWP44osvzCYRsumhMTWT7ieWkyOLoUNCZBLUuLEsiM4/RY2IiMrc/fv3ERsbizfeeAOdO3dGeno6du/ebfDr5OXlYf369RgwYIBWEqRWvnx5ODjo7nfYt28fypcvX+zj008/LfK9Dx06hPbt28PZ2VnTFhISghs3buDy5ct6xb9161bcunULQ3SsWzd27FhUrVoVzz33HJYuXYo8dR1JPj179oSnpyfatGmDjRs3FoqvS5cuWm0hISE4duwYsrOz9YrP1Gy6R8jkrl0D+vcH9u2TxyNHAv/7H+DqqmxcRERGFBwMpKSU/ftWry7/XVlS69evR/369TXDOn379kVkZCQ6dOhg0OvcunULd+/eRUBAgMExBAcH48SJE8Wek3+oqaCUlJRCtUpeXl6a5+rUqfPEGCIjIxESEoKaNWtqtf/3v//Ff/7zH7i6umLXrl149913cevWLXz44YcAZII3Z84ctGnTBnZ2dti6dStCQ0OxatUqvPHGG5oY1PHkjy8nJwe3bt2Ct7f3E+MzNSZCprRvn3xUqAAsWwaEhiodERGR0aWkKLgYbSlERkZqvrAB4I033kC7du1w7949g+p51IXJJRlSc3V1Rb169Qy+Lr+C72tIPNeuXcNPP/2EDRs2FHpOnfAAQJMmTQAAM2fO1LRXrVoVYWFhmnOCg4Nx9+5dzJ49W+v/a2niKwtMhEypXz/g8mWgTx+glL/oRETm6gklJGb5vmfPnsVvv/2Go0eP4v3339e05+bmYt26dXj77bcBAO7u7khNTS10/b179zSFw9WqVUOlSpVwrgQbVO7btw/d1BNnivDBBx/ggw8+0Plc9erVkVKgO+7mzZsAUKgnRpeVK1eiSpUq6Nmz5xPPbdmyJdLS0vD3338X+dotW7bE8uXLnxifg4MDqlSp8sT3LAtMhIwpKQkICwOWLgWqVZNtU6YoGxMRkYmVZnhKKZGRkWjXrh0WLVqk1b5mzRpERkZqEqGAgACtKeVqR48exdNPPw1ArrQdGhqKNWvWICIiolCdUHp6OpydnXXWCZV2aKxVq1b44IMPkJWVBScnJwDAzp074ePjU2jIrCAhBFauXIlBgwbB0dGx2HMBICEhAS4uLsX2liUkJGgNd7Vq1Qrff/+91jk7d+5EcHCwXu9ZJoSNSU1NFQBEamqq8PUVAhDC19cIL/zdd0JUqiRf8PXXjfCCRETm5eHDh+Ls2bPi4cOHSodSKllZWaJatWpiyZIlhZ47f/68ACBOnDghhBDi0KFDws7OTsyYMUOcOXNGnDlzRsycOVPY2dmJw4cPa667c+eOCAgIEDVq1BCrVq0SZ86cEefPnxeRkZGiXr164u7duyb5LPfu3RNeXl6iX79+4tSpU2Lz5s3C3d1dfPHFF5pzfvvtN/H000+La9euaV37888/CwDi7NmzhV5369at4uuvvxanTp0Sf/31l1i2bJlwd3cX48aN05wTFRUloqOjxdmzZ8Uff/whPv/8c+Ho6CjmzJmjOefSpUuiXLlyIiwsTJw9e1ZERkYKR0dHsXHjRp2fp7jfsfzf38bERKi0idCjR0JMmCBfCBDiueeEuHTJaPESEZkLa0mENm7cKOzs7ERKSorO55955hnxzjvvaI7j4uJE27ZtRaVKlUSlSpXE888/L+Li4gpdd+/ePTF58mRRv3594eTkJLy8vESnTp3Eli1bRF5ensk+z8mTJ0Xbtm2Fs7OzqF69upg+fbrW+/36668CgEhMTNS6rl+/fqJ169Y6X3P79u2iSZMmonz58qJcuXKiUaNGYu7cuSI7O1tzTlRUlAgMDBTlypUTFSpUEEFBQWLNmjWFXmv37t2iadOmwsnJSdSuXVtnAqqmRCKkEsKEy12aobS0NHh4eCA1NRUNGrjj+nU5ff7atRK8WGKiLIBWd5uGhQH/93/Av92TRETWJDMzE4mJiahTp47J1sUh21bc71j+7293d3ejvafN1ght2VLKWQ6HDsmVoVNTgUqVgKgoQI9iMyIiIjIfNpsIffLJ458rVCjBCzRsCFStCjRoAKxbB/j5GS02IiIiKhs2mwg9ePD4Z733F7t+Xe4Ur1IB7u7Arl3y2Fwq34mIiMggNr/Fht7ba8TEAIGBQP6pln5+TIKIiIgsmM0nQk/08KHcGqNvX7lR6nffyflhREREZPGYCBXnzz+Bli2Br7+Ww2FTpwLbt8ufiYhslI1NNqYypMTvls3WCD3RN98Ao0YB6emAp6c87txZ6aiIiBSjXgk4IyMDrtw8mkwgKysLAGBvb19m72mziVBycjFPXrgADBkC5OYCHToA0dGAGeyQS0SkJHt7e1SsWFGzl1W5cuXMZuNMsnx5eXn4559/UK5cOZ3bkZiKzSZCajqnztevD8yaBWRkAB9+CJRhZkpEZM6q/7vTqToZIjImOzs71KpVq0wTbJtPhP77X8ji51WrgOeek+sDAcDEiYrGRURkjlQqFby9veHp6Yns7GylwyEr4+TkBDu7si1fVjwRWrx4MT7//HMkJyejYcOGmDt3Ltq2bVvk+Xv27EF4eDjOnDkDHx8fTJo0CaNGjSrRe/v6Aq91fQAMHg2sWSMXRzx6FChXrqQfh4jIJtjb25dpHQeRqSg6aywmJgYTJkzA1KlTkZCQgLZt26Jbt25ISkrSeX5iYiK6d++Otm3bIiEhAR988AHGjRuHTZs2lej9A7NPAsHBMgmyswMGDAC4fw4REZHNUHTT1RYtWqBZs2ZYsmSJpi0wMBC9evXCrFmzCp3//vvvY+vWrTh37pymbdSoUfj9999x6NAhvd5TvWnbYMzFUrwPFzySXUPr1gHF9EQRERGRcky16apiPUJZWVmIj49Hly5dtNq7dOmCgwcP6rzm0KFDhc4PCQnBsWPHDB6rno8JMgnq1g04cYJJEBERkQ1SrEbo1q1byM3NhZeXl1a7l5cXUlJSdF6TkpKi8/ycnBzcunUL3jqmuD969AiPHj3SHKempgIA7kCF2RVmYNLad+SwWFpaaT8SERERmUjav9/Txh7IUrxYuuAUOSFEsdPmdJ2vq11t1qxZmDFjRqH2OhDA/Wn4pNI0Q0MmIiIihdy+fRseHh5Gez3FEqGqVavC3t6+UO/PzZs3C/X6qFWvXl3n+Q4ODqhSpYrOa6ZMmYLw8HDN8b179+Dn54ekpCSj/o+kkklLS0PNmjVx9epVo475kuF4L8wH74X54L0wH6mpqahVqxYqV65s1NdVLBFycnJCUFAQ4uLi0Lt3b017XFwcXn75ZZ3XtGrVCt9//71W286dOxEcHKxZ+r0gZ2dnODs7F2r38PDgL7UZcXd35/0wE7wX5oP3wnzwXpgPY68zpOj0+fDwcCxfvhwrVqzAuXPnEBYWhqSkJM26QFOmTMGgQYM0548aNQpXrlxBeHg4zp07hxUrViAyMhLvvfeeUh+BiIiILJiiNUKhoaG4ffs2Zs6cieTkZDRq1Ajbtm2Dn58fACA5OVlrTaE6depg27ZtCAsLw6JFi+Dj44P58+fj1VdfVeojEBERkQVTvFh69OjRGD16tM7noqKiCrW1b98ex48fL/H7OTs7IyIiQudwGZU93g/zwXthPngvzAfvhfkw1b1QdEFFIiIiIiUpWiNEREREpCQmQkRERGSzmAgRERGRzWIiRERERDbLKhOhxYsXo06dOnBxcUFQUBD27dtX7Pl79uxBUFAQXFxcULduXSxdurSMIrV+htyLzZs3o3PnzqhWrRrc3d3RqlUr/PTTT2UYrfUz9M+G2oEDB+Dg4IAmTZqYNkAbYui9ePToEaZOnQo/Pz84OzvjqaeewooVK8ooWutm6L2Ijo7Gs88+i3LlysHb2xtDhw7F7du3yyha67V371706NEDPj4+UKlU+Pbbb594jVG+v4WVWb9+vXB0dBTLli0TZ8+eFePHjxdubm7iypUrOs+/dOmSKFeunBg/frw4e/asWLZsmXB0dBQbN24s48itj6H3Yvz48eKzzz4TR44cEefPnxdTpkwRjo6O4vjx42UcuXUy9H6o3bt3T9StW1d06dJFPPvss2UTrJUryb3o2bOnaNGihYiLixOJiYnit99+EwcOHCjDqK2Tofdi3759ws7OTsybN09cunRJ7Nu3TzRs2FD06tWrjCO3Ptu2bRNTp04VmzZtEgDEli1bij3fWN/fVpcINW/eXIwaNUqrLSAgQEyePFnn+ZMmTRIBAQFabSNHjhQtW7Y0WYy2wtB7oUuDBg3EjBkzjB2aTSrp/QgNDRUffvihiIiIYCJkJIbei+3btwsPDw9x+/btsgjPphh6Lz7//HNRt25drbb58+eLGjVqmCxGW6RPImSs72+rGhrLyspCfHw8unTpotXepUsXHDx4UOc1hw4dKnR+SEgIjh07huzsbJPFau1Kci8KysvLw/37942+wZ4tKun9WLlyJS5evIiIiAhTh2gzSnIvtm7diuDgYMyePRu+vr7w9/fHe++9h4cPH5ZFyFarJPeidevWuHbtGrZt2wYhBP7++29s3LgRL774YlmETPkY6/tb8ZWljenWrVvIzc0ttHu9l5dXoV3r1VJSUnSen5OTg1u3bsHb29tk8VqzktyLgr788kukp6fj9ddfN0WINqUk9+PChQuYPHky9u3bBwcHq/qrQlEluReXLl3C/v374eLigi1btuDWrVsYPXo07ty5wzqhUijJvWjdujWio6MRGhqKzMxM5OTkoGfPnliwYEFZhEz5GOv726p6hNRUKpXWsRCiUNuTztfVToYz9F6orVu3DtOnT0dMTAw8PT1NFZ7N0fd+5Obmon///pgxYwb8/f3LKjybYsifjby8PKhUKkRHR6N58+bo3r075syZg6ioKPYKGYEh9+Ls2bMYN24cpk2bhvj4eOzYsQOJiYmazcKpbBnj+9uq/plXtWpV2NvbF8rkb968WShrVKtevbrO8x0cHFClShWTxWrtSnIv1GJiYjBs2DDExsaiU6dOpgzTZhh6P+7fv49jx44hISEBY8eOBSC/jIUQcHBwwM6dO9GxY8cyid3alOTPhre3N3x9feHh4aFpCwwMhBAC165dQ/369U0as7Uqyb2YNWsW2rRpg4kTJwIAGjduDDc3N7Rt2xYff/wxRxHKkLG+v62qR8jJyQlBQUGIi4vTao+Li0Pr1q11XtOqVatC5+/cuRPBwcFwdHQ0WazWriT3ApA9QUOGDMHatWs55m5Eht4Pd3d3nDp1CidOnNA8Ro0ahaeffhonTpxAixYtyip0q1OSPxtt2rTBjRs38ODBA03b+fPnYWdnhxo1apg0XmtWknuRkZEBOzvtr057e3sAj3sjqGwY7fvboNJqC6CeChkZGSnOnj0rJkyYINzc3MTly5eFEEJMnjxZDBw4UHO+evpdWFiYOHv2rIiMjOT0eSMx9F6sXbtWODg4iEWLFonk5GTN4969e0p9BKti6P0oiLPGjMfQe3H//n1Ro0YN8dprr4kzZ86IPXv2iPr164vhw4cr9RGshqH3YuXKlcLBwUEsXrxYXLx4Uezfv18EBweL5s2bK/URrMb9+/dFQkKCSEhIEADEnDlzREJCgmYpA1N9f1tdIiSEEIsWLRJ+fn7CyclJNGvWTOzZs0fz3ODBg0X79u21zt+9e7do2rSpcHJyErVr1xZLliwp44itlyH3on379gJAocfgwYPLPnArZeifjfyYCBmXoffi3LlzolOnTsLV1VXUqFFDhIeHi4yMjDKO2joZei/mz58vGjRoIFxdXYW3t7cYMGCAuHbtWhlHbX1+/fXXYr8DTPX9rRKCfXlERERkm6yqRoiIiIjIEEyEiIiIyGYxESIiIiKbxUSIiIiIbBYTISIiIrJZTISIiIjIZjERIiIiIpvFRIiItERFRaFixYpKh1FitWvXxty5c4s9Z/r06WjSpEmZxENE5o2JEJEVGjJkCFQqVaHHX3/9pXRoiIqK0orJ29sbr7/+OhITE43y+kePHsVbb72lOVapVPj222+1znnvvfewa9cuo7xfUQp+Ti8vL/To0QNnzpwx+HUsOTElMndMhIisVNeuXZGcnKz1qFOnjtJhAZCbuiYnJ+PGjRtYu3YtTpw4gZ49eyI3N7fUr12tWjWUK1eu2HPKly9v0O7UJZX/c/74449IT0/Hiy++iKysLJO/NxHph4kQkZVydnZG9erVtR729vaYM2cOnnnmGbi5uaFmzZoYPXq01q7mBf3+++/o0KEDKlSoAHd3dwQFBeHYsWOa5w8ePIh27drB1dUVNWvWxLhx45Cenl5sbCqVCtWrV4e3tzc6dOiAiIgInD59WtNjtWTJEjz11FNwcnLC008/jTVr1mhdP336dNSqVQvOzs7w8fHBuHHjNM/lHxqrXbs2AKB3795QqVSa4/xDYz/99BNcXFxw7949rfcYN24c2rdvb7TPGRwcjLCwMFy5cgV//vmn5pzi7sfu3bsxdOhQpKamanqWpk+fDgDIysrCpEmT4OvrCzc3N7Ro0QK7d+8uNh4iKoyJEJGNsbOzw/z583H69GmsWrUKv/zyCyZNmlTk+QMGDECNGjVw9OhRxMfHY/LkyXB0dAQAnDp1CiEhIXjllVdw8uRJxMTEYP/+/Rg7dqxBMbm6ugIAsrOzsWXLFowfPx7vvvsuTp8+jZEjR2Lo0KH49ddfAQAbN27E//73P3z11Ve4cOECvv32WzzzzDM6X/fo0aMAgJUrVyI5OVlznF+nTp1QsWJFbNq0SdOWm5uLDRs2YMCAAUb7nPfu3cPatWsBQPP/Dyj+frRu3Rpz587V9CwlJyfjvffeAwAMHToUBw4cwPr163Hy5En06dMHXbt2xYULF/SOiYgAq9x9nsjWDR48WNjb2ws3NzfN47XXXtN57oYNG0SVKlU0xytXrhQeHh6a4woVKoioqCid1w4cOFC89dZbWm379u0TdnZ24uHDhzqvKfj6V69eFS1bthQ1atQQjx49Eq1btxYjRozQuqZPnz6ie/fuQgghvvzyS+Hv7y+ysrJ0vr6fn5/43//+pzkGILZs2aJ1TkREhHj22Wc1x+PGjRMdO3bUHP/000/CyclJ3Llzp1SfE4Bwc3MT5cqV0+yk3bNnT53nqz3pfgghxF9//SVUKpW4fv26Vvt//vMfMWXKlGJfn4i0OSibhhGRqXTo0AFLlizRHLu5uQEAfv31V3z66ac4e/Ys0tLSkJOTg8zMTKSnp2vOyS88PBzDhw/HmjVr0KlTJ/Tp0wdPPfUUACA+Ph5//fUXoqOjNecLIZCXl4fExEQEBgbqjC01NRXly5eHEAIZGRlo1qwZNm/eDCcnJ5w7d06r2BkA2rRpg3nz5gEA+vTpg7lz56Ju3bro2rUrunfvjh49esDBoeR/nQ0YMACtWrXCjRs34OPjg+joaHTv3h2VKlUq1eesUKECjh8/jpycHOzZsweff/45li5dqnWOofcDAI4fPw4hBPz9/bXaHz16VCa1T0TWhIkQkZVyc3NDvXr1tNquXLmC7t27Y9SoUfjvf/+LypUrY//+/Rg2bBiys7N1vs706dPRv39//Pjjj9i+fTsiIiKwfv169O7dG3l5eRg5cqRWjY5arVq1ioxNnSDY2dnBy8ur0Be+SqXSOhZCaNpq1qyJP//8E3Fxcfj5558xevRofP7559izZ4/WkJMhmjdvjqeeegrr16/H22+/jS1btmDlypWa50v6Oe3s7DT3ICAgACkpKQgNDcXevXsBlOx+qOOxt7dHfHw87O3ttZ4rX768QZ+dyNYxESKyIceOHUNOTg6+/PJL2NnJEsENGzY88Tp/f3/4+/sjLCwM/fr1w8qVK9G7d280a9YMZ86cKZRwPUn+BKGgwMBA7N+/H4MGDdK0HTx4UKvXxdXVFT179kTPnj0xZswYBAQE4NSpU2jWrFmh13N0dNRrNlr//v0RHR2NGjVqwM7ODi+++KLmuZJ+zoLCwsIwZ84cbNmyBb1799brfjg5ORWKv2nTpsjNzcXNmzfRtm3bUsVEZOtYLE1kQ5566ink5ORgwYIFuHTpEtasWVNoqCa/hw8fYuzYsdi9ezeuXLmCAwcO4OjRo5qk5P3338ehQ4cwZswYnDhxAhcuXMDWrVvxzjvvlDjGiRMnIioqCkuXLsWFCxcwZ84cbN68WVMkHBUVhcjISJw+fVrzGVxdXeHn56fz9WrXro1du3YhJSUFd+/eLfJ9BwwYgOPHj+OTTz7Ba6+9BhcXF81zxvqc7u7uGD58OCIiIiCE0Ot+1K5dGw8ePMCuXbtw69YtZGRkwN/fHwMGDMCgQYOwefNmJCYm4ujRo/jss8+wbds2g2IisnlKFigRkWkMHjxYvPzyyzqfmzNnjvD29haurq4iJCRErF69WgAQd+/eFUJoF+c+evRI9O3bV9SsWVM4OTkJHx8fMXbsWK0C4SNHjojOnTuL8uXLCzc3N9G4cWPxySefFBmbruLfghYvXizq1q0rHB0dhb+/v1i9erXmuS1btogWLVoId3d34ebmJlq2bCl+/vlnzfMFi6W3bt0q6tWrJxwcHISfn58QonCxtNpzzz0nAIhffvml0HPG+pxXrlwRDg4OIiYmRgjx5PshhBCjRo0SVapUEQBERESEEEKIrKwsMW3aNFG7dm3h6OgoqlevLnr37i1OnjxZZExEVJhKCCGUTcWIiIiIlMGhMSIiIrJZTISIiIjIZjERIiIiIpvFRIiIiIhsFhMhIiIisllMhIiIiMhmMREiIiIim8VEiIiIiGwWEyEiIiKyWUyEiIiIyGYxESIiIiKbxUSIiIiIbNb/AyiWdaOYfNZSAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# calculate false positive rate and the true positive rate in order to construct a receiver operating curve (ROC)\n",
    "fpr, tpr, thresholds = roc_curve(y_train, y_proba, pos_label=1)\n",
    "\n",
    "plt.title('ROC (Train)')\n",
    "ax1 = plt.plot(fpr, tpr, 'b', label = '%s AUC = %0.4f' % ('', auc), linewidth=2)\n",
    "plt.legend(loc = 'lower right')\n",
    "plt.plot([0, 1], [0, 1],'r--')\n",
    "plt.xlim([0, 1])\n",
    "plt.ylim([0, 1])\n",
    "plt.ylabel('True Positive Rate')\n",
    "plt.xlabel('False Positive Rate')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bc9a4a79-b270-44de-b9da-733b6fe8ae34",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "### Precision-Recall Curve\n",
    "\n",
    "The curve plots the prcesion and recall at different decision thresholds. Recall or the true positive rate tells us how well the model correctly identifies all positive cases, whereas precision estimates the proportion of cases the model identified as positive that were truely positive."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 93,
   "id": "305cc2e0-125a-48f4-b62a-bfbd77c03144",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0.5, 0, 'Recall')"
      ]
     },
     "execution_count": 93,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAkIAAAHFCAYAAAAe+pb9AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8hTgPZAAAACXBIWXMAAA9hAAAPYQGoP6dpAABSYElEQVR4nO3deXhM1x8G8Heyb5LUkp2IrWhaJDSWEktFUUUXVGtJUUtLSWmpllIVtf1aJWlLUC2qtFRVkS72VC0J2igqIZZEJGRBZD2/P6aZuJlJZCaTubO8n+eZR86Zc2femTvJfN3lXIUQQoCIiIjIAlnJHYCIiIhILiyEiIiIyGKxECIiIiKLxUKIiIiILBYLISIiIrJYLISIiIjIYrEQIiIiIovFQoiIiIgsFgshIiIislgshMjirF27FgqFQnWzsbGBn58fwsPDcfXqVYPnGTlyJBo2bKjVMhcvXoRCocDatWtrJNODjBw5UvIe2tnZoXHjxpg6dSpycnJkyXQ/Te9P6Xq/ePFilR7j1KlTCA8PR0BAABwcHODi4oKgoCAsXLgQN2/erJngBnDhwgXY29sjLi4Oe/fulazHym7V1bVrV3Tt2lWnZd977z0EBQWhpKSk2jmIyrOROwCRXNasWYPmzZsjLy8P+/fvR2RkJPbt24fTp0/D2dnZYDnee+89vPHGG1ot4+3tjbi4ODRu3LiGUj2Yo6MjfvvtNwBAVlYWtmzZgiVLluDUqVPYs2ePbLn0YeXKlZgwYQIefvhhTJs2DS1btkRhYSGOHTuGzz77DHFxcdi6davcMXUydepU9OzZEx06dEBOTg7i4uIk9w8cOBCNGzfG4sWL9fq8UVFROi87depULF++HF9++SXCw8P1mIqIhRBZsMDAQLRt2xYA0K1bNxQXF+ODDz7Atm3b8NJLL2lc5u7du3ByctJrDl2KGXt7e7Rv316vObRlZWUlyfDUU08hKSkJsbGxSE5ORkBAgIzpdBcXF4fx48ejZ8+e2LZtG+zt7VX39ezZE2+++SZ27dqll+fKy8uDg4ODXra4VMWZM2ewbds2VX5XV1e1z5G9vT3c3d0r/XwJIXDv3j04OjpW+blbtmypW2gAbm5uePnll7FgwQLV1kgifeGuMaL/lP7hv3TpEgDl7h8XFxecPn0aYWFhqFWrFnr06AEAKCgowLx589C8eXPY29ujXr16CA8Px40bN9Qed8OGDejQoQNcXFzg4uKC1q1bIyYmRnW/pl1jmzdvRkhICNzc3ODk5IRGjRrhlVdeUd1f0a6xgwcPokePHqhVqxacnJzQsWNH/PTTT5IxpbuIfv/9d4wfPx5169ZFnTp18Oyzz+LatWs6v38AVIXl9evXJf2bNm1Chw4d4OzsDBcXF/Tq1Qvx8fFqyx85cgT9+vVDnTp14ODggMaNG2Py5Mmq+//991+Eh4ejadOmcHJygq+vL/r164fTp09XK/f95s+fD4VCgS+++EJSBJWys7PDM888o2orFAq8//77auMaNmyIkSNHqtql7/uePXvwyiuvoF69enBycsKmTZugUCjw66+/qj1GdHQ0FAoFTp06peo7duwYnnnmGdSuXRsODg5o06YNvv322yq9tujoaHh5eaFnz55VGn//a3z99dfx2WefoUWLFrC3t8eXX34JAJgzZw5CQkJQu3ZtuLq6IigoCDExMSh/Pe/yu8ZKP8OLFy/G0qVLERAQABcXF3To0AF//PGHWoZhw4bh3Llz+P3337XKTvQgLISI/vPvv/8CAOrVq6fqKygowDPPPIPu3bvjhx9+wJw5c1BSUoL+/ftjwYIFGDp0KH766ScsWLAAsbGx6Nq1K/Ly8lTLz5o1Cy+99BJ8fHywdu1abN26FSNGjFAVW5rExcVh8ODBaNSoEb755hv89NNPmDVrFoqKiirNv2/fPnTv3h3Z2dmIiYnBxo0bUatWLfTr1w+bNm1SGz969GjY2tpiw4YNWLhwIfbu3YuXX35Z27dNIjk5GTY2NmjUqJGqb/78+XjxxRfRsmVLfPvtt/jqq6+Qm5uLzp07IzExUTVu9+7d6Ny5M1JSUrB06VL8/PPPePfddyVF1bVr11CnTh0sWLAAu3btwooVK2BjY4OQkBCcPXu2WtkBoLi4GL/99huCg4NRv379aj+eJq+88gpsbW3x1VdfYcuWLRg4cCA8PDywZs0atbFr165FUFAQHnvsMQDA77//jk6dOiErKwufffYZfvjhB7Ru3RqDBw+u0vFiP/30E7p06QIrK+3/9G/btg3R0dGYNWuWal0ByoJm7Nix+Pbbb/H999/j2WefxcSJE/HBBx9U6XFXrFiB2NhYfPzxx1i/fj3u3LmDPn36IDs7WzIuODgYLi4uaoU9UbUJIguzZs0aAUD88ccforCwUOTm5oodO3aIevXqiVq1aom0tDQhhBAjRowQAMTq1asly2/cuFEAEN99952k/+jRowKAiIqKEkIIkZSUJKytrcVLL71UaZ4RI0YIf39/VXvx4sUCgMjKyqpwmeTkZAFArFmzRtXXvn174eHhIXJzc1V9RUVFIjAwUPj5+YmSkhLJ658wYYLkMRcuXCgAiNTU1ErzlmZ2dnYWhYWForCwUGRkZIjo6GhhZWUl3nnnHdW4lJQUYWNjIyZOnChZPjc3V3h5eYlBgwap+ho3biwaN24s8vLyHvj897++goIC0bRpUzFlyhRVv6b3p/R1JycnV/h4aWlpAoAYMmRIlTMAELNnz1br9/f3FyNGjFB7/uHDh6uNjYiIEI6OjpJ1npiYKACITz/9VNXXvHlz0aZNG1FYWChZ/umnnxbe3t6iuLi4wpzXr18XAMSCBQsqfT3+/v6ib9++aq/Rzc1N3Lx5s9Jli4uLRWFhoZg7d66oU6eO6jMnhBChoaEiNDRU1S5dR48++qgoKipS9f/5558CgNi4caPa43fq1EmEhIRUmoFIW9wiRBarffv2sLW1Ra1atfD000/Dy8sLP//8Mzw9PSXjnnvuOUl7x44dcHd3R79+/VBUVKS6tW7dGl5eXti7dy8AIDY2FsXFxXjttde0ytWuXTsAwKBBg/Dtt99W6Uy2O3fu4MiRI3j++efh4uKi6re2tsawYcNw5coVtS0m9+/eAaDa6lC6taqkpETy+oqLi9We09bWFra2tqhbty7Gjx+PwYMH48MPP1SN2b17N4qKijB8+HDJYzk4OCA0NFT1Xp07dw4XLlzAqFGj4ODgUOHrLCoqwvz589GyZUvY2dnBxsYGdnZ2OH/+PM6cOfPA98kYlP88AcqtRHl5eZItd2vWrIG9vT2GDh0KQLnF8p9//lEdv3b/+9mnTx+kpqZWulWsdLenh4eHTrm7d++Ohx56SK3/t99+w5NPPgk3NzdYW1vD1tYWs2bNQmZmJtLT0x/4uH379oW1tbWqXf5zeD8PDw9Zzuwk88ZCiCzWunXrcPToUcTHx+PatWs4deoUOnXqJBnj5OQEV1dXSd/169eRlZUFOzs7VSFQektLS0NGRgYAqI4X8vPz0ypXly5dsG3bNlUB4efnh8DAQGzcuLHCZW7dugUhBLy9vdXu8/HxAQBkZmZK+uvUqSNplx4PU7prb+7cuZLXVv6gbkdHRxw9ehRHjx7Fjz/+iK5du2Ljxo1YsGCBakzpbq127dqpvVebNm3S+r2KiIjAe++9hwEDBuDHH3/EkSNHcPToUbRq1UqyS1JXdevWhZOTE5KTk6v9WBXRtI4eeeQRtGvXTrV7rLi4GF9//TX69++P2rVrAyh7L6dOnar2Xk6YMAEAVO+nJqXvT2WFpra5//zzT4SFhQFQnml36NAhHD16FDNnzpQ8Z2Ue9Dm8n4ODg17WM9H9eNYYWawWLVqoDu6tiKazU0oPLq7ozKFatWoBKDvW6MqVK1ofb9K/f3/0798f+fn5+OOPPxAZGYmhQ4eiYcOG6NChg9r4hx56CFZWVkhNTVW7r3RLQN26dbXK8Oqrr+Lpp59WtcsfOGxlZSV5/3r27Ing4GDMmTMHL730EurXr696zi1btsDf37/C57r/varM119/jeHDh2P+/PmS/oyMDLi7u1fpdVXG2toaPXr0wM8//4wrV65UqYi1t7dHfn6+Wn/5wrNURWc8hYeHY8KECThz5gySkpKQmpoqOVW89L2cMWMGnn32WY2P8fDDD1eYs3R5XedA0pT7m2++ga2tLXbs2CEpsLZt26bTczzIzZs3tf4cEz0ICyEiLT399NP45ptvUFxcjJCQkArHhYWFwdraGtHR0RqLl6qwt7dHaGgo3N3dsXv3bsTHx2t8LGdnZ4SEhOD777/H4sWLVac1l5SU4Ouvv4afnx+aNWum1XP7+PiotiZVNeuKFSvQtWtXzJs3D59//jl69eoFGxsbXLhwQeMuoVLNmjVD48aNsXr1akRERGg8WwtQfhmXv++nn37C1atX0aRJkypnrcyMGTOwc+dOjBkzBj/88APs7Owk9xcWFmLXrl3o168fAOXZYfef1QUodxfdvn1bq+d98cUXERERgbVr1yIpKQm+vr6qrS2Asshp2rQpTp48qVYIVoW/vz8cHR1x4cIFrZetSOmEpPfv2srLy8NXX32lt+e4X1JSEgIDA2vksclysRAi0tKQIUOwfv169OnTB2+88QYef/xx2Nra4sqVK/j999/Rv39/DBw4EA0bNsQ777yDDz74AHl5eXjxxRfh5uaGxMREZGRkYM6cORoff9asWbhy5Qp69OgBPz8/ZGVl4ZNPPoGtrS1CQ0MrzBUZGYmePXuiW7dumDp1Kuzs7BAVFYW//voLGzduNMjcK6GhoejTpw/WrFmD6dOnIyAgAHPnzsXMmTORlJSEp556Cg899BCuX7+OP//8E87Ozqr3YcWKFejXrx/at2+PKVOmoEGDBkhJScHu3buxfv16AMoidO3atWjevDkee+wxHD9+HIsWLdJ692NlOnTogOjoaEyYMAHBwcEYP348HnnkERQWFiI+Ph5ffPEFAgMDVYXQsGHD8N5772HWrFkIDQ1FYmIili9fDjc3N62e193dHQMHDsTatWuRlZWFqVOnqp3d9fnnn6N3797o1asXRo4cCV9fX9y8eRNnzpzBiRMnsHnz5gof387OrsJT03XVt29fLF26FEOHDsWrr76KzMxMLF68uMJCtjoyMzNx/vx5TJw4Ue+PTRZO7qO1iQyt9Oydo0ePVjqu9MwoTQoLC8XixYtFq1athIODg3BxcRHNmzcXY8eOFefPn5eMXbdunWjXrp1qXJs2bSRnM5U/a2zHjh2id+/ewtfXV9jZ2QkPDw/Rp08fceDAAdUYTWdFCSHEgQMHRPfu3YWzs7NwdHQU7du3Fz/++GOVXv/vv/8uAIjff/+90vflQe/N6dOnhZWVlQgPD1f1bdu2TXTr1k24uroKe3t74e/vL55//nnxyy+/SJaNi4sTvXv3Fm5ubsLe3l40btxYcjbYrVu3xKhRo4SHh4dwcnISTzzxhDhw4ECFZyRpe9bY/RISEsSIESNEgwYNhJ2dnXB2dhZt2rQRs2bNEunp6apx+fn54q233hL169cXjo6OIjQ0VCQkJFR41lhln7s9e/YIAAKAOHfunMYxJ0+eFIMGDRIeHh7C1tZWeHl5ie7du4vPPvvsga8pJiZGWFtbi2vXrlU4pqKzxl577TWN41evXi0efvhhYW9vLxo1aiQiIyNFTEyM2ntd0TpatGiR2mNCw5l4MTExwtbWVnVWJ5G+KIQoN+sVERGZpXv37qFBgwZ488038fbbb8sdRyudO3dGgwYNVFsHifSFhRARkQWJjo7G+++/j6SkJINeU6869u/fj7CwMCQmJkom6yTSBx4jRERkQV599VVkZWUhKSkJjz76qNxxqiQzMxPr1q1jEUQ1gluEiIiIyGLJOqHi/v370a9fP/j4+EChUFRp7ol9+/YhODgYDg4OaNSoET777LOaD0pERERmSdZC6M6dO2jVqhWWL19epfHJycno06cPOnfujPj4eLzzzjuYNGkSvvvuuxpOSkRERObIaHaNKRQKbN26FQMGDKhwzNtvv43t27dLrik0btw4nDx5EnFxcQZISURERObEpA6WjouLk8y0CgC9evVCTEwMCgsLYWtrq7ZMfn6+ZPr7kpIS3Lx5E3Xq1DHIBHNERERUfUII5ObmwsfHR22y0eowqUIoLS1N7crgnp6eKCoqQkZGhsaLAkZGRlY4gy8RERGZlsuXL+t1NnmTKoQA9Qv/le7Zq2jrzowZMxAREaFqZ2dno0GDBrh8+bLaVcUt0alTwLBhwMWLZX1DhgCffy5bJCIiIjU5OTmoX7++6sLW+mJShZCXlxfS0tIkfenp6bCxsUGdOnU0LmNvb6/xujeurq4shAA88QRw4QJw3zUT8c03QEwM4OQkXy4iIiJN9H1Yi6xnjWmrQ4cOiI2NlfTt2bMHbdu21Xh8EFWNlRXw66/Svg0b5MlCRERkSLIWQrdv30ZCQgISEhIAKE+PT0hIQEpKCgDlbq3hw4erxo8bNw6XLl1CREQEzpw5g9WrVyMmJgZTp06VI75Z6dpV2h4zBnB0BHbsAPbuVd7S02UIRkREVINk3TV27NgxdOvWTdUuPZZnxIgRWLt2LVJTU1VFEQAEBARg586dmDJlClasWAEfHx8sW7YMzz33nMGzmxsrK2D5cuD118v67t0D+vUra7u4AGfOAHo8Ro2IiEhWRjOPkKHk5OTAzc0N2dnZPEZIg0mTgE8/rXzMp59KCyYiIqKaVlPf3yyESE1BgfKssWvXlO0FC9TH7N0LhIYaNBYREVmwmvr+Nqmzxsgw7OyAiRPL2v36AaNHK3eLlTp5koUQERGZPpM6a4zk0bEjkJgIdOpU1nf5snx5iIiI9IWFEFXZ8eNlPy9eDAQHA7dvy5eHiIioulgIUZUtXy5tnzgB1K4N/POPPHmIiIiqi4UQVdmoUcC0adK+wkKgRQt58hAREVUXCyHSysKFwK5d6v2Wde4hERGZCxZCpLVevYCrV6V9bdvKk4WIiKg6WAiRTnx8pO0TJ4DUVHmyEBER6YqFEOksOVna9vEBsrPlyUJERKQLFkKks4YNgUcflfb98ossUYiIiHTCQoiq5dgxafuFF4CiInmyEBERaYuFEFWLnR3QuHFZWwjA01O+PERERNpgIUTVtmOHtH3zJk+nJyIi08BCiKqteXPgyBFp35Yt8mQhIiLSBgsh0ovHH5e2Bw0CrlyRJwsREVFVsRAivfngA2m7fn0gL0+eLERERFXBQoj05t13AX9/aZ+TE5CZKU8eIiKiB2EhRHp14YJ63wsvGD4HERFRVbAQIr2ytgYuX5b2/f478PXX8uQhIiKqDAsh0js/P2DfPmnfsGHApUvy5CEiIqoICyGqEV26ABER0r6GDYHCQlniEBERacRCiGrMkiWAra20z84OKCmRJw8REVF5LISoRpU/XghQHkd07ZrhsxAREZXHQohqlKcnkJys3r9hg+GzEBERlcdCiGpcw4bqW4amTQOuXpUlDhERkQoLITIIPz9g8mT1vvfeAxYtAtLTZYlFREQWTiGEZV0nPCcnB25ubsjOzoarq6vccSzKiRNAcHDF91++rCyOiIiIyqup729uESKDCQoC/v234vvr1wfeecdweYiIiLhFiAzu2jUgMVF5Gn2vXur3W9YnkoiIqoJbhMhs+PgATz4JhIUBGRnA8OHS+5cvlycXERFZHhZCJKs6dYAvv5T2TZwIHDggTx4iIrIsLITIKEydKm136aL5SvZERET6xEKIjMKiRUCDBtK+Rx+VJwsREVkOFkJkNP76S9rOywP69ZMnCxERWQYWQmQ0atUCEhKkfTt28CwyIiKqOSyEyKi0aqUsfu43eLA8WYiIyPyxECKj07evtL15szw5iIjI/LEQIqP07bdyJyAiIkvAQoiMUseO0nZJiTw5iIjIvLEQIqPk6yttFxfLk4OIiMwbCyEyCevXy52AiIjMEQshMgnh4UBamtwpiIjI3LAQIqM1bpy0/cIL8uQgIiLzxUKIjNb//idtHzwIxMfLk4WIiMwTCyEyWg4OwJ9/SvuOHJEnCxERmScWQmTU2rWTtsePlycHERGZJxZCZPRefFHanjJFnhxERGR+WAiR0Zs/X9r++GPgjz9kiUJERGaGhRAZvYYNgd27pX2ffipLFCIiMjMshMgkhIUBtWuXtTdskC8LERGZDxZCZDImTZK2L1+WJwcREZkPFkJkMspPqNigAbB5My/ISkREumMhRCajZUvAyUnaN2iQ+sSLREREVcVCiExKaqp639Sphs9BRETmgYUQmRRXV+D6dfX+FSsMn4WIiEwfCyEyOR4ewM6d0r7XXwfu3JEnDxERmS4WQmSSevdWP3ja0VGeLEREZLpYCJHJ+vxzaXvWLCAzU54sRERkmlgIkclyc5O2P/wQ8PGRJwsREZkmFkJksqysgHHjpH0FBfJkISIi08RCiExaVBQQHS3ty8qSJQoREZkgFkJk0hQKoH17ad9DDwHbtskSh4iITIzshVBUVBQCAgLg4OCA4OBgHDhwoNLx69evR6tWreDk5ARvb2+Eh4cjk0fIWrRHH1XvGzjQ8DmIiMj0yFoIbdq0CZMnT8bMmTMRHx+Pzp07o3fv3khJSdE4/uDBgxg+fDhGjRqFv//+G5s3b8bRo0cxevRoAycnY2JtDWRnA02aSPsvXpQlDhERmRBZC6GlS5di1KhRGD16NFq0aIGPP/4Y9evXR3T5gz7+88cff6Bhw4aYNGkSAgIC8MQTT2Ds2LE4duyYgZOTsXF1Bc6fl/bFxcmThYiITIdshVBBQQGOHz+OsLAwSX9YWBgOHz6scZmOHTviypUr2LlzJ4QQuH79OrZs2YK+fftW+Dz5+fnIycmR3MgyDB0qdwIiIjJ2shVCGRkZKC4uhqenp6Tf09MTaWlpGpfp2LEj1q9fj8GDB8POzg5eXl5wd3fHp59+WuHzREZGws3NTXWrX7++Xl8HGRdfX2mbp9MTEVFlZD9YWqFQSNpCCLW+UomJiZg0aRJmzZqF48ePY9euXUhOTsa48pPJ3GfGjBnIzs5W3S5fvqzX/GRcEhKk7d27ZYlBREQmwkauJ65bty6sra3Vtv6kp6erbSUqFRkZiU6dOmHatGkAgMceewzOzs7o3Lkz5s2bB29vb7Vl7O3tYW9vr/8XQEapbl1pe8oUoF8/ebIQEZHxk22LkJ2dHYKDgxEbGyvpj42NRceOHTUuc/fuXVhZSSNbW1sDUG5JIgKAevXKfr5wAeCx9EREVBFZd41FRERg1apVWL16Nc6cOYMpU6YgJSVFtatrxowZGD58uGp8v3798P333yM6OhpJSUk4dOgQJk2ahMcffxw+vMgU/WfECGm7XTt5chARkfGTbdcYAAwePBiZmZmYO3cuUlNTERgYiJ07d8Lf3x8AkJqaKplTaOTIkcjNzcXy5cvx5ptvwt3dHd27d8dHH30k10sgIzRlCrB4sbTv5k2gdm158hARkfFSCAvbp5STkwM3NzdkZ2fD1dVV7jhUQ5KTgUaNlD+7uQE3bgC2tvJmIiIi3dXU97fsZ40R1YR//in7OTsbuHtXvixERGS8WAiRWbpyRdr+8EN5chARkXFjIURmqfys0osWAamp8mQhIiLjxUKIzJKzMzBhgrTvjz/kyUJERMaLhRCZrRkzpO1nnwVOn5YnCxERGScWQmS2/PyA/+bbVHnsMXmyEBGRcWIhRGbtvmmoAADNm8uTg4iIjBMLITJrPj7Am2+WtT085MtCRETGh4UQmb0lS8p+zsyULwcRERkfFkJk9oKCyn4eO1a+HEREZHxYCJHZO3eu7OfyZ5IREZFlYyFEZs/qvk/5nTvy5SAiIuPDQojM3qxZZT/XqiVfDiIiMj4shMjsrVxZ9nNuLpCTI18WIiIyLiyEyOyNHy9tJyXJk4OIiIwPCyEyexMnStvTp8uTg4iIjA8LITJ7VuU+5bt3y5ODiIiMDwshsgivvSZ3AiIiMkYshMgi+PhI2zEx8uQgIiLjwkKILEJEhLQ9ejRw9qw8WYiIyHiwECKL4OAADBgg7eOV6ImIiIUQWYzJk+VOQERExoaFEFmM0FDOIURERFIshMii/PijtH31qjw5iIjIOLAQIovSsaO0femSPDmIiMg4sBAii9K2rbS9apU8OYiIyDiwECKLtmaN3AmIiEhOLITI4jz7rLR94YI8OYiISH4shMjiTJggbW/YIE8OIiKSHwshsjg9ekjbnGGaiMhysRAii+TrW/bz+vWAEPJlISIi+bAQIotU/vIa6eny5CAiInmxECKLNGuWtN2ihTw5iIhIXiyEyCJ16SJt37olTw4iIpIXCyGyWJcvy52AiIjkxkKILNa5c3InICIiubEQIotVq5bcCYiISG4shMhi+fvLnYCIiOTGQogs1g8/lP3cs6d8OYiISD4shMhixcSU/RwRIV8OIiKSDwshskiFhcCRI2Xtrl1li0JERDJiIUQW6ehRafvwYXlyEBGRvFgIkUVKSpK2e/QA0tLkyUJERPJhIUQWqfwV6AHA25tzCxERWRoWQmSRvL2Be/fU+x9+GLhzx/B5iIhIHiyEyGLZ2wMXLqj3//mn4bMQEZE8WAiRRWvUCMjOlvZZ8beCiMhi8E8+WTxXV2nb2VmeHEREZHgshMjiFRZK2+3a8TghIiJLwUKILJ6mXWG9ehk+BxERGR4LIbJ41tbqW4AOHZInCxERGRYLISIATk7KXWKlJk2SLwsRERkOCyGi/9x/2Y3Bg+XLQUREhsNCiAjA9evSdocO8uQgIiLDYiFEBODiRWlboZAlBhERGRgLISIAzZrJnYCIiOTAQogIwLFjcicgIiI5sBAiAnD5srS9d68sMYiIyMBYCBEB6NpV2u7WDRBClihERGRALISIoLz4ankNGxo8BhERGRgLIaL/nD4tbaekAOnp8mQhIiLDYCFE9J/AQCAuTtrn6ChPFiIiMgwWQkT3qVdP2j55Up4cRERkGLIXQlFRUQgICICDgwOCg4Nx4MCBSsfn5+dj5syZ8Pf3h729PRo3bozVq1cbKC2Zuzp1pO0PP5QnBxERGYashdCmTZswefJkzJw5E/Hx8ejcuTN69+6NlJSUCpcZNGgQfv31V8TExODs2bPYuHEjmjdvbsDUZM7c3aXtXbuApUtliUJERAagEEK+k4RDQkIQFBSE6OhoVV+LFi0wYMAAREZGqo3ftWsXhgwZgqSkJNSuXVun58zJyYGbmxuys7Ph6uqqc3YyXx06AH/8Ie3jqfRERPKqqe9v2bYIFRQU4Pjx4wgLC5P0h4WF4fDhwxqX2b59O9q2bYuFCxfC19cXzZo1w9SpU5GXl1fh8+Tn5yMnJ0dyI6rM99+r9125YvgcRERU82zkeuKMjAwUFxfD09NT0u/p6Ym0tDSNyyQlJeHgwYNwcHDA1q1bkZGRgQkTJuDmzZsVHicUGRmJOXPm6D0/mS9vbyAjA6hbt6wvIwPw85MvExER1QzZD5ZWlLvMtxBCra9USUkJFAoF1q9fj8cffxx9+vTB0qVLsXbt2gq3Cs2YMQPZ2dmq2+Xy11Ig0qD8QdNXr8qTg4iIapZshVDdunVhbW2ttvUnPT1dbStRKW9vb/j6+sLNzU3V16JFCwghcKWCfRf29vZwdXWV3Ii09fTTwBtvyJ2CiIj0TbZCyM7ODsHBwYiNjZX0x8bGomPHjhqX6dSpE65du4bbt2+r+s6dOwcrKyv4cb8F1bBly+ROQERE+ibrrrGIiAisWrUKq1evxpkzZzBlyhSkpKRg3LhxAJS7tYYPH64aP3ToUNSpUwfh4eFITEzE/v37MW3aNLzyyitw5BTApGfnzwM2sh1FR0REhiDrn/nBgwcjMzMTc+fORWpqKgIDA7Fz5074+/sDAFJTUyVzCrm4uCA2NhYTJ05E27ZtUadOHQwaNAjz5s2T6yWQGWvSBOjdG/jxR7mTEBFRTZF1HiE5cB4h0saIEcC6dWVty/ptISIyHmY3jxCRKQgKkrYLC+XJQURENYOFEFElXFykbTs7IDNTnixERKR/LISIKtGvn3rff4ewERGRGdDpYOk7d+5gwYIF+PXXX5Geno6SkhLJ/UlJSXoJRyQ3Dw/gu++A554r67tzR748RESkXzoVQqNHj8a+ffswbNgweHt7VzgTNJE5ePZZ4NAhoFOnsr5t24ABA+RKRERE+qLTWWPu7u746aef0On+bwYTwbPGSBdFRYCtrbRv9WogPFyePERElsaozhp76KGHULt2bb2FIDJ2NjbqW4BeeQX46itZ4hARkZ7oVAh98MEHmDVrFu7evavvPERGa+tW6RXpAeC+ic+JiMgE6XSM0JIlS3DhwgV4enqiYcOGsC23z+DEiRN6CUdkbK5fB6ytpX0+PsqDqjdsAFq2lCcXERHpRqdCaACPEiULZWWlvAZZ06ZlfampytvkycCePbJFIyIiHehUCM2ePVvfOYhMRpMmwLRpwKJF0v769eXJQ0REuqvWRVePHz+OM2fOQKFQoGXLlmjTpo2+chEZtYULgVdflW4Zat9evjxERKQbnQqh9PR0DBkyBHv37oW7uzuEEMjOzka3bt3wzTffoF69evrOSWR00tKk7VdfBcaMkScLERHpRqezxiZOnIicnBz8/fffuHnzJm7duoW//voLOTk5mDRpkr4zEhklTdNoTZwIXLhg+CxERKQbnSZUdHNzwy+//IJ27dpJ+v/880+EhYUhKytLX/n0jhMqkj7t3An07aver/1vFRERVcaoJlQsKSlRO2UeAGxtbdWuO0Zkznr3Btzd5U5BRES60qkQ6t69O9544w1cu3ZN1Xf16lVMmTIFPXr00Fs4ImOnUACZmdI+Dw95shARkfZ0KoSWL1+O3NxcNGzYEI0bN0aTJk0QEBCA3NxcfPrpp/rOSGTUrMr9Fnl5yZODiIi0p9NZY/Xr18eJEycQGxuLf/75B0IItGzZEk8++aS+8xEZvfJ7g99+W54cRESkPZ0OljZlPFia9C0+HggKKmuXlCh3mRERkf7U1Pd3lbcILVu2DK+++iocHBywbNmySsfyFHqyJLt3S9ssgoiITEeVtwgFBATg2LFjqFOnDgICAip+QIUCSUlJeguob9wiRPo2dizwxRdl7StXAF9f+fIQEZkj2bcIJScna/yZyNIFBkrbnTsDRvx/ASIiuo9OZ42VV1xcjISEBNy6dUsfD0dkUgYOlLaTk4Hnn5cnCxERaUenQmjy5MmIiYkBoCyCunTpgqCgINSvXx979+7VZz4io+fnB0yYIO377jt5shARkXZ0KoS2bNmCVq1aAQB+/PFHXLx4Ef/88w8mT56MmTNn6jUgkSn48ENpm4efERGZBp0KoYyMDHj9N2vczp078cILL6BZs2YYNWoUTp8+rdeARKag/GU2hgyRJQYREWlJp0LI09MTiYmJKC4uxq5du1QTKd69exfW1tZ6DUhkCsqfe7lhgzw5iIhIOzrNLB0eHo5BgwbB29sbCoUCPXv2BAAcOXIEzZs312tAIlNQfu6g27eB3FygVi158hARUdXoVAi9//77CAwMxOXLl/HCCy/A3t4eAGBtbY3p06frNSCRqRgzBli5sqzt4CBfFiIiqhpeYoNIT7ZvB/r3L2sXF6tfkJWIiHQj+4SKvMQGUeUOHpS2WQQRERk/XmKDSE/atQOOHStrX7sGeHvLl4eIyJzIvkWIl9ggqlz5U+h5jBARkfHjxnsiPWnTpuznJ54AHnpIvixERFQ1OhVCzz//PBYsWKDWv2jRIrzwwgvVDkVkakpKgEWLytoHDwK7d8uXh4iIqkanQmjfvn3o27evWv9TTz2F/fv3VzsUkanJyVHve+op4Nw5w2chIqKq06kQun37Nuzs7NT6bW1tkaPpG4HIzLm7A99+q97ftq3BoxARkRZ0KoQCAwOxadMmtf5vvvkGLVu2rHYoIlP0wgtAQYG0LzdXnixERFQ1Os0s/d577+G5557DhQsX0L17dwDAr7/+io0bN2Lz5s16DUhkSmxtgc6dgQMHyvpu3wZcXOTLREREFdOpEHrmmWewbds2zJ8/H1u2bIGjoyMee+wx/PLLLwgNDdV3RiKTUn6vsYa9yEREZCR0KoQAoG/fvhoPmCaydOUPk7O3B/75B3j4YXnyEBFRxXSeRygrKwurVq3CO++8g5s3bwIATpw4gatXr+otHJEp0vT/g23bDB6DiIiqQKctQqdOncKTTz4JNzc3XLx4EaNHj0bt2rWxdetWXLp0CevWrdN3TiKTMXs24O8PhIeX9fGgaSIi46TTFqGIiAiMHDkS58+fh8N91xHo3bs35xEiAvD009L2hx8CvDINEZHx0akQOnr0KMaOHavW7+vri7S0tGqHIjJ1deuq9zVqZPgcRERUOZ0KIQcHB40TJ549exb16tWrdigic7Bnj9wJiIjoQXQqhPr374+5c+eisLAQAKBQKJCSkoLp06fjueee02tAIlPVsydQv77cKYiIqDI6FUKLFy/GjRs34OHhgby8PISGhqJJkyaoVasWPvzwQ31nJDJZly+X/fz22/LlICIizXQ6a8zV1RUHDx7Eb7/9hhMnTqCkpARBQUF48skn9Z2PyGQlJUnbU6bIk4OIiCqmdSFUVFQEBwcHJCQkoHv37qpLbBCR1KxZ0ranpzw5iIioYlrvGrOxsYG/vz+Ki4trIg+R2Sh/unx+vjw5iIioYjodI/Tuu+9ixowZqhmliUhd587S9syZ8uQgIqKKKYQQQtuF2rRpg3///ReFhYXw9/eHs7Oz5P4TJ07oLaC+5eTkwM3NDdnZ2XB1dZU7DpmxixeBgICydoMGwKVLssUhIjJpNfX9rdPB0gMGDIBCoYAONRSRxSj/e7p4sTw5iIioYloVQnfv3sW0adOwbds2FBYWokePHvj0009RV9M0ukQWrlMnafv55+XJQUREFdPqGKHZs2dj7dq16Nu3L1588UX88ssvGD9+fE1lIzJp//wjbW/YIE8OIiKqmFbHCDVu3BgffvghhgwZAgD4888/0alTJ9y7dw/W1tY1FlKfeIwQGYpCod6Xnw/Y2Rk+CxGRqaup72+ttghdvnwZne87Febxxx+HjY0Nrl27prdAROZi4kT1PhZBRETGRatCqLi4GHbl/pLb2NigqKhIr6GIzMGyZUDv3tK+uDiA5xgQERkPrQ6WFkJg5MiRsLe3V/Xdu3cP48aNk5xC//333+svIZEJa9JE2u7YEfjlF6BHD3nyEBGRlFbHCIWHh1dp3Jo1a3QOVNN4jBAZ0rlzwMMPS/t69gT27JEnDxGRqaqp72+dJlQ0ZSyEyNC2bgWefVbat3o1UMX/VxAREYzkYOmaEBUVhYCAADg4OCA4OBgHDhyo0nKHDh2CjY0NWrduXbMBiapp4ED1vldeAXiFGiIi+claCG3atAmTJ0/GzJkzER8fj86dO6N3795ISUmpdLns7GwMHz4cPXigBZkITR/pMWMMn4OIiKRk3TUWEhKCoKAgREdHq/patGiBAQMGIDIyssLlhgwZgqZNm8La2hrbtm1DQkJClZ+Tu8ZILqdOAa1aSftKSjTPN0RERFJmt2usoKAAx48fR1hYmKQ/LCwMhw8frnC5NWvW4MKFC5g9e3aVnic/Px85OTmSG5EcHntMve/99w0eg4iI7iNbIZSRkYHi4mJ4enpK+j09PZGWlqZxmfPnz2P69OlYv349bGyqduZ/ZGQk3NzcVLf69etXOzuRrqZOlbbnzpUnBxERKcl+sLSi3H4BIYRaH6CczHHo0KGYM2cOmjVrVuXHnzFjBrKzs1W3y5cvVzszka7mzVPvq+L5AUREVAO0mlBRn+rWrQtra2u1rT/p6elqW4kAIDc3F8eOHUN8fDxef/11AEBJSQmEELCxscGePXvQvXt3teXs7e0lE0ASycneHjh4EHjiibK+Ll2AxESgRQv5chERWSrZtgjZ2dkhODgYsbGxkv7Y2Fh07NhRbbyrqytOnz6NhIQE1W3cuHF4+OGHkZCQgJCQEENFJ6oWDR9vtGwJrF1r8ChERBZPti1CABAREYFhw4ahbdu26NChA7744gukpKRg3LhxAJS7ta5evYp169bBysoKgYGBkuU9PDzg4OCg1k9kzBQK4MYNoF49aX94OPD778CXX8qTi4jIEslaCA0ePBiZmZmYO3cuUlNTERgYiJ07d8Lf3x8AkJqa+sA5hYhMUd26wPnzQNOm0v5165TXIRs+XJ5cRESWhpfYIJLRvXtAcLDyGKHyDh3SvBuNiMgSmd08QkQEODgAf/+t+b5OnYD9+w2bh4jI0rAQIjICV68C/fur99+7Z/gsRESWhLvGiIxIy5bAmTNl7QEDlP8WFCiPJ3r3XeXxRURElqamvr9lPViaiKQ8PaWF0LZt0vt375beT0RE1cNdY0RGZP36yu//5x/lbjQiItIPFkJERsTHBygqAq5dK7u1aSMdc+OGPNmIiMwRCyEiI2NtDXh7l91WrJDeX353GRER6Y6FEJGRKz9xekCAPDmIiMwRCyEiIyYEMGiQtG/YMHmyEBGZIxZCREbsww+BXbvK2omJgBV/a4mI9IZ/UomM2HvvSdsNG8oSg4jIbLEQIjJitrbSdlSUPDmIiMwVCyEiI7ZwobQ9dao8OYiIzBULISIjVv5A6aAgeXIQEZkrFkJERmzrVml7715ZYhARmS0WQkRG7KuvpO1ateTJQURkrlgIERmxI0fkTkBEZN5YCBEZqfK7xQDgwAHD5yAiMmcshIiM1A8/qPfNnWv4HERE5oyFEJGRmjIFaN1a2tevnyxRiIjMFgshIiPVqhVw6ZK0b+JEebIQEZkrFkJERuzWLWn7u+/kyUFEZK5YCBEZsfK7wr79Vp4cRETmioUQkRF79VVp+9o1eXIQEZkrFkJERszdXdr285MlBhGR2WIhRGTELl6Utp9/XpYYRERmi4UQkRH7+GNpu0sXWWIQEZktFkJERuzyZWn7zh15chARmSsWQkRGLD1d2g4IkCcHEZG5YiFEZKRu3JC2e/SQJwcRkTljIURkpDw8pO1vvpEnBxGROWMhRGSEduyQths2BOrWlSUKEZFZYyFEZIS+/lra9veXJwcRkbljIURkhDZvlrYnTZInBxGRuWMhRGRkUlKAkpKyduvWwLPPyhaHiMissRAiMjIZGdJ2QoIsMYiILAILISIjc+WKtP3SS/LkICKyBCyEiIzIunVA//5l7aAg4Isv5MtDRGTuWAgRGYmTJ4ERI6R90dGAk5M8eYiILAELISIjUf66YgAQEgJMmQKcOAEIYfhMRETmjoUQkZHo3Vtz/8cfA8HBQK9eBo1DRGQRWAgRGQlra+Vp87/8ovn+2FjlZTccHACFQnkrPwM1ERFph4UQkRFRKJQXV83OBj75RP3+GzeA/Pyydr9+wA8/GC4fEZG5YSFEZIRcXZWzSZ89C7Rooezz9gYaNVIfe/GiQaMREZkVFkJERqxZMyAxUXmg9LVrwOefq48ZNszwuYiIzAULISITomnrT/nrkhERUdWxECIyIf36qfc1b274HERE5oKFEJEJ0XQV+hdfLDuLbOBA4N49w+ciIjJVLISITIimC7Cmppb9vG0b8McfhkpDRGT6WAgRmZCffgLq11f+7Oqqfn/r1kCHDgaNRERk0mzkDkBEVdekCZCSUtZWKKT3r1wJ2NsbNhMRkSnjFiEiM5KdLXcCIiLTwkKIyAQJAURESPtGjQK6d5cnDxGRqWIhRGSC9u8H/vc/ad/zz6vvKiMiosqxECIyQT4+6n1+fobPQURk6lgIEZmgevXU+wIDDZ+DiMjUsRAiMkErV6r37d5t+BxERKaOhRCRCdK0G+ypp4CrVw2fhYjIlLEQIjJBmq45BijPJiMioqpjIURkglxcgNBQ9f5PPzV8FiIiU8aZpYlMVMuWwL590r6FC6Wn0AsBZGQArVoBY8YAjo6GzUhEZOxYCBGZqO7dgeho9f6PPtI8Pj8fmDatZjMREZka7hojMlH16wNWWvwG+/vXXBYiIlMleyEUFRWFgIAAODg4IDg4GAcOHKhw7Pfff4+ePXuiXr16cHV1RYcOHbCb5wyThQoJAS5dAvbuld62bFEfO3ky0LQpEB9fdrt0yaBxiYiMkkII+c4z2bRpE4YNG4aoqCh06tQJn3/+OVatWoXExEQ0aNBAbfzkyZPh4+ODbt26wd3dHWvWrMHixYtx5MgRtGnTpkrPmZOTAzc3N2RnZ8PV1VXfL4lIdsePA23bVm2sry/w11+Au3uNRiIiqraa+v6WtRAKCQlBUFAQou870KFFixYYMGAAIiMjq/QYjzzyCAYPHoxZs2ZVaTwLITJ3u3YBvXtrv5xCAcybB4wcqfkSHkREcqqp72/ZDpYuKCjA8ePHMX36dEl/WFgYDh8+XKXHKCkpQW5uLmrXrl3hmPz8fOTn56vaOTk5ugUmMhG9egFRUUBsrHJLj4ND2X3FxcAXX2heTghg5kzlbfp0oIr/FyEiMmmyFUIZGRkoLi6Gp6enpN/T0xNpaWlVeowlS5bgzp07GDRoUIVjIiMjMWfOnGplJTIlCgUwfrzypsn//gd06gQkJFT8GAsWsBAiIssg+8HSivsnPQEghFDr02Tjxo14//33sWnTJnh4eFQ4bsaMGcjOzlbdLl++XO3MRKbMyUl5sLQQytu8eepjWrY0fC4iIjnItkWobt26sLa2Vtv6k56erraVqLxNmzZh1KhR2Lx5M5588slKx9rb28Pe3r7aeYnMjRDKM8kuXJD29+ih+cwzIiJzJNsWITs7OwQHByM2NlbSHxsbi44dO1a43MaNGzFy5Ehs2LABffv2remYRGYrOlq9CNq1C/jlF55FRkSWQ9ZdYxEREVi1ahVWr16NM2fOYMqUKUhJScG4ceMAKHdrDR8+XDV+48aNGD58OJYsWYL27dsjLS0NaWlpyM7OluslEJksTccIxcQYPAYRkaxkvcTG4MGDkZmZiblz5yI1NRWBgYHYuXMn/P+bAjc1NRUpKSmq8Z9//jmKiorw2muv4bXXXlP1jxgxAmvXrjV0fCKTVXp8UHn3/boREVkEWecRkgPnESICNm8GKjrZ8sYNoG5dw+YhInqQmvr+lv2sMSIyvMrOR7h/3iEiInPHQojIAnXpAnTurN6/dCng4mL4PEREcpH1GCEikkdmJlD++sZ//835g4jI8nCLEJEF0nTl+UaNDJ+DiEhu3CJEZIGCg9X7Fi0CCgqUB0v37g3072/4XEREhsazxogsTFIS0Ljxg8d98QUQGirts7UFGjZUXs+MiMiQzO7q80QkDyenqo179VXN/b17Az/9xGKIiMwDCyEiC+PlBaSlAX/8AdjYKAua3FxgyJCqLf/zz8D582WFkJ0d0KABCyMiMk0shIgskKen+jFATk7AZ58pi6TAQGXfunWal3/4YWm7Tx/lViIiIlPDY4SIqEJXriiLnNOnHzzWsv6SEJGhcWZpIjI4Pz/g1CngxAnlMUMeHprHDR5s2FxERPrCQoiIHqhNG2DBAuChh9Tve+89YONGw2ciItIHFkJE9EDXrwOPPgqcPat+3wcfAFZWQE6O4XMREVUXCyEieqCUFODq1crHuLkBPXoAJ08aJhMRkT6wECKiB2rbFli+XHktsnbtgA4dNI/77Tdg2jTDZiMiqg4WQkT0QAoF8Npryguz/vkncPgwsGOH5rFBQYbNRkRUHSyEiEhrOTnK3WWaREYaNgsRUXWwECIirURGKo8HmjBB8/2cWJGITAkLISKqsuxs4J13Kh/z9tuGyUJEpA8shIioylxdgZdfrnzMiBGGyUJEpA8shIioyhQK4KuvlLvGKvLYY4bLQ0RUXSyEiEhrY8dWfN/27YbLQURUXSyEiEhrQ4ZUfN/MmYbLQURUXSyEiKjKLlwAatfWPFdQQABw8SLg62vwWEREOmMhRERV9vHHwK1bmu9LTgacnQ0ah4io2lgIEVGVPfss4OFR8f316gGdOilvjRsrD65WKIB58wyXkYhIGwohhJA7hCHl5OTAzc0N2dnZcHV1lTsOkckRQnmzttZuuX//VRZHRES6qKnvb24RIiKtKBSAlRXw0UfaLVdQUDN5iIiqg4UQEenkrbeA4mKgqKjsNnduxeP79FGOISIyJiyEiEhnVlbKXWSlt8omU7x4EbC1BbZtM1Q6IqIHYyFERHrj6PjgMa+/XvM5iIiqykbuAERkPkJDgTffBJYsqXhMz55AerruzyEE4O4O2Nvr/hhERKV41hgR6UVGBtChg/LssJrm5gbs2gW0b1/zz0VExoFnjRGRUTt3zjBFEABkZwP79hnmuYjIvHHXmAZCCBQVFaG4uFjuKCQja2tr2NjYQKFQyB3FJHToAMyfD7zzjrLdsSNQp472j/Pjjw8e07cvMGaM9o9NRFQed42VU1BQgNTUVNy9e1eGdGRsnJyc4O3tDTs7O7mjWITr1wEvL/X+Ro2UB1n36AEEBirPViMiy1JTu8a4Reg+JSUlSE5OhrW1NXx8fGBnZ8etARZKCIGCggLcuHEDycnJaNq0Kaz47VsjSkqApUuBadMqHjNrFvDCC4CTk+FyEZFlYCF0n4KCApSUlKB+/fpw4l9ci+fo6AhbW1tcunQJBQUFcHBwkDuSWVq8GHj77crHjBypvHXvrp/nzM4Gjh8va//vf8Dkyfp5bCIyLSyENOD//KkUPws1T5u3+LffaibDlCnKySD1VWgRkelgIUREspoyBWjVCvj+e6BNG+X8QPPnK89CM6QePZQ3TdLTgdOny9rvvqucL6mqhFCe8s+6msj48GDp+9y7dw/JyckICAjgbhACwM+EnAoK9Hdtsq+/BsaO1c9jVUdkpG5n0umipERZTBYVARMnAk2aGOZ5iWpKjc0DKCxMdna2ACCys7PV7svLyxOJiYkiLy9PhmQ1p1mzZsLW1lZcuXJF7b7Q0FABQAAQdnZ2omnTpuLDDz8URUVFNZanpKREzJ49W3h7ewsHBwcRGhoq/vrrr0qX+e6770RwcLBwc3MTTk5OolWrVmLdunWSMfPnzxdt27YVLi4uol69eqJ///7in3/+UXucsLAwUadOHQFAxMfHV/q85vqZsDTJyUL4+wuh3DZjeTcXFyFu3pR7LRBVT2Xf39XBDbVm7uDBg7h37x5eeOEFrF27VuOYMWPGIDU1FWfPnsWkSZPw7rvvYvHixTWWaeHChVi6dCmWL1+Oo0ePwsvLCz179kRubm6Fy9SuXRszZ85EXFwcTp06hfDwcISHh2P37t2qMfv27cNrr72GP/74A7GxsSgqKkJYWBju3LmjGnPnzh106tQJCxYsqLHXR8anYUMgKQnIyQFyc7W7nTgBdOtW9lheXkBYGPDUUxXfjE1JifKCt0SkjrvG7mOOu0HCw8Ph5eWF0NBQvPbaa/j3338lUwJ07doVrVu3xscff6zq69mzJ27fvo24uDi95xFCwMfHB5MnT8bb/50qlJ+fD09PT3z00UcYq8X+i6CgIPTt2xcffPCBxvtv3LgBDw8P7Nu3D126dJHcd/HiRQQEBCA+Ph6tW7eu8DnM8TNBNa+4GPjlF2D9eiAoCNB1K/7t28pjk06eVN4KCrR/jJAQYMMG5VxMRKaM8wiR1nJzc7F582YcOXIEzZs3x507d7B37150u/+/txo4Ojri1q1bFd7fu3dvHDhwoNLHuH37tsb+5ORkpKWlISwsTNVnb2+P0NBQHD58uEqFkBACv/32G86ePYuPPvqownHZ2dkAlFuTiAzJ2hro1Ut5q4qSEiA5GfjrL2Xhk5CgLHwuXFDu3HoQDw+gXTugbVvlv8HBmiemJCJ1LISqoG1bIC3N8M/r5QUcO6b78t988w2aNm2KRx55BAAwZMgQxMTEVFgIlZSUYM+ePdi9ezcmVzKpyqpVq5CXl6dTprT/3khPT09Jv6enJy5dulTpstnZ2fD19UV+fj6sra0RFRWFnj17ahwrhEBERASeeOIJBAYG6pSVqKZ98on28xf5+yvPsmvVCmjdWln4+PkBnPuVSDcshKogLQ24elXuFNqLiYnByy+/rGq//PLL6NKlC7KysuDu7q7qj4qKwqpVq1Dw33b3YcOGYfbs2RU+rq+vb7WzlZ+xWwjxwFm8a9WqhYSEBNy+fRu//vorIiIi0KhRI3Tt2lVt7Ouvv45Tp07h4MGD1c5KpC+ZmcCZM8A//wB79gCbNz94mdGjlXMctWql/Pe+X10i0gMWQlUg1ybm6jxvYmIijhw5gqNHj6qOxQGA4uJibNy4EePHj1f1vfTSS5g5cybs7e3h4+MDa2vrSh+7OrvGvP57UWlpafD29lb1p6enq20lKs/KygpN/jsHuHXr1jhz5gwiIyPVCqGJEydi+/bt2L9/P/z8/Cp9TKKatm0bMHCgbstevqzc2kNENYeFUBVUZ/eUXGJiYtClSxesWLFC0v/VV18hJiZGUgi5ubmpCoyqqM6usYCAAHh5eSE2NhZt2rQBoLy0yb59+yo93kcTIQTy8/Ml7YkTJ2Lr1q3Yu3cvAgICdMpIpC8lJcpLg+jqr79YCBHVNBZCZqiwsBBfffUV5s6dq3Z8zOjRo7Fw4UKcPHkSrVq10unxq7NrTKFQYPLkyZg/fz6aNm2Kpk2bYv78+XBycsLQoUNV44YPHw5fX19ERkYCACIjI9G2bVs0btwYBQUF2LlzJ9atW4fo6GjVMq+99ho2bNiAH374AbVq1VIdj+Tm5gZHR0cAwM2bN5GSkoJr164BAM6ePQtAuaXKi0eXkp5ZWQHPPQesXq39so0bK3eHEVHNYiFkhrZv347MzEwM1LA9vmnTpnj00UcRExODZcuWyZAOeOutt5CXl4cJEybg1q1bCAkJwZ49e1CrVi3VmJSUFMl1vu7cuYMJEybgypUrcHR0RPPmzfH1119j8ODBqjGlRVH5XWVr1qzByP/+W759+3aEh4er7hsyZAgAYPbs2Xj//ff1/ErJEggBZGUBKSnK28WLyjPASm9JSdo/Zlyc8iQNG/6FJqpxnEfoPpwzhsrjZ4I0OXoUePxx/T6mgwPQuzfwxRdA3br6fWwic8B5hIiIjIS2p7xXxtUV2L+fu8GI5MJLbBARaSErS3n8jr7k5ChnoSYieXCLEBHRAxQXAzdvAmPGAD/8oN/HfvJJYPhw/T4mEVUdCyEisjh5ecD160BGRsW39PSyWyVXnKnUoEGAr6/y5uMj/fm/ExmJSGYshIjIonz9NTBsWM0/z+HDQIcONf88RFQ9LIQ0sLAT6agS/CyYnrt3lbuxsrI03957r+aee8kSICKi5h6fiPSPhdB9bG1tAQB3795VTcBHlu3u3bsAyj4bZNzefBNYulS+57e2BjZsUO8vLgZu3wZyc5UHR+fmlv08cSLQpQsvmkokF84jVE5qaiqysrLg4eEBJyenB14IlMyTEAJ3795Feno63N3dJddFI+NlTr+uc+YA06criytNHnBJQCKzU1PzCLEQKkcIgbS0NGRlZRk+HBkdd3d3eHl5sSA2EZa0mt5+G1iwQO4URIbDQkhPqvpGFhcXo7Cw0IDJyNjY2trCmv/tNinp6cC0acCRI4C9PWBnp/z3/hsA3LunvOXnS/8t31dUJO/rqUydOsoz3+4v/qw4MxyZMbOdWToqKgqLFi1CamoqHnnkEXz88cfo3LlzheP37duHiIgI/P333/Dx8cFbb72FcePG6T2XtbU1vwSJTEx0NLBundwpDCMzU/1aZAMHAvddfo80EEJZ5BYUKP+t7FY6ZswY5XxPlrTF0ZLIukVo06ZNGDZsGKKiotCpUyd8/vnnWLVqFRITE9GgQQO18cnJyQgMDMSYMWMwduxYHDp0CBMmTMDGjRvx3HPPVek5a6qiJCL5+fsrL3xKpG/r1hlm2gWqmFnuGgsJCUFQUJDqquEA0KJFCwwYMACRkZFq499++21s374dZ86cUfWNGzcOJ0+eRFxcXJWek4WQZSn9dAuh/nNl9+lzGT6n4Z7zhx80n7VFVF3R0UAN7HwgLZjdrrGCggIcP34c06dPl/SHhYXh8OHDGpeJi4tDWFiYpK9Xr16IiYlBYWGhVqc4u7lpn5mIiCzTnj3Ar7/KncKy1dRhu7IVQhkZGSguLoanp6ek39PTE2lpaRqXSUtL0zi+qKgIGRkZGk9xzs/PR35+vqqdnZ3930851XsBRERkMbZulTsBlX5v63tHluwHS5c/LVkIUempyprGa+ovFRkZiTlz5mi4p752QYmIiEh2mZmZcNPjbh3ZCqG6devC2tpabetPenq62lafUl5eXhrH29jYoE6dOhqXmTFjBiLum/M+KysL/v7+SElJ0esbSbrJyclB/fr1cfnyZR6zJTOuC+PBdWE8uC6MR3Z2Nho0aIDatWvr9XFlK4Ts7OwQHByM2NhYDBw4UNUfGxuL/v37a1ymQ4cO+PHHHyV9e/bsQdu2bSs8Psje3h72pZOH3MfNzY0faiPi6urK9WEkuC6MB9eF8eC6MB5Wep4wS9bptyIiIrBq1SqsXr0aZ86cwZQpU5CSkqKaF2jGjBkYPny4avy4ceNw6dIlRERE4MyZM1i9ejViYmIwdepUuV4CERERmTBZjxEaPHgwMjMzMXfuXKSmpiIwMBA7d+6Ev78/AOV1v1LumxQkICAAO3fuxJQpU7BixQr4+Phg2bJlVZ5DiIiIiOh+sh8sPWHCBEyYMEHjfWvXrlXrCw0NxYkTJ3R+Pnt7e8yePVvj7jIyPK4P48F1YTy4LowH14XxqKl1YXHXGiMiIiIqxUv0ERERkcViIUREREQWi4UQERERWSwWQkRERGSxzLIQioqKQkBAABwcHBAcHIwDBw5UOn7fvn0IDg6Gg4MDGjVqhM8++8xASc2fNuvi+++/R8+ePVGvXj24urqiQ4cO2L17twHTmj9tfzdKHTp0CDY2NmjdunXNBrQg2q6L/Px8zJw5E/7+/rC3t0fjxo2xevVqA6U1b9qui/Xr16NVq1ZwcnKCt7c3wsPDkZmZaaC05mv//v3o168ffHx8oFAosG3btgcuo5fvb2FmvvnmG2FraytWrlwpEhMTxRtvvCGcnZ3FpUuXNI5PSkoSTk5O4o033hCJiYli5cqVwtbWVmzZssXAyc2PtuvijTfeEB999JH4888/xblz58SMGTOEra2tOHHihIGTmydt10eprKws0ahRIxEWFiZatWplmLBmTpd18cwzz4iQkBARGxsrkpOTxZEjR8ShQ4cMmNo8absuDhw4IKysrMQnn3wikpKSxIEDB8QjjzwiBgwYYODk5mfnzp1i5syZ4rvvvhMAxNatWysdr6/vb7MrhB5//HExbtw4SV/z5s3F9OnTNY5/6623RPPmzSV9Y8eOFe3bt6+xjJZC23WhScuWLcWcOXP0Hc0i6bo+Bg8eLN59910xe/ZsFkJ6ou26+Pnnn4Wbm5vIzMw0RDyLou26WLRokWjUqJGkb9myZcLPz6/GMlqiqhRC+vr+NqtdYwUFBTh+/DjCwsIk/WFhYTh8+LDGZeLi4tTG9+rVC8eOHUNhYWGNZTV3uqyL8kpKSpCbm6v3C+xZIl3Xx5o1a3DhwgXMnj27piNaDF3Wxfbt29G2bVssXLgQvr6+aNasGaZOnYq8vDxDRDZbuqyLjh074sqVK9i5cyeEELh+/Tq2bNmCvn37GiIy3Udf39+yzyytTxkZGSguLla7er2np6faVetLpaWlaRxfVFSEjIwMeHt711hec6bLuihvyZIluHPnDgYNGlQTES2KLuvj/PnzmD59Og4cOAAbG7P6UyErXdZFUlISDh48CAcHB2zduhUZGRmYMGECbt68yeOEqkGXddGxY0esX78egwcPxr1791BUVIRnnnkGn376qSEi03309f1tVluESikUCklbCKHW96DxmvpJe9qui1IbN27E+++/j02bNsHDw6Om4lmcqq6P4uJiDB06FHPmzEGzZs0MFc+iaPO7UVJSAoVCgfXr1+Pxxx9Hnz59sHTpUqxdu5ZbhfRAm3WRmJiISZMmYdasWTh+/Dh27dqF5ORk1cXCybD08f1tVv/Nq1u3LqytrdUq+fT0dLWqsZSXl5fG8TY2NqhTp06NZTV3uqyLUps2bcKoUaOwefNmPPnkkzUZ02Jouz5yc3Nx7NgxxMfH4/XXXweg/DIWQsDGxgZ79uxB9+7dDZLd3Ojyu+Ht7Q1fX1+4ubmp+lq0aAEhBK5cuYKmTZvWaGZzpcu6iIyMRKdOnTBt2jQAwGOPPQZnZ2d07twZ8+bN414EA9LX97dZbRGys7NDcHAwYmNjJf2xsbHo2LGjxmU6dOigNn7Pnj1o27YtbG1tayyrudNlXQDKLUEjR47Ehg0buM9dj7RdH66urjh9+jQSEhJUt3HjxuHhhx9GQkICQkJCDBXd7Ojyu9GpUydcu3YNt2/fVvWdO3cOVlZW8PPzq9G85kyXdXH37l1YWUm/Oq2trQGUbY0gw9Db97dWh1abgNJTIWNiYkRiYqKYPHmycHZ2FhcvXhRCCDF9+nQxbNgw1fjS0++mTJkiEhMTRUxMDE+f1xNt18WGDRuEjY2NWLFihUhNTVXdsrKy5HoJZkXb9VEezxrTH23XRW5urvDz8xPPP/+8+Pvvv8W+fftE06ZNxejRo+V6CWZD23WxZs0aYWNjI6KiosSFCxfEwYMHRdu2bcXjjz8u10swG7m5uSI+Pl7Ex8cLAGLp0qUiPj5eNZVBTX1/m10hJIQQK1asEP7+/sLOzk4EBQWJffv2qe4bMWKECA0NlYzfu3evaNOmjbCzsxMNGzYU0dHRBk5svrRZF6GhoQKA2m3EiBGGD26mtP3duB8LIf3Sdl2cOXNGPPnkk8LR0VH4+fmJiIgIcffuXQOnNk/arotly5aJli1bCkdHR+Ht7S1eeuklceXKFQOnNj+///57pd8BNfX9rRCC2/KIiIjIMpnVMUJERERE2mAhRERERBaLhRARERFZLBZCREREZLFYCBEREZHFYiFEREREFouFEBEREVksFkJERAAaNmyIjz/+WNVWKBTYtm2bbHmIyDBYCBGR7EaOHAmFQgGFQgEbGxs0aNAA48ePx61bt+SORkRmjoUQERmFp556Cqmpqbh48SJWrVqFH3/8ERMmTJA7FhGZORZCRGQU7O3t4eXlBT8/P4SFhWHw4MHYs2eP6v41a9agRYsWcHBwQPPmzREVFSVZ/sqVKxgyZAhq164NZ2dntG3bFkeOHAEAXLhwAf3794enpydcXFzQrl07/PLLLwZ9fURknGzkDkBEVF5SUhJ27doFW1tbAMDKlSsxe/ZsLF++HG3atEF8fDzGjBkDZ2dnjBgxArdv30ZoaCh8fX2xfft2eHl54cSJEygpKQEA3L59G3369MG8efPg4OCAL7/8Ev369cPZs2fRoEEDOV8qEcmMhRARGYUdO3bAxcUFxcXFuHfvHgBg6dKlAIAPPvgAS5YswbPPPgsACAgIQGJiIj7//HOMGDECGzZswI0bN3D06FHUrl0bANCkSRPVY7dq1QqtWrVStefNm4etW7di+/bteP311w31EonICLEQIiKj0K1bN0RHR+Pu3btYtWoVzp07h4kTJ+LGjRu4fPkyRo0ahTFjxqjGFxUVwc3NDQCQkJCANm3aqIqg8u7cuYM5c+Zgx44duHbtGoqKipCXl4eUlBSDvDYiMl4shIjIKDg7O6u24ixbtgzdunXDnDlzVFtsVq5ciZCQEMky1tbWAABHR8dKH3vatGnYvXs3Fi9ejCZNmsDR0RHPP/88CgoKauCVEJEpYSFEREZp9uzZ6N27N8aPHw9fX18kJSXhpZde0jj2sccew6pVq3Dz5k2NW4UOHDiAkSNHYuDAgQCUxwxdvHixJuMTkYngWWNEZJS6du2KRx55BPPnz8f777+PyMhIfPLJJzh37hxOnz6NNWvWqI4hevHFF+Hl5YUBAwbg0KFDSEpKwnfffYe4uDgAyuOFvv/+eyQkJODkyZMYOnSo6kBqIrJsLISIyGhFRERg5cqV6NWrF1atWoW1a9fi0UcfRWhoKNauXYuAgAAAgJ2dHfbs2QMPDw/06dMHjz76KBYsWKDadfa///0PDz30EDp27Ih+/fqhV69eCAoKkvOlEZGRUAghhNwhiIiIiOTALUJERERksVgIERERkcViIUREREQWi4UQERERWSwWQkRERGSxWAgRERGRxWIhRERERBaLhRARERFZLBZCREREZLFYCBEREZHFYiFEREREFouFEBEREVms/wOH9zAXyGLyrQAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "y_proba = fit_lr_grid_search.predict_proba(X_train)[:,1]\n",
    "\n",
    "precision, recall, thresholds = precision_recall_curve(y_train, y_proba, pos_label=1, sample_weight=None)\n",
    "avg_p = average_precision_score(y_train, y_proba, pos_label=1, sample_weight=None)\n",
    "\n",
    "plt.title('Precision-Recall Curve (Train)')\n",
    "ax1= plt.plot(precision, recall, 'b', label = '%s AP = %0.3f' % ('', avg_p), linewidth=2)\n",
    "plt.legend(loc = 'lower left')\n",
    "plt.xlim([0, 1])\n",
    "plt.ylim([0, 1])\n",
    "plt.ylabel('Precision')\n",
    "plt.xlabel('Recall')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "74e43cba-127e-4d45-a450-870a2335c9b3",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "## Evaluation on Held-out Test Set\n",
    "\n",
    "Let's see how well our trained model does on data it has never seen before"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 94,
   "id": "bd51af1a-b66e-42a5-87ba-6f44766252f8",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>recordid</th>\n",
       "      <th>SAPS-I</th>\n",
       "      <th>SOFA</th>\n",
       "      <th>Length_of_stay</th>\n",
       "      <th>Survival</th>\n",
       "      <th>In-hospital_death</th>\n",
       "      <th>Age</th>\n",
       "      <th>Gender</th>\n",
       "      <th>Height</th>\n",
       "      <th>Weight</th>\n",
       "      <th>CCU</th>\n",
       "      <th>CSRU</th>\n",
       "      <th>SICU</th>\n",
       "      <th>DiasABP_first</th>\n",
       "      <th>GCS_first</th>\n",
       "      <th>Glucose_first</th>\n",
       "      <th>HR_first</th>\n",
       "      <th>MAP_first</th>\n",
       "      <th>NIDiasABP_first</th>\n",
       "      <th>NIMAP_first</th>\n",
       "      <th>NISysABP_first</th>\n",
       "      <th>RespRate_first</th>\n",
       "      <th>SaO2_first</th>\n",
       "      <th>Temp_first</th>\n",
       "      <th>DiasABP_last</th>\n",
       "      <th>GCS_last</th>\n",
       "      <th>Glucose_last</th>\n",
       "      <th>HR_last</th>\n",
       "      <th>MAP_last</th>\n",
       "      <th>NIDiasABP_last</th>\n",
       "      <th>NIMAP_last</th>\n",
       "      <th>NISysABP_last</th>\n",
       "      <th>RespRate_last</th>\n",
       "      <th>SaO2_last</th>\n",
       "      <th>Temp_last</th>\n",
       "      <th>DiasABP_lowest</th>\n",
       "      <th>GCS_lowest</th>\n",
       "      <th>Glucose_lowest</th>\n",
       "      <th>HR_lowest</th>\n",
       "      <th>MAP_lowest</th>\n",
       "      <th>NIDiasABP_lowest</th>\n",
       "      <th>NIMAP_lowest</th>\n",
       "      <th>NISysABP_lowest</th>\n",
       "      <th>RespRate_lowest</th>\n",
       "      <th>SaO2_lowest</th>\n",
       "      <th>Temp_lowest</th>\n",
       "      <th>DiasABP_highest</th>\n",
       "      <th>GCS_highest</th>\n",
       "      <th>Glucose_highest</th>\n",
       "      <th>HR_highest</th>\n",
       "      <th>MAP_highest</th>\n",
       "      <th>NIDiasABP_highest</th>\n",
       "      <th>NIMAP_highest</th>\n",
       "      <th>NISysABP_highest</th>\n",
       "      <th>RespRate_highest</th>\n",
       "      <th>SaO2_highest</th>\n",
       "      <th>Temp_highest</th>\n",
       "      <th>DiasABP_median</th>\n",
       "      <th>GCS_median</th>\n",
       "      <th>Glucose_median</th>\n",
       "      <th>HR_median</th>\n",
       "      <th>MAP_median</th>\n",
       "      <th>NIDiasABP_median</th>\n",
       "      <th>NIMAP_median</th>\n",
       "      <th>NISysABP_median</th>\n",
       "      <th>RespRate_median</th>\n",
       "      <th>SaO2_median</th>\n",
       "      <th>Temp_median</th>\n",
       "      <th>ALP_first</th>\n",
       "      <th>ALT_first</th>\n",
       "      <th>AST_first</th>\n",
       "      <th>Albumin_first</th>\n",
       "      <th>BUN_first</th>\n",
       "      <th>Bilirubin_first</th>\n",
       "      <th>Cholesterol_first</th>\n",
       "      <th>Creatinine_first</th>\n",
       "      <th>FiO2_first</th>\n",
       "      <th>HCO3_first</th>\n",
       "      <th>HCT_first</th>\n",
       "      <th>K_first</th>\n",
       "      <th>Lactate_first</th>\n",
       "      <th>Mg_first</th>\n",
       "      <th>Na_first</th>\n",
       "      <th>PaCO2_first</th>\n",
       "      <th>PaO2_first</th>\n",
       "      <th>Platelets_first</th>\n",
       "      <th>SysABP_first</th>\n",
       "      <th>TroponinI_first</th>\n",
       "      <th>TroponinT_first</th>\n",
       "      <th>WBC_first</th>\n",
       "      <th>Weight_first</th>\n",
       "      <th>pH_first</th>\n",
       "      <th>ALP_last</th>\n",
       "      <th>ALT_last</th>\n",
       "      <th>AST_last</th>\n",
       "      <th>Albumin_last</th>\n",
       "      <th>BUN_last</th>\n",
       "      <th>Bilirubin_last</th>\n",
       "      <th>Cholesterol_last</th>\n",
       "      <th>Creatinine_last</th>\n",
       "      <th>FiO2_last</th>\n",
       "      <th>HCO3_last</th>\n",
       "      <th>HCT_last</th>\n",
       "      <th>K_last</th>\n",
       "      <th>Lactate_last</th>\n",
       "      <th>Mg_last</th>\n",
       "      <th>Na_last</th>\n",
       "      <th>PaCO2_last</th>\n",
       "      <th>PaO2_last</th>\n",
       "      <th>Platelets_last</th>\n",
       "      <th>SysABP_last</th>\n",
       "      <th>TroponinI_last</th>\n",
       "      <th>TroponinT_last</th>\n",
       "      <th>WBC_last</th>\n",
       "      <th>Weight_last</th>\n",
       "      <th>pH_last</th>\n",
       "      <th>MechVentStartTime</th>\n",
       "      <th>MechVentDuration</th>\n",
       "      <th>MechVentLast8Hour</th>\n",
       "      <th>UrineOutputSum</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>142675</td>\n",
       "      <td>27</td>\n",
       "      <td>14</td>\n",
       "      <td>9</td>\n",
       "      <td>7</td>\n",
       "      <td>1</td>\n",
       "      <td>70.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>175.3</td>\n",
       "      <td>85.00</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>60.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>194.0</td>\n",
       "      <td>76.0</td>\n",
       "      <td>76.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>98.0</td>\n",
       "      <td>35.7</td>\n",
       "      <td>43.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>125.0</td>\n",
       "      <td>22.0</td>\n",
       "      <td>61.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>98.0</td>\n",
       "      <td>37.6</td>\n",
       "      <td>40.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>125.0</td>\n",
       "      <td>10.0</td>\n",
       "      <td>41.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>87.0</td>\n",
       "      <td>33.4</td>\n",
       "      <td>98.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>194.0</td>\n",
       "      <td>95.0</td>\n",
       "      <td>108.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>98.0</td>\n",
       "      <td>37.8</td>\n",
       "      <td>56.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>159.5</td>\n",
       "      <td>62.0</td>\n",
       "      <td>72.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>98.0</td>\n",
       "      <td>36.65</td>\n",
       "      <td>24.0</td>\n",
       "      <td>45.0</td>\n",
       "      <td>106.0</td>\n",
       "      <td>1.7</td>\n",
       "      <td>13.0</td>\n",
       "      <td>0.9</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.8</td>\n",
       "      <td>1.0</td>\n",
       "      <td>22.0</td>\n",
       "      <td>33.7</td>\n",
       "      <td>4.3</td>\n",
       "      <td>14.4</td>\n",
       "      <td>21.0</td>\n",
       "      <td>142.0</td>\n",
       "      <td>32.0</td>\n",
       "      <td>378.0</td>\n",
       "      <td>185.0</td>\n",
       "      <td>104.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>7.9</td>\n",
       "      <td>NaN</td>\n",
       "      <td>7.50</td>\n",
       "      <td>23.0</td>\n",
       "      <td>137.0</td>\n",
       "      <td>1106.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>24.0</td>\n",
       "      <td>2.8</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2.1</td>\n",
       "      <td>1.00</td>\n",
       "      <td>23.0</td>\n",
       "      <td>29.5</td>\n",
       "      <td>4.4</td>\n",
       "      <td>3.9</td>\n",
       "      <td>1.3</td>\n",
       "      <td>147.0</td>\n",
       "      <td>37.0</td>\n",
       "      <td>117.0</td>\n",
       "      <td>41.0</td>\n",
       "      <td>105.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>6.1</td>\n",
       "      <td>NaN</td>\n",
       "      <td>7.38</td>\n",
       "      <td>99.0</td>\n",
       "      <td>2559.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>13.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>142676</td>\n",
       "      <td>12</td>\n",
       "      <td>1</td>\n",
       "      <td>31</td>\n",
       "      <td>468</td>\n",
       "      <td>0</td>\n",
       "      <td>57.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>15.0</td>\n",
       "      <td>229.0</td>\n",
       "      <td>86.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>51.0</td>\n",
       "      <td>77.33</td>\n",
       "      <td>130.0</td>\n",
       "      <td>22.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>35.3</td>\n",
       "      <td>NaN</td>\n",
       "      <td>15.0</td>\n",
       "      <td>198.0</td>\n",
       "      <td>92.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>77.0</td>\n",
       "      <td>100.70</td>\n",
       "      <td>148.0</td>\n",
       "      <td>14.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>37.6</td>\n",
       "      <td>NaN</td>\n",
       "      <td>14.0</td>\n",
       "      <td>79.0</td>\n",
       "      <td>72.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>37.0</td>\n",
       "      <td>63.00</td>\n",
       "      <td>110.0</td>\n",
       "      <td>11.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>35.3</td>\n",
       "      <td>NaN</td>\n",
       "      <td>15.0</td>\n",
       "      <td>273.0</td>\n",
       "      <td>109.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>79.0</td>\n",
       "      <td>100.70</td>\n",
       "      <td>148.0</td>\n",
       "      <td>30.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>37.9</td>\n",
       "      <td>NaN</td>\n",
       "      <td>15.0</td>\n",
       "      <td>192.5</td>\n",
       "      <td>89.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>53.0</td>\n",
       "      <td>81.670</td>\n",
       "      <td>130.0</td>\n",
       "      <td>17.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>36.70</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>35.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.7</td>\n",
       "      <td>NaN</td>\n",
       "      <td>7.0</td>\n",
       "      <td>43.8</td>\n",
       "      <td>3.1</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2.0</td>\n",
       "      <td>148.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>616.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>22.4</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>10.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.4</td>\n",
       "      <td>NaN</td>\n",
       "      <td>21.0</td>\n",
       "      <td>32.5</td>\n",
       "      <td>3.2</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.8</td>\n",
       "      <td>134.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>517.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>15.1</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>142680</td>\n",
       "      <td>12</td>\n",
       "      <td>7</td>\n",
       "      <td>17</td>\n",
       "      <td>16</td>\n",
       "      <td>1</td>\n",
       "      <td>60.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>182.9</td>\n",
       "      <td>78.80</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>49.0</td>\n",
       "      <td>9.0</td>\n",
       "      <td>127.0</td>\n",
       "      <td>90.0</td>\n",
       "      <td>61.0</td>\n",
       "      <td>74.0</td>\n",
       "      <td>89.33</td>\n",
       "      <td>120.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>93.0</td>\n",
       "      <td>38.0</td>\n",
       "      <td>55.0</td>\n",
       "      <td>7.0</td>\n",
       "      <td>135.0</td>\n",
       "      <td>98.0</td>\n",
       "      <td>70.0</td>\n",
       "      <td>55.0</td>\n",
       "      <td>74.33</td>\n",
       "      <td>113.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>93.0</td>\n",
       "      <td>38.4</td>\n",
       "      <td>45.0</td>\n",
       "      <td>7.0</td>\n",
       "      <td>127.0</td>\n",
       "      <td>53.0</td>\n",
       "      <td>61.0</td>\n",
       "      <td>41.0</td>\n",
       "      <td>58.00</td>\n",
       "      <td>90.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>93.0</td>\n",
       "      <td>36.2</td>\n",
       "      <td>92.0</td>\n",
       "      <td>10.0</td>\n",
       "      <td>153.0</td>\n",
       "      <td>115.0</td>\n",
       "      <td>246.0</td>\n",
       "      <td>74.0</td>\n",
       "      <td>95.33</td>\n",
       "      <td>148.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>93.0</td>\n",
       "      <td>39.7</td>\n",
       "      <td>56.5</td>\n",
       "      <td>8.0</td>\n",
       "      <td>135.0</td>\n",
       "      <td>87.0</td>\n",
       "      <td>70.0</td>\n",
       "      <td>56.0</td>\n",
       "      <td>75.665</td>\n",
       "      <td>113.5</td>\n",
       "      <td>NaN</td>\n",
       "      <td>93.0</td>\n",
       "      <td>38.00</td>\n",
       "      <td>77.0</td>\n",
       "      <td>44.0</td>\n",
       "      <td>26.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>21.0</td>\n",
       "      <td>0.7</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.8</td>\n",
       "      <td>0.5</td>\n",
       "      <td>30.0</td>\n",
       "      <td>38.8</td>\n",
       "      <td>4.3</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.9</td>\n",
       "      <td>138.0</td>\n",
       "      <td>65.0</td>\n",
       "      <td>71.0</td>\n",
       "      <td>233.0</td>\n",
       "      <td>91.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>14.5</td>\n",
       "      <td>79.4</td>\n",
       "      <td>7.33</td>\n",
       "      <td>77.0</td>\n",
       "      <td>53.0</td>\n",
       "      <td>29.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>23.0</td>\n",
       "      <td>0.7</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.7</td>\n",
       "      <td>0.70</td>\n",
       "      <td>31.0</td>\n",
       "      <td>37.7</td>\n",
       "      <td>3.5</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.8</td>\n",
       "      <td>136.0</td>\n",
       "      <td>53.0</td>\n",
       "      <td>76.0</td>\n",
       "      <td>207.0</td>\n",
       "      <td>104.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>17.6</td>\n",
       "      <td>79.5</td>\n",
       "      <td>7.45</td>\n",
       "      <td>207.0</td>\n",
       "      <td>2570.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>20.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>142683</td>\n",
       "      <td>19</td>\n",
       "      <td>15</td>\n",
       "      <td>17</td>\n",
       "      <td>-1</td>\n",
       "      <td>0</td>\n",
       "      <td>64.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>180.3</td>\n",
       "      <td>145.20</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>76.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>191.0</td>\n",
       "      <td>87.0</td>\n",
       "      <td>75.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>95.0</td>\n",
       "      <td>37.4</td>\n",
       "      <td>59.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>166.0</td>\n",
       "      <td>68.0</td>\n",
       "      <td>84.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>97.0</td>\n",
       "      <td>37.0</td>\n",
       "      <td>40.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>166.0</td>\n",
       "      <td>62.0</td>\n",
       "      <td>29.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>89.0</td>\n",
       "      <td>36.6</td>\n",
       "      <td>90.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>191.0</td>\n",
       "      <td>87.0</td>\n",
       "      <td>113.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>97.0</td>\n",
       "      <td>38.0</td>\n",
       "      <td>58.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>178.5</td>\n",
       "      <td>74.0</td>\n",
       "      <td>77.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>92.5</td>\n",
       "      <td>37.20</td>\n",
       "      <td>39.0</td>\n",
       "      <td>160.0</td>\n",
       "      <td>186.0</td>\n",
       "      <td>1.6</td>\n",
       "      <td>11.0</td>\n",
       "      <td>0.7</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.9</td>\n",
       "      <td>1.0</td>\n",
       "      <td>18.0</td>\n",
       "      <td>33.9</td>\n",
       "      <td>4.9</td>\n",
       "      <td>1.8</td>\n",
       "      <td>1.6</td>\n",
       "      <td>138.0</td>\n",
       "      <td>45.0</td>\n",
       "      <td>294.0</td>\n",
       "      <td>104.0</td>\n",
       "      <td>137.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>18.3</td>\n",
       "      <td>NaN</td>\n",
       "      <td>7.33</td>\n",
       "      <td>49.0</td>\n",
       "      <td>1593.0</td>\n",
       "      <td>1933.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>19.0</td>\n",
       "      <td>1.8</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.4</td>\n",
       "      <td>0.65</td>\n",
       "      <td>24.0</td>\n",
       "      <td>28.3</td>\n",
       "      <td>5.0</td>\n",
       "      <td>4.5</td>\n",
       "      <td>2.4</td>\n",
       "      <td>137.0</td>\n",
       "      <td>42.0</td>\n",
       "      <td>107.0</td>\n",
       "      <td>79.0</td>\n",
       "      <td>140.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>18.3</td>\n",
       "      <td>NaN</td>\n",
       "      <td>7.36</td>\n",
       "      <td>374.0</td>\n",
       "      <td>2420.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>14.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>142688</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>9</td>\n",
       "      <td>-1</td>\n",
       "      <td>0</td>\n",
       "      <td>51.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>172.7</td>\n",
       "      <td>90.00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>55.0</td>\n",
       "      <td>15.0</td>\n",
       "      <td>165.0</td>\n",
       "      <td>91.0</td>\n",
       "      <td>74.0</td>\n",
       "      <td>63.0</td>\n",
       "      <td>76.33</td>\n",
       "      <td>103.0</td>\n",
       "      <td>24.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>36.8</td>\n",
       "      <td>68.0</td>\n",
       "      <td>14.0</td>\n",
       "      <td>130.0</td>\n",
       "      <td>113.0</td>\n",
       "      <td>81.0</td>\n",
       "      <td>66.0</td>\n",
       "      <td>86.33</td>\n",
       "      <td>127.0</td>\n",
       "      <td>20.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>38.7</td>\n",
       "      <td>53.0</td>\n",
       "      <td>14.0</td>\n",
       "      <td>130.0</td>\n",
       "      <td>88.0</td>\n",
       "      <td>69.0</td>\n",
       "      <td>50.0</td>\n",
       "      <td>71.00</td>\n",
       "      <td>99.0</td>\n",
       "      <td>13.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>36.8</td>\n",
       "      <td>74.0</td>\n",
       "      <td>15.0</td>\n",
       "      <td>165.0</td>\n",
       "      <td>115.0</td>\n",
       "      <td>89.0</td>\n",
       "      <td>68.0</td>\n",
       "      <td>86.33</td>\n",
       "      <td>127.0</td>\n",
       "      <td>27.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>38.7</td>\n",
       "      <td>58.0</td>\n",
       "      <td>15.0</td>\n",
       "      <td>147.5</td>\n",
       "      <td>100.5</td>\n",
       "      <td>74.5</td>\n",
       "      <td>60.0</td>\n",
       "      <td>73.500</td>\n",
       "      <td>106.5</td>\n",
       "      <td>20.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>37.75</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>19.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.9</td>\n",
       "      <td>1.0</td>\n",
       "      <td>26.0</td>\n",
       "      <td>38.4</td>\n",
       "      <td>5.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.9</td>\n",
       "      <td>141.0</td>\n",
       "      <td>50.0</td>\n",
       "      <td>67.0</td>\n",
       "      <td>215.0</td>\n",
       "      <td>126.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>16.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>7.37</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>19.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.7</td>\n",
       "      <td>0.50</td>\n",
       "      <td>29.0</td>\n",
       "      <td>29.4</td>\n",
       "      <td>3.7</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.7</td>\n",
       "      <td>141.0</td>\n",
       "      <td>50.0</td>\n",
       "      <td>67.0</td>\n",
       "      <td>164.0</td>\n",
       "      <td>107.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>12.5</td>\n",
       "      <td>NaN</td>\n",
       "      <td>7.37</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3995</th>\n",
       "      <td>152849</td>\n",
       "      <td>21</td>\n",
       "      <td>13</td>\n",
       "      <td>6</td>\n",
       "      <td>752</td>\n",
       "      <td>0</td>\n",
       "      <td>78.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>180.3</td>\n",
       "      <td>83.18</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>58.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>110.0</td>\n",
       "      <td>110.0</td>\n",
       "      <td>77.0</td>\n",
       "      <td>51.0</td>\n",
       "      <td>79.33</td>\n",
       "      <td>136.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>98.0</td>\n",
       "      <td>36.8</td>\n",
       "      <td>71.0</td>\n",
       "      <td>15.0</td>\n",
       "      <td>110.0</td>\n",
       "      <td>103.0</td>\n",
       "      <td>97.0</td>\n",
       "      <td>56.0</td>\n",
       "      <td>86.33</td>\n",
       "      <td>147.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>98.0</td>\n",
       "      <td>36.3</td>\n",
       "      <td>49.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>110.0</td>\n",
       "      <td>78.0</td>\n",
       "      <td>64.0</td>\n",
       "      <td>44.0</td>\n",
       "      <td>71.00</td>\n",
       "      <td>107.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>93.0</td>\n",
       "      <td>36.1</td>\n",
       "      <td>78.0</td>\n",
       "      <td>15.0</td>\n",
       "      <td>110.0</td>\n",
       "      <td>116.0</td>\n",
       "      <td>108.0</td>\n",
       "      <td>66.0</td>\n",
       "      <td>95.67</td>\n",
       "      <td>159.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>98.0</td>\n",
       "      <td>37.4</td>\n",
       "      <td>62.0</td>\n",
       "      <td>15.0</td>\n",
       "      <td>110.0</td>\n",
       "      <td>103.0</td>\n",
       "      <td>83.0</td>\n",
       "      <td>56.0</td>\n",
       "      <td>83.330</td>\n",
       "      <td>143.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>97.5</td>\n",
       "      <td>36.85</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>25.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.9</td>\n",
       "      <td>1.0</td>\n",
       "      <td>23.0</td>\n",
       "      <td>33.0</td>\n",
       "      <td>5.2</td>\n",
       "      <td>1.1</td>\n",
       "      <td>2.9</td>\n",
       "      <td>136.0</td>\n",
       "      <td>35.0</td>\n",
       "      <td>374.0</td>\n",
       "      <td>103.0</td>\n",
       "      <td>111.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>20.0</td>\n",
       "      <td>88.7</td>\n",
       "      <td>7.43</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>25.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.2</td>\n",
       "      <td>0.70</td>\n",
       "      <td>20.0</td>\n",
       "      <td>25.0</td>\n",
       "      <td>5.2</td>\n",
       "      <td>1.1</td>\n",
       "      <td>1.9</td>\n",
       "      <td>131.0</td>\n",
       "      <td>33.0</td>\n",
       "      <td>120.0</td>\n",
       "      <td>118.0</td>\n",
       "      <td>155.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>15.8</td>\n",
       "      <td>87.5</td>\n",
       "      <td>7.45</td>\n",
       "      <td>437.0</td>\n",
       "      <td>1260.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>8.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3996</th>\n",
       "      <td>152851</td>\n",
       "      <td>28</td>\n",
       "      <td>14</td>\n",
       "      <td>10</td>\n",
       "      <td>39</td>\n",
       "      <td>0</td>\n",
       "      <td>90.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>177.8</td>\n",
       "      <td>87.00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>50.0</td>\n",
       "      <td>13.0</td>\n",
       "      <td>120.0</td>\n",
       "      <td>97.0</td>\n",
       "      <td>72.0</td>\n",
       "      <td>32.0</td>\n",
       "      <td>50.33</td>\n",
       "      <td>87.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>93.0</td>\n",
       "      <td>37.2</td>\n",
       "      <td>64.0</td>\n",
       "      <td>10.0</td>\n",
       "      <td>90.0</td>\n",
       "      <td>91.0</td>\n",
       "      <td>96.0</td>\n",
       "      <td>43.0</td>\n",
       "      <td>61.00</td>\n",
       "      <td>97.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>93.0</td>\n",
       "      <td>36.6</td>\n",
       "      <td>34.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>64.0</td>\n",
       "      <td>70.0</td>\n",
       "      <td>48.0</td>\n",
       "      <td>30.0</td>\n",
       "      <td>48.33</td>\n",
       "      <td>85.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>93.0</td>\n",
       "      <td>36.3</td>\n",
       "      <td>65.0</td>\n",
       "      <td>13.0</td>\n",
       "      <td>246.0</td>\n",
       "      <td>110.0</td>\n",
       "      <td>96.0</td>\n",
       "      <td>91.0</td>\n",
       "      <td>109.70</td>\n",
       "      <td>147.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>97.0</td>\n",
       "      <td>37.7</td>\n",
       "      <td>47.0</td>\n",
       "      <td>7.5</td>\n",
       "      <td>120.0</td>\n",
       "      <td>82.0</td>\n",
       "      <td>71.0</td>\n",
       "      <td>38.0</td>\n",
       "      <td>63.835</td>\n",
       "      <td>108.5</td>\n",
       "      <td>NaN</td>\n",
       "      <td>93.0</td>\n",
       "      <td>37.15</td>\n",
       "      <td>74.0</td>\n",
       "      <td>12.0</td>\n",
       "      <td>25.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>49.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2.3</td>\n",
       "      <td>NaN</td>\n",
       "      <td>11.0</td>\n",
       "      <td>34.2</td>\n",
       "      <td>3.0</td>\n",
       "      <td>5.8</td>\n",
       "      <td>1.4</td>\n",
       "      <td>141.0</td>\n",
       "      <td>31.0</td>\n",
       "      <td>69.0</td>\n",
       "      <td>332.0</td>\n",
       "      <td>112.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.09</td>\n",
       "      <td>27.4</td>\n",
       "      <td>87.0</td>\n",
       "      <td>7.13</td>\n",
       "      <td>74.0</td>\n",
       "      <td>12.0</td>\n",
       "      <td>25.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>60.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.9</td>\n",
       "      <td>NaN</td>\n",
       "      <td>18.0</td>\n",
       "      <td>31.5</td>\n",
       "      <td>3.8</td>\n",
       "      <td>1.3</td>\n",
       "      <td>2.3</td>\n",
       "      <td>140.0</td>\n",
       "      <td>30.0</td>\n",
       "      <td>62.0</td>\n",
       "      <td>191.0</td>\n",
       "      <td>147.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.12</td>\n",
       "      <td>21.4</td>\n",
       "      <td>87.4</td>\n",
       "      <td>7.41</td>\n",
       "      <td>131.0</td>\n",
       "      <td>2700.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>21.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3997</th>\n",
       "      <td>152858</td>\n",
       "      <td>12</td>\n",
       "      <td>4</td>\n",
       "      <td>19</td>\n",
       "      <td>334</td>\n",
       "      <td>0</td>\n",
       "      <td>70.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>152.4</td>\n",
       "      <td>74.00</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>61.0</td>\n",
       "      <td>14.0</td>\n",
       "      <td>103.0</td>\n",
       "      <td>65.0</td>\n",
       "      <td>86.0</td>\n",
       "      <td>68.0</td>\n",
       "      <td>94.67</td>\n",
       "      <td>148.0</td>\n",
       "      <td>15.0</td>\n",
       "      <td>98.0</td>\n",
       "      <td>37.3</td>\n",
       "      <td>67.0</td>\n",
       "      <td>15.0</td>\n",
       "      <td>141.0</td>\n",
       "      <td>97.0</td>\n",
       "      <td>91.0</td>\n",
       "      <td>56.0</td>\n",
       "      <td>78.67</td>\n",
       "      <td>124.0</td>\n",
       "      <td>18.0</td>\n",
       "      <td>98.0</td>\n",
       "      <td>36.2</td>\n",
       "      <td>54.0</td>\n",
       "      <td>14.0</td>\n",
       "      <td>103.0</td>\n",
       "      <td>65.0</td>\n",
       "      <td>70.0</td>\n",
       "      <td>47.0</td>\n",
       "      <td>71.67</td>\n",
       "      <td>118.0</td>\n",
       "      <td>11.0</td>\n",
       "      <td>98.0</td>\n",
       "      <td>36.0</td>\n",
       "      <td>77.0</td>\n",
       "      <td>15.0</td>\n",
       "      <td>141.0</td>\n",
       "      <td>97.0</td>\n",
       "      <td>93.0</td>\n",
       "      <td>105.0</td>\n",
       "      <td>119.30</td>\n",
       "      <td>163.0</td>\n",
       "      <td>31.0</td>\n",
       "      <td>98.0</td>\n",
       "      <td>37.3</td>\n",
       "      <td>60.5</td>\n",
       "      <td>15.0</td>\n",
       "      <td>120.0</td>\n",
       "      <td>72.0</td>\n",
       "      <td>82.5</td>\n",
       "      <td>67.0</td>\n",
       "      <td>90.670</td>\n",
       "      <td>143.0</td>\n",
       "      <td>17.0</td>\n",
       "      <td>98.0</td>\n",
       "      <td>36.60</td>\n",
       "      <td>88.0</td>\n",
       "      <td>11.0</td>\n",
       "      <td>21.0</td>\n",
       "      <td>2.8</td>\n",
       "      <td>28.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.5</td>\n",
       "      <td>NaN</td>\n",
       "      <td>14.0</td>\n",
       "      <td>32.1</td>\n",
       "      <td>4.2</td>\n",
       "      <td>1.4</td>\n",
       "      <td>1.9</td>\n",
       "      <td>141.0</td>\n",
       "      <td>30.0</td>\n",
       "      <td>108.0</td>\n",
       "      <td>227.0</td>\n",
       "      <td>129.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>14.8</td>\n",
       "      <td>78.1</td>\n",
       "      <td>7.35</td>\n",
       "      <td>88.0</td>\n",
       "      <td>11.0</td>\n",
       "      <td>21.0</td>\n",
       "      <td>2.8</td>\n",
       "      <td>32.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.6</td>\n",
       "      <td>NaN</td>\n",
       "      <td>16.0</td>\n",
       "      <td>31.1</td>\n",
       "      <td>3.3</td>\n",
       "      <td>1.4</td>\n",
       "      <td>1.8</td>\n",
       "      <td>142.0</td>\n",
       "      <td>34.0</td>\n",
       "      <td>127.0</td>\n",
       "      <td>160.0</td>\n",
       "      <td>126.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>15.1</td>\n",
       "      <td>79.3</td>\n",
       "      <td>7.35</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3998</th>\n",
       "      <td>152862</td>\n",
       "      <td>5</td>\n",
       "      <td>0</td>\n",
       "      <td>12</td>\n",
       "      <td>-1</td>\n",
       "      <td>0</td>\n",
       "      <td>49.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>57.00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>15.0</td>\n",
       "      <td>129.0</td>\n",
       "      <td>115.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>86.0</td>\n",
       "      <td>100.70</td>\n",
       "      <td>130.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>67.0</td>\n",
       "      <td>36.1</td>\n",
       "      <td>NaN</td>\n",
       "      <td>15.0</td>\n",
       "      <td>131.0</td>\n",
       "      <td>119.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>57.0</td>\n",
       "      <td>83.00</td>\n",
       "      <td>135.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>67.0</td>\n",
       "      <td>36.1</td>\n",
       "      <td>NaN</td>\n",
       "      <td>15.0</td>\n",
       "      <td>129.0</td>\n",
       "      <td>86.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>50.0</td>\n",
       "      <td>77.33</td>\n",
       "      <td>113.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>67.0</td>\n",
       "      <td>36.1</td>\n",
       "      <td>NaN</td>\n",
       "      <td>15.0</td>\n",
       "      <td>131.0</td>\n",
       "      <td>136.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>103.0</td>\n",
       "      <td>119.00</td>\n",
       "      <td>161.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>67.0</td>\n",
       "      <td>37.1</td>\n",
       "      <td>NaN</td>\n",
       "      <td>15.0</td>\n",
       "      <td>130.0</td>\n",
       "      <td>111.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>80.5</td>\n",
       "      <td>100.500</td>\n",
       "      <td>137.5</td>\n",
       "      <td>NaN</td>\n",
       "      <td>67.0</td>\n",
       "      <td>36.20</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>16.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.6</td>\n",
       "      <td>NaN</td>\n",
       "      <td>24.0</td>\n",
       "      <td>30.6</td>\n",
       "      <td>3.7</td>\n",
       "      <td>4.5</td>\n",
       "      <td>1.6</td>\n",
       "      <td>131.0</td>\n",
       "      <td>48.0</td>\n",
       "      <td>189.0</td>\n",
       "      <td>204.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>13.6</td>\n",
       "      <td>57.0</td>\n",
       "      <td>7.34</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>12.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.7</td>\n",
       "      <td>NaN</td>\n",
       "      <td>26.0</td>\n",
       "      <td>30.4</td>\n",
       "      <td>4.4</td>\n",
       "      <td>4.5</td>\n",
       "      <td>1.5</td>\n",
       "      <td>131.0</td>\n",
       "      <td>53.0</td>\n",
       "      <td>38.0</td>\n",
       "      <td>185.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>16.6</td>\n",
       "      <td>57.0</td>\n",
       "      <td>7.36</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3999</th>\n",
       "      <td>152864</td>\n",
       "      <td>18</td>\n",
       "      <td>8</td>\n",
       "      <td>14</td>\n",
       "      <td>88</td>\n",
       "      <td>0</td>\n",
       "      <td>82.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>44.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>169.0</td>\n",
       "      <td>59.0</td>\n",
       "      <td>76.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>98.0</td>\n",
       "      <td>37.6</td>\n",
       "      <td>40.0</td>\n",
       "      <td>15.0</td>\n",
       "      <td>161.0</td>\n",
       "      <td>63.0</td>\n",
       "      <td>68.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>98.0</td>\n",
       "      <td>36.2</td>\n",
       "      <td>36.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>78.0</td>\n",
       "      <td>58.0</td>\n",
       "      <td>48.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>98.0</td>\n",
       "      <td>35.6</td>\n",
       "      <td>61.0</td>\n",
       "      <td>15.0</td>\n",
       "      <td>169.0</td>\n",
       "      <td>96.0</td>\n",
       "      <td>95.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>98.0</td>\n",
       "      <td>37.7</td>\n",
       "      <td>46.0</td>\n",
       "      <td>12.5</td>\n",
       "      <td>161.0</td>\n",
       "      <td>75.0</td>\n",
       "      <td>69.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>98.0</td>\n",
       "      <td>37.00</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>19.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.9</td>\n",
       "      <td>1.0</td>\n",
       "      <td>20.0</td>\n",
       "      <td>30.3</td>\n",
       "      <td>5.2</td>\n",
       "      <td>7.5</td>\n",
       "      <td>3.5</td>\n",
       "      <td>134.0</td>\n",
       "      <td>39.0</td>\n",
       "      <td>198.0</td>\n",
       "      <td>211.0</td>\n",
       "      <td>145.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>17.3</td>\n",
       "      <td>NaN</td>\n",
       "      <td>7.33</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>18.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.9</td>\n",
       "      <td>0.40</td>\n",
       "      <td>21.0</td>\n",
       "      <td>30.6</td>\n",
       "      <td>5.0</td>\n",
       "      <td>1.3</td>\n",
       "      <td>2.2</td>\n",
       "      <td>134.0</td>\n",
       "      <td>42.0</td>\n",
       "      <td>94.0</td>\n",
       "      <td>137.0</td>\n",
       "      <td>124.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>10.3</td>\n",
       "      <td>NaN</td>\n",
       "      <td>7.40</td>\n",
       "      <td>192.0</td>\n",
       "      <td>1320.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>9.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>4000 rows × 120 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "      recordid  SAPS-I  SOFA  Length_of_stay  Survival  In-hospital_death  \\\n",
       "0       142675      27    14               9         7                  1   \n",
       "1       142676      12     1              31       468                  0   \n",
       "2       142680      12     7              17        16                  1   \n",
       "3       142683      19    15              17        -1                  0   \n",
       "4       142688       3     0               9        -1                  0   \n",
       "...        ...     ...   ...             ...       ...                ...   \n",
       "3995    152849      21    13               6       752                  0   \n",
       "3996    152851      28    14              10        39                  0   \n",
       "3997    152858      12     4              19       334                  0   \n",
       "3998    152862       5     0              12        -1                  0   \n",
       "3999    152864      18     8              14        88                  0   \n",
       "\n",
       "       Age  Gender  Height  Weight  CCU  CSRU  SICU  DiasABP_first  GCS_first  \\\n",
       "0     70.0     1.0   175.3   85.00    0     1     0           60.0        3.0   \n",
       "1     57.0     0.0     NaN     NaN    0     0     0            NaN       15.0   \n",
       "2     60.0     1.0   182.9   78.80    0     0     0           49.0        9.0   \n",
       "3     64.0     1.0   180.3  145.20    0     1     0           76.0        3.0   \n",
       "4     51.0     1.0   172.7   90.00    0     0     1           55.0       15.0   \n",
       "...    ...     ...     ...     ...  ...   ...   ...            ...        ...   \n",
       "3995  78.0     1.0   180.3   83.18    0     1     0           58.0        3.0   \n",
       "3996  90.0     1.0   177.8   87.00    0     0     0           50.0       13.0   \n",
       "3997  70.0     0.0   152.4   74.00    0     1     0           61.0       14.0   \n",
       "3998  49.0     0.0     NaN   57.00    0     0     0            NaN       15.0   \n",
       "3999  82.0     0.0     NaN     NaN    0     1     0           44.0        3.0   \n",
       "\n",
       "      Glucose_first  HR_first  MAP_first  NIDiasABP_first  NIMAP_first  \\\n",
       "0             194.0      76.0       76.0              NaN          NaN   \n",
       "1             229.0      86.0        NaN             51.0        77.33   \n",
       "2             127.0      90.0       61.0             74.0        89.33   \n",
       "3             191.0      87.0       75.0              NaN          NaN   \n",
       "4             165.0      91.0       74.0             63.0        76.33   \n",
       "...             ...       ...        ...              ...          ...   \n",
       "3995          110.0     110.0       77.0             51.0        79.33   \n",
       "3996          120.0      97.0       72.0             32.0        50.33   \n",
       "3997          103.0      65.0       86.0             68.0        94.67   \n",
       "3998          129.0     115.0        NaN             86.0       100.70   \n",
       "3999          169.0      59.0       76.0              NaN          NaN   \n",
       "\n",
       "      NISysABP_first  RespRate_first  SaO2_first  Temp_first  DiasABP_last  \\\n",
       "0                NaN             NaN        98.0        35.7          43.0   \n",
       "1              130.0            22.0         NaN        35.3           NaN   \n",
       "2              120.0             NaN        93.0        38.0          55.0   \n",
       "3                NaN             NaN        95.0        37.4          59.0   \n",
       "4              103.0            24.0         NaN        36.8          68.0   \n",
       "...              ...             ...         ...         ...           ...   \n",
       "3995           136.0             NaN        98.0        36.8          71.0   \n",
       "3996            87.0             NaN        93.0        37.2          64.0   \n",
       "3997           148.0            15.0        98.0        37.3          67.0   \n",
       "3998           130.0             NaN        67.0        36.1           NaN   \n",
       "3999             NaN             NaN        98.0        37.6          40.0   \n",
       "\n",
       "      GCS_last  Glucose_last  HR_last  MAP_last  NIDiasABP_last  NIMAP_last  \\\n",
       "0          3.0         125.0     22.0      61.0             NaN         NaN   \n",
       "1         15.0         198.0     92.0       NaN            77.0      100.70   \n",
       "2          7.0         135.0     98.0      70.0            55.0       74.33   \n",
       "3          3.0         166.0     68.0      84.0             NaN         NaN   \n",
       "4         14.0         130.0    113.0      81.0            66.0       86.33   \n",
       "...        ...           ...      ...       ...             ...         ...   \n",
       "3995      15.0         110.0    103.0      97.0            56.0       86.33   \n",
       "3996      10.0          90.0     91.0      96.0            43.0       61.00   \n",
       "3997      15.0         141.0     97.0      91.0            56.0       78.67   \n",
       "3998      15.0         131.0    119.0       NaN            57.0       83.00   \n",
       "3999      15.0         161.0     63.0      68.0             NaN         NaN   \n",
       "\n",
       "      NISysABP_last  RespRate_last  SaO2_last  Temp_last  DiasABP_lowest  \\\n",
       "0               NaN            NaN       98.0       37.6            40.0   \n",
       "1             148.0           14.0        NaN       37.6             NaN   \n",
       "2             113.0            NaN       93.0       38.4            45.0   \n",
       "3               NaN            NaN       97.0       37.0            40.0   \n",
       "4             127.0           20.0        NaN       38.7            53.0   \n",
       "...             ...            ...        ...        ...             ...   \n",
       "3995          147.0            NaN       98.0       36.3            49.0   \n",
       "3996           97.0            NaN       93.0       36.6            34.0   \n",
       "3997          124.0           18.0       98.0       36.2            54.0   \n",
       "3998          135.0            NaN       67.0       36.1             NaN   \n",
       "3999            NaN            NaN       98.0       36.2            36.0   \n",
       "\n",
       "      GCS_lowest  Glucose_lowest  HR_lowest  MAP_lowest  NIDiasABP_lowest  \\\n",
       "0            3.0           125.0       10.0        41.0               NaN   \n",
       "1           14.0            79.0       72.0         NaN              37.0   \n",
       "2            7.0           127.0       53.0        61.0              41.0   \n",
       "3            3.0           166.0       62.0        29.0               NaN   \n",
       "4           14.0           130.0       88.0        69.0              50.0   \n",
       "...          ...             ...        ...         ...               ...   \n",
       "3995         3.0           110.0       78.0        64.0              44.0   \n",
       "3996         3.0            64.0       70.0        48.0              30.0   \n",
       "3997        14.0           103.0       65.0        70.0              47.0   \n",
       "3998        15.0           129.0       86.0         NaN              50.0   \n",
       "3999         3.0            78.0       58.0        48.0               NaN   \n",
       "\n",
       "      NIMAP_lowest  NISysABP_lowest  RespRate_lowest  SaO2_lowest  \\\n",
       "0              NaN              NaN              NaN         87.0   \n",
       "1            63.00            110.0             11.0          NaN   \n",
       "2            58.00             90.0              NaN         93.0   \n",
       "3              NaN              NaN              NaN         89.0   \n",
       "4            71.00             99.0             13.0          NaN   \n",
       "...            ...              ...              ...          ...   \n",
       "3995         71.00            107.0              NaN         93.0   \n",
       "3996         48.33             85.0              NaN         93.0   \n",
       "3997         71.67            118.0             11.0         98.0   \n",
       "3998         77.33            113.0              NaN         67.0   \n",
       "3999           NaN              NaN              NaN         98.0   \n",
       "\n",
       "      Temp_lowest  DiasABP_highest  GCS_highest  Glucose_highest  HR_highest  \\\n",
       "0            33.4             98.0          3.0            194.0        95.0   \n",
       "1            35.3              NaN         15.0            273.0       109.0   \n",
       "2            36.2             92.0         10.0            153.0       115.0   \n",
       "3            36.6             90.0          3.0            191.0        87.0   \n",
       "4            36.8             74.0         15.0            165.0       115.0   \n",
       "...           ...              ...          ...              ...         ...   \n",
       "3995         36.1             78.0         15.0            110.0       116.0   \n",
       "3996         36.3             65.0         13.0            246.0       110.0   \n",
       "3997         36.0             77.0         15.0            141.0        97.0   \n",
       "3998         36.1              NaN         15.0            131.0       136.0   \n",
       "3999         35.6             61.0         15.0            169.0        96.0   \n",
       "\n",
       "      MAP_highest  NIDiasABP_highest  NIMAP_highest  NISysABP_highest  \\\n",
       "0           108.0                NaN            NaN               NaN   \n",
       "1             NaN               79.0         100.70             148.0   \n",
       "2           246.0               74.0          95.33             148.0   \n",
       "3           113.0                NaN            NaN               NaN   \n",
       "4            89.0               68.0          86.33             127.0   \n",
       "...           ...                ...            ...               ...   \n",
       "3995        108.0               66.0          95.67             159.0   \n",
       "3996         96.0               91.0         109.70             147.0   \n",
       "3997         93.0              105.0         119.30             163.0   \n",
       "3998          NaN              103.0         119.00             161.0   \n",
       "3999         95.0                NaN            NaN               NaN   \n",
       "\n",
       "      RespRate_highest  SaO2_highest  Temp_highest  DiasABP_median  \\\n",
       "0                  NaN          98.0          37.8            56.0   \n",
       "1                 30.0           NaN          37.9             NaN   \n",
       "2                  NaN          93.0          39.7            56.5   \n",
       "3                  NaN          97.0          38.0            58.0   \n",
       "4                 27.0           NaN          38.7            58.0   \n",
       "...                ...           ...           ...             ...   \n",
       "3995               NaN          98.0          37.4            62.0   \n",
       "3996               NaN          97.0          37.7            47.0   \n",
       "3997              31.0          98.0          37.3            60.5   \n",
       "3998               NaN          67.0          37.1             NaN   \n",
       "3999               NaN          98.0          37.7            46.0   \n",
       "\n",
       "      GCS_median  Glucose_median  HR_median  MAP_median  NIDiasABP_median  \\\n",
       "0            3.0           159.5       62.0        72.0               NaN   \n",
       "1           15.0           192.5       89.0         NaN              53.0   \n",
       "2            8.0           135.0       87.0        70.0              56.0   \n",
       "3            3.0           178.5       74.0        77.0               NaN   \n",
       "4           15.0           147.5      100.5        74.5              60.0   \n",
       "...          ...             ...        ...         ...               ...   \n",
       "3995        15.0           110.0      103.0        83.0              56.0   \n",
       "3996         7.5           120.0       82.0        71.0              38.0   \n",
       "3997        15.0           120.0       72.0        82.5              67.0   \n",
       "3998        15.0           130.0      111.0         NaN              80.5   \n",
       "3999        12.5           161.0       75.0        69.0               NaN   \n",
       "\n",
       "      NIMAP_median  NISysABP_median  RespRate_median  SaO2_median  \\\n",
       "0              NaN              NaN              NaN         98.0   \n",
       "1           81.670            130.0             17.0          NaN   \n",
       "2           75.665            113.5              NaN         93.0   \n",
       "3              NaN              NaN              NaN         92.5   \n",
       "4           73.500            106.5             20.0          NaN   \n",
       "...            ...              ...              ...          ...   \n",
       "3995        83.330            143.0              NaN         97.5   \n",
       "3996        63.835            108.5              NaN         93.0   \n",
       "3997        90.670            143.0             17.0         98.0   \n",
       "3998       100.500            137.5              NaN         67.0   \n",
       "3999           NaN              NaN              NaN         98.0   \n",
       "\n",
       "      Temp_median  ALP_first  ALT_first  AST_first  Albumin_first  BUN_first  \\\n",
       "0           36.65       24.0       45.0      106.0            1.7       13.0   \n",
       "1           36.70        NaN        NaN        NaN            NaN       35.0   \n",
       "2           38.00       77.0       44.0       26.0            NaN       21.0   \n",
       "3           37.20       39.0      160.0      186.0            1.6       11.0   \n",
       "4           37.75        NaN        NaN        NaN            NaN       19.0   \n",
       "...           ...        ...        ...        ...            ...        ...   \n",
       "3995        36.85        NaN        NaN        NaN            NaN       25.0   \n",
       "3996        37.15       74.0       12.0       25.0            NaN       49.0   \n",
       "3997        36.60       88.0       11.0       21.0            2.8       28.0   \n",
       "3998        36.20        NaN        NaN        NaN            NaN       16.0   \n",
       "3999        37.00        NaN        NaN        NaN            NaN       19.0   \n",
       "\n",
       "      Bilirubin_first  Cholesterol_first  Creatinine_first  FiO2_first  \\\n",
       "0                 0.9                NaN               0.8         1.0   \n",
       "1                 NaN                NaN               0.7         NaN   \n",
       "2                 0.7                NaN               0.8         0.5   \n",
       "3                 0.7                NaN               0.9         1.0   \n",
       "4                 NaN                NaN               0.9         1.0   \n",
       "...               ...                ...               ...         ...   \n",
       "3995              NaN                NaN               0.9         1.0   \n",
       "3996              1.0                NaN               2.3         NaN   \n",
       "3997              NaN                NaN               1.5         NaN   \n",
       "3998              NaN                NaN               0.6         NaN   \n",
       "3999              NaN                NaN               0.9         1.0   \n",
       "\n",
       "      HCO3_first  HCT_first  K_first  Lactate_first  Mg_first  Na_first  \\\n",
       "0           22.0       33.7      4.3           14.4      21.0     142.0   \n",
       "1            7.0       43.8      3.1            NaN       2.0     148.0   \n",
       "2           30.0       38.8      4.3            NaN       1.9     138.0   \n",
       "3           18.0       33.9      4.9            1.8       1.6     138.0   \n",
       "4           26.0       38.4      5.0            NaN       1.9     141.0   \n",
       "...          ...        ...      ...            ...       ...       ...   \n",
       "3995        23.0       33.0      5.2            1.1       2.9     136.0   \n",
       "3996        11.0       34.2      3.0            5.8       1.4     141.0   \n",
       "3997        14.0       32.1      4.2            1.4       1.9     141.0   \n",
       "3998        24.0       30.6      3.7            4.5       1.6     131.0   \n",
       "3999        20.0       30.3      5.2            7.5       3.5     134.0   \n",
       "\n",
       "      PaCO2_first  PaO2_first  Platelets_first  SysABP_first  TroponinI_first  \\\n",
       "0            32.0       378.0            185.0         104.0              NaN   \n",
       "1             NaN         NaN            616.0           NaN              NaN   \n",
       "2            65.0        71.0            233.0          91.0              NaN   \n",
       "3            45.0       294.0            104.0         137.0              NaN   \n",
       "4            50.0        67.0            215.0         126.0              NaN   \n",
       "...           ...         ...              ...           ...              ...   \n",
       "3995         35.0       374.0            103.0         111.0              NaN   \n",
       "3996         31.0        69.0            332.0         112.0              NaN   \n",
       "3997         30.0       108.0            227.0         129.0              1.0   \n",
       "3998         48.0       189.0            204.0           NaN              NaN   \n",
       "3999         39.0       198.0            211.0         145.0              NaN   \n",
       "\n",
       "      TroponinT_first  WBC_first  Weight_first  pH_first  ALP_last  ALT_last  \\\n",
       "0                 NaN        7.9           NaN      7.50      23.0     137.0   \n",
       "1                 NaN       22.4           NaN       NaN       NaN       NaN   \n",
       "2                 NaN       14.5          79.4      7.33      77.0      53.0   \n",
       "3                 NaN       18.3           NaN      7.33      49.0    1593.0   \n",
       "4                 NaN       16.0           NaN      7.37       NaN       NaN   \n",
       "...               ...        ...           ...       ...       ...       ...   \n",
       "3995              NaN       20.0          88.7      7.43       NaN       NaN   \n",
       "3996             0.09       27.4          87.0      7.13      74.0      12.0   \n",
       "3997              NaN       14.8          78.1      7.35      88.0      11.0   \n",
       "3998              NaN       13.6          57.0      7.34       NaN       NaN   \n",
       "3999              NaN       17.3           NaN      7.33       NaN       NaN   \n",
       "\n",
       "      AST_last  Albumin_last  BUN_last  Bilirubin_last  Cholesterol_last  \\\n",
       "0       1106.0           2.0      24.0             2.8               NaN   \n",
       "1          NaN           NaN      10.0             NaN               NaN   \n",
       "2         29.0           NaN      23.0             0.7               NaN   \n",
       "3       1933.0           3.0      19.0             1.8               NaN   \n",
       "4          NaN           NaN      19.0             NaN               NaN   \n",
       "...        ...           ...       ...             ...               ...   \n",
       "3995       NaN           NaN      25.0             NaN               NaN   \n",
       "3996      25.0           NaN      60.0             1.0               NaN   \n",
       "3997      21.0           2.8      32.0             NaN               NaN   \n",
       "3998       NaN           NaN      12.0             NaN               NaN   \n",
       "3999       NaN           NaN      18.0             NaN               NaN   \n",
       "\n",
       "      Creatinine_last  FiO2_last  HCO3_last  HCT_last  K_last  Lactate_last  \\\n",
       "0                 2.1       1.00       23.0      29.5     4.4           3.9   \n",
       "1                 0.4        NaN       21.0      32.5     3.2           NaN   \n",
       "2                 0.7       0.70       31.0      37.7     3.5           NaN   \n",
       "3                 1.4       0.65       24.0      28.3     5.0           4.5   \n",
       "4                 0.7       0.50       29.0      29.4     3.7           NaN   \n",
       "...               ...        ...        ...       ...     ...           ...   \n",
       "3995              1.2       0.70       20.0      25.0     5.2           1.1   \n",
       "3996              1.9        NaN       18.0      31.5     3.8           1.3   \n",
       "3997              1.6        NaN       16.0      31.1     3.3           1.4   \n",
       "3998              0.7        NaN       26.0      30.4     4.4           4.5   \n",
       "3999              0.9       0.40       21.0      30.6     5.0           1.3   \n",
       "\n",
       "      Mg_last  Na_last  PaCO2_last  PaO2_last  Platelets_last  SysABP_last  \\\n",
       "0         1.3    147.0        37.0      117.0            41.0        105.0   \n",
       "1         1.8    134.0         NaN        NaN           517.0          NaN   \n",
       "2         1.8    136.0        53.0       76.0           207.0        104.0   \n",
       "3         2.4    137.0        42.0      107.0            79.0        140.0   \n",
       "4         1.7    141.0        50.0       67.0           164.0        107.0   \n",
       "...       ...      ...         ...        ...             ...          ...   \n",
       "3995      1.9    131.0        33.0      120.0           118.0        155.0   \n",
       "3996      2.3    140.0        30.0       62.0           191.0        147.0   \n",
       "3997      1.8    142.0        34.0      127.0           160.0        126.0   \n",
       "3998      1.5    131.0        53.0       38.0           185.0          NaN   \n",
       "3999      2.2    134.0        42.0       94.0           137.0        124.0   \n",
       "\n",
       "      TroponinI_last  TroponinT_last  WBC_last  Weight_last  pH_last  \\\n",
       "0                NaN             NaN       6.1          NaN     7.38   \n",
       "1                NaN             NaN      15.1          NaN      NaN   \n",
       "2                NaN             NaN      17.6         79.5     7.45   \n",
       "3                NaN             NaN      18.3          NaN     7.36   \n",
       "4                NaN             NaN      12.5          NaN     7.37   \n",
       "...              ...             ...       ...          ...      ...   \n",
       "3995             NaN             NaN      15.8         87.5     7.45   \n",
       "3996             NaN            0.12      21.4         87.4     7.41   \n",
       "3997             1.0             NaN      15.1         79.3     7.35   \n",
       "3998             NaN             NaN      16.6         57.0     7.36   \n",
       "3999             NaN             NaN      10.3          NaN     7.40   \n",
       "\n",
       "      MechVentStartTime  MechVentDuration  MechVentLast8Hour  UrineOutputSum  \n",
       "0                  99.0            2559.0                1.0            13.0  \n",
       "1                   NaN               NaN                NaN             NaN  \n",
       "2                 207.0            2570.0                1.0            20.0  \n",
       "3                 374.0            2420.0                1.0            14.0  \n",
       "4                   NaN               NaN                NaN             NaN  \n",
       "...                 ...               ...                ...             ...  \n",
       "3995              437.0            1260.0                0.0             8.0  \n",
       "3996              131.0            2700.0                1.0            21.0  \n",
       "3997                NaN               NaN                NaN             NaN  \n",
       "3998                NaN               NaN                NaN             NaN  \n",
       "3999              192.0            1320.0                0.0             9.0  \n",
       "\n",
       "[4000 rows x 120 columns]"
      ]
     },
     "execution_count": 94,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test = pd.read_csv('../challenge2012/data/PhysionetChallenge2012-set-b.csv')\n",
    "test"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "21d5be10-f40e-4ed6-ac60-288e56ebb002",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "source": [
    "We have to make sure our data is structured simiarly "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 95,
   "id": "9ba04e40-c34c-4f2e-890b-91111000e0fb",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "X_test = test[['Age', 'Gender', 'Weight', 'CCU', 'CSRU', 'SICU', 'GCS_first',\n",
    "       'Glucose_first', 'HR_first', 'NIDiasABP_first', 'NIMAP_first',\n",
    "       'NISysABP_first', 'Temp_first', 'BUN_first', 'Creatinine_first',\n",
    "       'HCO3_first', 'HCT_first', 'K_first', 'Mg_first', 'Na_first',\n",
    "       'PaCO2_first', 'PaO2_first', 'Platelets_first', 'WBC_first',\n",
    "       'pH_first']]\n",
    "\n",
    "# we will drop the small handful of patients without a gender\n",
    "X_test = X_test.dropna(subset=['Gender'])\n",
    "\n",
    "## format the y_test variable\n",
    "y_test = test.copy()\n",
    "\n",
    "# merge with the index for train to remove the handful of patients without gender\n",
    "y_test = pd.merge(y_test['In-hospital_death'], X_test,\n",
    "                  left_index=True, \n",
    "                  right_index=True)\n",
    "\n",
    "# select only the outcome of interest\n",
    "y_test = y_test['In-hospital_death']"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7505925f-0580-4641-862e-622b3cba2993",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "### Final Test-set Evaluation Metrics"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "17adc363-b250-426b-8466-421f0b1a5b8d",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "source": [
    "#### Confusion Matrix (Test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 96,
   "id": "19daf447-0979-4968-92e3-f9be3f1494fb",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "classification report (Test):\n",
      "                precision    recall  f1-score   support\n",
      "\n",
      "           0      0.930     0.673     0.781      3427\n",
      "           1      0.261     0.695     0.379       568\n",
      "\n",
      "    accuracy                          0.676      3995\n",
      "   macro avg      0.595     0.684     0.580      3995\n",
      "weighted avg      0.835     0.676     0.724      3995\n",
      "\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<sklearn.metrics._plot.confusion_matrix.ConfusionMatrixDisplay at 0x23b8c11c190>"
      ]
     },
     "execution_count": 96,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAgQAAAGwCAYAAADWsX1oAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8hTgPZAAAACXBIWXMAAA9hAAAPYQGoP6dpAABGqklEQVR4nO3de1xUdd4H8M9wGy7CyIAzA4WIeSNBRTBESzENpfCSW2q4pBthrbf1UbM1t7RtlfLZ1NLVdV1XTTHtadXcMgzvmeIFxVJZ8oIKyQgqDHIfZs7zB3lqBEeGmQHhfN77Oq+Xc87vd+Y75Dpfvr/LkQmCIICIiIgkzaG5AyAiIqLmx4SAiIiImBAQEREREwIiIiICEwIiIiICEwIiIiICEwIiIiIC4NTcAVjDaDTi+vXr8PT0hEwma+5wiIjIQoIg4M6dO/D394eDg/1+R62srER1dbXV93FxcYGrq6sNInr4tOiE4Pr16wgICGjuMIiIyEq5ubl49NFH7XLvyspKBAW2gbbAYPW9NBoNcnJyWmVS0KITAk9PTwDA1VMd4NWGox/UOoWvf7W5QyCyG2NVJa58+Gfx33N7qK6uhrbAgKsZHeDl2fjvipI7RgSGX0F1dTUTgofN3WECrzYOVv1HJnqYObbCf3iI7tUUw75tPGVo49n49zGidQ9Nt+iEgIiIqKEMghEGK57eYxCMtgvmIcSEgIiIJMEIAUY0PiOwpm9LwDo7ERERMSEgIiJpMNrgf5ZITk5Gnz594OnpCZVKhVGjRiE7O1u8rtfr8eabbyI0NBQeHh7w9/fHyy+/jOvXr5vcJzo6GjKZzOQYN26cSZuioiIkJCRAoVBAoVAgISEBxcXFFsXLhICIiCTBIAhWH5Y4ePAgpkyZgvT0dKSlpaGmpgYxMTEoKysDAJSXl+PUqVN4++23cerUKWzbtg0//vgjRowYUedeSUlJyM/PF4/Vq1ebXI+Pj0dmZiZSU1ORmpqKzMxMJCQkWBQv5xAQERHZQWpqqsnrdevWQaVSISMjAwMGDIBCoUBaWppJm+XLl+OJJ57AtWvX0L59e/G8u7s7NBpNve+TlZWF1NRUpKenIzIyEgCwZs0aREVFITs7G127dm1QvKwQEBGRJNydVGjNAQAlJSUmR1VVVYPeX6fTAQCUSqXZNjKZDG3btjU5n5KSAl9fX3Tv3h2zZ8/GnTt3xGtHjx6FQqEQkwEA6Nu3LxQKBY4cOdLQHw8rBEREJA1GCDDYYJXBvTvkzp8/HwsWLDDbVxAEzJw5E08++SRCQkLqbVNZWYk//vGPiI+Ph5eXl3h+/PjxCAoKgkajwdmzZzF37lycOXNGrC5otVqoVKo691OpVNBqtQ3+fEwIiIiILJCbm2vyhS2Xyx/YZ+rUqfj+++9x+PDheq/r9XqMGzcORqMRK1euNLmWlJQk/jkkJASdO3dGREQETp06hd69ewOof2MnQRAs2vCJCQEREUmCrfYh8PLyMkkIHmTatGnYuXMnDh06VO/zGvR6PcaMGYOcnBzs27fvgffu3bs3nJ2dceHCBfTu3RsajQY3btyo066wsBBqtbrBcXIOARERSUJTrzIQBAFTp07Ftm3bsG/fPgQFBdVpczcZuHDhAvbs2QMfH58H3vfcuXPQ6/Xw8/MDAERFRUGn0+H48eNim2PHjkGn06Ffv34NjpcVAiIiIjuYMmUKNm/ejC+++AKenp7ieL5CoYCbmxtqamrwwgsv4NSpU/jyyy9hMBjENkqlEi4uLrh06RJSUlLw7LPPwtfXF+fPn8esWbMQFhaG/v37AwCCg4MxbNgwJCUlicsRJ02ahLi4uAavMABYISAiIokw2uCwxKpVq6DT6RAdHQ0/Pz/x2Lp1KwAgLy8PO3fuRF5eHnr16mXS5u7qABcXF+zduxdDhw5F165dMX36dMTExGDPnj1wdHQU3yslJQWhoaGIiYlBTEwMevTogY0bN1oULysEREQkCQYrVxlY2ld4wBBDhw4dHtgmICAABw8efOB7KZVKbNq0yaL47sWEgIiIJMEgwMqnHdoulocRhwyIiIiIFQIiIpKGxswDuLd/a8aEgIiIJMEIGQxo+EY99fVvzThkQERERKwQEBGRNBiF2sOa/q0ZEwIiIpIEg5VDBtb0bQk4ZEBERESsEBARkTSwQmAeEwIiIpIEoyCDUbBilYEVfVsCDhkQERERKwRERCQNHDIwjwkBERFJggEOMFhRGDfYMJaHERMCIiKSBMHKOQQC5xAQERFRa8cKARERSQLnEJjHhICIiCTBIDjAIFgxh6CVb13MIQMiIiJihYCIiKTBCBmMVvwebETrLhEwISAiIkngHALzOGRARERErBAQEZE0WD+pkEMGRERELV7tHAIrHm7EIQMiIiJq7VghICIiSTBa+SwDrjIgIiJqBTiHwDwmBEREJAlGOHAfAjM4h4CIiIhYISAiImkwCDIYrHiEsTV9WwImBEREJAkGKycVGjhkQERERK0dKwRERCQJRsEBRitWGRi5yoCIiKjl45CBeRwyICIisoPk5GT06dMHnp6eUKlUGDVqFLKzs03aCIKABQsWwN/fH25uboiOjsa5c+dM2lRVVWHatGnw9fWFh4cHRowYgby8PJM2RUVFSEhIgEKhgEKhQEJCAoqLiy2KlwkBERFJghG/rDRozGG08P0OHjyIKVOmID09HWlpaaipqUFMTAzKysrENosXL8aSJUuwYsUKnDhxAhqNBs888wzu3LkjtpkxYwa2b9+OLVu24PDhwygtLUVcXBwMBoPYJj4+HpmZmUhNTUVqaioyMzORkJBgUbwcMiAiIkmwfmOi2r4lJSUm5+VyOeRyeZ32qampJq/XrVsHlUqFjIwMDBgwAIIgYNmyZZg3bx5Gjx4NANiwYQPUajU2b96M1157DTqdDmvXrsXGjRsxZMgQAMCmTZsQEBCAPXv2YOjQocjKykJqairS09MRGRkJAFizZg2ioqKQnZ2Nrl27NujzsUJARERkgYCAALE0r1AokJyc3KB+Op0OAKBUKgEAOTk50Gq1iImJEdvI5XIMHDgQR44cAQBkZGRAr9ebtPH390dISIjY5ujRo1AoFGIyAAB9+/aFQqEQ2zQEKwRERCQJ1j/LoLZvbm4uvLy8xPP1VQfuJQgCZs6ciSeffBIhISEAAK1WCwBQq9UmbdVqNa5evSq2cXFxgbe3d502d/trtVqoVKo676lSqcQ2DcGEgIiIJMEIGYxo/G6Dd/t6eXmZJAQNMXXqVHz//fc4fPhwnWsymWlMgiDUOXeve9vU174h9/k1DhkQEZEk3K0QWHM0xrRp07Bz507s378fjz76qHheo9EAQJ3f4gsKCsSqgUajQXV1NYqKisy2uXHjRp33LSwsrFN9MIcJARERkR0IgoCpU6di27Zt2LdvH4KCgkyuBwUFQaPRIC0tTTxXXV2NgwcPol+/fgCA8PBwODs7m7TJz8/H2bNnxTZRUVHQ6XQ4fvy42ObYsWPQ6XRim4bgkAEREUmC9RsTWdZ3ypQp2Lx5M7744gt4enqKlQCFQgE3NzfIZDLMmDEDixYtQufOndG5c2csWrQI7u7uiI+PF9smJiZi1qxZ8PHxgVKpxOzZsxEaGiquOggODsawYcOQlJSE1atXAwAmTZqEuLi4Bq8wAJgQEBGRRBgFGYxWPLHQ0r6rVq0CAERHR5ucX7duHSZOnAgAmDNnDioqKjB58mQUFRUhMjIS33zzDTw9PcX2S5cuhZOTE8aMGYOKigoMHjwY69evh6Ojo9gmJSUF06dPF1cjjBgxAitWrLAoXpkgtNzNmUtKSqBQKFD0Y0d4eXL0g1qnbmsmN3cIRHZjqKzE5UVvQafTWTxRr6HuflcsPvEU3No0/vfgitIazOnzrV1jbU6sEBARkSQYrRwysGZTo5aACQEREUmC9U87bN0JQev+dERERNQgrBAQEZEkGCCDwYqNiazp2xIwISAiIkngkIF5rfvTERERUYOwQkBERJJggHVlf4PtQnkoMSEgIiJJ4JCBeUwIiIhIEmz1+OPWqnV/OiIiImoQVgiIiEgSBMhgtGIOgcBlh0RERC0fhwzMa92fjoiIiBqEFQIiIpKEpn78cUvDhICIiCTBYOXTDq3p2xK07k9HREREDcIKARERSQKHDMxjQkBERJJghAOMVhTGrenbErTuT0dEREQNwgoBERFJgkGQwWBF2d+avi0BEwIiIpIEziEwjwkBERFJgmDl0w4F7lRIRERErR0rBEREJAkGyGCw4gFF1vRtCZgQEBGRJBgF6+YBGAUbBvMQ4pABERERsUIgNVuWq/DdrrbIvSiHi6sRj0eUI3HedQR0qhLbbPyrBge+aIvC685wdhHQKbQCv/tjPrr1LhfbVFfJsObP/jiwwxtVlTKEPVmKqcl5aOevBwCcOdIGc17oVG8MH+/KRtdeFfb9oCRZEZrrSOyRie6+hVB5lGPKN8Ow92qQeP2ZDpcxNvg8uvsWwtu1EqP+/SL+e9tXvK6QV2Ja+An0fyQXmjZlKKp0xd4rQfjoZB+U6uViOy+XKszrdxhPB14BAOy72gF/OfIk7lT/0oYeLkYrJxVa07claN2fjur4/mgbDJ94E8u+vIDkLZdgMABvvfQYKst/+avwSMdKTFmYh9X7svHhjovQBFRj7kuPofiWo9jm7/MfwZFUBeauuoIlOy6iotwB77zcEQZD7fXHI8rwaeZZk2NY/C2oA6rQpSeTAbIfNyc9/nvbB+8deeq+109pNfjweGS911XuZVC5l2HxsX4Y8fkYzD04CE8FXMPCAQdM2v316TQE+9xE0tfPIenr5xDscxOLo/fa+uOQDRkhs/pozZo9IVi5ciWCgoLg6uqK8PBwfPvtt80dUqu2aPNlxIy9jQ5dK/FY90rMWnoNBT+54ML3bmKbp0cXo/eAUvgFVqND10pMWvATyu84Iud8bZuyEgfs/lSJpHeuo/eAUnQKrcCby6/iyn9dcfpbTwCAs4sApapGPLy8a5D+jReGjrsNWev+/xQ1s2/zAvHRyUikXelY7/WdF7ti5ekIHP3p0XqvXyjywfQ9w7D/Wgfk3lHg2PVHsfREJAYFXoGjzAgA6Ni2CAMCcvGnQ9HILNAgs0CDt78diEGBVxGkKLLbZyOyp2ZNCLZu3YoZM2Zg3rx5OH36NJ566inExsbi2rVrzRmWpJSV1P7W79nWUO91fbUMuzb5wMPLgI6P1/5mf+F7d9ToHRA+8I7YzkdTg8BulTh/wqPe+xz9RoGS2054ZsxtG38CIvvzdKlCabULDD+XjHuptCipcsH3hWqxzZkCDUqqXBCmvtFcYdID3N2p0JqjNWvWhGDJkiVITEzEq6++iuDgYCxbtgwBAQFYtWpVc4YlGYIA/GPBI+j+RCk6dKs0uZae5oWRnUIxPKgHtq9ph+QtF6HwqU0abhc4wdnFWCeJ8PbVo6iw/mkpuz/1QXj0Hage0dvnwxDZSVt5JX4floGt/31cPNfOvRy3K93qtL1d6QZft/I65+nhcHcOgTVHa9Zsn666uhoZGRmIiYkxOR8TE4MjR47U26eqqgolJSUmBzXe3956BDlZbpi78mqda736l2JlWjaW7ryAiOg7WPhaBxTfND8HVRBkqG+IrfC6MzIOeGLoS7dsFTpRk/Bwrsbfh36FS8Xe+FtGhMk14T5L0Fr5yjRqxZotIbh58yYMBgPUarXJebVaDa1WW2+f5ORkKBQK8QgICGiKUFulv817BEe/UWDx5xfFlQG/5upuxCNB1QgOL8fMJblwdAJSP1UCAJSqGuirHXCn2NGkT/EtJ3j71tS51zdblfD0rkFUjM4+H4bIDjycq/HP2C9RXuOMqWnDUCP88ve9sNwdPm51J8cqXStwq8K9KcMkCxghE59n0KjDwkmFhw4dwvDhw+Hv7w+ZTIYdO3aYXJfJZPUe//u//yu2iY6OrnN93LhxJvcpKipCQkKC+N2YkJCA4uJii38+zV7/kN0zw0wQhDrn7po7dy50Op145ObmNkWIrYogACveegTffa3A4v+7CE376gb301fV/nXp3KMcTs5GnDrkKV6/dcMJV//risf7lNXp981WJYa8UAQnZ9t9DiJ78nCuxtrYL6E3OGLy7lhUG0yrY5kFGnjJqxHa7pf5Aj3a3YCXvBqnb6jvvR09JAQrVxgIFiYEZWVl6NmzJ1asWFHv9fz8fJPjX//6F2QyGX7zm9+YtEtKSjJpt3r1apPr8fHxyMzMRGpqKlJTU5GZmYmEhATLfjhoxn0IfH194ejoWKcaUFBQUKdqcJdcLodczjW+1ljx1qPYv90bC9ZdhlsbI24X1P4V8PA0QO4moLLcAZs/UiMqRgelWo+S2074coMvbuY746nhxbVtvYwY+tJt/ONdf3h518CzrQFr3vNHh26VCHvqjsn7ZR5uA+01OYbFc7iAmoa7kx7tvX6pRj3qWYJuypvQVcmRX+YJhbwSfh6lUHnUJq9BbYsBADcr3HGzwv3nZOA/cHOqwRv7B6ONix5tUFtFu13pCqPggMvF3jiUG4D3njqI+YcHAAD+/ORB7L8aiBydd9N+YGqwpn7aYWxsLGJjY+97XaPRmLz+4osvMGjQIHTsaLpCxt3dvU7bu7KyspCamor09HRERtYupV2zZg2ioqKQnZ2Nrl27NjjeZksIXFxcEB4ejrS0NDz//PPi+bS0NIwcObK5wmr1vtxQuwHLG7/pbHJ+1tJriBl7Gw4OAvIuyvHe/3VAyW0neHob0KVnOT7cfgEduv4y8fD1BT/B0VHAwtc7oLrCAb2evIN3N1yGo+koAlI/9cHjEaVo37kKRE0hpF0BPonbKb6eG1U7J2n7j10x9+DTeLr9FSRH7xevLx2cBgBYkRGBFaf6oLtvIXqpCwAAaeM2m9x78Kfj8VOpFwDgjf1DMC/qMNbGfgmgdmOi++19QK3LvfPXbPHL6o0bN/DVV19hw4YNda6lpKRg06ZNUKvViI2Nxfz58+HpWVuhPXr0KBQKhZgMAEDfvn2hUChw5MiRlpEQAMDMmTORkJCAiIgIREVF4R//+AeuXbuG119/vTnDatV2X880e93FVcA7a6888D4urgKmLPwJUxb+ZLZdfRMWiezpeP4j6Lbm9/e9vv1CN2y/0K3R/e/SVblizoEhjYqRmoetdiq8d/7a/PnzsWDBAmtCw4YNG+Dp6YnRo0ebnB8/fjyCgoKg0Whw9uxZzJ07F2fOnEFaWm0iq9VqoVKp6txPpVLddz7e/TRrQjB27FjcunULf/7zn5Gfn4+QkBDs2rULgYGBzRkWERG1QrYaMsjNzYWXl5d43hZD2f/6178wfvx4uLq6mpxPSkoS/xwSEoLOnTsjIiICp06dQu/evQHUnYsHmJ+Pdz/N/iyDyZMnY/Lkyc0dBhERUYN4eXmZJATW+vbbb5GdnY2tW7c+sG3v3r3h7OyMCxcuoHfv3tBoNLhxo+5mWIWFhfedj3c/zb7KgIiIqCk8rM8yWLt2LcLDw9GzZ88Htj137hz0ej38/PwAAFFRUdDpdDh+/LjY5tixY9DpdOjXr59FcTR7hYCIiKgpNPUqg9LSUly8eFF8nZOTg8zMTCiVSrRv3x5A7QTF//u//8OHH35Yp/+lS5eQkpKCZ599Fr6+vjh//jxmzZqFsLAw9O/fHwAQHByMYcOGISkpSVyOOGnSJMTFxVk0oRBghYCIiMguTp48ibCwMISFhQGonUgfFhaGd955R2yzZcsWCIKAl156qU5/FxcX7N27F0OHDkXXrl0xffp0xMTEYM+ePXD81ZKulJQUhIaGIiYmBjExMejRowc2btxocbysEBARkSQ0dYUgOjoawv32uP7ZpEmTMGnSpHqvBQQE4ODBgw98H6VSiU2bNlkUW32YEBARkSQ0dULQ0nDIgIiIiFghICIiaWCFwDwmBEREJAkCYNXSwdb+aGsmBEREJAmsEJjHOQRERETECgEREUkDKwTmMSEgIiJJYEJgHocMiIiIiBUCIiKSBlYIzGNCQEREkiAIMghWfKlb07cl4JABERERsUJARETSYITMqo2JrOnbEjAhICIiSeAcAvM4ZEBERESsEBARkTRwUqF5TAiIiEgSOGRgHhMCIiKSBFYIzOMcAiIiImKFgIiIpEGwcsigtVcImBAQEZEkCAAEwbr+rRmHDIiIiIgVAiIikgYjZJBxp8L7YkJARESSwFUG5nHIgIiIiFghICIiaTAKMsi4MdF9MSEgIiJJEAQrVxm08mUGHDIgIiIiVgiIiEgaOKnQPCYEREQkCUwIzGNCQEREksBJheZxDgERERExISAiImm4u8rAmsMShw4dwvDhw+Hv7w+ZTIYdO3aYXJ84cSJkMpnJ0bdvX5M2VVVVmDZtGnx9feHh4YERI0YgLy/PpE1RURESEhKgUCigUCiQkJCA4uJii38+TAiIiEgSar/UZVYclr1fWVkZevbsiRUrVty3zbBhw5Cfny8eu3btMrk+Y8YMbN++HVu2bMHhw4dRWlqKuLg4GAwGsU18fDwyMzORmpqK1NRUZGZmIiEhwbJgwTkEREREdhEbG4vY2FizbeRyOTQaTb3XdDod1q5di40bN2LIkCEAgE2bNiEgIAB79uzB0KFDkZWVhdTUVKSnpyMyMhIAsGbNGkRFRSE7Oxtdu3ZtcLysEBARkSRYVx34ZYVCSUmJyVFVVdXomA4cOACVSoUuXbogKSkJBQUF4rWMjAzo9XrExMSI5/z9/RESEoIjR44AAI4ePQqFQiEmAwDQt29fKBQKsU1DMSEgIiJJEGxwAEBAQIA4Xq9QKJCcnNyoeGJjY5GSkoJ9+/bhww8/xIkTJ/D000+LCYZWq4WLiwu8vb1N+qnVami1WrGNSqWqc2+VSiW2aSgOGRAREVkgNzcXXl5e4mu5XN6o+4wdO1b8c0hICCIiIhAYGIivvvoKo0ePvm8/QRAgk/2yBPLXf75fm4ZghYCIiCTBVkMGXl5eJkdjE4J7+fn5ITAwEBcuXAAAaDQaVFdXo6ioyKRdQUEB1Gq12ObGjRt17lVYWCi2aSgmBEREJA22GjOwk1u3biE3Nxd+fn4AgPDwcDg7OyMtLU1sk5+fj7Nnz6Jfv34AgKioKOh0Ohw/flxsc+zYMeh0OrFNQ3HIgIiIpMHKrYthYd/S0lJcvHhRfJ2Tk4PMzEwolUoolUosWLAAv/nNb+Dn54crV67grbfegq+vL55//nkAgEKhQGJiImbNmgUfHx8olUrMnj0boaGh4qqD4OBgDBs2DElJSVi9ejUAYNKkSYiLi7NohQHAhICIiMguTp48iUGDBomvZ86cCQCYMGECVq1ahR9++AGffPIJiouL4efnh0GDBmHr1q3w9PQU+yxduhROTk4YM2YMKioqMHjwYKxfvx6Ojo5im5SUFEyfPl1cjTBixAizex/cDxMCIiKShMbsNnhvf0tER0dDMNNp9+7dD7yHq6srli9fjuXLl9+3jVKpxKZNmywLrh5MCIiISBL4tEPzOKmQiIiIWCEgIiKJEGQWTwys078VY0JARESS0NRzCFoaDhkQERERKwRERCQR1m4u1MorBEwIiIhIErjKwLwGJQQff/xxg284ffr0RgdDREREzaNBCcHSpUsbdDOZTMaEgIiIHl6tvOxvjQYlBDk5OfaOg4iIyK44ZGBeo1cZVFdXIzs7GzU1NbaMh4iIyD4e8qcdNjeLE4Ly8nIkJibC3d0d3bt3x7Vr1wDUzh14//33bR4gERER2Z/FCcHcuXNx5swZHDhwAK6uruL5IUOGYOvWrTYNjoiIyHZkNjhaL4uXHe7YsQNbt25F3759IZP98sN5/PHHcenSJZsGR0REZDPch8AsiysEhYWFUKlUdc6XlZWZJAhERETUclicEPTp0wdfffWV+PpuErBmzRpERUXZLjIiIiJb4qRCsyweMkhOTsawYcNw/vx51NTU4KOPPsK5c+dw9OhRHDx40B4xEhERWY9POzTL4gpBv3798N1336G8vByPPfYYvvnmG6jVahw9ehTh4eH2iJGIiIjsrFHPMggNDcWGDRtsHQsREZHd8PHH5jUqITAYDNi+fTuysrIgk8kQHByMkSNHwsmJz0oiIqKHFFcZmGXxN/jZs2cxcuRIaLVadO3aFQDw448/ol27dti5cydCQ0NtHiQRERHZl8VzCF599VV0794deXl5OHXqFE6dOoXc3Fz06NEDkyZNskeMRERE1rs7qdCaoxWzuEJw5swZnDx5Et7e3uI5b29vLFy4EH369LFpcERERLYiE2oPa/q3ZhZXCLp27YobN27UOV9QUIBOnTrZJCgiIiKb4z4EZjUoISgpKRGPRYsWYfr06fj888+Rl5eHvLw8fP7555gxYwY++OADe8dLREREdtCgIYO2bduabEssCALGjBkjnhN+XosxfPhwGAwGO4RJRERkJW5MZFaDEoL9+/fbOw4iIiL74rJDsxqUEAwcONDecRAREVEzavROQuXl5bh27Rqqq6tNzvfo0cPqoIiIiGyOFQKzLE4ICgsL8bvf/Q5ff/11vdc5h4CIiB5KTAjMsnjZ4YwZM1BUVIT09HS4ubkhNTUVGzZsQOfOnbFz5057xEhERER2ZnGFYN++ffjiiy/Qp08fODg4IDAwEM888wy8vLyQnJyM5557zh5xEhERWYerDMyyuEJQVlYGlUoFAFAqlSgsLARQ+wTEU6dO2TY6IiIiG7m7U6E1R2vWqJ0Ks7OzAQC9evXC6tWr8dNPP+Hvf/87/Pz8bB4gERFRS3To0CEMHz4c/v7+kMlk2LFjh3hNr9fjzTffRGhoKDw8PODv74+XX34Z169fN7lHdHQ0ZDKZyTFu3DiTNkVFRUhISIBCoYBCoUBCQgKKi4stjrdRcwjy8/MBAPPnz0dqairat2+Pjz/+GIsWLbI4ACIioibRxFsXl5WVoWfPnlixYkWda+Xl5Th16hTefvttnDp1Ctu2bcOPP/6IESNG1GmblJSE/Px88Vi9erXJ9fj4eGRmZiI1NRWpqanIzMxEQkKCZcGiEXMIxo8fL/45LCwMV65cwX//+1+0b98evr6+FgdARETUkpSUlJi8lsvlkMvlddrFxsYiNja23nsoFAqkpaWZnFu+fDmeeOIJXLt2De3btxfPu7u7Q6PR1HufrKwspKamIj09HZGRkQCANWvWICoqCtnZ2ejatWuDP5fFFYJ7ubu7o3fv3kwGiIjooSaDlXMIfr5PQECAWJ5XKBRITk62SXw6nQ4ymQxt27Y1OZ+SkgJfX190794ds2fPxp07d8RrR48ehUKhEJMBAOjbty8UCgWOHDli0fs3qEIwc+bMBt9wyZIlFgVARETUkuTm5sLLy0t8XV91wFKVlZX44x//iPj4eJN7jx8/HkFBQdBoNDh79izmzp2LM2fOiNUFrVYrTvT/NZVKBa1Wa1EMDUoITp8+3aCb/foBSE3p+S6hcJI5N8t7E9lbx075zR0Ckd3UGKpwuanezEbLDr28vEy+tK2l1+sxbtw4GI1GrFy50uRaUlKS+OeQkBB07twZEREROHXqFHr37g2g/u9eQRAs/k7mw42IiEgaHsKdCvV6PcaMGYOcnBzs27fvgYlG79694ezsjAsXLqB3797QaDS4ceNGnXaFhYVQq9UWxWL1HAIiIiKy3N1k4MKFC9izZw98fHwe2OfcuXPQ6/XiMv+oqCjodDocP35cbHPs2DHodDr069fPonga/XAjIiKiFqWJKwSlpaW4ePGi+DonJweZmZlQKpXw9/fHCy+8gFOnTuHLL7+EwWAQx/yVSiVcXFxw6dIlpKSk4Nlnn4Wvry/Onz+PWbNmISwsDP379wcABAcHY9iwYUhKShKXI06aNAlxcXEWrTAAmBAQEZFEWLvboKV9T548iUGDBomv707QnzBhAhYsWCA+/6dXr14m/fbv34/o6Gi4uLhg7969+Oijj1BaWoqAgAA899xzmD9/PhwdHcX2KSkpmD59OmJiYgAAI0aMqHfvgwdhQkBERGQH0dHREIT7ZxHmrgG1yxsPHjz4wPdRKpXYtGmTxfHdiwkBERFJw0M4qfBh0qhJhRs3bkT//v3h7++Pq1evAgCWLVuGL774wqbBERER2UwTb13c0licEKxatQozZ87Es88+i+LiYhgMBgBA27ZtsWzZMlvHR0RERE3A4oRg+fLlWLNmDebNm2cyqSEiIgI//PCDTYMjIiKyFT7+2DyL5xDk5OQgLCysznm5XI6ysjKbBEVERGRzNtqpsLWyuEIQFBSEzMzMOue//vprPP7447aIiYiIyPY4h8AsiysEb7zxBqZMmYLKykoIgoDjx4/j008/RXJyMv75z3/aI0YiIiKyM4sTgt/97neoqanBnDlzUF5ejvj4eDzyyCP46KOPMG7cOHvESEREZLWm3piopWnUPgRJSUlISkrCzZs3YTQa6330IhER0UOF+xCYZdXGRL6+vraKg4iIiJqRxQlBUFCQ2WcsX77cZE+2JiIiajhrlw6yQmBqxowZJq/1ej1Onz6N1NRUvPHGG7aKi4iIyLY4ZGCWxQnBH/7wh3rP/+1vf8PJkyetDoiIiIiaXqOeZVCf2NhY/Pvf/7bV7YiIiGyL+xCYZbOnHX7++edQKpW2uh0REZFNcdmheRYnBGFhYSaTCgVBgFarRWFhIVauXGnT4IiIiKhpWJwQjBo1yuS1g4MD2rVrh+joaHTr1s1WcREREVETsighqKmpQYcOHTB06FBoNBp7xURERGR7XGVglkWTCp2cnPD73/8eVVVV9oqHiIjILvj4Y/MsXmUQGRmJ06dP2yMWIiIiaiYWzyGYPHkyZs2ahby8PISHh8PDw8Pkeo8ePWwWHBERkU218t/yrdHghOCVV17BsmXLMHbsWADA9OnTxWsymQyCIEAmk8FgMNg+SiIiImtxDoFZDU4INmzYgPfffx85OTn2jIeIiIiaQYMTAkGoTY0CAwPtFgwREZG9cGMi8yyaQ2DuKYdEREQPNQ4ZmGVRQtClS5cHJgW3b9+2KiAiIiJqehYlBO+++y4UCoW9YiEiIrIbDhmYZ1FCMG7cOKhUKnvFQkREZD8cMjCrwRsTcf4AERFR62XxKgMiIqIWiRUCsxqcEBiNRnvGQUREZFecQ2CexVsXExERtUisEJhl8cONiIiI6MEOHTqE4cOHw9/fHzKZDDt27DC5LggCFixYAH9/f7i5uSE6Ohrnzp0zaVNVVYVp06bB19cXHh4eGDFiBPLy8kzaFBUVISEhAQqFAgqFAgkJCSguLrY4XiYEREQkDYINDguUlZWhZ8+eWLFiRb3XFy9ejCVLlmDFihU4ceIENBoNnnnmGdy5c0dsM2PGDGzfvh1btmzB4cOHUVpairi4OJPnBsXHxyMzMxOpqalITU1FZmYmEhISLAsWHDIgIiKJsNUcgpKSEpPzcrkccrm8TvvY2FjExsbWey9BELBs2TLMmzcPo0ePBlD7zCC1Wo3Nmzfjtddeg06nw9q1a7Fx40YMGTIEALBp0yYEBARgz549GDp0KLKyspCamor09HRERkYCANasWYOoqChkZ2eja9euDf58rBAQERFZICAgQCzPKxQKJCcnW3yPnJwcaLVaxMTEiOfkcjkGDhyII0eOAAAyMjKg1+tN2vj7+yMkJERsc/ToUSgUCjEZAIC+fftCoVCIbRqKFQIiIpIGG00qzM3NhZeXl3i6vurAg2i1WgCAWq02Oa9Wq3H16lWxjYuLC7y9veu0udtfq9XWu2GgSqUS2zQUEwIiIpIEWw0ZeHl5mSQEVsV0z6Z/giA8cCPAe9vU174h97kXhwyIiIiamEajAYA6v8UXFBSIVQONRoPq6moUFRWZbXPjxo069y8sLKxTfXgQJgRERCQNTbzKwJygoCBoNBqkpaWJ56qrq3Hw4EH069cPABAeHg5nZ2eTNvn5+Th79qzYJioqCjqdDsePHxfbHDt2DDqdTmzTUBwyICIiaWjijYlKS0tx8eJF8XVOTg4yMzOhVCrRvn17zJgxA4sWLULnzp3RuXNnLFq0CO7u7oiPjwcAKBQKJCYmYtasWfDx8YFSqcTs2bMRGhoqrjoIDg7GsGHDkJSUhNWrVwMAJk2ahLi4OItWGABMCIiIiOzi5MmTGDRokPh65syZAIAJEyZg/fr1mDNnDioqKjB58mQUFRUhMjIS33zzDTw9PcU+S5cuhZOTE8aMGYOKigoMHjwY69evh6Ojo9gmJSUF06dPF1cjjBgx4r57H5gjE1rwU4tKSkqgUCgQjZFwkjk3dzhEduHYKai5QyCymxpDFfZe/hg6nc5mE/Xudfe74vHJi+Aod230fQxVlTi/8i27xtqcWCEgIiJp4LMMzGJCQEREksCnHZrHVQZERETECgEREUkEhwzMYkJARETS0cq/1K3BIQMiIiJihYCIiKSBkwrNY0JARETSwDkEZnHIgIiIiFghICIiaeCQgXlMCIiISBo4ZGAWhwyIiIiIFQIiIpIGDhmYx4SAiIikgUMGZjEhICIiaWBCYBbnEBARERErBEREJA2cQ2AeEwIiIpIGDhmYxSEDIiIiYoWAiIikQSYIkAmN/zXfmr4tARMCIiKSBg4ZmMUhAyIiImKFgIiIpIGrDMxjQkBERNLAIQOzOGRARERErBAQEZE0cMjAPCYEREQkDRwyMIsJARERSQIrBOZxDgERERGxQkBERBLBIQOzmBAQEZFktPayvzU4ZEBERERMCIiISCIEwfrDAh06dIBMJqtzTJkyBQAwceLEOtf69u1rco+qqipMmzYNvr6+8PDwwIgRI5CXl2ezH8mvMSEgIiJJuLvKwJrDEidOnEB+fr54pKWlAQBefPFFsc2wYcNM2uzatcvkHjNmzMD27duxZcsWHD58GKWlpYiLi4PBYLD653EvziEgIiKyg3bt2pm8fv/99/HYY49h4MCB4jm5XA6NRlNvf51Oh7Vr12Ljxo0YMmQIAGDTpk0ICAjAnj17MHToUJvGywoBERFJg2CDA0BJSYnJUVVV9cC3rq6uxqZNm/DKK69AJpOJ5w8cOACVSoUuXbogKSkJBQUF4rWMjAzo9XrExMSI5/z9/RESEoIjR440/udwH0wIiIhIEmRG6w8ACAgIgEKhEI/k5OQHvveOHTtQXFyMiRMniudiY2ORkpKCffv24cMPP8SJEyfw9NNPiwmGVquFi4sLvL29Te6lVquh1Wpt9nO5i0MGREREFsjNzYWXl5f4Wi6XP7DP2rVrERsbC39/f/Hc2LFjxT+HhIQgIiICgYGB+OqrrzB69Oj73ksQBJMqg60wISCERJbixcmF6BxaDh9NDRa80gFHUxXi9d3Xz9Tbb817fvh8lQoAMP2DXIQ9VQoftR4V5Q7IOumBtQv9kHvRtUk+A9H9PDsyB8+NyoFaUw4AuJrjiU83dMPJY2oAQFvvSvzu9XPo3acQHm30OHvGB3//qAeu57UR7/H+R9+iR9gtk/se3PsIPni3T9N9ELKejTYm8vLyMkkIHuTq1avYs2cPtm3bZradn58fAgMDceHCBQCARqNBdXU1ioqKTKoEBQUF6Nevn+XxPwATAoKruxGXz7nimy3eeGft1TrXx/V83OR1n6fv4H8+zMXhr35JGi58745927xR+JMLPL1r8NtZN7Do08uYEBkMo9H2mSxRQ90sdMW61Y8j/+cv+MHDruHtRemYljgI16544u2Fx2AwOODPb0WivMwJz4+9hEVLvsNrLw9GVeUv/0R+vTMQm/4VLL6uqnJs8s9C1mmuZxmsW7cOKpUKzz33nNl2t27dQm5uLvz8/AAA4eHhcHZ2RlpaGsaMGQMAyM/Px9mzZ7F48eLGBWNGs84hOHToEIYPHw5/f3/IZDLs2LGjOcORrJP7vbBhsR+++7ptvdeLCp1NjqihOpz5rg20134pk32d4oOzx9rgRp4LLv7gjg0faKB6RA91QHUTfQqi+h0/4oeT6Rr8lNcGP+W1wSf/fByVFU7o1v02Hnm0DMEhRVjxYU9c+K83fsr1xMolPeHqVoPowaZrvauqHFF021U8ysucm+kTUaM18T4EAGA0GrFu3TpMmDABTk6/JJilpaWYPXs2jh49iitXruDAgQMYPnw4fH198fzzzwMAFAoFEhMTMWvWLOzduxenT5/Gb3/7W4SGhoqrDmypWROCsrIy9OzZEytWrGjOMMgCbX31eGJwCXZvUd63jdzNgJixt5F/1QWF1/mPJj08HBwEDHg6D66uBmSdVcLZpXYtd3X1L7/tG40y1NQ44PEepkMEg57Jw6c7d2HVhr1InHwWbm76Jo2dWqY9e/bg2rVreOWVV0zOOzo64ocffsDIkSPRpUsXTJgwAV26dMHRo0fh6ekptlu6dClGjRqFMWPGoH///nB3d8d//vMfODravkLVrEMGsbGxiI2NbXD7qqoqk+UdJSUl9giLzHhmTBEqSh1xeJeizrW4CTfx6p/y4eZhxLULcswd1xE1ei5koebXoaMOH648BBcXIyoqHPHen55A7lUvODoacSPfDb+bdA7L/9oLlZVOeH7sRSh9qqD0+eXfmv1pAbiR746i264IDCrBxNfOo+NjOsyb1b8ZPxVZqjmGDGJiYiDUU1lwc3PD7t27H9jf1dUVy5cvx/Llyy1/cwu1qDkEycnJePfdd5s7DEkbOu429m1vC31V3S/6fdu8ceqQJ5QqPV74fSHmrb6K/xnZqd62RE0p75onpiYOQps2evQfeB2z3jqFOdOeRO5VLyx8OxJ/ePMUPtu1C4YaGU5ntMOJdLVJ/91fdhD/fDXHC9fz2uDjfx7AY12KcenHtk37Yajx+LRDs1rUv9Rz586FTqcTj9zc3OYOSVJCnihFQKcqpG72qfd6+R1HXM+R4+yxNvhLUiACOlWhf6yuiaMkqqumxgH5P7XBhWxvrP9Hd1y+qMDIFy8DAC7+2BbTEp/GC7HPYfzzw/DOG/3g5VWNG/nu973fxR8V0OtleOTR0qb6CER216IqBHK5vEHrPck+hr50Gz+eccPl824N6yAT4OzSylNqapFkMgHOzqZ7wd+dJOj/aCk6dS3CJ2uD6+sKAAgMugNnZwG3b3FZbUvSXKsMWooWlRCQfbi6G+Af9MtqAE1ANTp2r8CdYkcU/uQCAHBvY8CA4Tr8412/Ov017aswcEQxMg56QnfbCb4aPcZMKUB1hQOO7/Ws056oKU1IOo+Tx1QoLHCDu3sNBjz9E0J73cQ7b9Su434y+ifoil1QeMMdHR4rwWvTvkf6YT+cPlG7x4bGvwyDnsnFyXQ1dDoXtO9wB69OOYuLPypw/of6q2X0kGrkSgGT/q0YEwJCl54V+N9/XxJfv/7udQDAN1u98eH/tAcADBxZDMgE7N/hXad/dZUDQiLL8HzSTbRRGFB80wk/pHvgf0Z2gu4WVxlQ82qrrMTseRlQ+lShrMwJOZcUeOeNfjh9svYLX+lTiaSpZ9HWuxJFt1yxd3cAPt3QTexfUyNDr/BCjHzhEtzcDCgscMOJdDVS1nXjHhvUqsiE+qY/NpHS0lJcvHgRABAWFoYlS5Zg0KBBUCqVaN++/QP7l5SUQKFQIBoj4STjFw+1To6dgpo7BCK7qTFUYe/lj6HT6Sza/c8Sd78romL/DCfnxg/z1OgrcfTrd+waa3Nq1grByZMnMWjQIPH1zJkzAQATJkzA+vXrmykqIiJqlbjKwKxmTQiio6PrXZ9JRERETYtzCIiISBK4ysA8JgRERCQNRqH2sKZ/K8aEgIiIpIFzCMxqUTsVEhERkX2wQkBERJIgg5VzCGwWycOJCQEREUkDdyo0i0MGRERExAoBERFJA5cdmseEgIiIpIGrDMzikAERERGxQkBERNIgEwTIrJgYaE3floAJARERSYPx58Oa/q0YhwyIiIiIFQIiIpIGDhmYx4SAiIikgasMzGJCQERE0sCdCs3iHAIiIiJihYCIiKSBOxWax4SAiIikgUMGZnHIgIiIiFghICIiaZAZaw9r+rdmTAiIiEgaOGRgFocMiIiIiBUCIiKSCG5MZBYrBEREJAl3ty625rDEggULIJPJTA6NRiNeFwQBCxYsgL+/P9zc3BAdHY1z586Z3KOqqgrTpk2Dr68vPDw8MGLECOTl5dnk53EvJgRERER20r17d+Tn54vHDz/8IF5bvHgxlixZghUrVuDEiRPQaDR45plncOfOHbHNjBkzsH37dmzZsgWHDx9GaWkp4uLiYDAYbB4rhwyIiEgammFSoZOTk0lV4JdbCVi2bBnmzZuH0aNHAwA2bNgAtVqNzZs347XXXoNOp8PatWuxceNGDBkyBACwadMmBAQEYM+ePRg6dGjjP0s9WCEgIiJpEAAYrTh+zgdKSkpMjqqqqvu+5YULF+Dv74+goCCMGzcOly9fBgDk5ORAq9UiJiZGbCuXyzFw4EAcOXIEAJCRkQG9Xm/Sxt/fHyEhIWIbW2JCQEREkmCrOQQBAQFQKBTikZycXO/7RUZG4pNPPsHu3buxZs0aaLVa9OvXD7du3YJWqwUAqNVqkz5qtVq8ptVq4eLiAm9v7/u2sSUOGRAREVkgNzcXXl5e4mu5XF5vu9jYWPHPoaGhiIqKwmOPPYYNGzagb9++AACZTGbSRxCEOufu1ZA2jcEKARERSYOAX+YRNOqovY2Xl5fJcb+E4F4eHh4IDQ3FhQsXxHkF9/6mX1BQIFYNNBoNqqurUVRUdN82tsSEgIiIpMGqZMDKCYmoXUKYlZUFPz8/BAUFQaPRIC0tTbxeXV2NgwcPol+/fgCA8PBwODs7m7TJz8/H2bNnxTa2xCEDIiIiO5g9ezaGDx+O9u3bo6CgAH/5y19QUlKCCRMmQCaTYcaMGVi0aBE6d+6Mzp07Y9GiRXB3d0d8fDwAQKFQIDExEbNmzYKPjw+USiVmz56N0NBQcdWBLTEhICIiaTACsGbo3cKHG+Xl5eGll17CzZs30a5dO/Tt2xfp6ekIDAwEAMyZMwcVFRWYPHkyioqKEBkZiW+++Qaenp7iPZYuXQonJyeMGTMGFRUVGDx4MNavXw9HR0crPkj9ZILQcp/WUFJSAoVCgWiMhJPMubnDIbILx05BzR0Ckd3UGKqw9/LH0Ol0JhP1bOnud8XgkDlwcmzYeH99agxV2Ht2sV1jbU6cQ0BEREQcMiAiIong44/NYkJARETSwITALA4ZEBERESsEREQkEawQmMWEgIiIpKGJlx22NEwIiIhIEn79gKLG9m/NOIeAiIiIWCEgIiKJ4BwCs5gQEBGRNBgFQGbFl7qxdScEHDIgIiIiVgiIiEgiOGRgFhMCIiKSCCsTArTuhIBDBkRERMQKARERSQSHDMxiQkBERNJgFGBV2Z+rDIiIiKi1Y4WAiIikQTDWHtb0b8WYEBARkTRwDoFZTAiIiEgaOIfALM4hICIiIlYIiIhIIjhkYBYTAiIikgYBViYENovkocQhAyIiImKFgIiIJIJDBmYxISAiImkwGgFYsZeAsXXvQ8AhAyIiImKFgIiIJIJDBmYxISAiImlgQmAWhwyIiIiIFQIiIpIIbl1sFhMCIiKSBEEwQrDiiYXW9G0JmBAQEZE0CIJ1v+VzDgERERFZKjk5GX369IGnpydUKhVGjRqF7OxskzYTJ06ETCYzOfr27WvSpqqqCtOmTYOvry88PDwwYsQI5OXl2TxeJgRERCQNd1cZWHNY4ODBg5gyZQrS09ORlpaGmpoaxMTEoKyszKTdsGHDkJ+fLx67du0yuT5jxgxs374dW7ZsweHDh1FaWoq4uDgYDAarfyS/xiEDIiKSBqMRkFkxD+DnOQQlJSUmp+VyOeRyeZ3mqampJq/XrVsHlUqFjIwMDBgwwKS/RqOp9y11Oh3Wrl2LjRs3YsiQIQCATZs2ISAgAHv27MHQoUMb/3nuwQoBERGRBQICAqBQKMQjOTm5Qf10Oh0AQKlUmpw/cOAAVCoVunTpgqSkJBQUFIjXMjIyoNfrERMTI57z9/dHSEgIjhw5YoNP8wtWCIiISBoEK5cd/jxkkJubCy8vL/F0fdWBul0FzJw5E08++SRCQkLE87GxsXjxxRcRGBiInJwcvP3223j66aeRkZEBuVwOrVYLFxcXeHt7m9xPrVZDq9U2/rPUgwkBERFJgmA0QrBiyODuskMvLy+ThKAhpk6diu+//x6HDx82OT927FjxzyEhIYiIiEBgYCC++uorjB492kwsAmQymUUxPAiHDIiIiOxo2rRp2LlzJ/bv349HH33UbFs/Pz8EBgbiwoULAACNRoPq6moUFRWZtCsoKIBarbZpnEwIiIhIGpp4lYEgCJg6dSq2bduGffv2ISgo6IF9bt26hdzcXPj5+QEAwsPD4ezsjLS0NLFNfn4+zp49i379+ln2+R+AQwZERCQNRgGQNd3GRFOmTMHmzZvxxRdfwNPTUxzzVygUcHNzQ2lpKRYsWIDf/OY38PPzw5UrV/DWW2/B19cXzz//vNg2MTERs2bNgo+PD5RKJWbPno3Q0FBx1YGtMCEgIiKyg1WrVgEAoqOjTc6vW7cOEydOhKOjI3744Qd88sknKC4uhp+fHwYNGoStW7fC09NTbL906VI4OTlhzJgxqKiowODBg7F+/Xo4OjraNF4mBEREJA2CAMCafQgsHzIwx83NDbt3737gfVxdXbF8+XIsX77cove3FBMCIiKSBMEoQLBiyOBBX/AtHRMCIiKSBsEI6yoErftph1xlQERERKwQEBGRNHDIwDwmBEREJA0cMjCrRScEd7O1Guit2p6a6GEmGKqaOwQiu6kx1v79borfvq39rqiB3nbBPIRadEJw584dAMBh7HpAS6IW7HJzB0Bkf3fu3IFCobDLvV1cXKDRaHBYa/13hUajgYuLiw2ievjIhBY8KGI0GnH9+nV4enra/CEPVL+SkhIEBATUedoXUWvAv99NTxAE3LlzB/7+/nBwsN8898rKSlRXV1t9HxcXF7i6utogoodPi64QODg4PPBBEWQfjXnaF1FLwb/fTctelYFfc3V1bbVf5LbCZYdERETEhICIiIiYEJCF5HI55s+fD7lc3tyhENkc/36TlLXoSYVERERkG6wQEBERERMCIiIiYkJAREREYEJAREREYEJAFli5ciWCgoLg6uqK8PBwfPvtt80dEpFNHDp0CMOHD4e/vz9kMhl27NjR3CERNTkmBNQgW7duxYwZMzBv3jycPn0aTz31FGJjY3Ht2rXmDo3IamVlZejZsydWrFjR3KEQNRsuO6QGiYyMRO/evbFq1SrxXHBwMEaNGoXk5ORmjIzItmQyGbZv345Ro0Y1dyhETYoVAnqg6upqZGRkICYmxuR8TEwMjhw50kxRERGRLTEhoAe6efMmDAYD1Gq1yXm1Wg2tVttMURERkS0xIaAGu/cR04Ig8LHTREStBBMCeiBfX184OjrWqQYUFBTUqRoQEVHLxISAHsjFxQXh4eFIS0szOZ+WloZ+/fo1U1RERGRLTs0dALUMM2fOREJCAiIiIhAVFYV//OMfuHbtGl5//fXmDo3IaqWlpbh48aL4OicnB5mZmVAqlWjfvn0zRkbUdLjskBps5cqVWLx4MfLz8xESEoKlS5diwIABzR0WkdUOHDiAQYMG1Tk/YcIErF+/vukDImoGTAiIiIiIcwiIiIiICQERERGBCQERERGBCQERERGBCQERERGBCQERERGBCQERERGBCQERERGBCQGR1RYsWIBevXqJrydOnIhRo0Y1eRxXrlyBTCZDZmbmfdt06NABy5Yta/A9169fj7Zt21odm0wmw44dO6y+DxHZDxMCapUmTpwImUwGmUwGZ2dndOzYEbNnz0ZZWZnd3/ujjz5q8Ha3DfkSJyJqCny4EbVaw4YNw7p166DX6/Htt9/i1VdfRVlZGVatWlWnrV6vh7Ozs03eV6FQ2OQ+RERNiRUCarXkcjk0Gg0CAgIQHx+P8ePHi2Xru2X+f/3rX+jYsSPkcjkEQYBOp8OkSZOgUqng5eWFp59+GmfOnDG57/vvvw+1Wg1PT08kJiaisrLS5Pq9QwZGoxEffPABOnXqBLlcjvbt22PhwoUAgKCgIABAWFgYZDIZoqOjxX7r1q1DcHAwXF1d0a1bN6xcudLkfY4fP46wsDC4uroiIiICp0+ftvhntGTJEoSGhsLDwwMBAQGYPHkySktL67TbsWMHunTpAldXVzzzzDPIzc01uf6f//wH4eHhcHV1RceOHfHuu++ipqbG4niIqPkwISDJcHNzg16vF19fvHgRn332Gf7973+LJfvnnnsOWq0Wu3btQkZGBnr37o3Bgwfj9u3bAIDPPvsM8+fPx8KFC3Hy5En4+fnV+aK+19y5c/HBBx/g7bffxvnz57F582ao1WoAtV/qALBnzx7k5+dj27ZtAIA1a9Zg3rx5WLhwIbKysrBo0SK8/fbb2LBhAwCgrKwMcXFx6Nq1KzIyMrBgwQLMnj3b4p+Jg4MDPv74Y5w9exYbNmzAvn37MGfOHJM25eXlWLhwITZs2IDvvvsOJSUlGDdunHh99+7d+O1vf4vp06fj/PnzWL16NdavXy8mPUTUQghErdCECROEkSNHiq+PHTsm+Pj4CGPGjBEEQRDmz58vODs7CwUFBWKbvXv3Cl5eXkJlZaXJvR577DFh9erVgiAIQlRUlPD666+bXI+MjBR69uxZ73uXlJQIcrlcWLNmTb1x5uTkCACE06dPm5wPCAgQNm/ebHLuvffeE6KiogRBEITVq1cLSqVSKCsrE6+vWrWq3nv9WmBgoLB06dL7Xv/ss88EHx8f8fW6desEAEJ6erp4LisrSwAgHDt2TBAEQXjqqaeERYsWmdxn48aNgp+fn/gagLB9+/b7vi8RNT/OIaBW68svv0SbNm1QU1MDvV6PkSNHYvny5eL1wMBAtGvXTnydkZGB0tJS+Pj4mNynoqICly5dAgBkZWXh9ddfN7keFRWF/fv31xtDVlYWqqqqMHjw4AbHXVhYiNzcXCQmJiIpKUk8X1NTI85PyMrKQs+ePeHu7m4Sh6X279+PRYsW4fz58ygpKUFNTQ0qKytRVlYGDw8PAICTkxMiIiLEPt26dUPbtm2RlZWFJ554AhkZGThx4oRJRcBgMKCyshLl5eUmMRLRw4sJAbVagwYNwqpVq+Ds7Ax/f/86kwbvfuHdZTQa4efnhwMHDtS5V2OX3rm5uVncx2g0AqgdNoiMjDS55ujoCAAQBKFR8fza1atX8eyzz+L111/He++9B6VSicOHDyMxMdFkaAWoXTZ4r7vnjEYj3n33XYwePbpOG1dXV6vjJKKmwYSAWi0PDw906tSpwe179+4NrVYLJycndOjQod42wcHBSE9Px8svvyyeS09Pv+89O3fuDDc3N+zduxevvvpqnesuLi4Aan+jvkutVuORRx7B5cuXMX78+Hrv+/jjj2Pjxo2oqKgQkw5zcdTn5MmTqKmpwYcffggHh9rpRJ999lmddjU1NTh58iSeeOIJAEB2djaKi4vRrVs3ALU/t+zsbIt+1kT08GFCQPSzIUOGICoqCqNGjcIHH3yArl274vr169i1axdGjRqFiIgI/OEPf8CECRMQERGBJ598EikpKTh37hw6duxY7z1dXV3x5ptvYs6cOXBxcUH//v1RWFiIc+fOITExESqVCm5ubkhNTcWjjz4KV1dXKBQKLFiwANOnT4eXlxdiY2NRVVWFkydPoqioCDNnzkR8fDzmzZuHxMRE/OlPf8KVK1fw17/+1aLP+9hjj6GmpgbLly/H8OHD8d133+Hvf/97nXbOzs6YNm0aPv74Yzg7O2Pq1Kno27evmCC88847iIuLQ0BAAF588UU4ODjg+++/xw8//IC//OUvlv+HIKJmwVUGRD+TyWTYtWsXBgwYgFdeeQVdunTBuHHjcOXKFXFVwNixY/HOO+/gzTffRHh4OK5evYrf//73Zu/79ttvY9asWXjnnXcQHByMsWPHoqCgAEDt+PzHH3+M1atXw9/fHyNHjgQAvPrqq/jnP/+J9evXIzQ0FAMHDsT69evFZYpt2rTBf/7zH5w/fx5hYWGYN28ePvjgA4s+b69evbBkyRJ88MEHCAkJQUpKCpKTk+u0c3d3x5tvvon4+HhERUXBzc0NW7ZsEa8PHToUX375JdLS0tCnTx/07dsXS5YsQWBgoEXxEFHzkgm2GIwkIiKiFo0VAiIiImJCQEREREwIiIiICEwIiIiICEwIiIiICEwIiIiICEwIiIiICEwIiIiICEwIiIiICEwIiIiICEwIiIiICMD/A5LodCdjqzDSAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 640x480 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# predicted classes using default 0.5 threshold\n",
    "y_hat_test = fit_lr_grid_search.predict(X_test) \n",
    "\n",
    "print(\"classification report (Test):\\n \", classification_report(y_test, y_hat_test, digits=3))\n",
    "\n",
    "# Create the confusion matrix\n",
    "cm = confusion_matrix(y_test, y_hat_test)\n",
    "ConfusionMatrixDisplay(confusion_matrix=cm).plot()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "964dbf93-1011-4acd-95c2-d77d69b43446",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "#### Receiver Operating Characteristic Curve (Test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 97,
   "id": "996bdc9c-c3e9-43ef-ad62-2c0dc17bda14",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "# calculate the predicted probabilities on all of the training data\n",
    "y_proba_test = fit_lr_grid_search.predict_proba(X_test)[:,1] \n",
    "\n",
    "# calculate the area under the curve (AUC)\n",
    "auc = roc_auc_score(y_test, y_proba_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 98,
   "id": "b54736fc-7cd0-43d4-8217-e0320f960f45",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0.5, 0, 'False Positive Rate')"
      ]
     },
     "execution_count": 98,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAkIAAAHFCAYAAAAe+pb9AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8hTgPZAAAACXBIWXMAAA9hAAAPYQGoP6dpAABlZklEQVR4nO3deViUVfsH8O+wIwIuKAIquCGkmQq5pqa5pKVpb4b7kpqmZsKrFllub73+NDNzo1LEJRfEpXxLU7LcNRUx19SUXCEDA1RkP78/TjOyDMjAzDyzfD/XNZc8zzwzc49Dze0597mPSgghQERERGSFbJQOgIiIiEgpTISIiIjIajERIiIiIqvFRIiIiIisFhMhIiIislpMhIiIiMhqMREiIiIiq8VEiIiIiKwWEyEiIiKyWkyEiKhMVq9eDZVKpbnZ2dnBy8sLAwYMwJUrV7Q+JicnBxEREWjbti3c3d3h7OyMwMBAvPfee0hJSdH6mPz8fKxbtw5du3aFh4cH7O3tUbNmTbz88sv43//+h/z8/CfGmpqaCg8PD2zatAl//PFHobhLu/3xxx8V+SsCANy5cwezZs3C6dOni9334YcfomXLlmV6D0RkHCpusUFEZbF69WqMHDkSUVFRCAgIQGZmJg4fPoyPP/4Yrq6u+O2331C1alXN9RkZGejVqxcOHTqEN998Ey+//DKcnZ1x9OhRLFiwAJUrV0ZsbCwaN26seUxmZib69u2LPXv2YMCAAejXrx9q1aqFv/76Cz/88APWrl2L6OhovPLKK6XGGhoair179+LXX39FdnY24uPjC90/fvx4pKWlYf369YXOt2jRAo6OjhX6ezp58iSeffZZREVFYcSIEYXuS0tLg5+fHxYuXIiRI0dW6HWISE8EEVEZREVFCQDixIkThc7Pnj1bABCrVq0qdP7NN98UAMSmTZuKPdelS5eEu7u7aNKkicjNzdWcf+uttwQAsWbNGq0xXL58Wfz666+lxpmSkiKcnZ3FF198UeI1nTp1Ek2aNCn1ecrrxIkTAoCIiorSev/EiROFv7+/yM/PN8jrE5FuODVGRBUSHBwMAPjzzz8155KSkrBq1Sr06NEDISEhxR7j7++Pd999F+fPn8c333yjeczKlSvRo0cPDBs2TOtrNWrUCM2aNSs1ntWrVyM3N1fr65YmPT0dU6ZMQb169eDg4AAfHx9MnjwZDx8+LHRdTEwMWrduDXd3d1SqVAn169fHG2+8AQDYt28fnn32WQDAyJEjNVNus2bN0jx+6NChuHz5Mn7++Wed4iMiw2AiREQVkpCQAEAmN2o///wzcnNz0bdv3xIfp74vNjZW85icnJxSH1MW33//PVq0aIEqVaqU+TEZGRno1KkT1qxZg0mTJmHXrl149913sXr1avTp0wfinwqCo0ePIiQkBPXr18emTZvw/fffY8aMGcjNzQUAtGzZElFRUQCADz74AEePHsXRo0cxevRozWsFBQWhcuXK+P777yv0PolIP+yUDoCIzEteXh5yc3M1NUIfffQROnbsiD59+miuuXHjBgCgXr16JT6P+j71tWV5TFkcO3asxBGlkixevBhnzpzBL7/8ohnheuGFF+Dj44PXXnsNP/zwA3r27IkjR45ACIEvvvgC7u7umsera4Hc3NzQtGlTAECDBg3Qpk2bYq9la2uLZ555BocPHy7nOyQifeKIEBHppE2bNrC3t4erqytefPFFVK1aFd9++y3s7Mr37yqVSqW32FJTU5GRkYGaNWvq9LjvvvsOTZs2RfPmzZGbm6u59ejRAyqVCvv27QMAzbTX66+/js2bN+P27dvlirNmzZrlfiwR6RcTISLSydq1a3HixAn89NNPGDt2LC5evIiBAwcWuqZu3boAHk+baaO+r06dOmV+zJM8evQIAODk5KTT4/7880+cOXMG9vb2hW6urq4QQiA5ORkA0LFjR3zzzTfIzc3FsGHDULt2bTRt2hQbN27U6fWcnJw0sRKRsjg1RkQ6CQwM1Ewfde7cGXl5eVi5ciW2bNmC1157TXPezs4O33zzDcaNG6f1edRF0t26ddM8xt7evtTHPEn16tUBAPfu3dPpcR4eHnB2dsaqVatKvF/tlVdewSuvvIKsrCwcO3YMc+fOxaBBg+Dn54e2bduW6fXu3btX6DmJSDkcESKiCpk/fz6qVq2KGTNmaBoF1qpVC2+88QZ2796N6OjoYo+5fPky5s2bhyZNmmiKo2vVqoXRo0dj9+7dWLt2rdbXunr1Ks6cOVNiLA4ODqhfvz6uXr2q03t4+eWXcfXqVVSvXh3BwcHFbn5+fsUe4+joiE6dOmHevHkAoOlVpO5DVNqIz7Vr1/DUU0/pFCMRGQZHhIioQqpWrYrw8HBMmzYNGzZswJAhQwAACxcuxKVLlzBkyBAcOHAAvXv3hqOjI44dO4YFCxbA1dUVW7duha2trea5Fi5ciGvXrmHEiBHYvXs3+vXrB09PTyQnJyM2NhZRUVHYtGlTqUvon3/+eezatUun9zB58mRs3boVHTt2RGhoKJo1a4b8/HzcuHEDe/bswb///W+0bt0aM2bMwK1bt/DCCy+gdu3aSE1Nxeeffw57e3t06tQJgCySdnZ2xvr16xEYGIjKlSvD29sb3t7eAICUlBRcuXIFb7/9tq5/1URkCAr3MSIiM1FSQ0UhhHj06JGoW7euaNSoUaEGidnZ2WLZsmWidevWonLlysLR0VE0btxYTJs2TSQnJ2t9ndzcXLFmzRrRpUsXUa1aNWFnZydq1KghevbsKTZs2CDy8vJKjXPv3r0CgDh+/HiJ12hrqPjgwQPxwQcfiMaNGwsHBwfh7u4unn76aREaGiqSkpKEEEJ89913omfPnsLHx0c4ODiImjVril69eomDBw8Weq6NGzeKgIAAYW9vLwCImTNnau6LjIwU9vb2muckImVxiw0isjjNmjVD+/btERERoXQoxXTo0AF169Yttr0HESmDiRARWZwffvgB/fr1w5UrV1C7dm2lw9E4cOAAunfvjgsXLqB+/fpKh0NEYLE0EVmgF198EZ988kmFluIbQkpKCtauXcskiMiEcESIiIiIrJaiI0LqlSTe3t5QqVSaviKl2b9/P4KCguDk5IT69evjiy++MHygREREZJEUTYQePnyIZ555BkuXLi3T9QkJCejVqxc6dOiA+Ph4vP/++5g0aRK2bt1q4EiJiIjIEpnM1JhKpcL27dtL3Xn63XffxY4dO3Dx4kXNuXHjxuHXX3/F0aNHjRAlERERWRKzaqh49OhRdO/evdC5Hj16IDIyEjk5ObC3ty/2mKysLGRlZWmO8/Pzce/ePVSvXl2vmz0SERGR4QghcP/+fXh7e8PGRn8TWmaVCCUlJcHT07PQOU9PT+Tm5iI5ORleXl7FHjN37lzMnj3bWCESERGRAd28eVOvbTHMKhECUGwURz2zV9LoTnh4OMLCwjTHaWlpqFu3Lm7evAk3NzfDBUpEREQa27cDI0bo9pjO2IszaIYU1ACQDqAOXF1d9RqXWSVCtWrVQlJSUqFzd+/ehZ2dnWbX6aIcHR01myAW5ObmxkSIiIjoHzExwIwZwP37hnn+27cLH/v4lHytncjB1PQPMeHBPOxz7I6h1XchTwCJiSUPfJSXWSVCbdu2xf/+979C5/bs2YPg4GCt9UFERESWxlAJS9FExZBiYoDXXivhzps3gQEDgCNHAADPj2qIm5/lIj0TcHfXfyyKJkIPHjzA77//rjlOSEjA6dOnUa1aNdStWxfh4eG4ffs21q5dC0CuEFu6dCnCwsIwZswYHD16FJGRkdi4caNSb4GIiMgo1AnQb78Z/rVKG62pCFdX4D//KSUJ+u47YPhw4N49wM0NWLkS6N9f3peZaZCYFE2ETp48ic6dO2uO1bU8w4cPx+rVq5GYmIgbN25o7q9Xrx527tyJ0NBQLFu2DN7e3li8eDH+9a9/GT12IiIiQys4+qNtxEbfCcsTExVDyckBwsOBTz+Vx0FBQHQ00KCBwV/aZPoIGUt6ejrc3d2RlpbGGiEiIlJcaVNdJU1XBQQolLAYyv37QMuWwO+/A++8A8ybBxSp7zXU97dZ1QgRERFZgieN9Gjj46PgiI2huboCmzcD168DpTRWNgQmQkRERHpUlmLmkpIfbVNdFpn8ZGUB06bJqa9Jk+S5Fi3kzciYCBEREelBeYuZLXqkR5urV4GQECAuTk5/vfYa4O2tWDhMhIiIiFDxZem6FjNbVfKjFhMDjB4NpKcD1asDa9YomgQBTISIiMhKPCnR0WcfHYsrZq6ozEwgLAyIiJDH7dsDmzYBetwqo7yYCBERkUUpKeHRJdEp77J0qxzleZLcXKBjR+DECXkcHg7MmQPYmUYKYhpREBER6aA8S84LKinRYSJjAHZ28i/0jz+AdeuAHj2UjqgQ9hEiIiKzEhMDvP562a4tmvAw0TGSjAzg7l3Az08e5+cDyclAzZrlfkr2ESIiIqunLQmymiXn5uLiRfkh5eXJ6TAXF8DGpkJJkCExESIiIrOgLQkqdfNOMr41a4Dx4+WIkKenXCrfrJnSUZXKRukAiIiItImJAQID5cKi2rWZBJm0hw+BESPkLSMDeOEF4PRpk0+CAI4IERGRiShaAF1a0TOTIBNy7pzMUi9elFNgs2YB778P2NoqHVmZMBEiIiJFlaUjs7oOiLU/Jujdd2US5O0NbNgAdOqkdEQ6YSJERESKKWkFGBMfM7JyJTB1KvDZZ0CNGkpHozMmQkREZHBlbXLIjsxm4PRpYNcu2RgRALy8gK+/VjSkimAiREREBlEw+SlLk0PW/Zg4IYAvvgBCQ+Xu8YGBQN++SkdVYUyEiIhIL8pa7Mwmh2YoLQ0YM0Z+yADw8stAhw7KxqQnTISIiKhEuuzIXtqoj48PEx6zdfIkEBICXLsmt8uYN0+OCqlUSkemF0yEiIioEF2ntLRhsbOFWLECmDAByMkBfH2B6GigdWulo9IrJkJERFZM24hPWae0tGHiY2Fq1JBJUN++wKpVQNWqSkekd0yEiIisWFn69zC5sTIPH8r9wQCZAO3bB3TsaDFTYUUxESIisnCl1fkkJso/bWzkKmg1Jj9WSAhg4ULg00+B48flviaA2TVI1BUTISIiC1ZSw8Ki/P1lc2CyUikpcp+w776Tx1FRwIcfKhqSsTARIiKyIE9awq6tzkc9+kNW6vBhYMAA4NYtwNFRdogeN07pqIyGiRARkQUoy35dbFhIheTnA/PnAx98AOTlAY0aAZs3A82bKx2ZUdkoHQAREVWctiTIx0feAgKYBJEWS5bIbTLy8oBBg4C4OKtLggCOCBERmb2YmMdJkI2NrPdhoTM90ZgxwPr1wNixwBtvWOyqsCdhIkREZAZKW/lVsA6IRc9Uorw8YONGOfpjYwNUqgQcOyZ/tmJMhIiITFR5Ojyz6Jm0SkoChgwB9u6VRdHvvSfPW3kSBDARIiIyCRXt8My+P1SivXuBwYOBP/+Uo0BlaRFuRZgIEREprCy9ftjhmXSWlwfMng189JFslti0qVwVFhiodGQmhYkQEZECSpv2KvgPdiY/VC537shaoP375fHo0cDnn8sRISqEiRARkZGUpeaHy9xJL/78Ezh6FKhcGfjyS5kUkVZMhIiIDORJXZ7VOO1FeteiBbBunewL5O+vdDQmjYkQEZGelaXLM5Mf0qubN+VeYfPmAcHB8lxZNpkjJkJERPpUUuGzuu6HyQ/p3fffA8OGAffuAW++KTtEW2lzxPJgAwEiIj2IiZGLcYomQertLW7dkreLF5kEkZ7k5ABTpgAvvyyToKAg+cvGJEgnHBEiIqqA0qbBWPhMBvPHH3LH+F9+kceTJskNVB0dFQ3LHDERIiIqp5KmwQICOP1FBnT5MtC6NZCaClSpAqxaBfTrp3RUZouJEBGRjkoaBWICREbRsCHQti2QkgJERwN+fkpHZNaYCBERPUFZlsFzGowM6to1wNMTcHGR+4Nt2CCbIzo4KB2Z2WOxNBFRCQoWQP/2m0yAiiZB6mJoJkFkMDExsi/Q228/PlelCpMgPeGIEBGRFlwGT4rLzATCwoCICHl86RKQkcFtMvSMiRARkRYzZhQ+Zv0PGdWVKzITP31aHr/3HjBnDmBvr2hYlohTY0REBainwy5fLnyO/X/IaDZuBFq2lEmQhwewaxcwdy6TIAPhiBARWbUnFUIHBDABIiNKSwPeeQd48ADo2FEWRavnY8kgmAgRkVUqy35g6ukwIqNxd5ebpR46BMycCdjxa9rQ+DdMRBar6GhPQdqWwLMQmhSxdq38pVM3RezRQ97IKJgIEZHFetKIjxoLoUkRDx8CEycCq1fLkaBnnwVq11Y6KqvDRIiILFJMzOMkyMYG8PIqfg1Hfkgx587JVWEXL8pf0H//W/svKRkcEyEiskgFl7/7+8vvGyLFCSH3Bps4UfYJ8vKSq8Q6dVI6MqvF5fNEZDHUS99r1y68/J0Fz2QS8vKAoUOB0aNlEtSjh1wizyRIURwRIiKzV9oKMC5/J5NhawtUqyb//OgjYNo0OS1GimIiRERmT1sS5OPzuAaISDFCyKLoypXl8SefyFGhZ59VNi7SYCJERGZPvTzexkbWA7EAmkxCWhrw5ptAUhKwd6/sCeToyCTIxDARIiKzoq03UGKi/NPLi0XRZCLi4oCQEODqVZkAHTsGPPec0lGRFkyEiMislNYbyNXVuLEQFSMEsHQpMGUKkJ0N+PoCmzYBbdooHRmVgIkQEZmkkrpCq0d/ivYGYj0QKe7vv4FRo4Dt2+Vx375yqXzVqoqGRaVjIkREJicmRvaaKw17A5HJGToU+P57uUv8ggXA228DKpXSUdETMBEiIpNTsBkiUHzzbY7+kEmaNw+4fh2IigKCg5WOhsqIiRARmYSCU2Hq6S/1ea4AI5N07x6wbx/w6qvyuEkT4Ndf2RvIzDARIiLFFEx+tO0Gz2aIZLKOHAEGDADu3AH27wfat5fnmQSZHSZCRGQwJRU8q2lLfgA2QyQTlp8vmyJOny63zGjUCHBxUToqqgDFU9fly5ejXr16cHJyQlBQEA4ePFjq9evXr8czzzyDSpUqwcvLCyNHjkRKSoqRoiWislIXPP/2m0x4tN0K8vGRI0AxMcCtW7IQmqNBZFL++gt4+WXgvfdkEjRwoOwX1Ly50pFRBSiaCEVHR2Py5MmYPn064uPj0aFDB/Ts2RM3btzQev2hQ4cwbNgwjBo1CufPn0dMTAxOnDiB0aNHGzlyIiqNtlVfPj7ab0x+yCwcOCATnl27ACcnYMUKYP16Nq+yACohhFDqxVu3bo2WLVsiIiJCcy4wMBB9+/bF3Llzi12/YMECRERE4OrVq5pzS5Yswfz583Hz5s0yvWZ6ejrc3d2RlpYGNze3ir8JIiomMLBw00MWPJPZW7IEmDRJZu6bNwNPP610RFbHUN/fio0IZWdnIy4uDt27dy90vnv37jhy5IjWx7Rr1w63bt3Czp07IYTAn3/+iS1btuCll14q8XWysrKQnp5e6EZEhhMTwySILETBcYKJE2UydOIEkyALo1gilJycjLy8PHh6ehY67+npiaSkJK2PadeuHdavX4+QkBA4ODigVq1aqFKlCpYsWVLi68ydOxfu7u6aW506dfT6PojosaJTYlz1RWbrp5+Ajh0B9T+eVSqZDKl3kSeLoXixtKpI100hRLFzahcuXMCkSZMwY8YMxMXF4YcffkBCQgLGjRtX4vOHh4cjLS1NcyvrFBoR6UZbXRBXfZHZycsDZs4EunYFDh0CPvpI6YjIwBRbPu/h4QFbW9tioz93794tNkqkNnfuXLRv3x5Tp04FADRr1gwuLi7o0KEDPvroI3gV3HjoH46OjnB0dNT/GyCiQop2g+aUGJmdO3eAwYNlk0RA7hs2a5aSEZERKDYi5ODggKCgIMTGxhY6Hxsbi3bt2ml9TEZGBmyKNKuytbUFIEeSiMj4YmJkcfTly4XPMQkis7Jnj1wVtm+f7Av09dfAypVApUpKR0YGpmhDxbCwMAwdOhTBwcFo27YtvvrqK9y4cUMz1RUeHo7bt29j7dq1AIDevXtjzJgxiIiIQI8ePZCYmIjJkyejVatW8Pb2VvKtEFmV0jpCsy6IzM66dcCwYfLnZ56Rq8L8/ZWNiYxG0UQoJCQEKSkpmDNnDhITE9G0aVPs3LkTvr6+AIDExMRCPYVGjBiB+/fvY+nSpfj3v/+NKlWqoEuXLpg3b55Sb4HIqqgToIKrwgoKCGBdEJmhF18EvL2BPn2AhQsBZ2elIyIjUrSPkBLYR4hIN0/aD6zgdhgcCSKzceYM0KzZ4+OUFKB6deXioSeyuD5CRGT6im6TURA7QpNZyskBpk6VU2Bff/34PJMgq8VNV4moREVXgnH0h8za9etyx/hjx+TxuXPKxkMmgYkQEWnFDtFkUb79FhgxAkhNBdzdgVWrgFdfVToqMgGcGiOiQtTL4dkhmixCdjYweTLQt69Mgp59FoiPZxJEGhwRIiINbd2hAa4EIzN29Cjw+efy57AwYO5cwMFB2ZjIpDARIrJiBVeEAdoLolkPRGatUyfg44/lRqm9eysdDZkgLp8nsmKBgSX3BGJNEJmlzEzggw+ACROAevWUjob0iMvniajC1PU/tWvLm3pbDBsbuSLMx+fxsngmQWR2rlwB2rUDPv0UGDQIyM9XOiIyA5waI7ICT+oI7e8vewERma2NG4E33wQePAA8POQO8jb8tz49GRMhIiugLQny8ZF/qvsCEZmlR4+Ad94BVqyQxx07Ahs2PP4FJ3oCJkJEFq5gPyAbGzn6wwJosgi3bgG9egFnzwIqFTB9uhwJsuNXG5Udf1uILJS26TBOgZFF8fCQSU/NmsD69UDXrkpHRGaIiRCRhdI2HcYpMDJ7GRmAoyNgaws4OQFbt8o/vbyUjozMFCvJiCxQ0ekwrgQji3D+vOwMPWfO43P16jEJogphIkRkgQpulqqeDmMSRGZLCLk32LPPAhcuAJGRj7uAElUQEyEiC6LuE6TuDwRwOozM3IMHwNChwKhRcoVY9+7AqVNyuSORHrBGiMjMFdwmQ9sWGRwJIrP1669y87vLl+Uc73/+A7z3HvsDkV4xESIyYyVtkgo83ieMyCw9eAB06QLcuyd7Am3cCHTooHRUZIGYCBGZKW1JkI/P4waJHAkis1a5MvDJJ3JV2Jo1cqk8kQFw01UiM6QtCeKqMDJ7p04BublAq1byWP31pFIpFxOZDG66SkSaYmgmQWRRhACWLgXatpW/yPfuyfMqFZMgMjhOjRGZiZLqgZgEkVlLTZUrwrZtk8ctWzL5IaPiiBCRGdCWBLFJIpm948eBFi1kEmRvDyxaBGzfDlStqnRkZEU4IkRkBgo2SASYAJGZE0ImPe++C+TkyO7Q0dGyYSKRkXFEiMiEaWuQyCSILMKBAzIJ+te/ZJE0kyBSCEeEiEyQtp3jATZIJDMnxOMC6FWrgJdfBt54gzVBpCiOCBGZGHU9kLYkiA0SySzl5wPz5wPDhz9eEl+1qiySZhJECuOIEJEJeNI2GWyQSGbrr79kArRrlzweOhTo1k3ZmIgKYCJEpKCSpsAK3s8EiMzWwYPAgAHAnTuAkxPw+edA165KR0VUCBMhIoWU1BeI22SQ2cvPB+bOlVl+fj7QuDGweTPQrJnSkREVw0SISCFFl8RzCowsxsiRwNq18uehQ4Hly+XeYUQmiMXSREZW0pL4ixeZBJGFGDlSJj5RUTIhYhJEJoybrhIZWWBg4ZqggACZBBGZrbw84Pz5wlNf9+4B1aopFxNZHG66SmQBYmIeJ0E2NlwSTxYgMVEWQLdvX3iYk0kQmQnWCBEZUcG6IH9/jgSRmduzBxgyRC6Rd3GRiZC/v9JREemEI0JERnT//uOfORJEZis3F5g+HXjxRZkENWsGnDwpO0UTmRmOCBEZSUzM42aJPj4sjCYzdesWMGiQ7BEEAGPHAp99Bjg7KxsXUTkxESIykoLTYq6uysVBVCErVsgkyNUV+Oor2TCRyIwxESIygoJF0gCnxciMffCBLJCeNg1o2FDpaIgqjDVCRAak7hlUsIM0d5Ans3LjBjBhApCTI4/t7eVIEJMgshAcESLSs9I2UAU4GkRmZMcOYMQI4O+/5W7xH32kdEREescRISI9U2+iqm0XeW6iSmYhOxsIDQVeeUUmQc8+C4wapXRURAbBRIhIT4punWFjI1eHqRMgbqFBZiEhAXjuOWDRInkcGgocOgTUq6doWESGUq6psdzcXOzbtw9Xr17FoEGD4Orqijt37sDNzQ2VuacMWSFtO8mzYSKZndhYoH9/IC1NToWtXg306aN0VEQGpXMidP36dbz44ou4ceMGsrKy0K1bN7i6umL+/PnIzMzEF198YYg4iUySuh6o4IowgFtnkJny85P7hrVtC2zaBNStq3RERAan89TYO++8g+DgYPz9999wLtBAq1+/fti7d69egyMyddqSIE6DkVlJS3v8c6NGwP798sYkiKyEzonQoUOH8MEHH8DBwaHQeV9fX9zWtkSGyEJp20CVxdBkVjZtkqNAP//8+FzLlnKJPJGV0HlqLD8/H3l5ecXO37p1C65sl0tWhBuoktl69AiYPFn2AwKAiAigc2dFQyJSis4jQt26dcMi9WoCACqVCg8ePMDMmTPRq1cvfcZGZLLYKZrM1qVLQJs2MglSqeTmqRs2KB0VkWJUQgihywPu3LmDzp07w9bWFleuXEFwcDCuXLkCDw8PHDhwADVr1jRUrHqRnp4Od3d3pKWlwc3NTelwyAwVXSEWEMDRIDITX38NjBsHPHwI1KgBrF8PdOumdFREZWKo72+dp8a8vb1x+vRpbNq0CXFxccjPz8eoUaMwePDgQsXTROauYIfogoqWwnE0iMzC/v3A0KHy586dZRLk5aVsTEQmQOcRoQMHDqBdu3awsyucQ+Xm5uLIkSPo2LGjXgPUN44IUVkFBhZfEVYUi6PJbAgBDB8O1K8PfPghYGurdEREOjHU97fOiZCtrS0SExOLTYGlpKSgZs2aWgupTQkTISqLgtNfNjbF/+Hs6ipHgpgEkckSAti4EXjxRaBatcfnVCpl4yIqJ5OZGhNCQKXlP6SUlBS4uLjoJSgipWhrkMgVYWR2HjwAxo8H1q2T+4Vt3y4TICZBRMWUORF69dVXAchVYiNGjICjo6Pmvry8PJw5cwbt2rXTf4RERqStQSJrgMisnDkjhzMvXZLDma1acSSIqBRlToTc3d0ByBEhV1fXQoXRDg4OaNOmDcaMGaP/CImMQD0SVHDDVH9/Tn+RGRECWLECeOcdIDNT7vi7cSPQoYPSkRGZtDInQlFRUQAAPz8/TJkyhdNgZFE4HUZmLT0dGDtWdooGgJ49gbVrAQ8PZeMiMgM6N1ScOXMmkyCyKNq2yuB0GJmV3FzgyBG5Emz+fOC775gEEZWRzsXSALBlyxZs3rwZN27cQHZ2dqH7Tp06pZfAiIyFW2WQWVIv+FWp5KqwmJjHO8cTUZnpPCK0ePFijBw5EjVr1kR8fDxatWqF6tWr49q1a+jZs6chYiQyiJgY2StIXRcEcCSIzERqKtC/PxAZ+fhcq1ZMgojKQec+QgEBAZg5cyYGDhwIV1dX/Prrr6hfvz5mzJiBe/fuYenSpYaKVS/YR4i0LZEHuFUGmYkTJ4CQECAhAXBzA65fB6pUUToqIoMz1Pe3ziNCN27c0CyTd3Z2xv1/9h8YOnQoNm7cqLfAiAylpCSIo0Fk0oQAFi0C2reXSZCfHxAbyySIqIJ0ToRq1aqFlJQUAICvry+OHTsGAEhISICOg0tERlV0KkxdGB0TI0eCuEyeTNa9e0DfvkBoKJCTA7z6KhAfL6fDiKhCdE6EunTpgv/9738AgFGjRiE0NBTdunVDSEgI+vXrp3MAy5cvR7169eDk5ISgoCAcPHiw1OuzsrIwffp0+Pr6wtHREQ0aNMCqVat0fl2yPuqRoPx8eawujGYCRCYtIwMIDgZ27AAcHIClS4EtWzgSRKQnOq8a++qrr5D/zzfJuHHjUK1aNRw6dAi9e/fGuHHjdHqu6OhoTJ48GcuXL0f79u3x5ZdfomfPnrhw4QLq1q2r9TGvv/46/vzzT0RGRqJhw4a4e/cucnNzdX0bZGWKLpFXN0skMnmVKgHDhgFffw1s3gy0bKl0REQWRedi6dLcvn0bPj4+Zb6+devWaNmyJSIiIjTnAgMD0bdvX8ydO7fY9T/88AMGDBiAa9euoZp6E0EdsVjaOhXcSZ5F0WTykpPlfmF+fvI4L0+ODLm6KhoWkZJMplham6SkJLz99tto2LBhmR+TnZ2NuLg4dO/evdD57t2748iRI1ofs2PHDgQHB2P+/Pnw8fGBv78/pkyZgkePHpX4OllZWUhPTy90I+ugrgmqXZtL5MmMHDwING8O9Osnt8oAZKNEJkFEBlHmRCg1NRWDBw9GjRo14O3tjcWLFyM/Px8zZsxA/fr1cezYMZ1qdZKTk5GXlwdPT89C5z09PZGUlKT1MdeuXcOhQ4dw7tw5bN++HYsWLcKWLVswYcKEEl9n7ty5cHd319zq1KlT5hjJfMXEyH0nf/sNuH37cV1QQABrgshE5ecD//0v0Lmz/KXNyABK+H8hEelPmWuE3n//fRw4cADDhw/HDz/8gNDQUPzwww/IzMzErl270KlTp3IFoCqyI7IQotg5tfz8fKhUKqxfv16zCezChQvx2muvYdmyZYU2glULDw9HWFiY5jg9PZ3JkIVTJ0EF+fjIf1BzNIhM0t27wJAhcjk8IH+OiAAqV1Y2LiIrUOZE6Pvvv0dUVBS6du2K8ePHo2HDhvD398eiRYvK9cIeHh6wtbUtNvpz9+7dYqNEal5eXvDx8dEkQYCsKRJC4NatW2jUqFGxxzg6OsLR0bFcMZL50ZYExcRwFIhM2M8/A4MGydEfZ2dg2TJgxAi5dQYRGVyZp8bu3LmDp556CgBQv359ODk5YfTo0eV+YQcHBwQFBSFW/S+gf8TGxmoaNhbVvn173LlzBw8ePNCcu3z5MmxsbFC7du1yx0KWgUkQmR0hgA8/lEnQU0/JrtEjRzIJIjKiMidC+fn5sLe31xzb2tpWeBf6sLAwrFy5EqtWrcLFixcRGhqKGzduaJbhh4eHY9iwYZrrBw0ahOrVq2PkyJG4cOECDhw4gKlTp+KNN97QOi1G1qXg5qkAkyAyAyoVsH49MHEicPw40KSJ0hERWZ0yT40JITBixAjNNFNmZibGjRtXLBnatm1bmV88JCQEKSkpmDNnDhITE9G0aVPs3LkTvr6+AIDExETcuHFDc33lypURGxuLt99+G8HBwahevTpef/11fPTRR2V+TbI86r3DCq4MYxJEJuvHH+XIT3i4PPb1BZYsUTYmIitW5j5CI0eOLNMTRkVFVSggQ2MfIctTsEcQwD5BZKJyc4FZs+TKMCFkQvTCC0pHRWQ2DPX9XeYRIVNPcMj6FB0JYsdoMlm3b8uC6AMH5PHYsUAJtZBEZFw6b7FBZAq0FUar9w4jMim7dsktMpKTZQ+Hr74CBgxQOioi+odeOksTGVvRwuiAAI4EkQmaMwfo1UsmQS1aAHFxTIKITAxHhMjsFNxAVX3MwmgySepthyZMABYsAJyclI2HiIphIkRmpeiUGLfMIJOTmgpUqSJ/HjRIztkGBysZERGVglNjZFaKTolxOoxMRnY2EBYmewHdvfv4PJMgIpNWrkRo3bp1aN++Pby9vXH9+nUAwKJFi/Dtt9/qNTgiNfVO8uwVRCYpIQHo0AH47DPgzh1gxw6lIyKiMtI5EYqIiEBYWBh69eqF1NRU5OXlAQCqVKlS7n3HiJ5kxgxZF8Rd5MnkbNsmC6GPHweqVgW+/RaowPZDRGRcOidCS5YswYoVKzB9+nTY2tpqzgcHB+Ps2bN6DY5I7f59+aeNDVeIkYnIygLefhv417+AtDSgTRsgPh7o00fpyIhIBzoXSyckJKBFixbFzjs6OuLhw4d6CYqooJgY2Y8OALy82CuITMRHHwFLl8qfp04FPv4YKLAfIxGZB51HhOrVq4fTp08XO79r1y7N7vRE+lSwQNrVVbk4iAqZOlWOAn33HTB/PpMgIjOl84jQ1KlTMWHCBGRmZkIIgePHj2Pjxo2YO3cuVq5caYgYyUpp20yVU2KkmMxMYM0a4M035a7xbm7AkSPyZyIyWzonQiNHjkRubi6mTZuGjIwMDBo0CD4+Pvj8888xgB1TSU+0baHBAmlSzKVL8hfyzBlZGzRpkjzPJIjI7JWroeKYMWMwZswYJCcnIz8/HzVr1tR3XGSl1KNABTtHAyyQJgWtXy83SX34EKhZU/ZxICKLoXON0OzZs3H16lUAgIeHB5Mg0ittSVBMjCyQ5mgQGVVGhlwGP2SITII6dwZOnwa6dVM6MiLSI50Toa1bt8Lf3x9t2rTB0qVL8ddffxkiLrJSRZfJs2kiKeLCBaBVKyAyUk5/zZwJxMbKZYtEZFF0ToTOnDmDM2fOoEuXLli4cCF8fHzQq1cvbNiwARkZGYaIkayQepk8kyBSRGqqHJqsVQv48Udg1iygQN80IrIcKiGEqMgTHD58GBs2bEBMTAwyMzORnp6ur9gMIj09He7u7khLS4Obm5vS4VABBQukfXyAW7eUjYesjBCFi59jYoCOHQFPT+ViIiINQ31/V3jTVRcXFzg7O8PBwQE5OTn6iImsFPsFkWLOnpWbo5479/hc//5MgoisQLkSoYSEBHz88cd46qmnEBwcjFOnTmHWrFlISkrSd3xkJWJiChdJc4UYGYUQwIoVsh7o1CkgNFTpiIjIyHRePt+2bVscP34cTz/9NEaOHKnpI0RUHtqWy7NfEBlFerpcFr9pkzzu2RNYu1bZmIjI6HROhDp37oyVK1eiSZMmhoiHrIi2pokAR4PICOLj5S/f77/LIuj//heYMkUuVyQiq1LhYmlzw2Jp01BS5+j//IejQWRgx48DHToA2dlAnTpyRKhdO6WjIqInMNT3d5lGhMLCwvCf//wHLi4uCAsLK/XahQsX6iUwsmwFC6MB9gsiIwoKAtq2lXuFrV4NVKumdEREpKAyJULx8fGaFWHx8fEGDYgsX9HCaCZBZHC//go0bgw4OcmpsB075NJE7hVGZPU4NUZGFxj4OBEKCJCNE4kMQghg8WJg6lS5a/zSpUpHRETlZDJ9hN544w3cV++DUMDDhw/xxhtv6CUosjwxMTIBql0buHz58XkWRpPB/P038OqrwOTJQE4OkJQE5OYqHRURmRidR4RsbW2RmJhYbLPV5ORk1KpVC7km/j8ajggZX0mrwzgaRAZz7BgwYABw/Trg4AAsXAiMH8+pMCIzpmixtDoAIQSEELh//z6cnJw09+Xl5WHnzp3ciZ60KloY7eMjyzM4GkR6l58vk57wcDn606ABsHkz0LKl0pERkYkqcyJUpUoVqFQqqFQq+Pv7F7tfpVJh9uzZeg2OLEPBmVQWRpNBJSUBH38sk6CQEOCrr+TqMCKiEpQ5Efr5558hhECXLl2wdetWVCuw5NTBwQG+vr7w9vY2SJBkntRdoxMT5bGPD5MgMjBvb7kkPilJFkdzKoyInkDnGqHr16+jbt26UJnp/2BYI2Q8BVeHAawJIgPIzwfmzQOaN5dbZBCRxVK0RujMmTNo2rQpbGxskJaWhrNnz5Z4bbNmzfQWHJmvgr2CbGwAf3/WBJGe3b0LDB0K7NkDVK8OXLok/yQi0kGZEqHmzZsjKSkJNWvWRPPmzaFSqaBtIEmlUiEvL0/vQZL5KVgg7e/PkSDSs337gEGD5LyrszMwfz47RBNRuZQpEUpISECNGjU0PxM9ScECaY4Ekd7k5cli6Nmz5bTYU0/JVWHcBJqIyqlMiZCvr6/Wn4m0iYkBbt+WP7NAmvQmMxN46SXgp5/k8ciRwJIlgIuLsnERkVnTubP0mjVr8P3332uOp02bhipVqqBdu3a4fv26XoMj81O0eaKrq3KxkIVxcgL8/GTis3YtsGoVkyAiqjCdE6H//ve/cHZ2BgAcPXoUS5cuxfz58+Hh4YHQ0FC9B0jmQ1sHaU6LUYXk5gJpaY+PlywBTp2SRdJERHpQ5j5Cajdv3kTDhg0BAN988w1ee+01vPnmm2jfvj2ef/55fcdHZqRoB2k2T6QKuX1bFkQ7OwM7d8rlh5Uqyep7IiI90XlEqHLlykhJSQEA7NmzB127dgUAODk54dGjR/qNjswKO0iT3vzwg+wNdOAAcPgwlx0SkcHoPCLUrVs3jB49Gi1atMDly5fx0ksvAQDOnz8PPz8/fcdHZoIF0qQXOTnAhx/KJokA0KIFEB0NNGqkbFxEZLF0HhFatmwZ2rZti7/++gtbt25F9X8amMXFxWHgwIF6D5BMHwukSS9u3gSef/5xEjRhAnDkCJMgIjIonbfYMHfcYkO/tBVIc1qMdCYE0LYt8MsvcpPUyEj+EhFRIYpusVFUamoqIiMjcfHiRahUKgQGBmLUqFFwd3fXW2BkHlggTXqhUgEREcCkScCaNUD9+kpHRERWQuepsZMnT6JBgwb47LPPcO/ePSQnJ+Ozzz5DgwYNcOrUKUPESCaMBdJUbn/8AWzZ8vi4RQtZHM0kiIiMSOepsQ4dOqBhw4ZYsWIF7OzkgFJubi5Gjx6Na9eu4cCBAwYJVF84NaY/BafFfHyAW7eUjYfMyPbtwBtvABkZsg4oKEjpiIjIxJnM1NjJkycLJUEAYGdnh2nTpiE4OFhvgZFpY4E0lUtWFjB1qmyMCABt2gAeHsrGRERWTeepMTc3N9y4caPY+Zs3b8KV34ZWo2htEDtI0xNdvQq0b/84CZo6VU6Fcf9CIlKQzolQSEgIRo0ahejoaNy8eRO3bt3Cpk2bMHr0aC6ftyKsDSKdxMQALVsCcXFA9erAd98B8+cD9vZKR0ZEVk7nqbEFCxZApVJh2LBhyM3NBQDY29vjrbfewv/93//pPUAybWyeSGVy9SqQng489xywcSNQu7bSERERAahAH6GMjAxcvXoVQgg0bNgQlSpV0ndsBsFi6YpjkTSViRByWTwA5OfLZfFDhwJ25eraQURWzlDf32WeGsvIyMCECRPg4+ODmjVrYvTo0fDy8kKzZs3MJgmiimORNJXJ+vWyQeLDh/LYxgYYOZJJEBGZnDInQjNnzsTq1avx0ksvYcCAAYiNjcVbb71lyNjIxGjrIs0iaSokIwMYPRoYMkR2iV6+XOmIiIhKVeZ/nm3btg2RkZEYMGAAAGDIkCFo37498vLyYGtra7AAyXSwizSV6uJFmSmfOyenxGbMAMLClI6KiKhUZR4RunnzJjp06KA5btWqFezs7HDnzh2DBEamJSYG+O23wsdMgkhjzRogOFgmQbVqAT/+CMyaBfAfSURk4sqcCOXl5cHBwaHQOTs7O83KMbJcRafEAgKYBFEBCxYAI0bIabGuXYHTp4EuXZSOioioTMo8NSaEwIgRI+Do6Kg5l5mZiXHjxsHFxUVzbtu2bfqNkBQTEyNnNwqOBAGsC6IiBg4EPv0UmDgReO89jgIRkVkpcyI0fPjwYueGDBmi12DItGhLgjglRhACOHZMrgoDZA+Fy5e5hJCIzFKZE6GoqChDxkEmpmBNkI0N4O8vR4KYBFm5+/eBceOADRuArVuBV1+V55kEEZGZYlMPKqZoTZC/v1wQRFbu9Gn5i3Hlipz+un1b6YiIiCpM573GyLKxVxAVIwQQESF3ir9yBahTR26W+vbbSkdGRFRhHBEiDW1JEGuCrFxaGjBmjPxFAIDevYGoKLlxKhGRBeCIEAFgEkQlOHBA/iLY2QELFwLffsskiIgsiuKJ0PLly1GvXj04OTkhKCgIBw8eLNPjDh8+DDs7OzRv3tywAVoJdo0mrXr3Bj76CDh8GAgNfbyJKhGRhShXIrRu3Tq0b98e3t7euH79OgBg0aJF+Pbbb3V6nujoaEyePBnTp09HfHw8OnTogJ49e+LGjRulPi4tLQ3Dhg3DCy+8UJ7wqQh2jSaNv/8GRo0qXAg9fTrQqpVyMRERGZDOiVBERATCwsLQq1cvpKamIi8vDwBQpUoVLFq0SKfnWrhwIUaNGoXRo0cjMDAQixYtQp06dRAREVHq48aOHYtBgwahrbqPCZUbu0aTxi+/AC1aAKtWyZ3iiYisgM6J0JIlS7BixQpMnz690GarwcHBOHv2bJmfJzs7G3FxcejevXuh8927d8eRI0dKfFxUVBSuXr2KmTNnlul1srKykJ6eXuhGjxWdEuMKMSskhOwM/dxzwPXrQIMGwNy5SkdFRGQUOidCCQkJaNGiRbHzjo6OePjwYZmfJzk5GXl5efD09Cx03tPTE0lJSVofc+XKFbz33ntYv3497OzKtuBt7ty5cHd319zq1KlT5hgtWUwMEBgoGwIXPMfRICuTkgL06QNMmQLk5srhwbg4IChI6ciIiIxC50SoXr16OH36dLHzu3btwlNPPaVzAKoixZdCiGLnALnp66BBgzB79mz4+/uX+fnDw8ORlpamud28eVPnGC2RevuM/Hx5zCkxK3TxItC8OfDdd4CjI/DFF8CmTYC7u9KREREZjc59hKZOnYoJEyYgMzMTQggcP34cGzduxNy5c7Fy5coyP4+HhwdsbW2Ljf7cvXu32CgRANy/fx8nT55EfHw8Jk6cCADIz8+HEAJ2dnbYs2cPumjZ8drR0bHQRrFU8vYZZGXq1gXc3OQvwObNwDPPKB0REZHR6ZwIjRw5Erm5uZg2bRoyMjIwaNAg+Pj44PPPP8eAAQPK/DwODg4ICgpCbGws+vXrpzkfGxuLV155pdj1bm5uxWqQli9fjp9++glbtmxBvXr1dH0rVonbZ1i5e/eAKlVkBuziIkeDPDy4VxgRWa1ydZYeM2YMxowZg+TkZOTn56NmzZrlevGwsDAMHToUwcHBaNu2Lb766ivcuHED48aNAyCntW7fvo21a9fCxsYGTZs2LfT4mjVrwsnJqdh5KhmLo63Y/v3AwIHA5MnAtGnyHP8BQURWrkINFT08PMqdBAFASEgIFi1ahDlz5qB58+Y4cOAAdu7cCV9fXwBAYmLiE3sKUdmwONqK5eXJjLdLFyAxEVi/HsjJUToqIiKToBJCCF0eUK9ePa3FzGrXrl2rcFCGlJ6eDnd3d6SlpcHNzU3pcIxC2/YZAQGcErMKSUnAkCHA3r3yeMQIYOlSOS1GRGRGDPX9rfPU2OTJkwsd5+TkID4+Hj/88AOmTp2qr7hIj4pOhwUEcErMKuzdCwweDPz5J1CpktxBftgwpaMiIjIpOidC77zzjtbzy5Ytw8mTJyscEOnf/fuPf+Z0mJX480/g5ZeBzEygaVP5wQcEKB0VEZHJ0dumqz179sTWrVv19XRkAD4+TIKshqcnMH8+MGYMcPw4kyAiohKUa9WYNlu2bEG1atX09XREpKvdu4GaNeV+YQAwcSJ3iyciegKdE6EWLVoUKpYWQiApKQl//fUXli9frtfgqGJiYmR9UGKi0pGQQeXmAh9+CPzf/8l9wk6dko0SmQQRET2RzolQ3759Cx3b2NigRo0aeP755xHA4XeTot5GQ4098yzQzZuyN9Dhw/K4Rw/AwUHZmIiIzIhOiVBubi78/PzQo0cP1KpVy1AxkR5wGw0r8P33chXYvXtyBGjlSqB/f6WjIiIyKzolQnZ2dnjrrbdwkQ1oTBq30bBwublAeDiwYIE8Dg4GoqOB+vWVjYuIyAzpvGqsdevWiI+PN0QspCfcRsPC2dgA6n333nkHOHSISRARUTnpXCM0fvx4/Pvf/8atW7cQFBQElyIdaps1a6a34Kh82DfIQuXnyyTIxgZYuxb45Regd2+loyIiMmtl3mLjjTfewKJFi1ClSpXiT6JSQQgBlUqFvLw8fceoV9awxUbt2sDt27Jv0K1bSkdDFZaVBUydCmRkyDogIiIrZKjv7zInQra2tkhMTMSjR49KvU69YaqpsvREqGB9EBMhC3D1KhASAsTFyeNTpx73CSIisiKK7zWmzpdMPdGxZkWLpLlc3szFxACjRwPp6UC1anI6jEkQEZFe6VQsXdqu86QsbTvMs0jaTGVmAuPHyw80PR1o3x44fRp46SWlIyMisjg6FUv7+/s/MRm6d+9ehQIi3WlLglgkbcb69AFiY+XP4eHAnDmAnd52wyEiogJ0+r/r7Nmz4e7ubqhYqJyKLpdnEmTmQkOBX3+VU2E9eigdDRGRRdMpERowYABq1qxpqFioHAp2kFYfMwkyMxkZwIULsjEiAPTsCVy7BhRpTUFERPpX5hoh1geZlpgYIDCw8JRYQACTILNz8SLQujXQrRvwxx+PzzMJIiIyijInQmVcZU9GUnRDVYDF0WZnzRo5CnTuHODoCCQmKh0REZHVKXMilJ+fz2kxE6LuHm1jI0eCOCVmRh4+BEaMkLeMDOCFF+SqsLZtFQ6MiMj6cCmKGYqJkZ2jAcDLixuqmpVz5+R85sWLMoudPVuuDLO1VToyIiKrxETIDBVcJcamiWZm5UqZBHl7Axs2AJ06KR0REZFVYyJkhgpuqsq6IDPzf/8n/5w+HahRQ9lYiIhIt87SpLyC02I+PqwLMnmnTwOjRgHqzYidnIBFi5gEERGZCCZCZobTYmZCCCAiAmjTBli1Cvj0U6UjIiIiLTg1ZmY4LWYG0tKAN98ENm+Wxy+/LEeFiIjI5HBEyExxWsxExcUBLVvKJMjOTo4E7dgBVK+udGRERKQFR4SI9GXDBmDkSCA7G/D1BaKjZddoIiIyWRwRMhPqLTXYfNiENWsm+wH16wfExzMJIiIyAxwRMhNFt9RgobSJuHsXUHdcb9oUOHlSZqzcm4+IyCxwRMgMFNxhXr2lBgulFZafL+t//PyAo0cfn3/qKSZBRERmhCNCZqDgknl/f26pobiUFGD4cOD77+VxdDT3CSMiMlMcETJh6rqgy5cfn+NIkMIOHwaaN5dJkKOj7BX02WdKR0VEROXEESETFRMj9+YsKCCAS+YVk58PzJ8PfPCB7BLdqJFcIt+8udKRERFRBXBEyEQVnA4DWBekuG++kbvE5+UBgwbJfkFMgoiIzB5HhExITIxMgO7fL7xMPiaGI0GK69dPJkCdO8su0SyIJiKyCEyETIA6ASq4PF6N02EKycsDli0DRowA3Nxk4rN+vdJRERGRnjERMgHakiAfH9kriNNhCkhKAoYMAfbulUvjN2zgCBARkYViImQC1Bup2tjI5fH/+Q9HgRSzdy8weDDw559ApUrAiy8yCSIismBMhBQWEwPcvi1/9vJijyDF5OUBc+bILFQI2SU6Olo2SCQiIovFREhBRZfIc9sMhSQlAQMGAPv3y+PRo4HPP5cjQkREZNGYCCmo6BJ51gMpxMZGdq2sXBn48ku5OoyIiKwCEyEFqWuDAC6RN7r8fJkAAXLT1K1bgerVZZEWERFZDTZUVEjB2iAfHyZBRnXzJtCxo1wNpta2LZMgIiIrxERIIQWnxVgbZETffSc7Qh8+DEybBmRlKR0REREpiImQQgpOi7E2yAiys4EpU4DevYF794CgIFkc7eiodGRERKQg1ggpgNNiRvbHH3JV2C+/yONJk+QGqkyCiIisHhMhBXBazIhSUuToz717QJUqwKpVct8wIiIiMBFSBKfFjKh6dblJ6v79skGin5/SERERkQlhImRknBYzgmvXADs7oG5defzxx7JbtIODsnEREZHJYbG0EbGTtBFs2QK0aAGEhAA5OfKcvT2TICIi0oqJkJEUTYIATovpVWYmMH480L8/kJ4umyWmpSkdFRERmTgmQkZSdDsNdpLWoytXZEPEiAh5/N57wL59gIeHomEREZHpY42QEcTEAL/9VviYSZCebNwIvPkm8OCBTHzWrQNefFHpqIiIyEwwETKCgqNBAQFMgvQmN1f2A3rw4PGWGT4+SkdFRERmhImQgRUdDWJdkB7Z2QGbN8sEaPp0eUxERKQDlRBCKB2EMaWnp8Pd3R1paWlwc3Mz+OsFBj5OhAICgIsXDf6Slm3tWiAxEXj3XaUjISIiIzLU9zf/CW1gbJ6oJw8fAhMnAqtXAyoV0KUL8OyzSkdFRERmjomQkbB5YgWcOyd7D1y8KJfFz5oFtGypdFRERGQBmAgZUMEu0lQOQsi9wd5+G3j0CPDykvVAzz+vdGRERGQhmAgZEDdXraCxY4EVK+TPPXrI+qCaNZWNiYiILAobKhoQ64MqqFUrwNYWmDsX2LmTSRAREekdR4SMgPVBZSQEcPcu4Okpj0eNAp57Ti63IyIiMgDFR4SWL1+OevXqwcnJCUFBQTh48GCJ127btg3dunVDjRo14ObmhrZt22L37t1GjJYMJj0dGDBAjgL9/bc8p1IxCSIiIoNSNBGKjo7G5MmTMX36dMTHx6NDhw7o2bMnbty4ofX6AwcOoFu3bti5cyfi4uLQuXNn9O7dG/Hx8UaOnPQqLk6uAtu8GbhzByglGSYiItInRRsqtm7dGi1btkSEerNMAIGBgejbty/mzp1bpudo0qQJQkJCMKPorqYlMFZDxYK7zfv4ALduGeylzJcQwNKlwJQpQHY24OsLbNoEtGmjdGRERGRiDPX9rdiIUHZ2NuLi4tC9e/dC57t3744jR46U6Tny8/Nx//59VKtWrcRrsrKykJ6eXuhmaAWTIIArxrT6+2/gX/8CJk2SSVDfvkB8PJMgIiIyKsUSoeTkZOTl5cFTXRj7D09PTyQlJZXpOT799FM8fPgQrxfMOoqYO3cu3N3dNbc6depUKO4nKZoEAVwxptX77wPbtwP29sDnnwPbtgFVqyodFRERWRnFi6VVKlWhYyFEsXPabNy4EbNmzUJ0dDRqlrKsOjw8HGlpaZrbzZs3KxxzaYrO0MXEcMWYVh9/DHTuDBw5IkeFyvCZExER6Ztiy+c9PDxga2tbbPTn7t27xUaJioqOjsaoUaMQExODrl27lnqto6MjHB0dKxxvWRXsHcQkqIB792RDxHfekUlPtWrATz8pHRUREVk5xUaEHBwcEBQUhNjY2ELnY2Nj0a5duxIft3HjRowYMQIbNmzASy+9ZOgwdVJwSw32DirgyBGgeXMgNBSIjFQ6GiIiIg1FGyqGhYVh6NChCA4ORtu2bfHVV1/hxo0bGDduHAA5rXX79m2sXbsWgEyChg0bhs8//xxt2rTRjCY5OzvD3d1dsfehxi01isjPBz75BJg+HcjLAxo1AoKDlY6KiIhIQ9FEKCQkBCkpKZgzZw4SExPRtGlT7Ny5E76+vgCAxMTEQj2FvvzyS+Tm5mLChAmYMGGC5vzw4cOxevVqY4dfDLfUKOCvv4Dhw4Fdu+TxwIHAl18yQyQiIpOiaB8hJRiqDwH7BhVw6BAQEiKbIzo5AUuWyO0yWBBNRETlZKjvb+41piecFisgJwdITJTbY2zeDDz9tNIRERERacVESE+sflosL0/uFA/IZfHbtgFduwKVKysbFxERUSkU7yNkaaxytdjevUBgIHDlyuNzffsyCSIiIpPHRKiCYmJkDpCYqHQkCsjLA2bOBLp1k0nQzJlKR0RERKQTTo1V0IwZwG+/PT62mvqgO3eAwYOBffvk8ahRwOLFioZERESkKyZCFaSuDbKxAfz9raQ+aPduYOhQuUTexUUuix88WOmoiIiIdMZESE+8vICLF5WOwgh27QJ69ZI/P/OMXBXm769sTEREROXERKgCCm6pYTW6dgXatJFbZixcCDg7Kx0RERFRuTERqgCr6R20fz/Qrh1gby9vP/3EBIiIiCwCV41VgMX3DsrJAaZNA55/Hvjgg8fnmQQREZGF4IiQHlhk76Dr14EBA4Bjx+RxVhYgBLfJICIii8JEiIr75htg5EggNRVwdwdWrQJefVXpqIiIiPSOU2PlZJGF0tnZwOTJQL9+Mglq1QqIj2cSREREFouJUDlZZKH0zZvAypXy57Aw4OBBoF49ZWMiIiIyIE6NlZNFFko3aABERQFOTkDv3kpHQ0REZHAcEaogsy6UzswE3n778TYZANC/P5MgIiKyGhwRKgeLqA+6cgUICZE1QNu2Ab//zmXxRERkdTgiVA5mXx+0aRPQsqVMgjw8ZF0QkyAiIrJCTITKwWzrgx49AsaOBQYOBB48ADp0AE6fBnr2VDoyIiIiRXBqrALMqj4oNRXo2BE4e1Y2RZw+HZg5E7DjrwAREVkvfgvqyGzrg9zdgSZNgD//BL7+GujWTemIiIiIFMdESAcxMcDrrz8+Nvn6oIcPgdxcmQSpVMCXX8pzXl5KR0ZERGQSWCOkg4JF0oCJ1wedPy87Q48YIfcIAwA3NyZBREREBTAR0kHBIumYGBOtDxJC7g327LPAhQvAL78At24pHRUREZFJYiJUDiZbJP3gATB0KDBqlFwh1r27XBVWp47SkREREZkkJkKW4tdfgaAgYP16wNYW+O9/gV27gJo1lY6MiIjIZLFYuoxMerVYXp6s4r58WQ5XbdoEPPec0lERERGZPI4IlZFJd5O2tZWbpb7yipwKYxJERERUJkyEysjkukmfOiWHqdTatQO++UZumUFERERlwkSoDApOiyleKC0EsHQp0LYtMHy4XCZPRERE5cIaoTIwmWmx1FS5ImzbNnncpw/7AhEREVUAR4TKwCSmxY4fB1q0kEmQvT2waJGcCqtWTaGAiIiIzB9HhHSg2LTY558DU6cCOTlAvXpAdLRsmEhEREQVwhGhJzCJZfP37skk6F//kkXSTIKIiIj0giNCT6BYfVBuLmBn9ziIp5+WiZBKZcQgiIiKE0IgNzcXeXl5SodCFsbe3h62trZGfU0mQqWIiQF+++3xsVHqg/LzgQULZC3Q/v2Ao6PsE2SSe3oQkbXJzs5GYmIiMjIylA6FLJBKpULt2rVRuXJlo70mE6FSFBwNCggwQi7y119ySfyuXfJ440a5ezwRkQnIz89HQkICbG1t4e3tDQcHB6g4Sk16IoTAX3/9hVu3bqFRo0ZGGxliIlQCo48GHTgADBwI3LkDODkBixfLpIiIyERkZ2cjPz8fderUQaVKlZQOhyxQjRo18McffyAnJ8doiRCLpUtgtNGg/Hzg44+Bzp1lEtS4MfDLL8CYMawHIiKTZGPDrw4yDCVGGPnbXAKj9Q6aNg344AOZEA0dCpw8CTRrZsAXJCIiIjUmQk9g8N5BEycC3t7AqlXAmjWAEQvEiIiIrB0TIWPLywN+/PHxsZ8fcPUqMHIkp8KIiBTQuHFjODg44LaWpnF+fn5YtGhRsfOLFi2Cn59foXPp6emYPn06AgIC4OTkhFq1aqFr167Ytm0bhBAGih44e/YsOnXqBGdnZ/j4+GDOnDmlvt6+ffugUqm03k6cOFHs+pSUFNSuXRsqlQqpqalan/P333+Hq6srqlSpUuy+ZcuWITAwEM7OzmjcuDHWrl1b3rdqEEyEjCkxEejWTd7UK8MAWRxNRERGd+jQIWRmZqJ///5YvXp1uZ8nNTUV7dq1w9q1axEeHo5Tp07hwIEDCAkJwbRp05CWlqa/oAtIT09Ht27d4O3tjRMnTmDJkiVYsGABFi5cWOJj2rVrh8TExEK30aNHw8/PD8HBwcWuHzVqFJqVUrKRk5ODgQMHokOHDsXui4iIQHh4OGbNmoXz589j9uzZmDBhAv73v/+V7w0bAFeNaWGQbtKxscCQIcDdu4CLS+EiJCIiUkRkZCQGDRqETp06YcKECXj//ffLVbD7/vvv448//sDly5fh7e2tOe/v74+BAwfCyUD/4F2/fj0yMzOxevVqODo6omnTprh8+TIWLlyIsLAwre/FwcEBtWrV0hzn5ORgx44dmDhxYrHrIyIikJqaihkzZmBXwX/AF/DBBx8gICAAL7zwAo4cOVLovnXr1mHs2LEICQkBANSvXx/Hjh3DvHnz0Lt374q+fb3giJAWeu0mnZsri6F79JBJULNmsiD69dcr+MRERFQR9+/fR0xMDIYMGYJu3brh4cOH2Ldvn87Pk5+fj02bNmHw4MGFkiC1ypUrw85O+7jDwYMHUbly5VJv//3vf0t87aNHj6JTp05wdHTUnOvRowfu3LmDP/74o0zx79ixA8nJyRhRpG/dhQsXMGfOHKxdu7bElYI//fQTYmJisGzZMq33Z2VlFUsCnZ2dcfz4ceTk5JQpPkPjiJAWelsxdusWMGgQcPCgPB47FvjsM8DZuULxERGZkuBgICnJ+K9bq5b8d2V5bdq0CY0aNUKTJk0AAAMGDEBkZCQ6d+6s0/MkJyfj77//RkBAgM4xBAcH4/Tp06VeU61atRLvS0pKKlar5OnpqbmvXr16T4whMjISPXr0QJ06dTTnsrKyMHDgQHzyySeoW7curl27VuxxKSkpGDFiBL7++mu4ublpfe4ePXpg5cqV6Nu3L1q2bIm4uDisWrUKOTk5SE5OhpeX1xPjMzQmQkUUnBar8IqxgwflzdUVWLEC+GdokIjIkiQlmcDm1OUQGRmJIUOGaI6HDBmCjh07IjU1VWvRb0nUhcnlmVJzdnZGw4YNdX5cQUVfV5d4bt26hd27d2Pz5s2FzoeHhyMwMLDQ309RY8aMwaBBg9CxY8cSr/nwww+RlJSENm3aQAgBT09PjBgxAvPnzzf6nmIlYSJUhF6nxQYOBP74A+jfH6jgLzoRkakqUG5iNq974cIF/PLLLzhx4gTeffddzfm8vDxs3LgRb731FgDAzc1Na6Fzamoq3N3dAchuyFWrVsXFixd1juPgwYPo2bNnqde8//77eP/997XeV6tWLSQVGY67e/cugMcjQ6WJiopC9erV0adPn0Lnf/rpJ5w9exZbtmwB8Di58vDwwPTp0zF79mz89NNP2LFjBxYsWKC5Jj8/H3Z2dvjqq6/wxhtvwNnZGatWrcKXX36JP//8E15eXvjqq6/g6uoKDw+PJ8ZnDEyEiqjQtNiNG0BoKPDFF0CNGvJceLjeYiMiMkUVmZ5SSmRkJDp27FistmXdunWIjIzUJEIBAQFal5SfOHECjRs3BiA7bYeEhGDdunWYOXNmsTqhhw8fwtHRUWudUEWnxtq2bYv3338f2dnZcHBwAADs2bMH3t7exabMihJCICoqCsOGDYO9vX2h+7Zu3YpHjx5pjk+cOIE33ngDBw8eRIMGDQDI+qS8vDzNNd9++y3mzZuHI0eOwMfHp9Dz2dvbo3bt2gDklOTLL79sOh3KhZVJS0sTAERaWlqx+zZvFgKQNx8fHZ/422+FqFpVPvj11/UTLBGRCXn06JG4cOGCePTokdKhVEh2draoUaOGiIiIKHbf5cuXBQBx+vRpIYQQR48eFTY2NmL27Nni/Pnz4vz582LOnDnCxsZGHDt2TPO4e/fuiYCAAFG7dm2xZs0acf78eXH58mURGRkpGjZsKP7++2+DvJfU1FTh6ekpBg4cKM6ePSu2bdsm3NzcxIIFCzTX/PLLL6Jx48bi1q1bhR77448/CgDiwoULT3ydn3/+WQAo9X1ERUUJd3f3QucuXbok1q1bJy5fvix++eUXERISIqpVqyYSEhK0Pkdpv2OlfX9XhImkY6ahXNNi2dlyFOiVV4C//waefRb4v/8zSHxERFRxO3bsQEpKCvr161fsvkaNGuHpp59GZGQkAKBNmzbYvXs3fvzxRzz33HN47rnnsGfPHuzevRutW7fWPK5q1ao4duwYhgwZgo8++ggtWrRAhw4dsHHjRnzyySeaaTR9c3d3R2xsLG7duoXg4GCMHz8eYWFhCAsL01yTkZGBS5cuFVulFRkZiXbt2iEwMNAgsQFyqvHTTz/FM888g27duiEzMxNHjhx54miVMamEMGC7SxOUnp4Od3d3pKWlFapyj4kpvKI9JqYMhdIJCbIAWj1sGhoqk6B/hieJiCxJZmYmEhISUK9ePYP1xSHrVtrvWEnf3xXFGqF/6Lzb/NGjQM+eQFoaULUqsHo1UKTYjIiIiEwbEyHI0Z/ffnt8XKYi6SZNAA8P4KmngI0bAV9fg8VHREREhsFECDqMBt2+LXeKV6kANzdg7155XKTanoiIiMyD1RdLl3k0KDoaCAwECi619PVlEkRERGTGrD4ReuJo0KNHcmuMAQNkk6Fvv5Ur7ImIiMjsWX0iVGoDxUuXgDZtgK++ktNh06cDu3bJn4mIrJSVLTYmI1Lid4s1Qv8otq/Y118D48YBDx8CNWvK427dFIuPiEhp6u7DGRkZcObm0WQA2dnZAGDUfcisOhEquMFqIVeuACNGAHl5QOfOwPr1gAnskEtEpCRbW1tUqVJFs5dVpUqVyrXRKJE2+fn5+Ouvv1CpUiWt25EYilUnQiV2km7UCJg7F8jIAD74ADCRHXKJiJRW65+dTtXJEJE+2djYoG7dukZNsK06EXpcHySwtssa4Pyzsj8QAEydqlRYREQmS6VSwcvLCzVr1iy2ZQNRRTk4OBh9M1bFE6Hly5fjk08+QWJiIpo0aYJFixahQ4cOJV6/f/9+hIWF4fz58/D29sa0adMwbtw4nV93+3Y5LeaCB1jtPB7PLl8H7HtKbpdRqVJF3hIRkcWztbU1ah0HkaEoumosOjoakydPxvTp0xEfH48OHTqgZ8+euHHjhtbrExIS0KtXL3To0AHx8fF4//33MWnSJGzdulXn1/74Y+BpnMFJBOO1R+sAGxtg8GCA++cQERFZDUU3XW3dujVatmyJiIgIzbnAwED07dsXc+fOLXb9u+++ix07duDixYuac+PGjcOvv/6Ko0ePluk11Zu2DccifIF34YQsZFTzQaVvNgKljEQRERGRcgy16apiI0LZ2dmIi4tD9+7dC53v3r07jhw5ovUxR48eLXZ9jx49cPLkSZ3nqhdjMpyQhQMuPVHp0mkmQURERFZIsRqh5ORk5OXlwdPTs9B5T09PJCUlaX1MUlKS1utzc3ORnJwMLy1L3LOyspCVlaU5TktLAwDcgwofYDaCl76N5g42QHp6Rd8SERERGUj6P9/T+p7IUrxYuugSOSFEqcvmtF2v7bza3LlzMXv27GLn60EAmAGMnAGM1DFoIiIiUkRKSgrc3d319nyKJUIeHh6wtbUtNvpz9+7dYqM+arVq1dJ6vZ2dHapXr671MeHh4QgLC9Mcp6amwtfXFzdu3NDrXySVT3p6OurUqYObN2/qdc6XdMfPwnTwszAd/CxMR1paGurWrYtq1arp9XkVS4QcHBwQFBSE2NhY9OvXT3M+NjYWr7zyitbHtG3bFv/73/8KnduzZw+Cg4M1rd+LcnR0hKOjY7Hz7u7u/KU2IW5ubvw8TAQ/C9PBz8J08LMwHfruM6To8vmwsDCsXLkSq1atwsWLFxEaGoobN25o+gKFh4dj2LBhmuvHjRuH69evIywsDBcvXsSqVasQGRmJKVOmKPUWiIiIyIwpWiMUEhKClJQUzJkzB4mJiWjatCl27twJX19fAEBiYmKhnkL16tXDzp07ERoaimXLlsHb2xuLFy/Gv/71L6XeAhEREZkxxYulx48fj/Hjx2u9b/Xq1cXOderUCadOnSr36zk6OmLmzJlap8vI+Ph5mA5+FqaDn4Xp4GdhOgz1WSjaUJGIiIhISYrWCBEREREpiYkQERERWS0mQkRERGS1mAgRERGR1bLIRGj58uWoV68enJycEBQUhIMHD5Z6/f79+xEUFAQnJyfUr18fX3zxhZEitXy6fBbbtm1Dt27dUKNGDbi5uaFt27bYvXu3EaO1fLr+t6F2+PBh2NnZoXnz5oYN0Iro+llkZWVh+vTp8PX1haOjIxo0aIBVq1YZKVrLputnsX79ejzzzDOoVKkSvLy8MHLkSKSkpBgpWst14MAB9O7dG97e3lCpVPjmm2+e+Bi9fH8LC7Np0yZhb28vVqxYIS5cuCDeeecd4eLiIq5fv671+mvXrolKlSqJd955R1y4cEGsWLFC2Nvbiy1bthg5csuj62fxzjvviHnz5onjx4+Ly5cvi/DwcGFvby9OnTpl5Mgtk66fh1pqaqqoX7++6N69u3jmmWeME6yFK89n0adPH9G6dWsRGxsrEhISxC+//CIOHz5sxKgtk66fxcGDB4WNjY34/PPPxbVr18TBgwdFkyZNRN++fY0cueXZuXOnmD59uti6dasAILZv317q9fr6/ra4RKhVq1Zi3Lhxhc4FBASI9957T+v106ZNEwEBAYXOjR07VrRp08ZgMVoLXT8LbZ566ikxe/ZsfYdmlcr7eYSEhIgPPvhAzJw5k4mQnuj6WezatUu4u7uLlJQUY4RnVXT9LD755BNRv379QucWL14sateubbAYrVFZEiF9fX9b1NRYdnY24uLi0L1790Lnu3fvjiNHjmh9zNGjR4td36NHD5w8eRI5OTkGi9XSleezKCo/Px/379/X+wZ71qi8n0dUVBSuXr2KmTNnGjpEq1Gez2LHjh0IDg7G/Pnz4ePjA39/f0yZMgWPHj0yRsgWqzyfRbt27XDr1i3s3LkTQgj8+eef2LJlC1566SVjhEwF6Ov7W/HO0vqUnJyMvLy8YrvXe3p6Ftu1Xi0pKUnr9bm5uUhOToaXl5fB4rVk5fksivr000/x8OFDvP7664YI0aqU5/O4cuUK3nvvPRw8eBB2dhb1vwpFleezuHbtGg4dOgQnJyds374dycnJGD9+PO7du8c6oQooz2fRrl07rF+/HiEhIcjMzERubi769OmDJUuWGCNkKkBf398WNSKkplKpCh0LIYqde9L12s6T7nT9LNQ2btyIWbNmITo6GjVr1jRUeFanrJ9HXl4eBg0ahNmzZ8Pf399Y4VkVXf7byM/Ph0qlwvr169GqVSv06tULCxcuxOrVqzkqpAe6fBYXLlzApEmTMGPGDMTFxeGHH35AQkKCZrNwMi59fH9b1D/zPDw8YGtrWyyTv3v3brGsUa1WrVpar7ezs0P16tUNFqulK89noRYdHY1Ro0YhJiYGXbt2NWSYVkPXz+P+/fs4efIk4uPjMXHiRADyy1gIATs7O+zZswddunQxSuyWpjz/bXh5ecHHxwfu7u6ac4GBgRBC4NatW2jUqJFBY7ZU5fks5s6di/bt22Pq1KkAgGbNmsHFxQUdOnTARx99xFkEI9LX97dFjQg5ODggKCgIsbGxhc7HxsaiXbt2Wh/Ttm3bYtfv2bMHwcHBsLe3N1islq48nwUgR4JGjBiBDRs2cM5dj3T9PNzc3HD27FmcPn1acxs3bhwaN26M06dPo3Xr1sYK3eKU57+N9u3b486dO3jw4IHm3OXLl2FjY4PatWsbNF5LVp7PIiMjAzY2hb86bW1tATwejSDj0Nv3t06l1WZAvRQyMjJSXLhwQUyePFm4uLiIP/74QwghxHvvvSeGDh2quV69/C40NFRcuHBBREZGcvm8nuj6WWzYsEHY2dmJZcuWicTERM0tNTVVqbdgUXT9PIriqjH90fWzuH//vqhdu7Z47bXXxPnz58X+/ftFo0aNxOjRo5V6CxZD188iKipK2NnZieXLl4urV6+KQ4cOieDgYNGqVSul3oLFuH//voiPjxfx8fECgFi4cKGIj4/XtDIw1Pe3xSVCQgixbNky4evrKxwcHETLli3F/v37NfcNHz5cdOrUqdD1+/btEy1atBAODg7Cz89PREREGDliy6XLZ9GpUycBoNht+PDhxg/cQun630ZBTIT0S9fP4uLFi6Jr167C2dlZ1K5dW4SFhYmMjAwjR22ZdP0sFi9eLJ566inh7OwsvLy8xODBg8WtW7eMHLXl+fnnn0v9DjDU97dKCI7lERERkXWyqBohIiIiIl0wESIiIiKrxUSIiIiIrBYTISIiIrJaTISIiIjIajERIiIiIqvFRIiIiIisFhMhIipk9erVqFKlitJhlJufnx8WLVpU6jWzZs1C8+bNjRIPEZk2JkJEFmjEiBFQqVTFbr///rvSoWH16tWFYvLy8sLrr7+OhIQEvTz/iRMn8Oabb2qOVSoVvvnmm0LXTJkyBXv37tXL65Wk6Pv09PRE7969cf78eZ2fx5wTUyJTx0SIyEK9+OKLSExMLHSrV6+e0mEBkJu6JiYm4s6dO9iwYQNOnz6NPn36IC8vr8LPXaNGDVSqVKnUaypXrqzT7tTlVfB9fv/993j48CFeeuklZGdnG/y1iahsmAgRWShHR0fUqlWr0M3W1hYLFy7E008/DRcXF9SpUwfjx48vtKt5Ub/++is6d+4MV1dXuLm5ISgoCCdPntTcf+TIEXTs2BHOzs6oU6cOJk2ahIcPH5Yam0qlQq1ateDl5YXOnTtj5syZOHfunGbEKiIiAg0aNICDgwMaN26MdevWFXr8rFmzULduXTg6OsLb2xuTJk3S3FdwaszPzw8A0K9fP6hUKs1xwamx3bt3w8nJCampqYVeY9KkSejUqZPe3mdwcDBCQ0Nx/fp1XLp0SXNNaZ/Hvn37MHLkSKSlpWlGlmbNmgUAyM7OxrRp0+Dj4wMXFxe0bt0a+/btKzUeIiqOiRCRlbGxscHixYtx7tw5rFmzBj/99BOmTZtW4vWDBw9G7dq1ceLECcTFxeG9996Dvb09AODs2bPo0aMHXn31VZw5cwbR0dE4dOgQJk6cqFNMzs7OAICcnBxs374d77zzDv7973/j3LlzGDt2LEaOHImff/4ZALBlyxZ89tln+PLLL3HlyhV88803ePrpp7U+74kTJwAAUVFRSExM1BwX1LVrV1SpUgVbt27VnMvLy8PmzZsxePBgvb3P1NRUbNiwAQA0f39A6Z9Hu3btsGjRIs3IUmJiIqZMmQIAGDlyJA4fPoxNmzbhzJkz6N+/P1588UVcuXKlzDEREWCRu88TWbvhw4cLW1tb4eLiorm99tprWq/dvHmzqF69uuY4KipKuLu7a45dXV3F6tWrtT526NCh4s033yx07uDBg8LGxkY8evRI62OKPv/NmzdFmzZtRO3atUVWVpZo166dGDNmTKHH9O/fX/Tq1UsIIcSnn34q/P39RXZ2ttbn9/X1FZ999pnmGIDYvn17oWtmzpwpnnnmGc3xpEmTRJcuXTTHu3fvFg4ODuLevXsVep8AhIuLi6hUqZJmJ+0+ffpovV7tSZ+HEEL8/vvvQqVSidu3bxc6/8ILL4jw8PBSn5+ICrNTNg0jIkPp3LkzIiIiNMcuLi4AgJ9//hn//e9/ceHCBaSnpyM3NxeZmZl4+PCh5pqCwsLCMHr0aKxbtw5du3ZF//790aBBAwBAXFwcfv/9d6xfv15zvRAC+fn5SEhIQGBgoNbY0tLSULlyZQghkJGRgZYtW2Lbtm1wcHDAxYsXCxU7A0D79u3x+eefAwD69++PRYsWoX79+njxxRfRq1cv9O7dG3Z25f/f2eDBg9G2bVvcuXMH3t7eWL9+PXr16oWqVatW6H26urri1KlTyM3Nxf79+/HJJ5/giy++KHSNrp8HAJw6dQpCCPj7+xc6n5WVZZTaJyJLwkSIyEK5uLigYcOGhc5dv34dvXr1wrhx4/Cf//wH1apVw6FDhzBq1Cjk5ORofZ5Zs2Zh0KBB+P7777Fr1y7MnDkTmzZtQr9+/ZCfn4+xY8cWqtFRq1u3bomxqRMEGxsbeHp6FvvCV6lUhY6FEJpzderUwaVLlxAbG4sff/wR48ePxyeffIL9+/cXmnLSRatWrdCgQQNs2rQJb731FrZv346oqCjN/eV9nzY2NprPICAgAElJSQgJCcGBAwcAlO/zUMdja2uLuLg42NraFrqvcuXKOr13ImvHRIjIipw8eRK5ubn49NNPYWMjSwQ3b978xMf5+/vD398foaGhGDhwIKKiotCvXz+0bNkS58+fL5ZwPUnBBKGowMBAHDp0CMOGDdOcO3LkSKFRF2dnZ/Tp0wd9+vTBhAkTEBAQgLNnz6Jly5bFns/e3r5Mq9EGDRqE9evXo3bt2rCxscFLL72kua+877Oo0NBQLFy4ENu3b0e/fv3K9Hk4ODgUi79FixbIy8vD3bt30aFDhwrFRGTtWCxNZEUaNGiA3NxcLFmyBNeuXcO6deuKTdUU9OjRI0ycOBH79u3D9evXcfjwYZw4cUKTlLz77rs4evQoJkyYgNOnT+PKlSvYsWMH3n777XLHOHXqVKxevRpffPEFrly5goULF2Lbtm2aIuHVq1cjMjIS586d07wHZ2dn+Pr6an0+Pz8/7N27F0lJSfj7779LfN3Bgwfj1KlT+Pjjj/Haa6/ByclJc5++3qebmxtGjx6NmTNnQghRps/Dz88PDx48wN69e5GcnIyMjAz4+/tj8ODBGDZsGLZt24aEhAScOHEC8+bNw86dO3WKicjqKVmgRESGMXz4cPHKK69ovW/hwoXCy8tLODs7ix49eoi1a9cKAOLvv/8WQhQuzs3KyhIDBgwQderUEQ4ODsLb21tMnDixUIHw8ePHRbdu3UTlypWFi4uLaNasmfj4449LjE1b8W9Ry5cvF/Xr1xf29vbC399frF27VnPf9u3bRevWrYWbm5twcXERbdq0ET/++KPm/qLF0jt27BANGzYUdnZ2wtfXVwhRvFha7dlnnxUAxE8//VTsPn29z+vXrws7OzsRHR0thHjy5yGEEOPGjRPVq1cXAMTMmTOFEEJkZ2eLGTNmCD8/P2Fvby9q1aol+vXrJ86cOVNiTERUnEoIIZRNxYiIiIiUwakxIiIislpMhIiIiMhqMREiIiIiq8VEiIiIiKwWEyEiIiKyWkyEiIiIyGoxESIiIiKrxUSIiIiIrBYTISIiIrJaTISIiIjIajERIiIiIqvFRIiIiIis1v8DLqlLL+Ying0AAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# calculate false positive rate and the true positive rate in order to construct a receiver operating curve (ROC)\n",
    "fpr_test, tpr_test, thresholds_test = roc_curve(y_test, y_proba_test, pos_label=1)\n",
    "\n",
    "plt.title('ROC (Test)')\n",
    "ax1 = plt.plot(fpr_test, tpr_test, 'b', label = '%s AUC = %0.4f' % ('', auc), linewidth=2)\n",
    "plt.legend(loc = 'lower right')\n",
    "plt.plot([0, 1], [0, 1],'r--')\n",
    "plt.xlim([0, 1])\n",
    "plt.ylim([0, 1])\n",
    "plt.ylabel('True Positive Rate')\n",
    "plt.xlabel('False Positive Rate')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "287ba96e-90da-43cf-b079-abb7c4279085",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "#### Precision-Recall Curve (Test)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 99,
   "id": "c27539a6-cda2-4df4-bd22-1d5d5e6a6b50",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0.5, 0, 'Recall')"
      ]
     },
     "execution_count": 99,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAkIAAAHFCAYAAAAe+pb9AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8hTgPZAAAACXBIWXMAAA9hAAAPYQGoP6dpAABSfklEQVR4nO3deVhUZf8G8HvYFwVSZFdE3CU33M1wSQzNrUxcciE1t3IhK8ncS8zMn5lipaiVa1qaJqmUe5K5gJpQlqC4gITKoqhsz++PeRk5zICAM3OGmftzXXO98zznnJl75mDzfc/yPAohhAARERGRCTKTOwARERGRXFgIERERkcliIUREREQmi4UQERERmSwWQkRERGSyWAgRERGRyWIhRERERCaLhRARERGZLBZCREREZLJYCJFJ27BhAxQKhephYWEBLy8vhISE4MaNG3rPM3r0aNStW7dC21y5cgUKhQIbNmzQSaYnGT16tOQ7tLKygq+vL2bMmIGsrCxZMhWn6fsp2u9Xrlwp12ucP38eISEh8PHxgY2NDapVq4bWrVtjyZIluHPnjm6C68Hly5dhbW2NmJgYHD58WLIfy3poQ3x8PObNm6dxH4wYMQIDBgzQyvsQPYmF3AGIDMH69evRuHFjPHjwAEePHkV4eDiOHDmCCxcuwN7eXm85Zs+ejalTp1ZoG3d3d8TExMDX11dHqZ7M1tYWBw8eBABkZGRgx44d+PTTT3H+/HkcOHBAtlzasGbNGkyaNAmNGjXCO++8g6ZNmyIvLw+nT5/GF198gZiYGOzcuVPumJUyY8YM9OzZEx07dkRWVhZiYmIkywcOHAhfX18sXbpU6+8dHx+P+fPno2vXrmrF/7x589C4cWMcPHgQ3bt31/p7ExXHQogIgJ+fH9q0aQMA6NatGwoKCrBw4ULs2rULw4cP17hNTk4O7OzstJqjMsWMtbU1OnTooNUcFWVmZibJ8OKLLyIxMRHR0dFISkqCj4+PjOkqLyYmBhMnTkTPnj2xa9cuWFtbq5b17NkTb7/9Nvbt26eV93rw4AFsbGy0dsTlSRISErBr1y5VfgcHB7W/I2trazg5Oen978vX1xcvvvgiFi9ezEKIdI6nxog0KPoP/9WrVwEoT/9Uq1YNFy5cQGBgIKpXr44ePXoAAHJzc/Hhhx+icePGsLa2Rq1atRASEoL//vtP7XU3b96Mjh07olq1aqhWrRpatmyJyMhI1XJNp8a2b9+O9u3bw9HREXZ2dqhXrx5ef/111fLSTo0dP34cPXr0QPXq1WFnZ4dOnTph7969knWKThEdOnQIEydOhLOzM2rWrImXX34ZN2/erPT3B0BVWN66dUvSv23bNnTs2BH29vaoVq0aevXqhdjYWLXtT548ib59+6JmzZqwsbGBr68vpk2bplr+77//IiQkBA0aNICdnR08PT3Rt29fXLhw4alyF7do0SIoFAp89dVXkiKoiJWVFfr166dqKxQKzJs3T229unXrYvTo0ap20fd+4MABvP7666hVqxbs7Oywbds2KBQK/Prrr2qvsXr1aigUCpw/f17Vd/r0afTr1w81atSAjY0NWrVqhe+++65cn2316tVwc3NDz549y7V+kdTUVIwfPx5eXl6wsrKCj48P5s+fj/z8fLXXb9GiBapVq4bq1aujcePGeP/991Wf/9VXXwWg/D8eRafciv8NjxgxAr/88gsuX75coXxEFcVCiEiDf//9FwBQq1YtVV9ubi769euH7t2748cff8T8+fNRWFiI/v37Y/HixRg2bBj27t2LxYsXIzo6Gl27dsWDBw9U28+ZMwfDhw+Hh4cHNmzYgJ07d2LUqFGqYkuTmJgYBAcHo169eti6dSv27t2LOXPmqP3olHTkyBF0794dmZmZiIyMxJYtW1C9enX07dsX27ZtU1t/7NixsLS0xObNm7FkyRIcPnwYr732WkW/NomkpCRYWFigXr16qr5FixZh6NChaNq0Kb777jt8++23yM7ORpcuXRAfH69ab//+/ejSpQuSk5OxbNky/Pzzz/jggw8kRdXNmzdRs2ZNLF68GPv27cOqVatgYWGB9u3b4++//36q7ABQUFCAgwcPwt/fH7Vr137q19Pk9ddfh6WlJb799lvs2LEDAwcOhIuLC9avX6+27oYNG9C6dWs0b94cAHDo0CF07twZGRkZ+OKLL/Djjz+iZcuWCA4OLtf1Ynv37sXzzz8PM7Py/wykpqaiXbt22L9/P+bMmYOff/4ZY8aMQXh4OMaNG6dab+vWrZg0aRICAgKwc+dO7Nq1C9OnT8f9+/cBAH369MGiRYsAAKtWrUJMTAxiYmLQp08f1Wt07doVQghERUWVOx9RpQgiE7Z+/XoBQPz+++8iLy9PZGdni59++knUqlVLVK9eXaSmpgohhBg1apQAINatWyfZfsuWLQKA+P777yX9p06dEgBERESEEEKIxMREYW5uLoYPH15mnlGjRglvb29Ve+nSpQKAyMjIKHWbpKQkAUCsX79e1dehQwfh4uIisrOzVX35+fnCz89PeHl5icLCQsnnnzRpkuQ1lyxZIgCIlJSUMvMWZba3txd5eXkiLy9PpKeni9WrVwszMzPx/vvvq9ZLTk4WFhYW4q233pJsn52dLdzc3MTgwYNVfb6+vsLX11c8ePDgie9f/PPl5uaKBg0aiOnTp6v6NX0/RZ87KSmp1NdLTU0VAMSQIUPKnQGAmDt3rlq/t7e3GDVqlNr7jxw5Um3d0NBQYWtrK9nn8fHxAoD4/PPPVX2NGzcWrVq1Enl5eZLtX3rpJeHu7i4KCgpKzXnr1i0BQCxevLjMz+Pt7S369Omjao8fP15Uq1ZNXL16VbJe0d/pxYsXhRBCvPnmm8LJyanM196+fbsAIA4dOlTqOp6eniI4OLjM1yF6WjwiRATlqTBLS0tUr14dL730Etzc3PDzzz/D1dVVst4rr7wiaf/0009wcnJC3759kZ+fr3q0bNkSbm5uOHz4MAAgOjoaBQUFmDx5coVytW3bFgAwePBgfPfdd+W6k+3+/fs4efIkBg0ahGrVqqn6zc3NMWLECFy/fl3tiEnx0zsAVEcdio5WFRYWSj5fQUGB2ntaWlrC0tISzs7OmDhxIoKDg/HRRx+p1tm/fz/y8/MxcuRIyWvZ2NggICBA9V1dunQJly9fxpgxY2BjY1Pq58zPz8eiRYvQtGlTWFlZwcLCAlZWVvjnn3+QkJDwxO/JEJT8ewKUR4kePHggOXK3fv16WFtbY9iwYQCURyz/+usv1fVrxb/P3r17IyUlpcyjYkWnPV1cXCqU96effkK3bt3g4eEhec+goCAAyiORANCuXTtkZGRg6NCh+PHHH5Genl6h9yni4uIiy92bZFpYCBEB+Oabb3Dq1CnExsbi5s2bOH/+PDp37ixZx87ODg4ODpK+W7duISMjA1ZWVqpCoOiRmpqq+gEoul7Iy8urQrmef/557Nq1S1VAeHl5wc/PD1u2bCl1m7t370IIAXd3d7VlHh4eAIDbt29L+mvWrClpF10PU3Rqb8GCBZLPVvKibltbW5w6dQqnTp3Cnj170LVrV2zZsgWLFy9WrVN0Wqtt27Zq39W2bdsq/F2FhoZi9uzZGDBgAPbs2YOTJ0/i1KlTaNGiheSUZGU5OzvDzs4OSUlJT/1apdG0j5o1a4a2bduqTo8VFBRg48aN6N+/P2rUqAHg8Xc5Y8YMte9y0qRJAFBm8VH0/ZRVaGpy69Yt7NmzR+09mzVrJnnPESNGYN26dbh69SpeeeUVuLi4oH379oiOjq7Q+9nY2GhlXxKVhXeNEQFo0qSJ6uLe0mi6m6fo4uLS7hyqXr06gMfXGl2/fr3C15v0798f/fv3x6NHj/D7778jPDwcw4YNQ926ddGxY0e19Z955hmYmZkhJSVFbVnRkQBnZ+cKZXjjjTfw0ksvqdolLxw2MzOTfH89e/aEv78/5s+fj+HDh6N27dqq99yxYwe8vb1Lfa/i31VZNm7ciJEjR6quNSmSnp4OJyencn2uspibm6NHjx74+eefcf369XIVsdbW1nj06JFaf8nCs0hpd4iFhIRg0qRJSEhIQGJiIlJSUhASEqJaXvRdhoWF4eWXX9b4Go0aNSo1Z9H2FR0DydnZGc2bN5cc6SuuqNAu+gwhISG4f/8+jh49irlz5+Kll17CpUuXytz/xd25c6fC42oRVRQLIaKn8NJLL2Hr1q0oKChA+/btS10vMDAQ5ubmWL16tcbipTysra0REBAAJycn7N+/H7GxsRpfy97eHu3bt8cPP/yApUuXwtbWFoDy9NbGjRvh5eWFhg0bVui9PTw8JD9y5cm6atUqdO3aFR9++CG+/PJL9OrVCxYWFrh8+bLGU0JFGjZsCF9fX6xbtw6hoaEa79YClEVEyWV79+7FjRs3UL9+/XJnLUtYWBiioqIwbtw4/Pjjj7CyspIsz8vLw759+9C3b18AyrvDit/VBQAHDx7EvXv3KvS+Q4cORWhoKDZs2IDExER4enoiMDBQtbxRo0Zo0KABzp07p1YIloe3tzdsbW0rfEfWSy+9hKioKPj6+uKZZ54p1zb29vYICgpCbm4uBgwYgIsXL8Lb21vtqGNJ+fn5uHbtGnr37l2hjEQVxUKI6CkMGTIEmzZtQu/evTF16lS0a9cOlpaWuH79Og4dOoT+/ftj4MCBqFu3Lt5//30sXLgQDx48wNChQ+Ho6Ij4+Hikp6dj/vz5Gl9/zpw5uH79Onr06AEvLy9kZGTgs88+g6WlJQICAkrNFR4ejp49e6Jbt26YMWMGrKysEBERgT///BNbtmzRy1g1AQEB6N27N9avX4+ZM2fCx8cHCxYswKxZs5CYmIgXX3wRzzzzDG7duoU//vgD9vb2qu9h1apV6Nu3Lzp06IDp06ejTp06SE5Oxv79+7Fp0yYAyh/lDRs2oHHjxmjevDnOnDmDTz75pMKnH8vSsWNHrF69GpMmTYK/vz8mTpyIZs2aIS8vD7Gxsfjqq6/g5+enKoRGjBiB2bNnY86cOQgICEB8fDxWrlwJR0fHCr2vk5MTBg4ciA0bNiAjIwMzZsxQu7vryy+/RFBQEHr16oXRo0fD09MTd+7cQUJCAs6ePYvt27eX+vpWVlbo2LEjfv/99wrlWrBgAaKjo9GpUydMmTIFjRo1wsOHD3HlyhVERUXhiy++gJeXF8aNGwdbW1t07twZ7u7uSE1NRXh4OBwdHVXXvfn5+QEAvvrqK1SvXh02Njbw8fFRnaY9f/48cnJy0K1btwplJKowua/WJpJT0d07p06dKnO9ojujNMnLyxNLly4VLVq0EDY2NqJatWqicePGYvz48eKff/6RrPvNN9+Itm3bqtZr1aqV5G6mkneN/fTTTyIoKEh4enoKKysr4eLiInr37i2OHTumWkfTXVFCCHHs2DHRvXt3YW9vL2xtbUWHDh3Enj17yvX5Dx069MQ7esrz3Vy4cEGYmZmJkJAQVd+uXbtEt27dhIODg7C2thbe3t5i0KBB4pdffpFsGxMTI4KCgoSjo6OwtrYWvr6+krvB7t69K8aMGSNcXFyEnZ2deO6558SxY8dEQECACAgIKPP7Kc9dY8XFxcWJUaNGiTp16ggrKythb28vWrVqJebMmSPS0tJU6z169Ei8++67onbt2sLW1lYEBASIuLi4Uu8aK+vv7sCBAwKAACAuXbqkcZ1z586JwYMHCxcXF2FpaSnc3NxE9+7dxRdffPHEzxQZGSnMzc3FzZs3S12n5F1jQgjx33//iSlTpggfHx9haWkpatSoIfz9/cWsWbPEvXv3hBBCfP3116Jbt27C1dVVWFlZCQ8PDzF48GBx/vx5yWstX75c+Pj4CHNzc7V9NHv2bOHs7CwePnz4xM9C9DQUQgghVxFGRETyePjwIerUqYO3334b7733ntxxJAoKClC/fn0MGzas1OuRiLSFd40REZkgGxsbzJ8/H8uWLVMNdGgoNm7ciHv37uGdd96ROwqZAF4jRERkot544w1kZGQgMTERzz77rNxxVAoLC7Fp0yat3P1H9CQ8NUZEREQmS9ZTY0ePHkXfvn3h4eEBhUKBXbt2PXGbI0eOwN/fHzY2NqhXrx6++OIL3QclIiIioyRrIXT//n20aNECK1euLNf6SUlJ6N27N7p06YLY2Fi8//77mDJlCr7//nsdJyUiIiJjZDCnxhQKBXbu3IkBAwaUus57772H3bt3S+YRmjBhAs6dO4eYmBg9pCQiIiJjUqUulo6JiZGMrgoAvXr1QmRkJPLy8mBpaam2zaNHjyRD3hcWFuLOnTuoWbOmXgaVIyIioqcnhEB2djY8PDzUBhh9GlWqEEpNTVWbDdzV1RX5+flIT0/XOIFheHh4qaP2EhERUdVy7do1rY4gX6UKIUB9ksKiM3ulHd0JCwtDaGioqp2ZmYk6derg2rVrajOJm7LJk4GNG6V9/fsDffsCr74qTyYiIqIiWVlZqF27tmoya22pUoWQm5sbUlNTJX1paWmwsLBQzU9TkrW1tcZJGx0cHFgIFfPNN4C9PfDll4/7fvxR+Rg7Vrnc3Bzo1g3QcOCNiIhIL7R9WUuVKoQ6duyIPXv2SPoOHDiANm3aaLw+iMpPoQC++EJZDC1bpr585MjHzwsLlesTERFVdbLePn/v3j3ExcUhLi4OgPL2+Li4OCQnJwNQntYaWewXeMKECbh69SpCQ0ORkJCAdevWITIyEjNmzJAjvlH69FMgLQ0oa0in8+f1FoeIiEinZC2ETp8+jVatWqFVq1YAgNDQULRq1Qpz5swBAKSkpKiKIgDw8fFBVFQUDh8+jJYtW2LhwoVYsWIFXnnlFVnyG6tatZTXB2VmKk+JrVolXb55szy5iIiItM1gxhHSl6ysLDg6OiIzM5PXCFVA/frA5cuP23FxQIsWssUhIiITo6vfb84+T+Uye7a03bIlYFolNBERGSMWQlQuQ4cqT5kV5+IiTxYiIiJtYSFE5WJlBaSkSPvS04HDh4GkJFkiERERPTUWQlRu5ubKa4OK69YNqFcPmDlTlkhERERPhYUQVUiLFkCJWU4AAB9/rBxrKC9P/5mIiIgqi4UQVdiffwJz5wLDh0v7v/1WeQqNF1ETEVFVwUKIKszZGZg3Tzk32fvvqy8PCtJ7JCIiokphIURP5aOPgIQEad/+/TwqREREVQMLIXpqjRsDiYnSvtxcebIQERFVBAsh0gofH2nbokpN50tERKaKhRDpxLRpcicgIiJ6MhZCpBMrVwK3bsmdgoiIqGwshEhrfvxR2r5xQ54cRERE5cVCiLSmXz9p+/XX5clBRERUXiyESGfOnQP27ZM7BRERUelYCJFW/fuvtB0UBNy+LU8WIiKiJ2EhRFrl6wtMnSrtc3ZWXjydny9PJiIiotKwECKtW75cve+tt4ChQ/UehYiIqEwshEgnjh9X79uxA0hP138WIiKi0rAQIp3o3BlITlY/OjRkiCxxiIiINFIIYVrTY2ZlZcHR0RGZmZlwcHCQO45JUCikbdP6iyMiIm3Q1e83jwiRzu3aJW3Xrw8UFMgShYiISIKFEOlcy5bS9uXLQGSkLFGIiIgkWAiRztWpAyxYIO0bP16eLERERMWxECKdUyiA2bOBWbOk/YmJ8uQhIiIqwkKI9GbePGk7JUWWGERERCoshEhvLCyk7S5dgLg4WaIQEREBYCFEMhICaNVK7hRERGTKWAiRXm3erN7XqBFQWKj/LERERCyESK+GDgUyMqR9ly4BW7bIEoeIiEwcCyHSO0dH4OBBad9rr8mThYiITBsLIZJFt25A9+7SPk69QURE+sZCiGSzcaO0/e678uQgIiLTxUKIZOPuLm0vXQr89JM8WYiIyDSxECJZrVghbW/dKk8OIiIyTQohTOvKjKysLDg6OiIzMxMODg5yxzF5QgBmZup9RERExenq95tHhEhWCgUQESHtYyFERET6wkKIZDdypLS9bZs8OYiIyPSwECLZ2dtL2/v2yZODiIhMDwshMgh16jx+/vXXQG6ufFmIiMh0sBAigzB+vLT9wgvy5CAiItPCQogMQsnBFI8d40XTRESkeyyEyCBYWABnzkj7OCM9ERHpGgshMhitW0vbDx/Kk4OIiEwHCyEyWFu2yJ2AiIiMHQshMljjxvGoEBER6RYLITIoc+ZI2w0ayJODiIhMAwshMijz50vb168D9eoBp07Jk4eIiIwbCyEyOFevSttJScpBFomIiLSNhRAZnDp11Aufc+fkyUJERMaNhRAZpJEjgf79H7e7dJEvCxERGS8WQmSw7t59/Dw8XL4cRERkvFgIkcHq2lXa/uUXWWIQEZERYyFEBqvkHWT//SdPDiIiMl4shMig1anz+LmTk2wxiIjISLEQIoOWnPz4ee/eQG6ufFmIiMj4sBAig1b8zjEAaNiQxRAREWkPCyEyaDt3SttXrwJNmsiThYiIjA8LITJoCgWwYoW0LzFRnixERGR8WAiRwXvrLeDyZblTEBGRMWIhRFVCrVrS9uLF8uQgIiLjwkKIqgRbW2k7LAzw8wNeew346it5MhERUdXHQoiqBAsL9aNAFy8CmzYB48cDFy7Ik4uIiKo22QuhiIgI+Pj4wMbGBv7+/jh27FiZ62/atAktWrSAnZ0d3N3dERISgtu3b+spLcnp3XeBiRM1L9u8Wb9ZiIjIOMhaCG3btg3Tpk3DrFmzEBsbiy5duiAoKAjJxUfRK+b48eMYOXIkxowZg4sXL2L79u04deoUxo4dq+fkJAeFAoiIADIzgX//lS7jNUNERFQZshZCy5Ytw5gxYzB27Fg0adIEy5cvR+3atbF69WqN6//++++oW7cupkyZAh8fHzz33HMYP348Tp8+refkJCcHB8DXV3lKrMjkyfLlISKiqku2Qig3NxdnzpxBYGCgpD8wMBAnTpzQuE2nTp1w/fp1REVFQQiBW7duYceOHejTp0+p7/Po0SNkZWVJHmQccnIePw8IkC8HERFVXbIVQunp6SgoKICrq6uk39XVFampqRq36dSpEzZt2oTg4GBYWVnBzc0NTk5O+Pzzz0t9n/DwcDg6OqoetWvX1urnIPl8+63cCYiIqKqT/WJphUIhaQsh1PqKxMfHY8qUKZgzZw7OnDmDffv2ISkpCRMmTCj19cPCwpCZmal6XLt2Tav5yTB4ecmdgIiIqiILud7Y2dkZ5ubmakd/0tLS1I4SFQkPD0fnzp3xzjvvAACaN28Oe3t7dOnSBR9++CHc3d3VtrG2toa1tbX2PwAZlORkoGNHuVMQEVFVI9sRISsrK/j7+yM6OlrSHx0djU6dOmncJicnB2Zm0sjm5uYAlEeSyHQNGQI8eCB3CiIiqmpkPTUWGhqKtWvXYt26dUhISMD06dORnJysOtUVFhaGkSNHqtbv27cvfvjhB6xevRqJiYn47bffMGXKFLRr1w4eHh5yfQwyEJyVnoiIKkq2U2MAEBwcjNu3b2PBggVISUmBn58foqKi4O3tDQBISUmRjCk0evRoZGdnY+XKlXj77bfh5OSE7t274+OPP5brI5CM9u4Fit8wePUqkJICaDhDSkREpJFCmNg5paysLDg6OiIzMxMODg5yx6GndPGics6x4nJzAUtLefIQEZFu6Or3W/a7xoieRrNm6n2lDENFRESkhoUQVXklL5JOSJAnBxERVT0shKjKs7GRtl1c5MlBRERVDwshMjqlTFVHRESkhoUQGZ1ffgEKC+VOQUREVQELITIK33wjbZubA/fuyZOFiIiqDhZCZBSGD1fve/11/ecgIqKqhYUQGQUzM+V8Y8Vt3w7MnClPHiIiqhpYCJHRqF0b+PFHad/HHwN//SVPHiIiMnwshMio9OsHfPihtC8vT54sRERk+FgIkdGZNUva1jT6NBEREcBCiEzAihVyJyAiIkPFQoiM3oULcicgIiJDxUKIjF5UlNwJiIjIULEQIqNkZ/f4eceO8uUgIiLDxkKIjJKl5ePnO3fKl4OIiAwbCyEySv36SdsdOsiTg4iIDBsLITJKJWegP3kSiI6WJwsRERkuFkJklOztgV27pH1Hj8oShYiIDJhCCCHkDqFPWVlZcHR0RGZmJhwcHOSOQzqmUEjbhYXqfUREZPh09fvNI0Jk1AYPlra3b5cnBxERGSYWQmTUNm6UtoOD5clBRESGiYUQGTVLS8DWVu4URERkqFgIkdF7/325ExARkaFiIURGz8tL2s7OlicHEREZHhZCZPR695a2Y2LkyUFERIaHhRAZPRcXabtXL+DhQ3myEBGRYWEhRCZh8mRp29YWuH1bnixERGQ4WAiRSVi5Ur3P2ZlHhoiITB0LITIZycnqfRcu6D8HEREZDhZCZDJq1wZu3JD27dkjTxYiIjIMLITIpHh4SNsLF8qTg4iIDAMLITI5deo8fu7uLl8OIiKSHwshMjk9ez5+PmWKfDmIiEh+LITI5ERGPn5eq5Z8OYiISH4shMikCCFtBwXJk4OIiAwDCyEyKZcvS9slL54mIiLTwkKITMqdO3InICIiQ8JCiExKSoq0zZGliYhMGwshMim+vtK2rS1w4IA8WYiISH4shMikNGum3sfZ6ImITBcLITIpCgVw6ZJ6f2am/rMQEZH8WAiRyWnQACgslPatWiVPFiIikhcLITJJCoW0zTnHiIhMEwshMlkffSRtR0TIk4OIiOSjEKLkWLvGLSsrC46OjsjMzISDg4PccUhGQgBmZup9RERkeHT1+80jQmSyFApg2LDHbUtL+bIQEZE8WAiRSbtw4fHzGjXky0FERPJgIUQmzcLi8fP0dPlyEBGRPFgIkUlzcXn8vKAAePZZ5aNbN+DcOflyERGRfrAQIpP28svS9p9/Kh+HD6vfVUZERMaHhRCZtFGjlAMsAsp5x4rbvl3/eYiISL9YCJFJs7ZWTrkhBJCTAwwYIHciIiLSJxZCRMVMmyZt//23LDGIiEhPWAgRFdOokbS9fr08OYiISD9YCBEV4+YmbX/8sTw5iIhIP1gIEZVQshjitBtERMaLhRBRCSXvFouJkScHERHpHgshohL8/aXtatXkyUFERLrHQoiohJLjCfXvD9y4IU8WIiLSLRZCRE9w5Qrg5SV3CiIi0gUWQkQavP223AmIiEgfWAgRabB4MfDOO9K+f/6RJwsREemO7IVQREQEfHx8YGNjA39/fxw7dqzM9R89eoRZs2bB29sb1tbW8PX1xbp16/SUlkyFhQUwdKi0LydHnixERKQ7FnK++bZt2zBt2jRERESgc+fO+PLLLxEUFIT4+HjUqVNH4zaDBw/GrVu3EBkZifr16yMtLQ35+fl6Tk6moHFjaXvjRqBFC3myEBGRbiiEkG+4uPbt26N169ZYvXq1qq9JkyYYMGAAwsPD1dbft28fhgwZgsTERNSoUaNS75mVlQVHR0dkZmbCwcGh0tnJ+OXmKidlLRIaCnz6qXx5iIhMma5+v2U7NZabm4szZ84gMDBQ0h8YGIgTJ05o3Gb37t1o06YNlixZAk9PTzRs2BAzZszAgwcPSn2fR48eISsrS/IgKg8rK2l72TLgl1/kyUJERLohWyGUnp6OgoICuLq6SvpdXV2RmpqqcZvExEQcP34cf/75J3bu3Inly5djx44dmDx5cqnvEx4eDkdHR9Wjdu3aWv0cZFp69gQWLJA7BRERaYvsF0srFApJWwih1leksLAQCoUCmzZtQrt27dC7d28sW7YMGzZsKPWoUFhYGDIzM1WPa9euaf0zkPH67z/1vrlz9Z+DiIh0Q7ZCyNnZGebm5mpHf9LS0tSOEhVxd3eHp6cnHB0dVX1NmjSBEALXr1/XuI21tTUcHBwkD6LycnYGHj5U78/I0HsUIiLSAdkKISsrK/j7+yM6OlrSHx0djU6dOmncpnPnzrh58ybu3bun6rt06RLMzMzgxaF/SUesrYGffpL2nTolTxYiItIuWU+NhYaGYu3atVi3bh0SEhIwffp0JCcnY8KECQCUp7VGjhypWn/YsGGoWbMmQkJCEB8fj6NHj+Kdd97B66+/DtuSE0QRaVHv3tL25cvy5CAiIu2SdRyh4OBg3L59GwsWLEBKSgr8/PwQFRUFb29vAEBKSgqSk5NV61erVg3R0dF466230KZNG9SsWRODBw/Ghx9+KNdHIBNR8rK1iROB/9XrRERUhck6jpAcOI4QVVbJYigzE+CfEBGRfhjdOEJEVc2sWdK2oyNPkRERVXUshIjK6aWX1Pu+/lr/OYiISHtYCBGVU4cOwM2b0r5Hj+TJQkRE2lGpi6Xv37+PxYsX49dff0VaWhoKCwslyxMTE7USjsjQuLtL25yRnoioaqtUITR27FgcOXIEI0aMgLu7e6kjQRMZu5Urgc8/lzsFERFVVqUKoZ9//hl79+5F586dtZ2HyOA5OADF5+7NzwcsZB2IgoiIKqtS1wg988wzqFGjhrazEFUJxYsgQPNF1EREVDVUqhBauHAh5syZgxxeIEEmaMgQaXv/fnlyEBHR06vUgIqtWrXC5cuXIYRA3bp1YWlpKVl+9uxZrQXUNg6oSE8rIwN45pnHbWtrzROzEhGR9ujq97tSVzYMGDBAawGIqhoHB6B6dSA7W9k+cEDePEREVHmVKoTmzp2r7RxEVcbdu4+LIAC4dAl4/nn58hARUeU91b0uZ86cQUJCAhQKBZo2bYpWrVppKxeRwXJykrbHjQMCA4E6dWSJQ0RET6FShVBaWhqGDBmCw4cPw8nJCUIIZGZmolu3bti6dStq1aql7ZxEBsPcHOjeHTh48HGftzcQHw80aSJfLiIiqrhK3TX21ltvISsrCxcvXsSdO3dw9+5d/Pnnn8jKysKUKVO0nZHI4Pz6q3rfwIH6z0FERE+nUneNOTo64pdffkHbtm0l/X/88QcCAwORkZGhrXxax7vGSFv+/Rdo0EDaV/F/TUREVB66+v2u1BGhwsJCtVvmAcDS0lJt3jEiY1W/vrTNewiIiKqeShVC3bt3x9SpU3Gz2FTcN27cwPTp09GjRw+thSMyZJcuPX5uaQlMny5fFiIiqpxKFUIrV65EdnY26tatC19fX9SvXx8+Pj7Izs7G55yBkkxE8SNAPj6Ao6N8WYiIqHIqdddY7dq1cfbsWURHR+Ovv/6CEAJNmzbFCy+8oO18RAZr69bHz3m3GBFR1VSpi6WrMl4sTdqiUEjb06YBs2YBzs6yxCEiMmqyT7GxYsUKvPHGG7CxscGKFSvKXJe30JMpqFsXuHLlcXv5cqCgAHjCPw8iIjIg5T4i5OPjg9OnT6NmzZrw8fEp/QUVCiQmJmotoLbxiBBpy+3bmo/+pKcDNWvqPw8RkTGT/YhQUlKSxudEpqpmTaCwEDArccvBzJnAmjXyZCIiooqp1F1jJRUUFCAuLg53797VxssRVRkKBXD2rLRv7Vp5shARUcVVqhCaNm0aIiMjASiLoOeffx6tW7dG7dq1cfjwYW3mIzJ4JecafvFFeXIQEVHFVaoQ2rFjB1q0aAEA2LNnD65cuYK//voL06ZNw6xZs7QakKiqiYvjVBtERFVFpQqh9PR0uLm5AQCioqLw6quvomHDhhgzZgwuXLig1YBEVU1qqvK6oVu35E5CRERPUqlCyNXVFfHx8SgoKMC+fftUAynm5OTA3NxcqwGJqoJ69dT7ONsMEZHhq1QhFBISgsGDB8PPzw8KhQI9e/YEAJw8eRKNGzfWakCiquC334BRo6R9/ztoSkREBqzSI0vv2LED165dw6uvvgovLy8AwNdffw0nJyf0799fqyG1ieMIkS6VHG2a1woREWmHrn6/OcUGkZYIoT6m0B9/AG3bypOHiMiYyD6gIqfYICqbQgEsXQrMmPG4b+pU4MQJ+TIREVHZOMUGkRbl5wOWltI+0zrmSkSkG7IfEeIUG0RPZlHiX1SHDvLkICKi8tHKFBtEpBQbK23v3StPDiIiKp9KFUKDBg3C4sWL1fo/+eQTvPrqq08diqiqCg+XtqtXlycHERGVT6UKoSNHjqBPnz5q/S+++CKOHj361KGIqqqSw2hZWQE5OfJkISKiJ6tUIXTv3j1YWVmp9VtaWiIrK+upQxFVVcXvGCuyaZP+cxARUflUqhDy8/PDtm3b1Pq3bt2Kpk2bPnUooqrKwQE4fVrax8HWiYgMV7nvGitu9uzZeOWVV3D58mV0794dAPDrr79iy5Yt2L59u1YDElU1hw5J2126yJODiIierNIjS+/duxeLFi1CXFwcbG1t0bx5c8ydOxcBAQHazqhVHEeIdK3kNBsxMbyNnojoaXGKDS1hIUS61rAh8M8/0j7T+ldGRKR9uvr9rvQ4QhkZGVi7di3ef/993LlzBwBw9uxZ3LhxQ2vhiKqiI0fkTkBEROVVqWuEzp8/jxdeeAGOjo64cuUKxo4dixo1amDnzp24evUqvvnmG23nJKoy3N2BBg3UjwoREZHhqdQRodDQUIwePRr//PMPbGxsVP1BQUEcR4gI0iKoRw/5chARUdkqVQidOnUK48ePV+v39PREamrqU4ciqspu3pS2BwyQJQYREZVDpQohGxsbjQMn/v3336hVq9ZThyKqym7dkrZ37pQnBxERPVmlCqH+/ftjwYIFyMvLAwAoFAokJydj5syZeOWVV7QakKiqadVK2q5RQ54cRET0ZJUqhJYuXYr//vsPLi4uePDgAQICAlC/fn1Ur14dH330kbYzElUpaWnS9sSJ8uQgIqInq9RdYw4ODjh+/DgOHjyIs2fPorCwEK1bt8YLL7yg7XxEVU7Pno+fu7sD/xt8nYiIDFCFB1TMz8+HjY0N4uLi4Ofnp6tcOsMBFUnXSo4snZUFVK8uTxYiImNhMAMqWlhYwNvbGwUFBVoLQWTMHByUxRARERmeSl0j9MEHHyAsLEw1ojQRPbZpk3rfrFn6z0FERE9WqWuEVqxYgX///RceHh7w9vaGvb29ZPnZs2e1Eo6oKhoyBLC1BV5++XFfQoJ8eYiIqHSVKoQGDBgAhUIBE5uvlahczMzUR5P+9Vd5shARUdkqVAjl5OTgnXfewa5du5CXl4cePXrg888/h7Ozs67yEVVJmi6OPnQI6NZN/1mIiKh0FbpGaO7cudiwYQP69OmDoUOH4pdffsFEDpJCpEahAKZNk/Z17w6sWydLHCIiKkWFbp/39fXFRx99hCFDhgAA/vjjD3Tu3BkPHz6Eubm5zkJqE2+fJ32JjQVat5b2uboCnI6PiKjiDOL2+WvXrqFLly6qdrt27WBhYYGbJWeZJCK0agV8+aW079Yt4OpVefIQEZG6ChVCBQUFsLKykvRZWFggPz9fq6GIjMUbbwB160r76tYFTpyQIw0REZVUoYulhRAYPXo0rK2tVX0PHz7EhAkTJLfQ//DDD9pLSFTF/fILUL++tK9zZ+DMGfVTZ0REpF8VKoRGjRql1vfaa69pLQyRMfL1BW7cADw9pf3+/gBHoCAikleFCqH169frKgeRUfPwAAYPBr77Ttp/7x5QrZo8mYiIqJJTbGhTREQEfHx8YGNjA39/fxw7dqxc2/3222+wsLBAy5YtdRuQSEu2bgU+/VTal5cnTxYiIlKStRDatm0bpk2bhlmzZiE2NhZdunRBUFAQkpOTy9wuMzMTI0eORI+Sw/cSGTCFAihxrwFq1AD+/VeePEREVMFxhLStffv2aN26NVavXq3qa9KkCQYMGIDw8PBStxsyZAgaNGgAc3Nz7Nq1C3FxceV+T44jRHK6dQtwc5P2DR8ObNwoTx4ioqrCIMYR0qbc3FycOXMGgYGBkv7AwECcKOPe4vXr1+Py5cuYO3duud7n0aNHyMrKkjyI5OLqCnz+ubSP4woREclHtkIoPT0dBQUFcHV1lfS7uroitZShd//55x/MnDkTmzZtgoVF+a7zDg8Ph6Ojo+pRu3btp85O9DTGjZO2jx/nESEiIrnIfrG0QqGQtIUQan2AcjDHYcOGYf78+WjYsGG5Xz8sLAyZmZmqx7Vr1546M9HT0PQnOGIEb6UnIpJDhW6f1yZnZ2eYm5urHf1JS0tTO0oEANnZ2Th9+jRiY2Px5ptvAgAKCwshhICFhQUOHDiA7t27q21nbW0tGQCSSG716ytPj731lrTfwwO4eVN5UTUREemHbEeErKys4O/vj+joaEl/dHQ0OnXqpLa+g4MDLly4gLi4ONVjwoQJaNSoEeLi4tC+fXt9RSd6am++CezcKe1LTQXu3JEnDxGRqZLtiBAAhIaGYsSIEWjTpg06duyIr776CsnJyZgwYQIA5WmtGzdu4JtvvoGZmRn8/Pwk27u4uMDGxkatn6gq6NcPGDhQWhC9+CKwYwfg7S1fLiIiUyJrIRQcHIzbt29jwYIFSElJgZ+fH6KiouD9v1+BlJSUJ44pRFRVmZkB48dLC6HTp5WTsgYEKEehdnGRLR4RkUmQdRwhOXAcITIkhYVAmzZAbKz6smXLgOnT9Z+JiMgQGd04QkSkPCp09qzyUfIi6Xbt5MlERGRKWAgRGYBWrYAVK6R9Gu4ZICIiLWMhRGQAUlKkt9OHhPA2eiIifWAhRGQAPDyk7YYNOcAiEZE+sBAikllOjnpfWBiwZIn+sxARmRoWQkQys7VVXiNU0l9/6T8LEZGpYSFEJDOFAjhzRn3i1Q0bgOxsWSIREZkMFkJEBkChAPLy1Ptfekn/WYiITAkLISID8eqrQN++0r6jR+XJQkRkKlgIERkIe3vgk0+kfWFh8mQhIjIVLISIDERhIdC48eN2ixbARx/Jl4eIyBSwECIyENHR0va5c8ALLwCXLsmTh4jIFLAQIjIQdeuq9x08CHz9td6jEBGZDBZCRAaiUSPg5En1/t699Z+FiMhUsBAiMiDt2gG7d0v7nnsOOH1anjxERMaOhRCRgWneXL2vbVvg4UP9ZyEiMnYshIgMjLc3sH27en+3bpyIlYhI21gIERmgQYOA99+X9v3+O2BpCaSny5OJiMgYsRAiMlCzZ6v3FRQA4eH6z0JEZKxYCBEZKBsbYO9e5Smx4mrVkicPEZExYiFEZMB69wbWrZP2ff65PFmIiIwRCyEiA+fmJm3Xry9PDiIiY8RCiMjArV0rbX/xhTw5iIiMEQshIgO3ZIm0XbOmPDmIiIwRCyEiA3ftmrSdny9PDiIiY8RCiMiAJSVJ288/D3h4yJOFiMgYsRAiMlAFBUC9etK+X3+VJwsRkbFiIURkoD74QNoeMQLIy5MnCxGRsWIhRGSgFi+Wtr/9Fvj6a3myEBEZKxZCRAZq5Ur1Pmtr/ecgIjJmLISIDNTkyUCTJtI+jipNRKRdLISIDNjAgdJ2bCwwZAjw119Abq48mYiIjIlCCCHkDqFPWVlZcHR0RGZmJhwcHOSOQ/RECkXpy+ztgZQUoHp1/eUhIpKDrn6/eUSIyMDt21f6svv3AQcHICdHf3mIiIyJhdwBiKhsvXopC53ffwfmzweOHFFfx94e+Oyzx+2i47wdOwLt2uknJxFRVcRTY0RV0LZtymuFyuPHH4Fnn1U+r1MHMDfXXS4iIl3hqTEiUgkOBiZOLN+6/fsrR6iuV095F9qDB7rNRkRUlfCIEFEVdvYs8Pff0r5hw8reZvx44IsvdJeJiEgXdPX7zUKIyMjExyvHG7pyBXjmGWDLFvV1cnIAW1u9RyMiqjRd/X7zYmkiI9O0KbB69eN2kybAnDnSdV55Bdi7t+xb84mITAGvESIycu+/r97388/lv9iaiMiYsRAiMnLm5sCKFer9t2/rPwsRkaFhIURkAl5/HXB0lPYtXy5LFCIig8JCiMgEPHwIZGZK+3ivABERCyEik6Cp6HFx0X8OIiJDw0KIyAR8/716X0GB/nMQERkaFkJEJuCZZ9T7Bg3Sfw4iIkPDQojIBAQGAn5+0r6yZrUnIjIVLISITIBCoRxtmoiIpFgIEZmAEyeAbt3U+xUK4M4d/echIjIULISITMAff5S+rGZNZUHk6AhEROgvExGRIeBcY0QmYOxY4Pp14Pffgd9+07xOVhYwebLyf2fO1G8+IiK58IgQkQmoVg1YuhQ4flx52/wbb5S+blgY8OWX+stGRCQnFkJEJsbMTFnoCKF8tGqlvs7s2frPRUQkBxZCRCYuMFC9r6wjRkRExkQhhBByh9CnrKwsODo6IjMzEw6cbIlM3IULQOvWQH6++rLCQuVF1EREhkBXv988IkRkwm7d0lwEAcDhw3qNQkQkCxZCRCasRw9g82bNy3x89JuFiEgOLISITJhCAfz1l3r/X38BdevqPQ4Rkd5xHCEiE7ZrF7BggbQvO1t5uz0RkSngESEiExUbCwwcKO37v/9jEUREpoWFEJEJKrpbrLiwMGDKFHnyEBHJhafGiEzMr78CL7wg7du9G+jbV548RERykv2IUEREBHx8fGBjYwN/f38cO3as1HV/+OEH9OzZE7Vq1YKDgwM6duyI/fv36zEtUdX2/ffqRVB0NIsgIjJdshZC27Ztw7Rp0zBr1izExsaiS5cuCAoKQnJyssb1jx49ip49eyIqKgpnzpxBt27d0LdvX8TGxuo5OVHVk5MDDBok7WvdGoiKUk6pcfKkPLmIiOQk68jS7du3R+vWrbF69WpVX5MmTTBgwACEh4eX6zWaNWuG4OBgzJkzp1zrc2RpMlU3bwKenmWvM26cch4yjihNRIbG6EaWzs3NxZkzZxBYYqKjwMBAnDhxolyvUVhYiOzsbNSoUaPUdR49eoSsrCzJg8gUeXg8eZ01a4Dr13WfhYjIUMhWCKWnp6OgoACurq6SfldXV6SmppbrNT799FPcv38fgwcPLnWd8PBwODo6qh61a9d+qtxEVdnDh8ChQ8rTYT//rHmdBw/0m4mISE6y3zWmKHEMXgih1qfJli1bMG/ePPz4449wcXEpdb2wsDCEhoaq2llZWSyGyGRZWwNduyqfCwGYmwMFBdJ1GjVSLiMiMgWyHRFydnaGubm52tGftLQ0taNEJW3btg1jxozBd999hxdK3gJTgrW1NRwcHCQPIlJeB7RwoeZlr71W+mSsRETGRLZCyMrKCv7+/oiOjpb0R0dHo1OnTqVut2XLFowePRqbN29Gnz59dB2TyKiFhWk++rNpE/DWW/rPQ0Skb7LePh8aGoq1a9di3bp1SEhIwPTp05GcnIwJEyYAUJ7WGjlypGr9LVu2YOTIkfj000/RoUMHpKamIjU1FZmZmXJ9BCKjcPSoel96uv5zEBHpm6zXCAUHB+P27dtYsGABUlJS4Ofnh6ioKHh7ewMAUlJSJGMKffnll8jPz8fkyZMxefJkVf+oUaOwYcMGfccnMhrPPKPe93//p/8cRET6Jus4QnLgOEJEUmlpQIcOQFKStN+0/stARIbO6MYRIiLDcPq0ehEEAFOnArm5+s9DRKRPLISITFzJ2+eLrFgB7Nmj3yxERPrGQojIxDVtWvoyZ2feRk9Exo2FEJEJu38f6N279OVduwL16wMZGfpKRESkXyyEiExYejpw6VLZ61y9yvnHiMh4yT7FBhHJx9sb+PprYN065TxkDx8C585J12nQAGjcWJ58RES6xiNCRCZu5Ejg8GHg99+B+fPVl//zD7B5s95jERHpBQshIlLp21dzf506+s1BRKQvLISICIDyOqAuXdT7jx17PGM9EZGxYSFEZOL++QcYOxaoVw84cUK67ORJ4Lnn5MlFRKQPLISITNiKFUDDhkBkJJCX97i/c2fl3WLt2smXjYhIH3jXGJEJmzpVve/WLcDFRf9ZiIjkwCNCRKRSqxaLICIyLSyEiEyMEMC+fUBQkPqyyZP1n4eISE48NUZkQoQAqlUDcnLUl/XtC8yZo/9MRERy4hEhIhPy/feaiyBAOZiiQqHfPEREcmMhRGRCLC1LX7Zqlf5yEBEZChZCRCakXz/g4481L3vlFf1mISIyBCyEiEyIQgGsWaN5Wa9e+s1CRGQIWAgRmZDYWODff9X733oLMON/DYjIBPE/fUQm4No1oHZtoHVr9WX29kCjRkB+vv5zERHJjYUQkQkYM0Y5qaom9+8Db74J+PvrNxMRkSFgIURkApo0efI6588rxxkiIjIlLISITMDy5cAffwBvvAE4OWlep2dPjiNERKaHI0sTGbk7dwA3N+ns8iX93/9pnoCViMjYsRAiMlKXLimnzbh0SX1Z3brAa68Bw4cDjRvrPRoRkcFgIURkhO7eVV78fO+e5uWrVgG9e+s3ExGRIeI1QkRGKC8PKCgofXmfPqXfRUZEZEpYCBEZIRcX4NQp4L33gHHjNK/Tvr1+MxERGSKeGiMyUs2aAWFhpd8l1qmTXuMQERkkHhEiMmKjR5e+7Ntv9RaDiMhgsRAiMlIXLgC7dmle1rcvYGOj1zhERAaJhRCRESosBJo3L335zz8DO3fqLw8RkaFiIURkhJ40VUZ+PvDRR/rJQkRkyHixtAZCCOTn56OgrPuPyeiZm5vDwsICiio474S5OfDdd8DgwaWv07w5cPiw8ujR7dtA9+5AzZp6i0hEZBBYCJWQm5uLlJQU5OTkyB2FDICdnR3c3d1hZWUld5QKS0kpe/n69cpHEXNz4N9/laNOExGZChZCxRQWFiIpKQnm5ubw8PCAlZVVlTwaQE9PCIHc3Fz8999/SEpKQoMGDWBmVrXOJA8aVLH5wwoKgP/+YyFERKaFhVAxubm5KCwsRO3atWFnZyd3HJKZra0tLC0tcfXqVeTm5sKmit1m5eEBJCQAH38MHDwIJCeXvX5YGNC2rX6yEREZChZCGlS1/+dPulPV/xbs7IAaNUovgtq2BTp3Bt5+G/Dy0m82IiJDwEKIyMjcuaMcSHHPnrLXS00FXF31EomIyGCxECIyIunpwCuvAEePlr6OlRXw6JH+MhERGbKqfdyfyqVRo0awsrLCjRs31JZ17doVCoUCCoUC1tbWaNiwIRYtWqTToQOEEJg3bx48PDxga2uLrl274uLFi2Vu88MPP6BNmzZwcnKCvb09WrZsiW81zBEREREBHx8f2NjYwN/fH8eOHVN7nV69esHZ2RkKhQJxcXHa/GiySEsDOnQAFAqgVq2yiyDgycuJiEwJCyEjd/z4cTx8+BCvvvoqNmzYoHGdcePGISUlBX///TemTJmCDz74AEuXLtVZpiVLlmDZsmVYuXIlTp06BTc3N/Ts2RPZ2dmlblOjRg3MmjULMTExOH/+PEJCQhASEoL9+/er1tm2bRumTZuGWbNmITY2Fl26dEFQUBCSi10gc//+fXTu3BmLFy/W2efTtXv3gK5dlYWPQqE8vXXypOZ1zc2Vk6tOnw5s3QrcvctZ54mIilMI8aQxaI1LVlYWHB0dkZmZCQcHB8myhw8fIikpSXVEwRiEhITAzc0NAQEBmDx5Mv7991/JkABdu3ZFy5YtsXz5clVfz549ce/ePcTExGg9jxACHh4emDZtGt577z0AwKNHj+Dq6oqPP/4Y48ePL/drtW7dGn369MHChQsBAO3bt0fr1q2xevVq1TpNmjTBgAEDEB4eLtn2ypUr8PHxQWxsLFq2bFnqe8j9N1FYCFy+DJw9C8TGKv83OvrJ261dC7zwAlCnjrJYIiKq6sr6/X4avEbIiGVnZ2P79u04efIkGjdujPv37+Pw4cPo1q1bmdvZ2tri7t27pS4PCgpSO+VU0r179zT2JyUlITU1FYGBgao+a2trBAQE4MSJE+UqhIQQOHjwIP7++298/PHHAJRDH5w5cwYzZ86UrBsYGIgTJ0488TUNyfnzylGeb98u/zbTpgF+fkCbNsCzzwJV/GY3IiK9YSFUDm3aKO+w0Tc3N+D06cpvv3XrVjRo0ADNmjUDAAwZMgSRkZGlFkKFhYU4cOAA9u/fj2nTppX6umvXrsWDBw8qlSn1f1+ka4nblVxdXXH16tUyt83MzISnpycePXoEc3NzREREoGfPngCA9PR0FBQUaHzdVDl2XiU9eqQ8lXX/fvm3uXcPsLfXXSYiImPGQqgcUlMBDdcZG7zIyEi89tprqvZrr72G559/HhkZGXByclL1R0REYO3atcjNzQUAjBgxAnPnzi31dT09PZ86W8kRu4UQTxzFu3r16oiLi8O9e/fw66+/IjQ0FPXq1UPXrl2f6nUNiUKhLGrKWwjVqwfY2uo2ExGRMWMhVA5ublXvfePj43Hy5EmcOnVKdS0OABQUFGDLli2YOHGiqm/48OGYNWsWrK2t4eHhAXNz8zJf+2lOjbn970OlpqbC3d1d1Z+WlqZ2NKckMzMz1K9fHwDQsmVLJCQkIDw8HF27doWzszPMzc3Vjv6U53UNRVwc0KpV+da1tQXeeANYsICnwYiIngYLoXJ4mtNTcomMjMTzzz+PVatWSfq//fZbREZGSgohR0dHVYFRHk9zaszHxwdubm6Ijo5Gq//96ufm5uLIkSOq633KSwiBR/8bEMfKygr+/v6Ijo7GwIEDVetER0ejf//+lcqqb8UnQH2S994DyjhoR0RE5cRCyAjl5eXh22+/xYIFC+Dn5ydZNnbsWCxZsgTnzp1DixYtKvX6T3NqTKFQYNq0aVi0aBEaNGiABg0aYNGiRbCzs8OwYcNU640cORKenp6qu73Cw8PRpk0b+Pr6Ijc3F1FRUfjmm28kd4iFhoZixIgRaNOmDTp27IivvvoKycnJmDBhgmqdO3fuIDk5GTdv3gQA/P333wCUR6rc5Dr09z8hIcDevcq7xJ5k3jzlEcMK3GRHREQasBAyQrt378bt27clR0aKNGjQAM8++ywiIyOxYsUKGdIB7777Lh48eIBJkybh7t27aN++PQ4cOIDq1aur1klOTpbM83X//n1MmjQJ169fh62tLRo3boyNGzciODhYtU5wcDBu376NBQsWICUlBX5+foiKioK3t7dqnd27dyMkJETVHjJkCABg7ty5mDdvng4/ddmEUI4HFBQErFxZvm0++4yFEBHR0+I4QsXIPWYMGR59/E389hvw3HOV29bCAsjPf/J6y5cDU6dW7j2IiAyBrsYR4mWWRDL74YfKb1ueIggA1qyp/HsQERkzFkJEMpswQTlXmC5V8Dp0IiKTwWuEiGTWoAFQNJvJjh3Aq68+/Wt6eCin16hWTdlevlz5AJTTdty9q5ysNS0NyMtT3/6115Tr16z59FmIiAwZCyEiA1JQoJ3XuXlT+aisjRuB1q2Vk7USERkzFkJEBmTwYMDFBfjoI8DLS/lwdwcsLYGEhMdHdfSh2IDdRERGi4WQBiZ2Ix2VQd9/CwqF8g6y5cuBAQPKN6ZQRTg5KQutJz3q1QOsrbX73kREhoiFUDGWlpYAgJycHNhyAieC8m8BePy3oQt//QV07AhkZOjsLfD118CIEcpCi4iIHmMhVIy5uTmcnJyQlpYGALCzs6tSE3aS9gghkJOTg7S0NDg5OT1x/rWKys9Xzhqfna0cFFGXRRAALFkCjByp2/cgIqqKWAiVUDTNQlExRKbNyclJbeqNrCzgq6+A1FTlXVe3bj2+A6tRI6BJk8dFzr17mp//b4o0vXnzTf2+HxFRVcFCqASFQgF3d3e4uLggT9N9xWTwCguBhw+VjwcP1J/n5JS+7MGDx4+0NEtkZJgjJweqx9WryukwSnPzJnDokG4+V5cugKOj8uHgUL7nDg7K0aeJiEgz2f8TGRERgU8++QQpKSlo1qwZli9fji5dupS6/pEjRxAaGoqLFy/Cw8MD7777rmRSTW0xNzfX+ukQ0p7CQiA3V/m4dQto2rT8oyzLwc5OOaZPtWpA9erqzy9dUo4l5On5+OHiolw+fjzg6yv3JyAiMk6yFkLbtm3DtGnTEBERgc6dO+PLL79EUFAQ4uPjUadOHbX1k5KS0Lt3b4wbNw4bN27Eb7/9hkmTJqFWrVp45ZVXZPgExkmIx0VGbq7yNE5p7bKW6XJbfRY9ZmaAvb2ymLGzA5KSSl+3Rg1g0ybAx+dxkWNvD7CmJiIyTLJOutq+fXu0bt0aq1evVvU1adIEAwYMQHh4uNr67733Hnbv3o2EhARV34QJE3Du3DnEFA3N+wS6mrTNGEyeDEREyJ3CsC1cCHh7K4vFon85fF7x57p8jx49lHfImXECISKjoqvfb9mOCOXm5uLMmTOYOXOmpD8wMBAnTpzQuE1MTAwCAwMlfb169UJkZCTy8vIqdIuzo2PFMxPNni13AnqSb78FRo8GBg2SOwkRaZOuLtuVrRBKT09HQUEBXF1dJf2urq5ITU3VuE1qaqrG9fPz85Geng53d3e1bR49eoRHxW7RyczM/N+zrKf7AERk0HbskDsBEWmX8ndb2yeyZL9YuuQ4PUKIMsfu0bS+pv4i4eHhmD9/voYltSsWlIiIiGR3+/ZtOGrxtI5shZCzszPMzc3Vjv6kpaWpHfUp4ubmpnF9CwsL1CxlmuywsDCEhoaq2hkZGfD29kZycrJWv0iqnKysLNSuXRvXrl3jNVsy474wHNwXhoP7wnBkZmaiTp06qFGjhlZfV7ZCyMrKCv7+/oiOjsbAgQNV/dHR0ejfv7/GbTp27Ig9e/ZI+g4cOIA2bdqUen2QtbU1rDVMmuTo6Mg/agPi4ODA/WEguC8MB/eF4eC+MBxmWr4TQtb7KkJDQ7F27VqsW7cOCQkJmD59OpKTk1XjAoWFhWFksXkBJkyYgKtXryI0NBQJCQlYt24dIiMjMWPGDLk+AhEREVVhsl4jFBwcjNu3b2PBggVISUmBn58foqKi4O3tDQBISUlBcnKyan0fHx9ERUVh+vTpWLVqFTw8PLBixQqOIURERESVIvvF0pMmTcKkSZM0LtuwYYNaX0BAAM6ePVvp97O2tsbcuXM1ni4j/eP+MBzcF4aD+8JwcF8YDl3tC1kHVCQiIiKSE8deJSIiIpPFQoiIiIhMFgshIiIiMlkshIiIiMhkGWUhFBERAR8fH9jY2MDf3x/Hjh0rc/0jR47A398fNjY2qFevHr744gs9JTV+FdkXP/zwA3r27IlatWrBwcEBHTt2xP79+/WY1vhV9N9Gkd9++w0WFhZo2bKlbgOakIrui0ePHmHWrFnw9vaGtbU1fH19sW7dOj2lNW4V3RebNm1CixYtYGdnB3d3d4SEhOD27dt6Smu8jh49ir59+8LDwwMKhQK7du164jZa+f0WRmbr1q3C0tJSrFmzRsTHx4upU6cKe3t7cfXqVY3rJyYmCjs7OzF16lQRHx8v1qxZIywtLcWOHTv0nNz4VHRfTJ06VXz88cfijz/+EJcuXRJhYWHC0tJSnD17Vs/JjVNF90eRjIwMUa9ePREYGChatGihn7BGrjL7ol+/fqJ9+/YiOjpaJCUliZMnT4rffvtNj6mNU0X3xbFjx4SZmZn47LPPRGJiojh27Jho1qyZGDBggJ6TG5+oqCgxa9Ys8f333wsAYufOnWWur63fb6MrhNq1aycmTJgg6WvcuLGYOXOmxvXfffdd0bhxY0nf+PHjRYcOHXSW0VRUdF9o0rRpUzF//nxtRzNJld0fwcHB4oMPPhBz585lIaQlFd0XP//8s3B0dBS3b9/WRzyTUtF98cknn4h69epJ+lasWCG8vLx0ltEUlacQ0tbvt1GdGsvNzcWZM2cQGBgo6Q8MDMSJEyc0bhMTE6O2fq9evXD69Gnk5eXpLKuxq8y+KKmwsBDZ2dlan2DPFFV2f6xfvx6XL1/G3LlzdR3RZFRmX+zevRtt2rTBkiVL4OnpiYYNG2LGjBl48OCBPiIbrcrsi06dOuH69euIioqCEAK3bt3Cjh070KdPH31EpmK09fst+8jS2pSeno6CggK12etdXV3VZq0vkpqaqnH9/Px8pKenw93dXWd5jVll9kVJn376Ke7fv4/BgwfrIqJJqcz++OeffzBz5kwcO3YMFhZG9Z8KWVVmXyQmJuL48eOwsbHBzp07kZ6ejkmTJuHOnTu8TugpVGZfdOrUCZs2bUJwcDAePnyI/Px89OvXD59//rk+IlMx2vr9NqojQkUUCoWkLYRQ63vS+pr6qeIqui+KbNmyBfPmzcO2bdvg4uKiq3gmp7z7o6CgAMOGDcP8+fPRsGFDfcUzKRX5t1FYWAiFQoFNmzahXbt26N27N5YtW4YNGzbwqJAWVGRfxMfHY8qUKZgzZw7OnDmDffv2ISkpSTVZOOmXNn6/jer/5jk7O8Pc3Fytkk9LS1OrGou4ublpXN/CwgI1a9bUWVZjV5l9UWTbtm0YM2YMtm/fjhdeeEGXMU1GRfdHdnY2Tp8+jdjYWLz55psAlD/GQghYWFjgwIED6N69u16yG5vK/Ntwd3eHp6cnHB0dVX1NmjSBEALXr19HgwYNdJrZWFVmX4SHh6Nz58545513AADNmzeHvb09unTpgg8//JBnEfRIW7/fRnVEyMrKCv7+/oiOjpb0R0dHo1OnThq36dixo9r6Bw4cQJs2bWBpaamzrMauMvsCUB4JGj16NDZv3sxz7lpU0f3h4OCACxcuIC4uTvWYMGECGjVqhLi4OLRv315f0Y1OZf5tdO7cGTdv3sS9e/dUfZcuXYKZmRm8vLx0mteYVWZf5OTkwMxM+tNpbm4O4PHRCNIPrf1+V+jS6iqg6FbIyMhIER8fL6ZNmybs7e3FlStXhBBCzJw5U4wYMUK1ftHtd9OnTxfx8fEiMjKSt89rSUX3xebNm4WFhYVYtWqVSElJUT0yMjLk+ghGpaL7oyTeNaY9Fd0X2dnZwsvLSwwaNEhcvHhRHDlyRDRo0ECMHTtWro9gNCq6L9avXy8sLCxERESEuHz5sjh+/Lho06aNaNeunVwfwWhkZ2eL2NhYERsbKwCIZcuWidjYWNVQBrr6/Ta6QkgIIVatWiW8vb2FlZWVaN26tThy5Ihq2ahRo0RAQIBk/cOHD4tWrVoJKysrUbduXbF69Wo9JzZeFdkXAQEBAoDaY9SoUfoPbqQq+m+jOBZC2lXRfZGQkCBeeOEFYWtrK7y8vERoaKjIycnRc2rjVNF9sWLFCtG0aVNha2sr3N3dxfDhw8X169f1nNr4HDp0qMzfAF39fiuE4LE8IiIiMk1GdY0QERERUUWwECIiIiKTxUKIiIiITBYLISIiIjJZLISIiIjIZLEQIiIiIpPFQoiIiIhMFgshIiIAdevWxfLly1VthUKBXbt2yZaHiPSDhRARyW706NFQKBRQKBSwsLBAnTp1MHHiRNy9e1fuaERk5FgIEZFBePHFF5GSkoIrV65g7dq12LNnDyZNmiR3LCIyciyEiMggWFtbw83NDV5eXggMDERwcDAOHDigWr5+/Xo0adIENjY2aNy4MSIiIiTbX79+HUOGDEGNGjVgb2+PNm3a4OTJkwCAy5cvo3///nB1dUW1atXQtm1b/PLLL3r9fERkmCzkDkBEVFJiYiL27dsHS0tLAMCaNWswd+5crFy5Eq1atUJsbCzGjRsHe3t7jBo1Cvfu3UNAQAA8PT2xe/duuLm54ezZsygsLAQA3Lt3D71798aHH34IGxsbfP311+jbty/+/vtv1KlTR86PSkQyYyFERAbhp59+QrVq1VBQUICHDx8CAJYtWwYAWLhwIT799FO8/PLLAAAfHx/Ex8fjyy+/xKhRo7B582b8999/OHXqFGrUqAEAqF+/vuq1W7RogRYtWqjaH374IXbu3Indu3fjzTff1NdHJCIDxEKIiAxCt27dsHr1auTk5GDt2rW4dOkS3nrrLfz333+4du0axowZg3HjxqnWz8/Ph6OjIwAgLi4OrVq1UhVBJd2/fx/z58/HTz/9hJs3byI/Px8PHjxAcnKyXj4bERkuFkJEZBDs7e1VR3FWrFiBbt26Yf78+aojNmvWrEH79u0l25ibmwMAbG1ty3ztd955B/v378fSpUtRv3592NraYtCgQcjNzdXBJyGiqoSFEBEZpLlz5yIoKAgTJ06Ep6cnEhMTMXz4cI3rNm/eHGvXrsWdO3c0HhU6duwYRo8ejYEDBwJQXjN05coVXcYnoiqCd40RkUHq2rUrmjVrhkWLFmHevHkIDw/HZ599hkuXLuHChQtYv3696hqioUOHws3NDQMGDMBvv/2GxMREfP/994iJiQGgvF7ohx9+QFxcHM6dO4dhw4apLqQmItPGQoiIDFZoaCjWrFmDXr16Ye3atdiwYQOeffZZBAQEYMOGDfDx8QEAWFlZ4cCBA3BxcUHv3r3x7LPPYvHixapTZ//3f/+HZ555Bp06dULfvn3Rq1cvtG7dWs6PRkQGQiGEEHKHICIiIpIDjwgRERGRyWIhRERERCaLhRARERGZLBZCREREZLJYCBEREZHJYiFEREREJouFEBEREZksFkJERERkslgIERERkcliIUREREQmi4UQERERmSwWQkRERGSy/h/hH+3e8xMKkQAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "precision, recall, thresholds = precision_recall_curve(y_test, y_proba_test, pos_label=1, sample_weight=None)\n",
    "avg_p_test = average_precision_score(y_test, y_proba_test, pos_label=1, sample_weight=None)\n",
    "\n",
    "plt.title('Precision-Recall Curve (Test)')\n",
    "ax1= plt.plot(precision, recall, 'b', label = '%s AP = %0.3f' % ('', avg_p_test), linewidth=2)\n",
    "plt.legend(loc = 'lower left')\n",
    "plt.xlim([0, 1])\n",
    "plt.ylim([0, 1])\n",
    "plt.ylabel('Precision')\n",
    "plt.xlabel('Recall')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1cdabf91-7a3d-4eaf-8bbb-b79da4f83dc0",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "#### **Additional Questions to Consider:**\n",
    "\n",
    "* Do the most predictive features make clinical sense?"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e5e4013f-2162-4aec-87e7-7fc266999173",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "source": [
    "* Which patients is are model classifying incorrectly? - During training, we can look at which patients the model may be predicting wrong. What stands out about the FP or FN patients? Perhaps this could inform our training process."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "62ac492c-62d7-4afe-8211-f9b19190194a",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "source": [
    "* What metrics should we prioritize when evaluating this model?\n",
    "\n",
    "    * In this case, we focused on AUC, but performance metrics are often chosen based on the clinical outcome of interest and the dataset. For example, if our model is trying to classify patients with a criticall ilnness, there may be a high cost to missing a potential diagnosis. We may want to emphasize the sensitivity of our model. On the otherhand, if our model is predicting every patient as a potential diagnosis, healthcare workers might get fatigued from constant false positives."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b070dcc7-7e91-43a3-829f-d1f6336f4082",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "source": [
    "* How might the following affect model performance:\n",
    "\n",
    "    * Missing values\n",
    "\n",
    "    * Class imbalance"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b33162b8-6ce4-48b1-a215-366391b5ae6c",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "source": [
    "* What other types of experiments could we perform with this dataset? Is mortality prediction the best use case? Why or why not?"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "be2ac429-3f89-4adb-aeb3-0e5c3f1b4b18",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "# An Extra Exercise!"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "588e52f7",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "## Random Forest Algorithm\n",
    "\n",
    "<img src=\"lecture_images/decision_tree.png\" alt=\"title\" width=\"400\" align=\"left\">\n",
    "\n",
    "* Random forest is an esembling algorithm that learns from generating multiple (hundreds-thousands) of decision trees\n",
    "* Randomly selects a subset of features when growing each tree \n",
    "* This approach allows us to average the predictions of the \"crowd\"\n",
    "\n",
    "<b></b>\n",
    "* Many **hyperparameters** that can be tuned during training of a random forest algorithm:\n",
    "<br></br>\n",
    "* `n_estimators`: Number of trees\n",
    "* `max_features`: Number of featues to be considered at each split \n",
    "* `max_depth`: Maximum number of levels in tree (can help regularize the tree to prevent overfitting)\n",
    "* `min_samples_split`: Minimum samples to split a node \n",
    "* `min_samples_leaf`: Minimum number of samples required at each leaf node "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cd4c57a7-96e5-452c-a9e6-16c713220abd",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "For random forest, we don't need to standardize our features as the tree-based models are able to deal with different feature ranges more intrinsically, thus, we'll need to specify a slightly different pipeline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 100,
   "id": "79b0b11a-c63e-4d55-b1dd-8fccee12a645",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "# construct pipeline for preprocessing data \n",
    "numeric_transformer_rf = Pipeline(steps=[\n",
    "    ('imputer', SimpleImputer(strategy='median')),\n",
    "    #('scaler', StandardScaler())\n",
    "])\n",
    "\n",
    "non_numeric_transformer_rf = 'passthrough'  # Pass non-numeric features unchanged\n",
    "\n",
    "preprocessor_rf = ColumnTransformer(\n",
    "    transformers=[\n",
    "        ('num', numeric_transformer_rf, numeric_features),\n",
    "        ('non_num', non_numeric_transformer_rf, non_numeric_features)\n",
    "    ]\n",
    ")\n",
    "\n",
    "# Create a random forest classifier\n",
    "rf = RandomForestClassifier(criterion='entropy', random_state=12345, class_weight = 'balanced')\n",
    "\n",
    "# Create a pipeline with preprocessing and a model\n",
    "model_pipeline_rf = Pipeline(steps=[\n",
    "    ('preprocessor', preprocessor_rf),\n",
    "    ('classifier', rf)  # Replace with your model of choice\n",
    "])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d4a9d3a8-d97c-4999-96be-39239575169f",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "Specify the hyperparameter grid and initiate `GridSearchCV`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 103,
   "id": "802d4a35",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "### tuning RF hyperparameters\n",
    "# Number of trees in random forest\n",
    "n_estimators = [100, 300, 500, 1000]\n",
    "# Number of features to consider at every split\n",
    "max_features = [2, 5]\n",
    "# Maximum number of levels in tree\n",
    "max_depth = [5, 8]\n",
    "# Minimum number of samples required to split a node\n",
    "min_samples_split = [5, 10]\n",
    "# Minimum number of samples required at each leaf node\n",
    "min_samples_leaf = [10, 15]\n",
    "\n",
    "# create grid of hypterparameter settings\n",
    "# we will permute through this grid\n",
    "rf_param_grid = {'classifier__n_estimators': n_estimators,\n",
    "               'classifier__max_features': max_features,\n",
    "               'classifier__max_depth': max_depth,\n",
    "               'classifier__min_samples_split': min_samples_split,\n",
    "               'classifier__min_samples_leaf': min_samples_leaf}\n",
    "\n",
    "# Set up GridSearchCV\n",
    "rf_grid_search = GridSearchCV(estimator=model_pipeline_rf, # specific our pipeline as our estimator \n",
    "                              param_grid=rf_param_grid,\n",
    "                              cv=5,\n",
    "                              scoring='roc_auc', # this specifies that we are going to use the AUC as a metric to evaluate how well our model does across folds\n",
    "                              return_train_score=True,\n",
    "                              refit=True,\n",
    "                              n_jobs=-1)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4be2f9ff-18f3-47aa-b2c1-294e7619c891",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "Let's train the model!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 104,
   "id": "f2e006f2",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<style>#sk-container-id-2 {\n",
       "  /* Definition of color scheme common for light and dark mode */\n",
       "  --sklearn-color-text: #000;\n",
       "  --sklearn-color-text-muted: #666;\n",
       "  --sklearn-color-line: gray;\n",
       "  /* Definition of color scheme for unfitted estimators */\n",
       "  --sklearn-color-unfitted-level-0: #fff5e6;\n",
       "  --sklearn-color-unfitted-level-1: #f6e4d2;\n",
       "  --sklearn-color-unfitted-level-2: #ffe0b3;\n",
       "  --sklearn-color-unfitted-level-3: chocolate;\n",
       "  /* Definition of color scheme for fitted estimators */\n",
       "  --sklearn-color-fitted-level-0: #f0f8ff;\n",
       "  --sklearn-color-fitted-level-1: #d4ebff;\n",
       "  --sklearn-color-fitted-level-2: #b3dbfd;\n",
       "  --sklearn-color-fitted-level-3: cornflowerblue;\n",
       "\n",
       "  /* Specific color for light theme */\n",
       "  --sklearn-color-text-on-default-background: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, black)));\n",
       "  --sklearn-color-background: var(--sg-background-color, var(--theme-background, var(--jp-layout-color0, white)));\n",
       "  --sklearn-color-border-box: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, black)));\n",
       "  --sklearn-color-icon: #696969;\n",
       "\n",
       "  @media (prefers-color-scheme: dark) {\n",
       "    /* Redefinition of color scheme for dark theme */\n",
       "    --sklearn-color-text-on-default-background: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, white)));\n",
       "    --sklearn-color-background: var(--sg-background-color, var(--theme-background, var(--jp-layout-color0, #111)));\n",
       "    --sklearn-color-border-box: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, white)));\n",
       "    --sklearn-color-icon: #878787;\n",
       "  }\n",
       "}\n",
       "\n",
       "#sk-container-id-2 {\n",
       "  color: var(--sklearn-color-text);\n",
       "}\n",
       "\n",
       "#sk-container-id-2 pre {\n",
       "  padding: 0;\n",
       "}\n",
       "\n",
       "#sk-container-id-2 input.sk-hidden--visually {\n",
       "  border: 0;\n",
       "  clip: rect(1px 1px 1px 1px);\n",
       "  clip: rect(1px, 1px, 1px, 1px);\n",
       "  height: 1px;\n",
       "  margin: -1px;\n",
       "  overflow: hidden;\n",
       "  padding: 0;\n",
       "  position: absolute;\n",
       "  width: 1px;\n",
       "}\n",
       "\n",
       "#sk-container-id-2 div.sk-dashed-wrapped {\n",
       "  border: 1px dashed var(--sklearn-color-line);\n",
       "  margin: 0 0.4em 0.5em 0.4em;\n",
       "  box-sizing: border-box;\n",
       "  padding-bottom: 0.4em;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "}\n",
       "\n",
       "#sk-container-id-2 div.sk-container {\n",
       "  /* jupyter's `normalize.less` sets `[hidden] { display: none; }`\n",
       "     but bootstrap.min.css set `[hidden] { display: none !important; }`\n",
       "     so we also need the `!important` here to be able to override the\n",
       "     default hidden behavior on the sphinx rendered scikit-learn.org.\n",
       "     See: https://github.com/scikit-learn/scikit-learn/issues/21755 */\n",
       "  display: inline-block !important;\n",
       "  position: relative;\n",
       "}\n",
       "\n",
       "#sk-container-id-2 div.sk-text-repr-fallback {\n",
       "  display: none;\n",
       "}\n",
       "\n",
       "div.sk-parallel-item,\n",
       "div.sk-serial,\n",
       "div.sk-item {\n",
       "  /* draw centered vertical line to link estimators */\n",
       "  background-image: linear-gradient(var(--sklearn-color-text-on-default-background), var(--sklearn-color-text-on-default-background));\n",
       "  background-size: 2px 100%;\n",
       "  background-repeat: no-repeat;\n",
       "  background-position: center center;\n",
       "}\n",
       "\n",
       "/* Parallel-specific style estimator block */\n",
       "\n",
       "#sk-container-id-2 div.sk-parallel-item::after {\n",
       "  content: \"\";\n",
       "  width: 100%;\n",
       "  border-bottom: 2px solid var(--sklearn-color-text-on-default-background);\n",
       "  flex-grow: 1;\n",
       "}\n",
       "\n",
       "#sk-container-id-2 div.sk-parallel {\n",
       "  display: flex;\n",
       "  align-items: stretch;\n",
       "  justify-content: center;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "  position: relative;\n",
       "}\n",
       "\n",
       "#sk-container-id-2 div.sk-parallel-item {\n",
       "  display: flex;\n",
       "  flex-direction: column;\n",
       "}\n",
       "\n",
       "#sk-container-id-2 div.sk-parallel-item:first-child::after {\n",
       "  align-self: flex-end;\n",
       "  width: 50%;\n",
       "}\n",
       "\n",
       "#sk-container-id-2 div.sk-parallel-item:last-child::after {\n",
       "  align-self: flex-start;\n",
       "  width: 50%;\n",
       "}\n",
       "\n",
       "#sk-container-id-2 div.sk-parallel-item:only-child::after {\n",
       "  width: 0;\n",
       "}\n",
       "\n",
       "/* Serial-specific style estimator block */\n",
       "\n",
       "#sk-container-id-2 div.sk-serial {\n",
       "  display: flex;\n",
       "  flex-direction: column;\n",
       "  align-items: center;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "  padding-right: 1em;\n",
       "  padding-left: 1em;\n",
       "}\n",
       "\n",
       "\n",
       "/* Toggleable style: style used for estimator/Pipeline/ColumnTransformer box that is\n",
       "clickable and can be expanded/collapsed.\n",
       "- Pipeline and ColumnTransformer use this feature and define the default style\n",
       "- Estimators will overwrite some part of the style using the `sk-estimator` class\n",
       "*/\n",
       "\n",
       "/* Pipeline and ColumnTransformer style (default) */\n",
       "\n",
       "#sk-container-id-2 div.sk-toggleable {\n",
       "  /* Default theme specific background. It is overwritten whether we have a\n",
       "  specific estimator or a Pipeline/ColumnTransformer */\n",
       "  background-color: var(--sklearn-color-background);\n",
       "}\n",
       "\n",
       "/* Toggleable label */\n",
       "#sk-container-id-2 label.sk-toggleable__label {\n",
       "  cursor: pointer;\n",
       "  display: flex;\n",
       "  width: 100%;\n",
       "  margin-bottom: 0;\n",
       "  padding: 0.5em;\n",
       "  box-sizing: border-box;\n",
       "  text-align: center;\n",
       "  align-items: start;\n",
       "  justify-content: space-between;\n",
       "  gap: 0.5em;\n",
       "}\n",
       "\n",
       "#sk-container-id-2 label.sk-toggleable__label .caption {\n",
       "  font-size: 0.6rem;\n",
       "  font-weight: lighter;\n",
       "  color: var(--sklearn-color-text-muted);\n",
       "}\n",
       "\n",
       "#sk-container-id-2 label.sk-toggleable__label-arrow:before {\n",
       "  /* Arrow on the left of the label */\n",
       "  content: \"▸\";\n",
       "  float: left;\n",
       "  margin-right: 0.25em;\n",
       "  color: var(--sklearn-color-icon);\n",
       "}\n",
       "\n",
       "#sk-container-id-2 label.sk-toggleable__label-arrow:hover:before {\n",
       "  color: var(--sklearn-color-text);\n",
       "}\n",
       "\n",
       "/* Toggleable content - dropdown */\n",
       "\n",
       "#sk-container-id-2 div.sk-toggleable__content {\n",
       "  max-height: 0;\n",
       "  max-width: 0;\n",
       "  overflow: hidden;\n",
       "  text-align: left;\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-2 div.sk-toggleable__content.fitted {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-2 div.sk-toggleable__content pre {\n",
       "  margin: 0.2em;\n",
       "  border-radius: 0.25em;\n",
       "  color: var(--sklearn-color-text);\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-2 div.sk-toggleable__content.fitted pre {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-2 input.sk-toggleable__control:checked~div.sk-toggleable__content {\n",
       "  /* Expand drop-down */\n",
       "  max-height: 200px;\n",
       "  max-width: 100%;\n",
       "  overflow: auto;\n",
       "}\n",
       "\n",
       "#sk-container-id-2 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {\n",
       "  content: \"▾\";\n",
       "}\n",
       "\n",
       "/* Pipeline/ColumnTransformer-specific style */\n",
       "\n",
       "#sk-container-id-2 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
       "  color: var(--sklearn-color-text);\n",
       "  background-color: var(--sklearn-color-unfitted-level-2);\n",
       "}\n",
       "\n",
       "#sk-container-id-2 div.sk-label.fitted input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
       "  background-color: var(--sklearn-color-fitted-level-2);\n",
       "}\n",
       "\n",
       "/* Estimator-specific style */\n",
       "\n",
       "/* Colorize estimator box */\n",
       "#sk-container-id-2 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-2);\n",
       "}\n",
       "\n",
       "#sk-container-id-2 div.sk-estimator.fitted input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-2);\n",
       "}\n",
       "\n",
       "#sk-container-id-2 div.sk-label label.sk-toggleable__label,\n",
       "#sk-container-id-2 div.sk-label label {\n",
       "  /* The background is the default theme color */\n",
       "  color: var(--sklearn-color-text-on-default-background);\n",
       "}\n",
       "\n",
       "/* On hover, darken the color of the background */\n",
       "#sk-container-id-2 div.sk-label:hover label.sk-toggleable__label {\n",
       "  color: var(--sklearn-color-text);\n",
       "  background-color: var(--sklearn-color-unfitted-level-2);\n",
       "}\n",
       "\n",
       "/* Label box, darken color on hover, fitted */\n",
       "#sk-container-id-2 div.sk-label.fitted:hover label.sk-toggleable__label.fitted {\n",
       "  color: var(--sklearn-color-text);\n",
       "  background-color: var(--sklearn-color-fitted-level-2);\n",
       "}\n",
       "\n",
       "/* Estimator label */\n",
       "\n",
       "#sk-container-id-2 div.sk-label label {\n",
       "  font-family: monospace;\n",
       "  font-weight: bold;\n",
       "  display: inline-block;\n",
       "  line-height: 1.2em;\n",
       "}\n",
       "\n",
       "#sk-container-id-2 div.sk-label-container {\n",
       "  text-align: center;\n",
       "}\n",
       "\n",
       "/* Estimator-specific */\n",
       "#sk-container-id-2 div.sk-estimator {\n",
       "  font-family: monospace;\n",
       "  border: 1px dotted var(--sklearn-color-border-box);\n",
       "  border-radius: 0.25em;\n",
       "  box-sizing: border-box;\n",
       "  margin-bottom: 0.5em;\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-2 div.sk-estimator.fitted {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-0);\n",
       "}\n",
       "\n",
       "/* on hover */\n",
       "#sk-container-id-2 div.sk-estimator:hover {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-2);\n",
       "}\n",
       "\n",
       "#sk-container-id-2 div.sk-estimator.fitted:hover {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-2);\n",
       "}\n",
       "\n",
       "/* Specification for estimator info (e.g. \"i\" and \"?\") */\n",
       "\n",
       "/* Common style for \"i\" and \"?\" */\n",
       "\n",
       ".sk-estimator-doc-link,\n",
       "a:link.sk-estimator-doc-link,\n",
       "a:visited.sk-estimator-doc-link {\n",
       "  float: right;\n",
       "  font-size: smaller;\n",
       "  line-height: 1em;\n",
       "  font-family: monospace;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "  border-radius: 1em;\n",
       "  height: 1em;\n",
       "  width: 1em;\n",
       "  text-decoration: none !important;\n",
       "  margin-left: 0.5em;\n",
       "  text-align: center;\n",
       "  /* unfitted */\n",
       "  border: var(--sklearn-color-unfitted-level-1) 1pt solid;\n",
       "  color: var(--sklearn-color-unfitted-level-1);\n",
       "}\n",
       "\n",
       ".sk-estimator-doc-link.fitted,\n",
       "a:link.sk-estimator-doc-link.fitted,\n",
       "a:visited.sk-estimator-doc-link.fitted {\n",
       "  /* fitted */\n",
       "  border: var(--sklearn-color-fitted-level-1) 1pt solid;\n",
       "  color: var(--sklearn-color-fitted-level-1);\n",
       "}\n",
       "\n",
       "/* On hover */\n",
       "div.sk-estimator:hover .sk-estimator-doc-link:hover,\n",
       ".sk-estimator-doc-link:hover,\n",
       "div.sk-label-container:hover .sk-estimator-doc-link:hover,\n",
       ".sk-estimator-doc-link:hover {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-3);\n",
       "  color: var(--sklearn-color-background);\n",
       "  text-decoration: none;\n",
       "}\n",
       "\n",
       "div.sk-estimator.fitted:hover .sk-estimator-doc-link.fitted:hover,\n",
       ".sk-estimator-doc-link.fitted:hover,\n",
       "div.sk-label-container:hover .sk-estimator-doc-link.fitted:hover,\n",
       ".sk-estimator-doc-link.fitted:hover {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-3);\n",
       "  color: var(--sklearn-color-background);\n",
       "  text-decoration: none;\n",
       "}\n",
       "\n",
       "/* Span, style for the box shown on hovering the info icon */\n",
       ".sk-estimator-doc-link span {\n",
       "  display: none;\n",
       "  z-index: 9999;\n",
       "  position: relative;\n",
       "  font-weight: normal;\n",
       "  right: .2ex;\n",
       "  padding: .5ex;\n",
       "  margin: .5ex;\n",
       "  width: min-content;\n",
       "  min-width: 20ex;\n",
       "  max-width: 50ex;\n",
       "  color: var(--sklearn-color-text);\n",
       "  box-shadow: 2pt 2pt 4pt #999;\n",
       "  /* unfitted */\n",
       "  background: var(--sklearn-color-unfitted-level-0);\n",
       "  border: .5pt solid var(--sklearn-color-unfitted-level-3);\n",
       "}\n",
       "\n",
       ".sk-estimator-doc-link.fitted span {\n",
       "  /* fitted */\n",
       "  background: var(--sklearn-color-fitted-level-0);\n",
       "  border: var(--sklearn-color-fitted-level-3);\n",
       "}\n",
       "\n",
       ".sk-estimator-doc-link:hover span {\n",
       "  display: block;\n",
       "}\n",
       "\n",
       "/* \"?\"-specific style due to the `<a>` HTML tag */\n",
       "\n",
       "#sk-container-id-2 a.estimator_doc_link {\n",
       "  float: right;\n",
       "  font-size: 1rem;\n",
       "  line-height: 1em;\n",
       "  font-family: monospace;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "  border-radius: 1rem;\n",
       "  height: 1rem;\n",
       "  width: 1rem;\n",
       "  text-decoration: none;\n",
       "  /* unfitted */\n",
       "  color: var(--sklearn-color-unfitted-level-1);\n",
       "  border: var(--sklearn-color-unfitted-level-1) 1pt solid;\n",
       "}\n",
       "\n",
       "#sk-container-id-2 a.estimator_doc_link.fitted {\n",
       "  /* fitted */\n",
       "  border: var(--sklearn-color-fitted-level-1) 1pt solid;\n",
       "  color: var(--sklearn-color-fitted-level-1);\n",
       "}\n",
       "\n",
       "/* On hover */\n",
       "#sk-container-id-2 a.estimator_doc_link:hover {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-3);\n",
       "  color: var(--sklearn-color-background);\n",
       "  text-decoration: none;\n",
       "}\n",
       "\n",
       "#sk-container-id-2 a.estimator_doc_link.fitted:hover {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-3);\n",
       "}\n",
       "</style><div id=\"sk-container-id-2\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>GridSearchCV(cv=5,\n",
       "             estimator=Pipeline(steps=[(&#x27;preprocessor&#x27;,\n",
       "                                        ColumnTransformer(transformers=[(&#x27;num&#x27;,\n",
       "                                                                         Pipeline(steps=[(&#x27;imputer&#x27;,\n",
       "                                                                                          SimpleImputer(strategy=&#x27;median&#x27;))]),\n",
       "                                                                         [&#x27;Age&#x27;,\n",
       "                                                                          &#x27;Weight&#x27;,\n",
       "                                                                          &#x27;GCS_first&#x27;,\n",
       "                                                                          &#x27;Glucose_first&#x27;,\n",
       "                                                                          &#x27;HR_first&#x27;,\n",
       "                                                                          &#x27;NIDiasABP_first&#x27;,\n",
       "                                                                          &#x27;NIMAP_first&#x27;,\n",
       "                                                                          &#x27;NISysABP_first&#x27;,\n",
       "                                                                          &#x27;Temp_first&#x27;,\n",
       "                                                                          &#x27;BUN_first&#x27;,\n",
       "                                                                          &#x27;Creatinine_first&#x27;,\n",
       "                                                                          &#x27;HCO3_first&#x27;,\n",
       "                                                                          &#x27;HCT_first&#x27;,\n",
       "                                                                          &#x27;K_first&#x27;,\n",
       "                                                                          &#x27;Mg_f...\n",
       "                                       (&#x27;classifier&#x27;,\n",
       "                                        RandomForestClassifier(class_weight=&#x27;balanced&#x27;,\n",
       "                                                               criterion=&#x27;entropy&#x27;,\n",
       "                                                               random_state=12345))]),\n",
       "             n_jobs=-1,\n",
       "             param_grid={&#x27;classifier__max_depth&#x27;: [5, 8],\n",
       "                         &#x27;classifier__max_features&#x27;: [2, 5],\n",
       "                         &#x27;classifier__min_samples_leaf&#x27;: [10, 15],\n",
       "                         &#x27;classifier__min_samples_split&#x27;: [5, 10],\n",
       "                         &#x27;classifier__n_estimators&#x27;: [100, 300, 500, 1000]},\n",
       "             return_train_score=True, scoring=&#x27;roc_auc&#x27;)</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item sk-dashed-wrapped\"><div class=\"sk-label-container\"><div class=\"sk-label fitted sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-10\" type=\"checkbox\" ><label for=\"sk-estimator-id-10\" class=\"sk-toggleable__label fitted sk-toggleable__label-arrow\"><div><div>GridSearchCV</div></div><div><a class=\"sk-estimator-doc-link fitted\" rel=\"noreferrer\" target=\"_blank\" href=\"https://scikit-learn.org/1.6/modules/generated/sklearn.model_selection.GridSearchCV.html\">?<span>Documentation for GridSearchCV</span></a><span class=\"sk-estimator-doc-link fitted\">i<span>Fitted</span></span></div></label><div class=\"sk-toggleable__content fitted\"><pre>GridSearchCV(cv=5,\n",
       "             estimator=Pipeline(steps=[(&#x27;preprocessor&#x27;,\n",
       "                                        ColumnTransformer(transformers=[(&#x27;num&#x27;,\n",
       "                                                                         Pipeline(steps=[(&#x27;imputer&#x27;,\n",
       "                                                                                          SimpleImputer(strategy=&#x27;median&#x27;))]),\n",
       "                                                                         [&#x27;Age&#x27;,\n",
       "                                                                          &#x27;Weight&#x27;,\n",
       "                                                                          &#x27;GCS_first&#x27;,\n",
       "                                                                          &#x27;Glucose_first&#x27;,\n",
       "                                                                          &#x27;HR_first&#x27;,\n",
       "                                                                          &#x27;NIDiasABP_first&#x27;,\n",
       "                                                                          &#x27;NIMAP_first&#x27;,\n",
       "                                                                          &#x27;NISysABP_first&#x27;,\n",
       "                                                                          &#x27;Temp_first&#x27;,\n",
       "                                                                          &#x27;BUN_first&#x27;,\n",
       "                                                                          &#x27;Creatinine_first&#x27;,\n",
       "                                                                          &#x27;HCO3_first&#x27;,\n",
       "                                                                          &#x27;HCT_first&#x27;,\n",
       "                                                                          &#x27;K_first&#x27;,\n",
       "                                                                          &#x27;Mg_f...\n",
       "                                       (&#x27;classifier&#x27;,\n",
       "                                        RandomForestClassifier(class_weight=&#x27;balanced&#x27;,\n",
       "                                                               criterion=&#x27;entropy&#x27;,\n",
       "                                                               random_state=12345))]),\n",
       "             n_jobs=-1,\n",
       "             param_grid={&#x27;classifier__max_depth&#x27;: [5, 8],\n",
       "                         &#x27;classifier__max_features&#x27;: [2, 5],\n",
       "                         &#x27;classifier__min_samples_leaf&#x27;: [10, 15],\n",
       "                         &#x27;classifier__min_samples_split&#x27;: [5, 10],\n",
       "                         &#x27;classifier__n_estimators&#x27;: [100, 300, 500, 1000]},\n",
       "             return_train_score=True, scoring=&#x27;roc_auc&#x27;)</pre></div> </div></div><div class=\"sk-parallel\"><div class=\"sk-parallel-item\"><div class=\"sk-item\"><div class=\"sk-label-container\"><div class=\"sk-label fitted sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-11\" type=\"checkbox\" ><label for=\"sk-estimator-id-11\" class=\"sk-toggleable__label fitted sk-toggleable__label-arrow\"><div><div>best_estimator_: Pipeline</div></div></label><div class=\"sk-toggleable__content fitted\"><pre>Pipeline(steps=[(&#x27;preprocessor&#x27;,\n",
       "                 ColumnTransformer(transformers=[(&#x27;num&#x27;,\n",
       "                                                  Pipeline(steps=[(&#x27;imputer&#x27;,\n",
       "                                                                   SimpleImputer(strategy=&#x27;median&#x27;))]),\n",
       "                                                  [&#x27;Age&#x27;, &#x27;Weight&#x27;, &#x27;GCS_first&#x27;,\n",
       "                                                   &#x27;Glucose_first&#x27;, &#x27;HR_first&#x27;,\n",
       "                                                   &#x27;NIDiasABP_first&#x27;,\n",
       "                                                   &#x27;NIMAP_first&#x27;,\n",
       "                                                   &#x27;NISysABP_first&#x27;,\n",
       "                                                   &#x27;Temp_first&#x27;, &#x27;BUN_first&#x27;,\n",
       "                                                   &#x27;Creatinine_first&#x27;,\n",
       "                                                   &#x27;HCO3_first&#x27;, &#x27;HCT_first&#x27;,\n",
       "                                                   &#x27;K_first&#x27;, &#x27;Mg_first&#x27;,\n",
       "                                                   &#x27;Na_first&#x27;, &#x27;PaCO2_first&#x27;,\n",
       "                                                   &#x27;PaO2_first&#x27;,\n",
       "                                                   &#x27;Platelets_first&#x27;,\n",
       "                                                   &#x27;WBC_first&#x27;, &#x27;pH_first&#x27;]),\n",
       "                                                 (&#x27;non_num&#x27;, &#x27;passthrough&#x27;,\n",
       "                                                  [&#x27;Gender&#x27;, &#x27;CCU&#x27;, &#x27;CSRU&#x27;,\n",
       "                                                   &#x27;SICU&#x27;])])),\n",
       "                (&#x27;classifier&#x27;,\n",
       "                 RandomForestClassifier(class_weight=&#x27;balanced&#x27;,\n",
       "                                        criterion=&#x27;entropy&#x27;, max_depth=8,\n",
       "                                        max_features=2, min_samples_leaf=15,\n",
       "                                        min_samples_split=5,\n",
       "                                        random_state=12345))])</pre></div> </div></div><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-serial\"><div class=\"sk-item sk-dashed-wrapped\"><div class=\"sk-label-container\"><div class=\"sk-label fitted sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-12\" type=\"checkbox\" ><label for=\"sk-estimator-id-12\" class=\"sk-toggleable__label fitted sk-toggleable__label-arrow\"><div><div>preprocessor: ColumnTransformer</div></div><div><a class=\"sk-estimator-doc-link fitted\" rel=\"noreferrer\" target=\"_blank\" href=\"https://scikit-learn.org/1.6/modules/generated/sklearn.compose.ColumnTransformer.html\">?<span>Documentation for preprocessor: ColumnTransformer</span></a></div></label><div class=\"sk-toggleable__content fitted\"><pre>ColumnTransformer(transformers=[(&#x27;num&#x27;,\n",
       "                                 Pipeline(steps=[(&#x27;imputer&#x27;,\n",
       "                                                  SimpleImputer(strategy=&#x27;median&#x27;))]),\n",
       "                                 [&#x27;Age&#x27;, &#x27;Weight&#x27;, &#x27;GCS_first&#x27;, &#x27;Glucose_first&#x27;,\n",
       "                                  &#x27;HR_first&#x27;, &#x27;NIDiasABP_first&#x27;, &#x27;NIMAP_first&#x27;,\n",
       "                                  &#x27;NISysABP_first&#x27;, &#x27;Temp_first&#x27;, &#x27;BUN_first&#x27;,\n",
       "                                  &#x27;Creatinine_first&#x27;, &#x27;HCO3_first&#x27;, &#x27;HCT_first&#x27;,\n",
       "                                  &#x27;K_first&#x27;, &#x27;Mg_first&#x27;, &#x27;Na_first&#x27;,\n",
       "                                  &#x27;PaCO2_first&#x27;, &#x27;PaO2_first&#x27;,\n",
       "                                  &#x27;Platelets_first&#x27;, &#x27;WBC_first&#x27;, &#x27;pH_first&#x27;]),\n",
       "                                (&#x27;non_num&#x27;, &#x27;passthrough&#x27;,\n",
       "                                 [&#x27;Gender&#x27;, &#x27;CCU&#x27;, &#x27;CSRU&#x27;, &#x27;SICU&#x27;])])</pre></div> </div></div><div class=\"sk-parallel\"><div class=\"sk-parallel-item\"><div class=\"sk-item\"><div class=\"sk-label-container\"><div class=\"sk-label fitted sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-13\" type=\"checkbox\" ><label for=\"sk-estimator-id-13\" class=\"sk-toggleable__label fitted sk-toggleable__label-arrow\"><div><div>num</div></div></label><div class=\"sk-toggleable__content fitted\"><pre>[&#x27;Age&#x27;, &#x27;Weight&#x27;, &#x27;GCS_first&#x27;, &#x27;Glucose_first&#x27;, &#x27;HR_first&#x27;, &#x27;NIDiasABP_first&#x27;, &#x27;NIMAP_first&#x27;, &#x27;NISysABP_first&#x27;, &#x27;Temp_first&#x27;, &#x27;BUN_first&#x27;, &#x27;Creatinine_first&#x27;, &#x27;HCO3_first&#x27;, &#x27;HCT_first&#x27;, &#x27;K_first&#x27;, &#x27;Mg_first&#x27;, &#x27;Na_first&#x27;, &#x27;PaCO2_first&#x27;, &#x27;PaO2_first&#x27;, &#x27;Platelets_first&#x27;, &#x27;WBC_first&#x27;, &#x27;pH_first&#x27;]</pre></div> </div></div><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-estimator fitted sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-14\" type=\"checkbox\" ><label for=\"sk-estimator-id-14\" class=\"sk-toggleable__label fitted sk-toggleable__label-arrow\"><div><div>SimpleImputer</div></div><div><a class=\"sk-estimator-doc-link fitted\" rel=\"noreferrer\" target=\"_blank\" href=\"https://scikit-learn.org/1.6/modules/generated/sklearn.impute.SimpleImputer.html\">?<span>Documentation for SimpleImputer</span></a></div></label><div class=\"sk-toggleable__content fitted\"><pre>SimpleImputer(strategy=&#x27;median&#x27;)</pre></div> </div></div></div></div></div></div></div><div class=\"sk-parallel-item\"><div class=\"sk-item\"><div class=\"sk-label-container\"><div class=\"sk-label fitted sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-15\" type=\"checkbox\" ><label for=\"sk-estimator-id-15\" class=\"sk-toggleable__label fitted sk-toggleable__label-arrow\"><div><div>non_num</div></div></label><div class=\"sk-toggleable__content fitted\"><pre>[&#x27;Gender&#x27;, &#x27;CCU&#x27;, &#x27;CSRU&#x27;, &#x27;SICU&#x27;]</pre></div> </div></div><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-estimator fitted sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-16\" type=\"checkbox\" ><label for=\"sk-estimator-id-16\" class=\"sk-toggleable__label fitted sk-toggleable__label-arrow\"><div><div>passthrough</div></div></label><div class=\"sk-toggleable__content fitted\"><pre>passthrough</pre></div> </div></div></div></div></div></div></div><div class=\"sk-item\"><div class=\"sk-estimator fitted sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-17\" type=\"checkbox\" ><label for=\"sk-estimator-id-17\" class=\"sk-toggleable__label fitted sk-toggleable__label-arrow\"><div><div>RandomForestClassifier</div></div><div><a class=\"sk-estimator-doc-link fitted\" rel=\"noreferrer\" target=\"_blank\" href=\"https://scikit-learn.org/1.6/modules/generated/sklearn.ensemble.RandomForestClassifier.html\">?<span>Documentation for RandomForestClassifier</span></a></div></label><div class=\"sk-toggleable__content fitted\"><pre>RandomForestClassifier(class_weight=&#x27;balanced&#x27;, criterion=&#x27;entropy&#x27;,\n",
       "                       max_depth=8, max_features=2, min_samples_leaf=15,\n",
       "                       min_samples_split=5, random_state=12345)</pre></div> </div></div></div></div></div></div></div></div></div></div></div>"
      ],
      "text/plain": [
       "GridSearchCV(cv=5,\n",
       "             estimator=Pipeline(steps=[('preprocessor',\n",
       "                                        ColumnTransformer(transformers=[('num',\n",
       "                                                                         Pipeline(steps=[('imputer',\n",
       "                                                                                          SimpleImputer(strategy='median'))]),\n",
       "                                                                         ['Age',\n",
       "                                                                          'Weight',\n",
       "                                                                          'GCS_first',\n",
       "                                                                          'Glucose_first',\n",
       "                                                                          'HR_first',\n",
       "                                                                          'NIDiasABP_first',\n",
       "                                                                          'NIMAP_first',\n",
       "                                                                          'NISysABP_first',\n",
       "                                                                          'Temp_first',\n",
       "                                                                          'BUN_first',\n",
       "                                                                          'Creatinine_first',\n",
       "                                                                          'HCO3_first',\n",
       "                                                                          'HCT_first',\n",
       "                                                                          'K_first',\n",
       "                                                                          'Mg_f...\n",
       "                                       ('classifier',\n",
       "                                        RandomForestClassifier(class_weight='balanced',\n",
       "                                                               criterion='entropy',\n",
       "                                                               random_state=12345))]),\n",
       "             n_jobs=-1,\n",
       "             param_grid={'classifier__max_depth': [5, 8],\n",
       "                         'classifier__max_features': [2, 5],\n",
       "                         'classifier__min_samples_leaf': [10, 15],\n",
       "                         'classifier__min_samples_split': [5, 10],\n",
       "                         'classifier__n_estimators': [100, 300, 500, 1000]},\n",
       "             return_train_score=True, scoring='roc_auc')"
      ]
     },
     "execution_count": 104,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Fit the random search object to the data\n",
    "rf_grid_search.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "703b6f48-cb05-4045-b33c-b8a407cedf59",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "#### **Your turn! Evaluate the trained model!**\n",
    "\n",
    "Use this notebook and the previous logistic regression code to determine how to evaluate the trained model. \n",
    "\n",
    "Consider changing the training process:\n",
    "\n",
    "- What happens if you change the parameters in `rf_param_grid`? There are other parameters (as described in the [documentation](https://scikit-learn.org/1.5/modules/generated/sklearn.ensemble.RandomForestClassifier.html) you could add. Does this change training performance?\n",
    "- What happens if we change the number of folds (`cv=k`)? Does the model improve or get worse?\n",
    "- Try a different scoring metric (`scoring='x'`; see [list of possible metrics'](https://scikit-learn.org/dev/modules/model_evaluation.html#scoring-parameter))"
   ]
  }
 ],
 "metadata": {
  "celltoolbar": "Raw Cell Format",
  "kernelspec": {
   "display_name": "medAI_edu (Python 3.13.1)",
   "language": "python",
   "name": "medai_edu"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.13.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}