DT_Diabetes.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": [],
      "authorship_tag": "ABX9TyNs7o4v9xiO12MQ3l1LbgeY",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/frm1789/100DaysOfPython/blob/main/DT_Diabetes.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# CART vs Random Forest vs Assemble models\n",
        "\n",
        "This dataset is originally from the National Institute of Diabetes and Digestive and Kidney Diseases. The objective of the dataset is to diagnostically predict whether or not a patient has diabetes, based on certain diagnostic measurements included in the dataset."
      ],
      "metadata": {
        "id": "_ue0PjOU3Uzn"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## CART\n",
        "\n",
        "How CART (Classification and Regression Trees) works?\n",
        "\n",
        "1. Feature Selection: Finds the best feature to split the dataset.\n",
        "2. Dataset Splitting: Divides the dataset into subsets based on features.\n",
        "3. Recursion: Continues splitting subsets until stopping criteria are met.\n",
        "4. Tree Construction: Constructs a decision tree with nodes and leaves.\n",
        "5. Tree Pruning (optional): Removes subtrees to prevent overfitting.\n",
        "6. Prediction: Uses the tree to predict outcomes for new samples.\n"
      ],
      "metadata": {
        "id": "io97_qHI4Sfx"
      }
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "metadata": {
        "id": "BQsI_67rJu6h"
      },
      "outputs": [],
      "source": [
        "import numpy as np\n",
        "import pandas as pd\n",
        "from sklearn import datasets\n",
        "from sklearn.model_selection import train_test_split\n",
        "from sklearn.tree import DecisionTreeClassifier\n",
        "from sklearn.metrics import mean_squared_error\n",
        "import matplotlib.pyplot as plt\n",
        "from sklearn.metrics import accuracy_score\n",
        "from sklearn.datasets import load_diabetes\n",
        "from sklearn.preprocessing import StandardScaler, MinMaxScaler, RobustScaler\n",
        "from sklearn.impute import KNNImputer\n",
        "from sklearn.preprocessing import MinMaxScaler\n",
        "from sklearn.metrics import confusion_matrix, recall_score\n",
        "from sklearn.impute import KNNImputer\n",
        "from sklearn.preprocessing import MinMaxScaler\n",
        "\n",
        "df= pd.read_csv('diabetes.csv')\n"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "from sklearn.ensemble import RandomForestClassifier\n",
        "from sklearn.model_selection import GridSearchCV, cross_validate, validation_curve\n",
        "from sklearn.preprocessing import MinMaxScaler\n",
        "from sklearn.impute import KNNImputer\n",
        "\n",
        "pd.set_option('display.max_columns', None)\n",
        "pd.set_option('display.width', 500)"
      ],
      "metadata": {
        "id": "Bz0NXfqwrDY2"
      },
      "execution_count": 2,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "def check_data_quality(data):\n",
        "    # Check for NaN values\n",
        "    nan_count = np.sum(np.isnan(data))\n",
        "\n",
        "    # Check for outliers (assuming outliers are values more than 3 standard deviations away from the mean)\n",
        "    mean = np.mean(data)\n",
        "    std_dev = np.std(data)\n",
        "    outliers_count = np.sum(np.abs(data - mean) > 3 * std_dev)\n",
        "\n",
        "\n",
        "    return nan_count, outliers_count\n"
      ],
      "metadata": {
        "id": "1A1TwCJZbnbP"
      },
      "execution_count": 3,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "def preprocess_data(df):\n",
        "     # Replace zeros with NaN\n",
        "    for col in df:\n",
        "        df[col] = df[col].replace(0, np.nan)\n",
        "\n",
        "    # Impute NaN values using KNN Imputer\n",
        "    imputer = KNNImputer(n_neighbors=5)\n",
        "    imputed_df = pd.DataFrame(imputer.fit_transform(df), columns=df.columns)\n",
        "\n",
        "\n",
        "    return imputed_df\n",
        "\n"
      ],
      "metadata": {
        "id": "X5Z5SjJOeJxV"
      },
      "execution_count": 4,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Out of the Box: DT, DT with max_depth and RF"
      ],
      "metadata": {
        "id": "Ej8_9RYn3s_R"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "y =  df['Outcome']\n",
        "X =  df.drop('Outcome', axis=1)"
      ],
      "metadata": {
        "id": "gt0rqS2FjO0L"
      },
      "execution_count": 5,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "df['Outcome'].value_counts()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "na-KBOKGBhm2",
        "outputId": "6a0e0d51-64f5-422d-97b2-b88578a6bee4"
      },
      "execution_count": 6,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0    500\n",
              "1    268\n",
              "Name: Outcome, dtype: int64"
            ]
          },
          "metadata": {},
          "execution_count": 6
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)\n",
        "clf = DecisionTreeClassifier(random_state=42)\n",
        "clf.fit(X_train, y_train)\n",
        "y_pred = clf.predict(X_test)\n",
        "from sklearn.metrics import accuracy_score\n",
        "accuracy = accuracy_score(y_test, y_pred)\n",
        "print(\"Accuracy:\", accuracy)\n",
        "# Out of the box: Accuracy: 0.7467532467532467"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "v8Et3q61boqT",
        "outputId": "42dd6b66-6db2-439f-ceee-0c20467c02f3"
      },
      "execution_count": 7,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Accuracy: 0.7467532467532467\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "confusion_matrix(y_test, y_pred)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "mfk149vWClRM",
        "outputId": "4fa12a92-cb48-4be0-ece0-0fc794ab47ef"
      },
      "execution_count": 8,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[75, 24],\n",
              "       [15, 40]])"
            ]
          },
          "metadata": {},
          "execution_count": 8
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "clf = DecisionTreeClassifier(random_state=42, max_depth=10)\n",
        "clf.fit(X_train, y_train)\n",
        "y_pred = clf.predict(X_test)\n",
        "accuracy = accuracy_score(y_test, y_pred)\n",
        "print(\"Accuracy:\", accuracy)\n",
        "# Max depth: 10\n",
        "# Accuracy: 0.7532467532467533"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "kZBNPROXZVCM",
        "outputId": "84a7c4ea-5045-47b8-e864-f0a59a8b5972"
      },
      "execution_count": 9,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Accuracy: 0.7532467532467533\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "confusion_matrix(y_test, y_pred)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "dLt3Wl99Cu1r",
        "outputId": "1ae9d74e-eee0-43b9-89cf-aa28a2d50631"
      },
      "execution_count": 10,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[77, 22],\n",
              "       [16, 39]])"
            ]
          },
          "metadata": {},
          "execution_count": 10
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "rf_model = RandomForestClassifier(random_state=17)\n",
        "rf_model.get_params()\n",
        "'''\n",
        "{'bootstrap': True,\n",
        " 'ccp_alpha': 0.0,\n",
        " 'class_weight': None,\n",
        " 'criterion': 'gini',\n",
        " 'max_depth': None,\n",
        " 'max_features': 'sqrt',\n",
        " 'max_leaf_nodes': None,\n",
        " 'max_samples': None,\n",
        " 'min_impurity_decrease': 0.0,\n",
        " 'min_samples_leaf': 1,\n",
        " 'min_samples_split': 2,\n",
        " 'min_weight_fraction_leaf': 0.0,\n",
        " 'n_estimators': 100,\n",
        " 'n_jobs': None,\n",
        " 'oob_score': False,\n",
        " 'random_state': 17,\n",
        " 'verbose': 0,\n",
        " 'warm_start': False}\n",
        "'''\n",
        "\n",
        "#Accuracy before GridSearchCV\n",
        "cv_results = cross_validate(rf_model, X, y, cv=5, scoring=[\"accuracy\", \"precision\", \"recall\", \"f1\", \"roc_auc\"])\n",
        "cv_results\n",
        "\n",
        "#\n",
        "\n",
        "#{'fit_time': array([0.40921831, 0.39054608, 0.42077708, 0.54269505, 0.37673283]),\n",
        "# 'score_time': array([0.10616827, 0.05290127, 0.10224581, 0.04395771, 0.05506802]),\n",
        "# 'test_accuracy': array([0.74675325, 0.72077922, 0.78571429, 0.81699346, 0.75816993]),\n",
        "# 'test_precision': array([0.65306122, 0.6       , 0.78378378, 0.80487805, 0.69047619]),\n",
        "# 'test_recall': array([0.59259259, 0.61111111, 0.53703704, 0.62264151, 0.54716981]),\n",
        "# 'test_f1': array([0.62135922, 0.60550459, 0.63736264, 0.70212766, 0.61052632]),\n",
        "# 'test_roc_auc': array([0.80907407, 0.78509259, 0.83601852, 0.87971698, 0.83141509])}\n",
        "\n",
        "\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "R-vP5mcuq7XR",
        "outputId": "ca1460a7-17e3-4724-e72c-1f842fafe1e2"
      },
      "execution_count": 11,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{'fit_time': array([0.92464256, 0.40524197, 0.36967802, 0.49381351, 0.56689739]),\n",
              " 'score_time': array([0.11081028, 0.03660917, 0.06742048, 0.04082465, 0.10150886]),\n",
              " 'test_accuracy': array([0.74675325, 0.72077922, 0.78571429, 0.81699346, 0.75816993]),\n",
              " 'test_precision': array([0.65306122, 0.6       , 0.78378378, 0.80487805, 0.69047619]),\n",
              " 'test_recall': array([0.59259259, 0.61111111, 0.53703704, 0.62264151, 0.54716981]),\n",
              " 'test_f1': array([0.62135922, 0.60550459, 0.63736264, 0.70212766, 0.61052632]),\n",
              " 'test_roc_auc': array([0.80907407, 0.78509259, 0.83601852, 0.87971698, 0.83141509])}"
            ]
          },
          "metadata": {},
          "execution_count": 11
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Pre-Processing"
      ],
      "metadata": {
        "id": "3qd3QArc8lZA"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "zero_counts = (df.iloc[:, :-1] == 0).sum()\n",
        "print(zero_counts)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "N8Jpi-4x8nd3",
        "outputId": "795f1016-6fe5-413b-ce5d-2c45cf7a0ce3"
      },
      "execution_count": 12,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Pregnancies                 111\n",
            "Glucose                       5\n",
            "BloodPressure                35\n",
            "SkinThickness               227\n",
            "Insulin                     374\n",
            "BMI                          11\n",
            "DiabetesPedigreeFunction      0\n",
            "Age                           0\n",
            "dtype: int64\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Replacing NaN with mean values\n",
        "df[\"Glucose\"].replace(0,df[\"Glucose\"].mean(), inplace = True)\n",
        "df[\"BloodPressure\"].replace(0,df[\"BloodPressure\"].mean(), inplace = True)\n",
        "df[\"SkinThickness\"].replace(0,df[\"SkinThickness\"].mean(), inplace = True)\n",
        "df[\"Insulin\"].replace(0,df[\"Insulin\"].mean(), inplace = True)\n",
        "df[\"BMI\"].replace(0,df[\"BMI\"].mean(), inplace = True)"
      ],
      "metadata": {
        "id": "57VUnnSy8q-N"
      },
      "execution_count": 13,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "y =  df['Outcome']\n",
        "X =  df.drop('Outcome', axis=1)"
      ],
      "metadata": {
        "id": "peADGKlsDmXT"
      },
      "execution_count": 14,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Normalization\n",
        "scaler = StandardScaler()\n",
        "scaled_X = scaler.fit_transform(X)"
      ],
      "metadata": {
        "id": "1bmG2A9s8z25"
      },
      "execution_count": 15,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Again the models"
      ],
      "metadata": {
        "id": "ofoL4UxC883K"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "X_train, X_test, y_train, y_test = train_test_split(scaled_X, y, test_size=0.2, random_state=42)\n",
        "clf = DecisionTreeClassifier(random_state=42)\n",
        "clf.fit(X_train, y_train)\n",
        "y_pred = clf.predict(X_test)\n",
        "from sklearn.metrics import accuracy_score\n",
        "accuracy = accuracy_score(y_test, y_pred)\n",
        "print(\"Accuracy:\", accuracy)\n",
        "# Out of the box: Accuracy: 0.7467532467532467\n",
        "# Pre processing: 0.7207792207792207"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "8c194f14-17b6-45c8-9d43-98e1ca39cf82",
        "id": "1NTxp8SZ9L63"
      },
      "execution_count": 16,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Accuracy: 0.7207792207792207\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "confusion_matrix(y_test, y_pred)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "KS6Q8jl6C4SL",
        "outputId": "ded5039b-e33a-416d-aa98-5b6ee28d24d7"
      },
      "execution_count": 17,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[73, 26],\n",
              "       [17, 38]])"
            ]
          },
          "metadata": {},
          "execution_count": 17
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "clf = DecisionTreeClassifier(random_state=42, max_depth=10)\n",
        "clf.fit(X_train, y_train)\n",
        "y_pred = clf.predict(X_test)\n",
        "accuracy = accuracy_score(y_test, y_pred)\n",
        "print(\"Accuracy:\", accuracy)\n",
        "# Max depth: 10\n",
        "# Accuracy: 0.7532467532467533\n",
        "# Accuracy: 0.7272727272727273\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "84316c04-629e-404d-da16-a4b9e066a273",
        "id": "20UD2zek9L64"
      },
      "execution_count": 18,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Accuracy: 0.7272727272727273\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "rf_model = RandomForestClassifier(random_state=17)\n",
        "rf_model.get_params()\n",
        "'''\n",
        "{'bootstrap': True,\n",
        " 'ccp_alpha': 0.0,\n",
        " 'class_weight': None,\n",
        " 'criterion': 'gini',\n",
        " 'max_depth': None,\n",
        " 'max_features': 'sqrt',\n",
        " 'max_leaf_nodes': None,\n",
        " 'max_samples': None,\n",
        " 'min_impurity_decrease': 0.0,\n",
        " 'min_samples_leaf': 1,\n",
        " 'min_samples_split': 2,\n",
        " 'min_weight_fraction_leaf': 0.0,\n",
        " 'n_estimators': 100,\n",
        " 'n_jobs': None,\n",
        " 'oob_score': False,\n",
        " 'random_state': 17,\n",
        " 'verbose': 0,\n",
        " 'warm_start': False}\n",
        "'''\n",
        "\n",
        "#Accuracy before GridSearchCV\n",
        "cv_results = cross_validate(rf_model, X, y, cv=5, scoring=[\"accuracy\", \"precision\", \"recall\", \"f1\", \"roc_auc\"])\n",
        "cv_results\n",
        "\n",
        "#\n",
        "\n",
        "#{'fit_time': array([0.40921831, 0.39054608, 0.42077708, 0.54269505, 0.37673283]),\n",
        "# 'score_time': array([0.10616827, 0.05290127, 0.10224581, 0.04395771, 0.05506802]),\n",
        "# 'test_accuracy': array([0.74675325, 0.72077922, 0.78571429, 0.81699346, 0.75816993]),\n",
        "# 'test_precision': array([0.65306122, 0.6       , 0.78378378, 0.80487805, 0.69047619]),\n",
        "# 'test_recall': array([0.59259259, 0.61111111, 0.53703704, 0.62264151, 0.54716981]),\n",
        "# 'test_f1': array([0.62135922, 0.60550459, 0.63736264, 0.70212766, 0.61052632]),\n",
        "# 'test_roc_auc': array([0.80907407, 0.78509259, 0.83601852, 0.87971698, 0.83141509])}\n",
        "\n",
        "#'test_accuracy': array([0.75974026, 0.74025974, 0.75974026, 0.82352941, 0.73202614]),\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "c36eaf68-15e9-41c2-9b95-176e65687ba1",
        "id": "Xtsjn75h9L65"
      },
      "execution_count": 19,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{'fit_time': array([0.24629617, 0.22035623, 0.21037507, 0.20779586, 0.20407557]),\n",
              " 'score_time': array([0.02680731, 0.03101563, 0.02951336, 0.02539897, 0.02453732]),\n",
              " 'test_accuracy': array([0.75974026, 0.74025974, 0.75974026, 0.82352941, 0.73202614]),\n",
              " 'test_precision': array([0.66666667, 0.64      , 0.72972973, 0.80952381, 0.63636364]),\n",
              " 'test_recall': array([0.62962963, 0.59259259, 0.5       , 0.64150943, 0.52830189]),\n",
              " 'test_f1': array([0.64761905, 0.61538462, 0.59340659, 0.71578947, 0.57731959]),\n",
              " 'test_roc_auc': array([0.81555556, 0.77768519, 0.83981481, 0.89641509, 0.80924528])}"
            ]
          },
          "metadata": {},
          "execution_count": 19
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## SMOTE"
      ],
      "metadata": {
        "id": "So4GUr-HAeEW"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "df= pd.read_csv('diabetes.csv')"
      ],
      "metadata": {
        "id": "B_PTKqL8DH0d"
      },
      "execution_count": 28,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Replacing NaN with mean values\n",
        "df[\"Glucose\"].replace(0,df[\"Glucose\"].mean(), inplace = True)\n",
        "df[\"BloodPressure\"].replace(0,df[\"BloodPressure\"].mean(), inplace = True)\n",
        "df[\"SkinThickness\"].replace(0,df[\"SkinThickness\"].mean(), inplace = True)\n",
        "df[\"Insulin\"].replace(0,df[\"Insulin\"].mean(), inplace = True)\n",
        "df[\"BMI\"].replace(0,df[\"BMI\"].mean(), inplace = True)"
      ],
      "metadata": {
        "id": "R9W8M8psDca6"
      },
      "execution_count": 29,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "y =  df['Outcome']\n",
        "X =  df.drop('Outcome', axis=1)"
      ],
      "metadata": {
        "id": "ayMbK2veDRCH"
      },
      "execution_count": 30,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "from imblearn.over_sampling import SMOTE\n",
        "sm = SMOTE(random_state=42, k_neighbors=5)\n",
        "X_res, y_res = sm.fit_resample(scaled_X, y)"
      ],
      "metadata": {
        "id": "3Lth1X9rAdp7"
      },
      "execution_count": 31,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "X_res"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "B3YruH6HEph6",
        "outputId": "385291fd-004c-4125-c95d-ce8bbc9af1af"
      },
      "execution_count": 32,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[ 0.63994726,  0.86527574, -0.0210444 , ...,  0.16725546,\n",
              "         0.46849198,  1.4259954 ],\n",
              "       [-0.84488505, -1.20598931, -0.51658286, ..., -0.85153454,\n",
              "        -0.36506078, -0.19067191],\n",
              "       [ 1.23388019,  2.01597855, -0.68176235, ..., -1.33182125,\n",
              "         0.60439732, -0.10558415],\n",
              "       ...,\n",
              "       [-0.12460603,  1.86277092,  0.95952058, ...,  0.44248846,\n",
              "         1.9829989 ,  1.61427145],\n",
              "       [-0.26693054,  0.19815313,  1.57742511, ...,  0.55362796,\n",
              "         1.45659024, -0.11931871],\n",
              "       [ 1.10398471,  1.01939681,  0.36611784, ...,  0.03080233,\n",
              "         0.03432433,  1.09360178]])"
            ]
          },
          "metadata": {},
          "execution_count": 32
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "y_res.value_counts()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "7TLiRS-SAxcG",
        "outputId": "d8b2c1e3-d477-4b23-d675-c58b0efc7271"
      },
      "execution_count": 33,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "1    500\n",
              "0    500\n",
              "Name: Outcome, dtype: int64"
            ]
          },
          "metadata": {},
          "execution_count": 33
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Normalization\n",
        "scaler = StandardScaler()\n",
        "scaled_X = scaler.fit_transform(X)"
      ],
      "metadata": {
        "id": "TE86fggSDca9"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Again the models"
      ],
      "metadata": {
        "id": "ZV0aeYYwAxkL"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "X_train, X_test, y_train, y_test = train_test_split(X_res, y_res, test_size=0.2, random_state=42)\n",
        "clf = DecisionTreeClassifier(random_state=42)\n",
        "clf.fit(X_train, y_train)\n",
        "y_pred = clf.predict(X_test)\n",
        "from sklearn.metrics import accuracy_score\n",
        "accuracy = accuracy_score(y_test, y_pred)\n",
        "print(\"Accuracy:\", accuracy)\n",
        "# Out of the box: Accuracy: 0.7467532467532467"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "aa41ce57-6d24-43fa-9a9b-4f660807ec9a",
        "id": "vbP-OEquAxkL"
      },
      "execution_count": 34,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Accuracy: 0.785\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "clf = DecisionTreeClassifier(random_state=42, max_depth=8)\n",
        "clf.fit(X_train, y_train)\n",
        "y_pred = clf.predict(X_test)\n",
        "accuracy = accuracy_score(y_test, y_pred)\n",
        "print(\"Accuracy:\", accuracy)\n",
        "# Max depth: 10\n",
        "# Accuracy: 0.7532467532467533"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "968ebfbf-605d-4119-c989-8e2704b86018",
        "id": "3pCMI6PrAxkL"
      },
      "execution_count": 58,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Accuracy: 0.81\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "clf_model = DecisionTreeClassifier(random_state=42, max_depth=8)\n",
        "clf_model.get_params()\n",
        "\n",
        "#Accuracy before GridSearchCV\n",
        "cv_results = cross_validate(clf_model, X, y, cv=5, scoring=[\"accuracy\", \"precision\", \"recall\", \"f1\", \"roc_auc\"])\n",
        "cv_results\n",
        "\n",
        "# Calcular la precisión promedio\n",
        "accuracy_mean = np.mean(cv_results['test_accuracy'])\n",
        "print(\"Precisión promedio:\", accuracy_mean)\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "peipIzRWLvjE",
        "outputId": "39193dd0-938e-4f8d-ecff-9ba8006ce675"
      },
      "execution_count": 57,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Precisión promedio: 0.717519735166794\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "confusion_matrix(y_test, y_pred)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "2t33uWdfCTPN",
        "outputId": "a9a95325-a1b6-4ace-87bf-1ee3b64a173d"
      },
      "execution_count": 36,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[76, 23],\n",
              "       [28, 73]])"
            ]
          },
          "metadata": {},
          "execution_count": 36
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "rf_model = RandomForestClassifier(random_state=17)\n",
        "rf_model.get_params()\n",
        "\n",
        "#Accuracy before GridSearchCV\n",
        "cv_results = cross_validate(rf_model, X, y, cv=5, scoring=[\"accuracy\", \"precision\", \"recall\", \"f1\", \"roc_auc\"])\n",
        "cv_results\n",
        "\n",
        "# Calcular la precisión promedio\n",
        "accuracy_mean = np.mean(cv_results['test_accuracy'])\n",
        "print(\"Precisión promedio:\", accuracy_mean)\n",
        "\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "57dd4beb-f119-40ef-a1a5-943ac1e7588b",
        "id": "pd9mtLF3AxkM"
      },
      "execution_count": 54,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Precisión promedio: 0.7630591630591631\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "confusion_matrix(y_test, y_pred)"
      ],
      "metadata": {
        "id": "sZf8s-7DHAqh"
      },
      "execution_count": null,
      "outputs": []
    }
  ]
}