3_classification_and_results.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "3-classification-and-results.ipynb",
      "provenance": [],
      "collapsed_sections": [
        "ptNp4gASm2H5",
        "hk2oSP7V3mJ2",
        "bcTvfZtCWLOE",
        "1iFzjTax3q7v",
        "aBLWa3wX-W1g",
        "IrqfjdME18qj",
        "M1Fqi2bHnKam",
        "nB28ZakajI_o",
        "Zo1STHWKYH50",
        "WwqOdP7uqPpx",
        "ESh6-eXGqPp2",
        "bwK12ayosDm9",
        "hQ3Ev0J1sDm-",
        "spj4fTY0vdsh",
        "D0t7vWeqmfbW",
        "v7yarOzvkUMj",
        "CN-094zg4XxO",
        "16ezt6XAagLS",
        "TSyNbn51xeD2",
        "41vig1y9_Jdh",
        "HStijZ_XyLrH",
        "82BbOk_PyrUQ",
        "IshY0sEOID_Q",
        "QoFQd_1COUES",
        "ufTnoiNQOUuD",
        "DbaJZjxAmLo7",
        "wJUQN7M8vjI5",
        "i3csfAX3v1UU",
        "tOOqHUCTv991",
        "3kQSykLHy8pN",
        "vGJcfRPY5Fku",
        "tftwwd_su2G-",
        "-6yNFchiwGGQ",
        "Cb8btmSrwGrW",
        "zBayy8bzwHVR",
        "__Jbzt-4wflf",
        "PVTcypAIwfli",
        "h9IY8r0Xwflm",
        "ej_5dyl0wfln",
        "-20A8d866qsZ",
        "95WkQ6Uk6qsa",
        "u71hw-986qsc",
        "vxGrsO3fr4dW",
        "IgN64NHjvV4Q",
        "leGMP8mEvXoj",
        "DnwCo_KxvZCp",
        "jpjhSreZBQNG",
        "2BUe0v-lq8gi",
        "49nN4PyrrEBa",
        "DP9-mTOPrOag",
        "VUmSkVRGrUCW",
        "iFVGucUcsCzf",
        "HnzNg9wGsCzf",
        "OK84vZfdsCzf",
        "53RZH1TksCzg",
        "Ys7ylgVKuxVY",
        "9O7cqiwFu7IN",
        "Hb7KKixDvAST",
        "11LLB7WcvQID",
        "f97rbg-Cw-is",
        "ZnEJLQZhw-it",
        "0Ao-q9zAw-iw",
        "YBCfR04Ow-ix",
        "XrqMAs3Es6Mq",
        "xYljcuwrs6Mq",
        "iOtkFd6fs6Mq",
        "TxmjRKg9s6Mr",
        "2ZLIvAwHzGoc",
        "WCHVwyX1zI7Q",
        "VBHokkLszLDY",
        "hNdSCnNiNzvQ"
      ],
      "toc_visible": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "widgets": {
      "application/vnd.jupyter.widget-state+json": {
        "62ecd42cbce342b390386aa76fb12942": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "state": {
            "_view_name": "HBoxView",
            "_dom_classes": [],
            "_model_name": "HBoxModel",
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "box_style": "",
            "layout": "IPY_MODEL_56d0259d0a784d8682eee74a94bf27b1",
            "_model_module": "@jupyter-widgets/controls",
            "children": [
              "IPY_MODEL_ef73856034a247ebb26a89a4be5592d5",
              "IPY_MODEL_ca7eb627a9ca4c19bdb800a28693ec12"
            ]
          }
        },
        "56d0259d0a784d8682eee74a94bf27b1": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "ef73856034a247ebb26a89a4be5592d5": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "state": {
            "_view_name": "ProgressView",
            "style": "IPY_MODEL_85194ddcf0bb420f915cb22adc142fd8",
            "_dom_classes": [],
            "description": "100%",
            "_model_name": "FloatProgressModel",
            "bar_style": "success",
            "max": 31,
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "value": 31,
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "orientation": "horizontal",
            "min": 0,
            "description_tooltip": null,
            "_model_module": "@jupyter-widgets/controls",
            "layout": "IPY_MODEL_3b0895e075684c6abb200d1b95800961"
          }
        },
        "ca7eb627a9ca4c19bdb800a28693ec12": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "state": {
            "_view_name": "HTMLView",
            "style": "IPY_MODEL_320cf6e2445745b58061ee04e4eb4215",
            "_dom_classes": [],
            "description": "",
            "_model_name": "HTMLModel",
            "placeholder": "​",
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "value": " 31/31 [00:55&lt;00:00,  1.78s/it]",
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "description_tooltip": null,
            "_model_module": "@jupyter-widgets/controls",
            "layout": "IPY_MODEL_42ce100ca6c54ae4b3eeff514aa6b282"
          }
        },
        "85194ddcf0bb420f915cb22adc142fd8": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "state": {
            "_view_name": "StyleView",
            "_model_name": "ProgressStyleModel",
            "description_width": "initial",
            "_view_module": "@jupyter-widgets/base",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.2.0",
            "bar_color": null,
            "_model_module": "@jupyter-widgets/controls"
          }
        },
        "3b0895e075684c6abb200d1b95800961": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "320cf6e2445745b58061ee04e4eb4215": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_view_name": "StyleView",
            "_model_name": "DescriptionStyleModel",
            "description_width": "",
            "_view_module": "@jupyter-widgets/base",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.2.0",
            "_model_module": "@jupyter-widgets/controls"
          }
        },
        "42ce100ca6c54ae4b3eeff514aa6b282": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "69d11f99ef0747539318f877de2427ef": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "state": {
            "_view_name": "HBoxView",
            "_dom_classes": [],
            "_model_name": "HBoxModel",
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "box_style": "",
            "layout": "IPY_MODEL_05db429e21fd4fbca5c14c5ffe50aff2",
            "_model_module": "@jupyter-widgets/controls",
            "children": [
              "IPY_MODEL_8133c63affb1488fb223438b49636949",
              "IPY_MODEL_b8f37bb5e3b54f37b092cf10f98a0ce5"
            ]
          }
        },
        "05db429e21fd4fbca5c14c5ffe50aff2": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "8133c63affb1488fb223438b49636949": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "state": {
            "_view_name": "ProgressView",
            "style": "IPY_MODEL_bee26e650caa4cd0a5f6075cbb645694",
            "_dom_classes": [],
            "description": "100%",
            "_model_name": "FloatProgressModel",
            "bar_style": "success",
            "max": 31,
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "value": 31,
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "orientation": "horizontal",
            "min": 0,
            "description_tooltip": null,
            "_model_module": "@jupyter-widgets/controls",
            "layout": "IPY_MODEL_dfa7147cc28c4f248b7ce51b2ff1dc18"
          }
        },
        "b8f37bb5e3b54f37b092cf10f98a0ce5": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "state": {
            "_view_name": "HTMLView",
            "style": "IPY_MODEL_bd2611bb39c845df914fecabf3de3e47",
            "_dom_classes": [],
            "description": "",
            "_model_name": "HTMLModel",
            "placeholder": "​",
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "value": " 31/31 [00:42&lt;00:00,  1.39s/it]",
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "description_tooltip": null,
            "_model_module": "@jupyter-widgets/controls",
            "layout": "IPY_MODEL_578814ba6f5d4889ab9340c7a8d73f81"
          }
        },
        "bee26e650caa4cd0a5f6075cbb645694": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "state": {
            "_view_name": "StyleView",
            "_model_name": "ProgressStyleModel",
            "description_width": "initial",
            "_view_module": "@jupyter-widgets/base",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.2.0",
            "bar_color": null,
            "_model_module": "@jupyter-widgets/controls"
          }
        },
        "dfa7147cc28c4f248b7ce51b2ff1dc18": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "bd2611bb39c845df914fecabf3de3e47": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_view_name": "StyleView",
            "_model_name": "DescriptionStyleModel",
            "description_width": "",
            "_view_module": "@jupyter-widgets/base",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.2.0",
            "_model_module": "@jupyter-widgets/controls"
          }
        },
        "578814ba6f5d4889ab9340c7a8d73f81": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "6b7f761706ed4200b0d9e8331e61bae4": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "state": {
            "_view_name": "HBoxView",
            "_dom_classes": [],
            "_model_name": "HBoxModel",
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "box_style": "",
            "layout": "IPY_MODEL_60e9a8a36cf8474695df9e06084dea33",
            "_model_module": "@jupyter-widgets/controls",
            "children": [
              "IPY_MODEL_aa99c30de1334b76bcee62f2841624cd",
              "IPY_MODEL_90196ffae891408686c9095e0a921f71"
            ]
          }
        },
        "60e9a8a36cf8474695df9e06084dea33": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "aa99c30de1334b76bcee62f2841624cd": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "state": {
            "_view_name": "ProgressView",
            "style": "IPY_MODEL_9ca316c066214c17b19350f614beea7e",
            "_dom_classes": [],
            "description": "100%",
            "_model_name": "FloatProgressModel",
            "bar_style": "success",
            "max": 31,
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "value": 31,
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "orientation": "horizontal",
            "min": 0,
            "description_tooltip": null,
            "_model_module": "@jupyter-widgets/controls",
            "layout": "IPY_MODEL_8031cb6339334b7f9f31b93836c470f5"
          }
        },
        "90196ffae891408686c9095e0a921f71": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "state": {
            "_view_name": "HTMLView",
            "style": "IPY_MODEL_a555a955044945789738c63a44f7a05a",
            "_dom_classes": [],
            "description": "",
            "_model_name": "HTMLModel",
            "placeholder": "​",
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "value": " 31/31 [00:45&lt;00:00,  1.46s/it]",
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "description_tooltip": null,
            "_model_module": "@jupyter-widgets/controls",
            "layout": "IPY_MODEL_711512324b304e38acc021a573c3c776"
          }
        },
        "9ca316c066214c17b19350f614beea7e": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "state": {
            "_view_name": "StyleView",
            "_model_name": "ProgressStyleModel",
            "description_width": "initial",
            "_view_module": "@jupyter-widgets/base",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.2.0",
            "bar_color": null,
            "_model_module": "@jupyter-widgets/controls"
          }
        },
        "8031cb6339334b7f9f31b93836c470f5": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "a555a955044945789738c63a44f7a05a": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_view_name": "StyleView",
            "_model_name": "DescriptionStyleModel",
            "description_width": "",
            "_view_module": "@jupyter-widgets/base",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.2.0",
            "_model_module": "@jupyter-widgets/controls"
          }
        },
        "711512324b304e38acc021a573c3c776": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "ba8c2adbcdc8417da2428d9c3638ffdf": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "state": {
            "_view_name": "HBoxView",
            "_dom_classes": [],
            "_model_name": "HBoxModel",
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "box_style": "",
            "layout": "IPY_MODEL_f885c01e489a4e93b9fd8c3d205e51c2",
            "_model_module": "@jupyter-widgets/controls",
            "children": [
              "IPY_MODEL_1376a820502e4057a81968176327af42",
              "IPY_MODEL_327a5fdb5c6e49c2872b6c5e41d6db9c"
            ]
          }
        },
        "f885c01e489a4e93b9fd8c3d205e51c2": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "1376a820502e4057a81968176327af42": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "state": {
            "_view_name": "ProgressView",
            "style": "IPY_MODEL_d76aa4e0c9864877b7e98ec8956d3a7e",
            "_dom_classes": [],
            "description": "100%",
            "_model_name": "FloatProgressModel",
            "bar_style": "success",
            "max": 31,
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "value": 31,
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "orientation": "horizontal",
            "min": 0,
            "description_tooltip": null,
            "_model_module": "@jupyter-widgets/controls",
            "layout": "IPY_MODEL_7a0df8bdcdc14776a6c660897f46137d"
          }
        },
        "327a5fdb5c6e49c2872b6c5e41d6db9c": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "state": {
            "_view_name": "HTMLView",
            "style": "IPY_MODEL_445ac7c9d5a049bdb4e375506f16b3d9",
            "_dom_classes": [],
            "description": "",
            "_model_name": "HTMLModel",
            "placeholder": "​",
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "value": " 31/31 [1:44:05&lt;00:00, 201.47s/it]",
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "description_tooltip": null,
            "_model_module": "@jupyter-widgets/controls",
            "layout": "IPY_MODEL_013d62e7881640f5979811e85269ba21"
          }
        },
        "d76aa4e0c9864877b7e98ec8956d3a7e": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "state": {
            "_view_name": "StyleView",
            "_model_name": "ProgressStyleModel",
            "description_width": "initial",
            "_view_module": "@jupyter-widgets/base",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.2.0",
            "bar_color": null,
            "_model_module": "@jupyter-widgets/controls"
          }
        },
        "7a0df8bdcdc14776a6c660897f46137d": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "445ac7c9d5a049bdb4e375506f16b3d9": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_view_name": "StyleView",
            "_model_name": "DescriptionStyleModel",
            "description_width": "",
            "_view_module": "@jupyter-widgets/base",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.2.0",
            "_model_module": "@jupyter-widgets/controls"
          }
        },
        "013d62e7881640f5979811e85269ba21": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "b8979a45f03141a6ba4546ce9f15cf91": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "state": {
            "_view_name": "HBoxView",
            "_dom_classes": [],
            "_model_name": "HBoxModel",
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "box_style": "",
            "layout": "IPY_MODEL_e3db188ba93144e4a5afb67be9c6b18a",
            "_model_module": "@jupyter-widgets/controls",
            "children": [
              "IPY_MODEL_7cc228a2a497422885ffedb5eb3201e5",
              "IPY_MODEL_8766229068ab433ab05fac6b4fa24c56"
            ]
          }
        },
        "e3db188ba93144e4a5afb67be9c6b18a": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "7cc228a2a497422885ffedb5eb3201e5": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "state": {
            "_view_name": "ProgressView",
            "style": "IPY_MODEL_dfa7f218e0db4c66826d0a85bc71ae93",
            "_dom_classes": [],
            "description": "100%",
            "_model_name": "FloatProgressModel",
            "bar_style": "success",
            "max": 31,
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "value": 31,
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "orientation": "horizontal",
            "min": 0,
            "description_tooltip": null,
            "_model_module": "@jupyter-widgets/controls",
            "layout": "IPY_MODEL_fdafaa20d7fb488486fe472300b9149b"
          }
        },
        "8766229068ab433ab05fac6b4fa24c56": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "state": {
            "_view_name": "HTMLView",
            "style": "IPY_MODEL_0bd9bbbeae3e41e281f86106215902df",
            "_dom_classes": [],
            "description": "",
            "_model_name": "HTMLModel",
            "placeholder": "​",
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "value": " 31/31 [07:44&lt;00:00, 14.97s/it]",
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "description_tooltip": null,
            "_model_module": "@jupyter-widgets/controls",
            "layout": "IPY_MODEL_2d69efe57d5f47f4b8017922bf3f3f02"
          }
        },
        "dfa7f218e0db4c66826d0a85bc71ae93": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "state": {
            "_view_name": "StyleView",
            "_model_name": "ProgressStyleModel",
            "description_width": "initial",
            "_view_module": "@jupyter-widgets/base",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.2.0",
            "bar_color": null,
            "_model_module": "@jupyter-widgets/controls"
          }
        },
        "fdafaa20d7fb488486fe472300b9149b": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "0bd9bbbeae3e41e281f86106215902df": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_view_name": "StyleView",
            "_model_name": "DescriptionStyleModel",
            "description_width": "",
            "_view_module": "@jupyter-widgets/base",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.2.0",
            "_model_module": "@jupyter-widgets/controls"
          }
        },
        "2d69efe57d5f47f4b8017922bf3f3f02": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "b0179299f6354a3cac4de319bdb22e02": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "state": {
            "_view_name": "HBoxView",
            "_dom_classes": [],
            "_model_name": "HBoxModel",
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "box_style": "",
            "layout": "IPY_MODEL_8b15f1198d514dfcbc8dfc8e3abc26b3",
            "_model_module": "@jupyter-widgets/controls",
            "children": [
              "IPY_MODEL_f6fd1ad5e11e48a68f4d9522dacd6a3f",
              "IPY_MODEL_29d030e6f5cd43efadcc414c5b547bec"
            ]
          }
        },
        "8b15f1198d514dfcbc8dfc8e3abc26b3": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "f6fd1ad5e11e48a68f4d9522dacd6a3f": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "state": {
            "_view_name": "ProgressView",
            "style": "IPY_MODEL_b6db9171ec884784b252b32718f4e921",
            "_dom_classes": [],
            "description": "100%",
            "_model_name": "FloatProgressModel",
            "bar_style": "success",
            "max": 31,
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "value": 31,
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "orientation": "horizontal",
            "min": 0,
            "description_tooltip": null,
            "_model_module": "@jupyter-widgets/controls",
            "layout": "IPY_MODEL_89847cfeea5f4e67bd7bc5f956183347"
          }
        },
        "29d030e6f5cd43efadcc414c5b547bec": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "state": {
            "_view_name": "HTMLView",
            "style": "IPY_MODEL_4d74748538be45f79a1a7fa8591cf462",
            "_dom_classes": [],
            "description": "",
            "_model_name": "HTMLModel",
            "placeholder": "​",
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "value": " 31/31 [10:35&lt;00:00, 20.50s/it]",
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "description_tooltip": null,
            "_model_module": "@jupyter-widgets/controls",
            "layout": "IPY_MODEL_7f15bac0ebbe431badace764a3cb8948"
          }
        },
        "b6db9171ec884784b252b32718f4e921": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "state": {
            "_view_name": "StyleView",
            "_model_name": "ProgressStyleModel",
            "description_width": "initial",
            "_view_module": "@jupyter-widgets/base",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.2.0",
            "bar_color": null,
            "_model_module": "@jupyter-widgets/controls"
          }
        },
        "89847cfeea5f4e67bd7bc5f956183347": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "4d74748538be45f79a1a7fa8591cf462": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_view_name": "StyleView",
            "_model_name": "DescriptionStyleModel",
            "description_width": "",
            "_view_module": "@jupyter-widgets/base",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.2.0",
            "_model_module": "@jupyter-widgets/controls"
          }
        },
        "7f15bac0ebbe431badace764a3cb8948": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "24562794d30748e4b60a39cb0eab088c": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "state": {
            "_view_name": "HBoxView",
            "_dom_classes": [],
            "_model_name": "HBoxModel",
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "box_style": "",
            "layout": "IPY_MODEL_102a953fd7604e82a00db467557e2d0f",
            "_model_module": "@jupyter-widgets/controls",
            "children": [
              "IPY_MODEL_81bd274a196c4afc868a98ddb8506ccd",
              "IPY_MODEL_b02ae3089e634530be377a64b1572be5"
            ]
          }
        },
        "102a953fd7604e82a00db467557e2d0f": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "81bd274a196c4afc868a98ddb8506ccd": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "state": {
            "_view_name": "ProgressView",
            "style": "IPY_MODEL_77fcddcc920f46fa8ea846aaabccaa4d",
            "_dom_classes": [],
            "description": "100%",
            "_model_name": "FloatProgressModel",
            "bar_style": "success",
            "max": 31,
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "value": 31,
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "orientation": "horizontal",
            "min": 0,
            "description_tooltip": null,
            "_model_module": "@jupyter-widgets/controls",
            "layout": "IPY_MODEL_96dc7eff17c84ec8978d41c24b1f5004"
          }
        },
        "b02ae3089e634530be377a64b1572be5": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "state": {
            "_view_name": "HTMLView",
            "style": "IPY_MODEL_5955591fec2e40018d8619688bd58429",
            "_dom_classes": [],
            "description": "",
            "_model_name": "HTMLModel",
            "placeholder": "​",
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "value": " 31/31 [19:51&lt;00:00, 38.43s/it]",
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "description_tooltip": null,
            "_model_module": "@jupyter-widgets/controls",
            "layout": "IPY_MODEL_6a8b3729e92a40fc89b1c6bf249abc44"
          }
        },
        "77fcddcc920f46fa8ea846aaabccaa4d": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "state": {
            "_view_name": "StyleView",
            "_model_name": "ProgressStyleModel",
            "description_width": "initial",
            "_view_module": "@jupyter-widgets/base",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.2.0",
            "bar_color": null,
            "_model_module": "@jupyter-widgets/controls"
          }
        },
        "96dc7eff17c84ec8978d41c24b1f5004": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "5955591fec2e40018d8619688bd58429": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_view_name": "StyleView",
            "_model_name": "DescriptionStyleModel",
            "description_width": "",
            "_view_module": "@jupyter-widgets/base",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.2.0",
            "_model_module": "@jupyter-widgets/controls"
          }
        },
        "6a8b3729e92a40fc89b1c6bf249abc44": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "f8c9032d01964ce090844df8aa870d2a": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "state": {
            "_view_name": "HBoxView",
            "_dom_classes": [],
            "_model_name": "HBoxModel",
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "box_style": "",
            "layout": "IPY_MODEL_57f593d07d6448b585f7bdbce7622b26",
            "_model_module": "@jupyter-widgets/controls",
            "children": [
              "IPY_MODEL_c91b9c27330d43d6b9bb85f29db35221",
              "IPY_MODEL_6d083ed68e274299b15d33bb3cc07f5a"
            ]
          }
        },
        "57f593d07d6448b585f7bdbce7622b26": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "c91b9c27330d43d6b9bb85f29db35221": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "state": {
            "_view_name": "ProgressView",
            "style": "IPY_MODEL_157fd78879024057b27452c3eed8c289",
            "_dom_classes": [],
            "description": "100%",
            "_model_name": "FloatProgressModel",
            "bar_style": "success",
            "max": 31,
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "value": 31,
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "orientation": "horizontal",
            "min": 0,
            "description_tooltip": null,
            "_model_module": "@jupyter-widgets/controls",
            "layout": "IPY_MODEL_79101dd37410416495a27b139322be1a"
          }
        },
        "6d083ed68e274299b15d33bb3cc07f5a": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "state": {
            "_view_name": "HTMLView",
            "style": "IPY_MODEL_62debe0ca82c4be792593ac2d9ba6e5a",
            "_dom_classes": [],
            "description": "",
            "_model_name": "HTMLModel",
            "placeholder": "​",
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "value": " 31/31 [01:48&lt;00:00,  3.50s/it]",
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "description_tooltip": null,
            "_model_module": "@jupyter-widgets/controls",
            "layout": "IPY_MODEL_8704112e92cc47deafa2d110906694bb"
          }
        },
        "157fd78879024057b27452c3eed8c289": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "state": {
            "_view_name": "StyleView",
            "_model_name": "ProgressStyleModel",
            "description_width": "initial",
            "_view_module": "@jupyter-widgets/base",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.2.0",
            "bar_color": null,
            "_model_module": "@jupyter-widgets/controls"
          }
        },
        "79101dd37410416495a27b139322be1a": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "62debe0ca82c4be792593ac2d9ba6e5a": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_view_name": "StyleView",
            "_model_name": "DescriptionStyleModel",
            "description_width": "",
            "_view_module": "@jupyter-widgets/base",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.2.0",
            "_model_module": "@jupyter-widgets/controls"
          }
        },
        "8704112e92cc47deafa2d110906694bb": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "971be0d0d3374da195be95dcbbc34576": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "state": {
            "_view_name": "HBoxView",
            "_dom_classes": [],
            "_model_name": "HBoxModel",
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "box_style": "",
            "layout": "IPY_MODEL_f0208c2a18164458bfa5173b740ed16d",
            "_model_module": "@jupyter-widgets/controls",
            "children": [
              "IPY_MODEL_ad18af4045334c568a0250dd4fe3500b",
              "IPY_MODEL_f20f6eb795b04b16b51a2be8043259e8"
            ]
          }
        },
        "f0208c2a18164458bfa5173b740ed16d": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "ad18af4045334c568a0250dd4fe3500b": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "state": {
            "_view_name": "ProgressView",
            "style": "IPY_MODEL_f558998c92634c13b7b2b9bf269e6354",
            "_dom_classes": [],
            "description": "100%",
            "_model_name": "FloatProgressModel",
            "bar_style": "success",
            "max": 31,
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "value": 31,
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "orientation": "horizontal",
            "min": 0,
            "description_tooltip": null,
            "_model_module": "@jupyter-widgets/controls",
            "layout": "IPY_MODEL_81975408dcae4f8a87dac1325e11c9a5"
          }
        },
        "f20f6eb795b04b16b51a2be8043259e8": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "state": {
            "_view_name": "HTMLView",
            "style": "IPY_MODEL_3fbb6e2144d141f1bbd635e9a9dd7f55",
            "_dom_classes": [],
            "description": "",
            "_model_name": "HTMLModel",
            "placeholder": "​",
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "value": " 31/31 [3:05:02&lt;00:00, 358.16s/it]",
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "description_tooltip": null,
            "_model_module": "@jupyter-widgets/controls",
            "layout": "IPY_MODEL_23f06ace008a4e078f9841493a264b9f"
          }
        },
        "f558998c92634c13b7b2b9bf269e6354": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "state": {
            "_view_name": "StyleView",
            "_model_name": "ProgressStyleModel",
            "description_width": "initial",
            "_view_module": "@jupyter-widgets/base",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.2.0",
            "bar_color": null,
            "_model_module": "@jupyter-widgets/controls"
          }
        },
        "81975408dcae4f8a87dac1325e11c9a5": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "3fbb6e2144d141f1bbd635e9a9dd7f55": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_view_name": "StyleView",
            "_model_name": "DescriptionStyleModel",
            "description_width": "",
            "_view_module": "@jupyter-widgets/base",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.2.0",
            "_model_module": "@jupyter-widgets/controls"
          }
        },
        "23f06ace008a4e078f9841493a264b9f": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "c56844f691da4e8eb2fbdd3c4080452e": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "state": {
            "_view_name": "HBoxView",
            "_dom_classes": [],
            "_model_name": "HBoxModel",
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "box_style": "",
            "layout": "IPY_MODEL_74a5f20891534b6181cc9656c669815a",
            "_model_module": "@jupyter-widgets/controls",
            "children": [
              "IPY_MODEL_6ceea209ef4c40f2bc15ee4507425e5b",
              "IPY_MODEL_de5fedc407804b80be62f209d4fb3e81"
            ]
          }
        },
        "74a5f20891534b6181cc9656c669815a": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "6ceea209ef4c40f2bc15ee4507425e5b": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "state": {
            "_view_name": "ProgressView",
            "style": "IPY_MODEL_e7939d5e977c4d6dab424caa8e2eb9fd",
            "_dom_classes": [],
            "description": "100%",
            "_model_name": "FloatProgressModel",
            "bar_style": "success",
            "max": 31,
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "value": 31,
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "orientation": "horizontal",
            "min": 0,
            "description_tooltip": null,
            "_model_module": "@jupyter-widgets/controls",
            "layout": "IPY_MODEL_404f9a49e03e4a14a63fe2eb64f606af"
          }
        },
        "de5fedc407804b80be62f209d4fb3e81": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "state": {
            "_view_name": "HTMLView",
            "style": "IPY_MODEL_448eed7c906941eca4885aae92485f96",
            "_dom_classes": [],
            "description": "",
            "_model_name": "HTMLModel",
            "placeholder": "​",
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "value": " 31/31 [00:26&lt;00:00,  1.17it/s]",
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "description_tooltip": null,
            "_model_module": "@jupyter-widgets/controls",
            "layout": "IPY_MODEL_c90d5e00d666472baaf4683a705ab783"
          }
        },
        "e7939d5e977c4d6dab424caa8e2eb9fd": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "state": {
            "_view_name": "StyleView",
            "_model_name": "ProgressStyleModel",
            "description_width": "initial",
            "_view_module": "@jupyter-widgets/base",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.2.0",
            "bar_color": null,
            "_model_module": "@jupyter-widgets/controls"
          }
        },
        "404f9a49e03e4a14a63fe2eb64f606af": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "448eed7c906941eca4885aae92485f96": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_view_name": "StyleView",
            "_model_name": "DescriptionStyleModel",
            "description_width": "",
            "_view_module": "@jupyter-widgets/base",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.2.0",
            "_model_module": "@jupyter-widgets/controls"
          }
        },
        "c90d5e00d666472baaf4683a705ab783": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "06a6a1df12a848b48746bc7ae4569307": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "state": {
            "_view_name": "HBoxView",
            "_dom_classes": [],
            "_model_name": "HBoxModel",
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "box_style": "",
            "layout": "IPY_MODEL_82f29e9a77ed43b0a90b70ea548d31de",
            "_model_module": "@jupyter-widgets/controls",
            "children": [
              "IPY_MODEL_868b630837b3406a8736f900a617c863",
              "IPY_MODEL_0935081bd44140b6b99b8c5bcf46ad69"
            ]
          }
        },
        "82f29e9a77ed43b0a90b70ea548d31de": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "868b630837b3406a8736f900a617c863": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "state": {
            "_view_name": "ProgressView",
            "style": "IPY_MODEL_5104d3a802d942a1a0de67cc33e40acf",
            "_dom_classes": [],
            "description": "100%",
            "_model_name": "FloatProgressModel",
            "bar_style": "success",
            "max": 31,
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "value": 31,
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "orientation": "horizontal",
            "min": 0,
            "description_tooltip": null,
            "_model_module": "@jupyter-widgets/controls",
            "layout": "IPY_MODEL_a6a1b6b4dc0c43cf93c9cd1752524fe3"
          }
        },
        "0935081bd44140b6b99b8c5bcf46ad69": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "state": {
            "_view_name": "HTMLView",
            "style": "IPY_MODEL_790a2ec874984690847cb52aa00bb902",
            "_dom_classes": [],
            "description": "",
            "_model_name": "HTMLModel",
            "placeholder": "​",
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "value": " 31/31 [00:22&lt;00:00,  1.38it/s]",
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "description_tooltip": null,
            "_model_module": "@jupyter-widgets/controls",
            "layout": "IPY_MODEL_cad99c1460dc4c0f9ec646b347795a20"
          }
        },
        "5104d3a802d942a1a0de67cc33e40acf": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "state": {
            "_view_name": "StyleView",
            "_model_name": "ProgressStyleModel",
            "description_width": "initial",
            "_view_module": "@jupyter-widgets/base",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.2.0",
            "bar_color": null,
            "_model_module": "@jupyter-widgets/controls"
          }
        },
        "a6a1b6b4dc0c43cf93c9cd1752524fe3": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "790a2ec874984690847cb52aa00bb902": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_view_name": "StyleView",
            "_model_name": "DescriptionStyleModel",
            "description_width": "",
            "_view_module": "@jupyter-widgets/base",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.2.0",
            "_model_module": "@jupyter-widgets/controls"
          }
        },
        "cad99c1460dc4c0f9ec646b347795a20": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "24e0ce58aadd4d4ab4f329a0fdb8214b": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "state": {
            "_view_name": "HBoxView",
            "_dom_classes": [],
            "_model_name": "HBoxModel",
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "box_style": "",
            "layout": "IPY_MODEL_634acdb702bb4a3a9877f418d729f1aa",
            "_model_module": "@jupyter-widgets/controls",
            "children": [
              "IPY_MODEL_4841652ff03d427195f8e077db32d84c",
              "IPY_MODEL_55dd302f9bc9422580ba41f06bf1f4ad"
            ]
          }
        },
        "634acdb702bb4a3a9877f418d729f1aa": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "4841652ff03d427195f8e077db32d84c": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "state": {
            "_view_name": "ProgressView",
            "style": "IPY_MODEL_0fbcb853616a46c3b3a92a5567718da7",
            "_dom_classes": [],
            "description": "100%",
            "_model_name": "FloatProgressModel",
            "bar_style": "success",
            "max": 31,
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "value": 31,
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "orientation": "horizontal",
            "min": 0,
            "description_tooltip": null,
            "_model_module": "@jupyter-widgets/controls",
            "layout": "IPY_MODEL_f35d59edfa3f4b2281a495a6303350e6"
          }
        },
        "55dd302f9bc9422580ba41f06bf1f4ad": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "state": {
            "_view_name": "HTMLView",
            "style": "IPY_MODEL_b138eddb4933452b9d30409b9a0dd2d6",
            "_dom_classes": [],
            "description": "",
            "_model_name": "HTMLModel",
            "placeholder": "​",
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "value": " 31/31 [00:44&lt;00:00,  1.42s/it]",
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "description_tooltip": null,
            "_model_module": "@jupyter-widgets/controls",
            "layout": "IPY_MODEL_53af98e32a3b4b7983eea69476ab6f1c"
          }
        },
        "0fbcb853616a46c3b3a92a5567718da7": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "state": {
            "_view_name": "StyleView",
            "_model_name": "ProgressStyleModel",
            "description_width": "initial",
            "_view_module": "@jupyter-widgets/base",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.2.0",
            "bar_color": null,
            "_model_module": "@jupyter-widgets/controls"
          }
        },
        "f35d59edfa3f4b2281a495a6303350e6": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "b138eddb4933452b9d30409b9a0dd2d6": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_view_name": "StyleView",
            "_model_name": "DescriptionStyleModel",
            "description_width": "",
            "_view_module": "@jupyter-widgets/base",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.2.0",
            "_model_module": "@jupyter-widgets/controls"
          }
        },
        "53af98e32a3b4b7983eea69476ab6f1c": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "06b148547e184ab4b2bcc60b390f813e": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "state": {
            "_view_name": "HBoxView",
            "_dom_classes": [],
            "_model_name": "HBoxModel",
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "box_style": "",
            "layout": "IPY_MODEL_a0a44a3fd68149e3bbe2c07723cf1a07",
            "_model_module": "@jupyter-widgets/controls",
            "children": [
              "IPY_MODEL_6c77d0a54d01406981554bc8fc06e0db",
              "IPY_MODEL_b7ffaa34300c4536816e9f9f744e62b6"
            ]
          }
        },
        "a0a44a3fd68149e3bbe2c07723cf1a07": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "6c77d0a54d01406981554bc8fc06e0db": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "state": {
            "_view_name": "ProgressView",
            "style": "IPY_MODEL_47cd8f675cca49aba9f16d33d9411fcc",
            "_dom_classes": [],
            "description": "100%",
            "_model_name": "FloatProgressModel",
            "bar_style": "success",
            "max": 31,
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "value": 31,
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "orientation": "horizontal",
            "min": 0,
            "description_tooltip": null,
            "_model_module": "@jupyter-widgets/controls",
            "layout": "IPY_MODEL_8811cd697a1a42019d0e77677db5694a"
          }
        },
        "b7ffaa34300c4536816e9f9f744e62b6": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "state": {
            "_view_name": "HTMLView",
            "style": "IPY_MODEL_54b3eaf42f964cd192ca35b26382825a",
            "_dom_classes": [],
            "description": "",
            "_model_name": "HTMLModel",
            "placeholder": "​",
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "value": " 31/31 [03:04&lt;00:00,  5.95s/it]",
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "description_tooltip": null,
            "_model_module": "@jupyter-widgets/controls",
            "layout": "IPY_MODEL_c0f822e4081c4404906e6aad5d614e6d"
          }
        },
        "47cd8f675cca49aba9f16d33d9411fcc": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "state": {
            "_view_name": "StyleView",
            "_model_name": "ProgressStyleModel",
            "description_width": "initial",
            "_view_module": "@jupyter-widgets/base",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.2.0",
            "bar_color": null,
            "_model_module": "@jupyter-widgets/controls"
          }
        },
        "8811cd697a1a42019d0e77677db5694a": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "54b3eaf42f964cd192ca35b26382825a": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_view_name": "StyleView",
            "_model_name": "DescriptionStyleModel",
            "description_width": "",
            "_view_module": "@jupyter-widgets/base",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.2.0",
            "_model_module": "@jupyter-widgets/controls"
          }
        },
        "c0f822e4081c4404906e6aad5d614e6d": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        }
      }
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "59zzSDpHn7Uf"
      },
      "source": [
        "---\n",
        "---\n",
        "\n",
        "# 1. Setup\n",
        "\n",
        "---\n",
        "---"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ptNp4gASm2H5"
      },
      "source": [
        "### Import Packages"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hk2oSP7V3mJ2"
      },
      "source": [
        "\n",
        "##### Import the general packages"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "S-9Kv0qNm7UZ"
      },
      "source": [
        "from google.colab import drive\n",
        "import os\n",
        "import numpy as np\n",
        "import pandas as pd\n",
        "from tqdm.auto import tqdm"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bcTvfZtCWLOE"
      },
      "source": [
        "##### Import the packages for the classification"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "c4T7_A5mWBZs"
      },
      "source": [
        "from sklearn.pipeline import make_pipeline\n",
        "from sklearn.preprocessing import StandardScaler\n",
        "from sklearn.svm import SVC\n",
        "from sklearn.tree import DecisionTreeClassifier\n",
        "from sklearn.neural_network import MLPClassifier\n",
        "from sklearn.neighbors import KNeighborsClassifier\n",
        "\n",
        "from sklearn.metrics import classification_report\n",
        "\n",
        "from sklearn.model_selection import KFold\n",
        "\n",
        "from sklearn.model_selection import GridSearchCV\n",
        "from sklearn.model_selection import cross_val_score\n",
        "from sklearn.model_selection import train_test_split"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "1iFzjTax3q7v"
      },
      "source": [
        "##### Import and setup the Json package for loading in and storing data"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "tywjCwViiLw5"
      },
      "source": [
        "import json\n",
        "\n",
        "class JSONEncoder(json.JSONEncoder):\n",
        "    def default(self, obj):\n",
        "        if hasattr(obj, 'to_json'):\n",
        "            return obj.to_json(orient='records')\n",
        "        return json.JSONEncoder.default(self, obj)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "aBLWa3wX-W1g"
      },
      "source": [
        "### Set the maximum number of rows and column in the outputs"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "OoKWuPvN-XA7"
      },
      "source": [
        "pd.set_option('max_rows', 50)\n",
        "pd.set_option('max_columns', 50)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "IrqfjdME18qj"
      },
      "source": [
        "### Mount the drive"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "UnU2Cgu818yz",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "2472f16c-2971-4697-aad2-8a50e4c05af9"
      },
      "source": [
        "drive.mount('/content/drive')"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Mounted at /content/drive\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "M1Fqi2bHnKam"
      },
      "source": [
        "### Set the directory for reading the data"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "cZjGedIWnFKj"
      },
      "source": [
        "# os.chdir('/content/drive/My Drive/4th Year/CA4015/assignment2/Submission/data')    # Ivan's directory\n",
        "os.chdir('/content/drive/My Drive/assignment2/Submission/data')                     # Nathan's directory"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "0pQxo4jEocYc"
      },
      "source": [
        "---\n",
        "---\n",
        "\n",
        "# 2. Read in the data\n",
        "\n",
        "---\n",
        "---"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "nB28ZakajI_o"
      },
      "source": [
        "### Read in the filtered extracted_features dictionary from the Json file"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "rPfGo394iFWq"
      },
      "source": [
        "json_filtered_extract_features_df_data = json.load(open('map_subject_id_to_its_filtered_extracted_features_df.json'))"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "K94UIxSQiFZu",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "referenced_widgets": [
            "62ecd42cbce342b390386aa76fb12942",
            "56d0259d0a784d8682eee74a94bf27b1",
            "ef73856034a247ebb26a89a4be5592d5",
            "ca7eb627a9ca4c19bdb800a28693ec12",
            "85194ddcf0bb420f915cb22adc142fd8",
            "3b0895e075684c6abb200d1b95800961",
            "320cf6e2445745b58061ee04e4eb4215",
            "42ce100ca6c54ae4b3eeff514aa6b282"
          ]
        },
        "outputId": "0bc3ed3a-4f69-4750-c119-a5fc093cf627"
      },
      "source": [
        "subject_id_to_filtered_extracted_feature_df_map = {}\n",
        "for subject_id, sensor_df in tqdm(json_filtered_extract_features_df_data.items()):\n",
        "    subject_id_to_filtered_extracted_feature_df_map[subject_id] = pd.read_json(json_filtered_extract_features_df_data[subject_id])"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "62ecd42cbce342b390386aa76fb12942",
              "version_minor": 0,
              "version_major": 2
            },
            "text/plain": [
              "HBox(children=(FloatProgress(value=0.0, max=31.0), HTML(value='')))"
            ]
          },
          "metadata": {
            "tags": []
          }
        },
        {
          "output_type": "stream",
          "text": [
            "\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Zo1STHWKYH50"
      },
      "source": [
        "##### Manually inspect the features in this dictionary"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "JrQuUDPkP7la",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "43dbe785-ae47-458a-83d0-84e4002abd62"
      },
      "source": [
        "for student_id, df in subject_id_to_filtered_extracted_feature_df_map.items():\n",
        "    print(\"--------------------\", student_id, \"--------------------\")\n",
        "    print(\"This subject has the shape\", df.shape, \"\\n\")"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "-------------------- 46343 --------------------\n",
            "This subject has the shape (553, 357) \n",
            "\n",
            "-------------------- 759667 --------------------\n",
            "This subject has the shape (474, 385) \n",
            "\n",
            "-------------------- 781756 --------------------\n",
            "This subject has the shape (980, 270) \n",
            "\n",
            "-------------------- 844359 --------------------\n",
            "This subject has the shape (897, 362) \n",
            "\n",
            "-------------------- 1066528 --------------------\n",
            "This subject has the shape (947, 653) \n",
            "\n",
            "-------------------- 1360686 --------------------\n",
            "This subject has the shape (925, 263) \n",
            "\n",
            "-------------------- 1449548 --------------------\n",
            "This subject has the shape (953, 395) \n",
            "\n",
            "-------------------- 1455390 --------------------\n",
            "This subject has the shape (955, 207) \n",
            "\n",
            "-------------------- 1818471 --------------------\n",
            "This subject has the shape (958, 293) \n",
            "\n",
            "-------------------- 2598705 --------------------\n",
            "This subject has the shape (954, 384) \n",
            "\n",
            "-------------------- 2638030 --------------------\n",
            "This subject has the shape (948, 183) \n",
            "\n",
            "-------------------- 3509524 --------------------\n",
            "This subject has the shape (416, 168) \n",
            "\n",
            "-------------------- 3997827 --------------------\n",
            "This subject has the shape (958, 249) \n",
            "\n",
            "-------------------- 4018081 --------------------\n",
            "This subject has the shape (499, 254) \n",
            "\n",
            "-------------------- 4314139 --------------------\n",
            "This subject has the shape (961, 304) \n",
            "\n",
            "-------------------- 4426783 --------------------\n",
            "This subject has the shape (979, 202) \n",
            "\n",
            "-------------------- 5132496 --------------------\n",
            "This subject has the shape (464, 118) \n",
            "\n",
            "-------------------- 5383425 --------------------\n",
            "This subject has the shape (977, 255) \n",
            "\n",
            "-------------------- 5498603 --------------------\n",
            "This subject has the shape (744, 310) \n",
            "\n",
            "-------------------- 5797046 --------------------\n",
            "This subject has the shape (939, 251) \n",
            "\n",
            "-------------------- 6220552 --------------------\n",
            "This subject has the shape (954, 364) \n",
            "\n",
            "-------------------- 7749105 --------------------\n",
            "This subject has the shape (123, 58) \n",
            "\n",
            "-------------------- 8000685 --------------------\n",
            "This subject has the shape (958, 204) \n",
            "\n",
            "-------------------- 8173033 --------------------\n",
            "This subject has the shape (955, 203) \n",
            "\n",
            "-------------------- 8258170 --------------------\n",
            "This subject has the shape (971, 282) \n",
            "\n",
            "-------------------- 8530312 --------------------\n",
            "This subject has the shape (949, 167) \n",
            "\n",
            "-------------------- 8686948 --------------------\n",
            "This subject has the shape (955, 143) \n",
            "\n",
            "-------------------- 8692923 --------------------\n",
            "This subject has the shape (935, 215) \n",
            "\n",
            "-------------------- 9106476 --------------------\n",
            "This subject has the shape (960, 223) \n",
            "\n",
            "-------------------- 9618981 --------------------\n",
            "This subject has the shape (944, 300) \n",
            "\n",
            "-------------------- 9961348 --------------------\n",
            "This subject has the shape (719, 403) \n",
            "\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WwqOdP7uqPpx"
      },
      "source": [
        "### Read in the unfiltered_extracted_features dictionary from the Json file"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "JyYbeluXqPpy"
      },
      "source": [
        "json_unfiltered_extract_features_df_data = json.load(open('map_subject_id_to_its_unfiltered_extracted_features_df.json'))"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 66,
          "referenced_widgets": [
            "69d11f99ef0747539318f877de2427ef",
            "05db429e21fd4fbca5c14c5ffe50aff2",
            "8133c63affb1488fb223438b49636949",
            "b8f37bb5e3b54f37b092cf10f98a0ce5",
            "bee26e650caa4cd0a5f6075cbb645694",
            "dfa7147cc28c4f248b7ce51b2ff1dc18",
            "bd2611bb39c845df914fecabf3de3e47",
            "578814ba6f5d4889ab9340c7a8d73f81"
          ]
        },
        "id": "gblapU8KqPpz",
        "outputId": "6f33a6f4-92b1-42e0-fdef-7fa307fc50a5"
      },
      "source": [
        "subject_id_to_unfiltered_extracted_feature_df_map = {}\n",
        "for subject_id, sensor_df in tqdm(json_unfiltered_extract_features_df_data.items()):\n",
        "    subject_id_to_unfiltered_extracted_feature_df_map[subject_id] = pd.read_json(json_unfiltered_extract_features_df_data[subject_id])"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "69d11f99ef0747539318f877de2427ef",
              "version_minor": 0,
              "version_major": 2
            },
            "text/plain": [
              "HBox(children=(FloatProgress(value=0.0, max=31.0), HTML(value='')))"
            ]
          },
          "metadata": {
            "tags": []
          }
        },
        {
          "output_type": "stream",
          "text": [
            "\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ESh6-eXGqPp2"
      },
      "source": [
        "##### Manually inspect the features in this dictionary"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "SEuj3nnMqPp3",
        "outputId": "6a731e01-1a2e-47fb-ca2b-9ba39b3d36a1"
      },
      "source": [
        "for subject_id, df in subject_id_to_unfiltered_extracted_feature_df_map.items():\n",
        "    print(\"--------------------\", subject_id, \"--------------------\")\n",
        "    print(\"This subject has the shape\", df.shape, \"\\n\")"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "-------------------- 46343 --------------------\n",
            "This subject has the shape (553, 1058) \n",
            "\n",
            "-------------------- 759667 --------------------\n",
            "This subject has the shape (474, 1058) \n",
            "\n",
            "-------------------- 781756 --------------------\n",
            "This subject has the shape (980, 1058) \n",
            "\n",
            "-------------------- 844359 --------------------\n",
            "This subject has the shape (897, 1058) \n",
            "\n",
            "-------------------- 1066528 --------------------\n",
            "This subject has the shape (947, 1755) \n",
            "\n",
            "-------------------- 1360686 --------------------\n",
            "This subject has the shape (925, 1058) \n",
            "\n",
            "-------------------- 1449548 --------------------\n",
            "This subject has the shape (953, 1058) \n",
            "\n",
            "-------------------- 1455390 --------------------\n",
            "This subject has the shape (955, 1058) \n",
            "\n",
            "-------------------- 1818471 --------------------\n",
            "This subject has the shape (958, 1058) \n",
            "\n",
            "-------------------- 2598705 --------------------\n",
            "This subject has the shape (954, 1058) \n",
            "\n",
            "-------------------- 2638030 --------------------\n",
            "This subject has the shape (948, 1058) \n",
            "\n",
            "-------------------- 3509524 --------------------\n",
            "This subject has the shape (416, 1058) \n",
            "\n",
            "-------------------- 3997827 --------------------\n",
            "This subject has the shape (958, 1058) \n",
            "\n",
            "-------------------- 4018081 --------------------\n",
            "This subject has the shape (499, 1138) \n",
            "\n",
            "-------------------- 4314139 --------------------\n",
            "This subject has the shape (961, 1058) \n",
            "\n",
            "-------------------- 4426783 --------------------\n",
            "This subject has the shape (979, 1058) \n",
            "\n",
            "-------------------- 5132496 --------------------\n",
            "This subject has the shape (464, 1058) \n",
            "\n",
            "-------------------- 5383425 --------------------\n",
            "This subject has the shape (977, 1058) \n",
            "\n",
            "-------------------- 5498603 --------------------\n",
            "This subject has the shape (744, 1058) \n",
            "\n",
            "-------------------- 5797046 --------------------\n",
            "This subject has the shape (939, 1058) \n",
            "\n",
            "-------------------- 6220552 --------------------\n",
            "This subject has the shape (954, 1058) \n",
            "\n",
            "-------------------- 7749105 --------------------\n",
            "This subject has the shape (123, 1395) \n",
            "\n",
            "-------------------- 8000685 --------------------\n",
            "This subject has the shape (958, 1058) \n",
            "\n",
            "-------------------- 8173033 --------------------\n",
            "This subject has the shape (955, 1058) \n",
            "\n",
            "-------------------- 8258170 --------------------\n",
            "This subject has the shape (971, 1058) \n",
            "\n",
            "-------------------- 8530312 --------------------\n",
            "This subject has the shape (949, 1058) \n",
            "\n",
            "-------------------- 8686948 --------------------\n",
            "This subject has the shape (955, 1058) \n",
            "\n",
            "-------------------- 8692923 --------------------\n",
            "This subject has the shape (935, 1058) \n",
            "\n",
            "-------------------- 9106476 --------------------\n",
            "This subject has the shape (960, 1058) \n",
            "\n",
            "-------------------- 9618981 --------------------\n",
            "This subject has the shape (944, 1058) \n",
            "\n",
            "-------------------- 9961348 --------------------\n",
            "This subject has the shape (719, 1058) \n",
            "\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bwK12ayosDm9"
      },
      "source": [
        "### Read in the no step filtered extracted_features dictionary from the Json file"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "2yMPLRaEsDm9"
      },
      "source": [
        "json_no_step_filtered_extract_features_df_data = json.load(open('no_step_map_subject_id_to_its_filtered_extracted_features_df.json'))"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 66,
          "referenced_widgets": [
            "6b7f761706ed4200b0d9e8331e61bae4",
            "60e9a8a36cf8474695df9e06084dea33",
            "aa99c30de1334b76bcee62f2841624cd",
            "90196ffae891408686c9095e0a921f71",
            "9ca316c066214c17b19350f614beea7e",
            "8031cb6339334b7f9f31b93836c470f5",
            "a555a955044945789738c63a44f7a05a",
            "711512324b304e38acc021a573c3c776"
          ]
        },
        "id": "27GDYGlUsDm-",
        "outputId": "f37d0a16-5b86-427b-87ee-583b6ce121f4"
      },
      "source": [
        "subject_id_to_no_step_filtered_extracted_feature_df_map = {}\n",
        "for subject_id, sensor_df in tqdm(json_no_step_filtered_extract_features_df_data.items()):\n",
        "    subject_id_to_no_step_filtered_extracted_feature_df_map[subject_id] = pd.read_json(json_no_step_filtered_extract_features_df_data[subject_id])"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "6b7f761706ed4200b0d9e8331e61bae4",
              "version_minor": 0,
              "version_major": 2
            },
            "text/plain": [
              "HBox(children=(FloatProgress(value=0.0, max=31.0), HTML(value='')))"
            ]
          },
          "metadata": {
            "tags": []
          }
        },
        {
          "output_type": "stream",
          "text": [
            "\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hQ3Ev0J1sDm-"
      },
      "source": [
        "##### Manually inspect the features in this dictionary"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "8_3iIfpgsDm-",
        "outputId": "83fb5702-03b0-4ab4-cb6b-1a6dafd634df"
      },
      "source": [
        "for subject_id, df in subject_id_to_no_step_filtered_extracted_feature_df_map.items():\n",
        "    print(\"--------------------\", subject_id, \"--------------------\")\n",
        "    print(\"This subject has the shape\", df.shape, \"\\n\")"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "-------------------- 46343 --------------------\n",
            "This subject has the shape (553, 357) \n",
            "\n",
            "-------------------- 759667 --------------------\n",
            "This subject has the shape (474, 385) \n",
            "\n",
            "-------------------- 781756 --------------------\n",
            "This subject has the shape (980, 270) \n",
            "\n",
            "-------------------- 844359 --------------------\n",
            "This subject has the shape (897, 362) \n",
            "\n",
            "-------------------- 1066528 --------------------\n",
            "This subject has the shape (947, 653) \n",
            "\n",
            "-------------------- 1360686 --------------------\n",
            "This subject has the shape (925, 263) \n",
            "\n",
            "-------------------- 1449548 --------------------\n",
            "This subject has the shape (953, 395) \n",
            "\n",
            "-------------------- 1455390 --------------------\n",
            "This subject has the shape (955, 207) \n",
            "\n",
            "-------------------- 1818471 --------------------\n",
            "This subject has the shape (958, 293) \n",
            "\n",
            "-------------------- 2598705 --------------------\n",
            "This subject has the shape (954, 384) \n",
            "\n",
            "-------------------- 2638030 --------------------\n",
            "This subject has the shape (948, 183) \n",
            "\n",
            "-------------------- 3509524 --------------------\n",
            "This subject has the shape (416, 168) \n",
            "\n",
            "-------------------- 3997827 --------------------\n",
            "This subject has the shape (958, 249) \n",
            "\n",
            "-------------------- 4018081 --------------------\n",
            "This subject has the shape (499, 254) \n",
            "\n",
            "-------------------- 4314139 --------------------\n",
            "This subject has the shape (961, 304) \n",
            "\n",
            "-------------------- 4426783 --------------------\n",
            "This subject has the shape (979, 202) \n",
            "\n",
            "-------------------- 5132496 --------------------\n",
            "This subject has the shape (464, 118) \n",
            "\n",
            "-------------------- 5383425 --------------------\n",
            "This subject has the shape (977, 255) \n",
            "\n",
            "-------------------- 5498603 --------------------\n",
            "This subject has the shape (744, 310) \n",
            "\n",
            "-------------------- 5797046 --------------------\n",
            "This subject has the shape (939, 251) \n",
            "\n",
            "-------------------- 6220552 --------------------\n",
            "This subject has the shape (954, 364) \n",
            "\n",
            "-------------------- 7749105 --------------------\n",
            "This subject has the shape (123, 58) \n",
            "\n",
            "-------------------- 8000685 --------------------\n",
            "This subject has the shape (958, 204) \n",
            "\n",
            "-------------------- 8173033 --------------------\n",
            "This subject has the shape (955, 203) \n",
            "\n",
            "-------------------- 8258170 --------------------\n",
            "This subject has the shape (971, 282) \n",
            "\n",
            "-------------------- 8530312 --------------------\n",
            "This subject has the shape (949, 167) \n",
            "\n",
            "-------------------- 8686948 --------------------\n",
            "This subject has the shape (955, 143) \n",
            "\n",
            "-------------------- 8692923 --------------------\n",
            "This subject has the shape (935, 215) \n",
            "\n",
            "-------------------- 9106476 --------------------\n",
            "This subject has the shape (960, 223) \n",
            "\n",
            "-------------------- 9618981 --------------------\n",
            "This subject has the shape (944, 300) \n",
            "\n",
            "-------------------- 9961348 --------------------\n",
            "This subject has the shape (719, 403) \n",
            "\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "spj4fTY0vdsh"
      },
      "source": [
        "### Read in the dictionary mapping the psg_status' to the session_id's that are classed as tis psg_status"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "jP7G9K7avtrl"
      },
      "source": [
        "map_subject_id_to_a_map_of_the_session_id_to_psg_status = json.load(open('map_subject_id_to_a_map_of_the_session_id_to_psg_status.json'))"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "D0t7vWeqmfbW"
      },
      "source": [
        "### Create an inverted version of this map - mapping psg_status to the session_id's"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "1dQeoIK5g7le"
      },
      "source": [
        "map_subject_id_to_a_map_of_the_psg_status_to_session_id = {}\n",
        "\n",
        "for subject_id, subjects_session_to_psg_map in map_subject_id_to_a_map_of_the_session_id_to_psg_status.items():\n",
        "\n",
        "    subjects_psg_to_session_map = {0:[], 1:[], 2:[], 3:[], 4:[], 5:[]}\n",
        "    for session_id, psg_status in subjects_session_to_psg_map.items():\n",
        "      \n",
        "        subjects_psg_to_session_map[psg_status].append(int(session_id))\n",
        "\n",
        "    map_subject_id_to_a_map_of_the_psg_status_to_session_id[subject_id] = subjects_psg_to_session_map"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "v7yarOzvkUMj"
      },
      "source": [
        "##### Manually inspect this dictionary"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "6a2QvWTnobgf",
        "outputId": "8b276a3c-8ef9-4a5f-9b8e-abc78c0c9a20"
      },
      "source": [
        "i = 0\n",
        "for sub_id, psg_to_id_dict in map_subject_id_to_a_map_of_the_psg_status_to_session_id.items():\n",
        "\n",
        "    # print the subject_id\n",
        "    print(\"----------------\", sub_id, \"---------------------\")\n",
        "\n",
        "    # iterate through the dictionary and print the values\n",
        "    for psg_status, session_id in psg_to_id_dict.items():\n",
        "        print(psg_status, \"->\", session_id)\n",
        "\n",
        "    # stop after 2 subjects\n",
        "    if i == 1:\n",
        "        break\n",
        "    i = i + 1"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "---------------- 46343 ---------------------\n",
            "0 -> [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 110, 111, 112, 113, 132, 305, 384, 397, 398, 399, 461, 462, 493, 494, 510, 511, 515, 520, 521, 522, 523, 524, 525, 526, 527, 528, 529, 530, 531, 532, 533, 534, 535, 536, 537, 538, 539, 540, 541, 542, 543, 544, 545, 546, 547, 548, 549, 550, 551, 552]\n",
            "1 -> [34, 35, 36, 37, 114, 306, 307, 320, 323, 324, 325, 327, 328, 385, 386, 400, 413, 463, 464, 495, 496, 500, 512, 513, 514, 516, 517, 518, 519]\n",
            "2 -> [38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 115, 116, 117, 118, 119, 133, 134, 194, 195, 196, 197, 198, 199, 200, 201, 202, 203, 204, 205, 206, 207, 208, 209, 210, 211, 212, 213, 214, 215, 216, 217, 218, 219, 220, 221, 222, 223, 224, 225, 226, 227, 228, 308, 309, 310, 311, 312, 313, 314, 315, 316, 317, 318, 319, 321, 322, 326, 387, 388, 389, 390, 391, 392, 393, 394, 395, 396, 401, 402, 403, 404, 405, 406, 407, 408, 409, 410, 411, 412, 414, 415, 416, 417, 418, 419, 420, 421, 422, 423, 424, 425, 426, 427, 428, 429, 430, 431, 432, 433, 434, 435, 436, 437, 438, 439, 440, 441, 442, 443, 444, 445, 446, 465, 466, 467, 468, 469, 470, 471, 472, 473, 474, 475, 476, 477, 478, 479, 480, 481, 482, 483, 484, 485, 486, 487, 488, 489, 490, 491, 492, 497, 498, 499, 501, 502, 503, 504, 505, 506, 507, 508, 509]\n",
            "3 -> [56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, 131, 229, 230, 231, 232, 233, 234, 235, 236, 237, 238, 239, 240, 241, 242, 243, 244, 245, 246, 247, 248, 249, 250, 251, 252, 253, 254, 255, 256, 257, 258, 259, 260, 261, 262, 263, 264, 265, 266, 267, 268, 269, 270, 271, 272, 273, 274, 275, 276, 277, 278, 279, 280, 281, 282, 283, 284, 285, 286, 287, 288, 289, 290, 291, 292, 293, 294, 295, 296, 297, 298, 299, 300, 301, 302, 303, 304, 447, 448, 449, 450, 451, 452, 453, 454, 455, 456, 457, 458, 459, 460]\n",
            "4 -> []\n",
            "5 -> [135, 136, 137, 138, 139, 140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 171, 172, 173, 174, 175, 176, 177, 178, 179, 180, 181, 182, 183, 184, 185, 186, 187, 188, 189, 190, 191, 192, 193, 329, 330, 331, 332, 333, 334, 335, 336, 337, 338, 339, 340, 341, 342, 343, 344, 345, 346, 347, 348, 349, 350, 351, 352, 353, 354, 355, 356, 357, 358, 359, 360, 361, 362, 363, 364, 365, 366, 367, 368, 369, 370, 371, 372, 373, 374, 375, 376, 377, 378, 379, 380, 381, 382, 383]\n",
            "---------------- 759667 ---------------------\n",
            "0 -> [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 109, 226, 415, 441, 460, 471]\n",
            "1 -> [12, 13, 14, 15, 16, 142, 227, 228, 389, 390, 395, 396, 397, 440, 442, 470, 472, 473]\n",
            "2 -> [17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, 229, 230, 231, 232, 233, 234, 235, 236, 237, 238, 239, 240, 241, 242, 243, 244, 245, 246, 247, 248, 249, 250, 251, 252, 253, 254, 255, 256, 257, 258, 259, 260, 261, 262, 263, 264, 288, 289, 290, 291, 292, 293, 294, 295, 296, 297, 298, 299, 300, 301, 302, 303, 304, 305, 306, 307, 308, 309, 310, 311, 312, 313, 314, 315, 316, 317, 318, 319, 320, 321, 322, 323, 324, 325, 326, 327, 328, 329, 330, 331, 332, 333, 334, 335, 336, 380, 381, 382, 383, 384, 385, 386, 387, 388, 391, 392, 393, 394]\n",
            "3 -> [36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 265, 266, 267, 268, 269, 270, 271, 272, 273, 274, 275, 276, 277, 278, 279, 280, 281, 282, 283, 284, 285, 286, 287, 337, 338, 339, 340, 341, 342, 343, 344, 345, 346, 347, 348, 349, 350, 351, 352, 353, 354, 355, 356, 357, 358, 359, 360, 361, 362, 363, 364, 365, 366, 367, 368, 369, 370, 371, 372, 373, 374, 375, 376, 377, 378, 379]\n",
            "4 -> []\n",
            "5 -> [131, 132, 133, 134, 135, 136, 137, 138, 139, 140, 141, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 171, 172, 173, 174, 175, 176, 177, 178, 179, 180, 181, 182, 183, 184, 185, 186, 187, 188, 189, 190, 191, 192, 193, 194, 195, 196, 197, 198, 199, 200, 201, 202, 203, 204, 205, 206, 207, 208, 209, 210, 211, 212, 213, 214, 215, 216, 217, 218, 219, 220, 221, 222, 223, 224, 225, 398, 399, 400, 401, 402, 403, 404, 405, 406, 407, 408, 409, 410, 411, 412, 413, 414, 416, 417, 418, 419, 420, 421, 422, 423, 424, 425, 426, 427, 428, 429, 430, 431, 432, 433, 434, 435, 436, 437, 438, 439, 443, 444, 445, 446, 447, 448, 449, 450, 451, 452, 453, 454, 455, 456, 457, 458, 459, 461, 462, 463, 464, 465, 466, 467, 468, 469]\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WxA9RM7DXvGw"
      },
      "source": [
        "---\n",
        "---\n",
        "\n",
        "# 3. Create the functions needed for the classification\n",
        "\n",
        "---\n",
        "---"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "CN-094zg4XxO"
      },
      "source": [
        "### Create a function to create the dataset and y needed in the classification using these extracted features"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "6dpeRBZjwDK7"
      },
      "source": [
        "def create_map_of_subject_id_to_the_dataset_and_y_for_all_psg_status(subject_id_to_extracted_feature_df_map, map_subject_id_to_a_map_of_the_psg_status_to_session_id, map_subject_id_to_a_map_of_the_session_id_to_psg_status):\n",
        "\n",
        "    map_subject_id_to_all_psg_dataset_and_y = {}\n",
        "    map_subject_id_to_binary_dataset_and_y = {}\n",
        "\n",
        "    for subject_id, extracted_features_df in tqdm(subject_id_to_extracted_feature_df_map.items()):\n",
        "\n",
        "        # get the map of psg_status to session_id for this subject\n",
        "        subjects_psg_to_session_map = map_subject_id_to_a_map_of_the_psg_status_to_session_id[subject_id]\n",
        "\n",
        "        # get the map of session_id to psg_status for this subject\n",
        "        subjects_session_to_psg_map = map_subject_id_to_a_map_of_the_session_id_to_psg_status[subject_id]\n",
        "\n",
        "        # group the features based on their sleep states \n",
        "        zero_features = extracted_features_df.iloc[subjects_psg_to_session_map[0], :].values\n",
        "        one_features = extracted_features_df.iloc[subjects_psg_to_session_map[1], :].values\n",
        "        two_features = extracted_features_df.iloc[subjects_psg_to_session_map[2], :].values\n",
        "        three_features = extracted_features_df.iloc[subjects_psg_to_session_map[3], :].values\n",
        "        four_features = extracted_features_df.iloc[subjects_psg_to_session_map[4], :].values\n",
        "        five_features = extracted_features_df.iloc[subjects_psg_to_session_map[5], :].values\n",
        "\n",
        "        # Creating the feature vector for all the psg_status' to be used as the training data\n",
        "        all_psg_status_dataset = np.vstack((zero_features, one_features, two_features, three_features, four_features, five_features))\n",
        "\n",
        "        # Creating labels here as the target vector\n",
        "        all_psg_status_y = list(subjects_session_to_psg_map.values())\n",
        "\n",
        "        # ensure that the dataset and y are the same size\n",
        "        assert len(binary_psg_status_dataset) == len(binary_psg_status_y)\n",
        "        \n",
        "        # create a dictionary mapping each subject_id to a tuple of the dataset and 'y' for all of the different psg_status'\n",
        "        map_subject_id_to_all_psg_dataset_and_y[subject_id] = (all_psg_status_dataset, all_psg_status_y)\n",
        "\n",
        "    return map_subject_id_to_all_psg_dataset_and_y"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "5wg2Lk7KK71w"
      },
      "source": [
        "def create_map_of_subject_id_to_the_dataset_and_y_for_only_awake_and_asleep_psg_status(subject_id_to_extracted_feature_df_map, map_subject_id_to_a_map_of_the_psg_status_to_session_id, map_subject_id_to_a_map_of_the_session_id_to_psg_status):\n",
        "\n",
        "    map_subject_id_to_all_psg_dataset_and_y = {}\n",
        "    map_subject_id_to_binary_dataset_and_y = {}\n",
        "\n",
        "    for subject_id, extracted_features_df in tqdm(subject_id_to_extracted_feature_df_map.items()):\n",
        "\n",
        "        # get the map of psg_status to session_id for this subject\n",
        "        subjects_psg_to_session_map = map_subject_id_to_a_map_of_the_psg_status_to_session_id[subject_id]\n",
        "\n",
        "        # get the map of session_id to psg_status for this subject\n",
        "        subjects_session_to_psg_map = map_subject_id_to_a_map_of_the_session_id_to_psg_status[subject_id]\n",
        "\n",
        "        # group the features based on whether the person is awake or asleep \n",
        "        awake_features = extracted_features_df.iloc[subjects_psg_to_session_map[0], :].values\n",
        "        asleep_features = extracted_features_df.iloc[subjects_psg_to_session_map[1] + subjects_psg_to_session_map[2] + subjects_psg_to_session_map[3] + subjects_psg_to_session_map[4] + subjects_psg_to_session_map[5], :].values\n",
        "\n",
        "        # Creating the feature vector for the awake and asleep features to be used as the training data\n",
        "        binary_psg_status_dataset = np.vstack((awake_features, asleep_features))\n",
        "\n",
        "        # Creating labels here as the target vector - as the above 'y' list contains all psg_status', we need to change it so that it only contains binary values\n",
        "        all_psg_status_y = list(subjects_session_to_psg_map.values())\n",
        "        binary_psg_status_y = []\n",
        "        for binary_val in all_psg_status_y:\n",
        "            # replace the multiple sleep stats with a binary value fo sleep and awake\n",
        "            if binary_val in [2, 3, 4, 5]:\n",
        "                binary_val = 1\n",
        "            binary_psg_status_y.append(binary_val)\n",
        "\n",
        "        # ensure that the dataset and y are the same size\n",
        "        assert len(binary_psg_status_dataset) == len(binary_psg_status_y)\n",
        "\n",
        "        # create a dictionary mapping each subject_id to a tuple of the dataset and 'y' for all of the different psg_status'\n",
        "        map_subject_id_to_binary_dataset_and_y[subject_id] = (binary_psg_status_dataset, binary_psg_status_y)\n",
        "\n",
        "    return map_subject_id_to_binary_dataset_and_y"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "16ezt6XAagLS"
      },
      "source": [
        "### Create functions to classify the dataset and y"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "vlA-j-rAagUr"
      },
      "source": [
        "def create_fit_and_score_the_model(classifier_dict, X_train, y_train, X_test, y_test, optimised_boolean, cv_settings):\n",
        "\n",
        "    if optimised_boolean:\n",
        "        # create an optimised model by cross validating with the various parameters\n",
        "        clf = make_pipeline(StandardScaler(), GridSearchCV(classifier_dict['model'], classifier_dict['parameters'], cv=cv_settings, refit=True))\n",
        "\n",
        "    else:\n",
        "        # create the default model\n",
        "        clf = make_pipeline(StandardScaler(), classifier_dict['model'])\n",
        "\n",
        "    # train the classifier\n",
        "    clf.fit(X_train, y_train)\n",
        "\n",
        "    # get a prediction for y based on a test set\n",
        "    y_pred = clf.predict(X_test)\n",
        "\n",
        "    # output a report on the precision and recall to ensure that the model is not cheating by just predicting the most common class\n",
        "    class_report_dict = classification_report(y_test, y_pred, output_dict=True, zero_division=0)\n",
        "\n",
        "    # we now have the best parameters to use with the classifier\n",
        "    score = clf.score(X_test, y_test)\n",
        "\n",
        "    return score, class_report_dict"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "WouehcewV8Wy"
      },
      "source": [
        "def classify(map_subject_id_to_dataset_and_y, names_of_models_to_test, cv_settings, optimise_params_bool, classifier_report_bool):\n",
        "\n",
        "    # create dataframe for storing results\n",
        "    score_results = pd.DataFrame(columns=[\"subject_id\"].extend(names_of_models_to_test))\n",
        "    map_subject_to_classifier_to_classification_report = {}\n",
        "\n",
        "    for subject_id, tuple_of_dataset_and_y in tqdm(map_subject_id_to_dataset_and_y.items()):\n",
        "\n",
        "        # skip the subjects that are causing us problems\n",
        "        # we suspect that it is becauce there are some rows with psg_status = '4' which, based on the input data and the paper asssociated with the dataset, shoulcn't be there\n",
        "        if subject_id in ['5383425', '8258170', '9618981']:\n",
        "            continue\n",
        "\n",
        "        # assign the dataset and the 'y' variable\n",
        "        dataset = tuple_of_dataset_and_y[0]\n",
        "        y = tuple_of_dataset_and_y[1]\n",
        "\n",
        "        # create train-test split\n",
        "        X_train, X_test, y_train, y_test = train_test_split(dataset, y, test_size=0.33, random_state=42)\n",
        "\n",
        "        # create a dictionery to map classifier to a score\n",
        "        row_of_scores_df = pd.DataFrame(columns=[\"subject_id\"].extend(names_of_models_to_test), index=[0])\n",
        "\n",
        "        # populate the row with subject_id\n",
        "        row_of_scores_df.loc[0, 'subject_id'] = subject_id\n",
        "\n",
        "        map_classifiers_to_classification_report = {}\n",
        "\n",
        "        # for each model we score the parameters and we can then get an average for each set of parameters\n",
        "        for classifier_name, classifier_dict in classifiers.items():\n",
        "\n",
        "            # skip the mulit-layer perceptron classifier as the output to this classifier is messy\n",
        "            # the reuslts from this classifier are still valid\n",
        "            if classifier_name == 'MLP':\n",
        "                continue\n",
        "            \n",
        "            # create a model, train it and test it to produce a score\n",
        "            score, classification_report_dict = create_fit_and_score_the_model(classifier_dict, X_train, y_train, X_test, y_test, optimise_params_bool, cv_settings)\n",
        "\n",
        "            # add this score to the scores from the other classifiers for this subject\n",
        "            row_of_scores_df.loc[0, classifier_name] = score\n",
        "\n",
        "            # add the classification report to a dictionary\n",
        "            map_classifiers_to_classification_report[classifier_name] = classification_report_dict\n",
        "\n",
        "        map_subject_to_classifier_to_classification_report[subject_id] = map_classifiers_to_classification_report\n",
        "\n",
        "        # Add the row of scores to the row of scores recorded for the other subjects\n",
        "        score_results = pd.concat([score_results, row_of_scores_df], axis=0)\n",
        "\n",
        "    if classifier_report_bool:\n",
        "        return score_results.reset_index(drop=True), map_subject_to_classifier_to_classification_report\n",
        "\n",
        "    else:\n",
        "        return score_results.reset_index(drop=True)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zUblxpl_DopF"
      },
      "source": [
        "---\n",
        "---\n",
        "\n",
        "# 4. Classify the data\n",
        "\n",
        "---\n",
        "---"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "TSyNbn51xeD2"
      },
      "source": [
        "### Configurate the model variables needed for the classification"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "41vig1y9_Jdh"
      },
      "source": [
        "##### Set out the names and parameters of the classifier models we will use"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "fCcFaG-g95Y7"
      },
      "source": [
        "names_of_models_to_test = ['SVC', 'Decision Tree', 'K-Neighbours', 'MLP']\n",
        "\n",
        "model_function_for_these_models = [SVC(), DecisionTreeClassifier(), KNeighborsClassifier(), MLPClassifier()]\n",
        "\n",
        "params_for_these_models = [{'kernel':('linear', 'rbf'), 'degree':[2, 3], 'gamma':['auto', 'scale'], 'C':[1, 10]}, #{'kernel':('linear', 'rbf', 'poly', 'sigmoid'), 'degree':[2, 10], 'gamma':['auto', 'scale'], 'C':[1, 10]}\n",
        "                           {'criterion':('gini', 'entropy'), 'splitter':('best', 'random')}, #plenty more to add if needed\n",
        "                           {'n_neighbors':[3,10], 'weights':('uniform', 'distance')},\n",
        "                           {'activation':('identity', 'logistic', 'tanh', 'relu'), 'alpha':[0.0001,0.0002]},\n",
        "                          ]"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "HStijZ_XyLrH"
      },
      "source": [
        "##### For each model we will use, create a dictionary to store its variables"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "lnVjw31VOqQX",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "7a691124-e3e4-490a-8b34-a1dee3a0fe15"
      },
      "source": [
        "classifiers = {}\n",
        "for i in range(len(names_of_models_to_test)):\n",
        "\n",
        "    model_name = names_of_models_to_test[i]\n",
        "    model_function = model_function_for_these_models[i]\n",
        "    model_params = params_for_these_models[i]\n",
        "\n",
        "    classifiers[model_name] = {}\n",
        "    classifiers[model_name][\"model\"] = model_function\n",
        "    classifiers[model_name][\"parameters\"] = model_params\n",
        "\n",
        "print(classifiers)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "{'SVC': {'model': SVC(C=1.0, break_ties=False, cache_size=200, class_weight=None, coef0=0.0,\n",
            "    decision_function_shape='ovr', degree=3, gamma='scale', kernel='rbf',\n",
            "    max_iter=-1, probability=False, random_state=None, shrinking=True,\n",
            "    tol=0.001, verbose=False), 'parameters': {'kernel': ('linear', 'rbf'), 'degree': [2, 3], 'gamma': ['auto', 'scale'], 'C': [1, 10]}}, 'Decision Tree': {'model': DecisionTreeClassifier(ccp_alpha=0.0, class_weight=None, criterion='gini',\n",
            "                       max_depth=None, max_features=None, max_leaf_nodes=None,\n",
            "                       min_impurity_decrease=0.0, min_impurity_split=None,\n",
            "                       min_samples_leaf=1, min_samples_split=2,\n",
            "                       min_weight_fraction_leaf=0.0, presort='deprecated',\n",
            "                       random_state=None, splitter='best'), 'parameters': {'criterion': ('gini', 'entropy'), 'splitter': ('best', 'random')}}, 'K-Neighbours': {'model': KNeighborsClassifier(algorithm='auto', leaf_size=30, metric='minkowski',\n",
            "                     metric_params=None, n_jobs=None, n_neighbors=5, p=2,\n",
            "                     weights='uniform'), 'parameters': {'n_neighbors': [3, 10], 'weights': ('uniform', 'distance')}}, 'MLP': {'model': MLPClassifier(activation='relu', alpha=0.0001, batch_size='auto', beta_1=0.9,\n",
            "              beta_2=0.999, early_stopping=False, epsilon=1e-08,\n",
            "              hidden_layer_sizes=(100,), learning_rate='constant',\n",
            "              learning_rate_init=0.001, max_fun=15000, max_iter=200,\n",
            "              momentum=0.9, n_iter_no_change=10, nesterovs_momentum=True,\n",
            "              power_t=0.5, random_state=None, shuffle=True, solver='adam',\n",
            "              tol=0.0001, validation_fraction=0.1, verbose=False,\n",
            "              warm_start=False), 'parameters': {'activation': ('identity', 'logistic', 'tanh', 'relu'), 'alpha': [0.0001, 0.0002]}}}\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "82BbOk_PyrUQ"
      },
      "source": [
        "##### Set how many splits we use in cross-validation"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "sCyDZhy3yvNO"
      },
      "source": [
        "cv_settings = KFold(n_splits=4, shuffle=True)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "0C5Fqz6aCrgC"
      },
      "source": [
        ""
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "IshY0sEOID_Q"
      },
      "source": [
        "---\n",
        "\n",
        "## Our Hypothesis\n",
        "\n",
        "\n",
        "---"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Q9tJWlTFJFgo"
      },
      "source": [
        "Throughout the course of our project, we made a couple of tough decicions in relation to what way we wanted to deal with the data in order to try get the most accurate model we could when it came to the classification task. These decisions were based on hypotheses we had on what we thought would positively effect the model.\n",
        "1.   We decided to include the the step count data\n",
        "2.   We decided to subset the extracted features by selecting the most relevant ones\n",
        "3.   We decided to run our classificaton models using multiple variations of the different parameters to then use cross-validation to optimise these parameters\n",
        "\n",
        "These hypothesese results in us running our classification algorithms using optimised parameters, on a subset of the extracted features where the step count data is included.\n",
        "\n",
        "The below code will emulate this approach to see what sort of accuracy our classification algorithm has under these conditions.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "QoFQd_1COUES"
      },
      "source": [
        "#### Create a dictionary mapping subject_id to the dataset and the y needed for the classification"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 66,
          "referenced_widgets": [
            "ba8c2adbcdc8417da2428d9c3638ffdf",
            "f885c01e489a4e93b9fd8c3d205e51c2",
            "1376a820502e4057a81968176327af42",
            "327a5fdb5c6e49c2872b6c5e41d6db9c",
            "d76aa4e0c9864877b7e98ec8956d3a7e",
            "7a0df8bdcdc14776a6c660897f46137d",
            "445ac7c9d5a049bdb4e375506f16b3d9",
            "013d62e7881640f5979811e85269ba21"
          ]
        },
        "id": "tso164SzIFEd",
        "outputId": "3c98006a-dd30-4959-9c96-8c9f46bb2ef5"
      },
      "source": [
        "filtered_features_all_psg_dataset_and_y = create_map_of_subject_id_to_the_dataset_and_y_for_all_psg_status(subject_id_to_filtered_extracted_feature_df_map, map_subject_id_to_a_map_of_the_psg_status_to_session_id, map_subject_id_to_a_map_of_the_session_id_to_psg_status)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "ba8c2adbcdc8417da2428d9c3638ffdf",
              "version_minor": 0,
              "version_major": 2
            },
            "text/plain": [
              "HBox(children=(FloatProgress(value=0.0, max=31.0), HTML(value='')))"
            ]
          },
          "metadata": {
            "tags": []
          }
        },
        {
          "output_type": "stream",
          "text": [
            "\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ufTnoiNQOUuD"
      },
      "source": [
        "#### Classify the sleep states of the sessions for each subject"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 66,
          "referenced_widgets": [
            "b8979a45f03141a6ba4546ce9f15cf91",
            "e3db188ba93144e4a5afb67be9c6b18a",
            "7cc228a2a497422885ffedb5eb3201e5",
            "8766229068ab433ab05fac6b4fa24c56",
            "dfa7f218e0db4c66826d0a85bc71ae93",
            "fdafaa20d7fb488486fe472300b9149b",
            "0bd9bbbeae3e41e281f86106215902df",
            "2d69efe57d5f47f4b8017922bf3f3f02"
          ]
        },
        "id": "0FDFrW5AIFCI",
        "outputId": "c76923a8-21fa-456d-c619-5858b49ac638"
      },
      "source": [
        "optimise_params_bool = True\n",
        "classification_report_bool = True\n",
        "baseline_results, baseline_classification_report_dict = classify(filtered_features_all_psg_dataset_and_y, names_of_models_to_test, cv_settings, optimise_params_bool, classification_report_bool)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "b8979a45f03141a6ba4546ce9f15cf91",
              "version_minor": 0,
              "version_major": 2
            },
            "text/plain": [
              "HBox(children=(FloatProgress(value=0.0, max=31.0), HTML(value='')))"
            ]
          },
          "metadata": {
            "tags": []
          }
        },
        {
          "output_type": "stream",
          "text": [
            "\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "DbaJZjxAmLo7"
      },
      "source": [
        "#### Analyse these results"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "wJUQN7M8vjI5"
      },
      "source": [
        "##### Output the results of the score each classifier got for each subject"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "RgfbNo4Nmjf2"
      },
      "source": [
        "* 3 subjects are not included here because they were giving an error when creating the dataset and y"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 917
        },
        "id": "DcRQYHwRIPUe",
        "outputId": "d0967b57-45ca-4f25-db49-c4e22369e500"
      },
      "source": [
        "baseline_results"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>subject_id</th>\n",
              "      <th>SVC</th>\n",
              "      <th>Decision Tree</th>\n",
              "      <th>K-Neighbours</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>46343</td>\n",
              "      <td>0.524590</td>\n",
              "      <td>0.535519</td>\n",
              "      <td>0.420765</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>759667</td>\n",
              "      <td>0.554140</td>\n",
              "      <td>0.649682</td>\n",
              "      <td>0.503185</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>781756</td>\n",
              "      <td>0.589506</td>\n",
              "      <td>0.496914</td>\n",
              "      <td>0.561728</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>844359</td>\n",
              "      <td>0.548822</td>\n",
              "      <td>0.612795</td>\n",
              "      <td>0.535354</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>1066528</td>\n",
              "      <td>0.523962</td>\n",
              "      <td>0.549521</td>\n",
              "      <td>0.533546</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>5</th>\n",
              "      <td>1360686</td>\n",
              "      <td>0.620915</td>\n",
              "      <td>0.653595</td>\n",
              "      <td>0.637255</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>6</th>\n",
              "      <td>1449548</td>\n",
              "      <td>0.685714</td>\n",
              "      <td>0.698413</td>\n",
              "      <td>0.666667</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>7</th>\n",
              "      <td>1455390</td>\n",
              "      <td>0.705696</td>\n",
              "      <td>0.705696</td>\n",
              "      <td>0.715190</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>8</th>\n",
              "      <td>1818471</td>\n",
              "      <td>0.533123</td>\n",
              "      <td>0.564669</td>\n",
              "      <td>0.545741</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>9</th>\n",
              "      <td>2598705</td>\n",
              "      <td>0.609524</td>\n",
              "      <td>0.536508</td>\n",
              "      <td>0.523810</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>10</th>\n",
              "      <td>2638030</td>\n",
              "      <td>0.709265</td>\n",
              "      <td>0.619808</td>\n",
              "      <td>0.648562</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>11</th>\n",
              "      <td>3509524</td>\n",
              "      <td>0.630435</td>\n",
              "      <td>0.760870</td>\n",
              "      <td>0.681159</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>12</th>\n",
              "      <td>3997827</td>\n",
              "      <td>0.643533</td>\n",
              "      <td>0.738170</td>\n",
              "      <td>0.668770</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>13</th>\n",
              "      <td>4018081</td>\n",
              "      <td>0.606061</td>\n",
              "      <td>0.709091</td>\n",
              "      <td>0.606061</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>14</th>\n",
              "      <td>4314139</td>\n",
              "      <td>0.569182</td>\n",
              "      <td>0.490566</td>\n",
              "      <td>0.575472</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>15</th>\n",
              "      <td>4426783</td>\n",
              "      <td>0.638889</td>\n",
              "      <td>0.583333</td>\n",
              "      <td>0.620370</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>16</th>\n",
              "      <td>5132496</td>\n",
              "      <td>0.655844</td>\n",
              "      <td>0.512987</td>\n",
              "      <td>0.623377</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>17</th>\n",
              "      <td>5498603</td>\n",
              "      <td>0.573171</td>\n",
              "      <td>0.552846</td>\n",
              "      <td>0.573171</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>18</th>\n",
              "      <td>5797046</td>\n",
              "      <td>0.603226</td>\n",
              "      <td>0.596774</td>\n",
              "      <td>0.593548</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>19</th>\n",
              "      <td>6220552</td>\n",
              "      <td>0.574603</td>\n",
              "      <td>0.615873</td>\n",
              "      <td>0.603175</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>20</th>\n",
              "      <td>7749105</td>\n",
              "      <td>0.536585</td>\n",
              "      <td>0.878049</td>\n",
              "      <td>0.609756</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>21</th>\n",
              "      <td>8000685</td>\n",
              "      <td>0.589905</td>\n",
              "      <td>0.605678</td>\n",
              "      <td>0.596215</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>22</th>\n",
              "      <td>8173033</td>\n",
              "      <td>0.566456</td>\n",
              "      <td>0.610759</td>\n",
              "      <td>0.585443</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>23</th>\n",
              "      <td>8530312</td>\n",
              "      <td>0.624204</td>\n",
              "      <td>0.538217</td>\n",
              "      <td>0.627389</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>24</th>\n",
              "      <td>8686948</td>\n",
              "      <td>0.632911</td>\n",
              "      <td>0.598101</td>\n",
              "      <td>0.601266</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>25</th>\n",
              "      <td>8692923</td>\n",
              "      <td>0.579288</td>\n",
              "      <td>0.627832</td>\n",
              "      <td>0.521036</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>26</th>\n",
              "      <td>9106476</td>\n",
              "      <td>0.602524</td>\n",
              "      <td>0.589905</td>\n",
              "      <td>0.630915</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>27</th>\n",
              "      <td>9961348</td>\n",
              "      <td>0.605042</td>\n",
              "      <td>0.617647</td>\n",
              "      <td>0.613445</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "   subject_id       SVC  Decision Tree  K-Neighbours\n",
              "0       46343  0.524590       0.535519      0.420765\n",
              "1      759667  0.554140       0.649682      0.503185\n",
              "2      781756  0.589506       0.496914      0.561728\n",
              "3      844359  0.548822       0.612795      0.535354\n",
              "4     1066528  0.523962       0.549521      0.533546\n",
              "5     1360686  0.620915       0.653595      0.637255\n",
              "6     1449548  0.685714       0.698413      0.666667\n",
              "7     1455390  0.705696       0.705696      0.715190\n",
              "8     1818471  0.533123       0.564669      0.545741\n",
              "9     2598705  0.609524       0.536508      0.523810\n",
              "10    2638030  0.709265       0.619808      0.648562\n",
              "11    3509524  0.630435       0.760870      0.681159\n",
              "12    3997827  0.643533       0.738170      0.668770\n",
              "13    4018081  0.606061       0.709091      0.606061\n",
              "14    4314139  0.569182       0.490566      0.575472\n",
              "15    4426783  0.638889       0.583333      0.620370\n",
              "16    5132496  0.655844       0.512987      0.623377\n",
              "17    5498603  0.573171       0.552846      0.573171\n",
              "18    5797046  0.603226       0.596774      0.593548\n",
              "19    6220552  0.574603       0.615873      0.603175\n",
              "20    7749105  0.536585       0.878049      0.609756\n",
              "21    8000685  0.589905       0.605678      0.596215\n",
              "22    8173033  0.566456       0.610759      0.585443\n",
              "23    8530312  0.624204       0.538217      0.627389\n",
              "24    8686948  0.632911       0.598101      0.601266\n",
              "25    8692923  0.579288       0.627832      0.521036\n",
              "26    9106476  0.602524       0.589905      0.630915\n",
              "27    9961348  0.605042       0.617647      0.613445"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 41
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "i3csfAX3v1UU"
      },
      "source": [
        "##### Describe this dataframe better with summary statistics"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 297
        },
        "id": "hXgFb6uvIPUf",
        "outputId": "7e10f27a-bdfa-474e-a614-03e890d1f7d9"
      },
      "source": [
        "baseline_results.describe()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>SVC</th>\n",
              "      <th>Decision Tree</th>\n",
              "      <th>K-Neighbours</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>count</th>\n",
              "      <td>28.000000</td>\n",
              "      <td>28.000000</td>\n",
              "      <td>28.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>mean</th>\n",
              "      <td>0.601326</td>\n",
              "      <td>0.616065</td>\n",
              "      <td>0.593656</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>std</th>\n",
              "      <td>0.050517</td>\n",
              "      <td>0.086878</td>\n",
              "      <td>0.061696</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>min</th>\n",
              "      <td>0.523962</td>\n",
              "      <td>0.490566</td>\n",
              "      <td>0.420765</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>25%</th>\n",
              "      <td>0.568501</td>\n",
              "      <td>0.552014</td>\n",
              "      <td>0.557732</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>50%</th>\n",
              "      <td>0.602875</td>\n",
              "      <td>0.608219</td>\n",
              "      <td>0.602220</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>75%</th>\n",
              "      <td>0.631054</td>\n",
              "      <td>0.650660</td>\n",
              "      <td>0.628270</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>max</th>\n",
              "      <td>0.709265</td>\n",
              "      <td>0.878049</td>\n",
              "      <td>0.715190</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "             SVC  Decision Tree  K-Neighbours\n",
              "count  28.000000      28.000000     28.000000\n",
              "mean    0.601326       0.616065      0.593656\n",
              "std     0.050517       0.086878      0.061696\n",
              "min     0.523962       0.490566      0.420765\n",
              "25%     0.568501       0.552014      0.557732\n",
              "50%     0.602875       0.608219      0.602220\n",
              "75%     0.631054       0.650660      0.628270\n",
              "max     0.709265       0.878049      0.715190"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 38
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "tOOqHUCTv991"
      },
      "source": [
        "##### Plot these summary statistics to better visualise them"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ireKt_rmnIsM"
      },
      "source": [
        "A box plot is perfect for this situation as it gives us a comprehenensive comparison between the different models we looked at"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 286
        },
        "id": "aYWvo-tlIPUh",
        "outputId": "0969af84-7acd-4b9a-b83f-374329fa85c6"
      },
      "source": [
        "baseline_results.boxplot()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<matplotlib.axes._subplots.AxesSubplot at 0x7fe6ee455a90>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 39
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD8CAYAAABn919SAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAU7UlEQVR4nO3df5BdZ33f8fcHCQdigUMwqIxtsJMqiYwAA4pdGqWRMD9MyNhNTVNvMiSmCkrasUoJZLAjxhinmphAfkFMqILAhJmuMaalKlYtaNBClEAru9gGeWtXyFDLaYcfBoclBCPx7R/3iFwvK+3d3Svv3mffr5mdveec5577vffs/exznnPuuakqJEmj7zGLXYAkaTgMdElqhIEuSY0w0CWpEQa6JDXCQJekRgwU6EkuSnJPkoNJrpxh+TOS/HmSu5JMJDlz+KVKkk4ks52HnmQFcC/wYuAwsB8Yq6q7+9p8EPhIVb0vyQuBV1XVK09e2ZKk6QbpoZ8PHKyqQ1X1MHAjcMm0NucCH+9u751huSTpJFs5QJszgPv7pg8DF0xrcyfwz4A/An4eeEKSJ1fVV4+30tNPP73OPvvsuVU7Qr75zW9y6qmnLnYZmge33WhrffvdfvvtX6mqp8y0bJBAH8TrgT9OcjnwSeAB4Oj0Rkm2AFsAVq9ezdve9rYhPfzSMzU1xapVqxa7DM2D2260tb79Nm3a9MXjLRsk0B8AzuqbPrOb9z1V9df0eugkWQVcWlVfn76iqtoB7ABYv359bdy4cYCHH00TExO0/Pxa5rYbbct5+w0yhr4fWJPknCSnAJcBu/obJDk9ybF1XQW8Z7hlSpJmM2ugV9UR4ApgDzAJ3FRVB5Jcm+TirtlG4J4k9wKrge0nqV5J0nEMNIZeVbuB3dPmXd13+2bg5uGWJkmaCz8pKkmNMNClzvj4OOvWrePCCy9k3bp1jI+PL3ZJ0pwM67RFaaSNj4+zbds2du7cydGjR1mxYgWbN28GYGxsbJGrkwZjD10Ctm/fzs6dO9m0aRMrV65k06ZN7Ny5k+3bPb6v0WGgS8Dk5CQbNmx4xLwNGzYwOTm5SBVJc2egS8DatWvZt2/fI+bt27ePtWvXLlJF0twZ6BKwbds2Nm/ezN69ezly5Ah79+5l8+bNbNu2bbFLkwbmQVGJvz/wuXXrViYnJ1m7di3bt2/3gKhGioEudcbGxhgbG1vW1wLRaHPIRZIaYaBLUiMMdElqhIEuSY0w0CWpEQa6JDXCQJekRhjoktQIA12SGmGgS1IjDHRJaoSBLkmNMNAlqREGuiQ1wkCXpEYY6JLUCANdkhphoEtSIwx0SWqEgS5JjTDQJakRBrokNcJAl6RGGOiS1AgDXZIaMVCgJ7koyT1JDia5coblT0+yN8lnktyV5GeHX6ok6URmDfQkK4DrgZcB5wJjSc6d1uyNwE1V9VzgMuCdwy5UknRig/TQzwcOVtWhqnoYuBG4ZFqbAp7Y3T4N+OvhlShJGsTKAdqcAdzfN30YuGBam2uAjybZCpwKvGgo1UmSBjZIoA9iDLihqn4vyQuA9ydZV1Xf7W+UZAuwBWD16tVMTEwM6eGXnqmpqaafX8vcdqNtOW+/QQL9AeCsvukzu3n9NgMXAVTVp5I8Djgd+FJ/o6raAewAWL9+fW3cuHF+VY+AiYkJWn5+LXPbjbblvP0GGUPfD6xJck6SU+gd9Nw1rc3/AS4ESLIWeBzw5WEWKkk6sVkDvaqOAFcAe4BJemezHEhybZKLu2avA16d5E5gHLi8qupkFS1J+n4DjaFX1W5g97R5V/fdvhv4qeGWJkmaCz8pKkmNMNAlqREGuiQ1wkCXpEYY6JLUCANdkhphoEtSIwx0SWqEgS5JjTDQJakRBrokNcJAl6RGDOsLLqSRkWQo6/GColpq7KFr2amqE/484w0fmbWNYa6lyECXpEYY6JLUCANdkhphoEtSIwx0SWqEgS5JjTDQJakRBrokNcJAl6RGGOiS1AgDXZIaYaBLUiMMdElqhIEuSY0w0CWpEQa6JDXCQJekRhjoktQIv1NU0kgZxnfCtvoVgvbQJY2UYXwnbKvsoc+TvQRJS81APfQkFyW5J8nBJFfOsPwPktzR/dyb5OvDL3VpsZcgaamZtYeeZAVwPfBi4DCwP8muqrr7WJuqem1f+63Ac09CrZKkExikh34+cLCqDlXVw8CNwCUnaD8GjA+jOEnS4AYZQz8DuL9v+jBwwUwNkzwDOAf4+HGWbwG2AKxevZqJiYm51DpyWn9+LXPbjbbluv2GfVD0MuDmqjo608Kq2gHsAFi/fn1t3LhxyA+/hNx6C00/v5a57UbbMt5+gwy5PACc1Td9ZjdvJpfhcIskLYpBAn0/sCbJOUlOoRfau6Y3SvITwJOATw23REnSIGYN9Ko6AlwB7AEmgZuq6kCSa5Nc3Nf0MuDG8nw8SVoUA42hV9VuYPe0eVdPm75meGVJkubKj/5LUiMMdElqhIEuSY0w0CWpEQa6JDXCQJekRhjoktQIA12SGmGgS1IjDHRJaoSBLkmNMNAlqREGuiQ1wkCXpEYY6JLUCANdkhox7C+Jlhbdc978UR761ncWtI6zr7xlQfc/7fGP5c43vWRB65DmykBXcx761nf4wnUvn/f9JyYmFvyt8Qv9hyDNh0MuktQIA12SGmGgS1IjDHRJaoSBLkmNMNAlqRGetjiDYZzHDJ7LLOnRZaDPYKHnMYPnMkvzYWdqYQx0SUuGnamFcQxdkhphoEtSIwx0SWqEgS5JjTDQJakRBrokNcJAl6RGDHQeepKLgD8CVgDvrqrrZmjzC8A1QAF3VtUvDrFOaWBPWHslz3rflQtbyfsWWgPAws6nluZq1kBPsgK4HngxcBjYn2RXVd3d12YNcBXwU1X1tSRPPVkFS7P5xuR1fmORlqVBeujnAwer6hBAkhuBS4C7+9q8Gri+qr4GUFVfGnahj6ah9PDAXp40R773FmaQQD8DuL9v+jBwwbQ2PwaQ5C/pDctcU1W3Tl9Rki3AFoDVq1czMTExj5JPvm9MXscNF526oHVMTU2xatWqBa3j8lu/uWRfo6VuIa/b1NTUUF53t93c+d5bmGFdy2UlsAbYCJwJfDLJs6rq6/2NqmoHsANg/fr1tdDd2pPm1lsWvMs9jN32YdSxLC3wdXPbLSLfewsyyFkuDwBn9U2f2c3rdxjYVVXfqar7gHvpBbwk6VEySKDvB9YkOSfJKcBlwK5pbT5Mr3dOktPpDcEcGmKdkqRZzBroVXUEuALYA0wCN1XVgSTXJrm4a7YH+GqSu4G9wG9W1VdPVtGSpO830Bh6Ve0Gdk+bd3Xf7QJ+o/uRJC0CPykqSY0w0CWpEQa6JDXCQJekRhjoktQIA12SGmGgS1IjDHRJasSwLs7VnKFcz/rWha3jtMc/duE1SFo2DPQZLOTLEY45+8pbhrIeSRqUQy6S1AgDXZIaYaBLUiMcQ5e0pHhCwvwZ6JKWDE9IWBgDXU1acC9vmfbwNNoMdDVnob2z5dzD02jzoKgkNcJAl6RGGOiS1AgDXZIaYaBLUiMMdElqhIEuSY0w0CWpEQa6JDXCQJekRvjR/3lKMnubt5x4eVUNqRpJsoc+b1V1wp+9e/fO2kaShslAl6RGGOiS1AgDXZIaYaBLUiMMdElqxECBnuSiJPckOZjkyhmWX57ky0nu6H5+dfilSpJOZNbz0JOsAK4HXgwcBvYn2VVVd09r+oGquuIk1ChJGsAgHyw6HzhYVYcAktwIXAJMD3RJOun8UN/xDTLkcgZwf9/04W7edJcmuSvJzUnOGkp1kjSNH+o7vmF99P+/AONV9e0kvwa8D3jh9EZJtgBbAFavXs3ExMSQHn7pmZqaavr5tc5tN7qW83tvkEB/AOjvcZ/Zzfueqvpq3+S7gd+daUVVtQPYAbB+/frauHHjXGodKRMTE7T8/Jp26y1uuxG2nN97gwy57AfWJDknySnAZcCu/gZJntY3eTEwObwSJUmDmLWHXlVHklwB7AFWAO+pqgNJrgVuq6pdwL9JcjFwBHgQuPwk1ixJmsFAY+hVtRvYPW3e1X23rwKuGm5p0skxjLMkoN0zJTS6/KSolp1hnCVhmGspMtAlqREGuiQ1wkCXpEYY6JLUCANdkhphoEtSIwx0SWqEgS5JjTDQJakRBrokNcJAl6RGGOiS1AgDXZIaYaBLUiMMdElqhIEuSY0w0CWpEQa6JDXCQJekRhjoktQIA12SGmGgS1IjDHRJaoSBLkmNMNAlqREGuqQmjI+Ps27dOi688ELWrVvH+Pj4Ypf0qFu52AVI0kKNj4+zbds2du7cydGjR1mxYgWbN28GYGxsbJGre/TYQ5c08rZv387OnTvZtGkTK1euZNOmTezcuZPt27cvdmmPKgNd0sibnJxkw4YNj5i3YcMGJicnF6mixWGgSxp5a9euZd++fY+Yt2/fPtauXbtIFS0OA13SyNu2bRubN29m7969HDlyhL1797J582a2bdu22KU9qjwoKmnkHTvwuXXrViYnJ1m7di3bt29fVgdEwUCX1IixsTHGxsaYmJhg48aNi13OohhoyCXJRUnuSXIwyZUnaHdpkkqyfnglSpIGMWugJ1kBXA+8DDgXGEty7gztngC8Bvjvwy5SkjS7QXro5wMHq+pQVT0M3AhcMkO73wbeAvzdEOuTJA1okEA/A7i/b/pwN+97kjwPOKuqbhlibZKkOVjwQdEkjwF+H7h8gLZbgC0Aq1evZmJiYqEPv2RNTU01/fxa5rYbbct5+6WqTtwgeQFwTVW9tJu+CqCqfqebPg34PDDV3eUfAA8CF1fVbSdY75eBLy70CSxhpwNfWewiNC9uu9HW+vZ7RlU9ZaYFgwT6SuBe4ELgAWA/8ItVdeA47SeA158ozJeDJLdVlWf7jCC33Whbzttv1jH0qjoCXAHsASaBm6rqQJJrk1x8sguUJA1m1h665mc59xJGndtutC3n7ee1XE6eHYtdgObNbTfalu32s4cuSY2why5JjTDQ5yHJtiQHktyV5I4kb0ryO9PanJdksru9Ksm/T/L5JLcnmUhyweJUv/QlOdq9rgeS3Jnkdd3nHeazrmuTvOgEy389yS/Pv1pI8qyu3juSPJjkvu72f1vIekdVkqm+2z+b5N4kz5jW5gtJPtQ3/YokN8yy3vVJ3j5Lm7OTfO44yyZav86UV1uco+68/J8DnldV305yOr1r3NwAXNXX9DLg2LfUvhu4D1hTVd9Nck53H83sW1V1HkCSpwL/AXgi8Ka5rqiqrp5l+bvmVeEj1/FZ4Fi9NwAfqaqb+9skWdmdMbZsJLkQeDvw0qqa6TMnz09yblXdPcj6ulOhl9Tp0ElCb+j6u4tdC9hDn4+nAV+pqm8DVNVXquqTwNem9bp/ARhP8qPABcAbj230qrrPyyQMpqq+RO/TxVekZ0WStybZ3+0h/dqxtknekOSzXa/+um7eDUle0d2+Lsnd3f3e1s27Jsnru9vnJfl0t/w/JXlSN38iyVuS/I+ut/nTg9Te3e8Pk9wGvCbJ85N8ottL25PkaV27H01yazf/L5L8xBBfwkWR5J8Afwr8XFV9/jjNfg/4vm+gSHJqkvd0r/dnklzSzd+Y5CPd7ack+Vi3F/fuJF/sOlcAK5L8abfso0ke37f6V3Z7T59Lcn63rh9O8uFuu386ybO7+d/72+imP9ftAZyd3tVn/wz4HHBW93f2ue7v77ULe/Xmz0Cfu4/S24D3Jnlnkp/p5o/T65WT5B8BD1bV/waeCdxRVUcXp9zRV1WHgBXAU4HNwENV9ZPATwKvTnJOkpfRu2jcBVX1HOB3+9eR5MnAzwPPrKpnA/9uhof6M+AN3fLP8sg9gpVVdT7wb5nbnsIp3Sl0bwfeAbyiqp4PvAc49g3GO4Ct3fzXA++cw/qXoh8APgz806r6XydodxPwvCT/cNr8bcDHu9d7E/DWJKdOa/Omrs0zgZuBp/ctWwNc3y37OnBp37If7Pb+/jW9bQDwZuAz3Xb/LXp/B7NZA7yze4zTgTOqal1VPQt47wD3PykccpmjqppK8nzgp+n9sX0gvWvEfwD4qySv45HDLRqulwDPPtbrBk6j9+Z6EfDeqvpbgKp6cNr9HqJ3JdCdXS/vI/0L07uExQ9V1Se6We8DPtjX5D92v28Hzp5DvR/ofv84sA74WG8vnRXA/02yCvjHwAe7+dALxFH2HeCv6P3zfc0J2h0F3kpvqPK/9s1/CXBxX+/4cTwysAE20PsHTVXdmuRrfcvuq6o7utvTt9d4d59PJnlikh/q1nVpN//jSZ6c5ImzPMcvVtWnu9uHgB9J8g7gFnqdvkVhoM9D19ueACaSfBb4laq6Icl9wM/Q++N4Qdf8APCcJCvspc9Pkh+h9+b/EhB6vdk909q89ETrqKoj3S72hcAr6H36+YVzKOPb3e+jzO19881jJQIHquoF/Qu74Pj6sWMGjfguvSHHP0/yW/Quq317t2zXtOMa76cX6P0HMgNcWlX39K80yeoBH//bfbePAv1DLtPP0z7RedtHeOQoxuP6bh/brlTV15I8B3gp8Ov0nvu/HLDWoXLIZY6S/HiSNX2zzuPvLzI2DvwBcKiqDgN044e3AW/uDqAcOxL/8kex7JGV5CnAu4A/rt6HJvYA/yrJY7vlP9btjn8MeFWSH+zm//C09awCTquq3cBrgef0L6+qh+gdBzk2Pv5K4BMMzz3AU9I7qE6SxyZ5ZlX9DXBfkn/ezU8XDiOt21N6OfBLwOVVdV73c/W0dt+h957pH3feA2zte788d4aH+Et6wUmSlwBPGrC0f9HdZwO9obuHgL/o6iTJRnrHyP4G+ALwvG7+84BzZlphN3b/mKr6EPDGY/dZDPbQ524V8I5uV+0IcJDuksD0dtHfDmyddp9fpXcA6GCSb9G7EtxvPjrljqTHJ7kDeCy91/j99C7RDL0zhs4G/mf3hv8yvbHaW5OcB9yW5GFgN73x0GOeAPznJI+j1wP8jRke91eAd3X/FA4BrxrWE6qqh7thord3wzsrgT+ktwf3S8CfJHlj95xvBO4c1mMvlqp6MMlFwCeTfLmqdh2n6U56QXjMb9N7be5K73TV++idWdbvzfROOngl8Cng/wHfoPf+PJG/S/IZeq/zsV70NcB7ktwF/C29vwOADwG/nOQAvW9iu/c46zwDeG/+/tTaq47T7qTzk6KSRk6SHwCOdkNpLwD+pLFhq3mxhy5pFD0duKnrFT8MvHqR61kS7KFLUiM8KCpJjTDQJakRBrokNcJAl6RGGOiS1AgDXZIa8f8Bo1Aoy5VjTEAAAAAASUVORK5CYII=\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Vo1BkLOPIE55"
      },
      "source": [
        "The above plot tells us a number of important things about the results of these classifiers. Firstly, we can see that the interquartile range of these data is biggest in the Decision Tree classifier, meaning that the classification accuracy of the middle 50% of data points is more spread out than that of the  the other classifiers. We can also see that the boxplots minimum and maximum values for each of these classifiers gave a greater range in the Decision Tree Classifier than in the others. This means that the accuracy of this model in particular is more inconsistant than the others we tested.\n",
        "\n",
        "Outside of the interquartile range, we can also see the average classification scores of each of these models. The green line shows that the Decision Tree Classifier gives the best average classification score but this is not by much. The SVC and K-Neighbours classifier have a similar average accuracy, however, the K-Neighbours classifier's Q3 value is lower than that of the SVC model and it also contains an outlier much below the rest of the data. This suggests to me that K-Neighbours is possibly not a good fit for the data as unlike the Decision Tree model, its outliers are on the poorer side and even the bulk of its classifications do not beat those of the SVC model.\n",
        "\n",
        "From all of this, we concluded that the Decision Tree was the best model and it could be used as a baseline of our hypothesised optimal classifier. Using this, we also wanted to test the results in the case where we didn't make the decisions that we did to arrive at our hypothsised optimal model."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "t6tOYFzL_otC"
      },
      "source": [
        "Our next step involves experimenting with the alternate approach to the one we took through the course of 3 experiments. These experaments are to test the following scenario:\n",
        "\n",
        "1.   Test the results of not filtering the extracted features Vs the baseline filtered features\n",
        "2.   Test the results of not including the step count in the data Vs the baseline data with the step count\n",
        "3.   Test the results of ot optimising the parameters Vs the basline optimised parameters\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3kQSykLHy8pN"
      },
      "source": [
        "---\n",
        "\n",
        "## Experiment 1 - Classify the sleep state using filtered extracted features Vs the unfiltered extracted features\n",
        "\n",
        "\n",
        "---"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "T9JhPUTlpJ-0"
      },
      "source": [
        "This is our first experiment, where we test if it was a good decision to filter the extracted features. Here, we will follow the same approach as we did with our hypothesis results and then we will compare the results to make a conclusion."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "vGJcfRPY5Fku"
      },
      "source": [
        "#### Get the dataset and y for the unfiltered data"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 66,
          "referenced_widgets": [
            "b0179299f6354a3cac4de319bdb22e02",
            "8b15f1198d514dfcbc8dfc8e3abc26b3",
            "f6fd1ad5e11e48a68f4d9522dacd6a3f",
            "29d030e6f5cd43efadcc414c5b547bec",
            "b6db9171ec884784b252b32718f4e921",
            "89847cfeea5f4e67bd7bc5f956183347",
            "4d74748538be45f79a1a7fa8591cf462",
            "7f15bac0ebbe431badace764a3cb8948"
          ]
        },
        "id": "JB9emQ0h5CnY",
        "outputId": "4aa29ed1-9be0-4977-cf12-1960473ede92"
      },
      "source": [
        "unfiltered_features_all_psg_dataset_and_y = create_map_of_subject_id_to_the_dataset_and_y_for_all_psg_status(subject_id_to_unfiltered_extracted_feature_df_map, map_subject_id_to_a_map_of_the_psg_status_to_session_id, map_subject_id_to_a_map_of_the_session_id_to_psg_status)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "b0179299f6354a3cac4de319bdb22e02",
              "version_minor": 0,
              "version_major": 2
            },
            "text/plain": [
              "HBox(children=(FloatProgress(value=0.0, max=31.0), HTML(value='')))"
            ]
          },
          "metadata": {
            "tags": []
          }
        },
        {
          "output_type": "stream",
          "text": [
            "\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "tftwwd_su2G-"
      },
      "source": [
        "#### Classify the dataset and y for the unfiltered data"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "nrYRekpKjPmQ"
      },
      "source": [
        "** We could not run this experiment due to the below error and didn't have enough time to fix it so this experiment cannot be run **"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 66,
          "referenced_widgets": [
            "24562794d30748e4b60a39cb0eab088c",
            "102a953fd7604e82a00db467557e2d0f",
            "81bd274a196c4afc868a98ddb8506ccd",
            "b02ae3089e634530be377a64b1572be5",
            "77fcddcc920f46fa8ea846aaabccaa4d",
            "96dc7eff17c84ec8978d41c24b1f5004",
            "5955591fec2e40018d8619688bd58429",
            "6a8b3729e92a40fc89b1c6bf249abc44"
          ]
        },
        "id": "57Nq4eZl6Mqf",
        "outputId": "1be9a2da-4df1-4e4b-99a7-2b7d1fc6796f"
      },
      "source": [
        "optimise_params_bool = True\n",
        "classification_report_bool = False\n",
        "unfiltered_results = classify(unfiltered_features_all_psg_dataset_and_y, names_of_models_to_test, cv_settings, optimise_params_bool, classification_report_bool)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "24562794d30748e4b60a39cb0eab088c",
              "version_minor": 0,
              "version_major": 2
            },
            "text/plain": [
              "HBox(children=(FloatProgress(value=0.0, max=31.0), HTML(value='')))"
            ]
          },
          "metadata": {
            "tags": []
          }
        },
        {
          "output_type": "stream",
          "text": [
            "\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "IMPB66UKqKOS"
      },
      "source": [
        "### Analyse the results"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-6yNFchiwGGQ"
      },
      "source": [
        "##### Output the results of the score each classifier got for each subject"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 917
        },
        "id": "37cDiH9hKwAf",
        "outputId": "51523407-39d0-4492-cdfc-4d11f4829ab6"
      },
      "source": [
        "unfiltered_results"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>subject_id</th>\n",
              "      <th>SVC</th>\n",
              "      <th>Decision Tree</th>\n",
              "      <th>K-Neighbours</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>46343</td>\n",
              "      <td>0.453552</td>\n",
              "      <td>0.612022</td>\n",
              "      <td>0.387978</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>759667</td>\n",
              "      <td>0.522293</td>\n",
              "      <td>0.707006</td>\n",
              "      <td>0.541401</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>781756</td>\n",
              "      <td>0.506173</td>\n",
              "      <td>0.506173</td>\n",
              "      <td>0.561728</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>844359</td>\n",
              "      <td>0.508418</td>\n",
              "      <td>0.535354</td>\n",
              "      <td>0.518519</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>1066528</td>\n",
              "      <td>0.488818</td>\n",
              "      <td>0.498403</td>\n",
              "      <td>0.485623</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>5</th>\n",
              "      <td>1360686</td>\n",
              "      <td>0.529412</td>\n",
              "      <td>0.555556</td>\n",
              "      <td>0.568627</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>6</th>\n",
              "      <td>1449548</td>\n",
              "      <td>0.650794</td>\n",
              "      <td>0.590476</td>\n",
              "      <td>0.653968</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>7</th>\n",
              "      <td>1455390</td>\n",
              "      <td>0.702532</td>\n",
              "      <td>0.686709</td>\n",
              "      <td>0.693038</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>8</th>\n",
              "      <td>1818471</td>\n",
              "      <td>0.533123</td>\n",
              "      <td>0.586751</td>\n",
              "      <td>0.548896</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>9</th>\n",
              "      <td>2598705</td>\n",
              "      <td>0.584127</td>\n",
              "      <td>0.501587</td>\n",
              "      <td>0.539683</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>10</th>\n",
              "      <td>2638030</td>\n",
              "      <td>0.584665</td>\n",
              "      <td>0.575080</td>\n",
              "      <td>0.546326</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>11</th>\n",
              "      <td>3509524</td>\n",
              "      <td>0.572464</td>\n",
              "      <td>0.710145</td>\n",
              "      <td>0.630435</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>12</th>\n",
              "      <td>3997827</td>\n",
              "      <td>0.618297</td>\n",
              "      <td>0.690852</td>\n",
              "      <td>0.608833</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>13</th>\n",
              "      <td>4018081</td>\n",
              "      <td>0.490909</td>\n",
              "      <td>0.696970</td>\n",
              "      <td>0.509091</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>14</th>\n",
              "      <td>4314139</td>\n",
              "      <td>0.522013</td>\n",
              "      <td>0.462264</td>\n",
              "      <td>0.525157</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>15</th>\n",
              "      <td>4426783</td>\n",
              "      <td>0.567901</td>\n",
              "      <td>0.533951</td>\n",
              "      <td>0.530864</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>16</th>\n",
              "      <td>5132496</td>\n",
              "      <td>0.577922</td>\n",
              "      <td>0.571429</td>\n",
              "      <td>0.584416</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>17</th>\n",
              "      <td>5498603</td>\n",
              "      <td>0.524390</td>\n",
              "      <td>0.536585</td>\n",
              "      <td>0.491870</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>18</th>\n",
              "      <td>5797046</td>\n",
              "      <td>0.558065</td>\n",
              "      <td>0.551613</td>\n",
              "      <td>0.551613</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>19</th>\n",
              "      <td>6220552</td>\n",
              "      <td>0.504762</td>\n",
              "      <td>0.558730</td>\n",
              "      <td>0.552381</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>20</th>\n",
              "      <td>7749105</td>\n",
              "      <td>0.390244</td>\n",
              "      <td>0.512195</td>\n",
              "      <td>0.365854</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>21</th>\n",
              "      <td>8000685</td>\n",
              "      <td>0.586751</td>\n",
              "      <td>0.536278</td>\n",
              "      <td>0.570978</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>22</th>\n",
              "      <td>8173033</td>\n",
              "      <td>0.528481</td>\n",
              "      <td>0.582278</td>\n",
              "      <td>0.496835</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>23</th>\n",
              "      <td>8530312</td>\n",
              "      <td>0.531847</td>\n",
              "      <td>0.490446</td>\n",
              "      <td>0.541401</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>24</th>\n",
              "      <td>8686948</td>\n",
              "      <td>0.607595</td>\n",
              "      <td>0.541139</td>\n",
              "      <td>0.579114</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>25</th>\n",
              "      <td>8692923</td>\n",
              "      <td>0.495146</td>\n",
              "      <td>0.556634</td>\n",
              "      <td>0.472492</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>26</th>\n",
              "      <td>9106476</td>\n",
              "      <td>0.545741</td>\n",
              "      <td>0.561514</td>\n",
              "      <td>0.558360</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>27</th>\n",
              "      <td>9961348</td>\n",
              "      <td>0.596639</td>\n",
              "      <td>0.588235</td>\n",
              "      <td>0.563025</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "   subject_id       SVC  Decision Tree  K-Neighbours\n",
              "0       46343  0.453552       0.612022      0.387978\n",
              "1      759667  0.522293       0.707006      0.541401\n",
              "2      781756  0.506173       0.506173      0.561728\n",
              "3      844359  0.508418       0.535354      0.518519\n",
              "4     1066528  0.488818       0.498403      0.485623\n",
              "5     1360686  0.529412       0.555556      0.568627\n",
              "6     1449548  0.650794       0.590476      0.653968\n",
              "7     1455390  0.702532       0.686709      0.693038\n",
              "8     1818471  0.533123       0.586751      0.548896\n",
              "9     2598705  0.584127       0.501587      0.539683\n",
              "10    2638030  0.584665       0.575080      0.546326\n",
              "11    3509524  0.572464       0.710145      0.630435\n",
              "12    3997827  0.618297       0.690852      0.608833\n",
              "13    4018081  0.490909       0.696970      0.509091\n",
              "14    4314139  0.522013       0.462264      0.525157\n",
              "15    4426783  0.567901       0.533951      0.530864\n",
              "16    5132496  0.577922       0.571429      0.584416\n",
              "17    5498603  0.524390       0.536585      0.491870\n",
              "18    5797046  0.558065       0.551613      0.551613\n",
              "19    6220552  0.504762       0.558730      0.552381\n",
              "20    7749105  0.390244       0.512195      0.365854\n",
              "21    8000685  0.586751       0.536278      0.570978\n",
              "22    8173033  0.528481       0.582278      0.496835\n",
              "23    8530312  0.531847       0.490446      0.541401\n",
              "24    8686948  0.607595       0.541139      0.579114\n",
              "25    8692923  0.495146       0.556634      0.472492\n",
              "26    9106476  0.545741       0.561514      0.558360\n",
              "27    9961348  0.596639       0.588235      0.563025"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 154
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Cb8btmSrwGrW"
      },
      "source": [
        "##### Describe this dataframe better with summary statistics"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 297
        },
        "id": "r9QI7d9-K1vd",
        "outputId": "3fed3425-d74d-41a8-8c76-b234a5179ca5"
      },
      "source": [
        "unfiltered_results.describe()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>SVC</th>\n",
              "      <th>Decision Tree</th>\n",
              "      <th>K-Neighbours</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>count</th>\n",
              "      <td>28.000000</td>\n",
              "      <td>28.000000</td>\n",
              "      <td>28.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>mean</th>\n",
              "      <td>0.545824</td>\n",
              "      <td>0.572728</td>\n",
              "      <td>0.542089</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>std</th>\n",
              "      <td>0.061756</td>\n",
              "      <td>0.068636</td>\n",
              "      <td>0.067519</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>min</th>\n",
              "      <td>0.390244</td>\n",
              "      <td>0.462264</td>\n",
              "      <td>0.365854</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>25%</th>\n",
              "      <td>0.507856</td>\n",
              "      <td>0.535003</td>\n",
              "      <td>0.516162</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>50%</th>\n",
              "      <td>0.532485</td>\n",
              "      <td>0.557682</td>\n",
              "      <td>0.547611</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>75%</th>\n",
              "      <td>0.584261</td>\n",
              "      <td>0.588796</td>\n",
              "      <td>0.569215</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>max</th>\n",
              "      <td>0.702532</td>\n",
              "      <td>0.710145</td>\n",
              "      <td>0.693038</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "             SVC  Decision Tree  K-Neighbours\n",
              "count  28.000000      28.000000     28.000000\n",
              "mean    0.545824       0.572728      0.542089\n",
              "std     0.061756       0.068636      0.067519\n",
              "min     0.390244       0.462264      0.365854\n",
              "25%     0.507856       0.535003      0.516162\n",
              "50%     0.532485       0.557682      0.547611\n",
              "75%     0.584261       0.588796      0.569215\n",
              "max     0.702532       0.710145      0.693038"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 155
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "MTXxNPwSdb09"
      },
      "source": [
        "From this, we see that the minimum accuracy score for the SVC and K-Neighbours classifiers are very low at only 39% and 36% respectively. We also see that the average mean score across the clasifiers is below that of the baseline.\n",
        "The Decision Tree classifier also comes out on top as the best classifier here."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zBayy8bzwHVR"
      },
      "source": [
        "##### Plot these summary statistics to better visualise them"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 282
        },
        "id": "_dMkBH_iFkVg",
        "outputId": "efe3bd97-ff11-4d8b-ecd0-8358a206c49b"
      },
      "source": [
        "unfiltered_results.boxplot()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<matplotlib.axes._subplots.AxesSubplot at 0x7fe6f4a222b0>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 156
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD4CAYAAADiry33AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAccUlEQVR4nO3dcXhddZ3n8feH1KIiSrEYoGlpcMtMSFCgmbKO1UmmA9R1nuI+sG5TZegQKe7ajqujDzBhQcvkEYZdZ0aLQodWnHmepjjDymalUjskV62sTsOC0jYLllakxX1UijhFbGn57h/npNxektzT5Da39+Tzep775J7f+Z1zv+ee3O/9nd8553cVEZiZWX6dUO0AzMzs2HKiNzPLOSd6M7Occ6I3M8s5J3ozs5ybUu0ASk2fPj1mz55d7TCOmRdffJGTTjqp2mHYGHn/1a6877tHHnnklxFx2nDzjrtEP3v2bAYGBqodxjFTKBRoa2urdhg2Rt5/tSvv+07S0yPNc9eNmVnOOdGbmeWcE72ZWc450ZuZ5ZwTvZlZzjnRm2XQ09NDS0sLCxYsoKWlhZ6enmqHZJbZcXd5pdnxpqenh66uLtasWcOhQ4eoq6ujs7MTgI6OjipHZ1aeE71ZGd3d3SxZsoQVK1YwODhIU1MTS5Ysobu724neaoITvVkZ27dv58UXX2Tt2rWHW/RXX301Tz894v0pZscV99GblTF16lRWrFhBe3s7U6ZMob29nRUrVjB16tRqh2aWiRP9BPHJvNp14MABVq1aRX9/PwcPHqS/v59Vq1Zx4MCBaodmlkmmrhtJC4G/BeqAuyPi1pL5fw20p5NvBN4WEaek864Cbkzn/WVEfLUSgdcSn8yrbeeeey4f+MAHXtNHf//991c7NLNsImLUB0lyfwo4G5gK/BA4d5T6K4C16fNTgZ3p32np82mjvd7cuXMjb5qbm6Ovry8iIvr7+yMioq+vL5qbm6sYlWW1bt26aGxsjL6+vti0aVP09fVFY2NjrFu3rtqh2VEY+uzlFTAQI+TVLC36ecCOiNgJIGk9cBmwfYT6HcDN6fNLgU0RsTdddhOwEJhU/RaDg4PMnz//iLL58+czODhYpYjsaHR0dPDwww/zvve9j/3793PiiSdyzTXX+GjMakaWRD8DeKZoejdw0XAVJZ0FNAJ9oyw7Y5jllgHLAOrr6ykUChnCqh2zZs1i1apVXHDBBezbt49CocCjjz7KrFmzcretefTQQw9x33338bnPfY7GxkZ27drF7bffzimnnMKCBQuqHZ5lNPTZm5RGaurHq10xV5D0yw9NXwmsGqHudcAXi6Y/BdxYNP1fgU+N9np57LrxoX9tc9dbbVu3bl00NzfHCSecEM3Nzbn93DHOrps9wMyi6Ya0bDiLgY+VLNtWsmwhw2vmytAhfvHJPN9sUzsGBwfZvXs3LS0th/ffdddd5663GuALIVIjfQPEq63wKSQnURt59WRs8zD1fhf4CaCislOBXSQnYqelz08d7fXy2KIvlvcTQnnU0NAQp59++hFHZKeffno0NDRUOzQrYzIdjTGeFn1EHJS0HNhIcgXO2ojYJmlluuLetOpiYH36gkPL7pV0C7AlLVoZ6YlZs1oiadRpOz75QohEpuvoI2IDsKGk7KaS6c+MsOxaYO0Y4zOrumeffZZrr732iKturr76au66665qh2ZlNDU1sXnzZtrb2w+Xbd68maampipGNfE81o1ZGWeeeSZf//rX+eY3v3m4n3fJkiWceeaZ1Q7Nyujq6qKzs/NwH31/fz+dnZ10d3dXO7QJ5URvloG7bmqTL4RIONGblfHss89yzz33HJEsbrvtNpYuXVrt0CyDjo4OOjo6KBQKtLW1VTucqvCgZmZlNDU10dDQwNatW3nooYfYunUrDQ0Nk66f12qXE71ZGUP9vMWjV3Z2dtLV1VXt0MwycdeNWRnu57Va50RvloH7ea2WuevGzCznnOjNzHLOid7MLOec6M3Mcs6J3sws55zozcxyzonezCznnOjNzHLOid7MLOec6M3Mcs6J3sxyraenh5aWFhYsWEBLSws9PT3VDmnCeawbM8utnp4eurq6Dv/CVF1dHZ2dnQCTalA6t+jNLLe6u7tZs2YN7e3tTJkyhfb2dtasWTPpfkowU6KXtFDSE5J2SLp+hDoflLRd0jZJ64rKD0l6LH30VipwM7NyBgcHmT9//hFl8+fPZ3BwsEoRVUfZrhtJdcAdwMXAbmCLpN6I2F5UZw5wA/DuiHhe0tuKVvFSRJxf4bjNzMpqampi8+bNtLe3Hy7bvHnzpPt1sCx99POAHRGxE0DSeuAyYHtRnWuAOyLieYCI+HmlA60VlfrR6IioyHrMJrOhXwcb6qMf+nWwydZ1kyXRzwCeKZreDVxUUuccAEnfA+qAz0TEg+m810saAA4Ct0bE/aUvIGkZsAygvr6eQqFwNNtwXOnv7x91/tIHX+SehSeVXU8tvwd5tm/fPu+bGnLGGWfwoQ99iKuvvpqf/vSnzJo1iw9/+MOcccYZk2o/VuqqmynAHKANaAC+I+m8iPgVcFZE7JF0NtAn6fGIeKp44YhYDawGaG1tjVz/gs+DD/gXimqYf2Gq9rS1tXHLLbdM6n2X5WTsHmBm0XRDWlZsN9AbES9HxC7gSZLET0TsSf/uBArABeOM2czMjkKWRL8FmCOpUdJUYDFQevXM/SSteSRNJ+nK2SlpmqQTi8rfzZF9+2ZmdoyV7bqJiIOSlgMbSfrf10bENkkrgYGI6E3nXSJpO3AI+HREPCfp94G7JL1C8qVya/HVOmZmduxl6qOPiA3AhpKym4qeB/DJ9FFc52HgvPGHaWZmY+U7Y83Mcs6J3sws55zozcxyzonezCznnOjNzHLO49GbWS54nKmRuUVvZrkQEaM+zrruG2Xr5DHJgxO9mVnuOdGbmeWcE72ZWc450ZuZ5ZwTvZlZzjnRm5nlnBO9mVnOOdGbmeWcE72ZWc55CASzlG+ht7xyi94sleX2+Cy30Zsdb5zozcxyLlOil7RQ0hOSdki6foQ6H5S0XdI2SeuKyq+S9OP0cVWlAjczs2zK9tFLqgPuAC4GdgNbJPVGxPaiOnOAG4B3R8Tzkt6Wlp8K3Ay0AgE8ki77fOU3xczMhpOlRT8P2BEROyPiALAeuKykzjXAHUMJPCJ+npZfCmyKiL3pvE3AwsqEbmZmWWS56mYG8EzR9G7gopI65wBI+h5QB3wmIh4cYdkZpS8gaRmwDKC+vp5CoZAx/NqU9+3LO++/2jVZ912lLq+cAswB2oAG4DuSzsu6cESsBlYDtLa2RltbW4XCOg49+AC53r688/6rXZN432XputkDzCyabkjLiu0GeiPi5YjYBTxJkvizLGtmZsdQlkS/BZgjqVHSVGAx0FtS536S1jySppN05ewENgKXSJomaRpwSVpmZmYTpGzXTUQclLScJEHXAWsjYpuklcBARPTyakLfDhwCPh0RzwFIuoXkywJgZUTsPRYbYmZmw8vURx8RG4ANJWU3FT0P4JPpo3TZtcDa8YVpZmZj5TtjzcxyzoOa2aTxzs9+ixdeennc65l9/QPjWv4tb3gdP7z5knHHYZaVE71NGi+89DI/ufX941pHoVAY9yV64/2iMDta7roxM8s5t+iPUiUO/33ob2YTyYn+KI338N+H/mY20dx1Y2aWc070ZmY550RvZpZzTvRmZjnnRG9mlnNO9GZmOefLK82sJvgelrFzojezmuB7WMbOXTdmZjnnRG9mlnNO9GZmOedEb2aWc070ZmY556tubNI4uel6zvvq9eNf0VfHGwfA+H4AxexoZEr0khYCfwvUAXdHxK0l85cCtwN70qJVEXF3Ou8Q8Hha/tOIWFSBuM2O2r8O3upfmLJJqWyil1QH3AFcDOwGtkjqjYjtJVXvjYjlw6zipYg4f/yhmpnZWGTpo58H7IiInRFxAFgPXHZswzIzs0rJ0nUzA3imaHo3cNEw9S6X9F7gSeATETG0zOslDQAHgVsj4v7SBSUtA5YB1NfXUygUsm9BFYwnvn379lVk+4739+h4Nd73zfuvuvzZG6OIGPUBXEHSLz80fSVJH3xxnbcCJ6bPrwX6iubNSP+eDfwEePtorzd37tw4np113TfGtXx/f3/VY5isKvG+ef9Vjz97owMGYoS8mqVFvweYWTTdwKsnXYe+LJ4rmrwb+KuieXvSvzslFYALgKcyvK6Z2WEVuWpqkl4xlSXRbwHmSGokSfCLgSXFFSSdERE/SycXAYNp+TTgNxGxX9J04N0UfQmYmWU13qumJvMVU2UTfUQclLQc2EhyeeXaiNgmaSXJoUIv8GeSFpH0w+8FlqaLNwF3SXqF5MTvrfHaq3XMzOwYynQdfURsADaUlN1U9PwG4IZhlnsYOG+cMZqZ2Th4CAQzs5xzojczyzknejOznPOgZjapVOSqiQfH/7ujZhPJif4o+Vre2jXeAc0g+aKoxHrMJpIT/VHytbxmVmvcR29mlnNO9GZmOedEb2aWc070ZmY550RvZpZzTvRmZjnnRG9mlnNO9GZmOecbpsysZoz7ZsFJOnyFE72Z1YTxDj0xmYevcNeNmVnOOdGbmeWcu27GwP2EZlZLnOiPkvsJzazWZOq6kbRQ0hOSdkh6zWDskpZK+oWkx9LHR4rmXSXpx+njqkoGb2Zm5ZVt0UuqA+4ALgZ2A1sk9UbE9pKq90bE8pJlTwVuBlqBAB5Jl32+ItGbmVlZWVr084AdEbEzIg4A64HLMq7/UmBTROxNk/smYOHYQjUzs7HI0kc/A3imaHo3cNEw9S6X9F7gSeATEfHMCMvOKF1Q0jJgGUB9fT2FQiFT8LUq79uXd95/tWuy7rtKnYz9X0BPROyXdC3Jr6L+YdaFI2I1sBqgtbU1xvtTe8e1Bx8Y908JWhV5/9WuSbzvsnTd7AFmFk03pGWHRcRzEbE/nbwbmJt1WTMzO7ayJPotwBxJjZKmAouB3uIKks4omlwEDKbPNwKXSJomaRpwSVpmZmYTpGzXTUQclLScJEHXAWsjYpuklcBARPQCfyZpEXAQ2AssTZfdK+kWki8LgJURsfcYbIeZmY0gUx99RGwANpSU3VT0/AbghhGWXQusHUeMZmY2Dh7rxsws55zozcxyzonezCznnOjNzHLOid7MLOec6M3Mcs6J3sws55zozcxyzonezCznnOjNzHLOvxlrZrkgqXyd28qvJyIqEM3xxS16M8uFiBj10d/fX7ZOHpM8ONGbmeWeE72ZWc450ZuZ5ZwTvZlZzvmqG7NUlqs2oPyVG3k9oWe1yy16s1SWKzKyXLlhdrxxojczy7lMiV7SQklPSNoh6fpR6l0uKSS1ptOzJb0k6bH0cWelAjczs2zK9tFLqgPuAC4GdgNbJPVGxPaSeicDHwd+ULKKpyLi/ArFa2ZmRynLydh5wI6I2AkgaT1wGbC9pN4twG3ApysaYY3xbdhmdrzJ0nUzA3imaHp3WnaYpAuBmRHxwDDLN0p6VNK3Jb1n7KHWBt+GbWbHm3FfXinpBODzwNJhZv8MmBURz0maC9wvqTkifl2yjmXAMoD6+noKhcJ4wzpu7du3L9fbl3fef7VrMu+7LIl+DzCzaLohLRtyMtACFNJui9OBXkmLImIA2A8QEY9Iego4BxgofoGIWA2sBmhtbY22trYxbUwtKBQK5Hn78s77r3ZN5n2XpetmCzBHUqOkqcBioHdoZkS8EBHTI2J2RMwGvg8siogBSaelJ3ORdDYwB9hZ8a0wM7MRlW3RR8RBScuBjUAdsDYitklaCQxERO8oi78XWCnpZeAV4KMRsbcSgZuZWTaZ+ugjYgOwoaTsphHqthU9vw+4bxzxmZnZOPnOWDOznHOiNzPLOSd6M7Occ6I3M8s5J3ozs5xzojczyzknejOznHOiNzPLOSd6M7Occ6I3M8s5J3ozs5xzojczyzknejOznHOiNzPLOSd6M7Occ6I3M8s5J3ozs5xzojczyzknejOznHOiNzPLuUyJXtJCSU9I2iHp+lHqXS4pJLUWld2QLveEpEsrEbSZmWU3pVwFSXXAHcDFwG5gi6TeiNheUu9k4OPAD4rKzgUWA83AmcA/SzonIg5VbhPMzGw0WVr084AdEbEzIg4A64HLhql3C3Ab8NuissuA9RGxPyJ2ATvS9ZmZ2QQp26IHZgDPFE3vBi4qriDpQmBmRDwg6dMly36/ZNkZpS8gaRmwDKC+vp5CoZAp+Fq0b9++XG9f3nn/1a7JvO+yJPpRSToB+DywdKzriIjVwGqA1tbWaGtrG29Yx61CoUCety/vvP9q12Ted1kS/R5gZtF0Q1o25GSgBShIAjgd6JW0KMOyZmZ2jGXpo98CzJHUKGkqycnV3qGZEfFCREyPiNkRMZukq2ZRRAyk9RZLOlFSIzAH+JeKb0UN6OnpoaWlhQULFtDS0kJPT0+1QzKzSaJsiz4iDkpaDmwE6oC1EbFN0kpgICJ6R1l2m6SvAduBg8DHJuMVNz09PXR1dbFmzRoOHTpEXV0dnZ2dAHR0dFQ5OrN86+npobu7m8HBQZqamujq6pp8n7uIOK4ec+fOjbxpbm6Ovr6+iIjo7++PiIi+vr5obm6uYlQ2FkP7z2rDunXrorGxMfr6+mLTpk3R19cXjY2NsW7dumqHVnEkDe9h86rvjJ0Ag4ODzJ8//4iy+fPnMzg4WKWIzCaH7u5u1qxZQ3t7O1OmTKG9vZ01a9bQ3d1d7dAmlBP9BGhqamLz5s1HlG3evJmmpqYqRWQ2ObiRlXCinwBdXV10dnbS39/PwYMH6e/vp7Ozk66urmqHZpZrbmQlxn0dvZU3dOJnxYoVh08IdXd3T74TQmYTbKiRNXQhxFAja7J13TjRT5COjg46Ojom9U0bZhPNjayEE72Z5ZobWe6jNzPLPSd6M7Occ6I3M8s5J3ozs5xzojczyzknejPLNY8c68srzSzHPHJswi16M8stD2qWcKI3s9zyoGYJJ3ozyy0PapZwojez3PLIsQmfjDWz3PKgZgknejPLNQ9q5q4bM7Pcy5ToJS2U9ISkHZKuH2b+RyU9LukxSZslnZuWz5b0Ulr+mKQ7K70BZmY2urJdN5LqgDuAi4HdwBZJvRGxvajauoi4M62/CPg8sDCd91REnF/ZsM3MLKssLfp5wI6I2BkRB4D1wGXFFSLi10WTJwFRuRDNzGw8spyMnQE8UzS9G7iotJKkjwGfBKYCf1g0q1HSo8CvgRsj4rvDLLsMWAZQX19PoVDIGn/N2bdvX663L++8/2rXZN53ihi98S3pCmBhRHwknb4SuCgilo9QfwlwaURcJelE4E0R8ZykucD9QHPJEUDp8r8Anh7b5tSE6cAvqx2EjZn3X+3K+747KyJOG25Glhb9HmBm0XRDWjaS9cCXASJiP7A/ff6IpKeAc4CBkRYeKdC8kDQQEa3VjsPGxvuvdk3mfZelj34LMEdSo6SpwGKgt7iCpDlFk+8HfpyWn5aezEXS2cAcYGclAjczs2zKtugj4qCk5cBGoA5YGxHbJK0EBiKiF1gu6Y+Al4HngavSxd8LrJT0MvAK8NGI2HssNsTMzIZXto/eKkvSsohYXe04bGy8/2rXZN53TvRmZjnnIRDMzHLOid7MLOec6CtMUpekbZJ+lI7vc7Okz5XUOV/SYPr8TZLukvSUpEckFSS95oY0S0g6lL6v2yT9UNKfSxrT/7GklelFBCPN/6ikPxl7tCDpvKKxnvZK2pU+/+fxrLcWSdpX9PzfSXpS0lkldX4i6b6i6Ssk3VNmva2SvlCmzmxJW0eYV5CU68suPUxxBUl6F/DHwIURsV/SdOBc4B7ghqKqi4Ghn6K/G9gFzImIVyQ1psvY8F4aGjtJ0tuAdcCbgZuPdkURcVOZ+eMehC8iHgeG4r0H+EZE/FNxHUlTIuLgeF+rVkhaAHyB5MbK4W6OnCvp3JLxtEYUEQOMcm9ONUgSyTnQV6odC7hFX2lnAL9MbxQjIn4ZEd8Bni9ppX8Q6JH0dpLhJG4c+oeIiF0R8cBEB16LIuLnJENnLFeiTtLtkrakR1TXDtWVdF06wuoPJd2alt2T3vmNpFslbU+X+29p2WckfSp9fr6k76fzvy5pWlpekHSbpH9JW6jvyRJ7utzfSBoAPi5prqRvp0d1GyWdkdZ7u6QH0/LvSvrdCr6FE07Se4G/A/44Ip4aodp/B17zE1CSTpK0Nn2vH5V0WVreJukb6fPTJG1Kj/julvR02uACqJP0d+m8b0l6Q9Hqr0yPtLZKmpeu61RJ96f7/PuS3pGWH/6/SKe3pkcMs5WM8vv3wFZgZvo/tjX93/vE+N69sXOir6xvkezcJyV9SdIfpOU9JK14JP1bYG9E/BhoBh6LiEPVCbf2RcROkvs73gZ0Ai9ExO8Bvwdco+RGv/eRDMR3UUS8E/ir4nVIeivw70mG53gH8JfDvNTfA9el8x/nyCOIKRExD/gvHN2RxdT0Ts0vAF8EroiIucBaoDutsxpYkZZ/CvjSUaz/eHMiyTAoH4iI/ztKva8BF0r6NyXlXUBf+l63A7dLOqmkzs1pnWbgn4BZRfPmAHek834FXF40743pkeJ/Jnn/AT4LPJru878g+R8oZw7wpfQ1pgMzIqIlIs4DvpJh+WPCXTcVFBH7lIzp8x6Sf8R7lYzffy/wsKQ/58huG6usS4B3DLXSgbeQfPD+CPhKRPwGYJib9l4AfgusSVuG3yieKektwCkR8e206KvAPxZV+R/p30eA2UcR773p398BWoBNyRE/dcDPJL0J+H3gH9NySJJlrXoZeJjkC/njo9Q7BNxO0t35zaLyS4BFRa3p13NkIgeYT/KlTUQ8KOn5onm7IuKx9HnpvupJl/mOpDdLOiVd1+VpeZ+kt0p6c5ltfDoivp8+3wmcLemLwAMkDcGqcKKvsLR1XgAKkh4HroqIeyTtAv6A5B/nXWn1bcA7JdW5VT82SobWOAT8HBBJ63djSZ1LR1tHevf3PGABcAWwnCNHYC1nf/r3EEf3mXpxKERgW0S8q3hmmlR+laPfc3iFpNvyIUl/AdxGknABekvOmfwDSaIvPoEq4PKIeKJ4pZLqM77+/qLnh4DirpvSG4pGu8HoIEf2hry+6PnQPiUinpf0TuBS4KMk2351xlgryl03FSTpd3TkuD/n8+pInD3AXwM7I2I3QNpHOQB8Nj15M3R1wPsnMOyaJek04E5gVSR3/m0E/pOk16Xzz0kP7TcBfyrpjWn5qSXreRPwlojYAHwCeGfx/Ih4geQ8y1D/+5XAt6mcJ4DTlJzMR9LrJA2N8rpL0n9Iy5UmjpqVHlW9H/gQsDQizk8fN5XUe5nk81Lcr70RWFH0WblgmJf4HklCRdIlwLSMof3HdJn5JN1/LwDfTeNEUhvJ+bdfAz8BLkzLLwQah1them7ghIi4D7hxaJlqcIu+st4EfDE97DsI7CAdZ5/kUP8LwIqSZT5CcvJph6SXSIZR/fTEhFuT3iDpMeB1JO/xP5D8ohkkVzDNBv5Pmgx+QdIf/KCk84EBSQeADSR9rkNOBv6npNeTtBo/OczrXgXcmX5Z7AT+tFIbFBEH0u6mL6TdRFOAvyE54vsQ8GVJN6bbvB74YaVeuxoiYq+khcB3JP0iHS9rOGtIEuSQW0jelx8puaR2F8lVbsU+S3Khw5XA/wb+H/CvJJ/N0fxWye9mvI5XW92fAdZK+hHwG14dw+s+4E8kbQN+ADw5wjpnAF/Rq5f/3jBCvWPOQyCYWW4o+Q2MQ2l33LuAL+eo62vM3KI3szyZBXwtbUUfAK6pcjzHBbfozcxyzidjzcxyzonezCznnOjNzHLOid7MLOec6M3Mcu7/A2sFAnosfiuSAAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "0BA8aCqtf_Eu"
      },
      "source": [
        "This plot illustrates the data in a really nice way and delecately displays the insights in a sussinct way. It shows the decision tree classifier as head and shoulders above the other classifiers in terms of acuracy with the box containing the majority of its values havingn a higher accuracy than that of the other models."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WskpygXsnEM2"
      },
      "source": [
        "### Compare this with the results of our hypothesised optimal model"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "tJJUDlkthUVG"
      },
      "source": [
        "When it comes to comparing the difference betweeen this approach and the baseline approach, we want too first join their results dataframes together and then use this joined dataframe to analyse the difference."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "__Jbzt-4wflf"
      },
      "source": [
        "##### Join the two datafarames together so we can easily compare them"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "WrQXUQpdwflg"
      },
      "source": [
        "col_order = [\"subject_id\", \"SVC_Baseline\", \"SVC_unfiltered\", \"Decision Tree_Baseline\", \"Decision Tree_unfiltered\", \"K-Neighbours_Baseline\", \"K-Neighbours_unfiltered\"]\n",
        "\n",
        "baseline_and_unfiltered = baseline_results.merge(unfiltered_results, how='inner', on='subject_id', suffixes=['_Baseline', '_unfiltered'])[col_order]"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "PVTcypAIwfli"
      },
      "source": [
        "##### Output the joined dataframe to compare the classifier scores manually"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 917
        },
        "id": "9KfGFddYwfli",
        "outputId": "1ac45bee-f3e3-472f-aa6c-f319b4d0dc67"
      },
      "source": [
        "baseline_and_unfiltered"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>subject_id</th>\n",
              "      <th>SVC_Baseline</th>\n",
              "      <th>SVC_unfiltered</th>\n",
              "      <th>Decision Tree_Baseline</th>\n",
              "      <th>Decision Tree_unfiltered</th>\n",
              "      <th>K-Neighbours_Baseline</th>\n",
              "      <th>K-Neighbours_unfiltered</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>46343</td>\n",
              "      <td>0.524590</td>\n",
              "      <td>0.453552</td>\n",
              "      <td>0.557377</td>\n",
              "      <td>0.612022</td>\n",
              "      <td>0.453552</td>\n",
              "      <td>0.387978</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>759667</td>\n",
              "      <td>0.566879</td>\n",
              "      <td>0.522293</td>\n",
              "      <td>0.700637</td>\n",
              "      <td>0.707006</td>\n",
              "      <td>0.503185</td>\n",
              "      <td>0.541401</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>781756</td>\n",
              "      <td>0.589506</td>\n",
              "      <td>0.506173</td>\n",
              "      <td>0.493827</td>\n",
              "      <td>0.506173</td>\n",
              "      <td>0.561728</td>\n",
              "      <td>0.561728</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>844359</td>\n",
              "      <td>0.548822</td>\n",
              "      <td>0.508418</td>\n",
              "      <td>0.545455</td>\n",
              "      <td>0.535354</td>\n",
              "      <td>0.542088</td>\n",
              "      <td>0.518519</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>1066528</td>\n",
              "      <td>0.523962</td>\n",
              "      <td>0.488818</td>\n",
              "      <td>0.549521</td>\n",
              "      <td>0.498403</td>\n",
              "      <td>0.520767</td>\n",
              "      <td>0.485623</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>5</th>\n",
              "      <td>1360686</td>\n",
              "      <td>0.620915</td>\n",
              "      <td>0.529412</td>\n",
              "      <td>0.666667</td>\n",
              "      <td>0.555556</td>\n",
              "      <td>0.640523</td>\n",
              "      <td>0.568627</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>6</th>\n",
              "      <td>1449548</td>\n",
              "      <td>0.685714</td>\n",
              "      <td>0.650794</td>\n",
              "      <td>0.682540</td>\n",
              "      <td>0.590476</td>\n",
              "      <td>0.666667</td>\n",
              "      <td>0.653968</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>7</th>\n",
              "      <td>1455390</td>\n",
              "      <td>0.718354</td>\n",
              "      <td>0.702532</td>\n",
              "      <td>0.737342</td>\n",
              "      <td>0.686709</td>\n",
              "      <td>0.715190</td>\n",
              "      <td>0.693038</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>8</th>\n",
              "      <td>1818471</td>\n",
              "      <td>0.533123</td>\n",
              "      <td>0.533123</td>\n",
              "      <td>0.605678</td>\n",
              "      <td>0.586751</td>\n",
              "      <td>0.555205</td>\n",
              "      <td>0.548896</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>9</th>\n",
              "      <td>2598705</td>\n",
              "      <td>0.609524</td>\n",
              "      <td>0.584127</td>\n",
              "      <td>0.561905</td>\n",
              "      <td>0.501587</td>\n",
              "      <td>0.523810</td>\n",
              "      <td>0.539683</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>10</th>\n",
              "      <td>2638030</td>\n",
              "      <td>0.709265</td>\n",
              "      <td>0.584665</td>\n",
              "      <td>0.680511</td>\n",
              "      <td>0.575080</td>\n",
              "      <td>0.648562</td>\n",
              "      <td>0.546326</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>11</th>\n",
              "      <td>3509524</td>\n",
              "      <td>0.630435</td>\n",
              "      <td>0.572464</td>\n",
              "      <td>0.673913</td>\n",
              "      <td>0.710145</td>\n",
              "      <td>0.681159</td>\n",
              "      <td>0.630435</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>12</th>\n",
              "      <td>3997827</td>\n",
              "      <td>0.643533</td>\n",
              "      <td>0.618297</td>\n",
              "      <td>0.719243</td>\n",
              "      <td>0.690852</td>\n",
              "      <td>0.668770</td>\n",
              "      <td>0.608833</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>13</th>\n",
              "      <td>4018081</td>\n",
              "      <td>0.606061</td>\n",
              "      <td>0.490909</td>\n",
              "      <td>0.666667</td>\n",
              "      <td>0.696970</td>\n",
              "      <td>0.606061</td>\n",
              "      <td>0.509091</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>14</th>\n",
              "      <td>4314139</td>\n",
              "      <td>0.534591</td>\n",
              "      <td>0.522013</td>\n",
              "      <td>0.490566</td>\n",
              "      <td>0.462264</td>\n",
              "      <td>0.575472</td>\n",
              "      <td>0.525157</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>15</th>\n",
              "      <td>4426783</td>\n",
              "      <td>0.638889</td>\n",
              "      <td>0.567901</td>\n",
              "      <td>0.567901</td>\n",
              "      <td>0.533951</td>\n",
              "      <td>0.620370</td>\n",
              "      <td>0.530864</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>16</th>\n",
              "      <td>5132496</td>\n",
              "      <td>0.655844</td>\n",
              "      <td>0.577922</td>\n",
              "      <td>0.538961</td>\n",
              "      <td>0.571429</td>\n",
              "      <td>0.623377</td>\n",
              "      <td>0.584416</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>17</th>\n",
              "      <td>5498603</td>\n",
              "      <td>0.573171</td>\n",
              "      <td>0.524390</td>\n",
              "      <td>0.552846</td>\n",
              "      <td>0.536585</td>\n",
              "      <td>0.573171</td>\n",
              "      <td>0.491870</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>18</th>\n",
              "      <td>5797046</td>\n",
              "      <td>0.603226</td>\n",
              "      <td>0.558065</td>\n",
              "      <td>0.574194</td>\n",
              "      <td>0.551613</td>\n",
              "      <td>0.600000</td>\n",
              "      <td>0.551613</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>19</th>\n",
              "      <td>6220552</td>\n",
              "      <td>0.574603</td>\n",
              "      <td>0.504762</td>\n",
              "      <td>0.682540</td>\n",
              "      <td>0.558730</td>\n",
              "      <td>0.593651</td>\n",
              "      <td>0.552381</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>20</th>\n",
              "      <td>7749105</td>\n",
              "      <td>0.536585</td>\n",
              "      <td>0.390244</td>\n",
              "      <td>0.707317</td>\n",
              "      <td>0.512195</td>\n",
              "      <td>0.560976</td>\n",
              "      <td>0.365854</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>21</th>\n",
              "      <td>8000685</td>\n",
              "      <td>0.589905</td>\n",
              "      <td>0.586751</td>\n",
              "      <td>0.624606</td>\n",
              "      <td>0.536278</td>\n",
              "      <td>0.602524</td>\n",
              "      <td>0.570978</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>22</th>\n",
              "      <td>8173033</td>\n",
              "      <td>0.566456</td>\n",
              "      <td>0.528481</td>\n",
              "      <td>0.579114</td>\n",
              "      <td>0.582278</td>\n",
              "      <td>0.585443</td>\n",
              "      <td>0.496835</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>23</th>\n",
              "      <td>8530312</td>\n",
              "      <td>0.624204</td>\n",
              "      <td>0.531847</td>\n",
              "      <td>0.579618</td>\n",
              "      <td>0.490446</td>\n",
              "      <td>0.627389</td>\n",
              "      <td>0.541401</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>24</th>\n",
              "      <td>8686948</td>\n",
              "      <td>0.632911</td>\n",
              "      <td>0.607595</td>\n",
              "      <td>0.613924</td>\n",
              "      <td>0.541139</td>\n",
              "      <td>0.601266</td>\n",
              "      <td>0.579114</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>25</th>\n",
              "      <td>8692923</td>\n",
              "      <td>0.579288</td>\n",
              "      <td>0.495146</td>\n",
              "      <td>0.644013</td>\n",
              "      <td>0.556634</td>\n",
              "      <td>0.514563</td>\n",
              "      <td>0.472492</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>26</th>\n",
              "      <td>9106476</td>\n",
              "      <td>0.602524</td>\n",
              "      <td>0.545741</td>\n",
              "      <td>0.621451</td>\n",
              "      <td>0.561514</td>\n",
              "      <td>0.630915</td>\n",
              "      <td>0.558360</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>27</th>\n",
              "      <td>9961348</td>\n",
              "      <td>0.605042</td>\n",
              "      <td>0.596639</td>\n",
              "      <td>0.596639</td>\n",
              "      <td>0.588235</td>\n",
              "      <td>0.592437</td>\n",
              "      <td>0.563025</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "   subject_id  SVC_Baseline  SVC_unfiltered  Decision Tree_Baseline  \\\n",
              "0       46343      0.524590        0.453552                0.557377   \n",
              "1      759667      0.566879        0.522293                0.700637   \n",
              "2      781756      0.589506        0.506173                0.493827   \n",
              "3      844359      0.548822        0.508418                0.545455   \n",
              "4     1066528      0.523962        0.488818                0.549521   \n",
              "5     1360686      0.620915        0.529412                0.666667   \n",
              "6     1449548      0.685714        0.650794                0.682540   \n",
              "7     1455390      0.718354        0.702532                0.737342   \n",
              "8     1818471      0.533123        0.533123                0.605678   \n",
              "9     2598705      0.609524        0.584127                0.561905   \n",
              "10    2638030      0.709265        0.584665                0.680511   \n",
              "11    3509524      0.630435        0.572464                0.673913   \n",
              "12    3997827      0.643533        0.618297                0.719243   \n",
              "13    4018081      0.606061        0.490909                0.666667   \n",
              "14    4314139      0.534591        0.522013                0.490566   \n",
              "15    4426783      0.638889        0.567901                0.567901   \n",
              "16    5132496      0.655844        0.577922                0.538961   \n",
              "17    5498603      0.573171        0.524390                0.552846   \n",
              "18    5797046      0.603226        0.558065                0.574194   \n",
              "19    6220552      0.574603        0.504762                0.682540   \n",
              "20    7749105      0.536585        0.390244                0.707317   \n",
              "21    8000685      0.589905        0.586751                0.624606   \n",
              "22    8173033      0.566456        0.528481                0.579114   \n",
              "23    8530312      0.624204        0.531847                0.579618   \n",
              "24    8686948      0.632911        0.607595                0.613924   \n",
              "25    8692923      0.579288        0.495146                0.644013   \n",
              "26    9106476      0.602524        0.545741                0.621451   \n",
              "27    9961348      0.605042        0.596639                0.596639   \n",
              "\n",
              "    Decision Tree_unfiltered  K-Neighbours_Baseline  K-Neighbours_unfiltered  \n",
              "0                   0.612022               0.453552                 0.387978  \n",
              "1                   0.707006               0.503185                 0.541401  \n",
              "2                   0.506173               0.561728                 0.561728  \n",
              "3                   0.535354               0.542088                 0.518519  \n",
              "4                   0.498403               0.520767                 0.485623  \n",
              "5                   0.555556               0.640523                 0.568627  \n",
              "6                   0.590476               0.666667                 0.653968  \n",
              "7                   0.686709               0.715190                 0.693038  \n",
              "8                   0.586751               0.555205                 0.548896  \n",
              "9                   0.501587               0.523810                 0.539683  \n",
              "10                  0.575080               0.648562                 0.546326  \n",
              "11                  0.710145               0.681159                 0.630435  \n",
              "12                  0.690852               0.668770                 0.608833  \n",
              "13                  0.696970               0.606061                 0.509091  \n",
              "14                  0.462264               0.575472                 0.525157  \n",
              "15                  0.533951               0.620370                 0.530864  \n",
              "16                  0.571429               0.623377                 0.584416  \n",
              "17                  0.536585               0.573171                 0.491870  \n",
              "18                  0.551613               0.600000                 0.551613  \n",
              "19                  0.558730               0.593651                 0.552381  \n",
              "20                  0.512195               0.560976                 0.365854  \n",
              "21                  0.536278               0.602524                 0.570978  \n",
              "22                  0.582278               0.585443                 0.496835  \n",
              "23                  0.490446               0.627389                 0.541401  \n",
              "24                  0.541139               0.601266                 0.579114  \n",
              "25                  0.556634               0.514563                 0.472492  \n",
              "26                  0.561514               0.630915                 0.558360  \n",
              "27                  0.588235               0.592437                 0.563025  "
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 158
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "U2S8rHwwhrc8"
      },
      "source": [
        "Looking at this plot doesn't make anything in particular stand out so we will deed to summarise these results."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "h9IY8r0Xwflm"
      },
      "source": [
        "##### Describe this dataframe"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 297
        },
        "id": "NH1GuWJgwflm",
        "outputId": "65aa9b26-c56b-4fd4-97cd-e805b5538b0e"
      },
      "source": [
        "baseline_and_unfiltered.describe()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>SVC_Baseline</th>\n",
              "      <th>SVC_unfiltered</th>\n",
              "      <th>Decision Tree_Baseline</th>\n",
              "      <th>Decision Tree_unfiltered</th>\n",
              "      <th>K-Neighbours_Baseline</th>\n",
              "      <th>K-Neighbours_unfiltered</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>count</th>\n",
              "      <td>28.000000</td>\n",
              "      <td>28.000000</td>\n",
              "      <td>28.000000</td>\n",
              "      <td>28.000000</td>\n",
              "      <td>28.000000</td>\n",
              "      <td>28.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>mean</th>\n",
              "      <td>0.600997</td>\n",
              "      <td>0.545824</td>\n",
              "      <td>0.614820</td>\n",
              "      <td>0.572728</td>\n",
              "      <td>0.592458</td>\n",
              "      <td>0.542089</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>std</th>\n",
              "      <td>0.052381</td>\n",
              "      <td>0.061756</td>\n",
              "      <td>0.068533</td>\n",
              "      <td>0.068636</td>\n",
              "      <td>0.059108</td>\n",
              "      <td>0.067519</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>min</th>\n",
              "      <td>0.523962</td>\n",
              "      <td>0.390244</td>\n",
              "      <td>0.490566</td>\n",
              "      <td>0.462264</td>\n",
              "      <td>0.453552</td>\n",
              "      <td>0.365854</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>25%</th>\n",
              "      <td>0.566773</td>\n",
              "      <td>0.507856</td>\n",
              "      <td>0.560773</td>\n",
              "      <td>0.535003</td>\n",
              "      <td>0.559533</td>\n",
              "      <td>0.516162</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>50%</th>\n",
              "      <td>0.602875</td>\n",
              "      <td>0.532485</td>\n",
              "      <td>0.609801</td>\n",
              "      <td>0.557682</td>\n",
              "      <td>0.596825</td>\n",
              "      <td>0.547611</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>75%</th>\n",
              "      <td>0.631054</td>\n",
              "      <td>0.584261</td>\n",
              "      <td>0.675563</td>\n",
              "      <td>0.588796</td>\n",
              "      <td>0.628270</td>\n",
              "      <td>0.569215</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>max</th>\n",
              "      <td>0.718354</td>\n",
              "      <td>0.702532</td>\n",
              "      <td>0.737342</td>\n",
              "      <td>0.710145</td>\n",
              "      <td>0.715190</td>\n",
              "      <td>0.693038</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "       SVC_Baseline  SVC_unfiltered  Decision Tree_Baseline  \\\n",
              "count     28.000000       28.000000               28.000000   \n",
              "mean       0.600997        0.545824                0.614820   \n",
              "std        0.052381        0.061756                0.068533   \n",
              "min        0.523962        0.390244                0.490566   \n",
              "25%        0.566773        0.507856                0.560773   \n",
              "50%        0.602875        0.532485                0.609801   \n",
              "75%        0.631054        0.584261                0.675563   \n",
              "max        0.718354        0.702532                0.737342   \n",
              "\n",
              "       Decision Tree_unfiltered  K-Neighbours_Baseline  \\\n",
              "count                 28.000000              28.000000   \n",
              "mean                   0.572728               0.592458   \n",
              "std                    0.068636               0.059108   \n",
              "min                    0.462264               0.453552   \n",
              "25%                    0.535003               0.559533   \n",
              "50%                    0.557682               0.596825   \n",
              "75%                    0.588796               0.628270   \n",
              "max                    0.710145               0.715190   \n",
              "\n",
              "       K-Neighbours_unfiltered  \n",
              "count                28.000000  \n",
              "mean                  0.542089  \n",
              "std                   0.067519  \n",
              "min                   0.365854  \n",
              "25%                   0.516162  \n",
              "50%                   0.547611  \n",
              "75%                   0.569215  \n",
              "max                   0.693038  "
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 159
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "St2nao3Dh65l"
      },
      "source": [
        "This is where we really start to see the insights. The baseline is displayed beside its corrosponding row under this unfiltered approach and as we compare these side-by-side columns, we can see the common pattern where our baselike comes out ahead. In all of the important categories, our baseline models\n",
        " are more accurate than the models trained on the unfiltered data.\n",
        "\n",
        " The below plot can show this more clearly."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ej_5dyl0wfln"
      },
      "source": [
        "##### Plot the classifiers"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 446
        },
        "id": "0TNUsRIVwfln",
        "outputId": "3b7c6955-3dd5-4065-c28b-d15058e73988"
      },
      "source": [
        "baseline_and_unfiltered.boxplot(figsize=(14,7))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<matplotlib.axes._subplots.AxesSubplot at 0x7fe6ee5a9b00>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 160
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAzgAAAGcCAYAAAAVngVTAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3dfZhdZXno/+9tAkoFEQvmIG/Bnng6NFTU/KBqameaA+KhB+zRehhfKnU05VwletkjJZzxB4hOJbXWHiutRieF+jLxreVEk4IWZ1fja0IFDZmCIaCE9vxUQGwsQhLu3x9rTbKzmZedzA5rz5rv57r2NWuv/ay17r3Ws9fsez3PenZkJpIkSZJUB0+qOgBJkiRJ6hQTHEmSJEm1YYIjSZIkqTZMcCRJkiTVhgmOJEmSpNqYX3UArY499thcuHBh1WFU7mc/+xlPfepTqw5DFbMeCKwHKlgPBNYD7WNdgFtuueXHmXlc6/yuS3AWLlzI5s2bqw6jco1Gg97e3qrDUMWsBwLrgQrWA4H1QPtYFyAivj/RfLuoSZIkSaqNthKciDg3Iu6IiG0RsXKC198XEbeWjzsj4idNr+1pem1dJ4OXJEmSpGbTdlGLiHnAtcDZwA5gU0Ssy8yt42Uy861N5VcAz2taxcOZeUbnQpYkSZKkibXTgnMmsC0zt2fmo8Ba4IIpyvcDI50ITpIkSZIORGTm1AUiXgmcm5lvLJ+/DjgrMy+ZoOwpwDeAEzNzTzlvN3ArsBu4JjNvmGC55cBygAULFrxg7dq1M3pTdbBz506OPPLIqsNQxawHAuuBCtYDgfVA+1gXoK+v75bMXNI6v9OjqF0IfGY8uSmdkpn3RcSzgS9FxHcz867mhTJzNbAaYMmSJTnXR4QAR8ZQwXogsB6oYD0QWA+0j3Vhcu10UbsPOKnp+YnlvIlcSEv3tMy8r/y7HWiw//05kiRJktQx7SQ4m4BFEXFqRBxOkcQ8bjS0iPhl4Bjg603zjomIJ5fTxwIvBra2LitJkiRJnTBtF7XM3B0RlwA3AfOANZl5e0RcDWzOzPFk50Jgbe5/U08P8KGIeIwimbqmefQ1SZIkSeqktu7BycwNwIaWeVe0PL9qguW+Bpw+g/gkSZIkqW1t/dCnJEmSJM0GJjiSJEmSasMER5IkSVJtmOBIkiRJqg0THEmSJEm1YYIjSZIkqTbaGiZaknTwIqLqENj/J8okSaovW3Ak6RDLzBk9Trns8zNehyRJc4UJjiRJkqTaMMGRJEmSVBsmOJIkSZJqwwRHkiRJUm2Y4EiSJEmqDRMcSZIkSbVhgiNJkiSpNkxwJEmSJNWGCY4kSZKk2jDBkSRJklQbJjiSJEmSasMER5IkSVJtmOBIkiRJqg0THEmSJEm1YYIjSZIkqTbmVx1AHUVE1SEAkJlVhyBJkiQ9oWzBOQQyc8aPUy77/IzXIUmSJM01JjiSJEmSasMuapIkSU8Qu7FLh54tOJIkSU+QbujCbnKjujPBkSSpi42MjLB48WKWLVvG4sWLGRkZqTokSepqdlGTJKlLjYyMMDg4yPDwMHv27GHevHkMDAwA0N/fX3F0ktSdbMGRJKlLDQ0NMTw8TF9fH/Pnz6evr4/h4WGGhoaqDk2SupYJjiRJXWpsbIwdO3bs10Vtx44djI2NVR2aJHUtu6hJktSlnvWsZ/FHf/RHfOITn9jbRe3Vr341z3rWs6oOTZK6li04kiR1sdZhhbtlmGFJ6la24EiS1KX+5V/+heuuu44VK1YwNjZGT08Pq1at4qKLLqo6NEnqWrbgSF3IYWElAfT09HDiiSeyZcsWbr75ZrZs2cKJJ55IT09P1aFJUteyBUfqMg4LK2nc4OAgAwMDe88Ho6OjDAwMOIqaJE2hrQQnIs4F/jcwD/hIZl7T8vr7gL7y6S8Az8zMp5evvR54e/nauzLz+k4ELtVV87CwjUaD3t5ehoeHWbFihQmONMeMf+abu6gNDQ15LpCkKUyb4ETEPOBa4GxgB7ApItZl5tbxMpn51qbyK4DnldPPAK4ElgAJ3FIu+2BH34VUI2NjYyxdunS/eUuXLnVYWGmO6u/vp7+/f+8FD0nS1Nq5B+dMYFtmbs/MR4G1wAVTlO8Hxm8YeCnwxcx8oExqvgicO5OApbrr6elh48aN+83buHGjfe4lSZLa0E4XtROAe5ue7wDOmqhgRJwCnAp8aYplT5hgueXAcoAFCxbQaDTaCKv+3A9z02//9m/zmte8hksvvZRTTz2V973vfbznPe9hYGDAOjGHeeznrptvvpmPfexj/OAHP+Dkk0/mta99LcuWLas6LFXI84EAdu7caV2YRKcHGbgQ+Exm7jmQhTJzNbAaYMmSJWkTPHDjersizFG9vb2cdtppDA0N7e1z/973vtc+93OZ54M5a2RkhI9//OOsWbNmv0FHTjvtNM8Jc5XnA5Xstjq5drqo3Qec1PT8xHLeRC5kX/e0A11WUqm/v3+/YWH9IiPNTc2DjsyfP5++vj6Gh4cdRU2SptBOgrMJWBQRp0bE4RRJzLrWQhHxy8AxwNebZt8EnBMRx0TEMcA55TxJkjQNBx2RpAM3bYKTmbuBSygSkzHgU5l5e0RcHRHnNxW9EFibmdm07APAOymSpE3A1eU8SZI0DQcdkdTKHwOfXlv34GTmBmBDy7wrWp5fNcmya4A1BxmfJElzlj/0KamZPwbenna6qEmSpAr09/dz3nnn8bKXvYyzzz6bl73sZZx33nl+kZHmKO/La0+nR1GTJEkdMjIywvr16/n7v//7/a7WvuhFLzLJkeYg78trjy04kiR1Ka/WSmrmfXntMcGRJKlLebVWUrPx+/JGR0fZvXv33vvyBgcHqw6tq9hFTZKkLtXT08M73vEObrjhhr0//Pvyl7/cq7XSHDXeNXXFihV7zwlDQ0N2WW1hgiNJUpfq6+tj1apVrFq1itNOO42tW7dy2WWXcfHFF1cdmqSK9Pf309/fT6PRoLe3t+pwupIJjiRJXWp0dJTLLruMNWvW7L1ae9lll3HDDTdUHZokdS0THEmSutTY2Bjf/va3ede73rX3au2uXbt497vfXXVoktS1HGRAkqQu5YhJknTgTHAkSepSjpgkSQfOLmqSJHUpR0ySpANngiNJUhdzxCRJOjB2UZMkSZJUGyY4kiRJkmrDBEeSJElSbZjgSJIkSaoNExxJkiRJteEoapI0hee+4ws89PCuqsNg4cr1lW7/6CMO47Yrz6k0BkmS2mGCI0lTeOjhXdxzzXmVxtANwwNXnWBJktQuu6hJkiRJqg0THEmSJEm1YYIjSZIkqTZMcCRJkiTVhgmOJEmSpNowwZEkSZJUGyY4kiRJkmrDBEeSJElSbZjgSJIkSaoNExxJkiRJtWGCI0mSJKk2THAkSZIk1YYJjiRJkqTaMMGRJEmSVBsmOJIkSZJqwwRHkiRJmiVGRkZYvHgxy5YtY/HixYyMjFQdUteZX3UAkiRJkqY3MjLC4OAgw8PD7Nmzh3nz5jEwMABAf39/xdF1D1twJEmSpFlgaGiI4eFh+vr6mD9/Pn19fQwPDzM0NFR1aF3FBEeSJEmaBcbGxli6dOl+85YuXcrY2FhFEXWnthKciDg3Iu6IiG0RsXKSMq+KiK0RcXtEfKJp/p6IuLV8rOtU4JIkSdJc0tPTw8aNG/ebt3HjRnp6eiqKqDtNew9ORMwDrgXOBnYAmyJiXWZubSqzCLgceHFmPhgRz2xaxcOZeUaH45YkSZLmlMHBQQYGBvbegzM6OsrAwIBd1Fq0M8jAmcC2zNwOEBFrgQuArU1l3gRcm5kPAmTmDzsdqDTbRETVIQCQmVWHIEmSOmB8IIEVK1YwNjZGT08PQ0NDDjDQop0E5wTg3qbnO4CzWso8ByAivgrMA67KzBvL154SEZuB3cA1mXlD6wYiYjmwHGDBggU0Go0DeQ+15X6Y3UZHR2e8jotu/BnXnfvUGa3DejRzVe/DnTt3Vh4DVL8f5rpuqQeqnvVgbjv++OP5wAc+wM6dOznyyCMB60SrTg0TPR9YBPQCJwJfjojTM/MnwCmZeV9EPBv4UkR8NzPval44M1cDqwGWLFmSvb29HQprFrtxPe4HWQ+6QBccg0ajUXkM3bAf5rquqAeqnp9FlTwnTK6dQQbuA05qen5iOa/ZDmBdZu7KzLuBOykSHjLzvvLvdqABPG+GMUuSJEnShNpJcDYBiyLi1Ig4HLgQaB0N7QaK1hsi4liKLmvbI+KYiHhy0/wXs/+9O5IkSZLUMdN2UcvM3RFxCXATxf01azLz9oi4GticmevK186JiK3AHuDSzLw/Il4EfCgiHqNIpq5pHn1NkiRJkjqprXtwMnMDsKFl3hVN0wn8YfloLvM14PSZhylJkiRJ02vrhz4lSZIkaTYwwZEkSZJUGyY4kiRJkmrDBEeSJElSbZjgSJIkSaoNExxJkiRJtWGCI0mSJKk2THAkSZIk1YYJjiRJkqTaMMGRJEmSVBsmOJIkSZJqwwRHkiRJUm3MrzqAbvTcd3yBhx7eVXUYLFy5vrJtH33EYdx25TmVbV+SJKmuIqLqEADIzKpDOCRMcCbw0MO7uOea8yqNodFo0NvbW9n2q0yuJEmS6qwTicXClesr/77arUxwJEmS2tQNvTyqvghpLw91OxMcSZKkNlXdy6PqHh5QfYIlTcdBBiRJkiTVhgmOJEmSpNowwZEkSZJUGyY4kiRJkmrDBEeSJElSbZjgSJIkSaoNExxJkiRJtWGCI0mSJKk2THAkSZIk1YYJjiRJkqTaMMGRJEmSVBsmOJIkSZJqwwRHkiRJUm3MrzoASZLmgoioOgQys+oQJOmQswVHkqQnQGbO6HHKZZ+f8TokaS6wBUeSpnBUz0pOv35l1WHA9dVu/qgegPOqDUKSpDaY4EjSFP5t7BruuabaL/aNRoPe3t5KY1i4cn2l25ckqV12UZMkSZJUGyY4kiRJkmrDBEeSJElSbZjgSJIkSaqNthKciDg3Iu6IiG0RMeFwQhHxqojYGhG3R8Qnmua/PiK+Vz5e36nAJUmSJKnVtKOoRcQ84FrgbGAHsCki1mXm1qYyi4DLgRdn5oMR8cxy/jOAK4ElQAK3lMs+2Pm30jkOC+uQsJIkSZqd2hkm+kxgW2ZuB4iItcAFwNamMm8Crh1PXDLzh+X8lwJfzMwHymW/CJwLjHQm/EPDYWEdElaSJEmzUztd1E4A7m16vqOc1+w5wHMi4qsR8Y2IOPcAlpUkSZKkjujUD33OBxYBvcCJwJcj4vR2F46I5cBygAULFtBoNDoU1sGrOoadO3dWHkPV21fB41C9qo9BN5wPoPr9II9Bt6jyOHg+UDOPw8TaSXDuA05qen5iOa/ZDuCbmbkLuDsi7qRIeO6jSHqal220biAzVwOrAZYsWZJV/2I3N66v/FfDq+6i1g37QHgcukEXHIPKzwfQFfthzvMYdIeKj4PnA+3lcZhUO13UNgGLIuLUiDgcuBBY11LmBspEJiKOpeiyth24CTgnIo6JiGOAc8p5kiRJktRx07bgZObuiLiEIjGZB6zJzNsj4mpgc2auY18isxXYA1yamfcDRMQ7KZIkgKvHBxyQJEmabbpipNUKR1kFR1pV92vrHpzM3ABsaJl3RdN0An9YPlqXXQOsmVmYkiRJ1at6pNVu6KLmSKvqdm390KckSZIkzQYmOJIkSZJqwwRHkiRJUm2Y4EiSJEmqDRMcSZIkSbXR1ihqkiTNdc99xxd46OFdlcZQ9ehVRx9xGLddeU6lMUjSdExwJElqw0MP73J4YIcHljQL2EVNkiRJUm2Y4EiSJEmqDbuoSZPohv72UG2XEPvbS5Kk2cYER5pE1f3tofo+9/a3lyRJs41d1CRJkiTVhgmOJEmSpNowwZEkSZJUGyY4kiRJkmrDBEeSJElSbZjgSJIkSaoNExxJkiRJtWGCI0mSJKk2THAkSZIk1YYJjiRJkqTaMMGRJEmSVBsmOJIkSZJqY37VAUiSJEmzyXPf8QUeenhX1WGwcOX6yrZ99BGHcduV51S2/amY4EiSJEkH4KGHd3HPNedVGkOj0aC3t7ey7VeZXE3HLmqSJEmSasMWnEl0RVZ6Y7XNjpIkSdJsY4IzgaqbHKFIsLohDkmSJGk2sYuaJEmSpNowwZEkSZJUGyY4kiRJkmrDe3AkaRpzfdARcOARSdLsYYIjSVPohsE+HHRE6i6VX/Twgoc0JRMcSZKkNlV9scELHtL0THAkSWrDUT0rOf36ldUGcX21mz+qB8Av15K6mwmOJElt+Lexayq9ct5oNOjt7a1s+9AFXbMkqQ2OoiZJkiSpNkxwJEmSJNVGWwlORJwbEXdExLaIeFwH5Ii4KCJ+FBG3lo83Nr22p2n+uk4GL0mSJEnNpr0HJyLmAdcCZwM7gE0RsS4zt7YU/WRmXjLBKh7OzDNmHqokSZIkTa2dFpwzgW2ZuT0zHwXWAhcc2rAkSZIk6cC1M4raCcC9Tc93AGdNUO4VEfES4E7grZk5vsxTImIzsBu4JjNvaF0wIpYDywEWLFhAo9Fo/x3UmPuhelUfg507d1YeQ9XbV8Hj0B2qPA7dcD4A62I38Bh0h6qPQzecE6re/mQ6NUz054CRzHwkIn6fYqT+3yxfOyUz74uIZwNfiojvZuZdzQtn5mpgNcCSJUuy6mEwu8KN6ysfDnTO64JjUPmwsF2wD4THoVtUfBwqPx9A5ftAeAy6RRcch8rPCV2wDybTToJzH3BS0/MTy3l7Zeb9TU8/AvxJ02v3lX+3R0QDeB6wX4IjSZIkzRZd8cO/UOmP/3bzD/+2k+BsAhZFxKkUic2FwKubC0TE8Zn5r+XT84Gxcv4xwL+XLTvHAi+mKfmRJEmSZpuqf/gXqm/B6eYf/p02wcnM3RFxCXATMA9Yk5m3R8TVwObMXAe8OSLOp7jP5gHgonLxHuBDEfEYxYAG10ww+pokSZIkdURb9+Bk5gZgQ8u8K5qmLwcun2C5rwGnzzBGSZIkSWpLWz/0KUmSJEmzgQmOJEmSpNowwZEkSZJUGyY4kiRJkmrDBEeSJElSbZjgSJIkSaoNExxJkiRJtWGCI0mSJKk22vqhT0mSBAtXrq82gBur3f7RRxxW6fYlqR0mOJIkteGea86rdPsLV66vPAZJmg1McKRJHNWzktOvX1l1GHB9dZs+qgfAL1SSJGn2MMGRJvFvY9dUfrW00WjQ29tb2fYr744jSZJ0gBxkQJIkSVJtmOBIkiRJqg0THEmSJEm1YYIjSZIkqTZMcCRJkiTVhgmOJEmSpNpwmOhDICI6s55VM1s+MzsShyRJkjRb2IJzCGTmjB+jo6MzXockSZI015jgSJIkSaoNExxJkiRJtWGCI0mSJKk2THAkSZIk1YYJjiRJkqTaMMGRJEmSVBv+Do4kSZJ0gBauXF91CHBjdTEcfcRhlW17OiY4kiRJ0gG455rzqg6BhSvXd0Uc3cguapIkSZJqwwRHkiRJUm2Y4EiSJEmqDRMcSZIkSbVhgiNJkiSpNhxFTZqCQ0B27xCQkiRJEzHBkSbRDUMvOgSkJEnSgbGLmiRJkqTaMMGRJEmSVBsmOJIkSZJqo60EJyLOjYg7ImJbRKyc4PWLIuJHEXFr+Xhj02uvj4jvlY/XdzJ4SZKk2SQiZvT4/qrfmvE6IqLq3SAdUtMOMhAR84BrgbOBHcCmiFiXmVtbin4yMy9pWfYZwJXAEiCBW8plH+xI9JIkSbNIZs5o+UajQW9vb2eCkWqqnRacM4Ftmbk9Mx8F1gIXtLn+lwJfzMwHyqTmi8C5BxeqJEmSJE2tnWGiTwDubXq+AzhrgnKviIiXAHcCb83MeydZ9oTWBSNiObAcYMGCBTQajbaCr7OdO3e6HwRgPRBgPVDBeiC/H6iZdWFinfodnM8BI5n5SET8PnA98JvtLpyZq4HVAEuWLEmbXm2CVunG9dYDWQ9UsB4Ivx+oieeESbXTRe0+4KSm5yeW8/bKzPsz85Hy6UeAF7S7rCRJkiR1SjsJziZgUUScGhGHAxcC65oLRMTxTU/PB8bK6ZuAcyLimIg4BjinnCdJkiRJHTdtF7XM3B0Rl1AkJvOANZl5e0RcDWzOzHXAmyPifGA38ABwUbnsAxHxTookCeDqzHzgELwPSZIkSWrvHpzM3ABsaJl3RdP05cDlkyy7BlgzgxglSZIkqS1t/dCnJEmSJM0GJjiSJEmSaqNTw0RLkiYRETNfx6qZLT/TX0+XJGm2sAVHkg6xzJzRY3R0dMbrkCRprjDBkSRJklQbJjiSJEmSasMER5IkSVJtmOBIkiRJqg0THEmSJEm1YYIjSZIkqTZMcCRJkiTVhgmOJEmSpNowwZEkSZJUGyY4kiRJkmrDBEeSJElSbZjgSJIkSaoNExxJkiRJtWGCI0mSJKk2THAkSZIk1cb8qgOQJEmS5pKI6Mx6Vs1s+czsSBzdxhYcSZIk6QmUmTN+jI6OzngddWWCI0mSJKk2THAkSZIk1YYJjiRJkqTaMMGRJEmSVBsmOJIkSZJqwwRHkiRJUm2Y4EiSJEmqDRMcSZIkSbUxv+oAJEmaCzrxy+X+arkkTc8WHEmSngD+arkkPTFMcCRJkiTVhgmOJEmSpNowwZEkSZJUGyY4kiRJXW5kZITFixezbNkyFi9ezMjISNUhSV3LUdQkSZK62MjICIODgwwPD7Nnzx7mzZvHwMAAAP39/RVHJ3UfW3AkSZK62NDQEMPDw/T19TF//nz6+voYHh5maGio6tCkrtRWghMR50bEHRGxLSJWTlHuFRGREbGkfL4wIh6OiFvLxwc7FbjU7SJixo/vr/qtGa9DkjS7jY2NsXTp0v3mLV26lLGxsYoikrrbtAlORMwDrgVeBpwG9EfEaROUOwp4C/DNlpfuyswzysfFHYhZmhVm+nsV/u6FJAmgp6eHjRs37jdv48aN9PT0VBSR1N3aacE5E9iWmdsz81FgLXDBBOXeCawCft7B+CRJkua0wcFBBgYGGB0dZffu3YyOjjIwMMDg4GDVoUldqZ1BBk4A7m16vgM4q7lARDwfOCkz10fEpS3LnxoR3wZ+Crw9M7/SuoGIWA4sB1iwYAGNRqP9d1BTO3fudD/IeiDAeqCC9WDuOv7443nNa17DG97wBn7wgx9w8skn89rXvpbjjz/eOjGHeU6Y3IxHUYuIJwF/Blw0wcv/CpycmfdHxAuAGyLiVzLzp82FMnM1sBpgyZIl2dvbO9OwZr1Go4H7QdYDgfVABevB3Nbb28s73/lO64H2si5Mrp0uavcBJzU9P7GcN+4oYDHQiIh7gF8D1kXEksx8JDPvB8jMW4C7gOd0InBJkiRJatVOgrMJWBQRp0bE4cCFwLrxFzPzocw8NjMXZuZC4BvA+Zm5OSKOKwcpICKeDSwCtnf8XUiSJEkSbXRRy8zdEXEJcBMwD1iTmbdHxNXA5sxcN8XiLwGujohdwGPAxZn5QCcClyRJkqRWbd2Dk5kbgA0t866YpGxv0/Rngc/OID5JkiRJaltbP/QpSZIkSbOBCY4kSZKk2jDBkSRJklQbJjiSJEmSasMER5IkSVJtmOBIkiRJqg0THEmSJEm1YYIjSZIkqTZMcCRJkiTVhgmOJEmSpNowwZEkSZJUGyY4kiRJkmrDBEeSJElSbZjgSJIkSaoNExxJkiRJtWGCI0mSJKk2THAkSZIk1YYJjiRJkqTaMMGRJEmSVBsmOJIkSZJqwwRHkiRJUm2Y4EiSJEmqDRMcSZIkSbVhgiNJkiSpNkxwJEmSJNWGCY4kSZKk2jDBkSRJklQbJjiSJEmSasMER5IkSVJtmOBIkiRJqg0THEmSJEm1YYIjSZIkqTZMcCRJkiTVhgmOJEmSpNowwZEkSZJUGyY4kiRJkmrDBEeSJElSbZjgSJIkSaqNthKciDg3Iu6IiG0RsXKKcq+IiIyIJU3zLi+XuyMiXtqJoCVJkiRpIvOnKxAR84BrgbOBHcCmiFiXmVtbyh0FvAX4ZtO804ALgV8BngX8Q0Q8JzP3dO4tSJIkSVKhnRacM4Ftmbk9Mx8F1gIXTFDuncAq4OdN8y4A1mbmI5l5N7CtXJ8kSZIkddy0LTjACcC9Tc93AGc1F4iI5wMnZeb6iLi0ZdlvtCx7QusGImI5sBxgwYIFNBqNtoKvs507d7ofZD0QYD1QwXogsB5oH+vC5NpJcKYUEU8C/gy46GDXkZmrgdUAS5Ysyd7e3pmGNes1Gg3cD7IeCKwHKlgPBNYD7WNdmFw7Cc59wElNz08s5407ClgMNCIC4D8A6yLi/DaWlSRJkqSOaecenE3Aoog4NSIOpxg0YN34i5n5UGYem5kLM3MhRZe08zNzc1nuwoh4ckScCiwCvtXxdyFJkiRJtNGCk5m7I+IS4CZgHrAmM2+PiKuBzZm5boplb4+ITwFbgd3AHziCmiRJkqRDpa17cDJzA7ChZd4Vk5TtbXk+BAwdZHySJEmS1La2fuhT0hNrZGSExYsXs2zZMhYvXszIyEjVIUmSJM0KMx5FTVJnjYyMMDg4yPDwMHv27GHevHkMDAwA0N/fX3F0kiRJ3c0WHKnLDA0NMTw8TF9fH/Pnz6evr4/h4WGGhuzpKUnSXGcvj+nZgiN1mbGxMZYuXbrfvKVLlzI2NlZRRJIkqRvYy6M9tuBIXaanp4eNGzfuN2/jxo309PRUFJEkSeoG9vJojwmO1GUGBwcZGBhgdHSU3bt3Mzo6ysDAAIODg1WHJkmSKmQvj/bYRU3qMuNNzCtWrGBsbIyenh6GhoZsepYkaY4b7+XR19e3d569PB7PFhypC/X397NlyxZuvvlmtmzZYnIjSZLs5dEmW3AkSZKkWcBeHu0xwZEkSZJmif7+fvr7+2k0GvT29lYdTleyi5okSZKk2jDBkSRJklQbJjiSJEmSasMER5IkSVJtmOBIkiRJqg0THEmSJEm1YYIjSZIkqTZMcCRJkiTVhgmOJEmSpNowwZEkSZJUGyY4kiRJkmrDBEeSJElSbZjgSJIkSaoNExxJkiRplhgZGWHx4sUsW7aMxYsXMzIyUnVIXWd+1QFIkiRJmt7IyAiDg4MMDw+zZ88e5s2bx8DAAKyIq9gAABKXSURBVAD9/f0VR9c9bMGRJEmSZoGhoSGGh4fp6+tj/vz59PX1MTw8zNDQUNWhdRUTHEmSJGkWGBsbY+nSpfvNW7p0KWNjYxVF1J1McCRJkqRZoKenh40bN+43b+PGjfT09FQUUXcywZEkSZJmgcHBQQYGBhgdHWX37t2Mjo4yMDDA4OBg1aF1FQcZkCRJkmaB8YEEVqxYwdjYGD09PQwNDTnAQAsTHEmSJGmW6O/vp7+/n0ajQW9vb9XhdCW7qEmSJEmqDRMcSZIkSbVhgiNJkiSpNkxwJEmSJNWGCY4kSZKk2jDBkSRJklQbbSU4EXFuRNwREdsiYuUEr18cEd+NiFsjYmNEnFbOXxgRD5fzb42ID3b6DUiSJEnSuGl/Byci5gHXAmcDO4BNEbEuM7c2FftEZn6wLH8+8GfAueVrd2XmGZ0NW5IkSZIer50WnDOBbZm5PTMfBdYCFzQXyMyfNj19KpCdC1GSJEmS2hOZU+ciEfFK4NzMfGP5/HXAWZl5SUu5PwD+EDgc+M3M/F5ELARuB+4Efgq8PTO/MsE2lgPLARYsWPCCtWvXzvBtzX47d+7kyCOPrDoMVcx6ILAeqGA9EFgPtI91Afr6+m7JzCWt86ftotauzLwWuDYiXg28HXg98K/AyZl5f0S8ALghIn6lpcWHzFwNrAZYsmRJ9vb2diqsWavRaOB+kPVAYD1QwXogsB5oH+vC5NrponYfcFLT8xPLeZNZC7wcIDMfycz7y+lbgLuA5xxcqJIkSZI0tXZacDYBiyLiVIrE5kLg1c0FImJRZn6vfHoe8L1y/nHAA5m5JyKeDSwCtk+1sVtuueXHEfH9A3sbtXQs8OOqg1DlrAcC64EK1gOB9UD7WBfglIlmTpvgZObuiLgEuAmYB6zJzNsj4mpgc2auAy6JiP8M7AIepOieBvAS4OqI2AU8BlycmQ9Ms73j2n1HdRYRmyfqU6i5xXogsB6oYD0QWA+0j3Vhcm3dg5OZG4ANLfOuaJp+yyTLfRb47EwClCRJkqR2tfVDn5IkSZI0G5jgdK/VVQegrmA9EFgPVLAeCKwH2se6MIlpfwdHkiRJkmYLW3AkSZIk1YYJjiRJkqTaMMGRJEmSVBsmOBOIiMGIuD0ivhMRt0bElRHx7pYyZ0TEWDl9ZER8KCLuiohbIqIREWdNsf495Xpvi4h/iogXdTj+6yLileX0RyLitE6uv+4O9fE/yJhGynjeGhFXl787RbmtJeX0/+rkNlu2f1FEfOBQrX+mmj5Tt5efq/8ZEQd1fmvev5O8fnFE/O7BRwsRcXoZ760R8UBE3F1O/8NM1jvJtq6KiPvK9f9zRPzVwe6bSda/MCK2lNNLIuL9nVp3m9v32HdQRPx6uS9vjYgTIuIz5fzeiPh803RH/2+1xHBPRBw7g+V3Nk3/l4i4MyJOaSlzT0R8tun5KyPiumnWO239bv48TPDa3vN1VcrvB+N17p8j4soOr7+5npwfESs7uf42Y/D4d1BE/E5EjEXEaPM+aP5eEBEvP5TfNZuPabva+h2cuSQiXgj8FvD8zHykPMmeBlwHXN5U9EJgpJz+CHA3sCgzH4uIU8tlJvNwZp5Rbu+lwLuB3+joGyll5hsPxXrr6gk6/gca038A/p/M/I/TFP1fwB8f4LrnZeaegw6uezR/pp4JfAJ4GnDA/7ybf+Nrktc/eFAR7r+O7wLj8V4HfD4zP9NcJiLmZ+bumW6r9L7M/NPyi/+XKc43ox1a916ZuRnY3On1TsNj31mvAd6dmR8rn79ygjK9wE7ga+2utIr3FBHLgPcDL83M709Q5AURcVpmbm1nfRXV7ylFRFAMGPXYASx2aWZ+JiKeAmyNiL/JzLs7HVv5Q/DrOr3ednn8O2YAeFNmbiyfT7QPXg58HmhrX8KhPyfYgvN4xwM/zsxHADLzx5n5ZeDB2P+q/KuAkYj4JeAs4O3jFSwz787M9W1u72nAg7C3JeDmKFp1vhsRF5TznxoR68urk1si4r+X818QEf8YRavBTRFxfOvKY/8r/DsjYqhczzciYkE5/7iI+GxEbCofLz7w3VYbh/T4t17diYi3RcRV5XQjIlZFxLfKK06/Xhb7AnBCecXt16Opha5pPdcAR5RlPl7Oe225rlujaGGaV87fGRHvjYjbgBdOUe73yji+BcyaOpGZPwSWA5dEYV5EvKes29+JiN8fLxsRl5WftdvKfdjaAnpNRGwtl/vTct5VEfG2cvqM8rP0nYj4u4g4ppw/2bGcUrncn0fEZuAtk33GI+KXIuLGcv5XIuKX29w9hwNPYd85503lfrmtPAf8Qjn/d8pzzW0R8eVy3qT7sSn+5qu3V0XEmvI9bY+INzeVm7DOzZTH/nHr3O9cEeVV0PI4NSLiM1Fcxf94ub/eSHFue2c573FXoyNiIXAx8NbYd06a8H9Iub8+GhFfBT46RblfjIgvRNFy9BEg2tln0+zPlwAfBn4rM++apNh7gcEJln1qWXe/FRHfjn3/i5vr93ER8cXxmCPi+7Gv1WleRHy4fO0LEXFE0+pfV+63LRFxZrmuZ0TEDWVd+kZE/GrT/ntbU1xbymOyMCLuiIi/AbYAJ5XHektZp9/a5m56Svn3Z+X6ryiPy5aIWB0RUc5/c9NnYe1U+6hlPzZf4b8uIt4fEV+L4nzQXC8vjX2f0Xe0GfuUPP6T7pfm74THRsQ95fRFEfG3UZxbvhcRf1LOvwJYCgxHcS7duw+a1vki4HzgPeV7+6WY5DxVxvnBiPgm8CdTlDs1Ir5evp93TfZ+ppSZPpoewJHArcCdwF8Cv1HOfxvFVVCAXwM2l9PnA393gNvYU27jn4GHgBeU8+cDTyunjwW2UZzoXwF8uGn5o4HDKK6eHVfO++/AmnL6OuCV5XQDWFJOJ/Bfy+k/ofhSDsUVz6Xl9MnAWNXHoa7HH1gIbGl6/jbgqqZj9d5y+r8A/zDJMpMd351NZXqAzwGHlc//EvjdpnrwqqnKUSR6PwCOo/hS/FXgA1Ufnyn2684J5v0EWEDxhXe8rj+Z4urTqcDLys/QL5SvPaN5/wK/CNwBe4fTf3r59yrgbeX0d5rqyNXAn091LCeJvfV4/mU5PdVn/GaKFkMoEuwvTbH+q4D7KOr1g8Anml77xabpdwEryunvAie0vO/J9uPe+klxZf/zTdv9Wln2WOD+8j1NWjc99h0/9nvX37yvyuP0EHAixYXOr7Pvf0BzTFMd27c1rXfC/yFluVuAI6Yp937ginL6PIpz1LEzqBO7gAeAX52izD1lHRkD/mN53K8rX/tj4LXjx57i/8FTW/bBB4DLy+lzx2Mu99lu4IzytU81ratB+b8ceEnTvv0L4Mpy+jeBWyfZz1vK9S8EHgN+rZz/AuCLTeWePk2duJvifLAT+OOm157RNP1R9n1f+BfgyS2fhXb20UWU/zfK7X6aor6dBmwr559D8VsuUb72eeAlB3vsPf7THv8G+74zHAvc03SstlN8v3wK8H3gpAmWmer4Np9rJjxPleU+D8ybptw69n1n+QMmOM9P97CLWovM3BkRLwB+HegDPhlFH9JPAl+LiP/J/t2TDkZzl4oXAn8TEYspPuB/XF55eAw4geID+F3gvRGxiqJifaUsvxj4YnmRZR7wr9Ns91GKigXFP52zy+n/DJxWrgfgaRFxZGYecJ/H2e4JOv5T+dvy7y0UJ7GDtYzipLepPK5HAD8sX9sDfHaacmcBjcz8EUBEfBJ4zgziqdI5wK82XTE8GlhEUe//OjP/HSAzH2hZ7iHg5xRXrj7Pvs8OABFxNMU/kn8sZ11P8Q983MEey0+Wf/8TE3zGI+JI4EXAp5s+s0+eZp3jXdQOAz4TERdm5lpgcXl17OkUyf1NZfmvAtdFxKea3sdk+/HOKba7PovW0Eci4ocU57Op6maneewn963M3AEQEbeWcW6cconJTfg/pJxel5kPT1PuJcB/A8jM9RHx4EHGMW4XRYI4ALxlinJ7gPdQdD/++6b55wDnN109fwpFQtZsKfDbZcw3tsR8d2beWk631oGRcpkvR8TTIuLp5bpeUc7/UhQtWk+b5j1+PzO/UU5vB54dEX8BrKdo9Z/KeBe1I4GbI+JFmfk1oC8i/gj4BeAZwO0UFyO+A3w8Im4AbijX0c4+anVDFj0dtkbZg6RczznAt8vnR1J8Rr88zbqm4vE/ODdn5kMAEbEVOAW490BX0sZ56tOZuWeaci+m3CcUyfaqA43DBGcCWdyT0AAaEfFd4PWZeV1E3E3Rd/0VwAvL4rcDz42DvJchM78eRbPmcRRX+46jaNHZVTYdPiUz74yI55evvysibgb+Drg9M1842bonsCvLdJjigz1+/J9EcSXg5wcafx0d4uO/m/27hj6l5fVHyr/Nx+dgBHB9Zl4+wWs/b4p1wnIR8fIZbLtyEfFsin34Q4r3uCIzb2op89Kp1pGZu6PoQrCM4ureJRRX19p1sMfyZ+MhMsFnvPzH95PxiyQHojyv3EjxhXItxdW0l2fmbRFxEcXVOTLz4ii6ZJ4H3FIm/ZPtx4VTbPKRpunx/TBV3Zwxj/1+9p5vorj/6vAJYjyYOFtN+D+k/NLyszbLddJjFF3tbo5i8JVVFF80oUi4mu+1+ijFF9zmrngBvCIz72iJcwHtad23zV2UsqVs6/NmU/2/2LtfM/PBiHgu8FKK7oOvAt4wXZDlBb0GsDQi/omiNXVJZt4bRdfp8e2dR3HO+K/AYESczsHto+b9Ek1/352ZH5ou3gPg8Z/8+Devc7LvH+NxH+w54UlMfZ76WZvlpto3bQWhJhHxnyJiUdOsMyia6qDIvN8HbB+/8pVF387NwDsi9vZXXRgR57W5vV+muDp3P8XVxR+WX0L6KLJnIuJZwL9ncdPne4DnU3SfOK5sASIiDouIXznIt/0FYEVTTAf8xakunoDj//8Bzyyv0DyZYkCDTtlVXqGHotn3lVHcdD3ex/eUCZaZrNw3gd8o4zwM+J0OxnlIRcRxwAcpms6TolXif4zvm4h4TkQ8Ffgi8Hux776TZ7Ss50jg6MzcALwVeG7z6+WVrgdj3z0WrwP+kc6Z8DOemT8F7o6I3ynnR/nPbVplHX0xMN4n/SiKloHDKG4uHy/3S5n5zfKLwI+Ak5h8Px6oduvmAfPYP849FK1lUHSnPWzyogfk3yjqzrh2/4dMVu7LwKvLeS8DjplpgGXr3HkU9fqizDyjfFzRUm4XxXm9+b6Fm4AVTef0502wia9SfJEkIs45gJjH76FdCjxU1qWvlHESEb0U94H+lOL4Pb+c/3yK7pWPU14kfVJmfhZ4+/gy04mI+RSt9Xex78vuj8v6P34v2pMouiqNApdRfE8Zb+2dbh+14ybgDeU2iWLkvmce5Lr28vhP6h72nRMmGkDkYO09J7R7npqm3FcpestA0/+mA2ELzuMdCfxFFM2Guynug1levvZpir7CK1qWeSPFzWrbIuJh4MfApVNs44gougRAcaXg9WVz3ceBz0XRarCZ4h4dgNMpbt56jKLp9X9k5qNRdLt4fxTdJeYDf07RonCg3gxcGxHfKdfzZYqrAHPRIT3+ZfJ6NfAtivsi/nmicgdpNfCdiPinzHxNRLwd+EL5D2oXRT/W/UaSycytE5XLzG9EcQXv6xT3M9xKdxv/TB1Gcdw+CvxZ+dpHKLoI/FP5D+tHFK0WN5ZfsDZHxKPABoqR6MYdBfyfKEYaCuAPJ9ju64EPll+UtwO/16k3NM1n/DXAX5XH7jCK1pjbpljdWyPitWXZ71BcqQX4fymS2R+Vf8e/tL4nikQ/KBKS28rlFtKyHw/ifU1Y52ipmwfAYz/5sf9w+T5uA25k/9aUmfgcRVfHCyjOh+3+D5ms3DsoBm25naJr0Q86EWRmPhAR5wJfjogfZTGq10SGKb4Yjnsnxf7+TllH7+bxF6PGY34dxXny/1J8yTuSqf08Ir5NcezGr7JfBawp98u/U9QtKLoS/265X77J5N1BTwD+OvYNjz5d6+h7yvpzOMXn+28zMyPiwxQtGf8X2FSWnQd8rKyHAbw/M38SEe3so2ll5hciogf4eplP7AReSwe6rXr8J/SnwKciYjlFd7ZOWQt8OIrBZF5J++epycq9BfhERFwG/J+DCWj8BkpJkiS1IYoW+D1ld8YXAn91MN1GNTt5/LufLTiSJEkH5mSKK+FPohjA500Vx6Mnlse/y9mCc4hExC9SNP22WpaZ9z/R8eiJ5fHXEy0iBnn8vVKfzsyhKuLRE8djr1YRcS2P//2y/52Zf11FPHpiefxNcCRJkiTViKOoSZIkSaoNExxJkiRJtWGCI0mSJKk2THAkSZIk1cb/D5FnDpMk5XhXAAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<Figure size 1008x504 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7KkjlDdlifBd"
      },
      "source": [
        "This plot shows that the basline models in this experiment are much better than the models trained on the unfiltered data.\n",
        "This tells us that selecting and filtering the features increased the accuracy of our model by quite a bit.\n",
        "\n",
        "While we were confident in our hypothesis, this experiment was the one where we weren't quite sure if it was true.\n",
        "One concern we had was whether more features might mean the model has more data to use to classify the features and so might be more accurate.\n",
        "This proved to be very unture and showed how the tsfresh select_features is a massive asset to our pipeline and improves the accuracy greatly."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-20A8d866qsZ"
      },
      "source": [
        "---\n",
        "\n",
        "## Experiment 2 - Classify the sleep state using the step count column in our data Vs without the step count column\n",
        "\n",
        "\n",
        "---"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "nfdYTeozjrlz"
      },
      "source": [
        "This experiment was a verification of our intuition. The reason we wanted to test this was because during one of our lectures, Eoin had said that we don't need the step count data for this assignment. We thought that this sounded slightly counte intuative as we thouught a step count would be a great assett to our classification.\n",
        "\n",
        "Our logic behind this thought was that if somebody is asleep, they are are not going to be clocking up any steps. As a result, this feature would be an easy way to help classify if the subject is asleep or awake.\n",
        "\n",
        "We wanted to verify if what we thought was correct or if there was something we were missing in our cassification that Eoin picked up on. This experiment is that test."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "95WkQ6Uk6qsa"
      },
      "source": [
        "\n",
        "### Get the dataset and y for both the data without the step count"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 66,
          "referenced_widgets": [
            "f8c9032d01964ce090844df8aa870d2a",
            "57f593d07d6448b585f7bdbce7622b26",
            "c91b9c27330d43d6b9bb85f29db35221",
            "6d083ed68e274299b15d33bb3cc07f5a",
            "157fd78879024057b27452c3eed8c289",
            "79101dd37410416495a27b139322be1a",
            "62debe0ca82c4be792593ac2d9ba6e5a",
            "8704112e92cc47deafa2d110906694bb"
          ]
        },
        "id": "T4cbj7wE6qsb",
        "outputId": "ce9043ba-aa52-4aba-c445-f62fb1babde3"
      },
      "source": [
        "no_steps_features_all_psg_dataset_and_y = create_map_of_subject_id_to_the_dataset_and_y_for_all_psg_status(subject_id_to_no_step_filtered_extracted_feature_df_map, map_subject_id_to_a_map_of_the_psg_status_to_session_id, map_subject_id_to_a_map_of_the_session_id_to_psg_status)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "f8c9032d01964ce090844df8aa870d2a",
              "version_minor": 0,
              "version_major": 2
            },
            "text/plain": [
              "HBox(children=(FloatProgress(value=0.0, max=31.0), HTML(value='')))"
            ]
          },
          "metadata": {
            "tags": []
          }
        },
        {
          "output_type": "stream",
          "text": [
            "\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "u71hw-986qsc"
      },
      "source": [
        "### Classify the dataset and y for the data without the step count"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 66,
          "referenced_widgets": [
            "971be0d0d3374da195be95dcbbc34576",
            "f0208c2a18164458bfa5173b740ed16d",
            "ad18af4045334c568a0250dd4fe3500b",
            "f20f6eb795b04b16b51a2be8043259e8",
            "f558998c92634c13b7b2b9bf269e6354",
            "81975408dcae4f8a87dac1325e11c9a5",
            "3fbb6e2144d141f1bbd635e9a9dd7f55",
            "23f06ace008a4e078f9841493a264b9f"
          ]
        },
        "id": "fB8GS-vS6qsc",
        "outputId": "9b7d6486-3662-4ff0-9567-02c4781eba2f"
      },
      "source": [
        "optimise_params_bool = True\n",
        "classification_report_bool = False\n",
        "no_steps_results = classify(no_steps_features_all_psg_dataset_and_y, names_of_models_to_test, cv_settings, optimise_params_bool, classification_report_bool)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "971be0d0d3374da195be95dcbbc34576",
              "version_minor": 0,
              "version_major": 2
            },
            "text/plain": [
              "HBox(children=(FloatProgress(value=0.0, max=31.0), HTML(value='')))"
            ]
          },
          "metadata": {
            "tags": []
          }
        },
        {
          "output_type": "stream",
          "text": [
            "\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "vxGrsO3fr4dW"
      },
      "source": [
        "### Analyse these classification results"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "IgN64NHjvV4Q"
      },
      "source": [
        "##### Output the results of the score each classifier got for each subject"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 917
        },
        "id": "030kiFAw6qsd",
        "outputId": "1a9c3e74-a00d-4b14-8fe3-385716a1b994"
      },
      "source": [
        "no_steps_results"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>subject_id</th>\n",
              "      <th>SVC</th>\n",
              "      <th>Decision Tree</th>\n",
              "      <th>K-Neighbours</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>46343</td>\n",
              "      <td>0.524590</td>\n",
              "      <td>0.557377</td>\n",
              "      <td>0.453552</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>759667</td>\n",
              "      <td>0.554140</td>\n",
              "      <td>0.611465</td>\n",
              "      <td>0.503185</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>781756</td>\n",
              "      <td>0.589506</td>\n",
              "      <td>0.481481</td>\n",
              "      <td>0.561728</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>844359</td>\n",
              "      <td>0.548822</td>\n",
              "      <td>0.531987</td>\n",
              "      <td>0.535354</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>1066528</td>\n",
              "      <td>0.523962</td>\n",
              "      <td>0.565495</td>\n",
              "      <td>0.533546</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>5</th>\n",
              "      <td>1360686</td>\n",
              "      <td>0.620915</td>\n",
              "      <td>0.666667</td>\n",
              "      <td>0.637255</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>6</th>\n",
              "      <td>1449548</td>\n",
              "      <td>0.685714</td>\n",
              "      <td>0.666667</td>\n",
              "      <td>0.666667</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>7</th>\n",
              "      <td>1455390</td>\n",
              "      <td>0.718354</td>\n",
              "      <td>0.715190</td>\n",
              "      <td>0.699367</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>8</th>\n",
              "      <td>1818471</td>\n",
              "      <td>0.561514</td>\n",
              "      <td>0.536278</td>\n",
              "      <td>0.555205</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>9</th>\n",
              "      <td>2598705</td>\n",
              "      <td>0.590476</td>\n",
              "      <td>0.546032</td>\n",
              "      <td>0.523810</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>10</th>\n",
              "      <td>2638030</td>\n",
              "      <td>0.709265</td>\n",
              "      <td>0.638978</td>\n",
              "      <td>0.648562</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>11</th>\n",
              "      <td>3509524</td>\n",
              "      <td>0.601449</td>\n",
              "      <td>0.710145</td>\n",
              "      <td>0.681159</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>12</th>\n",
              "      <td>3997827</td>\n",
              "      <td>0.643533</td>\n",
              "      <td>0.738170</td>\n",
              "      <td>0.637224</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>13</th>\n",
              "      <td>4018081</td>\n",
              "      <td>0.606061</td>\n",
              "      <td>0.678788</td>\n",
              "      <td>0.606061</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>14</th>\n",
              "      <td>4314139</td>\n",
              "      <td>0.569182</td>\n",
              "      <td>0.547170</td>\n",
              "      <td>0.575472</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>15</th>\n",
              "      <td>4426783</td>\n",
              "      <td>0.638889</td>\n",
              "      <td>0.617284</td>\n",
              "      <td>0.620370</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>16</th>\n",
              "      <td>5132496</td>\n",
              "      <td>0.564935</td>\n",
              "      <td>0.584416</td>\n",
              "      <td>0.603896</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>17</th>\n",
              "      <td>5498603</td>\n",
              "      <td>0.573171</td>\n",
              "      <td>0.609756</td>\n",
              "      <td>0.573171</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>18</th>\n",
              "      <td>5797046</td>\n",
              "      <td>0.603226</td>\n",
              "      <td>0.609677</td>\n",
              "      <td>0.600000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>19</th>\n",
              "      <td>6220552</td>\n",
              "      <td>0.574603</td>\n",
              "      <td>0.565079</td>\n",
              "      <td>0.593651</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>20</th>\n",
              "      <td>7749105</td>\n",
              "      <td>0.536585</td>\n",
              "      <td>0.829268</td>\n",
              "      <td>0.560976</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>21</th>\n",
              "      <td>8000685</td>\n",
              "      <td>0.589905</td>\n",
              "      <td>0.589905</td>\n",
              "      <td>0.596215</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>22</th>\n",
              "      <td>8173033</td>\n",
              "      <td>0.566456</td>\n",
              "      <td>0.601266</td>\n",
              "      <td>0.585443</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>23</th>\n",
              "      <td>8530312</td>\n",
              "      <td>0.624204</td>\n",
              "      <td>0.570064</td>\n",
              "      <td>0.627389</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>24</th>\n",
              "      <td>8686948</td>\n",
              "      <td>0.632911</td>\n",
              "      <td>0.566456</td>\n",
              "      <td>0.585443</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>25</th>\n",
              "      <td>8692923</td>\n",
              "      <td>0.579288</td>\n",
              "      <td>0.631068</td>\n",
              "      <td>0.546926</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>26</th>\n",
              "      <td>9106476</td>\n",
              "      <td>0.602524</td>\n",
              "      <td>0.605678</td>\n",
              "      <td>0.630915</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>27</th>\n",
              "      <td>9961348</td>\n",
              "      <td>0.600840</td>\n",
              "      <td>0.613445</td>\n",
              "      <td>0.592437</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "   subject_id       SVC  Decision Tree  K-Neighbours\n",
              "0       46343  0.524590       0.557377      0.453552\n",
              "1      759667  0.554140       0.611465      0.503185\n",
              "2      781756  0.589506       0.481481      0.561728\n",
              "3      844359  0.548822       0.531987      0.535354\n",
              "4     1066528  0.523962       0.565495      0.533546\n",
              "5     1360686  0.620915       0.666667      0.637255\n",
              "6     1449548  0.685714       0.666667      0.666667\n",
              "7     1455390  0.718354       0.715190      0.699367\n",
              "8     1818471  0.561514       0.536278      0.555205\n",
              "9     2598705  0.590476       0.546032      0.523810\n",
              "10    2638030  0.709265       0.638978      0.648562\n",
              "11    3509524  0.601449       0.710145      0.681159\n",
              "12    3997827  0.643533       0.738170      0.637224\n",
              "13    4018081  0.606061       0.678788      0.606061\n",
              "14    4314139  0.569182       0.547170      0.575472\n",
              "15    4426783  0.638889       0.617284      0.620370\n",
              "16    5132496  0.564935       0.584416      0.603896\n",
              "17    5498603  0.573171       0.609756      0.573171\n",
              "18    5797046  0.603226       0.609677      0.600000\n",
              "19    6220552  0.574603       0.565079      0.593651\n",
              "20    7749105  0.536585       0.829268      0.560976\n",
              "21    8000685  0.589905       0.589905      0.596215\n",
              "22    8173033  0.566456       0.601266      0.585443\n",
              "23    8530312  0.624204       0.570064      0.627389\n",
              "24    8686948  0.632911       0.566456      0.585443\n",
              "25    8692923  0.579288       0.631068      0.546926\n",
              "26    9106476  0.602524       0.605678      0.630915\n",
              "27    9961348  0.600840       0.613445      0.592437"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 53
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "leGMP8mEvXoj"
      },
      "source": [
        "##### Describe this dataframe better with summary statistics"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 297
        },
        "id": "aWmkyvWT6qsd",
        "outputId": "3f6e3450-a387-4ea9-8980-06b9f0d0269d"
      },
      "source": [
        "no_steps_results.describe()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>SVC</th>\n",
              "      <th>Decision Tree</th>\n",
              "      <th>K-Neighbours</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>count</th>\n",
              "      <td>28.000000</td>\n",
              "      <td>28.000000</td>\n",
              "      <td>28.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>mean</th>\n",
              "      <td>0.597679</td>\n",
              "      <td>0.613759</td>\n",
              "      <td>0.590535</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>std</th>\n",
              "      <td>0.049369</td>\n",
              "      <td>0.073333</td>\n",
              "      <td>0.054633</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>min</th>\n",
              "      <td>0.523962</td>\n",
              "      <td>0.481481</td>\n",
              "      <td>0.453552</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>25%</th>\n",
              "      <td>0.566076</td>\n",
              "      <td>0.565391</td>\n",
              "      <td>0.559533</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>50%</th>\n",
              "      <td>0.590191</td>\n",
              "      <td>0.607678</td>\n",
              "      <td>0.593044</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>75%</th>\n",
              "      <td>0.621737</td>\n",
              "      <td>0.645900</td>\n",
              "      <td>0.628270</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>max</th>\n",
              "      <td>0.718354</td>\n",
              "      <td>0.829268</td>\n",
              "      <td>0.699367</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "             SVC  Decision Tree  K-Neighbours\n",
              "count  28.000000      28.000000     28.000000\n",
              "mean    0.597679       0.613759      0.590535\n",
              "std     0.049369       0.073333      0.054633\n",
              "min     0.523962       0.481481      0.453552\n",
              "25%     0.566076       0.565391      0.559533\n",
              "50%     0.590191       0.607678      0.593044\n",
              "75%     0.621737       0.645900      0.628270\n",
              "max     0.718354       0.829268      0.699367"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 54
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "DnwCo_KxvZCp"
      },
      "source": [
        "##### Plot these summary statistics to better visualise them"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 282
        },
        "id": "duxZgk0G6qse",
        "outputId": "e7f4c6bd-61e9-4541-867c-bd623f7f4d08"
      },
      "source": [
        "no_steps_results.boxplot()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<matplotlib.axes._subplots.AxesSubplot at 0x7fe6f3374588>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 55
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD4CAYAAADiry33AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAarElEQVR4nO3df5RcZZ3n8ffHDogjysQJ9roBSZgNY0OjKD1hGTJOtxlCdpwDenDd9LAOzLZGdjcZFx1OwObwI9BHGNZhFjf+iHTE8Zzp6OhutkeyREa6xPhr0iwoSXqDIeiS7OwRCeC0MkA33/3j3sabspOq7qru6nr68zqnTt/73Oe5/b11q75167l176OIwMzM0vWKRgdgZmYzy4nezCxxTvRmZolzojczS5wTvZlZ4hY0OoByixYtiiVLljQ6jBnz85//nFe/+tWNDsOmyfuveaW+7x588MGfRsTJky2bc4l+yZIlDA8PNzqMGVMqlejs7Gx0GDZN3n/NK/V9J+nHR1vmrhszs8Q50ZuZJc6J3swscU70ZmaJc6I3M0ucE71ZFQYGBmhvb2flypW0t7czMDDQ6JDMqjbnfl5pNtcMDAzQ29tLf38/4+PjtLS00NPTA0B3d3eDozOrzEf0ZhX09fXR399PV1cXCxYsoKuri/7+fvr6+hodmllVnOjNKhgZGWHFihVHlK1YsYKRkZEGRWQ2NU70ZhW0tbWxc+fOI8p27txJW1tbgyIymxonerMKent76enpYWhoiLGxMYaGhujp6aG3t7fRoZlVxSdjzSqYOOG6fv16RkZGaGtro6+vzydirWk40ZtVobu7m+7u7uRvjGVpcteNmVninOjNzBLnRG9mljgnejOzxDnRm5klzonezCxxTvRmZomrKtFLWi1pn6T9kq6ZZPkbJQ1JekjSDyT9QWHZtXm7fZIuqmfwZmZWWcULpiS1AJuAC4GDwC5JgxGxt1DtOuBLEfEpSWcC24El+fQa4CzgnwN/J+mMiBiv94aYmdnkqjmiXw7sj4gDEfECsBW4pKxOAK/Np08C/m8+fQmwNSKej4jHgf35+szMbJZUk+gXA08U5g/mZUU3Av9W0kGyo/n1U2hrZmYzqF73uukG7o6Ij0s6H/iCpPZqG0taC6wFaG1tpVQq1SmsuWd0dDTp7Uud91/zms/7rppEfwg4tTB/Sl5W1AOsBoiI70g6AVhUZVsiYjOwGaCjoyNSvmmUb4rV3Lz/mtd83nfVdN3sApZJWirpeLKTq4Nldf4PsBJAUhtwAvBkXm+NpFdKWgosA/6+XsGbmVllFY/oI2JM0jpgB9ACbImIPZI2AsMRMQh8BPispKvITsxeEREB7JH0JWAvMAb8R//ixsxsdlXVRx8R28lOshbLri9M7wUuOErbPsCjKJuZNYivjDUzS5wTvZlZ4pzozcwS50RvZpY4J3ozs8Q50ZuZJc6J3swscU70ZmaJc6I3M0ucE72ZWeKc6M3MEudEb2aWOCd6M7PEOdGbmSXOid7MLHFO9GZmiXOiNzNLnBO9mVniqkr0klZL2idpv6RrJll+h6SH88ejkp4pLBsvLCsfVNzMzGZYxTFjJbUAm4ALgYPALkmD+TixAETEVYX664G3FlbxXEScU7+QzcxsKqo5ol8O7I+IAxHxArAVuOQY9buBgXoEZ2Zmtat4RA8sBp4ozB8EzpusoqTTgKXA/YXiEyQNA2PArRGxbZJ2a4G1AK2trZRKpaqCb0ajo6NJb1/qvP+a13zed9Uk+qlYA3w5IsYLZadFxCFJpwP3S3okIh4rNoqIzcBmgI6Ojujs7KxzWHNHqVQi5e1Lnfdf85rP+66arptDwKmF+VPyssmsoazbJiIO5X8PACWO7L83M7MZVk2i3wUsk7RU0vFkyfxXfj0j6U3AQuA7hbKFkl6ZTy8CLgD2lrc1M7OZU7HrJiLGJK0DdgAtwJaI2CNpIzAcERNJfw2wNSKi0LwN+Iykl8g+VG4t/lrHzMxmXlV99BGxHdheVnZ92fyNk7T7NnB2DfGZmVmN6n0y1qxpSarLeo78UmvWeL4FglkuIio+Ttvw1Yp1zOYaJ3ozs8Q50ZuZJc6J3swscU70ZmaJc6I3M0ucE72ZWeKc6GfJwMAA7e3trFy5kvb2dgYGfCdnM5sdvmBqFgwMDNDb20t/fz/j4+O0tLTQ09MDQHd3d4OjM7PU+Yh+FvT19dHf309XVxcLFiygq6uL/v5++vr6Gh2amc0DTvSzYGRkhBUrVhxRtmLFCkZGRhoUkZnNJ+66mQVtbW3cdNNNbNu2jZGREdra2njXu95FW1tbo0Mzs3nAiX4WdHV1cdttt3Hbbbdx5plnsnfvXjZs2MCVV17Z6NDMbB5wop8FQ0NDbNiwgS1btrx8RL9hwwa2bfuV4XPNzOrOiX4WjIyM8NBDD3HLLbe8PG7liy++yMc+9rFGh2Zm84BPxs6CtrY2du7ceUTZzp073UdvZrOiqkQvabWkfZL2S7pmkuV3SHo4fzwq6ZnCsssl/TB/XF7P4JtFb28vPT09DA0NMTY2xtDQED09PfT29jY6NDObByp23UhqATYBFwIHgV2SBotjv0bEVYX664G35tOvA24AOoAAHszbPl3XrZjjJi6KWr9+/ct99H19fb5YysxmRTVH9MuB/RFxICJeALYClxyjfjcwcX3/RcB9EXE4T+73AatrCbhZdXd3s3v3br7+9a+ze/duJ3kzmzXVnIxdDDxRmD8InDdZRUmnAUuB+4/RdvEk7dYCawFaW1splUpVhNWcRkdHk96++cD7rznN5/devX91swb4ckSMT6VRRGwGNgN0dHREZ2dnncOaOyZ+dWNN6t57vP/mKA/ufnTVdN0cAk4tzJ+Sl01mDb/stplqWzOzaavHwO4pJnmoLtHvApZJWirpeLJkPlheSdKbgIXAdwrFO4BVkhZKWgisysvMzGyWVOy6iYgxSevIEnQLsCUi9kjaCAxHxETSXwNsjcJHYkQclnQz2YcFwMaIOFzfTTAzs2Opqo8+IrYD28vKri+bv/EobbcAW6YZn5mZ1chXxpqZJc6J3swscb6pWZ35J15mNtf4iL7O/BMvM5trnOjNzBLnRG9mljgnejOzxDnRm5klzonezCxxTvRmZolzojczS5wTvZlZ4pzozcwS50RvZpY4J3ozs8Q50ZuZJc6J3swscVUlekmrJe2TtF/SNUep815JeyXtkfTXhfJxSQ/nj18Za9bMzGZWxfvRS2oBNgEXAgeBXZIGI2Jvoc4y4Frggoh4WtLrC6t4LiLOqXPcZmZWpWqO6JcD+yPiQES8AGwFLimr8wFgU0Q8DRARP6lvmGZmNl3VJPrFwBOF+YN5WdEZwBmSviXpu5JWF5adIGk4L39XjfGamdkU1WsowQXAMqATOAV4QNLZEfEMcFpEHJJ0OnC/pEci4rFiY0lrgbUAra2tlEqlOoU1N6W+fanz/mte83XfVZPoDwGnFuZPycuKDgLfi4gXgcclPUqW+HdFxCGAiDggqQS8FTgi0UfEZmAzQEdHR3R2dk59S5rFvfeQ9Palzvuvec3jfVdNot8FLJO0lCzBrwH+qKzONqAb+JykRWRdOQckLQR+ERHP5+UXAH9et+jNpuAtN32NZ597seb1LLnmnpran/Sq4/j+DatqjsOsWhUTfUSMSVoH7ABagC0RsUfSRmA4IgbzZask7QXGgasj4ilJvwN8RtJLZOcDbi3+WsdsNj373Iv86NZ31rSOUqlU81FhrR8UZlNVVR99RGwHtpeVXV+YDuDD+aNY59vA2bWHaWZm0+UrY83MEudEb2aWOCd6M7PEOdGbmSXOid7MLHFO9GZmiavXLRDMzGZUPS54m68XuznRm1lTqPWCt/l8sZu7bszMEucj+iny10czazZO9FPkr49m1mzcdWNmljgnejOzxDnRm5klzonezCxxTvRmZolzojczS5x/XmnzxmvaruHsz19T+4o+X2scALUNaWg2FVUlekmrgf9CNmbsXRFx6yR13gvcCATw/Yj4o7z8cuC6vNotEVHj28Rsev5x5FaPGWvzUsVEL6kF2ARcCBwEdkkaLA7yLWkZcC1wQUQ8Len1efnrgBuADrIPgAfztk/Xf1PMzGwy1fTRLwf2R8SBiHgB2ApcUlbnA8CmiQQeET/Jyy8C7ouIw/my+4DV9QndzMyqUU3XzWLgicL8QeC8sjpnAEj6Fln3zo0Rce9R2i4u/weS1gJrAVpbWymVSlWG3xi1xDc6OlqX7Zvrz9FcVevz5v3XWH7vTU+9TsYuAJYBncApwAOSzq62cURsBjYDdHR0RK19oDPq3ntq6qOtRx9vrTHMW3V43rz/GsjvvWmrpuvmEHBqYf6UvKzoIDAYES9GxOPAo2SJv5q2ZmY2g6pJ9LuAZZKWSjoeWAMMltXZRnY0j6RFZF05B4AdwCpJCyUtBFblZWZmNksqdt1ExJikdWQJugXYEhF7JG0EhiNikF8m9L3AOHB1RDwFIOlmsg8LgI0RcXgmNsTM0laX6yDm6TUQVfXRR8R2YHtZ2fWF6QA+nD/K224BttQW5tzhF5tZY9R6HcR8vgbCV8ZOkV9sZtZsfK8bM7PEOdGbmSXOid7MLHFO9GZmiXOiNzNLnBO9mVninOjNzBLnRG9mljgnejOzxDnRm5klzonezCxxvtfNNNR8r5l7a2t/0quOq+3/z2N1uU+Q9581GSf6KarlhmaQJZpa12HTU4/n3fvPmpG7bszMEucjejNrGu42nR4nejNrCu42nb6qum4krZa0T9J+Sb8yvJKkKyQ9Kenh/PH+wrLxQnn5WLNmZjbDKh7RS2oBNgEXAgeBXZIGI2JvWdUvRsS6SVbxXEScU3uoZmY2HdUc0S8H9kfEgYh4AdgKXDKzYZmZWb1U00e/GHiiMH8QOG+SepdKejvwKHBVREy0OUHSMDAG3BoR28obSloLrAVobW2lVCpVvwVNKPXtS533X/Oar/uuXidj/xYYiIjnJX0Q+DzwjnzZaRFxSNLpwP2SHomIx4qNI2IzsBmgo6Mjah08e067956aBwe3BvL+a17zeN9V03VzCDi1MH9KXvayiHgqIp7PZ+8Czi0sO5T/PQCUgLfWEK+ZmU1RNYl+F7BM0lJJxwNrgCN+PSPpDYXZi4GRvHyhpFfm04uAC4Dyk7hmZjaDKnbdRMSYpHXADqAF2BIReyRtBIYjYhD4U0kXk/XDHwauyJu3AZ+R9BLZh8qtk/xax8zMZlBVffQRsR3YXlZ2fWH6WuDaSdp9Gzi7xhjNzKwGvteNmVninOjNzBLnRG9mljgnejOzxDnRm5klzonezCxxTvRmZolzojczS5xHmKozSZXr3FZ5PRFRh2jMzHxEX3cRcczH0NBQxTpO8mZWT070ZmaJc6I3M0ucE72ZWeKc6M3MEudEb2aWOCd6M7PEOdGbmSWuqkQvabWkfZL2S7pmkuVXSHpS0sP54/2FZZdL+mH+uLyewZuZWWUVr4yV1AJsAi4EDgK7JA1OMvbrFyNiXVnb1wE3AB1AAA/mbZ+uS/RmZlZRNbdAWA7sj4gDAJK2ApcA1QzyfRFwX0QcztveB6wGBqYXrpnZ5Hz7kaOrJtEvBp4ozB8Ezpuk3qWS3g48ClwVEU8cpe3i8oaS1gJrAVpbWymVSlUF34xGR0eT3r75wPtvbhoaGjrm8tHRUU488cSK60lx/9brpmZ/CwxExPOSPgh8HnhHtY0jYjOwGaCjoyM6OzvrFNbcUyqVSHn7knfvPd5/TWo+v/eqORl7CDi1MH9KXvayiHgqIp7PZ+8Czq22rZmZzaxqEv0uYJmkpZKOB9YAg8UKkt5QmL0YGMmndwCrJC2UtBBYlZeZmdksqdh1ExFjktaRJegWYEtE7JG0ERiOiEHgTyVdDIwBh4Er8raHJd1M9mEBsHHixKyZmc2OqvroI2I7sL2s7PrC9LXAtUdpuwXYUkOMZmZWA18Za2aWOCd6M7PEOdGbmSXOid7MLHH1umDKrOlVcwk9VL6MPsVL6K25+YjeLBcRFR9DQ0MV65jNNU70ZmaJc6I3M0ucE72ZWeKc6M3MEudEb2aWOCd6M7PEOdGbmSXOid7MLHFO9GZmiXOiNzNLnBO9mSVtYGCA9vZ2Vq5cSXt7OwMDA40Oadb5pmZmlqyBgQF6e3vp7+9nfHyclpYWenp6AOju7m5wdLOnqiN6Sasl7ZO0X9I1x6h3qaSQ1JHPL5H0nKSH88en6xW4mVklfX199Pf309XVxYIFC+jq6qK/v5++vr5GhzarKh7RS2oBNgEXAgeBXZIGI2JvWb3XAB8Cvle2isci4pw6xWtmVrWRkRFWrFhxRNmKFSsYGRlpUESNUc0R/XJgf0QciIgXgK3AJZPUuxm4DfinOsZnZjZtbW1t7Ny584iynTt30tbW1qCIGqOaPvrFwBOF+YPAecUKkt4GnBoR90i6uqz9UkkPAT8DrouIb5b/A0lrgbUAra2tlEql6regyYyOjia9fanz/msu7373u7nsssu4+uqrWbp0KXfccQe33347PT0982o/1nwyVtIrgL8Arphk8T8Ab4yIpySdC2yTdFZE/KxYKSI2A5sBOjo6orOzs9aw5qxSqUTK25c677/m0tnZyZlnnklfXx8jIyO0tbXx8Y9/fF6diIXqEv0h4NTC/Cl52YTXAO1AKR+K7Z8Bg5Iujohh4HmAiHhQ0mPAGcBwHWI3M6uou7ub7u7uef0hXU0f/S5gmaSlko4H1gCDEwsj4tmIWBQRSyJiCfBd4OKIGJZ0cn4yF0mnA8uAA3XfCjMzO6qKR/QRMSZpHbADaAG2RMQeSRuB4YgYPEbztwMbJb0IvARcGRGH6xG4mZlVp6o++ojYDmwvK7v+KHU7C9NfAb5SQ3xmZlYj3wLBzCxxTvRmZolTRDQ6hiNIehL4caPjmEGLgJ82OgibNu+/5pX6vjstIk6ebMGcS/SpkzQcER2NjsOmx/uvec3nfeeuGzOzxDnRm5klzol+9m1udABWE++/5jVv95376M3MEucjejOzxDnRm5klzom+ziT1Stoj6Qf58Ik3SPpYWZ1zJI3k0ydK+oykxyQ9KKkk6bzJ126SxvPndY+k70v6SH6r7Omsa6Ok3z/G8isl/fH0owVJZxeG0jws6fF8+u9qWW8zkjRamP4DSY9KOq2szo8kfaUw/x5Jd1dYb4ekOyvUWSJp91GWlSaGP02VBwevI0nnA38IvC0inpe0CDgTuBu4tlB1DTAxFP1dwOPAsoh4SdLSvI1N7rmJoSklvR74a+C1wA1TXdHR7tdUWF7zGMcR8QgwEe/dwFcj4svFOpIWRMRYrf+rWUhaCdwJXBQRk10cea6kM8uHKz2a/Hboc+rW58ru2a6IeKnRsYCP6OvtDcBPI2LiHvw/jYgHgKfLjtLfCwxI+k2y0bqum3hBRMTjEXHPbAfejCLiJ2Qjk61TpkXS7ZJ25d+oPjhRV9IGSY/k3wJuzcvulvSefPpWSXvzdv85L7tR0p/l0+dI+m6+/L9LWpiXlyTdJunv8yPU360m9rzdX0oaBj4k6VxJ38i/1e2Q9Ia83m9Kujcv/6akN9XxKZx1kt4OfBb4w4h47CjVPg70TtL21ZK25M/1Q5Iuycs7JX01nz5Z0n35N767JP04P+ACaJH02XzZ1yS9qrD69+XftHZLWp6v63WStuX7/LuS3pyXv/y6yOd3598YlkjaJ+mvgN3AqflrbHf+2ruqtmdv+pzo6+trZDv3UUmflPR7efkA2VE8kv4lcDgifgicBTwcEeONCbf5RcQBsttnvx7oAZ6NiN8Gfhv4gLJxFP4V2TjH50XEW4A/L65D0m8A7wbOiog3A7dM8q/+CtiQL3+EI79BLIiI5cB/YmrfLI7Pr9S8E/gE8J6IOBfYAvTldTYD6/PyPwM+OYX1zzWvBLYB74qI/32Mel8C3ibpX5SV9wL35891F3C7pFeX1bkhr3MW8GXgjYVly4BN+bJngEsLy34t/6b4H8ief4CbgIfyff5RstdAJcuAT+b/YxGwOCLaI+Js4HNVtJ8R7rqpo4gYVTZk4u+SvRC/KOka4IvAtyV9hCO7bay+VgFvnjhKB04ie+P9PvC5iPgFwCRjIjxLNqh9f35k+NXiQkknAb8eEd/Iiz4P/E2hyn/L/z4ILJlCvF/M//4W2Sht92Xf+GkB/kHSicDvAH+Tl0OWLJvVi8C3yT6QP3SMeuPA7WTdnf+zUL4KuLhwNH0CRyZygBVkH9pExL2Sni4sezwiHs6ny/fVQN7mAUmvlfTr+bouzcvvl/Qbkl5bYRt/HBHfzacPAKdL+gRwD9mBYEM40ddZfnReIhta8RHg8oi4W9LjwO+RvXDOz6vvAd4iqcVH9dOjbOSyceAngMiOfneU1bnoWOvIB9dZDqwE3gOsA94xhTCez/+OM7X31M8nQgT2RMT5xYV5Unlm4pxEAl4i67b8uqSPAreRJVyAwbJzJl8gS/TFE6gCLo2IfcWVSmqt8v8/X5geB4pdN+UXFB3rAqMxjuwNOaEwPbFPiYinJb0FuAi4kmzb/12VsdaVu27qSNJvSVpWKDqHX96JcwC4AzgQEQcB8j7KYeCm/OTNxK8D3jmLYTctSScDnwb+a2RX/u0A/r2k4/LlZ+Rf7e8D/kTSr+Xlrytbz4nASfkAO1cBbykuj4hnyc6zTPS/vw/4BvWzDzhZ2cl8JB0n6ayI+BnwuKR/nZcrTxxNK/9W9U7gMuCKiDgnf1xfVu9FsvdLsV97B7C+8F556yT/4ltkCRVJq4CFVYb2b/I2K8i6/54FvpnHiaROsvNvPwN+BLwtL38bsHSyFebnBl6RD8B03USbRvARfX2dCHwi/9o3BuwnO1kI2Vf9O4H1ZW3eT3byab+k58huo3r17ITblF4l6WHgOLLn+AvAX+TL7iL7Ov6/8mTwJFl/8L2SzgGGJb1ANlraRwvrfA3wPySdQHbU+OFJ/u/lwKfzD4sDwJ/Ua4Mi4oW8u+nOvJtoAfCXZN/4LgM+Jem6fJu3At+v1/9uhIg4LGk18ICkJ48xHGk/WYKccDPZ8/IDZT+pfZzsV25FN5H90OF9wHeA/wf8I9l781j+SdJDZM/xxFH3jcAWST8AfkH2GoBs1Lw/lrQH+B7w6FHWuRj4nH75899rj1JvxvkWCGaWDEmvBMbz7rjzgU8l1PU1bT6iN7OUvBH4Un4U/QLwgQbHMyf4iN7MLHE+GWtmljgnejOzxDnRm5klzonezCxxTvRmZon7/4ZZzLrm69N/AAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "jpjhSreZBQNG"
      },
      "source": [
        "### Compare this against the baseline"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2BUe0v-lq8gi"
      },
      "source": [
        "##### Join the two datafarames together so we can easily compare them"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "B04u8fMRrCsl"
      },
      "source": [
        "col_order = [\"subject_id\", \"SVC_Baseline\", \"SVC_No_Steps\", \"Decision Tree_Baseline\", \"Decision Tree_No_Steps\", \"K-Neighbours_Baseline\", \"K-Neighbours_No_Steps\"]\n",
        "\n",
        "baseline_and_no_steps = baseline_results.merge(no_steps_results, how='inner', on='subject_id', suffixes=['_Baseline', '_No_Steps'])[col_order]"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "49nN4PyrrEBa"
      },
      "source": [
        "##### Output the joined dataframe to compare the classifier scores manually"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "N4kT9NLj6qse",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 917
        },
        "outputId": "0eec0a05-72d8-47f2-aa8c-20386e5e3574"
      },
      "source": [
        "baseline_and_no_steps"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>subject_id</th>\n",
              "      <th>SVC_Baseline</th>\n",
              "      <th>SVC_No_Steps</th>\n",
              "      <th>Decision Tree_Baseline</th>\n",
              "      <th>Decision Tree_No_Steps</th>\n",
              "      <th>K-Neighbours_Baseline</th>\n",
              "      <th>K-Neighbours_No_Steps</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>46343</td>\n",
              "      <td>0.524590</td>\n",
              "      <td>0.524590</td>\n",
              "      <td>0.535519</td>\n",
              "      <td>0.557377</td>\n",
              "      <td>0.420765</td>\n",
              "      <td>0.453552</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>759667</td>\n",
              "      <td>0.554140</td>\n",
              "      <td>0.554140</td>\n",
              "      <td>0.649682</td>\n",
              "      <td>0.611465</td>\n",
              "      <td>0.503185</td>\n",
              "      <td>0.503185</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>781756</td>\n",
              "      <td>0.589506</td>\n",
              "      <td>0.589506</td>\n",
              "      <td>0.496914</td>\n",
              "      <td>0.481481</td>\n",
              "      <td>0.561728</td>\n",
              "      <td>0.561728</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>844359</td>\n",
              "      <td>0.548822</td>\n",
              "      <td>0.548822</td>\n",
              "      <td>0.612795</td>\n",
              "      <td>0.531987</td>\n",
              "      <td>0.535354</td>\n",
              "      <td>0.535354</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>1066528</td>\n",
              "      <td>0.523962</td>\n",
              "      <td>0.523962</td>\n",
              "      <td>0.549521</td>\n",
              "      <td>0.565495</td>\n",
              "      <td>0.533546</td>\n",
              "      <td>0.533546</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>5</th>\n",
              "      <td>1360686</td>\n",
              "      <td>0.620915</td>\n",
              "      <td>0.620915</td>\n",
              "      <td>0.653595</td>\n",
              "      <td>0.666667</td>\n",
              "      <td>0.637255</td>\n",
              "      <td>0.637255</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>6</th>\n",
              "      <td>1449548</td>\n",
              "      <td>0.685714</td>\n",
              "      <td>0.685714</td>\n",
              "      <td>0.698413</td>\n",
              "      <td>0.666667</td>\n",
              "      <td>0.666667</td>\n",
              "      <td>0.666667</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>7</th>\n",
              "      <td>1455390</td>\n",
              "      <td>0.705696</td>\n",
              "      <td>0.718354</td>\n",
              "      <td>0.705696</td>\n",
              "      <td>0.715190</td>\n",
              "      <td>0.715190</td>\n",
              "      <td>0.699367</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>8</th>\n",
              "      <td>1818471</td>\n",
              "      <td>0.533123</td>\n",
              "      <td>0.561514</td>\n",
              "      <td>0.564669</td>\n",
              "      <td>0.536278</td>\n",
              "      <td>0.545741</td>\n",
              "      <td>0.555205</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>9</th>\n",
              "      <td>2598705</td>\n",
              "      <td>0.609524</td>\n",
              "      <td>0.590476</td>\n",
              "      <td>0.536508</td>\n",
              "      <td>0.546032</td>\n",
              "      <td>0.523810</td>\n",
              "      <td>0.523810</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>10</th>\n",
              "      <td>2638030</td>\n",
              "      <td>0.709265</td>\n",
              "      <td>0.709265</td>\n",
              "      <td>0.619808</td>\n",
              "      <td>0.638978</td>\n",
              "      <td>0.648562</td>\n",
              "      <td>0.648562</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>11</th>\n",
              "      <td>3509524</td>\n",
              "      <td>0.630435</td>\n",
              "      <td>0.601449</td>\n",
              "      <td>0.760870</td>\n",
              "      <td>0.710145</td>\n",
              "      <td>0.681159</td>\n",
              "      <td>0.681159</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>12</th>\n",
              "      <td>3997827</td>\n",
              "      <td>0.643533</td>\n",
              "      <td>0.643533</td>\n",
              "      <td>0.738170</td>\n",
              "      <td>0.738170</td>\n",
              "      <td>0.668770</td>\n",
              "      <td>0.637224</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>13</th>\n",
              "      <td>4018081</td>\n",
              "      <td>0.606061</td>\n",
              "      <td>0.606061</td>\n",
              "      <td>0.709091</td>\n",
              "      <td>0.678788</td>\n",
              "      <td>0.606061</td>\n",
              "      <td>0.606061</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>14</th>\n",
              "      <td>4314139</td>\n",
              "      <td>0.569182</td>\n",
              "      <td>0.569182</td>\n",
              "      <td>0.490566</td>\n",
              "      <td>0.547170</td>\n",
              "      <td>0.575472</td>\n",
              "      <td>0.575472</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>15</th>\n",
              "      <td>4426783</td>\n",
              "      <td>0.638889</td>\n",
              "      <td>0.638889</td>\n",
              "      <td>0.583333</td>\n",
              "      <td>0.617284</td>\n",
              "      <td>0.620370</td>\n",
              "      <td>0.620370</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>16</th>\n",
              "      <td>5132496</td>\n",
              "      <td>0.655844</td>\n",
              "      <td>0.564935</td>\n",
              "      <td>0.512987</td>\n",
              "      <td>0.584416</td>\n",
              "      <td>0.623377</td>\n",
              "      <td>0.603896</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>17</th>\n",
              "      <td>5498603</td>\n",
              "      <td>0.573171</td>\n",
              "      <td>0.573171</td>\n",
              "      <td>0.552846</td>\n",
              "      <td>0.609756</td>\n",
              "      <td>0.573171</td>\n",
              "      <td>0.573171</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>18</th>\n",
              "      <td>5797046</td>\n",
              "      <td>0.603226</td>\n",
              "      <td>0.603226</td>\n",
              "      <td>0.596774</td>\n",
              "      <td>0.609677</td>\n",
              "      <td>0.593548</td>\n",
              "      <td>0.600000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>19</th>\n",
              "      <td>6220552</td>\n",
              "      <td>0.574603</td>\n",
              "      <td>0.574603</td>\n",
              "      <td>0.615873</td>\n",
              "      <td>0.565079</td>\n",
              "      <td>0.603175</td>\n",
              "      <td>0.593651</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>20</th>\n",
              "      <td>7749105</td>\n",
              "      <td>0.536585</td>\n",
              "      <td>0.536585</td>\n",
              "      <td>0.878049</td>\n",
              "      <td>0.829268</td>\n",
              "      <td>0.609756</td>\n",
              "      <td>0.560976</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>21</th>\n",
              "      <td>8000685</td>\n",
              "      <td>0.589905</td>\n",
              "      <td>0.589905</td>\n",
              "      <td>0.605678</td>\n",
              "      <td>0.589905</td>\n",
              "      <td>0.596215</td>\n",
              "      <td>0.596215</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>22</th>\n",
              "      <td>8173033</td>\n",
              "      <td>0.566456</td>\n",
              "      <td>0.566456</td>\n",
              "      <td>0.610759</td>\n",
              "      <td>0.601266</td>\n",
              "      <td>0.585443</td>\n",
              "      <td>0.585443</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>23</th>\n",
              "      <td>8530312</td>\n",
              "      <td>0.624204</td>\n",
              "      <td>0.624204</td>\n",
              "      <td>0.538217</td>\n",
              "      <td>0.570064</td>\n",
              "      <td>0.627389</td>\n",
              "      <td>0.627389</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>24</th>\n",
              "      <td>8686948</td>\n",
              "      <td>0.632911</td>\n",
              "      <td>0.632911</td>\n",
              "      <td>0.598101</td>\n",
              "      <td>0.566456</td>\n",
              "      <td>0.601266</td>\n",
              "      <td>0.585443</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>25</th>\n",
              "      <td>8692923</td>\n",
              "      <td>0.579288</td>\n",
              "      <td>0.579288</td>\n",
              "      <td>0.627832</td>\n",
              "      <td>0.631068</td>\n",
              "      <td>0.521036</td>\n",
              "      <td>0.546926</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>26</th>\n",
              "      <td>9106476</td>\n",
              "      <td>0.602524</td>\n",
              "      <td>0.602524</td>\n",
              "      <td>0.589905</td>\n",
              "      <td>0.605678</td>\n",
              "      <td>0.630915</td>\n",
              "      <td>0.630915</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>27</th>\n",
              "      <td>9961348</td>\n",
              "      <td>0.605042</td>\n",
              "      <td>0.600840</td>\n",
              "      <td>0.617647</td>\n",
              "      <td>0.613445</td>\n",
              "      <td>0.613445</td>\n",
              "      <td>0.592437</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "   subject_id  SVC_Baseline  SVC_No_Steps  Decision Tree_Baseline  \\\n",
              "0       46343      0.524590      0.524590                0.535519   \n",
              "1      759667      0.554140      0.554140                0.649682   \n",
              "2      781756      0.589506      0.589506                0.496914   \n",
              "3      844359      0.548822      0.548822                0.612795   \n",
              "4     1066528      0.523962      0.523962                0.549521   \n",
              "5     1360686      0.620915      0.620915                0.653595   \n",
              "6     1449548      0.685714      0.685714                0.698413   \n",
              "7     1455390      0.705696      0.718354                0.705696   \n",
              "8     1818471      0.533123      0.561514                0.564669   \n",
              "9     2598705      0.609524      0.590476                0.536508   \n",
              "10    2638030      0.709265      0.709265                0.619808   \n",
              "11    3509524      0.630435      0.601449                0.760870   \n",
              "12    3997827      0.643533      0.643533                0.738170   \n",
              "13    4018081      0.606061      0.606061                0.709091   \n",
              "14    4314139      0.569182      0.569182                0.490566   \n",
              "15    4426783      0.638889      0.638889                0.583333   \n",
              "16    5132496      0.655844      0.564935                0.512987   \n",
              "17    5498603      0.573171      0.573171                0.552846   \n",
              "18    5797046      0.603226      0.603226                0.596774   \n",
              "19    6220552      0.574603      0.574603                0.615873   \n",
              "20    7749105      0.536585      0.536585                0.878049   \n",
              "21    8000685      0.589905      0.589905                0.605678   \n",
              "22    8173033      0.566456      0.566456                0.610759   \n",
              "23    8530312      0.624204      0.624204                0.538217   \n",
              "24    8686948      0.632911      0.632911                0.598101   \n",
              "25    8692923      0.579288      0.579288                0.627832   \n",
              "26    9106476      0.602524      0.602524                0.589905   \n",
              "27    9961348      0.605042      0.600840                0.617647   \n",
              "\n",
              "    Decision Tree_No_Steps  K-Neighbours_Baseline  K-Neighbours_No_Steps  \n",
              "0                 0.557377               0.420765               0.453552  \n",
              "1                 0.611465               0.503185               0.503185  \n",
              "2                 0.481481               0.561728               0.561728  \n",
              "3                 0.531987               0.535354               0.535354  \n",
              "4                 0.565495               0.533546               0.533546  \n",
              "5                 0.666667               0.637255               0.637255  \n",
              "6                 0.666667               0.666667               0.666667  \n",
              "7                 0.715190               0.715190               0.699367  \n",
              "8                 0.536278               0.545741               0.555205  \n",
              "9                 0.546032               0.523810               0.523810  \n",
              "10                0.638978               0.648562               0.648562  \n",
              "11                0.710145               0.681159               0.681159  \n",
              "12                0.738170               0.668770               0.637224  \n",
              "13                0.678788               0.606061               0.606061  \n",
              "14                0.547170               0.575472               0.575472  \n",
              "15                0.617284               0.620370               0.620370  \n",
              "16                0.584416               0.623377               0.603896  \n",
              "17                0.609756               0.573171               0.573171  \n",
              "18                0.609677               0.593548               0.600000  \n",
              "19                0.565079               0.603175               0.593651  \n",
              "20                0.829268               0.609756               0.560976  \n",
              "21                0.589905               0.596215               0.596215  \n",
              "22                0.601266               0.585443               0.585443  \n",
              "23                0.570064               0.627389               0.627389  \n",
              "24                0.566456               0.601266               0.585443  \n",
              "25                0.631068               0.521036               0.546926  \n",
              "26                0.605678               0.630915               0.630915  \n",
              "27                0.613445               0.613445               0.592437  "
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 97
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "sGCGqS7RrfKE"
      },
      "source": [
        "From looking at this table, the values look pretty similar.\n",
        "We need to summarise these values somewhat to get a better understanding of which approach is better"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "DP9-mTOPrOag"
      },
      "source": [
        "##### Describe this dataframe"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 297
        },
        "id": "eJpFCxQirTrK",
        "outputId": "64ce304e-594e-4416-d001-5051d67997e0"
      },
      "source": [
        "baseline_and_no_steps.describe()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>SVC_Baseline</th>\n",
              "      <th>SVC_No_Steps</th>\n",
              "      <th>Decision Tree_Baseline</th>\n",
              "      <th>Decision Tree_No_Steps</th>\n",
              "      <th>K-Neighbours_Baseline</th>\n",
              "      <th>K-Neighbours_No_Steps</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>count</th>\n",
              "      <td>28.000000</td>\n",
              "      <td>28.000000</td>\n",
              "      <td>28.000000</td>\n",
              "      <td>28.000000</td>\n",
              "      <td>28.000000</td>\n",
              "      <td>28.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>mean</th>\n",
              "      <td>0.601326</td>\n",
              "      <td>0.597679</td>\n",
              "      <td>0.616065</td>\n",
              "      <td>0.613759</td>\n",
              "      <td>0.593656</td>\n",
              "      <td>0.590535</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>std</th>\n",
              "      <td>0.050517</td>\n",
              "      <td>0.049369</td>\n",
              "      <td>0.086878</td>\n",
              "      <td>0.073333</td>\n",
              "      <td>0.061696</td>\n",
              "      <td>0.054633</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>min</th>\n",
              "      <td>0.523962</td>\n",
              "      <td>0.523962</td>\n",
              "      <td>0.490566</td>\n",
              "      <td>0.481481</td>\n",
              "      <td>0.420765</td>\n",
              "      <td>0.453552</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>25%</th>\n",
              "      <td>0.568501</td>\n",
              "      <td>0.566076</td>\n",
              "      <td>0.552014</td>\n",
              "      <td>0.565391</td>\n",
              "      <td>0.557732</td>\n",
              "      <td>0.559533</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>50%</th>\n",
              "      <td>0.602875</td>\n",
              "      <td>0.590191</td>\n",
              "      <td>0.608219</td>\n",
              "      <td>0.607678</td>\n",
              "      <td>0.602220</td>\n",
              "      <td>0.593044</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>75%</th>\n",
              "      <td>0.631054</td>\n",
              "      <td>0.621737</td>\n",
              "      <td>0.650660</td>\n",
              "      <td>0.645900</td>\n",
              "      <td>0.628270</td>\n",
              "      <td>0.628270</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>max</th>\n",
              "      <td>0.709265</td>\n",
              "      <td>0.718354</td>\n",
              "      <td>0.878049</td>\n",
              "      <td>0.829268</td>\n",
              "      <td>0.715190</td>\n",
              "      <td>0.699367</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "       SVC_Baseline  SVC_No_Steps  Decision Tree_Baseline  \\\n",
              "count     28.000000     28.000000               28.000000   \n",
              "mean       0.601326      0.597679                0.616065   \n",
              "std        0.050517      0.049369                0.086878   \n",
              "min        0.523962      0.523962                0.490566   \n",
              "25%        0.568501      0.566076                0.552014   \n",
              "50%        0.602875      0.590191                0.608219   \n",
              "75%        0.631054      0.621737                0.650660   \n",
              "max        0.709265      0.718354                0.878049   \n",
              "\n",
              "       Decision Tree_No_Steps  K-Neighbours_Baseline  K-Neighbours_No_Steps  \n",
              "count               28.000000              28.000000              28.000000  \n",
              "mean                 0.613759               0.593656               0.590535  \n",
              "std                  0.073333               0.061696               0.054633  \n",
              "min                  0.481481               0.420765               0.453552  \n",
              "25%                  0.565391               0.557732               0.559533  \n",
              "50%                  0.607678               0.602220               0.593044  \n",
              "75%                  0.645900               0.628270               0.628270  \n",
              "max                  0.829268               0.715190               0.699367  "
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 98
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "t52FwO7mrdOp"
      },
      "source": [
        "After describing this dataframe, we can more clearly see that the mean scores of the classifiers when using the baseline configuration are marginally higher than.\n",
        "\n",
        "When we look at the Decision Tree classifier, the gap between the two approaches becomes even more apparent as the maximum accuracy for one of the subjects using the baseline configuratio is almost 5% higher than the variation where no step count is decorded.\n",
        "\n",
        "This points towards our hypothesis being correct for this experiment but we can double check this using a plot."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "VUmSkVRGrUCW"
      },
      "source": [
        "##### Plot the classifiers"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 450
        },
        "id": "LVRoQ-dup3Uz",
        "outputId": "42cf497e-e9e0-4325-9f8c-fd4eb27d87a9"
      },
      "source": [
        "baseline_and_no_steps.boxplot(figsize=(14,7))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<matplotlib.axes._subplots.AxesSubplot at 0x7fe6f155be80>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 103
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAzIAAAGgCAYAAABxBFJMAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3de5wlZ10n/s+XGRBMAMXAqIAEXVybHRVhFmSddXscwSi74AoKvV6Itkb3tRnXC/wItj8M7Lag6KIoXiKNoK4dAW+RZAmK3eIoagBJIGlhsyEuAV0uIu4gQmZ49o+qZs50+jbTPZxTM+/369WvrlOnTtVzqp5Tpz71PFWnWmsBAAAYkruNuwAAAACnS5ABAAAGR5ABAAAGR5ABAAAGR5ABAAAGR5ABAAAGZ1tBpqouqap3VNWtVXXFOs8/pKpeX1U3VdVyVT1o94sKAADQqa1+R6aq9iR5Z5LHJbkjyQ1JZlprt4xM86okr2mtvaKqvirJt7fWvvXsFRsAADifbadF5tFJbm2t3dZa+3iSq5M8ac00D0/yh/3w0jrPAwAA7Jq925jmgUnePfL4jiSPWTPNjUm+IclPJ/n3Se5dVZ/VWvvgRjO96KKL2sUXX3x6pT3HfOQjH8kFF1ww7mIwZuoBq9QFEvWAjnpAoh4kyZvf/OYPtNbuv95z2wky2/GMJD9bVZcmeUOS9yQ5sXaiqrosyWVJsm/fvvzET/zELi1+mI4dO5YLL7xw3MVgzNQDVqkLJOoBHfWARD1IkkOHDv31Rs9tJ8i8J8mDRx4/qB/3Sa2196ZrkUlVXZjkya21v187o9baVUmuSpIDBw606enpbSz+3LW8vJzzfR2gHnCSukCiHtBRD0jUg61s5xqZG5I8rKoeWlX3SPK0JNeMTlBVF1XV6ryeneRlu1tMAACAk7YMMq2140kuT3J9kpUkr2yt3VxVz6uqJ/aTTSd5R1W9M8m+JPNnqbwAAADbu0amtXZdkuvWjHvOyPCrk7x6d4sGAACwvm39ICYAAMAkEWQAAIDBEWQAAIDBEWQAAIDBEWQAAIDBEWQAAIDBEWQAAIDBEWQAAIDBEWQAAIDBEWQAxmxxcTH79+/P4cOHs3///iwuLo67SAAw8faOuwAA57PFxcXMzc1lYWEhJ06cyJ49ezI7O5skmZmZGXPpAGByaZEBGKP5+fksLCzk0KFD2bt3bw4dOpSFhYXMz8+Pu2gAMNEEGYAxWllZycGDB08Zd/DgwaysrIypRAAwDIIMwBhNTU3l6NGjp4w7evRopqamxlQiABgGQQZgjObm5jI7O5ulpaUcP348S0tLmZ2dzdzc3LiLBgATzcX+AGO0ekH/kSNHsrKykqmpqczPz7vQHwC2IMgAjNnMzExmZmayvLyc6enpcRcHAAZB1zIAAGBwBBkAAGBwBBkAAGBwBBkAAGBwBBkAAGBwBBkAAGBwBBkAAGBwBBkAAGBwBBkAAGBwBBkAAGBwBBkAAGBwBBkAAGBwBBkAAGBwBBkAAGBwBBkAAGBwBBkAAGBwBBkAAGBwBBkAAGBwBBkAAGBwBBkAAGBwBBkAAGBwBBkAAGBwBBkAAGBwBBkAAGBwBBkAAGBwBBkAAGBwBBkAAGBwBBkAAGBwBBkAAGBwBBkAmACLi4vZv39/Dh8+nP3792dxcXHcRQKYaHvHXQAAON8tLi5mbm4uCwsLOXHiRPbs2ZPZ2dkkyczMzJhLBzCZtMgAwJjNz89nYWEhhw4dyt69e3Po0KEsLCxkfn5+3EUDmFiCDACM2crKSg4ePHjKuIMHD2ZlZWVMJQKYfIIMAIzZ1NRUjh49esq4o0ePZmpqakwlAph8ggwAjNnc3FxmZ2eztLSU48ePZ2lpKbOzs5mbmxt30QAmlov9AWDMVi/oP3LkSFZWVjI1NZX5+XkX+gNsQpABgAkwMzOTmZmZLC8vZ3p6etzFAZh4upYBAACDI8gAAACDI8gAAACDI8gAAACDI8gAAACDI8gAAACDI8gAAACDI8gAAACDI8gAAACDI8gAAACDI8gAAACDI8gAAACDI8gAAACDs60gU1WXVNU7qurWqrpinec/r6qWquovq+qmqvq63S8qAABAZ8sgU1V7krwkydcmeXiSmap6+JrJfjjJK1trX5bkaUl+brcLCgAAsGo7LTKPTnJra+221trHk1yd5ElrpmlJ7tMP3zfJe3eviAAAAKfau41pHpjk3SOP70jymDXTXJnkdVV1JMkFSb56V0oHAACwjmqtbT5B1VOSXNJa+87+8bcmeUxr7fKRaX6gn9dPVtVjkywk2d9a+8SaeV2W5LIk2bdv36OuvvrqXX0zQ3Ps2LFceOGF4y4GY6YesEpdIFEP6KgHJOpBkhw6dOjNrbUD6z23nRaZ9yR58MjjB/XjRs0muSRJWmtvrKp7JrkoyftGJ2qtXZXkqiQ5cOBAm56e3k75z1nLy8s539cB6gEnqQsk6gEd9YBEPdjKdq6RuSHJw6rqoVV1j3QX81+zZpr/neRwklTVVJJ7Jnn/bhYUAABg1ZZBprV2PMnlSa5PspLu7mQ3V9XzquqJ/WQ/mOS7qurGJItJLm1b9VkDAAA4Q9vpWpbW2nVJrlsz7jkjw7ck+YrdLRoAAMD6tvWDmAAAAJNEkAEAAAZHkAEAAAZHkAEAAAZHkAEAAAZHkAEAAAZHkAEAAAZHkAEAAAZHkAEAAAZHkAEAAAZHkAEAAAZHkAEAAAZHkAEAAAZHkAEAAAZHkAEAAAZHkAEAAAZHkAEAAAZHkAEAAAZHkAEAAAZHkAEAAAZHkAEAAAZHkAEAAAZHkAEAAAZHkAEAAAZn77gLAHAuqKpxFyFJ0lobdxEA4FNCiwzALmit7fjvIc96zY7nAQDnC0EGAAAYHEEGAAAYHEEGAAAYHEEGAAAYHEEGAAAYHEEGAAAYHEEGAAAYHEEGAAAYHEEGAAAYHEEGAAAYHEEGAAAYHEEGAAAYHEEGAAAYHEEGAAAYHEEGAAAYnL3jLgAAnEuqatxFSGtt3EUAOOu0yADALmqt7ejvIc96zY7nAXA+EGQAAIDBEWQAAIDBEWQAAIDBEWQAAIDBEWQAAIDBEWQAAIDBEWQAAIDBEWQAAIDBEWQAAIDBEWQAAIDBEWRgTBYXF7N///4cPnw4+/fvz+Li4riLBAAwGHvHXQA4Hy0uLmZubi4LCws5ceJE9uzZk9nZ2STJzMzMmEsHADD5tMjAGMzPz2dhYSGHDh3K3r17c+jQoSwsLGR+fn7cRQMAGARBBsZgZWUlBw8ePGXcwYMHs7KyMqYSAQAMiyADYzA1NZWjR4+eMu7o0aOZmpoaU4kAAIZFkIExmJuby+zsbJaWlnL8+PEsLS1ldnY2c3Nz4y4aAMAguNgfxmD1gv4jR45kZWUlU1NTmZ+fd6E/wDmiqsZdhLTWxl0EOKu0yAAA7LLW2o7+HvKs1+x4HnCu0yIDY+D2ywAAO6NFBsbA7ZcBAHZGi8wO6P/KmXL7ZQCAndEiswP6v3Km3H4ZAGBnBBkYA7dfBgDYmW11LauqS5L8dJI9SV7aWnvBmudflORQ//DTkzygtfYZu1lQOJe4/TIAwM5sGWSqak+SlyR5XJI7ktxQVde01m5Znaa19v0j0x9J8mVnoaxwTpmZmcnMzEyWl5czPT097uIAAAzKdrqWPTrJra2121prH09ydZInbTL9TJLF3SgcAADAerYTZB6Y5N0jj+/ox91FVT0kyUOT/OHOiwYAALC+3b798tOSvLq1dmK9J6vqsiSXJcm+ffuyvLy8y4sfHuuAY8eOqQd8krpAoh7QUQ9wjLC57QSZ9yR58MjjB/Xj1vO0JP9poxm11q5KclWSHDhwoJ331wW89lrXRuAaGU6yTyBRD+ioB+cEvzl4dm2na9kNSR5WVQ+tqnukCyvXrJ2oqr4oyWcmeePuFhEAAIbHbw6eXVsGmdba8SSXJ7k+yUqSV7bWbq6q51XVE0cmfVqSq9u5vsYAAICx29Y1Mq2165Jct2bcc9Y8vnL3igUAALCx7XQtAwAAmCiCDAAAMDiCDAAAMDiCDAAAMDiCDAAAMDiCDAAAMDjbuv0ysLFJ+NXe5Nz+5V4AgLW0yMAOTcKv9goxAMD5RpABAAAGR5ABAAAGR5ABAAAGR5ABAAAGR5ABAAAGR5ABAAAGR5ABAAAGR5ABAAAGR5ABAAAGR5ABAAAGR5ABAAAGR5ABAAAGR5ABAAAGR5ABAAAGR5ABAAAGR5ABAAAGR5ABAAAGR5ABAAAGR5ABAAAGR5ABAAAGR5ABAAAGR5ABAAAGR5ABAAAGR5ABAAAGR5ABAAAGZ++4CwAwCb70ua/Lhz9657iLkYuvuHZsy77vve6eG3/k8WNbPgCcDkEGIMmHP3pnbn/BE8ZahuXl5UxPT49t+eMMUQBwunQtAwAABkeLDAD0dDHUxRAYDkEGAHq6GOpiCAyHrmUAAMDgCDIAAMDgCDIAAMDgCDIAAMDgCDIAAMDgCDIAAMDgCDIAAMDgCDIAAMDgCDIAAMDgCDIAAMDg7B13AcbpS5/7unz4o3eOtQwXX3HtWJd/33vdPTf+yOPHWgYAADhd53WQ+fBH78ztL3jC2Ja/vLyc6enpsS0/GX+QAgCAM3FeBxkAgLUmocdGMv6TjXptTEZdUA82JsgAAIwYd4+NRK+NSTHuuqAebM7F/gAAwOAIMgAAwOAIMgAAwOC4Robzngv5JvtCPgCA9QgynPdcyDf+IAUAcLp0LQMAAAZHkAEAAAZHkAEAAAZHkAEAAAZHkAEAAAZHkAEAAAZHkAEAAAZHkAEAAAbHD2ICJLn31BX54ldcMe5iJK8Y36LvPZUk4/tx2EmgHqgHwHAIMgBJ/u/KC3L7C8Z78La8vJzp6emxLf/iK64d27InhXqgHgDDsa2uZVV1SVW9o6purap1T1VV1TdV1S1VdXNV/fruFhMAAOCkLVtkqmpPkpckeVySO5LcUFXXtNZuGZnmYUmeneQrWmsfqqoHnK0CAwAAbKdr2aOT3Npauy1JqurqJE9KcsvINN+V5CWttQ8lSWvtfbtd0LNhIvpCj7EfdKIvNACsNRHHB4ljBNjCdoLMA5O8e+TxHUkes2aaL0ySqvqTJHuSXNlae+2ulPAsGndf6HH3g070hQaAtcZ9fJA4RoDt2K2L/fcmeViS6SQPSvKGqvri1trfj05UVZcluSxJ9u3bl+Xl5V1a/JkbZxmOHTt23q+DSaEeqAfJ+NfBJNSFcS9/Eox7HagHk2Hc62AS6kEy/vUwCRwjTG492E6QeU+SB488flA/btQdSf68tXZnkndV1TvTBZsbRidqrV2V5KokOXDgQBv3mYa89tqxnu2YhLMt414HE0E9GPs6mAgTsA7GXhcmYB2M3QSsA/VgAkzAOhh7PUgmYj2MnWOEsa+DzWznrmU3JHlYVT20qu6R5GlJrlkzze+ka41JVV2UrqvZbbtYTgAAgE/aMsi01o4nuTzJ9UlWkryytXZzVT2vqp7YT3Z9kg9W1S1JlpI8s7X2wbNVaAAA4Py2rWtkWmvXJbluzbjnjAy3JD/Q/wEAAJxVu3WxPwAAnFMm4lbcbsO9IUEGAADWMe5bcU/Cxf6TfBtuQYbznrMtk322BQBgPYIM5z1nWyb7bAsAwHq2c/tlAACAiSLIAAAAgyPIAAAAgyPIAAAAgyPIAAAAgyPIAAAAgyPIAAAAgyPIAAAAgyPIAAAAgyPIAAAAgyPIAAAAgyPIAAAAgyPIAAAAgyPIAAAAgyPIAAAAg7N33AUYt4uvuHa8BXjteJd/33vdfazLBwCAM3FeB5nbX/CEsS7/4iuuHXsZAABgiM7rIAOrtMxpmYNVY98fJGPdJ9gfAEMhyHDeG3ermJY5mByT8Fm0TwDYHhf7AwAAgyPIAAAAgyPIAAAAg+MaGQAA2MDYbwDihkAbEmQAANYY+8Fr4gB2Aoz7xhtu/rE5QQYAYMQkHDg6gIWtuUYGAAAYHEEGAAAYHF3LAHrne594/eEBGBJBBiD6xAPA0OhaBgAADI4gAwAADI4gAwAADI4gAwAADI4gAwAADI4gAwAADI4gAwAADI4gAwAADI4gAwAADI4gAwAADI4gAwAADI4gAwAADI4gAwAADI4gAwAADI4gAwAADI4gAwAADI4gAwAADI4gAwAADI4gAwAADI4gAwAADM7ecRdgyKpq5/P4sZ29vrW24zKwM5NQDxJ1AQA4v2iR2YHW2o7+lpaWdjwPxm8S6oG6AACcbwQZAABgcAQZAABgcAQZAABgcAQZAABgcAQZAABgcAQZAABgcAQZAABgcAQZAABgcAQZAABgcAQZAABgcAQZAABgcAQZAABgcLYVZKrqkqp6R1XdWlVXrPP8pVX1/qp6a//3nbtfVAAAgM7erSaoqj1JXpLkcUnuSHJDVV3TWrtlzaS/0Vq7/CyUEQBgUKpq5/P4sZ29vrW24zKwM+rB2bWdFplHJ7m1tXZba+3jSa5O8qSzWywAgOFqre3ob2lpacfzYPzUg7NrO0HmgUnePfL4jn7cWk+uqpuq6tVV9eBdKR0AAMA6tuxatk2/l2SxtfaxqvruJK9I8lVrJ6qqy5JcliT79u3L8vLyLi1+mI4dO3berwPUA06lLpCoB/huoKMebG47QeY9SUZbWB7Uj/uk1toHRx6+NMmPrzej1tpVSa5KkgMHDrTp6enTKes5Z3l5Oef7OkA9YMRrr1UXUA9I4ruBjnqwue10LbshycOq6qFVdY8kT0tyzegEVfU5Iw+fmGRl94oIAABwqi1bZFprx6vq8iTXJ9mT5GWttZur6nlJ3tRauybJ91bVE5McT/J3SS49i2UGAADOc9u6Rqa1dl2S69aMe87I8LOTPHt3iwYAALC+bf0gJgAAwCQRZAAAgMHZrdsvA5zXduPXmxO/4AwA26VFBmAX7PSXl/2CMwCcHkEGAAAYHEEGAAAYHEEGAAAYHEEGAAAYHEEGAAAYHLdfBoBdtBu34nYbboCtaZEBgF3kNtwAnxqCDAAAMDiCDAAAMDiCDAAAMDiCDAAAMDiCDAAAMDiCDAAAMDiCDAAAMDiCDAAAMDiCDAAAMDiCDAAAMDiCDAAAMDiCDAAAMDiCDAAAMDiCDAAAMDiCDAAAMDiCDAAAMDiCDAAATJDFxcXs378/hw8fzv79+7O4uDjuIk2kveMuAAAA0FlcXMzc3FwWFhZy4sSJ7NmzJ7Ozs0mSmZmZMZdusmiRAQCACTE/P5+FhYUcOnQoe/fuzaFDh7KwsJD5+flxF23iCDIAADAhVlZWcvDgwVPGHTx4MCsrK2Mq0eQSZAAAYEJMTU3l6NGjp4w7evRopqamxlSiySXIAADAhJibm8vs7GyWlpZy/PjxLC0tZXZ2NnNzc+Mu2sRxsT8AAEyI1Qv6jxw5kpWVlUxNTWV+ft6F/usQZAAAYILMzMxkZmYmy8vLmZ6eHndxJpauZQAAwOAIMgAAwOAIMgAAwOAIMgAAwOAIMgAAwOAIMgAAwOAIMgAAwOAIMgAAwOAIMgAAwOAIMgAAwOAIMgAAwOAIMgAAwOAIMgAAwOAIMgAAwOAIMgAAwOAIMgAAwOAIMgAAwOAIMgAAE2JxcTH79+/P4cOHs3///iwuLo67SDCx9o67AAAAdCFmbm4uCwsLOXHiRPbs2ZPZ2dkkyczMzJhLB5NHiwwAwASYn5/PwsJCDh06lL179+bQoUNZWFjI/Pz8uIsGE0mQAQCYACsrKzl48OAp4w4ePJiVlZUxlQgmmyADADABpqamcvTo0VPGHT16NFNTU2MqEUw2QQYAYALMzc1ldnY2S0tLOX78eJaWljI7O5u5ublxFw0mkov9AQAmwOoF/UeOHMnKykqmpqYyPz/vQn/YgCADADAhZmZmMjMzk+Xl5UxPT4+7ODDRdC0DAAAGR5ABAAAGR5ABAAAGR5ABAAAGR5ABAAAGR5ABAAAGZ1tBpqouqap3VNWtVXXFJtM9uapaVR3YvSICAACcassgU1V7krwkydcmeXiSmap6+DrT3TvJf07y57tdSAAAgFHbaZF5dJJbW2u3tdY+nuTqJE9aZ7r/kuTHkvzTLpYPAADgLrYTZB6Y5N0jj+/ox31SVT0yyYNba9fuYtkAAADWtXenM6iquyX5b0ku3ca0lyW5LEn27duX5eXlnS5+0I4dO3berwPUA05SF0jUAzrqAYl6sJXtBJn3JHnwyOMH9eNW3TvJ/iTLVZUkn53kmqp6YmvtTaMzaq1dleSqJDlw4ECbnp4+85KfA5aXl3O+rwPUA05SF0jUAzrqAYl6sJVqrW0+QdXeJO9McjhdgLkhyX9ord28wfTLSZ6xNsSsM937k/z1GZT5XHJRkg+MuxCMnXrAKnWBRD2gox6QqAdJ8pDW2v3Xe2LLFpnW2vGqujzJ9Un2JHlZa+3mqnpekje11q45kxJtVKDzSVW9qbXmVtXnOfWAVeoCiXpARz0gUQ+2sq1rZFpr1yW5bs2452ww7fTOiwUAALCxbf0gJgAAwCQRZMbrqnEXgImgHrBKXSBRD+ioByTqwaa2vNgfAABg0miRAQAABkeQAQAABue8DjJVNVdVN1fVTVX11qr6kap6/pppHlFVK/3whVX1i1X1v6rqzVW1XFWP2WT+J/r53lhVb6mqf7XL5X95VT2lH35pVT18N+d/rvsUbP9WVT858vgZVXXlGZTzn/fLemtVrVTVVSNl+7rTnd+QjXymbu4/Vz9YVWe0H6uq51XVV2/y/PdU1bedeWmTqvrivrxvraq/q6p39cN/sJP5brCsK6vqPf38/6qqfv5M180G87+4qt7eDx+oqhfv1rxPowy2/8bLurKq/rGqHjAy7tgZzmvtvvEx/fjvq6pP360y79To+6uqr6uqd1bVQ9ZMc3tV/ebI46dU1cu3mO+W9Xv087DOc8tVNdbb5fbHB6v17a+q6kd2ef7TVfWafviJVXXFbs5/m2Ww/TfQb//3VNWn9Y8vqqrbz2A+d6uqF1fV26vqbVV1Q1U9tH/uh3a52GdkW7dfPhdV1WOT/Nskj2ytfayqLkry8CQvT/LskUmflmSxH35pkncleVhr7RP9xtwsPHy0tfaIfnlfk+T5Sf7Nrr6RXmvtO8/GfM9Vn6Lt/7Ek31BVz2+t7eTHrF6c5EWttd/ty/7F/fhHJDmQNbdGP8eNfqYekOTXk9wnyWl/SW90C/mR53/hjEp46jzelm47pf/yfE1r7dWj01TV3tba8Z0uq/ei1tpP9Af3b0i3v1napXl/Uv+Dx5v+6PFZYvtv7gNJfjDJs850BhvsG+/RP/19SX4tyT/utKC7qaoOp9tPfk1rbb0f2n5UVT28tXbLduY3xvq9oaqqdNc1f+I0XvbM1tqrq+qeSW6pql9prb1rt8vW/57gGf2m4G6w/Td0Isl3JPn5HSz6qUk+N8mX9Mc9D0rykf65H0ryozuY9644n1tkPifJB1prH0uS1toHWmtvSPKhOvUs+zclWayqL0jymCQ/vFqRWmvvaq1du83l3SfJh5JPntl/fXWtNG+rqif14y+oqmv7M41vr6qn9uMfVVV/VF0rwPVV9TlrZz56BqCqjlXVfD+fP6uqff34+1fVb/aJ+oaq+orTX23njE/F9j+e7m4j37/2if5szh/2ZzxfX1Wft0VZ71h90Fp7W1XdI8nzkjy1P+P21L7+vKyq/qKq/nKkXl1aVb/b15H/uXpmbqP6NhSttfcluSzJ5dXZU1Uv7Ov2TVX13avTVtWz+s/ajVX1gn7caIvmC6rqlv51P9GPu7KqntEPP6L/LN1UVb9dVZ/Zj1+uqh/r1/k7q+pfb6fs/et+qqrelOQ/b/QZr6ovqKrX9uP/uKq+aJur5x5J7pmT+5zv6tfLjf0+4NP78d/Yb/sbq+oN/bgN1+NI+UfPxl7Z17vlqrqtqr53ZLpv6dfNW6trzdyzzfJvyfZf18vS7RPut84yf6Df1m+vqu/bZB7r7Rvf22/Xz02yVFVL/TwfX1VvrO677FVVdWE//vaq+vF+nf9FVf2zfvxd6ttOVdVXJvmlJP+2tfa/NpjsJ5PMrfPajfaZo/X7/lX1+9W1UL20qv66unCXJHuq6pf6515XVfcamf239vX+7VX16H5e96uq3+nr0Z9V1Zf04z9Z1/rHb6/uO+LiqnpHVf1KkrcneXBfb1fPjt/lu2UD9+z/f6Sf/3P6z8nbq+qqqqp+/PeOfA6u3mwdrVmPl1bVz/bDL6/uDP6fVrc/eMrIdM8c+Xw+d5tl35Ttv6mfSvL9VXVKo0V1Xjgyn82++z8nyd+MHPfc0Vr7UHX70Xv17/G/9/Ndd39f3THpi/r19Pqqun8//i717Yy01s7LvyQXJnlrkncm+bkk/6Yf/4x0ZzWT5MuTvKkffmKS3z7NZZzol/FXST6c5FH9+L1J7tMPX5Tk1iSV5MlJfmnk9fdNcvckf5rk/v24pyZ5WT/88iRP6YeXkxzoh1uSf9cP/3i6g++kO3t5sB/+vCQr494O5/j2P5YuwN7eb8tnJLmyf+73kjy9H/6OJL+zyXy+va8//yNdKPqMfvylSX52ZLofTfIt/fBn9O/tgn66v0nyWUnulW6HeGC9+jbu7bKddbrOuL9Psi/dQe1qXf+0dGfUHprka/vP0Kf3z92v///yJE/p18s7kk/exXF1/V6Z5Bn98E0jdeR5SX6qH15O8pP98Ncl+YNNyr728/pz/fBmn/HXp2sBTLog/YebzP/KJO/p6/WHkvz6yHOfNTL8X5Mc6YffluSBa973Ruvx4iRv78dPp2tdWF3un/bTXpTkg/17mkpXz+/eT/dzSb7N9j+r2/8ZSZ6T5Lmj6yvJo/ptfUG6fd/NSb5sg/msu2/sn7s9yUX98EXpWv0u6B8/K8lzRqab64e/baSu3KW+7bA+3Jnk79KdLd5omtv7+rGS5J/12/zl/XMb7TOnR8r8s0me3Q9fku779aJ0n4fjSR7RP/fKkXktp9+3JvnKnPzc/EySH+mHvyrJW9fWtf7x2/v5X5zkE0m+fGQ7/v7IdBuuw3T17V39tjyW5EdHnrvfyPCv5uTxwnuTfNqaz8F21tGl6b+L+uW+Kt2J8ocnudNNat4AAAoXSURBVLUf//h0J/aqf+41Sb7S9j+r2/8p6U5ufHtf5tv7556c5PeT7OnXzf9O8jkbzOdB/Tp8a7pA+GUjzx0bGd5wf9+vs2/uh58zUlfuUt/O5O+8bZFprR1LVykuS/L+JL9RVZcm+Y0kT6mua8Zot6Iz8dHW2iNaa1+U7gPwK/2Zj0ryo1V1U5I/SPLAdJXpbUkeV90Zvn/dWvtwkn+eZH+S36+qtyb54XQVazMfT7eTSJI3p/swJMlXJ/nZfj7XJLlP9WfQzjefou2f1to/JPmVJN+75qnHpguWSfdFcnCTefxyup3Eq9LtYP+s+n6vazw+yRX99l1OdxZutaXn91trH2ytfTTJb/XLW6++Ddnjk3xb//7/PN0B6sPS1ftfbq39Y5K01v5uzes+nOSfkixU1TdkTbeZqrpvup3sH/WjXpHuy2nVb/X/Rz9r2/Eb/f91P+P9Z/NfJXlVP/4X050d28yLWtf16gFJLqiqp/Xj91d3Rv9tSb45yb/ox/9JkpdX1Xel+1JLNl6Pm7m2tfax1nWhfF+6/dnhdJ+xG/p5HU7y+VvMZyds/86Lkzy9qu49Mu5guhMxH+n3fb+VZN3Wo032jWt9eboD1T/py/f0JA8ZeX5x5P9j++H16ttO3JkuBM5uMd2JJC/Mqd2Gk833masOJrk6SVprr03fytl7V2vtrf3w2u2/2L/mDem+az+jn9ev9uP/MMlnVdV9tij7X7fW/qwfvi3J51fVz1TVJUn+YYvXPrPfH3x2ksN18jrdQ1X15/3+4Ktycn9wU5L/XlXfku4gPdneOlrrd1prn2hdV659I/N5fJK/TPKWJF+UrfcrW7H9t/b8JM/MqT2wDiZZbK2daK39nyR/lORfrvfi1tod6fZRz04Xql5fXVe+tTbb338iJ/d3v5aTxzvr1bfTdt5eI5MkrbUT6Srvcv+Bfnpr7eVV9a50fcufnJM74JuTfGlV7elfd7rLemPfHHn/dGfu7p+uhebO6i7Aumdr7Z1V9cj++f9aVa9P8ttJbm6tPXajea/jztZH3HQf4NXtfLd0yf6fTrf856JP4fb/qXQ77l/eQVnfm+7Mysuqu8Bw/zqTVZInt9beccrIrqtcWzNtW6++tdaed6ZlHIeq+vx0dfx96d7/kdba9Wum+ZrN5tFaO943/R9Odwbr8nRf7tv1sf7/6GdtO1b7GVfW+Yz3X3B/3x+InJZ+v/LadAfcV6c7O/f1rbUb+4PS6X667+nrxxOSvLmqHpWN1+PFmyzyYyPDq+uhkryitbb24GHX2P531Vr7+6r69ST/6XRet2Yed9k3pqtDpxQx3QmSmY1ms3Z4vfrWWvvgmZYz3QHSN6U7uPqhJD+W7oAySa5pp14H9avpDsZGL9DeaJ+5L9uztt6Pdi26yz53k/kcz6kHmvccGV6tJ2ldl54vTfI1Sb4n3Xv/jq0K2Vo7VlXLSQ5W1VvSnS0/0Fp7d3U3oFld3hPS7TP+XZK56q7HPJN1NLpeauT/81trv7hVeU+D7b/F9m+t/c8+WHzTZtNtMY+PpesR8j+q6v8k+fp0rcWjTmd/v7ou7lLf2hlcL3jetshUdyeo0bMBj0iyepHYYpIXJbmtT6NpXd/LNyV5bt+qsnqdwxO2ubwvSncG6oPpuhm9rz/YOJT+LFZVfW6Sf2yt/Vq6swePTNfl4f7VXYCZqrp7Vf2L9ZaxDa9LcmSkTKd9gHSu+FRu//4M8Ctz6lmjP03X4pN0Z8j/eJOyXlJVd++HPzvdmeb3JPm/SUbPul6f5MhI+b5s5LnHVdc/917pdkJ/skF9G4zq+tn+Qrpm6pbu/f/HkXX1hVV1Qbom9G+vk9eF3G/NfC5M163uunRd97509Pm+pepDdfL6h29NdwZrt6z7Ge9b895VVd/Yj6/+S2xLfR34iiSrfcbvneRv+nXzzSPTfUFr7c/7L/z3J3lwNl6Pp+v16Vo3H9DP53615o5CO2H7b+q/JfnunAxWf5zk66vq0/t18u+zwT5ni33j6D7nz5J8RZ28/uWCqvrCkdc9deT/G/tp1qtvO9K3tD0hXb2+tO8F8Yg1B7Fprd2Zbr8+el3BZvvMVX+S/iCwqh6f5DO3WbTVa1wPJvlwX4/+uC9nqmo63bVI/5Cu684j+/GPTNcl8i6qOxl6t9bab6ZrudvWPru6ayQek25/sHqQ/IG+7q9eJ3a3JA9urS2l6yZ433TdDLezjrbj+iTfUSevo3pgjdxh70zZ/tsyn67b6ao/Tnct3Z5+P/qVSf5ig2U+sj9WWK0jX5KT+4M7V/e32Xx/f7f09SzJf0hydJP6dtrO5xaZC5P8THXNfcfTXadyWf/cq9I1zx9Z85rvTNdH8Naq+mi6O8Q8c5Nl3KtPwkmXVp/eWjtR3YVRv1fdma43pbuGJkm+OMkLq+oT6ZpM/2Nr7ePVXSz34uq6OOxNd4b/5jN4z9+b5CXVdWnbm65/8/ecwXzOBZ+K7T/qJ9Od6V11JMkvV9Uz032hf/smr318kp+uqtWWtGe21v62ugtuV5vFn5/kv6SrGzf1O4l3pbv7UNLtpH4zXbfEX2utvam6M9Wn1LdtvpdxWv1M3T3ddvvVdAdtSXdXuYuTvKX/Ynp/ulaI1/ah/U1V9fF0d3kbvW3kvZP8bnV39qkkP7DOcp+e5Bf6g+Hbsvn2Oi1bfMa/OcnPV9UP9+/56iQ3bjK776+umf7u6Zrtf64f//+n6271/v7/6sHoC/uD1kr3RXRj/7qLs2Y9nsH7uqUv9+v6+nhnulaC9e4qtF22/+bbf3WeH6iq305/0NZae0t1d01bPVh5aWvtLzd4+Wb7xquSvLaq3ttaO1Rd695inezq+sPprjNIks/sv2s+lmS11Wa9+rZjrbW/q66rzRuq6v2tu4vWehb6Mq7abJ+56rnp3uO3pgtkf5su0G110PVPVfWX6bbb6lnzK9O1qt+Urgvj0/vxv5muW+TN6T6f78z6Hpjue2P1JPRWZ79f2Nede6Rb37/VWmtV9UvpWib+NskN/bR7kvxaXwcryYv71r3trKMttdZeV1VTSd7Y54ZjSb4lXWvqjtj+m2ut3VxdS9xq8PntdL1NbkzXOvL/tdb+doOXPyDJL418xv8i3XVDSbc/uKmq3tJa++ZN9vcfSfLo/vn3pQt569a37byftVYvbgTOUf3BxoHW2uVbTQuwU9V1lz7Qdnbb+YnQH8Cd6LsgPjbJz59Jd0+Gyfbfuao61lo7a9djn88tMgAAm/m8JK/szzJ/PMl3jbk8fGrZ/hNOi8wOVdVn5a4XPSXJ4R1exMgA7Ob2r6q5JN+4ZvSrWmvzZ1o+zj3qyfltt7a/765zQ1W9JN31cKN+unV3u+Qct1vbv7obO/zqmtEfa609Zr3pJ4kgAwAADM55e9cyAABguAQZAABgcAQZAABgcAQZAABgcAQZAABgcP4f9i8/G3jJ5NIAAAAASUVORK5CYII=\n",
            "text/plain": [
              "<Figure size 1008x504 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "MDBErJFQtgTe"
      },
      "source": [
        "This plot backs up what we saw from the above describe. We can see for each classifier, the boxplot on the left (the baseline results) shows more promising signs of its results being more accurate. While the improvement is not one of great magnitude, it still adds value to our classifier.\n",
        "\n",
        "The step count feature will help to distinguish between the subject being asleep Vs being awake but it wont help to distinguish between the multiple sleep states. This is perhaps the reason why the results are minimal in some cases, as it only helps classify one particula psg status."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "iFVGucUcsCzf"
      },
      "source": [
        "---\n",
        "\n",
        "## Experiment 3 - Classify the sleep state using the optimised parameters Vs the default parameters\n",
        "\n",
        "\n",
        "---"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "DD-N6c6Z2J0M"
      },
      "source": [
        "We were fairly confident that our approach in relation to this step was the best one to take. It makes sense that if you optimise parameters to make the best prediction of the psg_status based on the training data then your results will be mroe accurate.\n",
        "\n",
        "Nevertheless, we felt it was important to validate this and ensure that the parameters that we gave the cross-validation optimiser were better than the default. We understood that if a value for one parameter is worse than the default, even if it is optimised, it will still be worse."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "HnzNg9wGsCzf"
      },
      "source": [
        "### Get the dataset and y for the models with the default parameters"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 66,
          "referenced_widgets": [
            "c56844f691da4e8eb2fbdd3c4080452e",
            "74a5f20891534b6181cc9656c669815a",
            "6ceea209ef4c40f2bc15ee4507425e5b",
            "de5fedc407804b80be62f209d4fb3e81",
            "e7939d5e977c4d6dab424caa8e2eb9fd",
            "404f9a49e03e4a14a63fe2eb64f606af",
            "448eed7c906941eca4885aae92485f96",
            "c90d5e00d666472baaf4683a705ab783"
          ]
        },
        "id": "UnSzyDGxsCzf",
        "outputId": "b3e223ee-fe7f-490e-8ee6-a377ee71e8a8"
      },
      "source": [
        "default_params_features_all_psg_dataset_and_y = create_map_of_subject_id_to_the_dataset_and_y_for_all_psg_status(subject_id_to_filtered_extracted_feature_df_map, map_subject_id_to_a_map_of_the_psg_status_to_session_id, map_subject_id_to_a_map_of_the_session_id_to_psg_status)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "c56844f691da4e8eb2fbdd3c4080452e",
              "version_minor": 0,
              "version_major": 2
            },
            "text/plain": [
              "HBox(children=(FloatProgress(value=0.0, max=31.0), HTML(value='')))"
            ]
          },
          "metadata": {
            "tags": []
          }
        },
        {
          "output_type": "stream",
          "text": [
            "\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "OK84vZfdsCzf"
      },
      "source": [
        "### Classify the dataset and y for the models with the default parameters"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 66,
          "referenced_widgets": [
            "06a6a1df12a848b48746bc7ae4569307",
            "82f29e9a77ed43b0a90b70ea548d31de",
            "868b630837b3406a8736f900a617c863",
            "0935081bd44140b6b99b8c5bcf46ad69",
            "5104d3a802d942a1a0de67cc33e40acf",
            "a6a1b6b4dc0c43cf93c9cd1752524fe3",
            "790a2ec874984690847cb52aa00bb902",
            "cad99c1460dc4c0f9ec646b347795a20"
          ]
        },
        "id": "ftjqJiOAsCzf",
        "outputId": "012ccb6d-d9a6-4c90-bdd6-5af84f02c002"
      },
      "source": [
        "optimise_params_bool = False\n",
        "classification_report_bool = False\n",
        "default_param_results = classify(default_params_features_all_psg_dataset_and_y, names_of_models_to_test, cv_settings, optimise_params_bool, classification_report_bool)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "06a6a1df12a848b48746bc7ae4569307",
              "version_minor": 0,
              "version_major": 2
            },
            "text/plain": [
              "HBox(children=(FloatProgress(value=0.0, max=31.0), HTML(value='')))"
            ]
          },
          "metadata": {
            "tags": []
          }
        },
        {
          "output_type": "stream",
          "text": [
            "\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "53RZH1TksCzg"
      },
      "source": [
        "### Analyse these classified results"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Ys7ylgVKuxVY"
      },
      "source": [
        "##### Output the results of the score each classifier got for each subject"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 917
        },
        "id": "dhVAFd-GsCzg",
        "outputId": "0fbe4111-1cc2-451f-9a59-ebbf4eee46d8"
      },
      "source": [
        "default_param_results"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>subject_id</th>\n",
              "      <th>SVC</th>\n",
              "      <th>Decision Tree</th>\n",
              "      <th>K-Neighbours</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>46343</td>\n",
              "      <td>0.480874</td>\n",
              "      <td>0.622951</td>\n",
              "      <td>0.415301</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>759667</td>\n",
              "      <td>0.566879</td>\n",
              "      <td>0.649682</td>\n",
              "      <td>0.611465</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>781756</td>\n",
              "      <td>0.537037</td>\n",
              "      <td>0.475309</td>\n",
              "      <td>0.558642</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>844359</td>\n",
              "      <td>0.572391</td>\n",
              "      <td>0.535354</td>\n",
              "      <td>0.525253</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>1066528</td>\n",
              "      <td>0.523962</td>\n",
              "      <td>0.533546</td>\n",
              "      <td>0.507987</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>5</th>\n",
              "      <td>1360686</td>\n",
              "      <td>0.565359</td>\n",
              "      <td>0.653595</td>\n",
              "      <td>0.620915</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>6</th>\n",
              "      <td>1449548</td>\n",
              "      <td>0.622222</td>\n",
              "      <td>0.641270</td>\n",
              "      <td>0.692063</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>7</th>\n",
              "      <td>1455390</td>\n",
              "      <td>0.705696</td>\n",
              "      <td>0.696203</td>\n",
              "      <td>0.680380</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>8</th>\n",
              "      <td>1818471</td>\n",
              "      <td>0.533123</td>\n",
              "      <td>0.523659</td>\n",
              "      <td>0.526814</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>9</th>\n",
              "      <td>2598705</td>\n",
              "      <td>0.609524</td>\n",
              "      <td>0.546032</td>\n",
              "      <td>0.546032</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>10</th>\n",
              "      <td>2638030</td>\n",
              "      <td>0.638978</td>\n",
              "      <td>0.632588</td>\n",
              "      <td>0.635783</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>11</th>\n",
              "      <td>3509524</td>\n",
              "      <td>0.594203</td>\n",
              "      <td>0.688406</td>\n",
              "      <td>0.652174</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>12</th>\n",
              "      <td>3997827</td>\n",
              "      <td>0.596215</td>\n",
              "      <td>0.731861</td>\n",
              "      <td>0.652997</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>13</th>\n",
              "      <td>4018081</td>\n",
              "      <td>0.509091</td>\n",
              "      <td>0.575758</td>\n",
              "      <td>0.612121</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>14</th>\n",
              "      <td>4314139</td>\n",
              "      <td>0.534591</td>\n",
              "      <td>0.490566</td>\n",
              "      <td>0.556604</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>15</th>\n",
              "      <td>4426783</td>\n",
              "      <td>0.623457</td>\n",
              "      <td>0.472222</td>\n",
              "      <td>0.601852</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>16</th>\n",
              "      <td>5132496</td>\n",
              "      <td>0.564935</td>\n",
              "      <td>0.564935</td>\n",
              "      <td>0.610390</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>17</th>\n",
              "      <td>5498603</td>\n",
              "      <td>0.459350</td>\n",
              "      <td>0.544715</td>\n",
              "      <td>0.536585</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>18</th>\n",
              "      <td>5797046</td>\n",
              "      <td>0.587097</td>\n",
              "      <td>0.551613</td>\n",
              "      <td>0.583871</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>19</th>\n",
              "      <td>6220552</td>\n",
              "      <td>0.520635</td>\n",
              "      <td>0.539683</td>\n",
              "      <td>0.561905</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>20</th>\n",
              "      <td>7749105</td>\n",
              "      <td>0.536585</td>\n",
              "      <td>0.707317</td>\n",
              "      <td>0.634146</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>21</th>\n",
              "      <td>8000685</td>\n",
              "      <td>0.605678</td>\n",
              "      <td>0.621451</td>\n",
              "      <td>0.602524</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>22</th>\n",
              "      <td>8173033</td>\n",
              "      <td>0.534810</td>\n",
              "      <td>0.506329</td>\n",
              "      <td>0.566456</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>23</th>\n",
              "      <td>8530312</td>\n",
              "      <td>0.592357</td>\n",
              "      <td>0.519108</td>\n",
              "      <td>0.544586</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>24</th>\n",
              "      <td>8686948</td>\n",
              "      <td>0.623418</td>\n",
              "      <td>0.601266</td>\n",
              "      <td>0.585443</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>25</th>\n",
              "      <td>8692923</td>\n",
              "      <td>0.508091</td>\n",
              "      <td>0.601942</td>\n",
              "      <td>0.508091</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>26</th>\n",
              "      <td>9106476</td>\n",
              "      <td>0.545741</td>\n",
              "      <td>0.552050</td>\n",
              "      <td>0.634069</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>27</th>\n",
              "      <td>9961348</td>\n",
              "      <td>0.605042</td>\n",
              "      <td>0.600840</td>\n",
              "      <td>0.567227</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "   subject_id       SVC  Decision Tree  K-Neighbours\n",
              "0       46343  0.480874       0.622951      0.415301\n",
              "1      759667  0.566879       0.649682      0.611465\n",
              "2      781756  0.537037       0.475309      0.558642\n",
              "3      844359  0.572391       0.535354      0.525253\n",
              "4     1066528  0.523962       0.533546      0.507987\n",
              "5     1360686  0.565359       0.653595      0.620915\n",
              "6     1449548  0.622222       0.641270      0.692063\n",
              "7     1455390  0.705696       0.696203      0.680380\n",
              "8     1818471  0.533123       0.523659      0.526814\n",
              "9     2598705  0.609524       0.546032      0.546032\n",
              "10    2638030  0.638978       0.632588      0.635783\n",
              "11    3509524  0.594203       0.688406      0.652174\n",
              "12    3997827  0.596215       0.731861      0.652997\n",
              "13    4018081  0.509091       0.575758      0.612121\n",
              "14    4314139  0.534591       0.490566      0.556604\n",
              "15    4426783  0.623457       0.472222      0.601852\n",
              "16    5132496  0.564935       0.564935      0.610390\n",
              "17    5498603  0.459350       0.544715      0.536585\n",
              "18    5797046  0.587097       0.551613      0.583871\n",
              "19    6220552  0.520635       0.539683      0.561905\n",
              "20    7749105  0.536585       0.707317      0.634146\n",
              "21    8000685  0.605678       0.621451      0.602524\n",
              "22    8173033  0.534810       0.506329      0.566456\n",
              "23    8530312  0.592357       0.519108      0.544586\n",
              "24    8686948  0.623418       0.601266      0.585443\n",
              "25    8692923  0.508091       0.601942      0.508091\n",
              "26    9106476  0.545741       0.552050      0.634069\n",
              "27    9961348  0.605042       0.600840      0.567227"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 58
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "9O7cqiwFu7IN"
      },
      "source": [
        "##### Describe this dataframe better with summary statistics"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 297
        },
        "id": "szuvooD6sCzg",
        "outputId": "d77f75e6-dc24-4bc4-d65f-632870ae953c"
      },
      "source": [
        "default_param_results.describe()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>SVC</th>\n",
              "      <th>Decision Tree</th>\n",
              "      <th>K-Neighbours</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>count</th>\n",
              "      <td>28.000000</td>\n",
              "      <td>28.000000</td>\n",
              "      <td>28.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>mean</th>\n",
              "      <td>0.567762</td>\n",
              "      <td>0.585009</td>\n",
              "      <td>0.583274</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>std</th>\n",
              "      <td>0.053260</td>\n",
              "      <td>0.071605</td>\n",
              "      <td>0.059866</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>min</th>\n",
              "      <td>0.459350</td>\n",
              "      <td>0.472222</td>\n",
              "      <td>0.415301</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>25%</th>\n",
              "      <td>0.534224</td>\n",
              "      <td>0.534902</td>\n",
              "      <td>0.545670</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>50%</th>\n",
              "      <td>0.566119</td>\n",
              "      <td>0.570346</td>\n",
              "      <td>0.584657</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>75%</th>\n",
              "      <td>0.605201</td>\n",
              "      <td>0.634758</td>\n",
              "      <td>0.624204</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>max</th>\n",
              "      <td>0.705696</td>\n",
              "      <td>0.731861</td>\n",
              "      <td>0.692063</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "             SVC  Decision Tree  K-Neighbours\n",
              "count  28.000000      28.000000     28.000000\n",
              "mean    0.567762       0.585009      0.583274\n",
              "std     0.053260       0.071605      0.059866\n",
              "min     0.459350       0.472222      0.415301\n",
              "25%     0.534224       0.534902      0.545670\n",
              "50%     0.566119       0.570346      0.584657\n",
              "75%     0.605201       0.634758      0.624204\n",
              "max     0.705696       0.731861      0.692063"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 59
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Hb7KKixDvAST"
      },
      "source": [
        "##### Plot these summary statistics to better visualise them"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 282
        },
        "id": "yZdR95JPsCzg",
        "outputId": "7720e6c7-9332-4490-d5ab-c2e7dabf741f"
      },
      "source": [
        "default_param_results.boxplot()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<matplotlib.axes._subplots.AxesSubplot at 0x7fe6f594fc50>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 60
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD4CAYAAADiry33AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAXp0lEQVR4nO3df7RV5X3n8fcnF3+kxlgM5k4GiWCKLYqRyC02jU0vtSJtumRm6WSgXal2SEhmBpsxrStQXGqwrMFmpu0kJU2oENOuVTCJHeeOMBA6cmJMasplQqLAYBDMElZmxQgxvYkBwe/8sZ/bbI/3x+aeDfee535ea511z3728+zz7LPv/dx9nv3jKCIwM7N8vW60O2BmZqeXg97MLHMOejOzzDnozcwy56A3M8vchNHuQLNJkybF1KlTR7sbp82PfvQjzjvvvNHuho2Qt1/7yn3b7dy58/sRcdFA88Zc0E+dOpXe3t7R7sZp02g06O7uHu1u2Ah5+7Wv3LedpO8MNs9DN2ZmmXPQm5llzkFvZpY5B72ZWeYc9GZmmXPQm5llzkFvZpY5B72ZWebG3AVTZqNFUi3L8Xc82FjjPXqzJCKGfVzy0UeGrWM21jjozcwy56A3M8ucg97MLHMOejOzzDnozcwy56A3M8ucg97MLHMOejOzzDnozcwy56A3M8ucg97MLHMOejOzzDnozcwy56A3M8ucg97MLHOVgl7SfEn7JO2XtGyA+X8maVd6PC3pB6V5t0j6dnrcUmfnxyJJQz7mzp07bJ26vgDDzAwqBL2kDmAN8BvA5cAiSZeX60TE7RExKyJmAZ8E/i61vRC4G7gGmAPcLWlivaswttTxxRX+8gozq1OVPfo5wP6IOBARx4GNwIIh6i8CNqTnNwDbIuJIRBwFtgHzW+mwmZmdmirfGTsZeK40fYhiD/01JF0CTAMeHaLt5AHaLQGWAHR2dtJoNCp0q33lvn658/ZrT319feN229X95eALgS9GxMlTaRQRa4G1AF1dXdHd3V1zt8aQLZvIev1y5+3XthqNxrjddlWGbg4DU0rTF6eygSzkp8M2p9rWzMxOgypBvwOYLmmapLMpwrynuZKkXwAmAv9QKt4KzJM0MR2EnZfKzMzsDBl26CYiTkhaShHQHcD6iNgtaSXQGxH9ob8Q2BilU0Yi4oikeyn+WQCsjIgj9a6CmZkNpdIYfURsBjY3ld3VNH3PIG3XA+tH2D8zs0rquv4kx9ObfWWsmWXB17AMzkFvZpY5B72ZWeYc9GZmmXPQm5llzkFvZpY5B72ZWeYc9GZmmXPQm5llzkFvZpY5B72ZWeYc9GZmmXPQm5llzkFvZpY5B72ZWeYc9GZmmXPQm5llzkFvZpY5B72ZWeYc9GZmmXPQm5llzkFvZpa5SkEvab6kfZL2S1o2SJ33Stojabekvy2Vn5S0Kz166uq4mZlVM2G4CpI6gDXA9cAhYIeknojYU6ozHVgOvCsijkp6c2kRL0XErJr7bWZmFVXZo58D7I+IAxFxHNgILGiq8wFgTUQcBYiI79XbTTMzG6lh9+iBycBzpelDwDVNdS4DkPRVoAO4JyK2pHnnSuoFTgCrI+Lh5heQtARYAtDZ2Umj0TiVdWg7ua9f7rz92td43XZVgr7qcqYD3cDFwGOSroyIHwCXRMRhSZcCj0p6MiKeKTeOiLXAWoCurq7o7u6uqVtj0JZNZL1+ufP2a1/jeNtVGbo5DEwpTV+cysoOAT0R8XJEHASepgh+IuJw+nkAaADvaLHPZmZ2CqoE/Q5guqRpks4GFgLNZ888TLE3j6RJFEM5ByRNlHROqfxdwB7MzOyMGXboJiJOSFoKbKUYf18fEbslrQR6I6InzZsnaQ9wErgjIl6Q9MvAZyS9QvFPZXX5bB0zMzv9Ko3RR8RmYHNT2V2l5wF8JD3Kdb4GXNl6N83MbKR8ZayZWeYc9GZmmXPQm5llzkFvZpa5ui6YMhvzrvrYl3jxpZdbXs7UZZtaan/B68/im3fPa7kfZlU56G3cePGll3l29XtaWkaj0Wj56spW/1GYnSoP3ZiZZc5Bb2aWOQe9mVnmHPRmZpnzwVgzawt1nDU1Xs+YctCbWVto9ayp8XzGlIduzMwy56A3M8ucg97MLHMOejOzzDnozcwy56A3M8ucg97MLHMOejOzzDnozcwy56A3M8tcpVsgSJoP/DegA7g/IlYPUOe9wD1AAN+MiN9O5bcAd6ZqfxwRn6uh36PG99sws3YzbNBL6gDWANcDh4AdknoiYk+pznRgOfCuiDgq6c2p/ELgbqCL4h/AztT2aP2rcmb4fhtm1m6qDN3MAfZHxIGIOA5sBBY01fkAsKY/wCPie6n8BmBbRBxJ87YB8+vpupmZVVEl6CcDz5WmD6WyssuAyyR9VdITaainalszMzuN6rpN8QRgOtANXAw8JunKqo0lLQGWAHR2dtJoNGrq1unRSv/6+vpqWb+x/h6NVa2+b95+o8t/eyNTJegPA1NK0xensrJDwNcj4mXgoKSnKYL/MEX4l9s2ml8gItYCawG6urqi1THs02rLppbG2OsYo2+1D+NWDe+bt98o8t/eiFUZutkBTJc0TdLZwEKgp6nOw6RAlzSJYijnALAVmCdpoqSJwLxUZmZmZ8iwe/QRcULSUoqA7gDWR8RuSSuB3ojo4aeBvgc4CdwRES8ASLqX4p8FwMqIOHI6VsTMzAZWaYw+IjYDm5vK7io9D+Aj6dHcdj2wvrVumpnZSPnKWDOzzDnozcwy56A3M8ucg97MLHN1XTBlZnZanT9jGVd+bllrC2nxlornzwAY+b2uRouD3szawj/tXe0bCo6Qh27MzDLnoDczy5yD3swscw56M7PMOejNzDLnoDczy5yD3swscz6P3saNWi64gXF70Y21Lwe9jRutXnAD4/uiG2tfHroxM8uc9+hPke+3YWbtxkF/iny/DTNrNx66MTPLnIPezCxzDnozs8w56M3MMuegNzPLXKWglzRf0j5J+yW95txCSbdKel7SrvR4f2neyVJ5T52dNzOz4Q17eqWkDmANcD1wCNghqSci9jRVfTAilg6wiJciYlbrXTUzs5Goskc/B9gfEQci4jiwEVhwertlZmZ1qXLB1GTgudL0IeCaAerdJOndwNPA7RHR3+ZcSb3ACWB1RDzc3FDSEmAJQGdnJ41Go/oajIJW+tfX11fL+o3192isavV98/YbXf7bG6GIGPIB3AzcX5p+H/AXTXXeBJyTnn8QeLQ0b3L6eSnwLPC2oV5v9uzZMZZd8tFHWmq/ffv2Ue/DeFXH++btN3r8tzc0oDcGydUqQzeHgSml6YtTWfmfxQsRcSxN3g/MLs07nH4eABrAOyr+DzIzsxpUGbrZAUyXNI0i4BcCv12uIOktEfHdNHkjsDeVTwR+HBHHJE0C3gX8SV2dN7PxpeX7PG1prf0Frz+rtdcfJcMGfUSckLQU2Ap0AOsjYreklRQfFXqA35d0I8U4/BHg1tR8BvAZSa9QHPhdHa89W8fMbFitfpfA1GWbWl5Gu6p098qI2Axsbiq7q/R8ObB8gHZfA65ssY9mZtYCXxlrZpY5B72ZWeYc9GZmmfM3TNm4Usu3c43TMzesfTnoR8CneLWnOs64GM9nblj7ctCfIp/iZWbtxmP0ZmaZc9CbmWXOQW9mljkHvZlZ5hz0ZmaZc9CbmWXOQW9mljkHvZlZ5hz0ZmaZc9CbmWXOQW9mljkHvZlZ5hz0ZmaZc9CbmWXOtyk2syxIGr7OfcMvJyJq6M3Y4j16M8tCRAz52L59+7B1cgx5qBj0kuZL2idpv6RlA8y/VdLzknalx/tL826R9O30uKXOzpuZ2fCGHbqR1AGsAa4HDgE7JPVExJ6mqg9GxNKmthcCdwNdQAA7U9ujtfTezMyGVWWPfg6wPyIORMRxYCOwoOLybwC2RcSRFO7bgPkj66qZmY1ElYOxk4HnStOHgGsGqHeTpHcDTwO3R8Rzg7Sd3NxQ0hJgCUBnZyeNRqNS59tV7uuXO2+/9tTX1zdut11dZ938T2BDRByT9EHgc8CvVW0cEWuBtQBdXV3R3d1dU7fGoC2byHr9cuft17Yajca43XZVhm4OA1NK0xensn8WES9ExLE0eT8wu2pbMzM7vaoE/Q5guqRpks4GFgI95QqS3lKavBHYm55vBeZJmihpIjAvlZmZ2Rky7NBNRJyQtJQioDuA9RGxW9JKoDcieoDfl3QjcAI4Atya2h6RdC/FPwuAlRFx5DSsh5mZDaLSGH1EbAY2N5XdVXq+HFg+SNv1wPoW+mhmZi3wlbFmZplz0JuZZc5Bb2aWOQe9mVnmHPRmZplz0JuZZc5Bb2aWOQe9mVnmHPRmZplz0JuZZc5Bb2aWubruR2/W9iRVq3ff0PNz/YJpa1/eozdLImLYx/bt24etYzbWOOjNzDLnoDczy5yD3swscw56M7PMOejNzDLnoDczy5zPo69ZlXOxhzsPG3wutpnVx3v0NavjPGyHvJnVyUFvZpa5SkEvab6kfZL2S1o2RL2bJIWkrjQ9VdJLknalx6fr6riZmVUz7Bi9pA5gDXA9cAjYIaknIvY01Tsf+DDw9aZFPBMRs2rqr5mZnaIqe/RzgP0RcSAijgMbgQUD1LsXuA/4SY39MzOzFlU562Yy8Fxp+hBwTbmCpKuBKRGxSdIdTe2nSfoG8EPgzoj4SvMLSFoCLAHo7Oyk0WhUX4M209fXl/X65c7br32N523X8umVkl4H/Clw6wCzvwu8NSJekDQbeFjSFRHxw3KliFgLrAXo6uqK7u7uVrs1ZjUaDXJev9x5+7Wv8bztqgzdHAamlKYvTmX9zgdmAg1JzwK/BPRI6oqIYxHxAkBE7ASeAS6ro+NmZlZNlaDfAUyXNE3S2cBCoKd/ZkS8GBGTImJqREwFngBujIheSRelg7lIuhSYDhyofS3MzGxQww7dRMQJSUuBrUAHsD4idktaCfRGRM8Qzd8NrJT0MvAK8KGIOFJHx83MrJpKY/QRsRnY3FR21yB1u0vPHwIeaqF/ZmbWIl8Za2aWOQe9mVnmHPRmZplz0JuZZc5Bb2aWOQe9mVnmHPRmZplz0JuZZc5Bb2aWOQe9mVnmHPRmZplz0JuZZc5Bb2aWOQe9mWVtw4YNzJw5k+uuu46ZM2eyYcOG0e7SGdfyVwmamY1VGzZsYMWKFaxbt46TJ0/S0dHB4sWLAVi0aNEo9+7M8R69mWVr1apVrFu3jrlz5zJhwgTmzp3LunXrWLVq1Wh37Yxy0JtZtvbu3cu11177qrJrr72WvXv3jlKPRoeD3syyNWPGDB5//PFXlT3++OPMmDFjlHo0Ohz0ZpatFStWsHjxYrZv386JEyfYvn07ixcvZsWKFaPdtTPKB2PNLFv9B1xvu+029u7dy4wZM1i1atW4OhALDnozy9yiRYtYtGgRjUaD7u7u0e7OqPDQjZlZ5ioFvaT5kvZJ2i9p2RD1bpIUkrpKZctTu32Sbqij02ZmVt2wQzeSOoA1wPXAIWCHpJ6I2NNU73zgw8DXS2WXAwuBK4B/Cfy9pMsi4mR9q2BmZkOpskc/B9gfEQci4jiwEVgwQL17gfuAn5TKFgAbI+JYRBwE9qflmZnZGVLlYOxk4LnS9CHgmnIFSVcDUyJik6Q7mto+0dR2cvMLSFoCLAHo7Oyk0WhU6nw76uvry3r9cuft177G87Zr+awbSa8D/hS4daTLiIi1wNq0vOfnzp37nVb7NYZNAr4/2p2wEfP2a1+5b7tLBptRJegPA1NK0xensn7nAzOBhiSAfwH0SLqxQtvXiIiLKvSpbUnqjYiu4WvaWOTt177G87arMka/A5guaZqksykOrvb0z4yIFyNiUkRMjYipFEM1N0ZEb6q3UNI5kqYB04F/rH0tzMxsUMPu0UfECUlLga1AB7A+InZLWgn0RkTPEG13S/o8sAc4AfxHn3FjZnZmKSJGuw/jiqQl6ZiEtSFvv/Y1nredg97MLHO+BYKZWeYc9GZmmXPQ10zSCkm7JX1L0i5Jd0v6z011Zknam56/QdJnJD0jaaekhqRrBl66STqZ3tfdkr4p6Q/StRwjWdZKSb8+xPwPSfrdkfcWJF2Z+rtL0hFJB9Pzv29lue1IUl/p+W9KelrSJU11npX0UGn6ZkkPDLPcLkmfGKbOVElPDTKvUb4/V458m+IaSXon8FvA1RFxTNIk4HLgAWB5qepCoP+r6O8HDgLTI+KVdBrq5Weu123npYiYBSDpzcDfAm8E7j7VBUXEXcPM//SIevjqZTwJ9Pf3AeCRiPhiuY6kCRFxotXXaheSrgM+AdwQEQNdHDlb0uXN99MaTDqVu7fOPrZKxUVFiohXRrsv4D36ur0F+H5EHAOIiO9HxGPA0aa99PcCGyS9jeJ2Enf2/0JExMGI2HSmO96OIuJ7FLfOWKpCh6SPS9qRPlF9sL+upI9KejJ9Clidyh6QdHN6vlrSntTuv6SyeyT9YXo+S9ITaf5/lzQxlTck3SfpH9Me6q9U6Xtq9+eSeoEPS5ot6cvpU91WSW9J9d4maUsq/4qkX6jxLTzjJL0b+CvgtyLimUGq/VfgNV8BJek8SevTe/0NSQtSebekR9LziyRtS5/47pf0nbTDBdAh6a/SvC9Jen1p8e9Ln7SekjQnLetCSQ+nbf6EpLen8n/+vUjTT6VPDFNV3KX3r4GngCnpd+yp9Lt3e2vv3sg56Ov1JYqN+7SkT0n61VS+gWIvHkm/BByJiG9T3NVzl68tGLmIOEBxfcebgcXAixHxi8AvAh9QcaHfb1DcYO+aiLgK+JPyMiS9CfjXwBUR8Xbgjwd4qb8GPprmP8mrP0FMiIg5wH/i1D5ZnJ2u1PwE8Eng5oiYDawHVqU6a4HbUvkfAp86heWPNecADwP/KiL+7xD1Pg9cLennmspXAI+m93ou8HFJ5zXVuTvVuQL4IvDW0rzpwJo07wfATaV5P5M+Kf4Hivcf4GPAN9I2/yOK34HhTAc+lV5jEjA5ImZGxJXAZyu0Py08dFOjiOiTNBv4FYpfxAdV3L//QeBrkv6AVw/bWL3mAW/v30sHLqD4w/t14LMR8WOAiDjS1O5Firuurkt7ho+UZ0q6APjZiPhyKvoc8IVSlb9LP3cCU0+hvw+mnz9PcRuRbcUnfjqA70p6A/DLwBdSORRh2a5eBr5G8Q/5w0PUOwl8nGK483+VyucBN5b2ps/l1UEOcC3FP20iYouko6V5ByNiV3revK02pDaPSXqjpJ9Ny7oplT8q6U2S3jjMOn4nIvpv5HgAuFTSJ4FNFDuCo8JBX7O0d96guPfPk8AtEfGApIPAr1L84rwzVd8NXCWpw3v1IyPpUopg+B4gir3frU11hvzCm3T19xzgOuBmYCnwa6fQjWPp50lO7W/qR/1dBHZHxDvLM1Oo/KD/mEQGXqEYtvzfkv6I4rbmO9O8nqZjJn9DEfTlA6gCboqIfeWFSuqs+PrHSs9PAuWhm+YLioa6wOgErx4NObf0vH+bEhFHJV0F3AB8iGLd/13FvtbKQzc1kvTzkqaXimYB/QebNgB/BhyIiEMAaYyyF/hYOnjTf3bAe85gt9uWpIuATwN/EcWVf1uBfy/prDT/svTRfhvwe5J+JpVf2LScNwAXRMRm4HbgqvL8iHiR4jhL//j7+4AvU599wEUqDuYj6SxJV0TED4GDkv5NKlcKjraVPlW9B/gd4NaImJUedzXVe5ni76U8rr0VuK30t/KOAV7iqxSBiqR5wMSKXfu3qc21FMN/LwJfSf1EUjfF8bcfAs8CV6fyq4FpAy0wHRt4XUQ8BNzZ32Y0eI++Xm8APpk+9p2g+KKVJWneFyjGYm9ravN+ioNP+yW9RHEb1Tuwwbxe0i7gLIr3+G8obpMNxRlMU4H/k8LgeYrx4C2SZgG9ko4DmynGXPudD/wPSedS7DV+ZIDXvQX4dPpncQD4vbpWKCKOp+GmT6RhognAn1N84vsd4C8l3ZnWeSPwzbpeezRExBFJ84HHJD0/xP2y1lEEZL97Kd6Xb6k4pfYgxVluZR+jONHhfcA/AP8P+CeKv82h/ETSNyje4/697nuA9ZK+BfyY4ncA4CHgdyXtpvhGvacHWeZk4LP66em/ywepd9r5Fghmlg1J5wAn03DcO4G/zGjoa8S8R29mOXkr8Pm0F30c+MAo92dM8B69mVnmfDDWzCxzDnozs8w56M3MMuegNzPLnIPezCxz/x/TOzcm6yovTQAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "11LLB7WcvQID"
      },
      "source": [
        "### Compare this against the baseline"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "f97rbg-Cw-is"
      },
      "source": [
        "##### Join the two datafarames together so we can easily compare them"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Rd8GgpRfw-is"
      },
      "source": [
        "col_order = [\"subject_id\", \"SVC_Baseline\", \"SVC_Default_Param\", \"Decision Tree_Baseline\", \"Decision Tree_Default_Param\", \"K-Neighbours_Baseline\", \"K-Neighbours_Default_Param\"]\n",
        "\n",
        "baseline_and_default_param = baseline_results.merge(default_param_results, how='inner', on='subject_id', suffixes=['_Baseline', '_Default_Param'])[col_order]"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ZnEJLQZhw-it"
      },
      "source": [
        "##### Output the joined dataframe to compare the classifier scores manually"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 917
        },
        "id": "Yl8jBLPew-iu",
        "outputId": "bf7586de-b419-4075-8cf7-c8b07b864378"
      },
      "source": [
        "baseline_and_default_param"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>subject_id</th>\n",
              "      <th>SVC_Baseline</th>\n",
              "      <th>SVC_Default_Param</th>\n",
              "      <th>Decision Tree_Baseline</th>\n",
              "      <th>Decision Tree_Default_Param</th>\n",
              "      <th>K-Neighbours_Baseline</th>\n",
              "      <th>K-Neighbours_Default_Param</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>46343</td>\n",
              "      <td>0.524590</td>\n",
              "      <td>0.480874</td>\n",
              "      <td>0.535519</td>\n",
              "      <td>0.622951</td>\n",
              "      <td>0.420765</td>\n",
              "      <td>0.415301</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>759667</td>\n",
              "      <td>0.554140</td>\n",
              "      <td>0.566879</td>\n",
              "      <td>0.649682</td>\n",
              "      <td>0.649682</td>\n",
              "      <td>0.503185</td>\n",
              "      <td>0.611465</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>781756</td>\n",
              "      <td>0.589506</td>\n",
              "      <td>0.537037</td>\n",
              "      <td>0.496914</td>\n",
              "      <td>0.475309</td>\n",
              "      <td>0.561728</td>\n",
              "      <td>0.558642</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>844359</td>\n",
              "      <td>0.548822</td>\n",
              "      <td>0.572391</td>\n",
              "      <td>0.612795</td>\n",
              "      <td>0.535354</td>\n",
              "      <td>0.535354</td>\n",
              "      <td>0.525253</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>1066528</td>\n",
              "      <td>0.523962</td>\n",
              "      <td>0.523962</td>\n",
              "      <td>0.549521</td>\n",
              "      <td>0.533546</td>\n",
              "      <td>0.533546</td>\n",
              "      <td>0.507987</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>5</th>\n",
              "      <td>1360686</td>\n",
              "      <td>0.620915</td>\n",
              "      <td>0.565359</td>\n",
              "      <td>0.653595</td>\n",
              "      <td>0.653595</td>\n",
              "      <td>0.637255</td>\n",
              "      <td>0.620915</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>6</th>\n",
              "      <td>1449548</td>\n",
              "      <td>0.685714</td>\n",
              "      <td>0.622222</td>\n",
              "      <td>0.698413</td>\n",
              "      <td>0.641270</td>\n",
              "      <td>0.666667</td>\n",
              "      <td>0.692063</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>7</th>\n",
              "      <td>1455390</td>\n",
              "      <td>0.705696</td>\n",
              "      <td>0.705696</td>\n",
              "      <td>0.705696</td>\n",
              "      <td>0.696203</td>\n",
              "      <td>0.715190</td>\n",
              "      <td>0.680380</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>8</th>\n",
              "      <td>1818471</td>\n",
              "      <td>0.533123</td>\n",
              "      <td>0.533123</td>\n",
              "      <td>0.564669</td>\n",
              "      <td>0.523659</td>\n",
              "      <td>0.545741</td>\n",
              "      <td>0.526814</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>9</th>\n",
              "      <td>2598705</td>\n",
              "      <td>0.609524</td>\n",
              "      <td>0.609524</td>\n",
              "      <td>0.536508</td>\n",
              "      <td>0.546032</td>\n",
              "      <td>0.523810</td>\n",
              "      <td>0.546032</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>10</th>\n",
              "      <td>2638030</td>\n",
              "      <td>0.709265</td>\n",
              "      <td>0.638978</td>\n",
              "      <td>0.619808</td>\n",
              "      <td>0.632588</td>\n",
              "      <td>0.648562</td>\n",
              "      <td>0.635783</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>11</th>\n",
              "      <td>3509524</td>\n",
              "      <td>0.630435</td>\n",
              "      <td>0.594203</td>\n",
              "      <td>0.760870</td>\n",
              "      <td>0.688406</td>\n",
              "      <td>0.681159</td>\n",
              "      <td>0.652174</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>12</th>\n",
              "      <td>3997827</td>\n",
              "      <td>0.643533</td>\n",
              "      <td>0.596215</td>\n",
              "      <td>0.738170</td>\n",
              "      <td>0.731861</td>\n",
              "      <td>0.668770</td>\n",
              "      <td>0.652997</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>13</th>\n",
              "      <td>4018081</td>\n",
              "      <td>0.606061</td>\n",
              "      <td>0.509091</td>\n",
              "      <td>0.709091</td>\n",
              "      <td>0.575758</td>\n",
              "      <td>0.606061</td>\n",
              "      <td>0.612121</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>14</th>\n",
              "      <td>4314139</td>\n",
              "      <td>0.569182</td>\n",
              "      <td>0.534591</td>\n",
              "      <td>0.490566</td>\n",
              "      <td>0.490566</td>\n",
              "      <td>0.575472</td>\n",
              "      <td>0.556604</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>15</th>\n",
              "      <td>4426783</td>\n",
              "      <td>0.638889</td>\n",
              "      <td>0.623457</td>\n",
              "      <td>0.583333</td>\n",
              "      <td>0.472222</td>\n",
              "      <td>0.620370</td>\n",
              "      <td>0.601852</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>16</th>\n",
              "      <td>5132496</td>\n",
              "      <td>0.655844</td>\n",
              "      <td>0.564935</td>\n",
              "      <td>0.512987</td>\n",
              "      <td>0.564935</td>\n",
              "      <td>0.623377</td>\n",
              "      <td>0.610390</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>17</th>\n",
              "      <td>5498603</td>\n",
              "      <td>0.573171</td>\n",
              "      <td>0.459350</td>\n",
              "      <td>0.552846</td>\n",
              "      <td>0.544715</td>\n",
              "      <td>0.573171</td>\n",
              "      <td>0.536585</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>18</th>\n",
              "      <td>5797046</td>\n",
              "      <td>0.603226</td>\n",
              "      <td>0.587097</td>\n",
              "      <td>0.596774</td>\n",
              "      <td>0.551613</td>\n",
              "      <td>0.593548</td>\n",
              "      <td>0.583871</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>19</th>\n",
              "      <td>6220552</td>\n",
              "      <td>0.574603</td>\n",
              "      <td>0.520635</td>\n",
              "      <td>0.615873</td>\n",
              "      <td>0.539683</td>\n",
              "      <td>0.603175</td>\n",
              "      <td>0.561905</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>20</th>\n",
              "      <td>7749105</td>\n",
              "      <td>0.536585</td>\n",
              "      <td>0.536585</td>\n",
              "      <td>0.878049</td>\n",
              "      <td>0.707317</td>\n",
              "      <td>0.609756</td>\n",
              "      <td>0.634146</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>21</th>\n",
              "      <td>8000685</td>\n",
              "      <td>0.589905</td>\n",
              "      <td>0.605678</td>\n",
              "      <td>0.605678</td>\n",
              "      <td>0.621451</td>\n",
              "      <td>0.596215</td>\n",
              "      <td>0.602524</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>22</th>\n",
              "      <td>8173033</td>\n",
              "      <td>0.566456</td>\n",
              "      <td>0.534810</td>\n",
              "      <td>0.610759</td>\n",
              "      <td>0.506329</td>\n",
              "      <td>0.585443</td>\n",
              "      <td>0.566456</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>23</th>\n",
              "      <td>8530312</td>\n",
              "      <td>0.624204</td>\n",
              "      <td>0.592357</td>\n",
              "      <td>0.538217</td>\n",
              "      <td>0.519108</td>\n",
              "      <td>0.627389</td>\n",
              "      <td>0.544586</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>24</th>\n",
              "      <td>8686948</td>\n",
              "      <td>0.632911</td>\n",
              "      <td>0.623418</td>\n",
              "      <td>0.598101</td>\n",
              "      <td>0.601266</td>\n",
              "      <td>0.601266</td>\n",
              "      <td>0.585443</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>25</th>\n",
              "      <td>8692923</td>\n",
              "      <td>0.579288</td>\n",
              "      <td>0.508091</td>\n",
              "      <td>0.627832</td>\n",
              "      <td>0.601942</td>\n",
              "      <td>0.521036</td>\n",
              "      <td>0.508091</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>26</th>\n",
              "      <td>9106476</td>\n",
              "      <td>0.602524</td>\n",
              "      <td>0.545741</td>\n",
              "      <td>0.589905</td>\n",
              "      <td>0.552050</td>\n",
              "      <td>0.630915</td>\n",
              "      <td>0.634069</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>27</th>\n",
              "      <td>9961348</td>\n",
              "      <td>0.605042</td>\n",
              "      <td>0.605042</td>\n",
              "      <td>0.617647</td>\n",
              "      <td>0.600840</td>\n",
              "      <td>0.613445</td>\n",
              "      <td>0.567227</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "   subject_id  SVC_Baseline  SVC_Default_Param  Decision Tree_Baseline  \\\n",
              "0       46343      0.524590           0.480874                0.535519   \n",
              "1      759667      0.554140           0.566879                0.649682   \n",
              "2      781756      0.589506           0.537037                0.496914   \n",
              "3      844359      0.548822           0.572391                0.612795   \n",
              "4     1066528      0.523962           0.523962                0.549521   \n",
              "5     1360686      0.620915           0.565359                0.653595   \n",
              "6     1449548      0.685714           0.622222                0.698413   \n",
              "7     1455390      0.705696           0.705696                0.705696   \n",
              "8     1818471      0.533123           0.533123                0.564669   \n",
              "9     2598705      0.609524           0.609524                0.536508   \n",
              "10    2638030      0.709265           0.638978                0.619808   \n",
              "11    3509524      0.630435           0.594203                0.760870   \n",
              "12    3997827      0.643533           0.596215                0.738170   \n",
              "13    4018081      0.606061           0.509091                0.709091   \n",
              "14    4314139      0.569182           0.534591                0.490566   \n",
              "15    4426783      0.638889           0.623457                0.583333   \n",
              "16    5132496      0.655844           0.564935                0.512987   \n",
              "17    5498603      0.573171           0.459350                0.552846   \n",
              "18    5797046      0.603226           0.587097                0.596774   \n",
              "19    6220552      0.574603           0.520635                0.615873   \n",
              "20    7749105      0.536585           0.536585                0.878049   \n",
              "21    8000685      0.589905           0.605678                0.605678   \n",
              "22    8173033      0.566456           0.534810                0.610759   \n",
              "23    8530312      0.624204           0.592357                0.538217   \n",
              "24    8686948      0.632911           0.623418                0.598101   \n",
              "25    8692923      0.579288           0.508091                0.627832   \n",
              "26    9106476      0.602524           0.545741                0.589905   \n",
              "27    9961348      0.605042           0.605042                0.617647   \n",
              "\n",
              "    Decision Tree_Default_Param  K-Neighbours_Baseline  \\\n",
              "0                      0.622951               0.420765   \n",
              "1                      0.649682               0.503185   \n",
              "2                      0.475309               0.561728   \n",
              "3                      0.535354               0.535354   \n",
              "4                      0.533546               0.533546   \n",
              "5                      0.653595               0.637255   \n",
              "6                      0.641270               0.666667   \n",
              "7                      0.696203               0.715190   \n",
              "8                      0.523659               0.545741   \n",
              "9                      0.546032               0.523810   \n",
              "10                     0.632588               0.648562   \n",
              "11                     0.688406               0.681159   \n",
              "12                     0.731861               0.668770   \n",
              "13                     0.575758               0.606061   \n",
              "14                     0.490566               0.575472   \n",
              "15                     0.472222               0.620370   \n",
              "16                     0.564935               0.623377   \n",
              "17                     0.544715               0.573171   \n",
              "18                     0.551613               0.593548   \n",
              "19                     0.539683               0.603175   \n",
              "20                     0.707317               0.609756   \n",
              "21                     0.621451               0.596215   \n",
              "22                     0.506329               0.585443   \n",
              "23                     0.519108               0.627389   \n",
              "24                     0.601266               0.601266   \n",
              "25                     0.601942               0.521036   \n",
              "26                     0.552050               0.630915   \n",
              "27                     0.600840               0.613445   \n",
              "\n",
              "    K-Neighbours_Default_Param  \n",
              "0                     0.415301  \n",
              "1                     0.611465  \n",
              "2                     0.558642  \n",
              "3                     0.525253  \n",
              "4                     0.507987  \n",
              "5                     0.620915  \n",
              "6                     0.692063  \n",
              "7                     0.680380  \n",
              "8                     0.526814  \n",
              "9                     0.546032  \n",
              "10                    0.635783  \n",
              "11                    0.652174  \n",
              "12                    0.652997  \n",
              "13                    0.612121  \n",
              "14                    0.556604  \n",
              "15                    0.601852  \n",
              "16                    0.610390  \n",
              "17                    0.536585  \n",
              "18                    0.583871  \n",
              "19                    0.561905  \n",
              "20                    0.634146  \n",
              "21                    0.602524  \n",
              "22                    0.566456  \n",
              "23                    0.544586  \n",
              "24                    0.585443  \n",
              "25                    0.508091  \n",
              "26                    0.634069  \n",
              "27                    0.567227  "
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 108
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hRV6rEs75GPW"
      },
      "source": [
        "Again in this experiment, it is difficult to pinpoint which moddle is best from this table. For some subject_ids the classifier using the default parameters is best while in others the basline is best.\n",
        "\n",
        "Due to the nature of parameters in general, this is always going to be the case as some parameter values work better in different situations.\n",
        "\n",
        "It is important to agreate these values and get a summary of the overall performance of these parameters across all subjects."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "0Ao-q9zAw-iw"
      },
      "source": [
        "##### Describe this dataframe"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 297
        },
        "id": "YpevB4tJw-iw",
        "outputId": "e6f28808-c980-4e54-c585-8b4e21a74693"
      },
      "source": [
        "baseline_and_default_param.describe()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>SVC_Baseline</th>\n",
              "      <th>SVC_Default_Param</th>\n",
              "      <th>Decision Tree_Baseline</th>\n",
              "      <th>Decision Tree_Default_Param</th>\n",
              "      <th>K-Neighbours_Baseline</th>\n",
              "      <th>K-Neighbours_Default_Param</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>count</th>\n",
              "      <td>28.000000</td>\n",
              "      <td>28.000000</td>\n",
              "      <td>28.000000</td>\n",
              "      <td>28.000000</td>\n",
              "      <td>28.000000</td>\n",
              "      <td>28.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>mean</th>\n",
              "      <td>0.601326</td>\n",
              "      <td>0.567762</td>\n",
              "      <td>0.616065</td>\n",
              "      <td>0.585009</td>\n",
              "      <td>0.593656</td>\n",
              "      <td>0.583274</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>std</th>\n",
              "      <td>0.050517</td>\n",
              "      <td>0.053260</td>\n",
              "      <td>0.086878</td>\n",
              "      <td>0.071605</td>\n",
              "      <td>0.061696</td>\n",
              "      <td>0.059866</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>min</th>\n",
              "      <td>0.523962</td>\n",
              "      <td>0.459350</td>\n",
              "      <td>0.490566</td>\n",
              "      <td>0.472222</td>\n",
              "      <td>0.420765</td>\n",
              "      <td>0.415301</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>25%</th>\n",
              "      <td>0.568501</td>\n",
              "      <td>0.534224</td>\n",
              "      <td>0.552014</td>\n",
              "      <td>0.534902</td>\n",
              "      <td>0.557732</td>\n",
              "      <td>0.545670</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>50%</th>\n",
              "      <td>0.602875</td>\n",
              "      <td>0.566119</td>\n",
              "      <td>0.608219</td>\n",
              "      <td>0.570346</td>\n",
              "      <td>0.602220</td>\n",
              "      <td>0.584657</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>75%</th>\n",
              "      <td>0.631054</td>\n",
              "      <td>0.605201</td>\n",
              "      <td>0.650660</td>\n",
              "      <td>0.634758</td>\n",
              "      <td>0.628270</td>\n",
              "      <td>0.624204</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>max</th>\n",
              "      <td>0.709265</td>\n",
              "      <td>0.705696</td>\n",
              "      <td>0.878049</td>\n",
              "      <td>0.731861</td>\n",
              "      <td>0.715190</td>\n",
              "      <td>0.692063</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "       SVC_Baseline  SVC_Default_Param  Decision Tree_Baseline  \\\n",
              "count     28.000000          28.000000               28.000000   \n",
              "mean       0.601326           0.567762                0.616065   \n",
              "std        0.050517           0.053260                0.086878   \n",
              "min        0.523962           0.459350                0.490566   \n",
              "25%        0.568501           0.534224                0.552014   \n",
              "50%        0.602875           0.566119                0.608219   \n",
              "75%        0.631054           0.605201                0.650660   \n",
              "max        0.709265           0.705696                0.878049   \n",
              "\n",
              "       Decision Tree_Default_Param  K-Neighbours_Baseline  \\\n",
              "count                    28.000000              28.000000   \n",
              "mean                      0.585009               0.593656   \n",
              "std                       0.071605               0.061696   \n",
              "min                       0.472222               0.420765   \n",
              "25%                       0.534902               0.557732   \n",
              "50%                       0.570346               0.602220   \n",
              "75%                       0.634758               0.628270   \n",
              "max                       0.731861               0.715190   \n",
              "\n",
              "       K-Neighbours_Default_Param  \n",
              "count                   28.000000  \n",
              "mean                     0.583274  \n",
              "std                      0.059866  \n",
              "min                      0.415301  \n",
              "25%                      0.545670  \n",
              "50%                      0.584657  \n",
              "75%                      0.624204  \n",
              "max                      0.692063  "
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 109
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "VgBMTgvH55dO"
      },
      "source": [
        "This summary informs us a bit more about the performance of the default parameters Vs our baseline optimised parameters.\n",
        "\n",
        "This data description unresoundingly points towards the baseline being better. As expected, across all classifiers, the minimum classification, the maximum classificaton, and the mean classification is better using the optimised parameters.\n",
        "\n",
        "This validates our hypothesis yet again."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "YBCfR04Ow-ix"
      },
      "source": [
        "##### Plot the classifiers"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 449
        },
        "id": "Z6vZcCnYw-ix",
        "outputId": "6dcfe70f-171d-4332-8228-a42440c494ec"
      },
      "source": [
        "baseline_and_default_param.boxplot(figsize=(18,7))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<matplotlib.axes._subplots.AxesSubplot at 0x7fe6f22fc438>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 111
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAABBEAAAGfCAYAAAAJeUpkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3de7hkd1kn+u9LOggmISrBHAWkMxq1mSAo/YBodLoNIooGHXCkVTROMKPPIXjDQ5h2IKCtQVC8gU6kMfEyHbkoE0kmiUJvQxQxiZJA0obJJPEQHIeLkGMjkgu/88daO6ne7Muvk+quqvTn8zz72VWrVq1696p3r6r6rt9aVa21AAAAAGzkIbMuAAAAAFgMQgQAAACgixABAAAA6CJEAAAAALoIEQAAAIAuQgQAAACgS1eIUFXPrKqbqurmqjpnldsfV1XvqKrrq2qpqh4z/VIBAACAWarW2vozVB2V5ANJvjnJ7UmuTrKjtXbjxDxvTvL21tqFVfVNSX6otfb8Q1c2AAAAcLht6pjnKUlubq3dkiRVdVGSZye5cWKexyf5yfHy3iRv22ihJ5xwQtu8efNBFcvaPvnJT+aYY46ZdRnwWfQm80x/Mq/0JvNKbzLP9Of0XHvttR9trT1qtdt6QoRHJ/ngxPXbkzx1xTzXJfn3SX41yXclOa6qHtla+9jkTFV1VpKzkuTEE0/Ma17zmr6/gA3t378/xx577KzLgM+iN5ln+pN5pTeZV3qTeaY/p2f79u1/v9ZtPSFCjxcn+Y2qOiPJlUk+lOSelTO11s5Pcn6SbN26tW3btm1KD8/S0lKsT+aR3mSe6U/mld5kXulN5pn+PDx6QoQPJXnsxPXHjNPu1Vr7hwwjEVJVxyZ5TmvtE9MqEgAAAJi9nm9nuDrJyVV1UlU9NMnzklw8OUNVnVBVy8t6aZI3TrdMAAAAYNY2DBFaa3cneWGSy5PsS/Km1toNVfXKqjp9nG1bkpuq6gNJTkyy6xDVCwAAAMxI1zkRWmuXJrl0xbSXTVx+S5K3TLc0AAAAYJ70HM4AAAAAIEQAAAAA+ggRAAAAgC5CBAAAAKCLEAEAAADoIkQAAAAAuggRAAAAgC5CBAAAAKCLEAGAI86ePXtyyimn5LTTTsspp5ySPXv2zLokAICFsGnWBQDA4bRnz57s3Lkzu3fvzj333JOjjjoqZ555ZpJkx44dM64OAGC+GYkAwBFl165d2b17d7Zv355NmzZl+/bt2b17d3bt2jXr0gAA5p4QAYAjyr59+3LqqaceMO3UU0/Nvn37ZlQRAMDiECIAcETZsmVLrrrqqgOmXXXVVdmyZcuMKgIAWBxCBACOKDt37syZZ56ZvXv35u67787evXtz5plnZufOnbMuDQBg7jmxIgBHlOWTJ5599tnZt29ftmzZkl27djmpIgBAByECAEecHTt2ZMeOHVlaWsq2bdtmXQ4AwMJwOAMAAADQRYgAAAAAdBEiAAAAAF2ECAAAAEAXIQIAAADQRYgAAAAAdBEiAAAAAF2ECAAAAEAXIQIAAADQRYgAAAAAdBEiAAAAAF2ECAAAAEAXIQIAAADQRYgAAAAAdBEiAAAAAF2ECAAAAEAXIQIAAADQRYgAAAAAdBEiAAAAAF2ECAAAAEAXIQIAAADQRYgAAAAAdBEiAAAAAF2ECAAAAEAXIQIAAADQRYgAAAAAdBEiAAAAAF2ECAAAAEAXIQIAAADQRYgAAAAAdBEiAAAAAF2ECAAAAEAXIQIAAADQRYgAAAAAdBEiAAAAAF2ECAAAAEAXIQIAAADQRYgAAAAAdBEiAAAAAF2ECAAAAEAXIQIAAADQRYgAAAAAdBEiAAAAAF2ECAAAAEAXIQIAAADQRYgAAAAAdBEiAAAAAF2ECAAAAEAXIQIAAADQRYgAAAAAdOkKEarqmVV1U1XdXFXnrHL7l1TV3qr626q6vqq+bfqlAgAAALO0YYhQVUcleV2Sb03y+CQ7qurxK2b7mSRvaq19dZLnJXn9tAsFAAAAZqtnJMJTktzcWrultXZnkouSPHvFPC3JI8bLxyf5h+mVCAAAAMyDaq2tP0PVc5M8s7X2gvH685M8tbX2wol5vijJFUk+P8kxSZ7eWrt2lWWdleSsJDnxxBOffNFFF03r7zji7d+/P8cee+ysy4DPojeZZ/qTeaU3mVd6k3mmP6dn+/bt17bWtq5226YpPcaOJBe01n6pqp6W5Peq6pTW2mcmZ2qtnZ/k/CTZunVr27Zt25QenqWlpVifzCO9yTzTn8wrvcm80pvMM/15ePQczvChJI+duP6YcdqkM5O8KUlaa+9O8rAkJ0yjQAAAAGA+9IQIVyc5uapOqqqHZjhx4sUr5vl/k5yWJFW1JUOI8JFpFgoAAADM1oYhQmvt7iQvTHJ5kn0ZvoXhhqp6ZVWdPs72U0l+uKquS7InyRlto5MtAAAAAAul65wIrbVLk1y6YtrLJi7fmOTrp1saAAAAME96DmcAAAAAECIAAAAAfYQIAAAAQBchAgAAANBFiAAAAAB0ESIAAAAAXYQIAAAAQBchAgAAANBFiAAAAAB0ESIAAAAAXYQIAAAAQBchAgAAANBFiAAAAAB0ESIAAAAAXYQIAAAAQBchAgAAANBFiAAAAAB0ESIAAAAAXYQIAAAAQBchAgAAANBFiAAAAAB0ESIAAAAAXYQIAAAAQJdNsy4AAHpU1axL6NZam3UJAACHhJEIACyE1trUfx73krcfkuUCADxYCREAAACALkIEAAAAoIsQAQAAAOgiRAAAAAC6CBEAAACALkIEAAAAoIsQAQAAAOgiRAAAAAC6CBEAAACALkIEAAAAoIsQAQAAAOgiRAAAAAC6CBEAAACALkIEAAAAoIsQAQAAAOgiRAAAAAC6CBEAAACALkIEAAAAoIsQAQAAAOgiRAAAAAC6bJp1AQAAi6yqZl1Ct9barEsAYMEZiQAA8AC01qb+87iXvP2QLBcAHighAgAAANBFiAAAAAB0ESIAAAAAXYQIAAAAQBchAgAAANBFiAAAAAB0ESIAAAAAXYQIAAAAQBchAgAAANBFiAAAAAB02TTrAgAAgOmrqlmX0K21NusSgE5GIgAAwINQa23qP497ydsPyXKBxSFEAAAAALoIEQAAAIAuQgQAAACgixMrzoCT3AAAALCIjESYASe5AQAAYBEJEQAAAIAuDmcA7uVQGwAAYD1dIxGq6plVdVNV3VxV56xy+2ur6r3jzweq6hPTLxU41BxqAwAArGfDkQhVdVSS1yX55iS3J7m6qi5urd24PE9r7Scm5j87yVcfgloBAACAGeoZifCUJDe31m5prd2Z5KIkz15n/h1J9kyjOAAAAGB+9JwT4dFJPjhx/fYkT11txqp6XJKTkrxzjdvPSnJWkpx44olZWlo6mFrZgPXJvNKbzDP9ybzSm8wrvcm82r9/v/48DKZ9YsXnJXlLa+2e1W5srZ2f5Pwk2bp1a9u2bduUH/4IdtklsT6ZS3qTeaY/mVd6k3mlN5ljS0tL+vMw6Dmc4UNJHjtx/THjtNU8Lw5lAAAAgAelnhDh6iQnV9VJVfXQDEHBxStnqqqvTPL5Sd493RIBAACAebBhiNBauzvJC5NcnmRfkje11m6oqldW1ekTsz4vyUXNd68BAADAg1LXORFaa5cmuXTFtJetuH7u9MoCAAAA5s20T6wIAAAA66qqWZfQzWD7A/WcEwEAAACmprU29Z/HveTth2S5HEiIAAAAAHQRIgAAAABdhAgAAABAFyECAAAA0EWIAAAAAHQRIgAAAABdhAgAAABAFyECAAAA0EWIAAAAAHQRIgAAAABdhAgAAABAFyECAAAA0EWIAAAAAHQRIgAAAABdhAgAAABAFyECAAAA0EWIAAAAAHQRIgAAAABdhAgAAABAFyECAAAA0EWIAAAAAHQRIgAAAABdhAgAAABAFyECAAAA0EWIAAAAAHQRIgAAAABdhAgAAABAFyECAAAA0EWIAAAAAHTZNOsCAHjweeIrrsgdn7pr1mV02XzOJbMuYV3HP/zoXPfyZ8y6DACAJEIEAA6BOz51V24771mzLmNDS0tL2bZt26zLWNe8hxwAwJHF4QwAAABAFyECAAAA0EWIAAAAAHQRIgAAAABdhAgAAABAFyECAAAA0EWIAAAAAHQRIgAAAABdhAgAAABAFyECAAAA0GXTrAsAADhcnviKK3LHp+6adRldNp9zyaxL2NDxDz861738GbMuA4DDSIgAABwx7vjUXbntvGfNuowNLS0tZdu2bbMuY0OLEHQAMF1ChA3YYzFd9lgAAAAsLiHCBuyxmK5FCDoAAABYnRABAADmwKKMgF2EnUJGv8KhI0QAAIA5sAgjYI1+BXzFIwAAANDFSAQAAADWtCiH2iSLMQpl0Q+3ESIAAACwpkU41CZxuM3h4nAGAAAAoIsQAQAAAOgiRAAAAAC6CBEAAACALkIEAAAAoIsQAQAAAOgiRAAAAAC6CBEAAACALkIEAAAAoIsQAQAAAOiyadYFAPDgc9yWc/KEC8+ZdRl9Lpx1Aes7bkuSPGvWZQAAJBEiAHAI/PO+83LbefP/wXdpaSnbtm2bdRnr2nzOJbMuAQDgXg5nAAAAALp0jUSoqmcm+dUkRyV5Q2vtvFXm+Q9Jzk3SklzXWvveKdYJrPDEV1yROz5116zL6LIIe1KPf/jRue7lz5h1GQAAMNc2DBGq6qgkr0vyzUluT3J1VV3cWrtxYp6Tk7w0yde31j5eVV94qAo+3BzXO12O7Z2eOz51l+HiU7QIQQcAAMxaz0iEpyS5ubV2S5JU1UVJnp3kxol5fjjJ61prH0+S1tqHp13orDiud7p8UAMAWN3C7Lyy4wqOaD0hwqOTfHDi+u1Jnrpini9Pkqr6iwyHPJzbWrtsKhUCAMARYBF2XtlxBUzr2xk2JTk5ybYkj0lyZVU9obX2icmZquqsJGclyYknnpilpaUpPfyhtQh17t+/fyHqTBZjfS6KRViXevPItQjrc1H6cxFqXCSLsD4XpTeTxVifi2Te16fePHItwvrUn4dHT4jwoSSPnbj+mHHapNuTvKe1dleSW6vqAxlChasnZ2qtnZ/k/CTZunVrW4QUM5ddshBp66KkwouyPhfCgqxLvXmEWpD1uRD9uSDrcmEsyPpciN5MFmZ9LowFWJ968wi1IOtTfx4ePV/xeHWSk6vqpKp6aJLnJbl4xTxvyzAKIVV1QobDG26ZYp0AAADAjG0YIrTW7k7ywiSXJ9mX5E2ttRuq6pVVdfo42+VJPlZVNybZm+SnW2sfO1RFAwAAAIdf1zkRWmuXJrl0xbSXTVxuSX5y/AEAAAAehHoOZwAAAAAQIgAAAAB9hAgAAABAFyECAAAA0EWIAAAAAHQRIgAAAABdur7iEQAAgCPTcVvOyRMuPGfWZfS5cNYFbOy4LUnyrFmXcb8JEQAAAFjTP+87L7edN/8fepeWlrJt27ZZl7GhzedcMusSHhCHMwAAAABdhAgAAABAFyECAAAA0EWIAAAAAHQRIgAAAABdhAgAAABAFyECAAAA0EWIAAAAAHQRIgAAAABdNs26AACAw+W4LefkCReeM+sy+lw46wI2dtyWJHnWrMsA4DASInTYfM4lsy6hz2XzX+fxDz961iUAcAT7533n5bbz5v9D79LSUrZt2zbrMja0MO+RAJgaIcIGFuGNRjK8iC9KrQAAACwmIQIsKENyp8uQXAAA2JgQARaUIbnTZUguAABszLczAAAAAF2ECAAAAEAXIQIAAADQRYgAAAAAdHFiRQAAmBMLcaLfy+a/xuMffvSsS4AHLSECAADMgUX41qXN51yyEHUCh47DGQAAAIAuQgQAAACgi8MZADgkFuK43mTuj+11XC8AME+ECABM3aIcL+vYXgCAg+NwBgAAAKCLEAEAAADoIkQAAAAAujgnAgAAAOtywuTpWfSTJgsRAAAAWNOinITYCZMPD4czAAAAAF2ECAAAAEAXIQIAAADQRYgAAAAAdBEiAAAAAF2ECAAAAEAXX/EIABxRfNf59Cz6d50DcPCECLDAvBGeHm+E4ciwKN8f7rvOAZhXQgRYUIvy5tIbYQAAePBwTgQAAACgixABAAAA6CJEAAAAALoIEQAAAIAuQgQAAACgi29nmIGqOjTLfdX0l9lam/5CAQAAWEhGIsxAa23qP3v37j0kywUAAIBlQgQAAACgixABAAAA6CJEAAAAALoIEQAAAIAuQgQAAACgixABAAAA6CJEAAAAALoIEQAAAIAuQgQAAACgixABAAAA6LJp1gUAAABwZKmqQ7PcV01/ma216S90gRmJAAAAwGHVWpv6z969ew/JcjmQEAEAAADo0hUiVNUzq+qmqrq5qs5Z5fYzquojVfXe8ecF0y8VAAAAmKUNz4lQVUcleV2Sb05ye5Krq+ri1tqNK2b9w9baCw9BjQAAwEFyzDlwKPSMRHhKkptba7e01u5MclGSZx/asgAAgAfCMefAodDz7QyPTvLBieu3J3nqKvM9p6q+MckHkvxEa+2DK2eoqrOSnJUkJ554YpaWlg66YFa3f/9+65O5pTeZZ/qTeaU3mUfeczLP9OfhMa2vePyTJHtaa5+uqv+U5MIk37Ryptba+UnOT5KtW7e2bdu2TenhWVpaivXJXLrsEr3J/NKfzCu9yZzynpN5pj8Pj57DGT6U5LET1x8zTrtXa+1jrbVPj1ffkOTJ0ykPAAAAmBc9IcLVSU6uqpOq6qFJnpfk4skZquqLJq6enmTf9EoEAAAA5sGGhzO01u6uqhcmuTzJUUne2Fq7oapemeSa1trFSV5UVacnuTvJPyU54xDWDAAAAMxA1zkRWmuXJrl0xbSXTVx+aZKXTrc0AAAAYJ70HM4AAAAAMLVvZwCAQ6qqDs1yXzX9ZfrOcwDgwcpIBAAWQmtt6j979+49JMsFAHiwEiIAAAAAXYQIAAAAQBchAgAAANBFiAAAAAB0ESIAAAAAXYQIAAAAQBchAgAAANBFiAAAAAB0ESIAAAAAXYQIAAAAQBchAgAAANBFiAAAAAB02TTrAgAAFllVHZrlvmr6y2ytTX+hABxRjEQAAHgAWmtT/9m7d+8hWS4APFBCBAAAAKCLEAEAAADoIkQAAAAAuggRAAAAgC5CBAAAAKCLEAEAAADoIkQAAAAAuggRAAAAgC6bZl0AMD+q6tAs91XTX2ZrbfoLBQAA1mUkAnCv1trUf/bu3XtIlgsAABx+QgQAAACgixABAAAA6CJEAAAAALoIEQAAAIAuQgQAAACgixABAAAA6CJEAAAAALoIEQAAAIAuQgQAAACgixABAAAA6CJEAAAAALoIEQAAAIAuQgQAAACgixABAAAA6CJEAAAAALoIEQAAAIAuQgQAAACgixABAAAA6CJEAAAAALoIEQAAAIAuQgQAAACgixABAAAA6CJEAAAAALoIEQAAAIAuQgQAAGBde/bsySmnnJLTTjstp5xySvbs2TPrkoAZ2TTrAgAAgPm1Z8+e7Ny5M7t3784999yTo446KmeeeWaSZMeOHTOuDjjcjEQAAADWtGvXruzevTvbt2/Ppk2bsn379uzevTu7du2adWnADAgRAACANe3bty+nnnrqAdNOPfXU7Nu3b0YVAbMkRAAAANa0ZcuWXHXVVQdMu+qqq7Jly5YZVQQHcs6Ow8s5EQAAgDXt3LkzZ5555r3nRNi7d2/OPPNMhzMwF5yz4/ATIgAAAGta/iB29tlnZ9++fdmyZUt27drlAxpzYfKcHUtLS9m2bVt2796ds88+W48eIkIEAABgXTt27MiOHTvu/ZAG88I5Ow4/50QAAABgITlnx+EnRAAAAGAhLZ+zY+/evbn77rvvPWfHzp07Z13ag5bDGQAAAFhIztlx+AkRAAAAWFjO2XF4OZwBAAAA6CJEAAAAALoIEQAAAIAuXSFCVT2zqm6qqpur6px15ntOVbWq2jq9EgEAAIB5sGGIUFVHJXldkm9N8vgkO6rq8avMd1ySH0vynmkXCQAAAMxez0iEpyS5ubV2S2vtziQXJXn2KvP9bJJXJfnXKdYHAAAAzImer3h8dJIPTly/PclTJ2eoqq9J8tjW2iVV9dNrLaiqzkpyVpKceOKJWVpaOuiCWd3+/futT+aS3mSe6U/mld5kXulN5pn+PDx6QoR1VdVDkvxykjM2mre1dn6S85Nk69atzXd4To/vRGVe6U3mmf5kXulN5pXeZJ7pz8Oj53CGDyV57MT1x4zTlh2X5JQkS1V1W5KvTXKxkysCAADAg0tPiHB1kpOr6qSqemiS5yW5ePnG1todrbUTWmubW2ubk/xVktNba9cckooBAACAmdgwRGit3Z3khUkuT7IvyZtaazdU1Sur6vRDXSAAAAAwH7rOidBauzTJpSumvWyNebc98LIAAACAeVOttdk8cNVHkvz9TB78wemEJB+ddRGwCr3JPNOfzCu9ybzSm8wz/Tk9j2utPWq1G2YWIjBdVXVNa83JLJk7epN5pj+ZV3qTeaU3mWf68/DoObEiAAAAgBABAAAA6CNEePA4f9YFwBr0JvNMfzKv9CbzSm8yz/TnYeCcCAAAAEAXIxEAAACALkIEAAAAoIsQAQAAAOgiRJiSqtpZVTdU1fVV9d6qenlV/cKKeZ5UVfvGy8dW1X+tqv9VVddW1VJVPXWd5d8zLve6qvqbqvq6Kdd/QVU9d7z8hqp6/DSXzwNzGPvrhrHHfqqqNtw+VNWrx/u8+n7+XfvH35ur6ns3mHdbVd0x1rmvql5+fx6Tg3N/e2ONZb2yqp6+zu0/UlU/cP+rTarqCWO9762qf6qqW8fLf/ZAlrvGY51bVR8al/93VfWb93fdrLH8zVX1/vHy1qr6tWkte17or3Ufa7K//mdV/VHPa3NVfeV4n7+tqi+9n4/74vHyGVX1xRvMv1RVN43P319U1Vcc7GM+2Cy/to2Xv62qPlBVj1sxz21V9daJ68+tqgs2WO6G24HJ7cYqty1V1dauP+IQGd9vLv/f/N20X8vH9wpvHy+fXlXnTHP580B/rW2iv64b18vvVtVjOu73DePr0Hur6uH383GXP0f9eFV97gbz31ZV76vhff0VVfV/HexjztqmWRfwYFBVT0vy7Um+prX26ao6Icnjk1yQ5KUTsz4vyZ7x8huS3Jrk5NbaZ6rqpPE+a/lUa+1J4+N9S5JfSPLvpvqHjFprLzgUy+X+mUF/fWGS/5bkEUk2enE/K8kXtNbuObi/6rNsTvK94+Ou512ttW+vqmOSvLeq/qS19jcbLbyqNrXW7n6ANR6p7m9vfJbW2ss2uP237leFBy7jfUmW670gydtba2+ZnGfK/fDa1tprxg++V2bYLu+d0rLv1Vq7Jsk1017uHNBf63tta+0143K/J8k7q+oJrbWPrHOf70zyltbaz03h8c9I8v4k/7DBfN/XWrumqs5K8uokp2+04CNhu1xVpyX5tSTf0lr7+1VmeXJVPb61dmPP8uZxO1BVleFE7Z85iLv9dGvtLVX1sCQ3VtXvttZunXZtrbWLk1w87eXOC/21puX+qiQ/nmG7eUpr7c517vN9SX6htfb7D6Te0Y8n+f0k/7LBfNtbax+tqp9P8p+TvGijBc/TdtNIhOn4oiQfba19Oklaax9trV2Z5ON14N7f/5Bkz7hn4KlJfmb5n6K1dmtr7ZLOx3tEko8n9+5xfkcNoxPeV1XPHqcfU1WXjEnc+8c3H6mqJ1fVn9ewd/ryqvqilQufTBKran9V7RqX81dVdeI4/VFV9daqunr8+fqDX210Oqz91Vr7cIZw4IU1OKqGEQdXj4npf0qSqro4ybFJrq2q76mq76iq99Sw9+vPJnrl3r1a4/X3V9XmFQ97XpJvGBPgn+io8ZNJrk3yZVX1srG291fV+eOLxnIf/0pVXZPkxzao78KqeldV/X1V/fuq+sXx/+myqjq6Z70dCXp7I0mq6iXjOryuqs4bp00m9edV1Y3j/ZY/JE3uAX3SuM25vqr+uKo+f5y+VFWvqqq/rmEvwzf01L5KP6y6LayqLx2f92vHnvjKztXz0CQPy33b5h8e18t147byc8fp3z326nVVdeU4bc31OFH/5N61c6vqjePfdEtVvWhivu8f1817axiNdFRn/TOnvzZcP3+Y5IoMgeuqr+dV9W0Z3sD+aFXtHed72zjPDTV8yF+ueXJv5mftpRzX5dYkf1D9e+euzLBd3jz+fX9TE6Mnxz5+Vw2vHzduVF/dN9rtz6rqKRM9v2FIMWtV9Y1JfjvJt7fW/tcas/1Skp2r3PeY8X/8r2t4zVp+bze5HXhUVf3puH7eUMPr1wnjIo6qqt8eb7tixXP3/PH5fH9VPWVc1heMz8P14//FV43TV339Hn9uqqrfzRAyPXb8/3v/+H+54ev46GHj70+Oy1/r9fxFE//PF623jlasxzOq6jfGyxdU1a9V1V+OPfTcifl+uu7bzryis/aZ0l8ba4PXJvnHJN86PsYzqurd43bpzTV8jnpBhvfQP1tVf1Brf7Y6YBRGVb24qs5dsW5flOSLk+ytcRvcYXm7+ZSxtr8d+/QrxmWeUVUXV9U7k7xjg/r+blxXHxj/lqfXMELsfy4/H1PTWvPzAH8yfJB6b5IPJHl9kn83Tn9xhr0ISfK1Sa4ZL5+e5I8P8jHuGR/j75LckeTJ4/RNSR4xXj4hyc1JKslzkvz2xP2PT3J0kr9M8qhx2vckeeN4+YIkzx0vLyXZOl5uSb5jvPyLGT6YJsPeolPHy1+SZN+sn4cH689h6q/9q0z7RJITM7ypX37ePydDSn3Syvsl+fzk3q+NfUGSXxovn5vkxRPzvT/J5sn7J9mWYY/eejXeO0+SRya5Lcm/zTASYnme35vo16Ukr++s76rx/+OJGZLjbx1v++Mk3znrHphx/x10b2R4sf7LJJ873vYF4+8Lkjx3fP5umng+Pm9lryS5fqLXX5nkVyae1+Xn7tuS/Nk6tV+QA7drrx8vr7ctfEeGETzJEMa9c53ln5vkQxn+Pz+e5L9N3PbIics/l+Ts8fL7kjx6xd+91nrcnOT9q/T/uWP9n5Nhu/+x8W/akuRPkhw9zvf6JNPrIYYAAAtgSURBVD8w6x7SXw+ov168YtqPJ/nNDR7jgPtNrJ+HZ9j+PnLluh/X2wWrrKeljO8H1qnz3nmS/HSSP0zyuUkeNk47Ofe9Pm3L8IHxpI76Wg7cFl+R+7bT7511726wTu5K8k9JvmqdeW7L0Of7knzZiufg55N8/3L/Znj9PyYHbgd+I8lLx8vPHNfXCRm2G3cnedJ425smlrWU8b1hkm/MfduXX0/y8vHyNy2v31V66f3j8jcn+UySrx2nPznJn07M93kb/N/cmmG7uT/Jz6/shfHy5Ov5PyT5nBX/zz3r6IwkvzHxuG/OsAP18UluHqc/I8n5Gd47PyTJ25N846x7SH89oP567oppv5LkJWP9VyY5Zpz+kiQvW3m/rP3ZavPy3zTe9uIk565y/9uSnLDBc3jvPOO6flWGncSbxmlPT/LWiT6+PfdtK9er7+4kT8jQy9cmeeN427OTvG2afehwhilore2vqicn+YYk25P8YQ3HYP1hkr+sqp/KgUPN74/JIZ9PS/K7VXVKhsb4+TGR/EySR2fYaLwvyS9V1asybBDeNc5/SpI/HcPdo5L87w0e984MG9RkaMZvHi8/Pcnjx+UkySOq6tjW2v4wVYepv9bzjCRfNZHaH5/hTeGtK+Z7zFjbF2XYK7vy9mn4hqr62wy9fl5r7Yaqek5V/T8Z3rR+QZIbMnyQSoZ11FPf/2it3VVV78vwf3HZOP19GTbKrG6t3nh6kt9prf1LkrTW/mnF/e5I8q9Jdo97Pd4+eWNVHZ/hTcKfj5MuzPDmb9kfjb+vzcE9P8v98BVZZVtYVccm+bokb57Ytn3OBstcPpzh6CRvqarntdYuSnJKVf1chjdoxya5fJz/L5JcUFVvmvg71lqPH1jncS9pw+ikT1fVhzNs90/L8Gbr6rH+hyf58Ab1zzP99dmW77jqY6xxnxdV1XeNlx+bYR1+7CAfdyN/UFWfyvDG+OwMH/Z/o6qelGEnyJdPzPvX7cCh62vVd2cO3BZ/emI7vXnK9U/bXRlCnjOT/Ng6892T4fCPlyb5HxPTn5Hk9Im9tA/LsMNm0qlJvitJWmuXVdXHJ267tbX23vHyyj7eM97nyqp6RFV93ris54zT31lVj6yqR2zwN/59a+2vxsu3JPk3VfXrSS7JEPisZ3m4+bEZ9qx+XWvtL5NsX+P1/PoMPfa2JG8bl9GzjlZ6WxtGaN5Y42jEcTnPSPK34/VjM/TglRssa5b018FZ3m5+bYYA6S/G7eZDk7x7jflX+2w1bXur6p4M/f0zGV7jLqyqkzOENpMjYf904rVuvfpubcNhd6mqG5K8o7XWDsV2U4gwJW04JnwpydL4RP1ga+2Cqro1wzGyz0nytHH2G5I8saqOavfjWPLW2rtrGFL0qAx7Sh6VYWTCXVV1W4b0/wNV9TXj7T9XVe/IkOTf0Fp72lrLXsVdbYy6MmyMlnvmIRkSwn892Po5eIezv5Kkqv5Nhuf7wxk2Vme31i5f/1759SS/3Fq7uKq2ZUiYkyEVnTx06mG5/97VWvv2iToflmFv69bW2gfHYWWTy/9kR31JsnyoyGeqarLnPxPbyQP09EYN521ZU2vt7nFY3WkZ9o68MMPeiV6fHn9PbpN6LPdDZZVt4fim5hPLge3BGLe/l2XY+3JRhr0S39lau66qzsiwhyettR+p4TCkZ2U4FOjJWXs9bl7nIT89cXl5PVSSC1trL139LvNPf23oqzOMyFj1MVYat3VPT/K01tq/VNVS7ttGtolZH8h2ORnPiTDxuOcm+T8ZRg08JEOos+yTE/OtV9/KbfHkdnret8ufyTA8+h1V9Z8z7GW8drzt4nbg+Tt+L8OHvMmT1VWS57TWbppc6MQH342s3D5MDjdvK+ZdeX3Seq/f9z6PrbWPV9UTk3xLkh/J8Lf/x42KHHeSLCU5tar+Jmu/nj8rw7b1O5LsrKon5P6to8n1UhO/f6G19l83qneO6K+O/prw1RlGgVWGD+M7Npj/+7LKZ6sN6r0/trfWPrp8pap+Jcne1tp3ja//SxPzTr6fXau+5MDn5jMT16f+ftY5Eaagqr5iTI2WPSnJ8glO9iR5bZJbWmu3J0kbjl26Jskrqu493mtzVT2r8/G+MsNeh49lSK0+PDbR9iSPG+f54iT/0oYThLw6yddkGN75qBpGMqSqjq6qf3s//+wrMuxtWK7pgbwpYh0z6K9HJfmtDEMAW4Y9qD867mlNVX15DSc2XOn4DEO7k+QHJ6bflqH/MgZbJ61y339OclxPfSssbzQ/Ou7ReO46865VH50Oojf+NMkP1X3nAfiCFcs5NsnxrbVLk/xEhg8a92qt3ZHhnB/Lx6M/P8mfZ3pW3Ra21v6/JLdW1XeP02t847Kh8X/t65MsH5t6XIa9z0dneMFfnu9LW2vvGd/kfSTDntfe/7GNvCPJc2s4QeHycaiPux/LmQn9tb6qek6GPYh71nqMVe52fJKPjx/QvzLDnrhl/6eqttRwUtDvWuW+yf3fNh+f5H+Pe32fn+E9y1rzrVXfQhtHyjwrw///Ga21J40/L1sx310ZXscnj/O+PMnZE6/hX73KQ/xFhg9TqapnZDhkr8fyObJOTXLH+P/wrrHO5WDno2O/3paNX79Tw46th7TW3pphj+rX9BQyhkFPzbDdXPX1fOzPx7bW9mYYfn587hvdtdE66nF5kv84Pmaq6tHL29B5pr82Nm5jX5Th3GKXJfmrJF9fVV823n5MVX35Kndd9bNVhmD0C2sYSfE5GU56vpoHst1cfp96xgbzrVbfYTXvSe6iODbJr9cwZOfuDMemLJ8c6M0Zzpx69or7vCDDCU9urmEI4EczHEu4lodX1fLQocqwJ/qeqvqDJH9Sw97pazKcMyEZjod5dVV9JsOwpx9trd1Zw7DQX6thOOemDMcJ3XA//uYXJXldVV0/LufKDOkg03c4++vo8TF+L8kvj7e9IcMQqL8ZX3A+kuHs3yudm2GY7seTvDP3vRi8NckP1DCs6j1ZfYj29UnuqarrMhyz99p1ar1Xa+0TVfXbGRL2f0xy9Tqzr1Uf6zvo3hiHPj4pyTVVdWeSSzOceXjZcUn+ew0jSSrJT67yuD+Y5LfGD4q3JPmhaf1BG2wLvy/Jb1bVz4x/80VJrltncT9RVd8/znt9hj1pSfJfMvT7R8bfy28oXl1DKFgZPvRfN95vczb+H9vo77pxrPuK8Y33XUn+79wXOs4j/dXXX8dk2M59Uxu/maHz9fyyJD9Sw9f/3pThTfSyczIc6vGRDO8fjl3l8S/IsJ4+lWG0wKf61kJen+StNXyl5mU5cC9ab30Lr7X2T1X1zCRXVtVH2vBtAavZneHD0bKfzfB8Xj/+L9+az/7A8ooMJ1N+foYh2f+Y4cPLas/jpH+t4bDAo3Pf3txzk7xxfE/3L7kvaO95/U6G4dS/U/d9PetGo6FePf4PPDTDdvCPxiHXq72eH5Xk98c+ryS/Nr7296yjDbXWrqiqLUnePX6m3p/k+7MAh4LprzW9uqr+S4bDYv4qwx7/O5N8pIaRgXvGECAZ1svKx131s9X4of2VSf46w4f9v8vqzk9yWVX9Q2tt+wa1TvrFDIcz/EyGwzbWstZnv8Nq+aRDAADAAhg/BN0zHr7ztCS/+QAPlYF76S82YiQCAAAsli9J8qZx7+ydSX54xvXw4KK/WJeRCHOkqh6ZYVjXSqe11qZ9JmWOMIvSXzWcNO1VKybf2lpb65hdOKSqameS714x+c2ttV2zqIcHl0Xpr6r643z2YWAv6TjpLkegqnpdhvPETPrV1trvzKIeHlwWpb+q6j357G/heX4bv0FhkQkRAAAAgC6+nQEAAADoIkQAAAAAuggRAAAAgC5CBAAAAKDL/w/qh31Bknq1IAAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<Figure size 1296x504 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Ux0lUyGk7iR_"
      },
      "source": [
        "Again these plots show that the baseline optimised parameters are much better than the default parameters. As opposed to the box plots in the previous experiments, there is a more apparent positional difference between the boxes on these plots containing the middle 50% of the data.\n",
        "\n",
        "Again, the Decision Tree model comes out trumps as the most accurate model."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "XrqMAs3Es6Mq"
      },
      "source": [
        "---\n",
        "---\n",
        "\n",
        "# 5. Binary Classification of Asleep Vs Awake\n",
        "\n",
        "---\n",
        "---"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ZRWJnoFpQgij"
      },
      "source": [
        "One thing we also wanted to do was to see how good our classifier was at classifying the subjects sessions if it just focused on being awake and asleep.\n",
        "\n",
        "Throught the whole project, we had focused on classifying the sessions using all of the psg_status values, whereas here, we merge all of the different sleep states into one to create a binary classification task.\n",
        "\n",
        "We knew that because there were less classes, our classifiers would get much better accuracy so we wanted to test this."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xYljcuwrs6Mq"
      },
      "source": [
        "### Get the dataset and y for the models with the default parameters"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 66,
          "referenced_widgets": [
            "24e0ce58aadd4d4ab4f329a0fdb8214b",
            "634acdb702bb4a3a9877f418d729f1aa",
            "4841652ff03d427195f8e077db32d84c",
            "55dd302f9bc9422580ba41f06bf1f4ad",
            "0fbcb853616a46c3b3a92a5567718da7",
            "f35d59edfa3f4b2281a495a6303350e6",
            "b138eddb4933452b9d30409b9a0dd2d6",
            "53af98e32a3b4b7983eea69476ab6f1c"
          ]
        },
        "id": "YRSmaSvKs6Mq",
        "outputId": "2698ca01-e747-441d-972f-f0b5b018e300"
      },
      "source": [
        "binary_psg_status_dataset_and_y = create_map_of_subject_id_to_the_dataset_and_y_for_only_awake_and_asleep_psg_status(subject_id_to_filtered_extracted_feature_df_map, map_subject_id_to_a_map_of_the_psg_status_to_session_id, map_subject_id_to_a_map_of_the_session_id_to_psg_status)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "24e0ce58aadd4d4ab4f329a0fdb8214b",
              "version_minor": 0,
              "version_major": 2
            },
            "text/plain": [
              "HBox(children=(FloatProgress(value=0.0, max=31.0), HTML(value='')))"
            ]
          },
          "metadata": {
            "tags": []
          }
        },
        {
          "output_type": "stream",
          "text": [
            "\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "iOtkFd6fs6Mq"
      },
      "source": [
        "### Classify the dataset and y for the models with the default parameters"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 66,
          "referenced_widgets": [
            "06b148547e184ab4b2bcc60b390f813e",
            "a0a44a3fd68149e3bbe2c07723cf1a07",
            "6c77d0a54d01406981554bc8fc06e0db",
            "b7ffaa34300c4536816e9f9f744e62b6",
            "47cd8f675cca49aba9f16d33d9411fcc",
            "8811cd697a1a42019d0e77677db5694a",
            "54b3eaf42f964cd192ca35b26382825a",
            "c0f822e4081c4404906e6aad5d614e6d"
          ]
        },
        "id": "GiQLaYbAs6Mq",
        "outputId": "ff4c84f1-cca2-485b-fe12-e0d058d400d6"
      },
      "source": [
        "optimise_params_bool = True\n",
        "classification_report_bool = True\n",
        "binary_results, binary_classification_report_dict = classify(binary_psg_status_dataset_and_y, names_of_models_to_test, cv_settings, optimise_params_bool, classification_report_bool)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "06b148547e184ab4b2bcc60b390f813e",
              "version_minor": 0,
              "version_major": 2
            },
            "text/plain": [
              "HBox(children=(FloatProgress(value=0.0, max=31.0), HTML(value='')))"
            ]
          },
          "metadata": {
            "tags": []
          }
        },
        {
          "output_type": "stream",
          "text": [
            "\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "TxmjRKg9s6Mr"
      },
      "source": [
        "### Analyse the results"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2ZLIvAwHzGoc"
      },
      "source": [
        "##### Output the results of the score each classifier got for each subject"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 917
        },
        "id": "FSbKKJPgs6Mr",
        "outputId": "780292b6-7253-4f6f-9669-88a8133fa806"
      },
      "source": [
        "binary_results"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>subject_id</th>\n",
              "      <th>SVC</th>\n",
              "      <th>Decision Tree</th>\n",
              "      <th>K-Neighbours</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>46343</td>\n",
              "      <td>0.846995</td>\n",
              "      <td>0.819672</td>\n",
              "      <td>0.841530</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>759667</td>\n",
              "      <td>0.942675</td>\n",
              "      <td>0.961783</td>\n",
              "      <td>0.942675</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>781756</td>\n",
              "      <td>0.925926</td>\n",
              "      <td>0.882716</td>\n",
              "      <td>0.932099</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>844359</td>\n",
              "      <td>0.905724</td>\n",
              "      <td>0.912458</td>\n",
              "      <td>0.919192</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>1066528</td>\n",
              "      <td>0.843450</td>\n",
              "      <td>0.840256</td>\n",
              "      <td>0.821086</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>5</th>\n",
              "      <td>1360686</td>\n",
              "      <td>0.915033</td>\n",
              "      <td>0.905229</td>\n",
              "      <td>0.911765</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>6</th>\n",
              "      <td>1449548</td>\n",
              "      <td>0.882540</td>\n",
              "      <td>0.892063</td>\n",
              "      <td>0.888889</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>7</th>\n",
              "      <td>1455390</td>\n",
              "      <td>0.901899</td>\n",
              "      <td>0.857595</td>\n",
              "      <td>0.901899</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>8</th>\n",
              "      <td>1818471</td>\n",
              "      <td>0.990536</td>\n",
              "      <td>0.977918</td>\n",
              "      <td>0.990536</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>9</th>\n",
              "      <td>2598705</td>\n",
              "      <td>0.987302</td>\n",
              "      <td>0.958730</td>\n",
              "      <td>0.987302</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>10</th>\n",
              "      <td>2638030</td>\n",
              "      <td>0.894569</td>\n",
              "      <td>0.888179</td>\n",
              "      <td>0.888179</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>11</th>\n",
              "      <td>3509524</td>\n",
              "      <td>0.855072</td>\n",
              "      <td>0.876812</td>\n",
              "      <td>0.876812</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>12</th>\n",
              "      <td>3997827</td>\n",
              "      <td>0.981073</td>\n",
              "      <td>0.958991</td>\n",
              "      <td>0.974763</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>13</th>\n",
              "      <td>4018081</td>\n",
              "      <td>0.763636</td>\n",
              "      <td>0.830303</td>\n",
              "      <td>0.818182</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>14</th>\n",
              "      <td>4314139</td>\n",
              "      <td>0.908805</td>\n",
              "      <td>0.858491</td>\n",
              "      <td>0.908805</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>15</th>\n",
              "      <td>4426783</td>\n",
              "      <td>0.935185</td>\n",
              "      <td>0.922840</td>\n",
              "      <td>0.935185</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>16</th>\n",
              "      <td>5132496</td>\n",
              "      <td>0.915584</td>\n",
              "      <td>0.902597</td>\n",
              "      <td>0.935065</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>17</th>\n",
              "      <td>5498603</td>\n",
              "      <td>0.833333</td>\n",
              "      <td>0.861789</td>\n",
              "      <td>0.841463</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>18</th>\n",
              "      <td>5797046</td>\n",
              "      <td>0.896774</td>\n",
              "      <td>0.893548</td>\n",
              "      <td>0.906452</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>19</th>\n",
              "      <td>6220552</td>\n",
              "      <td>0.939683</td>\n",
              "      <td>0.904762</td>\n",
              "      <td>0.936508</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>20</th>\n",
              "      <td>7749105</td>\n",
              "      <td>0.658537</td>\n",
              "      <td>0.829268</td>\n",
              "      <td>0.926829</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>21</th>\n",
              "      <td>8000685</td>\n",
              "      <td>0.965300</td>\n",
              "      <td>0.949527</td>\n",
              "      <td>0.962145</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>22</th>\n",
              "      <td>8173033</td>\n",
              "      <td>0.971519</td>\n",
              "      <td>0.949367</td>\n",
              "      <td>0.962025</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>23</th>\n",
              "      <td>8530312</td>\n",
              "      <td>0.926752</td>\n",
              "      <td>0.847134</td>\n",
              "      <td>0.933121</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>24</th>\n",
              "      <td>8686948</td>\n",
              "      <td>0.955696</td>\n",
              "      <td>0.968354</td>\n",
              "      <td>0.952532</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>25</th>\n",
              "      <td>8692923</td>\n",
              "      <td>0.902913</td>\n",
              "      <td>0.860841</td>\n",
              "      <td>0.902913</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>26</th>\n",
              "      <td>9106476</td>\n",
              "      <td>0.905363</td>\n",
              "      <td>0.880126</td>\n",
              "      <td>0.905363</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>27</th>\n",
              "      <td>9961348</td>\n",
              "      <td>0.899160</td>\n",
              "      <td>0.857143</td>\n",
              "      <td>0.903361</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "   subject_id       SVC  Decision Tree  K-Neighbours\n",
              "0       46343  0.846995       0.819672      0.841530\n",
              "1      759667  0.942675       0.961783      0.942675\n",
              "2      781756  0.925926       0.882716      0.932099\n",
              "3      844359  0.905724       0.912458      0.919192\n",
              "4     1066528  0.843450       0.840256      0.821086\n",
              "5     1360686  0.915033       0.905229      0.911765\n",
              "6     1449548  0.882540       0.892063      0.888889\n",
              "7     1455390  0.901899       0.857595      0.901899\n",
              "8     1818471  0.990536       0.977918      0.990536\n",
              "9     2598705  0.987302       0.958730      0.987302\n",
              "10    2638030  0.894569       0.888179      0.888179\n",
              "11    3509524  0.855072       0.876812      0.876812\n",
              "12    3997827  0.981073       0.958991      0.974763\n",
              "13    4018081  0.763636       0.830303      0.818182\n",
              "14    4314139  0.908805       0.858491      0.908805\n",
              "15    4426783  0.935185       0.922840      0.935185\n",
              "16    5132496  0.915584       0.902597      0.935065\n",
              "17    5498603  0.833333       0.861789      0.841463\n",
              "18    5797046  0.896774       0.893548      0.906452\n",
              "19    6220552  0.939683       0.904762      0.936508\n",
              "20    7749105  0.658537       0.829268      0.926829\n",
              "21    8000685  0.965300       0.949527      0.962145\n",
              "22    8173033  0.971519       0.949367      0.962025\n",
              "23    8530312  0.926752       0.847134      0.933121\n",
              "24    8686948  0.955696       0.968354      0.952532\n",
              "25    8692923  0.902913       0.860841      0.902913\n",
              "26    9106476  0.905363       0.880126      0.905363\n",
              "27    9961348  0.899160       0.857143      0.903361"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 127
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "n-rua4uDQb84"
      },
      "source": [
        "From this output table, we can already see that the accuracy scores are much better. It will be interesting to roll up on this information to find out a higher level summary of it."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WCHVwyX1zI7Q"
      },
      "source": [
        "##### Describe this dataframe better with summary statistics"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 297
        },
        "id": "w_EPJDuGs6Mr",
        "outputId": "e92de0b7-3fc6-4e34-c219-11b07c1db1f3"
      },
      "source": [
        "binary_results.describe()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>SVC</th>\n",
              "      <th>Decision Tree</th>\n",
              "      <th>K-Neighbours</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>count</th>\n",
              "      <td>28.000000</td>\n",
              "      <td>28.000000</td>\n",
              "      <td>28.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>mean</th>\n",
              "      <td>0.901823</td>\n",
              "      <td>0.894589</td>\n",
              "      <td>0.914524</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>std</th>\n",
              "      <td>0.069691</td>\n",
              "      <td>0.046666</td>\n",
              "      <td>0.045414</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>min</th>\n",
              "      <td>0.658537</td>\n",
              "      <td>0.819672</td>\n",
              "      <td>0.818182</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>25%</th>\n",
              "      <td>0.891561</td>\n",
              "      <td>0.858267</td>\n",
              "      <td>0.898646</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>50%</th>\n",
              "      <td>0.907264</td>\n",
              "      <td>0.890121</td>\n",
              "      <td>0.915478</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>75%</th>\n",
              "      <td>0.940431</td>\n",
              "      <td>0.929471</td>\n",
              "      <td>0.938050</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>max</th>\n",
              "      <td>0.990536</td>\n",
              "      <td>0.977918</td>\n",
              "      <td>0.990536</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "             SVC  Decision Tree  K-Neighbours\n",
              "count  28.000000      28.000000     28.000000\n",
              "mean    0.901823       0.894589      0.914524\n",
              "std     0.069691       0.046666      0.045414\n",
              "min     0.658537       0.819672      0.818182\n",
              "25%     0.891561       0.858267      0.898646\n",
              "50%     0.907264       0.890121      0.915478\n",
              "75%     0.940431       0.929471      0.938050\n",
              "max     0.990536       0.977918      0.990536"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 128
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "kqafkzmQStWM"
      },
      "source": [
        "One thing that sticks out immediately is the fact that, unlike when we were classifying with multiple psg_status values, the Decision Tree classifier is actually the worst of the few classifiers that we used.\n",
        "We can see that in comparison to the Support Vector Classifier (SVC) and the K-Neighbours Classifier, the Decision Tree classifier had the worst mean, minimum and maximum accuracy across the subjects.\n",
        "\n",
        "One other thing that also strook me was that for some subjects, we were able to get an accuracy score of >99%. This is a massive victory for both svc and the k-neighbours classifiers.\n",
        "\n",
        "While out of the two other classifiers, when we discount the Decision Tree, the results are similar enough, the K-Neighbours classsifier does pull out just ahead. We can talk further on this point after we plot this"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "VBHokkLszLDY"
      },
      "source": [
        "##### Plot these summary statistics to better visualise them"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 282
        },
        "id": "878WKuX3s6Mr",
        "outputId": "0694c582-e928-499b-f395-309551b13fb2"
      },
      "source": [
        "binary_results.boxplot()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<matplotlib.axes._subplots.AxesSubplot at 0x7fe6f1515fd0>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 129
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD4CAYAAADiry33AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAaQklEQVR4nO3df3Rc5X3n8fenMoYUAjEx1cnaYJuu08iIQIJqSnFbuQ7G+XHipKRdCzaBrDbe7NY+XTbpCVQcIE504mySTQMhECdygJxTORS2rE/w2tCiCXEDW5vy09aaGgPFTvaExEBqoBiJ7/5xr5LrQfaMPSON55nP65w5uvP8uPPceaSvnnmeO/cqIjAzs3T9WqMbYGZmE8uB3swscQ70ZmaJc6A3M0ucA72ZWeKmNLoB5aZPnx6zZ89udDMmzEsvvcTxxx/f6GbYEXL/Na/U++7BBx/8WUScMl5exUAvaS3wAeCnEdE5Tr6ArwHvA14GLouIf8zzLgWuyot+PiJuqfR6s2fPZuvWrZWKNa1SqUR3d3ejm2FHyP3XvFLvO0nPHCyvmqmbm4Elh8h/LzA3fywHbsxf9GTgGuBcYD5wjaRp1TXZzMzqpWKgj4j7gL2HKLIUuDUyDwBvkfQ24ELgnojYGxHPA/dw6H8YZmY2AeqxGDsDeLbwfHeedrB0MzObREfFYqyk5WTTPrS3t1MqlRrboAm0b9++pI8vde6/5tXKfVePQL8HOLXwfGaetgfoLksvjbeDiFgDrAHo6uqKlBdMUl8QSp37r3m1ct/VY+pmPfAxZX4HeDEifgJsAhZLmpYvwi7O08zMbBJVc3rlINnIfLqk3WRn0hwDEBE3ARvITq3cSXZ65cfzvL2SPgdsyXe1KiIOtahrZmYToGKgj4ieCvkB/OlB8tYCa4+saWZmVg9HxWJsSrLvj9XO9wkwOzz+2zs4X+umziLikI9Zn/l+xTIp/qKZTTT/7R2cA72ZWeIc6M3MEudAb2aWOAd6M7PEOdCbmSXOgd7MLHE+j94s5/OwLVUe0ZvlqjnHuppzsc2ONg70ZmaJc6A3M0ucA72ZWeIc6M3MEudAb2aWOAd6M7PEOdCbmSXOgd7MLHEO9GZmiXOgNzNLnAO9mVniqgr0kpZI2iFpp6QrxsmfJenvJD0qqSRpZiFvVNLD+WN9PRtvZmaVVbx6paQ24AbgAmA3sEXS+ojYXij2ZeDWiLhF0h8CXwA+mue9EhFn17ndZmZWpWpG9POBnRGxKyL2A+uApWVl5gH35ttD4+SbmVmDVHM9+hnAs4Xnu4Fzy8o8AvwR8DXgw8CbJb01In4OHCdpKzACrI6IO8tfQNJyYDlAe3s7pVLpcI+jqaR+fKlz/zWvVu27et145NPA1yVdBtwH7AFG87xZEbFH0unAvZIei4gni5UjYg2wBqCrqyu6u7vr1Kyj0Ma7SPr4Uuf+a14t3HfVBPo9wKmF5zPztF+KiB+TjeiRdAJwUUS8kOftyX/uklQC3gUcEOjNzGziVDNHvwWYK2mOpKnAMuCAs2ckTZc0tq8rgbV5+jRJx46VAc4Hiou4ZmY2wSoG+ogYAVYAm4Bh4LaI2CZplaQP5sW6gR2SngDagf48vQPYKukRskXa1WVn65iZ2QSrao4+IjYAG8rSri5s3w7cPk69HwFn1thGMzOrQb0WY1vGWZ+9mxdfea2mfcy+4q6a6p/0pmN45JrFNe3DzFqHA/1hevGV13h69fuPuH6pVKp55b/WfxRmzciDrCPnQG9mTcGDrCPni5qZmSXOgd7MLHEO9GZmiXOgNzNLnBdjrWXU46wNaN0zN6x5OdBby6j1rA1o7TM3rHl56sbMLHEO9GZmiXOgNzNLnAO9mVniHOjNzBLnQG9mljifXmlmTeHNHVdw5i1X1LaTW2ptA0Btp+g2ggO9mTWFfxle7atXHiFP3ZiZJc6B3swscQ70ZmaJ8xz9YfKCkJk1m6oCvaQlwNeANuDbEbG6LH8WsBY4BdgL/PuI2J3nXQpclRf9fETUGOYaywtCZtZsKk7dSGoDbgDeC8wDeiTNKyv2ZeDWiHgnsAr4Ql73ZOAa4FxgPnCNpGn1a76ZmVVSzRz9fGBnROyKiP3AOmBpWZl5wL359lAh/0LgnojYGxHPA/cAS2pvtpmZVauaqZsZwLOF57vJRuhFjwB/RDa982HgzZLeepC6M8pfQNJyYDlAe3s7pVKpyuY3Ri3t27dvX12O72h/j45Wtb5v7r/G8t/ekanXYuynga9Lugy4D9gDjFZbOSLWAGsAurq6otY57Am18a6a5tjrMUdfaxtaVh3eN/dfA228i8s2vlTDDgTUUj+7O1gz9l01gX4PcGrh+cw87Zci4sdkI3oknQBcFBEvSNoDdJfVLdXQXjNrUbXeHWz2FXfVvI9mVc0c/RZgrqQ5kqYCy4D1xQKSpksa29eVZGfgAGwCFkuali/CLs7TzMxsklQM9BExAqwgC9DDwG0RsU3SKkkfzIt1AzskPQG0A/153b3A58j+WWwBVuVpZmY2Saqao4+IDcCGsrSrC9u3A7cfpO5afjXCT0LN57FvrK3+SW86prbXN7OW4m/GHibPEzavunyrGfzNZms6DvTWMmr9VjP4m83WnHxRMzOzxDnQm5klzoHezCxxDvRmZolzoDczS5wDvZlZ4hzozcwS50BvZpY4B3ozs8Q50JuZJc6B3swscQ70ZmaJc6A3M0ucA72ZWeIc6M3MEudAb2aWOAd6M7PE+Q5T1lLqcncn3/PXmowDvbWMetyr1/f8tWZU1dSNpCWSdkjaKekNd1eWdJqkIUkPSXpU0vvy9NmSXpH0cP64qd4HYGZmh1ZxRC+pDbgBuADYDWyRtD4itheKXQXcFhE3SpoHbABm53lPRsTZ9W22mZlVq5oR/XxgZ0Tsioj9wDpgaVmZAE7Mt08Cfly/JpqZWS2qmaOfATxbeL4bOLeszLXA3ZJWAscD7ynkzZH0EPAL4KqI+GH5C0haDiwHaG9vp1QqVdv+ppT68aXO/de8WrXv6rUY2wPcHBFfkXQe8F1JncBPgNMi4ueSzgHulHRGRPyiWDki1gBrALq6uqK7u7tOzToKbbyLpI8vde6/5tXCfVfN1M0e4NTC85l5WlEvcBtARNwPHAdMj4hXI+LnefqDwJPA22tttJmZVa+aQL8FmCtpjqSpwDJgfVmZfwYWAUjqIAv0z0k6JV/MRdLpwFxgV70ab2ZmlVWcuomIEUkrgE1AG7A2IrZJWgVsjYj1wKeAb0m6nGxh9rKICEm/D6yS9BrwOvDJiNg7YUdjZmZvUNUcfURsIDtlsph2dWF7O3D+OPXuAO6osY1mZhVJqlzmi5X3ExF1aM3Rxde6MbMkRMQhH0NDQxXLpBjkwYHezCx5DvRmZolzoDczS5yvXllnXhAys6ONR/R15gUhMzvaONCbmSXOgd7MLHEO9GaWtMHBQTo7O1m0aBGdnZ0MDg42ukmTzouxZpaswcFB+vr6GBgYYHR0lLa2Nnp7ewHo6elpcOsmjwO9mSWrv7+fiy++mJUrVzI8PExHRwcXX3wx/f39DvRmZinYvn07L7/88htG9E8//XSjmzapPEdvlpNU8fHMFz9QsYwdPaZOncqKFStYuHAhU6ZMYeHChaxYsYKpU6c2ummTyoHeLFfN9xuq+R6EHT3279/P9ddfz9DQECMjIwwNDXH99dezf//+RjdtUnnqxsySNW/ePD70oQ8dMEd/ySWXcOeddza6aZPKgd7MktXX1zfuWTf9/f2NbtqkcqA3s2SNnVlTHNG32hk34EBvZonr6emhp6eHUqlEd3d3o5vTEF6MNTNLnAO9mVniHOjNzBJXVaCXtETSDkk7JV0xTv5pkoYkPSTpUUnvK+RdmdfbIenCejbezMwqq7gYK6kNuAG4ANgNbJG0PiK2F4pdBdwWETdKmgdsAGbn28uAM4B/A/ytpLdHxGi9D8TMzMZXzYh+PrAzInZFxH5gHbC0rEwAJ+bbJwE/zreXAusi4tWIeArYme/PzMwmSTWnV84Ani083w2cW1bmWuBuSSuB44H3FOo+UFZ3RvkLSFoOLAdob2+nVCpV0azmtG/fvqSPL3Xuv+bVyn1Xr/Poe4CbI+Irks4Dviups9rKEbEGWAPQ1dUVKZ/r2srn8qbA/de8Wrnvqgn0e4BTC89n5mlFvcASgIi4X9JxwPQq65qZ2QSqZo5+CzBX0hxJU8kWV9eXlflnYBGApA7gOOC5vNwyScdKmgPMBf6hXo03M7PKKo7oI2JE0gpgE9AGrI2IbZJWAVsjYj3wKeBbki4nW5i9LLLrtW6TdBuwHRgB/tRn3JiZTa6q5ugjYgPZKZPFtKsL29uB8w9Stx9orUvFmZkdRfzNWDOzxDnQm5klzoHezCxxDvRmZolzoDczS5wDvZlZ4hzozcwS50BvZpY4B3ozs8Q50JuZJc6B3swscQ70ZmaJc6CfJIODg3R2drJo0SI6OzsZHBxsdJPMrEXU6w5TdgiDg4P09fUxMDDA6OgobW1t9Pb2AtDT09Pg1plZ6jyinwT9/f0MDAywcOFCpkyZwsKFCxkYGKC/31dvNrOJ50A/CYaHh1mwYMEBaQsWLGB4eLhBLTKzVuJAPwk6OjrYvHnzAWmbN2+mo6OjQS0ys1biQD8J+vr66O3tZWhoiJGREYaGhujt7aWvr6/RTTOzFuDF2EkwtuC6cuVKhoeH6ejooL+/3wuxZjYpHOgnSU9PDz09PZRKJbq7uxvdHDNrIVVN3UhaImmHpJ2Srhgn/6uSHs4fT0h6oZA3WshbX8/Gm5lZZRVH9JLagBuAC4DdwBZJ6yNi+1iZiLi8UH4l8K7CLl6JiLPr12QzMzsc1Yzo5wM7I2JXROwH1gFLD1G+B/DXPs3MjhLVzNHPAJ4tPN8NnDteQUmzgDnAvYXk4yRtBUaA1RFx5zj1lgPLAdrb2ymVSlU1vhnt27cv6eNLnfuvebVy39V7MXYZcHtEjBbSZkXEHkmnA/dKeiwinixWiog1wBqArq6uSHmx0ouxzc3917xaue+qmbrZA5xaeD4zTxvPMsqmbSJiT/5zF1DiwPl7MzObYNUE+i3AXElzJE0lC+ZvOHtG0juAacD9hbRpko7Nt6cD5wPby+uamdnEqTh1ExEjklYAm4A2YG1EbJO0CtgaEWNBfxmwLiKiUL0D+Kak18n+qawunq1jZmYTr6o5+ojYAGwoS7u67Pm149T7EXBmDe0zM7Ma+Vo3ZmaJc6A3M0ucA72ZWeIc6M3MEudAb2aWOAd6M7PEOdCbmSXOgd7MLHEO9GZmiXOgNzNLnAO9mVniHOjNzBLnQG9mljgHejOzxDnQm5klzoHezCxxDvRmZolzoDczS5wDvZlZ4hzozcwS50BvZpa4qgK9pCWSdkjaKemKcfK/Kunh/PGEpBcKeZdK+qf8cWk9G29mZpVNqVRAUhtwA3ABsBvYIml9RGwfKxMRlxfKrwTelW+fDFwDdAEBPJjXfb6uR2FmZgdVzYh+PrAzInZFxH5gHbD0EOV7gMF8+0LgnojYmwf3e4AltTTYzMwOT8URPTADeLbwfDdw7ngFJc0C5gD3HqLujHHqLQeWA7S3t1MqlapoVnPat29f0seXOvdf82rlvqsm0B+OZcDtETF6OJUiYg2wBqCrqyu6u7vr3KyjR6lUIuXjS537r3m1ct9VM3WzBzi18HxmnjaeZfxq2uZw65qZ2QSoJtBvAeZKmiNpKlkwX19eSNI7gGnA/YXkTcBiSdMkTQMW52lmZjZJKk7dRMSIpBVkAboNWBsR2yStArZGxFjQXwasi4go1N0r6XNk/ywAVkXE3voegpmZHUpVc/QRsQHYUJZ2ddnzaw9Sdy2w9gjbZ2ZmNfI3Y83MEudAb2aWOAd6M7PEOdCbmSXOgd7MLHEO9GZmiXOgNzNLnAO9mVniHOjNzBLnQG9mljgHejOzxDnQm5klzoHezCxxDvRmZolzoDczS5wDvZlZ4hzoJ8ng4CCdnZ0sWrSIzs5OBgcHK1cyM6uDqu4wZbUZHBykr6+PgYEBRkdHaWtro7e3F4Cenp4Gt87MUucR/STo7+9nYGCAhQsXMmXKFBYuXMjAwAD9/f2NbpqZtQAH+kkwPDzMggULDkhbsGABw8PDDWqRmbUSB/pJ0NHRwebNmw9I27x5Mx0dHQ1qkZm1kqoCvaQlknZI2inpioOU+RNJ2yVtk/RXhfRRSQ/nj/X1angz6evro7e3l6GhIUZGRhgaGqK3t5e+vr5GN83MWkDFxVhJbcANwAXAbmCLpPURsb1QZi5wJXB+RDwv6TcKu3glIs6uc7ubytiC68qVKxkeHqajo4P+/n4vxJrZpKjmrJv5wM6I2AUgaR2wFNheKPMJ4IaIeB4gIn5a74Y2u56eHnp6eiiVSnR3dze6OWbWQqoJ9DOAZwvPdwPnlpV5O4CkvwfagGsjYmOed5ykrcAIsDoi7ix/AUnLgeUA7e3tlEqlwzmGprJv376kjy917r/m1cp9V6/z6KcAc4FuYCZwn6QzI+IFYFZE7JF0OnCvpMci4sli5YhYA6wB6OrqipRHvB7RNzf3X/Nq5b6rZjF2D3Bq4fnMPK1oN7A+Il6LiKeAJ8gCPxGxJ/+5CygB76qxzWZmdhiqCfRbgLmS5kiaCiwDys+euZNsNI+k6WRTObskTZN0bCH9fA6c2zczswlWceomIkYkrQA2kc2/r42IbZJWAVsjYn2et1jSdmAU+POI+Lmk3wW+Kel1sn8qq4tn65iZ2cRTRDS6DQeQ9BzwTKPbMYGmAz9rdCPsiLn/mlfqfTcrIk4ZL+OoC/Spk7Q1Iroa3Q47Mu6/5tXKfedLIJiZJc6B3swscQ70k29NoxtgNXH/Na+W7TvP0ZuZJc4jejOzxDnQm5klzoG+ziT15dfkfzS/Bv81kr5QVuZsScP59gmSvinpSUkPSipJKr9onOUK9zfYJukRSZ+SdES/x5JWSXrPIfI/KeljR95akHRm4X4MeyU9lW//bS37bUaS9hW23yfpCUmzyso8LemOwvOPSLq5wn67JF1XocxsSY8fJK8kKenTLn1z8DqSdB7wAeDdEfFqftmHecDNZNfrH7MMGMy3vw08BcyNiNclzcnr2Ph+eX+D/L4HfwWcCFxzuDuKiKsr5N90RC08cB+PAWPtvRn4fkTcXiwjaUpEjNT6Ws1C0iLgOuDCiBjvy5HnSJpX7bfoI2IrsLWebayVJJGtgb7e6LaAR/T19jbgZxHxKkBE/Cwi7gOeLxul/wkwKOk3yS75fNXYL0REPBURd012w5tRft+D5cAKZdokfUnSlvwT1X8aKyvpM5Ieyz8FrM7Tbpb0kXx7dX6HtEclfTlPu1bSp/PtsyU9kOf/jaRpeXpJ0hcl/UM+Qv29atqe1/vL/BLefybpHEk/yD/VbZL0trzcb0ramKf/UNI76vgWTjpJvw98C/hA+VVsC74CvOH2a5KOl7Q2f68fkrQ0T++W9P18+xRJ9+Sf+L4t6Zl8wAXQJulbed7dkt5U2P1H809aj0uan+/rZEl35n3+gKR35um//L3Inz+ef2KYrexOfLcCjwOn5r9jj+e/e5fX9u4dOQf6+rqbrHOfkPQNSX+Qpw+SjeKR9DvA3oj4J+AM4OGIGG1Mc5tfflXUNuA3gF7gxYj4beC3gU8ouxjfe8lulnNuRJwF/PfiPiS9FfgwcEZEvBP4/DgvdSvwmTz/MQ78BDElIuYD/5XD+2QxNf+m5nXA9cBHIuIcYC3Qn5dZA6zM0z8NfOMw9n+0OZbsAogfioj/e4hytwHvlvRvy9L7gHvz93oh8CVJx5eVuSYvcwZwO3BaIW8u2Q2SzgBeAC4q5P16/knxv5C9/wCfBR7K+/wvyH4HKpkLfCN/jenAjIjojIgzge9UUX9CeOqmjiJin6RzgN8j+0X8nrJ77H4P+JGkT3HgtI3V12LgnWOjdOAksj+89wDfiYiXASJib1m9F4F/BQbykeH3i5mSTgLeEhE/yJNuAf66UOR/5j8fBGYfRnu/l//8LaATuCf7xE8b8BNJJwC/C/x1ng5ZsGxWrwE/IvuH/GeHKDcKfIlsuvN/F9IXAx8sjKaP48BADrCA7J82EbFR0vOFvKci4uF8u7yvBvM690k6UdJb8n1dlKffK+mtkk6scIzPRMQD+fYu4HRJ1wN3kQ0EG8KBvs7y0XkJKEl6DLg0Im6W9BTwB2S/OOflxbcBZ0lq86j+yCi7oc0o8FNAZKPfTWVlLjzUPvIrtM4HFgEfAVYAf3gYzXg1/znK4f1NvTTWRGBbRJxXzMyDygsJ3XP5dbJpy7+T9BfAF8kCLmT3syiumXyXLNAXF1AFXBQRO4o7ldRe5eu/WtgeBYpTN+VfKDrUF4xGOHA25LjC9lifkt8/+yzgQuCTZMf+H6psa1156qaOJP2WshuljzmbX12JcxD4KrArInYD5HOUW4HP5os3Y2cHvH8Sm920JJ0C3AR8PbJv/m0C/rOkY/L8t+cf7e8BPi7p1/P0k8v2cwJwUkRsAC4HzirmR8SLZOssY/PvHwV+QP3sAE5RtpiPpGMknRERvwCekvTHebrywNG08k9V7wcuAS6LiLPzx9Vl5V4j+3spzmtvAlYW/lbGu4nR35MFVCQtBqZV2bR/l9dZQDb99yLww7ydSOomW3/7BfA08O48/d3AnPF2mK8N/FpE3AFcNVanETyir68TgOvzj30jwE7ye+GSfdS/DlhZVuc/ki0+7ZT0CtllVP98cprblN4k6WHgGLL3+LvA/8jzvk32cfwf82DwHNl88EZJZwNbJe0HNpDNuY55M/C/JB1HNmr8b+O87qXATfk/i13Ax+t1QBGxP59uui6fJpoC/CXZJ75LgBslXZUf8zrgkXq9diNExF5JS8huOfpcfk+L8QyQBcgxnyN7Xx5VdkrtU2RnuRV9luxEh48C9wP/D/gXsr/NQ/lXSQ+Rvcdjo+5rgbWSHgVeJvsdALgD+JikbcD/Ibuj3nhmAN/Rr07/vfIg5SacL4FgZslQdke70Xw67jzgxoSmvo6YR/RmlpLTgNvyUfR+4BMNbs9RwSN6M7PEeTHWzCxxDvRmZolzoDczS5wDvZlZ4hzozcwS9/8BJmG6I++QQ3sAAAAASUVORK5CYII=\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Okmr_a10UDEn"
      },
      "source": [
        "This plot clearly illustrates the point noted above, that the K-Neighbours classifier is the best clasifier in the case of just looking for a binary, sleep Vs awake, classifier.\n",
        "\n",
        "While it still contains the few outliers to do downside that are seen in every K-Neighbours box plot thus far in this notebook, in this case, these outliers are still much higher than those in the SVC classifier.\n",
        "\n",
        "Using our knowledge of these classifiers, it makes sense for this data that the K-Neighbours classifier is the best when it comes to the binary task of classifying the the subject is awake or asleep. K-Neighbours would take advantage of the massive similarity between the time series when the subject is awake and also the similarity betwee time series when the subject is asleep."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "48F1hBuXVYW7"
      },
      "source": [
        "When we use the knowledge we have on decision tree classifiers, it also makes sense that when it performed better than the other classifiers in the case when there were multiple psg_status that need to be classified.\n",
        "Decision trees involve a complexity that the other two models don't seem to be able to breach. This allows the decision tree to optimise itself to take better advantage of the all the features and more accurately predict the psg_status."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hNdSCnNiNzvQ"
      },
      "source": [
        "---\n",
        "---\n",
        "\n",
        "# 6. Analyse the distribution of the precision & recall values with each psg_status\n",
        "\n",
        "---\n",
        "---"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zOcNaRJgNvvH"
      },
      "source": [
        "Using the data below, we had planned to create a diagram for each model showing average precision and recall values across all the subjects for each model and psg status pair.\n",
        "\n",
        "We wanted to show on average which model and psg status pair gave the best recall, precision and F1 score to see how the different models might have had different effects on these metrics and also to see how the different psg_status' influenced these metrics.\n",
        "\n",
        "Due to the structure of the classification report dictionary returned and the time contraints, we were unable to dig our heals into this problem properly and perform this analysis.\n",
        "\n",
        "The below are the outputs to the classification reports dictionaries with:\n",
        "1.   All the psg_status being classified\n",
        "2.   Just awake and asleep being classified\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "vnm3DTnzPPDf",
        "outputId": "e70be683-4dd3-42ef-c532-86aea82c998b"
      },
      "source": [
        "baseline_classification_report_dict"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{'1066528': {'Decision Tree': {'0': {'f1-score': 0.4158415841584158,\n",
              "    'precision': 0.39622641509433965,\n",
              "    'recall': 0.4375,\n",
              "    'support': 48},\n",
              "   '1': {'f1-score': 0.14035087719298248,\n",
              "    'precision': 0.14285714285714285,\n",
              "    'recall': 0.13793103448275862,\n",
              "    'support': 29},\n",
              "   '2': {'f1-score': 0.574074074074074,\n",
              "    'precision': 0.6019417475728155,\n",
              "    'recall': 0.5486725663716814,\n",
              "    'support': 113},\n",
              "   '3': {'f1-score': 0.3018867924528302,\n",
              "    'precision': 0.3076923076923077,\n",
              "    'recall': 0.2962962962962963,\n",
              "    'support': 27},\n",
              "   '5': {'f1-score': 0.7738693467336684,\n",
              "    'precision': 0.7475728155339806,\n",
              "    'recall': 0.8020833333333334,\n",
              "    'support': 96},\n",
              "   'accuracy': 0.549520766773163,\n",
              "   'macro avg': {'f1-score': 0.44120453492239414,\n",
              "    'precision': 0.43925808575011727,\n",
              "    'recall': 0.44449664609681394,\n",
              "    'support': 313},\n",
              "   'weighted avg': {'f1-score': 0.5474228195885923,\n",
              "    'precision': 0.5471432113165113,\n",
              "    'recall': 0.549520766773163,\n",
              "    'support': 313}},\n",
              "  'K-Neighbours': {'0': {'f1-score': 0.39285714285714285,\n",
              "    'precision': 0.34375,\n",
              "    'recall': 0.4583333333333333,\n",
              "    'support': 48},\n",
              "   '1': {'f1-score': 0.15,\n",
              "    'precision': 0.2727272727272727,\n",
              "    'recall': 0.10344827586206896,\n",
              "    'support': 29},\n",
              "   '2': {'f1-score': 0.5702127659574469,\n",
              "    'precision': 0.5491803278688525,\n",
              "    'recall': 0.5929203539823009,\n",
              "    'support': 113},\n",
              "   '3': {'f1-score': 0.06666666666666667,\n",
              "    'precision': 0.3333333333333333,\n",
              "    'recall': 0.037037037037037035,\n",
              "    'support': 27},\n",
              "   '5': {'f1-score': 0.6698564593301435,\n",
              "    'precision': 0.6194690265486725,\n",
              "    'recall': 0.7291666666666666,\n",
              "    'support': 96},\n",
              "   'accuracy': 0.5207667731629393,\n",
              "   'macro avg': {'f1-score': 0.36991860696227996,\n",
              "    'precision': 0.4236919920956262,\n",
              "    'recall': 0.38418113337628135,\n",
              "    'support': 313},\n",
              "   'weighted avg': {'f1-score': 0.4912057683898662,\n",
              "    'precision': 0.49500157989438914,\n",
              "    'recall': 0.5207667731629393,\n",
              "    'support': 313}},\n",
              "  'SVC': {'0': {'f1-score': 0.19354838709677416,\n",
              "    'precision': 0.42857142857142855,\n",
              "    'recall': 0.125,\n",
              "    'support': 48},\n",
              "   '1': {'f1-score': 0.0, 'precision': 0.0, 'recall': 0.0, 'support': 29},\n",
              "   '2': {'f1-score': 0.6264150943396226,\n",
              "    'precision': 0.5460526315789473,\n",
              "    'recall': 0.7345132743362832,\n",
              "    'support': 113},\n",
              "   '3': {'f1-score': 0.06896551724137931,\n",
              "    'precision': 0.5,\n",
              "    'recall': 0.037037037037037035,\n",
              "    'support': 27},\n",
              "   '5': {'f1-score': 0.6141078838174274,\n",
              "    'precision': 0.5103448275862069,\n",
              "    'recall': 0.7708333333333334,\n",
              "    'support': 96},\n",
              "   'accuracy': 0.5239616613418531,\n",
              "   'macro avg': {'f1-score': 0.3006073764990407,\n",
              "    'precision': 0.3969937775473166,\n",
              "    'recall': 0.3334767289413307,\n",
              "    'support': 313},\n",
              "   'weighted avg': {'f1-score': 0.4501330800415745,\n",
              "    'precision': 0.46251910347643926,\n",
              "    'recall': 0.5239616613418531,\n",
              "    'support': 313}}},\n",
              " '1360686': {'Decision Tree': {'0': {'f1-score': 0.6666666666666667,\n",
              "    'precision': 0.6451612903225806,\n",
              "    'recall': 0.6896551724137931,\n",
              "    'support': 29},\n",
              "   '1': {'f1-score': 0.08695652173913043,\n",
              "    'precision': 0.08333333333333333,\n",
              "    'recall': 0.09090909090909091,\n",
              "    'support': 11},\n",
              "   '2': {'f1-score': 0.7441860465116278,\n",
              "    'precision': 0.7671232876712328,\n",
              "    'recall': 0.7225806451612903,\n",
              "    'support': 155},\n",
              "   '3': {'f1-score': 0.6774193548387097,\n",
              "    'precision': 0.6176470588235294,\n",
              "    'recall': 0.75,\n",
              "    'support': 56},\n",
              "   '5': {'f1-score': 0.5576923076923076,\n",
              "    'precision': 0.5918367346938775,\n",
              "    'recall': 0.5272727272727272,\n",
              "    'support': 55},\n",
              "   'accuracy': 0.6666666666666666,\n",
              "   'macro avg': {'f1-score': 0.5465841794896884,\n",
              "    'precision': 0.5410203409689107,\n",
              "    'recall': 0.5560835271513803,\n",
              "    'support': 306},\n",
              "   'weighted avg': {'f1-score': 0.6674746832542834,\n",
              "    'precision': 0.6721232332592925,\n",
              "    'recall': 0.6666666666666666,\n",
              "    'support': 306}},\n",
              "  'K-Neighbours': {'0': {'f1-score': 0.40816326530612246,\n",
              "    'precision': 0.5,\n",
              "    'recall': 0.3448275862068966,\n",
              "    'support': 29},\n",
              "   '1': {'f1-score': 0.0, 'precision': 0.0, 'recall': 0.0, 'support': 11},\n",
              "   '2': {'f1-score': 0.7345679012345678,\n",
              "    'precision': 0.7041420118343196,\n",
              "    'recall': 0.7677419354838709,\n",
              "    'support': 155},\n",
              "   '3': {'f1-score': 0.6542056074766356,\n",
              "    'precision': 0.6862745098039216,\n",
              "    'recall': 0.625,\n",
              "    'support': 56},\n",
              "   '5': {'f1-score': 0.5289256198347108,\n",
              "    'precision': 0.48484848484848486,\n",
              "    'recall': 0.5818181818181818,\n",
              "    'support': 55},\n",
              "   'accuracy': 0.6405228758169934,\n",
              "   'macro avg': {'f1-score': 0.46517247877040735,\n",
              "    'precision': 0.4750530012973452,\n",
              "    'recall': 0.46387754070178994,\n",
              "    'support': 306},\n",
              "   'weighted avg': {'f1-score': 0.6255594199177655,\n",
              "    'precision': 0.6167975524509993,\n",
              "    'recall': 0.6405228758169934,\n",
              "    'support': 306}},\n",
              "  'SVC': {'0': {'f1-score': 0.39999999999999997,\n",
              "    'precision': 0.47619047619047616,\n",
              "    'recall': 0.3448275862068966,\n",
              "    'support': 29},\n",
              "   '1': {'f1-score': 0.0, 'precision': 0.0, 'recall': 0.0, 'support': 11},\n",
              "   '2': {'f1-score': 0.7335423197492164,\n",
              "    'precision': 0.7134146341463414,\n",
              "    'recall': 0.7548387096774194,\n",
              "    'support': 155},\n",
              "   '3': {'f1-score': 0.5523809523809524,\n",
              "    'precision': 0.5918367346938775,\n",
              "    'recall': 0.5178571428571429,\n",
              "    'support': 56},\n",
              "   '5': {'f1-score': 0.5396825396825397,\n",
              "    'precision': 0.4788732394366197,\n",
              "    'recall': 0.6181818181818182,\n",
              "    'support': 55},\n",
              "   'accuracy': 0.6209150326797386,\n",
              "   'macro avg': {'f1-score': 0.4451211623625417,\n",
              "    'precision': 0.45206301689346295,\n",
              "    'recall': 0.44714105138465543,\n",
              "    'support': 306},\n",
              "   'weighted avg': {'f1-score': 0.6075651391405279,\n",
              "    'precision': 0.6008812987388168,\n",
              "    'recall': 0.6209150326797386,\n",
              "    'support': 306}}},\n",
              " '1449548': {'Decision Tree': {'0': {'f1-score': 0.46875,\n",
              "    'precision': 0.5555555555555556,\n",
              "    'recall': 0.40540540540540543,\n",
              "    'support': 37},\n",
              "   '1': {'f1-score': 0.08695652173913043,\n",
              "    'precision': 0.1111111111111111,\n",
              "    'recall': 0.07142857142857142,\n",
              "    'support': 14},\n",
              "   '2': {'f1-score': 0.7944444444444445,\n",
              "    'precision': 0.7566137566137566,\n",
              "    'recall': 0.8362573099415205,\n",
              "    'support': 171},\n",
              "   '3': {'f1-score': 0.37209302325581395,\n",
              "    'precision': 0.4,\n",
              "    'recall': 0.34782608695652173,\n",
              "    'support': 23},\n",
              "   '5': {'f1-score': 0.6857142857142857,\n",
              "    'precision': 0.6857142857142857,\n",
              "    'recall': 0.6857142857142857,\n",
              "    'support': 70},\n",
              "   'accuracy': 0.6825396825396826,\n",
              "   'macro avg': {'f1-score': 0.481591655030735,\n",
              "    'precision': 0.5017989417989417,\n",
              "    'recall': 0.469326331889261,\n",
              "    'support': 315},\n",
              "   'weighted avg': {'f1-score': 0.6697437486959732,\n",
              "    'precision': 0.6625144872763921,\n",
              "    'recall': 0.6825396825396826,\n",
              "    'support': 315}},\n",
              "  'K-Neighbours': {'0': {'f1-score': 0.20833333333333334,\n",
              "    'precision': 0.45454545454545453,\n",
              "    'recall': 0.13513513513513514,\n",
              "    'support': 37},\n",
              "   '1': {'f1-score': 0.0, 'precision': 0.0, 'recall': 0.0, 'support': 14},\n",
              "   '2': {'f1-score': 0.7791563275434243,\n",
              "    'precision': 0.6767241379310345,\n",
              "    'recall': 0.9181286549707602,\n",
              "    'support': 171},\n",
              "   '3': {'f1-score': 0.3888888888888889,\n",
              "    'precision': 0.5384615384615384,\n",
              "    'recall': 0.30434782608695654,\n",
              "    'support': 23},\n",
              "   '5': {'f1-score': 0.6356589147286822,\n",
              "    'precision': 0.6949152542372882,\n",
              "    'recall': 0.5857142857142857,\n",
              "    'support': 70},\n",
              "   'accuracy': 0.6666666666666666,\n",
              "   'macro avg': {'f1-score': 0.40240749289886574,\n",
              "    'precision': 0.47292927703506316,\n",
              "    'recall': 0.3886651803814275,\n",
              "    'support': 315},\n",
              "   'weighted avg': {'f1-score': 0.6170940756149559,\n",
              "    'precision': 0.6144974367797278,\n",
              "    'recall': 0.6666666666666666,\n",
              "    'support': 315}},\n",
              "  'SVC': {'0': {'f1-score': 0.2978723404255319,\n",
              "    'precision': 0.7,\n",
              "    'recall': 0.1891891891891892,\n",
              "    'support': 37},\n",
              "   '1': {'f1-score': 0.0, 'precision': 0.0, 'recall': 0.0, 'support': 14},\n",
              "   '2': {'f1-score': 0.7858942065491183,\n",
              "    'precision': 0.6902654867256637,\n",
              "    'recall': 0.9122807017543859,\n",
              "    'support': 171},\n",
              "   '3': {'f1-score': 0.4545454545454545,\n",
              "    'precision': 0.47619047619047616,\n",
              "    'recall': 0.43478260869565216,\n",
              "    'support': 23},\n",
              "   '5': {'f1-score': 0.671875,\n",
              "    'precision': 0.7413793103448276,\n",
              "    'recall': 0.6142857142857143,\n",
              "    'support': 70},\n",
              "   'accuracy': 0.6857142857142857,\n",
              "   'macro avg': {'f1-score': 0.44203740030402094,\n",
              "    'precision': 0.5215670546521934,\n",
              "    'recall': 0.43010764278498836,\n",
              "    'support': 315},\n",
              "   'weighted avg': {'f1-score': 0.6441110519688552,\n",
              "    'precision': 0.6564581933543091,\n",
              "    'recall': 0.6857142857142857,\n",
              "    'support': 315}}},\n",
              " '1455390': {'Decision Tree': {'0': {'f1-score': 0.2545454545454546,\n",
              "    'precision': 0.30434782608695654,\n",
              "    'recall': 0.21875,\n",
              "    'support': 32},\n",
              "   '1': {'f1-score': 0.125,\n",
              "    'precision': 0.125,\n",
              "    'recall': 0.125,\n",
              "    'support': 16},\n",
              "   '2': {'f1-score': 0.840909090909091,\n",
              "    'precision': 0.8185840707964602,\n",
              "    'recall': 0.8644859813084113,\n",
              "    'support': 214},\n",
              "   '5': {'f1-score': 0.7428571428571428,\n",
              "    'precision': 0.7647058823529411,\n",
              "    'recall': 0.7222222222222222,\n",
              "    'support': 54},\n",
              "   'accuracy': 0.7373417721518988,\n",
              "   'macro avg': {'f1-score': 0.49082792207792214,\n",
              "    'precision': 0.5031594448090895,\n",
              "    'recall': 0.48261455088265837,\n",
              "    'support': 316},\n",
              "   'weighted avg': {'f1-score': 0.7285262206148283,\n",
              "    'precision': 0.7221843013679871,\n",
              "    'recall': 0.7373417721518988,\n",
              "    'support': 316}},\n",
              "  'K-Neighbours': {'0': {'f1-score': 0.10810810810810811,\n",
              "    'precision': 0.4,\n",
              "    'recall': 0.0625,\n",
              "    'support': 32},\n",
              "   '1': {'f1-score': 0.08333333333333333,\n",
              "    'precision': 0.125,\n",
              "    'recall': 0.0625,\n",
              "    'support': 16},\n",
              "   '2': {'f1-score': 0.8232758620689655,\n",
              "    'precision': 0.764,\n",
              "    'recall': 0.8925233644859814,\n",
              "    'support': 214},\n",
              "   '5': {'f1-score': 0.5981308411214954,\n",
              "    'precision': 0.6037735849056604,\n",
              "    'recall': 0.5925925925925926,\n",
              "    'support': 54},\n",
              "   'accuracy': 0.7151898734177216,\n",
              "   'macro avg': {'f1-score': 0.4032120361579756,\n",
              "    'precision': 0.4731933962264151,\n",
              "    'recall': 0.4025289892696435,\n",
              "    'support': 316},\n",
              "   'weighted avg': {'f1-score': 0.6749142173927601,\n",
              "    'precision': 0.6674043467876762,\n",
              "    'recall': 0.7151898734177216,\n",
              "    'support': 316}},\n",
              "  'SVC': {'0': {'f1-score': 0.15384615384615383,\n",
              "    'precision': 0.42857142857142855,\n",
              "    'recall': 0.09375,\n",
              "    'support': 32},\n",
              "   '1': {'f1-score': 0.07692307692307693,\n",
              "    'precision': 0.1,\n",
              "    'recall': 0.0625,\n",
              "    'support': 16},\n",
              "   '2': {'f1-score': 0.8318965517241379,\n",
              "    'precision': 0.772,\n",
              "    'recall': 0.9018691588785047,\n",
              "    'support': 214},\n",
              "   '5': {'f1-score': 0.5825242718446603,\n",
              "    'precision': 0.6122448979591837,\n",
              "    'recall': 0.5555555555555556,\n",
              "    'support': 54},\n",
              "   'accuracy': 0.7183544303797469,\n",
              "   'macro avg': {'f1-score': 0.4112975135845073,\n",
              "    'precision': 0.4782040816326531,\n",
              "    'recall': 0.40341867860851505,\n",
              "    'support': 316},\n",
              "   'weighted avg': {'f1-score': 0.6823924648810865,\n",
              "    'precision': 0.6758971841901317,\n",
              "    'recall': 0.7183544303797469,\n",
              "    'support': 316}}},\n",
              " '1818471': {'Decision Tree': {'0': {'f1-score': 0.3333333333333333,\n",
              "    'precision': 0.3333333333333333,\n",
              "    'recall': 0.3333333333333333,\n",
              "    'support': 3},\n",
              "   '1': {'f1-score': 0.1111111111111111,\n",
              "    'precision': 0.08695652173913043,\n",
              "    'recall': 0.15384615384615385,\n",
              "    'support': 13},\n",
              "   '2': {'f1-score': 0.7471264367816092,\n",
              "    'precision': 0.7262569832402235,\n",
              "    'recall': 0.7692307692307693,\n",
              "    'support': 169},\n",
              "   '3': {'f1-score': 0.5454545454545454,\n",
              "    'precision': 0.5581395348837209,\n",
              "    'recall': 0.5333333333333333,\n",
              "    'support': 45},\n",
              "   '5': {'f1-score': 0.4487179487179487,\n",
              "    'precision': 0.5072463768115942,\n",
              "    'recall': 0.40229885057471265,\n",
              "    'support': 87},\n",
              "   'accuracy': 0.6056782334384858,\n",
              "   'macro avg': {'f1-score': 0.43714867507970945,\n",
              "    'precision': 0.4423865500016005,\n",
              "    'recall': 0.4384084880636604,\n",
              "    'support': 317},\n",
              "   'weighted avg': {'f1-score': 0.6066016667017429,\n",
              "    'precision': 0.612348829030229,\n",
              "    'recall': 0.6056782334384858,\n",
              "    'support': 317}},\n",
              "  'K-Neighbours': {'0': {'f1-score': 0.0,\n",
              "    'precision': 0.0,\n",
              "    'recall': 0.0,\n",
              "    'support': 3},\n",
              "   '1': {'f1-score': 0.0, 'precision': 0.0, 'recall': 0.0, 'support': 13},\n",
              "   '2': {'f1-score': 0.7276887871853548,\n",
              "    'precision': 0.5932835820895522,\n",
              "    'recall': 0.9408284023668639,\n",
              "    'support': 169},\n",
              "   '3': {'f1-score': 0.2,\n",
              "    'precision': 0.4,\n",
              "    'recall': 0.13333333333333333,\n",
              "    'support': 45},\n",
              "   '5': {'f1-score': 0.1896551724137931,\n",
              "    'precision': 0.3793103448275862,\n",
              "    'recall': 0.12643678160919541,\n",
              "    'support': 87},\n",
              "   'accuracy': 0.555205047318612,\n",
              "   'macro avg': {'f1-score': 0.22346879191982957,\n",
              "    'precision': 0.27451878538342767,\n",
              "    'recall': 0.24011970346187855,\n",
              "    'support': 317},\n",
              "   'weighted avg': {'f1-score': 0.46838929032910076,\n",
              "    'precision': 0.47717642073543953,\n",
              "    'recall': 0.555205047318612,\n",
              "    'support': 317}},\n",
              "  'SVC': {'0': {'f1-score': 0.0,\n",
              "    'precision': 0.0,\n",
              "    'recall': 0.0,\n",
              "    'support': 3},\n",
              "   '1': {'f1-score': 0.0, 'precision': 0.0, 'recall': 0.0, 'support': 13},\n",
              "   '2': {'f1-score': 0.6954732510288066,\n",
              "    'precision': 0.5331230283911672,\n",
              "    'recall': 1.0,\n",
              "    'support': 169},\n",
              "   '3': {'f1-score': 0.0, 'precision': 0.0, 'recall': 0.0, 'support': 45},\n",
              "   '5': {'f1-score': 0.0, 'precision': 0.0, 'recall': 0.0, 'support': 87},\n",
              "   'accuracy': 0.5331230283911672,\n",
              "   'macro avg': {'f1-score': 0.13909465020576134,\n",
              "    'precision': 0.10662460567823344,\n",
              "    'recall': 0.2,\n",
              "    'support': 317},\n",
              "   'weighted avg': {'f1-score': 0.37077280575352783,\n",
              "    'precision': 0.28422016340096923,\n",
              "    'recall': 0.5331230283911672,\n",
              "    'support': 317}}},\n",
              " '2598705': {'Decision Tree': {'0': {'f1-score': 0.0,\n",
              "    'precision': 0.0,\n",
              "    'recall': 0.0,\n",
              "    'support': 4},\n",
              "   '1': {'f1-score': 0.15789473684210525,\n",
              "    'precision': 0.125,\n",
              "    'recall': 0.21428571428571427,\n",
              "    'support': 14},\n",
              "   '2': {'f1-score': 0.6900584795321637,\n",
              "    'precision': 0.6900584795321637,\n",
              "    'recall': 0.6900584795321637,\n",
              "    'support': 171},\n",
              "   '3': {'f1-score': 0.4367816091954023,\n",
              "    'precision': 0.5135135135135135,\n",
              "    'recall': 0.38,\n",
              "    'support': 50},\n",
              "   '5': {'f1-score': 0.4868421052631579,\n",
              "    'precision': 0.4868421052631579,\n",
              "    'recall': 0.4868421052631579,\n",
              "    'support': 76},\n",
              "   'accuracy': 0.5619047619047619,\n",
              "   'macro avg': {'f1-score': 0.3543153861665658,\n",
              "    'precision': 0.36308281966176703,\n",
              "    'recall': 0.35423725981620724,\n",
              "    'support': 315},\n",
              "   'weighted avg': {'f1-score': 0.5684114500811416,\n",
              "    'precision': 0.5791291291291292,\n",
              "    'recall': 0.5619047619047619,\n",
              "    'support': 315}},\n",
              "  'K-Neighbours': {'0': {'f1-score': 0.0,\n",
              "    'precision': 0.0,\n",
              "    'recall': 0.0,\n",
              "    'support': 4},\n",
              "   '1': {'f1-score': 0.22222222222222224,\n",
              "    'precision': 0.23076923076923078,\n",
              "    'recall': 0.21428571428571427,\n",
              "    'support': 14},\n",
              "   '2': {'f1-score': 0.6646884272997032,\n",
              "    'precision': 0.6746987951807228,\n",
              "    'recall': 0.6549707602339181,\n",
              "    'support': 171},\n",
              "   '3': {'f1-score': 0.4081632653061225,\n",
              "    'precision': 0.4166666666666667,\n",
              "    'recall': 0.4,\n",
              "    'support': 50},\n",
              "   '5': {'f1-score': 0.37974683544303794,\n",
              "    'precision': 0.36585365853658536,\n",
              "    'recall': 0.39473684210526316,\n",
              "    'support': 76},\n",
              "   'accuracy': 0.5238095238095238,\n",
              "   'macro avg': {'f1-score': 0.3349641500542172,\n",
              "    'precision': 0.3375976702306412,\n",
              "    'recall': 0.33279866332497915,\n",
              "    'support': 315},\n",
              "   'weighted avg': {'f1-score': 0.5271166823439282,\n",
              "    'precision': 0.5309284907580528,\n",
              "    'recall': 0.5238095238095238,\n",
              "    'support': 315}},\n",
              "  'SVC': {'0': {'f1-score': 0.0,\n",
              "    'precision': 0.0,\n",
              "    'recall': 0.0,\n",
              "    'support': 4},\n",
              "   '1': {'f1-score': 0.0, 'precision': 0.0, 'recall': 0.0, 'support': 14},\n",
              "   '2': {'f1-score': 0.7455357142857144,\n",
              "    'precision': 0.6028880866425993,\n",
              "    'recall': 0.9766081871345029,\n",
              "    'support': 171},\n",
              "   '3': {'f1-score': 0.3235294117647059,\n",
              "    'precision': 0.6111111111111112,\n",
              "    'recall': 0.22,\n",
              "    'support': 50},\n",
              "   '5': {'f1-score': 0.29166666666666663,\n",
              "    'precision': 0.7,\n",
              "    'recall': 0.18421052631578946,\n",
              "    'support': 76},\n",
              "   'accuracy': 0.6095238095238096,\n",
              "   'macro avg': {'f1-score': 0.2721463585434174,\n",
              "    'precision': 0.3827998395507421,\n",
              "    'recall': 0.27616374269005844,\n",
              "    'support': 315},\n",
              "   'weighted avg': {'f1-score': 0.5264436330087592,\n",
              "    'precision': 0.5931727567347302,\n",
              "    'recall': 0.6095238095238096,\n",
              "    'support': 315}}},\n",
              " '2638030': {'Decision Tree': {'0': {'f1-score': 0.5714285714285714,\n",
              "    'precision': 0.5306122448979592,\n",
              "    'recall': 0.6190476190476191,\n",
              "    'support': 42},\n",
              "   '1': {'f1-score': 0.23076923076923075,\n",
              "    'precision': 0.1875,\n",
              "    'recall': 0.3,\n",
              "    'support': 10},\n",
              "   '2': {'f1-score': 0.7539936102236421,\n",
              "    'precision': 0.7919463087248322,\n",
              "    'recall': 0.7195121951219512,\n",
              "    'support': 164},\n",
              "   '3': {'f1-score': 0.4266666666666667,\n",
              "    'precision': 0.38095238095238093,\n",
              "    'recall': 0.48484848484848486,\n",
              "    'support': 33},\n",
              "   '5': {'f1-score': 0.8264462809917354,\n",
              "    'precision': 0.8771929824561403,\n",
              "    'recall': 0.78125,\n",
              "    'support': 64},\n",
              "   'accuracy': 0.6805111821086262,\n",
              "   'macro avg': {'f1-score': 0.5618608720159692,\n",
              "    'precision': 0.5536407834062625,\n",
              "    'recall': 0.5809316598036111,\n",
              "    'support': 313},\n",
              "   'weighted avg': {'f1-score': 0.6930837264148265,\n",
              "    'precision': 0.711666735991081,\n",
              "    'recall': 0.6805111821086262,\n",
              "    'support': 313}},\n",
              "  'K-Neighbours': {'0': {'f1-score': 0.46341463414634143,\n",
              "    'precision': 0.475,\n",
              "    'recall': 0.4523809523809524,\n",
              "    'support': 42},\n",
              "   '1': {'f1-score': 0.0, 'precision': 0.0, 'recall': 0.0, 'support': 10},\n",
              "   '2': {'f1-score': 0.7405247813411078,\n",
              "    'precision': 0.7094972067039106,\n",
              "    'recall': 0.774390243902439,\n",
              "    'support': 164},\n",
              "   '3': {'f1-score': 0.37837837837837834,\n",
              "    'precision': 0.34146341463414637,\n",
              "    'recall': 0.42424242424242425,\n",
              "    'support': 33},\n",
              "   '5': {'f1-score': 0.7350427350427351,\n",
              "    'precision': 0.8113207547169812,\n",
              "    'recall': 0.671875,\n",
              "    'support': 64},\n",
              "   'accuracy': 0.6485623003194888,\n",
              "   'macro avg': {'f1-score': 0.4634721057817125,\n",
              "    'precision': 0.46745627521100763,\n",
              "    'recall': 0.46457772410516307,\n",
              "    'support': 313},\n",
              "   'weighted avg': {'f1-score': 0.6403792341958771,\n",
              "    'precision': 0.6373813510679073,\n",
              "    'recall': 0.6485623003194888,\n",
              "    'support': 313}},\n",
              "  'SVC': {'0': {'f1-score': 0.5384615384615384,\n",
              "    'precision': 0.5833333333333334,\n",
              "    'recall': 0.5,\n",
              "    'support': 42},\n",
              "   '1': {'f1-score': 0.0, 'precision': 0.0, 'recall': 0.0, 'support': 10},\n",
              "   '2': {'f1-score': 0.7924528301886792,\n",
              "    'precision': 0.7101449275362319,\n",
              "    'recall': 0.8963414634146342,\n",
              "    'support': 164},\n",
              "   '3': {'f1-score': 0.3829787234042553,\n",
              "    'precision': 0.6428571428571429,\n",
              "    'recall': 0.2727272727272727,\n",
              "    'support': 33},\n",
              "   '5': {'f1-score': 0.75,\n",
              "    'precision': 0.8035714285714286,\n",
              "    'recall': 0.703125,\n",
              "    'support': 64},\n",
              "   'accuracy': 0.7092651757188498,\n",
              "   'macro avg': {'f1-score': 0.4927786184108946,\n",
              "    'precision': 0.5479813664596274,\n",
              "    'recall': 0.47443874722838136,\n",
              "    'support': 313},\n",
              "   'weighted avg': {'f1-score': 0.6812011074717841,\n",
              "    'precision': 0.6824492819769942,\n",
              "    'recall': 0.7092651757188498,\n",
              "    'support': 313}}},\n",
              " '3509524': {'Decision Tree': {'0': {'f1-score': 0.553191489361702,\n",
              "    'precision': 0.48148148148148145,\n",
              "    'recall': 0.65,\n",
              "    'support': 20},\n",
              "   '1': {'f1-score': 0.14285714285714288,\n",
              "    'precision': 0.16666666666666666,\n",
              "    'recall': 0.125,\n",
              "    'support': 8},\n",
              "   '2': {'f1-score': 0.8176100628930817,\n",
              "    'precision': 0.8441558441558441,\n",
              "    'recall': 0.7926829268292683,\n",
              "    'support': 82},\n",
              "   '3': {'f1-score': 0.4827586206896552,\n",
              "    'precision': 0.5384615384615384,\n",
              "    'recall': 0.4375,\n",
              "    'support': 16},\n",
              "   '5': {'f1-score': 0.5185185185185186,\n",
              "    'precision': 0.4666666666666667,\n",
              "    'recall': 0.5833333333333334,\n",
              "    'support': 12},\n",
              "   'accuracy': 0.6739130434782609,\n",
              "   'macro avg': {'f1-score': 0.5029871668640201,\n",
              "    'precision': 0.4994864394864395,\n",
              "    'recall': 0.5177032520325204,\n",
              "    'support': 138},\n",
              "   'weighted avg': {'f1-score': 0.6753411031926129,\n",
              "    'precision': 0.6840516434719333,\n",
              "    'recall': 0.6739130434782609,\n",
              "    'support': 138}},\n",
              "  'K-Neighbours': {'0': {'f1-score': 0.4375,\n",
              "    'precision': 0.5833333333333334,\n",
              "    'recall': 0.35,\n",
              "    'support': 20},\n",
              "   '1': {'f1-score': 0.0, 'precision': 0.0, 'recall': 0.0, 'support': 8},\n",
              "   '2': {'f1-score': 0.8268156424581006,\n",
              "    'precision': 0.7628865979381443,\n",
              "    'recall': 0.9024390243902439,\n",
              "    'support': 82},\n",
              "   '3': {'f1-score': 0.41379310344827586,\n",
              "    'precision': 0.46153846153846156,\n",
              "    'recall': 0.375,\n",
              "    'support': 16},\n",
              "   '5': {'f1-score': 0.5,\n",
              "    'precision': 0.4375,\n",
              "    'recall': 0.5833333333333334,\n",
              "    'support': 12},\n",
              "   'accuracy': 0.6811594202898551,\n",
              "   'macro avg': {'f1-score': 0.43562174918127533,\n",
              "    'precision': 0.4490516785619879,\n",
              "    'recall': 0.4421544715447155,\n",
              "    'support': 138},\n",
              "   'weighted avg': {'f1-score': 0.6461563212807004,\n",
              "    'precision': 0.6294056745087673,\n",
              "    'recall': 0.6811594202898551,\n",
              "    'support': 138}},\n",
              "  'SVC': {'0': {'f1-score': 0.14285714285714288,\n",
              "    'precision': 0.25,\n",
              "    'recall': 0.1,\n",
              "    'support': 20},\n",
              "   '1': {'f1-score': 0.0, 'precision': 0.0, 'recall': 0.0, 'support': 8},\n",
              "   '2': {'f1-score': 0.7830687830687831,\n",
              "    'precision': 0.6915887850467289,\n",
              "    'recall': 0.9024390243902439,\n",
              "    'support': 82},\n",
              "   '3': {'f1-score': 0.4166666666666667,\n",
              "    'precision': 0.625,\n",
              "    'recall': 0.3125,\n",
              "    'support': 16},\n",
              "   '5': {'f1-score': 0.4444444444444445,\n",
              "    'precision': 0.4,\n",
              "    'recall': 0.5,\n",
              "    'support': 12},\n",
              "   'accuracy': 0.6304347826086957,\n",
              "   'macro avg': {'f1-score': 0.35740740740740745,\n",
              "    'precision': 0.39331775700934574,\n",
              "    'recall': 0.36298780487804877,\n",
              "    'support': 138},\n",
              "   'weighted avg': {'f1-score': 0.572962196150602,\n",
              "    'precision': 0.5544223215495057,\n",
              "    'recall': 0.6304347826086957,\n",
              "    'support': 138}}},\n",
              " '3997827': {'Decision Tree': {'0': {'f1-score': 0.42857142857142855,\n",
              "    'precision': 0.375,\n",
              "    'recall': 0.5,\n",
              "    'support': 6},\n",
              "   '1': {'f1-score': 0.0, 'precision': 0.0, 'recall': 0.0, 'support': 8},\n",
              "   '2': {'f1-score': 0.7844311377245508,\n",
              "    'precision': 0.8506493506493507,\n",
              "    'recall': 0.7277777777777777,\n",
              "    'support': 180},\n",
              "   '3': {'f1-score': 0.7999999999999999,\n",
              "    'precision': 0.7368421052631579,\n",
              "    'recall': 0.875,\n",
              "    'support': 32},\n",
              "   '5': {'f1-score': 0.7096774193548387,\n",
              "    'precision': 0.6947368421052632,\n",
              "    'recall': 0.7252747252747253,\n",
              "    'support': 91},\n",
              "   'accuracy': 0.7192429022082019,\n",
              "   'macro avg': {'f1-score': 0.5445359971301637,\n",
              "    'precision': 0.5314456596035544,\n",
              "    'recall': 0.5656105006105007,\n",
              "    'support': 317},\n",
              "   'weighted avg': {'f1-score': 0.7380116041739371,\n",
              "    'precision': 0.7639333852267606,\n",
              "    'recall': 0.7192429022082019,\n",
              "    'support': 317}},\n",
              "  'K-Neighbours': {'0': {'f1-score': 0.15384615384615383,\n",
              "    'precision': 0.14285714285714285,\n",
              "    'recall': 0.16666666666666666,\n",
              "    'support': 6},\n",
              "   '1': {'f1-score': 0.0, 'precision': 0.0, 'recall': 0.0, 'support': 8},\n",
              "   '2': {'f1-score': 0.7544303797468355,\n",
              "    'precision': 0.6930232558139535,\n",
              "    'recall': 0.8277777777777777,\n",
              "    'support': 180},\n",
              "   '3': {'f1-score': 0.3, 'precision': 0.75, 'recall': 0.1875, 'support': 32},\n",
              "   '5': {'f1-score': 0.6292134831460674,\n",
              "    'precision': 0.6436781609195402,\n",
              "    'recall': 0.6153846153846154,\n",
              "    'support': 91},\n",
              "   'accuracy': 0.668769716088328,\n",
              "   'macro avg': {'f1-score': 0.36749800334781135,\n",
              "    'precision': 0.4459117119181273,\n",
              "    'recall': 0.35946581196581195,\n",
              "    'support': 317},\n",
              "   'weighted avg': {'f1-score': 0.6422049597596198,\n",
              "    'precision': 0.6567067556698191,\n",
              "    'recall': 0.668769716088328,\n",
              "    'support': 317}},\n",
              "  'SVC': {'0': {'f1-score': 0.0,\n",
              "    'precision': 0.0,\n",
              "    'recall': 0.0,\n",
              "    'support': 6},\n",
              "   '1': {'f1-score': 0.0, 'precision': 0.0, 'recall': 0.0, 'support': 8},\n",
              "   '2': {'f1-score': 0.734793187347932,\n",
              "    'precision': 0.6536796536796536,\n",
              "    'recall': 0.8388888888888889,\n",
              "    'support': 180},\n",
              "   '3': {'f1-score': 0.3157894736842105,\n",
              "    'precision': 1.0,\n",
              "    'recall': 0.1875,\n",
              "    'support': 32},\n",
              "   '5': {'f1-score': 0.5529411764705883,\n",
              "    'precision': 0.5949367088607594,\n",
              "    'recall': 0.5164835164835165,\n",
              "    'support': 91},\n",
              "   'accuracy': 0.6435331230283912,\n",
              "   'macro avg': {'f1-score': 0.32070476750054616,\n",
              "    'precision': 0.4497232725080826,\n",
              "    'recall': 0.3085744810744811,\n",
              "    'support': 317},\n",
              "   'weighted avg': {'f1-score': 0.607841274256612,\n",
              "    'precision': 0.6429071866519457,\n",
              "    'recall': 0.6435331230283912,\n",
              "    'support': 317}}},\n",
              " '4018081': {'Decision Tree': {'0': {'f1-score': 0.6571428571428573,\n",
              "    'precision': 0.7419354838709677,\n",
              "    'recall': 0.5897435897435898,\n",
              "    'support': 39},\n",
              "   '1': {'f1-score': 0.0, 'precision': 0.0, 'recall': 0.0, 'support': 8},\n",
              "   '2': {'f1-score': 0.7272727272727273,\n",
              "    'precision': 0.6984126984126984,\n",
              "    'recall': 0.7586206896551724,\n",
              "    'support': 58},\n",
              "   '3': {'f1-score': 0.7128712871287128,\n",
              "    'precision': 0.6923076923076923,\n",
              "    'recall': 0.7346938775510204,\n",
              "    'support': 49},\n",
              "   '5': {'f1-score': 0.56,\n",
              "    'precision': 0.5,\n",
              "    'recall': 0.6363636363636364,\n",
              "    'support': 11},\n",
              "   'accuracy': 0.6666666666666666,\n",
              "   'macro avg': {'f1-score': 0.5314573743088595,\n",
              "    'precision': 0.5265311749182716,\n",
              "    'recall': 0.5438843586626837,\n",
              "    'support': 165},\n",
              "   'weighted avg': {'f1-score': 0.6600065616951306,\n",
              "    'precision': 0.6597969533453404,\n",
              "    'recall': 0.6666666666666666,\n",
              "    'support': 165}},\n",
              "  'K-Neighbours': {'0': {'f1-score': 0.5714285714285714,\n",
              "    'precision': 0.75,\n",
              "    'recall': 0.46153846153846156,\n",
              "    'support': 39},\n",
              "   '1': {'f1-score': 0.0, 'precision': 0.0, 'recall': 0.0, 'support': 8},\n",
              "   '2': {'f1-score': 0.6666666666666666,\n",
              "    'precision': 0.6785714285714286,\n",
              "    'recall': 0.6551724137931034,\n",
              "    'support': 58},\n",
              "   '3': {'f1-score': 0.65625,\n",
              "    'precision': 0.5316455696202531,\n",
              "    'recall': 0.8571428571428571,\n",
              "    'support': 49},\n",
              "   '5': {'f1-score': 0.23529411764705885,\n",
              "    'precision': 0.3333333333333333,\n",
              "    'recall': 0.18181818181818182,\n",
              "    'support': 11},\n",
              "   'accuracy': 0.6060606060606061,\n",
              "   'macro avg': {'f1-score': 0.4259278711484594,\n",
              "    'precision': 0.45871006630500305,\n",
              "    'recall': 0.43113438285852074,\n",
              "    'support': 165},\n",
              "   'weighted avg': {'f1-score': 0.579981007554537,\n",
              "    'precision': 0.5959057117284966,\n",
              "    'recall': 0.6060606060606061,\n",
              "    'support': 165}},\n",
              "  'SVC': {'0': {'f1-score': 0.5161290322580646,\n",
              "    'precision': 0.6956521739130435,\n",
              "    'recall': 0.41025641025641024,\n",
              "    'support': 39},\n",
              "   '1': {'f1-score': 0.0, 'precision': 0.0, 'recall': 0.0, 'support': 8},\n",
              "   '2': {'f1-score': 0.6942148760330579,\n",
              "    'precision': 0.6666666666666666,\n",
              "    'recall': 0.7241379310344828,\n",
              "    'support': 58},\n",
              "   '3': {'f1-score': 0.6341463414634146,\n",
              "    'precision': 0.527027027027027,\n",
              "    'recall': 0.7959183673469388,\n",
              "    'support': 49},\n",
              "   '5': {'f1-score': 0.37499999999999994,\n",
              "    'precision': 0.6,\n",
              "    'recall': 0.2727272727272727,\n",
              "    'support': 11},\n",
              "   'accuracy': 0.6060606060606061,\n",
              "   'macro avg': {'f1-score': 0.4438980499509074,\n",
              "    'precision': 0.49786917352134735,\n",
              "    'recall': 0.44060799627302083,\n",
              "    'support': 165},\n",
              "   'weighted avg': {'f1-score': 0.5793434290890255,\n",
              "    'precision': 0.5952813683248466,\n",
              "    'recall': 0.6060606060606061,\n",
              "    'support': 165}}},\n",
              " '4314139': {'Decision Tree': {'0': {'f1-score': 0.24242424242424246,\n",
              "    'precision': 0.22857142857142856,\n",
              "    'recall': 0.25806451612903225,\n",
              "    'support': 31},\n",
              "   '1': {'f1-score': 0.125,\n",
              "    'precision': 0.09523809523809523,\n",
              "    'recall': 0.18181818181818182,\n",
              "    'support': 22},\n",
              "   '2': {'f1-score': 0.6394984326018809,\n",
              "    'precision': 0.6666666666666666,\n",
              "    'recall': 0.6144578313253012,\n",
              "    'support': 166},\n",
              "   '3': {'f1-score': 0.31999999999999995,\n",
              "    'precision': 0.375,\n",
              "    'recall': 0.27906976744186046,\n",
              "    'support': 43},\n",
              "   '5': {'f1-score': 0.5357142857142857,\n",
              "    'precision': 0.5357142857142857,\n",
              "    'recall': 0.5357142857142857,\n",
              "    'support': 56},\n",
              "   'accuracy': 0.49056603773584906,\n",
              "   'macro avg': {'f1-score': 0.3725273921480817,\n",
              "    'precision': 0.3802380952380952,\n",
              "    'recall': 0.3738249164857323,\n",
              "    'support': 318},\n",
              "   'weighted avg': {'f1-score': 0.5037166393932822,\n",
              "    'precision': 0.5219264749925128,\n",
              "    'recall': 0.49056603773584906,\n",
              "    'support': 318}},\n",
              "  'K-Neighbours': {'0': {'f1-score': 0.16666666666666666,\n",
              "    'precision': 0.6,\n",
              "    'recall': 0.0967741935483871,\n",
              "    'support': 31},\n",
              "   '1': {'f1-score': 0.0, 'precision': 0.0, 'recall': 0.0, 'support': 22},\n",
              "   '2': {'f1-score': 0.6981132075471698,\n",
              "    'precision': 0.5736434108527132,\n",
              "    'recall': 0.891566265060241,\n",
              "    'support': 166},\n",
              "   '3': {'f1-score': 0.3508771929824562,\n",
              "    'precision': 0.7142857142857143,\n",
              "    'recall': 0.23255813953488372,\n",
              "    'support': 43},\n",
              "   '5': {'f1-score': 0.45360824742268047,\n",
              "    'precision': 0.5365853658536586,\n",
              "    'recall': 0.39285714285714285,\n",
              "    'support': 56},\n",
              "   'accuracy': 0.5754716981132075,\n",
              "   'macro avg': {'f1-score': 0.3338530629237946,\n",
              "    'precision': 0.4849028981984172,\n",
              "    'recall': 0.32275114820013096,\n",
              "    'support': 318},\n",
              "   'weighted avg': {'f1-score': 0.507997610922681,\n",
              "    'precision': 0.5490184666781164,\n",
              "    'recall': 0.5754716981132075,\n",
              "    'support': 318}},\n",
              "  'SVC': {'0': {'f1-score': 0.17647058823529413,\n",
              "    'precision': 1.0,\n",
              "    'recall': 0.0967741935483871,\n",
              "    'support': 31},\n",
              "   '1': {'f1-score': 0.0, 'precision': 0.0, 'recall': 0.0, 'support': 22},\n",
              "   '2': {'f1-score': 0.6799116997792495,\n",
              "    'precision': 0.5365853658536586,\n",
              "    'recall': 0.927710843373494,\n",
              "    'support': 166},\n",
              "   '3': {'f1-score': 0.0, 'precision': 0.0, 'recall': 0.0, 'support': 43},\n",
              "   '5': {'f1-score': 0.30952380952380953,\n",
              "    'precision': 0.4642857142857143,\n",
              "    'recall': 0.23214285714285715,\n",
              "    'support': 56},\n",
              "   'accuracy': 0.5345911949685535,\n",
              "   'macro avg': {'f1-score': 0.23318121950767062,\n",
              "    'precision': 0.4001742160278746,\n",
              "    'recall': 0.25132557881294765,\n",
              "    'support': 318},\n",
              "   'weighted avg': {'f1-score': 0.4266329048175561,\n",
              "    'precision': 0.4593495934959349,\n",
              "    'recall': 0.5345911949685535,\n",
              "    'support': 318}}},\n",
              " '4426783': {'Decision Tree': {'0': {'f1-score': 0.24489795918367344,\n",
              "    'precision': 0.21428571428571427,\n",
              "    'recall': 0.2857142857142857,\n",
              "    'support': 21},\n",
              "   '1': {'f1-score': 0.16666666666666666,\n",
              "    'precision': 0.14285714285714285,\n",
              "    'recall': 0.2,\n",
              "    'support': 20},\n",
              "   '2': {'f1-score': 0.6440677966101696,\n",
              "    'precision': 0.6737588652482269,\n",
              "    'recall': 0.6168831168831169,\n",
              "    'support': 154},\n",
              "   '3': {'f1-score': 0.32,\n",
              "    'precision': 0.25,\n",
              "    'recall': 0.4444444444444444,\n",
              "    'support': 18},\n",
              "   '5': {'f1-score': 0.6893203883495146,\n",
              "    'precision': 0.7473684210526316,\n",
              "    'recall': 0.6396396396396397,\n",
              "    'support': 111},\n",
              "   'accuracy': 0.5679012345679012,\n",
              "   'macro avg': {'f1-score': 0.41299056216200486,\n",
              "    'precision': 0.4056540286887431,\n",
              "    'recall': 0.43733629733629736,\n",
              "    'support': 324},\n",
              "   'weighted avg': {'f1-score': 0.586225908212817,\n",
              "    'precision': 0.6128824161796664,\n",
              "    'recall': 0.5679012345679012,\n",
              "    'support': 324}},\n",
              "  'K-Neighbours': {'0': {'f1-score': 0.0,\n",
              "    'precision': 0.0,\n",
              "    'recall': 0.0,\n",
              "    'support': 21},\n",
              "   '1': {'f1-score': 0.0, 'precision': 0.0, 'recall': 0.0, 'support': 20},\n",
              "   '2': {'f1-score': 0.7336956521739131,\n",
              "    'precision': 0.6308411214953271,\n",
              "    'recall': 0.8766233766233766,\n",
              "    'support': 154},\n",
              "   '3': {'f1-score': 0.2222222222222222,\n",
              "    'precision': 0.3333333333333333,\n",
              "    'recall': 0.16666666666666666,\n",
              "    'support': 18},\n",
              "   '5': {'f1-score': 0.6116504854368933,\n",
              "    'precision': 0.6631578947368421,\n",
              "    'recall': 0.5675675675675675,\n",
              "    'support': 111},\n",
              "   'accuracy': 0.6203703703703703,\n",
              "   'macro avg': {'f1-score': 0.31351367196660573,\n",
              "    'precision': 0.32546646991310046,\n",
              "    'recall': 0.3221715221715222,\n",
              "    'support': 324},\n",
              "   'weighted avg': {'f1-score': 0.5706244886366597,\n",
              "    'precision': 0.5455557377347835,\n",
              "    'recall': 0.6203703703703703,\n",
              "    'support': 324}},\n",
              "  'SVC': {'0': {'f1-score': 0.23076923076923073,\n",
              "    'precision': 0.6,\n",
              "    'recall': 0.14285714285714285,\n",
              "    'support': 21},\n",
              "   '1': {'f1-score': 0.0, 'precision': 0.0, 'recall': 0.0, 'support': 20},\n",
              "   '2': {'f1-score': 0.746031746031746,\n",
              "    'precision': 0.6294642857142857,\n",
              "    'recall': 0.9155844155844156,\n",
              "    'support': 154},\n",
              "   '3': {'f1-score': 0.0909090909090909,\n",
              "    'precision': 0.25,\n",
              "    'recall': 0.05555555555555555,\n",
              "    'support': 18},\n",
              "   '5': {'f1-score': 0.6169154228855721,\n",
              "    'precision': 0.6888888888888889,\n",
              "    'recall': 0.5585585585585585,\n",
              "    'support': 111},\n",
              "   'accuracy': 0.6388888888888888,\n",
              "   'macro avg': {'f1-score': 0.33692509811912796,\n",
              "    'precision': 0.43367063492063485,\n",
              "    'recall': 0.3345111345111345,\n",
              "    'support': 324},\n",
              "   'weighted avg': {'f1-score': 0.5859537602213113,\n",
              "    'precision': 0.5879758230452675,\n",
              "    'recall': 0.6388888888888888,\n",
              "    'support': 324}}},\n",
              " '46343': {'Decision Tree': {'0': {'f1-score': 0.6301369863013699,\n",
              "    'precision': 0.5897435897435898,\n",
              "    'recall': 0.6764705882352942,\n",
              "    'support': 34},\n",
              "   '1': {'f1-score': 0.28571428571428564,\n",
              "    'precision': 0.2727272727272727,\n",
              "    'recall': 0.3,\n",
              "    'support': 10},\n",
              "   '2': {'f1-score': 0.4854368932038835,\n",
              "    'precision': 0.4716981132075472,\n",
              "    'recall': 0.5,\n",
              "    'support': 50},\n",
              "   '3': {'f1-score': 0.5684210526315789,\n",
              "    'precision': 0.6136363636363636,\n",
              "    'recall': 0.5294117647058824,\n",
              "    'support': 51},\n",
              "   '5': {'f1-score': 0.6486486486486486,\n",
              "    'precision': 0.6666666666666666,\n",
              "    'recall': 0.631578947368421,\n",
              "    'support': 38},\n",
              "   'accuracy': 0.5573770491803278,\n",
              "   'macro avg': {'f1-score': 0.5236715732999533,\n",
              "    'precision': 0.5228944011962879,\n",
              "    'recall': 0.5274922600619195,\n",
              "    'support': 183},\n",
              "   'weighted avg': {'f1-score': 0.5584249583849332,\n",
              "    'precision': 0.562799171135082,\n",
              "    'recall': 0.5573770491803278,\n",
              "    'support': 183}},\n",
              "  'K-Neighbours': {'0': {'f1-score': 0.5,\n",
              "    'precision': 0.5,\n",
              "    'recall': 0.5,\n",
              "    'support': 34},\n",
              "   '1': {'f1-score': 0.0, 'precision': 0.0, 'recall': 0.0, 'support': 10},\n",
              "   '2': {'f1-score': 0.5,\n",
              "    'precision': 0.48148148148148145,\n",
              "    'recall': 0.52,\n",
              "    'support': 50},\n",
              "   '3': {'f1-score': 0.45614035087719296,\n",
              "    'precision': 0.4126984126984127,\n",
              "    'recall': 0.5098039215686274,\n",
              "    'support': 51},\n",
              "   '5': {'f1-score': 0.39999999999999997,\n",
              "    'precision': 0.4375,\n",
              "    'recall': 0.3684210526315789,\n",
              "    'support': 38},\n",
              "   'accuracy': 0.453551912568306,\n",
              "   'macro avg': {'f1-score': 0.3712280701754386,\n",
              "    'precision': 0.3663359788359788,\n",
              "    'recall': 0.37964499484004127,\n",
              "    'support': 183},\n",
              "   'weighted avg': {'f1-score': 0.4396893874029336,\n",
              "    'precision': 0.4303097984791974,\n",
              "    'recall': 0.453551912568306,\n",
              "    'support': 183}},\n",
              "  'SVC': {'0': {'f1-score': 0.5151515151515151,\n",
              "    'precision': 0.53125,\n",
              "    'recall': 0.5,\n",
              "    'support': 34},\n",
              "   '1': {'f1-score': 0.0, 'precision': 0.0, 'recall': 0.0, 'support': 10},\n",
              "   '2': {'f1-score': 0.5050505050505051,\n",
              "    'precision': 0.5102040816326531,\n",
              "    'recall': 0.5,\n",
              "    'support': 50},\n",
              "   '3': {'f1-score': 0.5714285714285714,\n",
              "    'precision': 0.5245901639344263,\n",
              "    'recall': 0.6274509803921569,\n",
              "    'support': 51},\n",
              "   '5': {'f1-score': 0.5641025641025641,\n",
              "    'precision': 0.55,\n",
              "    'recall': 0.5789473684210527,\n",
              "    'support': 38},\n",
              "   'accuracy': 0.5245901639344263,\n",
              "   'macro avg': {'f1-score': 0.43114663114663115,\n",
              "    'precision': 0.42320884911341583,\n",
              "    'recall': 0.4412796697626419,\n",
              "    'support': 183},\n",
              "   'weighted avg': {'f1-score': 0.5100897887783133,\n",
              "    'precision': 0.4985071171709749,\n",
              "    'recall': 0.5245901639344263,\n",
              "    'support': 183}}},\n",
              " '5132496': {'Decision Tree': {'0': {'f1-score': 0.2857142857142857,\n",
              "    'precision': 0.375,\n",
              "    'recall': 0.23076923076923078,\n",
              "    'support': 13},\n",
              "   '1': {'f1-score': 0.1081081081081081,\n",
              "    'precision': 0.1,\n",
              "    'recall': 0.11764705882352941,\n",
              "    'support': 17},\n",
              "   '2': {'f1-score': 0.6233766233766234,\n",
              "    'precision': 0.6,\n",
              "    'recall': 0.6486486486486487,\n",
              "    'support': 74},\n",
              "   '3': {'f1-score': 0.6250000000000001,\n",
              "    'precision': 0.6521739130434783,\n",
              "    'recall': 0.6,\n",
              "    'support': 50},\n",
              "   'accuracy': 0.538961038961039,\n",
              "   'macro avg': {'f1-score': 0.41054975429975427,\n",
              "    'precision': 0.43179347826086956,\n",
              "    'recall': 0.3992662345603522,\n",
              "    'support': 154},\n",
              "   'weighted avg': {'f1-score': 0.5385194394934655,\n",
              "    'precision': 0.5427512704686618,\n",
              "    'recall': 0.538961038961039,\n",
              "    'support': 154}},\n",
              "  'K-Neighbours': {'0': {'f1-score': 0.5217391304347826,\n",
              "    'precision': 0.6,\n",
              "    'recall': 0.46153846153846156,\n",
              "    'support': 13},\n",
              "   '1': {'f1-score': 0.19047619047619047,\n",
              "    'precision': 0.5,\n",
              "    'recall': 0.11764705882352941,\n",
              "    'support': 17},\n",
              "   '2': {'f1-score': 0.7283950617283949,\n",
              "    'precision': 0.6704545454545454,\n",
              "    'recall': 0.7972972972972973,\n",
              "    'support': 74},\n",
              "   '3': {'f1-score': 0.5686274509803922,\n",
              "    'precision': 0.5576923076923077,\n",
              "    'recall': 0.58,\n",
              "    'support': 50},\n",
              "   'accuracy': 0.6233766233766234,\n",
              "   'macro avg': {'f1-score': 0.50230945840494,\n",
              "    'precision': 0.5820367132867132,\n",
              "    'recall': 0.489120704414822,\n",
              "    'support': 154},\n",
              "   'weighted avg': {'f1-score': 0.5996968250043393,\n",
              "    'precision': 0.6090795568068295,\n",
              "    'recall': 0.6233766233766234,\n",
              "    'support': 154}},\n",
              "  'SVC': {'0': {'f1-score': 0.42105263157894735,\n",
              "    'precision': 0.6666666666666666,\n",
              "    'recall': 0.3076923076923077,\n",
              "    'support': 13},\n",
              "   '1': {'f1-score': 0.0, 'precision': 0.0, 'recall': 0.0, 'support': 17},\n",
              "   '2': {'f1-score': 0.7485380116959065,\n",
              "    'precision': 0.6597938144329897,\n",
              "    'recall': 0.8648648648648649,\n",
              "    'support': 74},\n",
              "   '3': {'f1-score': 0.6666666666666666,\n",
              "    'precision': 0.673469387755102,\n",
              "    'recall': 0.66,\n",
              "    'support': 50},\n",
              "   'accuracy': 0.6558441558441559,\n",
              "   'macro avg': {'f1-score': 0.45906432748538006,\n",
              "    'precision': 0.49998246721368955,\n",
              "    'recall': 0.45813929313929314,\n",
              "    'support': 154},\n",
              "   'weighted avg': {'f1-score': 0.6116807169438748,\n",
              "    'precision': 0.5919797293666428,\n",
              "    'recall': 0.6558441558441559,\n",
              "    'support': 154}}},\n",
              " '5498603': {'Decision Tree': {'0': {'f1-score': 0.4675324675324675,\n",
              "    'precision': 0.45,\n",
              "    'recall': 0.4864864864864865,\n",
              "    'support': 37},\n",
              "   '1': {'f1-score': 0.0625,\n",
              "    'precision': 0.0625,\n",
              "    'recall': 0.0625,\n",
              "    'support': 16},\n",
              "   '2': {'f1-score': 0.5937499999999999,\n",
              "    'precision': 0.7125,\n",
              "    'recall': 0.5089285714285714,\n",
              "    'support': 112},\n",
              "   '3': {'f1-score': 0.5982905982905983,\n",
              "    'precision': 0.5303030303030303,\n",
              "    'recall': 0.6862745098039216,\n",
              "    'support': 51},\n",
              "   '5': {'f1-score': 0.6756756756756758,\n",
              "    'precision': 0.5681818181818182,\n",
              "    'recall': 0.8333333333333334,\n",
              "    'support': 30},\n",
              "   'accuracy': 0.5528455284552846,\n",
              "   'macro avg': {'f1-score': 0.4795497482997483,\n",
              "    'precision': 0.4646969696969697,\n",
              "    'recall': 0.5155045802104625,\n",
              "    'support': 246},\n",
              "   'weighted avg': {'f1-score': 0.5511454962674474,\n",
              "    'precision': 0.575369549150037,\n",
              "    'recall': 0.5528455284552846,\n",
              "    'support': 246}},\n",
              "  'K-Neighbours': {'0': {'f1-score': 0.375,\n",
              "    'precision': 0.4444444444444444,\n",
              "    'recall': 0.32432432432432434,\n",
              "    'support': 37},\n",
              "   '1': {'f1-score': 0.0, 'precision': 0.0, 'recall': 0.0, 'support': 16},\n",
              "   '2': {'f1-score': 0.6880000000000001,\n",
              "    'precision': 0.6231884057971014,\n",
              "    'recall': 0.7678571428571429,\n",
              "    'support': 112},\n",
              "   '3': {'f1-score': 0.5384615384615384,\n",
              "    'precision': 0.5283018867924528,\n",
              "    'recall': 0.5490196078431373,\n",
              "    'support': 51},\n",
              "   '5': {'f1-score': 0.5263157894736842,\n",
              "    'precision': 0.5555555555555556,\n",
              "    'recall': 0.5,\n",
              "    'support': 30},\n",
              "   'accuracy': 0.573170731707317,\n",
              "   'macro avg': {'f1-score': 0.4255554655870445,\n",
              "    'precision': 0.43029805851791086,\n",
              "    'recall': 0.428240215004921,\n",
              "    'support': 246},\n",
              "   'weighted avg': {'f1-score': 0.5454553339258089,\n",
              "    'precision': 0.5278520682390307,\n",
              "    'recall': 0.573170731707317,\n",
              "    'support': 246}},\n",
              "  'SVC': {'0': {'f1-score': 0.34782608695652173,\n",
              "    'precision': 0.375,\n",
              "    'recall': 0.32432432432432434,\n",
              "    'support': 37},\n",
              "   '1': {'f1-score': 0.10526315789473684,\n",
              "    'precision': 0.3333333333333333,\n",
              "    'recall': 0.0625,\n",
              "    'support': 16},\n",
              "   '2': {'f1-score': 0.6612244897959183,\n",
              "    'precision': 0.6090225563909775,\n",
              "    'recall': 0.7232142857142857,\n",
              "    'support': 112},\n",
              "   '3': {'f1-score': 0.576923076923077,\n",
              "    'precision': 0.5660377358490566,\n",
              "    'recall': 0.5882352941176471,\n",
              "    'support': 51},\n",
              "   '5': {'f1-score': 0.6181818181818183,\n",
              "    'precision': 0.68,\n",
              "    'recall': 0.5666666666666667,\n",
              "    'support': 30},\n",
              "   'accuracy': 0.573170731707317,\n",
              "   'macro avg': {'f1-score': 0.46188372595041444,\n",
              "    'precision': 0.5126787251146735,\n",
              "    'recall': 0.45298811416458484,\n",
              "    'support': 246},\n",
              "   'weighted avg': {'f1-score': 0.555201016542201,\n",
              "    'precision': 0.5556373340545719,\n",
              "    'recall': 0.573170731707317,\n",
              "    'support': 246}}},\n",
              " '5797046': {'Decision Tree': {'0': {'f1-score': 0.4,\n",
              "    'precision': 0.5,\n",
              "    'recall': 0.3333333333333333,\n",
              "    'support': 33},\n",
              "   '1': {'f1-score': 0.10810810810810811,\n",
              "    'precision': 0.1111111111111111,\n",
              "    'recall': 0.10526315789473684,\n",
              "    'support': 19},\n",
              "   '2': {'f1-score': 0.7151162790697675,\n",
              "    'precision': 0.6795580110497238,\n",
              "    'recall': 0.754601226993865,\n",
              "    'support': 163},\n",
              "   '3': {'f1-score': 0.4489795918367347,\n",
              "    'precision': 0.44,\n",
              "    'recall': 0.4583333333333333,\n",
              "    'support': 48},\n",
              "   '5': {'f1-score': 0.4651162790697675,\n",
              "    'precision': 0.5128205128205128,\n",
              "    'recall': 0.425531914893617,\n",
              "    'support': 47},\n",
              "   'accuracy': 0.5741935483870968,\n",
              "   'macro avg': {'f1-score': 0.42746405161687556,\n",
              "    'precision': 0.44869792699626954,\n",
              "    'recall': 0.4154125932897771,\n",
              "    'support': 310},\n",
              "   'weighted avg': {'f1-score': 0.5652564292479628,\n",
              "    'precision': 0.5632310677896135,\n",
              "    'recall': 0.5741935483870968,\n",
              "    'support': 310}},\n",
              "  'K-Neighbours': {'0': {'f1-score': 0.24390243902439024,\n",
              "    'precision': 0.625,\n",
              "    'recall': 0.15151515151515152,\n",
              "    'support': 33},\n",
              "   '1': {'f1-score': 0.0, 'precision': 0.0, 'recall': 0.0, 'support': 19},\n",
              "   '2': {'f1-score': 0.7324675324675325,\n",
              "    'precision': 0.6351351351351351,\n",
              "    'recall': 0.8650306748466258,\n",
              "    'support': 163},\n",
              "   '3': {'f1-score': 0.3, 'precision': 0.375, 'recall': 0.25, 'support': 48},\n",
              "   '5': {'f1-score': 0.5894736842105263,\n",
              "    'precision': 0.5833333333333334,\n",
              "    'recall': 0.5957446808510638,\n",
              "    'support': 47},\n",
              "   'accuracy': 0.6,\n",
              "   'macro avg': {'f1-score': 0.3731687311404898,\n",
              "    'precision': 0.4436936936936937,\n",
              "    'recall': 0.3724581014425682,\n",
              "    'support': 310},\n",
              "   'weighted avg': {'f1-score': 0.5469233917351852,\n",
              "    'precision': 0.5469957861086893,\n",
              "    'recall': 0.6,\n",
              "    'support': 310}},\n",
              "  'SVC': {'0': {'f1-score': 0.28571428571428575,\n",
              "    'precision': 0.6666666666666666,\n",
              "    'recall': 0.18181818181818182,\n",
              "    'support': 33},\n",
              "   '1': {'f1-score': 0.0, 'precision': 0.0, 'recall': 0.0, 'support': 19},\n",
              "   '2': {'f1-score': 0.7362924281984334,\n",
              "    'precision': 0.6409090909090909,\n",
              "    'recall': 0.8650306748466258,\n",
              "    'support': 163},\n",
              "   '3': {'f1-score': 0.32558139534883723,\n",
              "    'precision': 0.3684210526315789,\n",
              "    'recall': 0.2916666666666667,\n",
              "    'support': 48},\n",
              "   '5': {'f1-score': 0.5777777777777778,\n",
              "    'precision': 0.6046511627906976,\n",
              "    'recall': 0.5531914893617021,\n",
              "    'support': 47},\n",
              "   'accuracy': 0.603225806451613,\n",
              "   'macro avg': {'f1-score': 0.38507317740786684,\n",
              "    'precision': 0.45612959459960684,\n",
              "    'recall': 0.3783414025386353,\n",
              "    'support': 310},\n",
              "   'weighted avg': {'f1-score': 0.5555732250232769,\n",
              "    'precision': 0.5566806354698722,\n",
              "    'recall': 0.603225806451613,\n",
              "    'support': 310}}},\n",
              " '6220552': {'Decision Tree': {'0': {'f1-score': 0.3125,\n",
              "    'precision': 0.38461538461538464,\n",
              "    'recall': 0.2631578947368421,\n",
              "    'support': 19},\n",
              "   '1': {'f1-score': 0.08,\n",
              "    'precision': 0.1,\n",
              "    'recall': 0.06666666666666667,\n",
              "    'support': 15},\n",
              "   '2': {'f1-score': 0.7500000000000001,\n",
              "    'precision': 0.727810650887574,\n",
              "    'recall': 0.7735849056603774,\n",
              "    'support': 159},\n",
              "   '3': {'f1-score': 0.6906474820143885,\n",
              "    'precision': 0.676056338028169,\n",
              "    'recall': 0.7058823529411765,\n",
              "    'support': 68},\n",
              "   '5': {'f1-score': 0.7169811320754716,\n",
              "    'precision': 0.7307692307692307,\n",
              "    'recall': 0.7037037037037037,\n",
              "    'support': 54},\n",
              "   'accuracy': 0.6825396825396826,\n",
              "   'macro avg': {'f1-score': 0.5100257228179721,\n",
              "    'precision': 0.5238503208600717,\n",
              "    'recall': 0.5025991047417533,\n",
              "    'support': 315},\n",
              "   'weighted avg': {'f1-score': 0.6732333647906472,\n",
              "    'precision': 0.6665490642738746,\n",
              "    'recall': 0.6825396825396826,\n",
              "    'support': 315}},\n",
              "  'K-Neighbours': {'0': {'f1-score': 0.0,\n",
              "    'precision': 0.0,\n",
              "    'recall': 0.0,\n",
              "    'support': 19},\n",
              "   '1': {'f1-score': 0.0, 'precision': 0.0, 'recall': 0.0, 'support': 15},\n",
              "   '2': {'f1-score': 0.7047146401985112,\n",
              "    'precision': 0.5819672131147541,\n",
              "    'recall': 0.8930817610062893,\n",
              "    'support': 159},\n",
              "   '3': {'f1-score': 0.4485981308411215,\n",
              "    'precision': 0.6153846153846154,\n",
              "    'recall': 0.35294117647058826,\n",
              "    'support': 68},\n",
              "   '5': {'f1-score': 0.4941176470588235,\n",
              "    'precision': 0.6774193548387096,\n",
              "    'recall': 0.3888888888888889,\n",
              "    'support': 54},\n",
              "   'accuracy': 0.5936507936507937,\n",
              "   'macro avg': {'f1-score': 0.32948608361969123,\n",
              "    'precision': 0.3749542366676158,\n",
              "    'recall': 0.3269823652731533,\n",
              "    'support': 315},\n",
              "   'weighted avg': {'f1-score': 0.5372592178728127,\n",
              "    'precision': 0.5427288441037781,\n",
              "    'recall': 0.5936507936507937,\n",
              "    'support': 315}},\n",
              "  'SVC': {'0': {'f1-score': 0.08333333333333333,\n",
              "    'precision': 0.2,\n",
              "    'recall': 0.05263157894736842,\n",
              "    'support': 19},\n",
              "   '1': {'f1-score': 0.0, 'precision': 0.0, 'recall': 0.0, 'support': 15},\n",
              "   '2': {'f1-score': 0.6935866983372921,\n",
              "    'precision': 0.5572519083969466,\n",
              "    'recall': 0.9182389937106918,\n",
              "    'support': 159},\n",
              "   '3': {'f1-score': 0.45999999999999996,\n",
              "    'precision': 0.71875,\n",
              "    'recall': 0.3382352941176471,\n",
              "    'support': 68},\n",
              "   '5': {'f1-score': 0.3188405797101449,\n",
              "    'precision': 0.7333333333333333,\n",
              "    'recall': 0.2037037037037037,\n",
              "    'support': 54},\n",
              "   'accuracy': 0.5746031746031746,\n",
              "   'macro avg': {'f1-score': 0.3111521222761541,\n",
              "    'precision': 0.441867048346056,\n",
              "    'recall': 0.30256191409588223,\n",
              "    'support': 315},\n",
              "   'weighted avg': {'f1-score': 0.5090825703914621,\n",
              "    'precision': 0.5742160426511571,\n",
              "    'recall': 0.5746031746031746,\n",
              "    'support': 315}}},\n",
              " '759667': {'Decision Tree': {'0': {'f1-score': 0.4,\n",
              "    'precision': 0.5,\n",
              "    'recall': 0.3333333333333333,\n",
              "    'support': 9},\n",
              "   '1': {'f1-score': 0.0, 'precision': 0.0, 'recall': 0.0, 'support': 6},\n",
              "   '2': {'f1-score': 0.711111111111111,\n",
              "    'precision': 0.7272727272727273,\n",
              "    'recall': 0.6956521739130435,\n",
              "    'support': 46},\n",
              "   '3': {'f1-score': 0.8095238095238095,\n",
              "    'precision': 0.85,\n",
              "    'recall': 0.7727272727272727,\n",
              "    'support': 44},\n",
              "   '5': {'f1-score': 0.7387387387387386,\n",
              "    'precision': 0.6949152542372882,\n",
              "    'recall': 0.7884615384615384,\n",
              "    'support': 52},\n",
              "   'accuracy': 0.7006369426751592,\n",
              "   'macro avg': {'f1-score': 0.5318747318747319,\n",
              "    'precision': 0.5544375963020031,\n",
              "    'recall': 0.5180348636870376,\n",
              "    'support': 157},\n",
              "   'weighted avg': {'f1-score': 0.7028316760800837,\n",
              "    'precision': 0.7101282718145505,\n",
              "    'recall': 0.7006369426751592,\n",
              "    'support': 157}},\n",
              "  'K-Neighbours': {'0': {'f1-score': 0.30769230769230765,\n",
              "    'precision': 0.5,\n",
              "    'recall': 0.2222222222222222,\n",
              "    'support': 9},\n",
              "   '1': {'f1-score': 0.0, 'precision': 0.0, 'recall': 0.0, 'support': 6},\n",
              "   '2': {'f1-score': 0.5416666666666667,\n",
              "    'precision': 0.52,\n",
              "    'recall': 0.5652173913043478,\n",
              "    'support': 46},\n",
              "   '3': {'f1-score': 0.4444444444444444,\n",
              "    'precision': 0.5714285714285714,\n",
              "    'recall': 0.36363636363636365,\n",
              "    'support': 44},\n",
              "   '5': {'f1-score': 0.5691056910569106,\n",
              "    'precision': 0.49295774647887325,\n",
              "    'recall': 0.6730769230769231,\n",
              "    'support': 52},\n",
              "   'accuracy': 0.5031847133757962,\n",
              "   'macro avg': {'f1-score': 0.3725818219720659,\n",
              "    'precision': 0.4168772635814889,\n",
              "    'recall': 0.36483058004797136,\n",
              "    'support': 157},\n",
              "   'weighted avg': {'f1-score': 0.48939457914912327,\n",
              "    'precision': 0.5044373245844493,\n",
              "    'recall': 0.5031847133757962,\n",
              "    'support': 157}},\n",
              "  'SVC': {'0': {'f1-score': 0.0,\n",
              "    'precision': 0.0,\n",
              "    'recall': 0.0,\n",
              "    'support': 9},\n",
              "   '1': {'f1-score': 0.0, 'precision': 0.0, 'recall': 0.0, 'support': 6},\n",
              "   '2': {'f1-score': 0.5263157894736841,\n",
              "    'precision': 0.6666666666666666,\n",
              "    'recall': 0.43478260869565216,\n",
              "    'support': 46},\n",
              "   '3': {'f1-score': 0.5979381443298969,\n",
              "    'precision': 0.5471698113207547,\n",
              "    'recall': 0.6590909090909091,\n",
              "    'support': 44},\n",
              "   '5': {'f1-score': 0.6349206349206349,\n",
              "    'precision': 0.5405405405405406,\n",
              "    'recall': 0.7692307692307693,\n",
              "    'support': 52},\n",
              "   'accuracy': 0.5668789808917197,\n",
              "   'macro avg': {'f1-score': 0.3518349137448432,\n",
              "    'precision': 0.3508754037055924,\n",
              "    'recall': 0.3726208574034661,\n",
              "    'support': 157},\n",
              "   'weighted avg': {'f1-score': 0.5320743801412608,\n",
              "    'precision': 0.527708576260433,\n",
              "    'recall': 0.5668789808917197,\n",
              "    'support': 157}}},\n",
              " '7749105': {'Decision Tree': {'0': {'f1-score': 0.9285714285714286,\n",
              "    'precision': 0.9285714285714286,\n",
              "    'recall': 0.9285714285714286,\n",
              "    'support': 14},\n",
              "   '1': {'f1-score': 0.0, 'precision': 0.0, 'recall': 0.0, 'support': 3},\n",
              "   '2': {'f1-score': 0.7222222222222222,\n",
              "    'precision': 0.6190476190476191,\n",
              "    'recall': 0.8666666666666667,\n",
              "    'support': 15},\n",
              "   '3': {'f1-score': 0.5,\n",
              "    'precision': 1.0,\n",
              "    'recall': 0.3333333333333333,\n",
              "    'support': 9},\n",
              "   'accuracy': 0.7073170731707317,\n",
              "   'macro avg': {'f1-score': 0.5376984126984127,\n",
              "    'precision': 0.6369047619047619,\n",
              "    'recall': 0.5321428571428571,\n",
              "    'support': 41},\n",
              "   'weighted avg': {'f1-score': 0.6910569105691058,\n",
              "    'precision': 0.7630662020905923,\n",
              "    'recall': 0.7073170731707317,\n",
              "    'support': 41}},\n",
              "  'K-Neighbours': {'0': {'f1-score': 0.7741935483870968,\n",
              "    'precision': 0.7058823529411765,\n",
              "    'recall': 0.8571428571428571,\n",
              "    'support': 14},\n",
              "   '1': {'f1-score': 0.0, 'precision': 0.0, 'recall': 0.0, 'support': 3},\n",
              "   '2': {'f1-score': 0.5641025641025641,\n",
              "    'precision': 0.4583333333333333,\n",
              "    'recall': 0.7333333333333333,\n",
              "    'support': 15},\n",
              "   '3': {'f1-score': 0.0, 'precision': 0.0, 'recall': 0.0, 'support': 9},\n",
              "   'accuracy': 0.5609756097560976,\n",
              "   'macro avg': {'f1-score': 0.3345740281224152,\n",
              "    'precision': 0.29105392156862747,\n",
              "    'recall': 0.3976190476190476,\n",
              "    'support': 41},\n",
              "   'weighted avg': {'f1-score': 0.470737759486776,\n",
              "    'precision': 0.4087159253945481,\n",
              "    'recall': 0.5609756097560976,\n",
              "    'support': 41}},\n",
              "  'SVC': {'0': {'f1-score': 0.6000000000000001,\n",
              "    'precision': 0.5625,\n",
              "    'recall': 0.6428571428571429,\n",
              "    'support': 14},\n",
              "   '1': {'f1-score': 0.0, 'precision': 0.0, 'recall': 0.0, 'support': 3},\n",
              "   '2': {'f1-score': 0.6666666666666667,\n",
              "    'precision': 0.5416666666666666,\n",
              "    'recall': 0.8666666666666667,\n",
              "    'support': 15},\n",
              "   '3': {'f1-score': 0.0, 'precision': 0.0, 'recall': 0.0, 'support': 9},\n",
              "   'accuracy': 0.5365853658536586,\n",
              "   'macro avg': {'f1-score': 0.3166666666666667,\n",
              "    'precision': 0.27604166666666663,\n",
              "    'recall': 0.3773809523809524,\n",
              "    'support': 41},\n",
              "   'weighted avg': {'f1-score': 0.44878048780487817,\n",
              "    'precision': 0.3902439024390244,\n",
              "    'recall': 0.5365853658536586,\n",
              "    'support': 41}}},\n",
              " '781756': {'Decision Tree': {'0': {'f1-score': 0.4727272727272728,\n",
              "    'precision': 0.41935483870967744,\n",
              "    'recall': 0.5416666666666666,\n",
              "    'support': 24},\n",
              "   '1': {'f1-score': 0.19642857142857142,\n",
              "    'precision': 0.21568627450980393,\n",
              "    'recall': 0.18032786885245902,\n",
              "    'support': 61},\n",
              "   '2': {'f1-score': 0.5808580858085809,\n",
              "    'precision': 0.5945945945945946,\n",
              "    'recall': 0.567741935483871,\n",
              "    'support': 155},\n",
              "   '3': {'f1-score': 0.13333333333333333,\n",
              "    'precision': 0.2,\n",
              "    'recall': 0.1,\n",
              "    'support': 10},\n",
              "   '5': {'f1-score': 0.5766871165644172,\n",
              "    'precision': 0.5280898876404494,\n",
              "    'recall': 0.6351351351351351,\n",
              "    'support': 74},\n",
              "   'accuracy': 0.49382716049382713,\n",
              "   'macro avg': {'f1-score': 0.3920068759724351,\n",
              "    'precision': 0.39154511909090506,\n",
              "    'recall': 0.40497432122762633,\n",
              "    'support': 324},\n",
              "   'weighted avg': {'f1-score': 0.48570611315440626,\n",
              "    'precision': 0.482908002227425,\n",
              "    'recall': 0.49382716049382713,\n",
              "    'support': 324}},\n",
              "  'K-Neighbours': {'0': {'f1-score': 0.4782608695652174,\n",
              "    'precision': 0.5,\n",
              "    'recall': 0.4583333333333333,\n",
              "    'support': 24},\n",
              "   '1': {'f1-score': 0.1794871794871795,\n",
              "    'precision': 0.4117647058823529,\n",
              "    'recall': 0.11475409836065574,\n",
              "    'support': 61},\n",
              "   '2': {'f1-score': 0.6666666666666666,\n",
              "    'precision': 0.5781990521327014,\n",
              "    'recall': 0.7870967741935484,\n",
              "    'support': 155},\n",
              "   '3': {'f1-score': 0.20000000000000004,\n",
              "    'precision': 0.2,\n",
              "    'recall': 0.2,\n",
              "    'support': 10},\n",
              "   '5': {'f1-score': 0.5797101449275363,\n",
              "    'precision': 0.625,\n",
              "    'recall': 0.5405405405405406,\n",
              "    'support': 74},\n",
              "   'accuracy': 0.5617283950617284,\n",
              "   'macro avg': {'f1-score': 0.42082497212931996,\n",
              "    'precision': 0.4629927516030108,\n",
              "    'recall': 0.42014494928561563,\n",
              "    'support': 324},\n",
              "   'weighted avg': {'f1-score': 0.5267248854205375,\n",
              "    'precision': 0.540087963393186,\n",
              "    'recall': 0.5617283950617284,\n",
              "    'support': 324}},\n",
              "  'SVC': {'0': {'f1-score': 0.4680851063829787,\n",
              "    'precision': 0.4782608695652174,\n",
              "    'recall': 0.4583333333333333,\n",
              "    'support': 24},\n",
              "   '1': {'f1-score': 0.11267605633802817,\n",
              "    'precision': 0.4,\n",
              "    'recall': 0.06557377049180328,\n",
              "    'support': 61},\n",
              "   '2': {'f1-score': 0.6925207756232686,\n",
              "    'precision': 0.6067961165048543,\n",
              "    'recall': 0.8064516129032258,\n",
              "    'support': 155},\n",
              "   '3': {'f1-score': 0.0, 'precision': 0.0, 'recall': 0.0, 'support': 10},\n",
              "   '5': {'f1-score': 0.6496815286624203,\n",
              "    'precision': 0.6144578313253012,\n",
              "    'recall': 0.6891891891891891,\n",
              "    'support': 74},\n",
              "   'accuracy': 0.5895061728395061,\n",
              "   'macro avg': {'f1-score': 0.38459269340133917,\n",
              "    'precision': 0.4199029634790746,\n",
              "    'recall': 0.4039095811835103,\n",
              "    'support': 324},\n",
              "   'weighted avg': {'f1-score': 0.5355692448532005,\n",
              "    'precision': 0.5413627729811417,\n",
              "    'recall': 0.5895061728395061,\n",
              "    'support': 324}}},\n",
              " '8000685': {'Decision Tree': {'0': {'f1-score': 0.0,\n",
              "    'precision': 0.0,\n",
              "    'recall': 0.0,\n",
              "    'support': 11},\n",
              "   '1': {'f1-score': 0.12499999999999997,\n",
              "    'precision': 0.13636363636363635,\n",
              "    'recall': 0.11538461538461539,\n",
              "    'support': 26},\n",
              "   '2': {'f1-score': 0.7317073170731708,\n",
              "    'precision': 0.7258064516129032,\n",
              "    'recall': 0.7377049180327869,\n",
              "    'support': 183},\n",
              "   '3': {'f1-score': 0.7058823529411764,\n",
              "    'precision': 0.6,\n",
              "    'recall': 0.8571428571428571,\n",
              "    'support': 14},\n",
              "   '5': {'f1-score': 0.5783132530120482,\n",
              "    'precision': 0.5783132530120482,\n",
              "    'recall': 0.5783132530120482,\n",
              "    'support': 83},\n",
              "   'accuracy': 0.6246056782334385,\n",
              "   'macro avg': {'f1-score': 0.4281805846052791,\n",
              "    'precision': 0.4080966681977175,\n",
              "    'recall': 0.4577091287144615,\n",
              "    'support': 317},\n",
              "   'weighted avg': {'f1-score': 0.6152517096705574,\n",
              "    'precision': 0.6081010573836462,\n",
              "    'recall': 0.6246056782334385,\n",
              "    'support': 317}},\n",
              "  'K-Neighbours': {'0': {'f1-score': 0.13333333333333333,\n",
              "    'precision': 0.25,\n",
              "    'recall': 0.09090909090909091,\n",
              "    'support': 11},\n",
              "   '1': {'f1-score': 0.0, 'precision': 0.0, 'recall': 0.0, 'support': 26},\n",
              "   '2': {'f1-score': 0.7363636363636363,\n",
              "    'precision': 0.6303501945525292,\n",
              "    'recall': 0.8852459016393442,\n",
              "    'support': 183},\n",
              "   '3': {'f1-score': 0.5333333333333333,\n",
              "    'precision': 0.5,\n",
              "    'recall': 0.5714285714285714,\n",
              "    'support': 14},\n",
              "   '5': {'f1-score': 0.3252032520325203,\n",
              "    'precision': 0.5,\n",
              "    'recall': 0.24096385542168675,\n",
              "    'support': 83},\n",
              "   'accuracy': 0.6025236593059937,\n",
              "   'macro avg': {'f1-score': 0.3456467110125646,\n",
              "    'precision': 0.37607003891050583,\n",
              "    'recall': 0.35770948387973867,\n",
              "    'support': 317},\n",
              "   'weighted avg': {'f1-score': 0.5384219202100252,\n",
              "    'precision': 0.5255649388110815,\n",
              "    'recall': 0.6025236593059937,\n",
              "    'support': 317}},\n",
              "  'SVC': {'0': {'f1-score': 0.0,\n",
              "    'precision': 0.0,\n",
              "    'recall': 0.0,\n",
              "    'support': 11},\n",
              "   '1': {'f1-score': 0.0, 'precision': 0.0, 'recall': 0.0, 'support': 26},\n",
              "   '2': {'f1-score': 0.7246376811594203,\n",
              "    'precision': 0.6493506493506493,\n",
              "    'recall': 0.819672131147541,\n",
              "    'support': 183},\n",
              "   '3': {'f1-score': 0.7058823529411764,\n",
              "    'precision': 0.6,\n",
              "    'recall': 0.8571428571428571,\n",
              "    'support': 14},\n",
              "   '5': {'f1-score': 0.35714285714285715,\n",
              "    'precision': 0.43859649122807015,\n",
              "    'recall': 0.30120481927710846,\n",
              "    'support': 83},\n",
              "   'accuracy': 0.5899053627760252,\n",
              "   'macro avg': {'f1-score': 0.3575325782486908,\n",
              "    'precision': 0.33758942811574394,\n",
              "    'recall': 0.3956039615135013,\n",
              "    'support': 317},\n",
              "   'weighted avg': {'f1-score': 0.5430091663602761,\n",
              "    'precision': 0.5161977211454216,\n",
              "    'recall': 0.5899053627760252,\n",
              "    'support': 317}}},\n",
              " '8173033': {'Decision Tree': {'0': {'f1-score': 0.3333333333333333,\n",
              "    'precision': 0.3333333333333333,\n",
              "    'recall': 0.3333333333333333,\n",
              "    'support': 9},\n",
              "   '1': {'f1-score': 0.26229508196721313,\n",
              "    'precision': 0.2857142857142857,\n",
              "    'recall': 0.24242424242424243,\n",
              "    'support': 33},\n",
              "   '2': {'f1-score': 0.6713780918727915,\n",
              "    'precision': 0.6168831168831169,\n",
              "    'recall': 0.7364341085271318,\n",
              "    'support': 129},\n",
              "   '3': {'f1-score': 0.5263157894736842,\n",
              "    'precision': 0.5833333333333334,\n",
              "    'recall': 0.4794520547945205,\n",
              "    'support': 73},\n",
              "   '5': {'f1-score': 0.6131386861313869,\n",
              "    'precision': 0.6461538461538462,\n",
              "    'recall': 0.5833333333333334,\n",
              "    'support': 72},\n",
              "   'accuracy': 0.5791139240506329,\n",
              "   'macro avg': {'f1-score': 0.4812921965556818,\n",
              "    'precision': 0.49308358308358313,\n",
              "    'recall': 0.47499541448251226,\n",
              "    'support': 316},\n",
              "   'weighted avg': {'f1-score': 0.5722485746504651,\n",
              "    'precision': 0.5731421005155183,\n",
              "    'recall': 0.5791139240506329,\n",
              "    'support': 316}},\n",
              "  'K-Neighbours': {'0': {'f1-score': 0.0,\n",
              "    'precision': 0.0,\n",
              "    'recall': 0.0,\n",
              "    'support': 9},\n",
              "   '1': {'f1-score': 0.2727272727272727,\n",
              "    'precision': 0.5454545454545454,\n",
              "    'recall': 0.18181818181818182,\n",
              "    'support': 33},\n",
              "   '2': {'f1-score': 0.6796116504854369,\n",
              "    'precision': 0.5833333333333334,\n",
              "    'recall': 0.813953488372093,\n",
              "    'support': 129},\n",
              "   '3': {'f1-score': 0.45614035087719296,\n",
              "    'precision': 0.6341463414634146,\n",
              "    'recall': 0.3561643835616438,\n",
              "    'support': 73},\n",
              "   '5': {'f1-score': 0.6153846153846153,\n",
              "    'precision': 0.5714285714285714,\n",
              "    'recall': 0.6666666666666666,\n",
              "    'support': 72},\n",
              "   'accuracy': 0.5854430379746836,\n",
              "   'macro avg': {'f1-score': 0.4047727778949036,\n",
              "    'precision': 0.4668725583359731,\n",
              "    'recall': 0.40372054408371705,\n",
              "    'support': 316},\n",
              "   'weighted avg': {'f1-score': 0.5515058254251543,\n",
              "    'precision': 0.5717896837648304,\n",
              "    'recall': 0.5854430379746836,\n",
              "    'support': 316}},\n",
              "  'SVC': {'0': {'f1-score': 0.0,\n",
              "    'precision': 0.0,\n",
              "    'recall': 0.0,\n",
              "    'support': 9},\n",
              "   '1': {'f1-score': 0.25,\n",
              "    'precision': 0.7142857142857143,\n",
              "    'recall': 0.15151515151515152,\n",
              "    'support': 33},\n",
              "   '2': {'f1-score': 0.6645367412140575,\n",
              "    'precision': 0.5652173913043478,\n",
              "    'recall': 0.8062015503875969,\n",
              "    'support': 129},\n",
              "   '3': {'f1-score': 0.4444444444444445,\n",
              "    'precision': 0.5909090909090909,\n",
              "    'recall': 0.3561643835616438,\n",
              "    'support': 73},\n",
              "   '5': {'f1-score': 0.5751633986928105,\n",
              "    'precision': 0.5432098765432098,\n",
              "    'recall': 0.6111111111111112,\n",
              "    'support': 72},\n",
              "   'accuracy': 0.5664556962025317,\n",
              "   'macro avg': {'f1-score': 0.3868289168702625,\n",
              "    'precision': 0.4827244146084725,\n",
              "    'recall': 0.3849984393151007,\n",
              "    'support': 316},\n",
              "   'weighted avg': {'f1-score': 0.5311121796422159,\n",
              "    'precision': 0.5656074265733043,\n",
              "    'recall': 0.5664556962025317,\n",
              "    'support': 316}}},\n",
              " '844359': {'Decision Tree': {'0': {'f1-score': 0.3636363636363637,\n",
              "    'precision': 0.4444444444444444,\n",
              "    'recall': 0.3076923076923077,\n",
              "    'support': 26},\n",
              "   '1': {'f1-score': 0.22222222222222224,\n",
              "    'precision': 0.23076923076923078,\n",
              "    'recall': 0.21428571428571427,\n",
              "    'support': 28},\n",
              "   '2': {'f1-score': 0.6275862068965518,\n",
              "    'precision': 0.6148648648648649,\n",
              "    'recall': 0.6408450704225352,\n",
              "    'support': 142},\n",
              "   '3': {'f1-score': 0.375,\n",
              "    'precision': 0.375,\n",
              "    'recall': 0.375,\n",
              "    'support': 32},\n",
              "   '5': {'f1-score': 0.6338028169014085,\n",
              "    'precision': 0.6164383561643836,\n",
              "    'recall': 0.6521739130434783,\n",
              "    'support': 69},\n",
              "   'accuracy': 0.5454545454545454,\n",
              "   'macro avg': {'f1-score': 0.4444495219313092,\n",
              "    'precision': 0.4563033792485848,\n",
              "    'recall': 0.4379994010888071,\n",
              "    'support': 297},\n",
              "   'weighted avg': {'f1-score': 0.5404929408157415,\n",
              "    'precision': 0.5382564020311357,\n",
              "    'recall': 0.5454545454545454,\n",
              "    'support': 297}},\n",
              "  'K-Neighbours': {'0': {'f1-score': 0.43999999999999995,\n",
              "    'precision': 0.4583333333333333,\n",
              "    'recall': 0.4230769230769231,\n",
              "    'support': 26},\n",
              "   '1': {'f1-score': 0.10526315789473682,\n",
              "    'precision': 0.2,\n",
              "    'recall': 0.07142857142857142,\n",
              "    'support': 28},\n",
              "   '2': {'f1-score': 0.6646884272997031,\n",
              "    'precision': 0.5743589743589743,\n",
              "    'recall': 0.7887323943661971,\n",
              "    'support': 142},\n",
              "   '3': {'f1-score': 0.17391304347826086,\n",
              "    'precision': 0.2857142857142857,\n",
              "    'recall': 0.125,\n",
              "    'support': 32},\n",
              "   '5': {'f1-score': 0.5203252032520325,\n",
              "    'precision': 0.5925925925925926,\n",
              "    'recall': 0.463768115942029,\n",
              "    'support': 69},\n",
              "   'accuracy': 0.5420875420875421,\n",
              "   'macro avg': {'f1-score': 0.3808379663849466,\n",
              "    'precision': 0.42219983719983717,\n",
              "    'recall': 0.3744012009627441,\n",
              "    'support': 297},\n",
              "   'weighted avg': {'f1-score': 0.5058612172165153,\n",
              "    'precision': 0.5020450742672964,\n",
              "    'recall': 0.5420875420875421,\n",
              "    'support': 297}},\n",
              "  'SVC': {'0': {'f1-score': 0.35555555555555557,\n",
              "    'precision': 0.42105263157894735,\n",
              "    'recall': 0.3076923076923077,\n",
              "    'support': 26},\n",
              "   '1': {'f1-score': 0.0, 'precision': 0.0, 'recall': 0.0, 'support': 28},\n",
              "   '2': {'f1-score': 0.6568914956011729,\n",
              "    'precision': 0.5628140703517588,\n",
              "    'recall': 0.7887323943661971,\n",
              "    'support': 142},\n",
              "   '3': {'f1-score': 0.25531914893617025,\n",
              "    'precision': 0.4,\n",
              "    'recall': 0.1875,\n",
              "    'support': 32},\n",
              "   '5': {'f1-score': 0.5692307692307692,\n",
              "    'precision': 0.6065573770491803,\n",
              "    'recall': 0.5362318840579711,\n",
              "    'support': 69},\n",
              "   'accuracy': 0.5488215488215489,\n",
              "   'macro avg': {'f1-score': 0.3673993938647336,\n",
              "    'precision': 0.3980848157959773,\n",
              "    'recall': 0.36403131722329524,\n",
              "    'support': 297},\n",
              "   'weighted avg': {'f1-score': 0.5049500763053586,\n",
              "    'precision': 0.4899643953784371,\n",
              "    'recall': 0.5488215488215489,\n",
              "    'support': 297}}},\n",
              " '8530312': {'Decision Tree': {'0': {'f1-score': 0.18181818181818185,\n",
              "    'precision': 0.14705882352941177,\n",
              "    'recall': 0.23809523809523808,\n",
              "    'support': 21},\n",
              "   '1': {'f1-score': 0.28571428571428575,\n",
              "    'precision': 0.24242424242424243,\n",
              "    'recall': 0.34782608695652173,\n",
              "    'support': 23},\n",
              "   '2': {'f1-score': 0.7063197026022305,\n",
              "    'precision': 0.7142857142857143,\n",
              "    'recall': 0.6985294117647058,\n",
              "    'support': 136},\n",
              "   '3': {'f1-score': 0.33333333333333337,\n",
              "    'precision': 0.34375,\n",
              "    'recall': 0.3235294117647059,\n",
              "    'support': 34},\n",
              "   '5': {'f1-score': 0.6923076923076923,\n",
              "    'precision': 0.7682926829268293,\n",
              "    'recall': 0.63,\n",
              "    'support': 100},\n",
              "   'accuracy': 0.5796178343949044,\n",
              "   'macro avg': {'f1-score': 0.4398986391551448,\n",
              "    'precision': 0.4431622926332396,\n",
              "    'recall': 0.44759602971623436,\n",
              "    'support': 314},\n",
              "   'weighted avg': {'f1-score': 0.5955834156293514,\n",
              "    'precision': 0.6188650264503671,\n",
              "    'recall': 0.5796178343949044,\n",
              "    'support': 314}},\n",
              "  'K-Neighbours': {'0': {'f1-score': 0.24,\n",
              "    'precision': 0.75,\n",
              "    'recall': 0.14285714285714285,\n",
              "    'support': 21},\n",
              "   '1': {'f1-score': 0.07692307692307691,\n",
              "    'precision': 0.3333333333333333,\n",
              "    'recall': 0.043478260869565216,\n",
              "    'support': 23},\n",
              "   '2': {'f1-score': 0.710843373493976,\n",
              "    'precision': 0.6020408163265306,\n",
              "    'recall': 0.8676470588235294,\n",
              "    'support': 136},\n",
              "   '3': {'f1-score': 0.35294117647058826,\n",
              "    'precision': 0.5294117647058824,\n",
              "    'recall': 0.2647058823529412,\n",
              "    'support': 34},\n",
              "   '5': {'f1-score': 0.6804123711340208,\n",
              "    'precision': 0.7021276595744681,\n",
              "    'recall': 0.66,\n",
              "    'support': 100},\n",
              "   'accuracy': 0.6273885350318471,\n",
              "   'macro avg': {'f1-score': 0.41222399960433237,\n",
              "    'precision': 0.5833827147880429,\n",
              "    'recall': 0.39573766898063567,\n",
              "    'support': 314},\n",
              "   'weighted avg': {'f1-score': 0.5844750531140559,\n",
              "    'precision': 0.6162642791226803,\n",
              "    'recall': 0.6273885350318471,\n",
              "    'support': 314}},\n",
              "  'SVC': {'0': {'f1-score': 0.13333333333333333,\n",
              "    'precision': 0.2222222222222222,\n",
              "    'recall': 0.09523809523809523,\n",
              "    'support': 21},\n",
              "   '1': {'f1-score': 0.16,\n",
              "    'precision': 1.0,\n",
              "    'recall': 0.08695652173913043,\n",
              "    'support': 23},\n",
              "   '2': {'f1-score': 0.7165109034267912,\n",
              "    'precision': 0.6216216216216216,\n",
              "    'recall': 0.8455882352941176,\n",
              "    'support': 136},\n",
              "   '3': {'f1-score': 0.2608695652173913,\n",
              "    'precision': 0.5,\n",
              "    'recall': 0.17647058823529413,\n",
              "    'support': 34},\n",
              "   '5': {'f1-score': 0.6893203883495145,\n",
              "    'precision': 0.6698113207547169,\n",
              "    'recall': 0.71,\n",
              "    'support': 100},\n",
              "   'accuracy': 0.6242038216560509,\n",
              "   'macro avg': {'f1-score': 0.3920068380654061,\n",
              "    'precision': 0.6027310329197121,\n",
              "    'recall': 0.3828506881013275,\n",
              "    'support': 314},\n",
              "   'weighted avg': {'f1-score': 0.5787486844534597,\n",
              "    'precision': 0.6248036282887863,\n",
              "    'recall': 0.6242038216560509,\n",
              "    'support': 314}}},\n",
              " '8686948': {'Decision Tree': {'0': {'f1-score': 0.5,\n",
              "    'precision': 0.4444444444444444,\n",
              "    'recall': 0.5714285714285714,\n",
              "    'support': 14},\n",
              "   '1': {'f1-score': 0.09756097560975609,\n",
              "    'precision': 0.08,\n",
              "    'recall': 0.125,\n",
              "    'support': 16},\n",
              "   '2': {'f1-score': 0.7020648967551623,\n",
              "    'precision': 0.7300613496932515,\n",
              "    'recall': 0.6761363636363636,\n",
              "    'support': 176},\n",
              "   '3': {'f1-score': 0.6666666666666667,\n",
              "    'precision': 0.6730769230769231,\n",
              "    'recall': 0.660377358490566,\n",
              "    'support': 53},\n",
              "   '5': {'f1-score': 0.5217391304347826,\n",
              "    'precision': 0.5172413793103449,\n",
              "    'recall': 0.5263157894736842,\n",
              "    'support': 57},\n",
              "   'accuracy': 0.6139240506329114,\n",
              "   'macro avg': {'f1-score': 0.4976063338932735,\n",
              "    'precision': 0.4889648193049928,\n",
              "    'recall': 0.511851616605837,\n",
              "    'support': 316},\n",
              "   'weighted avg': {'f1-score': 0.6240407000214577,\n",
              "    'precision': 0.636547010481016,\n",
              "    'recall': 0.6139240506329114,\n",
              "    'support': 316}},\n",
              "  'K-Neighbours': {'0': {'f1-score': 0.4666666666666667,\n",
              "    'precision': 0.4375,\n",
              "    'recall': 0.5,\n",
              "    'support': 14},\n",
              "   '1': {'f1-score': 0.0, 'precision': 0.0, 'recall': 0.0, 'support': 16},\n",
              "   '2': {'f1-score': 0.6925064599483204,\n",
              "    'precision': 0.6350710900473934,\n",
              "    'recall': 0.7613636363636364,\n",
              "    'support': 176},\n",
              "   '3': {'f1-score': 0.6060606060606061,\n",
              "    'precision': 0.6521739130434783,\n",
              "    'recall': 0.5660377358490566,\n",
              "    'support': 53},\n",
              "   '5': {'f1-score': 0.38775510204081637,\n",
              "    'precision': 0.4634146341463415,\n",
              "    'recall': 0.3333333333333333,\n",
              "    'support': 57},\n",
              "   'accuracy': 0.6012658227848101,\n",
              "   'macro avg': {'f1-score': 0.4305977669432819,\n",
              "    'precision': 0.43763192744744267,\n",
              "    'recall': 0.43214694110920526,\n",
              "    'support': 316},\n",
              "   'weighted avg': {'f1-score': 0.5779674785499253,\n",
              "    'precision': 0.5660676056518578,\n",
              "    'recall': 0.6012658227848101,\n",
              "    'support': 316}},\n",
              "  'SVC': {'0': {'f1-score': 0.4615384615384615,\n",
              "    'precision': 0.5,\n",
              "    'recall': 0.42857142857142855,\n",
              "    'support': 14},\n",
              "   '1': {'f1-score': 0.0, 'precision': 0.0, 'recall': 0.0, 'support': 16},\n",
              "   '2': {'f1-score': 0.7204030226700252,\n",
              "    'precision': 0.6470588235294118,\n",
              "    'recall': 0.8125,\n",
              "    'support': 176},\n",
              "   '3': {'f1-score': 0.6185567010309277,\n",
              "    'precision': 0.6818181818181818,\n",
              "    'recall': 0.5660377358490566,\n",
              "    'support': 53},\n",
              "   '5': {'f1-score': 0.4615384615384615,\n",
              "    'precision': 0.6176470588235294,\n",
              "    'recall': 0.3684210526315789,\n",
              "    'support': 57},\n",
              "   'accuracy': 0.6329113924050633,\n",
              "   'macro avg': {'f1-score': 0.45240732935557526,\n",
              "    'precision': 0.48930481283422456,\n",
              "    'recall': 0.4351060434104128,\n",
              "    'support': 316},\n",
              "   'weighted avg': {'f1-score': 0.6086824933980834,\n",
              "    'precision': 0.6083056928179789,\n",
              "    'recall': 0.6329113924050633,\n",
              "    'support': 316}}},\n",
              " '8692923': {'Decision Tree': {'0': {'f1-score': 0.6875,\n",
              "    'precision': 0.6470588235294118,\n",
              "    'recall': 0.7333333333333333,\n",
              "    'support': 30},\n",
              "   '1': {'f1-score': 0.13333333333333333,\n",
              "    'precision': 0.1111111111111111,\n",
              "    'recall': 0.16666666666666666,\n",
              "    'support': 12},\n",
              "   '2': {'f1-score': 0.7092198581560285,\n",
              "    'precision': 0.7194244604316546,\n",
              "    'recall': 0.6993006993006993,\n",
              "    'support': 143},\n",
              "   '3': {'f1-score': 0.4842105263157895,\n",
              "    'precision': 0.4791666666666667,\n",
              "    'recall': 0.48936170212765956,\n",
              "    'support': 47},\n",
              "   '5': {'f1-score': 0.7074829931972789,\n",
              "    'precision': 0.7428571428571429,\n",
              "    'recall': 0.6753246753246753,\n",
              "    'support': 77},\n",
              "   'accuracy': 0.6440129449838188,\n",
              "   'macro avg': {'f1-score': 0.544349342200486,\n",
              "    'precision': 0.5399236409191974,\n",
              "    'recall': 0.5527974153506069,\n",
              "    'support': 309},\n",
              "   'weighted avg': {'f1-score': 0.6500890774412449,\n",
              "    'precision': 0.6580699974572028,\n",
              "    'recall': 0.6440129449838188,\n",
              "    'support': 309}},\n",
              "  'K-Neighbours': {'0': {'f1-score': 0.25,\n",
              "    'precision': 0.3333333333333333,\n",
              "    'recall': 0.2,\n",
              "    'support': 30},\n",
              "   '1': {'f1-score': 0.0, 'precision': 0.0, 'recall': 0.0, 'support': 12},\n",
              "   '2': {'f1-score': 0.6606606606606608,\n",
              "    'precision': 0.5789473684210527,\n",
              "    'recall': 0.7692307692307693,\n",
              "    'support': 143},\n",
              "   '3': {'f1-score': 0.23684210526315788,\n",
              "    'precision': 0.3103448275862069,\n",
              "    'recall': 0.19148936170212766,\n",
              "    'support': 47},\n",
              "   '5': {'f1-score': 0.4563758389261745,\n",
              "    'precision': 0.4722222222222222,\n",
              "    'recall': 0.44155844155844154,\n",
              "    'support': 77},\n",
              "   'accuracy': 0.5145631067961165,\n",
              "   'macro avg': {'f1-score': 0.3207757209699986,\n",
              "    'precision': 0.338969550312563,\n",
              "    'recall': 0.3204557144982677,\n",
              "    'support': 309},\n",
              "   'weighted avg': {'f1-score': 0.479763731453587,\n",
              "    'precision': 0.4651676106533119,\n",
              "    'recall': 0.5145631067961165,\n",
              "    'support': 309}},\n",
              "  'SVC': {'0': {'f1-score': 0.4827586206896552,\n",
              "    'precision': 0.5,\n",
              "    'recall': 0.4666666666666667,\n",
              "    'support': 30},\n",
              "   '1': {'f1-score': 0.0, 'precision': 0.0, 'recall': 0.0, 'support': 12},\n",
              "   '2': {'f1-score': 0.6958904109589041,\n",
              "    'precision': 0.5720720720720721,\n",
              "    'recall': 0.8881118881118881,\n",
              "    'support': 143},\n",
              "   '3': {'f1-score': 0.2985074626865672,\n",
              "    'precision': 0.5,\n",
              "    'recall': 0.2127659574468085,\n",
              "    'support': 47},\n",
              "   '5': {'f1-score': 0.48275862068965514,\n",
              "    'precision': 0.717948717948718,\n",
              "    'recall': 0.36363636363636365,\n",
              "    'support': 77},\n",
              "   'accuracy': 0.5792880258899676,\n",
              "   'macro avg': {'f1-score': 0.39198302300495635,\n",
              "    'precision': 0.45800415800415806,\n",
              "    'recall': 0.38623617517234543,\n",
              "    'support': 309},\n",
              "   'weighted avg': {'f1-score': 0.5346192619002752,\n",
              "    'precision': 0.5682471119364324,\n",
              "    'recall': 0.5792880258899676,\n",
              "    'support': 309}}},\n",
              " '9106476': {'Decision Tree': {'0': {'f1-score': 0.43636363636363634,\n",
              "    'precision': 0.5454545454545454,\n",
              "    'recall': 0.36363636363636365,\n",
              "    'support': 33},\n",
              "   '1': {'f1-score': 0.1081081081081081,\n",
              "    'precision': 0.125,\n",
              "    'recall': 0.09523809523809523,\n",
              "    'support': 21},\n",
              "   '2': {'f1-score': 0.7228260869565217,\n",
              "    'precision': 0.6820512820512821,\n",
              "    'recall': 0.7687861271676301,\n",
              "    'support': 173},\n",
              "   '3': {'f1-score': 0.6065573770491804,\n",
              "    'precision': 0.6166666666666667,\n",
              "    'recall': 0.5967741935483871,\n",
              "    'support': 62},\n",
              "   '5': {'f1-score': 0.5,\n",
              "    'precision': 0.5416666666666666,\n",
              "    'recall': 0.4642857142857143,\n",
              "    'support': 28},\n",
              "   'accuracy': 0.6214511041009464,\n",
              "   'macro avg': {'f1-score': 0.47477104169548934,\n",
              "    'precision': 0.5021678321678322,\n",
              "    'recall': 0.45774409877523803,\n",
              "    'support': 317},\n",
              "   'weighted avg': {'f1-score': 0.6098603807280685,\n",
              "    'precision': 0.6057409204885547,\n",
              "    'recall': 0.6214511041009464,\n",
              "    'support': 317}},\n",
              "  'K-Neighbours': {'0': {'f1-score': 0.26666666666666666,\n",
              "    'precision': 0.5,\n",
              "    'recall': 0.18181818181818182,\n",
              "    'support': 33},\n",
              "   '1': {'f1-score': 0.0, 'precision': 0.0, 'recall': 0.0, 'support': 21},\n",
              "   '2': {'f1-score': 0.7639902676399027,\n",
              "    'precision': 0.6596638655462185,\n",
              "    'recall': 0.9075144508670521,\n",
              "    'support': 173},\n",
              "   '3': {'f1-score': 0.4807692307692307,\n",
              "    'precision': 0.5952380952380952,\n",
              "    'recall': 0.4032258064516129,\n",
              "    'support': 62},\n",
              "   '5': {'f1-score': 0.4528301886792452,\n",
              "    'precision': 0.48,\n",
              "    'recall': 0.42857142857142855,\n",
              "    'support': 28},\n",
              "   'accuracy': 0.6309148264984227,\n",
              "   'macro avg': {'f1-score': 0.3928512707510091,\n",
              "    'precision': 0.44698039215686275,\n",
              "    'recall': 0.38422597354165505,\n",
              "    'support': 317},\n",
              "   'weighted avg': {'f1-score': 0.5787295075470484,\n",
              "    'precision': 0.5708725887831474,\n",
              "    'recall': 0.6309148264984227,\n",
              "    'support': 317}},\n",
              "  'SVC': {'0': {'f1-score': 0.2978723404255319,\n",
              "    'precision': 0.5,\n",
              "    'recall': 0.21212121212121213,\n",
              "    'support': 33},\n",
              "   '1': {'f1-score': 0.0, 'precision': 0.0, 'recall': 0.0, 'support': 21},\n",
              "   '2': {'f1-score': 0.7293577981651376,\n",
              "    'precision': 0.6045627376425855,\n",
              "    'recall': 0.9190751445086706,\n",
              "    'support': 173},\n",
              "   '3': {'f1-score': 0.4318181818181818,\n",
              "    'precision': 0.7307692307692307,\n",
              "    'recall': 0.3064516129032258,\n",
              "    'support': 62},\n",
              "   '5': {'f1-score': 0.2926829268292683,\n",
              "    'precision': 0.46153846153846156,\n",
              "    'recall': 0.21428571428571427,\n",
              "    'support': 28},\n",
              "   'accuracy': 0.6025236593059937,\n",
              "   'macro avg': {'f1-score': 0.35034624944762394,\n",
              "    'precision': 0.4593740859900556,\n",
              "    'recall': 0.3303867367637645,\n",
              "    'support': 317},\n",
              "   'weighted avg': {'f1-score': 0.5393581562793632,\n",
              "    'precision': 0.5656786209556357,\n",
              "    'recall': 0.6025236593059937,\n",
              "    'support': 317}}},\n",
              " '9961348': {'Decision Tree': {'0': {'f1-score': 0.2105263157894737,\n",
              "    'precision': 0.2857142857142857,\n",
              "    'recall': 0.16666666666666666,\n",
              "    'support': 24},\n",
              "   '1': {'f1-score': 0.05128205128205128,\n",
              "    'precision': 0.058823529411764705,\n",
              "    'recall': 0.045454545454545456,\n",
              "    'support': 22},\n",
              "   '2': {'f1-score': 0.7158671586715868,\n",
              "    'precision': 0.6783216783216783,\n",
              "    'recall': 0.7578125,\n",
              "    'support': 128},\n",
              "   '3': {'f1-score': 0.5,\n",
              "    'precision': 0.5555555555555556,\n",
              "    'recall': 0.45454545454545453,\n",
              "    'support': 11},\n",
              "   '5': {'f1-score': 0.6481481481481481,\n",
              "    'precision': 0.6363636363636364,\n",
              "    'recall': 0.660377358490566,\n",
              "    'support': 53},\n",
              "   'accuracy': 0.5966386554621849,\n",
              "   'macro avg': {'f1-score': 0.42516473477825195,\n",
              "    'precision': 0.4429557370733842,\n",
              "    'recall': 0.4169713050314465,\n",
              "    'support': 238},\n",
              "   'weighted avg': {'f1-score': 0.5784188439872583,\n",
              "    'precision': 0.5664488200326064,\n",
              "    'recall': 0.5966386554621849,\n",
              "    'support': 238}},\n",
              "  'K-Neighbours': {'0': {'f1-score': 0.07142857142857142,\n",
              "    'precision': 0.25,\n",
              "    'recall': 0.041666666666666664,\n",
              "    'support': 24},\n",
              "   '1': {'f1-score': 0.08,\n",
              "    'precision': 0.3333333333333333,\n",
              "    'recall': 0.045454545454545456,\n",
              "    'support': 22},\n",
              "   '2': {'f1-score': 0.7148014440433214,\n",
              "    'precision': 0.6644295302013423,\n",
              "    'recall': 0.7734375,\n",
              "    'support': 128},\n",
              "   '3': {'f1-score': 0.4799999999999999,\n",
              "    'precision': 0.42857142857142855,\n",
              "    'recall': 0.5454545454545454,\n",
              "    'support': 11},\n",
              "   '5': {'f1-score': 0.5619834710743802,\n",
              "    'precision': 0.5,\n",
              "    'recall': 0.6415094339622641,\n",
              "    'support': 53},\n",
              "   'accuracy': 0.592436974789916,\n",
              "   'macro avg': {'f1-score': 0.3816426973092546,\n",
              "    'precision': 0.43526685842122087,\n",
              "    'recall': 0.40950453830760436,\n",
              "    'support': 238},\n",
              "   'weighted avg': {'f1-score': 0.5463613215074495,\n",
              "    'precision': 0.5445151214848355,\n",
              "    'recall': 0.592436974789916,\n",
              "    'support': 238}},\n",
              "  'SVC': {'0': {'f1-score': 0.0,\n",
              "    'precision': 0.0,\n",
              "    'recall': 0.0,\n",
              "    'support': 24},\n",
              "   '1': {'f1-score': 0.0, 'precision': 0.0, 'recall': 0.0, 'support': 22},\n",
              "   '2': {'f1-score': 0.7173252279635258,\n",
              "    'precision': 0.5870646766169154,\n",
              "    'recall': 0.921875,\n",
              "    'support': 128},\n",
              "   '3': {'f1-score': 0.0, 'precision': 0.0, 'recall': 0.0, 'support': 11},\n",
              "   '5': {'f1-score': 0.5777777777777777,\n",
              "    'precision': 0.7027027027027027,\n",
              "    'recall': 0.49056603773584906,\n",
              "    'support': 53},\n",
              "   'accuracy': 0.6050420168067226,\n",
              "   'macro avg': {'f1-score': 0.2590206011482607,\n",
              "    'precision': 0.25795347586392364,\n",
              "    'recall': 0.2824882075471698,\n",
              "    'support': 238},\n",
              "   'weighted avg': {'f1-score': 0.5144531571493846,\n",
              "    'precision': 0.47221647836222025,\n",
              "    'recall': 0.6050420168067226,\n",
              "    'support': 238}}}}"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 146
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "qhC0V1ngPbV7",
        "outputId": "6bdfe0b8-f776-4c05-9e6a-27ed1c36779b"
      },
      "source": [
        "binary_classification_report_dict"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{'1066528': {'Decision Tree': {'0': {'f1-score': 0.4074074074074074,\n",
              "    'precision': 0.36666666666666664,\n",
              "    'recall': 0.4583333333333333,\n",
              "    'support': 48},\n",
              "   '1': {'f1-score': 0.8764478764478765,\n",
              "    'precision': 0.8972332015810277,\n",
              "    'recall': 0.8566037735849057,\n",
              "    'support': 265},\n",
              "   'accuracy': 0.7955271565495208,\n",
              "   'macro avg': {'f1-score': 0.641927641927642,\n",
              "    'precision': 0.6319499341238471,\n",
              "    'recall': 0.6574685534591195,\n",
              "    'support': 313},\n",
              "   'weighted avg': {'f1-score': 0.804518347649338,\n",
              "    'precision': 0.8158683655558222,\n",
              "    'recall': 0.7955271565495208,\n",
              "    'support': 313}},\n",
              "  'K-Neighbours': {'0': {'f1-score': 0.3,\n",
              "    'precision': 0.375,\n",
              "    'recall': 0.25,\n",
              "    'support': 48},\n",
              "   '1': {'f1-score': 0.8974358974358975,\n",
              "    'precision': 0.8718861209964412,\n",
              "    'recall': 0.9245283018867925,\n",
              "    'support': 265},\n",
              "   'accuracy': 0.8210862619808307,\n",
              "   'macro avg': {'f1-score': 0.5987179487179487,\n",
              "    'precision': 0.6234430604982206,\n",
              "    'recall': 0.5872641509433962,\n",
              "    'support': 313},\n",
              "   'weighted avg': {'f1-score': 0.8058163348898173,\n",
              "    'precision': 0.7956863324730253,\n",
              "    'recall': 0.8210862619808307,\n",
              "    'support': 313}},\n",
              "  'SVC': {'0': {'f1-score': 0.19672131147540983,\n",
              "    'precision': 0.46153846153846156,\n",
              "    'recall': 0.125,\n",
              "    'support': 48},\n",
              "   '1': {'f1-score': 0.9132743362831858,\n",
              "    'precision': 0.86,\n",
              "    'recall': 0.9735849056603774,\n",
              "    'support': 265},\n",
              "   'accuracy': 0.8434504792332268,\n",
              "   'macro avg': {'f1-score': 0.5549978238792979,\n",
              "    'precision': 0.6607692307692308,\n",
              "    'recall': 0.5492924528301887,\n",
              "    'support': 313},\n",
              "   'weighted avg': {'f1-score': 0.8033876104340699,\n",
              "    'precision': 0.7988940771688376,\n",
              "    'recall': 0.8434504792332268,\n",
              "    'support': 313}}},\n",
              " '1360686': {'Decision Tree': {'0': {'f1-score': 0.4333333333333333,\n",
              "    'precision': 0.41935483870967744,\n",
              "    'recall': 0.4482758620689655,\n",
              "    'support': 29},\n",
              "   '1': {'f1-score': 0.9384057971014491,\n",
              "    'precision': 0.9418181818181818,\n",
              "    'recall': 0.9350180505415162,\n",
              "    'support': 277},\n",
              "   'accuracy': 0.8888888888888888,\n",
              "   'macro avg': {'f1-score': 0.6858695652173912,\n",
              "    'precision': 0.6805865102639296,\n",
              "    'recall': 0.6916469563052409,\n",
              "    'support': 306},\n",
              "   'weighted avg': {'f1-score': 0.890539452495974,\n",
              "    'precision': 0.8923036819811014,\n",
              "    'recall': 0.8888888888888888,\n",
              "    'support': 306}},\n",
              "  'K-Neighbours': {'0': {'f1-score': 0.4285714285714286,\n",
              "    'precision': 0.6923076923076923,\n",
              "    'recall': 0.3103448275862069,\n",
              "    'support': 29},\n",
              "   '1': {'f1-score': 0.9578947368421052,\n",
              "    'precision': 0.931740614334471,\n",
              "    'recall': 0.9855595667870036,\n",
              "    'support': 277},\n",
              "   'accuracy': 0.9215686274509803,\n",
              "   'macro avg': {'f1-score': 0.693233082706767,\n",
              "    'precision': 0.8120241533210817,\n",
              "    'recall': 0.6479521971866052,\n",
              "    'support': 306},\n",
              "   'weighted avg': {'f1-score': 0.9077301095876947,\n",
              "    'precision': 0.9090492589789919,\n",
              "    'recall': 0.9215686274509803,\n",
              "    'support': 306}},\n",
              "  'SVC': {'0': {'f1-score': 0.4090909090909091,\n",
              "    'precision': 0.6,\n",
              "    'recall': 0.3103448275862069,\n",
              "    'support': 29},\n",
              "   '1': {'f1-score': 0.9542253521126761,\n",
              "    'precision': 0.9312714776632303,\n",
              "    'recall': 0.9783393501805054,\n",
              "    'support': 277},\n",
              "   'accuracy': 0.9150326797385621,\n",
              "   'macro avg': {'f1-score': 0.6816581306017926,\n",
              "    'precision': 0.7656357388316151,\n",
              "    'recall': 0.6443420888833562,\n",
              "    'support': 306},\n",
              "   'weighted avg': {'f1-score': 0.9025622839831622,\n",
              "    'precision': 0.8998764683422051,\n",
              "    'recall': 0.9150326797385621,\n",
              "    'support': 306}}},\n",
              " '1449548': {'Decision Tree': {'0': {'f1-score': 0.3870967741935484,\n",
              "    'precision': 0.48,\n",
              "    'recall': 0.32432432432432434,\n",
              "    'support': 37},\n",
              "   '1': {'f1-score': 0.9330985915492958,\n",
              "    'precision': 0.9137931034482759,\n",
              "    'recall': 0.9532374100719424,\n",
              "    'support': 278},\n",
              "   'accuracy': 0.8793650793650793,\n",
              "   'macro avg': {'f1-score': 0.660097682871422,\n",
              "    'precision': 0.6968965517241379,\n",
              "    'recall': 0.6387808671981334,\n",
              "    'support': 315},\n",
              "   'weighted avg': {'f1-score': 0.8689650447487793,\n",
              "    'precision': 0.862839627805145,\n",
              "    'recall': 0.8793650793650793,\n",
              "    'support': 315}},\n",
              "  'K-Neighbours': {'0': {'f1-score': 0.28571428571428575,\n",
              "    'precision': 0.5833333333333334,\n",
              "    'recall': 0.1891891891891892,\n",
              "    'support': 37},\n",
              "   '1': {'f1-score': 0.9397590361445782,\n",
              "    'precision': 0.900990099009901,\n",
              "    'recall': 0.9820143884892086,\n",
              "    'support': 278},\n",
              "   'accuracy': 0.8888888888888888,\n",
              "   'macro avg': {'f1-score': 0.612736660929432,\n",
              "    'precision': 0.7421617161716172,\n",
              "    'recall': 0.5856017888391989,\n",
              "    'support': 315},\n",
              "   'weighted avg': {'f1-score': 0.862934732125782,\n",
              "    'precision': 0.8636780344701137,\n",
              "    'recall': 0.8888888888888888,\n",
              "    'support': 315}},\n",
              "  'SVC': {'0': {'f1-score': 0.2448979591836735,\n",
              "    'precision': 0.5,\n",
              "    'recall': 0.16216216216216217,\n",
              "    'support': 37},\n",
              "   '1': {'f1-score': 0.93631669535284,\n",
              "    'precision': 0.8976897689768977,\n",
              "    'recall': 0.9784172661870504,\n",
              "    'support': 278},\n",
              "   'accuracy': 0.8825396825396825,\n",
              "   'macro avg': {'f1-score': 0.5906073272682567,\n",
              "    'precision': 0.6988448844884488,\n",
              "    'recall': 0.5702897141746063,\n",
              "    'support': 315},\n",
              "   'weighted avg': {'f1-score': 0.8551024311043981,\n",
              "    'precision': 0.850977002462151,\n",
              "    'recall': 0.8825396825396825,\n",
              "    'support': 315}}},\n",
              " '1455390': {'Decision Tree': {'0': {'f1-score': 0.12000000000000002,\n",
              "    'precision': 0.16666666666666666,\n",
              "    'recall': 0.09375,\n",
              "    'support': 32},\n",
              "   '1': {'f1-score': 0.9243986254295532,\n",
              "    'precision': 0.9026845637583892,\n",
              "    'recall': 0.9471830985915493,\n",
              "    'support': 284},\n",
              "   'accuracy': 0.8607594936708861,\n",
              "   'macro avg': {'f1-score': 0.5221993127147766,\n",
              "    'precision': 0.5346756152125279,\n",
              "    'recall': 0.5204665492957746,\n",
              "    'support': 316},\n",
              "   'weighted avg': {'f1-score': 0.8429405367784591,\n",
              "    'precision': 0.8281511058250502,\n",
              "    'recall': 0.8607594936708861,\n",
              "    'support': 316}},\n",
              "  'K-Neighbours': {'0': {'f1-score': 0.06060606060606061,\n",
              "    'precision': 1.0,\n",
              "    'recall': 0.03125,\n",
              "    'support': 32},\n",
              "   '1': {'f1-score': 0.9482470784641069,\n",
              "    'precision': 0.9015873015873016,\n",
              "    'recall': 1.0,\n",
              "    'support': 284},\n",
              "   'accuracy': 0.9018987341772152,\n",
              "   'macro avg': {'f1-score': 0.5044265695350838,\n",
              "    'precision': 0.9507936507936507,\n",
              "    'recall': 0.515625,\n",
              "    'support': 316},\n",
              "   'weighted avg': {'f1-score': 0.8583593804531656,\n",
              "    'precision': 0.9115531444645368,\n",
              "    'recall': 0.9018987341772152,\n",
              "    'support': 316}},\n",
              "  'SVC': {'0': {'f1-score': 0.06060606060606061,\n",
              "    'precision': 1.0,\n",
              "    'recall': 0.03125,\n",
              "    'support': 32},\n",
              "   '1': {'f1-score': 0.9482470784641069,\n",
              "    'precision': 0.9015873015873016,\n",
              "    'recall': 1.0,\n",
              "    'support': 284},\n",
              "   'accuracy': 0.9018987341772152,\n",
              "   'macro avg': {'f1-score': 0.5044265695350838,\n",
              "    'precision': 0.9507936507936507,\n",
              "    'recall': 0.515625,\n",
              "    'support': 316},\n",
              "   'weighted avg': {'f1-score': 0.8583593804531656,\n",
              "    'precision': 0.9115531444645368,\n",
              "    'recall': 0.9018987341772152,\n",
              "    'support': 316}}},\n",
              " '1818471': {'Decision Tree': {'0': {'f1-score': 0.0,\n",
              "    'precision': 0.0,\n",
              "    'recall': 0.0,\n",
              "    'support': 3},\n",
              "   '1': {'f1-score': 0.9888357256778308,\n",
              "    'precision': 0.9904153354632588,\n",
              "    'recall': 0.9872611464968153,\n",
              "    'support': 314},\n",
              "   'accuracy': 0.9779179810725552,\n",
              "   'macro avg': {'f1-score': 0.4944178628389154,\n",
              "    'precision': 0.4952076677316294,\n",
              "    'recall': 0.49363057324840764,\n",
              "    'support': 317},\n",
              "   'weighted avg': {'f1-score': 0.9794776588733087,\n",
              "    'precision': 0.981042319670231,\n",
              "    'recall': 0.9779179810725552,\n",
              "    'support': 317}},\n",
              "  'K-Neighbours': {'0': {'f1-score': 0.0,\n",
              "    'precision': 0.0,\n",
              "    'recall': 0.0,\n",
              "    'support': 3},\n",
              "   '1': {'f1-score': 0.9952456418383517,\n",
              "    'precision': 0.9905362776025236,\n",
              "    'recall': 1.0,\n",
              "    'support': 314},\n",
              "   'accuracy': 0.9905362776025236,\n",
              "   'macro avg': {'f1-score': 0.49762282091917587,\n",
              "    'precision': 0.4952681388012618,\n",
              "    'recall': 0.5,\n",
              "    'support': 317},\n",
              "   'weighted avg': {'f1-score': 0.9858269133666955,\n",
              "    'precision': 0.9811621172466638,\n",
              "    'recall': 0.9905362776025236,\n",
              "    'support': 317}},\n",
              "  'SVC': {'0': {'f1-score': 0.0,\n",
              "    'precision': 0.0,\n",
              "    'recall': 0.0,\n",
              "    'support': 3},\n",
              "   '1': {'f1-score': 0.9952456418383517,\n",
              "    'precision': 0.9905362776025236,\n",
              "    'recall': 1.0,\n",
              "    'support': 314},\n",
              "   'accuracy': 0.9905362776025236,\n",
              "   'macro avg': {'f1-score': 0.49762282091917587,\n",
              "    'precision': 0.4952681388012618,\n",
              "    'recall': 0.5,\n",
              "    'support': 317},\n",
              "   'weighted avg': {'f1-score': 0.9858269133666955,\n",
              "    'precision': 0.9811621172466638,\n",
              "    'recall': 0.9905362776025236,\n",
              "    'support': 317}}},\n",
              " '2598705': {'Decision Tree': {'0': {'f1-score': 0.0,\n",
              "    'precision': 0.0,\n",
              "    'recall': 0.0,\n",
              "    'support': 4},\n",
              "   '1': {'f1-score': 0.9855072463768115,\n",
              "    'precision': 0.9870967741935484,\n",
              "    'recall': 0.9839228295819936,\n",
              "    'support': 311},\n",
              "   'accuracy': 0.9714285714285714,\n",
              "   'macro avg': {'f1-score': 0.49275362318840576,\n",
              "    'precision': 0.4935483870967742,\n",
              "    'recall': 0.4919614147909968,\n",
              "    'support': 315},\n",
              "   'weighted avg': {'f1-score': 0.9729928686450424,\n",
              "    'precision': 0.9745622119815669,\n",
              "    'recall': 0.9714285714285714,\n",
              "    'support': 315}},\n",
              "  'K-Neighbours': {'0': {'f1-score': 0.0,\n",
              "    'precision': 0.0,\n",
              "    'recall': 0.0,\n",
              "    'support': 4},\n",
              "   '1': {'f1-score': 0.9936102236421726,\n",
              "    'precision': 0.9873015873015873,\n",
              "    'recall': 1.0,\n",
              "    'support': 311},\n",
              "   'accuracy': 0.9873015873015873,\n",
              "   'macro avg': {'f1-score': 0.4968051118210863,\n",
              "    'precision': 0.4936507936507937,\n",
              "    'recall': 0.5,\n",
              "    'support': 315},\n",
              "   'weighted avg': {'f1-score': 0.980992950961002,\n",
              "    'precision': 0.9747644242882338,\n",
              "    'recall': 0.9873015873015873,\n",
              "    'support': 315}},\n",
              "  'SVC': {'0': {'f1-score': 0.0,\n",
              "    'precision': 0.0,\n",
              "    'recall': 0.0,\n",
              "    'support': 4},\n",
              "   '1': {'f1-score': 0.9936102236421726,\n",
              "    'precision': 0.9873015873015873,\n",
              "    'recall': 1.0,\n",
              "    'support': 311},\n",
              "   'accuracy': 0.9873015873015873,\n",
              "   'macro avg': {'f1-score': 0.4968051118210863,\n",
              "    'precision': 0.4936507936507937,\n",
              "    'recall': 0.5,\n",
              "    'support': 315},\n",
              "   'weighted avg': {'f1-score': 0.980992950961002,\n",
              "    'precision': 0.9747644242882338,\n",
              "    'recall': 0.9873015873015873,\n",
              "    'support': 315}}},\n",
              " '2638030': {'Decision Tree': {'0': {'f1-score': 0.6436781609195403,\n",
              "    'precision': 0.6222222222222222,\n",
              "    'recall': 0.6666666666666666,\n",
              "    'support': 42},\n",
              "   '1': {'f1-score': 0.9424860853432282,\n",
              "    'precision': 0.9477611940298507,\n",
              "    'recall': 0.9372693726937269,\n",
              "    'support': 271},\n",
              "   'accuracy': 0.9009584664536742,\n",
              "   'macro avg': {'f1-score': 0.7930821231313843,\n",
              "    'precision': 0.7849917081260365,\n",
              "    'recall': 0.8019680196801968,\n",
              "    'support': 313},\n",
              "   'weighted avg': {'f1-score': 0.9023904533119346,\n",
              "    'precision': 0.9040786482920858,\n",
              "    'recall': 0.9009584664536742,\n",
              "    'support': 313}},\n",
              "  'K-Neighbours': {'0': {'f1-score': 0.5070422535211268,\n",
              "    'precision': 0.6206896551724138,\n",
              "    'recall': 0.42857142857142855,\n",
              "    'support': 42},\n",
              "   '1': {'f1-score': 0.9369369369369369,\n",
              "    'precision': 0.9154929577464789,\n",
              "    'recall': 0.959409594095941,\n",
              "    'support': 271},\n",
              "   'accuracy': 0.8881789137380192,\n",
              "   'macro avg': {'f1-score': 0.7219895952290318,\n",
              "    'precision': 0.7680913064594463,\n",
              "    'recall': 0.6939905113336847,\n",
              "    'support': 313},\n",
              "   'weighted avg': {'f1-score': 0.8792513883635694,\n",
              "    'precision': 0.8759346871135372,\n",
              "    'recall': 0.8881789137380192,\n",
              "    'support': 313}},\n",
              "  'SVC': {'0': {'f1-score': 0.41176470588235287,\n",
              "    'precision': 0.5384615384615384,\n",
              "    'recall': 0.3333333333333333,\n",
              "    'support': 42},\n",
              "   '1': {'f1-score': 0.9283154121863799,\n",
              "    'precision': 0.9024390243902439,\n",
              "    'recall': 0.955719557195572,\n",
              "    'support': 271},\n",
              "   'accuracy': 0.8722044728434505,\n",
              "   'macro avg': {'f1-score': 0.6700400590343664,\n",
              "    'precision': 0.7204502814258912,\n",
              "    'recall': 0.6445264452644527,\n",
              "    'support': 313},\n",
              "   'weighted avg': {'f1-score': 0.8590018988804083,\n",
              "    'precision': 0.8535985949685005,\n",
              "    'recall': 0.8722044728434505,\n",
              "    'support': 313}}},\n",
              " '3509524': {'Decision Tree': {'0': {'f1-score': 0.6818181818181818,\n",
              "    'precision': 0.625,\n",
              "    'recall': 0.75,\n",
              "    'support': 20},\n",
              "   '1': {'f1-score': 0.939655172413793,\n",
              "    'precision': 0.956140350877193,\n",
              "    'recall': 0.923728813559322,\n",
              "    'support': 118},\n",
              "   'accuracy': 0.8985507246376812,\n",
              "   'macro avg': {'f1-score': 0.8107366771159874,\n",
              "    'precision': 0.7905701754385965,\n",
              "    'recall': 0.836864406779661,\n",
              "    'support': 138},\n",
              "   'weighted avg': {'f1-score': 0.9022874926173277,\n",
              "    'precision': 0.9081489956775998,\n",
              "    'recall': 0.8985507246376812,\n",
              "    'support': 138}},\n",
              "  'K-Neighbours': {'0': {'f1-score': 0.48275862068965514,\n",
              "    'precision': 0.7777777777777778,\n",
              "    'recall': 0.35,\n",
              "    'support': 20},\n",
              "   '1': {'f1-score': 0.9392712550607288,\n",
              "    'precision': 0.8992248062015504,\n",
              "    'recall': 0.9830508474576272,\n",
              "    'support': 118},\n",
              "   'accuracy': 0.8913043478260869,\n",
              "   'macro avg': {'f1-score': 0.711014937875192,\n",
              "    'precision': 0.8385012919896642,\n",
              "    'recall': 0.6665254237288136,\n",
              "    'support': 138},\n",
              "   'weighted avg': {'f1-score': 0.8731100037026022,\n",
              "    'precision': 0.8816237875894095,\n",
              "    'recall': 0.8913043478260869,\n",
              "    'support': 138}},\n",
              "  'SVC': {'0': {'f1-score': 0.0,\n",
              "    'precision': 0.0,\n",
              "    'recall': 0.0,\n",
              "    'support': 20},\n",
              "   '1': {'f1-score': 0.921875,\n",
              "    'precision': 0.855072463768116,\n",
              "    'recall': 1.0,\n",
              "    'support': 118},\n",
              "   'accuracy': 0.855072463768116,\n",
              "   'macro avg': {'f1-score': 0.4609375,\n",
              "    'precision': 0.427536231884058,\n",
              "    'recall': 0.5,\n",
              "    'support': 138},\n",
              "   'weighted avg': {'f1-score': 0.7882699275362319,\n",
              "    'precision': 0.731148918294476,\n",
              "    'recall': 0.855072463768116,\n",
              "    'support': 138}}},\n",
              " '3997827': {'Decision Tree': {'0': {'f1-score': 0.25,\n",
              "    'precision': 0.2,\n",
              "    'recall': 0.3333333333333333,\n",
              "    'support': 6},\n",
              "   '1': {'f1-score': 0.9805825242718447,\n",
              "    'precision': 0.9869706840390879,\n",
              "    'recall': 0.9742765273311897,\n",
              "    'support': 311},\n",
              "   'accuracy': 0.9621451104100947,\n",
              "   'macro avg': {'f1-score': 0.6152912621359223,\n",
              "    'precision': 0.593485342019544,\n",
              "    'recall': 0.6538049303322615,\n",
              "    'support': 317},\n",
              "   'weighted avg': {'f1-score': 0.9667544638755322,\n",
              "    'precision': 0.972075339861692,\n",
              "    'recall': 0.9621451104100947,\n",
              "    'support': 317}},\n",
              "  'K-Neighbours': {'0': {'f1-score': 0.0,\n",
              "    'precision': 0.0,\n",
              "    'recall': 0.0,\n",
              "    'support': 6},\n",
              "   '1': {'f1-score': 0.9872204472843451,\n",
              "    'precision': 0.9809523809523809,\n",
              "    'recall': 0.9935691318327974,\n",
              "    'support': 311},\n",
              "   'accuracy': 0.9747634069400631,\n",
              "   'macro avg': {'f1-score': 0.49361022364217255,\n",
              "    'precision': 0.49047619047619045,\n",
              "    'recall': 0.4967845659163987,\n",
              "    'support': 317},\n",
              "   'weighted avg': {'f1-score': 0.9685348867679222,\n",
              "    'precision': 0.9623854589154273,\n",
              "    'recall': 0.9747634069400631,\n",
              "    'support': 317}},\n",
              "  'SVC': {'0': {'f1-score': 0.0,\n",
              "    'precision': 0.0,\n",
              "    'recall': 0.0,\n",
              "    'support': 6},\n",
              "   '1': {'f1-score': 0.9904458598726115,\n",
              "    'precision': 0.9810725552050473,\n",
              "    'recall': 1.0,\n",
              "    'support': 311},\n",
              "   'accuracy': 0.9810725552050473,\n",
              "   'macro avg': {'f1-score': 0.49522292993630573,\n",
              "    'precision': 0.49053627760252366,\n",
              "    'recall': 0.5,\n",
              "    'support': 317},\n",
              "   'weighted avg': {'f1-score': 0.9716992505374831,\n",
              "    'precision': 0.9625033585765606,\n",
              "    'recall': 0.9810725552050473,\n",
              "    'support': 317}}},\n",
              " '4018081': {'Decision Tree': {'0': {'f1-score': 0.6470588235294118,\n",
              "    'precision': 0.7586206896551724,\n",
              "    'recall': 0.5641025641025641,\n",
              "    'support': 39},\n",
              "   '1': {'f1-score': 0.9083969465648855,\n",
              "    'precision': 0.875,\n",
              "    'recall': 0.9444444444444444,\n",
              "    'support': 126},\n",
              "   'accuracy': 0.8545454545454545,\n",
              "   'macro avg': {'f1-score': 0.7777278850471486,\n",
              "    'precision': 0.8168103448275862,\n",
              "    'recall': 0.7542735042735043,\n",
              "    'support': 165},\n",
              "   'weighted avg': {'f1-score': 0.8466261174837736,\n",
              "    'precision': 0.8474921630094043,\n",
              "    'recall': 0.8545454545454545,\n",
              "    'support': 165}},\n",
              "  'K-Neighbours': {'0': {'f1-score': 0.53125,\n",
              "    'precision': 0.68,\n",
              "    'recall': 0.4358974358974359,\n",
              "    'support': 39},\n",
              "   '1': {'f1-score': 0.887218045112782,\n",
              "    'precision': 0.8428571428571429,\n",
              "    'recall': 0.9365079365079365,\n",
              "    'support': 126},\n",
              "   'accuracy': 0.8181818181818182,\n",
              "   'macro avg': {'f1-score': 0.709234022556391,\n",
              "    'precision': 0.7614285714285715,\n",
              "    'recall': 0.6862026862026862,\n",
              "    'support': 165},\n",
              "   'weighted avg': {'f1-score': 0.8030801435406698,\n",
              "    'precision': 0.8043636363636364,\n",
              "    'recall': 0.8181818181818182,\n",
              "    'support': 165}},\n",
              "  'SVC': {'0': {'f1-score': 0.0,\n",
              "    'precision': 0.0,\n",
              "    'recall': 0.0,\n",
              "    'support': 39},\n",
              "   '1': {'f1-score': 0.865979381443299,\n",
              "    'precision': 0.7636363636363637,\n",
              "    'recall': 1.0,\n",
              "    'support': 126},\n",
              "   'accuracy': 0.7636363636363637,\n",
              "   'macro avg': {'f1-score': 0.4329896907216495,\n",
              "    'precision': 0.38181818181818183,\n",
              "    'recall': 0.5,\n",
              "    'support': 165},\n",
              "   'weighted avg': {'f1-score': 0.6612933458294283,\n",
              "    'precision': 0.5831404958677686,\n",
              "    'recall': 0.7636363636363637,\n",
              "    'support': 165}}},\n",
              " '4314139': {'Decision Tree': {'0': {'f1-score': 0.2388059701492537,\n",
              "    'precision': 0.2222222222222222,\n",
              "    'recall': 0.25806451612903225,\n",
              "    'support': 31},\n",
              "   '1': {'f1-score': 0.9103690685413005,\n",
              "    'precision': 0.9184397163120568,\n",
              "    'recall': 0.9024390243902439,\n",
              "    'support': 287},\n",
              "   'accuracy': 0.839622641509434,\n",
              "   'macro avg': {'f1-score': 0.5745875193452771,\n",
              "    'precision': 0.5703309692671394,\n",
              "    'recall': 0.5802517702596381,\n",
              "    'support': 318},\n",
              "   'weighted avg': {'f1-score': 0.8449022256162898,\n",
              "    'precision': 0.8505694574542428,\n",
              "    'recall': 0.839622641509434,\n",
              "    'support': 318}},\n",
              "  'K-Neighbours': {'0': {'f1-score': 0.1714285714285714,\n",
              "    'precision': 0.75,\n",
              "    'recall': 0.0967741935483871,\n",
              "    'support': 31},\n",
              "   '1': {'f1-score': 0.9517470881863561,\n",
              "    'precision': 0.910828025477707,\n",
              "    'recall': 0.9965156794425087,\n",
              "    'support': 287},\n",
              "   'accuracy': 0.9088050314465409,\n",
              "   'macro avg': {'f1-score': 0.5615878298074638,\n",
              "    'precision': 0.8304140127388535,\n",
              "    'recall': 0.5466449364954479,\n",
              "    'support': 318},\n",
              "   'weighted avg': {'f1-score': 0.8756783019615405,\n",
              "    'precision': 0.8951498217361695,\n",
              "    'recall': 0.9088050314465409,\n",
              "    'support': 318}},\n",
              "  'SVC': {'0': {'f1-score': 0.1714285714285714,\n",
              "    'precision': 0.75,\n",
              "    'recall': 0.0967741935483871,\n",
              "    'support': 31},\n",
              "   '1': {'f1-score': 0.9517470881863561,\n",
              "    'precision': 0.910828025477707,\n",
              "    'recall': 0.9965156794425087,\n",
              "    'support': 287},\n",
              "   'accuracy': 0.9088050314465409,\n",
              "   'macro avg': {'f1-score': 0.5615878298074638,\n",
              "    'precision': 0.8304140127388535,\n",
              "    'recall': 0.5466449364954479,\n",
              "    'support': 318},\n",
              "   'weighted avg': {'f1-score': 0.8756783019615405,\n",
              "    'precision': 0.8951498217361695,\n",
              "    'recall': 0.9088050314465409,\n",
              "    'support': 318}}},\n",
              " '4426783': {'Decision Tree': {'0': {'f1-score': 0.40816326530612246,\n",
              "    'precision': 0.35714285714285715,\n",
              "    'recall': 0.47619047619047616,\n",
              "    'support': 21},\n",
              "   '1': {'f1-score': 0.9515859766277128,\n",
              "    'precision': 0.9628378378378378,\n",
              "    'recall': 0.9405940594059405,\n",
              "    'support': 303},\n",
              "   'accuracy': 0.9104938271604939,\n",
              "   'macro avg': {'f1-score': 0.6798746209669176,\n",
              "    'precision': 0.6599903474903475,\n",
              "    'recall': 0.7083922677982084,\n",
              "    'support': 324},\n",
              "   'weighted avg': {'f1-score': 0.9163641342272394,\n",
              "    'precision': 0.9235798298298298,\n",
              "    'recall': 0.9104938271604939,\n",
              "    'support': 324}},\n",
              "  'K-Neighbours': {'0': {'f1-score': 0.0,\n",
              "    'precision': 0.0,\n",
              "    'recall': 0.0,\n",
              "    'support': 21},\n",
              "   '1': {'f1-score': 0.9665071770334929,\n",
              "    'precision': 0.9351851851851852,\n",
              "    'recall': 1.0,\n",
              "    'support': 303},\n",
              "   'accuracy': 0.9351851851851852,\n",
              "   'macro avg': {'f1-score': 0.48325358851674644,\n",
              "    'precision': 0.4675925925925926,\n",
              "    'recall': 0.5,\n",
              "    'support': 324},\n",
              "   'weighted avg': {'f1-score': 0.9038631933368776,\n",
              "    'precision': 0.8745713305898493,\n",
              "    'recall': 0.9351851851851852,\n",
              "    'support': 324}},\n",
              "  'SVC': {'0': {'f1-score': 0.0,\n",
              "    'precision': 0.0,\n",
              "    'recall': 0.0,\n",
              "    'support': 21},\n",
              "   '1': {'f1-score': 0.9665071770334929,\n",
              "    'precision': 0.9351851851851852,\n",
              "    'recall': 1.0,\n",
              "    'support': 303},\n",
              "   'accuracy': 0.9351851851851852,\n",
              "   'macro avg': {'f1-score': 0.48325358851674644,\n",
              "    'precision': 0.4675925925925926,\n",
              "    'recall': 0.5,\n",
              "    'support': 324},\n",
              "   'weighted avg': {'f1-score': 0.9038631933368776,\n",
              "    'precision': 0.8745713305898493,\n",
              "    'recall': 0.9351851851851852,\n",
              "    'support': 324}}},\n",
              " '46343': {'Decision Tree': {'0': {'f1-score': 0.5797101449275363,\n",
              "    'precision': 0.5714285714285714,\n",
              "    'recall': 0.5882352941176471,\n",
              "    'support': 34},\n",
              "   '1': {'f1-score': 0.9023569023569022,\n",
              "    'precision': 0.9054054054054054,\n",
              "    'recall': 0.8993288590604027,\n",
              "    'support': 149},\n",
              "   'accuracy': 0.8415300546448088,\n",
              "   'macro avg': {'f1-score': 0.7410335236422192,\n",
              "    'precision': 0.7384169884169884,\n",
              "    'recall': 0.7437820765890248,\n",
              "    'support': 183},\n",
              "   'weighted avg': {'f1-score': 0.8424116031623752,\n",
              "    'precision': 0.8433550646665399,\n",
              "    'recall': 0.8415300546448088,\n",
              "    'support': 183}},\n",
              "  'K-Neighbours': {'0': {'f1-score': 0.5084745762711865,\n",
              "    'precision': 0.6,\n",
              "    'recall': 0.4411764705882353,\n",
              "    'support': 34},\n",
              "   '1': {'f1-score': 0.9055374592833876,\n",
              "    'precision': 0.879746835443038,\n",
              "    'recall': 0.9328859060402684,\n",
              "    'support': 149},\n",
              "   'accuracy': 0.8415300546448088,\n",
              "   'macro avg': {'f1-score': 0.707006017777287,\n",
              "    'precision': 0.7398734177215189,\n",
              "    'recall': 0.6870311883142519,\n",
              "    'support': 183},\n",
              "   'weighted avg': {'f1-score': 0.8317662132592629,\n",
              "    'precision': 0.8277720135574462,\n",
              "    'recall': 0.8415300546448088,\n",
              "    'support': 183}},\n",
              "  'SVC': {'0': {'f1-score': 0.5333333333333333,\n",
              "    'precision': 0.6153846153846154,\n",
              "    'recall': 0.47058823529411764,\n",
              "    'support': 34},\n",
              "   '1': {'f1-score': 0.9084967320261438,\n",
              "    'precision': 0.8853503184713376,\n",
              "    'recall': 0.9328859060402684,\n",
              "    'support': 149},\n",
              "   'accuracy': 0.8469945355191257,\n",
              "   'macro avg': {'f1-score': 0.7209150326797386,\n",
              "    'precision': 0.7503674669279765,\n",
              "    'recall': 0.701737070667193,\n",
              "    'support': 183},\n",
              "   'weighted avg': {'f1-score': 0.8387942426515232,\n",
              "    'precision': 0.8351927561492145,\n",
              "    'recall': 0.8469945355191257,\n",
              "    'support': 183}}},\n",
              " '5132496': {'Decision Tree': {'0': {'f1-score': 0.5714285714285714,\n",
              "    'precision': 0.45454545454545453,\n",
              "    'recall': 0.7692307692307693,\n",
              "    'support': 13},\n",
              "   '1': {'f1-score': 0.945054945054945,\n",
              "    'precision': 0.9772727272727273,\n",
              "    'recall': 0.9148936170212766,\n",
              "    'support': 141},\n",
              "   'accuracy': 0.9025974025974026,\n",
              "   'macro avg': {'f1-score': 0.7582417582417582,\n",
              "    'precision': 0.7159090909090909,\n",
              "    'recall': 0.8420621931260229,\n",
              "    'support': 154},\n",
              "   'weighted avg': {'f1-score': 0.9135150563721991,\n",
              "    'precision': 0.93314639905549,\n",
              "    'recall': 0.9025974025974026,\n",
              "    'support': 154}},\n",
              "  'K-Neighbours': {'0': {'f1-score': 0.4444444444444444,\n",
              "    'precision': 0.8,\n",
              "    'recall': 0.3076923076923077,\n",
              "    'support': 13},\n",
              "   '1': {'f1-score': 0.9655172413793103,\n",
              "    'precision': 0.9395973154362416,\n",
              "    'recall': 0.9929078014184397,\n",
              "    'support': 141},\n",
              "   'accuracy': 0.935064935064935,\n",
              "   'macro avg': {'f1-score': 0.7049808429118773,\n",
              "    'precision': 0.8697986577181208,\n",
              "    'recall': 0.6503000545553737,\n",
              "    'support': 154},\n",
              "   'weighted avg': {'f1-score': 0.9215305767029904,\n",
              "    'precision': 0.9278131264708447,\n",
              "    'recall': 0.935064935064935,\n",
              "    'support': 154}},\n",
              "  'SVC': {'0': {'f1-score': 0.0,\n",
              "    'precision': 0.0,\n",
              "    'recall': 0.0,\n",
              "    'support': 13},\n",
              "   '1': {'f1-score': 0.9559322033898304,\n",
              "    'precision': 0.9155844155844156,\n",
              "    'recall': 1.0,\n",
              "    'support': 141},\n",
              "   'accuracy': 0.9155844155844156,\n",
              "   'macro avg': {'f1-score': 0.4779661016949152,\n",
              "    'precision': 0.4577922077922078,\n",
              "    'recall': 0.5,\n",
              "    'support': 154},\n",
              "   'weighted avg': {'f1-score': 0.8752366277790006,\n",
              "    'precision': 0.8382948220610559,\n",
              "    'recall': 0.9155844155844156,\n",
              "    'support': 154}}},\n",
              " '5498603': {'Decision Tree': {'0': {'f1-score': 0.5074626865671642,\n",
              "    'precision': 0.5666666666666667,\n",
              "    'recall': 0.4594594594594595,\n",
              "    'support': 37},\n",
              "   '1': {'f1-score': 0.9223529411764707,\n",
              "    'precision': 0.9074074074074074,\n",
              "    'recall': 0.937799043062201,\n",
              "    'support': 209},\n",
              "   'accuracy': 0.8658536585365854,\n",
              "   'macro avg': {'f1-score': 0.7149078138718175,\n",
              "    'precision': 0.737037037037037,\n",
              "    'recall': 0.6986292512608302,\n",
              "    'support': 246},\n",
              "   'weighted avg': {'f1-score': 0.8599507484100304,\n",
              "    'precision': 0.8561577838000602,\n",
              "    'recall': 0.8658536585365854,\n",
              "    'support': 246}},\n",
              "  'K-Neighbours': {'0': {'f1-score': 0.33898305084745767,\n",
              "    'precision': 0.45454545454545453,\n",
              "    'recall': 0.2702702702702703,\n",
              "    'support': 37},\n",
              "   '1': {'f1-score': 0.9099307159353349,\n",
              "    'precision': 0.8794642857142857,\n",
              "    'recall': 0.9425837320574163,\n",
              "    'support': 209},\n",
              "   'accuracy': 0.8414634146341463,\n",
              "   'macro avg': {'f1-score': 0.6244568833913963,\n",
              "    'precision': 0.6670048701298701,\n",
              "    'recall': 0.6064270011638433,\n",
              "    'support': 246},\n",
              "   'weighted avg': {'f1-score': 0.8240564736253696,\n",
              "    'precision': 0.8155537298067786,\n",
              "    'recall': 0.8414634146341463,\n",
              "    'support': 246}},\n",
              "  'SVC': {'0': {'f1-score': 0.0,\n",
              "    'precision': 0.0,\n",
              "    'recall': 0.0,\n",
              "    'support': 37},\n",
              "   '1': {'f1-score': 0.9186813186813186,\n",
              "    'precision': 0.8495934959349594,\n",
              "    'recall': 1.0,\n",
              "    'support': 209},\n",
              "   'accuracy': 0.8495934959349594,\n",
              "   'macro avg': {'f1-score': 0.4593406593406593,\n",
              "    'precision': 0.4247967479674797,\n",
              "    'recall': 0.5,\n",
              "    'support': 246},\n",
              "   'weighted avg': {'f1-score': 0.7805056731886,\n",
              "    'precision': 0.7218091083349858,\n",
              "    'recall': 0.8495934959349594,\n",
              "    'support': 246}}},\n",
              " '5797046': {'Decision Tree': {'0': {'f1-score': 0.4150943396226416,\n",
              "    'precision': 0.55,\n",
              "    'recall': 0.3333333333333333,\n",
              "    'support': 33},\n",
              "   '1': {'f1-score': 0.9453262786596119,\n",
              "    'precision': 0.9241379310344827,\n",
              "    'recall': 0.9675090252707581,\n",
              "    'support': 277},\n",
              "   'accuracy': 0.9,\n",
              "   'macro avg': {'f1-score': 0.6802103091411268,\n",
              "    'precision': 0.7370689655172413,\n",
              "    'recall': 0.6504211793020457,\n",
              "    'support': 310},\n",
              "   'weighted avg': {'f1-score': 0.8888822335363216,\n",
              "    'precision': 0.8843103448275861,\n",
              "    'recall': 0.9,\n",
              "    'support': 310}},\n",
              "  'K-Neighbours': {'0': {'f1-score': 0.21621621621621626,\n",
              "    'precision': 1.0,\n",
              "    'recall': 0.12121212121212122,\n",
              "    'support': 33},\n",
              "   '1': {'f1-score': 0.9502572898799314,\n",
              "    'precision': 0.9052287581699346,\n",
              "    'recall': 1.0,\n",
              "    'support': 277},\n",
              "   'accuracy': 0.9064516129032258,\n",
              "   'macro avg': {'f1-score': 0.5832367530480739,\n",
              "    'precision': 0.9526143790849673,\n",
              "    'recall': 0.5606060606060606,\n",
              "    'support': 310},\n",
              "   'weighted avg': {'f1-score': 0.8721174336512134,\n",
              "    'precision': 0.9153173097195867,\n",
              "    'recall': 0.9064516129032258,\n",
              "    'support': 310}},\n",
              "  'SVC': {'0': {'f1-score': 0.0,\n",
              "    'precision': 0.0,\n",
              "    'recall': 0.0,\n",
              "    'support': 33},\n",
              "   '1': {'f1-score': 0.9437819420783645,\n",
              "    'precision': 0.8935483870967742,\n",
              "    'recall': 1.0,\n",
              "    'support': 277},\n",
              "   'accuracy': 0.8935483870967742,\n",
              "   'macro avg': {'f1-score': 0.47189097103918226,\n",
              "    'precision': 0.4467741935483871,\n",
              "    'recall': 0.5,\n",
              "    'support': 310},\n",
              "   'weighted avg': {'f1-score': 0.8433148321151838,\n",
              "    'precision': 0.7984287200832466,\n",
              "    'recall': 0.8935483870967742,\n",
              "    'support': 310}}},\n",
              " '6220552': {'Decision Tree': {'0': {'f1-score': 0.3125,\n",
              "    'precision': 0.38461538461538464,\n",
              "    'recall': 0.2631578947368421,\n",
              "    'support': 19},\n",
              "   '1': {'f1-score': 0.9632107023411371,\n",
              "    'precision': 0.9536423841059603,\n",
              "    'recall': 0.972972972972973,\n",
              "    'support': 296},\n",
              "   'accuracy': 0.9301587301587302,\n",
              "   'macro avg': {'f1-score': 0.6378553511705686,\n",
              "    'precision': 0.6691288843606724,\n",
              "    'recall': 0.6180654338549075,\n",
              "    'support': 315},\n",
              "   'weighted avg': {'f1-score': 0.9239614853745288,\n",
              "    'precision': 0.919320120644624,\n",
              "    'recall': 0.9301587301587302,\n",
              "    'support': 315}},\n",
              "  'K-Neighbours': {'0': {'f1-score': 0.0,\n",
              "    'precision': 0.0,\n",
              "    'recall': 0.0,\n",
              "    'support': 19},\n",
              "   '1': {'f1-score': 0.9672131147540984,\n",
              "    'precision': 0.9394904458598726,\n",
              "    'recall': 0.9966216216216216,\n",
              "    'support': 296},\n",
              "   'accuracy': 0.9365079365079365,\n",
              "   'macro avg': {'f1-score': 0.4836065573770492,\n",
              "    'precision': 0.4697452229299363,\n",
              "    'recall': 0.4983108108108108,\n",
              "    'support': 315},\n",
              "   'weighted avg': {'f1-score': 0.9088732760863909,\n",
              "    'precision': 0.8828227681730866,\n",
              "    'recall': 0.9365079365079365,\n",
              "    'support': 315}},\n",
              "  'SVC': {'0': {'f1-score': 0.0,\n",
              "    'precision': 0.0,\n",
              "    'recall': 0.0,\n",
              "    'support': 19},\n",
              "   '1': {'f1-score': 0.9689034369885434,\n",
              "    'precision': 0.9396825396825397,\n",
              "    'recall': 1.0,\n",
              "    'support': 296},\n",
              "   'accuracy': 0.9396825396825397,\n",
              "   'macro avg': {'f1-score': 0.4844517184942717,\n",
              "    'precision': 0.46984126984126984,\n",
              "    'recall': 0.5,\n",
              "    'support': 315},\n",
              "   'weighted avg': {'f1-score': 0.910461642376536,\n",
              "    'precision': 0.8830032753842277,\n",
              "    'recall': 0.9396825396825397,\n",
              "    'support': 315}}},\n",
              " '759667': {'Decision Tree': {'0': {'f1-score': 0.42857142857142855,\n",
              "    'precision': 0.6,\n",
              "    'recall': 0.3333333333333333,\n",
              "    'support': 9},\n",
              "   '1': {'f1-score': 0.9733333333333333,\n",
              "    'precision': 0.9605263157894737,\n",
              "    'recall': 0.9864864864864865,\n",
              "    'support': 148},\n",
              "   'accuracy': 0.9490445859872612,\n",
              "   'macro avg': {'f1-score': 0.7009523809523809,\n",
              "    'precision': 0.7802631578947368,\n",
              "    'recall': 0.6599099099099099,\n",
              "    'support': 157},\n",
              "   'weighted avg': {'f1-score': 0.9421049438883833,\n",
              "    'precision': 0.9398592021454912,\n",
              "    'recall': 0.9490445859872612,\n",
              "    'support': 157}},\n",
              "  'K-Neighbours': {'0': {'f1-score': 0.0,\n",
              "    'precision': 0.0,\n",
              "    'recall': 0.0,\n",
              "    'support': 9},\n",
              "   '1': {'f1-score': 0.9704918032786886,\n",
              "    'precision': 0.9426751592356688,\n",
              "    'recall': 1.0,\n",
              "    'support': 148},\n",
              "   'accuracy': 0.9426751592356688,\n",
              "   'macro avg': {'f1-score': 0.4852459016393443,\n",
              "    'precision': 0.4713375796178344,\n",
              "    'recall': 0.5,\n",
              "    'support': 157},\n",
              "   'weighted avg': {'f1-score': 0.914858515192649,\n",
              "    'precision': 0.8886364558399936,\n",
              "    'recall': 0.9426751592356688,\n",
              "    'support': 157}},\n",
              "  'SVC': {'0': {'f1-score': 0.0,\n",
              "    'precision': 0.0,\n",
              "    'recall': 0.0,\n",
              "    'support': 9},\n",
              "   '1': {'f1-score': 0.9704918032786886,\n",
              "    'precision': 0.9426751592356688,\n",
              "    'recall': 1.0,\n",
              "    'support': 148},\n",
              "   'accuracy': 0.9426751592356688,\n",
              "   'macro avg': {'f1-score': 0.4852459016393443,\n",
              "    'precision': 0.4713375796178344,\n",
              "    'recall': 0.5,\n",
              "    'support': 157},\n",
              "   'weighted avg': {'f1-score': 0.914858515192649,\n",
              "    'precision': 0.8886364558399936,\n",
              "    'recall': 0.9426751592356688,\n",
              "    'support': 157}}},\n",
              " '7749105': {'Decision Tree': {'0': {'f1-score': 0.9285714285714286,\n",
              "    'precision': 0.9285714285714286,\n",
              "    'recall': 0.9285714285714286,\n",
              "    'support': 14},\n",
              "   '1': {'f1-score': 0.9629629629629629,\n",
              "    'precision': 0.9629629629629629,\n",
              "    'recall': 0.9629629629629629,\n",
              "    'support': 27},\n",
              "   'accuracy': 0.9512195121951219,\n",
              "   'macro avg': {'f1-score': 0.9457671957671958,\n",
              "    'precision': 0.9457671957671958,\n",
              "    'recall': 0.9457671957671958,\n",
              "    'support': 41},\n",
              "   'weighted avg': {'f1-score': 0.9512195121951219,\n",
              "    'precision': 0.9512195121951219,\n",
              "    'recall': 0.9512195121951219,\n",
              "    'support': 41}},\n",
              "  'K-Neighbours': {'0': {'f1-score': 0.88,\n",
              "    'precision': 1.0,\n",
              "    'recall': 0.7857142857142857,\n",
              "    'support': 14},\n",
              "   '1': {'f1-score': 0.9473684210526316,\n",
              "    'precision': 0.9,\n",
              "    'recall': 1.0,\n",
              "    'support': 27},\n",
              "   'accuracy': 0.926829268292683,\n",
              "   'macro avg': {'f1-score': 0.9136842105263159,\n",
              "    'precision': 0.95,\n",
              "    'recall': 0.8928571428571428,\n",
              "    'support': 41},\n",
              "   'weighted avg': {'f1-score': 0.9243645699614892,\n",
              "    'precision': 0.9341463414634146,\n",
              "    'recall': 0.926829268292683,\n",
              "    'support': 41}},\n",
              "  'SVC': {'0': {'f1-score': 0.0,\n",
              "    'precision': 0.0,\n",
              "    'recall': 0.0,\n",
              "    'support': 14},\n",
              "   '1': {'f1-score': 0.7941176470588235,\n",
              "    'precision': 0.6585365853658537,\n",
              "    'recall': 1.0,\n",
              "    'support': 27},\n",
              "   'accuracy': 0.6585365853658537,\n",
              "   'macro avg': {'f1-score': 0.39705882352941174,\n",
              "    'precision': 0.32926829268292684,\n",
              "    'recall': 0.5,\n",
              "    'support': 41},\n",
              "   'weighted avg': {'f1-score': 0.5229555236728838,\n",
              "    'precision': 0.43367043426531826,\n",
              "    'recall': 0.6585365853658537,\n",
              "    'support': 41}}},\n",
              " '781756': {'Decision Tree': {'0': {'f1-score': 0.425531914893617,\n",
              "    'precision': 0.43478260869565216,\n",
              "    'recall': 0.4166666666666667,\n",
              "    'support': 24},\n",
              "   '1': {'f1-score': 0.9550748752079866,\n",
              "    'precision': 0.9534883720930233,\n",
              "    'recall': 0.9566666666666667,\n",
              "    'support': 300},\n",
              "   'accuracy': 0.9166666666666666,\n",
              "   'macro avg': {'f1-score': 0.6903033950508018,\n",
              "    'precision': 0.6941354903943378,\n",
              "    'recall': 0.6866666666666666,\n",
              "    'support': 324},\n",
              "   'weighted avg': {'f1-score': 0.9158494707402556,\n",
              "    'precision': 0.9150657229524771,\n",
              "    'recall': 0.9166666666666666,\n",
              "    'support': 324}},\n",
              "  'K-Neighbours': {'0': {'f1-score': 0.45,\n",
              "    'precision': 0.5625,\n",
              "    'recall': 0.375,\n",
              "    'support': 24},\n",
              "   '1': {'f1-score': 0.9638157894736843,\n",
              "    'precision': 0.9512987012987013,\n",
              "    'recall': 0.9766666666666667,\n",
              "    'support': 300},\n",
              "   'accuracy': 0.9320987654320988,\n",
              "   'macro avg': {'f1-score': 0.7069078947368421,\n",
              "    'precision': 0.7568993506493507,\n",
              "    'recall': 0.6758333333333333,\n",
              "    'support': 324},\n",
              "   'weighted avg': {'f1-score': 0.9257553606237817,\n",
              "    'precision': 0.9224987974987975,\n",
              "    'recall': 0.9320987654320988,\n",
              "    'support': 324}},\n",
              "  'SVC': {'0': {'f1-score': 0.07692307692307693,\n",
              "    'precision': 0.5,\n",
              "    'recall': 0.041666666666666664,\n",
              "    'support': 24},\n",
              "   '1': {'f1-score': 0.9614147909967846,\n",
              "    'precision': 0.9285714285714286,\n",
              "    'recall': 0.9966666666666667,\n",
              "    'support': 300},\n",
              "   'accuracy': 0.9259259259259259,\n",
              "   'macro avg': {'f1-score': 0.5191689339599308,\n",
              "    'precision': 0.7142857142857143,\n",
              "    'recall': 0.5191666666666667,\n",
              "    'support': 324},\n",
              "   'weighted avg': {'f1-score': 0.8958968862505842,\n",
              "    'precision': 0.8968253968253967,\n",
              "    'recall': 0.9259259259259259,\n",
              "    'support': 324}}},\n",
              " '8000685': {'Decision Tree': {'0': {'f1-score': 0.19999999999999998,\n",
              "    'precision': 0.15789473684210525,\n",
              "    'recall': 0.2727272727272727,\n",
              "    'support': 11},\n",
              "   '1': {'f1-score': 0.9602649006622517,\n",
              "    'precision': 0.9731543624161074,\n",
              "    'recall': 0.9477124183006536,\n",
              "    'support': 306},\n",
              "   'accuracy': 0.9242902208201893,\n",
              "   'macro avg': {'f1-score': 0.5801324503311258,\n",
              "    'precision': 0.5655245496291064,\n",
              "    'recall': 0.6102198455139631,\n",
              "    'support': 317},\n",
              "   'weighted avg': {'f1-score': 0.9338834687780726,\n",
              "    'precision': 0.944864596228997,\n",
              "    'recall': 0.9242902208201893,\n",
              "    'support': 317}},\n",
              "  'K-Neighbours': {'0': {'f1-score': 0.14285714285714288,\n",
              "    'precision': 0.3333333333333333,\n",
              "    'recall': 0.09090909090909091,\n",
              "    'support': 11},\n",
              "   '1': {'f1-score': 0.9806451612903226,\n",
              "    'precision': 0.9681528662420382,\n",
              "    'recall': 0.9934640522875817,\n",
              "    'support': 306},\n",
              "   'accuracy': 0.9621451104100947,\n",
              "   'macro avg': {'f1-score': 0.5617511520737327,\n",
              "    'precision': 0.6507430997876857,\n",
              "    'recall': 0.5421865715983363,\n",
              "    'support': 317},\n",
              "   'weighted avg': {'f1-score': 0.9515736527642502,\n",
              "    'precision': 0.9461244281915785,\n",
              "    'recall': 0.9621451104100947,\n",
              "    'support': 317}},\n",
              "  'SVC': {'0': {'f1-score': 0.0,\n",
              "    'precision': 0.0,\n",
              "    'recall': 0.0,\n",
              "    'support': 11},\n",
              "   '1': {'f1-score': 0.9823434991974318,\n",
              "    'precision': 0.9652996845425867,\n",
              "    'recall': 1.0,\n",
              "    'support': 306},\n",
              "   'accuracy': 0.9652996845425867,\n",
              "   'macro avg': {'f1-score': 0.4911717495987159,\n",
              "    'precision': 0.48264984227129337,\n",
              "    'recall': 0.5,\n",
              "    'support': 317},\n",
              "   'weighted avg': {'f1-score': 0.9482558698877417,\n",
              "    'precision': 0.9318034809780175,\n",
              "    'recall': 0.9652996845425867,\n",
              "    'support': 317}}},\n",
              " '8173033': {'Decision Tree': {'0': {'f1-score': 0.19999999999999998,\n",
              "    'precision': 0.18181818181818182,\n",
              "    'recall': 0.2222222222222222,\n",
              "    'support': 9},\n",
              "   '1': {'f1-score': 0.9738562091503268,\n",
              "    'precision': 0.9770491803278688,\n",
              "    'recall': 0.9706840390879479,\n",
              "    'support': 307},\n",
              "   'accuracy': 0.9493670886075949,\n",
              "   'macro avg': {'f1-score': 0.5869281045751634,\n",
              "    'precision': 0.5794336810730253,\n",
              "    'recall': 0.5964531306550851,\n",
              "    'support': 316},\n",
              "   'weighted avg': {'f1-score': 0.9518160006618681,\n",
              "    'precision': 0.9544001961930992,\n",
              "    'recall': 0.9493670886075949,\n",
              "    'support': 316}},\n",
              "  'K-Neighbours': {'0': {'f1-score': 0.0,\n",
              "    'precision': 0.0,\n",
              "    'recall': 0.0,\n",
              "    'support': 9},\n",
              "   '1': {'f1-score': 0.985553772070626,\n",
              "    'precision': 0.9715189873417721,\n",
              "    'recall': 1.0,\n",
              "    'support': 307},\n",
              "   'accuracy': 0.9715189873417721,\n",
              "   'macro avg': {'f1-score': 0.492776886035313,\n",
              "    'precision': 0.48575949367088606,\n",
              "    'recall': 0.5,\n",
              "    'support': 316},\n",
              "   'weighted avg': {'f1-score': 0.9574842026129182,\n",
              "    'precision': 0.9438491427655824,\n",
              "    'recall': 0.9715189873417721,\n",
              "    'support': 316}},\n",
              "  'SVC': {'0': {'f1-score': 0.0,\n",
              "    'precision': 0.0,\n",
              "    'recall': 0.0,\n",
              "    'support': 9},\n",
              "   '1': {'f1-score': 0.985553772070626,\n",
              "    'precision': 0.9715189873417721,\n",
              "    'recall': 1.0,\n",
              "    'support': 307},\n",
              "   'accuracy': 0.9715189873417721,\n",
              "   'macro avg': {'f1-score': 0.492776886035313,\n",
              "    'precision': 0.48575949367088606,\n",
              "    'recall': 0.5,\n",
              "    'support': 316},\n",
              "   'weighted avg': {'f1-score': 0.9574842026129182,\n",
              "    'precision': 0.9438491427655824,\n",
              "    'recall': 0.9715189873417721,\n",
              "    'support': 316}}},\n",
              " '844359': {'Decision Tree': {'0': {'f1-score': 0.44776119402985076,\n",
              "    'precision': 0.36585365853658536,\n",
              "    'recall': 0.5769230769230769,\n",
              "    'support': 26},\n",
              "   '1': {'f1-score': 0.9297912713472486,\n",
              "    'precision': 0.95703125,\n",
              "    'recall': 0.9040590405904059,\n",
              "    'support': 271},\n",
              "   'accuracy': 0.8754208754208754,\n",
              "   'macro avg': {'f1-score': 0.6887762326885497,\n",
              "    'precision': 0.6614424542682926,\n",
              "    'recall': 0.7404910587567414,\n",
              "    'support': 297},\n",
              "   'weighted avg': {'f1-score': 0.8875933521208097,\n",
              "    'precision': 0.9052783295351893,\n",
              "    'recall': 0.8754208754208754,\n",
              "    'support': 297}},\n",
              "  'K-Neighbours': {'0': {'f1-score': 0.46511627906976744,\n",
              "    'precision': 0.5882352941176471,\n",
              "    'recall': 0.38461538461538464,\n",
              "    'support': 26},\n",
              "   '1': {'f1-score': 0.9582577132486388,\n",
              "    'precision': 0.9428571428571428,\n",
              "    'recall': 0.974169741697417,\n",
              "    'support': 271},\n",
              "   'accuracy': 0.9225589225589226,\n",
              "   'macro avg': {'f1-score': 0.7116869961592032,\n",
              "    'precision': 0.7655462184873949,\n",
              "    'recall': 0.6793925631564008,\n",
              "    'support': 297},\n",
              "   'weighted avg': {'f1-score': 0.9150870826471215,\n",
              "    'precision': 0.9118128059304529,\n",
              "    'recall': 0.9225589225589226,\n",
              "    'support': 297}},\n",
              "  'SVC': {'0': {'f1-score': 0.3,\n",
              "    'precision': 0.42857142857142855,\n",
              "    'recall': 0.23076923076923078,\n",
              "    'support': 26},\n",
              "   '1': {'f1-score': 0.9494584837545127,\n",
              "    'precision': 0.9293286219081273,\n",
              "    'recall': 0.9704797047970479,\n",
              "    'support': 271},\n",
              "   'accuracy': 0.9057239057239057,\n",
              "   'macro avg': {'f1-score': 0.6247292418772563,\n",
              "    'precision': 0.6789500252397779,\n",
              "    'recall': 0.6006244677831394,\n",
              "    'support': 297},\n",
              "   'weighted avg': {'f1-score': 0.8926035323147237,\n",
              "    'precision': 0.8854912918517159,\n",
              "    'recall': 0.9057239057239057,\n",
              "    'support': 297}}},\n",
              " '8530312': {'Decision Tree': {'0': {'f1-score': 0.21052631578947367,\n",
              "    'precision': 0.23529411764705882,\n",
              "    'recall': 0.19047619047619047,\n",
              "    'support': 21},\n",
              "   '1': {'f1-score': 0.9491525423728814,\n",
              "    'precision': 0.9427609427609428,\n",
              "    'recall': 0.9556313993174061,\n",
              "    'support': 293},\n",
              "   'accuracy': 0.9044585987261147,\n",
              "   'macro avg': {'f1-score': 0.5798394290811775,\n",
              "    'precision': 0.5890275302040008,\n",
              "    'recall': 0.5730537948967983,\n",
              "    'support': 314},\n",
              "   'weighted avg': {'f1-score': 0.8997539730790867,\n",
              "    'precision': 0.8954462824826257,\n",
              "    'recall': 0.9044585987261147,\n",
              "    'support': 314}},\n",
              "  'K-Neighbours': {'0': {'f1-score': 0.0909090909090909,\n",
              "    'precision': 1.0,\n",
              "    'recall': 0.047619047619047616,\n",
              "    'support': 21},\n",
              "   '1': {'f1-score': 0.9669966996699669,\n",
              "    'precision': 0.9361022364217252,\n",
              "    'recall': 1.0,\n",
              "    'support': 293},\n",
              "   'accuracy': 0.9363057324840764,\n",
              "   'macro avg': {'f1-score': 0.5289528952895289,\n",
              "    'precision': 0.9680511182108626,\n",
              "    'recall': 0.5238095238095238,\n",
              "    'support': 314},\n",
              "   'weighted avg': {'f1-score': 0.9084048532241759,\n",
              "    'precision': 0.9403756537311002,\n",
              "    'recall': 0.9363057324840764,\n",
              "    'support': 314}},\n",
              "  'SVC': {'0': {'f1-score': 0.08,\n",
              "    'precision': 0.25,\n",
              "    'recall': 0.047619047619047616,\n",
              "    'support': 21},\n",
              "   '1': {'f1-score': 0.9618573797678275,\n",
              "    'precision': 0.9354838709677419,\n",
              "    'recall': 0.9897610921501706,\n",
              "    'support': 293},\n",
              "   'accuracy': 0.9267515923566879,\n",
              "   'macro avg': {'f1-score': 0.5209286898839137,\n",
              "    'precision': 0.592741935483871,\n",
              "    'recall': 0.5186900698846091,\n",
              "    'support': 314},\n",
              "   'weighted avg': {'f1-score': 0.9028796569171129,\n",
              "    'precision': 0.8896394082597081,\n",
              "    'recall': 0.9267515923566879,\n",
              "    'support': 314}}},\n",
              " '8686948': {'Decision Tree': {'0': {'f1-score': 0.6250000000000001,\n",
              "    'precision': 0.5555555555555556,\n",
              "    'recall': 0.7142857142857143,\n",
              "    'support': 14},\n",
              "   '1': {'f1-score': 0.98,\n",
              "    'precision': 0.9865771812080537,\n",
              "    'recall': 0.9735099337748344,\n",
              "    'support': 302},\n",
              "   'accuracy': 0.9620253164556962,\n",
              "   'macro avg': {'f1-score': 0.8025,\n",
              "    'precision': 0.7710663683818046,\n",
              "    'recall': 0.8438978240302744,\n",
              "    'support': 316},\n",
              "   'weighted avg': {'f1-score': 0.9642721518987342,\n",
              "    'precision': 0.9674812864006644,\n",
              "    'recall': 0.9620253164556962,\n",
              "    'support': 316}},\n",
              "  'K-Neighbours': {'0': {'f1-score': 0.36363636363636365,\n",
              "    'precision': 0.5,\n",
              "    'recall': 0.2857142857142857,\n",
              "    'support': 14},\n",
              "   '1': {'f1-score': 0.9770491803278688,\n",
              "    'precision': 0.9675324675324676,\n",
              "    'recall': 0.9867549668874173,\n",
              "    'support': 302},\n",
              "   'accuracy': 0.9556962025316456,\n",
              "   'macro avg': {'f1-score': 0.6703427719821162,\n",
              "    'precision': 0.7337662337662338,\n",
              "    'recall': 0.6362346263008515,\n",
              "    'support': 316},\n",
              "   'weighted avg': {'f1-score': 0.9498726631326755,\n",
              "    'precision': 0.9468190037810291,\n",
              "    'recall': 0.9556962025316456,\n",
              "    'support': 316}},\n",
              "  'SVC': {'0': {'f1-score': 0.0,\n",
              "    'precision': 0.0,\n",
              "    'recall': 0.0,\n",
              "    'support': 14},\n",
              "   '1': {'f1-score': 0.9773462783171522,\n",
              "    'precision': 0.9556962025316456,\n",
              "    'recall': 1.0,\n",
              "    'support': 302},\n",
              "   'accuracy': 0.9556962025316456,\n",
              "   'macro avg': {'f1-score': 0.4886731391585761,\n",
              "    'precision': 0.4778481012658228,\n",
              "    'recall': 0.5,\n",
              "    'support': 316},\n",
              "   'weighted avg': {'f1-score': 0.9340461267461391,\n",
              "    'precision': 0.9133552315334081,\n",
              "    'recall': 0.9556962025316456,\n",
              "    'support': 316}}},\n",
              " '8692923': {'Decision Tree': {'0': {'f1-score': 0.4528301886792453,\n",
              "    'precision': 0.5217391304347826,\n",
              "    'recall': 0.4,\n",
              "    'support': 30},\n",
              "   '1': {'f1-score': 0.9486725663716814,\n",
              "    'precision': 0.9370629370629371,\n",
              "    'recall': 0.9605734767025089,\n",
              "    'support': 279},\n",
              "   'accuracy': 0.9061488673139159,\n",
              "   'macro avg': {'f1-score': 0.7007513775254633,\n",
              "    'precision': 0.7294010337488599,\n",
              "    'recall': 0.6802867383512545,\n",
              "    'support': 309},\n",
              "   'weighted avg': {'f1-score': 0.9005325297025129,\n",
              "    'precision': 0.8967402373903008,\n",
              "    'recall': 0.9061488673139159,\n",
              "    'support': 309}},\n",
              "  'K-Neighbours': {'0': {'f1-score': 0.16666666666666669,\n",
              "    'precision': 0.5,\n",
              "    'recall': 0.1,\n",
              "    'support': 30},\n",
              "   '1': {'f1-score': 0.9484536082474228,\n",
              "    'precision': 0.9108910891089109,\n",
              "    'recall': 0.989247311827957,\n",
              "    'support': 279},\n",
              "   'accuracy': 0.9029126213592233,\n",
              "   'macro avg': {'f1-score': 0.5575601374570447,\n",
              "    'precision': 0.7054455445544554,\n",
              "    'recall': 0.5446236559139785,\n",
              "    'support': 309},\n",
              "   'weighted avg': {'f1-score': 0.8725519634337572,\n",
              "    'precision': 0.8709987503604729,\n",
              "    'recall': 0.9029126213592233,\n",
              "    'support': 309}},\n",
              "  'SVC': {'0': {'f1-score': 0.0,\n",
              "    'precision': 0.0,\n",
              "    'recall': 0.0,\n",
              "    'support': 30},\n",
              "   '1': {'f1-score': 0.9489795918367347,\n",
              "    'precision': 0.9029126213592233,\n",
              "    'recall': 1.0,\n",
              "    'support': 279},\n",
              "   'accuracy': 0.9029126213592233,\n",
              "   'macro avg': {'f1-score': 0.4744897959183674,\n",
              "    'precision': 0.45145631067961167,\n",
              "    'recall': 0.5,\n",
              "    'support': 309},\n",
              "   'weighted avg': {'f1-score': 0.8568456508817119,\n",
              "    'precision': 0.8152512018097842,\n",
              "    'recall': 0.9029126213592233,\n",
              "    'support': 309}}},\n",
              " '9106476': {'Decision Tree': {'0': {'f1-score': 0.4642857142857143,\n",
              "    'precision': 0.5652173913043478,\n",
              "    'recall': 0.3939393939393939,\n",
              "    'support': 33},\n",
              "   '1': {'f1-score': 0.9480968858131488,\n",
              "    'precision': 0.9319727891156463,\n",
              "    'recall': 0.9647887323943662,\n",
              "    'support': 284},\n",
              "   'accuracy': 0.9053627760252366,\n",
              "   'macro avg': {'f1-score': 0.7061913000494315,\n",
              "    'precision': 0.748595090209997,\n",
              "    'recall': 0.67936406316688,\n",
              "    'support': 317},\n",
              "   'weighted avg': {'f1-score': 0.8977316849916808,\n",
              "    'precision': 0.893793205116363,\n",
              "    'recall': 0.9053627760252366,\n",
              "    'support': 317}},\n",
              "  'K-Neighbours': {'0': {'f1-score': 0.25,\n",
              "    'precision': 0.7142857142857143,\n",
              "    'recall': 0.15151515151515152,\n",
              "    'support': 33},\n",
              "   '1': {'f1-score': 0.9494949494949495,\n",
              "    'precision': 0.9096774193548387,\n",
              "    'recall': 0.9929577464788732,\n",
              "    'support': 284},\n",
              "   'accuracy': 0.9053627760252366,\n",
              "   'macro avg': {'f1-score': 0.5997474747474747,\n",
              "    'precision': 0.8119815668202766,\n",
              "    'recall': 0.5722364489970124,\n",
              "    'support': 317},\n",
              "   'weighted avg': {'f1-score': 0.8766768632699232,\n",
              "    'precision': 0.8893369579438574,\n",
              "    'recall': 0.9053627760252366,\n",
              "    'support': 317}},\n",
              "  'SVC': {'0': {'f1-score': 0.28571428571428575,\n",
              "    'precision': 0.6666666666666666,\n",
              "    'recall': 0.18181818181818182,\n",
              "    'support': 33},\n",
              "   '1': {'f1-score': 0.9493243243243243,\n",
              "    'precision': 0.9123376623376623,\n",
              "    'recall': 0.9894366197183099,\n",
              "    'support': 284},\n",
              "   'accuracy': 0.9053627760252366,\n",
              "   'macro avg': {'f1-score': 0.6175193050193051,\n",
              "    'precision': 0.7895021645021645,\n",
              "    'recall': 0.5856274007682458,\n",
              "    'support': 317},\n",
              "   'weighted avg': {'f1-score': 0.8802418912829009,\n",
              "    'precision': 0.8867630791921012,\n",
              "    'recall': 0.9053627760252366,\n",
              "    'support': 317}}},\n",
              " '9961348': {'Decision Tree': {'0': {'f1-score': 0.21739130434782608,\n",
              "    'precision': 0.22727272727272727,\n",
              "    'recall': 0.20833333333333334,\n",
              "    'support': 24},\n",
              "   '1': {'f1-score': 0.9162790697674419,\n",
              "    'precision': 0.9120370370370371,\n",
              "    'recall': 0.9205607476635514,\n",
              "    'support': 214},\n",
              "   'accuracy': 0.8487394957983193,\n",
              "   'macro avg': {'f1-score': 0.566835187057634,\n",
              "    'precision': 0.5696548821548821,\n",
              "    'recall': 0.5644470404984424,\n",
              "    'support': 238},\n",
              "   'weighted avg': {'f1-score': 0.8458029925822705,\n",
              "    'precision': 0.8429851738675268,\n",
              "    'recall': 0.8487394957983193,\n",
              "    'support': 238}},\n",
              "  'K-Neighbours': {'0': {'f1-score': 0.07999999999999999,\n",
              "    'precision': 1.0,\n",
              "    'recall': 0.041666666666666664,\n",
              "    'support': 24},\n",
              "   '1': {'f1-score': 0.9490022172949002,\n",
              "    'precision': 0.9029535864978903,\n",
              "    'recall': 1.0,\n",
              "    'support': 214},\n",
              "   'accuracy': 0.9033613445378151,\n",
              "   'macro avg': {'f1-score': 0.5145011086474501,\n",
              "    'precision': 0.9514767932489452,\n",
              "    'recall': 0.5208333333333334,\n",
              "    'support': 238},\n",
              "   'weighted avg': {'f1-score': 0.8613717416012968,\n",
              "    'precision': 0.9127397794560862,\n",
              "    'recall': 0.9033613445378151,\n",
              "    'support': 238}},\n",
              "  'SVC': {'0': {'f1-score': 0.0,\n",
              "    'precision': 0.0,\n",
              "    'recall': 0.0,\n",
              "    'support': 24},\n",
              "   '1': {'f1-score': 0.9469026548672567,\n",
              "    'precision': 0.8991596638655462,\n",
              "    'recall': 1.0,\n",
              "    'support': 214},\n",
              "   'accuracy': 0.8991596638655462,\n",
              "   'macro avg': {'f1-score': 0.47345132743362833,\n",
              "    'precision': 0.4495798319327731,\n",
              "    'recall': 0.5,\n",
              "    'support': 238},\n",
              "   'weighted avg': {'f1-score': 0.8514166728638358,\n",
              "    'precision': 0.808488101122802,\n",
              "    'recall': 0.8991596638655462,\n",
              "    'support': 238}}}}"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 150
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "J9jcH1zTkgV_"
      },
      "source": [
        "At the start of this project, we had one concern in our heads and it revolved around sneaky algorithms with low precision and recall. We didn't want our model too jst predict the most common class for all classes and thus to get a good accuracy.\n",
        "This is partly why we decided not to go with the binary sleep Vs awake from the get go as in this situation, it is easier for a model to predict the exact same class for each session - this is because the person is awake most of the time.\n",
        "\n",
        "But our worries are just that, worries, and did not materialise for us.\n",
        "\n",
        "From a quick analysis of these returned dictionaries, you can see that our precision and recall scores lay in the region of 0.4 to 0.9 and the F1 scores are similar. This is a healthy range and nothing to be concerned about."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ZPTVst2f-RZp"
      },
      "source": [
        "---\n",
        "---\n",
        "\n",
        "# 7. Conclustion and Results\n",
        "\n",
        "---\n",
        "---"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JV7DSbSVl4Rn"
      },
      "source": [
        "The project for this assignent was different to the last one where there wa s alot more to it. With the lat project, we had one dataset to use and one classifier type to use whereas the scope for this project was more endless. \n",
        "\n",
        "This posed some interesting new challenges for us but also put us under a bit more pressure in tackling these and made us have to work hard and plan our our approaches more. This all meant lots of learning which is definitely a plus, and we were able to perform some interesting analysis.\n",
        "\n",
        "Due to the bulk in th eproject between prepering the data, extracting features and then formatting everything, we didnt get a lot of time to focus on the classification and analysis so we decided to take a simple approach using just three classifiers thw whole way through. While this limited us when choosing the best model, it also gave us a chance to dive further into these specific models and understand them better.\n",
        "\n",
        "This understanding held true throughout the project as seen by our experiments where all of our hypotheses were correct. While it is ok to think a certain way using intuition, it is always important to test and verify along the way to ensure you understand what is going on and are in sinc with the process."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "EqcoUPGHnu2I"
      },
      "source": [
        ""
      ],
      "execution_count": null,
      "outputs": []
    }
  ]
}