Guided Investigation - Fusion Incident.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "# Guided investigation - Fusion Incidents\n",
        "<details>\n",
        "    <summary> <u>Details...</u></summary>\n",
        "Notebook Version: 1.1<br>\n",
        "\n",
        "**Data Sources Used**:<br>\n",
        "- Microsoft Sentinel\n",
        "    - Sentinel Fusion Incidents (https://learn.microsoft.com/en-us/azure/sentinel/fusion)\n",
        "<br>\n",
        "- Threat Intelligence Providers\n",
        "    - OTX (https://otx.alienvault.com/)\n",
        "    - VirusTotal (https://www.virustotal.com/)\n",
        "    - XForce (https://www.ibm.com/security/xforce)\n",
        "    - GreyNoise (https://www.greynoise.io)\n",
        "</details>\n",
        "\n",
        "Microsoft Sentinel uses Fusion, a correlation engine based on scalable machine learning algorithms, to automatically detect multistage attacks (also known as advanced persistent threats or APT) by identifying combinations of anomalous behaviors and suspicious activities that are observed at various stages of the kill chain. On the basis of these discoveries, Microsoft Sentinel generates incidents that would otherwise be difficult to catch. These incidents comprise two or more alerts or activities. By design, these incidents are low-volume, high-fidelity, and high-severity. For more information https://aka.ms/SentinelFusion<br>\n",
        "<br>\n",
        "This notebook takes you through a guided investigation of a Microsoft Sentinel Fusion Incident. The investigation focuses on the entities that attached to a Microsoft Sentinel Fusion Incident. This notebook can be extended with additional investigation steps based on specific processes and workflows."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### Notebook initialization\n",
        "The next cell:\n",
        "- Checks for the correct Python version\n",
        "- Checks versions and optionally installs required packages\n",
        "- Imports the required packages into the notebook\n",
        "- Sets a number of configuration options.\n",
        "\n",
        "<details>\n",
        "    <summary><u>More details...</u></summary>\n",
        "\n",
        "This should complete without errors. If you encounter errors or warnings look at the following two notebooks:\n",
        "- [TroubleShootingNotebooks](https://github.com/Azure/Azure-Sentinel-Notebooks/blob/master/TroubleShootingNotebooks.ipynb)\n",
        "- [ConfiguringNotebookEnvironment](https://github.com/Azure/Azure-Sentinel-Notebooks/blob/master/ConfiguringNotebookEnvironment.ipynb)\n",
        "\n",
        "If you are running in the Microsoft Sentinel Notebooks environment (Azure Notebooks or Azure ML) you can run live versions of these notebooks:\n",
        "- [Run TroubleShootingNotebooks](./TroubleShootingNotebooks.ipynb)\n",
        "- [Run ConfiguringNotebookEnvironment](./ConfiguringNotebookEnvironment.ipynb)\n",
        "\n",
        "You may also need to do some additional configuration to successfully use functions such as Threat Intelligence service lookup and Geo IP lookup. \n",
        "There are more details about this in the `ConfiguringNotebookEnvironment` notebook and in these documents:\n",
        "- [msticpy configuration](https://msticpy.readthedocs.io/en/latest/getting_started/msticpyconfig.html)\n",
        "- [Threat intelligence provider configuration](https://msticpy.readthedocs.io/en/latest/data_acquisition/TIProviders.html#configuration-file)\n",
        "\n",
        "</details>\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "gather": {
          "logged": 1669671608170
        },
        "jupyter": {
          "outputs_hidden": false,
          "source_hidden": false
        },
        "nteract": {
          "transient": {
            "deleting": false
          }
        }
      },
      "outputs": [],
      "source": [
        "from IPython.display import HTML, display\n",
        "import sys\n",
        "!{sys.executable} -m pip install azure-mgmt-resourcegraph\n",
        "!{sys.executable} -m pip install msticpy\n",
        "\n",
        "extra_imports = [\n",
        "    \"json\",\n",
        "    \"bokeh.plotting,show\",\n",
        "    \"msticnb,nb\",\n",
        "    \"msticpy.nbwidgets,SelectAlert\",\n",
        "    \"msticpy.nbwidgets,Progress\",\n",
        "    \"msticpy.context.azure, MicrosoftSentinel\",\n",
        "    \"msticpy.vis.entity_graph_tools, EntityGraph\",\n",
        "]\n",
        "\n",
        "display(HTML(\"<h3>Starting Notebook setup...</h3>\"))\n",
        "\n",
        "import msticpy as mp\n",
        "\n",
        "mp.init_notebook(\n",
        "    extra_imports=extra_imports,\n",
        "    additional_packages=[\"msticnb>=1.0\"],\n",
        ")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "gather": {
          "logged": 1669671609129
        },
        "tags": []
      },
      "outputs": [],
      "source": [
        "from IPython.display import HTML, display\n",
        "extra_imports = [\n",
        "    \"json\",\n",
        "    \"bokeh.plotting,show\",\n",
        "    \"msticnb,nb\",\n",
        "    \"msticpy.nbwidgets,SelectAlert\",\n",
        "    \"msticpy.nbwidgets,Progress\",\n",
        "    \"msticpy.context.azure.azure_sentinel,AzureSentinel\",\n",
        "    \"msticpy.vis.entity_graph_tools, EntityGraph\",\n",
        "]\n",
        "\n",
        "display(HTML(\"<h3>Starting Notebook setup...</h3>\"))\n",
        "\n",
        "import msticpy as mp\n",
        "\n",
        "mp.init_notebook(\n",
        "    extra_imports=extra_imports,\n",
        "    additional_packages=[\"msticnb>=1.0\"],\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "<div class=\"alert alert-block alert-info\">\n",
        "<b>Note:</b> The following cell creates some helper functions used later in the notebook. This cell has no output.\n",
        "</div>"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "gather": {
          "logged": 1669671609288
        }
      },
      "outputs": [],
      "source": [
        "import re\n",
        "\n",
        "def check_ent(items, entity):\n",
        "    \"\"\"Check if entity is present\"\"\"\n",
        "    for item in items:\n",
        "        if item[0].casefold() == entity.casefold():\n",
        "            return True\n",
        "    return False\n",
        "\n",
        "def extract_resourcegroup(azure_id : str) -> str:\n",
        "\n",
        "    azure_id = azure_id.lower()\n",
        "    m = re.match(r'^/?subscriptions/[^/]+/resourcegroups/[^/]+', azure_id)\n",
        "    if m == None:\n",
        "        return \"\"\n",
        "    if 'providers/microsoft.devices/iothubs' in azure_id:\n",
        "      return \"\"\n",
        "    start, end = m.span()\n",
        "    resource_group = azure_id[start:end]\n",
        "    if resource_group[0] != '/':\n",
        "        resource_group = \"/\" + resource_group\n",
        "    return resource_group\n",
        "\n",
        "def ti_color_cells(val):\n",
        "    \"\"\"Color cells of output dataframe based on severity\"\"\"\n",
        "    color = \"none\"\n",
        "    if isinstance(val, str):\n",
        "        if val.casefold() == \"high\":\n",
        "            color = \"Red\"\n",
        "        elif val.casefold() == \"warning\" or val.casefold() == \"medium\":\n",
        "            color = \"Orange\"\n",
        "        elif val.casefold() == \"information\" or val.casefold() == \"low\":\n",
        "            color = \"Green\"\n",
        "    return f\"background-color: {color}\"\n",
        "\n",
        "\n",
        "def ent_color_cells(val):\n",
        "    \"\"\"Color table cells based on values in the cells\"\"\"\n",
        "    if isinstance(val, int):\n",
        "        color = \"yellow\" if val < 3 else \"none\"\n",
        "    elif isinstance(val, float):\n",
        "        color = \"yellow\" if val > 4.30891 or val < 2.72120 else \"none\"\n",
        "    else:\n",
        "        color = \"none\"\n",
        "    return \"background-color: %s\" % color\n",
        "\n",
        "def ent_alerts(ent_val):\n",
        "    query = f\" SecurityAlert | where TimeGenerated between(datetime({start})..datetime({end})) | where Entities contains '{ent_val}'\"\n",
        "    alerts_df = qry_prov.exec_query(query)\n",
        "    if isinstance(alerts_df, pd.DataFrame) and not alerts_df.empty:\n",
        "        display_timeline(\n",
        "            data=alerts_df,\n",
        "            source_columns=[\"DisplayName\", \"AlertSeverity\", \"ProviderName\"],\n",
        "            title=f\"Alerts involving {ent_val}\",\n",
        "            group_by=\"AlertSeverity\",\n",
        "            height=300,\n",
        "            time_column=\"TimeGenerated\",\n",
        "        )"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "gather": {
          "logged": 1669671609445
        },
        "tags": [
          "parameters"
        ]
      },
      "outputs": [],
      "source": [
        "from datetime import datetime, timedelta, timezone\n",
        "# papermill default parameters\n",
        "ws_name = \"default\"\n",
        "incident_id = None\n",
        "end = datetime.now(timezone.utc)\n",
        "start = end - timedelta(days=5)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### Authenticate to Microsoft Sentinel APIs and Select Subscriptions\n",
        "\n",
        "This cell connects to the Microsoft Sentinel APIs and gets a list of subscriptions the user has access to for them to select. In order to use this the user must have at least read permissions on the Microsoft Sentinel workspace.\n",
        "In the drop down select the name of the subscription that contains the Microsoft Sentinel workspace you want to triage incidents from."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "gather": {
          "logged": 1669671609601
        }
      },
      "outputs": [],
      "source": [
        "print(\n",
        "    \"Configured workspaces: \",\n",
        "    \", \".join(msticpy.settings.get_config(\"AzureSentinel.Workspaces\").keys()),\n",
        ")\n",
        "import ipywidgets as widgets\n",
        "\n",
        "ws_param = widgets.Combobox(\n",
        "    description=\"Workspace Name\",\n",
        "    value=ws_name,\n",
        "    options=list(msticpy.settings.get_config(\"AzureSentinel.Workspaces\").keys()),\n",
        ")\n",
        "ws_param"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Now select the name of the Microsoft Sentinel workspace in the subscription you want to triage incidents from."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "\n",
        "### Authenticate to Microsoft Sentinel, TI providers and load Notebooklets\n",
        "<details>\n",
        "    <summary><u>Details...</u></summary>\n",
        "If you are using user/device authentication, run the following cell. \n",
        "- Click the 'Copy code to clipboard and authenticate' button.\n",
        "- This will pop up an Azure Active Directory authentication dialog (in a new tab or browser window). The device code will have been copied to the clipboard. \n",
        "- Select the text box and paste (Ctrl-V/Cmd-V) the copied value. \n",
        "- You should then be redirected to a user authentication page where you should authenticate with a user account that has permission to query your Log Analytics workspace.\n",
        "\n",
        "Note: you may occasionally see a JavaScript error displayed at the end of the authentication - you can safely ignore this.<br>\n",
        "On successful authentication you should see a ```popup schema``` button.\n",
        "To find your Workspace Id go to [Log Analytics](https://ms.portal.azure.com/#blade/HubsExtension/Resources/resourceType/Microsoft.OperationalInsights%2Fworkspaces). Look at the workspace properties to find the ID.\n",
        "    \n",
        "Note that you may see a warning relating to the IPStack service when running this cell. This can be safely ignored as its not used in this case.\n",
        "</details>"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "gather": {
          "logged": 1669671678626
        }
      },
      "outputs": [],
      "source": [
        "from msticpy.context.azure import MicrosoftSentinel\n",
        "from msticpy.common.pkg_config import get_config\n",
        "config_items = get_config(\"AzureSentinel.Workspaces\")[ws_param.value]\n",
        "\n",
        "try:\n",
        "    sent_prov = MicrosoftSentinel(ws_name=ws_param.value)\n",
        "    sent_prov.connect()\n",
        "except KeyError as e:\n",
        "    raise MsticpyUserConfigError(\"Unable to retreive Sentinel workspace items from config. Ensure you have SubscriptionId, ResourceGroup and WorkspaceName specified.\") from e"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "gather": {
          "logged": 1669671732972
        }
      },
      "outputs": [],
      "source": [
        "from msticpy.common.timespan import TimeSpan\n",
        "\n",
        "#Authentication\n",
        "qry_prov = QueryProvider(\"MSSentinel\")\n",
        "qry_prov.connect(WorkspaceConfig(workspace=ws_param.value))\n",
        "nb_timespan = TimeSpan(start, end)\n",
        "qry_prov.query_time.timespan = nb_timespan\n",
        "md(\"<hr>\")\n",
        "md(\"Confirm time range to search\", \"bold\")\n",
        "qry_prov.query_time"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### Authentication and Configuration Problems\n",
        "\n",
        "<br>\n",
        "<details>\n",
        "    <summary>Click for details about configuring your authentication parameters</summary>\n",
        "    \n",
        "    \n",
        "The notebook is expecting your Microsoft Sentinel Tenant ID and Workspace ID to be configured in one of the following places:\n",
        "- `config.json` in the current folder\n",
        "- `msticpyconfig.yaml` in the current folder or location specified by `MSTICPYCONFIG` environment variable.\n",
        "    \n",
        "For help with setting up your `config.json` file (if this hasn't been done automatically) see the [`ConfiguringNotebookEnvironment`](https://github.com/Azure/Azure-Sentinel-Notebooks/blob/master/ConfiguringNotebookEnvironment.ipynb) notebook in the root folder of your Azure-Sentinel-Notebooks project. This shows you how to obtain your Workspace and Subscription IDs from the Microsoft Sentinel Portal. You can use the SubscriptionID to find your Tenant ID). To view the current `config.json` run the following in a code cell.\n",
        "\n",
        "```%pfile config.json```\n",
        "\n",
        "For help with setting up your `msticpyconfig.yaml` see the [Setup](#Setup) section at the end of this notebook and the [ConfigureNotebookEnvironment notebook](https://github.com/Azure/Azure-Sentinel-Notebooks/blob/master/ConfiguringNotebookEnvironment.ipynb)\n",
        "</details>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Import and initialize notebooklets\n",
        "\n",
        "This imports the **msticnb** package and the notebooklets classes.\n",
        "\n",
        "These are needed for the notebook operations"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "gather": {
          "logged": 1669671733398
        }
      },
      "outputs": [],
      "source": [
        "import msticnb as nb\n",
        "\n",
        "nb.init(query_provider=qry_prov)\n",
        "ti = nb.DataProviders.instance.tilookup\n",
        "timespan = TimeSpan(start=start, end=end)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": []
      },
      "source": [
        "\n",
        "### Timeline View of Fusion Incidents\n",
        "This timeline shows you all fusion incidents in the selected workspace, grouped by the severity of the incidents."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "nteract": {
          "transient": {
            "deleting": false
          }
        }
      },
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "gather": {
          "logged": 1669671734147
        }
      },
      "outputs": [],
      "source": [
        "from msticpy.vis.timeline import display_timeline\n",
        "params = {\"$top\" : 500 , \"$filter\": f\"properties/relatedAnalyticRuleIds/any(x:x eq 'BuiltInFusion') and properties/createdTimeUtc gt {start.isoformat()} and properties/createdTimeUtc lt {end.isoformat()}\"}\n",
        "\n",
        "incidents = sent_prov.list_incidents(params)\n",
        "if isinstance(incidents, pd.DataFrame) and not incidents.empty:\n",
        "    incidents['Title'] = incidents[\"properties.title\"]\n",
        "    incidents['Status'] = incidents[\"properties.status\"]\n",
        "    incidents['date'] = incidents[\"properties.createdTimeUtc\"]\n",
        "    display_timeline(\n",
        "        data=incidents,\n",
        "        source_columns=[\"Title\", \"Status\"],\n",
        "        title=\"Fusion Incidents over time - grouped by severity\",\n",
        "        height=300,\n",
        "        group_by=\"properties.severity\",\n",
        "        time_column=\"date\",\n",
        "    )\n",
        "else:\n",
        "    md(\"No incidents found\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "\n",
        "### Select Fusion Incident to Investigate\n",
        "From the table below select the incident you wish to investigate."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "gather": {
          "logged": 1669671735268
        }
      },
      "outputs": [],
      "source": [
        "from IPython.display import HTML\n",
        "md(\"Select an incident to triage:\", \"bold\")\n",
        "\n",
        "def display_incident(incident):\n",
        "    details = f\"\"\"\n",
        "\n",
        "Selected Incident:\n",
        "{incident['properties.title']},\n",
        "            Incident time:  {incident['properties.createdTimeUtc']} -\n",
        "            Severity:  {incident['properties.severity']} -\n",
        "            Assingned to: {incident['properties.owner.userPrincipalName']} -\n",
        "            Status:  {incident['properties.status']}\n",
        "            \"\"\"\n",
        "    new_idx = [idx.split(\".\")[-1] for idx in incident.index]\n",
        "    incident.set_axis(new_idx, inplace=True)\n",
        "    return (HTML(details),pd.DataFrame(incident))\n",
        "\n",
        "filtered_incidents = incidents\n",
        "filtered_incidents[\"short_id\"] = filtered_incidents[\"id\"].apply(\n",
        "    lambda x: x.split(\"/\")[-1]\n",
        ")\n",
        "\n",
        "alert_sel = SelectAlert(\n",
        "    alerts=filtered_incidents,\n",
        "    default_alert=incident_id,\n",
        "    columns=[\"properties.title\", \"properties.severity\", \"properties.status\", \"name\"],\n",
        "    time_col=\"properties.createdTimeUtc\",\n",
        "    id_col=\"short_id\",\n",
        "    action=display_incident,\n",
        ")\n",
        "alert_sel.display()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Fusion creates correlations on entites including host, account, IP addresses and azure resources. To investigate Fusion incidents, we recommend you to start the investigaiton with the joined entites.\n",
        "The cell below shows you key details and context relating to this fusion incident, including:\n",
        "- **All the associated entities**: with column 'IsFusedEntity' indicating if the incident is fused on the entity\n",
        "- **Summary of associated incidents created on those entities in last 14 days**: with the column \"Number of associated incidents\", \"status of associated incidents\" and \"classification of associated incidents\"\n",
        "- **Related alerts**: including the expanded alerts that are indicated in the column 'IsExpandedAlert', which fired on the Fused entities in last 7 days if exists"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "gather": {
          "logged": 1669675835689
        }
      },
      "outputs": [],
      "source": [
        "incident_details = sent_prov.get_incident(\n",
        "    alert_sel.selected_alert.id.split(\"/\")[-1], entities=True, alerts=True\n",
        ")\n",
        "ent_dfs = []\n",
        "alert_out = []\n",
        "full_alerts = []\n",
        "#fused entities\n",
        "all_entities_count = {}\n",
        "fusion_join_ent_map = {\n",
        "    \"azure-resource\": \"resourceid\",\n",
        "    \"azureresource\": \"resourceid\",\n",
        "    \"account\": \"aaduserid\",\n",
        "    \"host\": \"hostname\",\n",
        "    \"ip\": \"address\",\n",
        "    \"url\": \"url\",\n",
        "    \"iotdevice\": \"deviceid\"\n",
        "}\n",
        "\n",
        "\n",
        "\n",
        "if \"Alerts\" in incident_details.columns:\n",
        "    for alert in incident_details.iloc[0][\"Alerts\"]:\n",
        "        qry = f\"SecurityAlert | where TimeGenerated between((datetime({start})-7d)..datetime({end})) | where SystemAlertId == '{alert['ID']}'\"\n",
        "        df = qry_prov.exec_query(qry)\n",
        "        df[\"IsExpansionAlert\"] = False\n",
        "        full_alerts.append(df)\n",
        "        if df.empty or not df[\"Entities\"].iloc[0]:\n",
        "            alert_full = {\"ID\": alert[\"ID\"], \"Name\": alert[\"Name\"], \"Entities\": None}\n",
        "        else:\n",
        "            alert_full = {\n",
        "                \"ID\": alert[\"ID\"],\n",
        "                \"Name\": alert[\"Name\"],\n",
        "                \"Entities\": json.loads(df[\"Entities\"].iloc[0].lower()),\n",
        "            }\n",
        "            for ent in alert_full['Entities']:\n",
        "                if 'type' in ent:\n",
        "                    ent_type = ent['type'].lower()\n",
        "                    if ent_type in fusion_join_ent_map:\n",
        "                        if ent_type == 'azure-resource':\n",
        "                            rg = extract_resourcegroup(ent[fusion_join_ent_map[ent_type]])\n",
        "                            if rg:\n",
        "                                all_entities_count[rg] = all_entities_count.get(rg, 0) + 1\n",
        "                        elif  ent_type == 'account' and  \"objectguid\" in ent:\n",
        "                            ent_key = ent['objectguid']\n",
        "                            all_entities_count[ent_key] = all_entities_count.get(ent_key, 0) + 1\n",
        "                        elif fusion_join_ent_map[ent_type] in ent:\n",
        "                            ent_key = ent[fusion_join_ent_map[ent_type]]\n",
        "                            all_entities_count[ent_key] = all_entities_count.get(ent_key, 0) + 1\n",
        "\n",
        "    incident_details[\"Alerts\"] = [alert_out]\n",
        "\n",
        "full_alerts = pd.concat(full_alerts, axis=0, ignore_index=True)\n",
        "full_alerts.sort_values(by=['SystemAlertId','TimeGenerated'])\n",
        "full_alerts = full_alerts.drop_duplicates(subset=['SystemAlertId'],keep='last')\n",
        "# expanded alerts on fused entities\n",
        "incident_created_time = incident_details.iloc[0]['properties.createdTimeUtc']\n",
        "for (ent_key,count) in all_entities_count.items():\n",
        "    if count > 1:\n",
        "        query = f\" SecurityAlert | where TimeGenerated between((datetime({incident_created_time})-7d)..(datetime({incident_created_time})+7d)) | where Entities contains '{ent_key}'\"\n",
        "        alerts_df = qry_prov.exec_query(query)\n",
        "        alerts_df[\"IsExpansionAlert\"] = False\n",
        "        if isinstance(alerts_df, pd.DataFrame) and not alerts_df.empty:\n",
        "            for idx, one_alert in alerts_df.iterrows():\n",
        "                if not (one_alert['SystemAlertId'] in set(alert_full['ID'])):\n",
        "                    one_alert['IsExpansionAlert'] = True\n",
        "                    full_alerts = full_alerts.append(one_alert, ignore_index=True)\n",
        "                    alert_full = {\n",
        "                        \"ID\": one_alert[\"SystemAlertId\"],\n",
        "                        \"Name\": alert[\"Name\"],\n",
        "                        \"Entities\": json.loads(one_alert[\"Entities\"].lower()),\n",
        "                    }\n",
        "                    alert_out.append(alert_full)\n",
        "\n",
        "\n",
        "# query all the incidents contains those entities and summarize their status\n",
        "incidents = sent_prov.get_incidents()\n",
        "all_inincident_details = []\n",
        "if isinstance(incidents, pd.DataFrame) and not incidents.empty:\n",
        "    incidents[\"date\"] = pd.to_datetime(incidents[\"properties.createdTimeUtc\"], utc=True)\n",
        "    filtered_incidents=incidents[incidents[\"date\"].between(pd.to_datetime(incident_created_time)- timedelta(days=14), pd.to_datetime(incident_created_time) + timedelta(days=7))]\n",
        "    for idx, incident_tmp in filtered_incidents.iterrows():\n",
        "        incident_details_tmp = sent_prov.get_incident(incident_tmp.id.split(\"/\")[-1], entities=True)\n",
        "        all_inincident_details.append(incident_details_tmp)\n",
        "all_inincident_details_df = pd.concat(all_inincident_details, axis=0, ignore_index=True)\n",
        "\n",
        "ent_map = {\n",
        "    \"FileHash\": \"hashValue\",\n",
        "    \"Malware\": \"malwareName\",\n",
        "    \"File\": \"fileName\",\n",
        "    \"CloudApplication\": \"appId\",\n",
        "    \"AzureResource\": \"resourceId\",\n",
        "    \"RegistryValue\": \"registryName\",\n",
        "    \"SecurityGroup\": \"SID\",\n",
        "    \"IoTDevice\": \"deviceId\",\n",
        "    \"Mailbox\": \"mailboxPrimaryAddress\",\n",
        "    \"MailMessage\": \"networkMessageId\",\n",
        "    \"SubmissionMail\": \"submissionId\",\n",
        "    \"Account\": \"accountName\",\n",
        "    \"Host\": \"hostName\",\n",
        "    \"Ip\": \"address\",\n",
        "}\n",
        "\n",
        "for ent in incident_details[\"Entities\"][0]:\n",
        "    all_inincident_details_matched = pd.DataFrame()\n",
        "    entities = {}\n",
        "    for k, v in ent[1].items():\n",
        "        entities[k.lower()] = v\n",
        "    ent_df = pd.json_normalize(entities)\n",
        "    ent_type = ent[0].lower()\n",
        "    ent_df[\"type\"] = ent_type\n",
        "    ent_df['IsFusedEntity'] = False\n",
        "    ent_key = None\n",
        "    if ent_type in fusion_join_ent_map:\n",
        "        if ent_type == 'azureresource':\n",
        "            ent_key = ent_df[fusion_join_ent_map[ent_type]].values[0]\n",
        "            rg = extract_resourcegroup(ent_df[fusion_join_ent_map[ent_type]].values[0])\n",
        "            if rg and all_entities_count.get(rg, 0) > 1:\n",
        "                ent_df['IsFusedEntity'] = True\n",
        "        elif  ent_type == 'account' and  \"objectguid\" in ent_df:\n",
        "            ent_key = ent_df['objectguid'].values[0]\n",
        "            if all_entities_count.get(ent_key, 0) > 1:\n",
        "                ent_df['IsFusedEntity'] = True\n",
        "        elif fusion_join_ent_map[ent_type] in ent_df:\n",
        "            ent_key = ent_df[fusion_join_ent_map[ent_type]].values[0]\n",
        "            if all_entities_count.get(ent_key, 0) > 1:\n",
        "                ent_df['IsFusedEntity'] = True\n",
        "    elif ent[0] in ent_map:\n",
        "        try:\n",
        "            ent_key = ent[1][ent_map[ent[0]]]\n",
        "        except:\n",
        "            ent_key = None\n",
        "    # check if the incident have the entity\n",
        "    incident_id_set = set([incident_details.iloc[0]['id']])\n",
        "\n",
        "    if ent_key != None:\n",
        "        for idx, incident_details_tmp in all_inincident_details_df.iterrows():\n",
        "            for ent_tmp in incident_details_tmp['Entities']:\n",
        "                if ent_type == ent_tmp[0].lower() and not incident_details_tmp['id'] in incident_id_set:\n",
        "                    for k, v in ent_tmp[1].items():\n",
        "                        if  ent_key in v.lower():\n",
        "                            all_inincident_details_matched = all_inincident_details_matched.append(incident_details_tmp, ignore_index=True)\n",
        "                            incident_id_set.add(incident_details_tmp['id'])\n",
        "\n",
        "    #summarize the incidents created on the entity\n",
        "    associated_incidents_status = \"none\"\n",
        "    associated_incidents_classification = \"none\"\n",
        "    count_associated_incidents = 0\n",
        "    if not all_inincident_details_matched.empty:\n",
        "        count_associated_incidents = len(all_inincident_details_matched.index)\n",
        "        associated_incidents_status = json.dumps(all_inincident_details_matched['properties.status'].value_counts().to_dict())\n",
        "        if \"properties.classification\" in all_inincident_details_matched.columns:\n",
        "            associated_incidents_classification = json.dumps(all_inincident_details_matched['properties.classification'].value_counts().to_dict())\n",
        "\n",
        "    ent_df['Number of associated incidents'] = count_associated_incidents\n",
        "    ent_df['Status of associated incidents'] = associated_incidents_status\n",
        "    ent_df['Classification of associated incidents'] = associated_incidents_classification\n",
        "    ent_dfs.append(ent_df)\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "md(\"Incident Entities:\", \"bold\")\n",
        "if ent_dfs:\n",
        "    new_df = pd.concat(ent_dfs, axis=0, ignore_index=True)\n",
        "    grp_df = new_df.groupby(\"type\")\n",
        "    for grp in grp_df:\n",
        "        md(grp[0], \"bold\")\n",
        "        display(grp[1].dropna(axis=1))\n",
        "\n",
        "md(\"Graph of incident entities:\", \"bold\")\n",
        "graph = EntityGraph(incident_details.iloc[0])\n",
        "graph.plot(timeline=True)\n",
        "\n",
        "md(\"Fusion incidents with expanded alerts:\", \"bold\")\n",
        "full_alerts.sort_values(by='TimeGenerated')\n",
        "display(full_alerts)\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### Entity Analysis\n",
        "Below is an analysis of the incident's entities that appear in threat intelligence sources."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "gather": {
          "logged": 1669675509820
        },
        "tags": [
          "output"
        ]
      },
      "outputs": [],
      "source": [
        "sev = []\n",
        "resps = pd.DataFrame()\n",
        "\n",
        "# For each entity look it up in Threat Intelligence data\n",
        "md(\"Looking up entities in TI feeds...\")\n",
        "prog = Progress(completed_len=len(incident_details[\"Entities\"].iloc[0]))\n",
        "i = 0\n",
        "result_dfs = []\n",
        "for ent in incident_details[\"Entities\"].iloc[0]:\n",
        "    i += 1\n",
        "    prog.update_progress(i)\n",
        "    if ent[0] == \"Ip\":\n",
        "        resp = ti.lookup_ioc(observable=ent[1][\"address\"], ioc_type=\"ipv4\")\n",
        "        result_dfs.append(ti.result_to_df(resp))\n",
        "        for response in resp[1]:\n",
        "            sev.append(response[1].severity)\n",
        "    if ent[0] == \"Url\" or ent[0] == \"DnsResolution\":\n",
        "        if \"url\" in ent[1]:\n",
        "            lkup_dom = ent[1][\"url\"]\n",
        "        else:\n",
        "            lkup_dom = ent[1][\"domainName\"]\n",
        "        resp = ti.lookup_ioc(lkup_dom, ioc_type=\"url\")\n",
        "        result_dfs.append(ti.result_to_df(resp))\n",
        "        for response in resp[1]:\n",
        "            sev.append(response[1].severity)\n",
        "    if ent[0] == \"FileHash\":\n",
        "        resp = ti.lookup_ioc(ent[1][\"hashValue\"])\n",
        "        result_dfs.append(ti.result_to_df(resp))\n",
        "        for response in resp[1]:\n",
        "            sev.append(response[1].severity)\n",
        "    if result_dfs:\n",
        "        resps = pd.concat(result_dfs)\n",
        "    else:\n",
        "        resps = pd.DataFrame()\n",
        "\n",
        "# Take overall severity of the entities based on the highest score\n",
        "if \"high\" in sev:\n",
        "    severity = \"High\"\n",
        "elif \"warning\" in sev:\n",
        "    severity = \"Warning\"\n",
        "elif \"information\" in sev:\n",
        "    severity = \"Information\"\n",
        "else:\n",
        "    severity = \"None\"\n",
        "\n",
        "md(\"Checking to see if incident entities appear in TI data...\")\n",
        "\n",
        "incident_details[\"TI Severity\"] = severity\n",
        "# Output TI hits of high or warning severity\n",
        "if (\n",
        "    incident_details[\"TI Severity\"].iloc[0] == \"High\"\n",
        "    or incident_details[\"TI Severity\"].iloc[0] == \"Warning\"\n",
        "    or incident_details[\"TI Severity\"].iloc[0] == \"Information\"\n",
        "):\n",
        "    print(\"Incident:\")\n",
        "    display(\n",
        "        incident_details[\n",
        "            [\n",
        "                \"properties.createdTimeUtc\",\n",
        "                \"properties.incidentNumber\",\n",
        "                \"properties.title\",\n",
        "                \"properties.status\",\n",
        "                \"properties.severity\",\n",
        "                \"TI Severity\",\n",
        "            ]\n",
        "        ]\n",
        "        .style.applymap(ti_color_cells)\n",
        "        .hide(axis='index')\n",
        "    )\n",
        "    md(\"TI Results:\", \"bold\")\n",
        "    display(\n",
        "        resps[[\"Ioc\", \"IocType\", \"Provider\", \"Severity\", \"Details\"]]\n",
        "        .sort_values(by=\"Severity\")\n",
        "        .style.applymap(ti_color_cells)\n",
        "        .hide(axis='index')\n",
        "    )\n",
        "else:\n",
        "    md(\"None of the Entities appeared in TI data\", \"bold\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### IP Entity Analysis\n",
        "Below is an analysis of all IP entities attached to the incident."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "gather": {
          "logged": 1669678503574
        }
      },
      "outputs": [],
      "source": [
        "# Enrich IP entities using the IP Summary notebooklet\n",
        "ip_ent_nb = nb.nblts.azsent.network.IpAddressSummary()\n",
        "\n",
        "if not resps.empty and \"ipv4\" in resps[\"IocType\"].unique():\n",
        "    for ip_addr in resps[resps[\"IocType\"] == \"ipv4\"][\"Ioc\"].unique():\n",
        "        folium_map = FoliumMap(width=\"50%\", height=\"50%\")\n",
        "        try:\n",
        "            display(HTML(f\"<h1>Summary of Activity Related to{ip_addr}:</h1>\"))\n",
        "            ip_ent_nb_out = ip_ent_nb.run(value=ip_addr, timespan=timespan, silent=True)\n",
        "            md(\n",
        "                f\"{ip_addr} - {ip_ent_nb_out.ip_origin} - {ip_ent_nb_out.ip_type}\",\n",
        "                \"bold\",\n",
        "            )\n",
        "            if (\n",
        "                isinstance(ip_ent_nb_out.whois, pd.DataFrame)\n",
        "                and not ip_ent_nb_out.whois.empty\n",
        "            ):\n",
        "                md(f\"Whois information for {ip_addr}\", \"bold\")\n",
        "                display(ip_ent_nb_out.whois)\n",
        "            if ip_ent_nb_out.location:\n",
        "                md(f\"Geo IP details for {ip_addr}\", \"bold\")\n",
        "                folium_map.add_ip_cluster([ip_ent_nb_out.ip_entity])\n",
        "                display(folium_map)\n",
        "            if (\n",
        "                isinstance(ip_ent_nb_out.related_alerts, pd.DataFrame)\n",
        "                and not ip_ent_nb_out.related_alerts.empty\n",
        "            ):\n",
        "                md(f\"Alerts for {ip_addr}\", \"bold\")\n",
        "                tl = nbdisplay.display_timeline(\n",
        "                    data=ip_ent_nb_out.related_alerts,\n",
        "                    source_columns=[\"AlertName\", \"Severity\"],\n",
        "                    title=f\"Alerts associated with {ip_addr}\",\n",
        "                    height=300,\n",
        "                )\n",
        "                display(tl)\n",
        "            if (\n",
        "                isinstance(ip_ent_nb_out.ti_results, pd.DataFrame)\n",
        "                and not ip_ent_nb_out.ti_results.empty\n",
        "            ):\n",
        "                md(f\"TI results for {ip_addr}\", \"bold\")\n",
        "                display(ip_ent_nb_out.ti_results)\n",
        "            if (\n",
        "                isinstance(ip_ent_nb_out.passive_dns, pd.DataFrame)\n",
        "                and not ip_ent_nb_out.passive_dns.empty\n",
        "            ):\n",
        "                md(f\"Passive DNS results for {ip_addr}\", \"bold\")\n",
        "                display(ip_ent_nb_out.passive_dns)\n",
        "            if ip_ent_nb_out.host_entities[0][\"IpAddresses\"]:\n",
        "                md(f\"{ip_addr} belongs to a known host\", \"bold\")\n",
        "                display(\n",
        "                    pd.DataFrame.from_records(\n",
        "                        [\n",
        "                            {\n",
        "                                x: ip_ent_nb_out.host_entities[x]\n",
        "                                for x in ip_ent_nb_out.host_entities.__iter__()\n",
        "                            }\n",
        "                        ]\n",
        "                    )\n",
        "                )\n",
        "            print(\n",
        "                \"------------------------------------------------------------------------------------------------------------------------------------------------------------------\"\n",
        "            )\n",
        "            display(HTML(\"<br><br>\"))\n",
        "        except:\n",
        "            md(f\"Error processing {ip_addr}\", \"bold\")\n",
        "else:\n",
        "    md(\"No IP entities present\", \"bold\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### URL Entity Analysis\n",
        "Below is an analysis of all URL entities attached to the incident."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "gather": {
          "logged": 1669671750633
        }
      },
      "outputs": [],
      "source": [
        "url_nb = nb.nblts.azsent.url.URLSummary()\n",
        "\n",
        "domain_records = pd.DataFrame()\n",
        "if not resps.empty and \"url\" in resps[\"IocType\"].unique():\n",
        "    md(\"Domain entity enrichment\", \"bold\")\n",
        "    for url in resps[resps[\"IocType\"] == \"url\"][\"Ioc\"].unique():\n",
        "        md(f\"Summary of {url}\", \"bold\")\n",
        "        url_nb_result = url_nb.run(timespan=timespan, value=url)\n",
        "        url_nb_result.display_alert_timeline()\n",
        "        url_nb_result.browse()\n",
        "else:\n",
        "    md(\"No URL entities present\", \"bold\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### User Entity Analysis\n",
        "Below is an analysis of all User entities attached to the incident."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "gather": {
          "logged": 1669671750672
        }
      },
      "outputs": [],
      "source": [
        "# Enrich Account entities using the AccountSummary notebooklet\n",
        "account_nb = nb.nblts.azsent.account.AccountSummary()\n",
        "user = None\n",
        "uent = None\n",
        "\n",
        "if check_ent(incident_details[\"Entities\"][0], \"account\") or check_ent(\n",
        "    incident_details[\"Entities\"][0], \"mailbox\"\n",
        "):\n",
        "    md(\"Account entity enrichment\", \"bold\")\n",
        "    for ent in incident_details[\"Entities\"][0]:\n",
        "        if ent[0] == \"Account\" or ent[0] == \"Mailbox\":\n",
        "            if \"accountName\" in ent[1].keys():\n",
        "                uent = ent[1][\"accountName\"]\n",
        "            elif \"aadUserId\" in ent[1].keys():\n",
        "                uent = ent[1][\"aadUserId\"]\n",
        "            elif \"upn\" in ent[1].keys():\n",
        "                uent = ent[1][\"upn\"]\n",
        "            if \"upnSuffix\" in ent[1].keys():\n",
        "                user = uent + \"@\" + ent[1][\"upnSuffix\"]\n",
        "            else:\n",
        "                user = uent\n",
        "            if user:\n",
        "                try:\n",
        "                    ac_nb = account_nb.run(timespan=timespan, value=user, silent=True)\n",
        "                    ac_nb.get_additional_data()\n",
        "                    if (\n",
        "                        isinstance(ac_nb.account_activity, pd.DataFrame)\n",
        "                        and not ac_nb.account_activity.empty\n",
        "                    ):\n",
        "                        display(HTML(f\"<h1>Summary of Activity Related to {user}:</h1>\"))\n",
        "                        md(\"Recent activity\")\n",
        "                        display(ac_nb.account_activity)\n",
        "                    else:\n",
        "                        md(f\"No activity found for {user}\")\n",
        "                    if (\n",
        "                        isinstance(ac_nb.related_alerts, pd.DataFrame)\n",
        "                        and not ac_nb.related_alerts.empty\n",
        "                    ):\n",
        "                        show(ac_nb.alert_timeline)\n",
        "                    if (\n",
        "                        isinstance(ac_nb.host_logon_summary, pd.DataFrame)\n",
        "                        and not ac_nb.host_logon_summary.empty\n",
        "                    ):\n",
        "                        md(f\"Host logons by {user}\")\n",
        "                        display(ac_nb.host_logon_summary)\n",
        "                    if (\n",
        "                        isinstance(ac_nb.azure_activity_summary, pd.DataFrame)\n",
        "                        and not ac_nb.azure_activity_summary.empty\n",
        "                    ):\n",
        "                        md(f\"Azure activity by {user}\")\n",
        "                        display(ac_nb.azure_activity_summary)\n",
        "                        show(ac_nb.azure_timeline_by_provider)\n",
        "                    if (\n",
        "                        isinstance(ac_nb.ip_summary, pd.DataFrame)\n",
        "                        and not ac_nb.ip_summary.empty\n",
        "                    ):\n",
        "                        md(f\"IPs used by {user}\")\n",
        "                        display(ac_nb.ip_summary)\n",
        "                except:\n",
        "                    print(f\"Error processing {user}\")\n",
        "else:\n",
        "    md(\"No Account entities present\", \"bold\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### Host Entity Analysis\n",
        "Below is an analysis of all Host entities attached to the incident."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "gather": {
          "logged": 1669671750720
        }
      },
      "outputs": [],
      "source": [
        "# Enrich Host entities using the HostSummary notebooklet\n",
        "host_nb = nb.nblts.azsent.host.HostSummary()\n",
        "\n",
        "if check_ent(incident_details[\"Entities\"][0], \"host\"):\n",
        "    md(\"Host entity enrichment\", \"bold\")\n",
        "    for ent in incident_details[\"Entities\"][0]:\n",
        "        if ent[0] == \"Host\":\n",
        "            if \"dnsDomain\" in ent[1]:\n",
        "                host_name = ent[1][\"hostName\"] + \".\" + ent[1][\"dnsDomain\"], \"\"\n",
        "            else:\n",
        "                host_name = ent[1][\"hostName\"]\n",
        "            md(f\"Host summary for {host_name}\", \"bold\")\n",
        "            try:\n",
        "                display(HTML(f\"<h1>Summary of Activity Related to{host_name}:</h1>\"))\n",
        "                host_sum_out = host_nb.run(value=host_name, timespan=timespan)\n",
        "            except:\n",
        "                print(f\"Error processing {host_name}\")\n",
        "else:\n",
        "    md(\"No Host entities present\", \"bold\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### Timeline of other alerts with the same entities\n",
        "If there are other entity types not analyzed above, a timeline of their appearance in security alerts appears below."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "gather": {
          "logged": 1669671750744
        }
      },
      "outputs": [],
      "source": [
        "ent_map = {\n",
        "    \"FileHash\": \"hashValue\",\n",
        "    \"Malware\": \"malwareName\",\n",
        "    \"File\": \"fileName\",\n",
        "    \"CloudApplication\": \"appId\",\n",
        "    \"AzureResource\": \"resourceId\",\n",
        "    \"RegistryValue\": \"registryName\",\n",
        "    \"SecurityGroup\": \"SID\",\n",
        "    \"IoTDevice\": \"deviceId\",\n",
        "    \"Mailbox\": \"mailboxPrimaryAddress\",\n",
        "    \"MailMessage\": \"networkMessageId\",\n",
        "    \"SubmissionMail\": \"submissionId\",\n",
        "    \"Account\": \"accountName\",\n",
        "    \"Host\": \"hostName\",\n",
        "    \"Ip\": \"address\",\n",
        "}\n",
        "for ent in incident_details[\"Entities\"][0]:\n",
        "    if ent[0] in ent_map:\n",
        "        ent_alerts(ent[1][ent_map[ent[0]]])"
      ]
    }
  ],
  "metadata": {
    "celltoolbar": "Tags",
    "kernel_info": {
	  "name": "python310-sdkv2"
    },
	"kernelspec": {
	 "display_name": "Python 3.10 - SDK v2",
	  "language": "python",
	  "name": "python310-sdkv2"
	},
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.8.5"
    },
    "microsoft": {
      "host": {
        "AzureML": {
          "notebookHasBeenCompleted": true
        }
      }
    },
    "nteract": {
      "version": "nteract-front-end@1.0.0"
    },
    "vscode": {
      "interpreter": {
        "hash": "0a54084e6b208ee8d1ce3989ffc20924477a5f55f5a43e22e699a6741623861e"
      }
    },
    "widgets": {
      "application/vnd.jupyter.widget-state+json": {
        "state": {
          "0040cad1767f4b95a00aefbc70d06bc1": {
            "model_module": "@jupyter-widgets/base",
            "model_module_version": "1.2.0",
            "model_name": "LayoutModel",
            "state": {}
          },
          "01069a28b66840f5908dbf5167e08482": {
            "model_module": "@jupyter-widgets/controls",
            "model_module_version": "1.5.0",
            "model_name": "DescriptionStyleModel",
            "state": {
              "description_width": ""
            }
          },
          "14eaa102acb64e0b8073da21f45e83b5": {
            "model_module": "@jupyter-widgets/controls",
            "model_module_version": "1.5.0",
            "model_name": "VBoxModel",
            "state": {
              "children": [
                "IPY_MODEL_e28b52485fcb486c91ec902f5b7c61db",
                "IPY_MODEL_db98f91c5e394abeb6b76ed5fb4284ad",
                "IPY_MODEL_6125ee42c10541e3bca956f190da4496"
              ],
              "layout": "IPY_MODEL_47880b93603046cea618b9d10a7f192c"
            }
          },
          "195c48cd5c414746b418dac44c964c18": {
            "model_module": "@jupyter-widgets/base",
            "model_module_version": "1.2.0",
            "model_name": "LayoutModel",
            "state": {
              "width": "100px"
            }
          },
          "2483892439044ba69f374f2c0bb01c72": {
            "model_module": "@jupyter-widgets/controls",
            "model_module_version": "1.5.0",
            "model_name": "DescriptionStyleModel",
            "state": {
              "description_width": ""
            }
          },
          "2b5f387ba79b44b9a764af8c44827521": {
            "model_module": "@jupyter-widgets/controls",
            "model_module_version": "1.5.0",
            "model_name": "LabelModel",
            "state": {
              "layout": "IPY_MODEL_af3398b31ebc4934a2dc1ed71efeb55c",
              "style": "IPY_MODEL_01069a28b66840f5908dbf5167e08482"
            }
          },
          "3338fc72841a43b983a7a5d7ab114635": {
            "model_module": "@jupyter-widgets/controls",
            "model_module_version": "1.5.0",
            "model_name": "DescriptionStyleModel",
            "state": {
              "description_width": "initial"
            }
          },
          "34aa9860ecc94a71b4ec1920d9554694": {
            "model_module": "@jupyter-widgets/base",
            "model_module_version": "1.2.0",
            "model_name": "LayoutModel",
            "state": {
              "width": "70%"
            }
          },
          "3c3908a4efe34c62a0a008f31eefd1c8": {
            "model_module": "@jupyter-widgets/controls",
            "model_module_version": "1.5.0",
            "model_name": "HBoxModel",
            "state": {
              "children": [
                "IPY_MODEL_d8ba0aa2aad54cd0aa69760a5354b452",
                "IPY_MODEL_94d2f3d7a77d475fa56029c6c1b102a2"
              ],
              "layout": "IPY_MODEL_b108e5c7f71d41aa896d812172987157"
            }
          },
          "3c990d1c0dac4fac9be32569b11c773a": {
            "model_module": "@jupyter-widgets/controls",
            "model_module_version": "1.5.0",
            "model_name": "DescriptionStyleModel",
            "state": {
              "description_width": "initial"
            }
          },
          "3de1a85313a54edab63eab8a97d013cd": {
            "model_module": "@jupyter-widgets/controls",
            "model_module_version": "1.5.0",
            "model_name": "DescriptionStyleModel",
            "state": {
              "description_width": ""
            }
          },
          "41ce3c950bf641e982dd88dc1439cc85": {
            "model_module": "@jupyter-widgets/controls",
            "model_module_version": "1.5.0",
            "model_name": "VBoxModel",
            "state": {
              "children": [
                "IPY_MODEL_6d846f2a40af446183bbc9999494c25c",
                "IPY_MODEL_db4165bc3d9e4a3182d06b7fca3178e4",
                "IPY_MODEL_aad0360f0cb94309b96360ef49ef7474"
              ],
              "layout": "IPY_MODEL_7601235d1fbb40478da390e5c635bde4"
            }
          },
          "424d17d0ee4b4d929ea90e253f069ed7": {
            "model_module": "@jupyter-widgets/controls",
            "model_module_version": "1.5.0",
            "model_name": "IntRangeSliderModel",
            "state": {
              "_model_name": "IntRangeSliderModel",
              "_view_name": "IntRangeSliderView",
              "description": "Time Range",
              "layout": "IPY_MODEL_847bf20ca3504a108341090f44a5d399",
              "max": 28,
              "min": -28,
              "style": "IPY_MODEL_ecd5058213f7439db39bd205512843e5",
              "value": [
                -1,
                1
              ]
            }
          },
          "442fd1263e334d45ad7d171c3457d7cd": {
            "model_module": "@jupyter-widgets/controls",
            "model_module_version": "1.5.0",
            "model_name": "TextModel",
            "state": {
              "description": "Time (24hr)",
              "layout": "IPY_MODEL_d1f7f46ac3b64c818536834c60392fea",
              "style": "IPY_MODEL_3de1a85313a54edab63eab8a97d013cd",
              "value": "22:38:45.575783"
            }
          },
          "47880b93603046cea618b9d10a7f192c": {
            "model_module": "@jupyter-widgets/base",
            "model_module_version": "1.2.0",
            "model_name": "LayoutModel",
            "state": {}
          },
          "4bf47ca149934c799ae842ef53122f0d": {
            "model_module": "@jupyter-widgets/base",
            "model_module_version": "1.2.0",
            "model_name": "LayoutModel",
            "state": {
              "width": "50%"
            }
          },
          "511af61975284928bb839f6a946ae2b6": {
            "model_module": "@jupyter-widgets/controls",
            "model_module_version": "1.5.0",
            "model_name": "DescriptionStyleModel",
            "state": {
              "description_width": ""
            }
          },
          "56dbb96a024b4062bb16295c93bb8386": {
            "model_module": "@jupyter-widgets/controls",
            "model_module_version": "1.5.0",
            "model_name": "DescriptionStyleModel",
            "state": {
              "description_width": ""
            }
          },
          "5d170286dedd41889a7345023f650a33": {
            "model_module": "@jupyter-widgets/base",
            "model_module_version": "1.2.0",
            "model_name": "LayoutModel",
            "state": {
              "width": "50%"
            }
          },
          "6125ee42c10541e3bca956f190da4496": {
            "model_module": "@jupyter-widgets/controls",
            "model_module_version": "1.5.0",
            "model_name": "TextModel",
            "state": {
              "description": "Query end time (UTC) :  ",
              "layout": "IPY_MODEL_4bf47ca149934c799ae842ef53122f0d",
              "style": "IPY_MODEL_c8d4203d9a214a2faf7c33159211f3ad",
              "value": "2021-07-03 22:38:35.047441"
            }
          },
          "621702c9f018481183b96930e706cc54": {
            "model_module": "@jupyter-widgets/base",
            "model_module_version": "1.2.0",
            "model_name": "LayoutModel",
            "state": {}
          },
          "6d846f2a40af446183bbc9999494c25c": {
            "model_module": "@jupyter-widgets/controls",
            "model_module_version": "1.5.0",
            "model_name": "HTMLModel",
            "state": {
              "layout": "IPY_MODEL_af92228a3694413aacca38bdf0d6db3a",
              "style": "IPY_MODEL_ff485c7bb935466a8425b6a4ae8f8da9",
              "value": "<h4>Set time range for pivot functions.</h4>"
            }
          },
          "722e044af71d41c6b6a82690cd8a577c": {
            "model_module": "@jupyter-widgets/controls",
            "model_module_version": "1.5.0",
            "model_name": "DropdownModel",
            "state": {
              "_options_labels": [
                "minute",
                "hour",
                "day",
                "week"
              ],
              "index": 2,
              "layout": "IPY_MODEL_96fa332bb44f43b19d7a373d509f7afb",
              "style": "IPY_MODEL_2483892439044ba69f374f2c0bb01c72"
            }
          },
          "7601235d1fbb40478da390e5c635bde4": {
            "model_module": "@jupyter-widgets/base",
            "model_module_version": "1.2.0",
            "model_name": "LayoutModel",
            "state": {}
          },
          "777e16e2c58745df8e5d76524f5a1433": {
            "model_module": "@jupyter-widgets/base",
            "model_module_version": "1.2.0",
            "model_name": "LayoutModel",
            "state": {}
          },
          "7a9485f145724c38b81876500dbb43f2": {
            "model_module": "@jupyter-widgets/controls",
            "model_module_version": "1.5.0",
            "model_name": "HTMLModel",
            "state": {
              "layout": "IPY_MODEL_fa1b0cab6957493e935f4e9a8d428cb2",
              "style": "IPY_MODEL_94439f6f9f9b400496af079cd6e37604",
              "value": "<h4>Set query time boundaries</h4>"
            }
          },
          "81676b130d4a4af3a1b88a9a378dd60d": {
            "model_module": "@jupyter-widgets/controls",
            "model_module_version": "1.5.0",
            "model_name": "DescriptionStyleModel",
            "state": {
              "description_width": ""
            }
          },
          "82a64b7fb54640eda6c0c87406c628d2": {
            "model_module": "@jupyter-widgets/base",
            "model_module_version": "1.2.0",
            "model_name": "LayoutModel",
            "state": {
              "width": "50%"
            }
          },
          "847bf20ca3504a108341090f44a5d399": {
            "model_module": "@jupyter-widgets/base",
            "model_module_version": "1.2.0",
            "model_name": "LayoutModel",
            "state": {
              "width": "70%"
            }
          },
          "8c8eab82b42945dc89a91bbb570eb170": {
            "model_module": "@jupyter-widgets/controls",
            "model_module_version": "1.5.0",
            "model_name": "DescriptionStyleModel",
            "state": {
              "description_width": ""
            }
          },
          "8f45b292a74043d0a1cee653bc57689a": {
            "model_module": "@jupyter-widgets/base",
            "model_module_version": "1.2.0",
            "model_name": "LayoutModel",
            "state": {}
          },
          "8f4f079d2f044d36a7adfaf4efdd27cc": {
            "model_module": "@jupyter-widgets/base",
            "model_module_version": "1.2.0",
            "model_name": "LayoutModel",
            "state": {
              "width": "95%"
            }
          },
          "927b173de2034b0dbe37d9e1ae388c9a": {
            "model_module": "@jupyter-widgets/controls",
            "model_module_version": "1.5.0",
            "model_name": "VBoxModel",
            "state": {
              "children": [
                "IPY_MODEL_7a9485f145724c38b81876500dbb43f2",
                "IPY_MODEL_d1fa63f01abb4c59a0158af8c3af9d8d",
                "IPY_MODEL_14eaa102acb64e0b8073da21f45e83b5"
              ],
              "layout": "IPY_MODEL_8f45b292a74043d0a1cee653bc57689a"
            }
          },
          "94439f6f9f9b400496af079cd6e37604": {
            "model_module": "@jupyter-widgets/controls",
            "model_module_version": "1.5.0",
            "model_name": "DescriptionStyleModel",
            "state": {
              "description_width": ""
            }
          },
          "949f237e510542789ff9e149560d5181": {
            "model_module": "@jupyter-widgets/controls",
            "model_module_version": "1.5.0",
            "model_name": "DatePickerModel",
            "state": {
              "description": "Origin Date",
              "disabled": false,
              "layout": "IPY_MODEL_dcd9308cd0474c24827bcc6cec31e96d",
              "style": "IPY_MODEL_81676b130d4a4af3a1b88a9a378dd60d",
              "value": {
                "date": 2,
                "month": 6,
                "year": 2021
              }
            }
          },
          "94d2f3d7a77d475fa56029c6c1b102a2": {
            "model_module": "@jupyter-widgets/controls",
            "model_module_version": "1.5.0",
            "model_name": "DropdownModel",
            "state": {
              "_options_labels": [
                "minute",
                "hour",
                "day",
                "week"
              ],
              "index": 2,
              "layout": "IPY_MODEL_195c48cd5c414746b418dac44c964c18",
              "style": "IPY_MODEL_8c8eab82b42945dc89a91bbb570eb170"
            }
          },
          "96fa332bb44f43b19d7a373d509f7afb": {
            "model_module": "@jupyter-widgets/base",
            "model_module_version": "1.2.0",
            "model_name": "LayoutModel",
            "state": {
              "width": "100px"
            }
          },
          "a445ee0e426c4f13af21b260b700e05f": {
            "model_module": "@jupyter-widgets/base",
            "model_module_version": "1.2.0",
            "model_name": "LayoutModel",
            "state": {
              "width": "99%"
            }
          },
          "a4d3df3b4447488f93e32ac4b3ada8c9": {
            "model_module": "@jupyter-widgets/controls",
            "model_module_version": "1.5.0",
            "model_name": "SliderStyleModel",
            "state": {
              "description_width": "initial"
            }
          },
          "aad0360f0cb94309b96360ef49ef7474": {
            "model_module": "@jupyter-widgets/controls",
            "model_module_version": "1.5.0",
            "model_name": "VBoxModel",
            "state": {
              "children": [
                "IPY_MODEL_3c3908a4efe34c62a0a008f31eefd1c8",
                "IPY_MODEL_f5fe70128cdf4eadaf8455cdbbd719cb",
                "IPY_MODEL_ae8c36ba95ab4ffb883b6cef6ffa6ac7"
              ],
              "layout": "IPY_MODEL_777e16e2c58745df8e5d76524f5a1433"
            }
          },
          "ae8c36ba95ab4ffb883b6cef6ffa6ac7": {
            "model_module": "@jupyter-widgets/controls",
            "model_module_version": "1.5.0",
            "model_name": "TextModel",
            "state": {
              "description": "Query end time (UTC) :  ",
              "layout": "IPY_MODEL_82a64b7fb54640eda6c0c87406c628d2",
              "style": "IPY_MODEL_f85fee408dda49ffadbae38d3f49eab9",
              "value": "2021-07-02 22:38:45.575783"
            }
          },
          "af3398b31ebc4934a2dc1ed71efeb55c": {
            "model_module": "@jupyter-widgets/base",
            "model_module_version": "1.2.0",
            "model_name": "LayoutModel",
            "state": {
              "width": "99%"
            }
          },
          "af92228a3694413aacca38bdf0d6db3a": {
            "model_module": "@jupyter-widgets/base",
            "model_module_version": "1.2.0",
            "model_name": "LayoutModel",
            "state": {}
          },
          "b108e5c7f71d41aa896d812172987157": {
            "model_module": "@jupyter-widgets/base",
            "model_module_version": "1.2.0",
            "model_name": "LayoutModel",
            "state": {}
          },
          "b7cffa19b0144d8eb66f2c87e47282f6": {
            "model_module": "@jupyter-widgets/base",
            "model_module_version": "1.2.0",
            "model_name": "LayoutModel",
            "state": {
              "width": "50%"
            }
          },
          "c8d4203d9a214a2faf7c33159211f3ad": {
            "model_module": "@jupyter-widgets/controls",
            "model_module_version": "1.5.0",
            "model_name": "DescriptionStyleModel",
            "state": {
              "description_width": "initial"
            }
          },
          "d1f7f46ac3b64c818536834c60392fea": {
            "model_module": "@jupyter-widgets/base",
            "model_module_version": "1.2.0",
            "model_name": "LayoutModel",
            "state": {}
          },
          "d1fa63f01abb4c59a0158af8c3af9d8d": {
            "model_module": "@jupyter-widgets/controls",
            "model_module_version": "1.5.0",
            "model_name": "HBoxModel",
            "state": {
              "children": [
                "IPY_MODEL_949f237e510542789ff9e149560d5181",
                "IPY_MODEL_e0f304182b2543cc88d79e19c5bae2c6"
              ],
              "layout": "IPY_MODEL_621702c9f018481183b96930e706cc54"
            }
          },
          "d43e2535709949c7bbe3b426483ff4ae": {
            "model_module": "@jupyter-widgets/base",
            "model_module_version": "1.2.0",
            "model_name": "LayoutModel",
            "state": {
              "width": "70%"
            }
          },
          "d6fa1751257f48c5a6161393ac096efc": {
            "model_module": "@jupyter-widgets/base",
            "model_module_version": "1.2.0",
            "model_name": "LayoutModel",
            "state": {}
          },
          "d8ba0aa2aad54cd0aa69760a5354b452": {
            "model_module": "@jupyter-widgets/controls",
            "model_module_version": "1.5.0",
            "model_name": "IntRangeSliderModel",
            "state": {
              "_model_name": "IntRangeSliderModel",
              "_view_name": "IntRangeSliderView",
              "description": "Time Range",
              "layout": "IPY_MODEL_d43e2535709949c7bbe3b426483ff4ae",
              "max": 28,
              "min": -28,
              "style": "IPY_MODEL_a4d3df3b4447488f93e32ac4b3ada8c9",
              "value": [
                -1,
                0
              ]
            }
          },
          "db4165bc3d9e4a3182d06b7fca3178e4": {
            "model_module": "@jupyter-widgets/controls",
            "model_module_version": "1.5.0",
            "model_name": "HBoxModel",
            "state": {
              "children": [
                "IPY_MODEL_ed5543a1b31c4370ab95fdb59cf1453e",
                "IPY_MODEL_442fd1263e334d45ad7d171c3457d7cd"
              ],
              "layout": "IPY_MODEL_f3278667c52b4c6986fe59f6854be181"
            }
          },
          "db98f91c5e394abeb6b76ed5fb4284ad": {
            "model_module": "@jupyter-widgets/controls",
            "model_module_version": "1.5.0",
            "model_name": "TextModel",
            "state": {
              "description": "Query start time (UTC):",
              "layout": "IPY_MODEL_5d170286dedd41889a7345023f650a33",
              "style": "IPY_MODEL_3c990d1c0dac4fac9be32569b11c773a",
              "value": "2021-07-01 22:38:35.047441"
            }
          },
          "dcd9308cd0474c24827bcc6cec31e96d": {
            "model_module": "@jupyter-widgets/base",
            "model_module_version": "1.2.0",
            "model_name": "LayoutModel",
            "state": {}
          },
          "debbd947449c4860aef557d0d8c54d08": {
            "model_module": "@jupyter-widgets/base",
            "model_module_version": "1.2.0",
            "model_name": "LayoutModel",
            "state": {
              "width": "70%"
            }
          },
          "e0f304182b2543cc88d79e19c5bae2c6": {
            "model_module": "@jupyter-widgets/controls",
            "model_module_version": "1.5.0",
            "model_name": "TextModel",
            "state": {
              "description": "Time (24hr)",
              "layout": "IPY_MODEL_d6fa1751257f48c5a6161393ac096efc",
              "style": "IPY_MODEL_56dbb96a024b4062bb16295c93bb8386",
              "value": "22:38:35.047441"
            }
          },
          "e28b52485fcb486c91ec902f5b7c61db": {
            "model_module": "@jupyter-widgets/controls",
            "model_module_version": "1.5.0",
            "model_name": "HBoxModel",
            "state": {
              "children": [
                "IPY_MODEL_424d17d0ee4b4d929ea90e253f069ed7",
                "IPY_MODEL_722e044af71d41c6b6a82690cd8a577c"
              ],
              "layout": "IPY_MODEL_0040cad1767f4b95a00aefbc70d06bc1"
            }
          },
          "e41efc3621cc45e399078cf24332bf67": {
            "model_module": "@jupyter-widgets/base",
            "model_module_version": "1.2.0",
            "model_name": "LayoutModel",
            "state": {
              "height": "150px",
              "width": "300px"
            }
          },
          "ec9a50e5fb5646669c679ce5dcf6f26a": {
            "model_module": "@jupyter-widgets/base",
            "model_module_version": "1.2.0",
            "model_name": "LayoutModel",
            "state": {}
          },
          "ecd5058213f7439db39bd205512843e5": {
            "model_module": "@jupyter-widgets/controls",
            "model_module_version": "1.5.0",
            "model_name": "SliderStyleModel",
            "state": {
              "description_width": "initial"
            }
          },
          "ed5543a1b31c4370ab95fdb59cf1453e": {
            "model_module": "@jupyter-widgets/controls",
            "model_module_version": "1.5.0",
            "model_name": "DatePickerModel",
            "state": {
              "description": "Origin Date",
              "disabled": false,
              "layout": "IPY_MODEL_ec9a50e5fb5646669c679ce5dcf6f26a",
              "style": "IPY_MODEL_511af61975284928bb839f6a946ae2b6",
              "value": {
                "date": 2,
                "month": 6,
                "year": 2021
              }
            }
          },
          "f3278667c52b4c6986fe59f6854be181": {
            "model_module": "@jupyter-widgets/base",
            "model_module_version": "1.2.0",
            "model_name": "LayoutModel",
            "state": {}
          },
          "f5fe70128cdf4eadaf8455cdbbd719cb": {
            "model_module": "@jupyter-widgets/controls",
            "model_module_version": "1.5.0",
            "model_name": "TextModel",
            "state": {
              "description": "Query start time (UTC):",
              "layout": "IPY_MODEL_b7cffa19b0144d8eb66f2c87e47282f6",
              "style": "IPY_MODEL_3338fc72841a43b983a7a5d7ab114635",
              "value": "2021-07-01 22:38:45.575783"
            }
          },
          "f662b92adde64b5da4e13750c0fa88c0": {
            "model_module": "@jupyter-widgets/base",
            "model_module_version": "1.2.0",
            "model_name": "LayoutModel",
            "state": {
              "width": "70%"
            }
          },
          "f85fee408dda49ffadbae38d3f49eab9": {
            "model_module": "@jupyter-widgets/controls",
            "model_module_version": "1.5.0",
            "model_name": "DescriptionStyleModel",
            "state": {
              "description_width": "initial"
            }
          },
          "f90cc4c565664db79c4aa5220fe7f94c": {
            "model_module": "@jupyter-widgets/base",
            "model_module_version": "1.2.0",
            "model_name": "LayoutModel",
            "state": {
              "height": "150px",
              "width": "300px"
            }
          },
          "fa1b0cab6957493e935f4e9a8d428cb2": {
            "model_module": "@jupyter-widgets/base",
            "model_module_version": "1.2.0",
            "model_name": "LayoutModel",
            "state": {}
          },
          "ff485c7bb935466a8425b6a4ae8f8da9": {
            "model_module": "@jupyter-widgets/controls",
            "model_module_version": "1.5.0",
            "model_name": "DescriptionStyleModel",
            "state": {
              "description_width": ""
            }
          }
        },
        "version_major": 2,
        "version_minor": 0
      }
    }
  },
  "nbformat": 4,
  "nbformat_minor": 4
}