examples/Vulture/02-vulture_operators_NER.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "254cd3b1-e36b-42b5-9297-9b9d174a1011",
   "metadata": {},
   "source": [
    "# Vulture"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "20f74105-f696-41c1-ad8e-1414f1e874b4",
   "metadata": {},
   "source": [
    "## Introduction to Text Operations with Vulture"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "1c0a0ad2-f847-4cdb-a35e-7ad4f298505b",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import os\n",
    "import pickle\n",
    "import pathlib\n",
    "import pandas as pd\n",
    "\n",
    "from TELF.pre_processing import Vulture"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4a1dc990-7de8-40e6-baca-0ecef35c6477",
   "metadata": {},
   "source": [
    "## 0. Load Dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9b12074f-691b-4a4a-b070-ec94307f3fd4",
   "metadata": {},
   "source": [
    "### Input"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "521bce7b-4e83-4639-b784-a066c22478c6",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "DATA_DIR = os.path.join('..', '..', 'data')\n",
    "DATA_DIR = pathlib.Path(DATA_DIR).resolve()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "d292d91b-c0da-487b-bb34-bdcee4fc4075",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "DATA_FILE = 'documents.p'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "6f73de82-bd13-4c8b-acf3-5c0d11073fb8",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "9"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "documents = pickle.load(open(os.path.join(DATA_DIR, DATA_FILE), 'rb'))\n",
    "len(documents)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "387ef96c-492f-4eab-a72e-bcdb3417efa8",
   "metadata": {},
   "source": [
    "### Output"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "21aaaf2c-8c07-4126-a68e-aeebb4d01b59",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "RESULTS_DIR = 'results'\n",
    "RESULTS_DIR = pathlib.Path(RESULTS_DIR).resolve()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "6f278716-0ba2-4037-8146-59e1386cf0da",
   "metadata": {},
   "outputs": [],
   "source": [
    "RESULTS_FILE = 'operated_documents'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "eeec913a",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "try:\n",
    "    os.mkdir(RESULTS_DIR)\n",
    "except FileExistsError:\n",
    "    pass"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "845d3357-ac44-464e-b9f8-cfd683697a38",
   "metadata": {},
   "source": [
    "### Examine Data Format"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "d76a0f09-cdcf-4cf2-ac54-1362136e1736",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['ad68055e-677f-11ee-95d4-4ab2673ea3f0',\n",
       " 'ad680626-677f-11ee-95d4-4ab2673ea3f0',\n",
       " 'ad680658-677f-11ee-95d4-4ab2673ea3f0',\n",
       " 'ad680680-677f-11ee-95d4-4ab2673ea3f0',\n",
       " 'ad6806a8-677f-11ee-95d4-4ab2673ea3f0',\n",
       " 'ad6806d0-677f-11ee-95d4-4ab2673ea3f0',\n",
       " 'ad6806f8-677f-11ee-95d4-4ab2673ea3f0',\n",
       " 'ad680716-677f-11ee-95d4-4ab2673ea3f0',\n",
       " 'ad68073e-677f-11ee-95d4-4ab2673ea3f0']"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# key serve as document unique ids\n",
    "list(documents.keys())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "b2f2f25a-c3ad-40b1-bba1-652833367173",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Supervisory Control and Data Acquisition (SCADA) systems often serve as the nervous system for substations within power grids. These systems facilitate real-time monitoring, data acquisition, control of equipment, and ensure smooth and efficient operation of the substation and its connected devices. As the dependence on these SCADA systems grows, so does the risk of potential malicious intrusions that could lead to significant outages or even permanent damage to the grid. Previous work has shown that dimensionality reduction-based approaches, such as Principal Component Analysis (PCA), can be used for accurate identification of anomalies in SCADA systems. While not specifically applied to SCADA, non-negative matrix factorization (NMF) has shown strong results at detecting anomalies in wireless sensor networks. These unsupervised approaches model the normal or expected behavior and detect the unseen types of attacks or anomalies by identifying the events that deviate from the expected behavior. These approaches; however, do not model the complex and multi-dimensional interactions that are naturally present in SCADA systems. Differently, non-negative tensor decomposition is a powerful unsupervised machine learning (ML) method that can model the complex and multi-faceted activity details of SCADA events. In this work, we novelly apply the tensor decomposition method Canonical Polyadic Alternating Poisson Regression (CP-APR) with a probabilistic framework, which has previously shown state-of-the-art anomaly detection results on cyber network data, to identify anomalies in SCADA systems. We showcase that the use of statistical behavior analysis of SCADA communication with tensor decomposition improves the specificity and accuracy of identifying anomalies in electrical grid systems. In our experiments, we model real-world SCADA system data collected from the electrical grid operated by Los Alamos National Laboratory (LANL) which provides transmission and distribution service through a partnership with Los Alamos County, and detect synthetically generated anomalies.'"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# values are the text that needs to be cleaned\n",
    "documents[next(iter(documents))]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a708c17b-f4ad-480c-b31f-f221e034a981",
   "metadata": {},
   "source": [
    "## NER Operation"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "da2deb7d-a3a4-4ac1-8232-00b3a116d3fd",
   "metadata": {},
   "source": [
    "The Vulture library is composed of multiple operation modules that can work one after another to perform a custom operation on the text. These modules are flexible and their order can be re-arranged depending on the user's preferences. By default Vulture implements NER pipeline so that new users can quickly get started. In this section we will examine the Vulture default pipeline and apply the name entity recognition to the sample text."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3b316563-4e41-40cc-a31b-f56451b2d102",
   "metadata": {},
   "source": [
    "The pipeline is a just list of Vulture modules that are to be updated sequentially. The default pipeline contains a single module - the ```NEDetector```."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "1c59c011-9d3c-41a4-8785-53cf45cf96ab",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[NEDetector(module_type='OPERATOR', backend=None)]"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Vulture.DEFAULT_OPERATOR_PIPELINE"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "302365f7-5aa8-4e8f-befa-a2e94eaf0f70",
   "metadata": {},
   "source": [
    "### Setup Vulture"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "41a53e96-23d6-4e10-a538-430e6d6bc43b",
   "metadata": {},
   "source": [
    "Create a single-node multi-process Vulture object"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "ea8b8813-e8c7-42bc-9aa0-538187487841",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "vulture = Vulture(n_jobs  = 1, \n",
    "                  verbose = 10,  # Disable == 0, Verbose >= 1\n",
    "                 )"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7f936d14-4fd1-4f43-bac7-0fc95348c72e",
   "metadata": {},
   "source": [
    "### Apply NER"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If we do not pass the ```save_path```, it will return a list of results where each entry in the list is for the given operation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Vulture]: Cleaning 9 documents\n",
      "  0%|          | 0/1 [00:00<?, ?it/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Vulture]: Running NEDetector module\n",
      "/Users/maksim/Desktop/Code/T-ELF_public/.eggs/tqdm-4.66.1-py3.11.egg/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html\n",
      "  from .autonotebook import tqdm as notebook_tqdm\n",
      "/Users/maksim/miniconda3/envs/TELF_public/lib/python3.11/site-packages/transformers/utils/generic.py:441: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead.\n",
      "  _torch_pytree._register_pytree_node(\n",
      "/Users/maksim/miniconda3/envs/TELF_public/lib/python3.11/site-packages/transformers/utils/generic.py:309: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead.\n",
      "  _torch_pytree._register_pytree_node(\n",
      "100%|██████████| 9/9 [00:02<00:00,  3.12it/s]\n",
      "100%|██████████| 1/1 [00:02<00:00,  2.89s/it]\n"
     ]
    }
   ],
   "source": [
    "operation_results =  vulture.operate(documents)                   "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Each entry is a tuple where index 0 is the name of the operation and index 1 is the results of the operation in dictionary format."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'NEDetector'"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "operation_results[0][0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Operation for each document is given in dictionary format where key is the document ID and its value is the operation results, in this case NER"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'ad68055e-677f-11ee-95d4-4ab2673ea3f0': {'ORG': {'LANL',\n",
       "   'Los Alamos National Laboratory',\n",
       "   'SCADA'},\n",
       "  'GPE': {'Los Alamos County'}},\n",
       " 'ad680626-677f-11ee-95d4-4ab2673ea3f0': {'PRODUCT': {'SeNMFk'},\n",
       "  'CARDINAL': {'two'}},\n",
       " 'ad680658-677f-11ee-95d4-4ab2673ea3f0': {'ORG': {'NVIDIA'},\n",
       "  'PRODUCT': {'NMFk'},\n",
       "  'CARDINAL': {'4096', 'approximately 25,000'},\n",
       "  'QUANTITY': {'11 Exabyte', '340 Terabyte'}},\n",
       " 'ad680680-677f-11ee-95d4-4ab2673ea3f0': {'PRODUCT': {'HNMFk Classifier',\n",
       "   'the HNMFk Classifier'},\n",
       "  'CARDINAL': {'0.80', 'nearly 2,900', 'nearly 388,000'}},\n",
       " 'ad6806a8-677f-11ee-95d4-4ab2673ea3f0': {},\n",
       " 'ad6806d0-677f-11ee-95d4-4ab2673ea3f0': {'PRODUCT': {'Malware-DNA',\n",
       "   'R&D100',\n",
       "   'SmartTensors AI Platform'},\n",
       "  'DATE': {'2021'},\n",
       "  'ORDINAL': {'first'},\n",
       "  'CARDINAL': {'one'}},\n",
       " 'ad6806f8-677f-11ee-95d4-4ab2673ea3f0': {'CARDINAL': {'One',\n",
       "   'two',\n",
       "   '~2 million+'},\n",
       "  'ORG': {'SeNMFk', 'arXiv'}},\n",
       " 'ad680716-677f-11ee-95d4-4ab2673ea3f0': {'CARDINAL': {'One', 'one', 'two'},\n",
       "  'ORDINAL': {'first'},\n",
       "  'PRODUCT': {'MovieLens'}},\n",
       " 'ad68073e-677f-11ee-95d4-4ab2673ea3f0': {'CARDINAL': {'1'}}}"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "operation_results[0][1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "d409a060-2039-4e05-a2a0-a64851ea4b2f",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Vulture]: Cleaning 9 documents\n",
      "  0%|          | 0/1 [00:00<?, ?it/s][Vulture]: Running NEDetector module\n",
      "100%|██████████| 9/9 [00:01<00:00,  6.21it/s]\n",
      "100%|██████████| 1/1 [00:01<00:00,  1.45s/it]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 8.29 s, sys: 7.11 s, total: 15.4 s\n",
      "Wall time: 1.45 s\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    }
   ],
   "source": [
    "%time vulture.operate(documents, save_path=RESULTS_DIR, file_name=RESULTS_FILE)                   "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['operated_documents_NEDetector.p']"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "saved_file = ! ls $RESULTS_DIR\n",
    "saved_file"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a96259b6-c967-4102-913a-9e866e4ee111",
   "metadata": {},
   "source": [
    "### Look at Cleaned Documents"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "9aad256c-2645-4f1e-b98a-f44132e61c15",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "operated_documents = pickle.load(open(os.path.join(RESULTS_DIR, saved_file[0]), 'rb'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "69859fe2-48e5-4a8e-8b8d-ccc64abd51b7",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "def to_df(documents, operated_documents):\n",
    "    data = {\n",
    "        'id': [],\n",
    "        'text': [],\n",
    "        'operation_result': []\n",
    "    }\n",
    "\n",
    "    for i, text in documents.items():\n",
    "        data['id'].append(i)\n",
    "        data['text'].append(text)\n",
    "        data['operation_result'].append(operated_documents.get(i))\n",
    "\n",
    "    return pd.DataFrame.from_dict(data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "087aa962-1a60-40c3-8de2-be90df8a975c",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>text</th>\n",
       "      <th>operation_result</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>ad68055e-677f-11ee-95d4-4ab2673ea3f0</td>\n",
       "      <td>Supervisory Control and Data Acquisition (SCAD...</td>\n",
       "      <td>{'ORG': {'LANL', 'Los Alamos National Laborato...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>ad680626-677f-11ee-95d4-4ab2673ea3f0</td>\n",
       "      <td>Highly specific datasets of scientific literat...</td>\n",
       "      <td>{'PRODUCT': {'SeNMFk'}, 'CARDINAL': {'two'}}</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>ad680658-677f-11ee-95d4-4ab2673ea3f0</td>\n",
       "      <td>We propose an efficient distributed out-of-mem...</td>\n",
       "      <td>{'ORG': {'NVIDIA'}, 'PRODUCT': {'NMFk'}, 'CARD...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>ad680680-677f-11ee-95d4-4ab2673ea3f0</td>\n",
       "      <td>Identification of the family to which a malwar...</td>\n",
       "      <td>{'PRODUCT': {'the HNMFk Classifier', 'HNMFk Cl...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>ad6806a8-677f-11ee-95d4-4ab2673ea3f0</td>\n",
       "      <td>Malware is one of the most dangerous and costl...</td>\n",
       "      <td>{}</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>ad6806d0-677f-11ee-95d4-4ab2673ea3f0</td>\n",
       "      <td>Malware is one of the most dangerous and costl...</td>\n",
       "      <td>{'PRODUCT': {'SmartTensors AI Platform', 'Malw...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>ad6806f8-677f-11ee-95d4-4ab2673ea3f0</td>\n",
       "      <td>Topic modeling is one of the key analytic tech...</td>\n",
       "      <td>{'CARDINAL': {'~2 million+', 'One', 'two'}, 'O...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>ad680716-677f-11ee-95d4-4ab2673ea3f0</td>\n",
       "      <td>Non-negative matrix factorization (NMF) with m...</td>\n",
       "      <td>{'CARDINAL': {'one', 'One', 'two'}, 'ORDINAL':...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>ad68073e-677f-11ee-95d4-4ab2673ea3f0</td>\n",
       "      <td>We propose an efficient, distributed, out-of-m...</td>\n",
       "      <td>{'CARDINAL': {'1'}}</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                     id  \\\n",
       "0  ad68055e-677f-11ee-95d4-4ab2673ea3f0   \n",
       "1  ad680626-677f-11ee-95d4-4ab2673ea3f0   \n",
       "2  ad680658-677f-11ee-95d4-4ab2673ea3f0   \n",
       "3  ad680680-677f-11ee-95d4-4ab2673ea3f0   \n",
       "4  ad6806a8-677f-11ee-95d4-4ab2673ea3f0   \n",
       "5  ad6806d0-677f-11ee-95d4-4ab2673ea3f0   \n",
       "6  ad6806f8-677f-11ee-95d4-4ab2673ea3f0   \n",
       "7  ad680716-677f-11ee-95d4-4ab2673ea3f0   \n",
       "8  ad68073e-677f-11ee-95d4-4ab2673ea3f0   \n",
       "\n",
       "                                                text  \\\n",
       "0  Supervisory Control and Data Acquisition (SCAD...   \n",
       "1  Highly specific datasets of scientific literat...   \n",
       "2  We propose an efficient distributed out-of-mem...   \n",
       "3  Identification of the family to which a malwar...   \n",
       "4  Malware is one of the most dangerous and costl...   \n",
       "5  Malware is one of the most dangerous and costl...   \n",
       "6  Topic modeling is one of the key analytic tech...   \n",
       "7  Non-negative matrix factorization (NMF) with m...   \n",
       "8  We propose an efficient, distributed, out-of-m...   \n",
       "\n",
       "                                    operation_result  \n",
       "0  {'ORG': {'LANL', 'Los Alamos National Laborato...  \n",
       "1       {'PRODUCT': {'SeNMFk'}, 'CARDINAL': {'two'}}  \n",
       "2  {'ORG': {'NVIDIA'}, 'PRODUCT': {'NMFk'}, 'CARD...  \n",
       "3  {'PRODUCT': {'the HNMFk Classifier', 'HNMFk Cl...  \n",
       "4                                                 {}  \n",
       "5  {'PRODUCT': {'SmartTensors AI Platform', 'Malw...  \n",
       "6  {'CARDINAL': {'~2 million+', 'One', 'two'}, 'O...  \n",
       "7  {'CARDINAL': {'one', 'One', 'two'}, 'ORDINAL':...  \n",
       "8                                {'CARDINAL': {'1'}}  "
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = to_df(documents, operated_documents)\n",
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "TELF_public",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}