Demos/21b. Bert 微调抽取式问答任务 - 作业样例代码.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "xvSGDbExff_I"
   },
   "source": [
    "# 作业 - Bert (Extractive Question Answering)\n",
    "\n",
    "> 本作业迁移自李宏毅老师 2023 年机器学习春的作业 7，原链接：[ML2023_HW7_Question_Answering](https://colab.research.google.com/drive/1m0fQjJfkK9vAovxPj9Nd3-hQuxezB2w1)\n",
    ">\n",
    "> 课程主页：[Machine Learning 2023 Spring](https://speech.ee.ntu.edu.tw/~hylee/ml/2023-spring.php)\n",
    ">\n",
    "> 指导文章：[22a. 微调 LLM：实现抽取式问答](../Guide/22a.%20微调%20LLM：实现抽取式问答.md) | 作业文章：[22b. 作业 - Bert 微调抽取式问答](../Guide/22b.%20作业%20-%20Bert%20微调抽取式问答.md)\n",
    "\n",
    "请根据**提示**完成相应的代码修改，从而达到学习的目的。\n",
    "\n",
    "[作业 PPT](https://github.com/Hoper-J/AI-Guide-and-Demos-zh_CN/blob/master/Demos/PDF/HW_BERT_Question_Answering.pdf) | [Kaggle 加入链接](https://www.kaggle.com/t/e001cad568dc4d77b6a5e762172f44d6)\n",
    "\n",
    "在线链接：[Kaggle](https://www.kaggle.com/code/aidemos/21-bert) | [Colab](https://colab.research.google.com/drive/1zHR2Cztmo49j3yrdT3GgkzNcAcHd7M0M?usp=sharing)\n",
    "\n",
    "\n",
    "## 作业提示\n",
    "\n",
    "> 如果理解了每个提示到底需要做什么，那么恭喜你已经完成了学习。\n",
    ">\n",
    "> 搜索代码文件中的 TODO 可以快速索引到需要修改的地方。\n",
    "\n",
    "- **Simple baseline**\n",
    "  - 直接运行当前代码文件\n",
    "- **Medium baseline**\n",
    "  - 使用学习率调度器：线性衰减，warmup 等，选择一个即可\n",
    "- **Strong baseline**\n",
    "  - 修改模型的预处理过程 (TODO: Preprocessing)\n",
    "  - 尝试替换为其他预训练模型，在 [Hugging Face](https://huggingface.co/models?language=zh&sort=trending) 中进行选择\n",
    "- **Boss baseline**\n",
    "  - 修改后处理部分 (TODO: Postprocessing)\n",
    "  - 尝试梯度累积\n",
    "\n",
    "## Kaggle 加入“比赛”\n",
    "\n",
    "访问 [Kaggle 加入链接](https://www.kaggle.com/t/e001cad568dc4d77b6a5e762172f44d6)，点击右上角的 `Late Submission`，然后点击 `I Understand and Accept` 同意：\n",
    "\n",
    "![image-20241110191905666](../Guide/assets/image-20241110191905666.png)\n",
    "\n",
    "样例代码运行后会生成文件：`result.csv`，如果想要提交查看分数，再次点击 `Late Submission`，上传后点击提交：\n",
    "\n",
    "![image-20241110192134533](../Guide/assets/image-20241110192134533.png)\n",
    "\n",
    "你可以在 `Submissions` 中看到自己的提交成绩：\n",
    "\n",
    "![image-20241110192753327](../Guide/assets/image-20241110192753327.png)\n",
    "\n",
    "> [!note]\n",
    ">\n",
    "> 在评估效果时最好仅参考 Public Score，Private Score 是测试集上的分数，因为比赛已经过了Deadline，所以显示了出来，比赛进行的过程中这个分数是隐藏的。\n",
    "\n",
    "点击 `Leaderboard` 你将能够看到曾经参与者的成绩：\n",
    "\n",
    "![image-20241110192649655](../Guide/assets/image-20241110192649655.png)\n",
    "\n",
    "\n",
    "## 任务目标：中文抽取式问答\n",
    "\n",
    "学习如何使用 transformers 微调预训练模型，以完成下游抽取式问答任务。\n",
    "\n",
    "- **任务描述**: 通过 fine-tune BERT 模型，使其能够从给定的段落中抽取出问题的具体答案。\n",
    "  - **目标**: 模型将预测答案的开始（Start）和结束（End）位置，并从文本中提取该片段作为答案。\n",
    "    \n",
    "    ![image-20240920002112364](../Guide/assets/e6a56c94334632bae17b286f42264de4.png)\n",
    "    \n",
    "  - **输入**: 段落 + 问题\n",
    "  - **输出**: 答案\n",
    " \n",
    "- **TODO**\n",
    "  - 微调预训练的中文 BERT 模型\n",
    "  - 调整超参数（如 `doc_stride`）\n",
    "  - 应用线性学习率衰减策略\n",
    "  - 尝试其他预训练模型\n",
    "  - 改进数据的预处理（preprocessing）\n",
    "  - 改进模型的后处理（postprocessing）\n",
    "\n",
    "- **训练技巧**\n",
    "  - 自动混合精度（Automatic Mixed Precision）\n",
    "  - 梯度累积（Gradient Accumulation）\n",
    "  - 集成方法（Ensemble）\n",
    "\n",
    "- **预计训练时间**（在 Tesla T4 上启用自动混合精度时的运行时间）：\n",
    "  - Simple baseline：8 分钟\n",
    "  - Medium baseline：8 分钟\n",
    "  - Strong baseline：25 分钟\n",
    "  - Boss baseline：2 小时\n",
    "\n",
    "## 性能指标（EM）\n",
    "\n",
    "- **准确率 (Exact Match)**: 该指标用于衡量模型的预测答案与真实答案**完全一致**的比例。\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "NYAHsHNbzdKm"
   },
   "source": [
    "## 前置准备"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "TJ1fSAJE2oaC"
   },
   "source": [
    "### 下载数据集\n",
    "\n",
    "#### Kaggle\n",
    "\n",
    "> To use the Kaggle API, sign up for a Kaggle account at [https://www.kaggle.com](https://www.kaggle.com/). Then go to the 'Account' tab of your user profile (`https://www.kaggle.com/<username>/account`) and select 'Create API Token'. This will trigger the download of `kaggle.json`, a file containing your API credentials. Place this file in the location `~/.kaggle/kaggle.json` (on Windows in the location `C:\\Users\\<Windows-username>\\.kaggle\\kaggle.json` - you can check the exact location, sans drive, with `echo %HOMEPATH%`). You can define a shell environment variable `KAGGLE_CONFIG_DIR` to change this location to `$KAGGLE_CONFIG_DIR/kaggle.json` (on Windows it will be `%KAGGLE_CONFIG_DIR%\\kaggle.json`).\n",
    ">\n",
    "> -\\- [Official Kaggle API](https://github.com/Kaggle/kaggle-api)\n",
    "\n",
    "替换\\<username\\>为你自己的用户名，`https://www.kaggle.com/<username>/account`，然后点击 `Create New API Token`，将下载下来的文件放去应该放的位置：\n",
    "\n",
    "- Mac 和 Linux 放在 `~/.kaggle`\n",
    "- Windows 放在 `C:\\Users\\<Windows-username>\\.kaggle`\n",
    "\n",
    "```bash\n",
    "pip install kaggle\n",
    "# 你需要先在 Kaggle -> Account -> Create New API Token 中下载 kaggle.json\n",
    "# mv kaggle.json ~/.kaggle/kaggle.json\n",
    "kaggle competitions download -c ml2023spring-hw7\n",
    "unzip ml2023spring-hw7.zip   \n",
    "```\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "YPrc4Eie9Yo5",
    "outputId": "1640c875-58d1-4288-e382-e9fa9db39b9f"
   },
   "outputs": [],
   "source": [
    "# 或者通过以下命令进行下载\n",
    "!wget https://github.com/Hoper-J/HUNG-YI_LEE_Machine-Learning_Homework/raw/refs/heads/master/HW07/ml2023spring-hw7.zip\n",
    "!unzip ml2023spring-hw7.zip"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "TevOvhC03m0h"
   },
   "source": [
    "### 安装库 "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "tbxWFX_jpDom",
    "outputId": "138488c2-96fa-4c41-d450-9d0a1b74f36c"
   },
   "outputs": [],
   "source": [
    "!pip install transformers\n",
    "!pip install accelerate"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "XapBp31gytyD"
   },
   "source": [
    "## 开始微调\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "8dKM4yCh4LI_"
   },
   "source": [
    "### 导入包\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "WOTHHtWJoahe"
   },
   "outputs": [],
   "source": [
    "import json\n",
    "import numpy as np\n",
    "import random\n",
    "import torch\n",
    "from torch.utils.data import DataLoader, Dataset\n",
    "from transformers import AdamW\n",
    "\n",
    "from tqdm.auto import tqdm\n",
    "\n",
    "device = \"cuda\" if torch.cuda.is_available() else \"cpu\"\n",
    "\n",
    "# 为了可重复性，固定随机数种子\n",
    "def same_seeds(seed):\n",
    "    torch.manual_seed(seed)\n",
    "    if torch.cuda.is_available():\n",
    "        torch.cuda.manual_seed(seed)\n",
    "        torch.cuda.manual_seed_all(seed)\n",
    "    np.random.seed(seed)\n",
    "    random.seed(seed)\n",
    "    torch.backends.cudnn.benchmark = False\n",
    "    torch.backends.cudnn.deterministic = True\n",
    "same_seeds(42)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "2YgXHuVLp_6j"
   },
   "source": [
    "### 加载模型和分词器\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 220,
     "referenced_widgets": [
      "641665cb9ec6433a81719d57eeaf8298",
      "ba9c1cd3cfe04add90aa2edf2aa87378",
      "1e20837bf23548249678da0a07745d58",
      "c1d24cb7607c44b989774113a2baf675",
      "c9144705a33848a6b142f9887da6ac49",
      "ebfa4ba9f18b4945947d326581d3b8e9",
      "28fc746a3c0548f4a4ff669b5dbb7320",
      "d26e50974c52452389c5abb001056546",
      "9eefbb580f0146658d18658f210d3a67",
      "c813a6f6ce09424582c06a3ac8a7d712",
      "ff6cd33dca3c44ba9832b2d705fd2de1",
      "8b264499afca4694849b08e2091f6826",
      "3eaf82623d964c2abb92b4509ac5ca9d",
      "aa3ff7831fbf47efa404805886ca4696",
      "6be63de77ed14eee9c37e93d3a75f743",
      "5195ed241f09440eaf84a7e0b3dc8155",
      "33fa92f225be477d8cd176a0a4e8894e",
      "b80166eeee984661a611a02d1eae4458",
      "20cd68a7e03f4353aa99fcefc1f16d36",
      "fb908d9c3e374983aa47478a95411ba7",
      "304609dbefa74735baa08ce7aadff92d",
      "250451c12cee4aacb91a32d7da4ab861",
      "2c37badbe0504073b31abd67a8c0b808",
      "bfd7d35860c14dd3b8e5eea07032311d",
      "db2c74f92f6d4a2c8bfdcb91d5bd3d64",
      "b0af063597ea4de5a8c8bf4f370f782a",
      "5b1ac1a0f04f4c29b3c1127ec3d95e81",
      "b91c8359a6444497b3665e66afbe237d",
      "1181117783fd4bca9d7f5f296507aff3",
      "941a345ba82a44d5ad597a15c0af0d5c",
      "23afbee21a644745b24b8ffefed6b286",
      "a075ee85c2d34ab3b2d011c63dc02286",
      "052cccb5d69d476f9caac95848e0353a"
     ]
    },
    "id": "xyBCYGjAp3ym",
    "outputId": "81f25c6d-cd7f-4be1-e085-f8a54bd1c27d",
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "from transformers import (\n",
    "  AutoTokenizer,\n",
    "  AutoModelForQuestionAnswering,\n",
    ")\n",
    "\n",
    "model = AutoModelForQuestionAnswering.from_pretrained(\"bert-base-chinese\").to(device)\n",
    "tokenizer = AutoTokenizer.from_pretrained(\"bert-base-chinese\")\n",
    "\n",
    "# 你可以忽略警告消息（它弹出是因为新的 QA 预测头是随机初始化的）"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "3Td-GTmk5OW4"
   },
   "source": [
    "## 数据部分\n",
    "\n",
    "> 使用两个繁体中文阅读理解数据集：[DRCD](https://github.com/DRCKnowledgeTeam/DRCD) 和 [ODSQA](https://github.com/Chia-Hsuan-Lee/ODSQA)。\n",
    "\n",
    "- **训练集（DRCD + DRCD-backtrans）**：包含 15,329 个段落和 26,918 个问题。一个段落可能对应多个问题。\n",
    "- **开发集（DRCD + DRCD-backtrans）**：包含 1,255 个段落和 2,863 个问题。用于验证。\n",
    "- **测试集（DRCD + ODSQA）**：包含 1,606 个段落和 3,504 个问题。测试集的段落没有提供答案，需要模型进行预测。\n",
    "\n",
    "所有数据集的格式相同：\n",
    "\n",
    "- `id`：问题编号\n",
    "- `paragraph_id`：段落编号\n",
    "- `question_text`：问题文本\n",
    "- `answer_text`：答案文本\n",
    "- `answer_start`：答案在段落中的起始字符位置\n",
    "- `answer_end`：答案在段落中的结束字符位置\n",
    "\n",
    "![数据集格式](../Guide/assets/4215768313590de87aab01adcad78c90.png)\n",
    "\n",
    "### 读取数据"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "NvX7hlepogvu"
   },
   "outputs": [],
   "source": [
    "def read_data(file):\n",
    "    with open(file, 'r', encoding=\"utf-8\") as reader:\n",
    "        data = json.load(reader)\n",
    "    return data[\"questions\"], data[\"paragraphs\"]\n",
    "\n",
    "train_questions, train_paragraphs = read_data(\"hw7_train.json\")\n",
    "dev_questions, dev_paragraphs = read_data(\"hw7_dev.json\")\n",
    "test_questions, test_paragraphs = read_data(\"hw7_test.json\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Fm0rpTHq0e4N"
   },
   "source": [
    "### 分词处理"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "rTZ6B70Hoxie",
    "outputId": "db13805e-02a9-4cbe-e6d2-66c0e3860c2b"
   },
   "outputs": [],
   "source": [
    "# 分别对问题和段落进行分词\n",
    "# 「add_special_tokens」设置为 False，因为在数据集的 __getitem__ 中，当合并分词后的问题和段落时会添加特殊标记\n",
    "\n",
    "train_questions_tokenized = tokenizer([train_question[\"question_text\"] for train_question in train_questions], add_special_tokens=False)\n",
    "dev_questions_tokenized = tokenizer([dev_question[\"question_text\"] for dev_question in dev_questions], add_special_tokens=False)\n",
    "test_questions_tokenized = tokenizer([test_question[\"question_text\"] for test_question in test_questions], add_special_tokens=False)\n",
    "\n",
    "train_paragraphs_tokenized = tokenizer(train_paragraphs, add_special_tokens=False)\n",
    "dev_paragraphs_tokenized = tokenizer(dev_paragraphs, add_special_tokens=False)\n",
    "test_paragraphs_tokenized = tokenizer(test_paragraphs, add_special_tokens=False)\n",
    "\n",
    "# 你可以忽略警告消息，因为在传递给模型之前，将在数据集的 __getitem__ 中进一步处理分词后的序列"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Ws8c8_4d5UCI"
   },
   "source": [
    "### 数据集（Medium / Strong 相关）\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "Xjooag-Swnuh"
   },
   "outputs": [],
   "source": [
    "class QA_Dataset(Dataset):\n",
    "    \"\"\"\n",
    "    自定义的问答数据集类，用于处理问答任务的数据。\n",
    "\n",
    "    参数：\n",
    "    - split (str): 数据集的类型，'train'、'dev' 或 'test'。\n",
    "    - questions (list): 问题列表，每个元素是一个字典，包含问题的详细信息。\n",
    "    - tokenized_questions (BatchEncoding): 分词后的问题，由 tokenizer 生成。\n",
    "    - tokenized_paragraphs (BatchEncoding): 分词后的段落列表，由 tokenizer 生成。\n",
    "    \n",
    "    属性（即 __init_() 中的 self.xxx）：\n",
    "    - max_question_len (int): 问题的最大长度（以分词后的 token 数计）。\n",
    "    - max_paragraph_len (int): 段落的最大长度（以分词后的 token 数计）。\n",
    "    - doc_stride (int): 段落窗口滑动步长。\n",
    "    - max_seq_len (int): 输入序列的最大长度。\n",
    "    \"\"\"\n",
    "    def __init__(self, split, questions, tokenized_questions, tokenized_paragraphs):\n",
    "        self.split = split\n",
    "        self.questions = questions\n",
    "        self.tokenized_questions = tokenized_questions\n",
    "        self.tokenized_paragraphs = tokenized_paragraphs\n",
    "        self.max_question_len = 60\n",
    "        self.max_paragraph_len = 150\n",
    "\n",
    "        ##### TODO: 更改 doc_stride 的值 #####\n",
    "        self.doc_stride = 150\n",
    "\n",
    "        # 输入序列长度 = [CLS] + question + [SEP] + paragraph + [SEP]\n",
    "        self.max_seq_len = 1 + self.max_question_len + 1 + self.max_paragraph_len + 1\n",
    "\n",
    "    def __len__(self):\n",
    "        \"\"\"\n",
    "        返回数据集中样本的数量。\n",
    "\n",
    "        返回：\n",
    "        - (int): 数据集的长度。\n",
    "        \"\"\"\n",
    "        return len(self.questions)\n",
    "\n",
    "    def __getitem__(self, idx):\n",
    "        \"\"\"\n",
    "        获取数据集中指定索引的样本。\n",
    "\n",
    "        参数：\n",
    "        - idx (int): 样本的索引。\n",
    "\n",
    "        返回：\n",
    "        - 对于训练集，返回一个输入张量和对应的答案位置：\n",
    "          (input_ids, token_type_ids, attention_mask, answer_start_token, answer_end_token)\n",
    "        - 对于验证和测试集，返回包含多个窗口的输入张量列表：\n",
    "          (input_ids_list, token_type_ids_list, attention_mask_list)\n",
    "        \"\"\"\n",
    "        question = self.questions[idx]\n",
    "        tokenized_question = self.tokenized_questions[idx]\n",
    "        tokenized_paragraph = self.tokenized_paragraphs[question[\"paragraph_id\"]]\n",
    "\n",
    "        ##### TODO: 预处理 #####\n",
    "        # 提示：思考怎么防止模型学习到不应该学习的东西\n",
    "        if self.split == \"train\":\n",
    "            # 将答案在段落文本中的起始/结束位置转换为在分词后段落中的起始/结束位置\n",
    "            answer_start_token = tokenized_paragraph.char_to_token(question[\"answer_start\"])\n",
    "            answer_end_token = tokenized_paragraph.char_to_token(question[\"answer_end\"])\n",
    "\n",
    "            # 通过切片包含答案的段落部分来获得一个窗口\n",
    "            mid = (answer_start_token + answer_end_token) // 2\n",
    "            paragraph_start = max(0, min(mid - self.max_paragraph_len // 2, len(tokenized_paragraph) - self.max_paragraph_len))\n",
    "            paragraph_end = paragraph_start + self.max_paragraph_len\n",
    "\n",
    "            # 切片问题/段落并添加特殊标记（101：CLS，102：SEP）\n",
    "            input_ids_question = [101] + tokenized_question.ids[:self.max_question_len] + [102]\n",
    "            input_ids_paragraph = tokenized_paragraph.ids[paragraph_start : paragraph_end] + [102]\n",
    "\n",
    "            # 将答案在分词后段落中的起始/结束位置转换为窗口中的起始/结束位置\n",
    "            answer_start_token += len(input_ids_question) - paragraph_start\n",
    "            answer_end_token += len(input_ids_question) - paragraph_start\n",
    "\n",
    "            # 填充序列并获取模型的输入\n",
    "            input_ids, token_type_ids, attention_mask = self.padding(input_ids_question, input_ids_paragraph)\n",
    "            return torch.tensor(input_ids), torch.tensor(token_type_ids), torch.tensor(attention_mask), answer_start_token, answer_end_token\n",
    "\n",
    "        # 验证/测试\n",
    "        else:\n",
    "            input_ids_list, token_type_ids_list, attention_mask_list = [], [], []\n",
    "\n",
    "            # 段落被分割成多个窗口，每个窗口的起始位置由步长 \"doc_stride\" 分隔\n",
    "            for i in range(0, len(tokenized_paragraph), self.doc_stride):\n",
    "\n",
    "                # 切片问题/段落并添加特殊标记（101：CLS，102：SEP）\n",
    "                input_ids_question = [101] + tokenized_question.ids[:self.max_question_len] + [102]\n",
    "                input_ids_paragraph = tokenized_paragraph.ids[i : i + self.max_paragraph_len] + [102]\n",
    "\n",
    "                # 填充序列并获取模型的输入\n",
    "                input_ids, token_type_ids, attention_mask = self.padding(input_ids_question, input_ids_paragraph)\n",
    "\n",
    "                input_ids_list.append(input_ids)\n",
    "                token_type_ids_list.append(token_type_ids)\n",
    "                attention_mask_list.append(attention_mask)\n",
    "\n",
    "            return torch.tensor(input_ids_list), torch.tensor(token_type_ids_list), torch.tensor(attention_mask_list)\n",
    "\n",
    "    def padding(self, input_ids_question, input_ids_paragraph):\n",
    "        \"\"\"\n",
    "        对输入的序列进行填充，生成统一长度的模型输入。\n",
    "\n",
    "        参数：\n",
    "        - input_ids_question (list): 问题部分的输入 ID 列表。\n",
    "        - input_ids_paragraph (list): 段落部分的输入 ID 列表。\n",
    "\n",
    "        返回：\n",
    "        - input_ids (list): 填充后的输入 ID 列表。\n",
    "        - token_type_ids (list): 区分问题和段落的标记列表。\n",
    "        - attention_mask (list): 注意力掩码列表，指示哪些位置是有效的输入。\n",
    "        \"\"\"\n",
    "        # 计算需要填充的长度\n",
    "        padding_len = self.max_seq_len - len(input_ids_question) - len(input_ids_paragraph)\n",
    "        # 填充输入序列\n",
    "        input_ids = input_ids_question + input_ids_paragraph + [0] * padding_len\n",
    "        # 构造区分问题和段落的 token_type_ids\n",
    "        token_type_ids = [0] * len(input_ids_question) + [1] * len(input_ids_paragraph) + [0] * padding_len\n",
    "        # 构造注意力掩码，有效位置为 1，填充位置为 0\n",
    "        attention_mask = [1] * (len(input_ids_question) + len(input_ids_paragraph)) + [0] * padding_len\n",
    "\n",
    "        return input_ids, token_type_ids, attention_mask\n",
    "\n",
    "train_set = QA_Dataset(\"train\", train_questions, train_questions_tokenized, train_paragraphs_tokenized)\n",
    "dev_set = QA_Dataset(\"dev\", dev_questions, dev_questions_tokenized, dev_paragraphs_tokenized)\n",
    "test_set = QA_Dataset(\"test\", test_questions, test_questions_tokenized, test_paragraphs_tokenized)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> 这里解释一下QA_Dataset，如果觉得太长，可以只查看重点部分。\n",
    "\n",
    "1. **初始化（`__init__`）**：\n",
    "   - `split` 决定数据集的类型（训练、验证或测试）。\n",
    "   - `questions`, `tokenized_questions`, 和 `tokenized_paragraphs` 是原问题和 tokenized 后的问题和段落。\n",
    "   - `max_question_len` 和 `max_paragraph_len` 分别设定了问题和段落的最大长度。\n",
    "   - `self.doc_stride`：段落的窗口滑动步长（决定每个窗口之间的重叠部分）。\n",
    "     - `Sample code` 中将其设置为 150，和 `max_paragraph_len` 一样，意味着窗口之间完全不重叠。\n",
    "   - `self.max_seq_len`：定义了整个输入序列的最大长度（包含问题和段落）。\n",
    "\n",
    "2. **`__getitem__`**：\n",
    "   - 针对给定的索引 `idx`，获取对应问题和段落数据，返回模型需要的输入。\n",
    "   - **训练集**：定位答案的起始和结束位置，将包含答案的段落部分截取为一个窗口（中心在答案位置附近）。然后将问题和段落合并为一个输入序列，并进行填充。\n",
    "   - **验证/测试集**：将段落分成多个窗口，每个窗口之间的步长由 `self.doc_stride` 决定，然后将每个窗口作为模型的输入。验证和测试时不需要答案位置，因此只需生成多个窗口作为输入。\n",
    "\n",
    "3. **填充（`padding`）**：\n",
    "   - 输入序列可能比最大序列长度短，填充部分用 0 表示。对于问题部分和段落部分，`token_type_ids` 被用来区分它们（0 表示问题，1 表示段落）。`attention_mask` 用于标记有效的输入部分，防止模型对填充部分进行注意力计算。\n",
    "\n",
    "#### 重点\n",
    "\n",
    "- `self.doc_stride` 通过控制窗口之间的滑动步长，确保即使答案位于窗口边缘，模型也能通过多个窗口重叠的方式找到答案。\n",
    "- **训练阶段**不需要使用 `doc_stride`，因为训练时我们已经知道答案的位置，可以直接截取包含答案的窗口。但在**验证和测试**阶段，由于模型并不知道答案的位置，`doc_stride` 保证每个窗口之间有足够的重叠，避免遗漏答案。\n",
    "- 所以这里存在一个问题，训练过程中模型可能学习到：答案就在中间这一模式。这是我们在 Strong baseline 中需要解决的。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "5_H1kqhR8CdM"
   },
   "source": [
    "### 评估函数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "SqeA3PLPxOHu"
   },
   "outputs": [],
   "source": [
    "def evaluate(data, output):\n",
    "    ##### TODO: 后处理 #####\n",
    "    # 后处理存在一个错误和改进的空间\n",
    "    # 提示：打开你的预测文件，看看有什么问题\n",
    "\n",
    "    answer = ''\n",
    "    max_prob = float('-inf')\n",
    "    num_of_windows = data[0].shape[1]\n",
    "\n",
    "    for k in range(num_of_windows):\n",
    "        # 通过选择最可能的起始位置/结束位置来获得答案\n",
    "        start_prob, start_index = torch.max(output.start_logits[k], dim=0)\n",
    "        end_prob, end_index = torch.max(output.end_logits[k], dim=0)\n",
    "\n",
    "        # 答案的概率计算为 start_prob 和 end_prob 的和\n",
    "        prob = start_prob + end_prob\n",
    "\n",
    "        # 如果计算的概率大于之前的窗口，则替换答案\n",
    "        if prob > max_prob:\n",
    "            max_prob = prob\n",
    "            # 将标记转换为字符（例如，[1920, 7032] --> \"大 金\"）\n",
    "            answer = tokenizer.decode(data[0][0][k][start_index : end_index + 1])\n",
    "\n",
    "    # 移除答案中的空格（例如，\"大 金\" --> \"大金\"）\n",
    "    return answer.replace(' ','')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "rzHQit6eMnKG"
   },
   "source": [
    "### 训练"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 778,
     "referenced_widgets": [
      "53c1ce79c38b45fc8d1b675515642c54",
      "9370d0df1bca4b9a9c15e1d3e10142d9",
      "611f0f0c70a5471b891bd6f6b8cb7863",
      "986b627aef004704ac0806a9aa189938",
      "f2b466617f0247638e0934c8fe95ed1d",
      "f9b28a5fbcd14568983ed36f9a1ca997",
      "04c0aa3d24944485b475601f786b56f1",
      "78818b2d8cc748efbf1b5784a9ee2a85",
      "7c61a280288d4608b6ef341ce3adb7a8",
      "8710bb7686df4fb3a73e5af839be5305",
      "c7896f62a30f4eec95e3c84aef08e1b2",
      "29a7b6bb49004c3c96a6d48cd55e1ec3",
      "06e4e53bd3e748ad90d14eeec117aac9",
      "6a47a3bc614a44e7a5b0dfef50cadaf7",
      "fba6bb8f65904ae6ae4f5bca95e942e2",
      "6d5ad2f51bb3459b85201f983a499ea2",
      "9c7a21dc260a4ad7b87f2123947058a2",
      "7b7a0c46d9794d6fa66401f03684dec0",
      "11dcde71ff984d829ec9612e7f43e3a3",
      "a987cb79f4904973958a98f492c6443a",
      "8d791392eed34a9b8e155da9a4b0a0d6",
      "bab8f16e038b440a8016ef087ee12904"
     ]
    },
    "id": "3Q-B6ka7xoCM",
    "outputId": "69c6f306-8a93-4b59-a64d-41f55889a6e6"
   },
   "outputs": [],
   "source": [
    "from accelerate import Accelerator\n",
    "\n",
    "# 超参数\n",
    "num_epoch = 1\n",
    "validation = True\n",
    "logging_step = 100\n",
    "learning_rate = 1e-5\n",
    "optimizer = AdamW(model.parameters(), lr=learning_rate)\n",
    "train_batch_size = 8\n",
    "\n",
    "#### TODO: 梯度累积（可选）####\n",
    "# 注意：train_batch_size * gradient_accumulation_steps = 有效批次大小\n",
    "# 如果 CUDA 内存不足，你可以降低 train_batch_size 并提高 gradient_accumulation_steps\n",
    "# 文档：https://huggingface.co/docs/accelerate/usage_guides/gradient_accumulation\n",
    "gradient_accumulation_steps = 16\n",
    "\n",
    "# 数据加载器\n",
    "# 注意：不要更改 dev_loader / test_loader 的批次大小！\n",
    "# 虽然批次大小=1，但它实际上是由同一对 QA 的多个窗口组成的批次\n",
    "train_loader = DataLoader(train_set, batch_size=train_batch_size, shuffle=True, pin_memory=True)\n",
    "dev_loader = DataLoader(dev_set, batch_size=1, shuffle=False, pin_memory=True)\n",
    "test_loader = DataLoader(test_set, batch_size=1, shuffle=False, pin_memory=True)\n",
    "\n",
    "# 将 \"fp16_training\" 更改为 True 以支持自动混合精度训练（fp16）\n",
    "fp16_training = True\n",
    "if fp16_training:\n",
    "    accelerator = Accelerator(mixed_precision=\"fp16\")\n",
    "else:\n",
    "    accelerator = Accelerator()\n",
    "\n",
    "# 文档：https://huggingface.co/docs/accelerate/\n",
    "model, optimizer, train_loader = accelerator.prepare(model, optimizer, train_loader)\n",
    "\n",
    "model.train()\n",
    "\n",
    "print(\"开始训练...\")\n",
    "\n",
    "for epoch in range(num_epoch):\n",
    "    step = 1\n",
    "    train_loss = train_acc = 0\n",
    "\n",
    "    for data in tqdm(train_loader):\n",
    "        # 将所有数据加载到设备\n",
    "        data = [i.to(device) for i in data]\n",
    "\n",
    "        # 模型输入：input_ids, token_type_ids, attention_mask, start_positions, end_positions（注意：只有 \"input_ids\" 是必需的）\n",
    "        # 模型输出：start_logits, end_logits, loss（提供 start_positions/end_positions 时返回）\n",
    "        output = model(input_ids=data[0], token_type_ids=data[1], attention_mask=data[2], start_positions=data[3], end_positions=data[4])\n",
    "        # 选择最可能的起始位置/结束位置\n",
    "        start_index = torch.argmax(output.start_logits, dim=1)\n",
    "        end_index = torch.argmax(output.end_logits, dim=1)\n",
    "\n",
    "        # 只有当 start_index 和 end_index 都正确时，预测才正确\n",
    "        train_acc += ((start_index == data[3]) & (end_index == data[4])).float().mean()\n",
    "\n",
    "        train_loss += output.loss\n",
    "\n",
    "        accelerator.backward(output.loss)\n",
    "\n",
    "        step += 1\n",
    "        optimizer.step()\n",
    "        optimizer.zero_grad()\n",
    "\n",
    "        ##### TODO: 应用线性学习率衰减 #####\n",
    "\n",
    "        # 每经过 logging_step，打印训练损失和准确率\n",
    "        if step % logging_step == 0:\n",
    "            print(f\"Epoch {epoch + 1} | Step {step} | loss = {train_loss.item() / logging_step:.3f}, acc = {train_acc / logging_step:.3f}\")\n",
    "            train_loss = train_acc = 0\n",
    "\n",
    "    if validation:\n",
    "        print(\"评估开发集...\")\n",
    "        model.eval()\n",
    "        with torch.no_grad():\n",
    "            dev_acc = 0\n",
    "            for i, data in enumerate(tqdm(dev_loader)):\n",
    "                output = model(input_ids=data[0].squeeze(dim=0).to(device), token_type_ids=data[1].squeeze(dim=0).to(device),\n",
    "                       attention_mask=data[2].squeeze(dim=0).to(device))\n",
    "                # 只有当答案文本完全匹配时，预测才正确\n",
    "                dev_acc += evaluate(data, output) == dev_questions[i][\"answer_text\"]\n",
    "            print(f\"Validation | Epoch {epoch + 1} | acc = {dev_acc / len(dev_loader):.3f}\")\n",
    "        model.train()\n",
    "\n",
    "# 将模型及其配置文件保存到目录「saved_model」\n",
    "# 即，在目录「saved_model」下有两个文件：「pytorch_model.bin」和「config.json」\n",
    "# 可以使用「model = BertForQuestionAnswering.from_pretrained(\"saved_model\")」重新加载保存的模型\n",
    "print(\"保存模型...\")\n",
    "model_save_dir = \"saved_model\"\n",
    "model.save_pretrained(model_save_dir)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "kMmdLOKBMsdE"
   },
   "source": [
    "### 测试"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 84,
     "referenced_widgets": [
      "9c84cbc758db46e78d7fa361a385ccce",
      "642db3bee97a4bea8a4c1079e2b7526c",
      "ed9f5d147e7844a187d8610130f1ecd9",
      "42dd646b18b24deda274c857d337c973",
      "9e2635394d5b4965b41a05a9bc37a4ec",
      "3366101d46a847dfa5700ffb652dcc86",
      "a1602e58c3a7413490ce0a93f35a7ca9",
      "d26f522a28d24494914df1dd8432af49",
      "c24b174369fb48cdbc3d4afaa4cceccb",
      "a0f74d882c63403c8116c19745e011ba",
      "55a7b680dae94cba8758288dacff2988"
     ]
    },
    "id": "U5scNKC9xz0C",
    "outputId": "841c70f2-2f3c-4c0f-866e-143f127eb6b5"
   },
   "outputs": [],
   "source": [
    "print(\"评估测试集...\")\n",
    "\n",
    "result = []\n",
    "\n",
    "model.eval()\n",
    "with torch.no_grad():\n",
    "    for data in tqdm(test_loader):\n",
    "        output = model(input_ids=data[0].squeeze(dim=0).to(device), token_type_ids=data[1].squeeze(dim=0).to(device),\n",
    "                       attention_mask=data[2].squeeze(dim=0).to(device))\n",
    "        result.append(evaluate(data, output))\n",
    "\n",
    "result_file = \"result.csv\"\n",
    "with open(result_file, 'w') as f:\n",
    "    f.write(\"ID,Answer\\n\")\n",
    "    for i, test_question in enumerate(test_questions):\n",
    "        # 将答案中的逗号替换为空字符串（因为 csv 以逗号分隔）\n",
    "        # Kaggle 中的答案也以同样的方式处理\n",
    "        f.write(f\"{test_question['id']},{result[i].replace(',','')}\\n\")\n",
    "\n",
    "print(f\"完成！结果保存在 {result_file}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 拓展\n",
    "\n",
    "### Question 1\n",
    "\n",
    ">微调（Finetune）和提示（Prompt）之间存在一些差异。上下文学习（In-context Learning）可以让预训练模型在不进行梯度下降的情况下，通过少量示例给出许多下游任务的正确预测。请回答以下问题：\n",
    ">\n",
    ">a. **Encoder-only 模型（如 BERT 系列）如何在抽取式问答任务中确定答案？**\n",
    ">\n",
    ">b. **Decoder-only 模型（如 GPT 系列）如何在抽取式问答任务中确定答案？**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "0_JVNKOCywbR"
   },
   "source": [
    "### Question 2 (In-context learning)\n",
    "\n",
    "> 尝试不同的 Prompt 并观察 fine-tuning 和 in-context learning 的区别。\n",
    ">\n",
    "> 代码所下载的模型是 [facebook/xglm-1.7B](https://huggingface.co/facebook/xglm-1.7B)，实际上你也可以直接去 GPT 或者其他 AI 平台提问，这里的目的是去调整自己的 prompt，从而使模型不经过微调也能获取到正确答案。\n",
    ">\n",
    "> 对于“老手”来说，完全可以跳过这个问题。\n",
    "\n",
    "#### Prompt 对比（错误对比）\n",
    "\n",
    "1. **Prompt 示例 1**: \"根据文章找出问题的答案：{问题}\"。\n",
    "2. **Prompt 示例 2**: \"请阅读文章并回答以下问题：{问题}\"。\n",
    "3. **Prompt 示例 3**: \"请根据文章信息回答下列问题：{问题}\"。\n",
    "\n",
    "#### Prompt 对比（正确对比）\n",
    "\n",
    "> ![image-20240920173255150](../Guide/assets/9805a280ab816c41ec21aee5a223bf9d.png)\n",
    "\n",
    "- 中英对比\n",
    "- 不同 prompt 对比\n",
    "\n",
    "**样例 prompt**:\n",
    "```\n",
    "請從最後一篇的文章中找出最後一個問題的答案：\n",
    "文章：<文章1 內容>\n",
    "問題：<問題1 敘述>\n",
    "答案：<答案1>\n",
    "...\n",
    "文章：<文章n 內容>\n",
    "問題：<問題n 敘述>\n",
    "答案：\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "sGUxltxqzKpm"
   },
   "outputs": [],
   "source": [
    "import torch\n",
    "import random\n",
    "import numpy as np\n",
    "\n",
    "# 为避免 CUDA_OUT_OF_MEMORY\n",
    "torch.set_default_tensor_type(torch.cuda.FloatTensor)\n",
    "\n",
    "# 为了可重复性，固定随机种子\n",
    "def same_seeds(seed):\n",
    "    torch.manual_seed(seed)\n",
    "    if torch.cuda.is_available():\n",
    "        torch.cuda.manual_seed(seed)\n",
    "        torch.cuda.manual_seed_all(seed)\n",
    "    np.random.seed(seed)\n",
    "    random.seed(seed)\n",
    "    torch.backends.cudnn.benchmark = False\n",
    "    torch.backends.cudnn.deterministic = True\n",
    "same_seeds(2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 241,
     "referenced_widgets": [
      "fd1e236692ec43a18a1750a29515050d",
      "7b3c48ae548848c880439841c0b9cb6b",
      "1023013dc30a4731b61383bd59f2dc91",
      "ee0cb4c1f31346b5b180a7b826688eb4",
      "0f7a38ddfe7b42eaa5c8ffdb3c150641",
      "eb405f797a1441d280029e6e203befce",
      "7a7081fd9d5942af9148e29054b18f51",
      "eb4bdd1ff3cf4b0db273c53e32cad5de",
      "7c852afee0de49fe8dac74f3af82a725",
      "24b96f20dd5f4a6d8cf9788e97a74f73",
      "a5bf4132f3dd4c7a807f27c8106fdd59",
      "0549ffe5a6a444dd9fbf8c4513ebe7f7",
      "388d3d2566ff4a1cb68048ac78dd374f",
      "6235eaf236d643c6a163dddb86d3664d",
      "faf78b2739f841d8a213e4311926b598",
      "91f03b6ed1c94cb68970f45ffa64695e",
      "9e312a0ec6a2478fb0f55691caf6d9ae",
      "39d35e6e28264b5ca157745d0c1e1491",
      "c497cc2cfabb451aad71f5d7b19873e4",
      "7f7f2c3afc2647c086877a2f4eff6c12",
      "0386ad4eea2b4387b48cd58e95bcd4ba",
      "028ec097e62b4ee8a463708f4c3b8f24",
      "f607835a7acf4037821885704bde736d",
      "ddd59aac5a5d48ceace9ede35bf8f11b",
      "20e14c8de0f84979ac3f0d896263f49d",
      "4d7c3ea23e9b42a2ba5b0b11ee190c69",
      "5d9e4fd05f964fd7a11d66b964ea0873",
      "7fb466a306e6443287cb2fa2bea90f06",
      "1b2b45ecc27b4f28bc0194709fed0638",
      "6a2f5722362c4ec08ae8ed7d681dc7ba",
      "f51192579d014a52b132f99abf8901b2",
      "6912ca44b1864902a3e32d4a19a4972d",
      "acbfe8b88e6945ab9a5959fbb5f294de",
      "fda75dc7c2d24f7b827b2c78a6988586",
      "2d1a48d240b949ad9b465985e90557c5",
      "5a0f5ba8ad024f5b915e34e48f269437",
      "bc4b72ff74d84c9e83b328e8823044b1",
      "901f4ebc70f144c188d03a4978920441",
      "98ce66c91c5c45968a3844fb0de50624",
      "a24d04e7064048c2819ef88a50ce9093",
      "154be9baa03943d08c975dbc85c4edf8",
      "2fe4fa69d9544fd393a4d72c02245245",
      "c4da9fed45254ebdaa49312f16d9b159",
      "53a2b78a732c4522805de5c9aed92c41",
      "0440f9e26a0b420b86175bc62e212d01",
      "c9bf7b5808f74db9811f5b64e878dcd5",
      "66a35d11743141b1b20904b13a4b86ea",
      "c0145fad2cb843e9b75ac47d307cc346",
      "d61e0c79febd45c8b00e3566296d7e8e",
      "2a59b752f9414e69b8c1d24bd24159f2",
      "447bb12155cf4dc193ffea3f64c6a8f8",
      "16b30759d76a4c8f979212d1ae30c297",
      "1639410507da4b54996064d9fcc8621c",
      "e515df89088042dbbe23594e5976e10d",
      "0dccf470d87f4585af65826582399466",
      "7dee4e5320d349159422709a69af61f4",
      "86dbdec7087f4312b945c3a3940d8965",
      "cdbf6c3b5b3948229946fd51c227b51e",
      "d132ff798f284c9cbe2a73d35d776592",
      "fead01a1a86c40bcb1ba9133ef351d7c",
      "f19d9d34c978402691a89114849d2bb9",
      "7a729fb76a374da99334e7d4b4de102e",
      "cccbdce89ff545f6a66304352b320ad7",
      "cbd776799c18488eb74a8f17cc27c6fd",
      "b1cd7728deff4d50a18cb22805351330",
      "550e7aaac7fa4c3db37ceca6a5595ddb",
      "745f78a378b5412eae007f029e8c321a",
      "2c449a0e664c41a79b3a1de62e94b5b7",
      "a9a16bf8795c402a90cf4504b1cd30a1",
      "715ad9f6bdaa46298317e049404f9755",
      "667f153201d347a0a01e39b438fa736b",
      "d652cdca44e04070a420c5d1a24b4fd1",
      "b5c71e0a94014278a22ba45a6f58b77d",
      "7f928c5e620a49c1ae137868e34818a9",
      "d403f126d48746948c0f627dfdce5d65",
      "d41e81f720f44b52b9a914e4c619bfee",
      "bee7c0220d4e476dbc035fbc5dc44fa3"
     ]
    },
    "id": "wIS23s79zItf",
    "outputId": "c686be61-221a-4f55-ca79-67715343f1db"
   },
   "outputs": [],
   "source": [
    "from transformers import AutoTokenizer, AutoModelForCausalLM\n",
    "\n",
    "# 你可以尝试不同大小的模型\n",
    "# 当使用 Colab 或 Kaggle 时，超过 20 亿（2B）参数的模型可能会耗尽内存\n",
    "tokenizer = AutoTokenizer.from_pretrained(\"facebook/xglm-1.7B\")\n",
    "model = AutoModelForCausalLM.from_pretrained(\"facebook/xglm-1.7B\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "2d97YtavzFHA"
   },
   "outputs": [],
   "source": [
    "# 清理模型输出。如果你尝试不同的提示，可能需要自行修正此函数\n",
    "def clean_text(text):\n",
    "    # 注意：当你使用单语模型时，冒号可能会变成全角\n",
    "    text = text.split(\"答案:\")[-1]\n",
    "    text = text.split(\" \")[0]\n",
    "    return text"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "jdf3gdP_yykH",
    "outputId": "b9a784dd-e5d7-4212-c089-e0180157bd11"
   },
   "outputs": [],
   "source": [
    "import random\n",
    "import json\n",
    "\n",
    "with open(\"hw7_in-context-learning-examples.json\", \"r\") as f:\n",
    "    test = json.load(f)\n",
    "\n",
    "# K-shot 学习\n",
    "# 给模型 K 个示例，使其获得更好的准确率\n",
    "# 注意：(1) 当 K >= 4 时，可能会发生 CUDA_OUT_OF_MEMORY。\n",
    "#       (2) XGLM 的最大输入长度为 2048\n",
    "K = 2\n",
    "\n",
    "question_ids = [qa[\"id\"] for qa in test[\"questions\"]]\n",
    "\n",
    "with open(\"in-context-learning-result.txt\", \"w\") as f:\n",
    "    print(\"ID,Ground-Truth,Prediction\", file = f)\n",
    "    with torch.no_grad():\n",
    "        for idx, qa in enumerate(test[\"questions\"]):\n",
    "            # 你可以尝试不同的提示\n",
    "            prompt = \"請從最後一篇的文章中找出最後一個問題的答案\\n\"\n",
    "            exist_question_indexs = [question_ids.index(qa[\"id\"])]\n",
    "\n",
    "            # K-shot 学习：给模型 K 个带有答案的示例\n",
    "            for i in range(K):\n",
    "                question_index = question_ids.index(qa[\"id\"])\n",
    "                while(question_index in exist_question_indexs):\n",
    "                    question_index = random.randint(0, len(question_ids) - 1)\n",
    "                exist_question_indexs.append(question_index)\n",
    "                paragraph_id = test[\"questions\"][question_index][\"paragraph_id\"]\n",
    "                prompt += f'文章：{test[\"paragraphs\"][paragraph_id]}\\n'\n",
    "                prompt += f'問題：{test[\"questions\"][question_index][\"question_text\"]}\\n'\n",
    "                prompt += f'答案：{test[\"questions\"][question_index][\"answer_text\"]}\\n'\n",
    "\n",
    "            # 最后一个没有答案的问题\n",
    "            paragraph_id = qa[\"paragraph_id\"]\n",
    "            prompt += f'文章：{test[\"paragraphs\"][paragraph_id]}\\n'\n",
    "            prompt += f'問題：{qa[\"question_text\"]}\\n'\n",
    "            prompt += f'答案：'\n",
    "\n",
    "            inputs = tokenizer(prompt, add_special_tokens=False, return_tensors=\"pt\")\n",
    "            sample = model.generate(**inputs, max_new_tokens = 20)\n",
    "            text = tokenizer.decode(sample[0], skip_special_tokens=True)\n",
    "\n",
    "            # 注意：你可以删除这一行，看看会发生什么\n",
    "            text = clean_text(text)\n",
    "\n",
    "            print(prompt)\n",
    "            print(f'正確答案: {qa[\"answer_text\"]}')\n",
    "            print(f'模型輸出: {text}')\n",
    "            print()\n",
    "\n",
    "            print(f\"{idx},{qa['answer_text']},{text}\", file = f)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "k3prlGCdjf38"
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "colab": {
   "provenance": []
  },
  "gpuClass": "standard",
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.16"
  },
  "widgets": {
   "application/vnd.jupyter.widget-state+json": {
    "028ec097e62b4ee8a463708f4c3b8f24": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "0386ad4eea2b4387b48cd58e95bcd4ba": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "0440f9e26a0b420b86175bc62e212d01": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_c9bf7b5808f74db9811f5b64e878dcd5",
       "IPY_MODEL_66a35d11743141b1b20904b13a4b86ea",
       "IPY_MODEL_c0145fad2cb843e9b75ac47d307cc346"
      ],
      "layout": "IPY_MODEL_d61e0c79febd45c8b00e3566296d7e8e"
     }
    },
    "04c0aa3d24944485b475601f786b56f1": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "052cccb5d69d476f9caac95848e0353a": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "0549ffe5a6a444dd9fbf8c4513ebe7f7": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_388d3d2566ff4a1cb68048ac78dd374f",
       "IPY_MODEL_6235eaf236d643c6a163dddb86d3664d",
       "IPY_MODEL_faf78b2739f841d8a213e4311926b598"
      ],
      "layout": "IPY_MODEL_91f03b6ed1c94cb68970f45ffa64695e"
     }
    },
    "06e4e53bd3e748ad90d14eeec117aac9": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_9c7a21dc260a4ad7b87f2123947058a2",
      "placeholder": "​",
      "style": "IPY_MODEL_7b7a0c46d9794d6fa66401f03684dec0",
      "value": "100%"
     }
    },
    "0dccf470d87f4585af65826582399466": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "0f7a38ddfe7b42eaa5c8ffdb3c150641": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "1023013dc30a4731b61383bd59f2dc91": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_eb4bdd1ff3cf4b0db273c53e32cad5de",
      "max": 335,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_7c852afee0de49fe8dac74f3af82a725",
      "value": 335
     }
    },
    "1181117783fd4bca9d7f5f296507aff3": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "11dcde71ff984d829ec9612e7f43e3a3": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "154be9baa03943d08c975dbc85c4edf8": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "1639410507da4b54996064d9fcc8621c": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "16b30759d76a4c8f979212d1ae30c297": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "1b2b45ecc27b4f28bc0194709fed0638": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "1e20837bf23548249678da0a07745d58": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_d26e50974c52452389c5abb001056546",
      "max": 29,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_9eefbb580f0146658d18658f210d3a67",
      "value": 29
     }
    },
    "20cd68a7e03f4353aa99fcefc1f16d36": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "20e14c8de0f84979ac3f0d896263f49d": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_6a2f5722362c4ec08ae8ed7d681dc7ba",
      "max": 9032074,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_f51192579d014a52b132f99abf8901b2",
      "value": 9032074
     }
    },
    "23afbee21a644745b24b8ffefed6b286": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "24b96f20dd5f4a6d8cf9788e97a74f73": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "250451c12cee4aacb91a32d7da4ab861": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "28fc746a3c0548f4a4ff669b5dbb7320": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "29a7b6bb49004c3c96a6d48cd55e1ec3": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_06e4e53bd3e748ad90d14eeec117aac9",
       "IPY_MODEL_6a47a3bc614a44e7a5b0dfef50cadaf7",
       "IPY_MODEL_fba6bb8f65904ae6ae4f5bca95e942e2"
      ],
      "layout": "IPY_MODEL_6d5ad2f51bb3459b85201f983a499ea2"
     }
    },
    "2a59b752f9414e69b8c1d24bd24159f2": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "2c37badbe0504073b31abd67a8c0b808": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_bfd7d35860c14dd3b8e5eea07032311d",
       "IPY_MODEL_db2c74f92f6d4a2c8bfdcb91d5bd3d64",
       "IPY_MODEL_b0af063597ea4de5a8c8bf4f370f782a"
      ],
      "layout": "IPY_MODEL_5b1ac1a0f04f4c29b3c1127ec3d95e81"
     }
    },
    "2c449a0e664c41a79b3a1de62e94b5b7": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_d652cdca44e04070a420c5d1a24b4fd1",
      "placeholder": "​",
      "style": "IPY_MODEL_b5c71e0a94014278a22ba45a6f58b77d",
      "value": "Downloading (…)neration_config.json: 100%"
     }
    },
    "2d1a48d240b949ad9b465985e90557c5": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_98ce66c91c5c45968a3844fb0de50624",
      "placeholder": "​",
      "style": "IPY_MODEL_a24d04e7064048c2819ef88a50ce9093",
      "value": "Downloading (…)cial_tokens_map.json: 100%"
     }
    },
    "2fe4fa69d9544fd393a4d72c02245245": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "304609dbefa74735baa08ce7aadff92d": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "3366101d46a847dfa5700ffb652dcc86": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "33fa92f225be477d8cd176a0a4e8894e": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "388d3d2566ff4a1cb68048ac78dd374f": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_9e312a0ec6a2478fb0f55691caf6d9ae",
      "placeholder": "​",
      "style": "IPY_MODEL_39d35e6e28264b5ca157745d0c1e1491",
      "value": "Downloading (…)tencepiece.bpe.model: 100%"
     }
    },
    "39d35e6e28264b5ca157745d0c1e1491": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "3eaf82623d964c2abb92b4509ac5ca9d": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_33fa92f225be477d8cd176a0a4e8894e",
      "placeholder": "​",
      "style": "IPY_MODEL_b80166eeee984661a611a02d1eae4458",
      "value": "Downloading (…)solve/main/vocab.txt: 100%"
     }
    },
    "42dd646b18b24deda274c857d337c973": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_a0f74d882c63403c8116c19745e011ba",
      "placeholder": "​",
      "style": "IPY_MODEL_55a7b680dae94cba8758288dacff2988",
      "value": " 3524/3524 [01:20&lt;00:00, 46.55it/s]"
     }
    },
    "447bb12155cf4dc193ffea3f64c6a8f8": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "4d7c3ea23e9b42a2ba5b0b11ee190c69": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_6912ca44b1864902a3e32d4a19a4972d",
      "placeholder": "​",
      "style": "IPY_MODEL_acbfe8b88e6945ab9a5959fbb5f294de",
      "value": " 9.03M/9.03M [00:00&lt;00:00, 35.9MB/s]"
     }
    },
    "5195ed241f09440eaf84a7e0b3dc8155": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "53a2b78a732c4522805de5c9aed92c41": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "53c1ce79c38b45fc8d1b675515642c54": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_9370d0df1bca4b9a9c15e1d3e10142d9",
       "IPY_MODEL_611f0f0c70a5471b891bd6f6b8cb7863",
       "IPY_MODEL_986b627aef004704ac0806a9aa189938"
      ],
      "layout": "IPY_MODEL_f2b466617f0247638e0934c8fe95ed1d"
     }
    },
    "550e7aaac7fa4c3db37ceca6a5595ddb": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "55a7b680dae94cba8758288dacff2988": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "5a0f5ba8ad024f5b915e34e48f269437": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_154be9baa03943d08c975dbc85c4edf8",
      "max": 276,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_2fe4fa69d9544fd393a4d72c02245245",
      "value": 276
     }
    },
    "5b1ac1a0f04f4c29b3c1127ec3d95e81": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "5d9e4fd05f964fd7a11d66b964ea0873": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "611f0f0c70a5471b891bd6f6b8cb7863": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_78818b2d8cc748efbf1b5784a9ee2a85",
      "max": 3365,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_7c61a280288d4608b6ef341ce3adb7a8",
      "value": 3365
     }
    },
    "6235eaf236d643c6a163dddb86d3664d": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_c497cc2cfabb451aad71f5d7b19873e4",
      "max": 4920706,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_7f7f2c3afc2647c086877a2f4eff6c12",
      "value": 4920706
     }
    },
    "641665cb9ec6433a81719d57eeaf8298": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_ba9c1cd3cfe04add90aa2edf2aa87378",
       "IPY_MODEL_1e20837bf23548249678da0a07745d58",
       "IPY_MODEL_c1d24cb7607c44b989774113a2baf675"
      ],
      "layout": "IPY_MODEL_c9144705a33848a6b142f9887da6ac49"
     }
    },
    "642db3bee97a4bea8a4c1079e2b7526c": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_3366101d46a847dfa5700ffb652dcc86",
      "placeholder": "​",
      "style": "IPY_MODEL_a1602e58c3a7413490ce0a93f35a7ca9",
      "value": "100%"
     }
    },
    "667f153201d347a0a01e39b438fa736b": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "66a35d11743141b1b20904b13a4b86ea": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_16b30759d76a4c8f979212d1ae30c297",
      "max": 548,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_1639410507da4b54996064d9fcc8621c",
      "value": 548
     }
    },
    "6912ca44b1864902a3e32d4a19a4972d": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "6a2f5722362c4ec08ae8ed7d681dc7ba": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "6a47a3bc614a44e7a5b0dfef50cadaf7": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_11dcde71ff984d829ec9612e7f43e3a3",
      "max": 2863,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_a987cb79f4904973958a98f492c6443a",
      "value": 2863
     }
    },
    "6be63de77ed14eee9c37e93d3a75f743": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_304609dbefa74735baa08ce7aadff92d",
      "placeholder": "​",
      "style": "IPY_MODEL_250451c12cee4aacb91a32d7da4ab861",
      "value": " 110k/110k [00:00&lt;00:00, 1.53MB/s]"
     }
    },
    "6d5ad2f51bb3459b85201f983a499ea2": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "715ad9f6bdaa46298317e049404f9755": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_d41e81f720f44b52b9a914e4c619bfee",
      "placeholder": "​",
      "style": "IPY_MODEL_bee7c0220d4e476dbc035fbc5dc44fa3",
      "value": " 168/168 [00:00&lt;00:00, 4.67kB/s]"
     }
    },
    "745f78a378b5412eae007f029e8c321a": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_2c449a0e664c41a79b3a1de62e94b5b7",
       "IPY_MODEL_a9a16bf8795c402a90cf4504b1cd30a1",
       "IPY_MODEL_715ad9f6bdaa46298317e049404f9755"
      ],
      "layout": "IPY_MODEL_667f153201d347a0a01e39b438fa736b"
     }
    },
    "78818b2d8cc748efbf1b5784a9ee2a85": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "7a7081fd9d5942af9148e29054b18f51": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "7a729fb76a374da99334e7d4b4de102e": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "7b3c48ae548848c880439841c0b9cb6b": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_eb405f797a1441d280029e6e203befce",
      "placeholder": "​",
      "style": "IPY_MODEL_7a7081fd9d5942af9148e29054b18f51",
      "value": "Downloading (…)okenizer_config.json: 100%"
     }
    },
    "7b7a0c46d9794d6fa66401f03684dec0": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "7c61a280288d4608b6ef341ce3adb7a8": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "7c852afee0de49fe8dac74f3af82a725": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "7dee4e5320d349159422709a69af61f4": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_86dbdec7087f4312b945c3a3940d8965",
       "IPY_MODEL_cdbf6c3b5b3948229946fd51c227b51e",
       "IPY_MODEL_d132ff798f284c9cbe2a73d35d776592"
      ],
      "layout": "IPY_MODEL_fead01a1a86c40bcb1ba9133ef351d7c"
     }
    },
    "7f7f2c3afc2647c086877a2f4eff6c12": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "7f928c5e620a49c1ae137868e34818a9": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "7fb466a306e6443287cb2fa2bea90f06": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "86dbdec7087f4312b945c3a3940d8965": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_f19d9d34c978402691a89114849d2bb9",
      "placeholder": "​",
      "style": "IPY_MODEL_7a729fb76a374da99334e7d4b4de102e",
      "value": "Downloading pytorch_model.bin: 100%"
     }
    },
    "8710bb7686df4fb3a73e5af839be5305": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "8b264499afca4694849b08e2091f6826": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_3eaf82623d964c2abb92b4509ac5ca9d",
       "IPY_MODEL_aa3ff7831fbf47efa404805886ca4696",
       "IPY_MODEL_6be63de77ed14eee9c37e93d3a75f743"
      ],
      "layout": "IPY_MODEL_5195ed241f09440eaf84a7e0b3dc8155"
     }
    },
    "8d791392eed34a9b8e155da9a4b0a0d6": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "901f4ebc70f144c188d03a4978920441": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "91f03b6ed1c94cb68970f45ffa64695e": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "9370d0df1bca4b9a9c15e1d3e10142d9": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_f9b28a5fbcd14568983ed36f9a1ca997",
      "placeholder": "​",
      "style": "IPY_MODEL_04c0aa3d24944485b475601f786b56f1",
      "value": "100%"
     }
    },
    "941a345ba82a44d5ad597a15c0af0d5c": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "986b627aef004704ac0806a9aa189938": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_8710bb7686df4fb3a73e5af839be5305",
      "placeholder": "​",
      "style": "IPY_MODEL_c7896f62a30f4eec95e3c84aef08e1b2",
      "value": " 3365/3365 [09:20&lt;00:00,  6.34it/s]"
     }
    },
    "98ce66c91c5c45968a3844fb0de50624": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "9c7a21dc260a4ad7b87f2123947058a2": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "9c84cbc758db46e78d7fa361a385ccce": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_642db3bee97a4bea8a4c1079e2b7526c",
       "IPY_MODEL_ed9f5d147e7844a187d8610130f1ecd9",
       "IPY_MODEL_42dd646b18b24deda274c857d337c973"
      ],
      "layout": "IPY_MODEL_9e2635394d5b4965b41a05a9bc37a4ec"
     }
    },
    "9e2635394d5b4965b41a05a9bc37a4ec": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "9e312a0ec6a2478fb0f55691caf6d9ae": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "9eefbb580f0146658d18658f210d3a67": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "a075ee85c2d34ab3b2d011c63dc02286": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "a0f74d882c63403c8116c19745e011ba": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "a1602e58c3a7413490ce0a93f35a7ca9": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "a24d04e7064048c2819ef88a50ce9093": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "a5bf4132f3dd4c7a807f27c8106fdd59": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "a987cb79f4904973958a98f492c6443a": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "a9a16bf8795c402a90cf4504b1cd30a1": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_7f928c5e620a49c1ae137868e34818a9",
      "max": 168,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_d403f126d48746948c0f627dfdce5d65",
      "value": 168
     }
    },
    "aa3ff7831fbf47efa404805886ca4696": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_20cd68a7e03f4353aa99fcefc1f16d36",
      "max": 109540,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_fb908d9c3e374983aa47478a95411ba7",
      "value": 109540
     }
    },
    "acbfe8b88e6945ab9a5959fbb5f294de": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "b0af063597ea4de5a8c8bf4f370f782a": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_a075ee85c2d34ab3b2d011c63dc02286",
      "placeholder": "​",
      "style": "IPY_MODEL_052cccb5d69d476f9caac95848e0353a",
      "value": " 269k/269k [00:00&lt;00:00, 2.44MB/s]"
     }
    },
    "b1cd7728deff4d50a18cb22805351330": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "b5c71e0a94014278a22ba45a6f58b77d": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "b80166eeee984661a611a02d1eae4458": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "b91c8359a6444497b3665e66afbe237d": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "ba9c1cd3cfe04add90aa2edf2aa87378": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_ebfa4ba9f18b4945947d326581d3b8e9",
      "placeholder": "​",
      "style": "IPY_MODEL_28fc746a3c0548f4a4ff669b5dbb7320",
      "value": "Downloading (…)okenizer_config.json: 100%"
     }
    },
    "bab8f16e038b440a8016ef087ee12904": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "bc4b72ff74d84c9e83b328e8823044b1": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_c4da9fed45254ebdaa49312f16d9b159",
      "placeholder": "​",
      "style": "IPY_MODEL_53a2b78a732c4522805de5c9aed92c41",
      "value": " 276/276 [00:00&lt;00:00, 14.5kB/s]"
     }
    },
    "bee7c0220d4e476dbc035fbc5dc44fa3": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "bfd7d35860c14dd3b8e5eea07032311d": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_b91c8359a6444497b3665e66afbe237d",
      "placeholder": "​",
      "style": "IPY_MODEL_1181117783fd4bca9d7f5f296507aff3",
      "value": "Downloading (…)/main/tokenizer.json: 100%"
     }
    },
    "c0145fad2cb843e9b75ac47d307cc346": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_e515df89088042dbbe23594e5976e10d",
      "placeholder": "​",
      "style": "IPY_MODEL_0dccf470d87f4585af65826582399466",
      "value": " 548/548 [00:00&lt;00:00, 21.8kB/s]"
     }
    },
    "c1d24cb7607c44b989774113a2baf675": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_c813a6f6ce09424582c06a3ac8a7d712",
      "placeholder": "​",
      "style": "IPY_MODEL_ff6cd33dca3c44ba9832b2d705fd2de1",
      "value": " 29.0/29.0 [00:00&lt;00:00, 1.06kB/s]"
     }
    },
    "c24b174369fb48cdbc3d4afaa4cceccb": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "c497cc2cfabb451aad71f5d7b19873e4": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "c4da9fed45254ebdaa49312f16d9b159": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "c7896f62a30f4eec95e3c84aef08e1b2": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "c813a6f6ce09424582c06a3ac8a7d712": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "c9144705a33848a6b142f9887da6ac49": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "c9bf7b5808f74db9811f5b64e878dcd5": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_2a59b752f9414e69b8c1d24bd24159f2",
      "placeholder": "​",
      "style": "IPY_MODEL_447bb12155cf4dc193ffea3f64c6a8f8",
      "value": "Downloading (…)lve/main/config.json: 100%"
     }
    },
    "cbd776799c18488eb74a8f17cc27c6fd": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "cccbdce89ff545f6a66304352b320ad7": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "cdbf6c3b5b3948229946fd51c227b51e": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_cccbdce89ff545f6a66304352b320ad7",
      "max": 3465946427,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_cbd776799c18488eb74a8f17cc27c6fd",
      "value": 3465946427
     }
    },
    "d132ff798f284c9cbe2a73d35d776592": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_b1cd7728deff4d50a18cb22805351330",
      "placeholder": "​",
      "style": "IPY_MODEL_550e7aaac7fa4c3db37ceca6a5595ddb",
      "value": " 3.47G/3.47G [00:19&lt;00:00, 179MB/s]"
     }
    },
    "d26e50974c52452389c5abb001056546": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "d26f522a28d24494914df1dd8432af49": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "d403f126d48746948c0f627dfdce5d65": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "d41e81f720f44b52b9a914e4c619bfee": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "d61e0c79febd45c8b00e3566296d7e8e": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "d652cdca44e04070a420c5d1a24b4fd1": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "db2c74f92f6d4a2c8bfdcb91d5bd3d64": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_941a345ba82a44d5ad597a15c0af0d5c",
      "max": 268943,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_23afbee21a644745b24b8ffefed6b286",
      "value": 268943
     }
    },
    "ddd59aac5a5d48ceace9ede35bf8f11b": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_7fb466a306e6443287cb2fa2bea90f06",
      "placeholder": "​",
      "style": "IPY_MODEL_1b2b45ecc27b4f28bc0194709fed0638",
      "value": "Downloading (…)/main/tokenizer.json: 100%"
     }
    },
    "e515df89088042dbbe23594e5976e10d": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "eb405f797a1441d280029e6e203befce": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "eb4bdd1ff3cf4b0db273c53e32cad5de": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "ebfa4ba9f18b4945947d326581d3b8e9": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "ed9f5d147e7844a187d8610130f1ecd9": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_d26f522a28d24494914df1dd8432af49",
      "max": 3524,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_c24b174369fb48cdbc3d4afaa4cceccb",
      "value": 3524
     }
    },
    "ee0cb4c1f31346b5b180a7b826688eb4": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_24b96f20dd5f4a6d8cf9788e97a74f73",
      "placeholder": "​",
      "style": "IPY_MODEL_a5bf4132f3dd4c7a807f27c8106fdd59",
      "value": " 335/335 [00:00&lt;00:00, 11.3kB/s]"
     }
    },
    "f19d9d34c978402691a89114849d2bb9": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "f2b466617f0247638e0934c8fe95ed1d": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "f51192579d014a52b132f99abf8901b2": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "f607835a7acf4037821885704bde736d": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_ddd59aac5a5d48ceace9ede35bf8f11b",
       "IPY_MODEL_20e14c8de0f84979ac3f0d896263f49d",
       "IPY_MODEL_4d7c3ea23e9b42a2ba5b0b11ee190c69"
      ],
      "layout": "IPY_MODEL_5d9e4fd05f964fd7a11d66b964ea0873"
     }
    },
    "f9b28a5fbcd14568983ed36f9a1ca997": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "faf78b2739f841d8a213e4311926b598": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_0386ad4eea2b4387b48cd58e95bcd4ba",
      "placeholder": "​",
      "style": "IPY_MODEL_028ec097e62b4ee8a463708f4c3b8f24",
      "value": " 4.92M/4.92M [00:00&lt;00:00, 41.4MB/s]"
     }
    },
    "fb908d9c3e374983aa47478a95411ba7": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "fba6bb8f65904ae6ae4f5bca95e942e2": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_8d791392eed34a9b8e155da9a4b0a0d6",
      "placeholder": "​",
      "style": "IPY_MODEL_bab8f16e038b440a8016ef087ee12904",
      "value": " 2863/2863 [01:03&lt;00:00, 45.98it/s]"
     }
    },
    "fd1e236692ec43a18a1750a29515050d": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_7b3c48ae548848c880439841c0b9cb6b",
       "IPY_MODEL_1023013dc30a4731b61383bd59f2dc91",
       "IPY_MODEL_ee0cb4c1f31346b5b180a7b826688eb4"
      ],
      "layout": "IPY_MODEL_0f7a38ddfe7b42eaa5c8ffdb3c150641"
     }
    },
    "fda75dc7c2d24f7b827b2c78a6988586": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_2d1a48d240b949ad9b465985e90557c5",
       "IPY_MODEL_5a0f5ba8ad024f5b915e34e48f269437",
       "IPY_MODEL_bc4b72ff74d84c9e83b328e8823044b1"
      ],
      "layout": "IPY_MODEL_901f4ebc70f144c188d03a4978920441"
     }
    },
    "fead01a1a86c40bcb1ba9133ef351d7c": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "ff6cd33dca3c44ba9832b2d705fd2de1": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    }
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}