Demos/20. 嵌入层 nn.Embedding() 代码示例.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "bb641062-5756-4915-aae0-d3ed2fc5c0b9",
   "metadata": {},
   "source": [
    "# 嵌入层 nn.Embedding() 代码示例\n",
    "\n",
    "> 引导文章：[g. 嵌入层 nn.Embedding() 详解和要点提醒（PyTorch）](https://github.com/Hoper-J/AI-Guide-and-Demos-zh_CN/blob/master/Guide/g.%20嵌入层%20nn.Embedding()%20详解和要点提醒（PyTorch）.md)\n",
    "\n",
    "在线链接：[Kaggle](https://www.kaggle.com/code/aidemos/20-nn-embedding) | [Colab](https://colab.research.google.com/drive/1BLgmxasxOD1HQGrI0L1sPlYG40iba9SY?usp=sharing)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fca454d7-797e-4cd9-9e68-54833787ddbf",
   "metadata": {},
   "source": [
    "> `torch.nn.Embedding(num_embeddings, embedding_dim, padding_idx=None, max_norm=None, norm_type=2.0, scale_grad_by_freq=False, sparse=False, _weight=None, _freeze=False, device=None, dtype=None)`\n",
    ">\n",
    "> A simple lookup table that stores embeddings of a fixed dictionary and size.\n",
    ">\n",
    "> 一个简单的查找表，用于存储固定大小的字典中每个词的嵌入向量。\n",
    "\n",
    "### 参数\n",
    "\n",
    "- **num_embeddings** (int): 嵌入字典的大小，即词汇表的大小 (vocab size)。\n",
    "- **embedding_dim** (int): 每个嵌入向量的维度大小。\n",
    "- **padding_idx** (int, 可选): 指定填充（`<PAD>`）对应的索引值。`padding_idx` 对应的嵌入向量在训练过程中不会更新，即梯度不参与反向传播。对于新构建的 `Embedding`，`padding_idx` 处的嵌入向量默认为全零，但可以手动更新为其他值。\n",
    "- **max_norm** (float, 可选):  如果设置，嵌入向量的范数超过此值时将被重新归一化，使其范数等于 `max_norm`。\n",
    "- **norm_type** (float, 可选): 用于计算 `max_norm` 的 p-范数，默认为 2，即计算 2 范数。\n",
    "- **scale_grad_by_freq** (bool, 可选): 如果为 `True`，梯度将根据单词在 mini-batch 中的频率的倒数进行缩放，适用于高频词的梯度调整。默认为 `False`。\n",
    "- **sparse** (bool, 可选): 如果设置为 `True`，则权重矩阵的梯度为稀疏张量，适合大规模词汇表的内存优化。\n",
    "\n",
    "### 属性\n",
    "\n",
    "- **weight** (Tensor): 模块的可学习权重，形状为 `(num_embeddings, embedding_dim)`，初始值从正态分布 `N(0, 1)` 中采样。\n",
    "\n",
    "### 方法\n",
    "\n",
    "> `from_pretrained(embeddings, freeze=True, padding_idx=None, max_norm=None, norm_type=2.0, scale_grad_by_freq=False, sparse=False)`\n",
    ">\n",
    "> Create Embedding instance from given 2-dimensional FloatTensor.\n",
    ">\n",
    "> 用于从给定的 2 维浮点张量（FloatTensor）创建一个 `Embedding` 实例，通俗来讲就是自定义二维的权重矩阵。\n",
    "\n",
    "#### 参数\n",
    "\n",
    "- **embeddings** (Tensor): 一个包含嵌入权重的 `FloatTensor`。第一个维度代表 `num_embeddings`（词汇表大小，vocab_size），第二个维度代表 `embedding_dim`（嵌入向量维度）。\n",
    "- **freeze** (bool, 可选): 如果为 `True`，则嵌入矩阵在训练过程中保持不变，相当于设置 `embedding.weight.requires_grad = False`。默认值为 `True`。\n",
    "- 其余参数参考之前定义。\n",
    "\n",
    "## 数学公式\n",
    "\n",
    "假设词汇表大小为 $V$，嵌入维度为 $D$，则嵌入层可以表示为一个矩阵 $E \\in \\mathbb{R}^{V \\times D}$。\n",
    "\n",
    "给定一个输入的 token ID 序列 $\\{x_1, x_2, \\dots, x_n\\}$，嵌入层的输出为对应的嵌入向量序列 $\\{E_{x_1}, E_{x_2}, \\dots, E_{x_n}\\}$，其中每个 $E_{x_i} \\in \\mathbb{R}^D$。\n",
    "\n",
    "嵌入层接受 $x_i$ 作为输入，返回对应的行向量，公式如下：\n",
    "\n",
    "$$\n",
    "E(x_i) = E_{x_i}\n",
    "$$\n",
    "其中，$E$ 是嵌入矩阵，$x_i$ 是输入的 token ID，$E(x_i)$ 是对应的嵌入向量。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "657b37f1-e9e0-46fa-bdd1-5c898f894231",
   "metadata": {},
   "source": [
    "## 使用示例\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "2d473fa4-2377-4d20-8b80-3896d91f66fe",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "权重矩阵：\n",
      "tensor([[ 0.3367,  0.1288,  0.2345],\n",
      "        [ 0.2303, -1.1229, -0.1863],\n",
      "        [ 2.2082, -0.6380,  0.4617],\n",
      "        [ 0.2674,  0.5349,  0.8094],\n",
      "        [ 1.1103, -1.6898, -0.9890]])\n",
      "\n",
      "Embedding 输出：\n",
      "tensor([[ 0.3367,  0.1288,  0.2345],\n",
      "        [ 2.2082, -0.6380,  0.4617],\n",
      "        [ 1.1103, -1.6898, -0.9890]], grad_fn=<EmbeddingBackward0>)\n"
     ]
    }
   ],
   "source": [
    "import torch\n",
    "import torch.nn as nn\n",
    "\n",
    "# 设置随机种子以确保结果可复现\n",
    "torch.manual_seed(42)\n",
    "\n",
    "# 定义嵌入层参数\n",
    "num_embeddings = 5  # 假设词汇表中有 5 个 token\n",
    "embedding_dim = 3   # 每个 token 对应 3 维嵌入向量\n",
    "\n",
    "# 初始化嵌入层\n",
    "embedding = nn.Embedding(num_embeddings, embedding_dim)\n",
    "\n",
    "# 定义整数索引\n",
    "input_indices = torch.tensor([0, 2, 4])\n",
    "\n",
    "# 查找嵌入向量\n",
    "output = embedding(input_indices)\n",
    "\n",
    "# 打印结果\n",
    "print(\"权重矩阵：\")\n",
    "print(embedding.weight.data)\n",
    "print(\"\\nEmbedding 输出：\")\n",
    "print(output)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0a397f47-238b-4792-ac79-d29ff3ad85d3",
   "metadata": {},
   "source": [
    "## `padding_idx` 的作用"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "5dcbc0c3-443b-43d7-b64f-ed672884d6f2",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "权重矩阵：\n",
      "tensor([[ 0.0000,  0.0000,  0.0000],\n",
      "        [-0.7658, -0.7506,  1.3525],\n",
      "        [ 0.6863, -0.3278,  0.7950],\n",
      "        [ 0.2815,  0.0562,  0.5227],\n",
      "        [-0.2384, -0.0499,  0.5263]])\n"
     ]
    }
   ],
   "source": [
    "import torch\n",
    "import torch.nn as nn\n",
    "\n",
    "# 定义嵌入层，指定 padding_idx=0\n",
    "embedding = nn.Embedding(num_embeddings=5, embedding_dim=3, padding_idx=0)\n",
    "\n",
    "# 打印权重矩阵\n",
    "print(\"权重矩阵：\")\n",
    "print(embedding.weight.data)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3b770a15-389b-4b31-bb86-2ceabde92126",
   "metadata": {},
   "source": [
    "**验证 `padding_idx` 对应的嵌入是否在训练过程中保持不变**：\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "dda62291-7c57-4e84-a12f-d0cae4ccbbaa",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "梯度：\n",
      "tensor([[ 0.0000,  0.0000,  0.0000],\n",
      "        [ 0.0000,  0.0000,  0.0000],\n",
      "        [-0.0395,  0.1529,  0.3742],\n",
      "        [ 0.0000,  0.0000,  0.0000],\n",
      "        [-0.0863,  0.0353,  0.2030]])\n",
      "原权重矩阵：\n",
      "tensor([[ 0.0000,  0.0000,  0.0000],\n",
      "        [-0.7658, -0.7506,  1.3525],\n",
      "        [ 0.6863, -0.3278,  0.7950],\n",
      "        [ 0.2815,  0.0562,  0.5227],\n",
      "        [-0.2384, -0.0499,  0.5263]])\n",
      "权重矩阵更新后:\n",
      "tensor([[ 0.0000,  0.0000,  0.0000],\n",
      "        [-0.7658, -0.7506,  1.3525],\n",
      "        [ 0.6903, -0.3430,  0.7576],\n",
      "        [ 0.2815,  0.0562,  0.5227],\n",
      "        [-0.2297, -0.0534,  0.5060]])\n"
     ]
    }
   ],
   "source": [
    "import torch\n",
    "import torch.optim as optim\n",
    "\n",
    "# 注意之前设置了 padding_idx=0\n",
    "input_indices = torch.tensor([0, 2, 4])\n",
    "\n",
    "# 定义一个简单的损失函数\n",
    "loss_fn = nn.MSELoss()\n",
    "\n",
    "# 目标值\n",
    "target = torch.randn(3, 3)\n",
    "\n",
    "# 定义优化器\n",
    "optimizer = optim.SGD(embedding.parameters(), lr=0.1)\n",
    "\n",
    "# 清空之前的梯度\n",
    "optimizer.zero_grad()\n",
    "\n",
    "# 前向传播\n",
    "output = embedding(input_indices)\n",
    "\n",
    "# 计算损失\n",
    "loss = loss_fn(output, target)\n",
    "\n",
    "# 反向传播，注意此时不会更新权重\n",
    "loss.backward()\n",
    "\n",
    "# 查看梯度\n",
    "print(\"梯度：\")\n",
    "print(embedding.weight.grad)\n",
    "\n",
    "# 打印原权重矩阵\n",
    "print(\"原权重矩阵：\")\n",
    "print(embedding.weight.data)\n",
    "\n",
    "# 更新权重\n",
    "optimizer.step()\n",
    "\n",
    "# 打印权重矩阵更新后\n",
    "print(\"权重矩阵更新后:\")\n",
    "print(embedding.weight.data)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "498b10f1-8685-49a8-9954-52f78694c726",
   "metadata": {},
   "source": [
    "## 使用预训练的嵌入向量\n",
    "有时候，我们希望使用预训练的嵌入（如 GloVe、Word2Vec）来初始化嵌入层。这时候可以使用 `from_pretrained` 方法："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "9f11cb0b-927e-4de0-a82d-246402776b43",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "权重矩阵：\n",
      "tensor([[ 0.3367,  0.1288,  0.2345],\n",
      "        [ 0.2303, -1.1229, -0.1863],\n",
      "        [ 2.2082, -0.6380,  0.4617],\n",
      "        [ 0.2674,  0.5349,  0.8094],\n",
      "        [ 1.1103, -1.6898, -0.9890]])\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/tmp/ipykernel_194008/4236655584.py:8: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).\n",
      "  pretrained_embeddings = torch.tensor(torch.randn(5, 3))\n"
     ]
    }
   ],
   "source": [
    "import torch\n",
    "import torch.nn as nn\n",
    "\n",
    "# 设置随机种子以确保结果可复现\n",
    "torch.manual_seed(42)\n",
    "\n",
    "# 假设我们有一个预训练的嵌入矩阵，这里只是随机初始化\n",
    "pretrained_embeddings = torch.tensor(torch.randn(5, 3))\n",
    "\n",
    "# 使用 from_pretrained 方法创建嵌入层，不冻结权重层（默认冻结）\n",
    "embedding = nn.Embedding.from_pretrained(pretrained_embeddings, freeze=False)\n",
    "\n",
    "# 查看权重矩阵\n",
    "print(\"权重矩阵：\")\n",
    "print(embedding.weight.data)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "10a944cb-6fd2-4946-b72d-412278606b92",
   "metadata": {},
   "source": [
    "现在设置一下 `padding_idx`："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "0f5692e3-dfc1-40d4-916f-eee6b2061be3",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "权重矩阵：\n",
      "tensor([[ 0.3367,  0.1288,  0.2345],\n",
      "        [ 0.2303, -1.1229, -0.1863],\n",
      "        [ 2.2082, -0.6380,  0.4617],\n",
      "        [ 0.2674,  0.5349,  0.8094],\n",
      "        [ 1.1103, -1.6898, -0.9890]])\n"
     ]
    }
   ],
   "source": [
    "embedding = nn.Embedding.from_pretrained(pretrained_embeddings, freeze=False, padding_idx=0)\n",
    "\n",
    "# 查看权重矩阵\n",
    "print(\"权重矩阵：\")\n",
    "print(embedding.weight.data)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0af460b8-76f1-4225-ba7f-68c23b969e33",
   "metadata": {},
   "source": [
    "**注意**：虽然指定了 `padding_idx=0`，但预训练的嵌入矩阵第 0 行不会自动变为零向量。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a008de45-18a0-4d2b-a449-c53ecc0048c8",
   "metadata": {},
   "source": [
    "## 可视化"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8c556b20-2d6d-4ece-9cf6-9d85f7d39dc9",
   "metadata": {},
   "source": [
    "### 环境配置\n",
    "\n",
    "假设已经安装好了 PyTorch"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "c63fef98-6494-44f0-ad73-b6f34bfe4f5a",
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install transformers scikit-learn matplotlib seaborn"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2237de1e-6438-4214-8be6-bdb37e00b00b",
   "metadata": {},
   "source": [
    "### 完整代码"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "2ed89523-e69a-4e33-87e9-dd15f9ed7542",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA1sAAANXCAYAAADD/kV+AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8fJSN1AAAACXBIWXMAAA9hAAAPYQGoP6dpAABy7klEQVR4nO3deZzN5f//8efBrGZl9uyM3VhL9kFCEpJqki2UGiSyZddCJEpU9AmVlCJKH1t2ErKE0liyfTB2M8Y2Y+b9+8NvzrdjBjPM5RjzuN9u59ac63q/r/N6nzNH5znX+30dm2VZlgAAAAAAWSqXswsAAAAAgPsRYQsAAAAADCBsAQAAAIABhC0AAAAAMICwBQAAAAAGELYAAAAAwADCFgAAAAAYQNgCAAAAAAMIWwAAAABgAGELAP5l5cqVstlsWrlypbNLsStSpIgef/xx449z4MAB2Ww2TZ8+/ZbbduzYUUWKFHFos9lsGj58uJHa7qbp06fLZrPpwIED91wdkZGRioyMvKt1OOM9kZnfRQC4lxG2ANx1s2fPls1m0w8//JCmr2LFirLZbFqxYkWavkKFCqlmzZp3o8RbSv0gfKPbb7/95uwS8f898cQT8vT01Pnz52+4Tdu2beXq6qrTp0/fxcpwL/rrr780fPjwTIXtxMREffDBB6pcubJ8fHzk5+encuXK6cUXX9Tff/9t3y713w13d3cdOXIkzTiRkZEqX768Q1uRIkVu+O9MkyZNbvs4AdwdeZxdAICcp3bt2pKktWvXqlWrVvb2+Ph47dy5U3ny5NG6detUv359e9/hw4d1+PBhPfvss3e93psZOXKkihYtmqa9RIkSTqjGuS5duqQ8ee69/620bdtWP/30k3744Qe1b98+Tf/Fixc1f/58NWnSRPnz51e7du307LPPys3NzQnV3tySJUucXcJdUbhwYV26dEkuLi53/bH/+usvjRgxQpGRkWlmb2+kdevWWrhwoaKiotS1a1clJSXp77//1oIFC1SzZk2VLl3aYfsrV65o9OjRmjhxYobGr1Spkvr06ZOmPSwsLEP7A3Cee+//igDue2FhYSpatKjWrl3r0L5+/XpZlqU2bdqk6Uu9nxrUbpdlWbp8+bI8PDzuaJxUTZs2VbVq1bJkrOzO3d3d2SWk64knnpC3t7e+/vrrdMPW/PnzdeHCBbVt21aSlDt3buXOnftul5khrq6uzi7hrkid/ckONm3apAULFujtt9/WG2+84dD30Ucf6dy5c2n2qVSpkqZOnaqBAwdmKDA98MADev7557OqZAB3EacRAnCK2rVra+vWrbp06ZK9bd26dSpXrpyaNm2q3377TSkpKQ59NptNtWrVkiRdvXpVb775pooXLy43NzcVKVJEb7zxhq5cueLwOKnXOy1evFjVqlWTh4eHPv30U0nS//73P7Vs2VJ58+ZVUFCQXnvttTT736nUa0/ee+89TZo0ScWKFZOnp6ceffRRHT58WJZl6c0331SBAgXk4eGhFi1a6MyZM+mOtWTJElWqVEnu7u4qW7as5s6dm2abc+fOqVevXipYsKDc3NxUokQJvfvuuw7PZep2HTt2lK+vr/z8/NShQ4d0PxRK0rx581S+fHm5u7urfPny6Z7+KaW9Zmv48OGy2Wzau3evOnbsKD8/P/n6+qpTp066ePGiw76XLl1Sz549FRAQIG9vbz3xxBM6cuRImjHPnz+vXr16qUiRInJzc1NQUJAaNWqkLVu2pFuTJHl4eOjJJ5/UsmXLdOLEiTT9X3/9tf0xpfSvlfr999/VuHFjBQQEyMPDQ0WLFtULL7xg77/RdU3pXXu0fft2dezYUcWKFZO7u7tCQkL0wgsvZOgUxuuv2brZKWb/ruXIkSN64YUXFBwcLDc3N5UrV06ff/55mvHv5D2R3nV80v/9Hvzb0qVLVbt2bfn5+cnLy0ulSpVyCCrpPW8dO3aUl5eXjhw5opYtW8rLy0uBgYF6/fXXlZyc7DD+6dOn1a5dO/spfR06dNAff/xxy+vApk+frjZt2kiS6tevn+5zeb19+/ZJkv3fpn/LnTu38ufPn6b9jTfeUHJyskaPHn3DcQHcH5jZAuAUtWvX1pdffqkNGzbYPzyuW7dONWvWVM2aNRUXF6edO3cqIiLC3le6dGn7B5cuXbpoxowZeuqpp9SnTx9t2LBBo0aN0q5du9KEgZiYGEVFRemll15S165dVapUKV26dEkNGzbUoUOH1LNnT4WFhenLL7/U8uXLM3UccXFxOnXqlEObzWZL8wFr5syZSkxMVI8ePXTmzBmNGTNGTz/9tBo0aKCVK1eqf//+2rt3ryZOnKjXX389zQfhPXv26JlnnlG3bt3UoUMHTZs2TW3atNGiRYvUqFEjSddOh6tXr56OHDmil156SYUKFdKvv/6qgQMH6tixY5owYYKka7N7LVq00Nq1a9WtWzeVKVNGP/zwgzp06JDm+JYsWaLWrVurbNmyGjVqlE6fPq1OnTqpQIECGX6Onn76aRUtWlSjRo3Sli1b9NlnnykoKEjvvvuufZuOHTtq9uzZateunR5++GGtWrVKzZo1SzNWt27d9P3336t79+4qW7asTp8+rbVr12rXrl2qUqXKDWto27atZsyYodmzZ6t79+729jNnzmjx4sWKioq64WzniRMn9OijjyowMFADBgyQn5+fDhw4kG7YzYilS5fqn3/+UadOnRQSEqI///xTU6ZM0Z9//qnffvstTTC5mQkTJighIcGhbfz48dq2bZv9d/D48eN6+OGHZbPZ1L17dwUGBmrhwoXq3Lmz4uPj1atXL0nKsvfErfz55596/PHHFRERoZEjR8rNzU179+7VunXrbrlvcnKyGjdurOrVq+u9997TL7/8onHjxql48eJ6+eWXJUkpKSlq3ry5Nm7cqJdfflmlS5fW/Pnz0/39vl7dunXVs2dPffjhh3rjjTdUpkwZSbL/Nz2FCxeWdO09XqtWrQydSlu0aFG1b99eU6dO1YABA245u5WUlJTm3xlJyps3b5bN0gMwxAIAJ/jzzz8tSdabb75pWZZlJSUlWXnz5rVmzJhhWZZlBQcHW5MmTbIsy7Li4+Ot3LlzW127drUsy7K2bdtmSbK6dOniMObrr79uSbKWL19ubytcuLAlyVq0aJHDthMmTLAkWbNnz7a3XbhwwSpRooQlyVqxYsVN6582bZolKd2bm5ubfbv9+/dbkqzAwEDr3Llz9vaBAwdakqyKFStaSUlJ9vaoqCjL1dXVunz5cppjmDNnjr0tLi7OCg0NtSpXrmxve/PNN628efNau3fvdqh1wIABVu7cua1Dhw5ZlmVZ8+bNsyRZY8aMsW9z9epVq06dOpYka9q0afb2SpUqWaGhoQ61L1myxJJkFS5c2OFxJFnDhg2z3x82bJglyXrhhRcctmvVqpWVP39++/3NmzdbkqxevXo5bNexY8c0Y/r6+lrR0dFWZl29etUKDQ21atSo4dD+ySefWJKsxYsX29tSX9v9+/dblmVZP/zwgyXJ2rRp0w3HX7FiRbq/N6mv/7+f04sXL6bZf9asWZYka/Xq1Tesw7Isq169ela9evVuWMfs2bMtSdbIkSPtbZ07d7ZCQ0OtU6dOOWz77LPPWr6+vvZ67vQ90aFDhzS/E5b1f78HqcaPH29Jsk6ePHnDsdJ73jp06JDm2CzLsipXrmxVrVrVfn/OnDmWJGvChAn2tuTkZKtBgwZpxkzPd999l6HjTZWSkmLVq1fPkmQFBwdbUVFR1qRJk6yDBw+m2Tb1Nd20aZO1b98+K0+ePFbPnj3t/fXq1bPKlSvnsE/q+z+926hRozJUIwDn4TRCAE5RpkwZ5c+f334t1h9//KELFy7YVxusWbOm/S/d69evV3Jysv16rf/+97+SpN69ezuMmXoB+c8//+zQXrRoUTVu3Nih7b///a9CQ0P11FNP2ds8PT314osvZuo4Jk2apKVLlzrcFi5cmGa7Nm3ayNfX136/evXqkqTnn3/e4S/h1atXV2JiYpqVysLCwhwWE/Hx8VH79u21detWxcbGSpK+++471alTR/7+/jp16pT99sgjjyg5OVmrV6+2H3uePHnsMwHStdOdevTo4fCYx44d07Zt29ShQweH2hs1aqSyZctm+Dnq1q2bw/06dero9OnTio+PlyQtWrRIkvTKK684bHd9PZLk5+enDRs26OjRoxl+fOna8T377LNav369w+mBX3/9tYKDg9WwYcMb7uvn5ydJWrBggZKSkjL1uOn590zE5cuXderUKT388MOSdNPTIW/lr7/+0gsvvKAWLVpo8ODBkq7NYs6ZM0fNmzeXZVkOvxeNGzdWXFyc/TGz6j1xK6nP5/z589Oc3poR6f0+/fPPP/b7ixYtkouLi7p27Wpvy5Url6Kjo2+v4Fuw2WxavHix3nrrLfn7+2vWrFmKjo5W4cKF9cwzz9zw9NxixYqpXbt2mjJlio4dO3bTx6hevXqaf2eWLl2qqKgoA0cEICsRtgA4hc1mU82aNe3XZq1bt05BQUH2Vfz+HbZS/5satg4ePKhcuXKlWfEvJCREfn5+OnjwoEN7eqsFHjx4UCVKlEhzylapUqUydRwPPfSQHnnkEYfbv1dRTFWoUCGH+6nhpWDBgum2nz171qE9vVpLliwpSfbwsGfPHi1atEiBgYEOt0ceeUSS7NcrHTx4UKGhofLy8nIY7/pjT30ew8PD0xxPZp6n64/d39/f4RhTX8/rX6f0VnQcM2aMdu7cqYIFC+qhhx7S8OHDHT5o30zqAhhff/21pGvXJ61Zs0bPPvvsTRfEqFevnlq3bq0RI0YoICBALVq00LRp0277+r4zZ87o1VdfVXBwsDw8PBQYGGg/9ri4uNsaMz4+Xk8++aQeeOABffHFF/bflZMnT+rcuXOaMmVKmt+LTp06SXL8vciK98StPPPMM6pVq5a6dOmi4OBgPfvss5o9e3aGgpe7u7sCAwMd2vz9/R3eL6m/356eng7b3ekKoXFxcYqNjbXf/n1tpZubmwYNGqRdu3bp6NGjmjVrlh5++OE0p61eb/Dgwbp69eotr90KCAhI8+/MI488Yj+FEcC9i7AFwGlq166tuLg47dixw369VqqaNWvq4MGDOnLkiNauXauwsDAVK1bMYf+MXttyL1zTcKMP8zdqtywr04+RkpKiRo0apfsX8KVLl6p169aZHjMrZOUxPv300/rnn380ceJEhYWFaezYsSpXrly6s4nXq1q1qkqXLq1Zs2ZJkmbNmiXLsuwh7EZsNpu+//57rV+/Xt27d7cvNlG1alX79VI3+l28fuGG1GOYOnWqunXrprlz52rJkiX22b3bmemRrl3zdvToUc2bN08+Pj729tTxnn/++Rv+XqS3sMPtyOhz4OHhodWrV+uXX35Ru3bttH37dj3zzDNq1KhRus/XvzlzlchXX31VoaGh9tuTTz6Z7nahoaF69tlntXr1aoWHh2v27Nm6evVqutsWK1ZMzz//fIZmtwBkTyyQAcBp/v19W+vWrbNfqC9d+2Ds5uamlStXasOGDXrsscfsfYULF1ZKSor27NnjcOH68ePHde7cuQz9tbdw4cLauXOnLMty+JAYExOTBUeW9fbu3Zum1t27d0uSfQW44sWLKyEhwT6TdSOFCxfWsmXLlJCQ4DC7df2xpz6Pe/bsSTNGVj5Pqa/n/v37HWbR9u7dm+72oaGheuWVV/TKK6/oxIkTqlKlit5++201bdr0lo/Vtm1bDRkyRNu3b9fXX3+t8PBwPfjggxmq8+GHH9bDDz+st99+W19//bXatm2rb775Rl26dLHP1l1/ytj1s6xnz57VsmXLNGLECA0dOtTent5znFGjR4/WvHnzNHfu3DTf5xQYGChvb28lJydn6PfiTt4T/v7+6Z4yd/1zIF07ra9hw4Zq2LCh3n//fb3zzjsaNGiQVqxYccs6b6Vw4cJasWKFLl686DC7daPfp+vdKDT269fPYfn11Nf8RlxcXBQREaE9e/bo1KlTCgkJSXe7wYMH66uvvnJYMAbA/YOZLQBOU61aNbm7u2vmzJk6cuSIw8yWm5ubqlSpokmTJunChQsO36+VGrxSV9dL9f7770tSuqvYXe+xxx7T0aNH9f3339vbLl68qClTptzJIRlz9OhRh1UW4+Pj9cUXX6hSpUr2D3FPP/201q9fr8WLF6fZ/9y5c/a/rj/22GO6evWqPv74Y3t/cnJymi9YDQ0NVaVKlTRjxgyH09uWLl2qv/76K8uOLfV6usmTJzu0X19PcnJymtPsgoKCFBYWluFT+lJnsYYOHapt27bdclZLuhaQrp+Fq1SpkiTZH7dw4cLKnTu3/bq4VNcfU+rMzPXjXf+7nFG//PKLBg8erEGDBqlly5Zp+nPnzq3WrVtrzpw52rlzZ5r+kydP2n++0/dE8eLFFRcXp+3bt9vbjh07lmZ10PS+2uD65/NONG7cWElJSZo6daq9LSUlRZMmTcrQ/nnz5pWUNjiXLVvW4RS+qlWrSroWlA8dOpRmnHPnzmn9+vXy9/dPc+rjvxUvXlzPP/+8Pv30U/v1lwDuH8xsAXAaV1dXPfjgg1qzZo3c3NzsH15S1axZU+PGjZPk+GXGFStWVIcOHTRlyhSdO3dO9erV08aNGzVjxgy1bNky3Wumrte1a1d99NFHat++vTZv3qzQ0FB9+eWXaa7zuJWFCxfq77//TtNes2bNNKc93omSJUuqc+fO2rRpk4KDg/X555/r+PHjmjZtmn2bvn376scff9Tjjz+ujh07qmrVqrpw4YJ27Nih77//XgcOHFBAQICaN2+uWrVqacCAATpw4ID9O7vSu15o1KhRatasmWrXrq0XXnhBZ86c0cSJE1WuXLk0S47frqpVq6p169aaMGGCTp8+bV/6PXXmLnWm4fz58ypQoICeeuopVaxYUV5eXvrll1+0adMm++/JrRQtWlQ1a9bU/PnzJSlDYWvGjBmaPHmyWrVqpeLFi+v8+fOaOnWqfHx87MHf19dXbdq00cSJE2Wz2VS8eHEtWLAgzfd6+fj4qG7duhozZoySkpL0wAMPaMmSJdq/f3+Gn69/i4qKUmBgoMLDw/XVV1859DVq1EjBwcEaPXq0VqxYoerVq6tr164qW7aszpw5oy1btuiXX36xh587fU88++yz6t+/v1q1aqWePXvq4sWL+vjjj1WyZEmHhT9Gjhyp1atXq1mzZipcuLBOnDihyZMnq0CBAnf8peWS1LJlSz300EPq06eP9u7dq9KlS+vHH3+0H+etTj+uVKmScufOrXfffVdxcXFyc3NTgwYNFBQUlO72f/zxh5577jk1bdpUderUUb58+XTkyBHNmDFDR48e1YQJE255+uOgQYP05ZdfKiYmRuXKlUvTf+TIkTSvryR5eXmlG7IB3EOctQwiAFjW/y2BXrNmzTR9c+fOtSRZ3t7e1tWrVx36kpKSrBEjRlhFixa1XFxcrIIFC1oDBw50WDLdsq4tm9ysWbN0H/vgwYPWE088YXl6eloBAQHWq6++ai1atOiOl37Xv5aXTl3CeuzYsQ77py4V/t1336U77r+XGU89hsWLF1sRERGWm5ubVbp06TT7WpZlnT9/3ho4cKBVokQJy9XV1QoICLBq1qxpvffee1ZiYqJ9u9OnT1vt2rWzfHx8LF9fX6tdu3bW1q1b010ae86cOVaZMmUsNzc3q2zZstbcuXPTXeZbN1j6/folvtNb0vzChQtWdHS0lS9fPsvLy8tq2bKlFRMTY0myRo8ebVmWZV25csXq27evVbFiRcvb29vKmzevVbFiRWvy5MnpvkY3MmnSJEuS9dBDD6Xbf319W7ZssaKioqxChQpZbm5uVlBQkPX4449bv//+u8N+J0+etFq3bm15enpa/v7+1ksvvWTt3LkzzXP6v//9z2rVqpXl5+dn+fr6Wm3atLGOHj2a5vnLyNLvN/sd/Pfv8PHjx63o6GirYMGClouLixUSEmI1bNjQmjJlisMx3Ml7wrKufS1A+fLlLVdXV6tUqVLWV199lWbp92XLllktWrSwwsLCLFdXVyssLMyKiopy+MqCGy39njdv3jSPef34lnXttXjuuecsb29vy9fX1+rYsaO1bt06S5L1zTff3PI4pk6dahUrVszKnTv3LY/9+PHj1ujRo6169epZoaGhVp48eSx/f3+rQYMG1vfff++wbXrv738fn6RMLf2e3lL7AO4tNsu6jSuUAQAwbNu2bapcubK++uqrDM1AATczb948tWrVSmvXrs2yRUEA4Fa4ZgsA4HSXLl1K0zZhwgTlypVLdevWdUJFyM6u/31KvSbRx8dHVapUcVJVAHIirtkCADjdmDFjtHnzZtWvX1958uTRwoULtXDhQr344otpvosMuJUePXro0qVLqlGjhq5cuaK5c+fq119/1TvvvHNPfBUEgJyD0wgBAE63dOlSjRgxQn/99ZcSEhJUqFAhtWvXToMGDVKePPxdEJnz9ddfa9y4cdq7d68uX76sEiVK6OWXX77pFwwDgAmELQAAAAAwgGu2AAAAAMAAwhYAAAAAGMCJ8NdJSUnR0aNH5e3tfcsvPgQAAABw/7IsS+fPn1dYWJhy5cr8PBVh6zpHjx5l5SsAAAAAdocPH1aBAgUyvR9h6zre3t6Srj2hPj4+Tq4GAAAAgLPEx8erYMGC9oyQWYSt66SeOujj40PYAgAAAHDblxexQAYAAAAAGEDYAgAAAAADCFsAAAAAYABhCwAAAAAMIGwBAAAAgAGELQAAAAAwgLAFAAAAAAYQtgAAAADAAMIWAAAAABhA2AIAAAAAAwhbAAAAAGAAYQsAAAAADCBsAQAAAIABhC0AAAAAMICwBQAAAAAGELYAAAAAwADCFgAAAAAYQNgCAAAAAAMIWwAAAABgAGELAAAAAAwgbAEAAACAAYQtAAAAADCAsAUA0IULF9S+fXt5eXkpNDRU48aNU2RkpHr16iVJstlsmjdvnsM+fn5+mj59uv3+4cOH9fTTT8vPz0/58uVTixYtdODAAYd9PvvsM5UpU0bu7u4qXbq0Jk+ebO87cOCAbDab5s6dq/r168vT01MVK1bU+vXrDR01AABmEbYAAOrbt69WrVql+fPna8mSJVq5cqW2bNmS4f2TkpLUuHFjeXt7a82aNVq3bp28vLzUpEkTJSYmSpJmzpypoUOH6u2339auXbv0zjvvaMiQIZoxY4bDWIMGDdLrr7+ubdu2qWTJkoqKitLVq1ez9HgBALgb8ji7AACAcyUkJOg///mPvvrqKzVs2FCSNGPGDBUoUCDDY3z77bdKSUnRZ599JpvNJkmaNm2a/Pz8tHLlSj366KMaNmyYxo0bpyeffFKSVLRoUf3111/69NNP1aFDB/tYr7/+upo1ayZJGjFihMqVK6e9e/eqdOnSWXXIAADcFYQtAMjh9u3bp8TERFWvXt3eli9fPpUqVSrDY/zxxx/au3evvL29HdovX76sffv26cKFC9q3b586d+6srl272vuvXr0qX19fh30iIiLsP4eGhkqSTpw4QdgCAGQ7hC0AwC3ZbDZZluXQlpSUZP85ISFBVatW1cyZM9PsGxgYqISEBEnS1KlTHUKdJOXOndvhvouLi8PjSlJKSsqdHQAAAE5A2AKAHK548eJycXHRhg0bVKhQIUnS2bNntXv3btWrV0/StcB07Ngx+z579uzRxYsX7ferVKmib7/9VkFBQfLx8UnzGL6+vgoLC9M///yjtm3bGj4iAADuDYQtAMhh4i4m6lRCouIvJ8nHw0UBeV3VuXNn9e3bV/nz51dQUJAGDRqkXLn+bw2lBg0a6KOPPlKNGjWUnJys/v37O8xAtW3bVmPHjlWLFi00cuRIFShQQAcPHtTcuXPVr18/FShQQCNGjFDPnj3l6+urJk2a6MqVK/r999919uxZ9e7d2xlPBQAARhG2ACAHOXrukvrP2a41e07Z2+qGB2jIoJFKSEhQ8+bN5e3trT59+iguLs6+zbhx49SpUyfVqVNHYWFh+uCDD7R582Z7v6enp1avXq3+/fvrySef1Pnz5/XAAw+oYcOG9pmuLl26yNPTU2PHjlXfvn2VN29eVahQwb68PAAA9xubdf1J+DlcfHy8fH19FRcXl+6pMACQXcVdTFT3WVsdglaquuEBmhhVWb6erva2yMhIVapUSRMmTLiLVQIAcO+402zA92wBQA5xKiEx3aAlSav3nNKphMS7XBEAAPc3whYA5BDxl5Nu2n/+Fv0AACBzuGYLAHIIH3eXm/Z7X9e/cuVKg9UAAHD/Y2YLAHKIAC9X1Q0PSLevbniAArxc0+0DAAC3h7AFADmEr6erRreOSBO46oYH6N3WEQ6LYwAAgDvHaYQAkIOE+XloYlRlnUpI1PnLSfJ2d1GAlytBCwAAAwhbAJDD+HoSrgAAuBs4jRAAAAAADCBsAQAAAIABhC0A96XIyEj16tUr3b6OHTuqZcuWd7UeAACQ83DNFoAc54MPPpBlWc4uAwAA3OcIWwByHF9fX2eXAAAAcgBOIwSQI/z888/y9fXVzJkz05xGGBkZqZ49e6pfv37Kly+fQkJCNHz4cIf9//77b9WuXVvu7u4qW7asfvnlF9lsNs2bN++uHgcAAMg+CFsA7ntff/21oqKiNHPmTLVt2zbdbWbMmKG8efNqw4YNGjNmjEaOHKmlS5dKkpKTk9WyZUt5enpqw4YNmjJligYNGnQ3DwEAAGRDnEYI4L42adIkDRo0SD/99JPq1at3w+0iIiI0bNgwSVJ4eLg++ugjLVu2TI0aNdLSpUu1b98+rVy5UiEhIZKkt99+W40aNborxwAAALInwhaA+9b333+vEydOaN26dXrwwQdvum1ERITD/dDQUJ04cUKSFBMTo4IFC9qDliQ99NBDWV8wAAC4r3AaIYD7VuXKlRUYGKjPP//8lqsPuri4ONy32WxKSUkxWR4AALjPEbYA3LeKFy+uFStWaP78+erRo8dtj1OqVCkdPnxYx48ft7dt2rQpK0oEAAD3MU4jBJDtxV1M1KmERMVfTpKPh4sC8rra+0qWLKkVK1YoMjJSefLk0YQJEzI9fqNGjVS8eHF16NBBY8aM0fnz5zV48GBJ12bAAAAA0kPYApCtHT13Sf3nbNeaPafsbXXDA5R49f9OASxVqpSWL1+uyMhI5c6dO9OPkTt3bs2bN09dunTRgw8+qGLFimns2LFq3ry53N3ds+Q4AADA/cdm3epChhwmPj5evr6+iouLk4+Pj7PLAXATcRcT1X3WVoeglapueIAmRlWWr6drOnveuXXr1ql27drau3evihcvbuQxAACAc91pNmBmC0C2dSohMd2gJUmr95zSqYTELAtbP/zwg7y8vBQeHq69e/fq1VdfVa1atQhaAADghghbALKt+MtJN+0/f4v+zDh//rz69++vQ4cOKSAgQI888ojGjRuXZeMDAID7D2ELQLbl4+5y037vW/RnRvv27dW+ffssGw8AANz/WPodQLYV4OWquuEB6fbVDQ9QgJeZ67UAAAAygrAFINvy9XTV6NYRaQJX3fAAvds6wtjiGAAAABnBaYQAsrUwPw9NjKqsUwmJOn85Sd7uLgrwciVoAQAApyNsAcj2fD0JVwAA4N7DaYQAAAAAYABhCwAAAAAMIGwBAAAAgAGELQAAAAAwgLAFAAAAAAYQtgAAAADAAMIWAAAAABhA2AIAAAAAAwhbAAAAAGAAYQsAAAAADCBsAQAAAIABhC0AAAAAMICwBQAAAAAGELYAAAAAwADCFgAAAAAYQNgCAAAAAAMIWwAAAABgAGELAAAAAAwgbAEAAACAAYQtAAAAADCAsAUAAAAABhC2AAAAAMAAwhYAAAAAGEDYAgAAAAADsm3YGj16tGw2m3r16mVvu3z5sqKjo5U/f355eXmpdevWOn78uPOKBAAAAJBjZcuwtWnTJn366aeKiIhwaH/ttdf0008/6bvvvtOqVat09OhRPfnkk06qEgAAAEBOlu3CVkJCgtq2baupU6fK39/f3h4XF6f//Oc/ev/999WgQQNVrVpV06ZN06+//qrffvvNiRUDAAAAyImyXdiKjo5Ws2bN9Mgjjzi0b968WUlJSQ7tpUuXVqFChbR+/fobjnflyhXFx8c73AAAAADgTuVxdgGZ8c0332jLli3atGlTmr7Y2Fi5urrKz8/PoT04OFixsbE3HHPUqFEaMWJEVpcKAAAAIIfLNjNbhw8f1quvvqqZM2fK3d09y8YdOHCg4uLi7LfDhw9n2dgAAAAAcq5sE7Y2b96sEydOqEqVKsqTJ4/y5MmjVatW6cMPP1SePHkUHBysxMREnTt3zmG/48ePKyQk5Ibjurm5ycfHx+EGAAAAAHcq25xG2LBhQ+3YscOhrVOnTipdurT69++vggULysXFRcuWLVPr1q0lSTExMTp06JBq1KjhjJIBAAAA5GDZJmx5e3urfPnyDm158+ZV/vz57e2dO3dW7969lS9fPvn4+KhHjx6qUaOGHn74YWeUDAAAACAHyzZhKyPGjx+vXLlyqXXr1rpy5YoaN26syZMnO7ssAAAAADmQzbIsy9lF3Evi4+Pl6+uruLg4rt8CAAAAcrA7zQbZZoEMAAAAAMhOCFsAAAAAYABhCwAAAAAMIGwBAAAAgAGELQAAAAAwgLAFAAAAAAYQtgAAAADAAMIWAAAAABhA2AIAAAAAAwhbAAAAAGAAYQsAAAAADCBsAQAAAIABhC0AAAAAMICwBQAAAAAGELYAAAAAwADCFgAAAAAYQNgCAAAAAAMIWwAAAABgAGELAAAAAAwgbAEAAACAAYQtAAAAADCAsAUAAAAABhC2AAAAAMAAwhYAAAAAGEDYAgAAAAADCFsAAAAAYABhCwAAAAAMIGwBAAAAgAGELQAAAAAwgLAFAAAAAAYQtgAAAADAAMIWAAAAABhA2AIAAAAAAwhbAAAAAGAAYQsAAAAADCBsAQAAAIABhC0AAAAAMICwBQAAAAAGELYAAAAAwADCFgAAAAAYQNgCAAAAAAMIWwAAAABgAGELAAAAAAwgbAEAAACAAYQtAAAAADCAsAUAAAAABhC2AAAAAMAAwhYAAAAAGEDYAgAAAAADCFsAAAAAYABhCwAAAAAMIGwBAAAAgAGELQAAAAAwgLAFAAAAAAYQtgAAAADAAMIWAAAAABhA2AIAAAAAAwhbAAAAAGAAYQsAAAAADCBsAQAAAIABhC0AAAAAMICwBQAAAAAGELYAAAAAwADCFgAAAAAYQNgCAAAAAAMIWwAAAABgAGELAAAAAAwgbAEAAACAAYQtAAAAADCAsAUAAAAABhC2AAAAAMAAwhYAAAAAGEDYAgAAAAADCFsAAAAAYABhCwAAAAAMIGwBAAAAgAGELQAAAAAwgLAFAAAAAAYQtgAAAADAAMIWAAAAABhA2AIAAAAAAwhbAAAAAGAAYQsAAAAADCBsAQAAAIABhC0AAAAAMICwBQAAAAAGELYAAAAAwADCFgAAAAAYQNgCAAAAAAMIWwAAAABgAGELAAAAAAwgbAEAAACAAYQtAAAAADCAsAUAAAAABhC2AAAAAMAAwhYAAAAAGJBtwtaoUaP04IMPytvbW0FBQWrZsqViYmIctrl8+bKio6OVP39+eXl5qXXr1jp+/LiTKgYAAACQk2WbsLVq1SpFR0frt99+09KlS5WUlKRHH31UFy5csG/z2muv6aefftJ3332nVatW6ejRo3ryySedWDUAAACAnMpmWZbl7CJux8mTJxUUFKRVq1apbt26iouLU2BgoL7++ms99dRTkqS///5bZcqU0fr16/Xwww9naNz4+Hj5+voqLi5OPj4+Jg8BAAAAwD3sTrNBtpnZul5cXJwkKV++fJKkzZs3KykpSY888oh9m9KlS6tQoUJav379Dce5cuWK4uPjHW4AAAAAcKeyZdhKSUlRr169VKtWLZUvX16SFBsbK1dXV/n5+TlsGxwcrNjY2BuONWrUKPn6+tpvBQsWNFk6AAAAgBwiW4at6Oho7dy5U998880djzVw4EDFxcXZb4cPH86CCgEAAADkdHmcXUBmde/eXQsWLNDq1atVoEABe3tISIgSExN17tw5h9mt48ePKyQk5Ibjubm5yc3NzWTJAAAAAHKgbDOzZVmWunfvrh9++EHLly9X0aJFHfqrVq0qFxcXLVu2zN4WExOjQ4cOqUaNGne7XAAAAAA5XLaZ2YqOjtbXX3+t+fPny9vb234dlq+vrzw8POTr66vOnTurd+/eypcvn3x8fNSjRw/VqFEjwysRAgAAAEBWyTZLv9tstnTbp02bpo4dO0q69qXGffr00axZs3TlyhU1btxYkydPvulphNdj6XcAAAAA0p1ng2wTtu4WwhYAAAAAKQd/zxYAAAAA3MsIWwAAAABytOnTp6f5vt6sQNgCAAAAAAMIWwAAAABgAGELAAAAwD1r0aJFql27tvz8/JQ/f349/vjj2rdvnyTpwIEDstls+uabb1SzZk25u7urfPnyWrVqlX3/lStXymaz6eeff1ZERITc3d318MMPa+fOnTd93Pnz56tOnTqSpIiICI0YMUJXr17NVO2ELQAAAAD3rAsXLqh37976/ffftWzZMuXKlUutWrVSSkqKfZu+ffuqT58+2rp1q2rUqKHmzZvr9OnTDuP07dtX48aN06ZNmxQYGKjmzZsrKSkp3cdcs2aN2rdvr5dfflmSNGHCBE2fPl1vv/12pmpn6ffrsPQ7AAAAcO86deqUAgMDtWPHDnl5ealo0aIaPXq0+vfvL0m6evWqihYtqh49eqhfv35auXKl6tevr2+++UbPPPOMJOnMmTMqUKCApk+frqefflrTp09Xr169dO7cOUnSI488ooYNGyo6OtqeDX788Uf169dPR48ezXCtzGwBAAAAuGft2bNHUVFRKlasmHx8fFSkSBFJ0qFDh+zb1KhRw/5znjx5VK1aNe3atcthnH9vky9fPpUqVSrNNqn++OMPjRw5UmFhYZKksLAwde3aVceOHdPFixczXHueDG8JAAAAAHdZ8+bNVbhwYU2dOlVhYWFKSUlR+fLllZiYaOwxExISNGLECDVq1EhVqlTRmjVr5O3tLUlyd3fP8DjMbAEAAAC4J50+fVoxMTEaPHiwGjZsqDJlyujs2bNptvvtt9/sP1+9elWbN29WmTJlbrjN2bNntXv37jTbpKpSpYpiYmJUvHhxSVLx4sVVokQJlShRQrlyZTxCMbMFAAAAwOniLibqVEKi4i8nycfDRQF5XeXv76/8+fNrypQpCg0N1aFDhzRgwIA0+06aNEnh4eEqU6aMxo8fr7Nnz+qFF15w2GbkyJHKnz+/goODNWjQIAUEBKhly5bp1jJ06FA9/vjjCg4OliTFxMRo37592rlzp956660MHxNhCwAAAIBTHT13Sf3nbNeaPafsbXXDAzS6dYS++eYb9ezZU+XLl1epUqX04YcfKjIy0mH/0aNHa/To0dq2bZtKlCihH3/8UQEBAWm2efXVV7Vnzx5VqlRJP/30k1xdXdOtp3HjxlqwYIGGDRsm6dqCGaVLl1aXLl0ydVysRngdViMEAAAA7p64i4nqPmurQ9BKVTc8QBOjKsvXM/1QdODAARUtWlRbt25VpUqV0t0mdTXCs2fPys/PL1O13Wk24JotAAAAAE5zKiEx3aAlSav3nNKpBHMLYZhG2AIAAADgNPGX0/9i4VTnb9F/L+OaLQAAAABO4+PuctN+75v0FylSRLe6KioyMvKW25jCzBYAAAAApwnwclXd8IB0++qGByjAK/3rtbIDwhYAAAAAp/H1dNXo1hFpAlfd8AC92zrihotjZAecRggAAADAqcL8PDQxqrJOJSTq/OUkebu7KMDLNVsHLYmwBQAAAOAe4OuZ/cPV9TiNEAAAAAAMIGwBAAAAgAGELQAAAAAwgLAFAAAAAAYQtgAAAADAAMIWAAAAABhA2AIAAAAAAwhbAAAAAGAAYQsAAAAADCBsAQAAAIABhC0AAAAAMICwBQAAAAAGELYAAAAAwADCFgAAAAAYQNgCAAAAAAMIWwAAAABgAGELAAAAAAwgbAEAAACAAYQtAAAAADCAsAUAAAAABhC2AAAAAMAAwhYAAAAAGEDYAgAAAAADCFsAAAAAYABhCwAAAAAMIGwBAAAAgAGELQAAAAAwgLAFAAAAAAYQtgAAAADAAMIWAAAAABhA2AIAAAAAAwhbAAAAAGAAYQsAAAAADCBsAQAAAIABhC0AAAAAMICwBQAAAAAGELYAAAAAwADCFgAAAAAYQNgCAAAAAAMIWwAAAABgAGELAAAAAAwgbAEAAACAAYQtAAAAADCAsAUAAAAABhC2AAAAAMAAwhYAAAAAGEDYAgAAAAADCFsAAAAAYABhCwAAAAAMIGwBAAAAgAGELQAAAAAwgLAFAAAAAAYQtgAAAADAAMIWAAAAABhA2AIAAAAAAwhbAAAAAGAAYQsAAAAADCBsAQAAAIABhC0AAAAAMICwBQAAAAAGELYAAAAAwADCFgAAAAAYQNgCAAAAAAMIWwAAAABgAGELAAAAAAwgbAEAAACAAYQtAAAAADCAsAUAAAAABhC2AAAAAMAAwhYAAAAAGEDYAgAAAHBHLMvS1atXnV3GPYewBQAAACCNK1euqGfPngoKCpK7u7tq166tTZs2SZJWrlwpm82mhQsXqmrVqnJzc9PatWu1b98+tWjRQsHBwfLy8tKDDz6oX375xWHcIkWK6J133tELL7wgb29vFSpUSFOmTHHY5tdff1WlSpXk7u6uatWqad68ebLZbNq2bZt9m507d6pp06by8vJScHCw2rVrp1OnThl/XjKDsAUAAAAgjX79+mnOnDmaMWOGtmzZohIlSqhx48Y6c+aMfZsBAwZo9OjR2rVrlyIiIpSQkKDHHntMy5Yt09atW9WkSRM1b95chw4dchh73LhxqlatmrZu3apXXnlFL7/8smJiYiRJ8fHxat68uSpUqKAtW7bozTffVP/+/R32P3funBo0aKDKlSvr999/16JFi3T8+HE9/fTT5p+YTLBZlmU5u4h7SXx8vHx9fRUXFycfHx9nlwMAAADcdRcuXJC/v7+mT5+u5557TpKUlJSkIkWKqFevXnrwwQdVv359zZs3Ty1atLjpWOXLl1e3bt3UvXt3SddmturUqaMvv/xS0rVTEENCQjRixAh169ZNn3zyiQYPHqz//e9/cnd3lyR99tln6tq1q7Zu3apKlSrprbfe0po1a7R48WL74/zvf/9TwYIFFRMTo5IlS2bJ83Cn2YCZLQAAAAAO9u3bp6SkJNWqVcve5uLiooceeki7du2yt1WrVs1hv4SEBL3++usqU6aM/Pz85OXlpV27dqWZ2YqIiLD/bLPZFBISohMnTkiSYmJiFBERYQ9akvTQQw857P/HH39oxYoV8vLyst9Kly5tr/1ekcfZBQAAAADInvLmzetw//XXX9fSpUv13nvvqUSJEvLw8NBTTz2lxMREh+1cXFwc7ttsNqWkpGT4cRMSEtS8eXO9++67afpCQ0MzcQRm3ZczW5MmTVKRIkXk7u6u6tWra+PGjc4uCQAAAMg2ihcvLldXV61bt87elpSUpE2bNqls2bI33G/dunXq2LGjWrVqpQoVKigkJEQHDhzI1GOXKlVKO3bs0JUrV+xtqQtzpKpSpYr+/PNPFSlSRCVKlHC4XR8Anem+C1vffvutevfurWHDhmnLli2qWLGiGjdubJ+WBAAAAPB/4i4mat+JBG09dFb7TiYo7mKi8ubNq5dffll9+/bVokWL9Ndff6lr1666ePGiOnfufMOxwsPDNXfuXG3btk1//PGHnnvuuUzNWEmy7/Piiy9q165dWrx4sd577z1J12bAJCk6OlpnzpxRVFSUNm3apH379mnx4sXq1KmTkpOTb//JyGL3Xdh6//331bVrV3Xq1Elly5bVJ598Ik9PT33++efOLg0AAAC4pxw9d0ndZ21Vw/dXqdXkX9Vw3Cr1mLVVR89d0ujRo9W6dWu1a9dOVapU0d69e7V48WL5+/vfcLz3339f/v7+qlmzppo3b67GjRurSpUqmarJx8dHP/30k7Zt26ZKlSpp0KBBGjp0qCTZr+MKCwvTunXrlJycrEcffVQVKlRQr1695Ofnp1y57p2Ic1+tRpiYmChPT099//33atmypb29Q4cOOnfunObPn59mnytXrjhMUcbHx6tgwYKsRggAAID7WtzFRHWftVVr9qT9bqq64QGaGFVZvp6uTqgsrZkzZ6pTp06Ki4uTh4fHXXtcViP8l1OnTik5OVnBwcEO7cHBwYqNjU13n1GjRsnX19d+K1iw4N0oFQAAAHCqUwmJ6QYtSVq955ROJSSm23c3fPHFF1q7dq3279+vefPmqX///nr66afvatDKCvdV2LodAwcOVFxcnP12+PBhZ5cEAAAAGBd/Oemm/edv0W9SbGysnn/+eZUpU0avvfaa2rRpoylTpjitntt1Xy39HhAQoNy5c+v48eMO7cePH1dISEi6+7i5ucnNze1ulAcAAADcM3zcXW7a732LfpP69eunfv36Oe3xs8p9NbPl6uqqqlWratmyZfa2lJQULVu2TDVq1HBiZQAAAMC9JcDLVXXDA9LtqxseoACve+N6rezsvgpbktS7d29NnTpVM2bM0K5du/Tyyy/rwoUL6tSpk7NLAwAAAO4Zvp6uGt06Ik3gqhseoHdbR9wzi2NkZ/fVaYSS9Mwzz+jkyZMaOnSoYmNjValSJS1atCjNohkAAABAThfm56GJUZV1KiFR5y8nydvdRQFergStLHJfLf2eFe50eUcAAAAA9weWfgcAAACAexBhCwAAAAAMIGwBAAAAgAGELQAAAAAwgLAFAAAAAAYQtgAAAADAAMIWAAAAABhA2AIAAAAAAwhbAAAAAGAAYQsAAAAADCBsAQAAAIABhC0AAAAAMICwBQAAAAAGELYAAAAAwADCFgAAAAAYQNgCAAAAAAMIWwAAAABgAGELAAAAAAwgbAEAAACAAYQtAAAAADCAsAUAAAAABhC2AAAAAMAAwhYAAAAAGEDYAgAAAAADCFsAAAAAYABhCwAAAAAMIGwBAAAAgAGELQAAAAAwgLAFAAAAAAYQtgAAAADAAMIWAAAAABhA2AIAAAAAAwhbAAAAAGBApsLWpUuXtHbtWv31119p+i5fvqwvvvgiywoDAAAAgOwsw2Fr9+7dKlOmjOrWrasKFSqoXr16OnbsmL0/Li5OnTp1MlIkAAAAAGQ3GQ5b/fv3V/ny5XXixAnFxMTI29tbtWrV0qFDh0zWBwAAAADZUobD1q+//qpRo0YpICBAJUqU0E8//aTGjRurTp06+ueff0zWCAAAAADZTobD1qVLl5QnTx77fZvNpo8//ljNmzdXvXr1tHv3biMFAgAAAEB2lOfWm1xTunRp/f777ypTpoxD+0cffSRJeuKJJ7K2MgAAAADIxjI8s9WqVSvNmjUr3b6PPvpIUVFRsiwrywoDAAAAgOzMZpGQHMTHx8vX11dxcXHy8fFxdjkAAAAAnOROswFfagwAAAAABhC2AAAAAMAAwhYAAAAAGEDYAgAAAAADCFsAAAAAYECGv2fr3/bs2aMVK1boxIkTSklJcegbOnRolhQGAAAAANlZpsPW1KlT9fLLLysgIEAhISGy2Wz2PpvNRtgCAAAAAN1G2Hrrrbf09ttvq3///ibqAQAAAID7Qqav2Tp79qzatGljohYAAAAAuG9kOmy1adNGS5YsMVELAAAAANw3Mn0aYYkSJTRkyBD99ttvqlChglxcXBz6e/bsmWXFAQAAAEB2ZbMsy8rMDkWLFr3xYDab/vnnnzsuypni4+Pl6+uruLg4+fj4OLscAAAAAE5yp9kg0zNb+/fvz/SDAAAAAEBOc0dfamxZljI5MQYAAAAAOcJtha0vvvhCFSpUkIeHhzw8PBQREaEvv/wyq2sDAAAAgGwr06cRvv/++xoyZIi6d++uWrVqSZLWrl2rbt266dSpU3rttdeyvEgAAAAAyG5ua4GMESNGqH379g7tM2bM0PDhw7P9NV0skAEAAABAuvNskOnTCI8dO6aaNWumaa9Zs6aOHTuW6QIAAAAA4H6U6bBVokQJzZ49O037t99+q/Dw8CwpCgAAAACyu0xfszVixAg988wzWr16tf2arXXr1mnZsmXphjAAAAAAyIkyPbPVunVrbdiwQQEBAZo3b57mzZungIAAbdy4Ua1atTJRIwAAAABkO5leION+xwIZAAAAAKQ7zwYZOo0wPj7ePnh8fPxNtyWgAAAAAEAGw5a/v7+OHTumoKAg+fn5yWazpdnGsizZbDYlJydneZEAAAAAkN1kKGwtX75c+fLlkyStWLHCaEEAAAAAcD/gmq3rcM0WAAAAAMkJX2q8aNEirV271n5/0qRJqlSpkp577jmdPXs20wUAAAAAwP0o02Grb9++9kUyduzYod69e+uxxx7T/v371bt37ywvEAAAAACyo0x/qfH+/ftVtmxZSdKcOXPUvHlzvfPOO9qyZYsee+yxLC8QAAAAALKjTM9subq66uLFi5KkX375RY8++qgkKV++fLdcFh4AAAAAcopMz2zVrl1bvXv3Vq1atbRx40Z9++23kqTdu3erQIECWV4gAAAAAGRHmZ7Z+uijj5QnTx59//33+vjjj/XAAw9IkhYuXKgmTZpkeYEAAAAAkB2x9Pt1WPodAAAAgHTn2SDTpxFKUkpKivbu3asTJ04oJSXFoa9u3bq3MyQAAAAA3FcyHbZ+++03Pffcczp48KCunxSz2WxKTk7OsuIAAAAAILvKdNjq1q2bqlWrpp9//lmhoaGy2Wwm6gIAAACAbC3TYWvPnj36/vvvVaJECRP1AAAAAMB9IdOrEVavXl179+41UQsAAAAA3DcyPbPVo0cP9enTR7GxsapQoYJcXFwc+iMiIrKsOAAAAADIrjK99HuuXGknw2w2myzLui8WyGDpdwAAAACSE5Z+379/f6YfBAAAAABymkyHrcKFC5uoAwAAAADuK5leIEOSvvzyS9WqVUthYWE6ePCgJGnChAmaP39+lhYHAAAAANlVpsPWxx9/rN69e+uxxx7TuXPn7Ndo+fn5acKECVldHwAAAABkS5kOWxMnTtTUqVM1aNAg5c6d295erVo17dixI0uLAwAAAIDsKtNha//+/apcuXKadjc3N124cCFLigIAAACA7C7TYato0aLatm1bmvZFixapTJkyWVETAAAAAGR7mV6NsHfv3oqOjtbly5dlWZY2btyoWbNmadSoUfrss89M1AgAAAAA2U6mZ7a6dOmid999V4MHD9bFixf13HPP6eOPP9YHH3ygZ5991kSNAIAMWLBggfz8/OwLF23btk02m00DBgywb9OlSxc9//zzkqQ5c+aoXLlycnNzU5EiRTRu3DiH8YoUKaK33npL7du3l5eXlwoXLqwff/xRJ0+eVIsWLeTl5aWIiAj9/vvv9n1Onz6tqKgoPfDAA/L09FSFChU0a9Ysh3EjIyPVs2dP9evXT/ny5VNISIiGDx9u6FkBAMB5bmvp97Zt22rPnj1KSEhQbGys/ve//6lz585ZXRsAIBPq1Kmj8+fPa+vWrZKkVatWKSAgQCtXrrRvs2rVKkVGRmrz5s16+umn9eyzz2rHjh0aPny4hgwZounTpzuMOX78eNWqVUtbt25Vs2bN1K5dO7Vv317PP/+8tmzZouLFi6t9+/ayLEuSdPnyZVWtWlU///yzdu7cqRdffFHt2rXTxo0bHcadMWOG8ubNqw0bNmjMmDEaOXKkli5davT5AQDgbrNZqf+HhCQpPj5evr6+iouLk4+Pj7PLAYBMqVq1qqKiovT666+rVatWevDBBzVixAidPn1acXFxKlCggHbv3q3hw4fr5MmTWrJkiX3ffv366eeff9aff/4p6drMVp06dfTll19KkmJjYxUaGqohQ4Zo5MiRkqTffvtNNWrU0LFjxxQSEpJuTY8//rhKly6t9957T9K1ma3k5GStWbPGvs1DDz2kBg0aaPTo0UaeFwAAbsedZoNMz2ydPn1a0dHRKlu2rAICApQvXz6HGwDAeerVq6eVK1fKsiytWbNGTz75pMqUKaO1a9dq1apVCgsLU3h4uHbt2qVatWo57FurVi3t2bPHfhqiJEVERNh/Dg4OliRVqFAhTduJEyckScnJyXrzzTdVoUIF5cuXT15eXlq8eLEOHTrk8Fj/HleSQkND7WMAAHC/yPQCGe3atdPevXvVuXNnBQcHy2azmagLAHAbIiMj9fnnn+uPP/6Qi4uLSpcurcjISK1cuVJnz55VvXr1MjWei4uL/efUf+/Ta0tJSZEkjR07Vh988IEmTJigChUqKG/evOrVq5cSExNvOG7qOKljAABwv8h02FqzZo3Wrl2rihUrmqgHAHAHUq/bGj9+vD1YRUZGavTo0Tp79qz69OkjSSpTpozWrVvnsO+6detUsmRJhy+sz6x169apRYsW9kU4UlJStHv3bpUtW/a2xwQAILvK9GmEpUuX1qVLl0zUAgDIoLiLidp3IkFbD53VvpMJirt4bebI399fERERmjlzpiIjIyVJdevW1ZYtW7R79257AOvTp4+WLVumN998U7t379aMGTP00Ucf6fXXX7+jusLDw7V06VL9+uuv2rVrl1566SUdP378jsYEACC7ynTYmjx5sgYNGqRVq1bp9OnTio+Pd7iZcODAAXXu3FlFixaVh4eHihcvrmHDhqU5LWX79u2qU6eO3N3dVbBgQY0ZM8ZIPQDgTEfPXVL3WVvV8P1VajX5VzUct0o9Zm3V0XPX/hBWr149JScn28NWvnz5VLZsWYWEhKhUqVKSpCpVqmj27Nn65ptvVL58eQ0dOlQjR45Ux44d76i2wYMHq0qVKmrcuLEiIyMVEhKili1b3tGYAABkV5lejXDPnj167rnntGXLFod2y7Jks9kcLqzOKosWLdK3336rqKgolShRQjt37lTXrl3Vrl07++pW8fHxKlmypB555BENHDhQO3bs0AsvvKAJEyboxRdfzPBjsRohgHtZ3MVEdZ+1VWv2nErTVzc8QBOjKsvX09UJlQEAcP+502yQ6Wu22rZtKxcXF3399dd3bYGMJk2aqEmTJvb7xYoVU0xMjD7++GN72Jo5c6YSExP1+eefy9XVVeXKldO2bdv0/vvvZypsAcC97FRCYrpBS5JW7zmlUwmJhC0AAO4RmQ5bO3fu1NatW+2nojhLXFycw1Lz69evV926deXq+n8fMho3bqx3331XZ8+elb+/f7rjXLlyRVeuXLHfN3UqJABkhfjLSTftP3+LfgAAcPdk+pqtatWq6fDhwyZqybC9e/dq4sSJeumll+xtsbGx9u97SZV6PzY29oZjjRo1Sr6+vvZbwYIFzRQNAFnAx93lpv3et+gHAAB3T6bDVo8ePfTqq69q+vTp2rx5s7Zv3+5wy4wBAwbIZrPd9Pb333877HPkyBE1adJEbdq0UdeuXTNbfhoDBw5UXFyc/ebsIAkANxPg5aq64QHp9tUND1CAF6cQAgBwr8j0Ahm5cqXNZzab7bYWyDh58qROnz59022KFStmPzXw6NGjioyM1MMPP6zp06c71NK+fXvFx8dr3rx59rYVK1aoQYMGOnPmzA1PI7weC2QAuNcdPXdJA+Zs1+p/XbtVNzxA77aOUKifhxMrAwDg/nLXF8jYv39/ph/kRgIDAxUYGJihbY8cOaL69euratWqmjZtWprQV6NGDQ0aNEhJSUlycbl2Gs3SpUtVqlSpDActAMgOwvw8NDGqsk4lJOr85SR5u7sowMuVhTEAALjHZHpmyxmOHDmiyMhIFS5cWDNmzFDu3LntfSEhIZKuLZhRqlQpPfroo+rfv7927typF154QePHj2fpdwAAAACZdldmtn788Uc1bdpULi4u+vHHH2+67RNPPJHpIm5l6dKl2rt3r/bu3asCBQo49KVmRV9fXy1ZskTR0dGqWrWqAgICNHToUJZ9BwAAAOAUGZrZypUrl2JjYxUUFJTuNVv2wQx9qfHdxMwWAAAAAOnOs0GGViNMSUlRUFCQ/ecb3bJ70AJwb4mMjFSPHj3Uq1cv+fv7Kzg4WFOnTtWFCxfUqVMneXt7q0SJElq4cKEkKTk5WZ07d1bRokXl4eGhUqVK6YMPPnAYs2PHjmrZsqXee+89hYaGKn/+/IqOjlZSEt9PBQAAslaml34HgLtpxowZCggI0MaNG9WjRw+9/PLLatOmjWrWrKktW7bo0UcfVbt27XTx4kWlpKSoQIEC+u677/TXX39p6NCheuONNzR79myHMVesWKF9+/ZpxYoVmjFjhqZPn67p06c75wABAMB9K1MLZKSkpGj69OmaO3euDhw4IJvNpqJFi+qpp55Su3btZLPZTNZ6V3AaIXDviIyMVHJystasWSPp2syVr6+vnnzySX3xxReSrn1peWhoqNavX6+HH344zRjdu3dXbGysvv/+e0nXZrZWrlypffv22Rfbefrpp5UrVy598803d+nIAABAdnBXTiOUri1E8cQTT6hLly46cuSIKlSooHLlyungwYPq2LGjWrVqlekHB4BbiYiIsP+cO3du5c+fXxUqVLC3BQcHS5JOnDghSZo0aZKqVq2qwMBAeXl5acqUKTp06JDDmOXKlXNY1TQ0NNS+PwAAQFbJ8PdsTZ8+XatXr9ayZctUv359h77ly5erZcuW+uKLL9S+ffssLxJAzpX6vXmpbDabQ1vqjHpKSoq++eYbvf766xo3bpxq1Kghb29vjR07Vhs2bLjlmCkpKYaOAAAA5FQZntmaNWuW3njjjTRBS5IaNGigAQMGaObMmVlaHABkxrp161SzZk298sorqly5skqUKKF9+/Y5uywAAJBDZThsbd++XU2aNLlhf9OmTfXHH39kSVEAcp64i4nadyJBWw+d1b6TCYq7mJjpMcLDw/X7779r8eLF2r17t4YMGaJNmzYZqBYAAODWMnwa4ZkzZ+zXRqQnODhYZ8+ezZKiAOQsR89dUv8527Vmzyl7W93wACVezdypfS+99JK2bt2qZ555RjabTVFRUXrllVfsS8MDAADcTRlejTB37tyKjY1VYGBguv3Hjx9XWFhYtv+uLVYjBO6uuIuJ6j5rq0PQSlU3PEAToyrL19PVCZUBAICc7k6zQYZntizLUseOHeXm5pZu/5UrVzL94ABwKiEx3aAlSav3nNKphETCFgAAyJYyHLY6dOhwy21YiRBAZsVfTrpp//lb9AMAANyrMhy2pk2bZrIOADmUj7vLTfu9b9EPAABwr8rwaoQAYEKAl6vqhgek21c3PEABXpxCCAAAsifCFgCn8vV01ejWEWkCV93wAL3bOoLrtQAAQLaV4dMIAcCUMD8PTYyqrFMJiTp/OUne7i4K8HIlaAEAgGyNsAXgnuDrSbgCAAD3F04jBAAAAAADCFsAAAAAYABhCwAAAAAMIGwBAAAAgAGELQAAAAAwgLAFAAAAAAYQtgAAAADAAMIWAAAAABhA2AIAAAAAAwhbAAAAAGAAYQsAAAAADCBsAQAAAIABhC0AAAAAMICwBQAAAAAGELYAAAAAwADCFgAAAAAYQNgCAAAAAAMIWwAAAABgAGELAAAAAAwgbAEAAACAAYQtAAAAADCAsAUAAAAABhC2AAAAAMAAwhYAAAAAGEDYAgAAAAADCFsAAAAAYABhCwAAAAAMIGwBAAAAgAGELQAAAAAwgLAFAAAAAAYQtgAAAADAAMIWAAAAABhA2AIAAAAAAwhbAAAAAGAAYQsAAAAADCBsAQAAAIABhC0AAAAAMICwBQAAAAAGELYAAAAAwADCFgAAAAAYQNgCAAAAAAMIWwAAAABgAGELAAAAAAwgbAEAAACAAYQtAAAAADCAsAUAAAAABhC2AAAAAMAAwhYAAAAAGEDYAgAAAAADCFsAAAAAYABhCwAAAAAMIGwBAAAAgAGELQAAAAAwgLAFAAAAAAYQtgAAAADAAMIWAAAAABhA2AIAAAAAAwhbAAAAAGAAYQsAAAAADCBsAQAAAIABhC0AAAAAMICwBQAAAAAGELYAAAAAwADCFgAAAAAYQNgCAAAAAAMIWwAAAABgAGELAAAAAAwgbAEAAACAAYQtAAAAADCAsAUAAAAABhC2AAAAAMAAwhYAAAAAGEDYAgAAAAADCFsAAAAAYABhCwAAAAAMIGwBAAAAgAGELQAAAAAwgLAFAAAAAAYQtgAAAADAAMIWAAAAABhA2AIAAAAAA7Jd2Lpy5YoqVaokm82mbdu2OfRt375dderUkbu7uwoWLKgxY8Y4p0gAAAAAOV62C1v9+vVTWFhYmvb4+Hg9+uijKly4sDZv3qyxY8dq+PDhmjJlihOqBAAAAJDT5XF2AZmxcOFCLVmyRHPmzNHChQsd+mbOnKnExER9/vnncnV1Vbly5bRt2za9//77evHFF2845pUrV3TlyhX7/fj4eGP1AwAAAMg5ss3M1vHjx9W1a1d9+eWX8vT0TNO/fv161a1bV66urva2xo0bKyYmRmfPnr3huKNGjZKvr6/9VrBgQSP1AwAAAMhZskXYsixLHTt2VLdu3VStWrV0t4mNjVVwcLBDW+r92NjYG449cOBAxcXF2W+HDx/OusIBAAAA5FhODVsDBgyQzWa76e3vv//WxIkTdf78eQ0cODDLa3Bzc5OPj4/DDQAAAADulFOv2erTp486dux4022KFSum5cuXa/369XJzc3Poq1atmtq2basZM2YoJCREx48fd+hPvR8SEpKldQMAAADArTg1bAUGBiowMPCW23344Yd666237PePHj2qxo0b69tvv1X16tUlSTVq1NCgQYOUlJQkFxcXSdLSpUtVqlQp+fv7mzkAAAAAALiBbLEaYaFChRzue3l5SZKKFy+uAgUKSJKee+45jRgxQp07d1b//v21c+dOffDBBxo/fvxdrxcAAAAAskXYyghfX18tWbJE0dHRqlq1qgICAjR06NCbLvsOAAAAAKbYLMuynF3EvSQ+Pl6+vr6Ki4tjsQwAAAAgB7vTbJAtln4HAAAAgOyGsAUAAAAABhC2AAAAAMAAwhYAAAAAGEDYAgAAAAADCFsAAAAAYABhCwAAAAAMIGwBAAAAgAGELQAAAAAwgLAFAAAAAAYQtgAAAADAAMIWAAAAABhA2AIAAAAAAwhbAAAAAGAAYQsAAAAADCBsAQAAAIABhC0AAAAAMICwBQAAAAAGELYAAAAAwADCFgAAAAAYQNgCAAAAAAMIWwAAAABgAGELAAAAAAwgbAEAAACAAYQtAAAAADCAsAUAAAAABhC2AAAAAMAAwhYAAAAAGEDYAgAAAAADCFsAAAAAYABhCwAAAAAMIGwBAAAAgAGELQAAAAAwgLAFAAAAAAYQtgAAAADAAMIWAAAAABhA2AIAAAAAAwhbAAAAAGAAYQsAAAAADCBsAQAAAIABhC0AAAAAMICwBQAAAAAGELYAAAAAwADCFgAAAAAYQNgCAAAAAAMIWwAAAABgAGELAAAAAAwgbAEAAACAAYQtAAAAADCAsAUAAAAABhC2AAAAAMAAwhYAAAAAGEDYAgAAAAADCFsAAAAAYABhCwAAAAAMIGwBAAAAgAGELQAAAAAwgLAFAAAAAAYQtgAAAADAAMIWAAAAABhA2AIAAAAAAwhbAAAAAGAAYQsAAAAADCBsAQAAAIABhC0AAAAAMICwBQAAAAAGELYAAAAAwADCFgAAAAAYQNgCAAAAAAMIWwAAAABgAGELAAAAAAwgbAEAAACAAYQtAAAAADCAsAUAAAAABhC2AAAAAMAAwhYAAAAAGEDYAgAAAAADCFsAAAAAYABhCwAAAAAMIGwBAAAAgAGELQAAAAAwgLAFAAAAAAYQtgAAAADAAMIWAAAAABhA2AIAAAAAAwhbAAAAAGAAYQsAAAAADCBsAQAAAIABhC0AAAAAMICwBQAAAAAGELYAAAAAwADCFgAAAAAYQNgCAAAAAAMIWwAAAABgAGELAAAAAAwgbAEAAACAAdkqbP3888+qXr26PDw85O/vr5YtWzr0Hzp0SM2aNZOnp6eCgoLUt29fXb161TnFAgAAAMjR8ji7gIyaM2eOunbtqnfeeUcNGjTQ1atXtXPnTnt/cnKymjVrppCQEP366686duyY2rdvLxcXF73zzjtOrBwAAABATmSzLMtydhG3cvXqVRUpUkQjRoxQ586d091m4cKFevzxx3X06FEFBwdLkj755BP1799fJ0+elKura4YeKz4+Xr6+voqLi5OPj0+WHQMAAACA7OVOs0G2OI1wy5YtOnLkiHLlyqXKlSsrNDRUTZs2dZjZWr9+vSpUqGAPWpLUuHFjxcfH688//7zh2FeuXFF8fLzDDQAAAADuVLYIW//8848kafjw4Ro8eLAWLFggf39/RUZG6syZM5Kk2NhYh6AlyX4/Njb2hmOPGjVKvr6+9lvBggUNHQUAAACAnMSpYWvAgAGy2Ww3vf39999KSUmRJA0aNEitW7dW1apVNW3aNNlsNn333Xd3VMPAgQMVFxdnvx0+fDgrDg0AAABADufUBTL69Omjjh073nSbYsWK6dixY5KksmXL2tvd3NxUrFgxHTp0SJIUEhKijRs3Oux7/Phxe9+NuLm5yc3N7XbKBwAAAIAbcmrYCgwMVGBg4C23q1q1qtzc3BQTE6PatWtLkpKSknTgwAEVLlxYklSjRg29/fbbOnHihIKCgiRJS5culY+Pj0NIAwAAAIC7IVss/e7j46Nu3bpp2LBhKliwoAoXLqyxY8dKktq0aSNJevTRR1W2bFm1a9dOY8aMUWxsrAYPHqzo6GhmrgAAAADcddkibEnS2LFjlSdPHrVr106XLl1S9erVtXz5cvn7+0uScufOrQULFujll19WjRo1lDdvXnXo0EEjR450cuUAAAAAcqJs8T1bdxPfswUAAABAyiHfswUAAAAA2Q1hCwAAAAAMIGwBAAAAgAGELQAAAAAwgLAFAAAAAAYQtgAAAADAAMIWAAAAABhA2AIAAAAAAwhbAAAAAGAAYQsAAAAADCBsAQAAAIABhC0AAAAAMICwBQAAAAAGELYAAAAAwADCFgAAAAAYQNgCAAAAAAMIWwAAAABgAGELAAAAAAwgbAEAAACAAYQtAAAAADCAsAUAAAAABhC2AAAAAMAAwhYAAAAAGEDYAgAAAAADCFsAAAAAYABhCwAAAAAMIGwBAAAAgAGELQAAAAAwgLAFAAAAAAYQtgAAAADAAMIWAAAAABhA2AIAAAAAAwhbAAAAAGAAYQsAAAAADCBsAQAAAIABhC0AAAAAMICwBQAAAAAGELYAAAAAwADCFgAAAAAYQNgCAAAAAAMIWwAAAABgAGELAAAAAAwgbAEAAACAAYQtAAAAADCAsAUAAAAABhC2AAAAAMAAwhYAAAAAGEDYAgAAAAADCFsAAAAAYABhCwAAAAAMIGwBAAAAgAGELQAAAAAwgLAFAAAAAAYQtgAAAADAAMIWAAAAABhA2AIAAAAAAwhbAAAAAGAAYQsAAAAADCBsAQAAAIABhC0AAAAAMICwBQAAAAAGELYAAAAAwADCFgAAAAAYQNgCAAAAAAMIWwAAAABgAGELAAAAAAwgbAEAAACAAYQt5DiRkZHq1auXs8sAAADAfY6wBQAAAAAGELYAAAAAwADCFu5rFy5cUPv27eXl5aXQ0FCNGzfOof/s2bNq3769/P395enpqaZNm2rPnj0O20ydOlUFCxaUp6enWrVqpffff19+fn538SgAAACQHRG2cF/r27evVq1apfnz52vJkiVauXKltmzZYu/v2LGjfv/9d/34449av369LMvSY489pqSkJEnSunXr1K1bN7366qvatm2bGjVqpLfffttZhwMAAIBsxGZZluXsIu4l8fHx8vX1VVxcnHx8fJxdDu5AQkKC8ufPr6+++kpt2rSRJJ05c0YFChTQiy++qOjoaJUsWVLr1q1TzZo1JUmnT59WwYIFNWPGDLVp00bPPvusEhIStGDBAvu4zz//vBYsWKBz584547AAAABwl9xpNmBmC/etffv2KTExUdWrV7e35cuXT6VKlZIk7dq1S3ny5HHoz58/v0qVKqVdu3ZJkmJiYvTQQw85jHv9fQAAACA9hC0AAAAAMICwhftW8eLF5eLiog0bNtjbzp49q927d0uSypQpo6tXrzr0nz59WjExMSpbtqwkqVSpUtq0aZPDuNffBwAAANKTx9kFAFkh7mKiTiUkKv5yknw8XBSQ11W+Xl7q3Lmz+vbtq/z58ysoKEiDBg1SrlzX/sYQHh6uFi1aqGvXrvr000/l7e2tAQMG6IEHHlCLFi0kST169FDdunX1/vvvq3nz5lq+fLkWLlwom83mzMMFAABANkDYQrZ39Nwl9Z+zXWv2nLK31Q0P0OjWERo7dqwSEhLUvHlzeXt7q0+fPoqLi7NvN23aNL366qt6/PHHlZiYqLp16+q///2vXFxcJEm1atXSJ598ohEjRmjw4MFq3LixXnvtNX300Ud3/TgBAACQvbAa4XVYjTB7ibuYqO6ztjoErVR1wwM0MaqyfD1ds/Qxu3btqr///ltr1qzJ0nEBAABwb7nTbMDMFrK1UwmJ6QYtSVq955ROJSTecdh677331KhRI+XNm1cLFy7UjBkzNHny5DsaEwAAAPc/whaytfjLSTftP3+L/ozYuHGjxowZo/Pnz6tYsWL68MMP1aVLlzseFwAAAPc3whayNR93l5v2e9+iPyNmz559x2MAAAAg52Hpd2RrAV6uqhsekG5f3fAABXhl7fVaAAAAQEYRtpCt+Xq6anTriDSBq254gN5tHZHli2MAAAAAGcVphMj2wvw8NDGqsk4lJOr85SR5u7sowMuVoAUAAACnImzhvuDrSbgCAADAvYXTCAEAAADAAMIWAAAAABhA2AIAAAAAAwhbAAAAAGAAYSsHGT58uCpVquTsMgAAAIAcgbAFAAAAAAYQtrKZlJQUjRkzRiVKlJCbm5sKFSqkt99+W5LUv39/lSxZUp6enipWrJiGDBmipKQkSdL06dM1YsQI/fHHH7LZbLLZbJo+fboTjwQAAAC4v/E9W9nMwIEDNXXqVI0fP161a9fWsWPH9Pfff0uSvL29NX36dIWFhWnHjh3q2rWrvL291a9fPz3zzDPauXOnFi1apF9++UWS5Ovr68xDAQAAAO5rNsuyLGcXcS+Jj4+Xr6+v4uLi5OPj4+xyHJw/f16BgYH66KOP1KVLl1tu/9577+mbb77R77//LunaNVvz5s3Ttm3bDFcKAAAAZH93mg2Y2cpGdu3apStXrqhhw4bp9n/77bf68MMPtW/fPiUkJOjq1av3XGAEAAAAcopsc83W7t271aJFCwUEBMjHx0e1a9fWihUrHLY5dOiQmjVrJk9PTwUFBalv3766evWqkyrOeh4eHjfsW79+vdq2bavHHntMCxYs0NatWzVo0CAlJibexQoBAAAApMo2Yevxxx/X1atXtXz5cm3evFkVK1bU448/rtjYWElScnKymjVrpsTERP3666+aMWOGpk+frqFDhzq58qwTHh4uDw8PLVu2LE3fr7/+qsKFC2vQoEGqVq2awsPDdfDgQYdtXF1dlZycfLfKBQAAAHK0bHEa4alTp7Rnzx795z//UUREhCRp9OjRmjx5snbu3KmQkBAtWbJEf/31l3755RcFBwerUqVKevPNN9W/f38NHz5crq6uTj6KzIm7mKhTCYmKv5wkHw8XBeR1la+nu/r3769+/frJ1dVVtWrV0smTJ/Xnn38qPDxchw4d0jfffKMHH3xQP//8s3744QeHMYsUKaL9+/dr27ZtKlCggLy9veXm5uakIwQAAADub9liZit//vwqVaqUvvjiC124cEFXr17Vp59+qqCgIFWtWlXStdPoKlSooODgYPt+jRs3Vnx8vP78888bjn3lyhXFx8c73Jzt6LlL6j5rqxq+v0qtJv+qhuNWqcesrTp67pKGDBmiPn36aOjQoSpTpoyeeeYZnThxQk888YRee+01de/eXZUqVdKvv/6qIUOGOIzbunVrNWnSRPXr11dgYKBmzZrlpCMEAAAA7n/ZZjXC//3vf2rZsqW2bNmiXLlyKSgoSD///LMqV64sSXrxxRd18OBBLV682L7PxYsXlTdvXv33v/9V06ZN0x13+PDhGjFiRJp2Z61GGHcxUd1nbdWaPafS9NUND9DEqMry9cxes3QAAABAdnSnqxE6dWZrwIAB9i/YvdHt77//lmVZio6OVlBQkNasWaONGzeqZcuWat68uY4dO3ZHNQwcOFBxcXH22+HDh7Po6G7PqYTEdIOWJK3ec0qnEljwAgAAAMgOnHrNVp8+fdSxY8ebblOsWDEtX75cCxYs0NmzZ+2JcvLkyVq6dKlmzJihAQMGKCQkRBs3bnTY9/jx45KkkJCQG47v5uZ2T123FH856ab952/RDwAAAODe4NSwFRgYqMDAwFtud/HiRUlSrlyOE3G5cuVSSkqKJKlGjRp6++23deLECQUFBUmSli5dKh8fH5UtWzaLKzfHx93lpv3et+gHAAAAcG/IFgtk1KhRQ/7+/urQoYP++OMP7d69W3379tX+/fvVrFkzSdKjjz6qsmXLql27dvrjjz+0ePFiDR48WNHR0ffUzNWtBHi5qm54QLp9dcMDFODF9VoAAABAdpAtwlZAQIAWLVqkhIQENWjQQNWqVdPatWs1f/58VaxYUZKUO3duLViwQLlz51aNGjX0/PPPq3379ho5cqSTq88cX09XjW4dkSZw1Q0P0LutI1gcAwAAAMgmss1qhHfLna44klVSv2fr/OUkebu7KMDLlaAFAAAA3EV3mg2yxZca50S+noQrAAAAIDvLFqcRAgAAAEB2Q9gCAAAAAAMIWwAAAABgAGELAAAAAAwgbAEAAACAAYQtAAAAADCAsAUAAAAABhC2AAAAAMAAwhYAAAAAGEDYAgAAAAADCFsAAAAAYABhCwAAAAAMIGwBAAAAgAGELQAAAAAwgLAFAAAAAAYQtgAAAADAAMIWAAAAABhA2AIAAAAAAwhbAAAAAGAAYQsAAAAADCBsAQAAAIABhC0AAAAAMICwBQAAAAAGELYAAAAAwADCFgAAAAAYQNgCAAAAAAMIWwAAAABgAGELAAAAAAzI4+wC7jWWZUmS4uPjnVwJAAAAAGdKzQSpGSGzCFvXOX/+vCSpYMGCTq4EAAAAwL3g/Pnz8vX1zfR+Nut2Y9p9KiUlRUePHpW3t7dsNtttjxMfH6+CBQvq8OHD8vHxycIKkZV4nbIPXqvsgdcpe+B1yh54nbIHXqfs4XZfJ8uydP78eYWFhSlXrsxfgcXM1nVy5cqlAgUKZNl4Pj4+vPGyAV6n7IPXKnvgdcoeeJ2yB16n7IHXKXu4ndfpdma0UrFABgAAAAAYQNgCAAAAAAMIW4a4ublp2LBhcnNzc3YpuAlep+yD1yp74HXKHnidsgdep+yB1yl7cNbrxAIZAAAAAGAAM1sAAAAAYABhCwAAAAAMIGwBAAAAgAGELQAAAAAwgLBlQJEiRWSz2Rxuo0ePdthm+/btqlOnjtzd3VWwYEGNGTPGSdXmTAcOHFDnzp1VtGhReXh4qHjx4ho2bJgSExMdtrn+dbTZbPrtt9+cWHnONGnSJBUpUkTu7u6qXr26Nm7c6OyScrRRo0bpwQcflLe3t4KCgtSyZUvFxMQ4bBMZGZnmvdOtWzcnVZwzDR8+PM1rULp0aXv/5cuXFR0drfz588vLy0utW7fW8ePHnVhxzpTeZwabzabo6GhJvJecZfXq1WrevLnCwsJks9k0b948h37LsjR06FCFhobKw8NDjzzyiPbs2eOwzZkzZ9S2bVv5+PjIz89PnTt3VkJCwl08ipzhZq9VUlKS+vfvrwoVKihv3rwKCwtT+/btdfToUYcxMvLZ/XYRtgwZOXKkjh07Zr/16NHD3hcfH69HH31UhQsX1ubNmzV27FgNHz5cU6ZMcWLFOcvff/+tlJQUffrpp/rzzz81fvx4ffLJJ3rjjTfSbPvLL784vJZVq1Z1QsU517fffqvevXtr2LBh2rJliypWrKjGjRvrxIkTzi4tx1q1apWio6P122+/aenSpUpKStKjjz6qCxcuOGzXtWtXh/cOf1S6+8qVK+fwGqxdu9be99prr+mnn37Sd999p1WrVuno0aN68sknnVhtzrRp0yaH12jp0qWSpDZt2ti34b109124cEEVK1bUpEmT0u0fM2aMPvzwQ33yySfasGGD8ubNq8aNG+vy5cv2bdq2bas///xTS5cu1YIFC7R69Wq9+OKLd+sQcoybvVYXL17Uli1bNGTIEG3ZskVz585VTEyMnnjiiTTb3uyz+x2xkOUKFy5sjR8//ob9kydPtvz9/a0rV67Y2/r372+VKlXqLlSHGxkzZoxVtGhR+/39+/dbkqytW7c6ryhYDz30kBUdHW2/n5ycbIWFhVmjRo1yYlX4txMnTliSrFWrVtnb6tWrZ7366qvOKwrWsGHDrIoVK6bbd+7cOcvFxcX67rvv7G27du2yJFnr16+/SxUiPa+++qpVvHhxKyUlxbIs3kv3AknWDz/8YL+fkpJihYSEWGPHjrW3nTt3znJzc7NmzZplWZZl/fXXX5Yka9OmTfZtFi5caNlsNuvIkSN3rfac5vrXKj0bN260JFkHDx60t93qs/udYGbLkNGjRyt//vyqXLmyxo4dq6tXr9r71q9fr7p168rV1dXe1rhxY8XExOjs2bPOKBeS4uLilC9fvjTtTzzxhIKCglS7dm39+OOPTqgs50pMTNTmzZv1yCOP2Nty5cqlRx55ROvXr3diZfi3uLg4SUrz/pk5c6YCAgJUvnx5DRw4UBcvXnRGeTnanj17FBYWpmLFiqlt27Y6dOiQJGnz5s1KSkpyeG+VLl1ahQoV4r3lRImJifrqq6/0wgsvyGaz2dt5L91b9u/fr9jYWIf3j6+vr6pXr25//6xfv15+fn6qVq2afZtHHnlEuXLl0oYNG+56zfg/cXFxstls8vPzc2i/2Wf3O5EnS0aBg549e6pKlSrKly+ffv31Vw0cOFDHjh3T+++/L0mKjY1V0aJFHfYJDg629/n7+9/1mnO6vXv3auLEiXrvvffsbV5eXho3bpxq1aqlXLlyac6cOWrZsqXmzZuX7vQzst6pU6eUnJxsf3+kCg4O1t9//+2kqvBvKSkp6tWrl2rVqqXy5cvb25977jkVLlxYYWFh2r59u/r376+YmBjNnTvXidXmLNWrV9f06dNVqlQpHTt2TCNGjFCdOnW0c+dOxcbGytXVNc2HjeDgYMXGxjqnYGjevHk6d+6cOnbsaG/jvXTvSX2PpPf/ptS+2NhYBQUFOfTnyZNH+fLl4z3mRJcvX1b//v0VFRUlHx8fe/utPrvfCcJWBg0YMEDvvvvuTbfZtWuXSpcurd69e9vbIiIi5OrqqpdeekmjRo2Sm5ub6VJztMy8TqmOHDmiJk2aqE2bNuratau9PSAgwOG1fPDBB3X06FGNHTuWsAX8f9HR0dq5c6fDtUCSHK5LqFChgkJDQ9WwYUPt27dPxYsXv9tl5khNmza1/xwREaHq1aurcOHCmj17tjw8PJxYGW7kP//5j5o2baqwsDB7G+8lIGskJSXp6aeflmVZ+vjjjx36TH52J2xlUJ8+fRz+0pSeYsWKpdtevXp1Xb16VQcOHFCpUqUUEhKSZsWn1PshISFZUm9OldnX6ejRo6pfv75q1qyZoQVKqlevbr94GeYFBAQod+7c6b5feK84X/fu3e0XfRcoUOCm21avXl3StVlkPiA6h5+fn0qWLKm9e/eqUaNGSkxM1Llz5xxmt3hvOc/Bgwf1yy+/3HLGiveS86W+R44fP67Q0FB7+/Hjx1WpUiX7Ntcv5HT16lWdOXOG95gTpAatgwcPavny5Q6zWum5/rP7nSBsZVBgYKACAwNva99t27YpV65c9unkGjVqaNCgQUpKSpKLi4skaenSpSpVqhSnEN6hzLxOR44cUf369VW1alVNmzZNuXLd+hLGbdu2OfzDCrNcXV1VtWpVLVu2TC1btpR07bS1ZcuWqXv37s4tLgezLEs9evTQDz/8oJUrV6Y5LTo927ZtkyTeP06UkJCgffv2qV27dqpatapcXFy0bNkytW7dWpIUExOjQ4cOqUaNGk6uNGeaNm2agoKC1KxZs5tux3vJ+YoWLaqQkBAtW7bMHq7i4+O1YcMGvfzyy5KufdY7d+6cNm/ebF/FePny5UpJSbEHZtwdqUFrz549WrFihfLnz3/Lfa7/7H4nCFtZbP369dqwYYPq168vb29vrV+/Xq+99pqef/55e5B67rnnNGLECHXu3Fn9+/fXzp079cEHH2j8+PFOrj7nOHLkiCIjI1W4cGG99957OnnypL0v9S9OM2bMkKurqypXrixJmjt3rj7//HN99tlnTqk5p+rdu7c6dOigatWq6aGHHtKECRN04cIFderUydml5VjR0dH6+uuvNX/+fHl7e9uvP/D19ZWHh4f27dunr7/+Wo899pjy58+v7du367XXXlPdunUVERHh5Opzjtdff13NmzdX4cKFdfToUQ0bNky5c+dWVFSUfH191blzZ/Xu3Vv58uWTj4+PevTooRo1aujhhx92duk5TkpKiqZNm6YOHTooT57/+2jGe8l5EhIStHfvXvv9/fv3a9u2bcqXL58KFSqkXr166a233lJ4eLiKFi2qIUOGKCwszP6HwTJlyqhJkybq2rWrPvnkEyUlJal79+569tlnHU4TxZ272WsVGhqqp556Slu2bNGCBQuUnJxs/39Wvnz55OrqmqHP7nfEyBqHOdjmzZut6tWrW76+vpa7u7tVpkwZ65133rEuX77ssN0ff/xh1a5d23Jzc7MeeOABa/To0U6qOGeaNm2aJSndW6rp06dbZcqUsTw9PS0fHx/roYceclgmGXfPxIkTrUKFClmurq7WQw89ZP3222/OLilHu9F7Z9q0aZZlWdahQ4esunXrWvny5bPc3NysEiVKWH379rXi4uKcW3gO88wzz1ihoaGWq6ur9cADD1jPPPOMtXfvXnv/pUuXrFdeecXy9/e3PD09rVatWlnHjh1zYsU51+LFiy1JVkxMjEM77yXnWbFiRbr/znXo0MGyrGvLvw8ZMsQKDg623NzcrIYNG6Z5/U6fPm1FRUVZXl5elo+Pj9WpUyfr/PnzTjia+9vNXqvUr/FJ77ZixQrLsjL+2f122SzLsu48sgEAAAAA/o3v2QIAAAAAAwhbAAAAAGAAYQsAAAAADCBsAQAAAIABhC0AAAAAMICwBQAAAAAGELYAAAAAwADCFgAAAAAYQNgCANyzbDab5s2b5+wybmrlypWy2Ww6d+6cs0sBANxjCFsAgLuqY8eOstlsstlscnFxUXBwsBo1aqTPP/9cKSkpDtseO3ZMTZs2dVKlGVOzZk0dO3ZMvr6+Rh9n9erVat68ucLCwrJFCAUAELYAAE7QpEkTHTt2TAcOHNDChQtVv359vfrqq3r88cd19epV+3YhISFyc3NzYqW35urqqpCQENlsNqOPc+HCBVWsWFGTJk0y+jgAgKxD2AIA3HVubm4KCQnRAw88oCpVquiNN97Q/PnztXDhQk2fPt2+3b9ncA4cOCCbzabZs2erTp068vDw0IMPPqjdu3dr06ZNqlatmry8vNS0aVOdPHnS4fE+++wzlSlTRu7u7ipdurQmT55s70sdd+7cuapfv748PT1VsWJFrV+/3r7NwYMH1bx5c/n7+ytv3rwqV66c/vvf/0pK/zTCOXPmqFy5cnJzc1ORIkU0btw4h3qKFCmid955Ry+88IK8vb1VqFAhTZky5abPWdOmTfXWW2+pVatWmXmqAQBORNgCANwTGjRooIoVK2ru3Lk33W7YsGEaPHiwtmzZojx58ui5555Tv3799MEHH2jNmjXau3evhg4dat9+5syZGjp0qN5++23t2rVL77zzjoYMGaIZM2Y4jDto0CC9/vrr2rZtm0qWLKmoqCj7LFt0dLSuXLmi1atXa8eOHXr33Xfl5eWVbn2bN2/W008/rWeffVY7duzQ8OHDNWTIEIcQKUnjxo1TtWrVtHXrVr3yyit6+eWXFRMTcxvPHADgXpXH2QUAAJCqdOnS2r59+023ef3119W4cWNJ0quvvqqoqCgtW7ZMtWrVkiR17tzZIdgMGzZM48aN05NPPilJKlq0qP766y99+umn6tChg8O4zZo1kySNGDFC5cqV0969e1W6dGkdOnRIrVu3VoUKFSRJxYoVu2F977//vho2bKghQ4ZIkkqWLKm//vpLY8eOVceOHe3bPfbYY3rllVckSf3799f48eO1YsUKlSpVKiNPFQAgG2BmCwBwz7As65bXPkVERNh/Dg4OliR7CEptO3HihKRr1znt27dPnTt3lpeXl/321ltvad++fTccNzQ0VJLs4/Ts2VNvvfWWatWqpWHDht00EO7atcse/FLVqlVLe/bsUXJycrqPZ7PZFBISYn88AMD9gbAFALhn7Nq1S0WLFr3pNi4uLvafU4PZ9W2pqxomJCRIkqZOnapt27bZbzt37tRvv/12y3FTx+nSpYv++ecftWvXTjt27FC1atU0ceLE2z3MNI93fd0AgPsDYQsAcE9Yvny5duzYodatW2fZmMHBwQoLC9M///yjEiVKONxuFequV7BgQXXr1k1z585Vnz59NHXq1HS3K1OmjNatW+fQtm7dOpUsWVK5c+e+7WMBAGQ/XLMFALjrrly5otjYWCUnJ+v48eNatGiRRo0apccff1zt27fP0scaMWKEevbsKV9fXzVp0kRXrlzR77//rrNnz6p3794ZGqNXr15q2rSpSpYsqbNnz2rFihUqU6ZMutv26dNHDz74oN58800988wzWr9+vT766COHFRBvR0JCgvbu3Wu/v3//fm3btk358uVToUKF7mhsAIAZhC0AwF23aNEihYaGKk+ePPL391fFihX14YcfqkOHDsqVK2tPuujSpYs8PT01duxY9e3bV3nz5lWFChXUq1evDI+RnJys6Oho/e9//5OPj4+aNGmi8ePHp7ttlSpVNHv2bA0dOlRvvvmmQkNDNXLkSIfFMW7H77//rvr169vvpwbFDh06pFnpEABwb7BZlmU5uwgAAAAAuN9wzRYAAAAAGEDYAgAAAAADCFsAAAAAYABhCwAAAAAMIGwBAAAAgAGELQAAAAAwgLAFAAAAAAYQtgAAAADAAMIWAAAAABhA2AIAAAAAAwhbAAAAAGDA/wOG+zSez+QAXQAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 1000x1000 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import torch\n",
    "from transformers import AutoTokenizer, AutoModel\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "import seaborn as sns\n",
    "from sklearn.manifold import TSNE\n",
    "\n",
    "# 选择预训练的 BERT 模型\n",
    "model_name = 'bert-base-uncased'  # 对于中文模型，可使用 'bert-base-chinese'\n",
    "\n",
    "# 加载分词器和模型\n",
    "tokenizer = AutoTokenizer.from_pretrained(model_name)\n",
    "model = AutoModel.from_pretrained(model_name)\n",
    "\n",
    "# 要可视化的词语列表\n",
    "words = ['cat', 'dog', 'apple', 'orange', 'king', 'queen', 'man', 'woman']\n",
    "# words = ['猫', '狗', '苹果', '橙子', '国王', '王后', '男人', '女人']\n",
    "\n",
    "def get_input_embeddings(tokenizer, model, words):\n",
    "    # 获取词表中的索引\n",
    "    word_ids = [tokenizer.convert_tokens_to_ids(word) for word in words]\n",
    "    # 从嵌入层提取对应的向量\n",
    "    embeddings = model.embeddings.word_embeddings.weight[word_ids]\n",
    "    return embeddings.detach().numpy()\n",
    "\n",
    "def get_output_embeddings(tokenizer, model, words):\n",
    "    embeddings = []\n",
    "    for word in words:\n",
    "        inputs = tokenizer(word, return_tensors='pt')\n",
    "        outputs = model(**inputs)\n",
    "        # 获取 [CLS] 向量\n",
    "        cls_embedding = outputs.last_hidden_state[0][0]  # [CLS] 的向量\n",
    "        embeddings.append(cls_embedding.detach().numpy())\n",
    "    return np.array(embeddings)\n",
    "    \n",
    "# 获取输入嵌入向量\n",
    "embeddings = get_input_embeddings(tokenizer, model, words)\n",
    "\n",
    "# 降维处理\n",
    "tsne = TSNE(n_components=2, perplexity=2, n_iter=1000, random_state=42)\n",
    "embeddings_2d = tsne.fit_transform(embeddings)\n",
    "\n",
    "# 可视化\n",
    "plt.figure(figsize=(10, 10))\n",
    "sns.scatterplot(x=embeddings_2d[:, 0], y=embeddings_2d[:, 1])\n",
    "\n",
    "for i, word in enumerate(words):\n",
    "    plt.text(embeddings_2d[i, 0]+0.5, embeddings_2d[i, 1]+0.5, word)\n",
    "\n",
    "plt.title('Word Embeddings Visualized using t-SNE')\n",
    "plt.xlabel('Dimension 1')\n",
    "plt.ylabel('Dimension 2')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9005bf77-ae18-4239-83d2-a54328123c4e",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.16"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}