5. Fashion MNIST Train .ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.6.7"
    },
    "colab": {
      "name": "Part 4 - Fashion-MNIST (Exercises).ipynb",
      "provenance": [],
      "include_colab_link": true
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/avinregmi/PyTorch-Lessons/blob/master/5.%20Fashion%20MNIST%20Train%20.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "sdyBR4QhX_PB",
        "colab_type": "text"
      },
      "source": [
        "# Classifying Fashion-MNIST\n",
        "\n",
        "Now it's your turn to build and train a neural network. You'll be using the [Fashion-MNIST dataset](https://github.com/zalandoresearch/fashion-mnist), a drop-in replacement for the MNIST dataset. MNIST is actually quite trivial with neural networks where you can easily achieve better than 97% accuracy. Fashion-MNIST is a set of 28x28 greyscale images of clothes. It's more complex than MNIST, so it's a better representation of the actual performance of your network, and a better representation of datasets you'll use in the real world.\n",
        "\n",
        "<img src='https://github.com/avinregmi/deep-learning-v2-pytorch/blob/master/intro-to-pytorch/assets/fashion-mnist-sprite.png?raw=1' width=500px>\n",
        "\n",
        "In this notebook, you'll build your own neural network. For the most part, you could just copy and paste the code from Part 3, but you wouldn't be learning. It's important for you to write the code yourself and get it to work. Feel free to consult the previous notebooks though as you work through this.\n",
        "\n",
        "First off, let's load the dataset through torchvision."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "xIBae4iZX_PE",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 275
        },
        "outputId": "03879a8c-9422-44a2-a4d0-04a99cbf9e64"
      },
      "source": [
        "import torch\n",
        "from torchvision import datasets, transforms\n",
        "import helper\n",
        "\n",
        "# Define a transform to normalize the data\n",
        "transform = transforms.Compose([transforms.ToTensor(),\n",
        "                                transforms.Normalize((0.5,), (0.5,))])\n",
        "# Download and load the training data\n",
        "trainset = datasets.FashionMNIST('~/.pytorch/F_MNIST_data/', download=True, train=True, transform=transform)\n",
        "trainloader = torch.utils.data.DataLoader(trainset, batch_size=64, shuffle=True)\n",
        "\n",
        "# Download and load the test data\n",
        "testset = datasets.FashionMNIST('~/.pytorch/F_MNIST_data/', download=True, train=False, transform=transform)\n",
        "testloader = torch.utils.data.DataLoader(testset, batch_size=64, shuffle=True)"
      ],
      "execution_count": 1,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "\r0it [00:00, ?it/s]"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "Downloading http://fashion-mnist.s3-website.eu-central-1.amazonaws.com/train-images-idx3-ubyte.gz to /root/.pytorch/F_MNIST_data/FashionMNIST/raw/train-images-idx3-ubyte.gz\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "26427392it [00:07, 3572094.33it/s]                              \n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "Extracting /root/.pytorch/F_MNIST_data/FashionMNIST/raw/train-images-idx3-ubyte.gz to /root/.pytorch/F_MNIST_data/FashionMNIST/raw\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "\r0it [00:00, ?it/s]"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "Downloading http://fashion-mnist.s3-website.eu-central-1.amazonaws.com/train-labels-idx1-ubyte.gz to /root/.pytorch/F_MNIST_data/FashionMNIST/raw/train-labels-idx1-ubyte.gz\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "32768it [00:00, 72160.26it/s]                            \n",
            "0it [00:00, ?it/s]"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "Extracting /root/.pytorch/F_MNIST_data/FashionMNIST/raw/train-labels-idx1-ubyte.gz to /root/.pytorch/F_MNIST_data/FashionMNIST/raw\n",
            "Downloading http://fashion-mnist.s3-website.eu-central-1.amazonaws.com/t10k-images-idx3-ubyte.gz to /root/.pytorch/F_MNIST_data/FashionMNIST/raw/t10k-images-idx3-ubyte.gz\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "4423680it [00:01, 3066276.31it/s]                            \n",
            "0it [00:00, ?it/s]"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "Extracting /root/.pytorch/F_MNIST_data/FashionMNIST/raw/t10k-images-idx3-ubyte.gz to /root/.pytorch/F_MNIST_data/FashionMNIST/raw\n",
            "Downloading http://fashion-mnist.s3-website.eu-central-1.amazonaws.com/t10k-labels-idx1-ubyte.gz to /root/.pytorch/F_MNIST_data/FashionMNIST/raw/t10k-labels-idx1-ubyte.gz\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "8192it [00:00, 23620.85it/s]            "
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "Extracting /root/.pytorch/F_MNIST_data/FashionMNIST/raw/t10k-labels-idx1-ubyte.gz to /root/.pytorch/F_MNIST_data/FashionMNIST/raw\n",
            "Processing...\n",
            "Done!\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "\n"
          ],
          "name": "stderr"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5RqUjuIlX_PI",
        "colab_type": "text"
      },
      "source": [
        "Here we can see one of the images."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "k0coD7T3X_PJ",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 248
        },
        "outputId": "c6668126-5d53-426b-f1fd-a7a1c616dedd"
      },
      "source": [
        "image, label = next(iter(trainloader))\n",
        "imshow(image[0,:]);"
      ],
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAOcAAADnCAYAAADl9EEgAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjIsIGh0\ndHA6Ly9tYXRwbG90bGliLm9yZy8li6FKAAAKlElEQVR4nO3dy0/b6RXG8eMrtoFAYBImV2iahG6i\nNqNRNZW66n/dVuqmaXczo9lE7bSZJk1CgEC4lIvBty4qVVMp73MqLIsn+PvZHr3GNnnykzg6562M\nRqMA4Kd62W8AwMcRTsAU4QRMEU7AFOEETNVV8Te//rntn3IrlYqsT/Kv0K1WS9Yf/OSBrP/yyy+L\nte///jd5dnt7W9Yj+V7u3Lot62trq8Xab3/3e3l2e3tL1od0Bj7qD8++++gvjScnYIpwAqYIJ2CK\ncAKmCCdginACpggnYEr2OZ1lfcxr8/PF2trqmjx74+YNWW82GrLe7Z7p8zPNYu1XX30lzw4GA1mv\n1/SvNGmDxt7efrG2vv5Ynv3ii6ey3j09lfWd3Q/F2tbWpjy7lfV/E5fZNy/hyQmYIpyAKcIJmCKc\ngCnCCZginIApwgmY+mT7nFk/cHlpuVjrdrvy7GnSj8vq2et/8+23xdrjR7qXuLi4IOsHpwey/vLV\nK1nf2Ngo1mY7s/LsycmJrDeS/vBny0vF2trqfXn2zdu3sq6+84jL6WNmeHICpggnYIpwAqYIJ2CK\ncAKmCCdgaqKtFDWGk/3p+mfr67KuRsIiIjbFiFE6dlXXX0u1ov9Pa9R1y+DktNxy+Pqbr+XZWjIS\n1u/3ZL1a1e9dtUsqVT1WNejr7zX73lUrZm+/PMoWEbG8VG7DRETcu3tX1l+/eSPrl4EnJ2CKcAKm\nCCdginACpggnYIpwAqYIJ2Dq0kbGarWarN++dUvWj46OZX00LPdRs9GlbE1ijDldtLi4WKxlfchs\nLKtW1d9r1l8+Oy+v9Rz19dnsd5r9bPXZh8OhPLt/oEfl7ty+I+v0OQH83wgnYIpwAqYIJ2CKcAKm\nCCdginACpiba51R9rdX7etVh1u/LdGY7xVq2ujKTvbd2uy3r2+K6uo137+TZbNa02SxfLxgRUU16\nuLdv3S7Wsv7vaVevDM2M8zvP3ttMa0bWV27elPVxrxi8CJ6cgCnCCZginIApwgmYIpyAKcIJmCKc\ngKlLm+dU/bSIiG63PFcYEbGU7CkdDMs7UrM+Z9Zvy+ovXryQ9Z3d3WKt1WrJswsL+grA3vm5rL9J\n5hbVe/vpgwfy7NzcnKxnVyeqXmXWO24k/d9+ry/rd+7oeU/6nAD+i3ACpggnYIpwAqYIJ2CKcAKm\nJtpKaTbK40vZCM/h4aGsq1ZJhL7Kbmu4Jc/OzeqWwF++/6us7+3tybq6Ci+7qu7atWuyfnysV2fe\nvXdP1t+JkbXnz5/Ls09/8VTWWzO6TXR0fFSs3U/ed6+vWyWvX7+W9axFpdZ+ZlcbXhRPTsAU4QRM\nEU7AFOEETBFOwBThBEwRTsDURPucjx49LP/gZMQnW3V4dqZHym5+dqNYuy6u4IuI2Hi3Keu7O+Wx\nqoiIekN/NnW94UFylV12BWA2znZ8rK9ObM2U+8+HyTja8Yl+7U67vK40Qo+FZdcHHuzr763X68l6\nvaZ/Z2urq8Xaix9+kGcviicnYIpwAqYIJ2CKcAKmCCdginACpggnYGqifc73OzvFWqeje15L16/L\netL2ikq13Cdtil5eRMThYdIz6+ueWa1env2L0POiy8vL8uxwMJT1f7x6Kevj9EnVTGNEvtbzNFlJ\nuv74UbG2vKS/l+3372X9evLvaf9gX9b7ybzoJPDkBEwRTsAU4QRMEU7AFOEETBFOwBThBExNtM+5\nLa5NU7WIiLnZ8t7ZiIinT/WOVDX/l/7s5Co7PWka8fnKiqx/EHtt5+fn5dlGoyHraldwRMRJ6D6n\n7GUmzeWsh5p9th0xJ5v1Gf+Z7KXdFT33iIhuMh98GXhyAqYIJ2CKcAKmCCdginACpggnYIpwAqYm\n2udUsr20R8l+1T8+e3bh+vrjx/Jsdj9nS+xXjYg4PdVzi2qHavdMn816rNl+1kayL1jtE56b033K\nWrL7NdsNq+7nfPbnP8mzVxFPTsAU4QRMEU7AFOEETBFOwBThBExdWislu9Jtks6Tq+zmP9ctg6xd\nsftBXxGoRqcq2deSXPGXfa3DoV6tWa2UX78zr9eZNpKrDzPZtY7ThicnYIpwAqYIJ2CKcAKmCCdg\ninACpggnYOrS+pyX6V9H5dGkiIhGU6+fzK7Cy/p1qsfb6w/k2XZbX7M3iuxuRD101umUx+GuL+pr\n9NIeak0/C7JRu2nDkxMwRTgBU4QTMEU4AVOEEzBFOAFThBMw9cn2ObPVmqqXeNbV/bTBQPcaHz18\nKOsvX76U9Var3Kuc7eiZyWwONhsHza7pq4h50YXFBXn2YP9A1rNr/AYDXZ82PDkBU4QTMEU4AVOE\nEzBFOAFThBMwRTgBU59sn3Mcw6RXmO2lXbim+31PnjyR9c3NzWItmzXN+pzVpP/bTq4vbDbKs6xZ\n/zeb18zmPXs9+pw/xpMTMEU4AVOEEzBFOAFThBMwRTgBU4QTMDWVfc5GfbyP3T3T86BZL3FlZaVY\nOz7W85bZez/v6btHs/dWiXKfdJDs1K0n7y3rcw5Huj5teHICpggnYIpwAqYIJ2CKcAKmCCdgaipb\nKdk1eNl+yewKwNHw4iNpbbE2MyK/4q8ZTVnPridUK0ezVklDjJtFRJyf6zZPNu6mjLMq1RVPTsAU\n4QRMEU7AFOEETBFOwBThBEwRTsDUVPY5035aUq6Ka/Ii8p6b0uvrtZzpWFYydpX1+2rVcg/3/Fy/\nt2ZT91izOv4XT07AFOEETBFOwBThBEwRTsAU4QRMEU7A1HT2OSfYp4zIrxBUr5+tjxx3LjH7bINh\nef3lUNQiIoYD/d5nZrI+53jf+1XDkxMwRTgBU4QTMEU4AVOEEzBFOAFThBMwZdvnHLfXKF876XNm\ne2cz2UxltTK5/xPH3d+qzlfFrGdEfjVio6n32jbTPuh04ckJmCKcgCnCCZginIApwgmYIpyAKdtW\nyjh/8s+o9Y8R+RV/2VV23tfR6femWyn6bL/fl/Xsc7dmZmR92vDkBEwRTsAU4QRMEU7AFOEETBFO\nwBThBEzZ9jkz4/QKO522rGf9PLU+MiKi0dCjUYNB+fwkR+X+8/qTe+1srWe2MrRe19+bcrm948ng\nyQmYIpyAKcIJmCKcgCnCCZginIApwgmY+mT7nONot3WfU/UhI8ZfnTnJnlx2vWE2c6lkc7BZj/bs\n7Ey/fo1nxY/xbQCmCCdginACpggnYIpwAqYIJ2CKcAKmprLPOTc7J+vDpI9ZSeY9M5Oc2cx27mY/\nW9VHob+Xel3/c8quPuyPLt6DvYp4cgKmCCdginACpggnYIpwAqYIJ2CKcAKmprLPmfXjznvj3b+Z\nyfa7Ktks6Lj3muoer37f2c8ejJI52Su4e3YcPDkBU4QTMEU4AVOEEzBFOAFThBMwdWVbKc1Gs1xr\nlmsREaNz/Sf9bCwrW085zmtn1wtmBn3dzqhWy5+9Esm4WTJKl60UHed7u4r4NgBThBMwRTgBU4QT\nMEU4AVOEEzBFOAFTV7bPORiU1yy+3Xgrz2YjXVkvMqPGtrKRrqzPmY1dZdfwqZ+frgwdc+NnvT5e\nD/eq4ckJmCKcgCnCCZginIApwgmYIpyAKcIJmKqwjhDwxJMTMEU4AVOEEzBFOAFThBMwRTgBU/8G\nDySTg+bJFKMAAAAASUVORK5CYII=\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2ts4X4BSX_PO",
        "colab_type": "text"
      },
      "source": [
        "## Building the network\n",
        "\n",
        "Here you should define your network. As with MNIST, each image is 28x28 which is a total of 784 pixels, and there are 10 classes. You should include at least one hidden layer. We suggest you use ReLU activations for the layers and to return the logits or log-softmax from the forward pass. It's up to you how many layers you add and the size of those layers."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "n9Yg-6JfX_PP",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# TODO: Define your network architecture here\n",
        "import torch.nn.functional as F\n",
        "\n",
        "class Network(nn.Module):\n",
        "  def __init__(self):\n",
        "    super().__init__()\n",
        "    self.fc1 = nn.Linear(784,256)\n",
        "    self.fc2 = nn.Linear(256,128)\n",
        "    self.fc3 = nn.Linear(128,64)\n",
        "    self.fc4 = nn.Linear(64,10)\n",
        "\n",
        "  def forward(self,x):\n",
        "    # make sure input tensor is flattened\n",
        "    x = x.view(-1, 784)\n",
        "\n",
        "    x = self.fc1(x)\n",
        "    x = F.relu(x)\n",
        "\n",
        "    x = self.fc2(x)\n",
        "    x = F.relu(x)\n",
        "\n",
        "    x = self.fc3(x)\n",
        "    x = F.relu(x)\n",
        "\n",
        "    x = self.fc4(x)\n",
        "    x = F.log_softmax(x,dim=1)\n",
        "    return x\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qJPlDTB5X_PT",
        "colab_type": "text"
      },
      "source": [
        "# Train the network\n",
        "\n",
        "Now you should create your network and train it. First you'll want to define [the criterion](http://pytorch.org/docs/master/nn.html#loss-functions) ( something like `nn.CrossEntropyLoss`) and [the optimizer](http://pytorch.org/docs/master/optim.html) (typically `optim.SGD` or `optim.Adam`).\n",
        "\n",
        "Then write the training code. Remember the training pass is a fairly straightforward process:\n",
        "\n",
        "* Make a forward pass through the network to get the logits \n",
        "* Use the logits to calculate the loss\n",
        "* Perform a backward pass through the network with `loss.backward()` to calculate the gradients\n",
        "* Take a step with the optimizer to update the weights\n",
        "\n",
        "By adjusting the hyperparameters (hidden units, learning rate, etc), you should be able to get the training loss below 0.4."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "gExMG31CX_PU",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 119
        },
        "outputId": "dc109c58-fc21-4668-abe5-08fbdd8ffa88"
      },
      "source": [
        "# TODO: Create the network, define the criterion and optimizer\n",
        "\n",
        "model = Network()\n",
        "criterion = nn.NLLLoss()\n",
        "optimizer = optim.Adam(model.parameters(),lr=0.001)\n",
        "print(model)"
      ],
      "execution_count": 44,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Network(\n",
            "  (fc1): Linear(in_features=784, out_features=256, bias=True)\n",
            "  (fc2): Linear(in_features=256, out_features=128, bias=True)\n",
            "  (fc3): Linear(in_features=128, out_features=64, bias=True)\n",
            "  (fc4): Linear(in_features=64, out_features=10, bias=True)\n",
            ")\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "IPhG0JbaX_PW",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 102
        },
        "outputId": "090d24d3-6c5b-4f6f-d14c-87b3330e98d2"
      },
      "source": [
        "# TODO: Train the network here\n",
        "\n",
        "epoch = 5\n",
        "\n",
        "for i in range(epoch):\n",
        "  running_loss = 0\n",
        "  for images, labels in trainloader:\n",
        " \n",
        "    log_ps = model(images)\n",
        "    loss = criterion(log_ps,labels)\n",
        "\n",
        "    optimizer.zero_grad()\n",
        "    loss.backward()\n",
        "    optimizer.step()\n",
        "\n",
        "    running_loss += loss.item()\n",
        "\n",
        "  else:\n",
        "    print(f\"Epoch: {i}. Running Loss: {running_loss/len(trainloader)}\")\n"
      ],
      "execution_count": 45,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Epoch: 0. Running Loss: 0.5180570711967534\n",
            "Epoch: 1. Running Loss: 0.37806684533352536\n",
            "Epoch: 2. Running Loss: 0.3412134214552608\n",
            "Epoch: 3. Running Loss: 0.3136682447943606\n",
            "Epoch: 4. Running Loss: 0.29647466113000537\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "nYtNjLViX_Pa",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 284
        },
        "outputId": "449a35cf-8501-443c-8de3-d6f58eff06e1"
      },
      "source": [
        "%matplotlib inline\n",
        "%config InlineBackend.figure_format = 'retina'\n",
        "\n",
        "import helper\n",
        "\n",
        "# Test out your network!\n",
        "\n",
        "dataiter = iter(testloader)\n",
        "images, labels = dataiter.next()\n",
        "img = images[0]\n",
        "# Convert 2D image to 1D vector\n",
        "img = img.resize_(1, 784)\n",
        "\n",
        "# TODO: Calculate the class probabilities (softmax) for img\n",
        "ps = torch.exp(model(img))\n",
        "print(ps)\n",
        "# Plot the image and probabilities\n",
        "view_classify(img.resize_(1, 28, 28), ps, version='Fashion')"
      ],
      "execution_count": 47,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "tensor([[5.8123e-04, 2.4609e-07, 4.3805e-06, 1.0045e-06, 7.3789e-06, 2.2603e-05,\n",
            "         3.0936e-04, 5.8756e-07, 9.9907e-01, 2.4883e-07]],\n",
            "       grad_fn=<ExpBackward>)\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAA1EAAAGwCAYAAABIAu+kAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAAWJQAAFiUBSVIk8AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjIsIGh0\ndHA6Ly9tYXRwbG90bGliLm9yZy8li6FKAAAgAElEQVR4nOzdeZhkVX3/8fdnelZghh1UTAQVBfdA\nNIEYEaNRUVFR4x4xi1ES16hxjxqMJvFn3EJcUFExiuIuCoKCkqCCgxsuIPsmqzDDALP29/fHvS1l\n2T3Tp5menul5v56nnlt17/fcc25V9Ux/+yw3VYUkSZIkaXLmzHQDJEmSJGlLYhIlSZIkSQ1MoiRJ\nkiSpgUmUJEmSJDUwiZIkSZKkBiZRkiRJktTAJEqSJEmSGphESZIkSVIDkyhJkiRJamASJUmSJEkN\nTKIkSZIkqYFJlCRJkiQ1MImSJEmSpAYmUZIkSUCS6h97znRbthYz9Z7fnnqTHNOXfeNkz5vk8H7/\naVNrsTY3JlGSJGlWSbJNkhck+XKSS5PckuTmJBclOT7Js5Ismul2bipJLh745X7ssS7J9UlOT/LS\nJNvMdDu3Vn2C9cYkD5jptmjy5s50AyRJkjaWJI8DPgDcYWD3zcAosGf/eBLwb0meXVXf3NRtnEE3\nAyv65/OBnYAH94+/SXJwVV0zU43bgvwKOBe4rqHMsr7MpeMcOxw4CLgY+OHtbJs2EXuiJEnSrJDk\ncOALdAnUucCzgV2qaruqWgLsADwZOA24E/CQmWnpjHl7Vd2hf+wE7AK8BSjgXnTJpzagql5dVftU\n1Xsbyny+L/OX09k2bTomUZIkaYuX5P7A++h+t/kq8AdVdWxVXT8WU1XLquqzVXUw8DTgpplp7eah\nqq6vqtcBH+l3PT7JnWayTdKWwiRKkiTNBkcCC4ArgGdU1a3rC66q44B3TObESUaSPDrJ+5MsTXJ1\nktVJrkzy+SQPW0/ZOf2cl1P7OUhrklyb5KdJPpzkUeOU2SvJfyc5L8mt/ZyuS5KcluTVSXaZTLsb\nfHLg+X4D7fjNAgpJFiR5bZIfJ7mp37/DULsPTvK5JFf1789VG3p/hsrfJ8mn+nIrk/wiyeuTLJgg\nfnH/3n46yTlJbuzfr/OTfCDJ3tNU74QLS6ynjt9ZWGJsH91QPoCPDM1bu7iP+3D/+vgN1PGmPu6M\nybZLU+ecKEmStEVLsgfwmP7lu6tq2WTKVVVNsop96Xq3xiwHVgN3BJ4APCHJa6rqreOU/TjwjIHX\ny4AldEPp7tU/Thw7mGQ/uuGGi/tda+jmMv1+/zgI+MFgmY3gioHnS8Y5vhD4NvCgvj23DAckORJ4\nbf+y6K5zN257f95WVa9eTxsOpBtOuC3d+xvgnsCbgUOSPKKqVgyVeQ7wnv75ur7OOcDd+sczkjyh\nqk7ZyPVuLLcCV9PNTZvX1z+Y/F/bb48Gngs8LsnOg72rY5LMoXs/AD48Te3VAHuiJEnSlu6hdL/8\nAnxpGs6/mu4X00cC21fV9lW1HbA78Hq6X+DfkuSPBgsleQhdArUOeCmwpKp2oEtK7kS3oMD/DtX1\ndroE6nvAflU1v6p2pPsl/4HAO+mShY3p9wee3zjO8b8H7kE3BHK7/hr2pEvuSPI0bkug3gvs1rd5\nV25Lcl6V5FnracNRwM+A+1XV9nTvwXPpkoo/Zvxew+vo5nQ9CNimqname2/3BT5B9579T5JtN3K9\nG0VVHVdVdwDGeo5ePDBn7Q5V9cA+7oy+jfOBZ05wuocBd6H7TI6brjbrNiZRkiRpS7dvv11Ft6DE\nRlVV51XVX1fV16tq+cD+a6rqSOBNdEnc84eK/nG/Pbmq3llVN/Xlqqp+VVUfraqXT1DmxVX1g4G6\nbqmq71fVS6vqOxv1AuFv++0ocNY4x7cDntr/0r+6b88lVbUmSYB/6eM+VVUvrKrr+pjrq+pF3DZc\n8F/6HpPxrAIeVVU/6cuurqpjgCP643+dZDDZo6o+VVWvq6qzBtpVVfULukVFTqFL5J68nmtvrneG\nHN1vnzvB8b/qt8ePfc80vUyiJEnSlm7nfntDwxC9jenL/fZPhvaPJVy7rSd5GDZW5o63u1XrkWR+\nknslOZpuyXeA46rq2nHCf1xVX5/gVA8A7t4/P3KCmDf12z3peo3G876q+vU4+z8GXE73O+thE5T9\nHf334IT+5fDnMm31TqOP0fWIPiDJHwwe6OemPbF/6VC+TcQkSpIkaQOSLOpvSntakmv6BSKqXxhg\nrMdoeGW7b9D94rsfcFq6m/xuaPW7sblXH0vytiR/nGTeRrqMfx5o8yrgp8Bf98e+y229L8PW1/M1\nthDFtVX10/ECqupcbpt3td94MXTzwMYrOwqcPlHZJHdO8m/9gh83pruJ8Ng1/mcftr73fEr1bmr9\nPKgv9C+He6OeTjeM8ZdV9e1N2rCtmEmUJEna0o1NtN+xH162USW5I91NUN9Bt7DDrnRJyLV0CwOM\n3XT1t+beVNUvgRfQza/5U7pFJq5IclG/+t5v9Sj0XkE3R2Yx8E90CczyJN9M8oIki27Hpdzct/dq\n4Erg58Dn6Ia+/WlVjTcfCm5b4GA8u/bbK9YTA12vzmD8sPWVHzv2W2WTHER3Da+kS3S2p1u2fuwa\nx3r11jcnqrneGTQ2pO8ZSeYP7B8byvcRtMmYREmSpC3dz/vtArqV1Ta2d9ItrHAh3dC3nfob+O7W\nLwzwxxMVrKoPA3sBLwG+SJfw7Uk3f2ppktcMxV8PPBh4BPBuul6u+cDBdIsgnJPkzlO8jsGb7e5R\nVfeqqif199Nau55y6yZx7oVTbNOU9L1zx9LN1zqF7sbJi6pqh7FrBF42Fr4p2zaNTgEuohu+eih0\ny7MDf0j3GX105pq29TGJkiRJW7pv0S2rDf0vlxtL/xf/x/cvn1lVn6uqG4bCdl/fOarq6qp6V1U9\nga5X40HA5+l+uf+XJPcbiq+qOqWqXlxV+9Eth/53wK+Bu3LbMLXNwVgv1e9tIG4s8ZuoV2t9Q+7G\njg2WPaA/56+Bx1fV6VW1cqjcej+XKdY7Y/p5XmNznsaG9I31Qp1UVVdu+lZtvUyiJEnSFq2qLue2\nuUQvTDLevY5+xySH/u1C18MFt819GvbwydQHv0mQzgKewm0LFzx4A2VuqKoPAGO9VgetL34TO7vf\nbptk3EUjktwD2GMofti419R/Rg8Zp+xYUnZeVf3Ofat6k/lcWuudDqNj1U4i9iN0vU6PTHIXYGzZ\neBeU2MRMoiRJ0mzwOrp5SnemuzfQeoeXJfkLbhvutT43cVsv133HOc8dgRdOUMf88fYDVNU6uhvX\nQp+kJZmTZO562nLrYPxm4ofA+f3z10wQ88Z+ezFw5gQxL+hXmRv2LLrPdJRu/taYsXtl7T3eZ53k\nz+mGQG5Ia73TYWzu1njt+C1VdQXwNWCE7l5Yu9L1lE3H/dG0HiZRkiRpi1dVP6S7KWwBjwF+0K+G\nt9NYTJLtkxyW5FS6G5IunsR5b6JbuQ7gw0ke0J9rTpI/oxtKOFEPwr8mOT7JE4basXuSd9PNlSrg\n5P7QEuD8JK9Nct8kI0N1vaWPO2nD78im0Q8xe13/8vFJ3pNkZ4AkO/fX+fT++Ov6Ve/GsxA4sZ/j\nQ5J5SZ4DvK8//qGqunQg/v+AW+jmB32sT2bHVlH8K+Cz3LbgyPq01jsdxlY1PCzJ9pOIH1tgYmzp\n9mOras1EwZoe6/trhyRJ0hajqj6U5Hrg/cA+dKvhkWQFXbIymDRdAnxzkqd+KXAqXU/UD5LcTPeH\n6EV0c3L+ituWnx40l24hiif17VhOl3ANtuN1VXXOwOu70N1v6UhgTZKb6FadG+mPX8jketA2mao6\nLsl9gdcC/wAckWQZXbvH/mD/tqr6xHpOcwTwQeAnfdlFdAtqQJfE/tY1V9WNSV4NvItuaORT+nLb\n0r3vP6Qb4vbuDTS/qd5p8nHg5XTDOq9Lcg1dL+XlVTXeUM8TgF9x273EHMo3A+yJkiRJs0ZVfYFu\n8YW/p5sndTndL9Vz6YaTHQ88A7jnZO+pU1Xfo1vI4AvADcA84Bq6ZO0BwI8mKPqfwIvoVuU7jy6B\nWgBcRtcT9pCq+teB+OXAY+lWAzyTbpjWYrqlyc+iS1Ie0M8B26xU1euAP6O71uvoVs27nm6Y2cOr\n6tUbOMUZwB8Bn6YbllnAucAbgIdW1Ypx6nw33Y1wx3ql5gK/AP4ZOJBuKOaGNNe7sVXVL+hWYzyR\nbpjiHeiS6XFXYexXUhy7wfNZQ0m4NpHMzI29JUmSJE1FkvOAvYEXVNX7NhSvjc8kSpIkSdpC9PPj\nTqHrobxTVS3fQBFNA4fzSZIkSVuAJLsA/9G//LAJ1MyxJ0qSJEnajCV5O/AXdPOl5tHNO7t3VV0z\now3bitkTJUmSJG3edgF+j+5eYV8HHmYCNbPsiZIkSZKkBvZESZIkSVIDkyhJkiRJajB3qgUfMecp\njgOUpGl28uhnMtNtkCRJv82eKEmSJElqYBIlSZIkSQ2mPJxPkiRNjyQXAUuAi2e4KZI0m+0JLK+q\nvVoLmkRJkrT5WbJo0aKd9t13351muiGSNFv9/Oc/59Zbb51SWZMoSZI2Pxfvu+++Oy1dunSm2yFJ\ns9b+++/P2WefffFUyjonSpIkSZIamERJkiRJUgOTKEmSJElqYBIlSZIkSQ1MoiRJkiSpgUmUJEmS\nJDUwiZIkSZKkBiZRkiRJktTAJEqSJEmSGphESZIkSVIDkyhJkiRJamASJUmSJEkNTKIkSZIkqYFJ\nlCRJkiQ1MImSJEmSpAYmUZIkSZLUwCRKkiRJkhqYREmSJElSA5MoSZIkSWpgEiVJ2uiSHJNkxSTi\nTkty2iZokiRJG41JlCQJgCRHJKkk35vptkxVn7zVwGNtksuSfCrJvaa57m2SvDHJQ6ezHknSzJs7\n0w2QJG02nglcDDwoyd2r6vwZbs9UrQL+pn8+F7gb8HzgUUnuVVVXTlO92wD/3D8/bZrqkCRtBkyi\nJEkk2Qs4EDgMeD9dQvWmGW3U1K2tqmMHdyT5LvAV4DHAB2ekVZKkWcPhfJIk6JKmG4ATgOP7178l\nyZ79ELmXJ3lekguSrEpyVpIHbqiCJA9Icm0/D2q79cQtSPKmJOf3578syb8nWXA7ru+qfrt2qK67\nJvlMkl8nuSXJd5M8Zpw27ZbkQ0muTrIyyY+SPGfg+J7Atf3Lfx4YTvjG29FmSdJmyp4oSRJ0SdPn\nqmp1kk8CL0jywKo6a5zYZwCL6XqsCngl8Lkkd62qNeOdvE+yTgK+Dzy+qm6dIG4O8CXgwcAHgJ8D\n9wVeCtwDeMJkLibJLv3TEeCuwL8B19P1Ro3F7A6cQTcM79398ecAX0ry5Kr6fB+3iG543t2B9wIX\nAU8BjkmyQ1W9iy6BegHw38Dngc/11fx4Mu2VJG1ZTKIkaSuXZH9gH+CF/a7/BS6nS6zGS6J+H9i7\nqm7oy58LfBF4JANJysD5/wT4KnA68KSqWrWe5jwDeDhwUFX978A5zgHel+TAqjpjA5e0Lbf1Co25\nAvjzqhrc/ypgd+BPx+pK8kG6xOcdSb5YVaPA84B9gWdV1Sf6uPcB3wKOTPLhqropyfF0SdSPh4cT\nTiTJ0gkO7TOZ8pKkmeFwPknSM4GrgVMBqqqA44CnJRkZJ/64sQSqd3q/vetwYJKD6XqgvgEctoEE\nCroenp8Dv0iyy9gD+GZ//OBJXM9K4BH945HA3wErgK8mucdA3CHAmYPJWlWtoOsB2xO410DcVcAn\nB+LW0PVebQccNIk2SZJmEXuiJGkr1idJT6NLoPZKMnboe8A/An8GfH2o2KWDL6rqhr7cjkNxC+nm\nWC0F/qKq1rJhe9P1+gz3JI3ZbRLnWFdVpwzuSPJV4JfAW4En9bvvQnedw34+cPycfvvLvldqorgp\nqar9x9vf91DtN9XzSpKml0mUJG3dHgbckS6Reto4x5/J7yZR6yY4V4Zer6Ibxvd44FGMM9RvHHOA\nnwAvm+D4ZZM4x++oqsv7YYcPmUp5SZIGmURJ0tbtmcA1wN+Pc+ww4IlJnj/RQhAbUP35vwh8Jsmj\nq+q0DZS5ALg/8I1+WOHGNJdu+N2YS4B7jhO3z8Dxse39kswZ6o0ajtvY7ZUkbaacEyVJW6l+1bnD\ngK9U1fHDD7qV6BYDh061jqpa3ddxFvDlJA/aQJFPA3sAfztee5NsO5V29HOh7gn8aGD3V+luLHzA\nQNy2dAtJXAz8bCDuDsBTB+Lm0i3EsYJugQmAW/rtDlNpoyRpy2FPlCRtvQ6lS5K+NMHx79LNTXom\n3UITU1JVtyZ5LN3iEF9LclBVnTNB+MeBv6Bbie9g4P/olinfp9//SLpl0tdnbpJn9c/n0C0S8fz+\n+eANhN8GPL1v07uBX9Mtcb4X3SqCY71OH6BbnOKYfiXDi4EnA38CvKSqbhq4zp8BT01yXn++c9Zz\nrZKkLZRJlCRtvZ5Jt5LdyeMdrKrRJCcAz0yy8+2pqKqWJ3kk8G3g5CR/WlXnT1DnE+juC/WXwBPp\nenguBN4FnDeJ6hbQJWNjltP1hD27qr4xUNfVSQ6ku4fUC+kWwvgx8LiqOmEg7tYkD6VLup4DLAHO\nBZ5bVccM1f03wHuA/wTm0yVtJlGSNMtkqkPOHzHnKY79lqRpdvLoZ4YXa9BWIMnS/fbbb7+lSye6\njZQk6fbaf//9Ofvss8+eaKXU9XFOlCRJkiQ1MImSJEmSpAYmUZIkSZLUwCRKkiRJkhqYREmSJElS\nA5MoSZIkSWpgEiVJkiRJDUyiJEmSJKmBSZQkSZIkNTCJkiRJkqQGc2e6AbNdHXj/5jIXPWGbpvht\nL09zHWsXtcWn2uJrCun5aOO3ce4tbfFrtmuLB5izpjF+XXsdIyvby7TI6BQKTaVMoxppi597a9uX\ncOejv9NWgSRJ0iSZREmStBk654pl7PmqE2a6GZI0rS5+22NmuglT4nA+SZIkSWpgEiVJkiRJDUyi\nJEmSJKmBSZQkSZIkNTCJkiRJkqQGJlGSJEmS1MAkSpK0VUhyeJIaelyT5NQkj57p9kmSthzeJ0qS\ntLV5A3AREGB34HDgq0keV1VfmcmGSZK2DCZRkqStzdeq6vtjL5J8CLgaeDpgEiVJ2iCH80mStnY3\nArcCa8d2JHl5kjOSXJ/k1iRLkzx5uGCSRUneneS6JDcl+VKSPfqhgm/chNcgSdqEpt4TlbSFj4w0\nV1Fr1244aMDqR/5hcx2XPL2a4nf47vym+Hr0DU3xAGuvm9cUv2zJFHLheaNt8Wsa60jb+zoVcxev\naYrfY5cbm+u49Fc7NcXvtuvy5jquu3G7pvh1K9q+H8yZwmfR9uNNRtrrqLVtlcxZuK4pftk9DmiK\nB9jtflc3xV/zw92b67jrq77TXEYb3fZJdqH7pu8GvBDYDjh2IObFwJeATwDzgacBn0ny2Ko6YSDu\nGOAvgI8D3wUOAgaPr1eSpRMc2mey55AkbXoO55MkbW1OGXq9Cvirqjp5YN89qurWsRdJ3gucDbyM\nPklKsh9dAvXOqnppH3pUko8A95+uxkuSZp5JlCRpa/P3wHn9892BZwFHJ7mpqj4HMJRA7QiMAKfT\nzZsa86h+e9TQ+d9Dt1jFBlXV/uPt73uo9pvMOSRJm55JlCRpa3Pm0MISnwR+ALw3yVeqanWSxwKv\nAx4ALBgoOzh29S7AKN1Kf4POn55mS5I2Fy4sIUnaqlXVKHAqcEdg7yR/SjcfaiVwBHAI8Ajgf2ie\nMShJmo3siZIk6bb/D7cDnkSXQD2yqlaNBSR57lCZS+j+GLkX8MuB/XefxnZKkjYD9kRJkrZqSeYB\nfw6sBn4OrKMbtjcyELMn8IShoif12yOG9r9wOtopSdp82BMlSdraPDrJ2BLiuwHPAPYG3lZVy5Oc\nQLcK34lJ/qeP+Xu6uU73GztJVS1N8lngJUl25rYlzu8xFrJJrkaStMmZREmStjZvHni+EvgF8ALg\n/QBV9c0kfw28Cngn3cIR/wTsyUAS1ftL4Cq6VfueSLd8+lOBc/tzS5JmIZMoSdJWoaqOobs57mRi\nPwx8eJxDbxyKuwX4h/4BQJIH9E8vn0IzJUlbAOdESZI0RUkWjbP7JXRLn397EzdHkrSJ2BMlSdLU\nvTLJ/nRLpK8FHt0/PlBVl81oyyRJ02bKSVRGRjYcNKDWrp1qVZO2bK95zWXe8+CPNMV/7d73b4r/\nwfV7NMUDrFjU9l4t3Gl1cx03L1/YFN88O3rNFDo5GytZu7Lt63vTqvltFQC1su17vmLlgg0HDRld\n3VZH5o+2xY+0xQPMGWn7MObNb//5Xjh/TVP8LSvbPr81d2qfjvKau3+1Kf6Fv3xOcx2adc6gu4fU\n6+mWR7+UbsjfW2awTZKkaWZPlCRJU1RVJwMnz3Q7JEmblkmUJEmbofvssT1L3/aYmW6GJGkcLiwh\nSZIkSQ1MoiRJkiSpgUmUJEmSJDUwiZIkSZKkBiZRkiRJktTAJEqSJEmSGphESZIkSVIDkyhJkiRJ\namASJUmSJEkNTKIkSZIkqcHcKZccGWmLX7u2uYoL/+2ApvhXHPrF5jouW71zU/zFN+/UFL9yTftb\nvGTxLU3xi+a1v7dJtcU3nn/1FK57dLStlrlz1zXFL16wuike4OYdV7bVsagtHmDt2ra/ZcydO9oU\nP2+k7X0CmDvSVsfcKdRR1fZ577jTsqb4K2/Yvike4KQb79sU/8FDjm6u418/f3hzGUmStHmxJ0qS\nJEmSGphESZK2Okn2TFJJXj7TbZEkbXlMoiRJ0yLJfZMcn+SSJCuTXJHk5CQvnOm2SZJ0e5hESZI2\nuiQHAt8H7g98EPgH4GhgFHjxDDZNkqTbbeoLS0iSNLHXAsuAB1bVjYMHkuw2M03atJLMBeZUVfuq\nNpKkzZo9UZKk6XA34KfDCRRAVV0z9ryfl/TeJE9Ick6SVUl+muRRw+WS7JHkw0muHoj7q6GY+Une\nnGRpkmVJbk5yepKDN9TgdD6QZHWSwwb275DknUku6+s9P8k/JZkzEPObOVZJXpLkAmAVcK9Jv2OS\npC2GPVGSpOlwCXBAkvtU1TkbiH0wcBhwFHAT8CLgs0l+v6quB0iyO/BdoID3AtcCjwY+lGRJVb2z\nP9cS4G+AT9INI1wM/DVwUpIHVdUPx2tAkhHgw8BTgSdW1Qn9/m2AbwF7AO8HLgUOBN4K3BF4ydCp\nngssBD5Al0T9en0XnmTpBIf2WV85SdLMMomSJE2HtwNfA36Y5EzgdOAbwKlVtWYodl/gXlV1AUCS\nU4EfAU+nS5gA3gKMAPcdS6yA9yX5JPDGJO+vqluBG4A9B4fQJfkg8AvghXQJ1W/ph90dCxwKHFpV\nXx84/DK6XrU/qKpf9vven+RK4BVJ/l9VXTYQf2fg7lV17eTeJknSlsjhfJKkja6qTgYOAL5Et7jE\nK4GTgCuSHDoUfspYAtWX/TGwHLgrdMPsgCcBX+5f7jL26M+5PbBfX3bdWAKVZE6Snej+YPj9sZgh\n84HPAI8FDhlKoACeQpcA3jBU7yl0Sd1DhuI/25JAVdX+4z3okj5J0mbKnihJ0rSoqrOAw5LMp0uk\nngi8FDg+yQOq6md96KXjFL8B2LF/viuwA/C8/jGe3yxWkeQ5wD/SDYmbNxBz0TjlXg1sBzy6qk4b\n5/jewP3ohg+ut9711CFJmmVMoiRJ06rvGToLOCvJecBH6Hp43tSHrJugaPrt2KiJY4GPThD7Y4Ak\nzwKOAb4A/AdwTX/+V9MNyxt2EvAo4JVJTquqlUPH5wAnA/8+Qb3nDb2+dYI4SdIsMuUkqlataoof\nWbKkuY5XHPrFpvj/Pm94VMWGPezOw///rd9OC25uir/khh03HDRkyaLh/8PXb11lw0FDtpk/PCVh\n/UbmjDbF39oYPxUjc6opfsHI2uY60vjWLpnf9nMBsGph249hNX7ec0faP4v5c9veq3lT+LxbP49r\nVmzXFL/T4rafVYBLb96pKf6y7XZuruPCp4w0l5llvt9v79hQ5lq6BSdGquqUDcQ+GbgQOKyqfvOP\nRJI3TRD/XeB9wFeAzyR5YlUNfjkvALabRL2SpK2Ic6IkSRtdkoP7uUzDDum35072XFW1Dvgs8KQk\n9xmnrl0HXo71amXg+B/Rzc+a6PynAE+j65H6+ODS5cCn6VYZfOQ49e7QL0ohSdrK+I+/JGk6vAfY\nJsnn6RZJmE+3NPhTgYvphvS1eBVwMPC9frW9nwE70S0W8fD+OXQ9SocBn09yArAX8Pw+fsLuzKr6\nQpLnAh+jW9Ti7/pD/0G3at9XkhwDLAW2Be5L1+u1J3Bd47VIkrZwJlGSpOnwcrp5T4fQLQYxn24B\niaOAI8e7Ce/6VNXVSR4EvIEuSToCuB74KfBPA6HHAHegS4IeSZc8Patvy0M3UMexSRYDRyVZXlWv\nqKpbkhwEvKY/x1/SJVnnAf8MLGu5DknS7GASJUna6KrqRODEScSNO8mvqvYcZ981wD/0j4nOV3Q3\nwn3r0KEThuIuZmDI38D+/wb+e2jfCrok6jXrqXfc80mSZifnREmSJElSA5MoSZIkSWpgEiVJkiRJ\nDUyiJEmSJKmBSZQkSZIkNTCJkiRJkqQGJlGSJEmS1MAkSpIkSZIabLKb7V7wins3l1mx7vKm+LX/\nt1NzHTy1LXzlunlN8QvnrW2rAJg3Z7QpfnT8e1Wu1+p1I9NaR1JN8QDzRtquu9W281Y1lxlpbNPi\n+Sub61i+ekFT/NrGz27uyLqmeGj/Do40xgOsrba/4ey23Yqm+JvXzG+KB1i2emFT/Fevu29zHTsv\nbfv8JEnS5seeKEmSJElqYBIlSZIkSQ1MoiRJkiSpgUmUJEmSJDUwiZIkSZKkBiZRkiRJktTAJEqS\nJEmSGphESZK2KkkqyXsnEXd4H7vn9LdKkrQlMYmSJM0aSe6b5PgklyRZmeSKJCcneeEmqPs1SZ4w\n3fVIkmaeSZQkaVZIciDwfeD+wAeBfwCOBkaBF0/hlB8HFgGXTDL+NYBJlCRtBebOdAMkSdpIXgss\nAx5YVTcOHkiyW+vJqmodsG59MUkCLKyqW1vPL0nacm2yJCr3XNFc5icr9mir48AbmutYMGdtU/yO\n829pij9/dJemeIAFI21tWrhjaY0AACAASURBVLZqYXMdq9eONMWPzKmm+NHR9k7OdWmLH5kz2hS/\n/fyVbRUA8+e2fRYLGz87gFVr2n4Mk7bPYipWrWv8fkzh857TeB1z5rbFzxtZ7+++G8Wzdv9Oc5n/\nOvr6tgIfaK5ia3Y34KfDCRRAVV0zvK8fencksDdwPvCPVXXiwPHDgY8Ae1XVxf2+i4FzgPcAbwHu\nA7wqyX/2xZ6T5Dn9849W1eEb48IkSZsXe6IkSbPFJcABSe5TVedsIPbBwGHAUcBNwIuAzyb5/ara\nUKZ7T+CTwPvphg2eCzybbujgmdyW+l6woQYnWTrBoX02VFaSNHNMoiRJs8Xbga8BP0xyJnA68A3g\n1KpaMxS7L3CvqroAIMmpwI+ApwMbWrnv7sCjquqkwZ1J3gdcWFXH3u4rkSRt1kyiJEmzQlWdnOQA\n4NXAI4EDgFcC1yb5m6r60kD4KWMJVF/2x0mWA3edRFUXDSdQt6PN+4+3v++h2m9j1CFJ2vhcnU+S\nNGtU1VlVdRiwI/Ag4K3AYuD4JPcaCL10nOI39OU25KLb3VBJ0hbNJEqSNOtU1eo+oXoN8AJgHvCU\ngZCJVh6ZzBI3rsQnSVs5kyhJ0mz3/X57x2muZ/qXzpQkbRZMoiRJs0KSg/v7Ng07pN+eO81NuBnY\nYZrrkCRtBlxYQpI0W7wH2CbJ54FfAPOBA4GnAhfT3fNpOi0FHp7kZcCVdAtQfG+a65QkzQCTKEnS\nbPFyunlPhwDPo0uiLqW7F9SR492EdyN7Gd09oo4EFgEfBUyiJGkWMomSJM0KVXUicOIk4sZdPKKq\n9hx6fQxwzPpiho6dCxy0wYZKkrZ4zomSJEmSpAabrCdq3lmLm8s89w9Pb4q/8MjnNdfxyUP+uCn+\ndQd/acNBA069cO+meIBdt725KX7daHsunLQtIjUyZ7Qpfu7IRKsHT2xOY5tarRkdaS6zzfw1TfGj\n4/+Be6OaN9L2WUzFdH8WUzFvTtt3asXq+c117LpN28/eu5/3tOY6Rji7uYwkSdq82BMlSZIkSQ1M\noiRJkiSpgUmUJEmSJDUwiZIkSZKkBiZRkiRJktTAJEqSJEmSGphESZIkSVIDkyhJkiRJamASJUmS\nJEkNTKIkSZIkqYFJlCRJkiQ1MImSJEmSpAZzN1VFq/9wRXOZd1z2yKb4hV85s7mOw9+yuin+1Bv2\naYqfO3e0KR5gbbXltgvmrm2uo9Wc1LTGA4xWprWOOZnCZzHa9lnstvCm5jouX7BDU/y6xvdpZAqf\nRet7mynUUY3X0fr9mDen/fNeOLKmKX71jaua62h/pyRJ0ubGnihJkiRJamASJUmSJEkNTKIkSRqS\n5PAklWTPKZQ9JsnFG71RkqTNhkmUJGmzkOS+SY5PckmSlUmuSHJykhfOdNskSRpkEiVJmnFJDgS+\nD9wf+CDwD8DRwCjw4hlsmiRJv2OTrc4nSdJ6vBZYBjywqm4cPJBkt5lpkiRJ47MnSpK0Obgb8NPh\nBAqgqq4Ze57kuUm+meSaJKuS/CzJC4bLJLk4yVeSPDjJmf3wwAuT/OU4sffuz3lrksuTvI5x/n9M\n8vgkJyS5sq/7giSvTzJy+y9fkrQlsSdKkrQ5uAQ4IMl9quqc9cS9APgp8CVgLfA44Kgkc6rqv4Zi\n7w4cD3wI+CjwV8AxSZZW1U8BktwBOJXu/8O3ATcDzwNuHafuw4EVwDv67cOANwNLgFe0XnBf/9IJ\nDrXdlFCStEmZREmSNgdvB74G/DDJmcDpwDeAU6tq8C7IB1XVYILz3iQnAi8DhpOoewIPqarTAZJ8\nGrgMeC7w8j7mn4BdgT+qqjP7uI8Cvxynjc8Yqvt9Sd4HHJHkdVXVfvdlSdIWyeF8kqQZV1UnAwfQ\n9TDdH3glcBJwRZJDB+J+k8Qk2T7JLsC3gLsm2X7otD8bS6D6stcC5wJ3HYg5BPjuWAI1EPeJcdo4\nWPfivu7TgW2YYs9RVe0/3gP4xVTOJ0naNEyiJEmbhao6q6oOA3YEHgS8FVgMHJ/kXgBJ/iTJKUlu\nBm4ErgX+tT/FcBJ16TjV3NCff8xdGL/X6dzhHf3cqc8nWQYs7+s+doK6JUmz2JSH8+UP7t0U/4x9\nvt9cxxn3n99cptWjl/yoKf471z+hKX7e3HVN8QDz5rSVGR1Jcx1zUs1lplvrzOzRarvueRltrAFG\n5rSVuWrlkmmvY07jdbeeH9q/H3OY/u/T3Nb3aQrf8cVz20ZjXXnX7Zrr2PYHzUW2OlW1GjgLOCvJ\necBHgKckOZZuiN8v6IbvXQasputNeim/+4fBif4xbf5HM8kOdD1ey4E3ABcAK4H9gH8bp25J0izm\nnChJ0uZs7C9wd6RbRGIBcGhV/aaXKcnBt+P8lwB7j7P/nkOvHwrsDBxWVd8eqHuv21G3JGkL5V/O\nJEkzLsnBScbrITqk357LbT1Lv4nr50E993ZU/VXgj5M8aOCcuwLPHIobr+75wBG3o25J0hbKnihJ\n0ubgPcA2ST5PN1xvPnAg8FTgYrohfbvTDd/7cpL3A9sBfwtcQ9dTNRX/DjwbODHJu7htifNLgPsN\nxJ1BN5/qo0neDVRfrn08tSRpi2dPlCRpc/Byuvs1HUJ3H6Z30C0ucRTd8uM3VtW5wJPpEpi3A88H\nPgC8a6qVVtWvgIOBHwOvAl4CfGz4nFV1PfBY4FfAkX17T6ZbRVCStJWxJ0qSNOOq6kTgxEnEfRn4\n8jiHPjIUt+cE5R86zr6f0M15Gvbhobgz6JZhH5ahuMPHq1uSNHvYEyVJkiRJDUyiJEmSJKmBSZQk\nSZIkNTCJkiRJkqQGJlGSJEmS1MAkSpIkSZIaTHmJ8xV3264p/gc3/t4Uarl6CmXajFZbHrlqXdtb\ntmj+mqb4qZiT2iRlpttotd2zshrjt527qike4Lplbd/zZ//+95rr+Db3aIq/cNnOTfHbzl/dFA/t\nn8Wm+D61tmntaPvfiFaNtv18r5vvfVYlSdoa2RMlSZIkSQ1MoiRJkiSpgUmUJEmSJDUwiZIkSZKk\nBiZRkiRJktTAJEqSJEmSGphESZIkSVIDkyhJkiRJamASJUmSJEkNTKIkSZIkqYFJlCRpVkpytyTv\nT3JhkpVJlif5vyQvTrJomup8RpKXTMe5JUmbj7kz3QBJkja2JI8BPgOsAj4GnAPMBx4M/Adwb+B5\n01D1M4D7AO+chnNLkjYTU06iRlZWU/w2c1c313Frc4l2K2teU/y60bbOu/kj65rip6K1TQDMGW0K\nH600xa9ZN9IUD7Bo3pqm+HvueFVT/Heu2qspHuDZ+57ZFP/eXzy0uY6773xdU/ye2/+6Kf6S5Ts2\nxQMsnLu2KX51tX/erd/bkcbv7NzGeIA5aSuzdlHbz4U2jSR7AZ8CLgEeVlW/Gjj8X0nuDjxmRhon\nSZoVHM4nSZptXglsB/z1UAIFQFWdX1XvAkgyN8nrk1yQZFWSi5P8a5IFg2WSPD7JCUmu7OMu6MuN\nDMScRpec3SVJ9Y+Lp/NCJUkzw+F8kqTZ5nHAhVV1xiRijwaeAxwP/D/gj4BXA/sCTxyIOxxYAbyj\n3z4MeDOwBHhFH/MWYHvgzsBL+30r1ld5kqUTHNpnEm2XJM0QkyhJ0qyRZAmwB/DFScTeny6BOrqq\n/rbffVSSa4CXJzm4qk7t9z+jqgZHmb8vyfuAI5K8rqpWVdXJSa4AdqyqYzfeVUmSNjcO55MkzSZL\n+u1Nk4g9pN++Y2j//+u3v5k3NZhAJVmcZBfgdGAbbkevUVXtP94D+MVUzylJmn72REmSZpPl/Xbx\nJGLvAowC5w/urKqrktzYHwcgyb2BI+mG8S3ht20/5dZKkrZIJlGSpFmjqpYnuZJumfFJF1vfwSQ7\nAN+iS9DeAFwArAT2A/4NR3VI0lbHJEqSNNt8BXhekgOq6jvribuELgHaG/j52M4kuwM79McBHgrs\nDBxWVd8eiBvvvglt9/+QJG2R/OuZJGm2+XfgZuDoPiH6LUnuluTFwFf7XS8ZCnlZvz2h347d8O83\nNwZLMh84Ypy6b8bhfZI069kTJUmaVarqgiTPAI4Dfp7kY8A5wHzgQOApwDFV9a4kH6XrtRobsvcg\nuhX7vjCwMt8ZwA3AR5O8m6636dkMJFUDlgJPTfIO4CxgRVV9ebquVZI0M0yiJEmzTlV9Kcn96O7h\n9HjgBcAq4MfAPwIf7EP/BriQ7j5QTwSuAt4KvGngXNcneSzdqn1H0iVUxwLfAE4aqvoo4AHAc+nu\nFXUJYBIlSbOMSZQkaVaqql8Cz9tAzFq6m+a+eQNxZwAHjHMoQ3E3A89sa6kkaUsz5SRq4VfObIrf\n9vXbNtdxwzbbNMWP3nJLcx1fX96ygBPMG1m34aABq9a2v8Xrqm2qWtI+j3m0xhuFMrG5c0ab4tes\nG2mKBzho1182xX/l3x/aFL/zuTc3xQPM+3Db5z3y7fapEKuPvaopfsmX1zbF77jw1g0HDVm1ru17\nO2cK38HVjd+R1m9U63ccYNHImqb4KVQhSZJmAReWkCRJkqQGJlGSJEmS1MAkSpIkSZIamERJkiRJ\nUgOTKEmSJElqYBIlSZIkSQ1MoiRJkiSpgUmUJEmSJDUwiZIkSZKkBiZRkiRJktTAJEqSJEmSGphE\nSZIkSVKDuZuqom99437NZfb68mVtBf7sluY6Trxs36b4u+90XVP8VTcvaYoHWLNupCl+wcja5jpG\n5ow2xa8bbcu3t5u/qike4MpVOzTFb3/sd5viL3nDgU3xAMddtF9T/E1/sLK5jnXvvL4p/kdXt31n\n97vD5U3xAL+8cdem+EXz1jTXsbjxO7Kupv9vPjevXdAUv+Sy9uuWJElbPnuiJEmSJKmBSZQkSZIk\nNTCJkiRJkqQGJlGSJEmS1MAkSpK0RUhyeJIaeKxMcmWSk5K8KMnimW6jJGnrsMlW55MkaSN5A3AR\nMA+4A/BQ4J3Ay5IcWlU/nsG2SZK2AiZRkqQtzdeq6vsDr9+a5GHAV4AvJdm3qm4dr2CSbavq5k3S\nSknSrOVwPknSFq+qvgn8C3AX4FkASY5JsiLJ3ZJ8NclNwCfGyiT5oyQnJlmW5JYk30ryJ4PnTbI4\nyTuTXJxkVZJrkpycZL+BmL2TfDbJVf0Qw8uTfCrJ9pvm6iVJm5o9UZKk2eLjwL8Cfw58sN83FzgJ\n+F/g5cAtAH3P1deApcCbgFHgucA3k/xpVZ3Zl38f8GTgvcDPgJ2BBwP7Amcnmd+ffwHwHuAqYA/g\nscAOwLL1NTjJ0gkO7dNw3ZKkTcwkSpI0K1TV5UmWAXcb2L0A+ExVvXpsR5LQJUenAo+uqur3vx/4\nKXAkXSIG8Bjgg1X1jwPn/PeB5/cC9gKeUlXHD+x/88a5KknS5sgkSpI0m6wAhlfp+++h1w8A9qZL\nlnbucqrf+Abw7CRzqmoUuBH4oyR3qqorx6lvrKfpkUm+WlW3tDS2qvYfb3/fQ7XfeMckSTNvkyVR\ne73mO81l7nd2Nhw04JQjDmyu404vurwpfvGnVjXFn79qflM8wPaLVjbFzxtZ11zHmnUjzWVarFw7\nr7nMvtv8qin+uy84pCn+Ll+7qSke4MTnH9cU/4C3HdFcx9qHjfs71ITecK/PNMW/48KHN8UDLJ7f\n9j1fta79n5LRtP18rxttm8K5zbzVTfEAV9zcNoVl/te/v+EgbWrbAdcMvF4LDP9Dv3e//eh6zrM9\ncAPwyj7usj6x+Srwsaq6EKCqLkryDuBlwDOTnA58CTi2qtY7lE+StOVyYQlJ0qyQ5M50yc/5A7tX\n9T1Kg8b+73sF8IgJHisAqurTwF2BFwJX9mV+muTRYyfrh/rdj24+1iLg3X3MnTfm9UmSNh8O55Mk\nzRbP7rcnbSDugn67vKpO2dBJq+pXwFHAUUl2A84GXku3MMVYzE+AnwBHJjkQ+D/g+cDrmq5AkrRF\nsCdKkrTF61fbez3dTXg/sYHwpXSJ1MuTbDfOuXbttyPDy5RX1TV0PVIL+pglSYb/IPkTutX+Fkzh\nUiRJWwB7oiRJW5pHJ9mH7v+w3YGH0Q3BuwQ4tKrWO7G0qkaT/A1dT9JPk3wEuIJuafKDgeXA4+gW\nqLg8yfHAj+iG+D0ceCAwtlrfw4D3JvkMcF7fpmcD64DPbrQrliRtVkyiJElbmrHlw1cDv6br+XkJ\n8JGqmtQKMlV1WpID6Hqv/oFuQYqrgO8B7+/DbqEbxvfnwGF0ozfOB46oqrEV/35EN3zwcXRJ2C39\nvkdX1XdvxzVKkjZjJlGSpC1CVR0DHNMQfzhw+HqO/xB40nqOr6Zbne+V64m5CPjrybZJkjQ7OCdK\nkiRJkhqYREmSJElSA5MoSZIkSWpgEiVJkiRJDUyiJEmSJKnB1FfnmzPSFj+6rrmKH+9XTfHLPrbe\nW4OMa8GyOzXF32fBpU3xd1g8qdV2f8vCkTVN8fNH2t/bOfPa3tu11ZZvz5/T3qaLV+7cFP+PL/l0\nU/x3lt+9KR5gry8+ryn+Lo+9srmOD/zjsU3xR/7q0U3xd9x2eVM8wC1r5zfFL56/qrmO5asWNpdp\nscc2y5rLnPk/92+KvwOXNNeRuS6KKknSls6eKEmSJElqYBIlSZIkSQ1MoiRJkiSpgUmUJEmSJDUw\niZIkSZKkBiZRkiRJktTAJEqSJEmSGphESZIkSVIDkyhJkiRJamASJUmSJEkNTKIkSVuMJKclOW3g\n9Z5JKsnhM9cqSdLWxiRKkjRtkhzeJzljj5VJzkvy3iS7z3T7JEmairlTLlmjTeGZN7+9ijWrm+L3\n/suzm+s47+g/bIrfd9GVTfHzsq4pHuBXK7dvit9lwYrmOhbOWdMUv2Ltgqb4BXPWNsUD3DLa9h05\ne8VdmuLnzmn/LFjYVma00lzFm648pCl+JNUUP3dO28/qVMu0usO2y6f1/Gd84f7NZe78rjOmoSW/\nrda2/2zMEm8ALgIWAg8GXgAckuQ+VXXLjLZMkqRGU0+iJEmavK9V1ff750cnuR54GfB44JMz16zp\nlWTbqrp5ptshSdq4HM4nSZoJ3+y3eyV5Y/K7XawDQwH3bD15koclOT3JzUluTPLFJPsOHH9yf+6D\nxin7d/2x+wzs2yfJ8Ul+3Q9J/H6SQydo70FJjkpyDXB5a9slSZs/e6IkSTPhbv32euCOG/PESR4O\nfA24EHgjsAh4IfB/SfarqouBE4AVwF8A3xo6xVOBn1bVOf357g38H3AF8Dbg5r7cF5I8qao+P1T+\nKOBa4M3Athto69IJDu2zwQuVJM0YkyhJ0qawfZJd6OZE/QndHKlbga8Af7uR6/oP4NfAAVX1a4Ak\nXwB+ALwJeE5V3Zrky8CTk7yoqtb1cXcADqJLvsa8C7gUeGBVrerjjgL+F/g3YDiJ+jXwZ2PnlCTN\nPiZRkqRN4ZSh15cAz6yqK5L2BVkmkuSOwAOAfx9LoACq6sdJTgYGV3I5Dng68FDgG/2+J9MNdT+u\nP99OwMPokr7FSRYPlD8JeFOSParqioH9H5xsAlVV+09wHUuB/SZzDknSpmcSJUnaFP4eOA9YC1wN\nnFvVuMzr5Iwt23nuOMd+DjxyYLGHE4FldMP3xpKopwI/rKrz+td3BwL8S/8Yz250Q/3GXDT15kuS\ntgQmUZKkTeHMgdX5hk20bv/IdDUGoKpW9cP8npjkCGB3uqGGrxkIG1uA6e10PU/jOX/o9a0btaGS\npM2OSZQkaabdAJBkh6q6cWB/283gOpf023uOc2wf4LqhJcePA54D/BmwL12v03EDxy/st2uqanhI\noiRpK+US55KkmXZBv33I2I4k29IlN02q6lfAD4HnJNlh4Hz3Af4c+OpQkVPoFoJ4av84s6p+Mxyv\nqq4BTgP+rp9v9VuS7NraRknSls+eKEnSTPs63ep3H0ryH8A64K/olgn//Smc7xV0S5x/J8mHuG2J\n82X89qp7VNWaJJ8Dnka3HPnLxznf39OtxPeTJB+k653aHTgAuDNw/ym0UZK0BbMnSpI0o6pqDfBE\nuh6pfwFeBBwNvHeK5zsFeBTdPajeTJcYfRf4k8FepgHHAdv1zz89zvl+Bvwh3b2lDgf+C3g+MNqf\nX5K0lZl6T1RNNA94gvC1a5qrmLNwYVP86MqVzXXsuHReU/yn7vrApvjLb9xhw0FDdtimbU7y5fPa\n69h+QVsda0fb8u1b1s5vigeoalvmeGRO28Jeh+7+o6Z4gK/Mv29TfGubAA7c/oINBw24dNXOTfHz\n5rTfqub61dttOGjA0uvu3FzHtb9uu7/qPd60vCn+zued0RSv6VFVxwDHTCLubOCPxzl0zFDcQ4de\nX0w3j2n4fN/gthX3NlT3KeOdYyjmQjYwvHCy1ypJ2vLZEyVJkiRJDUyiJEmSJKmBSZQkSZIkNTCJ\nkiRJkqQGJlGSJEmS1MAkSpIkSZIamERJkiRJUgOTKEmSJElqYBIlSZIkSQ1MoiRJkiSpgUmUJEmS\nJDUwiZIkSZKkBnOnXDJpi69qrqLWrm0u02q7K9c1xV+zYrum+D22X9YUD7Bizfym+KtuWtxcx61r\n5zXFzxtpe58Wz1vVFA+w68IVTfF33+aapvjPPu8RTfEAb//IZ5riX3/Ooc11fHG/32srcI89m8Ln\nLL+l7fzAul9d3RS/ZNUFzXUsaYxv+wZC5rb/81ajjf9O1WhzHVP5t1CSJG1e7ImSJEmSpAYmUZIk\nSZLUwCRKkiRJkhqYREmSJElSA5MoSZIkSWpgEiVJmnZJapKPh850WyVJ2pCpL3EuSdLkPXvo9V8C\njxhn/883TXMkSZq6/9/enUdJVpZ3HP/+hi0IRDYJ6hARFCY5h2AYlUlQRAiIIEgUYk7QiCZxicpR\nMYnHLKJijDkRccclSAQ3QIxLEBSRCCKJDIvbwADKFiTIMmGRGYF58se97SmL6uV2V1d34fdzzj13\n6r3vfd+33n67p556732vQZQkad5V1am9r5OsAPbvT59Mkk2BtVXj96CtJI+oqu4PbJMkLVpezidJ\nWlSSHNhe2ve8JO9McjNwL7BJe/yJSc5MsibJz5JclOSAvjJe0Zax/SRlr+hJ+60k/57kf5OsTXJj\nkk8m2azv3JcmuSzJfUluT3Jqkkf35bk4ySVJViS5MMl9wD8MuYskSQvMmShJ0mL1NuBnwD8DmwEP\nJlkKXETz/9d7gTXAS4GzkhxaVWd1qaCd4fpq+/IE4FZgB+BQYHOa4I0kbwPeBHwa+DCwPXA0sGeS\n362qe3qK/Q3gy8ApwCeA/5mi/pWTHFrW5X1IkkbLIEqStFgtAZ5WVesmEpL8LbANsGdVfadN+1fg\nB8DxQKcgCtgdWAocUlVf7kk/tqfOXWgCqL+qquN70r8IfAd4WVv3hKXAUVX1bx3bIkkaE7MPokZw\nWXo98MC817HpF/67U/7Nv7lVp/xrnrVrp/wAdy7rdpXlukc92LmOn2/X7Ue/ZMn6TvlvXbJ5p/wA\nt2262fSZelxz17ad8m984eWd8gO857r9OuXf8Otbdq5j/dq13U747pXdyu9W+shkk026nfBgt3E+\nir8fmncf7w2gWgcBF0wEUABV9X9JPga8OckTquqaDnWsafcHJjm3qgb9Qj4fKOBzSXr/8NwAXAc8\nk18Oou4GZnSvV1UtH5TezlDtMZMyJEmj5z1RkqTF6se9L5IsobnU7qoBeSdW9Xtclwqq6krgg8Cr\ngNuTfCXJK5Ns0ZPticAGNAHTT/u2nYDt+oq9saq6f7slSRobXs4nSVqs7pvDuZNdLrHBQzJWvSrJ\nR2nugzoA+ADwN0lWVNUtNF843k8zCzbIXX2v59JuSdIYMIiSJI2Fqlqf5EZg0HXSEwsxXN/u72z3\nWwK39OQbOFNVVZcDlwNvTbIv8HXgz4HjgGuBjYDVVXXDnN6EJOlhwcv5JEnj5Czg6Ul+cb9Qkl+n\nCXiu6rkf6tp2v3dPvo2Av+gtLMkjk/TPTl3R7idu3DuDZmbrzf2NSbIkydazfC+SpDHlTJQkaZy8\nHTgcODfJe2kupXsJ8BjgkIlMVbUyyWXAvyT5jTbfkUD/vUrPBv45yenA1TSB058C64Az27JWJXkr\n7cIVwJdolj7fCXge8G7g/fPzdiVJi5FBlCRpbFTVTUn2At4JvA7YmOYyvIOq6py+7H8MnAj8LXAH\n8BGaJcl7lzJfSXPp3h8Cj6YJji4DnlVVl/XUe2ySVTTPhjqWZmbqxrasrwz3XUqSFjuDKEnSyFXV\nq4FXT3LsbCBTnLuaJuiZro7VwL4DDqUnz9U0M1nTqqrPAp+dJs+KmZQlSRpv3hMlSZIkSR0YREmS\nJElSBwZRkiRJktSBQZQkSZIkdeDCEh09eOed02fqscVnLu5cxxadz9B82Xj/66fP1GM7uuX/VVbr\n1i10EyRJkmbFmShJkiRJ6sAgSpIkSZI6MIiSJEmSpA4MoiRJkiSpA4MoSZIkSerAIEqSJEmSOjCI\nkiRJkqQODKIkSZIkqQODKEmSJEnqwCBKkiRJkjowiJIkSZKkDgyiJEkjleTUJGtmkG/DJJXk70bR\nLkmSZsogSpIEQBuwzGTbZ6HbOkiS5yT5h2nyvCfJd9t/Py3JsUl+fTQtlCQ9XGy40A2QJC0aL+p7\n/afA/gPSV42iMVX1QJJNgftneMpzgD8H3jpFnoOAM9p/Pw14M/Ax4K7ZtlOS9KvHIEqSBEBVndr7\nOskKYP/+9FGqqrXT5UmyWVXdO4N8uwBPAP5jGG2TJP3q8nI+SdLQJNkoyVuSXJNkbZLbklyQZL8B\neXdI8sUk9yT5aZJ3JlnSc/wh90QlOa5N2zXJZ9t7q85PcirwcmCDnssOH+ir8mDgTuDbSY4D3tGm\n39hzztKe9/HmJD9Ksi7Jj5O8LcnGfe/hpiT/nuTZSa5o3/MPkhw2jP6UJC1OzkRJkobpOOCvgI8A\nlwCPBJ4C/C7w9Z58GwFfBS4E3gAcAPw1cA3w0RnUcyZwFfDG9vV3gUcD+wAvbtPW951zEHBOVT2Y\n5HSaWakXAEfTBFcAtnRAVgAADHVJREFUd7T7jwNHAqcB7wJWAH8HLAOO6Ct3GfAp4EPAycCfAWck\nOaCqzpvBe5EkjRmDKEnSMB0MfLGqXjFNvkcAn6iqidmgE5NcQROAzCSIurSqfulerSRXA88YdPlh\nks2BvdvyqaorklxOE0R9vqpu6sm7nCaAOrGqXtkmfyDJbcBrkzy9qi7oKX5X4LlV9cX2/I/TBHj/\nBDx1qjeRZOUkh5ZNdZ4kaWF5OZ8kaZjWALslecIM8n647/WFwE4zrOdDnVoFf0DzxeHZM8h7ULs/\nvi/9Xe3+4L70GyYCKICqWgOcAjwlybYd2ylJGgPOREmSOkuyfV/SmnYRiL8HPg9cneR7wFeAU6rq\n+33576mqO/rS7gS2mmETftyxyQcD/1VVt80g7+OAB4BrexOr6qYkd7fHe10zoIzV7X5HYNI6q2r5\noPR2hmqPGbRVkrQAnImSJHWSZEPgJ33b4QBV9Q1gZ5rL5n4IvAy4PMlRfcU8OFnxM2zGfd1azbOB\nszqeI0nSQM5ESZK6epDm+VG9fjHTVFW3AycBJyXZguYyvWNpFl2YTzUoMcmTgMfy0KXNB+YHrqf5\n/3Fn4Oqech4LbNEe7zXo0sVd2v11U7ZYkjSWnImSJHVSjXP7tlsAkmzTl/dumsviNhlB0+6lWeJ8\n8770g4Cbq+qyAfkBtuxLn5ixem1f+uvbfX8w9ptJDp14kWRLmgcUXzLDywclSWPGmShJ0jCtTvI1\nYCXNPU5PBQ4DThhB3RMr3b0vybnA/VV1Gs39UIMu5ZvI/4/tkuf3A1+oqpVJPgn8ZZKtgQtoljh/\nEXBG38p80KzE929JPkhz/9OfAdvSrPAnSXoYMoiSJA3TCcAhwLNoZp+uA94E/MsI6j4NeBrwRzTP\nilrfBlN7Dqq/qr6d5Fia+7YOprk6YwfgJuAlNAtGvBh4Ps19X28H3jqg3iuB19Esab4rzaIXR1TV\nuUN8b5KkRSRVk10SPrX9lxwxuxMlSTP2tfWnz3ShBQ2Q5E9oHpy7TVXdMw/l30Rz2d5hQy535R57\n7LHHypWTPUZKkjRXy5cv59JLL710spVSp+I9UZKkh7M7gKPnI4CSJP3q8nI+SdLDVlXN5OG6kiR1\n4kyUJEmSJHXgTJQkSbNUVUsXug2SpNFzJkqSJEmSOjCIkiRJkqQODKIkSZIkqQODKEmSJEnqwCBK\nkiRJkjowiJIkSZKkDgyiJEmSJKkDgyhJkiRJ6sAgSpIkSZI6MIiSJEmSpA4MoiRJkiSpA4MoSZIk\nSerAIEqSJEmSOjCIkiRJkqQODKIkSZIkqYMNF7oBkiTpIXZctWoVy5cvX+h2SNLD1qpVqwB2nM25\nBlGSJC0+m993330PXnrppVcsdEPG0LJ2f+WCtmI82XezZ9/N3kL23Y7AXbM50SBKkqTF5/sAVeVU\nVEdJVoJ9Nxv23ezZd7M3rn3nPVGSJEmS1MGsZ6K+tv70DLMhkiRJkjQOnImSJEmSpA4MoiRJkiSp\nA4MoSZIkSeogVbXQbZAkSZKkseFMlCRJkiR1YBAlSZIkSR0YREmSJElSBwZRkiRJktSBQZQkSZIk\ndWAQJUmSJEkdGERJkiRJUgcGUZIkDUmSpUlOSnJzknVJrktyQpKtOpazdXvedW05N7flLp3vuhfK\nXNufZLMkRyb5VJIrk9yb5O4klyQ5JsnGk5xXU2wXD/ddzo9h/OyTnD9NX/zaJOf9dpLTktyaZG2S\nq5K8Jcmmw3uH82cI426fafptYtuh77yxHXdJDk/yviQXJLmrbfOpsyyrc/8vljHnw3YlSRqCJDsD\nFwHbAV8ArgSeCjwTuArYq6pun0E527Tl7AKcB3wHWAY8F7gV+L2q+tF81L1QhtH+JAcCXwHuAL4B\nXANsBRwKbN+Wv19Vre07r4DrgZMHFHtTVX1s1m9sBIY47s4HngG8ZZIsx1XVA33n7EkzRjcCzgBu\nBPYFngx8i6a/13V/V6MxpHG3I3DUJId3A54HfL+qdus7b2zHXZLLgd2Be4CbaP4+fbKqXtixnM79\nv6jGXFW5ubm5ubm5zXEDzgEKeE1f+vFt+okzLOfDbf539aUf3aafPV91j3PfAU8CjgQ27kvfAljZ\nlnPMgPMKOH+h+2ARjLvzm4+FM653A+CHbR2H9qQvoflwW8AbF7p/RtF3U5T/6bacowccG9txRxPk\nPBEIsE/7Xk6d7/5fbGPOmShJkuao/Ub1GuA6YOeqWt9zbAvgJzQfOLarqnunKGdzmtmm9cCjq+ru\nnmNLgB8Bj2vr+NEw614oo2h/kj8BPgl8uaoO6TtWwH9W1T6zegMLaJh9NzETVVWZYd37Al8HvllV\nz+g7thNwLc1My+NrEX7YnO9xl2Rbmlma9cBjqmpN3/GxHXe9kuxDM/PbaSZqNv2/2Mac90RJkjR3\nz2z3X+39MADQBkLfAh4BrJimnBXApsC3egOotpz1NN/c9tY3zLoXyijaf3+7f2CS41smeWmSNyV5\nVZLF2lf9ht53SV6Q5I1JXp/k2Uk2mSTrvu3+7P4DbYC/mibg32mmdY/YfI+7FwObAKf3B1A9xnXc\nDcNs+n9RjTmDKEmS5m7Xdr96kuNXt/td5qGcYdW9UEbR/pe2+4d8+GrtDvwr8Hbg/cC3k1yeZLdJ\n8i8W89F3nwHeAbwLOAu4IcnhI6p7lOa7/X/R7j88RZ5xHXfDMPZ/6wyiJEmau0e2+/+b5PhE+pbz\nUM6w6l4o89r+JK8GDgQuB04akOV4YC/gUTT3Tz2F5v6K3YHzkjx2NvWOyDD77gvAIcBSmtnQZTTB\n1JbAZ9uFO+ar7oUwb+1P8gyaD/zfr6qLJsk2zuNuGMb+b51BlCRJelhK8jzgBOAW4PlVdX9/nqo6\npqouqqrbquqeqrqkqo4APgdsC7xhtK1eGFX17qr6clX9T1WtraqrqupNwDE0nxffscBNHCcva/cf\nmSyD4278GURJkjR3E9+APnKS4xPpk90bMZdyhlX3QpmX9ic5jObStFuBfapvWfgZOLHd793xvFEa\nxc/+YzT3kj2pveF/lHXPp/kad1sDzwfuA06ZRbvGYdwNw9j/rTOIkiRp7q5q95Ndi//Edj/Ztfxz\nKWdYdS+Uobc/yRHA6cD/0qw4d9U0pwzy03a/2SzOHZV5/9lX81ytiUVOevvCcTfYxIISp02xoMRU\nxmHcDcPY/60ziJIkae6+0e4PaJci/4X22/u9gJ8BF09TzsU032Dv1fet/8QS5wf01TfMuhfKUNuf\n5Eia5/PcTBNAXT3NKZOZWBWs6wzWKM37zz7JrjQPLb4buK3n0Hntvv9eqYnlpnehWW56sfbffPXd\nxIISk17KN41xGHfDMJv+X1RjziBKkqQ5qqprga8COwKv6jv8FppvlU/pfd5MkmVJlvWVcw/NJUCb\nAcf2lfPqtvxzei9Nm03di8mw+q5NfzHwCeAGYO/pLuFL8jtJNhqUTrNiGsCpM383ozWsvkvy+PYy\nNPrSHwV8vH35marqXSL+P4FVwN5JDu05ZwnwzvbliYvxGVEw3HHXc/zpwG8x9YISYz/uukiyUdtv\nO/emz/Lv1qIacz5sV5KkIWg/JFwEbEez0tkqYE+a56GsBn6/qm7vyV8A/Q83TbJNW84uNN+8/jfN\nB7Pn0tzf8/vtB5BZ173YDKPvkjwTOJfmC+KTgBsHVLWmqk7oOedkmhXpLmjzr6NZle5AYAPgo8DL\nF2sgAEPru6No7sW5kOZb/DuA3wQOornP5BJg/wEPjN2TZoxuRLOy3A3AfsCTaZ7zs19VrRv2ex6W\nYf3O9hw/BXghcHRVvW+Kek9mjMdde7/hYe3L7YFn0YybC9q026rqDW3eHYEfA9dX1Y595XT+u7Wo\nxlxVubm5ubm5uQ1hA3ag+eb+J8DPaS4tOQHYakDeav4bHljO1sB72vN/3pZ3ErB0GHUvxm2ufQcc\nNZE+xXZd3zmHAWcC1wB39fT1l4BDF7pPRth3uwEnA98Dbqd5OPEdNB+KXwNsPEXdv01z/9ltNMHA\napqZhE0Xul9G0Xc9x7aiuRT3Z8CW09Q51uOOZpZ8Rr9nNDNND/ndm03/L7Yx50yUJEmSJHXgPVGS\nJEmS1IFBlCRJkiR1YBAlSZIkSR0YREmSJElSBwZRkiRJktSBQZQkSZIkdWAQJUmSJEkdGERJkiRJ\nUgcGUZIkSZLUgUGUJEmSJHVgECVJkiRJHRhESZIkSVIHBlGSJEmS1IFBlCRJkiR1YBAlSZIkSR0Y\nREmSJElSBwZRkiRJktTB/wMxb6bxuj4C0AAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<Figure size 432x648 with 2 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "image/png": {
              "width": 424,
              "height": 216
            }
          }
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "zrAbdAmjYMAw",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "import matplotlib.pyplot as plt\n",
        "import numpy as np\n",
        "from torch import nn, optim\n",
        "from torch.autograd import Variable\n",
        "\n",
        "\n",
        "def test_network(net, trainloader):\n",
        "\n",
        "    criterion = nn.MSELoss()\n",
        "    optimizer = optim.Adam(net.parameters(), lr=0.001)\n",
        "\n",
        "    dataiter = iter(trainloader)\n",
        "    images, labels = dataiter.next()\n",
        "\n",
        "    # Create Variables for the inputs and targets\n",
        "    inputs = Variable(images)\n",
        "    targets = Variable(images)\n",
        "\n",
        "    # Clear the gradients from all Variables\n",
        "    optimizer.zero_grad()\n",
        "\n",
        "    # Forward pass, then backward pass, then update weights\n",
        "    output = net.forward(inputs)\n",
        "    loss = criterion(output, targets)\n",
        "    loss.backward()\n",
        "    optimizer.step()\n",
        "\n",
        "    return True\n",
        "\n",
        "\n",
        "def imshow(image, ax=None, title=None, normalize=True):\n",
        "    \"\"\"Imshow for Tensor.\"\"\"\n",
        "    if ax is None:\n",
        "        fig, ax = plt.subplots()\n",
        "    image = image.numpy().transpose((1, 2, 0))\n",
        "\n",
        "    if normalize:\n",
        "        mean = np.array([0.485, 0.456, 0.406])\n",
        "        std = np.array([0.229, 0.224, 0.225])\n",
        "        image = std * image + mean\n",
        "        image = np.clip(image, 0, 1)\n",
        "\n",
        "    ax.imshow(image)\n",
        "    ax.spines['top'].set_visible(False)\n",
        "    ax.spines['right'].set_visible(False)\n",
        "    ax.spines['left'].set_visible(False)\n",
        "    ax.spines['bottom'].set_visible(False)\n",
        "    ax.tick_params(axis='both', length=0)\n",
        "    ax.set_xticklabels('')\n",
        "    ax.set_yticklabels('')\n",
        "\n",
        "    return ax\n",
        "\n",
        "\n",
        "def view_recon(img, recon):\n",
        "    ''' Function for displaying an image (as a PyTorch Tensor) and its\n",
        "        reconstruction also a PyTorch Tensor\n",
        "    '''\n",
        "\n",
        "    fig, axes = plt.subplots(ncols=2, sharex=True, sharey=True)\n",
        "    axes[0].imshow(img.numpy().squeeze())\n",
        "    axes[1].imshow(recon.data.numpy().squeeze())\n",
        "    for ax in axes:\n",
        "        ax.axis('off')\n",
        "        ax.set_adjustable('box-forced')\n",
        "\n",
        "def view_classify(img, ps, version=\"MNIST\"):\n",
        "    ''' Function for viewing an image and it's predicted classes.\n",
        "    '''\n",
        "    ps = ps.data.numpy().squeeze()\n",
        "\n",
        "    fig, (ax1, ax2) = plt.subplots(figsize=(6,9), ncols=2)\n",
        "    ax1.imshow(img.resize_(1, 28, 28).numpy().squeeze())\n",
        "    ax1.axis('off')\n",
        "    ax2.barh(np.arange(10), ps)\n",
        "    ax2.set_aspect(0.1)\n",
        "    ax2.set_yticks(np.arange(10))\n",
        "    if version == \"MNIST\":\n",
        "        ax2.set_yticklabels(np.arange(10))\n",
        "    elif version == \"Fashion\":\n",
        "        ax2.set_yticklabels(['T-shirt/top',\n",
        "                            'Trouser',\n",
        "                            'Pullover',\n",
        "                            'Dress',\n",
        "                            'Coat',\n",
        "                            'Sandal',\n",
        "                            'Shirt',\n",
        "                            'Sneaker',\n",
        "                            'Bag',\n",
        "                            'Ankle Boot'], size='small');\n",
        "    ax2.set_title('Class Probability')\n",
        "    ax2.set_xlim(0, 1.1)\n",
        "\n",
        "    plt.tight_layout()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "JOoHcVQSYMqR",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    }
  ]
}