DataMining_week12.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "DataMining-week12.ipynb",
      "provenance": [],
      "toc_visible": true,
      "authorship_tag": "ABX9TyOu11P1DMHe1qp6M+b15bma"
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "UWlV6OxwMKhU",
        "colab_type": "text"
      },
      "source": [
        "# Model Selection\n",
        "\n",
        "The task of producing the most suitable predictive model from a dataset has many nuances. In particular, there are two aspects that need to be defined experimentally before starting to adjust a model:\n",
        "\n",
        "* At first, it may not be clear which learning algorithm is best suited to be used in a given situation.\n",
        "\n",
        "* In addition, each learning algorithm has its own set of *hyperparameters*, which are the parameters of the algorithm itself, to counter the concept of *parameter* of the ML model to be adjusted.\n",
        "\n",
        "Model selection is the process of selecting one of different learning algorithms (e.g., SVM, decision tree learning, logistic regression, etc.) or, once the learning algorithm is selected, choosing between different values ​​of hyperparameters (e.g., in polynomial regression, deciding which polynomial degree to use during training).\n",
        "\n",
        "For a given prediction task, the choice of the most appropriate learning algorithm (and its respective hyperparameter values) is made based on some *estimate* of the generalization error obtained on unused data during training. In other words, different candidate models are evaluated during model selection. The model that has the best predictive performance (according to some pre-established assessment measure) is selected as a result of the model selection process."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Vvv1GLjaNKI1",
        "colab_type": "text"
      },
      "source": [
        "## Holdout method\n",
        "\n",
        "We have already seen several examples in which the original dataset is split into two datasets, one for training and the other for test. This two-way split is illustrated in the image below ([source](https://www.datavedas.com/holdout-cross-validation/)).\n",
        "\n",
        "![alt text](https://www.datavedas.com/wp-content/uploads/2018/04/image001-768x368.jpg)\n",
        "\n",
        "However, since the test dataset should ideally be used only to produce an estimate of the generalization error **after** training, it is common (at least for relatively large data sets) to subdivide the available training set into two subsets. The result of this is the production of three datasets, as described below.\n",
        "\n",
        "* Training set: used to adjust the parameters of the model.\n",
        "* Validation set: used to adjust the hyperparameters of the ML algorithm.\n",
        "* Test set: used once to obtain an estimate of the generalization error.\n",
        "\n",
        "The image below ([source](https://www.datavedas.com/holdout-cross-validation/)) illustrates the three-way split of the original dataset.\n",
        "\n",
        "![alt text](https://www.datavedas.com/wp-content/uploads/2018/04/image003.jpg)\n",
        "\n",
        "The image below ([source](https://datascience.stackexchange.com/questions/52632/cross-validation-vs-train-validate-test)) summarizes the model selection process for a given learning algorithm using the three data sets described above. Initially, the original data set is divided into two subsets, training and testing. Then, the training set is again divided into two, which generates the training, validation and test sets. After that, for each desired combination of hyperparameter values, a model is adjusted on the training set, and its performance on the validation set is computed. In this way, for each of the $ q $ combinations of hyperparameter values, $ q $ performance estimates are obtained. The predictive model corresponding to the combination of hyperparameter values ​​that produces the best performance estimate is selected. Finally, the generalization error estimate for this selected model is produced using the test set.\n",
        "\n",
        "![alt text](https://i.stack.imgur.com/osBuF.png)\n",
        "\n",
        "The procedure illustrated in the figure above can be performed once for each of the different learning algorithms candidates in a supervised learning task."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "9Vq5YxWvN5_F",
        "colab_type": "text"
      },
      "source": [
        "### Split proportions\n",
        "\n",
        "Reserving  a test dataset to compute an estimate of the generalization error has the negative side effect of retaining potentially valuable information that the learning algorithm could benefit from. Therefore, choosing the size of the test set  involves a trade-off: allocating a very large test set can retain a lot of information that could be used to improve model fit during training. On the other hand, the smaller the test set, the more inaccurate the generalization error estimate is.\n",
        "\n",
        "In most practical cases, the most used divisions are 60:40,\n",
        "70:30 or 80:20, depending on the size of the initial data set.\n",
        "\n",
        "However, for large data sets, 90:10 or even 99: 1 may be appropriate, since 10% (or even 1%) of a very large sample can still be statistically significant.\n",
        "\n",
        "A common practice is to re-train the estimator on the entire data set (i.e., rejoining the training and test sets), in order to improve predictive performance of the resulting final model."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "piO8DLsf93Cv",
        "colab_type": "text"
      },
      "source": [
        "## $k$-fold Cross-validation\n",
        "\n",
        "There are two disadvantages of the holdout method:\n",
        "\n",
        "* the performance estimate can be very sensitive to the way used to subdivide the training set into the training and validation sets: this estimate will vary for different data samples. Repeating the holdout validation on different subsets of the data would provide a better estimate of the algorithm's performance.\n",
        "\n",
        "* a significant part of the data is not used for training the model. For example, if the split ratio is 50%, half of the data set does not contribute to model training! This can cause problems, especially if the initial data set is small.\n",
        "\n",
        "A more robust technique for performing model selection is *k-fold cross-validtion*. This technique corresponds to applying the holdout method $k$ times over $k$ different configurations of training and validation subsets.\n",
        "\n",
        "The following figure ([source](https://sebastianraschka.com/faq/docs/evaluate-a-model.html)) illustrates $k$-fold cross validation. This is an iterative technique. In the first iteration, the training set is subdivided into $k$ disjunct sets (in the figure, $k = 10$). Then, $k-1$ of these subsets are used to fit the predictive model. Once the model has been adjusted, its error is estimated in the examples of the subset that was not used during the adjustment, which produces the value $E_1$ in the figure. In the second iteration, another fold is selected to compute the estimate and again a model is adjusted, which allows to compute another estimate of the error, $E_2$. In all, $k$ iterations are performed, which results in $k$ performance estimates. Finally, a more robust (i.e., less biased) estimate of the model error is obtained by computing the average of the previously produced $k$ estimates.\n",
        "\n",
        "![alt text](https://sebastianraschka.com/images/faq/evaluate-a-model/k-fold.png)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "LUXqo-lsHaPo",
        "colab_type": "text"
      },
      "source": [
        "### Function cross_val_score"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "VJu2J83AX-NW",
        "colab_type": "text"
      },
      "source": [
        "In the following example, the $k$-fold cross-validation technique is applied, with $k = 2$. The result is two performance estimates, which can be combined (for example, taking the average) to obtain a better measure of the model's generalization performance. This specific form of cross-validation is double cross-validation: we split the original data into two subsets and use each one as a validation set."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "D33MZ2_iXSwY",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        },
        "outputId": "d34e8d88-5451-44ba-a6bf-c760a4a331f1"
      },
      "source": [
        "from sklearn.model_selection import train_test_split\n",
        "from sklearn.datasets import load_iris\n",
        "from sklearn.neighbors import KNeighborsClassifier\n",
        "from sklearn.metrics import accuracy_score\n",
        "\n",
        "# Configures k-NN with 1 neighbor\n",
        "model = KNeighborsClassifier(n_neighbors=1)\n",
        "\n",
        "iris = load_iris()\n",
        "X = iris.data\n",
        "y = iris.target\n",
        "\n",
        "# divide the original dataset in 2 equal parts\n",
        "X1, X2, y1, y2 = train_test_split(X, y, random_state=0, train_size=0.5)\n",
        "\n",
        "# ajusta e avalia DOIS modelos\n",
        "\n",
        "y2_model = model.fit(X1, y1).predict(X2)\n",
        "\n",
        "y1_model = model.fit(X2, y2).predict(X1)\n",
        "\n",
        "accuracy_score(y1, y1_model), accuracy_score(y2, y2_model)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(0.96, 0.9066666666666666)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 2
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "jOlBbOMaGw1h",
        "colab_type": "text"
      },
      "source": [
        "Implementing $k$-fold cross validation using the approach illustrated in the previous example can be time consuming for greater values of $k$.\n",
        "\n",
        "Scikit-Learn provides the function [`cross_val_score`](https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.cross_val_score.html). This function makes it easier to apply $k$-fold cross validation. \n",
        "\n",
        "It returns a NumPy array with an accuracy value for each fold. Notice that, for each fold, the training and tests sets will be different. Therefore, different accuracy values for each fold will be produced. Also notice that we can compute the mean and standard deviation of the resulting accuracies.\n",
        "\n",
        "The following example illustrates the application of the function cross_val_score to the case $k = 10$."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "tSLzxCB_YERc",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 124
        },
        "outputId": "70c675e7-13a7-4f4b-89c6-fb35ef94bbf5"
      },
      "source": [
        "from sklearn.model_selection import cross_val_score\n",
        "scores = cross_val_score(model, X, y, cv=10)\n",
        "print('Scores in each fold: ', scores)\n",
        "print('Average score: ', scores.mean())\n",
        "print('Std deviation of scores: ', scores.std())"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Scores in each fold:  [1.  1.  1.  1.  0.  0.  0.  0.5 0.  1. ]\n",
            "Average score:  0.55\n",
            "Std deviation of scores:  0.4716990566028302\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.6/dist-packages/sklearn/model_selection/_split.py:667: UserWarning: The least populated class in y has only 6 members, which is less than n_splits=10.\n",
            "  % (min_groups, self.n_splits)), UserWarning)\n"
          ],
          "name": "stderr"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mUefpkmoY4v2",
        "colab_type": "text"
      },
      "source": [
        "Scikit-Learn implements several cross-validation schemes that are useful in particular situations; these are implemented through iterators in the cross_validation module. In practice, we have to define the iterator through the `cv` parameter.\n",
        "\n",
        "As an example, we might want to go to the extreme case where the number of subsets is equal to the number of examples contained in the training set. In this case, in each iteration, a model is fitted using all but one example, and accuracy is measured against the only example not used during training. This type of cross-validation is known as **leave-one-out cross-validation**.\n",
        "\n",
        "The example below illustrates the use of this particular case leave-one-out cross-validation. Since there are 150 samples, the leave-one-out cross-validation produces 150 estimates (1.0 indicates a successful forecast, and 0.0 indicates an unsuccessful one)."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "paZ5oipGZIoJ",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 173
        },
        "outputId": "dbf573c2-143c-443d-8011-52720752e43f"
      },
      "source": [
        "from sklearn.model_selection import LeaveOneOut\n",
        "\n",
        "scores = cross_val_score(model, X, y, cv=LeaveOneOut())\n",
        "scores"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.,\n",
              "       1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.,\n",
              "       1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.,\n",
              "       1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.,\n",
              "       1., 1., 0., 1., 0., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 0., 1.,\n",
              "       1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.,\n",
              "       1., 1., 1., 1., 0., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.,\n",
              "       0., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 0., 1., 1.,\n",
              "       1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 7
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "APSamO7pZZ2F",
        "colab_type": "text"
      },
      "source": [
        "The average of these 150 values ​​corresponds to the estimated error rate of the model produced:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "aXh4DFwqZgeG",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 52
        },
        "outputId": "24c45ba7-c67b-4f11-9488-85b72d42bfb1"
      },
      "source": [
        "print('Average score: ', scores.mean())\n",
        "print('Std deviation of scores: ', scores.std())"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Average score:  0.96\n",
            "Std deviation of scores:  0.19595917942265423\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "yHk15WRYiziA",
        "colab_type": "text"
      },
      "source": [
        "## Stratified k-fold cross-validation\n",
        "\n",
        "In the context of the classification task, an unbalanced presents as unequal proportion of examples for each class. See the image below ([source](https://www.researchgate.net/publication/306376881_Survey_of_resampling_techniques_for_improving_classification_performance_in_unbalanced_datasets)) for an example.\n",
        "\n",
        "![alt text](https://miro.medium.com/max/698/1*cd6AorHoJYMFyj7IZd2nOg.png)\n",
        "\n",
        "For unbalanced datasets, if the vanilla version of $k$-fold cross-validation is used, it can be the case that one or more produced folds end up with zero examples of a particular class. In such situations, it is better to use *stratified $k$-fold cross-validation*. In stratified $k$ cross-validation, class proportions are preserved in each of the $k$ subsets, so that each one is representative of the class proportions in the training data set. The image below (source) illustrates how stratified $k$-fold crors-validation works.\n",
        "\n",
        "![alt text](https://miro.medium.com/max/562/0*QKJTHrcriSx2ZNYr.png)\n",
        "\n",
        "In Scikit-Learn, the `StratifiedKFold` class implements the stratified cross-validation method. See the following example, that applies stratified cross-validation to a toy dataset with two classes. Notice that the proportions os classes in each fold is approximately the same as the original dataset."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "bB_y_3iml9Gs",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 69
        },
        "outputId": "05f0b464-abde-4d13-b703-f4358b62ba22"
      },
      "source": [
        "import numpy as np\n",
        "from sklearn.model_selection import StratifiedKFold\n",
        "\n",
        "# data matrix\n",
        "X = np.array([[1, 2], [3, 4], [1, 2], [3, 4], [5, 6], [7, 8], \n",
        "              [9, 10], [9, 10], [11, 12], [11, 12],\n",
        "              [1, 2], [3, 4], [5, 6], [7, 8], [4, 5], [5, 6], [7, 8], [4, 5]])\n",
        "\n",
        "# response vector\n",
        "y = np.array([0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1])\n",
        "\n",
        "skf = StratifiedKFold(n_splits=2, random_state=42)\n",
        "skf.get_n_splits(X, y)\n",
        "\n",
        "print(skf)  \n",
        "\n",
        "# print the indices of examples in each fold\n",
        "for train_index, test_index in skf.split(X, y):\n",
        "   print(\"TRAIN:\", train_index, \"TEST:\", test_index)\n",
        "   X_train, X_test = X[train_index], X[test_index]\n",
        "   y_train, y_test = y[train_index], y[test_index]"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "StratifiedKFold(n_splits=2, random_state=None, shuffle=False)\n",
            "TRAIN: [ 3  4  5 12 13 14 15 16 17] TEST: [ 0  1  2  6  7  8  9 10 11]\n",
            "TRAIN: [ 0  1  2  6  7  8  9 10 11] TEST: [ 3  4  5 12 13 14 15 16 17]\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "UJw6vFxDix6O",
        "colab_type": "text"
      },
      "source": [
        "## Hyperparameter optimization\n",
        "\n",
        "In contrast to the model parameters that are learned during training, the values ​​of the hyperparameters are set before the training of a model begins. Some examples of hyperparameters are given below.\n",
        "\n",
        "* the learning rate in logistic regression, \n",
        "* the height of trees in decision tree learning,\n",
        "* the value of $k$ in $k$-NN, \n",
        "* the value of $k$ in $k$-means, \n",
        "* the values of $\\epsilon$ and $\\operatorname{minPoints}$ in DBSCAN,\n",
        "* the regularization term in linear regression,\n",
        "* the polynomial degree in polynomial regression.\n",
        "\n",
        "\n",
        "Hyperparameters can be considered as configurations of a learning algorithm. In general, the ideal settings for the algorithm to generate a suitable model for one dataset may not be the same for another dataset. \n",
        "\n",
        "*Hyperparameter optimization* (aka *hyperparameter tuning*) is the procedure of defining appropriate values for the hyperparameters of a given learning algorithm. The general approach is to generate several combinations of hyperparameter values. Then, a model is trained for each combinatation. Finally, the best combination (according to some evaluation measure applied on the candidate models) is chosen.\n",
        "\n",
        "When the learning algorithm generates an estimator (either a classifier or a regressor), there are two main strategies to *automatically* tune its hyperparameters: *Grid Search* and *Random Search*. Both strategies generate several hyperparameter combinations. Then, for each combination, an estimator is trained a model (on the training set) and evaluateed (on the validation set). The difference between these two strategies is in the way the several combinations are generated, as described below."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "iB4Yw9xzVy4j",
        "colab_type": "text"
      },
      "source": [
        "**Grid search** first defines a grid of hyperparameter combinations. The amount of hyperparameters defines the number of dimensions in the grid. Then, each combination (a point in the grid) ​​is used to produce a model.\n",
        "\n",
        "The image below ([source](https://www.researchgate.net/publication/271771573_TuPAQ_An_Efficient_Planner_for_Large-scale_Predictive_Analytic_Queries/figures?lo=1)) illustrates the GS procedure for a learning algorithm with two hyperparameters. The star represents the combination of hyperparameters that gives the best model. as measured in the validation set. The red dot in picture on the left represents the first combination test by GS. The picture on the center represents the second combination. The picture on the right illustrates all the tested combinations in the 2-dimensional grid.\n",
        "\n",
        "![texto alternativo](https://www.researchgate.net/profile/Michael_Jordan13/publication/271771573/figure/fig4/AS:668513593217027@1536397469229/Illustration-of-naive-grid-search-for-a-single-model-family-with-two-hyperparameters_W640.jpg)\n",
        "\n",
        "**Random search** selects random combinations of hyperparameter values ​​(within preconfigured ranges of values for each hyperparameter) to train and evaluate the candidate models. Instead of trying all possible combinations of values, RS performs a pre-defined number of iterations, testing a random combination of hyperparameters ​​in each iteration.\n",
        "\n",
        "Both GS and RS can be very computationally expensive. For example, searching for 20 different values ​​for each of 4 hyperparameters would require 160,000 k-fold cross-validation runs. If $k = 10$ then 1,600,000 model adjustments and 1,600,000 evaluations should be performed\n",
        "\n",
        "The image below ([source](https://community.alteryx.com/t5/Data-Science/Hyperparameter-Tuning-Black-Magic/ba-p/449289)) illustrates the difference between GS and RS. Think of a learning algorithm with just two hyperparameters. This way, each combination of its hyperparameters is a pair of numbers. Suppose one of these hyperparameters (x-axis) has more influence than the other (y-axis) on the predictive performance of the generated models. The plot on the left shows several of these pairs organized in a grid. In total, there are thirty combination in this grid. The picture on the right shows other combinations of pairs; this times these combinations where randomly selected. Notice that RS has the potential to explore more promising combinations than GS.\n",
        "\n",
        "![texto alternativo](https://pvsmt99345.i.lithium.com/t5/image/serverpage/image-id/74545i97245FDAA10376E9/image-size/large?v=1.0&px=999)\n",
        "\n",
        "In Scikit-Learn, the classes [GridSearchCV](https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.GridSearchCV.html) and [RandomizedSearchCV](https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.RandomizedSearchCV.html) implement GS and RS, respectively."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hQg_3AcwaZjQ",
        "colab_type": "text"
      },
      "source": [
        "### Class GridSearchCV"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "n59pZ0EI4I_d",
        "colab_type": "text"
      },
      "source": [
        "Scikit-Learn provides the class `GridSearchCV` to perform grid search.\n",
        "\n",
        "The following code block presents an example of using the `GridSearchCV` class to find the ideal polynomial model for a data set. In this example, a three-dimensional grid of hyperparameters is explored:\n",
        "\n",
        "* the polynomial degree,\n",
        "* a boolean indicator that indicates whether the linear coefficient should be adjusted,\n",
        "* an indicator (boolean) indicating whether the data should be normalized."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "HyyBkg2E0eAP",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "from sklearn.preprocessing import PolynomialFeatures\n",
        "from sklearn.linear_model import LinearRegression\n",
        "from sklearn.pipeline import make_pipeline\n",
        "\n",
        "def PolynomialRegression(degree=2, **kwargs):\n",
        "    return make_pipeline(PolynomialFeatures(degree),\n",
        "                         LinearRegression(**kwargs))"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "SAW_e44vz86E",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "from sklearn.model_selection import GridSearchCV\n",
        "import numpy as np\n",
        "\n",
        "param_grid = {'polynomialfeatures__degree': np.arange(21),\n",
        "              'linearregression__fit_intercept': [True, False],\n",
        "              'linearregression__normalize': [True, False]}\n",
        "\n",
        "def make_data(N, err=1.0, rseed=1):\n",
        "    # randomly sample the data\n",
        "    rng = np.random.RandomState(rseed)\n",
        "    X = rng.rand(N, 1) ** 2\n",
        "    y = 10 - 1. / (X.ravel() + 0.1)\n",
        "    if err > 0:\n",
        "        y += err * rng.randn(N)\n",
        "    return X, y"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "VYV99pMK0FB8",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 71
        },
        "outputId": "e95b5e59-aabd-45a8-88e7-243f803dcf45"
      },
      "source": [
        "X, y = make_data(40)\n",
        "grid = GridSearchCV(PolynomialRegression(), param_grid, cv=7)\n",
        "grid.fit(X, y);"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.6/dist-packages/sklearn/model_selection/_search.py:814: DeprecationWarning: The default of the `iid` parameter will change from True to False in version 0.22 and will be removed in 0.24. This will change numeric results when test-set sizes are unequal.\n",
            "  DeprecationWarning)\n"
          ],
          "name": "stderr"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Uw8ZQ4025wje",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 291
        },
        "outputId": "e282cb97-3bff-4748-a6a4-c9f52be86290"
      },
      "source": [
        "%matplotlib inline\n",
        "import matplotlib.pyplot as plt\n",
        "import seaborn; seaborn.set()  # plot formatting\n",
        "\n",
        "X_test = np.linspace(-0.1, 1.1, 500)[:, None]\n",
        "\n",
        "plt.scatter(X.ravel(), y, color='black')\n",
        "axis = plt.axis()\n",
        "for degree in [1, 3, 5]:\n",
        "    y_test = PolynomialRegression(degree).fit(X, y).predict(X_test)\n",
        "    plt.plot(X_test.ravel(), y_test, label='degree={0}'.format(degree))\n",
        "plt.xlim(-0.1, 1.0)\n",
        "plt.ylim(-2, 12)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(-2, 12)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 13
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX8AAAEBCAYAAACQbKXWAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzs3Xd4XNWd//H3napR77KsasmW5N5t\nbDAYFwwuQDAtGNgQShI2ZUOyWTb+bYCAkzjJ+kmWEsImG7K7ToiJNwQXMDYGbCz33mTZsnrvbfrc\n+/tjbOEi2yozavN9PY8emSl3zkHSZ86ce+75KpqmaQghhAgouv5ugBBCiL4n4S+EEAFIwl8IIQKQ\nhL8QQgQgCX8hhAhAEv5CCBGAJPyFECIASfgLIUQA6lb4r169mnnz5pGdnU1+fj4AjY2NPP300yxa\ntIhly5bxzW9+k4aGBr80VgghhG90K/znz5/P2rVrSUpK6rhNURSeeuoptmzZwoYNG0hJSeGXv/yl\nzxsqhBDCd7oV/tOmTSMxMfGy2yIjI5k5c2bHf0+aNImKigrftE4IIYRf+HTOX1VV/vznPzNv3jxf\nHlYIIYSP+TT8X375ZYKDg3n00Ud9eVghhBA+ZvDVgVavXk1xcTFvvvkmOl3331MaG9tR1b7dYDQm\nJpT6+rY+fc3+Jn0ODNLnwcXtUdl3uoZPD5fT1OYgOS6U+dOSGZMWhaIonT5Hp1OIigrp8Wv6JPzX\nrFnDiRMneOuttzCZTD06hqpqfR7+F1830EifA4P0eeBzOD18eqScD/eW0NzuZGRSBCsWZjFuRDSK\noqBp4K9d97sV/q+88gofffQRdXV1PPHEE0RGRvKrX/2K3/72t6Snp/Pwww8DkJyczOuvv+6XBgsh\nxGBnc7jZfqiMj/aX0mp1kZMayTN3jyUnNfKaI31fUwZKMZf6+rY+f9eOiwujtra1T1+zv0mfA4P0\neWBqt7vYdqCMbQdKabe7GZcRzbLZ6YxKjuz2sXQ6hZiY0B63xWdz/kIIITrXanXy0f5Sth8qw+bw\nMGlkLMtuTmdEYni/tUnCXwgh/KS5zcGWfaV8crgcp8vD1Ow4ls5OJzUhrL+bJuEvhBC+1tBi54O9\nJew4WoHbozJzTAJLZqWTFNvz1Tm+JuEvhBA+UttkY/OeYj4/VgnArHHDWHJTGgnRwf3csqtJ+Ash\nRC9VN1jZuLuI3Seq0elgzsThLJ6ZSmykpb+bdk0S/kII0UPlde1syi1i7+lqDHod86YmceeMVKLD\ng/q7aTck4S+EEN1UUt3KhtwiDp2pxWTUs2hGKotmpBIR0rOLXPuDhL8QQnTR+YoWNuYWceRcHRaz\nniWz01g4LYWw4MET+hdJ+AshxA3klzaxIbeIk4UNhAQZuHfOCBZMTSY4yNjfTesxCX8hhOiEpmnk\nFTeyIbeIvJImwoKN3D83k9snJ2ExD/7oHPw9EEIIH9I0jePnG9iQW0hBeQsRoSYenj+K2yYNx2zU\n93fzfEbCXwghAFXTOHq2jg25RRRVtRIdbubRO7KYMyERo2HohP5FEv5CiICmqhoHztSwMbeYsto2\n4iKD+MpdOcweNwyD3qf1rgYUCX8hREDyqCr7TtWwcXcRlfVWhkUH89TS0cwck4C+BwWpBhsJfyFE\nQHF7VHJPVLF5dzE1TTaS40L4+j1jmZYdj07XN3vpDwQS/kKIgOBye/j8WCWb9xRT3+IgLSGMb943\nnkmjYtH1UQGVgUTCXwgxpDlcHj47UsGHe4tpanOSmRTOY4tyGJ8R3WdVs/xBc9mBnhdzGfoTW0L4\nyfr165gyZSwJCRFMmTKW9evX9XeTxCVsDjcf7CnmX36TyzsfnyUhKpjvPzyJHz46lQmZMYM6+NXm\nKuwfvdqrY8jIX4geWL9+Hc899y1sNhsAZWWlPPfctwBYvvzB/mxawLPaXWw7WMbW/d5SiWNHeEsl\nZqV0v1TiQOQuO4Ft2xsYI+J7dRwJfyF6YNWqlzqC/yKbzcaqVS9J+PeTlnYn/7fjPB8fLMXm8DAx\nM4alN6eTOTyiv5vmE5qm4Tr2IY5969BFJRG06Du9Op6EvxA9UF5e1q3bhf80tzvZsq+ETw+XY3de\nKJU4K520Yf1fKtFXNHsbtk9/h6fkCIYR0wia+xQ6c+9qBUj4C9EDSUnJlJWVdnq76BuNrQ4+2FvM\njiMVuDwqcyYlsXBKEklxPT8JOhB5agqwbXsDzdqEefYKjGMX+OR8RbfCf/Xq1WzZsoXy8nI2bNhA\nVlYWAIWFhTz//PM0NTURGRnJ6tWrSU9P73XjhBioVq584bI5fwCLxcLKlS/0Y6sCQ12zjc17Svj8\nWAWqCrPGeevjjs9OoLa2tb+b5zOapuI6sRXH3nUoIVEE370SfXyGz47frdU+8+fPZ+3atSQlJV12\n+wsvvMAjjzzCli1beOSRR/jRj37kswYKMRAtX/4ga9a8SnJyCoqikJycwpo1rwbkfH9frXqqbrTy\nX5tP86+/3cPOoxXcPD6Rn37tJp5cMoZhA7BGbm+obQ3YNv87jt1/xpAygZD7XvJp8EM3R/7Tpk27\n6rb6+npOnTrFH/7wBwCWLl3Kyy+/TENDA9HR0b5ppRAD0PLlDwZk2F+qL1Y9VdS1s2l3EXtOeUsl\nzp2cxF0zB0epxO7SNA13wR7sn/8PqB7Mc76CMec2vyxL7fU6/8rKShISEtDrvbve6fV64uPjqays\n7HXjhBAD2/VWPfVWaU0bb7x3gn/73V4O5tdyx/QUfv71WaxYmDWgg7+nn4Q0exv2j3+Dfftv0UUN\nJ2T5jzGNnuu36xEGzAnfmJj+OUkTFzd0VgR0lfQ5MPRFn6+36qmnr3+2tJG/bM1n78kqLGYD988f\nxT23ZhIRar7hc/v757x27Vq+971vY7VaAe8noe9979uEhQWxYsWKTp+jaRptJ3ZQv+1tVHs7UXMf\nIXLWvSg6/24j3evwT0xMpLq6Go/Hg16vx+PxUFNTQ2JiYreOU1/fhqpqvW1Ot8TFhQ2pE0RdIX0O\nDH3V5+uteuru658ra+b93EJOnG8g2GzgnltGsGBaMiFBRpw2J7U253WfPxB+zs8//68dwX+R1Wrl\n+ef/lTvuuPuqx6stNdh3/hFP+Ul08ZkEL/4B7uhk6uqtVz32Sjqd0qtBc6/DPyYmhtGjR7Nx40bu\nueceNm7cyOjRo2W+X4gA0NtVT5qmkVfSxMbcIk4XNxJqMbL8tgzmTUkelKUSu3r9h+Zx4Ty+BefB\n90Gnw3zzYxjH3I6i9N2OO936v/vKK6/w0UcfUVdXxxNPPEFkZCSbNm3ixRdf5Pnnn+eNN94gPDyc\n1atX+6u9QogB5OJJ3VWrXqK8vIykpGRWrnzhhid7NU3jZGED7+cWca6smYgQEw/NG8ncSUmYTYO3\nataNrv/QNA138SEcu99Ba63FkD4V882PoguJ6uumomia1rdzLdcg0z59Q/ocGAZqnzVN4+i5ejbk\nFlJY2UpUmJnFN6UxZ0Iipl7Wxx0Ifb5y9RN4PwmtWfMq994+C0fun/BUnEYXlYR51iMYksf2+LX6\nfdpHCCFuRNU0Dp2pZUNuEaU1bcRGBPEPd2Zz8/jEXpdKXL9+Xbc/efhLZ5+EXnn+n7k9phnr+h+B\nKRjzzY9iHH2730/o3oiM/Afg6MifpM+BYaD02aOq7Dtdw8Zcb6nEhOhgls5KY+aYBJ/Ux73eSLu/\nr8FQrU04D72PK+8zUBSMY+ZjnrwMJcg3Kxt7O/KX8B8AfyB9SfocGPq7z26Pyu6TVWzaXUxNo42k\n2BCWzk5neo5vSyVOmTK20zn25OQUDh066bPX6Q61rQHn8S24Tn0Cqgdjzq2Yptzt83l9mfYRQgwY\nLrfKruPeUol1zXZSE0L5xy+NY3JWnF9KJQ6k3VXVpkqcRzfjOpsLmoZh5E2Yp96LLrx3++77i4S/\nEKLXnC4Pnx2t4MO9JTS2OsgYHs6KhVl+r5jV37urapqGpzIP14ltuIsOgd6AcfRcTBPuRBcW1ydt\n6CkJfyFEj9mdbj49XMGH+0poaXeSlRzBVxePZkx6VJ+USeyv3VU1lx3X2VxcJ7ehNlaAOQTT5KUY\nxy1EZwn362t7VA95jec403SWr8V8ucfHkfAXQnSb1e7m40PeUoltNhdj0qNYds9YslO7Nq/tUT24\nNQ+apqJqGhoaase/VQD0ip5NG9/nlz//GeWlpSQOG87KH16+kqen1xn0hKZpeKrP4c7/HFfBPnDZ\n0MWmEXTbkxgyZ6IYTD5/zYuaHa3kNeRzquEMpxvyaXdZSQkf3qtjyglfORE45EmffafN5mLbgVK2\nHizBrrYzakQQU8dEEhGh0O6y0u5qv/DditVtw+Fx4PA4cV78Ul04PE5UTe3R62uqil7RYzFZMOvN\nmPQmzHoTZr2ZMIsFxaPHpDMRZDATbLBgMVoINlz4uvBviyGYYKMFo65rY1+1tRZXfi6us7vQWmrA\nYMIwYhqmMfPQxWf6/BOO0+Oksr2a4pZSilpKKWopodpaC0CYKZQx0dlMiBvLhLgxJMT1vESljPyF\nEJfRNI0WZyu1tnrqbQ002JuobW/kXE0VddYmMNhRxjsJUqAUKC0Hyr3PVVAINlgIMQYTbAwmSG8m\nxBiCWW/CpDNh0hsxXfi3QadHURR0KCiK7sK/degUBQ148aX/h91hQ9Hr0Bl06PQ6FL2OiKhIvvzo\nY5e9sTg8DhqsTbQ7vG849gv3XY9RZyTYEITFGNzxBmExBGExBGH2eDC31GGqL8fcXEOQqhEcnUro\nuLmEpk3BYIlE6+Kbx0Wqpnrb5nZgc9tpdrbQ7PB+NTlaqLXVUW2tpcHe2PGcMGMo6REp3DRsGqNj\nskgKTUR3YQuI3q6akvAXohMD6cIhf9A0jSZHM1XtNdTY6qiz1VNrq6fOVk+drQGX6rr88W4jmjOI\ncFM4mXGZJEXEEGmOINwcRogxhBBjsDfwDZaOcOqN9evXcfy9fZ3epygK7728/qrbr/y041E9WN02\n75fL+912yb+tbiu2i/92WWlqr6XS0YrN48CuaKiKAhbAcnF03QI1n3q/LmHQGTAoBgw6PQadAQXl\nkmksFQ0Nl+rGeZ03I4shiFhLDBkRacxOnE5CSDxpYclEB/nv3ImEvxBX6IsCJX1F1VQa7c1UWaup\nbPd+VbXXUNVejd3j6HicSWck1hJDnCWW0dFZWAinoMjN8dN2VLuZm8YMZ8msNBJjQvqk3derB9DV\nlTx6nZ4wUyhhps7XwqutdbjLT+KpPom7/CQ42gEF/bBR6NOnoKaOx2EJxe62Y3Pbsbrt2N02bG47\ndo8Dt+q+8OXBrbpxad7/RvO+QekU7ycaHQoGnYEgvZkgQ9CF72bCTeFEmiOIMIdh0vvvfMG1yJy/\nzAUPed3t80C8cKgrXB4XFe1VlLaWU+eu5WxtMRXtVZeNOMNNYQwLSSAxJJ7EkASGBccTHxxPuCkU\nRVGoabKxeXcRu45XAXDz+EQWz0ojPtLSp31JSIjgWtH0m9/8rtM34ev9nDXVg9pQhqf6XMeX1uqd\nR1eCI9Enj8WQNBZ98ji/r9bxFbnISwgf6+mFQ5qmgeq58OVGu/hvTQUU0Ok6viuX/bce9IZubedr\nc9soa62gtK3C+721nCprTceJ1GCjhaSQRGYnTveGfEgCiSEJhBgvr3V7cXqryaoxYe7jxKRPR6/X\nc9uk4dw1M42YiP6pmHWt9fvR0dE33jHU7URtqkCtK8HTUIpaX4Kntgjc3k86iiUCfcJI9OMWeMM+\ncnifLEsdaCT8hbiEpmmMzkjD3dpAbKiZmFAzsaFBxIaaSYmLxrb1NTSXHc1pA5cNzWlHc1rB7QLN\n07sX1xvBYELp+G4CgxGPwUiFQaHECCU6N6WKgzrtktG8zkxyUAzjoseTHJpISngK2WkjaWj1oOiv\n/Se+fv06/u2Vn5My8UEmZt+M6nZRfHQzT9w7ixV3zO1dX3rpWuv3V636OeBd9aO1N6C21KA2V6G2\n1FBlq8deW47aXHXhDRcwmNBFp2DMusU7nZOQiRIaG5BhfyWZ9pEpkCHvyj5rqorWWoOnsQKtpRa1\n1fultdahttZ1jBAv5fKoeAwWgiNjwGhBMVkor6nj4NHj1DQ2YwwKZtbNtzJ67ATQGbw7Nur1oDOA\nooCmXfhSr/quaR5wu9DcTjS3g3q3lWK1lRLVRjF2KhQ3ngtZFe7RSHG4SbY5Ge5wMdzhJsxznWWT\negOK0QLGIBSTBcUYBCYLVo+e3GNFaMHDsLoVKsvyKS44SnNbK0Fhkbz22/9GMQWhGIM6ns+F1Tn+\noGkauOxojnbvl9PK7k+3smXjenROK5mJsdw8dQKJkSFo7U1o1ubL32z1RozRw9BC4tBFJaGLSUEf\nnYoSHo+i67sCKX1JNnbrBQnCoU9ztBPqrKah4BRqQzlqYzlqUyV4LlnNYgxCFxaHLiwWJSwWXVgs\n+46f5r/+9A55RWUYQqP5zvdXsvz+hzqe4qvdJNtd1o613EUtJRS3lNLu8pbwM+mMpIYnkx6eeuEr\nhaigSG+/NBVcjks+hXi/ay4bOG2EmDVaG5q+uO/CpxRbWxstzS1oThsmnJg1B0Fd/vyvgN7gfUPR\nG71vbHqj99OF/sK/L526uiJaNC5Mi3lcaG6X92fgdqJ5XOBxXvX4S7XYnDTYPIQlJJOQnoMuJAol\nNAZdRAK6iASUkCji4yMC6ndbwr8XAi0IYWj3WVNV1IYSPJX5eGrP46ktRGuu7rhfCYlGF52ELioJ\nfVQSuqjh6MITwBzS7RFtT04Ka5pGjbWW883FnG8u4nxzMVXWGm/bUBgWEt8R8unhqSSGJKDv4Z7v\nV/6cz5Q08v6uL0ol3jE9hRe/u5ySwrPodQrBJgOhZu9XRkoSf/jP/7rkjePCG4zqBo/7Qlh/8R2P\nC031fr8qwK/8/6rTe6ez9Ebvm4Xh4ncTiikYzMEo5hAUcwjbd+7iR6+8TGVDCzaXd5R/vTfYofy7\n3RkJ/14ItF8WGFp91jQVtb4UT0Ue7orTeKrOgNM7EldCotDHjUAXl0H0qDG0GRNQzL5bpnit1SiK\nolBd3Qx4V98Ut5ZdCHpv2F8c1QcbLGREpDEiIp0R4amkhidjMfju5GpcXBg1NS2cKmpkw65C8sua\nCQ8xceeMVOZOHk6QyTCg98KH7r/BDvTfbV9fOyKrfcSA4u+LozSXHXfZSdzFR/CUHkWztQCghCdg\nzJiBfngO+sScy/ZOD44Lo93HodDZahRzhIXsWeNYf3YDhc3FlLSW47kwLx0fHMv42DFkRKSRGZFO\nfHCcTy6G6oymaew7VcXaD05zvqKFqDAzjywYxa0Th19WKrEv98XpiYG0XXNvDcRrR2TkP4BHCv7g\nzz77aySp2dtwnd+Pu+ggnoo87/SDyYIheTyG1Inoh49GFxp9zef7ss8X39zKysuISIkmJiuBmKwE\nYrOHEZrgvRLUoDOQFpZMRkT6hdF92jUvNPIlVdM4nO8tlVhS7S2VuPimNG4en4jRMPhOeg6lkb8/\nrh2RaZ9eGMi/LP7izz778hdcc9lxFx3CdW4PnrKToHlQIhIwpE32Bv6wUShd3FvFF322ux38z6b/\n4S9b/0JERgwxWQkYg83e+5qstJe2MDNzGvfMuYeUsKQubxrWFTf6NKWqGvvyqtmUW0x5XTsJURYe\nviOHsakRPimV2F+6O5gYyH/PXZkm7C6Z9hEDRm8/pncUxsj7DHfhIfA4UUKiMU1YhCFzJrqY1D5b\nn91gb/zixGxTEWVtlWhhGln3TqKlrIGSXeeoz6+mLr+KKGOE3678vd50wT333s/eU9Vs3F1MdYOV\n4bEhPLNsDNNHxzMsYfCvfBno01Ld0d9FZzrjs/D/5JNP+PWvf42maWiaxje/+U3uuOMOXx1eDAI9\n/QVXbS2483fhzPsMrbkKTBaM2bdgGHkT+oSR3brytSc8qofytkoKLjkx2+TwjsZMehPp4ancmT6P\nH379e9TnV+GyXr5Bl1XxX8iuWvXSZSNfAIfTxZt/3sremmTqmu2kxIfy7L3jmJLtn1KJ/Wn58gcH\nZdhfqb+KzlyPT8Jf0zR+8IMfsHbtWrKyssjLy+PLX/4yCxYsQDdEL7AQV+vuL7inoRTnsY9wn9sN\nqht9wiiMk5dhyJiGYjD7rZ1Wl5XClhLON3mDvqilBOeFXSyjzJFkRqR75+sj00gKSexYbvmjuueu\nCn7w7+jt0k9NOoOJ1HELyJx+H5awWMKCjTyyIIuJI/1bKlH03kD8FOOzkb9Op6O11TsCam1tJT4+\nXoI/wHTlF1zTNDxlJ3Ae+xBP+UkwmDDm3Ipx7Hz0UUk+b5OmaVS11rC/8mTHNE5lu3ftv07RkRya\nyKzhM8iMSCMjIr3jIqrO9MfoLSkpmcqqGtIm3knG1HsICo2mofwUFYf+wu83/1VCfxAZaJ9ifBL+\niqLwq1/9imeffZbg4GDa29t56623fHFoMchc6xdc01TcRYdxHvo7an0JSnAkphn3Y8qZixLku5Uw\nLtVNaWuZN+gvjOxbXW2Ad8/0EeFpTI2fREZEGmnhKQR14xNGX4/ebA43D33jZxwv82CyhFNXcozD\nm9dgrS9gzZpXJfhFr/hktY/b7eapp57iW9/6FlOnTuXgwYN873vfY9OmTYSE9M3+32Jg0jSV9jN7\nadr5Ls6aYozRiUTOvo/QcXO8WwT0Upujnby6c+TVFXCmtoCCxhLvnupAQmgc2bEZZMdkkh2bQXJE\not/W1vtSm9XJhp3neX/nedpsLhLD3Xz+91fJO7KD1NRUVq1axYoVK/q7mWKQ88nI//Tp09TU1DB1\n6lQApk6disVioaCggAkTJnTpGLLUs2/0ZZ/dpcdx7F2H2lCKEjGMoNufwZA5E7tOj73BDti7fcxG\nexPnmgo511xIQVNhxxSOXtGTGpbMbcmzO9bXh5vCgC/6XF/X7svu+VyL1cnW/aV8fLAMu9PD5FGx\nLJ2dzojEcHj28sUTN/oZyu/20DcglnoOGzaMqqoqzp8/T0ZGBgUFBdTX15OamuqLw4tBxlNXjGPv\nOjzlJ1HC4i6E/k3d3l1R0zSqrTXesG8qoqC5sKO+aZDezIiINKYlTCIzYgRp4SmYfPBJwp+utV6/\nqc3Bln0lfHK4HJdLZWpOPMtmp5MS7/8Lw0Tg8kn4x8XF8eKLL/Kd73ynYx7yJz/5CZGR1z55JoYe\ntb0Rx76/4j6bC+ZgzLO+jHHMvC5P71wM+zONBeQ3nuNcUyFtLu9oPcwYSmbkCOalzCEzMv2yVTiD\nQWfr9f/1317gRE0I5W0heFSVm8YksGRWOsNjZapU+J9c4RtAHxPBP33WVA+uE9twHPwbeNwYxy3E\nPHlplzZSq7c1kt94jjON58hvPEez09u26KAoRkVmMDJyBJmRI4i39LwAx0D4OV969bMlPJ6RM5aT\nMnYeiqLj1knJLJ6VRkJU8A2O0nUDoc99LdD6PCCmfUTgclfl4/j8v1EbytCnjCdo9qPoIhKu+Xi7\n28GZxnOcrM/jTOM56mz1gHdknxWVSXb0SLKjRhITFD1gV7P0ZPO68vIyQiKHM3Lm/SSNvg1NVSk5\nvo3zB/7GH86d66OWC/EFCX/RI5qjHceev+A6swMlJJqghd/CkD7lqsD2TuXUcrI+j5P1eZxrKsSj\neQjSmxkVlcnc5JvJjhpJYkjCgA37S/Vkd8by2jZm37eSyJTJqB43RYc3cf7ge9jbGkhOTumztgtx\nKQl/0W3ukiPYd/4RzdqMaeJiTFPuQTF+sV5e1VQKmoo4Unuc43Wnqbc3AJAYksDtKbcwNiaHjIg0\nDD7c/KyvdLbdgs1mY9Wql64K/+KqVjbmFnEwv5bY9KkUHNpA/t71OK3erSP6+/J+EdgG31+f6Dea\nox377j/hzt+FLioZyx3fRh83AvDuj5PfWMDh2uMcqz1Jq6sNg85ATtQoFqbdxpjoHGIsUTd4hYGv\nK5vXFVQ0s3FXEUcL6rGY9Sydnc4d01PYsrmeVWe3Um5rGRCX94vAJuE/RPi7iIq7/BT2T95Cs7Vg\nmnI3psnL0HR6zjUVsq/qIIdrjmN12zDpTYyLyWFS3HjGxuR06wraweB6m9fllzaxYVchJ4saCQky\n8KU5I5g/NZngIO9qp4F2eb8IbBL+Q4A/qwRpHjfOA/+H8+gH6CITsdz5T9QFh7CveDv7qw5Rb2/E\npDcxMXYck+PHMzo6a8Cvt++Nzvb3GT5yGvMe+hd+tvYQ4cFGHpibydzJSVjM8uclBi5Z6jkEloZ1\np4hKd/qsNldj2/4mam0hSvatnBo1jp2VBzjfXISCQk70KGYMm8LEuHGY9Saf9MUffP1zvvgpy2mM\nZ+ycFYTEZhAZauKumWncOmk4ZmP/X38wVH63uyPQ+ixLPYVfap26zu3BvvNtGowGDk29lT3WUtry\n8oi1xHBv5mKmD5tMpDmix8cfrFRNY8T427nn2yMormolJtzM4pvSuGVCIkZD/4e+EF0l4T8EXGse\nOjKy+ydYNdWNY886Cs5u57Ph8ZwyuqH5DONjxzAn6SZyokcNis3RfE1VNQ6cqWFjbhFlte3ER1p4\n4q4cZo0bNqhLJYrAJeE/BKxc+QLf/vY3cLlcl93e3t7G+vXrujzv72lv5Ohnr7Jda6AwJZpgg5E7\nkm7llqSZRAcN/pU6PeFRVfaeqmbT7mIq660kxgTz9NIxzBgTj17qVYhBTOb8h8gcYU5OOg0NDVfd\nfuW8f2d91jSNowXb2VTwARVGHZF6C/NHzGf28JlDYrVOT37Obo9K7okqNu0uorbJTnJcKMtuTmdq\nVhw63cC/GG0o/W53VaD1Web8BQCNjY2d3n6jef+zjQX8/eS7FDobiFF0rEhZwIzMeYPyAixfcLk9\n7DxWyQd7iqlvcZA2LIxv3TeKiaNih1x9XBHYAvMvfAjqbvH08rZK/nZuE6cb8gl3e1juCePW276L\nIdi/J3H9fT1CTzlcHj47XM4H+0pobnMyMimCx+/MYdyIgbvHkBC9IeE/RHS1vmybo52/nHmPneW7\nsWgKi+tbmRM3hbD5T6Do/fvr4M/rEXrK5nDzyeFytuwrodXqIic1kmeWjiEnLUpCXwxpEv5DxI3q\ny6qayu6K/Wz4/EPanFZmOQ0ut4dQAAAgAElEQVQsKKsgctpyTBOX+DzoOhvhd2dfHH+z2l1sO1jG\n1v2ltNvdjBsRzdLZ6WSlSA0KERjkhG8AnCCqsdaxNu9dzjUVkhOVxp3nSxnWVE/Q7c9gzJju89e7\ncoQP3k8hVwb/RYqiUF3d7PN2XHTpz7nV6mTrAW+pRJvDw6SR3lKJGcPD/fb6/SFQfrcvFWh9lhO+\n4ppUTeWT0s/ZcH4LBp2eL6cuZPL+D1HtViyLv48hMdsvr3utEb5er8fj8Vz1+Gudl/Cl5nant1Ti\noXKcLg9Ts+NYOjud1IQwv7+2EAORhP8QVW9r4O1Tf+Z8czHjY0fzYMw0TNt+i2IwEHz3v6KP8V99\n5WutMPJ4PFd9AvD3tsaNrQ7+tquID3cX4faozBydwJLZ6SRJqUQR4CT8h6BDNcf4U95f0TT4hzEP\nM9llwr7lVbCEM/yxF2ly+zf4rrXyKDk5pWPu39+rfeqabGzeU8znxyvRNJg1dhhLZqWREO27UolC\nDGYS/kOI0+Pkr2ffZ1fFPtLDU3li7CNE1pVh++hX6CKGYVn8fYxRw8DP86LXW3nk722NqxutbMot\nZvfJKhQFbhmfyKNLxqLrZLpJiEAm4T9E1Nsaeev4Hylrq2Bh6lyWZSxCKzuB7aNX0UUNJ3jJD1CC\nen5yqDtutPLIHyrq2tm4u4i9p6ox6HXcPjmJO2emEh0eRFx0cECdCBSiKyT8h4CzjQW8fvD32Bw2\n9vzHVnbXbiLqnwuY0n4UXXQSwYv/uc+C/6K+KlxSUn2hVOKZWkxGPYump7JoRgoRoYN/Wwoh/Mln\n4e9wOPjJT37C7t27MZvNTJo0iZdfftlXhxfXsKNsN+vOvEdrdRM7f76ZtspmFuQkMr5pP82mKJKW\n/ADFPPRObhZWtrBhVxFHztURZNKzeFYad0xPISx44NYVEGIg8Vn4/+IXv8BsNrNlyxYURaGurs5X\nhxadUDWV9wo283HJDhpPV/PpLzbhtjm5bVQCrz48k5MVTazceoQdTw6t4D9b1sSGXUWcKGwgJMjA\nvbeMYP60ZEKChm71MCH8wSfh397eznvvvcdnn33WcaVobGysLw4tOuFRPfzP6XfZX32IW5Nm8+1H\nvoKmqkxPj+GNR24iv6aFr/xxF20Od3831Sc0TSOvxFsfN6+kibBgI/fPzeR2KZUoRI/55C+ntLSU\nyMhIXnvtNfbu3UtISAjf+c53mDZtmi8OP6T0dmMzu9vB7078D6cb8lmWsYhFafNYPTyJKLWV/3x0\nNmWN7Tzx9ue02l0kJ6f4sSf+p2kaJwob2JBbxLmyZiJCTTw8byS3TUrCbJKqWUL0hk+2dzh58iT3\n3Xcfv/zlL1m2bBlHjx7l61//Olu3biU0tG9PNA5ka9eu5ZlnnsFqtXbcFhwczFtvvcWKFStu+Hyr\n08aqHa9yrqGIr01bwbyMmwH4vz/8hrRzm2lzuHjorc+obrV367gDjaZp7DtZxTvb8jlX2kRspIX7\n541i4YxUTAOgPq4QQ4FPwr+hoYE5c+Zw4sSJjmmfxYsXs3r1asaPH9+lYwTC3j7dKbR+JZvbxmtH\nfk9JaxlPjnuUSXHjAFBbarG+vwqbzcYTa/ez/3TBdT9RDOT9T1RN4+CZWjbsKqKsto24yCCWzEpn\ndi9LJQ7kPvuL9HnoGxB7+0RHRzNz5kx27drFLbfcQmFhIfX19aSlpfni8ENGTwut29x2Xr8Q/E+N\ne4yJcWMB0OxtWD/4dzSPi+jlP+KJ6F1UXJhSWrXqJaD/tkruDo+qsu+0tz5uZb2VYdHBPLlkNDeN\nTZBSiUL4ic/Olr300kv88Ic/ZPXq1RgMBn7+858THj60dkrsre4WXIGLwf87ilvLeGrco18Ev9uJ\ndcuv0NrqsCz5F977ZNeA2yv/Rtweld0nqti0p5iaRhtJcSF8/Z6xTMuOHxSlEoUYzGRL5z78mHit\nrY7XrHm104B2eVy8fvT3FDQXXTbVo6kq9m2v4y46RNCCZzFmTO/ylNJA+Gjscqt8frySzbuLqW+x\nk5YQxtLZ6UzO8k+pxIHQ574mfR76BsS0j+ia7mx7oGoqb596h7NN5/mHMQ9/EfyahmPPn3EXHcQ8\n65GO/fh7OqXUlxwuDzuOVPDB3mKa2pxkDg/nsUVZjM+IkapZQvQxCf8+1pVtDzRN4938v3Ok9jjL\nRy5lxrApHfe5jn+E68RW1h2t4of/9kDHG0hPppT6it15oVTi3hJarC6yUiJ5cukYxkipRCH6jYT/\nALS15FN2lO9mYepc5qXe2nG7u+QI9j1/5qPTlfzwr7vRtC/m9h9+eAXvvLO2T/fKvxGr3c3HB0v5\n6EKpxLHpUSydnU52alS/tUkI4SXhP8AcrT3B+wUfMjV+Ivdk3tVxu6exHNvHb5Jf285z6/Zx6Zka\nm83G1q1bWLPm1T7dSfNa2mwutu4vZdvBMmwONxMzY1g6O53MpIg+b4sQonMS/gNIWWsFb596h9Sw\nZB4d/WDHlIhmb8P24a9QDCaeensndtfVe9OXl5f12U6a19LS7mTL/hK2HyrH4fQwNctbKjFtmJRK\nFGKgkUXUfWj9+nVMmTKWhIQIpkwZy/r16zrua3G28uaxtwk2WPjahH/ApPduVKapbmzbXkezNmK5\n49vowmI6PXZ/zu03tjr487az/OA3uXy4p4SJmTH8+MkZ/ON94yX4hRigZOTfR65c5nnpOvx7v7Sc\nP5z4E22uNp6b+iwR5i+uj3Dk/glPxWmC5j6NPmHkdatk9bW6Zhsf7Clh57EKVBVmjU1g8aw0EmOG\n1k6iQgxFEv59ZNWqly4LbPDO1a9a9RLGSeHkNxVw7p2jTH9obMd8/d1jh+E6tR3jhLswZnn38emP\nKllXqmm0sml3MbknqgC4eXwii2elER9p6bM2CCF6R8K/j1xrvb0nXseW4u2UfJbP4fd2A95PBW+t\nXsn8p+ZgShqLecYDlz2nv+b2K+vb2ZhbzN5T1eh0CnMnJXHXTd5SiUKIwUXCv490tg4/ODaUm745\nn/byZvb/7rOO28ODjKy5bxL1bXZGzPsaSj/vb1NW08aG3CIO5NVgNOpYMC2ZO2emEimlEoUYtCT8\n+8hVc/WKwk3fWojZEsTWH65HvbCCR1Hgl/dPY1i4hUd+v4Mtz/Xf/khFVd5SiYfPflEqceH0FMKl\nVKIQg56Efx+5cq5++mO3EZOdwCOj7+dzwwbaaAbg67dmMz8nkRc2HKFW658Tp+fKm9mwq4jj5+sJ\nNhu4++Z0FkxLIdQipRKFGCok/PvQxbn6kpYyfnHwNSbGjWPmsKkdnwomJ4by3flj+PvREtYfq2TN\nmlf7rG2apnGmpIkNuUWcLm4k1GJk+W0ZzJuSLKUShRiC5K+6jzk9Tt4+9WfCTWF8Ofs+FEVh+fIH\nMakOxpd/yPm6Vt7YX3vNnT59TdM0ThY1sGFXEWfLmgkPMfHg7SO5fbKUShRiKJPw72Mbz39EtbWW\nb096hhBjMACapjI/tAZPWDApj68m9//5/4ItTdM4WlDPhl1FFFa2EBVmZsXCLOZMSJRSiUIEAAn/\nLuht0fWLilpK2F66k1uGzyQ7emTH7a7jH+EpPY75lsfRR/s3+FVV40Cet2pWSU0bsRFBPH5nNjeP\nS8RokAu+hQgUEv43cL0rc7vzBuBS3fzv6XeJMIdz78glHbd7aotw7HsXQ/pUjKNv923jL6GqGvvy\nqvlwXyklVa0kRFl4csloZo5J6FV9XCHE4CThfwPXuzK3O+G/pWg7le3VfGPCE1gM3ouiNKcN28e/\nQbFEEHTrE37Z297tUdlzsppNu4uobrSROiyMZ+4ew4ycBCmVKEQAk/C/AV9UyKpoq2JL8XamJ0xh\nXOzojtvtu/4XrbUGy9LnUYJ6Xo6tMy63yq4T3lKJdc12UuND+ccvjeOO2RnU17f59LWEEIOPhP8N\n9LZClqZprMt/jyC9meWjlnbc7jq/H/fZXZim3IMhMdtn7XW6POw4WsEHe0tobHUwIjGcRxZmMTHT\nWypRRvtCCJDwv6He7qJ5sPoIZ5vO83D2lwgzeUf3qrUJx84/oosbgWnKMp+00+508+nhCj7cV0JL\nu5NRyRE8sTiHsenRUipRCHEVCf8b6M0umja3nf87t5HUsCRuHj4T8H4SsO/4A5rbgeX2p1F0vfsR\n2BxuPj5Yxkf7S2mzuRidFsU37hkrpRKFENfl8/B/7bXXePXVV9mwYQNZWVm+Pny/6OkumpsLt9Li\nbOOZCf+ATvGuqHGf2Ymn5CjmWV9GHzm8x21qt18olXigDKvDzYQLpRJHSqlEIUQX+DT8T548yZEj\nR0hKSvLlYQelqvYaPi3bxazE6aSHpwKgttRi3/0n9Ik5GMct7NFxW6xOPtpXyvZDZdidHiaPimXZ\nzemkD+u/DeCEEIOPz8Lf6XTy4x//mH//93/n8ccf99VhB633CjZj0hm5O/NOwHsVr/2z3wEQNPcp\nFKV7a+ub2hx8uLeET4+U43KpTB8dz5JZ6aTE+3aVkBAiMPgs/H/9619z9913k5zcf7VkB4r8xgKO\n153inoy7Ok7yuk5+jKfyDEG3PYkuLLbLx2posbN5TzE7jlaiqhozxySwdLaUShRC9I5Pwv/w4cOc\nOHGC73//+z0+RkxM/4xg4+J8W2Bc1VTeP7yZmOAoHph8JyaDib++/Rajizazt7CWH//ns6xatYoV\nK1Zc9zhV9e38dftZPt5fgqbB/Omp3D9vFImxvQ99X/d5MJA+B4ZA7HNP+ST89+/fT0FBAfPnzweg\nqqqKJ598kp/+9KfccsstXTpGfX0bqqr5ojldFhcXRm1tq0+Pua/qEIWNpfzDmIdpbnSw/q//TeiB\ntXiSIvh/fz9MZbONp59+mtZWe6cnkSvr29m8u5jdJ6vR6WDOxOEsnplGTEQQaGqv2+uPPg900ufA\nEGh91umUXg2afRL+zzzzDM8880zHf8+bN48333xzyKz26SqXx8X7BR+SGpbEtIRJABz46+v8cF4G\nL2w4QmWz91qBzraHKKttY2NuEftP12A0eEslLpqRSlSYlEoUQvierPP3oV0V+2h0NPHo6AfQKTpU\naxPP3pTMgeI61u47f9ljL24PUVzVyobcIg7l12I26bnzplQWTU8lPERKJQoh/Mcv4b99+3Z/HHZA\nc3qcfFj8MaMiM8iO8m7X7Nj1vwSbDfzr3w6hXTGjNWr8zfzq3aMcK6jHYjawbHY6C6dLqUQhRN+Q\nkb+PfFaWS6uzjafGPYaiKLgKD+IuPEBxaA6V7Vs6HhedNIbs2V8mJmU8BeXNfOnWDOZPSSY4SH4U\nQoi+I4njA3a3na0lnzImOpuRkSPQnDYcu/4HXXQKk+/7Af9uGcl//O4dokbeTkzyWEx6lXtuzeT2\nyUkEmeRHIIToe5I8PvBJ6S7aXVaWZtwBgOPge2jWZoIWfpNj55s4Y81g5NxvEhVm5s6Zqdw2cbiU\nShRC9CsJ/16yue18XLqD8bFjSAtPwVNfguvEVpoSZ7BmcyPF1SXEhAfx2KJsbhkvpRKFEAODhH8v\nfV6+B5vbxl3p8/F4PNR99DvQTKw+lU5opJsnFucwa+wwKZUohBhQJPx7welx8XHpDnKiRlFWbGDP\n7v9mMSVsYi4rlk5mxuh49DoJfSHEwCPh3wu7yvfR6myjuCCB/KrD/FvUHqzhI3jgwccl9IUQA5qE\nfw+43B4+PVLG3+s/wmOPJEwbxnfH7MdS5Sb4jicl+IUQA56Efzc4nB4+PVLOh3tLaAsqxJRpY1nS\nEhbFRmN7fz/GCXehj5ZdTYUQA5+EfxfYHG62H/KWSmy1ushOjaAtpRKLKZG7xkzF9vdXUIIjMU+9\np7+bKoQQXSLhfx3tdhcfHyhj64FS2u1uxmVEs2x2Oi5LNa8frePxkQ/hOZuLWltI0NynUYxB/d1k\nIYToEgn/TrRanXy031sq0ebwlkpcOjudEYneUomvHfkrEaYwpkRl4di2El18BoZRs/q51UII0XUS\n/pdobnOwZV8pnxwux+nyMDUnnqWz0khN+KJARGV7Nacb8lmWsQj16AdothYsi/6p22UZhRCiP0n4\n4y2V+MHeEnYcrcDtUZk5JoEls9JJ6qRq1ielOzHqDMwOy8C57c8Ysm5GH5/RD60WQoieC+jwr6pv\nZ+2HeXx+vBJNg1njhrHkpjQSooM7fXybs519VYeYMWwqxgPv49YbMM94oI9bLYQQvReQ4V/dYGXj\n7iJvqUQFbpkwnMUzU4mNtFz3eTvL9+BS3dxqTMBdvAHTjAfQBUf2TaOFEMKHAir8y+va2ZRbxN7T\n1Rj0OpbePILbJiR2qVSiR/Wws3w3o6OyiDr8IVpYHKbxd/RBq4UQwvcCIvxLqi+USjxTi8moZ9GM\nVBbNSGVkekyXCz4frztFs7OFB0KzURvKCJr/LIpeqm4JIQanIR3+5yta2JhbxJFzdVjMepbMTmPh\ntBTCgrtfH3dn+R6izBFknsxFF5eBIWO6H1oshBB9Y0iGf35pExtyizhZ2EBIkIF754xgwdRkgoN6\nNlKvsdaS13iWu4LSUNrPYp73dRRF8XGrhRCi7wyZ8Nc0jbziRjbkFpFX0kRYsJEH5mYyd3ISFnPv\nurmzfA86RceUsycwpE3GkJjto1YLIUT/GPThr2kax883sDG3iHPlzUSEmnh4/ihumzQcsw9KJTo9\nLvZUHmC8EkaYswbTTFnaKYQY/AZt+KuaxtGzdWzILaKoqpWYcDOP3ZHFLRMSMRp8Vx/3UM1RrG4b\nM8qrMObchj5yuM+OLYQQ/cUn4d/Y2MgPfvADSkpKMJlMpKWl8eMf/5jo6GhfHP4yqqpx4EwNG3OL\nKattIy4yiK/clcPscf4plbirYh9xmoEMF5hk104hxBDhk/BXFIWnnnqKmTNnArB69Wp++ctf8pOf\n/MQXhwfAo6rsO1XDxt1FVNZbGRYdzFNLRzNzTILfiqdUt9dwvrmIu+rbME+8Sy7oEkIMGT5JzcjI\nyI7gB5g0aRIVFRW+ODRuj8qOoxWsfGsv/7nxFHqdwtfvGcsrT81k9rhEv1bN2lN1EEXTyKxtZfS9\nTzJlyljWr1/nt9cTQoi+omiapvnygKqq8tWvfpV58+bx+OOP9/g4TpeHbftL+Ov2s9Q22hiZHMGD\nC7KZOXYYOp3/l1l6VA9f/ct3SLNZqfivnfx+17mO+2JiYvj1r3/NihUr/N4OIYTwB5+f8H355ZcJ\nDg7m0Ucf7dbz6uvbUFUNh8vDZ0cq+HBvMU1tTjKTwlmxIIvxGdEoikJ9fVuv27h+/TpWrXqJ8vIy\nkpKSWbnyBZYvf/Cyx5yoO41N52FkVTM/33v+irbW8/TTT9Paar/qeQNdXFxYl69qHiqkz4Eh0Pqs\n0ynExIT2+Pk+Df/Vq1dTXFzMm2++ia6b0zF2p4ePD5SyZV8JLVYX2SmRPLV0DKPTonx6QdX69et4\n7rlvYbPZACgrK+W5574FcFmQ5xZsI8Sj8sH7R3C41auOY7PZWLXqpUEX/kIIAT4M/zVr1nDixAne\neustTKbub5/ws/89SFFVK2NHeEslZqX45+TqqlUvdQT/RVcGeaujlRNtpYyva+PFA4XXPFZ5eZlf\n2iiEEP7mk/A/e/Ysv/3tb0lPT+fhhx8GIDk5mddff73Lx0hPDOfRO7LJGB7uiyZd07UC+9Lb955+\nH48Cw/UZGExBuK54s7goKSnZL20UQgh/80n4jxo1ijNnzvTqGF+5KwdV9em5504lJSVTVlba6e0A\n//fXdzhm3keMpvHC6j/y8MMreO+9/6OxseGyx1ssFlaufMHv7RVCCH8IuMKzK1e+gMVyedGWi0G+\nfv06tr/3KrUhZspyz1FSWso776zlJz/5Ob/5ze9ITk5BURSSk1NYs+ZVme8XQgxag3Z7h566GNid\nrfaZPnUs3/jObA5qGtv+dhj44nzAoUMnJeyFEENGwI38wfsGcOjQSVRVZeXKF1i16iUSEiKYHANF\n8WG4C+uwN7Z3PF5O7AohhpqADP+L1q5dy3PPfYuyslIUNB5YMpF6k4EjW09c9jg5sSuEGGoCOvxX\nrlzZsexz6YQUalKi0dweyvZ+sbxTTuwKIYaigA7/kpISAHQKPDs3h8PBJsoPFuOyOgHQ6/Udc/6y\np48QYigJ6PBPTU0FYMn4ZLTUGGwmAyW7zhIVFY3FYsHj8QBfXAUsbwBCiKEioMN/8eLF6HUK/zg3\nh50GBZfVQdPpGhSFa14FLIQQQ0HAhv/69ev44x//yJ1jhpMRH05+uIWKg8WgQkNDQ6fPkVU/Qoih\nImDDf9Wql7BarTw7N4cdTjeKxUTpngLa29uv+RxZ9SOEGCoCNvzLy8uYmzWMnGERfOB047I6qT52\n7ZG9rPoRQgwlARv+SUnJfOO2LEqarbgy46g4VIzq8nT6WNnOQQgx1ATc9g4X/eIH32Jayx5+dLQE\n89R0yvYWdPq45OQUDh062cetE0II/wrYkf+sSBtWj8LZqBDcdhdVR67e6VOmeoQQQ1VAhv+Wv/wO\nT8lRfvPpSRKmpnc65SNTPUKIoSzgpn3Wr1+Hc9c62kbFsbnRyvQIC2VX1OhVFEWmeoQQQ1rAjfz/\n6z9+xl1jEvnzvkIiJ6XgdrioOlJy2WNkSacQYqgLuPBfNjIUt6ryX7nnSJqWTvXxMjwOd8f9Ms8v\nhAgEARX+qrWZ+6em87fDJTiiQwiODaPiQFHH/TLPL4QIFAE15+868REmg47/3l9C0qx0NFWj8nAJ\nFotFQl8IEVACZuSvOW04T23HOGIa//Rvq0mblUX92WriwmIl+IUQASdgRv6u/M/BacM0cTHzwqPZ\nnnuAr034CrO/Mau/myaEEH3OZyP/wsJCHnroIRYtWsRDDz1EUVGRrw7da5qq4jz+EbqEkejjMzhW\ndwqAaUkT+rllQgjRP3wW/i+88AKPPPIIW7Zs4ZFHHuFHP/qRrw7da+7iQ2ittZjGLwLgeN0p4oNj\nSQof1s8tE0KI/uGT8K+vr+fUqVMsXboUgKVLl3Lq1Klr7ovf11zHtqCExWFIn4rNbSOv/ixHNu9F\np9MxZcpYqdAlhAg4Ppnzr6ysJCEhAb1eD3hr38bHx1NZWUl0dHSXjhETE+qLplzFXp5Pa/VZYhY+\nQURCBD//06/Q9Bp5nxxH0zTKykp59tmnOXbsIG+88YZf2jDQxMWF9XcT+pz0OTAEYp97asCc8K2v\nb0NVNZ8f17bjb2C04EiaQW1tK5sPbSMiJ476/OqOx2iaxptvvsmECVOH/KqfuLgwamtb+7sZfUr6\nHBgCrc86ndKrQbNPpn0SExOprq7uKHju8XioqakhMTHRF4fvMbW1DnfhAYyjb0MxWVA1lfBRMVQf\nKwXt8jcaTdOkRq8QImD4JPxjYmIYPXo0GzduBGDjxo2MHj26y1M+/uI8uQ0A07iFAJS1VhAUGUzl\nFXv5XCQ1eoUQgcJn0z4vvvgizz//PG+88Qbh4eGsXr3aV4fuEc1pw3X6MwwZ09GFxgBwsv4MADXX\nKNcoG7oJIQKFz8I/MzOTd99911eH6zXXmR3gsnUs7wQ4WZ9HWlgKX75vBW+//Xu0S6Z+ZEM3IUQg\nGZLbO2iqivPEVvTDstDHZwDQ5mqnqKWEMTHZrF69hjfe+E/S0tJQFEU2dBNCBJwBs9rHlzwlR9Fa\n6zDOfKjjtrz6fDQ0xsbkALB8+YN8/etPBtTqACGEuGhIjvydJ7ehhERjSJ/ScdvJhjOEGINJC5d5\nfSGEGHLh72mqwFN+EuPouSg670VnqqZyqv4Mo6Oz0ClDrstCCNFtQy4JXSe3g86AcfTcjttKW8tp\nc7V3TPkIIUSgG1LhrzltuPI/9y7vtIR33H66IR+A0dFZ/dU0IYQYUIZU+LvO7QaXHdO4BZfdntdw\nluTQ4YSZ/LN/kBBCDDZDJvw1TcN1chu6uBHo4jI6bnd4nJxvLiYnelQ/tk4IIQaWIRP+nso81MYK\nTGPnoyhKx+3nmgrxaB5yoiT8hRDioiET/q4T21DMoRgyZlx2e15DPgadgczIEf3UMiGEGHiGRPir\nbfW4iw9hzLkVxWC67L4zjefIiEjHpDf2U+uEEGLgGRLh7zr1CQDGMbdfdnuLs5XytkpGy5SPEEJc\nZtCHv+Zx4cr7DEPqJHRhcZfdd6bhHADZ0SP7o2lCCDFgDfrwdxceRLO3Yhw7/6r78hrOEmIIJiUs\nqR9aJoQQA9egD3/X6U9RwuLQJ4257HZN08hrPEtWVKZs6SCEEFcY1KmoNlXhqczzlmm8IuCrrbU0\nOZplfb8QQnRiUIe/M+9TUPQYs2656r78xgIAsqJkvl8IIa40aMNf87hwn/kcQ/pkdMGRV91/tqmA\nSHMEcZaYfmidEEIMbIM2/N2FB9EcbZft3nmRpmmcbTzPqMjMy672FUII4TVow/9aJ3oBqq01tLra\nyIrK6OSZQgghBmX4X+9EL0B+43kARkVm9nXThBBiUBiU4X+9E73wxXx/rCW6bxsmhBCDRK8LuL/0\n0kvs3r0bk8lEcHAwK1euZPz48b5oW6c0jwt3/q5rnui9ON+fE50l8/1CCHENvR7533rrrWzYsIH3\n33+fr33ta3z3u9/1RbuuqeOK3k5O9ILM9wshRFf0euR/++1fbKY2adIkqqqqUFUVna577ys6XddG\n6WrZUYzDczCmjO10ZF/WXkFccDRjYrK7dMyuvu5QIn0ODNLnoa23fVU0TdN81BZee+018vLyeO21\n13x1SCGEEH5ww5H/l770JSoqKjq9Lzc3F71eD8CmTZvYsGEDa9eu9W0LhRBC+JxPRv5bt25l9erV\nvP322yQnJ/uiXUIIIfyo13P+n3zyCT/96U/5wx/+IMEvhBCDRK9H/jfddBNGo5Ho6C/W1L/99ttE\nRUX1unFCCCH8w6cnfIUQQgwOg/IKXyGEEL0j4S+EEAFIwl8IIQKQhL8QQgSgIR/+hYWFPPTQQyxa\ntIiHHnqIoqKiqx7j8TovpTgAAATYSURBVHh46aWXWLBgAQsXLuTdd9/t+4b6UFf6/Prrr7NkyRKW\nLVvGfffdx86dO/u+oT7UlT5fdP78eSZOnMjq1av7roF+0NU+b968mWXLlrF06VKWLVtGXV1d3zbU\nh7rS5/r6ep555hmWLVvGXXfdxYsvvojb7e77xvrA6tWrmTdvHtnZ2eTn53f6mB7nlzbEPfbYY9p7\n772naZqmvffee9pjjz121WP+9re/aV/96lc1j8ej1dfXa3PmzNFKS0v7uqk+05U+79ixQ7NarZqm\nadrp06e1qVOnajabrU/b6Utd6bOmaZrb7dYeffRR7bnnntN+9rOf9WUTfa4rfT527Jh21113aTU1\nNZqmaVpLS4tmt9v7tJ2+1JU+v/LKKx0/W6fTqd1///3apk2b+rSdvrJ//36toqJCu/3227UzZ850\n+pie5teQHvnX19dz6tQpli5dCsDSpUs5deoUDQ0Nlz1u8+bNPPDAA+h0OqKjo1mwYAEffvhhfzS5\n17ra5zlz5mCxWADIzs5G0zSampr6vL2+0NU+A7z11lvMnTuX9PT0Pm6lb3W1z2+//TZf/epXiYuL\nAyAsLAyz2fz/27t7kMbhMAzgjwpOOgnWBHE4EA2OCoLo0oDGUgpKcXGwgzi4OThWlEI3Bwu6iugg\nuih0cJAOIaAVQchUwSLqED/AzaUhfW84KModd3+a2v+1eX/QoZDheSh96Eea1j1vLYh2bmlpwcfH\nB8rlMkqlElzXRSgUkhHZt5GRESiK8tdjqt2vph5/x3EQCoUq1x9qa2tDd3c3HMf57ThVVSv3FUXB\n8/NzXbPWimjnz05OTtDX14eenp56xawp0c6FQgGWZSGRSEhIWVuinYvFIp6enjA/P4+ZmRns7OyA\nGvSnPaKdl5eXcX9/j/Hx8cpteHhYRuS6qHa/mnr82b9dXV1ha2sLm5ubsqN8K9d1kUwmsbGxURmP\nIPA8D7e3t9jd3cX+/j5M08Tp6ansWN/q7OwMAwMDsCwLpmni+vq6Yd/Jf6emHn9FUfDy8gLP8wD8\neiK8vr7+9jZKUZQvVy51HKdhXwWLdgaAm5sbrK6uYnt7Gz9+NO6f34h0fnt7w+PjI5aWlhAOh7G3\nt4ejoyMkk0lZsX0RfZxVVYVhGGhvb0dHRwd0XYdt2zIi+yba+eDgALFYDK2trejs7EQ4HEY+n5cR\nuS6q3a+mHv+uri5omoZsNgsAyGaz0DTty3WIAMAwDBwfH6NcLuP9/R3n5+eYmpqSEdk30c62bWNl\nZQWZTAZDQ0MyotaMSGdVVZHP55HL5ZDL5bCwsIC5uTmkUilZsX0RfZyj0SgsywIRwXVdXF5eYnBw\nUEZk30Q79/b2wjRNAECpVMLFxQX6+/vrnrdeqt6vmn41/R+6u7ujeDxOk5OTFI/HqVgsEhHR4uIi\n2bZNRL/OAFlbWyNd10nXdTo8PJQZ2TeRzrOzszQ6OkqxWKxyKxQKMmP7ItL5s0wm0/Bn+4h09jyP\n0uk0GYZBkUiE0uk0eZ4nM7YvIp0fHh4okUhQNBql6elpWl9fJ9d1ZcauWiqVoomJCdI0jcbGxigS\niRBRbfaLL+zGGGMB1NQf+zDGGPszHn/GGAsgHn/GGAsgHn/GGAsgHn/GGAsgHn/GGAsgHn/GGAsg\nHn/GGAugn/F49A9+Az8OAAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "vGm6ipNT0Frz",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 68
        },
        "outputId": "cb2cf362-75bc-46d4-c4f8-9f6596775724"
      },
      "source": [
        "grid.best_params_"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{'linearregression__fit_intercept': False,\n",
              " 'linearregression__normalize': True,\n",
              " 'polynomialfeatures__degree': 4}"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 10
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "7FXeoFiP5RTB",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 274
        },
        "outputId": "e7d2c3d6-4340-44da-cf65-9ed449949cc1"
      },
      "source": [
        "import matplotlib.pyplot as plt\n",
        "\n",
        "model = grid.best_estimator_\n",
        "\n",
        "plt.scatter(X.ravel(), y)\n",
        "lim = plt.axis()\n",
        "y_test = model.fit(X, y).predict(X_test)\n",
        "plt.plot(X_test.ravel(), y_test);\n",
        "plt.axis(lim);"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAEBCAYAAAB2RW6SAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3Xl8VPW9N/DPLNknk2Uy2cgCCVsM\nATQoIlAooJAaUCktLlWs9fHp5dq9tffV16OC9dpS296rVW/LfXxUSluvtIoVRKqCIBiwrCEJCTBZ\nSMhMlplsk0wy23n+CIkEJmGSOTNzZs7n/XrlpSSTM9+fkU9+53d+i0IQBAFERBS2lMEugIiI/ItB\nT0QU5hj0RERhjkFPRBTmGPRERGGOQU9EFOYY9EREYY5BT0QU5hj0RERhjkFPRBTmGPRERGGOQU9E\nFOYY9EREYU4d7AI6Onrhdvu2gaZOp4HZbBWpotAi57YDbD/bL7/2K5UKJCXFjet7gh70brfgc9AP\nXUeu5Nx2gO1n++Xdfm9w6IaIKMwx6ImIwhyDnogozDHoiYjCXNAfxhJJVVmlCW8fMMDcPQCdNgpr\nl+RjQWF6sMsiGjcGPZEHZZUmvLGnGnanGwBg7h7AG3uqAYBhTyGHQzdEHrx9wDAc8kPsTjfePmAI\nUkVEE8egJ/LA3D0wrs8TSRmDnsgDnTZqXJ8nkjIGPZEHa5fkI1I98q9HpFqJtUvyg1QR0cRdN+i3\nbNmCZcuWYcaMGTh37tzw5+vq6rB+/XqsXLkS69evR319vT/rJAqoBYXp2FAyc7gHr9NGYUPJTD6I\npZB03Vk3y5cvx0MPPYQHHnhgxOeffvpp3H///bjrrrvw7rvv4qmnnsK2bdv8VihRoC0oTGewe8Bp\np6Hnuj36efPmISMjY8TnzGYzqqqqUFpaCgAoLS1FVVUVLBaLf6okIkkYmnY69FB6aNppWaUpyJXR\nWCY0Rm80GpGWlgaVSgUAUKlUSE1NhdFoFLU4IpIWTjsNTUFfMKXTaUS5jl4fL8p1QpGc2w6w/YFs\nv2WU6aWW7oGg/Ryk8vP/5Hgjtu05i/YOG1KSYvBQSQGWFmcHuywAEwz6jIwMtLS0wOVyQaVSweVy\nobW19ZohHm+YzVaf95PW6+PR1tbj0zVClZzbDrD9gW5/sjbK41qCZG1UUH4OUvn5X72Suq3Dht+9\ndQrdPf2iP79QKhXj7iBPaOhGp9OhoKAAu3btAgDs2rULBQUFSE5OnsjliChEcNqpZ1If0rpuj/7Z\nZ5/FP/7xD7S3t+Ob3/wmEhMTsXv3bmzatAn/9m//hldeeQVarRZbtmwJRL1EFERDvVPOuhlJ6iup\nFYIgBPUcLg7d+EbObQfYfrZfGu3/ySuHPYa6ThuF5zcuFPW9AjZ0Q0REX5D6kFbQZ90QEY3X0KIt\nS/cAkiUwfCT1IS0GPRGFFKmeFSDlldQcuiGikCL1GS5SxKAnopAi9RkuUsSgJ6KQwrMCxo9j9EQk\neW63gE7rACzdA5gzNQUHTjXDdcW0bIUCUCgUePL/HoXD5YbDOfgBACqVAirl4IdapURstBqa6AjE\nxURAExOBhLhI6BKioU+MQUpCNDQxEVAoFMFqql8w6IlIElxuN9o6+2Ey98Fo6YXJ3IcWSx/M3f3o\n6LHDPcqSH6VSgZSEaKQlxSJSrUSEWgm1WokIlfLydQW43G643AKcTjd6+53osA6gqc0Kq82JAYdr\nxPU0MRHITtUMf+RPSkBaUkxIhz+DnogCShAEmLv70dhixcVWKxpbrTCae9HaYRvRS9fGRSI9KQbT\ns5OgS4hCcnw0krXRSNZGIVEThdgoNdLStD4vmLINONHe1Y/2ThvauvrR3D5Y0/6Tl4bvChI0kZiR\nnYgZOUmYnaeDLiHap/cMNAY9EfmN2y2g2dyLBlMPLrZY0dg6+M++AScAQAEgNTkWk1LicOM0PXr7\nHTh5rg3dfQ5EqBRYcuMkv09ZjIlSD/fer+Ryu2Ey9+F8UxdqGjtRc7EDn59tBQDkpsfjxmkpuGm6\nHll6cXbg9ScGPRGJprffAcOlbhgudcHQ3IU6YzdsA4NDI5FqJbJSNbilIBXZafHISdUgS69BVOTg\nuRZSmx+vUioxSa/BJL0GS2+cBEEQYLL04dT5dpw434Z3P63Dzk/rkJsej8WzM3DrDWmIjY4IeJ3e\nYNAT0YS4BQHN7b2DoX6pG4bmLhjNfQAGH45m6TWYf0M68jO1mJKhRXpyLJTK0ce5//xhzajz46Ww\nEEmhUCBDF4cMXRxKbs1Fl3UA/6xuxaFyI7b/4xz+Z98FzJuhxx035yA3XRp75A9h0JMs8dzT8evt\nd6C2+XJv/VIXaq/orcdFq5E/KQG3FqZjaqYWkzO0iInyPl7KKk3o7Xd5/JpU58cnaKKwYl42VszL\nRoOpB5+WN+OzChPKKltQkJuEkvk5KJySLImHuAx6kh2pDRFIkVsQYGzvhaG5GxcuB/uVvfVJKRrM\nL0hD/qQEUWaljLWqNRTmx+emxyM3fQbWfikPB04348N/NuK3b53G1KwEfPVLeZiRkxTU+hj0JDtj\nLaGXa9D3Xe6tX7jUBUNzN2qbu2G7/MB0uLd+w2CwTxlnb90bY/XapbIDpDdioyNQMj8Xt8/Lxqfl\nRrx3uA5b/nwShVOSce/yaZiUEheUuhj0JDtyX0LvFgQYzX0wXOoao7eeKlpv3Ru6UY4o1MSoQ/KX\nr1qlxJdvnISFs9Kx78Ql7C6rx9Ovfo7lxVm4a9EUxEYHNnoZ9CQ7o4VKKAwRTITV5kBtcxcOnGpG\nRa0ZDtcXc9UD0Vv3xtol+SOG04DBWTr3rZge8FrEFBmhwqr5OVhYlI63D9bio2ONOHq2BQ/eMR3F\nM1IDVgeDnmRntFAJpSGC0Thdblxq64WheXAmTG1zF1o6bB5fG6FS4L4V03DbrIwAV3ktqe/n7qv4\n2EhsWDUTX5qTiTc+qMbL71TgloJUPHD7dMTHRvr9/XmUYIiTc9uBibc/HGbduAUBToUSJ6qMaDD1\noK65G/WmnuFfYAlxkYPDL5la7P38Irr7HNdcwx9H3QVSKP7/73S5sedIA/5+uB5x0Wp8q/QGFOXp\nvP7+iRwlyB49yZI3h0RI6ZeBWxDQYulDg6kH9aaewZWmrT3D0xsj1ErkpA4u7MnL1CI/MwHJ2qjh\nsfUdn3ie1SKX5xJSolYpsXrhFNw4TY+t71XiP946jZJbc3DP4jyoVf7ZUJhBT+RBMKdgOpwuNLf3\noaltcM+VelMPLrb0oN/+Rahnp2pwa2E6iqbqodNEIkMXO2ZIhMpzCSn9cvW3rFQN/s9D8/Dmx+ex\n58hFnG/swr/cPQtJ8eL/TBj0RB4EYgrm0OZeTa29aGqzXv4Y3LVxaKfGoVC/bVY6ctPjMTldOyLU\nvR26CIXnEnJc3xAZocJDq2ZiZm4SXttTjWfe+Ce+s3Y28jK1or4Pg57IAzGnYLrdAtq7+2Ey98Jo\n7oPR3Idmcy8utVmHh14AICUhGll6DW6anoIs/eA+MGnJMVApfb+dD4WHneG6vsGbu5RbCtKQmRKH\nF/9ajl/+6QQe+cpM3Cpimxn0dF1yup0eMpGhjn67Ey0WG4xDgW7pg8ncC5PFBqfriwCLj41ARnIs\nFhSmDwZ6qgaTUuL8Pq1RyodXA+G5vmE8dylZeg2e3DAPr7xTga3vVaG104bVt00WZQ0Dg57GJMfb\nacDzUEeESoGSW3NhuNSF1k4b2jpsaO20Df97V699+LUKBaBPjEFGcixmTdEhXReLTF0c0nWx0MRI\nc4fDYAuV5wjjMd67lPjYSPzo3rl47f1q7Py0Dj29Dtx3+zQofQx7Bj2NKVxvp0dz+IwRfztgQKfV\njki1AhFqBRxOASqlAlAosP0f50a8Pik+CvrEGBTl6aBPGgz2DF0sUpNiEaHmkczjEQrPEcZrIncp\napUS3yotgDYuAns/b0SPzY5HS2/waUYOg57GFI630739DrR12tDe2Y+2TtvgR1c/Glut6L6iV253\nDj4QTYiLRG56PPSJMUhNjIE+afCfKQnRiIxQiVKTHIfHrhYKzxHGa6J3KUqFAuuXTYM2LhI79htg\nd7ix8Z5ZEw57Bj2NKRRvp50uN8xd/WjrsqHtijAfCvah042GaGIikJIQjf6rPj9ErVLg+1+b47d6\n5To85onUnyOMl693KSXzcxEVocL2f5zDf+2swL/cPQuRyvF3Lhj0NCap3k673QIs3f1otNhQU9uO\nFosNJsvgg09LTz+uXO+tVimQkhADfWIM8iZpob/87/rEaKQkxAxvMPXIL/d5fC9/373IbXhMTsS4\nS1l2UxYEAfjTh4Nh//hXi8Zdh89Bv3//frzwwgsQBAGCIODxxx/HHXfc4etlSSKCfTvtdLlhsgwu\nHrp0eY65qaMPLVfNZImOVCE9ORbTshKQmpR+OcgHPxI0kV49zArW3Us4Do/RF8S4S1lenAVgMOzf\n/PgCvnvvTeP6fp+CXhAEPPHEE/jTn/6E6dOno7q6Gvfddx9WrFgBpQhzf0kaAnU7bbU5UG/sRmOr\nFY1tVjS19sJo7oXr8l5IKqUC+sQYpCfHomiKDmnJMZiZl4JoJaCNi/R5Glqw7l5CcXiMAm95cRbs\nDhcOnG4e9/f63KNXKpXo6RlcmdfT04PU1FSGPF3XgN2FhpYe1Bm7hz/aOvuHv54UH4UsvQZF+cnD\ni4c8LfMXc1OrYN29SHV4jKRn1fwcuK//smv4vHtlWVkZvv/97yM2Nha9vb3YunUr5s6d68slKQxZ\nbQ5U1ZlRaTCjorYdF5q6hnct1SfFYFp2IqZnJ2FaTiLyMhOgCcDWrVLyyfFGbNtzFu0dNqQkxeCh\nkgIsLc4OdlkkQW63MOYh6574FPROpxOPPvoovvOd76C4uBjHjx/Hj370I+zevRtxcd4dmcVtin0j\n1bY7XW5caOrCmVozKustaGyxQsDg8MuUTC1mZCcOH3SREDfxUJdq+wOF7Zdf+wO+TfHZs2fR2tqK\n4uJiAEBxcTFiYmJgMBgwe/ZsXy5NIajTOoBygxlnDGZUNVhgG3BBpVRg6qQErF44GTNykpCfqRVt\n7jkRecenoE9PT4fJZEJtbS3y8vJgMBhgNpuRk5MjVn0kcR09Azhe04p/VrfiQlMXBAyOr988Mw1F\neTrcMDkpKEfTBQsXPpEU+fQ3UK/XY9OmTfje9743POPhueeeQ2JioijFkTT19TvxeXULyipMw+E+\nSR+HuxZNwY3T9cjSx/n9MGkp4sInkiqfu1pr1qzBmjVrxKiFJMwtCKhp6MChM0Ycr2mD3elGZspg\nuM+bmYrMFO+eyYQzLnwiqZLPPTVNSF+/E4fKm/HxiSa0dfYjJkqN24oysHh2Bianx8uy5z4aLnwi\nqWLQk0ctHX34+FgTPj1jxIDdhWlZCbh7cR6Kp+vD/mHqRMfZufCJpIpBTyNcbOnBe4frceJcG5RK\nBW4pSMPtN2dhcrq4R5tJlS/j7Fz4RFLFoCcAQL2pG38/VI9TF9oRE6XGnbflYtlNWUjUyKs36ss4\ne7D3BSIaDYNe5kyWPvz1EwNOnGtDXLQady+eghXFWYiNlucpSL6Os4fbNrsUHhj0EuXv+djdvXa8\ne7gOB042IyJCibsXTcHtN2fLas67Jxxnp3Ak77/VEuXP+dgutxsfH2vCzkN1sDvcWDI3E2sWTfFp\nG4JwwnF2CkcMegny13zsC01d2La3Bk1tVhTl6XDv8qnI0HH++5U4zk7hiEEvQWLPx+7rd+Ct/Rdw\n8LQRSfFR+Nd7inDT9BTOgR8Fx9kp3DDoJUjMceKKOjNee78aXVY7Vt2SgzWLJiM6kj92Ijnh33gJ\n8jRODACz83VeX6Pf7sSO/QbsP3kJGbpYPP5QMaZkyGMuPBGNxKCXoAWF6bjQ1In9J0ceGXb4jAlT\nsxKvO6zQYOrBf71bgbYOG1beko17FueF/WpWIhodg16iyg3maz53vQeygiBg/8lLePPj84iPjcQT\n99+IGTlJ/i6ViCSOQS9R430gaxtw4vU91fhndSuK8nR4tLQA8TI7jo+IPGPQS9R4HsiaLH148a/l\naO2w4atL8lByay6UnFFDRJcpg10AebZ2ST4i1SN/PJ4W7pw614pn3zgGq82Bn9w3F3cumMyQJ6IR\n2KOXKG8W7uw70YQ/f3QeGcmx+O662dAnxgSrXB6hRyRhDHoJG23hjlsQ8OZH5/HR8SbMK0jDN1fN\nCMgeNaOFOY/QI5I2Bn2IcbrceHX3WRytasGKeVl4fP1NsJitfn/fscKcR+gRSRuDPoQM2F14+Z0z\nqKiz4KtL8vCVW3OhUgZmPH6sMOcRekTSxqAPEVabAy/sOI1aYzceLpmJL83JDOj7jxXm3NqXSNo4\n6yYEWG0O/PrNk2hosWLj3UUBD3lg9NAeGqv3ZoYQEQUHg17irDYHfvPmKTS39+E7Xy1C8Qx9UOoY\nK8wXFKZjQ8nM4V8GOm0UNpTM5Pg8kURw6EbCevsd+M3/nMKldiseXzsbRXneb2omtutN9+TWvkTS\nxaCXqL5+J37z5ilcarPi8bVF49q50l8Y5kShiUM3EuRwuvC7v5WjsdWKjfcUYXZ+SrBLIqIQxh69\nxLjdAra+V4Waxk5oYtR48a/lXGlKRD5hj15CBEHAnz48h+M1bVApFbDanAC+WJxUVmkKcoVEFIp8\nDvqBgQE8/fTTuOOOO7B69Wo8+eSTYtQlS+99Vo/9Jy8hOlIFl1sY8bWhxUlEROPl89DN888/j6io\nKOzduxcKhQLt7e1i1CU7ZZUm7Py0DgtnpeNwheeeO1eaEtFE+BT0vb292LlzJw4cOADF5a1xU1L4\n4HC8DM1deO39aszMScSGkpmovtjBlaZEJBqfgr6xsRGJiYl46aWXcPToUcTFxeF73/se5s2bJ1Z9\nkiXWtryW7n689LczSIqPxMZ7iqBWKT0eDs6VpkQ0UT4FvcvlQmNjI2644Qb89Kc/xenTp/Htb38b\nH374ITQajVfX0Om8e9316PXxolzHG58cb8S2D2ow4HABGBxS2fZBDbTx0VhanO31dfrtTvz79uOw\nO934940LkZuuBQCsWRoPbXw0tu05i/YOG1KSYvBQScGo1w5k26WI7Wf7aWw+BX1GRgbUajVKS0sB\nAHPmzEFSUhLq6upQVFTk1TXMZivcVz14HC+9Ph5tbT0+XWM8Xt9VORzyQwYcLry+qxKFOYleXUMQ\nBPzh75WoberCd9fNRqxKMaINhTmJ2PK/F4z4Hk9tDHTbpYbtZ/vl1n6lUjHuDrJPQZ+cnIz58+fj\n8OHDWLRoEerq6mA2m5Gbm+vLZSVPjG15PzrehM/PtmLd0nzMmer5uQZPbSIiMfg862bz5s342c9+\nhi1btkCtVuNXv/oVtFqtGLVJlq/b8hqau/DWvguYOzUFJfNzPL6GpzYRkVh8Dvrs7Gz88Y9/FKOW\nkOHLw1KrzYHf76xAUnwUvlVaMDxb6Wo8tYmIxMItECbAm4O7PXELAl7dVYVOqx0/e7AYcdERo76W\npzYRkVgY9BM0kZ0c9x69iNMGMx64fTqmZIw9vMVTm4hILAz6AGkw9eDtg7UonqHHspsmjfiap4eu\nnEtPRGLhpmYBYHe4sPW9SsTHRmDDqpkjxuWHHroO9d6vfOjKU5uISAzs0QfAXw8YYDT34Ufr50IT\nM3JcfqyHrs9vXMhgJyKfsUfvZ5X1Fnx0rAnLi7NQOCX5mq/zoSsR+RuD3o96+x34f7vPIkMXi3VL\nPY+tj/ZwlQ9diUgsHLqZIG9Wrf75w/Po7rXj8bXFiIpQebwOH7oSkb8x6CfAm1WrFbVmlFWaUHrb\n5DGnUk50Tj4RkbcY9BNwvVWrB083Y9sHg8H/2ZlmZOhixwzuiczJJyLyFsfoJ2CsB6hllSZs+6Aa\nQxtyWnrsPO+ViIKKQT8BYz1AfWvfeVy96zLPeyWiYGLQT8DaJfmIVI/8TxepVuLuxXno6nV4/B5O\nlySiYOEY/QSM9gC10zp6mHO6JBEFC4N+gq5+gNrRM4Cf/fcR5KRpYDL3cbokEUkGh25EsuOTC3C5\nBGy8exb3qCEiSWGPXgTnGjtxpLIFpbflIjUpFqlJY0+nJCIKJPbofeR2C/jzh+eQrI3CnbdODnY5\nRETXYND76MCpS7jYasXXvzwVUZGetzkgIgomWQ3deLM/zXhYbQ68fbAWM3MScfPMVBErJSISj2yC\n3pv9acZr12f16Btw4v4V00c95JuIKNhkM3Qz1v40E9HWacPHx5uwqCgDWakaMUokIvIL2QS92Ad8\nvHOwFiqlAncvzvOlLCIiv5NN0It5wEe9qRtHqlpw+83ZSIrnilcikjbZBP1o+9OMd8WqIAjYsd8A\nTUwESubnilkiEZFfyOZhrFgHfFTUWXC2oQP3LZ+G2GjZ/OcjohAmq6Ty9YAPt1vAjv0XoE+Mxpdv\nmiRiZURE/iOboRsx/LO6FU1tvbhncR7UKv6nI6LQIFpavfTSS5gxYwbOnTsn1iUlxe0W8PfDdchM\nicMtBWnBLoeIyGuiBH1lZSVOnTqFSZPCdzjj87MtMJr7sGbhZCiVXBxFRKHD56C32+145plnsGnT\nJhHKkabB3nw9JunjMI9bHRBRiPE56F944QWsWbMGWVlZYtQjSUerWmCy9OGuhVOg5FYHRBRifJp1\nc/LkSVRUVODHP/7xhK+h04mzfYBeHy/Kda7mcrmx+0gDJmdosXJhHg6ebMK2PWfR3mFDSlIMHiop\nwNLibL+8t7f81fZQwfaz/TQ2hSAIwkS/eevWrdi2bRsiIyMBACaTCTqdDr/4xS+waNEir65hNlvh\ndk+4BACDP+i2th6frjGaw2eMeHX3WfzrPUWwO10jNkYDBhddBfMEKX+2PRSw/Wy/3NqvVCrG3UH2\naejmsccew6FDh7Bv3z7s27cP6enpePXVV70OealzuwW891k9clI1uGl6iugboxERBQIng4/hWE0r\nWjtsKL1tMhQKhegboxERBYKoK2P37dsn5uWCShAE7DlyEWnJsbhpuh7A4AZonkJ9IhujEREFCnv0\no6iq70BDSw9K5ucMz5sXa2M0IqJAktVeN+Px/pEGJGoiRzxkFWtjNCKiQGLQe1Bn7MbZhg58/ctT\nEXFVD97XjdGIiAKNQzcevH+kAbFRaiyZmxnsUoiIfMagv4rR3IsTNW1YVjwJMVG84SGi0Megv8re\nzy9CrVZiRZBXuxIRiYVBf4XuPjs+q2jBwlnp0MZFBrscIiJRMOivcOBUM5wuN1bMY2+eiMIHg/4y\np8uN/SeaUDglGZkpccEuh4hINAz6y47VtKLTasft88J3u2UikicG/WUfHWtCWnIsZuXpgl0KEZGo\nGPQADJe6UNvcjRXFWTxYhIjCDoMewIfHGhETpcJts7jilYjCj+yDvqNnAMdr2rB4diYXSBFRWJJ9\n0O8/2QS3W8CyYj6EJaLwJOugd7rc+PS0EbPzdUhNjAl2OUREfiHroD91vh1dvXYsvXFSsEshIvIb\nWQf9J6cuQaeNQhGnVBJRGJNt0Ld09KGqvgOL52QOnyBFRBSOZBv0B041Q6lQYPFs7jlPROFNlkHv\ncLpxqNyIudNSkBTPg72JKLzJMuiPn2uF1ebA0hvZmyei8CfLoD9wshn6xGjcMDk52KUQEfmd7ILe\naO5FTWMnvjQnk/vaEJEsyC7oPz1thEqpwCI+hCUimZBV0LvcbnxWacLsfB0SeFQgEcmErIL+TK0F\n3b12LCrKCHYpREQBI6ugP3zGiPjYCBTlcyUsEcmHT/vydnR04IknnsDFixcRGRmJ3NxcPPPMM0hO\nlt5sFqvNgVPn27HspiyoVbL6/UZEMudT4ikUCjz66KPYu3cv3nvvPWRnZ+PXv/61WLWJ6kilCS63\ngEWzOWxDRPLiU9AnJiZi/vz5w3+eO3cumpubfS7KHw6fMSEnTYPsVE2wSyEiCijRjlRyu934y1/+\ngmXLlol1SdE0tlrR0NKD2CgVHvnlPui0UVi7JB8LCnl0IBGFP4UgCIIYF9q8eTNaWlrw0ksvQamU\n1hj4pv8uw/Hq1ms+Hx8bgcfuLsLS4uwgVEVEFBii9Oi3bNmChoYG/P73vx93yJvNVrjdvv2u0evj\n0dbW4/FrTpcbJ2quDXkA6Olz4HdvnUJ3T3/I9u7HarscsP1sv9zar1QqoNONbwja56D/7W9/i4qK\nCmzduhWRkYFdhFRWacLbBwywdA8geZThmIo6C8a6Z7E73Xj7gCFkg56I6Hp8Cvrz58/jD3/4AyZP\nnox7770XAJCVlYWXX35ZlOLGUlZpwht7qmF3ugEA5u4BvLGnGgBGhPbRqhYoFBgz7M3dA36tlYgo\nmHwK+mnTpqGmpkasWsbl7QOG4ZAfcnXvvN/uxMnzbZiZkwTDpa5rXj9Ep+We9EQUvqT11HQcRuuF\nX/n5U+fbYXe4cdeiKdhQMhNx0aprXh+pVmLtkny/1UlEFGwhG/Sj9cKHPl9WacJrl4dytv69AgDw\nu+8vwf9afcPwa3TaKGwomcnxeSIKa6LNow+0tUvyR4zRA1/0zssqTXj9/bNwuAYH5i099hHj9wx2\nIpKTkO3RLyhMx4aSmdBpo6DAyN752wcMwyE/ZGj8nohIbkK2Rw980Tu/ei6tN+P3RERyEdJBP+ST\n4414fVclzN0D0GmjEBOlhG3g2hk2nF1DRHIU8kFfVmnCtg9qMOBwARjstSs9HAXL2TVEJFchO0Y/\n5O0DhuGQH+IWAKUCnF1DRIQw6NGPNu7uFoDnNy4McDVERNIT8kGv00Z5DPtEzeC+O0P74QyN33N7\nYiKSm5Afulm7JB9RESNXvCoAfO3LU4f3wxn6RTC0H05ZpSkIlRIRBUfIB/2CwnQsn5c14gFs3iTt\n8Hz60fbDISKSi5AP+rJKEz4+1oQrt7S/aOpBWaWJ8+mJiBAGQe9p1o3DJeDVXVWjfg/n0xORnIR8\n0I8168YTzqcnIrkJ+aAfT++c8+mJSI5CPug9zboZzfMbFzLkiUh2Qj7oFxSmY8OdBdd9HcfliUiu\nQj7oAeB8Y+eYX+e4PBHJWUivjL1y1etouBqWiOQuZIN+aNXraAd+D+F+N0QkdyE7dONp1evVOC5P\nRBTCQX+91a0clyciGhSyQT/9bApiAAAHz0lEQVRWb53z5YmIvhCyY/Rrl+Tjtd1n4bxiCWykWsmA\nJyK6Ssj26BcUpiM/KwFDm1ayF09E5FnI9ujdbgHG9l7cXJCKJx9dgLa2nmCXREQkSSHbo681dqO7\nz4G501KCXQoRkaSFbNCfPN8GlVKB2Xm6YJdCRCRpPgd9XV0d1q9fj5UrV2L9+vWor68XoazrO3W+\nHdOzExEbHRGQ9yMiClU+B/3TTz+N+++/H3v37sX999+Pp556Soy6xmSy9MFo7sONHLYhIroun4Le\nbDajqqoKpaWlAIDS0lJUVVXBYrGIUtxoTp1vBwCOzxMRecGnWTdGoxFpaWlQqQb3g1epVEhNTYXR\naERycrJX19DpNON+38qGDkzJ1KKlawDPv3kK7R02pCTF4KGSAiwtzh739UKdXh8f7BKCiu1n+2ls\nQZ9eaTZb4R7t3D8P+vodOFtnwex8Hf7zLyfguvytbR02/OdfTqC7p19Wc+n1+nhZTy1l+9l+ubVf\nqVSMu4Ps09BNRkYGWlpa4HINHs7tcrnQ2tqKjIwMXy47psr6DrgFAdUXLcMhP8QlAH/+sMZv701E\nFIp8CnqdToeCggLs2rULALBr1y4UFBR4PWwzEeWGdsRFq9Fv97xzZW+/y2/vTUQUinyedbNp0yZs\n374dK1euxPbt27F582Yx6vLILQioqLWgcIr/fpEQEYUbn8fo8/PzsWPHDjFqua7GFiu6eu0oytOh\nqt4Cq815zWs0MUF/7EBEJCkhtTK23DA4rbIoT4f7VkyHWqUY8XW1SoH7VkwPRmlERJIVUt3fM7UW\nTE6PhzYucnhmzdsHDLB0DyCZZ8MSEXkUMkFvtTlgaO7C6tsmD39uQWE6FhSmy3KKFRGRt0Jm6Kai\nzgxBAIryuYkZEdF4hEzQnzFYoImJwJR0bbBLISIKKSER9G5BwJlaM2blJUOpVFz/G4iIaFhIBH29\nsQdWm4N7zxMRTUDQH8Z600OvM3UjLSkGc6amjPp6Off05dx2gO1n++XV/om0VyEIgvc7ihERUcgJ\niaEbIiKaOAY9EVGYY9ATEYU5Bj0RUZhj0BMRhTkGPRFRmGPQExGFOQY9EVGYY9ATEYW5kAn6uro6\nrF+/HitXrsT69etRX19/zWtcLhc2b96MFStW4Pbbbw/YEYeB4E37X375Zdx5551YvXo11q5di08/\n/TTwhfqBN20fUltbizlz5mDLli2BK9DPvG3/+++/j9WrV6O0tBSrV69Ge3t7YAv1E2/abzab8dhj\nj2H16tUoKSnBpk2b4HRee9SobAkh4sEHHxR27twpCIIg7Ny5U3jwwQevec0777wjPPLII4LL5RLM\nZrOwePFiobGxMdCl+oU37T948KDQ19cnCIIgnD17ViguLhZsNltA6/QHb9ouCILgdDqFb3zjG8IP\nf/hD4Ze//GUgS/Qrb9pfXl4ulJSUCK2trYIgCEJ3d7fQ398f0Dr9xZv2P/vss8M/c7vdLqxbt07Y\nvXt3QOuUspDo0ZvNZlRVVaG0tBQAUFpaiqqqKlgslhGve//99/G1r30NSqUSycnJWLFiBT744INg\nlCwqb9u/ePFixMTEAABmzJgBQRDQ2dkZ8HrF5G3bAWDr1q1YunQpJk+eHOAq/cfb9r/++ut45JFH\noNfrAQDx8fGIiooKeL1i87b9CoUCvb29cLvdsNvtcDgcSEtLC0bJkhQSQW80GpGWlgaVSgUAUKlU\nSE1NhdFovOZ1mZmZw3/OyMiAyWQKaK3+4G37r7Rz507k5OQgPT20z9D1tu3V1dU4dOgQHn744SBU\n6T/ett9gMKCxsREPPPAA7rnnHrzyyisQwmC/Qm/bv3HjRtTV1WHRokXDH8XFxcEoWZJCIuhpfD7/\n/HO88MIL+M1vfhPsUgLC4XDgySefxObNm4cDQW5cLhdqamrw2muv4Y9//CMOHjyId999N9hlBcwH\nH3yAGTNm4NChQzh48CCOHTsWFnfzYgmJoM/IyEBLSwtcLheAwf+pW1tbkZGRcc3rmpubh/9sNBpD\nvkcLeN9+ADh58iR+8pOf4OWXX0ZeXl6gSxWdN21va2vDxYsX8dhjj2HZsmV444038NZbb+HJJ58M\nVtmi8fZnn5mZiVWrViEyMhIajQbLly9HeXl5MEoWlbft3759O9asWQOlUon4+HgsW7YMR48eDUbJ\nkhQSQa/T6VBQUIBdu3YBAHbt2oWCggIkJyePeN2qVauwY8cOuN1uWCwWfPTRR1i5cmUwShaVt+0v\nLy/HD37wA7z44osoLCwMRqmi86btmZmZOHr0KPbt24d9+/Zhw4YN+PrXv46f//znwSpbNN7+7EtL\nS3Ho0CEIggCHw4EjR45g5syZwShZVN62PysrCwcPHgQA2O12lJWVYdq0aQGvV7KC/DDYaxcuXBDW\nrVsn3HHHHcK6desEg8EgCIIgPProo0J5ebkgCIOzLp566ilh+fLlwvLly4U333wzmCWLypv2r127\nVpg/f76wZs2a4Y/q6upgli0Kb9p+pRdffDGsZt14036XyyU899xzwqpVq4SvfOUrwnPPPSe4XK5g\nli0ab9rf0NAgPPzww0JpaalQUlIibNq0SXA4HMEsW1J4whQRUZgLiaEbIiKaOAY9EVGYY9ATEYU5\nBj0RUZhj0BMRhTkGPRFRmGPQExGFOQY9EVGY+/+x/rZvT7qCJgAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "jTlVoHTlY7EQ",
        "colab_type": "text"
      },
      "source": [
        "### Class RandomizedSearchCV\n",
        "\n",
        "This allows you to explicitly control the number of parameter combinations that are attempted. The number of search iterations is defined based on time or resources. Scikit Learn offers the RandomizedSearchCV function for this process.\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "sNmXxgGsPhkZ",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 428
        },
        "outputId": "3c8445cd-0336-4491-db70-f43be404a3f2"
      },
      "source": [
        "from sklearn.model_selection import train_test_split\n",
        "from sklearn.model_selection import StratifiedKFold\n",
        "from sklearn.model_selection import RandomizedSearchCV\n",
        "from sklearn.model_selection import GridSearchCV\n",
        "from timeit import default_timer as timer\n",
        "from sklearn.svm import LinearSVC\n",
        "\n",
        "def linear_SVC(x, y, param, kfold):\n",
        "    param_grid = {'C':param}\n",
        "    k = StratifiedKFold(n_splits=kfold)\n",
        "    grid = GridSearchCV(LinearSVC(dual=False), param_grid=param_grid, cv=k, n_jobs=4, verbose=1)\n",
        "\n",
        "    return grid.fit(x, y)\n",
        "\n",
        "def Linear_SVC_Rand(x, y, param, kfold, n):\n",
        "    param_grid = {'C':param}\n",
        "    k = StratifiedKFold(n_splits=kfold)\n",
        "    randsearch = RandomizedSearchCV(LinearSVC(dual=False), param_distributions=param_grid, cv=k, n_jobs=4,\n",
        "                                    verbose=1, n_iter=n)\n",
        "\n",
        "    return randsearch.fit(x, y)\n",
        "\n",
        "from sklearn.datasets import load_iris\n",
        "iris=load_iris()\n",
        "X=iris.data\n",
        "y=iris.target\n",
        "\n",
        "x_train,x_test,y_train,y_test=train_test_split(X,y,test_size=0.2)\n",
        "\n",
        "\n",
        "start = timer()\n",
        "param = [i/1000 for i in range(1,1000)]\n",
        "param1 = [i for i in range(1,101)]\n",
        "param.extend(param1)\n",
        "\n",
        "#progress = progressbar.bar.ProgressBar()\n",
        "clf = Linear_SVC_Rand(x=x_train, y=y_train, param=param, kfold=3, n=100)\n",
        "\n",
        "print('LinearSVC:')\n",
        "print('Best cv accuracy: {}' .format(clf.best_score_))\n",
        "print('Test set score:   {}' .format(clf.score(x_test, y_test)))\n",
        "print('Best parameters:  {}' .format(clf.best_params_))\n",
        "print()\n",
        "\n",
        "duration = timer() - start\n",
        "print('time to run RandomizedSearchCV: {}' .format(duration))\n",
        "\n",
        "\n",
        "print('########')\n",
        "\n",
        "#high C means more chance of overfitting\n",
        "\n",
        "start = timer()\n",
        "param = [i/1000 for i in range(1,1000)]\n",
        "param1 = [i for i in range(1,101)]\n",
        "param.extend(param1)\n",
        "\n",
        "clf = linear_SVC(x=x_train, y=y_train, param=param, kfold=3)\n",
        "\n",
        "print('LinearSVC:')\n",
        "print('Best cv accuracy: {}' .format(clf.best_score_))\n",
        "print('Test set score:   {}' .format(clf.score(x_test, y_test)))\n",
        "print('Best parameters:  {}' .format(clf.best_params_))\n",
        "print()\n",
        "\n",
        "duration = timer() - start\n",
        "print('time to run GridSearchCV: {}' .format(duration))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Fitting 3 folds for each of 100 candidates, totalling 300 fits\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=4)]: Using backend LokyBackend with 4 concurrent workers.\n",
            "[Parallel(n_jobs=4)]: Done 300 out of 300 | elapsed:    0.3s finished\n",
            "/usr/local/lib/python3.6/dist-packages/sklearn/model_selection/_search.py:814: DeprecationWarning: The default of the `iid` parameter will change from True to False in version 0.22 and will be removed in 0.24. This will change numeric results when test-set sizes are unequal.\n",
            "  DeprecationWarning)\n",
            "[Parallel(n_jobs=4)]: Using backend LokyBackend with 4 concurrent workers.\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "LinearSVC:\n",
            "Best cv accuracy: 0.9666666666666667\n",
            "Test set score:   0.9\n",
            "Best parameters:  {'C': 0.226}\n",
            "\n",
            "time to run RandomizedSearchCV: 0.3045790639998813\n",
            "########\n",
            "Fitting 3 folds for each of 1099 candidates, totalling 3297 fits\n",
            "LinearSVC:\n",
            "Best cv accuracy: 0.9666666666666667\n",
            "Test set score:   0.9\n",
            "Best parameters:  {'C': 0.187}\n",
            "\n",
            "time to run GridSearchCV: 2.47029100199984\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=4)]: Done 3297 out of 3297 | elapsed:    2.5s finished\n",
            "/usr/local/lib/python3.6/dist-packages/sklearn/model_selection/_search.py:814: DeprecationWarning: The default of the `iid` parameter will change from True to False in version 0.22 and will be removed in 0.24. This will change numeric results when test-set sizes are unequal.\n",
            "  DeprecationWarning)\n"
          ],
          "name": "stderr"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "bPaJ9EBODzzR",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 479
        },
        "outputId": "15c02ec0-e907-4a42-c66e-ea4e251f2ff6"
      },
      "source": [
        "\n",
        "import numpy as np\n",
        "\n",
        "from time import time\n",
        "from scipy.stats import randint as sp_randint\n",
        "\n",
        "from sklearn.model_selection import GridSearchCV\n",
        "from sklearn.model_selection import RandomizedSearchCV\n",
        "from sklearn.datasets import load_digits\n",
        "from sklearn.ensemble import RandomForestClassifier\n",
        "\n",
        "# get some data\n",
        "digits = load_digits()\n",
        "X, y = digits.data, digits.target\n",
        "\n",
        "# build a classifier\n",
        "clf = RandomForestClassifier(n_estimators=20)\n",
        "\n",
        "\n",
        "# Utility function to report best scores\n",
        "def report(results, n_top=3):\n",
        "    for i in range(1, n_top + 1):\n",
        "        candidates = np.flatnonzero(results['rank_test_score'] == i)\n",
        "        for candidate in candidates:\n",
        "            print(\"Model with rank: {0}\".format(i))\n",
        "            print(\"Mean validation score: {0:.3f} (std: {1:.3f})\".format(\n",
        "                  results['mean_test_score'][candidate],\n",
        "                  results['std_test_score'][candidate]))\n",
        "            print(\"Parameters: {0}\".format(results['params'][candidate]))\n",
        "            print(\"\")\n",
        "\n",
        "\n",
        "# specify parameters and distributions to sample from\n",
        "param_dist = {\"max_depth\": [3, None],\n",
        "              \"max_features\": sp_randint(1, 11),\n",
        "              \"min_samples_split\": sp_randint(2, 11),\n",
        "              \"bootstrap\": [True, False],\n",
        "              \"criterion\": [\"gini\", \"entropy\"]}\n",
        "\n",
        "# run randomized search\n",
        "n_iter_search = 20\n",
        "random_search = RandomizedSearchCV(clf, param_distributions=param_dist,\n",
        "                                   n_iter=n_iter_search, cv=5, iid=False)\n",
        "\n",
        "start = time()\n",
        "random_search.fit(X, y)\n",
        "print(\"RandomizedSearchCV took %.2f seconds for %d candidates\"\n",
        "      \" parameter settings.\" % ((time() - start), n_iter_search))\n",
        "report(random_search.cv_results_)\n",
        "\n",
        "# use a full grid over all parameters\n",
        "param_grid = {\"max_depth\": [3, None],\n",
        "              \"max_features\": [1, 3, 10],\n",
        "              \"min_samples_split\": [2, 3, 10],\n",
        "              \"bootstrap\": [True, False],\n",
        "              \"criterion\": [\"gini\", \"entropy\"]}\n",
        "\n",
        "# run grid search\n",
        "grid_search = GridSearchCV(clf, param_grid=param_grid, cv=5, iid=False)\n",
        "start = time()\n",
        "grid_search.fit(X, y)\n",
        "\n",
        "print(\"GridSearchCV took %.2f seconds for %d candidate parameter settings.\"\n",
        "      % (time() - start, len(grid_search.cv_results_['params'])))\n",
        "report(grid_search.cv_results_)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "RandomizedSearchCV took 4.72 seconds for 20 candidates parameter settings.\n",
            "Model with rank: 1\n",
            "Mean validation score: 0.941 (std: 0.022)\n",
            "Parameters: {'bootstrap': False, 'criterion': 'gini', 'max_depth': None, 'max_features': 6, 'min_samples_split': 5}\n",
            "\n",
            "Model with rank: 2\n",
            "Mean validation score: 0.933 (std: 0.011)\n",
            "Parameters: {'bootstrap': False, 'criterion': 'gini', 'max_depth': None, 'max_features': 4, 'min_samples_split': 8}\n",
            "\n",
            "Model with rank: 3\n",
            "Mean validation score: 0.931 (std: 0.019)\n",
            "Parameters: {'bootstrap': False, 'criterion': 'entropy', 'max_depth': None, 'max_features': 5, 'min_samples_split': 2}\n",
            "\n",
            "GridSearchCV took 16.61 seconds for 72 candidate parameter settings.\n",
            "Model with rank: 1\n",
            "Mean validation score: 0.939 (std: 0.015)\n",
            "Parameters: {'bootstrap': False, 'criterion': 'entropy', 'max_depth': None, 'max_features': 3, 'min_samples_split': 2}\n",
            "\n",
            "Model with rank: 2\n",
            "Mean validation score: 0.934 (std: 0.017)\n",
            "Parameters: {'bootstrap': True, 'criterion': 'entropy', 'max_depth': None, 'max_features': 10, 'min_samples_split': 2}\n",
            "\n",
            "Model with rank: 3\n",
            "Mean validation score: 0.930 (std: 0.022)\n",
            "Parameters: {'bootstrap': False, 'criterion': 'gini', 'max_depth': None, 'max_features': 10, 'min_samples_split': 2}\n",
            "\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "EkB2DK0laaGr",
        "colab_type": "text"
      },
      "source": [
        "## Model selection: practical tips\n",
        "\n",
        "* It is appropriate to start with a simple algorithm that can be used quickly.\n",
        "\n",
        "* Use learning curves to decide for more (or less) data, more (or less) attributes, etc.\n",
        "\n",
        "* Another technique used in model selection is *error analysis*, which corresponds to manually checking the examples (in the validation set) that the algorithm has erred. It is sometimes possible to detect a pattern of systematic error in these examples.\n",
        "\n",
        "* The motivation for using k-fold cross-validation is to be able to use a good amount of test or validation data, without significantly reducing the training data set. In situations where there is enough data to have a good-sized training data set, in addition to reasonable amounts of test and validation data, holdout cross validation can be used."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5GeybIdRaZvd",
        "colab_type": "text"
      },
      "source": [
        "\n",
        "# Case Study: Housing dataset\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "fFMl9Q2dwfpw",
        "colab_type": "text"
      },
      "source": [
        "In this case study, we will use the *Housing* dataset to practice some concepts related to hyperparameter optimization. This dataset contains information about residences in Boston/USA. There are 506 samples and 13 variables in this dataset. This dataset is used to train regressors, in which the objective is to predict the value of home prices (MEDV feature) using the predictive features provided. This dataset can be loaded with the Scikit-Learn library, according to the code block below."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Zu6suAhQmRjp",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "import numpy as np\n",
        "import matplotlib.pyplot as plt \n",
        "\n",
        "import pandas as pd  \n",
        "import seaborn as sns \n",
        "\n",
        "from sklearn.datasets import load_boston\n",
        "boston_dataset = load_boston()\n",
        "\n",
        "print(boston_dataset.keys())\n",
        "print(boston_dataset['DESCR'])\n",
        "\n",
        "boston = pd.DataFrame(boston_dataset.data, columns=boston_dataset.feature_names)\n",
        "boston['MEDV'] = boston_dataset.target\n",
        "boston.head()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "oSioPypYksrd",
        "colab_type": "text"
      },
      "source": [
        "**(1)** \n",
        "\n",
        "a) Perform the split of the Housing dataset into training and test subsets using holdout strategy. Set aside 80% of the data for training. After that, build a OLS regression model with that split, and evaluate the MSE in the test dataset.\n",
        "\n",
        "b) Repeat the previous experiment, this time using $k$-fold cross validation instead of the hold-out strategy. Use $k = 5$."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "QZpAEr8ntiEa",
        "colab_type": "text"
      },
      "source": [
        "**(2)** Consider the following code block, which is intended to perform Grid Search on the `Housing` dataset. The `param_grid` parameter tells Scikit-Learn to first evaluate all 3 × 4 = 12 combinations of` n_estimators` and `max_features` with hyperparameters specified in the first dict (don't worry about what these hyperparameters mean for the moment), and then try all 2 × 3 = 6 combinations of hyperparameter values ​​in the second dict, but this time with the `bootstrap` hyperparameter set to` False` instead of `True` (which is the default value).\n",
        "In short, GS will explore 12 + 6 = 18 combinations of hyperparameter values ​​for the `RandomForestRegressor` algorithm.\n",
        "\n",
        "  a) Based on the value of $k$ defined to perform the cross-validation in the code below, how many models will be adjusted during GS?\n",
        "\n",
        "  b) Complete the code given below to tune the hyperparameters using Grid Search. What is the RMSE value for the best model?\n",
        "  \n",
        "  c) Use Random Search with 18 iterations. Is there a change in the  produced best combination of hyperparameter values, when compared to Grid Search? What is the new RMSE value?"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "4P0gkhvutlq2",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 380
        },
        "outputId": "9a834746-fb4b-433d-b5e3-cd1bfdefc423"
      },
      "source": [
        "from sklearn.model_selection import GridSearchCV\n",
        "from sklearn.ensemble import RandomForestRegressor\n",
        "\n",
        "param_grid = [\n",
        "    {'n_estimators': [3, 10, 30], 'max_features': [2, 4, 6, 8]},\n",
        "    {'bootstrap': [False], 'n_estimators': [3, 10], 'max_features': [2, 3, 4]},\n",
        "  ]\n",
        "\n",
        "forest_reg = RandomForestRegressor()\n",
        "grid_search = GridSearchCV(forest_reg, param_grid, cv=5,\n",
        "                           scoring='neg_mean_squared_error')\n",
        "grid_search.fit(X, y)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "GridSearchCV(cv=5, error_score=nan,\n",
              "             estimator=RandomForestRegressor(bootstrap=True, ccp_alpha=0.0,\n",
              "                                             criterion='mse', max_depth=None,\n",
              "                                             max_features='auto',\n",
              "                                             max_leaf_nodes=None,\n",
              "                                             max_samples=None,\n",
              "                                             min_impurity_decrease=0.0,\n",
              "                                             min_impurity_split=None,\n",
              "                                             min_samples_leaf=1,\n",
              "                                             min_samples_split=2,\n",
              "                                             min_weight_fraction_leaf=0.0,\n",
              "                                             n_estimators=100, n_jobs=None,\n",
              "                                             oob_score=False, random_state=None,\n",
              "                                             verbose=0, warm_start=False),\n",
              "             iid='deprecated', n_jobs=None,\n",
              "             param_grid=[{'max_features': [2, 4, 6, 8],\n",
              "                          'n_estimators': [3, 10, 30]},\n",
              "                         {'bootstrap': [False], 'max_features': [2, 3, 4],\n",
              "                          'n_estimators': [3, 10]}],\n",
              "             pre_dispatch='2*n_jobs', refit=True, return_train_score=False,\n",
              "             scoring='neg_mean_squared_error', verbose=0)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 13
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "uW2yvX0wEbtD",
        "colab_type": "text"
      },
      "source": [
        "To load the data, the `load_boston` function is imported. To standardize the data, the normalization function was used, which places the data with $ \\ mu = 0 $ and $ \\ sigma ^ 2 = 1 $"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "upsQ2LjPFDFC",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 218
        },
        "outputId": "2e74fe20-a328-4fa0-f2e2-4459d00f1ec4"
      },
      "source": [
        "import pandas as pd   \n",
        "from sklearn.datasets import load_boston\n",
        "from sklearn.preprocessing import normalize\n",
        "\n",
        "boston_dataset = load_boston()\n",
        "\n",
        "boston = pd.DataFrame(normalize(boston_dataset.data), columns=boston_dataset.feature_names)\n",
        "boston['MEDV'] = boston_dataset.target\n",
        "boston.head()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>CRIM</th>\n",
              "      <th>ZN</th>\n",
              "      <th>INDUS</th>\n",
              "      <th>CHAS</th>\n",
              "      <th>NOX</th>\n",
              "      <th>RM</th>\n",
              "      <th>AGE</th>\n",
              "      <th>DIS</th>\n",
              "      <th>RAD</th>\n",
              "      <th>TAX</th>\n",
              "      <th>PTRATIO</th>\n",
              "      <th>B</th>\n",
              "      <th>LSTAT</th>\n",
              "      <th>MEDV</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>0.000013</td>\n",
              "      <td>0.035997</td>\n",
              "      <td>0.004620</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.001076</td>\n",
              "      <td>0.013149</td>\n",
              "      <td>0.130388</td>\n",
              "      <td>0.008179</td>\n",
              "      <td>0.002000</td>\n",
              "      <td>0.591945</td>\n",
              "      <td>0.030597</td>\n",
              "      <td>0.793727</td>\n",
              "      <td>0.009959</td>\n",
              "      <td>24.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>0.000058</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.014977</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000994</td>\n",
              "      <td>0.013602</td>\n",
              "      <td>0.167140</td>\n",
              "      <td>0.010522</td>\n",
              "      <td>0.004237</td>\n",
              "      <td>0.512648</td>\n",
              "      <td>0.037707</td>\n",
              "      <td>0.840785</td>\n",
              "      <td>0.019362</td>\n",
              "      <td>21.6</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>0.000059</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.015174</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.001007</td>\n",
              "      <td>0.015421</td>\n",
              "      <td>0.131140</td>\n",
              "      <td>0.010661</td>\n",
              "      <td>0.004293</td>\n",
              "      <td>0.519409</td>\n",
              "      <td>0.038204</td>\n",
              "      <td>0.843138</td>\n",
              "      <td>0.008650</td>\n",
              "      <td>34.7</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>0.000071</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.004785</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.001005</td>\n",
              "      <td>0.015360</td>\n",
              "      <td>0.100527</td>\n",
              "      <td>0.013306</td>\n",
              "      <td>0.006585</td>\n",
              "      <td>0.487268</td>\n",
              "      <td>0.041045</td>\n",
              "      <td>0.866174</td>\n",
              "      <td>0.006453</td>\n",
              "      <td>33.4</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>0.000151</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.004755</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000999</td>\n",
              "      <td>0.015587</td>\n",
              "      <td>0.118209</td>\n",
              "      <td>0.013222</td>\n",
              "      <td>0.006543</td>\n",
              "      <td>0.484177</td>\n",
              "      <td>0.040784</td>\n",
              "      <td>0.865631</td>\n",
              "      <td>0.011625</td>\n",
              "      <td>36.2</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "       CRIM        ZN     INDUS  CHAS  ...   PTRATIO         B     LSTAT  MEDV\n",
              "0  0.000013  0.035997  0.004620   0.0  ...  0.030597  0.793727  0.009959  24.0\n",
              "1  0.000058  0.000000  0.014977   0.0  ...  0.037707  0.840785  0.019362  21.6\n",
              "2  0.000059  0.000000  0.015174   0.0  ...  0.038204  0.843138  0.008650  34.7\n",
              "3  0.000071  0.000000  0.004785   0.0  ...  0.041045  0.866174  0.006453  33.4\n",
              "4  0.000151  0.000000  0.004755   0.0  ...  0.040784  0.865631  0.011625  36.2\n",
              "\n",
              "[5 rows x 14 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 4
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "MSNFYGeIFKvK",
        "colab_type": "text"
      },
      "source": [
        "### Answer: 1a\n",
        "\n",
        "We can use the function `train_test_split` to create a training dataset with  80% of the examples, as requested."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "8XeiO1bdFMSp",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 69
        },
        "outputId": "18349b3b-3323-4e0e-8da0-329fe14044f2"
      },
      "source": [
        "from sklearn.model_selection import train_test_split\n",
        "from sklearn.linear_model import LinearRegression\n",
        "from sklearn.metrics import mean_squared_error\n",
        "\n",
        "X = boston.drop(columns=['MEDV'])\n",
        "y = boston['MEDV']\n",
        "\n",
        "X_train, X_test, y_train, y_test = train_test_split(X, y, \n",
        "                                                    train_size=0.8, \n",
        "                                                    random_state=42)\n",
        "\n",
        "print(X_train.shape)\n",
        "print(X_test.shape)\n",
        "\n",
        "reg = LinearRegression()\n",
        "\n",
        "reg.fit(X_train, y_train)\n",
        "\n",
        "y_pred = reg.predict(X_test)\n",
        "\n",
        "print('MSE:', mean_squared_error(y_test, y_pred))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "(404, 13)\n",
            "(102, 13)\n",
            "MSE: 23.111795411360358\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "AjvFQrB8GLjw",
        "colab_type": "text"
      },
      "source": [
        "### Answer: 1b\n",
        "\n",
        "For the k-fold validation, five data divisions were performed between training and testing using the cross_value function with $ MSE $ validation metric"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "_DKu59nNGbVL",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 89
        },
        "outputId": "2aaa0c31-c6e7-4714-ec59-5de5f1eb2574"
      },
      "source": [
        "import numpy as np\n",
        "from sklearn.model_selection import cross_validate\n",
        "from sklearn.linear_model import LinearRegression\n",
        "\n",
        "X = boston.drop(columns=['MEDV'])\n",
        "y = boston['MEDV']\n",
        "\n",
        "reg = LinearRegression()\n",
        "\n",
        "scores = cross_validate(reg, X, y, cv=5, scoring='neg_mean_squared_error')\n",
        "\n",
        "print(scores)\n",
        "print(-scores['test_score'].mean())"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "{'fit_time': array([0.00324059, 0.00256824, 0.00249481, 0.00248075, 0.00245643]), 'score_time': array([0.00126147, 0.00105405, 0.00102663, 0.00103593, 0.00105   ]), 'test_score': array([-10.80088053, -24.71134007, -24.49247001, -70.29785965,\n",
            "       -45.91573759])}\n",
            "35.24365757024931\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "138_5SHdG3u0",
        "colab_type": "text"
      },
      "source": [
        "### Answer: 2a\n",
        "\n",
        "In total, 90 models will be trained by Grid Search as there are 18 parameters that will be combined and trained 5 times each"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2DlOiV0NCXOk",
        "colab_type": "text"
      },
      "source": [
        "### Answer: 2b\n",
        "The code below performs the training using Grid Search"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "i83XrXyhG8AT",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 218
        },
        "outputId": "c7b934c4-4181-45ef-d763-03e036dc0ded"
      },
      "source": [
        "housing = pd.read_csv('housing.csv')\n",
        "housing.head()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>longitude</th>\n",
              "      <th>latitude</th>\n",
              "      <th>housing_median_age</th>\n",
              "      <th>total_rooms</th>\n",
              "      <th>total_bedrooms</th>\n",
              "      <th>population</th>\n",
              "      <th>households</th>\n",
              "      <th>median_income</th>\n",
              "      <th>median_house_value</th>\n",
              "      <th>ocean_proximity</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>-122.23</td>\n",
              "      <td>37.88</td>\n",
              "      <td>41.0</td>\n",
              "      <td>880.0</td>\n",
              "      <td>129.0</td>\n",
              "      <td>322.0</td>\n",
              "      <td>126.0</td>\n",
              "      <td>8.3252</td>\n",
              "      <td>452600.0</td>\n",
              "      <td>NEAR BAY</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>-122.22</td>\n",
              "      <td>37.86</td>\n",
              "      <td>21.0</td>\n",
              "      <td>7099.0</td>\n",
              "      <td>1106.0</td>\n",
              "      <td>2401.0</td>\n",
              "      <td>1138.0</td>\n",
              "      <td>8.3014</td>\n",
              "      <td>358500.0</td>\n",
              "      <td>NEAR BAY</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>-122.24</td>\n",
              "      <td>37.85</td>\n",
              "      <td>52.0</td>\n",
              "      <td>1467.0</td>\n",
              "      <td>190.0</td>\n",
              "      <td>496.0</td>\n",
              "      <td>177.0</td>\n",
              "      <td>7.2574</td>\n",
              "      <td>352100.0</td>\n",
              "      <td>NEAR BAY</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>-122.25</td>\n",
              "      <td>37.85</td>\n",
              "      <td>52.0</td>\n",
              "      <td>1274.0</td>\n",
              "      <td>235.0</td>\n",
              "      <td>558.0</td>\n",
              "      <td>219.0</td>\n",
              "      <td>5.6431</td>\n",
              "      <td>341300.0</td>\n",
              "      <td>NEAR BAY</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>-122.25</td>\n",
              "      <td>37.85</td>\n",
              "      <td>52.0</td>\n",
              "      <td>1627.0</td>\n",
              "      <td>280.0</td>\n",
              "      <td>565.0</td>\n",
              "      <td>259.0</td>\n",
              "      <td>3.8462</td>\n",
              "      <td>342200.0</td>\n",
              "      <td>NEAR BAY</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "   longitude  latitude  ...  median_house_value  ocean_proximity\n",
              "0    -122.23     37.88  ...            452600.0         NEAR BAY\n",
              "1    -122.22     37.86  ...            358500.0         NEAR BAY\n",
              "2    -122.24     37.85  ...            352100.0         NEAR BAY\n",
              "3    -122.25     37.85  ...            341300.0         NEAR BAY\n",
              "4    -122.25     37.85  ...            342200.0         NEAR BAY\n",
              "\n",
              "[5 rows x 10 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 16
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "qlZ8no0G-7xH",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "housing = pd.get_dummies(housing, drop_first=True)\n",
        "housing = housing.fillna(0)\n",
        "from sklearn.preprocessing import StandardScaler\n",
        "\n",
        "scaler = StandardScaler()\n",
        "\n",
        "housing_prepared = scaler.fit_transform(housing.drop(columns='median_house_value'))\n",
        "housing_labels = housing.loc[:,'median_house_value'].values"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "AYU3feCs_w9K",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        },
        "outputId": "11d41e11-cb3c-462b-a5c3-ad18af5cedef"
      },
      "source": [
        "housing_labels"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([452600., 358500., 352100., ...,  92300.,  84700.,  89400.])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 24
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "d5OXx67cG-Nu",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 415
        },
        "outputId": "a15afce8-c95e-48aa-a61f-594b5f5cf1ff"
      },
      "source": [
        "from sklearn.model_selection import GridSearchCV\n",
        "from sklearn.ensemble import RandomForestRegressor\n",
        "\n",
        "param_grid = [\n",
        "    {'n_estimators': [3, 10, 30], 'max_features': [2, 4, 6, 8]},\n",
        "    {'bootstrap': [False], 'n_estimators': [3, 10], 'max_features': [2, 3, 4]},\n",
        "  ]\n",
        "\n",
        "forest_reg = RandomForestRegressor()\n",
        "grid_search = GridSearchCV(forest_reg, param_grid, cv=5,\n",
        "                           scoring='neg_mean_squared_error')\n",
        "%time grid_search.fit(housing_prepared, housing_labels)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "CPU times: user 1min 2s, sys: 223 ms, total: 1min 2s\n",
            "Wall time: 1min 2s\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "GridSearchCV(cv=5, error_score=nan,\n",
              "             estimator=RandomForestRegressor(bootstrap=True, ccp_alpha=0.0,\n",
              "                                             criterion='mse', max_depth=None,\n",
              "                                             max_features='auto',\n",
              "                                             max_leaf_nodes=None,\n",
              "                                             max_samples=None,\n",
              "                                             min_impurity_decrease=0.0,\n",
              "                                             min_impurity_split=None,\n",
              "                                             min_samples_leaf=1,\n",
              "                                             min_samples_split=2,\n",
              "                                             min_weight_fraction_leaf=0.0,\n",
              "                                             n_estimators=100, n_jobs=None,\n",
              "                                             oob_score=False, random_state=None,\n",
              "                                             verbose=0, warm_start=False),\n",
              "             iid='deprecated', n_jobs=None,\n",
              "             param_grid=[{'max_features': [2, 4, 6, 8],\n",
              "                          'n_estimators': [3, 10, 30]},\n",
              "                         {'bootstrap': [False], 'max_features': [2, 3, 4],\n",
              "                          'n_estimators': [3, 10]}],\n",
              "             pre_dispatch='2*n_jobs', refit=True, return_train_score=False,\n",
              "             scoring='neg_mean_squared_error', verbose=0)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 30
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "rr4PgbUQHB1j",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        },
        "outputId": "a1d69334-7d6b-454a-f328-e0337854061c"
      },
      "source": [
        "grid_search.best_params_"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{'max_features': 2, 'n_estimators': 30}"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 26
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "9G9pmH7OHFGK",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        },
        "outputId": "5a88b92d-5111-403c-8d64-2b1fc2241e80"
      },
      "source": [
        "import numpy as np\n",
        "\n",
        "rmse = np.sqrt(-grid_search.best_score_)\n",
        "\n",
        "rmse"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "72005.41921556783"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 28
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "o4H47tctDEa2",
        "colab_type": "text"
      },
      "source": [
        "### Answer: 2c\n",
        "\n",
        "The code below performs the training using Random Search\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "tcMgkMY1HImY",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        },
        "outputId": "6bd6c7cd-47fd-45b3-b3a6-0183633cd43e"
      },
      "source": [
        "from sklearn.model_selection import RandomizedSearchCV\n",
        "from sklearn.ensemble import RandomForestRegressor\n",
        "\n",
        "param_grid = [\n",
        "    {'n_estimators': [3, 10, 30], 'max_features': [2, 4, 6, 8]},\n",
        "    {'bootstrap': [False], 'n_estimators': [3, 10], 'max_features': [2, 3, 4]},\n",
        "  ]\n",
        "\n",
        "forest_reg = RandomForestRegressor()\n",
        "random_search = RandomizedSearchCV(forest_reg, \n",
        "                                   param_grid, \n",
        "                                   cv = 5,\n",
        "                                   scoring='neg_mean_squared_error',\n",
        "                                   n_iter = 18,\n",
        "                                   verbose = 2)\n",
        "%time random_search.fit(housing_prepared, housing_labels)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Fitting 5 folds for each of 18 candidates, totalling 90 fits\n",
            "[CV] n_estimators=3, max_features=2 ..................................\n",
            "[CV] ................... n_estimators=3, max_features=2, total=   0.1s\n",
            "[CV] n_estimators=3, max_features=2 ..................................\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=1)]: Using backend SequentialBackend with 1 concurrent workers.\n",
            "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.1s remaining:    0.0s\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[CV] ................... n_estimators=3, max_features=2, total=   0.1s\n",
            "[CV] n_estimators=3, max_features=2 ..................................\n",
            "[CV] ................... n_estimators=3, max_features=2, total=   0.1s\n",
            "[CV] n_estimators=3, max_features=2 ..................................\n",
            "[CV] ................... n_estimators=3, max_features=2, total=   0.1s\n",
            "[CV] n_estimators=3, max_features=2 ..................................\n",
            "[CV] ................... n_estimators=3, max_features=2, total=   0.1s\n",
            "[CV] n_estimators=10, max_features=2 .................................\n",
            "[CV] .................. n_estimators=10, max_features=2, total=   0.3s\n",
            "[CV] n_estimators=10, max_features=2 .................................\n",
            "[CV] .................. n_estimators=10, max_features=2, total=   0.3s\n",
            "[CV] n_estimators=10, max_features=2 .................................\n",
            "[CV] .................. n_estimators=10, max_features=2, total=   0.3s\n",
            "[CV] n_estimators=10, max_features=2 .................................\n",
            "[CV] .................. n_estimators=10, max_features=2, total=   0.3s\n",
            "[CV] n_estimators=10, max_features=2 .................................\n",
            "[CV] .................. n_estimators=10, max_features=2, total=   0.3s\n",
            "[CV] n_estimators=30, max_features=2 .................................\n",
            "[CV] .................. n_estimators=30, max_features=2, total=   0.9s\n",
            "[CV] n_estimators=30, max_features=2 .................................\n",
            "[CV] .................. n_estimators=30, max_features=2, total=   1.0s\n",
            "[CV] n_estimators=30, max_features=2 .................................\n",
            "[CV] .................. n_estimators=30, max_features=2, total=   0.9s\n",
            "[CV] n_estimators=30, max_features=2 .................................\n",
            "[CV] .................. n_estimators=30, max_features=2, total=   0.9s\n",
            "[CV] n_estimators=30, max_features=2 .................................\n",
            "[CV] .................. n_estimators=30, max_features=2, total=   0.9s\n",
            "[CV] n_estimators=3, max_features=4 ..................................\n",
            "[CV] ................... n_estimators=3, max_features=4, total=   0.1s\n",
            "[CV] n_estimators=3, max_features=4 ..................................\n",
            "[CV] ................... n_estimators=3, max_features=4, total=   0.2s\n",
            "[CV] n_estimators=3, max_features=4 ..................................\n",
            "[CV] ................... n_estimators=3, max_features=4, total=   0.1s\n",
            "[CV] n_estimators=3, max_features=4 ..................................\n",
            "[CV] ................... n_estimators=3, max_features=4, total=   0.2s\n",
            "[CV] n_estimators=3, max_features=4 ..................................\n",
            "[CV] ................... n_estimators=3, max_features=4, total=   0.1s\n",
            "[CV] n_estimators=10, max_features=4 .................................\n",
            "[CV] .................. n_estimators=10, max_features=4, total=   0.5s\n",
            "[CV] n_estimators=10, max_features=4 .................................\n",
            "[CV] .................. n_estimators=10, max_features=4, total=   0.5s\n",
            "[CV] n_estimators=10, max_features=4 .................................\n",
            "[CV] .................. n_estimators=10, max_features=4, total=   0.5s\n",
            "[CV] n_estimators=10, max_features=4 .................................\n",
            "[CV] .................. n_estimators=10, max_features=4, total=   0.5s\n",
            "[CV] n_estimators=10, max_features=4 .................................\n",
            "[CV] .................. n_estimators=10, max_features=4, total=   0.5s\n",
            "[CV] n_estimators=30, max_features=4 .................................\n",
            "[CV] .................. n_estimators=30, max_features=4, total=   1.5s\n",
            "[CV] n_estimators=30, max_features=4 .................................\n",
            "[CV] .................. n_estimators=30, max_features=4, total=   1.5s\n",
            "[CV] n_estimators=30, max_features=4 .................................\n",
            "[CV] .................. n_estimators=30, max_features=4, total=   1.4s\n",
            "[CV] n_estimators=30, max_features=4 .................................\n",
            "[CV] .................. n_estimators=30, max_features=4, total=   1.5s\n",
            "[CV] n_estimators=30, max_features=4 .................................\n",
            "[CV] .................. n_estimators=30, max_features=4, total=   1.4s\n",
            "[CV] n_estimators=3, max_features=6 ..................................\n",
            "[CV] ................... n_estimators=3, max_features=6, total=   0.2s\n",
            "[CV] n_estimators=3, max_features=6 ..................................\n",
            "[CV] ................... n_estimators=3, max_features=6, total=   0.2s\n",
            "[CV] n_estimators=3, max_features=6 ..................................\n",
            "[CV] ................... n_estimators=3, max_features=6, total=   0.2s\n",
            "[CV] n_estimators=3, max_features=6 ..................................\n",
            "[CV] ................... n_estimators=3, max_features=6, total=   0.2s\n",
            "[CV] n_estimators=3, max_features=6 ..................................\n",
            "[CV] ................... n_estimators=3, max_features=6, total=   0.2s\n",
            "[CV] n_estimators=10, max_features=6 .................................\n",
            "[CV] .................. n_estimators=10, max_features=6, total=   0.7s\n",
            "[CV] n_estimators=10, max_features=6 .................................\n",
            "[CV] .................. n_estimators=10, max_features=6, total=   0.7s\n",
            "[CV] n_estimators=10, max_features=6 .................................\n",
            "[CV] .................. n_estimators=10, max_features=6, total=   0.7s\n",
            "[CV] n_estimators=10, max_features=6 .................................\n",
            "[CV] .................. n_estimators=10, max_features=6, total=   0.7s\n",
            "[CV] n_estimators=10, max_features=6 .................................\n",
            "[CV] .................. n_estimators=10, max_features=6, total=   0.7s\n",
            "[CV] n_estimators=30, max_features=6 .................................\n",
            "[CV] .................. n_estimators=30, max_features=6, total=   2.0s\n",
            "[CV] n_estimators=30, max_features=6 .................................\n",
            "[CV] .................. n_estimators=30, max_features=6, total=   2.0s\n",
            "[CV] n_estimators=30, max_features=6 .................................\n",
            "[CV] .................. n_estimators=30, max_features=6, total=   2.0s\n",
            "[CV] n_estimators=30, max_features=6 .................................\n",
            "[CV] .................. n_estimators=30, max_features=6, total=   2.0s\n",
            "[CV] n_estimators=30, max_features=6 .................................\n",
            "[CV] .................. n_estimators=30, max_features=6, total=   2.0s\n",
            "[CV] n_estimators=3, max_features=8 ..................................\n",
            "[CV] ................... n_estimators=3, max_features=8, total=   0.3s\n",
            "[CV] n_estimators=3, max_features=8 ..................................\n",
            "[CV] ................... n_estimators=3, max_features=8, total=   0.3s\n",
            "[CV] n_estimators=3, max_features=8 ..................................\n",
            "[CV] ................... n_estimators=3, max_features=8, total=   0.3s\n",
            "[CV] n_estimators=3, max_features=8 ..................................\n",
            "[CV] ................... n_estimators=3, max_features=8, total=   0.3s\n",
            "[CV] n_estimators=3, max_features=8 ..................................\n",
            "[CV] ................... n_estimators=3, max_features=8, total=   0.3s\n",
            "[CV] n_estimators=10, max_features=8 .................................\n",
            "[CV] .................. n_estimators=10, max_features=8, total=   0.8s\n",
            "[CV] n_estimators=10, max_features=8 .................................\n",
            "[CV] .................. n_estimators=10, max_features=8, total=   0.9s\n",
            "[CV] n_estimators=10, max_features=8 .................................\n",
            "[CV] .................. n_estimators=10, max_features=8, total=   0.8s\n",
            "[CV] n_estimators=10, max_features=8 .................................\n",
            "[CV] .................. n_estimators=10, max_features=8, total=   0.9s\n",
            "[CV] n_estimators=10, max_features=8 .................................\n",
            "[CV] .................. n_estimators=10, max_features=8, total=   0.9s\n",
            "[CV] n_estimators=30, max_features=8 .................................\n",
            "[CV] .................. n_estimators=30, max_features=8, total=   2.5s\n",
            "[CV] n_estimators=30, max_features=8 .................................\n",
            "[CV] .................. n_estimators=30, max_features=8, total=   2.6s\n",
            "[CV] n_estimators=30, max_features=8 .................................\n",
            "[CV] .................. n_estimators=30, max_features=8, total=   2.5s\n",
            "[CV] n_estimators=30, max_features=8 .................................\n",
            "[CV] .................. n_estimators=30, max_features=8, total=   2.5s\n",
            "[CV] n_estimators=30, max_features=8 .................................\n",
            "[CV] .................. n_estimators=30, max_features=8, total=   2.5s\n",
            "[CV] n_estimators=3, max_features=2, bootstrap=False .................\n",
            "[CV] .. n_estimators=3, max_features=2, bootstrap=False, total=   0.1s\n",
            "[CV] n_estimators=3, max_features=2, bootstrap=False .................\n",
            "[CV] .. n_estimators=3, max_features=2, bootstrap=False, total=   0.1s\n",
            "[CV] n_estimators=3, max_features=2, bootstrap=False .................\n",
            "[CV] .. n_estimators=3, max_features=2, bootstrap=False, total=   0.1s\n",
            "[CV] n_estimators=3, max_features=2, bootstrap=False .................\n",
            "[CV] .. n_estimators=3, max_features=2, bootstrap=False, total=   0.1s\n",
            "[CV] n_estimators=3, max_features=2, bootstrap=False .................\n",
            "[CV] .. n_estimators=3, max_features=2, bootstrap=False, total=   0.1s\n",
            "[CV] n_estimators=10, max_features=2, bootstrap=False ................\n",
            "[CV] . n_estimators=10, max_features=2, bootstrap=False, total=   0.5s\n",
            "[CV] n_estimators=10, max_features=2, bootstrap=False ................\n",
            "[CV] . n_estimators=10, max_features=2, bootstrap=False, total=   0.5s\n",
            "[CV] n_estimators=10, max_features=2, bootstrap=False ................\n",
            "[CV] . n_estimators=10, max_features=2, bootstrap=False, total=   0.5s\n",
            "[CV] n_estimators=10, max_features=2, bootstrap=False ................\n",
            "[CV] . n_estimators=10, max_features=2, bootstrap=False, total=   0.5s\n",
            "[CV] n_estimators=10, max_features=2, bootstrap=False ................\n",
            "[CV] . n_estimators=10, max_features=2, bootstrap=False, total=   0.5s\n",
            "[CV] n_estimators=3, max_features=3, bootstrap=False .................\n",
            "[CV] .. n_estimators=3, max_features=3, bootstrap=False, total=   0.2s\n",
            "[CV] n_estimators=3, max_features=3, bootstrap=False .................\n",
            "[CV] .. n_estimators=3, max_features=3, bootstrap=False, total=   0.2s\n",
            "[CV] n_estimators=3, max_features=3, bootstrap=False .................\n",
            "[CV] .. n_estimators=3, max_features=3, bootstrap=False, total=   0.2s\n",
            "[CV] n_estimators=3, max_features=3, bootstrap=False .................\n",
            "[CV] .. n_estimators=3, max_features=3, bootstrap=False, total=   0.2s\n",
            "[CV] n_estimators=3, max_features=3, bootstrap=False .................\n",
            "[CV] .. n_estimators=3, max_features=3, bootstrap=False, total=   0.2s\n",
            "[CV] n_estimators=10, max_features=3, bootstrap=False ................\n",
            "[CV] . n_estimators=10, max_features=3, bootstrap=False, total=   0.6s\n",
            "[CV] n_estimators=10, max_features=3, bootstrap=False ................\n",
            "[CV] . n_estimators=10, max_features=3, bootstrap=False, total=   0.6s\n",
            "[CV] n_estimators=10, max_features=3, bootstrap=False ................\n",
            "[CV] . n_estimators=10, max_features=3, bootstrap=False, total=   0.6s\n",
            "[CV] n_estimators=10, max_features=3, bootstrap=False ................\n",
            "[CV] . n_estimators=10, max_features=3, bootstrap=False, total=   0.6s\n",
            "[CV] n_estimators=10, max_features=3, bootstrap=False ................\n",
            "[CV] . n_estimators=10, max_features=3, bootstrap=False, total=   0.6s\n",
            "[CV] n_estimators=3, max_features=4, bootstrap=False .................\n",
            "[CV] .. n_estimators=3, max_features=4, bootstrap=False, total=   0.2s\n",
            "[CV] n_estimators=3, max_features=4, bootstrap=False .................\n",
            "[CV] .. n_estimators=3, max_features=4, bootstrap=False, total=   0.2s\n",
            "[CV] n_estimators=3, max_features=4, bootstrap=False .................\n",
            "[CV] .. n_estimators=3, max_features=4, bootstrap=False, total=   0.2s\n",
            "[CV] n_estimators=3, max_features=4, bootstrap=False .................\n",
            "[CV] .. n_estimators=3, max_features=4, bootstrap=False, total=   0.2s\n",
            "[CV] n_estimators=3, max_features=4, bootstrap=False .................\n",
            "[CV] .. n_estimators=3, max_features=4, bootstrap=False, total=   0.2s\n",
            "[CV] n_estimators=10, max_features=4, bootstrap=False ................\n",
            "[CV] . n_estimators=10, max_features=4, bootstrap=False, total=   0.8s\n",
            "[CV] n_estimators=10, max_features=4, bootstrap=False ................\n",
            "[CV] . n_estimators=10, max_features=4, bootstrap=False, total=   0.8s\n",
            "[CV] n_estimators=10, max_features=4, bootstrap=False ................\n",
            "[CV] . n_estimators=10, max_features=4, bootstrap=False, total=   0.8s\n",
            "[CV] n_estimators=10, max_features=4, bootstrap=False ................\n",
            "[CV] . n_estimators=10, max_features=4, bootstrap=False, total=   0.8s\n",
            "[CV] n_estimators=10, max_features=4, bootstrap=False ................\n",
            "[CV] . n_estimators=10, max_features=4, bootstrap=False, total=   0.7s\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=1)]: Done  90 out of  90 | elapsed:  1.0min finished\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "CPU times: user 1min 2s, sys: 249 ms, total: 1min 2s\n",
            "Wall time: 1min 2s\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "RandomizedSearchCV(cv=5, error_score=nan,\n",
              "                   estimator=RandomForestRegressor(bootstrap=True,\n",
              "                                                   ccp_alpha=0.0,\n",
              "                                                   criterion='mse',\n",
              "                                                   max_depth=None,\n",
              "                                                   max_features='auto',\n",
              "                                                   max_leaf_nodes=None,\n",
              "                                                   max_samples=None,\n",
              "                                                   min_impurity_decrease=0.0,\n",
              "                                                   min_impurity_split=None,\n",
              "                                                   min_samples_leaf=1,\n",
              "                                                   min_samples_split=2,\n",
              "                                                   min_weight_fraction_leaf=0.0,\n",
              "                                                   n_estimators=100,\n",
              "                                                   n_jobs=None, oob_score=False,\n",
              "                                                   random_state=None, verbose=0,\n",
              "                                                   warm_start=False),\n",
              "                   iid='deprecated', n_iter=18, n_jobs=None,\n",
              "                   param_distributions=[{'max_features': [2, 4, 6, 8],\n",
              "                                         'n_estimators': [3, 10, 30]},\n",
              "                                        {'bootstrap': [False],\n",
              "                                         'max_features': [2, 3, 4],\n",
              "                                         'n_estimators': [3, 10]}],\n",
              "                   pre_dispatch='2*n_jobs', random_state=None, refit=True,\n",
              "                   return_train_score=False, scoring='neg_mean_squared_error',\n",
              "                   verbose=2)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 34
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "crebtgnXHLNs",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        },
        "outputId": "19210af6-aca8-4a64-b435-30c2d5b7986f"
      },
      "source": [
        "random_search.best_params_"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{'max_features': 2, 'n_estimators': 30}"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 35
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "mJQhCxMHHNbg",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        },
        "outputId": "f4804002-7884-496f-d229-2681e2221673"
      },
      "source": [
        "import numpy as np\n",
        "rmse = np.sqrt(-random_search.best_score_)\n",
        "rmse"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "71881.3543860473"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 36
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "cJuVwtxRjlkh",
        "colab_type": "text"
      },
      "source": [
        "# Case Study: Wine Quality dataset\n",
        "\n",
        "Let us see another case study of applying hyperparameter optimization using the Wine Quality dataset."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "-BNaX00jXwUt",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "import pandas as pd\n",
        "from sklearn.model_selection import train_test_split\n",
        "\n",
        "# Read in and create target\n",
        "wine_data = pd.read_csv('winequality-red.csv')\n",
        "wine_data['high_quality'] = [1 if x >=7 else 0 for x in wine_data['quality']]\n",
        "\n",
        "# Split data\n",
        "X = wine_data.drop(['quality'], axis=1)\n",
        "y = wine_data['high_quality']\n",
        "\n",
        "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=42)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "79d2My1npcRP",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 221
        },
        "outputId": "0a2ca968-c4e8-4d45-f624-121d9772dd11"
      },
      "source": [
        "import pandas as pd\n",
        "from sklearn.model_selection import train_test_split\n",
        "\n",
        "# Read in and create target\n",
        "wine_df = pd.read_csv('winequality-red.csv')\n",
        "\n",
        "wine_df.head()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>fixed acidity</th>\n",
              "      <th>volatile acidity</th>\n",
              "      <th>citric acid</th>\n",
              "      <th>residual sugar</th>\n",
              "      <th>chlorides</th>\n",
              "      <th>free sulfur dioxide</th>\n",
              "      <th>total sulfur dioxide</th>\n",
              "      <th>density</th>\n",
              "      <th>pH</th>\n",
              "      <th>sulphates</th>\n",
              "      <th>alcohol</th>\n",
              "      <th>quality</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>7.4</td>\n",
              "      <td>0.70</td>\n",
              "      <td>0.00</td>\n",
              "      <td>1.9</td>\n",
              "      <td>0.076</td>\n",
              "      <td>11.0</td>\n",
              "      <td>34.0</td>\n",
              "      <td>0.9978</td>\n",
              "      <td>3.51</td>\n",
              "      <td>0.56</td>\n",
              "      <td>9.4</td>\n",
              "      <td>5</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>7.8</td>\n",
              "      <td>0.88</td>\n",
              "      <td>0.00</td>\n",
              "      <td>2.6</td>\n",
              "      <td>0.098</td>\n",
              "      <td>25.0</td>\n",
              "      <td>67.0</td>\n",
              "      <td>0.9968</td>\n",
              "      <td>3.20</td>\n",
              "      <td>0.68</td>\n",
              "      <td>9.8</td>\n",
              "      <td>5</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>7.8</td>\n",
              "      <td>0.76</td>\n",
              "      <td>0.04</td>\n",
              "      <td>2.3</td>\n",
              "      <td>0.092</td>\n",
              "      <td>15.0</td>\n",
              "      <td>54.0</td>\n",
              "      <td>0.9970</td>\n",
              "      <td>3.26</td>\n",
              "      <td>0.65</td>\n",
              "      <td>9.8</td>\n",
              "      <td>5</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>11.2</td>\n",
              "      <td>0.28</td>\n",
              "      <td>0.56</td>\n",
              "      <td>1.9</td>\n",
              "      <td>0.075</td>\n",
              "      <td>17.0</td>\n",
              "      <td>60.0</td>\n",
              "      <td>0.9980</td>\n",
              "      <td>3.16</td>\n",
              "      <td>0.58</td>\n",
              "      <td>9.8</td>\n",
              "      <td>6</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>7.4</td>\n",
              "      <td>0.70</td>\n",
              "      <td>0.00</td>\n",
              "      <td>1.9</td>\n",
              "      <td>0.076</td>\n",
              "      <td>11.0</td>\n",
              "      <td>34.0</td>\n",
              "      <td>0.9978</td>\n",
              "      <td>3.51</td>\n",
              "      <td>0.56</td>\n",
              "      <td>9.4</td>\n",
              "      <td>5</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "   fixed acidity  volatile acidity  citric acid  ...  sulphates  alcohol  quality\n",
              "0            7.4              0.70         0.00  ...       0.56      9.4        5\n",
              "1            7.8              0.88         0.00  ...       0.68      9.8        5\n",
              "2            7.8              0.76         0.04  ...       0.65      9.8        5\n",
              "3           11.2              0.28         0.56  ...       0.58      9.8        6\n",
              "4            7.4              0.70         0.00  ...       0.56      9.4        5\n",
              "\n",
              "[5 rows x 12 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 47
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "eFCCA0-B8zGK",
        "colab_type": "text"
      },
      "source": [
        ""
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "08hFy2Ow80AU",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 340
        },
        "outputId": "0c1ca340-36ab-48d4-83c6-f2e999ce514e"
      },
      "source": [
        "wine_df.info()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "<class 'pandas.core.frame.DataFrame'>\n",
            "RangeIndex: 1599 entries, 0 to 1598\n",
            "Data columns (total 12 columns):\n",
            " #   Column                Non-Null Count  Dtype  \n",
            "---  ------                --------------  -----  \n",
            " 0   fixed acidity         1599 non-null   float64\n",
            " 1   volatile acidity      1599 non-null   float64\n",
            " 2   citric acid           1599 non-null   float64\n",
            " 3   residual sugar        1599 non-null   float64\n",
            " 4   chlorides             1599 non-null   float64\n",
            " 5   free sulfur dioxide   1599 non-null   float64\n",
            " 6   total sulfur dioxide  1599 non-null   float64\n",
            " 7   density               1599 non-null   float64\n",
            " 8   pH                    1599 non-null   float64\n",
            " 9   sulphates             1599 non-null   float64\n",
            " 10  alcohol               1599 non-null   float64\n",
            " 11  quality               1599 non-null   int64  \n",
            "dtypes: float64(11), int64(1)\n",
            "memory usage: 150.0 KB\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "gOUJtffE_4oW",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 331
        },
        "outputId": "d35f5c13-a733-4df5-f0f2-33a0d72961d7"
      },
      "source": [
        "wine_df.describe()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>fixed acidity</th>\n",
              "      <th>volatile acidity</th>\n",
              "      <th>citric acid</th>\n",
              "      <th>residual sugar</th>\n",
              "      <th>chlorides</th>\n",
              "      <th>free sulfur dioxide</th>\n",
              "      <th>total sulfur dioxide</th>\n",
              "      <th>density</th>\n",
              "      <th>pH</th>\n",
              "      <th>sulphates</th>\n",
              "      <th>alcohol</th>\n",
              "      <th>quality</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>count</th>\n",
              "      <td>1599.000000</td>\n",
              "      <td>1599.000000</td>\n",
              "      <td>1599.000000</td>\n",
              "      <td>1599.000000</td>\n",
              "      <td>1599.000000</td>\n",
              "      <td>1599.000000</td>\n",
              "      <td>1599.000000</td>\n",
              "      <td>1599.000000</td>\n",
              "      <td>1599.000000</td>\n",
              "      <td>1599.000000</td>\n",
              "      <td>1599.000000</td>\n",
              "      <td>1599.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>mean</th>\n",
              "      <td>8.319637</td>\n",
              "      <td>0.527821</td>\n",
              "      <td>0.270976</td>\n",
              "      <td>2.538806</td>\n",
              "      <td>0.087467</td>\n",
              "      <td>15.874922</td>\n",
              "      <td>46.467792</td>\n",
              "      <td>0.996747</td>\n",
              "      <td>3.311113</td>\n",
              "      <td>0.658149</td>\n",
              "      <td>10.422983</td>\n",
              "      <td>5.636023</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>std</th>\n",
              "      <td>1.741096</td>\n",
              "      <td>0.179060</td>\n",
              "      <td>0.194801</td>\n",
              "      <td>1.409928</td>\n",
              "      <td>0.047065</td>\n",
              "      <td>10.460157</td>\n",
              "      <td>32.895324</td>\n",
              "      <td>0.001887</td>\n",
              "      <td>0.154386</td>\n",
              "      <td>0.169507</td>\n",
              "      <td>1.065668</td>\n",
              "      <td>0.807569</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>min</th>\n",
              "      <td>4.600000</td>\n",
              "      <td>0.120000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.900000</td>\n",
              "      <td>0.012000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>6.000000</td>\n",
              "      <td>0.990070</td>\n",
              "      <td>2.740000</td>\n",
              "      <td>0.330000</td>\n",
              "      <td>8.400000</td>\n",
              "      <td>3.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>25%</th>\n",
              "      <td>7.100000</td>\n",
              "      <td>0.390000</td>\n",
              "      <td>0.090000</td>\n",
              "      <td>1.900000</td>\n",
              "      <td>0.070000</td>\n",
              "      <td>7.000000</td>\n",
              "      <td>22.000000</td>\n",
              "      <td>0.995600</td>\n",
              "      <td>3.210000</td>\n",
              "      <td>0.550000</td>\n",
              "      <td>9.500000</td>\n",
              "      <td>5.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>50%</th>\n",
              "      <td>7.900000</td>\n",
              "      <td>0.520000</td>\n",
              "      <td>0.260000</td>\n",
              "      <td>2.200000</td>\n",
              "      <td>0.079000</td>\n",
              "      <td>14.000000</td>\n",
              "      <td>38.000000</td>\n",
              "      <td>0.996750</td>\n",
              "      <td>3.310000</td>\n",
              "      <td>0.620000</td>\n",
              "      <td>10.200000</td>\n",
              "      <td>6.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>75%</th>\n",
              "      <td>9.200000</td>\n",
              "      <td>0.640000</td>\n",
              "      <td>0.420000</td>\n",
              "      <td>2.600000</td>\n",
              "      <td>0.090000</td>\n",
              "      <td>21.000000</td>\n",
              "      <td>62.000000</td>\n",
              "      <td>0.997835</td>\n",
              "      <td>3.400000</td>\n",
              "      <td>0.730000</td>\n",
              "      <td>11.100000</td>\n",
              "      <td>6.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>max</th>\n",
              "      <td>15.900000</td>\n",
              "      <td>1.580000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>15.500000</td>\n",
              "      <td>0.611000</td>\n",
              "      <td>72.000000</td>\n",
              "      <td>289.000000</td>\n",
              "      <td>1.003690</td>\n",
              "      <td>4.010000</td>\n",
              "      <td>2.000000</td>\n",
              "      <td>14.900000</td>\n",
              "      <td>8.000000</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "       fixed acidity  volatile acidity  ...      alcohol      quality\n",
              "count    1599.000000       1599.000000  ...  1599.000000  1599.000000\n",
              "mean        8.319637          0.527821  ...    10.422983     5.636023\n",
              "std         1.741096          0.179060  ...     1.065668     0.807569\n",
              "min         4.600000          0.120000  ...     8.400000     3.000000\n",
              "25%         7.100000          0.390000  ...     9.500000     5.000000\n",
              "50%         7.900000          0.520000  ...    10.200000     6.000000\n",
              "75%         9.200000          0.640000  ...    11.100000     6.000000\n",
              "max        15.900000          1.580000  ...    14.900000     8.000000\n",
              "\n",
              "[8 rows x 12 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 49
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "5qaydnuG8oSi",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Split data\n",
        "X = wine_df.drop(['quality'], axis=1)\n",
        "y = wine_df['quality']\n",
        "\n",
        "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=42)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "BKkyePbF_9Qm",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "from sklearn.preprocessing import StandardScaler\n",
        "\n",
        "scaler = StandardScaler()\n",
        "\n",
        "scaler.fit(X_train)\n",
        "\n",
        "X_train = scaler.transform(X_train)\n",
        "X_test = scaler.transform(X_test)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "AMIHfSgDldTX",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 255
        },
        "outputId": "558f7f1b-dff7-45f4-fbc0-047aebb4222c"
      },
      "source": [
        "from sklearn.model_selection import GridSearchCV\n",
        "from sklearn.svm import SVC\n",
        "\n",
        "# Create parameter grid\n",
        "param_grid = {'C': [.01, .1, 1, 10, 100, 1000, 10000],\n",
        "                  'gamma': [0.0001, 0.001, 0.01, .1, 1, 10, 100, 1000]}\n",
        "\n",
        "# Create grid search \n",
        "grid_search = GridSearchCV(SVC(),\n",
        "                           param_grid, \n",
        "                           cv=5)\n",
        "# Fit grid search\n",
        "%time grid_search.fit(X_train, y_train)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "CPU times: user 40.5 s, sys: 6.95 ms, total: 40.5 s\n",
            "Wall time: 40.5 s\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "GridSearchCV(cv=5, error_score=nan,\n",
              "             estimator=SVC(C=1.0, break_ties=False, cache_size=200,\n",
              "                           class_weight=None, coef0=0.0,\n",
              "                           decision_function_shape='ovr', degree=3,\n",
              "                           gamma='scale', kernel='rbf', max_iter=-1,\n",
              "                           probability=False, random_state=None, shrinking=True,\n",
              "                           tol=0.001, verbose=False),\n",
              "             iid='deprecated', n_jobs=None,\n",
              "             param_grid={'C': [0.01, 0.1, 1, 10, 100, 1000, 10000],\n",
              "                         'gamma': [0.0001, 0.001, 0.01, 0.1, 1, 10, 100, 1000]},\n",
              "             pre_dispatch='2*n_jobs', refit=True, return_train_score=False,\n",
              "             scoring=None, verbose=0)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 53
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "IF8SpsThm6fu",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 409
        },
        "outputId": "d097bf01-b240-4f5e-939f-a1f4e235c74b"
      },
      "source": [
        "import matplotlib.pyplot as plt\n",
        "import seaborn as sns\n",
        "\n",
        "# Pull results from grid search\n",
        "results = pd.DataFrame(grid_search.cv_results_)\n",
        "hm_data = results.pivot(index='param_C', \n",
        "                   columns='param_gamma',\n",
        "                   values='mean_test_score')\n",
        "\n",
        "# Plot heatmap\n",
        "fix, ax = plt.subplots(figsize=(9, 6))\n",
        "\n",
        "ax = sns.heatmap(hm_data,\n",
        "                 annot=True,\n",
        "                 cmap='RdBu',\n",
        "                 linecolor='white')\n",
        "\n",
        "ax.set_title('Hyperparameter Gridsearch', fontsize=16)\n",
        "ax.set_ylabel('C Parameter', fontsize=12)\n",
        "ax.set_xlabel('Gamma Parameter', fontsize=12)\n",
        "\n",
        "plt.show()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAg8AAAGICAYAAADGeANfAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzdd3hUVfrA8e+bSegthCRAQi8iiNIJihSVotJdu6uoi4odlQUW5YeIIjbAXRDRta0FUHcVFERFAUVBegtVCCQBkhBCb8nM+f1xb8LMJCETksykvJ/nuQ+Zc8+9932HSe6Zc869V4wxKKWUUkr5KijQASillFKqZNHGg1JKKaXyRRsPSimllMoXbTwopZRSKl+08aCUUkqpfNHGg1JKKaXyRRsPCgARGSoiRkSa5rAu2F43PgChqTyIyJMiMsTPxxQRuUNEfhCRQyKSLiIpIvKjiDwsIhV93M8HIhLnQ73Mz2fDAoYeECKyRER+DXQcShUWbTwoVfI9Cfit8SAiwcAXwEdAAvAgcA3wELAHeAN43MfdvQAMLoIwlVJFKDjQAShVUCJS3hhz1g/HESDEGHOuqI8VaHm8p89infBvMsb8z2vdlyIyCWjhy/6NMX8WQrgBUZY+D0p5054HlW8i0t7uQh6Yw7oPRCRBRBz26zgR+VhEhonILhE5IyJrRaRnDtt2F5HFInJcRE6KyCIRucyrzhIR+VVE+ovIOhE5CzxsrzMi8qKIjLVjOC0iy0Skjdc+eovIAhE5ICKnRGSziDydGbNbvczY7xORbcA54EZ73fN2HsfsbvufRCTGa/sedkyDRORtETksIkdEZKqIOESko53LSRHZIiJ98vue2F3+DYA77WMZEfnAbf0VIjJPRNLs92O5iFydy/9ZFxH5TUROA694x2LXLQ+MAObn0HAAwBiz2xizwG2bzCGHbiLyuYgcAVa6HTvO6xiNReRb+/8mRUSmAeVziOUO+zNwwv5/2CQiD+bn/bPrFMbn4QoR+Z+IpNrv83YRGZNDzNfZn5vM42iviyqRtOdBeXOI1S3tUeb+whizRkRWYXVXf51ZLiI1gFuAV4wxTrdNegDtgbHAWWAUsFBErjDGbLe3vdHe17fAXfZ2o4BfRORyY0y82/6aA29idXnvBg67rbsb2Ac8inXCmQAsFpFmxpjMeo2BxcA/gTNAB2A8EA6M9sq9J9AGeB5IBuLs8ihgCla3fWU75mUi0t4Ys8lrH1OB/wK3At2wvrk7gOuAV4FEu+y/ItLAGHMoH+/JYGABsMHOASDF3r4d8AuwDhgGnMIaWvhRRK40xqxxi7E6MBt4DfgHcJqcdQCqAd/ksv5CPgE+A/5CLn97RKQc8ANQEXgE6z1/EK9hGRHpCnyM9TkYifVFqAVQw62Or5+pAn0eRKQTsATYhdWwSgCaAZd7bdsEmAZMAg4BTwOfi0gLY8yunN4PpYotY4wuugAMBUwey3iv+k6ggVvZ40AGEO1WFof1Da2eW1lVrBP+f9zKdgGLvWKqhvVHdqpb2RLABbTJIQdj16/sVtYQSAdeyCVvwTqRjQXSgCCv2E8BtfN47xz2PrYD09zKe9gxvedVf61d3tWt7HK77J6LeE/igI9ziGsxsBUo5xXrVuArt7IP7GMP9OFzcqtdt08u72Pm4sjhszUlh/19AMS5vR5m141xKwsCttjlDe2yZ4DDecTq0/tX0M8DsAyIBypdIJYl9uewmVtZBNbv0D8K83dZF138seiwhfI2GOjotcTkUG82cATrj32mB4FvjTEJXnVXGLeeA2PMcaxvg10ARKQZ1reyT8S6siPY7v04BfyO9W3dXZwxZn0u8S8wxpx0O1YcsCLzWPbx6tjDCHuxGjbpwESsb60ROcR+0PsgdvfzzyKSitVgSsfqEbkkh5gWer3eBpw0xvzqVQZQz95/ft8T7/gqAt2BzwGX2/YC/JjD9ulcXG9CplvtfWQuS3Ook+Mwh5cuQLwxZkVmgTHGBcz1qrcKCLWHEfrZvV5Z8vP+FeTzICKVgKuAT4wxp/LIbacxZqdbXslYvRf189hOqWJHGw/K22ZjzGr3BVjjXckYcwZ4H7jP/sN8NdASmJnDPpNyKYuyf878A/1vPE9A6UA/IMxr2wMXiP+CxxKRIGCevd+JWFcJdARetOtWyOtY9nDAAuAEcD9W46oj1tCB9/ZgfYN1dw6r4ZXFnJ90l7l9ft8TbzWxehmey2H7R7FOvO6//ynGc6gpN5kNQ+8T3iLONzbX5rLthf7fMtUh9//DLMaYpcDNWI2t/wGZl4lmDhX49P4VwuchFOvvqHeDOSeHcyg7m8MxlCr2dM6DKoi3gKeAgVg9FnFYJxFvkbmUJdo/p9r/jsH6VuzNezb7hZ4jn9exmmCNaf/VGPNxZgUR6Z/L/nI61k1YvQ1DjDHpbvsIxatRUAD5fU+8HcEa3pmOdUllNvY3+qyXPsa1GjiGdbJ9x21fafY6ROQ4Of9t8eUYB4BWOZRn+381xnwBfCEiVbCGiCYD34lINL6/fwX9PKRhvc9ROdRVqtTSxoO6aMaYP0Xke6wJa22ACV4npEwxIlIvc+hCRKpizVL/1l6/Havh0coY83IBw7pBRCpnDl2IdVOhGCBzv5Xsf91P+iHAnfk4RiWsseqsE4mIXIP1bXzPxQbuJT/vyVmsCYZZjDEnReQX4ApgbS7/L/lmjDlrX/3wrIgMMsZ8VRj7dfM7cK+IxGQOXdi9A7dcIKYTwDci0hhrQmIYvr9/Bfo8GGNOiXXzp7tEZIIxJreJpkqVKtp4UAU1A2tGezpWF3FOkoDvxbpDZebVFpWxrpbAGGNE5BHga3u2/VysSW2RwJXAPmPMGz7Gc9o+1qtYV1s8j/VNeYq9fiuwF3hRRJx23CN8ztbyHdaNmT4Qkfex5jo8x/nejQLL53sSC1wtIv2Ag8Ahe67HU1iT+RaJyL+xvtXXAtphTWj0vpLAVxOwJnh+ISIfYc2VSMa6YqMTVoNlRe6bX9CHWFc4/FdE/mHv9yGsiY5ZRGQC1nvxM7AfiMaasLveGJN5tYkv719hfB6ewZrj8buIvI41hNEYa1LvY/ncl1Ilgs55UAX1LdYJ+2tjTE5j1WD9YX0deAmYgzXGe70xZkdmBWPdF6AbVqPiXazhj1eA2ljfRn31kR3Tv7BORCnAtca+TNOeWzAI6yT7EVa3/jLO90zkyRizCOtEdRXWifM+rEtEC/Vyu3y8J2OwvmnPxZpION7efi3W+H0q1iWN32N9M2+NlfPFxpWBdenkUKx7TLyDdRL/AOs9+QcXeddI+/+nF7Aeq2H6IVZvzkSvqiuxrqSZgnVp52Ssz9mNbvvK8/0rpM/DKqy847Eu91yA1RvnyzwIpUokMcbXoU6lshORXlgnpeuMMYtzWB8H/GqMuct7XRHEYoAXjTHPFvWxlFKqLNNhC3VRRKQJVtfsFKwx9WwNB6WUUqWTDluoi/Uc1v0LzmJ12SullCojdNhCKaWUUvmiPQ9KKaVUCSEife0Hr+0SkRyvmBKRW0QkVqwH7n3qVu4UkfX2Ms+tvJGIrLT3Oce+QunCcWjPg1JKKVX8ifWk1x1YVyQlYF1ddbsxJtatTjOsK6+uMcakiUiEfSt0ROSEMaZKDvudC/zXGDNbRGYCG4wxb10olhIxYfIhaagtHKWUUhdtpomToj5Gubb3FehcdW7de3nF2AnYZYzZDSAis7Hu8BvrVmcYMN2+62vmM1RyJSKCdVv2O+yiD7Eu975g40GHLZRSSqlCIEGOAi0+iMK6n0imBLLfGr050FxElovIChHp67augoistssH2WVhwBH7/i257TObEtHzoJRSSpV2IvIA8IBb0SxjzKx87iYYaIb1vJdoYJmItDbGHAEaGGMS7Vu5/yQim4CjFxOrNh6UUkqpQuBj70Gu7IbChRoLiVhPks0UTfbb4icAK+2H9u0RkR1YjYlVxphE+zi7RWQJ0Bb4EqghIsF270NO+8xGhy2UUkqpQuCHYYtVQDP76ohywG1Yj5R39xVWrwMiUgtrGGO3iISKSHm38quAWGNdNfEz8Bd7+3uwnld0QdrzoJRSShWCgvY85MUYkyEij2I9p8UBvGeM2WI/KG61MWaeva63iMRiPf13pDEmVUSuBN4WERdWx8HLbldpjAJmi8hEYB25P+QwS4m4VFOvtlBKKVUQ/rjaolKXxwt0rjr1+5tFHmNh0Z4HpZRSqhCIo2h7HooTbTwopZRShSCoiIctihNtPCillFKFoKjnPBQn2nhQSimlCkFZajzopZpKKaWUyhfteVBKKaUKgQSVne/j2nhQSimlCkFZGrbQxoNSSilVCLTxoJRSSql8KUuNh7IzQKOUUkqpQqE9D0oppVQhKEt3mCwTPQ8t+3Rn/LbFTNi5hD6jhudar+2Qvsw0cdRv3xqAhh2vYOy6BYxdt4Bn1y+kzaA+/go5R5pH8coDSk8umofmURRKSx6+8sNTNYuNUt/zIEFB3D59AtN63UVawkHGrJrHxnk/cGDrLo965atU5pon7mX3inVZZYmbtzOpQ39cTifVaofz7IaFbJz/Iy6n099paB4Urzyg9OSieWgeRaG05JEfJa0BUBClvuehYac2JO/ay6E98TjT01k1ez6XD+ydrd6AF55m0eSZZJw5m1WWfvpM1oc1pEJ5COATSDWP4pUHlJ5cNA/NoyiUljxUzkp94yE0KpK0+P1Zr48kHCA0KtKjTr22rQitV4fNC37Otn3DTm0Yt/l7ntu0iE8fejZgLV/Nw1Jc8oDSk4vmYdE8CldpySM/goIcBVpKkoA2HkSkRSCPb8fAzW88x5dPv5jj+rg/1jPhst683HEAfccMJ7h8eT9H6BvNo/gpLbloHsWL5lF8laU5D4Huefg+txUi8oCIrBaR1bEcv+gDpCUmEVqvbtbrGtF1SEtMynpdvmoV6l7WnKeWzObFPb/SKKYtD897N2viTqaD2/7kzIlT1L2s+UXHUhCaR/HKA0pPLpqH5lEUSkse+VGWGg9FPmFSRN7MbRVQI7ftjDGzgFkAD0nDix7w2rtqAxHNGhLWMJojiUl0vK0//77j8az1Z44d55nwdlmvn/p5Nl888yL71mwirGE0afEHcDmd1KwfRe0WTUiNS7jYUApE8yheeUDpyUXz0DyKQmnJIz9KWgOgIPxxtcW9wNPA2RzW3V7UB3c5ncx5dByPL/qIIIeD396by4HYnfR/fgR7V29i4/wfc922adeO9Bk9HGd6Bsbl4rOHn+NkalpRh5wjzaN45QGlJxfNQ/MoCqUlD5UzMUU8i1VEfgKeNcb8lsO6PcaYRnntoyA9D0oppdRMEydFfYzoO/9doHNVwif3F3mMhcUfPQ9/Ac7ktMKXhoNSSilVEuiwRSEyxhzO/FlEanqXKaWUUqVBWbo9tT8mTNYHXgGuBY5YRVIN+AkYbYyJK+oYlFJKqaJWlnoe/HGp5hzgf0BtY0wzY0xToA7wFTDbD8dXSimlVCHyR+OhljFmjjEm6/ZgxhinMWY2EOaH4yullFJFTu/zULjWiMgM4EMg3i6rB9wDrMt1K6WUUqoEKWkNgILwR+PhbuB+4Hkgyi5LAOYD//bD8ZVSSqkiFxRUYq60LDB/XG1xDnjLXpRSSilVwgX6wVj9Anl8pZRSqrBIkBRo8ekYIn1FZLuI7BKR0bnUuUVEYkVki4h8ape1EZHf7bKNInKrW/0PRGSPiKy3lzZ5xeGPYYsL6Qh8E+AYlFJKqQITKdphCxFxANOBXljD/6tEZJ4xJtatTjNgDHCVMSZNRCLsVaeAu40xO0WkLtZ8xEXGmCP2+pHGmC98jcUvjQf70dsDOT/nIRGYZ4z5P38cXymllCpqfpjz0AnYZYzZDSAis7HOrbFudYYB040xaQDGmGT73x2ZFYwx+0UkGQjHuv9SvhX5sIWIjMK6n4MAf9iLAJ/l1uWilFJKlTR+GLaI4vxVi2D1PkR51WkONBeR5SKyQkT6ZotTpBNQDvjTrfhFezhjioiUzysQf/Q83A+0MsakuxeKyBvAFuBlP8SglFJKFWsi8gDwgFvRLGPMrHzuJhhoBvQAooFlItI6c3hCROoA/wHuMca47G3GAAexGhSzgFHAhLwOUtRcQF1gr1d5HXudUkopVeL5OukxN3ZD4UKNhUSs+yRlirbL3CUAK+0v7HtEZAdWY2KV/WiIb4GxxpgVbsc9YP94VkTeB57JK1Z/NB6eBBaLyE7Od7fUB5oCj/rh+EoppVSRCyriCZPAKqCZiDTCajTcBtzhVecr4HbgfRGphTWMsVtEymE9KuIj74mRIlLHGHNArBmfg4DNeQXij/s8fCcizbEmerhPmFzlfstqpZRSqiQraM9DXowxGSLyKLAIcADvGWO2iMgEYLUxZp69rreIxAJOrKsoUkXkLqAbECYiQ+1dDjXGrAc+EZFwrPmI64GH8opFjDGFnV+he0gaFv8glVJKFVszTVyRdwu0/vu3BTpXbXrlxhJzi8qA3iRKKaWUUiVPoG8SpZRSSpUK+mwLpS5g2L71gQ6hUMyLPRjoEApNUt9sl3IrpfxMylBfvjYelFJKqUJQ1LenLk7KUDtJKaWUUoVBex6UUkqpQqBzHpRSSimVL0V9n4fiRBsPSimlVCHQxoNSSiml8sUPt6cuNnTCpFJKKaXyRXselFJKqUKgwxZKKaWUyhdtPCillFIqX/RSTaWUUkrli95hUimllFIqF9rzoJRSShUCfTCWUkoppfJF5zwopZRSKl/K0tUWZaiTRSmllFKFQXselFJKqUJQlq620MaDUkopVQjK0pyHMjFs0bJPd8ZvW8yEnUvoM2p4rvXaDunLTBNH/fatAWjY8QrGrlvA2HULeHb9QtoM6uOvkHNUWvLY8MfvPD30ZkbcfRPzPvsw2/qli77hwZv6MObBuxjz4F38vODrrHWHkg4yadRjPHPfrYy871ZSDu73Z+geDsSu5duJw/lmwoPE/vBFtvW7Vy7mf2P+yneTn+S7yU/y52/fZ61b//WHLJz0GAsnPca+tb/4M+wclZbPluaheQSSBEmBlpKk1Pc8SFAQt0+fwLRed5GWcJAxq+axcd4PHNi6y6Ne+SqVueaJe9m9Yl1WWeLm7Uzq0B+X00m12uE8u2EhG+f/iMvp9HcapSYPl9PJ+/98lTGT/0lYeATPPjKUdldeTXSDxh71Ynpcx72Pjcy2/VuTn2fQnUNp3b4zZ06fQgJ0bZTL5WT152/T85HnqVgjjB9ee4aoyzpRvU59j3r123Wl/c0PepTt37KatIQ/6fP3qbgy0vnpn2Opc2l7QipW8mcKWUrLZ0vz0DwCzVHCGgAFUep7Hhp2akPyrr0c2hOPMz2dVbPnc/nA3tnqDXjhaRZNnknGmbNZZemnz2R9WEMqlAdj/Ba3t9KSx67tsUTWjSaybhTBISF06dGLNcuX+bRtwt7dOJ0ZtG7fGYAKFStRvkKFogw3V4f37qRqeG2q1KqNIziE+u2uJnHTHz5te/TgPsKbtCLI4SC4fAWq123Iga1rizji3JWWz5bmoXko/yn1jYfQqEjS4s93bR9JOEBoVKRHnXptWxFarw6bF/ycbfuGndowbvP3PLdpEZ8+9GzAWr6lJY+0Q8mERZyPu2Z4BIdTU7LVW/XLz4wadidTnx9NanISAAcS4qlcpSpTxo9izIN/5ZO33wxYHqePpFKpRq2s1xVrhHH6aGq2evEbfmfhy4/z679f5mSalWeNuo04uHUtGefOcvbEMZJ3buLUkUN+i91baflsaR4WzSNwHEFSoKUkCWjjQUQ2BfL4dgzc/MZzfPn0izmuj/tjPRMu683LHQfQd8xwgsuX93OEvikteQC0i7maaR9/xeR3PqF1+0689crzALicGWzbtJ47HniciTPeJ/lAIku//zbA0eYu6rKO9P+/d7h+9JvUbtGGlR9PA6DOpW2p07I9P04ZxW8fvkathpcEbPjFF6Xls6V5FC+lJQ932ngoRCIyJJflJqD2BbZ7QERWi8jqWI5f9PHTEpMIrVc363WN6DqkJSZlvS5ftQp1L2vOU0tm8+KeX2kU05aH572bNXEn08Ftf3LmxCnqXtb8omMpiNKSR2itiKyeBIDDKcnUDAv3qFO1enVCypUDoOf1A9mzYxsANWtF0KBpcyLrRuFwBNPhqu7E7dzmv+DdVKwR5tFbcPpIKhWrh3nUKV+5Go6QEAAad+lFWvyfWeta9bmFvqOm0vORCRigakRdAqW0fLY0D80j0LTxULjmAAOA/l5LPyDXAWtjzCxjTAdjTIeWVL3og+9dtYGIZg0JaxiNIySEjrf1Z+O8H7LWnzl2nGfC2zG2UVfGNurKnhXrmDHgb+xbs4mwhtEEORwA1KwfRe0WTUiNS7joWAqitOTR5JJLOZgYT/KB/WSkp/P7kh9of2U3jzppqedPymt+/4Wo+g3tbVty6sRxjh1JA2DL+tVENWjkt9jd1azfjOMpBziRmoQzI519a38hqnUnjzqnjx7O+nn/pj+oFhkNWJMtz548BsCRxDiO7o+jdou2/gveS2n5bGkemkeglaXGgz+uttgIvGaM2ey9QkSuK+qDu5xO5jw6jscXfUSQw8Fv783lQOxO+j8/gr2rN7Fx/o+5btu0a0f6jB6OMz0D43Lx2cPPcTI1rahDzlFpycPhCGboY8/w8ujHcblc9Ojbn+iGjfn8g7dp3PxS2l/ZjUX/m8Oa33/B4XBQpWo1Hvz7OACCHA7ufPBxXhz5KBhDo+YtuOaGQQHJI8jhoP1fHmDpjPG4XC4ax1xL9Tr12fTtJ9Ss35So1p3ZsfQbEjf/QVCQg3KVqtD5ricAME4ni6eOASCkQiVi/joi6w9lIJSWz5bmoXmUBSLSF5gGOIB3jTEv51DnFmA8YIANxpg77PJ7gGftahONMR/a5e2BD4CKwALgCWMuPEtV8lhfYCJyNbDXGLMvh3UdjDGr89rHQ9JQp9oWI8P2rQ90CIViXuzBQIdQaJL69g10CEoVazNNXJF/tb/1gz8KdK6aM7TTBWMUEQewA+gFJACrgNuNMbFudZoBc4FrjDFpIhJhjEkWkZrAaqADVqNiDdDervMH8DiwEqvx8KYxZuGFYinyYQtjzC85NRzsdXk2HJRSSqmSwA/DFp2AXcaY3caYc8BsYKBXnWHAdGNMGoAxJtku7wP8YIw5bK/7AegrInWAasaYFXZvw0dAnl26gb7aol8gj6+UUkoVFj80HqKAeLfXCXaZu+ZAcxFZLiIr7GGOC20bZf98oX1mE+jrwzoG+PhKKaVUseB+laG9PHARuwkGmgE9gNuBd0SkRmHGmXmQIiciLbC6VjJbM4nAPGPM//nj+EoppVRRcwQV7Pu4MWYWMOsCVRKBem6vo+0ydwnASmNMOrBHRHZgNSYSsRoU7tsuscuj89hnNv64z8MorHEZAf6wFwE+E5HRRX18pZRSyh/8MGyxCmgmIo1EpBxwGzDPq85X2I0EEamFNYyxG1gE9BaRUBEJBXoDi4wxB4BjIhIj1jPF7wa+Jg/+6Hm4H2hlt4KyiMgbwBYg22UmSimlVElT1PdqMMZkiMijWA0BB/CeMWaLiEwAVhtj5nG+kRALOIGRxphUABF5AasBAjDBGJN5M5qHOX+p5kJ7uSB/NB5cQF1gr1d5HXudUkopVeL540ZPxpgFWJdTupeNc/vZAE/Zi/e27wHv5VC+GrgsP3H4o/HwJLBYRHZyfqZnfaAp8Kgfjq+UUkqpQlTkjQdjzHci0hzr+lT3CZOrjDHF/zFpSimllA8cUrJuMV0QfrnawhjjAlb441hKKaVUIJS051MUhF8aD0oppVRpV5YaD4G+SZRSSimlShjteVBKKaUKQXAZ6nnQxoNSSilVCMrSsIU2HpRSSqlCoI0HpZRSSuVLWWo86IRJpZRSSuWL9jwopZRShaAs9Txo48GPbtm5Ku9KJcDm5OOBDqFQfL8mz6fOlhhr2lwT6BAKxfYfpgU6hEJRq1Lp+NMaUnbOhYVCGw9KKaWUyhdtPCillFIqX8pS40EnTCqllFIqX7TnQSmllCoEZannQRsPSimlVCHQxoNSSiml8qUsNR50zoNSSiml8kV7HpRSSqlC4JCy0/OgjQellFKqEARp40EppZRS+eEoO20HbTwopZRShSFIJ0wqpZRSSuVMex6UUkqpQqATJpVSSimVLzphUimllFL5ohMmlVJKKZUvOmFSKaWUUsWOiPQVke0isktERuewfqiIpIjIenv5m13e061svYicEZFB9roPRGSP27o2ecWhPQ9KKaVUISjqOQ8i4gCmA72ABGCViMwzxsR6VZ1jjHnUvcAY8zPQxt5PTWAX8L1blZHGmC98jUUbD0oppVQh8MOch07ALmPMbgARmQ0MBLwbD3n5C7DQGHPqYgMpE42Hln26c8u0cQQ5HCx/dw6LJr+VY722Q/ry4JczealDf/at2UTDjldw56xJAIgI34yfyvqvFvkzdA9bVq9g7qxpGJeLq3r3o88tf/VY//sPC/jvezOoEVYLgO79b6Jrn/5s37CWL955M6vewYR93D9qPG26dPNr/Jn+3PAHP3w0A+NycUXP67lywO0e6zcuXcTiT2dRtaaVR4feA2nT8wYAfvp0FrvWrcQYQ6PW7eh19yNIgGY4xzSuyZO9muMQYd6G/fzn973Z6lx7aQT3X90YYwy7kk/wf19vAWDKrW1oFVWNjfFHeebzDf4OPZveV17GGyPvIChIeP+rX3j1/QXZ6vylV0eee2ggxhg27ojn7n/MAuD06nfZvCsBgPiDqQx58p9+jd3d6hW/8dbU13C5XPTtP4hb/zrUY/33387n3zOmEVYrAoD+N93C9QMGAZB88CBTX36BlOQkRIQJr02jdp26/k4BgN+WL+e1V1/B5XIxaNBght53n8f6+fO+ZtqUqUREhANwy623MWjIkKz1J06c4JabhtC9Z09GjR7j19jdLV++nMmvWHkMHjyY+73y+Prrr5kydSoR4VYet912G0O88hg8ZAg9e/bkH2MCl4ev/HC1RRQQ7/Y6AeicQ72bRKQbsAMYYYyJ91p/G/CGV9mLIjIOWAyMNsdlyMkAACAASURBVMacvVAgpb7xIEFB3D59AtN63UVawkHGrJrHxnk/cGDrLo965atU5pon7mX3inVZZYmbtzOpQ39cTifVaofz7IaFbJz/Iy6n099p4HI6mf3WGzw+cQqhtSJ4ecTfuDymK3XqN/Ko177bNdw2/CmPskuuaMfYf30AwMnjxxj3t1tp2baTv0L34HI5WfT+P7l9zGSqhYXz/rOP0KzdlYRHN/Co1zKmB33ufcyjLGHHFhJ2bOFvk62T1n/GP8m+rRto0DLP4blCFyTwdJ9LeOKzdSQfO8t793bkl52HiDt0MqtOdGhF7u7SkAc/Ws3xMxmEVgrJWvfJyr1UCHYwqG2U32P3FhQkTBt9FzcMf52EpMP8/sk4vlm6nq2792fVaVo/gr/fdwPdh77EkeOnCA+tmrXu9NlzdLxtfAAi9+R0Opn++mRemjqdWhGRPP63u4np2o0GjRp71Ot2TS8eeXpUtu1fnTiO2+++j3adYjh96hQSFJgpYU6nk8kvT2L6WzOJjIzk7jvvpFv37jRu0sSjXq8+vXNtGMycMZ227dr5I9xcOZ1OXpo0ibdnWnncceed9OjenSZeefTu3TvXhsH06dNpH+A8/ElEHgAecCuaZYyZlc/dzAc+M8acFZEHgQ+Ba9yOUQdoDbh/Ex4DHATKAbOAUcCECx2k1E+YbNipDcm79nJoTzzO9HRWzZ7P5QN7Z6s34IWnWTR5Jhlnzje20k+fyWoohFQoD8b4LW5vcTu2El43mvA6UQSHhNCh23VsWPFrvvez9tefadUhhnIVKhRBlHnbv2s7oZF1CY2siyM4hJZderBzzXIftxYyzp3DmZGBMz0dp9NJ5eqhRRpvblrWrUZC2mn2HzlDhsvwY2wS3ZrV8qgzsE0UX6xJ4PiZDADSTqVnrVsdl8bJcxl+jTk3HS9rzJ/xyexJTCE9w8ncRSvp38OzQXb/4O68Nfcnjhy3ejlT0o4HItQL2r51C3Wi61EnKpqQkBC6X9ub339Z6tO2e/fsxul00q5TDAAVK1WiQoB+R7Zs3ky9evWIjrby6N2nD0uXLPF5+62xsaSmHiamS5eiC9IHm73y6NunD0vykUdsbCyphw/TJcB55IcjSAq0GGNmGWM6uC3eDYdEoJ7b62i7LIsxJtWt1+BdoL3XPm4B/meMSXfb5oCxnAXexxoeuaBS33gIjYokLf78N6gjCQcIjYr0qFOvbStC69Vh84Kfs23fsFMbxm3+nuc2LeLTh54NSK8DwJHUFELtrlaA0FrhHElNyVZv3fKlTHzkHma99CyHU5KyrV+9bDEdu19XpLFeyPG0Q1QLO59H1ZrhHD+cmq3etlW/8M6oYXw59XmOpSYDEN28JQ1ateHNh2/hzYdvofHlHagV1SDbtv4QXrUCycfOZL1OPn6W8KrlPerUq1mJ+jUr8fZf2/POPR2IaVzT32H6JCqiBglJh7NeJyalUTfcs1HWrEEkzevXZsn7Y/jlw7H0vvKyrHUVyoXw+yfj+OXDsQzo0dZvcXtLTUkmPOL873atiAhSU5Kz1ft16U88dPdtTBz7d1KSDgKQGL+PKlWqMmHMSB4Zegfv/GsazgD9ricnJxMZWTvrdURkJMk55PHT4sXcdsvN/P2ZZzh40MrD5XIx5Y3XefKpp7LV97fk5GRq1/bMIyk5ex6LFy/mLzffzNNeebz++us8XQzyyI8gkQItPlgFNBORRiJSDmv4YZ57BbtnIdMAYKvXPm4HPstpG7HGgAcBm/PM1ZdoC4OIRIpIO3uJ9KH+AyKyWkRWx1J033JEhJvfeI4vn34xx/Vxf6xnwmW9ebnjAPqOGU5w+fI51isOWne+ionvf86z0z/k0rYd+PANz5yOHj7E/rjdtGyX0xBZ8dG0XQyPTPuYYZPfoVHr9sx/6xUADh9M5FDiXh7712wemz6HvVvWsW/bpgBHm7vgIKFezYo8/Mlaxn21mdE3XEqV8iVzpNDhcNC0fiTXDXuFv455m7eeG0r1KhUBaHrDSLrcOYG7/zGL10beTuPo8ABHm7uYrlfz4RfzmfnRbNp27MxrE8cD4HRmsHnDOoY9+gRvvvsRB/cn8MOC+QGN9UKu7tad+d8uYPbcz+kcE8P4cc8B8PncuVzVtSuRkXn+iS0WunfvzsIFC/ji88+JiYnh2eesPObMnUvXEpRHJocUbMmLMSYDeBRryGErMNcYs0VEJojIALva4yKyRUQ2AI8DQzO3F5GGWD0X3l1yn4jIJmATUAuYmFcsRf6XzL5edCZQnfPdK9EicgR42BizNqft7O6aWQAPScOLHi9IS0witN75SU81ouuQlnj+G3n5qlWoe1lznloyG4BqtcN5eN67zBjwN/atOX9iOrjtT86cOEXdy5p7lPtLjbBw0g6db7WnHUqhRpjnH+kq1apn/XxV7/789z3PiaFrfvmJNl2uxhEcuBNY1dBaWT0JAMcPp1C1ZphHnUpVz+fRpuf1/Pyp1XO3Y9WvRDVtSbkK1kmrcZtOJO6MpX6L1n6I3FPK8TNEVDvfrR1RtTwpxz3nFyUfP8OW/cdwugwHjp4h/vAp6tWsyNYDxavLPzH5CNGR53tFoiJD2Z+S5lXnMH9s2kNGhpO4/YfYufcgTetHsiY2jv0pRwDYk5jCstXbaNOiPrsTsveKFbWw8AhSks//bh9KTiYsPMKjTrXqNbJ+7tt/EP+eYU0krhUeSZNml1AnKhqALt16sG1Lnl++ikRERARJdo8IQHJSEhFeedSocT6PQYMH8+a0qQBs2riBdevW8cXcuZw6fZqM9HQqVazEY0884Z/g3URERGT1JICVR2RE7nkMGTyYqVOtPDZu2MDadeuYa+eRnp5OpUqVeDIAeeSHP25PbYxZACzwKhvn9vMYrDkMOW0bhzXp0rv8muy1L8wfPQ8fAE8YYy41xlxnLy2AJ7HGVorU3lUbiGjWkLCG0ThCQuh4W382zvsha/2ZY8d5JrwdYxt1ZWyjruxZsS6r4RDWMJoghwOAmvWjqN2iCalxCUUdco4aNG9BcmI8hw7uJyM9ndXLfuTyzld51Dl6+FDWzxtX/krtep5d+quW/kiH7r38Em9u6ja5hLSDiRxJPoAzI53Y35fQrP2VHnVOpJ0fxti55nfCouoDUK1WBPu2bsDldOLMyGDf1o3Uqlvfr/Fn2rr/OPVCK1GnegWCg4TrWkbyy85DHnWW7UihXX2r+796xRDq1axE4pHTgQj3glZv2UPT+pE0rFuLkGAHt/TpzDdL1nvUmffzOrp3uASAsBpVaNagNnsSU6hRtRLlQoKzyru0acbW3Qf8ngPAJS1asj8hnoP7E0lPT2fp4u+J6ep5RVHqofP/Ryt+XUb9BtaE4+aXtuTEieMcSbMaTRvWrKZ+Q8/JyP7SslUr4vftIzHRyuP7RYvo1qO7R51DKecbZ8uWLqVRIyvWiS9N4tuF3zF/wUKeHDGCG/r1C0jDAaBVq1bs27ePBDuP7xYtont3zzxS3PJY4pbHpEmTWPTddyxcuJCnRoygX79+xb7hUNb44ytoZWPMSu9CY8wKEalc1Ad3OZ3MeXQcjy/6iCCHg9/em8uB2J30f34Ee1dvYuP8H3PdtmnXjvQZPRxnegbG5eKzh5/jZGparvWLksMRzG3Dn+Kfzz2Fy+Xiyl43UrdBY+b/513qN2vBFTFd+XneF2xc+StBDgeVq1TjnhFjs7ZPTTpA2qFkmrX2/5UJ7oIcDnoPfYzZL4/G5XJxRY++hEc3ZOnnH1CncXOat7+SVYv+x841vxPkcFChSlX6Pfh3AFp07kbclvW8M2oYCDS5vCPN2gdmMpXTGF7/fjtTb2tLUBB8s+EAew6dZFi3xmw9cIxfdx5ixe7DdGoUxqcPxOByGf710y6OnbYmSb711/Y0CKtEpRAHXz96FS99u5WVew7ncdQiysXp4snJH/PtjKcICgriw69/JXb3fv5v+CDWxMbxzdL1fP/bZq7r0ooNX07E6XQxZupcDh89ScwVTZgx9h5cxhAkwqvvL/C4SsOfHMHBPDxiJGOfegyX00nvfgNo2LgJH70zk2YtLqXL1d35+vPZrPh1GY5gB1WrVuPpZ8db2zocDHvkCUY/MRyMoekll3L9gMEBySM4OJiRo0bz2MPDcbpcDBg4kCZNmjJzxgwubdmS7j16MPuzz1i2dAkORzDVqldj/PMXnBgfEMHBwYwZPZrhw4dbl5wOHEjTpk2ZPmMGrVq2pEePHnz62WcsWbKE4OBgqlWrxgsTil8e+eEoQ7enFlPEVxCIyJtAE+Ajzl+fWg+4G9jjfResnBRk2KI4uWXnqkCHUCjij57Ju1IJMHPRjkCHUGjWfP5JoEMoFNt/mBboEApFrUolc26Lt5BSdC6sULFikWfz066UAp2rrmkaXmLe8SL/hBtjHheR67HugpU51pIITLfHbpRSSqkSz6GP5C5cxpiFwEJ/HEsppZQKBH9MmCwuAnqfB/tuWkoppZQqQQI9MFd2mmlKKaVKNUepv+3ieYFuPJwL8PGVUkqpQqHDFm7E0th+jnhhe74I9qmUUkr5nUOkQEtJkmfPgzHG2LetrJpX3ZyIyMbcVgEl696jSimllPJ52GId0BzYdhHHiAT6AN53VxLgt4vYn1JKKVXslKVhC18bD0uA70TkA6wbPWXdCMMY814e234DVDHGrPdeISJLfDy+UkopVazphMnsrgL2AN29yg1wwcaDMeb+C6y7w8fjK6WUUsWa9jx4Mcb0LOpAlFJKqZKsDLUdfL9JlIiEichfRWSk/bquiEQXXWhKKaWUKo58ajyISHdgO3AnkPnc8GbAW0UUl1JKKVWiBCEFWkoSX+c8TAVuNcYsFpHMqyZWAp2KJiyllFKqZClLwxa+Nh4aGmMW2z9nXmlxLh/bK6WUUqVaUBlqPPg65yFWRPp4lV0HbCrkeJRSSqkSSaRgS0nia8/B08A3IvItUFFE3gb6AwOLLDKllFJKFUu+Xqq5QkQuB+7Cuq9DPNDJGJNQlMEppZRSJUVJm/RYED41HkTkGWPMa8ArXuVPGWPeKJLI3Nz555qiPoRfHDhxNtAhFIpFWw4GOoRCcezQqUCHUGgqhtYOdAiFIsMV6AgKh8vkXackcJW0vvQAK0tvl69zHsblUv5sYQWilFJKlWRBUrClJLlgz4OIXGP/6BCRnuDRJ9MYOF5UgSmllFKqeMpr2OLf9r8V8HyGhQGSgMeKIiillFKqpClhnQcFcsHGgzGmEYCIfGSMuds/ISmllFIljz4Yy4sx5m4RCQFigLrGmDkiUtled7IoA1RKKaVKgjLUdvD52RatgR3AO5wfyuhOHo/jVkoppcqKoAIuJYmv8b4FjDPGtADS7bKlQNciiUoppZRS2YhIXxHZLiK7RGR0DuuHikiKiKy3l7+5rXO6lc9zK28kIivtfc4RkXJ5xeFr46EV8LH9s4Gs4YqKPm6vlFJKlWoiUqDFh/07gOnA9UBL4HYRaZlD1TnGmDb28q5b+Wm38gFu5ZOBKcaYpkAacH9esfjaeIgD2nsl0QnY5eP2SimlVKnmh/s8dAJ2GWN2G2POAbMp4GMixGq1XAN8YRd9CAzKaztfGw/PAd+KyPNAOREZA3yO3iRKKaWUAvzyYKworMdDZEqwy7zdJCIbReQLEannVl5BRFaLyAoRyWwghAFHjDEZeezTg0+NB2PMN0BfIBxrrkMDYIgx5ntftldKKaVKu4JOmBSRB+yTe+bywEWEMR9oaIy5HPgBqychUwNjTAfgDmCqiDS5iP0Dvj9VE2PMOuDhiz2QUkoppXJnjJkFzLpAlUTAvSch2i5z30eq28t3cXsmlTEm0f53t4gsAdoCXwI1RCTY7n3Its+c+PpgrGDgdvtAVbwCvZiWkVJKKVWq+DLpsYBWAc1EpBHWCf42rF4E9xjqGGMO2C8HAFvt8lDglDHmrIjUAq4CXjHGGBH5GfgL1hyKe4Cv8wrE156Hj4HWwEKs21IrpZRSyk1RP9zKGJMhIo8CiwAH8J4xZouITABWG2PmAY+LyAAgAzgMDLU3vxR4W0RcWKMkLxtjYu11o4DZIjIRWMf5+znlytfGQ1+gnjFGH4SllFJK5cAfN5g0xiwAFniVjXP7eQwwJoftfsPqBMhpn7uxruTwma9XW2wBauZnx0oppZQqnXztefgr8K6IfI/XsIUx5qNCj0oppZQqYYp62KI48bXxMBS4GggFTruVG0AbD0oppco8P0yYLDZ8bTw8AbQ1xmwtymCKyubVK/hs5lRcLidX9+3PDbd4Pl18+Q/f8vm70wmtFQ5Az/430a3vALZtWMOcWW9m1TsQv5cHRz9P2yu7+zX+TDvX/cG37/8L43LS/tob6TbYY5Ita3/+jkX/mUm1mrUA6Hz9YDpceyMAi/4zk+1rV2CMoenl7bnh3seKxQf98rrVuLtjfYJE+HlXCvM3H8xWp3ODUG66wrpnyd60U0z/Zbe/w8xR1xbhjB50GY4g4csV+3j3p+w3XO1zRR0e6XMJBti+/yh//3gdLepW47m/tKZKhRCcLsOsH3fy3fr9/k/AzbUdmzH50X44goL4aMEqpny2LFudwd1bM/qeazEYNv95kL+9OCdrXdVK5Vn5/pN8uzyWkW/O92foHtas/I1Z017D5XLRu98gbr5rqMf6HxfM570Z0wgLjwCg35Bb6NPfuldOctJB/jn5BVKSkxCE8a9OI7JOXX+nAMDvy5fz+muv4HK6GDh4MPfce5/H+m/mfc2bU6cSHmH9zbr51tsYNHgIADEd2tGkaVMAateuw+tTp/k3eDfLly/n1Vcm43K5GDR4MPfd53nX43lff82UqVOIsP8/br3tNoYMGZK1/sSJE9w0ZDA9e/Zk9Jh/+DX2i6E9D9klAfuKMpCi4nI6+WT6azz10jRCa0Uw8Yn7adP5auo2aORRr2P3a7nz4ac9ylpc0Z7/m27dX+PE8WP8476badmus99id+dyOpn/72kMfe5VqtUMZ+aYh2jR4Uoi6jX0qNf6yp70+9sTHmX7tm9m3/bNPPqaNYH23eceJy52A41atfFX+DkSgXs7N2DSDztIPXWOiTe0ZG38ERKPnsmqU7tqeQa2rsPz323l5Dkn1Sr4fGuSIhUkMHZIa4bNXEHS0dPMGXE1P285yJ9JJ7Lq1K9VmWHXNuOufy7n2Ol0alaxnjVzOt3JmE/Xs+/QScKrlefzp7qxfFsyx89k5Ha4os0lSHj9iQEMGvkeiSnH+Pmth1nw2za2703OqtM4Koyn7uhOn8dncuTEGWrVqOyxj7H39uK3jXF+jtyT0+nkrTcmM3HKdMLCIxkx7G46X9WN+o0ae9S7+tpeDB8xKtv2b0wcx61330fbjjGcPnUKCQrMcw6dTievTJ7Ev2bMJCIyknvuupOru3encWPP+/n06t2bkaOzzYujfPnyfDJ7rr/CzZXT6eTlSS/x1sy3iYyM5M4776B79x40aeKZR5/evXNtGMyYPp127drnuK44KkNtB58nTE4BPhaRGBFp7L4UZXCFYc+OWCLqRhNeJ4rgkBA6db+O9St+yfd+1vzyE607dKF8hQpFEGXeEnZtI6x2XWpG1iU4JITWV13D1tXLfdxayDh3DmdGBhkZ6TidGVSuHlqk8fqiaVhlko6fJfnEWZwuw+9xh2lfzzOuns3C+X5bMifPOQE4FqATrLfW9UOJP3SShMOnSHcaFqzbT8/LanvUuTmmPp8tj+PYaetBtIdPnANgb8pJ9h06CUDKsbMcPnGW0Crl/ZuAm/YtotmdmErcgTTSM5z896eN3HjlpR51ht7YkXe+XsGRE1bD7tCRk1nr2jSrS0RoFX5avdOvcXvbsXULdaLqUbtuNCEhIXS7tjcrfl3q07b79uzG5XTStmMMABUrVaJCgH7Xt2zeTHR0PaKirTx69+nDsiVLAhJLQWzevJl69eoRbefRp09fluQjj9jYWFIPp9KlS5eiC1JdNF+/xk23//V+AIfButa02Eo7lEJoeGTW69Ba4ezeHput3tpfl7Bj03pqR9Xj1gefoKbbNgCrlv1Ir8G3F3m8uTl2+BDVwyKyXlevGU7CzuyjSFtWLiNu60bC6kRzw9BHqF4rgvqXtKLRZW155YGbMAY69x1ERHQDf4afo9BK5Ug9eS7r9eFT52hay/MbbZ1q1h/w/+vbgiARvtyQyMb9x/waZ04iq1fgwJHz03+Sjpzh8gY1POo0CLfup/bxY1cRFCTMWLSdX7eleNRpXb8GwY4g4lNPEih1a1UnMflo1uvEQ0fpcGk9jzpNoq2hsEVvPogjSJj04WIWr9qJiDBx+A088NJcerRv6te4vaWmJBMecf73tlZ4BNu3bs5W77clP7Fl/Trq1qvPsMeeIjyyNonx+6hcpSovjh1J0oFE2rTvzD0PPYrD4f8/bykpyUTWPt8QjYiIZMvmTdnq/fTTYtatXUv9Bg0Y8fQzWducO3eOu++8g2CHg7vvvZcePa/xW+zukpM984iMjGDzpux5LF68mLV2Hs88M5LatWvjcrl44/XXefGll1i5YoU/wy6QoGIwFOwvvj7bIiiXxaffLBGpLiK3ishT9nKriNTIe0v/uKJzV17+4Euef+s/tGzXifdef8Fj/ZHDh0jYs5tW7QMzZOGrFh268PSMz3j09X/T9Ir2fPmvlwFIPZBISsJenpn5OSPf/pw9m9cRt3VjgKP1TVCQULtaBSYu2s6/fvmTYV0aUSmkWLdXsziChPrhlRk6/TdG/mcN42++gqpuwy61qpZn0h1teXb2eowJYKA+CHYE0Tg6jBtHvMP9E+fw5tODqV65An8b2JkfVm5n/6HAN+h80emqq3nv8/n868PZtO3YmSkvjQfA6cxgy8Z13P/IE0yZ9REHDySweGHg5m7kpWu37nz9zQI+nfs5nTrHMH7cc1nrvv52AR998ikvvDSJKa+9SkJ8/AX2FFjdunfn2wULmfv5F8TExDDuOetZi3PnzqFr165ERkbmsYfixQ8Pxio2inxQT0TuBtYCPYBK9tITWGOvy227rAeEzPvsw9yq5Sm0VjhpKeevLk07lEJoWLhHnSrVqhNSzhqPvrpPf/bu3O6xfvWyxbS7shvBwYEbb69WsxZHU8+PQR89nELVsFoedSpVrU5wiJVH+2tuZP/uHQBs/eMXopu3pHzFipSvWJFmbTsRv2OL/4LPRdqpc4RVLpf1umalchw+le5R5/DJc6yNP4LTGFJOnOPAsTPUrhaY7mR3SUfPUKdGxazXkTUqkOQ2V8Oqc5qfNyeR4TIkHj7N3pQTNAi3elYqlw/mrWGdeXPBNjbuPeLX2L3tP3SUqIjqWa+jalXnQIpnY2B/ylEW/raVDKeLvQfT+DMhlSbRYXRqWZ9hg7qw8dORTHzoem7r1Zbxw/r4OwUAwsIjSEk+/7t+KCWZsFoRHnWqVa+R9bveu98gdm23eu9qRUTSuOkl1K4bjSM4mJiuPfhzh+ffAX8JD48g6eD5icPJyUmER3jmUaNGDcrZeQwcPJht2873QkbYvS9R0dG069CB7du3+SHq7CIiPPNISvLsGQLPPAYPHsLWrVYeGzdsZM6c2dxw/fVMmfIG33zzDdOmTfVf8BdJjCnQUpL41HgQkWAReVxEvhSRpSKyLHPxYfOxQHtjzHBjzER7eQjowAUe6W2MmWWM6WCM6TDg9nt8yyYHDZtfStL+BFIO7icjPZ0/lv7IFTFdPeocOXwo6+f1K36ljtckxD+W/EinHr0uOobCENW0BakHEklLOkBGejqblv9Eiw5XetQ5nnb+eSjbVv9GeHR9AKrXiiAudgNOpxNnRgZxsRsIjwr8sMWfqSepXbU84VXK4QgSujSsyZr4NI86q+PTuLR2VQCqlg+mTrUKJJ84k9Pu/Gpz/BHqh1cmqmZFQhzCDW3r8rPXlSI/bT5Ip6ZhANSoXI4G4VWITz1FiEN4894OzFsdz/cbD+S0e79auy2RJlG1aFA7lJBgB0OuuZwFv3sOiX2zPJauV1hTnGpWq0ST6DD2HDjMsJfmctntr3D5Ha/y7MyFzP5hHePfWRSINGjeoiX7E+I5uD+R9PR0li3+ns5du3nUOXzo/O/6yuXLqGdPnG7WoiUnThznaJr1+du4djX1GnpOqvaXlq1aER+/j8REK4/vFy3i6u6eV3gdSjk//LVs6VIa2bEeO3aMc+esocAjaWlsXL+eRo0DMzWtVatW7Nu3j8TEBNLT01m06Dt6eOWR4pbH0qVLaNTIyuOlSZNY+N0iFixcyIgRT9GvXz+eeOJJv8Z/UYyrYEsJ4utX6SnANVhP+3oRq0EwHOshGnkRrLkR3lz4YXKqwxHMHcOfYuqzI3A5nVzVux9RDRrz1Ufv0LB5C9rEXM3irz9nw4pfCXI4qFy1Gvc+PTZr+0NJBzh8KInmrdsWdagX5HA46Hf/43z44t9xuVy063k9kfUasXj2e9RtcgmXdryK3xf8l22rlxPkcFCpSjWGPDIagFYx3dm9eR3/evo+BKFZm47ZGh6B4DLwwR/7GH3dJQQJLNl1iMSjZ/jLFXXZnXqKtQlH2Lj/GJfXrc4rAy7DZQyfronnxFlnoEPH6TK8+N/NzHoghqAg4X9/xPNn0gke7XsJW+KP8POWJH7dlsKVzcOZ9/ceOI3h9fmxHD2VTr/2UbRvEkaNyuUY1NGaWzD2s/VsC9BcDqfLxTP/nMd/J9+LwyF8vHAN2+KS+cfQ61i3I4GFv21j8aqdXNOhGSvfexKny8W4t78j7djpvHfuR47gYB4aMZJxTz+Gy+Wk140DaNCoCR+/O5NmLS6lc9fuzPtiNn8sX0aQw0HVatV48h/jrW0dDu5/5AnGPjkcg6Fp80vp039wQPIIDg5m5KjRPP7IcFwuF/0HDKRJk6a8/dYMLm3Zkm7dezBn9mcsW7oEhyOY6tWrMe75mqW5eQAAIABJREFUCQDE7dnNpBcnIhKEMS7uvve+bFdp+DOPUaPH8PBwK4+BAwfRpGlTZsyYTsuWrejRowefffYpS5cswREcTPVq1Xh+wgt571gVC2J86CoRkUSgizFmn4gcMcbUEJEWwNvGmAve9EBE7gHGAd8DmYNv9YFewAvGmA/yOv4vu1NLVn9OLg6cOBvoEArFV+vzfFpribBhffZ7SpRUCet/D3QIhWL17OyXHpZEEZWLxyXFBRVSim5cUKlihSJP5uzJ4wU6V5WvXLXEvOG+fsIrcf7Ef1pEKhljtolInl/HjTEfisg8oA8QZRcvAcYYY9Jy3VAppZQqSf6fvfsOj6pKHzj+fTOB0JJAOim00LEgRUCRgC5NQYo/V1d2QWwrNsQG6sq6FhBsgAsI68rqiiK6uiBSVBRYQXqTAEFqSAikQggtycz5/TFDyCSEBJK5k8D7eZ77MPeec+9938yQnDn33HOr2KWH8ihr42EH0BFYC6wHXhaRbJzPEy+Vq5FQlkscSimlVNVUxQY9lkdZ77YYCZwdBv8U0A7oDzxUnpOLyMzy7K+UUkop65Xa8yAiNpzPAJ8NYIz5DfhdBZ1/RgUdRymllPKuK+iyRak9D8YYO/COMabCR/sZYzZU9DGVUkopbxDjKNdSlZT1ssU3ItL/Uk7gml3yDRHZKSKZIpIhIjtc2yrNLJNKKaVUueg8D8XUAL4UkV9w3nVRMCrEGFPiLJEuc4Efge7GmMMAIhIBDHOV9brYoJVSSqlKp4o1AMqjrI2Hba7lUjQyxkwovMHViJggIveVsI9SSimlKqkyNR6MMX8rxzkOiMhzwEfGmCMAIhIO3Mu5uSOUUkqpqk17HooTkepACyCEQtNKG2N+LGXXu4AxwHIROft0lyPAfODOi4pWKaWUqqwc2nhwIyJdgS8APyAAyAb8cfYcXPCpK64Joka7lqLHHQ7MuriQlVJKqcqnqt0xUR5lvdviXWCiMSYIOO7691VgWjnPX57LIUoppZTygrJetmgOTC6y7Q1gH/DWhXYUka0lFQHhJZQppZRSVcsV1PNQ1sbDMZyXK44CKSLSGsgA6pRh33CcD8Uq+hAsAVaV8fxKKaVU5XYFPduirI2Hr4BbgU+BD4GfcD7r4ssy7LsAqGOM2Vy0QESWlfH8SimlVOWmPQ/ujDFPFnr9loisxjlgckkZ9r3/AmX3lOX8SimlVGWnAyZdRKSFiKwSkWwRWSYijQGMMT8bYxYZcwX9pJRSSikvE5E+IpIgIrtFZMx5yu8VkTQR2exaHnBtbysiv4hIvIhsFZG7Cu3zLxHZV2iftqXFUVrPw3vAXuA1YAjOuy4GXkyiSiml1BXBw9+nXU+5ngr0BJKAdSIy3xizvUjVz40xjxXZdhIYaoz5TUQigQ0issQYc9RV/qwxpixDEYDSGw/tgGhjzGkRWQHsKuuBlVJKqSuK5zvjrwd2G2P2AojIHGAAULTxUIwxZleh14dEJBUIxXkjxEUrrfFQ3Rhz2nWyHBGpcSknKa+Dx05547QVzs+3rNNqVG4nc+3eDqFCBNUvy81CVUPKdq/811RKFeb5xkMU7o91SAI6nafeHSLSDecX/lHGGLdHQYjI9UB1YE+hza+LyFhgKTDGGHPmQoGU1njwE5FXCq3XLLKOMWZsKcdQSimlLnvlHTApIg8BDxXaNNMYM/MiD/MN8Jkx5oyI/Bn4CLi50DnqA/8GhhUat/g8cBhng2Imzhmh3f7WF1Va4+FTIKbQ+pwi61fOTa1KKaWUB7kaChdqLCTj/jc42rWt8DEyCq1+AEw8uyIiAcC3wIvGmNWF9klxvTwjIrOAZ0qL9YKNB2PM8NIOoJRSSimseDDWOqCZ687HZOBuwG3KAxGpX6gxcDuww7W9OvA18HHRgZFn9xERwXlTxLbSAinzUzWVUkopdQEenmHSGJMvIo/hnGPJBnxojIl3DSdYb4yZDzwhIrcD+UAmcK9r998D3YBgETm77V7XBI6zRSQU58zPm4GHS4tFGw9KKaVURbBg6iNjzEJgYZFtYwu9fh7nGIai+30CfFLCMW8+3/YLuTyG/yullFLKMtrzoJRSSlUAnZ7aRURuFJEJJZS9ISKdPROWUkopVcUYR/mWKqS0yxYvACtKKFsGvFih0SillFJV1RXUeCjtskVbYHEJZT/gfDy3UkoppRyXx+y7ZVFaz0MAzhmnzqcazsdyK6WUUuoKUlrjYSfQq4SyXq5ypZRS6opnHI5yLVVJaZct3gVmuB4D+l9jjENEfHDOQDUVeMrTASqllFJVwhV02aK06ak/FZEInA/W8BORdCAEOAP81RjzmQUxKqWUUpWfNh7OMca8IyIfAF2AYCAD+MUYk+3p4JRSSqmqwti18eDG1VBY4uFYlFJKKVUF6AyTSimlVEWoYoMey0MbD0oppVRF0DEPl5fdm9ey+KOpOBwO2t18K10H/MGtfPOyxXw/eyb+QSEAXN97AO1uvg2A72fP4LdNazAOQ5Nr2tNn2KM4H3luvV2b1rJg1t9xOOx0vOU24ga5PcadDT8tZtG/3yfQlUfnPoPo+DtnHov+/T4JG1ZjjKHpNe3pd9/jXsujsOuiA3mgcyN8RPg+IZWvth4qVufGxkHc3S4aA+zPOMk7y3ZbH+h5XN8oiJG3NMNHYMHWFGavTSxWp0eLUO67oTEG2J2awyvfbgfgrf+7htb1A/g1+Rijv/rV4siLu6V9LONG9Mbm48O/F29i8tyVxeoMvKk1o/8Yh8Gwbe8RHprwdUGZf63q/DLjEb79ZSejp5U0r5znbVizipmT38LhcNCr30Du/OO9buU/LPyGD6dNJjg0DIB+g39P7/4DAUg9cpj3JrxKWuoRBOHlNycTXj/S6hQA+GXlSt5+ayIOu4MBgwYxbPh9buUL5s9jyqRJhIaFAnDnXXczcNBgADp3aEds06YARETU5+1Jk60NvpCVK1fy5sQJOBwOBg4axH333e9WPn/ePN6d9C5hrvfjrrvvZvDgwQXlOTk53DF4ED169GDM8y9YGvulMNp4uHw4HHYWfjiFP704kYDgUP7xwiO0aN+F0OhGbvXadOnOrfc94bbtYEI8BxPieXjiPwCY9deRHNi+hUZt2loVfgGH3c78DyZz39g3CQgKZdqYh2nZ4QbCYxq51bvmhh7c/sBIt20Hdm7jwM5tPPH2PwGY8dIT7IvfQpOrrM+jMB+BP9/QmL8u2kHGiVzeHHAVaxOzSDp6qqBO/YAa3HFtFGO+iedErp3AGpXjI+sj8FTP5oyau5m042f4x586sHJPOvszThbUia5bkz92asiITzeScyafurWqFZR9tvYgftV8GHCtd/44FebjI0x8tC+DX/iEQ+nZLJ3yAItXJ5CQmF5Qp0lkEE/edSN9np7FsZzThATWcjvGC0N7sGrbAatDd2O325n+zgRee3cqwaHhjHpwKJ1u7EaDxk3c6t10S09GjBpdbP93XhvLXUPv47qOnTl18iTi452HDtvtdiZOGM/fp71PWHg4w/44hJvi4mjSJNatXs9evXh2TLEnL+Pn58fsOXOtCrdEdrudN8aPY/r7MwgPD2fIkHuIi+tObKx7Hr179SqxYTBt6lTatWtvRbjqIl32j+RO3r2ToIgo6oVHYvOtRpsberBz/aqy7SyQn5eLPT8fe14edrud2nXreTbgEiTt3klwRCRB4ZH4VqvGNTfezI51xb8dno+IFOSRn5+HIz+fOl7Ko7BmoXVIyT7NkeNnyHcYft6bQaeG7nH1ahnGwh2HOZHrbNEfO53vjVCLaVU/gOSsU6QcO02+w7B05xG6Ng1xq9P/2ki+3pRMzhlnzEdP5hWUbUjM4mRu5fiW0r5FFPtSsjhw+Ch5+Q6+Wh5P3y4t3OoM7duOfy5Yz7Gc0wCkHzvXSLq2aX1C69bmp417LY27qF074qkfFUNEZDTVqlWj2y29WP3z8jLtm7hvLw67nes6Op/1V7NWLWrUqOHJcEsUv20b0dExREU78+jVuzcrli3zSizlsW3bNmJiYoh25dG7dx+WXUQe27dvJyMzgy5dunguyIrmcJRvqUIqx9c4DzqemU5AcGjBekBQKMm7dxSrt2Pt/ziwcyvBEdH0HvoIgSFhxDRvQ6PWbXn74TvBQMfeAwiNamhl+AWOZaYTGBJWsB4YHMrB34rnEb96Bfu3byU4Mprb7n2UuiFhNGjRhiZtrmP8g3dggC59BhIW7Z08CguqVZ30E7kF6xkncmkWWsetTmSA8xf4+P5t8BGYszGJTUnHLI3zfELr+JF6/HTBetrxM7SqH+BWJ6ZeTQCm3dMOH4EPV+5n7f5MS+Msi/rB/iSnnfuZHkrPpn2LKLc6sVFBACx6ezg2H2HCJ8tZumEPIvDqQz15eOLXxF3n/g3fahlpqYSGhResh4SGkbBjW7F6q5b9SPzmTUTGNODBx58iNDyC5IOJ1K7jz+svPsuRlGTatu/EsIcfw2azWZkCAGlpqYRHRBSsh4WFE7+t+KWtH39cyqaNG2nQsCGjnn6mYJ/c3FyGDrkHX5uNocOH073HzZbFXlhqqnse4eFhbPu1eB5Lly5loyuPZ555loiICBwOB++8/TavjxvHmtWrrQy7XK6kyxaW9DyISG8RmS4i813LdBHpU8o+D4nIehFZ/+N/Zns0vubtuzDyvdmMmPgBTa5pz3+nO59Cnnk4mfRDiTw17XOemv45++M3cWDHVo/GUh6tOnTh2emf8cQ7/6TpNe358u9vAJCRkkxa8gFGz/iCMTO+YM+2TezbXnnzKMzHR6gfUIO/LNjO2z/u5tGuTahd3fpf6JfC5iNE16vJ43M28bcF23mudwvq+FXN9rqvzYcmkUH0f+4jHnjjKyY92Y+A2n7c368j36/dzaH0494OsUyuv/EmPvziG/7+0Ryu69iJd8e9DIDdnk/81k3c/+hI3p35MYdTkli66BvvBnsBXbvFMW/BQj6d+wXXd+rMy2NfKiib9+1CPp79Ka+OG8+7b71J0sGDXoz0wrrFxfHtwkXM/eJLOnfuzNiX/gLA3Lmf07VrV8LDw0s5QiXjsJdvqUI8/ptMRCYBzYGPgSTX5mjgCRHpa4wZeb79jDEzgZkAn25KMpd6fv+gELIz0grWszPTCgZGnlXLP7Dgdbubb+WH2c4xDjvW/UxU01ZUr+H8Btm07fUk/badhq2uudRwLllgUAjH0lML1o9lpBFwgTw63nIbiz+ZCUD82v8R06w1fjWdeTS/7noO7oqncWvr8ygs82QuIbXPPXctuHZ1Mk/mutXJOJHLrtQc7MaQmnOGQ8dOUz+gBrvTT1gdrpu0nDOE+Z/r1g719yM954xbndTjZ9iRko3dYUg5dpqkrFNE16vJzsOV6w9tSsZxokLPfXYiQwJIyXCP8VB6Nht2JpNvd5B45Ci7kzKJjQqmY6toulzVgPv7d6B2jepU97Vx4lQer8xaanUaBIeGkZZ6pGA9PS2V4EK9dQABgXULXvfqN5BZ06cAEBIWTpOmLYiIjAagc9fuJGwv3mthhdDQMI4cPlywnpp6hNAw9zzq1j2Xx4BBg3hvyqSC9TBX70tUdDTtOnQgIWEn0TExHo66uLAw9zyOHHHvGQL3PAYNGszkSc48tm7ZyqZNG5k7dy6nTp0kLy+PmrVqMXLkk9YEf6mq2KWH8rCi5+FWY8ytxpg5xpifXcsc4DbgVk+fPCq2JRmHk8lKTcGen0f8qp9o0f4GtzrHszIKXies/4WQqAYABAaHcWDHVhx2O/b8fA5s31pQZrWopi1JT0km80gK+Xl5bF35I606uueRXSiPHetXEeaKtW5IGPu2b8HuymNf/BZCK8Fli9/ScqgfUIOwOn74+ghdmwSz9kCWW501BzK5ynU5wN/Pl8jAGhw5fuZ8h7PUzpTjRNerSf3AGvj6CLe0DOfn3eludf73WzptY5y/HANrViO6Xk0OFRoMWllsTEimSWQQDcLrUs3Xh8FxbVi8epdbnYWrErjxmkYABAXUpGl0EPtTsvjzxK+5Zuhk2g6bwtgPvmfO0i1eaTgANG/ZmkNJBzl8KJm8vDxWLP2OTl27udXJTD/3Hq1ZuYKYho0BaNayNTk5xzmW5fz8bd24nphGja0LvpDWbdpw8GAiycnOPL5bsoSb4uLc6qSnnftCtGL5chq7Ys3OziY319kAP5qVxdbNm2ncxDuXk9q0aUNiYiLJyUnk5eWxZMliuhfJI61QHsuXL6NxY2ce48aPZ9HiJSxctIhRo56iX79+lb/hcIWxog/1tIh0NMasK7K9I3D6fDtUJB+bjVuHP84n40ZjHA7a9uhLWEwjfpo7i8gmLWjR4QbWLP6aXRtW4eNjo2YdfwaOeA6A1p27sS9+E9OffQAEml7bsVjDwyo2m43bH3iCWa89h3E4aH9zX8JjGvP9nA+Jjm1Bq4438svCr9ixbiU+Nhs16wRwx2NjALiqcxx7tm1iylP3gQjN23akVQfv5FGYw8A/Vu3nr31bYhPhh12pHDx6ij+0i2Z3+gnWJWaxKekYbaPq8t4d1+Aw8K+1iRw/4/1Bk3ZjePeHXbz9f9fi4yN8+2sK+zNOcv+Njdl5OJuVezJYuz+T6xsH8e/h12M3hunL95DtGvD59z9cR8OgWtSsZuM/D3dhwuIEr42HsDsMz01bxJevD8HmI8z+bjM7D6Tx/J+6s+m3QyxevYulG/bQo30sv8wYgd3h4K8f/EDW8crVELL5+vLwqGcZ+/TjOBx2et52Ow0bx/LJB+/TrGUrOnWNY/6Xc1i7cgU+Nhv+AQE8+cLLzn1tNu5/dCQvPjkCg6Fp81b07j/IK3n4+vry7OgxPPHoCBwOB/1vH0BsbFNmTJ9Gq9at6RbXnc/nfMaK5cuw2XwJDAxg7N9eAWD/vr2Mf/01RHwwxsHQ4fcVu0vDyjxGj3meR0Y48xgwYCCxTZsybdpUWrduQ/fu3fnss09ZvmwZNl9fAgMC+Nsrr3ol1opyJU1PLcZc8hWBsp1ApB0wHfDn3GWLGOAY8KgxZkNpxyjPZYvKxM/38ri55d9ris9nUBVlZFWuP37lEb/0f94OoUKs+XiUt0OoEGG1q+bYlqKq+Xh/LpiKUqtmDY8nc+bHj8v1t8rv5qFV5gfu8U+4MWYj0Mn1dM6zQ7iTjTGHL7CbUkopVbVUsUGP5WFZ89jVWNAGg1JKqcuS0QGT1hCRjd48v1JKKaUunlcvzBlj2nnz/EoppVSF0csWFU9EwnEf83DkQvWVUkqpKkUbDxVHRNoC7wOBQLJrc7SIHAUecQ2oVEoppao0HfNQsf4FjDTGtDLG/M61tASeBGZZcH6llFLqsiAifUQkQUR2i8iY85TfKyJpIrLZtTxQqGyYiPzmWoYV2t5eRH51HXOKiJR6y6gVly1qG2PWFN1ojFktIrUtOL9SSinleR6+bCEiNmAq0BPnvEnrRGS+MWZ7kaqfG2MeK7JvEPBXoANggA2ufbNwzsX0ILAGWAj0ARZdKBYrGg+LRORbnM+2OPuElhhgKLDYgvMrpZRSnuf5MQ/XA7uNMXsBRGQOMAAo2ng4n97A98aYTNe+3wN9RGQZEGCMWe3a/jEwEG83HowxT4hIX5wJFgyYBKYaYxZ6+vxKKaWUFSyYnjqKc1/Cwdn70Ok89e4QkW7ALmCUMeZgCftGuZak82y/IEvutjDGLKKUVoxSSilVpZVzwKSIPAQ8VGjTTNcTpi/GN8BnxpgzIvJn4CPg5nIFdh4eHzApIoEi8oaI7BCRTBHJcL1+Q0Tqln4EpZRS6vJnjJlpjOlQaCnacEjGedn/rGjO3cV49hgZxpizjx7+AGhfyr7JrtclHvN8rLjbYi6QBfQwxgQZY4KBHsBRV5lSSilV9Tns5VtKtw5oJiKNRaQ6cDcwv3AFEalfaPV2YIfr9RKgl4jUE5F6QC9giTEmBcgWkc6uuyyGAvNKC8SKyxaNjDETCm9wPefiDREZbsH5lVJKKY8zHh4waYzJF5HHcDYEbMCHxph4EXkFWG+MmQ88ISK3A/lAJnCva99MEXkVZwME4JWzgyeBR3BOq1AT5xCDUocZWNF4OCAizwEfnZ1V0jXb5L24D95QSimlqiwrJoly3WiwsMi2sYVePw88X8K+HwIfnmf7euCqi4nDissWdwHBwHLXmIdMYBkQBNxpwfmVUkopjzN2R7mWqsSKWzWzgNGuxY3rsoXOMqmUUkpVIV59JDfwNy+fXymllKoQ2vNQgURka0lFQHhZjmE3FRePNy3ZkertECrE0eNnSq9UBfjVrObtEJRSl5Er6cFYVgyYDMc5LWZWke0CrLLg/EoppZTHVbXeg/KwovGwAKhjjNlctMA1p7ZSSimlqhArBkzef4Gyezx9fqWUUsoK2vOglFJKqYvi8PyDsSoNbTwopZRSFUAHTCqllFLqolxJly28Pc+DUkoppaoY7XlQSimlKsCV1POgjQellFKqAuiYB6WUUkpdFIf2PCillFLqYlxJly10wKRSSimlLor2PCillFIV4ErqedDGg1JKKVUBdMCkUkoppS7KldTzoGMelFJKKXVRtOdBKaWUqgBXUs+DNh6UUkqpCuDQMQ+Xlz2b17Lk46kYh4O2PW7lxgF/cCvfsnwxS2fPxD8oBIAOvQZw3c23AbB09gx+27QGYwxNrm5Pr2GPIiKW51BUmwh/7mobhY8IP+/LYPHO1GJ12kfXpX+bCMBw8Ohp/rnmgPWBnsf1DevxWPem2HyEb7el8Om6g8XqdG8eyr2dG2KAPWk5vLZoJwATB11N64gAfj10jOfnbbM4cncdG9TlkZua4CPCou1HmLMxqViduKYhDL2+AcYY9macYNx3uwAY378NrSL82ZaSzV8WbLc69GJuaR/LuBG9sfn48O/Fm5g8d2WxOgNvas3oP8ZhMGzbe4SHJnxdUOZfqzq/zHiEb3/Zyehpi60M3c2GNauYOfktHA4HvfoN5M4/3utW/sPCb/hw2mSCQ8MA6Df49/TuPxCA1COHeW/Cq6SlHkEQXn5zMuH1I61OAYBfVq7k7bcm4rA7GDBoEMOG3+dWvmD+PKZMmkRoWCgAd951NwMHDQagc4d2xDZtCkBERH3enjTZ2uALWblyJW9OnIDD4WDgoEHcd9/9buXz583j3UnvEuZ6P+66+24GDx5cUJ6Tk8MdgwfRo0cPxjz/gqWxXwrtebiMOBx2Fs2awpAXJhIQHMo/X3yE5u27EBrdyK1e6y7d6TP8CbdtB3fFc3BXPA9N/AcAH708kgM7ttCodVurwj8vEbinXTTvLt9D1qk8Xvhdc7YcOkZK9pmCOmF1qtO3VRgTf/yNk3l2/P0qx1vtIzDy5mY889VW0o6f4f172rFyTwYHMk8W1ImqW5MhHWN47PPN5JzJp27NagVlc9YfxK+ajduvru+N8Av4CDweF8voedtIy8ll6u/bsmpfBolZpwrqRAXW4A/toxn5ny3knLG75TF3UxJ+vjb6XRXhjfDd+PgIEx/ty+AXPuFQejZLpzzA4tUJJCSmF9RpEhnEk3fdSJ+nZ3Es5zQhgbXcjvHC0B6s2ubdxqndbmf6OxN47d2pBIeGM+rBoXS6sRsNGjdxq3fTLT0ZMWp0sf3feW0sdw29j+s6dubUyZOIj3eGhNntdiZOGM/fp71PWHg4w/44hJvi4mjSJNatXs9evXh2zPPF9vfz82P2nLlWhVsiu93OG+PHMf39GYSHhzNkyD3ExXUnNtY9j969epXYMJg2dSrt2rW3ItwKYex2b4dgmct+wOSh3TsJioiiXngkNt9qtOnSg13rV5VpXwHy83Kx5+djz8vDkW+nTmA9zwZcBo2DapGac4b0E7nYHYZ1iVlcGxnoVuemJsEs253OyTznh/n4mXxvhFpMy4gAko+eIuXYafIdhh8TUrkxNtitTr+r6/PfLYfIccV89FReQdnGg0c5lev9XFqE+3Po2GlSss+Q7zAs+y2NG5u453Frmwjm/ZpCzhnne1A4j01JxziVVzl+0bRvEcW+lCwOHD5KXr6Dr5bH07dLC7c6Q/u2458L1nMs5zQA6cfONfaubVqf0Lq1+WnjXkvjLmrXjnjqR8UQERlNtWrV6HZLL1b/vLxM+ybu24vDbue6jp0BqFmrFjVq1PBkuCWK37aN6OgYoqKdefTq3ZsVy5Z5JZby2LZtGzExMUS78ujduw/LLiKP7du3k5GZQZcuXTwXpLpklePrqAcdz0onIDi0YN0/OJRDu3cUq7dz7f9I3LGVoPrR9Bz6CIHBYUQ3b0Oj1m2ZNOJOMNCh9wBCohpaGf551a1ZjcyT5/4QHT2VR+Mg92+C4f7OX3zP3dwUHxG+iT9M/OHjlsZ5PqF1qpN2/FwPSVrOGVpHBLjVialbE4D37mqLTYR//bKftQeyrAyzVCG1q5NaJI+W4f5udaJdeUy64xpsAh+vTWRd4lFL4yyL+sH+JKcdK1g/lJ5N+xZRbnVio4IAWPT2cGw+woRPlrN0wx5E4NWHevLwxK+Ju879G77VMtJSCQ0LL1gPCQ0jYUfxS1urlv1I/OZNRMY04MHHnyI0PILkg4nUruPP6y8+y5GUZNq278Swhx/DZrNZmQIAaWmphEec65EKCwsnftuvxer9+ONSNm3cSIOGDRn19DMF++Tm5jJ0yD342mwMHT6c7j1utiz2wlJT3fMIDw9j26/F81i6dCkbXXk888yzRERE4HA4eOftt3l93DjWrF5tZdjlciXN83DZ9zyURbN2XXhsymwemvgBja9uz/xpEwDIPJxMenIiI6d+zshpn7M/fhOJO7d6Odqy8REIq+PH2z/t5h+rD/CnDjHUrGb9L8JLYfMRouvW5MkvtvDKwh0807M5dfyqRuyF2XyEqMCaPP31r7y+JIFRPZpRu3rVywPA1+ZDk8gg+j/3EQ+88RWTnuxHQG0/7u/Xke/1E+SZAAAgAElEQVTX7uZQuvcbpmVx/Y038eEX3/D3j+ZwXcdOvDvuZQDs9nzit27i/kdH8u7MjzmcksTSRd94N9gL6NotjnkLFvLp3C+4vlNnXh77UkHZvG8X8vHsT3l13HjefetNkg4WH1NUWXSLi+PbhYuY+8WXdO7cmbEv/QWAuXM/p2vXroSHh5dyhMrF2B3lWqoSjzceRCRQRN4QkZ0ikikiGSKyw7Wt7gX2e0hE1ovI+p++mn3J5/evF0J2RlrB+vGMNPzrhbjVqeUfiG+16gBcd/OtHN73GwAJ634mqlkrqteoSfUaNYm99nqSdnl/cNvRU3kE1Tp3/bxuzWpkFeoSB8g6lceWQ9nYDWScyOXI8TOE1aludajFpOXkEurvV7AeWsePtJwzReqcYeWedOwOw+Hs0xzMOkVU3VpFD+VV6SdyCSuSR8aJXLc6aTln+GV/hjOP42dIOnqqoDeiMknJOE5U6LnLXpEhAaRkuDcGDqVns3h1Avl2B4lHjrI7KZPYqGA6tormwds7svmjJ3jlgZ7cfcu1jB1+i9UpABAcGkZa6pGC9fS0VIJDwtzqBATWpVp15/+DXv0GsjvB2QsZEhZOk6YtiIiMxubrS+eu3dmzK8G64AsJDQ3jyOHDBeupqUcIDXPPo27dulR35TFg0CB27jzXmxrm6n2Jio6mXYcOJCTstCDq4sLC3PM4csS9Zwjc8xg0aDA7djjz2LplK59/Podb+/bl3XffYcGCBUyePMm64C+RNh4q1lwgC+hujAkyxgQDPVzbShzVY4yZaYzpYIzp0GPwkEs+eWRsSzIPJ5OVmoI9P4/4X36iefsb3Oocz8ooeL1rwy+ERDUAICAkjAM7tuKw27Hn55O4Y2tBmTftzzxJWB0/gmtXx+YjdGxQjy2Hst3qbE4+RvOwOgDUqW4j3N+P9CJ/3Lwh4XA20fVqEhFQA18f4eYWYazam+FW5+fd6bSNcbYrA2v4ElOvJinHTp3vcF6TcOQ4UYE1ifD3w9dH6N4slFX7Mt3qrNqbwbVRzj/KATV8ia5bk5Ts094I94I2JiTTJDKIBuF1qebrw+C4NixevcutzsJVCdx4TSMAggJq0jQ6iP0pWfx54tdcM3QybYdNYewH3zNn6RZembXUC1lA85atOZR0kMOHksnLy2PF0u/o1LWbW53M9HODQNesXEFMw8YANGvZmpyc4xzLcl4e27pxPTGNGlsXfCGt27Th4MFEkpOdeXy3ZAk3xcW51UlPO/eFaMXy5TR2xZqdnU1urvP/+dGsLLZu3kzjJt65nNSmTRsSExNJTk4iLy+PJUsW071IHmmF8li+fBmNGzvzGDd+PIsWL2HhokWMGvUU/fr1Y+TIJy2N/1I47I5yLWUhIn1EJEFEdovImAvUu0NEjIh0cK0PEZHNhRaHiLR1lS1zHfNsWVhJxz3LijEPjYwxEwpvMMYcBiaIyH0l7FNhfGw2+tz7OJ+NH43D4aBt976ExjRi2ReziGzcguYdbmDd4q/ZtWEVPjYbNev40//h5wBo1akb++M3MeO5BxCB2Gs7Fmt4eIPDwGcbk3iym/M2wZX7MknJPs3tbSI4kHWSLYeyiT98nNbh/rzcuyXGGP6z5RAncr0/QM9uYPKPu3lz8NXOWxzjD7M/4yTDuzQi4chxVu3NYO2BLDo0DOJfQzvgMIb3V+wl+7RzkOSU37elQb2a1Kxu44sHOjPx+wTWeWE8hMPAeyv28MaAq/ARWLz9CAcyTzLs+gbsSs3hl/2ZrEs8SvsG9fjnPe1wGMPMVfsK8nh38NXE1KtFzWo+fHZvR97+8TfWe2k8hN1heG7aIr58fQg2H2H2d5vZeSCN5//UnU2/HWLx6l0s3bCHHu1j+WXGCOwOB3/94AeyjleuBp3N15eHRz3L2Kcfx+Gw0/O222nYOJZPPnifZi1b0alrHPO/nMPalSvwsdnwDwjgyRdedu5rs3H/oyN58ckRGAxNm7eid/9BXsnD19eXZ0eP4YlHR+BwOOh/+wBiY5syY/o0WrVuTbe47nw+5zNWLF+GzeZLYGAAY//2CgD79+1l/OuvIeKDMQ6GDr+v2F0aVuYxeszzPDLCmceAAQOJbdqUadOm0rp1G7p3785nn33K8mXLsPn6EhgQwN9eedUrsVYVImIDpgI9gSRgnYjMN8ZsL1LPHxgJrDm7zRgzG5jtKr8a+K8xZnOh3YYYY9aXORZjzCUnUqYTiHwH/AB8ZIw54toWDtwL9DTG/K60Y/x7Y5Jng7TI/3anl16pCtiVfKz0SlWAbxUZA1IWGxf86O0QKsSaj0d5O4QKEVb78hiLXs3H+3PaVJRaNWt4PJmdDwws19+qlh/894IxikgX4GVjTG/X+vMAxpjxRepNAr4HngWeKdooEJFxzt3Mi671ZeerdyFWXLa4CwgGlrvGPGQCy4Ag4PcWnF8ppZTyOAvGPEQBhUfAJrm2FRCRdkCMMebbCxznLuCzIttmuS5ZvCRlmAnR481jY0wWMNq1KKWUUpclYy9fJ7mIPAQ8VGjTTGPMzIvY3wd4B2fPfkl1OgEnjTGF72MeYoxJdl3u+A/wJ+DjC53Lq31rItLOGLPRmzEopZRSlYGroXChxkIyEFNoPdq17Sx/4CpgmavzIAKYLyK3F7okcTdFeh2MMcmuf4+LyKfA9ZTSePD2PA8jvHx+pZRSqkJYcLfFOqCZiDQWkeo4GwLzzxYaY44ZY0KMMY2MMY2A1UBBw8HVM/F7YM7ZfUTEV0RCXK+rAf2AUh8c5NWeB2PMg948v1JKKVVRjMOzY/uNMfki8hiwBLABHxpj4kXkFWC9MWb+hY9AN+CgMabwXPJ+wBJXw8GG8waHf5QWiyWNBxEJBPpwbmBHMrDEGFP55upVSimlLoGjnGMeysIYsxBYWGTb2BLqdi+yvgzoXGTbCeCinz5mxQyTQ4GNQHeglmvpAWxwlSmllFJV3pU0w6QVPQ8vAu2L9jKISD2cE1hccFCGUkoppSoXKxoPApyvL8fhKlNKKaWqvPLeqlmVWNF4eB3Y6Jpp8uzkFg1wTq+pc5EqpZS6LFgx5qGysGKSqI9EZD7Qm3MDJpcBz7smkFJKKaWqvKo2bqE8LLnbwhiTJSI/UehuC204KKWUUlWTxxsPrkd+vg8E4pyHW4BoETkKPKIzTCqllLocODw8z0NlYkXPw7+APxtj1hTeKCKdgVnAtRbEoJRSSnmUDpisWLWLNhwAjDGrRaS2BedXSimlPK6MU0xfFqxoPCwSkW9xzudw9m6LGGAosNiC8yullFIepz0PFcgY84SI9AUG4D499VTXNJtKKaWUqkKsuttiEbDIinMppZRS3qA9DxXI9VCs53H2PITjnG0yFZgHvKEPx1JKKXU50DEPFWsu8CPQwxhzGEBEIoB7XWW9LIihUmgWXsfbIVSI1OzT3g6hQvjX8OoT6ZVSlxlPP5K7MrHit2cjY8yEwhtcjYg3RGS4BedXSimlPO5Kmp7a44/kBg6IyHMiEn52g4iEi8hozt19oZRSSqkqworGw11AMLBcRDJFJBPnsy2CgDstOL9SSinlccbuKNdSlVhxq2YWMNq1uHFdtpjl6RiUUkopT7uS7rawoufhQv7m5fMrpZRSFcJhN+VaqhIrbtXcWlIRzls3lVJKKVWFWHG3RTjQGyj6CG4BVllwfqWUUsrjjKNqjVsoDysaDwuAOsaYzUULRGSZBedXSimlPK6qXXooDysGTN5/gbJ7PH1+pZRSygpX0oBJnWJPKaWUqgBV7XbL8vD23RZKKaWUqmK050EppZSqADrmQSmllFIXRcc8KKWUUuqiOIw2HpRSSil1EexXUONBB0wqpZRSVYSI9BGRBBHZLSJjLlDvDhExItLBtd5IRE6JyGbX8n6huu1F5FfXMaeIiJQWh/Y8KKWUUhXA00MeRMQGTAV6AknAOhGZb4zZXqSePzASWFPkEHuMMW3Pc+jpwIOu+guBPsCiC8WiPQ9KKaVUBbAbU66lDK4Hdhtj9hpjcoE5wIDz1HsVmACcLu2AIlIfCDDGrDbGGOBjYGBp+10RPQ97Nq9lycdTMQ4HbXvcyo0D/uBWvmX5YpbOnol/UAgAHXoN4LqbbwNg6ewZ/LZpDcYYmlzdnl7DHqUMPToecXDbelZ/PhPjcNCiay+u7ft7t/Jdq75n7ZcfUqtuMACte/Sn5U29AVj7nw85+Ot6ANredjexHbtZG3wJrosO5IHOjfAR4fuEVL7aeqhYnRsbB3F3u2gMsD/jJO8s2219oGVwdf0A/tSxAT4Cy3ansyD+cLE61zeox+BrIjFAYtZJpq/cZ32g53FL+1jGjeiNzceHfy/exOS5K4vVGXhTa0b/MQ6DYdveIzw04euCMv9a1fllxiN8+8tORk9bbGXobjasWcXMyW/hcDjo1W8gd/7xXrfyHxZ+w4fTJhMcGgZAv8G/p3d/5+/J1COHeW/Cq6SlHkEQXn5zMuH1I61OAYBfVq7k7bcm4rA7GDBoEMOG3+dWvmD+PKZMmkRoWCgAd951NwMHDQagc4d2xDZtCkBERH3enjTZ2uALWblyJW9OnIDD4WDgoEHcd5/7hMPz583j3UnvEuZ6P+66+24GDx5cUJ6Tk8MdgwfRo0cPxjz/gqWxX4ry9jyIyEPAQ4U2zTTGzCy0HgUcLLSeBHQqcox2QIwx5lsRebbIKRqLyCYgG/iLMeZ/rmMmFTlmVGmxXvaNB4fDzqJZUxjywkQCgkP554uP0Lx9F0KjG7nVa92lO32GP+G27eCueA7uiuehif8A4KOXR3JgxxYatT5fr49nORx2Vn06nb6jXqN2vRDmjRtFg2s7Uy+ygVu9Jh26ccM9I9y2JW5dS3riHga99B72/Dy+fWsMMVd1oHrNWlamUIyPwJ9vaMxfF+0g40Qubw64irWJWSQdPVVQp35ADe64Noox38RzItdOYI3K+ZEVgWHXN2DC0l1knszjlb6t2Jh0lEPHzjX8w/396H9VfV75bicnc+0E+FWOXHx8hImP9mXwC59wKD2bpVMeYPHqBBIS0wvqNIkM4sm7bqTP07M4lnOakED3z84LQ3uwatsBq0N3Y7fbmf7OBF57dyrBoeGMenAonW7sRoPGTdzq3XRLT0aMGl1s/3deG8tdQ+/juo6dOXXyJOLjnY5Zu93OxAnj+fu09wkLD2fYH4dwU1wcTZrEutXr2asXz455vtj+fn5+zJ4z16pwS2S323lj/Dimvz+D8PBwhgy5h7i47sTGuufRu1evEhsG06ZOpV279laEWym4GgozS61YAhHxAd4B7j1PcQrQwBiTISLtgf+KSJtLPddlf9ni0O6dBEVEUS88EptvNdp06cGu9WV7mKcA+Xm52PPzsefl4ci3UyewnmcDLkHavl0EhEUSEFofm281mnTsxoEtq8u079GUg9RvdhU+NhvV/GoQFN2IpPgNHo64dM1C65CSfZojx8+Q7zD8vDeDTg3df769WoaxcMdhTuTaATh2Ot8boZYqNrg2R46fIS0nF7vDsHp/Ju2j67rV6dE0lB92pXLSlUv2mcqRS/sWUexLyeLA4aPk5Tv4ank8fbu0cKsztG87/rlgPcdynI2h9GMnC8qubVqf0Lq1+WnjXkvjLmrXjnjqR8UQERlNtWrV6HZLL1b/vLxM+ybu24vDbue6jp0BqFmrFjVq1PBkuCWK37aN6OgYoqKdefTq3ZsVy5Z5JZby2LZtGzExMUS78ujduw/LLiKP7du3k5GZQZcuXTwXZAWz4LJFMhBTaD3ate0sf+AqYJmI7Ac6A/NFpIMx5owxJgPAGLMB2AM0d+0ffYFjnlfl+OrjQcez0gkIDi1Y9w8O5dDuHcXq7Vz7PxJ3bCWofjQ9hz5CYHAY0c3b0Kh1WyaNuBMMdOg9gJCohlaGX+Dk0Qxquy6rANSuG0LavoRi9fZtXEnKb9sIDI+i8+8fpE5QKEHRjdm44FOu7jmI/NwzpCRspW79BsX2tVpQreqkn8gtWM84kUuz0DpudSIDnL/Ax/dvg4/AnI1JbEo6ZmmcZVGvVnUyT57LJfNkLrEh7rlEBPgB8FKvlvgIfLX1EL+mZFsa5/nUD/YnOe3cz/RQejbtW7j3WsZGBQGw6O3h2HyECZ8sZ+mGPYjAqw/15OGJXxN3nfs3fKtlpKUSGhZesB4SGkbCjm3F6q1a9iPxmzcRGdOABx9/itDwCJIPJlK7jj+vv/gsR1KSadu+E8MefgybzWZlCgCkpaUSHhFRsB4WFk78tl+L1fvxx6Vs2riRBg0bMurpZwr2yc3NZeiQe/C12Rg6fDjde9xsWeyFpaa65xEeHsa2X4vnsXTpUja68njmmWeJiIjA4XDwzttv8/q4caxZXbYvSZWBBXNErQOaiUhjnH/g7wYKHjBpjDkGFPyhcD25+hljzHoRCQUyjTF2EWkCNAP2GmMyRSRbRDrjHDA5FHivtEAsazyISDjnrqMkG2OOWHXu0jRr14U2N9yMb7XqbPjhG+ZPm8CfXnqbzMPJpCcnMnLq5wDMHvcsiTu30qDlNV6O+PwaXNOJ2I7dsVWrxo7li1g+6x1ue3o80W3akbZ/F/MnPEMN/0DCmrTCx0tdshfLx0eoH1CDvyzYTnDt6ozr15qRX20t6ImoSnxECPf3Y9z3CQTVqsaLvVrywoJ4TuZV/lx8bT40iQyi/3MfERkSwLdvDePGh9/n9zdfw/drd3Mo/bi3QyyT62+8ibjf9aZa9eosmvcf3h33MuMmv4/dnk/81k1M+XA2oWERTHj5eZYu+oZe/UodN+YVXbvF0atPX6pXr85XX37Jy2NfYvpM5+XVed8uJCwsnOSkJB7584M0bdqM6JiYUo7oHd3i4ujT15nHl19+wdiX/sLMf3zA3Lmf07VrV8LDw0s/SCXi6XkejDH5IvIYsASwAR8aY+JF5BVgvTFm/gV27wa8IiJ5gAN42BiT6Sp7BPgXUBPnXRYXvNMCLGg8iEhb4H0gkHNdIdEichR4xBizsYT9CgaODH/xDXoMHnJJ5/evF0J2RlrB+vGMNPzrhbjVqeUfWPD6uptv5cdPnf8JE9b9TFSzVlSvUROA2GuvJ2nXdq80HmrVDeZE5rlr0CeOplOrXrBbnRp1Agpet7ipF2v/82HB+nW33c11t90NwE8fTCQwvNTxMB6XeTKXkNrVC9aDa7t/ewdnb8Su1BzsxpCac4ZDx05TP6AGu9NPWB3uBWWdzCWo1rlcgmpVJ6tILpknc9mTfgK7MaSdyOVw9mnCA/zYl3Gy6OEslZJxnKjQc/8HIkMCSMlwbwwcSs9mw85k8u0OEo8cZXdSJrFRwXRsFU2Xqxpwf/8O1K5Rneq+Nk6cyuOVWUutToPg0DDSUs99J0lPSyU4JMytTkDguUtJvfoNZNb0KQCEhIXTpGkLIiKdvbedu3YnYXvxXgsrhIaGceTwucG2qalHCA1zz6Nu3XN5DBg0iPemTCpYD3P1vkRFR9OuQwcSEnZ6pfEQFuaex5Ej7j1D4J7HoEGDmTzJmcfWLVvZtGkjc+fO5dSpk+Tl5VGzVi1GjnzSmuAvkRWzUxtjFuK8nbLwtrEl1O1e6PV/gP+UUG89zssdZWbF189/ASONMa2MMb9zLS2BJ4FZJe1kjJlpjOlgjOlwqQ0HgMjYlmQeTiYrNQV7fh7xv/xE8/Y3uNU5npVR8HrXhl8IiXJ26QeEhHFgx1Ycdjv2/HwSd2wtKLNaaKPmZKcmczz9MPb8PPauW0HDa90G2XLyaGbB68Qta6hb3/kLw+GwczrH2T2ekbSPzKT9RLVuZ13wJfgtLYf6ATUIq+OHr4/QtUkwaw9kudVZcyCTq+o7G0X+fr5EBtbgyPEz3gj3gvZmnCDCvwahtatj8xE6NwpiY9JRtzobDh6lVbg/AHX8fIkIqEFaJchlY0IyTSKDaBBel2q+PgyOa8Pi1bvc6ixclcCN1zQCICigJk2jg9ifksWfJ37NNUMn03bYFMZ+8D1zlm7xSsMBoHnL1hxKOsjhQ8nk5eWxYul3dOrqfldRZvq5BvialSuIadgYgGYtW5OTc5xjWc7P39aN64lp1Ni64Atp3aYNBw8mkpzszOO7JUu4KS7OrU562rkvRCuWL6exK9bs7Gxyc52N1qNZWWzdvJnGTbxzOalNmzYkJiaSnJxEXl4eS5YspnuRPNIK5bF8+TIaN3bmMW78eBYtXsLCRYsYNeop+vXrV+kbDlcaKy5b1DbGFJ2oAmPMahGp7emT+9hs9Ln3cT4bPxqHw0Hb7n0JjWnEsi9mEdm4Bc073MC6xV+za8MqfGw2atbxp//DzwHQqlM39sdvYsZzDyACsdd2LNbwsIqPzcYNfxjBokkvYRwOmt/Yk3qRDdkw79+ENGxGw7adif9xPge2rMHHZsOvVh3i7h0FgMNuZ8Gbzpyq1ahF9/ufxscL13KLchj4x6r9/LVvS2wi/LArlYNHT/GHdtHsTj/BusQsNiUdo21UXd674xocBv61NpHjlWSgYWEOAx+vS+TZW5rjI7BiTwbJx04z+JpI9mWeYFPSMX5NyebqyADe6NcGhzHM2XiQnEpw+cXuMDw3bRFfvj4Em48w+7vN7DyQxvN/6s6m3w6xePUulm7YQ4/2sfwyYwR2h4O/fvADWcdPlX5wC9l8fXl41LOMffpxHA47PW+7nYaNY/nkg/dp1rIVnbrGMf/LOaxduQIfmw3/gACefOFl5742G/c/OpIXnxyBwdC0eSt69x/klTx8fX15dvQYnnh0BA6Hg/63DyA2tikzpk+jVevWdIvrzudzPmPF8mXYbL4EBgYw9m+vALB/317Gv/4aIj4Y42Do8PuK3aVhZR6jxzzPIyOceQwYMJDYpk2ZNm0qrVu3oXv37nz22acsX7YMm68vgQEB/O2VV70Sa0W5gp6LhRgPX6MRkSlALM6JJ87enxqDc1DGPmPMY6Ud498bky6Lt+Tw8VLn66gSVv6WXnqlKsC/kt72eSm+/WRh6ZWqgDUfj/J2CBUirPbl8dmq5uOdOW08oVbNGh5P5u+BLcr1t+qxYwlV5gfu8U+4MeYJEemLcxasggGTwFTXtRullFKqyruSeh4saR4bY8o0elMppZSqqvSpmhZx3VGhlFJKqSrE2xfmqsz1HaWUUupC9LKFRYwxM7x5fqWUUqqiXEmXLSxpPIhIb5yP+Cw8YHKeMcZ7j99TSimlKpD2PFQgEZmE8+EbH3PusZ/RwBMi0tcYM9LTMSillFKq4ljR83CrMaZ50Y0i8jmwC9DGg1JKqSrvSrpsYcXdFqdFpON5tncELo9Zk5RSSl3xHOVcqhIreh7uBaaLiD/nLlvEAMdcZUoppVSVdyX1PFgxw+RGoJOIROD+SO7DF9hNKaWUqlJ0wGQFE5FAII5CjQcRWWKMOXqB3ZRSSilVCXl8zIOIDAU2At2BWq6lB7DBVaaUUkpVeXZjyrVUJVb0PLwItC/ayyAi9YA1OG/hVEoppao0vWxRsQQ434/UgU5PrZRS6jJR1XoPysOKxsPrwEYR+Q446NrWAOgJvGrB+ZVSSimPu5J6Hjw+5sEY8xHQAVgOnHEty4AOxph/efr8SimllKpYltxtYYzJEpGfcL9VM8uKcyullFJW0MsWFUhE2gLvA4E4J4kSIFpEjgKPuOaBUEoppaq0K+myhRgPt5REZDPwZ2PMmiLbOwMzjDHXejSAMhKRh4wxM70dR0W4XHLRPCoXzaPyuVxyuVzyuJJY8WyL2kUbDgDGmNVAbQvOX1YPeTuACnS55KJ5VC6aR+VzueRyueRxxbBizMMiEfkW53wOZ++2iAGGAostOL9SSimlKpAVz7Z4QkT6AgMoNGASmGqMWejp8yullFKqYll1t8UiYJEV5yqHy+l62+WSi+ZRuWgelc/lksvlkscVw4oBk4HA8zh7HsJxzjaZCswD3tCHYymllFJVixUDJucCWUAPY0yQMSYY54OxjrrKlFJKKVWFWNF4aGSMmWCMOXx2gzHmsDHmDaBhRZ5IRPqISIKI7BaRMecp9xORz13la0SkUaGy513bE0Skd2nHFJHHXNuMiIRUZB4W5PShiKSKyDZPxX0hl5qTiASLyE8ikiMif7c67tKUIa9uIrJRRPJF5P+8EWNpSvtsiNMUV45bRaSd1TGW5Hyxi0iQiHwvIr+5/q1Xwr7DXHV+E5Fh1kXtFkOZ4y/r+yAi7UXkV1e9KSLikecJVVTsZXkfyvqeKg8zxnh0Ab4DngPCC20LB0YDP1TgeWzAHqAJUB3YArQuUucR4H3X67uBz12vW7vq+wGNXcexXeiYwHVAI2A/EOKhn12F5+Qq6wa0A7Z5+v2v4JxqA12Bh4G/Wx17BeTVCLgG551H/+ftmEvI44KfDeBWnOOXBOgMrPF2zBeKHZgIjHG9HgNMOM9+QcBe17/1XK/rVeb4y/o+AGtd5eKq37eyxl7W96Es76kunl+s6Hm4CwgGlotIpohk4ny2RRBwZwWe53pgtzFmrzEmF5iDc5xFYQOAj1yvvwRucbXEBwBzjDFnjDH7gN2u45V4TGPMJmPM/gqM36qcMMasADI9HHtJLjknY8wJY8zPwGnrwi2zUvMyxuw3xmzF+UTZSqkMn40BwMfGaTVQV0TqWxPdhZUQe+HP0kfAwPPs2hv43hiTaZzT5n8P9PFYoCW4yPhLfR9c6wHGmNXGGIOz0Xq+/CtL7GV9H8rynioPs+LBWFnGmNHGmJbGOeYhyBjTyhgzmop906M4N48EOKfCjiqpjjEmHziGs2FT0r5lOaYneSInbytPTpVZZf15V7Sqlme4MSbF9fowzl7PoipzTiXFX9b/R0ml1PGki429rO9DWd5T5WFW9DxcyN+8fH6l1BXC9e27yj59oCrH76nYq/LPpKrzeOPBNSDmfMuvVGyLMRnnzJVnRbu2nbeOiPjifFhXxgX2LWXE820AAAZOSURBVMsxPckTOXlbeXKqzCrrz7uiVbU8j5ztznf9m3qeOpU5p5LiL+v/o+hS6njSxcZe1vehLO+p8jAreh7CcU5F3f88S0X+QVgHNBORxiJSHedAu/lF6swHzo7g/T/gR1fLdT5wt2uUf2OgGc6BRmU5pid5IidvK09OlZm3PytWmQ8MdY2Y7wwcK9SFXBkV/iwNwzm/TFFLgF4iUs81cr+Xa1tlUFL8pb4PrvVsEensGgc1lPPn7ykXG3tZ34eyvKfK0zw9IhP4J9C1hLJPK/hctwK7cI56f9G17RXgdtfrGsAXOAcPrgWaFNr3Rdd+CRQakXy+Y7q2P4Hzmlw+cAj4wEM/P0/k9P/t3WuIlFUcx/Hvzy2i8katZVIo5JtUUEGTDDJIMDKIoDS6m70wX4QUFESU2YUoSiOTQiuR0izDoBeGVEShVpCmlhSFVqabpqmZ6Jr578U5gw/jrjuP7tX9fWBg5pn/nMszszz/PXPmnCVAA/Bv7sPUtv4ctGKffiFNzPont31Ie7b9FPs1Orf5AClx/r6j29xEH477bJB+3TItPy/gldzHjcCojm5zC20/H/gE+An4GDgvx44q/s0C9+TP28/AlC7Q/mbfB+Dbwv1RwHc5bi55YcBO3PYm3wdgQSWuuXJ9a99bm68waWZmZqeXjp4waWZmZl2MkwczMzMrxcmDmZmZleLkwczMzEpx8mBmZmalOHkwMzOzUpw8WLch6RalLb4P5O2Dv5I0va22KW4rkq6WdFRpa/L9StuAT+nodrVE0kxJb3V0O8zs1Dl5sG5B0oPAS8DzQH/SyqfTgCtJW2h3NdsjoifQm7S9/XxJQ8oUkJf+7jK6WnvNTmdOHuy0J6kPaaXH6RGxLCL2R7IuIm6LiMYcN1HSOkl/S9oqaWahjEGSQtKU/NweSdMkjc57teyVNLcQf7ekVZJm5+c2Sxqbj2/NIx93FeKbrftEcj8+APYAQ2rsw1RJvwGf5uPvSfpD0j5Jn0saWnjNQknzJK3IIx2rJPWXNCefgx8kjSzED5D0vqQ/JW2RdH8+fi3wCDA5l7O+8t5Iel1Sg6Rtkp6SVNfEOdwN1HROzKztOXmw7uAK4CxaXgP/AGn9/77AROA+SdXbxo8h7RMyGZhDWgJ8PDAUmCRpXFXsBtJyuouBd0hLVA8GbgfmSupZou7jSOoh6cb8uo01ljMOuAyYkB+vyH26AFgLvF0VPwl4FKgHGoE1Oa4eWAa8WGkL8CGwnrSV8jXADEkTIuIj4BlgaUT0jIjhueyFpCXeBwMjSfsZ3Ft1DjeTRoqebul8mFn7cPJg3UE9sCsijlQOSFqdRwQOSroKICI+i4iNEXE0IjaQ1usfV1XWkxFxKCJWki7USyJiZ0RsA74gXQArtkTEmxHxH7CUtGPgrIhozK8/TLpo1lp30QBJe4FdwOPAHRHxY43lzIyIAxFxMNf9Rh6NaST9dz88j9ZULI+IbyLiELAcOBQRiwr9qvR5NNAvImZFxOGI2AzMJ20SdhxJF5L2A5mR27MTmF0Vvz0iXo6II5X2mlnH83eI1h3sBuolnVFJICJiLICk38lJtKQxwLPAMNI8iLNIG3QV7SjcP9jE454niCUimoyvse6i7RFxcfXBGsvZWoivI/1HfzPQDzian6oH9jXTj+b6PJBjSU1FHSmpaspA4EygoTBntUexfVX3zayT8MiDdQdrSMPtN7QQt5i03e8lEdEHeJW0A2B7aK26aymnuBveraTzMh7oAwzKx0+m7q2k0Za+hVuviLiuiXor8Y1AfSG+d0QMLcR45z6zTsjJg532ImIv8AQwT9JNknrluQIjgHMLob2AvyLikKTLSRfW9tJadZctpxfpAr4bOIc0L+FkfQ3sl/SwpLMl1UkaJml0fn4HMCjPjSAiGoCVwAuSeuf35NKqeSNm1gk5ebBuISKeAx4AHiJdxHYAr5F+5rg6h00HZknaDzwGvNuOTWytusuWswj4FdgGbAK+PMl6yXMgrgdGAFtI8zEWkEY04NjXJ7slrc337yR9vbKJ9IuRZcBFJ9sGM2sfivCooJmZmdXOIw9mZmZWipMHMzMzK8XJg5mZmZXi5MHMzMxKcfJgZmZmpTh5MDMzs1KcPJiZmVkpTh7MzMysFCcPZmZmVsr/o28kzSJVOtgAAAAASUVORK5CYII=\n",
            "text/plain": [
              "<Figure size 648x432 with 2 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "zVoAy7LjnTy-",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 68
        },
        "outputId": "c9ce8c34-e445-401a-af99-4175e01d01c3"
      },
      "source": [
        "# Print best parameters and best score\n",
        "print(\"Best parameters: {}\".format(grid_search.best_params_))\n",
        "print(\"Best cross-validation score: {:.2f}\".format(grid_search.best_score_))\n",
        "\n",
        "### Best parameters: {'C': 100, 'gamma': 0.0001}\n",
        "### Best cross-validation score: 1.00\n",
        "\n",
        "# Plug best parameters into the model, train it, and test on held out data\n",
        "best_svc = SVC(**grid_search.best_params_).fit(X_train, y_train)\n",
        "best_svc.score(X_test, y_test)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Best parameters: {'C': 1, 'gamma': 1}\n",
            "Best cross-validation score: 0.65\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0.655"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 55
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "H2FtRgBNnjKA",
        "colab_type": "text"
      },
      "source": [
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "AjS1R5SLn-cn",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 119
        },
        "outputId": "28affd8a-69b0-4fc0-b0e8-58d9e3280e6d"
      },
      "source": [
        "from sklearn.metrics import confusion_matrix\n",
        "\n",
        "y_pred = best_svc.predict(X_test)\n",
        "\n",
        "print(confusion_matrix(y_test, y_pred))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[[  0   0   0   1   0   0]\n",
            " [  0   0   6   7   0   0]\n",
            " [  0   0 121  43   0   0]\n",
            " [  0   0  43 122   4   0]\n",
            " [  0   0   5  24  19   0]\n",
            " [  0   0   0   4   1   0]]\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Mmm8WrH6nnzW",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 292
        },
        "outputId": "c16dbffe-2981-4eaa-a507-142c37edc228"
      },
      "source": [
        "from sklearn.metrics import classification_report\n",
        "\n",
        "print(classification_report(y_test, y_pred))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "              precision    recall  f1-score   support\n",
            "\n",
            "           3       0.00      0.00      0.00         1\n",
            "           4       0.00      0.00      0.00        13\n",
            "           5       0.69      0.74      0.71       164\n",
            "           6       0.61      0.72      0.66       169\n",
            "           7       0.79      0.40      0.53        48\n",
            "           8       0.00      0.00      0.00         5\n",
            "\n",
            "    accuracy                           0.66       400\n",
            "   macro avg       0.35      0.31      0.32       400\n",
            "weighted avg       0.63      0.66      0.63       400\n",
            "\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.6/dist-packages/sklearn/metrics/_classification.py:1272: UndefinedMetricWarning: Precision and F-score are ill-defined and being set to 0.0 in labels with no predicted samples. Use `zero_division` parameter to control this behavior.\n",
            "  _warn_prf(average, modifier, msg_start, len(result))\n"
          ],
          "name": "stderr"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "cDXK5Ti9rQWr",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 272
        },
        "outputId": "3d45ea0c-0fbf-43d6-cd0c-e516bc574dee"
      },
      "source": [
        "from sklearn.model_selection import GridSearchCV\n",
        "from sklearn.svm import SVC\n",
        "\n",
        "# Create parameter grid\n",
        "param_grid = {'C': [100, 500, 1000, 5000, 10000],\n",
        "                  'gamma': [0.001/3, 0.001/2, 0.001, 0.001*2, 0.001*3]}\n",
        "\n",
        "# Create grid search \n",
        "grid_search = GridSearchCV(SVC(),\n",
        "                           param_grid, \n",
        "                           cv=5)\n",
        "# Fit grid search\n",
        "%time grid_search.fit(X_train, y_train)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "CPU times: user 30.6 s, sys: 2.94 ms, total: 30.6 s\n",
            "Wall time: 30.6 s\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "GridSearchCV(cv=5, error_score=nan,\n",
              "             estimator=SVC(C=1.0, break_ties=False, cache_size=200,\n",
              "                           class_weight=None, coef0=0.0,\n",
              "                           decision_function_shape='ovr', degree=3,\n",
              "                           gamma='scale', kernel='rbf', max_iter=-1,\n",
              "                           probability=False, random_state=None, shrinking=True,\n",
              "                           tol=0.001, verbose=False),\n",
              "             iid='deprecated', n_jobs=None,\n",
              "             param_grid={'C': [100, 500, 1000, 5000, 10000],\n",
              "                         'gamma': [0.0003333333333333333, 0.0005, 0.001, 0.002,\n",
              "                                   0.003]},\n",
              "             pre_dispatch='2*n_jobs', refit=True, return_train_score=False,\n",
              "             scoring=None, verbose=0)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 58
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "jHWLvg-Ir5s8",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 527
        },
        "outputId": "8b4ec278-5bf5-4d61-9543-b9455fe7af6b"
      },
      "source": [
        "import matplotlib.pyplot as plt\n",
        "import seaborn as sns\n",
        "\n",
        "# Pull results from grid search\n",
        "results = pd.DataFrame(grid_search.cv_results_)\n",
        "hm_data = results.pivot(index='param_C', \n",
        "                   columns='param_gamma',\n",
        "                   values='mean_test_score')\n",
        "\n",
        "# Plot heatmap\n",
        "fix, ax = plt.subplots(figsize=(9, 6))\n",
        "\n",
        "ax = sns.heatmap(hm_data,\n",
        "                 annot=True,\n",
        "                 cmap='RdBu',\n",
        "                 linecolor='white')\n",
        "\n",
        "ax.set_title('Hyperparameter Gridsearch', fontsize=16)\n",
        "ax.set_ylabel('C Parameter', fontsize=12)\n",
        "ax.set_xlabel('Gamma Parameter', fontsize=12)\n",
        "\n",
        "plt.show()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAg8AAAH+CAYAAAAbA+2DAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzdd3gU5fbA8e/ZTULvJKGEXkRFRASkiBSpVixX/dkr6rV7bdjrvWK5iFesWPB6FRUbCgiIhg4GUOlNiiSEJKRQQks25/fHbMJuCGRDks0uOZ/nmYfMO+/MnJ0s2bNvmRFVxRhjjDEmUK6KDsAYY4wx4cWSB2OMMcaUiCUPxhhjjCkRSx6MMcYYUyKWPBhjjDGmRCx5MMYYY0yJWPJgABCR60VERaRtEdsivNueroDQTDFE5F4RuTjI5xQRuVJEZojIDhHJEZE0EflJRP4uItUCPM5HIrI5gHr578+WpQy9QohIvIjMreg4jCkrljwYE/7uBYKWPIhIBDAR+BhIBG4FBgC3AZuAfwN3B3i454CLyiFMY0w5iqjoAIwpLRGpoqoHgnAeASJV9WB5n6uiFXNNH8f5wL9EVb8ptO0rEfkX0CGQ46vqn2UQboWoTO8HYwqzlgdTYiJyurcJ+cIitn0kIoki4vaubxaRT0TkFhHZICL7RWSpiPQvYt++IjJTRHaLSLaITBORjoXqxIvIXBE5X0R+E5EDwN+921REXhCRx7wx7BOR2SLSudAxBovIFBFJFpG9IrJCRP6RH7NPvfzYbxSRNcBB4Fzvtme8r2OXt9n+ZxHpUWj/ft6YhovIOyKSISJZIvKaiLhFpJv3tWSLyEoRGVLSa+Jt8m8BXOU9l4rIRz7bTxWRSSKS6b0e80SkzxF+Zz1FZL6I7ANeKhyLt24V4D7g+yISBwBUdaOqTvHZJ7/L4SwR+VJEsoBFPufeXOgcrUVksvd3kyYiY4AqRcRypfc9sMf7e1guIreW5Pp565TF++FUEflGRNK913mtiIwsIuaB3vdN/nms1cWEJWt5MIW5xWmW9ivzXVHVJSKSgNNc/V1+uYjUBS4DXlJVj88u/YDTgceAA8DDwFQROVVV13r3Pdd7rMnA1d79HgbmiEgnVd3qc7z2wOs4Td4bgQyfbdcCfwF34nzgPAvMFJF2qppfrzUwE/gPsB/oCjwNRAOPFHrt/YHOwDNAKrDZW94UGI3TbF/DG/NsETldVZcXOsZrwNfA5cBZON/c3cBA4GUgyVv2tYi0UNUdJbgmFwFTgD+8rwEgzbt/F2AO8BtwC7AXp2vhJxHppapLfGKsA0wAXgEeBfZRtK5AbeCHI2w/mv8BnwGXcoS/PSISBcwAqgF34FzzWynULSMiZwKf4LwPHsT5ItQBqOtTJ9D3VKneDyLSHYgHNuAkVolAO6BToX3bAGOAfwE7gH8AX4pIB1XdUNT1MCZkqaottgBcD2gxy9OF6nuAFj5ldwO5QJxP2Wacb2jNfMpq4Xzg/9enbAMws1BMtXH+yL7mUxYP5AGdi3gN6q1fw6esJZADPHeE1y04H2SPAZmAq1Dse4FGxVw7t/cYa4ExPuX9vDF9UKj+Um/5mT5lnbxl1x3DNdkMfFJEXDOB1UBUoVhXA9/6lH3kPfeFAbxPLvfWHXKE65i/uIt4b40u4ngfAZt91m/x1u3hU+YCVnrLW3rLHgAyiok1oOtX2vcDMBvYClQ/Sizx3vdhO5+yGJz/Q4+W5f9lW2wJxmLdFqawi4BuhZYeRdSbAGTh/LHPdyswWVUTC9VdqD4tB6q6G+fbYE8AEWmH863sf+LM7Ijwtn7sBRbgfFv3tVlVfz9C/FNUNdvnXJuBhfnn8p6vsbcbYQtOYpMDPI/zrTWmiNi3Fz6Jt/n5FxFJx0mYcnBaRE4oIqaphdbXANmqOrdQGUAz7/FLek0Kx1cN6At8CeT57C/AT0Xsn8OxtSbku9x7jPxlVhF1iuzmKKQnsFVVF+YXqGoe8EWheglAPW83wnneVq8CJbl+pXk/iEh1oDfwP1XdW8xrW6+q631eVypO60XzYvYzJuRY8mAKW6Gqi30XYEnhSqq6H/gQuNH7h7kPcBLwdhHHTDlCWVPvz/l/oN/H/wMoBzgPaFBo3+SjxH/Uc4mIC5jkPe7zOLMEugEveOtWLe5c3u6AKcAe4Cac5KobTtdB4f3B+Qbr6yBO4lVADw26y9+/pNeksPo4rQxPFLH/nTgfvL7//9PUv6vpSPITw8IfeNM4lGwuPcK+R/u95WvMkX+HBVR1FvA3nGTrGyB/mmh+V0FA168M3g/1cP6OFk6Yi5JRRNmBIs5hTMizMQ+mNN4C7gcuxGmx2IzzIVJY7BHKkrw/p3v/HYnzrbiwwqPZj/Yc+eLO1QanT/saVf0kv4KInH+E4xV1rktwWhsuVtUcn2PUo1BSUAolvSaFZeF074zFmVJ5GO83+oLVAONaDOzC+bB9z+dYmd5tiMhuiv7bEsg5koGTiyg/7PeqqhOBiSJSE6eLaBTwo4jEEfj1K+37IRPnOjctoq4xxy1LHswxU9U/RWQ6zoC1zsCzhT6Q8vUQkWb5XRciUgtnlPpk7/a1OInHyar6YinDOkdEauR3XYhzU6EeQP5xq3v/9f3QjwSuKsE5quP0VRd8kIjIAJxv45uONfBCSnJNDuAMMCygqtkiMgc4FVh6hN9LianqAe/sh8dFZLiqflsWx/WxALhBRHrkd114WwcuO0pMe4AfRKQ1zoDEBgR+/Ur1flDVveLc/OlqEXlWVY800NSY44olD6a03sQZ0Z6D00RclBRgujh3qMyfbVEDZ7YEqqoicgfwnXe0/Rc4g9pigV7AX6r67wDj2ec918s4sy2ewfmmPNq7fTWwBXhBRDzeuO8L+NU6fsS5MdNHIvIhzliHJzjUulFqJbwmq4A+InIesB3Y4R3rcT/OYL5pIvI+zrf6hkAXnAGNhWcSBOpZnAGeE0XkY5yxEqk4Mza64yQsC4+8+1GNx5nh8LWIPOo97m04Ax0LiMizONfiF2AbEIczYPd3Vc2fbRLI9SuL98MDOGM8FojIqzhdGK1xBvXeVcJjGRMWbMyDKa3JOB/Y36lqUX3V4PxhfRX4J/A5Th/vMFVdl19BnfsCnIWTVIzD6f54CWiE8200UB97Y3oD54MoDThbvdM0vWMLhuN8yH6M06w/m0MtE8VS1Wk4H1S9cT44b8SZIlqm0+1KcE1G4nzT/gJnIOHT3v2X4vTfp+NMaZyO8838FJzXfKxx5eJMnbwe5x4T7+F8iH+Ec00e5RjvGun9/QwCfsdJTMfjtOY8X6jqIpyZNKNxpnaOwnmfnetzrGKvXxm9HxJwXvdWnOmeU3Ba4wIZB2FMWBLVQLs6jTmciAzC+VAaqKozi9i+GZirqlcX3lYOsSjwgqo+Xt7nMsaYysy6LcwxEZE2OE2zo3H61A9LHIwxxhyfrNvCHKsncO5fcACnyd4YY0wlYd0WxhhjjCkRa3kwxhhjTImExZiHFVeda80jpTDze3vmTmms3l3uT/s+7o1d+VFFhxDWkhp1regQwl7z+jWlvM8RddqNpfqsOvjbB+UeY1mxlgdjjDHGlEhYtDwYY4wxoU5c7ooOIWgseTDGGGPKQGVKHqzbwhhjjDElYi0PxhhjTBmoTC0PljwYY4wxZcCSB2OMMcaUiLgrT/JgYx6MMcYYUyLW8mCMMcaUAZd1WxhjjDGmJGzMgzHGGGNKpDIlDzbmwRhjjCkD4nKVagnoHCJDRWStiGwQkUeOUOcyEVklIitF5FOf8utEZL13uc6nPN57zN+9S0xxcVjLgzHGGBMGRMQNjAUGAYlAgohMUtVVPnXaASOB3qqamZ8IiEh94CmgK6DAEu++md5dr1LVxYHGYi0PxhhjTBkQl7tUSwC6AxtUdaOqHgQmABcWqnMLMDY/KVDVVG/5EGCGqmZ4t80Ahh7ra7XkwRhjjCkDpU0eRGSEiCz2WUYUOkVTYKvPeqK3zFd7oL2IzBORhSIyNMB9P/R2WTwhIsU+Gty6LYwxxpgyUNoBk6r6LvBuKcOIANoB/YA4YLaInFLMPlepapKI1AK+Aq4BPj7aDtbyYIwxxpQBcbtLtQQgCWjmsx7nLfOVCExS1RxV3QSsw0kmjrivqub/uxv4FKd75KgseTDGGGPCQwLQTkRaiUgUcAUwqVCdb3FaHRCRhjjdGBuBacBgEaknIvWAwcA0EYnw1kNEIoHzgBXFBWLdFsYYY0wZKO/7PKhqrojciZMIuIEPVHWliDwLLFbVSRxKElYBHuBBVU0HEJHncBIQgGdVNUNEauAkEZHeY/4EvFdcLJY8GGOMMWUgGDeJUtUpwJRCZU/6/KzA/d6l8L4fAB8UKssGTi9pHJY8GGOMMWWgMj3bwsY8GGOMMaZErOXBGGOMKQOV6dkWljwYY4wxZcCSB2OMMcaUiCUPxhhjjCmRypQ82IBJY4wxxpSItTwYY4wxZaAytTxY8lCMmp1Op/E1I8DlIjN+Oju+/9Jve92zBtLo/24kJzMdgIzp35MZPx2A2P+7gVqdu4G4yF7xG8kfvxP0+Ctai4F96DvqMcTtYuX4L1k82v/GZSdeeRFnPv8Q2dtSAPjj3U9Y+fFEAHo/+yCthvRFXC7++mUesx56Iejxh4KThvTlsjFP4nK7mTfuc6aNestve8/rLuXil0eSleRcw/g3xjPv/c8BuHjUI3Q8dwDicrF6xhy+uOeZoMdf0eYsXcG/3v8CT14elw48k1su8X8K8Tc/z+eV8V8RU78uAFed059LB50JwKsff8Wsxc6dem+/7ByGndktuMGHgIQF83nztVfI83gYdsFwrrj2Br/t0yZP4r03xtAgOgaACy+9jHMuuAiA98a+zq/z5wJw1Q0302/g4OAGH2QBPp/iuGDJw9GIiybX386mfz1ObsYOWj83mt1LF3IgaatftZ0LZ5M8/m2/smrtTqR6+5PY8MidALR+6iVqnHgK2auXBy38iiYuF/1efZJvLryBPUkpXBE/kY1TfiZj7Z9+9dZ/PYX4B57zK2vc/TSa9OjC/3peAMDfpn9K0zO7kzT316DFHwrE5eL/xj7LmEFXk5m4nZEJk1g2aQbJqzf41Vvy+Q9MuOspv7LWPbvQpndXnuvkfFg+OHci7fv2YN2shUGLv6J5PHk8/+5njHv6XmIb1OPyh/5F/+6daNusiV+9Yb278viI//Mrm7V4Oas2buXr0Y9zMCeX6594lT5dOlKzerVgvoQK5fF4+M+rLzJqzJs0jInlzhuvoWefvrRo1dqvXt+zB3PXAw/7lS2aN4cNa9fw9vhPOZiTwwN3jKBbz17UqFEzmC8hqCpTy4ONeTiKam3acyBlGzlp21FPLjsXzqbW6T0C21kVV2QUEhGBREYi7ghyd2aVb8AhJrZrJ3Zu3MKuzYnk5eSw7qvJtD737ID2VRR3lShcUZHOvxGR7E3dUc4Rh56W3TuTumELOzZtxZOTQ8KE7+l0YWDf3lQhomoVIqIiiagShTsygl0paeUccWhZvn4TzRvH0KxRNFGREQw7sys///pHQPtu2LqNrie1I8LtpnrVKrRvEcec31aWc8ShZe2qlTSJa0bjpnFERkbSb+Bg5s+OD2jfLZs2cUrn03BHRFCtWjVat2nH4gXzyzfgCiYud6mWcGLJw1FE1m9ATvqhD6zcjB1E1mtwWL3a3XrT9l9v0OyekUTWbwjAvg1ryF61jA5j/0uHsf9l97KlHNi29bB9j2c1G8eyO3F7wfqebSnUbBJ7WL22FwzmqvmTOOfjMdRs2giA7b/+TuKcRdyybi43r5vLlplzyFy3MWixh4p6TWPJ3LqtYD0rMZl6TQ+/hqddMozH/5jKiC/fpF5cYwA2LVzKul8WMCo5gZeSf2XVtNlsX/PnYfsez1IysmjUsF7BeqMG9UhNPzyJn75wKcPvfZZ7X3qH5B0ZAHRo1Yy5v61k34GDZO7aw68r1rJ9R2bQYg8FO9JSiY459H5rGBPLjrTDE9C58TMZcfXlPPvoQ6SmOP/nW7drR8LCBezfv4+dWZn8vnQxqakpQYvdlK+gdVuISCzQ1LuapKpHfReJyAhgBMCT3Tvyt7bNyznCY7N76SJ2zo9Hc3OpN2AoTW+7n83/fJSo2MZUadKMtXddB0DLkc+zZ9nJ7F1bub65FGfTj7+wbuIPeA7m0PGGyxn89ii+Pv866rRuTv0T2vD+iX0BuOi7D9jy0+lsW7CkgiMOPcu+/4mEzyaRe/AgfUZcyXXjX+W1s68kuk0LGp3YlpFxTmvZPTM+oe2Z3dgwN6GYI1Yu/bt24tw+3YiKjOTzabN5dMxHfPjc/fTufBLL12/mykdGUb9OLU49oTVul1R0uCGn55ln0X/QUKKiovjhm694+bmnePmNd+h6Rk/Wrl7FPSNupG7depzU8RTcYfbtuqTCrfWgNMq95UFEOovIQiAeeMm7zBKRhSLS5Uj7qeq7qtpVVbtWVOKQk5FOZIOGBesR9RsWDIzM59mzG83NBSDzl+lUa9UWgNpde7J3wxryDuwn78B+dv+xhOrtOgQv+BCwJzmFWnGNCtZrNollzzb/nHF/RhaegzkArBz/JTGdTwagzXmD2J7wBznZe8nJ3svmGXNo3P204AUfIjKTUqjn0z9fN64xmUn+1zA7I4vcgwcBmDtuAi1O7whA54uGsGnhbxzI3suB7L2smBpP655H/C93XIqtX9evtWB7eiYxDer61albuyZRkZEAXDrwTFZu3FKw7ba/ncM3o5/g/afvBVVaFNFydjxrGB1Dmk9rwY7UFBpGR/vVqV2nLlFRUQAMu2A469asLth21fU38c7HnzHq9TdRVZo2D80vgWXF5ZJSLeEkGN0WHwH3qOqJqjrQu3QA7gU+DML5j9m+jeuo0qgpkdGxiDuCOj3OYveSRX51IuoeahKtdfoZBV0TB9PTqHHiKeBygdtNjQ4dDxtoebxLWbKcuq1bUrtFHK7ISNpfci4bp/zsV6d67KE/RK3PGUDGOqdZfXfiNpr27oa43bgiIojr3e2wgZaVwZaEP4hp15IGLeNwR0bS7YrzWTZphl+d2o0OXcNTLxhE8mrnOmX8tY12fc/A5b2G7fuecdhAy+Ndx3Yt2ZKcSmLKDg7m5DJ17mL6dzvVr05axs6Cn39J+IPW3m4fjyePrF17AFi7OZG1m5Po3fmk4AUfAk448SSStm4leVsSOTk5xP80nZ59+vrVSd9xqBtjwZxZNG/ZCnAGW+7yjvPauGE9m/7cQNfuAY4ZC1PiklIt4SQY3RY1VHVR4UJVXSgiNYJw/mOXl8e2j96i5cPPIS4XmbNmcCDpL2IuuZp9m9aze+kiGgy5gFpdzkA9HjzZe0h8ezQAuxbNo+ZJp9L2xTcBZc8fS9j9W+WaKaAeD/EPPsvwb8Yhbjer/vsVGWs20OOxu0lZuoJNU3+m823X0PqcAeTletifuZMZt40EYMO302h2Vg+uXvg9qsqWn+aw6cdfKvgVBV+ex8Pndz7J3dM+xuV2M/+DL0hetZ7zn7mPLYuXs+z7nxhw9w10umAgebkesjOyGH/9AwAsnTiFEwb04onl00CVlT/OYvkPMyv4FQVXhNvNY7dcwS3PjCEvL4+Lzu5Nu+ZN+M+nkzi5bQsGdD+V/07+mV8S/iDC7aZOzer8867rAcj1eLj6sVcAqFm9KqPuu5GISjQVD8AdEcGd/3iIkffeSV6ehyHnXUjL1m346N23aH/iSfTq05dvv5jAgrmzcbvd1KpdmwcffxoAT24u9912MwDVa9Tg4aeewx1hE/yOF6Kq5XsCkdeBNsDHQP5X72bAtcAmVb2zuGOsuOrc8g3yODfz+8r1bbOsrd59oKJDCHtjV35U0SGEtaRGXSs6hLDXvH7Ncv9q3+6Ob0r1WbV+7EVh0/xQ7mmgqt4tIsOAC/EZMAmMVdUp5X1+Y4wxJhjCbdxCaQSlDUlVpwJTg3EuY4wxpiKE27iF0gjGbIs6IvKiiKwWkQwRSff+/KKI1C3+CMYYY0zoq0wDJoMx2+ILIBPor6r1VbUB0B/I8m4zxhhjTBgJRrdFS1Ud5VugqtuBF0XkhiPsY4wxxoQVl4RX60FpBKPlYYuIPOS9wyTg3G1SRB7m0OwLY4wxJqxZt0XZuhxogHNXyQwRycC522R94G9BOL8xxhhT7ipT8hCMqZqZwMPexY+32yKk7zJpjDHGBKIyTdWs6KdqPlPB5zfGGGNMCZV7y4OILDvSJqByPWXGGGPMcUsq+ut4EAVjtkUsMARnuqYvAeYH4fzGGGNMuZNKNNsiGMnDD0BNVf298AYRiQ/C+Y0xxphyV5nGPARjwORNR9l2ZXmf3xhjjDFly56PaowxxpSBcJtuWRqWPBhjjDFlwJIHY4wxxpRIZbo9tSUPxhhjTBmoTC0PlWhWqjHGGGPKgrU8GGOMMWWgMrU8WPJgjDHGlAG7z4MxxhhjSsTuMGmMMcaYEqlMz7aoRC/VGGOMMWXBWh6MMcaYMmBjHowxxhhTIjbbwhhjjDElUpkGTNqYB2OMMcaUSFi0PLzx6cqKDiGsXdiiTkWHENauTvqtokMIe19uzKzoEMLaiIufq+gQwt6u+FHlfg4b82CMMcaYEqlMYx6s28IYY4wpA26XlGoJhIgMFZG1IrJBRB45Qp3LRGSViKwUkU99yq8TkfXe5Tqf8tNFZLn3mK9LAIM3rOXBGGOMKQOBJgDHSkTcwFhgEJAIJIjIJFVd5VOnHTAS6K2qmSIS4y2vDzwFdAUUWOLdNxN4C7gFWARMAYYCU48Wi7U8GGOMMeGhO7BBVTeq6kFgAnBhoTq3AGO9SQGqmuotHwLMUNUM77YZwFARaQzUVtWFqqrAx8Dw4gKx5MEYY4wpA6XtthCRESKy2GcZUegUTYGtPuuJ3jJf7YH2IjJPRBaKyNBi9m3q/floxzyMdVsYY4wxZaC03Raq+i7wbinDiADaAf2AOGC2iJxSymMWeRJjjDHGlFJ5j3kAkoBmPutx3jJficAiVc0BNonIOpxkIgknofDdN95bHlfMMQ9j3RbGGGNMeEgA2olIKxGJAq4AJhWq8y3eJEFEGuJ0Y2wEpgGDRaSeiNQDBgPTVDUZ2CUiPbyzLK4FvisuEGt5MMYYY8pARDm3PKhqrojciZMIuIEPVHWliDwLLFbVSRxKElYBHuBBVU0HEJHncBIQgGdVNcP789+Bj4BqOLMsjjrTAix5MMYYY8pEELotUNUpONMpfcue9PlZgfu9S+F9PwA+KKJ8MdCxJHFY8mCMMcaUgWAkD6HCkgdjjDGmDLhdlWcYYeV5pcYYY4wpE9byYIwxxpQB67YwxhhjTIlY8mCMMcaYErHkwRhjjDEl4i7+SdbHDRswaYwxxpgSsZYHY4wxpgxYt4UxxhhjSsSSB2OMMcaUSHk/2yKU2JgHY4wxxpSItTwYY4wxZcC6LYwxxhhTIpY8GGOMMaZELHkwxhhjTIlUpuTBBkwaY4wxpkSs5cEYY4wpA5Wp5cGSh2KcNKQvl415EpfbzbxxnzNt1Ft+23tedykXvzySrKQUAOLfGM+89z8H4OJRj9Dx3AGIy8XqGXP44p5ngh5/RWvYtxcnPvkw4naR+Pk3bHzrA7/tTS+9gA4j72N/SioAW8ZPIPHzbwA44ZF7iR5wFuISdsxZyOpnRgU9/lCwcP48XnvlZTx5eZw/fDjXXn+j3/bJ309i7JjRRMfEAHDJZZdzwfCLARj7+hjmz50DwA0338LAwUOCG3yIWf/7r0z98A00L48uZ59Dn+FXHlZnxfx44r8cDwKNWrTh0nseD36gIWRg9/aMuvMC3G5h/OQERn8af1idi/p1YuT1A1GFFX9u46bnJwDw9Us30vWk5ixcvpnLRn4U1LgrgiUPBgBxufi/sc8yZtDVZCZuZ2TCJJZNmkHy6g1+9ZZ8/gMT7nrKr6x1zy606d2V5zoNBeDBuRNp37cH62YtDFr8Fc7l4uRnH+XXq29l//YUek36lNQZ8ezZsNGvWvIP01n11L/8yup2OZV6XTszd+ilAPSY+BH1e3QlY+HioIUfCjweD6+MepExY98iJjaWm669ij5n9aVV6zZ+9c4eNIR/PPyIX9m8uXNYt2Y14z+dQE5ODnfcejM9e/WmRs2awXwJISMvz8Pk98dw7eMvU7tBNO+OvJ0TuvYiJq5lQZ305ETmfPspNz33OtVq1mLPzsyKCzgEuFzCq/cM58IHxpGUtpP4t+9kyrxVrN2SWlCnTdMG3H9VPwbf+RZZe/bRsG6Ngm1jJsyiWpUobrzgjIoIP+gqU/JgYx6OomX3zqRu2MKOTVvx5OSQMOF7Ol04OKB9VSGiahUioiKJqBKFOzKCXSlp5RxxaKnbuSPZW7ayb2sSmpNL8vc/EjO4X4B7K64qVXBFRuKKisIVEcGBtPTyDDckrVq5grhmzWgaF0dkZCQDBw9hzqz4gPbdvHEjnbt0ISIigmrVqtG2bTsWLphfvgGHsKQNa6jfqCn1Y5sQERFJx14DWJPgfz2WzJxM9yEXUq1mLQBq1qlXEaGGjK4dmrExKZ3NyRnk5Hr46uc/OLf3SX51rjuvO+99u4CsPfsA2JGVXbBt1tI/2bPvQFBjrkhul5RqCSdBSx5EJFZEuniX2GCdtzTqNY0lc+u2gvWsxGTqNT089NMuGcbjf0xlxJdvUi+uMQCbFi5l3S8LGJWcwEvJv7Jq2my2r/kzaLGHgqqxMezftr1gfX9yKlVjD79+scPOpvfULzntzVeo2tjZnrV0GekLEhiQ8BMDfv2JtNnzyf5zU9BiDxVpqanE+lyz6JhY0lIPT0Ljf57JNVdcxqMPPUDKdueat23fnoXz57N//z6ysjJZumQxKSnbD9u3stiVsYM6DWIK1us0aMjuDP9rmb4tkfTkRMY9cRfvPXYH63//NdhhhpTG0XVITMsqWN+WtpMm0XX86rRtFk3buIZM/8/tzHzzDgZ2bx/sME0FKPduCxHpDLwN1AGSvMVxIpIF/F1Vlx5hvxHACIA+1OckapV3qMdk2fc/kfDZJHIPHqTPiCu5bvyrvHb2lUS3aUGjE9syMq4HAPfM+IS2Z1U7zeAAACAASURBVHZjw9yECo44tKT+NIvkSVPJO5hDsysvpdOrz/PrlbdQvUUzarZtxS89nJaebp+8w45up5GZ8FsFRxx6zuxzFoOGDCUqKopvv5rIc08/yRtvv8sZPXqyeuVKbr3xeurWrUfHUzrhdrkrOtyQlpfnIT05kRueGs2ujDQ+eOpe/v7K+1SrUTm7egIR4XbRJq4h59z7Dk2j6zD19dvoeeNodu7ZX9GhBV24tR6URjBaHj4C7lHVE1V1oHfpANwLfHiknVT1XVXtqqpdKypxyExKoV6zJgXrdeMak+kdGJkvOyOL3IMHAZg7bgItTu8IQOeLhrBp4W8cyN7Lgey9rJgaT+ueXYIXfAjYn5JK1SaNCtarNo5hf4r/9cvJ2knewRwAtk74mtodTwQgdsgAsn5bjmfvPjx797Ejfh51u5wavOBDRHRMDCk+1ywtNYXomGi/OnXq1iUqKgqA84dfxNrVqwu2XX/TzYz/9HPGvPk2qkqz5s2DE3gIql2/ITvTD/XV70zfQa360YXqRNOhay/cERHUi2lMg8ZxZCQnBjvUkJGctpO46LoF602i67AtbadfnaS0nUyZt5pcTx5btmeyYesO2jRtGOxQQ4J1W5StGqq6qHChqi4EahRRP2RsSfiDmHYtadAyDndkJN2uOJ9lk2b41and6NAfn1MvGETyaqdrIuOvbbTrewYutxtXRATt+55x2EDL493OP1ZSo2VzqsU1RSIjaHz+UFJnzPKrUyX60B+Z2EH9Crom9m3bTv0zTkfcbiQignpnnE72hsrXbXHiSSeTuPUvtiUlkZOTw0/Tp3HmWf386uzYcajpfe7sWbRs1QpwBlvuzHKanDesX8eG9evp3qNn0GIPNU3adCAjOYnM1GRyc3NYMf9nOnT1vx4duvdm08o/AMjetZP05ETqxTauiHBDwpK1ibSOa0CLRvWIjHBzyYBTmTJ/tV+dyXNX0qdzawDq16lO22YN2ZycURHhVji3SKmWcBKM2RZTRWQy8DGw1VvWDLgW+DEI5z9meR4Pn9/5JHdP+xiX2838D74gedV6zn/mPrYsXs6y739iwN030OmCgeTlesjOyGL89Q8AsHTiFE4Y0Isnlk8DVVb+OIvlP8ys4FcUXOrxsOrJf9Ht47ecqZpffMue9X/S7r6/s3P5SlJ/mkWLG64kZmA/1JNLTtYulj3wBADbp8ygQa/unDltIqiSNms+qTNnFXPG409ERAT3P/gw9931dzyePM674EJat2nDe2+/SYcTT6JP3358OeEz5s6ehdvtpnbtOjz2tDMlODc3l9tvcaZ11qhRk6eee4GIiMo7wcrtdnPOjXfx3xceJi/Pw2n9hxHTrBU/f/4hTdq0p0PX3rQ9tRt//rGYN+67AXG5GHz1rVSvVaf4gx+nPJ48HhzzHd+8fBNul4v/Tk1gzeYUHrthEEvXJjJ1/mp++nUdA7q259eP7seTl8cTb08hY9deAH58/TbaN4+mRrUqrP7yUe58aSIzE9ZV8KsyZUFUtfxPIjIMuBBo6i1KAiap6pRA9r9NWpZ/kMexC1tU3j9+ZaH78gUVHULYm7Gxck95LK0R97xe0SGEvV3xo8r9q/07i7aU6rPq1jNahE3zQ1C+hqjqVGBqMM5ljDHGVAR32Hz0l14wZlvUAUbitDzEAgqkAt8BL6pq1lF2N8YYY8KCK8wGPZZGMAZMfgFkAv1Vtb6qNgD6A1nebcYYY0zYq0wDJoORPLRU1VGqWnB3GlXdrqovAi2CcH5jjDHGlKFgJA9bROQh37tKeu82+TCHZl8YY4wxYc0lUqolnAQjebgcaADMEpFMEckA4oH6wGVBOL8xxhhT7txSuiWclPuASVXNBB72LohIH6A7sFxVK+edRIwxxhx3bMBkGRKRX31+vhl4HagJPCUijxxxR2OMMSaMWLdF2Yr0+flWYLCqPgMMBq4KwvmNMcYYU4aCcZMol4jUw0lURFXTAFQ1W0Ryg3B+Y4wxptyF27iF0ghG8lAHWAIIoCLSWFWTRaSmt8wYY4wJe+HW9VAawRgw2fIIm/KAi8r7/MYYY0wwhNtjtUujwh6xp6p7gcr3jGVjjDEmzFXe5/MaY4wxZci6LYwxxhhTIjZg0hhjjDElYi0PxhhjjCmRyjRgMhg3iTLGGGPMccSSB2OMMaYMuKR0SyBEZKiIrBWRDUU94kFErheRNBH53bvc7LNtlIis8C6X+5R/JCKbfPbpXFwc1m1hjDHGlAF3OY95EBE3MBYYBCQCCSIySVVXFar6uareWWjfc4EuQGegChAvIlNVdZe3yoOqOjHQWKzlwRhjjCkDQXgwVndgg6puVNWDwATgwgDDOwmYraq5qpoNLAOGHtMLxZIHY4wxpky4XaVbRGSEiCz2WUYUOkVTYKvPeqK3rLBLRGSZiEwUkWbesj+AoSJSXUQaAv2BZj77vODdZ7SIVCnutVryYIwxxoQAVX1XVbv6LO8ew2G+B1qqaidgBjDee+zpwBRgPvAZsADwePcZCXQAugH1gYeLO4klD8YYY0wZCEK3RRL+rQVx3rICqpquqge8q+OA0322vaCqnVV1EM6DKdd5y5PVcQD4EKd75OivtbgK4mjtHahhjDHGmCK4RUq1BCABaCcirUQkCrgCmORbQUQa+6xeAKz2lrtFpIH3505AJ2C67z4iIsBwYEVxgRQ720JVVUSWA7WKf13GGGNM5VTed5hU1VwRuROYBriBD1R1pYg8CyxW1UnA3SJyAZALZADXe3ePBOY4+QG7gKtVNde77X8iEo3TGvE7cFtxsQQ6VfM3oD2wJsD6xhhjjCljqjoFZ+yCb9mTPj+PxBnDUHi//TgzLoo65oCSxhFo8hAP/CgiH+GM9FSfk35Q0pOW1FkNq5f3KY5r/d++o6JDCGsfrkyt6BDC3vifNlR0CGEtfULhQfcmFLkr0SjCQJOH3sAmoG+hcgXKPXkwxhhjQp09GKsQVe1f3oEYY4wx4awS5Q6B357aO0rzHKCRqr4sIk0Al6omllt0xhhjTJhwUXmyh4B6aESkL7AWuArIH5jRDnirnOIyxhhjTIgKtOXhNeByVZ0pIpneskUEcCMJY4wxpjKwbovDtVTVmd6f82daHCzB/sYYY8xxLdDHah8PAp1YskpEhhQqGwgsL+N4jDHGmLAkUrolnATacvAP4AcRmQxUE5F3gPMJ/FGgxhhjzHHNBkwWoqoLce6DvRLnvg6bgO6qmlCOsRljjDEmBAXU8iAiD6jqK8BLhcrvV9V/l0tkxhhjTBgJt66H0gh0zMOTRyh/vKwCMcYYY8KZS0q3hJOjtjyISP7DMtwi0h/8OnRaA7vLKzBjjDEmnITZ53+pFNdt8b7336r4P8NCgRTgrvIIyhhjjDGh66jJg6q2AhCRj1X12uCEZIwxxoQfezBWIap6rYhEAj2AJqr6uYjU8G7LLs8AjTHGmHBQiXKHgJ9tcQqwDniPQ10ZfbHHcRtjjDGA84FamiWcBBrvW8CTqtoByPGWzQLOLJeojDHGmDAjIqVawkmgycPJwCfenxUKuiuqlUdQxhhjjAldgSYPm4HTfQtEpDuwoawDMsYYY8KR3efhcE8Ak0XkbSBKREYCtwG3lFtkxhhjTBgJs56HUgn02RY/AEOBaJyxDi2Ai1V1ejnGZowxxoSNyjRgMtCWB1T1N+Dv5RiLMcYYY8JAoA/GigD+DzgNqOm7TVVHlENcxhhjTFgJtxkTpRFoy8MnwCnAVJzbUhtjjDHGR7gNeiyNQJOHoUAzVbUHYRljjDFFqES5Q8DJw0qgPvYUTWOMMaZI1vJwuGuAcSIynULdFqr6cZlHZYwxxpiQFWjycD3QB6gH7PMpV8CSB2OMMZWeDZg83D3Aaaq6ujyDCUWNB5zJ6S+MRNxu/vxkIqteH+e3vdUVwzntqQfYtz0VgHXv/48/P/kKgM5P/oMmg/oiLmH7rAUsefSfQY+/os1bvYlRX/9CXp5yUY+O3DToDL/t3y1awejvZhNT15nEc0WfzlzcsxMAoyfNZs7KjQCMGNKDoV06BDf4ELR5WQKz/vc2eXkeOvYdRrfzLj+szrpFs1j4rXM3+ejmrRl2+8hghxmyerRuwH1D2uMSYdLvSfx3/pbD6px9Ygw3n9UaBdan7OGpb1cEP9AQMnfRYl78zzt48vK45Nwh3HzVZX7bv506g1ffep+Y6IYA/N9F53HpeUMB+PfbHzB7YQIAt157BcMG9A1u8EFm3RaHSwH+Ks9AQpG4XHR98XF+/tvN7NuWwpDpn5P44y/sWvenX72/vpvK4kde8Ctr2K0z0WecxtS+wwEY9MMnxPTqRur8hKDFX9E8eXn888uZvPP3S4mtW4srX/0f/U5pS5tGDfzqDe5yAo9eerZf2eyVG1mzNYUvHrqWg7kebv7P55x5UitqVq0SzJcQUvLyPPzy8Vgufuhf1KzfkM+evovWp/WgQdMWBXUytyeR8MPnXPb4v6laoxZ7d2VVYMShxSXwwLATuPt/v5G6az8f3tSdOet2sHlHdkGdZvWqcW3vVowYv5jd+3OpVz2yAiOueB6Ph+dfe5P3Xn2BRtENufzWe+nfuwdtWjb3qzd0wFk8dq//bYBmLfiVVes2MHHcGxzMyeGGex6mzxndqFmjejBfQlBVotwh4JtajQY+EZEeItLadynP4Cpagy6nsGfzX2RvSSQvJ4ct304lbtiAwHZWxV2lCq6oSFxVopDICPanpZdvwCFmxZbtNIuuS1zDukRGuBna5QTilwf2OJSN29Pp0jaOCLeL6lUiadckmnmrN5dvwCFu+8a11IltQp2YxrgjIml/Rj/+XLrAr86KWVM59ezzqVqjFgDVa9etiFBD0klN6pCYsY9tWfvIzVNmrEzhrPbRfnUuPK0pXy3eyu79uQBk7s0p6lCVxvLV62jetAnNmjQmMjKSYQPO4ue5C4rfEfhz8190PbUjERFuqlerSvs2rZi7aHE5R1yxXCKlWsJJoMnDWOBCYD7Ow7Dyl/XlFFdIqNY4luyk7QXre7dtp3rjmMPqNTtvMMPiv+HMD0ZTvUkjAHYs/oOUub9y0YpZXLRiFsm/zGPX+o1Biz0UpO7cQ6O6tQrWY+rWImXnnsPqzfxjPZe+OJ5/fDCJ7Zm7AGjfNJr5qzez72AOmXv2krBhK9szK/dkn+zMdGrVP/RhV6t+Q7Izd/jVydyeSGZKEp8/dx8Tnr2HzcsqT0tXcaJrVSF11/6C9dTd+4mu5d+S1axBdZrXr86713Vl3PXd6NG6QeHDVCqpO9JpFNOwYD02uiGpOw7/EjRj1jwuuuHv3PfkCySnpgFwQtvWzP11Cfv27yczaycJvy1je9qOw/Y14SmgbgtVLdVtt0WkDs69Ipp6i5KAaap6xDZVERkBjAC4qWYjBlStV5oQyk3StF/Y8vVk8g7m0Pbay+jxxj/5+eIbqdmqObXbt+bbU52WigFfjiO5x+mkLVxSwRGHlr4d2zDs9A5ERUTw5bw/ePx/PzLuzsvo1aElK//aznWvfUa9GtU4tWVj3JWpQ/EYqcdD1vYkLh35Mnsyd/DlP//B1c+/Q9UaNYvf2eB2CXH1q3P7f5cQU7sKb1/blaveWcieA7kVHVrI6tfrDM45ux9RUZF8MWkKj/3zVT547UV6d+vCijXruPqOB6hXpzanntwBtyvcnuBQMmHWeFAq5f6bFJFrgaVAP6C6d+kPLPFuK5KqvquqXVW1a0UlDvuSU6jRtFHBevUmjdibnOpX52DmTvIOOk2bf34ykfqnngxAs3MGkr74D3Kz95KbvZdtM+fQsOupwQs+BMTUqcn2rEOtBalZu4mt4/8hVrdGNaIinBz24p6nsHrroZnAtwzuwRcPXcs7d/wNVWgRHZoJZLDUqNeA3RlpBeu7M3ZQo15Dvzo16zek9Wk9cEdEUCe6EfUaxZGVkhTsUENS2u4DxNSuWrAeU6sqabsP+NVJ3XWAOevT8OQpyVn7+St9L83qH7999MWJadiA7amHWgtS0nYQ09C/NaZundpERTljQy45dwir1h3qmrz1miv46v03GPfvf6KqtGjWlOOZqJZqCScBJQ8iEiEid4vIVyIyS0Rm5y8B7P4YcLqq3q6qz3uX24CuwOOlCb68pf+2glqtWlCjeVNckZG0GD6MpB9/8atTNfbQH++mQ/uza53TNZGdtI2YXt0QtxuJiCCmV7eCbZXFyc0b8VdaFonpO8nJ9fDj0rX07djGr06aTzdG/PI/aRXr/GHy5OWRle3MCl6XlMa6bWn07NAyaLGHokatTiArJYmdadvx5OawblE8bU7r4VenTZdeJK5ZBsC+3TvJ3J5InZjGFRFuyFm9bRfN6lejcd2qRLiEQSfHMmddml+d2WvT6NLCSVLrVIukeYPqJGXtK+pwlULHDu35K3EbicnbycnJYerPs+nf2/89l5aeUfDzL/MW0bpFM8AZbJm10+mGXPvnJtZt3Eyvrl2CF3xF0LzSLWEk0NkWo4EBwLvACzgJwe3AhAD2FZz7QRSWR4gPTlWPh8UjX6D/F+8hLhcbP/uGnWs3cMrDd5Lx+0qSpv3CCbdcQ9Mh/dHcXA5m7WThXY8CsHXSdGLP7ME5s78FheSf55A0Pb5iX1CQRbhdjLxkALe/9RV5eXkM79GRto0bMnbKPE5uFku/U9ry6ezfiF/xJxEuF7WrV+W5q4YAkOvJ44YxzturRtUq/POac4hwH99NnsVxud30v+YOvnn5UTQvj5PPGkyDuJYs+Ho8MS3b06ZLT1qc0pUtK5by8chbEJeLPpffQrWatSs69JDgUeWVH9cy5v9Ow+USfvh9G5t2ZHNL39as2baLOet3sHBjOme0rs9nt/bAo/Cfn9aza1/lHTQZEeHm0Xtv59YHHseTl8dF5wymbasWvPH+fzm5Qzv69+7BJ199R/y8RbjdburUqsXzj9wPQG6uh2vvehCAmjWq8+JjDxAR4a7Il2PKkGgATSUikgT0VNW/RCRLVeuKSAfgHVU96sRdEbkOeBKYDmz1FjcHBgHPqepHxZ3/0+iTwqs9J8Rc/N97KzqEsPZh3cEVHULYG/9TYLNsTNHm3NyqokMIe5GN2pT7l9UD2btL9VlVpUatkP5C7SvQlofqHPrg3yci1VV1jYicVtyOqjpeRCYBQzg0YDIeGKmqmSUN2BhjjAlJYdb1UBqBJg+rgW7Ar8Bi4GkR2YUza6JYqpopIr/gM9vCEgdjjDHHlTAb9FgaJbk9df5cpfuBt4BaeKdSHo2IdAbeBuoAiTjjHOJEJAv4u6ouLWnQxhhjTMixlodDRMQNnAL8D0BV1wMDS3COj4BbVXVRoeP2AD4EKtf8RWOMMSbMFTt8XVU9wL9V9UBxdY+gRuHEwXvchUCNYzymMcYYE1JE80q1hJNAuy2+F5HzVfX7YzjHVBGZjPPo7vxBl82Aa4Efj+F4xhhjTOgJswSgNAJNHqoCE0VkAU4CUDAqRFWPeJdI7/a7RWQYzrMxfG9PPVZVp5Q8ZGOMMSYEWfJwmBXe5Zio6lRg6rHub4wxxoS8ICQPIjIUGAO4gXGq+mKh7dcDL3NoNuQbqjrOu20UcK63/DlV/dxb3grnpo8NgCXANap68GhxBPpgrGcCqVcU70OxRuK0PMTitFqkAt8BLx7t4VjGGGOMcXgnMIzFucliIpAgIpNUdVWhqp+r6p2F9j0X6AJ0BqoA8SIyVVV3AaOA0ao6QUTeBm7CmVV5RAHf71dEokTkFBHpLyID8pcAdv0CyAT6q2p9VW2A82CsLO82Y4wxJvzl5ZVuKV53YIOqbvS2DEzA+WIeiJOA2aqaq6rZwDJgqIgIzuMnJnrrjQeGF3ewQB+MdSawBZgFzPCeZBowLoDdW6rqKFXdnl+gqtu9TS0tAjm/McYYE+pKO9tCREaIyGKfpfC9lJpyaOIBOK0PRT2q9BIRWSYiE0WkmbfsD5xkobqINMT5Et8Mp6siS1Xz7+V0pGP6KcmDsV5S1dEikqmq9UXkSWBvAPtuEZGHgPGqmgIgIrHA9fhfBGOMMSZ8lXLMg6q+i/MAytL4HvhMVQ+IyK04LQkDVHW6iHQD5gNpwALAc6wnCbTboj3OAA1fLwL3BbDv5TiZzSwRyRCRDJxnW9QH/hbg+Y0xxpjKLgmntSBfHIUeE6Gq6T73ZRoHnO6z7QVV7ayqg3Du9rwOSAfqikjEkY5ZlECTh51A/nN9k0XkJKAeULO4HVU1U1UfVtUO3jEP9VX1RFV9mAD6VYwxxpiwoFq6pXgJQDsRaSUiUcAVwCTfCiLS2Gf1ApxnUyEibhFp4P25E9AJmK7Oo7V/AS717nMdzoSGowq02+Jr4BzgU+AD74lyODTA4lg9g3OLamOMMSa8lfNUTVXNFZE7ccYcuoEPVHWliDwLLFbVScDdInIBzvOoMnCGCABEAnOc8ZHsAq72GefwMDBBRJ4HfgPeLy6WQKdq3uvz8ysishDnwVjTittXRJYdaRPO1E1jjDEm7AXjFtPemytOKVT2pM/PI3Fuj1B4v/04My6KOuZGnJkcATtq8iAiJ+C0DHQElgI3qOomVZ1bgnPEAkNwpmv6HR5n4IYxxhgT/irRHSaLG/PwH2AjTr9KEs6si5L6AaipqlsKLZtxBk4aY4wxJowU123RBYhT1f0iMhtnZGaJqOpNR9l2ZUmPZ4wxxoSkStTyUFzyEOXtJ0FV94hI1SDEZIwxxoQfSx4KVPGO4sxXrdC630ANY4wxprIKxoDJUFFc8vAp/jekmFBoPaCJqcYYY8xxL7DnUxwXjpo8qOoNwQrEGGOMMeEh0JtEGWOMMeZoArtL5HHBkgdjjDGmLNiYB2OMMcaURGUaMBnog7GMMcYYY4BikgcR6S0io46w7UUR6VE+YRljjDFhRvNKt4QR0aMM8BCRycCbqjq5iG1DgTtU9fxyjA+AvzL2VJ5RKCbkzN+6s6JDCHv1qkZWdAhh7d15myo6hLD31Y1nSHmfI2/DwlJ9Vrna9ij3GMtKcWMeOgM/HmHbTziP5zbGGGNMnqeiIwia4pKH2kAUsK+IbZE4j+U2xhhjKj2tRDeJKm7A5Bpg8BG2DfZuN8YYY0wlUlzLw2jgHRFxA9+qap6IuIDhwFjg/vIO0BhjjAkL1m3hUNVPRaQRMB7nIVk7gIbAAeApVf0sCDEaY4wxoc+Sh0NU9d8iMg7oCTQA0oEFqrqrvIMzxhhjwoV6LHnw400UppVzLMYYY0z4sgGTxhhjjDFFs2dbGGOMMWXBxjwYY4wxpiTUkgdjjDHGlIiNeTDGGGOMKZq1PBhjjDFlwLotjDHGGFMyljwYY4wxpkQq0ZgHSx6MMcaYMlCZ7jBpAyaNMcYYUyLW8mCMMcaUBRvzYIwxxpgSseTBGGOMMSWhNmDSGGOMMSVSiVoebMCkMcYYY0rEWh6MMcaYslCJWh4seTDGGGPKgI15MMYYY0zJVKKWBxvzYIwxxpgSsZYHY4wxpixUopYHSx6MMcaYMlCZnm1hyUMxEhbM583XXiHP42HYBcO54tob/LZPmzyJ994YQ4PoGAAuvPQyzrngIgDeG/s6v86fC8BVN9xMv4GDgxt8CLDrV7bW//4rUz98A83Lo8vZ59Bn+JWH1VkxP574L8eDQKMWbbj0nseDH2iIWrVkEV+PG0OeJ4+eg89j0KVXH1Zn6dyfmfrZBwhC01Ztue6Bpyog0tDUuWkdbuzRApcIM9el8s2y5MPq9GpVn8s6xwHK5oy9vDbrz+AHWlFswKQB8Hg8/OfVFxk15k0axsRy543X0LNPX1q0au1Xr+/Zg7nrgYf9yhbNm8OGtWt4e/ynHMzJ4YE7RtCtZy9q1KgZzJdQoez6la28PA+T3x/DtY+/TO0G0bw78nZO6NqLmLiWBXXSkxOZ8+2n3PTc61SrWYs9OzMrLuAQk+fx8OU7/+aOZ0dTt0E0r/zjFjp2703j5q0K6qRu28qMLz/hvlFvUb1mLXZn2fXL5xK4pWdLnp22hvTsg4y64GQS/soiMWtfQZ3GtatwUacmPDZ5JdkHPdSuWsk+YipRt4UNmDyKtatW0iSuGY2bxhEZGUm/gYOZPzs+oH23bNrEKZ1Pwx0RQbVq1Wjdph2LF8wv34BDjF2/spW0YQ31GzWlfmwTIiIi6dhrAGsS/K/JkpmT6T7kQqrVrAVAzTr1KiLUkLRl/WqiGzelYaMmRERG0qXP2SxfNNevzoJp39Pn3Iuo7r1+tera9cvXtmFNtu/aT8ruA+TmKXM3ZtCtuf/1Gdg+hh9Xp5B90PkQ3bU/tyJCNUEQlORBROqIyOUicr93uVxE6gbj3KWxIy2V6JjYgvWGMbHsSEs7rN7c+JmMuPpynn30IVJTtgPQul07EhYuYP/+fezMyuT3pYtJTU0JWuyhwK5f2dqVsYM6DWIK1us0aMjuDP/rmb4tkfTkRMY9cRfvPXYH63//Ndhhhqys9DTqNjx0/eo2jGZn+g6/OqnbtpKWtJXRD93Oqw/cyqoli4IdZsiqXyOKHdkHC9Yzsg/SoHqkX50mdarSpHbV/2/vvuOkKs/+j3++uwtKEaRIF0FQAYmCUhQ1YJ5YMD6giUmMid3YHp9oii0m0WgwavKoSZREYo+JSvSXBAuWqCixgQqKigii0nsRLAi71++POQuzK222zNnZ/b5fr/Ni5j73zLn2Bnauucu5GfW1Pvz66L3p17llvsNMVZSVVusoJLWePEg6CXgNGAY0TY5DgVeTc1t63ZmSXpH0yt/uur22w6yyAw/+Mn/5fw8z5p772W/gYH5zVWZ8dMDgAxk05CDOP/M0rv7FZfTp+yWKi4pTjrbucfvVrLKyUpYvnMepl9/Acef/jHG3/B+ffrw27bAKRllpKUsXzuMHV/+BU35yOffdfB2frF2TdlgFo0iiY8sd+cWj07lhwizOOag7TRs3nP+3Av0h0gAAIABJREFUUVZWraOQ5KPn4TJg/4g4JyJ+lRxnAwOALc7kiogxETEgIgaccPJpeQjzi9ru0o6lWd92ly1ZTNtddqlQp0XLnWncuDEAw0ccw7vvTN947runnM4td9/Ltb8fTUTQuWvX/AReR7j9alaL1m1ZvXzJxuerly9jp9aV2rP1LvQaMITikhJatetIm45dWLFwXr5DrZN2brMLq5Ztar9Vy5bSsk3binXatqPvoIMoLimhTYdOtOvUhaVuPyDT09C2WeONz1s3a8zyT9ZXqLP8k8+ZPGclpREsWbuOBR99RscWO+Y71NREaVm1ju0h6UhJMyTNknTJZs6fImmppKnJcUbWueskvSVpuqTfS1JSPiF5z/LXtKv8vpXlI3kQEJspL0vO1Vl79e7D/LlzWbhgPuvXr2fCv5/gwEOGVqizfNmmbuMXJz5L126ZyVelpaV8tHoVALNnzeT992YxYNAB+Qu+DnD71axOPXqxYuF8Vi5ZyIYN63nzhafpNeDACnV6DTqI9996HYCPP1rN8oXzaNW+Yxrh1jld9+jF0gXzWL5oARvWr+e1iU/xpcEHV6jzpcGHMGvaVADWfrSKJQvm0bZ9pzTCrXNmLVtLx5Y70q75DpQUiYN3b80rcypOKJ304Ur27tACgJ12KKFTix1ZvGZdGuGmoraTB0nFwM3AcKAP8B1JfTZT9f6I6JcctyavHQIcBOwD9AUGAtm/kL+b9ZolX3zLivIxFXYU8JqkJ4C5SVlX4DDgqjxcv8qKS0o478cXcekF51FWVsoRR4+k2+49uHPMH9mzdx+GHDKUf469jxf/8xzFxcXs1KIFF/7sCgBKN2zgh2dnEr6mzZpx8eVXUVzSsGYeu/1qVnFxMUed9r/8ZdTFlJWV0v/Q4bTbtTtP338HnXrsSa8BB9Fz34G89/or3PTDU1FREYd/7yya7tSwxp23pLi4hOPO+iGjr/gxZWVlHPDVr9Gxa3ce+eutdO3Ziy8NPpje+w3inamTGPU/36OoqJiRp5xDsxZuP4CygFtf/ICfH7EXRRJPz1zK3FWfcnz/zsxa9jGvzF3F1Pmr6de5JTceuw9lEdw9eQ5r13nSZA0aBMyKiNkAku4DRgJvb8drA9gRaEzmi3sjoMoTyRSxuU6BmiWpFXAE0Dkpmg88HhHbtQ5qzoq1tR+k2Ra8MHd12iEUvFY7Ntp2JduiMc+/n3YIBe/B0wbXek/36tt+Vq3Pqp3PGHUWcGZW0ZiIGFP+RNJxwJERcUby/ERgcEScl1XnFODXwFLgXeCHETE3Ofdb4AwyycNNEXFZUj4BaAOUAg8Cv4ptJAd5+SoXESslPUNW8rC9iYOZmVkh2N55C1t8fSZRGLPNilv3EHBvRKyTdBZwF/AVST2B3kCXpN6Tkg6JiIlkhizmS9qJTPJwInD31i5S68mDpH7An4CWwDwyGU8XSauAcyPitdqOwczMrLZVN3nYDvOBXbOed0nKNsUQsTzr6a3AdcnjY4GXImItgKTxwIHAxIiYn7x2jaS/kRke2WrykI8Jk3cC50dE74g4LCK+GhG9gAuAO/JwfTMzs/pgMrCHpO6SGgPHA+OyK0jKniE9AihfwjYHGCqpRFIjMpMlpyfP2yavbQQcDby5rUDyMWzRLCK+cKeViHhJUrM8XN/MzKzWldXyxlgRsUHSecDjQDFwe0S8JelK4JWIGAf8QNIIYAOwAjglefkDwFeAaWQmTz4WEQ8ln8OPJ4lDMfBv4M/biiUfycN4SY+Q6QIpX22xK3AS8Fgerm9mZlbr8nGjp4h4FHi0Utkvsh5fCly6mdeVAmdtpvxjYP9c46j15CEifiBpOJnlJNmrLW5OGsHMzKzg5WHOQ52Rr9UW44Hx+biWmZlZGhpS8pCPvS1aSromuR3mCknLk8fXFMLmWGZmZlZRPlZbjAVWAodGROuIaENmY6xVyTkzM7OC15A2xsrHsEW3iLg2uyAiFgHXSDo1D9c3MzOrdWUNaNgiH8nDh5IuAu6KiMUAktqTWT4yd2svNDMzKxSe81Czvk3mntnPSlopaQUwAWgNfCsP1zczM7MalI+lmiuBi5MDSYeQufXltIhYUdvXNzMzywf3PNQgSZOyHp8B/B5oDlwu6ZLavr6ZmVk+eMJkzcrei/cs4PCIWJpsDfoScE0eYjAzM6tVDannIR/JQ5GkVmR6ORQRSyFzS0xJG/JwfTMzs1rn5KFmtQReJbMVd0jqGBELJTVPyszMzKyA5GPCZLctnCojs7+4mZlZwSsrsHkL1ZGXvS02JyI+Ad5P6/pmZmY1ycMWZmZmlpMoLU07hLxx8mBmZlYDCm25ZXXk4w6TZmZmVo+458HMzKwGeM6DmZmZ5cTJg5mZmeWkIW3J7TkPZmZmlhP3PJiZmdWAhrTaoiCSh12aFkSYddabSz5NO4SC9vVPXko7hILX+Rez0w6hoK2eMz3tEArfaYNr/RKe82BmZmY5idJIO4S8cfJgZmZWAzxh0szMzGwL3PNgZmZWA6LMwxZmZmaWgzLPeTAzM7NceLWFmZmZ5aQhrbbwhEkzMzPLiXsezMzMaoDnPJiZmVlOPOfBzMzMclLWgJZqes6DmZmZ5cQ9D2ZmZjWgIa22cPJgZmZWAxrS3hZOHszMzGqAex7MzMwsJw0pefCESTMzM8uJex7MzMxqgOc8mJmZWU68JbeZmZnlxLenNjMzs5w0pNtTe8KkmZmZ5cTJg5mZWQ2I0qjWsT0kHSlphqRZki7ZzPlTJC2VNDU5zsg6d52ktyRNl/R7SUrK95c0LXnPjeVb42ELMzOzGlDbcx4kFQM3A4cB84DJksZFxNuVqt4fEedVeu0Q4CBgn6ToP8BQYALwR+D7wMvAo8CRwPitxeKeBzMzsxoQZWXVOrbDIGBWRMyOiM+B+4CR2xsesCPQGNgBaAQsltQRaBERL0VEAHcDx2zrzZw8mJmZ1QGSzpT0StZxZqUqnYG5Wc/nJWWVfUPSG5IekLQrQES8CDwDLEyOxyNievL6edvxnhV42MLMzKwGVHfYIiLGAGOqGcZDwL0RsU7SWcBdwFck9QR6A12Sek9KOgT4tCoXcc+DmZlZDcjDhMn5wK5Zz7skZZtiiFgeEeuSp7cC+yePjwVeioi1EbGWzJyGA5PXd8l6iy+85+a452Ebnn/+ea679lrKyso49thjOe300yuc/9e//sWNN9zALu3aAXD88cfz9a9/HYAbb7iBiRMnAnDmmWdyxJFH5jf4OuD1yS/yl9E3UFZWxrDhIxhx/EkVzj/3+MPc++ebaNVmFwAOG3kchx6VGcK77883MXXSCwAc891TOWDYYfkNvo74zxvvcu3fHqa0rIyvf3kgZxw9tML5f058levHjqfdzi0B+M5XD+AbQwcCcP3Yx5j4+jsAnDXiKxw5eB8asq/078ao7/8XxUXiniff4PcPTvpCnZEH7cWF3xlCBLz1/hLOvv6RFCKtOw4f0pfrLzyBoiJxxz8n8ps7Hv1CneMOG8jPzx5JRPDGu3M56aeZL88P3fRDBu/Tg+enzOTY83+X79DzLg/3eZgM7CGpO5kP+OOBE7IrSOoYEQuTpyOA6cnjOcD3Jf0aEJnJkjdGxEJJH0k6gMyEyZOAP2wrECcPW1FaWsqvr76aP91yC+3bt+e7J5zA0GHD6NGjR4V6hx9+OJf+9KcVyp577jmmv/MO948dy/rPP+f0M87goIMPpnnz5vn8EVJVVlrKXX/4LZdc+3tat23HL847lf0PPITOu3WvUO+AoV/l5P/9SYWyKS8/zwezZjDqT3ez/vP1jPrJuewzcAhNmzXL54+QutKyMkb9ZRxjLjyNDq1bcPwvR3No/1706Ny+Qr0jBu3DZSeOqFD23NR3mP7hAv5+5f/y+YZSTrvmzxy8z540b7JjPn+EOqOoSFxz1mF88/KxLFi+hid+eyKPTXqPd+cu31hn9447c/5xg/naxX9j9cfraNuyaYoRp6+oSPzuku9x1Dn/x7zFK3jxr7/g4WenMn32go11enZtx0WnHcXQU65m1ZpP2KXVThvPXX/3YzTdsTFnfGNYCtHnX22vtoiIDZLOAx4HioHbI+ItSVcCr0TEOOAHkkYAG4AVwCnJyx8AvgJMIzN58rGIeCg5dy5wJ9CETI/EVldagIctturNN99k1113pUuXLjRq1IgjjjySCRMmbNdrZ8+ezf777UdJSQlNmjZlzz324Pnnn6/dgOuY92a8TftOXWjXsTMljRpxwLDDePWF57brtfM/fJ+9vtSf4uISdmzShK679+SNV16s5Yjrnmmz59G1fRt2bdeaRiUlDB+8D89Mmb7tFwLvLVjC/nt2o6S4mKY7NGbPLh34z7R3azniumu/PTrywaKVfLh4Nes3lPHPie8wfFDPCnW+d/i+3P7oFFZ/nOn1Xbb6kzRCrTMG9t2d9+Yu4f35S1m/oZSxj7/Mfw/rV6HO6ccO5Y9jn2bVmkxbLV25ZuO5ZyZNZ83Hn+U15vouIh6NiD0jokdEjErKfpEkDkTEpRGxd0TsGxGHRsQ7SXlpRJwVEb0jok9E/CjrPV+JiL7Je56XrLrYKicPW7FkyRI6dOiw8Xn7du1YsnjxF+o99dRTfPO44/jJj3/MokWLANhzzz15/oUX+PTTT1m5ciWTJ09mcXKuoVi5bCmtd2m38Xnrtu1YuWzpF+pN+s8zXHrmd/ndlZeyfEmmfXfbfQ/emPwi6z77jDWrV/H21FdZseSLbV/fLVm5mg6tW2583r5VSxav/OgL9f79ylt8/We/50c3/ZVFy1cBsFfXjjw/7V0+Xfc5K9d8zKR3ZrN4+eq8xV7XdGzTnPnLNn2wLVi+ho5tKvYE9ujUit07teaRa05g/HXf5Sv9u+U5yrqlc7udmbd4xcbn8xevpNMurSrU2WO39uzZtQMT7riUiXddxuFD+uY7zDojHzeJqivyMmwh6Qgy60bLl3/MB/4VEY9t5TVnAmcC/OGmmzi90lyDumLo0KEMHz6cxo0b88Df/87Pf/Yz/nzrrQwZMoS33nqLk08+mVatWrHPvvtSVFycdrh1Tv8DD+HAQw+nUePGPPXwP7jlN1fy09/czJcGDGb2jLf55fnfp8XOO7NHn75uvy0Y1r83Rx2wL40blTD2mZe57NYHuO3iMxjSdw/enD2PE391C61aNGPfHl0pKvL3ha0pKS5i906tGHnZfXRqsxPjfn08X/7BnXz08bptv7iBKi4upmfX9nz1+9fRpV0rnrrtEvb75s9ZvbZKk/gLWtm2v7DXG7X+m0TSjcD5wLPAdcnxLJlxmS3OoImIMRExICIGpJU4tGvXbmNPAsDiJUto177iWPPOO+9M48aNATj2619n+vRNXcrf//73GTt2LLfccgsRwW677ZafwOuIVm13YcXSJRufr1i2hFZtd6lQZ6cWLWmUtN+hw0fw/rvvbDw38runcvUtf+GSa/9ABHTo3DU/gdch7Vq1ZNGKTb0Fi1eupn2rFhXq7Ny8KY0bZb4HfGPoQN7+YNNE6TNHHMoDV/0vf77wNIJgtw5t8xN4HbRw+Vo6t900Ht+pzU4sXL62Qp0Fy9fw+KRZbCgtY86S1bw3fyW7d2xV+a0ajPlLVtGlfeuNzzu3b8WCpSsr1VnBQ89OZcOGUj5YsIyZHy6iZ9f2ld+qQSiNqNZRSPLxNeSoiDgqIu6LiP8kx33A14Cj8nD9Ktt7772ZM2cO8+fNY/369Tz+2GMMHVpxpvvSpZu64Z+dMIHu3TOTAUtLS1m1KtN9/O677zLz3Xc58MAD8xd8HbD7Xr1ZNH8uSxYuYMP69bw04Un2O/CQCnVWLl+28fGrL06kU9duQGay5ZqPMh+ac2bPZO77s/jSgEF5i72u6Nu9Mx8uXsa8pStYv2ED419+g2H9e1eos3TVpmGMCVOms3vHzFBRaVkZq9ZmxqFnzF3IzLmLGNK34hh/QzJl5kK6d2xF13YtaVRSxDGH9OKxSbMq1Bn/0kwO6ptZCdd6pyb06NyKDxevSiPcOuGVt96nZ9f2dOvUlkYlxXzriME8PGFqhTrjnpnC0AF7AdBm5+bssVsH3p//xeHJhqA0qncUknwMW3wmaWBETK5UPhCo0zNpSkpKuOTSSznnnHMoKytj5DHH0LNnT0bffDN99t6bYcOGce/f/saECRMoKSmhRYsWXHnVVQBs2LCB0049FYBmzZox6uqrKSlpWItbiotLOPm8n3DdpedTVlbG0COOpku33XngzjF037MX+w/5Mk/8cyyvvTiR4uJimu3UgrMu/DkAG0o3cNUPzwKgSdNmnHPxFRQXN6z2AygpLuan3xvB2b+9g9Ky4NhD9qdn5/bc9P+eZO/uXTi0f2/++uSLTJgyneLiIlo2a8JVZ3wDgA0bSjn56lsAaL7jjvz6zG9R0oCHfkrLgkvH/JuxVxxHUVER9z41jRlzl3PxCQcxddYiHp/0Hk9P+YBh/bvzn5tOpbQ0uOLOZ1m5pk7/mqpVpaVlXHDtPTwy+kcUFRVx17/+w9uzF3D5Ocfw6tsf8PCzU3nihTf56oF78/qDv6K0tIxLbxzLitUfA/D0bZewV/eONG+yA7Mf+y1n/fIOnnzxrZR/KqsJ2o5JldW7gLQfmU03dmLTLTB3BVYD/xMRr27rPT797LMCy8nqljeXNLyxx5q07/yn0w6h4HW+ZnbaIRS01XO2b4WNbdnnU27f5k6R1XV/uz7V+qz69pK3az3GmlLrX+Ui4jVgsKQOZE2YjIiGtfTAzMzqtUIbeqiOfK22aEnmblYbkwdJj0dEwx1MNDOzeqXQJj1WRz5WW5wEvAYMA5omx6HAq8k5MzMzKyD56Hm4DNi/ci+DpFZk7qN9dx5iMDMzq1UetqhZInMf7crKknNmZmYFryENW+QjeRgFvCbpCWBuUtYVOAy4Kg/XNzMzq3XueahBEXGXpHHAEWyaMDkBuDQiVm7xhWZmZgXEyUMNi4iVkp6h4lJNJw5mZmYFqNaTB0n9gD8BLcncJEpAF0mrgHOT+0CYmZkVNM95qFl3AmdFxMvZhZIOAO4A9s1DDGZmZrXKwxY1q1nlxAEgIl6S1CwP1zczM6t17nmoWeMlPULmfg7lqy12BU4CHsvD9c3MzGqdex5qUET8QNJwYCRZEyaBmyPi0dq+vpmZmdWsfK22GA+Mz8e1zMzM0tCQhi3ysbdFS0nXSJouaYWk5cnjayTtXNvXNzMzy4fSqN5RSGo9eQDGAiuBQyOidUS0IbMx1qrknJmZWcErjajWUUjykTx0i4hrI2JReUFELIqIa4Dd8nB9MzMzq0H5SB4+lHSRpPblBZLaS7qYTasvzMzMClpZNY9Cko/k4dtAG+DZZM7DCjJ7W7QGvpmH65uZmdW6hjRskY+lmiuBi5OjAkmnkrnLpJmZWUErtEmP1ZGPnoet+WXK1zczM6sR7nmoQZLe2NIpoP0WzpmZmVkdlY+bRLUHjiCzXDObgBfycH0zM7Na15CGLfKRPDwMNI+IqZVPSJqQh+ubmZnVukIbeqiOfEyYPH0r506o7eubmZnlg3sezMzMLCcNqech7dUWZmZmVmDc82BmZlYDGtKwhaIBdbPUFklnRsSYtOMoZG7D6nH7VY/br/rchg2Lhy1qxplpB1APuA2rx+1XPW6/6nMbNiBOHszMzCwnTh7MzMwsJ04eaobH+arPbVg9br/qcftVn9uwAfGESTMzM8uJex7MzMwsJ04ezMzMLCdOHszMzCwnTh7MzMwsJ04eqkDSIEkDk8d9JP1I0lFpx1UIJO2T9biRpJ9JGifpaklN04ytPpB0atoxFAJJvST9l6TmlcqPTCumQuLfgebVFjmSdDkwnMy+IE8Cg4FngMOAxyNiVIrh1XmSXouI/ZLH/we0Ae4AjgHaRMRJacZX6CTNiYiuacdRl0n6AfA/wHSgH3B+RPwrObfx36dtnn8HGjh5yJmkaWR+4ewALAK6RMRHkpoAL0fEPlt9gwZO0pSI6J88ngoMjIj1kgS87vbbNklvbOkUsGdE7JDPeApN8n/4wIhYK6kb8ADwl4j4Xfa/T9s8/w408K6aVbEhIkqBTyS9FxEfAUTEp5LKUo6tELSUdCyZIbMdImI9QESEJGey26c9cASwslK5gBfyH07BKYqItQAR8YGkYcADknYj04a2df4daE4equBzSU0j4hNg//JCSS0B/8fZtmeBEcnjlyS1j4jFkjoAy1KMq5A8DDSPiKmVT0iakP9wCs5iSf3K2y/pgTgauB34UrqhFQT/DjQPW+RK0g4RsW4z5W2BjhExLYWwzGw7SepC5tvzos2cOygink8hrILh34EGTh6qRFIRQESUSWoM9AU+iIgV6UZWGCT1AkYCnZOi+cC4iJieXlSFJZkjMoiKbTgp/B96u7j9qsftZ04eciTpGOAWMt1zZwM/BdYCewHnRMRDKYZX50m6GPgOcB8wLynuAhwP3BcR16QVW6GQdDgwGphJ5pc2ZNqwJ3BuRDyRVmyFwO1XPW4/AycPOZM0hcwypSbA62RWC8xIJls9GBEDUg2wjpP0LrB3+UTJrPLGwFsRsUc6kRUOSdOB4RHxQaXy7sCjEdE7lcAKhNuvetx+Br5JVJVExKKIeB+YExEzkrIPcXtujzKg02bKO+LJVturhE29NtnmA43yHEshcvtVj9vPvNqiKiQVRUQZcFpWWTHQOL2oCsYFwFOSZgJzk7KuZLo8z0stqsJyOzBZ0n1sasNdyQz93JZaVIXD7Vc9bj/zsEWukluyTouIzyqVdwMOjoh70oirkCQTTitPtpqcrB237SCpD5klr5Unnb6dXlSFw+1XPW4/c/JgeeeZ2jVHUmsAr/SpGrdf9bj9Gi6P0eco2VBnvKRHJPWQdKekVZImSfJEoW1IZmrPBK4AjkqOXwIzk3O2DZK6SrpP0hLgZWCSpCVJWbd0o6v73H7V4/YzcM9DziQ9B/wGaA5cA1wM3A8cDVwQEf+VYnh1nmdqV5+kF4EbgQfKh3qSOTffJPNv8IA046vr3H7V4/YzcPKQs0obO82KiJ5Z57wj3zYkEyV7R8SGSuWNgbez29M2T9LMLS1p3do5y3D7VY/bz8CrLaqiOOvx9ZXOebXFtnmmdvW9Kmk0cBcV2/BkYEpqURUOt1/1uP3MPQ+5knQW8NfyXfmyynsC50XEBelEVjg8U7t6kl6a06l4i+95wEPAbZvbd8A2cftVj9vPwMmDpcgztc3MCpNXW+RIUlNJF0m6UNKOkk6RNE7SdZKapx1fXeeZ2rUr2VraqsjtVz1uv4bDyUPu7gTaA92BR4ABZFZfCPhjemEVjPuBf5DZunePZIJkR+CfZDbLsuoZmHYABc7tVz1uvwbCwxY5kjQ1IvolNzpaSOZDMJLnr0fEPimHWKd5pnbN8Lbm1eP2qx63n7nnoYqSuyE+Wn5XxORPZ2Lb9qqk0ZIGS+qUHIOT2dueqb0dkm3N7yPT2zUpOQTcK+mSNGMrBG6/6nH7GbjnIWeSbiVzI5TKqy16AHdFxMHpRFYYPFO7+rytefW4/arH7Wfg+zzkLCLO2EL5e5IOyXc8hSYiPiczN8TzQ6qufFvzDyuVe1vz7eP2qx63nzl5qApJg8iMVExO7llwJPBORDyacmgFTdLREfFw2nEUAG9rXj1uv+px+5mHLXIl6XJgOJnE60lgMPAMcBjweESMSjG8gibplxFxedpxFAJva149br/qcfuZk4ccSZoG9AN2ABYBXSLiI0lNgJe92mLbPFPbzKywebVF7jZERGlEfAK8FxEfAUTEp3i8b5s8U9vMrPC55yFHkl4GDo2ITyQVRURZUt4SeMa7am6dZ2qbmRU+9zzk7stJrwPliUOiEZld5WzrymdqV+aZ2mZmBcKrLXIUEeuSyUJERFnyjbkv8EFETEs3uoLgmdpmZgXOwxY5knQMcAuZb8lnAz8F1gJ7AedExEMphlcQPFPbzKywOXnIkaQpZJZqNgFeBwZGxAxJuwEPRsSAVAM0MzOrZR62qIKIWAQgaU5EzEjKPiwfzjAzM6vP/GFXBVlJwmlZZcVA43QiMjMzyx8PW+RI0kBgWkR8Vqm8G3BwRNyTRlxmZmb54uTBzMzMcuJhixxJ6iVpvKRHJPWQdKekVZImSeqddnxmZma1zclD7sYAo4F7gKeBx4BWwFXATSnGZWZmlhcetsiRpCkR0T95PCsiemade823pzYzs/rOPQ+5K856fH2lc15tYWZm9Z6Th9zdLKk5QESMLi+U1BP4d2pRmZmZ5YmHLczMzCwn7nnIkaSmki6SdKGkHSWdImmcpOvKeyTMzMzqM/c85EjSWDK7QTYhsxnWdOB+YATQISJOTDE8MzOzWufkIUeSpkZEP0kCFgIdIyKS569HxD4ph2hmZlarPGxRRZHJuh5N/ix/7kzMzMzqPScPuXsla7VF9sZYPYA1qUVlZmaWJx62qEGSFG5QMzOr50rSDqAQSRpEZqRisqQ+wJHAOxHxaMqhmZmZ1Tr3PORI0uXAcDKJ15PAYOAZ4DDg8YgYlWJ4ZmZmtc7JQ44kTQP6ATsAi4AuEfGRpCbAy15tYWZm9Z0nTOZuQ0SURsQnwHsR8RFARHwKlKUbmpmZWe1z8pC7zyU1TR7vX14oqSVOHszMrAHwsEWOJO0QEes2U96WzA2jpqUQlpmZWd44eagCSUUAEVEmqTHQF/ggIlakG5mZmVnt87BFjiQdQ+a21PMljQQmAr8B3pD036kGZ2ZmlgfueciRpClklmo2AV4HBkbEDEm7AQ9GxIBUAzQzM6tlvklUFUTEIgBJcyJiRlL2YflwhpmZWX3mD7sqyEoSsve2KAYapxORmZlZ/njYIkeSBgLTIuKzSuXdgIMj4p404jIzM8sXJw9mZmaWEw9b5EhSL0njJT0iqYekOyWtkjRJUu+04zMzM6ttTh5yNwYYDdx/4LWyAAAJ4ElEQVQDPA08BrQCrgJuSjEuMzOzvPCwRY4kTYmI/snjWRHRM+vcaxGxX3rRmZmZ1T73POSuOOvx9ZXOebWFmZnVe04ecnezpOYAETG6vFBST+DfqUVlZmaWJx62MDMzs5y45yFHkppKukjShZJ2lHSKpHGSrivvkTAzM6vP3POQI0ljgblk9rbYC5gO3A+MADpExIkphmdmZlbrnDzkSNLUiOgnSWR21+wYEZE8fz0i9kk5RDMzs1rlYYsqikzW9WjyZ/lzZ2JmZlbvOXnI3StZqy2yN8bqAaxJLSozM7M88bBFDZKkcIOamVk9V5J2AIVI0iAyIxWTJfUBjgTeiYhHUw7NzMys1rnnIUeSLgeGk0m8ngQGA88AhwGPR8SoFMMzMzOrdU4eciRpGtAP2AFYBHSJiI8kNQFe9moLMzOr7zxhMncbIqI0Ij4B3ouIjwAi4lOgLN3QzMzMap+Th9x9Lqlp8nj/8kJJLXHyYGZmDYCHLXIkaYeIWLeZ8rZkbhg1LYWwzMzM8sbJQxVIKgKIiDJJjYG+wAcRsSLdyMzMzGqfhy1yJOkYMrelni9pJDAR+A3whqT/TjU4MzOzPHDPQ44kTSGzVLMJ8DowMCJmSNoNeDAiBqQaoJmZWS3zTaKqICIWAUiaExEzkrIPy4czzMzM6jN/2FVBVpKQvbdFMdA4nYjMzMzyx8MWOZI0EJgWEZ9VKu8GHBwR96QRl5mZWb44eTAzM7OceNgiR5J6SRov6RFJPSTdKWmVpEmSeqcdn5mZWW1z8pC7McBo4B7gaeAxoBVwFXBTinGZmZnlhYctciRpSkT0Tx7PioieWedei4j90ovOzMys9rnnIXfFWY+vr3TOqy3MzKzec/KQu5slNQeIiNHlhZJ6Av9OLSozM7M88bCFmZmZ5cQ9DzmS1FTSRZIulLSjpFMkjZN0XXmPhJmZWX3mnoccSRoLzCWzt8VewHTgfmAE0CEiTkwxPDMzs1rn5CFHkqZGRD9JIrO7ZseIiOT56xGxT8ohmpmZ1SoPW1RRZLKuR5M/y587EzMzs3rPyUPuXslabZG9MVYPYE1qUZmZmeWJhy1qkCSFG9TMzOq5krQDKESSegEjgc5J0XxgXERMTy8qMzOz/PCwRY4kXQzcBwiYlBwC7pV0SZqxmZmZ5YOHLXIk6V1g74hYX6m8MfBWROyRTmRmZmb54Z6H3JUBnTZT3jE5Z2ZmVq95zkPuLgCekjSTzM2iALoCPYHzUovKzMwsTzxsUQWSioBBVJwwOTkiStOLyszMLD/c81A1kXWUP/eQhZmZNQjueciRpMOB0cBMMj0OAF3IDFucGxFPpBWbmZlZPjh5yJGk6cDwiPigUnl3Mrer7p1KYGZmZnni1Ra5KwHmbaZ8PtAoz7GYmZnlnec85O52YLKk+9i02mJX4HjgttSiMjMzyxMPW1SBpD7ACL54e+q304vKzMwsP5w8VIOk1gARsSLtWMzMzPLFcx5yJKmrpPskLQFeBiZJWpKUdUs3OjMzs9rn5CF39wP/ADpGxB4R0ZPMran/SWbDLDMzs3rNwxY5kjRzS5tfbe2cmZlZfeHVFrl7VdJo4C4qrrY4GZiSWlRmZmZ54p6HHCVbb58OjGTTaot5wEPAbRGxLq3YzMzM8sHJg5mZmeXEEyZrkKSj047BzMystjl5qFkD0w7AzMystnnYogok9aLinIfyO0xOTy8qMzOz/HDPQ44kXUzmfg4CJiWHgHslXZJmbGZmZvngnoccSXoX2Dsi1lcqbwy85fs8mJlZfeeeh9yVAZ02U94xOWdmZlav+SZRubsAeErSTDbdJKor0BM4L7WozMzM8sTDFlUgqQgYRMUJk5MjojS9qMzMzPLDyYOZmZnlxHMezMzMLCdOHszMzCwnTh7MzMwsJ04erMGQdLyklyV9LGlJ8vhcSUo7tlxIGiapTNJaSWskzZB0atpxbYukKyTdk3YcZlZ9Th6sQZD0Y+B3wG+ADkB74GzgIKBxiqFV1YKIaA60AC4G/iypTy5vIKmglmoXWrxm9ZmTB6v3JLUErgTOjYgHImJNZEyJiO9GxLqk3tckTZH0kaS5kq7Ieo9ukkLSqcm5lZLOljRQ0huSVkm6Kav+KZKel3RDcm62pCFJ+dyk5+PkrPpbvPbWJD/HP4GVQJ/t/BlOlzQHeDop/7ukRZJWS3pO0t5Zr7lT0mhJ45OejucldZB0Y9IG70jqn1W/k6QHJS2V9L6kHyTlRwI/Bb6dvM/r5X83km6TtFDSfEm/klS8mTZcDmxXm5hZ7XPyYA3BgcAOwL+2Ue9j4CRgZ+BrwDmSjqlUZzCwB/Bt4EbgMuCrwN7AtyQNrVT3DaAN8Dcye6IMJHNDse8BN0lqnsO1v0BSkaRjk9dN2873GQr0Bo5Ino9PfqZ2wGvAXyvV/xbwM6AtsA54ManXFngAuL48FuAh4HUy90D5L+ACSUdExGPA1cD9EdE8IvZN3vtOYEPSJv2Bw4EzKrXhbDI9RaO21R5mlh9OHqwhaAssi4gN5QWSXkh6BD6V9GWAiJgQEdMioiwi3gDuJfNBm+2qiPgsIp4g80F9b0QsiYj5wEQyH4Dl3o+IO5Kbh90P7ApcGRHrktd/TuZDc3uvna2TpFXAMuBy4MSImLGd73NFRHwcEZ8m17496Y1ZR+bb/b5Jb025f0TEqxHxGfAP4LOIuDvr5yr/mQcCu0TElRHxeUTMBv4MHL+5H0BSe+Ao4IIkniXADZXqL4iIP0TEhvJ4zSx9HkO0hmA50FZSSXkCERFDACTNI0miJQ0GrgH6kpkHsQPw90rvtTjr8aebed58K3WJiM3W385rZ1sQEV0qF27n+8zNql9M5hv9N4Fd2LQ/S1tg9RZ+ji39zLuxKakpV0wmqdqc3YBGwMKsOatF2fFVemxmdYR7HqwheJFMd/vIbdT7GzAO2DUiWgJ/IrPdej7U1LW3532ybyt7Apl2+SrQEuiWlFfl2nPJ9LbsnHXsFBFHbea65fXXAW2z6reIiL2z6vgWuGZ1kJMHq/ciYhXwS2C0pOMk7ZTMFegHNMuquhOwIiI+kzSIzAdrvtTUtXN9n53IfIAvB5qSmZdQVZOANZIultREUrGkvpIGJucXA92SuRFExELgCeD/JLVI/k56VJo3YmZ1kJMHaxAi4jrgR8BFZD7EFgO3kFnm+EJS7VzgSklrgF8AY/MYYk1dO9f3uRv4kMzmbm8DL1XxuiRzII4G+gHvk5mPcSuZHg3YNHyyXNJryeOTyAyvvE1mxcgDZLa3N7M6zBtjmZmZWU7c82BmZmY5cfJgZmZmOXHyYGZmZjlx8mBmZmY5cfJgZmZmOXHyYGZmZjlx8mBmZmY5cfJgZmZmOfn/rQVZqLsSAPsAAAAASUVORK5CYII=\n",
            "text/plain": [
              "<Figure size 648x432 with 2 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Beta1kA0saS9",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 68
        },
        "outputId": "5227046b-1fcb-4a96-faa8-459c9ffe0e4c"
      },
      "source": [
        "# Print best parameters and best score\n",
        "print(\"Best parameters: {}\".format(grid_search.best_params_))\n",
        "print(\"Best cross-validation score: {:.2f}\".format(grid_search.best_score_))\n",
        "\n",
        "### Best parameters: {'C': 100, 'gamma': 0.0001}\n",
        "### Best cross-validation score: 1.00\n",
        "\n",
        "# Plug best parameters into the model, train it, and test on held out data\n",
        "best_svc = SVC(**grid_search.best_params_).fit(X_train, y_train)\n",
        "best_svc.score(X_test, y_test)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Best parameters: {'C': 10000, 'gamma': 0.003}\n",
            "Best cross-validation score: 0.61\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0.605"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 60
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "moYVH8CDsz6Z",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 119
        },
        "outputId": "1c1d6738-ae0e-4dae-cc2c-b583847df088"
      },
      "source": [
        "from sklearn.metrics import confusion_matrix\n",
        "\n",
        "y_pred = best_svc.predict(X_test)\n",
        "\n",
        "print(confusion_matrix(y_test, y_pred))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[[  0   0   1   0   0   0]\n",
            " [  1   0  11   1   0   0]\n",
            " [  3   1 125  32   3   0]\n",
            " [  1   4  51  99  12   2]\n",
            " [  0   0   2  27  18   1]\n",
            " [  0   0   0   2   3   0]]\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "iuhCPa3BvF4i",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 238
        },
        "outputId": "a388c826-52f5-42a5-b120-b997707cee6f"
      },
      "source": [
        "from sklearn.metrics import classification_report\n",
        "\n",
        "print(classification_report(y_test, y_pred))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "              precision    recall  f1-score   support\n",
            "\n",
            "           3       0.00      0.00      0.00         1\n",
            "           4       0.00      0.00      0.00        13\n",
            "           5       0.66      0.76      0.71       164\n",
            "           6       0.61      0.59      0.60       169\n",
            "           7       0.50      0.38      0.43        48\n",
            "           8       0.00      0.00      0.00         5\n",
            "\n",
            "    accuracy                           0.60       400\n",
            "   macro avg       0.30      0.29      0.29       400\n",
            "weighted avg       0.59      0.60      0.59       400\n",
            "\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "gesHMgo86W8I",
        "colab_type": "text"
      },
      "source": [
        "# Recommended books"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "AYP7fj4B6Z0P",
        "colab_type": "text"
      },
      "source": [
        "![alt text](https://images-na.ssl-images-amazon.com/images/I/51%2BkYprYK1L._SX379_BO1,204,203,200_.jpg)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ZB9PuoBw6kYS",
        "colab_type": "text"
      },
      "source": [
        "![alt text](https://images-na.ssl-images-amazon.com/images/I/41LEirGUWAL._SX404_BO1,204,203,200_.jpg)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-jsraWQF6yBy",
        "colab_type": "text"
      },
      "source": [
        "![alt text](https://images-na.ssl-images-amazon.com/images/I/51oSnWCYsxL._SX378_BO1,204,203,200_.jpg)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "i0K5rt4VHEmK",
        "colab_type": "text"
      },
      "source": [
        "![alt text](https://images-na.ssl-images-amazon.com/images/I/511MLXf56iL._SX398_BO1,204,203,200_.jpg)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "IGn5YpO78J0h",
        "colab_type": "text"
      },
      "source": [
        "![The Elements of Statistical Learning](https://images-na.ssl-images-amazon.com/images/I/41aQrQaPseL._SX331_BO1,204,203,200_.jpg)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "tyXh3j-27Eu9",
        "colab_type": "text"
      },
      "source": [
        "![alt text](https://images-na.ssl-images-amazon.com/images/I/514G9tMBXHL.jpg)"
      ]
    }
  ]
}