6.4-sequence-processing-with-convnets.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Using TensorFlow backend.\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "'2.2.2'"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import keras\n",
    "keras.__version__"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 컨브넷을 사용한 시퀀스 처리\n",
    "\n",
    "이 노트북은 [케라스 창시자에게 배우는 딥러닝](https://tensorflow.blog/%EC%BC%80%EB%9D%BC%EC%8A%A4-%EB%94%A5%EB%9F%AC%EB%8B%9D/) 책의 6장 4절의 코드 예제입니다. 책에는 더 많은 내용과 그림이 있습니다. 이 노트북에는 소스 코드에 관련된 설명만 포함합니다.\n",
    "\n",
    "\n",
    "## 1D 컨브넷 구현\n",
    "\n",
    "케라스에서 1D 컨브넷은 `Conv1D` 층을 사용하여 구현합니다. `Conv1D`는 `Conv2D`와 인터페이스가 비슷합니다. `(samples, time, features)` 크기의 3D 텐서를 입력받고 비슷한 형태의 3D 텐서를 반환합니다. 합성곱 윈도우는 시간 축의 1D 윈도우입니다. 즉, 입력 텐서의 두 번째 축입니다.\n",
    "\n",
    "간단한 두 개 층으로 된 1D 컨브넷을 만들어 익숙한 IMDB 감성 분류 문제에 적용해 보죠.\n",
    "\n",
    "기억을 되살리기 위해 데이터를 로드하고 전처리하는 코드를 다시 보겠습니다:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "데이터 로드...\n",
      "25000 훈련 시퀀스\n",
      "25000 테스트 시퀀스\n",
      "시퀀스 패딩 (samples x time)\n",
      "x_train 크기: (25000, 500)\n",
      "x_test 크기: (25000, 500)\n"
     ]
    }
   ],
   "source": [
    "from keras.datasets import imdb\n",
    "from keras.preprocessing import sequence\n",
    "\n",
    "max_features = 10000  # 특성으로 사용할 단어의 수\n",
    "max_len = 500  # 사용할 텍스트의 길이(가장 빈번한 max_features 개의 단어만 사용합니다)\n",
    "\n",
    "print('데이터 로드...')\n",
    "(x_train, y_train), (x_test, y_test) = imdb.load_data(num_words=max_features)\n",
    "print(len(x_train), '훈련 시퀀스')\n",
    "print(len(x_test), '테스트 시퀀스')\n",
    "\n",
    "print('시퀀스 패딩 (samples x time)')\n",
    "x_train = sequence.pad_sequences(x_train, maxlen=max_len)\n",
    "x_test = sequence.pad_sequences(x_test, maxlen=max_len)\n",
    "print('x_train 크기:', x_train.shape)\n",
    "print('x_test 크기:', x_test.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "1D 컨브넷은 5장에서 사용한 2D 컨브넷과 비슷한 방식으로 구성합니다. `Conv1D`와 `MaxPooling1D` 층을 쌓고 전역 풀링 층이나 `Flatten` 층으로 마칩니다. 이 구조는 3D 입력을 2D 출력으로 바꾸므로 분류나 회귀를 위해 모델에 하나 이상의 `Dense` 층을 추가할 수 있습니다.\n",
    "\n",
    "한 가지 다른 점은 1D 컨브넷에 큰 합성곱 윈도우를 사용할 수 있다는 것입니다. 2D 합성곱 층에서 3 × 3 합성곱 윈도우는 3 × 3 = 9 특성을 고려합니다. 하지만 1D 합성곱 층에서 크기 3인 합성곱 윈도우는 3개의 특성만 고려합니다. 그래서 1D 합성곱에 크기 7이나 9의 윈도우를 사용할 수 있습니다.\n",
    "\n",
    "다음은 IMDB 데이터셋을 위한 1D 컨브넷의 예입니다:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "embedding_1 (Embedding)      (None, 500, 128)          1280000   \n",
      "_________________________________________________________________\n",
      "conv1d_1 (Conv1D)            (None, 494, 32)           28704     \n",
      "_________________________________________________________________\n",
      "max_pooling1d_1 (MaxPooling1 (None, 98, 32)            0         \n",
      "_________________________________________________________________\n",
      "conv1d_2 (Conv1D)            (None, 92, 32)            7200      \n",
      "_________________________________________________________________\n",
      "global_max_pooling1d_1 (Glob (None, 32)                0         \n",
      "_________________________________________________________________\n",
      "dense_1 (Dense)              (None, 1)                 33        \n",
      "=================================================================\n",
      "Total params: 1,315,937\n",
      "Trainable params: 1,315,937\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n",
      "Train on 20000 samples, validate on 5000 samples\n",
      "Epoch 1/10\n",
      "20000/20000 [==============================] - 3s 165us/step - loss: 0.8337 - acc: 0.5088 - val_loss: 0.6875 - val_acc: 0.5638\n",
      "Epoch 2/10\n",
      "20000/20000 [==============================] - 2s 96us/step - loss: 0.6700 - acc: 0.6399 - val_loss: 0.6642 - val_acc: 0.6582\n",
      "Epoch 3/10\n",
      "20000/20000 [==============================] - 2s 102us/step - loss: 0.6235 - acc: 0.7547 - val_loss: 0.6076 - val_acc: 0.7436\n",
      "Epoch 4/10\n",
      "20000/20000 [==============================] - 2s 102us/step - loss: 0.5253 - acc: 0.8096 - val_loss: 0.4848 - val_acc: 0.8062\n",
      "Epoch 5/10\n",
      "20000/20000 [==============================] - 2s 102us/step - loss: 0.4114 - acc: 0.8485 - val_loss: 0.4244 - val_acc: 0.8316\n",
      "Epoch 6/10\n",
      "20000/20000 [==============================] - 2s 103us/step - loss: 0.3486 - acc: 0.8666 - val_loss: 0.4164 - val_acc: 0.8376\n",
      "Epoch 7/10\n",
      "20000/20000 [==============================] - 2s 101us/step - loss: 0.3108 - acc: 0.8640 - val_loss: 0.4502 - val_acc: 0.8200\n",
      "Epoch 8/10\n",
      "20000/20000 [==============================] - 2s 100us/step - loss: 0.2802 - acc: 0.8519 - val_loss: 0.4321 - val_acc: 0.8038\n",
      "Epoch 9/10\n",
      "20000/20000 [==============================] - 2s 97us/step - loss: 0.2531 - acc: 0.8361 - val_loss: 0.4445 - val_acc: 0.7868\n",
      "Epoch 10/10\n",
      "20000/20000 [==============================] - 2s 100us/step - loss: 0.2307 - acc: 0.8103 - val_loss: 0.5007 - val_acc: 0.7538\n"
     ]
    }
   ],
   "source": [
    "from keras.models import Sequential\n",
    "from keras import layers\n",
    "from keras.optimizers import RMSprop\n",
    "\n",
    "model = Sequential()\n",
    "model.add(layers.Embedding(max_features, 128, input_length=max_len))\n",
    "model.add(layers.Conv1D(32, 7, activation='relu'))\n",
    "model.add(layers.MaxPooling1D(5))\n",
    "model.add(layers.Conv1D(32, 7, activation='relu'))\n",
    "model.add(layers.GlobalMaxPooling1D())\n",
    "model.add(layers.Dense(1))\n",
    "\n",
    "model.summary()\n",
    "\n",
    "model.compile(optimizer=RMSprop(lr=1e-4),\n",
    "              loss='binary_crossentropy',\n",
    "              metrics=['acc'])\n",
    "history = model.fit(x_train, y_train,\n",
    "                    epochs=10,\n",
    "                    batch_size=128,\n",
    "                    validation_split=0.2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "그림 6-27과 6-28은 훈련과 검증 결과를 보여줍니다. 검증 정확도는 LSTM보다 조금 낮지만 CPU나 GPU에서 더 빠르게 실행됩니다(속도 향상은 환경에 따라 많이 다릅니다). 여기에서 적절한 에포크 수(4개)로 모델을 다시 훈련하고 테스트 세트에서 확인할 수 있습니다. 이 예는 단어 수준의 감성 분류 작업에 순환 네트워크를 대신하여 빠르고 경제적인 1D 컨브넷을 사용할 수 있음을 보여줍니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAAEYCAYAAAAJeGK1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3Xl8VNX5x/HPQwCRVWRRy5KgoqIIGCOooOBGoT8FxQ1Ef+ICakWtS1sqtFAVtVrcqrWiov40gqhVoRVxA3EtBBAUrIqAGkGMgLigQuD5/XEmMIQsE0hyZybf9+s1r8y999w7z9xk8sw599xzzN0RERFJNrWiDkBERKQkSlAiIpKUlKBERCQpKUGJiEhSUoISEZGkpAQlIiJJSQlKKoWZZZjZ92bWtjLLRsnM9jWzSr8Pw8yON7PlccsfmtlRiZTdgdd6wMyu3dH9RaJUO+oAJBpm9n3cYn3gZ2BTbPkid8+tyPHcfRPQsLLL1gTuvn9lHMfMLgTOdvdecce+sDKOLRIFJagayt23JIjYN/QL3f3l0sqbWW13L6yO2ETKo7/HmkFNfFIiM7vBzJ4ws4lm9h1wtpkdYWbvmNk3ZrbSzO4yszqx8rXNzM0sK7b8WGz7NDP7zszeNrN2FS0b297XzD4ys3Vm9jcze9PMhpQSdyIxXmRmS8xsrZndFbdvhpndbmarzewToE8Z52eUmU0qtu4eM7st9vxCM/sg9n4+idVuSjtWvpn1ij2vb2aPxmJbBBxawusujR13kZn1i60/GLgbOCrWfPp13LkdE7f/xbH3vtrMnjWzvRI5NxU5z0XxmNnLZrbGzL40s9/Fvc4fY+fkWzPLM7NflNScamZvFP2eY+dzVux11gCjzKy9mc2IvZevY+etSdz+mbH3WBDbfqeZ1YvF3CGu3F5mtt7MmpX2fiUi7q5HDX8Ay4Hji627AdgAnET4IrMrcBjQjVDz3hv4CBgeK18bcCArtvwY8DWQA9QBngAe24GyLYHvgP6xbVcBG4EhpbyXRGJ8DmgCZAFrit47MBxYBLQGmgGzwkekxNfZG/geaBB37K+AnNjySbEyBhwL/Ah0im07Hlged6x8oFfs+V+BmUBTIBNYXKzsGcBesd/JWbEY9ohtuxCYWSzOx4Axsee9YzF2AeoBfwdeTeTcVPA8NwFWAVcAuwCNga6xbX8AFgDtY++hC7A7sG/xcw28UfR7jr23QuASIIPw97gfcBxQN/Z38ibw17j3837sfDaIle8e2zYeGBv3OlcDz0T9OdSjhL+zqAPQI/oHpSeoV8vZ7xrgydjzkpLOP+LK9gPe34Gy5wOvx20zYCWlJKgEYzw8bvs/gWtiz2cRmjqLtv2q+D/NYsd+Bzgr9rwv8FEZZf8FXBp7XlaC+iz+dwH8Or5sCcd9H/if2PPyEtQjwI1x2xoTrju2Lu/cVPA8nwPklVLuk6J4i61PJEEtLSeG04A5sedHAV8CGSWU6w4sAyy2/C4woLI/V3rs/ENNfFKWz+MXzOwAM/t3rMnmW+A6oHkZ+38Z93w9ZXeMKK3sL+Lj8PAfJb+0gyQYY0KvBXxaRrwAjwODYs/PArZ0LDGzE83sP7Emrm8ItZeyzlWRvcqKwcyGmNmCWDPVN8ABCR4Xwvvbcjx3/xZYC7SKK5PQ76yc89wGWFJKDG0ISWpHFP973NPMJpvZF7EYHi4Ww3IPHXK24e5vEmpjPcysI9AW+PcOxiRVSAlKylK8i/V9hG/s+7p7Y+BPhBpNVVpJ+IYPgJkZ2/5DLW5nYlxJ+MdWpLxu8E8Ax5tZa0IT5OOxGHcFngJuIjS/7Qa8mGAcX5YWg5ntDdxLaOZqFjvuf+OOW16X+BWEZsOi4zUiNCV+kUBcxZV1nj8H9illv9K2/RCLqX7cuj2LlSn+/v5C6H16cCyGIcViyDSzjFLi+D/gbEJtb7K7/1xKOYmQEpRURCNgHfBD7CLzRdXwmv8Css3sJDOrTbiu0aKKYpwM/MbMWsUumP++rMLuvorQDPUQ8KG7fxzbtAvhukgBsMnMTiRcK0k0hmvNbDcL94kNj9vWkPBPuoCQqy8k1KCKrAJax3dWKGYicIGZdTKzXQgJ9HV3L7VGWoayzvMUoK2ZDTezumbW2My6xrY9ANxgZvtY0MXMdick5i8JnXEyzGwYccm0jBh+ANaZWRtCM2ORt4HVwI0WOp7sambd47Y/SmgSPIuQrCQJKUFJRVwNnEvotHAfoQZRpWJJ4EzgNsI/nH2A+YRvzpUd473AK8B7wBxCLag8jxOuKT0eF/M3wJXAM4SOBqcREm0iRhNqcsuBacT983T3hcBdwOxYmQOA/8Tt+xLwMbDKzOKb6or2f4HQFPdMbP+2wOAE4yqu1PPs7uuAE4BTCZ0yPgJ6xjbfCjxLOM/fEjos1Is13Q4FriV0mNm32HsryWigKyFRTgGejouhEDgR6ECoTX1G+D0UbV9O+D1vcPe3KvjepZoUXSQUSQmxJpsVwGnu/nrU8UjqMrP/I3S8GBN1LFIy3agrSc/M+hCabH4idFMuJNQiRHZI7Hpef+DgqGOR0qmJT1JBD2ApoemnD3CyLmrLjjKzmwj3Yt3o7p9FHY+UTk18IiKSlFSDEhGRpJR016CaN2/uWVlZUYchIiJVZO7cuV+7e1m3iwBJmKCysrLIy8uLOgwREakiZlbeKC2AmvhERCRJKUGJiEhSUoISEZGklHTXoEqyceNG8vPz+emnn6IORcpQr149WrduTZ06pQ0FJyKSuJRIUPn5+TRq1IisrCzCYNaSbNyd1atXk5+fT7t27crfQUSkHCnRxPfTTz/RrFkzJackZmY0a9ZMtVwRqTQpkaAAJacUoN9RdHJzISsLatUKP3Nzy9tDJPmlRBOfiJQuNxeGDYP168Pyp5+GZYDBOzqZhkgSSJkaVJRWr15Nly5d6NKlC3vuuSetWrXasrxhw4aEjnHeeefx4YcfllnmnnvuIVdffaWCRo7cmpyKrF8f1kdNNTvZGUk3WGxOTo4XH0nigw8+oEOHDgkfIzc3fDg/+wzatoWxYyvvm+SYMWNo2LAh11xzzTbr3R13p1atmp3zK/q7kp1XqxaU9DE2g82bqz+eIsVrdgD168P48arZ1XRmNtfdc8orl3b/TYs+FJ9+Gj60Rc0dVfHNbcmSJXTs2JGLL76Y7OxsVq5cybBhw8jJyeGggw7iuuuu21K2R48evPvuuxQWFrLbbrsxYsQIOnfuzBFHHMFXX30FwKhRo7jjjju2lB8xYgRdu3Zl//335623wqSfP/zwA6eeeiqdO3dm0KBB5OTk8O67724X2+jRoznssMO2xFf0ReSjjz7i2GOPpXPnzmRnZ7N8+XIAbrzxRg4++GA6d+7MyGT46i0Ja9u2YuurSzLX7CQ1pF2Cqu4PxeLFi7nggguYP38+rVq14uabbyYvL48FCxbw0ksvsXjx4u32WbduHT179mTBggUcccQRTJgwocRjuzuzZ8/m1ltv3ZLs/va3v7HnnnuyYMECRowYwfz580vc94orrmDOnDm89957rFu3jhdeeAGAQYMGceWVV7JgwQLeeustWrZsydSpU5k2bRqzZ89mwYIFXH311ZV0dqQ6jB0baibx6tcP66P0WSkzLZW2XqS4tEtQ1f2h2GeffTjssMO2LE+cOJHs7Gyys7P54IMPSkxQu+66K3379gXg0EMP3VKLKW7AgAHblXnjjTcYOHAgAJ07d+aggw4qcd9XXnmFrl270rlzZ1577TUWLVrE2rVr+frrrznppJOAcGNt/fr1efnllzn//PPZddddAdh9990rfiIkMoMHh2azzMzQrJeZmRzNaMlas5PUkXa9+Nq2Dc16Ja2vCg0aNNjy/OOPP+bOO+9k9uzZ7Lbbbpx99tkl3hdUt27dLc8zMjIoLCws8di77LLLdmUSuWa4fv16hg8fzrx582jVqhWjRo3aEkdJXcHdXV3EU9zgwdEnpOLGji35GlTUNTtJHWlXg4qyuePbb7+lUaNGNG7cmJUrVzJ9+vRKf40ePXowefJkAN57770Sa2g//vgjtWrVonnz5nz33Xc8/fTTADRt2pTmzZszdepUINwAvX79enr37s2DDz7Ijz/+CMCaNWsqPW6peZK1ZiepI+0SVJQfiuzsbA488EA6duzI0KFD6d69e6W/xmWXXcYXX3xBp06dGDduHB07dqRJkybblGnWrBnnnnsuHTt25JRTTqFbt25btuXm5jJu3Dg6depEjx49KCgo4MQTT6RPnz7k5OTQpUsXbr/99kqPW2qmwYNh+fLQm3D58uRKTuoCn/wS6mZuZn2AO4EM4AF3v7nY9rbAI8BusTIj3P15M8sCPgCKbgB6x90vLuu1KqObeTorLCyksLCQevXq8fHHH9O7d28+/vhjatdOjtZa/a4kFagLfLQqrZu5mWUA9wB9gQOBQWZ2YLFio4DJ7n4IMBD4e9y2T9y9S+xRZnKS8n3//fd0796dzp07c+qpp3LfffclTXKqCfStOz2oC3xqSOQ/W1dgibsvBTCzSUB/IP7ihwONY8+bACsqM0jZarfddmPu3LlRh1EjaUih9KEu8KkhkWtQrYDP45bzY+vijQHONrN84Hngsrht7cxsvpm9ZmZHlfQCZjbMzPLMLK+goCDx6EWqkb51pw91gU8NiSSokvofF79wNQh42N1bA78CHjWzWsBKoG2s6e8q4HEza1xsX9x9vLvnuHtOixYtKvYORKqJvnWnj2S9uVm2lUiCygfaxC23ZvsmvAuAyQDu/jZQD2ju7j+7++rY+rnAJ8B+Oxu0SBT0rTt9qAt8akgkQc0B2ptZOzOrS+gEMaVYmc+A4wDMrAMhQRWYWYtYJwvMbG+gPbC0soIXqU761p1ekrkLvATlJih3LwSGA9MJXcYnu/siM7vOzPrFil0NDDWzBcBEYIiH/utHAwtj658CLnb3lLsLtFevXtvddHvHHXfw61//usz9GjZsCMCKFSs47bTTSj128W71xd1xxx2sj7v48atf/YpvvvkmkdClEulbt0g1K5omIlkehx56qBe3ePHi7dZVp3/84x8+ZMiQbdZ169bNZ82aVeZ+DRo0KPfYPXv29Dlz5pRZJjMz0wsKCsoPNAlE/bsSSXWPPeaemeluFn4+9ljUEVU+IM8TyAdpN5JEVTjttNP417/+xc8//wzA8uXLWbFiBT169OD777/nuOOOIzs7m4MPPpjnnntuu/2XL19Ox44dgTAM0cCBA+nUqRNnnnnmluGFAC655JItU3WMHj0agLvuuosVK1ZwzDHHcMwxxwCQlZXF119/DcBtt91Gx44d6dix45apOpYvX06HDh0YOnQoBx10EL17997mdYpMnTqVbt26ccghh3D88cezatUqINxrdd5553HwwQfTqVOnLUMlvfDCC2RnZ9O5c2eOO+64Sjm3smMKC2HlSpg/H6ZNg4cegptugjFj4JVXoIQhICUFVOd0Qakg5e7w/M1voITpj3ZKly4Q+99eombNmtG1a1deeOEF+vfvz6RJkzjzzDMxM+rVq8czzzxD48aN+frrrzn88MPp169fqYOv3nvvvdSvX5+FCxeycOFCsrOzt2wbO3Ysu+++O5s2beK4445j4cKFXH755dx2223MmDGD5s2bb3OsuXPn8tBDD/Gf//wHd6dbt2707NmTpk2b8vHHHzNx4kTuv/9+zjjjDJ5++mnOPvvsbfbv0aMH77zzDmbGAw88wC233MK4ceO4/vrradKkCe+99x4Aa9eupaCggKFDhzJr1izatWun8fqqgDusWQNffhkeq1ZtfV788fXXpU9S6A677go9e0Lv3vDLX0KHDmGbJLeybmWoiU3JKZegojJo0CAmTZq0JUEVzeHk7lx77bXMmjWLWrVq8cUXX7Bq1Sr23HPPEo8za9YsLr/8cgA6depEp06dtmybPHky48ePp7CwkJUrV7J48eJtthf3xhtvcMopp2wZUX3AgAG8/vrr9OvXj3bt2tGlSxeg9Ck98vPzOfPMM1m5ciUbNmygXbt2ALz88stMmjRpS7mmTZsydepUjj766C1lNCVH4r7/vuQkUzwBrVoFGzduv/8uu8Cee4bH3nvDkUfCHntsXVf02GMP2LQJXnsNXnwxPK66KhyjVauQrHr3huOPh2LfdSRJ6FaGbaVcgiqrplOVTj75ZK666irmzZvHjz/+uKXmk5ubS0FBAXPnzqVOnTpkZWWVOMVGvJJqV8uWLeOvf/0rc+bMoWnTpgwZMqTc43gZ4ygWTdUBYbqOkpr4LrvsMq666ir69evHzJkzGTNmzJbjFo+xpHU12YYNZddw4rf98MP2+9eqBS1bbk0uHTtun2yKnjdpUrHaz4knhgeEJqKXXgrJ6tlnQ1OgGWRnb01YRx4JcTPASISqe7qgZJdyCSoqDRs2pFevXpx//vkMGjRoy/p169bRsmVL6tSpw4wZM/i0pL+uOEcffTS5ubkcc8wxvP/++yxcuBAIU3U0aNCAJk2asGrVKqZNm0avXr0AaNSoEd999912TXxHH300Q4YMYcSIEbg7zzzzDI8++mjC72ndunW0ahUGBXnkkUe2rO/duzd33333lmtaa9eu5YgjjuDSSy9l2bJlW5r4alot6ocfYPJkuP9+ePvtksvsvvvW5NK16/a1nKLk07w5ZGRUfcyZmXDhheGxaRPMnbu1dnXrreG6VYMG0KvX1oS1//5qDoyK5tDalhJUBQwaNIgBAwZs0/w1ePBgTjrppC1TVRxwwAFlHuOSSy7hvPPOo1OnTnTp0oWuXbsCYXbcQw45hIMOOoi99957m6k6hg0bRt++fdlrr72YMWPGlvXZ2dkMGTJkyzEuvPBCDjnkkFJn6C1uzJgxnH766bRq1YrDDz+cZcuWATBq1CguvfRSOnbsSEZGBqNHj2bAgAGMHz+eAQMGsHnzZlq2bMlLL72U0OukunnzQlJ6/HH49tvwD3zUqPCtNj7xtGwZmuOSVUZGSJpdu4b4v/0WZs4MyWr6dPj3v0O5Nm22JqvjjoNmzSINu0Ypus40cmRo1mvbNiSnmnj9CRKcbqM6abqN1JYuv6tvvw0J6f77Q4KqVw9OOw2GDoWjjkrPGsbSpVubA195BdatC+8zJ2drwjr8cDUHys6rtOk2RGoKd3jnHbjgAthrL7jkktBp4a67YMUKePRROPro9ExOEDpgXHQRPP106CX41lswejTUqQM33xx6BTZrBv36wd13w0cfldyTUKSyqIlParw1a+Cxx0Jt6f33wzWZQYNCbalr1/RNSGWpXRuOOCI8Ro+Gb76BGTO2NgdOnRrKZWZu2xzYtGm0cUt6SZkEpV5kyS/ZmovL4g6zZoWk9NRT8PPPoSnrvvtg4EBovN2Y+zXbbrvBKaeEB8Ann2ztbDFpUjiPtWqFhF6UsLp2DbUvkR2VEtegli1bRqNGjWjWrJmSVJJyd1avXs1333235V6pZPTVV/DII/DAA6GJqnFjOPvsUFuK3TYmFbRxI8yevTVhzZ4dBmBt3BiOPTYkqxNOgH32qZm1UdleotegUiJBbdy4kfz8/HLvC5Jo1atXj9atW1Mnyb42b94cLvrff3+4F2jjRujePSSl00/ffoRy2Tlr18Krr25tDiy686JNm9Cd/ZhjwiMrK8ooJUpplaBEdsSKFeHG1AcfhGXLwj1K//u/ITEdeGDU0dUM7rBkSegdOGNG6NYeG0aSzMytCatXr7AsNYMSlNRIhYXwwguhtvTvf4ebU485JiSlU04J3cUlOps3w+LFIVHNmBGGZVq9Omxr127bhNWmTRkHkpSmBCU1yqefhprShAnwxRdhtIYhQ8IICvvuG3V0UprNm0PPyfiEtXZt2LbPPtsmrNigJxKR3NzKu4FYCUpSWiIfho0bYcqUUFt68cWw7pe/DLWlk05SD7JUtHkzLFwYEtbMmSFhFc3N2b59SFRFj1/8IrIwa5yiaUCKD8G0oxN2KkFJyirvw7BkSeiF99BDoVde69Zw/vnhoesY6WXTppCwiq5fzZoVRrgA2G+/rbWrXr3CcFNSNbKySh7ENjMTEhxZbRtKUJKySvswNG8OBx8c/lllZIQRu4cOhT59qmfgVYnepk1hPrj4hPXdd2HbAQdsm7Batoww0DRTq1bp849t3lzx4ylBScoq7cMA4UL6hReG60tq4pHCwjCrcFHCev31MP8WhJ6aRQmrZ09o0SLKSFObalAxSlBS2oehZcswzXktjSAppSgsDFOKFHW6eOONrfNxdey4tdPF0Udr0saKiOoaVEIfdTPrY2YfmtkSMxtRwva2ZjbDzOab2UIz+1Xctj/E9vvQzH5ZsbchNdGYMds32dWvD7fdpuQkZatdG7p1g9//PtxusHZtGPT2xhtDjXvCBDj11FCb6twZrrgidLQpuq4lJRs8OCSjzMzQrJeZuePJqSLKrUGZWQbwEXACkA/MAQa5++K4MuOB+e5+r5kdCDzv7lmx5xOBrsAvgJeB/dx9U2mvpxpUzfbjj+F+penTw421a9dqThypPBs2QF7e1ibBN98Mf3MZGXDYYXD88WHQ2yOOSO65vVJdZdagugJL3H2pu28AJgH9i5VxoGh4zSbAitjz/sAkd//Z3ZcBS2LHE9nODz+Ejg8vvhh66a1eHS7ALl+u5CSVo27dMMX9yJFhdIu1a0Oy+sMfwvYbbwxNgE2bhs43f/1ruMa1Ix0BZOclMpp5K+DzuOV8oFuxMmOAF83sMqABcHzcvu8U23e72+3MbBgwDKBt27aJxC1p5rvv4H/+J3yjfeQROOecqCOSmmCXXbb2+rv++tDU99pr8PLL4fHb34ZyzZqFgW+PPz489t47yqhrjkQSVEnjDxdvFxwEPOzu48zsCOBRM+uY4L64+3hgPIQmvgRikjSybh307RtGwX78cTjzzKgjkpqqSZMwIWO/fmF5xYow0PDLL4efTz4Z1mdlbU1Wxx6rHoJVJZEElQ/Ej4rVmq1NeEUuAPoAuPvbZlYPaJ7gvlKDrVkTRn9YsCB8+IvmGxJJBr/4RajNn3NOuPXhww+3JqwnnwxN0RA6XBRdvzrqKGjYMNq400UinSRqEzpJHAd8QegkcZa7L4orMw14wt0fNrMOwCuEprwDgcfZ2kniFaC9OkkIhFGtTzghDB769NPh+pNIqigshHnztjYHvvlm6IRRp07oZHHccSFpHXaYht0qrlLvg4p1G78DyAAmuPtYM7sOyHP3KbHeevcDDQlNeL9z9xdj+44EzgcKgd+4+7SyXksJqmZYtSp8gD/5BJ57LkxqJ5LK1q8PSaqoOXDevFDratQo3Chc1CR44IGauFE36krSWrEiJKfPPoOpU0Mbvki6Wb069BAsahJcsiSs33PP8PdfVMOqidOKKEFJUvr885CQvvwSnn8+tNeL1ASffrpth4uvvgrr99tva7Iq6uKe7pSgJOksWxaS09q14S7/ww+POiKRaLiHebCKktXMmeE+QDM49NCQrM45J31nflaCkqTy8cchOf3wQ7hB8tBDo45IJHls2BBusyiqYb3zThi5/ayzYPToMBdWOqnUsfhEdsYHH4SLxD/9FNrklZxEtlW3LvToEZLR66+HQZF/+1v45z+hQ4cw19mOjBqe6pSgpEq9/364S3/z5tCM0blz1BGJJL/mzeEvf4GlS2H48HAD+377wSWXQH5+1NFVHyUoqTLz54fkVLt2GD7moIOijkgktey5J9xxR+gBeMEF8OCDsO++8JvfhI5G6U4JSqrEnDnhmlODBmHW0/33jzoikdTVujXcey989FEYOPnuu8N4gL/7XbjhPV0pQUmle+ut0AupadOQnPbZJ+qIRNJDVlaoRX3wAQwYEEZbb9cORo0KvWPTjRKUVKpZs8KoEHvsEZ5nZkYdkUj6ad8eHnssXOPt2zfMl9auXRiR/dtvo46u8ihBSaV55ZUwh07btuGaU+vWUUckkt4OPBAmTw7Xe3v2hD/9KSSqW27ZOtV9KlOCkkrxwgthsNd99w299fbaK+qIRGqOLl3CmJazZ0PXrmHK+733Dh0sfvop6uh2nBKU7LSpU6F//3C/xowZ0LJl1BGJ1EyHHQbTpsEbb4Res1deGb403ntvuBk41ShByU55+ulwsbZLl9DE16xZ1BGJSPfu8Oqr4TOZmQm//nW4j2rChDBNSKpQgpIdNnFimP22a9cwfFFNGORSJJUce2yoTU2bFmb9veCC0NLx2GNhKKVkpwQlO+SRR8L9GD16wPTp0Lhx1BGJSEnMQuel2bPh2Wehfv0wEO3BB4dZgTdvjjrC0ilBSYXdfz+cd16YIuD55zW9tUgqMAvXiufPhyeeCCOqn3EGZGfDlClhOdkoQUmF3HMPDBsWvpFNnRq+jYlI6qhVKySm99+HRx8N3dH794du3UJv3GRKVEpQkrDbbgsDV/bvD888A/XqRR2RiOyojAw4++wwKsWDD4YJFPv2DZOIzpgRdXSBElQNl5sbhk+pVSv8zM0tudxNN8HVV8Ppp4d26112qc4oRaSq1K4dpvP46CP4+9+3Tix67LHw5pvRxqYEVYPl5obmuk8/DdX6Tz8Ny/FJyh3+/Ge49trQKeLxx6FOnehiFpGqUbdumM5jyRK4/XZYtCh0gurbNwz+HAUlqBps5EhYv37bdevXh/UQktPIkTBmDAwZEnru1a5d3VGKSHXaddcwncfSpWFOqqLRKfr3hwULqjeWhBKUmfUxsw/NbImZjShh++1m9m7s8ZGZfRO3bVPctimVGbzsnM8+K329O1xzTWjau+ii0EadkVG98YlIdBo0CNN5LFsG110Xxtfs0iXUsqpLud+HzSwDuAc4AcgH5pjZFHdfXFTG3a+MK38ZcEjcIX509y6VF7JUlrZtQ7NecW3awOWXhzlnLrsM7rwzdFEVkZqncWP44x9DB6lx48JMBdUlkRpUV2CJuy919w3AJKB/GeUHARMrIzipWmPHbt9NfNddw/xNd98dalBKTiICYaSYG24IX1qrSyIJqhXwedxyfmzddswsE2gHvBq3up6Z5ZnZO2Z2cin7DYuVySsoKEgwdNlZgwfD+PFhrC6zUKPKyQldTEeODEP2KzmSp81xAAAUaklEQVSJSFQSSVAl/Ysq7VaugcBT7h4/ylNbd88BzgLuMLPt5ld19/HunuPuOS1atEggJKksgwfD8uVhpOPu3eH110N78w03KDmJSLQSSVD5QJu45dbAilLKDqRY8567r4j9XArMZNvrU5IENmyAgQPD4K833xzam0VEopZIgpoDtDezdmZWl5CEtuuNZ2b7A02Bt+PWNTWzXWLPmwPdgcXF95XobN4MZ50Vps247bYw0ZmISDIotxefuxea2XBgOpABTHD3RWZ2HZDn7kXJahAwyX2bkZw6APeZ2WZCMrw5vvefRG/cuJCcbr01TG4mIpIszJNpZEAgJyfH8/Lyog6jRnjzTejZE045BSZP1jUnEakeZjY31jehTBpJoob6+usw2WBWFjzwgJKTiCQfDVxTA23eHCYsKyiAt9+GJk2ijkhEZHtKUDXQLbeEeV/+/vcwWZmISDJSE18N8/rrMGpUaN67+OKooxERKZ0SVA3y1Vfhfqe99w4jSOi6k4gkMzXx1RBF151Wr4Z//zsMACkiksyUoGqIG2+EF1+E++4LQ+aLiCQ7NfHVADNmwOjRYcSIoUOjjkZEJDFKUGlu1aqQmNq3D7UnXXcSkVShJr40tmlTGK38m29g+nRo2DDqiEREEqcElcZuuAFeeSWMFNGpU9TRiIhUjJr40tQrr8Cf/xx67p1/ftTRiIhUnBJUGlq5Mlx3OuAAuPdeXXcSkdSkJr40U1gYktP338Orr0KDBlFHJCKyY5Sg0syf/wwzZ8LDD8NBB0UdjYjIjlMTXxp58UUYOxbOOw/OPTfqaEREdo4SVJpYsQLOPhsOPBDuvjvqaEREdp4SVBooLAyDwK5fD08+CfXrRx2RiMjO0zWoNPCnP4VpNB59FDp0iDoaEZHKoRpUips2DW66CS68MDTxiYikCyWoFPb55+FG3E6d4K67oo5GRKRyJZSgzKyPmX1oZkvMbEQJ2283s3djj4/M7Ju4beea2cexh/qWVZKNG8N1p59/Dteddt016ohERCpXudegzCwDuAc4AcgH5pjZFHdfXFTG3a+MK38ZcEjs+e7AaCAHcGBubN+1lfouaqBRo+Ctt+Dxx2G//aKORkSk8iVSg+oKLHH3pe6+AZgE9C+j/CBgYuz5L4GX3H1NLCm9BPTZmYAF/vUvuOUWuOgiGDQo6mhERKpGIgmqFfB53HJ+bN12zCwTaAe8WpF9zWyYmeWZWV5BQUEicddYn30WbsLt0gXuuCPqaEREqk4iCaqkoUa9lLIDgafcfVNF9nX38e6e4+45LVq0SCCkmmnDBjjzzHD96cknoV69qCMSEak6iSSofKBN3HJrYEUpZQeytXmvovtKOf7wB3jnHXjwQdh336ijERGpWokkqDlAezNrZ2Z1CUloSvFCZrY/0BR4O271dKC3mTU1s6ZA79g6qaDnnoPbboNLL4XTT486GhGRqlduLz53LzSz4YTEkgFMcPdFZnYdkOfuRclqEDDJ3T1u3zVmdj0hyQFc5+5rKvctpL/ly2HIEMjOhnHjoo5GRKR6WFw+SQo5OTmel5cXdRhJY8MGOOoo+O9/Yd482GefqCMSEdk5ZjbX3XPKK6ex+JLc734Hs2fDU08pOYlIzaKhjpLYP/8Jd94Jl18Op54adTQiItVLCSpJLV0K558Phx0Gt94adTQiItVPCSoJ/fwznHEGmMHkyVC3btQRiYhUP12DSkLXXANz58Kzz0JWVtTRiIhEQzWoJPPkk2HK9quugv5ljXgoIpLmlKCSyJIlcMEF0K1bmIRQRKQmU4JKEj/9FEaIqF0bnnhC151ERHQNKklceSW8+y5MnQqZmVFHIyISPdWgksDEifCPf8Bvfwsnnhh1NCIiyUEJKmIffQTDhsGRR8LYsVFHIyKSPJSgIvTjj+G60y67hOtOdepEHZGISPLQNagIXXEFLFwIzz8PrVtHHY2ISHJRDSoiublw//0wYgT07Rt1NCIiyUcJKgL//S9cdFGYRuP666OORkQkOSlBVbP168N1p/r1Q++92mpkFREpkf49VrPhw2HRInjhBWjVKupoRESSl2pQ1eiRR+Chh2DkSOjdO+poRESSmxJUNVmyBH79a+jVC8aMiToaEZHkpwRVDTZtgiFDwn1Ojz4KGRlRRyQikvwSSlBm1sfMPjSzJWY2opQyZ5jZYjNbZGaPx63fZGbvxh5TKivwVHLbbfDmm2EaDd3vJCKSmHI7SZhZBnAPcAKQD8wxsynuvjiuTHvgD0B3d19rZi3jDvGju3ep5LhTxvvvw6hRMGAADB4cdTQiIqkjkRpUV2CJuy919w3AJKD4VHpDgXvcfS2Au39VuWGmpo0b4X//F5o0CYPBmkUdkYhI6kgkQbUCPo9bzo+ti7cfsJ+ZvWlm75hZn7ht9cwsL7b+5JJewMyGxcrkFRQUVOgNJLMbboD582H8eGjRIupoRERSSyL3QZX0vd9LOE57oBfQGnjdzDq6+zdAW3dfYWZ7A6+a2Xvu/sk2B3MfD4wHyMnJKX7slJSXF0YnP+ccOLnEtCwiImVJpAaVD7SJW24NrCihzHPuvtHdlwEfEhIW7r4i9nMpMBM4ZCdjTno//hia9vbcE+66K+poRERSUyIJag7Q3szamVldYCBQvDfes8AxAGbWnNDkt9TMmprZLnHruwOLSXN//CN88AFMmAC77RbW5eZCVhbUqhV+5uZGGaGISPIrt4nP3QvNbDgwHcgAJrj7IjO7Dshz9ymxbb3NbDGwCfitu682syOB+8xsMyEZ3hzf+y8dzZoVupVfcsnW0SJyc8OkhOvXh+VPPw3LoJ59IiKlMffkuuSTk5PjeXl5UYexQ77/Hjp1Cr31FiyAhg3D+qyskJSKy8yE5curM0IRkeiZ2Vx3zymvnAaLrUTXXBMSzmuvbU1OAJ99VnL50taLiIiGOqo006fDfffB1VeHeZ7itW1b8j6lrRcRESWoSrF2LZx/Phx4YMkTEI4dG+Z/ile/flgvIiIlU4KqBJdfDqtWwf/9H9Srt/32wYPDzbqZmeH6VGZmWFYHCRGR0uka1E765z/hscdg9Gg49NDSyw0erIQkIlIRqkHthK++gosvhuzsMAmhiIhUHiWoHeQOF10E334bmvbq1Ik6IhGR9KImvh302GPw7LNw661w0EFRRyMikn5Ug9oB+flw2WXQowdceWXU0YiIpCclqApyhwsuCHM9Pfywpm8XEakqauKroPvugxdfhL//HfbZJ+poRETSl2pQFfDJJ2E4oxNOCL33RESk6ihBJWjTJhgyBGrXDtNoaPp2EZGqpSa+BN1+O7zxRuhS3rp11NGIiKQ/1aASsGhRuBH35JPh7LOjjkZEpGZQgirHxo1h+vbGjUMHCTXtiYhUDzXxlePGG2HePHj6aWjZMupoRERqDtWgyjB3LtxwQxjkdcCAqKMREalZlKBK8dNPoWmvZUv429+ijkZEpOZRE18p/vhHWLwYpk2Dpk2jjkZEpOZRDaoEb7wB48aF0cr79Ik6GhGRmimhBGVmfczsQzNbYmYjSilzhpktNrNFZvZ43Ppzzezj2OPcygq8qnz/PZx7LmRlhZHKRUQkGuU28ZlZBnAPcAKQD8wxsynuvjiuTHvgD0B3d19rZi1j63cHRgM5gANzY/uurfy3Ujl+9ztYtgxmzoRGjaKORkSk5kqkBtUVWOLuS919AzAJ6F+szFDgnqLE4+5fxdb/EnjJ3dfEtr0EJG2j2Ysvwr33hik0jj466mhERGq2RBJUK+DzuOX82Lp4+wH7mdmbZvaOmfWpwL6Y2TAzyzOzvIKCgsSjr0TffAPnnw8dOsDYsZGEICIicRLpxVfS2AlewnHaA72A1sDrZtYxwX1x9/HAeICcnJzttleHK66AL78Ms+TWqxdFBCIiEi+RGlQ+0CZuuTWwooQyz7n7RndfBnxISFiJ7Bu5Z58Ng8COHAk5OVFHIyIikFiCmgO0N7N2ZlYXGAhMKVbmWeAYADNrTmjyWwpMB3qbWVMzawr0jq1LGgUFMGwYHHJISFAiIpIcym3ic/dCMxtOSCwZwAR3X2Rm1wF57j6FrYloMbAJ+K27rwYws+sJSQ7gOndfUxVvZEe4h4kH162DV1+FunWjjkhERIqYeySXfEqVk5PjeXl51fJaublh+oy//CV0LxcRkapnZnPdvdwLKjV2JIkvvoDhw+HII+Hqq6OORkREiquRCcodLrgANmyARx6BjIyoIxIRkeJq5GCx48fD9Olw992w775RRyMiIiWpcTWopUtDk97xx8Mll0QdjYiIlKZGJahNm2DIkNCkN2EC1KpR715EJLXUqCa+O++E11+Hhx+GNm3KLS4iIhGqMXWIxYvh2muhX78wU66IiCS3GpGgNm4Mczw1bBg6SFhJIwSKiEhSqRFNfDfdBHl58OSTsMceUUcjIiKJSPsa1Lx5cP31cNZZcNppUUcjIiKJSusE9fPP4XpTixbwt79FHY2IiFREWjfx/elPsGgRPP887L571NGIiEhFpG0N6q234NZbYehQ6Ns36mhERKSi0jJB/fBDaNrLzIRx46KORkREdkRaNvH9/vfwyScwcyY0ahR1NCIisiPSrgY1dy7ccw/85jfQs2fU0YiIyI5KuxpUdjZMnAj9+0cdiYiI7Iy0S1BmMHBg1FGIiMjOSrsmPhERSQ9KUCIikpSUoEREJCkllKDMrI+ZfWhmS8xsRAnbh5hZgZm9G3tcGLdtU9z6KZUZvIiIpK9yO0mYWQZwD3ACkA/MMbMp7r64WNEn3H14CYf40d277HyoIiJSkyRSg+oKLHH3pe6+AZgEqBO3iIhUqUQSVCvg87jl/Ni64k41s4Vm9pSZxU+oXs/M8szsHTM7uaQXMLNhsTJ5BQUFiUcvIiJpK5EEVdL8s15seSqQ5e6dgJeBR+K2tXX3HOAs4A4z22e7g7mPd/ccd89p0aJFgqGLiEg6SyRB5QPxNaLWwIr4Au6+2t1/ji3eDxwat21F7OdSYCZwyE7EKyIiNUQiCWoO0N7M2plZXWAgsE1vPDPbK26xH/BBbH1TM9sl9rw50B0o3rlCRERkO+X24nP3QjMbDkwHMoAJ7r7IzK4D8tx9CnC5mfUDCoE1wJDY7h2A+8xsMyEZ3lxC7z8REZHtmHvxy0nRysnJ8by8vKjDEBGRKmJmc2N9E8qkkSRERCQpKUGJiEhSUoISEZGkpAQlIiJJSQlKRESSkhKUiIgkJSUoERFJSkpQIiKSlJSgREQkKSlBiYhIUlKCEhGRpKQEJSIiSUkJSkREkpISlIiIJCUlKBERSUppl6BycyErC2rVCj9zc6OOSEREdkS5M+qmktxcGDYM1q8Py59+GpYBBg+OLi4REam4tKpBjRy5NTkVWb8+rBcRkdSSVgnqs88qtl5ERJJXWiWotm0rtl5ERJJXQgnKzPqY2YdmtsTMRpSwfYiZFZjZu7HHhXHbzjWzj2OPcysz+OLGjoX69bddV79+WC8iIqml3E4SZpYB3AOcAOQDc8xsirsvLlb0CXcfXmzf3YHRQA7gwNzYvmsrJfpiijpCjBwZmvXatg3JSR0kRERSTyK9+LoCS9x9KYCZTQL6A8UTVEl+Cbzk7mti+74E9AEm7li45Rs8WAlJRCQdJNLE1wr4PG45P7auuFPNbKGZPWVmbSqyr5kNM7M8M8srKChIMHQREUlniSQoK2GdF1ueCmS5eyfgZeCRCuyLu4939xx3z2nRokUCIYmISLpLJEHlA23illsDK+ILuPtqd/85tng/cGii+4qIiJQkkQQ1B2hvZu3MrC4wEJgSX8DM9opb7Ad8EHs+HehtZk3NrCnQO7ZORESkTOV2knD3QjMbTkgsGcAEd19kZtcBee4+BbjczPoBhcAaYEhs3zVmdj0hyQFcV9RhQkREpCzmvt0loUjl5OR4Xl5e1GGIiEgVMbO57p5TbrlkS1BmVgB8GnUcVag58HXUQaQQna+K0fmqOJ2ziqmM85Xp7uX2iEu6BJXuzCwvkW8OEuh8VYzOV8XpnFVMdZ6vtBqLT0RE0ocSlIiIJCUlqOo3PuoAUozOV8XofFWczlnFVNv50jUoERFJSqpBiYhIUlKCEhGRpKQEVQ3MrI2ZzTCzD8xskZldEXVMqcDMMsxsvpn9K+pYUoGZ7RabTeC/sb+1I6KOKZmZ2ZWxz+P7ZjbRzOpFHVOyMbMJZvaVmb0ft253M3spNgntS7Fh7KqEElT1KASudvcOwOHApWZ2YMQxpYIr2Dquo5TvTuAFdz8A6IzOXanMrBVwOZDj7h0Jw7gNjDaqpPQwYQ6/eCOAV9y9PfBKbLlKKEFVA3df6e7zYs+/I/zjKGlOLYkxs9bA/wAPRB1LKjCzxsDRwIMA7r7B3b+JNqqkVxvY1cxqA/XRTAvbcfdZhPFV4/Vn65RKjwAnV9XrK0FVMzPLAg4B/hNtJEnvDuB3wOaoA0kRewMFwEOxZtEHzKxB1EElK3f/Avgr8BmwEljn7i9GG1XK2MPdV0L48g20rKoXUoKqRmbWEHga+I27fxt1PMnKzE4EvnL3uVHHkkJqA9nAve5+CPADVdj0kupi1036A+2AXwANzOzsaKOS4pSgqomZ1SEkp1x3/2fU8SS57kA/M1sOTAKONbPHog0p6eUD+e5eVDN/ipCwpGTHA8vcvcDdNwL/BI6MOKZUsapoDsDYz6+q6oWUoKqBmRnh2sAH7n5b1PEkO3f/g7u3dvcswoXrV91d327L4O5fAp+b2f6xVccBiyMMKdl9BhxuZvVjn8/jUKeSRE0Bzo09Pxd4rqpeqNwJC6VSdAfOAd4zs3dj66519+cjjEnSz2VAbmzm66XAeRHHk7Tc/T9m9hQwj9DLdj4a8mg7ZjYR6AU0N7N8YDRwMzDZzC4gJPrTq+z1NdSRiIgkIzXxiYhIUlKCEhGRpKQEJSIiSUkJSkREkpISlIiIJCUlKBERSUpKUCIikpT+H4gQEA4X4/F2AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAAEYCAYAAAAJeGK1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3Xl8VNX9//HXh01kERCwKAgJiOxbjAhFBRUtagFFq2CoYlXUulurVG3dSt2oWpSvP2nr0hKhFDdcqa1Ya7VAWBUQQdYIIqCgLAqBz++PMwkhmSQTmDAzyfv5eOSRuXfO3Plk0Lxzzj33XHN3REREkk21RBcgIiISjQJKRESSkgJKRESSkgJKRESSkgJKRESSkgJKRESSkgJKEsbMqpvZVjNrGc+2iWRmx5hZ3K/dMLP+Zray0PYSMzsplrb78V5/MrPb9/f1pRz3t2b2bLyPK5VXjUQXIKnDzLYW2qwDfA/sjmxf6e7Z5Tmeu+8G6sW7bVXg7u3icRwzuxwY7u79Ch378ngcW+RAKaAkZu5eEBCRv9Avd/d/ltTezGq4e97BqE1EKh8N8UncRIZw/mZmE83sW2C4mfU2s/+Z2WYzW2dmY82sZqR9DTNzM0uLbE+IPP+mmX1rZh+aWXp520aeP9PMPjWzLWb2uJn918xGlFB3LDVeaWbLzOxrMxtb6LXVzexRM9tkZp8BA0r5fO40s0lF9o0zs0cijy83s8WRn+ezSO+mpGPlmlm/yOM6ZvbXSG0LgeOivO/yyHEXmtmgyP4uwBPASZHh042FPtu7C73+qsjPvsnMXjazI2P5bMpiZudE6tlsZu+YWbtCz91uZmvN7Bsz+6TQz9rLzOZE9q83s4djfT9JQe6uL32V+wtYCfQvsu+3wE5gIOGPn0OB44ETCL311sCnwLWR9jUAB9Ii2xOAjUAmUBP4GzBhP9oeAXwLDI48dzOwCxhRws8SS42vAA2ANOCr/J8duBZYCLQAGgPvhf+tor5Pa2ArULfQsb8EMiPbAyNtDDgV2AF0jTzXH1hZ6Fi5QL/I4zHAu0AjoBWwqEjbC4AjI/8mF0Vq+EHkucuBd4vUOQG4O/L4jEiN3YHawP8B78Ty2UT5+X8LPBt53CFSx6mRf6PbI597TaATsApoFmmbDrSOPJ4FDIs8rg+ckOj/F/RVcV/qQUm8ve/ur7r7Hnff4e6z3H2Gu+e5+3JgPNC3lNdPcfccd98FZBN+MZa37Y+Bee7+SuS5RwlhFlWMNd7v7lvcfSUhDPLf6wLgUXfPdfdNwAOlvM9y4GNCcAKcDmx295zI86+6+3IP3gH+BUSdCFHEBcBv3f1rd19F6BUVft/J7r4u8m/yPOGPi8wYjguQBfzJ3ee5+3fAKKCvmbUo1Kakz6Y0Q4Gp7v5O5N/oAeAwwh8KeYQw7BQZJl4R+ewg/KHR1swau/u37j4jxp9DUpACSuJtTeENM2tvZq+b2Rdm9g1wL9CklNd/UejxdkqfGFFS26MK1+HuTuhxRBVjjTG9F+Ev/9I8DwyLPL6IEKz5dfzYzGaY2VdmtpnQeynts8p3ZGk1mNkIM5sfGUrbDLSP8bgQfr6C47n7N8DXQPNCbcrzb1bScfcQ/o2au/sS4BeEf4cvI0PGzSJNLwU6AkvMbKaZnRXjzyEpSAEl8VZ0ivVThF7DMe5+GPAbwhBWRVpHGHIDwMyMfX+hFnUgNa4Dji60XdY0+L8B/SM9kMGEwMLMDgWmAPcTht8aAv+IsY4vSqrBzFoDTwJXA40jx/2k0HHLmhK/ljBsmH+8+oShxM9jqKs8x61G+Df7HMDdJ7h7H8LwXnXC54K7L3H3oYRh3N8DL5hZ7QOsRZKUAkoqWn1gC7DNzDoAVx6E93wNyDCzgWZWA7gBaFpBNU4GbjSz5mbWGLittMbuvh54H3gGWOLuSyNPHQLUAjYAu83sx8Bp5ajhdjNraOE6sWsLPVePEEIbCFl9OaEHlW890CJ/UkgUE4HLzKyrmR1CCIr/uHuJPdJy1DzIzPpF3vuXhPOGM8ysg5mdEnm/HZGv3YQf4Kdm1iTS49oS+dn2HGAtkqQUUFLRfgFcQvjl8xShB1GhIiFwIfAIsAloA8wlXLcV7xqfJJwr+ohwAn9KDK95njDp4flCNW8GbgJeIkw0OJ8QtLG4i9CTWwm8Cfyl0HEXAGOBmZE27YHC523eBpYC682s8FBd/uvfIgy1vRR5fUvCeakD4u4LCZ/5k4TwHAAMipyPOgR4iHDe8AtCj+3OyEvPAhZbmCU6BrjQ3XceaD2SnCwMz4tUXmZWnTCkdL67/yfR9YhIbNSDkkrJzAaYWYPIMNGvCTPDZia4LBEpBwWUVFYnAssJw0QDgHPcvaQhPhFJQhriExGRpKQelIiIJKWELRbbpEkTT0tLS9Tbi4hIgsyePXuju5d26QeQwIBKS0sjJycnUW8vIiIJYmZlrbgCaIhPRESSlAJKRESSkgJKRESSku6oKyIpY9euXeTm5vLdd98luhSJQe3atWnRogU1a5a01GPpFFAikjJyc3OpX78+aWlphEXqJVm5O5s2bSI3N5f09PSyXxCFhvhEJGV89913NG7cWOGUAsyMxo0bH1BvN2UDKjsb0tKgWrXwPTu7rFeISGWgcEodB/pvlZJDfNnZMHIkbN8etletCtsAWQd8IwAREUkGKdmDuuOOveGUb/v2sF9EpKJs2rSJ7t270717d5o1a0bz5s0LtnfujO22VJdeeilLliwptc24cePIjtOw0Iknnsi8efPicqyDLSV7UKtXl2+/iFRN2dnhD9fVq6FlSxg9+sBGWRo3blzwy/7uu++mXr163HLLLfu0cXfcnWrVov/9/8wzz5T5Ptdcc83+F1mJpGQPqmXL8u0Xkaon/1TAqlXgvvdUQEWcr162bBmdO3fmqquuIiMjg3Xr1jFy5EgyMzPp1KkT9957b0Hb/B5NXl4eDRs2ZNSoUXTr1o3evXvz5ZdfAnDnnXfy2GOPFbQfNWoUPXv2pF27dnzwwQcAbNu2jfPOO49u3boxbNgwMjMzy+wpTZgwgS5dutC5c2duv/12APLy8vjpT39asH/s2LEAPProo3Ts2JFu3boxfPjwuH9msUjJgBo9GurU2XdfnTphv4gIHPxTAYsWLeKyyy5j7ty5NG/enAceeICcnBzmz5/P22+/zaJFi4q9ZsuWLfTt25f58+fTu3dvnn766ajHdndmzpzJww8/XBB2jz/+OM2aNWP+/PmMGjWKuXPnllpfbm4ud955J9OnT2fu3Ln897//5bXXXmP27Nls3LiRjz76iI8//piLL74YgIceeoh58+Yxf/58nnjiiQP8dPZPSgZUVhaMHw+tWoFZ+D5+vCZIiMheB/tUQJs2bTj++OMLtidOnEhGRgYZGRksXrw4akAdeuihnHnmmQAcd9xxrFy5MuqxhwwZUqzN+++/z9ChQwHo1q0bnTp1KrW+GTNmcOqpp9KkSRNq1qzJRRddxHvvvccxxxzDkiVLuOGGG5g2bRoNGjQAoFOnTgwfPpzs7Oz9vtD2QKVkQEEIo5UrYc+e8F3hJCKFHexTAXXr1i14vHTpUv7whz/wzjvvsGDBAgYMGBD1eqBatWoVPK5evTp5eXlRj33IIYcUa1Pem82W1L5x48YsWLCAE088kbFjx3LllVcCMG3aNK666ipmzpxJZmYmu3fvLtf7xUPKBpSISGkSeSrgm2++oX79+hx22GGsW7eOadOmxf09TjzxRCZPngzARx99FLWHVlivXr2YPn06mzZtIi8vj0mTJtG3b182bNiAu/OTn/yEe+65hzlz5rB7925yc3M59dRTefjhh9mwYQPbi46XHgQpOYtPRKQs+aMq8ZzFF6uMjAw6duxI586dad26NX369In7e1x33XVcfPHFdO3alYyMDDp37lwwPBdNixYtuPfee+nXrx/uzsCBAzn77LOZM2cOl112Ge6OmfHggw+Sl5fHRRddxLfffsuePXu47bbbqF+/ftx/hrJYebuJ8ZKZmem6YaGIlMfixYvp0KFDostICnl5eeTl5VG7dm2WLl3KGWecwdKlS6lRI7n6HdH+zcxstrtnlvXa5PpJREQkJlu3buW0004jLy8Pd+epp55KunA6UDH9NGY2APgDUB34k7s/UOT5lsBzQMNIm1Hu/kacaxURkYiGDRsye/bsRJdRocqcJGFm1YFxwJlAR2CYmXUs0uxOYLK79wCGAv8X70JFRKRqiWUWX09gmbsvd/edwCRgcJE2DhwWedwAWBu/EkVEpCqKJaCaA2sKbedG9hV2NzDczHKBN4Droh3IzEaaWY6Z5WzYsGE/yhURkaoiloCKdkOPolP/hgHPunsL4Czgr2ZW7NjuPt7dM909s2nTpuWvVkREqoxYAioXOLrQdguKD+FdBkwGcPcPgdpAk3gUKCKSLPr161fsotvHHnuMn//856W+rl69egCsXbuW888/v8Rjl3XpzWOPPbbPBbNnnXUWmzdvjqX0Ut19992MGTPmgI8Tb7EE1CygrZmlm1ktwiSIqUXarAZOAzCzDoSA0hieiFQqw4YNY9KkSfvsmzRpEsOGDYvp9UcddRRTpkzZ7/cvGlBvvPEGDRs23O/jJbsyA8rd84BrgWnAYsJsvYVmdq+ZDYo0+wVwhZnNByYCIzxRVwCLiFSQ888/n9dee43vv/8egJUrV7J27VpOPPHEguuSMjIy6NKlC6+88kqx169cuZLOnTsDsGPHDoYOHUrXrl258MIL2bFjR0G7q6++uuBWHXfddRcAY8eOZe3atZxyyimccsopAKSlpbFx40YAHnnkETp37kznzp0LbtWxcuVKOnTowBVXXEGnTp0444wz9nmfaObNm0evXr3o2rUr5557Ll9//XXB+3fs2JGuXbsWLFL773//u+CGjT169ODbb7/d7882mpiug4pc0/RGkX2/KfR4ERD/tTxEREpw440Q7xvFdu8Okd/tUTVu3JiePXvy1ltvMXjwYCZNmsSFF16ImVG7dm1eeuklDjvsMDZu3EivXr0YNGgQZtFO48OTTz5JnTp1WLBgAQsWLCAjI6PgudGjR3P44Yeze/duTjvtNBYsWMD111/PI488wvTp02nSZN8zKLNnz+aZZ55hxowZuDsnnHACffv2pVGjRixdupSJEyfyxz/+kQsuuIAXXnih1Ps7XXzxxTz++OP07duX3/zmN9xzzz089thjPPDAA6xYsYJDDjmkYFhxzJgxjBs3jj59+rB161Zq165djk+7bFosVkSkHAoP8xUe3nN3br/9drp27Ur//v35/PPPWb9+fYnHee+99wqComvXrnTt2rXgucmTJ5ORkUGPHj1YuHBhmQvBvv/++5x77rnUrVuXevXqMWTIEP7zn/8AkJ6eTvfu3YHSb+kB4f5Umzdvpm/fvgBccsklvPfeewU1ZmVlMWHChIIVK/r06cPNN9/M2LFj2bx5c9xXsqhc62KISJVRWk+nIp1zzjncfPPNzJkzhx07dhT0fLKzs9mwYQOzZ8+mZs2apKWlRb3FRmHRelcrVqxgzJgxzJo1i0aNGjFixIgyj1PaGZX8W3VAuF1HWUN8JXn99dd57733mDp1Kvfddx8LFy5k1KhRnH322bzxxhv06tWLf/7zn7Rv336/jh+NelAiIuVQr149+vXrx89+9rN9Jkds2bKFI444gpo1azJ9+nRWrVpV6nFOPvlksiP3n//4449ZsGABEG7VUbduXRo0aMD69et58803C15Tv379qOd5Tj75ZF5++WW2b9/Otm3beOmllzjppJPK/bM1aNCARo0aFfS+/vrXv9K3b1/27NnDmjVrOOWUU3jooYfYvHkzW7du5bPPPqNLly7cdtttZGZm8sknn5T7PUujHpSISDkNGzaMIUOG7DOjLysri4EDB5KZmUn37t3L7ElcffXVXHrppXTt2pXu3bvTs2dPINwdt0ePHnTq1KnYrTpGjhzJmWeeyZFHHsn06dML9mdkZDBixIiCY1x++eX06NGj1OG8kjz33HNcddVVbN++ndatW/PMM8+we/duhg8fzpYtW3B3brrpJho2bMivf/1rpk+fTvXq1enYsWPB3YHjRbfbEJGUodttpJ4Dud2GhvhERCQpKaBERCQpKaBEJKVoDYDUcaD/VgooEUkZtWvXZtOmTQqpFODubNq06YAu3tUsPhFJGS1atCA3Nxfdric11K5dmxYtWuz36xVQIpIyatasSXp6eqLLkINEQ3wiIpKUFFAiIpKUFFAiIpKUFFAiIpKUFFAiIpKUFFAiIpKUFFAiIpKUFFAiIpKUFFAiIpKUFFAiIpKUFFAiIpKUFFAiIpKUFFAiIpKUFFAiIpKUFFAiIpKUYgooMxtgZkvMbJmZjYry/KNmNi/y9amZbY5/qSIiUpWUecNCM6sOjANOB3KBWWY21d0X5bdx95sKtb8O6FEBtYqISBUSSw+qJ7DM3Ze7+05gEjC4lPbDgInxKE5ERKquWAKqObCm0HZuZF8xZtYKSAfeKeH5kWaWY2Y5GzZsKG+tIiJShcQSUBZln5fQdigwxd13R3vS3ce7e6a7ZzZt2jTWGkVEpAqKJaBygaMLbbcA1pbQdiga3hMRkTiIJaBmAW3NLN3MahFCaGrRRmbWDmgEfBjfEqPbtg1efBFycw/Gu4mIyMFW5iw+d88zs2uBaUB14Gl3X2hm9wI57p4fVsOASe5e0vBfXM2aBeedFx4fdRT07Ln3KzMTGjQ4GFWIiEhFsYOUJ8VkZmZ6Tk7Ofr/+++9h/nyYORNmzAjfP/107/Pt24ewOuGE8L1rV6hVKw6Fi4jIATGz2e6eWWa7VA2oaL7+GnJy9g2t9evDc7VqQY8ee3tZJ5wAxxwDFm0KiIiIVJgqGVBFucOaNSGo8r9ycsL5K4BGjeD44/cdHvzBDyq0JBGRKk8BVYLdu2HRon1D66OPwn6AVq32DayMDKhXL/bjZ2fDHXfA6tXQsiWMHg1ZWRXzs4iIpCIFVDls2wZz5+4bWitWhOeqVYPOnfcNrU6doEaU6SXZ2TByJGzfvndfnTowfrxCSkQknwLqAG3YEGYK5p/LmjkTvvoqPHfooXDccXsnYPTsGXpe6emwalXxY7VqBStXHtTyRUSSlgIqztxh+fJ9J2DMmRNmEwI0bRpCLRoz2LPn4NUqIpLMYg2oMq+DksAM2rQJX8OGhX27doXzV/k9rAkTwr6iWrY8uLWKiFQGumHhAahZM0yiuOoqePppeOaZMPxXVMeO8OWXB78+EZFUpoCKo6ws+OMfwzkns7DCxUknwbRp0Lo1/OpXe89jiYhI6RRQcZaVFSZE7NkDn38O770XprUPGgQPPhgmUtx9N2zZkuhKRUSSmwLqIGjXDp5/HhYsgP794Z57QlD97newdWuiqxMRSU4KqIOoc2d44YUw+69Pn3BBb3o6/P73+147JSIiCqiE6NEDXn0VPvwwPL7lljA78PHH905bFxGp6hRQCdSrF/zjH/Dvf8Oxx8L110PbtmHliWjT1UVEqhIFVBI4+WR49114+21o3hyuvDKct3r2WcjLS3R1IiKJoYBKEmZhAsUHH8Drr4eV1i+9NKz7N3GiVqIQkapHAZVkzOCss8JtQV58MdzH6qKLwg0XX3wxLLkkIlIVKKCSlBmce264a/DEiWGo77zzwiK1r72moBKRyk8BleSqVYOhQ+Hjj+G558IFvgMHQu/e4ZxVeYIqOxvS0sIx09LCtohIslJApYgaNeDii+GTT8Isv7Vr4YwzoG/fMAuwLPn3qlq1KoTaqlVhWyElIslKAZViataEK66ApUvhiSdg2TLo1y9MsPjww5Jfd8cdxS8G3r497BcRSUYKqBR1yCFwzTXw2WdhJYoFC+CHP4Szz4bZs4u3X706+nFK2i8ikmgKqBR36KFw883hZor33x96UZmZMGRIuFdVvpLuSaV7VYlIslJAVRL16sGoUbBiRVgt/V//gm7dwgSLTz6B0aOhTp19X1OnTtgvIpKMFFCVTIMGcNddIahGjQpT0jt1Cksq3Xff3ntVtWoVJltkZSW6YhGR6BRQldThh4fbeSxfDjfdBJMnw623wumnhxl8K1cqnEQkucUUUGY2wMyWmNkyMxtVQpsLzGyRmS00s+fjW6bsryOOgDFjwmSKq6+Gv/wlrKC+ZEmiKxMRKV2ZAWVm1YFxwJlAR2CYmXUs0qYt8Cugj7t3Am6sgFrlABx1VLidx4IF4ULdAQNg3bpEVyUiUrJYelA9gWXuvtzddwKTgMFF2lwBjHP3rwHc/cv4linx0q5dWIx2w4aw5t833yS6IhGR6GIJqObAmkLbuZF9hR0LHGtm/zWz/5nZgGgHMrORZpZjZjkbNmzYv4rlgB1/PEyZEpZPGjJEN0kUkeQUS0BZlH1FV4CrAbQF+gHDgD+ZWcNiL3If7+6Z7p7ZtGnT8tYqcTRgAPz5z2E6+ogRup2HiCSfGjG0yQWOLrTdAlgbpc3/3H0XsMLMlhACa1ZcqpQKcfHF4TzUqFHhHNXvf5/oikRE9oqlBzULaGtm6WZWCxgKTC3S5mXgFAAza0IY8lsez0KlYtx6a7jV/COPKKBEJLmU2YNy9zwzuxaYBlQHnnb3hWZ2L5Dj7lMjz51hZouA3cAv3X1TRRYu8WEGjz4KX3wBt9wCzZrp+igRSQ7mCbrzXWZmpufk5CTkvaW4778P56X++98wy+/00xNdkYhUVmY2290zy2qnlSQECKujv/wydOgQZvbNmZPoikSkqlNASYEGDeDNN8MySWeeGVafEBFJFAWU7OOoo2DaNMjLC0N+X+qSaxFJEAWUFNO+fVgF/fPP4cc/hq1bE12RiFRFCiiJqndv+Nvfwt15f/IT2LUr0RWJSFWjgJISDRwITz0Fb70Fl18OCZrwKSJVVCwrSUgVdvnlsHZtuAli8+bhHlMiIgeDAkrK9Otfh5C6/3448ki47rpEVyQiVYECSspkBuPGhdUmbrghrDbxk58kuioRqex0DkpiUr06TJwYJk8MHw7vvpvoikSkslNAScwOPRRefRXatIHBg8PdeUVEKooCSsrl8MPDrL769cNqE6tXJ7oiEamsFFBSbi1bhiWRtm2DH/0INmndehGpAAoo2S9dusArr8CKFeF6qe3bE12RiFQ2CijZb337QnY2/O9/MGxYWL9PRCReFFByQM47D8aOhalT4ec/12oTIhI/ug5KDti118K6dWGViebNw6oTIiIHSgElcfHb34bVJu6+O6w2MXJkoisSkVSnIT6JCzMYPz5MPb/66jCBojyysyEtDapVC9+zsyuiShFJJQooiZuaNeHvf4fjjoOhQ+GDD2J7XXZ26HGtWhXOYa1aFbYVUiJVmwJK4qpuXXj9dTj66HCzw8WLy37NHXcUn6a+fXvYLyJVlwJK4q5p03Db+Fq1woW8n39eevuSVqPQKhUiVZsCSipEenpYbWLz5nBeavPmktu2bFm+/SJSNSigpML06AEvvgiffALnnAPffRe93ejRUKfOvvvq1An7RaTqUkBJherfH557Dv79b/jpT2H37uJtsrLCDMBWrcJswFatwnZW1sGvV0SSh66Dkgo3bFi4kPcXv4AbbwwrT5jt2yYrS4EkIvuKqQdlZgPMbImZLTOzUVGeH2FmG8xsXuTr8viXKqns5ptDQD3xBDz4YKKrEZFUUGYPysyqA+OA04FcYJaZTXX3RUWa/s3dr62AGqWSeOih0JP61a/CahOXXJLoikQkmcXSg+oJLHP35e6+E5gEDK7YsqQyqlYNnnkmnJe67LIwy09EUsPq1XDllWHS08ESS0A1B9YU2s6N7CvqPDNbYGZTzOzoaAcys5FmlmNmORs2bNiPciXV1aoFL7wAXbvC+efDzJmJrkhESvP552FB6LZt4dlnYcaMg/fesQSURdlX9KYKrwJp7t4V+CfwXLQDuft4d89098ymTZuWr1KpNA47DN54A37wAzj7bFi6NNEViUhRX3wBN90EbdrAU0/BiBHh/9WDOTQfS0DlAoV7RC2AtYUbuPsmd/8+svlH4Lj4lCeVVbNm8NZb4fGPfhT+ZxCRxNu4EW69FVq3hscfh4sugk8/DSF1sC+ejyWgZgFtzSzdzGoBQ4GphRuY2ZGFNgcBMazAJlXdsceGdfvWr4ezzoJvv010RSJV11dfhfUv09NhzJhwM9LFi+Hpp8O+RCgzoNw9D7gWmEYInsnuvtDM7jWzQZFm15vZQjObD1wPjKiogqVy6dkTpkyBBQtgyBDYuTPRFYlULVu2hPu4paeHm46efTYsXAh//Ws475RI5gm6R3dmZqbn5OQk5L0l+Tz7LFx6KZx0Ejz6aLhlh4hUnG+/DRfNjxkT1socMiQEVZcuFf/eZjbb3TPLaqeljiQpjBgRhhIWL4bMzLCqxMqVia5KpPLZti1ck5ieDnfeGf4onDMnzK49GOFUHgooSRqXXgrLlsHtt4dFZtu1C6tPfPVVoisTSX07dsBjj4XJD7fdBscfH6aMT50aFnZORgooSSoNGoRVzJcuheHDw3BfmzZhGKKk1dBFpGTffw/jxsExx4Rp4126wPvvhwvle/ZMdHWlU0BJUmrRAv78Z5g/H374Q/jlL0OPasIE2LMn0dWJJL+dO8NdAdq2DRfatmkD774L//wn9OmT6Opio4CSpNalS5iK/q9/QZMm4ZYdxx0X/icTkeLy8sKSYu3ahaWJmjeHt98Ot7zp2zfR1ZWPAkpSwqmnwqxZ8PzzYcbR6afDgAGhhyUi4V5rEyZAhw7ws59B48ZhxZYPPgjrXxa9xU0qUEBJyqhWLdxb6pNP4JFHwjp+PXqEGYBr1pT5cpFKac8e+NvfoHPnMMJQpw68/HL4g+7MM1MzmPIpoCTlHHJIONn72Wdwyy0waVIYZx81KvSuRKoC9zDbtVs3GDo0/AH397/D3LkweHBqB1M+BZSkrEaNwvUcS5bABReEx23ahKm0339f9utFUpE7vPpqOBd73nlhMsTzz4fVWM4/PwRVZVGJfhSpqlq1gr/8BWbPhoyM0Lvq0CEMe2jGn1QW7mGB5RNOgEGDwhJFzz0XliUaNgyqV090hfGngJKmBXDLAAANTElEQVRKo0ePMFtp2jSoXz8Me/TqFabWiqQq9zCL9cQTwzmlL7+EP/0pnIu9+GKoUeZ90VOXAkoqnTPOCEu3PPdcuI3HKafAwIHhL02peDt2wGuvwciRcPLJ4e7Jv/99uDB01Sr1asvjP/8J//327x8+uyefDLe+uOwyqFkz0dVVPC0WK5Xajh3hnja/+11YHPNnP4N77oGjjkp0ZZXLhg0hlKZOhX/8A7ZvD73YLl3CZJb16/e2rVs3DMF27LjvV1pa5Rym2h8ffgi/+U243q9Zs7D81xVXQO3aia4sPmJdLFYBJVXCpk1hCaUnnghDIr/4RVid4rDDEl1Z6lqyBF55JYTSBx+Eoaijjw7nRwYNgn79oFat0HbTprAQ8KJF+359/vne49WuDe3bFw+uNm0q3zDWnj0h1Nesgdzc8D3/8bJlYYp406Zhzbyrrw5TxysTBZRIFMuXhxWcJ04MvwDuuisMRVWF4ZIDtXt3+Mt+6tQQTJ9+GvZnZOwNpe7dyze9ecuW6MG1atXeNjVrhlURigZX27Z7AzCZuIdAzg+daCGUm1v83me1aoUlvlq0CDfwvOYaqFcvMT9DRVNAiZQiJyf0oN59N/yiu//+cD+cynDtSDxt2xaG7KZODUN4GzeGwDjllHCtzcCBodcUb1u3hkkAhUNr8eIwXJj/K6t69bAAatHgatcODj00/jVBeO+vvy4eOoW3c3OLL2xcs2ZYcqhFi/B55X8V3m7SpHJNES+NAkqkDO7hxP2tt4YJFL16wcMPh9lSVdm6deE6m6lTwzmQ77+Hhg3DnVYHDQpLTCVqaHTHjtBzK9rjWro09PAg/JHRunXx4GrfvvQeiTt8803pPZ81a8L5tcKqVy8ePkUf/+AHVSd8YqGAEolRXl6Y8ffrX4dfzuecE3pU7dsnurKDwz0EdP75pJkzw/709NBLGjQohHYyD4Pu3BlCqmhwLVkCu3btbdeq1d7AqleveAht3brvcatVgyOPLDl4jj46TGLQ5I7yUUCJlNO2bWEVigcfDH8lX3FFOEfVrFmiK4u/XbvCPYHyzyetWBH29+y5N5Q6dUr9Ic9du8J5x6LB9cknoWfYrFnx4Cm8feSRlW+CRjJQQInspy+/hPvug//3/8K6f7/8ZZj1l+onrL/5JqxE8MorYZXrzZvDz9e/fwikgQPDL+SqYPfuMJMumXuFlZkCSuQALV0arj+ZMiWcQ+jdO9zC4PDD9/1edF8yXauyevXe80nTp4ceRZMm8OMfh1A644xwXZLIwRRrQKnzKlKCtm3D6tAffhjOSS1bFs7PbNpU+mK0deoUD7GSwiz/++GHx+eveXeYN2/v+aS5c8P+Y4+FG28ModS7t86ZSGpQQImUoXfv8Ms+n3s4R/XVVyGsSvqe//jjj8P3r74KEzJKUr9+6SEW7XvDhmG46t1394ZSbm44d/TDH4bzaYMHh6nXIqlGASVShuxsuOOOMFzWsmVYkSIrKwyNlecaIPew3FK0EIv2fcWK8Pjrr/de+1OUWeh57dwZem5nnAH33humhB9xRHx+fpFEUUCJlCI7O6w0kX/ty6pVYRtCSJWHWbh+6LDDwhTuWO3eHVZcKKmXtmMH9O0Lp51WcReoiiSCJkmIlCItbd9ld/K1agUrVx7sakQqh1gnScR0bbOZDTCzJWa2zMxGldLufDNzMyvzjUVSwerV5dsvIvFTZkCZWXVgHHAm0BEYZmYdo7SrD1wPzIh3kSKJ0rJl+faLSPzE0oPqCSxz9+XuvhOYBAyO0u4+4CHguyjPiaSk0aOL3+qgTp2wX0QqViwB1RxYU2g7N7KvgJn1AI5299fiWJtIwmVlwfjx4ZyTWfg+fnz5J0iISPnFMosv2mpcBTMrzKwa8CgwoswDmY0ERgK01BiJpIisLAWSSCLE0oPKBQpf7dECWFtouz7QGXjXzFYCvYCp0SZKuPt4d89098ymTZvuf9UiIlLpxRJQs4C2ZpZuZrWAoUDBdfXuvsXdm7h7mrunAf8DBrm75pCLiMh+KzOg3D0PuBaYBiwGJrv7QjO718wGVXSBIiJSNcW0koS7vwG8UWTfb0po2+/AyxIRkapONyEWEZGkpIASEZGkpIASEZGkpIASEZGkpIASEZGkpIASEZGkpIASEZGkpIASEZGkpIASEZGkpIASEZGkpIASSVHZ2ZCWBtWqhe/Z2YmuSCS+YlqLT0SSS3Y2jBwJ27eH7VWrwjbo3lVSeagHJZKC7rhjbzjl27497BepLBRQIilo9ery7RdJRQookRTUsmX59oukIgWUSAoaPRrq1Nl3X506Yb9IZaGAEklBWVkwfjy0agVm4fv48ZogIZWLZvGJpKisLAWSVG7qQYmISFJSQImISFJSQImISFJSQImISFJSQImISFJSQIlI3GkhW4kHTTMXkbjSQrYSL+pBiUhcaSFbiRcFlIjElRaylXiJKaDMbICZLTGzZWY2KsrzV5nZR2Y2z8zeN7OO8S9VRFKBFrKVeCkzoMysOjAOOBPoCAyLEkDPu3sXd+8OPAQ8EvdKRSQlaCFbiZdYelA9gWXuvtzddwKTgMGFG7j7N4U26wIevxJFJJVoIVuJl1hm8TUH1hTazgVOKNrIzK4BbgZqAadGO5CZjQRGArRUf1+k0tJCthIPsfSgLMq+Yj0kdx/n7m2A24A7ox3I3ce7e6a7ZzZt2rR8lYqISJUSS0DlAkcX2m4BrC2l/STgnAMpSkREJJaAmgW0NbN0M6sFDAWmFm5gZm0LbZ4NLI1fiSIiUhWVGVDungdcC0wDFgOT3X2hmd1rZoMiza41s4VmNo9wHuqSCqtYROQAaBmm1GHuiZlwl5mZ6Tk5OQl5bxGpmoouwwRhCrxmGR5cZjbb3TPLaqeVJESkytAyTKlFASUiVYaWYUotCigRqTK0DFNqUUCJSJWhZZhSiwJKRKoMLcOUWnTDQhGpUrQMU+pQD0pERJKSAkpEJEnoIuJ9aYhPRCQJFL2IeNWqsA1Vd0hSPSgRkSSgi4iLU0CJiCQBXURcnAJKRCQJ6CLi4hRQIiJJQBcRF6eAEhFJArqIuDjN4hMRSRK6iHhf6kGJiEiZEnGNlnpQIiJSqkRdo6UelIiIlCpR12gpoEREpFSJukZLASUiIqVK1DVaCigRESlVoq7RUkCJiEipEnWNlmbxiYhImRJxjZZ6UCIikpQUUCIikpQUUCIikpRiCigzG2BmS8xsmZmNivL8zWa2yMwWmNm/zKxV/EsVEZGqpMyAMrPqwDjgTKAjMMzMOhZpNhfIdPeuwBTgoXgXKiIiVUssPaiewDJ3X+7uO4FJwODCDdx9urvnL4TxP6BFfMsUEZGqJpZp5s2BNYW2c4ETSml/GfBmtCfMbCQQWWKQrWa2JJYiU1QTYGOii0hB+tzKT5/Z/tHntn/i8bnFdBooloCyKPs8akOz4UAm0Dfa8+4+HhgfS2Gpzsxy3D0z0XWkGn1u5afPbP/oc9s/B/NziyWgcoGjC223ANYWbWRm/YE7gL7u/n18yhMRkaoqlnNQs4C2ZpZuZrWAocDUwg3MrAfwFDDI3b+Mf5kiIlLVlBlQ7p4HXAtMAxYDk919oZnda2aDIs0eBuoBfzezeWY2tYTDVSVVYiizAuhzKz99ZvtHn9v+OWifm7lHPZ0kIiKSUFpJQkREkpICSkREkpICKs7M7Ggzm25mi81soZndkOiaUoWZVTezuWb2WqJrSRVm1tDMppjZJ5H/5nonuqZUYGY3Rf7//NjMJppZ7UTXlGzM7Gkz+9LMPi6073Aze9vMlka+N6rIGhRQ8ZcH/MLdOwC9gGuiLA0l0d1AmIgjsfsD8Ja7twe6oc+vTGbWHLiesDxbZ6A6YXay7OtZYECRfaOAf7l7W+Bfke0Ko4CKM3df5+5zIo+/JfzCaJ7YqpKfmbUAzgb+lOhaUoWZHQacDPwZwN13uvvmxFaVMmoAh5pZDaAOUa7trOrc/T3gqyK7BwPPRR4/B5xTkTUooCqQmaUBPYAZia0kJTwG3ArsSXQhKaQ1sAF4JjI0+iczq5voopKdu38OjAFWA+uALe7+j8RWlTJ+4O7rIPwxDhxRkW+mgKogZlYPeAG40d2/SXQ9yczMfgx86e6zE11LiqkBZABPunsPYBsVPORSGUTOmwwG0oGjgLqRZdokySigKoCZ1SSEU7a7v5joelJAH2CQma0krJZ/qplNSGxJKSEXyHX3/B76FEJgSen6AyvcfYO77wJeBH6Y4JpSxXozOxIg8r1CVw5SQMWZmRnhnMBid38k0fWkAnf/lbu3cPc0wsnqd9xdf9GWwd2/ANaYWbvIrtOARQksKVWsBnqZWZ3I/6+nocklsZoKXBJ5fAnwSkW+WSyLxUr59AF+CnxkZvMi+2539zcSWJNUXtcB2ZF1MpcDlya4nqTn7jPMbAowhzDrdi5a9qgYM5sI9AOamFkucBfwADDZzC4jBP1PKrQGLXUkIiLJSEN8IiKSlBRQIiKSlBRQIiKSlBRQIiKSlBRQIiKSlBRQIiKSlBRQIiKSlP4/iihD5l7/rUgAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "acc = history.history['acc']\n",
    "val_acc = history.history['val_acc']\n",
    "loss = history.history['loss']\n",
    "val_loss = history.history['val_loss']\n",
    "\n",
    "epochs = range(1, len(acc) + 1)\n",
    "\n",
    "plt.plot(epochs, acc, 'bo', label='Training acc')\n",
    "plt.plot(epochs, val_acc, 'b', label='Validation acc')\n",
    "plt.title('Training and validation accuracy')\n",
    "plt.legend()\n",
    "\n",
    "plt.figure()\n",
    "\n",
    "plt.plot(epochs, loss, 'bo', label='Training loss')\n",
    "plt.plot(epochs, val_loss, 'b', label='Validation loss')\n",
    "plt.title('Training and validation loss')\n",
    "plt.legend()\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## CNN과 RNN을 연결하여 긴 시퀀스를 처리하기\n",
    "\n",
    "1D 컨브넷이 입력 패치를 독립적으로 처리하기 때문에 RNN과 달리 (합성곱 윈도우 크기의 범위를 넘어선) 타임스텝의 순서에 민감하지 않습니다. 물론 장기간 패턴을 인식하기 위해 많은 합성곱 층과 풀링 층을 쌓을 수 있습니다. 상위 층은 원본 입력에서 긴 범위를 보게 될 것입니다. 이런 방법은 순서를 감지하기엔 부족합니다. 온도 예측 문제에 1D 컨브넷을 적용하여 이를 확인해 보겠습니다. 이 문제는 순서를 감지해야 좋은 예측을 만들어 낼 수 있습니다. 다음은 이전에 정의한 float_data, train_gen, val_gen, val_steps를 다시 사용합니다:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import numpy as np\n",
    "\n",
    "data_dir = './datasets/jena_climate/'\n",
    "fname = os.path.join(data_dir, 'jena_climate_2009_2016.csv')\n",
    "\n",
    "f = open(fname)\n",
    "data = f.read()\n",
    "f.close()\n",
    "\n",
    "lines = data.split('\\n')\n",
    "header = lines[0].split(',')\n",
    "lines = lines[1:]\n",
    "\n",
    "float_data = np.zeros((len(lines), len(header) - 1))\n",
    "for i, line in enumerate(lines):\n",
    "    values = [float(x) for x in line.split(',')[1:]]\n",
    "    float_data[i, :] = values\n",
    "    \n",
    "mean = float_data[:200000].mean(axis=0)\n",
    "float_data -= mean\n",
    "std = float_data[:200000].std(axis=0)\n",
    "float_data /= std\n",
    "\n",
    "def generator(data, lookback, delay, min_index, max_index,\n",
    "              shuffle=False, batch_size=128, step=6):\n",
    "    if max_index is None:\n",
    "        max_index = len(data) - delay - 1\n",
    "    i = min_index + lookback\n",
    "    while 1:\n",
    "        if shuffle:\n",
    "            rows = np.random.randint(\n",
    "                min_index + lookback, max_index, size=batch_size)\n",
    "        else:\n",
    "            if i + batch_size >= max_index:\n",
    "                i = min_index + lookback\n",
    "            rows = np.arange(i, min(i + batch_size, max_index))\n",
    "            i += len(rows)\n",
    "\n",
    "        samples = np.zeros((len(rows),\n",
    "                           lookback // step,\n",
    "                           data.shape[-1]))\n",
    "        targets = np.zeros((len(rows),))\n",
    "        for j, row in enumerate(rows):\n",
    "            indices = range(rows[j] - lookback, rows[j], step)\n",
    "            samples[j] = data[indices]\n",
    "            targets[j] = data[rows[j] + delay][1]\n",
    "        yield samples, targets\n",
    "        \n",
    "lookback = 1440\n",
    "step = 6\n",
    "delay = 144\n",
    "batch_size = 128\n",
    "\n",
    "train_gen = generator(float_data,\n",
    "                      lookback=lookback,\n",
    "                      delay=delay,\n",
    "                      min_index=0,\n",
    "                      max_index=200000,\n",
    "                      shuffle=True,\n",
    "                      step=step, \n",
    "                      batch_size=batch_size)\n",
    "val_gen = generator(float_data,\n",
    "                    lookback=lookback,\n",
    "                    delay=delay,\n",
    "                    min_index=200001,\n",
    "                    max_index=300000,\n",
    "                    step=step,\n",
    "                    batch_size=batch_size)\n",
    "test_gen = generator(float_data,\n",
    "                     lookback=lookback,\n",
    "                     delay=delay,\n",
    "                     min_index=300001,\n",
    "                     max_index=None,\n",
    "                     step=step,\n",
    "                     batch_size=batch_size)\n",
    "\n",
    "# 전체 검증 세트를 순회하기 위해 val_gen에서 추출할 횟수\n",
    "val_steps = (300000 - 200001 - lookback) // batch_size\n",
    "\n",
    "# 전체 테스트 세트를 순회하기 위해 test_gen에서 추출할 횟수\n",
    "test_steps = (len(float_data) - 300001 - lookback) // batch_size"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/20\n",
      "500/500 [==============================] - 10s 21ms/step - loss: 0.4194 - val_loss: 0.4395\n",
      "Epoch 2/20\n",
      "500/500 [==============================] - 10s 20ms/step - loss: 0.3637 - val_loss: 0.4604\n",
      "Epoch 3/20\n",
      "500/500 [==============================] - 10s 20ms/step - loss: 0.3391 - val_loss: 0.4559\n",
      "Epoch 4/20\n",
      "500/500 [==============================] - 10s 20ms/step - loss: 0.3223 - val_loss: 0.4638\n",
      "Epoch 5/20\n",
      "500/500 [==============================] - 10s 20ms/step - loss: 0.3080 - val_loss: 0.4446\n",
      "Epoch 6/20\n",
      "500/500 [==============================] - 10s 19ms/step - loss: 0.3008 - val_loss: 0.4486\n",
      "Epoch 7/20\n",
      "500/500 [==============================] - 10s 20ms/step - loss: 0.2921 - val_loss: 0.4983\n",
      "Epoch 8/20\n",
      "500/500 [==============================] - 10s 20ms/step - loss: 0.2865 - val_loss: 0.4869\n",
      "Epoch 9/20\n",
      "500/500 [==============================] - 10s 20ms/step - loss: 0.2787 - val_loss: 0.4539\n",
      "Epoch 10/20\n",
      "500/500 [==============================] - 10s 20ms/step - loss: 0.2741 - val_loss: 0.4568\n",
      "Epoch 11/20\n",
      "500/500 [==============================] - 10s 20ms/step - loss: 0.2708 - val_loss: 0.4796\n",
      "Epoch 12/20\n",
      "500/500 [==============================] - 10s 20ms/step - loss: 0.2669 - val_loss: 0.4892\n",
      "Epoch 13/20\n",
      "500/500 [==============================] - 10s 20ms/step - loss: 0.2615 - val_loss: 0.4874\n",
      "Epoch 14/20\n",
      "500/500 [==============================] - 10s 20ms/step - loss: 0.2575 - val_loss: 0.4907\n",
      "Epoch 15/20\n",
      "500/500 [==============================] - 10s 20ms/step - loss: 0.2564 - val_loss: 0.4733\n",
      "Epoch 16/20\n",
      "500/500 [==============================] - 10s 19ms/step - loss: 0.2535 - val_loss: 0.5018\n",
      "Epoch 17/20\n",
      "500/500 [==============================] - 10s 20ms/step - loss: 0.2503 - val_loss: 0.4870\n",
      "Epoch 18/20\n",
      "500/500 [==============================] - 10s 20ms/step - loss: 0.2470 - val_loss: 0.4639\n",
      "Epoch 19/20\n",
      "500/500 [==============================] - 10s 20ms/step - loss: 0.2444 - val_loss: 0.4718\n",
      "Epoch 20/20\n",
      "500/500 [==============================] - 10s 20ms/step - loss: 0.2447 - val_loss: 0.4726\n"
     ]
    }
   ],
   "source": [
    "from keras.models import Sequential\n",
    "from keras import layers\n",
    "from keras.optimizers import RMSprop\n",
    "\n",
    "model = Sequential()\n",
    "model.add(layers.Conv1D(32, 5, activation='relu',\n",
    "                        input_shape=(None, float_data.shape[-1])))\n",
    "model.add(layers.MaxPooling1D(3))\n",
    "model.add(layers.Conv1D(32, 5, activation='relu'))\n",
    "model.add(layers.MaxPooling1D(3))\n",
    "model.add(layers.Conv1D(32, 5, activation='relu'))\n",
    "model.add(layers.GlobalMaxPooling1D())\n",
    "model.add(layers.Dense(1))\n",
    "\n",
    "model.compile(optimizer=RMSprop(), loss='mae')\n",
    "history = model.fit_generator(train_gen,\n",
    "                              steps_per_epoch=500,\n",
    "                              epochs=20,\n",
    "                              validation_data=val_gen,\n",
    "                              validation_steps=val_steps)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "다음은 훈련 MAE와 검증 MAE입니다:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAAEYCAYAAAAJeGK1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3Xl8VNX5x/HPI7KIoCCgIltA0QoYIEbEirJULW6oqBWEuhdxr7b9ScV9qXWnLtVSq1WJUqtV0Wq1KorYqkRlERQFBIlQRJQdhMDz++PcwBAmyQQmmTvJ9/16zWvm3rnLmclknjnnPPccc3dERETiZodMF0BERCQZBSgREYklBSgREYklBSgREYklBSgREYklBSgREYklBSiJBTOrY2YrzaxtOrfNJDPbx8zSfh2HmR1hZnMTlmea2WGpbLsN53rYzK7a1v3LOe7NZvbXdB9XapYdM10AyU5mtjJhsSHwA7AhWj7f3Qsqczx33wA0Sve2tYG775eO45jZecBQd++TcOzz0nFskW2hACXbxN03BYjoF/p57v56Wdub2Y7uXlwdZRORmkFNfFIloiacv5nZU2a2AhhqZoeY2XtmttTMFprZvWZWN9p+RzNzM8uJlsdEz79iZivM7L9m1r6y20bPH21mn5vZMjO7z8zeNbOzyih3KmU838xmmdn3ZnZvwr51zOweM1tiZrOB/uW8P1eb2dhS6x4ws7ujx+eZ2afR65kd1W7KOlaRmfWJHjc0syeisk0HDkxy3jnRcaeb2YBo/QHA/cBhUfPptwnv7fUJ+w+PXvsSM3vezFqm8t5UxMxOjMqz1MzeNLP9Ep67yswWmNlyM/ss4bX2NLOPovWLzOyOVM8nWcLdddNtu27AXOCIUutuBtYBxxN+CO0EHAQcTKi5dwA+By6Ott8RcCAnWh4DfAvkA3WBvwFjtmHb3YEVwAnRc1cA64GzyngtqZTxBWBXIAf4ruS1AxcD04HWQDNgQvgXS3qeDsBKYOeEY38D5EfLx0fbGNAPWAPkRs8dAcxNOFYR0Cd6fCfwFtAUaAfMKLXtz4CW0d/k9KgMe0TPnQe8VaqcY4Dro8dHRWXsBjQA/gi8mcp7k+T13wz8NXq8f1SOftHf6Krofa8LdAbmAXtG27YHOkSPJwGDo8eNgYMz/b+gW3pvqkFJVZro7i+6+0Z3X+Puk9z9fXcvdvc5wGigdzn7P+Puhe6+HiggfDFWdtvjgMnu/kL03D2EYJZUimW81d2XuftcQjAoOdfPgHvcvcjdlwC/L+c8c4BPCIET4EhgqbsXRs+/6O5zPHgTeANImghRys+Am939e3efR6gVJZ73aXdfGP1NniT8uMhP4bgAQ4CH3X2yu68FRgC9zax1wjZlvTflGQSMc/c3o7/R74FdCD8UignBsHPUTPxl9N5B+KHR0cyaufsKd38/xdchWUIBSqrS/MQFM/uRmf3TzP5nZsuBG4Hm5ez/v4THqyk/MaKsbfdKLIe7O6HGkVSKZUzpXIRf/uV5EhgcPT6dEFhLynGcmb1vZt+Z2VJC7aW896pEy/LKYGZnmdmUqCltKfCjFI8L4fVtOp67Lwe+B1olbFOZv1lZx91I+Bu1cveZwK8If4dvoibjPaNNzwY6ATPN7AMzOybF1yFZQgFKqlLpFOs/EWoN+7j7LsC1hCasqrSQ0OQGgJkZW36hlrY9ZVwItElYrigN/m/AEVEN5ARCwMLMdgKeAW4lNL81AV5LsRz/K6sMZtYBeBC4AGgWHfezhONWlBK/gNBsWHK8xoSmxK9TKFdljrsD4W/2NYC7j3H3QwnNe3UI7wvuPtPdBxGace8CnjWzBttZFokRBSipTo2BZcAqM9sfOL8azvkSkGdmx5vZjsBlQIsqKuPTwC/NrJWZNQOuLG9jd18ETAQeBWa6+xfRU/WBesBiYIOZHQf8pBJluMrMmli4TuzihOcaEYLQYkKsPo9QgyqxCGhdkhSSxFPAuWaWa2b1CYHiHXcvs0ZaiTIPMLM+0bl/Q+g3fN/M9jezvtH51kS3DYQX8HMzax7VuJZFr23jdpZFYkQBSqrTr4AzCV8+fyLUIKpUFAROA+4GlgB7Ax8TrttKdxkfJPQVTSN04D+Twj5PEpIenkwo81LgcuA5QqLBKYRAm4rrCDW5ucArwOMJx50K3At8EG3zIyCx3+bfwBfAIjNLbKor2f9fhKa256L92xL6pbaLu08nvOcPEoJnf2BA1B9VH7id0G/4P0KN7epo12OATy1kid4JnObu67a3PBIfFprkRWoHM6tDaFI6xd3fyXR5RKRsqkFJjWdm/c1s16iZ6BpCZtgHGS6WiFRAAUpqg17AHEIzUX/gRHcvq4lPRGJCTXwiIhJLqkGJiEgsxW6w2ObNm3tOTk6miyEiIlXkww8//Nbdy7vcA4hhgMrJyaGwsDDTxRARkSpiZhWNsgKoiU9ERGJKAUpERGJJAUpERGIpdn1Qyaxfv56ioiLWrl2b6aJICho0aEDr1q2pW7esId1ERCqWFQGqqKiIxo0bk5OTQxiMWuLK3VmyZAlFRUW0b9++4h1ERMqQFU18a9eupVmzZgpOWcDMaNasmWq7IrLdsiJAAQpOWUR/KxFJh5QCVDTY5kwzm2VmI5I8f5aZLTazydHtvITnzjSzL6LbmeksvIiI1FwVBqhoeoIHgKMJ0ysPNrNOSTb9m7t3i24PR/vuRpif5mCgB3CdmTVNW+mryZIlS+jWrRvdunVjzz33pFWrVpuW161LbfqZs88+m5kzZ5a7zQMPPEBBQUG526SqV69eTJ48OS3HEqkNli+Hs86CZ1KZxUuqRSo1qB7ALHefE00GNpYwPXUqfgr8292/c/fvCROi9d+2oqauoABycmCHHcL99n7nN2vWjMmTJzN58mSGDx/O5Zdfvmm5Xr16QEgO2Lix7Mk8H330Ufbbb79yz3PRRRcxZMh2z/8mIpXkDuecA489BqeeCoMGwZIlmS6VpBKgWgHzE5aLonWlnWxmU83sGTNrU5l9zWyYmRWaWeHixYtTLHpyBQUwbBjMmxc+dPPmheU0VUy2MGvWLLp06cLw4cPJy8tj4cKFDBs2jPz8fDp37syNN964aduSGk1xcTFNmjRhxIgRdO3alUMOOYRvvvkGgKuvvppRo0Zt2n7EiBH06NGD/fbbj//85z8ArFq1ipNPPpmuXbsyePBg8vPzK6wpjRkzhgMOOIAuXbpw1VVXAVBcXMzPf/7zTevvvfdeAO655x46depE165dGTp0aNrfM5E4GjUKnn0Wbr0VbroJ/vEP6NwZxo3LdMlqt1QCVLIe79JzdLwI5Lh7LvA68Fgl9sXdR7t7vrvnt2hR4fiB5Ro5Elav3nLd6tVhfVWYMWMG5557Lh9//DGtWrXi97//PYWFhUyZMoV///vfzJgxY6t9li1bRu/evZkyZQqHHHIIjzzySNJjuzsffPABd9xxx6Zgd99997HnnnsyZcoURowYwccff1xu+YqKirj66qsZP348H3/8Me+++y4vvfQSH374Id9++y3Tpk3jk08+4YwzzgDg9ttvZ/LkyUyZMoX7779/O9+d1Nx4I5xwAkyaVC2nE9nCu+/C//0fnHQSXHklXH11+CzusUf4XJ51FixdmulS1k6pBKgioE3CcmvClNmbuPuShAng/gwcmOq+6fbVV5Vbv7323ntvDjrooE3LTz31FHl5eeTl5fHpp58mDVA77bQTRx99NAAHHnggc+fOTXrsgQMHbrXNxIkTGTRoEABdu3alc+fO5Zbv/fffp1+/fjRv3py6dety+umnM2HCBPbZZx9mzpzJZZddxquvvsquu+4KQOfOnRk6dCgFBQXVcqHt//4Ht9wCL74IPXrAiSfCtGlVfloRAL75Bn72s9AV8OijUJKA2rVrCFJXXw1jxkCXLvDqqxktaq2USoCaBHQ0s/ZmVg8YBGxR8TWzlgmLA4BPo8evAkeZWdMoOeKoaF2Vadu2cuu3184777zp8RdffMEf/vAH3nzzTaZOnUr//v2TXg9U0m8FUKdOHYqLi5Meu379+lttU9kJJsvavlmzZkydOpVevXpx7733cv755wPw6quvMnz4cD744APy8/PZsGFDpc5XWQ8+COvXQ2FhqEmNHx++HAYPhs8/r9JTSy23YUP4nH33XUiMiH6jbVKvXmju++9/YZddoH//0F2wYkVmylsbVRig3L0YuJgQWD4Fnnb36WZ2o5kNiDa71Mymm9kU4FLgrGjf74CbCEFuEnBjtK7K3HILNGy45bqGDcP6qrZ8+XIaN27MLrvswsKFC3m1Cn5y9erVi6effhqAadOmJa2hJerZsyfjx49nyZIlFBcXM3bsWHr37s3ixYtxd0499VRuuOEGPvroIzZs2EBRURH9+vXjjjvuYPHixawu3V6aRmvXhgB13HGQlwfXXANffgkjRoS2/06d4NxzQz+iSLpdey28+Wb4DHbtWvZ2Bx0EH30Ev/kNPPww5OaGH1JS9VIa6sjdXwZeLrXu2oTHvwV+W8a+jwDJO1mqQEkS3MiRoVmvbdsQnKojOS4vL49OnTrRpUsXOnTowKGHHpr2c1xyySWcccYZ5ObmkpeXR5cuXTY1zyXTunVrbrzxRvr06YO7c/zxx3Psscfy0Ucfce655+LumBm33XYbxcXFnH766axYsYKNGzdy5ZVX0rhx47S/hhIFBbB4MVx++eZ1u+0Gv/sdXHZZ6LB+8EF44gk4/3y46ipo2bLs44mk6qWXwufsvPNCH1NFGjSA228PTdBnnQX9+sEll4TPaEIjiqSbu8fqduCBB3ppM2bM2GpdbbV+/Xpfs2aNu7t//vnnnpOT4+vXr89wqbZW0d9s40b3zp3du3YNj8vy1Vfuv/iFe5067jvt5P6b37h/+22aCyu1ypw57k2auHfv7h79K1XKqlXul17qDu777OM+cWL6y1jTAYWeQjzImqGOJFi5ciWHHnooXbt25eSTT+ZPf/oTO+6YFWP+buH112H69FB7Km9kpDZtYPRo+OwzOPlkuPNOaN8err8+XFgp2WXFCvjwQ3jqqfA3vPhimDWr+s6/di2cckp4/MwzoWZUWQ0bwh/+EJr5iovhsMNC85+Gn0w/80p2ule1/Px8Lz3l+6effsr++++foRLJtqjob3bsseGLat48iHJBUjJ9Olx3XbhmZbfdQlrwRRepmSUVixbBW2+FDv8WLaB583DfsGH5PxIqa/16mDsXZs4MiS6J9wsXbt7ODOrWDed/8kmIElur1Pnnhx8848bB8cdv//FWrAjB6U9/gv33Dxf6JiT1ShnM7EN3z69ou+z76S1Z77PP4OWX4YYbKhecIFw8+cwzIbhdc00IUHffHfochw2r/PFqg8mTw4WoTz0FyUbmatBgy4BV0f1uu4VRWr75ZsvgU/J49uxQsyjRrBnstx/89Kfhft99w/3ee8OCBTBwYPjBctNNoZ+xqsYafvzxEJxGjEhPcAJo3Bgeeii8hnPPhUMOCce/9tqQBSjbRzUoqRLl/c0uuCBcc/LVV7D77tt3nokTw7Uqb78dEmLuumtzE05ttmFDuLZs1Kjw3jRsCGefDWeeGZ5bvBi+/bb8+7KaUM1CUFuzZvO6+vWhY8fNwSfxvlmz8su6enVIVnjqqXCx7GOPhS/+dJo2DQ4+GHr2hNdeg6poFV+6NDRZ//WvIdPv6afDeyBbUw1KYmnJkvAFNHTo9gcngF69Ql/AG2+E0QAGD4Z99oFu3bb/2Nlo2TJ45BG4776Qst+uHdxxR/h137SSwzSvWxeCVUnASgxeK1eGi1tLAlGbNlCnzraVuWHDkNF50EGhuezgg+G559L35b5sWei/bNIkBMGq6rJt0iT88Dr55DCuX9++8M47oaYo2yiVTIrqvCmLr2Yo62/2u9+F7Kdp09J/ziVL3PfYw71bN/d169J//Dj74ouQWdaoUXh/e/Vyf+YZ9xgmeJbrzTfdmzd332UX93Hjtv94Gze6DxwYskAnTNj+46Vq2jT3Zs3c27ULmaiyJZTFlz59+vTZ6qLbUaNGceGFF5a7X6NGjQBYsGABp5TR7tSnTx9KN2mWNmrUqC0umD3mmGNYmobBwa6//nruvPPO7T5Oqtatg/vvhyOPDEPHpNtuu4X+gMmT4fe/T//x48Y9XGg6YECoyTz4YGgiKywMv9xPPrnqagtVpW/f0L+4zz7hdd1wA5QzSUCF7rknDPx6220h2666lAyN9P33cMQRIUFFKk8BKgWDBw9m7NixW6wbO3YsgwcPTmn/vfbai2e2Y5KZ0gHq5ZdfpkmTJtt8vEz5+99Dp3jihbnpduKJoZnvpptg6tSqO08mrVkDf/lLGP3gJz8JQ/FcfXXIiHz8cTjwwIqPEWdt24a+xTPOCKnoJ54Ymukqa+LE0Ow7cCBccUXai1mhAw+Ef/4TiorCj7LvqnQMnZpJASoFp5xyCi+99BI//BDGw507dy4LFiygV69erFy5kp/85Cfk5eVxwAEH8MILL2y1/9y5c+kSVRnWrFnDoEGDyM3N5bTTTmNNQk/zBRdcsGmqjuuuuw6Ae++9lwULFtC3b1/69u0LQE5ODt9++y0Ad999N126dKFLly6bpuqYO3cu+++/P7/4xS/o3LkzRx111BbnSWby5Mn07NmT3NxcTjrpJL7//vtN5+/UqRO5ubmbBql9++23N03Y2L17d1akMDiZe/g1+6MfhWyuqnTvvaG/5eyzQ8pzTbFgQchcbNs2JBWYhf6m+fPDOIY1aZSNnXYKyQb33QevvBIGEq5gVK8tLFoUBoFt3z68R1WVGViRXr3ghRdChuPRR+vavUpLpR2wOm8V9UFddpl7797pvV12WcVtpsccc4w///zz7u5+6623+q9//Wt3DyM7LFu2zN3dFy9e7HvvvbdvjIZG2Hnnnd3d/csvv/TOnTu7u/tdd93lZ599tru7T5kyxevUqeOTJk1yd/clS5a4u3txcbH37t3bp0yZ4u7u7dq188WLF28qS8lyYWGhd+nSxVeuXOkrVqzwTp06+UcffeRffvml16lTxz/++GN3dz/11FP9iSee2Oo1XXfddX7HHXe4u/sBBxzgb731lru7X3PNNX5Z9Ka0bNnS165d6+7u33//vbu7H3fccT4xunx+xYoVSUeyKN0HNWFC6Bt56KEK3ug0efbZcL6bb66e81Wl+fPdhwxxr1vX3cx9wIDQV1PeCBw1yYQJ7rvvHvrXnn224u3Xr3fv2zeMPBL9C2XcCy+EfrDDDw8jUdR2qA8qvRKb+RKb99ydq666itzcXI444gi+/vprFpXT4DxhwoRNEwHm5uaSm5u76bmnn36avLw8unfvzvTp0yscCHbixImcdNJJ7LzzzjRq1IiBAwfyzjvvANC+fXu6Rals5U3pAWF+qqVLl9K7d28AzjzzTCZMmLCpjEOGDGHMmDGbRqw49NBDueKKK7j33ntZunRpSiNZ3HNP6CP6+c8r3DQtBg6E004LfRiffFI956wKCxZAnz4hq+3CC+GLL8Iv8r59M1crqG6HHRb6pTp1Cv1qI0eGVPmyXHttyOx88MGQ7h0HAwaEaTtK+gZ/+KHifSQL08yjVqxqd+KJJ3LFFVfw0UcfsWbNGvLy8gAoKChg8eLFfPjhh9StW5ecnJykU2wksiTfLF9++SV33nknkyZNomnTppx11lkVHsfLuYatfsIVq3Xq1Kmwia8s//znP5kwYQLjxo3jpptuYvr06YwYMYJjjz2Wl19+mZ49e/L666/zox/9qMxjzJkDzz8fLmAsPdJ8VbrvvpBEcPbZoZ8m2xIGvvkm9DEtWhTS6Hv2zHSJMqd1a5gwIQyN9LvfhdHFn3xy69T5F18MA7j+4hfhmq84GTQIVq0KzbODB4frpLLtM1ndVINKUaNGjejTpw/nnHPOFskRy5YtY/fdd6du3bqMHz+eeRXMDXH44YdTEM0//8knnzA16slfvnw5O++8M7vuuiuLFi3ilVde2bRP48aNk/bzHH744Tz//POsXr2aVatW8dxzz3HYNqQq7brrrjRt2nRT7euJJ56gd+/ebNy4kfnz59O3b19uv/12li5dysqVK5k9ezYHHHAAV155Jfn5+Xz22WflHv+++8I1MhddVOmibZcWLeCBB0JWWzUmK6bFd9+FjvV580JHe20OTiXq14c//zkMK/TGG+G6qcTJLefMCYkVeXmhHzKOzj03/Mh+7rnww2l7MhRrA8XvShg8eDADBw7cIqNvyJAhHH/88eTn59OtW7dyaxIQEiHOPvtscnNz6datGz169ADC7Ljdu3enc+fOW03VMWzYMI4++mhatmzJ+ISJaPLy8jjrrLM2HeO8886je/fu5TbnleWxxx5j+PDhrF69mg4dOvDoo4+yYcMGhg4dyrJly3B3Lr/8cpo0acI111zD+PHjqVOnDp06ddo0O3Ayy5eHjLPTToNWrSpdrO126qlhZInrrgvNLJ06VX8ZKmvZspBI8tlnYVqIww/PdIniZdgwOOCA0FTWs2e4OHbAgPC3hm0fBLa6XHZZqEmNHBnGkHzwwdrTXFtpqXRUVedNF+rWDCV/s7vvDskKhYWZK8uiReGiyR494n/h6sqV7oce6r7jjum5ULUmW7DA/cc/Dp+v3Nxw/+KLmS5V6kaMCGX+1a9qT8JLCVJMklANSqrMhg2hqeWwwzJ7bc7uu4emvkGDwsCy//d/mStLedasCTWB//4Xxo5N34CmNVXLliEZ4pe/DLWQ3/42zM6cLX73uzBk1F13hbEHoytLqsQPP4SLhtesCdOCrF27+XFl1+XlhSGpqoMClFSZ558P0y7cfXemSxKuifnb30KG14AB4XqsOFm3LjRFjh8fLrYtaa6S8tWrB3/8Y7gQN9vGvDML80qtWhUuSN55Z/j1r9N3/A0bwudpzJgwPc3KlZU/xo47hmvSGjTYfN+iRfrKWOH5q+9U28fdk2a/Sfx4lF14zz3QoUMICJlmFr7IOncOndMTJ2774KbpVlwcancvvxwSAKKrEKQS9tkn0yXYNjvsEBI/Vq0KtZJGjWD48O075pQpISg9+WS4TGHXXcPnKy9v62BT+r7040xnGWZFgGrQoAFLliyhWbNmClIx5+4sWbKEtWsb8O67IWMpLoFgzz1DRuGQIaFcv/pVpksUfuWeeWbI6vrDH0ICgNQuderAE0+EaUcuvDDUpCp7vWBRUQhIY8aEzMYdd4RjjgnHOe64eCeNlCcr5oNav349RUVFFV4XJPHQoEEDbr65NX//e12KisIMrnHhHgZUffXVMKhsJufr2bgxBKS//CUMbnvllZkri2Te2rVh4sa33grjVg4cWP72y5eHprsxY0JTnnuYMHHo0NCk3bx5tRR7m6Q6H1TGs/ZK35Jl8Ul2mT8/ZKFdfnmmS5LcggXuTZuGDLDi4syUYeNG94svDllc116bmTJI/KxYET6Xdeu6v/LK1s+vWxcyFU87zb1Bg/D52Xtv9+uvD1OuZAs01JFkyv33h9rBpZdmuiTJtWwZsgv/85/Q5Ffd3ENt6f77Q6f49ddXfxkknho1Chdmd+kSavpvvx0+L++/D5dcAnvtFbI7X389XPT73/+G4a+uuy57++HKkxVNfBI6Ud94I0yx0K5dpktTtlWrwuyq/fqFCybjyj0kb7zxRuhU7tix+s59/fVhjMCLLgoBUt2qUtrixdC7dxipfq+94PPPw0gaAwaEfqWf/jRkMGarVJv4VIOKseJieO218IHcYw844YSQiTNxYqZLVrbHHw/XW1TlnE/pYBYy5urXD9NzV9eQM7fdFoLTOeeEWpyCkyTTokWoJXXsGGr8Dz8cxmR8+ulQg8rm4FQZClAx4w4ffxyu62jTJvxSeumlkHn2/PPhg3vEEeGanrjZuDFkxx10EPz4x5kuTcX22iuUd+LE0NxW1e67LwyYO3gwjB4dUoxFyrLXXmFQ3LfeCs15u+6a6RJVv6xIM8+EtWvDdTz16kF+fqi5NG5cdef76isoKAgZOTNmQN26IT106NCQLlqSJnrYYWGG0UGDwkCiv/lNfH6Fv/JKaIp48sn4lKkiZ5wRfpWOGBHe56pqx3/44dAnd9JJ8Nhj8Um9F4kz9UElsWpVaE57443N68zC6AP5+aGGkJ8P3bqFi9q21dKloZ9mzJjQGQphBs6hQ8NIArvtlny/tWvDxaZjx8IFF4SmokxfUAehZvfZZ/DllyHAZouvvw4X8HbtGtJ1012zGTMmBML+/cP1TgkzoYjUSkoz30bffx/SPHfYwf2vfw0Djb78svsNN7gff7z7nnuG1E4IM2R27ep+7rnuDz7oPmmS+w8/lH/8H35wf/5591NOca9fPxxn333db7rJffbs1Mu5YYP7lVeG/Y87LgwymklTpoSy3HprZsuxrf7yl1D+++9P73H//vfwWerXz3316vQeWyRbkWKauWpQCb79Fo46KszA+uSTYWy0ZL7+OswxVFgIkyaF+yVLwnP16oVZPPPzN9e29t8/bDdmTOg7+u670Jc0eHCoLeXnb3uT2EMPhWyw7t1DX9Wee27bcbbXOeeE1zZ/ftk1vzhzh6OPDv1RU6eGIZoq6/vvYfbsMC/R7Nkwa1ZIGjn4YPjXv0IKsYikXoNSgIosWBAmiJszB/7xj/BllSr30B+UGLAKC8OV3hD6GzZsCM2BJ54YgtKRR6avGeyll8J8Sy1ahH6g/fdPz3FTtWgRtG0bOnL/+MfqPXc6zZ8fmvoOPDA075Zu6tuwIQwpkxiEEh8vXbrl9nvsEZJFHn20dnZwi5RFAaoS5s4NU2t/802YMrpPn+0/5saN4Rd0YWG4zmb//cPQJVU17E9hYUiq+OGHkO3Xu3fVnCeZkut6Pvsss0MHpcPDD4fpwn/zm1AbTQxAc+fC+vWbt61bF3JyQm1r773DreRx+/aqMYmUJa0Bysz6A38A6gAPu/vvy9juFODvwEHuXmhmOcCnwMxok/fcvdyxeqs7QM2cGTr3V64MzTAHH1xtp067uXNDJtrs2eFX++mnV/05164NtacePUJNLtu5h2SG114Ly7vuunXwKXncpo2y8US2RaoBqsLcLzOrAzwAHAlqDR9WAAAUM0lEQVQUAZPMbJy7zyi1XWPgUuD9UoeY7e7dUi55NZo6NTS1uYdrDbp2zXSJtk9ODrz7bkhlHjIkpK5feWXVpnw/+WS46j3uF+amyiwMwDlzZqgFNW2aPSnzIjVNKgm1PYBZ7j7H3dcBY4ETkmx3E3A7kBVDjn/wQWjKq1cP3nkn+4NTiaZNw0jdp58eZhgdPjyMSFEV3MOFrrm5YWijmqJRo9APtdtuCk4imZRKgGoFzE9YLorWbWJm3YE27p6skae9mX1sZm+b2WHbXtT0efvt0OfUtGkITtneb1Ja/fphfpmrrgojFgwYACtWpP88b74Z5p755S/1RS4i6ZfK5Z3Jvno2dVyZ2Q7APcBZSbZbCLR19yVmdiDwvJl1dvflW5zAbBgwDKBt27YpFn3bvPJKSFZo3z6MdbXXXlV6uozZYQe45ZYwsOyFF4akiZde2vbX6x4yHadO3Xx76y3YffeQLi8ikm6pBKgioE3CcmtgQcJyY6AL8FY02+2ewDgzG+DuhcAPAO7+oZnNBvYFtsiCcPfRwGgISRLb9lIq9uyz4cu0S5fQDNaiRVWdKT6GDQud+aeeCj17hgDduXP5+6xeHYZbmjo1ZCCWBKTvvtu8Tdu2YfinSy7J3tk6RSTeUglQk4COZtYe+BoYBGzKD3P3ZcCmuRvN7C3g11EWXwvgO3ffYGYdgI7AnDSWP2WPPx6GB+rZM8y30qRJJkqRGUcfDRMmhNk6Dz00XOfVr9/m67cSa0VTp4b5ZUpG927YEA44AE4+OfQ15eaG5aZNM/uaRKTmqzBAuXuxmV0MvEpIM3/E3aeb2Y2E4SrGlbP74cCNZlYMbACGu/t35WxfJf74xzDaQr9+8MILtfP6lLw8eO+9kIbev38Y4eKTTzZfTAwhdTo3N1z0m5sbEkc6dNCo2yKSGTX+Qt3bbw+p1scfH0atru3NUUuXwvnnhxERunYNt9zc0OxZlaO1i4iUSNt1UNnKHa69Fm6+OdQInngiu0bYripNmsRzLikRkdJqZIByDxP+jRoVBjEdPVpX/IuIZJsa17uwYUPIXBs1KkwQ9+c/KziJiGSjGhegpk4NM5aOHBmClDr4RUSyU41r4uvePWSn7btvpksiIiLbo0bWLxScRESyX40MUCIikv0UoEREJJYUoEREJJYUoEREJJYUoEREJJYUoEREJJYUoEREJJYUoEREJJYUoEREJJYUoEREJJYUoEREJJZqXIAqKICcnDCKeU5OWBYRkexTo0YzLygIc0GtXh2W580LywBDhmSuXCIiUnk1qgY1cuTm4FRi9eqwXkREskuNClBffVW59SIiEl81KkC1bVu59SIiEl81KkDdcgs0bLjluoYNw3oREckuNSpADRkCo0dDu3ZgFu5Hj1aChIhINqpRWXwQgpECkohI9qtRNSgREak5FKBERCSWFKBERCSWFKBERCSWFKBERCSWFKBERCSWFKBERCSWFKBERCSWUgpQZtbfzGaa2SwzG1HOdqeYmZtZfsK630b7zTSzn6aj0CIiUvNVOJKEmdUBHgCOBIqASWY2zt1nlNquMXAp8H7Cuk7AIKAzsBfwupnt6+4b0vcSRESkJkqlBtUDmOXuc9x9HTAWOCHJdjcBtwNrE9adAIx19x/c/UtgVnQ8ERGRcqUSoFoB8xOWi6J1m5hZd6CNu79U2X2j/YeZWaGZFS5evDilgouISM2WSoCyJOt805NmOwD3AL+q7L6bVriPdvd8d89v0aJFCkUSEZGaLpXRzIuANgnLrYEFCcuNgS7AW2YGsCcwzswGpLCviIhIUqnUoCYBHc2svZnVIyQ9jCt50t2XuXtzd89x9xzgPWCAuxdG2w0ys/pm1h7oCHyQ9lchIiI1ToU1KHcvNrOLgVeBOsAj7j7dzG4ECt19XDn7Tjezp4EZQDFwkTL4REQkFea+VZdQRuXn53thYWGmiyEiIlXEzD509/yKttNIEiIiEksKUCIiEksKUCIiEksKUCIiEksKUCIiEksKUCIiEksKUCIiEksKUCIiEksKUCIiEksKUCIiEksKUCIiEksKUCIiEksKUCIiEksKUCIiEksKUCIiEksKUCIiEksKUCIiEksKUCIiEksKUCIiEksKUCIiEksKUCIiEksKUGUoKICcHNhhh3BfUJDpEomI1C47ZroAcVRQAMOGwerVYXnevLAMMGRI5solIlKbqAaVxMiRm4NTidWrw3oREakeClBJfPVV5daLiEj6KUAl0bZt5daLiEj6KUAlccst0LDhlusaNgzrRUSkeihAJTFkCIweDe3agVm4Hz1aCRIiItVJWXxlGDJEAUlEJJNUgxIRkVhSgBIRkVhSgBIRkVhKKUCZWX8zm2lms8xsRJLnh5vZNDObbGYTzaxTtD7HzNZE6yeb2UPpfgEiIlIzVZgkYWZ1gAeAI4EiYJKZjXP3GQmbPenuD0XbDwDuBvpHz812927pLbaIiNR0qdSgegCz3H2Ou68DxgInJG7g7ssTFncGPH1FFBGR2iiVANUKmJ+wXBSt24KZXWRms4HbgUsTnmpvZh+b2dtmdliyE5jZMDMrNLPCxYsXV6L4IiJSU6USoCzJuq1qSO7+gLvvDVwJXB2tXgi0dffuwBXAk2a2S5J9R7t7vrvnt2jRIvXSi4hIjZVKgCoC2iQstwYWlLP9WOBEAHf/wd2XRI8/BGYD+25bUUVEpDZJJUBNAjqaWXszqwcMAsYlbmBmHRMWjwW+iNa3iJIsMLMOQEdgTjoKLiIiNVuFWXzuXmxmFwOvAnWAR9x9upndCBS6+zjgYjM7AlgPfA+cGe1+OHCjmRUDG4Dh7v5dVbwQERGpWcw9Xgl3+fn5XlhYmOliiIhIFTGzD909v6LtNJJEFSsogJwc2GGHcF9QkOkSiYhkB41mXoUKCmDYsM3Tx8+bF5ZBI6WLiFRENagqNHLk5uBUYvXqsF5ERMqnAFWFvvqqcutFRGQzBagq1LZt5daLiMhmClBV6JZboGHDLdc1bBjWi4hI+RSgqtCQITB6NLRrB2bhfvRoJUiIiKRCWXxVbMgQBSQRkW2hGpSIiMSSApSIiMSSApSIiMSSApSIiMSSApSIiMSSApSIiMSSAlSW0KjoIlLb6DqoLKBR0UWkNlINKgtoVHQRqY0UoLKARkUXkdpIASoLaFR0EamNFKCygEZFF5HaSAEqC2hUdBGpjZTFlyU0KrqI1DaqQYmISCwpQImISCwpQImISCwpQImISCwpQNUyGtNPRLKFsvhqEY3pJyLZRDWoWkRj+olINlGAqkU0pp+IZBMFqFpEY/qJSDZRgKpFNKafiGSTlAKUmfU3s5lmNsvMRiR5friZTTOzyWY20cw6JTz322i/mWb203QWXionnWP6KRtQRKqauXv5G5jVAT4HjgSKgEnAYHefkbDNLu6+PHo8ALjQ3ftHgeopoAewF/A6sK+7byjrfPn5+V5YWLh9r0qqVOlsQAg1MQ1gKyKpMLMP3T2/ou1SqUH1AGa5+xx3XweMBU5I3KAkOEV2Bkqi3gnAWHf/wd2/BGZFx5MspmxAEakOqVwH1QqYn7BcBBxceiMzuwi4AqgH9EvY971S+7ZKsu8wYBhAW/XYx56yAUWkOqRSg7Ik67ZqF3T3B9x9b+BK4OpK7jva3fPdPb9FixYpFEkySdmAIlIdUglQRUCbhOXWwIJyth8LnLiN+0oWUDagiFSHVALUJKCjmbU3s3rAIGBc4gZm1jFh8Vjgi+jxOGCQmdU3s/ZAR+CD7S+2ZJJm+BWR6lBhgHL3YuBi4FXgU+Bpd59uZjdGGXsAF5vZdDObTOiHOjPadzrwNDAD+BdwUXkZfJI9hgyBuXNh48Zwv63BSenqIlKWCtPMq5vSzGsPpauL1E7pTDMXqRJKVxeR8ihAScYoXV1EyqMAJRmTznR19WWJ1DwKUJIx6UpXL+nLmjcP3DdPxKggJZLdFKAkY9KVrq6+LJGaSVl8kvV22CHUnEozC2nwIhIvyuKTWkNDL4nUTApQkvU09JJIzaQAJVlPQy+J1EwKUFIjaOglkZonlfmgRGqF0kMvlaSrg2pjIpmgGpRIROnqIvGiACUS0dBLIvGiACUS0dBLIvGiACUS0dBLIvGiACUSiePQS6qJSW2moY5E0ixdQy9pQkepqTTUkUiGpKsvS1mFUtspQImkWbr6spRVKLWdApRImqWrL0tZhVLbKUCJVIF0DL2krEKp7RSgRGIqjlmFItVJWXwiNZwmdJS4URafiADqy5LspQAlUsOpL0uylQKUSA2nvizJVgpQIrVAOrIK031dlpoLpSIKUCKSknT3ZaWruVCBruZSgBKRlKSrLwvS11yofrGaTQFKRFKSrr4sSF9zofrFajYFKBFJWTr6siB9zYXp7BdTU2H8KECJSLVLV3NhugKdmgrjKaUAZWb9zWymmc0ysxFJnr/CzGaY2VQze8PM2iU8t8HMJke3ceksvIhkp3Q1F6Yr0GmSyXiqcKgjM6sDfA4cCRQBk4DB7j4jYZu+wPvuvtrMLgD6uPtp0XMr3b1RqgXSUEciUhkFBSGQfPVVqDndckvlA50mmaxe6RzqqAcwy93nuPs6YCxwQuIG7j7e3Uv+JO8BrStbYBGRbZGOfjFNMhlPqQSoVsD8hOWiaF1ZzgVeSVhuYGaFZvaemZ24DWUUEalScZxkUk2FsGMK21iSdUnbBc1sKJAP9E5Y3dbdF5hZB+BNM5vm7rNL7TcMGAbQdluu+hMR2Q4lta7tbSps2zYkWCRbXxmlmwpLkjYSy1obpFKDKgLaJCy3BhaU3sjMjgBGAgPc/YeS9e6+ILqfA7wFdC+9r7uPdvd8d89v0aJFpV6AiEg6xGmSyTgmbWSkRufu5d4Itaw5QHugHjAF6Fxqm+7AbKBjqfVNgfrR4+bAF0Cn8s534IEHuohIthozxr1dO3ezcD9mTOWPYeYe0ja2vJlVviwNG255jIYNK1+mdB2nBFDoFcQed09twkIzOwYYBdQBHnH3W8zsxugk48zsdeAAYGG0y1fuPsDMfgz8CdhIqK2Ncve/lHcuZfGJSG2Xk5O8qbBdu1C7y9bjlEg1i08z6oqIxEy60tXTlT6f7lmZNaOuiEiWSteFzOlKn0/nSPaVoQAlIhJDcUraSOdI9pWhACUiUkOlqyaWzpHsK0N9UCIiUq3UByUiIllNAUpERGJJAUpERGJJAUpERGJJAUpERGJJAUpERGIpdmnmZrYYSDLqU6w1B77NdCEqSWWuHipz9cnGctfWMrdz9wqnrohdgMpGZlaYSk5/nKjM1UNlrj7ZWG6VuXxq4hMRkVhSgBIRkVhSgEqP0ZkuwDZQmauHylx9srHcKnM51AclIiKxpBqUiIjEkgKUiIjEkgJUCsysjZmNN7NPzWy6mV2WZJs+ZrbMzCZHt2szUdbSzGyumU2LyrTVPCYW3Gtms8xsqpnlZaKcCeXZL+E9nGxmy83sl6W2yfh7bWaPmNk3ZvZJwrrdzOzfZvZFdN+0jH3PjLb5wszOzHCZ7zCzz6K//XNm1qSMfcv9HFWlMsp9vZl9nfAZOKaMffub2czo8z0iw2X+W0J555rZ5DL2zch7Xdb3XEY/1+6uWwU3oCWQFz1uDHwOdCq1TR/gpUyXNUnZ5wLNy3n+GOAVwICewPuZLnNC2eoA/yNc1Ber9xo4HMgDPklYdzswIno8ArgtyX67AXOi+6bR46YZLPNRwI7R49uSlTmVz1EGyn098OsUPj+zgQ5APWBK6f/b6ixzqefvAq6N03td1vdcJj/XqkGlwN0XuvtH0eMVwKdAq8yWKm1OAB734D2giZm1zHShIj8BZrt77EYWcfcJwHelVp8APBY9fgw4McmuPwX+7e7fufv3wL+B/lVW0ATJyuzur7l7cbT4HtC6OspSGWW816noAcxy9znuvg4YS/gbVbnyymxmBvwMeKo6ypKqcr7nMva5VoCqJDPLAboD7yd5+hAzm2Jmr5hZ52otWNkceM3MPjSzYUmebwXMT1guIj7BdxBl/xPH8b3ew90XQvhnB3ZPsk2c3+9zCLXpZCr6HGXCxVHT5CNlNDvF9b0+DFjk7l+U8XzG3+tS33MZ+1wrQFWCmTUCngV+6e7LSz39EaEpqitwH/B8dZevDIe6ex5wNHCRmR1e6nlLsk/Grz0ws3rAAODvSZ6O63udiri+3yOBYqCgjE0q+hxVtweBvYFuwEJCk1lpsXyvgcGUX3vK6HtdwfdcmbslWbfd77UCVIrMrC7hj1bg7v8o/by7L3f3ldHjl4G6Zta8mou5FXdfEN1/AzxHaPZIVAS0SVhuDSyontKV62jgI3dfVPqJuL7XwKKS5tHo/psk28Tu/Y46tI8DhnjUoVBaCp+jauXui9x9g7tvBP5cRnni+F7vCAwE/lbWNpl8r8v4nsvY51oBKgVRm/FfgE/d/e4yttkz2g4z60F4b5dUXymTlmlnM2tc8pjQIf5Jqc3GAWdE2Xw9gWUl1fkMK/NXZhzf68g4oCR76UzghSTbvAocZWZNo2apo6J1GWFm/YErgQHuvrqMbVL5HFWrUv2kJ5G8PJOAjmbWPqqRDyL8jTLpCOAzdy9K9mQm3+tyvucy97mu7kyRbLwBvQjV1anA5Oh2DDAcGB5tczEwnZAp9B7w4xiUu0NUnilR2UZG6xPLbcADhGynaUB+DMrdkBBwdk1YF6v3mhA8FwLrCb8ezwWaAW8AX0T3u0Xb5gMPJ+x7DjArup2d4TLPIvQdlHyuH4q23Qt4ubzPUYbL/UT0eZ1K+AJtWbrc0fIxhGy02dVZ7mRljtb/teRznLBtLN7rcr7nMva51lBHIiISS2riExGRWFKAEhGRWFKAEhGRWFKAEhGRWFKAEhGRWFKAEhGRWFKAEhGRWPp/ElQvC+z7hV4AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "loss = history.history['loss']\n",
    "val_loss = history.history['val_loss']\n",
    "\n",
    "epochs = range(1, len(loss) + 1)\n",
    "\n",
    "plt.figure()\n",
    "\n",
    "plt.plot(epochs, loss, 'bo', label='Training loss')\n",
    "plt.plot(epochs, val_loss, 'b', label='Validation loss')\n",
    "plt.title('Training and validation loss')\n",
    "plt.legend()\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "검증 MAE는 0.40 대에 머물러 있습니다. 작은 컨브넷을 사용해서 상식 수준의 기준점을 넘지 못 했습니다. 이는 컨브넷이 입력 시계열에 있는 패턴을 보고 이 패턴의 시간 축의 위치(시작인지 끝 부분인지 등)를 고려하지 않기 때문입니다. 최근 데이터 포인트일수록 오래된 데이터 포인트와는 다르게 해석해야 하기 때문에 컨브넷이 의미 있는 결과를 만들지 못합니다. 이런 컨브넷의 한계는 IMDB 데이터에서는 문제가 되지 않습니다. 긍정 또는 부정적인 감성과 연관된 키워드 패턴의 중요성은 입력 시퀀스에 나타난 위치와 무관하기 때문입니다.\n",
    "\n",
    "컨브넷의 속도와 경량함을 RNN의 순서 감지 능력과 결합하는 한가지 전략은 1D 컨브넷을 RNN 이전에 전처리 단계로 사용하는 것입니다. 수천 개의 스텝을 가진 시퀀스 같이 RNN으로 처리하기엔 현실적으로 너무 긴 시퀀스를 다룰 때 특별히 도움이 됩니다. 컨브넷이 긴 입력 시퀀스를 더 짧은 고수준 특성의 (다운 샘플된) 시퀀스로 변환합니다. 추출된 특성의 시퀀스는 RNN 파트의 입력이 됩니다."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "이 기법이 연구 논문이나 실전 애플리케이션에 자주 등장하지는 않습니다. 아마도 널리 알려지지 않았기 때문일 것입니다. 이 방법은 효과적이므로 많이 사용되기를 바랍니다. 온도 예측 문제에 적용해 보죠. 이 전략은 훨씬 긴 시퀀스를 다룰 수 있으므로 더 오래전 데이터를 바라보거나(데이터 제너레이터의 `lookback` 매개변수를 증가시킵니다), 시계열 데이터를 더 촘촘히 바라볼 수 있습니다(제너레이터의 `step` 매개변수를 감소시킵니다). 여기서는 그냥 `step`을 절반으로 줄여서 사용하겠습니다. 온도 데이터가 30분마다 1 포인트씩 샘플링되기 때문에 결과 시계열 데이터는 두 배로 길어집니다. 앞서 정의한 제너레이터 함수를 다시 사용합니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 이전에는 6이었습니다(시간마다 1 포인트); 이제는 3 입니다(30분마다 1 포인트)\n",
    "step = 3\n",
    "lookback = 1440  # 변경 안 됨\n",
    "delay = 144 # 변경 안 됨\n",
    "\n",
    "train_gen = generator(float_data,\n",
    "                      lookback=lookback,\n",
    "                      delay=delay,\n",
    "                      min_index=0,\n",
    "                      max_index=200000,\n",
    "                      shuffle=True,\n",
    "                      step=step)\n",
    "val_gen = generator(float_data,\n",
    "                    lookback=lookback,\n",
    "                    delay=delay,\n",
    "                    min_index=200001,\n",
    "                    max_index=300000,\n",
    "                    step=step)\n",
    "test_gen = generator(float_data,\n",
    "                     lookback=lookback,\n",
    "                     delay=delay,\n",
    "                     min_index=300001,\n",
    "                     max_index=None,\n",
    "                     step=step)\n",
    "val_steps = (300000 - 200001 - lookback) // 128\n",
    "test_steps = (len(float_data) - 300001 - lookback) // 128"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "이 모델은 두 개의 `Conv1D` 층 다음에 `GRU` 층을 놓았습니다:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "conv1d_6 (Conv1D)            (None, None, 32)          2272      \n",
      "_________________________________________________________________\n",
      "max_pooling1d_4 (MaxPooling1 (None, None, 32)          0         \n",
      "_________________________________________________________________\n",
      "conv1d_7 (Conv1D)            (None, None, 32)          5152      \n",
      "_________________________________________________________________\n",
      "gru_1 (GRU)                  (None, 32)                6240      \n",
      "_________________________________________________________________\n",
      "dense_3 (Dense)              (None, 1)                 33        \n",
      "=================================================================\n",
      "Total params: 13,697\n",
      "Trainable params: 13,697\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n",
      "Epoch 1/20\n",
      "500/500 [==============================] - 80s 160ms/step - loss: 0.3401 - val_loss: 0.2890\n",
      "Epoch 2/20\n",
      "500/500 [==============================] - 79s 158ms/step - loss: 0.3084 - val_loss: 0.2786\n",
      "Epoch 3/20\n",
      "500/500 [==============================] - 79s 158ms/step - loss: 0.2945 - val_loss: 0.2726\n",
      "Epoch 4/20\n",
      "500/500 [==============================] - 79s 158ms/step - loss: 0.2862 - val_loss: 0.2744\n",
      "Epoch 5/20\n",
      "500/500 [==============================] - 79s 158ms/step - loss: 0.2782 - val_loss: 0.2812\n",
      "Epoch 6/20\n",
      "500/500 [==============================] - 79s 158ms/step - loss: 0.2745 - val_loss: 0.2924\n",
      "Epoch 7/20\n",
      "500/500 [==============================] - 79s 158ms/step - loss: 0.2671 - val_loss: 0.2816\n",
      "Epoch 8/20\n",
      "500/500 [==============================] - 79s 158ms/step - loss: 0.2609 - val_loss: 0.2801\n",
      "Epoch 9/20\n",
      "500/500 [==============================] - 79s 158ms/step - loss: 0.2565 - val_loss: 0.2850\n",
      "Epoch 10/20\n",
      "500/500 [==============================] - 79s 158ms/step - loss: 0.2537 - val_loss: 0.2787\n",
      "Epoch 11/20\n",
      "500/500 [==============================] - 79s 158ms/step - loss: 0.2496 - val_loss: 0.2864\n",
      "Epoch 12/20\n",
      "500/500 [==============================] - 79s 158ms/step - loss: 0.2446 - val_loss: 0.2885\n",
      "Epoch 13/20\n",
      "500/500 [==============================] - 79s 158ms/step - loss: 0.2444 - val_loss: 0.2949\n",
      "Epoch 14/20\n",
      "500/500 [==============================] - 79s 158ms/step - loss: 0.2403 - val_loss: 0.3011\n",
      "Epoch 15/20\n",
      "500/500 [==============================] - 79s 158ms/step - loss: 0.2364 - val_loss: 0.2903\n",
      "Epoch 16/20\n",
      "500/500 [==============================] - 79s 158ms/step - loss: 0.2353 - val_loss: 0.2862\n",
      "Epoch 17/20\n",
      "500/500 [==============================] - 79s 158ms/step - loss: 0.2326 - val_loss: 0.2901\n",
      "Epoch 18/20\n",
      "500/500 [==============================] - 79s 158ms/step - loss: 0.2300 - val_loss: 0.2908\n",
      "Epoch 19/20\n",
      "500/500 [==============================] - 79s 158ms/step - loss: 0.2285 - val_loss: 0.2944\n",
      "Epoch 20/20\n",
      "500/500 [==============================] - 79s 158ms/step - loss: 0.2254 - val_loss: 0.2929\n"
     ]
    }
   ],
   "source": [
    "model = Sequential()\n",
    "model.add(layers.Conv1D(32, 5, activation='relu',\n",
    "                        input_shape=(None, float_data.shape[-1])))\n",
    "model.add(layers.MaxPooling1D(3))\n",
    "model.add(layers.Conv1D(32, 5, activation='relu'))\n",
    "model.add(layers.GRU(32, dropout=0.1, recurrent_dropout=0.5))\n",
    "model.add(layers.Dense(1))\n",
    "\n",
    "model.summary()\n",
    "\n",
    "model.compile(optimizer=RMSprop(), loss='mae')\n",
    "history = model.fit_generator(train_gen,\n",
    "                              steps_per_epoch=500,\n",
    "                              epochs=20,\n",
    "                              validation_data=val_gen,\n",
    "                              validation_steps=val_steps)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAAEYCAYAAAAJeGK1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3Xl8VOXZ//HPBbLIJghYFWRTqwJCwIhYUUAtD2DFpS4gqLgUUWm1Pu1PCmgtfWwVrFIstdJWHxUUlz5UqihaxbrUDWRRQAQ0CEIRUTZBMOT6/XGfwBAmyYRMZk6S7/v1mldmzrnPmWtOJrnmvudezN0RERGJmxrZDkBERCQZJSgREYklJSgREYklJSgREYklJSgREYklJSgREYklJSiJBTOraWZbzaxVOstmk5kdZWZpH8dhZmeaWV7C46VmdmoqZffjuf5iZqP29/gSzvs/Zva/6T6vVC0HZDsAqZzMbGvCw3rADmBX9Pgad59alvO5+y6gQbrLVgfufkw6zmNmVwND3L1XwrmvTse5RfaHEpTsF3ffnSCiT+hXu/s/iytvZge4e34mYhORqkFNfFIhoiacx83sMTPbAgwxs5PN7C0z22hma81sopnVisofYGZuZm2ix1Oi/c+Z2RYze9PM2pa1bLS/n5l9ZGabzOxeM3vDzIYWE3cqMV5jZsvN7Cszm5hwbE0zu8fMNpjZCqBvCddnjJlNK7JtkpndHd2/2syWRK9nRVS7Ke5cq82sV3S/npk9EsW2CDghyfN+HJ13kZkNiLYfD/wBODVqPv0i4drelnD88Oi1bzCzv5vZYalcm9KY2blRPBvN7GUzOyZh3ygzW2Nmm83sw4TX2t3M3ou2rzOz8ak+n1QS7q6bbuW6AXnAmUW2/Q+wEzib8EHoQOBE4CRCzb0d8BEwIip/AOBAm+jxFOALIBeoBTwOTNmPsocAW4Bzon03Ad8CQ4t5LanE+DRwENAG+LLwtQMjgEVAS6Ap8Gr4E0v6PO2ArUD9hHN/DuRGj8+OyhhwOrAd6BTtOxPISzjXaqBXdP8u4BWgCdAaWFyk7EXAYdHv5JIohu9E+64GXikS5xTgtuh+nyjGHKAu8Efg5VSuTZLX/z/A/0b3j4viOD36HY2KrnstoAOwEjg0KtsWaBfdfxcYFN1vCJyU7b8F3dJ7Uw1KKtLr7v4Pdy9w9+3u/q67v+3u+e7+MTAZ6FnC8U+5+xx3/xaYSvjHWNayPwDmu/vT0b57CMksqRRj/K27b3L3PEIyKHyui4B73H21u28A7ijheT4GPiAkToDvAxvdfU60/x/u/rEHLwMvAUk7QhRxEfA/7v6Vu68k1IoSn/cJd18b/U4eJXy4yE3hvACDgb+4+3x3/wYYCfQ0s5YJZYq7NiUZCMxw95ej39EdQCPCB4V8QjLsEDUTfxJdOwgfNI42s6buvsXd307xdUgloQQlFWlV4gMzO9bMnjWz/5jZZmAs0KyE4/+TcH8bJXeMKK7s4YlxuLsTahxJpRhjSs9F+ORfkkeBQdH9SwiJtTCOH5jZ22b2pZltJNReSrpWhQ4rKQYzG2pmC6KmtI3AsSmeF8Lr230+d98MfAW0SChTlt9ZcectIPyOWrj7UuC/Cb+Hz6Mm40OjolcA7YGlZvaOmfVP8XVIJaEEJRWpaBfr+wm1hqPcvRFwK6EJqyKtJTS5AWBmxt7/UIsqT4xrgSMSHpfWDf5x4MyoBnIOIWFhZgcCTwG/JTS/NQZeSDGO/xQXg5m1A+4DrgWaRuf9MOG8pXWJX0NoNiw8X0NCU+JnKcRVlvPWIPzOPgNw9ynufgqhea8m4brg7kvdfSChGfd3wN/MrG45Y5EYUYKSTGoIbAK+NrPjgGsy8JzPAF3N7GwzOwC4AWheQTE+AdxoZi3MrClwc0mF3X0d8DrwILDU3ZdFu+oAtYH1wC4z+wFwRhliGGVmjS2MExuRsK8BIQmtJ+Tqqwk1qELrgJaFnUKSeAy4ysw6mVkdQqJ4zd2LrZGWIeYBZtYreu6fE743fNvMjjOz3tHzbY9uuwgv4FIzaxbVuDZFr62gnLFIjChBSSb9N3A54Z/P/YQaRIWKksDFwN3ABuBIYB5h3Fa6Y7yP8F3R+4Qv8J9K4ZhHCZ0eHk2IeSPwU2A6oaPBBYREm4pfEmpyecBzwMMJ510ITATeicocCyR+b/MisAxYZ2aJTXWFxz9PaGqbHh3fivC9VLm4+yLCNb+PkDz7AgOi76PqAOMI3xv+h1BjGxMd2h9YYqGX6F3Axe6+s7zxSHxYaJIXqR7MrCahSekCd38t2/GISPFUg5Iqz8z6mtlBUTPRLYSeYe9kOSwRKYUSlFQHPYCPCc1EfYFz3b24Jj4RiQk18YmISCypBiUiIrEUu8limzVr5m3atMl2GCIiUkHmzp37hbuXNNwDiGGCatOmDXPmzMl2GCIiUkHMrLRZVgA18YmISEwpQYmISCwpQYmISCzF7jsoEZHifPvtt6xevZpvvvkm26FICurWrUvLli2pVau46R1LpgQlIpXG6tWradiwIW3atCFMTC9x5e5s2LCB1atX07Zt29IPSEJNfCJSaXzzzTc0bdpUyakSMDOaNm1artpuSgkqmstsqZktN7ORSfYPN7P3zWy+mb1uZu2L7G9lZlvN7Gf7HWmKpk6FNm2gRo3wc+rU0o4QkcpEyanyKO/vqtQEFc3+PAnoR1i9clDRBAQ86u7Hu3sOYWr8u4vsv4cw9X+FmjoVhg2DlSvBPfwcNkxJSkSkMkqlBtUNWO7uH0drrUwjrP65W7T0c6H6JKzMaWbnEibqXFT+cEs2ejRs27b3tm3bwnYRkfLasGEDOTk55OTkcOihh9KiRYvdj3fuTG0pqiuuuIKlS5eWWGbSpElMTdMn6x49ejB//vy0nCvTUukk0QJYlfB4NXBS0UJmdj1wE2El0NOjbfUJq4p+Hyi2ec/MhgHDAFq1Km2V7OJ9+mnZtotI1TZ1aviA+umn0KoV3H47DC7HEotNmzbd/c/+tttuo0GDBvzsZ3v/a3N33J0aNZJ//n/wwQdLfZ7rr79+/4OsQlKpQSVrRNxnCnR3n+TuRxISUuGKl78C7nH3rSU9gbtPdvdcd89t3rzU6ZmKVVxuK0fOE5FKKpNN/suXL6djx44MHz6crl27snbtWoYNG0Zubi4dOnRg7Nixu8sW1mjy8/Np3LgxI0eOpHPnzpx88sl8/vnnAIwZM4YJEybsLj9y5Ei6devGMcccw7///W8Avv76a374wx/SuXNnBg0aRG5ubqk1pSlTpnD88cfTsWNHRo0aBUB+fj6XXnrp7u0TJ04E4J577qF9+/Z07tyZIUOGpP2apSKVBLUaOCLhcUvCiqTFmQacG90/CRhnZnnAjcAoMxuxH3Gm5PbboV69vbfVqxe2i0j1kukm/8WLF3PVVVcxb948WrRowR133MGcOXNYsGABL774IosXL97nmE2bNtGzZ08WLFjAySefzAMPPJD03O7OO++8w/jx43cnu3vvvZdDDz2UBQsWMHLkSObNm1difKtXr2bMmDHMnj2befPm8cYbb/DMM88wd+5cvvjiC95//30++OADLrvsMgDGjRvH/PnzWbBgAX/4wx/KeXX2TyoJ6l3gaDNra2a1gYHAjMQCZnZ0wsOzgGUA7n6qu7dx9zbABOA37l5hr3TwYJg8GVq3BrPwc/Lk8lXpRaRyynST/5FHHsmJJ564+/Fjjz1G165d6dq1K0uWLEmaoA488ED69esHwAknnEBeXl7Sc59//vn7lHn99dcZOHAgAJ07d6ZDhw4lxvf2229z+umn06xZM2rVqsUll1zCq6++ylFHHcXSpUu54YYbmDVrFgcddBAAHTp0YMiQIUydOnW/B9qWV6kJyt3zgRHALGAJ8IS7LzKzsWY2ICo2wswWmdl8wvdQl1dYxKUYPBjy8qCgIPxUchKpnjLd5F+/fv3d95ctW8bvf/97Xn75ZRYuXEjfvn2TjgeqXbv27vs1a9YkPz8/6bnr1KmzT5myLjZbXPmmTZuycOFCevTowcSJE7nmmmsAmDVrFsOHD+edd94hNzeXXbt2len50iGlcVDuPtPdv+vuR7r77dG2W919RnT/Bnfv4O457t7b3ffpsefut7n7XekNX0QkuWw2+W/evJmGDRvSqFEj1q5dy6xZs9L+HD169OCJJ54A4P33309aQ0vUvXt3Zs+ezYYNG8jPz2fatGn07NmT9evX4+5ceOGF/OpXv+K9995j165drF69mtNPP53x48ezfv16thVtL80ATXUkIlVSYetJOnvxpapr1660b9+ejh070q5dO0455ZS0P8ePf/xjLrvsMjp16kTXrl3p2LHj7ua5ZFq2bMnYsWPp1asX7s7ZZ5/NWWedxXvvvcdVV12Fu2Nm3HnnneTn53PJJZewZcsWCgoKuPnmm2nYsGHaX0NprKzVxIqWm5vrWrBQRJJZsmQJxx13XLbDiIX8/Hzy8/OpW7cuy5Yto0+fPixbtowDDohXvSPZ78zM5rp7bmnHxuuViIhISrZu3coZZ5xBfn4+7s79998fu+RUXlXr1YiIVBONGzdm7ty52Q6jQmk2cxERiSUlKBERiSUlKBERiSUlKBERiSUlKBGRFPXq1WufQbcTJkzguuuuK/G4Bg0aALBmzRouuOCCYs9d2hCbCRMm7DVgtn///mzcuDGV0Et02223cddd8ZtHQQlKRCRFgwYNYtq0aXttmzZtGoMGDUrp+MMPP5ynnnpqv5+/aIKaOXMmjRs33u/zxZ0SlIhIii644AKeeeYZduzYAUBeXh5r1qyhR48eu8clde3aleOPP56nn356n+Pz8vLo2LEjANu3b2fgwIF06tSJiy++mO3bt+8ud+211+5equOXv/wlABMnTmTNmjX07t2b3r17A9CmTRu++OILAO6++246duxIx44ddy/VkZeXx3HHHcePfvQjOnToQJ8+ffZ6nmTmz59P9+7d6dSpE+eddx5fffXV7udv3749nTp12j1J7b/+9a/dCzZ26dKFLVu27Pe1TUbjoESkUrrxRkj3QrE5ORD9b0+qadOmdOvWjeeff55zzjmHadOmcfHFF2Nm1K1bl+nTp9OoUSO++OILunfvzoABAzBLtqQe3HfffdSrV4+FCxeycOFCunbtunvf7bffzsEHH8yuXbs444wzWLhwIT/5yU+4++67mT17Ns2aNdvrXHPnzuXBBx/k7bffxt056aST6NmzJ02aNGHZsmU89thj/PnPf+aiiy7ib3/7W4nrO1122WXce++99OzZk1tvvZVf/epXTJgwgTvuuINPPvmEOnXq7G5WvOuuu5g0aRKnnHIKW7dupW7dumW42qVTDUpEpAwSm/kSm/fcnVGjRtGpUyfOPPNMPvvsM9atW1fseV599dXdiaJTp0506tRp974nnniCrl270qVLFxYtWlTqRLCvv/465513HvXr16dBgwacf/75vPbaawC0bduWnJwcoOQlPSCsT7Vx40Z69uwJwOWXX86rr766O8bBgwczZcqU3TNWnHLKKdx0001MnDiRjRs3pn0mC9WgRKRSKqmmU5HOPfdcbrrpJt577z22b9++u+YzdepU1q9fz9y5c6lVqxZt2rRJusRGomS1q08++YS77rqLd999lyZNmjB06NBSz1PSnKqFS3VAWK6jtCa+4jz77LO8+uqrzJgxg1//+tcsWrSIkSNHctZZZzFz5ky6d+/OP//5T4499tj9On8yqkGJiJRBgwYN6NWrF1deeeVenSM2bdrEIYccQq1atZg9ezYrV64s8TynnXYaU6P15z/44AMWLlwIhKU66tevz0EHHcS6det47rnndh/TsGHDpN/znHbaafz9739n27ZtfP3110yfPp1TTz21zK/toIMOokmTJrtrX4888gg9e/akoKCAVatW0bt3b8aNG8fGjRvZunUrK1as4Pjjj+fmm28mNzeXDz/8sMzPWRLVoEREymjQoEGcf/75e/XoGzx4MGeffTa5ubnk5OSUWpO49tprueKKK+jUqRM5OTl069YNCKvjdunShQ4dOuyzVMewYcPo168fhx12GLNnz969vWvXrgwdOnT3Oa6++mq6dOlSYnNecR566CGGDx/Otm3baNeuHQ8++CC7du1iyJAhbNq0CXfnpz/9KY0bN+aWW25h9uzZ1KxZk/bt2+9eHThdtNyGiFQaWm6j8inPchtq4hMRkVhSghIRkVhSghKRSiVuX0tI8cr7u1KCEpFKo27dumzYsEFJqhJwdzZs2FCuwbvqxScilUbLli1ZvXo169evz3YokoK6devSsmXL/T5eCUpEKo1atWrRtm3bbIchGaImPhERiSUlKBERiSUlKBERiSUlKBERiSUlKBERiaWUEpSZ9TWzpWa23MxGJtk/3MzeN7P5Zva6mbWPtn/fzOZG++aa2enpfgEiIlI1lZqgzKwmMAnoB7QHBhUmoASPuvvx7p4DjAPujrZ/AZzt7scDlwOPpC1yERGp0lKpQXUDlrv7x+6+E5gGnJNYwN03JzysD3i0fZ67r4m2LwLqmlkdRERESpHKQN0WwKqEx6uBk4oWMrPrgZuA2kCyprwfAvPcfUeSY4cBwwBatWqVQkgiIlLVpVKD2ndN4qiGtNcG90nufiRwMzBmrxOYdQDuBK5J9gTuPtndc909t3nz5imEJCIiVV0qCWo1cETC45bAmmLKQmgCPLfwgZm1BKYDl7n7iv0JUkREqp9UEtS7wNFm1tbMagMDgRmJBczs6ISHZwHLou2NgWeBX7j7G+kJWUREqoNSE5S75wMjgFnAEuAJd19kZmPNbEBUbISZLTKz+YTvoS4v3A4cBdwSdUGfb2aHpP9liIhIVWNxW1clNzfX58yZk+0wRESkgpjZXHfPLa2cZpIQEZFYUoISEZFYUoISEZFYUoISEZFYUoISEZFYUoISEZFYUoISEZFYUoISEZFYUoISEZFYUoISEZFYUoISEZFYUoISEZFYUoISEZFYUoIqxtSp0KYN1KgRfk6dmu2IRESqlwOyHUAcTZ0Kw4bBtm3h8cqV4THA4MHZi0tEpDpRDSqJ0aP3JKdC27aF7SIikhlKUEl8+mnZtouISPopQSXRqlXZtouISPopQSVx++1Qr97e2+rVC9tFRCQzlKCSGDwYJk+G1q3BLPycPFkdJEREMkm9+IoxeLASkohINqkGJSIisaQEJSIisaQEJSIisaQEJSIisaQEJSIisaQEJSIisaQEJSIisZRSgjKzvma21MyWm9nIJPuHm9n7ZjbfzF43s/YJ+34RHbfUzP4rncGLSOW1ahX89rfw5ZfZjkTiqtQEZWY1gUlAP6A9MCgxAUUedffj3T0HGAfcHR3bHhgIdAD6An+Mzici1dSuXXDvvdC+PYwaBWedBV9/ne2oJI5SqUF1A5a7+8fuvhOYBpyTWMDdNyc8rA94dP8cYJq773D3T4Dl0flEpBpauBC+9z34yU+gRw+YNAneeQcuugi+/Tbb0UncpDLVUQtgVcLj1cBJRQuZ2fXATUBt4PSEY98qcmyLJMcOA4YBtNKU4SJVzrZtMHYs3HUXHHwwPPooDBwY5ro84AC45hq4+mp48MGwirUIpFaDsiTbfJ8N7pPc/UjgZmBMGY+d7O657p7bvHnzFEISkcrixRfh+OPhzjth6FD48EMYNCgkJwirVY8dCw8/DDffnNVQJWZSqUGtBo5IeNwSWFNC+WnAfft5rIhUEevXw3//NzzyCHz3uzB7NvTqlbzsmDGwbl2oYX3nO/Czn2U0VImpVGpQ7wJHm1lbM6tN6PQwI7GAmR2d8PAsYFl0fwYw0MzqmFlb4GjgnfKHLSJx5Q4PPQTHHQfTpsEtt8CCBcUnJwi1qd//Hi68EH7+81CbEim1BuXu+WY2ApgF1AQecPdFZjYWmOPuM4ARZnYm8C3wFXB5dOwiM3sCWAzkA9e7+64Kei0ikmXLlsHw4fDyy3DKKWEdtfZF+/wWo2bNUNv68ku48kpo1gz696/YeCXezH2fr4SyKjc31+fMmZPtMESkDHbuDM1zY8dCnTowbhz86Ef71+Fh82bo3Tt8V/XSS9C9e/rjlewys7nunltaOfWXEZFyefNNOOEEGD0azj4bliwJvfL2tzdeo0YwcyYcdlgYI7VkSXrjlcpDCaqCTZ0KbdqEP9Y2bcJjkapg0ya4/vrQlLdxIzz9NDz5JBx+ePnP/Z3vwAsvQK1a0KdPmHVCqh8lqAo0dWroQrtyZfjieOXK8FhJSiq76dPDd0v33RcG3S5eDAMGpPc52rWD558PTX59+2pKpOpICaoCjR4dBigm2rYtbBepjNatgwsugPPPh+bN4a23YMIEaNiwYp4vJyfUzJYvhx/8YN+/J0nNhg3wxRfZjqLsUhkHJfvp00/Ltl0krtxDzf+GG8K8eb/5TRirVKtWxT93r15h5okLLwxTIk2fnpnnrey2bw/J/eGHYdYsKCgIs3gcc8y+tyOPDJ1b4kYJqgK1ahWa9ZJtF6ksPvssdB1/5hk4+WT461/DGKdM+uEPQ3Pi8OGhd+CDD+6ZiUL2KCiA118PSenJJ0Pz6BFHhBk6mjWDpUvD7fnn4X//d89xNWpA27bJk9ehh2bvWitBVaDbbw/fOSU2S9SrF7aLxJ07PPAA3HRTmMj1nnvgxz8O45Wy4ZprQhPjL38ZOlHceWd24oijZcvCGLJHHoG8PGjQIDTFXnYZ9OyZvEflpk3w0Ud7klbh7eWX4Ztv9pRr1CjMBFKYsLp3h+9/P0MvzN1jdTvhhBO8Kpkyxb11a3ez8HPKlGxHlD4vv+z+5pvZjkIqwiefuJ95pju49+rlvnx5tiMKCgrcr7suxPW732U7muzasMH9vvvcTz45XI8aNdz79An/Y7Zu3f/z7trlnpfnPmuW+8SJ7tdfH94LRxwRnueCC8ofO2GSh1LzgQbqyn6ZNSuMUalVK0wG2qNHtiOSdCgoCE1pN98cmnXGjw+tAHGaYXzXrjAT+lNPhaasSy/NdkSZs3NnaJ57+GH4xz/C4w4d4PLL4ZJLoMU+a0Wk19dfw9atoQZbHqkO1FUTn5TZ/Pmh+aBDB9ixIwzOfO016Ngx25FJeSxbBlddFX6XffrAn/8cz+9La9aEKVNCz7TCKZH69ct2VBXHHebODUnpscdCb7xDDoHrrgtNeDk5mfuOqH79cMsU1aCkTFatCm3QNWqELsb5+WEBOoB//xtat85ufFJ2u3aFruJjxoSeXPfcE5bFiHsnhM2bw/crH30Uvjc5aZ9V6uKvoCC8jq++CoOdi/5cvx5mzAizadSpA+eeG5LS979fuXsyqgYlabdxY/ikunVr6ClU2Jzw/PNw6qnwX/8Vtjdrlt04JXWLF4dayNtvh5rwn/6UnpkgMqFRI3juuTCTRf/+IcledFH2u0sXNsN9+umeRJMs+Xz1VUhOJdURatQIHwAnTw7d7Bs3ztzriAPVoCQlO3eG5PTaa+GP7/TT997/2mvhU11OTpjgM5PNAFJ2334bJnQdOzYMsr333j0r3FY2K1bsmQOwefOwMu/w4Zlvnly+PDSLPvhgqPkUqlcPmjQJyaWsPxs2rJy/k9KkWoPKeq+9orfy9uIrKHB/4AH3bdvKdRpJUFDgfumloQfPww8XX2769NCTqF8/9507MxeflM28ee45OeH3edFF7uvWZTui8tu1y/2FF9zPOSe8B2vUCPdfeCHsqyg7drg/8YT7GWeE61mzpvu557rPnBmu644dFffclRkp9uLLekIqeitvgpozJ3Tp/sEP9E8yXcaMCe+UX/+69LKTJ4eyl10WEpvEQ0GB+6efht/lAQe4f+c77n/7W7ajqhh5ee6/+IV7s2bhvfjd77pPmOD+1Vfpe47ly91vvtn9kEPCc7RqFf4+Pvssfc9RlVXbBOUexgaA+6BB7vn55T5dtfbnP4drefXVqSecX/86HPP//l/FxibF27DB/fnn3ceOdT/77JCQwrcd4cPDhg3ZjrDiffON+yOPuHfvHl53vXruw4a5L1iwf+fbudP9ySfdv/993z3u6Jxz3J99Vv9nyqpaJyh39zvuCK/ummv0SX5/zZwZmiz69i1bbbSgIAzu02DKzPj6a/fXX3e/++7woezII/ckI3A/9tjQRHvvvaF5rzqaM8f9iivc69YN16RHD/fHHkutCW7FilAjK0zyRxzh/qtfua9eXfFxV1XVPkG5u48cueeTfGVPUpmekWLuXPf69d27dHHfvLnsx+fnu194Ybj+VWn2jGz79lv3+fNDU+qPfuTeuXP4EFGYjFq2dD//fPff/tb9pZfcN27MdsTx8sUX7uPHu7drF67XoYe633rrvslm587QBNqnz57a0tlnuz/zjGpL6aAE5SEpXXtteJW/+U3aTptxU6aE5onET8X16lXcP/68vPCH26pV+drUv/nGvXfv8J3H88+nL77q5uOP3W+6yf2UU9wPPHDPe6Bx4/APdPRo96efdl+zJtuRVh67doWmuf79w4e+mjXDFD7/+If7qFHh/V+Y8G+7zX3VqmxHXLWkmqCqfDfzgoIwFcqjj8KkSWH0dWXTpk3yWdFbtw4TQ6bTV1+FaYs++wzeeCPMFlEemzaF5RKWLQuDKbt1S0uY1casWTBoUJhw+IQT4MQTwzU88UQ46qiq2QU501asCOO//vrX8P6vUSOMqxo2LAytOECjRdMu1W7mVT5BQRjzccEFYUT2I4/AkCFpPX2Fq1Ej+WA+s5CA02XHjrBy6RtvhOW2e/VKz3n/858w2HDLlnDu7343PeetytzhjjvC4pbHHx/WQGrXLttRVW3bt4cxfJ06xXOKp6ok1QQVoykgK06tWvD449C7d5jC5emnsx1R2RT3x5LOPyL3MKPAK6+EdWLSlZwgrCfzwgshofbpA2vWpO/c6ZaXB9deGz7QLF2anRi2bAmzBowaBRdfHKaQUnKqeAceGFbtVXKKj2qRoADq1g2JKTc3TIfy0kvZjih1t98eRqMnSve6UqNHh2bQ3/wmzIqcbkcdFaal2bAhNJts3Jj+5yiPTz4JC+EdfXRYA+mf/4TOneG3vw018ExZtizMdTiEztXjAAAUpklEQVR9Ovzud+F3olk5pNpK5YuqTN4qej2oDRvcO3YMPdQq01pGFdmL709/8ox1yX/xRfdatdxPO819+/aKfa5UrFjhfuWVoSNH7dqhe/yqVe5r14YvzSHMuvDeexUfyzPPuB90kHvTpqEHnkhVhXrxFW/NmjBWpHHj/R+0V1U8+2zoQtu/f+jCnAnTpoVEe9552euyu3x5GBdTs6Z7nTruP/5x8nEtf/tb6NFVs2YYtlARSXXXrjCuxix068/LS/9ziMSJElQpPvnEvUWLMPjuo48y8pSxM2dOqEmecIL7li2Zfe6JE8O7b9iwzI5RW7bM/fLLQ8KpW9f9hhtK70r/5ZchmRVOm/Paa+mLZ+NG9wEDfPcMD5pDUqoDJagULFkS5utq1SrMU1adfPJJSM6tW4fmrGwYNSq8A3/5y4p/ro8+CgmgRo2QmG68sezjhl54wb1NmxDz9dfv3wDmREuWuB9zTEiWEydW/sHkIqlSgkrR3LnujRqFfxRVYVbnVHz5ZZj+pkkT98WLsxdHQUH4/gfcf/KT0PT33nvprc19+KH7kCEhMR14YBjwWp6EvGVLSG5mYcqbmTP37zzTp7s3bBgmG33llf2PR6QySmuCAvoCS4HlwMgk+28CFgMLgZeA1gn7xgGLgCXARKKxV8XdMp2g3EOTzYEHhi/D0znjcRx9/LH7iSeGDgH/+le2ownfew0c6HvNkgHuhx/u3qtXaAK86y73GTNCskl1+YIlS9wvuSQkpnr13H/2M/f//Cd9cf/73+7HHRdivfTSMIVOKvLz98wO362bZiiQ6iltCQqoCawA2gG1gQVA+yJlegP1ovvXAo9H978HvBGdoybwJtCrpOfLRoJyd3/uudC77JRT3LduzUoIFaqgwP2hh8Kn9kaN3P/+92xHtLdt29wXLnR/6qkwLdXQoe7f+96eJRMKbzVruh91VOjUceON7n/8o/s//xmaaHftCjXCQYNCDadePfef/7ziasbffON+yy2hB+Ahh4R1gUpqpvvqqxA3hJpjHHoximRDOhPUycCshMe/AH5RQvkuwBsJx84FDgTqAXOA40p6vmwlKPcwlX6NGmF+s2++yVoYabdhw56JW087rfL1Etuwwf2tt8JiiWPGhEX2cnL2nZ/wwANDYqpfP6zV8/nnmYlvwYLQ0QTCYnXJOl28/35IrLVqheVg9H2TVGepJqhUZplqAaxKeLwaOKmE8lcBzwG4+5tmNhtYCxjwB3dfUvQAMxsGDANolcVh3BdcEJZsvuoqGDwYpk2r/PNwvfhimD1j/fowdc7hh0PPnvDpp2HE/O23h9caZwcfDCedFG6JCgrCrBQffRRuS5eGZbKvvx6aNctcfJ06wVtvwYQJcMst0L59GGR75ZVh9ownn4QrrgjLd7/ySpj2SURSUFoGAy4E/pLw+FLg3mLKDgHeAupEj48CngUaRLc3gdNKer5s1qAK3XNP+DQ8dGjFLhddkbZvD01gEL4rmTs387OiV0fLlrn37Bmu7emn7/kdnHyyVlsVKUSKNahUpjpaDRyR8LglsM9samZ2JjAaGODuO6LN5wFvuftWd99KqFl1L0sCzYYbb4Tbbgtz0vXrF2Y7rkwWLAhTOk2YACNGwNy50LVrmM5o27a9y27bFrZLehx1VJi1/U9/gnffDb+D4cNDzenww7MdnUjlkkqCehc42szamlltYCAwI7GAmXUB7ickp88Tdn0K9DSzA8ysFtCT0Jsv9m69Ff7wB3jzTejYMTSF7dyZ7ahKVlAAd90VlmPYsCHMfXfvvWESTAjNeskUt132T40acM01sGRJmNPvvvugdu1sRyVS+ZSaoNw9HxgBzCIklyfcfZGZjTWzAVGx8YQmvCfNbL6ZFSawpwg9AN8n9P5b4O7/SPeLqAhm4buMJUvCDMdjxkBODrz6arYjS27VKjjzTPj5z8NaNu+/H5bOSJSJWdFljxYt4Iwzsh2FSOVVLdaDSoeZM0PCyssLX3iPG5fZL+JLMm1aaEbatQt+//sQX7KF7KZODYuwJTbz1asHkyfHv6OEiFQdWg8qzfr3h0WLYOTIsOjhscfCgw8mX0gwUzZuDIll0KDQc2z+/D09x5IZPDgko9atQ5nWrZWcRCS+lKDKoF69sD7QvHkhQV15ZVjYb/HizMfyyiuhe/Pjj8PYsaHp8cgjSz9u8OBQCywoCD/3NzlNnRqWoq9RI/ycOnX/ziMiUhwlqP3QsWNICH/5S/iuJycn9ITbvr3in3vHDrj5Zjj9dKhTJ6y2esstmR2vVdhUuHJlqEGuXBkeK0mJSDopQe2nGjXCgN6lS0MT229+ExLXrFkV83wFBTBnThisOm5cWP113rzQYy/T1F1dRDJBCaqcmjeHhx4KY19q1Qo95wYOhLVr9/+cmzfD66/DH/8Yuit37x5mITjxxDBzwowZcP/90KBB+l5HWai7uohkQiWfyCc+evcOA2THjQtjpp57LtSqhg+HmjWTH1NQAJ98Eo5buDD8XLAgbCvUpAl07hxqTJ07hy7vzZtn5jUVp1Wr0KyXbLuISLqom3kFWLYMrrsuDNI88cRQ2zn66PB9VWIyWrgQtm4Nx9SoEcp07rz3rUWL4nvlZYu6q4tIeaTazVw1qApw9NHwwgthfNJPfwonnBC2F34WOOig0ANv6NA9iahDh/BPvjIoTEKjR1euSWdFpHJRDaqCbdwY5mOrWTMkok6d9oxDEhGpjlSDionGjcPEsyIiUjbqxSciIrGkBCUiIrGkBCUiIrGkBCUiIrGkBCUiIrGkBCVZpVnRRaQ46mYuWVN0RorCWdFBg35FRDUoySLNii4iJVGCkqzRrOgiUhIlKMma4mY/16zoIgJKUJJFt9++7wS59eqF7SIiSlCSNYMHhyU6CifPbd1aS3aIyB7qxSdZNXiwEpKIJKcalIiIxJISlIiIxJISlIiIxJISlFQJmjJJpOpRJwmp9DRlkkjVlFINysz6mtlSM1tuZiOT7L/JzBab2UIze8nMWifsa2VmL5jZkqhMm/SFL6Ipk0SqqlITlJnVBCYB/YD2wCAza1+k2Dwg1907AU8B4xL2PQyMd/fjgG7A5+kIXKSQpkwSqZpSqUF1A5a7+8fuvhOYBpyTWMDdZ7t74WfYt4CWAFEiO8DdX4zKbU0oJ5IWmjJJpGpKJUG1AFYlPF4dbSvOVcBz0f3vAhvN7P/MbJ6ZjY9qZHsxs2FmNsfM5qxfvz7V2EUATZkkUlWlkqAsyTZPWtBsCJALjI82HQCcCvwMOBFoBwzd52Tuk909191zmzdvnkJIIntoyiSRqimVXnyrgSMSHrcE1hQtZGZnAqOBnu6+I+HYee7+cVTm70B34K/lCVqkKE2ZJFL1pFKDehc42szamlltYCAwI7GAmXUB7gcGuPvnRY5tYmaF1aLTgcXlD1tERKq6UhOUu+cDI4BZwBLgCXdfZGZjzWxAVGw80AB40szmm9mM6NhdhOa9l8zsfUJz4Z8r4HWIpIUG/IrEh7kn/Topa3Jzc33OnDnZDkOqoaIDfiF0ttD3WSLpZWZz3T23tHKa6kgkogG/IvGiBCUS0YBfkXhRghKJaMCvSLwoQYlE0jngV50tRMpPCUokkq4Bv4WdLVauBPc9s6srSYmUjXrxiaRZmzYhKRXVujXk5WU6GpH4US8+kSxRZwuR9FCCEkkzdbYQSQ8lKJE00+zqIumhBCWSZppdXSQ9lKBEKsDgwaFDREFB+Lm/yUnd1aU6S2W5DRHJgqJzAxZ2VwfVxqR6UA1KJKY0N6BUd0pQIjGl7upS3SlBicSUuqtLdacEJRJT6q4u1Z0SlEhMqbu6VHfqxScSY4MHKyFJ9aUalIiIxJISlIiIxJISlEg1kK4ZKTSzhWSSvoMSqeLSNSOFZraQTNOChSJVXLoWUNRCjJIuWrBQRID0zUihmS0k05SgRKq4dM1IoZktJNOUoESquHTNSKGZLSTTlKBEqrh0zUihmS0k09RJQkREMiqtnSTMrK+ZLTWz5WY2Msn+m8xssZktNLOXzKx1kf2NzOwzM/tD6i9BRKoyjamS0pSaoMysJjAJ6Ae0BwaZWfsixeYBue7eCXgKGFdk/6+Bf5U/XBGpCgrHVK1cCe57xlQpSUmiVGpQ3YDl7v6xu+8EpgHnJBZw99nuXrj251tAy8J9ZnYC8B3ghfSELCKVnVYLllSkkqBaAKsSHq+OthXnKuA5ADOrAfwO+HlJT2Bmw8xsjpnNWb9+fQohiUhlpjFVkopUEpQl2Za0Z4WZDQFygfHRpuuAme6+Kln53Sdzn+zuue6e27x58xRCEpHKTGOqJBWpJKjVwBEJj1sCa4oWMrMzgdHAAHffEW0+GRhhZnnAXcBlZnZHuSIWkUpPY6okFalMFvsucLSZtQU+AwYClyQWMLMuwP1AX3f/vHC7uw9OKDOU0JFin16AIlK9FI6dGj06NOu1ahWSk8ZUSaJSa1Dung+MAGYBS4An3H2RmY01swFRsfFAA+BJM5tvZjMqLGIRqRIGDw6TzBYUhJ/7m5zUXb3q0kBdEam0ii4BAqGpUDNcxJtmMxeRKi+d3dVVE4sfLVgoIpVWurqrazHGeFINSkQqrXR1V9fA4XhSghKRSitd3dU1cDielKBEpNJK1xIgGjgcT0pQIlKppaO7ejoHDquzRfooQYlItZeumphmaU8vjYMSEUmTNm1CUiqqdetQu5NA46BERDJMnS3SSwlKRCRN1NkivZSgRETSRJ0t0ksJSkQkTdTZIr2UoERE0igd3d41x2CgufhERGJGcwwGqkGJiMSM5hgMlKBERGJGcwwGSlAiIjGjOQYDJSgRkRiK2xyD2aAEJSJSRaWrJgbZ6Q2oXnwiIlXY4MHl77GXrd6AqkGJiEiJstUbUAlKRERKlK3egEpQIiJSomz1BlSCEhGREmWrN6ASlIiIlCidvQHLQr34RESkVOnoDVhWqkGJiEgsKUGJiEgspZSgzKyvmS01s+VmNjLJ/pvMbLGZLTSzl8ysdbQ9x8zeNLNF0b6L0/0CRESkaio1QZlZTWAS0A9oDwwys/ZFis0Dct29E/AUMC7avg24zN07AH2BCWbWOF3Bi4hI1ZVKDaobsNzdP3b3ncA04JzEAu4+290Lxxm/BbSMtn/k7sui+2uAz4Hm6QpeRESqrlQSVAtgVcLj1dG24lwFPFd0o5l1A2oDK5LsG2Zmc8xszvr161MISUREqrpUuplbkm2etKDZECAX6Flk+2HAI8Dl7l6wz8ncJwOTo7LrzWxlCnHFSTPgi2wHUUaKOTMUc+ZUxrira8ytUymUSoJaDRyR8LglsKZoITM7ExgN9HT3HQnbGwHPAmPc/a3SnszdK10ToJnNcffcbMdRFoo5MxRz5lTGuBVzyVJp4nsXONrM2ppZbWAgMCOxgJl1Ae4HBrj75wnbawPTgYfd/cn0hS0iIlVdqQnK3fOBEcAsYAnwhLsvMrOxZjYgKjYeaAA8aWbzzawwgV0EnAYMjbbPN7Oc9L8MERGpalKa6sjdZwIzi2y7NeH+mcUcNwWYUp4AK4nJ2Q5gPyjmzFDMmVMZ41bMJTD3pP0dREREskpTHYmISCwpQYmISCwpQaXAzI4ws9lmtiSaV/CGJGV6mdmmhM4gtyY7V6aZWZ6ZvR/FNCfJfjOzidE8iwvNrGs24kyI55iEazjfzDab2Y1FymT9WpvZA2b2uZl9kLDtYDN70cyWRT+bFHPs5VGZZWZ2eZZjHm9mH0a/++nFTUVW2vuoIhUT921m9lnCe6B/MceWOI9ohmN+PCHePDObX8yxWbnWxf2fy+r72t11K+UGHAZ0je43BD4C2hcp0wt4JtuxJok9D2hWwv7+hJk/DOgOvJ3tmBNiqwn8B2gdt2tN6J3aFfggYds4YGR0fyRwZ5LjDgY+jn42ie43yWLMfYADovt3Jos5lfdRFuK+DfhZCu+fFUA7wiw2C4r+3WYy5iL7fwfcGqdrXdz/uWy+r1WDSoG7r3X396L7Wwjd7Uua7qkyOYcwTs09DKRuHM38EQdnACvcPXYzi7j7q8CXRTafAzwU3X8IODfJof8FvOjuX7r7V8CLhImUK1yymN39BQ9DSSBhHs04KeZap6LUeUQrSkkxm5kRhuA8lolYUlXC/7msva+VoMrIzNoAXYC3k+w+2cwWmNlzZtYho4EVz4EXzGyumQ1Lsr+scy1m0kCK/yOO47X+jruvhfDHDhySpEycr/eVJJlHM1La+ygbRkRNkw8U0+wU12t9KrDOo4m0k8j6tS7yfy5r72slqDIwswbA34Ab3X1zkd3vEZqiOgP3An/PdHzFOMXduxKWS7nezE4rsj/luRYzKZqFZACQbAaSuF7rVMT1eo8G8oGpxRQp7X2UafcBRwI5wFpCk1lRsbzWwCBKrj1l9VqX8n+u2MOSbCv3tVaCSpGZ1SL80qa6+/8V3e/um919a3R/JlDLzJplOMx9eFjmBA9TUE0nNHskSmmuxSzoB7zn7uuK7ojrtQbWFTaPRj8/T1Imdtc7+kL7B8Bgj75QKCqF91FGufs6d9/lYfLpPxcTTxyv9QHA+cDjxZXJ5rUu5v9c1t7XSlApiNqM/woscfe7iylzaFSucGmRGsCGzEWZNKb6Ztaw8D7hC/EPihSbAVwW9ebrDmwqrM5nWbGfMuN4rSMzgMLeS5cDTycpMwvoY2ZNomapPtG2rDCzvsDNhHk0txVTJpX3UUYV+Z70PJLHU+o8ollwJvChu69OtjOb17qE/3PZe19nuqdIZbwBPQjV1YXA/OjWHxgODI/KjAAWEXoKvQV8LwZxt4viWRDFNjranhi3EVZMXgG8T1gZOdtx1yMknIMStsXqWhOS51rgW8Knx6uApsBLwLLo58FR2VzgLwnHXgksj25XZDnm5YTvDgrf13+Kyh4OzCzpfZTluB+J3q8LCf9ADysad/S4P6E32opMxp0s5mj7/xa+jxPKxuJal/B/Lmvva011JCIisaQmPhERiSUlKBERiSUlKBERiSUlKBERiSUlKBERiSUlKBERiSUlKBERiaX/D0ihwC1GfanzAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "loss = history.history['loss']\n",
    "val_loss = history.history['val_loss']\n",
    "\n",
    "epochs = range(1, len(loss) + 1)\n",
    "\n",
    "plt.figure()\n",
    "\n",
    "plt.plot(epochs, loss, 'bo', label='Training loss')\n",
    "plt.plot(epochs, val_loss, 'b', label='Validation loss')\n",
    "plt.title('Training and validation loss')\n",
    "plt.legend()\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "검증 손실로 비교해 보면 이 설정은 규제가 있는 GRU 모델만큼 좋지는 않습니다. 하지만 훨씬 빠르기 때문에 데이터를 두 배 더 많이 처리할 수 있습니다. 여기서는 큰 도움이 안 되었지만 다른 데이터셋에서는 중요할 수 있습니다."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 정리\n",
    "\n",
    "다음은 이번 절에서 배운 것들입니다.\n",
    "\n",
    "* 2D 컨브넷이 2D 공간의 시각적 패턴을 잘 처리하는 것과 같이 1D 컨브넷은 시간에 따른 패턴을 잘 처리합니다. 1D 컨브넷은 특정 자연어 처리 같은 일부 문제에 RNN을 대신할 수 있는 빠른 모델입니다.\n",
    "* 전형적으로 1D 컨브넷은 컴퓨터 비전 분야의 2D 컨브넷과 비슷하게 구성합니다. `Conv1D` 층과 `Max-Pooling1D` 층을 쌓고 마지막에 전역 풀링 연산이나 `Flatten` 층을 둡니다.\n",
    "* RNN으로 아주 긴 시퀀스를 처리하려면 계산 비용이 많이 듭니다. 1D 컨브넷은 비용이 적게 듭니다. 따라서 1D 컨브넷을 RNN 이전의 전처리 단계로 사용하는 것은 좋은 생각입니다. 시퀀스 길이를 줄이고 RNN이 처리할 유용한 표현을 추출해 줄 것입니다.\n",
    "\n",
    "유용하고 중요한 개념이지만 여기서 다루지 않은 것은 팽창 커널을 사용한 1D 합성곱입니다."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}