6.1-using-word-embeddings.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Using TensorFlow backend.\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "'2.2.2'"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import keras\n",
    "keras.__version__"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 단어 임베딩 사용하기\n",
    "\n",
    "이 노트북은 [케라스 창시자에게 배우는 딥러닝](https://tensorflow.blog/%EC%BC%80%EB%9D%BC%EC%8A%A4-%EB%94%A5%EB%9F%AC%EB%8B%9D/) 책의 6장 1절의 코드 예제입니다. 책에는 더 많은 내용과 그림이 있습니다. 이 노트북에는 소스 코드에 관련된 설명만 포함합니다.\n",
    "\n",
    "---\n",
    "\n",
    "단어와 벡터를 연관짓는 강력하고 인기 있는 또 다른 방법은 단어 임베딩이라는 밀집 단어 벡터를 사용하는 것입니다. 원-핫 인코딩으로 만든 벡터는 희소하고(대부분 0으로 채워집니다) 고차원입니다(어휘 사전에 있는 단어의 수와 차원이 같습니다). 반면 단어 임베딩은 저차원의 실수형 벡터입니다(희소 벡터의 반대인 밀집 벡터입니다). 그림 6-2를 참고하세요. 원-핫 인코딩으로 얻은 단어 벡터와 달리 단어 임베딩은 데이터로부터 학습됩니다. 보통 256차원, 512차원 또는 큰 어휘 사전을 다룰 때는 1,024차원의 단어 임베딩을 사용합니다. 반면 원-핫 인코딩은 (20,000개의 토큰으로 이루어진 어휘 사전을 만들려면) 20,000차원 또는 그 이상의 벡터일 경우가 많습니다. 따라서 단어 임베딩이 더 많은 정보를 적은 차원에 저장합니다."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![word embeddings vs. one hot encoding](https://s3.amazonaws.com/book.keras.io/img/ch6/word_embeddings.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "단어 임베딩을 만드는 방법은 두 가지입니다.\n",
    "\n",
    "* (문서 분류나 감성 예측과 같은) 관심 대상인 문제와 함께 단어 임베딩을 학습합니다. 이런 경우에는 랜덤한 단어 벡터로 시작해서 신경망의 가중치를 학습하는 것과 같은 방식으로 단어 벡터를 학습합니다.\n",
    "* 풀려는 문제가 아니고 다른 머신 러닝 작업에서 미리 계산된 단어 임베딩을 로드합니다. 이를 사전 훈련된 단어 임베딩이라고 합니다.\n",
    "\n",
    "두 가지 모두 살펴보겠습니다."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## `Embedding` 층을 사용해 단어 임베딩 학습하기\n",
    "\n",
    "단어와 밀집 벡터를 연관짓는 가장 간단한 방법은 랜덤하게 벡터를 선택하는 것입니다. 이 방식의 문제점은 임베딩 공간이 구조적이지 않다는 것입니다. 예를 들어 accurate와 exact 단어가 대부분 문장에서 비슷한 의미로 사용되지만 완전히 다른 임베딩을 가지게 됩니다. 심층 신경망이 이런 임의의 구조적이지 않은 임베딩 공간을 이해하기는 어렵습니다.\n",
    "\n",
    "단어 벡터 사이에 조금 더 추상적이고 기하학적인 관계를 얻으려면 단어 사이에 있는 의미 관계를 반영해야 합니다. 단어 임베딩은 언어를 기하학적 공간에 매핑하는 것입니다. 예를 들어 잘 구축된 임베딩 공간에서는 동의어가 비슷한 단어 벡터로 임베딩될 것입니다. 일반적으로 두 단어 벡터 사이의 거리(L2 거리)는 이 단어 사이의 의미 거리와 관계되어 있습니다(멀리 떨어진 위치에 임베딩된 단어의 의미는 서로 다르고 반면 비슷한 단어들은 가까이 임베딩됩니다). 거리외에 임베딩 공간의 특정 방향도 의미를 가질 수 있습니다.\n",
    "\n",
    "[...]\n",
    "\n",
    "실제 단어 임베딩 공간에서 의미 있는 기하학적 변환의 일반적인 예는 '성별' 벡터와 '복수(plural)' 벡터입니다. 예를 들어 'king' 벡터에 'female' 벡터를 더하면 'queen' 벡터가 됩니다. 'plural' 벡터를 더하면 'kings'가 됩니다. 단어 임베딩 공간은 전형적으로 이런 해석 가능하고 잠재적으로 유용한 수천 개의 벡터를 특성으로 가집니다.\n",
    "\n",
    "사람의 언어를 완벽하게 매핑해서 어떤 자연어 처리 작업에도 사용할 수 있는 이상적인 단어 임베딩 공간이 있을까요? 아마도 가능하겠지만 아직까지 이런 종류의 공간은 만들지 못했습니다. 사람의 언어에도 그런 것은 없습니다. 세상에는 많은 다른 언어가 있고 언어는 특정 문화와 환경을 반영하기 때문에 서로 동일하지 않습니다. 실제로 좋은 단어 임베딩 공간을 만드는 것은 문제에 따라 크게 달라집니다. 영어로 된 영화 리뷰 감성 분석 모델을 위한 완벽한 단어 임베딩 공간은 영어로 된 법률 문서 분류 모델을 위한 완벽한 임베딩 공간과 다를 것 같습니다. 특정 의미 관계의 중요성이 작업에 따라 다르기 때문입니다.\n",
    "\n",
    "따라서 새로운 작업에는 새로운 임베딩을 학습하는 것이 타당합니다. 다행히 역전파를 사용해 쉽게 만들 수 있고 케라스를 사용하면 더 쉽습니다. `Embedding` 층의 가중치를 학습하면 됩니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "from keras.layers import Embedding\n",
    "\n",
    "# Embedding 층은 적어도 두 개의 매개변수를 받습니다.\n",
    "# 가능한 토큰의 개수(여기서는 1,000으로 단어 인덱스 최댓값 + 1입니다)와 임베딩 차원(여기서는 64)입니다\n",
    "embedding_layer = Embedding(1000, 64)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`Embedding` 층을 (특정 단어를 나타내는) 정수 인덱스를 밀집 벡터로 매핑하는 딕셔너리로 이해하는 것이 가장 좋습니다. 정수를 입력으로 받아 내부 딕셔너리에서 이 정수에 연관된 벡터를 찾아 반환합니다. 딕셔너리 탐색은 효율적으로 수행됩니다."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`Embedding` 층은 크기가 `(samples, sequence_length)`인 2D 정수 텐서를 입력으로 받습니다. 각 샘플은 정수의 시퀀스입니다. 가변 길이의 시퀀스를 임베딩할 수 있습니다. 예를 들어 위 예제의 `Embedding` 층에 `(32, 10)` 크기의 배치(길이가 10인 시퀀스 32개로 이루어진 배치)나 `(64, 15)` 크기의 배치(길이가 15인 시퀀스 64개로 이루어진 배치)를 주입할 수 있습니다. 배치에 있는 모든 시퀀스는 길이가 같아야 하므로(하나의 텐서에 담아야 하기 때문에) 작은 길이의 시퀀스는 0으로 패딩되고 길이가 더 긴 시퀀스는 잘립니다.\n",
    "\n",
    "`Embedding` 층은 크기가 `(samples, sequence_length, embedding_dimensionality)`인 3D 실수형 텐서를 반환합니다. 이런 3D 텐서는 RNN 층이나 1D 합성곱 층에서 처리됩니다(둘 다 이어지는 절에서 소개하겠습니다).\n",
    "\n",
    "`Embedding` 층의 객체를 생성할 때 가중치(토큰 벡터를 위한 내부 딕셔너리)는 다른 층과 마찬가지로 랜덤하게 초기화됩니다. 훈련하면서 이 단어 벡터는 역전파를 통해 점차 조정되어 이어지는 모델이 사용할 수 있도록 임베팅 공간을 구성합니다. 훈련이 끝나면 임베딩 공간은 특정 문제에 특화된 구조를 많이 가지게 됩니다.\n",
    "\n",
    "이를 익숙한 IMDB 영화 리뷰 감성 예측 문제에 적용해 보죠. 먼저 데이터를 준비합니다. 영화 리뷰에서 가장 빈도가 높은 10,000개의 단어를 추출하고(처음 이 데이터셋으로 작업했던 것과 동일합니다) 리뷰에서 20개 단어 이후는 버립니다. 이 네트워크는 10,000개의 단어에 대해 8 차원의 임베딩을 학습하여 정수 시퀀스 입력(2D 정수 텐서)를 임베딩 시퀀스(3D 실수형 텐서)로 바꿀 것입니다. 그 다음 이 텐서를 2D로 펼쳐서 분류를 위한 `Dense` 층을 훈련하겠습니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "from keras.datasets import imdb\n",
    "from keras import preprocessing\n",
    "\n",
    "# 특성으로 사용할 단어의 수\n",
    "max_features = 10000\n",
    "# 사용할 텍스트의 길이(가장 빈번한 max_features 개의 단어만 사용합니다)\n",
    "maxlen = 20\n",
    "\n",
    "# 정수 리스트로 데이터를 로드합니다.\n",
    "(x_train, y_train), (x_test, y_test) = imdb.load_data(num_words=max_features)\n",
    "\n",
    "# 리스트를 (samples, maxlen) 크기의 2D 정수 텐서로 변환합니다.\n",
    "x_train = preprocessing.sequence.pad_sequences(x_train, maxlen=maxlen)\n",
    "x_test = preprocessing.sequence.pad_sequences(x_test, maxlen=maxlen)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "embedding_2 (Embedding)      (None, 20, 8)             80000     \n",
      "_________________________________________________________________\n",
      "flatten_1 (Flatten)          (None, 160)               0         \n",
      "_________________________________________________________________\n",
      "dense_1 (Dense)              (None, 1)                 161       \n",
      "=================================================================\n",
      "Total params: 80,161\n",
      "Trainable params: 80,161\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n",
      "Train on 20000 samples, validate on 5000 samples\n",
      "Epoch 1/10\n",
      "20000/20000 [==============================] - 1s 75us/step - loss: 0.6759 - acc: 0.6050 - val_loss: 0.6398 - val_acc: 0.6814\n",
      "Epoch 2/10\n",
      "20000/20000 [==============================] - 1s 43us/step - loss: 0.5657 - acc: 0.7427 - val_loss: 0.5467 - val_acc: 0.7206\n",
      "Epoch 3/10\n",
      "20000/20000 [==============================] - 1s 43us/step - loss: 0.4752 - acc: 0.7808 - val_loss: 0.5113 - val_acc: 0.7384\n",
      "Epoch 4/10\n",
      "20000/20000 [==============================] - 1s 43us/step - loss: 0.4263 - acc: 0.8077 - val_loss: 0.5008 - val_acc: 0.7452\n",
      "Epoch 5/10\n",
      "20000/20000 [==============================] - 1s 43us/step - loss: 0.3930 - acc: 0.8258 - val_loss: 0.4981 - val_acc: 0.7538\n",
      "Epoch 6/10\n",
      "20000/20000 [==============================] - 1s 44us/step - loss: 0.3668 - acc: 0.8395 - val_loss: 0.5014 - val_acc: 0.7530\n",
      "Epoch 7/10\n",
      "20000/20000 [==============================] - 1s 43us/step - loss: 0.3435 - acc: 0.8533 - val_loss: 0.5052 - val_acc: 0.7520\n",
      "Epoch 8/10\n",
      "20000/20000 [==============================] - 1s 44us/step - loss: 0.3223 - acc: 0.8657 - val_loss: 0.5132 - val_acc: 0.7486\n",
      "Epoch 9/10\n",
      "20000/20000 [==============================] - 1s 44us/step - loss: 0.3022 - acc: 0.8766 - val_loss: 0.5213 - val_acc: 0.7490\n",
      "Epoch 10/10\n",
      "20000/20000 [==============================] - 1s 44us/step - loss: 0.2839 - acc: 0.8860 - val_loss: 0.5303 - val_acc: 0.7466\n"
     ]
    }
   ],
   "source": [
    "from keras.models import Sequential\n",
    "from keras.layers import Flatten, Dense, Embedding\n",
    "\n",
    "model = Sequential()\n",
    "# 나중에 임베딩된 입력을 Flatten 층에서 펼치기 위해 Embedding 층에 input_length를 지정합니다.\n",
    "model.add(Embedding(10000, 8, input_length=maxlen))\n",
    "# Embedding 층의 출력 크기는 (samples, maxlen, 8)가 됩니다.\n",
    "\n",
    "# 3D 임베딩 텐서를 (samples, maxlen * 8) 크기의 2D 텐서로 펼칩니다.\n",
    "model.add(Flatten())\n",
    "\n",
    "# 분류기를 추가합니다.\n",
    "model.add(Dense(1, activation='sigmoid'))\n",
    "model.compile(optimizer='rmsprop', loss='binary_crossentropy', metrics=['acc'])\n",
    "model.summary()\n",
    "\n",
    "history = model.fit(x_train, y_train,\n",
    "                    epochs=10,\n",
    "                    batch_size=32,\n",
    "                    validation_split=0.2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "약 75% 정도의 검증 정확도가 나옵니다. 리뷰에서 20개의 단어만 사용한 것치고 꽤 좋은 결과입니다. 하지만 임베딩 시퀀스를 펼치고 하나의 Dense 층을 훈련했으므로 입력 시퀀스에 있는 각 단어를 독립적으로 다루었습니다. 단어 사이의 관계나 문장의 구조를 고려하지 않았습니다(예를 들어 이 모델은 “this movie is a bomb”와 “this movie is the bomb”를 부정적인 리뷰로 동일하게 다룰 것입니다). 각 시퀀스 전체를 고려한 특성을 학습하도록 임베딩 층 위에 순환 층이나 1D 합성곱 층을 추가하는 것이 좋습니다. 다음 절에서 이에 관해 집중적으로 다루겠습니다."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 사전 훈련된 단어 임베딩 사용하기\n",
    "\n",
    "이따금 훈련 데이터가 부족하면 작업에 맞는 단어 임베딩을 학습할 수 없습니다. 이럴 땐 어떻게 해야 할까요?\n",
    "\n",
    "풀려는 문제와 함께 단어 임베딩을 학습하는 대신에 미리 계산된 임베딩 공간에서 임베딩 벡터를 로드할 수 있습니다. 이런 임베딩 공간은 뛰어난 구조와 유용한 성질을 가지고 있어서 언어 구조의 일반적인 측면을 잡아낼 수 있습니다. 자연어 처리에서 사전 훈련된 단어 임베딩을 사용하는 이유는 이미지 분류 문제에서 사전 훈련된 컨브넷을 사용하는 이유와 거의 동일합니다. 충분한 데이터가 없어서 자신만의 좋은 특성을 학습하지 못하지만 꽤 일반적인 특성이 필요할 때입니다. 이런 경우에는 다른 문제에서 학습한 특성을 재사용하는 것이 합리적입니다.\n",
    "\n",
    "단어 임베딩은 일반적으로 (문장이나 문서에 같이 등장하는 단어를 관찰하는) 단어 출현 통계를 사용하여 계산됩니다. 여기에는 여러 가지 기법이 사용되는데 신경망을 사용하는 것도 있고 그렇지 않은 방법도 있습니다. 단어를 위해 밀집된 저차원 임베딩 공간을 비지도 학습 방법으로 계산하는 아이디어는 요슈아 벤지오 등이 2000년대 초에 조사했습니다. 연구나 산업 애플리케이션에 적용되기 시작된 것은 Word2vec 알고리즘이 등장한 이후입니다. 이 알고리즘은 2013년 구글의 토마스 미코로프가 개발하였으며 가장 유명하고 성공적인 단어 임베딩 방법입니다. Word2vec의 차원은 성별 같은 구체적인 의미가 있는 속성을 잡아냅니다.\n",
    "\n",
    "케라스의 `Embedding` 층을 위해 내려받을 수 있는 미리 계산된 단어 임베딩 데이터베이스가 여럿 있습니다. Word2vec은 그 중 하나입니다. 인기 있는 또 다른 하나는 2014년 스탠포드 대학의 연구자들이 개발한 GloVe(Global Vectors for Word Representation)입니다. 이 임베딩 기법은 단어의 동시 출현 통계를 기록한 행렬을 분해하는 기법을 사용합니다. 이 개발자들은 위키피디아 데이터와 커먼 크롤 데이터에서 가져온 수백만 개의 영어 토큰에 대해서 임베딩을 미리 계산해 놓았습니다.\n",
    "\n",
    "GloVe 임베딩을 케라스 모델에 어떻게 사용하는지 알아보죠. Word2vec 임베딩이나 다른 단어 임베딩 데이터베이스도 방법은 같습니다. 앞서 보았던 텍스트 토큰화 기법도 다시 살펴보겠습니다. 원본 텍스트에서 시작해서 완전한 모델을 구성해 보겠습니다."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 모든 내용을 적용하기: 원본 텍스트에서 단어 임베딩까지\n",
    "\n",
    "앞서 만들었던 것과 비슷한 모델을 사용하겠습니다. 문장들을 벡터의 시퀀스로 임베딩하고 펼친 다음 그 위에 `Dense` 층을 훈련합니다. 여기서는 사전 훈련된 단어 임베딩을 사용하겠습니다. 케라스에 포함된 IMDB 데이터는 미리 토큰화가 되어 있습니다. 이를 사용하는 대신 원본 텍스트 데이터를 다운로딩해서 처음부터 시작하겠습니다."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 원본 IMDB 텍스트 다운로드하기\n",
    "\n",
    "먼저 http://mng.bz/0tIo 에서 IMDB 원본 데이터셋을 다운로드하고 압축을 해제합니다.\n",
    "\n",
    "훈련용 리뷰 하나를 문자열 하나로 만들어 훈련 데이터를 문자열의 리스트로 구성해 보죠. 리뷰 레이블(긍정/부정)도 `labels` 리스트로 만들겠습니다:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "\n",
    "imdb_dir = './datasets/aclImdb'\n",
    "train_dir = os.path.join(imdb_dir, 'train')\n",
    "\n",
    "labels = []\n",
    "texts = []\n",
    "\n",
    "for label_type in ['neg', 'pos']:\n",
    "    dir_name = os.path.join(train_dir, label_type)\n",
    "    for fname in os.listdir(dir_name):\n",
    "        if fname[-4:] == '.txt':\n",
    "            f = open(os.path.join(dir_name, fname), encoding='utf8')\n",
    "            texts.append(f.read())\n",
    "            f.close()\n",
    "            if label_type == 'neg':\n",
    "                labels.append(0)\n",
    "            else:\n",
    "                labels.append(1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 데이터 토큰화\n",
    "\n",
    "이전 절에서 소개한 개념을 사용해 텍스트를 벡터로 만들고 훈련 세트와 검증 세트로 나누겠습니다. 사전 훈련된 단어 임베딩은 훈련 데이터가 부족한 문제에 특히 유용합니다(그렇지 않으면 문제에 특화된 임베딩이 훨씬 성능이 좋습니다). 그래서 다음과 같이 훈련 데이터를 처음 200개의 샘플로 제한합니다. 이 모델은 200개의 샘플을 학습한 후에 영화 리뷰를 분류할 것입니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "88582개의 고유한 토큰을 찾았습니다.\n",
      "데이터 텐서의 크기: (25000, 100)\n",
      "레이블 텐서의 크기: (25000,)\n"
     ]
    }
   ],
   "source": [
    "from keras.preprocessing.text import Tokenizer\n",
    "from keras.preprocessing.sequence import pad_sequences\n",
    "import numpy as np\n",
    "\n",
    "maxlen = 100  # 100개 단어 이후는 버립니다\n",
    "training_samples = 200  # 훈련 샘플은 200개입니다\n",
    "validation_samples = 10000  # 검증 샘플은 10,000개입니다\n",
    "max_words = 10000  # 데이터셋에서 가장 빈도 높은 10,000개의 단어만 사용합니다\n",
    "\n",
    "tokenizer = Tokenizer(num_words=max_words)\n",
    "tokenizer.fit_on_texts(texts)\n",
    "sequences = tokenizer.texts_to_sequences(texts)\n",
    "\n",
    "word_index = tokenizer.word_index\n",
    "print('%s개의 고유한 토큰을 찾았습니다.' % len(word_index))\n",
    "\n",
    "data = pad_sequences(sequences, maxlen=maxlen)\n",
    "\n",
    "labels = np.asarray(labels)\n",
    "print('데이터 텐서의 크기:', data.shape)\n",
    "print('레이블 텐서의 크기:', labels.shape)\n",
    "\n",
    "# 데이터를 훈련 세트와 검증 세트로 분할합니다.\n",
    "# 샘플이 순서대로 있기 때문에 (부정 샘플이 모두 나온 후에 긍정 샘플이 옵니다) \n",
    "# 먼저 데이터를 섞습니다.\n",
    "indices = np.arange(data.shape[0])\n",
    "np.random.shuffle(indices)\n",
    "data = data[indices]\n",
    "labels = labels[indices]\n",
    "\n",
    "x_train = data[:training_samples]\n",
    "y_train = labels[:training_samples]\n",
    "x_val = data[training_samples: training_samples + validation_samples]\n",
    "y_val = labels[training_samples: training_samples + validation_samples]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### GloVe 단어 임베딩 내려받기\n",
    "\n",
    "https://nlp.stanford.edu/projects/glove 에서 2014년 영문 위키피디아를 사용해 사전에 계산된 임베딩을 내려받습니다. 이 파일의 이름은 glove.6B.zip이고 압축 파일 크기는 823MB입니다. 400,000만개의 단어(또는 단어가 아닌 토큰)에 대한 100차원의 임베딩 벡터를 포함하고 있습니다. datasets 폴더 아래에 파일 압축을 해제합니다.(이 저장소에는 이미 포함되어 있습니다)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 임베딩 전처리\n",
    "\n",
    "압축 해제한 파일(.txt 파일)을 파싱하여 단어(즉 문자열)와 이에 상응하는 벡터 표현(즉 숫자 벡터)를 매핑하는 인덱스를 만듭니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "400000개의 단어 벡터를 찾았습니다.\n"
     ]
    }
   ],
   "source": [
    "glove_dir = './datasets/'\n",
    "\n",
    "embeddings_index = {}\n",
    "f = open(os.path.join(glove_dir, 'glove.6B.100d.txt'), encoding=\"utf8\")\n",
    "for line in f:\n",
    "    values = line.split()\n",
    "    word = values[0]\n",
    "    coefs = np.asarray(values[1:], dtype='float32')\n",
    "    embeddings_index[word] = coefs\n",
    "f.close()\n",
    "\n",
    "print('%s개의 단어 벡터를 찾았습니다.' % len(embeddings_index))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "그다음 `Embedding` 층에 주입할 수 있도록 임베딩 행렬을 만듭니다. 이 행렬의 크기는 `(max_words, embedding_dim)`이어야 합니다. 이 행렬의 `i`번째 원소는 (토큰화로 만든) 단어 인덱스의 `i`번째 단어에 상응하는 `embedding_dim` 차원 벡터입니다. 인덱스 `0`은 어떤 단어나 토큰도 아닐 경우를 나타냅니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "embedding_dim = 100\n",
    "\n",
    "embedding_matrix = np.zeros((max_words, embedding_dim))\n",
    "for word, i in word_index.items():\n",
    "    embedding_vector = embeddings_index.get(word)\n",
    "    if i < max_words:\n",
    "        if embedding_vector is not None:\n",
    "            # 임베딩 인덱스에 없는 단어는 모두 0이 됩니다.\n",
    "            embedding_matrix[i] = embedding_vector"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 모델 정의하기\n",
    "\n",
    "이전과 동일한 구조의 모델을 사용하겠습니다:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "embedding_3 (Embedding)      (None, 100, 100)          1000000   \n",
      "_________________________________________________________________\n",
      "flatten_2 (Flatten)          (None, 10000)             0         \n",
      "_________________________________________________________________\n",
      "dense_2 (Dense)              (None, 32)                320032    \n",
      "_________________________________________________________________\n",
      "dense_3 (Dense)              (None, 1)                 33        \n",
      "=================================================================\n",
      "Total params: 1,320,065\n",
      "Trainable params: 1,320,065\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "from keras.models import Sequential\n",
    "from keras.layers import Embedding, Flatten, Dense\n",
    "\n",
    "model = Sequential()\n",
    "model.add(Embedding(max_words, embedding_dim, input_length=maxlen))\n",
    "model.add(Flatten())\n",
    "model.add(Dense(32, activation='relu'))\n",
    "model.add(Dense(1, activation='sigmoid'))\n",
    "model.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 모델에 GloVe 임베딩 로드하기\n",
    "\n",
    "`Embedding` 층은 하나의 가중치 행렬을 가집니다. 이 행렬은 2D 부동 소수 행렬이고 각 `i`번째 원소는 `i`번째 인덱스에 상응하는 단어 벡터입니다. 간단하네요. 모델의 첫 번째 층인 `Embedding` 층에 준비된 GloVe 행렬을 로드하세요:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.layers[0].set_weights([embedding_matrix])\n",
    "model.layers[0].trainable = False"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "추가적으로 `Embedding` 층을 동결합니다(`trainable` 속성을 `False`로 설정합니다). 사전 훈련된 컨브넷 특성을 사용할 때와 같은 이유입니다. 모델의 일부는 (`Embedding` 층처럼) 사전 훈련되고 다른 부분은 (최상단 분류기처럼) 랜덤하게 초기화되었다면 훈련하는 동안 사전 훈련된 부분이 업데이트되면 안됩니다. 이미 알고 있던 정보를 모두 잃게 됩니다. 랜덤하게 초기화된 층에서 대량의 그래디언트 업데이트가 발생하면 이미 학습된 특성을 오염시키기 때문입니다."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 모델 훈련과 평가\n",
    "\n",
    "모델을 컴파일하고 훈련합니다:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train on 200 samples, validate on 10000 samples\n",
      "Epoch 1/10\n",
      "200/200 [==============================] - 0s 2ms/step - loss: 1.6733 - acc: 0.5400 - val_loss: 0.6907 - val_acc: 0.5338\n",
      "Epoch 2/10\n",
      "200/200 [==============================] - 0s 964us/step - loss: 0.6077 - acc: 0.6700 - val_loss: 0.8459 - val_acc: 0.5060\n",
      "Epoch 3/10\n",
      "200/200 [==============================] - 0s 976us/step - loss: 0.4950 - acc: 0.7650 - val_loss: 0.6870 - val_acc: 0.5583\n",
      "Epoch 4/10\n",
      "200/200 [==============================] - 0s 990us/step - loss: 0.3349 - acc: 0.8950 - val_loss: 0.8019 - val_acc: 0.5098\n",
      "Epoch 5/10\n",
      "200/200 [==============================] - 0s 986us/step - loss: 0.2251 - acc: 0.9600 - val_loss: 0.7647 - val_acc: 0.5462\n",
      "Epoch 6/10\n",
      "200/200 [==============================] - 0s 983us/step - loss: 0.3254 - acc: 0.8150 - val_loss: 0.6875 - val_acc: 0.5835\n",
      "Epoch 7/10\n",
      "200/200 [==============================] - 0s 982us/step - loss: 0.1144 - acc: 0.9950 - val_loss: 1.0806 - val_acc: 0.5026\n",
      "Epoch 8/10\n",
      "200/200 [==============================] - 0s 1ms/step - loss: 0.1141 - acc: 0.9800 - val_loss: 0.7699 - val_acc: 0.5488\n",
      "Epoch 9/10\n",
      "200/200 [==============================] - 0s 996us/step - loss: 0.0671 - acc: 0.9950 - val_loss: 0.7842 - val_acc: 0.5610\n",
      "Epoch 10/10\n",
      "200/200 [==============================] - 0s 974us/step - loss: 0.0814 - acc: 0.9850 - val_loss: 1.6252 - val_acc: 0.5076\n"
     ]
    }
   ],
   "source": [
    "model.compile(optimizer='rmsprop',\n",
    "              loss='binary_crossentropy',\n",
    "              metrics=['acc'])\n",
    "history = model.fit(x_train, y_train,\n",
    "                    epochs=10,\n",
    "                    batch_size=32,\n",
    "                    validation_data=(x_val, y_val))\n",
    "model.save_weights('pre_trained_glove_model.h5')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "이제 모델의 성능을 그래프로 그려 보겠습니다:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAAEYCAYAAAAJeGK1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3Xl4VOX5//H3zWYIu4BVQQilrmCAGHEDq2LdquDWCgUrbqlatK1tLYot/FRqv3Vt61Kp1toSoahV0VJptVq3qgRRFKiyCBhQCAgIBIGQ+/fHM0mGkGUSJ5kzyed1XXNlzplnztxzJnPueZbzHHN3REREoqZFqgMQERGpihKUiIhEkhKUiIhEkhKUiIhEkhKUiIhEkhKUiIhEkhKU1JuZtTSzLWbWK5llU8nMvmZmST/3wsxONrPlccsfmNnQRMrW47UeNLMb6vt8kaholeoApPGY2Za4xUxgO7Artvw9d8+vy/bcfRfQPtllmwN3PzgZ2zGzy4Ax7n5C3LYvS8a2RVJNCaoZcffyBBH7hX6Zuz9fXXkza+XuJY0Rm0ht9P/Y/KiJT8qZ2S1m9lczm2Zmm4ExZnaMmb1hZhvN7BMz+62ZtY6Vb2VmbmZZseWpscf/YWabzey/ZtanrmVjj59uZh+a2SYz+52ZvWZmY6uJO5EYv2dmS8xsg5n9Nu65Lc3sLjNbb2ZLgdNq2D83mtn0SuvuNbM7Y/cvM7NFsfezNFa7qW5bhWZ2Qux+ppn9JRbbAuCIKl53WWy7C8xseGz94cA9wNBY8+m6uH07Ke75V8Te+3oze8rM9ktk39RlP5fFY2bPm9lnZvapmV0X9zo/j+2Tz82swMz2r6o51cxeLfucY/vz5djrfAbcaGYHmtmLsfeyLrbfOsU9v3fsPRbFHv+NmWXEYj40rtx+ZlZsZl2re78SAe6uWzO8AcuBkyutuwXYAZxF+PHSFjgSOIpQ2/4q8CEwLla+FeBAVmx5KrAOyAVaA38Fptaj7D7AZmBE7LFrgZ3A2GreSyIxPg10ArKAz8reOzAOWAD0BLoCL4evRZWv81VgC9AubttrgdzY8lmxMgacBGwDsmOPnQwsj9tWIXBC7P7twEtAF6A3sLBS2W8D+8U+k+/EYvhK7LHLgJcqxTkVmBS7f0osxoFABnAf8O9E9k0d93MnYA3wA2AvoCMwOPbY9cC7wIGx9zAQ2Bv4WuV9Dbxa9jnH3lsJcCXQkvD/eBAwDGgT+z95Dbg97v28H9uf7WLlj4s9NgWYHPc6PwaeTPX3ULdajlOpDkC3FH3w1Seof9fyvJ8Aj8XuV5V0fh9Xdjjwfj3KXgK8EveYAZ9QTYJKMMaj4x7/G/CT2P2XCU2dZY+dUfmgWWnbbwDfid0/HfiwhrLPAt+P3a8pQa2M/yyAq+LLVrHd94Fvxu7XlqAeAX4Z91hHQr9jz9r2TR3384VAQTXllpbFW2l9IglqWS0xnA/Mid0fCnwKtKyi3HHAR4DFlt8Bzk3290q35N7UxCeVfRy/YGaHmNnfY002nwM3Ad1qeP6ncfeLqXlgRHVl94+Pw8MRpbC6jSQYY0KvBayoIV6AR4FRsfvfAcoHlpjZmWb2ZqyJayOh9lLTviqzX00xmNlYM3s31ky1ETgkwe1CeH/l23P3z4ENQI+4Mgl9ZrXs5wOAJdXEcAAhSdVH5f/Hfc1shpmtisXwp0oxLPcwIGc37v4aoTY2xMz6A72Av9czJmkkSlBSWeUh1g8QfrF/zd07Ar8g1Gga0ieEX/gAmJmx+wG1si8T4yeEA1uZ2obB/xU42cx6EpogH43F2BZ4HLiV0PzWGfhngnF8Wl0MZvZV4H5CM1fX2Hb/F7fd2obEryY0G5ZtrwOhKXFVAnFVVtN+/hjoW83zqntsayymzLh1+1YqU/n9/R9h9OnhsRjGVoqht5m1rCaOPwNjCLW9Ge6+vZpyEhFKUFKbDsAmYGusk/l7jfCazwI5ZnaWmbUi9Gt0b6AYZwA/NLMesQ7zn9VU2N3XEJqhHgY+cPfFsYf2IvSLFAG7zOxMQl9JojHcYGadLZwnNi7usfaEg3QRIVdfRqhBlVkD9IwfrFDJNOBSM8s2s70ICfQVd6+2RlqDmvbzTKCXmY0zszZm1tHMBsceexC4xcz6WjDQzPYmJOZPCYNxWppZHnHJtIYYtgKbzOwAQjNjmf8C64FfWhh40tbMjot7/C+EJsHvEJKVRJwSlNTmx8BFhEELDxBqEA0qlgQuAO4kHHD6AvMIv5yTHeP9wAvAe8AcQi2oNo8S+pQejYt5I/Aj4EnCQIPzCYk2ERMJNbnlwD+IO3i6+3zgt8BbsTKHAG/GPfdfwGJgjZnFN9WVPf85QlPck7Hn9wJGJxhXZdXuZ3ffBHwDOI8wKOND4Ouxh28DniLs588JAxYyYk23lwM3EAbMfK3Se6vKRGAwIVHOBJ6Ii6EEOBM4lFCbWkn4HMoeX074nHe4++t1fO+SAmUdhiKRFWuyWQ2c7+6vpDoeSV9m9mfCwItJqY5FaqcTdSWSzOw0QpPNF4RhyiWEWoRIvcT680YAh6c6FkmMmvgkqoYAywhNP6cBZ6tTW+rLzG4lnIv1S3dfmep4JDFq4hMRkUhSDUpERCIpZX1Q3bp186ysrFS9vIiIpMjcuXPXuXtNp44AKUxQWVlZFBQUpOrlRUQkRcysthlbADXxiYhIRClBiYhIJClBiYhIJEXqRN2dO3dSWFjIF198kepQpAYZGRn07NmT1q2rm/5NROTLqzVBmdkfCfNbrXX3/lU8bsBvCNfRKSZcy+Xt+gRTWFhIhw4dyMrKImxWosbdWb9+PYWFhfTp06f2J4iI1FMiTXx/oobLYBMu2nZg7JZHmHyzXr744gu6du2q5BRhZkbXrl1VyxWRBldrgnL3lwmzM1dnBPBnD94AOpvZfvUNSMkp+vQZSW3y8yErC1q0CH/z82t7hoD2W2XJ6IPqwe5XvSyMrfukcsHY9V7yAHr1qu26cCKSjvLzIS8PiovD8ooVYRlgdH0v9NEMaL/tKRmj+Kr6OV3lBH/uPsXdc909t3v3Wk8ibnTr169n4MCBDBw4kH333ZcePXqUL+/YsSOhbVx88cV88MEHNZa59957yW/uP42kyZowoeIgW6a4OKyX6mm/7SkZNahCdr9cdU/CtXsaXH5++PBWroRevWDy5C/3S6Nr16688847AEyaNIn27dvzk5/8ZLcy7o6706JF1bn94YcfrvV1vv/979c/SJGIW1nNXOHVrZdA+21PyahBzQS+G7uU89HAJnffo3kv2cqqwytWgHtFdbghKiZLliyhf//+XHHFFeTk5PDJJ5+Ql5dHbm4u/fr146abbiovO2TIEN555x1KSkro3Lkz48ePZ8CAARxzzDGsXbsWgBtvvJG77767vPz48eMZPHgwBx98MK+/Hi70uXXrVs477zwGDBjAqFGjyM3NLU+e8SZOnMiRRx5ZHl/Z7PQffvghJ510EgMGDCAnJ4fly5cD8Mtf/pLDDz+cAQMGMKE5/zSTBlNd671a9WsW9f2Wkv6xshpBdTdgGqE/aSehtnQpcAVwRexxA+4FlhIup5xb2zbdnSOOOMIrW7hw4R7rqtO7t3tITbvfevdOeBM1mjhxot92223u7r548WI3M3/rrbfKH1+/fr27u+/cudOHDBniCxYscHf34447zufNm+c7d+50wGfNmuXu7j/60Y/81ltvdXf3CRMm+F133VVe/rrrrnN396efftpPPfVUd3e/9dZb/aqrrnJ393feecdbtGjh8+bN2yPOsjhKS0t95MiR5a+Xk5PjM2fOdHf3bdu2+datW33mzJk+ZMgQLy4u3u259VGXz0qal6lT3TMzd/9eZmaG9VK9KO+3ZMcGFHgCeSKRUXyj3H0/d2/t7j3d/SF3/727/z72uLv79929r7sf7u6NMgNsY1eH+/bty5FHHlm+PG3aNHJycsjJyWHRokUsXLhwj+e0bduW008/HYAjjjiivBZT2bnnnrtHmVdffZWRI0cCMGDAAPr161flc1944QUGDx7MgAED+M9//sOCBQvYsGED69at46yzzgLCibWZmZk8//zzXHLJJbRt2xaAvffeu+47QqQWo0fDlCnQuzeYhb9TpkSnoz+qI+WivN9S1T8WqZkk6qJXr9CsV9X6htCuXbvy+4sXL+Y3v/kNb731Fp07d2bMmDFVnhfUpk2b8vstW7akpKSkym3vtddee5TxBC4kWVxczLhx43j77bfp0aMHN954Y3kcVQ0Fd3cNEZdGMXp0NA6slUV9pFxU91uq+sfSdi6+yZMhM3P3dZmZYX1D+/zzz+nQoQMdO3bkk08+Yfbs2Ul/jSFDhjBjxgwA3nvvvSpraNu2baNFixZ069aNzZs388QTTwDQpUsXunXrxjPPPAOEE6CLi4s55ZRTeOihh9i2bRsAn31W0+ltIk2PRsrVT6r6x9I2QaWyOpyTk8Nhhx1G//79ufzyyznuuOOS/hpXX301q1atIjs7mzvuuIP+/fvTqVOn3cp07dqViy66iP79+3POOedw1FFHlT+Wn5/PHXfcQXZ2NkOGDKGoqIgzzzyT0047jdzcXAYOHMhdd92V9LhFokwj5eonVRUCS6QpqSHk5uZ65QsWLlq0iEMPPTQl8URNSUkJJSUlZGRksHjxYk455RQWL15Mq1bRaJXVZyXpKCur6q6B3r2hmi5iiUnmaT1mNtfdc2srF42jnexhy5YtDBs2jJKSEtydBx54IDLJSSRdTZ68ex8UNF7XQLpLRf+YjngR1blzZ+bOnZvqMESalLIDbDJP8JeGowQlIs1KVEfKyZ7SdpCEiIg0bUpQIrWI6omdIk2dmvhEahD1EztFmjLVoOKccMIJe5x0e/fdd3PVVVfV+Lz27dsDsHr1as4///xqt115WH1ld999N8Vxw4vOOOMMNm7cmEjo0kB0YqdI6ihBxRk1ahTTp0/fbd306dMZNWpUQs/ff//9efzxx+v9+pUT1KxZs+jcuXO9tydfnk7sFEkdJag4559/Ps8++yzbt28HYPny5axevZohQ4aUn5eUk5PD4YcfztNPP73H85cvX07//v2BMA3RyJEjyc7O5oILLiifXgjgyiuvLL9Ux8SJEwH47W9/y+rVqznxxBM58cQTAcjKymLdunUA3HnnnfTv35/+/fuXX6pj+fLlHHrooVx++eX069ePU045ZbfXKfPMM89w1FFHMWjQIE4++WTWrFkDhHOtLr74Yg4//HCys7PLp0p67rnnyMnJYcCAAQwbNiwp+zZdRf0SCCJNWWT7oH74Q6ji8kdfysCBEDu2V6lr164MHjyY5557jhEjRjB9+nQuuOACzIyMjAyefPJJOnbsyLp16zj66KMZPnx4tZOv3n///WRmZjJ//nzmz59PTk5O+WOTJ09m7733ZteuXQwbNoz58+dzzTXXcOedd/Liiy/SrVu33bY1d+5cHn74Yd58803cnaOOOoqvf/3rdOnShcWLFzNt2jT+8Ic/8O1vf5snnniCMWPG7Pb8IUOG8MYbb2BmPPjgg/z617/mjjvu4Oabb6ZTp0689957AGzYsIGioiIuv/xyXn75Zfr06dPs5+vTiZ0iqaMaVCXxzXzxzXvuzg033EB2djYnn3wyq1atKq+JVOXll18uTxTZ2dlkZ2eXPzZjxgxycnIYNGgQCxYsqHIi2Hivvvoq55xzDu3ataN9+/ace+65vPLKKwD06dOHgQMHAtVf0qOwsJBTTz2Vww8/nNtuu40FCxYA8Pzzz+92dd8uXbrwxhtvcPzxx9OnTx9Al+SI8iUQRJq6yNagaqrpNKSzzz6ba6+9lrfffptt27aV13zy8/MpKipi7ty5tG7dmqysrCovsRGvqtrVRx99xO23386cOXPo0qULY8eOrXU7Nc2XWHapDgiX66iqie/qq6/m2muvZfjw4bz00ktMmjSpfLuVY9QlOfakEztFUkM1qErat2/PCSecwCWXXLLb4IhNmzaxzz770Lp1a1588UVWVDXjZJzjjz+e/NgJM++//z7z588HwqU62rVrR6dOnVizZg3/+Mc/yp/ToUMHNm/eXOW2nnrqKYqLi9m6dStPPvkkQ4cOTfg9bdq0iR49egDwyCOPlK8/5ZRTuOeee8qXN2zYwDHHHMN//vMfPvroI0CX5BCR1FGCqsKoUaN49913y69oCzB69GgKCgrIzc0lPz+fQw45pMZtXHnllWzZsoXs7Gx+/etfM3jwYCBcHXfQoEH069ePSy65ZLdLdeTl5XH66aeXD5Iok5OTw9ixYxk8eDBHHXUUl112GYMGDUr4/UyaNIlvfetbDB06dLf+rRtvvJENGzbQv39/BgwYwIsvvkj37t2ZMmUK5557LgMGDOCCCy5I+HVERJJJl9uQetFnJSL1lejlNlSDEhGRSFKCEhGRSIpcgkpVk6MkTp+RiDSGSCWojIwM1q9frwNghLk769evJyMjI9WhiEgTF6nzoHr27ElhYSFFRUWpDkVqkJGRQc+ePVMdhog0cZFKUK1bty6fwUBERJq3SDXxiYiIlFGCEhGRSFKCEhGRSFKCEhGRSFKCEhGRSFKCEhGRSFKCEhGRSFKCEhGRSFKCEhGRSEooQZnZaWb2gZktMbPxVTze28xeMLP5ZvaSmWkeHBER+VJqTVBm1hK4FzgdOAwYZWaHVSp2O/Bnd88GbgJuTXagIiLSvCRSgxoMLHH3Ze6+A5gOjKhU5jDghdj9F6t4XKRG+fmQlQUtWoS/+fmpjkhEUi2RBNUD+DhuuTC2Lt67wHmx++cAHcysa+UNmVmemRWYWYFmLJcy+fmQlwcrVoB7+JuXpyQl0twlkqCsinWVL9j0E+DrZjYP+DqwCijZ40nuU9w9191zu3fvXudgpWmaMAGKi3dfV1wc1otI85XI5TYKgQPilnsCq+MLuPtq4FwAM2sPnOfum5IVpDRtK1fWbb2INA+J1KDmAAeaWR8zawOMBGbGFzCzbmZWtq3rgT8mN0xpynr1qtt6EWkeak1Q7l4CjANmA4uAGe6+wMxuMrPhsWInAB+Y2YfAV4DJDRSvNEGTJ0Nm5u7rMjPDehFpvsy9cndS48jNzfWCgoKUvLZET35+6HNauTLUnCZPhtGjUx2ViDQEM5vr7rm1lYvUJd+l+Ro9WglJRHanqY5ERCSSlKBERCSSlKBERCSSlKBERCSSlKBERCSSlKBERCSSlKBERCSSlKBERCSSlKBERCSSlKBERCSSlKBERCSSlKBERCSSlKBERCSSlKBERCSSlKBERCSSlKBERCSSlKBERCSSlKBERCSSlKBERCSSlKBERCSSlKBERCSSlKBERCSSlKBERCSSlKBERCSSlKBERCSSlKBERCSSlKBERCSSlKBE0lR+PmRlQYsW4W9+fqojEkmuVqkOQETqLj8f8vKguDgsr1gRlgFGj05dXCLJpBqUSBqaMKEiOZUpLg7rRZoKJSiRNLRyZd3Wi6QjJSiRNNSrV93Wi6QjJSiRNDR5MmRm7r4uMzOsF2kqEkpQZnaamX1gZkvMbHwVj/cysxfNbJ6ZzTezM5IfqoiUGT0apkyB3r3BLPydMkUDJKRpMXevuYBZS+BD4BtAITAHGOXuC+PKTAHmufv9ZnYYMMvds2rabm5urhcUFHzJ8EVEJN2Y2Vx3z62tXCI1qMHAEndf5u47gOnAiEplHOgYu98JWF2XYEVERCpLJEH1AD6OWy6MrYs3CRhjZoXALODqqjZkZnlmVmBmBUVFRfUIV0REmotEEpRVsa5yu+Ao4E/u3hM4A/iLme2xbXef4u657p7bvXv3ukcrIiLNRiIJqhA4IG65J3s24V0KzABw9/8CGUC3ZAQoIiLNUyIJag5woJn1MbM2wEhgZqUyK4FhAGZ2KCFBqQ1PRETqrdYE5e4lwDhgNrAImOHuC8zsJjMbHiv2Y+ByM3sXmAaM9dqGB4qIiNQgocli3X0WYfBD/LpfxN1fCByX3NBERKQ500wSIiISSUpQIiISSUpQIiISSUpQIiISSUpQIiISSUpQIiISSUpQzUh+PmRlQYsW4W9+fqojEhGpXkLnQUn6y8+HvDwoLg7LK1aEZdA1hEQkmlSDaiYmTKhITmWKi8N6EZEoUoJqJlaurNt6EZFUU4JqJnr1qtt6EZFUU4JqJiZPhszM3ddlZob1IiJRpATVTIweDVOmQO/eYBb+TpmiARIiEl0axdeMjB6thCQi6UM1KBERiSQlKBERiSQlKBERiSQlKBERiSQlKBERiSQlKBERiSQlKBERiSQlKBERiSQlKBERiSQlKBERiSQlKBERiSQlKBERiSQlKBERiSQlKBERiSQlKBERiSQlKBERiSQlKBERiSQlKBERiSQlKBERiSQlKBERiaSEEpSZnWZmH5jZEjMbX8Xjd5nZO7Hbh2a2MfmhiohIc9KqtgJm1hK4F/gGUAjMMbOZ7r6wrIy7/yiu/NXAoAaIVUREmpFEalCDgSXuvszddwDTgRE1lB8FTEtGcCIi0nwlkqB6AB/HLRfG1u3BzHoDfYB/V/N4npkVmFlBUVFRXWMVEZFmJJEEZVWs82rKjgQed/ddVT3o7lPcPdfdc7t3755ojCIi0gwlkqAKgQPilnsCq6spOxI174mISBIkkqDmAAeaWR8za0NIQjMrFzKzg4EuwH+TG6KIiDRHtSYody8BxgGzgUXADHdfYGY3mdnwuKKjgOnuXl3zn4iISMJqHWYO4O6zgFmV1v2i0vKk5IUlIiLNnWaSEBGRSFKCEhGRSFKCEhGRSFKCEhGRSFKCSrL8fMjKghYtwt/8/FRHJCKSnhIaxSeJyc+HvDwoLg7LK1aEZYDRo1MXl4hIOlINKokmTKhITmWKi8N6ERGpGyWoJFq5sm7rRUSkekpQSdSrV93Wi4hI9ZSgkmjyZMjM3H1dZmZYLyIidaMElUSjR8OUKdC7N5iFv1OmaICEiEh9aBRfko0erYQkIpIMqkGJiEgkKUGJiEgkKUGJiEgkKUGJiEgkKUGJiEgkKUGJiEgkKUGJiEgkKUGJiEgkKUGJiEgkKUGJiEgkKUGJiEgkKUGJiEgkKUGJiEgkKUGJiEgkKUGJiEgkKUGJiEgkKUGJiEgkKUGJiEgkKUGJiEgkKUGJiEgkKUGJiEgkKUGJiEgkJZSgzOw0M/vAzJaY2fhqynzbzBaa2QIzezS5YYqISHPTqrYCZtYSuBf4BlAIzDGzme6+MK7MgcD1wHHuvsHM9mmogEVEpHlIpAY1GFji7svcfQcwHRhRqczlwL3uvgHA3dcmN0wREWluEklQPYCP45YLY+viHQQcZGavmdkbZnZaVRsyszwzKzCzgqKiovpFLCIizUIiCcqqWOeVllsBBwInAKOAB82s8x5Pcp/i7rnuntu9e/e6xioiIs1IIgmqEDggbrknsLqKMk+7+053/wj4gJCwRERE6iWRBDUHONDM+phZG2AkMLNSmaeAEwHMrBuhyW9ZMgMVEZHmpdYE5e4lwDhgNrAImOHuC8zsJjMbHis2G1hvZguBF4Gfuvv6hgpaRESaPnOv3J3UOHJzc72goCAlry0iIqljZnPdPbe2cppJQiSNffIJTJsG27enOhKR5FOCEkkz7vDSS/Dtb0OvXvCd78AZZ8Dnn6c6MpHkUoISSRObNsE990C/fnDiifD88/CDH8BvfgMvvwzHHx9qVCJNRa1THYlIas2fD/fdB1OnwtatcOSR8PDDcMEF0LZtKHPwwXDeeXDssfDcc2FZJN2pBiUSQdu3w6OPwpAhMGAAPPJIaNJ7661wGzu2IjkBnHpqaPbbuhWOOw7efDNVkYskjxKUSISsWAETJoS+pdGjYc0auOMOWLUK/vjHUHuqTm4uvP46dOoEJ50Es2Y1XtwiDUEJSiTFSkth9mwYMQK++lX41a/gmGPCug8+gGuvhb33TmxbX/taSFKHHALDh8Of/tSgoYs0KPVBiaTIZ5+FvqT774elS2GffeD66yEvL9Sg6usrXwnNfeedBxdfHAZOjB8PVtWsmiIRpgQl0sjmzAmDHqZPhy++CP1MN98M554Le+2VnNfo0AGefRYuuQRuuAFWr4a774aWLZOzfZHGoAQl0gi2bQsJ6b77oKAA2rULAx2uvBKysxvmNdu0gT//GfbbD26/HT79FP7yF8jIaJjXE0k2JSiRBrRkSWjCe/hh2LABDjssnMt04YXQsWPDv36LFnDbbSFJ/fjHUFQETz0Fnfe4GI6kwo4dYWDM0qXhtmwZLF8ORx8N11yTvBp1ulKCakbcYebMMDLswgt3H6YsybNrF/z976G2NHs2tGoVmu+uuiqcTJuKvqBrr4V99w21tuOPD+dK7b9/48fRHG3cGBJPWRIqS0RLl8LHH4dBMmUyMsKPib/9Df7wB/jtb+G0Ki//2jxosthmYv78MOvASy+F5Z494aab4LvfVb9EsqxZAw89BA88ACtXhgTwve/BZZdFJxk8/zycc04YFTh7dhjtJ19OaWk4DSA+8cTf/+yz3ct37w59+4YRm3377n5/v/3CD5h//CN8XxcvDqM777oL+vRJzftrCIlOFqsE1cStXw8//3k4aHbuDLfcEmYZuP76cMJnv35w661w5pka5VUf7vDaa6G29PjjsHMnDBsWaktnnQWtW6c6wj29/XaYu2/nzjCQ4phjUh1R9G3bBh99tGcSWro0rN+xo6Jsy5bQu/eeyadv35BkEm3a3b49JKabbw618vHj4Wc/axotH0pQzVxJSej7mDgxTCJ61VUwaVLF+TTu8MQTYYTX4sVhJNn//V+YKkdqt3kz5OeHxPTee+Hk2LFj4Yor0qNWsmxZmH1i1SqYMSP8QGnO3MOPuaqa4ZYuDaMg47VvX5F0KiehAw5I7g+TwkL46U/DIJusLLjzTjj77PT+QZlogsLdU3I74ogjXBrG88+79+vnDu7Dhrm/9171ZXfscP/979333TeUP/ts94ULGy/WdPP+++7f/757hw5hfw0c6P6HP7hv2ZLqyOpuzRr3I490b9nS/cEHUx1N4yktdf/Xv9x/+lP3884Ln2HHjuHzjL/tv7/70KHuF13kftNN7vnz+3CpAAANYklEQVT57v/9r/vatWEbje3FFyu+16ec4v6//zV+DMkCFHgCeUIJqglZtsz9nHPCp9qnj/uTTyb+Rdqyxf2WW8KBt0UL90svdf/444aNN11s3+7+17+6f/3rYd+2aeM+Zkw4WKXiQJVMmze7n3ZaeF8335z+76c2L70Ukk7Z53jwwe5nnOE+bpz7XXe5z5wZfoRs3ZrqSKu2Y4f73XeHhNq6tft117l//nmqo6q7Jp+gpk51793b3Sz8nTr1S20urW3e7H7DDe577eXerp375Mnu27bVb1tFRe4//GH458/IcP/Zz9w/+yy58aaLZcvcx49332ef8E3JynL/1a/CL+imZMcO9+9+N7zHK65wLylJdUTJ9/rroTWhrGZ0773uX3yR6qjq79NP3ceOrXg/jz6aXj8umnSCmjrVPTPTd6uOZ2Y2vyRVWhre8/77h30wZox7YWFytr1sWdiemXuXLu633Vb/pJdOdu4MNc9TTw3vvUUL9+HD3f/+96Z54C5TWhp+jECohTeVz7qgINSQIPzQuPNO9+LiVEeVPK+/7p6TE97f8ce7v/tuqiNKTJNOUL17+x7txRDWNxdz5rgfc0x437m57q+91jCvM29eRRPQAQe4P/xw0zxQr1zp/otfVCT7/fd3nzgxrG9O7r47JOahQ9O75jx/fkVz9957h5pvOvYTJqKkxP2BB8L7bNnS/Zpr3DdsSHVUNWvSCcqs6gRlVu9Npo1PP3W/5JLwXvfZx/2hh9x37Wr4133hhZAIIXTUPvNMejUpVKWkJNSMzjor1JTMQjJ+6qlQk2qupk8P/TP9+qVfP+SiRe4XXBA+y44d3f/f/3PftCnVUTWOdevcr7wyvPfu3Rvv2FAfTTpBNcca1PbtoZmtQ4fQP/STn7hv3Ni4MZSWus+Y4f61r4X9PXRoaGJIN6tXhwEhvXp5edPP9deHZk0JXngh/K/17Om+YEGqo6nd0qWhH61Fi9APe8MN7uvXpzqq1Jg71/3YY8P/9uDBobUlapp0gpo6NXTgxyenjIym2wf197+7H3RQeJ/f/Kb7Bx+kNp4dO9zvu8/9K1/x8j6LRYtSG1Ntdu0KQ4vPO8+9VasQ90knhYS7fXuqo4umefPC6Qdduri/+mqqo6naihXul18ePtOMDPcf/7jpDWKpj9JS9z//OXxHzcI+KipKdVQVmnSCcq8YahmfpLKz3X/+89Axmu7NT+7hPIeyDt6DDnKfNSvVEe1u8+Zwfkj79uGX6+WXJ2+QRrKsXev+61+79+0b9mPXruEgluokny4++ij872VkhMEjUbF6dRga3qZNuI0bF9bJ7jZudL/22tA31aVLGL0YhT7kJp+g4i1e7H777aHJqUWL8K569nS/6ir32bPT7xfyxo3hINqqVUjCt98e7fewdq37D34Qmh7btg3NZanspC0tDee7jBoVDl5lzZFTpzad0WmNqagoNBW1aBFO6k6ltWvDdyMjI3w/8vJCLUpq9v77ocWg7OTyV15JbTzNKkHFW7s2jDQ7++xwsIRwkB850n3atMbvt6mLXbtCx+Y++4Rq+aWXhkER6WLpUvfRo7185NTttzduQli/PpxsecghIYZOncKIpvffb7wYmqotWypq8xMnNn4Lxfr14YdPu3YhUV50Ufh/k8SVlro/9lgYjVt2Wkqqap3NNkHFKy4OZ4ZfemnFyZatW7t/4xvu99wTrSHEr73mfsQRIcZjj41mx2ai3n47nEdUNjT9T39quGaF0tIwUOOiiyr6JY86KvxIiepsAOlqx46Kk0Pz8hpnpOPGje6TJoUfmWbhh2bU+zujbssW9wkTQutChw7hh+SOHY0bgxJUJSUlIQlcd12Y3qSs3yonJwxFfeed1PRbFRZW1Dr23z80QzWF/jP3MCdgWdLt39/92WeT9942bQrt6dnZYfvt24dZEObNS872pWqlpeHgBu4jRjTcSa9btrjfemvoNykbiDN/fsO8VnO1eHFFrfjQQ8MgosaiBFWLRYvCyXvHHFNxXlVWVmgSeuGFhv9FsW1bGOqcmRmmKJowIQw6aGp27Qrz2JUNUjj++DCHXX0VFLhfdllo6gH3QYPCSYrpOB9ZOrvnnvC9OfbY5A7nLi4Osz107+7lo1bnzk3e9mVPzzzj/tWvhv19/vmN06enBFUHn34aZqQ+88yKZqIuXUIb7WOPJffgV1rq/re/hclcy34ZNoe29B07Qo2nrKn13HMTn4158+bw+ZTVxtq2DScrv/VW06ltpqPHHgvNRIce+uUPal98EZJe2UweJ5+cnufYpatt28JkwW3bhtsttzRs/7ESVD1t2RISyEUXhSHJZbMen356GMH0ZToV33uvYsLKfv1CE1hzs3lzaFJt3z4Mfc3Lc1+1quqy774bRmKWXdqif3/33/0u+tO4NCcvvRQGo/ToUfNlXaqzY0f48VF20vTQoWGbkhorVoRaFIRa1TPPNMzrKEElwc6d7v/5TziPoKyJquzs7MmTw+iwRH7Br18fztMoOxfhd79r3lPpuIdrEV19dcXQ9BtuCB3ixcVhUEXZPIN77eV+4YXhRFHVlqJp/vxQ8+nUKXxfElFSEk4kLfteDR4cTgnRZxwN//pXxWjYb34z9FclkxJUkpWWhoQ0eXL4MpUlq759QwJ7+eU9R6rt3BlmXOjaNQyNvfLKMF+WVFiyJJyvVDY0vaxT/KCDQl+E9ld6WL48HND22sv9iSeqL7drV5jrr+zgN3Bg05jXsSnavj2M8GvfPrQiTZiQvAl3laAa2KpVocnv9NMrTgbt1i0Mw33ySfd//rNihNkJJ6TPNPipMndu6JcaOTJcOVQHrPSzbl3FoKP77tv9sdLSMAlv2XfisMPcH388upOZSoXVq0N/fNlpI7Nnf/ltJjVBAacBHwBLgPFVPD4WKALeid0uq22b6Z6g4n3+eegwHjOmogZQNnntY4/pYCvNx9atYXZ4cL/xxvC/P2tWxUz4Bx4YLp0ehel2pG5eeSXUeBNtxq1JognKQtnqmVlL4EPgG0AhMAcY5e4L48qMBXLdfVyNG4uTm5vrBQUFiRZPGzt3wquvwooVcMEF0LZtqiMSaVwlJXDllfDgg9C7d/guZGXBL34BF14IrVqlOkKpr9JSaNHiy2/HzOa6e25t5RL5VxkMLHH3ZbENTwdGAAtrfFYz1bo1nHhiqqMQSZ1WrWDKFOjZE6ZNg/vvh0sugTZtUh2ZfFnJSE51er0EyvQAPo5bLoytq+w8M5tvZo+b2QFVbcjM8syswMwKioqK6hGuiKQDM5g4Ef73P7jiCiUnqZ9EEpRVsa5yu+AzQJa7ZwPPA49UtSF3n+Luue6e271797pFKiIizUoiCaoQiK8R9QRWxxdw9/Xuvj22+AfgiOSEJyIizVUiCWoOcKCZ9TGzNsBIYGZ8ATPbL25xOLAoeSGKiEhzVOsgCXcvMbNxwGygJfBHd19gZjcRhgrOBK4xs+FACfAZYdi5iIhIvdU6zLyhNNVh5iIiUrNEh5k38qBBERGRxChBiYhIJClBiYhIJClBiYhIJKVskISZFQErUvLijaMbsC7VQaQh7be60z6rH+23+knGfuvt7rXO1pCyBNXUmVlBIqNUZHfab3WnfVY/2m/105j7TU18IiISSUpQIiISSUpQDWdKqgNIU9pvdad9Vj/ab/XTaPtNfVAiIhJJqkGJiEgkKUGJiEgkKUElmZkdYGYvmtkiM1tgZj9IdUzpwsxamtk8M3s21bGkCzPrHLuK9f9i/3PHpDqmdGBmP4p9P983s2lmlpHqmKLGzP5oZmvN7P24dXub2b/MbHHsb5eGjEEJKvlKgB+7+6HA0cD3zeywFMeULn6AriVWV78BnnP3Q4ABaP/Vysx6ANcAue7en3AZoZGpjSqS/gScVmndeOAFdz8QeCG23GCUoJLM3T9x97dj9zcTDhg9UhtV9JlZT+CbwIOpjiVdmFlH4HjgIQB33+HuG1MbVdpoBbQ1s1ZAJpWuEi7g7i8Tru8XbwTwSOz+I8DZDRmDElQDMrMsYBDwZmojSQt3A9cBpakOJI18FSgCHo41jT5oZu1SHVTUufsq4HZgJfAJsMnd/5naqNLGV9z9Ewg/xoF9GvLFlKAaiJm1B54Afujun6c6nigzszOBte4+N9WxpJlWQA5wv7sPArbSwE0uTUGs32QE0AfYH2hnZmNSG5VURQmqAZhZa0Jyynf3v6U6njRwHDDczJYD04GTzGxqakNKC4VAobuX1dAfJyQsqdnJwEfuXuTuO4G/AcemOKZ0scbM9gOI/V3bkC+mBJVkZmaEPoFF7n5nquNJB+5+vbv3dPcsQmf1v91dv2hr4e6fAh+b2cGxVcOAhSkMKV2sBI42s8zY93UYGlySqJnARbH7FwFPN+SLtWrIjTdTxwEXAu+Z2TuxdTe4+6wUxiRN19VAvpm1AZYBF6c4nshz9zfN7HHgbcKo23lo2qM9mNk04ASgm5kVAhOBXwEzzOxSQqL/VoPGoKmOREQkitTEJyIikaQEJSIikaQEJSIikaQEJSIikaQEJSIikaQEJSIikaQEJSIikfT/AZH5wNsTfXVyAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAAEYCAYAAAAJeGK1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3Xl8VNX9//HXh0V2QQEVQQTUqoCAkFKtUnCrYBXEYgGDVgql7lptK3Wpy1d+4lLXWi1atUsSsO61Km5URCsSFiOLVKQIEYSIsgkKIZ/fH2eCScgyIZPcmcn7+XjkMXPvnLn3MzfJfO4599xzzN0RERFJNg2iDkBERKQ8SlAiIpKUlKBERCQpKUGJiEhSUoISEZGkpAQlIiJJSQlKImNmDc1si5l1TmTZKJnZoWaW8Hs3zOxkM1tRYnmpmQ2Ip+we7OsRM7tmT99fyXZvMbPHE71dSV+Nog5AUoeZbSmx2Bz4BtgZW/6Fu2dVZ3vuvhNomeiy9YG7H56I7ZjZeGCMuw8qse3xidi2SE0pQUnc3H1XgoidoY9399cqKm9mjdy9sC5iE5H0oyY+SZhYE840M8sxs83AGDM71szeNbMNZrbGzO4zs8ax8o3MzM2sS2z577HXXzKzzWb2HzPrWt2ysdeHmNl/zWyjmd1vZm+b2fkVxB1PjL8ws2Vm9qWZ3VfivQ3N7G4zW29mHwODKzk+15nZ1DLrHjCzu2LPx5vZktjn+ThWu6loW/lmNij2vLmZ/S0W2yKgXzn7XR7b7iIzGxpbfxTwB2BArPn08xLH9sYS778g9tnXm9mzZtYhnmNTFTM7MxbPBjN7w8wOL/HaNWa22sw2mdmHJT7rMWY2L7Z+rZndEe/+JAW5u370U+0fYAVwcpl1twDbgTMIJz/NgO8C3yPU1rsB/wUuiZVvBDjQJbb8d+BzIANoDEwD/r4HZfcDNgPDYq9dCewAzq/gs8QT43NAa6AL8EXxZwcuARYBnYC2wMzwb1XufroBW4AWJba9DsiILZ8RK2PAicA2oFfstZOBFSW2lQ8Mij2/E/g3sA9wMLC4TNmfAB1iv5NzYjHsH3ttPPDvMnH+Hbgx9vyHsRj7AE2BPwJvxHNsyvn8twCPx54fGYvjxNjv6JrYcW8M9AA+AQ6Ile0KdIs9nwOMjj1vBXwv6v8F/dTej2pQkmiz3P2f7l7k7tvcfY67z3b3QndfDkwBBlby/ifdPdfddwBZhC/G6pY9HVjg7s/FXrubkMzKFWeMt7r7RndfQUgGxfv6CXC3u+e7+3pgciX7WQ4sJCROgFOADe6eG3v9n+6+3IM3gNeBcjtClPET4BZ3/9LdPyHUikru9wl3XxP7nWQTTi4y4tguQCbwiLsvcPevgYnAQDPrVKJMRcemMqOA5939jdjvaDKwN+FEoZCQDHvEmon/Fzt2EE40DjOztu6+2d1nx/k5JAUpQUmirSq5YGZHmNm/zOwzM9sE3Ay0q+T9n5V4vpXKO0ZUVPbAknG4uxNqHOWKM8a49kU4869MNjA69vwcQmItjuN0M5ttZl+Y2QZC7aWyY1WsQ2UxmNn5ZvZ+rCltA3BEnNuF8Pl2bc/dNwFfAh1LlKnO76yi7RYRfkcd3X0pcBXh97Au1mR8QKzoWKA7sNTM3jOz0+L8HJKClKAk0cp2sf4TodZwqLvvDfyO0IRVm9YQmtwAMDOj9BdqWTWJcQ1wUInlqrrBTwNOjtVAhhESFmbWDHgSuJXQ/NYGeCXOOD6rKAYz6wY8CFwItI1t98MS262qS/xqQrNh8fZaEZoSP40jrupstwHhd/YpgLv/3d2PIzTvNSQcF9x9qbuPIjTj/h54ysya1jAWSVJKUFLbWgEbga/M7EjgF3WwzxeAvmZ2hpk1Ai4H2tdSjE8AV5hZRzNrC1xdWWF3XwvMAh4Dlrr7R7GXmgB7AQXATjM7HTipGjFcY2ZtLNwndkmJ11oSklABIVePJ9Sgiq0FOhV3CilHDjDOzHqZWRNConjL3SuskVYj5qFmNii2718TrhvONrMjzeyE2P62xX52Ej7AuWbWLlbj2hj7bEU1jEWSlBKU1LargJ8Svnz+RKhB1KpYEhgJ3AWsBw4B5hPu20p0jA8SrhV9QLiA/2Qc78kmdHrILhHzBuCXwDOEjgYjCIk2HjcQanIrgJeAv5bYbh5wH/BerMwRQMnrNq8CHwFrzaxkU13x+18mNLU9E3t/Z8J1qRpx90WEY/4gIXkOBobGrkc1AW4nXDf8jFBjuy721tOAJRZ6id4JjHT37TWNR5KTheZ5kfRlZg0JTUoj3P2tqOMRkfioBiVpycwGm1nrWDPR9YSeYe9FHJaIVIMSlKSr44HlhGaiwcCZ7l5RE5+IJCE18YmISFJSDUpERJJSZIPFtmvXzrt06RLV7kVEJCJz58793N0ru/UDiDBBdenShdzc3Kh2LyIiETGzqkZcAdTEJyIiSUoJSkREkpISlIiIJCUlKBERSUpKUCIikpSUoEREJCmlbILKyoIuXaBBg/CYlVXVO0REJJVEdh9UTWRlwYQJsHVrWP7kk7AMkFnjiQBERCQZpGQN6tprv01OxbZuDetFRCQ9pGSCWrmyeutFRCT1pGSC6ty5eutFRCT1pGSCmjQJmjcvva5587BeRETSQ0omqMxMmDIFDj4YzMLjlCnqICEikk5SshcfhGSkhCQikr5SsgYlIiJ1yx3uvz/c1lNXqkxQZvaoma0zs4WVlBlkZgvMbJGZvZnYEEVEJGrz58Nll8H06XW3z3hqUI8Dgyt60czaAH8Ehrp7D+DsxIQmIiLJIjsbGjeGH/+47vZZZYJy95nAF5UUOQd42t1XxsqvS1BsIiKSBIqKYNo0OPVUaNu27vabiGtQ3wH2MbN/m9lcMzsvAdsUEZEkMWsW5OfD6NF1u99E9OJrBPQDTgKaAf8xs3fd/b9lC5rZBGACQGfdVSsikhKys8O9pkOH1u1+E1GDygdedvev3P1zYCbQu7yC7j7F3TPcPaN9+/YJ2LWIiNSmHTvgySdDcmrZsm73nYgE9RwwwMwamVlz4HvAkgRsV0REIvbqq7B+fd0370EcTXxmlgMMAtqZWT5wA9AYwN0fcvclZvYykAcUAY+4e4Vd0kVEJHVkZ8M++8DgCvty154qE5S7V5k33f0O4I6ERCQiIklh61Z49tlQe9prr7rfv0aSEBGRcr3wAnz1VTTNe6AEJSIiFcjOhg4dYODAaPavBCUiIrvZsAFeeglGjoSGDaOJQQlKRER28/TTsH17dM17oAQlIiLlyM6GQw6B7343uhiUoEREpJTPPoMZM0LtySy6OJSgRESklCeeCAPEnnNOtHEoQYmISCnZ2dC7Nxx5ZLRxKEGJiMguy5fD7NnRdo4opgQlIiK75OSEx1Gjoo0DlKBERKSEnBw47jg4+OCoI1GCEhGRmA8+gEWLkqN5D5SgREQkJjs7jBpx9tlRRxIoQYmICO4wdSqcfDLst1/U0QRKUCIiwrvvwooVydO8B0pQIiJCaN5r0gSGD486km8pQYmI1HOFhWH0iNNPh733jjqabylBiYjUczNmwLp1ydW8B3EkKDN71MzWmdnCKsp918x2mtmIxIUnIiK1LTs71JxOOy3qSEqLpwb1ODC4sgJm1hC4DZiegJhERKSOfP11mPtp+HBo1izqaEqrMkG5+0zgiyqKXQo8BaxLRFAiIlI3XnoJNm2KfuTy8tT4GpSZdQSGAw/FUXaCmeWaWW5BQUFNdy0iIjWUnR3uezrxxKgj2V0iOkncA1zt7jurKujuU9w9w90z2rdvn4Bdi4jIntq0CV54IYwc0ahR1NHsLhEhZQBTLUy72A44zcwK3f3ZBGxbRERqyXPPhWtQydi8BwlIUO7etfi5mT0OvKDkJCKS/LKzw6jlxx4bdSTlqzJBmVkOMAhoZ2b5wA1AYwB3r/K6k4iIJJ+CAnj1VfjVryA0gCWfKhOUu8d965a7n1+jaEREpE784x+wc2fyNu+BRpIQEamXcnKge3c46qioI6mYEpSISD2zciXMmhWGNkrW5j1QghIRqXemTg2PyTb2XllKUCIi9UxODvTvD4ccEnUklVOCEhGpR5YsgQULkr/2BEpQIiL1Sk5OuO40cmTUkVRNCUpEpJ5wDwnqhBOgQ4eoo6maEpSISD0xdy4sW5bc9z6VpAQlIlJPZGdD48Zw1llRRxIfJSgRkXpg506YNg2GDIF99ok6mvgoQYmI1ANvvQWrV6dO8x4oQYmI1AvZ2dCiBZxxRtSRxE8JSkQkzW3fDk8+CcOGQfPmUUcTPyUoEZE098or8OWXqdW8B0pQIiJpLzsb9t0XTjkl6kiqRwlKRCSNffVVmNp9xAjYa6+oo6keJSgRkTT2z3/C1q2p17wHcSQoM3vUzNaZ2cIKXs80s7zYzztm1jvxYYqIyJ7IzoaOHWHAgKgjqb54alCPA4Mref1/wEB37wX8HzAlAXGJiEgNffEFvPxyGBi2QQq2lzWqqoC7zzSzLpW8/k6JxXeBTjUPS0REauqpp2DHjtRs3oPEX4MaB7yU4G2KiMgeyMmBww6Dvn2jjmTPJCxBmdkJhAR1dSVlJphZrpnlFhQUJGrXIiJSxurV8O9/h4kJzaKOZs8kJEGZWS/gEWCYu6+vqJy7T3H3DHfPaN++fSJ2LSIi5Zg2Lcz/lAoz51akxgnKzDoDTwPnuvt/ax6SiIjUVE4OHH00HHFE1JHsuSo7SZhZDjAIaGdm+cANQGMAd38I+B3QFvijhXpkobtn1FbAIiJSuWXLYM4cuOOOqCOpmXh68VVaQXT38cD4hEUkIiI1kpMTHkeOjDaOmkrBnvEiIlIR95CgBgyAgw6KOpqaUYISEUkjeXmwZEnq3vtUkhKUiEgayc6GRo3C4LCpTglKRCRNFBXB1KlhWo127aKOpuaUoERE0sR//gMrV6ZH8x4oQYmIpI3sbGjaNEztng6UoERE0kBhIfzjH3DGGdCqVdTRJIYSlIhIGnj9dSgoSJ/mPVCCEhFJC9nZ0Lo1DBkSdSSJowQlIrXiiy/guefCjaNSu7Ztg2eegbPOgiZNoo4mcZSgRCThCgth+HA480y45pqoo0l///oXbN6cXs17oAQlIrXgpptg5kz4/vdh8mS49daoI0pvOTmw//5wwglRR5JYSlAiklCvvQaTJsHYsSFJnXNOqEU98EDUkaWnjRtDDWrkSGjYMOpoEqvK0cxFROK1Zg1kZsKRR8L994cvzMcfhy1b4JJLoGVL+OlPo44yvTzzDHzzTWpPTFgR1aBEJCF27gzJafNmeOIJaNEirG/cOMzuetJJ8LOfwVNPRRtnusnJga5d4XvfizqSxFOCEpGEuOUWmDEjNOX16FH6taZN4dlnw5fo6NEwfXo0MaabtWvD/U+jR0OYLza9KEGJSI298UboGHHuuXD++eWXadkSXnwxJK/hw+Gtt+o0xLT0j3+Emms6Nu+BEpSI1NDataFp7/DD4Y9/rPxMvk2bUHvq3Bl+9CPIza27ONNRTg707Bl+0lGVCcrMHjWzdWa2sILXzczuM7NlZpZnZn0TH6aIJKOdO2HMGNiwIVx3atmy6vfst1/o6bfvvjB4MCxaVPtxpqMVK+Cdd9Lv3qeS4qlBPQ4MruT1IcBhsZ8JwIM1D0tEUsGtt4Zkc//9cNRR8b+vU6dw7WSvvcLcRR9/XHsxpqupU8PjqFHRxlGbqkxQ7j4T+KKSIsOAv3rwLtDGzDokKkARSU5vvgk33BDO4MeNq/77DzkEXn01dJE++WTIz098jOksJweOOSb04EtXibgG1RFYVWI5P7ZuN2Y2wcxyzSy3oKAgAbsWkSisWxcS06GHwkMP7XkPsh49wjWp9etDTWrdusTGma4WLYK8vPRu3oPEJKjy/jTLHR7S3ae4e4a7Z7Rv3z4BuxaRulZUFHrrrV8frjvVdO6hjIwwEsInn8Cpp4brWVK5nBxo0AB+8pOoI6ldiUhQ+cBBJZY7AasTsF0RSUK33QavvAL33gu9eydmmwMGwNNPh5rBaaeFkSekfO4hQZ14Yhh/L50lIkE9D5wX6813DLDR3dckYLsikmRmzYLrrw/jvk2YkNhtDx4cvnhnzw6joH/9dWK3ny7mzIHly9O/eQ/i62aeA/wHONzM8s1snJldYGYXxIq8CCwHlgEPAxfVWrQiEpnPPw89xrp2hSlTamfkgh//GB59NPTwGzUKduxI/D5SXXZ26P04fHjUkdS+KgeLdfdK71F2dwcuTlhEIpJ0iorCIK8FBfDuu7D33rW3r5/+NIznd+mlYUT0v/41XG+RcN/ZtGnhJuc2baKOpvZpNHMRqdLvfx+GKfrDH+Doo2t/f5dcEpLUNdeEThhVjVBRX7z5Jnz2WfoObVSWEpSIVOqdd+C3v4URI+CiOmzA/+1vw1xHt90WktRttylJZWeH0TpOPz3qSOqGEpSIVGj9+nAt6OCD4ZFH6j5B3HorbNoEd9wBrVvDtdfW7f6TyTffhKlKhg+HZs2ijqZuKEGJSLncwzWgzz4LtajWres+BrPQrLhlC1x3XahJXXZZ3ceRDF5+OdwjVl+a90AJSkQqcPfd8M9/hvudMjKii6NBg9Czb8sWuPzykKTGjo0unqjk5EDbtmFYqPpCfWNEZDezZ8PVV4fmpEsvjToaaNQofEGfcgqMHx/mQapPtmyB558PI0c0bhx1NHVHCUpESvnyy3AjbqdO8Oc/J0/HhCZN4Jln4Nhjw/xTL74YdUR157nnYNu2+tW8B0pQIlKCO/zsZ/Dpp2E6h332iTqi0lq0COP2HXVUuKn3zTejjqhu5OSEE4bjjos6krqlBCUiu9x3Hzz7LNx+O3zve1FHU77WrUOHga5dQ3fr996LOqLatX59GPF99Oj6d8NyPfu4IlKROXPg17+GoUPhiiuijqZy7duHuaTatw9j+H3wQdQR1Z4nn4TCwvrXvAdKUCJC6L48ciR06ACPPZY8150q07FjmM23WbPQeeKjj6KOqHbk5MDhh0OfPlFHUveUoETqOffQM27VqnDdad99o44oft26hZrUzp2h+/WqVVW/J5Xk58PMmWHk8lQ4aUg0JSiReu6PfwwjFPy//xd6yKWa7t3DNZoNG0KSSqdZeadNCycQ9bF5D5SgasVnn4WZRjdvjjoSkcrNmwdXXhkmCbzqqqij2XN9+4Zu5/n58MMfhq7y6SAnB/r1g8MOizqSaChBJdCGDWH05UMOCe35hxwShmnZvj3qyER2t2lTuPFzv/3gL39J/R5ixx0XeiAuWZIes/L+978wd279mJiwIin+J5kctm4NIy136xYGtxw6NNxY1717uAu/e/dQVS8qijpSkcAdfv5zWLEiXHdq1y7qiBLjlFPC55kzB4YNS+1ZeXNywnWnkSOjjiQ6SlA1sGMHPPQQHHooTJwY2u/nzw9/WEOHwowZ4abCZs3CiNDf+x688UbUUYvAn/4UmqFvuSX9bv4cPhwefzz8r/3kJ6k5K697+B4ZODD0Vqyv4kpQZjbYzJaa2TIzm1jO653NbIaZzTezPDM7LfGhJo+iojAvy5FHwoUXhprTzJkhGZXsCmoWmhoWLAj/MGvXwkknwZAh8P77kYUv9dyCBeE+p8GD4Te/iTqa2jFmTOj88c9/hhl6d+6MOqLqWbAAli6tv50jdnH3Sn+AhsDHQDdgL+B9oHuZMlOAC2PPuwMrqtpuv379PNUUFbm/8IJ7797u4N6rV1guKorv/du2ud9xh/s++7ibuZ97rvuKFbUbs0hJmza5H3aY+4EHuq9bF3U0te+228L/6s9/Hv//aTL41a/cGzVy//zzqCOpHUCuV5Ej3D2u6Tb6A8vcfTmAmU0FhgGLS+Y5YO/Y89bA6pqlzeQza1aY4XPWrND5ISsrNNtV58Jy06bwq1/BuHEweXKYxmDatDC99TXXhKH066tVq8KF+nXroEuXMIxN8WObNlFHlx7c4YIL4OOPQ/Nz+/ZRR1T7fvOb0Blk0qQwTceddyb//URFReE62uDB9fs7AYirBjUCeKTE8rnAH8qU6QB8AOQDXwL9KtjWBCAXyO3cuXPdpOoaWrDA/bTTwllYhw7uDz7ovn17Yra9cqX72LGhNtW6tfutt7pv3ZqYbaeC7dvdn3rKfciQcAzAvWXL8Fjyp3Vr9z593M880/2KK9zvvdf9uefc8/JCjUDi8/DD4XjeckvUkdStoiL3Sy8Nn/2mm6KOpmpvvhlizcqKOpLaQ5w1KAtlK2ZmZwOnuvv42PK5QH93v7REmSsBc/ffm9mxwJ+Bnu5eYb+1jIwMz83N3YOUWjeWLYPf/S5cqGzTJnSCuPRSaN488ftauDDUzl54IVwQvemm0G7eKE2nk1y6NEzjUFxj6tgxjKA9dmyoNX3xRehd9r//hceyz7duLb29ffctXePq0uXb5wcfHEbAru/y8kInnQEDwkCrqd6lvLqKikLLxeOPw113wS9/GXVE4fTr669Dd/gtW8J9k1u2wD33hO+CdeugZcuoo6wdZjbX3aucBjOer8B84KASy53YvQlvHDAYwN3/Y2ZNgXZAyt3TvXo13Hxz+ALda6/Q9PbrX9duM1PPnuFi7syZoUli/PjwT3TrrXDGGcnfJBGPrVvDaAUPPwxvvQUNG4bPNn48nHpq6WTctm346ddv9+24Q0FB+Qls4cLwj/3NN6Xfs99+5SevLl1CAmvatJY+dJLYsiX0ZmvTBv72t/qXnCB85ocfDsfiyitDc9/48dXbRmHh7smk+LG8dfG8VlHnjczM9E1O1RFPgpoDHGZmXYFPgVFA2VvHVgInAY+b2ZFAU6AgkYHWti++CPcy3X9/+EP8xS/guuvggAPqLoYf/AD+8x94+umQGIcNg+OPD1MfpOIQNBC63T/ySLhmt3Fj6JI/eXKoIe7JsTULCWe//aB//91fLyoKvSXLS2Bz54ZjW7bbcYcO5Sevrl3hoIPCiUqqcoeLLgoDqb72Guy/f9QRRadRo/B3uGULTJgQ/iZatKg4eZRNMNW5p6p585AEW7b89rFdu/B3Vbxc8rWyj/VxYNjyVNnEBxDrNn4PoUffo+4+ycxuJrQjPm9m3YGHgZaEDhO/cfdXKttmsjTxffVV6Kxw++3hYmpmZmhi69Yt2rh27Ai1uBtvDF+4w4eHGtXhh0cbVzw2bgxNow8/HIbSadIERowIZ6wDB0ZbI9y5E9asKZ3ASiaylStLn9WahSbIrl3hu98NY70NGJA6Z7ePPRaaT2+8EW64IepoksPWraH2XnxPYqNGITFUlCwqSyTllW3RIrQQSMXibeKLK0HVhqgT1PbtMGVKuFFx7drwBztpUpipM5ls2QJ33x0S6LZtoR39hhvgwAOjjqw0d3j77VBbeuKJEGuvXmG0gszM5JuZtSKFhWE22bIJ7OOPITc3NB82agTHHBOS1UknhWs7jRtHHfnuFi0KSfXYY+GVV/SlWZJ7aDVp2TKcQEndijdBVdmLorZ+oroPqrDQ/a9/de/aNfSUGTjQ/e23IwmlWtauDT2RGjd2b9bM/dpr3TdsiDqqENedd7ofccS3vfAmTHCfMye17juJx9at7q++6j5xontGxrc9D1u0CD09f/979/ffd9+5M+pI3bdsce/e3X3//d3XrIk6GpHSiLMXX71JUEVFoWtyz57hUx99tPvLL6fel+iyZe6jRoXP0Lat+z33uH/9dd3GUFgYjt2IESFhgvv3v+/+6KPumzfXbSxR+uIL96efdr/oIvfDD/+2W3z79u4jR4Zu3cuXRxNb8e0Lr70Wzf5FKqMEVcK//+1+7LHh0x52mPu0aclxllsTc+a4n3hi+Exdu4Z7Jmr7M61c6X7jje6dO3+bIK+80n3Rotrdb6pYtcr98cfDCCEdOnybsLp1CyMZTJ1aN6M3/OUvYb/XX1/7+xLZE0pQ7j53rvupp4ZP2bGj+5QpibvJNhkUFYWaTPHQS0cf7f7KK4ndR3k3055ySkjydV1zSyVFRe6LF7vff7/7sGHhZuPihNW7d0jsL76Y+Brn4sXuzZuHpuvCwsRuWyRR4k1QadlJ4r//heuvDxfr9903dNm+6KIwqng6Kh689rrr4JNPwpQDkyeHSdz2VGU303btmrjY64vCwtDN/fXXw8+sWaGjTqNGoRPDSSfVvMPF1q3h/WvXhsFGk60jjUixetlJYtUq9/Hj3Rs2DBeur78+OToS1JWvv3a/6y73ffcNZ+rnnFO9ayBffRU6kAwYEN7fsGEYXuhf/9LZeKIVd7i4+urSHS5attzzDhfjx4ftTJ9ee3GLJAL1qYnv88/dr7rKvUkT9732cr/sstC7rL7asMH9t78Nvf0aN3a//PLKr33Mmxcu9Bc3Qx16qPvkyer9VZfWrw9NqRdd5P6d73ipDhejRlXd4SIrK5S/5pq6i1lkT9WLBLV5s/vNN7vvvbd7gwbu55+v6StKys8PZ9UNGri3ahUGCd2yJby2YUMY+LZv3/BX0KSJe2Zm6FCSaj0b01Fxh4sxY8rvcDFt2rcnHR9+GGpeAwa479gRbdwi8Yg3QaXsNailS8Md/QUFYZSFW24JU6vL7pYsCYPRPvdcGNbnBz+A559P3Ztp6xt3+PDDMFTR66+HqTI2bQqv9e4dbubesCFcd+rUKdpYReKR9iNJ7NwZZrMdNy5cGJaqvf02XH11GFR11Kgw9FC/fukxGG19UrLDxWuvhdmZs7LC/EEiqSDtE5SIiKSmeBNUPRx4X0REUoESlIiIJCUlKBERSUpKUCIikpSUoEREJCkpQYmISFJSgkqwrCzo0gUaNAiPWVlRRyQikpriSlBmNtjMlprZMjObWEGZn5jZYjNbZGbZiQ0zNWRlwYQJYURx9/A4YYKSlIjInqjyRl0zawj8FzgFyAfmAKPdfXGJMocBTwAnuvuXZrafu6+rbLvpeKNuly4hKZV18MGwYkVdRyMikpwSeaNuf2CZuy939+3AVGBYmTI/Bx5w9y8BqkpO6WrlyuqtFxGRisWToDoCq0os58fWlfQd4Dtm9raZvWtm5Y4KZmYTzCzXzHILCgr2LOIk1rlz9daLiEjF4klQ5Q0lWrZdsBFwGDAIGA08YmZtdntesVImAAASvUlEQVST+xR3z3D3jPbt21c31qQ3aRI0b156XfPmYb2IiFRPPAkqHzioxHInYHU5ZZ5z9x3u/j9gKSFh1SuZmTBlSrjmZBYep0wJ60VEpHriSVBzgMPMrKuZ7QWMAp4vU+ZZ4AQAM2tHaPJbnshAU0VmZugQUVQUHpWcRET2TJUJyt0LgUuA6cAS4Al3X2RmN5vZ0Fix6cB6M1sMzAB+7e7raytoERFJf5oPSkRE6pTmgxIRkZSmBCUiIklJCUpERJKSEpSIiCQlJSgREUlKSlD1iKYCEZFU0ijqAKRuFE8FsnVrWC6eCgR0M7GIJCfVoOqJa6/9NjkV27o1rBcRSUZKUPWEpgIRkVSjBFVPaCoQEUk1SlD1hKYCEZFUowRVT2gqEBFJNerFV49kZiohiUjqUA1KRESSkhKUiIgkJSUoERFJSkpQIiKSlOJKUGY22MyWmtkyM5tYSbkRZuZmVuVMiSIiIpWpMkGZWUPgAWAI0B0YbWbdyynXCrgMmJ3oIEVEpP6JpwbVH1jm7svdfTswFRhWTrn/A24Hvk5gfCIiUk/Fk6A6AqtKLOfH1u1iZkcDB7n7C5VtyMwmmFmumeUWFBRUO1gREak/4klQVs463/WiWQPgbuCqqjbk7lPcPcPdM9q3bx9/lCIiUu/Ek6DygYNKLHcCVpdYbgX0BP5tZiuAY4Dn1VFCRERqIp4ENQc4zMy6mtlewCjg+eIX3X2ju7dz9y7u3gV4Fxjq7rm1ErGIiNQLVSYody8ELgGmA0uAJ9x9kZndbGZDaztAERGpn+IaLNbdXwReLLPudxWUHVTzsEREpL7TSBKSFLKyoEsXaNAgPGZlRR2RiERNCUoil5UFEybAJ5+Ae3icMEFJKh5K7JLOlKAkctdeC1u3ll63dWtYLxVTYpd0pwQlkVu5snrrJVBil3SnBCWR69y5euslUGKXdKcEJZGbNAmaNy+9rnnzsF4qpsQu6U4JSiKXmQlTpsDBB4NZeJwyJayXiimxS7qL6z4okdqWmamEVF3Fx+vaa0OzXufOITnpOEq6UIISSWFK7JLO1MQnIiJJSQlKRESSUlI18e3YsYP8/Hy+/lqT8qaCpk2b0qlTJxo3bhx1KCKShpIqQeXn59OqVSu6dOmCWXnzJEqycHfWr19Pfn4+Xbt2jTocEUlDSdXE9/XXX9O2bVslpxRgZrRt21a1XRGpNUmVoAAlpxRSX35XGpBVJBpJ1cQnkmyKB2QtHvOueEBWUPdukdqWdDWo6kj0me369evp06cPffr04YADDqBjx467lrdv3x7XNsaOHcvSpUsrLfPAAw+QlaDT8OOPP54FCxYkZFuyOw3IKhKduGpQZjYYuBdoCDzi7pPLvH4lMB4oBAqAn7n7JwmOtZTaOLNt27btri/7G2+8kZYtW/KrX/2qVBl3x91p0KD83P7YY49VuZ+LL754zwKUOqcBWUWiU2UNyswaAg8AQ4DuwGgz616m2Hwgw917AU8Ctyc60LLq8sx22bJl9OzZkwsuuIC+ffuyZs0aJkyYQEZGBj169ODmm2/eVba4RlNYWEibNm2YOHEivXv35thjj2XdunUAXHfdddxzzz27yk+cOJH+/ftz+OGH88477wDw1Vdf8eMf/5jevXszevRoMjIyqqwp/f3vf+eoo46iZ8+eXHPNNQAUFhZy7rnn7lp/3333AXD33XfTvXt3evfuzZgxYxJ+zNKFBmQViU48TXz9gWXuvtzdtwNTgWElC7j7DHcvThfvAp0SG+bu6vrMdvHixYwbN4758+fTsWNHJk+eTG5uLu+//z6vvvoqixcv3u09GzduZODAgbz//vsce+yxPProo+Vu29157733uOOOO3Ylu/vvv58DDjiA999/n4kTJzJ//vxK48vPz+e6665jxowZzJ8/n7fffpsXXniBuXPn8vnnn/PBBx+wcOFCzjvvPABuv/12FixYwPvvv88f/vCHGh6d9KUBWUWiE0+C6gisKrGcH1tXkXHASzUJKh51fWZ7yCGH8N3vfnfXck5ODn379qVv374sWbKk3ATVrFkzhgwZAkC/fv1YsWJFuds+66yzdisza9YsRo0aBUDv3r3p0aNHpfHNnj2bE088kXbt2tG4cWPOOeccZs6cyaGHHsrSpUu5/PLLmT59Oq1btwagR48ejBkzhqysLN1oWwmNtC4SnXgSVHl9ib3cgmZjgAzgjgpen2BmuWaWW1BQEH+U5ajrM9sWLVrsev7RRx9x77338sYbb5CXl8fgwYPLvR9or7322vW8YcOGFBYWlrvtJk2a7FbGvdxDXKGKyrdt25a8vDyOP/547rvvPn7xi18AMH36dC644ALee+89MjIy2LlzZ7X2V59kZsKKFVBUFB6VnETqRjwJKh84qMRyJ2B12UJmdjJwLTDU3b8pb0PuPsXdM9w9o3379nsS7y5Rntlu2rSJVq1asffee7NmzRqmT5+e8H0cf/zxPPHEEwB88MEH5dbQSjrmmGOYMWMG69evp7CwkKlTpzJw4EAKCgpwd84++2xuuukm5s2bx86dO8nPz+fEE0/kjjvuoKCggK1lL+iJiEQsnl58c4DDzKwr8CkwCjinZAEzOxr4EzDY3dclPMoKRDXVQN++fenevTs9e/akW7duHHfccQnfx6WXXsp5551Hr1696Nu3Lz179tzVPFeeTp06cfPNNzNo0CDcnTPOOIMf/ehHzJs3j3HjxuHumBm33XYbhYWFnHPOOWzevJmioiKuvvpqWrVqlfDPICJSExZPU5KZnQbcQ+hm/qi7TzKzm4Fcd3/ezF4DjgLWxN6y0t2HVrbNjIwMz83NLbVuyZIlHHnkkXvwMdJPYWEhhYWFNG3alI8++ogf/vCHfPTRRzRqlFz3Vut3JiLVZWZz3T2jqnJxfdu5+4vAi2XW/a7E85OrHaFUasuWLZx00kkUFhbi7vzpT39KuuQkIlKb9I2XpNq0acPcuXOjDkNEJDIpPdSRiIikLyUoERFJSkpQIpJwyTxFSTLHJqXpGpSIJFQyT1GSzLHJ7lSDKmHQoEG73XR7zz33cNFFF1X6vpYtWwKwevVqRowYUeG2y3arL+uee+4pdcPsaaedxoYNG+IJvVI33ngjd955Z423IxKPZJ6iJJljk90pQZUwevRopk6dWmrd1KlTGT16dFzvP/DAA3nyySf3eP9lE9SLL75ImzZt9nh7IlFI5ilKkjk22V3SNvFdcQUkeh6+Pn0gNstFuUaMGMF1113HN998Q5MmTVixYgWrV6/m+OOPZ8uWLQwbNowvv/ySHTt2cMsttzBsWKlB3VmxYgWnn346CxcuZNu2bYwdO5bFixdz5JFHsm3btl3lLrzwQubMmcO2bdsYMWIEN910E/fddx+rV6/mhBNOoF27dsyYMYMuXbqQm5tLu3btuOuuu3aNhj5+/HiuuOIKVqxYwZAhQzj++ON555136NixI8899xzNmjWr8DMuWLCACy64gK1bt3LIIYfw6KOPss8++3Dffffx0EMP0ahRI7p3787UqVN58803ufzyy4EwvfvMmTM14oRUqXPn0HRW3vqoJXNsyS4rK9Q0V64Mx2vSpNpvFlUNqoS2bdvSv39/Xn75ZSDUnkaOHImZ0bRpU5555hnmzZvHjBkzuOqqqyod0PXBBx+kefPm5OXlce2115a6p2nSpEnk5uaSl5fHm2++SV5eHpdddhkHHnggM2bMYMaMGaW2NXfuXB577DFmz57Nu+++y8MPP7xr+o2PPvqIiy++mEWLFtGmTRueeuqpSj/jeeedx2233UZeXh5HHXUUN910EwCTJ09m/vz55OXl8dBDDwFw55138sADD7BgwQLeeuutShOfSLFknqIkmWNLZsXX7j75BNy/vXZX2x1MkrYGVVlNpzYVN/MNGzaMqVOn7qq1uDvXXHMNM2fOpEGDBnz66aesXbuWAw44oNztzJw5k8suuwyAXr160atXr12vPfHEE0yZMoXCwkLWrFnD4sWLS71e1qxZsxg+fPiuEdXPOuss3nrrLYYOHUrXrl3p06cPUPmUHhDmp9qwYQMDBw4E4Kc//Slnn332rhgzMzM588wzOfPMMwE47rjjuPLKK8nMzOSss86iU6dan+ZL0kDxWXVdn23HI5ljg2hqKfGo7NpdbcanGlQZZ555Jq+//jrz5s1j27Zt9O3bF4CsrCwKCgqYO3cuCxYsYP/99y93io2SzHafqeR///sfd955J6+//jp5eXn86Ec/qnI7ldXUiqfqgMqn9KjKv/71Ly6++GLmzp1Lv379KCwsZOLEiTzyyCNs27aNY445hg8//HCPti31TzJPUZKssUVVS4lHVNfulKDKaNmyJYMGDeJnP/tZqc4RGzduZL/99qNx48bMmDGDT8pryC7hBz/4AVmxv6yFCxeSl5cHhKk6WrRoQevWrVm7di0vvfTt3I6tWrVi8+bN5W7r2WefZevWrXz11Vc888wzDBgwoNqfrXXr1uyzzz689dZbAPztb39j4MCBFBUVsWrVKk444QRuv/12NmzYwJYtW/j444856qijuPrqq8nIyFCCEqlFydzDsK4niC2WtE18URo9ejRnnXVWqR59mZmZnHHGGWRkZNCnTx+OOOKISrdx4YUXMnbsWHr16kWfPn3o378/EGbHPfroo+nRo8duU3VMmDCBIUOG0KFDh1LXofr27cv555+/axvjx4/n6KOPrrQ5ryJ/+ctfdnWS6NatG4899hg7d+5kzJgxbNy4EXfnl7/8JW3atOH6669nxowZNGzYkO7du++aHVhEEi+ZexhOmlT6/jGom2t3cU23URs03UZ60O9MJDG6dCm/h+HBB4emyKgl8vpYvNNtqIlPRCQJJHsPwyiu3SlBiYgkgcxMmDIl1JjMwuOUKcnTiSMKSXcNqnhqckl+UTUPi6SrzMz6nZDKSqoaVNOmTVm/fr2++FKAu7N+/XqaNm0adSgikqbiqkGZ2WDgXqAh8Ii7Ty7zehPgr0A/YD0w0t1XVDeYTp06kZ+fT0FBQXXfKhFo2rSpbt4VkVpTZYIys4bAA8ApQD4wx8yed/fFJYqNA75090PNbBRwGzCyusE0btyYrl27VvdtIiKShuJp4usPLHP35e6+HZgKDCtTZhjwl9jzJ4GTTBeSRESkBuJJUB2BVSWW82Pryi3j7oXARqBt2Q2Z2QQzyzWzXDXjiYhIZeJJUOXVhMr2YoinDO4+xd0z3D2jffv28cQnIiL1VDydJPKBg0osdwJWV1Am38waAa2BLyrb6Ny5cz83s8oHtEtt7YDPow4iBem47Rkdt+rTMdsziThuB8dTKJ4ENQc4zMy6Ap8Co4BzypR5Hvgp8B9gBPCGV9FX3N3TugplZrnxDOUhpem47Rkdt+rTMdszdXncqkxQ7l5oZpcA0wndzB9190VmdjOQ6+7PA38G/mZmywg1p1G1GbSIiKS/uO6DcvcXgRfLrPtdiedfA2cnNjQREanPkmokiTQzJeoAUpSO257Rcas+HbM9U2fHLbLpNkRERCqjGpSIiCQlJSgREUlKSlAJZmYHmdkMM1tiZovM7PKoY0oVZtbQzOab2QtRx5IqzKyNmT1pZh/G/uaOjTqmVGBmv4z9fy40sxwz07D8ZZjZo2a2zswWlli3r5m9amYfxR73qc0YlKASrxC4yt2PBI4BLjaz7hHHlCouB5ZEHUSKuRd42d2PAHqj41clM+sIXAZkuHtPwu0zujVmd48Dg8usmwi87u6HAa/HlmuNElSCufsad58Xe76Z8IVRduxCKcPMOgE/Ah6JOpZUYWZ7Az8g3IeIu2939w3RRpUyGgHNYiPfNGf30XHqPXefye4jApUcGPwvwJm1GYMSVC0ysy7A0cDsaCNJCfcAvwGKog4khXQDCoDHYk2jj5hZi6iDSnbu/ilwJ7ASWANsdPdXoo0qZezv7msgnIwD+9XmzpSgaomZtQSeAq5w901Rx5PMzOx0YJ27z406lhTTCOgLPOjuRwNfUctNLukgdt1kGNAVOBBoYWZjoo1KyqMEVQvMrDEhOWW5+9NRx5MCjgOGmtkKwnxjJ5rZ36MNKSXkA/nuXlxDf5KQsKRyJwP/c/cCd98BPA18P+KYUsVaM+sAEHtcV5s7U4JKsNhEjX8Glrj7XVHHkwrc/bfu3snduxAuVr/h7jqjrYK7fwasMrPDY6tOAhZX8hYJVgLHmFnz2P/rSahzSbyKBwYn9vhcbe4srrH4pFqOA84FPjCzBbF118TGMxRJtEuBLDPbC1gOjI04nqTn7rPN7ElgHqHX7Xw07NFuzCwHGAS0M7N84AZgMvCEmY0jJPpaHYNVQx2JiEhSUhOfiIgkJSUoERFJSkpQIiKSlJSgREQkKSlBiYhIUlKCEhGRpKQEJSIiSen/A4z9DmlUgsePAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "acc = history.history['acc']\n",
    "val_acc = history.history['val_acc']\n",
    "loss = history.history['loss']\n",
    "val_loss = history.history['val_loss']\n",
    "\n",
    "epochs = range(1, len(acc) + 1)\n",
    "\n",
    "plt.plot(epochs, acc, 'bo', label='Training acc')\n",
    "plt.plot(epochs, val_acc, 'b', label='Validation acc')\n",
    "plt.title('Training and validation accuracy')\n",
    "plt.legend()\n",
    "\n",
    "plt.figure()\n",
    "\n",
    "plt.plot(epochs, loss, 'bo', label='Training loss')\n",
    "plt.plot(epochs, val_loss, 'b', label='Validation loss')\n",
    "plt.title('Training and validation loss')\n",
    "plt.legend()\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "이 모델은 과대적합이 빠르게 시작됩니다. 훈련 샘플 수가 작기 때문에 놀라운 일은 아닙니다. 같은 이유로 검증 정확도와 훈련 정확도 사이에 차이가 큽니다. 검증 정확도는 50% 후반을 달성한 것 같습니다.\n",
    "\n",
    "훈련 샘플 수가 적기 때문에 어떤 샘플 200개를 선택했는지에 따라 성능이 크게 좌우됩니다. 여기서는 샘플들을 랜덤하게 선택했습니다. 만약 선택한 샘플에서 성능이 나쁘면 예제를 위해서 랜덤하게 200개의 샘플을 다시 추출하세요(실전에서는 훈련 데이터를 고르지 않습니다).\n",
    "\n",
    "사전 훈련된 단어 임베딩을 사용하지 않거나 임베딩 층을 동결하지 않고 같은 모델을 훈련할 수 있습니다. 이런 경우 해당 작업에 특화된 입력 토큰의 임베딩을 학습할 것입니다. 데이터가 풍부하게 있다면 사전 훈련된 단어 임베딩보다 일반적으로 훨씬 성능이 높습니다. 여기서는 훈련 샘플이 200개뿐이지만 한 번 시도해 보죠:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "embedding_4 (Embedding)      (None, 100, 100)          1000000   \n",
      "_________________________________________________________________\n",
      "flatten_3 (Flatten)          (None, 10000)             0         \n",
      "_________________________________________________________________\n",
      "dense_4 (Dense)              (None, 32)                320032    \n",
      "_________________________________________________________________\n",
      "dense_5 (Dense)              (None, 1)                 33        \n",
      "=================================================================\n",
      "Total params: 1,320,065\n",
      "Trainable params: 1,320,065\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n",
      "Train on 200 samples, validate on 10000 samples\n",
      "Epoch 1/10\n",
      "200/200 [==============================] - 1s 3ms/step - loss: 0.6894 - acc: 0.5200 - val_loss: 0.6953 - val_acc: 0.5117\n",
      "Epoch 2/10\n",
      "200/200 [==============================] - 0s 1ms/step - loss: 0.4895 - acc: 0.9650 - val_loss: 0.7149 - val_acc: 0.5138\n",
      "Epoch 3/10\n",
      "200/200 [==============================] - 0s 1ms/step - loss: 0.2904 - acc: 0.9800 - val_loss: 0.7029 - val_acc: 0.5150\n",
      "Epoch 4/10\n",
      "200/200 [==============================] - 0s 1ms/step - loss: 0.1236 - acc: 1.0000 - val_loss: 0.7255 - val_acc: 0.5194\n",
      "Epoch 5/10\n",
      "200/200 [==============================] - 0s 1ms/step - loss: 0.0553 - acc: 1.0000 - val_loss: 0.7111 - val_acc: 0.5195\n",
      "Epoch 6/10\n",
      "200/200 [==============================] - 0s 1ms/step - loss: 0.0275 - acc: 1.0000 - val_loss: 0.7580 - val_acc: 0.5212\n",
      "Epoch 7/10\n",
      "200/200 [==============================] - 0s 1ms/step - loss: 0.0152 - acc: 1.0000 - val_loss: 0.7334 - val_acc: 0.5197\n",
      "Epoch 8/10\n",
      "200/200 [==============================] - 0s 1ms/step - loss: 0.0085 - acc: 1.0000 - val_loss: 0.7509 - val_acc: 0.5228\n",
      "Epoch 9/10\n",
      "200/200 [==============================] - 0s 1ms/step - loss: 0.0052 - acc: 1.0000 - val_loss: 0.7437 - val_acc: 0.5208\n",
      "Epoch 10/10\n",
      "200/200 [==============================] - 0s 1ms/step - loss: 0.0032 - acc: 1.0000 - val_loss: 0.7570 - val_acc: 0.5243\n"
     ]
    }
   ],
   "source": [
    "from keras.models import Sequential\n",
    "from keras.layers import Embedding, Flatten, Dense\n",
    "\n",
    "model = Sequential()\n",
    "model.add(Embedding(max_words, embedding_dim, input_length=maxlen))\n",
    "model.add(Flatten())\n",
    "model.add(Dense(32, activation='relu'))\n",
    "model.add(Dense(1, activation='sigmoid'))\n",
    "model.summary()\n",
    "\n",
    "model.compile(optimizer='rmsprop',\n",
    "              loss='binary_crossentropy',\n",
    "              metrics=['acc'])\n",
    "history = model.fit(x_train, y_train,\n",
    "                    epochs=10,\n",
    "                    batch_size=32,\n",
    "                    validation_data=(x_val, y_val))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAAEYCAYAAAAJeGK1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3X14VPWd9/H3x4CGAAICVgtCaGvrQ0ggRtBbVKyWRWuxPrRKca+iVba22FbtvTdV75XLXbrd+lBr63ZLrba7plJuXVt1fdhqsdZalaCCgqtQRYxQDYiooGLwe/9xJnEYJskkTpgT8nld11wz55zfOfOdM8l85pzzm3MUEZiZmaXNbqUuwMzMLB8HlJmZpZIDyszMUskBZWZmqeSAMjOzVHJAmZlZKjmgrMsklUl6S9KoYrYtJUmfkFT0315IOk7S6qzhZyUdWUjbLjzX9ZIu7ur8ZmnRp9QF2M4j6a2swQrgXWBbZvjvIqK+M8uLiG3AgGK37Q0i4lPFWI6kc4AzI2Jy1rLPKcayzUrNAdWLRERrQGS+oZ8TEfe11V5Sn4ho3hm1mXXEf4+9j3fxWStJ/yTp15JulvQmcKakwyU9Iul1SeskXSupb6Z9H0khqTIzfFNm+t2S3pT0Z0ljOts2M/14Sc9J2iTpR5L+JGlmG3UXUuPfSVolaaOka7PmLZP0A0kbJP0FmNrO+rlU0oKccddJujrz+BxJz2Rez18yWzdtLatR0uTM4wpJ/5GpbTlwSJ7nfT6z3OWSpmXGjwV+DByZ2X26Pmvdzs2a/6uZ175B0m8k7VvIuunMem6pR9J9kl6T9FdJf5/1PP83s07ekNQg6aP5dqdKeqjlfc6szwczz/MacKmk/SUtyryW9Zn1Nihr/tGZ19iUmf5DSeWZmg/MarevpC2Shrb1ei0FIsK3XngDVgPH5Yz7J2Ar8DmSLy/9gEOBiSRb2x8DngNmZ9r3AQKozAzfBKwH6oC+wK+Bm7rQdm/gTeCkzLQLgfeAmW28lkJq/C0wCKgEXmt57cBsYDkwEhgKPJj8W+R9no8BbwH9s5b9KlCXGf5cpo2ATwNvA9WZaccBq7OW1QhMzjy+EngAGAKMBlbktP0isG/mPflSpoaPZKadAzyQU+dNwNzM4ymZGscB5cC/Ar8vZN10cj0PAl4BvgnsAewJTMhM+w6wFNg/8xrGAXsBn8hd18BDLe9z5rU1A+cBZSR/j58EjgV2z/yd/Am4Muv1PJ1Zn/0z7Y/ITJsPzMt6nouA20r9f+hbB59TpS7AtxK98W0H1O87mO/bwP/LPM4XOv+W1XYa8HQX2p4N/DFrmoB1tBFQBdZ4WNb0/wS+nXn8IMmuzpZpJ+R+aOYs+xHgS5nHxwPPtdP2TuDrmcftBdSa7PcC+Fp22zzLfRr4bOZxRwH1S+C7WdP2JDnuOLKjddPJ9fy3QEMb7f7SUm/O+EIC6vkOajgNWJx5fCTwV6AsT7sjgBcAZYafBE4p9v+Vb8W9eRef5Xope0DSAZL+K7PL5g3gcmBYO/P/NevxFtrvGNFW249m1xHJJ0pjWwspsMaCngt4sZ16AX4FTM88/hLQ2rFE0omSHs3s4nqdZOulvXXVYt/2apA0U9LSzG6q14EDClwuJK+vdXkR8QawERiR1aag96yD9bwfsKqNGvYjCamuyP173EfSQkkvZ2r4RU4NqyPpkLOdiPgTydbYJElVwCjgv7pYk+0kDijLldvF+qck39g/ERF7Av9AskXTndaRfMMHQJLY/gM114epcR3JB1uLjrrB/xo4TtJIkl2Qv8rU2A+4Bfhnkt1vg4H/LrCOv7ZVg6SPAT8h2c01NLPc/8labkdd4teS7DZsWd5Akl2JLxdQV6721vNLwMfbmK+taZszNVVkjdsnp03u6/sXkt6nYzM1zMypYbSksjbq+HfgTJKtvYUR8W4b7SwlHFDWkYHAJmBz5iDz3+2E57wTqJX0OUl9SI5rDO+mGhcC35I0InPA/P+01zgiXiHZDXUj8GxErMxM2oPkuEgTsE3SiSTHSgqt4WJJg5X8Tmx21rQBJB/STSRZfQ7JFlSLV4CR2Z0VctwMfEVStaQ9SAL0jxHR5hZpO9pbz7cDoyTNlrS7pD0lTchMux74J0kfV2KcpL1IgvmvJJ1xyiTNIitM26lhM7BJ0n4kuxlb/BnYAHxXSceTfpKOyJr+HyS7BL9EElaWcg4o68hFwJdJOi38lGQLoltlQuB04GqSD5yPA0+QfHMudo0/Ae4HngIWk2wFdeRXJMeUfpVV8+vABcBtJB0NTiMJ2kJcRrIltxq4m6wPz4hYBlwLPJZpcwDwaNa8vwNWAq9Iyt5V1zL/PSS74m7LzD8KmFFgXbnaXM8RsQn4DHAqSaeM54CjM5OvAH5Dsp7fIOmwUJ7ZdXsucDFJh5lP5Ly2fC4DJpAE5e3ArVk1NAMnAgeSbE2tIXkfWqavJnmft0bEw5187VYCLQcMzVIrs8tmLXBaRPyx1PVYzyXp30k6XswtdS3WMf9Q11JJ0lSSXTbvkHRTbibZijDrkszxvJOAsaWuxQrjXXyWVpOA50l2/UwFPu+D2tZVkv6Z5LdY342INaWuxwrjXXxmZpZK3oIyM7NUKtkxqGHDhkVlZWWpnt7MzEpkyZIl6yOivZ+OACUMqMrKShoaGkr19GZmViKSOjpjC+BdfGZmllIOKDMzSyUHlJmZpZIDyszMUskBZWZmqeSAMjOzVOowoCTdIOlVSU+3MV2SrpW0StIySbXFL9N2dfX1UFkJu+2W3NfXdzTHzpPW2tJaF7i2rnJtOTq65C5wFFBL5nLceaafQHKJAAGHAY8WcinfQw45JMwiIm66KaKiIgI+uFVUJONLLa21pbUu1+baCgE0RAE5UdC5+CRVAndGRFWeaT8FHoiImzPDzwKTI2Jde8usq6sL/1DXIPk29mKen+2NHg2rV+/saraX1trSWhe4tq7qTbVJWhIRdR21K8YxqBEkFwdr0Ugbl+eWNEtSg6SGpqamIjy17QrWtHFu6bbG70xprS2tdbVXg2trn2vbUTECSnnG5d0si4j5EVEXEXXDh3d4GibrJUaN6tz4nSmttaW1rvZqcG3tc207KkZANQL7ZQ2PJLn6qVlB5s2Diortx1VUJONLLa21pbUucG1d5dryKORAFVBJ250kPsv2nSQeK2SZ7iRh2W66KWL06AgpuU/DgeEWaa0trXVFuLau6i21UaxOEpJuBiYDw4BXgMuAvplw+zdJAn5MctXTLcBZEdFh7wd3kjAz650K7STR4eU2ImJ6B9MD+HonajMzM+uQzyTRi6T5R4BmZrlKdsFC27nq62HWLNiyJRl+8cVkGGDGjNLVZWbWFm9B9RKXXPJBOLXYsiUZb2aWRg6oXiLNPwI0M8vHAdVLpPlHgGZm+Tigeok0/wjQzCwfB1SRpbWn3IwZMH9+cnJHKbmfP98dJMwsvdyLr4jS3lNuxox01GFmVghvQRWRe8qZmRWPA6qI3FPOzKx4HFBF5J5yZmbF44AqIveUMzMrHgdUEbmnnJlZ8bgXX5G5p5yZWXF4C8rMzFLJAWVmZqnkgDIzs1RyQJmZWSo5oMzMLJUcUGZmlkoOKDMzSyUHlJmZpZIDyszMUskBZWZmqeSAMjOzVHJAmZlZKjmgzMwslRxQZmaWSg4oMzNLJQeUmZmlkgPKzMxSyQFlZmap5IAyM7NUckCZmVkqFRRQkqZKelbSKklz8kwfLel+ScskPSBpZPFLNTOz3qTDgJJUBlwHHA8cBEyXdFBOsyuBf4+IauBy4J+LXaiZmfUuhWxBTQBWRcTzEbEVWACclNPmIOD+zONFeaabmZl1SiEBNQJ4KWu4MTMu21Lg1Mzjk4GBkobmLkjSLEkNkhqampq6Uq+ZmfUShQSU8oyLnOFvA0dLegI4GngZaN5hpoj5EVEXEXXDhw/vdLFmZtZ79CmgTSOwX9bwSGBtdoOIWAucAiBpAHBqRGwqVpFmZtb7FLIFtRjYX9IYSbsDZwC3ZzeQNExSy7K+A9xQ3DLNzKy36TCgIqIZmA3cCzwDLIyI5ZIulzQt02wy8Kyk54CPAPO6qV4zM+slFJF7OGnnqKuri4aGhpI8t5mZlY6kJRFR11E7n0nCzMxSyQFlZmap5IAyM7NUckCZmVkqOaDMzCyVHFBmZpZKDigzM0slB5SZmaWSA8rMzFLJAWVmZqnkgDIzs1RyQJmZWSo5oMzMLJUcUGZmlkoOKDMzSyUHlJmZpZIDyszMUskBZWZmqeSAMjOzVHJAmZlZKjmgzMwslRxQZmaWSg4oMzNLJQeUmZmlkgPKzMxSyQFlZmap5IAyM7NUckCZmVkqOaDMzCyVHFBmZpZKDigzM0slB5SZmaWSA8rMzFLJAWVmZqnkgDIzs1QqKKAkTZX0rKRVkubkmT5K0iJJT0haJumE4pdqZma9SYcBJakMuA44HjgImC7poJxmlwILI2I8cAbwr8Uu1MzMepdCtqAmAKsi4vmI2AosAE7KaRPAnpnHg4C1xSvRzMx6o0ICagTwUtZwY2ZctrnAmZIagbuA8/MtSNIsSQ2SGpqamrpQrpmZ9RaFBJTyjIuc4enALyJiJHAC8B+Sdlh2RMyPiLqIqBs+fHjnqzUzs16jkIBqBPbLGh7JjrvwvgIsBIiIPwPlwLBiFGhmZr1TnwLaLAb2lzQGeJmkE8SXctqsAY4FfiHpQJKA8j48M+sW7733Ho2NjbzzzjulLsXaUV5ezsiRI+nbt2+X5u8woCKiWdJs4F6gDLghIpZLuhxoiIjbgYuAn0m6gGT338yIyN0NaGZWFI2NjQwcOJDKykqkfEchrNQigg0bNtDY2MiYMWO6tIxCtqCIiLtIOj9kj/uHrMcrgCO6VIGZWSe98847DqeUk8TQoUP5MB3ifCYJM+uRHE7p92HfIweUmVknbdiwgXHjxjFu3Dj22WcfRowY0Tq8devWgpZx1lln8eyzz7bb5rrrrqO+vr4YJfdIBe3iMzPryerr4ZJLYM0aGDUK5s2DGTO6vryhQ4fy5JNPAjB37lwGDBjAt7/97e3aRAQRwW675d8OuPHGGzt8nq9//etdL3IX4C0oM9ul1dfDrFnw4osQkdzPmpWML7ZVq1ZRVVXFV7/6VWpra1m3bh2zZs2irq6Ogw8+mMsvv7y17aRJk3jyySdpbm5m8ODBzJkzh5qaGg4//HBeffVVAC699FKuueaa1vZz5sxhwoQJfOpTn+Lhhx8GYPPmzZx66qnU1NQwffp06urqWsMz22WXXcahhx7aWl9LP7bnnnuOT3/609TU1FBbW8vq1asB+O53v8vYsWOpqanhkksuKf7KKoADysx2aZdcAlu2bD9uy5ZkfHdYsWIFX/nKV3jiiScYMWIE3/ve92hoaGDp0qX87ne/Y8WKFTvMs2nTJo4++miWLl3K4Ycfzg033JB32RHBY489xhVXXNEadj/60Y/YZ599WLp0KXPmzOGJJ57IO+83v/lNFi9ezFNPPcWmTZu45557AJg+fToXXHABS5cu5eGHH2bvvffmjjvu4O677+axxx5j6dKlXHTRRUVaO53jgDKzXdqaNZ0b/2F9/OMf59BDD20dvvnmm6mtraW2tpZnnnkmb0D169eP448/HoBDDjmkdSsm1ymnnLJDm4ceeogzzjgDgJqaGg4++OC8895///1MmDCBmpoa/vCHP7B8+XI2btzI+vXr+dznPgckv1uqqKjgvvvu4+yzz6Zfv34A7LXXXp1fEUXgY1BmtksbNSrZrZdvfHfo379/6+OVK1fywx/+kMcee4zBgwdz5pln5v1x8e677976uKysjObm5rzL3mOPPXZoU8hPTrds2cLs2bN5/PHHGTFiBJdeemlrHfl62kVEKnpJegvKzHZp8+ZBRcX24yoqkvHd7Y033mDgwIHsueeerFu3jnvvvbfozzFp0iQWLlwIwFNPPZV3C+3tt99mt912Y9iwYbz55pvceuutAAwZMoRhw4Zxxx13AMnvy7Zs2cKUKVP4+c9/zttvvw3Aa6+9VvS6C+GAMrNd2owZMH8+jB4NUnI/f/6H68VXqNraWg466CCqqqo499xzOeKI4p/P4Pzzz+fll1+murqaq666iqqqKgYNGrRdm6FDh/LlL3+ZqqoqTj75ZCZOnNg6rb6+nquuuorq6momTZpEU1MTJ554IlOnTqWuro5x48bxgx/8oOh1F0KlOiNRXV1dNDQ0lOS5zaxne+aZZzjwwANLXUYqNDc309zcTHl5OStXrmTKlCmsXLmSPn3ScQQn33slaUlE1HU0bzpegZmZdclbb73FscceS3NzMxHBT3/609SE04e1a7wKM7NeavDgwSxZsqTUZXQLH4MyM7NUckCZmVkqOaDMzCyVHFBmZpZKDigzs06aPHnyDj+6veaaa/ja177W7nwDBgwAYO3atZx22mltLrujn+Bcc801bMk6weAJJ5zA66+/XkjpPYoDysysk6ZPn86CBQu2G7dgwQKmT59e0Pwf/ehHueWWW7r8/LkBdddddzF48OAuLy+tHFBmZp102mmnceedd/Luu+8CsHr1atauXcukSZNaf5dUW1vL2LFj+e1vf7vD/KtXr6aqqgpITkN0xhlnUF1dzemnn956eiGA8847r/VSHZdddhkA1157LWvXruWYY47hmGOOAaCyspL169cDcPXVV1NVVUVVVVXrpTpWr17NgQceyLnnnsvBBx/MlClTtnueFnfccQcTJ05k/PjxHHfccbzyyitA8lurs846i7Fjx1JdXd16qqR77rmH2tpaampqOPbYY4uybrP5d1Bm1qN961uQ5/JHH8q4cZD5bM9r6NChTJgwgXvuuYeTTjqJBQsWcPrppyOJ8vJybrvtNvbcc0/Wr1/PYYcdxrRp09o8+epPfvITKioqWLZsGcuWLaO2trZ12rx589hrr73Ytm0bxx57LMuWLeMb3/gGV199NYsWLWLYsGHbLWvJkiXceOONPProo0QEEydO5Oijj2bIkCGsXLmSm2++mZ/97Gd88Ytf5NZbb+XMM8/cbv5JkybxyCOPIInrr7+e73//+1x11VX84z/+I4MGDeKpp54CYOPGjTQ1NXHuuefy4IMPMmbMmG45X5+3oMzMuiB7N1/27r2I4OKLL6a6uprjjjuOl19+uXVLJJ8HH3ywNSiqq6uprq5unbZw4UJqa2sZP348y5cvz3si2GwPPfQQJ598Mv3792fAgAGccsop/PGPfwRgzJgxjBs3Dmj7kh6NjY38zd/8DWPHjuWKK65g+fLlANx3333bXd13yJAhPPLIIxx11FGMGTMG6J5LcngLysx6tPa2dLrT5z//eS688EIef/xx3n777dYtn/r6epqamliyZAl9+/alsrIy7yU2suXbunrhhRe48sorWbx4MUOGDGHmzJkdLqe9c6u2XKoDkst15NvFd/7553PhhRcybdo0HnjgAebOndu63Nwad8YlObwFZWbWBQMGDGDy5MmcffbZ23WO2LRpE3vvvTd9+/Zl0aJFvJjvYlRZjjrqKOoz159/+umnWbZsGZBcqqN///4MGjSIV155hbvvvrt1noEDB/Lmm2/mXdZvfvMbtmzZwubNm7nttts48sgjC35NmzZtYsSIEQD88pe/bB0/ZcoUfvzjH7cOb9y4kcMPP5w//OEPvPDCC0D3XJLDAWVm1kXTp09n6dKlrVe0BZgxYwYNDQ3U1dVRX1/PAQcc0O4yzjvvPN566y2qq6v5/ve/z4QJE4Dk6rjjx4/n4IMP5uyzz97uUh2zZs3i+OOPb+0k0aK2tpaZM2cyYcIEJk6cyDnnnMP48eMLfj1z587lC1/4AkceeeR2x7cuvfRSNm7cSFVVFTU1NSxatIjhw4czf/58TjnlFGpqajj99NMLfp5C+XIbZtbj+HIbPceHudyGt6DMzCyVHFBmZpZKDigzM0slB5SZ9UilOn5uhfuw75EDysx6nPLycjZs2OCQSrGIYMOGDZSXl3d5Gf6hrpn1OCNHjqSxsZGmpqZSl2LtKC8vZ+TIkV2e3wFlZj1O3759W0+xY7su7+IzM7NUckCZmVkqFRRQkqZKelbSKklz8kz/gaQnM7fnJO16l3Y0M7OdqsNjUJLKgOuAzwCNwGJJt0dE63nfI+KCrPbnA4Wf/MnMzCyPQragJgCrIuL5iNgKLABOaqf9dODmYhRnZma9VyEBNQJ4KWu4MTNuB5JGA2OA37cxfZakBkkN7h5qZmbtKSSg8l2Rqq1fx50B3BIR2/JNjIj5EVEXEXXDhw8vtEYzM+uFCgmoRmC/rOGRwNo22p6Bd++ZmVkRFBJQi4H9JY2RtDtJCN2e20jSp4AhwJ+LW6KZmfVGHQZURDQDs4F7gWeAhRGxXNLlkqZlNZ0OLAifHMvMzIqgoFMdRcRdwF054/4hZ3hu8coyM7PezmeSMDOzVHJAmZlZKjmgzMwslRxQZmaWSg4oMzNLJQeUmZmlkgPKzMxSyQFlZmap5IAyM7NUckCZmVkqOaDMzCyVHFBmZpZKDigzM0slB5SZmaWSA8rMzFLJAWVmZqnkgDIzs1RyQJmZWSo5oMzMLJUcUGZmlkoOKDMzSyUHlJmZpZIDyszMUskBZWZmqeSAMjOzVHJAmZlZKjmgzMwslRxQZmaWSg4oMzNLJQeUmZmlkgPKzMxSyQFlZmap5IAyM7NUckCZmVkqOaDMzCyVCgooSVMlPStplaQ5bbT5oqQVkpZL+lVxyzQzs96mT0cNJJUB1wGfARqBxZJuj4gVWW32B74DHBERGyXt3V0Fm5lZ71DIFtQEYFVEPB8RW4EFwEk5bc4FrouIjQAR8WpxyzQzs96mkIAaAbyUNdyYGZftk8AnJf1J0iOSpuZbkKRZkhokNTQ1NXWtYjMz6xUKCSjlGRc5w32A/YHJwHTgekmDd5gpYn5E1EVE3fDhwztbq5mZ9SKFBFQjsF/W8EhgbZ42v42I9yLiBeBZksAyMzPrkkICajGwv6QxknYHzgBuz2nzG+AYAEnDSHb5PV/MQs3MrHfpMKAiohmYDdwLPAMsjIjlki6XNC3T7F5gg6QVwCLgf0fEhu4q2szMdn2KyD2ctHPU1dVFQ0NDSZ7bzMxKR9KSiKjrqJ3PJGFmZqnkgDIzs1RyQJmZWSo5oMzMLJUcUGZmlkoOKDMzSyUHlJmZpZIDyszMUskBZWZmqeSAMjOzVHJAmZlZKjmgzMwslRxQZmaWSg4oMzNLJQeUmZmlkgPKzMxSyQFlZmap5IAyM7NUckCZmVkqOaDMzCyVHFBmZpZKDigzM0slB5SZmaWSA8rMzFLJAWVmZqnkgDIzsw5FwDvvwNatO+85++y8pyqu+nq45BJYswZGjYJ582DGjFJXZWal9N578PbbO97eeWfHce+/D7vvDn37Jvctt84Ml5WBVOpXDdu2wZYtxblt3tz2tAj4+c/h7LN3zuvqkQFVXw+zZiUrDODFF5NhcEhZ93v//eSDsLk5uW/r1t70DzNv7vRt25IPyj59klvfvvkftzetu+aRCguLQkOlozbbtu3cvwXpwwVcW8PNzZ0Lkq5s1ZSVQf/+UFGx4+0jH8k/vqICDjmk+OuxLYqInfdsWerq6qKhoaFL81ZWJqGUa/RoWL36Q5VlXfD++8k/y5tvJre33tr+vq3H+ca9994Hy83+08z9My10WrGWE/FBKLz/fv710B3KypIPrHy37CDYtu2D+pqbd3ycPdwT7LEHlJdDv37t3wpp01a73XZL1snWrR/ct9xKPdy3b9sBUeitrfBpufXtW7r3V9KSiKjrqF2P3IJas6Zz4217zc2dC42O2m7eXPhz9+sHAwcmtwEDkvvhw2HMmGR4jz22b5+9+yR3V0p7w90xrSUQcgOiowD5MNOLvfsoYvstwEJDravt3n+/86FSXp6Eh1mPDKhRo/JvQY0a1fVlbtu247eYtm7vvttxm5Z/0pZvttn3+cbtzGmFbgFISWi0BElLqIwYsX3A5E5v6/GAAckWgZWOlLwHfh+sJ+iRATVvHpxzTrL/uUVZGey7L5xySmEhk3vrzn3XLR8Iffrkv+/MtD326Np82fe77/5BcLQVKgMGJLsB/E3WzEqlRwbUjBnwl7/A3LnJLouyMthrL9i4MdndlH3AccCA7Yd31q1lF81uu6Wjl4+ZWU/TIztJmJlZz1VoJ4mCduBImirpWUmrJM3JM32mpCZJT2Zu53SlaDMzsxYd7uKTVAZcB3wGaAQWS7o9IlbkNP11RMzuhhrNzKwXKmQLagKwKiKej4itwALgpO4ty8zMertCAmoE8FLWcGNmXK5TJS2TdIuk/fItSNIsSQ2SGpqamrpQrpmZ9RaFBFS+Pmi5PSvuACojohq4D/hlvgVFxPyIqIuIuuHDh3euUjMz61UKCahGIHuLaCSwNrtBRGyIiHczgz8DduLZmszMbFdUSEAtBvaXNEbS7sAZwO3ZDSTtmzU4DXimeCWamVlv1GEvvoholjQbuBcoA26IiOWSLgcaIuJ24BuSpgHNwGvAzG6s2czMegH/UNfMzHaqQn+oW7KAktQE5Dnl6y5jGLC+1EX0QF5vned11jVeb11TjPU2OiI67ClXsoDa1UlqKOQbgm3P663zvM66xuuta3bmevO5qs3MLJUcUGZmlkoOqO4zv9QF9FBeb53nddY1Xm9ds9PWm49BmZlZKnkLyszMUskBZWZmqeSAKjJJ+0laJOkZScslfbPUNfUUksokPSHpzlLX0lNIGpy5gsD/ZP7mDi91TT2BpAsy/59PS7pZUnmpa0obSTdIelXS01nj9pL0O0krM/dDurMGB1TxNQMXRcSBwGHA1yUdVOKaeopv4vM4dtYPgXsi4gCgBq+/DkkaAXwDqIuIKpJTuJ1R2qpS6RfA1Jxxc4D7I2J/4P7McLdxQBVZRKyLiMczj98k+cDId/0syyJpJPBZ4PpS19JTSNoTOAr4OUBEbI2I10tbVY/RB+gnqQ9QQc4VGgwi4kGSc6tmO4kPLqf0S+Dz3VmDA6obSaoExgOPlraSHuEa4O+B90tdSA/yMaAJuDGza/R6Sf1LXVTaRcTLwJXAGmCPq12xAAABbUlEQVQdsCki/ru0VfUYH4mIdZB8GQf27s4nc0B1E0kDgFuBb0XEG6WuJ80knQi8GhFLSl1LD9MHqAV+EhHjgc108y6XXUHmuMlJwBjgo0B/SWeWtirLxwHVDST1JQmn+oj4z1LX0wMcAUyTtBpYAHxa0k2lLalHaAQaI6JlC/0WksCy9h0HvBARTRHxHvCfwP8qcU09xSst1//L3L/anU/mgCoySSI5JvBMRFxd6np6goj4TkSMjIhKkoPVv48If6PtQET8FXhJ0qcyo44FVpSwpJ5iDXCYpIrM/+uxuHNJoW4Hvpx5/GXgt935ZB1esNA67Qjgb4GnJD2ZGXdxRNxVwpps13U+UJ+52vXzwFklrif1IuJRSbcAj5P0un0Cn/ZoB5JuBiYDwyQ1ApcB3wMWSvoKSdB/oVtr8KmOzMwsjbyLz8zMUskBZWZmqeSAMjOzVHJAmZlZKjmgzMwslRxQZmaWSg4oMzNLpf8PufE20mvutPQAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAAEYCAYAAAAJeGK1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3X14VOW57/HvzZuRF0EBRQgQUKoGBMSR6pEWVNoNvgBaVDBatbqpbq1a3ftI1XYrR66t1qOWbo6VenT31ChSrEpblLbK9qWtSEAaJYggAkYQIgXkVQjc549nQibJJJmESWYl+X2ua66ZteaZNfdMYH7zPOuZtczdERERiZpWmS5AREQkGQWUiIhEkgJKREQiSQElIiKRpIASEZFIUkCJiEgkKaAkY8ystZntNLM+6WybSWZ2opml/bcbZjbazNYmLK80s2+k0rYez/Wkmd1V38fXsN37zey/0r1dab7aZLoAaTrMbGfCYnvgK+BAfPn77p5fl+25+wGgY7rbtgTuflI6tmNm1wNXuvuohG1fn45tixwuBZSkzN0PBUT8G/r17v7n6tqbWRt3L22M2kSk+dEQn6RNfAjneTN7zsx2AFea2Vlm9o6ZbTOzjWY2w8zaxtu3MTM3s5z48jPx+18xsx1m9jcz61fXtvH7x5rZR2a23cx+bmZ/MbNrqqk7lRq/b2arzWyrmc1IeGxrM3vUzLaY2cfAmBren3vMbHaldTPN7JH47evNbEX89Xwc791Ut61iMxsVv93ezH4dr205cHqS510T3+5yMxsXX38q8J/AN+LDp18kvLf3Jjz+hvhr32JmL5nZ8am8N7UxswnxeraZ2etmdlLCfXeZ2QYz+9LMPkx4rWea2dL4+k1m9tNUn0+aIHfXRZc6X4C1wOhK6+4H9gEXEb78HAmcAXyd0FvvD3wE3Bxv3wZwICe+/AzwBRAD2gLPA8/Uo+2xwA5gfPy+24H9wDXVvJZUanwZ6AzkAP8oe+3AzcByIBvoCrwZ/lslfZ7+wE6gQ8K2NwOx+PJF8TYGnAvsAQbH7xsNrE3YVjEwKn77YeC/gaOBvkBRpbaXAcfH/yZXxGs4Ln7f9cB/V6rzGeDe+O1vx2scCmQB/wd4PZX3Jsnrvx/4r/jtU+J1nBv/G90Vf9/bAgOBdUCPeNt+QP/47cXA5PjtTsDXM/1/QZeGu6gHJen2trv/zt0Puvsed1/s7ovcvdTd1wCzgJE1PH6uuxe4+34gn/DBWNe2FwLL3P3l+H2PEsIsqRRr/A933+7uawlhUPZclwGPunuxu28BHqjhedYAHxCCE+BbwDZ3L4jf/zt3X+PB68BrQNKJEJVcBtzv7lvdfR2hV5T4vHPcfWP8b/Is4ctFLIXtAuQBT7r7MnffC0wFRppZdkKb6t6bmkwC5rn76/G/0QPAUYQvCqWEMBwYHyb+JP7eQfiiMcDMurr7DndflOLrkCZIASXp9mnigpmdbGZ/MLPPzexLYBrQrYbHf55wezc1T4yorm3PxDrc3Qk9jqRSrDGl5yJ886/Js8Dk+O0rCMFaVseFZrbIzP5hZtsIvZea3qsyx9dUg5ldY2Z/jw+lbQNOTnG7EF7foe25+5fAVqBXQpu6/M2q2+5Bwt+ol7uvBO4g/B02x4eMe8SbXgvkAivN7F0zOz/F1yFNkAJK0q3yFOsnCL2GE939KOAnhCGshrSRMOQGgJkZFT9QKzucGjcCvROWa5sG/zwwOt4DGU8ILMzsSGAu8B+E4bcuwB9TrOPz6mows/7A48CNQNf4dj9M2G5tU+I3EIYNy7bXiTCU+FkKddVlu60If7PPANz9GXc/mzC815rwvuDuK919EmEY938DL5hZ1mHWIhGlgJKG1gnYDuwys1OA7zfCc/4eGGZmF5lZG+BWoHsD1TgHuM3MeplZV+DOmhq7+ybgbeBpYKW7r4rfdQTQDigBDpjZhcB5dajhLjPrYuF3Yjcn3NeREEIlhKy+ntCDKrMJyC6bFJLEc8B1ZjbYzI4gBMVb7l5tj7QONY8zs1Hx5/43wn7DRWZ2ipmdE3++PfHLAcILuMrMusV7XNvjr+3gYdYiEaWAkoZ2B3A14cPnCUIPokHFQ+By4BFgC3AC8B7hd1vprvFxwr6i9wk78Oem8JhnCZMenk2oeRvwQ+BFwkSDiYSgTcW/E3pya4FXgP+XsN1CYAbwbrzNyUDifps/AauATWaWOFRX9vhXCUNtL8Yf34ewX+qwuPtywnv+OCE8xwDj4vujjgAeIuw3/JzQY7sn/tDzgRUWZok+DFzu7vsOtx6JJgvD8yLNl5m1JgwpTXT3tzJdj4ikRj0oaZbMbIyZdY4PE/2YMDPs3QyXJSJ1oICS5moEsIYwTDQGmODu1Q3xiUgEaYhPREQiST0oERGJpIwdLLZbt26ek5OTqacXEZEMWbJkyRfuXtNPP4AMBlROTg4FBQWZenoREckQM6vtiCuAhvhERCSiFFAiIhJJCigREYkkBZSIiESSAkpERCJJASUiIpGkgBIRkUhSQImISCRl7Ie6InJ4Pv0U5syBN96Avn1h8GA49VQYNAg6pnLSdZFa7NgBGzfC55+H640bYfTo8G+sMSigRJqQDRvgN7+B55+Hv/0trBswABYuhJ07y9udcEIIrLLQGjwY+veH1q0zU7dEx8GD8MUX5YGTGD6Vb+/aVfXxM2cqoEQkbvNmmDs3hNJbb4E7DBkC06fDZZfBiSeGD51166CwsPzy/vvw8svhPoD27cMHS1lglYVX166ZfX0Nbdeu8N588gmsXVt+vWsXdOoULkcdVfG6pnVZWWCW6VdV1d69IVRqCpyNG2HTJjhwoOrjjzoKjj8+XM44o/x2jx4Vbx9zTOO9poydbiMWi7mOxSeS3JYt8NvfhlBauDCEzCmnwKRJIZROPjm17ezeDUVF5YFVFl5ffFHepmfP8sAqu5x0ErRr1zCvLd327oX166sGUNn15s0V22dlQU5OCJwdO8Llyy9DDzSVj8PWrVMLslTWtW9fc9i5w/btqfV2tm6t+ngzOPbY8oBJFjhl1+3bp/6eHy4zW+LusVrbKaBEomHbNnjppRBKf/4zlJaG3tHll4fLoEHp+ebuHr5FJ/a2CgthxQrYty+0adMmBGLlYcKePRu/97B/f9jfVl0AbdhQsX3btmGfXL9+IYgqXx93XPLXcPBgCPQvvywPrcQAq8u6HTvKe641adWqamh16hR6d2XBs3dv1cdlZVUfNom3u3cPf8uoUUCJNAE7dsC8eSGUFiwIAZGTE3pJl18Op53WeIGwfz989FHFIcLCwhAOZY45pmJgDR4MAwdChw71f94DB+Czz5KHzyefQHFxxQ/71q2hd++q4VN2+/jjM7+vzT2EXX3C7csvQ2+mpuDp3Dmaw4ypUkCJRNSuXfCHP4RQmj8/fEPu1as8lIYPj9aHz9atIawShwjff798B7pZxUkZZQHWv3/oIRw8GHoC1QXQ+vWht1jGLLwfyXo//fpBdnY0ewWSOgWUSITs3QuvvBJC6Xe/C9+ue/SAiRPDfqWzzgof5k3FwYMhYCpPyli1qnw/TocO4TUWF8NXX1V8fI8e1QdQ795wxBGN+3qkcaUaUPoeItJA9u2DP/4xhNLLL4fhm27d4KqrQk/pm9/M/FBUfbVqFXpI/fvDhAnl63fvhuXLywPr88/hkksqBlDfvnDkkRkrXZoQBZRk1NatYed8UVG4hjC807NnuC67nZWV2TpTtX8/vP56CKUXXwwTH44+OgzfXXYZnHtu8x6eat8+TFE+44xMVyLNQTP+ryJRUlJSHkJFReWXjRvL22RlhW/mu3dXffwxx5QHVrIA69UrzFjKxDDZgQPhaA7PPw8vvBCmiHfqBBdfHHpKo0c3nSnbIlGigJK0cQ9DOokBVHZJ/N1Nx46Qmwv/9E/huuzSt2/YQb59e5jVtWFDuC67lC3//e9hmnTlabxt24YZTsnCK3E5HYcBOngQ/vKXEEpz54Z6OnSAiy4KoTRmTNPp9YlElQIqDcp+V7JqFaxeHa5LSsIv9I89Nvzu4thjyy9R/W1CqtzD1ONkPaJt28rbdekSpiBffHH4TU1ZEGVn1zxLrUuX8sdWp7Q0vOeVw6vsUlQEf/pTmLJb2VFHVR9eZbePO67q38gdFi0KofSb34TnycqCCy4IoXTBBY37Y0eR5q4Jf0w2rrLeQWIIrV5dfkk8DlqbNmFn+JYtYZ9EMmXhlSzAKi936pSZacdlM7Uq94ZWrKj4ert3D8EzeXLFHlF1P4hMhzZtysOkJjt3Vt8T++yzcJSGjRsrTnOGMFR43HHlz3H00aHtunVhuG7MGHjoodBj6tSpYV6jSEuX0jRzMxsD/AxoDTzp7g9Uuv9R4Jz4YnvgWHfvUtM2ozjN3D18WCWGT+LtxAMntmkTZjCdeGI4WGfidd++4f6yw5Rs3hwumzaV3062nOxQJRC+pdcUYInL3bqFoa66KC2Fjz+u2iP68EPYs6e8Xc+eFXtCublhuXv3ur/XUXLwYOjx1jSsuGkTnH566ClNmBB+KCki9ZO230GZWWvgI+BbQDGwGJjs7kXVtP8BcJq7f6+m7WYqoNzDh05i+CSGUOIO+rZtqw+hPn3SP0y3b1/4oEwlzDZtSq13lizQSksr9og++qj8EDcQXltiCJUFUZcav3KIiKQmnb+DGg6sdvc18Q3PBsYDSQMKmAz8e6qFNoSDB2sOocReQdu24VfwJ54I550XrsuCqHfvxt1X1K5dasNWULF3VlOYFRaG5cR9QxCG3vr3D8Fz/vnlQXTyyRqyEpFoSOXjtxeQcDQuioGvJ2toZn2BfsDr1dw/BZgC0KdPnzoVWtmvfw0/+lEYfjnmmPBL/HbtQhB9/HHFEGrXLnwYDxgQpvwm9oZ6926aP5Y0K59M8LWv1d4+sXdmFo5WrR9LikiUpRJQyXZzVzcuOAmY6+5JzjYC7j4LmAVhiC+lCpPIz4d//ufyw6f84x/h2GY9e0IsBt/+dsUQys5umiGUTnXpnYmIREEqAVUM9E5YzgY2VNN2EnDT4RZVm7vvrnpsLwjDdS+/3NDPLiIijSGV390vBgaYWT8za0cIoXmVG5nZScDRwN/SW2JV69fXbb2IiDQ9tQaUu5cCNwMLgBXAHHdfbmbTzGxcQtPJwGxvhMOjV7f76jB3a4mISISkNEfN3ecD8yut+0ml5XvTV1bNpk+HKVMqTglv3z6sFxGR5qEJnYGmXF4ezJpVfuy2vn3Dcl5episTEZF0abKHOsrLUyCJiDRnTbIHJSIizZ8CSkREIkkBJSIikaSAEhGRSFJAiYhIJCmgREQkkhRQIiISSQooERGJJAWUiIhEkgJKREQiSQElIiKRpIASEZFIUkCJiEgkKaBERCSSFFAiIhJJCigREYkkBZSIiERSSgFlZmPMbKWZrTazqdW0uczMisxsuZk9m94yRUSkpan1lO9m1hqYCXwLKAYWm9k8dy9KaDMA+BFwtrtvNbNjG6pgERFpGVLpQQ0HVrv7GnffB8wGxldq88/ATHffCuDum9NbpoiItDSpBFQv4NOE5eL4ukRfA75mZn8xs3fMbEyyDZnZFDMrMLOCkpKS+lUsIiItQioBZUnWeaXlNsAAYBQwGXjSzLpUeZD7LHePuXuse/fuda1VRERakFQCqhjonbCcDWxI0uZld9/v7p8AKwmBJSIiUi+pBNRiYICZ9TOzdsAkYF6lNi8B5wCYWTfCkN+adBYqIiItS60B5e6lwM3AAmAFMMfdl5vZNDMbF2+2ANhiZkXAQuDf3H1LQxUtIiLNn7lX3p3UOGKxmBcUFGTkuUVEJHPMbIm7x2prpyNJiIhIJCmgREQkkhRQIiISSQooERGJJAWUiIhEkgJKREQiSQElIiKRpIASEZFIUkCJiEgkKaBERCSSFFAiIhJJCigREYkkBZSIiESSAkpERCJJASUiIpGkgBIRkUhSQImISCQpoEREJJIUUCIiEkkpBZSZjTGzlWa22symJrn/GjMrMbNl8cv16S9VRERakja1NTCz1sBM4FtAMbDYzOa5e1Glps+7+80NUKOIiLRAqfSghgOr3X2Nu+8DZgPjG7YsERFp6VIJqF7ApwnLxfF1lX3HzArNbK6Z9U5LdU1Qfj7k5ECrVuE6Pz/TFYmINE2pBJQlWeeVln8H5Lj7YODPwK+SbshsipkVmFlBSUlJ3SptAvLzYcoUWLcO3MP1lCkKKRGR+kgloIqBxB5RNrAhsYG7b3H3r+KLvwROT7Yhd5/l7jF3j3Xv3r0+9Uba3XfD7t0V1+3eHdaLiEjdpBJQi4EBZtbPzNoBk4B5iQ3M7PiExXHAivSV2HSsX1+39SIiUr1aZ/G5e6mZ3QwsAFoDT7n7cjObBhS4+zzgFjMbB5QC/wCuacCaI6tPnzCsl2y9iIjUjblX3p3UOGKxmBcUFGTkuRtK2T6oxGG+9u1h1izIy8tcXSIiUWJmS9w9Vls7HUkijfLyQhj17Qtm4VrhJCJSP7UO8Und5OUpkERE0kE9KBERiSQFlIiIRJICSkREIkkBJSIikaSAEhGRSFJAiYhIJCmgREQkkhRQIiISSQooERGJJAWUiIhEkgJKREQiSQElIiKRpIASEZFIUkCJiEgkKaBERCSSFFAiIhJJCigREYmklALKzMaY2UozW21mU2toN9HM3MxqPde8iIhITWoNKDNrDcwExgK5wGQzy03SrhNwC7Ao3UWKiEjLk0oPajiw2t3XuPs+YDYwPkm7/wU8BOxNY30iItJCpRJQvYBPE5aL4+sOMbPTgN7u/vs01iYiIi1YKgFlSdb5oTvNWgGPAnfUuiGzKWZWYGYFJSUlqVcpIiItTioBVQz0TljOBjYkLHcCBgH/bWZrgTOBeckmSrj7LHePuXuse/fu9a9aRESavVQCajEwwMz6mVk7YBIwr+xOd9/u7t3cPcfdc4B3gHHuXtAgFYuISItQa0C5eylwM7AAWAHMcfflZjbNzMY1dIEiItIytUmlkbvPB+ZXWveTatqOOvyyRESkpdORJEREJJIUUCIiEkkKKBERiSQFlIiIRJICSkREIkkBJSIikaSAEhGRSFJAiYhIJCmgREQkkhRQIiISSQooERGJJAWUiIhEkgJKREQiSQElIiKRpIASEZFIUkCJiEgkKaBERCSSFFAiIhJJCigREYkkBZSIiERSSgFlZmPMbKWZrTazqUnuv8HM3jezZWb2tpnlpr9UERFpSWoNKDNrDcwExgK5wOQkAfSsu5/q7kOBh4BH0l6piIi0KKn0oIYDq919jbvvA2YD4xMbuPuXCYsdAE9fiZIu+fmQkwOtWoXr/PxMVyQiUr02KbTpBXyasFwMfL1yIzO7CbgdaAecm2xDZjYFmALQp0+futYqhyE/H6ZMgd27w/K6dWEZIC8vc3WJiFQnlR6UJVlXpYfk7jPd/QTgTuCeZBty91nuHnP3WPfu3etWqRyWu+8uD6cyu3eH9SIiUZRKQBUDvROWs4ENNbSfDUw4nKIk/davr9t6EZFMSyWgFgMDzKyfmbUDJgHzEhuY2YCExQuAVekrUdKhuhFVjbSKSFTVGlDuXgrcDCwAVgBz3H25mU0zs3HxZjeb2XIzW0bYD3V1g1Us9TJ9OrRvX3Fd+/ZhvYhIFKUySQJ3nw/Mr7TuJwm3b01zXZJmZRMh7r47DOv16RPCSRMkRCSqUgooaR7y8hRIItJ06FBHIiISSQooERGJJAWUiIhEkgJKREQiSQElIiKRpIASEZFIUkCJiEgkKaBERCSSFFAiIhJJCigREYkkBZSIiESSAkpERCJJASUiIpGkgBIRkUhSQImISCQpoEREJJIUUCIiEkkKKBERiaSUAsrMxpjZSjNbbWZTk9x/u5kVmVmhmb1mZn3TX6qIiLQktQaUmbUGZgJjgVxgspnlVmr2HhBz98HAXOChdBcqIiItSyo9qOHAandf4+77gNnA+MQG7r7Q3XfHF98BstNbpoiItDSpBFQv4NOE5eL4uupcB7yS7A4zm2JmBWZWUFJSknqVIiLS4qQSUJZknSdtaHYlEAN+mux+d5/l7jF3j3Xv3j31KkVEpMVpk0KbYqB3wnI2sKFyIzMbDdwNjHT3r9JTnoiItFSp9KAWAwPMrJ+ZtQMmAfMSG5jZacATwDh335z+MkVEpKWpNaDcvRS4GVgArADmuPtyM5tmZuPizX4KdAR+Y2bLzGxeNZsTERFJSSpDfLj7fGB+pXU/Sbg9Os11iYhIC6cjSYiISCQpoEREJJIUUCIiEkkKKBERiSQFlIiIRJICSkREIkkBJSIikaSAEhGRSFJAiYhIJCmgREQkkhRQIiISSQooERGJJAWUREJ+PuTkQKtW4To/P9MViUimpXQ0c5GGlJ8PU6bA7t1hed26sAyQl5e5ukQks9SDkoy7++7ycCqze3dYLyItlwJKMm79+rqtF5GWQQElGdenT93Wi0jLoICSjJs+Hdq3r7iuffuwXkRaLgWUZFxeHsyaBX37glm4njVLEyREWrqUAsrMxpjZSjNbbWZTk9z/TTNbamalZjYx/WVKc5eXB2vXwsGD4VrhJCK1BpSZtQZmAmOBXGCymeVWarYeuAZ4Nt0FiohIy5TK76CGA6vdfQ2Amc0GxgNFZQ3cfW38voMNUKOIiLRAqQRUL+DThOVi4Ov1eTIzmwJMAeiTZIrW/v37KS4uZu/evfXZvDSyrKwssrOzadu2baZLEZFmKJWAsiTrvD5P5u6zgFkAsVisyjaKi4vp1KkTOTk5mCV7WokKd2fLli0UFxfTr1+/TJcjIs1QKpMkioHeCcvZwIaGKGbv3r107dpV4dQEmBldu3ZVb1dEGkwqAbUYGGBm/cysHTAJmNdQBSmcmg79rUSkIdUaUO5eCtwMLABWAHPcfbmZTTOzcQBmdoaZFQOXAk+Y2fKGLFpERJq/lH4H5e7z3f1r7n6Cu0+Pr/uJu8+L317s7tnu3sHdu7r7wIYsuky6T9GwZcsWhg4dytChQ+nRowe9evU6tLxv376UtnHttdeycuXKGtvMnDmT/DSdT2LEiBEsW7YsLdsSEYmSJnu6jYY4RUPXrl0Pfdjfe++9dOzYkX/913+t0MbdcXdatUqe7U8//XStz3PTTTfVr0ARkRakyR7qqDFP0bB69WoGDRrEDTfcwLBhw9i4cSNTpkwhFosxcOBApk2bdqhtWY+mtLSULl26MHXqVIYMGcJZZ53F5s2bAbjnnnt47LHHDrWfOnUqw4cP56STTuKvf/0rALt27eI73/kOQ4YMYfLkycRisVp7Ss888wynnnoqgwYN4q677gKgtLSUq6666tD6GTNmAPDoo4+Sm5vLkCFDuPLKK9P+nomIHK4m24Nq7FM0FBUV8fTTT/OLX/wCgAceeIBjjjmG0tJSzjnnHCZOnEhubsUDbGzfvp2RI0fywAMPcPvtt/PUU08xdWqVI0Xh7rz77rvMmzePadOm8eqrr/Lzn/+cHj168MILL/D3v/+dYcOG1VhfcXEx99xzDwUFBXTu3JnRo0fz+9//nu7du/PFF1/w/vvvA7Bt2zYAHnroIdatW0e7du0OrRMRiZIm24Nq7FM0nHDCCZxxxhmHlp977jmGDRvGsGHDWLFiBUVFRVUec+SRRzJ27FgATj/9dNauXZt025dcckmVNm+//TaTJk0CYMiQIQwcWPNuvUWLFnHuuefSrVs32rZtyxVXXMGbb77JiSeeyMqVK7n11ltZsGABnTt3BmDgwIFceeWV5Ofn64e2IhJJTTagGvsUDR06dDh0e9WqVfzsZz/j9ddfp7CwkDFjxiT9PVC7du0O3W7dujWlpaVJt33EEUdUaeNet99CV9e+a9euFBYWMmLECGbMmMH3v/99ABYsWMANN9zAu+++SywW48CBA3V6vpYk3ZNxRCQ1TTagMnmKhi+//JJOnTpx1FFHsXHjRhYsWJD25xgxYgRz5swB4P3330/aQ0t05plnsnDhQrZs2UJpaSmzZ89m5MiRlJSU4O5ceuml3HfffSxdupQDBw5QXFzMueeey09/+lNKSkrYXXmHngDlk3HWrQP38sk4CimRhtdk90FBCKNMnJZh2LBh5ObmMmjQIPr378/ZZ5+d9uf4wQ9+wHe/+10GDx7MsGHDGDRo0KHhuWSys7OZNm0ao0aNwt256KKLuOCCC1i6dCnXXXcd7o6Z8eCDD1JaWsoVV1zBjh07OHjwIHfeeSedOnVK+2toDmqajKNTgog0LKvrUFK6xGIxLygoqLBuxYoVnHLKKRmpJ2pKS0spLS0lKyuLVatW8e1vf5tVq1bRpk20vlM0979Zq1ah51SZWTh3lYjUnZktcfdYbe2i9Wknh+zcuZPzzjuP0tJS3J0nnngicuHUEvTpE4b1kq0XkYalT7yI6tKlC0uWLMl0GS3e9OkVfxAODTsZR0TKNdlJEiKNIZOTcURaOgWUSC3y8mDt2rDPae3a6ISTpr9Lc6chPpEmqCGORSkSNepBiTRBjXksSpFMUUAlGDVqVJUf3T722GP8y7/8S42P69ixIwAbNmxg4sSJ1W678rT6yh577LEKP5g9//zz03KcvHvvvZeHH374sLcj0dHYx6IUyQQFVILJkycze/bsCutmz57N5MmTU3p8z549mTt3br2fv3JAzZ8/ny5dutR7e9J8NfaxKOtK+8ckHSK7D+q22yDd5+EbOhTiZ7lIauLEidxzzz189dVXHHHEEaxdu5YNGzYwYsQIdu7cyfjx49m6dSv79+/n/vvvZ/z48RUev3btWi688EI++OAD9uzZw7XXXktRURGnnHIKe/bsOdTuxhtvZPHixezZs4eJEydy3333MWPGDDZs2MA555xDt27dWLhwITk5ORQUFNCtWzceeeQRnnrqKQCuv/56brvtNtauXcvYsWMZMWIEf/3rX+nVqxcvv/wyRx55ZLWvcdmyZdxwww3s3r2bE044gaeeeoqjjz6aGTNm8Itf/II2bdqQm5vL7NmzeeONN7j11luBcHr3N998U0eciIgoT3/X/jFJF/WgEnTt2pXhw4fz6quvAqH3dPnll2NmZGVl8eKLL7J06VIWLlzIHXcaZYKPAAAIHklEQVTcUeMBXR9//HHat29PYWEhd999d4XfNE2fPp2CggIKCwt54403KCws5JZbbqFnz54sXLiQhQsXVtjWkiVLePrpp1m0aBHvvPMOv/zlL3nvvfeAcODam266ieXLl9OlSxdeeOGFGl/jd7/7XR588EEKCws59dRTue+++4Bw+pD33nuPwsLCQ6cUefjhh5k5cybLli3jrbfeqjH4pHFFefp71PePqXfXdES2B1VTT6chlQ3zjR8/ntmzZx/qtbg7d911F2+++SatWrXis88+Y9OmTfTo0SPpdt58801uueUWAAYPHszgwYMP3TdnzhxmzZpFaWkpGzdupKioqML9lb399ttcfPHFh46ofskll/DWW28xbtw4+vXrx9ChQ4GaT+kB4fxU27ZtY+TIkQBcffXVXHrppYdqzMvLY8KECUyYMAGAs88+m9tvv528vDwuueQSsrOzU3kLpZFk6liUtYny/rGo9+7y80OQr18fhmunT49GXZCZ2lLqQZnZGDNbaWarzazKGffM7Agzez5+/yIzy0l3oY1lwoQJvPbaayxdupQ9e/YcOlFgfn4+JSUlLFmyhGXLlnHcccclPcVGIjOrsu6TTz7h4Ycf5rXXXqOwsJALLrig1u3U1FMrO1UH1HxKj9r84Q9/4KabbmLJkiWcfvrplJaWMnXqVJ588kn27NnDmWeeyYcfflivbUvLEuX9Y1Hu3UX5yPmZqq3WgDKz1sBMYCyQC0w2s9xKza4Dtrr7icCjwIPpLrSxdOzYkVGjRvG9732vwuSI7du3c+yxx9K2bVsWLlzIumQHaEvwzW9+k/z4X++DDz6gsLAQCKfq6NChA507d2bTpk288sorhx7TqVMnduzYkXRbL730Ert372bXrl28+OKLfOMb36jza+vcuTNHH300b731FgC//vWvGTlyJAcPHuTTTz/lnHPO4aGHHmLbtm3s3LmTjz/+mFNPPZU777yTWCymgJKUNPa52uoiyr27KIdnpmpLZYhvOLDa3dcAmNlsYDyQeIKi8cC98dtzgf80M/NMHSr9ME2ePJlLLrmkwoy+vLw8LrroImKxGEOHDuXkk0+ucRs33ngj1157LYMHD2bo0KEMHz4cCGfHPe200xg4cGCVU3VMmTKFsWPHcvzxx1fYDzVs2DCuueaaQ9u4/vrrOe2002oczqvOr371q0OTJPr378/TTz/NgQMHuPLKK9m+fTvuzg9/+EO6dOnCj3/8YxYuXEjr1q3Jzc09dHZgkZqUDftEcagqygf/jXJ4Zqq2Wk+3YWYTgTHufn18+Srg6+5+c0KbD+JtiuPLH8fbfFFpW1OAKQB9+vQ5vXIvpLmfuqE50t9MmpLK+6Ag9O6iMMEkJyd5ePbtGw6xlUnpri3V022ksg+q6o4UqJxqqbTB3We5e8zdY927d0/hqUVE0ifKsx+jPDSaqdpSCahioHfCcjawobo2ZtYG6Az8Ix0FioikU1QP/hvl8MxUbansg1oMDDCzfsBnwCTgikpt5gFXA38DJgKv13f/U9mpySX6muguRpHIiupPByAztdXag3L3UuBmYAGwApjj7svNbJqZjYs3+79AVzNbDdwOVJmKnoqsrCy2bNmiD74mwN3ZsmULWVlZmS5FRJqpWidJNJRYLOaVD566f/9+iouLa/1dkERDVlYW2dnZtG3bNtOliEgTkuokiUgdSaJt27b069cv02WIiEgE6Fh8IiISSQooERGJJAWUiIhEUsYmSZhZCVDzAe2atm7AF7W2ksr0vtWd3rP60ftWP+l43/q6e61Ha8hYQDV3ZlaQyiwVqUjvW93pPasfvW/105jvm4b4REQkkhRQIiISSQqohjMr0wU0UXrf6k7vWf3ofaufRnvftA9KREQiST0oERGJJAWUiIhEkgIqzcyst5ktNLMVZrbczG7NdE1NhZm1NrP3zOz3ma6lqTCzLmY218w+jP+bOyvTNTUFZvbD+P/PD8zsOTPTYfkrMbOnzGxz/IzpZeuOMbM/mdmq+PXRDVmDAir9SoE73P0U4EzgJjPLzXBNTcWthFO6SOp+Brzq7icDQ9D7Vysz6wXcAsTcfRDQmnCeO6nov4AxldZNBV5z9wHAa9Tz1EqpUkClmbtvdPel8ds7CB8YvTJbVfSZWTZwAfBkpmtpKszsKOCbhPOx4e773H1bZqtqMtoAR8bPAN6eqmcJb/Hc/U2qnhl9PPCr+O1fARMasgYFVAMysxzgNGBRZitpEh4D/idwMNOFNCH9gRLg6fjQ6JNm1iHTRUWdu38GPAysBzYC2939j5mtqsk4zt03QvgyDhzbkE+mgGogZtYReAG4zd2/zHQ9UWZmFwKb3X1JpmtpYtoAw4DH3f00YBcNPOTSHMT3m4wH+gE9gQ5mdmVmq5JkFFANwMzaEsIp391/m+l6moCzgXFmthaYDZxrZs9ktqQmoRgodveyHvpcQmBJzUYDn7h7ibvvB34L/I8M19RUbDKz4wHi15sb8skUUGlmZkbYJ7DC3R/JdD1Ngbv/yN2z3T2HsLP6dXfXN9pauPvnwKdmdlJ81XlAUQZLairWA2eaWfv4/9fz0OSSVM0Dro7fvhp4uSGfLFKnfG8mzgauAt43s2XxdXe5+/wM1iTN1w+AfDNrB6wBrs1wPZHn7ovMbC6wlDDr9j102KMqzOw5YBTQzcyKgX8HHgDmmNl1hKC/tEFr0KGOREQkijTEJyIikaSAEhGRSFJAiYhIJCmgREQkkhRQIiISSQooERGJJAWUiIhE0v8H9cdUImPOpG4AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "acc = history.history['acc']\n",
    "val_acc = history.history['val_acc']\n",
    "loss = history.history['loss']\n",
    "val_loss = history.history['val_loss']\n",
    "\n",
    "epochs = range(1, len(acc) + 1)\n",
    "\n",
    "plt.plot(epochs, acc, 'bo', label='Training acc')\n",
    "plt.plot(epochs, val_acc, 'b', label='Validation acc')\n",
    "plt.title('Training and validation accuracy')\n",
    "plt.legend()\n",
    "\n",
    "plt.figure()\n",
    "\n",
    "plt.plot(epochs, loss, 'bo', label='Training loss')\n",
    "plt.plot(epochs, val_loss, 'b', label='Validation loss')\n",
    "plt.title('Training and validation loss')\n",
    "plt.legend()\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "검증 정확도는 50% 초반에 멈추어 있습니다. 이 예제에서는 사전 훈련된 단어 임베딩을 사용하는 것이 임베딩을 함께 훈련하는 것보다 낫습니다. 훈련 샘플의 수를 늘리면 금새 상황이 바뀝니다. 연습삼아 한 번 확인해 보세요."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "훈련 샘플의 수를 2000개로 늘려서 확인해 보겠습니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "training_samples = 2000\n",
    "x_train = data[:training_samples]\n",
    "y_train = labels[:training_samples]\n",
    "x_val = data[training_samples: training_samples + validation_samples]\n",
    "y_val = labels[training_samples: training_samples + validation_samples]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train on 2000 samples, validate on 10000 samples\n",
      "Epoch 1/10\n",
      "2000/2000 [==============================] - 0s 181us/step - loss: 0.6383 - acc: 0.6060 - val_loss: 0.6543 - val_acc: 0.6142\n",
      "Epoch 2/10\n",
      "2000/2000 [==============================] - 0s 177us/step - loss: 0.1578 - acc: 0.9880 - val_loss: 0.6246 - val_acc: 0.6631\n",
      "Epoch 3/10\n",
      "2000/2000 [==============================] - 0s 169us/step - loss: 0.0191 - acc: 0.9995 - val_loss: 0.6568 - val_acc: 0.6787\n",
      "Epoch 4/10\n",
      "2000/2000 [==============================] - 0s 168us/step - loss: 0.0016 - acc: 1.0000 - val_loss: 0.7071 - val_acc: 0.6916\n",
      "Epoch 5/10\n",
      "2000/2000 [==============================] - 0s 167us/step - loss: 1.2005e-04 - acc: 1.0000 - val_loss: 0.7635 - val_acc: 0.6972\n",
      "Epoch 6/10\n",
      "2000/2000 [==============================] - 0s 174us/step - loss: 8.1684e-06 - acc: 1.0000 - val_loss: 0.8398 - val_acc: 0.7043\n",
      "Epoch 7/10\n",
      "2000/2000 [==============================] - 0s 172us/step - loss: 7.4167e-07 - acc: 1.0000 - val_loss: 0.9032 - val_acc: 0.7046\n",
      "Epoch 8/10\n",
      "2000/2000 [==============================] - 0s 171us/step - loss: 1.7394e-07 - acc: 1.0000 - val_loss: 0.9660 - val_acc: 0.7040\n",
      "Epoch 9/10\n",
      "2000/2000 [==============================] - 0s 177us/step - loss: 1.1575e-07 - acc: 1.0000 - val_loss: 0.9998 - val_acc: 0.7039\n",
      "Epoch 10/10\n",
      "2000/2000 [==============================] - 0s 173us/step - loss: 1.1111e-07 - acc: 1.0000 - val_loss: 1.0030 - val_acc: 0.7046\n"
     ]
    }
   ],
   "source": [
    "history = model.fit(x_train, y_train,\n",
    "                    epochs=10,\n",
    "                    batch_size=32,\n",
    "                    validation_data=(x_val, y_val))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAEICAYAAACzliQjAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3XmYVOWd9vHvzaKIIgKNGkEWDYkLAmIH44BLNBI0KomaCGriGhIjJjrJzBjDOzioiZM4Rk18HYlLTCQSXh0TzLiMIkaNG01YFBwEEbWFaIuIIig2/t4/zmm6uuiluimo7j7357rq6rM859SvquGu0885dR5FBGZmlg0dSl2AmZltPw59M7MMceibmWWIQ9/MLEMc+mZmGeLQNzPLEId+BknqKGmdpH7FbFtKkj4tqejXH0v6oqQVOfNLJB1eSNsWPNctki5r6fZmhehU6gKsaZLW5cx2BT4CNqXz346Iac3ZX0RsAnYpdtssiIjPFmM/ks4HzoyIo3L2fX4x9m3WGId+GxARm0M3PZI8PyIeaai9pE4RUb09ajNriv89ti7u3mkHJF0p6Q+S7pL0PnCmpMMkPSPpXUmrJN0gqXPavpOkkDQgnb8zXf+ApPclPS1pYHPbpuuPk/SSpLWSfinpr5LObqDuQmr8tqRlktZIuiFn246SfiFptaSXgTGNvD+TJE3PW3ajpGvT6fMlvZi+npfTo/CG9lUp6ah0uquk36W1LQIOqed5l6f7XSTppHT5QcCvgMPTrrO3c97by3O2/0762ldL+qOkTxXy3jTnfa6pR9Ijkt6R9HdJ/5zzPP8nfU/ek1Qhaa/6utIkPVnze07fz8fT53kHmCRpkKTZ6Wt5O33fuuds3z99jVXp+usldUlr3j+n3ackrZfUq6HXa02ICD/a0ANYAXwxb9mVwEbgRJIP8p2AzwGHkvw1tw/wEjAxbd8JCGBAOn8n8DZQDnQG/gDc2YK2uwPvA2PTdf8IfAyc3cBrKaTGPwHdgQHAOzWvHZgILAL6Ar2Ax5N/zvU+zz7AOmDnnH2/BZSn8yembQQcDWwAhqTrvgisyNlXJXBUOn0N8BjQA+gPLM5r+3XgU+nv5PS0hj3SdecDj+XVeSdweTo9Oq1xGNAF+L/Ao4W8N818n7sDbwLfB3YEdgVGpOt+BCwABqWvYRjQE/h0/nsNPFnze05fWzVwAdCR5N/jZ4BjgB3Sfyd/Ba7JeT0vpO/nzmn7kem6qcBVOc/zA+DeUv8/bMuPkhfgRzN/YQ2H/qNNbPdD4P+l0/UF+X/mtD0JeKEFbc8FnshZJ2AVDYR+gTV+Pmf9fwE/TKcfJ+nmqll3fH4Q5e37GeD0dPo44KVG2v4ZuDCdbiz0X8v9XQDfzW1bz35fAL6cTjcV+ncAP8lZtyvJeZy+Tb03zXyfvwFUNNDu5Zp685YXEvrLm6jhVGBOOn048HegYz3tRgKvAErn5wMnF/v/VZYe7t5pP17PnZG0n6T/Tv9cfw+YApQ1sv3fc6bX0/jJ24ba7pVbRyT/Sysb2kmBNRb0XMCrjdQL8HtgfDp9OrD55LekEyQ9m3ZvvEtylN3Ye1XjU43VIOlsSQvSLop3gf0K3C8kr2/z/iLiPWAN0CenTUG/sybe572BZQ3UsDdJ8LdE/r/HPSXNkPRGWsNv8mpYEclFA3VExF9J/moYJWkw0A/47xbWZLhPvz3Jv1zxZpIjy09HxK7Av5IceW9Lq0iORAGQJOqGVL6tqXEVSVjUaOqS0j8AX5TUl6T76fdpjTsBdwM/Jel62Q34nwLr+HtDNUjaB7iJpIujV7rf/83Zb1OXl64k6TKq2V83km6kNwqoK19j7/PrwL4NbNfQug/SmrrmLNszr03+6/t3kqvODkprODuvhv6SOjZQx2+BM0n+KpkRER810M4K4NBvv7oBa4EP0hNh394Oz/lnYLikEyV1Iukn7r2NapwBXCypT3pS718aaxwRb5J0QdwOLImIpemqHUn6mauATZJOIOl7LrSGyyTtpuR7DBNz1u1CEnxVJJ9/55Mc6dd4E+ibe0I1z13AeZKGSNqR5EPpiYho8C+nRjT2Ps8E+kmaKGkHSbtKGpGuuwW4UtK+SgyT1JPkw+7vJBcMdJQ0gZwPqEZq+ABYK2lvki6mGk8Dq4GfKDk5vpOkkTnrf0fSHXQ6yQeAbQWHfvv1A+AskhOrN5Mc6W5TabCeBlxL8p94X2AeyRFesWu8CZgFPA/MITlab8rvSfrof59T87vAJcC9JCdDTyX58CrEZJK/OFYAD5ATSBGxELgBeC5tsx/wbM62DwNLgTcl5XbT1Gz/IEk3zL3p9v2AMwqsK1+D73NErAWOBU4hOXH8EnBkuvrnwB9J3uf3SE6qdkm77b4FXEZyUv/Tea+tPpOBESQfPjOBe3JqqAZOAPYnOep/jeT3ULN+BcnveWNEPNXM1255ak6OmBVd+uf6SuDUiHii1PVY2yXptyQnhy8vdS1tnb+cZUUlaQzJn+sfklzyV01ytGvWIun5kbHAQaWupT1w944V2yhgOcmf/WOAr/jEm7WUpJ+SfFfgJxHxWqnraQ/cvWNmliE+0jczy5BW16dfVlYWAwYMKHUZZmZtyty5c9+OiMYukQZaYegPGDCAioqKUpdhZtamSGrqW+mAu3fMzDLFoW9mliEOfTOzDHHom5lliEPfzCxDmgx9SbdJekvSCw2sVzos2jJJCyUNz1l3lqSl6eOsYhZuDZs2DQYMgA4dkp/TmjVsuutwHdmoIbN1NDXKCnAEMJx0dKR61h9PcodBAZ8Hnk2X9yT5On5PkvuALwd6NPV8hxxySFjL3XlnRNeuEVD76No1We46XEdrqKM11NAe66CBEdDyHwUNr0UyBmdDoX8zMD5nfgnJiELjgZsbatfQw6G/dfr3r/uPp+bRv7/rcB2to47WUEN7rKPQ0C9Gn34f6g6NVpkua2j5FiRNkFQhqaKqqqoIJWXXaw3ckqqh5a7DdWzvOlpDDVmuoxihX9+wctHI8i0XRkyNiPKIKO/du8lvEVsj+jUwaGBDy12H69jedbSGGrJcRzFCv5K644T2JRk4o6Hltg1ddRV07Vp3WdeuyXLX4TpaQx2toYZM11FIHxCN9+l/mboncp9Ll/cEXiE5idsjne7Z1HO15T79O+9M+uGk5Of2PiHkOlxHW6mjNdTQ3uqgwD79Ju+nL+ku4CigjGQw58lA5/QD4z8lCfgVyYAZ64FzIqIi3fZcknE0Aa6KiNub+hAqLy+PtnjDtWnTYMIEWL++dlnXrjB1KpzR0pFNzcwKJGluRJQ32a6p0N/e2mroDxgAr9Zzj7v+/WHFiu1djZllTaGh72/kFklruRLAzKwxDv0iaS1XApiZNcahXySt5UoAM7PGOPSL5IwzkpO2/fuDlPz0SVwza21a3XCJbdkZZzjkzax185G+mVmGOPTNzDLEoW9mliEOfTOzDHHom5lliEPfzCxDHPpmZhni0DczyxCHvplZhjj0zcwyxKFvZpYhBYW+pDGSlkhaJunSetb3lzRL0kJJj0nqm7Nuk6T56WNmMYs3M7PmafKGa5I6AjcCx5IMdj5H0syIWJzT7BrgtxFxh6SjgZ8C30jXbYiIYUWu28zMWqCQI/0RwLKIWB4RG4HpwNi8NgcAs9Lp2fWsNzOzVqCQ0O8DvJ4zX5kuy7UAOCWd/irQTVKvdL6LpApJz0j6Sn1PIGlC2qaiqqqqGeWbmVlzFBL6qmdZ/mjqPwSOlDQPOBJ4A6hO1/VLB+s9HbhO0r5b7CxiakSUR0R57969C6/ezMyapZBBVCqBvXPm+wIrcxtExErgZABJuwCnRMTanHVExHJJjwEHAy9vdeVmZtZshRzpzwEGSRooaQdgHFDnKhxJZZJq9vUj4LZ0eQ9JO9a0AUYCuSeAzcxsO2oy9COiGpgIPAS8CMyIiEWSpkg6KW12FLBE0kvAHkDNcOD7AxWSFpCc4L0676ofMzPbjhSR3z1fWuXl5VFRUVHqMszM2hRJc9Pzp43yN3LNzDLEoW9mliEOfTOzDHHom5lliEPfzCxDHPpmZhni0DczyxCHvplZhjj0zcwyxKFvZpYhDn0zswxx6JuZZYhD38wsQxz6ZmYZ4tA3M8sQh76ZWYYUFPqSxkhaImmZpEvrWd9f0ixJCyU9JqlvzrqzJC1NH2cVs3gzM2ueJkNfUkfgRuA44ABgvKQD8ppdA/w2IoYAU4Cfptv2BCYDhwIjgMmSehSvfDMza45CjvRHAMsiYnlEbASmA2Pz2hwAzEqnZ+es/xLwcES8ExFrgIeBMVtftpmZtUQhod8HeD1nvjJdlmsBcEo6/VWgm6ReBW6LpAmSKiRVVFVVFVq7mZk1UyGhr3qW5Y+m/kPgSEnzgCOBN4DqArclIqZGRHlElPfu3buAkszMrCU6FdCmEtg7Z74vsDK3QUSsBE4GkLQLcEpErJVUCRyVt+1jW1GvmZlthUKO9OcAgyQNlLQDMA6YmdtAUpmkmn39CLgtnX4IGC2pR3oCd3S6zMzMSqDJ0I+IamAiSVi/CMyIiEWSpkg6KW12FLBE0kvAHsBV6bbvAFeQfHDMAaaky8zMrAQUsUUXe0mVl5dHRUVFqcswM2tTJM2NiPKm2vkbuWZmGeLQNzPLEIe+mVmGOPTNzDLEoW9mliEOfTOzDHHom5lliEPfzCxDHPpmZhni0DczyxCHvplZhjj0zcwyxKFvZpYhDn0zswxx6JuZZYhD38wsQwoKfUljJC2RtEzSpfWs7ydptqR5khZKOj5dPkDSBknz08d/FvsFmJlZ4ZocGF1SR+BG4FiSQdLnSJoZEYtzmk0iGUbxJkkHAPcDA9J1L0fEsOKWbWZmLVHIkf4IYFlELI+IjcB0YGxemwB2Tae7AyuLV6KZmRVLIaHfB3g9Z74yXZbrcuBMSZUkR/kX5awbmHb7/EXS4fU9gaQJkiokVVRVVRVevZmZNUshoa96luWPpj4e+E1E9AWOB34nqQOwCugXEQcD/wj8XtKuedsSEVMjojwiynv37t28V2BmZgUrJPQrgb1z5vuyZffNecAMgIh4GugClEXERxGxOl0+F3gZ+MzWFm1mZi1TSOjPAQZJGihpB2AcMDOvzWvAMQCS9icJ/SpJvdMTwUjaBxgELC9W8WZm1jxNXr0TEdWSJgIPAR2B2yJikaQpQEVEzAR+APxa0iUkXT9nR0RIOgKYIqka2AR8JyLe2WavxszMGqWI/O750iovL4+KiopSl2Fm1qZImhsR5U218zdyzcwyxKFvZpYhDn0zswxx6JuZZYhD38wsQxz6ZmYZ4tA3M8sQh76ZWYY49M3MMsShb2aWIQ59M7MMceibmWWIQ9/MLEMc+mZmGeLQNzPLEIe+mVmGFBT6ksZIWiJpmaRL61nfT9JsSfMkLZR0fM66H6XbLZH0pWIWb2ZmzdPkcInpGLc3AseSDJI+R9LMiFic02wSMCMibpJ0AHA/MCCdHgccCOwFPCLpMxGxqdgvxMzMmlbIkf4IYFlELI+IjcB0YGxemwB2Tae7AyvT6bHA9Ij4KCJeAZal+zMzsxIoJPT7AK/nzFemy3JdDpwpqZLkKP+iZmxrZmbbSSGhr3qW5Y+mPh74TUT0BY4HfiepQ4HbImmCpApJFVVVVQWUZGZmLVFI6FcCe+fM96W2+6bGecAMgIh4GugClBW4LRExNSLKI6K8d+/ehVdvZmbNUkjozwEGSRooaQeSE7Mz89q8BhwDIGl/ktCvStuNk7SjpIHAIOC5YhVvZmbN0+TVOxFRLWki8BDQEbgtIhZJmgJURMRM4AfAryVdQtJ9c3ZEBLBI0gxgMVANXOgrd8zMSkdJNrce5eXlUVFRUeoyzMzaFElzI6K8qXb+Rq6ZWYY49M3MMsShb2aWIQ59M7MMceibmWWIQ9/MLEMc+mZmGeLQNzPLEIe+mVmGOPTNzDLEoW9mliEOfTOzDHHom5lliEPfzCxDHPpmZhni0DczyxCHvplZhhQU+pLGSFoiaZmkS+tZ/wtJ89PHS5LezVm3KWdd/ti6Zma2HTU5Rq6kjsCNwLFAJTBH0syIWFzTJiIuyWl/EXBwzi42RMSw4pVsZmYtVciR/ghgWUQsj4iNwHRgbCPtxwN3FaM4MzMrrkJCvw/wes58ZbpsC5L6AwOBR3MWd5FUIekZSV9pYLsJaZuKqqqqAks3M7PmKiT0Vc+yaKDtOODuiNiUs6xfOkL76cB1kvbdYmcRUyOiPCLKe/fuXUBJZmbWEoWEfiWwd858X2BlA23Hkde1ExEr05/Lgceo299vZmbbUSGhPwcYJGmgpB1Ign2Lq3AkfRboATyds6yHpB3T6TJgJLA4f1szM9s+mrx6JyKqJU0EHgI6ArdFxCJJU4CKiKj5ABgPTI+I3K6f/YGbJX1C8gFzde5VP2Zmtn2pbkaXXnl5eVRUVJS6DDOzNkXS3PT8aaP8jVwzswxx6JuZZYhD38wsQxz6ZmYZ4tA3M8sQh76ZWYY49M3MMsShb2aWIQ59M7MMceibmWWIQ9/MLEMc+mZmGeLQNzPLEIe+mVmGOPTNzDKkyUFUzKw4PvwQVq1KfkZs+fjkk/qXF7q+OfsAkKBjR+jQIXnkTufPN7Zua9p26JDU0Rp98gls2lT7c2umC23bvTscfvi2fV0Fhb6kMcD1JCNn3RIRV+et/wXwhXS2K7B7ROyWrjsLmJSuuzIi7ihG4WatRQSsXg1vvNH4Y/XqUlfaduR+ENRM17esqfWFblPzgZgbwKVw6KHwzDPb9jmaDH1JHYEbgWNJBkmfI2lm7rCHEXFJTvuLSAc/l9QTmAyUAwHMTbddU9RXYbaNfPghrFzZeJivXAkbN9bdToLdd4c+faB/f/iHf0im99oLunZN1uc/ao56G3oUe/0nn9Q+aoKuvvnG1jWnbUProPavj4amW7K+OdtA7V8gHTs2f7ql2+VP77JL4/8ei6GQI/0RwLKIWA4gaTowloYHOB9PEvQAXwIejoh30m0fBsYAd21N0WZba2uOzrt2rQ3wmjDPf3zqU9C58/Z/XWZNKST0+wCv58xXAofW11BSf2Ag8Ggj2/apZ7sJwASAfv36FVCS2ZYi4L334K23oKqq9lEzv2pV3aPzjz6qu31DR+f5j+7dW28/tFlTCgn9+v55NzSa+jjg7ojY1JxtI2IqMBWSgdELqMkyIALWrq0b3PnT+fMff1z/vnbZBfbcMwntww7z0bllVyGhXwnsnTPfF1jZQNtxwIV52x6Vt+1jhZdn7UkEvPtu08FdM//22w2HeLdu0Lt38ujXDw45JJnefffa5bnzXbps39dq1loVEvpzgEGSBgJvkAT76fmNJH0W6AE8nbP4IeAnknqk86OBH21VxdaqRSRdJ/PmJY/582HZstowr66uf7tdd60b4uXlWwZ37sMhbtYyTYZ+RFRLmkgS4B2B2yJikaQpQEVEzEybjgemR9SeE4+IdyRdQfLBATCl5qSutX2ffJIEek3A1zyqqmrbDBoE++0Hn/tcw0fhvXvDjjuW7nWYZYki/7qlEisvL4+KiopSl2F5Nm6ERYvqhvuCBbBuXbK+c2c48EA4+ODax9ChSTeMmW17kuZGRHlT7fyNXNvC++8ngZ4b8IsW1fav77wzDBsGZ59dG/AHHOCjdbO2wKGfcW+9tWX3zLJltV9c6d07CfUvfak24D/96eQLJWbW9jj0MyICVqzYMuBX5lyHNWBAEurf+EZtwO+1l69JN2tPHPrtUERytP7MM3Wvonn33WR9hw6w//5w9NG14T5sGPTo0fh+zaztc+i3Axs3wt/+Bk8+CX/9a/KouYKmSxcYMgROO6024A86CHbaqbQ1m1lpOPTboHfegaefrg35OXOSG4MB7LsvHHccjByZ3EZgv/2gk3/LZpZyHLRyEbB8eRLuNSG/OL3VXadOMHw4XHBBEvIjRya3GjAza4hDv5X5+OOkDz435N98M1nXvXty9H766UnAjxiR3PHRzKxQDv0Se/fdpKumJuSfew42bEjWDRwIxx5bexR/4IG+VNLMto5DfzuquWyy5mTrk08mX3qKSAZQOPhgmDChNuT32qvUFZtZe+PQ34aqq5NLJXNDftWqZF23bklXzde/XttVsz1GzTGzbHPoF9maNXDDDfD44/Dss/DBB8nyfv3gC1+oPYofPDg5ujcz254c+kX05JPJSdY33khuNnbuubUh37dvqaszM3PoF0V1NVx1FUyZkpx8feaZ5FbCZm3Nxx9/TGVlJR/WfPHDWp0uXbrQt29fOrdwmDeH/lZ67TU480x44onknjU33ujbCVvbVVlZSbdu3RgwYADyTZdanYhg9erVVFZWMnDgwBbtwxcAboV77km6cebNg9/9Dn77Wwe+tW0ffvghvXr1cuC3UpLo1avXVv0lVlDoSxojaYmkZZIubaDN1yUtlrRI0u9zlm+SND99zKxv27Zm/Xr4znfg1FOT2wzPm5cc7Zu1Bw781m1rfz9Ndu9I6gjcCBxLMtD5HEkzI2JxTptBJGPfjoyINZJ2z9nFhogYtlVVtiILF8L48cmtEP75n+GKK2CHHUpdlZlZYQo50h8BLIuI5RGxEZgOjM1r8y3gxohYAxARbxW3zNKLSPrrR4yA1avhoYfg3//dgW/ZNm1aMg5Dhw7Jz2nTtm5/q1evZtiwYQwbNow999yTPn36bJ7fuHFjQfs455xzWLJkSaNtbrzxRqZtbbFtVCEncvsAr+fMVwKH5rX5DICkv5IMnn55RDyYrusiqQKoBq6OiD/mP4GkCcAEgH79+jXrBWwPq1cnl1/OnJncwfI3v0kG9TbLsmnTkm+Qr1+fzL/6ajIPcMYZLdtnr169mD9/PgCXX345u+yyCz/84Q/rtIkIIoIODdyT5Pbbb2/yeS688MKWFdgOFHKkX18HUv5o6p2AQcBRwHjgFkm7pev6pYP1ng5cJ2nfLXYWMTUiyiOivHfv3gUXvz089lhysvaBB+AXv4A//9mBbwbw4x/XBn6N9euT5cW2bNkyBg8ezHe+8x2GDx/OqlWrmDBhAuXl5Rx44IFMmTJlc9tRo0Yxf/58qqur2W233bj00ksZOnQohx12GG+9lXRCTJo0ieuuu25z+0svvZQRI0bw2c9+lqeeegqADz74gFNOOYWhQ4cyfvx4ysvLN38g5Zo8eTKf+9znNtcX6VijL730EkcffTRDhw5l+PDhrFixAoCf/OQnHHTQQQwdOpQfb4s3qwmFhH4lsHfOfF9gZT1t/hQRH0fEK8ASkg8BImJl+nM58Bhw8FbWvF18/DFMmpSMLrXzzsm3ay++2Dc8M6vx2mvNW761Fi9ezHnnnce8efPo06cPV199NRUVFSxYsICHH36YxYsXb7HN2rVrOfLII1mwYAGHHXYYt912W737jgiee+45fv7zn2/+APnlL3/JnnvuyYIFC7j00kuZN29evdt+//vfZ86cOTz//POsXbuWBx9MOjnGjx/PJZdcwoIFC3jqqafYfffdue+++3jggQd47rnnWLBgAT/4wQ+K9O4UrpAImwMMkjRQ0g7AOCD/Kpw/Al8AkFRG0t2zXFIPSTvmLB8JbPmbaWVWrIAjj0y+cHXOOTB3bnIzNDOr1VBP7Lbqod133335XM63Hu+66y6GDx/O8OHDefHFF+sN/Z122onjjjsOgEMOOWTz0Xa+k08+eYs2Tz75JOPGjQNg6NChHHjggfVuO2vWLEaMGMHQoUP5y1/+wqJFi1izZg1vv/02J554IpB8oapr16488sgjnHvuueyUDl3Xs2fP5r8RW6nJ0I+IamAi8BDwIjAjIhZJmiLppLTZQ8BqSYuB2cA/RcRqYH+gQtKCdPnVuVf9tEZ/+EPSnbNoEUyfDrfe6huhmdXnqqu2HM+ha9dk+baw8847b55eunQp119/PY8++igLFy5kzJgx9V67vkPOlRYdO3akurq63n3vuOOOW7Sp6aZpzPr165k4cSL33nsvCxcu5Nxzz91cR32XVkZEyS+JLaizIiLuj4jPRMS+EXFVuuxfI2JmOh0R8Y8RcUBEHBQR09PlT6XzQ9Oft267l7J1PvgAzjsPxo2DAw5I7o552mmlrsqs9TrjDJg6Ffr3Byn5OXVqy0/iNsd7771Ht27d2HXXXVm1ahUPPfRQ0Z9j1KhRzJgxA4Dnn3++3r8kNmzYQIcOHSgrK+P999/nnnvuAaBHjx6UlZVx3333AcmX3tavX8/o0aO59dZb2ZAOmvHOO+8Uve6m+DYMJF+uGj8eXnopOQk1eTK08LYWZplyxhnbJ+TzDR8+nAMOOIDBgwezzz77MHLkyKI/x0UXXcQ3v/lNhgwZwvDhwxk8eDDdu3ev06ZXr16cddZZDB48mP79+3PoobUXNk6bNo1vf/vb/PjHP2aHHXbgnnvu4YQTTmDBggWUl5fTuXNnTjzxRK644oqi194YFfInzPZUXl4eFRUV2+W5IuD66+Ff/gXKyuDOO5PbH5tl1Ysvvsj+++9f6jJaherqaqqrq+nSpQtLly5l9OjRLF26lE6dSn+sXN/vSdLc9ErJRpW++hJ5663kJO3998NJJyV992Vlpa7KzFqLdevWccwxx1BdXU1EcPPNN7eKwN9abf8VtMDDD8M3v5kMePKrX8F3v5v0SZqZ1dhtt92YO3duqcsoukxddb5xY9KVM3o09OyZDEJ+4YUOfDPLjswc6b/8cnKyds4c+Pa34dprt7zczMysvctE6E+bltwKuVOn5B746fcwzMwyp11377z/ftJ3f+aZMGwYLFjgwDezbGu3oV9RAcOHJ0f5l18Os2dvu6+Hm1lxHHXUUVt80eq6667ju9/9bqPb7ZJ+bX7lypWceuqpDe67qcvBr7vuOtbn3EXu+OOP59133y2k9Daj3YX+J5/Az38Ohx0GH32U3CVz8uSka8fMWrfx48czffr0OsumT5/O+PHjC9p+r7324u67727x8+eH/v33389uu+3WyBZtT7uKwr//PenOefhhOOUU+PWvoUePUldl1jZdfHFyO5JiGjYM0jsa1+vUU09l0qRJfPTRR+y4446sWLGClStXMmrUKNatW8fYsWNZs2Ywi5VpAAAJDUlEQVQNH3/8MVdeeSVjx9Ydz2nFihWccMIJvPDCC2zYsIFzzjmHxYsXs//++2++9QHABRdcwJw5c9iwYQOnnnoq//Zv/8YNN9zAypUr+cIXvkBZWRmzZ89mwIABVFRUUFZWxrXXXrv5Lp3nn38+F198MStWrOC4445j1KhRPPXUU/Tp04c//elPm2+oVuO+++7jyiuvZOPGjfTq1Ytp06axxx57sG7dOi666CIqKiqQxOTJkznllFN48MEHueyyy9i0aRNlZWXMmjWraL+DdhP6S5fCyJGwbh3cfDN861u+FNOsrenVqxcjRozgwQcfZOzYsUyfPp3TTjsNSXTp0oV7772XXXfdlbfffpvPf/7znHTSSQ3ewOymm26ia9euLFy4kIULFzJ8+PDN66666ip69uzJpk2bOOaYY1i4cCHf+973uPbaa5k9ezZled/UnDt3LrfffjvPPvssEcGhhx7KkUceSY8ePVi6dCl33XUXv/71r/n617/OPffcw5l5g2aPGjWKZ555Bknccsst/OxnP+M//uM/uOKKK+jevTvPP/88AGvWrKGqqopvfetbPP744wwcOLDo9+dpN6G/zz7J0f1FFyU3TDOzrdPYEfm2VNPFUxP6NUfXEcFll13G448/TocOHXjjjTd488032XPPPevdz+OPP873vvc9AIYMGcKQIUM2r5sxYwZTp06lurqaVatWsXjx4jrr8z355JN89atf3Xynz5NPPpknnniCk046iYEDBzJsWDIMeEO3b66srOS0005j1apVbNy4kYEDBwLwyCOP1OnO6tGjB/fddx9HHHHE5jbFvv1yu+nTnz49Gd1q8ODijNVpZqXxla98hVmzZvG3v/2NDRs2bD5CnzZtGlVVVcydO5f58+ezxx571Hs75Vz1/RXwyiuvcM011zBr1iwWLlzIl7/85Sb309g9ympuywwN3775oosuYuLEiTz//PPcfPPNm5+vvlstb+vbL7eL0K8Zq/PVV5ObqNWM1engN2t7dtllF4466ijOPffcOidw165dy+67707nzp2ZPXs2r776aqP7OeKIIzYPfv7CCy+wcOFCILkt884770z37t158803eeCBBzZv061bN95///169/XHP/6R9evX88EHH3Dvvfdy+OGHF/ya1q5dS58+fQC44447Ni8fPXo0v/rVrzbPr1mzhsMOO4y//OUvvPLKK0Dxb7/cLkJ/e47VaWbb3vjx41mwYMHmkasAzjjjDCoqKigvL2fatGnst99+je7jggsuYN26dQwZMoSf/exnjBgxAkhGwTr44IM58MADOffcc+vclnnChAkcd9xxfCHvdrvDhw/n7LPPZsSIERx66KGcf/75HNyM4fQuv/xyvva1r3H44YfXOV8wadIk1qxZw+DBgxk6dCizZ8+md+/eTJ06lZNPPpmhQ4dyWpEH9ijo1sqSxgDXAx2BWyLi6nrafB24nGTQ9AURcXq6/CxgUtrsyoi4I3/bXC25tXKHDskR/pY1JZdwmllhfGvltmGb3lpZUkfgRuBYkgHQ50iamTvsoaRBwI+AkRGxRtLu6fKewGSgnOTDYG667ZqCX10B+vVLunTqW25mZrUK6d4ZASyLiOURsRGYDozNa/Mt4MaaMI+It9LlXwIejoh30nUPA2OKU3qt7T1Wp5lZW1VI6PcBXs+Zr0yX5foM8BlJf5X0TNodVOi2SJogqUJSRVVVVeHVp0o5VqdZe9PaRtOzurb291PIdfr1XTuU/6ydgEHAUUBf4AlJgwvcloiYCkyFpE+/gJq2UKqxOs3aky5durB69Wp69eq1TS8btJaJCFavXk2XLl1avI9CQr8S2Dtnvi+wsp42z0TEx8ArkpaQfAhUknwQ5G77WEuLNbNtq2/fvlRWVtKSv7ht++jSpQt9+/Zt8faFhP4cYJCkgcAbwDjg9Lw2fwTGA7+RVEbS3bMceBn4iaSaO+CMJjnha2atUOfOnTd/E9TapyZDPyKqJU0EHiK5ZPO2iFgkaQpQEREz03WjJS0GNgH/FBGrASRdQfLBATAlIor7TQMzMytYQdfpb08tuU7fzCzrCr1Ov118I9fMzArT6o70JVUBjd9Uo/UrA94udRGtiN+Puvx+1PJ7UdfWvB/9I6J3U41aXei3B5IqCvkzKyv8ftTl96OW34u6tsf74e4dM7MMceibmWWIQ3/bmFrqAloZvx91+f2o5feirm3+frhP38wsQ3ykb2aWIQ59M7MMcegXkaS9Jc2W9KKkRZK+X+qaSk1SR0nzJP251LWUmqTdJN0t6X/TfyOHlbqmUpJ0Sfr/5AVJd0lq+a0j2yBJt0l6S9ILOct6SnpY0tL0Z4/G9tESDv3iqgZ+EBH7A58HLpR0QIlrKrXvAy+WuohW4nrgwYjYDxhKht8XSX2A7wHlETGY5L5e4xrfqt35DVsOKnUpMCsiBgGz0vmicugXUUSsioi/pdPvk/yn3mLQmKyQ1Bf4MnBLqWspNUm7AkcAtwJExMaIeLe0VZVcJ2AnSZ2Armx5y/Z2LSIeB/JvQDkWqBlH/A7gK8V+Xof+NiJpAHAw8GxpKymp64B/Bjw8PewDVAG3p91dt0jaudRFlUpEvAFcA7wGrALWRsT/lLaqVmGPiFgFyUEksHuxn8Chvw1I2gW4B7g4It4rdT2lIOkE4K2ImFvqWlqJTsBw4KaIOBj4gG3wp3tbkfZVjwUGAnsBO0s6s7RVZYNDv8gkdSYJ/GkR8V+lrqeERgInSVoBTAeOlnRnaUsqqUqgMiJq/vK7m+RDIKu+CLwSEVXpiHv/BfxDiWtqDd6U9CmA9OdbxX4Ch34RKRlU9FbgxYi4ttT1lFJE/Cgi+kbEAJITdI9GRGaP5CLi78Drkj6bLjoGWFzCkkrtNeDzkrqm/2+OIcMntnPMBM5Kp88C/lTsJyhkuEQr3EjgG8Dzkuanyy6LiPtLWJO1HhcB0yTtQDKc6DklrqdkIuJZSXcDfyO56m0eGbslg6S7SMYQL5NUCUwGrgZmSDqP5IPxa0V/Xt+GwcwsO9y9Y2aWIQ59M7MMceibmWWIQ9/MLEMc+mZmGeLQNzPLEIe+mVmG/H+gr0aAGfmz5AAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAEICAYAAACktLTqAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3Xt4VNW9//H3l4BE7srFG0Kg9UKAGCBVFJSrFrCC+kMFoahHpVrvtj1StWqptNR60GI9KloplVTKwVooF2lVWmofi1ykKCgFESSCChQQjBcC398fa0ImIZdJSJjJzuf1PHkys2fN3t+ZJJ/ZWXvvtczdERGRaKmX7AJERKT6KdxFRCJI4S4iEkEKdxGRCFK4i4hEkMJdRCSCFO5SKjNLM7O9ZtauOtsmk5l93cyq/dxfMxtoZhvj7q81s3MTaVuFbT1jZndX9fnlrPdBM/tNda9Xkqd+sguQ6mFme+PuNgK+BPbH7n/H3XMrsz533w80qe62dYG7n1Yd6zGz64DR7t43bt3XVce6JfoU7hHh7gfDNbZneJ27v1xWezOr7+4FR6I2ETny1C1TR8T+7f69mT1vZnuA0WZ2tpn908x2mdlWM5tsZg1i7eubmZtZRuz+9NjjC8xsj5m9bmYdKts29vhgM/u3me02s8fM7B9mdnUZdSdS43fMbL2Z7TSzyXHPTTOzR8xsh5m9Bwwq5/2518xmlFj2uJlNit2+zszeib2e92J71WWtK8/M+sZuNzKz52K1rQZ6lLLdDbH1rjazobHlXYFfAefGury2x723D8Q9/4bYa99hZn80sxMSeW8qYmYXx+rZZWavmtlpcY/dbWZbzOxTM3s37rX2NLMVseUfm9kvEt2e1AB311fEvoCNwMASyx4EvgIuInyoHw18AziL8B9cR+DfwM2x9vUBBzJi96cD24EcoAHwe2B6Fdq2AfYAw2KP3QnsA64u47UkUuNsoDmQAfyn8LUDNwOrgbZAS2Bx+JUvdTsdgb1A47h1fwLkxO5fFGtjQH/gcyAr9thAYGPcuvKAvrHbDwN/BY4B2gNrSrS9HDgh9jO5MlbDcbHHrgP+WqLO6cADsdsXxGrMBtKB/wVeTeS9KeX1Pwj8Jna7U6yO/rGf0d2x970B0BnYBBwfa9sB6Bi7vRQYGbvdFDgr2X8LdflLe+51y2vu/id3P+Dun7v7Undf4u4F7r4BmAL0Kef5s9x9mbvvA3IJoVLZtt8CVrr77NhjjxA+CEqVYI0/c/fd7r6REKSF27oceMTd89x9BzCxnO1sAN4mfOgAnA/scvdlscf/5O4bPHgVeAUo9aBpCZcDD7r7TnffRNgbj9/uTHffGvuZ/I7wwZyTwHoBRgHPuPtKd/8CGAf0MbO2cW3Kem/KMwKY4+6vxn5GE4FmhA/ZAsIHSedY1977sfcOwof0KWbW0t33uPuSBF+H1ACFe92yOf6OmZ1uZvPM7CMz+xQYD7Qq5/kfxd3Op/yDqGW1PTG+Dnd3wp5uqRKsMaFtEfY4y/M7YGTs9pWED6XCOr5lZkvM7D9mtouw11zee1XohPJqMLOrzexfse6PXcDpCa4Xwus7uD53/xTYCZwU16YyP7Oy1nuA8DM6yd3XAt8j/Bw+iXXzHR9reg2QCaw1szfMbEiCr0NqgMK9bil5GuBThL3Vr7t7M+A+QrdDTdpK6CYBwMyM4mFU0uHUuBU4Oe5+Radq/h4YGNvzHUYIe8zsaGAW8DNCl0kL4M8J1vFRWTWYWUfgCeBGoGVsve/Grbei0za3ELp6CtfXlND982ECdVVmvfUIP7MPAdx9urv3InTJpBHeF9x9rbuPIHS9/Q/wgpmlH2YtUkUK97qtKbAb+MzMOgHfOQLbnAt0N7OLzKw+cBvQuoZqnAncbmYnmVlL4K7yGrv7x8BrwFRgrbuviz3UEDgK2AbsN7NvAQMqUcPdZtbCwnUAN8c91oQQ4NsIn3PXEfbcC30MtC08gFyK54FrzSzLzBoSQvbv7l7mf0KVqHmomfWNbfsHhOMkS8ysk5n1i23v89jXfsIL+LaZtYrt6e+OvbYDh1mLVJHCvW77HnAV4Q/3KcKea42KBegVwCRgB/A14E3CefnVXeMThL7xtwgH+2Yl8JzfEQ6Q/i6u5l3AHcCLhIOSwwkfUom4n/AfxEZgAfDbuPWuAiYDb8TanA7E91P/BVgHfGxm8d0rhc9/idA98mLs+e0I/fCHxd1XE97zJwgfPIOAobH+94bAQ4TjJB8R/lO4N/bUIcA7Fs7Gehi4wt2/Otx6pGosdHmKJIeZpRG6AYa7+9+TXY9IVGjPXY44MxtkZs1j/9r/iHAGxhtJLkskUhTukgy9gQ2Ef+0HARe7e1ndMiJSBeqWERGJIO25i4hEUNIGDmvVqpVnZGQka/MiIrXS8uXLt7t7eacPA0kM94yMDJYtW5aszYuI1EpmVtGV1oC6ZUREIknhLiISQQp3EZEISqmZmPbt20deXh5ffPFFskuRBKSnp9O2bVsaNChr6BMRSZaUCve8vDyaNm1KRkYGYbBASVXuzo4dO8jLy6NDhw4VP0FEjqiU6pb54osvaNmypYK9FjAzWrZsqf+yRFJUheFuZs+a2Sdm9nYZj5uFeS3Xm9kqM+t+OAUp2GsP/axEUlci3TK/IUwN9tsyHh8MnBL7OoswTOhZ1VGciMiRcOAA7NsXvgoKim6XvF9dj110EXzjGzX7mioMd3dfbLFZ7cswDPhtbLq0f8YmJTjB3bdWU41HzI4dOxgwIMzB8NFHH5GWlkbr1uFCsDfeeIOjjjqqwnVcc801jBs3jtNOO63MNo8//jgtWrRg1KjDHnqb3r1786tf/Yrs7ESmxhSpOz79FN58E1asCF9vvgn/+U/pYXukh9g68cQUCPcEnETxOSLzYssOCXczGwuMBWjXrqIZzyqWmwv33AMffADt2sGECXA4edmyZUtWrlwJwAMPPECTJk34/ve/X6zNwZnF65XeozV16tQKt3PTTTdVvUgROcTOnSG8ly8PQb58OaxbV/T4SSdBt25wzjnQoAHUrx++l3c70XaVfU5aGhyJHs3qCPfSyiz1c9DdpxBmrycnJ+ewPitzc2HsWMjPD/c3bQr34fACvjTr16/n4osvpnfv3ixZsoS5c+fy4x//mBUrVvD5559zxRVXcN999wFFe9JdunShVatW3HDDDSxYsIBGjRoxe/Zs2rRpw7333kurVq24/fbb6d27N7179+bVV19l9+7dTJ06lXPOOYfPPvuMMWPGsH79ejIzM1m3bh3PPPNMuXvo06dP5+c//znuztChQ/npT39KQUEB11xzDStXrsTdGTt2LLfeeiuPPPIITz/9NA0aNKBr165Mnz69et80kRqyfXvR3nhhmG/YUPR4+/bQvTtcdVX43r07HHdc8upNluoI9zyKTwDcljCzTo26556iYC+Unx+WV3e4A6xZs4apU6fy5JNPAjBx4kSOPfZYCgoK6NevH8OHDyczM7PYc3bv3k2fPn2YOHEid955J88++yzjxo07ZN3uzhtvvMGcOXMYP348L730Eo899hjHH388L7zwAv/617/o3r3849R5eXnce++9LFu2jObNmzNw4EDmzp1L69at2b59O2+99RYAu3btAuChhx5i06ZNHHXUUQeXiaSajz8uHuLLl4f/1At17Ag9esD114fv3bpBq1bJqzeVVEe4zwFuNrMZhAOpu49Ef3v8DziR5Yfra1/7Gt+I6yR7/vnn+fWvf01BQQFbtmxhzZo1h4T70UcfzeDBgwHo0aMHf/976bPIXXrppQfbbNy4EYDXXnuNu+4K8zmfccYZdO7cudz6lixZQv/+/WkV+82+8sorWbx4MXfddRdr167ltttuY8iQIVxwwQUAdO7cmdGjRzNs2DAuvvjiSr4bItXLHbZsOTTIt8TtJp56KvTqBbfcEvbGu3WDY45JXs2prsJwN7Pngb5AKzPLI0z42wDA3Z8E5hMmxl0P5APX1FSx8dq1C10xpS2vCY0bNz54e926dfzyl7/kjTfeoEWLFowePbrU873jD8CmpaVRUFBQ6robNmx4SJvKTqJSVvuWLVuyatUqFixYwOTJk3nhhReYMmUKCxcu5G9/+xuzZ8/mwQcf5O233yYtLa1S2xSpCnfYvLl4iK9YEfbSAerVg9NPh/79w9549+6QnQ3NmiW37tomkbNlRlbwuANH/AjhhAnF+9wBGjUKy2vap59+StOmTWnWrBlbt25l4cKFDBo0qFq30bt3b2bOnMm5557LW2+9xZo1a8pt37NnT37wgx+wY8cOmjdvzowZM/j+97/Ptm3bSE9P57LLLqNDhw7ccMMN7N+/n7y8PPr370/v3r3Jzc0lPz+fpk2bVutrEIEQ2n//e1GIr1gR+s0hHFzs3BkGDw4h3qMHnHEGxO1LSRWl1PADlVHYr16dZ8skqnv37mRmZtKlSxc6duxIr169qn0bt9xyC2PGjCErK4vu3bvTpUsXmjdvXmb7tm3bMn78ePr27Yu7c9FFF3HhhReyYsUKrr32WtwdM+PnP/85BQUFXHnllezZs4cDBw5w1113KdilWuXnw+zZ8Nvfwp//HM4jb9AAunSBiy8uCvKuXeHoo5NdbTQlbQ7VnJwcLzlZxzvvvEOnTp2SUk+qKSgooKCggPT0dNatW8cFF1zAunXrqF8/tT6P9TOTQgcOwOLFIdBnzYI9e8JO17e/HQK9a1eI9UDKYTCz5e6eU1G71EoKOWjv3r0MGDCAgoIC3J2nnnoq5YJdBODdd+G552D69PBfdNOmcNllIdTPOy/0ocuRp7RIUS1atGD58uXJLkOkVNu3w4wZIdTfeCME+De/CRMnwrBh4fiXJJfCXUQS8uWXMHduCPR588Il/NnZ8D//AyNHwgknJLtCiadwF5EyucPrr4dA//3vw2X+J5wAt98eul2yspJdoZRF4S4ih9iwIfSh//a38N57oZvlkktgzBgYMCCcwiipTeEuIgDs2gX/938h0F97LQxu1a8f/OhHcOml4UCp1B46jh2nb9++LFy4sNiyRx99lO9+97vlPq9JkyYAbNmyheHDh5e57pKnfpb06KOPkh93VdaQIUOqZdyXBx54gIcffviw1yPRs29f6Ee//HI4/vhwYeCOHfCzn4UrwF95JQzApWCvfRTucUaOHMmMGTOKLZsxYwYjR5Z7ke5BJ554IrNmzary9kuG+/z582nRokWV1ydSGvdwtehtt4WhcC+6CBYtgu98B5YuhdWrYdw4OPnkitclqUvhHmf48OHMnTuXL7/8EoCNGzeyZcsWevfuffC88+7du9O1a1dmz559yPM3btxIly5dAPj8888ZMWIEWVlZXHHFFXz++ecH2914443k5OTQuXNn7r//fgAmT57Mli1b6NevH/369QMgIyOD7bHrtCdNmkSXLl3o0qULjz766MHtderUieuvv57OnTtzwQUXFNtOaVauXEnPnj3JysrikksuYefOnQe3n5mZSVZWFiNGjADgb3/7G9nZ2WRnZ9OtWzf27NlT5fdWkm/z5nCqYufOkJMDTz4JffvCnDlhgK5f/jIs1+yJ0ZCyfe633w6xeTOqTXY2xHKxVC1btuTMM8/kpZdeYtiwYcyYMYMrrrgCMyM9PZ0XX3yRZs2asX37dnr27MnQoUPLnEf0iSeeoFGjRqxatYpVq1YVG7J3woQJHHvssezfv58BAwawatUqbr31ViZNmsSiRYsOjuxYaPny5UydOpUlS5bg7px11ln06dOHY445hnXr1vH888/z9NNPc/nll/PCCy8wevToMl/jmDFjeOyxx+jTpw/33XcfP/7xj3n00UeZOHEi77//Pg0bNjzYFfTwww/z+OOP06tXL/bu3Ut6enol3m1JBXv2wB/+EPrRFy0Ke+29e8NTT4ULjTSqYnRpz72E+K6Z+C4Zd+fuu+8mKyuLgQMH8uGHH/Jx4TB2pVi8ePHBkM3KyiIr7pyxmTNn0r17d7p168bq1asrHBTstdde45JLLqFx48Y0adKESy+99ODwwR06dDg4gUf8kMGl2b17N7t27aJPnz4AXHXVVSxevPhgjaNGjWL69OkHr4Tt1asXd955J5MnT2bXrl26QraW2L8fXn45nKp4/PFw9dWh//z++2H9+jCI19ixCvaoS9m/1vL2sGvSxRdfzJ133nlwlqXCPe7c3Fy2bdvG8uXLadCgARkZGaUO8xuvtL36999/n4cffpilS5dyzDHHcPXVV1e4nvLG/2kYN1hHWlpahd0yZZk3bx6LFy9mzpw5/OQnP2H16tWMGzeOCy+8kPnz59OzZ09efvllTj/99CqtX2reO++EPfTp0yEvD1q0gNGjwwHRs89Wd0tdoz33Epo0aULfvn35r//6r2IHUnfv3k2bNm1o0KABixYtYlNpg8nHOe+888jNzQXg7bffZtWqVUAYLrhx48Y0b96cjz/+mAULFhx8TtOmTUvt1z7vvPP44x//SH5+Pp999hkvvvgi5557bqVfW/PmzTnmmGMO7vU/99xz9OnThwMHDrB582b69evHQw89xK5du9i7dy/vvfceXbt25a677iInJ4d333230tuUmrVjBzz+OJx5JmRmwi9+EYbMnTkTtm4N3S/nnKNgr4tSds89mUaOHMmll15a7MyZUaNGcdFFF5GTk0N2dnaFe7A33ngj11xzDVlZWWRnZ3PmmWcCYValbt260blz50OGCx47diyDBw/mhBNOYNGiRQeXd+/enauvvvrgOq677jq6detWbhdMWaZNm8YNN9xAfn4+HTt2ZOrUqezfv5/Ro0eze/du3J077riDFi1a8KMf/YhFixaRlpZGZmbmwVmlJLm++goWLIBp08JpjPv2hUCfNCkMA3D88cmuUFKBhvyVw6Kf2ZHhHia5mDYNnn8+DNx13HFh/oIxY0K4S92gIX9FImDLltCHPm0arFkTxkMfNiwE+je/CTrGLWXRr4ZIisnPhz/+MQT6yy+HSTDOOSecl3755TrLRRKTcuFeOB2cpL5kdelF0YEDYTyXadPC+C579kD79nD33WEv/ZRTkl2h1DYpFe7p6ens2LGDli1bKuBTnLuzY8cOXdh0mN57L5y++Nxz8P770KQJDB8eTl/ULEZyOFIq3Nu2bUteXh7btm1LdimSgPT0dNq2bZvsMmqd3bvDqYrxoy8OGADjx4dhdRs3TnaFEgUpFe4NGjSgQ4cOyS5DpNoVFMBf/hK6XWbPhi++gNNPD6Mvjh4N+oyU6pZS4S4SNW+9FQI9Nxc++giOPRauvTZ0u2iQLqlJCneRarZzZwj0adPC4Hf168OFF4ZAHzIknM4oUtMU7iLVZP36MGzus8+G0xl79IDJk2HECGjdOtnVSV2jcBc5DO5hlMVJk8K46PXrh6tGb79dV41KcincRapg375wPvqkSWFWo5Yt4Z574LvfhRNOSHZ1Igp3kUrZuROmTIHHHoMPP4TTTgtXjn7729CoUbKrEymicBdJQMn+9AEDQsgPGqQLjSQ1JfRraWaDzGytma03s3GlPN7OzBaZ2ZtmtsrMhlR/qSJHVmF/+iWXwKmnFk1Nt3JlGPNlyBAFu6SuCvfczSwNeBw4H8gDlprZHHePnxvuXmCmuz9hZpnAfCCjBuoVqXHqT5coSKRb5kxgvbtvADCzGcAwID7cHWgWu90c2FKdRYocCTt3wtNPh9MX1Z8utV0i4X4SsDnufh5wVok2DwB/NrNbgMbAwNJWZGZjgbEA7dq1q2ytIjWisD996lT47DPo3z90wQwerG4Xqb0S+dUt7QLpkmO9jgR+4+5tgSHAc2Z2yLrdfYq757h7Tmtd1SFJVFp/+vDhoT/9lVfCFaUKdqnNEtlzzwNOjrvflkO7Xa4FBgG4++tmlg60Aj6pjiJFqsu+fTBrVuhPX7YsjPVy991w003qT5doSWTfZClwipl1MLOjgBHAnBJtPgAGAJhZJyAd0Li9kjJ27oSHHoKOHeHKK8NkGE88AZs3w4MPKtgleircc3f3AjO7GVgIpAHPuvtqMxsPLHP3OcD3gKfN7A5Cl83Vrml6JAW8917R+emF/elPPqn+dIm+hC5icvf5hNMb45fdF3d7DdCreksTqRr3MAnGI4+EuUjr14eRI+GOOyA7O9nViRwZukJVIqNwYulHHlF/uojCXWq1vXth/vxwkHTevBDwp54a+tPHjNH56VJ3Kdyl1vn0U5g7NwT6ggVhyrrjjguTYQwfDn37qj9dROEutcKuXfCnP4VAX7gQvvwSTjwRrr8+BHqvXpCWluwqRVKHwl1S1n/+EyaTnjUrTC69b1+YSPrGG0Ogn3229tBFyqJwl5SyfXs4KDprVrhStKAA2reHW28NIzJ+4xsKdJFEKNwl6T7+GF58MQT6X/8K+/eHi42+972wh96jB1hpg2CISJkU7pIUW7fCH/4QAn3xYjhwAE45Be66KwR6drYCXeRwKNzliMnLgxdeCIH+j3+Ei406dYJ77w2B3qWLAl2kuijcpUZt2lQU6K+/HpZ17QoPPBACPTMzqeWJRJbCXardhg0hzGfNgqVLw7LsbJgwAf7f/wuTYIhIzVK4S7X497+L9tBXrAjLcnJg4sQQ6F//enLrE6lrFO5SJV99FQ6EzpsXvtatC8t79oSHHw6BnpGR1BJF6jSFuyRs69Ywjsu8eeGior17oWFD6NcvnIc+dCho9kSR1KBwlzIdOBD6zAv3zgu7W9q2hVGjwlR0/ftD48bJrVNEDqVwl2J27Qpjt8ybBy+9BNu2hStCzz4bfvrTEOhdu+qURZFUp3Cv49xhzZqivfN//CNcIXrssTBoUAjzb34TWrZMdqUiUhkK9zro889h0aKiQN+0KSzPyoL//u8Q6D17apRFkdpM4V5HfPBBUZi/+moI+EaNYOBA+OEPYcgQOPnkZFcpItVF4R5RBQXhitDCQH/77bC8Y0e47rqwd96nD6SnJ7dOEakZCvcI2b49HASdNy8cFN25M0wOfe654dzzCy8MV4fqYKhI9Cnca7H8fHjrLXj55RDo//xnOEDapg0MGxbC/PzzoXnzZFcqIkdarQv33/0O/vd/wwiCXbqE0/K6dIn+2RyffAIrVxb/Wrs2nIsO4VL/++4Lgd6jhya0EKnralW45+bC7beH7ofXXy8KNoDjjy8K+sLQz8ysfRfYHDgA69cfGuRbtxa1adcuDMR12WXh+9lnh9cvIlKo1oR7bi6MHRu6IiCEYHo63HYbtG4dDhi+/TY8+WQ4EwRC33KHDoeG/qmnQoMGyXsthfLzQ83xIb5qFXz2WXi8fv3wAXX++SHEs7PhjDPCOegiIuUxd0/KhnNycnzZsmUJt8/IKDofO1779rBxY9H9/fvh/fdDaL71VlHor10bHoMQ7Keddmjot29fc90ZFXWrNGtWFOCFX5mZYewWEZFCZrbc3XMqbFdbwr1evXCwsCSz4t0zZfnyyxCmJUM//oOhcWPo3PnQ0G/TJvEzTCrTrRL/lZGhs1hEpGKJhnut6ZZp1670PfdERyFs2DBcgZmVVXz5p5+Gy+/jQ/9Pf4Jf/7qoTatWxQ/eFn7Vr19xt0rnznDBBaE7Rd0qInKk1JpwnzCheJ87hCssJ0w4vPU2axYute/Zs/jyTz4p2rsvDP2pU8Mwt4Xq1Sv6r6F58xDe111XtDfeqZO6VUQkOWpNuI8aFb7fc0+4lL5duxDshcurW5s2YTjb/v2Llh04ELZdGPhfflm0R65uFRFJJbWmz11ERBLvc0/o3BAzG2Rma81svZmNK6PN5Wa2xsxWm9nvKluwiIhUnwq7ZcwsDXgcOB/IA5aa2Rx3XxPX5hTgh0Avd99pZm1qqmAREalYInvuZwLr3X2Du38FzACGlWhzPfC4u+8EcPdPqrdMERGpjETC/SRgc9z9vNiyeKcCp5rZP8zsn2Y2qLQVmdlYM1tmZsu2bdtWtYpFRKRCiYR7aeeAlDwKWx84BegLjASeMbMWhzzJfYq757h7TuvWrStbq4iIJCiRcM8D4ufoaQtsKaXNbHff5+7vA2sJYS8iIkmQSLgvBU4xsw5mdhQwAphTos0fgX4AZtaK0E2zoToLFRGRxFUY7u5eANwMLATeAWa6+2ozG29mQ2PNFgI7zGwNsAj4gbvvqKmiRUSkfLqISUSkFqnWi5hERKR2UbiLiESQwl1EJIIU7iIiEaRwFxGJIIW7iEgEKdxFRCJI4S4iEkEKdxGRCFK4i4hEkMJdRCSCFO4iIhGkcBcRiSCFu4hIBCncRUQiSOEuIhJBCncRkQhSuIuIRJDCXUQkghTuIiIRpHAXEYkghbuISAQp3EVEIkjhLiISQQp3EZEIUriLiESQwl1EJIIU7iIiEaRwFxGJIIW7iEgEJRTuZjbIzNaa2XozG1dOu+Fm5maWU30liohIZVUY7maWBjwODAYygZFmlllKu6bArcCS6i5SREQqJ5E99zOB9e6+wd2/AmYAw0pp9xPgIeCLaqxPRESqIJFwPwnYHHc/L7bsIDPrBpzs7nPLW5GZjTWzZWa2bNu2bZUuVkREEpNIuFspy/zgg2b1gEeA71W0Inef4u457p7TunXrxKsUEZFKSSTc84CT4+63BbbE3W8KdAH+amYbgZ7AHB1UFRFJnkTCfSlwipl1MLOjgBHAnMIH3X23u7dy9wx3zwD+CQx192U1UrGIiFSownB39wLgZmAh8A4w091Xm9l4Mxta0wWKiEjl1U+kkbvPB+aXWHZfGW37Hn5ZIiJyOHSFqohIBCncRUQiSOEuIhJBCncRkQhSuIuIRJDCXUQkghTuIiIRpHAXEYkghbuISAQp3EVEIkjhLiISQQp3EZEIUriLiESQwl1EJIIU7iIiEaRwFxGJIIW7iEgEKdxFRCJI4S4iEkEKdxGRCFK4i4hEkMJdRCSCFO4iIhGkcBcRiSCFu4hIBCncRUQiSOEuIhJBCncRkQhSuIuIRJDCXUQkghIKdzMbZGZrzWy9mY0r5fE7zWyNma0ys1fMrH31lyoiIomqMNzNLA14HBgMZAIjzSyzRLM3gRx3zwJmAQ9Vd6EiIpK4RPbczwTWu/sGd/8KmAEMi2/g7os3HmmzAAAG6klEQVTcPT92959A2+otU0REKiORcD8J2Bx3Py+2rCzXAgtKe8DMxprZMjNbtm3btsSrFBGRSkkk3K2UZV5qQ7PRQA7wi9Ied/cp7p7j7jmtW7dOvEoREamU+gm0yQNOjrvfFthSspGZDQTuAfq4+5fVU56IiFRFInvuS4FTzKyDmR0FjADmxDcws27AU8BQd/+k+ssUEZHKqDDc3b0AuBlYCLwDzHT31WY23syGxpr9AmgC/J+ZrTSzOWWsTkREjoBEumVw9/nA/BLL7ou7PbCa6xIRkcOgK1SrIDcXMjKgXr3wPTc32RWJiBSX0J67FMnNhbFjIT92Vv+mTeE+wKhRyatLRCSe9twr6Z57ioK9UH5+WC4ikioU7pX0wQeVWy4ikgwK90pq165yy0VEkkHhXkkTJkCjRsWXNWoUlouIpAqFeyWNGgVTpkD79mAWvk+ZooOpIpJadLZMFYwapTAXkdSmPXcRkQhSuIuIRJDCXUQkghTuIiIRpHAXEYkghbuISAQp3EVEIkjhLiISQQp3EZEIUriLiESQwl1EJIIU7iIiEaRwFxGJIIW7iEgEKdxFRCJI4S4iEkEKdxGRCFK4i4hEkMJdRCSCFO61WG4uZGRAvXrhe25usisSkVShCbJrqdxcGDsW8vPD/U2bwn3Q5N0ioj33Wuuee4qCvVB+flguIqJwr6U++KByy0Wkbkko3M1skJmtNbP1ZjaulMcbmtnvY48vMbOM6i5UimvXrnLLa1Kq9P2nQh2pUIPqUB0AuHu5X0Aa8B7QETgK+BeQWaLNd4EnY7dHAL+vaL09evRwqbrp090bNXKHoq9GjcJy1ZGcOlKhBtUR/TqAZV5BvnpYfYXhfjawMO7+D4EflmizEDg7drs+sB2w8tarcD9806e7t2/vbha+H+lfVvew3fhf1sKv9u3rXh2pUIPqiH4diYa7hbZlM7PhwCB3vy52/9vAWe5+c1ybt2Nt8mL334u12V5iXWOBsQDt2rXrsWnTpsr+oyEppl698CtakhkcOFC36kiFGlRH9Osws+XunlPh9hJZVynLSpaYSBvcfYq757h7TuvWrRPYtKS6VOn7T4U6UqEG1aE6CiUS7nnAyXH32wJbympjZvWB5sB/qqNASW0TJkCjRsWXNWoUlte1OlKhBtWhOg6qqN+G0Ie+AehA0QHVziXa3ETxA6ozK1qv+tyjIxX6/lOljlSoQXVEuw6qq88dwMyGAI8Szpx51t0nmNn42EbmmFk68BzQjbDHPsLdN5S3zpycHF+2bFkVPo5EROquRPvcExp+wN3nA/NLLLsv7vYXwGWVLVJERGqGrlAVEYkghbuISAQp3EVEIkjhLiISQQmdLVMjGzbbBtT2S1RbEYZakEDvRxG9F8Xp/SjucN6P9u5e4VWgSQv3KDCzZYmcklRX6P0ooveiOL0fxR2J90PdMiIiEaRwFxGJIIX74ZmS7AJSjN6PInovitP7UVyNvx/qcxcRiSDtuYuIRJDCXUQkghTuVWBmJ5vZIjN7x8xWm9ltya4p2cwszczeNLO5ya4l2cyshZnNMrN3Y78jZye7pmQysztifydvm9nzsVFk6wQze9bMPonNVle47Fgz+4uZrYt9P6Ymtq1wr5oC4Hvu3gnoCdxkZplJrinZbgPeSXYRKeKXwEvufjpwBnX4fTGzk4BbgRx370IYNnxEcqs6on4DDCqxbBzwirufArwSu1/tFO5V4O5b3X1F7PYewh/vScmtKnnMrC1wIfBMsmtJNjNrBpwH/BrA3b9y913JrSrp6gNHx2Zpa8ShM7lFlrsv5tBZ6YYB02K3pwEX18S2Fe6HycwyCJOULEluJUn1KPDfwBGcbjhldQS2AVNj3VTPmFnjZBeVLO7+IfAw8AGwFdjt7n9OblVJd5y7b4Wwowi0qYmNKNwPg5k1AV4Abnf3T5NdTzKY2beAT9x9ebJrSRH1ge7AE+7eDfiMGvq3uzaI9ScPI0zTeSLQ2MxGJ7equkHhXkVm1oAQ7Lnu/odk15NEvYChZrYRmAH0N7PpyS0pqfKAPHcv/E9uFiHs66qBwPvuvs3d9wF/AM5Jck3J9rGZnQAQ+/5JTWxE4V4FZmaEPtV33H1SsutJJnf/obu3dfcMwoGyV929zu6ZuftHwGYzOy22aACwJoklJdsHQE8zaxT7uxlAHT7AHDMHuCp2+ypgdk1sJKE5VOUQvYBvA2+Z2crYsrtjc82K3ALkmtlRwAbgmiTXkzTuvsTMZgErCGeZvUkdGorAzJ4H+gKtzCwPuB+YCMw0s2sJH341Mv+0hh8QEYkgdcuIiESQwl1EJIIU7iIiEaRwFxGJIIW7iEgEKdxFRCJI4S4iEkH/HwM06dUgDc2vAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "acc = history.history['acc']\n",
    "val_acc = history.history['val_acc']\n",
    "loss = history.history['loss']\n",
    "val_loss = history.history['val_loss']\n",
    "\n",
    "epochs = range(1, len(acc) + 1)\n",
    "\n",
    "plt.plot(epochs, acc, 'bo', label='Training acc')\n",
    "plt.plot(epochs, val_acc, 'b', label='Validation acc')\n",
    "plt.title('Training and validation accuracy')\n",
    "plt.legend()\n",
    "\n",
    "plt.figure()\n",
    "\n",
    "plt.plot(epochs, loss, 'bo', label='Training loss')\n",
    "plt.plot(epochs, val_loss, 'b', label='Validation loss')\n",
    "plt.title('Training and validation loss')\n",
    "plt.legend()\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "훈련 샘플의 수를 늘리니 단어 임베딩을 같이 훈련하는 모델의 검증 정확도가 70%를 넘었습니다."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "마지막으로 테스트 데이터에서 모델을 평가해 보죠. 먼저 테스트 데이터를 토큰화해야 합니다:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "test_dir = os.path.join(imdb_dir, 'test')\n",
    "\n",
    "labels = []\n",
    "texts = []\n",
    "\n",
    "for label_type in ['neg', 'pos']:\n",
    "    dir_name = os.path.join(test_dir, label_type)\n",
    "    for fname in sorted(os.listdir(dir_name)):\n",
    "        if fname[-4:] == '.txt':\n",
    "            f = open(os.path.join(dir_name, fname), encoding=\"utf8\")\n",
    "            texts.append(f.read())\n",
    "            f.close()\n",
    "            if label_type == 'neg':\n",
    "                labels.append(0)\n",
    "            else:\n",
    "                labels.append(1)\n",
    "\n",
    "sequences = tokenizer.texts_to_sequences(texts)\n",
    "x_test = pad_sequences(sequences, maxlen=maxlen)\n",
    "y_test = np.asarray(labels)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "그다음 이 절의 첫 번째 모델을 로드하고 평가합니다:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "25000/25000 [==============================] - 0s 19us/step\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[1.6611276818060874, 0.50412]"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.load_weights('pre_trained_glove_model.h5')\n",
    "model.evaluate(x_test, y_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "테스트 정확도는 겨우 50% 정도입니다. 적은 수의 훈련 샘플로 작업하는 것은 어려운 일이군요!"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}