6.3-advanced-usage-of-recurrent-neural-networks.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Using TensorFlow backend.\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "'2.2.2'"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import keras\n",
    "keras.__version__"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 순환 신경망의 고급 사용법\n",
    "\n",
    "이 노트북은 [케라스 창시자에게 배우는 딥러닝](https://tensorflow.blog/%EC%BC%80%EB%9D%BC%EC%8A%A4-%EB%94%A5%EB%9F%AC%EB%8B%9D/) 책의 6장 3절의 코드 예제입니다. 책에는 더 많은 내용과 그림이 있습니다. 이 노트북에는 소스 코드에 관련된 설명만 포함합니다.\n",
    "\n",
    "---\n",
    "\n",
    "이 절에서는 순환 신경망의 성능과 일반화 능력을 향상시키기 위한 세 가지의 고급 기술을 살펴보겠습니다. 이 절을 마칠 때면 케라스에서 순환 신경망을 사용하는 대부분의 방법을 알게 될 것입니다. 온도 예측 문제로 세 가지 개념을 모두 시연해 보겠습니다. 이 시계열 데이터는 건물 옥상에 설치된 센서에서 취득한 온도, 기압, 습도 같은 데이터입니다. 이 데이터를 사용하여 마지막 데이터 포인트에서부터 24시간 이후의 온도를 예측하겠습니다. 이 문제는 시계열 데이터에서 일반적으로 나타나는 여러 가지 어려운 점을 가지고 있습니다. 전형적이고 꽤 도전적인 문제입니다.\n",
    "\n",
    "다음 기법들을 적용하겠습니다.\n",
    "\n",
    "* 순환 드롭아웃 : 순환 층에서 과대적합을 방지하기 위해 케라스에 내장되어 있는 드롭아웃을 사용합니다.\n",
    "* 순환 층 스태킹 : 네트워크의 표현 능력을 증가시킵니다(대신 계산 비용이 많이 듭니다).\n",
    "* 양방향 순환 층 : 순환 네트워크에 같은 정보를 다른 방향으로 주입하여 정확도를 높이고 기억을 좀 더 오래 유지시킵니다."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 기온 예측 문제\n",
    "지금까지 다룬 시퀀스 데이터는 IMDB 데이터셋이나 로이터 데이터셋과 같은 텍스트 데이터입니다. 시퀀스 데이터는 이런 언어 처리 분야뿐만 아니라 훨씬 많은 문제에서 등장합니다. 이 절에 있는 모든 예제는 날씨 시계열 데이터셋을 사용합니다. 이 데이터는 독일 예나 시에 있는 막스 플랑크 생물지구화학 연구소( http://www.bgc-jena.mpg.de/wetter/ )의 지상 관측소에서 수집한 것입니다.\n",
    "\n",
    "이 데이터셋에는 수년간에 걸쳐 (기온, 기압, 습도, 풍향 등과 같은) 14개의 관측치가 10분마다 기록되어 있습니다. 원본 데이터는 2003년부터 기록되어 있지만 이 예제에서는 2009~2016년 사이의 데이터만 사용합니다. 이 데이터셋은 시계열 수치 데이터를 다루는 법을 익히는 데 안성맞춤입니다. 최근 데이터(몇 일치 데이터 포인트)를 입력으로 사용해 모델을 만들고 24시간 이후의 기온을 예측하겠습니다."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "데이터를 살펴보죠:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['\"Date Time\"', '\"p (mbar)\"', '\"T (degC)\"', '\"Tpot (K)\"', '\"Tdew (degC)\"', '\"rh (%)\"', '\"VPmax (mbar)\"', '\"VPact (mbar)\"', '\"VPdef (mbar)\"', '\"sh (g/kg)\"', '\"H2OC (mmol/mol)\"', '\"rho (g/m**3)\"', '\"wv (m/s)\"', '\"max. wv (m/s)\"', '\"wd (deg)\"']\n",
      "420551\n"
     ]
    }
   ],
   "source": [
    "import os\n",
    "\n",
    "data_dir = './datasets/jena_climate/'\n",
    "fname = os.path.join(data_dir, 'jena_climate_2009_2016.csv')\n",
    "\n",
    "f = open(fname)\n",
    "data = f.read()\n",
    "f.close()\n",
    "\n",
    "lines = data.split('\\n')\n",
    "header = lines[0].split(',')\n",
    "lines = lines[1:]\n",
    "\n",
    "print(header)\n",
    "print(len(lines))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "420,551개 데이터 전체를 넘파이 배열로 바꿉니다:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "\n",
    "float_data = np.zeros((len(lines), len(header) - 1))\n",
    "for i, line in enumerate(lines):\n",
    "    values = [float(x) for x in line.split(',')[1:]]\n",
    "    float_data[i, :] = values"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "예를 들어 다음은 시간에 따른 기온(섭씨) 그래프입니다:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "from matplotlib import pyplot as plt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAAEYCAYAAAAJeGK1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJztnXeYFEX6x7+1EXaXzBIXWMIiOehKFANZQTGe6J3iqcfpGX8YDgOeARU9cz4MJ2fArCAYyBKUnHNcYGGBJeeN9ftjenZ7Z3tmOlR3V/e8n+fZZ2d6OrxT011v1VtvYJxzEARBEIRsxLktAEEQBEFoQQqKIAiCkBJSUARBEISUkIIiCIIgpIQUFEEQBCElpKAIgiAIKSEFRRAEQUiJMAXFGItnjK1gjE1R3jdnjC1ijG1hjH3JGEsSdS2CIAjC/4icQd0HYIPq/QsAXuWcZwE4AuA2gdciCIIgfA4TkUmCMZYBYAKAZwGMAnA5gHwADTjnxYyxngCe5JwPinSeunXr8szMTMvyEARBEPKybNmyg5zz9Gj7JQi63msAHgZQTXlfB8BRznmx8j4XQGOtAxljIwGMBICmTZti6dKlgkQiCIIgZIQxtlPPfpZNfIyxoQAOcM6XqTdr7Ko5VeOcj+ecZ3POs9PToypUgiAIIkYQMYPqDeAKxthlAKoAqI7AjKomYyxBmUVlANgr4FoEQRBEjGB5BsU5f4RznsE5zwQwHMAszvmfAcwGcK2y2wgAk6xeiyAIgogd7IyD+ieAUYyxrQisSX1o47UIgiAInyHKSQIAwDmfA2CO8no7gG4iz08QBEHEDpRJgiAIgpASUlAEQRCElJCCIgiCIKSEFBRBEIROPl6wA2tyj7ktRswg1EmCIAjCzzz543oAQM64IS5LEhvQDIogCIKQElJQBEEQJlm1+ygOnSxwWwzfQgqKIAjCJMPeXoAr3lrgthi+hRQUQVgg79gZbMg77rYYhIvsOXrGbRF8CzlJEIQFej4/CwAtmhOEHdAMiiB8zI+r9iJz9FQcP1vktigEYRhSUAThY96dsw0AsOvQaZclIQjjkILyIR8v2IHBr811WwyCIAhL0BqUDwkGExKEZhlrgvAINIMiCB/xwFerMHHxrkrbh7453wVp/Evm6KnIPyFv/NNDX6/CrI37NT9btP0QnpnijUEsKSiC8BHfLs/FI9+tcVuMmGB17lG3RQjL18tycevHSzU/u378Qnw4f4fDEpmDFJSPyRw9FYdPFbotBkFU4qc1edh92D+OG5xzTF61FyWl3jGqeiHpLSkon7NxHwWRxjKcy9lh/uOz5RXMjiWlHJe9Pg8z1mubpWTn+xV7cO/EFfhg3na3RcGZwhJd+63PIwVFEKY4dLLAU6NRwjjHzpTHZp04W4T1ecfxwNerXJTIPJv3nwQAKdal9h0/67YIwiAFZYG7PluOJTmH3RbDdxw8WYDzxs7AS9M2uS2KJgXFJThbpG+U6jYb951wWwRdBCd6pwqK3RXEAKt2l69BvffbNhcl8S+koExSUsoxdU0ernvvD7dFiQgDc1sEwwTXzWQ19/R96Te0GfOL22Lookntqm6LoItFOwIDvWIPzZo/XVTZW1IG6fWadSW1/laAFJTP4VI8Mv4imBzUC7Oo1CRvhDp6oS1D8boDkhd6BlJQJgnOS5j3JiiEII6epvx2ovDLQGrHwVNui6CbrQdOui1CVEhB+Rwvmvi8QlFJqdsiRGWXR1y5Zb1POec4YSDR7qyNB2yURh9M56jZC05IpKBMIv9PS4ik/yu/VcpvWOgBBXVap8ux28hqifh04U50fHIadh7yzsxI1tACM5CC8hkHfORiKhNbD5ys5BGXFO+tx6dU0hHzqt1HceC4++7ZWkxTHHVyDp32VccPANvyT+KrJbvdFiMi3lhBJXRzreRehX6ialK82yIY4pOFOzGiV6bbYlTq6Ie97Y2S6b+uk9Or1CzzthzEvC0HcV12hm6zoNN4awhIRMUraw6RkGmgWlrKce/EFW6LIYT9NLu2xNo98mZeKC4p1ZwhnyoojjpzDgYZy4hlBcUYq8IYW8wYW8UYW8cYe0rZ3pwxtogxtoUx9iVjLMm6uPIhU2fqF9zw6DpxtgiZo6fi/bkVU9UcOFGAyav2ah7jtd9eFnGPn/VOMG4wpGDfsTMuSxKZVo/9jKve/b3S9vb/+hUtHv0J78zZGvZYmT0oRcygCgD05Zx3BtAFwGDGWA8ALwB4lXOeBeAIgNsEXEsapqzW7rQI60z4fScAYIuDbrAHTwZiWj5btNOxazpNqSQadVu+vCP2ULbnB5wjvlmWK60jRxB1ZotQXvxFzqws0bCsoHiA4B2XqPxxAH0BfKNsnwDgSqvXkomHv1ld9jpz9FQXJYmM7A+VFit2HXFbBF2Etu3GfcelckQIXeuRRD95Fmo/5xGyBsUYi2eMrQRwAMB0ANsAHOWcB+fyuQAahzl2JGNsKWNsaX5+vghxYo78CGYoWncwRmgfFCnxprrDWrvnGAa/Ng9vzw5vSrGbaMrxt01yPF95RyPfk/9UDf5kwouDPT0UFMkbLiFEQXHOSzjnXQBkAOgGoK3WbmGOHc85z+acZ6enp4sQJ+a45b+Lce/EFTh6unLqlR9W7AEQUFRu13/5ZOFOZI6eioLiyLE5bnoU7Tx0Gp2e/LXsfYHOFDx7lbWKVRLX2Nm0P+AmH2lA4wS1UhMjfv7lUvlcnyWaGAtH5ng+oW7mnPOjjLE5AHoAqMkYS1BmURkAaNHGJjbkBWo+FRZXvtGCnX2fF2ajsKQUOeOGOCqbmtdnbAYAHD9TjPRq2i7anPOy7+MWXlrEVxOq18OZpG79eAnW7DmGPq3qolaq875LVRK95Z4PeCv41aikNapGHjC4iQgvvnTGWE3ldVUA/QFsADAbwLXKbiMATLJ6LZmQ6X4Nju7emRM+5b/MoyQ1pyTLfBBpNrdoxyGNre7dGHpnnnmKR5rMmcNlUwjLdx2V6pkXyfq98hY1FWHiawhgNmNsNYAlAKZzzqcA+CeAUYyxrQDqAPhQwLWkQcYOX0aZIrE053BZZxkOt7JcB6uSNqpZJew+d39eHh/1+7aAslq+K7wnldP4tD91jdwj3ogxNGogf3PWFlvkEIEIL77VnPOunPNOnPMOnPOnle3bOefdOOetOOfXcc7lzGXiI7QWyWUbiQLlSVavfe8P9H3pt4j7thnziysdQ9snfkFhcSnidM5K5m4OOCB4qQSDW/EvEt6SMU33FnXcFiEslEnCBMWSzlRkXqBX02vcrLLXZ4pKyoIhw/FuBNOlnRw5Xai7C98uYZkFrTXJwDb53dEm/J7jtgiVkDUdUCh+0v+koAwy7ueNaPXYz26LoYnWbMkLD9X9XwRMZZ8v2oWfVue5LE053Z+b6bYIlli3t/KA5Zb/LsbBk/IbM578cb2r1885eKrS8/TL2n0uSSOGjfu015pkntFSslgd3DtxBdbuOYZZD16M935zZzSvh2Z1Uipl3J618QByJBzdq1mSEwjMffT7NZqfu6lj5Vfv4dHKHRdcK3ODY6eL0PnpaWjXsLprMuhh0fZDuH78QrxwTccK2894sOqvmo8X5ODG7k0rbZd5DEsKKgTOOX5cnYeB7eqXucO6GTNihFop2i7Dt01Y4rAk0TFSLE3WYnayEVrN9S0Xg4a12HYwkHBmvcthBNHYqqRiWrnbGyZzvXyxZDe+0CivkRfFxO4mZOIL4fdth3DvxBUY9/PGSp95xYsnFBkrZx7RCComrHHJS3MqvJfNdKNlgjZSrdYpgmLK6GCkB621x0jMliTDiBakoEI4dibwwOw7VjkdywUvzHZaHF8RTMgKeMd0JrP5Ixqyda9a/b2EYyf8ui6w1qQ12/ACk1Z6w+KjB1JQYfBixxR0dfYCXnDeAOSbhXgZrabUm0rKSbw+u/fqzE8LUlA+Yq/GrE9WFmw96LYIvmbHwVPSxWSt1Ahijo+Tb6ByddcMAMBVXTXzW0vN6tyj0s2crUBOEipKS3lZ/IXRQcipgmKkJlNz6iXUI+p4lLWIL5fsQtemtdC6fjVb5AnXTcq4fqfm+NkipCYlVOroL3t9nksShWfh9soehDIqqMT4gEypyd7LGXjFWwuQlOCfeYd/vokAJq3ag0U7DlfYpreDav+vX6Pv5BI5h+Rz7gjtlr5cHNne/89v12Dgq3NtkyfcrxxaPmPnIf0u+2OnrLe1VlhRSSk6PTkNYyatrfSZLpdoh3WvllWXPDTFY9RJQmZIQak4ocpiHXyYiiTNGuF1Hgqp+XM6QpJYNz29co9UdMG96N9zdB/7wfwdgqUJsCb3GDjnZffm98v32HIdGbjvixXIHD0VR1wwV/poKcezkIJSob4hC5RRCN2kzhCpxMYPDnglhRvHy+bLMXdzPi5/az4+WeiP0vTR8gEGPdI27z8RcT/Cn5CCUqH2flmdK09Waq/zwFer8Or0zRH3kU0RyMomJVPIYpUp2nTSV4fb3MpgjwOYuWE/pq/fL0yeaNA96T6koFRUfH6Ysk2uKdTBkwW46/PlOFngnaJ63y7PxeszI6f0P3BCX364aGXN/c43y3IBAFNE5Cz0UFNyDtw2YSn+9r+ltl6nqKQUYyats/UahH5IQalQ932yjp5en7EFU1fn4Vulo/ILeteZSmPc5hos237NuRll27zsaJCvc2DiFCc9Wk3Zr5CCCkOwYN3ynXKZ+oKK00/BeACwef9Jt0XQxIxC/H5FxcGDHY42y3YKMPGFkHvktOP31QCdnpk3vL/QZkkqs8pkLj43HDqsEi7TuduQglKhfjhPFhTjqR/XGcpe/tH8HbY/4MGxstmr7D5sv8v51gMnbWsHu1o3XALTBVuNZ/8ODUjda0MyzpxDp8WY+RQ27z+BC16YjffnbRd2Ti9RWFyKlbsr/m5mHTO6PjNdhEiOMvg1+eLmAFJQFQgtRfDfBTmYbyDjwdNT1mPh9sPRd7RAMEWQ2f7/8R8qx8yIZEnOYfR/5TfbvMzs0v+rBDrFODUHyTtanjnE6oBglxIrt8jG+3eOxKm4nvtpA658e0FZJnOvMHz8H7bG2rlNTCuo42eLKgRe5hgIwgxHQbEzucXMdkdHThcic/RU21INBUs+rFaq+4aOSq0im9OKFk6tCJWolNKsjQeEnNPO1hWVlePzRbuEnEdNsLijbOmhomH3gNhtYlpBXfn2ggqBl15Y1lE7bxw/W4SPDAaDBhWHU2XUjWRe8AsT/tiJs6pMDnbdV+PnBn7Ds0WlpqrkqsUS7RR0tqjEtsFauMKWIlA3gwe6A98T0wpqe774zvOEQ15AnHOM+WEtnp5irjT2/K0HcdOHiwRLVY4XlL2dfOpAIO3ZonLni7V7xCxyi1o7bDPmF2SPnWHpHF4KpSDsIWYVlHqEG5zei3g4cw6ewl//u7jC+UWidik+dNKaOWLeFvFmvtCBuGhF5RXF57Q7vNU1PzvCKqwO1u78dFnYzw6ZmDFGwiv3VawRswpqjMpZYMgb8wGIKZ728vTNmL0p35bOH1C7mcuZaVstUeboqbj/y5WuyRKJhdsPxXyexRW7jrgtQkQiPUPnjZ1hS4Vr9Tq0n5KuepWYVVAbQvz+M0dPFXpDiqoisPvwaQx7ewGOKkXUJq0MJAZdknNYSgXlBZbtPILh4xfivi9WuC2KMMzMgPI8VD9Mi202mOh3Csr8/5VHq/HKRswqKC1OF4qzeccJ0lDvzNmKVbuP4qc1gTLUwbLp09bvxwkJbfTBb/3tcnsyXYgwxQSVfLBNAfHZGILejDKj1ZZeGvIETfMiEfX9v1mei2U7j/guoN5pSEGpEHkv2RGcGUqkDOBusd3mjlmEm7kTfcaRU+6VCNGLep0sqKBFt82SnMPIHD0VczaJcYNXU1zCpc3NuHjHYVzz7u/4fLF4l/hYImYVlN35y2K1pPl/DGTeMIOIDlRLyYl2ElDHKDkxWLHcLjY9Di/+shEAcMt/lwg/9yvTN+OqdxYIOVew+UQ3w7YDYgdspwuLbZudyzjbi1kFpdVJiTSZiVaAXghQ9QqnCip7WIrumNQPe2iGElnQ6o9+E5ztoajE3vt2Va5YM98pCc3mam6fsBSXvDTHbTEcw7KCYow1YYzNZoxtYIytY4zdp2yvzRibzhjbovyvZV1cDyGoxysoIk8iNSK6O62RIql/MazcfRT3TCx3PnEi96MIlu0MeDSKLo4pemBp52BHwgmUkBlUMYAHOOdtAfQAcBdjrB2A0QBmcs6zAMxU3scMB46L8ZD6bkVgQd8JM5EXEGGGcPo5lLV0S7jO08q6zpVvL8CPq8o7eQn7PMJDWFZQnPM8zvly5fUJABsANAYwDMAEZbcJAK60ei2R2L0GJd704EyOP6swm3tjMTOoytvslFrGkSkQUv9Mtf2QwHx0XsttJxpZf3stZBRV6BoUYywTQFcAiwDU55znAQElBqBemGNGMsaWMsaW5ufLm+3YKPGCO2oZFzC9ihNZHmZsKPdak3X9UN0M6rRCssrrRYJ15YpLSnH358uxaZ+5Eh5OsOPgSXy1VK74LWEKijGWBuBbAPdzznX7P3POx3POsznn2enp6aLEicqaPeJjKNSInkjY1WWIXhS3G9LT9lBhrdNkG8vq8u0mXyod/sZ9JzBldR7+T1BmFTvaesgb8/HwN6uFn9cKQhQUYywRAeX0Gef8O2XzfsZYQ+XzhgDEB0JIjGiT0ffKWpRoRny02Jbz2obg53KNYoqVdZ3ITirEQQn4/i0e/cn6SXxKsKlF3WfhCmxaoUDC1E4ivPgYgA8BbOCcv6L6aDKAEcrrEQAmWb2Wl4gT3OM5lSVddoQE6qpeX/5WIA+jnWuSss761GbjJTnqEvKEaIL3rahuQdZ7SjQiZlC9AdwEoC9jbKXydxmAcQAGMMa2ABigvI8Z7BiRP/j1qrLXHRvXEH8BD7B5v4CKpyEP9y9r96GoVL7Ro5OoCx7G4GTSdoIWOVEDIaez5btFgtUTcM7nI/w93c/q+b1KvKhssSq+WVae365G1UTh5xeB3Z3bT2vy0K15bUvnCJ2F3RGhrIOfKVUy4sfHsQoz/tjo+pzl/XnbAYhb+44VBRWzmSTsxm53a8I8MfJsR+X1mVvQ8tGfcKawRLhJmqjI1NV5Qs9XymMjNpIUlE146XFXB1b+7X9Lcfmb802fy+6+n3OOyav24u3ZWy2cQ6BAeq7n7OV0EyzXcvxsEc6oCmx66d61yrR1+7A696jbYhiGc25rRWxZsGziI7QRVW7DCe6ZuAKXd24EAJi+fr+lc9ldo2rGhgOY8Eegeuxdl7QydQ6K86lI/omCCgG1sdQ6Iz/xpnmXA9h9mGZQhEk8pJ88xR4BZg0y8VXkD0mT2RLhKS3lKLSpIrRMpkNSUDbhRZt+2zG/uC2CIXaZrH5K+qkidnV0hH3Yaak4dFKe9FSkoGzCbvU034Z6U+p1CC8wY4M1cyQR4KVpm9wWgTDIjR/4f/0JIAVlGyK8+HYekr9suJs8PWW9qeMcd5KQfMomu3x28NasLfhl7T63xZASmdZoyUlCYvKOiSnZQYQizwNIuMNL0za7LQKhA5pB2YR6AnXibBEuf3M+tuw3lsk4Fke2TuCldj12pqjs9VmHTLDdn5vpSdfrUDbkHcfPa8TGH8UCL03bjLU2J9PWCykoB5i/5SDW7Dlm2Nb//Yrc6DsJ4rmfNlg6ftXuo5i10RtrQk5H4Zs1mcxYvx+dn5qGxTsCefKcrK48e6O3stxrcenr83DnZ8vdFqMSOw7Kbbqfuzm/LEel25CCsgn1DCq4HmW0X8w94py75/i52y0dP+ztBbj146WCpCEA4I/tAffvstmMg46hxTGem9BOLnlpjtsiREUWKwMpKAcIxkRRuZzYRFSCUCcjF4pK6GYVxZEYrypsBVJQNqEegZTPoOihlwGnf4VwJr6zRSU4cbZI8zO3KaLYKGE8oKpCQBiDFJQDBGdQpJ7kwPGZbJjr9X1pDjo+OS38YTzyezspJgUlDHUpk1BOFxbj/bnbqRpxGEhBOUDQNKN3BnW6sBg/rNgjjR3YTxSVlKJEkvWVvRKHERi59dbuOYbM0VNtkyUag1+b61kz2r0TV+LZnzbgp7XkbagFKSibqJoYX/Y6uAah96Ef88M63P/lSqzYfcQGyWKbrMd+xoKt7uSeO1tUgke+W6O7M3UzYHJpjv57b6iF7Pci2LjvBLo+Mx0Fxd7KhHK2qKQsG8qZQm/J7hSkoGyiZkp5QcHgDKqwWN/Ifdq6QIT7WQfdign7CKqZ75bvwcTFu/Bvg+EGbtQWW5933PFrWmXC7zlui2AI2df5tuWfdF1xkoKyCbVJedXuQNDb7zqzRp8oKLZDJMJlgjOicKbbDXnH8fdPlpZ1XLKZeEtLOUZ9tRLr9soRxBmKlwZ0uUdO46P5OW6LEZF+L/+Gtk+4m0CaUh3ZRCnnyB47HRefU88XUfmEefTOfx78ehXW7T2OjXkn0DGjRuXjXVRYBcUl2HXoNL5bvgffLd+DnHFD8OH8HXjGZD7EWKffy7+hQKdFJUgsegHTDMomtuefwsGThfhmWa4nS294iT1Hz+C3zfJmPnC7WxnSsaHhY+qmJVd43+O5mRjw6twK2yYu3mVJLtFwDmzZfwKZo6dKn60hknJ6Z85WLN91BMUlpWUmtoLiEvR4fqZT4kkDzaBsQp1DbeM+Yzn4CGMMfnUuThQUI2fcELdF0UTvyDe4W7jxjFmnCTPHqQtu/rI2D0dOV47XkrEo53cr9gAAfvJwDr4XfwmsUV7UOh2/bc5Hzrgh+Meny7H/eIEr8hw5VYgpa/JwU49mjl+bZlCEbr5ZlotBIaNoADjp8pqZ7Gt2Zp0cgpVug3W6zFp4LsxKN3yMetZ/x6fa+exkswwwVm4OVQ8KvGQaU8uqtgrMjBBLpYdaKqcto9z/5UqM+WEtNrjgOBMzCupsUQk+mLfd1kqUfufBr1dhk0ZG9mU7yR0+EmY7yGBbb1b+m71zrz+/ieFj9OiegxJVXgUCCrw85rB8+6cLd7ojkAH2Hg3ExF3wwmxbzm9lMHHoVGDm5obXYcwoqNdnbsHYqRvwg2ICIKxRWFyKl6dtwikJZy+Zo6fike9Wuy2GYaIpILOKzswMTs8RB0+KNzm9feO5lo7Xijnceei0pXM6wWszA/Wp9hy1J0H0oA4NTB9bZnp2MluxQswoqOPKmtBpl8uae8ncEIk3Z23Bm7O24o2ZW1yVI1wnOXHx7krb/vzBQrvF8Q1uxF4BQCeV96IZJLM66sbubuGevq1MH1saZW3UTmJGQcnC+/OslbWQhTdnbQUAFJaUujCuKid77Azd+7qVQeL9eTuw9+gZnDgbebYZrh3L1lWEShWZOJd6hia1U9CsTorl8zg5DhzSybiXpJcIrj09M2W94ymtSEE5zNTV3vQuWpJzWHO7LJU3ZafXuFkY9/PGiPuE61PjTNYTs4KbDhDDujQ2fayW1HY3W7wHpm0i7p1FO7T7ADuJOQVVUlKK3Yflt0nLxppcbUW0xEDONkKb0HWHSv1dWTZ85zSUTF1uvWrJ0XcK4XShc2ujIvRTjarhvezOCliW8OrCghAFxRj7iDF2gDG2VrWtNmNsOmNsi/K/lohrWeW5nzaiz4v2eMr4mbUR0tscOS2XN5fX6D1ulq48jXtMVFju2rSmGZFcWYO68+KWAIC/9GhaYfuMBy7SdTxHuRvff1QVomVStuGI1NwFAlI4ebWch6gZ1McABodsGw1gJuc8C8BM5b3rFLqcoNGbtwkquOfP33KwwmcfLchxWBpvk3uk8gy+pJSHdaAJek998odz7tJudOr/HNwGAFCvWhXT54hk4jtVUIxVu4/i2ali0zOlJMVH3ykKkdq7VJBtN6NWVSHncRIhmSQ453MZY5khm4cBuFh5PQHAHAD/FHE9N1g+ZgCSE+Jww/sLsTqMuUsPXnXiUz9AoclCvTo6c4v5Ww9G30lFWWyPDbJEu6aXYGAAC99Kd3++HLM3iU+J1TI9zfI5Is1YSwR0GpwDzeumItfELNxN7FyDqs85zwMA5X89rZ0YYyMZY0sZY0vz8+3Lp2b1gaudmoTU5AS0bVBdjEA206GxWDnVi+ahj4vIAL6ZOs05Xkarv+HgUc1qRkMUkhLicF+/LEPHBNmWL08uO71OCNHW6FZZGFjazYmzlVNJBRExABQ1C3Ma150kOOfjOefZnPPs9HTjKVmcxqqiW7PnGPJP2J9Tq6QU6N9Wc0xgijhV4rWDIfKLXK8QMRp1mlt6ZVo+RykPr4DiTM6gNo+9FBefI+4ecAsj33vmhvApgQ7bVHW3X9v6ls9RVBL+W+7WMAkbxZvqyV4FtZ8x1hAAlP/WkkmZZO/RMygoLpHKtDbmh7XRd7II5xwdGlsLelSjVkEfzN8R9jO91K9u3DPLbWqnJuHV6ztX2p6caP0xUiun0Ih9kRH8tVOTsPKJAcLO5wRGZo5rXAh7aF431dZExde8+4fufa/qqu2i79UEAXYqqMkARiivRwCYZOO1NCkuKUWvcbNw/xcrhZ2zegR3UL0UO7Bmwzlwe58WAIC7LzEfRR5E9JqEOtblxWs7CTnnoFfnYsb6/ULOpQUDcFXXjErbuzax7qAa6ZbQyi9nloQ4hpopSdZPJCFOJjNtVU/OmX7vVnUBAH3bVJw5e3WZWJSb+UQAfwA4hzGWyxi7DcA4AAMYY1sADFDeO0rwR5mxYb/h4mBq+mTVLXs9akBrPD6kLbY/d5np883YYF8nCgA9W9TB89d0RFpyAnLGDcGDg86xfM5IgZtWldefso0nM9Vi0/4TuP1/S4Wcywgiyk7oGeFa6WOuV9p4YHtj5qhZG7Xv1cUOBm3qNSH/uk5b1g9DZvwiaJmeCgDo3ry28HNboX71ZKz610C895fzKmxPiGOupa+ygigvvhvCfNRPxPnNoh55irI/V0mML5uZyMrEkT2En9OD97ZwwrVBvAANpW8GZV5FZdUPjPiT4o25RN/6sbbC/9N/9JudrJKSaN2NWzS/rtsv3Kx3/rP603aFo49GaZXnruqIzLqpls/tBq47SdhJsNso5bwsWWy9JlyIAAAgAElEQVSs8rc+zQ3tPz3EVObk6KuboFGp+Iwh2m0gIi1QhfpFleZKlTN0m76OpMvlTwxtF/YzGSWecGs34ee06jwVLh7rxu5NNbd7AX8rKFb+YC+1ULPozotaam7/+b4+ps/Z6tGfHC398diQ8B2AFn8LMZUtiWDSWbdXrO3/q7/3FHIedcaQA8fPmj5PckLkxyRO8AwqdKIULK1upQNjLuTz08uDA1vj1guMDaDc4uqujZEzbgguai2fx/G5TaVI1iMUXyuoYMS+1YeyV6u6mtvbNjQfa1RcyjF26gbTx4cyakBrYefSYsuBk0LPl5pU2bpcN80+z74nf1xn+th5/7xEc3ubBtUAAM3rGDefvBlSpuR4hDiYIHrWfTY+E5rQJYCsFtoOjavj7r7RY7WSogwSnOL5azqG/SyYCcMthnVpZGj//gLc4+1GyBqUrDz6/Rq3RYiIyIJv9/RthVembxZ2Pjt59LI2GNErE5l1y8sqbHxmsK3rXFYGKXVSk9G7VR3cpfKGHHlhC4wa0BoFxaURE32G4+WQ32quqrx3OFnPz6xlOjlvo5qB9EFNa1svZSGSMTpm9pxzXN21Mb5YUrnGl5MkxjMkJ4RfD3NbiV59bmUP03AsfrQfaqQk4pzHf7FRIuv4WkE5WQHyll6Z+Pj3HMeuF4qXPHRGXhgwmapdzavYvBAeuqZmhPg4hs9uL3c8US+Oi5L77dnbUDct4P4dbjalRzmFU26D2jfAJ7d1Q++WAWvAwHb1MS1Km9id9X/a/12I1vWrRd1PFhfpaIMc7zyBQL3q5vMdch4964ko5Jg3C4ZzjrdmbcFem8ona/HkFe2FnMerAXWycqYwUKrAidgzK6hn03/+YJGlc/VrUw9J8RUfbcYY+mSll62XJcRH72DszvqvRzkB3k3T4zRav2iXJjUjfm4GR+uSOXcp59h9+AxemrYZ2w/Kk09MBi45R76FXbtp+4TcJgw1G/edsHwOxoAPbzkfm5+9NMp+co/3g6U3APtn13poWjslakB5sElTBWQ3F0HOuCH44a7elbbf7hGHFMCnCopGXNq8GxK8R/iLj/96vu7O3M2KuXoIOp5cd15gXUUtrhuiz334kqhrPDVTAmuRt7kUJ6m3XXqHcfrSi5MDf18qKC9z8KTxgOK3buyqaz/J+yTCIkYSw8p+K3RvEYiFC5dbTkaGdW6MF6/phHv6Wk8tZgeinv9JK50Lj/Glk4TIjjiY0sQpLntjnuFjhnbS517qpNMIITe1U+XOx9esTmgCVvnv3bg4hj+dLyZtlxn0mm2tBmtvyLNuitYLzaBCuKlHM3xwc3bZ+3f+HN0sNshgfrNw5J8ocKQUR6xR7EAV5RmjvFXHavSlbZDdzJuBnWTBN4coFV9c6lxVcl/OoBZsPWTp+P7tyhXOOQ0iexqJyse1fNcRXP3O70LOFQ4ZTHyt6zufBfqshUTBerE7u/X/fSkuIz8QcDy4qWczSxlWzHJV18a40kOmO7/Qt009zN6Uj0wTgeVq5thQlTgcvpxB/Wuy+XpL2ZnOjiqDSWxFlAo4t2lNvPKnyvWKgoTqJ5EFDfVSVSODhBZtogwMjHDEpkJ1TvK9g2mx7ObV67tYShUkw0DLi/ylRzOsemIgWqiKgr5wTUc0qV3VRaki40sFFak6ZTTUwaNOUKiM7s2uD93cs1nZ6+/+0Tuip1Gojfra89yzl0fj6zt6YuYDF2HtU4Msnyv3iHPxcIS9JCfE4cMR2dF3DOFsUYkN0ngLxhhqpFTMenL9+U0x7+G+LkkUHV8qKLdQ143Si9XRoJVSD9GufbqwGNvyxeTgM/o9q1VJRMv0NKQlW7dCZ9RyZoT4PxsyXNuJG67mn9/e3dRxQVHHDG2HlunGzan/XZBj6rqEu5CCEsj/bu2GzDrmcp2Z7SuMLBiHXiLaJUf+bxn6vfybkOwWjys5125wwcvJqbg4UWVCnMINU1m4xMt64QCa1DL+jL3wy0ZD+zeuKa/Zyy46Nq7htgiVIAUlEMYYZj1wsbFj7BFF+1ohF1OnQdFi/taDAMR4TdVJTULOuCEY3s352jQlkqc50qL14z8b2n/WA8a9CK0uljvJZR0aAgC6ZdYWUt4kGmOv6mD7NUQSWuLdKJvHXorv/9FLkDTi8KUXnwi+vbNXWQZoIxh9eA6dKsT14xdiYDv7U98zxjBmaDtcmFUXWTrzoAFAkYNupVrMfegSFJaUov8rv5k63in9JHJGUmjA8/CFazpWWPjWS1OTs303uCCrrvAKtn7hglZ18dEt51s6RzAT+0e3ZIetouwGpKDCcJ5DMSK/rtuHHQdP4f15200df+S0MQ+120zk4XI7Jb/VjtSsie/qro3xnY+850IhZzh9zBh1odsiRCRRR+JfvXRpIldsHJn4XCZeGXabHeVPWrlXoDT24WapcbMmvn9f1zlsAUAtkuLjcGmHBvjMpCOAEWpUTSzL/WYW7xk+HURpnBZ1U9GqnriQB9mRbdAS8woq2jqM3fy42pqC6Zwh38KmbBTpzCShdnBpmZ6K+DhmKJM2Ywzv/uU8y8k49bDqXwMxtFNgXcasN16xhXAMQh4eG9JW2LlkizGLeRPfwPb1sXL3Udeuf6rAWnzGvwTVobIbN9PTHDkdvZx6KInx8o7dgp3IQ4PaICUpwXTsnkjTkN84v3ltXNmlUVlxTZkROcOTLV+nvE+hQwxu38DV6++xUFQxZ9wQnNtULptxONxUUCM+WqxrP7UlUIYaROHokxXIwlCjaiIevayt6VLjdteEEpkNxGnSkhPw2vCuaNeoutuiOItc+okUVIv0NEe8g8IVC5RtSh3LlHJeViivZ8s6LksTHlFVl+2+9UKV/JCODW2+ImEVEYHxIol5BeUUyQnaI/IWdb0Ti2IFL6x2cO7uA9pe52hd1MzH7sFRaMQFDcbkx0pmGjsgBeUy1ata88TyCqJG/YQ47F5vkL1qLyE/pKAcokcL7TQ48fQQS0NJKS9bN2nX0Pm1B723gleUfaiC8obUhEyQgnKIEb0yNbebHWV+ObKHBWmcp0cLedd0gqQkxaNf2/qY9cBFuLyzvirF4bipR7PoO4Wgd0YjysSXHOJccUM3sXkSaexFWMV2BcUYG8wY28QY28oYG2339WQlXKdi9iHu7oEOX021KnItvmrRWMl6biZtUChPXdEeK8YMMHSM3nvhb32MZwPRQp2Wa+5Dl2BIR2tKORQ71zP+flEL284tArfjK/2CrQqKMRYP4G0AlwJoB+AGxlg7O6/pNfw6ynxwYGs8MKB12XsvWKXS05KFnSsujqFWapKw86lpboNjDQcXfi9WKkoo8B5I1Vn40i3Muv7LgBN5QfVidyt2A7CVc76dc14I4AsAw2y+pikWP9oPcx+6RMi5QtebIpXg8OtC8t19s3BPvyy3xTDE/f1bR9/JRvx2J6Qps+ZGNQJJl2vbpLDdWC+Mxo02Z+2fcs8Ftp27VT3rFgRR2K2gGgPYrXqfq2wrgzE2kjG2lDG2ND/feq17s5Uz61WvIiy784SQwnU/RriZ/KqgQqmaJG/ga5B4tzMruHAv1BU4awwleG/3yUrHc1d1xKOXiUvJE+T2C5rjPzedJ/y8Vrmya2M8c6V9JTs62Fi7SSZXc7sVlNY3rTDR55yP55xnc86z09O1g1mNYCUzgyjCxTxpEay5ZIR61ezrVJY93t+W84rIzKDOMB9pQT+cx2Q03PaO09stiBTzv7ecjyEdGyKjVorwGVzwfBwcN3ZvassgpWpSPBIEDywm3dVb6Pm8hkyDZrsVVC4AdU+SAcDW9Nv9XjZXM8hO7E4pY4XQfGx1bBxRW0UdRBupTf9iwoMOcH+dzI3Ld8yogbf/fG5g1Cz4Ng12dHa06y29MzGsSyPcfkEL4fFcnQU5OASlurF7ubnPLjOnSGJpBrUEQBZjrDljLAnAcACT7bpYQbG1xKuxyJonB5W9Vjs1yIhePW82tswNBdVLnVLJbQ0pmODPYEfByOpVEvH68K6okZIovaOR+md9LMTMKUoZikT0jNQKtiooznkxgLsB/ApgA4CvOOfr7LqeF0t7m+HgyQJh51Kb3oxU2XWbSH25WVOS2cKGVkhQZU0vMFBF1w5Ez0RY2QzK3natIXk2FrUCrZNWcQZ1UZb9pVmMEksmPnDOf+Kct+act+ScP2vntWRLFW8X9pnh5Fbw6l830jNkxKtLva8dCiqaS/hzV5UvpG/cd0L49Y0gul+KK5tB2Xtficw8n2LDOpn661s1948a0BrPXdXRokSRkSm7jXed9T1EqqCb/o9H+gKwzxLkJQtTRFkNPF+pyeW/jdNf//LOjZBRS4znqAjOa1YL156XYekcL17bqex1cCTuFcPGLb0y8ev94sq7l/fz5Q0QemsabZp7+2WVrWkF3fdFE0trUDHPprGDhTlJlN84Yp/4YGVWO5w5RJpf4uP03a5mA26d9uLbffi0o9eLRmJ8HF66rrOlc5yfWe5BGbydPKKfMLhDAzSpLW7AYLdF59f/uxALH+kn/LykoGxi4fZDbotQCSMu59Gw64Z/6or2uO2C5ujftp7wc88YdRGm3ismqLB788qdXyg/3NXbkKJVt6kd+imSJG5Wco6EkSDQfm0q3jNqk2atlMB6S4Pq8nmGNq5ZtdI2dRiDXYgcA1arkogGNsyi7AxjMYqvFNQZk0G6XiHoZn1dttiknnXSkjFmaLsKC/aiSK+WjPaNxAQV3tSz3H08nNm0qckRcM2URFtSCHlxWbRD4xq6i3h2yqiJ+/trZwzpk1UXb93YFQ8OOkekeJpYrd479soOSLTh/gfEDXy+vbOXmBNFIZ0UlD3o8eKb9cBFZa9XPmEsmadVfnvoYlPHfX1HT9zauzmqJsVj09jBeNiBB15GqiTGo5mS7eOKzo019wnGmTSoHn1kqb4X3vvLebYoaA/qp7DseP4ytNVwQLmvXxZWPjEA8x4OpArrlFEDXZrUBGMMQzs1EmpFCMdTV7Q3tH++yhO2f9v6pmPnIpGhJB9umZ6GKomBe8uKFcSJGR4gV9ym3BkXDaLHW0idqbp6FWfdU5vVMTdCPz+zdplt34mH3SpLbcpGAZQnCY32DPVvVw+fLtwV9vPsZrUq3AvyPJJyUqNqombHFR8X6NBqpiShpmLSm3y3fXniwmHUEaOwuBT39svCGzO32CMQgAtbp+ObO3ri3Ka1MKxLIxw8WYjDpwrLPk9OiMNVXRvjzVlbbZPBDLQGZRPRZlDf3NGzwnuJBgpS0dFini8787vp/c2ixXKEjpjtXsg3OsKXjZnKbDO0VWUZbRt1ZW/ToBo6Kfe5nW7w2Zm1ERfHUK96FbRrVL3C/btp7KVokZ6mOSt1E3XGFrfxlYIqjqKgzm1acYosy8MlG0FzhMxo9SnqbPThFFStlERMvrs3hnVRah/ZfAsE77GeLevoXteRjdqpSWEHHRvyjjssjTa9WxkPeA3OFJwM8M/SyBSepCNzQ80U56w9MmUzl0dVCkCmTBIdG9dwzGYsGpGBj6Lp16Ye1u09rrmQqycb/dPDOqBTRnl6mexmtbB4x2HbPJf8MAQqLgmf4UK20b9eOA8MGvpk1cWYoc6VqKtXvQpWPjEAhaqsIXosDiufGGinWNLiKwV1qqBY9752u1JGKrEhO6GlwI0QzqNLFPf3b42bemZG9TQKNzkOLeX+wMBzcPW5GUKq6EbCS0HQoRw/W/5chbZrfR3OKDJyXXYGqiTG45Pbujt+7eBaXZD7+7fGzI0HHJfDC/hKQa3dcyzi58GHa9Gj/VzL35VVLw1bDpx05dp6MeuI8eXIHraXoo+LY7rcYPXmE4uPY7aaNERbkVukpzquFNSWgNDvc3nnho7KEom6acmV8lT2bFEHf4TERz5/dUcMP19sqIYVUpLltVi4jfyLDQY4eqZI1371q1dxzYz194taunJdI5gtV21XifNwbB57KdY8qW36uCJkpuQ2XJAbxqwHLna8nPjoS9uUvX56WIcKZj2Z8l/WTat8/024tRvWPjWowrZW9dKkWn9Osin+yg/4qmU2u5xsUw9Ody5mSDDpZuq0GSspIQ7VwoQKZNWXY6FXpg7cLOr0Rec2rYUf7y4v6Gf2XnGKpIS4Sl5psplbtcpb3HFRS6QkxaNWSiI6Z9hXPVd25O8tDeBExLpV9HjsuE0kJbrqX95YrJVNMQQ7xXM8VNIkHOrZR5xECuqNG7riyi6NUDWKdaRNQ7l+g1oha1L929bH6EvbYP3Tg7HiiYGY5EJcmSz4SkFp8fiQ8gJhMkzrvTCDirTGIWPtnTFD21WKcZOFsoSpioL6YmQP94QRhEQ6qQKt61fDa8O7VqoSDVTM/O10gH40qiTGVwhBkKCbkgb5e0sDaE3dZSq+BQBJ8fIviLZItyEnnY3cdkFzZKvMUIA8VUGDDh1JCQF5nF6nswMZBnqRGKVUhu6jKgbYTlA+SCeQdQCwYHRfx6/pLwUV8v7SDg0woF19V2QJh9GO8xHVArVT6E2aKXM/lRgfh9kPXuy2GHh9eFc8f3VHtKpX2awUrbCikaziRDn1FAuAeu0puFb2kAeWAWQzTwfRygBvN75SUKG8+5fzkCpR2g5Af+cfZFu+8y7pWiYSLWRbbA7FluzkBqmdmoQbujXV/Cyagu9gMeWUVR4Y0NoXJkkAiFfuaZH1nvzM7Rc0d1sEAD5TUFoF52QqXwwYdyndd7wg+k6CSdBZGDB0MVqUK3WsYPTWtJoj0Sj39MtCD5vj2uxE3R1c2SWQ/d7pNjSDDF3W40PbSZGay1cKSgudfa1jJCZIcPdFQa/rcOcm8j/sMnNzz8ywn2kl7OwUw+7GRtC6ewe0q4+ccUOkmFV7HSuZZowil/3LBmRzkjBq4it1Ib+g3rpIdVIrZnSQ1XYuK38SXHiS8AeSdVkV+ODmbJxjsTikESSbX1hDqyuXTUEZNfEVl4ZP1GkXuh05QnYjE5857rpEX3YRal3CaXqGmHj7t6vv6DqerxSU1hMsm4nPaByUC/opqndZELlUv3fxQuiBlwiOSe2s82QnMlki3rqxq6vXl6z7tsaSnMOVtsnmJGE0NYwbMyi9eQplj4fxClqT6lOFlTPze7S/dQGP35cSia/X3G8XvlJQB05U9niTzcRXNcnYaFmiEleVCG3ZhjWcj5PwA1qKXksZSXYrS0tZ9g53xfAHLjeirxSUFrI91ClJxvxSZCrCGIq6bXPGDZEyDZIXiFfNqluQl5llujevjbppSbjrklZui+J53PY69pWC0lJGXjdDyaygvBwjIxNB/TSgXf2ywFgtV14y8emjZkoSlj4+AF2a1Iy+s4TI1GNVTYzHn7IzMPFv7gRs+97N3Ku0rp+GzftP4przMtwWJSy1UmjGJIKgGbpp7RSkKPFP8bImZCNiCsYYXry2s2vXtzSDYoxdxxhbxxgrZYxlh3z2CGNsK2NsE2NsULhzEOWoc101qRVw5WwmcWoWGtGLITjLLynlEUfPHjcGEIRhrJr41gK4GsBc9UbGWDsAwwG0BzAYwDuMMdt9ab3+/GoltpVZB1SnNSchBMPO1Km6tJQ/DQhiA/qZy7GkoDjnGzjnmzQ+GgbgC855Aed8B4CtALpZuZYevL7epMYLX8WN7MZ+JFj0r4TziL9716beXFMhCLPYtQbVGMBC1ftcZVslGGMjAYwEgKZNtbM+6+XxIW0xa+MBS+cgojNj1IVgjHlCiXqB4MCqlANVEuLRuUlN3HVx5ewS10m2HplRiwYotkBTqDKiKijG2AwADTQ+eoxzPincYRrbNJudcz4ewHgAyM7OtvTTtEhPs3K4lGhlaHeLYNqTYG2jnYdOuSmOb+jbph7GALjh/KaIi2OYdFfvSvs0rFFFKgvBmicHGs4rSRBGiaqgOOf9TZw3F4A6E2YGgL0mzhNTVFRG8nRG4ZApJYuXaVyzatjSBunVkpF/oiBi5nM3qCZZ2XTCn9g1BJoMYDhjLJkx1hxAFoDFNl3L18gzf/LGupjfSE8LZIxXly8n/A0lXS7Hqpv5VYyxXAA9AUxljP0KAJzzdQC+ArAewC8A7uKcl1gV1ixeWVyuMH+SUBn0bVOvwntag7CfsrQ91GcRMYglJwnO+fcAvg/z2bMAnrVyflF8clt3HDh+1m0xTCFTx5SdWbvC+zgKJrWd8rxyEt0IhK3I9My7TUxkkkhLTkCaBxwo1Dcmdf0EUL7OR50WEYvEhILyIg8OOgd7j51B71aU7y6WoczcsQcNRsohBSURajNO6/rVMOWePi5KUxma1TlPsM1lCjcgCKcgBUUQDvPzfX1wqqByQUJNlCkUqSf/k5QQh8JiF0poSwxF2kkEDZJjg7YNq1dyOAkHzVpjh5evcy9ruKyQgiIIiSE389ghIS44W6YfOwgpKImQ/baUMTbL75Q3uex3B2EVer4qQwqKICRmaKdGAIDGNeWtC0aIhWbL5ZCCkggZb8zJd1dOXOol6lVLdlsES/y1dyY2PD0YDWpUcVsUwmaClZWpmnI5vlVQddOS3BaBkACvp2NijKFqku21PgkJ6NumHm7plYmnh3VwWxRp8K2beb1qXhxxyjeFqpjdwnsjuzgy7BMeISE+Dk9e0d5tMaTCtzMor9KhcXXUTpVr9teuYXUA3lzEJQVFEN7FtzMoL/DlyB64fnx54WHOIV32CAAoVaZRWp39t3f2Qo2q8t5GpJ8IwrvI27NYRD5jWWW6t6iDVvXSsPXASbdFiUiZgtKYb5/XrJbD0hgjIZ40FEF4Fd+a+LySu2zGqIvw/NUdAcjpxZeUEIdSRS4vmsu8KDNBEAF8q6C8hMxdaOv61TC4fQMAQB3J1sb0QC67BOFdfKugaqYkui2CYWRKcfL0sPZIiGOIY8CoAa2x8okBqJPmvZiieJpBEYRn8e0a1K29m7stgm5kzLd2c89M3NwzE0BAvpop3ps9AYE4IoIgvIlvZ1BeXBynvlQ8CWTiIwjP4lsFRRAAEO/BgQpBEAFIQRG+htagCMK7+FZB9WpZ120RCAkgLz6C8C6+VVBVEinBJuH9ZLEEEcv41otPZibf3RuHThWWvW9aOxVAoBQ4IZZmdVLLXjepTcqKILwEKSgX6JRRs8L7ni3rYOq9F5QlZSXEobbwVUv2XmwcQcQypKAkoX2jGm6L4EtoDYogvItv16AIAqBAXYLwMqSgCF+jnkC1aVjNPUEIgjCMJQXFGPs3Y2wjY2w1Y+x7xlhN1WePMMa2MsY2McYGWReVIIyjzmb+3FUdXZSEIAijWJ1BTQfQgXPeCcBmAI8AAGOsHYDhANoDGAzgHcYY+X0TjqOeQVHoAUF4C0sKinM+jXNerLxdCCBDeT0MwBec8wLO+Q4AWwF0s3ItgjAD1YMiCO8icg3qVgA/K68bA9it+ixX2VYJxthIxthSxtjS/Px8geIQBCkogvAyUd3MGWMzADTQ+OgxzvkkZZ/HABQD+Cx4mMb+msUkOOfjAYwHgOzsbIkKThB+QKtMPUEQ3iCqguKc94/0OWNsBIChAPrx8jrruQCaqHbLALDXrJAEYRZyMycI72LVi28wgH8CuIJzflr10WQAwxljyYyx5gCyACy2ci2CMEOr9DS3RSAIwiRWM0m8BSAZwHRlpLqQc34H53wdY+wrAOsRMP3dxTkvsXgtgjBMk9op6JxRAzmHTkffmSAIqbCkoDjnrSJ89iyAZ62cnyBEMOnuC9wWgSAIE9ASMkEQBCElpKAIgiAIKSEFRRAEQUgJKSiCIAhCSkhBEQRBEFLiOwWVlkw1GAmCIPyA73rzOQ9djCOnCt0WgyAIgrCI7xRU3bRk1E1LdlsMgiAIwiK+M/ERBEEQ/oAUFEEQBCElpKAIgiAIKSEFRRAEQUgJKSiCIAhCSkhBEQRBEFJCCoogCIKQElJQBEEQhJSQgiIIgiCkhHHO3ZahDMZYPoCdFk9TF8BBAeLEOtSOYqB2FAO1oxhkacdmnPP0aDtJpaBEwBhbyjnPdlsOr0PtKAZqRzFQO4rBa+1IJj6CIAhCSkhBEQRBEFLiRwU13m0BfAK1oxioHcVA7SgGT7Wj79agCIIgCH/gxxkUQRAE4QNIQREEQRBS4isFxRgbzBjbxBjbyhgb7bY8bsEY+4gxdoAxtla1rTZjbDpjbIvyv5aynTHG3lDabDVj7FzVMSOU/bcwxkaotp/HGFujHPMGY4xFuoYXYYw1YYzNZoxtYIytY4zdp2yndjQAY6wKY2wxY2yV0o5PKdubM8YWKd/xS8ZYkrI9WXm/Vfk8U3WuR5Ttmxhjg1TbNZ/7cNfwMoyxeMbYCsbYFOW9v9uRc+6LPwDxALYBaAEgCcAqAO3clsultrgQwLkA1qq2vQhgtPJ6NIAXlNeXAfgZAAPQA8AiZXttANuV/7WU17WUzxYD6Kkc8zOASyNdw4t/ABoCOFd5XQ3AZgDtqB0NtyMDkKa8TgSwSGmfrwAMV7a/B+BO5fU/ALynvB4O4EvldTvlmU4G0Fx51uMjPffhruHlPwCjAHwOYEqk7+iXdnS9wQX+cD0B/Kp6/wiAR9yWy8X2yERFBbUJQEPldUMAm5TX/wFwQ+h+AG4A8B/V9v8o2xoC2KjaXrZfuGv44Q/AJAADqB0ttWEKgOUAuiOQzSBB2V727AL4FUBP5XWCsh8LfZ6D+4V77pVjNK/h1T8AGQBmAugLYEqk7+iXdvSTia8xgN2q97nKNiJAfc55HgAo/+sp28O1W6TtuRrbI13D0yjmka4IjP6pHQ2imKVWAjgAYDoCI/WjnPNiZRf1dy9rL+XzYwDqwHj71olwDa/yGoCHAZQq7yN9R1+0o58UFNPYRj700QnXbka3+xLGWBqAbwHczzk/HmlXjW3UjgA45yWc8y4IzAC6AWirtZvyX1Q7+qp9GWNDARzgnC9Tb9bY1Vft6CcFlQugiep9BoC9LtradTIAAAHQSURBVMkiI/sZYw0BQPl/QNkert0ibc/Q2B7pGp6EMZaIgHL6jHP+nbKZ2tEknPOjAOYgsAZVkzGWoHyk/u5l7aV8XgPAYRhv34MRruFFegO4gjGWA+ALBMx8r8Hn7egnBbUEQJbicZKEwMLgZJdlkonJAIIeZCMQWFMJbr9Z8ULrAeCYYlb6FcBAxlgtxYtsIAK25zwAJxhjPRSvs5tDzqV1Dc+hfLcPAWzgnL+i+oja0QCMsXTGWE3ldVUA/QFsADAbwLXKbqHtGPzu1wKYxQOLH5MBDFe805oDyELAyUTzuVeOCXcNz8E5f4RznsE5z0TgO87inP8Zfm9Htxf+BC8iXoaAt9U2AI+5LY+L7TARQB6AIgRGRrchYEueCWCL8r+2si8D8LbSZmsAZKvOcyuArcrfX1XbswGsVY55C+UZSTSv4cU/ABcgYMpYDWCl8ncZtaPhduwEYIXSjmsBPKFsb4FAx7gVwNcAkpXtVZT3W5XPW6jO9ZjSVpugeDwq2zWf+3DX8PofgItR7sXn63akVEcEQRCElPjJxEcQBEH4CFJQBEEQhJSQgiIIgiCkhBQUQRAEISWkoAiCIAgpIQVFEARBSAkpKIIgCEJK/h8WEsfHnDQyGwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "temp = float_data[:, 1]  # 온도(섭씨)\n",
    "plt.plot(range(len(temp)), temp)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "이 그래프에서 온도에 주기성이 있다는 것을 잘 볼 수 있습니다.\n",
    "\n",
    "다음은 기간을 좁혀서 처음 10일간의 온도 데이터를 나타낸 그래프입니다. 10분마다 데이터가 기록되므로 하루에 총 144개의 데이터 포인트가 있습니다:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAAEYCAYAAAAJeGK1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJztnXeYG+W1/7+vurTa1fa+ttfdaxtjY4wpNs10Qg25kAQIBAj5pSc3CcS5ucklhISQAiGEkAIBEkpIKKEbMGDA4ALuve6u7e1Vqy69vz9mRqtdaVVnNCPpfJ7Hj7Wj0ejsrDRnznnP+R7GOQdBEARBaA2d2gYQBEEQRCzIQREEQRCahBwUQRAEoUnIQREEQRCahBwUQRAEoUnIQREEQRCahBwUQRAEoUnIQREEQRCahBwUQRAEoUkMahsQSWVlJZ8yZYraZhAEQRAKsnHjxh7OeVWi/TTloKZMmYINGzaobQZBEAShIIyxw8nsp3iKjzF2PmNsN2NsH2PsNqXfjyAIgsgPFHVQjDE9gN8DuABAC4BrGGMtSr4nQRAEkR8oHUEtAbCPc36Ac+4D8CSASxV+T4IgCCIPUNpBNQBoi/i5XdwWhjF2C2NsA2NsQ3d3t8LmEARBELmC0g6Kxdg2ZgAV5/whzvlizvniqqqERR0EQRBEgaC0g2oH0BTxcyOAowq/J0EQBJEHKO2g1gOYwRhrZoyZAFwN4AWF35MgCILIAxTtg+KcBxhjXwXwGgA9gL9yzrcr+Z4EQRBEfqB4HxTn/GXO+UzO+TTO+Z1Kvx9RWPQ4vfjJf7bD7QuqbQpBEDJDWnyEJuGc46L71mDKbS/hUM9IzH3a+1340fPb8PD7h/D6jo4sW0gQhNKQgyJShnOOHz63FesP9Sn2Ho+uPYztR4cAAP/ZPLauZsfRIezpHMbyu1fj5a2CY9rVMayYLQRBqIOmtPiI3GBPpxOPf9iKjw70YdW3T5f9+K9uO4b/fWF0qbLf5Q8/DgRDuPC+NVGv+aS1X3Y7CIJQF4qgiJTpGPIAAIx6ZT4+j30o6Ej+9QuLManchh6nN/zc5vbBMfveevo0nD+3Fh+3DqBLtIsgiPyAHBSRMiPegGLHdnoDWHewD7csn4qzZtegrMiEfpcP33l6M6bc9hKu/MMHAACTQYfPLG7EbRfMxnfPnwV/MITHP2pVzC6CILIPpfiIlHF6BAfFYumEZMh7e7vhD3KcOasaAFBuM2LHsSF8sL93zH57fnpB+PG0KjsWTy7Dqh2d+PY5M+U3iiAIVSAHRaSM06ucg3prVxeKzQYsnlIGAGipL8Hq3YJG46vfXIZPWgcwudwW9bpzWmrws5d3ob3fhcay6OcBoSS90m6W32iCIBSBUnxESry9uwuPrj0EAGAxpRbj8/SGNvzytV1R27uGPPjsnz7E0xvacU5LTXh967y5tQCA6dV2zK4twTVLJuGU6ZVRrz+nRdjvtF+sxvObjoS3r9nbjX1dw7jr5Z1Y/NM3sPzu1fjSYxuwO6Lqb8fRIfz74/aUfxeCIJSFIigiJb7w8PrwY28gtebYYIjje89sAQDcsmwaHDZj+Ll7Xt8dTuPdvHxqePtxjaVY9a3lqEgQ+TRXFmFeQwm2HRnCN57chBe3HMOF82vxrac2j9mvtc+F1j4XXtveiV9dtQBlRUbc+IgwxXlGdTHmNzpS+p0IglAOclBE2ox4U3NQa/aOjlPpGfGGHVTnkAdPb2iHyaDDB7edFZWGm1FTnNTx//PV09Dt9OLWxzZi1Y5OrNrRCQA4cUoZljSXY2ZNMWwmA/pHfPjev7bgO/8c67xWPrcVz/2/U6HTKZC7JAgiZchBEWkz4kutmu+jg6ONvVKhRSAYwl/eOwgAWHnhnIzWiBhjqC624JlbT8HK57biiXVt+PY5M/H1s2dE7XvxgjpsOzKE376xB9cunQyXL4jv/HMz7n1zL2bXFuPBd/Zj2BvAEzcvRU2JJW2bCIJIH3JQRNq4UoygtrQPhB8Piw7q2r+sw9oDvTipuRzXnzJFFrt0Ooa7rjgOd11x3IT72EwGLGkuxz9uXgoACIU4/vDOftz75t4x+7289RhuOLVZFrsIgkgNKpIgkmb8mpMvGIIvEErqtRsP9+P9fb04dXoFAGDY44c/GMLaA8K607liMYRa6HQMj3/xJHxp+VQ8/sWT8N73z0RNiRmb2wYSv5ggCEWgCIpIGiktF8mINwCTwZTwtS9tOQYA+NpZM/D+vl68tr0Dh3pdAIAvnDIFN8gUPWVCrcOC2y+cE/75uMZSbDkyGOcVBEEoCTmoHCEY4rjl0Q3gAP503WLoVVjId8ZQkBjxBVBWlNhBHe4dwezaYsypLQEAPLfpKKThyteePFmThQnHNTiwakcnhj1+FFuMiV9AEISsUIovB9hwqA/TV76MN3d14a1dXWjtc6lix4tiFBTJDlFxPBGtfS5MKrfBbhl7T+SwGjGlokgW++TmuKZSAMC2I8n9jgSRS6ze3YUPD/Qm3lFFyEHlALc+/jE4H/15f5dTFTt++dru8GOrUQ8AUUUFseCco61fcFDjI78rFjWoEg0mw4JGBww6hte206wpIv+44eH1uPqhD9U2Iy7koHIAk37sBXx3p/qzj5bPrIRJr0OZLXF6r63PDY8/hKlVdgDA/IbRZtgSDafOSm0mnD+vNmoeFUHkOjzyjlfDkIPSOF3DHhwdHB0j4bAa8eo2de7oT4uQGDLodThpanlSvVDbjwqFBnPrhfWnJ29ZiutOngxA+1+UufUO9I74MOzxJ96ZIHKEIbdyEwnkhByUxhnvjD530iRsOzqIvhFf1m1x+0fLzA06hiKTIeHoDW8giMc+PAyjnmFWraAIUWQ24OZlU9FYZsVVi5sUtTlTJonCtG19bpUtIQj5ODqYG59nclAaZ+exYTisRjz4+RPwPxe34Ow51eAcio5bnwi3L9JB6VBkNsSVOxpw+XDtn9fhg/29OHNWNSziuhUANJXb8N73z0JTDGVyLSE5qNa+EZUtIQj56B4eHQKqxs1uspCD0jC+QAivbDuG2bXFOH9eLb54WjPm1jtgM+nx4xe2Y/Xurqza4/YHccq0CtSWWHDz8mbYzfqYpecS//fiDqw71IeaEjP+95K5WbRUPqZVF0HHgE9aqWGXyB88EdmQZCtx1YAclIbZ3D6AAZcfVy5qDG+zGPW449J56Hf5cNPfNqA/i3c/bl8QTWU2fPiDszG7tgRmo37MB308248M4eSpFfjoByvQUGrNmp1yYjMZcMq0Svzx3QNYu1/bJbkEkSyeCAWYg73azQ6Qg9IonHNc9eBaAMCpM8bOP7ryhEY8dO1ioXn3sQ3wB5OTG8oUtz8Iq2k0TWc26OALhmIWOox4A9jX7QwPHsxlfn7lfADAO3u6E+xJELlB5I1l95Anzp7qQg5Ko0TmiGtjqGkvn1mF758/G+sP9ePZj49EPa8Ebl9wzDqSSa8D50AgFO2gNrUNIBjiWDylPCu2KUljmQ0LGh148J392KuBEn+CyBRvRAR131v7NFtNSw5Ko6zZ2wMA+K/FTRM2st56ulAJ95f3DsIXCKFr2INgDGchB4FgCL5gCLaICMpkED4+sQRj1x/qg44BiyaVKmJPtvnmipkAgB88uzVh5SJBaB3vuNT84V511GkSQQ5Ko6w72IcymzGcXooFYww/uWQudncOY/ndq7Hkzjdx86MbFLkbknLWVmO0g/KOc1AuXwC/fWMvQhx5o2F35uxq3Hr6NKw/1I9P3f8eAllKqxKEEkgpvnuvPh5A6rPdsgU5KI2ys2MILfUlYCy+DNDZc2rwuZMmoUPMI7+1qwtv75Z/rcQlfoAtY9aghMfjI6hjYmPx8U35ET1JfPe8WThzVhUOdI+Ex9MTRC7iDYTAmKCWAgAevzZvuMhBaZBgiGN3x3BY+TsRd14+Hxt+uAJbfnyuUIL+n+1hhyIXHp/wAbbFiKDGO6hBt6C68I0Yk2xzGb2O4YHPnQCTQUf6fERO4/EHYTHoYRG/w/GqcdWEHJQGOdgzAm8ghNl1yTkoAKi0m1FiMeKHF7XgcK8Lm2Tu23H5BYdnjbEGNX6Q4ZDooEqs+ZHei8Rq0uP8ubX4+0et+PWqPZpdXCa0yerdXbhUAylijz8Es1EXLnoiB0Ukza4OoXFuTl1xyq89b24NAGC7zM13kopE5BqU3Sw8jmzW/bi1H79ZtQeAoBuYj9x5+Tyc1FyO+97ci4+pgZdIgbte3onN7YPYpnJzrDcgRFDSDSel+Iik4Jxj57EhGHQM06vtKb++wm5GncMSFmiVC0mHLzKCqigyAwB6nUKz8LqDfbjigQ+wuV147zpHdHl8PlBsMeKBzy0CAHx8uF9la4hcwmYS5qGpKT7cNezB0xvaMezxwyKuI7s1GkHRRF0NsXpXF254ZD0AQflbKkJIlbn1JVmJoCqLBQfV4/Ti2KAbX3psw5jXFJnz9+NVYTejqtiMPdQXRaSA1AYSCKqXGv7Jf3YAAEZ8QZiNsdP0WoEiKA3xr4/bAQA6BvzPxS1pH6el3oH93U4MuuS7S5PusGxjIiihAqjH6cUrWzvQ7/Jjdm0xzphVhbf/+wzZ3lurNJZZc0YVuhDxBUJ4O8t6lYmQmtpjNbdni63to9kVk37iXkYtQA5KZg71jISr2BIx4g3gXVE+xxcI4cMDfbhofh02/PAcLJ1akbYNZ8yqQogDq3Z2pn2M8bjECCpSScJi1KPYYkCP04c3d3ViWlURXv3mcjxywxJMqdTmGHc5mVxuw/v7etHWp80mx0Ln96v34QsPr8f7+3rUNiVMMCQ4AjWLJCKb+eM122sBclAywjnHGfe8jcsfeD+8zR8M4der9sRUDP7eM1tw3V/XYV/XMF7f0YEepxefPqER5UWJp9TGY0FjKUwGHXZ3yJfm88RYgwKAKrsZ6w724f19vVgxp0a298sFzhZ/35XPbVPZEiIWbf3CjcPBnhF0DwtpaLWRUnt+FSOoECcHVZD0iMUCB7pH8KPnt4Fzjnte24373tyLC+9bg7te2Ykn17WG95eipxW/fhdf/ccnaCi1YvnMqozt0OsYfIEQ/rTmoGxq59IalG2cg2oqt2HHMcERXr6oQZb3yhU+taAeZ82uxvqDfZot0y1kzOLFt2/EhxPvfAMn3/WWyhYBB3oE5XApkso2/SO+cCM9IAweZQzwaVQZhRyUjDz24eHw40fXHsa3ntqEP757AICwrvTHdw7gtn9vRSjEsbltAMPjNN1WXjRnQt29VPnsSZMAAA9/cEiW44VTfOMKN6ZUjA4cnKTx4YNKcOOpzXD7g6R0rkF8ASFS+MdHrQn2zA6Rab0j/epEc9/55+bw499dsxCMMZj0OoqgCoHXx6kLPLfpKADgpa+fhisiZjp1DHnwSatQnrxu5dm4aH4d7rlqAS6cXyebLT+7fD7OnFWF+97ci04Z5PQ9/iDMBh104xzo5IrRtSaphLaQOGlqOcpsRjz3SXYU5YnkkdRUeke8CfbMDtNXvhJ+fM/re/De3uyvjR0dEBzj2bOr8akF9QCENN94PU2tQA5KJjz+IPZ0DuOrZ07HpcfXh7ffcelczK13YOWFc3DNEiGqeXTtYfxYLPWsLDLj959bhE+f0BjzuJkgvd8PZVgjcfmCUek9AJhSWXhRUyRGvQ6XL2zEqh2dsstLEZkhRf1+FUu64/HmLvmKmJJFqsa1xJjrpkXIQcnEns5hhLjQg/Tb/zoeK+ZUAwBOmS4MGywrMuGnl82D2aDDg+/sD79ufEQiJ+fOrcXpM6uwUYZmUrc/OKYHSqK5UmgmlhQsCpFlMysRCHEaC68x3L4gDOO+X1paK2xVYcTFFDHjcU5EQVNBpvgYYz9mjB1hjG0S/12o1HupRWuvCzuPDWHj4T5ccr9QuScpkD/4+ROw8YcrMK1qVA1Cr2P4eoSA6hM3L1XcxtOmV6JvxJdxT5TbHxxz1yXRXFmEf9x0Eu67ZmFGx89lFk8ug0mvw0tbj6ltChGByx8Y8/0DgGGPOlFuLM3GHpkKmFJhapXgoC5bOFrQZDJo10EpvWjwG875PQq/h2qcfs9qjP/cSYUCBr0OFXZz1GsWNI6OoFjQ5FDUPgBoKrcCEEpuHbb03889QYoPGI0SC5ViixHLZlRi/cE+tU0hInD5gsI06ohM2gf7e3Dp8dmvNo3VmKtXLnkyId5ACJX2sW0sWnZQlOJLkzte3BHlnN749vKE85uOa3LAZtJjQVNpVooKqiLkiDLB7Yud4iME5tSV4EDPiGYlYwoRty8Y1VP4jSc3YfWu7KtL+GOs8egSXCuUwOsPRUmomTS8BqX0FfKrjLHrAGwA8B3OedRiCGPsFgC3AMCkSZMUNkce+kZ8+Mt7BwEA1yxpwoXz61BmM2F6dWL18RKLEWtvPxv2LOnUlYkDyfpdmaUT3P4gii2FV6WXLHPqShAMcWxuG8SS5nK1zSEgKLXEanrP9GYtHfyB6AhKjWZdb2BUf08ib9egGGNvMMa2xfh3KYA/AJgG4HgAxwD8KtYxOOcPcc4Xc84XV1Vl3qSqJN3DXqw72IdFd6wCACydWo4fXtSCZTOqMK8h+fSZw2qUrd8pEZLi+PYjQwhl8IWIl+IjgNNnVUGvY1izl/qhtILbH4yZpVBjDIw3GB1Ze1Uo2PBMFEFp1EFldEvMOV+RzH6MsT8BeDGT91Kbqx78AOsPjQaAp06vwN9vUr7IIVNKrMKf+M/vHYTNpMe3z52V1nEmquIjBOxmA2ZU28OjRgh18QdD8Ad5zJsqNdJZkaXukkNQwyl4A8GwwsaoPfqk9UOzjZJVfJFdp5cDyFnBsl6nd4xzApATzgnAmDWxf32cfjOpyxeM0uEjxrKgsRRb2wdoyq4GcEVIc333vFm48dTm8HNeFYbz+SOc0aM3LsGnT2hUpTnWGwhFOygNp/iUXFS4mzF2PAAO4BCALyn4Xopyz+u7AQDXnTwZNSWWcOFBrmAx6uDxh8Lzm9LB4w/CaqQ1qHgsaCrFUxvasPPYMFrqS9Q2p6AZ1Y404KZlwtr29adMxum/fBseFQpZIqM2u9kAs0GnSkGNNxBC6bgUp9mgg0+jxT2KRVCc82s55/M558dxzi/hnOdsk8gT69oAAN85Zxa+cuZ0fGZxk8oWpcZLX18Gm0mf9l0S51xI8Zmo6DMeF8yrhdWoxxPrtKH9VshIqh6RKT5p7UmNCCryu2cz6QV5IRXs8PqjU3xmI0kd5Ry+QAivbjuG1eLAs3NbauCwZX9xVQ6mVdlxzZJJ2Ns5nJa6uS8YQjDEaQ0qAWVFJsxvdGBzOylKqI2U4otMS0uzzNSIoCLLzI16HcwGPbwqrIV5AyGYx32P7WYDnF5tynSRg5qAX7y6C7c+/jFueFgYwf7tc2eqbFFmLJshyPHs7XKm/FqnR7obpRRfIk5qLseW9sGwKCehDq4Y42Gk6bEeNdagIookah0WMa0Wyvp6pdsXHUEVWwwY9gQ0uXZKDmoC3oiYRvurqxZgdm1uryk0ixNuD/WOpPzaI+LFtqHMKqtN+Yg0z2t3x7DKlhQ2oym+0ZsqnY6Jyt0qrEGJKbR/3nqyEEGJvUjZTK11DXnQMeSJGotTbDEiGOKqOO5EkINKghUtuS+E2lBqhUHH8Oc1B/Cp370Xs7N9Io4OeMLHIOIzU2zW3kUOSlUmGrBpUWntR/q+GcUoTupFyqaDkgaqzqwZq08oiQYMe7RXak4OagL6R3y46oRG7P7p+ao09smNQa9DY5kVezqd2HpkEOsP9WHnseRGwg+6hQ92aY6uwWUTh82IaVVFqsjpEKPESvEBgNmoVyWCuuERYanAKArwSWm2bNri9sdO1UsKMeMHqGoBclAxCARDGPIE0FBmjeq6zmUihwt+9k8f4YJ71ySlci418eWDo84GK1pq8ElbP+nyqcDHrf1Ys7cbLn90kQQw2nKhFpJjMkkOKou2jHiFc1JkHntOwg5KJaX3eJCDisGAeEGOpeOVy0SOZ5d4fUdHjD3HMuj2Q8eQNf3AXGd2bTH8QY62PiqUyCa+QAhXPPABrv3LOrhjrEEBQmot2zOhxlfwCXZkfw0q1rocIKxBAZTiyxmkUuxSW345qMgISmJPZ+K1kl6nDxV2c0KldkJAOs+H0yhIIdLnQM9ohWq4zNwYHUFlu+cnUkZo/BpUNhUcwhHURCk+iqByA2kxsTzPHNT4FF1tiQWHkpjq2TnkQU1JbqlnqIk0tTSZc0vIh+SUAODYgAfFFkOUKLNFhQjK5R19v0pxRtxoFV/2bAlHUONSfDXFFgDAsUFP1mxJFnJQMdjUJjRazqpNPD4jl1g2oxKlNiP+duMSbPrROZjf6EjqLr/H6UNVjOGLRGzKbEZUF5tpgGGW8UQ4qA8P9o4ZDiphNuqy7qAkmaN7rz4+vPakRopvZILCkVKbEXazAW192ruhIgcVg48O9mJaVVHOae4lorrEgk0/Ohenz6xCqc2EeoclqbumfpcvPFeKSAxjDBfMq8Wr2zuwhVQlsoY7wvG09blQU2KJ2sdi0Gc9xSetQUmNwoBKa1DeABgTzkEkjDE0llnR3k8OSvNwzrGpbQAnTC5T2xTFcViNcHoDCedEDbj8ebcepzRXiXqNf/vgsMqWFA6RKb4QByrs0Z9ZNSKo8T1QQEQfVBZtGfEFYTPqoYsxi66p3KbJoh5yUONo63NjwOXHgqbo9EC+UWI1gnPA6Zt4cdQXCMHpDaCMeqBSYl6DA0umlKO1jwolsoV73MX+9JnRA1DVjKCMBnUjqBFvANYJ5Mqaymxo7XNpTu6IHBSEaquLf7cGL245ik1iSiZW/jrfKBHLS+P1QkkVSNSkmzp1pRZ0DGlv4TlfGR8ZnTq9Mmofs1Gf9T4onzju3RQjgspmFZ+QCYn9PW4qt8LtD6I3DTFpJSEHBeC17R3YdmQIv3h1F7a0DcBk0OVdgUQsahyJq3cGXPlZcp8Nakos6Bryqm1GweD2JU6XmQ3qpfhMhtHUmhpafL0jXlTGSHsCCOvztWqsUIIcFIDdHUL/xLEBD17eegzHN5WOyRfnK5PFD2W8Sr5+MbqiIonUcViN8AZCWb8gFipSik/HgBe/dlrMfcqLTHB6A1kt7469BpX9MnOpnzEWc+oEMex393RnzZ5kyP+rcBLs7RKaVQMhjqODnpi563ykocwKvY7FvWvqd5EOX7qUiH1nQ27tdejnI25xGN+Buy7CvAZHzH1qxcq+bEa2sRyUSYU1qB6nF5UTqOPUl1rRUGrFYY317pGDgtCIetH8uvDP0miKfMeo16G+NH6zrrQ+RQ4qdaTG6EFyUGMIhjgeW3sIB7pTn00WD7cvGKW9N55qseG8M4trgz5xFtQYB6XPrhafNxDEkCcwYQQFCHOqOjTWrFvwDqq114XOIS8mVdhw46nNAIBFk/K/xFxiSkURWuOm+IQIilJ8qSM1N1OhxFh+9fpu/M/z23HnSztlPa7bF0w49blWXHftzGYEFYjugzLodTDoWNZSfH1i8UOs0nuJ2hJLVh13MuSlg/rNqj348uMbk9p3+S9XAwBqis1YedEcvPPdM8If4kKgscyG9v6J+x/6XX6Y9Lqo7nMiMdLcnT2d8kYKuYzHH8QDb+8HAOzpkndmlsuf2EFJsj7ZvGkYLTMf239kNmRPF7BXlG+rjBNB1ZQIVadaKjXPS3nqe9/cC0Ao4TQZkvPBdaXCekwsQdV8ps5hQe+ID95AMOZokUG3Dw6bkYRi06DCbkZFkQm7O5Kbu1UIvLz1WPix3NdBpycQFj6diFKbESaDDl1qOKhxhVdmoz5rZeY9TiFinKiKDwBqHWa4fEEMewPhFhS1ycsISkJKT02EVJZaaTfh7NnV2TBJc0jR4p6O2Hf5/SN+atLNgAVNpVhHmnxhuodHU2vt/W5ZFd+d3gDsCRwUYww1JeasRlCx1qAAIeWXrRTf6Ey3iR2UJA3VqaF1qLx2UFJYOxGS9tT/XNwCQwGUlcdCqmr61P3vxSwx7XP5qAcqA5bPqMShXhdaNVYdpRZSuvOrZ04HAHzpseRS8ckw7PEnNbOspji7ay2xtPgAQbR1JIneLTmQbsbjFZFI1wItrZnm1VV525FBLL97dfjnoQQDuHaII8+nVdkVtUvL1EWst726PXp4Ya/TS0rmGXBiczkA4JO2fpUt0Qb/+rgdAPCl06cCkLdiVkjxJY72axzZbaCW0njSuHeJWocFxwayo38n9eLFW6OTsilaquTLKwdl0I/t6XHGGcA16PLjG09uAjDapFaIRBaExCr77XH64lb+EPGZWVMMi1GHLe2DapuiGWwmPYotRsyolvfGcNgbSDqCymYxgD8YAmOImk3VUGqNW6AkJ26xnN1inPiSL6X4tNQLlVcOavzC3rB34ghqw2FhXWDFnOqoD04hEXnH2TdOh8sXCGHQ7Y9b+UPEx6jXYWqlHQd7SDRWuov/ipjec1iNGIijA5kKnHM4vYmLJIDRYgCnNzsTZH3BEIx6XVShUUOZFV3D3qysQ0nnfvyojUgsRj3mNziw8bB2ov28clDjP5zxIigpF/6rzxyvqE25wFO3LMWyGZVRDaWSwyIHlRn1pVYczVIqR8u8t7cHwGh/mMNqlK2JecQXBOdIKoKSdOe2HclOdaU/wKPWnwBgcoVgRzZuXjyiykasURuRVNpNGIkz3SDb5JWDKhonJT8c5w5pb9cwakrMUWPQC5GTplZgdm0xhtyj58sfDOH5TUcAxG/uIxJTX2ohBwXgpkc3ABhNKztsRuw4NoSH3t2f8bGlm9Fk1qCWzaiCXsewZm92dOf8wVDU+hMALGwSBAGe/fiI4jZ4/EFYEvSIAYDNbMBIliLLZMgrBxV5d2DUMwzHiaD2dTkxozr/FcuTpdRmgtsfDFf7PLGuFXe9sgsAMKe2cNfo5KDOYcWQJ6CpL362OTY46qCXiIUj0g3lz17eheEEBU2JcIrp/ERl5gBQZDbgxCllY3qylMQvpvjGI0VQf3z3APoVHnMx4guiKIlm+yKTfszgR7XJKwcFAP/68sl46palKLYY8fr2Dnx4oDdqn1CICw6qpnCr98YzRWxQ/so/PgYAHB0QKnkjpxTXAAAgAElEQVQay6yYJH6RiPSodWRf/01rSOm9l7++LHwnH7n2m2maS7oZLU4ixQcAS6aUo63fHS4BVxJfMLZgQOSaVOewsp8Nly8AWxLnxmaiCEpRTphcjpOmVsBuNmB/9wiufujDMc2BAHBsyAOXL4jpMlcR5TLLZgrD3drEKkh/MASbSY/3vn+WmmblBeVFgoMaX4RSSOw4NoQikx5z6kazFt86ZyYWTxbSXJmuRUkOKpkIChBGnAdDPCupV38w9hoUADz8hRMBIOoaJTeuZCMos15cz9OG3FHeOSiJYGj0BH/zqU/GVMr0iQ281cWFo7mXiBKLEZ9fOgndoiTKgMuPUlqfk4UKccRBT4LG8Xyme9iL6hLLmKjBYTXizsvnA8CY9c90kCrykqniA0YLJbJRUu0PxE7xAcIkW0D5mxeXNwjbBOPeI7GZDAiGeFbHgMQjbx3UOS014cfv7+vFn9ccBCB0m3/q/vcACHcLxCi1JRYMuPzw+INwev1J340S8ZGKTAo5gupxxp7mKtdIEknWLNmip/pSwTFEro0phT8YihKKlZCKOuKtl8vBiC+Q1PVOirK0sg6Vtw7q++fPxoe3n42fXDIXAHCgW8hxR94xJVOSWkjUOoQvbcegkAJN5o6LSEx5keSgCnf8e/ewN2a7glwOqmPQA72OJZ0VkeZCZUNRwjdBkQQwGvEp7aCS/T5L61QHup34+Su7srJGF4+8dVBWkx61DguuP2UKVsypxub2AQBj7wzIQY0lUovL7QvSiA2ZMBv0sJsNBZ3i63H6Yjooi1EHo54llCVLRHu/GzXF5qSb7s0GPRxWYzilrSQTVfEBgvSQXscyrmJMxIg32QhKqqzciQff2Y9Xt0XLn2WTvHVQkTSW2cIKvQMRCueUwhpLpBaXixyUrFSXmNGlcKWWVvEGghh0+1FVHO2gGGOyNOzu7RrG9JrU2kYcViOGsjDtOF6RBGMMxRaD4qoWyUZQ0jXRLyqwq115WhAOqs5hwbA3gNe2d2Ag4gNZUUQKCZFIwrFHBtwY8QVgpRSfbDSUWnEkS7prWiPRsLwSixH/+KgV33tmc0bvURPDAcaj2GJQPLUGTNyoKzHg8uPRtYex8bAyY1k458IaVBI3nFJBz4Bb+JupXSxREA5Kigy+9NjGMRFUIWvwxaLIbECl3YRfvrYbh3tdsCXReU4kR0OpFUcGCjOC+oM4Qbd0grliPnGd4+kN7Wm/x7An8Syo8WTLQfniVPFFsnqXMsoWkgxUMjecUkFPW59wMyWXFFW6FISDmhIxJfdnLwvqCD+6uEUtczRNg1jdBAA2qnKUjfpSK3qc3rBoZyFxROw1WjylLObzmTaGBkOSUGxqbRElFvm0ACfC4w/iUO8IypKYqWaIE2VlwqZWYf19dm3iFKhU0COhtMJFIgrCQS1oKsWXlk8ds+3G05pVskbbRH7JaQ1KPiTHX4iafCa9DjOq7RNW2GXaEiqJm5akGEHVl1rR3u9StCm1c8gDjz8UngsWi08tqAegXGm3VILfUGZNsKdQPBLJQC5HUIyxqxhj2xljIcbY4nHP3c4Y28cY280YOy8zMzPnGytmqG1CzkFl5vIhXRyOFKCD6hz2hGcNxSKyqT4dwjJHKTqoxjIrRnxBRaOoXjECqSiaOIL63TULUWozKhZdh6fpppGyj1wSUYNMI6htAK4A8G7kRsZYC4CrAcwFcD6ABxhjqt6O20yGsE7XIzecqKYpmiaymVTHaI1OLiTlgkMFOBeqa8iL6jgFDOkGMC5fAD96fht2HBXGZtjNqaX4pKpCJcv/pRRZWRwHBQhzmpRyUC4xwkw2I/J/l87FL66cjwvm1aJfpnld6ZKRg+Kc7+Sc747x1KUAnuSceznnBwHsA7Akk/eSg+OaHACoei8ePxYbmwl5qXNYUGTSY393YTkozjm6hj2ojhNBpZtiW3+oH4+uPYybxTEeqUZQUlVhr4K9UCNi9GJPsJ5rNenh8StTMecSHV9Rkn2f1508Bf914iSh/D0LRSTxUGoNqgFAW8TP7eI2Vbn36oW447J5mFtP4yMmYklzOb573iwAwExSe5cNxhimVduxr8uptilZ5d29PfAHedwUV7oZvp5xAqupOiipYk3JCCo8yTZBes1s0Cma4mNMeI9UsJuN6BjyYPXuLkXsSoaEFjPG3mCMbYvx79J4L4uxLebHkDF2C2NsA2NsQ3e3sgPEKu1mXLt0csKpkoXOjac245EbTsTZc2oS70wkzfSqwnNQX3/iEwBAII4XktpAgNSiqZ5xkY+URk0WKZPSq6AEVbIOymLUw61Yii8Im1EfNXI+EZLyxA0Pr1fCrKRI6KA45ys45/Ni/Hs+zsvaATRF/NwI4OgEx3+Ic76Yc764qqoqNesJRbCa9DhjVrXaZuQd02vs6BjyYFDlvH42WTipFADw+aWTJtzn3quPDw/vSyXN1eP0wmLU4a3vnI6/XL8YFRM0Ak9EeZEJjGkjgrKZ9OFiBrlx+YJpNd2HNDByQ6kU3wsArmaMmRljzQBmAFin0HsRRE4wv0FYA916ZFBlS7JH15AXZ82ujtujdFxjKW4S2z6kkvFkkPT9plbZ04r29TqGcptJ0TUoyeFaEqTXymwm9ClUMef2BdJqGamP6IkMqCQam2mZ+eWMsXYAJwN4iTH2GgBwzrcDeBrADgCvAvgK57zwOhQJIoLjGoRo4o2dnSpbkj3a+l1oTKL/RmppcHmTv0wIIzwyK3iqsJuwt8up2AXY4w/CqGcwJFCSKC8yKTaOJV1dzatPnIQFTcJn9tigB2/t6sQ9r8WqiVOOTKv4nuWcN3LOzZzzGs75eRHP3ck5n8Y5n8U5fyVzUwkit3HYjLhmSRMe+eAQdh4bUtscxRl0+zHsCSTloKT1jiGPHwMuX1JrUcIIj8QKDfEIBDnWHezDfW/ty+g4EzHsCSTVT1hWZMKAy59xT1gs3P70HJRex/A9sWCqrd+FGx/ZgPtXK3OeJqIglCQIQit85czpAIANh5QRBtUSHeIEgTpH8hHUxb97D8f/3yr8/JVdCV8z0QiPVJA08h5+72BGx5mIowPusAhzPKQydFcKKc5kyWS2W3iwY4SOZDbHwZODIogs0lBqRZnNWBDrUE6vUAxSksSU2/Gziv747oG4+wdDHH0jmaf4/vD5RSgvMmHYG0D3sPxrUV3D3rgqGhJSk/EPnt0muw0efxAWY3qX+sgZcRKPf9Qqi13JQA6KILIIYwxz6x3YeWxY9mPv7pD/mJkgSRAlMxjUakztDr/f5UOII+MU39QqO+6/ZiEAYG+n/OfPG0guvSY56P9sjlnsnBEefzBKYy9ZrCY9rEb9GMmjD/b1yGVaQshBEUSWmVEj9EPJmSp5cl0rzvvtu3hvb/YuHomQhvAl00A7vXpsU/icuvjN9OEZUynOgIqFNP5diem63kAIpiQaZCOduNwFG95ACOY0IyhAUKBwRhSvnBRH+FZuyEERRJZpKrPB7Q9iQMZ+qNd3CJWBb+1Sr+t/PM4UIiiTQReeOLCgqRTuBGsxUpNupik+AKgSVdbbFRgo6fWHklJwiOzhkrth1+MPpR1BAUJ0J62ZnjC5DNedPEUmyxJDDoogsoy08CynsrlBVEf56/sHs7qIHQ8pgkp2kOC3zpmJVd9ajjm1xQlHT1z7l48AyOOgHFYjZtcWY+3+3oyPNR5fMDnnMDlCBUNuB+UNpL8GBQgq6HtFBZTLFjZkVYmHHBRBZJkGBRxUZBpJ7SmoEtIaVFGSFWQWox4zaophTaCq0DXsCev3Sem5TJnf4MAuBdbwvP5gUim+siITVswR1Fs8PplTfBlGUL6IlGOqM7cyhRwUQWSZ+lIhpSTn8EJTRCNolwLVaOng9AZQZNJDn+Idt82kh8sfnDASXCWmM20mPUpSnKI7EXWlVvSOeGXvQxIiqOQus1csagQAuPzylZqHQhy+YCijCCrys5VMRaackIMiiCxTXmSCxaiT1UEZIy4iQxqJoJyeQNLpvUhsJgOCIQ5vIHYkcWzAA72OYeuP5ZuDWlFkAuej02flIBTi8Ad50tGLNFBQTk0+6RxmEkFF6nJSBEUQeQ5jDPWlVllTfEbDaJSi1OjwVHF6A0kVSIwn1oWac47frNqD3R3D6BjyoMpuTjkyi4e0lvXzV3YhJFMU5QkI9idbQScJysq5BuUNSGK16V/qv3veLFyzpAlVxWbMqs3uqCJyUAShAg2lVhyJ6M7PlMjKZK04qGFvAPY0UnBST9D+bid+8p/t8AdDcPmCuPfNvbjqwQ/QOeRBTRLqDKmwQBxm+szGdrwgUy/SkFtI1SWbhpT6pX79+h7ZomtJrDaTCEqvY7jriuOwfuWKtG44MoEcFEGoQL3DKmuKLxga9VBuGdcwMsHp8aM4nQhKLKr49INr8fD7h/BJ60DY6Q55AugY9KBWpuIIicYyG1782mkosxmxRqZeMqlYxZHkuo1VdFAbDvfjwvvWyGLD6LiP3LzU56bVBJHj1Jda0T3sDadgMiUQ4uHFbK1EUJmm+CR6nN4x6b6OIU9YgkdO5jU4UFNiwbBHnjU8yUGVWJM7B5G/t1w9cnKsQakJOSiCUIGGsmgRzkwIhnhYseHNnV2KyPakwtr9vdjT6Ux5DDsALJtROebn3hHfmMq2YU9A9hSfhN1sCPdvZcpQihHU+KGGcijeUwRFEETKyF1qHohwUG/t6sI5v3lXluOmQyjEcc2fPgSApJS8x2Mx6sMTdgHhQj8ybk6UEhEUIDQVS/1bmZJqiq+8aKyu4AX3rsm46VoSwKUIiiCIpJG7WTcgKhbIWdmWLpHK1w1JzIKKRWRhwZDbHxXVKOWgii1G2SKoVB2UXsfwu2sWYtGk0vC2TBXW/7mxDcVmA+bWZ7f6Ti7IQRGECtQ6pAhKvhSfXscUGXiXKpKQKwBUp+lIVl40B1cuakSxxYAhjx9dQ2PPU11peo4vEXazfBHUmr3d4WMmy6cW1OOs2aN9R5newPSP+DGvwYGyosxU39WCHBRBqIDZoEd1sRlt/S5ZjhcIcRj1Y6MntTT5ekZG7/qbK4rSOsbSqRX41WcWoLrYjCF3IEodY1KEdp2clFgMshVJDLj9mFpZlHDc+3h8wdG/W6Y3MN5gcmrqWiV3LSeIHGdmTTF2dcgz+j0Y4lEinhMpMSjNiJgie+LmpZhSmZ6DkiixGjHk8WPI7YfZoMM73z0Df7txiWKpTLvZAG8gBJ8M587jD2FqlT3xjuOYGnHO1h/qy+hGw+sPJi21pEVy13KCyHHm1pdgT4cTfhnm/3gDgubbBfNqw9vUKjf3is2hUiFIJvSP+LBmb4+g62c2YHJFEU6fWZXxcSdCKjSRI4rypjnJ9tLj6/HYF5cAAB754BD++v6htG3wUQRFEEQ6tNSXwBcMYX+3M+NjCQ5Kjwc+twh3XTEfwGgkk23k7L3xi+muXR3DUWPhlUAagNjjzFyTz+0PRvV0JQNjDMtmVOHi4+oAAHe8uCNtG3xJDkzUKrlrOUHkOC3i1NjtRzJP80mpHMZYuAJOrQjKJzYfy3FhvOOyuQCEcezJju3IhJoS+cr/Pf5gVG9TKtxz1YLwY1eCAY4TId245CrkoAhCJaZW2WEx6rBDhoZMXyAEs3gxtImRxkiaF7VMGY2gMr+8NFcKazhDHiHFpzQza4pRajPib2sPZXQczjlGvJkNCrQY9fjdNQsBAN98clNax/AFkh/3oUVy13KCyHH0OoZZtSXYfnQw42N5Iy5EUqQh59iGVPDJ6KAiG32T7SfKBIfViBVzarA7w+GFv3p9D3zBEPS6zM5Bs1gw8bo4AytVvIHkBiZqldy1nCDygLn1JdhxdCjjknBvYLRaS1LFVnMNSq9jKZdXxyIyRVZqy86wvDqHBV3DXgQyKF65f/U+AJmX+jdnWAXpC4TGDBzMNXLXcoLIA+Y3ODDkCWRcKOH1h8IXc6kxdEimhtOUbQkEZb0o/vozwlqMXA20iah1WBAMcVkKJTKlyGzAdSdPBoCUHWYgGEKIyxPJqkXuWk4QecCp0wRh1Pf39aZ9DM453BH9LnWlFjAGbDjUJ4uNqSKsh8l3abns+AZ89czp+OqZ02U7ZjyktOKxwcwLJeRIr80Ri2k6U5Q9ktYCKcVHEERaTKqwobHMig/2pz+DyBsIIRDi4fHqZoMetSUWPLm+DQMyjjBPxR4579p1Oob/Pm8WFjSVJt5ZBprKBJWKgz0jaR9jfoMwAPGW5VMztkcSke0fSe1v6SMHRRBEpqyYU4O3dnWlfccupb6KIwRWv3/+bADyaf2lgjfHe2+aK4tgNugyGneh0zEsn1k15m+SLtKaYqqj4HN9FhRADoogVOeLpzUjGOJ4cl1bWq+X1Lcjp9c2lQtiqp3D2XdQvhzvvTHodZhdW5xR+b9QnCCPHJPU7JtqVSZFUARBZExTuQ3HNZbiiXWtaE9DPFaS5YlUzS61CWkhaWheNomsKMxVWjKsrvTJWN4tjYJ/ZVtHeABhUjYE5WuYVovctZwg8oifXjYPAy4//vD2/pRf6wyn+EYdlBRNZavyLZJcT/EBgspHv8s/ZrZVKviDXLZKRimCemJdK376UvKyRx6/fP1oapG7lhNEHjGvwYFz59bgpa3H0ONMrVpLKie3Rzgo6bFcw/dSQe4iCTVoqc9MhkpODTxbhMTT4x+2Jt3fJqeih1rkruUEkWfcvGwqnJ5AyuKgo2tQowvyVqMeOjYaXWUTIYLK3TUoAJhdWwKjnuHFLUfTer2cKuIV9rHDBp9an9xapaTfZ8uChqFSkIMiCI2woKkUn1pQj/f29qS09uEU16AiU3yMMdjNBlUiKLcvAFsGIqlaoMhswOkzq/FJ20BarxeKJOQ5B8ZxqcJtSUpjSWLBUhVgLkIOiiA0xNKp5egd8WFfV/LKEsMxUnyAUHauxhrUiDcYZUsu0lJXjLY+V1qSR0rNYVo4qRSdSa6LuclBEQQhJyc1VwAAfvHq7qRf4/QGYDHqou60hQgq+1V8wx7/mIrCXKWu1IoQT13BgXMua5k5ADx1y1Lccdk8VBebsbfTiQ8PJFYeGY2gcvdvQQ6KIDTE5ApBxeCNnZ3oSrKHacgTgN0c3RBqt2Q/xcc5x4gvmJXhgkoj/S32dKambC4NWZQzgjppagWuXToZDqsRXcNeXP3QhwmjO2kNykoRFEEQcsAYwx2XzQMAfHw48fpHKMTxxLrWmM/ZzYasp/g8/hCCIZ6V2U1Ks7CpDA6rEb9ZtSelNJ8vqFyDbKSA7bK7V+PWxz+ecF9agyIIQnauOqERRj3DJ239Cffd2CrsE6s0vaLIhN4sK3L3idp/5TZTgj21j9Wkx08umYst7YNYvbs76deFFRwUGHMhHVvijZ0Tz4ly+QRV+fGp31widy0niDzFYtSjpd6BTa2JIyhJ0PT+zy6Meq7CbkKP05vxTKJU6BMdoiRwmutcdFwdyotMeHnrsaRf4w9HUPJHLr+86jjMFXu0JMY7LQmXL5DT6T0gQwfFGLuKMbadMRZijC2O2D6FMeZmjG0S/z2YuakEUTgsbCrFlvbBuKmlQbcf33tmCwDggnl1Uc9X2s3wBkIYyeJk3d4RIZIb37uTqxj1OiyaVIatR5KfeqykBl6dw4qXvr4Mf7puMb5wyhQAwKNrD8Xc1+ULoqiQHRSAbQCuAPBujOf2c86PF//dmuH7EERB0VJfArc/iPb+iRXOJd2+Jc3l0OuiK8Yq7WYASCoSk4u+ESmCMmftPZVmWnURWntdCIaSi0QlBQejjFV84zmnpQafXzoJAPDTl3bC7QtiV8eo6sU/N7ThmY3thR1Bcc53cs6Tr4clCCIpplUJo74P9EzcDzUoCsF+a8XMmM9LlXTXP7xOZusmZtRB5UcEBQjzoXzBED5uTbwmCIxGUEpLDE2rsoejtJsf3YDzf7smLCb7XTGyzuUSc0DZNahmxtgnjLF3GGPLFHwfgsg7plbaAQAHuqOH5vU4vVi7vxf/+/x2AECpLfbMoZPFab1V9uxFM70jPhj1DCV50KgrUSmmK696cG1S+0s3DiUyzIKKB2MM3ztvFgDgvX3CwMvxIzlyXbQ34aeIMfYGgNoYT63knD8/wcuOAZjEOe9ljJ0A4DnG2FzOeZTyImPsFgC3AMCkSZOSt5wg8pgyMQL5xau7cNOysVNZb3xkPba0j66J1JdaYx7DYTXi3JYatPalPsIjXfqcPpTZTGBMufRWtilLsSJRmmJcmoVKxvGR6tMb2jC1yh7+2RvI3vqjEiR0UJzzFakelHPuBeAVH29kjO0HMBPAhhj7PgTgIQBYvHhx9sqNCCIH8Ac5th8dxNx6R3jb/ggZpFtPnwaHdeI7dYtRn9IMoUzpHfHlVXoPAKpLLOHHnPOEzlcqtS8rUjaCAqId1F2v7BrzcyCY25dUReI/xlgVY0wvPp4KYAaAA0q8F0HkK/+46SQAwK2PbxyzXYqurEY9brtgdtxjWIy68KJ9Nugb8eadg2quLMKZs6oAjM5YisfofC7lHVRlgvRtsoUdWiXTMvPLGWPtAE4G8BJj7DXxqeUAtjDGNgN4BsCtnPO+zEwliMLilOmVaKkrQVufe0wUJDmAWodlopeGMRuyG0H15WEEBQDnzhVWOQbciRufJQUHaxYU3RvLYqd3JXLbPWVexfcs57yRc27mnNdwzs8Tt/+Lcz6Xc76Ac76Ic/4fecwliMLiK2dOB4Ax6uZFYmXWn69fHPM1kWQ/gvKhIg8dVKmYRh1wJRbfdfkE8d5Ypf9yU2oz4YxZVThvbk3M58+eXa24DUqSP6U2BJGHNIh3yJ1DHsxrENahjg66cdH8OkyLWAyfCKtRD7c/mNTaSab4gyEMeQJ51QMl4RArJd/f14M5dSVx9xUaZLN3aX3khiXYeWwIr20flT26dulkXHJ8PRZNKsuaHUqQ2zWIBJHnVBcLF/vOIUGh4UC3E4d7XZhZU5zU64stRnCOrKhJ9Es9UHmiIhFJnUO4UfjpSzsTSke5fMGsN8hGphMf/Pwi/Pe5s3DilNgN3LkEOSiC0DBSibPUW7PtqNCpcfac5FI3JVbhTn7IrfxcqN6R/BGKHU9zZRGkADRRms/lC2Q1ggLGKpafP68uHPHlOuSgCELDWIw66BgwIs512ivOJmquLErq9Xqd8BV/9pMjyhgYgeREJ2ocznV+d40gyPvvBOdSjQjKkuOSRhNBDoogNAxjDEUmYfBgKMTx3KYjOGVaRdLzlo5rFNat3klhXES6uMVqwVzXf5uIpjJhgOEdL+5AIBjCfzYfxbm/eWdMlWRbnwtObyDrAxuzUTGoBlQkQRAap8hswIg3gJ0dQ2jrc+MbZ8fW3ouFtFa17pDyXR5e8UJtUWDMhBaY3zDaLN3n8uFrT3wCQJCeaiyzocfpxbK7VwMAVsyJXVWnFEa9DrcsnzphNV+uQhEUQWicIrMeLl8Q+0VdvsgLZSqEFG7azPcISqdjWHnhHABAx6AnvL1zyIP+EV+4QReAKiPvf3DhHJwwuTzr76sk5KAIQuPYzUKKr2tIuCgm06AbyQ8vEi6qTp+y49/dPqHfKl/TTQBw8QJh7tYH+3vD2678w1osvGMVPBG6d7muIq4VyEERhMaxmYQUn7TWYUsxQikRm0yVruQLR1B57KDqHFbUOSzY0zEc9dyId9RB1ZakdhNBxIYcFEFonCIxgnL7g9DrGIz61L62krLD4V5lVc0lB2ox5fdlZXq1Hfu6o+d0SSrmAFBVnH/NymqQ358kgsgD7GY9nN4APP4QLGnM9zllWiWKLQY8v0nZUnOPPwjGAFOKDjTXmF5tHzPuRELqA1s6tRxXntCQbbPykvz+JBFEHlBVbEZ7vxutfa60ChCsJj0WTirD9qNR49hkxe0LwmrU59UsqFhMJDElFU788KIWmPO0kjHbkIMiCI1z1myhdHjVjs60L3zTq+w40D2iaCWfJxDM6/UniQvn12FBU2nU9l+v2gNAKGoh5IEcFEFonHkNo+KkRwbcaR1jerUdbn8QRwfTe30yuH0hWArAQZUXmfDvL5+Cr581PebzdaVUICEX5KAIQuMUW4z4+RXzAaQvIzStSpBGihzbITcef/YlftRCr2P49rmzYj5H6T35oFiUIHKAq5dMgsNqxLTqxCM2YjFdfN2+LifOmKXMjCBpDlIh8+o3l6ltQl5BDoogcoQL5tel/doKuxllNmNYjUIJ2vrdSYvY5gvXLGnCE+vacPeVx6HEasDs2vizoojUIAdFEAXCpIoitPcr1wvV1ufCmbOqFDu+Frnj0nn4+tkzwvOiCHkp7HicIAqIxjIr2vqUcVAefxDeQAileTgLKh4GvY6ck4KQgyKIAqGpzIYjA25FSs2HRaFUSVaJIOSAHBRBFAiNZVb4gxxdw17Zjy0NKyyx0KoBIR/koAiiQGgsE1JRSqxDOcWJv9SkSsgJOSiCKBAaxYmwn35wLV7YfFTWY0sj6ZOd9EsQyUAOiiAKBCmCAoCvi9Ng5WKEIihCAchBEUSBYDHqUaSQ0oPLl96sKoKIBzkogiggyoqUKQOnNShCCchBEUQBUa6Qg3KJ4+Rt5KAIGSEHRRAFROQ6lJxI485tBaBmTmQPclAEUUDceGpz+HHkiPJMeODtffj3J+2wmfTQ6fJ7WCGRXchBEUQBsXhKOR774hIAkG3C7t2v7kZbn5tKzAnZIQdFEAXG3HoHAGCHzCPglaoQJAoXclAEUWCUF5lgNxtkn647rSq9WVUEMRHkoAiiAKm0m3BA5tlQU6sKaxYUoTzkoAiiACkvMuH9fT3gXD5l8wGXX7ZjEQRADoogCpIVLTUIhDiGxQbbdIl0cP5gKFOzCGIM5KAIogCpKbYAAPqcmZWa+4OjDur7F8zO6FgEMR5yUARRgFTYBUWJ3pHMZkNJUdMPLpxNk2UJ2SEHRRAFSIgrOuMAAAsKSURBVEWRGQCwdn9vRseRHJRRT5cSQn7oU0UQBUi5GEHd8/qejI7jIwdFKAh9qgiiAKmym8OPM6nkk9agTOSgCAWgTxVBFCAmgw63iUUNbn8w7eP4A2IEZSANPkJ+MnJQjLFfMsZ2Mca2MMaeZYyVRjx3O2NsH2NsN2PsvMxNJQhCTqTZTcOe9EvNKcVHKEmmn6pVAOZxzo8DsAfA7QDAGGsBcDWAuQDOB/AAY4yEughCQxRbZHBQAXJQhHJk9KninL/OOZc+3R8CaBQfXwrgSc65l3N+EMA+AEsyeS+CIORl1EGlrwAhVfHRGhShBHJ+qm4E8Ir4uAFAW8Rz7eI2giA0gt1sBDA6rj0dwkUSBnJQhPwkHODCGHsDQG2Mp1Zyzp8X91kJIADg79LLYuwfs1SIMXYLgFsAYNKkSUmYTBCEHEgRlDODFB/1QRFKktBBcc5XxHueMXY9gIsBnM1H61XbATRF7NYI4OgEx38IwEMAsHjxYvmUKwmCiMv4IglBPBY4bUZl0scYLZKgKj5CfjKt4jsfwPcBXMI5d0U89QKAqxljZsZYM4AZANZl8l4EQchLeA1KTPF97s8f4fN/+SilY/ipSIJQkExnNN8PwAxgFWMMAD7knN/KOd/OGHsawA4Iqb+vcM7Tb7YgCEJ2pAhqyO3HW7s60zqGFEHRGhShBBk5KM759DjP3QngzkyOTxCEchj0OliNejzw9r4xquSpQGtQhJLQp4ogChi7xZC2cwIAf0B4La1BEUpADoogCphymymj11OKj1AS+lQRRAEzqcIWtY1zjq4hT8LXbmobwH1v7gVAjbqEMtCniiAKmHNbaqK2/fyVXVjyszfRMRjfSV32+/fRNSwMPKQ1KEIJ6FNFEAXMlYsao7b98d0DAIDOJKIoCXJQhBLQp4ogChidjqGlriTmc/EkkIKhsYUVVCRBKAE5KIIocF782mmwmaKHDcQTkX1vX8+Yn8U+SIKQFXJQBFHg6HQMS5rLo7YPxdHoSyX9RxDpQg6KIAjc/9lF+OxJY8WaPTEm7X71Hx/jpr+tR7dYHEEQSkIOiiAI2M0GnDmresw2ty/aQb245Rje2NlFDorICuSgCIIAANQ5LGN+dseIoCS6neSgCOUhB0UQBACgpa4En1nciH99+WSYDTq4/UEEgiH0is4oIKpGAEDXkAdmUo8gFIY+YQRBABCKJe7+9AKcMLkcVpMeHl8QK5/dhhN++gb8wRBGIlJ+3cNeOKxGFa0lCoFMx20QBJGHWI16uP1B/HNjOwDgwbf346kNbeHne5w+1JSYw0oSBKEE5KAIgohCcFAhSDOyf7Vqz5jnnd4AplnsKlhGFBKU4iMIIgqLUR+zii+SEgvd3xLKQp8wgiCisJr02NM5HHef6mILvnT6VFy+sCFLVhGFBjkogiCisBr1aO1zxd3HbNTh9gvmZMkiohChFB9BEFFYjNHafKPPCZcNmgFFKA19wgiCiMIaQzwWAHQMMOqEy8bWI4PZNIkoQCjFRxBEFFIBREWRCau/ewZsRj28gRD0OoaL7luD4e4Abl7WrLKVRL5DDoogiChqSwTZo1KbESUWoSHXIKb0Hv3iSXhzZyfOm1urmn1EYUAOiiCIKGpEXb5Yc54aSq247uQpWbaIKERoDYogiChqxAiKxhASakIOiiCIKKrsZgDA8plVKltCFDKU4iMIIoqW+hI8cfNSnDilTG1TiAKGHBRBEDE5eVqF2iYQBQ6l+AiCIAhNQg6KIAiC0CTkoAiCIAhNQg6KIAiC0CTkoAiCIAhNQg6KIAiC0CTkoAiCIAhNQg6KIAiC0CTkoAiCIAhNwjjnatsQhjHWDeCwDIeqBNAjw3HUIJdtB8h+tSH71YXsT47JnPOEQo+aclBywRjbwDlfrLYd6ZDLtgNkv9qQ/epC9ssLpfgIgiAITUIOiiAIgtAk+eqgHlLbgAzIZdsBsl9tyH51IftlJC/XoAiCIIjcJ18jKIIgCCLHIQdFEARBaJK8clCMsfMZY7sZY/sYY7epbU8sGGNNjLHVjLGdjLHtjLFviNvLGWOrGGN7xf/LxO2MMXaf+DttYYwtUvc3ABhjesbYJ4yxF8WfmxljH4m2P8UYM4nbzeLP+8Tnp6hpt2hTKWPsGcbYLvFvcHKOnftviZ+bbYyxJxhjFi2ff8bYXxljXYyxbRHbUj7fjLHrxf33MsauV9n+X4qfny2MsWcZY6URz90u2r+bMXZexHZVrk2x7I947r8ZY5wxVin+rLnzD855XvwDoAewH8BUACYAmwG0qG1XDDvrACwSHxcD2AOgBcDdAG4Tt98G4Bfi4wsBvAKAAVgK4CMN/A7fBvAPAC+KPz8N4Grx8YMAviw+/n8AHhQfXw3gKQ3Y/jcAN4mPTQBKc+XcA2gAcBCANeK8f0HL5x/AcgCLAGyL2JbS+QZQDuCA+H+Z+LhMRfvPBWAQH/8iwv4W8bpjBtAsXo/0al6bYtkvbm8C8BoEYYRKzZ7/bH9gFfxDnAzgtYifbwdwu9p2JWH38wDOAbAbQJ24rQ7AbvHxHwFcE7F/eD+V7G0E8CaAswC8KH6YeyK+sOG/g/gFOFl8bBD3YyraXiJe4Nm47bly7hsAtIkXCoN4/s/T+vkHMGXcBT6l8w3gGgB/jNg+Zr9s2z/uucsB/F18POaaI51/ta9NsewH8AyABQAOYdRBae7851OKT/rySrSL2zSLmHJZCOAjADWc82MAIP5fLe6mtd/rtwC+ByAk/lwBYIBzHhB/jrQvbLv4/KC4v1pMBdAN4GExRflnxlgRcuTcc86PALgHQCuAYxDO50bkzvmXSPV8a+rvMI4bIUQdQI7Yzxi7BMARzvnmcU9pzv58clAsxjbN1tAzxuwA/gXgm5zzoXi7xtimyu/FGLsYQBfnfGPk5hi78iSeUwMDhHTHHzjnCwGMQEgxTYSm7BfXai6FkD6qB1AE4IIYu2r1/CdiIns1+XswxlYCCAD4u7Qpxm6asp8xZgOwEsCPYj0dY5uq9ueTg2qHkFeVaARwVCVb4sIYM0JwTn/nnP9b3NzJGKsTn68D0CVu19LvdSqASxhjhwA8CSHN91sApYwxg7hPpH1h28XnHQD6smnwONoBtHPOPxJ/fgaCw8qFcw8AKwAc5Jx3c879AP4N4BTkzvmXSPV8a+3vALFQ4GIAn+Ni3gu5Yf80CDc4m8XvcSOAjxljtdCg/fnkoNYDmCFWNJkgLAq/oLJNUTDGGIC/ANjJOf91xFMvAJCqY66HsDYlbb9OrLBZCmBQSo9kG8757ZzzRs75FAjn9y3O+ecArAbwaXG38bZLv9Onxf1Vu/PlnHcAaGOMzRI3nQ1gB3Lg3Iu0AljKGLOJnyPJ/pw4/xGker5fA3AuY6xMjCLPFbepAmPsfADfB3AJ59wV8dQLAK4WqyebAcwAsA4aujZxzrdyzqs551PE73E7hKKtDmjx/GdroS5Li4EXQqiK2w9gpdr2TGDjaRDC4y0ANon/LoSwNvAmgL3i/+Xi/gzA78XfaSuAxWr/DqJdZ2C0im8qhC/iPgD/BGAWt1vEn/eJz0/VgN3HA9ggnv/nIFQl5cy5B/ATALsAbAPwGISKMc2efwBPQFgv80O4GH4xnfMNYa1nn/jvBpXt3wdhTUb6/j4Ysf9K0f7dAC6I2K7KtSmW/eOeP4TRIgnNnX+SOiIIgiA0ST6l+AiCIIg8ghwUQRAEoUnIQREEQRCahBwUQRAEoUnIQREEQRCahBwUQRAEoUnIQREEQRCa5P8DvJEK5MyqgS4AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(range(1440), temp[:1440])\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "이 그래프에서 일별 주기성을 볼 수 있습니다. 특히 마지막 4일간을 보면 확실합니다. 이 데이터는 아주 추운 겨울 중 10일입니다.\n",
    "\n",
    "지난 몇 달간의 데이터를 사용해 다음 달의 평균 온도를 예측하는 문제는 쉬운 편입니다. 연간 데이터 주기성은 안정적이기 때문입니다. 하지만 하루 하루 데이터를 살펴보면 온도 변화는 매우 불안정합니다. 일자별 수준의 시계열 데이터를 예측할 수 있을까요? 직접 확인해 보겠습니다."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Preparing the data\n",
    "\n",
    "이 문제의 정확한 정의는 다음과 같습니다. `lookback` 타임스텝(하나의 타임스텝은 10분입니다)만큼 이전으로 돌아가서 매 `steps` 타임스텝마다 샘플링합니다. 이 데이터를 바탕으로 `delay` 타임스텝 이후의 온도를 예측할 수 있을까요? 사용할 변수는 다음과 같습니다.\n",
    "\n",
    "* `lookback = 1440` : 10일 전 데이터로 돌아갑니다.\n",
    "* `steps = 6` : 한 시간마다 데이터 포인트 하나를 샘플링합니다.\n",
    "* `delay = 144` : 24시간이 지난 데이터가 타깃이 됩니다.\n",
    "\n",
    "시작하기 전에 두 가지 작업을 처리해야 합니다:\n",
    "\n",
    "* 신경망에 주입할 수 있는 형태로 데이터를 전처리합니다. 데이터가 이미 수치형이므로 추가적인 벡터화가 필요하지 않습니다. 하지만 데이터에 있는 각 시계열 특성의 범위가 서로 다릅니다(예를 들어 온도는 일반적으로 -20도에서 +30도 사이이고, 밀리바(mb)로 측정된 기압은 1,000 근처의 값입니다). 각 시계열 특성을 개별적으로 정규화하여 비슷한 범위를 가진 작은 값으로 바꾸겠습니다.\n",
    "* float_data 배열을 받아 과거 데이터의 배치와 미래 타깃 온도를 추출하는 파이썬 제너레이터를 만듭니다. 이 데이터셋에 있는 샘플은 중복이 많습니다(샘플 `N`과 샘플 `N + 1`은 대부분 타임스텝이 비슷합니다). 모든 샘플을 각기 메모리에 적재하는 것은 낭비가 심하므로 대신에 원본 데이터를 사용해 그때 그때 배치를 만들겠습니다.\n",
    "\n",
    "각 시계열 특성에 대해 평균을 빼고 표준 편차로 나누어 전처리합니다. 처음 200,000개 타임스텝을 훈련 데이터로 사용할 것이므로 전체 데이터에서 200,000개만 사용하여 평균과 표준 편차를 계산합니다:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "mean = float_data[:200000].mean(axis=0)\n",
    "float_data -= mean\n",
    "std = float_data[:200000].std(axis=0)\n",
    "float_data /= std"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "다음은 여기서 사용할 제너레이터입니다. 이 제너레이터 함수는 `(samples, targets)` 튜플을 반복적으로 반환합니다. `samples`는 입력 데이터로 사용할 배치이고 `targets`은 이에 대응되는 타깃 온도의 배열입니다. 이 제너레이터 함수는 다음과 같은 매개변수가 있습니다:\n",
    "\n",
    "* `data` : 코드 6-32에서 정규화한 부동 소수 데이터로 이루어진 원본 배열\n",
    "* `lookback` : 입력으로 사용하기 위해 거슬러 올라갈 타임스텝\n",
    "* `delay` : 타깃으로 사용할 미래의 타임스텝\n",
    "* `min_index`와 `max_index` : 추출할 타임스텝의 범위를 지정하기 위한 `data` 배열의 인덱스. 검증 데이터와 테스트 데이터를 분리하는 데 사용합니다.\n",
    "* `shuffle` : 샘플을 섞을지 시간 순서대로 추출할지 결정합니다.\n",
    "* `batch_size` : 배치의 샘플 수\n",
    "* `step` : 데이터를 샘플링할 타임스텝 간격. 한 시간에 하나의 데이터 포인트를 추출하기 위해 6으로 지정하겠습니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "def generator(data, lookback, delay, min_index, max_index,\n",
    "              shuffle=False, batch_size=128, step=6):\n",
    "    if max_index is None:\n",
    "        max_index = len(data) - delay - 1\n",
    "    i = min_index + lookback\n",
    "    while 1:\n",
    "        if shuffle:\n",
    "            rows = np.random.randint(\n",
    "                min_index + lookback, max_index, size=batch_size)\n",
    "        else:\n",
    "            if i + batch_size >= max_index:\n",
    "                i = min_index + lookback\n",
    "            rows = np.arange(i, min(i + batch_size, max_index))\n",
    "            i += len(rows)\n",
    "\n",
    "        samples = np.zeros((len(rows),\n",
    "                           lookback // step,\n",
    "                           data.shape[-1]))\n",
    "        targets = np.zeros((len(rows),))\n",
    "        for j, row in enumerate(rows):\n",
    "            indices = range(rows[j] - lookback, rows[j], step)\n",
    "            samples[j] = data[indices]\n",
    "            targets[j] = data[rows[j] + delay][1]\n",
    "        yield samples, targets"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "이제 generator 함수를 사용해 훈련용, 검증용, 테스트용으로 세 개의 제너레이터를 만들어 보죠. 각 제너레이터는 원본 데이터에서 다른 시간대를 사용합니다. 훈련 제너레이터는 처음 200,000개 타임스텝을 사용하고 검증 제너레이터는 그 다음 100,000개를 사용하고 테스트 제너레이터는 나머지를 사용합니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "lookback = 1440\n",
    "step = 6\n",
    "delay = 144\n",
    "batch_size = 128\n",
    "\n",
    "train_gen = generator(float_data,\n",
    "                      lookback=lookback,\n",
    "                      delay=delay,\n",
    "                      min_index=0,\n",
    "                      max_index=200000,\n",
    "                      shuffle=True,\n",
    "                      step=step, \n",
    "                      batch_size=batch_size)\n",
    "val_gen = generator(float_data,\n",
    "                    lookback=lookback,\n",
    "                    delay=delay,\n",
    "                    min_index=200001,\n",
    "                    max_index=300000,\n",
    "                    step=step,\n",
    "                    batch_size=batch_size)\n",
    "test_gen = generator(float_data,\n",
    "                     lookback=lookback,\n",
    "                     delay=delay,\n",
    "                     min_index=300001,\n",
    "                     max_index=None,\n",
    "                     step=step,\n",
    "                     batch_size=batch_size)\n",
    "\n",
    "# 전체 검증 세트를 순회하기 위해 val_gen에서 추출할 횟수\n",
    "val_steps = (300000 - 200001 - lookback) // batch_size\n",
    "\n",
    "# 전체 테스트 세트를 순회하기 위해 test_gen에서 추출할 횟수\n",
    "test_steps = (len(float_data) - 300001 - lookback) // batch_size"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 상식 수준의 기준점\n",
    "\n",
    "블랙 박스 같은 딥러닝 모델을 사용해 온도 예측 문제를 풀기 전에 간단한 상식 수준의 해법을 시도해 보겠습니다. 이는 정상 여부 확인을 위한 용도이고 고수준 머신 러닝 모델이라면 뛰어 넘어야 할 기준점을 만듭니다. 이런 상식 수준의 해법은 알려진 해결책이 없는 새로운 문제를 다루어야 할 때 유용합니다. 일부 클래스가 월등히 많아 불균형한 분류 문제가 고전적인 예입니다. 데이터셋에 클래스 A의 샘플이 90%, 클래스 B의 샘플이 10%가 있다면, 이 분류 문제에 대한 상식 수준의 접근법은 새로운 샘플을 항상 클래스 'A'라고 예측하는 것입니다. 이 분류기는 전반적으로 90%의 정확도를 낼 것입니다. 머신 러닝 기반의 모델이라면 90% 이상을 달성해야 유용하다고 볼 수 있습니다. 이따금 이런 기본적인 기준점을 넘어서기가 아주 어려운 경우가 있습니다.\n",
    "\n",
    "이 경우 온도 시계열 데이터는 연속성이 있고 일자별로 주기성을 가진다고 가정할 수 있습니다(오늘 온도는 내일 온도와 비슷할 가능성이 높습니다). 그렇기 때문에 상식 수준의 해결책은 지금으로부터 24시간 후의 온도는 지금과 동일하다고 예측하는 것입니다. 이 방법을 평균 절댓값 오차(MAE)로 평가해 보겠습니다:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "```python\n",
    "np.mean(np.abs(preds - targets))\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "다음은 평가를 위한 반복 루프입니다:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.2897359729905486\n"
     ]
    }
   ],
   "source": [
    "def evaluate_naive_method():\n",
    "    batch_maes = []\n",
    "    for step in range(val_steps):\n",
    "        samples, targets = next(val_gen)\n",
    "        preds = samples[:, -1, 1]\n",
    "        mae = np.mean(np.abs(preds - targets))\n",
    "        batch_maes.append(mae)\n",
    "    print(np.mean(batch_maes))\n",
    "    \n",
    "evaluate_naive_method()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2.5672247338393395"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "0.29 * std[1]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "출력된 MAE는 0.29입니다. 이 온도 데이터는 평균이 0이고 표준 편차가 1이므로 결괏값이 바로 와 닿지는 않습니다. 평균 절댓값 오차 0.29에 표준 편차를 곱하면 섭씨 2.57°C가 됩니다. 평균 절댓값 오차가 상당히 크네요. 이제 딥러닝 모델이 더 나은지 시도해 봅시다."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 기본적인 머신 러닝 방법\n",
    "\n",
    "머신 러닝 모델을 시도하기 전에 상식 수준의 기준점을 세워 놓았습니다. 비슷하게 RNN처럼 복잡하고 연산 비용이 많이 드는 모델을 시도하기 전에 간단하고 손쉽게 만들 수 있는 머신 러닝 모델(예를 들면 소규모의 완전 연결 네트워크)을 먼저 만드는 것이 좋습니다. 이를 바탕으로 더 복잡한 방법을 도입하는 근거가 마련되고 실제적인 이득도 얻게 될 것입니다.\n",
    "\n",
    "다음 코드는 데이터를 펼쳐서 두 개의 `Dense` 층을 통과시키는 완전 연결 네트워크를 보여줍니다. 전형적인 회귀 문제이므로 마지막 `Dense` 층에 활성화 함수를 두지 않았습니다. 손실 함수는 MAE입니다. 상식 수준의 방법에서 사용한 것과 동일한 데이터와 지표를 사용했으므로 결과를 바로 비교해볼 수 있습니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/20\n",
      "500/500 [==============================] - 10s 21ms/step - loss: 1.6545 - val_loss: 0.7454\n",
      "Epoch 2/20\n",
      "500/500 [==============================] - 10s 19ms/step - loss: 0.5458 - val_loss: 0.4677\n",
      "Epoch 3/20\n",
      "500/500 [==============================] - 10s 19ms/step - loss: 0.3097 - val_loss: 0.3011\n",
      "Epoch 4/20\n",
      "500/500 [==============================] - 10s 19ms/step - loss: 0.2696 - val_loss: 0.3534\n",
      "Epoch 5/20\n",
      "500/500 [==============================] - 10s 20ms/step - loss: 0.2541 - val_loss: 0.3437\n",
      "Epoch 6/20\n",
      "500/500 [==============================] - 10s 20ms/step - loss: 0.2439 - val_loss: 0.3076\n",
      "Epoch 7/20\n",
      "500/500 [==============================] - 10s 20ms/step - loss: 0.2361 - val_loss: 0.3385\n",
      "Epoch 8/20\n",
      "500/500 [==============================] - 10s 19ms/step - loss: 0.2315 - val_loss: 0.3103\n",
      "Epoch 9/20\n",
      "500/500 [==============================] - 10s 19ms/step - loss: 0.2271 - val_loss: 0.3226\n",
      "Epoch 10/20\n",
      "500/500 [==============================] - 10s 20ms/step - loss: 0.2225 - val_loss: 0.3122\n",
      "Epoch 11/20\n",
      "500/500 [==============================] - 10s 19ms/step - loss: 0.2194 - val_loss: 0.3086\n",
      "Epoch 12/20\n",
      "500/500 [==============================] - 10s 20ms/step - loss: 0.2172 - val_loss: 0.3364\n",
      "Epoch 13/20\n",
      "500/500 [==============================] - 10s 19ms/step - loss: 0.2125 - val_loss: 0.3463\n",
      "Epoch 14/20\n",
      "500/500 [==============================] - 10s 19ms/step - loss: 0.2125 - val_loss: 0.3247\n",
      "Epoch 15/20\n",
      "500/500 [==============================] - 10s 19ms/step - loss: 0.2096 - val_loss: 0.3220\n",
      "Epoch 16/20\n",
      "500/500 [==============================] - 10s 19ms/step - loss: 0.2083 - val_loss: 0.3218\n",
      "Epoch 17/20\n",
      "500/500 [==============================] - 10s 19ms/step - loss: 0.2051 - val_loss: 0.3349\n",
      "Epoch 18/20\n",
      "500/500 [==============================] - 10s 20ms/step - loss: 0.2037 - val_loss: 0.3253\n",
      "Epoch 19/20\n",
      "500/500 [==============================] - 10s 19ms/step - loss: 0.2031 - val_loss: 0.3285\n",
      "Epoch 20/20\n",
      "500/500 [==============================] - 10s 20ms/step - loss: 0.2017 - val_loss: 0.3222\n"
     ]
    }
   ],
   "source": [
    "from keras.models import Sequential\n",
    "from keras import layers\n",
    "from keras.optimizers import RMSprop\n",
    "\n",
    "model = Sequential()\n",
    "model.add(layers.Flatten(input_shape=(lookback // step, float_data.shape[-1])))\n",
    "model.add(layers.Dense(32, activation='relu'))\n",
    "model.add(layers.Dense(1))\n",
    "\n",
    "model.compile(optimizer=RMSprop(), loss='mae')\n",
    "history = model.fit_generator(train_gen,\n",
    "                              steps_per_epoch=500,\n",
    "                              epochs=20,\n",
    "                              validation_data=val_gen,\n",
    "                              validation_steps=val_steps)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "훈련 손실과 검증 손실의 그래프를 그려 보죠:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAAEYCAYAAAAJeGK1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3Xt8FPW9//HXBwhgBAEJKIIQvFQFRAgRoWDBy88DtF6PVRHvF4rXWrUPOWrVankcq1YRRVvq8Z6CHlurtSi1RypVixoQEEQKKmgEIaAgCF4SPr8/vpuwLLvJJtlkJ8n7+XjsY3dmvjv72clm3vudmZ0xd0dERCRqWmS7ABERkWQUUCIiEkkKKBERiSQFlIiIRJICSkREIkkBJSIikaSAkqwxs5ZmtsXMemaybTaZ2QFmlvHfbpjZsWa2Mm54mZkdmU7bWrzWQ2Z2fW2fX8V8f2Vmj2Z6vtJ0tcp2AdJ4mNmWuMFc4BugPDb8E3cvqsn83L0caJfpts2Bux+UifmY2UXAWe4+Mm7eF2Vi3iJ1pYCStLl7ZUDEvqFf5O5/T9XezFq5e1lD1CYiTY828UnGxDbhPGVm081sM3CWmQ01s7lmttHM1pjZFDPLibVvZWZuZvmx4Sdj0180s81m9i8z613TtrHpo83s32a2yczuM7PXzey8FHWnU+NPzGyFmX1hZlPintvSzO4xsw1m9gEwqorlc6OZzUgYN9XM7o49vsjMlsbezwex3k2qeZWY2cjY41wzeyJW2xJgUJLX/TA23yVmdkJs/KHA/cCRsc2n6+OW7S1xz58Qe+8bzOzPZtYtnWVTHTM7KVbPRjN7xcwOipt2vZmtNrMvzez9uPc6xMzmx8avNbM70309aYTcXTfdanwDVgLHJoz7FfAtcDzhy89uwOHAEYTe+n7Av4HLY+1bAQ7kx4afBNYDhUAO8BTwZC3adgU2AyfGpl0NfAecl+K9pFPjc0AHIB/4vOK9A5cDS4AeQGdgTvi3Svo6+wFbgN3j5r0OKIwNHx9rY8DRwDagf2zascDKuHmVACNjj+8C/gF0AnoB7yW0PQ3oFvubnBmrYa/YtIuAfyTU+SRwS+zxcbEaBwBtgQeAV9JZNkne/6+AR2OPD4nVcXTsb3R9bLnnAH2BVcDesba9gf1ij98GxsYetweOyPb/gm71d1MPSjLtNXf/i7tvd/dt7v62u7/p7mXu/iEwDRhRxfOfcfdid/8OKCKsGGva9kfAAnd/LjbtHkKYJZVmjf/t7pvcfSUhDCpe6zTgHncvcfcNwO1VvM6HwGJCcAL8P2CjuxfHpv/F3T/04BXg/4CkB0IkOA34lbt/4e6rCL2i+Nd92t3XxP4mfyB8uShMY74A44CH3H2Bu38NTARGmFmPuDaplk1VzgCed/dXYn+j24E9CF8Uyghh2De2mfij2LKD8EXjQDPr7O6b3f3NNN+HNEIKKMm0T+IHzOxgM/urmX1mZl8CtwJ5VTz/s7jHW6n6wIhUbfeJr8PdndDjSCrNGtN6LcI3/6r8ARgbe3wmIVgr6viRmb1pZp+b2UZC76WqZVWhW1U1mNl5ZrYwtiltI3BwmvOF8P4q5+fuXwJfAN3j2tTkb5ZqvtsJf6Pu7r4MuIbwd1gX22S8d6zp+UAfYJmZvWVmY9J8H9IIKaAk0xIPsf4doddwgLvvAdxE2IRVn9YQNrkBYGbGzivURHWpcQ2wb9xwdYfBPwUcG+uBnEgILMxsN+AZ4L8Jm986An9Ls47PUtVgZvsBDwKXAJ1j830/br7VHRK/mrDZsGJ+7QmbEj9No66azLcF4W/2KYC7P+nuwwib91oSlgvuvszdzyBsxv0N8Ecza1vHWiSiFFBS39oDm4CvzOwQ4CcN8JovAAVmdryZtQJ+CnSppxqfBq4ys+5m1hm4rqrG7r4WeA14BFjm7stjk9oArYFSoNzMfgQcU4MarjezjhZ+J3Z53LR2hBAqJWT1RYQeVIW1QI+Kg0KSmA5caGb9zawNISj+6e4pe6Q1qPkEMxsZe+2fE/Ybvmlmh5jZUbHX2xa7lRPewNlmlhfrcW2KvbftdaxFIkoBJfXtGuBcwsrnd4QeRL2KhcDpwN3ABmB/4B3C77YyXeODhH1F7xJ24D+TxnP+QDjo4Q9xNW8EfgY8SzjQ4FRC0KbjZkJPbiXwIvB43HwXAVOAt2JtDgbi99u8DCwH1ppZ/Ka6iue/RNjU9mzs+T0J+6XqxN2XEJb5g4TwHAWcENsf1Qa4g7Df8DNCj+3G2FPHAEstHCV6F3C6u39b13okmixsnhdpusysJWGT0qnu/s9s1yMi6VEPSpokMxtlZh1im4l+QTgy7K0slyUiNaCAkqZqOPAhYTPRKOAkd0+1iU9EIkib+EREJJLUgxIRkUjK2sli8/LyPD8/P1svLyIiWTJv3rz17l7VTz+ALAZUfn4+xcXF2Xp5ERHJEjOr7owrgDbxiYhIRCmgREQkkhRQIiISSbqirog0Gt999x0lJSV8/fXX2S5F0tC2bVt69OhBTk6qUz1WTQElIo1GSUkJ7du3Jz8/n3CSeokqd2fDhg2UlJTQu3fv6p+QhDbxiUij8fXXX9O5c2eFUyNgZnTu3LlOvd1GG1BFRZCfDy1ahPuiouqeISJNgcKp8ajr36pRbuIrKoLx42Hr1jC8alUYBhhX5wsBiIhIFDTKHtQNN+wIpwpbt4bxIiL1ZcOGDQwYMIABAwaw9957071798rhb79N77JU559/PsuWLauyzdSpUynK0Gah4cOHs2DBgozMq6E1yh7Uxx/XbLyINE9FReGL68cfQ8+eMGlS3baydO7cuXJlf8stt9CuXTuuvfbandq4O+5OixbJv/8/8sgj1b7OZZddVvsim5BG2YPq2bNm40Wk+anYFbBqFbjv2BVQH/urV6xYQb9+/ZgwYQIFBQWsWbOG8ePHU1hYSN++fbn11lsr21b0aMrKyujYsSMTJ07ksMMOY+jQoaxbtw6AG2+8kcmTJ1e2nzhxIoMHD+aggw7ijTfeAOCrr77iP//zPznssMMYO3YshYWF1faUnnzySQ499FD69evH9ddfD0BZWRlnn3125fgpU6YAcM8999CnTx8OO+wwzjrrrIwvs3Q0yoCaNAlyc3cel5sbxouIQMPvCnjvvfe48MILeeedd+jevTu33347xcXFLFy4kJdffpn33ntvl+ds2rSJESNGsHDhQoYOHcrDDz+cdN7uzltvvcWdd95ZGXb33Xcfe++9NwsXLmTixIm88847VdZXUlLCjTfeyOzZs3nnnXd4/fXXeeGFF5g3bx7r16/n3XffZfHixZxzzjkA3HHHHSxYsICFCxdy//3313Hp1E6jDKhx42DaNOjVC8zC/bRpOkBCRHZo6F0B+++/P4cffnjl8PTp0ykoKKCgoIClS5cmDajddtuN0aNHAzBo0CBWrlyZdN6nnHLKLm1ee+01zjjjDAAOO+ww+vbtW2V9b775JkcffTR5eXnk5ORw5plnMmfOHA444ACWLVvGT3/6U2bNmkWHDh0A6Nu3L2eddRZFRUW1/qFtXTXKgIIQRitXwvbt4V7hJCLxGnpXwO677175ePny5dx777288sorLFq0iFGjRiX9PVDr1q0rH7ds2ZKysrKk827Tps0ubWp6sdlU7Tt37syiRYsYPnw4U6ZM4Sc/+QkAs2bNYsKECbz11lsUFhZSXl5eo9fLhGoDysweNrN1Zra4ijYjzWyBmS0xs1czW6KISM1lc1fAl19+Sfv27dljjz1Ys2YNs2bNyvhrDB8+nKeffhqAd999N2kPLd6QIUOYPXs2GzZsoKysjBkzZjBixAhKS0txd3784x/zy1/+kvnz51NeXk5JSQlHH300d955J6WlpWxN3F7aANI5iu9R4H7g8WQTzawj8AAwyt0/NrOumStPRKR2KraqZPIovnQVFBTQp08f+vXrx3777cewYcMy/hpXXHEF55xzDv3796egoIB+/fpVbp5LpkePHtx6662MHDkSd+f444/nhz/8IfPnz+fCCy/E3TEzfv3rX1NWVsaZZ57J5s2b2b59O9dddx3t27fP+HuojqXTTTSzfOAFd++XZNqlwD7ufmNNXriwsNB1wUIRqYmlS5dyyCGHZLuMSCgrK6OsrIy2bduyfPlyjjvuOJYvX06rVtH69VCyv5mZzXP3wuqem4l38j0gx8z+AbQH7nX3VL2t8cB4gJ46JlxEpNa2bNnCMcccQ1lZGe7O7373u8iFU11l4t20AgYBxwC7Af8ys7nu/u/Ehu4+DZgGoQeVgdcWEWmWOnbsyLx587JdRr3KRECVAOvd/SvgKzObAxwG7BJQIiIi6crEYebPAUeaWSszywWOAJZmYL4iItKMVduDMrPpwEggz8xKgJuBHAB3/627LzWzl4BFwHbgIXdPeUi6iIhIOqoNKHcfm0abO4E7M1KRiIgIjfhMEiIiDW3kyJG7/Oh28uTJXHrppVU+r127dgCsXr2aU089NeW8q/vpzeTJk3f6weyYMWPYuHFjOqVX6ZZbbuGuu+6q83wyTQElIpKmsWPHMmPGjJ3GzZgxg7Fjq93QBMA+++zDM888U+vXTwyomTNn0rFjx1rPL+oUUCIiaTr11FN54YUX+OabbwBYuXIlq1evZvjw4ZW/SyooKODQQw/lueee2+X5K1eupF+/cL6Dbdu2ccYZZ9C/f39OP/10tm3bVtnukksuqbxUx8033wzAlClTWL16NUcddRRHHXUUAPn5+axfvx6Au+++m379+tGvX7/KS3WsXLmSQw45hIsvvpi+ffty3HHH7fQ6ySxYsIAhQ4bQv39/Tj75ZL744ovK1+/Tpw/9+/evPEntq6++WnnBxoEDB7J58+ZaL9tkmtavukSk2bjqKsj0hWIHDIDYuj2pzp07M3jwYF566SVOPPFEZsyYwemnn46Z0bZtW5599ln22GMP1q9fz5AhQzjhhBMws6TzevDBB8nNzWXRokUsWrSIgoKCymmTJk1izz33pLy8nGOOOYZFixZx5ZVXcvfddzN79mzy8vJ2mte8efN45JFHePPNN3F3jjjiCEaMGEGnTp1Yvnw506dP5/e//z2nnXYaf/zjH6u8vtM555zDfffdx4gRI7jpppv45S9/yeTJk7n99tv56KOPaNOmTeVmxbvuuoupU6cybNgwtmzZQtu2bWuwtKunHpSISA3Eb+aL37zn7lx//fX079+fY489lk8//ZS1a9emnM+cOXMqg6J///7079+/ctrTTz9NQUEBAwcOZMmSJdWeCPa1117j5JNPZvfdd6ddu3accsop/POf/wSgd+/eDBgwAKj6kh4Qrk+1ceNGRowYAcC5557LnDlzKmscN24cTz75ZOUZK4YNG8bVV1/NlClT2LhxY8bPZKEelIg0SlX1dOrTSSedxNVXX838+fPZtm1bZc+nqKiI0tJS5s2bR05ODvn5+UkvsREvWe/qo48+4q677uLtt9+mU6dOnHfeedXOp6pzqlZcqgPC5Tqq28SXyl//+lfmzJnD888/z2233caSJUuYOHEiP/zhD5k5cyZDhgzh73//OwcffHCt5p+MelAiIjXQrl07Ro4cyQUXXLDTwRGbNm2ia9eu5OTkMHv2bFatWlXlfH7wgx9QFLv+/OLFi1m0aBEQLtWx++6706FDB9auXcuLL75Y+Zz27dsn3c/zgx/8gD//+c9s3bqVr776imeffZYjjzyyxu+tQ4cOdOrUqbL39cQTTzBixAi2b9/OJ598wlFHHcUdd9zBxo0b2bJlCx988AGHHnoo1113HYWFhbz//vs1fs2qqAclIlJDY8eO5ZRTTtnpiL5x48Zx/PHHU1hYyIABA6rtSVxyySWcf/759O/fnwEDBjB48GAgXB134MCB9O3bd5dLdYwfP57Ro0fTrVs3Zs+eXTm+oKCA8847r3IeF110EQMHDqxyc14qjz32GBMmTGDr1q3st99+PPLII5SXl3PWWWexadMm3J2f/exndOzYkV/84hfMnj2bli1b0qdPn8qrA2dKWpfbqA+63IaI1JQut9H41OVyG9rEJyIikaSAEhGRSFJAiUijkq3dElJzdf1bKaBEpNFo27YtGzZsUEg1Au7Ohg0b6vTjXR3FJyKNRo8ePSgpKaG0tDTbpUga2rZtS48ePWr9fAWUiDQaOTk59O7dO9tlSAPRJj4REYkkBZSIiESSAkpERCJJASUiIpGkgBIRkUiqNqDM7GEzW2dmi6tpd7iZlZvZqZkrT0REmqt0elCPAqOqamBmLYFfA7MyUJOIiEj1AeXuc4DPq2l2BfBHYF0mihIREanzPigz6w6cDPw2jbbjzazYzIr1S3AREalKJg6SmAxc5+7l1TV092nuXujuhV26dMnAS4uISFOViVMdFQIzzAwgDxhjZmXu/ucMzFtERJqpOgeUu1eeGMvMHgVeUDiJiEhdVRtQZjYdGAnkmVkJcDOQA+Du1e53EhERqY1qA8rdx6Y7M3c/r07ViIiIxOhMEiIiEkkKKBERiSQFlIiIRJICSkREIkkBJSIikaSAEhGRSFJAiYhIJCmgREQkkhRQIiISSQooERGJJAWUiIhEkgJKREQiSQElIiKRpIASEZFIUkCJiEgkKaBERCSSFFAiIhJJCigREYkkBZSIiESSAkpERCKp2oAys4fNbJ2ZLU4xfZyZLYrd3jCzwzJfpoiINDfp9KAeBUZVMf0jYIS79wduA6ZloC4REWnmWlXXwN3nmFl+FdPfiBucC/Soe1kiItLcZXof1IXAi6kmmtl4Mys2s+LS0tIMv7SIiDQlGQsoMzuKEFDXpWrj7tPcvdDdC7t06ZKplxYRkSao2k186TCz/sBDwGh335CJeYqISPNW5x6UmfUE/gSc7e7/rntJIiIiafSgzGw6MBLIM7MS4GYgB8DdfwvcBHQGHjAzgDJ3L6yvgkVEpHlI5yi+sdVMvwi4KGMViYiIoDNJiIhIRCmgREQkkhRQIiISSQooERGJJAWUiIhEkgJKREQiSQElIiKRpIASEZFIUkCJiEgkKaBERCSSFFAiIhJJCigREYkkBZSIiESSAkpERCJJASUiIpGkgBIRkUhSQImISCQpoEREJJIUUCIiEkkKKBERiaRqA8rMHjazdWa2OMV0M7MpZrbCzBaZWUHmyxQRkeYmnR7Uo8CoKqaPBg6M3cYDD9a9LBERae6qDSh3nwN8XkWTE4HHPZgLdDSzbpkqUEREmqdM7IPqDnwSN1wSGyciIlJrmQgoSzLOkzY0G29mxWZWXFpamoGXFhGRpioTAVUC7Bs33ANYnayhu09z90J3L+zSpUsGXlpERJqqTATU88A5saP5hgCb3H1NBuYrIiLNWKvqGpjZdGAkkGdmJcDNQA6Au/8WmAmMAVYAW4Hz66tYERFpPqoNKHcfW810By7LWEUiIiLoTBIiIhJRCigREYkkBZSIiESSAkpERCJJASUiIpGkgBIRkUhSQImISCQpoEREJJIUUCIiEkkKKBERiSQFlIiIRJICSkREIkkBJSIikaSAEhGRSFJAiYhIJCmgREQkkhRQIiISSQooERGJJAWUiIhEkgJKREQiSQElIiKRlFZAmdkoM1tmZivMbGKS6T3NbLaZvWNmi8xsTOZLFRGR5qTagDKzlsBUYDTQBxhrZn0Smt0IPO3uA4EzgAcyXaiIiDQv6fSgBgMr3P1Dd/8WmAGcmNDGgT1ijzsAqzNXooiINEfpBFR34JO44ZLYuHi3AGeZWQkwE7gi2YzMbLyZFZtZcWlpaS3KFRGR5iKdgLIk4zxheCzwqLv3AMYAT5jZLvN292nuXujuhV26dKl5tSIi0mykE1AlwL5xwz3YdRPehcDTAO7+L6AtkJeJAkVEpHlKJ6DeBg40s95m1ppwEMTzCW0+Bo4BMLNDCAGlbXgiIlJr1QaUu5cBlwOzgKWEo/WWmNmtZnZCrNk1wMVmthCYDpzn7ombAUVERNLWKp1G7j6TcPBD/Lib4h6/BwzLbGkiItKcNeozSWzeDGVl2a5CRETqQ6MNqGXLoGdPeOaZbFciIiL1odEG1IEHwl57wZ13gvZ2iYg0PY02oFq0gGuugfnz4dVXs12NiIhkWqMNKICzz4auXeGuu7JdiYiIZFqjDqi2beHyy+Gvf4X33st2NSIikkmNOqAALrkEdtsN7r4725WIiEgmNfqAysuD886DJ56Azz7LdjUiIpIpjT6gAH72M/juO7j//mxXIiIimdIkAurAA+Gkk+CBB+Crr7JdjYiIZEKTCCiAa6+FL76ARx7JdiUiIpIJTSagvv99GDo0HCxRXp7takREpK6aTEBB6EV99BE8+2y2KxERkbpqUgF14omw//46/ZGISFPQpAKqZUu4+mp46y14/fVsVyMiInXRpAIKwm+iOnfW6Y9ERBq7JhdQublw6aXw/PPhkhwiItI4NbmAArjsMmjdGu65J9uViIhIbTXJgNprLzjnHHjsMVi3LtvViIhIbTTJgIJwsMTXX4ezS4iISOOTVkCZ2SgzW2ZmK8xsYoo2p5nZe2a2xMz+kNkya+7gg+H442HqVNi2LdvViIhITVUbUGbWEpgKjAb6AGPNrE9CmwOB/wKGuXtf4Kp6qLXGrr0W1q+Hxx/PdiUiIlJT6fSgBgMr3P1Dd/8WmAGcmNDmYmCqu38B4O6R2PNz5JFw+OHwm9/A9u3ZrkZERGoinYDqDnwSN1wSGxfve8D3zOx1M5trZqOSzcjMxptZsZkVl5aW1q7iGjALvajly+Evf6n3lxMRkQxKJ6AsybjEEwm1Ag4ERgJjgYfMrOMuT3Kf5u6F7l7YpUuXmtZaK6ecAvn5+uGuiEhjk05AlQD7xg33AFYnafOcu3/n7h8BywiBlXWtWoULGr72Gsydm+1qREQkXekE1NvAgWbW28xaA2cAzye0+TNwFICZ5RE2+X2YyULr4oILoGPHsC9KREQah2oDyt3LgMuBWcBS4Gl3X2Jmt5rZCbFms4ANZvYeMBv4ubtvqK+ia6pdO5gwAf70J/jgg2xXIyIi6Ujrd1DuPtPdv+fu+7v7pNi4m9z9+dhjd/er3b2Pux/q7jPqs+jauOKKcLbzyZN3Hl9UFPZRtWgR7ouKslGdiIgkarJnkki0zz4wbhw8/DBsiPXtiopg/HhYtSpcP2rVqjCskBIRyb5mE1AA11wDW7fCb38bhm+4IQzH27o1jBcRkexqVgHVrx+MGgX33RfO0/fxx8nbpRovIiINp1kFFIQf7q5dGzbj9eyZvE2q8SIi0nCaXUAdfTQMGBAOOb/ttnCBw3i5uTBpUnZqExGRHZpdQFWc/mjpUthzT5g2DXr1CuN79QrD48Zlu0oRETH3xLMWNYzCwkIvLi7Oymt/9x3stx8ccADMnp2VEkREmi0zm+fuhdW1a3Y9KICcHLjqKvjHPyBLGSkiItVolgEFcPHFsMceOv2RiEhUNduA2mOP8KPc//1fWLky29WIiEiiZhtQAFdeGQ6OuPfebFciIiKJmnVA7bsvnHEGPPQQbNyY7WpERCResw4oCKc/2rIFfve7bFciIiLxmn1ADRgAxx0Xfpy7YkW2qxERkQrNPqAg/Di3VSv48Y/DOfpERCT7FFCEM0g8/jgsWBB+HyUiItmngIr50Y/g5z8P+6L+8IdsVyMiIgqoOJMmwbBh4fdR77+f7WpERJo3BVScnByYMQN22y3sj0q8mKGIiDQcBVSCHj3giSdg8WK44opsVyMi0nylFVBmNsrMlpnZCjObWEW7U83Mzazas9RG2ahR4bLvDz8Mjz2W7WpERJqnagPKzFoCU4HRQB9grJn1SdKuPXAl8Gami8yGW26BESPgkktgyZJsVyMi0vyk04MaDKxw9w/d/VtgBnBikna3AXcATeKXRK1awfTp0L592B+1ZUu2KxIRaV7SCajuwCdxwyWxcZXMbCCwr7u/UNWMzGy8mRWbWXFpaWmNi21o3bqFQ87ffx8uvRSydG1HEZFmKZ2AsiTjKlfVZtYCuAe4proZufs0dy9098IuXbqkX2UWHXMM3HxzOHDif/4n29WIiDQf6QRUCbBv3HAPYHXccHugH/APM1sJDAGeb+wHSsS78UY49thwVN+iRdmuRkSkeUgnoN4GDjSz3mbWGjgDeL5iortvcvc8d89393xgLnCCuzeZi6m3bAlPPgmdOsGpp8KXX2a7IhGRpq/agHL3MuByYBawFHja3ZeY2a1mdkJ9FxgVe+0VDpr44INwpgntjxIRqV+t0mnk7jOBmQnjbkrRdmTdy4qmESPgttvCb6QqDkEXEZH6kVZAyQ4TJ8I//xnOen7EEVBQkNn5u8Mrr8Cjj8I330Be3o5b5847D+flQW5uuGy9iEhTo4CqoRYtwhF9AweG30fNnw8dOtR9vhs3hkt+PPAALFsGe+4JXbrA+vXw+eepNym2abNraCWG2f77w6BBYV+aSCplZbBhQ/jMVdxKS3cebt0a+vaFfv3CbZ999AVJ6o8Cqhby8uCpp8JmvgsugGeeqf0/6cKFMHUqFBWFk9MecUQ4vdJpp0HbtqFNeXkIsPgVReKKpGL4nXfC/Rdf7BxqnTvDf/wHjB4d7rNxlP8338Bbb8Hrr0O7djB0KPTvH07SK/Xj22/h3Xdh9erUoVMxbuPG1PPZY4/wuf/qK3jkkR3jO3YMQRUfWv36hbbNlbtCO1PMs7S3v7Cw0IuLG/eBfnfdFa4hde+9cOWV6T/vm29CqD3wALzxRgiiM88MPwYeNCgztZWXh5AqLQ2Hxs+cCS+9BOvWhX+eww8PYTVmDBQWhp5hpm3bBnPnwquvhtvcubtesTg3N9QydCh8//vhvjmv3OqqpCQs53/9K9zmzw+ft3ht2oQvKIm97mTjKnrjbdrseP769eH0X4sX77hfvDh83irstdeuodW3bwi6xmr79vBF8NNPQ+Cnui8tDWegqdiKkXifbFziMq4r99Aj/uabcHMPXwrbtIlGeJrZPHev9qdICqg6cIcTTwwr/htvDCeX/fhj6NkzXFtq3Lid269aFS6I+NBD4UN8wAEhlM49N2zSq2/bt4cV1osvhsB6883wHvLywglyK3pXnTtC6HPCAAALWUlEQVTXbv5ffRUCtyKQ3norfINv0QIGDAg9zhEj4Mgjd7SNX5GWlYX5HHhgCKqK0Orbt/42T379NXz2Wfh7tGsXVtJ77lk/gZ1pX38dlltFIM2dGwIKwopo0KCwDIcMgfz8HSFUH/st3WHNml1Da8mS8LeusO++IawOOgh23z1sMszJ2fk+1eNU4+I/G/HvK9XjVNO2bUseOhWPV6+G777b9b137Ro2dXbvHu67dg2nRovfslFxv3lz6mXYrt2uwdWqVfgfqgia+MeJw4mPk63aW7UK4VmXW48edd+toYBqIJ9/Hv7ZNmzY+QORmwvTpsHYsfDyy6G39ELsRFDHHx+C6dhjs7siXL8e/va3EFgvvRSGW7SAwYN39K4KClLX+OWXYXNdRSAVF4eQadkyrBwrAmn48Oo/0Nu2wbx5O0LrjTdCbw/CP8URR+wIrSFDwm/SqrJlS1hhrlkTViwVjxNv8d/6K7RosWNl3qVLWOFUPE42vOee9b9/zz18+YkPo/nzd6ww8/PDcqlYPgMGhBV4tm3fHr6YJQbX8uUhYLdvz3aFqbVvv3PwJLvfe++aLedvvgnrjGThlWxceXn4stG6dbhPfJzuNLPwP7F5c3q3ii+LyTz4IEyYULdlq4BqQN26hW/hiTp1Ct+EVqwIK7KLLw6/oerVq+FrrE55eQiIit7V22+HlWLXrjt6V0OHhs2FFYE0f35YweTkhM10FYH0/e+Hf+66cIePPtoRVv/6V9hfV7FCO+SQ8Dr77x+CLDF4kp3ct3Xr8LdKduvSJTyntDTc1q3b8bhiONU+mhYtwt+5IrA6dNixgmjbdtfHycYle7x1a+jlVoTSmjXh9XbbLSzvikA64ojwHhqj8vIQst9+u+t9uuMqPhPxq7JUj6ua1qbNzgFU189wY+UegjRVeA0aFP7v6kIB1YBatEh9lN2wYXDZZXDKKZndxlzfSkth1qwdvavPP98xrXXrsFIcMQJGjgwrydzc+q9py5YQnPGbBj//PGwqig+bffZJHkKdOtVt09Z33+04oCAxwOLHbd4c/sG//nrHJpeKx+XlNX/d/fffuXekA0uksVNANaD8/LAZI1G3bmHzUmNXXh6C4e234dBDQzjttlu2qwpfCrZuDQHVWMTvuE4VYhX3rVqFb6tdu2a7apHMSjegdJh5BkyaFDbdbd26Y1xuLtx5Z/ZqyqSWLcM39yFDsl3JzswaVzhBCJ1WrRpf3SLZ0AiOVYq+cePCARG9eoWVZq9eYTjxKL50FRWFXlmLFuG+qCiT1YqINA7qQWXIuHG1D6R4RUU798ZWrQrDFa8hItJcqAcVMTfcsPOmQgjDN9yQnXpERLJFARUxH39cs/FV0aZCEWnMFFAR07NnzcanUrGpcNWqcLRbxabC2oSUgk5EskEBFTGTJu36m6Lc3DC+JjK1qTCTQVcxP4WdiKRDARUxmToiMFObCjO5TyyqvTqFpkhEuXtWboMGDXKpP716uYcY2PnWq1fN5mOWfD5m2avpySfdc3N3nkdubhhfU5meV69eYdn06lW7eYg0B0Cxp5ET6kE1UZnaVJipfWIQzV5dFDeFqncoEpNOitXHTT2o+peJb/SZ7GFEsVeXqXmpd9iw84lqTZIe0uxBpRUmwChgGbACmJhk+tXAe8Ai4P+AXtXNUwHVeGRyRZCJFWamwiCT84pa0GVyXpn6u0UxfDNZU8X8ohSaUfxC4J7BgAJaAh8A+wGtgYVAn4Q2RwG5sceXAE9VN18FVPMUtV5d1EKzKfcOoxi+mawpaqEZxf+TCpkMqKHArLjh/wL+q4r2A4HXq5uvAkrqImrfDKMWdJmcV6aCLorhG8WDgKI2n0zPyz2zAXUq8FDc8NnA/VW0vx+4McW08UAxUNyzZ8/avTORiIpS0GVyXk15hZnJmqIWmlH8QlAhkwH14yQBdV+KtmcBc4E21c1XPSiR5Jpq7zCK4RvFg4CiNp9Mz8s9swGV1iY+4FhgKdA1nRdWQIk0Hk15p33UDgKK2nwyPS/3zAZUK+BDoHfcQRJ9E9oMjB1IcWA6L+oKKBFpgqIYmlH7QuCefkCldcl3MxsDTI4d0fewu08ys1tjL/K8mf0dOBRYE3vKx+5+QlXzbEqXfBcRkfRl9JLv7j4TmJkw7qa4x8fWuEIREZEq6FRHIiISSQooERGJJAWUiIhEkgJKREQiSQElIiKRlNZh5vXywmalwKqsvHjt5QHrs11ELajuhqW6G5bqbliZqLuXu3eprlHWAqoxMrPidI7djxrV3bBUd8NS3Q2rIevWJj4REYkkBZSIiESSAqpmpmW7gFpS3Q1LdTcs1d2wGqxu7YMSEZFIUg9KREQiSQElIiKRpIBKYGb7mtlsM1tqZkvM7KdJ2ow0s01mtiB2uynZvBqama00s3djNe1yLRMLppjZCjNbZGYF2agzoaaD4pbjAjP70syuSmgTieVtZg+b2TozWxw3bk8ze9nMlsfuO6V47rmxNsvN7NyGqzpl3Xea2fuxz8GzZtYxxXOr/EzVpxR132Jmn8Z9FsakeO4oM1sW+6xPbLiqU9b9VFzNK81sQYrnZmV5p1rvZf3znc5Fo5rTDegGFMQetwf+DfRJaDMSeCHbtSapfSWQV8X0McCLgAFDgDezXXNCfS2Bzwg/4ovc8gZ+ABQAi+PG3QFMjD2eCPw6yfP2JFz0c0+gU+xxpyzXfRzQKvb418nqTuczlYW6bwGuTeNz9AGwHzsustqnPmutru6E6b8BborS8k613sv251s9qATuvsbd58cebyZcxr57dqvKmBOBxz2YC3Q0s27ZLirOMcAH7h7JM4y4+xzg84TRJwKPxR4/BpyU5Kn/Abzs7p+7+xfAy8Coeis0QbK63f1v7l4WG5wL9GioetKVYnmnYzCwwt0/dPdvgRmEv1ODqKpuMzPgNGB6Q9WTjirWe1n9fCugqmBm+YTL2b+ZZPJQM1toZi+aWd8GLSw1B/5mZvPMbHyS6d2BT+KGS4hW+J5B6n/cKC5vgL3cfQ2Ef3Kga5I2UV/uFxB61slU95nKhstjmyYfTrHJKcrL+0hgrbsvTzE968s7Yb2X1c+3AioFM2sH/BG4yt2/TJg8n7AZ6jDgPuDPDV1fCsPcvQAYDVxmZj9ImG5JnhOJ3xmYWWvgBOB/k0yO6vJOV5SX+w1AGVCUokl1n6mG9iCwPzAAWEPYXJYosssbGEvVvaesLu9q1nspn5ZkXEaWtwIqCTPLIfyRitz9T4nT3f1Ld98SezwTyDGzvAYucxfuvjp2vw54lrCpI14JsG/ccA9gdcNUV63RwHx3X5s4IarLO2ZtxWbS2P26JG0iudxjO7N/BIzz2M6ERGl8phqUu69193J33w78PkU9UV3erYBTgKdStcnm8k6x3svq51sBlSC2jfh/gKXufneKNnvH2mFmgwnLcUPDVZm0pt3NrH3FY8JO8MUJzZ4HzokdzTcE2FTRfY+AlN8so7i84zwPVBy1dC7wXJI2s4DjzKxTbJPUcbFxWWNmo4DrgBPcfWuKNul8phpUwj7Tk0lez9vAgWbWO9YzP4Pwd8q2Y4H33b0k2cRsLu8q1nvZ/Xw39NEiUb8Bwwnd00XAgthtDDABmBBrczmwhHB00Fzg+xGoe79YPQtjtd0QGx9ftwFTCUc4vQsUZrvuWF25hMDpEDcucsubEKBrgO8I3xovBDoD/wcsj93vGWtbCDwU99wLgBWx2/kRqHsFYb9BxWf8t7G2+wAzq/pMZbnuJ2Kf3UWElWe3xLpjw2MIR6J9EIW6Y+MfrfhMx7WNxPKuYr2X1c+3TnUkIiKRpE18IiISSQooERGJJAWUiIhEkgJKREQiSQElIiKRpIASEZFIUkCJiEgk/X/gQ85pid/y1AAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "\n",
    "loss = history.history['loss']\n",
    "val_loss = history.history['val_loss']\n",
    "\n",
    "epochs = range(1, len(loss) + 1)\n",
    "\n",
    "plt.figure()\n",
    "\n",
    "plt.plot(epochs, loss, 'bo', label='Training loss')\n",
    "plt.plot(epochs, val_loss, 'b', label='Validation loss')\n",
    "plt.title('Training and validation loss')\n",
    "plt.legend()\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "일부 검증 손실은 학습을 사용하지 않은 기준점에 가깝지만 안정적이지 못합니다. 앞서 기준 모델을 만든 것이 도움이 됩니다. 이 문제는 기준 모델의 성능을 앞지르기가 쉽지 않습니다. 우리가 적용한 상식에는 머신 러닝 모델이 찾지 못한 핵심 정보가 많이 포함되어 있습니다.\n",
    "\n",
    "간단하고 괜찮은 성능을 내는 모델(상식 수준의 기준 모델)이 데이터와 타깃을 매핑할 수 있다면 왜 훈련한 모델은 이를 찾지 못하고 성능이 낮을까요? 훈련 과정이 찾는 것은 간단한 이 모델이 아니기 때문입니다. 문제 해결을 위해 탐색하는 모델의 공간, 즉 가설 공간은 우리가 매개변수로 설정한 두 개 층을 가진 네트워크의 모든 가능한 가중치의 조합입니다. 이 네트워크는 이미 매우 복잡합니다. 복잡한 모델 공간에서 해결책을 탐색할 때 간단하고 괜찮은 성능을 내는 모델은 찾지 못할 수 있습니다. 심지어 기술적으로 봤을 때 이 가설 공간에 포함되어 있을 때 조차도 말이죠. 이것이 일반적으로 머신 러닝이 가진 심각한 제약 사항입니다. 학습 알고리즘이 특정한 종류의 간단한 모델을 찾도록 하드코딩되지 않았다면, 모델 파라미터를 학습하는 방법은 간단한 문제를 위한 간단한 해결책을 찾지 못할 수 있습니다."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 첫 번째 순환 신경망\n",
    "\n",
    "첫 번째 완전 연결 네트워크는 잘 작동하지 않았습니다. 그렇다고 이 문제에 머신 러닝이 적합하지 않다는 뜻은 아닙니다. 앞선 모델은 시계열 데이터를 펼쳤기 때문에 입력 데이터에서 시간 개념을 잃어버렸습니다. 대신 인과 관계와 순서가 의미가 있는 시퀀스 데이터 그대로 사용해 보겠습니다. 이런 시퀀스 데이터에 아주 잘 들어맞는 순환 시퀀스 처리 모델을 시도해 보겠습니다. 이 모델은 앞선 모델과 달리 데이터 포인터의 시간 순서를 사용합니다.\n",
    "\n",
    "이전 절에서 소개한 `LSTM` 층 대신에 2014년에 정준영 등이 개발한 `GRU` 층을 사용하겠습니다. GRU 층은 LSTM과 같은 원리로 작동하지만 조금 더 간결하고 그래서 계산 비용이 덜 듭니다(LSTM 만큼 표현 학습 능력이 높지는 않을 수 있습니다). 계산 비용과 표현 학습 능력 사이의 트레이드오프(trade-off)는 머신 러닝 어디에서나 등장합니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/20\n",
      "500/500 [==============================] - 104s 207ms/step - loss: 0.2994 - val_loss: 0.2664\n",
      "Epoch 2/20\n",
      "500/500 [==============================] - 103s 206ms/step - loss: 0.2850 - val_loss: 0.2690\n",
      "Epoch 3/20\n",
      "500/500 [==============================] - 103s 207ms/step - loss: 0.2793 - val_loss: 0.2645\n",
      "Epoch 4/20\n",
      "500/500 [==============================] - 103s 207ms/step - loss: 0.2722 - val_loss: 0.2646\n",
      "Epoch 5/20\n",
      "500/500 [==============================] - 103s 207ms/step - loss: 0.2677 - val_loss: 0.2679\n",
      "Epoch 6/20\n",
      "500/500 [==============================] - 103s 206ms/step - loss: 0.2633 - val_loss: 0.2628\n",
      "Epoch 7/20\n",
      "500/500 [==============================] - 103s 206ms/step - loss: 0.2614 - val_loss: 0.2731\n",
      "Epoch 8/20\n",
      "500/500 [==============================] - 103s 206ms/step - loss: 0.2551 - val_loss: 0.2782\n",
      "Epoch 9/20\n",
      "500/500 [==============================] - 103s 207ms/step - loss: 0.2533 - val_loss: 0.2675\n",
      "Epoch 10/20\n",
      "500/500 [==============================] - 103s 207ms/step - loss: 0.2480 - val_loss: 0.2719\n",
      "Epoch 11/20\n",
      "500/500 [==============================] - 103s 206ms/step - loss: 0.2442 - val_loss: 0.2720\n",
      "Epoch 12/20\n",
      "500/500 [==============================] - 103s 207ms/step - loss: 0.2389 - val_loss: 0.2738\n",
      "Epoch 13/20\n",
      "500/500 [==============================] - 103s 207ms/step - loss: 0.2351 - val_loss: 0.2756\n",
      "Epoch 14/20\n",
      "500/500 [==============================] - 103s 206ms/step - loss: 0.2308 - val_loss: 0.2800\n",
      "Epoch 15/20\n",
      "500/500 [==============================] - 103s 206ms/step - loss: 0.2267 - val_loss: 0.2849\n",
      "Epoch 16/20\n",
      "500/500 [==============================] - 103s 206ms/step - loss: 0.2240 - val_loss: 0.2882\n",
      "Epoch 17/20\n",
      "500/500 [==============================] - 103s 207ms/step - loss: 0.2170 - val_loss: 0.2929\n",
      "Epoch 18/20\n",
      "500/500 [==============================] - 103s 206ms/step - loss: 0.2168 - val_loss: 0.2936\n",
      "Epoch 19/20\n",
      "500/500 [==============================] - 103s 206ms/step - loss: 0.2120 - val_loss: 0.3002\n",
      "Epoch 20/20\n",
      "500/500 [==============================] - 103s 206ms/step - loss: 0.2077 - val_loss: 0.2967\n"
     ]
    }
   ],
   "source": [
    "from keras.models import Sequential\n",
    "from keras import layers\n",
    "from keras.optimizers import RMSprop\n",
    "\n",
    "model = Sequential()\n",
    "model.add(layers.GRU(32, input_shape=(None, float_data.shape[-1])))\n",
    "model.add(layers.Dense(1))\n",
    "\n",
    "model.compile(optimizer=RMSprop(), loss='mae')\n",
    "history = model.fit_generator(train_gen,\n",
    "                              steps_per_epoch=500,\n",
    "                              epochs=20,\n",
    "                              validation_data=val_gen,\n",
    "                              validation_steps=val_steps)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "결과를 확인해 보죠:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAAEYCAYAAAAJeGK1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3Xl8VNX5x/HPwyayKBRUFJRFrSyKiJG6oKC1FlyAKi4IdS/ibm39iWKtUnlVwaqlUhWtthWUolZL3XBD0SpIgBB2QQWNIEQUBEEg5Pn9cW5gCJMwgUlmyff9es0rM/eee+eZO5N55px77jnm7oiIiKSbGqkOQEREJB4lKBERSUtKUCIikpaUoEREJC0pQYmISFpSghIRkbSkBCVpwcxqmtk6MzsomWVTycwOMbOkX8dhZqea2ZKYxwvN7MREyu7Ccz1uZrft6vbl7PduM/t7svcr2aVWqgOQzGRm62Ie1gM2Aluix1e6+9iK7M/dtwANkl22OnD3w5KxHzO7Ahjg7t1j9n1FMvYtsiuUoGSXuPvWBBH9Qr/C3d8sq7yZ1XL3oqqITUSyg5r4pFJETTj/MrNnzGwtMMDMjjOzKWa22syWm9lIM6sdla9lZm5mraLHY6L1r5rZWjP70MxaV7RstL6nmX1sZmvM7C9m9j8zu6SMuBOJ8UozW2xm35rZyJhta5rZA2a2ysw+AXqUc3xuN7NxpZaNMrP7o/tXmNn86PV8EtVuytpXgZl1j+7XM7OnotjmAkfHed5Po/3ONbNe0fIjgIeAE6Pm069jju2dMdsPil77KjN70cz2T+TY7IyZ9YniWW1mb5vZYTHrbjOzZWb2nZktiHmtx5rZjGj5CjMbkejzSYZwd910260bsAQ4tdSyu4FNwFmEH0J7AscAPyHU3NsAHwPXRuVrAQ60ih6PAb4GcoDawL+AMbtQdl9gLdA7WncTsBm4pIzXkkiM/wH2BloB35S8duBaYC7QAmgCTA7/YnGfpw2wDqgfs++VQE70+KyojAGnABuAjtG6U4ElMfsqALpH9+8D3gEaAy2BeaXKngfsH70nF0Yx7BetuwJ4p1ScY4A7o/unRTF2AuoCfwXeTuTYxHn9dwN/j+63i+I4JXqPbouOe22gA7AUaBaVbQ20ie5PA/pF9xsCP0n1/4Juyb2pBiWV6X13/6+7F7v7Bnef5u5T3b3I3T8FRgPdytn+OXfPdffNwFjCF2NFy54J5Ln7f6J1DxCSWVwJxvhHd1/j7ksIyaDkuc4DHnD3AndfBdxTzvN8CswhJE6AnwGr3T03Wv9fd//Ug7eBt4C4HSFKOQ+4292/dfelhFpR7POOd/fl0XvyNOHHRU4C+wXoDzzu7nnu/gMwGOhmZi1iypR1bMpzATDB3d+O3qN7gL0IPxSKCMmwQ9RM/Fl07CD80DjUzJq4+1p3n5rg65AMoQQllemL2Adm1tbMXjazr8zsO2Ao0LSc7b+Kub+e8jtGlFX2gNg43N0JNY64Eowxoeci/PIvz9NAv+j+hYTEWhLHmWY21cy+MbPVhNpLeceqxP7lxWBml5jZrKgpbTXQNsH9Qnh9W/fn7t8B3wLNY8pU5D0ra7/FhPeoubsvBH5DeB9WRk3GzaKilwLtgYVm9pGZnZ7g65AMoQQllal0F+tHCbWGQ9x9L+AOQhNWZVpOaHIDwMyM7b9QS9udGJcDB8Y83lk3+H8Bp0Y1kN6EhIWZ7Qk8B/yR0PzWCHg9wTi+KisGM2sDPAxcBTSJ9rsgZr876xK/jNBsWLK/hoSmxC8TiKsi+61BeM++BHD3Me5+AqF5rybhuODuC939AkIz7p+A582s7m7GImlECUqqUkNgDfC9mbUDrqyC53wJ6GxmZ5lZLeAGYJ9KinE8cKOZNTezJsAt5RV29xXA+8CTwEJ3XxSt2gOoAxQCW8zsTOCnFYjhNjNrZOE6sWtj1jUgJKFCQq6+glCDKrECaFHSKSSOZ4DLzayjme1BSBTvuXuZNdIKxNzLzLpHz30z4bzhVDNrZ2YnR8+3IbptIbyAX5pZ06jGtSZ6bcW7GYukESUoqUq/AS4mfPk8SqhBVKooCZwP3A+sAg4GZhKu20p2jA8TzhXNJpzAfy6BbZ4mdHp4Oibm1cCvgRcIHQ36EhJtIn5PqMktAV4F/hmz33xgJPBRVKYtEHve5g1gEbDCzGKb6kq2f43Q1PZCtP1BhPNSu8Xd5xKO+cOE5NkD6BWdj9oDGE44b/gVocZ2e7Tp6cB8C71E7wPOd/dNuxuPpA8LTfIi1YOZ1SQ0KfV19/dSHY+IlE01KMl6ZtbDzPaOmol+R+gZ9lGKwxKRnVCCkuqgK/ApoZmoB9DH3ctq4hORNKEmPhERSUuqQYmISFpKu8FimzZt6q1atUp1GCIiUkmmT5/+tbuXd7kHkIYJqlWrVuTm5qY6DBERqSRmtrNRVgA18YmISJpSghIRkbSkBCUiImlJCUpERNKSEpSIiKQlJSgREUlLCSWoaCyzhWa22MwGx1k/yMxmm1memb1vZu1j1t0abbfQzH6ezOBFRCR77TRBRaM/jwJ6Emav7BebgCJPu/sR7t6JMDT+/dG27QnTOXcgjIH212h/IiIi5UqkBtUFWOzun0ZzrYwjzP65VTT1c4n6bJuZszcwzt03uvtnwOJof5Vm7Fho1Qpq1Ah/x47d2RYiItlr/Xq49lq49Vb44otUR1MxiSSo5kDsyyogzpTZZnaNmX1CqEFdX8FtB5pZrpnlFhYWJhr7DsaOhYEDYelScA9/Bw5UkhKR6qmgAE48Ef76Vxg+HFq3hn794KMMmWwmkQRlcZbtMAS6u49y94MJ01yXzHiZ6Laj3T3H3XP22WenwzOVaciQ8Gsh1vr1YbmISHUydSoccwwsWgQTJsAnn8CNN8Irr8BPfgLHHw/PPgtFRamOtGyJJKgC4MCYxy0IM5KWZRzQZxe33S2ff16x5SIi2WjMGOjWDfbcEz78EM48M5zyuO++UKv6859hxQo47zw4+OCwfPXqVEe9o0QS1DTgUDNrbWZ1CJ0eJsQWMLNDYx6eASyK7k8ALjCzPcysNXAolTiT6UEHVWy5iEg2KS6GwYPhl7+EY48NTXkdOmxfpmFDuP56+PhjePHF0Ox3883QogVcdx0sXpya2OPZaYJy9yLgWmAiMB8Y7+5zzWyomfWKil1rZnPNLA+4Cbg42nYuMB6YB7wGXOPuWyrhdQAwbBjUq7f9snr1wnIRkWy2di306QP33hvOvb/+OjRtWnb5mjWhd2945x2YMQPOOQcefRR+/GPo1QsmTQrn8lMp7WbUzcnJ8d2ZbmPs2HDO6fPPQ81p2DDo3z+JAYqIpJnPPgtJZf58ePBBuOYasHg9AHbiq69Ch4qHH4avv4ZOncJ5qwsugD32SF68Zjbd3XN2Wi7bEpSISHUyeXKo/RQVwfjx8LOf7f4+N2yAp5+GBx6AuXNhv/3g6qth0CDYd9/d33+iCUpDHYmIZKjHH4ef/hSaNAm99pKRnCB0rrj8cpg9OzQVHn00/P73oVXq5puT8xyJUIISEckwRUWh6e1Xv4JTToEpU8K5o2QzC0nv5ZdD8+Fll0GjRsl/nrKk3ZTvIiJSttWr4fzzQ83mxhthxAioVQXf5G3bhvNTVUkJSkQkQ3z8MZx1VugU8dhjcMUVqY6ocilBiYhkgDfeCBfW1qoFb74JJ52U6ogqn85BiYikMXd46CHo2TNcTDttWvVITqAEJSKStjZtCl27r7sOzjgDPvggDFlUXaiJT0QkRdxh1SpYvjz+bc6c0Hvu1lvh7rvDNELViRKUiEiSbdkCK1fumHCWLdv+8VdfwebNO26/116w//7hdscdYSSH6kgJSkQkSbZsCePZ/e538M03O65v0mRb4mnbdtv9ktsBB0CzZlC/ftXHno6UoEREkuD998O5orw8OPlk6Ns3JJyS5LPffskdz646UIISEdkNy5fD//1fmIPpwAPDJIDnnLNrg7XK9qrZKTcRkeTYtClM9PfjH4dBWocMCR0a+vZVckoW1aBERCrojTfCpH8LFoTu3w8+CIcckuqoso9qUCIiCVq6NNSQTjst9L7773/hpZeUnCqLEpSIyE788AP84Q/Qrh288kq4JmnOHDjzzFRHlt3UxCciUgb3UEv69a/h00/h3HPDeaeDDkp1ZNWDalAiInEsWhTOL/XuHbqHv/lm6Ayh5FR1lKBERGKsWxeGFjr88HBt0/33w6xZYeZaqVpq4hMRIYwCMX58mNL8yy/hoovg3nvDyA6SGkpQIlKtrVwJjz8ehij6/HM46qiQqI4/PtWRiRKUiFQ77vDhh2EK82efDRfdnnJKaM7r0wdq1kx1hAJKUCJSjaxfD08/DaNGhTHz9toLrrwSrroqdCGX9KIEJSJZb9GiUFv6+99h9erQAeLhh2HAAGjQINXRSVmUoEQkK23ZAi+/HGpLr78OtWqFQVyvuQa6dtV4eZlA3czLMHZsmFq5Ro3wd+zYVEckIokoLIQ//hHatAnXMM2dC0OHhg4Q48bBiScqOWUK1aDiGDsWBg4M7dUQxt8aODDc798/dXGJSHzuMHVqqC2NHx86PZx8cuj00KsX1K6d6ghlV6gGFceQIduSU4n168NyEUkv770HxxwDxx0H//lP+DE5bx68/XZo0lNyylyqQcXx+ecVWy4iVW/VqjBR4BNPQMuW6vSQjVSDiqOssbY0BpdI6rnDP/4BbdvCP/8ZktTcuTBokJJTtlGCimPYMKhXb/tl9eqF5SKSOgsWhAtqL7kEDj0UZswIwxHVr5/qyKQyKEHF0b8/jB4dmg3Mwt/Ro9VBQiRVfvgB7rgDjjwyXGD76KNhINcjjkh1ZFKZdA6qDP37KyGJpIM33wwjPSxeHP4n//Qn2G+/VEclVUE1KBFJSytWhE4PP/tZePzGGzBmjJJTdaIEJSJppbg4NKm3bRsGcr3jDpg9G049NdWRSVVTE5+IpI3Zs0NvvA8+gO7dQ9fxtm1THZWkSkI1KDPrYWYLzWyxmQ2Os/4mM5tnZvlm9paZtYxZN9zM5prZfDMbaaZBRkRke99/D7fcAp07w8cfh27kb7+t5FTd7TRBmVlNYBTQE2gP9DOz9qWKzQRy3L0j8BwwPNr2eOAEoCNwOHAM0C1p0YtIxnv5ZejQAYYPD7PYLlgQ/uqnrCRSg+oCLHb3T919EzAO6B1bwN0nuXvJ4EBTgBYlq4C6QB1gD6A2sCIZgWcKDTorEt/SpdC3L5x5ZrjOcPJk+NvfoEmTVEcm6SKRBNUc+CLmcUG0rCyXA68CuPuHwCRgeXSb6O7zS29gZgPNLNfMcgsLCxONPe2VDDq7dGm4+r1k0FklKanO1qyBwYPhsMNC7WnYsHBt04knpjoySTeJJKh4FW2PW9BsAJADjIgeHwK0I9SomgOnmNlJO+zMfbS757h7zj777JNo7GlPg86KbFNUFCYNPOSQMPrDeeeF80233QZ16qQ6OklHiSSoAuDAmMctgGWlC5nZqcAQoJe7b4wW/wKY4u7r3H0doWZ17O6FnDk06KxIaD146aUw6sM114TzTbm5YRy9Aw/c+fZSfSWSoKYBh5pZazOrA1wATIgtYGZHAY8SktPKmFWfA93MrJaZ1SZ0kNihiS9badDZqjVtGkyalOooJFZeXrh+6ayzwgy3L74Y3qOjj051ZJIJdpqg3L0IuBaYSEgu4919rpkNNbNeUbERQAPgWTPLM7OSBPYc8AkwG5gFzHL3/yb7RaQrDTpbNdatgxtugJ/8BE47DT76KNURyZdfwqWXhm7jeXnw5z/DnDlhhlv1zpNEmXvc00kpk5OT47m5uakOI2nGjg3nnD7/PNSchg3TGH/J9Oab8KtfwZIlYby2l1+GmjVh5kzYe+9UR1f9fP89jBgRbkVFcN114fPfuHGqI5N0YmbT3T1nZ+U01FEl698/fHkWF4e/Sk7JsXo1XHFFGKetdu3QRfmvf4Vnngk/Bq68Mpz7kKqxZUuYOPDQQ+Guu+CMM2D+fLjvPiUn2XVKUJJxJkwIJ9qffDKMPjBr1rYuyscfD3/4A/zrX+ELUyrfm2+GprzLLw9T0/zvfzB+PLRpk+rIJNMpQUnGKCyEfv3CeYwmTWDqVLjnHthzz+3L3XIL/PSnoXlp3rzUxFodzJsXako/+xl89x2MGxfG0Dv++FRHJtlCCUrSnnv48mvfHp5/PjQh5eZCThkt2DVqwFNPhem/zz8fNmyo2niz2fr18M47YUDXjh3DpIHDh4fmvPPPVwcISS6NZi5p7csv4eqrQ7Nely5hKJzDD9/5dvvvH66z6dkTfvObcH5KKm7ZstBkV3LLywudH2rWDEnq97+HLLq2XtKMEpSkJfeQjH77W9i4MZxsv/HG8MWYqB494OabQ4+yn/4Uzjmn8uLNBlu2hK7gsQlp6dKwbs89ww+E//u/0IR33HHwox+lNl7JfupmLmnns8/CmIVvvgndusHjj4fhcXbFpk3QtWsYUicvLwzYK8HateE8XkkymjIlLINQAz3hhHA7/ng46qjQW1IkGRLtZq4alKSN4mJ46CG49dZQU3r44ZCoauzGmdI6dcL5q6OOggsvhHffrZ5ftO7wxRfb147y88MxNwvDEA0YsC0ptWyp80mSekpQkhYWLAjdlD/4IJw3evTR5I3T1qZNmEL8ggvgzjurx0gemzeHi5U/+GDb7csvw7r69eHYY+H220My+slPdFGzpCclqAyRzSNSPPZY6BJev37o2DBgQPJ/vZ9/fmgy/OMf4eSTw/hw2eTrr+HDD7clo2nTtvVebNkSTjopJKPjjgu972rpP18ygM5BZYCSeaVip+6oVy/UCjI9Sb3zTujAcOqpITntt1/lPdf69aFr+rffhvNRlflclam4ONQ4Y2tHCxeGdbVqhYtmjz9+W0JqXt7sbSIpkOg5KCWoDNCq1bbeVLFatgzDJ2Wq5cvDuaHGjcMArw0bVv5zzp4NxxwD3bvDK6/s3vmtZCguDs1xsbeioh0fFxZuqyF9+GFIshAuWC5JRscfHxJw6QuXRdKNOklkkWycV6qoKJwTWrsW3nqrapIThM4ADz4YBpa9//7Qjb0yfPNNaIb9739DN/mykk9Ffx+2bx+mST/++HA79FB1ZpDspQSVAQ46KH4NKpPnlbr99jDA65gxYVy9qnTlleF81K23hnMzXbokb98bN4aLgv/whzCg7ZlnhuuFatcOt1q1tt2vyOO99go1Pw28KtWJElQGGDYs/jmoTO2NNmFCmPJ70KDUnEMzCx0zpk0LtbhkTM3hDs89B4MHw6efws9/HoYA6tgxOTGLVEcaiy8D9O8fOkSUXJvSsmXmdpD49FO4+OIwo+oDD6QujsaNkzc1xwcfhHNA550XeiK+9lq4KTmJ7B4lqAyRDfNK/fADnHtuuP/ss1C3bmrj2d2pOT75JLyeE04I78nf/hZqYz//edJDFamWlKCkytx4I8yYEbqTt26d6miCW24JXdwrMjXHN9/Ar38N7drBq6+G0dUXLYLLLqvYWIEiUj4lqDKsWRN6YH31VaojyQ5PPRVGhxg8GM46K9XRbFMyNUfDhjufmmPjxtDz7+CDYeTI0FS5aBHccUdo2hOR5FKCivHdd+Gi2F69YN99w98f/zic0N+4MdXRJcfYseG6qho1wt+xYyv/OefMCR0iunULTWrpplmzUKubMwduumnH9e5hhth27cLUHcceGy70feyxMKiqiFQSd0+r29FHH+1Vae1a92eece/Tx32PPdzBvUUL95tucn/lFfdevcKyNm3cn3/evbi4SsNLqjFj3OvVC6+n5FavXlheWb77zv2ww9ybNXNfvrzynicZbr45HJPnntu27P333Y89Nizv2NF94sTUxSeSLYBcTyAfpDwhlb5VRYL6/nv38ePd+/Z133PPcBQOOMD9hhvc//c/9y1bti//+uvuHTqEct27u+flVXqIlaJly+2TU8mtZcvKeb7iYvfzznOvUcP9nXcq5zmSaeNG9y5d3Pfe2/3NN93POWfbZ+OJJ9yLilIdoUh2UIIqZf36UAM6//xttYj99nO/9lr3yZN3TEqlbd7sPmqUe5Mm4Qt34ED3FSsqJdSkW7/e/ckn4ycncDernOf9y1/C/u+5p3L2Xxk++cR9r71C3PXruw8d6r5uXaqjEskuiSaorB6Lb+NGmDgxdCOeMAHWrQvTU59zTjghfuKJFe919e23MHRomLeoXj343e/g+uvDvEPp5uOP4ZFH4O9/D3HXqhWG2iltr71CZ5BkjuE2ZUoYpaFHD3jxxdSPeVcRr78Ob7wRzjc1a5bqaESyT7UdLHbTpvDlMn58+GL87rswoObZZ4ek1K1bcqYaWLAgfIG98kqY7fVPfwq901I9LtrmzSEZP/xwGOOuVi34xS/C2HNffhkuSo0dkaJmzTDV96GHhplrTzpp92P4+uswonbNmqFbuYbnEZFYiSaolDfplb7tbhPfnDmheaZRI/fLLnN/7TX3TZt2a5flevVV93btwnOeeqr77NmV91zl+eIL9zvucN9//xDLgQe63333jh0TxowJ55zMwt8xY8I5tlatwnZXXum+evWux7Fli/vPf+5ep457bu7uvCIRyVZU53NQb70VTnhXlU2b3EeOdG/cOJyfuuoq98LCyn/eLVtCr7I+fdxr1gxJp2dP9wkTKn5Cf9260HOxRo3QKeDFF3ctpqFDw6fqkUd2bXsRyX7VOkGlytdfu193XUgWjRq5P/BA5dTevv7afcQI90MOCe9g06but9wSTvDvrqlT3Y84Iuz33HPdv/oq8W3feCMkyQEDMrs7vohUrkQTVNadg0oH8+aFCz4nToTDDgvnp7p3D+d6ioq23WIfl7eu5PH69fDvf4fzaxs3Qteu4dzSOefAHnskL/5Nm2DEiNAZpH79EP8ll5R/fq2gIEw+uN9+MHWqRlYQkbJV204S6cI9dKC46abQmy5ZGjaEX/4yjMxwxBHJ2288CxbAr34F778fxqt79FFo02bHcps3hwQ8axbk5kLbtpUbl4hkNiWoNLFpU5jWYeXK0KOuZs3wN5H78ZZ16AANGlRd/MXFITHdckuoyd19N9xwA4wbB0OGhOkqGjQIM+M+80yYX0lEpDya8j1N1KkTBhXNVDVqhGbEM8+Eq68OXetHjYJly8L0GRCSU61aoRlSRCRZMujySUmlAw8M11eNGxfmPipJTiWKikKNSkQkWZSgJGFm4WLn4uL46z//vGrjEZHspgQlFdayZfzlBx1U8X2lYvoPEckMSlBSYcOGhXEIY9WrF5ZXxNixMHAgLF0aej0uXRoeK0mJCChByS7o3x9Gjw41KbPwd/TosLwihgzZflxACI91LktEIMEEZWY9zGyhmS02s8Fx1t9kZvPMLN/M3jKzljHrDjKz181sflSmVfLCl1Tp3z90liguDn8rmpyg7HNWOpclIpBAgjKzmsAooCfQHuhnZu1LFZsJ5Lh7R+A5YHjMun8CI9y9HdAFWJmMwCXzlXXOalfOZYlI9kmkBtUFWOzun7r7JmAc0Du2gLtPcveSxpopQAuAKJHVcvc3onLrYspJNZesc1kikp0SSVDNgS9iHhdEy8pyOfBqdP/HwGoz+7eZzTSzEVGNbDtmNtDMcs0st7CwMNHYJcMl61yWiGSnREaSiDdEaNzxkcxsAJADdIvZ/4nAUcDnwL+AS4C/bbcz99HAaAhDHSUQk2SJ/v2VkEQkvkRqUAXAgTGPWwDLShcys1OBIUAvd98Ys+3MqHmwCHgR6Lx7IYuISHWQSIKaBhxqZq3NrA5wATAhtoCZHQU8SkhOK0tt29jM9okenwLM2/2wRUQk2+00QUU1n2uBicB8YLy7zzWzoWbWKyo2AmgAPGtmeWY2Idp2C/Bb4C0zm01oLnysEl6HiIhkGU23ISIiVSrR6TY0koSIiKQlJSgREUlLSlCSFTQqukj20Yy6kvFKRkUvGXi2ZFR00DVWIplMNSjJeBoVXSQ7KUFJxtOo6CLZSQlKMl4yR0XXuSyR9KEEJRlPM/yKZCclKMl4muFXJDtpJAmRSI0aoeZUmlmYOVhEkkMjSYhUkGb4FUkvSlAiEc3wK5JelKBEIprhVyS9aCQJkRia4VckfagGJSIiaUkJSkRE0pISlIiIpCUlKBERSUtKUCIikpaUoEQqgQadFdl96mYukmSaQFEkOVSDEkkyDTorkhxKUCJJpgkURZJDCUokyTTorEhyKEGJJJkGnRVJDiUokSTToLMiyaFefCKVQIPOiuw+1aBE0piup5LqTDUokTSl66mkulMNSiRN6Xoqqe6UoETSlK6nkupOCUokTel6KqnulKBE0pSup5LqTglKJE3peiqp7tSLTySN6Xoqqc5UgxIRkbSkBCUiImkpoQRlZj3MbKGZLTazwXHW32Rm88ws38zeMrOWpdbvZWZfmtlDyQpcRESy204TlJnVBEYBPYH2QD8za1+q2Ewgx907As8Bw0ut/wPw7u6HKyIi1UUiNaguwGJ3/9TdNwHjgN6xBdx9kruXXPM+BWhRss7Mjgb2A15PTsgiIlIdJJKgmgNfxDwuiJaV5XLgVQAzqwH8Cbi5vCcws4FmlmtmuYWFhQmEJCIVoUFnJRMl0s3c4izzuAXNBgA5QLdo0dXAK+7+hVm83UQ7cx8NjAbIycmJu28R2TUadFYyVSI1qALgwJjHLYBlpQuZ2anAEKCXu2+MFh8HXGtmS4D7gIvM7J7dilhEKkSDzkqmSqQGNQ041MxaA18CFwAXxhYws6OAR4Ee7r6yZLm7948pcwmhI8UOvQBFpPJo0FnJVDutQbl7EXAtMBGYD4x397lmNtTMekXFRgANgGfNLM/MJlRaxCJSIRp0VjKVuafXKZ+cnBzPzc1NdRgiWaP0OSgIg85qXD9JFTOb7u45OyunkSREspwGnZVMpcFiRaoBDTormUg1KBFJmK6nkqqkGpSIJETXU0lVUw1KRBKi66mkqilBiUhCdD2VVDUlKBFJiK6nkqqmBCUiCRk2LFw/FatevbBcpDIoQYlIQnQ9lVQ19eITkYTpeiqpSqpBiYhIWlKCEhGRtKQEJSIpoVEpZGeqI7hGAAARjklEQVR0DkpEqpxGpZBEqAYlIlVOo1JIIpSgRKTKaVQKSYQSlIhUOY1KIYlQghKRKqdRKSQRSlAiUuU0KoUkQr34RCQlNCqF7IxqUCIikpaUoEQko+mC3+ylJj4RyVi64De7qQYlIhlLF/xmNyUoEclYuuA3uylBiUjG0gW/2U0JSkQyli74zW5KUCKSsXTBb3ZTghKRjNa/PyxZAsXF4e+uJid1V08/6mYuItWeuqunJ9WgRKTaU3f19KQEJSLVnrqrpyclKBGp9tRdPT0pQYlItafu6ulJCUpEqj11V09P6sUnIoLmp0pHqkGJiEhaSihBmVkPM1toZovNbHCc9TeZ2Twzyzezt8ysZbS8k5l9aGZzo3XnJ/sFiIhIdtppgjKzmsAooCfQHuhnZu1LFZsJ5Lh7R+A5YHi0fD1wkbt3AHoAD5pZo2QFLyIi2SuRGlQXYLG7f+rum4BxQO/YAu4+yd1LLnObArSIln/s7oui+8uAlcA+yQpeRESyVyIJqjnwRczjgmhZWS4HXi290My6AHWAT+KsG2hmuWaWW1hYmEBIIiKS7RLpxWdxlnncgmYDgBygW6nl+wNPARe7e/EOO3MfDYwGyMnJ2WHfmzdvpqCggB9++CGBcCXV6tatS4sWLahdu3aqQxGRDJZIgioADox53AJYVrqQmZ0KDAG6ufvGmOV7AS8Dt7v7lF0JsqCggIYNG9KqVSvM4uVLSRfuzqpVqygoKKB169apDkdEMlgiTXzTgEPNrLWZ1QEuACbEFjCzo4BHgV7uvjJmeR3gBeCf7v7srgb5ww8/0KRJEyWnDGBmNGnSRLVdEdltO01Q7l4EXAtMBOYD4919rpkNNbNeUbERQAPgWTPLM7OSBHYecBJwSbQ8z8w67UqgSk6ZQ++VVGfJmldK81MlOJKEu78CvFJq2R0x908tY7sxwJjdCVBEJFMka14pzU8VZOVIEsn+5bFq1So6depEp06daNasGc2bN9/6eNOmTQnt49JLL2XhwoXllhk1ahRjk/QzqWvXruTl5SVlXyKSmGTNK6X5qYKsG4uvMn55NGnSZOuX/Z133kmDBg347W9/u10Zd8fdqVEjfs5/8sknd/o811xzza4FKCJpIVnzSml+qiDralBV+ctj8eLFHH744QwaNIjOnTuzfPlyBg4cSE5ODh06dGDo0KFby5bUaIqKimjUqBGDBw/myCOP5LjjjmPlytCv5Pbbb+fBBx/cWn7w4MF06dKFww47jA8++ACA77//nnPOOYcjjzySfv36kZOTs9Oa0pgxYzjiiCM4/PDDue222wAoKiril7/85dblI0eOBOCBBx6gffv2HHnkkQwYMCDpx0wkmyVrXinNTxVkXYKq6l8e8+bN4/LLL2fmzJk0b96ce+65h9zcXGbNmsUbb7zBvHnzdthmzZo1dOvWjVmzZnHcccfxxBNPxN23u/PRRx8xYsSIrcnuL3/5C82aNWPWrFkMHjyYmTNnlhtfQUEBt99+O5MmTWLmzJn873//46WXXmL69Ol8/fXXzJ49mzlz5nDRRRcBMHz4cPLy8pg1axYPPfTQbh4dkeolWfNKaX6qIOsSVFX/8jj44IM55phjtj5+5pln6Ny5M507d2b+/PlxE9See+5Jz549ATj66KNZsmRJ3H2fffbZO5R5//33ueCCCwA48sgj6dChQ7nxTZ06lVNOOYWmTZtSu3ZtLrzwQiZPnswhhxzCwoULueGGG5g4cSJ77703AB06dGDAgAGMHTtWF9qKVFCy5pXS/FRB1iWoqv7lUb9+/a33Fy1axJ///Gfefvtt8vPz6dGjR9zrgerUqbP1fs2aNSkqKoq77z322GOHMu5xB/EoU1nlmzRpQn5+Pl27dmXkyJFceeWVAEycOJFBgwbx0UcfkZOTw5YtWyr0fCLVXf/+sGQJFBeHv7uaVJK1n0yWdQkqlb88vvvuOxo2bMhee+3F8uXLmThxYtKfo2vXrowfPx6A2bNnx62hxTr22GOZNGkSq1atoqioiHHjxtGtWzcKCwtxd84991zuuusuZsyYwZYtWygoKOCUU05hxIgRFBYWsr70CT0RkSqSdb34IHUzY3bu3Jn27dtz+OGH06ZNG0444YSkP8d1113HRRddRMeOHencuTOHH3741ua5eFq0aMHQoUPp3r077s5ZZ53FGWecwYwZM7j88stxd8yMe++9l6KiIi688ELWrl1LcXExt9xyCw0bNkz6axARSYRVtMmosuXk5Hhubu52y+bPn0+7du1SFFF6KSoqoqioiLp167Jo0SJOO+00Fi1aRK1a6fVbQ++ZiJTFzKa7e87OymVdE1+2W7duHSeccAJHHnkk55xzDo8++mjaJScRSR+ZPGSSvtkyTKNGjZg+fXqqwxCRDJDpQyapBiUikqUyfcgkJSgRkSyV6UMmKUGJiGSpTB8ySQlKRCRLZfqQSUpQCejevfsOF90++OCDXH311eVu16BBAwCWLVtG3759y9x36W71pT344IPbXTB7+umns3r16kRCL9edd97Jfffdt9v7EZH0lOlDJilBJaBfv36MGzduu2Xjxo2jX79+CW1/wAEH8Nxzz+3y85dOUK+88gqNGjXa5f2JSPWRyUMmZVw38xtvhGTPw9epE0SzXMTVt29fbr/9djZu3Mgee+zBkiVLWLZsGV27dmXdunX07t2bb7/9ls2bN3P33XfTu3fv7bZfsmQJZ555JnPmzGHDhg1ceumlzJs3j3bt2rFhw4at5a666iqmTZvGhg0b6Nu3L3fddRcjR45k2bJlnHzyyTRt2pRJkybRqlUrcnNzadq0Kffff//W0dCvuOIKbrzxRpYsWULPnj3p2rUrH3zwAc2bN+c///kPe+65Z5mvMS8vj0GDBrF+/XoOPvhgnnjiCRo3bszIkSN55JFHqFWrFu3bt2fcuHG8++673HDDDUCY3n3y5MkacUJEkk41qAQ0adKELl268NprrwGh9nT++edjZtStW5cXXniBGTNmMGnSJH7zm9+UO6Drww8/TL169cjPz2fIkCHbXdM0bNgwcnNzyc/P59133yU/P5/rr7+eAw44gEmTJjFp0qTt9jV9+nSefPJJpk6dypQpU3jssce2Tr+xaNEirrnmGubOnUujRo14/vnny32NF110Effeey/5+fkcccQR3HXXXQDcc889zJw5k/z8fB555BEA7rvvPkaNGkVeXh7vvfdeuYlPRGRXZVwNqryaTmUqaebr3bs348aN21prcXduu+02Jk+eTI0aNfjyyy9ZsWIFzZo1i7ufyZMnc/311wPQsWNHOnbsuHXd+PHjGT16NEVFRSxfvpx58+Ztt760999/n1/84hdbR1Q/++yzee+99+jVqxetW7emU6dOQPlTekCYn2r16tV069YNgIsvvphzzz13a4z9+/enT58+9OnTB4ATTjiBm266if79+3P22WfTokWLRA6hiGSwsWPD9VOffx56AQ4bVvnNhapBJahPnz689dZbzJgxgw0bNtC5c2cAxo4dS2FhIdOnTycvL4/99tsv7hQbscxsh2WfffYZ9913H2+99Rb5+fmcccYZO91PeTW1kqk6oPwpPXbm5Zdf5pprrmH69OkcffTRFBUVMXjwYB5//HE2bNjAsccey4IFC3Zp3yKSGUpGpFi6FNy3jUhR2cMmKUElqEGDBnTv3p3LLrtsu84Ra9asYd9996V27dpMmjSJpUuXlrufk046ibHRuzpnzhzy8/OBMFVH/fr12XvvvVmxYgWvvvrq1m0aNmzI2rVr4+7rxRdfZP369Xz//fe88MILnHjiiRV+bXvvvTeNGzfmvffeA+Cpp56iW7duFBcX88UXX3DyySczfPhwVq9ezbp16/jkk0844ogjuOWWW8jJyVGCEslyqRqRIuOa+FKpX79+nH322dv16Ovfvz9nnXUWOTk5dOrUibZt25a7j6uuuopLL72Ujh070qlTJ7p06QKE2XGPOuooOnTosMNUHQMHDqRnz57sv//+252H6ty5M5dccsnWfVxxxRUcddRR5TbnleUf//jH1k4Sbdq04cknn2TLli0MGDCANWvW4O78+te/plGjRvzud79j0qRJ1KxZk/bt22+dHVhEslOqRqTQdBtSKfSeiWSPVq1Cs15pLVuGrusVpek2REQkKVI1IoUSlIiIlCtVI1JkzDmokqnJJf2lW7OxiOy+/v2rfhSKjKhB1a1bl1WrVumLLwO4O6tWraJu3bqpDkVEMlxG1KBatGhBQUEBhYWFqQ5FElC3bl1dvCsiuy0jElTt2rVp3bp1qsMQEZEqlBFNfCIiUv0oQYmISFpSghIRkbSUdiNJmFkhUP6AdumnKfB1qoOoIMVcNRRz1cnEuKtrzC3dfZ+dFUq7BJWJzCw3kWE70olirhqKuepkYtyKuXxq4hMRkbSkBCUiImlJCSo5Rqc6gF2gmKuGYq46mRi3Yi6HzkGJiEhaUg1KRETSkhKUiIikJSWoBJjZgWY2yczmm9lcM7shTpnuZrbGzPKi2x2piLU0M1tiZrOjmHLjrDczG2lmi80s38w6pyLOmHgOizmGeWb2nZndWKpMyo+1mT1hZivNbE7Msh+Z2Rtmtij627iMbS+Oyiwys4tTHPMIM1sQvfcvmFmjMrYt93NUmcqI+04z+zLmM3B6Gdv2MLOF0ed7cIpj/ldMvEvMLK+MbVNyrMv6nkvp59rdddvJDdgf6Bzdbwh8DLQvVaY78FKqY40T+xKgaTnrTwdeBQw4Fpia6phjYqsJfEW4qC+tjjVwEtAZmBOzbDgwOLo/GLg3znY/Aj6N/jaO7jdOYcynAbWi+/fGizmRz1EK4r4T+G0Cn59PgDZAHWBW6f/bqoy51Po/AXek07Eu63sulZ9r1aAS4O7L3X1GdH8tMB9ontqokqY38E8PpgCNzGz/VAcV+Snwibun3cgi7j4Z+KbU4t7AP6L7/wD6xNn058Ab7v6Nu38LvAH0qLRAY8SL2d1fd/ei6OEUIO3mSSnjWCeiC7DY3T91903AOMJ7VOnKi9nCzKvnAc9URSyJKud7LmWfayWoCjKzVsBRwNQ4q48zs1lm9qqZdajSwMrmwOtmNt3MBsZZ3xz4IuZxAemTfC+g7H/idDzW+7n7cgj/7MC+ccqk8/G+jFCbjmdnn6NUuDZqmnyijGandD3WJwIr3H1RGetTfqxLfc+l7HOtBFUBZtYAeB640d2/K7V6BqEp6kjgL8CLVR1fGU5w985AT+AaMzup1HqLs03Krz0wszpAL+DZOKvT9VgnIl2P9xCgCBhbRpGdfY6q2sPAwUAnYDmhyay0tDzWQD/Krz2l9Fjv5HuuzM3iLNvtY60ElSAzq01408a6+79Lr3f379x9XXT/FaC2mTWt4jB34O7Lor8rgRcIzR6xCoADYx63AJZVTXTl6gnMcPcVpVek67EGVpQ0j0Z/V8Ypk3bHOzqhfSbQ36MTCqUl8DmqUu6+wt23uHsx8FgZ8aTjsa4FnA38q6wyqTzWZXzPpexzrQSVgKjN+G/AfHe/v4wyzaJymFkXwrFdVXVRxo2pvpk1LLlPOCE+p1SxCcBFUW++Y4E1JdX5FCvzV2Y6HuvIBKCk99LFwH/ilJkInGZmjaNmqdOiZSlhZj2AW4Be7r6+jDKJfI6qVKnzpL8gfjzTgEPNrHVUI7+A8B6l0qnAAncviLcylce6nO+51H2uq7qnSCbegK6E6mo+kBfdTgcGAYOiMtcCcwk9haYAx6dB3G2ieGZFsQ2JlsfGbcAoQm+n2UBOGsRdj5Bw9o5ZllbHmpA8lwObCb8eLweaAG8Bi6K/P4rK5gCPx2x7GbA4ul2a4pgXE84dlHyuH4nKHgC8Ut7nKMVxPxV9XvMJX6D7l447enw6oTfaJ1UZd7yYo+V/L/kcx5RNi2Ndzvdcyj7XGupIRETSkpr4REQkLSlBiYhIWlKCEhGRtKQEJSIiaUkJSkRE0pISlIiIpCUlKBERSUv/D7frf3gA6fNCAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "loss = history.history['loss']\n",
    "val_loss = history.history['val_loss']\n",
    "\n",
    "epochs = range(1, len(loss) + 1)\n",
    "\n",
    "plt.figure()\n",
    "\n",
    "plt.plot(epochs, loss, 'bo', label='Training loss')\n",
    "plt.plot(epochs, val_loss, 'b', label='Validation loss')\n",
    "plt.title('Training and validation loss')\n",
    "plt.legend()\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "훨씬 좋군요! 상식 수준의 모델을 크게 앞질렀습니다. 이 그림은 시퀀스를 펼쳐서 처리하는 완전 연결 네트워크에 비해서 순환 네트워크가 이런 종류의 작업에 훨씬 뛰어나다는 것과 머신 러닝의 가치를 보여줍니다.\n",
    "\n",
    "새로운 검증 MAE는 0.265 이하(크게 과대적합되기 시작하는 곳)이고 정규화되기 전인 섭씨로 복원하면 MAE는 2.35°C입니다. 초기 에러 2.57°C보다는 확실히 낫지만 더 개선할 수 있을 것 같습니다."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 과대적합을 감소하기 위해 순환 드롭아웃 사용하기\n",
    "\n",
    "훈련 손실과 검증 손실 곡선을 보면 모델이 과대적합인지 알 수 있습니다. 몇 번의 에포크 이후에 훈련 손실과 검증 손실이 현저하게 벌어지기 시작합니다. 이런 현상을 해결하기 위해 잘 알려진 드롭아웃 기법을 이미 보았습니다. 훈련 데이터를 층에 주입할 때 데이터에 있는 우연한 상관 관계를 깨뜨리기 위해 입력 층의 유닛을 랜덤하게 끄는 기법입니다. 순환 신경망에 드롭아웃을 올바르게 적용하는 방법은 간단하지 않습니다. 순환 층 이전에 드롭아웃을 적용하면 규제에 도움이 되는 것보다 학습에 더 방해되는 것으로 오랫동안 알려졌습니다. 2015년 야린 갈이 베이지안 딥러닝에 관한 박사 논문에서 순환 네트워크에 적절하게 드롭아웃을 사용하는 방법을 알아 내었습니다. 타임스텝마다 랜덤하게 드롭아웃 마스크를 바꾸는 것이 아니라 동일한 드롭아웃 마스크(동일한 유닛의 드롭 패턴)를 모든 타임스텝에 적용해야 합니다. GRU이나 LSTM 같은 순환 게이트에 의해 만들어지는 표현을 규제하려면 순환 층 내부 계산에 사용된 활성화 함수에 타임스텝마다 동일한 드롭아웃 마스크를 적용해야 합니다(순환 드롭 아웃 마스크). 모든 타임스텝에 동일한 드롭아웃 마스크를 적용하면 네트워크가 학습 오차를 타임스템에 걸쳐 적절하게 전파시킬 것입니다. 타임스텝마다 랜덤한 드롭아웃 마스크를 적용하면 오차 신호가 전파되는 것을 방해하고 학습 과정에 해를 끼칩니다.\n",
    "\n",
    "야린 갈은 케라스를 사용해 연구를 하였고 케라스 순환 층에 이 기능을 구현하는 데 도움을 주었습니다. 케라스에 있는 모든 순환 층은 두 개의 드롭아웃 매개변수를 가지고 있습니다. `dropout`은 층의 입력에 대한 드롭아웃 비율을 정하는 부동 소수 값입니다. `recurrent_dropout`은 순환 상태의 드롭아웃 비율을 정합니다. GRU 층에 드롭아웃과 순환 드롭아웃을 적용해 과대적합에 어떤 영향을 미치는지 살펴보겠습니다. 드롭아웃으로 규제된 네트워크는 언제나 완전히 수렴하는 데 더 오래 걸립니다. 에포크를 두 배 더 늘려 네트워크를 훈련하겠습니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/40\n",
      "500/500 [==============================] - 117s 234ms/step - loss: 0.3403 - val_loss: 0.2780\n",
      "Epoch 2/40\n",
      "500/500 [==============================] - 117s 233ms/step - loss: 0.3171 - val_loss: 0.2775\n",
      "Epoch 3/40\n",
      "500/500 [==============================] - 117s 233ms/step - loss: 0.3091 - val_loss: 0.2706\n",
      "Epoch 4/40\n",
      "500/500 [==============================] - 116s 233ms/step - loss: 0.3029 - val_loss: 0.2700\n",
      "Epoch 5/40\n",
      "500/500 [==============================] - 117s 233ms/step - loss: 0.3009 - val_loss: 0.2698\n",
      "Epoch 6/40\n",
      "500/500 [==============================] - 117s 233ms/step - loss: 0.2983 - val_loss: 0.2697\n",
      "Epoch 7/40\n",
      "500/500 [==============================] - 117s 233ms/step - loss: 0.2969 - val_loss: 0.2674\n",
      "Epoch 8/40\n",
      "500/500 [==============================] - 117s 233ms/step - loss: 0.2957 - val_loss: 0.2692\n",
      "Epoch 9/40\n",
      "500/500 [==============================] - 117s 233ms/step - loss: 0.2936 - val_loss: 0.2689\n",
      "Epoch 10/40\n",
      "500/500 [==============================] - 116s 233ms/step - loss: 0.2936 - val_loss: 0.2670\n",
      "Epoch 11/40\n",
      "500/500 [==============================] - 117s 233ms/step - loss: 0.2898 - val_loss: 0.2667\n",
      "Epoch 12/40\n",
      "500/500 [==============================] - 117s 233ms/step - loss: 0.2891 - val_loss: 0.2651\n",
      "Epoch 13/40\n",
      "500/500 [==============================] - 117s 233ms/step - loss: 0.2861 - val_loss: 0.2679\n",
      "Epoch 14/40\n",
      "500/500 [==============================] - 117s 233ms/step - loss: 0.2869 - val_loss: 0.2667\n",
      "Epoch 15/40\n",
      "500/500 [==============================] - 117s 233ms/step - loss: 0.2852 - val_loss: 0.2643\n",
      "Epoch 16/40\n",
      "500/500 [==============================] - 117s 233ms/step - loss: 0.2841 - val_loss: 0.2656\n",
      "Epoch 17/40\n",
      "500/500 [==============================] - 117s 233ms/step - loss: 0.2827 - val_loss: 0.2650\n",
      "Epoch 18/40\n",
      "500/500 [==============================] - 117s 233ms/step - loss: 0.2821 - val_loss: 0.2675\n",
      "Epoch 19/40\n",
      "500/500 [==============================] - 117s 233ms/step - loss: 0.2801 - val_loss: 0.2669\n",
      "Epoch 20/40\n",
      "500/500 [==============================] - 117s 233ms/step - loss: 0.2792 - val_loss: 0.2623\n",
      "Epoch 21/40\n",
      "500/500 [==============================] - 117s 233ms/step - loss: 0.2787 - val_loss: 0.2641\n",
      "Epoch 22/40\n",
      "500/500 [==============================] - 117s 233ms/step - loss: 0.2795 - val_loss: 0.2662\n",
      "Epoch 23/40\n",
      "500/500 [==============================] - 117s 233ms/step - loss: 0.2766 - val_loss: 0.2693\n",
      "Epoch 24/40\n",
      "500/500 [==============================] - 117s 233ms/step - loss: 0.2757 - val_loss: 0.2689\n",
      "Epoch 25/40\n",
      "500/500 [==============================] - 117s 233ms/step - loss: 0.2769 - val_loss: 0.2646\n",
      "Epoch 26/40\n",
      "500/500 [==============================] - 117s 233ms/step - loss: 0.2761 - val_loss: 0.2666\n",
      "Epoch 27/40\n",
      "500/500 [==============================] - 116s 233ms/step - loss: 0.2745 - val_loss: 0.2669\n",
      "Epoch 28/40\n",
      "500/500 [==============================] - 117s 233ms/step - loss: 0.2747 - val_loss: 0.2652\n",
      "Epoch 29/40\n",
      "500/500 [==============================] - 117s 233ms/step - loss: 0.2743 - val_loss: 0.2662\n",
      "Epoch 30/40\n",
      "500/500 [==============================] - 117s 233ms/step - loss: 0.2732 - val_loss: 0.2643\n",
      "Epoch 31/40\n",
      "500/500 [==============================] - 116s 233ms/step - loss: 0.2728 - val_loss: 0.2689\n",
      "Epoch 32/40\n",
      "500/500 [==============================] - 117s 233ms/step - loss: 0.2715 - val_loss: 0.2674\n",
      "Epoch 33/40\n",
      "500/500 [==============================] - 117s 233ms/step - loss: 0.2711 - val_loss: 0.2656\n",
      "Epoch 34/40\n",
      "500/500 [==============================] - 116s 233ms/step - loss: 0.2701 - val_loss: 0.2660\n",
      "Epoch 35/40\n",
      "500/500 [==============================] - 116s 233ms/step - loss: 0.2697 - val_loss: 0.2695\n",
      "Epoch 36/40\n",
      "500/500 [==============================] - 116s 233ms/step - loss: 0.2688 - val_loss: 0.2652\n",
      "Epoch 37/40\n",
      "500/500 [==============================] - 117s 233ms/step - loss: 0.2697 - val_loss: 0.2694\n",
      "Epoch 38/40\n",
      "500/500 [==============================] - 117s 233ms/step - loss: 0.2691 - val_loss: 0.2650\n",
      "Epoch 39/40\n",
      "500/500 [==============================] - 117s 233ms/step - loss: 0.2681 - val_loss: 0.2653\n",
      "Epoch 40/40\n",
      "500/500 [==============================] - 117s 234ms/step - loss: 0.2671 - val_loss: 0.2653\n"
     ]
    }
   ],
   "source": [
    "from keras.models import Sequential\n",
    "from keras import layers\n",
    "from keras.optimizers import RMSprop\n",
    "\n",
    "model = Sequential()\n",
    "model.add(layers.GRU(32,\n",
    "                     dropout=0.2,\n",
    "                     recurrent_dropout=0.2,\n",
    "                     input_shape=(None, float_data.shape[-1])))\n",
    "model.add(layers.Dense(1))\n",
    "\n",
    "model.compile(optimizer=RMSprop(), loss='mae')\n",
    "history = model.fit_generator(train_gen,\n",
    "                              steps_per_epoch=500,\n",
    "                              epochs=40,\n",
    "                              validation_data=val_gen,\n",
    "                              validation_steps=val_steps)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAAEYCAYAAAAJeGK1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3Xt8VNW5//HPQ7jJTRDxqCAElaIBYsCIeqSiaD3g/VYVUfBWRKW1tbZSb1Uqv6p4lGJpK6dV24pSamulFos3Wi9tVVBAgSKIgBGLAUVF5BLy/P5YOzCESWYnmcxMku/79ZpXZu9Ze88zezLzzFp77bXM3REREck1zbIdgIiISDJKUCIikpOUoEREJCcpQYmISE5SghIRkZykBCUiIjlJCUpygpnlmdlGM+uezrLZZGYHm1nar+MwsxPNbGXC8lIz+2qcsrV4rl+a2Y213b6a/d5hZg+ne7/SuDTPdgDSMJnZxoTFNsAWYHu0fKW7T6vJ/tx9O9Au3WWbAnfvnY79mNkVwEXuflzCvq9Ix75FakMJSmrF3XckiOgX+hXu/lxV5c2subuXZSI2EWkc1MQn9SJqwvmdmT1mZp8DF5nZ0Wb2LzPbYGYfmtlkM2sRlW9uZm5m+dHyI9HjT5vZ52b2TzPrWdOy0ePDzOwdM/vUzO43s1fM7JIq4o4T45VmttzMPjGzyQnb5pnZfWa23szeBYZWc3xuNrPpldZNMbN7o/tXmNmS6PW8G9VuqtpXiZkdF91vY2a/jWJbBBye5HlXRPtdZGanR+v7AT8Fvho1n65LOLa3JWw/Jnrt683sT2a2X5xjk4qZnRnFs8HMXjCz3gmP3Whma8zsMzP7d8JrPcrM3ojWrzWziXGfTxoId9dNtzrdgJXAiZXW3QFsBU4j/BDaAzgCOJJQcz8QeAcYG5VvDjiQHy0/AqwDioEWwO+AR2pRdh/gc+CM6LHrgG3AJVW8ljgxPgnsCeQDH1e8dmAssAjoBnQGXgwfsaTPcyCwEWibsO+PgOJo+bSojAFDgC+BwuixE4GVCfsqAY6L7t8D/A3oBPQAFlcqex6wX/SeXBjF8F/RY1cAf6sU5yPAbdH9k6IYi4DWwM+AF+IcmySv/w7g4ej+oVEcQ6L36MbouLcA+gCrgH2jsj2BA6P7rwPDo/vtgSOz/VnQLb031aCkPr3s7n9293J3/9LdX3f3V929zN1XAFOBwdVs/7i7z3X3bcA0whdjTcueCsx39yejx+4jJLOkYsb4Y3f/1N1XEpJBxXOdB9zn7iXuvh64s5rnWQG8TUicAF8DNrj73OjxP7v7Cg9eAJ4HknaEqOQ84A53/8TdVxFqRYnPO8PdP4zek0cJPy6KY+wXYATwS3ef7+6bgXHAYDPrllCmqmNTnQuAme7+QvQe3Ql0IPxQKCMkwz5RM/F70bGD8EOjl5l1dvfP3f3VmK9DGgglKKlP7ycumNkhZvYXM/uPmX0GjAf2rmb7/yTc30T1HSOqKrt/Yhzu7oQaR1IxY4z1XIRf/tV5FBge3b+QkFgr4jjVzF41s4/NbAOh9lLdsaqwX3UxmNklZrYgakrbABwSc78QXt+O/bn7Z8AnQNeEMjV5z6rabznhPerq7kuB7xLeh4+iJuN9o6KXAgXAUjN7zcxOjvk6pIFQgpL6VLmL9QOEWsPB7t4BuJXQhFWfPiQ0uQFgZsauX6iV1SXGD4EDEpZTdYP/HXBiVAM5g5CwMLM9gMeBHxOa3zoCz8SM4z9VxWBmBwI/B64COkf7/XfCflN1iV9DaDas2F97QlPiBzHiqsl+mxHesw8A3P0Rdz+G0LyXRzguuPtSd7+A0Iz7v8AfzKx1HWORHKIEJZnUHvgU+MLMDgWuzMBzPgUMMLPTzKw5cC3QpZ5inAF828y6mlln4IbqCrv7WuBl4CFgqbsvix5qBbQESoHtZnYqcEINYrjRzDpauE5sbMJj7QhJqJSQq68g1KAqrAW6VXQKSeIx4HIzKzSzVoRE8ZK7V1kjrUHMp5vZcdFzf49w3vBVMzvUzI6Pnu/L6Lad8AIuNrO9oxrXp9FrK69jLJJDlKAkk74LjCJ8+TxAqEHUqygJnA/cC6wHDgLeJFy3le4Yf044V/QW4QT+4zG2eZTQ6eHRhJg3AN8BniB0NDiXkGjj+CGhJrcSeBr4TcJ+FwKTgdeiMocAiedtngWWAWvNLLGprmL7vxKa2p6Itu9OOC9VJ+6+iHDMf05InkOB06PzUa2AuwnnDf9DqLHdHG16MrDEQi/Re4Dz3X1rXeOR3GGhSV6kaTCzPEKT0rnu/lK24xGRqqkGJY2emQ01sz2jZqJbCD3DXstyWCKSghKUNAWDgBWEZqKhwJnuXlUTn4jkCDXxiYhITlINSkREclLODRa79957e35+frbDEBGRejJv3rx17l7d5R5ADiao/Px85s6dm+0wRESknphZqlFWADXxiYhIjlKCEhGRnKQEJSIiOSnnzkGJiFRl27ZtlJSUsHnz5myHIjG0bt2abt260aJFVcM7Vk8JSkQajJKSEtq3b09+fj5hYHrJVe7O+vXrKSkpoWfPnqk3SEJNfCLSYGzevJnOnTsrOTUAZkbnzp3rVNttdAlq2jTIz4dmzcLfadNSbSEiDYmSU8NR1/cqVoKKBttcambLzWxcksfHmNlbZjbfzF42s4JKj3c3s41mdn2dok1h2jQYPRpWrQL38Hf0aCUpEZGGKGWCiqYnmAIMI0yvPLxyAgIedfd+7l5EmLvl3kqP30eYm6Ze3XQTbNq067pNm8J6EZG6Wr9+PUVFRRQVFbHvvvvStWvXHctbt8abiurSSy9l6dKl1ZaZMmUK09L0y3rQoEHMnz8/LfvKtDidJAYCy919BYCZTSdMT724ooC7f5ZQvi0JU0eb2ZmEkaS/SEfA1Vm9umbrRaRxmzYt/EBdvRq6d4cJE2BEHaZY7Ny5844v+9tuu4127dpx/fW7Ngy5O+5Os2bJf/8/9NBDKZ/nmmuuqX2QjUicJr6uwPsJyyXRul2Y2TVm9i6hBvWtaF1bwrTXt1f3BGY22szmmtnc0tLSuLHvpnv3mq0XkcYrk03+y5cvp2/fvowZM4YBAwbw4YcfMnr0aIqLi+nTpw/jx4/fUbaiRlNWVkbHjh0ZN24chx12GEcffTQfffQRADfffDOTJk3aUX7cuHEMHDiQ3r17849//AOAL774gnPOOYfDDjuM4cOHU1xcnLKm9Mgjj9CvXz/69u3LjTfeCEBZWRkXX3zxjvWTJ08G4L777qOgoIDDDjuMiy66KO3HLI44CSrZWa7d5uhw9ynufhAhIVVMyXw7cJ+7b6zuCdx9qrsXu3txly4pxw+s0oQJ0KbNruvatAnrRaRpyXST/+LFi7n88st588036dq1K3feeSdz585lwYIFPPvssyxevHi3bT799FMGDx7MggULOProo3nwwQeT7tvdee2115g4ceKOZHf//fez7777smDBAsaNG8ebb75ZbXwlJSXcfPPNzJkzhzfffJNXXnmFp556innz5rFu3Treeust3n77bUaOHAnA3Xffzfz581mwYAE//elP63h0aidOgioBDkhY7kaYMrsq04Ezo/tHAneb2Urg28CNZja2FnHGMmIETJ0KPXqAWfg7dWrdqvQi0jBlusn/oIMO4ogjjtix/NhjjzFgwAAGDBjAkiVLkiaoPfbYg2HDhgFw+OGHs3LlyqT7Pvvss3cr8/LLL3PBBRcAcNhhh9GnT59q43v11VcZMmQIe++9Ny1atODCCy/kxRdf5OCDD2bp0qVce+21zJ49mz333BOAPn36cNFFFzFt2rRaX2hbV3ES1OtALzPraWYtgQuAmYkFzKxXwuIpwDIAd/+qu+e7ez4wCfh/7l6vqXjECFi5EsrLw18lJ5GmKdNN/m3btt1xf9myZfzkJz/hhRdeYOHChQwdOjTp9UAtW7bccT8vL4+ysrKk+27VqtVuZWo62WxV5Tt37szChQsZNGgQkydP5sorrwRg9uzZjBkzhtdee43i4mK2b99eo+dLh5QJyt3LgLHAbGAJMMPdF5nZeDM7PSo21swWmdl84DpgVL1FLCISQzab/D/77DPat29Phw4d+PDDD5k9e3ban2PQoEHMmDEDgLfeeitpDS3RUUcdxZw5c1i/fj1lZWVMnz6dwYMHU1pairvz9a9/ndtvv5033niD7du3U1JSwpAhQ5g4cSKlpaVsqtxemgGxhjpy91nArErrbk24f22MfdxW0+BERGqrovUknb344howYAAFBQX07duXAw88kGOOOSbtz/HNb36TkSNHUlhYyIABA+jbt++O5rlkunXrxvjx4znuuONwd0477TROOeUU3njjDS6//HLcHTPjrrvuoqysjAsvvJDPP/+c8vJybrjhBtq3b5/215CK1bSaWN+Ki4tdExaKSDJLlizh0EMPzXYYOaGsrIyysjJat27NsmXLOOmkk1i2bBnNm+fWEKvJ3jMzm+fuxam2za1XIiIisWzcuJETTjiBsrIy3J0HHngg55JTXTWuVyMi0kR07NiRefPmZTuMetXoBosVEZHGQQlKRERykhKUiIjkJCUoERHJSUpQIiIxHXfccbtddDtp0iSuvvrqardr164dAGvWrOHcc8+tct+pLrGZNGnSLhfMnnzyyWzYsCFO6NW67bbbuOeee+q8n3RTghIRiWn48OFMnz59l3XTp09n+PDhsbbff//9efzxx2v9/JUT1KxZs+jYsWOt95frlKBERGI699xzeeqpp9iyZQsAK1euZM2aNQwaNGjHdUkDBgygX79+PPnkk7ttv3LlSvr27QvAl19+yQUXXEBhYSHnn38+X3755Y5yV1111Y6pOn74wx8CMHnyZNasWcPxxx/P8ccfD0B+fj7r1q0D4N5776Vv37707dt3x1QdK1eu5NBDD+Ub3/gGffr04aSTTtrleZKZP38+Rx11FIWFhZx11ll88sknO56/oKCAwsLCHYPU/v3vf98xYWP//v35/PPPa31sk9F1UCLSIH3725DuiWKLiiD6bk+qc+fODBw4kL/+9a+cccYZTJ8+nfPPPx8zo3Xr1jzxxBN06NCBdevWcdRRR3H66adjlmzGIvj5z39OmzZtWLhwIQsXLmTAgAE7HpswYQJ77bUX27dv54QTTmDhwoV861vf4t5772XOnDnsvffeu+xr3rx5PPTQQ7z66qu4O0ceeSSDBw+mU6dOLFu2jMcee4z/+7//47zzzuMPf/hDtfM7jRw5kvvvv5/Bgwdz6623cvvttzNp0iTuvPNO3nvvPVq1arWjWfGee+5hypQpHHPMMWzcuJHWrVvX4GinphqUiEgNJDbzJTbvuTs33ngjhYWFnHjiiXzwwQesXbu2yv28+OKLOxJFYWEhhYWFOx6bMWMGAwYMoH///ixatCjlQLAvv/wyZ511Fm3btqVdu3acffbZvPTSSwD07NmToqIioPopPSDMT7VhwwYGDx4MwKhRo3jxxRd3xDhixAgeeeSRHSNWHHPMMVx33XVMnjyZDRs2pH0kC9WgRKRBqq6mU5/OPPNMrrvuOt544w2+/PLLHTWfadOmUVpayrx582jRogX5+flJp9hIlKx29d5773HPPffw+uuv06lTJy655JKU+6luTNWKqTogTNeRqomvKn/5y1948cUXmTlzJj/60Y9YtGgR48aN45RTTmHWrFkcddRRPPfccxxyyCG12n8yqkGJiNRAu3btOO6447jssst26Rzx6aefss8++9CiRQvmzJnDqlWrqt3Psccey7Ro/vm3336bhQsXAmGqjrZt27Lnnnuydu1ann766R3btG/fPul5nmOPPZY//elPbNq0iS+++IInnniCr371qzV+bXvuuSedOnXaUfv67W9/y+DBgykvL+f999/n+OOP5+6772bDhg1s3LiRd999l379+nHDDTdQXFzMv//97xo/Z3VUgxIRqaHhw4dz9tln79Kjb8SIEZx22mkUFxdTVFSUsiZx1VVXcemll1JYWEhRUREDBw4Ewuy4/fv3p0+fPrtN1TF69GiGDRvGfvvtx5w5c3asHzBgAJdccsmOfVxxxRX079+/2ua8qvz6179mzJgxbNq0iQMPPJCHHnqI7du3c9FFF/Hpp5/i7nznO9+hY8eO3HLLLcyZM4e8vDwKCgp2zA6cLppuQ0QaDE230fDUZboNNfGJiEhOUoISEZGcFCtBmdlQM1tqZsvNbFySx8eY2VtmNt/MXjazgmj9wGjdfDNbYGZnpfsFiEjTkmunJaRqdX2vUiYoM8sDpgDDgAJgeEUCSvCou/dz9yLgbuDeaP3bQHG0fijwgJmpY4aI1Err1q1Zv369klQD4O6sX7++ThfvxkkWA4Hl7r4CwMymA2cAO64cc/fPEsq3BTxavylhfeuK9SIitdGtWzdKSkooLS3NdigSQ+vWrenWrVutt4+ToLoC7ycslwBHVi5kZtcA1wEtgSEJ648EHgR6ABe7e1mSbUcDowG6d+9eg/BFpClp0aIFPXv2zHYYkiFxzkElG0hqt5qQu09x94OAG4CbE9a/6u59gCOAH5jZbvU9d5/q7sXuXtylS5f40YuISKMVJ0GVAAckLHcD1lRTfjpwZuWV7r4E+ALoW5MARUSkaYqToF4HeplZTzNrCVwAzEwsYGa9EhZPAZZF63tWdIowsx5Ab2BlGuIWEZFGLuU5KHcvM7OxwGwgD3jQ3ReZ2XhgrrvPBMaa2YnANuATYFS0+SBgnJltA8qBq919XX28EBERaVw01JGIiGSUhjoSEZEGTQlKRERykhKUiIjkJCUoERHJSUpQIiKSk5SgREQkJylBiYhITlKCEhGRnKQEJSIiOUkJSkREcpISlIiI5CQlKBERyUlKUCIikpOUoEREJCcpQYmISE5SghIRkZykBCUiIjlJCUpERHJSrARlZkPNbKmZLTezcUkeH2Nmb5nZfDN72cwKovVfM7N50WPzzGxIul+AiIg0TikTlJnlAVOAYUABMLwiASV41N37uXsRcDdwb7R+HXCau/cDRgG/TVvkIiLSqMWpQQ0Elrv7CnffCkwHzkgs4O6fJSy2BTxa/6a7r4nWLwJam1mruoctIiKNXfMYZboC7ycslwBHVi5kZtcA1wEtgWRNeecAb7r7liTbjgZGA3Tv3j1GSCIi0tjFqUFZknW+2wr3Ke5+EHADcPMuOzDrA9wFXJnsCdx9qrsXu3txly5dYoQkIiKNXZwEVQIckLDcDVhTRVkITYBnViyYWTfgCWCku79bmyBFRKTpiZOgXgd6mVlPM2sJXADMTCxgZr0SFk8BlkXrOwJ/AX7g7q+kJ2QREWkKUiYody8DxgKzgSXADHdfZGbjzez0qNhYM1tkZvMJ56FGVawHDgZuibqgzzezfdL/MkREpLEx991OJ2VVcXGxz507t972P20a3HQTrF4N3bvDhAkwYkS9PZ2IiFRiZvPcvThVuTi9+BqNadNg9GjYtCksr1oVlkFJSkQk1zSpoY5uumlncqqwaVNYLyIiuaVJJajVq2u2XkREsqdJJaiqrgHWtcEiIrmnSSWoCROgTZtd17VpE9aLiEhuaVIJasQImDoVevQAs/B36lR1kBARyUVNqhcfhGSkhCQikvuaVA1KREQaDiUoERHJSUpQIiKSk5SgREQkJylBiYhITlKCEhGRnKQEJSIiOUkJSkREcpISlIiI5CQlKBERyUlKUCIikpNiJSgzG2pmS81suZmNS/L4GDN7y8zmm9nLZlYQre9sZnPMbKOZ/TTdwYuISOOVMkGZWR4wBRgGFADDKxJQgkfdvZ+7FwF3A/dG6zcDtwDXpy9kERFpCuLUoAYCy919hbtvBaYDZyQWcPfPEhbbAh6t/8LdXyYkKhERkdjiJKiuwPsJyyXRul2Y2TVm9i6hBvWtmgRhZqPNbK6ZzS0tLa3JpvVi2jTIz4dmzcLfadOyHZGISNMTJ0FZknW+2wr3Ke5+EHADcHNNgnD3qe5e7O7FXbp0qcmmaTdtGoweDatWgXv4O3q0kpSISKbFSVAlwAEJy92ANdWUnw6cWZegsummm2DTpl3XbdoU1ouISObESVCvA73MrKeZtQQuAGYmFjCzXgmLpwDL0hdiZq1eXbP1IiJSP1JO+e7uZWY2FpgN5AEPuvsiMxsPzHX3mcBYMzsR2AZ8Aoyq2N7MVgIdgJZmdiZwkrsvTv9LSY/u3UOzXrL1IiKSOSkTFIC7zwJmVVp3a8L9a6vZNr+2wWXDhAnhnFNiM1+bNmG9iIhkjkaSqGTECJg6FXr0ALPwd+rUsF5ERDInVg2qqRkxQglJRCTbVIMSEZGcpAQlIiI5SQlKRERykhKUiIjkJCUoERHJSUpQIiKSk5SgREQkJylBiYhITlKCEhGRnKQEVUua1FBEpH5pqKNaqJjUsGJA2YpJDUFDJImIpItqULWgSQ1FROqfElQtaFJDEZH6pwRVC1VNXqhJDUVE0kcJqhYmTAiTGCZKNqmhOlKIiNSeElQtxJnUsKIjxapV4L6zI4WSlIhIPObuqQuZDQV+AuQBv3T3Oys9Pga4BtgObARGu/vi6LEfAJdHj33L3WdX91zFxcU+d+7cWryU3JKfH5JSZT16wMqVmY5GRCR3mNk8dy9OVS5lDcrM8oApwDCgABhuZgWVij3q7v3cvQi4G7g32rYAuADoAwwFfhbtr9FTRwoRkbqJ08Q3EFju7ivcfSswHTgjsYC7f5aw2BaoqJadAUx39y3u/h6wPNpfo6eOFCIidRMnQXUF3k9YLonW7cLMrjGzdwk1qG/VZNvGKG5HChERSS5OgrIk63Y7ceXuU9z9IOAG4OaabGtmo81srpnNLS0tjRFS7ovTkUJERKoWJ0GVAAckLHcD1lRTfjpwZk22dfep7l7s7sVdunSJEVLDMGJE6BBRXh7+JktO6oouIpJcnAT1OtDLzHqaWUtCp4eZiQXMrFfC4inAsuj+TOACM2tlZj2BXsBrdQ+7cVBXdBGRqqVMUO5eBowFZgNLgBnuvsjMxpvZ6VGxsWa2yMzmA9cBo6JtFwEzgMXAX4Fr3H17PbyOBklj+omIVC3WdVCZ1Fiug4qjWbNQc6rMLDQLiog0Rmm7DkrqT5yu6DpHJSJNlRJUFqXqiq5zVCLSlClBZVGqrug6RyUiTZkSVJZV1xU97nBJagYUkcZICSqHxT1HpWZAEWmMlKByWJzhktQMKCKNlRJUDoszXJJGTReRxqp5tgOQ6o0YUf34fd27J593SqOmi0hDpxpUA6dR00WksVKCauDijpoep6efegOKSC7RUEdNQEVPv8TOFG3a7JrI4pQREUmHuEMdKUE1Afn5yc9T9egRrr2KW0ZEJB00Fp/sEKenn3oDikiuUYJqAuJc8BunjIhIJilBNQFxevqpN6CI5BolqCYgTk+/uL0BRUQyRZ0kREQko9RJQkREGjQlKBERyUmxEpSZDTWzpWa23MzGJXn8OjNbbGYLzex5M+uR8NhdZvZ2dDs/ncFL5mm0CRHJlJQJyszygCnAMKAAGG5mBZWKvQkUu3sh8Dhwd7TtKcAAoAg4EviemXVIX/iSSZp7SkQyKU4NaiCw3N1XuPtWYDpwRmIBd5/j7hWD5PwL6BbdLwD+7u5l7v4FsAAYmp7QJdPizD2lGpaIpEucBNUVeD9huSRaV5XLgaej+wuAYWbWxsz2Bo4HDqi8gZmNNrO5Zja3tLQ0XuSScalGm1ANS0TSKU6CsiTrkvZNN7OLgGJgIoC7PwPMAv4BPAb8EyjbbWfuU9292N2Lu3TpEjN0ybRUo01odl8RSac4CaqEXWs93YA1lQuZ2YnATcDp7r6lYr27T3D3Inf/GiHZLatbyJItqUab0Hh+IpJOcRLU60AvM+tpZi2BC4CZiQXMrD/wACE5fZSwPs/MOkf3C4FC4Jl0BS+ZlWq0CY3nJyLplHLKd3cvM7OxwGwgD3jQ3ReZ2XhgrrvPJDTptQN+b2YAq939dKAF8FK07jPgInffrYlPGo7qpqCfMCH5nFIaz09EaiNlggJw91mEc0mJ625NuH9iFdttJvTkkyagInHddFNo1uvePSSn2oznN21aevYjIg1XrAQlEld1Nay4Ks/uW9EbsGL/ItI0aKgjyTnqDSgioAQlWZDqYl71BhQRUIKSDItzMa96A4oIKEFJhsVpvos7u6+GVRJp3JSgJKPiNN/Fmd1XwyqJNH6aUVcyKj8/JJPKevSAlSszvx8RyTzNqCs5KW7zXSpxO1KoGVCk4VKCkoyK03wXR5yOFGoGFGnY1MQnDVLli3kh1MQSk52aAUVyk5r4pFGLUxPT9VQiDZsSlDRYI0aEmlB5efhbuZkwbjOgzlGJ5CYlKGm0UnXIiHuOSklMJDuUoKTRStUMGOeiYXW0EMkedZKQJqtZs5B0KjMLzYagjhYi9UGdJERSiHOOSh0tRLJHCUqarDgXDWvgWpHsUYKSJitOV/V0jXwhIjUXK0GZ2VAzW2pmy81sXJLHrzOzxWa20MyeN7MeCY/dbWaLzGyJmU02M0vnCxCpi1Rd1eOOfKGefiLplzJBmVkeMAUYBhQAw82soFKxN4Fidy8EHgfujrb9b+AYoBDoCxwBDE5b9CIZkCqJZbKnnxKhNCVxalADgeXuvsLdtwLTgTMSC7j7HHev6LD7L6BbxUNAa6Al0ApoAaxNR+AiuSLuFPV1TS7q8i5NTZwE1RV4P2G5JFpXlcuBpwHc/Z/AHODD6Dbb3ZdU3sDMRpvZXDObW1paGjd2kZwQp6dfOpJL3EQo0ljESVDJzhklvXjKzC4CioGJ0fLBwKGEGlVXYIiZHbvbztynunuxuxd36dIlbuwiOSFOT7901LLU5V2amjgJqgQ4IGG5G7CmciEzOxG4CTjd3bdEq88C/uXuG919I6FmdVTdQhbJLXF6+qWjlqUu79LUxElQrwO9zKynmbUELgBmJhYws/7AA4Tk9FHCQ6uBwWbW3MxaEDpI7NbEJ9KQxenpl45alrq8S1OTMkG5exkwFphNSC4z3H2RmY03s9OjYhOBdsDvzWy+mVUksMeBd4G3gAXAAnf/c7pfRKKyMnjhBSgpST7I4f9kAAAWc0lEQVSMjUh9SNXTLx21rHRN9ijSUDS6sfiWLYOvfCXcb9sWevWC3r3Duopb796w555pClgkpmnTQm1o9epQc5owYdfkkmvj/qWKV6S24o7F1+gS1KZN8M9/wjvvwNKlO/9W/Lqt0Ls3PP88dK2uP6JIBsWZJTjufuqaWNIVi0gyTTZBVWXLFlixIiSrJUvgllvgW9+Ce+9N+1OJ1Fpdk0u6Ekuu1eakcVGCSmHkSPjjH8OHsHPnen86kYxIV2KJMxUJqBlQakfTbaTw/e/DF1/AlCnZjkQkfeJeK5VqVIs4vQ41soXUtyaboPr2hVNPhcmTQ6ISaQzSlVji9DrUyBZS35psggIYNw7Wr4cHH8x2JCLpka7EEqdLe7pGttAAuFIld8+p2+GHH+6ZNGiQe/fu7lu3ZvRpRerNI4+49+jhbhb+PvLIro+buYe60643s5o9T48eyffTo0f8WB55xL1Nm123b9Nm93LSuABzPUY+aNI1KAi1qNWrYfr0bEcikh6pLhpO15BJqWprcZoSMzUSvDRQcbJYJm+ZrkGVl7v37evep4/79u0ZfWqRrEhnraW6GlKcGlac2pxqWY0PqkHFYxZqUYsWwV/+ku1oROpfOodMqq62FuccVaZGgpeGqcleB5WorCwMibTffvDKK+FDKyJ1E+earDgXFse5JksjXzQsug6qBpo3h+uvD0MkvfxytqMRaRzi9CjM1EjwFVTLamDitANm8pbpc1AVvvjCvUsX95NPzsrTizRKqXrxxd1HqnNQOpfVsKBzUDXTpg1cey3MmgULF2Y7GpHGIVWPwrj7yGQtS3KHElSCq6+Gdu3grruyHYmIJMrEfFsV1AyYO5SgEnTqBFdeGa6JWrEi29GISFzpqmXFuXZLCSxz1Iuvkg8+gJ494Rvf0ECyIo1JnJ5+qXoeqrdgeqgXXy117Rqm4njwQVi7NtvRiEi6pGN8QZ3HyqxYCcrMhprZUjNbbmbjkjx+nZktNrOFZva8mfWI1h9vZvMTbpvN7Mx0v4h0+973wgSHP/lJtiMRkXSq6zBQOo+VWSkTlJnlAVOAYUABMNzMCioVexModvdC4HHgbgB3n+PuRe5eBAwBNgHPpDH+etG7N5x9Nvz4x+Gf69xz4c474bnn4JNPsh2diNSXVJ0t0jlPVrqSWKNOhqn6oQNHA7MTln8A/KCa8v2BV5KsHw1MS/V82boOqrJPPnGfONH9/PPdDzpo12snDjoorJ840f1Pf3J//nn3115zX7zY/f333TdscC8ry/YrEJHaqO7arTjXUsUd5T0d12Q11Gu7iHkdVMpOEmZ2LjDU3a+Ili8GjnT3sVWU/ynwH3e/o9L6F4B73f2pJNuMjhIY3bt3P3xVsrOUWfbxx/DGG/D66zB3brilmvemTRto3x722ANat05+a9UqlLnqKhgwIDOvRURqL9U093GGZoozDFQccYeTqi7ebIjbSSJOgvo68D+VEtRAd/9mkrIXAWOBwe6+JWH9fsBCYH9331bd82W7F19NfPQRlJTA55/vftu4cef9L7+EzZvDbcuWnfcrbv/5TzjRescdYcilvLyax1JWFnogduwIHTpoPMGmbsUK+P3vYd99Q6/Unj1h//1r978lNRMnacRJYlD3ZJirvQ7jJqi0NfEBJwJLgH2SPHYtMDVOlS5Xmvgyaf1693PPDdXzwYPdV6+Ov215ufsf/+jeu/fOKn7z5u777ON+6KFhQsYzznC//HL373/f/YknwjbSeC1YEN7/yk1MLVq4H3yw+9e+5j56tPuPf+z+xhvZjjbzXnzRvaSk/vYfp9mtW7f0NAMecED1+4nT3FjxXHUdkqomiNnEFydBNQdWAD2BlsACoE+lMv2Bd4FeVezjX8DxcQJqignKPSSNhx5yb9fOvWNH9+nTU2/z8svu//3f4V085BD3yZPd77nH/Qc/CF9A55zjftxx7v36ue+/v3urVqHskCHhfJk0Pq+/7t6pk3vXriFRLV3qPnu2+y9+4X7DDe7nned+xBHue+8d/hf22MP97bezHXXmzJy5M1mPGuX+1lv18zzVfeF/8YV7r167J41WrWp2Lqu83P3II3d/PDGJ5eoYhWlLUGFfnAy8EyWhm6J144HTo/vPAWuB+dFtZsK2+cAHQLM4z9VUE1SF5ct3/tONHOn+6ae7l1m8ONSKwH2//dynTnXfti31vsvK3H/2s5AAmzcPNarPP0//a5DseOkl9/bt3fPz3VesSF1+9epQ0yoocN+4sf7jy7b//CcMCF1Y6D52bEjO4D5smPsLL2SmZWHrVvdTTnFv1sz92mt3JqEWLdzbtnVfuXJn2VTJ5b77wvIFF+yazK68cuc+4tSg4tay0imtCSqTt6aeoNzDP/Gtt4Z/4p493V95Jaz/4AP3b3wjrG/f3v2OO2r3xfLRR+6XXRbe/a5d3WfMyHyz31NPhQ/SrFnq8ZgOzz4bfvV+5SuhJ2lNtjNzv+SS+ostF5SXh0TUuvXOGuO6de7jx4ekBe6HHx5aLuL82KttDKNGhef6xS92fWz5cvcOHdwHDnTfsiWsqy5xvPiie16e+1ln7fzslpWFUwRt27r/+99hXbpGgq/YV7qaAZWgGoGXXw6/hps1C+eo9tgj/NK69lr30tK67/8f/3AvKgr/BSee6L5kSd33mUp5ufvdd4d/8ry88Nz77x+aJd95p/6fv7b+9Cf3ESPCj4Knn3ZfuzbbEe305z+H5qF+/UItoaZuuSW8Dw8/nP7YcsX994fXeP/9uz+2aVNIGBXNbvn5obn8yy/TG8P3vhf2f/vtyR//wx/C49deG5arSi733+++777hx8iGDbvuo6QkNN8WFu6MP1ViyWS3+ApKUI3Ep5+Gpj5wv/BC93ffTe/+y8rcf/rT0OzXooX7uHHuy5bVT9Pf1q3uV1wRXst554UP1x/+sLPJA0KnjgcfzJ2mx61b3a+7LsTWseOuH9CuXd1PP939tttCkvjgg8zHN2NGaK4tLg41gtooKwvnKtu0cV+0KL3xJT5HtixaFGpOw4ZV31JQVhY6HB19dHh/u3d3/81v3Ldvr3sM99wT9nnNNdXH8O1vh3KPPx6WKyeXhx92/+pXw3tV1bnDv/wl7OPqq+PFlq5ru2pCCaqRSXYuKp3Wrg3NPIn/fG3bhouSjzkmdLi45prQJPLLX9b8l/rHH4fOGeB+8827f+g/+MD9zjvDr8KK577kEvd589L3GmuqpCS8dgjnLDZvDkl1zhz3//3f8IPhkEN2bSL52tdCUsuEhx8Oif2YY3b/JV1Ta9bU/nzU55+7L1wYmmunTg3N05dd5n7SSWF/HTqE2vJll7mvWlW3OGtq82b3ww4LtYoPP4y/3XPPuQ8YEN7ToqLQFFpbv/lN2M/Xv546UW/ZEs5Bd+gQfihW9p3vhH09+mj1+/nud3dNdKmkqmXFbQaMSwlKamX+fPdf/9r9rrvCh2H48JBYCgrc99pr5z9m69bhF1qck/HLl4du8C1ahH1Xp7w8nHO7/PLQozEvz33ChMz/An/22XBuom1b98ceq77s55+HDgo33RSOza231n98P/uZ72iaTVcHh2eeCV84l14ar/yWLeG9ad169y+t/fYLvQXPOsv9m98M505btgy3a6/NXBPp9deHmGbOrPm227e7T5u2s/bwP/8TPh81MWtWqOEOGRKSZRyrVoXPWlFRaH6s8LvfhTi++c3U+9iyJZzP2nNP9/feq1nMyagGpQTVIGzeHH4tX3FFSDh5eeHcTFXddV96yb1z5/CB+/vfa/ZcGzaEIaXA/YQTwq/8+rZ9e6glmoWkXNPu+CNHhlpNRceWdFu/fmcHl1NPTf95kptvDvtO9UPib38L19mB+9lnhy/PV14JX65V1SBXrQqxN2sWfnzcckvqmt+WLaE2853vhNp1QUE4HxinU89zz4X4xoxJXbY6mzeHGnOnTuH/YtSo5NcqlpeHc8Pz54fEdP/9oamsf/+at4A89VSIffTosLx4cfix9N//vbMTRSrvvhtqYkceWfdavc5BKUE1OCUl4fxM27Y7vzATv5h/+9vwi/krX0neXBFHeXloUtxjj1Cjefrpmsc4aVJoDps3b9dfpJWVloZfyRCSbm1qJp9+Gnpe5uent1m2vDz8mu/SJfwouOGG+F9UNbFtW+gJ1qZN8uRcWrqzKTg/P3yR1tSSJTsvTN9rrzCmZeL78uGH7r/6VUh87duHcq1auQ8dujMpnnBC9dcwrV8fzhH27h2uO0qHjz8OHR1atQq3UaPCudRBg8J7XnGdYeLtkENq13HFPZwPhlBbPuSQ0ARb0wuMZ8wI+/je92oXQyL14lOCapDWrQs9kzp3Dv9Rxx4bzleB+/HHhy+Lulq0KPRSg9BsU92X8/btobnqrLN29hSsuDVrFhLmOee4//CHoY1+6dKQWA84ICTUX/yibt3uX3klPM/IkbXfR6IVK3YmzoEDa97MVFMffBASYZ8+O7/ct28PSWOvvUKT1bhxdf/inzt35+vaf//QQaC4eOd71a1buBRh5sydPxa2bg097Dp1Csf46qt379FaXh4SYIsW9XMOc+VK94suCp1mevUKHUwuvDAkgUmT3H//+9BDduXKunVZ37YtdIiA8H88Z07t9jNmTNjHrFmhxr1ypfurr7o/+WQ4Z/ijH4XP67nnhvdj2LDQcenUU0MnoDPPDD8WzjknnEebOLH2r6mCEpRk3MaN4eLBimFcLrssvb/yN21yv+qqsO8jjti9R+O6deHDc/DBoczee4eaxjvvhF/tv/99OD901lnhi6Xyid/8/PClmQ633hr2GWdEkKps2xZezx57hCaxyZMzdy5u9uxwfC67LPQWq/iiHDQo/SNP/O1voedcs2ahCWvChJCEq/uRsG5d+FLNywuJYtKknc1YDz0UYr3zzvTGmQ0ffBB+mCXrHh/Xpk2h23lVHR0qeqgeckj4AXTEEeGasP79w3mwwkL3vn1D8+qhh4bjXldKUJI1W7aEL/r6uvj38cfDB6pDh9CB4R//cL/44p1NLIMGheawVCelv/giDA300EPh2qyPP05fjNu2uR91VIizNj3XXn995zVqp51Ws/EZ06Wi00deXqg5/epX6elynUx5efxOBInefjv0nKxoTnvwwZDMBw/WBeCJ3nsv1HonTAhN5n/+c/gfW726dse9ruImqJSjmWdaQxrNXLJn1SoYPhz++c+w3L49XHwxjBkD/fplN7YK774LRUVw+OHw/PPxRhLfsAFuvx0mT4b/+i+4//4weWY2RqcvK4ORI8Po1z/+MXTpkvkY4nCHp56C666D5cthzz1h4cKqJxeU7EvbdBuZpgQlcW3bFqYNaNECLrwQ2rXLdkS7e/hhuPTSMCPzDTdUXa6sLLyWH/4Q1q+HK68MSaFjx4yF2uBt3QoPPggFBXDssdmORqqjBCWSA9zhvPPgySdDbe/ww3cv8/TT8N3vwpIlMHgw3HuvJq+Uxi1ugmqWiWBEmiozeOAB2GefMEFc4sRxb78NQ4fCySeH2uATT8CcOUpOIhWUoETq2V57wW9+A++8E2pKH30EV10Fhx0Gr74K990HixbBmWdqJmSRRM2zHYBIUzBkCFx/PUycCI88Aps3w9ixcOut0LlztqMTyU1KUCIZcscdMH9+6BV3113Qu3e2IxLJbUpQIhnSsiU880y2oxBpOHQOSkREclKsBGVmQ81sqZktN7NxSR6/zswWm9lCM3vezHokPNbdzJ4xsyVRmfz0hS8iIo1VygRlZnnAFGAYUAAMN7OCSsXeBIrdvRB4HLg74bHfABPd/VBgIPBROgIXEZHGLU4NaiCw3N1XuPtWYDpwRmIBd5/j7hVXePwL6AYQJbLm7v5sVG5jQjkREZEqxUlQXYH3E5ZLonVVuRx4Orr/FWCDmf3RzN40s4lRjWwXZjbazOaa2dzS0tK4sYuISCMWJ0Elu3Qw6fhIZnYRUAxMjFY1B74KXA8cARwIXLLbztynunuxuxd3ydURKUVEJKPiJKgS4ICE5W7AmsqFzOxE4CbgdHffkrDtm1HzYBnwJ0ADuYiISEpxEtTrQC8z62lmLYELgJmJBcysP/AAITl9VGnbTmZWUS0aAiyue9giItLYpUxQUc1nLDAbWALMcPdFZjbezE6Pik0E2gG/N7P5ZjYz2nY7oXnveTN7i9Bc+H/18DpERKSRybnpNsysFFiVotjewLoMhJMuird+Kd76pXjrV1OMt4e7p+xwkHMJKg4zmxtnLpFcoXjrl+KtX4q3fineqmmoIxERyUlKUCIikpMaaoKamu0Aakjx1i/FW78Ub/1SvFVokOegRESk8WuoNSgREWnklKBERCQnNagElWpeqlxjZivN7K3o4uW52Y4nGTN70Mw+MrO3E9btZWbPmtmy6G+nbMaYqIp4bzOzD6LjPN/MTs5mjBXM7AAzmxPNhbbIzK6N1ufk8a0m3lw9vq3N7DUzWxDFe3u0vqeZvRod399FI+BkXTXxPmxm7yUc36Jsx5rIzPKiwb6fipYzdnwbTIKKOS9VLjre3Yty+DqHh4GhldaNA553917A89FyrniY3eMFuC86zkXuPivDMVWlDPhuNBfaUcA10f9srh7fquKF3Dy+W4Ah7n4YUAQMNbOjgLsI8fYCPiHMsJALqooX4HsJx3d+9kJM6lrCKEIVMnZ8G0yCIsa8VFJz7v4i8HGl1WcAv47u/xo4M6NBVaOKeHOSu3/o7m9E9z8nfMi7kqPHt5p4c5IHG6PFFtHNCWN+Ph6tz6XjW1W8OcvMugGnAL+Mlo0MHt+GlKBqOi9VLnDgGTObZ2ajsx1MDfyXu38I4UsL2CfL8cQx1swWRk2AOdFklsjM8oH+wKs0gONbKV7I0eMbNT/NJ8zU/SzwLrAhGkMUcux7onK87l5xfCdEx/c+M2uVxRArmwR8HyiPljuTwePbkBJU7Hmpcsgx7j6A0Cx5jZkdm+2AGqmfAwcRmk0+BP43u+HsyszaAX8Avu3un2U7nlSSxJuzx9fdt7t7EWEaoIHAocmKZTaqqlWO18z6Aj8ADiHMmbcXcEMWQ9zBzE4FPnL3eYmrkxStt+PbkBJUrHmpcom7r4n+fgQ8QfgANQRrzWw/gOjvRynKZ5W7r40++OWE0fJz5jibWQvCl/00d/9jtDpnj2+yeHP5+FZw9w3A3wjnzjqaWfPooZz8nkiId2jUtOrRPHoPkTvH9xjgdDNbSTilMoRQo8rY8W1ICSrlvFS5xMzamln7ivvAScDb1W+VM2YCo6L7o4AnsxhLShVf9pGzyJHjHLXX/wpY4u73JjyUk8e3qnhz+Ph2MbOO0f09gBMJ583mAOdGxXLp+CaL998JP1aMcD4nJ46vu//A3bu5ez7h+/YFdx9BBo9vgxpJIureOgnIAx509wlZDqlKZnYgodYE0Bx4NBfjNbPHgOMIQ+ivBX5ImPl4BtAdWA183d1zomNCFfEeR2h+cmAlcGXFOZ5sMrNBwEvAW+xsw7+RcF4n545vNfEOJzePbyHhJH0e4cf2DHcfH332phOay94ELkqY5Ttrqon3BaALoflsPjAmoTNFTjCz44Dr3f3UTB7fBpWgRESk6WhITXwiItKEKEGJiEhOUoISEZGcpAQlIiI5SQlKRERykhKUiIjkJCUoERHJSf8fO5D4s4k5le4AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "loss = history.history['loss']\n",
    "val_loss = history.history['val_loss']\n",
    "\n",
    "epochs = range(1, len(loss) + 1)\n",
    "\n",
    "plt.figure()\n",
    "\n",
    "plt.plot(epochs, loss, 'bo', label='Training loss')\n",
    "plt.plot(epochs, val_loss, 'b', label='Validation loss')\n",
    "plt.title('Training and validation loss')\n",
    "plt.legend()\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "성공이네요! 30번째 에포크까지 과대적합이 일어나지 않았습니다. 평가 점수는 안정적이지만 이전보다 더 나아지진 않았습니다."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 스태킹 순환 층\n",
    "\n",
    "과대적합은 더 이상 없지만 성능상의 병목이 있는 것 같으므로 네트워크의 용량을 늘려야 합니다. 일반적인 머신 러닝 작업 흐름을 기억하세요. (드롭아웃 등을 사용하여 과대적합을 줄이는 기본 단계를 거쳤다고 가정하고) 과대적합이 일어날 때까지 네트워크의 용량을 늘리는 것이 좋습니다. 너무 많이 과대적합되지 않는 한 아직 충분한 용량에 도달한 것이 아닙니다.\n",
    "\n",
    "네트워크의 용량을 늘리려면 일반적으로 층에 있는 유닛의 수를 늘리거나 층을 더 많이 추가합니다. 순환 층 스태킹은 더 강력한 순환 네트워크를 만드는 고전적인 방법입니다. 예를 들어 구글 번역 알고리즘의 현재 성능은 7개의 대규모 LSTM 층을 쌓은 대규모 모델에서 나온 것입니다.\n",
    "\n",
    "케라스에서 순환 층을 차례대로 쌓으려면 모든 중간 층은 마지막 타임스텝 출력만이 아니고 전체 시퀀스(3D 텐서)를 출력해야 합니다. `return_sequences=True`로 지정하면 됩니다: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/40\n",
      "500/500 [==============================] - 238s 475ms/step - loss: 0.3353 - val_loss: 0.2765\n",
      "Epoch 2/40\n",
      "500/500 [==============================] - 237s 473ms/step - loss: 0.3122 - val_loss: 0.2768\n",
      "Epoch 3/40\n",
      "500/500 [==============================] - 237s 473ms/step - loss: 0.3055 - val_loss: 0.2721\n",
      "Epoch 4/40\n",
      "500/500 [==============================] - 237s 473ms/step - loss: 0.3013 - val_loss: 0.2667\n",
      "Epoch 5/40\n",
      "500/500 [==============================] - 237s 473ms/step - loss: 0.2960 - val_loss: 0.2648\n",
      "Epoch 6/40\n",
      "500/500 [==============================] - 237s 474ms/step - loss: 0.2949 - val_loss: 0.2626\n",
      "Epoch 7/40\n",
      "500/500 [==============================] - 236s 473ms/step - loss: 0.2913 - val_loss: 0.2648\n",
      "Epoch 8/40\n",
      "500/500 [==============================] - 237s 473ms/step - loss: 0.2866 - val_loss: 0.2705\n",
      "Epoch 9/40\n",
      "500/500 [==============================] - 237s 473ms/step - loss: 0.2841 - val_loss: 0.2643\n",
      "Epoch 10/40\n",
      "500/500 [==============================] - 237s 473ms/step - loss: 0.2829 - val_loss: 0.2664\n",
      "Epoch 11/40\n",
      "500/500 [==============================] - 237s 473ms/step - loss: 0.2844 - val_loss: 0.2686\n",
      "Epoch 12/40\n",
      "500/500 [==============================] - 237s 473ms/step - loss: 0.2817 - val_loss: 0.2642\n",
      "Epoch 13/40\n",
      "500/500 [==============================] - 237s 473ms/step - loss: 0.2798 - val_loss: 0.2684\n",
      "Epoch 14/40\n",
      "500/500 [==============================] - 237s 473ms/step - loss: 0.2788 - val_loss: 0.2665\n",
      "Epoch 15/40\n",
      "500/500 [==============================] - 237s 473ms/step - loss: 0.2783 - val_loss: 0.2697\n",
      "Epoch 16/40\n",
      "500/500 [==============================] - 237s 473ms/step - loss: 0.2750 - val_loss: 0.2680\n",
      "Epoch 17/40\n",
      "500/500 [==============================] - 237s 473ms/step - loss: 0.2732 - val_loss: 0.2719\n",
      "Epoch 18/40\n",
      "500/500 [==============================] - 237s 473ms/step - loss: 0.2729 - val_loss: 0.2689\n",
      "Epoch 19/40\n",
      "500/500 [==============================] - 237s 473ms/step - loss: 0.2720 - val_loss: 0.2730\n",
      "Epoch 20/40\n",
      "500/500 [==============================] - 237s 473ms/step - loss: 0.2737 - val_loss: 0.2675\n",
      "Epoch 21/40\n",
      "500/500 [==============================] - 237s 473ms/step - loss: 0.2713 - val_loss: 0.2657\n",
      "Epoch 22/40\n",
      "500/500 [==============================] - 236s 473ms/step - loss: 0.2696 - val_loss: 0.2743\n",
      "Epoch 23/40\n",
      "500/500 [==============================] - 237s 473ms/step - loss: 0.2691 - val_loss: 0.2667\n",
      "Epoch 24/40\n",
      "500/500 [==============================] - 236s 473ms/step - loss: 0.2664 - val_loss: 0.2706\n",
      "Epoch 25/40\n",
      "500/500 [==============================] - 237s 473ms/step - loss: 0.2644 - val_loss: 0.2750\n",
      "Epoch 26/40\n",
      "500/500 [==============================] - 237s 473ms/step - loss: 0.2653 - val_loss: 0.2712\n",
      "Epoch 27/40\n",
      "500/500 [==============================] - 237s 473ms/step - loss: 0.2648 - val_loss: 0.2689\n",
      "Epoch 28/40\n",
      "500/500 [==============================] - 237s 473ms/step - loss: 0.2622 - val_loss: 0.2690\n",
      "Epoch 29/40\n",
      "500/500 [==============================] - 237s 473ms/step - loss: 0.2617 - val_loss: 0.2689\n",
      "Epoch 30/40\n",
      "500/500 [==============================] - 237s 473ms/step - loss: 0.2599 - val_loss: 0.2777\n",
      "Epoch 31/40\n",
      "500/500 [==============================] - 237s 473ms/step - loss: 0.2602 - val_loss: 0.2745\n",
      "Epoch 32/40\n",
      "500/500 [==============================] - 237s 473ms/step - loss: 0.2595 - val_loss: 0.2736\n",
      "Epoch 33/40\n",
      "500/500 [==============================] - 237s 473ms/step - loss: 0.2596 - val_loss: 0.2807\n",
      "Epoch 34/40\n",
      "500/500 [==============================] - 237s 473ms/step - loss: 0.2582 - val_loss: 0.2749\n",
      "Epoch 35/40\n",
      "500/500 [==============================] - 237s 473ms/step - loss: 0.2582 - val_loss: 0.2756\n",
      "Epoch 36/40\n",
      "500/500 [==============================] - 237s 473ms/step - loss: 0.2585 - val_loss: 0.2834\n",
      "Epoch 37/40\n",
      "500/500 [==============================] - 237s 473ms/step - loss: 0.2576 - val_loss: 0.2785\n",
      "Epoch 38/40\n",
      "500/500 [==============================] - 237s 473ms/step - loss: 0.2546 - val_loss: 0.2759\n",
      "Epoch 39/40\n",
      "500/500 [==============================] - 237s 473ms/step - loss: 0.2534 - val_loss: 0.2759\n",
      "Epoch 40/40\n",
      "500/500 [==============================] - 237s 473ms/step - loss: 0.2547 - val_loss: 0.2798\n"
     ]
    }
   ],
   "source": [
    "from keras.models import Sequential\n",
    "from keras import layers\n",
    "from keras.optimizers import RMSprop\n",
    "\n",
    "model = Sequential()\n",
    "model.add(layers.GRU(32,\n",
    "                     dropout=0.1,\n",
    "                     recurrent_dropout=0.5,\n",
    "                     return_sequences=True,\n",
    "                     input_shape=(None, float_data.shape[-1])))\n",
    "model.add(layers.GRU(64, activation='relu',\n",
    "                     dropout=0.1, \n",
    "                     recurrent_dropout=0.5))\n",
    "model.add(layers.Dense(1))\n",
    "\n",
    "model.compile(optimizer=RMSprop(), loss='mae')\n",
    "history = model.fit_generator(train_gen,\n",
    "                              steps_per_epoch=500,\n",
    "                              epochs=40,\n",
    "                              validation_data=val_gen,\n",
    "                              validation_steps=val_steps)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "결과를 확인해 보죠:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAAEYCAYAAAAJeGK1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3Xl8VOXZ//HPRWQRUEFArSAEl6cKiIDB4uMCrkUtbnVDcKvUpdVqbfsTRa3a0lpXimIr9nGpUKmPVUtdqxVFbR+QXRERxIAIIiAgOyS5fn/cMzCESeYkmcmcJN/36zWvzDlzlmtOkrnmXs59m7sjIiISN43yHYCIiEg6SlAiIhJLSlAiIhJLSlAiIhJLSlAiIhJLSlAiIhJLSlASC2ZWYGbrzKxjNrfNJzM70Myyfh+HmZ1oZsUpy3PN7Jgo21bjXH8ys5uru38lx/21mT2R7eNK/bJLvgOQusnM1qUsNgc2A6WJ5SvdfWxVjufupUDLbG/bELj7t7NxHDMbAgx2934pxx6SjWOLVIcSlFSLu29LEIlv6EPc/Y2KtjezXdy9pDZiE5H6QVV8khOJKpy/mtnTZrYWGGxmR5rZ/5nZajNbamYjzaxxYvtdzMzNrDCxPCbx+itmttbM/mNmnau6beL1U8zsEzNbY2YPmtl7ZnZpBXFHifFKM5tvZqvMbGTKvgVm9oCZrTSzT4H+lVyfW8xsXLl1o8zs/sTzIWY2J/F+Pk2Ubio61mIz65d43tzMnkrENhs4PM15FySOO9vMTk+sPxR4CDgmUX26IuXa3p6y/1WJ977SzF4ws29FuTaZmNmZiXhWm9mbZvbtlNduNrMlZvaNmX2c8l77mNm0xPplZnZP1PNJHeHueuhRowdQDJxYbt2vgS3AAMIXoV2B3sB3CCX3/YFPgGsS2+8COFCYWB4DrACKgMbAX4Ex1dh2L2AtcEbitRuArcClFbyXKDH+HdgDKAS+Tr534BpgNtABaANMDP9iac+zP7AOaJFy7K+AosTygMQ2BhwPbAS6J147EShOOdZioF/i+b3AW0BroBPwUbltzwO+lfidXJiIYe/Ea0OAt8rFOQa4PfH85ESMPYBmwMPAm1GuTZr3/2vgicTzQxJxHJ/4Hd2cuO6Nga7AQmCfxLadgf0Tz98HBiae7wZ8J9//C3pk96ESlOTSu+7+D3cvc/eN7v6+u09y9xJ3XwCMBvpWsv+z7j7F3bcCYwkfjFXd9nvADHf/e+K1BwjJLK2IMf7W3de4ezEhGSTPdR7wgLsvdveVwF2VnGcB8CEhcQKcBKx29ymJ1//h7gs8eBP4F5C2I0Q55wG/dvdV7r6QUCpKPe8z7r408Tv5C+HLRVGE4wIMAv7k7jPcfRMwFOhrZh1Stqno2lTmAmC8u7+Z+B3dBexO+KJQQkiGXRPVxJ8lrh2ELxoHmVkbd1/r7pMivg+pI5SgJJc+T10ws4PN7CUz+9LMvgHuBNpWsv+XKc83UHnHiIq23Tc1Dnd3QokjrYgxRjoX4Zt/Zf4CDEw8v5CQWJNxfM/MJpnZ12a2mlB6qexaJX2rshjM7FIzm5moSlsNHBzxuBDe37bjufs3wCqgfco2VfmdVXTcMsLvqL27zwV+Rvg9fJWoMt4nsellQBdgrplNNrNTI74PqSOUoCSXynexfoRQajjQ3XcHbiNUYeXSUkKVGwBmZuz4gVpeTWJcCuyXspypG/xfgRMTJZAzCAkLM9sVeBb4LaH6rRXwz4hxfFlRDGa2P/AH4GqgTeK4H6ccN1OX+CWEasPk8XYjVCV+ESGuqhy3EeF39gWAu49x96MI1XsFhOuCu8919wsI1bj3AX8zs2Y1jEViRAlKatNuwBpgvZkdAlxZC+d8EehlZgPMbBfgOqBdjmJ8BrjezNqbWRvgxso2dvdlwLvA48Bcd5+XeKkp0ARYDpSa2feAE6oQw81m1srCfWLXpLzWkpCElhNy9RBCCSppGdAh2SkkjaeBy82su5k1JSSKd9y9whJpFWI+3cz6Jc79C0K74SQzO8TMjkucb2PiUUp4AxeZWdtEiWtN4r2V1TAWiRElKKlNPwMuIXz4PEIoQeRUIgmcD9wPrAQOAKYT7tvKdox/ILQVfUBowH82wj5/IXR6+EtKzKuBnwLPEzoanENItFH8klCSKwZeAf6cctxZwEhgcmKbg4HUdpvXgXnAMjNLrapL7v8qoart+cT+HQntUjXi7rMJ1/wPhOTZHzg90R7VFLib0G74JaHEdkti11OBORZ6id4LnO/uW2oaj8SHhSp5kYbBzAoIVUrnuPs7+Y5HRCqmEpTUe2bW38z2SFQT3UroGTY5z2GJSAZKUNIQHA0sIFQT9QfOdPeKqvhEJCZUxSciIrGkEpSIiMRS7AaLbdu2rRcWFuY7DBERyZGpU6eucPfKbvcAYpigCgsLmTJlSr7DEBGRHDGzTKOsAKriExGRmFKCEhGRWFKCEhGRWIpdG5SISEW2bt3K4sWL2bRpU75DkQiaNWtGhw4daNy4ouEdK6cEJSJ1xuLFi9ltt90oLCwkDEwvceXurFy5ksWLF9O5c+fMO6ShKj4RqTM2bdpEmzZtlJzqADOjTZs2NSrt1rsENXYsFBZCo0bh59ixmfYQkbpEyanuqOnvql5V8Y0dC1dcARs2hOWFC8MywKAaTwogIiK1qV6VoIYN256ckjZsCOtFRGpq5cqV9OjRgx49erDPPvvQvn37bctbtkSbiuqyyy5j7ty5lW4zatQoxmap+ufoo49mxowZWTlWbatXJahFi6q2XkTqt7FjwxfURYugY0cYPrxmtSlt2rTZ9mF/++2307JlS37+85/vsI274+40apT++//jjz+e8Tw//vGPqx9kPVKvSlAdO1ZtvYjUX8kq/4ULwX17lX8u2qXnz59Pt27duOqqq+jVqxdLly7liiuuoKioiK5du3LnnXdu2zZZoikpKaFVq1YMHTqUww47jCOPPJKvvvoKgFtuuYURI0Zs237o0KEcccQRfPvb3+bf//43AOvXr+f73/8+hx12GAMHDqSoqChjSWnMmDEceuihdOvWjZtvvhmAkpISLrroom3rR44cCcADDzxAly5dOOywwxg8eHDWr1kU9SpBDR8OzZvvuK5587BeRBqW2q7y/+ijj7j88suZPn067du356677mLKlCnMnDmT119/nY8++minfdasWUPfvn2ZOXMmRx55JI899ljaY7s7kydP5p577tmW7B588EH22WcfZs6cydChQ5k+fXql8S1evJhbbrmFCRMmMH36dN577z1efPFFpk6dyooVK/jggw/48MMPufjiiwG4++67mTFjBjNnzuShhx6q4dWpnnqVoAYNgtGjoVMnMAs/R49WBwmRhqi2q/wPOOAAevfuvW356aefplevXvTq1Ys5c+akTVC77rorp5xyCgCHH344xcXFaY999tln77TNu+++ywUXXADAYYcdRteuXSuNb9KkSRx//PG0bduWxo0bc+GFFzJx4kQOPPBA5s6dy3XXXcdrr73GHnvsAUDXrl0ZPHgwY8eOrfaNtjVVrxIUhGRUXAxlZeGnkpNIw1TbVf4tWrTY9nzevHn8/ve/580332TWrFn0798/7f1ATZo02fa8oKCAkpKStMdu2rTpTttUdbLZirZv06YNs2bN4uijj2bkyJFceeWVALz22mtcddVVTJ48maKiIkpLS6t0vmyodwlKRATyW+X/zTffsNtuu7H77ruzdOlSXnvttayf4+ijj+aZZ54B4IMPPkhbQkvVp08fJkyYwMqVKykpKWHcuHH07duX5cuX4+6ce+653HHHHUybNo3S0lIWL17M8ccfzz333MPy5cvZUL6+tBbUq158IiJJydqTbPbii6pXr1506dKFbt26sf/++3PUUUdl/RzXXnstF198Md27d6dXr15069ZtW/VcOh06dODOO++kX79+uDsDBgzgtNNOY9q0aVx++eW4O2bG7373O0pKSrjwwgtZu3YtZWVl3Hjjjey2225Zfw+ZWFWLiblWVFTkmrBQRNKZM2cOhxxySL7DiIWSkhJKSkpo1qwZ8+bN4+STT2bevHnssku8yh3pfmdmNtXdizLtG693IiIikaxbt44TTjiBkpIS3J1HHnkkdsmppurXuxERaSBatWrF1KlT8x1GTqmThIiIxJISlIiIxJISlIiIxJISlIiIxJISlIhIRP369dvpptsRI0bwox/9qNL9WrZsCcCSJUs455xzKjx2pltsRowYscMNs6eeeiqrV6+OEnqlbr/9du69994aHyfblKBERCIaOHAg48aN22HduHHjGDhwYKT99913X5599tlqn798gnr55Zdp1apVtY8Xd0pQIiIRnXPOObz44ots3rwZgOLiYpYsWcLRRx+97b6kXr16ceihh/L3v/99p/2Li4vp1q0bABs3buSCCy6ge/funH/++WzcuHHbdldfffW2qTp++ctfAjBy5EiWLFnCcccdx3HHHQdAYWEhK1asAOD++++nW7dudOvWbdtUHcXFxRxyyCH88Ic/pGvXrpx88sk7nCedGTNm0KdPH7p3785ZZ53FqlWrtp2/S5cudO/efdsgtW+//fa2CRt79uzJ2rVrq31t09F9UCJSJ11/PWR7otgePSDx2Z5WmzZtOOKII3j11Vc544wzGDduHOeffz5mRrNmzXj++efZfffdWbFiBX369OH000/HzNIe6w9/+APNmzdn1qxZzJo1i169em17bfjw4ey5556UlpZywgknMGvWLH7yk59w//33M2HCBNq2bbvDsaZOncrjjz/OpEmTcHe+853v0LdvX1q3bs28efN4+umnefTRRznvvPP429/+Vun8ThdffDEPPvggffv25bbbbuOOO+5gxIgR3HXXXXz22Wc0bdp0W7Xivffey6hRozjqqKNYt24dzZo1q8LVzkwlKBGRKkit5kut3nN3br75Zrp3786JJ57IF198wbJlyyo8zsSJE7cliu7du9O9e/dtrz3zzDP06tWLnj17Mnv27IwDwb777rucddZZtGjRgpYtW3L22WfzzjvvANC5c2d69OgBVD6lB4T5qVavXk3fvn0BuOSSS5g4ceK2GAcNGsSYMWO2jVhx1FFHccMNNzBy5EhWr16d9ZEsVIISkTqpspJOLp155pnccMMNTJs2jY0bN24r+YwdO5bly5czdepUGjduTGFhYdopNlKlK1199tln3Hvvvbz//vu0bt2aSy+9NONxKhtTNTlVB4TpOjJV8VXkpZdeYuLEiYwfP55f/epXzJ49m6FDh3Laaafx8ssv06dPH9544w0OPvjgah0/HZWgRESqoGXLlvTr148f/OAHO3SOWLNmDXvttReNGzdmwoQJLFy4sNLjHHvssYxNzD//4YcfMmvWLCBM1dGiRQv22GMPli1bxiuvvLJtn9122y1tO8+xxx7LCy+8wIYNG1i/fj3PP/88xxxzTJXf2x577EHr1q23lb6eeuop+vbtS1lZGZ9//jnHHXccd999N6tXr2bdunV8+umnHHroodx4440UFRXx8ccfV/mclVEJSkSkigYOHMjZZ5+9Q4++QYMGMWDAAIqKiujRo0fGksTVV1/NZZddRvfu3enRowdHHHEEEGbH7dmzJ127dt1pqo4rrriCU045hW9961tMmDBh2/pevXpx6aWXbjvGkCFD6NmzZ6XVeRV58sknueqqq9iwYQP7778/jz/+OKWlpQwePJg1a9bg7vz0pz+lVatW3HrrrUyYMIGCggK6dOmybXbgbNF0GyJSZ2i6jbqnJtNtqIpPRERiSQlKRERiSQlKROqUuDVLSMVq+rtSghKROqNZs2asXLlSSaoOcHdWrlxZo5t31YtPROqMDh06sHjxYpYvX57vUCSCZs2a0aFDh2rvrwQlInVG48aN6dy5c77DkFqiKj4REYklJSgREYklJSgREYklJSgREYklJSgREYklJSgREYklJSgREYmlSAnKzPqb2Vwzm29mQ9O8fpWZfWBmM8zsXTPrklh/kplNTbw21cyOz/YbEBGR+iljgjKzAmAUcArQBRiYTEAp/uLuh7p7D+Bu4P7E+hXAAHc/FLgEeCprkYuISL0WpQR1BDDf3Re4+xZgHHBG6gbu/k3KYgvAE+unu/uSxPrZQDMza4qIiEgGUYY6ag98nrK8GPhO+Y3M7MfADUATIF1V3veB6e6+uRpxiohIAxOlBGVp1u00lLC7j3L3A4AbgVt2OIBZV+B3wJVpT2B2hZlNMbMpGgRSREQgWoJaDOyXstwBWFLBthCqAM9MLphZB+B54GJ3/zTdDu4+2t2L3L2oXbt2EUISEZH6LkqCeh84yMw6m1kT4AJgfOoGZnZQyuJpwLzE+lbAS8BN7v5edkIWEZGGIGOCcvcS4BrgNWAO8Iy7zzazO83s9MRm15jZbDObQWiHuiS5HjgQuDXRBX2Gme2V/bchIiL1jcVtZsqioiKfMmVKvsMQEZEcMbOp7l6UaTuNJCEiIrGkBCUiIrGkBCUiIrGkBCUiIrGkBCUiIrGkBCUiIrGkBCUiIrGkBCUiIrGkBCUiIrGkBCUiIrGkBCUiIrGkBCUiIrHU4BLU2LFQWAiNGoWfY8fmOyIREUknypTv9cbYsXDFFbBhQ1heuDAsAwwalL+4RERkZw2qBDVs2PbklLRhQ1gvIiLx0qAS1KJFVVsvIiL506ASVMeOVVsvIiL506AS1PDh0Lz5juuaNw/rRUQkXhpUgho0CEaPhk6dwCz8HD1aHSREROKoQfXig5CMlJBEROKvQZWgRESk7lCCEhGRWFKCEhGRWFKCEhGRWFKCEhGRWFKCEhGRWFKCEhGRWFKCEhGRWFKCEhGRWFKCSkOTGoqI5F+DG+ooE01qKCISDypBlaNJDUVE4kEJqhxNaigiEg9KUOVoUkMRkXhQgipHkxqKiMSDElQ5mtRQRCQe1IsvDU1qKCKSfypBiYhILClBiYhILClBiYhILClBVZOGQxIRyS11kqgGDYckIpJ7KkFVg4ZDEhHJvUgJysz6m9lcM5tvZkPTvH6VmX1gZjPM7F0z65Ly2k2J/eaa2XezGXy+aDgkEZHcy5igzKwAGAWcAnQBBqYmoIS/uPuh7t4DuBu4P7FvF+ACoCvQH3g4cbw6TcMhiYjkXpQS1BHAfHdf4O5bgHHAGakbuPs3KYstAE88PwMY5+6b3f0zYH7ieHWahkMSEcm9KAmqPfB5yvLixLodmNmPzexTQgnqJ1Xc9wozm2JmU5YvXx419rzRcEgiIrkXJUFZmnW+0wr3Ue5+AHAjcEsV9x3t7kXuXtSuXbsIIeXfoEFQXAxlZeGnkpOISHZFSVCLgf1SljsASyrZfhxwZjX3FRERAaIlqPeBg8yss5k1IXR6GJ+6gZkdlLJ4GjAv8Xw8cIGZNTWzzsBBwOSahy0iIvVdxht13b3EzK4BXgMKgMfcfbaZ3QlMcffxwDVmdiKwFVgFXJLYd7aZPQN8BJQAP3b30hy9FxERqUfMfacmobwqKiryKVOm5DsMERHJETOb6u5FmbbTSBIiIhJLSlAiIhJLSlA5pBHPRUSqT6OZ54hGPBcRqRmVoHJEI56LiNSMElSORB3xXNWAIiLpKUHlSJQRz5PVgAsXgvv2akAlKRERJaiciTLiuaoBRUQqpgSVI1FGPNfEhyIiFVMvvhwaNKjyHnsdO4ZqvXTrRUQaOpWg8kgTH4qIVEwJKo808aGISMVUxZdnmaoBRUQaKpWgREQklpSgYk438opIQ6UqvhjTeH4i0pCpBBVjupFXRBoyJagY0428ItKQKUHFWJTx/ERE6islqBjTjbwi0pApQcWYbuQVkYZMvfhiTjfyikhDpRKUiIjEkhKUiIjEkhKUiIjEkhKUiIjEkhJUPaDx+kSkPlIvvjpO4/WJSH2lElQdp/H6RKS+UoKq4zRen4jUV0pQdZzG6xOR+koJqo7TeH0iUl8pQdVxtT1en3oMikhtUYKqBwYNguJiKCsLP9Mlp2wklmSPwYULwX17j0ElKRHJBSWoBiBbiUU9BkWkNilBNQDZSizqMSgitUkJqgGImlgyVQOqx6BI7fvoI9i0Kd9R5IcSVAMQJbFEqQZUj0GR2jV/Phx6KJx/fvi/bGiUoBqAKIklSjWgZvgVqV0PPhg6P40fD48/nu9oap8SVAMQJbFErQbM1GNQ3dBFsmPNGnjssfA/1q8fXHcdLFiQ76hqlwaLbSAyTR3fsWOo1ku3PioNXCuSPY8/DuvWwU9/Cm3bQvfucMkl8NZbUFCQ7+hqh0pQAmSnfUnd0EWyo7QURo6Eo4+Gww8PtR4PPQTvvgv33pvv6GqPEpQA2WlfUjd0kex48UX47LNQrZc0eDB8//tw660wc2b+YqtNkRKUmfU3s7lmNt/MhqZ5/QYz+8jMZpnZv8ysU8prd5vZbDObY2Yjzcyy+QYke6KMSFEZdUOXhsIdVqzI3fFHjAj/N2eeuX2dGfzxj9CmTUhWDaHrecYEZWYFwCjgFKALMNDMupTbbDpQ5O7dgWeBuxP7/jdwFNAd6Ab0BvpmLXqJlajVhOpIIXXd8OHQvj3MmpX9Y8+aFdqZrrkGdinXS6Bt29Bx4sMP4ZZbsn/uuIlSgjoCmO/uC9x9CzAOOCN1A3ef4O7J1of/AzokXwKaAU2ApkBjYFk2Apf4iVJNqPH8pK775hu47z7YsgWuvTb79yf9/vfhi92QIelfP+UUuOoquP/+kMhq05IlsHFj7Z0vSoJqD3yesrw4sa4ilwOvALj7f4AJwNLE4zV3n1N+BzO7wsymmNmU5cuXR41dYihTNWHUjhQqZUlcPfwwrF4NV14JEyfCX/+avWMvXx7+1i+5BFq3rni7e++FAw4I261Zk73zV2b1ajjppNAOVluiJKh0bUZpvzOY2WCgCLgnsXwgcAihRNUeON7Mjt3pYO6j3b3I3YvatWsXNXapg6J0pFApS+Jqw4ZQcvnud2HUKOjZE37+c1i/PjvHf+QR2LwZfvKTyrdr0QKeegoWL96xI0WubNkCZ58N8+bBL36R+/MlRUlQi4H9UpY7AEvKb2RmJwLDgNPdfXNi9VnA/7n7OndfRyhZ9alZyFKXRelIoVKWxNWjj4ZSzi23hHuRHnwQvvgCfvObmh97y5aQ9Pr3h4MPzrx9nz7hf+LJJ+G552p+/oq4h+rGCRNC+9dxx+XuXGlO7pU+CDfzLgA6E9qSZgJdy23TE/gUOKjc+vOBNxLHaAz8CxhQ2fkOP/xwl/przBj35s3dw599eDRvHtYnme34evJhVrXjiGTTpk3u7du7H3vsjusHD3Zv0sR93ryaHX/MmPB3/Mor0ffZssX98MPd27Rx/+STmp2/IrfdFuK6887sHROY4hlyj4fTRtgITgU+SSShYYl1dxJKSySS0DJgRuIxPrG+AHgEmAN8BNyf6VxKUPXfmDHunTqFhNOp085JpVOn9AmqU6eqbSOSTaNHh7+x117bcf0XX7i3bOk+YED1j11W5l5U5H7wwe6lpVXbd84c9z33dN9jD/fx46sfQzqPPRbe8w9+EGLMlqwmqNp8KEFJtkpZ0vDMnOm+YEH2j7t1q/v++4ckku6D+ne/C39/L79cveO/917Y/+GHq7f/Z5+59+oVjjFsmHtJSfWOk+r119132cX9pJNCSS2blKCkTstGKUsalg0b3Fu3DiWJd9/N7rGfeir8fT3/fPrXN292/6//cj/ooFAVWFXnnefeqpX7unXVj3HjxlDSgZBUli+v/rFmzXLffXf3Qw91X7Om+sepiBJUGps2uX/+ufu0ae6zZ+fsNFIL1AYl5Y0dG/4O2rULfwvlq+Kqq7TUvUsX927dKq9+e+WVcP7f/a5qx1+0yL2gwP0Xv6hZnEmPPhraxDp2dH///arv/8UX7vvt577vviG2XGiwCWrZMvf/9//cL7vM/Xvfc//Od0LRfLfddv62rQ+zui1TKUsaluOOc+/c2X3JEvfDDnNv3Nj92Wdrfty//S18XvzlL5m3HTAgtEd98UX04994o3ujRu7FxdWPsbzJk0OCatIkJKyovvnGvUeP8B6mT89ePOVFTVAWto2PoqIinzJlSrX3X7QIDjwQ2rULj7322v489XHffWFIkenTww1vIlJ3ffpp+L//9a9D1+tVq+C002DSJPjTn+Cyy6p3XPcwmvjatfDxx5mnufj0U+jaFc45B8aMyXz89ethv/3ghBPgf/+3ejFWZMUKuPBCeP11uPzyMBp6s2YVb19SAmecAa+9Fgar7d8/u/GkMrOp7l6UccMoWaw2HzUtQZWVRettUlwc6nyPOCL7DYASHyplNQzDhoVSyOLF29etWxfaYsB9xIjqHffll8P+f/pT1WKBaO1gf/xj2Padd6oXXyYlJdvjKSx07907lJC6dg1tZp07u3fo4L733qHtDkJvxVyjoZagquLZZ+Hcc2HoUPjtb2vllFKLyk+gCGGMM01TX7+UlIRxH3v0gJde2vG1zZtDKeK55+D22+G228I4kVG4wzHHhFqZ+fOhSZNo+61fH260bdsWpkzZXurasgU++AAmT4b33w8/58wJcU+ZEj2u6vjHP8JNwACNG4f30rjx9kdy+Ygj4KKLchdHUoMtQVXVD38Yvl2/8UatnlZqgXr61dz48e5TpuQ7isq9+GL4vT73XPrXt251v/TSsM3110e/z+itt8I+I0dWPaZx48K+P/qR+zXXhLbwpk23/w22a+d+6qnut9/uvnBh1Y9f16ESVDTr10NRURhwcebM0D4l9UOjRulHmjYLg9lK5V59FU49NYz7NnFiGHcujs46C/797zAuXePG6bcpK4MbbggjhV92WZhXKVOJ6KSTQjt1cTHsumvVYnIP7UoTJoTrV1QEvXuHEkrv3ttH/G+oVIKqghkzQm+X007L7t3Skl9xK0Flqz3syy/dhwxxX7Eim9HtqLg4jE7QrVvocrz33rm5Abamli4NN5NG6aJdVhZKLMlbEk4+2f23v3X/z39CKSvVpElerS7jqb75xv2jj7Jz02x9Q0PtZl5dv/999YvzEk9R7pWqrU4U2bxv65Zbwv633Zb9ON3D/YJFReFGzXnZYgYMAAAUUklEQVTzwj2DrVuHm1BrcvNnLiRHcJgzJ/o+r74aqt26dt3+u2jZ0v2UU8LxJk8O3cVbtw5JRrJPCaqKyspCCapJk1CikvqhsgQUNWlkI4llqzS3dav7t74V9m3bNoyekG1XXhmO/8IL29e98457s2ahLWX9+uyfszrKykJPtKOOqv4xli1zf+YZ96uvdj/kkB1/N7/8ZdZClXKUoKrhq6/CP//BB9dsyBGpG6IkjaqUfJYscZ8/P/25sjV24AsvhP1++lPPSZfgJ54Ix73xxp1fe+650JX7e9/buUosHyZODLE+/nj2jrl0aejgcNttKj3lkhJUNb3xRvjQ+OEP8xqG1IIoSSNqyefBB8NwNRDuK8nV2IGnnRa+RG3dGgYHPeSQqo9+XZEZM0IpqV+/ihPQww+HuIcMyX977SWXhBFi9GWy7lGCqoGhQ8OV+d//zXckkktRkkaUJPbYY6Fkkamtq6ZtUAsXhvMMG7b9mOD+0ks1uQrBqlXuBxwQkt+XX1a+bfLGz3xWga1eHa7fFVfkLwapPiWoGtiyJYww0apV5n9WqbuiJI1MSaysbOdjVFQ6itKWVdk2v/xlWJ/sTbdlS5hA74QTanYdysrczzgj9IaLMvpBWdn2+4oeeaRm566u5AgMkybl5/xSM0pQNfTxx6HDxKBB+Y5EcunPf3bfddfwn9C0afjgS5Upif3mN+mTU/JRFZWdq6QkVB1+97s77pPsxVaTjj3JYzzwQPR9tmwJvd4aNcr+JHlR9O4dpoLIdzWjVI8SVBbcemu4Qhplov667rrwO7744vCF5OCDd77fp6JSzfPPb08i6ZLTHntULZbKSmvJ0RLKj8799dfuLVqE+KtjwoSQZM49t+of9mvXhu7ou+7q/o9/VO/81TFzZrgW1R1fT/JPCSoLNm4M9fIHHRSeS/2SvPft+uvD8ttvh3tf9t478zw6M2aExNC7d2iDKp+kCgpCldncudHjqay96/TTQ1zpBja+9towtURVpnhwD9vvtZf7t79d/R5ry5a5d+8e4rz00tCWlWs/+Un4MpHLG5Ult5SgsuSf//S8NwhHMX58qI7UyOzR/P3v4YP/zDN3vNP/o4/CqM/Nm1dcKli2LMy107596FruvnMp66GHQgnq+OOjl0wqKkG1bx8S3o03pi/NzZ8flm++Ofr7X78+3NPUokXNJ+/ctCmcu6AgTHL34os1O15lNm4MXyLOPz9355DcU4LKooEDwze2jz/OdyTprVwZbtqE0A1YKvf++yEB9e6d/qbTpUvdDz88VH394Q87vrZpU7gxdNddMw+imuyS/dRT0eKqqA3qnHPC83vvrbiN6uyzw9BEUbpcl5S4n3VWSGoVDbBaHe+/H4ZGgtAF/Ouvs3fspKefDsf/5z+zf2ypPUpQWbR0adW/DdemH/0ofJh26RKqbBrKDYZbt4YbKu+/P/roBsXFoaqssLDyHppr14Z7jpI3rZaW7th77ZlnMp+rtDSUUtq1C18ioihfQvrzn8PPE06ovI3q3XfD81GjMvcWvOEGr3KniKg2bQpDMRUUhC7r2e5AkbwO2br3S/JDCSrLqvptuLZMmxaS07XXbh/gMldjtMXJpk2h1JD8kN5rL/f77qs8Ua1aFZJ4q1ahKi+TrVu3D/tzwQXbe+xVpbp3xozwYV3dG79ffTWcc9y4ytuoysrCrRF77729V2K6XocPPRTWXXttbr9sTZkSetmB+0UXZU7QUSYaXbAgHO+OO7IXp+SHElSWVefbcK6VlrofeWSIKdk4ff754QMp2TaSC/kuRa5dG75JJ3tyTZy4fbmiRLV5cygBN27s/uab0c9VVhZGvE5+2J97btW/vf/sZ2HfKPcYlXf22aH6dtOmzPdkJecgqmibf/wjfJkZMKB2RtjevDn0hC0oCO1G3/72jjO4tm4dBmlt0mR78m3SJIwO0a5dGEX9wANDtWFRUeisZOa+aFHuY5fcipqgGvx8UFUxcyYcfjhcfjk88ki+o4Enn4RLL4XHHgtz3AB8+ikcckhYP3p0ds9XVhbm0bntNhg+HK68svrHKi6GN98Ms502axZ9v6+/DnMUTZkS3vfFF29/7Z134I474F//gr33hhtvDDHuuiv84AfwxBPhmqXuE9Uzz8DLL8PDD4dZeati3Tro0gX22AOmTat4zqLyvvwS9tsPrr8e7rkn8wzBJSWVH7t58/C38fbbYY6i2jJtGtx33/b4mjTZPoNr6s9GjcIMuJs2bf+ZfCSX//u/4Ve/qr3YJTc0H1SOJOvv33svv3GsWhVKC3367PyN/rrrwjflmvbOSrVggftxx4X3vueeoSTyn/9U71irVoVvwxC+UT/7bLRS2RdfhCkSmjbdcaTt8lJLVHvvvb2Twe23Vy/ebEgO8nrXXdH3SZbcUjvnZGpfat06fQmqoCBsv3RpFt6MSA2hKr7cWLs2VFF065bfLt3XXRc+pNL1JFu+PMzlM2BAzc9TWhoa3lu0CFUvjz4aqjgLC0MVTFXnByopCSMQ7LJLaKRP9vo69lj3qVMr3u/TT0Mya9kyehVdaqK6+OL8V02ecUZoH4oy8V9pqfv++7v37Vu1c4wevXNyMgvVvh9+WK2wRbJOCSqHkiMI3H13fs4/a1b4RnzVVRVvk/z2/dZb1T9PaqnppJPCYKVJU6aE9oKTT65ae8ZNN4XjJbtvb90anrdtGz5IL7ts5/azDz4IPcL23LN6Y6/NmxePWU0XLQqJfp99QtLfvLnibd94I1ynsWOrfp7+/bcnp2bNwt+KRkOROFGCyrHTTw/fSouLa/e8ZWXuxxzj3qZN5Z01NmwIJb3evaveqF9aGnotJktNo0enL30kv61H7dX217+G7dONQL1qVehM0LhxOO/w4eE9/Oc/odpq332zW2WZL5MmuR99tG+r3vzzn9Mnz/POCwm5OiOYFBeHKt599gnnSTdfUm3NJCySjhJUji1cGBJUz57Vb4upjuQUC1EmqktOPjduXPTjf/ZZxaWm8srKwg2ZZu6vvFL5cWfMCNfrv/+78pLDvHlhdAcIozW0aBGGm/rss+jvIe7Kytxfftm9R4/wPrt2DaXy5JeAr74KiTo5BFN1nHeeV3jLQTannxepDiWoWvDcc6ERHkJD/Cef5PZ8a9aEb8VFRdGqrEpKwjhpnTuHbsqV2bIljGrdvHnlpaby1q8P59hzz4pLk8uXh2/p7dtHb6R/882Q/Hv1qr8N+6WloVT5X/8V/oZ693Z//XX3e+4JyzUpMS5dGkpn6X6H2Zo80V0lMakeJahasnZtqOJq0SI0/F9zTfgGnAs/+1n4IKhKO0zyRs/KRn5+++1wAyuEhvzKSk3pfPJJ6JTRu/fOiXDr1lAia9rUffLkqh3XPf8dG2rD1q3uf/pT6HQCofR01FG5O1+2pp9XSUyqSwmqli1ZEkYdKCgIJZDhw6MPvxPF7NkhAQ4ZUrX9ysrcTzwxlHDKjzT91Vehii757bkmw9L87W/hOD/60Y7rk9NZPPlk9Y/dUGzcGL5IHHhgbqevyFYJKspxVMKSdJSg8mTOnO1tKPvu6/4//1PzHmRlZWEUhNatq96t2z0Mh2QWxpRzD1VLf/xjON4uu4Qp7qMMMppJcsSEZM+zZBtYTdpSJPuilnwyJZdMJTGVsKQiSlB59s474SZaCG1GM2dW7zhff719gNJRo6ofz0UXhWq28ePDkE0Q7rHJZs+4LVtCD7XmzUPPsaZNw31IW7dm7xySHZmST5TkkqkElc22LqlflKBioKwsTA+w116hpHLbbZk7K6R67rnQKaKgIJRyalISKy4OCQPCOGcVNaDX1BdfbO84UlioSeXqqqjVd5UlsWy1dUn9owQVIytWuA8e7Nu6FGfq5PDll2FQUghdkSsbYaEqHn00VMPlYp6eVG+/HUbWrm6pUfIvanKprCSmEpRURAkqhl56Kdw826hRSBTlO1GUlYXOBK1bh9LOb36jGXIlP7KRXNQGJRWJmqAa5WasWknn1FNh9mz44Q/D6M7du8Nbb4XXFi6EU06BSy4JI07PmAE33RR95GuRbBo+fOdR25s3D+ujGjQojLTeqROYhZ/JkddTjR0LhYVhNPPCwrAsAqgElS8TJoQREiBMv92iRXg8+KBmC5V4qI0u4iplNUxoPqj427ABbr0VRoyAk04Kc0x16pTvqERqT2FhqD0or1OnMGdY0tixMGwYLFoEHTuGklz5kpjUHVHng1KCioFVq6BVq1ANItKQNGoUyk3lmYUJMiHzRI1S90RNUGqDioHWrZWcpGHq2DHz+mHDdkxOEJaHDctdXBIPSlAikjdROmMsWpR+34rWS/0RKUGZWX8zm2tm881saJrXbzCzj8xslpn9y8w6pbzW0cz+aWZzEtsUZi98EanLovT0i1LKgmi9AdVjsI7J1IsCKAA+BfYHmgAzgS7ltjkOaJ54fjXw15TX3gJOSjxvmdyuokdD6cUnItFE6emXrW2S22UaBkoD4NYM2bpRFzgSeC1l+Sbgpkq27wm8l3jeBXg3SiDJhxKUiJSXKSlEubE4G8M3qVt8dkRNUBl78ZnZOUB/dx+SWL4I+I67X1PB9g8BX7r7r83sTGAIsAXoDLwBDHX30nL7XAFcAdCxY8fDF6brdyoiUoEovQGjbJOp23vUbvFSuWz24kvXvyxtVjOzwUARcE9i1S7AMcDPgd6EasJLdzqY+2h3L3L3onbt2kUISURkuyjtVFG2ydQho7Y7bDT0NrMoCWoxsF/KcgdgSfmNzOxEYBhwurtvTtl3ursvcPcS4AWgV81CFhHZUZTegFG2yZTEarPDRvL+r4ULQ8lv4cKw3KCSVKY6QEIpaAGhii7ZSaJruW16EjpSHFRufUFi+3aJ5ceBH1d2PrVBiUh1ROm8UNN5sGqzw0bUAXvrYqcNsjmaOXAq8EkiCQ1LrLuTUFqC0La0DJiReIxP2fckYBbwAfAE0KSycylBiUg+1bQXX7Y6bESZ8qSudtqImqA01JGISBbVVoeNqNvEkYY6EhHJg2x12NAoG0pQIiJZla0OG9kcZaOuUoISEcmiKIkl6mSOgwaFqrqysvCz/OvZmFgyztQGJSJSh9XFubKitkHtUhvBiIhIbgwaFP+EVF2q4hMRqceyNRpFPka1UAlKRKSeKj8bcXI0CqhaqStbx6kqtUGJiNRT2bpPKtv3W+k+KBGRBi7qfVKZqu/ydb+VEpSISD0V5T6pKIPS5ut+KyUoEZF6Ksp9UsOGbW9bStqwIayvynFyQQlKRKSeinJDcJTqu6g3FmebOkmIiDRg+RhwVp0kREQkozgPl6QEJSLSgOWr+i4K3agrItLAxXW4JJWgREQklpSgREQklpSgREQklpSgREQklpSgREQklpSgREQklmI3koSZLQfS3Ne8g7bAiloIJ1sUb24p3txSvLnVEOPt5O7tMm0UuwQVhZlNiTJMRlwo3txSvLmleHNL8VZMVXwiIhJLSlAiIhJLdTVBjc53AFWkeHNL8eaW4s0txVuBOtkGJSIi9V9dLUGJiEg9pwQlIiKxVKcSlJn1N7O5ZjbfzIbmO55MzKzYzD4wsxlmFstpgs3sMTP7ysw+TFm3p5m9bmbzEj9b5zPGVBXEe7uZfZG4zjPM7NR8xphkZvuZ2QQzm2Nms83susT6WF7fSuKN6/VtZmaTzWxmIt47Eus7m9mkxPX9q5k1yXesUGm8T5jZZynXt0e+Y01lZgVmNt3MXkws19r1rTMJyswKgFHAKUAXYKCZdclvVJEc5+49YnyfwxNA/3LrhgL/cveDgH8lluPiCXaOF+CBxHXu4e4v13JMFSkBfubuhwB9gB8n/mbjen0rihfieX03A8e7+2FAD6C/mfUBfkeI9yBgFXB5HmNMVVG8AL9Iub4z8hdiWtcBc1KWa+361pkEBRwBzHf3Be6+BRgHnJHnmOo8d58IfF1u9RnAk4nnTwJn1mpQlagg3lhy96XuPi3xfC3hn7w9Mb2+lcQbSx6sSyw2TjwcOB54NrE+Tte3onhjy8w6AKcBf0osG7V4fetSgmoPfJ6yvJgY//MkOPBPM5tqZlfkO5gq2Nvdl0L40AL2ynM8UVxjZrMSVYCxqDJLZWaFQE9gEnXg+paLF2J6fRPVTzOAr4DXgU+B1e5ektgkVp8T5eN19+T1HZ64vg+YWdM8hljeCOD/AWWJ5TbU4vWtSwnK0qyL9bcP4Ch370WolvyxmR2b74DqqT8ABxCqTZYC9+U3nB2ZWUvgb8D17v5NvuPJJE28sb2+7l7q7j2ADoRalkPSbVa7UVWsfLxm1g24CTgY6A3sCdyYxxC3MbPvAV+5+9TU1Wk2zdn1rUsJajGwX8pyB2BJnmKJxN2XJH5+BTxP+AeqC5aZ2bcAEj+/ynM8lXL3ZYl//DLgUWJ0nc2sMeHDfqy7P5dYHdvrmy7eOF/fJHdfDbxFaDtrZWa7JF6K5edESrz9E1Wr7u6bgceJz/U9CjjdzIoJTSrHE0pUtXZ961KCeh84KNGDpAlwATA+zzFVyMxamNluyefAycCHle8VG+OBSxLPLwH+nsdYMkp+2CecRUyuc6K+/n+AOe5+f8pLsby+FcUb4+vbzsxaJZ7vCpxIaDebAJyT2CxO1zddvB+nfFkxQntOLK6vu9/k7h3cvZDwefumuw+iFq9vnRpJItG9dQRQADzm7sPzHFKFzGx/QqkJYBfgL3GM18yeBvoRhtBfBvwSeAF4BugILALOdfdYdEyoIN5+hOonB4qBK5NtPPlkZkcD7wAfsL0O/2ZCu07srm8l8Q4knte3O6GRvoDwZfsZd78z8b83jlBdNh0YnCid5FUl8b4JtCNUn80ArkrpTBELZtYP+Lm7f682r2+dSlAiItJw1KUqPhERaUCUoEREJJaUoEREJJaUoEREJJaUoEREJJaUoEREJJaUoEREJJb+P4x3zu/ZvD6rAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "loss = history.history['loss']\n",
    "val_loss = history.history['val_loss']\n",
    "\n",
    "epochs = range(1, len(loss) + 1)\n",
    "\n",
    "plt.figure()\n",
    "\n",
    "plt.plot(epochs, loss, 'bo', label='Training loss')\n",
    "plt.plot(epochs, val_loss, 'b', label='Validation loss')\n",
    "plt.title('Training and validation loss')\n",
    "plt.legend()\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "층을 추가하여 성능을 조금 향상시켰지만 크지는 않습니다. 여기서 두 가지 결론을 얻을 수 있습니다:\n",
    "\n",
    "* 아직 충분히 과대적합을 만들지 못했기 때문에 검증 손실을 향상하기 위해서 층의 크기를 늘릴 수 있습니다. 하지만 적지 않은 계산 비용이 추가됩니다.\n",
    "* 층을 추가한 만큼 도움이 되지 않았으므로 여기서는 네트워크의 용량을 늘리는 것이 도움이 되지 않는다고 볼 수 있습니다."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 양방향 RNN 사용하기\n",
    "\n",
    "이 절에서 소개할 마지막 기법은 양방향 RNN입니다. 양방향 RNN은 RNN의 한 변종이고 특정 작업에서 기본 RNN 보다 훨씬 좋은 성능을 냅니다. 자연어 처리에서는 맥가이버 칼이라고 할 정도로 즐겨 사용됩니다.\n",
    "\n",
    "RNN은 특히 순서 또는 시간에 민감합니다. 즉, 입력 시퀀스의 타입스텝 순서대로 처리합니다. 타입스텝을 섞거나 거꾸로 하면 RNN이 시퀀스에서 학습하는 표현을 완전히 바꾸어 버립니다. 이는 온도 예측과 같이 순서에 의미가 있는 문제에 잘 맞는 이유이기도 합니다. 양방향 RNN은 RNN이 순서에 민감하다는 성질을 사용합니다. 앞서 보았던 GRU나 LSTM 같은 RNN 두 개를 사용합니다. 각 RNN은 입력 시퀀스를 한 방향(시간의 순서나 반대 순서)으로 처리한 다음 각 표현을 합칩니다. 시퀀스를 양쪽 방향으로 처리하기 때문에 양방향 RNN은 단방향 RNN이 놓치기 쉬운 패턴을 감지할 수 있습니다.\n",
    "\n",
    "놀랍게도 이 절에 있는 RNN 층이 시간의 순서대로 (오래된 타임스텝이 먼저 나오도록) 시퀀스를 처리하는 것은 근거 없는 결정입니다. 적어도 이 결정을 궁금해하지 않았습니다. 시간의 반대 방향으로 (최근 타임스텝이 먼저 나오도록) 입력 시퀀스를 처리하면 만족할만한 RNN 성능을 낼 수 있을까요? 실제 이렇게 해 보고 결과가 어떤지 확인해 보죠. 해야 할 일은 입력 시퀀스를 시간 차원을 따라 거꾸로 생성하는 데이터 제너레이터 만드는 것 뿐입니다(제너레이터 함수의 마지막 줄을 `yield samples[:, ::-1, :], targets`로 바꿉니다). 이 절의 첫 번째 예제와 동일하게 하나의 GRU 층을 가진 네트워크를 훈련합니다:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "def reverse_order_generator(data, lookback, delay, min_index, max_index,\n",
    "                            shuffle=False, batch_size=128, step=6):\n",
    "    if max_index is None:\n",
    "        max_index = len(data) - delay - 1\n",
    "    i = min_index + lookback\n",
    "    while 1:\n",
    "        if shuffle:\n",
    "            rows = np.random.randint(\n",
    "                min_index + lookback, max_index, size=batch_size)\n",
    "        else:\n",
    "            if i + batch_size >= max_index:\n",
    "                i = min_index + lookback\n",
    "            rows = np.arange(i, min(i + batch_size, max_index))\n",
    "            i += len(rows)\n",
    "\n",
    "        samples = np.zeros((len(rows),\n",
    "                           lookback // step,\n",
    "                           data.shape[-1]))\n",
    "        targets = np.zeros((len(rows),))\n",
    "        for j, row in enumerate(rows):\n",
    "            indices = range(rows[j] - lookback, rows[j], step)\n",
    "            samples[j] = data[indices]\n",
    "            targets[j] = data[rows[j] + delay][1]\n",
    "        yield samples[:, ::-1, :], targets\n",
    "        \n",
    "train_gen_reverse = reverse_order_generator(\n",
    "    float_data,\n",
    "    lookback=lookback,\n",
    "    delay=delay,\n",
    "    min_index=0,\n",
    "    max_index=200000,\n",
    "    shuffle=True,\n",
    "    step=step, \n",
    "    batch_size=batch_size)\n",
    "val_gen_reverse = reverse_order_generator(\n",
    "    float_data,\n",
    "    lookback=lookback,\n",
    "    delay=delay,\n",
    "    min_index=200001,\n",
    "    max_index=300000,\n",
    "    step=step,\n",
    "    batch_size=batch_size)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/20\n",
      "500/500 [==============================] - 106s 211ms/step - loss: 0.4776 - val_loss: 0.4767\n",
      "Epoch 2/20\n",
      "500/500 [==============================] - 105s 210ms/step - loss: 0.4467 - val_loss: 0.4743\n",
      "Epoch 3/20\n",
      "500/500 [==============================] - 105s 210ms/step - loss: 0.4060 - val_loss: 0.4571\n",
      "Epoch 4/20\n",
      "500/500 [==============================] - 105s 210ms/step - loss: 0.3806 - val_loss: 0.4548\n",
      "Epoch 5/20\n",
      "500/500 [==============================] - 105s 210ms/step - loss: 0.3529 - val_loss: 0.4844\n",
      "Epoch 6/20\n",
      "500/500 [==============================] - 105s 210ms/step - loss: 0.3295 - val_loss: 0.4544\n",
      "Epoch 7/20\n",
      "500/500 [==============================] - 105s 210ms/step - loss: 0.3158 - val_loss: 0.4682\n",
      "Epoch 8/20\n",
      "500/500 [==============================] - 105s 210ms/step - loss: 0.3039 - val_loss: 0.4738\n",
      "Epoch 9/20\n",
      "500/500 [==============================] - 105s 210ms/step - loss: 0.2918 - val_loss: 0.4601\n",
      "Epoch 10/20\n",
      "500/500 [==============================] - 105s 210ms/step - loss: 0.2858 - val_loss: 0.4656\n",
      "Epoch 11/20\n",
      "500/500 [==============================] - 105s 210ms/step - loss: 0.2761 - val_loss: 0.5020\n",
      "Epoch 12/20\n",
      "500/500 [==============================] - 105s 210ms/step - loss: 0.2689 - val_loss: 0.4802\n",
      "Epoch 13/20\n",
      "500/500 [==============================] - 105s 210ms/step - loss: 0.2619 - val_loss: 0.4712\n",
      "Epoch 14/20\n",
      "500/500 [==============================] - 105s 210ms/step - loss: 0.2558 - val_loss: 0.4973\n",
      "Epoch 15/20\n",
      "500/500 [==============================] - 105s 210ms/step - loss: 0.2528 - val_loss: 0.4941\n",
      "Epoch 16/20\n",
      "500/500 [==============================] - 105s 210ms/step - loss: 0.2466 - val_loss: 0.4857\n",
      "Epoch 17/20\n",
      "500/500 [==============================] - 105s 210ms/step - loss: 0.2423 - val_loss: 0.5008\n",
      "Epoch 18/20\n",
      "500/500 [==============================] - 105s 210ms/step - loss: 0.2400 - val_loss: 0.4976\n",
      "Epoch 19/20\n",
      "500/500 [==============================] - 105s 210ms/step - loss: 0.2357 - val_loss: 0.4958\n",
      "Epoch 20/20\n",
      "500/500 [==============================] - 105s 210ms/step - loss: 0.2339 - val_loss: 0.5031\n"
     ]
    }
   ],
   "source": [
    "model = Sequential()\n",
    "model.add(layers.GRU(32, input_shape=(None, float_data.shape[-1])))\n",
    "model.add(layers.Dense(1))\n",
    "\n",
    "model.compile(optimizer=RMSprop(), loss='mae')\n",
    "history = model.fit_generator(train_gen_reverse,\n",
    "                              steps_per_epoch=500,\n",
    "                              epochs=20,\n",
    "                              validation_data=val_gen_reverse,\n",
    "                              validation_steps=val_steps)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAAEYCAYAAAAJeGK1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3Xt8FNX9//HXh7tcBAUUBTF4qXKRmynVioLWKngBa/GrGOuVB2KxXtuvVGyrtLRqrbeWqtRq/dYoUi1KbZX6UxStRQlIUUDKxSARhIiCIIiEfH5/nAlswibZwGZ3kryfj8c+sjNzZuazs5v97Dlz5oy5OyIiInHTKNsBiIiIJKMEJSIisaQEJSIisaQEJSIisaQEJSIisaQEJSIisaQEJbFgZo3NbLOZdU1n2WwysyPMLO3XcZjZqWZWmDC9xMxOTKXsHuzrYTO7eU/Xr2K7vzCzP6V7u1K/NMl2AFI3mdnmhMmWwDZgRzR9pbvn12R77r4DaJ3usg2Bux+Vju2Y2SjgIncfnLDtUenYtsieUIKSPeLuOxNE9At9lLv/v8rKm1kTdy/JRGwiUj+oiU9qRdSE85SZPWlmm4CLzOx4M5ttZhvMbI2Z3W9mTaPyTczMzSwnmn48Wv6CmW0ys3+bWbealo2WDzWz/5rZRjP7rZn9y8wurSTuVGK80syWmdlnZnZ/wrqNzeweM1tvZsuBIVUcn1vMbEqFeZPM7O7o+SgzWxy9nuVR7aaybRWZ2eDoeUsz+3MU20Lg2CT7XRFtd6GZDYvmHwP8Djgxaj79JOHY3pqw/pjota83s2fN7KBUjk11zOycKJ4NZvaKmR2VsOxmM1ttZp+b2fsJr/U4M5sXzV9rZr9OdX9SR7i7Hnrs1QMoBE6tMO8XwFfA2YQfQvsAXwe+Qai5Hwb8F7g6Kt8EcCAnmn4c+ATIBZoCTwGP70HZA4BNwPBo2Q3AduDSSl5LKjE+B7QFcoBPy147cDWwEOgCtAdmhX+xpPs5DNgMtErY9jogN5o+OypjwCnAVqB3tOxUoDBhW0XA4Oj5XcCrwH7AocCiCmX/Bzgoek8ujGI4MFo2Cni1QpyPA7dGz0+LYuwLtAB+D7ySyrFJ8vp/Afwpet49iuOU6D26OTruTYGewEqgU1S2G3BY9HwOMDJ63gb4Rrb/F/RI70M1KKlNb7j739y91N23uvscd3/L3UvcfQUwGRhUxfpPu3uBu28H8glfjDUtexYw392fi5bdQ0hmSaUY46/cfaO7FxKSQdm+/ge4x92L3H09cHsV+1kBvEdInADfBja4e0G0/G/uvsKDV4CXgaQdISr4H+AX7v6Zu68k1IoS9zvV3ddE78kThB8XuSlsFyAPeNjd57v7l8A4YJCZdUkoU9mxqcoFwHR3fyV6j24H9iX8UCghJMOeUTPxB9Gxg/BD40gza+/um9z9rRRfh9QRSlBSm1YlTpjZ0Wb2dzP72Mw+ByYAHapY/+OE51uoumNEZWUPTozD3Z1Q40gqxRhT2hfhl39VngBGRs8vJCTWsjjOMrO3zOxTM9tAqL1UdazKHFRVDGZ2qZn9J2pK2wAcneJ2Iby+ndtz98+Bz4DOCWVq8p5Vtt1SwnvU2d2XADcS3od1UZNxp6joZUAPYImZvW1mZ6T4OqSOUIKS2lSxi/VDhFrDEe6+L/BTQhNWbVpDaHIDwMyM8l+oFe1NjGuAQxKmq+sG/xRwalQDGU5IWJjZPsDTwK8IzW/tgH+mGMfHlcVgZocBDwBXAe2j7b6fsN3qusSvJjQblm2vDaEp8aMU4qrJdhsR3rOPANz9cXc/gdC815hwXHD3Je5+AaEZ9zfAM2bWYi9jkRhRgpJMagNsBL4ws+7AlRnY5/NAfzM728yaANcCHWspxqnAdWbW2czaAzdVVdjd1wJvAI8CS9x9abSoOdAMKAZ2mNlZwLdqEMPNZtbOwnViVycsa01IQsWEXD2KUIMqsxboUtYpJIkngSvMrLeZNSckitfdvdIaaQ1iHmZmg6N9/4hw3vAtM+tuZidH+9saPXYQXsD3zKxDVOPaGL220r2MRWJECUoy6UbgEsKXz0OEGkStipLA+cDdwHrgcOAdwnVb6Y7xAcK5oncJJ/CfTmGdJwidHp5IiHkDcD0wjdDRYAQh0abiZ4SaXCHwAvB/CdtdANwPvB2VORpIPG/zErAUWGtmiU11Zeu/SGhqmxat35VwXmqvuPtCwjF/gJA8hwDDovNRzYE7CecNPybU2G6JVj0DWGyhl+hdwPnu/tXexiPxYaFJXqRhMLPGhCalEe7+erbjEZHKqQYl9Z6ZDTGztlEz0U8IPcPeznJYIlINJShpCAYCKwjNREOAc9y9siY+EYkJNfGJiEgsqQYlIiKxFLvBYjt06OA5OTnZDkNERGrJ3LlzP3H3qi73AGKYoHJycigoKMh2GCIiUkvMrLpRVgA18YmISEwpQYmISCwpQYmISCwpQYmISCwpQYmISCwpQYmISCwpQYmISCwpQYmISCyllKCi0aCXmNkyMxuXZPmlZlZsZvOjx6iEZZeY2dLocUk6gxcRkdr3xRfwzDOQlwc/+1nm9lttgorunzMJGAr0AEaaWY8kRZ9y977R4+Fo3f0JN1D7BjAA+JmZ7Ze26EUagBdegG9/Gz74INuRSEPy6afw2GMwfDh06AAjRsCMGVBSkrkYUhnqaACwzN1XAJjZFGA4sCiFdU8HXnL3T6N1XyLc7uDJPQtXpGHZsAEuvxw+/hhOPBFefhmOOirbUUlVZs+GH/0IduyAww8Pj8MO2/X8wAPBLNtRJvfRR/DsszBtGrz6angNXbrA6NHwne/AwIHQJIMD5KWyq87AqoTpIkKNqKLvmtlJwH+B6919VSXrdq64opmNBkYDdO3aNbXIRRqAceNg3Tp49FG46SY46SR46SXo3TvbkUlFW7bALbfAvfeGL/XDD4dZsyA/HxLvatSy5a6EVfFvTg40a5bZuP/735CQpk2Dt94K844+Gv73f+Hcc+HYY7OXUFNJUMlCq3gTqb8BT7r7NjMbAzwGnJLiurj7ZGAyQG5urm5QJQL861/w0ENw/fVw6aVw/PFw6qkweDC8+CIMGJDtCPfeJ59Aq1awzz7ZjmTvvPoqjBoFy5fD978Pt98ObdqEZdu2QWEhrFgRlpf9XbYM/vlP2Lp113YaNdqV3MoS18EHhya2Dh2gY8fwt02bPU8a7jB/Pvz1ryEpLVwY5ufmwsSJoabUvfveHI30SSVBFQGHJEx3AVYnFnD39QmTfwDuSFh3cIV1X61pkCINzVdfhWaVrl1hwoQw76ij4PXX4VvfCo/nn4dBg7Ib59546CG46qrwhVn2pXzEEbseZV/S++6b7Ugrt2lTqNk+8ECI9dVXd39PmjcP712ypln30HybmLjKnk+fHmrPyTRtWj5hJT4qzuvYEfbbD+bO3VVTKiwMyfDEE+G+++Ccc8JnLW6qvaOumTUhNNt9C/gImANc6O4LE8oc5O5rouffAW5y9+OiThJzgf5R0XnAsWXnpJLJzc113W5DGrqJE0Nz0d/+BmedVX7Z6tWhJvXBB+HLZsiQ7MS4N55/Ppx8P+WU0Gy5bFl4LF8Oa9eWL3vAAbsSVsUEtv/+2Wt+mjEj/IhYtSrUcn/+89B8l06bN4fj8ckn5R/FxcmnP630mzVo1ix0uDn3XDj77JC8ssHM5rp7brXlUrnlu5mdAdwLNAYecfeJZjYBKHD36Wb2K2AYUAJ8Clzl7u9H614O3BxtaqK7P1rVvpSgpKFbuhSOOSZ8gfzlL8nLFBfD6afDe+/BlCnhC6eumDMnNFN27x5qHK1bl1++adOumkRZ4ipLXqtWlS/brl1IVkceGY7X8OHpTxIVffYZ3HhjOC/YvTs88ggcd1zt7jNVJSUhvooJ7JNPQkIfOjQeNdK0JqhMUoKShsw91I4KCmDx4nD+oTIbNsAZZ8Dbb8Of/gQXXZSxMPfYihXhXFrLlvDvf0OnTjVbf+vWUHNMTF7Ll8OCBbBmTTg3c955cPHFofmqUZqHInjuudAsuW5daNr7yU+gRYv07qMhSDVBxe6OuiIN2Z//DK+8Ar//fdXJCULt4Z//DLWGiy8OF1NeeWVm4twT69eHX/Dbt4dru2qanCB0pujRIzwSlZaGHnP/938wdWqo1Rx6aEja3/ve3nfNLy6Ga64JtdXevUMTZf/+1a8ne8ndY/U49thjXaQhKi52b9/e/fjj3XfsSH29rVvdzzzTHdzvuqv24tsbW7a4n3CCe/Pm7q+/Xrv7+uIL9yeecB8yxL1Ro3BcvvEN90mT3D/5pGbbKi11nzLFvUMH96ZN3SdMcN+2rXbibkgIp4eqzQcai08kJn74Q9i4ESZPrlnTVIsWocvweeeFbdx2W/nrbrKttDTU8P71r1BDHDiwdvfXsiWMHBlqaUVFcNddoWlw7Fg46KDQjXratND9uypr1oRzexdcAN26wbx5oUkv09cpNWipZLFMPlSDkobo5ZfDL/0f/3jPt1FS4n7ppWE7N94Yfv3HwfXXx6N2N39+OC6dOoV49t/f/fvfd589u/yxKi11/9Of3Nu1c2/Rwv3Xv3bfvj17cddHpFiDynpCqvhQgpKGZutW9yOPdD/88NAUtjd27HC/+urwnz1mTM2aCmvDvfeGWH7wg/gkzO3b3V94wX3kyJCAwP1rX3P/+c/d//3v0DQI7gMHui9Zku1o6yclKJE64pZbwn/iSy+lZ3ulpe7jxoVtfu972fv1//TT7mbu3/lOqN3F0caN7n/8o/ugQeF4gXurVu6//W32k3t9lmqCUjfzBmbr1nC9SK9eYcwwya5Fi6BvXzj//HB+Jp1++UsYPz6cR3niiTCiQaa8+WYY7aJv39ArsS4MZVRYGMY5PPXUcM5Jao+ug5KkrrgidMEF+OMfw0jZkh2lpWEUhcWL4f33a+eq/vvug+uuC6NNPPNM7V/ECmHw0W9+Mwyv8+ab2RutQOIr1QSlXnwNyMMPh+R0883hV+LYsWHQSMmOhx8OPdvuuqv2vsSvvTbsZ8aMcFHvpk21s58y69aFa53MQi86JSfZG/UuQeXnhyHrGzUKf/Pzsx1RPMydC1dfHcbhmjAhNPm0bx9uQrZxY7aja3g+/jjczmDw4DBSeW264orwf/DGG3DyyWEk9B070r+fL74I4wauWRMuZD3iiPTvQxqWepWg8vPD4I0rV4bTnStXhumGnqQ+/TQkogMOCImpcePwy/app8IxuuyyeF030xBcd104H/jgg5kZ7HTkyHDtz8qVoYbTrRvcemuYToeSkrCPuXPhySfhG8nuGCdSU6n0pMjkY2968R166K6eOImPQw/d403WeTt2uA8dGq6Cf+ut3Zf/5jfhGN19d+Zja6j+8Y9wzG+7LfP73rbNfepU99NOCz3szEK36qef3vMREkpL3a+6Krym3/0uvfFK/URD7GZuljxBgftDD7n//e/hYr3i4vhck1HbJkwIr//3v0++vLQ0dANu0sT9jTcyG1tDtHlz+MF09NHuX36Z3Vg++MD9pz9179IlfEY6dnT/4Q/dFy+u2XZuvz2s/6Mf1UqYUg+lmqDqVS++nJzUmyyaN4fOncOjS5fyf8ued+oUbgxWV82YEZpz8vLCIJqVNSVt3Bhu6/zll2E4lwMOyGycDcmPfhQ6RcyaFUbbjoMdO8Kgsw8/HG6SV1IShiMaNSoMn1RVz78nngifrwsuCE3p6R49XOqnBtnNvOwc1JYtu+a1bBlGhj7llDAu10cf7fpb8XnFsbkaNw69oH71q7o3/tbKlWG05c6dYfbs6rsXz58fboMwcGA4id64cWbiTIevvgrnPmbNCq9z9OjMXvOTqvnzw221L7sM/vCHbEeT3Mcfhx8zDz8c7ku1774hAY0atfvo3a++CqedFj43//xnPI+5xFOqCSrrTXoVH3s7ksTjj4cmFLPw9/HHU1uvtDQ0/c2f7/7886FJsGxcswED3Fes2KuwMurLL92//nX3ffd1/+9/U1/v4YfD6/3pT2svtnT44gv3V15xv/VW91NOcd9nn/LNuUccEc7zxElJiXturvsBB7h/+mm2o6leaan7a6+FkSjKhgPq1y+MCP7ZZ+7vvefetq179+514/VIvNAQz0HVhmeeCf+Ibdu6//Wv2Y4mNWPGhHd22rSarVdaGpKyWRirLC42bAgJZ9w4929+M3T4gBBnv37u110X3pt169xnzHA/6qiwfPjw+PywuO++ENMTT2Q7kpr79NPQ+aFPn/AaWrQItwXp1Mm9sDDb0UldpASVRitWhBoJuF9zTfZPblflscdCnP/7v3u2/hdfuB9zTPgC+vDD9MaWquLikHCuu869f/9d9/Rp0iTcK+mmm0KHlw0bkq+/bVs4cd+qVfgyvfXWvR+EdW98+KF769bup59etzvnlJa6FxSEH0C9ernPnZvtiKSuUoJKs23bwhcmuB97rPvy5dmOaHfz54cv5MGD926A0CVL3Nu0cT/uuMzcnG39+lCzGDPGvUcP39lUV/ZafvazcDuKzZtrtt1Vq9zPPz9sq1s39+eey06CGD48NEPGpTYnkm1KULVk2rRwn5h99w3XjsTFZ5+F2zUcfLD7xx/v/fb+8pfw6bj22r3fVmVKSkLTUbt2YV9t2oRrcn75y9DlPV011Vde2ZX4hg51X7o0PdtNxV//GvZ7xx2Z26dI3ClB1aIPPggdJ8B97NhwP59s2rHDfdiw9F/LdO214TVOnZq+bZZ5881w/ghCR4c336zd20J89VW4GLlNG/dmzdzHj695jaymNmxw79zZvXfvsH8RCZSgatm2be433OA7ezdl8ld5Rb/6VYjj3nvTu91t20IzX+vW7u+/n55trlvnfvnlId6DD3Z/6qnMNrutXu1+0UVh/4ccEmrB6dr/5s3u/+//hV6QJ58cmvXMko/gIdKQKUFlyHPPue+3X/hl/tRTmd//yy+HTgTnn187X/Qffhg6TPTqFTpQ7KmSkjCaRbt2oab3ox+5f/55+uKsqVmzQs0G3L/97ZqPnuDu/skn7s8+G24jPmBAeF0Q3o9+/UKHmpkz0x66SJ2nBJVBhYWhpgFhTLJMNfmtWhWGp+ne3X3Tptrbz4svhprAJZfsWRKcPTv0xoNQs1i4MO0h7pHt28OdU9u2TS1prlwZrqu78srynTmaN3c/8UT3m28O3fM3bszcaxCpi5SgMuyrr8IXHLj37VuzC2T3xLZtoct169buixbV7r7cQ086CBfzpqq42H3UqLDeQQe5P/lkPLtZr13rftllvrPZ8Yknwnm9hQvdH3zQPS/PvWvXXQlp331DZ4tf/tL99dezfw5SpK5RgsqS559333//kDhq86LMH/wgvHuZalYsKXE/9dRQW3jnnerLPvhgaPps0iQ0gWWzOS9V//73rppeq1a7EtKBB7qPGBEutp03L7w+EdlzqSaoejUWX1ysWhUGz3zzzTAu3L33wj77pG/7Tz4JF14Y7il0zz3p2251iouhXz9o0QIKCqBdu93LzJkD3/9+WD54MPzud9CzZ+Zi3Fs7doS7Ds+ZA8cdFwZ0PeKIzNyzSaShaJCDxcbJ9u3wk5/AHXdA795w1VVw8MG7HgccAE2a1Hy7CxfCgAEhUcycmfnR1v/1r5B4zjoL/vrXXV/c69eHW8n/4Q9hFPjf/CYkaX2xi0hFSlAx8cIL4Zbe69aVn28GBx4YktVBB5VPXonTBxywa2Txzz+Hr3893B5j3rywPBvuuQduuCHcNuL668PI1z/+cYjr2mvhZz8Lo2CLiCSjBBUjJSWwdi2sWQOrV+96VJwuLt791uuNGu1KZFu3wpIl8PLLMGhQdl4LhBhHjIDnnoNjjgm3kTjpJJg0CXr1yl5cIlI3pJqg9qCRSWqqSZNdN0KsyvbtIZElS2Br1oRa2EMPZTc5Qaj9PfJIaGr8+GN4/PFwTkzNeSKSTiklKDMbAtwHNAYedvfbKyk3AvgL8HV3LzCzHGAxsCQqMtvdx+xt0PVV06bhTr5dumQ7kuq1bRuaGRs3Dp0mRETSrdoEZWaNgUnAt4EiYI6ZTXf3RRXKtQGuAd6qsInl7t43TfFKjLRqle0IRKQ+a5RCmQHAMndf4e5fAVOA4UnK/Ry4E/gyjfFlTX4+5OSEc0A5OWFaREQyJ5UE1RlYlTBdFM3bycz6AYe4+/NJ1u9mZu+Y2WtmdmKyHZjZaDMrMLOC4uLiVGOvNfn54fqllStDh4CVK8O0kpSISOakkqCSnfre2dfMzBoB9wA3Jim3Bujq7v2AG4AnzGy3DsjuPtndc909t2PHjqlFXovGj4ctW8rP27IlzBcRkcxIJUEVAYckTHcBVidMtwF6Aa+aWSFwHDDdzHLdfZu7rwdw97nAcuBr6Qi8Nn34Yc3mi4hI+qWSoOYAR5pZNzNrBlwATC9b6O4b3b2Du+e4ew4wGxgW9eLrGHWywMwOA44EVqT9VaRZ1641my8iIulXbYJy9xLgamAGocv4VHdfaGYTzGxYNaufBCwws/8ATwNj3P3TvQ26tk2cCC1blp/XsmWYLyIimaGRJCqRnx/OOX34Yag5TZwIeXnZjkpEpO7TSBJ7KS9PCUlEJJtSOQclIiKScUpQIiISS0pQIiISS0pQIiISS0pQIiISS0pQIiISS0pQIiISS0pQIiISS0pQIiISS0pQIiISS0pQIiISS0pQIiISS0pQIiISS0pQIiISS0pQIiISS0pQIiISS0pQIiISS0pQIiISS0pQIiISS0pQIiISS0pQIiISS0pQIiISS0pQIiISS0pQIiISS0pQIiISS0pQtSw/H3JyoFGj8Dc/P9sRiYjUDU2yHUB9lp8Po0fDli1heuXKMA2Ql5e9uERE6gLVoGrR+PG7klOZLVvCfBERqZoSVC368MOazRcRkV1SSlBmNsTMlpjZMjMbV0W5EWbmZpabMO/H0XpLzOz0dARdV3TtWrP5IiKyS7UJyswaA5OAoUAPYKSZ9UhSrg1wDfBWwrwewAVAT2AI8Ptoew3CxInQsmX5eS1bhvkiIlK1VGpQA4Bl7r7C3b8CpgDDk5T7OXAn8GXCvOHAFHff5u4fAMui7TUIeXkweTIceiiYhb+TJ6uDhIhIKlJJUJ2BVQnTRdG8ncysH3CIuz9f03Wj9UebWYGZFRQXF6cUeF2RlweFhVBaGv4qOYmIpCaVBGVJ5vnOhWaNgHuAG2u67s4Z7pPdPdfdczt27JhCSCIiUt+lch1UEXBIwnQXYHXCdBugF/CqmQF0Aqab2bAU1hUREUkqlRrUHOBIM+tmZs0InR6mly10943u3sHdc9w9B5gNDHP3gqjcBWbW3My6AUcCb6f9VYiISL1TbQ3K3UvM7GpgBtAYeMTdF5rZBKDA3adXse5CM5sKLAJKgLHuviNNsYuISD1m7rudEsqq3NxcLygoyHYYIiJSS8xsrrvnVldOI0mIiEgsKUGJiEgsKUGJiEgsKUGJiEgsKUGJiEgsKUGJiEgsKUGJiEgsKUHVEfn5kJMDjRqFv/n52Y5IRKR2pTIWn2RZfj6MHr3r9vErV4Zp0OjoIlJ/qQZVB4wfvys5ldmyJcwXEamvlKDqgA8/rNl8EZH6QAmqDujatWbzRUTqAyWoOmDiRGjZsvy8li3DfBGR+koJqg7Iy4PJk+HQQ8Es/J08WR0kRKR+Uy++OiIvTwlJRBoW1aBERCSWlKBERCSWlKBERCSWlKBERCSWlKBERCSWlKBERCSWlKBERCSWlKBERCSWlKBERCSWlKBERCSWlKBERCSWlKBERCSWlKBERCSWlKAamPx8yMmBRo3C3/z8bEckIpKcbrfRgOTnw+jRsGVLmF65MkyDbuUhIvGjGlQDMn78ruRUZsuWMF9EJG5SSlBmNsTMlpjZMjMbl2T5GDN718zmm9kbZtYjmp9jZluj+fPN7MF0vwBJ3Ycf1my+iEg2VdvEZ2aNgUnAt4EiYI6ZTXf3RQnFnnD3B6Pyw4C7gSHRsuXu3je9Ycue6No1NOslmy8iEjep1KAGAMvcfYW7fwVMAYYnFnD3zxMmWwGevhAlXSZOhJYty89r2TLMFxGJm1QSVGdgVcJ0UTSvHDMba2bLgTuBaxIWdTOzd8zsNTM7MdkOzGy0mRWYWUFxcXENwpeayMuDyZPh0EPBLPydPFkdJEQknsy96sqOmZ0HnO7uo6Lp7wED3P0HlZS/MCp/iZk1B1q7+3ozOxZ4FuhZocZVTm5urhcUFOzhyxERkbgzs7nunltduVRqUEXAIQnTXYDVVZSfApwD4O7b3H199HwusBz4Wgr7FBGRBi6VBDUHONLMuplZM+ACYHpiATM7MmHyTGBpNL9j1MkCMzsMOBJYkY7ARUSkfqu2F5+7l5jZ1cAMoDHwiLsvNLMJQIG7TweuNrNTge3AZ8Al0eonARPMrATYAYxx909r44WIiEj9Uu05qEzTOSgRkfotneegREREMk4JSkREYkkJSkREYkkJSkREYkkJSkREYkkJSkREYkkJSkREYkkJSvaIbh0vIrVNt3yXGtOt40UkE1SDkhrTreNFJBOUoKTGdOt4EckEJSipscpuEa9bx4tIOilBSY3p1vEikglKUFJjunW8iGSCevHJHsnLU0ISkdqlGpSIiMSSEpSIiMSSEpSIiMSSEpRklYZMEpHKqJOEZI2GTBKRqqgGJVmjIZNEpCpKUJI1GjJJRKqiBCVZoyGTRKQqSlCSNRoySUSqogQlWaMhk0SkKurFJ1mlIZNEpDKqQYmISCwpQUm9oAt+ReofNfFJnacLfkXqJ9WgpM7TBb8i9VNKCcrMhpjZEjNbZmbjkiwfY2bvmtl8M3vDzHokLPtxtN4SMzs9ncGLgC74Famvqk1QZtYYmAQMBXoAIxMTUOQJdz/G3fsCdwJ3R+v2AC4AegJDgN9H2xNJG13wK1I/pVKDGgAsc/cV7v4VMAUYnljA3T9PmGwFePR8ODDF3be5+wfAsmh7ImmjC35F6qdUElRnYFXCdFE0rxwzG2tmywk1qGtquO5oMysws4Li4uJUYxcBdMGvSH2VSoKyJPN8txnuk9z9cOAm4JYarjvZ3XPdPbdjx44phCQ1Qt6PAAASpklEQVRSXl4eFBZCaWn4q+QkUvelkqCKgEMSprsAq6soPwU4Zw/XFRERAVJLUHOAI82sm5k1I3R6mJ5YwMyOTJg8E1gaPZ8OXGBmzc2sG3Ak8Pbehy0iIvVdtQnK3UuAq4EZwGJgqrsvNLMJZjYsKna1mS00s/nADcAl0boLganAIuBFYKy776iF1yGSFhqRQiQ+zH23U0JZlZub6wUFBdkOQxqgiiNSQOgNqA4XIullZnPdPbe6chpJQiSiESlE4kUJSiSiESlE4kUJSiSSzhEpdC5LZO8pQYlE0jUiRdm5rJUrwX3X6OpKUiI1owQlEknXiBQ6lyWSHurFJ5JmjRqFmlNFZmGkC5GGTr34RLJEo6uLpIcSlEiaaXR1kfRQghJJM42uLpIeTbIdgEh9lJenhCSyt1SDEokxXU8lDZlqUCIxVXFswLLrqUC1M2kYVIMSiSldTyUNnRKUSExpbEBp6JSgRGJK11NJQ6cEJRJTup5KGjolKJGYSuf1VOoNKHWRevGJxFg6rqdSb0Cpq1SDEqnn1BtQ6qo6UYPavn07RUVFfPnll9kORVLQokULunTpQtOmTbMdiqDegFJ31YkEVVRURJs2bcjJycHMsh2OVMHdWb9+PUVFRXTr1i3b4Qih19/Klcnni8RZnWji+/LLL2nfvr2SUx1gZrRv31613RhJZ29AdbaQTKoTCQpQcqpD9F7FS7p6A+pW9pJpdSZBiciey8uDwsJwR9/Cwj3rvafOFpJp9TJBpbsZYv369fTt25e+ffvSqVMnOnfuvHP6q6++Smkbl112GUuWLKmyzKRJk8hP08/RgQMHMn/+/LRsSwTU2UIyr050kqiJ2rjmo3379ju/7G+99VZat27ND3/4w3Jl3B13p1Gj5Dn/0UcfrXY/Y8eO3bMARTJAnS0k0+pdDSqTzRDLli2jV69ejBkzhv79+7NmzRpGjx5Nbm4uPXv2ZMKECTvLltVoSkpKaNeuHePGjaNPnz4cf/zxrFu3DoBbbrmFe++9d2f5cePGMWDAAI466ijefPNNAL744gu++93v0qdPH0aOHElubm61NaXHH3+cY445hl69enHzzTcDUFJSwve+972d8++//34A7rnnHnr06EGfPn246KKL0n7MpO5K99BL6nAh1al3NahMN0MsWrSIRx99lAcffBCA22+/nf3335+SkhJOPvlkRowYQY8ePcqts3HjRgYNGsTtt9/ODTfcwCOPPMK4ceN227a78/bbbzN9+nQmTJjAiy++yG9/+1s6derEM888w3/+8x/69+9fZXxFRUXccsstFBQU0LZtW0499VSef/55OnbsyCeffMK7774LwIYNGwC48847WblyJc2aNds5TwR2tUCMHx/+n7p2DclpT4de0ugWUp16V4PK9AjQhx9+OF//+td3Tj/55JP079+f/v37s3jxYhYtWrTbOvvssw9Dhw4F4Nhjj6WwsDDpts8999zdyrzxxhtccMEFAPTp04eePXtWGd9bb73FKaecQocOHWjatCkXXnghs2bN4ogjjmDJkiVce+21zJgxg7Zt2wLQs2dPLrroIvLz83WhrewmHZ0tIL0tHaqJ1V/1LkFlegToVq1a7Xy+dOlS7rvvPl555RUWLFjAkCFDkl4P1KxZs53PGzduTElJSdJtN2/efLcy7l6j+Cor3759exYsWMDAgQO5//77ufLKKwGYMWMGY8aM4e233yY3N5cdO3bUaH8iqUhXS4e6vtdvKSUoMxtiZkvMbJmZ7dYWZWY3mNkiM1tgZi+b2aEJy3aY2fzoMT2dwSeTzhGga+rzzz+nTZs27LvvvqxZs4YZM2akfR8DBw5k6tSpALz77rtJa2iJjjvuOGbOnMn69espKSlhypQpDBo0iOLiYtyd8847j9tuu4158+axY8cOioqKOOWUU/j1r39NcXExWyr+zBVJg3S1dKjre/1W7TkoM2sMTAK+DRQBc8xsursnfjO+A+S6+xYzuwq4Ezg/WrbV3fumOe4qpWME6D3Rv39/evToQa9evTjssMM44YQT0r6PH/zgB1x88cX07t2b/v3706tXr53Nc8l06dKFCRMmMHjwYNyds88+mzPPPJN58+ZxxRVX4O6YGXfccQclJSVceOGFbNq0idLSUm666SbatGmT9tcgMnFi+XNQsGctHer6Xr9ZdU1GZnY8cKu7nx5N/xjA3X9VSfl+wO/c/YRoerO7t041oNzcXC8oKCg3b/HixXTv3j3VTdRrJSUllJSU0KJFC5YuXcppp53G0qVLadIkXv1d9J5JdfLz977DRU5O8q7vhx4azpFJPJnZXHfPra5cKk18nYFVCdNF0bzKXAG8kDDdwswKzGy2mZ2Twv6kCps3b+aEE06gT58+fPe73+Whhx6KXXISSUU6OlxonMH6LZVvtmQDqyWtdpnZRUAuMChhdld3X21mhwGvmNm77r68wnqjgdEAXXXVX5XatWvH3Llzsx2GSCykq+u7ur3HUyo1qCLgkITpLsDqioXM7FRgPDDM3beVzXf31dHfFcCrQL+K67r7ZHfPdffcjh071ugFiEjDpnEG669UEtQc4Egz62ZmzYALgHK98aLzTg8RktO6hPn7mVnz6HkH4ASg6m5nIiIZps4W8VRtgnL3EuBqYAawGJjq7gvNbIKZDYuK/RpoDfylQnfy7kCBmf0HmAncXqH3n4hI1qXzAn+dy0qflK6Dcvd/uPvX3P1wd58Yzfupu0+Pnp/q7ge6e9/oMSya/6a7H+PufaK/f6y9lyIismfS1dkinRcOK9HVw5EkasPgwYN3u+j23nvv5fvf/36V67VuHXrXr169mhEjRlS67Yrd6iu69957y10we8YZZ6RlnLxbb72Vu+66a6+3I1LXpesC/3Sdy9IIGYESVApGjhzJlClTys2bMmUKI0eOTGn9gw8+mKeffnqP918xQf3jH/+gXbt2e7w9EdldOjpbpOtcljptBHXuAprrroN034evb1+I7nKR1IgRI7jlllvYtm0bzZs3p7CwkNWrVzNw4EA2b97M8OHD+eyzz9i+fTu/+MUvGD58eLn1CwsLOeuss3jvvffYunUrl112GYsWLaJ79+5s3bp1Z7mrrrqKOXPmsHXrVkaMGMFtt93G/fffz+rVqzn55JPp0KEDM2fOJCcnh4KCAjp06MDdd9/NI488AsCoUaO47rrrKCwsZOjQoQwcOJA333yTzp0789xzz7HPPvtU+hrnz5/PmDFj2LJlC4cffjiPPPII++23H/fffz8PPvggTZo0oUePHkyZMoXXXnuNa6+9Fgi3d581a5ZGnBAhfffMUqeNQDWoFLRv354BAwbw4osvAqH2dP7552NmtGjRgmnTpjFv3jxmzpzJjTfeWOWArg888AAtW7ZkwYIFjB8/vtw1TRMnTqSgoIAFCxbw2muvsWDBAq655hoOPvhgZs6cycyZM8tta+7cuTz66KO89dZbzJ49mz/84Q+88847QBi4duzYsSxcuJB27drxzDPPVPkaL774Yu644w4WLFjAMcccw2233QaE24e88847LFiwYOctRe666y4mTZrE/Pnzef3116tMfCINSbrOZanTRlDnalBV1XRqU1kz3/Dhw5kyZcrOWou7c/PNNzNr1iwaNWrERx99xNq1a+nUqVPS7cyaNYtrrrkGgN69e9O7d++dy6ZOncrkyZMpKSlhzZo1LFq0qNzyit544w2+853v7BxR/dxzz+X1119n2LBhdOvWjb59wxCIVd3SA8L9qTZs2MCgQeH66ksuuYTzzjtvZ4x5eXmcc845nHNOGAjkhBNO4IYbbiAvL49zzz2XLl26pHIIReq9dF04nK6xCtN5AXI6hqaqKdWgUnTOOefw8ssvM2/ePLZu3brzRoH5+fkUFxczd+5c5s+fz4EHHpj0FhuJzHYfnOODDz7grrvu4uWXX2bBggWceeaZ1W6nqppa2a06oOpbelTn73//O2PHjmXu3Lkce+yxlJSUMG7cOB5++GG2bt3Kcccdx/vvv79H2xapj9JxLkudNgIlqBS1bt2awYMHc/nll5frHLFx40YOOOAAmjZtysyZM1mZrAE6wUknnUR+9K6+9957LFiwAAi36mjVqhVt27Zl7dq1vPDCruEM27Rpw6ZNm5Ju69lnn2XLli188cUXTJs2jRNPPLHGr61t27bst99+vP766wD8+c9/ZtCgQZSWlrJq1SpOPvlk7rzzTjZs2MDmzZtZvnw5xxxzDDfddBO5ublKUCK1QJ026mATXzaNHDmSc889t1yPvry8PM4++2xyc3Pp27cvRx99dJXbuOqqq7jsssvo3bs3ffv2ZcCAAUC4O26/fv3o2bPnbrfqGD16NEOHDuWggw4qdx6qf//+XHrppTu3MWrUKPr161dlc15lHnvssZ2dJA477DAeffRRduzYwUUXXcTGjRtxd66//nratWvHT37yE2bOnEnjxo3p0aPHzrsDi0i81PVOG9XebiPTdLuN+kHvmUj2VTwHBeFcVk2bC9N9W5N03m5DRETqoHSdy0rnbU1qQk18IiL1WDruMJ6u3ok1VWcSVNmtySX+4tZsLCJ7Lx2JrqbqRBNfixYtWL9+vb746gB3Z/369bRo0SLboYhIHVcnalBdunShqKiI4uLibIciKWjRooUu3hWRvVYnElTTpk3p1q1btsMQEZEMqhNNfCIi0vAoQYmISCwpQYmISCzFbiQJMysGqh7QLn46AJ9kO4gaUsyZoZgzpy7G3VBjPtTdO1ZXKHYJqi4ys4JUhu2IE8WcGYo5c+pi3Iq5amriExGRWFKCEhGRWFKCSo/J2Q5gDyjmzFDMmVMX41bMVdA5KBERiSXVoEREJJaUoEREJJaUoFJgZoeY2UwzW2xmC83s2iRlBpvZRjObHz1+mo1YKzKzQjN7N4qpIMlyM7P7zWyZmS0ws/7ZiDMhnqMSjuF8M/vczK6rUCbrx9rMHjGzdWb2XsK8/c3sJTNbGv3dr5J1L4nKLDWzS7Ic86/N7P3ovZ9mZu0qWbfKz1FtqiTuW83so4TPwBmVrDvEzJZEn+9xWY75qYR4C81sfiXrZuVYV/Y9l9XPtbvrUc0DOAjoHz1vA/wX6FGhzGDg+WzHmiT2QqBDFcvPAF4ADDgOeCvbMSfE1hj4mHBRX6yONXAS0B94L2HencC46Pk44I4k6+0PrIj+7hc93y+LMZ8GNIme35Es5lQ+R1mI+1bghyl8fpYDhwHNgP9U/L/NZMwVlv8G+GmcjnVl33PZ/FyrBpUCd1/j7vOi55uAxUDn7EaVNsOB//NgNtDOzA7KdlCRbwHL3T12I4u4+yzg0wqzhwOPRc8fA85JsurpwEvu/qm7fwa8BAyptUATJIvZ3f/p7iXR5GwgdvdJqeRYp2IAsMzdV7j7V8AUwntU66qK2cKdV/8HeDITsaSqiu+5rH2ulaBqyMxygH7AW0kWH29m/zGzF8ysZ0YDq5wD/zSzuWY2OsnyzsCqhOki4pN8L6Dyf+I4HusD3X0NhH924IAkZeJ8vC8n1KaTqe5zlA1XR02Tj1TS7BTXY30isNbdl1ayPOvHusL3XNY+10pQNWBmrYFngOvc/fMKi+cRmqL6AL8Fns10fJU4wd37A0OBsWZ2UoXllmSdrF97YGbNgGHAX5IsjuuxTkVcj/d4oATIr6RIdZ+jTHsAOBzoC6whNJlVFMtjDYyk6tpTVo91Nd9zla6WZN5eH2slqBSZWVPCm5bv7n+tuNzdP3f3zdHzfwBNzaxDhsPcjbuvjv6uA6YRmj0SFQGHJEx3AVZnJroqDQXmufvaigvieqyBtWXNo9HfdUnKxO54Rye0zwLyPDqhUFEKn6OMcve17r7D3UuBP1QSTxyPdRPgXOCpyspk81hX8j2Xtc+1ElQKojbjPwKL3f3uSsp0isphZgMIx3Z95qJMGlMrM2tT9pxwQvy9CsWmAxdHvfmOAzaWVeezrNJfmXE81pHpQFnvpUuA55KUmQGcZmb7Rc1Sp0XzssLMhgA3AcPcfUslZVL5HGVUhfOk3yF5PHOAI82sW1Qjv4DwHmXTqcD77l6UbGE2j3UV33PZ+1xnuqdIXXwAAwnV1QXA/OhxBjAGGBOVuRpYSOgpNBv4ZgziPiyK5z9RbOOj+YlxGzCJ0NvpXSA3BnG3JCSctgnzYnWsCclzDbCd8OvxCqA98DKwNPq7f1Q2F3g4Yd3LgWXR47Isx7yMcO6g7HP9YFT2YOAfVX2Oshz3n6PP6wLCF+hBFeOOps8g9EZbnsm4k8Uczf9T2ec4oWwsjnUV33NZ+1xrqCMREYklNfGJiEgsKUGJiEgsKUGJiEgsKUGJiEgsKUGJiEgsKUGJiEgsKUGJiEgs/X+EN+D4OU1CtgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "loss = history.history['loss']\n",
    "val_loss = history.history['val_loss']\n",
    "\n",
    "epochs = range(1, len(loss) + 1)\n",
    "\n",
    "plt.figure()\n",
    "\n",
    "plt.plot(epochs, loss, 'bo', label='Training loss')\n",
    "plt.plot(epochs, val_loss, 'b', label='Validation loss')\n",
    "plt.title('Training and validation loss')\n",
    "plt.legend()\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "순서를 뒤집은 GRU는 상식 수준의 기준점보다도 성능이 낮습니다. 이 경우에는 시간 순서대로 처리하는 것이 중요한 역할을 합니다. 사실 이는 당연합니다. 기본적인 GRU 층은 먼 과거보다 최근 내용을 잘 기억합니다. 또한, 최근에 가까운 날씨 데이터 포인트일수록 오래된 데이터 포인트보다 예측에 유용합니다(상식 수준의 기준점이 꽤 강력한 이유입니다). 시간 순서대로 처리하는 네트워크가 거꾸로 처리하는 것보다 성능이 높아야만 합니다. 하지만 자연어 처리를 포함하여 다른 많은 문제에서는 그렇지 않습니다. 문장을 이해하는데 있어서 단어의 중요성은 단어가 문장 어디에 놓여 있는지에 따라 결정되지 않습니다. 같은 기법을 이전 절의 LSTM IMDB 예제에 적용해 보죠:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train on 20000 samples, validate on 5000 samples\n",
      "Epoch 1/10\n",
      "20000/20000 [==============================] - 73s 4ms/step - loss: 0.4925 - acc: 0.7677 - val_loss: 0.3748 - val_acc: 0.8578\n",
      "Epoch 2/10\n",
      "20000/20000 [==============================] - 72s 4ms/step - loss: 0.3147 - acc: 0.8774 - val_loss: 0.4113 - val_acc: 0.8404\n",
      "Epoch 3/10\n",
      "20000/20000 [==============================] - 72s 4ms/step - loss: 0.2522 - acc: 0.9049 - val_loss: 0.3159 - val_acc: 0.8884\n",
      "Epoch 4/10\n",
      "20000/20000 [==============================] - 72s 4ms/step - loss: 0.2204 - acc: 0.9194 - val_loss: 0.4816 - val_acc: 0.8520\n",
      "Epoch 5/10\n",
      "20000/20000 [==============================] - 72s 4ms/step - loss: 0.1934 - acc: 0.9319 - val_loss: 0.3790 - val_acc: 0.8660\n",
      "Epoch 6/10\n",
      "20000/20000 [==============================] - 72s 4ms/step - loss: 0.1771 - acc: 0.9378 - val_loss: 0.3594 - val_acc: 0.8814\n",
      "Epoch 7/10\n",
      "20000/20000 [==============================] - 72s 4ms/step - loss: 0.1534 - acc: 0.9475 - val_loss: 0.4477 - val_acc: 0.8694\n",
      "Epoch 8/10\n",
      "20000/20000 [==============================] - 72s 4ms/step - loss: 0.1457 - acc: 0.9517 - val_loss: 0.6034 - val_acc: 0.8502\n",
      "Epoch 9/10\n",
      "20000/20000 [==============================] - 72s 4ms/step - loss: 0.1322 - acc: 0.9569 - val_loss: 0.4100 - val_acc: 0.8664\n",
      "Epoch 10/10\n",
      "20000/20000 [==============================] - 72s 4ms/step - loss: 0.1213 - acc: 0.9593 - val_loss: 0.4392 - val_acc: 0.8764\n"
     ]
    }
   ],
   "source": [
    "from keras.datasets import imdb\n",
    "from keras.preprocessing import sequence\n",
    "from keras import layers\n",
    "from keras.models import Sequential\n",
    "\n",
    "# 특성으로 사용할 단어의 수\n",
    "max_features = 10000\n",
    "# 사용할 텍스트의 길이(가장 빈번한 max_features 개의 단어만 사용합니다)\n",
    "maxlen = 500\n",
    "\n",
    "# 데이터 로드\n",
    "(x_train, y_train), (x_test, y_test) = imdb.load_data(num_words=max_features)\n",
    "\n",
    "# 시퀀스를 뒤집습니다\n",
    "x_train = [x[::-1] for x in x_train]\n",
    "x_test = [x[::-1] for x in x_test]\n",
    "\n",
    "# 시퀀스에 패딩을 추가합니다\n",
    "x_train = sequence.pad_sequences(x_train, maxlen=maxlen)\n",
    "x_test = sequence.pad_sequences(x_test, maxlen=maxlen)\n",
    "\n",
    "model = Sequential()\n",
    "model.add(layers.Embedding(max_features, 128))\n",
    "model.add(layers.LSTM(32))\n",
    "model.add(layers.Dense(1, activation='sigmoid'))\n",
    "\n",
    "model.compile(optimizer='rmsprop',\n",
    "              loss='binary_crossentropy',\n",
    "              metrics=['acc'])\n",
    "history = model.fit(x_train, y_train,\n",
    "                    epochs=10,\n",
    "                    batch_size=128,\n",
    "                    validation_split=0.2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "시간 순서로 훈련한 LSTM과 거의 동일한 성능을 얻을 수 있습니다. 놀랍게도 이런 텍스트 데이터셋에는 순서를 뒤집어 처리하는 것이 시간 순서대로 처리하는 것과 거의 동일하게 잘 작동합니다. 이는 언어를 이해하는 데 단어의 순서가 중요하지만 결정적이지는 않다는 가정을 뒷받침합니다. 거꾸로 된 시퀀스에서 훈련한 RNN은 원래 시퀀스에서 훈련한 것과는 다른 표현을 학습합니다. 이와 비슷하게 시작할 때 죽고 마지막 날 태어나는 삶처럼 실제 세상의 시간이 거꾸로 흘러간다면 우리의 정신 세계가 달라질 것입니다. 머신 러닝에서 다른 표현이 유용하다면 항상 사용할 가치가 있습니다. 이 표현이 많이 다를수록 더 좋습니다. 이 표현이 데이터를 바라보는 새로운 시각을 제공하고 다른 방식에서는 놓칠 수 있는 데이터의 특징을 잡아냅니다. 이런 표현은 작업의 성능을 올리는 데 도움을 줍니다. 이것이 다음 장에서 살펴볼 앙상블의 개념입니다.\n",
    "\n",
    "양방향 RNN은 이 아이디어를 사용하여 시간 순서대로 처리하는 RNN의 성능을 향상시킵니다. 입력 시퀀스를 양쪽 방향으로 바라보기 때문에, 드러나지 않은 다양한 표현을 얻어 시간 순서대로 처리할 때 놓칠 수 있는 패턴을 잡아 냅니다."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![bidirectional rnn](https://s3.amazonaws.com/book.keras.io/img/ch6/bidirectional_rnn.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "케라스에서는 `Bidirectional` 층을 사용하여 양방향 RNN을 만듭니다. 이 클래스는 첫 번째 매개변수로 순환 층의 객체를 전달받습니다. `Bidirectional` 클래스는 전달받은 순환 층으로 새로운 두 번째 객체를 만듭니다. 하나는 시간 순서대로 입력 시퀀스를 처리하고 다른 하나는 반대 순서로 입력 시퀀스를 처리합니다. IMDB 감성 분석 문제에 이를 적용해 보죠:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "from keras import backend as K\n",
    "K.clear_session()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train on 20000 samples, validate on 5000 samples\n",
      "Epoch 1/10\n",
      "20000/20000 [==============================] - 135s 7ms/step - loss: 0.5623 - acc: 0.7147 - val_loss: 0.4869 - val_acc: 0.7700\n",
      "Epoch 2/10\n",
      "20000/20000 [==============================] - 134s 7ms/step - loss: 0.3304 - acc: 0.8735 - val_loss: 0.3264 - val_acc: 0.8866\n",
      "Epoch 3/10\n",
      "20000/20000 [==============================] - 134s 7ms/step - loss: 0.2716 - acc: 0.9008 - val_loss: 0.4408 - val_acc: 0.7924\n",
      "Epoch 4/10\n",
      "20000/20000 [==============================] - 134s 7ms/step - loss: 0.2357 - acc: 0.9154 - val_loss: 0.3052 - val_acc: 0.8832\n",
      "Epoch 5/10\n",
      "20000/20000 [==============================] - 134s 7ms/step - loss: 0.2100 - acc: 0.9256 - val_loss: 0.3620 - val_acc: 0.8810\n",
      "Epoch 6/10\n",
      "20000/20000 [==============================] - 134s 7ms/step - loss: 0.1877 - acc: 0.9355 - val_loss: 0.4297 - val_acc: 0.8544\n",
      "Epoch 7/10\n",
      "20000/20000 [==============================] - 134s 7ms/step - loss: 0.1689 - acc: 0.9427 - val_loss: 0.3866 - val_acc: 0.8578\n",
      "Epoch 8/10\n",
      "20000/20000 [==============================] - 134s 7ms/step - loss: 0.1531 - acc: 0.9471 - val_loss: 0.3366 - val_acc: 0.8754\n",
      "Epoch 9/10\n",
      "20000/20000 [==============================] - 134s 7ms/step - loss: 0.1416 - acc: 0.9522 - val_loss: 0.3549 - val_acc: 0.8642\n",
      "Epoch 10/10\n",
      "20000/20000 [==============================] - 134s 7ms/step - loss: 0.1336 - acc: 0.9556 - val_loss: 0.3664 - val_acc: 0.8770\n"
     ]
    }
   ],
   "source": [
    "model = Sequential()\n",
    "model.add(layers.Embedding(max_features, 32))\n",
    "model.add(layers.Bidirectional(layers.LSTM(32)))\n",
    "model.add(layers.Dense(1, activation='sigmoid'))\n",
    "\n",
    "model.compile(optimizer='rmsprop', loss='binary_crossentropy', metrics=['acc'])\n",
    "history = model.fit(x_train, y_train, epochs=10, batch_size=128, validation_split=0.2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "여기서 얻은 검증 정확도는 88% 정도로 이전 절에서 사용했던 일반 LSTM보다 조금 더 성능이 높습니다. 이 모델은 조금 더 일찍 과대적합되는 것 같습니다. 양방향 순환 층이 단방향 LSTM보다 모델 파라미터가 두 배 많기 때문에 놀라운 일은 아닙니다. 규제를 조금 추가한다면 양항뱡 순환 층을 사용하는 것이 이 작업에 더 적합해 보입니다.\n",
    "\n",
    "이제 동일한 방식을 온도 예측 문제에 적용해 보죠:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/40\n",
      "500/500 [==============================] - 196s 391ms/step - loss: 0.2930 - val_loss: 0.2739\n",
      "Epoch 2/40\n",
      "500/500 [==============================] - 195s 390ms/step - loss: 0.2743 - val_loss: 0.2716\n",
      "Epoch 3/40\n",
      "500/500 [==============================] - 195s 390ms/step - loss: 0.2666 - val_loss: 0.2689\n",
      "Epoch 4/40\n",
      "500/500 [==============================] - 195s 390ms/step - loss: 0.2628 - val_loss: 0.2705\n",
      "Epoch 5/40\n",
      "500/500 [==============================] - 195s 390ms/step - loss: 0.2553 - val_loss: 0.2730\n",
      "Epoch 6/40\n",
      "500/500 [==============================] - 195s 390ms/step - loss: 0.2488 - val_loss: 0.2715\n",
      "Epoch 7/40\n",
      "500/500 [==============================] - 195s 390ms/step - loss: 0.2448 - val_loss: 0.2799\n",
      "Epoch 8/40\n",
      "500/500 [==============================] - 195s 390ms/step - loss: 0.2399 - val_loss: 0.2820\n",
      "Epoch 9/40\n",
      "500/500 [==============================] - 195s 390ms/step - loss: 0.2334 - val_loss: 0.2822\n",
      "Epoch 10/40\n",
      "500/500 [==============================] - 195s 390ms/step - loss: 0.2282 - val_loss: 0.2832\n",
      "Epoch 11/40\n",
      "500/500 [==============================] - 195s 390ms/step - loss: 0.2226 - val_loss: 0.2878\n",
      "Epoch 12/40\n",
      "500/500 [==============================] - 195s 390ms/step - loss: 0.2163 - val_loss: 0.2893\n",
      "Epoch 13/40\n",
      "500/500 [==============================] - 195s 390ms/step - loss: 0.2099 - val_loss: 0.2917\n",
      "Epoch 14/40\n",
      "500/500 [==============================] - 195s 390ms/step - loss: 0.2065 - val_loss: 0.2999\n",
      "Epoch 15/40\n",
      "500/500 [==============================] - 195s 390ms/step - loss: 0.2009 - val_loss: 0.3017\n",
      "Epoch 16/40\n",
      "500/500 [==============================] - 195s 390ms/step - loss: 0.1969 - val_loss: 0.3040\n",
      "Epoch 17/40\n",
      "500/500 [==============================] - 195s 390ms/step - loss: 0.1914 - val_loss: 0.3053\n",
      "Epoch 18/40\n",
      "500/500 [==============================] - 195s 390ms/step - loss: 0.1899 - val_loss: 0.3080\n",
      "Epoch 19/40\n",
      "500/500 [==============================] - 195s 390ms/step - loss: 0.1850 - val_loss: 0.3129\n",
      "Epoch 20/40\n",
      "500/500 [==============================] - 195s 389ms/step - loss: 0.1807 - val_loss: 0.3135\n",
      "Epoch 21/40\n",
      "500/500 [==============================] - 195s 390ms/step - loss: 0.1766 - val_loss: 0.3119\n",
      "Epoch 22/40\n",
      "500/500 [==============================] - 194s 389ms/step - loss: 0.1734 - val_loss: 0.3178\n",
      "Epoch 23/40\n",
      "500/500 [==============================] - 195s 390ms/step - loss: 0.1694 - val_loss: 0.3176\n",
      "Epoch 24/40\n",
      "500/500 [==============================] - 195s 390ms/step - loss: 0.1659 - val_loss: 0.3228\n",
      "Epoch 25/40\n",
      "500/500 [==============================] - 195s 390ms/step - loss: 0.1640 - val_loss: 0.3193\n",
      "Epoch 26/40\n",
      "500/500 [==============================] - 195s 390ms/step - loss: 0.1612 - val_loss: 0.3236\n",
      "Epoch 27/40\n",
      "500/500 [==============================] - 195s 390ms/step - loss: 0.1587 - val_loss: 0.3241\n",
      "Epoch 28/40\n",
      "500/500 [==============================] - 195s 390ms/step - loss: 0.1548 - val_loss: 0.3214\n",
      "Epoch 29/40\n",
      "500/500 [==============================] - 195s 390ms/step - loss: 0.1533 - val_loss: 0.3263\n",
      "Epoch 30/40\n",
      "500/500 [==============================] - 195s 391ms/step - loss: 0.1523 - val_loss: 0.3303\n",
      "Epoch 31/40\n",
      "500/500 [==============================] - 195s 390ms/step - loss: 0.1501 - val_loss: 0.3229\n",
      "Epoch 32/40\n",
      "500/500 [==============================] - 195s 390ms/step - loss: 0.1482 - val_loss: 0.3286\n",
      "Epoch 33/40\n",
      "500/500 [==============================] - 195s 390ms/step - loss: 0.1451 - val_loss: 0.3310\n",
      "Epoch 34/40\n",
      "500/500 [==============================] - 195s 390ms/step - loss: 0.1445 - val_loss: 0.3332\n",
      "Epoch 35/40\n",
      "500/500 [==============================] - 195s 390ms/step - loss: 0.1429 - val_loss: 0.3343\n",
      "Epoch 36/40\n",
      "500/500 [==============================] - 195s 390ms/step - loss: 0.1422 - val_loss: 0.3372\n",
      "Epoch 37/40\n",
      "500/500 [==============================] - 195s 389ms/step - loss: 0.1407 - val_loss: 0.3314\n",
      "Epoch 38/40\n",
      "500/500 [==============================] - 195s 390ms/step - loss: 0.1394 - val_loss: 0.3357\n",
      "Epoch 39/40\n",
      "500/500 [==============================] - 195s 390ms/step - loss: 0.1371 - val_loss: 0.3326\n",
      "Epoch 40/40\n",
      "500/500 [==============================] - 195s 390ms/step - loss: 0.1363 - val_loss: 0.3369\n"
     ]
    }
   ],
   "source": [
    "from keras.models import Sequential\n",
    "from keras import layers\n",
    "from keras.optimizers import RMSprop\n",
    "\n",
    "model = Sequential()\n",
    "model.add(layers.Bidirectional(\n",
    "    layers.GRU(32), input_shape=(None, float_data.shape[-1])))\n",
    "model.add(layers.Dense(1))\n",
    "\n",
    "model.compile(optimizer=RMSprop(), loss='mae')\n",
    "history = model.fit_generator(train_gen,\n",
    "                              steps_per_epoch=500,\n",
    "                              epochs=40,\n",
    "                              validation_data=val_gen,\n",
    "                              validation_steps=val_steps)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "이 네트워크는 일반 GRU 층과 비슷한 성능을 냅니다. 이유는 쉽게 이해할 수 있습니다. 모든 예측 성능은 시간 순서대로 처리하는 네트워크의 절반에서 옵니다. 시간 반대 순서로 처리하는 절반은 이런 작업에 성능이 매우 좋지 않기 때문입니다(최근의 정보가 오래 전의 정보보다 훨씬 더 중요합니다)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 더 나아가서\n",
    "\n",
    "온도 예측 문제의 성능을 향상하기 위해 시도해 볼 수 있는 것들이 많이 있습니다.\n",
    "\n",
    "* 스태킹한 각 순환 층의 유닛 수를 조정합니다. 지금 설정은 대부분 임의로 한 것이라 최적화가 덜 되었을 것입니다.\n",
    "* `RMSprop` 옵티마이저가 사용한 학습률을 조정합니다.\n",
    "* `GRU` 대신 `LSTM` 층을 사용합니다.\n",
    "* 순환 층 위에 용량이 큰 완전 연결된 회귀 층을 사용합니다. 즉 유닛 수가 많은 `Dense` 층이나 `Dense` 층을 스태킹합니다.\n",
    "* 최종적으로 (검증 MAE 기준으로 봤을 때) 최선의 모델을 테스트 세트에서 확인해야 합니다. 이를 잊으면 검증 세트에 과대적합된 네트워크 구조를 만들게 될 것입니다.\n",
    "\n",
    "늘 그렇듯이 딥러닝은 과학보다는 예술에 가깝습니다. 어떤 문제에 적합하거나 그렇지 않은 가이드라인은 제시할 수 있지만 결국 모든 문제는 다릅니다. 경험을 바탕으로 다른 전략들을 시도해 봐야 합니다. 현재는 문제를 해결하는 최선의 방법을 미리 알 수 있는 이론이 없습니다. 반복해서 시도해야 합니다.\n",
    "\n",
    "## 요약\n",
    "\n",
    "다음은 이번 절에서 배운 것들입니다.\n",
    "\n",
    "* 4장에서 처음 배웠던 것처럼 새로운 문제를 해결할 때는 선택한 지표에서 상식 수준의 기준점을 설정하는 것이 좋습니다. 기준점을 가지고 있지 않으면 실제 향상이 되었는지 알 수 없습니다.\n",
    "* 계산 비용을 추가할지 판단하기 위해서 비용이 비싼 모델 전에 간단한 모델을 시도합니다.\n",
    "* 시간 순서가 중요한 데이터가 있다면 순환 층이 적합합니다. 시계열 데이터를 펼쳐서 처리하는 모델의 성능을 쉽게 앞지를 것입니다.\n",
    "* 순환 네트워크에 드롭아웃을 사용하려면 타임스텝 동안 일정한 드롭아웃 마스크와 순환 드롭아웃 마스크를 사용해야 합니다. 둘 다 케라스 순환 층에 포함되어 있습니다. 순환 층에 있는 `dropout`과 `recurrent_dropout` 매개변수를 사용하면 됩니다.\n",
    "* 스태킹 RNN은 단일 RNN 층보다 더 강력한 표현 능력을 제공합니다. 하지만 계산 비용이 많이 들기 때문에 항상 시도할 가치가 있지는 않습니다. (기계 번역 같은) 복잡한 문제에서 확실히 도움이 되지만 작고 간단한 문제에서는 항상 그렇지 않습니다.\n",
    "* 양쪽 방향으로 시퀀스를 바라보는 양방향 RNN은 자연어 처리 문제에 유용합니다. 하지만 최근의 정보가 오래된 것보다 훨씬 의미 있는 시퀀스 데이터에는 잘 작동하지 않습니다.\n",
    "\n",
    "여기서 자세히 다루지 않은 두 가지 중요한 개념이 있습니다. 순환 어텐션과 시퀀스 마스킹입니다. 둘 다 자연어 처리에 깊게 관련되어 있고 온도 예측 문제에는 적합하지 않습니다. 이 책을 끝내고 앞으로 공부할 목록으로 남겨 두겠습니다."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}