02-1. 用CNN圖形辨識（還是MNIST）.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 主題 02-1. 用 CNN 做圖形辨識\n",
    "\n",
    "Yann LeCun 被譽為 Deep Learning 的三巨頭之一。他的 CNN (Convolutional Neural Networks) 是讓 Neural Network 重新受到重視的主因之一。\n",
    "\n",
    "【註】因 TensorFlow 2 已做了一些改變, 例如完全整合了 Keras。到 2021 年的今天, 有一些細節也做了調整。因此我們依新的規範修改了程式。最大的不同是, 以後大家直接安裝 `tensorflow` 即可, 不用再另外裝 `keras`。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. 初始準備\n",
    "\n",
    "基本上和之前是一樣的, 我們就不再說明。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. 讀入 MNIST 數據庫\n",
    "\n",
    "### 2.1 由 tf.Keras 讀入 MNIST\n",
    "\n",
    "基本上和我們上次一樣, 這次因為 tf.Keras 已偷偷把數據庫存在你的電腦, 所以會快很多!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "from tensorflow.keras.datasets import mnist"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "(x_train, y_train), (x_test, y_test) = mnist.load_data()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.2 輸入格式整理\n",
    "\n",
    "如果你還記得, 我們每筆輸入資料都是 28x28 的陣列, CNN 其實就是吃「圖」的, 所以基本上不用像之前把每筆資料拉平。「但。是。」平常的圖都有 R, G, B 三個 channels, 每個 channel 都是一個矩陣, 也就是一張圖可能是三個矩陣! 我們是灰階, 也就是只有一個 channel。但這件事也要明確的告訴 Keras。\n",
    "\n",
    "換句話說, 我們的輸入每筆資料型式要從 (28, 28) 換成 (28, 28, 1)!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "x_train = x_train.reshape(60000, 28, 28, 1)/255\n",
    "x_test = x_test.reshape(10000, 28, 28, 1)/255"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(28, 28, 1)"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x_train[9487].shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "X = x_train[9487][:,:,0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.image.AxesImage at 0x7f3324ed5d90>"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPsAAAD4CAYAAAAq5pAIAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAAMVUlEQVR4nO3df+hddR3H8ddLt8CtlK191WHSKhQcSSsuI1DCiIUTcfZH0cBYKnz7Y4piSFJC/qEgkUVCS1aOptYiKHF/SDVGIEMYXsfa5kb5g5XTsX2Hui0YlO7dH99jfLd977l395xzz93ezwdczr3nfe89b+6+r517z+fc+3FECMD574K2GwAwGoQdSIKwA0kQdiAJwg4kMWeUG1u0aFEsWbJklJsEUtm/f7+OHDni2WqVwm77Rkk/k3ShpF9FxKNl91+yZIm63W6VTQIo0el0etaGfhtv+0JJP5e0UtJSSattLx32+QA0q8pn9uWSXouINyLiP5J+J2lVPW0BqFuVsF8h6c0Ztw8U605he9J213Z3amqqwuYAVFEl7LMdBDjj3NuIWB8RnYjoTExMVNgcgCqqhP2ApCtn3P6EpLertQOgKVXC/pKkq2x/yvZHJH1T0uZ62gJQt6GH3iLifdt3SfqzpofeNkTEK7V1BqBWlcbZI+J5Sc/X1AuABnG6LJAEYQeSIOxAEoQdSIKwA0kQdiAJwg4kQdiBJAg7kARhB5Ig7EAShB1IgrADSRB2IAnCDiRB2IEkCDuQBGEHkiDsQBKEHUiCsANJjHTKZozewoULS+tHjx4tra9du7a0/vjjj591T2gHe3YgCcIOJEHYgSQIO5AEYQeSIOxAEoQdSIJx9vPAe++917MWEaWPtV1aX7duXWmdcfZzR6Ww294v6bikDyS9HxGdOpoCUL869uxfjogjNTwPgAbxmR1IomrYQ9JfbL9se3K2O9ietN213Z2amqq4OQDDqhr26yLiC5JWSlpr+0un3yEi1kdEJyI6ExMTFTcHYFiVwh4RbxfLw5KelbS8jqYA1G/osNueb/tjH16X9FVJe+pqDEC9qhyNv0zSs8U47RxJv42IP9XSFU7Rb6z8iSee6Fk7fvx43e2cYsuWLaX1FStWNLp9DG7osEfEG5I+V2MvABrE0BuQBGEHkiDsQBKEHUiCsANJ8BXXc8CxY8dK6w8++OCIOjnTtm3bSusMvY0P9uxAEoQdSIKwA0kQdiAJwg4kQdiBJAg7kATj7OeAuXPnltaXLl3as7Z3796628E5ij07kARhB5Ig7EAShB1IgrADSRB2IAnCDiTBOPs5YN68eaX122+/vWft/vvvr7udU2zatKm0ft999/WsXXLJJXW3gxLs2YEkCDuQBGEHkiDsQBKEHUiCsANJEHYgCcbZUcnrr79eWj9x4kTPGuPso9V3z257g+3DtvfMWLfQ9hbbrxbLBc22CaCqQd7G/1rSjaete0DS1oi4StLW4jaAMdY37BHxgqR3Tlu9StLG4vpGSbfW2xaAug17gO6yiDgoScXy0l53tD1pu2u7OzU1NeTmAFTV+NH4iFgfEZ2I6ExMTDS9OQA9DBv2Q7YXS1KxPFxfSwCaMGzYN0taU1xfI+m5etoB0JS+4+y2N0m6QdIi2wck/VDSo5J+b/tOSf+S9PUmm0S5u+++u2ft4YcfLn3s0aNH624HY6pv2CNidY/SV2ruBUCDOF0WSIKwA0kQdiAJwg4kQdiBJPiK63lgzpze/4y2R9gJxhl7diAJwg4kQdiBJAg7kARhB5Ig7EAShB1IgrADSRB2IAnCDiRB2IEkCDuQBGEHkiDsQBKEHUiC77Of526++ebS+tNPP13p+U+ePFla3759e8/aqlWrKm0bZ4c9O5AEYQeSIOxAEoQdSIKwA0kQdiAJwg4kwTj7ee6OO+4orT/zzDOVnv+CC8r3F4899ljP2ooVK0ofO2/evKF6wuz67tltb7B92PaeGesesv2W7Z3F5aZm2wRQ1SBv438t6cZZ1v80IpYVl+frbQtA3fqGPSJekPTOCHoB0KAqB+jusr2reJu/oNedbE/a7truTk1NVdgcgCqGDfsvJH1G0jJJByX1PAoTEesjohMRnYmJiSE3B6CqocIeEYci4oOIOCnpl5KW19sWgLoNFXbbi2fc/JqkPb3uC2A89B1nt71J0g2SFtk+IOmHkm6wvUxSSNov6TvNtYhz2YsvvtiztmPHjtLHXn/99XW3k1rfsEfE6llWP9lALwAaxOmyQBKEHUiCsANJEHYgCcIOJEHYgSQIO5AEYQeSIOxAEoQdSIKwA0kQdiAJwg4kwU9Jn+cuv/zy0nq/Xw9q8qfEJicnS+t79+5tbNsZsWcHkiDsQBKEHUiCsANJEHYgCcIOJEHYgSQYZz/PXX311aX1lStXltafeuqpOts5BdOBjRZ7diAJwg4kQdiBJAg7kARhB5Ig7EAShB1IgnH25B555JHSepPj7CdOnCit7969u7R+7bXX1tnOea/vnt32lbb/anuf7Vds31OsX2h7i+1Xi+WC5tsFMKxB3sa/L+m7EXGNpC9KWmt7qaQHJG2NiKskbS1uAxhTfcMeEQcjYkdx/bikfZKukLRK0sbibhsl3dpQjwBqcFYH6GwvkfR5SdslXRYRB6Xp/xAkXdrjMZO2u7a7nAsNtGfgsNv+qKQ/SLo3Io4N+riIWB8RnYjo9PtxQwDNGSjstudqOui/iYg/FqsP2V5c1BdLOtxMiwDq0HfozbYlPSlpX0T8ZEZps6Q1kh4tls810iEaNX/+/NJ6v+GtXbt2Db3tiy66qLR+zTXXDP3cONMg4+zXSfqWpN22dxbrvq/pkP/e9p2S/iXp6410CKAWfcMeEdskuUf5K/W2A6ApnC4LJEHYgSQIO5AEYQeSIOxAEnzFNbmLL764tH7LLbeU1quMs/czZw5/nnVizw4kQdiBJAg7kARhB5Ig7EAShB1IgrADSTCQiVK33XZbaf3dd98tra9bt67OdlABe3YgCcIOJEHYgSQIO5AEYQeSIOxAEoQdSMIRMbKNdTqd6Ha7I9sekE2n01G3253116DZswNJEHYgCcIOJEHYgSQIO5AEYQeSIOxAEn3DbvtK23+1vc/2K7bvKdY/ZPst2zuLy03NtwtgWIP8eMX7kr4bETtsf0zSy7a3FLWfRsSPm2sPQF0GmZ/9oKSDxfXjtvdJuqLpxgDU66w+s9teIunzkrYXq+6yvcv2BtsLejxm0nbXdndqaqpatwCGNnDYbX9U0h8k3RsRxyT9QtJnJC3T9J7/sdkeFxHrI6ITEZ2JiYnqHQMYykBhtz1X00H/TUT8UZIi4lBEfBARJyX9UtLy5toEUNUgR+Mt6UlJ+yLiJzPWL55xt69J2lN/ewDqMsjR+OskfUvSbts7i3Xfl7Ta9jJJIWm/pO800B+AmgxyNH6bpNm+H/t8/e0AaApn0AFJEHYgCcIOJEHYgSQIO5AEYQeSIOxAEoQdSIKwA0kQdiAJwg4kQdiBJAg7kARhB5IY6ZTNtqck/XPGqkWSjoysgbMzrr2Na18SvQ2rzt4+GRGz/v7bSMN+xsbtbkR0WmugxLj2Nq59SfQ2rFH1xtt4IAnCDiTRdtjXt7z9MuPa27j2JdHbsEbSW6uf2QGMTtt7dgAjQtiBJFoJu+0bbf/d9mu2H2ijh15s77e9u5iGuttyLxtsH7a9Z8a6hba32H61WM46x15LvY3FNN4l04y3+tq1Pf35yD+z275Q0j8krZB0QNJLklZHxN6RNtKD7f2SOhHR+gkYtr8k6d+SnoqIzxbrfiTpnYh4tPiPckFEfG9MentI0r/bnsa7mK1o8cxpxiXdKunbavG1K+nrGxrB69bGnn25pNci4o2I+I+k30la1UIfYy8iXpD0zmmrV0naWFzfqOk/lpHr0dtYiIiDEbGjuH5c0ofTjLf62pX0NRJthP0KSW/OuH1A4zXfe0j6i+2XbU+23cwsLouIg9L0H4+kS1vu53R9p/EepdOmGR+b126Y6c+raiPss00lNU7jf9dFxBckrZS0tni7isEMNI33qMwyzfhYGHb686raCPsBSVfOuP0JSW+30MesIuLtYnlY0rMav6moD304g26xPNxyP/83TtN4zzbNuMbgtWtz+vM2wv6SpKtsf8r2RyR9U9LmFvo4g+35xYET2Z4v6asav6moN0taU1xfI+m5Fns5xbhM491rmnG1/Nq1Pv15RIz8IukmTR+Rf13SD9rooUdfn5b0t+LyStu9Sdqk6bd1/9X0O6I7JX1c0lZJrxbLhWPU29OSdkvapelgLW6pt+s1/dFwl6SdxeWmtl+7kr5G8rpxuiyQBGfQAUkQdiAJwg4kQdiBJAg7kARhB5Ig7EAS/wMNkrriz2iDSgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.imshow(X, cmap='Greys')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.3 輸出格式整理\n",
    "\n",
    "和上次一樣, 我們用標準 1-hot 方式處理。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "from tensorflow.keras.utils import to_categorical"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "y_train = to_categorical(y_train, 10)\n",
    "y_test = to_categorical(y_test, 10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. 打造你的 CNN\n",
    "\n",
    "### 3.1 決定神經網路架構、讀入相關套件\n",
    "\n",
    "CNN 我們一樣要決定用幾層的 CNN, 然後是不是每次都要做 max-pooling。再來就是拉平、送入標準神經網路 (再度要決定幾層、幾個神經元)。\n",
    "\n",
    "* 做 <span style=\"color:red;\">3</span> 次 convolution, 每次都接 max-pooling\n",
    "* filter 大小都是 <span style=\"color:red;\">3x3</span>, max-pooling 都用 <span style=\"color:red;\">2x2</span> 為一小區塊\n",
    "\n",
    "CNN 一個小技巧是每層的 filters 數目是越來越多。做完 convolution 之後, 我們要拉平、再送入一個標準的神經網路。這個神經網路設計是這樣:\n",
    "\n",
    "* 只有 <span style=\"color:red;\">1</span> 個隱藏層, 使用 <span style=\"color:red;\">200</span> 個神經元。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "from tensorflow.keras.models import Sequential\n",
    "from tensorflow.keras.layers import Dense, Activation, Flatten\n",
    "from tensorflow.keras.layers import Conv2D, MaxPooling2D\n",
    "from tensorflow.keras.optimizers import SGD"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.2 建構我們的神經網路\n",
    "\n",
    "一開始一樣是打開個空白的神經網路。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "model = Sequential()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.add(Conv2D(32, (3,3), padding='same', input_shape=(28,28,1)))\n",
    "model.add(Activation('relu'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.add(MaxPooling2D(pool_size=(2,2)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.add(Conv2D(64, (3,3), padding='same'))\n",
    "model.add(Activation('relu'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.add(MaxPooling2D(pool_size=(2,2)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.add(Conv2D(128, (3,3), padding='same'))\n",
    "model.add(Activation('relu'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.add(MaxPooling2D(pool_size=(2,2)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.add(Flatten())\n",
    "model.add(Dense(200))\n",
    "model.add(Activation('relu'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.add(Dense(10))\n",
    "model.add(Activation('softmax'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.3 組裝\n",
    "\n",
    "和之前比較不一樣的是我們還要做 `compile` 才正式把我們的神經網路建好。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.compile(loss='mse', optimizer=SGD(learning_rate=0.05), metrics=['accuracy'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.4 檢視我們的神經網路"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"sequential\"\n",
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "conv2d (Conv2D)              (None, 28, 28, 32)        320       \n",
      "_________________________________________________________________\n",
      "activation (Activation)      (None, 28, 28, 32)        0         \n",
      "_________________________________________________________________\n",
      "max_pooling2d (MaxPooling2D) (None, 14, 14, 32)        0         \n",
      "_________________________________________________________________\n",
      "conv2d_1 (Conv2D)            (None, 14, 14, 64)        18496     \n",
      "_________________________________________________________________\n",
      "activation_1 (Activation)    (None, 14, 14, 64)        0         \n",
      "_________________________________________________________________\n",
      "max_pooling2d_1 (MaxPooling2 (None, 7, 7, 64)          0         \n",
      "_________________________________________________________________\n",
      "conv2d_2 (Conv2D)            (None, 7, 7, 128)         73856     \n",
      "_________________________________________________________________\n",
      "activation_2 (Activation)    (None, 7, 7, 128)         0         \n",
      "_________________________________________________________________\n",
      "max_pooling2d_2 (MaxPooling2 (None, 3, 3, 128)         0         \n",
      "_________________________________________________________________\n",
      "flatten (Flatten)            (None, 1152)              0         \n",
      "_________________________________________________________________\n",
      "dense (Dense)                (None, 200)               230600    \n",
      "_________________________________________________________________\n",
      "activation_3 (Activation)    (None, 200)               0         \n",
      "_________________________________________________________________\n",
      "dense_1 (Dense)              (None, 10)                2010      \n",
      "_________________________________________________________________\n",
      "activation_4 (Activation)    (None, 10)                0         \n",
      "=================================================================\n",
      "Total params: 325,282\n",
      "Trainable params: 325,282\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "model.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4. 訓練"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/12\n",
      "600/600 [==============================] - 3s 3ms/step - loss: 0.0899 - accuracy: 0.1019\n",
      "Epoch 2/12\n",
      "600/600 [==============================] - 2s 3ms/step - loss: 0.0892 - accuracy: 0.2558\n",
      "Epoch 3/12\n",
      "600/600 [==============================] - 2s 3ms/step - loss: 0.0882 - accuracy: 0.4014\n",
      "Epoch 4/12\n",
      "600/600 [==============================] - 2s 3ms/step - loss: 0.0842 - accuracy: 0.3984\n",
      "Epoch 5/12\n",
      "600/600 [==============================] - 2s 3ms/step - loss: 0.0548 - accuracy: 0.6564\n",
      "Epoch 6/12\n",
      "600/600 [==============================] - 2s 3ms/step - loss: 0.0230 - accuracy: 0.8650\n",
      "Epoch 7/12\n",
      "600/600 [==============================] - 2s 3ms/step - loss: 0.0158 - accuracy: 0.9026\n",
      "Epoch 8/12\n",
      "600/600 [==============================] - 2s 3ms/step - loss: 0.0127 - accuracy: 0.9189\n",
      "Epoch 9/12\n",
      "600/600 [==============================] - 2s 3ms/step - loss: 0.0109 - accuracy: 0.9304\n",
      "Epoch 10/12\n",
      "600/600 [==============================] - 2s 3ms/step - loss: 0.0091 - accuracy: 0.9426\n",
      "Epoch 11/12\n",
      "600/600 [==============================] - 2s 3ms/step - loss: 0.0083 - accuracy: 0.9476\n",
      "Epoch 12/12\n",
      "600/600 [==============================] - 2s 3ms/step - loss: 0.0075 - accuracy: 0.9528\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<tensorflow.python.keras.callbacks.History at 0x7f331ff9bac0>"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.fit(x_train, y_train, batch_size=100, epochs=12)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "## 5. 結果測試"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 5.1 分數"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "313/313 [==============================] - 1s 1ms/step - loss: 0.0062 - accuracy: 0.9615\n"
     ]
    }
   ],
   "source": [
    "score = model.evaluate(x_test, y_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "測試資料 loss:  0.0061867316253483295\n",
      "測試資料正確率:  0.9614999890327454\n"
     ]
    }
   ],
   "source": [
    "print('測試資料 loss: ', score[0])\n",
    "print('測試資料正確率: ', score[1])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "### 5.2 儲存結果\n",
    "\n",
    "結果看來還不差, 所以我們把結果存起來。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [],
   "source": [
    "model_json = model.to_json()\n",
    "open('handwriting_model_cnn.json', 'w').write(model_json)\n",
    "model.save_weights('handwriting_weights_cnn.h5')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "### 5.3 欣賞一下成果\n",
    "\n",
    "我們用另一個方式: 每次選 5 個顯示, 看是不是有正確辨識。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "y_predict = np.argmax(model.predict(x_test), axis=-1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAV0AAABYCAYAAABWMiSwAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAArJElEQVR4nO2da3Cb15nffy9IgARAgAQI8A7eKd4kkbrYli+xK1vZZmVb3vVmdpJ0J5182Ww7TTbdznRmd9uZZtt+6OxOk6bbtNMku840znqynmxsrWJF3sSWLcuWKIkSJfEiiQBvIHjHncT97QfmnBCiZEmOCIDM+5vRaIgLefDivP/znOd2FFVV0dDQ0NDIDbp8D0BDQ0PjNwlNdDU0NDRyiCa6GhoaGjlEE10NDQ2NHKKJroaGhkYO0URXQ0NDI4dooquhoaGRQwpKdBVF+ZyiKCOKokQVRRlXFOVT+R5TPlEUpVlRlJ8qiuJXFGVOUZS/VhSlON/jKgQURelQFCWmKMoP8j2WfKMoSomiKN9TFGVSUZSwoiiDiqL8dr7HVQgUoqYUjOgqivJp4L8BXwIswNOAO6+Dyj/fBhaAWqAfeAb41/kcUAHxv4CBfA+iQCgGplmfH+XAfwR+pChKcz4HlW8KVVMKyWr6OvAXqqp+9MufvfkcTIHQAvy1qqoxYE5RlJNAb57HlHcURfkcEADOAu35HU3+UVU1CvynDQ/9o6IoHuAAMJGPMRUIBakpBWHpKopSBBwEnIqi3FIUZeaXW2ljvseWZ/4H8DlFUUyKotQDvw2czPOY8oqiKFbgL4B/l++xFCqKolQDu4Dr+R5LvihkTSkI0QWqAT3wWeBTrG+l9wH/IY9jKgROs27ZhoAZ4ALwk3wOqAD4z8D3VFWdzvdAChFFUfTAq8D3VVUdzfd48kjBakqhiO7aL///n6qq+lRVXQL+O3A0j2PKK4qi6ICfAT8GzIADsLHuo/qNRFGUfuAI8I08D6Ug+eWc+X9AAvg3eR5OvilYTSkIn66qqn5FUWYAreXZr7ADLtZ9unEgrijK3wL/Bfj3eR1Z/vhnQDMwpSgKQBlQpChKj6qq+/M4rryjrF+Q77Fu4R1VVTWZ5yHllULWlEKxdAH+FviKoihViqLYgK8B/5jfIeWPX67MHuBfKYpSrChKBfAvgSt5HVh++b9AG+tbxX7g/wAngH+evyEVDP8b6AZeVFV17V4v/g2hIDWlkET3P7OeAnQDGAEGgf+a1xHln5eBzwCLwC0gBfzbvI4oj6iquqqq6pz4B0SAmKqqi/keWz5RFKUJ+DLrC9GcoiiRX/77F/kdWd4pSE1RtCbmGhoaGrmjkCxdDQ0NjR2PJroaGhoaOUQTXQ0NDY0coomuhoaGRg7RRFdDQ0Mjh9yrOOI3JbVBeYDXatfkzmjXZTPaNdnMb/w10SxdDQ0NjRyiia6GhoZGDimI3gsbUVUVVVVJpVKoqko6nUZVVRRFQVEUDAYDRUVF+R6mhobGNiOdTpPJZLI0BUCv1+dUUwpKdDOZDLFYjFgsxvnz55menubDDz/E7/dTW1uLzWbjy1/+Mo2NjfkeqoaGxjYilUrh9Xrx+/2cOnWKubk5ysrKMJvNfOELX6C+vl4adltNQYmuqqokEgnC4TDXrl3D7XZz8uRJFhYW6OjowOVy8YUvfCHfw9TQ0NhGiB3z4uIiU1NTXLx4kZs3b+JwOLDZbLzwwgvU1tbmzNotKNFNJpOMjIzgdrv52c9+xs2bNwkEAlmrTy5WIg0NjZ2BMORCoRB/8zd/w+DgIJOTk6ytreH1ejGZTExPT1NfX4/JZMJgMGz5mApGdDOZDKlUisnJSW7dusXY2Bhe7/qRRkVFRej1ekpKSjTR1dC4A6JxlYiJbHz8Tk2txFZaURR0up0bTxfxoWg0yqVLlxgYWD/LVOjI6uoqgUCAWCxGaWlpTsZUEKIbi8UYHR1lZmaG119/nfHxcVZWVuTzer2eI0eOsHv3biorK/M4Ug2NwiOTyRCPx4nH49y4cYNIJILf7ycajTI8PEw4HAZ+JcyKotDZ2UlDQwMHDhyQW+udGKBOJBIMDg4yOjrK4mJ2B1CTyYTFYsFsNmMwGHJm0BWE6CaTSUZHRxkdHeXs2bMsLCxkrc56vZ7W1lb27t2L0Zj3c+Vyxu0Wy90Qk0XbBTwYt1uHG62/7YAYdzqdllvoy5cvMz8/z/z8PKFQiFOnTrGwsCBfD+vz5MiRI3R1dVFbW4vD4QDYkaKbSqW4du0aN27cYG3tV73dRSZUaWkppaWlFBUV/WaIrpgsKysrnDlzBrfbTSQSkZNDp9NRXV1NbW0t/f39tLW1UVJSks8h54xkMkkgEGB1dZXZ2VlWV1fxer1Eo1Hm5+eJx+NUVlZiNpt59NFHcTqdOBwOTCZTvoe+bfD7/Xg8HjweDwMDA+zdu5fnnnsOs9lMWVlZwYpvKpVibW2NYDDIxYsXWVpa4tatW4TDYUZGRohEIkQiEWKxmLRyIXtRvnbtGj6fD1VVuXr1KocPH6ajoyMfH2dLENa/3+9nYGAgy+KHdW2pq6ujqakJp9OJ0WikuDg3cphX0RVO7mAwyMjICOPj48Tjcfm8TqfD4XDQ0NBAdXU1Vqt1R/ufNpJOp1laWmJlZYUPPviA5eVlbty4wfLyMlevXiUSidDS0kJVVRWqqrJv3z6sVqsmug9AIBDg3LlzfPTRR7z66qu8/PLL9PX1oaoqZrO5YEU3nU4TjUbx+XwcP36cubk5Ll26xOrqKtFolEwmk/X6O30OYQ1nMhlmZ2fp6OjYcaKbSCSIRCKMjo4yNjZGLBaTz+t0OpxOJ1VVVZSVleUkgCbIq+gmEglu3LjByMgI09PTBAKBrAlTUlLC008/TWdnJxUVFeh0uoK9EX5dVFUlk8mwtrbG+Pg4i4uLnDx5kpWVFSYnJ4lGoywsLEjfnaqqLC0tEY1GOX78OBcuXOArX/kKvb29O9Y/97BYW1tjbW2NS5cu8eabb+LxeDYJVaGiqip+v59XX32V6elpzp8/TygUIhQKyeT/B2F1dZX5+fmsrfdOIJlMMjw8zNjYGPPz88RisSwXXVFREX19fezduxer1ZrTseVddIeHhxkdHWV5eTnL/AcwGAzs2bOHffv2YTKZdqzgwq9yCSORCO+//z5ut5vXXnuNYDAoRRayrRZxs7399tvo9XqOHj3Krl27UBRFE92PIR6Ps7i4yNDQEKdOnfpYf3kh4vf7ef3115mdnWV2dvbXWjBisRh+vz/LCtwJJJNJBgcHpdtl4w4aoLi4mI6ODvr6+nIeJ8qL6Aqzf2pqinfffZepqamsL72oqAiHw4HL5aKrq4uGhgb0en0+hrrlpNNp6b8dGBhgYmKC06dPMzc3RyQSkeXQgqKiIlpbWykrK2NhYUH67dLpNAMDA9jtdnp7e6murs7jp3r4xGIxvF4vyeT6yeJGo5G6urpPNC9u3LjBa6+9xuDgIKqqUl5ejsPhoLW1lZqaGoxGY0Eu8CIGEo/HsVgs2Gw2uVjDupEi8k2rqqqyrk0ikSCZTHL27Fl8Pp98vKuriz179tDQ0JDzz7MVCLeC3+/n6tWreDweacWrqopOp6Ompoaamhp6e3tpbGzMeZwoL6KbTCbxer0MDQ3JLXQqlfrVoIqLqa+vp7W1FZfLhcPh2LGWWzqdJhaLMTExwXe/+12mpqYYHR2V4nK7FabX6+nt7cVmszE2Nsbi4iLxeFzmIUajUSorK3ec6MbjcT744AMikQgAdXV1m4TlfhkYGOAb3/iG/NlqtbJnzx7a29ux2WwFO9fS6TTxeJxYLIbZbMZms2XlllosFnp7e6moqKC7uxuLxQKsC1E4HCYSiTA9PS1FV1EUWltbeeKJJ3A6nXn5TA+bdDrN6uoqy8vLXLp0CbfbLV0LQnQbGhpobW2lsbExLymoeRHddDrN8vIyCwsLJJNJuT1SFIXS0lIsFgsdHR00NTVRWlq6I4NnYhJEIhGuXLkit0KBQIB0Ok1RUZH0Y8N6xDoUCqHX62lvb6ehoYFAIEA4HJavCQQCLC0t7bitIkA0GuXDDz8kmUzS09PziX6H1+tlcnKSK1euZD1eXl5OY2MjVVVVD2OoW0ZRURElJSU0NjbypS99ibW1NWnlwrqlW11dTUlJCTabTUbjxTxbXV2V6WE7lWQyicfjYXh4mOXlZSm4YudiMBjYvXs3HR0deUs/zZt7we12Mzc3RyKRkKKr0+koKyvDbrfT0dFBW1sbRqNxx4puJpNhYWGB119/HY/Hg9vt3rR9FlFVEfwxGAz09/fT0dHB8PAwMzMz6HQ6VFVleXkZg8GwI0U3Eonwk5/8BIPBQFtb2wO/X1VVBgYG+MEPfsDQ0FDWcw6Hg46ODmpqagrSrSAoKirCaDRiNBp58cUX7/iaO41fVVU5f+x2+1YPM6/EYjE+/PBDbt68ycrKyqbc3JKSEvr6+ti/f3/eMn1yKrqi1DcUCjE8PMzk5OQmt4LdbqepqYmDBw/S3Nyc01SOXLK2tsbi4iLDw8PcvHkTr9dLOp2mvLycvr4+qqur2bdvH6qqyiqjoqIiSktL5WRJJBIyRUhRFIxGIzabbUf5v2OxGMvLy4yPj5NIJB74s6mqSigUIhgMyiT5YDAIQFlZGTabjZ6eHh577LGCF92NPOg44/E4oVCIRCKR9bjFYqGmpmbb57+LnWM8Hsfn87G4uJilLYqioNfrMZvN1NfXU1VVlbO83NvJ6V8VPimfz8fPf/5z5ufnpWUH66Lb2NhIe3s7jz/+OJWVldvmJnhQVlZWePvtt7l06RJnzpyRN0NNTQ2/93u/R2dnJ4899hjJZJI333yTmZkZysvLKS4upqysTFovwWBQXkObzYbT6dz2N9BGotEo//RP/8S1a9eIx+OYzeYH/h3j4+OcO3eODz74gGvXrkk/ucPhoK+vjyeeeIL9+/fv2JREVVUJBAIyOLsRu91OXV3dJ7quhYTosbC6usqtW7dwu92bRNdoNFJRUUFLSwu1tbV5M05yKrqrq6uMjIxw8eJFafrfHpkvLy/HYrFQXFy8I28AQSwWY2pqiuXlZTk5FEWhrKxMVsno9XpSqZR0xVRVVVFaWsro6CiqqjIzM0M0GiWVSsmgSF9fHxUVFfn9cA+BWCxGIBBgcnKSCxcuMD09nbVA3w/xeFyWmF+6dImpqams+eZyudi3bx/Nzc07VnDhVz7dpaUlVldXs54TO6dCDR7eC2Hhrq2tMT09LXu4LC0tZfm7S0tLefTRR2lqaqKiooLi4uK8uS1zKro+n49vfetbeDwe5ubmslYiWLd0HQ4HTqdzR/pxNxIIBBgcHMxKzC8uLqayspLOzk4qKyvR6/VEIhGZQvbnf/7nOJ1OXnnlFa5du8b09DTRaBRYX7AOHjzIiy++uCMi0YFAgJMnT3LlyhVeeeWVTUGje6GqKsFgkJWVFU6dOsWrr7666f3d3d189rOfLfgA2q+LqqosLCwwOjqK3+/Pes5qtVJeXr5tXVIbYyPf+c53mJqaYmhoaFMxhMVi4dixY+zevRun05mzjmJ3Iieim8lkyGQyrK6usrS0RDAYzEroNhgMOJ1OXC4Xvb29dHZ25s3fkisSiQSzs7OyIMRgMFBRUUF5eTlFRUWyUEIUjYjG7larlampKQKBAICMVJeXl1NfX4/Vat0R1y6ZTOLz+VhZWSEej0vBNBqN7Nq1S1qnd0NVVSYmJhgaGtoUO2hubqazs5O9e/dit9spLS3dsVYurF+LaDRKOBx+4N3CdiCdTktLd2pqKiu3XafTYbFYqK6upq2tjbq6urzfHzn568KXK7aLgUAgaxWyWq0cPnyYtrY2Xn75ZcrLy3eUX/JOhEIhxsbGpC/XbDbT3d1NbW2t3A56vV7Gx8dlet23v/1tFEUhkUigqiplZWWUlZXxzDPP4HK52LNnz46JTou2hG63O2uBrqmp4fDhw5SXl3+s6KbTaX74wx/y3e9+d1M10rFjx/jSl75EbW0tTqdzRwuuYHFxEZ/PJ3dGgkwms+0q8jYigvNLS0ucPn1aplwKDAYD7e3tdHd309fXR1VVVd6/75yIbjKZxO/34/P5WFtbk6IhKCkpobm5mZaWFpmXe/uKLC5UUVGRTJESyc75voifBJ1Oh16vJ5PJkEwmSafThEIhFhYWuHr1Koqi4PF48Pl8cqu0traW1cZRURSKi4spLS2lrKwMvV6/La/FRtbW1vD7/dy6dYvZ2VmWlpZQVZWioiIsFgsOh+OeBwkmEgnZ6nCjyIh0q5qaGpxOJyaTace7scT22+/3y0IagIqKClnVlsu2hg8LoR+xWIyZmRnZunFjfETk/e/atStLW/JNTkTX7/fz7rvvMjAwwNLS0qa0FYfDwdGjR6mpqUGv15NOpwkEAlnCq9fr0el0WK1W+RpVVdHr9XnfLnwSjEYjtbW1+P1+lpaWpPtgZGSEt99+G1gviBAdpe6EXq/HYDDIm2c7XofbGR8fl37/s2fPyuKZiooKDh48SFtb28feOEJg/H4/oVAo6zmXy0VHRwf79u2jpqamIG7ArUaUCY+NjfH++++zurqKoigcOHCAzs5Ouru7C7bs+eMQn8vtdvOXf/mXeDyeTfnper2eqqoqfvd3f5e2tra8+nE3sqV3qegr4Pf7uXnzJvPz85v8LUajEYvFgk6nk+0MU6kUs7Oz8iKKhsPFxcVUVVVJC0VYP9tRbEwmE7t27WJycpLl5WVUVSWZTJJMJrMmz8dt/VKplOxH7PP5WFpaoqKigpKSkm2X35xOp0mn06ysrODxeJiamspq9GOxWNizZw+dnZ13tHJFypA49XViYkKeFFBaWkpJSQk9PT309vZSU1OzLaL1wkoV/9/OxsZGty8gouNYLBYjEokQDoeJxWLynquqqqKhoQGz2bwtFx9xnywtLckd4e0HH9jtdurr66mrq5N9Kj7Opy0MuaKiIpk9tRWL0ZaqVSQSYXZ2lrfffptXXnmFSCSS5W8pKyujp6eHpqYmFhcXmZubk53uL1++nGXhmc1mSkpK6Orqwmaz0dvbS1NTE4888gh1dXVb+TG2hIaGBr74xS9y9uxZxsbGHjjAIZL+w+Ewx48fp6SkhFAoxIEDBzh8+DBNTU1bNPKtQRSLXLx4kQ8//DBLcAF2797Nn/7pn2KxWO7o789kMiwvL+P3+/mrv/orfvGLXxAMBlEUhZaWFpqbm/mjP/ojPvWpT22bSL1YVJPJ5B13O3q9nrKyMoqKirKOmxEtQtfW1rhy5Qput5sbN26QSCSorq7GZrOxb98+nnrqqW17/FUgEODixYu8//77DA0NbZovVquV5557jo6ODrq6ujCZTLKB1J1QVZXFxUWCwSBNTU1UVlZSXFy8JXNlyy3dSCQiU3dudyvA+mqdTCaZnJyUddNerxev15tVwldaWorBYECn08kTE9LpND09PXdse1jo6PV6HA7HXU8oEFa8wWDIqjLLZDJy8qyursoGH9FolJmZGaxWK/39/dTU1FBcXFzwFp2wyILBIBMTE/J7F5ad6DdQUVFBWVnZx24R19bWCAQCeL1e5ufngfU5YbVacTgcVFZWFmSTd+FGSqVSWYtvLBZjdXVVFsHcTklJifRxl5WVodPpKC4ulgtQOBxmbGxM9mMuLi6WjV6am5ux2+3bZgG6nVQqxeLiomzbuNGXC+uWv9gdRyIR2az94/oGe71elpaWKC4uRlVVKioqpL/7YWrLlopuPB5neXmZYDBIIpHYlCeZSCTwer0sLCxw7do1UqkUc3NzxOPxTQItHrtx4wZFRUVMT0/jdDrp7u7G5XJRXFy8rdwMorLsbiJgt9t54YUXcLlcvPTSS7JjVCqVYnBwEJ/Px/Xr11lcXGRgYID5+Xneffddzp07J5Pd6+vrCz6bYXV1lZWVFd544w2++c1vEgqFsrbS5eXl7N27l7a2to+d+JlMhunpaa5fv56Vi6ooCm1tbQVbNCKa0S8sLPDBBx9k9YWIRCIEAgH8fj8LCwubXAwWi4Xm5masViu7du3CbrezZ88eMpkMJ06cwOfzMTAwIK9HdXU1f/zHf8zjjz+O1WqlpKSk4BfluxGPx5mfnycYDN7RBSes/cnJSb75zW+SSCQ4f/68vBa3u1Q2BrR37dpFfX09n//85/nUpz5FcXHxQ3XXbalKCR+K8JXcjvA5ifp6ETS6k/9qY201wPLyMolEguXlZZLJpIzkbwc2Hgu9sdcnrIux2WzG6XTS3NxMW1sbLpdL5pKK99ntdqLRKKWlpbLFozgba25ujtnZWWw2W8GKrsjdjkajLC4uMjU1hcfjueNrdTqdTDm8W2eoeDzO5OQkPp9vU9WV2WzG4XAUlJ9bzAERx5iYmODGjRtcvXpVCkI4HCYYDBIOh/H7/Vn3kIjMB4NB7HY7iqJIf346nZZ+cZ/PRyQSoaqqCqvVSn19vdw6b0fBzWQyMi93eXl5U1mzQNwnqVSKhYUFwuEwo6OjMi/+4xZwUTbtdrvp6+t76Mf55FWlksmkPGpdiOr9dsEPh8Osra1x9epV2tvbaW1t3RaVWEJsJicn+f73v5/VWQzA6XRy7Ngx2tra+P3f/315RLS4QQwGA93d3aTTaTo6OggEAtJ9I4IlZ86cYWVlhS9/+csF25xaWHHnz5/n3Xff5fLly3d8XTAY5Pz58wwPD/P+++/fdTucTCZlU/elpSX5uKIouFwu+vr6cn4sy90QkfeRkRFu3rzJiRMn+OijjwgEAkQiEVmiKoRRVVVKSkpkoFCkG6qqytzcHAsLC0xOTlJcXMzx48eBdZ9nIpEgFotRVFREV1dXlkthO7niNhKNRpmbm+Pdd9/l7//+72UD/9uJRCK89957KIoideVuWUC3s7i4iN/v54033mBqaopjx47x+OOPP7TPkPeDKe9UCiy6aW2cGCKKK3LxhC8wFAqxuLi4bYJpqVRKHhfjdrvxer2y36der8dms9HS0kJHRwd2u11uATdeC3Ej2u12Wc1XUVFBPB6Xea6Tk5N3tQLyifgew+Ewc3NzjI+P43a75THhtyPiAsKC/ySIExei0Sh6vV6m2uULYa35fD6Gh4flWV7CB1lUVITJZJK9pUWTeiGiIjdbnJAgvve7odPpMJlMmM3mvPYceBjEYjFZ6HF7w6yNpNPpLD/4gxSAiDzvqakpysvLZXbRw1qoCmo/LhKZa2pqeOKJJ7DZbMCvzP1IJMLx48cZGxuTj3u9Xi5cuIDL5cLlcuVz+PeF2+3mRz/6ESMjI1y5ckX6risqKujp6WHfvn28/PLLcqt4t8R1RVEwmUwYDAZ+67d+i/Lyck6cOMHAwACKohTsjRUOhwmFQvzd3/0dP/7xj6VVsVU9gNPpNN///vc5efIkFosFk8nEV7/6VZ577rkt+Xv3QgQNg8Egx48f56233mJ5eRmAvr4+uru76erqore3F6fTidPpJBwOy8DY/Pw8paWlVFVVMT4+zve+9z2WlpaYm5v72N4U4ro/SP+KQmRmZoYf/vCHjIyMZB2AsBWsra2xsrLy0OdmQYiuyMMVHbaqq6vp7++XXe6FVbC8vLxpiyj8XbeXehYawn2yuLjI4OAg4+PjMnlfNG9vamrC5XLJhhz3qhQSVovNZqO2tlYeGy7yNwttC6mqKrFYjFAoxM2bNzl37px8ToxZFLtsrDwUu5qNJasi8HE/FszExAQTExPodDp0Oh0vvPBC3kQX1rfIy8vLzMzMMDExITM0xDEy/f398gRsq9VKLBbD6XQSi8Wor6+XQVjRlS8UCt3TRyliJ4lEQrootioPdSuJRCK43W58Pp+cDxut0LvNBzG/RHrdxqwEkf0gdtDi94j2BXdLM/ukFIToOhwOXnjhBerr6/nMZz6DzWajurpapm6IxufT09MP/QLkimg0SjAY5MqVK1y6dIlwOIyqqhiNRiorK3n66af5kz/5EyorKzEajfctmqqqMjY2xtmzZ5mZmQHWreZC7ZG6srLC8PCw9OULqqqqaG5uZs+ePTz55JM4HA4aGhpYXV3F4/Hg9/txu93ypggEArzxxht3TKW6G/X19dTX11NbW/tQP9ODkE6nOXv2LAMDA4yNjaEoCnv37qW9vZ1jx45x6NAhysvLpR9fnHZQWVmJqqpUVVWxvLzM2bNnGRkZkU3ZP86CzWQyjI+Ps7CwwIkTJ1haWqKzsxO73S6FaLsgshLEDlGILnx8cKykpIRHH32U2tpaDh8+jMvlkgHF48ePMzo6ykcffcTk5GRWfGkrLOmCEN3S0lI6Ojpob29n165dmEwm6ewXFo242LdPLrGCFep2WiAyLebn52XVHSDzcGtra2ltbaWkpOSBegmLlKOpqSnpwzWZTA894vqwWFtby+rrKqzbyspKWltb6e7ulq4lh8NBLBajpqaGQCCA1WqVSfBzc3ObiiSE5SLyuW9/rra2lpaWlrwG1FRVZXp6mlu3bskFw+l0UldXR1tbmyyF33i+GSCFUcx1t9vNzMwMwWAwy58rLFjx/lQqJQ+mjMfj3Lx5E4PBQFVVFRaLJauqbbux0aq9l/AWFRVRW1tLQ0MDjzzyCDU1NfIMQo/HQzAYpKysbNPvvdPPvy4FIbomk4muri6Zb5tMJpmYmCAcDku3wqlTp2Sit0Cn09Hc3MzBgwcLvrJmamqKN954g6GhoSxfVENDAy+++CIHDx6kpKTkviPLIn0uHo8zOzuLx+MhEomgKArt7e0Fe8Krw+Fg7969nD17FoCXXnqJ5557jra2NpqammR/V7HwGAwG2aCmoaFBnjzidruzRFccTV9bW8sXv/hFuru7N/1tk8kkG97kCyH+jY2NDA8Ps7CwIH2yIl0ymUySSqXw+/1MT0+zuLjIrVu3iEQirKyssLKywtDQEIFAgHg8LoOwRqNR9ovt6+vDYDDw85//HI/HIzM7fvGLXzA0NMStW7doamrixRdfpK2tbVsYLrC+I3rmmWe4fv06Xq9308G2d0O0DUin07z//vsyKwjg5MmTXL9+PaugRrznXr/3k7Dlons/fiNh7YmVJpVK4fF4mJmZYWxsjOXlZc6cOSOPjgZkSo3NZqOxsTFvJ3veLysrK1y/fj2rabmiKJsKPB5k4ieTSdbW1uQZYBuP7WlpaSnI6iuTyYTD4ZCNizo6Ojhy5AhOp1MGTjciIvmAjOQnEgnC4XCWhSYsmdbWVo4cOVLQZdAWi0UGSmHd9eT3+zeVqS4sLMhjxC9duiRTw0QGDCCDpiUlJVgsFlpaWmT7S4PBgMfjIZFIEAwGicViTE5OyvaqjY2N9Pf309zcXNDB142YzWZ27drF4uKirL67PX/5TojHU6kU4+Pj8vF0Os3Q0BATExOyGdDG92zFNdlS0TWbzbS2ttLQ0EB5eTmxWOyOqS3T09N861vfwmw2Y7VayWQyzMzMEA6H5VZUNO2G9YvR399Pe3s7zz33HL29vQUruiJw4fV68Xg80pfpcDhoaWnh8ccf58knn7xnf9jbyWQyuN1uPB4Pk5OTsu2jaGpeVVVVcNdEHEdUWlrKV77yFX7nd36H5ubmBzqvKh6P8/bbb3Pz5k2Z6A7rXdueffZZ+vv7C7LyTCDKkm02G6WlpdJVEgwG+c53vsOpU6fka0UjI1GRJnY2oqWpqNarqanhkUceoaqqit7eXsxmM5WVleh0Or72ta/h9/t588038Xg8fPDBB/h8Plkw8KMf/QiPx8NTTz1Fe3t7wft4HQ4Hhw8fpqKiQhYwXLhw4Z6+11gsxtmzZ+UCvjHwNjc3x+rq6qb0M7PZTG1t7UN3R22p6Iq8U6fTmXWY4u0EAgH+4R/+4b5/rzgPrLe3l7a2tjtaSIWCyBENBAIsLCzIz2+1Wmlvb6e9vV32SXiQbUwmk2FqaorBwUFmZ2dJJBKyu5jFYqGsrKwgK/RE74Suri66uroe+P3xeJwrV67I/qmC4uJidu/eze7duwu+Ab6wSoXPXaSQ/fSnP73nezf2UbZYLHR0dNDS0sLRo0cpLy/H4XBkfe8VFRWkUimCwSBms5mxsTF8Ph+hUIhQKMSZM2fk+XvNzc0ABS26JpMJk8nE2toaPT09AFy6dOmeftdEIsHExIT8+X7uNZPJhNVqfegni2y56JaXl/PEE0/wZ3/2ZwwNDfHaa6+xtrb2iXLfdDodXV1dNDQ08NRTT7F///6CLXMViK2dOKZINHCvrKyko6ODxsbG+/aniUyOqakpFhYWeOuttxgaGsLn86EoCk8++SRdXV0cOnRoWzczuROxWIyJiQlu3brF4OAg09PTJBIJiouLcblctLS00N7entejte8HnU5He3s7DodDJt+L7/B2hMDW1dWxZ88eGSAtLy+nsbERh8NBb2+vPI5G9Jy+09/s6+ujoaFBFqFMT09nGQLvvPMOJSUldHd3S/EtZCorK3n++edl4H16eprTp08/lOOIqqqqsNvtfOYzn+Ho0aMP/Xpsuejq9XpcLhfHjh3Dbrdz4sQJGQR60KigTqejs7OT1tZW9u/fT0dHR0GmRW1E+B9FDb2goqJClmXe7+kXInfw2rVrDA0N8c4773Djxg1g/dr09PTw7LPP0tTUVJD+3F+HeDzO+fPnGRkZYWxsTLqbjEajPHVEROQLGZ1Oh91up7y8nEcffZR0Os3c3NxdRVen01FXVyfdJnV1ddTX19PZ2ZnVi/pef7OmpkaK9MLCAoFAIOvf+fPnUVUVs9lMU1NTwefvCt+uzWaTTaDOnDnzUETXbreza9cuDh06xP79+x+65Z8Tk0Cv12O1Wunr6+OrX/0qXq+XgYEBGZW9W46hqLoyGo0cOnSI6upqPv3pT8sD5gp1C72R0tJS7HY7DocDm80m/dqLi4sMDw9TXV19z8VHiG0oFCIQCPDee+9x9epV2WOgtLSU0tJSenp66O/vp7y8PBcfLadEo1Heeecd3G531i7JarXyuc99jt27dxdMb4V7IUSyr69PNtgWoipcQyKoXFRURE1NDY2NjRgMBkpLSzEajVit1gcqgBGpYU888QQOhwNFUWSDc3GoI6y7aUKhEJ2dnbS3txes+Ir+E3a7nccee0zGjsQ5jNFoFI/HQygUYnJy8mOLpyoqKjCZTBw4cID6+nq6urrkuWpbUTadE8USbRddLhef//znmZqakvmKk5OTHyu6ZWVl2Gw2nn76afr6+jh48GBBB0pux2AwYLVa5ZZFlLwuLy9z69Yt2tvb7/k7hOj6fD5mZmZ47733GBwcBNavkbB4uru7aWxs3OqPlBcikQhvvfWWTOsRWK1Wjh49WrCNfe6EEMCGhgbq6+uxWCzs3r1btuK0Wq1ZO5WHEUUXv6O9vR2Xy8XIyAjDw8OyY5fP52Nubg6v18vly5f5gz/4g3u208wnorpQVOc1NjayZ88eQqEQly9fxuv1cv78eZmOdzfRFd3ZqqureeaZZzh06JDcgW5VUDGnZqJw/jc2NvLyyy/j9/s5dOjQXavMFEXBbDZjMpno7+/HZrMVfJDkdsSXJj6HKGAIh8PcunWLK1eucPnyZaxWa9bJtKLRSyKRkGeoffjhh8zPz0vhMRqNlJSU8Pzzz9PW1kZ9fX1+PuQWkkqlZBu/jYuzyWRi3759dHd3F8zZVw+K+K5F20nR5OZOxR0P6++JgO3hw4exWCycPn2a9957L6tMOBQKFXxZvUB0ERMCbLFYZEe11tZWwuEwzzzzzCZ35sbFxGazYTab6enpwel0YrFYtjRvOeeiK/rF1tTUoKoqzz///H29t1BX3HshPrPYNgpfZCgU4saNG6TTaRRFobm5mWeffVYGv1ZXV7l58yZLS0tcuHCBxcVFzp07J/s1iAXJZrPx0ksv8dhjjxV8UPGTkEwmmZ+fZ2ZmJkt0LRYLR44c2daiC+vfo81my1kGjsiYOHDgALt37yaZTDI6OsrS0tK2FF3IrkQ0GAwyztPV1fVAcaNcaUxeHaLbVUg/CWazmerqatm5XmQirKysMDo6ysrKijxSBdaj9QsLC0SjUVlfn0gkZCDGbDZz5MgRXC4XXV1dcnXeaYTDYX76058yNjZGPB6Xxxy1t7dz6NAhmpqadlSWRq4QTXYee+wxVldXcbvdjI2N0dbWRltbGwcOHNgR92chfobCjkLtIMxmM3V1dSwtLTE2NiaTuUUTasiu3hMr9MaGHrBuqTQ3N9Pc3Mwf/uEf0t7ejtFoLMg+Cw+D2dlZvv71rxMIBMhkMlgsFvbu3Ut3d7c8dqYQb6xCR8yXgwcP0tPTw7Vr1zh9+rSM2G/HY9m3C5ro5gjhqHc4HDidTlmhFo/HZSrZnbZCJSUlNDc3Y7FYZPvGzs5Oqqurqaqqkm3qdip6vZ6qqioA/H4/ZrOZ3bt309XV9cAFJRqbKS4uxmg04nK5ePrpp2loaMBoNGq7hy1EE90c4XK5qKur46mnnmJ6eppz587xxhtv4PV6iUQid/U9WSwWjh49isvl4tFHH6WiooKamhoMBsO2PljwfjEajezfvx+3283FixdxOBy89NJL1NfX71jrPpcYDAb0ej0tLS20tLQA99cvReOTo4lujhAFEMK329/fTzQaJRAIMD8/f1fRtVqtPP744zgcDmpqamTGwm+KlWexWHjqqafo6uqiu7uburo6amtrH7hXhcbd0UQ2tyj3iO493EaShcuDzLhf65rcqUHyvSKswprd2GouBzfJg/6BLZkr4jyxjScEiGBjnkQ3Z3NlG6Fdk83c9ZpooruONmk2UxCiW4Boc2Uz2jXZzF2vibY/09DQ0MghmuhqaGho5BBNdDU0NDRyiCa6GhoaGjnkXoE0DQ0NDY2HiGbpamhoaOQQTXQ1NDQ0cogmuhoaGho5RBNdDQ0NjRyiia6GhoZGDtFEV0NDQyOH/H8fV0cZgQ4JvQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 5 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "pick = np.random.randint(1,9999, 5)\n",
    "\n",
    "for i in range(5):\n",
    "    plt.subplot(1,5,i+1)\n",
    "    plt.imshow(x_test[pick[i]].reshape(28,28), cmap='Greys')\n",
    "    plt.title(y_predict[pick[i]])\n",
    "    plt.axis(\"off\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5.4 小結論\n",
    "\n",
    "我們到此, 基本上是「亂做」的神經網路。有些同學在不斷試驗的過程中, 可能會發現有時會出現很糟糕的結果。因此, 接下來我們要介紹怎麼樣用些簡單的手法, 能讓學習效果比較穩定, 而且有可能可以增加學習效率。"
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python [conda env:tf2py38]",
   "language": "python",
   "name": "conda-env-tf2py38-py"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}