02-2. 作業說明_調校你的CNN.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 主題 02-2. 作業說明: 調校你的 CNN\n",
    "\n",
    "說實在我們 CNN 不算太好, 我們可以用幾個技巧, 就可以改善穩定度和學習成效。我們建議這次先不要改變結構 (例如用的 filter 個數等)。\n",
    "\n",
    "【註】因 TensorFlow 2 已做了一些改變, 例如完全整合了 Keras。到 2021 年的今天, 有一些細節也做了調整。因此我們依新的規範修改了程式。最大的不同是, 以後大家直接安裝 tensorflow 即可, 不用再另外裝 keras。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. 初始準備\n",
    "\n",
    "基本上和之前是一樣的, 我們就不再說明。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. 讀入 MNIST 數據庫\n",
    "\n",
    "這還是一樣, 然後大家記得可以做 nomarlization!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "from tensorflow.keras.datasets import mnist"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "(x_train, y_train), (x_test, y_test) = mnist.load_data()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "x_train = x_train/255\n",
    "x_test = x_test/255"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "然後的 reshape 等等是一樣的。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "x_train = x_train.reshape(60000, 28, 28, 1)\n",
    "x_test = x_test.reshape(10000, 28, 28, 1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. 改變 loss function\n",
    "\n",
    "大家記得 loss function 就是看我們和目標差多遠。我們最常用的「平均平方差」並不一定是唯一、甚至「最好」的。分類問題我們更常用 \"categorical crossentropy\", 在 Keras 中要使用很方便, 就這樣下指令:\n",
    "\n",
    "    model.compile(loss='categorical_crossentropy', optimizer=SGD(lr=0.05), metrics=['accuracy'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4. 我們來做 Regularization!\n",
    "\n",
    "任何函數的學習, 從迴歸、機器學習到神經網路, 都害怕 \"overfitting\"。所謂 overfitting 大概可以想成同學把考古題做得太熟、差不多都背下來, 但一碰到新的題目就不會!\n",
    "\n",
    "### 4.1 Regularization 的兩大手法\n",
    "\n",
    "為了要避免這件事發生, 我們基本手法大致有兩種, 我們大概選一種用就可以了。\n",
    "\n",
    "#### (1) Dropout\n",
    "\n",
    "Dropout 方式很有趣: 我們設一定的「休息比例」, 也就是每個 batch 讓一些神經元休息, 這樣就不會有「某些神經元拼命背答案」這樣的事發生。實作上其實是設某個比例, 讓有些神經元的輸出不要傳到下一層, 這樣就好像那些神經元 (暫時) 不存在。\n",
    "\n",
    "#### (2) Weights 不要太大\n",
    "\n",
    "這想法很簡單, 如果 weights 太大就「處罰」我們的神經網路! 等等, 怎麼處罰呢? 其實就是把某一層 weight 的大小送進我們的 loss function, 而我們的目標是要讓 loss function 的值最小, 自然達到我們的目的。\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5. 不要忘了還可換 Optimizer"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 6. 恭喜, 你已是神經網路調校大師!\n",
    "\n",
    "我們來總結一下怎麼去調我們的神經網路:\n",
    "\n",
    "1. 調整神經網路的結構\n",
    "2. 把數據做 normalization\n",
    "3. 改變你的 optimizer\n",
    "4. 選用不同的 loss function\n",
    "5. 用 regularization 防止 overfitting\n",
    "\n",
    "千萬千萬不要忘記, 也許你該「重新問你的問題」!\n",
    "\n",
    "我們還有最後還有兩個重點在課程最後會介紹:\n",
    "\n",
    "1. 選用適當的 weights 初始化\n",
    "2. 用 SELU 或是 Bach Normalization 讓過程中都是 normalized 的狀態"
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python [conda env:tf2py38]",
   "language": "python",
   "name": "conda-env-tf2py38-py"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}