CWRU_Dataset.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Prerequisites\n",
    "Understanding in\n",
    "- machine learning and deep learning\n",
    "- python syntax\n",
    "- python libraries: numpy, pandas, Pytorch"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Introduction\n",
    "This notebook shows the study of the [CWRU Bearing Dataset](https://csegroups.case.edu/bearingdatacenter/home), which contains data of normal and fault bearings. Artificial defects of different diameters (0.007 ~ 0.028 Inches) are manufactured at different locations of the bearings: inner raceway(IR), outer raceway(OR) and ball(B) defects. \n",
    "\n",
    "Vibration data was recorded for motor loads of 0 to 3 hp (motor speed of 1797 to 1720 RPM) using accelerometers at the drive end (DE) and fan end (FE) and the data is stored as Matlab files. The sampling rate is 12 kHz and each Matlab file contains between ~120k to ~240k sample points. For more information please refer to the [website](https://csegroups.case.edu/bearingdatacenter/home).\n",
    "\n",
    "This study focuses on the classification of the drive end bearing defects using only the signal data at **DE**. It is a **multiclass classification** problem. The input is the vibration signal data at DE and the output is the type of defects:\n",
    "- 0 : Normal (N), \n",
    "- 1 : Fault at Ball (B),\n",
    "- 2 : Fault at Inner Raceway (IR), \n",
    "- 3 : Fault at Outer Raceway (OR), \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "# Import"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "code_folding": []
   },
   "outputs": [],
   "source": [
    "# Data science libraries\n",
    "import scipy.io\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "# Pytorch\n",
    "import torch\n",
    "from torch import nn\n",
    "from torch.nn import functional as F\n",
    "from torch import Tensor\n",
    "from torch.utils.data import TensorDataset, DataLoader\n",
    "from torch import optim\n",
    "from torch.nn.modules.loss import CrossEntropyLoss\n",
    "\n",
    "# Others\n",
    "from IPython.core.debugger import set_trace\n",
    "from pathlib import Path\n",
    "\n",
    "from helper import get_df_all, download\n",
    "from train_helper import get_dataloader, fit, validate \n",
    "import nn_model\n",
    "from data_urls import URLS"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "working_dir = Path('.')\n",
    "DATA_PATH = Path(\"./Data\")\n",
    "save_model_path = working_dir / 'Model'\n",
    "DE_path = DATA_PATH / '12k_DE'\n",
    "\n",
    "for path in [DATA_PATH, save_model_path]:\n",
    "    if not path.exists():\n",
    "        path.mkdir(parents=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "# # Uncomment this to download the 12k_DE data if needed\n",
    "# for name, url in URLS[\"DE_12k\"].items():\n",
    "#     download(url, DE_path, name, suffix=\".mat\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "#### HYPERPARAMETERS ####\n",
    "bs = 64\n",
    "lr = 0.001\n",
    "wd = 1e-5\n",
    "betas=(0.99, 0.999)\n",
    "device = torch.device(\"cuda\") if torch.cuda.is_available() else torch.device(\"cpu\")\n",
    "random_seed = 42"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Load Data and Preprocessing"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_all = get_df_all(DE_path, segment_length=500, normalize=True)\n",
    "features = df_all.columns[2:]\n",
    "target = 'label'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>label</th>\n",
       "      <th>filename</th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "      <th>3</th>\n",
       "      <th>4</th>\n",
       "      <th>5</th>\n",
       "      <th>6</th>\n",
       "      <th>7</th>\n",
       "      <th>...</th>\n",
       "      <th>490</th>\n",
       "      <th>491</th>\n",
       "      <th>492</th>\n",
       "      <th>493</th>\n",
       "      <th>494</th>\n",
       "      <th>495</th>\n",
       "      <th>496</th>\n",
       "      <th>497</th>\n",
       "      <th>498</th>\n",
       "      <th>499</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>17802</th>\n",
       "      <td>3</td>\n",
       "      <td>OR021@6_3.mat</td>\n",
       "      <td>0.274785</td>\n",
       "      <td>0.018349</td>\n",
       "      <td>0.003093</td>\n",
       "      <td>0.212311</td>\n",
       "      <td>0.105523</td>\n",
       "      <td>-0.129121</td>\n",
       "      <td>-0.008530</td>\n",
       "      <td>0.181073</td>\n",
       "      <td>...</td>\n",
       "      <td>0.256624</td>\n",
       "      <td>0.516693</td>\n",
       "      <td>0.115693</td>\n",
       "      <td>-0.108780</td>\n",
       "      <td>0.142572</td>\n",
       "      <td>0.032151</td>\n",
       "      <td>-0.170528</td>\n",
       "      <td>-0.307827</td>\n",
       "      <td>-0.158905</td>\n",
       "      <td>0.005999</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14772</th>\n",
       "      <td>3</td>\n",
       "      <td>OR014@6_2.mat</td>\n",
       "      <td>-0.675053</td>\n",
       "      <td>-0.069198</td>\n",
       "      <td>1.270061</td>\n",
       "      <td>-0.380097</td>\n",
       "      <td>-0.266814</td>\n",
       "      <td>-0.426249</td>\n",
       "      <td>0.396941</td>\n",
       "      <td>-0.063743</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.448906</td>\n",
       "      <td>-1.651386</td>\n",
       "      <td>-0.475758</td>\n",
       "      <td>-0.178705</td>\n",
       "      <td>-0.233668</td>\n",
       "      <td>0.980140</td>\n",
       "      <td>0.290371</td>\n",
       "      <td>-0.067520</td>\n",
       "      <td>0.467429</td>\n",
       "      <td>0.532042</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2967</th>\n",
       "      <td>1</td>\n",
       "      <td>B028_0.mat</td>\n",
       "      <td>1.755181</td>\n",
       "      <td>1.672121</td>\n",
       "      <td>-1.291409</td>\n",
       "      <td>-1.508854</td>\n",
       "      <td>0.660901</td>\n",
       "      <td>1.297566</td>\n",
       "      <td>-0.496850</td>\n",
       "      <td>-0.891779</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.115635</td>\n",
       "      <td>0.524556</td>\n",
       "      <td>0.134330</td>\n",
       "      <td>-0.456887</td>\n",
       "      <td>-0.157165</td>\n",
       "      <td>0.446198</td>\n",
       "      <td>-0.090952</td>\n",
       "      <td>-0.251196</td>\n",
       "      <td>0.450507</td>\n",
       "      <td>0.045001</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11473</th>\n",
       "      <td>3</td>\n",
       "      <td>OR007@12_1.mat</td>\n",
       "      <td>-0.472892</td>\n",
       "      <td>0.070833</td>\n",
       "      <td>0.700586</td>\n",
       "      <td>-0.234298</td>\n",
       "      <td>-0.696699</td>\n",
       "      <td>0.232136</td>\n",
       "      <td>0.534579</td>\n",
       "      <td>-0.238331</td>\n",
       "      <td>...</td>\n",
       "      <td>0.328245</td>\n",
       "      <td>0.084275</td>\n",
       "      <td>-0.606639</td>\n",
       "      <td>-0.120714</td>\n",
       "      <td>0.437797</td>\n",
       "      <td>-0.264542</td>\n",
       "      <td>-0.437271</td>\n",
       "      <td>0.423011</td>\n",
       "      <td>0.570872</td>\n",
       "      <td>-0.134156</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2023</th>\n",
       "      <td>1</td>\n",
       "      <td>B021_0.mat</td>\n",
       "      <td>0.064694</td>\n",
       "      <td>0.384914</td>\n",
       "      <td>0.501358</td>\n",
       "      <td>-0.678848</td>\n",
       "      <td>-0.803782</td>\n",
       "      <td>0.706347</td>\n",
       "      <td>0.881013</td>\n",
       "      <td>-0.135444</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.632755</td>\n",
       "      <td>-0.499330</td>\n",
       "      <td>0.819152</td>\n",
       "      <td>1.008373</td>\n",
       "      <td>0.696644</td>\n",
       "      <td>0.024666</td>\n",
       "      <td>-0.243397</td>\n",
       "      <td>0.159304</td>\n",
       "      <td>0.255128</td>\n",
       "      <td>-0.009296</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 502 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "       label        filename         0         1         2         3  \\\n",
       "17802      3   OR021@6_3.mat  0.274785  0.018349  0.003093  0.212311   \n",
       "14772      3   OR014@6_2.mat -0.675053 -0.069198  1.270061 -0.380097   \n",
       "2967       1      B028_0.mat  1.755181  1.672121 -1.291409 -1.508854   \n",
       "11473      3  OR007@12_1.mat -0.472892  0.070833  0.700586 -0.234298   \n",
       "2023       1      B021_0.mat  0.064694  0.384914  0.501358 -0.678848   \n",
       "\n",
       "              4         5         6         7  ...       490       491  \\\n",
       "17802  0.105523 -0.129121 -0.008530  0.181073  ...  0.256624  0.516693   \n",
       "14772 -0.266814 -0.426249  0.396941 -0.063743  ... -0.448906 -1.651386   \n",
       "2967   0.660901  1.297566 -0.496850 -0.891779  ... -0.115635  0.524556   \n",
       "11473 -0.696699  0.232136  0.534579 -0.238331  ...  0.328245  0.084275   \n",
       "2023  -0.803782  0.706347  0.881013 -0.135444  ... -0.632755 -0.499330   \n",
       "\n",
       "            492       493       494       495       496       497       498  \\\n",
       "17802  0.115693 -0.108780  0.142572  0.032151 -0.170528 -0.307827 -0.158905   \n",
       "14772 -0.475758 -0.178705 -0.233668  0.980140  0.290371 -0.067520  0.467429   \n",
       "2967   0.134330 -0.456887 -0.157165  0.446198 -0.090952 -0.251196  0.450507   \n",
       "11473 -0.606639 -0.120714  0.437797 -0.264542 -0.437271  0.423011  0.570872   \n",
       "2023   0.819152  1.008373  0.696644  0.024666 -0.243397  0.159304  0.255128   \n",
       "\n",
       "            499  \n",
       "17802  0.005999  \n",
       "14772  0.532042  \n",
       "2967   0.045001  \n",
       "11473 -0.134156  \n",
       "2023  -0.009296  \n",
       "\n",
       "[5 rows x 502 columns]"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_all.sample(5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(17987, 502)"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_all.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "## Split the data into train and validation set\n",
    "X_train, X_valid, y_train, y_valid = train_test_split(df_all[features], \n",
    "                                                      df_all[target], \n",
    "                                                      test_size=0.20, random_state=random_seed, shuffle=True\n",
    "                                                     )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "## Create DataLoader of train and validation set\n",
    "X_train = torch.tensor(X_train.values, dtype=torch.float32)\n",
    "X_valid = torch.tensor(X_valid.values, dtype=torch.float32)\n",
    "y_train = torch.tensor(y_train.values, dtype=torch.long)\n",
    "y_valid = torch.tensor(y_valid.values, dtype=torch.long)\n",
    "\n",
    "train_ds = TensorDataset(X_train, y_train)\n",
    "valid_ds = TensorDataset(X_valid, y_valid)\n",
    "train_dl, valid_dl = get_dataloader(train_ds, valid_ds, bs)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Training with Adams Optimizer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "## Instantiate model, optimizer and loss function\n",
    "model = nn_model.CNN_1D_2L(len(features))\n",
    "model.to(device)\n",
    "opt = optim.Adam(model.parameters(), lr=lr, betas=betas, weight_decay=wd)\n",
    "loss_func = CrossEntropyLoss()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "EPOCH \t Train Loss \t Val Loss \t Train Acc \t Val Acc \t\n",
      "0 \t 0.00685 \t 0.37846 \t 0.00000 \t0.85937 \t\n",
      "1 \t 0.00220 \t 0.21542 \t 0.00000 \t0.91579 \t\n",
      "2 \t 0.00130 \t 0.12809 \t 0.00000 \t0.94803 \t\n",
      "3 \t 0.00093 \t 0.09389 \t 0.00000 \t0.96304 \t\n",
      "4 \t 0.00073 \t 0.10653 \t 0.00000 \t0.95414 \t\n",
      "5 \t 0.00055 \t 0.10663 \t 0.00000 \t0.95942 \t\n",
      "6 \t 0.00062 \t 0.06465 \t 0.00000 \t0.97360 \t\n",
      "7 \t 0.00036 \t 0.03604 \t 0.00000 \t0.98833 \t\n",
      "8 \t 0.00053 \t 0.02805 \t 0.00000 \t0.99138 \t\n",
      "9 \t 0.00040 \t 0.04993 \t 0.00000 \t0.98193 \t\n",
      "10 \t 0.00033 \t 0.08187 \t 0.00000 \t0.96665 \t\n",
      "11 \t 0.00035 \t 0.05626 \t 0.00000 \t0.98193 \t\n",
      "12 \t 0.00032 \t 0.12104 \t 0.00000 \t0.95664 \t\n",
      "13 \t 0.00045 \t 0.02969 \t 0.00000 \t0.98916 \t\n",
      "14 \t 0.00042 \t 0.05943 \t 0.00000 \t0.98277 \t\n",
      "15 \t 0.00027 \t 0.11805 \t 0.00000 \t0.95525 \t\n",
      "16 \t 0.00027 \t 0.04056 \t 0.00000 \t0.98749 \t\n",
      "17 \t 0.00026 \t 0.13035 \t 0.00000 \t0.95914 \t\n",
      "18 \t 0.00019 \t 0.03276 \t 0.00000 \t0.98777 \t\n",
      "19 \t 0.00028 \t 0.13098 \t 0.00000 \t0.95831 \t\n",
      "Wall time: 24min 10s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "## Train\n",
    "epochs = 20\n",
    "model, metrics = fit(epochs, model, loss_func, opt, train_dl, valid_dl, train_metric=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "EPOCH \t Train Loss \t Val Loss \t Train Acc \t Val Acc \t\n",
      "0 \t 0.00020 \t 0.04433 \t 0.00000 \t0.98527 \t\n",
      "1 \t 0.00017 \t 0.08623 \t 0.00000 \t0.96915 \t\n",
      "2 \t 0.00017 \t 0.02013 \t 0.00000 \t0.99277 \t\n",
      "3 \t 0.00021 \t 0.03686 \t 0.00000 \t0.98666 \t\n",
      "4 \t 0.00021 \t 0.01674 \t 0.00000 \t0.99389 \t\n",
      "5 \t 0.00020 \t 0.04552 \t 0.00000 \t0.98082 \t\n",
      "6 \t 0.00025 \t 0.04389 \t 0.00000 \t0.98694 \t\n",
      "7 \t 0.00016 \t 0.04647 \t 0.00000 \t0.98138 \t\n",
      "8 \t 0.00012 \t 0.06574 \t 0.00000 \t0.97415 \t\n",
      "9 \t 0.00007 \t 0.05641 \t 0.00000 \t0.98027 \t\n",
      "10 \t 0.00010 \t 0.03147 \t 0.00000 \t0.98722 \t\n",
      "11 \t 0.00010 \t 0.03779 \t 0.00000 \t0.98555 \t\n",
      "12 \t 0.00010 \t 0.01386 \t 0.00000 \t0.99500 \t\n",
      "13 \t 0.00009 \t 0.01199 \t 0.00000 \t0.99611 \t\n",
      "14 \t 0.00013 \t 0.02183 \t 0.00000 \t0.99361 \t\n",
      "15 \t 0.00022 \t 0.01676 \t 0.00000 \t0.99277 \t\n",
      "16 \t 0.00025 \t 0.01729 \t 0.00000 \t0.99555 \t\n",
      "17 \t 0.00016 \t 0.01913 \t 0.00000 \t0.99444 \t\n",
      "18 \t 0.00008 \t 0.01996 \t 0.00000 \t0.99305 \t\n",
      "19 \t 0.00008 \t 0.02320 \t 0.00000 \t0.99277 \t\n",
      "Wall time: 24min 19s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "## Train\n",
    "epochs = 20\n",
    "model, metrics = fit(epochs, model, loss_func, opt, train_dl, valid_dl, train_metric=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Save trained model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "torch.save(model.state_dict(), save_model_path / 'model.pth')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "model2 = nn_model.CNN_1D_2L(len(features))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "CNN_1D_2L(\n",
       "  (layer1): Sequential(\n",
       "    (0): Conv1d(1, 64, kernel_size=(9,), stride=(1,), padding=(4,))\n",
       "    (1): BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    (2): ReLU()\n",
       "    (3): Dropout(p=0.5)\n",
       "    (4): MaxPool1d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)\n",
       "  )\n",
       "  (layer2): Sequential(\n",
       "    (0): Conv1d(64, 128, kernel_size=(5,), stride=(1,), padding=(2,))\n",
       "    (1): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    (2): ReLU()\n",
       "    (3): Dropout(p=0.5)\n",
       "    (4): AvgPool1d(kernel_size=(2,), stride=(2,), padding=(0,))\n",
       "  )\n",
       "  (linear1): Linear(in_features=16000, out_features=4, bias=True)\n",
       ")"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model2.load_state_dict(torch.load(save_model_path / 'model.pth'))\n",
    "model2.eval()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(0.012104931890057517, 0.9952751528627015)\n",
      "Wall time: 92.8 ms\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "print(validate(model, valid_dl, loss_func))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}