Tobigs_Week4_Dimensonality_Reduction_Assignment1.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "zU92BX-5Pgsm"
   },
   "source": [
    "# 차원축소 Assignment1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "b15X2e8HPgsn"
   },
   "source": [
    "# ''' ? ''' 이 있는 부분을 채워주세요 \n",
    "\n",
    "참고하셔야 하는 함수들은 링크 달아드렸으니 들어가서 확인해보세요\n",
    "\n",
    "무작정 빈칸 채우기가 아니라 꼭 차원축소 과정을 이해하시면서 과제를 진행하시면 좋겠습니다!!!\n",
    "\n",
    "또 질문은 언제나 환영이니 과제하다가 궁금하신것 있으시면 자유롭게 물어보세요!!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "VHM2ow0gPgsn"
   },
   "source": [
    "# 1) PCA의 과정 차근차근 밟아보기"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "metadata": {
    "executionInfo": {
     "elapsed": 901,
     "status": "ok",
     "timestamp": 1629794503787,
     "user": {
      "displayName": "김건우",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14GifAuBZsXu6RawBPfBoYmIJaUQ0zRxVy1udxOxf=s64",
      "userId": "13140730798096026130"
     },
     "user_tz": -540
    },
    "id": "tpoi2PmKPgsn"
   },
   "outputs": [],
   "source": [
    "#기본 모듈 불러오기\n",
    "import numpy as np\n",
    "import numpy.linalg as lin\n",
    "import matplotlib.pyplot as plt\n",
    "import pandas as pd\n",
    "import random"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "metadata": {
    "executionInfo": {
     "elapsed": 16,
     "status": "ok",
     "timestamp": 1629794504693,
     "user": {
      "displayName": "김건우",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14GifAuBZsXu6RawBPfBoYmIJaUQ0zRxVy1udxOxf=s64",
      "userId": "13140730798096026130"
     },
     "user_tz": -540
    },
    "id": "ZcFtAQxoPgsn"
   },
   "outputs": [],
   "source": [
    "#설명변수 x1, x2, x3\n",
    "x1 = [95, 91, 66, 94, 68, 63, 12, 73, 93, 51, 13, 70, 63, 63, 97, 56, 67, 96, 75, 6]\n",
    "x2 = [56, 27, 25, 1, 9, 80, 92, 69, 6, 25, 83, 82, 54, 97, 66, 93, 76, 59, 94, 9]\n",
    "x3 = [57, 34, 9, 79, 4, 77, 100, 42, 6, 96, 61, 66, 9, 25, 84, 46, 16, 63, 53, 30]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "metadata": {
    "executionInfo": {
     "elapsed": 18,
     "status": "ok",
     "timestamp": 1629794504695,
     "user": {
      "displayName": "김건우",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14GifAuBZsXu6RawBPfBoYmIJaUQ0zRxVy1udxOxf=s64",
      "userId": "13140730798096026130"
     },
     "user_tz": -540
    },
    "id": "QB3ElSfuPgso"
   },
   "outputs": [],
   "source": [
    "#   설명변수들을 하나의 행렬로\n",
    "X = np.stack((x1,x2,x3),axis=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "metadata": {
    "executionInfo": {
     "elapsed": 18,
     "status": "ok",
     "timestamp": 1629794504695,
     "user": {
      "displayName": "김건우",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14GifAuBZsXu6RawBPfBoYmIJaUQ0zRxVy1udxOxf=s64",
      "userId": "13140730798096026130"
     },
     "user_tz": -540
    },
    "id": "WRc6gLv2Pgso"
   },
   "outputs": [],
   "source": [
    "X = pd.DataFrame(X.T,columns=['x1','x2','x3'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 665
    },
    "executionInfo": {
     "elapsed": 18,
     "status": "ok",
     "timestamp": 1629794504696,
     "user": {
      "displayName": "김건우",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14GifAuBZsXu6RawBPfBoYmIJaUQ0zRxVy1udxOxf=s64",
      "userId": "13140730798096026130"
     },
     "user_tz": -540
    },
    "id": "6YYdgIxXPgso",
    "outputId": "c21dd528-bba9-4d43-b09b-db339f267be0"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>x1</th>\n",
       "      <th>x2</th>\n",
       "      <th>x3</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>95</td>\n",
       "      <td>56</td>\n",
       "      <td>57</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>91</td>\n",
       "      <td>27</td>\n",
       "      <td>34</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>66</td>\n",
       "      <td>25</td>\n",
       "      <td>9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>94</td>\n",
       "      <td>1</td>\n",
       "      <td>79</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>68</td>\n",
       "      <td>9</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>63</td>\n",
       "      <td>80</td>\n",
       "      <td>77</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>12</td>\n",
       "      <td>92</td>\n",
       "      <td>100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>73</td>\n",
       "      <td>69</td>\n",
       "      <td>42</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>93</td>\n",
       "      <td>6</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>51</td>\n",
       "      <td>25</td>\n",
       "      <td>96</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>13</td>\n",
       "      <td>83</td>\n",
       "      <td>61</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>70</td>\n",
       "      <td>82</td>\n",
       "      <td>66</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>63</td>\n",
       "      <td>54</td>\n",
       "      <td>9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>63</td>\n",
       "      <td>97</td>\n",
       "      <td>25</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>97</td>\n",
       "      <td>66</td>\n",
       "      <td>84</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>56</td>\n",
       "      <td>93</td>\n",
       "      <td>46</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>67</td>\n",
       "      <td>76</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>96</td>\n",
       "      <td>59</td>\n",
       "      <td>63</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>75</td>\n",
       "      <td>94</td>\n",
       "      <td>53</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>6</td>\n",
       "      <td>9</td>\n",
       "      <td>30</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    x1  x2   x3\n",
       "0   95  56   57\n",
       "1   91  27   34\n",
       "2   66  25    9\n",
       "3   94   1   79\n",
       "4   68   9    4\n",
       "5   63  80   77\n",
       "6   12  92  100\n",
       "7   73  69   42\n",
       "8   93   6    6\n",
       "9   51  25   96\n",
       "10  13  83   61\n",
       "11  70  82   66\n",
       "12  63  54    9\n",
       "13  63  97   25\n",
       "14  97  66   84\n",
       "15  56  93   46\n",
       "16  67  76   16\n",
       "17  96  59   63\n",
       "18  75  94   53\n",
       "19   6   9   30"
      ]
     },
     "execution_count": 77,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "NTRZsZjkPgso"
   },
   "source": [
    "### 1-1) 먼저 PCA를 시작하기 전 데이터 scaling!!!! 매우 중요합니다!\n",
    "\n",
    "https://datascienceschool.net/03%20machine%20learning/02.01%20%EB%8D%B0%EC%9D%B4%ED%84%B0%20%EC%A0%84%EC%B2%98%EB%A6%AC%20%EA%B8%B0%EC%B4%88.html?highlight=%EC%A0%84%EC%B2%98%EB%A6%AC  를 참고하시면 도움이 될거에요"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "executionInfo": {
     "elapsed": 255,
     "status": "ok",
     "timestamp": 1629794771272,
     "user": {
      "displayName": "김건우",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14GifAuBZsXu6RawBPfBoYmIJaUQ0zRxVy1udxOxf=s64",
      "userId": "13140730798096026130"
     },
     "user_tz": -540
    },
    "id": "urxrwxdJvBre",
    "outputId": "fc424ba8-1f5e-47a6-b8d8-a5c51ba8a3f5"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 1.08573604,  0.02614175,  0.30684189],\n",
       "       [ 0.93801686, -0.86575334, -0.46445467],\n",
       "       [ 0.01477192, -0.92726334, -1.30282049],\n",
       "       [ 1.04880625, -1.66538341,  1.04460382],\n",
       "       [ 0.08863151, -1.41934339, -1.47049366],\n",
       "       [-0.09601747,  0.76426183,  0.97753455],\n",
       "       [-1.97943714,  1.13332186,  1.74883111],\n",
       "       [ 0.2732805 ,  0.42595679, -0.1961776 ],\n",
       "       [ 1.01187645, -1.5116084 , -1.40342439],\n",
       "       [-0.53917504, -0.92726334,  1.61469258],\n",
       "       [-1.94250735,  0.85652683,  0.44098042],\n",
       "       [ 0.16249111,  0.82577183,  0.60865359],\n",
       "       [-0.09601747, -0.03536825, -1.30282049],\n",
       "       [-0.09601747,  1.28709688, -0.76626636],\n",
       "       [ 1.15959564,  0.33369178,  1.21227698],\n",
       "       [-0.35452606,  1.16407687, -0.06203907],\n",
       "       [ 0.05170172,  0.64124181, -1.06807806],\n",
       "       [ 1.12266584,  0.11840676,  0.50804969],\n",
       "       [ 0.3471401 ,  1.19483187,  0.17270336],\n",
       "       [-2.20101593, -1.41934339, -0.5985932 ]])"
      ]
     },
     "execution_count": 78,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#scaling\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "\n",
    "scaler = StandardScaler()\n",
    "X_std = scaler.fit_transform(X)\n",
    "\n",
    "X_std\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "metadata": {
    "executionInfo": {
     "elapsed": 13,
     "status": "ok",
     "timestamp": 1629794504696,
     "user": {
      "displayName": "김건우",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14GifAuBZsXu6RawBPfBoYmIJaUQ0zRxVy1udxOxf=s64",
      "userId": "13140730798096026130"
     },
     "user_tz": -540
    },
    "id": "TjuIDQNMPgsp"
   },
   "outputs": [],
   "source": [
    "features = X_std.T"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 80,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "executionInfo": {
     "elapsed": 13,
     "status": "ok",
     "timestamp": 1629794504697,
     "user": {
      "displayName": "김건우",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14GifAuBZsXu6RawBPfBoYmIJaUQ0zRxVy1udxOxf=s64",
      "userId": "13140730798096026130"
     },
     "user_tz": -540
    },
    "id": "9Z5Hpvl3vnfp",
    "outputId": "1fc37c2f-6b20-46c6-aad9-9ebd48805658"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 1.08573604,  0.93801686,  0.01477192,  1.04880625,  0.08863151,\n",
       "        -0.09601747, -1.97943714,  0.2732805 ,  1.01187645, -0.53917504,\n",
       "        -1.94250735,  0.16249111, -0.09601747, -0.09601747,  1.15959564,\n",
       "        -0.35452606,  0.05170172,  1.12266584,  0.3471401 , -2.20101593],\n",
       "       [ 0.02614175, -0.86575334, -0.92726334, -1.66538341, -1.41934339,\n",
       "         0.76426183,  1.13332186,  0.42595679, -1.5116084 , -0.92726334,\n",
       "         0.85652683,  0.82577183, -0.03536825,  1.28709688,  0.33369178,\n",
       "         1.16407687,  0.64124181,  0.11840676,  1.19483187, -1.41934339],\n",
       "       [ 0.30684189, -0.46445467, -1.30282049,  1.04460382, -1.47049366,\n",
       "         0.97753455,  1.74883111, -0.1961776 , -1.40342439,  1.61469258,\n",
       "         0.44098042,  0.60865359, -1.30282049, -0.76626636,  1.21227698,\n",
       "        -0.06203907, -1.06807806,  0.50804969,  0.17270336, -0.5985932 ]])"
      ]
     },
     "execution_count": 80,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "features"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "3HHTB65iPgsp"
   },
   "source": [
    "### 1-2) 다음으로 공분산 행렬을 구합니다\n",
    "\n",
    "https://docs.scipy.org/doc/numpy/reference/generated/numpy.cov.html 를 참고하시면 도움이 될거에요"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 81,
   "metadata": {
    "executionInfo": {
     "elapsed": 11,
     "status": "ok",
     "timestamp": 1629794504697,
     "user": {
      "displayName": "김건우",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14GifAuBZsXu6RawBPfBoYmIJaUQ0zRxVy1udxOxf=s64",
      "userId": "13140730798096026130"
     },
     "user_tz": -540
    },
    "id": "lZ2KT6M3Pgsq",
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 1.05263158, -0.2037104 , -0.12079228],\n",
       "       [-0.2037104 ,  1.05263158,  0.3125801 ],\n",
       "       [-0.12079228,  0.3125801 ,  1.05263158]])"
      ]
     },
     "execution_count": 81,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cov_matrix = np.cov(features, bias=False)\n",
    "\n",
    "cov_matrix"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "LDPacia_Pgsq"
   },
   "source": [
    "### 1-3) 이제 고유값과 고유벡터를 구합니다\n",
    "\n",
    "실습코드를 참고해주세요"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 82,
   "metadata": {
    "executionInfo": {
     "elapsed": 3,
     "status": "ok",
     "timestamp": 1629794506645,
     "user": {
      "displayName": "김건우",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14GifAuBZsXu6RawBPfBoYmIJaUQ0zRxVy1udxOxf=s64",
      "userId": "13140730798096026130"
     },
     "user_tz": -540
    },
    "id": "u850XjOtv3b9"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Eigen Value:  [1.48756162 0.94435407 0.72597904]\n",
      "Eigen Vector:  [[ 0.47018528 -0.85137353 -0.23257022]\n",
      " [-0.64960236 -0.15545725 -0.74421087]\n",
      " [-0.59744671 -0.50099516  0.62614797]]\n"
     ]
    }
   ],
   "source": [
    "import numpy.linalg as lin\n",
    "\n",
    "X_eigenvalues, X_eigenvectors = np.linalg.eig(cov_matrix)\n",
    "\n",
    "print(\"Eigen Value: \", X_eigenvalues)\n",
    "print(\"Eigen Vector: \", X_eigenvectors)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "--C9kkKDPgsr"
   },
   "source": [
    "### 1-4) 고유값 분해: 고유값 분해의 곱으로 원래 공분산 행렬을 구해봅니다\n",
    "\n",
    "https://docs.scipy.org/doc/numpy/reference/generated/numpy.dot.html 를 참고해주세요\n",
    "\n",
    "A = 𝑃Λ𝑃`\n",
    "이 식 기억나죠?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 83,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[1.48756162, 0.        , 0.        ],\n",
       "       [0.        , 0.94435407, 0.        ],\n",
       "       [0.        , 0.        , 0.72597904]])"
      ]
     },
     "execution_count": 83,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 고유값으로 대칭행렬 만들기\n",
    "sym_matrix = np.zeros((3,3))\n",
    "\n",
    "sym_matrix[0][0] = X_eigenvalues[0]\n",
    "sym_matrix[1][1] = X_eigenvalues[1]\n",
    "sym_matrix[2][2] = X_eigenvalues[2]\n",
    "\n",
    "sym_matrix"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 84,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 1.05263158, -0.2037104 , -0.12079228],\n",
       "       [-0.2037104 ,  1.05263158,  0.3125801 ],\n",
       "       [-0.12079228,  0.3125801 ,  1.05263158]])"
      ]
     },
     "execution_count": 84,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.dot(np.dot(X_eigenvectors, sym_matrix),X_eigenvectors.T)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`1-2 부분에서 공분산행렬을 구한 것과 행렬곱으로 구한 것과 같은 것을 확인할 수 있다.`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "N4t2RjoNPgss"
   },
   "source": [
    "### 1-5) 마지막으로 고유 벡터 축으로 값을 변환합니다\n",
    "\n",
    "\n",
    "https://docs.scipy.org/doc/numpy/reference/generated/numpy.concatenate.html"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 85,
   "metadata": {
    "executionInfo": {
     "elapsed": 2,
     "status": "ok",
     "timestamp": 1629794510015,
     "user": {
      "displayName": "김건우",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14GifAuBZsXu6RawBPfBoYmIJaUQ0zRxVy1udxOxf=s64",
      "userId": "13140730798096026130"
     },
     "user_tz": -540
    },
    "id": "HkhBLg0jPgss"
   },
   "outputs": [],
   "source": [
    "# X, eigenvector 행렬 곱을 통해 데이터를 투영시킵니다.\n",
    "# 모든 고유 벡터 축으로 데이터를 투영한 값\n",
    "\n",
    "def new_coordinates(X,X_eigenvectors):\n",
    "    for i in range(X_eigenvectors.shape[0]) :\n",
    "        if i == 0 :\n",
    "            new = [X.dot(X_eigenvectors.T[i])]\n",
    "        else :\n",
    "            new = np.concatenate((new, [X.dot(X_eigenvectors.T[i])]),axis = 0)\n",
    "    \n",
    "    return new.T"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 86,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 0.31019368, -1.08215716, -0.07983642],\n",
       "       [ 1.28092404, -0.43132556,  0.13533091],\n",
       "       [ 1.38766381,  0.78428014, -0.12911446],\n",
       "       [ 0.95087515, -1.15737142,  1.6495519 ],\n",
       "       [ 1.84222365,  0.88189889,  0.11493111],\n",
       "       [-1.12563709, -0.52680338,  0.06564012],\n",
       "       [-2.71174416,  0.63290138,  0.71195473],\n",
       "       [-0.03100441, -0.20059783, -0.50339479],\n",
       "       [ 2.29618509,  0.07661447,  0.01087174],\n",
       "       [-0.61585248, -0.205764  ,  1.82651199],\n",
       "       [-1.73320252,  1.29971699,  0.09045178],\n",
       "       [-0.82366049, -0.57164535, -0.27123176],\n",
       "       [ 0.75619512,  0.73995175, -0.76710616],\n",
       "       [-0.42344386,  0.26555394, -1.41533681],\n",
       "       [-0.39581307, -1.64646874,  0.24104031],\n",
       "       [-0.88581498,  0.15195119, -0.82271209],\n",
       "       [ 0.24587691,  0.39139878, -1.15801831],\n",
       "       [ 0.14741103, -1.22874561, -0.03110396],\n",
       "       [-0.7161265 , -0.56781471, -0.86180345],\n",
       "       [ 0.24475107,  2.39442622,  1.19337361]])"
      ]
     },
     "execution_count": 86,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "new_coordinates(X_std, X_eigenvectors)\n",
    "# 고유벡터 축으로 값 변환"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "B7c1o5URPgst"
   },
   "source": [
    "# 2) PCA를 구현해보기\n",
    "\n",
    "위의 과정을 하나의 함수로 표현해주세요!\n",
    "\n",
    "1)에서 진행한 과정을 그대로 가져오면 됩니다..! , \n",
    "\n",
    "위에서 만든 함수(new_coordinate)도 가져오셔서 사용해도 좋아요!\n",
    "\n",
    "+하지만 이 함수는 number이라는 인자를 받아서 pca에서 주성분 개수를 고려하도록 합니다!\n",
    "\n",
    "고유값이 큰 축 순서로 정렬하는 과정 + 주성분 개수 number을 고려해야한다는 점을 유의해서 함수를 정의해주세요\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 87,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "executionInfo": {
     "elapsed": 272,
     "status": "ok",
     "timestamp": 1629794797564,
     "user": {
      "displayName": "김건우",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14GifAuBZsXu6RawBPfBoYmIJaUQ0zRxVy1udxOxf=s64",
      "userId": "13140730798096026130"
     },
     "user_tz": -540
    },
    "id": "P_cF1LnX6eBh",
    "outputId": "b644f825-9c82-4573-ea35-20b5a3ea0d56"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 0.47018528, -0.85137353, -0.23257022],\n",
       "       [-0.64960236, -0.15545725, -0.74421087],\n",
       "       [-0.59744671, -0.50099516,  0.62614797]])"
      ]
     },
     "execution_count": 87,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_eigenvectors"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 93,
   "metadata": {
    "executionInfo": {
     "elapsed": 405,
     "status": "ok",
     "timestamp": 1629794918195,
     "user": {
      "displayName": "김건우",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14GifAuBZsXu6RawBPfBoYmIJaUQ0zRxVy1udxOxf=s64",
      "userId": "13140730798096026130"
     },
     "user_tz": -540
    },
    "id": "5qOrSvvZPgst"
   },
   "outputs": [],
   "source": [
    "from sklearn.preprocessing import StandardScaler\n",
    "\n",
    "def MYPCA(X,number):\n",
    "  # Scaling\n",
    "  scaler = StandardScaler()\n",
    "  X_std = scaler.fit_transform(X)\n",
    "    \n",
    "  # Covariance\n",
    "  features = X_std.T\n",
    "  cov_matrix = np.cov(features)\n",
    "  \n",
    "  # eigenvalues, eigenvectors\n",
    "  X_eigenvalues, X_eigenvectors = np.linalg.eig(cov_matrix)\n",
    "\n",
    "  #np.dot(np.dot(eigenvectors,mat),eigenvectors.T)\n",
    "  sym_matrix = np.zeros((3,3))\n",
    "\n",
    "  sym_matrix[0][0] = X_eigenvalues[0]\n",
    "  sym_matrix[1][1] = X_eigenvalues[1]\n",
    "  sym_matrix[2][2] = X_eigenvalues[2]\n",
    "\n",
    "  np.dot(np.dot(X_eigenvectors, sym_matrix),X_eigenvectors.T)\n",
    "  \n",
    "  # 고유값 정렬 및 고유값 크기에 따라 고유벡터도 정렬\n",
    "  new_coordinates(X_std, X_eigenvectors)\n",
    "  new_coordinate = new_coordinates(X_std, X_eigenvectors)\n",
    "  #def new_coordinates(X,eigenvectors):\n",
    "  index = X_eigenvalues.argsort()\n",
    "  index = list(index)\n",
    "  \n",
    "  for i in range(X_eigenvectors.shape[0]) :\n",
    "    if i == 0 :\n",
    "      new = [X.dot(X_eigenvectors.T[i])]\n",
    "    else :\n",
    "      new = np.concatenate((new, [X.dot(X_eigenvectors.T[i])]),axis = 0)\n",
    "    \n",
    "  return new.T\n",
    "  #return new.T #새로운 축으로 변환되어 나타난 데이터 return"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 89,
   "metadata": {
    "id": "XAtXIzPHPgst",
    "outputId": "a31eef56-131b-4c5b-e852-db115b7c67a2",
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ -25.76459322, -118.14281563,  -28.07954506],\n",
       "       [   4.93440853,  -98.70617271,  -19.96855233],\n",
       "       [   9.41514902,  -64.58604088,  -28.31957436],\n",
       "       [  -3.65047618, -119.76318703,   26.85987816],\n",
       "       [  23.73639094,  -61.29649622,  -20.00808077],\n",
       "       [ -68.34991312, -104.64973959,  -25.97539951],\n",
       "       [-113.86586512,  -74.61806491,   -8.6434455 ],\n",
       "       [ -35.59179944,  -93.91861469,  -42.02996106],\n",
       "       [  36.24493662,  -83.11645317,  -22.33740768],\n",
       "       [ -49.61549403,  -95.40201664,   29.64385221],\n",
       "       [ -84.24883684,  -54.53151206,  -26.5978887 ],\n",
       "       [ -59.78590706, -105.40932209,  -35.97944043],\n",
       "       [ -10.83387534,  -66.54018042,  -49.20397885],\n",
       "       [ -48.32592432,  -81.24076455,  -71.18667863],\n",
       "       [ -47.45130749, -134.92700444,  -19.080799  ],\n",
       "       [ -61.56519276,  -85.18021914,  -53.43273629],\n",
       "       [ -27.42651318,  -76.87270009,  -62.12386301],\n",
       "       [ -30.8278953 , -122.46653186,  -26.78786006],\n",
       "       [ -57.46340178, -105.01873966,  -54.21274552],\n",
       "       [ -20.94871091,  -21.53721118,   10.69111995]])"
      ]
     },
     "execution_count": 89,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "MYPCA(X,3)\n",
    "\n",
    "# 새로운 축으로 잘 변환되어서 나타나나요?\n",
    "# 위에서 했던 PCA랑은 차이가 있을 수 있어요 \n",
    "# 왜냐하면 위에서는 고유값이 큰 축 순서로 정렬을 안했었거든요"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Q8E2iMssPgsu"
   },
   "source": [
    "# 3) sklearn이랑 비교를 해보기\n",
    "\n",
    "https://scikit-learn.org/stable/modules/generated/sklearn.decomposition.PCA.html 를 참고하시면 도움이 될거에요"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 90,
   "metadata": {
    "id": "YLelmhAXPgsu"
   },
   "outputs": [],
   "source": [
    "from sklearn.decomposition import PCA\n",
    "pca = PCA(n_components= 3) # 주성분 개수 3개로 사용\n",
    "\n",
    "#scaled된 X_std에대해서 pca 학습 시킴\n",
    "pca_scaled = pca.fit_transform(X_std)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 91,
   "metadata": {
    "id": "b23tW08RPgsu",
    "outputId": "ec080ca6-8984-4a5a-8588-62dd2278165e"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[-0.31019368, -1.08215716, -0.07983642],\n",
       "       [-1.28092404, -0.43132556,  0.13533091],\n",
       "       [-1.38766381,  0.78428014, -0.12911446],\n",
       "       [-0.95087515, -1.15737142,  1.6495519 ],\n",
       "       [-1.84222365,  0.88189889,  0.11493111],\n",
       "       [ 1.12563709, -0.52680338,  0.06564012],\n",
       "       [ 2.71174416,  0.63290138,  0.71195473],\n",
       "       [ 0.03100441, -0.20059783, -0.50339479],\n",
       "       [-2.29618509,  0.07661447,  0.01087174],\n",
       "       [ 0.61585248, -0.205764  ,  1.82651199],\n",
       "       [ 1.73320252,  1.29971699,  0.09045178],\n",
       "       [ 0.82366049, -0.57164535, -0.27123176],\n",
       "       [-0.75619512,  0.73995175, -0.76710616],\n",
       "       [ 0.42344386,  0.26555394, -1.41533681],\n",
       "       [ 0.39581307, -1.64646874,  0.24104031],\n",
       "       [ 0.88581498,  0.15195119, -0.82271209],\n",
       "       [-0.24587691,  0.39139878, -1.15801831],\n",
       "       [-0.14741103, -1.22874561, -0.03110396],\n",
       "       [ 0.7161265 , -0.56781471, -0.86180345],\n",
       "       [-0.24475107,  2.39442622,  1.19337361]])"
      ]
     },
     "execution_count": 91,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pca_scaled"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 92,
   "metadata": {
    "id": "4LuSOZgVPgsu",
    "outputId": "dd069c4a-cebe-47eb-ca61-c43ae1cc2550"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ -25.76459322, -118.14281563,  -28.07954506],\n",
       "       [   4.93440853,  -98.70617271,  -19.96855233],\n",
       "       [   9.41514902,  -64.58604088,  -28.31957436],\n",
       "       [  -3.65047618, -119.76318703,   26.85987816],\n",
       "       [  23.73639094,  -61.29649622,  -20.00808077],\n",
       "       [ -68.34991312, -104.64973959,  -25.97539951],\n",
       "       [-113.86586512,  -74.61806491,   -8.6434455 ],\n",
       "       [ -35.59179944,  -93.91861469,  -42.02996106],\n",
       "       [  36.24493662,  -83.11645317,  -22.33740768],\n",
       "       [ -49.61549403,  -95.40201664,   29.64385221],\n",
       "       [ -84.24883684,  -54.53151206,  -26.5978887 ],\n",
       "       [ -59.78590706, -105.40932209,  -35.97944043],\n",
       "       [ -10.83387534,  -66.54018042,  -49.20397885],\n",
       "       [ -48.32592432,  -81.24076455,  -71.18667863],\n",
       "       [ -47.45130749, -134.92700444,  -19.080799  ],\n",
       "       [ -61.56519276,  -85.18021914,  -53.43273629],\n",
       "       [ -27.42651318,  -76.87270009,  -62.12386301],\n",
       "       [ -30.8278953 , -122.46653186,  -26.78786006],\n",
       "       [ -57.46340178, -105.01873966,  -54.21274552],\n",
       "       [ -20.94871091,  -21.53721118,   10.69111995]])"
      ]
     },
     "execution_count": 92,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "MYPCA(X,3)"
   ]
  }
 ],
 "metadata": {
  "colab": {
   "name": "week5_dimensionality reduction_assignment1.ipynb",
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}