Report - Resale Price Prediction of HDB Resale Flats in Singapore.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<script>$(\"div.input\").hide()</script>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "24e3c1ed39d44694922a633fb5e00677",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "ToggleButton(value=False, description='Show code')"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<script>$(\"div.input\").show()</script>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<script>$(\"div.input\").hide()</script>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<script>$(\"div.input\").show()</script>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<script>$(\"div.input\").hide()</script>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import ipywidgets as widgets\n",
    "from IPython.display import display, HTML\n",
    "\n",
    "javascript_functions = {False: \"hide()\", True: \"show()\"}\n",
    "button_descriptions  = {False: \"Show code\", True: \"Hide code\"}\n",
    "\n",
    "\n",
    "def toggle_code(state):\n",
    "\n",
    "    \"\"\"\n",
    "    Toggles the JavaScript show()/hide() function on the div.input element.\n",
    "    \"\"\"\n",
    "\n",
    "    output_string = \"<script>$(\\\"div.input\\\").{}</script>\"\n",
    "    output_args   = (javascript_functions[state],)\n",
    "    output        = output_string.format(*output_args)\n",
    "\n",
    "    display(HTML(output))\n",
    "\n",
    "\n",
    "def button_action(value):\n",
    "\n",
    "    \"\"\"\n",
    "    Calls the toggle_code function and updates the button description.\n",
    "    \"\"\"\n",
    "\n",
    "    state = value.new\n",
    "\n",
    "    toggle_code(state)\n",
    "\n",
    "    value.owner.description = button_descriptions[state]\n",
    "\n",
    "\n",
    "state = False\n",
    "toggle_code(state)\n",
    "\n",
    "button = widgets.ToggleButton(state, description = button_descriptions[state])\n",
    "button.observe(button_action, \"value\")\n",
    "\n",
    "display(button)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Import required libraries\n",
    "import numpy as np\n",
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h1><center>Resale Price Prediction of HDB Resale Flats in Singapore</center></h1>\n",
    "<h3 style=\"margin:0\"><center>Lee Gang</center></h3>\n",
    "<h3 style=\"margin:0\"><center>29 February 2020</center></h3>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Executive Summary"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This project attempts to construct a high accuracy predictive model using publicly available transaction data and nearby venues data from Foursquare API to predict the resale price for HDB Flats in Singapore. The purpose of the project is to provide an easily accessible and free-to-use application that potential buyers and sellers of HDB Resale Flats can use to get a quick and accurate estimate of the resale price of a HDB Flat.\n",
    "\n",
    "A high performing model is successfully built using the **Extreme Gradient Boosted Trees** algorithm, achieving a mean absolute error of only **S\\\\$122** per square meter, or **2.7\\%** of average resale price. Around 66\\% of all predictions will have an error of less than **S\\\\$207**, while 95\\% of all predictions will have an error of less than **S\\\\$406**. The predictive model is able to explain **97\\%** of the variance in the data."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Table of Contents"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "1. [Introduction](#introduction)\n",
    "2. [Methodology](#methodology)\n",
    "3. [Results and Discussion](#results)\n",
    "4. [Conclusion](#conclusion)\n",
    "5. [References](#references)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. Introduction <a name=\"introduction\"></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.1 Business Problem"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Property valuations are usually costly and take time to prepare. Buyers and sellers of properties generally resort to comparing recent transaction prices of nearby and similar properties to estimate the property value. However, there are several limitations to this approach. Firstly, there might be limited transactions within the nearby vicinity. Secondly, the evaluation of similarity between properties is highly subjective, with a wide range of factors that might contribute to the final transaction price of a particular property.\n",
    "\n",
    "In the context of Singapore, a property valuation report is required and can only be requested after an Option To Purchase (OTP) is granted by the seller of a HDB unit to the buyer. The amount of loan that the buyer can take up will depend on the property value as stated in the valuation report, or the offer price stated in the OTP, whichever lower. A buyer may potentially face the situation where the offer price is higher than the valuation price, therefore requiring an upfront additional cash payment for the difference in value. If the buyer decides to not exercise the OTP, he/she will be forfeiting the deposit paid as part of the OTP. As such, the buyer will face significant financial risk if the assessment of the property price is not conducted diligently. On the other hand, a common challenge faced by sellers are the price setting of properties. Overpricing and underpricing properties will often result in loss of profit or prolonged holding of the property.\n",
    "\n",
    "As such, this project aims to develop a predictive model to provide both sellers and buyers a quick and easy way to estimate a property's value without spending a single cent. However, as the focus of this project is to develop a predictive model with high accuracy, we will not be delving deep into the effects of various predictors on the estimated price."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.2 Scope and Background"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Singapore's residential property market can be categorized into the public housing market, and the private housing market. The private housing market generally consist of condominiums or landed property developed by private property developers, and does not have any restrictions or requirements in regards to the buyers' citizenship. In contrast, public housing in Singapore is managed by the Housing and Development Board (HDB), and usually are on a 99-years leasehold basis. These high-rise apartments or flats are generally located within housing estates, which are self-contained satellite towns with schools, supermarkets and other facilities. These residential properties accounts for over 80% of the Singapore Resident population's housing needs<sup>[[1](#Ref1)]</sup>, and are called HDB Flats. However, HDB flats and are only available for purchase to Singaporean Citizens and Permanent Residents. \n",
    "\n",
    "Strictly speaking, only Singaporean Citizens are allowed to purchase new HDB flats. The price of the new HDB flat is set by the government agency and the entire process from application to occupation takes around two to four years. Singaporean Permanent Residents and Singaporean Citizens who do not want to wait for such long periods of time will have to purchase HDB flats from the resale market, where property prices are decided by various market forces.\n",
    "\n",
    "In view of the relative high transaction volume of HDB resale flats and free market pricing, we will focus on building a prediction model to estimate the transaction price for HDB Resale Flats for this project."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. Methodology <a name=\"methodology\"></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.1 Data Sources and Assumptions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "All transactions data on HDB Resale Flats are made available publicly at **Data.gov.sg** through their API every month<sup>[[2](#Ref2)]</sup>. The transactions exclude resale transactions that may not reflect the full market price such as resale between relatives and resale of part shares. We will only utilize transaction data from 2017 to 2020 due to a different assumption and data classification methods adopted by HDB prior to 2017. The HDB Resale Flats transaction data contains information on the address, floor area, transaction price, transaction registration date and other information regarding the property such as property type and storeys.\n",
    "\n",
    "A sample header of the HDB Resale Flats transaction data is as below:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>town</th>\n",
       "      <th>flat_type</th>\n",
       "      <th>flat_model</th>\n",
       "      <th>floor_area_sqm</th>\n",
       "      <th>street_name</th>\n",
       "      <th>resale_price</th>\n",
       "      <th>month</th>\n",
       "      <th>remaining_lease</th>\n",
       "      <th>lease_commence_date</th>\n",
       "      <th>storey_range</th>\n",
       "      <th>_id</th>\n",
       "      <th>block</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>JURONG EAST</td>\n",
       "      <td>3 ROOM</td>\n",
       "      <td>New Generation</td>\n",
       "      <td>67.0</td>\n",
       "      <td>JURONG EAST ST 21</td>\n",
       "      <td>270000.0</td>\n",
       "      <td>2018-07</td>\n",
       "      <td>63 years 08 months</td>\n",
       "      <td>1983</td>\n",
       "      <td>10 TO 12</td>\n",
       "      <td>1</td>\n",
       "      <td>214</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>JURONG EAST</td>\n",
       "      <td>3 ROOM</td>\n",
       "      <td>New Generation</td>\n",
       "      <td>67.0</td>\n",
       "      <td>JURONG EAST ST 21</td>\n",
       "      <td>248000.0</td>\n",
       "      <td>2018-07</td>\n",
       "      <td>63 years 09 months</td>\n",
       "      <td>1983</td>\n",
       "      <td>01 TO 03</td>\n",
       "      <td>2</td>\n",
       "      <td>211</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>JURONG EAST</td>\n",
       "      <td>3 ROOM</td>\n",
       "      <td>New Generation</td>\n",
       "      <td>84.0</td>\n",
       "      <td>JURONG EAST ST 24</td>\n",
       "      <td>318000.0</td>\n",
       "      <td>2018-07</td>\n",
       "      <td>63 years 07 months</td>\n",
       "      <td>1983</td>\n",
       "      <td>07 TO 09</td>\n",
       "      <td>3</td>\n",
       "      <td>241</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>JURONG EAST</td>\n",
       "      <td>3 ROOM</td>\n",
       "      <td>New Generation</td>\n",
       "      <td>67.0</td>\n",
       "      <td>JURONG EAST ST 24</td>\n",
       "      <td>305000.0</td>\n",
       "      <td>2018-07</td>\n",
       "      <td>66 years 11 months</td>\n",
       "      <td>1986</td>\n",
       "      <td>04 TO 06</td>\n",
       "      <td>4</td>\n",
       "      <td>250</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>JURONG EAST</td>\n",
       "      <td>3 ROOM</td>\n",
       "      <td>New Generation</td>\n",
       "      <td>67.0</td>\n",
       "      <td>JURONG EAST ST 24</td>\n",
       "      <td>258000.0</td>\n",
       "      <td>2018-07</td>\n",
       "      <td>65 years 11 months</td>\n",
       "      <td>1985</td>\n",
       "      <td>01 TO 03</td>\n",
       "      <td>5</td>\n",
       "      <td>252</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          town flat_type      flat_model  floor_area_sqm        street_name  \\\n",
       "0  JURONG EAST    3 ROOM  New Generation            67.0  JURONG EAST ST 21   \n",
       "1  JURONG EAST    3 ROOM  New Generation            67.0  JURONG EAST ST 21   \n",
       "2  JURONG EAST    3 ROOM  New Generation            84.0  JURONG EAST ST 24   \n",
       "3  JURONG EAST    3 ROOM  New Generation            67.0  JURONG EAST ST 24   \n",
       "4  JURONG EAST    3 ROOM  New Generation            67.0  JURONG EAST ST 24   \n",
       "\n",
       "   resale_price    month     remaining_lease  lease_commence_date  \\\n",
       "0      270000.0  2018-07  63 years 08 months                 1983   \n",
       "1      248000.0  2018-07  63 years 09 months                 1983   \n",
       "2      318000.0  2018-07  63 years 07 months                 1983   \n",
       "3      305000.0  2018-07  66 years 11 months                 1986   \n",
       "4      258000.0  2018-07  65 years 11 months                 1985   \n",
       "\n",
       "  storey_range  _id block  \n",
       "0     10 TO 12    1   214  \n",
       "1     01 TO 03    2   211  \n",
       "2     07 TO 09    3   241  \n",
       "3     04 TO 06    4   250  \n",
       "4     01 TO 03    5   252  "
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_hdb = pd.read_csv('./Data/dat_hdb.csv')\n",
    "df_hdb.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p><center><i><b>Table 1</b>: Header View of The HDB Resale Flat Transaction Data Extracted From Data.gov.sg API</i></center></p>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Property price are usually also affected by the venues and facilities available within the vicinity such as supermarkets, shopping malls, train station etc. In order to expand our dataset to include these information, we geocoded the address for each of the transacted properties to extract their map coordinates and leveraged on [**Foursquare's Places API**](https://developer.foursquare.com/) to retrieve all nearby venues within a 500m radius of the property. \n",
    "\n",
    "The nearby venues extracted are then categorized into major categories that might contribute to the property transacted price such as *Train Stations*, *Bus Stops*, *Restaurants*, *Schools*, *Park and Gardens*, *Supermarkets*, *Gyms*, *Theaters*, *Food Courts*, *Museums* and *Shopping Malls*. The number of venues for each of these categories were then merged with the transacted property dataset.\n",
    "\n",
    "To remove the effect of the property's *floor area* on the *resale price*, we transformed the *resale price* to *price per square meter*, which will be used as the target variable/ dependent variable for our analysis. A sample view of the combined dataset is as below:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>floor_area_sqm</th>\n",
       "      <th>month</th>\n",
       "      <th>remaining_lease</th>\n",
       "      <th>year</th>\n",
       "      <th>price_sqm</th>\n",
       "      <th>Bus Stop</th>\n",
       "      <th>Food Court</th>\n",
       "      <th>Gym</th>\n",
       "      <th>Park</th>\n",
       "      <th>Restaurant</th>\n",
       "      <th>...</th>\n",
       "      <th>storey_range_22 TO 24</th>\n",
       "      <th>storey_range_25 TO 27</th>\n",
       "      <th>storey_range_28 TO 30</th>\n",
       "      <th>storey_range_31 TO 33</th>\n",
       "      <th>storey_range_34 TO 36</th>\n",
       "      <th>storey_range_37 TO 39</th>\n",
       "      <th>storey_range_40 TO 42</th>\n",
       "      <th>storey_range_43 TO 45</th>\n",
       "      <th>storey_range_46 TO 48</th>\n",
       "      <th>storey_range_49 TO 51</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>67.0</td>\n",
       "      <td>7.0</td>\n",
       "      <td>63.666667</td>\n",
       "      <td>2018</td>\n",
       "      <td>4029.850746</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>17.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>67.0</td>\n",
       "      <td>7.0</td>\n",
       "      <td>63.750000</td>\n",
       "      <td>2018</td>\n",
       "      <td>3701.492537</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>17.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>84.0</td>\n",
       "      <td>7.0</td>\n",
       "      <td>63.583333</td>\n",
       "      <td>2018</td>\n",
       "      <td>3785.714286</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>67.0</td>\n",
       "      <td>7.0</td>\n",
       "      <td>66.916667</td>\n",
       "      <td>2018</td>\n",
       "      <td>4552.238806</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>67.0</td>\n",
       "      <td>7.0</td>\n",
       "      <td>65.916667</td>\n",
       "      <td>2018</td>\n",
       "      <td>3850.746269</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 84 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   floor_area_sqm  month  remaining_lease  year    price_sqm  Bus Stop  \\\n",
       "0            67.0    7.0        63.666667  2018  4029.850746       0.0   \n",
       "1            67.0    7.0        63.750000  2018  3701.492537       0.0   \n",
       "2            84.0    7.0        63.583333  2018  3785.714286       0.0   \n",
       "3            67.0    7.0        66.916667  2018  4552.238806       0.0   \n",
       "4            67.0    7.0        65.916667  2018  3850.746269       0.0   \n",
       "\n",
       "   Food Court  Gym  Park  Restaurant  ...  storey_range_22 TO 24  \\\n",
       "0         0.0  0.0   0.0        17.0  ...                      0   \n",
       "1         0.0  0.0   0.0        17.0  ...                      0   \n",
       "2         2.0  1.0   0.0         2.0  ...                      0   \n",
       "3         2.0  1.0   0.0         2.0  ...                      0   \n",
       "4         2.0  1.0   0.0         2.0  ...                      0   \n",
       "\n",
       "   storey_range_25 TO 27  storey_range_28 TO 30  storey_range_31 TO 33  \\\n",
       "0                      0                      0                      0   \n",
       "1                      0                      0                      0   \n",
       "2                      0                      0                      0   \n",
       "3                      0                      0                      0   \n",
       "4                      0                      0                      0   \n",
       "\n",
       "   storey_range_34 TO 36  storey_range_37 TO 39  storey_range_40 TO 42  \\\n",
       "0                      0                      0                      0   \n",
       "1                      0                      0                      0   \n",
       "2                      0                      0                      0   \n",
       "3                      0                      0                      0   \n",
       "4                      0                      0                      0   \n",
       "\n",
       "   storey_range_43 TO 45  storey_range_46 TO 48  storey_range_49 TO 51  \n",
       "0                      0                      0                      0  \n",
       "1                      0                      0                      0  \n",
       "2                      0                      0                      0  \n",
       "3                      0                      0                      0  \n",
       "4                      0                      0                      0  \n",
       "\n",
       "[5 rows x 84 columns]"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_final = pd.read_csv('./Data/dat_final.csv')\n",
    "df_final.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p><center><i><b>Table 2</b>: Header View of The Processed Data Ready For Model Building and Evaluation</i></center></p>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.2 Exploratory Data Analysis and Feature Selection"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Co-Linearity Between Variables**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Exploratory data analysis was performed on the combined dataset. A correlation plot is ploted to investigate the degree of co-linearity between independent variables (Figure 1). There are no independent variables with high degree of co-linearity, thus no columns were dropped."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"./Images/Correlation_Plot.png\">"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p><center><i><b>Figure 1</b>: Correlation Plot Between Variables in HDB Resale and Nearby Venues Dataset</i></center></p>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Zero and Near-Zero Variance Variables**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There are no zero variance variables in the dataset. However, there are 54 variables with less than 5% variance. Most of these variables with less than 5% variance is mainly due to one-hot encoding of the **'town'**, **'flat_type'**, **'flat_model'** and **'storey_range'** variables. These variables may hold important information, so we will retain these variables. \n",
    "\n",
    "On the other hand, the variables **'General College & University'**, **'High School'** and **'Musuem'** are also identified to have a variance of less than 5%. These categories of venues seem to be rare around residential areas, and might introduce more noise than information for our models. We will proceed to drop these columns from our dataset."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The final dataset is then randomly split into a **training set (75%)** and **testing set (25%)** for model development and evaluation."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.3 Model Development and Evaluation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Two different models were developed using **Lasso Regression** and **Extreme Gradient Boosted Trees (XGBoost)**. **Lasso Regression** is used because **Multivariate Linear Regression** is a common technique employed by real estate appraisers in determining the various adjustment factors<sup>[[3](#Ref3)]</sup>. As the dataset is sparse (with high number of features), **Lasso Regression** is used as it introduces the L1 regularization term into **(Multivariate) Linear Regression**, which regularizes the magnitudes of the coefficients (feature selection). The model is also easily interpretable and the various predictors' effect on the resale price can be easily quantified. \n",
    "\n",
    "For **Lasso Regression**, we performed a grid search via 5-fold cross-validation to select the best alpha value for the model. We then build the final **Lasso Regression** model using the best alpha value.\n",
    "\n",
    "**XGBoost** is used as it is one of the top performing algorithms in many machine learning competitions and is efficient considered its complexity. However, interpreting the relatonship between predictors and the target variable in **XGBoost** models can be challenging.\n",
    "\n",
    "For the **XGBoost** model, a randomized grid search with 5-fold cross-validation is used to tuned the hyper-parameters *max_depth*, *min_child_weight*, *gamma*, *subsample*, *ncol_subsample* and *learning_rate*. The best parameters selected is then used to select the best *num_boost_rounds* via 5-fold cross-validation, which is then used to build the final **XGBoost** model.\n",
    "\n",
    "Both models are evaluated using the testing set, and their **mean absolute error (MAE)**, **root mean squared error (RMSE)** and **R<sub>2</sub> scores** are computed and compared. The model with the smallest error is then selected."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. Results and Discussion <a name=\"results\"></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.1 Model 1: Lasso Regression"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Using the testing dataset for model evaluation, the final **Lasso Regression** model performed well with a **mean absolute error (mae)** of **S\\$406** (per sqm), or **9\\%** of **mean price** (per sqm). \n",
    "\n",
    "The **root mean squared error (rmse)** of the model evaluation is **S\\$527** (per sqm). This is also the standard deviation of the prediction error. We can derive the margin of error for the 95% confidence interval of our prediction to be **+-S\\$1,333** (per sqm).\n",
    "\n",
    "The **R<sup>2</sup> Score** of the model can be interpret as the percentage of variance that is explained by the model. In this case, the **Lasso Regression** model is able to explain **81\\%** of the variance in the data.\n",
    "\n",
    "The residual plot and histogram of residuals of the **Lasso Regression** model (Figure 2) does not suggest any major violation of the following assumptions of the model. The distribution of residuals in the histogram forms a normal distribution, while the standardize residuals are almost distributed randomly around the horizontal axis in the residual plot. There are no major bias observed from the residuals. As such, the **Lasso Regression** model is a valid model that can be used to predict the resale price of HDB Flats."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"./Images/Lasso_Residuals.png\">"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p><center><i><b>Figure 2</b>: Histogram and Residual Plot of Lasso Regression Model</i></center></p>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.2 Model 2: Extreme Gradient Boosted Trees (XGBoost)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Using the testing dataset for model evaluation, the final **XGBoost** model performed very well with a **mean absolute error (mae)** of **S\\$121** (per sqm), or **2.7\\%** of **mean price** (per sqm). \n",
    "\n",
    "The **root mean squared error (rmse)** of the model evaluation is **S\\$207** (per sqm). This is also the standard deviation of the prediction error. We can derive the margin of error for the 95% confidence interval of our prediction to be **+-S\\$406** (per sqm).\n",
    "\n",
    "The **R<sup>2</sup> Score** of the model can be interpret as the percentage of variance that is explained by the model. In this case, the **XGBoost** model is able to explain **97\\%** of the variance in the data.\n",
    "\n",
    "The residual plot and histogram of residuals of the **XGBoost** model (Figure 3) shows that the distribution of residuals in the histogram forms a normal distribution, while the standardize residuals are almost distributed randomly around the horizontal axis in the residual plot. There are no major bias observed from the residuals. As such, the **XGBoost** model is a valid model that can be used to predict the resale price of HDB Flats."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"./Images/XGB_Residuals.png\">"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p><center><i><b>Figure 3</b>: Histogram and Residual Plot of XGBoost Model</i></center></p>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.3 Model Selection"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Looking at the performance metric between both **Lasso Regression** and **XGBoost** models, **XGBoost** outperforms **Lasso Regression** significantly. For an example, the **mae** for **XGBoost** is just **S\\$121** (per sqm), or **2.7\\%** of the **mean resale price** (per sqm). This is a 70\\% reduction in the **mae** compared to the **Lasso Regression** model's **mae**. \n",
    "\n",
    "The standard deviation of the prediction error or **rmse** of **XGBoost** model is also 60\\% lesser than the **rmse** of the **Lasso Regression** model.\n",
    "\n",
    "Furthermore, the **XGBoost** model is able to explain **97\\%** of the variance in the data, as compared to **81\\%** by the **Lasso Regression** model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Metrics</th>\n",
       "      <th>Lasso Regression</th>\n",
       "      <th>XGBoost</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>MAE</td>\n",
       "      <td>406.29</td>\n",
       "      <td>121.92</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>RMSE</td>\n",
       "      <td>527.06</td>\n",
       "      <td>206.77</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>R2 Score</td>\n",
       "      <td>0.81</td>\n",
       "      <td>0.97</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    Metrics  Lasso Regression  XGBoost\n",
       "0       MAE            406.29   121.92\n",
       "1      RMSE            527.06   206.77\n",
       "2  R2 Score              0.81     0.97"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_results = pd.read_csv(\"./Data/dat_results.csv\")\n",
    "df_results"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p><i><b>Table 3</b>: Comparison of Performance Metrics Between Lasso Regression and XGBoost Models</i></p>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The following table (Table 4) displays ten predictions for the most recent HDB Flat Resale transactions (January 2020) from the testing dataset. The *resale_price* is the actual transacted price, whereas the *pred_resale_price* is the predicted resale price by the **XGBoost** model. The predictions are surprisingly accurate."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>resale_price</th>\n",
       "      <th>pred_resale_price</th>\n",
       "      <th>price_sqm</th>\n",
       "      <th>pred_price</th>\n",
       "      <th>town</th>\n",
       "      <th>flat_type</th>\n",
       "      <th>flat_model</th>\n",
       "      <th>floor_area_sqm</th>\n",
       "      <th>street_name</th>\n",
       "      <th>month</th>\n",
       "      <th>remaining_lease</th>\n",
       "      <th>lease_commence_date</th>\n",
       "      <th>storey_range</th>\n",
       "      <th>id</th>\n",
       "      <th>block</th>\n",
       "      <th>reg_date</th>\n",
       "      <th>year</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>426000.0</td>\n",
       "      <td>424631.0</td>\n",
       "      <td>4580.645161</td>\n",
       "      <td>4565.9233</td>\n",
       "      <td>PUNGGOL</td>\n",
       "      <td>4 ROOM</td>\n",
       "      <td>Premium Apartment</td>\n",
       "      <td>93.0</td>\n",
       "      <td>PUNGGOL CENTRAL</td>\n",
       "      <td>1.0</td>\n",
       "      <td>89.500000</td>\n",
       "      <td>2010</td>\n",
       "      <td>04 TO 06</td>\n",
       "      <td>67643</td>\n",
       "      <td>301C</td>\n",
       "      <td>2020-01-01</td>\n",
       "      <td>2020</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>600000.0</td>\n",
       "      <td>597799.0</td>\n",
       "      <td>5084.745763</td>\n",
       "      <td>5066.0900</td>\n",
       "      <td>CLEMENTI</td>\n",
       "      <td>5 ROOM</td>\n",
       "      <td>Improved</td>\n",
       "      <td>118.0</td>\n",
       "      <td>CLEMENTI WEST ST 2</td>\n",
       "      <td>1.0</td>\n",
       "      <td>60.083333</td>\n",
       "      <td>1981</td>\n",
       "      <td>19 TO 21</td>\n",
       "      <td>66801</td>\n",
       "      <td>724</td>\n",
       "      <td>2020-01-01</td>\n",
       "      <td>2020</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>433888.0</td>\n",
       "      <td>431828.0</td>\n",
       "      <td>3337.600000</td>\n",
       "      <td>3321.7534</td>\n",
       "      <td>WOODLANDS</td>\n",
       "      <td>5 ROOM</td>\n",
       "      <td>Improved</td>\n",
       "      <td>130.0</td>\n",
       "      <td>WOODLANDS ST 31</td>\n",
       "      <td>1.0</td>\n",
       "      <td>71.916667</td>\n",
       "      <td>1993</td>\n",
       "      <td>01 TO 03</td>\n",
       "      <td>101799</td>\n",
       "      <td>301</td>\n",
       "      <td>2020-01-01</td>\n",
       "      <td>2020</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>375588.0</td>\n",
       "      <td>374900.0</td>\n",
       "      <td>5523.352941</td>\n",
       "      <td>5513.2334</td>\n",
       "      <td>PUNGGOL</td>\n",
       "      <td>3 ROOM</td>\n",
       "      <td>Premium Apartment</td>\n",
       "      <td>68.0</td>\n",
       "      <td>PUNGGOL DRIVE</td>\n",
       "      <td>1.0</td>\n",
       "      <td>95.166667</td>\n",
       "      <td>2016</td>\n",
       "      <td>07 TO 09</td>\n",
       "      <td>67619</td>\n",
       "      <td>667A</td>\n",
       "      <td>2020-01-01</td>\n",
       "      <td>2020</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>428000.0</td>\n",
       "      <td>435935.0</td>\n",
       "      <td>4652.173913</td>\n",
       "      <td>4738.4280</td>\n",
       "      <td>YISHUN</td>\n",
       "      <td>4 ROOM</td>\n",
       "      <td>Model A</td>\n",
       "      <td>92.0</td>\n",
       "      <td>YISHUN AVE 9</td>\n",
       "      <td>1.0</td>\n",
       "      <td>94.750000</td>\n",
       "      <td>2015</td>\n",
       "      <td>13 TO 15</td>\n",
       "      <td>69215</td>\n",
       "      <td>317A</td>\n",
       "      <td>2020-01-01</td>\n",
       "      <td>2020</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>710000.0</td>\n",
       "      <td>688264.0</td>\n",
       "      <td>4797.297297</td>\n",
       "      <td>4650.4340</td>\n",
       "      <td>SERANGOON</td>\n",
       "      <td>EXECUTIVE</td>\n",
       "      <td>Maisonette</td>\n",
       "      <td>148.0</td>\n",
       "      <td>SERANGOON NORTH AVE 4</td>\n",
       "      <td>1.0</td>\n",
       "      <td>71.750000</td>\n",
       "      <td>1992</td>\n",
       "      <td>10 TO 12</td>\n",
       "      <td>101497</td>\n",
       "      <td>525</td>\n",
       "      <td>2020-01-01</td>\n",
       "      <td>2020</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>570000.0</td>\n",
       "      <td>573616.0</td>\n",
       "      <td>5135.135135</td>\n",
       "      <td>5167.7150</td>\n",
       "      <td>SENGKANG</td>\n",
       "      <td>5 ROOM</td>\n",
       "      <td>Premium Apartment</td>\n",
       "      <td>111.0</td>\n",
       "      <td>ANCHORVALE CRES</td>\n",
       "      <td>1.0</td>\n",
       "      <td>94.916667</td>\n",
       "      <td>2015</td>\n",
       "      <td>07 TO 09</td>\n",
       "      <td>68173</td>\n",
       "      <td>335B</td>\n",
       "      <td>2020-01-01</td>\n",
       "      <td>2020</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>850000.0</td>\n",
       "      <td>849683.0</td>\n",
       "      <td>6910.569106</td>\n",
       "      <td>6907.9927</td>\n",
       "      <td>BUKIT TIMAH</td>\n",
       "      <td>5 ROOM</td>\n",
       "      <td>Standard</td>\n",
       "      <td>123.0</td>\n",
       "      <td>FARRER RD</td>\n",
       "      <td>1.0</td>\n",
       "      <td>53.250000</td>\n",
       "      <td>1974</td>\n",
       "      <td>07 TO 09</td>\n",
       "      <td>100541</td>\n",
       "      <td>5</td>\n",
       "      <td>2020-01-01</td>\n",
       "      <td>2020</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>465000.0</td>\n",
       "      <td>466943.0</td>\n",
       "      <td>5054.347826</td>\n",
       "      <td>5075.4727</td>\n",
       "      <td>JURONG EAST</td>\n",
       "      <td>4 ROOM</td>\n",
       "      <td>Model A</td>\n",
       "      <td>92.0</td>\n",
       "      <td>TEBAN GDNS RD</td>\n",
       "      <td>1.0</td>\n",
       "      <td>92.666667</td>\n",
       "      <td>2013</td>\n",
       "      <td>31 TO 33</td>\n",
       "      <td>100825</td>\n",
       "      <td>21</td>\n",
       "      <td>2020-01-01</td>\n",
       "      <td>2020</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>420000.0</td>\n",
       "      <td>420164.0</td>\n",
       "      <td>4565.217391</td>\n",
       "      <td>4566.9976</td>\n",
       "      <td>SENGKANG</td>\n",
       "      <td>4 ROOM</td>\n",
       "      <td>Model A</td>\n",
       "      <td>92.0</td>\n",
       "      <td>FERNVALE RD</td>\n",
       "      <td>1.0</td>\n",
       "      <td>94.500000</td>\n",
       "      <td>2015</td>\n",
       "      <td>07 TO 09</td>\n",
       "      <td>68116</td>\n",
       "      <td>453B</td>\n",
       "      <td>2020-01-01</td>\n",
       "      <td>2020</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   resale_price  pred_resale_price    price_sqm  pred_price         town  \\\n",
       "0      426000.0           424631.0  4580.645161   4565.9233      PUNGGOL   \n",
       "1      600000.0           597799.0  5084.745763   5066.0900     CLEMENTI   \n",
       "2      433888.0           431828.0  3337.600000   3321.7534    WOODLANDS   \n",
       "3      375588.0           374900.0  5523.352941   5513.2334      PUNGGOL   \n",
       "4      428000.0           435935.0  4652.173913   4738.4280       YISHUN   \n",
       "5      710000.0           688264.0  4797.297297   4650.4340    SERANGOON   \n",
       "6      570000.0           573616.0  5135.135135   5167.7150     SENGKANG   \n",
       "7      850000.0           849683.0  6910.569106   6907.9927  BUKIT TIMAH   \n",
       "8      465000.0           466943.0  5054.347826   5075.4727  JURONG EAST   \n",
       "9      420000.0           420164.0  4565.217391   4566.9976     SENGKANG   \n",
       "\n",
       "   flat_type         flat_model  floor_area_sqm            street_name  month  \\\n",
       "0     4 ROOM  Premium Apartment            93.0        PUNGGOL CENTRAL    1.0   \n",
       "1     5 ROOM           Improved           118.0     CLEMENTI WEST ST 2    1.0   \n",
       "2     5 ROOM           Improved           130.0        WOODLANDS ST 31    1.0   \n",
       "3     3 ROOM  Premium Apartment            68.0          PUNGGOL DRIVE    1.0   \n",
       "4     4 ROOM            Model A            92.0           YISHUN AVE 9    1.0   \n",
       "5  EXECUTIVE         Maisonette           148.0  SERANGOON NORTH AVE 4    1.0   \n",
       "6     5 ROOM  Premium Apartment           111.0        ANCHORVALE CRES    1.0   \n",
       "7     5 ROOM           Standard           123.0              FARRER RD    1.0   \n",
       "8     4 ROOM            Model A            92.0          TEBAN GDNS RD    1.0   \n",
       "9     4 ROOM            Model A            92.0            FERNVALE RD    1.0   \n",
       "\n",
       "   remaining_lease  lease_commence_date storey_range      id block  \\\n",
       "0        89.500000                 2010     04 TO 06   67643  301C   \n",
       "1        60.083333                 1981     19 TO 21   66801   724   \n",
       "2        71.916667                 1993     01 TO 03  101799   301   \n",
       "3        95.166667                 2016     07 TO 09   67619  667A   \n",
       "4        94.750000                 2015     13 TO 15   69215  317A   \n",
       "5        71.750000                 1992     10 TO 12  101497   525   \n",
       "6        94.916667                 2015     07 TO 09   68173  335B   \n",
       "7        53.250000                 1974     07 TO 09  100541     5   \n",
       "8        92.666667                 2013     31 TO 33  100825    21   \n",
       "9        94.500000                 2015     07 TO 09   68116  453B   \n",
       "\n",
       "     reg_date  year  \n",
       "0  2020-01-01  2020  \n",
       "1  2020-01-01  2020  \n",
       "2  2020-01-01  2020  \n",
       "3  2020-01-01  2020  \n",
       "4  2020-01-01  2020  \n",
       "5  2020-01-01  2020  \n",
       "6  2020-01-01  2020  \n",
       "7  2020-01-01  2020  \n",
       "8  2020-01-01  2020  \n",
       "9  2020-01-01  2020  "
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_pred = pd.read_csv('./Data/dat_pred.csv')\n",
    "df_pred[0:10]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p><center><i><b>Table 4</b>: Comparison of Predicted vs Actual Resale Price of 10 Most Recent Transactions</i></center></p>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As we would want to select the model with the best predictive performance, **XGBoost** model is selected as the model of choice for deployment. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.4 Top Model Predictors"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this analysis, we will not delve deep into each of the predictors' effect and relationship with the target variable (resale price), as this is not the primary focus of the analysis. However, we will quickly look into some of the interesting trends and findings that might be of interested to the target users. \n",
    "\n",
    "Unsurprisingly, the **remaining lease** of a property and the **floor area** of a property are two of the most important determinants of the property's resale price (Figure 4). Properties with a longer **remaining lease** generally have a higher resale price, whereas properties with a larger **floor area** would generally see a reduction in their resale price by unit area (smaller units generally commands a higher per square meter price tag)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"./Images/Top_Predictors.png\">"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p><center><i><b>Figure 4</b>: Top 10 Most Important Features/ Predictors of XGBoost Model</i></center></p>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Interestingly, the **month** which the application for the transaction is recorded, regardless of the year, also have a high **F score** and is the third most important predictors in the model. This is followed by the **year** of the application for the transaction, but with a much reduced **F score**. The sum of the **F score** for the top 3 predictors accounts for more than 60\\% of the sum of **F score** for all predictors in the model.\n",
    "\n",
    "By looking at the resale price trend from January 2017 to January 2020, there seems to be a general downward trend from 2017 to 2020, with some degrees of fluctuations across the months (Figure 5). By separating the effect of the **year** of the transaction with the **month** of the transaction, we can see a clearer trend in both the impact of **year** and **month** on resale price (Figure 6). The number of restaurants within a 500m radius of the property also positively impact the resale price."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"./Images/Resale_Price_Trend.png\">"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p><center><i><b>Figure 5</b>: HDB Flat Resale Price (per sqm) from January 2017 to January 2020</i></center></p>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"./Images/Annual_Monthly_Price_Trend.png\">"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p><center><i><b>Figure 6</b>: HDB Flat Resale Price (per sqm) by Year and Month of Application of Transaction</i></center></p>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Analysis on Annual Trend**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Resale price have dipped sharply from 2017 to 2018, and continued to decline from 2018 to 2019. The relative high resale price in 2017 can be attributed to the 'en bloc' fever that started in 2016 and peaked in 2017. In March 2017, the Singapore Government have relaxed certain measures for property sellers such as reducing the amount of Seller's Stamp Duty<sup>[[4](#Ref4)]</sup>. These measures arguably increased the supply of HDB Resale Flats and drove down the resale price. \n",
    "\n",
    "In July 2018, the Singapore Government introduced more property cooling measures such as increasing the Additional Buyer Stamp Duty and reducing the Loan-To-Value limit, effectively resulting in buyer's required to pay more tax and getting less loan for property purchases<sup>[[5](#Ref5)]</sup>. These probably drove the property prices further into decline. The steadily increasing interest rates for home loans over the years arguably also cemented the declining trend for HDB Flat resale prices. \n",
    "\n",
    "Even though it seems that there is a slight recovery from 2019 to 2020, the number of transactions and the fluctuation of the error bar suggest that the trend can still go either ways. With the global market tensions and the recent COVID-19 outbreak, 2020 maybe another struggling year for the HDB Resale market.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Analysis on Monthly Trend**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Surprisingly, regardless of the year of transaction, there is a distinct cyclical pattern across the different months. Applications of transactions recorded in March seems to have the highest resale price, whereas resale prices are lowest in December. Currently we have yet to uncover any compelling reason behind this trend, but there might be a few potential confounders to take note of. \n",
    "\n",
    "As HDB Flats have a minimum occupancy period of 5 years (owners need to stay in the flat for at least 5 years before they can sell the flat, unless for some exceptional situations), December to March maybe a period where new and highly-sought after projects launch 5 years ago where made available to the resale market. And given the time from searching a new property by a buyer to the actual application of the transaction usually takes 2 to 6 months, we can deduce that there are more property buyers in the market from November to January.\n",
    "\n",
    "However, it is important to understand that property prices are very much driven by external factors and that these trends that were observed right now may not persist."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.5 Top 10 Most and Least Expensive Towns"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\"Location, location, location.\" have always been the mantra for property buyers and sellers. Undoubtly, which town the property is located will have an impact on its resale price. In the **HDB Flat Resale Data** from January 2017 to January 2020, all transactions recorded fall into 26 different towns across the city state. Naturally, which town has a bigger or lesser impact of the property price would be an interesting topic for property buyers and sellers.\n",
    "\n",
    "However, interpreting the relative impact of each town to the resale price from the **XGBoost** is difficult and time consuming. As such, we will extract the coefficients from the **Lasso Regression** model to compile a list of the most expensive towns down to the least expensive towns for HDB Resale Flats. Even though the **Lasso Regression** model is not the selected model and only explains 81\\% of the variance in the data, it is still a valid model that we can glean insights into the **relative impact** each town have towards the property resale price (not actual quantified impact).\n",
    "\n",
    "By arrange the coefficients of each town, we will display the top 10 most expensive towns and the top 10 least expensive town, with the assumption of a larger positive coefficient means being more expensive, while a larger negative coefficient means less expensive."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"./Images/Top_Towns.png\">"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p><center><i><b>Figure 7</b>: Top 10 Most and Least Expensive Towns for HDB Resale Flats</i></center></p>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Bukit Merah** and **Queenstown** takes the lead with the highest positive impact to resale price (per sqm) under an all-things-equal situation. It is then followed by **Kallang/Whampoa**, **Bishan** and **Toa Payoh**. Surprisingly, **Central Area** did not make it to the list, even with the common preception of it being the most expensive area to buy a property. Most of the towns in the expensive list are towns surrounding the **Central Area** (or Central Business District) of Singapore. The other cluster of towns are located Eastwards (**Marine Parade**, **Bedok** and **Geylang**). \n",
    "\n",
    "As for the least expensive towns, **Woodlands** tops the list followed by **Sengkang**, **Sembawang** and **Chua Chu Kang**. Out of the 10 towns that made the least expensive list, 7 are located at the North and North-West area of Singapore (additional to the towns mentioned above are **Yishun**, **Bukit Panjang**, and **Bukit Batok**)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4. Conclusion <a name=\"conclusion\"></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We managed to construct a model using **XGBoost** that achieved a **mean absolute error** of only **2.7%** of mean price, and 95\\% confidence interval margin of error of **+-S\\$406** per square meter area (if the model predicts a resale price of S\\\\$4,500, there is 95\\% chance that the actual resale price lie within the range of S\\\\$4,094 to S\\\\$4,906). The model also managed to explain **97\\%** of the variance in the dataset. \n",
    "\n",
    "The selected model will be integrated in an application for deployment to users, so that users are able to get an estimate of the HDB Flat resale price, quickly and cost-free."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5. References <a name=\"references\"></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[1] Singapore Housing Development Board (2020). *HDB Website*. Retrieved February 28, 2020, from https://www.hdb.gov.sg/cs/infoweb/about-us. <a name=\"Ref1\"></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[2] GovTech (2020). *Data.gov.sg Website*. Retrieved February 28, 2020 from https://data.gov.sg/group/infrastructure. <a name=\"Ref2\"></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[3] J.A.Swartz (2013). *Appraising with Regression*. Retrieved February 28, 2020 from http://www.workingre.com/appraising-regression/. <a name=\"Ref3\"></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[4] Monetary Authority of Singapore (2017). *Joint Press Release on Measures Relating to Residential Property*. Retrieved March 03, 2020, from [https://www.mas.gov.sg/news/media-releases/2017/joint-press-release-on-measures-relating-to-residential-property](https://www.mas.gov.sg/news/media-releases/2017/joint-press-release-on-measures-relating-to-residential-property)<a name=\"Ref4\"></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[5] Monetary Authority of Singapore (2018). *Raising Additional Buyer's Stamp Duty Rates and Tightening Loan-to-Value Limits to Promote a Stable and Sustainable Property Market*. Retrieved March 03, 2020, from [https://www.mas.gov.sg/news/media-releases/2018/raising-additional-buyers-stamp-duty-rates-and-tightening-loan-to-value-limits](https://www.mas.gov.sg/news/media-releases/2018/raising-additional-buyers-stamp-duty-rates-and-tightening-loan-to-value-limits) <a name=\"Ref5\"></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Link to Jupyter Notebook with full codes can be found [here](https://github.com/leegang1990/Coursera_Capstone/blob/master/Capstone%20Project%20Notebook.ipynb)."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}