activity_recognition.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "7a20018d-eeee-4426-8188-d22730c0c806",
   "metadata": {},
   "source": [
    "# Activity Recognition with Healthy Older People Using a Batteryless Wearable Sensor\n",
    "\n",
    "This notebook provides code for loading the [activity recognition dataset](https://archive.ics.uci.edu/dataset/427/activity+recognition+with+healthy+older+people+using+a+batteryless+wearable+sensor) into a suitable format for classification and sequence labelling. \n",
    "\n",
    "The code is divided into three sections:\n",
    "* Loading the data\n",
    "* Sequence labelling: processing the data into a suitable format for sequence labelling. \n",
    "* Classification: processing the data into a suitable format for use with an IID classifier.\n",
    "\n",
    "We recommend running all three sections, then using the variables produced by the code as required for sequence labelling and IID classification.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "abd005df-dbf6-459d-a56b-b4d5a312f684",
   "metadata": {},
   "source": [
    "## Loading the Data\n",
    "\n",
    "Run the following cells to load the data from disk. You will need to run code from the subsequent sections (or your own code) to get the data into a suitable format for classification and sequence labelling.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "313450fe-e849-4b29-81c2-3d145458680a",
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "\n",
    "data_path = \"activity_recognition_dataset/S1_Dataset\"\n",
    "\n",
    "files = os.listdir(data_path)\n",
    "print(files)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e3195079-d8f9-4b46-b06a-21f79e26003b",
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd  # use pandas to load data from CSV files\n",
    "\n",
    "combined_data = []\n",
    "\n",
    "for file in files:\n",
    "    \n",
    "    if file == \"README.txt\":\n",
    "        continue\n",
    "    \n",
    "    try:\n",
    "        # load a single sequence from the file\n",
    "        seq_dataframe = pd.read_csv(os.path.join(data_path, file), header=None, names=[\"time\", \"frontal acc\", \"vertical acc\", \"lateral acc\", \"antenna ID\", \"RSSI\", \"phase\", \"frequency\", \"label\"])\n",
    "        \n",
    "        # put the ID of the sequence into the dataframe as an extra column\n",
    "        seq_dataframe['seqID'] = file  # use filename as ID\n",
    "                \n",
    "        combined_data.append(seq_dataframe)  # put the data frame into a list\n",
    "    except:\n",
    "        print(f\"Could not load file {file}. Skipping.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e139f96d-b83c-4642-8f31-de687b68f02f",
   "metadata": {},
   "source": [
    "## Sequence Labelling\n",
    "\n",
    "The following cell creates two variables that you can use for sequence labelling:\n",
    "* X_by_seq is a list of 2-D numpy arrays. Each numpy array in this list corresponds to on data sequence and contains the input feature values for that sequence.\n",
    "* y_by_seq is a list of 1-D numpy arrays, where each array contains the target class labels for each sequence. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1854a0cc-9cfa-4a28-aba2-9e8e45f86b55",
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "\n",
    "# the pandas dataframe stores data in a table with headers\n",
    "input_cols = [\"frontal acc\", \"vertical acc\", \"lateral acc\", \"RSSI\", \"phase\", \"frequency\"]  # column headers for the input features\n",
    "output_col = \"label\"  # column header for the output label\n",
    "\n",
    "# get the relevant columns from the pandas dataframes and convert to numpy arrays\n",
    "X_by_seq = []  # store a list of numpy arrays containing the input features for each sequence\n",
    "y_by_seq = []  # store a list of 1-D numpy arrays containing the target activity labels for each sequence\n",
    "\n",
    "for seq_table in combined_data:\n",
    "    X_by_seq.append(seq_table[input_cols].values)\n",
    "    y_by_seq.append(seq_table[output_col].values - 1)    # subtract one from the label so that labels start from 0\n",
    "    \n",
    "n_states = np.unique(np.concatenate(y_by_seq)).size  # how many classes/states are there?"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e232b920-8ea0-4be5-8c50-1646d15e701f",
   "metadata": {},
   "source": [
    "### Variables for sequence labelling\n",
    "\n",
    "The cell below produces the data you need for sequence labelling... You should be able to work with these variables directly."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a08cc922-d89b-4d20-8bac-a7bbdd71b2f8",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "# create train/test split. Sequences are kept complete.\n",
    "X_by_seq_tr, X_by_seq_test, y_by_seq_tr, y_by_seq_test = train_test_split(X_by_seq, y_by_seq, test_size=0.2, random_state=21)  # You may wish to make further splits of the data or to modify this split."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1ef1a36f-2cbf-4f6c-90b0-25d4c88e8ff9",
   "metadata": {},
   "source": [
    "You may find the following code useful for creating a sequence labelling model:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6147f83f-64e3-4ba3-aea1-271f3319ec78",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Record which observations occur given each state (activity label)\n",
    "X_given_y = [[] for _ in range(n_states)]  # empty list where we will record the observations that occur given each activity label\n",
    "\n",
    "for s, X_seq in enumerate(X_by_seq_tr):\n",
    "    for i in range(X_seq.shape[0]):\n",
    "        state_i = y_by_seq_tr[s][i]\n",
    "        X_given_y[state_i].append(X_seq[i, :][None, :])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7e4dd756-a7f6-40f9-b83e-d075f1139985",
   "metadata": {},
   "outputs": [],
   "source": [
    "from hmmlearn import hmm  # We recommend using this class if builing an HMM\n",
    "\n",
    "# Record the mean feature values for observations in each state\n",
    "means = np.zeros((n_states, len(input_cols)))\n",
    "\n",
    "# Record the variance of feature values for observations in each state\n",
    "diagonal_covars = np.zeros((n_states, len(input_cols)))\n",
    "\n",
    "for state in range(n_states):\n",
    "    means[state] = np.mean(X_given_y[state], axis=0)\n",
    "    diagonal_covars[state, :] = np.var(X_given_y[state], axis=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "344abaad-2e8f-4575-92cf-1c0055c47870",
   "metadata": {},
   "source": [
    "## IID Classification\n",
    "\n",
    "In this code, we take the X and y lists produced for sequence labelling, and concatenate the data points for all sequences. This produces a single set of training data and a single set of test data, which are not divided into separate sequences.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e88a2b2c-58a6-4cf1-afd4-d91505f95ebf",
   "metadata": {},
   "outputs": [],
   "source": [
    "X_tr = np.concatenate(X_by_seq_tr, axis=0)  # combine features into one matrix -- use this as input features for training a classifier\n",
    "y_tr = np.concatenate(y_by_seq_tr)  # combine target labels into one list -- use this as target labels for training a classifier"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "60176b1d-2cfa-48c5-a219-26ac39660bc4",
   "metadata": {},
   "outputs": [],
   "source": [
    "X_test = np.concatenate(X_by_seq_test, axis=0)  # combine features into one matrix -- use this as input features for testing a classifier\n",
    "y_test = np.concatenate(y_by_seq_test)  # combine target labels into one list -- use this as target labels for evaluating a classifier"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "ml_labs",
   "language": "python",
   "name": "ml_labs"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}