Classification_serial.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# The problem: classification of hand-written digits\n",
    "\n",
    "The folder `data/` contains 3 datasets: \n",
    "\n",
    "- `train_images.npy` contains 2000 images of hand-written digits (28x28 pixels) that have been previously identified by humans\n",
    "- `train_labels.npy` contains the 2000 corresponding labels (integers from 0 to 9)\n",
    "- `test_images.npy` contains 1000 **unlabeled** images of hand-written digits\n",
    "\n",
    "**The aim is to automatically attribute the right label to the unlabeled digits.**\n",
    "\n",
    "\n",
    "Let us have a look at the images and corresponding labels:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Load the data\n",
    "train_images = np.load('./data/train_images.npy')\n",
    "train_labels = np.load('./data/train_labels.npy')\n",
    "test_images = np.load('./data/test_images.npy')\n",
    "\n",
    "# Define function to have a look at the data\n",
    "def show_random_digit( images, labels=None ):\n",
    "    \"\"\"\"Show a random image out of `images`, \n",
    "    with the corresponding label if available\"\"\"\n",
    "    i = np.random.randint(len(images))\n",
    "    image = images[i].reshape((28, 28))\n",
    "    plt.imshow( image, cmap='Greys' )\n",
    "    if labels is not None:\n",
    "        plt.title('Label: %d' %labels[i])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let us first have a look at the images from the training set (labeled images)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "show_random_digit( train_images, train_labels )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also have a look at the images from the test set (unlabeled images)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "show_random_digit( test_images )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "----\n",
    "\n",
    "# Creating a python script that predicts the labels\n",
    "\n",
    "In order to predict the labels, we will use the **nearest neighbor method**:  \n",
    "For each of the images in the **test set**, we will search for the **most similar** image in the **training set**, and return the corresponding label.\n",
    "\n",
    "Since the actual code for the nearest neighbor method is not of interest here, it has been **already implemented** in the folder `classification/` and we will only use it here as a **function call**.\n",
    "\n",
    "Note: The implementation in `classification/` is naive and inefficient. `scikit-learn` has much more efficient implementation, but it has built-in parallel functionalities, which would not be suitable for this tutorial."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from classification import nearest_neighbor_prediction\n",
    "nearest_neighbor_prediction?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In addition, instead of performing the calculation **directly in the Jupyter notebook**, we will write a **Python script** that performs the calculation, and we will **execute** this script from the terminal.  \n",
    "NB: This may seem odd, but it will make more sense when we compare this code with the corresponding `mpi4py` code.\n",
    "\n",
    "Below, the line magic `%%file` allows us to write a Python script and save it as `serial_script.py`, without having to leave the notebook and open a text editor."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "%%file serial_script.py\n",
    "\n",
    "import numpy as np\n",
    "from classification import nearest_neighbor_prediction\n",
    "\n",
    "# Load data\n",
    "train_images = np.load('./data/train_images.npy')\n",
    "train_labels = np.load('./data/train_labels.npy')\n",
    "test_images = np.load('./data/test_images.npy')\n",
    "\n",
    "# Predict the test labels and save it to a file\n",
    "test_labels = nearest_neighbor_prediction( test_images, train_images, train_labels )\n",
    "np.save('data/test_labels_serial.npy', test_labels )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the line below, the character `!` allows to run the command line as if we were in a terminal.  \n",
    "The line magic `%%time` allows to time the execution."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "%%time \n",
    "! python serial_script.py"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The execution takes a substantial amount of time: the nearest neighbor method is simple but computationally expensive."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "-----\n",
    "\n",
    "# Checking the results \n",
    "\n",
    "The script saved the predicted labels in `test_labels_serial.npy`.  \n",
    "Let us have a look at it, and check that the predicted labels match the images."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "test_labels = np.load('./data/test_labels_serial.npy')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "show_random_digit( test_images, test_labels )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "-----\n",
    "# Towards a parallel implementation\n",
    "\n",
    "Up to now, the prediction of the labels was done on a **single core**.  \n",
    "However, since the single-core execution is time consuming, it is desirable to execute the code on **several cores**.\n",
    "\n",
    "Because the prediction of each label is **independent**, the parallelization is conceptually trivial: the set of 1000 unlabeled images should be **split** between the different cores, so that **each core** only takes care of a **fraction** of the unlabeled images.\n",
    "\n",
    "Let us see how to implement this with `mpi4py` [here](./Classification_mpi4py.ipynb)."
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}