MicaSense Image Processing Tutorial 2.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Tutorial 2 - MicaSense library\n",
    "\n",
    "This tutorial assumes you have gone through the [basic setup](./Micasense Image Processing Setup.html) and builds on the basic radiance, irradiance, and reflectance concepts and code covered in the [first tutorial](./MicaSense Image Processing Tutorial 1.html). \n",
    "\n",
    "In this tutorial, we will cover usage of the MicaSense python library to access images and groups of images.  Most of the processing details are hidden away in the library, but the library code is open and available in the git repository. \n",
    "\n",
    "# Library Components\n",
    "\n",
    "In the first tutorial, we introduced `micasense.utils` which provided some helper functions for single image manipulation, and `micasense.plotutils` which provided some plotting helpers.\n",
    "\n",
    "For this second tutorial, we are going to introduce the usage of the included micasense libraries for opening, converting, and displaying images. This will allow us to discuss and visualize results at a high level, while the underlying source code is available for those interested in the implementation details.  In some cases, the libraries themselves may be enough to implement a custom workflow without the need to re-implement or translate the code to another system or language.\n",
    "\n",
    "The library code provides some basic classes to manage image data.  At the highest level is the `ImageSet`, which is able to load a list of files or recursively search a whole directory into data structures which are easy to access and manipulate.  `ImageSet`s are made up of `Capture`s, which hold the set of (usually 5) images as they are simultaneously gathered by the RedEdge camera.  Within `Capture`s are `Image`s, which hold a single image file and allow easy access to the image metadata.  The `Image` class also provides the ability to extract metadata from individual images and to convert individual images in similar ways to those described in the first tutorial. \n",
    "\n",
    "For the rest of this article, we will look at each of the objects available starting with the single `Image` object, and work our way up to the whole `ImageSet`.  Each section in this article is standalone, and can be copied into another workbook or edited in place to explore more of the functions associated with that object."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## micasense.Image\n",
    "\n",
    "An image is the lowest level object. It represents the data in a single tiff file as taken by the camera.  `Image` objects expose a set of data retrieval methods which provide access to raw, radiance, and reflectance corrected images, and to undistort any of those images. Note that when retrieving image data from an `Image` object, the data is stored internally in the object, increasing the object's memory footprint. If operating on a large number of images, it may be necessary to release this data memory after each image is processed to limit the program memory footprint. This can be done by calling the `Image.clear_image_data()` method."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import micasense.image as image\n",
    "%matplotlib inline\n",
    "\n",
    "image_path = os.path.join('.','data','0000SET','000','IMG_0000_1.tif')\n",
    "img = image.Image(image_path)\n",
    "img.plot_raw();"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Accessing `Image` Metadata\n",
    "\n",
    "Metadata for each image is available in the `Image.meta` parameter.  This object is a `micasense.Metadata` object and can be accessed directly for image specific metadata extraction. Below, we print the same metadata values as we did in Tutorial #1, but using direct access to the `Metadata` object parameters.\n",
    "\n",
    "A notebook for experimenting with the `Image` class can be found [here](Images.html)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print('{0} {1} firmware version: {2}'.format(img.meta.camera_make(),\n",
    "                                             img.meta.camera_model(), \n",
    "                                             img.meta.firmware_version()))\n",
    "print('Exposure Time: {0} seconds'.format(img.meta.exposure()))\n",
    "print('Imager Gain: {0}'.format(img.meta.gain()))\n",
    "print('Size: {0}x{1} pixels'.format(img.meta.image_size()[0],\n",
    "                                    img.meta.image_size()[1]))\n",
    "print('Band Name: {0}'.format(img.meta.band_name()))\n",
    "print('Center Wavelength: {0} nm'.format(img.meta.center_wavelength()))\n",
    "print('Bandwidth: {0} nm'.format(img.meta.bandwidth()))\n",
    "print('Capture ID: {0}'.format(img.meta.capture_id()))\n",
    "print('Flight ID: {0}'.format(img.meta.flight_id()))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## micasense.Capture\n",
    "\n",
    "The `Capture` class is a container for `Image`s which allows access to metadata common to the group of images. The internal `Image` objects are accessible via the `capture.images` properties, and images in this list are kept sorted by the `band` property.  Data which is different for each image can be accessed through composite methods, such as the `capture.dls_irradiance()` method, which returns a list of irradiances in band order."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os, glob\n",
    "import micasense.capture as capture\n",
    "\n",
    "images_path = os.path.join('.','data','0000SET','000')\n",
    "image_names = glob.glob(os.path.join(images_path,'IMG_0000_*.tif'))\n",
    "cap = capture.Capture.from_filelist(image_names)\n",
    "cap.plot_radiance();"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Acessing `Capture` metadata\n",
    "\n",
    "Metadata which is common to all captures can be accessed via methods on the `Capture` object.  Metadata which varies between the images of the capture, such as DLS information, is available as lists accessed from the capture object. \n",
    "\n",
    "<div class=\"alert alert-info\">\n",
    "  <strong>Note:</strong> The lists returned from metadata access on the `Capture` object are returned in `band_index` order.  All images within a capture are sorted by the image `band_index`, and all lists adhere to this ordering.  This ordering is consistent with the number at the end of each filename of a RedEdge image.\n",
    "</div>\n",
    "\n",
    "Below we plot the raw and tilt compensated DLS irradiance by center wavelength and by band name.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "\n",
    "print(cap.band_names())\n",
    "fig = plt.figure(figsize=(14,6))\n",
    "plt.subplot(1,2,1)\n",
    "plt.scatter(cap.center_wavelengths(), cap.dls_irradiance())\n",
    "plt.ylabel('Irradiance $(W/m^2/nm)$')\n",
    "plt.xlabel('Center Wavelength (nm)')\n",
    "plt.subplot(1,2,2)\n",
    "plt.scatter(cap.band_names(), [img.meta.exposure() for img in cap.images])\n",
    "plt.xlabel('Band Names')\n",
    "plt.ylim([0,2.5e-3])\n",
    "plt.ylabel('Exposure Time (s)')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A notebook for experimenting with the `Capture` class can be found [here](Captures.html)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## micasense.Panel\n",
    "\n",
    "The `Panel` class is a helper class which can automatically extract panel information from MicaSense calibrated reflectance panels by finding the QR code within an image and using the QR Code location and orientation information to find the lambertian panel area.  The class then allows extraction of statistics from the panel area such as mean raw values, mean radiance, standard deviation, and the number of saturated pixels in the panel region. The panel object can be included standalone, or used within the context of a `Capture` object.\n",
    "\n",
    "<div class=\"alert alert-info\">\n",
    "  <strong>Note:</strong> For the automatic panel QR code finding functions of the library to work, zbar and it's python bindings must be installed. We have made every effort to ensure this fails gracefully if zbar isn't available. Unfortunately zbar is only available using Python 2.7, not Python 3.  If you're using Python 3.x, the code available in '/micasense/panel.py' shows how to find QR codes in images and to find the panel area from the QR location.  We're currently looking for Python QR code finding options that work across platforms and Python versions, let us know if you have one that supports location!\n",
    "</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "import os, glob\n",
    "import micasense.image as image\n",
    "import micasense.panel as panel\n",
    "\n",
    "image_path = os.path.join('.','data','0000SET','000','IMG_0000_1.tif')\n",
    "img = image.Image(image_path)\n",
    "# panelCorners - if we dont have zbar installed to scan the QR codes, detect panel manually and \n",
    "panelCorners = [[[809,613],[648,615],[646,454],[808,452]],\n",
    "                [[772,623],[613,625],[610,464],[770,462]],\n",
    "                [[771,651],[611,653],[610,492],[770,490]],\n",
    "                [[829,658],[668,659],[668,496],[829,496]],\n",
    "                [[807,632],[648,634],[645,473],[805,471]]]\n",
    "\n",
    "pnl = panel.Panel(img,panelCorners = panelCorners[0])\n",
    "print(\"Panel found: {}\".format(pnl.panel_detected()))\n",
    "print(\"Panel serial: {}\".format(pnl.serial))\n",
    "print(\"QR Code Corners:\\n{}\".format(pnl.qr_corners()))\n",
    "mean, std, count, saturated_count = pnl.raw()\n",
    "print(\"Panel mean raw pixel value: {}\".format(mean))\n",
    "print(\"Panel raw pixel standard deviation: {}\".format(std))\n",
    "print(\"Panel region pixel count: {}\".format(count))\n",
    "print(\"Panel region saturated pixel count: {}\".format(count))\n",
    "\n",
    "pnl.plot();"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A notebook for experimenting with the `Panel` class can be found [here](Panels.html)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## micasense.ImageSet\n",
    "\n",
    "An `ImageSet` contains a group of `Capture`s. The captures can be loaded from image object, from a list of files, or by recursively searching a directory for images.\n",
    "\n",
    "Loading an `ImageSet` can be a time consuming process.  It uses python multithreading under the hood to maximize cpu usage on multi-core machines."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from ipywidgets import FloatProgress\n",
    "from IPython.display import display\n",
    "f = FloatProgress(min=0, max=1)\n",
    "display(f)\n",
    "def update_f(val):\n",
    "    f.value=val\n",
    "\n",
    "import micasense.imageset as imageset\n",
    "import os\n",
    "images_dir = os.path.join('.','data','0000SET')\n",
    "\n",
    "imgset = imageset.ImageSet.from_directory(images_dir, progress_callback=update_f)\n",
    "\n",
    "for cap in imgset.captures:\n",
    "    print (\"Opened Capture {} with bands {}\".format(cap.uuid,[str(band) for band in cap.band_names()]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Extended ImageSet examples\n",
    "\n",
    "A large group of images captured over a central California orchard are available for [download here](https://s3-us-west-2.amazonaws.com/sample.micasense.com/imageprocessing/RedEdgeImageSet.zip).\n",
    "\n",
    "With this set extracted to a working folder, the [extended ImageSet example](./ImageSets.html) notebook provides more usages of ImageSet data.\n",
    "\n",
    "# Conclusion\n",
    "\n",
    "In this tutorial, we have introduced the MicaSense library and provided some examples of opening Images, Captures, and ImageSets, as well as detecting and extracting panel information from images.\n",
    "\n",
    "The next tutorial covers basic usage of DLS information, and is available [here](./MicaSense%20Image%20Processing%20Tutorial%203.html)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "Copyright (c) 2017-2019 MicaSense, Inc.  For licensing information see the [project git repository](https://github.com/micasense/imageprocessing)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.1"
  },
  "toc": {
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": "block",
   "toc_window_display": true
  },
  "varInspector": {
   "cols": {
    "lenName": 16,
    "lenType": 16,
    "lenVar": 40
   },
   "kernels_config": {
    "python": {
     "delete_cmd_postfix": "",
     "delete_cmd_prefix": "del ",
     "library": "var_list.py",
     "varRefreshCmd": "print(var_dic_list())"
    },
    "r": {
     "delete_cmd_postfix": ") ",
     "delete_cmd_prefix": "rm(",
     "library": "var_list.r",
     "varRefreshCmd": "cat(var_dic_list()) "
    }
   },
   "types_to_exclude": [
    "module",
    "function",
    "builtin_function_or_method",
    "instance",
    "_Feature"
   ],
   "window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}