module_2.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Lecture: The (Py)Tesseract Library"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "B0P4ehjER8bq"
   },
   "outputs": [],
   "source": [
    "# We're going to start experimenting with tesseract using just a simple image of nice clean text.\n",
    "# Lets first import Image from PIL and display the image text.png.\n",
    "from PIL import Image\n",
    "\n",
    "image = Image.open(\"text.png\")\n",
    "display(image)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "4w4k52VwR8bu"
   },
   "outputs": [],
   "source": [
    "# Great, we have a base image of some big clear text\n",
    "# Lets import pytesseract and use the dir() fundtion to get a sense of what might be some interesting\n",
    "# functions to play with\n",
    "import pytesseract\n",
    "dir(pytesseract)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "dg9KlgaxR8b6",
    "outputId": "ac573a44-d53c-41f6-f566-9a203d94901d"
   },
   "outputs": [],
   "source": [
    "# It looks like there are just a handful of interesting functions, and I think image_to_string\n",
    "# is probably our best bet. Lets use the help() function to interrogate this a bit more \n",
    "help(pytesseract.image_to_string)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "B_SyR7QfR8b_",
    "outputId": "7cd8e9db-5311-4d7b-acc4-247c17b4ce82"
   },
   "outputs": [],
   "source": [
    "# So this function takes an image as the first parameter, then there are a bunch of optional parameters,\n",
    "# and it will return the results of the OCR. I think it's worth comparing this documentation string\n",
    "# with the documentation we were receiving from the PILLOW module. Lets run the help command on the \n",
    "# Image resize function()\n",
    "help(Image.Image.resize)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Notice how the PILLOW function has a bit more information in it. First it's using a specific format\n",
    "# called reStructuredText, which is similar in intent to document markups such as HTML, the language of\n",
    "# the web. The intent is to embed semantics in the documentation itself. For instance, in the resize()\n",
    "# function we see the words \"param size\" with colons surrounding it. This allows documentation engines\n",
    "# which create web docs from source code to link the parameter to the extended docs about that parameter.\n",
    "# In this case the extended docs tell us that the size should be passed as a tuple of width and height.\n",
    "# Notice how the docs for image_to_string, for instance, indicate that there is a \"lang\" parameter we can\n",
    "# use, but then fail to say anything about what that parameter is for or what its format is.\n",
    "#\n",
    "# What this really means is that we need to dig deeper. Here's a quick hack if you want to look at the\n",
    "# source code of a function -- you can use the inspect getsource() command and print the results\n",
    "import inspect\n",
    "src = inspect.getsource(pytesseract.image_to_string)\n",
    "print(src)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# There's actually another way in jupyter, and that's to append *two* question marks to the end of\n",
    "# a given function or module. Other editors have similar features, and is a great reason to use a \n",
    "# software development environment\n",
    "pytesseract.image_to_string??"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# We can see from the source code that there really isn't much more information about what the parameters\n",
    "# are for this image_to_string function. This is because underneath the pytesseract library is calling a C++\n",
    "# library which does all of the hard work, and the author just passes through all of the calls to the \n",
    "# underlying tesseract executable. This is a common issue when working with python libraries, and it means\n",
    "# we need to do some web sleuthing in order to understand how we can interact with tesseract.\n",
    "#\n",
    "# In a case like this I just googled \"tesseract command line parameters\" and the first hit was what I was\n",
    "# looking for, here's the URL: https://github.com/tesseract-ocr/tesseract/wiki/Command-Line-Usage\n",
    "#\n",
    "# This goes to a wiki page which describes how to call the tesseract executable, and as we read down we see\n",
    "# that we can actually have tesseract use multiple languages in its detection, such as English and Hindi, by\n",
    "# passing them in as \"eng+hin\". Very cool. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# One last thing to mention - the image_to_string() function takes in an \"image\", but the docs don't\n",
    "# really describe what this image is underneath. Is it a string to an image file? A PILLOW image?\n",
    "# Something else?\n",
    "#\n",
    "# Again we have to sleuth (and/or experiment) to understand what we should do. If we look at the source\n",
    "# code for the pytesseract library, we see that there is a function called run_and_get_output(). Here's\n",
    "# a link to that function on the author's github account:\n",
    "# https://github.com/madmaze/pytesseract/blob/d1596f7f59a517ad814b7d810ccdef7d33763221/src/pytesseract.py#L199\n",
    "#\n",
    "# In this function we see that one of the first things which happens is the image is saved through\n",
    "# the save_image() function. Here's that line of code:\n",
    "# https://github.com/madmaze/pytesseract/blob/d1596f7f59a517ad814b7d810ccdef7d33763221/src/pytesseract.py#L116\n",
    "#\n",
    "# And we see there that another function is called, prepare(image), which actually loads the image as a\n",
    "# PILLOW image file. So yes, sending a PIL image file is appropriate use for this function! It sure would\n",
    "# have been useful for the author to have included this information in reStructuredText to help us not have\n",
    "# to dig through the implementation. But, this is an open source project -- maybe you would like to contribute\n",
    "# back better documentation?\n",
    "#\n",
    "# Hint: The doc line we needed was :param image: A PIL Image.Image file or an ndarray of bytes\n",
    "#\n",
    "# In the end, we often don't do this full level of investigation, and we just experiment and try things. It\n",
    "# seems likely that a PIL Image.Image would work, given how well known PIL is in the python world. But still,\n",
    "# as you explore and use different libraries you'll see a breadth of different documentation norms, so it's\n",
    "# useful to know how to explore the source code. And now that you're at the end of this course, you've got\n",
    "# the skills to do so!\n",
    "#\n",
    "# Ok, lets try and run tesseract on this image\n",
    "text = pytesseract.image_to_string(image)\n",
    "print(text)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Looks great! We see that the output includes new line characters, and faithfully represents the text\n",
    "# but doesn't include any special formatting. Lets go on and look at something with a bit more nuance to it."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## More Tesseract"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# In the previous example, we were using a clear, unambiguous image for conversion. Sometimes there will \n",
    "# be noise in images you want to OCR, making it difficult to extract the text. Luckily, there are \n",
    "# techniques we can use to increase the efficacy of OCR with pytesseract and Pillow.\n",
    "#\n",
    "# Let's use a different image this time, with the same text as before but with added noise in the picture. \n",
    "# We can view this image using the following code. \n",
    "from PIL import Image\n",
    "img = Image.open(\"readonly/Noisy_OCR.PNG\")\n",
    "display(img)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# As you can see, this image had shapes of different opacities behind the text, which can confuse  \n",
    "# the tesseract engine. Let's see if OCR will work on this noisy image\n",
    "import pytesseract\n",
    "text = pytesseract.image_to_string(Image.open(\"readonly/Noisy_OCR.PNG\"))\n",
    "print(text)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# This is a bit surprising given how nicely tesseract worked previously! Let's experiment on the image \n",
    "# using techniqes that will allow for more effective image analysis. First up, lets change the size of\n",
    "# the image"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# First we will import PIL\n",
    "import PIL \n",
    "# Then set the base width of our image\n",
    "basewidth = 600 \n",
    "# Now lets open it\n",
    "img = Image.open(\"readonly/Noisy_OCR.PNG\")\n",
    "# We want to get the correct aspect ratio, so we can do this by taking the base width and dividing\n",
    "# it by the actual width of the image\n",
    "wpercent = (basewidth / float(img.size[0]))\n",
    "# With that ratio we can just get the appropriate height of the image.\n",
    "hsize = int((float(img.size[1]) * float(wpercent)))\n",
    "# Finally, lets resize the image. antialiasing is a specific way of resizing lines to try and make them \n",
    "# appear smooth\n",
    "img = img.resize((basewidth, hsize), PIL.Image.ANTIALIAS)\n",
    "# Now lets save this to a file\n",
    "img.save('resized_nois.png') # save the image as a jpg\n",
    "# And finally, lets display it\n",
    "display(img)\n",
    "# and run OCR\n",
    "text = pytesseract.image_to_string(Image.open('resized_nois.png')) \n",
    "print(text)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# hrm, no improvement for resizing the image. Let's convert the image to greyscale. Converting images \n",
    "# can be done in many different ways. If we poke around in the PILLOW documentation we find that one of\n",
    "# the easiest ways to do this is to use the convert() function and pass in the string 'L'\n",
    "img = Image.open('readonly/Noisy_OCR.PNG')\n",
    "img = img.convert('L')\n",
    "# Now lets save that image\n",
    "img.save('greyscale_noise.jpg')\n",
    "# And run OCR on the greyscale image\n",
    "text = pytesseract.image_to_string(Image.open('greyscale_noise.jpg')) \n",
    "print(text)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Wow, that worked really well! If we look at the help documentation using the help function\n",
    "# as in help(img.convert) we see that the conversion mechanism is the ITU-R 601-2 luma transform.\n",
    "# There's more information about this out there, but this method essentially takes a three channel image,\n",
    "# where there is information for the amount of red, green, and blue (R, G, and B), and reduces it\n",
    "# to a single channel to represent luminosity. This method actually comes from how standard\n",
    "# definition television sets encoded color onto black and while images. If you get really interested\n",
    "# in image manipulation and recognition, learning about color spaces and how we represent color, both\n",
    "# computationally and through human perception, is really an interesting field."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Even though we have now the complete text of the image, there are a few other techniques\n",
    "# we could use to help improve OCR detection in the event that the above two don't help.\n",
    "# The next approach I would use is called binarization, which means to separate into two\n",
    "# distinct parts - in this case, black and white. Binarization is enacted through a process \n",
    "# called thresholding. If a pixel value is greater than a threshold value, it will be converted\n",
    "# to a black pixel; if it is lower than the threshold it will be converted to a white pixel. \n",
    "# This process eliminates noise in the OCR process allowing greater image recognition accuracy. \n",
    "# With Pillow, this process is straightforward.\n",
    "# Lets open the noisy impage and convert it using binarization\n",
    "img = Image.open('readonly/Noisy_OCR.PNG').convert('1')\n",
    "# Now lets save and display that image\n",
    "img.save('black_white_noise.jpg')\n",
    "display(img)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# So, that was a bit magical, and really required a fine reading of the docs to figure out\n",
    "# that the number \"1\" is a string parameter to the convert function actually does the binarization.\n",
    "# But you actually have all of the skills you need to write this functionality yourself.\n",
    "# Lets walk through an example. First, lets define a function called binarize, which takes in\n",
    "# an image and a threshold value:\n",
    "def binarize(image_to_transform, threshold):\n",
    "    # now, lets convert that image to a single greyscale image using convert()\n",
    "    output_image=image_to_transform.convert(\"L\")\n",
    "    # the threshold value is usually provided as a number between 0 and 255, which\n",
    "    # is the number of bits in a byte.\n",
    "    # the algorithm for the binarization is pretty simple, go through every pixel in the\n",
    "    # image and, if it's greater than the threshold, turn it all the way up (255), and\n",
    "    # if it's lower than the threshold, turn it all the way down (0).\n",
    "    # so lets write this in code. First, we need to iterate over all of the pixels in the\n",
    "    # image we want to work with\n",
    "    for x in range(output_image.width):\n",
    "        for y in range(output_image.height):\n",
    "            # for the given pixel at w,h, lets check its value against the threshold\n",
    "            if output_image.getpixel((x,y))< threshold: #note that the first parameter is actually a tuple object\n",
    "                # lets set this to zero\n",
    "                output_image.putpixel( (x,y), 0 )\n",
    "            else:\n",
    "                # otherwise lets set this to 255\n",
    "                output_image.putpixel( (x,y), 255 )\n",
    "    #now we just return the new image\n",
    "    return output_image\n",
    "\n",
    "# lets test this function over a range of different thresholds. Remember that you can use\n",
    "# the range() function to generate a list of numbers at different step sizes. range() is called\n",
    "# with a start, a stop, and a step size. So lets try range(0, 257, 64), which should generate 5\n",
    "# images of different threshold values\n",
    "for thresh in range(0,257,64):\n",
    "    print(\"Trying with threshold \" + str(thresh))\n",
    "    # Lets display the binarized image inline\n",
    "    display(binarize(Image.open('readonly/Noisy_OCR.PNG'), thresh))\n",
    "    # And lets use tesseract on it. It's inefficient to binarize it twice but this is just for\n",
    "    # a demo\n",
    "    print(pytesseract.image_to_string(binarize(Image.open('readonly/Noisy_OCR.PNG'), thresh)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# We can see from this that a threshold of 0 essentially turns everything white,\n",
    "# that the text becomes more bold as we move towards a higher threshold, and that\n",
    "# the shapes, which have a filled in grey color, become more evident at higher\n",
    "# thresholds. In the next lecture we'll look a bit more at some of the challenges\n",
    "# you can expect when doing OCR on real data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Tesseract and Photographs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Lets try a new example and bring together some of the things we have learned.\n",
    "# Here's an image of a storefront, lets load it and try and get the name of the\n",
    "# store out of the image\n",
    "from PIL import Image\n",
    "import pytesseract\n",
    "# Lets read in the storefront image I've loaded into the course and display it\n",
    "image=Image.open('readonly/storefront.jpg')\n",
    "display(image)\n",
    "# Finally, lets try and run tesseract on that image and see what the results are\n",
    "pytesseract.image_to_string(image)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# We see at the very bottom there is just an empty string. Tesseract is unable to take\n",
    "# this image and pull out the name. But we learned how to crop the images in the\n",
    "# last set of lectures, so lets try and help Tesseract by cropping out certain pieces.\n",
    "#\n",
    "# First, lets set the bounding box. In this image the store name is in a box\n",
    "# bounded by (315, 170, 700, 270)\n",
    "bounding_box=(315, 170, 700, 270)\n",
    "\n",
    "# Now lets crop the image\n",
    "title_image=image.crop(bounding_box)\n",
    "\n",
    "# Now lets display it and pull out the text\n",
    "display(title_image)\n",
    "pytesseract.image_to_string(title_image)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Great, we see how with a bit of a problem reduction we can make that work. So now we have\n",
    "# been able to take an image, preprocess it where we expect to see text, and turn that text\n",
    "# into a string that python can understand.\n",
    "#\n",
    "# If you look back up at the image though, you'll see there is a small sign inside of the\n",
    "# shop that also has the shop name on it. I wonder if we're able to recognize the text on \n",
    "# that sign? Let's give it a try.\n",
    "#\n",
    "# First, we need to determine a bounding box for that sign. I'm going to show you a short-cut\n",
    "# to make this easier in an optional video in this module, but for now lets just use the bounding\n",
    "# box I decided on\n",
    "bounding_box=(900, 420, 940, 445)\n",
    "\n",
    "# Now, lets crop the image\n",
    "little_sign=image.crop((900, 420, 940, 445))\n",
    "display(little_sign)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# All right, that is a little sign! OCR works better with higher resolution images, so\n",
    "# lets increase the size of this image by using the pillow resize() function\n",
    "# Lets set the width and height equal to ten times the size it is now in a (w,h) tuple\n",
    "new_size=(little_sign.width*10,little_sign.height*10)\n",
    "\n",
    "# Now lets check the docs for resize()\n",
    "help(little_sign.resize)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# We can see that there are a number of different filters for resizing the image. The\n",
    "# default is Image.NEAREST. Lets see what that looks like\n",
    "display(little_sign.resize( new_size, Image.NEAREST))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# I think we should be able to find something better. I can read it, but it looks\n",
    "# really pixelated. Lets see what all the different resize options look like\n",
    "options=[Image.NEAREST, Image.BOX, Image.BILINEAR, Image.HAMMING, Image.BICUBIC, Image.LANCZOS]\n",
    "for option in options:\n",
    "    # lets print the option name\n",
    "    print(option)\n",
    "    # lets display what this option looks like on our little sign\n",
    "    display(little_sign.resize( new_size, option))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# From this we can notice two things. First, when we print out one of the resampling\n",
    "# values it actually just prints an integer! This is really common: that the\n",
    "# API developer writes a property, such as Image.BICUBIC, and then assigns it to an\n",
    "# integer value to pass it around. Some languages use enumerations of values, which is\n",
    "# common in say, Java, but in python this is a pretty normal way of doing things.\n",
    "# The second thing we learned is that there are a number of different algorithms for\n",
    "# image resampling. In this case, the Image.LANCZOS and Image.BICUBIC filters do a good\n",
    "# job. Lets see if we are able to recognize the text off of this resized image\n",
    "\n",
    "# First lets resize to the larger size\n",
    "bigger_sign=little_sign.resize(new_size, Image.BICUBIC)\n",
    "# Lets print out the text\n",
    "pytesseract.image_to_string(bigger_sign)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Well, no text there. Lets try and binarize this. First, let me just bring in the\n",
    "# binarization code we did earlier\n",
    "def binarize(image_to_transform, threshold):\n",
    "    output_image=image_to_transform.convert(\"L\")\n",
    "    for x in range(output_image.width):\n",
    "        for y in range(output_image.height):\n",
    "            if output_image.getpixel((x,y))< threshold:\n",
    "                output_image.putpixel( (x,y), 0 )\n",
    "            else:\n",
    "                output_image.putpixel( (x,y), 255 )\n",
    "    return output_image\n",
    "\n",
    "# Now, lets apply binarizations with, say, a threshold of 190, and try and display that\n",
    "# as well as do the OCR work\n",
    "binarized_bigger_sign=binarize(bigger_sign, 190)\n",
    "display(binarized_bigger_sign)\n",
    "pytesseract.image_to_string(binarized_bigger_sign)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Ok, that text is pretty useless. How should we pick the best binarization\n",
    "# to use? Well, there are some methods, but lets just try something very simple to\n",
    "# show how well this can work. We have an english word we are trying to detect, \"FOSSIL\".\n",
    "# If we tried all binarizations, from 0 through 255, and looked to see if there were\n",
    "# any english words in that list, this might be one way. So lets see if we can\n",
    "# write a routine to do this.\n",
    "#\n",
    "# First, lets load a list of english words into a list. I put a copy in the readonly\n",
    "# directory for you to work with\n",
    "eng_dict=[]\n",
    "with open (\"readonly/words_alpha.txt\", \"r\") as f:\n",
    "    data=f.read()\n",
    "    # now we want to split this into a list based on the new line characters\n",
    "    eng_dict=data.split(\"\\n\")\n",
    "\n",
    "# Now lets iterate through all possible thresholds and look for an english word, printing\n",
    "# it out if it exists\n",
    "for i in range(150,170):\n",
    "    # lets binarize and convert this to s tring values\n",
    "    strng=pytesseract.image_to_string(binarize(bigger_sign,i))\n",
    "    # We want to remove non alphabetical characters, like ([%$]) from the text, here's\n",
    "    # a short method to do that\n",
    "    # first, lets convert our string to lower case only\n",
    "    strng=strng.lower()\n",
    "    # then lets import the string package - it has a nice list of lower case letters\n",
    "    import string\n",
    "    # now lets iterate over our string looking at it character by character, putting it in\n",
    "    # the comaprison text\n",
    "    comparison=''\n",
    "    for character in strng:\n",
    "        if character in string.ascii_lowercase:\n",
    "            comparison=comparison+character\n",
    "    # finally, lets search for comparison in the dictionary file\n",
    "    if comparison in eng_dict:\n",
    "        # and print it if we find it\n",
    "        print(comparison)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Well, not perfect, but we see fossil there among other values which are in the dictionary.\n",
    "# This is not a bad way to clean up OCR data. It can useful to use a language or domain specific \n",
    "# dictionary in practice, especially if you are generating a search engine for specialized language\n",
    "# such as a medical knowledge base or locations. And if you scroll up and look at the data\n",
    "# we were working with - this small little wall hanging on the inside of the store - it's not\n",
    "# so bad.\n",
    "#\n",
    "# At this point you've now learned how to manipulate images and convert them into text. In the\n",
    "# next module in this course we're going to dig deeper further into a computer vision library\n",
    "# which allows us to detect faces among other things. Then, on to the culminating project!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Jupyter Widgets (Optional)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# In this brief lecture I want to introduce you to one of the more advanced features of the \n",
    "# Jupyter notebook development environment called widgets. Sometimes you want\n",
    "# to interact with a function you have created and call it multiple times with different\n",
    "# parameters. For instance, if we wanted to draw a red box around a portion of an\n",
    "# image to try and fine tune the crop location. Widgets are one way to do this quickly\n",
    "# in the browser without having to learn how to write a large desktop application.\n",
    "#\n",
    "# Lets check it out. First we want to import the Image and ImageDraw classes from the\n",
    "# PILLOW package\n",
    "from PIL import Image, ImageDraw\n",
    "\n",
    "# Then we want to import the interact class from the widgets package\n",
    "from ipywidgets import interact\n",
    "\n",
    "# We will use interact to annotate a function. Lets bring in an image that we know we \n",
    "# are interested in, like the storefront image from a previous lecture\n",
    "image=Image.open('readonly/storefront.jpg')\n",
    "\n",
    "# Ok, our setup is done. Now we're going to use the interact decorator to indicate\n",
    "# that we want to wrap the python function. We do this using the @ sign. This will\n",
    "# take a set of parameters which are identical to the function to be called. Then Jupyter\n",
    "# will draw some sliders on the screen to let us manipulate these values. Decorators,\n",
    "# which is what the @ sign is describing, are standard python statements and just a\n",
    "# short hand for functions which wrap other functions. They are a bit advanced though, so\n",
    "# we haven't talked about them in this course, and you might just have to have some faith\n",
    "@interact(left=100, top=100, right=200, bottom=200)\n",
    "\n",
    "# Now we just write the function we had before\n",
    "def draw_border(left, top, right, bottom):\n",
    "    img=image.copy()\n",
    "    drawing_object=ImageDraw.Draw(img)\n",
    "    drawing_object.rectangle((left,top,right,bottom), fill = None, outline ='red')\n",
    "    display(img)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Jupyter widgets is certainly advanced territory, but if you would like\n",
    "# to explore more you can read about what is available here: \n",
    "# https://ipywidgets.readthedocs.io/en/stable/examples/Using%20Interact.html"
   ]
  }
 ],
 "metadata": {
  "colab": {
   "name": "Module 2.ipynb",
   "provenance": [],
   "version": "0.3.2"
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}