02_times.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Times and places\n",
    "\n",
    "Elements of Data Science\n",
    "\n",
    "by [Allen Downey](https://allendowney.com)\n",
    "\n",
    "[MIT License](https://opensource.org/licenses/MIT)\n",
    "\n",
    "### Goals\n",
    "\n",
    "In the previous notebook, you learned about variables and two kinds of values: integers and floating-point numbers.\n",
    "\n",
    "In this notebook, you'll see some additional types:\n",
    "\n",
    "* Strings, which contain sequences of letters, are used to represent text data.\n",
    "\n",
    "* Timestamp values are used to represent dates and times.\n",
    "\n",
    "* And several ways to represent and display geographical locations.\n",
    "\n",
    "Not every data science project uses all of these types, but many projects use at least one."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### A note\n",
    "\n",
    "In these notebooks, I generally try to define terms when they are first used, and to explain the code examples as we go along.\n",
    "\n",
    "However, there are a few places where I provide code with the expectation that you will not understand all of the details yet.  When that happens, I'll explain what you should or should not understand.\n",
    "\n",
    "If you are able to do the exercises, that probably means you are understanding what you need to understand."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Strings\n",
    "\n",
    "A **string** is a sequence of letters, numbers, and punctuation marks.\n",
    "\n",
    "In Python you can create a string by typing letters between single or double quotation marks."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "'Data'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "\"Science\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And you can assign string values to variables."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "first = 'Data'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "last = \"Science\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Some arithmetic operators work with strings, but they might no do what you expect.  For example, the `+` operator \"concatenates\" two strings; that is, it creates a new string that contains the first string followed by the second string:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "first + last"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you want to put a space between the words, you can use a string that contains a space:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "first + ' ' + last"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Strings are used to store text data like names, addresses, titles, etc.\n",
    "\n",
    "When you read data from a file, you might see values that look like numbers, but they are actually strings, like this:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "not_actually_a_number = '123'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you try to do math with these strings, you might get an error:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "not_actually_a_number + 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Or you might get a surprising result:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "not_actually_a_number + '1'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Fortunately, you can convert strings to numbers.\n",
    "\n",
    "If you have a string that contains only digits, you can convert it to an integer using the `int` function:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "int('123')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Or you can convert it to a floating-point number using `float`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "float('123')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "But if the string contains a decimal point, you can't convert it to an `int`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "int('12.3')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Going in the other direction, you can convert almost any type of value to a string using `str`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "str(123)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "str(12.3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Exercise**: When personal names are stored in a database, they are usually stored in three variables: a given name, a family name, and sometimes a middle name.  For example, a list of great rock drummers might include:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "given = 'Neil'\n",
    "middle = 'Ellwood'\n",
    "family = 'Peart'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "But names are often displayed different ways in different contexts.  For example, the first time you mention someone in an article, you might give all three names, like \"Neil Ellwood Peart\".  But in the index of a book, you might put the family name first, like \"Peart, Neil Ellwood\".\n",
    "\n",
    "Write Python expressions that use the variables `given`, `middle`, and `family` to display Neil Peart's name in these two formats. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Solution goes here"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Solution goes here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Dates and times"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you read data from a file, you might also find that dates and times are represented with strings."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "not_really_a_date = 'June 4, 1989'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To confirm that this value is a string, we can use the `type` function, which takes a value and reports its type."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "type(not_really_a_date)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`str` indicates that the value of `not_really_a_date` is a string.\n",
    "\n",
    "We get the same result with `not_really_a_time`, below:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "not_really_a_time = '6:30:00'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "type(not_really_a_time)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Representing dates and times using strings provides human-readable values, but they are not useful for doing computation.\n",
    "\n",
    "Fortunately, Python provides libraries for working with date and time data; the one we'll use is called Pandas.\n",
    "\n",
    "As always, we have to import a library before we use it; it is conventional to import Pandas with the abbreviated name `pd`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Pandas provides a type called `Timestamp`, which represents a date and time.\n",
    "\n",
    "It also provides a function called `Timestamp`, which we can use to convert a string to a `Timestamp`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "pd.Timestamp('6:30:00')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Or we can do the same thing using the variable defined above."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "pd.Timestamp(not_really_a_time)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this example, the string specifies a time but no date, so Pandas fills in today's date.\n",
    "\n",
    "A `Timestamp` is a value, so you can assign it to a variable."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [],
   "source": [
    "date_of_birth = pd.Timestamp('June 4, 1989')\n",
    "date_of_birth"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If the string specifies a date but no time, Pandas fills in midnight as the default time.\n",
    "\n",
    "If you assign the `Timestamp` to a variable, you can use the variable name to get the year, month, and day, like this:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "date_of_birth.year, date_of_birth.month, date_of_birth.day"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can also gets the name of the month and the day of the week."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [],
   "source": [
    "date_of_birth.day_name(), date_of_birth.month_name()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`Timestamp` provides a function called `now` that returns the current date and time."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [],
   "source": [
    "now = pd.Timestamp.now()\n",
    "now"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Exercise:** Use the value of `now` to display the name of the current month and day of the week."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Solution goes here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Timedelta\n",
    "\n",
    "`Timestamp` values support some arithmetic operations.  For example, you can compute the difference between two `Timestamps`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [],
   "source": [
    "age = now - date_of_birth\n",
    "age"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The result is a `Timedelta` that represents the current age of someone born on `date_of_birth`.\n",
    "\n",
    "The `Timedelta` contains `components` that store the number of days, hours, etc. between the two `Timestamp` values."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [],
   "source": [
    "age.components"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can get one of the components like this:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [],
   "source": [
    "age.days"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The biggest component of `Timedelta` is days, not years, because days are well defined and years are problematic.\n",
    "\n",
    "Most years are 365 days, but some are 366.  The average calendar year is 365.24 days, which is a very good approximation of a solar year, [but it is not exact](https://pumas.jpl.nasa.gov/files/04_21_97_1.pdf).\n",
    "\n",
    "One way to compute age in years is to divide age in days by 365.24:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [],
   "source": [
    "age.days / 365.24"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "But people usually report their ages in integer years.  We can use the Numpy `floor` function to round down:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "\n",
    "np.floor(age.days / 365.24)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Or the `ceil` function (which stands for \"ceiling\") to round up:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [],
   "source": [
    "np.ceil(age.days / 365.24)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also compare `Timestamp` values to see which comes first.\n",
    "\n",
    "For example, let's see if a person with a given birthdate has already had a birthday this year.\n",
    "\n",
    "We can create a new `Timestamp` with the year from `now` and the month and day from `date_of_birth`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [],
   "source": [
    "bday_this_year = pd.Timestamp(now.year, date_of_birth.month, date_of_birth.day)\n",
    "bday_this_year"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The result represents the person's birthday this year.  Now we can use the `>` operator to check whether `now` is later than the birthday:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [],
   "source": [
    "now > bday_this_year"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The result is either `True` or `False`, which are special values in Python used to represent results from this kind of comparison.\n",
    "\n",
    "These values belong to a type called `bool`, short for \"Boolean algebra\", which is a branch of algebra where all values are either true or false. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [],
   "source": [
    "type(True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [],
   "source": [
    "type(False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Exercise:** Any two people with different birthdays have a \"Double Day\" when one is twice as old as the other.\n",
    "\n",
    "Suppose you are given two `Timestamp` values, `d1` and `d2`, that represent birthdays for two people.  Use `Timestamp` arithmetic to compute their double day.\n",
    "\n",
    "Here are two example dates; with these dates, the result should be December 19, 2009."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [],
   "source": [
    "d1 = pd.Timestamp('2003-07-12')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [],
   "source": [
    "d2 = pd.Timestamp('2006-09-30')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Solution goes here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Location\n",
    "\n",
    "There are many ways to represent geographical locations, but the most common, at least for global data, is latitude and longitude.\n",
    "\n",
    "When stored as strings, latitude and longitude are expressed in degrees with compass directions N, S, E, and W.  For example, this string represents the location of Boston, MA, USA:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [],
   "source": [
    "lat_lon_string = '42.3601° N, 71.0589° W'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When we compute with location information, we use floating-point numbers, with \n",
    "\n",
    "* Positive latitude for the northern hemisphere, negative latitude for the southern hemisphere, and \n",
    "\n",
    "* Positive longitude for the eastern hemisphere and negative latitude for the western hemisphere.\n",
    "\n",
    "Of course, the choice of the origin and the orientation of positive and negative are arbitrary choices that were made for historical reasons.  We might not be able to change conventions like these, but we should be aware that they are conventions.\n",
    "\n",
    "Here's how we might represent the location of Boston with two variables."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [],
   "source": [
    "lat = 42.3601\n",
    "lon = -71.0589"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It is also possible to combine two numbers into a composite value and assign it to a single variable:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [],
   "source": [
    "boston = lat, lon\n",
    "boston"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The type of this variable is `tuple`, which is a mathematical term for a value that contains a sequence of elements.  Math people pronounce it \"tuh' ple\", but computational people usually say \"too' ple\".  Take your pick."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [],
   "source": [
    "type(boston)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you have a tuple with two elements, you can assign them to two variables, like this:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [],
   "source": [
    "y, x = boston\n",
    "y"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [],
   "source": [
    "x"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Notice that I assigned latitude to `y` and longitude to `x`, because a `y` coordinate usually goes up and down like latitude, and an `x` coordinate usually goes side-to-side like longitude."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Exercise:** Find the latitude and longitude of the place you were born or someplace you think of as your \"home town\".  [You can use this web page to look it up](https://www.latlong.net/), among others.\n",
    "\n",
    "Make a tuple of floating-point numbers that represents this location."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Solution goes here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Distance\n",
    "\n",
    "If you are given two tuples that represent locations, you can compute the approximate distance between them, along the surface of the globe, using the haversine function.\n",
    "\n",
    "If you are curious about it, [you can read an explanation in this article](https://janakiev.com/blog/gps-points-distance-python/).\n",
    "\n",
    "To estimate a haversine distance, we have to compute the haversine function, which is defined:\n",
    "\n",
    "$\\mathrm{haversine}(θ)=\\sin^2(θ/2)$\n",
    "\n",
    "Where θ is an angle in radians.\n",
    "\n",
    "We can compute this function in Python like this:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "\n",
    "θ = 1\n",
    "np.sin(θ/2)**2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can use Greek letters in variable names, but there is currently no way to type them in Jupyter/Colab, so I usually copy them from a web page and paste them in.\n",
    "\n",
    "To avoid the inconvenience, it is more common to write out letter names, like this:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [],
   "source": [
    "theta = 1\n",
    "np.sin(theta/2)**2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Exercise:** This is a good time to remind you that the operator for exponentiation is `**`.  \n",
    "\n",
    "In some other languages the operator for exponentiation is `^`.  That is also an operator in Python, but it performs another operation altogether.\n",
    "\n",
    "Try out the previous expression, replacing `**` with `^`, and see what error message you get.  Remember this message in case you see it in the future!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Solution goes here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Defining functions\n",
    "\n",
    "At this point you don't have to understand how to define a new function.  But you will see function defininitions, so I want to explain the basics now. \n",
    "\n",
    "If we are planning to use an expression like `np.sin(theta/2)**2` more than a few times, we can define a new function that compute it, like this:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [],
   "source": [
    "def haversine(theta):\n",
    "    \"\"\"Compute the haversine function of theta.\"\"\"\n",
    "    return np.sin(theta/2)**2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "On the first line, `def` indicates that we are defining a function.\n",
    "\n",
    "The second line is a \"triple-quoted string\", which describes what the function does, but it has no effect when the program runs.\n",
    "\n",
    "On the third line, `return` indicates the result of the function.\n",
    "\n",
    "When you run the previous cell, it creates a new variable called `haversine`.  You can display its value like this:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [],
   "source": [
    "haversine"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And you can display its type like this:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {},
   "outputs": [],
   "source": [
    "type(haversine)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So `haversin` is a variable that refers to a function.\n",
    "\n",
    "To run the function and compute a result, we have to \"call\" the function and provide a value for `theta`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {},
   "outputs": [],
   "source": [
    "haversine(1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When you define a function, you create a new variable.  But the function doesn't actually run until you call it."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Haversine distance\n",
    "\n",
    "Now we can use `haversine` as part of a function that computes haversine distances.\n",
    "\n",
    "I won't explain this function in as much detail, but if you read through it, you might get a sense of how it works."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [],
   "source": [
    "def haversine_distance(coord1, coord2):\n",
    "    \"\"\"Haversine distance between two locations.\n",
    "    \n",
    "    coord1: lat-lon as tuple of float \n",
    "    coord2: lat-lon as tuple of float\n",
    "    \n",
    "    returns: distance in km\n",
    "    \"\"\"\n",
    "    R = 6372.8  # Earth radius in km\n",
    "    lat1, lon1 = coord1\n",
    "    lat2, lon2 = coord2\n",
    "    \n",
    "    phi1, phi2 = np.radians(lat1), np.radians(lat2) \n",
    "    dphi       = np.radians(lat2 - lat1)\n",
    "    dlambda    = np.radians(lon2 - lon1)\n",
    "    \n",
    "    a = haversine(dphi) + np.cos(phi1)*np.cos(phi2)*haversine(dlambda)\n",
    "    \n",
    "    distance = 2*R*np.arctan2(np.sqrt(a), np.sqrt(1 - a))\n",
    "    \n",
    "    return distance"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When we call this function, we provide two tuples, each representing a latitude and a longitude.\n",
    "\n",
    "We already have a tuple that represents the location of Boston.\n",
    "Now here's a tuple that represents the location of London, England, UK:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {},
   "outputs": [],
   "source": [
    "london = 51.5074, -0.1278"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And here's the haversine distance between Boston and London."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {},
   "outputs": [],
   "source": [
    "haversine_distance(boston, london)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The actual geographic distance is slightly different because Earth is not a perfect sphere.  But the error of this estimate is less than 1%."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Exercise:** Use `haversine_distance` to compute the distance between Boston and your \"home town\" from the previous exercise.\n",
    "\n",
    "If possible, use an online map to check the result."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Solution goes here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Geopandas\n",
    "\n",
    "Python provides libraries for working with geographical data.  One of the most popular is Geopandas, which is based on another library called Shapely.  I'll introduce these libraries here, and we'll come back to them later.\n",
    "\n",
    "Shapely provides `Point` and `LineString` values, which we'll use to represent geographic locations and lines between locations."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {},
   "outputs": [],
   "source": [
    "from shapely.geometry import Point, LineString"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can use the tuples we defined in the previous section to create Shapely `Point` values, but we have to reverse the order of the coordinates, providing them in x-y order rather than lat-lon order, because that's the order the `Point` function expects."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {},
   "outputs": [],
   "source": [
    "lat, lon = boston\n",
    "p1 = Point(lon, lat)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {},
   "outputs": [],
   "source": [
    "lat, lon = london\n",
    "p2 = Point(lon, lat)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If we display a `Point` value, we get a graphical representation, but not a very useful one."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {},
   "outputs": [],
   "source": [
    "p1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Soon we will see how to plot a `Point`, more usefully, on a map."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can use the points we just defined to create a `LineString`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {},
   "outputs": [],
   "source": [
    "line = LineString([p1, p2])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If we display the result, we get another not very useful graphical representation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {},
   "outputs": [],
   "source": [
    "line"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "However, now we can use Geopandas to show these points and lines on a map.\n",
    "\n",
    "If you are running this notebook on Colab, the following cell will install Geopandas, which should only take a few seconds.  It uses features we have not seen yet; you might be able to read it and guess how it works, but you don't have to."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {},
   "outputs": [],
   "source": [
    "import sys\n",
    "IN_COLAB = 'google.colab' in sys.modules\n",
    "\n",
    "if IN_COLAB:\n",
    "    !pip install geopandas"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now the following import statement should work."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {},
   "outputs": [],
   "source": [
    "import geopandas as gpd"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you are running this notebook on Colab, the libraries you install disappear when you shut down the notebook.  When you start the notebook again, you have to install them again."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The following code loads a map of the world and plots it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "metadata": {},
   "outputs": [],
   "source": [
    "path = gpd.datasets.get_path('naturalearth_lowres')\n",
    "world = gpd.read_file(path)\n",
    "world.plot(color='white', edgecolor='gray');"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here's a version that just plots North America and Europe:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "metadata": {},
   "outputs": [],
   "source": [
    "north_america = world.continent == 'North America'\n",
    "europe = world.continent == 'Europe'\n",
    "region = world[north_america | europe]\n",
    "region.plot(color='white', edgecolor='gray');"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Notice:\n",
    "\n",
    "* By default, Geopandas uses the political definition of \"Europe\", which includes the part of Russia that is on the Asian continent.\n",
    "\n",
    "* It also uses an [equirectangular projection](https://en.wikipedia.org/wiki/Equirectangular_projection), which provides a misleading picture of relative land areas.\n",
    "\n",
    "You can't make a map without making visualization decisions.\n",
    "\n",
    "Now let's put dots on the map for Boston and London.  We have to put the `Point` values and the `LineString` into a `GeoSeries`, which provides a `plot` function:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "metadata": {},
   "outputs": [],
   "source": [
    "t = [p1, p2, line]\n",
    "series = gpd.GeoSeries(t)\n",
    "series.plot();"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here's a first attempt to plot the maps and the lines together:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 80,
   "metadata": {},
   "outputs": [],
   "source": [
    "# plot the map\n",
    "region.plot(color='white', edgecolor='gray')\n",
    "\n",
    "# plot Boston, London, and the line\n",
    "series.plot();"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "GeoPandas puts the two plots on different axes, which is not what we want in this case.\n",
    "\n",
    "To get the points and the map on the same axes, we have to use a function from Matplotlib, which is a visualization library we will use extensively."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 81,
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The function is `gca`, which stands for \"get current axes\".  We can use the result to tell `plot` to put the points and lines on the current axes, rather than create a new one."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 82,
   "metadata": {},
   "outputs": [],
   "source": [
    "ax = plt.gca()\n",
    "\n",
    "# plot the map\n",
    "region.plot(color='white', edgecolor='gray', ax=ax)\n",
    "\n",
    "# plot Boston, London, and the line\n",
    "series.plot(ax=ax);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also use `plt.figure` to set the figure size."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 83,
   "metadata": {},
   "outputs": [],
   "source": [
    "plt.figure(figsize=(12, 12))\n",
    "ax = plt.gca()\n",
    "\n",
    "# plot the map\n",
    "world.plot(color='white', edgecolor='gray', ax=ax)\n",
    "\n",
    "# plot Boston, London, and the line\n",
    "series.plot(ax=ax);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There are a few features in this example we have not explained completely, but hopefully you get the idea.  We will come back to Geopandas later."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Exercise:** Modify the code in the previous section to plot a point that shows the \"home town\" you chose in a previous exercise and a line from there to Boston.\n",
    "\n",
    "Then go to [this online survey](https://forms.gle/2LDcqxJmcWY3Q3X47) and answer the questions there.  We will use your responses for an upcoming example."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}