data-sonification-with-python.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "NPqxpRFqMKn5"
   },
   "source": [
    "# Data sonification with Python\n",
    "\n",
    "## Instructor\n",
    "\n",
    "- Walt Gurley\n",
    "\n",
    "## Workshop description\n",
    "\n",
    "Data visualization is the process of using graphical elements to represent data. This workshop introduces the concept of data sonification, using characteristics of sound to represent information. Sonification can provide an alternate mode for communicating data with implications for accessibility, engagement, and discovery. Participants in this workshop will get an overview of sonification techniques and tools and learn basic processes for mapping data to sound using the Python programming language.\n",
    "\n",
    "## Learning objectives\n",
    "\n",
    "- A basic understanding of the properties of sound\n",
    "\n",
    "- A basic understanding of using properties of sound to represent data and different sonification methods\n",
    "\n",
    "- Experience using `pandas` data types and methods to manipulate data\n",
    "\n",
    "- An awareness of the `music21` library and its utility for producing audio from structured data\n",
    "\n",
    "- Experience applying different methods to map numerical data to audio frequency/pitch using Python\n",
    "\n",
    "## Questions during the workshop\n",
    "\n",
    "Please feel free to ask questions throughout the workshop.\n",
    "\n",
    "We have a second instructor who will available during the workshop. They will answer as able, and will collect questions with answers that might help everyone to be answered at the end of the workshop.\n",
    "\n",
    "## Using Jupyter Notebooks and Google Colaboratory\n",
    "\n",
    "Jupyter notebooks are a way to write and run Python code in an interactive way. They're quickly becoming a standard way of putting together data, code, and written explanations or visualizations into a single document and sharing that. There are a lot of ways that you can run Jupyter notebooks, including just locally on your computer, but we've decided to use Google's Colaboratory notebook platform for this workshop. Colaboratory is “a Google research project created to help disseminate machine learning education and research.” If you would like to know more about Colaboratory in general, you can visit the [Welcome Notebook](https://colab.research.google.com/notebooks/welcome.ipynb)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Q00n3-A8wrN4"
   },
   "source": [
    "## Notebook setup\n",
    "\n",
    "The next cell loads the necessary Python libraries and dependencies. If this notebook is run in Google Colab, it will also install and load extra dependencies to create audio files and play audio files.\n",
    "\n",
    "Libraries imported into this notebook:\n",
    "- [music21](https://web.mit.edu/music21/) - toolkit for computer-aided musicology\n",
    "- [pandas](https://pandas.pydata.org/) - a data analysis and manipulation tool\n",
    "- [NumPy](https://numpy.org/) - a library supporting numerical analysis\n",
    "- [matplotlib](https://matplotlib.org/) - a plotting library\n",
    "- [midi2audio](https://pypi.org/project/midi2audio/) - a tool for synthesizing or playing MIDI audio\n",
    "- [IPython Audio controls](https://ipython.readthedocs.io/en/stable/api/generated/IPython.display.html?highlight=audio#IPython.display.Audio) - a tool for playing audio and generating audio controls in a Jupyter notebook\n",
    "- [os](https://docs.python.org/3/library/os.html) - a Python module providing functions for manipulating files and directories\n",
    "\n",
    "Additional dependencies:\n",
    "- [Fluidsynth](http://www.fluidsynth.org/) - a synthesizer for processing MIDI files. This can be installed in Google Colab and can also be installed on your local machine\n",
    "- [Fluid (R3) Mono General MIDI SoundFont (GM) (FluidR3Mono_GM.sf3)](https://github.com/musescore/MuseScore/tree/master/share/sound) - a soundfont file containing the synthesized instruments to playback MIDI audio. This is an open source soundfont file obtained from the [MuseScore GitHub repository](https://github.com/musescore/MuseScore)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "executionInfo": {
     "elapsed": 44680,
     "status": "ok",
     "timestamp": 1615560735379,
     "user": {
      "displayName": "Walt Gurley",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14Gg9eQ8NtdhzyTtyTqhb-4LANYN8W8pA7TUUhK54=s64",
      "userId": "03624837261806285415"
     },
     "user_tz": 300
    },
    "id": "UdqdA8g9wrN5",
    "outputId": "c60d5d61-d8d3-41ef-903d-4246b62bf04e",
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Test if this notebook is running in Colab\n",
    "try:\n",
    "  import google.colab\n",
    "  IN_COLAB = True\n",
    "except:\n",
    "  IN_COLAB = False\n",
    "print(\"I am in Colab: \" + str(IN_COLAB))\n",
    "\n",
    "# If running in Colab install additional dependencies to create audio files from\n",
    "# MIDI files\n",
    "if IN_COLAB:\n",
    "  # Install synthesizer to process MIDI files\n",
    "  !apt install fluidsynth\n",
    "  # Copy the soundfonts file to our session storage space (No longer using installed soundfont. Using soundfont stored in GitHub repository)\n",
    "  # !cp /usr/share/sounds/sf2/FluidR3_GM.sf2 ./FluidR3_GM.sf2\n",
    "  # Install midi2audio module to convert MIDI files to audio files\n",
    "  !pip install midi2audio\n",
    "\n",
    "# Load midi2audio module to convert MIDI files to audio files\n",
    "from midi2audio import FluidSynth\n",
    "\n",
    "# Load modules from the music21 library for sonifying data\n",
    "from music21 import instrument, note, scale, stream, midi, tempo\n",
    "\n",
    "# Load data processing and visualization libraries\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "from matplotlib import pyplot as plt\n",
    "plt.style.use('ggplot')\n",
    "\n",
    "# Load the Audio display tool to play and control audio\n",
    "from IPython.display import Audio\n",
    "\n",
    "# Import os library to test if \"FluidR3Mono_GM.sf3\" file already exists in\n",
    "# current directory, fetch the soundfont file from GitHub if not\n",
    "import os\n",
    "if not os.path.exists('FluidR3Mono_GM.sf3'):\n",
    "  # Fetch the soundfont file from GitHub\n",
    "  !curl https://raw.githubusercontent.com/ncsu-libraries-data-vis/data-sonification-with-python/main/FluidR3Mono_GM.sf3 -o FluidR3Mono_GM.sf3"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "r1qGIoJIz_LI"
   },
   "source": [
    "## A brief overview of audio properties and sonification\n",
    "\n",
    "Sound travels through air like a wave as particles are compressed together and then stretched apart. By measuring how these particles change we can represent sound as a series of waves called a waveform.\n",
    "\n",
    "An audio waveform has two basic properties, **amplitude** and **wavelength**. Amplitude is measured as the magnitude of displacement of a particle from its original position and can be thought of as loudness. Wavelength is used to measure frequency. Frequency is a measure of how many times the waveform repeats over time. Frequency is directly related to pitch, lower frequencies have a lower pitch and higher frequencies a higher pitch. Humans have a general hearing frequency range of 20 Hz to 20,000 Hz (Hz = one cycle per second).\n",
    "\n",
    "Here is a wonderful [interactive guide to audio waveforms](https://pudding.cool/2018/02/waveforms/) by Josh Comeau.\n",
    "\n",
    "![An image representing a sound wave traveling through the air and as a waveform.](https://github.com/WaltGurley/rem-rem-cur/blob/gh-pages/MicIn/particleSound.png?raw=true \"As a sound wave travels through the air particles are compressed and stretched apart. This can be modeled as a waveform\")\n",
    "\n",
    "Beyond the basic properties of sound waves we can also consider the audio properties of timbre (the perceived quality of a sound, e.g., how a guitar sounds different than a trumpet), tempo (the speed at which a collection of sounds are played, e.g., beats per minute), and spatial positioning (where a sound is coming from in space, e.g., panning left or right in stereo sound).\n",
    "\n",
    "Just as we may map a data value to color, position, or size on a graph, we can use these properties of sound to represent data sonically. Sonification has the ability to represent data in a way that complements visualization and has valid application with regard to accessibility, engagement, and discovery:\n",
    "\n",
    "- Accessibility: [SAS Graphics Accelerator](https://support.sas.com/software/products/graphics-accelerator/samples/index.html)\n",
    "\n",
    "- Engagement: [Sounds from Around the Milky Way](https://www.nasa.gov/mission_pages/chandra/news/data-sonification-sounds-from-around-the-milky-way.html)\n",
    "\n",
    "- Discovery: [Sonification of Cyclone Sidr](https://youtu.be/RRluA1r3rTk)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "gk74DIs4wrN9"
   },
   "source": [
    "## Load and observe/clean the dataset\n",
    "\n",
    "We will be working with a dataset consisting of monthly atmospheric carbon dioxide (CO<sup>2</sup>) values measured at Mauna Loa Observatory, Hawaii from the Scripps CO<sup>2</sup> program website (visit the [Scripps CO<sup>2</sup> program website](https://scrippsco2.ucsd.edu/data/atmospheric_co2/primary_mlo_co2_record.html) for more information about the dataset we are using).\n",
    "\n",
    "We load the raw dataset directly from the Scripps website using the pandas method `read_csv()`. The raw data file requires some intial parsing and manipulation using arguments passed into `read_csv()`, such as identifying the starting row of the data (using the keyword argument `header=56`), filtering only the columns of data of interest (using the keyword argument `usecols=[0, 1, 2, 4, 5]`, and parsing dates (using the keyword argument `parse_dates=[2]` and the `date_parser` function). Comments are included in the code below to describe each argument."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 450
    },
    "executionInfo": {
     "elapsed": 45295,
     "status": "ok",
     "timestamp": 1615560736006,
     "user": {
      "displayName": "Walt Gurley",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14Gg9eQ8NtdhzyTtyTqhb-4LANYN8W8pA7TUUhK54=s64",
      "userId": "03624837261806285415"
     },
     "user_tz": 300
    },
    "id": "KTv0U4EKwrN-",
    "outputId": "4feb88f6-7868-44cf-fcb7-08e720964792",
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Read and format the csv file located at the provided URL into a Pandas\n",
    "# DataFrame\n",
    "co2_raw = pd.read_csv(\n",
    "  # The URL of the csv file\n",
    "  \"https://scrippsco2.ucsd.edu/assets/data/atmospheric/stations/in_situ_co2/monthly/monthly_in_situ_co2_mlo.csv\",\n",
    "  # The row position of the column headers for the dataset\n",
    "  header=56,\n",
    "  # Create new column header names to rename given headers\n",
    "  names=[\"year\", \"month\", \"date\", \"CO2ppm\", \"season_adj\"],\n",
    "  # Only take the specified columns from the csv file\n",
    "  usecols=[0,1,2,4,5],\n",
    "  # Parse dates from the data given in column 2\n",
    "  parse_dates=[2],\n",
    "  # How to parse the date values\n",
    "  date_parser=lambda x: pd.to_datetime(int(x), unit='D', origin=pd.Timestamp('01-01-1900')),\n",
    "  # Set the column index of the DataFrame\n",
    "  index_col=2\n",
    ")\n",
    "\n",
    "# Print the data\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 450
    },
    "executionInfo": {
     "elapsed": 45286,
     "status": "ok",
     "timestamp": 1615560736007,
     "user": {
      "displayName": "Walt Gurley",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14Gg9eQ8NtdhzyTtyTqhb-4LANYN8W8pA7TUUhK54=s64",
      "userId": "03624837261806285415"
     },
     "user_tz": 300
    },
    "id": "667sdEAqwrOB",
    "outputId": "5bdd1513-606d-42f0-d16c-9d482531c3e0"
   },
   "outputs": [],
   "source": [
    "# Replace missing values (-99.99) with NaN\n",
    "\n",
    "\n",
    "# Trim dataset to remove leading and trailing missing CO2 data\n",
    "\n",
    "\n",
    "# Print the data\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 371
    },
    "executionInfo": {
     "elapsed": 45711,
     "status": "ok",
     "timestamp": 1615560736441,
     "user": {
      "displayName": "Walt Gurley",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14Gg9eQ8NtdhzyTtyTqhb-4LANYN8W8pA7TUUhK54=s64",
      "userId": "03624837261806285415"
     },
     "user_tz": 300
    },
    "id": "rryyuA0WUTIm",
    "outputId": "0e1ba14f-ab86-4d52-d5e0-fc90ef60c8af"
   },
   "outputs": [],
   "source": [
    "# Configure the plot\n",
    "plt.figure(figsize=[8, 5])\n",
    "plt.xlabel(\"Time (months)\")\n",
    "plt.ylabel(\"CO2 ppm\")\n",
    "plt.title(\"Monthly atmospheric CO2 values measured at Mauna Loa Observatory, Hawaii\")\n",
    "\n",
    "# Plot the data\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "qelb72YFVKbs"
   },
   "source": [
    "**Question:** What properties of sound could we use to sonify this dataset (e.g., frequency, amplitude, timbre, tempo, and spatial position)?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "bEXeBuTMwrOE"
   },
   "source": [
    "## Audification\n",
    "Audification is a type of sonification in which a data series is directly translated into an audio waveform. This sonification process is applied in research ranging from medicine to seismology and astronomy.\n",
    "\n",
    "Audification is typically suitable for large datasets that have a cyclical component.\n",
    "\n",
    "Examples:\n",
    "\n",
    "- [Vibrations of the Sun](https://soundcloud.com/nasa/sun-sonification)\n",
    "- [Audification of brainwave data](https://youtu.be/y1Nl3De_frM)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "UzlygM1RSTxg"
   },
   "source": [
    "### Audification of a sine wave\n",
    "We will first demonstrate the process of audification by generating a sine wave over time and then converting that data into audio.\n",
    "\n",
    "When creating audio we need to consider two things, the sample rate and the shape of the sound wave."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "executionInfo": {
     "elapsed": 45702,
     "status": "ok",
     "timestamp": 1615560736442,
     "user": {
      "displayName": "Walt Gurley",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14Gg9eQ8NtdhzyTtyTqhb-4LANYN8W8pA7TUUhK54=s64",
      "userId": "03624837261806285415"
     },
     "user_tz": 300
    },
    "id": "36rB0aBTLzHJ",
    "outputId": "2394bacb-49bc-45fb-be8d-88bba36c0738"
   },
   "outputs": [],
   "source": [
    "# How many times per second (Hz) are we sampling our data? (44100 Hz is a\n",
    "# common sampling frequency)\n",
    "\n",
    "\n",
    "# Over how many seconds are we sampling our data?\n",
    "\n",
    "\n",
    "# Generate a series of time samples at a rate of 'sampleRate' Hz over 'time' \n",
    "# seconds (44100 Hz * 2 seconds = 88200 samples)\n",
    "\n",
    "\n",
    "# Print out time samples\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "executionInfo": {
     "elapsed": 45700,
     "status": "ok",
     "timestamp": 1615560736442,
     "user": {
      "displayName": "Walt Gurley",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14Gg9eQ8NtdhzyTtyTqhb-4LANYN8W8pA7TUUhK54=s64",
      "userId": "03624837261806285415"
     },
     "user_tz": 300
    },
    "id": "oItIPoS5cpme"
   },
   "outputs": [],
   "source": [
    "# Generate a sine wave with a frequency of 'frequency' Hz over 'timeSample'\n",
    "# samples \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 367
    },
    "executionInfo": {
     "elapsed": 46268,
     "status": "ok",
     "timestamp": 1615560737019,
     "user": {
      "displayName": "Walt Gurley",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14Gg9eQ8NtdhzyTtyTqhb-4LANYN8W8pA7TUUhK54=s64",
      "userId": "03624837261806285415"
     },
     "user_tz": 300
    },
    "id": "7InH00ORcsrp",
    "outputId": "6a784279-c9d5-4b6b-c662-be2618059325"
   },
   "outputs": [],
   "source": [
    "# Configure the sine wave plot\n",
    "plt.figure(figsize=[15, 5])\n",
    "plt.xlabel('Time (seconds)')\n",
    "plt.ylabel('Amplitude')\n",
    "plt.title(f'{frequency} Hz sine wave')\n",
    "\n",
    "# Only plot 1/4 of the data (0.25 seconds)\n",
    "time_max = sample_rate // 4\n",
    "plt.plot(time_samples[:time_max], sine_wave[:time_max])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 75
    },
    "executionInfo": {
     "elapsed": 46261,
     "status": "ok",
     "timestamp": 1615560737024,
     "user": {
      "displayName": "Walt Gurley",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14Gg9eQ8NtdhzyTtyTqhb-4LANYN8W8pA7TUUhK54=s64",
      "userId": "03624837261806285415"
     },
     "user_tz": 300
    },
    "id": "uhG2kwaiEeON",
    "outputId": "80137dd0-3cce-4687-fc63-3a65377fce8b"
   },
   "outputs": [],
   "source": [
    "# Generate audio from sine wave data\n",
    "Audio(sine_wave, rate=sample_rate)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "7y0Er_qM8-WV"
   },
   "source": [
    "**Question**: Would a 440 Hz sine wave have a higher or lower pitch than the 220 Hz sine wave?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "V2KxY0YnsXw-"
   },
   "source": [
    "### Audification of CO<sup>2</sup> concentration data\n",
    "\n",
    "We need to modify our data in order to create an audification. First, our data has a sample rate of 12 samples per year. That sample rate is approximately 0.0000004 Hz, and based on the lower limit of human hearing (20 Hz), this frequency is well below our ability to hear.\n",
    "\n",
    "To bring our dataset into an audible frequency range we must speed it up considerably. We will compress the time scale of our data to a sample rate of 3000 Hz (the lowest sample rate at which we can playback audio with the IPython Audio tool). This equates to a frequency increase of about 10^10."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "executionInfo": {
     "elapsed": 46283,
     "status": "ok",
     "timestamp": 1615560737049,
     "user": {
      "displayName": "Walt Gurley",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14Gg9eQ8NtdhzyTtyTqhb-4LANYN8W8pA7TUUhK54=s64",
      "userId": "03624837261806285415"
     },
     "user_tz": 300
    },
    "id": "sMWoQ9LtLfGq"
   },
   "outputs": [],
   "source": [
    "# Set our sample rate at 3000 Hz\n",
    "\n",
    "\n",
    "# Generate a series of time samples at a rate of 'data_sample_rate' Hz over 1 second\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "executionInfo": {
     "elapsed": 46291,
     "status": "ok",
     "timestamp": 1615560737059,
     "user": {
      "displayName": "Walt Gurley",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14Gg9eQ8NtdhzyTtyTqhb-4LANYN8W8pA7TUUhK54=s64",
      "userId": "03624837261806285415"
     },
     "user_tz": 300
    },
    "id": "nT1V20UY_vY_"
   },
   "outputs": [],
   "source": [
    "# Use a linear interpolation to fill NaN values, the audio player WILL NOT WORK\n",
    "# with data that contains any NaN values\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 371
    },
    "executionInfo": {
     "elapsed": 46441,
     "status": "ok",
     "timestamp": 1615560737219,
     "user": {
      "displayName": "Walt Gurley",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14Gg9eQ8NtdhzyTtyTqhb-4LANYN8W8pA7TUUhK54=s64",
      "userId": "03624837261806285415"
     },
     "user_tz": 300
    },
    "id": "V_PnFimTeUAa",
    "outputId": "715b65c5-2b6f-40ea-e169-8ff73c2f00bf"
   },
   "outputs": [],
   "source": [
    "# Configure the plot\n",
    "plt.figure(figsize=[8, 5])\n",
    "plt.xlabel('Time (seconds)')\n",
    "plt.ylabel('Amplitude')\n",
    "plt.title('CO2 concentration compressed to a sample rate of 3000 Hz')\n",
    "\n",
    "# Plot the time compressed waveform of CO2 concentration data\n",
    "plt.plot(data_time_samples[:len(co2_int)], co2_int)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 75
    },
    "executionInfo": {
     "elapsed": 46442,
     "status": "ok",
     "timestamp": 1615560737230,
     "user": {
      "displayName": "Walt Gurley",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14Gg9eQ8NtdhzyTtyTqhb-4LANYN8W8pA7TUUhK54=s64",
      "userId": "03624837261806285415"
     },
     "user_tz": 300
    },
    "id": "fm8zm69Wefml",
    "outputId": "2855ee2a-0cfb-4f81-d857-ebca4721b0fc"
   },
   "outputs": [],
   "source": [
    "# The sample rate of our data is approximately 0.0000004 Hz (12 samples / year),\n",
    "# we speed it up about 10^10 times (3000 Hz)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "88FxaUqO_IGL"
   },
   "source": [
    "**Question:** Why is our audification so short?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ivhZJCWnHVie"
   },
   "source": [
    "### Audification of normalized CO<sup>2</sup> concentration data\n",
    "\n",
    "Even when resampling our dataset at a higher frequency, we still don't really get a useful audio representation of our waveform. We need to modify our dataset even further to establish a central value about which we can measure displacement. To do this we will normalize our dataset by removing the longterm trend from the data (i.e., subtracting the seasonally adjusted CO<sup>2</sup> concentration values from the true CO<sup>2</sup> concentration values)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "executionInfo": {
     "elapsed": 46444,
     "status": "ok",
     "timestamp": 1615560737235,
     "user": {
      "displayName": "Walt Gurley",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14Gg9eQ8NtdhzyTtyTqhb-4LANYN8W8pA7TUUhK54=s64",
      "userId": "03624837261806285415"
     },
     "user_tz": 300
    },
    "id": "86OvHtAewrOH"
   },
   "outputs": [],
   "source": [
    "# Remove the longterm trend of increasing CO2 concentration\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "executionInfo": {
     "elapsed": 46438,
     "status": "ok",
     "timestamp": 1615560737238,
     "user": {
      "displayName": "Walt Gurley",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14Gg9eQ8NtdhzyTtyTqhb-4LANYN8W8pA7TUUhK54=s64",
      "userId": "03624837261806285415"
     },
     "user_tz": 300
    },
    "id": "lU3POvVJ_2Ez",
    "outputId": "585d7792-fe69-4f2e-ae09-7eefcc738f8c"
   },
   "outputs": [],
   "source": [
    "# Use a linear interpolation to fill NaN values–the audio player WILL NOT WORK\n",
    "# with data that contains any NaN values\n",
    "\n",
    "\n",
    "# Print the data\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 641
    },
    "executionInfo": {
     "elapsed": 47524,
     "status": "ok",
     "timestamp": 1615560738334,
     "user": {
      "displayName": "Walt Gurley",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14Gg9eQ8NtdhzyTtyTqhb-4LANYN8W8pA7TUUhK54=s64",
      "userId": "03624837261806285415"
     },
     "user_tz": 300
    },
    "id": "FKZvCW4ljJBG",
    "outputId": "b6cc371f-8235-4006-86b5-6297ec2d45b5"
   },
   "outputs": [],
   "source": [
    "# Plot CO2 concentration data\n",
    "plt.figure(figsize=[15, 10])\n",
    "plt.subplot(2, 2, 1)\n",
    "plt.xlabel('Time (months)')\n",
    "plt.ylabel('CO2 ppm')\n",
    "plt.title('CO2 concentration')\n",
    "plt.plot(co2[\"CO2ppm\"])\n",
    "\n",
    "# Plot seasonally adjusted CO2 concentration data\n",
    "plt.subplot(2, 2, 2)\n",
    "plt.xlabel('Time (months)')\n",
    "plt.ylabel('CO2 ppm')\n",
    "plt.title('Seasonally adjusted CO2 concentration')\n",
    "plt.plot(co2[\"season_adj\"])\n",
    "\n",
    "# Plot normalized CO2 concentration data over 3000 Hz sample frequency\n",
    "plt.subplot(2, 2, 3)\n",
    "plt.xlabel('Time (seconds)')\n",
    "plt.ylabel('Amplitude')\n",
    "plt.title('Normalized CO2 concentration artificially sampled at 3000 Hz')\n",
    "plt.plot(data_time_samples[:len(co2_fit_int)], co2_fit_int)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 75
    },
    "executionInfo": {
     "elapsed": 47510,
     "status": "ok",
     "timestamp": 1615560738335,
     "user": {
      "displayName": "Walt Gurley",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14Gg9eQ8NtdhzyTtyTqhb-4LANYN8W8pA7TUUhK54=s64",
      "userId": "03624837261806285415"
     },
     "user_tz": 300
    },
    "id": "0ZAQo4lnjMZz",
    "outputId": "56e11320-8079-4046-d62c-423dfc1689c8"
   },
   "outputs": [],
   "source": [
    "# The sample rate of our data is approximately 0.0000004 Hz (12 samples / year),\n",
    "# we speed it up about 10^10 times (3000 Hz)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "qPZf8HifA3lr"
   },
   "source": [
    "**Question:** Does this audification provide you with any insight into the data or help you pick out any discernable patterns?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "-1KTsWrUQZZ5"
   },
   "source": [
    "## Parameter mapping\n",
    "\n",
    "Parameter mapping is the process of mapping data to properties of sound. As previously mentioned, these properties can include pitch (frequency), amplitude (loudness), tempo (beat speed), and timbre (quality). The spatial position of sound (for example, panning audio to the left or right channel in a stereo mix) can also be used as a mapping property.\n",
    " \n",
    "As opposed to audification, mapping data to sound properties provides more options and opportunities to represent the features of a dataset.\n",
    "\n",
    "In this demonstration we will only explore using variations in pitch, tempo, and timbre to represent monthly atmospheric CO<sup>2</sup> values."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "er5glbZEwrOK"
   },
   "source": [
    "### Subsample CO<sup>2</sup> concentration data for sonification\n",
    "\n",
    "We will subsample our data to create a smaller dataset that will be appropriate for generating shorter demo sonifications. Additionally, due to some constraints with audio playback in Google Colab we have to create audio files of our sonifications and then load them into our notebook for playback. In this case, smaller files are preferable for shorter load times."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 450
    },
    "executionInfo": {
     "elapsed": 47496,
     "status": "ok",
     "timestamp": 1615560738335,
     "user": {
      "displayName": "Walt Gurley",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14Gg9eQ8NtdhzyTtyTqhb-4LANYN8W8pA7TUUhK54=s64",
      "userId": "03624837261806285415"
     },
     "user_tz": 300
    },
    "id": "9Ar1VxrswrOL",
    "outputId": "e4f9a2b2-137d-4c46-a7a7-b017b2aeeb2c"
   },
   "outputs": [],
   "source": [
    "# Create a subsample of the CO2 data from the year 2000 to present (co2_modern)\n",
    "\n",
    "\n",
    "# We are only going to be working with the measured CO2 concentration values\n",
    "# moving forward, so we will store it in a variable (co2_ppm)\n",
    "\n",
    "\n",
    "# Print the data\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 282
    },
    "executionInfo": {
     "elapsed": 47692,
     "status": "ok",
     "timestamp": 1615560738553,
     "user": {
      "displayName": "Walt Gurley",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14Gg9eQ8NtdhzyTtyTqhb-4LANYN8W8pA7TUUhK54=s64",
      "userId": "03624837261806285415"
     },
     "user_tz": 300
    },
    "id": "fC4WQzC0wrOP",
    "outputId": "d8c7c8b0-cb29-456c-99fa-b3a13214a70f"
   },
   "outputs": [],
   "source": [
    "# Plot the subsampled CO2 concentration data\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "executionInfo": {
     "elapsed": 47682,
     "status": "ok",
     "timestamp": 1615560738554,
     "user": {
      "displayName": "Walt Gurley",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14Gg9eQ8NtdhzyTtyTqhb-4LANYN8W8pA7TUUhK54=s64",
      "userId": "03624837261806285415"
     },
     "user_tz": 300
    },
    "id": "R20VNbUnCOv9",
    "outputId": "60ace97b-ac81-4416-ebba-7e665a7f04f1"
   },
   "outputs": [],
   "source": [
    "# Check out some basic statistics of the subsampled data\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "jlv18L5UEIiT"
   },
   "source": [
    "**Question:** Given the values of the descriptive statistics, could we directly use the CO<sup>2</sup> concentration data as frequencies (in Hz) in a sonification?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "6uZN64XgW4qI"
   },
   "source": [
    "### Functions for generating audio streams and creating audio files\n",
    "\n",
    "Three functions are provided for processing numerical data to create a sonification in which data values are represented as musical pitch (frequency). The three functions are `create_audio_stream`, `create_audio_file`, and `data_to_audio`. In this workshop we will mainly use the `data_to_audio`. Each function is briefly described here and documented in the code cell below.\n",
    "\n",
    "The function `create_audio_stream` creates a music21 audio stream that is playable in local Jupyter notebook using the call `[stream_name].show('midi')`. This functionality is not available in Google Colab and other notebook editors such as VS Code and Jupyter Lab, so we must also create an audio file for playback using the function `create_audio_file` to listen to sonifications in these environments.\n",
    "\n",
    "#### The `data_to_audio` function\n",
    "\n",
    "The two functions above, `create_audio_stream` and `create_audio_file` have been combined into the function `data_to_audio` to streamline the process of creating playable audio in environments outside of Jupyter Notebook. However, each of the previous two functions can still be called individually.\n",
    "\n",
    "`data_to_audio` has four parameters:\n",
    "\n",
    "- `notes` - a List of music21 Note objects\n",
    "\n",
    "- `file_name` - The name of the file to write. Name must include audio file extension type. Valid types include .flac and .wav\n",
    "\n",
    "- `bpm` - beats per minute, or the speed of the sonification (default value is 120, each note is treated as a 16th note and four notes constitute one beat)\n",
    "\n",
    "- `instrument_name` - the name of a synthesized instrument to play the notes (default value is \"Piano\"). A list of instrument names is available in the [music21 documentation](http://web.mit.edu/music21/doc/moduleReference/moduleInstrument.html#module-music21.instrument). (These synthesized instruments are provided through the soundfont file.)\n",
    "\n",
    "These three arguments provide the data mappings to frequency (the `notes` argument), tempo (the `bpm` argument), and timbre (the `instrument_name` argument).\n",
    "\n",
    "**Run the cell below to define these three functions**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "executionInfo": {
     "elapsed": 47680,
     "status": "ok",
     "timestamp": 1615560738554,
     "user": {
      "displayName": "Walt Gurley",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14Gg9eQ8NtdhzyTtyTqhb-4LANYN8W8pA7TUUhK54=s64",
      "userId": "03624837261806285415"
     },
     "user_tz": 300
    },
    "id": "-GGk-mHVXwWD"
   },
   "outputs": [],
   "source": [
    "def create_audio_stream(notes, bpm=120, instrument_name='Piano'):\n",
    "    \"\"\"\n",
    "    Creates and returns a music21 Stream object of notes.\n",
    "\n",
    "    Parameters\n",
    "    ----------\n",
    "    notes : Sequence[music21.note.Note]\n",
    "        A sequence of music21 Note objects.\n",
    "    bpm : int, optional\n",
    "        The beats per minute of the stream, the tempo (speed) of the sonification. The default value is 120 bpm, each note is treated as a 16th note and four notes constitute one beat.\n",
    "    instrument_name : str, optional\n",
    "        The name of a synthesized instrument to play the notes. If no value is provided, or the name is not a valid instrument, the default \"piano\" instrument is used. For a list of valid instrument names see the [music21 Instrument module documentation](https://web.mit.edu/music21/doc/moduleReference/moduleInstrument.html).\n",
    "\n",
    "    Returns\n",
    "    -------\n",
    "    music21.stream.base.Stream\n",
    "        A music21 Stream object\n",
    "\n",
    "    \"\"\"\n",
    "    # Create a new music21 stream object to add notes to\n",
    "    new_stream = stream.Stream()\n",
    "\n",
    "    # Set the tempo string of the stream\n",
    "    new_stream.append(tempo.MetronomeMark(number=bpm))\n",
    "\n",
    "    # Set the instrument to play the stream notes\n",
    "    new_stream.append(getattr(instrument, instrument_name)())\n",
    "\n",
    "    # Iterate over the notes provided in the series\n",
    "    for this_note in notes:\n",
    "        # Append the note to the new stream\n",
    "        new_stream.append(this_note)\n",
    "        # Set the duration of the note as a sixteenth note (0.25 of quarter note)\n",
    "        this_note.duration.quarterLength = 0.25\n",
    "    # Return a music21 stream object\n",
    "    return new_stream\n",
    "\n",
    "def create_audio_file(stream, file_name, keep_midi=False):\n",
    "    \"\"\"\n",
    "    Create an audio file using the given music21 stream and the given file_name.\n",
    "    \n",
    "    It writes a MIDI file using music21 and then uses the newly written MIDI file to create the specified audio file using FluidSynth midi to audio conversion.\n",
    "\n",
    "    Parameters\n",
    "    ----------\n",
    "    stream : music21.stream.base.Stream\n",
    "        A music 21 Stream object containing notes.\n",
    "    file_name : str\n",
    "        The name of the file to write. Name must include audio file extension type. Valid types include .flac and .wav.\n",
    "    keep_midi : bool, optional\n",
    "        Keep the intermediate MIDI file if True. Default is False.\n",
    "\n",
    "    Returns\n",
    "    -------\n",
    "    None\n",
    "\n",
    "    \"\"\"\n",
    "    # If the MIDI file is not needed, set the MIDI file name to \"None\" to create\n",
    "    # a temporary file with the music21 \"stream.write()\"\" method. If keeping\n",
    "    # MIDI file, create string for MIDI file path.\n",
    "    if not keep_midi:\n",
    "        midi_file_name = None\n",
    "    else:\n",
    "        # File name with MIDI extension\n",
    "        midi_file_name = file_name.split('.')[0] + '.mid'\n",
    "    \n",
    "    # Use the music21 stream object write function to create a MIDI file and\n",
    "    # store the path to the new file\n",
    "    midi_file_path = stream.write('midi', midi_file_name)\n",
    "\n",
    "    # Use the FluidSynth module to call the sound font and convert the newly \n",
    "    # created MIDI file to the specified file_name using the midi_to_audio function\n",
    "    FluidSynth('FluidR3Mono_GM.sf3').midi_to_audio(\n",
    "        midi_file_path, file_name\n",
    "    )\n",
    "\n",
    "    # Delete the MIDI file if \"keep_midi\" not set to True\n",
    "    # if not keep_midi:\n",
    "    #     os.remove(midi_file_name)\n",
    "\n",
    "def data_to_audio(notes, file_name, bpm=120, instrument_name='Piano'):\n",
    "    '''\n",
    "    Processes a sequence of music21 Note objects to create a music21 stream and generate an audio file with optional musical style parameters.\n",
    "\n",
    "    Combines the create_audio_stream and create_audio_file into one functions.\n",
    "\n",
    "    Parameters\n",
    "    ----------\n",
    "    notes : Sequence[music21.note.Note]\n",
    "        A sequence of music21 Note objects\n",
    "    file_name : str\n",
    "        The name of the file to write. Name must include audio file extension type. Valid types include .flac and .wav\n",
    "    bpm : int, optional\n",
    "        The beats per minute of the stream, the tempo (speed) of the sonification. The default value is 120 bpm, each note is treated as a 16th note and four notes constitute one beat.\n",
    "    instrument_name : str, optional\n",
    "        The name of a synthesized instrument to play the notes. If no value is provided, or the name is not a valid instrument, the default \"piano\" instrument is used.\n",
    "    \n",
    "    Returns\n",
    "    -------\n",
    "    None\n",
    "\n",
    "    '''\n",
    "\n",
    "    # Generate the music21 Stream object from the list of notes\n",
    "    notes_stream = create_audio_stream(\n",
    "        notes, bpm=bpm, instrument_name=instrument_name\n",
    "    )\n",
    "    # Generate an audio file from the music21 Stream object\n",
    "    create_audio_file(notes_stream, file_name=file_name)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Uncomment the line below to print out a list of instruments contained in the\n",
    "# provided soundfont\n",
    "\n",
    "# !echo \"inst 1\" | fluidsynth ./FluidR3Mono_GM.sf3"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "WQcth6LrwrOR"
   },
   "source": [
    "### Parameter mapping using CO<sup>2</sup> concentration data values as frequency (pitch)\n",
    "\n",
    "In this demonstration we will use a function that directly maps a numeric value to a frequency and the corresponding scientific pitch notation value.\n",
    "\n",
    "For example, a data value of 440 would be interpreted as the frequency 440 Hz. This frequency would then be converted to scientific pitch notation as the pitch A4 (A in octave 4).\n",
    "\n",
    "Moving forward we will be thinking of frequency in terms of Hz and scientific pitch notation. It might be helpful to reference this [table of piano key pitches and frequencies](https://en.wikipedia.org/wiki/Piano_key_frequencies#List)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "yxC5HdyPYDPC"
   },
   "source": [
    "We will use the defined function `value_to_pitch` to translate data values as frequency to pitch. This function takes one argument:\n",
    "\n",
    "- `value` - a data value interpreted as a frequency"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "executionInfo": {
     "elapsed": 47677,
     "status": "ok",
     "timestamp": 1615560738555,
     "user": {
      "displayName": "Walt Gurley",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14Gg9eQ8NtdhzyTtyTqhb-4LANYN8W8pA7TUUhK54=s64",
      "userId": "03624837261806285415"
     },
     "user_tz": 300
    },
    "id": "yhuk3043wrOU"
   },
   "outputs": [],
   "source": [
    "def value_to_pitch(value):\n",
    "    \"\"\"\n",
    "    Generate a music21 Note object with a set pitch based on the provided value interpreted as a frequency or music21 Rest object of silence.\n",
    "    \n",
    "    It returns the pitch of the provided frequency value if the value is greater than zero. If the value is less than zero it returns a rest (i.e., silence).\n",
    "\n",
    "    Parameters\n",
    "    ----------\n",
    "    value : int | float\n",
    "        A numerical data value to be directly mapped to frequency and used to generate a pitched musical note. \n",
    "\n",
    "    Returns\n",
    "    -------\n",
    "    music21.note.Note | music21.note.Rest\n",
    "        A music21 Note object with a set pitch based on the provided value interpreted as a frequency. Values less than zero will return a music21 Rest object.\n",
    "\n",
    "    \"\"\"\n",
    "    # Test if the frequency value is greater than zero\n",
    "    if (value > 0):\n",
    "        # Create a music21 Note object\n",
    "        converted_note = note.Note()\n",
    "        # Set the pitch of the Note object based on the supplied frequency value\n",
    "        converted_note.pitch.frequency = value\n",
    "        # Return the Note object with the assigned pitch\n",
    "        return converted_note\n",
    "    # Retrun a Rest value (silence)\n",
    "    return note.Rest()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "executionInfo": {
     "elapsed": 47676,
     "status": "ok",
     "timestamp": 1615560738556,
     "user": {
      "displayName": "Walt Gurley",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14Gg9eQ8NtdhzyTtyTqhb-4LANYN8W8pA7TUUhK54=s64",
      "userId": "03624837261806285415"
     },
     "user_tz": 300
    },
    "id": "JnNguBMc4PmW"
   },
   "outputs": [],
   "source": [
    "# Test the value_to_pitch function\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "PPc2WNAxMGKD"
   },
   "source": [
    "The `value_to_pitch` function has returned a [music21 Note object](http://web.mit.edu/music21/doc/moduleReference/moduleNote.html#module-music21.note). A Note object has many properties. For this workshop, the most important properties are the note name and pitch frequency. We can get a full descriptive name by calling the property `fullName` on a Note object. We can get the frequency of a note by accessing the property `pitch.frequency` on a Note object."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 35
    },
    "executionInfo": {
     "elapsed": 47665,
     "status": "ok",
     "timestamp": 1615560738556,
     "user": {
      "displayName": "Walt Gurley",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14Gg9eQ8NtdhzyTtyTqhb-4LANYN8W8pA7TUUhK54=s64",
      "userId": "03624837261806285415"
     },
     "user_tz": 300
    },
    "id": "zaxkfC8FvbRr",
    "outputId": "60ded02e-3394-42f7-be44-619840ef6162"
   },
   "outputs": [],
   "source": [
    "# Print out full note name information by calling \"fullName\"\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "executionInfo": {
     "elapsed": 47663,
     "status": "ok",
     "timestamp": 1615560738556,
     "user": {
      "displayName": "Walt Gurley",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14Gg9eQ8NtdhzyTtyTqhb-4LANYN8W8pA7TUUhK54=s64",
      "userId": "03624837261806285415"
     },
     "user_tz": 300
    },
    "id": "LuV6k2SD5-g6",
    "outputId": "c7d75f9c-0228-4788-99ad-733892a1cc25"
   },
   "outputs": [],
   "source": [
    "# Print out the frequency information by calling \"pitch.frequency\"\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "executionInfo": {
     "elapsed": 47657,
     "status": "ok",
     "timestamp": 1615560738557,
     "user": {
      "displayName": "Walt Gurley",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14Gg9eQ8NtdhzyTtyTqhb-4LANYN8W8pA7TUUhK54=s64",
      "userId": "03624837261806285415"
     },
     "user_tz": 300
    },
    "id": "80rVTGkawrOW",
    "outputId": "a6de2c3a-7f44-4000-8d6d-0ec2f7eea3b3"
   },
   "outputs": [],
   "source": [
    "# Map CO2 concentration values directly to pitch\n",
    "\n",
    "\n",
    "#Print the data\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "_The following cell contains the code for generating a music21 Stream object and playing the audio in the notebook without having to generate any files. This process uses the music21 Stream functionality to playback MIDI audio._\n",
    "\n",
    "_I have only found this to work in a Jupyter Notebook environment. As such, we will skip this cell, but note this ability if you are using a Jupyter Notebook environment._"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "executionInfo": {
     "elapsed": 47789,
     "status": "ok",
     "timestamp": 1615560738690,
     "user": {
      "displayName": "Walt Gurley",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14Gg9eQ8NtdhzyTtyTqhb-4LANYN8W8pA7TUUhK54=s64",
      "userId": "03624837261806285415"
     },
     "user_tz": 300
    },
    "id": "sP72yepSwrOY"
   },
   "outputs": [],
   "source": [
    "# Create an audio stream of the raw_pitch values using the create_audio_stream\n",
    "# function. Pass in the series of notes (raw_pitch), a tempo in bpm, and an\n",
    "# instrument name from the list located in the right-hand column of this\n",
    "# music21 documentation page: https://web.mit.edu/music21/doc/moduleReference/moduleInstrument.html\n",
    "raw_pitch_stream = create_audio_stream(raw_pitch, 180)\n",
    "\n",
    "# If running in Jupyter Notebook a local machine, uncomment the line below to\n",
    "# play the MIDI data in the Jupyter notebook without having to create a file\n",
    "\n",
    "# raw_pitch_stream.show('midi')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "i5Omr_loyux6"
   },
   "source": [
    "In the next cell we generate an audio file from the data. We will create a FLAC audio file, but it is also possible to create other standard audio file formats such as mp3, mp4, and wave files by changing the file extension.\n",
    "\n",
    "When running this notebook in Colab I found that FLAC audio files tended to be smaller and loaded faster than other audio file types when using the Audio tool."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create an audio file of the pitch data\n",
    "\n",
    "\n",
    "# Load the newly created audio file for playback\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 282
    },
    "executionInfo": {
     "elapsed": 53152,
     "status": "ok",
     "timestamp": 1615560744081,
     "user": {
      "displayName": "Walt Gurley",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14Gg9eQ8NtdhzyTtyTqhb-4LANYN8W8pA7TUUhK54=s64",
      "userId": "03624837261806285415"
     },
     "user_tz": 300
    },
    "id": "HJ0tcDEeyoGy",
    "outputId": "195f4c4a-447f-4d54-c1a4-0658f22abe21"
   },
   "outputs": [],
   "source": [
    "# Plot the data for visual reference\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "HCZWd0-QUcli"
   },
   "source": [
    "#### Activity: Sonify seasonally adjusted CO<sup>2</sup> concentrations using `value_to_pitch`\n",
    "\n",
    "Create a sonification of the seasonally adjusted CO<sup>2</sup> concentration data from the `co2_modern` dataset using the `value_to_pitch` method."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 75,
     "output_embedded_package_id": "1HwpocNNbjHe4nYaXbpIGIwtaMCgbJYlv"
    },
    "executionInfo": {
     "elapsed": 53141,
     "status": "ok",
     "timestamp": 1615560744082,
     "user": {
      "displayName": "Walt Gurley",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14Gg9eQ8NtdhzyTtyTqhb-4LANYN8W8pA7TUUhK54=s64",
      "userId": "03624837261806285415"
     },
     "user_tz": 300
    },
    "id": "8UH95jGTTa-P",
    "outputId": "e7d5469e-581e-43ab-e2f3-1c68d09fd32b"
   },
   "outputs": [],
   "source": [
    "# Store the seasonally adjusted concentration data from \"co2_modern\" in a variable\n",
    "\n",
    "\n",
    "# Use the apply() method to call value_to_pitch on each row of the\n",
    "# seasonally adjusted concentration data\n",
    "\n",
    "\n",
    "# Create an audio file of the pitch data\n",
    "\n",
    "\n",
    "# Load the audio file for playback\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "D7slkEh5YYBg"
   },
   "source": [
    "**Question:** Are there any shortcomings with the method we used to create this sonification?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "rueqdiqmwrOc"
   },
   "source": [
    "### Sonification using values mapped to linear pitch range\n",
    "\n",
    "In this demonstration we will use a function that maps a data value from the dataset range (dataMin - dataMax) to the pitch range of a standard 88 key piano (~27 Hz – ~4186 Hz).\n",
    "\n",
    "    (dataMin)         (value)         (dataMax)\n",
    "        |----------------o----------------| Data scale\n",
    "                          \\\n",
    "                           \\\n",
    "                            \\       \n",
    "     (27 Hz)           (mapped value)           (4186 Hz)\n",
    "        |---------------------o---------------------| Frequency range scale (88 key piano)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "t6F2Nh9Igz9U"
   },
   "source": [
    "We will use the defined function `map_value_to_pitch_range` to map data values from the data domain to a pitch range. This function takes two arguments:\n",
    "- `value` - the data value to be mapped to the new pitch range\n",
    "- `data_min_max` - a list containing the min and max of the dataset ([min, max])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "executionInfo": {
     "elapsed": 53139,
     "status": "ok",
     "timestamp": 1615560744082,
     "user": {
      "displayName": "Walt Gurley",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14Gg9eQ8NtdhzyTtyTqhb-4LANYN8W8pA7TUUhK54=s64",
      "userId": "03624837261806285415"
     },
     "user_tz": 300
    },
    "id": "z93Dh_EEwrOt"
   },
   "outputs": [],
   "source": [
    "def map_value_to_pitch_range(value, data_min_max):\n",
    "    \"\"\"\n",
    "    Map a data value from the dataset domain (data min - data max) to the frequency range of a standard 88 key piano (~27 Hz - ~4186 Hz).\n",
    "    \n",
    "    It returns a music21 Note object with the pitch of the data value mapped to the frequency range.\n",
    "\n",
    "    Parameters\n",
    "    ----------\n",
    "    value : int | float\n",
    "        A numerical data value to be mapped to the frequency range based on the minimum and maximum values of the whole dataset.\n",
    "    data_min_max : List[int | float]\n",
    "        A list containing the minimum and maximum of the dataset in the form [min, max].\n",
    "\n",
    "    Returns\n",
    "    -------\n",
    "    music21.note.Note\n",
    "        A music21 Note object with a specified pitch based on the mapping of the provided value to the new pitch range based on the min and max of the dataset.\n",
    "\n",
    "    \"\"\"\n",
    "    # Create the music21 Note object\n",
    "    converted_note = note.Note()\n",
    "\n",
    "    # Set the data range using supplied min and max\n",
    "    data_range = data_min_max[1] - data_min_max[0]\n",
    "\n",
    "    # Frequency of the lowest note on a standard 88 key piano\n",
    "    MIN_HZ = 27.50\n",
    "    # Frequency of the highest note on a standard 88 key piano\n",
    "    MAX_HZ = 4186.01\n",
    "\n",
    "    # Set the pitch of each Note object mapping from the data range to the\n",
    "    # frequency range of a standard 88 key piano\n",
    "    converted_note.pitch.frequency = (\n",
    "        ((value - data_min_max[0]) * (MAX_HZ - MIN_HZ)) / data_range\n",
    "    ) + MIN_HZ\n",
    "    return converted_note"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "executionInfo": {
     "elapsed": 53131,
     "status": "ok",
     "timestamp": 1615560744083,
     "user": {
      "displayName": "Walt Gurley",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14Gg9eQ8NtdhzyTtyTqhb-4LANYN8W8pA7TUUhK54=s64",
      "userId": "03624837261806285415"
     },
     "user_tz": 300
    },
    "id": "RVUWtlLn8GdE",
    "outputId": "b2b9c47c-4b78-4046-9671-ef85253d72c3"
   },
   "outputs": [],
   "source": [
    "# Test the map_value_to_pitch_range function\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "executionInfo": {
     "elapsed": 53130,
     "status": "ok",
     "timestamp": 1615560744084,
     "user": {
      "displayName": "Walt Gurley",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14Gg9eQ8NtdhzyTtyTqhb-4LANYN8W8pA7TUUhK54=s64",
      "userId": "03624837261806285415"
     },
     "user_tz": 300
    },
    "id": "mhEaI1pawrOv"
   },
   "outputs": [],
   "source": [
    "# Calculate the minimum and maximum values of the CO2 concentration data\n",
    "\n",
    "\n",
    "# Convert values to pitch range and store in a variable named \"pitch_range\"\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 75,
     "output_embedded_package_id": "1rAiAzFye7jOzRKPhunaJ-rmeFsAg2fIw"
    },
    "executionInfo": {
     "elapsed": 56030,
     "status": "ok",
     "timestamp": 1615560746997,
     "user": {
      "displayName": "Walt Gurley",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14Gg9eQ8NtdhzyTtyTqhb-4LANYN8W8pA7TUUhK54=s64",
      "userId": "03624837261806285415"
     },
     "user_tz": 300
    },
    "id": "0NAe1HKfe6HG",
    "outputId": "a861d08d-3a27-4eaa-e760-05b9118d61f1"
   },
   "outputs": [],
   "source": [
    "# Create an audio file of the pitch data\n",
    "\n",
    "\n",
    "# Load the newly created audio file for playback\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 369
    },
    "executionInfo": {
     "elapsed": 56014,
     "status": "ok",
     "timestamp": 1615560746998,
     "user": {
      "displayName": "Walt Gurley",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14Gg9eQ8NtdhzyTtyTqhb-4LANYN8W8pA7TUUhK54=s64",
      "userId": "03624837261806285415"
     },
     "user_tz": 300
    },
    "id": "3zuCH3KmdvTp",
    "outputId": "c5ce53cd-f928-4a2f-9c46-25c4bf9ade4e"
   },
   "outputs": [],
   "source": [
    "# Plot the actual concentration data and the mapped frequency data for visual reference\n",
    "plt.figure(figsize=[12, 5])\n",
    "plt.subplot(1, 2, 1)\n",
    "plt.title('CO2 concentration over time')\n",
    "plt.plot(co2_ppm)\n",
    "\n",
    "plt.subplot(1, 2, 2)\n",
    "plt.title('CO2 concentration mapped to\\n frequency range over time (seconds)')\n",
    "# Create the time scale for the audio file using the bpm (the bpm\n",
    "# value passed into the data_to_audio function for this sonification)\n",
    "bpm = 120\n",
    "time_scale = np.linspace(0, (len(pitch_range) / 4) / bpm * 60, len(pitch_range))\n",
    "plt.plot(time_scale, pitch_range.apply(lambda x: x.pitch.frequency))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "GrpJOvczUYVa"
   },
   "source": [
    "#### Activity: Sonify seasonally adjusted CO<sup>2</sup> concentrations using `map_value_to_pitch_range`\n",
    "\n",
    "Create a sonification of the seasonally adjusted CO<sup>2</sup> concentration data using the `map_value_to_pitch_range` method."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 75,
     "output_embedded_package_id": "136F2RhRrOZF8ZYyEYlGhd7SXtUPEDcw9"
    },
    "executionInfo": {
     "elapsed": 58328,
     "status": "ok",
     "timestamp": 1615560749323,
     "user": {
      "displayName": "Walt Gurley",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14Gg9eQ8NtdhzyTtyTqhb-4LANYN8W8pA7TUUhK54=s64",
      "userId": "03624837261806285415"
     },
     "user_tz": 300
    },
    "id": "dkfjRYl9VF7N",
    "outputId": "f2c6e1f5-4d0e-4170-961f-a72db95f6dd1"
   },
   "outputs": [],
   "source": [
    "# Calculate the min and max values of the seasonally adjusted C02 concentration\n",
    "# data\n",
    "\n",
    "\n",
    "# Use the apply() method to call map_value_to_pitch_range on each row of the\n",
    "# seasonally adjusted concentration data\n",
    "\n",
    "\n",
    "# Create the audio file from the audio stream\n",
    "\n",
    "\n",
    "# Load the audio file for playback\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "KvZvi4i-cnpn"
   },
   "source": [
    "### Sonification using values mapped to an exponential pitch range\n",
    "\n",
    "The frequency ranges between pitches are not linear. For example, the frequency of note A4 is 440 Hz. To get to the next octave of this note we must double the frequency of A4, giving A5 as 880 Hz (i.e., A6 = 1760, A3 = 220, and so on).\n",
    "\n",
    "      A3        A4              A5                                A6                                                              A7\n",
    "    (220hz)  (440 Hz)         (880 Hz)                         (1760 Hz)                                                       (3520 Hz)\n",
    "       |--------|----------------|--------------------------------|----------------------------------------------------------------|\n",
    "\n",
    "In order to map our values to an appropriate pitch scale, we can apply a log transform to pitch range. This makes it easier to discern pitches at lower frequencies and produces a more natural and pleasing sound.\n",
    "\n",
    "In this demonstration we will use a function that maps a data value from the dataset range (dataMin - dataMax) to the frequency range of a standard 88 key piano (\\~27 Hz – \\~4186 Hz) over an exponential scale."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "gaYvm_QxhTXr"
   },
   "source": [
    "We will use the defined function `map_value_to_pitch_range_exp_scale` to map data values from the data domain to a log scale pitch range. This function takes two arguments:\n",
    "- `data_value` - the data value to be mapped to the new pitch range\n",
    "- `data_min_max` - a list containing the min and max of the dataset ([min, max])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "executionInfo": {
     "elapsed": 58331,
     "status": "ok",
     "timestamp": 1615560749330,
     "user": {
      "displayName": "Walt Gurley",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14Gg9eQ8NtdhzyTtyTqhb-4LANYN8W8pA7TUUhK54=s64",
      "userId": "03624837261806285415"
     },
     "user_tz": 300
    },
    "id": "zvu14U2ewrOm"
   },
   "outputs": [],
   "source": [
    "\n",
    "def map_value_to_pitch_range_exp_scale(value, data_min_max):\n",
    "    \"\"\"\n",
    "    Map a data value from the dataset domain (data min - data\n",
    "    max) to the log2 frequency range of a standard 88 key piano (~27 Hz – ~4186\n",
    "    Hz). It returns a music21 Note object with the pitch of the mapped frequency\n",
    "    value.\n",
    "\n",
    "    Parameters\n",
    "    ----------\n",
    "    value : int | float\n",
    "        A numerical data value to be mapped to the frequency range based on the minimum and maximum values of the whole dataset.\n",
    "    data_min_max : List[int | float]\n",
    "        A list containing the minimum and maximum of the dataset in the form [min, max].\n",
    "\n",
    "    Returns\n",
    "    -------\n",
    "    music21.note.Note\n",
    "        A music21 Note object with a specified pitch based on the mapping of the provided value to the new pitch range generated by the min and max of the dataset.\n",
    "\n",
    "    \"\"\"\n",
    "    # Create the music21 Note object\n",
    "    converted_note = note.Note()\n",
    "\n",
    "    # Set the data range using supplied min and max\n",
    "    data_range = data_min_max[1] - data_min_max[0]\n",
    "\n",
    "    # Frequency of the lowest note on a standard 88 key piano\n",
    "    MIN_HZ = 27.50\n",
    "    # Frequency of the highest note on a standard 88 key piano\n",
    "    MAX_HZ = 4186.01\n",
    "\n",
    "    # Set the pitch of each Note object, mapping from the data range to a\n",
    "    # log base 2 frequency range of a standard 88 key piano\n",
    "    #\n",
    "    # https://stackoverflow.com/questions/19472747/convert-linear-scale-to-logarithmic\n",
    "    #           x - x0\n",
    "    # log(y) = ------- * (log(y1) - log(y0)) + log(y0)\n",
    "    #          x1 - x0\n",
    "    converted_note.pitch.frequency = np.exp2(\n",
    "        (value - data_min_max[0]) / data_range *\n",
    "        (np.log2(MAX_HZ) - np.log2(MIN_HZ)) +\n",
    "        np.log2(MIN_HZ)\n",
    "    )\n",
    "    return converted_note"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "executionInfo": {
     "elapsed": 58324,
     "status": "ok",
     "timestamp": 1615560749330,
     "user": {
      "displayName": "Walt Gurley",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14Gg9eQ8NtdhzyTtyTqhb-4LANYN8W8pA7TUUhK54=s64",
      "userId": "03624837261806285415"
     },
     "user_tz": 300
    },
    "id": "rBK8oSmrTAp2",
    "outputId": "ad72de5b-3126-4fd7-b718-e27661a137cb"
   },
   "outputs": [],
   "source": [
    "# Test the map_value_to_pitch_range_exp_scale function\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "executionInfo": {
     "elapsed": 58323,
     "status": "ok",
     "timestamp": 1615560749331,
     "user": {
      "displayName": "Walt Gurley",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14Gg9eQ8NtdhzyTtyTqhb-4LANYN8W8pA7TUUhK54=s64",
      "userId": "03624837261806285415"
     },
     "user_tz": 300
    },
    "id": "5nYRAPzkwrOo"
   },
   "outputs": [],
   "source": [
    "# Convert values to exponential pitch range and store in a variable named\n",
    "# \"pitch_range_exp\"\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 75
    },
    "executionInfo": {
     "elapsed": 3765,
     "status": "ok",
     "timestamp": 1615560798982,
     "user": {
      "displayName": "Walt Gurley",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14Gg9eQ8NtdhzyTtyTqhb-4LANYN8W8pA7TUUhK54=s64",
      "userId": "03624837261806285415"
     },
     "user_tz": 300
    },
    "id": "k7eFc7kufKed",
    "outputId": "f77a0598-c293-42df-fde7-76e77203c168"
   },
   "outputs": [],
   "source": [
    "# Create an audio file of the pitch data\n",
    "\n",
    "\n",
    "# Load the newly created audio file for playback\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 353
    },
    "executionInfo": {
     "elapsed": 3997,
     "status": "ok",
     "timestamp": 1615560799223,
     "user": {
      "displayName": "Walt Gurley",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14Gg9eQ8NtdhzyTtyTqhb-4LANYN8W8pA7TUUhK54=s64",
      "userId": "03624837261806285415"
     },
     "user_tz": 300
    },
    "id": "fFlUrzq_b_MI",
    "outputId": "2256c128-0716-4e0a-a04c-bdeb13e8037d"
   },
   "outputs": [],
   "source": [
    "# Plot the actual concentration data and the mapped frequency data for visual reference\n",
    "plt.figure(figsize=[12, 5])\n",
    "plt.subplot(1, 2, 1)\n",
    "plt.title('CO2 concentration over time')\n",
    "plt.plot(co2_ppm)\n",
    "\n",
    "plt.subplot(1, 2, 2)\n",
    "plt.title('CO2 concentration mapped to frequency over time')\n",
    "# Calculate the time scale for the audio we generated using the bpm (the bpm\n",
    "# value passed into the data_to_audio function for this sonification)\n",
    "bpm = 120\n",
    "time_scale = np.linspace(0, len(pitch_range) / 4 / bpm * 60, len(pitch_range))\n",
    "\n",
    "plt.plot(time_scale, pitch_range_exp.apply(lambda x: x.pitch.frequency))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "b2QkwZQ-UTlq"
   },
   "source": [
    "#### Activity: Sonify seasonally adjusted CO<sup>2</sup> concentrations using `map_value_to_pitch_range_exp_scale`\n",
    "\n",
    "Create a sonification of the seasonally adjusted CO<sup>2</sup> concentration data using the `map_value_to_pitch_range_exp_scale` method."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 75,
     "output_embedded_package_id": "1ZF1suEEpPQHmOdIIlQo0-ZkCaTyyo7fU"
    },
    "executionInfo": {
     "elapsed": 6691,
     "status": "ok",
     "timestamp": 1615560801921,
     "user": {
      "displayName": "Walt Gurley",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14Gg9eQ8NtdhzyTtyTqhb-4LANYN8W8pA7TUUhK54=s64",
      "userId": "03624837261806285415"
     },
     "user_tz": 300
    },
    "id": "0qYPT9xgVb7r",
    "outputId": "5603accf-38f6-4363-b873-a3ba1c3f2a72"
   },
   "outputs": [],
   "source": [
    "# Use the apply() method to call map_value_to_pitch_range on each row of the\n",
    "# seasonally adjusted concentration data\n",
    "\n",
    "\n",
    "# Create the audio file from the audio stream\n",
    "\n",
    "\n",
    "# Load the audio file for playback\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "u7DhEXAXwrO3"
   },
   "source": [
    "### Sonification using values mapped to a musical scale\n",
    "\n",
    "In this demonstration we will use a function that maps a data value from the dataset range (dataMin - dataMax) to a specified [musical scale](https://en.wikipedia.org/wiki/Scale_(music)) (i.e., an ordered group of pitches) with the given provided starting note (i.e., the tonic) and an octave range (i.e., the number of notes over which to map your data).\n",
    "\n",
    "Using a musical scale provides us with some options to make our sonification more \"musical\". Previously we were mapping values to frequencies and pitches without consideration for ordering of notes. Predefined musical scales can evoke culturally influenced emotions that can give our sonification an aesthetic feeling. For example, major scales are often interpreted as happy, while minor scales are interpreted as sad."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "FQblzbv6h8AN"
   },
   "source": [
    "We will use the defined function `map_value_to_scale` to map data values from the data domain to a musical scale. This function takes five arguments:\n",
    "- `value` - the data value to be mapped to the new pitch range\n",
    "- `data_min_max` - a list containing the min and max of the dataset ([min, max])\n",
    "- `tonic` - a string containing a valid scientific pitch notation for the first note of the scale\n",
    "- `scale_name` - a string containing a valid scale subclass name from the music21 scale class (default value is \"MajorScale\"). A list of scale names is available in the [music21 documentation](http://web.mit.edu/music21/doc/moduleReference/moduleScale.html#module-music21.scale)\n",
    "- `octave_range` - a list containing the min and max octave numbers over which to map the pitches (default value is [3, 5])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "executionInfo": {
     "elapsed": 6687,
     "status": "ok",
     "timestamp": 1615560801922,
     "user": {
      "displayName": "Walt Gurley",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14Gg9eQ8NtdhzyTtyTqhb-4LANYN8W8pA7TUUhK54=s64",
      "userId": "03624837261806285415"
     },
     "user_tz": 300
    },
    "id": "-TIcOeWuwrO3"
   },
   "outputs": [],
   "source": [
    "def map_value_to_scale(value, data_min_max, tonic='c',\n",
    "                       scale_name='MajorScale', octave_range=[3, 5]):\n",
    "    \"\"\"\n",
    "    Map a data value from the dataset domain (data min - data max) to a musical scale with a provided tonic note over a provided octave range.\n",
    "    \n",
    "    It returns a music21 Note object with the pitch of the mapped data value.\n",
    "\n",
    "    Parameters\n",
    "    ----------\n",
    "    value : int | float\n",
    "        A numerical data value to be mapped to the frequency range based on the minimum and maximum values of the whole dataset.\n",
    "    data_min_max : List[int | float]\n",
    "        A list containing the minimum and maximum of the dataset in the form [min, max].\n",
    "    tonic : str, optional\n",
    "        A string containing a valid scientific pitch notation for the first note of the scale.\n",
    "    scale_name : str, optional\n",
    "        A string containing a valid scale subclass name from the music21 scale class. A list of scale names is available in the [music21 documentation](http://web.mit.edu/music21/doc/moduleReference/moduleScale.html#module-music21.scale)\n",
    "    octave_range : List[int]\n",
    "        A list of two integers containing the lower and upper bounds of the octaves over which to map the pitches\n",
    "\n",
    "    Returns\n",
    "    -------\n",
    "    music21.note.Note\n",
    "        A music21 Note object with a specified pitch based on the mapping of the provided value to the new pitch range generated by the min and max of the dataset.\n",
    "    \"\"\"\n",
    "    # Create the music21 Note object\n",
    "    converted_note = note.Note()\n",
    "\n",
    "    # Set the scale using the supplied tonic value\n",
    "    this_scale = getattr(scale, scale_name)(tonic)\n",
    "    # Get the pitches of the scale based on the octave range\n",
    "    scale_pitches = this_scale.getPitches(\n",
    "        tonic + str(octave_range[0]),\n",
    "        tonic + str(octave_range[1]))\n",
    "    \n",
    "    # Get the data range using supplied min and max \n",
    "    dataRange = data_min_max[1] - data_min_max[0]\n",
    "\n",
    "    # Get the position of the note on the scale given the range of the scale\n",
    "    note_position = round(\n",
    "        ((value - data_min_max[0]) * (len(scale_pitches) - 1)) / dataRange)\n",
    "    # Set the pitch of the note\n",
    "    converted_note.pitch = scale_pitches[note_position]\n",
    "    return converted_note"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "f9WZxcng2VQB"
   },
   "source": [
    "The C major scale has a base note (tonic) of C and consists of the notes: C, D, E, F, G, A, B, C."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 35
    },
    "executionInfo": {
     "elapsed": 6685,
     "status": "ok",
     "timestamp": 1615560801923,
     "user": {
      "displayName": "Walt Gurley",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14Gg9eQ8NtdhzyTtyTqhb-4LANYN8W8pA7TUUhK54=s64",
      "userId": "03624837261806285415"
     },
     "user_tz": 300
    },
    "id": "cKNHwhlN2GjZ",
    "outputId": "cb57429c-379f-4ca4-e87d-e7a1557f4d7e"
   },
   "outputs": [],
   "source": [
    "# Test the map_value_to_scale function\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "executionInfo": {
     "elapsed": 6843,
     "status": "ok",
     "timestamp": 1615560802085,
     "user": {
      "displayName": "Walt Gurley",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14Gg9eQ8NtdhzyTtyTqhb-4LANYN8W8pA7TUUhK54=s64",
      "userId": "03624837261806285415"
     },
     "user_tz": 300
    },
    "id": "HGPC7-mdwrO5"
   },
   "outputs": [],
   "source": [
    "# Convert values to musical scale and store in a variable named \"musical_notes\"\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 75,
     "output_embedded_package_id": "1fwapKf46vectCPQRwSsGl6HCR_FPOaWA"
    },
    "executionInfo": {
     "elapsed": 11718,
     "status": "ok",
     "timestamp": 1615560806968,
     "user": {
      "displayName": "Walt Gurley",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14Gg9eQ8NtdhzyTtyTqhb-4LANYN8W8pA7TUUhK54=s64",
      "userId": "03624837261806285415"
     },
     "user_tz": 300
    },
    "id": "GJiYGjeDdyxo",
    "outputId": "066d442a-5e69-4642-b4f2-c9648fefe22d"
   },
   "outputs": [],
   "source": [
    "# Create an audio file of the pitch data\n",
    "\n",
    "\n",
    "# Load the newly created audio file for playback\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 353
    },
    "executionInfo": {
     "elapsed": 11723,
     "status": "ok",
     "timestamp": 1615560806979,
     "user": {
      "displayName": "Walt Gurley",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14Gg9eQ8NtdhzyTtyTqhb-4LANYN8W8pA7TUUhK54=s64",
      "userId": "03624837261806285415"
     },
     "user_tz": 300
    },
    "id": "pfyDmgPhwrO9",
    "outputId": "dcf6600c-e966-406a-f785-3c9637c3fabb"
   },
   "outputs": [],
   "source": [
    "# Plot the actual concentration data and the mapped frequency data for visual\n",
    "# reference\n",
    "plt.figure(figsize=[12, 5])\n",
    "plt.subplot(1, 2, 1)\n",
    "plt.title('CO2 concentration over time')\n",
    "plt.plot(co2_ppm)\n",
    "\n",
    "plt.subplot(1, 2, 2)\n",
    "plt.title('CO2 concentration mapped to frequency over time')\n",
    "# Create the time scale for the audio file using the bpm (the bpm\n",
    "# value passed into the data_to_audio function for this sonification)\n",
    "bpm = 120\n",
    "time_scale = np.linspace(0, (len(pitch_range) / 4) / bpm * 60, len(pitch_range))\n",
    "\n",
    "plt.plot(time_scale, musical_notes.apply(lambda x: x.pitch.frequency))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Gxikh7JcUOcj"
   },
   "source": [
    "#### Activity: Sonify seasonally adjusted CO<sup>2</sup> concentrations using `map_value_to_scale`\n",
    "\n",
    "Create a sonification of the seasonally adjusted CO<sup>2</sup> concentration data using the `map_value_to_scale` method."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 75,
     "output_embedded_package_id": "1dPkePaS-kWT6A9IcCylE_kPTTrmZAudO"
    },
    "executionInfo": {
     "elapsed": 15852,
     "status": "ok",
     "timestamp": 1615560811112,
     "user": {
      "displayName": "Walt Gurley",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14Gg9eQ8NtdhzyTtyTqhb-4LANYN8W8pA7TUUhK54=s64",
      "userId": "03624837261806285415"
     },
     "user_tz": 300
    },
    "id": "4CxWBLuHVmrf",
    "outputId": "874cb975-79b1-43f5-a67a-708ce1693366"
   },
   "outputs": [],
   "source": [
    "# Use the apply() method to call map_value_to_scale on each row of the\n",
    "# seasonally adjusted concentration data\n",
    "\n",
    "\n",
    "\n",
    "# Create the audio file from the audio stream\n",
    "\n",
    "\n",
    "# Load the audio file for playback\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "BA_c8TCS4ZiU"
   },
   "source": [
    "### Sonification playground\n",
    "\n",
    "Use the cells below to generate your own sonifications of the CO<sup>2</sup> concentration data using the provided functions.\n",
    "\n",
    "You can the call the following functions on a Pandas Series using the `apply()` function:\n",
    "\n",
    "- `value_to_pitch(value_as_frequency)`\n",
    "\n",
    "- `map_value_to_pitch_range(value, data_min_max)`\n",
    "\n",
    "- `map_value_to_pitch_range_exp_scale(value, data_min_max)`\n",
    "\n",
    "- `map_value_to_scale(value, data_min_max, tonic=\"c\", scale_name=\"MajorScale\",\n",
    "  octave_range=[3, 5])`\n",
    "\n",
    "For example, to create a sonification of the modern CO2 data you could do the following:\n",
    "\n",
    "1. Map the data to a log scale pitch range to return a series of music21 Note objects\n",
    "      ```python\n",
    "      pitch_range = co2_modern['season_adj'].apply(\n",
    "            map_value_to_pitch_range_exp_scale, data_min_max=co2_range.values\n",
    "      )\n",
    "      \n",
    "      ```\n",
    "\n",
    "1. Create an audio file from the data at 100 bpm using a Guitar sound and play it back\n",
    "      ```python\n",
    "      data_to_audio(pitch_range, 'demo_notes.flac', 100, 'Guitar')\n",
    "      Audio('demo_notes.flac')\n",
    "      \n",
    "      ```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create you own sonifications here\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "7K6a5vD5VxeK"
   },
   "source": [
    "## Other resources\n",
    "\n",
    "### Unfilled version of this notebook\n",
    "\n",
    "[Data Sonification with Python starter notebook](https://colab.research.google.com/github/ncsu-libraries-data-vis/data-sonification-with-python/blob/main/data-sonification-with-python.ipynb) - a version of this notebook without any exercise code filled in. Use this version of the notebook to practice concepts and coding or to lead a workshop.\n",
    "\n",
    "### Sonification development tools and applications\n",
    "\n",
    "- [Sonic Pi](https://sonic-pi.net/) - a code-based music creation and performance tool based on the Ruby programming language\n",
    "\n",
    "- [p5.js](https://p5js.org/) - a JavaScript library for creative coding that includes a library for interfacing with web-based audio.\n",
    "\n",
    "- [Web audio API](https://developer.mozilla.org/en-US/docs/Web/API/Web_Audio_API) - base API for controling audio on the web\n",
    "\n",
    "- [SAS Graphics Accelerator](https://support.sas.com/software/products/graphics-accelerator/samples/index.html) - a free Chrome extension that allows you to sonify data captured from web tables and data from web-based SAS visualization\n",
    "\n",
    "- [TwoTone](https://app.twotone.io/) - An interactive web application for easily creating sonifications with uploaded data\n",
    "\n",
    "### Sonification theory and practice\n",
    "\n",
    "- [The Sonification Handbook](https://sonification.de/handbook/) - an open access textbook published in 2011 providing an introduction to sonification and sonification research and techniques\n",
    "\n",
    "- [An Overview of Auditory Displays and Sonification](https://sonification.de/) - the website of Thomas Hermann, one of the editors of The Sonification Handbook, providing an overview of sonification\n",
    "\n",
    "- [DataSonificationBlog](https://www.saralenzi.com/datasonificationblog) - blog of Sara Lenzi, sonification researcher\n",
    "\n",
    "- [Intentionality and design in the data sonification of social issues](https://www.researchgate.net/publication/343692618_Intentionality_and_design_in_the_data_sonification_of_social_issues) - a journal article analyzing the role of intentionality when designing sonifications that communicate social issues to public audiences\n",
    "\n",
    "- [Thirteen Years of Reflection on Auditory Graphing: Promises, Pitfalls, and Potential New Directions](https://digitalcommons.unl.edu/cgi/viewcontent.cgi?referer=http://playitbyr.org/&httpsredir=1&article=1429&context=psychfacpub) - a 2005 journal article covering sonification methods and discussing the success of these methods\n",
    "\n",
    "- [Loud Numbers](https://www.loudnumbers.net/) - a data sonification podcast and mailing list\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "i6Sj-7OYR5JL"
   },
   "source": [
    "## Credits\n",
    "\n",
    "This workshop was created by Walt Gurley at the NC State University Libraries."
   ]
  }
 ],
 "metadata": {
  "colab": {
   "collapsed_sections": [],
   "name": "Data_Sonification_with_Python_filled.ipynb",
   "provenance": [],
   "toc_visible": true
  },
  "interpreter": {
   "hash": "fac5a5762ec420451b2489013a3546c3275ac1b170e5d848d93f56e02eb86e31"
  },
  "kernelspec": {
   "display_name": "Python 3.9.7 64-bit ('sonification': conda)",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}