Data_Manipulation_with_Python.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "uN0rPiA3E-7y"
      },
      "source": [
        "# **Objectives**\n",
        "\n",
        "\n",
        "### **Data Manipulation with Python**\n",
        "* **[Data Frames, Data Series and Pandas Library](#scrollTo=jBIdS1e4E-74)**\n",
        "* **Hands-on working on a data file with Pandas**\n",
        "  * [Creating a Dataframe Object and Saving/Exporting DataFrames](#scrollTo=CxQlMkrDE-76)\n",
        "  * [Exploring a Dataframe](#scrollTo=iVh0ikhOE-7-) (Columns, Datatypes, Shape etc)\n",
        "  * [Dataframe and Series Slicing and Indexing](#scrollTo=didszzDEE-8A)\n",
        "  * [Querying Dataframes](#scrollTo=mt9ieDjbE-8I)\n",
        "  * [Running Statistics on Dataframes](#scrollTo=u-K-K3ocE-8K)\n",
        "  * [Grouping and Aggregating Dataframes](#scrollTo=yh3Vg2UlE-8L)\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "\n",
        "## **Using LLM AI as a coding tutor when learning this workshop on your own.**\n",
        "There are several ways that LLM can help improve your self-learning process. LLM AI can be used as a coding tutor to help:\n",
        "* Explain new concepts in other words. Ask it to give basic, intermediate, or advanced explanations of coding concepts depending on your familiarity and comfort level.\n",
        "* Guide you through the exercises and explain the logic and code. If you get stuck with the exercises, you can ask a LLM to give you a step-by-step guide to the exercise so you better understand the learning objectives. For best results, be very specific with the LLM AI so it has the necessary context to best help you.\n",
        "\n",
        "* Give you more exercises. If you aren't sure you understand a concept and want to test your knowledge, ask a LLM AI to give additional exercises to help test your abilities and understanding. They can be coding problems, multiple choice, fill in the blank, or whatever will work best for your learning style. Be creative and get the support you want.  \n",
        "\n",
        "The AI has infinite patience, so ask as many questions as you want!\n",
        "\n",
        "### **The Animal Emojis**\n",
        "\n",
        "As you go through this workshop, you will notice **Animal Emojis** embedded throughout. They accompany bolded terms that are the fundamental concepts that python programming is built upon. The key thing here is **if you see an emoji, consider asking a LLM AI about the bolded concept**.\n",
        "  - To collect all the animals, you should follow the workshop section by section and add your found animal (new coding concept) to the end of the following prompt and take it to LLM AI:\n",
        "\n",
        " `You are going to take on the role of an excellent python coding tutor. I'm a beginner learning to program in python. In simple terms, please give me an overview and explanation of the following concept:  `\n",
        "\n",
        "  - You can add more surrounding components to the prompts to make it more clear. For example, maybe you know another language and want LLM AI to compare how the concept works in R or C++ to help you better understand.\n",
        "\n",
        "### **Disclaimer**\n",
        "- Outputs may be inaccurate, untruthful, and otherwise misleading.\n",
        "- It may produce biased content.\n",
        "- It will not cite sources.\n",
        "  \n",
        "Always use caution when interacting with LLM AI systems. Despite the large bounds in progress and capability, they are still in early versions and teams of people are working to fix short comings and mistakes each and every day. Despite how useful they can be as coding tutors, it is important to remember that their outputs should be met with a healthy amount of skepticism."
      ],
      "metadata": {
        "id": "XQWpAZo14Mcq"
      }
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "tt6PCd-NE-73"
      },
      "source": [
        "## **Data**\n",
        "We will be using files from the [Portal Project Teaching Database](https://figshare.com/articles/Portal_Project_Teaching_Database/1314459). This section will use the **'surveys.csv'** file, stored as a **'.csv'** and can be downloaded here: https://ndownloader.figshare.com/files/2292172<br>\n",
        "We are studying the species and weight of animals caught in sites in our study area.<br>\n",
        "Each row holds information for a single animal, and the columns represent:\n",
        "\n",
        "| **Column**      | **Description**               |\n",
        "|:----------------|:------------------------------|\n",
        "| record_id       | Unique id for the observation |\n",
        "| month           | month of observation          |\n",
        "| day             | day of observation            |\n",
        "| year            | year of observation           |\n",
        "| plot_id         | ID of a particular site       |\n",
        "| species_id      | 2-letter code for a species   |\n",
        "| sex             | sex of animal (“M”, “F”)      |\n",
        "| hindfoot_length | length of the hindfoot in mm  |\n",
        "| weight          | weight of the animal in grams |"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "jBIdS1e4E-74"
      },
      "source": [
        "## **🐺Python Dataframe**\n",
        "* 2-dimensional data structure that can store data of different types (including characters, integers, floating point values, factors and more) in a tabular form. Each row holds information for each column.\n",
        "* Similar to a spreadsheet or an SQL table or the **'data.frame'** in R.\n",
        "* Dataframes have indexes starting from 0.\n",
        "* An index refers to the position of an element in the data structure.\n",
        "\n",
        "## **🦁Python Series**\n",
        "* A series is a one-dimensional structure like an associative array.\n",
        "* Series have indexes starting from 0.\n",
        "* An index refers to the position of a value in the data structure.\n",
        "\n",
        "|index | value     |\n",
        "|------|-----------|\n",
        "| 0    | -84.5099  |\n",
        "| 1    | -84.48937 |\n",
        "| 2    | -84.4432  |\n",
        "\n",
        "\n",
        "## **🐯Pandas (Python Data Analysis Library)**\n",
        "* **Python Data Analysis Library** shortened as **pandas** is a third-party package for data manipulation\n",
        "* The **pandas** package is one of the best options for working with tabular data in Python\n",
        "* **pandas** ingest data as Dataframe objects and works with both Dataframes and Series\n",
        "* **pandas** uses sequential indexes by default\n",
        "* **pandas** provides data structures, produces high quality plots with **matplotlib** (a Python package graphing) and integrates nicely with other libraries that use **NumPy** (a Python package for arrays processing).\n",
        "* Operations on **numpy** arrays also work on **pandas** Series\n",
        "\n",
        "\n",
        "---\n",
        "Did you notice the first animal emoji - a wolf? Here is an example of how to use the sample prompt given above:\n",
        "\n",
        "`You are going to take on the role of an excellent python coding tutor. I'm a beginner learning to program in python. In simple terms, please give me an overview and explanation of the following concept: Python dataFrame`\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Importing Pandas**"
      ],
      "metadata": {
        "id": "n2Is9HaYFhpI"
      }
    },
    {
      "cell_type": "code",
      "execution_count": 8,
      "metadata": {
        "id": "Ed-DzV3OE-75"
      },
      "outputs": [],
      "source": [
        "import pandas as pd"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "## Run this cell if you're on Colab to download the 'data' file\n",
        "!wget https://raw.githubusercontent.com/The-CEAS-Library/Data_Manipulation_with_Python/master/data/surveys.csv"
      ],
      "metadata": {
        "id": "Sm-XQ6csQ-tS",
        "outputId": "cd8324f8-4267-4bef-90f3-141d4b3b7eeb",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "execution_count": 9,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "--2024-11-07 19:23:59--  https://raw.githubusercontent.com/The-CEAS-Library/Data_Manipulation_with_Python/master/data/surveys.csv\n",
            "Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.108.133, 185.199.109.133, 185.199.110.133, ...\n",
            "Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.108.133|:443... connected.\n",
            "HTTP request sent, awaiting response... 200 OK\n",
            "Length: 1021662 (998K) [text/plain]\n",
            "Saving to: ‘surveys.csv’\n",
            "\n",
            "surveys.csv         100%[===================>] 997.72K  --.-KB/s    in 0.03s   \n",
            "\n",
            "2024-11-07 19:24:00 (28.5 MB/s) - ‘surveys.csv’ saved [1021662/1021662]\n",
            "\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "CxQlMkrDE-76"
      },
      "source": [
        "### **🦒Reading CSV Data** - ***pd.read_csv( )***\n",
        "The pandas **'read_csv'** function pulls import from **'.csv'** files into a Dataframe objects\n",
        "\n",
        "*Syntax*:<br>\n",
        "**dataframe_object = pd.read_csv( file_path_name )**\n",
        "\n",
        "*Usage*:"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "surveys_df = pd.read_csv(\"surveys.csv\")\n",
        "surveys_df"
      ],
      "metadata": {
        "id": "suDcRSqJRCQJ",
        "outputId": "0370f2c7-86a1-49b4-db50-d7827cf6deb0",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 444
        }
      },
      "execution_count": 10,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "       record_id  month  day  year  plot_id species_id  sex  hindfoot_length  \\\n",
              "0              1      7   16  1977        2         NL    M             32.0   \n",
              "1              2      7   16  1977        3         NL    M             33.0   \n",
              "2              3      7   16  1977        2         DM    F             37.0   \n",
              "3              4      7   16  1977        7         DM    M             36.0   \n",
              "4              5      7   16  1977        3         DM    M             35.0   \n",
              "...          ...    ...  ...   ...      ...        ...  ...              ...   \n",
              "35544      35545     12   31  2002       15         AH  NaN              NaN   \n",
              "35545      35546     12   31  2002       15         AH  NaN              NaN   \n",
              "35546      35547     12   31  2002       10         RM    F             15.0   \n",
              "35547      35548     12   31  2002        7         DO    M             36.0   \n",
              "35548      35549     12   31  2002        5        NaN  NaN              NaN   \n",
              "\n",
              "       weight  \n",
              "0         NaN  \n",
              "1         NaN  \n",
              "2         NaN  \n",
              "3         NaN  \n",
              "4         NaN  \n",
              "...       ...  \n",
              "35544     NaN  \n",
              "35545     NaN  \n",
              "35546    14.0  \n",
              "35547    51.0  \n",
              "35548     NaN  \n",
              "\n",
              "[35549 rows x 9 columns]"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-24758217-73b0-415d-9d9c-f961aab40c4a\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>record_id</th>\n",
              "      <th>month</th>\n",
              "      <th>day</th>\n",
              "      <th>year</th>\n",
              "      <th>plot_id</th>\n",
              "      <th>species_id</th>\n",
              "      <th>sex</th>\n",
              "      <th>hindfoot_length</th>\n",
              "      <th>weight</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>1</td>\n",
              "      <td>7</td>\n",
              "      <td>16</td>\n",
              "      <td>1977</td>\n",
              "      <td>2</td>\n",
              "      <td>NL</td>\n",
              "      <td>M</td>\n",
              "      <td>32.0</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>2</td>\n",
              "      <td>7</td>\n",
              "      <td>16</td>\n",
              "      <td>1977</td>\n",
              "      <td>3</td>\n",
              "      <td>NL</td>\n",
              "      <td>M</td>\n",
              "      <td>33.0</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>3</td>\n",
              "      <td>7</td>\n",
              "      <td>16</td>\n",
              "      <td>1977</td>\n",
              "      <td>2</td>\n",
              "      <td>DM</td>\n",
              "      <td>F</td>\n",
              "      <td>37.0</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>4</td>\n",
              "      <td>7</td>\n",
              "      <td>16</td>\n",
              "      <td>1977</td>\n",
              "      <td>7</td>\n",
              "      <td>DM</td>\n",
              "      <td>M</td>\n",
              "      <td>36.0</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>5</td>\n",
              "      <td>7</td>\n",
              "      <td>16</td>\n",
              "      <td>1977</td>\n",
              "      <td>3</td>\n",
              "      <td>DM</td>\n",
              "      <td>M</td>\n",
              "      <td>35.0</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>...</th>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>35544</th>\n",
              "      <td>35545</td>\n",
              "      <td>12</td>\n",
              "      <td>31</td>\n",
              "      <td>2002</td>\n",
              "      <td>15</td>\n",
              "      <td>AH</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>35545</th>\n",
              "      <td>35546</td>\n",
              "      <td>12</td>\n",
              "      <td>31</td>\n",
              "      <td>2002</td>\n",
              "      <td>15</td>\n",
              "      <td>AH</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>35546</th>\n",
              "      <td>35547</td>\n",
              "      <td>12</td>\n",
              "      <td>31</td>\n",
              "      <td>2002</td>\n",
              "      <td>10</td>\n",
              "      <td>RM</td>\n",
              "      <td>F</td>\n",
              "      <td>15.0</td>\n",
              "      <td>14.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>35547</th>\n",
              "      <td>35548</td>\n",
              "      <td>12</td>\n",
              "      <td>31</td>\n",
              "      <td>2002</td>\n",
              "      <td>7</td>\n",
              "      <td>DO</td>\n",
              "      <td>M</td>\n",
              "      <td>36.0</td>\n",
              "      <td>51.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>35548</th>\n",
              "      <td>35549</td>\n",
              "      <td>12</td>\n",
              "      <td>31</td>\n",
              "      <td>2002</td>\n",
              "      <td>5</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>35549 rows × 9 columns</p>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-24758217-73b0-415d-9d9c-f961aab40c4a')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-24758217-73b0-415d-9d9c-f961aab40c4a button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-24758217-73b0-415d-9d9c-f961aab40c4a');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-98b8491f-49fe-4c79-a6da-0f2ec7628185\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-98b8491f-49fe-4c79-a6da-0f2ec7628185')\"\n",
              "            title=\"Suggest charts\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-98b8491f-49fe-4c79-a6da-0f2ec7628185 button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "\n",
              "  <div id=\"id_ec814745-ef1a-4fe4-bbcf-94561753c8ce\">\n",
              "    <style>\n",
              "      .colab-df-generate {\n",
              "        background-color: #E8F0FE;\n",
              "        border: none;\n",
              "        border-radius: 50%;\n",
              "        cursor: pointer;\n",
              "        display: none;\n",
              "        fill: #1967D2;\n",
              "        height: 32px;\n",
              "        padding: 0 0 0 0;\n",
              "        width: 32px;\n",
              "      }\n",
              "\n",
              "      .colab-df-generate:hover {\n",
              "        background-color: #E2EBFA;\n",
              "        box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "        fill: #174EA6;\n",
              "      }\n",
              "\n",
              "      [theme=dark] .colab-df-generate {\n",
              "        background-color: #3B4455;\n",
              "        fill: #D2E3FC;\n",
              "      }\n",
              "\n",
              "      [theme=dark] .colab-df-generate:hover {\n",
              "        background-color: #434B5C;\n",
              "        box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "        filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "        fill: #FFFFFF;\n",
              "      }\n",
              "    </style>\n",
              "    <button class=\"colab-df-generate\" onclick=\"generateWithVariable('surveys_df')\"\n",
              "            title=\"Generate code using this dataframe.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M7,19H8.4L18.45,9,17,7.55,7,17.6ZM5,21V16.75L18.45,3.32a2,2,0,0,1,2.83,0l1.4,1.43a1.91,1.91,0,0,1,.58,1.4,1.91,1.91,0,0,1-.58,1.4L9.25,21ZM18.45,9,17,7.55Zm-12,3A5.31,5.31,0,0,0,4.9,8.1,5.31,5.31,0,0,0,1,6.5,5.31,5.31,0,0,0,4.9,4.9,5.31,5.31,0,0,0,6.5,1,5.31,5.31,0,0,0,8.1,4.9,5.31,5.31,0,0,0,12,6.5,5.46,5.46,0,0,0,6.5,12Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "    <script>\n",
              "      (() => {\n",
              "      const buttonEl =\n",
              "        document.querySelector('#id_ec814745-ef1a-4fe4-bbcf-94561753c8ce button.colab-df-generate');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      buttonEl.onclick = () => {\n",
              "        google.colab.notebook.generateWithVariable('surveys_df');\n",
              "      }\n",
              "      })();\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "    </div>\n",
              "  </div>\n"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "dataframe",
              "variable_name": "surveys_df",
              "summary": "{\n  \"name\": \"surveys_df\",\n  \"rows\": 35549,\n  \"fields\": [\n    {\n      \"column\": \"record_id\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 10262,\n        \"min\": 1,\n        \"max\": 35549,\n        \"num_unique_values\": 35549,\n        \"samples\": [\n          19536,\n          24087,\n          30804\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"month\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 3,\n        \"min\": 1,\n        \"max\": 12,\n        \"num_unique_values\": 12,\n        \"samples\": [\n          5,\n          4,\n          7\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"day\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 8,\n        \"min\": 1,\n        \"max\": 31,\n        \"num_unique_values\": 31,\n        \"samples\": [\n          23,\n          4,\n          31\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"year\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 7,\n        \"min\": 1977,\n        \"max\": 2002,\n        \"num_unique_values\": 26,\n        \"samples\": [\n          1985,\n          1993,\n          1977\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"plot_id\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 6,\n        \"min\": 1,\n        \"max\": 24,\n        \"num_unique_values\": 24,\n        \"samples\": [\n          11,\n          20,\n          2\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"species_id\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 48,\n        \"samples\": [\n          \"UR\",\n          \"CU\",\n          \"CV\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"sex\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 2,\n        \"samples\": [\n          \"F\",\n          \"M\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"hindfoot_length\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 9.564758680929966,\n        \"min\": 2.0,\n        \"max\": 70.0,\n        \"num_unique_values\": 56,\n        \"samples\": [\n          32.0,\n          14.0\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"weight\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 36.63125947458399,\n        \"min\": 4.0,\n        \"max\": 280.0,\n        \"num_unique_values\": 255,\n        \"samples\": [\n          243.0,\n          8.0\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}"
            }
          },
          "metadata": {},
          "execution_count": 10
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "gK7BFlHYE-77"
      },
      "source": [
        "The first column is the **🦊'0-based' index** of the DataFrame. It is not an actual column of the DataFrame but used to identify the position of the data.\n",
        "\n",
        "### **Dataframe *head( )* and *tail( )* methods**\n",
        "The **'head( )'** method displays the first several lines of a dataframe object.\n",
        "The **'tail( )'** method displays the last several lines of a dataframe object.\n",
        "\n",
        "*Syntax*:<br>\n",
        "**data_frame_object_name.head ( [ row_count_limit ] )**<br>\n",
        "**data_frame_object_name.tail ( [ row_count_limit ] )**\n",
        "\n",
        "**# row_count_limit is optional. Default is 5**\n",
        "\n",
        "\n",
        "*Usage*:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 11,
      "metadata": {
        "id": "Zunp4ob-E-78",
        "outputId": "04d750b9-f835-483a-9251-c6c6b8cd927c",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 226
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "   record_id  month  day  year  plot_id species_id sex  hindfoot_length  \\\n",
              "0          1      7   16  1977        2         NL   M             32.0   \n",
              "1          2      7   16  1977        3         NL   M             33.0   \n",
              "2          3      7   16  1977        2         DM   F             37.0   \n",
              "3          4      7   16  1977        7         DM   M             36.0   \n",
              "4          5      7   16  1977        3         DM   M             35.0   \n",
              "\n",
              "   weight  \n",
              "0     NaN  \n",
              "1     NaN  \n",
              "2     NaN  \n",
              "3     NaN  \n",
              "4     NaN  "
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-966d2397-1dc5-4470-8f14-23d7a6e3a4a0\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>record_id</th>\n",
              "      <th>month</th>\n",
              "      <th>day</th>\n",
              "      <th>year</th>\n",
              "      <th>plot_id</th>\n",
              "      <th>species_id</th>\n",
              "      <th>sex</th>\n",
              "      <th>hindfoot_length</th>\n",
              "      <th>weight</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>1</td>\n",
              "      <td>7</td>\n",
              "      <td>16</td>\n",
              "      <td>1977</td>\n",
              "      <td>2</td>\n",
              "      <td>NL</td>\n",
              "      <td>M</td>\n",
              "      <td>32.0</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>2</td>\n",
              "      <td>7</td>\n",
              "      <td>16</td>\n",
              "      <td>1977</td>\n",
              "      <td>3</td>\n",
              "      <td>NL</td>\n",
              "      <td>M</td>\n",
              "      <td>33.0</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>3</td>\n",
              "      <td>7</td>\n",
              "      <td>16</td>\n",
              "      <td>1977</td>\n",
              "      <td>2</td>\n",
              "      <td>DM</td>\n",
              "      <td>F</td>\n",
              "      <td>37.0</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>4</td>\n",
              "      <td>7</td>\n",
              "      <td>16</td>\n",
              "      <td>1977</td>\n",
              "      <td>7</td>\n",
              "      <td>DM</td>\n",
              "      <td>M</td>\n",
              "      <td>36.0</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>5</td>\n",
              "      <td>7</td>\n",
              "      <td>16</td>\n",
              "      <td>1977</td>\n",
              "      <td>3</td>\n",
              "      <td>DM</td>\n",
              "      <td>M</td>\n",
              "      <td>35.0</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-966d2397-1dc5-4470-8f14-23d7a6e3a4a0')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-966d2397-1dc5-4470-8f14-23d7a6e3a4a0 button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-966d2397-1dc5-4470-8f14-23d7a6e3a4a0');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-f7c91339-b52e-470a-8330-819a65a0ae90\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-f7c91339-b52e-470a-8330-819a65a0ae90')\"\n",
              "            title=\"Suggest charts\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-f7c91339-b52e-470a-8330-819a65a0ae90 button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "\n",
              "    </div>\n",
              "  </div>\n"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "dataframe",
              "variable_name": "surveys_df",
              "summary": "{\n  \"name\": \"surveys_df\",\n  \"rows\": 35549,\n  \"fields\": [\n    {\n      \"column\": \"record_id\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 10262,\n        \"min\": 1,\n        \"max\": 35549,\n        \"num_unique_values\": 35549,\n        \"samples\": [\n          19536,\n          24087,\n          30804\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"month\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 3,\n        \"min\": 1,\n        \"max\": 12,\n        \"num_unique_values\": 12,\n        \"samples\": [\n          5,\n          4,\n          7\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"day\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 8,\n        \"min\": 1,\n        \"max\": 31,\n        \"num_unique_values\": 31,\n        \"samples\": [\n          23,\n          4,\n          31\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"year\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 7,\n        \"min\": 1977,\n        \"max\": 2002,\n        \"num_unique_values\": 26,\n        \"samples\": [\n          1985,\n          1993,\n          1977\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"plot_id\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 6,\n        \"min\": 1,\n        \"max\": 24,\n        \"num_unique_values\": 24,\n        \"samples\": [\n          11,\n          20,\n          2\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"species_id\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 48,\n        \"samples\": [\n          \"UR\",\n          \"CU\",\n          \"CV\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"sex\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 2,\n        \"samples\": [\n          \"F\",\n          \"M\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"hindfoot_length\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 9.564758680929966,\n        \"min\": 2.0,\n        \"max\": 70.0,\n        \"num_unique_values\": 56,\n        \"samples\": [\n          32.0,\n          14.0\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"weight\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 36.63125947458399,\n        \"min\": 4.0,\n        \"max\": 280.0,\n        \"num_unique_values\": 255,\n        \"samples\": [\n          243.0,\n          8.0\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}"
            }
          },
          "metadata": {},
          "execution_count": 11
        }
      ],
      "source": [
        "surveys_df.head()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 12,
      "metadata": {
        "id": "jUCMZSVtE-78",
        "outputId": "3210fa7f-3878-4267-f294-5efad46cb584",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 383
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "       record_id  month  day  year  plot_id species_id  sex  hindfoot_length  \\\n",
              "35539      35540     12   31  2002       15         PB    F             26.0   \n",
              "35540      35541     12   31  2002       15         PB    F             24.0   \n",
              "35541      35542     12   31  2002       15         PB    F             26.0   \n",
              "35542      35543     12   31  2002       15         PB    F             27.0   \n",
              "35543      35544     12   31  2002       15         US  NaN              NaN   \n",
              "35544      35545     12   31  2002       15         AH  NaN              NaN   \n",
              "35545      35546     12   31  2002       15         AH  NaN              NaN   \n",
              "35546      35547     12   31  2002       10         RM    F             15.0   \n",
              "35547      35548     12   31  2002        7         DO    M             36.0   \n",
              "35548      35549     12   31  2002        5        NaN  NaN              NaN   \n",
              "\n",
              "       weight  \n",
              "35539    23.0  \n",
              "35540    31.0  \n",
              "35541    29.0  \n",
              "35542    34.0  \n",
              "35543     NaN  \n",
              "35544     NaN  \n",
              "35545     NaN  \n",
              "35546    14.0  \n",
              "35547    51.0  \n",
              "35548     NaN  "
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-8b8913f1-d779-432e-981d-b1636c5c0269\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>record_id</th>\n",
              "      <th>month</th>\n",
              "      <th>day</th>\n",
              "      <th>year</th>\n",
              "      <th>plot_id</th>\n",
              "      <th>species_id</th>\n",
              "      <th>sex</th>\n",
              "      <th>hindfoot_length</th>\n",
              "      <th>weight</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>35539</th>\n",
              "      <td>35540</td>\n",
              "      <td>12</td>\n",
              "      <td>31</td>\n",
              "      <td>2002</td>\n",
              "      <td>15</td>\n",
              "      <td>PB</td>\n",
              "      <td>F</td>\n",
              "      <td>26.0</td>\n",
              "      <td>23.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>35540</th>\n",
              "      <td>35541</td>\n",
              "      <td>12</td>\n",
              "      <td>31</td>\n",
              "      <td>2002</td>\n",
              "      <td>15</td>\n",
              "      <td>PB</td>\n",
              "      <td>F</td>\n",
              "      <td>24.0</td>\n",
              "      <td>31.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>35541</th>\n",
              "      <td>35542</td>\n",
              "      <td>12</td>\n",
              "      <td>31</td>\n",
              "      <td>2002</td>\n",
              "      <td>15</td>\n",
              "      <td>PB</td>\n",
              "      <td>F</td>\n",
              "      <td>26.0</td>\n",
              "      <td>29.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>35542</th>\n",
              "      <td>35543</td>\n",
              "      <td>12</td>\n",
              "      <td>31</td>\n",
              "      <td>2002</td>\n",
              "      <td>15</td>\n",
              "      <td>PB</td>\n",
              "      <td>F</td>\n",
              "      <td>27.0</td>\n",
              "      <td>34.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>35543</th>\n",
              "      <td>35544</td>\n",
              "      <td>12</td>\n",
              "      <td>31</td>\n",
              "      <td>2002</td>\n",
              "      <td>15</td>\n",
              "      <td>US</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>35544</th>\n",
              "      <td>35545</td>\n",
              "      <td>12</td>\n",
              "      <td>31</td>\n",
              "      <td>2002</td>\n",
              "      <td>15</td>\n",
              "      <td>AH</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>35545</th>\n",
              "      <td>35546</td>\n",
              "      <td>12</td>\n",
              "      <td>31</td>\n",
              "      <td>2002</td>\n",
              "      <td>15</td>\n",
              "      <td>AH</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>35546</th>\n",
              "      <td>35547</td>\n",
              "      <td>12</td>\n",
              "      <td>31</td>\n",
              "      <td>2002</td>\n",
              "      <td>10</td>\n",
              "      <td>RM</td>\n",
              "      <td>F</td>\n",
              "      <td>15.0</td>\n",
              "      <td>14.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>35547</th>\n",
              "      <td>35548</td>\n",
              "      <td>12</td>\n",
              "      <td>31</td>\n",
              "      <td>2002</td>\n",
              "      <td>7</td>\n",
              "      <td>DO</td>\n",
              "      <td>M</td>\n",
              "      <td>36.0</td>\n",
              "      <td>51.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>35548</th>\n",
              "      <td>35549</td>\n",
              "      <td>12</td>\n",
              "      <td>31</td>\n",
              "      <td>2002</td>\n",
              "      <td>5</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-8b8913f1-d779-432e-981d-b1636c5c0269')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-8b8913f1-d779-432e-981d-b1636c5c0269 button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-8b8913f1-d779-432e-981d-b1636c5c0269');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-1267d061-2877-48c4-95bd-f1a3a35e2145\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-1267d061-2877-48c4-95bd-f1a3a35e2145')\"\n",
              "            title=\"Suggest charts\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-1267d061-2877-48c4-95bd-f1a3a35e2145 button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "\n",
              "    </div>\n",
              "  </div>\n"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "dataframe",
              "summary": "{\n  \"name\": \"surveys_df\",\n  \"rows\": 10,\n  \"fields\": [\n    {\n      \"column\": \"record_id\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 3,\n        \"min\": 35540,\n        \"max\": 35549,\n        \"num_unique_values\": 10,\n        \"samples\": [\n          35548,\n          35541,\n          35545\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"month\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0,\n        \"min\": 12,\n        \"max\": 12,\n        \"num_unique_values\": 1,\n        \"samples\": [\n          12\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"day\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0,\n        \"min\": 31,\n        \"max\": 31,\n        \"num_unique_values\": 1,\n        \"samples\": [\n          31\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"year\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0,\n        \"min\": 2002,\n        \"max\": 2002,\n        \"num_unique_values\": 1,\n        \"samples\": [\n          2002\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"plot_id\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 3,\n        \"min\": 5,\n        \"max\": 15,\n        \"num_unique_values\": 4,\n        \"samples\": [\n          10\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"species_id\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 5,\n        \"samples\": [\n          \"US\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"sex\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 2,\n        \"samples\": [\n          \"M\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"hindfoot_length\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 6.713171133426189,\n        \"min\": 15.0,\n        \"max\": 36.0,\n        \"num_unique_values\": 5,\n        \"samples\": [\n          24.0\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"weight\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 12.355835328567093,\n        \"min\": 14.0,\n        \"max\": 51.0,\n        \"num_unique_values\": 6,\n        \"samples\": [\n          23.0\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}"
            }
          },
          "metadata": {},
          "execution_count": 12
        }
      ],
      "source": [
        "surveys_df.tail(10)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "i6Wt_GaJE-79"
      },
      "source": [
        "### **🦝Writing out Data from Dataframe to CSV using *to_csv( )* method**\n",
        "The **'to_csv'** pandas DataFrame method is used to export a DataFrame in CSV format\n",
        "\n",
        "*Syntax*:<br>\n",
        "**data_frame_object_name.to_csv ( csv_file_path_name , [ index ] , [ sep ] , [ na_rep ] , [ mode ] )**\n",
        "\n",
        "*Note*:\n",
        "* **csv_file_path_name** is the file path. If **None** is provided the result is returned as a string. Default is **None**\n",
        "* **index** is boolean value for Write row index/names. Default is **True**. Set this to **False** so that pandas doesn’t include the index number for each line.\n",
        "* **sep** is a string of length 1 for field delimiter for the output file. Default is **' , '**\n",
        "* **na_rep** is a string for missing data representation. Default is **' '**\n",
        "* **mode** is a string for the Python write mode. Default is **'w'**\n",
        "\n",
        "*Usage*:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 13,
      "metadata": {
        "id": "0clm84efE-79"
      },
      "outputs": [],
      "source": [
        "surveys_df_head = pd.read_csv(\"surveys.csv\").head(30)\n",
        "surveys_df_head.to_csv('surveys_head.csv', index=False)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ZhcARp0_E-7-"
      },
      "source": [
        "**<h5><center>Note</center></h5>**\n",
        "\n",
        "**' to_excel '** pandas DataFrame method is used to export a DataFrame to an **Excel .xlsx** file"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## **Accessing Data Attributes of a DataFrame Object**"
      ],
      "metadata": {
        "id": "08yBvCg529Dd"
      }
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "iVh0ikhOE-7-"
      },
      "source": [
        "### **🐻Dataframe '*dtypes*' data attribute**\n",
        "Shows the the data types of the values in the columns.\n",
        "\n",
        "*Syntax*:<br>\n",
        "**data_frame_object_name.dtypes**\n",
        "\n",
        "*Usage*:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 14,
      "metadata": {
        "id": "AsdaaRVzE-7-",
        "outputId": "745de3b3-f3cc-4821-d134-4a3c8172c871",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 366
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "record_id            int64\n",
              "month                int64\n",
              "day                  int64\n",
              "year                 int64\n",
              "plot_id              int64\n",
              "species_id          object\n",
              "sex                 object\n",
              "hindfoot_length    float64\n",
              "weight             float64\n",
              "dtype: object"
            ],
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>0</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>record_id</th>\n",
              "      <td>int64</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>month</th>\n",
              "      <td>int64</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>day</th>\n",
              "      <td>int64</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>year</th>\n",
              "      <td>int64</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>plot_id</th>\n",
              "      <td>int64</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>species_id</th>\n",
              "      <td>object</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>sex</th>\n",
              "      <td>object</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>hindfoot_length</th>\n",
              "      <td>float64</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>weight</th>\n",
              "      <td>float64</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div><br><label><b>dtype:</b> object</label>"
            ]
          },
          "metadata": {},
          "execution_count": 14
        }
      ],
      "source": [
        "surveys_df.dtypes"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "od9b9kUTE-7-"
      },
      "source": [
        "### **🐻‍❄️Dataframe '*columns*' data attribute**\n",
        "Shows an index list of all the column names in the DataFrame.\n",
        "\n",
        "*Syntax*:<br>\n",
        "**data_frame_object_name.columns**\n",
        "\n",
        "*Usage*:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 15,
      "metadata": {
        "id": "ggmTXP06E-7_",
        "outputId": "87ac93f2-d131-477f-a189-89d81a142e7f",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "Index(['record_id', 'month', 'day', 'year', 'plot_id', 'species_id', 'sex',\n",
              "       'hindfoot_length', 'weight'],\n",
              "      dtype='object')"
            ]
          },
          "metadata": {},
          "execution_count": 15
        }
      ],
      "source": [
        "surveys_df.columns"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "xLXeLN_BE-7_"
      },
      "outputs": [],
      "source": [
        "surveys_df.columns[0:4]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "x5Vq3FSBE-7_"
      },
      "source": [
        "### **🐼Dataframe '*shape*' data attribute**\n",
        "Outputs a tuple of the row by column count of the DataFrame.\n",
        "\n",
        "*Syntax*:<br>\n",
        "**data_frame_object_name.shape**\n",
        "\n",
        "*Usage*:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "alTkWtkcE-7_"
      },
      "outputs": [],
      "source": [
        "surveys_df.shape"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "collapsed": true,
        "id": "didszzDEE-8A"
      },
      "source": [
        "### **🦓Selecting data using Column Names**\n",
        "* DataFrame objects are treated like dictionaries using column heads as keys\n",
        "* We use square brackets **[ ]** to select column data and for indexing<br>\n",
        "\n",
        "*Syntax*:<br>\n",
        "**data_frame_object_name [ 'column_name' ]**\n",
        "\n",
        "*Usage*:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "YiTPionJE-8A"
      },
      "outputs": [],
      "source": [
        "surveys_df['record_id']"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "e6t4gmSJE-8A"
      },
      "source": [
        "* You may selct more than one column at a time by making a list of the column names<br>\n",
        "\n",
        "*Syntax*:<br>\n",
        "**data_frame_object_name [ [ 'column_name_1' , 'column_name_2' ] ]**\n",
        "\n",
        "*Usage*:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "ylcPj9hSE-8A"
      },
      "outputs": [],
      "source": [
        "surveys_df[['month', 'day']]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "yBs05aoaE-8B"
      },
      "source": [
        "* You may select a column data also by using the column name as an 'attribute'<br>\n",
        "\n",
        "*Syntax*:<br>\n",
        "**data_frame_object_name.column_name**\n",
        "\n",
        "*Usage*:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "OT8cx9baE-8B"
      },
      "outputs": [],
      "source": [
        "surveys_df.record_id"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Vw6kEOqHE-8B"
      },
      "source": [
        "### **🐎Selecting Unique values from a column**\n",
        "#### Using pandas *'unique'* function\n",
        "The **pandas.unique()** function tells us all of the unique values in a column\n",
        "\n",
        "*Syntax*:<br>\n",
        "**pd.unique ( data_frame_object_name [ 'column_name' ] )**<br>\n",
        "or<br>\n",
        "**pd.unique ( data_frame_object_name.column_name )**\n",
        "\n",
        "*Usage*:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "cKkyP5P4E-8B"
      },
      "outputs": [],
      "source": [
        "pd.unique(surveys_df['species_id'])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "y4uBA4uLE-8B",
        "scrolled": true
      },
      "outputs": [],
      "source": [
        "pd.unique(surveys_df.species_id)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "LLqaUuNvE-8C"
      },
      "source": [
        "\n",
        "\n",
        "*Syntax*:<br>\n",
        "**data_series_object_name.unique ( )**<br>\n",
        "or<br>\n",
        "**data_frame_object_name [ 'column_name' ].unique ( )**<br>"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "jf_IoLK6E-8D"
      },
      "outputs": [],
      "source": [
        "surveys_df['species_id'].unique()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6Z6u17u0E-8E"
      },
      "source": [
        "### **🦄Extracting data Subsets by Indexing and Slicing**\n",
        "#### **Slicing and subsetting rows**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "3qUJxkGRE-8E"
      },
      "outputs": [],
      "source": [
        "surveys_df[0:5]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "I7zttvkKE-8E"
      },
      "outputs": [],
      "source": [
        "surveys_df[3:7]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "DLYFzBLPE-8E"
      },
      "outputs": [],
      "source": [
        "surveys_df[5:]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "ZrYzJiKfE-8E"
      },
      "outputs": [],
      "source": [
        "surveys_df[:5]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "1BunOpZtE-8F"
      },
      "outputs": [],
      "source": [
        "surveys_df[-1:]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "SpN5TJ4nE-8F"
      },
      "outputs": [],
      "source": [
        "surveys_df[-3:]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "yzTi26aPE-8F"
      },
      "outputs": [],
      "source": [
        "surveys_df[-3:-1]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4b6R03NCE-8F"
      },
      "source": [
        "#### **Selecting and slicing data from columns**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "xOu_ifb4E-8F"
      },
      "outputs": [],
      "source": [
        "surveys_df['record_id']"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Ewi62EyAE-8F"
      },
      "outputs": [],
      "source": [
        "surveys_df['record_id'][0]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "5NDXMLm6E-8F"
      },
      "outputs": [],
      "source": [
        "surveys_df.record_id[0]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "O3OZzvcYE-8G"
      },
      "outputs": [],
      "source": [
        "surveys_df['record_id'][0:4]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "5ez_DIZ7E-8G"
      },
      "outputs": [],
      "source": [
        "surveys_df[['record_id','plot_id']][0:5]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "SbUO0_q7E-8G"
      },
      "source": [
        "#### **Slicing Subsets of Rows and Columns**\n",
        "We can select a specific row and a column value (a specific data element) or ranges of the data in both the row and column directions using either the **'iloc'** data attribute with integer-based indexing or the **'loc'** data attribute with integers for rows and the column labels.\n",
        "\n",
        "\n",
        "##### **🦬iloc**\n",
        "* Integer based indexing in the row and column directions\n",
        "* Indexing follows the same **0-based** indexing and slicing concepts used earlier\n",
        "\n",
        "*Syntax*:<br>\n",
        "**# for a specific data element**\n",
        "\n",
        "**data_frame_object_name.iloc [ row_index, column_index ]**\n",
        "\n",
        "**# Remember that Python indexing begins at 0. So, the index location [2, 6] selects the element that is 3 rows down and 7 columns of the DataFrame.**\n",
        "\n",
        "*Usage*:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "4AskwDryE-8G"
      },
      "outputs": [],
      "source": [
        "surveys_df.iloc[2, 6]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "ylC8O6BaE-8G"
      },
      "outputs": [],
      "source": [
        "surveys_df.head(5)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pz0RrctNE-8G"
      },
      "source": [
        "*Syntax*:<br>\n",
        "**# for a range of data element**\n",
        "\n",
        "**data_frame_object_name.iloc [ starting_row_index : ending_row_index + 1 , starting_column_index : ending_column_index + 1 ]**\n",
        "\n",
        "**# Remember that in Python slicing, ending indices are not included in selection. So, the index location [1:4, 0:5] selects the  element in row 2 to 4 (index 1 to 3) and in columns 1 to 5 (index 0 to 4) of the DataFrame.**\n",
        "\n",
        "*Usage*:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "SFil84NZE-8G"
      },
      "outputs": [],
      "source": [
        "surveys_df.iloc[1:4, 0:5]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Jfva3FV_E-8H"
      },
      "outputs": [],
      "source": [
        "surveys_df.iloc[3:5, 5:8]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "6Qsk1xIgE-8H"
      },
      "outputs": [],
      "source": [
        "surveys_df.iloc[3:5, :]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "zpbuFVEHE-8H"
      },
      "outputs": [],
      "source": [
        "surveys_df.iloc[:, 5:8].head()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ElvFmbHOE-8H"
      },
      "source": [
        "*Syntax*:<br>\n",
        "**# selection by specifying a list of row indexes and list of column indexes**\n",
        "\n",
        "**data_frame_object_name.iloc [ [ list of row indexes ] , [ list of column indexes ] ]**\n",
        "\n",
        "*Usage*:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Hz10TN8kE-8H"
      },
      "outputs": [],
      "source": [
        "surveys_df.iloc[[2,5,23,1], [8,5,6,7]]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Mwqh0ATNE-8H"
      },
      "outputs": [],
      "source": [
        "surveys_df.iloc[[2,5,23], :]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "BSsqzBtKE-8H"
      },
      "outputs": [],
      "source": [
        "surveys_df.iloc[[2,5,23], 5:8]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "nnK0DXWpE-8I"
      },
      "source": [
        "##### **🐂loc**\n",
        "* Label based indexing.\n",
        "* Rows use Integer labels. When integers are used, they are interpreted as a label and not indexes.\n",
        "* Columns use a list of strings of the column names.\n",
        "* Indexing follows the same **0-based** indexing and slicing concepts used earlier\n",
        "\n",
        "NB: The integer row labels in **'loc'** are different from the integer indexes in **'iloc'**. With loc, both the start bound and the stop bound are inclusive.\n",
        "\n",
        "*Syntax*:<br>\n",
        "**# selection by specifying a list of integer row labels and list of column names**\n",
        "\n",
        "**data_frame_object_name.loc [ [list of row indexes ] , [list of column names ] ]**\n",
        "\n",
        "**# selection by specifying starting and ending integer row label and list of column names**\n",
        "\n",
        "**data_frame_object_name.loc [ starting_integer_row label : ending_integer_row label , [list of column names ] ]**\n",
        "\n",
        "\n",
        "*Usage*:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 5,
      "metadata": {
        "id": "uHtXHXrKE-8I",
        "outputId": "2568ef9e-8f2c-4bcd-e30a-accaf2d8fb1b",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 147
        }
      },
      "outputs": [
        {
          "output_type": "error",
          "ename": "NameError",
          "evalue": "name 'surveys_df' is not defined",
          "traceback": [
            "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
            "\u001b[0;31mNameError\u001b[0m                                 Traceback (most recent call last)",
            "\u001b[0;32m<ipython-input-5-33c48b0c3af4>\u001b[0m in \u001b[0;36m<cell line: 1>\u001b[0;34m()\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0msurveys_df\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mloc\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m2\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;36m5\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m'species_id'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m'plot_id'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m'weight'\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
            "\u001b[0;31mNameError\u001b[0m: name 'surveys_df' is not defined"
          ]
        }
      ],
      "source": [
        "surveys_df.loc[2:5,['species_id', 'plot_id', 'weight']]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "EIs4k_BlE-8I"
      },
      "outputs": [],
      "source": [
        "surveys_df.loc[[2,5,5,7],['species_id', 'plot_id', 'weight']]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "aQmOZ0BeE-8I"
      },
      "outputs": [],
      "source": [
        "surveys_df.loc[2:5,:]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "SERIXCivE-8I"
      },
      "outputs": [],
      "source": [
        "surveys_df.loc[:,['species_id', 'plot_id', 'weight']].tail()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mt9ieDjbE-8I"
      },
      "source": [
        "## **🦘Creating Queries**\n",
        "Similar to SQL and R, we can select a subset of our data using criteria\n",
        "\n",
        "*Syntax*:<br>\n",
        "**data_frame_object_name [ < condition > ]**<br>\n",
        "\n",
        "**# Conditions must be based on columns names**\n",
        "\n",
        "*Usage*:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "ke67nfOCE-8I"
      },
      "outputs": [],
      "source": [
        "surveys_df[surveys_df['weight'] == 88]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "hgdO5elIE-8J"
      },
      "outputs": [],
      "source": [
        "surveys_df[surveys_df.weight == 88]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "wvc2ChiOE-8J"
      },
      "outputs": [],
      "source": [
        "surveys_df[surveys_df.month == surveys_df.plot_id].head()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "iPpQnshBE-8J"
      },
      "source": [
        "**<h5><center>Note</center></h5>**\n",
        "\n",
        "1. **Remember the python Comparison Operators discussed in the Introduction to Python Programming workshop**\n",
        "   * Equals: **==**\n",
        "   * Not equals: **!=**\n",
        "   * Greater than, less than: **>** or **<**\n",
        "   * Greater than or equal to, Less than or equal to: **>=** or **<=**<br><br>\n",
        "2. **Remember the python Logical Operators discussed in the Introduction to Python Programming workshop. This time we use the BITWISE versions**<br>\n",
        "   * Logical AND ( **&** )<br>\n",
        "     *Syntax*:<br>\n",
        "     *Note the parenthesis around each condition*<br>\n",
        "     **data_frame_object_name [ ( < condition 1 > ) & ( < condition 2 > ) ]**<br><br>\n",
        "   * Logical OR ( **|** )<br>\n",
        "     *Syntax*:<br>\n",
        "     *Note the parenthesis around each condition*<br>\n",
        "     **data_frame_object_name [ ( < condition 1 > ) | ( < condition 2 > ) ]**<br><br>\n",
        "   * Logical NOT ( **~** )<br>\n",
        "     **Use the ' ~ ' symbol to return the OPPOSITE of the selection**<br>\n",
        "     *Syntax*:<br>\n",
        "     **data_frame_object_name [ ~ ( < condition > ) ]**<br><br>\n",
        "   \n",
        "4. You can use the **'isin'** method to query a DataFrame based upon a list of column value<br>\n",
        "   \n",
        "   *Syntax*:<br>\n",
        "   **data_frame_object_name [ data_frame_object_name[ ' column_name ' ] . isin ( [ list of column values to search ] ) ]**<br>\n",
        "   or<br>\n",
        "   **data_frame_object_name [ data_frame_object_name . column_name . isin ( [ list of column values to search ] ) ]**\n",
        "\n",
        "   *Usage*:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "hQ8aVCgdE-8J"
      },
      "outputs": [],
      "source": [
        "surveys_df[(surveys_df.weight == 88) & (surveys_df.month == surveys_df.plot_id)]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "ILwuelMHE-8J"
      },
      "outputs": [],
      "source": [
        "surveys_df[surveys_df['weight'].isin([88,0.5,1])]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "9bj24FolE-8J"
      },
      "outputs": [],
      "source": [
        "surveys_df[~(surveys_df.weight.isin([88,0.5,1]))]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "OHFTBYRfE-8J"
      },
      "source": [
        "## **Exercise 1**\n",
        "#### Queries\n",
        "Perform the following queries\n",
        "1. Select a subset of rows in the surveys_df DataFrame that contain data from the year 1999 and weights less than or equal to 8.\n",
        "   How many rows did you end up with? What did your neighbor get?\n",
        "2. Display **record_id** and **plot_id** of data that contain data of the years 1999, 2000, 1977 and 1989.\n",
        "3. Write query that selects all rows with sex equal to ‘M’ or ‘F’ in the data. Write two different queries that produce this same result.Write two different queries that do the same thing."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "I17Ge7HmE-8K"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "9u5gSP-_E-8K"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "gD_Kc9zjE-8K"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "pE3p_bLgE-8K"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "i5m4-7DOE-8K"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "u-K-K3ocE-8K"
      },
      "source": [
        "### **🐫Summary statistics methods**\n",
        "**Calculate basic statistics for all records in all columns, a group of columns or a single column in a dataframe**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "KH1qP9IZE-8K"
      },
      "outputs": [],
      "source": [
        "surveys_df.describe()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "rUcnygk6E-8K"
      },
      "outputs": [],
      "source": [
        "surveys_df[['hindfoot_length','weight']].describe()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "QUAhgaiRE-8K"
      },
      "outputs": [],
      "source": [
        "surveys_df['record_id'].describe()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Fs72c6FXE-8L"
      },
      "source": [
        "**Extract one specific metric**\n",
        "\n",
        "**data_frame_object . min( )<br>\n",
        "data_frame_object . max( )<br>\n",
        "data_frame_object . mean( )<br>\n",
        "data_frame_object . std( )<br>\n",
        "data_frame_object . sum( )<br>\n",
        "data_frame_object . count( )**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 4,
      "metadata": {
        "id": "qvM2vx5hE-8L",
        "outputId": "15d310e0-7b4a-4bec-9f10-f5053b4152fd",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 147
        }
      },
      "outputs": [
        {
          "output_type": "error",
          "ename": "NameError",
          "evalue": "name 'surveys_df' is not defined",
          "traceback": [
            "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
            "\u001b[0;31mNameError\u001b[0m                                 Traceback (most recent call last)",
            "\u001b[0;32m<ipython-input-4-c7d3a59b8db9>\u001b[0m in \u001b[0;36m<cell line: 1>\u001b[0;34m()\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0msurveys_df\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mmin\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
            "\u001b[0;31mNameError\u001b[0m: name 'surveys_df' is not defined"
          ]
        }
      ],
      "source": [
        "surveys_df.min()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "327OwDoRE-8L"
      },
      "outputs": [],
      "source": [
        "surveys_df.mean()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "flFs7ThGE-8L"
      },
      "outputs": [],
      "source": [
        "surveys_df[['hindfoot_length','weight']].min()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "nr8OCp_GE-8L"
      },
      "outputs": [],
      "source": [
        "surveys_df[['hindfoot_length','weight']].mean()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "NA0JA6wXE-8L"
      },
      "outputs": [],
      "source": [
        "surveys_df['record_id'].min()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "JlMHdZQ6E-8L"
      },
      "outputs": [],
      "source": [
        "surveys_df['record_id'].mean()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "yh3Vg2UlE-8L"
      },
      "source": [
        "## **🐘Group by**\n",
        "**To summarize by one or more column**\n",
        "1. Use the **groupby ( )** pandas dataframe method. This creates a group by object in memory\n",
        "2. Use one on of the following summary statistics methods to get the grouped by data; **describe ( )**,**min ( )**,**max ( )**,**mean ( )**,**std ( )**,**count ( )**\n",
        "3. Use the **🐇agg( )** function to get different statistics on different columns in a group by object"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "metadata": {
        "id": "zkJc6Y8JE-8M",
        "outputId": "b9eb0f2c-8dee-4ca9-8f1a-7583e1ca5241",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 147
        }
      },
      "outputs": [
        {
          "output_type": "error",
          "ename": "NameError",
          "evalue": "name 'surveys_df' is not defined",
          "traceback": [
            "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
            "\u001b[0;31mNameError\u001b[0m                                 Traceback (most recent call last)",
            "\u001b[0;32m<ipython-input-1-69a504b8ca95>\u001b[0m in \u001b[0;36m<cell line: 1>\u001b[0;34m()\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0ma\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0msurveys_df\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mgroupby\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'sex'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
            "\u001b[0;31mNameError\u001b[0m: name 'surveys_df' is not defined"
          ]
        }
      ],
      "source": [
        "a=surveys_df.groupby('sex')"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "a.describe()"
      ],
      "metadata": {
        "id": "V0H2nhkBRKID",
        "outputId": "b5452d34-ac78-4626-8e67-4e0856412ed5",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 147
        }
      },
      "execution_count": 2,
      "outputs": [
        {
          "output_type": "error",
          "ename": "NameError",
          "evalue": "name 'a' is not defined",
          "traceback": [
            "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
            "\u001b[0;31mNameError\u001b[0m                                 Traceback (most recent call last)",
            "\u001b[0;32m<ipython-input-2-aad08ddc3c82>\u001b[0m in \u001b[0;36m<cell line: 1>\u001b[0;34m()\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0ma\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mdescribe\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
            "\u001b[0;31mNameError\u001b[0m: name 'a' is not defined"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 3,
      "metadata": {
        "id": "bRMucywoE-8M",
        "outputId": "48b6e44a-d6e0-4972-e970-32b0ba036a54",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 147
        }
      },
      "outputs": [
        {
          "output_type": "error",
          "ename": "NameError",
          "evalue": "name 'a' is not defined",
          "traceback": [
            "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
            "\u001b[0;31mNameError\u001b[0m                                 Traceback (most recent call last)",
            "\u001b[0;32m<ipython-input-3-d6ac9701668a>\u001b[0m in \u001b[0;36m<cell line: 1>\u001b[0;34m()\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0ma\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mmean\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
            "\u001b[0;31mNameError\u001b[0m: name 'a' is not defined"
          ]
        }
      ],
      "source": [
        "a.mean()"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "surveys_df[['sex','hindfoot_length','weight']].groupby('sex').describe()"
      ],
      "metadata": {
        "id": "r1KaJ-I3RQZy"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "surveys_df[['sex','species_id','hindfoot_length','weight']].groupby(['sex','species_id']).max()"
      ],
      "metadata": {
        "id": "GGeZTgShRSxa"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "ZK1iqhC5E-8M"
      },
      "outputs": [],
      "source": [
        "surveys_df[['sex','species_id','hindfoot_length','weight']].groupby(['sex','species_id'])['hindfoot_length'].max()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "KxMnKNv6E-8M"
      },
      "outputs": [],
      "source": [
        "surveys_df[['sex','species_id','hindfoot_length','weight']].groupby(['sex','species_id'])['weight'].sum()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "GsiGw3qkE-8M"
      },
      "outputs": [],
      "source": [
        "surveys_df[['sex','species_id','hindfoot_length','weight']].groupby(['sex','species_id']).agg({'hindfoot_length':'max','weight':'sum'})"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pglfOE4DE-8M"
      },
      "source": [
        "## **Exercise 2**\n",
        "#### Data Workflows and Automation\n",
        "1. Create a script/module called **'survey_years.py'** with the following\n",
        "   * Import the **pandas** and **os** modules\n",
        "   * Create a new folder called **'yearly_files'** using the **os.mkdir( ' data/yearly_files ' )** function\n",
        "   * Create a first function called **generate_year_files** that imports **'surveys.csv'** into a dataframe and do the following<br>\n",
        "     a) Read all the unique years<br>\n",
        "     b) For each year between **1990** and **2000**, create new **CSV** files with data from only one particular year.<br>\n",
        "     c) The name of the new files should follow the format **'survey_year.csv'** example **'survey_1977.csv'**<br>\n",
        "     **HINT**: Use the **for loop** and **if block**<br><br>\n",
        "   * Create the second function called **display_year_file_names** which reads names of the files generated and print them out.<br>\n",
        "     **HINTS**:<br>\n",
        "     Use the **os.listdir ( ' path/survey_\\* ' )** function to get the list of all files<br>\n",
        "     Use the **for loop** to loop through and print the file names<br><br>\n",
        "2. Import the **'survey_years.py'** module into your current session and use the two functions to generate the files for the years and print the file names      "
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 17,
      "metadata": {
        "id": "4QAJjg1GE-8N"
      },
      "outputs": [],
      "source": [
        "import os"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 18,
      "metadata": {
        "id": "c3lbwbuBE-8N"
      },
      "outputs": [],
      "source": [
        "os.mkdir('data')\n",
        "os.mkdir('data/yearlyfiles')"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "def generate_year_files():\n",
        "    surveys\n",
        "    #get a lits of unique years\n",
        "    #for year in list of unique years:\n",
        "      # years betweekn 1990 and 2000\n",
        "        #create a new databaframe for each year\n",
        "        #save that dataframe to file inside data/yearlyfile\n",
        "        #surveys_[year].csv\n",
        "        #dataframe_object_to_csv(file_path_name)\n",
        "        surveys_df = pd.read.to_csv\n",
        "\n",
        "def display_year_file_names():\n"
      ],
      "metadata": {
        "id": "PlaiKf8vRYwy"
      },
      "execution_count": 21,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qZuW2p9MNmxO"
      },
      "source": [
        "## **Bonus Exercise**\n",
        "\n",
        "* Grouping on each month, find the average hindfoot length\n",
        "* Find the number of entries and average weight for each plot\n",
        "* For every entry not during the summer (months June, July, August), find the largest hindfoot length"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2mKS68T6E-8N"
      },
      "source": [
        "## **Advanced and Future Topics**\n",
        "* Concatenation of dataFrames with the pandas concat function\n",
        "* Left, Right, Inner and Outer joins with the pandas merge function\n",
        "* Python for Data Visualization\n",
        "* Error Handling and Exceptions\n",
        "* List Comprehension\n",
        "* Regular Expressions\n",
        "* Database Access\n",
        "* Multithreading\n",
        "\n",
        "## **Helpful Resources**\n",
        "* CEAS Library Python resources - http://guides.libraries.uc.edu/python\n",
        "* Online links & tutorials\n",
        " * Data Carpentry - https://datacarpentry.org/python-ecology-lesson/\n",
        " * Python documentation - https://www.python.org/doc/\n",
        " * Python Programming wiki book - http://en.wikibooks.org/wiki/Python_Programming\n",
        " * Python tutorials - Udemy, Code academy, etc  "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pD6WTLkoE-8N"
      },
      "source": [
        "\n",
        "$\\;\\;\\;\\;\\;\\;\\;\\;\\;\\;\\;\\;\\;\\;\\;\\;\\;\\;\\;\\;\\;\\;\\;\\;\\;\\;\\;\\;\\;\\;\\;\\;\\;\\;\\;\\;\\;\\;\\;\\;\\;\\;\\;\\;\\;\\;\\;\\;\\;\\;\\;\\;\\;\\;\\;\\;\\;\\;$**Thank you for attending the workshop !!**\n",
        "\n",
        "\n",
        "$\\;\\;\\;\\;\\;\\;\\;\\;\\;\\;\\;\\;\\;\\;\\;\\;\\;\\;\\;\\;\\;\\;\\;\\;\\;\\;\\;\\;\\;\\;\\;\\;\\;\\;\\;\\;\\;\\;\\;\\;\\;\\;\\;\\;\\;\\;$**Your kind suggestions/feedbacks are more than welcome**"
      ]
    }
  ],
  "metadata": {
    "colab": {
      "provenance": []
    },
    "kernelspec": {
      "display_name": "Python 3.10.6 64-bit",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.10.6"
    },
    "vscode": {
      "interpreter": {
        "hash": "aee8b7b246df8f9039afb4144a1f6fd8d2ca17a180786b69acc140d282b71a49"
      }
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}