22_ntb_layer_bronze.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": [],
      "authorship_tag": "ABX9TyP4dz3LMjnVcGEzaZuZPCVV",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/luasampaio/data-engineering/blob/main/22_ntb_layer_bronze.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Configurações Iniciais e Importações**\n",
        "\n",
        "Aqui está um exemplo de um notebook em PySpark para implementar a arquitetura Medallion com as camadas Bronze, Silver e Gold.\n",
        "\n",
        "**Explicações:**\n",
        "\n",
        "- Importar bibliotecas e funções necessárias.\n",
        "- Definir os caminhos de arquivo para as camadas Bronze, Silver e Gold.\n",
        "- Configurar as definições do Spark para um desempenho ótimo, como partições de shuffle automático."
      ],
      "metadata": {
        "id": "MIfC1eBzBE4F"
      }
    },
    {
      "source": [
        "!pip install pyspark\n",
        "from pyspark.sql import SparkSession\n",
        "import pandas as pd\n",
        "\n",
        "# Create a SparkSession\n",
        "spark = SparkSession.builder.appName(\"LucianaSampaio\").getOrCreate()\n",
        "\n",
        "# URL para o conteúdo bruto do arquivo CSV\n",
        "url = 'https://raw.githubusercontent.com/luasampaio/datasets/main/customers.csv'\n",
        "\n"
      ],
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "_2ZpPvmZ8-SJ",
        "outputId": "86ca4489-fb1b-4bed-d493-9af1cf784673"
      },
      "execution_count": 19,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Requirement already satisfied: pyspark in /usr/local/lib/python3.11/dist-packages (3.5.4)\n",
            "Requirement already satisfied: py4j==0.10.9.7 in /usr/local/lib/python3.11/dist-packages (from pyspark) (0.10.9.7)\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "-  Lendo a URL\n"
      ],
      "metadata": {
        "id": "ZNFv396BAyJQ"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Lê o arquivo CSV diretamente do URL using pandas\n",
        "pandas_df = pd.read_csv(url)"
      ],
      "metadata": {
        "id": "USq_X0Fn9HK9"
      },
      "execution_count": 20,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Create a Spark DataFrame from the pandas DataFrame\n",
        "df = spark.createDataFrame(pandas_df)"
      ],
      "metadata": {
        "id": "zd8axB-G9JbJ"
      },
      "execution_count": 21,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Registrando um Spark DataFrame em uma tabela temporaria \"bronze\"\n",
        "df.registerTempTable(\"bronze\")"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "-x4UG_fp9LBm",
        "outputId": "825332f4-cea5-473b-d117-81ffec87ca72"
      },
      "execution_count": 22,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "/usr/local/lib/python3.11/dist-packages/pyspark/sql/dataframe.py:329: FutureWarning: Deprecated in 2.0, use createOrReplaceTempView instead.\n",
            "  warnings.warn(\"Deprecated in 2.0, use createOrReplaceTempView instead.\", FutureWarning)\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Now you can access the table using spark.table()\n",
        "df = spark.table(\"bronze\")\n",
        "display(df)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        },
        "id": "4ZwnmgRb9MSR",
        "outputId": "bb1c2dc1-28ff-40a8-e1fe-8b32beb90346"
      },
      "execution_count": 23,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "DataFrame[customer_id: bigint, first_name: string, last_name: string, phone: string, email: string, street: string, city: string, state: string, zip_code: bigint]"
            ]
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "df.count()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "k7X9znOl9bNk",
        "outputId": "f7618d29-0f31-4ad9-ce90-9b36bd1caa98"
      },
      "execution_count": 24,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "1445"
            ]
          },
          "metadata": {},
          "execution_count": 24
        }
      ]
    },
    {
      "source": [
        "# Em vez de usar 'bronze' diretamente, acesse-o através do Spark:\n",
        "spark.table(\"bronze\").show()"
      ],
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "TB9lMXpH9y7W",
        "outputId": "a69c12d8-dc63-4af5-9821-8d0b831316a5"
      },
      "execution_count": 25,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "+-----------+----------+---------+--------------+--------------------+--------------------+---------------+-----+--------+\n",
            "|customer_id|first_name|last_name|         phone|               email|              street|           city|state|zip_code|\n",
            "+-----------+----------+---------+--------------+--------------------+--------------------+---------------+-----+--------+\n",
            "|          1|     Debra|    Burks|           NaN|debra.burks@yahoo...|   9273 Thorne Ave. |   Orchard Park|   NY|   14127|\n",
            "|          2|     Kasha|     Todd|           NaN|kasha.todd@yahoo.com|    910 Vine Street |       Campbell|   CA|   95008|\n",
            "|          3|    Tameka|   Fisher|           NaN|tameka.fisher@aol...|769C Honey Creek ...|  Redondo Beach|   CA|   90278|\n",
            "|          4|     Daryl|   Spence|           NaN|daryl.spence@aol.com|     988 Pearl Lane |      Uniondale|   NY|   11553|\n",
            "|          5|Charolette|     Rice|(916) 381-6003|charolette.rice@m...|      107 River Dr. |     Sacramento|   CA|   95820|\n",
            "|          6|   Lyndsey|     Bean|           NaN|lyndsey.bean@hotm...|      769 West Road |       Fairport|   NY|   14450|\n",
            "|          7|   Latasha|     Hays|(716) 986-3359|latasha.hays@hotm...|7014 Manor Statio...|        Buffalo|   NY|   14215|\n",
            "|          8| Jacquline|   Duncan|           NaN|jacquline.duncan@...|       15 Brown St. |Jackson Heights|   NY|   11372|\n",
            "|          9|  Genoveva|  Baldwin|           NaN|genoveva.baldwin@...|  8550 Spruce Drive |Port Washington|   NY|   11050|\n",
            "|         10|   Pamelia|   Newman|           NaN|pamelia.newman@gm...|  476 Chestnut Ave. |         Monroe|   NY|   10950|\n",
            "|         11|   Deshawn|  Mendoza|           NaN|deshawn.mendoza@y...|8790 Cobblestone ...|         Monsey|   NY|   10952|\n",
            "|         12|     Robby|    Sykes|(516) 583-7761|robby.sykes@hotma...|486 Rock Maple St...|      Hempstead|   NY|   11550|\n",
            "|         13|   Lashawn|    Ortiz|           NaN|lashawn.ortiz@msn...|  27 Washington Rd. |       Longview|   TX|   75604|\n",
            "|         14|     Garry| Espinoza|           NaN|garry.espinoza@ho...|7858 Rockaway Court |         Forney|   TX|   75126|\n",
            "|         15|    Linnie|   Branch|           NaN|linnie.branch@gma...|314 South Columbi...|    Plattsburgh|   NY|   12901|\n",
            "|         16|    Emmitt|  Sanchez|(212) 945-8823|emmitt.sanchez@ho...|461 Squaw Creek R...|       New York|   NY|   10002|\n",
            "|         17|     Caren| Stephens|           NaN|caren.stephens@ms...|      914 Brook St. |      Scarsdale|   NY|   10583|\n",
            "|         18| Georgetta|   Hardin|           NaN|georgetta.hardin@...|     474 Chapel Dr. |    Canandaigua|   NY|   14424|\n",
            "|         19|  Lizzette|    Stein|           NaN|lizzette.stein@ya...| 19 Green Hill Lane |   Orchard Park|   NY|   14127|\n",
            "|         20|     Aleta|  Shepard|           NaN|aleta.shepard@aol...|     684 Howard St. |     Sugar Land|   TX|   77478|\n",
            "+-----------+----------+---------+--------------+--------------------+--------------------+---------------+-----+--------+\n",
            "only showing top 20 rows\n",
            "\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Temos também a opção de ler os dados conforme exemplo abaixo\n"
      ],
      "metadata": {
        "id": "-VR58WBSBuAL"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "spark.sql(\"SELECT * FROM bronze\").show()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "X111c3uQ-DeJ",
        "outputId": "e673d1d0-4568-488d-a875-80bd30094566"
      },
      "execution_count": 26,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "+-----------+----------+---------+--------------+--------------------+--------------------+---------------+-----+--------+\n",
            "|customer_id|first_name|last_name|         phone|               email|              street|           city|state|zip_code|\n",
            "+-----------+----------+---------+--------------+--------------------+--------------------+---------------+-----+--------+\n",
            "|          1|     Debra|    Burks|           NaN|debra.burks@yahoo...|   9273 Thorne Ave. |   Orchard Park|   NY|   14127|\n",
            "|          2|     Kasha|     Todd|           NaN|kasha.todd@yahoo.com|    910 Vine Street |       Campbell|   CA|   95008|\n",
            "|          3|    Tameka|   Fisher|           NaN|tameka.fisher@aol...|769C Honey Creek ...|  Redondo Beach|   CA|   90278|\n",
            "|          4|     Daryl|   Spence|           NaN|daryl.spence@aol.com|     988 Pearl Lane |      Uniondale|   NY|   11553|\n",
            "|          5|Charolette|     Rice|(916) 381-6003|charolette.rice@m...|      107 River Dr. |     Sacramento|   CA|   95820|\n",
            "|          6|   Lyndsey|     Bean|           NaN|lyndsey.bean@hotm...|      769 West Road |       Fairport|   NY|   14450|\n",
            "|          7|   Latasha|     Hays|(716) 986-3359|latasha.hays@hotm...|7014 Manor Statio...|        Buffalo|   NY|   14215|\n",
            "|          8| Jacquline|   Duncan|           NaN|jacquline.duncan@...|       15 Brown St. |Jackson Heights|   NY|   11372|\n",
            "|          9|  Genoveva|  Baldwin|           NaN|genoveva.baldwin@...|  8550 Spruce Drive |Port Washington|   NY|   11050|\n",
            "|         10|   Pamelia|   Newman|           NaN|pamelia.newman@gm...|  476 Chestnut Ave. |         Monroe|   NY|   10950|\n",
            "|         11|   Deshawn|  Mendoza|           NaN|deshawn.mendoza@y...|8790 Cobblestone ...|         Monsey|   NY|   10952|\n",
            "|         12|     Robby|    Sykes|(516) 583-7761|robby.sykes@hotma...|486 Rock Maple St...|      Hempstead|   NY|   11550|\n",
            "|         13|   Lashawn|    Ortiz|           NaN|lashawn.ortiz@msn...|  27 Washington Rd. |       Longview|   TX|   75604|\n",
            "|         14|     Garry| Espinoza|           NaN|garry.espinoza@ho...|7858 Rockaway Court |         Forney|   TX|   75126|\n",
            "|         15|    Linnie|   Branch|           NaN|linnie.branch@gma...|314 South Columbi...|    Plattsburgh|   NY|   12901|\n",
            "|         16|    Emmitt|  Sanchez|(212) 945-8823|emmitt.sanchez@ho...|461 Squaw Creek R...|       New York|   NY|   10002|\n",
            "|         17|     Caren| Stephens|           NaN|caren.stephens@ms...|      914 Brook St. |      Scarsdale|   NY|   10583|\n",
            "|         18| Georgetta|   Hardin|           NaN|georgetta.hardin@...|     474 Chapel Dr. |    Canandaigua|   NY|   14424|\n",
            "|         19|  Lizzette|    Stein|           NaN|lizzette.stein@ya...| 19 Green Hill Lane |   Orchard Park|   NY|   14127|\n",
            "|         20|     Aleta|  Shepard|           NaN|aleta.shepard@aol...|     684 Howard St. |     Sugar Land|   TX|   77478|\n",
            "+-----------+----------+---------+--------------+--------------------+--------------------+---------------+-----+--------+\n",
            "only showing top 20 rows\n",
            "\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Quando tiver dúvida do tipo do dataframe.\n",
        "\n",
        "print(type(df))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "zd-pgM6KGI1X",
        "outputId": "5c360827-c3f5-4c34-f0b2-1c98203734c0"
      },
      "execution_count": 27,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "<class 'pyspark.sql.dataframe.DataFrame'>\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "df.printSchema()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "t17OlguLGiZG",
        "outputId": "87a8e264-f7fd-48d9-9657-fab4ce05a4ee"
      },
      "execution_count": 28,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "root\n",
            " |-- customer_id: long (nullable = true)\n",
            " |-- first_name: string (nullable = true)\n",
            " |-- last_name: string (nullable = true)\n",
            " |-- phone: string (nullable = true)\n",
            " |-- email: string (nullable = true)\n",
            " |-- street: string (nullable = true)\n",
            " |-- city: string (nullable = true)\n",
            " |-- state: string (nullable = true)\n",
            " |-- zip_code: long (nullable = true)\n",
            "\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "- Concatenando em uma nova coluna\n"
      ],
      "metadata": {
        "id": "BeEMtt9vHVbF"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from pyspark.sql.functions import concat, col, lit\n",
        "\n",
        "df = df.withColumn(\"NomeCompleto\", concat(col(\"first_name\"), lit(\" \"), col(\"last_name\")))\n"
      ],
      "metadata": {
        "id": "xGYHlD0JGv8J"
      },
      "execution_count": 32,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "df.show()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "s7lLJPPXHA5R",
        "outputId": "7217e2f3-c835-4a6d-80e7-ca89a17e7802"
      },
      "execution_count": 33,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "+-----------+----------+---------+--------------+--------------------+--------------------+---------------+-----+--------+----------------+----------------+\n",
            "|customer_id|first_name|last_name|         phone|               email|              street|           city|state|zip_code|       full_name|    NomeCompleto|\n",
            "+-----------+----------+---------+--------------+--------------------+--------------------+---------------+-----+--------+----------------+----------------+\n",
            "|          1|     Debra|    Burks|           NaN|debra.burks@yahoo...|   9273 Thorne Ave. |   Orchard Park|   NY|   14127|     Debra Burks|     Debra Burks|\n",
            "|          2|     Kasha|     Todd|           NaN|kasha.todd@yahoo.com|    910 Vine Street |       Campbell|   CA|   95008|      Kasha Todd|      Kasha Todd|\n",
            "|          3|    Tameka|   Fisher|           NaN|tameka.fisher@aol...|769C Honey Creek ...|  Redondo Beach|   CA|   90278|   Tameka Fisher|   Tameka Fisher|\n",
            "|          4|     Daryl|   Spence|           NaN|daryl.spence@aol.com|     988 Pearl Lane |      Uniondale|   NY|   11553|    Daryl Spence|    Daryl Spence|\n",
            "|          5|Charolette|     Rice|(916) 381-6003|charolette.rice@m...|      107 River Dr. |     Sacramento|   CA|   95820| Charolette Rice| Charolette Rice|\n",
            "|          6|   Lyndsey|     Bean|           NaN|lyndsey.bean@hotm...|      769 West Road |       Fairport|   NY|   14450|    Lyndsey Bean|    Lyndsey Bean|\n",
            "|          7|   Latasha|     Hays|(716) 986-3359|latasha.hays@hotm...|7014 Manor Statio...|        Buffalo|   NY|   14215|    Latasha Hays|    Latasha Hays|\n",
            "|          8| Jacquline|   Duncan|           NaN|jacquline.duncan@...|       15 Brown St. |Jackson Heights|   NY|   11372|Jacquline Duncan|Jacquline Duncan|\n",
            "|          9|  Genoveva|  Baldwin|           NaN|genoveva.baldwin@...|  8550 Spruce Drive |Port Washington|   NY|   11050|Genoveva Baldwin|Genoveva Baldwin|\n",
            "|         10|   Pamelia|   Newman|           NaN|pamelia.newman@gm...|  476 Chestnut Ave. |         Monroe|   NY|   10950|  Pamelia Newman|  Pamelia Newman|\n",
            "|         11|   Deshawn|  Mendoza|           NaN|deshawn.mendoza@y...|8790 Cobblestone ...|         Monsey|   NY|   10952| Deshawn Mendoza| Deshawn Mendoza|\n",
            "|         12|     Robby|    Sykes|(516) 583-7761|robby.sykes@hotma...|486 Rock Maple St...|      Hempstead|   NY|   11550|     Robby Sykes|     Robby Sykes|\n",
            "|         13|   Lashawn|    Ortiz|           NaN|lashawn.ortiz@msn...|  27 Washington Rd. |       Longview|   TX|   75604|   Lashawn Ortiz|   Lashawn Ortiz|\n",
            "|         14|     Garry| Espinoza|           NaN|garry.espinoza@ho...|7858 Rockaway Court |         Forney|   TX|   75126|  Garry Espinoza|  Garry Espinoza|\n",
            "|         15|    Linnie|   Branch|           NaN|linnie.branch@gma...|314 South Columbi...|    Plattsburgh|   NY|   12901|   Linnie Branch|   Linnie Branch|\n",
            "|         16|    Emmitt|  Sanchez|(212) 945-8823|emmitt.sanchez@ho...|461 Squaw Creek R...|       New York|   NY|   10002|  Emmitt Sanchez|  Emmitt Sanchez|\n",
            "|         17|     Caren| Stephens|           NaN|caren.stephens@ms...|      914 Brook St. |      Scarsdale|   NY|   10583|  Caren Stephens|  Caren Stephens|\n",
            "|         18| Georgetta|   Hardin|           NaN|georgetta.hardin@...|     474 Chapel Dr. |    Canandaigua|   NY|   14424|Georgetta Hardin|Georgetta Hardin|\n",
            "|         19|  Lizzette|    Stein|           NaN|lizzette.stein@ya...| 19 Green Hill Lane |   Orchard Park|   NY|   14127|  Lizzette Stein|  Lizzette Stein|\n",
            "|         20|     Aleta|  Shepard|           NaN|aleta.shepard@aol...|     684 Howard St. |     Sugar Land|   TX|   77478|   Aleta Shepard|   Aleta Shepard|\n",
            "+-----------+----------+---------+--------------+--------------------+--------------------+---------------+-----+--------+----------------+----------------+\n",
            "only showing top 20 rows\n",
            "\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        " - Buscando Nome de pessoas que inicia com \"A\"\n"
      ],
      "metadata": {
        "id": "3awZiPfcIKIl"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "df_filtrado = df.filter(df.first_name.startswith(\"A\"))\n",
        "df_filtrado.show()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "GC1knurpH3ap",
        "outputId": "32b4ce0c-06c0-46ed-fae1-073171e27017"
      },
      "execution_count": 35,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "+-----------+----------+---------+--------------+--------------------+--------------------+----------------+-----+--------+----------------+----------------+\n",
            "|customer_id|first_name|last_name|         phone|               email|              street|            city|state|zip_code|       full_name|    NomeCompleto|\n",
            "+-----------+----------+---------+--------------+--------------------+--------------------+----------------+-----+--------+----------------+----------------+\n",
            "|         20|     Aleta|  Shepard|           NaN|aleta.shepard@aol...|     684 Howard St. |      Sugar Land|   TX|   77478|   Aleta Shepard|   Aleta Shepard|\n",
            "|         22|    Adelle|   Larsen|           NaN|adelle.larsen@gma...|683 West Kirkland...|  East Northport|   NY|   11731|   Adelle Larsen|   Adelle Larsen|\n",
            "|         32|   Araceli|   Golden|           NaN|araceli.golden@ms...|  12 Ridgeview Ave. |       Fullerton|   CA|   92831|  Araceli Golden|  Araceli Golden|\n",
            "|         62|     Alica|   Hunter|           NaN|alica.hunter@hotm...|   8 San Juan Drive |   East Elmhurst|   NY|   11369|    Alica Hunter|    Alica Hunter|\n",
            "|         75|      Abby|   Gamble|           NaN| abby.gamble@aol.com|       60 Myers Dr. |      Amityville|   NY|   11701|     Abby Gamble|     Abby Gamble|\n",
            "|         79|   Ashanti|    Parks|           NaN|ashanti.parks@hot...|   846 N. Helen St. |         Baldwin|   NY|   11510|   Ashanti Parks|   Ashanti Parks|\n",
            "|         84|    Armand|Whitehead|           NaN|armand.whitehead@...|     639 Harvey St. |     Lindenhurst|   NY|   11757|Armand Whitehead|Armand Whitehead|\n",
            "|         89|   Arvilla|   Osborn|           NaN|arvilla.osborn@gm...|     60 Acacia Lane |          Upland|   CA|   91784|  Arvilla Osborn|  Arvilla Osborn|\n",
            "|        158|     Alane|  Kennedy|           NaN|alane.kennedy@yah...|34 Green Lake Str...|   Spring Valley|   NY|   10977|   Alane Kennedy|   Alane Kennedy|\n",
            "|        193|    Ayanna|   Rhodes|           NaN|ayanna.rhodes@msn...|  7425 Pheasant St. |      Hicksville|   NY|   11801|   Ayanna Rhodes|   Ayanna Rhodes|\n",
            "|        195|     Addie|     Hahn|           NaN|addie.hahn@hotmai...|    8165 Baker Ave. | Franklin Square|   NY|   11010|      Addie Hahn|      Addie Hahn|\n",
            "|        197|      Andy|  O'neill|           NaN|andy.o'neill@msn.com|9544 Mulberry Drive |       Rego Park|   NY|   11374|    Andy O'neill|    Andy O'neill|\n",
            "|        200|    Ashely|   Holmes|           NaN|ashely.holmes@aol...| 45 High Ridge Ave. |     Farmingdale|   NY|   11735|   Ashely Holmes|   Ashely Holmes|\n",
            "|        204|     Alpha|     King|           NaN|alpha.king@hotmai...| 14 Henry Smith St. |Rockville Centre|   NY|   11570|      Alpha King|      Alpha King|\n",
            "|        222|      Anya|Contreras|           NaN|anya.contreras@ms...|38 Old Fairground...|  East Northport|   NY|   11731|  Anya Contreras|  Anya Contreras|\n",
            "|        267|  Angelina|    Lloyd|           NaN|angelina.lloyd@ya...|9593 North Sherma...|    Apple Valley|   CA|   92307|  Angelina Lloyd|  Angelina Lloyd|\n",
            "|        301|    Annett|  Garrett|           NaN|annett.garrett@ya...|     19 Euclid Lane |       Scarsdale|   NY|   10583|  Annett Garrett|  Annett Garrett|\n",
            "|        309|  Anderson|   Martin|           NaN|anderson.martin@h...|  9051 Hillside Dr. |     Canandaigua|   NY|   14424| Anderson Martin| Anderson Martin|\n",
            "|        320|      Aron|  Wiggins|(442) 497-3353|aron.wiggins@aol.com|     6 Bradford St. |       Oceanside|   NY|   11572|    Aron Wiggins|    Aron Wiggins|\n",
            "|        331|     Aleta|     Mack|           NaN|aleta.mack@yahoo.com|8869 W. Edgemont ...|       Bay Shore|   NY|   11706|      Aleta Mack|      Aleta Mack|\n",
            "+-----------+----------+---------+--------------+--------------------+--------------------+----------------+-----+--------+----------------+----------------+\n",
            "only showing top 20 rows\n",
            "\n"
          ]
        }
      ]
    },
    {
      "source": [
        "df_filtrado2 = df.filter(df.first_name.startswith(\"A\")).count()\n",
        "print(df_filtrado2)\n",
        "print('Quantidade de Pessoas que iniciam o nome com a Letra \"A\" é: ', df_filtrado2)"
      ],
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "1XDjphCIIjMu",
        "outputId": "584c6514-9f4a-49e2-b954-8a6241a0a1bc"
      },
      "execution_count": 39,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "114\n",
            "Quantidade de Pessoas que iniciam o nome com a Letra \"A\" é:  114\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "8J74LGzPI6XN"
      },
      "execution_count": null,
      "outputs": []
    }
  ]
}