Udacity's Data Engineering With AWS Nano-Degree - Project: "STEDI Human Balance Analytics"

This is a fictional project for lesson 3 of Udacity's Data Engineering with AWS nano-degree to be reviewed by Udacity.

Although I think both, the data for this project and the rubric, are problematic (see question 972495 and question 972946 in Udacity's Knowledge Center as well as my sense_check.ipynb) and - after more than two weeks - Udacity was not able to provide answers (let alone solutions), I herewith try my best to come up with something to finish this project.

The overall task is to use the data provided in the project started folder for

• Customers,
• Accelerometer data, and
• Step Trainer data

to build a toy Datalake with

• Landing,
• Trusted, and
• Curated zones

using AWS services

• S3,
• Glue, and
• Athena

and document the various steps by saving the Glue job scripts and Athena table definitions and also making some screenshots.

Please find the documentation here:

1. Landing zone
   • customer data
   • accelerometer data
   • step trainer data
2. Trusted zone
   • customer data
   • accelerometer data
   • step trainer data
3. Curated zone
   • customer data
   • machine learning data (combination of accelerometer and step trainer data)

As the previous reviewer was somehow not able to see that customer data in the landing and trusted zone, differ by having sharewithresearchasofdata column with nulls and without nulls respectively, I added them more explicitely here:

• customer data in landing zone having data with sharewithresearchasofdata = NULL
• customer data in trusted zone having data with sharewithresearchasofdata = NULL