Weeks 7–12 | Milestone: M3 — Clean Dataset
Raw data is never analysis-ready. Phase 3 is about understanding what you have, fixing what's broken, and producing a clean dataset you can trust.
By the end of Week 12, you will have:
- A schema plan defining key tables/files, grain, and identifiers
- SQL queries answering at least 3 business questions from your data
- A clean, structured dataset saved in
/data/processed/ - A reproducible transform script with documented cleaning decisions and validation checks
- Completed Milestone 3 submission
| Week | Focus | Deliverable |
|---|---|---|
| Week 7 | Storage design & data modeling (⚡ moved from Phase 2) | Schema draft in README or separate design note |
| Week 8 | SQL fundamentals for data projects (⚡ moved from Phase 2) | SQL script with 3 answered business questions |
| Week 9 | Pandas fundamentals: cleaning & shaping | First cleaned dataset version committed |
| Week 10 | Advanced transformation & joins | Processed dataset + transformation logic documented |
| Week 11 | Data quality, validation & reproducibility | Validation notes + reproducible transform flow |
| Week 12 | Pipeline structuring & local orchestration | Clear script flow + repo instructions for running the project |
Due: By Week 12
Pass criteria:
- Processed dataset saved in
/data/processed/ - Schema plan documented (key tables/files, grain, identifiers, expected columns)
- SQL queries (Week 8) documented alongside the clean dataset
- Missing values handled and cleaning decisions logged
- Validation checks added to the transformation flow
- Transform script is reproducible — same output on every run
Submission: Open a Milestone Issue