Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replacing rtpa and mpo record_id with their key and name in dim_organizations #3786

Merged
merged 2 commits into from
Mar 19, 2025

Conversation

fsalemi
Copy link
Contributor

@fsalemi fsalemi commented Mar 19, 2025

Replacing rtpa and mpo record_id with their key and name in dim_organizations

The MPO and RTPA columns currently contain the source_record_id for these organizations, which is not ideal. In this PR, I am replacing these columns with the name and unique key of the organizations.

Before this change:

Before this change

After this change:

After this change

Resolves #[issue]

Type of change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation

How has this been tested?

jovyan@jupyter-fsalemi ~/data-infra/warehouse/models/mart/transit_database (rtpa_mpo_name_key) $ poetry run dbt run -s +dim_organizations
02:30:05  Running with dbt=1.5.1
02:30:08  [WARNING]: Configuration paths exist in your dbt_project.yml file which do not apply to any resources.
There are 1 unused configuration paths:
- models.calitp_warehouse.staging.ntd
02:30:09  Found 563 models, 1023 tests, 0 snapshots, 0 analyses, 852 macros, 0 operations, 12 seed files, 206 sources, 4 exposures, 0 metrics, 0 groups
02:30:09  
02:30:25  Concurrency: 8 threads (target='dev')
02:30:25  
02:30:25  1 of 3 START sql view model farhad_staging.stg_transit_database__organizations . [RUN]
02:30:26  1 of 3 OK created sql view model farhad_staging.stg_transit_database__organizations  [CREATE VIEW (0 processed) in 1.18s]
02:30:26  2 of 3 START sql table model farhad_staging.int_transit_database__organizations_dim  [RUN]
02:30:31  2 of 3 OK created sql table model farhad_staging.int_transit_database__organizations_dim  [CREATE TABLE (10.3k rows, 14.3 GiB processed) in 4.82s]
02:30:31  3 of 3 START sql table model farhad_mart_transit_database.dim_organizations .... [RUN]
02:30:33  3 of 3 OK created sql table model farhad_mart_transit_database.dim_organizations  [CREATE TABLE (10.3k rows, 2.2 MiB processed) in 2.65s]
02:30:33  
02:30:33  Finished running 1 view model, 2 table models in 0 hours 0 minutes and 24.65 seconds (24.65s).
02:30:34  
02:30:34  Completed successfully
02:30:34  
02:30:34  Done. PASS=3 WARN=0 ERROR=0 SKIP=0 TOTAL=3

Post-merge follow-ups

  • No action required
  • Actions required (specified below)

Copy link
Member

@evansiroky evansiroky left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work!

@fsalemi fsalemi merged commit 1619cdb into main Mar 19, 2025
3 of 4 checks passed
@fsalemi fsalemi deleted the rtpa_mpo_name_key branch March 19, 2025 23:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants