-
Notifications
You must be signed in to change notification settings - Fork 15
Fix: correct some Benefits app versions #4529
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
|
Terraform plan in iac/cal-itp-data-infra-staging/composer/us Plan: 0 to add, 1 to change, 0 to destroy.Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
!~ update in-place
Terraform will perform the following actions:
# google_composer_environment.calitp-staging-composer will be updated in-place
!~ resource "google_composer_environment" "calitp-staging-composer" {
id = "projects/cal-itp-data-infra-staging/locations/us-west2/environments/calitp-staging-composer"
name = "calitp-staging-composer"
# (5 unchanged attributes hidden)
!~ config {
# (8 unchanged attributes hidden)
!~ software_config {
!~ pypi_packages = {
+ "calitp-data-infra" = "==2025.6.5"
+ "pydantic" = ">=1.9,<2.0"
# (11 unchanged elements hidden)
}
# (6 unchanged attributes hidden)
# (1 unchanged block hidden)
}
# (8 unchanged blocks hidden)
}
# (1 unchanged block hidden)
}
Plan: 0 to add, 1 to change, 0 to destroy.📝 Plan generated in Plan Terraform for Warehouse and DAG changes #1074 |
|
Terraform plan in iac/cal-itp-data-infra/composer/us No changes. Your infrastructure matches the configuration.📝 Plan generated in Plan Terraform for Warehouse and DAG changes #1074 |
b13a4b8 to
42aa933
Compare
|
Terraform plan in iac/cal-itp-data-infra-staging/airflow/us Plan: 0 to add, 18 to change, 3 to destroy.Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
!~ update in-place
- destroy
Terraform will perform the following actions:
# google_storage_bucket_object.calitp-staging-composer["dags/airtable_loader_v2/generate_gtfs_download_configs.py"] will be updated in-place
!~ resource "google_storage_bucket_object" "calitp-staging-composer" {
!~ crc32c = "MijWlA==" -> (known after apply)
!~ detect_md5hash = "s632w01yc8uo408y4VdAyw==" -> "different hash"
!~ generation = 1763690417635115 -> (known after apply)
id = "calitp-staging-composer-dags/airtable_loader_v2/generate_gtfs_download_configs.py"
!~ md5hash = "s632w01yc8uo408y4VdAyw==" -> (known after apply)
name = "dags/airtable_loader_v2/generate_gtfs_download_configs.py"
# (17 unchanged attributes hidden)
}
# google_storage_bucket_object.calitp-staging-composer["dags/download_gtfs_schedule_v2/download_schedule_feeds.py"] will be updated in-place
!~ resource "google_storage_bucket_object" "calitp-staging-composer" {
!~ crc32c = "8qLecA==" -> (known after apply)
!~ detect_md5hash = "iGapm0xJ3U0wowUUkId1eQ==" -> "different hash"
!~ generation = 1763690417218909 -> (known after apply)
id = "calitp-staging-composer-dags/download_gtfs_schedule_v2/download_schedule_feeds.py"
!~ md5hash = "iGapm0xJ3U0wowUUkId1eQ==" -> (known after apply)
name = "dags/download_gtfs_schedule_v2/download_schedule_feeds.py"
# (17 unchanged attributes hidden)
}
# google_storage_bucket_object.calitp-staging-composer["dags/sync_ntd_data_xlsx/scrape_ntd_xlsx_urls.py"] will be updated in-place
!~ resource "google_storage_bucket_object" "calitp-staging-composer" {
!~ crc32c = "g2TBuw==" -> (known after apply)
!~ detect_md5hash = "PSes9rK7j0FP6JcNRQhPLg==" -> "different hash"
!~ generation = 1763690417804807 -> (known after apply)
id = "calitp-staging-composer-dags/sync_ntd_data_xlsx/scrape_ntd_xlsx_urls.py"
!~ md5hash = "PSes9rK7j0FP6JcNRQhPLg==" -> (known after apply)
name = "dags/sync_ntd_data_xlsx/scrape_ntd_xlsx_urls.py"
# (17 unchanged attributes hidden)
}
# google_storage_bucket_object.calitp-staging-composer["plugins/calitp_data_infra/__init__.py"] will be destroyed
# (because key ["plugins/calitp_data_infra/__init__.py"] is not in for_each map)
- resource "google_storage_bucket_object" "calitp-staging-composer" {
- bucket = "calitp-staging-composer" -> null
- content_type = "text/plain; charset=utf-8" -> null
- crc32c = "AAAAAA==" -> null
- detect_md5hash = "1B2M2Y8AsgTpgAmY7PhCfg==" -> null
- event_based_hold = false -> null
- generation = 1763690417199832 -> null
- id = "calitp-staging-composer-plugins/calitp_data_infra/__init__.py" -> null
- md5hash = "1B2M2Y8AsgTpgAmY7PhCfg==" -> null
- md5hexhash = "d41d8cd98f00b204e9800998ecf8427e" -> null
- media_link = "https://storage.googleapis.com/download/storage/v1/b/calitp-staging-composer/o/plugins%2Fcalitp_data_infra%2F__init__.py?generation=1763690417199832&alt=media" -> null
- metadata = {} -> null
- name = "plugins/calitp_data_infra/__init__.py" -> null
- output_name = "plugins/calitp_data_infra/__init__.py" -> null
- self_link = "https://www.googleapis.com/storage/v1/b/calitp-staging-composer/o/plugins%2Fcalitp_data_infra%2F__init__.py" -> null
- source = "../../../../airflow/plugins/calitp_data_infra/__init__.py" -> null
- storage_class = "STANDARD" -> null
- temporary_hold = false -> null
# (6 unchanged attributes hidden)
}
# google_storage_bucket_object.calitp-staging-composer["plugins/calitp_data_infra/auth.py"] will be destroyed
# (because key ["plugins/calitp_data_infra/auth.py"] is not in for_each map)
- resource "google_storage_bucket_object" "calitp-staging-composer" {
- bucket = "calitp-staging-composer" -> null
- content_type = "text/plain; charset=utf-8" -> null
- crc32c = "6lsUtA==" -> null
- detect_md5hash = "+/KTbwc3sd3B4wBkY+HoUw==" -> null
- event_based_hold = false -> null
- generation = 1763690417207979 -> null
- id = "calitp-staging-composer-plugins/calitp_data_infra/auth.py" -> null
- md5hash = "+/KTbwc3sd3B4wBkY+HoUw==" -> null
- md5hexhash = "fbf2936f0737b1ddc1e3006463e1e853" -> null
- media_link = "https://storage.googleapis.com/download/storage/v1/b/calitp-staging-composer/o/plugins%2Fcalitp_data_infra%2Fauth.py?generation=1763690417207979&alt=media" -> null
- metadata = {} -> null
- name = "plugins/calitp_data_infra/auth.py" -> null
- output_name = "plugins/calitp_data_infra/auth.py" -> null
- self_link = "https://www.googleapis.com/storage/v1/b/calitp-staging-composer/o/plugins%2Fcalitp_data_infra%2Fauth.py" -> null
- source = "../../../../airflow/plugins/calitp_data_infra/auth.py" -> null
- storage_class = "STANDARD" -> null
- temporary_hold = false -> null
# (6 unchanged attributes hidden)
}
# google_storage_bucket_object.calitp-staging-composer["plugins/calitp_data_infra/storage.py"] will be destroyed
# (because key ["plugins/calitp_data_infra/storage.py"] is not in for_each map)
- resource "google_storage_bucket_object" "calitp-staging-composer" {
- bucket = "calitp-staging-composer" -> null
- content_type = "text/plain; charset=utf-8" -> null
- crc32c = "b87VYA==" -> null
- detect_md5hash = "sq1Q+wmsL8o0RKJLFUFC7g==" -> null
- event_based_hold = false -> null
- generation = 1763690417395966 -> null
- id = "calitp-staging-composer-plugins/calitp_data_infra/storage.py" -> null
- md5hash = "sq1Q+wmsL8o0RKJLFUFC7g==" -> null
- md5hexhash = "b2ad50fb09ac2fca3444a24b154142ee" -> null
- media_link = "https://storage.googleapis.com/download/storage/v1/b/calitp-staging-composer/o/plugins%2Fcalitp_data_infra%2Fstorage.py?generation=1763690417395966&alt=media" -> null
- metadata = {} -> null
- name = "plugins/calitp_data_infra/storage.py" -> null
- output_name = "plugins/calitp_data_infra/storage.py" -> null
- self_link = "https://www.googleapis.com/storage/v1/b/calitp-staging-composer/o/plugins%2Fcalitp_data_infra%2Fstorage.py" -> null
- source = "../../../../airflow/plugins/calitp_data_infra/storage.py" -> null
- storage_class = "STANDARD" -> null
- temporary_hold = false -> null
# (6 unchanged attributes hidden)
}
# google_storage_bucket_object.calitp-staging-composer["plugins/hooks/kuba_hook.py"] will be updated in-place
!~ resource "google_storage_bucket_object" "calitp-staging-composer" {
!~ crc32c = "pIf6jA==" -> (known after apply)
!~ detect_md5hash = "M9n0Cr7dL9+4asfxnMHgjQ==" -> "different hash"
!~ generation = 1763690418428084 -> (known after apply)
id = "calitp-staging-composer-plugins/hooks/kuba_hook.py"
!~ md5hash = "M9n0Cr7dL9+4asfxnMHgjQ==" -> (known after apply)
name = "plugins/hooks/kuba_hook.py"
# (17 unchanged attributes hidden)
}
# google_storage_bucket_object.calitp-staging-composer["plugins/hooks/soda_hook.py"] will be updated in-place
!~ resource "google_storage_bucket_object" "calitp-staging-composer" {
!~ crc32c = "Wxj+aQ==" -> (known after apply)
!~ detect_md5hash = "CDAoj9pONq2nsD5BT43vEg==" -> "different hash"
!~ generation = 1763690418284709 -> (known after apply)
id = "calitp-staging-composer-plugins/hooks/soda_hook.py"
!~ md5hash = "CDAoj9pONq2nsD5BT43vEg==" -> (known after apply)
name = "plugins/hooks/soda_hook.py"
# (17 unchanged attributes hidden)
}
# google_storage_bucket_object.calitp-staging-composer["plugins/hooks/transitland_hook.py"] will be updated in-place
!~ resource "google_storage_bucket_object" "calitp-staging-composer" {
!~ crc32c = "xolZ/w==" -> (known after apply)
!~ detect_md5hash = "lMSV7OyTWTBE5ar8Ush1oA==" -> "different hash"
!~ generation = 1763690417210931 -> (known after apply)
id = "calitp-staging-composer-plugins/hooks/transitland_hook.py"
!~ md5hash = "lMSV7OyTWTBE5ar8Ush1oA==" -> (known after apply)
name = "plugins/hooks/transitland_hook.py"
# (17 unchanged attributes hidden)
}
# google_storage_bucket_object.calitp-staging-composer["plugins/operators/aggregator_to_gcs_operator.py"] will be updated in-place
!~ resource "google_storage_bucket_object" "calitp-staging-composer" {
!~ crc32c = "h26Utg==" -> (known after apply)
!~ detect_md5hash = "+2mzXgNPW36mWf5Z3ZZBag==" -> "different hash"
!~ generation = 1763690417144134 -> (known after apply)
id = "calitp-staging-composer-plugins/operators/aggregator_to_gcs_operator.py"
!~ md5hash = "+2mzXgNPW36mWf5Z3ZZBag==" -> (known after apply)
name = "plugins/operators/aggregator_to_gcs_operator.py"
# (17 unchanged attributes hidden)
}
# google_storage_bucket_object.calitp-staging-composer["plugins/operators/blackcat_to_gcs_operator.py"] will be updated in-place
!~ resource "google_storage_bucket_object" "calitp-staging-composer" {
!~ crc32c = "s7BJ7Q==" -> (known after apply)
!~ detect_md5hash = "husvfrVLOwWESpUczaAT3w==" -> "different hash"
!~ generation = 1763690417861255 -> (known after apply)
id = "calitp-staging-composer-plugins/operators/blackcat_to_gcs_operator.py"
!~ md5hash = "husvfrVLOwWESpUczaAT3w==" -> (known after apply)
name = "plugins/operators/blackcat_to_gcs_operator.py"
# (17 unchanged attributes hidden)
}
# google_storage_bucket_object.calitp-staging-composer["plugins/operators/dbt_manifest_to_dictionary_operator.py"] will be updated in-place
!~ resource "google_storage_bucket_object" "calitp-staging-composer" {
!~ crc32c = "4MyK1Q==" -> (known after apply)
!~ detect_md5hash = "FIHwYcjOm5NPB+nPb7enyg==" -> "different hash"
!~ generation = 1763690416913428 -> (known after apply)
id = "calitp-staging-composer-plugins/operators/dbt_manifest_to_dictionary_operator.py"
!~ md5hash = "FIHwYcjOm5NPB+nPb7enyg==" -> (known after apply)
name = "plugins/operators/dbt_manifest_to_dictionary_operator.py"
# (17 unchanged attributes hidden)
}
# google_storage_bucket_object.calitp-staging-composer["plugins/operators/dbt_manifest_to_metadata_operator.py"] will be updated in-place
!~ resource "google_storage_bucket_object" "calitp-staging-composer" {
!~ crc32c = "UDX9og==" -> (known after apply)
!~ detect_md5hash = "F4n9FAx9ExF1Zl3J7NaMWQ==" -> "different hash"
!~ generation = 1763690417995295 -> (known after apply)
id = "calitp-staging-composer-plugins/operators/dbt_manifest_to_metadata_operator.py"
!~ md5hash = "F4n9FAx9ExF1Zl3J7NaMWQ==" -> (known after apply)
name = "plugins/operators/dbt_manifest_to_metadata_operator.py"
# (17 unchanged attributes hidden)
}
# google_storage_bucket_object.calitp-staging-composer["plugins/operators/gtfs_csv_to_jsonl_hourly.py"] will be updated in-place
!~ resource "google_storage_bucket_object" "calitp-staging-composer" {
!~ crc32c = "pCzW9g==" -> (known after apply)
!~ detect_md5hash = "n34Iv1u6y/wZeOnqO2WDEA==" -> "different hash"
!~ generation = 1763690417647780 -> (known after apply)
id = "calitp-staging-composer-plugins/operators/gtfs_csv_to_jsonl_hourly.py"
!~ md5hash = "n34Iv1u6y/wZeOnqO2WDEA==" -> (known after apply)
name = "plugins/operators/gtfs_csv_to_jsonl_hourly.py"
# (17 unchanged attributes hidden)
}
# google_storage_bucket_object.calitp-staging-composer["plugins/operators/littlepay_raw_sync_feed_v3.py"] will be updated in-place
!~ resource "google_storage_bucket_object" "calitp-staging-composer" {
!~ crc32c = "TDDjLg==" -> (known after apply)
!~ detect_md5hash = "zXyVjenO8Z0Xcx4u85EVYQ==" -> "different hash"
!~ generation = 1763690417211071 -> (known after apply)
id = "calitp-staging-composer-plugins/operators/littlepay_raw_sync_feed_v3.py"
!~ md5hash = "zXyVjenO8Z0Xcx4u85EVYQ==" -> (known after apply)
name = "plugins/operators/littlepay_raw_sync_feed_v3.py"
# (17 unchanged attributes hidden)
}
# google_storage_bucket_object.calitp-staging-composer["plugins/operators/pod_operator.py"] will be updated in-place
!~ resource "google_storage_bucket_object" "calitp-staging-composer" {
!~ crc32c = "H81Llg==" -> (known after apply)
!~ detect_md5hash = "6vO0LHE3p5d/cOQ71Ghv8g==" -> "different hash"
!~ generation = 1763690417200759 -> (known after apply)
id = "calitp-staging-composer-plugins/operators/pod_operator.py"
!~ md5hash = "6vO0LHE3p5d/cOQ71Ghv8g==" -> (known after apply)
name = "plugins/operators/pod_operator.py"
# (17 unchanged attributes hidden)
}
# google_storage_bucket_object.calitp-staging-composer["plugins/operators/scrape_ntd_xlsx.py"] will be updated in-place
!~ resource "google_storage_bucket_object" "calitp-staging-composer" {
!~ crc32c = "800p7w==" -> (known after apply)
!~ detect_md5hash = "+sUY5347tlkwmkjx/59Ytg==" -> "different hash"
!~ generation = 1763690417819438 -> (known after apply)
id = "calitp-staging-composer-plugins/operators/scrape_ntd_xlsx.py"
!~ md5hash = "+sUY5347tlkwmkjx/59Ytg==" -> (known after apply)
name = "plugins/operators/scrape_ntd_xlsx.py"
# (17 unchanged attributes hidden)
}
# google_storage_bucket_object.calitp-staging-composer["plugins/operators/scrape_state_geoportal.py"] will be updated in-place
!~ resource "google_storage_bucket_object" "calitp-staging-composer" {
!~ crc32c = "HSTs0g==" -> (known after apply)
!~ detect_md5hash = "kroDAzyyod9g32UxYWccew==" -> "different hash"
!~ generation = 1763690418239800 -> (known after apply)
id = "calitp-staging-composer-plugins/operators/scrape_state_geoportal.py"
!~ md5hash = "kroDAzyyod9g32UxYWccew==" -> (known after apply)
name = "plugins/operators/scrape_state_geoportal.py"
# (17 unchanged attributes hidden)
}
# google_storage_bucket_object.calitp-staging-composer["plugins/scripts/gtfs_rt_parser.py"] will be updated in-place
!~ resource "google_storage_bucket_object" "calitp-staging-composer" {
!~ crc32c = "XjBzGA==" -> (known after apply)
!~ detect_md5hash = "BfvE2ctBSfWnSYBpIv2hOg==" -> "different hash"
!~ generation = 1763690417837883 -> (known after apply)
id = "calitp-staging-composer-plugins/scripts/gtfs_rt_parser.py"
!~ md5hash = "BfvE2ctBSfWnSYBpIv2hOg==" -> (known after apply)
name = "plugins/scripts/gtfs_rt_parser.py"
# (17 unchanged attributes hidden)
}
# google_storage_bucket_object.calitp-staging-composer["plugins/utils.py"] will be updated in-place
!~ resource "google_storage_bucket_object" "calitp-staging-composer" {
!~ crc32c = "2wseZw==" -> (known after apply)
!~ detect_md5hash = "L7u5luMr8c8OXJVHR7nzEg==" -> "different hash"
!~ generation = 1763690417826500 -> (known after apply)
id = "calitp-staging-composer-plugins/utils.py"
!~ md5hash = "L7u5luMr8c8OXJVHR7nzEg==" -> (known after apply)
name = "plugins/utils.py"
# (17 unchanged attributes hidden)
}
# google_storage_bucket_object.calitp-staging-composer-dags["models/mart/benefits/fct_benefits_events.sql"] will be updated in-place
!~ resource "google_storage_bucket_object" "calitp-staging-composer-dags" {
!~ crc32c = "AmoX1g==" -> (known after apply)
!~ detect_md5hash = "32e2YxZ8YVVpaVcbMfrV+w==" -> "different hash"
!~ generation = 1762380824121196 -> (known after apply)
id = "calitp-staging-composer-data/warehouse/models/mart/benefits/fct_benefits_events.sql"
!~ md5hash = "32e2YxZ8YVVpaVcbMfrV+w==" -> (known after apply)
name = "data/warehouse/models/mart/benefits/fct_benefits_events.sql"
# (17 unchanged attributes hidden)
}
Plan: 0 to add, 18 to change, 3 to destroy.📝 Plan generated in Plan Terraform for Warehouse and DAG changes #1074 |
|
Terraform plan in iac/cal-itp-data-infra/airflow/us Plan: 0 to add, 1 to change, 0 to destroy.Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
!~ update in-place
Terraform will perform the following actions:
# google_storage_bucket_object.calitp-composer-dags["models/mart/benefits/fct_benefits_events.sql"] will be updated in-place
!~ resource "google_storage_bucket_object" "calitp-composer-dags" {
!~ crc32c = "AmoX1g==" -> (known after apply)
!~ detect_md5hash = "32e2YxZ8YVVpaVcbMfrV+w==" -> "different hash"
!~ generation = 1762459059031940 -> (known after apply)
id = "calitp-composer-data/warehouse/models/mart/benefits/fct_benefits_events.sql"
!~ md5hash = "32e2YxZ8YVVpaVcbMfrV+w==" -> (known after apply)
name = "data/warehouse/models/mart/benefits/fct_benefits_events.sql"
# (17 unchanged attributes hidden)
}
Plan: 0 to add, 1 to change, 0 to destroy.📝 Plan generated in Plan Terraform for Warehouse and DAG changes #1074 |
|
Warehouse report 📦 DAGLegend (in order of precedence)
|
42aa933 to
c796186
Compare
caused by a bug in the Docker build process
c796186 to
4ac75b5
Compare
|
I'm pretty sure the DBT error is unrelated to these changes, it is referencing a different table:
|

Description
Caused by a bug in our Docker build process, see cal-itp/benefits#3314 for the fix at the source.
This PR adjusts versions for historical events going back to August 2025, when the bug was introduced.
Resolves cal-itp/benefits#3319
Type of change
How has this been tested?
Include commands/logs/screenshots as relevant.
If making changes to dbt models, please run the command
poetry run dbt run -s CHANGED_MODELandpoetry run dbt test -s CHANGED_MODEL, then include the output in this section of the PR.Post-merge follow-ups
Document any actions that must be taken post-merge to deploy or otherwise implement the changes in this PR (for example, running a full refresh of some incremental model in dbt). If these actions will take more than a few hours after the merge or if they will be completed by someone other than the PR author, please create a dedicated follow-up issue and link it here to track resolution.