Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CORE-41] Update BQ spending report query to remove duplicates #3161

Merged
merged 4 commits into from
Jan 8, 2025

Conversation

calypsomatic
Copy link
Contributor

@calypsomatic calypsomatic commented Jan 6, 2025

Ticket: https://broadworkbench.atlassian.net/jira/software/c/projects/CORE/boards/214
For unknown reasons, BigQuery sometimes has rows in the spending table with null project names, resulting in a single project ID receiving two results: one with and one without the project name. This causes weirdness in the UI as well as inaccurate results. This PR updates the query to ignore project name altogether when consolidating rows, since it's not used in the result.


PR checklist

  • Include the JIRA issue number in the PR description and title
  • Make sure Swagger is updated if API changes
    • ...and Orchestration's Swagger too!
  • If you changed anything in model/, then you should publish a new official rawls-model and perform the corresponding dependency updates as specified in the README:
    • in the automation subdirectory
    • in workbench-libs
    • in firecloud-orchestration
  • Get two thumbsworth of PR review
  • Verify all tests go green, including CI tests
  • Squash commits and merge to develop (branches are automatically deleted after merging)
  • Inform other teams of any substantial changes via Slack and/or email

@calypsomatic calypsomatic changed the title attempt to update query to remove duplicates [CORE-41] Update BQ spending report query to remove duplicates Jan 7, 2025
@calypsomatic calypsomatic marked this pull request as ready for review January 7, 2025 15:48
@calypsomatic calypsomatic requested a review from a team as a code owner January 7, 2025 15:48
@calypsomatic calypsomatic requested review from kevinpalis and marctalbott and removed request for a team January 7, 2025 15:48
Copy link

@kevinpalis kevinpalis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

@marctalbott marctalbott left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd recommend just removing the project name from the query entirely. We aren't using it in the results at all and the project id is sufficient to identify where the spend came from.

@calypsomatic
Copy link
Contributor Author

I'd recommend just removing the project name from the query entirely. We aren't using it in the results at all and the project id is sufficient to identify where the spend came from.

See this is why they pay you the big money

@calypsomatic calypsomatic merged commit 02e9f4f into develop Jan 8, 2025
29 checks passed
@calypsomatic calypsomatic deleted the core-41-spend-report branch January 8, 2025 13:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants