Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(insights): Update backend performance score function to handle missing vitals #82750

Merged

Conversation

edwardgou-sentry
Copy link
Contributor

@edwardgou-sentry edwardgou-sentry commented Dec 31, 2024

Currently, if a webvital is missing, we treat the individual score as 0. This can be misleading because it negatively affects the overall performance score of an organization/project.

Instead, we want to remove any missing vital factors from the overall performance score calculation, which is what this change aims to do by updating the performance_score function.

…ls, instead of just considering them to be zero
@github-actions github-actions bot added the Scope: Backend Automatically applied to PRs that change backend components label Dec 31, 2024
Copy link

codecov bot commented Dec 31, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

✅ All tests successful. No failed tests found.

Additional details and impacted files
@@            Coverage Diff             @@
##           master   #82750      +/-   ##
==========================================
+ Coverage   87.46%   87.66%   +0.20%     
==========================================
  Files        9407     9410       +3     
  Lines      536317   545069    +8752     
  Branches    21044    20977      -67     
==========================================
+ Hits       469088   477835    +8747     
- Misses      66859    66863       +4     
- Partials      370      371       +1     

# TODO: Is there a way to sum more than 2 values at once?
return Function(
"plus",
"divide",
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We divide by the maximum possible weight (which will be some value between 0 - 1.0 depending on which vitals are present). This will scale up the result if there are any vitals missing.

@edwardgou-sentry edwardgou-sentry marked this pull request as ready for review December 31, 2024 17:34
@edwardgou-sentry edwardgou-sentry requested a review from a team as a code owner December 31, 2024 17:34
"if",
[
Function(
"isZeroOrNull",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Both a 0 or a null value represent a missing vital score?

This makes sense to me, because you can't ever have an instantaneous load time for any vital metric.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The isZeroOrNull condition is ran on the count of the metric so we're really just checking for the existence of the vital, not the actual value. This is false if an lcp with value of 0ms is reported, because the count is > 0 in this case.

You're also right, it shouldn't really be possible for load time vitals to be 0ms. It might be possible for cls to be 0 though, and that would depend on if the sdk reports it.

Not sure if there's an easier clickhouse or snuba function to do this check, but this works. We'll see if there are any suggestions

@bcoe
Copy link
Member

bcoe commented Jan 2, 2025

Refs: #77071

Copy link
Member

@gggritso gggritso left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code makes sense! Could you add a small explanation to the function, though? The main thing that confused me is that there are weight columns in the actual metric buckets rows in ClickHouse and now there are weights when calculating the score. What's the difference between the weights?

@edwardgou-sentry
Copy link
Contributor Author

The code makes sense! Could you add a small explanation to the function, though? The main thing that confused me is that there are weight columns in the actual metric buckets rows in ClickHouse and now there are weights when calculating the score. What's the difference between the weights?

Added an explanation!

@edwardgou-sentry edwardgou-sentry merged commit 54ff4ec into master Jan 2, 2025
49 checks passed
@edwardgou-sentry edwardgou-sentry deleted the egou/feat/performance-score-functions-missing-vitals branch January 2, 2025 19:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Scope: Backend Automatically applied to PRs that change backend components
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants