Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Horreum] Infinispan metric calculation sometimes results to division to 0 #832 #833

Merged
merged 1 commit into from
May 28, 2024

Conversation

andyuk1986
Copy link
Contributor

Fixes #832

@andyuk1986
Copy link
Contributor Author

You can find a successful run on my fork here: https://github.com/andyuk1986/keycloak-benchmark/actions/runs/9267087730

@andyuk1986
Copy link
Contributor Author

So, I am afraid that the generated report data so far was only correct for memoryUsageTest, for other 2 test runs (cpuUsage and cpuUsageForGrants was incorrectly calculated).

@ahus1
Copy link
Contributor

ahus1 commented May 28, 2024

@andyuk1986 - thank you for figuring out the problem!

To help me understand the root cause: Can you please describe it to me? Looking at the diff, the rm seems to do the trick?

@ahus1 ahus1 self-assigned this May 28, 2024
@ahus1 ahus1 self-requested a review May 28, 2024 12:06
@andyuk1986
Copy link
Contributor Author

andyuk1986 commented May 28, 2024

@andyuk1986 - thank you for figuring out the problem!

To help me understand the root cause: Can you please describe it to me? Looking at the diff, the rm seems to do the trick?
@ahus1 actually there were 2 problems:

  1. The floating point math operation in bash sometimes were throwing error which was causing that data was completely missing for specific test. > I have fixed this by rounding the data already on prometheus level.
  2. The 'rm' part was the reason of devision to 0 issue. When the statistic is read from prometheus the data is stored in the file. This is done before and after the benchmark ran and is written on new line. Then the metric calculator reads the data and makes math operations to calculate that and theoretically should remove the file for next bechmark iteration as the next benchmark uses the same file name. I was deleting one of the files and missed the second, as for ispn metric calculation I am picking 2 time sub-metrics (seconds sum and count) and store them in different files for later processing. So as one of the files was not deleted, the later benchmarks were adding the new statistics to already existing file where already data was from previous benchmark run, and after that the whole calculation was going wrong.

@ahus1 ahus1 merged commit 6daa6a8 into keycloak:main May 28, 2024
2 checks passed
@ahus1
Copy link
Contributor

ahus1 commented May 28, 2024

Thank you for the explanation and the fix!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Horreum] Infinispan metric calculation sometimes results to division to 0
2 participants