Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

new benches using pixelbomber #16

Merged
merged 3 commits into from
Aug 6, 2023
Merged

Conversation

fabi321
Copy link
Contributor

@fabi321 fabi321 commented Jun 21, 2023

breakwater::test::helpers::pixelflut_commands is never used. should it be removed?

For the record:

Processed image, pixel commands bytes: 38142000, 18.394096 bytes per pixel, using no optimizations
parse_draw_commands_ordered/1920 x 1080
                        time:   [9.8654 ms 9.9548 ms 10.085 ms]
                        change: [-0.2438% +0.4584% +1.4045%] (p = 0.34 > 0.05)
                        No change in performance detected.
Found 5 outliers among 100 measurements (5.00%)
  3 (3.00%) high mild
  2 (2.00%) high severe

Processed image, pixel commands bytes: 38142000, 18.394096 bytes per pixel, using no optimizations
parse_draw_commands_shuffled/1920 x 1080
                        time:   [18.889 ms 18.936 ms 18.986 ms]
                        change: [+0.3597% +0.6845% +1.0536%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high mild

Processed image, pixel commands bytes: 28271640, 13.634086 bytes per pixel, using both gray and offset optimizations
parse_mixed_draw_commands/1920 x 1080
                        time:   [8.5315 ms 8.6067 ms 8.7171 ms]
Found 11 outliers among 100 measurements (11.00%)
  3 (3.00%) high mild
  8 (8.00%) high severe

Processed image, pixel commands bytes: 28271640, 13.634086 bytes per pixel, using both gray and offset optimizations
parse_mixed_draw_commands_shuffled/1920 x 1080
                        time:   [8.6021 ms 8.6253 ms 8.6513 ms]
Found 5 outliers among 100 measurements (5.00%)
  1 (1.00%) high mild
  4 (4.00%) high severe

So using offset actually helps performance significantly ;)

Fixes #4

@fabi321
Copy link
Contributor Author

fabi321 commented Jun 22, 2023

the new bench results for me:

Processed image, pixel commands bytes: 38142000, 18.394096 bytes per pixel, using no optimizations
Benchmarking parse_draw_commands_ordered/1920 x 1080: Warming up for 10.000 s
Warning: Unable to complete 100 samples in 30.0s. You may wish to increase target time to 49.6s, enable flat sampling, or reduce sample count to 50.
parse_draw_commands_ordered/1920 x 1080
                        time:   [9.7931 ms 9.8223 ms 9.8558 ms]
                        change: [-2.0787% -1.0964% -0.3418%] (p = 0.01 < 0.05)
                        Change within noise threshold.
Found 5 outliers among 100 measurements (5.00%)
  3 (3.00%) high mild
  2 (2.00%) high severe

Processed image, pixel commands bytes: 38142000, 18.394096 bytes per pixel, using no optimizations
parse_draw_commands_shuffled/1920 x 1080
                        time:   [18.789 ms 18.838 ms 18.890 ms]
                        change: [-0.8874% -0.5180% -0.1368%] (p = 0.01 < 0.05)
                        Change within noise threshold.
Found 3 outliers among 100 measurements (3.00%)
  2 (2.00%) high mild
  1 (1.00%) high severe

Processed image, pixel commands bytes: 37105200, 17.894096 bytes per pixel, using only gray optimization
Benchmarking parse_mixed_draw_commands/1920 x 1080: Warming up for 10.000 s
Warning: Unable to complete 100 samples in 30.0s. You may wish to increase target time to 50.1s, enable flat sampling, or reduce sample count to 50.
parse_mixed_draw_commands/1920 x 1080
                        time:   [9.8759 ms 9.9003 ms 9.9274 ms]
                        change: [+14.179% +15.254% +16.186%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high mild

Processed image, pixel commands bytes: 29308440, 14.134086 bytes per pixel, using only offset optimization
Benchmarking parse_draw_commands_with_offset/1920 x 1080: Warming up for 10.000 s
Warning: Unable to complete 100 samples in 30.0s. You may wish to increase target time to 40.9s, enable flat sampling, or reduce sample count to 60.
parse_draw_commands_with_offset/1920 x 1080
                        time:   [8.2587 ms 8.2849 ms 8.3130 ms]
Found 5 outliers among 100 measurements (5.00%)
  3 (3.00%) high mild
  2 (2.00%) high severe

@fabi321
Copy link
Contributor Author

fabi321 commented Jun 22, 2023

A note on the mixed bench: it is 50% rrggbb, 25% rrggbbaa and 25% gg rn. Let me know if you want to go with this, or rather have dedicated benches for each type.

@sbernauer
Copy link
Owner

Many thanks for the nice benchmarks, sorry for taking so long!
Mixed benchmark and breakwater::test::helpers::pixelflut_commands would be fine for me as of now, but happy to let you decide on that and create a follow-up PR if desired :)

@sbernauer sbernauer merged commit 4cc8e2a into sbernauer:master Aug 6, 2023
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

More realistic bench
2 participants