[DI] Reduce time it takes to run benchmarks

[watson]

What does this PR do?

Reduce the number of iterations the Dynamic Instrumentation benchmarks run from 500 to 250.

Motivation

The line-probe-with-snapshot-default benchmark took 6 minutes to run, which could make it flaky (because of timeout). Reducing it to 3 minutes should get us well below the threshold.

Plugin Checklist

• Unit tests.
• TypeScript definitions.
• TypeScript tests.
• API documentation.
• CircleCI jobs/workflows.
• Plugin is exported.

Additional Notes

[watson]

• [DI] Reduce time it takes to run benchmarks #5100 👈 (View in Graphite)
• master

This stack of pull requests is managed by Graphite. Learn more about stacking.

[github-actions]

Overall package size

Self size: 8.39 MB
Deduped: 94.74 MB
No deduping: 95.25 MB

Dependency sizes

| name | version | self size | total size | |------|---------|-----------|------------| | @datadog/libdatadog | 0.3.0 | 29.43 MB | 29.43 MB | | @datadog/native-appsec | 8.4.0 | 19.25 MB | 19.26 MB | | @datadog/native-iast-taint-tracking | 3.2.0 | 13.9 MB | 13.91 MB | | @datadog/pprof | 5.4.1 | 9.76 MB | 10.13 MB | | protobufjs | 7.2.5 | 2.77 MB | 5.16 MB | | @datadog/native-iast-rewriter | 2.6.1 | 2.59 MB | 2.73 MB | | @opentelemetry/core | 1.14.0 | 872.87 kB | 1.47 MB | | @datadog/native-metrics | 3.1.0 | 1.06 MB | 1.46 MB | | @opentelemetry/api | 1.8.0 | 1.21 MB | 1.21 MB | | import-in-the-middle | 1.11.2 | 112.74 kB | 826.22 kB | | source-map | 0.7.4 | 226 kB | 226 kB | | opentracing | 0.14.7 | 194.81 kB | 194.81 kB | | lru-cache | 7.18.3 | 133.92 kB | 133.92 kB | | pprof-format | 2.1.0 | 111.69 kB | 111.69 kB | | @datadog/sketches-js | 2.1.0 | 109.9 kB | 109.9 kB | | semver | 7.6.3 | 95.82 kB | 95.82 kB | | lodash.sortby | 4.7.0 | 75.76 kB | 75.76 kB | | ignore | 5.3.1 | 51.46 kB | 51.46 kB | | shell-quote | 1.8.1 | 44.96 kB | 44.96 kB | | istanbul-lib-coverage | 3.2.0 | 29.34 kB | 29.34 kB | | rfdc | 1.3.1 | 25.21 kB | 25.21 kB | | @isaacs/ttlcache | 1.4.1 | 25.2 kB | 25.2 kB | | tlhunter-sorted-set | 0.1.0 | 24.94 kB | 24.94 kB | | limiter | 1.1.5 | 23.17 kB | 23.17 kB | | dc-polyfill | 0.1.4 | 23.1 kB | 23.1 kB | | retry | 0.13.1 | 18.85 kB | 18.85 kB | | jest-docblock | 29.7.0 | 8.99 kB | 12.76 kB | | crypto-randomuuid | 1.0.0 | 11.18 kB | 11.18 kB | | path-to-regexp | 0.1.12 | 6.6 kB | 6.6 kB | | koalas | 1.0.2 | 6.47 kB | 6.47 kB | | module-details-from-path | 1.0.3 | 4.47 kB | 4.47 kB |

_{🤖 This report was automatically generated by heaviest-objects-in-the-universe}

[pr-commenter]

Benchmarks

Benchmark execution time: 2025-01-13 12:21:56

Comparing candidate commit 9d44fa2 in PR branch watson/DEBUG-2740/reduce-bench-time with baseline commit 684ead6 in branch master.

Found 49 performance improvements and 0 performance regressions! Performance is the same for 777 metrics, 17 unstable metrics.

scenario:debugger-control-18

• 🟩 cpu_user_time [-38.376ms; -31.714ms] or [-36.758%; -30.377%]
• 🟩 execution_time [-341.258ms; -328.404ms] or [-34.988%; -33.670%]
• 🟩 instructions [-194.0M instructions; -188.8M instructions] or [-35.909%; -34.949%]

scenario:debugger-control-20

• 🟩 cpu_user_time [-33.779ms; -29.125ms] or [-38.672%; -33.344%]
• 🟩 execution_time [-343.049ms; -323.113ms] or [-35.832%; -33.749%]
• 🟩 instructions [-189.0M instructions; -182.6M instructions] or [-39.397%; -38.062%]

scenario:debugger-control-22

• 🟩 cpu_user_time [-33.702ms; -28.366ms] or [-40.773%; -34.319%]
• 🟩 execution_time [-345.476ms; -330.651ms] or [-36.214%; -34.660%]
• 🟩 instructions [-179.6M instructions; -174.6M instructions] or [-38.873%; -37.786%]

scenario:debugger-enabled-but-breakpoint-not-hit-18

• 🟩 cpu_user_time [-541.968ms; -510.131ms] or [-43.547%; -40.989%]
• 🟩 execution_time [-823.737ms; -743.186ms] or [-43.340%; -39.102%]
• 🟩 instructions [-907.3M instructions; -901.2M instructions] or [-41.225%; -40.947%]
• 🟩 max_rss_usage [-18.288MB; -17.154MB] or [-18.161%; -17.035%]

scenario:debugger-enabled-but-breakpoint-not-hit-20

• 🟩 cpu_user_time [-457.609ms; -443.443ms] or [-45.360%; -43.956%]
• 🟩 execution_time [-786.499ms; -722.714ms] or [-43.061%; -39.569%]
• 🟩 instructions [-895.1M instructions; -889.6M instructions] or [-38.089%; -37.857%]
• 🟩 max_rss_usage [-7.615MB; -6.089MB] or [-9.240%; -7.389%]

scenario:debugger-enabled-but-breakpoint-not-hit-22

• 🟩 cpu_user_time [-413.041ms; -389.370ms] or [-44.963%; -42.386%]
• 🟩 execution_time [-756.703ms; -690.752ms] or [-42.973%; -39.227%]
• 🟩 instructions [-887.6M instructions; -880.3M instructions] or [-38.586%; -38.267%]

scenario:debugger-line-probe-with-snapshot-default-18

• 🟩 cpu_user_time [-18.433s; -17.858s] or [-50.026%; -48.466%]
• 🟩 execution_time [-19.105s; -18.408s] or [-49.850%; -48.031%]
• 🟩 instructions [-76.5G instructions; -76.4G instructions] or [-49.924%; -49.856%]

scenario:debugger-line-probe-with-snapshot-default-20

• 🟩 cpu_user_time [-17.211s; -16.618s] or [-50.117%; -48.389%]
• 🟩 execution_time [-17.984s; -17.269s] or [-49.878%; -47.895%]
• 🟩 instructions [-73.4G instructions; -73.2G instructions] or [-49.775%; -49.630%]

scenario:debugger-line-probe-with-snapshot-default-22

• 🟩 cpu_user_time [-16.389s; -15.921s] or [-50.351%; -48.913%]
• 🟩 execution_time [-17.159s; -16.598s] or [-50.017%; -48.383%]
• 🟩 instructions [-70.6G instructions; -70.5G instructions] or [-49.772%; -49.726%]

scenario:debugger-line-probe-with-snapshot-minimal-18

• 🟩 cpu_user_time [-224.013ms; -192.471ms] or [-32.706%; -28.101%]
• 🟩 execution_time [-566.874ms; -486.130ms] or [-39.504%; -33.877%]
• 🟩 instructions [-555.0M instructions; -533.1M instructions] or [-36.947%; -35.491%]

scenario:debugger-line-probe-with-snapshot-minimal-20

• 🟩 cpu_user_time [-202.612ms; -178.683ms] or [-34.843%; -30.728%]
• 🟩 execution_time [-516.433ms; -448.369ms] or [-37.847%; -32.859%]
• 🟩 instructions [-532.9M instructions; -511.5M instructions] or [-32.419%; -31.116%]

scenario:debugger-line-probe-with-snapshot-minimal-22

• 🟩 cpu_user_time [-194.271ms; -172.555ms] or [-36.006%; -31.982%]
• 🟩 execution_time [-522.570ms; -443.345ms] or [-39.327%; -33.365%]
• 🟩 instructions [-522.4M instructions; -504.0M instructions] or [-32.741%; -31.584%]

scenario:debugger-line-probe-without-snapshot-18

• 🟩 cpu_user_time [-544.776ms; -517.436ms] or [-45.167%; -42.900%]
• 🟩 execution_time [-824.415ms; -746.339ms] or [-44.194%; -40.009%]
• 🟩 instructions [-830.8M instructions; -826.0M instructions] or [-40.508%; -40.270%]
• 🟩 max_rss_usage [-19.548MB; -18.083MB] or [-19.480%; -18.021%]

scenario:debugger-line-probe-without-snapshot-20

• 🟩 cpu_user_time [-439.948ms; -422.603ms] or [-45.969%; -44.157%]
• 🟩 execution_time [-766.579ms; -695.292ms] or [-43.297%; -39.271%]
• 🟩 instructions [-814.9M instructions; -810.3M instructions] or [-37.250%; -37.041%]
• 🟩 max_rss_usage [-7.739MB; -5.610MB] or [-9.449%; -6.850%]

scenario:debugger-line-probe-without-snapshot-22

• 🟩 cpu_user_time [-389.849ms; -364.577ms] or [-44.860%; -41.952%]
• 🟩 execution_time [-726.373ms; -659.037ms] or [-42.710%; -38.750%]
• 🟩 instructions [-813.7M instructions; -808.8M instructions] or [-37.772%; -37.543%]