Reproducible benchmark code?

[craigbarnes]

Is the code used to produce the numbers in the readme available somewhere? Based on the methodology described, it seems likely that triehash is benefiting significantly from (unrealistically favourable) branch prediction:

... each hash function was run 1,000,000 times for each word

[julian-klode]

See https://github.com/julian-klode/hashbench

[julian-klode]

Needs access to the Debian porter boxes, and probably updates for changes there, to run the entire thing; but can also run locally (make run).

[julian-klode]

Not sure how you'd test that differently, though, you do need to run the function a lot to get usable results.

[craigbarnes]

Thanks for the pointers. I just wanted to see how unpredictable inputs would change the results.

To make it fair and avoid adding too much overhead to the loop, I was thinking the best way might be to pre-compute an array of randomly selected (known) words and use words[iteration % ARRAY_SIZE(words)] as the input. Otherwise the branch predictor is getting perfect conditions for most of the 1,000,000 iterations, when in practice most of the branches are probably highly unpredictable.

[julian-klode]

Won't that be affected too much by data caches?

[craigbarnes]

It'll increase cache pressure I guess, but not by much for a small array. It'd certainly be much closer to a real world scenario than repeating the same input a million times.