Provide performance benchmarks

[ngirard]

• What to benchmark: indexing time / querying time
• Comparison with mlocale / updatedb would be nice
• Should be performed against a standardized directory structure. I quick search did not help with finding some reusable prior art.
• Should be integrated with Github Actions.

[dufferzafar]

Hi, you do seem to be making some perf related commits. Like LZ4 etc. How are you measuring perf right now? Hyperfine on some directory?

[ngirard]

Hey @dufferzafar,

yes, I'm personally using hyperfine on a test configuration which indexes a few directories belonging to the same filesystem in my machine.

And there's an ongoing discussion on Reddit with a redditter willing to run his own benchmarks.

Like I said on Reddit, I changed the code for database updating so that it uses multiple threads. I noticed that there's an important overhead with my multithreaded implementation, so I'm suspecting there might be room for improvements. It would be nice if someone could take a look at it (9ec023f)

[dufferzafar]

Sorry, I wish I could help you, but I can't rust (or C++ for that matter.) I just let grown ups do the real work while I sit here and play with Python / bash.

But I would guess we could look at fd for this? It does multi-threaded file path search as well right?