Skip to content

Conversation

@unlimitedsola
Copy link
Contributor

@unlimitedsola unlimitedsola commented Aug 5, 2025

I'm trying to resolve #398 as I strongly agree with this comment.

In the process of coming up with benchmark numbers I noticed a few shortcomings of the benchmark crate, notably:

The rest of changes are some minor polishments and linting fixes.


I'm also planning to add an opinionated benchmark suite runner for making the benchmark numbers more accessible, reproducible, and provable.

I noticed from your comment that you're interested in how people benchmark this crate. Since I'm not an expert in lock implementations, I wanted to use this PR as a chance to ask for your input: what scenarios do you consider important and meaningful for benchmarking, and which ones do you think should be included in the suite to fairly represent this crate?

@Amanieu
Copy link
Owner

Amanieu commented Aug 11, 2025

I noticed from your comment that you're interested in how people benchmark this crate. Since I'm not an expert in lock implementations, I wanted to use this PR as a chance to ask for your input: what scenarios do you consider important and meaningful for benchmarking, and which ones do you think should be included in the suite to fairly represent this crate?

The number one most important benchmark is the performance of uncontended lock acquisition since this is going to be the most common case in practice. This is easy to measure reliably since only one thread is needed.

Benchmarking situations with contention between multiple threads is more difficult because it is very easy to "cheat" by simply sleeping more between each attempt to acquire the lock when contention is detected. Eventually this devolves into having one thread repeatedly acquiring and releasing the lock while all other threads are just sleeping.

@Amanieu Amanieu merged commit f412a68 into Amanieu:master Aug 11, 2025
35 checks passed
@unlimitedsola unlimitedsola deleted the bench branch August 11, 2025 03:17
@unlimitedsola
Copy link
Contributor Author

The number one most important benchmark is the performance of uncontended lock acquisition since this is going to be the most common case in practice. This is easy to measure reliably since only one thread is needed.

Benchmarking situations with contention between multiple threads is more difficult because it is very easy to "cheat" by simply sleeping more between each attempt to acquire the lock when contention is detected. Eventually this devolves into having one thread repeatedly acquiring and releasing the lock while all other threads are just sleeping.

Thank you for the feedback! I'll try to put something together.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants