par_sort_pairs: Use a growable pool of thread states

[progval]

It turns out that rayon can return Yield::Idle when there is an other task running the the same thread (lower in the call stack), because it cannot yield to that other task (it would need to jump back to that point of the call stack, then back to where we currently are). This means that we need a way to add new states somehow.

Instead of creating throwaway states (which would create many tiny BatchIterators we would have to merge with the others) or making each thread have a pool of throwaway states (which would create fewer of them, but still many because the extra ones can't be used by other states), this introduces a global pool with all states in it.

This does add synchronization overhead, but it should be negligeable because try_for_each_init calls this initializer only once per internal sequential iterators; and if there are many small such iterators, then rayon already adds too much overhead anyway.

[vigna]

I know this might seem like a stupid question, but could the problem be in the parallel iterator and not in the sorter?

[progval]

It doesn't matter why Rayon calls worker functions recursively, it's well-known that it will do that in many cases. It's the same reason we can't hold mutexes across calls to Rayon functions.