Improve CAGRA multi-CTA heuristic: max_iterations

[sherylll]

For multi-CTA mode, current max_iterations is assigned a large number. However, at low itopk, extra iterations only add latency and does not do much to recall. So the intuition is to use a lower max_iterations for smaller itopk.

minimum_depth = 8 * (search_quality - 1), since we don't provide a knob for search_quality for the moment, hardcoding this to 16. search_quality = 5 would mean the previous default 32.

When the problem is harder, or when more results are needed, it is still a good idea to turn up the max_iterations.

n_queries=1, k=10

n_queries=10, k=50

This is a subset of #2502, where the change to width or hashmap size seem to have different effect on different GPU generations, which still remains to be investigated. But the conclusion on max_iterations should hold.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.