[BUG] Test SubmitAsynchronousSearchSingleNodeIT.testSubmitAsynchronousSearchWithRetainedResponse reproducible seed

[mch2]

From 2.13 release tests here there is a failure without security plugin.

This seed is reproducible. I think the issue here is a low waitForCompletionTimeout value. The test fetches a random TimeValue between 1 and 5000 ms and with this seed it is set to 6s.

Test testSubmitAsynchronousSearchWithoutRetainedResponse also fails with setting this timeout to a similar value.

I dont have a lot of context here but I think setting a min value of 1k here is reasonable? CC @bharath-techie.

REPRODUCE WITH: ./gradlew ':integTest' --tests "org.opensearch.search.asynchronous.integTests.SubmitAsynchronousSearchSingleNodeIT.testSubmitAsynchronousSearchWithRetainedResponse" -Dtests.seed=DA4B7AB78276EF7A -Dtests.security.manager=false -Dtests.locale=ar-QA -Dtests.timezone=NZ-CHAT -Druntime.java=21

REPRODUCE WITH: ./gradlew ':integTest' --tests "org.opensearch.search.asynchronous.integTests.SubmitAsynchronousSearchSingleNodeIT.testSubmitAsynchronousSearchWithRetainedResponse" -Dtests.seed=DA4B7AB78276EF7A -Dtests.security.manager=false -Dtests.locale=ar-QA -Dtests.timezone=NZ-CHAT -Druntime.java=21

REPRODUCE WITH: ./gradlew ':integTest' --tests "org.opensearch.search.asynchronous.integTests.SubmitAsynchronousSearchSingleNodeIT" -Dtests.seed=DA4B7AB78276EF7A -Dtests.security.manager=false -Dtests.locale=en-US -Dtests.timezone=UTC -Druntime.java=21

REPRODUCE WITH: ./gradlew ':integTest' --tests "org.opensearch.search.asynchronous.integTests.SubmitAsynchronousSearchSingleNodeIT" -Dtests.seed=DA4B7AB78276EF7A -Dtests.security.manager=false -Dtests.locale=en-US -Dtests.timezone=UTC -Druntime.java=21

REPRODUCE WITH: ./gradlew ':integTest' --tests "org.opensearch.search.asynchronous.integTests.SubmitAsynchronousSearchSingleNodeIT" -Dtests.seed=DA4B7AB78276EF7A -Dtests.security.manager=false -Dtests.locale=en-US -Dtests.timezone=UTC -Druntime.java=21

REPRODUCE WITH: ./gradlew ':integTest' --tests "org.opensearch.search.asynchronous.integTests.SubmitAsynchronousSearchSingleNodeIT" -Dtests.seed=DA4B7AB78276EF7A -Dtests.security.manager=false -Dtests.locale=en-US -Dtests.timezone=UTC -Druntime.java=21

[dblock]

Catch All Triage - 1 2 3 4 5 6