Handle 404 Messages when processing

[chazzbg]

This one is a bit tricky.
There is a bug in the fetchAll logic when using fetch no wait ( consumer timeout 0 )
If we are waiting for a batch of 50 messages, the fetch logic has two options - either wait for a timeout or wait for the all 50 messages.
BUT when we publish a fetch no wait command ( PUB $JS.API.CONSUMER.MSG.NEXT.test.tester handler.0bf1604f 27 {"batch":50,"no_wait":true} ) to NATS it responds with all available messages and an empty message with 404 status as a termination. If the collected messages and more than 1 but less than the expected batch, the code will wait for the whole timeout or for all of the 50 expected messages.

Basically the fix makes the message handling to break the waiting it the moment it receives a message with 404 status and proceed to return whatever was fetched from the server until the 404 status.

This greatly improves situations where there a lot of clients fetching from NATS and the server itself is struggling to provide the whole expected batches from the clients

I'm struggling a bit to introduce this into the test suite and i would love some help there so we can verify the behavior and validate it with tests, until i figure it out on my own.

Tests is added handling both processing of a 404 message and performance testing ensuring timeout is not waited

[nekufa]

hi @chazzbg
thanks for the contribution!

[ro0NL]

this broke our tests https://github.com/etrias-nl/php-toolkit/actions/runs/21662573253/job/62450396026, they keep hanging

[ro0NL]

This one is a bit tricky.

@nekufa @chazzbg i figured out at some we assert the queue is empty (we fetch 1 by 1)

self::assertSame([], $transport->get());

This one keeps hanging now.

[chazzbg]

@ro0NL personally i experiencesd similar behavior in the perf tests for the 500k iterations when running locally. They just hanged there for over an hour. Never investigated what have happened.

[ro0NL]

To me it's just a bc break now. We have pinned the version. And we'll figure it out later.

[chazzbg]

I'm trying to se what could have cause this and fix it eventualy

[nekufa]

looks like we have incomplete edge cases test package.. :(

[ro0NL]

im not really sure it's an edge case, we rely on

if (null === $this->queue) {
    $this->queue = $this->getConsumer()->getQueue();
    $this->client->setTimeout(self::SOCKET_TIMEOUT);
}

$message = $this->queue->next();
$payload = $message->payload;

and the final next() - when empty - seems to be the hanging one.

[chazzbg]

i think i got a hold of it
#121

[nekufa]

im not really sure it's an edge case

i mean that this is not covered by tests. @ro0NL feel free to contribute any tests that cover your workflow - that would help improve release stability