Service Bus queue and subscription health check does not respect cancellation token

[andradf]

What happened:
When using the PeekMessagesAsync health check on a queue or subscription, the processor doesn't respect the cancellation token passed to the method. This is because internally the AMPQ client uses timeouts instead of cancellation tokens in their retry policies, by default 1 minute (+5 seconds that are always added for some reason)

What you expected to happen:
I would expect the health check to time out when the cancellation token is triggered.

How to reproduce it (as minimally and precisely as possible):
With connection string pointing to an unreachable destination, register a queue or subscription health check with a timeout. This timeout is not respected.

Source code sample:
builder.Services.AddHeathChecks.AddAzureServiceBusQueue(<connstr>, "queue", timeout: TimeSpan.FromSeconds(5));

Anything else we need to know?:
This is especially important for production environments with health checks enabled. Those usually implement a strategy of RequestTimeouts and OutputCaching to prevent cascading health probes causing denials of service. The aspnet core request timeout middleware is not aggressive. Meaning that it doesn't kill the request or respond even if the request hasn't finished. It only triggers the HttpContext.RequestAborted cancellation token. If a health probe doesn't respect the cancellation token, the request is not answered within the timeout and the output cache doesn't get set. The healthcheck should register a callback on the cancellation token and respond unhealthy on it, even if the call to PeekMessagesAsync hasn't completed.

Environment:

• .NET Core version 9.0
• Healthchecks version 9.0.0
• Operative system: Windows