Add support for cancellation tokens

[reecerussell]

It would be really helpful if this SDK supported cancellation tokens on the public APIs, such as Models.GenerateContentAsync. This would allow clients to continue to follow best practice by supporting cancellations in their projects, which you can and should do with I/O bound work.

This would be a relatively easy feature to add, as the underlying HttpApiClient already supports and just Models would need to be updated. It could also be done in a non-breaking way buy introducing overloads or default parameters to current method signatures.

I would happily open a pull request to tackle this, but the contributing guide said that contributions are not being accepted currently.

Thanks team!

[shivvaam0001]

Hi @reecerussell
Thank you for your feedback. We appreciate you taking the time to share your thoughts with us, and we'll be filing a feature request.
To help us prioritize this request effectively, any additional details you can provide regarding the impact this feature would have would be very helpful.

[ProRedCat]

I’d also really like to see support for CancellationToken in the generated client.

I pulled the repo down and tested it locally by adding CancellationToken parameters to the relevant GenerateContentAsync and GenerateContentStreamAsync overloads, and wiring the token through to:

RequestAsync(...)
RequestStreamAsync(...)
ReadAsStringAsync(cancellationToken)

I also added [EnumeratorCancellation] for the streaming variants. Everything worked as expected.

Since Models.cs is auto-generated (according to a comment), any changes would get overwritten the next time the client is regenerated. So this really needs to be added at the generator level rather than patched manually from what I can tell.

One implementation detail worth noting: Google AI Studio sends usage/billing metadata on every streamed chunk, but Vertex AI only includes the usage data in the final chunk of the stream. With cancellation, cancelling a Vertex stream early means that final chunk is never delivered, so any billing information is lost. Just flagging this in case it affects how cancellation support is implemented.

Happy to provide a patch if it would help, but I think this ultimately needs to be added to the generator.

[yyyu-google]

Thank you for the feature request, we have noticed that this is very important and we are prioritizing this item.

[phillip-haydon]

Thank you for the feature request, we have noticed that this is very important and we are prioritizing this item.

I also need this, ASAP. Thanks.

[christallire]

I also need this, ASAP. Thanks.

[Jaxter]

I also need this, ASAP. Thanks.