Document backoff semantics for create_profile

[bitglue]

Is your feature request related to a problem? Please describe.

I'm trying to implement an agent for Cloud Profiler, but I'm not sure how it should behave. According to the documentation, the server can request the client to back off through some GRPC metadata:

google-api-ruby-client/generated/google/apis/cloudprofiler_v2/service.rb

Lines 50 to 58 in 889330f

	# CreateProfile creates a new profile resource in the online mode. The server
	# ensures that the new profiles are created at a constant rate per deployment,
	# so the creation request may hang for some time until the next profile session
	# is available. The request may fail with ABORTED error if the creation is not
	# available within ~1m, the response will indicate the duration of the backoff
	# the client should take before attempting creating a profile again. The backoff
	# duration is returned in google.rpc.RetryInfo extension on the response status.
	# To a gRPC client, the extension will be return as a binary-serialized proto in
	# the trailing metadata item named "google.rpc.retryinfo-bin".

However, I can't find anything that specifies how this maps to the JSON API.

Sometimes, I've received an HTTP 409 response with an error like "generic::aborted: action throttled, backoff for 44m0s". Maybe this is what the documentation is talking about? I'm not sure. Parsing the duration out of a string doesn't seem ideal. It's difficult to explore, because it's difficult to solicit this response.

Oddly, the documentation says this should happen in about 1 minute, but when I've had it happen, it's taken more like 230 seconds. This is longer than the default 60 second timeout on this method.

Furthermore, I don't always get the 409 response even with a longer timeout. I've tried increasing the timeout to as much as 1 hour, and usually I'll get <Google::Apis::TransmissionError: execution expired> at the end of that hour, instead of the HTTP 409.

I definitely want to respect the prescribed client behavior here, because the alternative is getting rate limited and that's no good.

Not sure if there's already logic in this library that's handling the retries or what.

Describe the solution you'd like

Document what the backoff response looks like over the JSON HTTP and the interface exposed by google-api-ruby-client.

Describe alternatives you've considered

I'd use an agent implemented by Google, but that doesn't seem to be going anywhere: googleapis/google-cloud-ruby#2116

I'd also use the GRPC API directly if it was easy, but it seems like https://github.com/googleapis/google-cloud-ruby would be the place that would happen, and there doesn't seem to be any support even for the raw API in there.

Additional context

[quartzmo]

I'm not sure if this helps, but here is where retry with backoff is implemented for the generated gRPC clients in google-cloud-ruby:

https://github.com/googleapis/gapic-generator-ruby/blob/master/gapic-common/lib/gapic/call_options/retry_policy.rb

[fhinkel]

Greetings, we're closing this. Looks like the issue got resolved. Please let us know if the issue needs to be reopened.