Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce http client profile option for custom configuration #3165

Open
wants to merge 5 commits into
base: main
Choose a base branch
from

Conversation

wolfeidau
Copy link
Contributor

@wolfeidau wolfeidau commented Jan 22, 2025

Description

We have customers using the agent in a number of different environments, across different cloud services all over the world, so different configuration profiles for the HTTP client would be valueble.

Context

The Go HTTP client has a log of configurable timeouts, passing all these through is not sustainable so we need a way to bless a subset of configuration and provide this with a generic name, hence the idea of configuration profiles.

On top of this different version of Go introduce new features, and new timeouts, so we also need a way to introduce new profiles as the HTTP Client evolves.

Changes

This change introduces a flag which allows us to add HTTP client configurations which allows us to provide new profiles for customers, while also providing an unconfigured standard library profile to take advantage of the current defaults.

To get the ball rolling I have created a stdlib profile which applies the absolulte minimum configuration, this includes tlsconfig overrides, and the timeout value. Disabling H2 is not provided as this is avaliable via the GODEBUG environment variable supported by the Go Team. https://pkg.go.dev/net/http#hdr-HTTP_2

Testing

  • Tests have run locally (with go test ./...). Buildkite employees may check this if the pipeline has run automatically.
  • Code is formatted (with go fmt ./...)

@wolfeidau wolfeidau force-pushed the feat_http_client_profile branch 2 times, most recently from ec60825 to a6ea19e Compare January 28, 2025 01:03
@wolfeidau wolfeidau marked this pull request as ready for review January 28, 2025 01:03
Copy link
Contributor

@DrJosh9000 DrJosh9000 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like the concept!

internal/agenthttp/client.go Outdated Show resolved Hide resolved
internal/agenthttp/client.go Outdated Show resolved Hide resolved
Copy link
Contributor

@DrJosh9000 DrJosh9000 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍
On the HTTP/2 issue, I think it might lack visibility if someone is trying to use a profile and disable it the legacy way. May be worth checking and logging or even fatal-ing?

@wolfeidau wolfeidau force-pushed the feat_http_client_profile branch from 7f050e8 to 283cc36 Compare January 31, 2025 03:08
@wolfeidau wolfeidau force-pushed the feat_http_client_profile branch from 283cc36 to 89aeb92 Compare January 31, 2025 03:09
@wolfeidau
Copy link
Contributor Author

wolfeidau commented Jan 31, 2025

@DrJosh9000 great point, the agent now reports a conflict for that particular setting.

2025-01-31 14:07:04 FATAL NoHTTP2 is not supported with the standard library (stdlib) HTTP client profile, use GODEBUG see https://pkg.go.dev/net/http#hdr-HTTP_2

@wolfeidau wolfeidau requested a review from patrobinson January 31, 2025 03:33
Copy link
Contributor

@patrobinson patrobinson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like the idea, I have a question about if that patch has been released?

switch conf.HTTPClientProfile {
case ClientProfileStdlib:
// Base any modifications on the default transport.
transport := http.DefaultTransport.(*http.Transport).Clone()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As far as I can tell https://go-review.googlesource.com/c/net/+/617655 is merged but not released. So using the default option would re-introduce the bug we fixed in #3005?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants