HTTP proxy support

[cbartz]

Describe the bug

The charm does not respect the juju http proxy variables when making requests to lets encrypt.

To Reproduce

We assume an environment where http requests have to go through a proxy and are otherwise blocked by the firewall.

1. Set juju model-config juju-http-proxy="http-proxy-address"
2. Integrate charm application via tls-certificate integration (e.g. with an nginx-ingress-integrator application)
3. Observe in the juju debug-logs that the request to https://acme-v02.api.letsencrypt.org/directory times out.

Expected behavior

The lego request uses the http proxy, so either the request succeeds or the proxy returns a 403 forbidden.

It seems that LEGO supports http proxy via env variables (go-acme/lego#840).

Logs

unit-httprequest-lego-k8s-0: 09:51:43 INFO unit.httprequest-lego-k8s/0.juju-log Received Certificate Creation Request for domain example.com
unit-httprequest-lego-k8s-0: 09:52:13 ERROR unit.httprequest-lego-k8s/0.juju-log Exited with code 1. Stderr:
unit-httprequest-lego-k8s-0: 09:52:13 ERROR unit.httprequest-lego-k8s/0.juju-log     2024/04/16 09:52:13 Could not create client: get directory at 'https://acme-v02.api.letsencrypt.org/directory': Get "https://acme-v02.api.letsencrypt.org/directory": dial tcp 172.65.32.248:443: i/o timeout
unit-httprequest-lego-k8s-0: 09:52:13 ERROR unit.httprequest-lego-k8s/0.juju-log Failed to execute lego command                 will try again in during the next update status event.

Environment

• Charm / library version (if relevant):
• Juju version (output from juju --version):

Model                             Controller         Cloud/Region             Version  SLA          Timestamp
github-runner-webhook-router      k8s                k8s/default              3.1.8    unsupported  09:56:50Z

App                           Version  Status  Scale  Charm                         Channel  Rev  Address       Exposed  Message
github-runner-webhook-router           active      1  github-runner-webhook-router  edge       3  10.0.5.1  no       
httprequest-lego-k8s                   active      1  httprequest-lego-k8s          edge      70  10.0.5.2  no       
nginx-ingress-integrator      24.2.0   active      1  nginx-ingress-integrator      edge      84  10.0.5.3  no       Ingress IP(s): 10.100.0.135

• Kubernetes version (output from kubectl version --short):

Client Version: v1.29.3
Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3
Server Version: v1.26.15

[gruyaume]

Hello @cbartz, thank you for opening this issue. You are correct that httprequest does not read or do anything with this juju level config. We will have to investigate the value and effort related to implementing this feature. Is this something that is needed for an existing use case?

[cbartz]

Hi @gruyaume, this is required for deployment in an infrastructure with limited access to the outside world (hence proxy access required) See also this discourse post.

[mcarvalhor]

It would be better to have a charm config for proxy instead of using juju model-config

[shipperizer]

allegedly go supports the HTTP_PROXY variables from the environment when using the http.Client, worth giving it a shot when setting the pebble layer

effort should be small in this case @gruyaume

[mthaddon]

It would be better to have a charm config for proxy instead of using juju model-config

I'm not sure I agree that this should be separate charm configuration. In the IS DevOps team we've certainly been working on the understanding that if a charm needs to support a proxy the right way for us to do that is to use model-config proxy variables. This way it's consistent across every charm rather than being done differently in different places.

[gruyaume]

@mthaddon do you have example of charms who handle this in the way you recommend?

[mcarvalhor]

@mthaddon Makes sense.

I was thinking of the scenario: 1 application needs proxy, other should not use proxy (on the same model).
But I think that's why we have no-proxy anyway: we just have to document it properly.

[cbartz]

@mthaddon do you have example of charms who handle this in the way you recommend?

@gruyaume

For example:

• github runner charm: https://github.com/canonical/github-runner-operator/blob/82067e501bbb9e7e23330c433098b4083beaa433/src/charm_state.py#L674-L697
• discourse: https://github.com/canonical/discourse-k8s-operator/blob/3c2260e763dd796750875a41b91fc97e9e05bad9/src/charm.py#L462-L470
• loki push api lib: https://github.com/canonical/discourse-k8s-operator/blob/3c2260e763dd796750875a41b91fc97e9e05bad9/lib/charms/loki_k8s/v0/loki_push_api.py#L2135-L2139
• synapse: https://github.com/canonical/synapse-operator/blob/dee4fabc0a4994acda3f7f29d1847b5dad5a4c9e/src/charm_state.py#L296-L309

And probably more...

[saltiyazan]

Fixed by canonical/lego-base-k8s-operator#143 and #178