Trouble with latest adapter #2

heliostatic · 2020-02-14T17:23:26Z

I've tried building caddy with 09ce5f0 but I'm not getting the expected adapter behavior.

Output of caddy build-info:

Details

path: github.com/caddyserver/caddy/v2/cmd/caddy
main: github.com/caddyserver/caddy/v2 (devel) 
dependencies:
github.com/Masterminds/goutils v1.1.0 h1:zukEsf/1JZwCMgHiK3GZftabmxiCw4apj3a28RPBiVg=
github.com/Masterminds/semver/v3 v3.0.1 h1:2kKm5lb7dKVrt5TYUiAavE6oFc1cFT0057UVGT+JqLk=
github.com/Masterminds/sprig/v3 v3.0.0 h1:KSQz7Nb08/3VU9E4ns29dDxcczhOD1q7O1UfM4G3t3g=
github.com/alecthomas/chroma v0.7.0 h1:z+0HgTUmkpRDRz0SRSdMaqOLfJV4F+N1FPDZUZIDUzw=
github.com/andybalholm/brotli v0.0.0-20190821151343-b60f0d972eeb h1:ZSlUsEd11C/uRzhZHOgANARJ03fkwmjJEa6g2Cqjlo4=
github.com/caddyserver/nginx-adapter v0.0.0-20200209014910-09ce5f0856cf h1:n5c+KNx1KrQ+R+FkKFpBHTBIwcYH2ACwozUzJnCeLzg=
github.com/cenkalti/backoff/v3 v3.1.1 h1:UBHElAnr3ODEbpqPzX8g5sBcASjoLFtt3L/xwJ01L6E=
github.com/cheekybits/genny v1.0.0 h1:uGGa4nei+j20rOSeDeP5Of12XVm7TGUd4dJA9RDitfE=
github.com/codahale/hdrhistogram v0.0.0-20161010025455-3a0bb77429bd h1:qMd81Ts1T2OTKmB4acZcyKaMtRnY5Y44NuXGX2GFJ1w=
github.com/danwakefield/fnmatch v0.0.0-20160403171240-cbb64ac3d964 h1:y5HC9v93H5EPKqaS1UYVg1uYah5Xf51mBfIoWehClUQ=
github.com/dlclark/regexp2 v1.1.6 h1:CqB4MjHw0MFCDj+PHHjiESmHX+N7t0tJzKvC6M97BRg=
github.com/dustin/go-humanize v1.0.0 h1:VSnTsYCnlFHaM2/igO1h6X3HA71jcobQuxemgkq4zYo=
github.com/go-acme/lego/v3 v3.3.0 h1:6BePZsOiYA4/w+M7QDytxQtMfCipMPGnWAHs9pWks98=
github.com/golang/groupcache v0.0.0-20191002201903-404acd9df4cc h1:55rEp52jU6bkyslZ1+C/7NGfpQsEc6pxGLAGDOctqbw=
github.com/golang/protobuf v1.3.2 h1:6nsPYzhq5kReh6QImI3k5qWzO4PEbvbIW2cwSfR/6xs=
github.com/google/uuid v1.1.1 h1:Gkbcsh/GbpXz7lPftLA3P6TYMwjCLYm83jiFQZF/3gY=
github.com/hpcloud/tail v1.0.0 h1:nfCOvKYfkgYP8hkirhJocXT2+zOD8yUNjXaWfTlyFKI=
github.com/huandu/xstrings v1.2.0 h1:yPeWdRnmynF7p+lLYz0H2tthW9lqhMJrQV/U7yy4wX0=
github.com/ilibs/json5 v1.0.1 h1:3e14wUQM8PyK6Hf1bM+zAQFxfG+N5oZj35x5vCNeQ58=
github.com/imdario/mergo v0.3.8 h1:CGgOkSJeqMRmt0D9XLWExdT4m4F1vd3FV3VPt+0VxkQ=
github.com/jsternberg/zap-logfmt v1.2.0 h1:1v+PK4/B48cy8cfQbxL4FmmNZrjnIMr2BsnyEmXqv2o=
github.com/klauspost/compress v1.8.6 h1:970MQcQdxX7hfgc/aqmB4a3grW0ivUVV6i1TLkP8CiE=
github.com/klauspost/cpuid v1.2.2 h1:1xAgYebNnsb9LKCdLOvFWtAxGU/33mjJtyOVbmUa0Us=
github.com/lucas-clemente/quic-go v0.14.1 h1:c1aKoBZKOPA+49q96B1wGkibyPP0AxYh45WuAoq+87E=
github.com/mailgun/timetools v0.0.0-20141028012446-7e6055773c51 h1:Kg/NPZLLC3aAFr1YToMs98dbCdhootQ1hZIvZU28hAQ=
github.com/marten-seemann/chacha20 v0.2.0 h1:f40vqzzx+3GdOmzQoItkLX5WLvHgPgyYqFFIO5Gh4hQ=
github.com/marten-seemann/qpack v0.1.0 h1:/0M7lkda/6mus9B8u34Asqm8ZhHAAt9Ho0vniNuVSVg=
github.com/marten-seemann/qtls v0.4.1 h1:YlT8QP3WCCvvok7MGEZkMldXbyqgr8oFg5/n8Gtbkks=
github.com/mholt/certmagic v0.9.3 h1:RmzuNJ5mpFplDbyS41z+gGgE/py24IX6m0nHZ0yNTQU=
github.com/miekg/dns v1.1.25 h1:dFwPR6SfLtrSwgDcIq2bcU/gVutB4sNApq2HBdqcakg=
github.com/mitchellh/copystructure v1.0.0 h1:Laisrj+bAB6b/yJwB5Bt3ITZhGJdqmxquMKeZ+mmkFQ=
github.com/mitchellh/reflectwalk v1.0.0 h1:9D+8oIskB4VJBN5SFlmc27fSlIBZaov1Wpk/IfikLNY=
github.com/muhammadmuzzammil1998/jsonc v0.0.0-20190906142622-1265e9b150c6 h1:EajWCEv0scxMWyMHWxJbFK70brsPIl4TLQJ0zaOeOiI=
github.com/naoina/go-stringutil v0.1.0 h1:rCUeRUHjBjGTSHl0VC00jUPLz8/F9dDzYI70Hzifhks=
github.com/naoina/toml v0.1.1 h1:PT/lllxVVN0gzzSqSlHEmP8MJB4MY2U7STGxiouV4X8=
github.com/onsi/ginkgo v1.8.0 h1:VkHVNpR4iVnU8XQR6DBm8BqYjN7CRzw+xKUbVVbbW9w=
github.com/spf13/cast v1.3.0 h1:oget//CVOEoFewqQxwr0Ej5yjygnqGkvggSE/gB35Q8=
github.com/vulcand/oxy v1.0.0 h1:7vL5/pjDFzHGbtBEhmlHITUi6KLH4xXTDF33/wrdRKw=
github.com/yuin/goldmark v1.1.17 h1:t7bl7JAmainKs/Uhmr1XZVkRJsgL2UwdTCtTuYL7RAY=
github.com/yuin/goldmark-highlighting v0.0.0-20191202084645-78f32c8dd6d5 h1:QbH7ca1qtgZHrzvcVAEoiJIwBqrXxMOfHYfwZIniIK0=
go.uber.org/atomic v1.3.2 h1:2Oa65PReHzfn29GpvgsYwloV9AVFHPDk8tYxt2c2tr4=
go.uber.org/multierr v1.2.0 h1:6I+W7f5VwC5SV9dNrZ3qXrDB9mD0dyGOi/ZJmYw03T4=
go.uber.org/zap v1.10.0 h1:ORx85nbTijNz8ljznvCMR1ZBIPKFn3jQrag10X2AsuM=
golang.org/x/crypto v0.0.0-20191206172530-e9b2fee46413 h1:ULYEB3JvPRE/IfO+9uO7vKV/xzVTO7XPAwm8xbf4w2g=
golang.org/x/net v0.0.0-20191209160850-c0dbc17a3553 h1:efeOvDhwQ29Dj3SdAV/MJf8oukgn+8D8WgaCaRMchF8=
golang.org/x/sys v0.0.0-20191210023423-ac6580df4449 h1:gSbV7h1NRL2G1xTg/owz62CST1oJBmxy4QpMMregXVQ=
golang.org/x/text v0.3.2 h1:tW2bmiBqwgJj/UpqtC8EpXEZVYOwU0yG4iWbprSVAcs=
gopkg.in/fsnotify.v1 v1.4.7 h1:xOHLXZwVvI9hhs+cLKq5+I5onOuwQLhQwiu63xxlHs4=
gopkg.in/natefinch/lumberjack.v2 v2.0.0 h1:1Lc07Kr7qY4U2YPouBjpCLxpiyxIVoxqXgkXLknAOE8=
gopkg.in/square/go-jose.v2 v2.4.1 h1:H0TmLt7/KmzlrDOpa1F+zr0Tk90PbJYBfsVUmRLrf9Y=
gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7 h1:uRGJdciOHaEIrze2W8Q3AKkepLTh2hOroT7a+7czfdQ=
gopkg.in/yaml.v2 v2.2.2 h1:ZCJp+EgiOT7lHqUV2J862kp8Qj64Jo6az82+3Td9dZw=

Then running ./caddy adapt --config ~/Documents/example.conf --adapter nginx --pretty returned:

[WARNING][nginx] nginx.conf:1: map: unrecognized or unsupported nginx directive
[WARNING][nginx] nginx.conf:5: proxy_cache_path: unrecognized or unsupported nginx directive
[WARNING][nginx] nginx.conf:6: server: unrecognized or unsupported nginx directive
[WARNING][nginx] nginx.conf:19: server: unrecognized or unsupported nginx directive
{
        "apps": {
                "http": {}
        }
}

With an nginx.conf of:

Details

map $http_upgrade $connection_upgrade {
    default upgrade;
    '' close;
}
proxy_cache_path /var/cache/nginx levels=1:2 keys_zone=CACHE:10m inactive=7d max_size=1g;
server {
    listen 80;
    listen [::]:80;
    server_name SERVER_NAME_PLACEHOLDER;
    root /home/mastodon/mastodon/public;
# Useful for Let's Encrypt                                                                                                                                                                                                   
    location /.well-known/acme-challenge/ {
        allow all;
    }
    location / {
        return 301 https://$host$request_uri;
    }
}
server {
    listen 443 ssl http2;
    listen [::]:443 ssl http2;
    server_name SERVER_NAME_PLACEHOLDER;
    ssl_protocols TLSv1.2;
    ssl_ciphers EECDH+AESGCM:EECDH+AES;
    ssl_ecdh_curve prime256v1;
    ssl_prefer_server_ciphers on;
    ssl_session_cache shared:SSL:10m;
    ssl_certificate /etc/letsencrypt/live/SERVER_NAME_PLACEHOLDER/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/SERVER_NAME_PLACEHOLDER/privkey.pem;
    keepalive_timeout 70;
    sendfile on;
    client_max_body_size 80M;
    root /home/mastodon/mastdon/public;
    gzip on;
    gzip_disable "msie6";
    gzip_vary on;
    gzip_proxied any;
    gzip_comp_level 6;
    gzip_buffers 16 8k;
    gzip_http_version 1.1;
    gzip_types text/plain text/css application/json application/javascript text/xml application/xml application/xml+rss text/javascript;
    add_header Strict-Transport-Security "max-age=31536000; includeSubDomains";
    location / {
        try_files $uri @proxy;
    }
    location ~ ^/(emoji|packs|system/accounts/avatars|system/media_attachments/files) {
        add_header Cache-Control "public, max-age=31536000, immutable";
        add_header Strict-Transport-Security "max-age=31536000";
        try_files $uri @proxy;
    }
    location /sw.js {
        add_header Cache-Control "public, max-age=0";
        add_header Strict-Transport-Security "max-age=31536000";
        try_files $uri @proxy;
    }
    location @proxy {
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto https;
        proxy_set_header Proxy "";
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection $connection_upgrade;
        proxy_pass_header Server;
        proxy_pass http://localhost:3000;
        proxy_buffering on;
        proxy_redirect off;
        proxy_http_version 1.1;
        proxy_cache CACHE;
        proxy_cache_valid 200 7d;
        proxy_cache_use_stale error timeout updating http_500 http_502 http_503 http_504;
        add_header X-Cached $upstream_cache_status;
        add_header Strict-Transport-Security "max-age=31536000";
        tcp_nodelay on;
    }
    location /api/v1/streaming {
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto https;
        proxy_set_header Proxy "";
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection $connection_upgrade;
        proxy_pass http://localhost:4000;
        proxy_buffering off;
        proxy_redirect off;
        proxy_http_version 1.1;
        tcp_nodelay on;
    }
    error_page 500 501 502 503 504 /500.html;
}

The text was updated successfully, but these errors were encountered:

jschlier · 2020-02-14T17:33:23Z

I had a similar problem with no directives being recognized.
After taking another look at the supported directives in the README.
I saw that server is supported - provided it is within an http directive.
map and proxy_cache_path are not supported for now.

Wrapping everthing in a http { } helped to get at least some of serverand location settings adapted.

As the README states this adapter is still work in progress, so it is to be expected that not all directives are working right now.
Maybe the http { } tricks helps to get something useful out of the adapter though.

mholt · 2020-02-14T21:04:23Z

Thanks for the report, and the tip/workaround!

It's odd, all the docs I can find show the server context within the http context only. This page says:

The “server” context is declared within the “http” context.

And this page shows:

http {
    server {
    }
}

Indeed, this adapter is not yet complete. Contributions welcomed! This is a community project, so feel free to get involved. (Although @mohammed90 does seem to be having fun with it)

mohammed90 · 2020-02-14T21:06:13Z

I believe I've fixed it :) Can you check out the PR #3? It works for me now with the patch.

mohammed90 · 2020-02-14T21:32:50Z

That was hasty of me 😅 I've just tried with nginx to have the server directive without an enclosing http directive, and nginx failed to start with the error:
"server" directive is not allowed here

Is your config part of an include elsewhere?

heliostatic · 2020-02-17T00:08:41Z

It isn't, but perhaps I'm using an old version of nginx. In any case, I wrapped my server in an http and ran it through the latest adapter with greater success. The current list of unsupported directives is mostly not applicable (ssl) or WIP (gzip):

[WARNING][nginx] nginx.conf:1: proxy_cache_path: unrecognized or unsupported nginx directive
[WARNING][nginx] nginx.conf:22: ssl_protocols: unrecognized or unsupported nginx directive
[WARNING][nginx] nginx.conf:23: ssl_ciphers: unrecognized or unsupported nginx directive
[WARNING][nginx] nginx.conf:24: ssl_ecdh_curve: unrecognized or unsupported nginx directive
[WARNING][nginx] nginx.conf:25: ssl_prefer_server_ciphers: unrecognized or unsupported nginx directive
[WARNING][nginx] nginx.conf:26: ssl_session_cache: unrecognized or unsupported nginx directive
[WARNING][nginx] nginx.conf:27: ssl_certificate: unrecognized or unsupported nginx directive
[WARNING][nginx] nginx.conf:28: ssl_certificate_key: unrecognized or unsupported nginx directive
[WARNING][nginx] nginx.conf:29: keepalive_timeout: unrecognized or unsupported nginx directive
[WARNING][nginx] nginx.conf:30: sendfile: unrecognized or unsupported nginx directive
[WARNING][nginx] nginx.conf:31: client_max_body_size: unrecognized or unsupported nginx directive
[WARNING][nginx] nginx.conf:33: gzip: unrecognized or unsupported nginx directive
[WARNING][nginx] nginx.conf:34: gzip_disable: unrecognized or unsupported nginx directive
[WARNING][nginx] nginx.conf:35: gzip_vary: unrecognized or unsupported nginx directive
[WARNING][nginx] nginx.conf:36: gzip_proxied: unrecognized or unsupported nginx directive
[WARNING][nginx] nginx.conf:37: gzip_comp_level: unrecognized or unsupported nginx directive
[WARNING][nginx] nginx.conf:38: gzip_buffers: unrecognized or unsupported nginx directive
[WARNING][nginx] nginx.conf:39: gzip_http_version: unrecognized or unsupported nginx directive
[WARNING][nginx] nginx.conf:40: gzip_types: unrecognized or unsupported nginx directive
[WARNING][nginx] nginx.conf:41: add_header: unrecognized or unsupported nginx directive
[WARNING][nginx] nginx.conf:55: location: named locations marked by @ are unnsupported
[WARNING][nginx] nginx.conf:89: error_page: unrecognized or unsupported nginx directive

mholt · 2020-02-17T02:17:08Z

Great, sounds like there is no longer a problem then. 👍

Nice work, @mohammed90 !

Except for a few, the remaining directives will form a long tail that we can address in time. The SSL-related directives will need some special handling, after we decide whether to upgrade nginx's default logic (non-privacy-preserving, plaintext HTTP settings) to HTTPS, or to keep it the same, or make it configurable...

mohammed90 mentioned this issue Feb 14, 2020

support "server" directive in main context #3

Closed

mholt closed this as completed Feb 17, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Trouble with latest adapter #2

Trouble with latest adapter #2

heliostatic commented Feb 14, 2020

jschlier commented Feb 14, 2020

mholt commented Feb 14, 2020 •

edited

Loading

mohammed90 commented Feb 14, 2020

mohammed90 commented Feb 14, 2020 •

edited

Loading

heliostatic commented Feb 17, 2020

mholt commented Feb 17, 2020

Trouble with latest adapter #2

Trouble with latest adapter #2

Comments

heliostatic commented Feb 14, 2020

jschlier commented Feb 14, 2020

mholt commented Feb 14, 2020 • edited Loading

mohammed90 commented Feb 14, 2020

mohammed90 commented Feb 14, 2020 • edited Loading

heliostatic commented Feb 17, 2020

mholt commented Feb 17, 2020

mholt commented Feb 14, 2020 •

edited

Loading

mohammed90 commented Feb 14, 2020 •

edited

Loading