Compatibility with traefik-foward-auth?

[kvangent]

I'm trying to configure this plugin with https://github.com/thomseddon/traefik-forward-auth.

I'm running traefik-forward-auth as a Service with a middleware as such:

apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
  name: traefik-forward-auth
spec:
  forwardAuth:
    address: http://traefik-forward-auth.default.svc.cluster.local:4181
    authResponseHeaders:
      - X-Forwarded-User

Then I tried to apply a middleware for fail2ban:

apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
  name: fail2ban
  namespace: traefik
spec:
  plugin:
    fail2ban:
      logLevel: DEBUG
      rules:
        bantime: "5m"
        findtime: "10m"
        maxretry: 3
        enabled: true
        statuscode: "400-499"

Everything starts up fine, but testing with 10x requests that return 401 all successfully connect without an IP ban. I tried reversing the order of middleware (putting fail2ban first and traefik-forward-auth) and started to get the following error (from the first request):

2024-06-07T03:02:15Z DBG github.com/traefik/traefik/v3/pkg/middlewares/auth/forward.go:180 > Remote error http://traefik-forward-auth.default.svc.cluster.local:4181. StatusCode: 307 middlewareName=default-traefik-forward-auth@kubernetescrd middlewareType=ForwardAuth
2024-06-07T03:02:15Z DBG log/log.go:245 > http: superfluous response.WriteHeader call from github.com/traefik/traefik/v3/pkg/middlewares/capture.(*captureResponseWriter).WriteHeader (capture.go:169)

I'm unsure of what's exactly is causing this, and would appreciate any tips.

[tomMoulard]

Hello @kvangent,

Thanks for your interest in this Traefik plugin !

Indeed, there is a WriteHeader call here:

fail2ban/pkg/response/status/status.go

Line 68 in c27a62f

w.WriteHeader(catcher.getCode())

I have to investigate further on this behavior.

In the meantime, can you have a f2b middleware that protects your route downstream, and another f2b middleware after the forward auth ?
It could look like this:

sequenceDiagram
    participant request
    participant f2b1
    participant forward auth
    participant f2b2
    participant auth server
    participant server
    alt request without auth
        request->>+f2b1: 
        f2b1->>+server: 
    end
    alt request with auth
        request->>+forward auth: 
        forward auth->>+f2b2: 
        f2b2->>+auth server: 
        auth server->>+server: 
    end

Loading

[kvangent]

Hey Tom,

Appreciate the response. I'm not entirely sure what you are asking for. Currently I apply both middleware layers to all of the ingress routes in my cluster. The first time, I had the middlewares in the following order:

    middlewares:
      - default-traefik-forward-auth@kubernetescrd
      # - traefik-fail2ban@kubernetescrd

My theory is it was failing to do anything because the fail2ban middleware was never reached -- the forwardauth middleware sends a request to the AuthService, and if it fails then it returns that response:

flowchart TD
    request -->|service.example.com| traefik
    traefik --> fa(forward-auth)
    fa --> AuthService
    AuthService --> fa
    fa -->|4xx| traefik
    traefik -->|4xx| request
    fa -->|2xx| fail2ban
    fail2ban --> service.example.com

Loading

I was hoping that switching the order would allow fail2ban to record errors properly:

flowchart TD
    request -->|service.example.com| traefik
    traefik --> fail2ban
    fail2ban --> fa
    fa(forward-auth) --> AuthService
    AuthService --> fa
    fa -->|4xx| fail2ban
    fail2ban -->|4xx| traefik
    fa --> service.example.com

Loading

but I think the forward-auth middleware needs to be involved on all the requests for it to work effectively.

also -- I'm not entirely sure that the line you linked it the one throwing the error. The error seems like it's coming from a capture.go in middleware? Not entirely sure. It might be causing the one in capture.go to error though :)

[tomMoulard]

Oh, I was explaining something like :

flowchart TD
    request -->|service.example.com| traefik
    traefik --> fa(forward-auth)
    fa --> fail2ban2
    fail2ban2 --> AuthService
    AuthService --> fail2ban2
    fail2ban2 --> fa
    fa -->|4xx| traefik
    traefik -->|4xx| request
    fa -->|2xx| fail2ban1
    fail2ban1 --> service.example.com

Loading

But this would require to use the XFF header, as well as synchronize two f2b instance to work properly.
But still, this may be a workaround, there might be an underlying bug that needs to be though for and addressed.

[kvangent]

Will need to think about this some more. I played around with your suggested flow. The problem with adding the fail2ban2 layer behind forward-auth is that forward-auth needs a dns name, and so I need to use the service dns and route back to traefik a second time to hit the auth service (vs right now I'm using k8s service name which is a direct route in the cluster).

Even isolating to just fail2ban -> forward-auth -> auth-service seems to introduce the original bug.