No endpoints could be generated from HTTPRoute for no reason

[pkit]

time="2024-05-07T03:58:32Z" level=debug msg="No endpoints could be generated from HTTPRoute foo/bar"

And that's it. There's no other logs that can shed any light on why it cannot be generated.
Looking at this atrocity I see a lot of places where it can go wrong, but zero debug logging. Nice!

[pkit]

Namespace discrepancy between Gateway and HttpRoute. eh
Why not print a log line on that?

[jakoberpf]

We also stumbled about this. But we would really like to be able define gateways and route in different namespaces.

@pkit Do you know if Is this not possible (rbac or whatever) or just not implemented as a feature?

[abursavich]

Gateways can have many different Listeners and Routes aren't required to match all of them. The logs would be really noisy if every non-match was logged when this is a commonly expected case. However, it is not expected for a Gateway to have accepted a Route AND for no Listener match to be found. This case is logged in the code you linked.

external-dns/source/gateway.go

Line 360 in fa17f9d

log.Debugf("Gateway %s/%s section %q does not match %s %s/%s hostnames %q", namespace, ref.Name, section, c.src.rtKind, meta.Namespace, meta.Name, rtHosts)

It also logs several other potential issues...

• If the parent is an unknown/unsupported type (i.e. not a Gateway):

  external-dns/source/gateway.go

  Line 298 in fa17f9d

  log.Debugf("Unsupported parent %s/%s for %s %s/%s", group, kind, c.src.rtKind, meta.Namespace, meta.Name)

• If the Gateway couldn't be found:

  external-dns/source/gateway.go

  Line 305 in fa17f9d

  log.Debugf("Gateway %s/%s not found for %s %s/%s", namespace, ref.Name, c.src.rtKind, meta.Namespace, meta.Name)

• If the Gateway is attached to the Route but hasn't accepted it:

  external-dns/source/gateway.go

  Line 310 in fa17f9d

  log.Debugf("Gateway %s/%s has not accepted %s %s/%s", namespace, ref.Name, c.src.rtKind, meta.Namespace, meta.Name)

Gateways, Routes, and Services can cross namespace boundaries, but this must be explicitly allowed. It doesn't happen by default. In your case, @pkit, this is why the Gateway never attached to the Route. Please see the Gateway API's security model for more details (cc @jakoberpf).

I think external-dns should do its best to convert the state of your cluster resources to DNS entries. I don't think it should try to determine the user's true intentions and diagnose misconfigurations of resource specifications. It just takes what it's given.

That said, since it's already logging that no endpoints were found it could also specify that no Gateways were attached.

[jakoberpf]

@abursavich Thanks for elaborating that. Indeed providing the right namespace allow selectors on the gateway fixed my issue.

[zargor]

Faced completely the same issue.
For instance, I want to terminate TLS on GW side, so there is certificate attached to listener section.

listeners:  
    - allowedRoutes:
        namespaces:
          from: All
      name: web-secure
      port: 8443
      hostname: example.com
      protocol: HTTPS
      tls:
        certificateRefs:
          - kind: Secret
            group: ""
            name: example-secret
        mode: Terminate

So on GW side it is HTTPS protocol, while with HTTPRoute there is no protocol (per se) that you can specify.
Now why I think DNS cannot be provisioned.

It is b/c of this function.

There it says if protocols do not match between GW & HTTPRoute that route will be just skipped.

// Confirm that the Listener and Route protocols match.
if !gwProtocolMatches(rt.Protocol(), lis.Protocol) {
  continue
}

Regarding HTTPRoute the only protocol is the one defined by kubernetes service and when web based it is TCP.

Can you advise which protocol is taken into consideration on HTTPRoute side, or perhaps it can be good enough if that condition would be omitted? Thanks.

[abursavich]

@zargor, HTTPRoute matches both HTTP and HTTPS protocols on the Listener.

The first thing to look for is if the Route has been accepted by the Gateway. You can check the Route status for that. If the Gateway hasn't accepted the Route the Gateway won't serve the Route, so ExternalDNS won't match them together either. If the Route has been accepted by the Gateway, then there might be a problem with ExternalDNS. In either case, I'd be happy to help you debug your issue if you provide the full Gateway and HTTPRoute resources, including status. Feel free to obscure the IPs and hostnames if desired, but not so much that it changes the hostname matching semantics.

[zargor]

Sure @abursavich , thanks, here are described gateway and httroute respectfully:

Gateway:

Name:         traefik-g2
Namespace:    traefik
Labels:       app.kubernetes.io/instance=traefik-traefik
              app.kubernetes.io/name=traefik
Annotations:  <none>
API Version:  gateway.networking.k8s.io/v1
Kind:         Gateway
Metadata:
  Creation Timestamp:  2024-08-30T14:27:54Z
  Generation:          1
  Resource Version:    1361676557
  UID:                 cecaf3f9-ecaa-495e-805d-0856580077f7
Spec:
  Gateway Class Name:  traefik
  Listeners:
    Allowed Routes:
      Namespaces:
        From:  All
    Name:      web
    Port:      8000
    Protocol:  HTTP
    Allowed Routes:
      Namespaces:
        From:  All
    Name:      web-mtls
    Port:      8443
    Protocol:  HTTPS
    Tls:
      Certificate Refs:
        Group:
        Kind:   Secret
        Name:   server-secret
      Mode:     Terminate
Status:
  Conditions:
    Last Transition Time:  2024-08-30T14:27:54Z
    Message:               Gateway successfully scheduled
    Observed Generation:   1
    Reason:                Accepted
    Status:                True
    Type:                  Accepted
    Last Transition Time:  2024-08-30T14:27:54Z
    Message:               Gateway successfully scheduled
    Observed Generation:   1
    Reason:                Programmed
    Status:                True
    Type:                  Programmed
  Listeners:
    Attached Routes:  0
    Conditions:
      Last Transition Time:  2024-08-30T14:27:54Z
      Message:               No error found
      Observed Generation:   1
      Reason:                Accepted
      Status:                True
      Type:                  Accepted
      Last Transition Time:  2024-08-30T14:27:54Z
      Message:               No error found
      Observed Generation:   1
      Reason:                ResolvedRefs
      Status:                True
      Type:                  ResolvedRefs
      Last Transition Time:  2024-08-30T14:27:54Z
      Message:               No error found
      Observed Generation:   1
      Reason:                Programmed
      Status:                True
      Type:                  Programmed
    Name:                    web
    Supported Kinds:
      Group:          gateway.networking.k8s.io
      Kind:           HTTPRoute
    Attached Routes:  1
    Conditions:
      Last Transition Time:  2024-08-30T14:27:54Z
      Message:               No error found
      Observed Generation:   1
      Reason:                Accepted
      Status:                True
      Type:                  Accepted
      Last Transition Time:  2024-08-30T14:27:54Z
      Message:               No error found
      Observed Generation:   1
      Reason:                ResolvedRefs
      Status:                True
      Type:                  ResolvedRefs
      Last Transition Time:  2024-08-30T14:27:54Z
      Message:               No error found
      Observed Generation:   1
      Reason:                Programmed
      Status:                True
      Type:                  Programmed
    Name:                    web-mtls
    Supported Kinds:
      Group:  gateway.networking.k8s.io
      Kind:   HTTPRoute
Events:       <none>

HTTPRoute:

Name:         route-ws
Namespace:    some-namespace
Labels:       <none>
Annotations:  external-dns.alpha.kubernetes.io/hostname: example.com
API Version:  gateway.networking.k8s.io/v1
Kind:         HTTPRoute
Metadata:
  Creation Timestamp:  2024-08-29T12:22:29Z
  Generation:          1
  Resource Version:    1361676554
  UID:                 b5535d72-5b5e-400f-80f3-9654f21f92a3
Spec:
  Hostnames:
    example.com
  Parent Refs:
    Group:         gateway.networking.k8s.io
    Kind:          Gateway
    Name:          traefik-g2
    Namespace:     traefik
    Port:          8443
    Section Name:  web-mtls
  Rules:
    Backend Refs:
      Group:
      Kind:    Service
      Name:    ws-deployment
      Port:    8443
      Weight:  1
    Matches:
      Path:
        Type:   PathPrefix
        Value:  /
Status:
  Parents:
    Conditions:
      Last Transition Time:  2024-08-30T14:27:54Z
      Message:
      Observed Generation:   1
      Reason:                Accepted
      Status:                True
      Type:                  Accepted
      Last Transition Time:  2024-08-30T14:27:54Z
      Message:
      Observed Generation:   1
      Reason:                ResolvedRefs
      Status:                True
      Type:                  ResolvedRefs
    Controller Name:         traefik.io/gateway-controller
    Parent Ref:
      Group:         gateway.networking.k8s.io
      Kind:          Gateway
      Name:          traefik-g2
      Namespace:     traefik
      Port:          8443
      Section Name:  web-mtls
Events:              <none>

[abursavich]

@zargor, the only thing that's jumping out at me is that the Gateway doesn't have any IP addresses in its status. The Gateway and the HTTPRoute are attached and ExternalDNS should agree, but it doesn't have any addresses to use for DNS records.

https://github.com/kubernetes-sigs/external-dns/blob/master/docs/sources/gateway.md#targets

[zargor]

@zargor, the only thing that's jumping out at me is that the Gateway doesn't have any IP addresses in its status. The Gateway and the HTTPRoute are attached and ExternalDNS should agree, but it doesn't have any addresses to use for DNS records.

https://github.com/kubernetes-sigs/external-dns/blob/master/docs/sources/gateway.md#targets

In this case, it is about external NLB which comes with its own generic/default dns record. But yes, I suppose there can be an issue, since gateway Address is not populated. On the other hand ,my sense is that external dns should try to create CNAME based on the k8s service, which is responsible for NLB.

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle stale
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle rotten
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten