diff --git a/.markdownlint.yml b/.markdownlint.yml index fd23fe852..64bc85e39 100644 --- a/.markdownlint.yml +++ b/.markdownlint.yml @@ -6,3 +6,6 @@ ol-prefix: false no-duplicate-heading: false single-h1: false no-emphasis-as-heading: false +no-hard-tabs: + code_blocks: true + spaces_per_tab: 2 \ No newline at end of file diff --git a/docs/proposal/dual_engine_dns.md b/docs/proposal/dual_engine_dns.md new file mode 100644 index 000000000..44ba05acf --- /dev/null +++ b/docs/proposal/dual_engine_dns.md @@ -0,0 +1,242 @@ +--- +title: Support DNS Resolution in Dual Engine Mode +authors: +- "@Flying-Tom" +reviewers: +- +approvers: +- +creation-date: 2025-07-09 +--- + +## Support DNS Resolution in Dual Engine Mode + + + +### Summary + + + +This proposal adds DNS resolution support for Dual-Engine mode workloads, enabling seamless Istio migration by supporting ServiceEntry resources with DNS-based endpoints. Workloads dynamically resolve hostnames to IP addresses without static configuration. + +### Motivation + + + +In Istio, [ExternalName services](https://kubernetes.io/docs/concepts/services-networking/service/#externalname) and DNS-typed [ServiceEntry](https://istio.io/latest/docs/reference/config/networking/service-entry/#ServiceEntry-Resolution) rely on client-side DNS resolution. When istiod processes these configurations, it generates workloads without pre-resolved IP addresses. + +Example ServiceEntry: + +```yaml +apiVersion: networking.istio.io/v1 +kind: ServiceEntry +metadata: + name: external-svc-google + namespace: default +spec: + hosts: + - news.google.com + ports: + - number: 80 + name: http + protocol: HTTP + resolution: DNS +``` + +In Release 1.1, Kmesh refactored the DNS module, extracting DNS logic from the kernel-native Ads controller into a standalone `AdsDnsController`. This decoupling enables reuse in Dual-Engine mode. + +![](./pics/dns-evolution.png) + +#### Goals + + + +- Support DNS resolution for workloads generated from ServiceEntry resources in Dual-Engine mode +- Implement asynchronous DNS resolution to avoid blocking workload processing +- Provide automatic cleanup when workloads with DNS hostnames are removed +- Support both IPv4 and IPv6 address resolution + +#### Non-Goals + + + +This KEP does not aim to implement or provide a DNS proxy or DNS server functionality. Specifically, we do not support resolving DNS names on behalf of client workloads. As a result, if a `ServiceEntry` uses a non-resolvable or fake DNS domain, client workloads may fail to resolve and access the intended service. Handling such DNS resolution scenarios is explicitly out of scope for this proposal. + +### Proposal + + + +Thanks to the `DNSResolver` in `pkg/dns` which extracted independent DNS resolution logic, we can reuse the DNS capabilities in Dual-engine mode. Inspired by the `AdsDnsController` in kernel-native mode, we implement a similar `WorkloadDnsController` to handle DNS resolution for workloads generated by `ServiceEntry` without address information. + +The controller receives workloads needing DNS resolution from the Processor via a channel, handles the DNS resolution asynchronously, and sends the resolved results back to the Processor through per-workload result channels. This design ensures that DNS resolution does not block the workload processing pipeline. + +### Design Details + + + +#### Architecture Overview + +The DNS resolution flow in Dual-Engine mode follows this pattern: + +```txt +Processor → WorkloadDnsController → DNSResolver → Upstream DNS + ↑ ↓ + └────── Resolved Workload ────────────┘ +``` + +![](./pics/dual-engine-dns.png) + +#### Key Components + +##### WorkloadDnsController + +The WorkloadDnsController manages asynchronous DNS resolution for ServiceEntry-generated workloads. Key data structures: + +- **Resolution Queue**: Buffered channel receiving workloads from Processor for non-blocking submission +- **Result Channel Registry**: Per-workload result channels (indexed by UID) for independent resolution tracking +- **Domain Resolution Cache**: Bidirectional index between hostnames and pending workloads for batch resolution + +The controller operates through three concurrent workers: + +1. **Domain Processor**: Consumes workloads, groups by domain, delegates to DNS Resolver +2. **Refresh Worker**: Receives resolved addresses, constructs workload objects, delivers via result channels +3. **DNS Resolver**: Executes DNS queries (A/AAAA), maintains TTL-based cache (reused from `pkg/dns`) + +##### Resolution Flow + +The DNS resolution mechanism follows a producer-consumer pattern with timeout protection: + +**Workload Submission**: When the Processor encounters a workload without addresses, it registers a result channel and enqueues the workload for resolution. The Processor blocks on the result channel with a 3-second timeout to prevent pipeline blocking. + +**Domain Aggregation**: The Domain Processor maintains a hostname-indexed cache to aggregate workloads requiring the same domain, reducing redundant DNS queries. It checks for cached resolutions before initiating new queries. + +**Address Resolution**: The DNS Resolver performs parallel IPv4 (A) and IPv6 (AAAA) queries, respecting DNS TTL values. Resolved addresses are stored in a protocol-agnostic format. + +**Result Distribution**: The Refresh Worker reconstructs workload objects with resolved addresses and delivers them via result channels. Channel operations include a 100ms send timeout to prevent deadlocks. After delivery, the controller removes the result channel registration to prevent memory leaks. + +##### Cleanup Mechanism + +When a workload is deleted, the controller: + +- Removes the workload from pending hostname tracking +- Removes the workload from the hostname's pending domain cache +- If no more workloads depend on the hostname, unwatches the domain from DNS resolver + +This ensures no memory leaks and prevents unnecessary DNS queries. + +##### Design Rationale + +The WorkloadDnsController design diverges from AdsDnsController in several key aspects to better accommodate workload-level resolution requirements: + +| Design Decision | Approach | Rationale | +|----------------|----------|-----------| +| **Result Delivery** | Dedicated per-workload channels | Eliminates result filtering overhead and spurious wake-ups; enables direct workload-specific blocking without cross-workload interference | +| **Timeout Strategy** | Two-tier mechanism (3s processor + 100ms channel) | Processor-level timeout prevents indefinite pipeline blocking; channel-level timeout prevents deadlocks from abandoned consumers | +| **Address Format** | `netip.ParseAddr().AsSlice()` byte representation | Provides protocol-agnostic representation supporting both IPv4 and IPv6 without conditional logic | +| **Refresh Interval** | Fixed 200ms rate | Simplifies implementation while maintaining adequate freshness for typical ServiceEntry use cases; trades configurability for consistency | + +#### Integration Points + +The WorkloadDnsController integrates into the Kmesh control plane through two primary integration points: + +| Component | Integration Method | Lifecycle | +|-----------|-------------------|-----------| +| **WorkloadController** | Instantiated during `NewController()` | Started via `Run()` context; shutdown via context cancellation | +| **Processor** | Reference-based invocation | Synchronous blocking on resolution for address-less workloads; ensures data consistency before processing | + +**Initialization Sequence**: + +1. WorkloadController creates WorkloadDnsController instance +2. DnsController reference stored in Processor +3. DnsController goroutines started when WorkloadController.Run() is invoked +4. Lifecycle bound to WorkloadController's context + +**Runtime Interaction**: + +1. Processor detects workload without addresses (ServiceEntry-originated) +2. Processor registers result channel in DnsController +3. Processor submits workload to resolution queue +4. Processor blocks on result channel with timeout +5. DnsController delivers resolved workload or timeout expires +6. Processor continues with workload processing + +#### Comparison with AdsDnsController + +| Aspect | AdsDnsController | WorkloadDnsController | +|--------|------------------|----------------------| +| **Input** | Clusters with DNS endpoints | Workloads without addresses | +| **Processing Unit** | Cluster | Workload | +| **Result Delivery** | Shared channel | Per-workload channels | +| **Timeout** | No processor timeout | 3s processor + 100ms channel | +| **Refresh Rate** | From cluster config | Fixed 200ms | +| **Cleanup** | On cluster removal | On workload removal | + +#### Test Plan + + + +**Unit Tests** + +DNS controller unit tests cover IPv4, IPv6, and dual-stack resolution scenarios, workload address overwriting logic, cleanup on workload deletion, and concurrent resolution of multiple workloads. + +**E2E Tests** + +E2E test validates the end-to-end ServiceEntry DNS resolution flow by creating a ServiceEntry with DNS resolution pointing to a fake hostname, creating a VirtualService routing the fake hostname to a real service, and verifying traffic flows successfully. + +Note: DNS proxy is disabled in IPv6-only environments, so tests skip in that configuration. + +### Alternatives + + + +**Synchronous Resolution in Processor**: Embedding DNS resolution directly within the Processor's main workload handling loop would introduce blocking behavior, eliminating the possibility of batching concurrent resolutions for identical domains and complicating timeout implementation. This approach violates the separation of concerns principle by coupling network I/O with workload state management. + +**Shared Result Channel**: Reusing AdsDnsController's single shared channel pattern would necessitate complex result filtering logic to match responses with their corresponding workload requests. The additional synchronization overhead and potential for spurious wake-ups make this approach less suitable for workload-level granularity. + +**Kernel-Space DNS Resolution**: Implementing DNS resolution within eBPF programs would require reimplementing the DNS protocol stack in a constrained execution environment with strict complexity limits. This approach would duplicate existing userspace functionality, significantly increase maintenance burden, and provide minimal performance benefits given the infrequent nature of DNS lookups. + + \ No newline at end of file diff --git a/docs/proposal/pics/dns-evolution.png b/docs/proposal/pics/dns-evolution.png new file mode 100644 index 000000000..e402e1194 Binary files /dev/null and b/docs/proposal/pics/dns-evolution.png differ diff --git a/docs/proposal/pics/dual-engine-dns.png b/docs/proposal/pics/dual-engine-dns.png new file mode 100644 index 000000000..666d000fb Binary files /dev/null and b/docs/proposal/pics/dual-engine-dns.png differ