vmod_dynamic

Varnish dynamic backends module

Manual section:	3

SYNOPSIS

import dynamic [as name] [from "path"]

new xdirector = dynamic.director(STRING port, STRING host_header, ENUM share, PROBE probe, ACL whitelist, DURATION ttl, DURATION connect_timeout, DURATION first_byte_timeout, DURATION between_bytes_timeout, DURATION domain_usage_timeout, DURATION first_lookup_timeout, INT max_connections, INT proxy_header, BLOB resolver, ENUM ttl_from, DURATION retry_after, BACKEND via, INT keep, STRING authority, DURATION wait_timeout, INT wait_limit)

    BACKEND xdirector.backend(STRING host, STRING port, STRING authority)

    BACKEND xdirector.service(STRING service)

    VOID xdirector.debug(BOOL)

new xresolver = dynamic.resolver(BOOL set_from_os, INT parallel)

    BLOB xresolver.use()

    BOOL xresolver.set_resolution_type(ENUM)

    BOOL xresolver.clear_namespaces()

    BOOL xresolver.add_namespace(ENUM)

    BOOL xresolver.set_namespaces()

    BOOL xresolver.clear_transports()

    BOOL xresolver.add_transport(ENUM)

    BOOL xresolver.set_transports()

    BOOL xresolver.set_idle_timeout(DURATION)

    BOOL xresolver.set_limit_outstanding_queries(INT)

    BOOL xresolver.set_timeout(DURATION)

    BOOL xresolver.set_follow_redirects(ENUM)

DESCRIPTION

This module provides a varnish director for dynamic creation of backends based on calls to

the system's network address resolution service which, in turn, typically use information from the /etc/hosts file and the Domain Name Service (DNS), but can be configured to use other sources like LDAP (see nsswitch.conf(5)).
or more advanced DNS resolution where getdns is available.

While standard varnish backends defined in VCL may also be defined in terms of host names, changes of the name service information will only be picked up with a VCL reload.

In contrast, for dynamic backends provided by this module,

name resolution information will be refreshed by background threads after a configurable time to live (ttl) or after the ttl from DNS with a getdns dynamic.resolver().
resolution to multiple network addresses is supported

In addition, with a getdns dynamic.resolver(), service discovery by DNS SRV records is possible, in which case this module also allows to configure host names (targets), their ports, priority and weight though DNS. See https://en.wikipedia.org/wiki/SRV_record for a good basic explanation and xdirector.service() for details.

BACKEND SHARING

The share parameter specifies if backends are shared per director or per hostname.

share = "DIRECTOR"

Default if via is not specified.

Backends are shared per director: Only one backend per director instance and ip address (and optionally port) is created.
share = "HOST"

Default if via is specified.

Sharing of backends is limited to the same hostname (host argument to the .backend() method).

This is the default for via for the common use case of TLS: For TLS connections, the TLS onloader usually should verify the server's certificate based on a hostname. For this purpose, the dynamic backend's name is put into the authority TLV, so, consequently, the backend needs to be defined per hostname.

PROBING

A probe to be used with dynamically created backends can be specified.

With share = HOST, the Host: header for probes defaults to the backend's hostname (host argument to the .backend() method).

With share = DIRECTOR, the probe is not specific to any particular host. If the probe has the .request attribute set, it will be used as the probe request. Otherwise, if a host_header argument was given to dynamic.director(), it will be used for the probe's Host header, otherwise Varnish-Cache core code will send the backend's IP Address as the Host header.

Consider setting the initial attribute of probes at least as high as the threshold attribute. Otherwise transactions that trigger the first lookup of a domain will see a sick backend and fail.

Irrespective of the initial attribute, transactions may still fail for backends which are actually sick. This can be mitigated using the retry transition in VCL.

TTLs from DNS

With the default system resolver, TTLs from DNS are not supported optimally. While a good combination of a ttl parameter in combination with a system name service caching service like nscd(8) can achieve good results, to use TTLs from DNS, we recommend to compile this module with getdns support, configure a dynamic.resolver() object and set the ttl_from parameter to either dns, min or max as in this example:

sub vcl_init {
        new r = dynamic.resolver();
        new d = dynamic.director(
                resolver = r.use(),
                ttl_from = dns
                );
}

See ttl_from for details.

NAMES

Directors and backends created by this vmod follow this naming scheme, which will be referred to as <name> in the following documentation

<directorname>

The name of the VCL object created by dynamic.director()
<directorname>(<hostname>:<port>)

A director (internally called domain), created as a result of a xdirector.backend() or, indirectly, for targets resulting from a xdirector.service() call.
<directorname>(service)

A director created as a result of a xdirector.backend() or xdirector.service() call.
<directorname>(address:port)

A dynamic backend for a <hostname>:<port> with DIRECTOR scope sharing
<directorname>(hostname.address:port)

A dynamic backend for a <hostname>:<port> with HOST scope sharing

port may be represented symbolically (http by default)

If an authority is set, it is appended to address:portseparated by /.

STATISTICS

Dynamic backends are created and deleted on demand and can be monitored just like VCL-defined backends. Their statistics will appear in VSM-tools like varnishstat as:

VBE.<configname>.<name>.*

LOGGING

This module may log VCL_Log, Error, and Debug records following a common pattern:

vmod-dynamic %s %s %s %s [ %s ]
             |  |  |  |    |
             |  |  |  |    +- Additional information
             |  |  |  +------ Event
             |  |  +--------- <name> without the <director> part
             |  +------------ Director name
             +--------------- VCL name

Lookup timestamps are also logged to help troubleshooting, using regular Timestamp records with the following pattern for event labels:

vmod-dynamic <vcl>.<name> <Lookup|Results|Update>

When a lookup thread is terminated, either because the VCL is cooling down or the domain_usage_timeout triggered, Timestamp records are logged with the event:

vmod-dynamic <vcl>.<name> Done

Not all logs belong to HTTP transactions, especially since DNS lookups happen in the background. In order to capture all logs from this module the simplest way with varnishlog is the following:

varnishlog -g raw -q '* ~ vmod-dynamic'

It displays any individual record that contains the string vmod-dynamic whether it belongs to a transaction or not.

When a lookup fails, the backends are left untouched and the error will be logged with the following event:

getaddrinfo <errno> (<reason>)

new xdirector = dynamic.director(STRING port, STRING host_header, ENUM share, PROBE probe, ACL whitelist, DURATION ttl, DURATION connect_timeout, DURATION first_byte_timeout, DURATION between_bytes_timeout, DURATION domain_usage_timeout, DURATION first_lookup_timeout, INT max_connections, INT proxy_header, BLOB resolver, ENUM ttl_from, DURATION retry_after, BACKEND via, INT keep, STRING authority, DURATION wait_timeout, INT wait_limit)

new xdirector = dynamic.director(
   STRING port="http",
   STRING host_header=0,
   ENUM {DEFAULT, DIRECTOR, HOST} share=DEFAULT,
   PROBE probe=0,
   ACL whitelist=0,
   DURATION ttl=3600,
   DURATION connect_timeout=-1,
   DURATION first_byte_timeout=-1,
   DURATION between_bytes_timeout=-1,
   DURATION domain_usage_timeout=7200,
   DURATION first_lookup_timeout=10,
   INT max_connections=0,
   INT proxy_header=0,
   BLOB resolver=NULL,
   ENUM {cfg, dns, min, max} ttl_from=cfg,
   DURATION retry_after=30,
   BACKEND via=NULL,
   INT keep=3,
   STRING authority=NULL,
   DURATION wait_timeout=-1,
   INT wait_limit=0
)

Description

Create a DNS director.

The director creates backends with DNS lookups and chooses them in a round robin fashion. It accepts the following

Parameters:

port (defaults to "http")

The port to conncet to
share (defaults to "DIRECTOR")

Backend sharing scope, see BACKEND SHARING
host_header (defaults to none)

host_header attribute for dynamically created backends. See also PROBING
probe (defaults to none)

Probe to use. See also PROBING
whitelist - an acl (defaults to none)

Only name resolution results matching the acl will be used.
ttl - interval between lookups (defaults to one hour)

Minimum configured backend lifetime before address resultion is re-checked, see also ttl_from

Related parameter: keep
domain_usage_timeout

Delay until an unused domain and its backends are removed (defaults to two hours)
first_lookup_timeout

Delay until the director fails lookup when a domain is requested for the first time and there is no response from the name service (defaults to ten seconds)
resolver

Use a particular resolver instance created with the dynamic.resolver() constructor, if available.

The argument to the resolver parameter must be the return value of the xresolver.use() method.

ttl_from

How to determine the minimum backend lifetime before address resultion is re-checked when a resolver is also used:

cfg: Always use the ttl argument value (default)

dns: Always use the ttl from the DNS response(s), falling back to the ttl argument value

min: Use the minimum of the DNS response and the ttl argument value

max: Use the maximum of the DNS response and the ttl argument value

If there is more than one DNS response, the minimum if taken as the DNS ttl.

For no resolver, only "cfg" is valid.

retry_after

Delay to retry lookups after DNS failures (defaults to 30 seconds).

Notice that, for all practical purposes, this only concerns lookup errors (NXDOMAIN). After successful resolution, dynamic directors preserve existing address information if DNS resources are temporarily unavailable.

keep (default: 3)

For how many subsequent updates to keep backends configured which are no longer returned by name resolution.

Some DNS services do not return a stable set of IP addresses for consecutive queries, but rather a varying subset from a larger pool. The keep parameter is intended to keep backends configured but unused for such scenarios, in case they re-appear later.

keep = 0 releases backends immediately as soon as they are no longer contained in a resolution result.

Any other number specifies for how many updates (in ttl intervals) to keep backends configured while they are not contained in the resolution result.

Until Varnish-Cache issue 3949 is addressed, it is recommended to use a keep value of at least 1.

The keep value must not be negative and it is capped to a high positive value (UINT_MAX, usually 4 294 967 295, see limits.h(7POSIX)).

Parameters to set attributes of backends

See varnish documentation for details

connect_timeout (defaults to global connect_timeout)

first_byte_timeout (defaults to global first_byte_timeout)

between_bytes_timeout (defaults to global between_bytes_timeout)

max_connections (defaults to zero, unlimited)

proxy_header - version of the PROXY protocol to use, or zero to disable it (defaults to zero, valid versions are one or two)

via backend to make the connection through with an additional PROXY protocol header containing the actual address to connect to. Primary use case is SSL/TLS onloading via haproxy.

authority default for the xdirector.backend() argument. The main use case for this argument to dynamic.director() is to disable SNI by default by setting it to the empty string as authority = "".

wait_timeout (defaults to global backend_wait_timeout)

wait_limit (defaults to global backend_wait_limit)

Example

probe www_probe {
     .window = 8;
     .initial = 7;
     .threshold = 6;
     .interval = 5s;
}

acl www_acl {
     "192.168"/24;
}

sub vcl_init {
     new www_dir = dynamic.director(
             port = "80",
             probe = www_probe,
             whitelist = www_acl,
             ttl = 5m);
}

sub vcl_recv {
     set req.backend_hint = www_dir.backend("production.acme.com");
}

BACKEND xdirector.backend(STRING host, STRING port, STRING authority)

BACKEND xdirector.backend(
      STRING host="",
      STRING port="",
      STRING authority=NULL
)

Description

Return a backend from the director for a given host name and, optionally, authority. If the host name resolves to multiple addresses, they are used in a round-robin fashion.

If possible, a healhy backend is returned. If no healthy backends are known, any backend is returned. Note that it may or may not be actually responsive.

If the host is not specified, it is picked from either bereq or req.

If the port is not speficied, it is taken from the director.

If authority is set, it's value is used as the authority TLV in the PROXY header sent to the via host, which, in turn, will probably use it as the SNI for an outgoing TLS connection.

If authority is not set, it will fall back to these options in order:

authority from the director
host_header from the director
host argument

If authority is set to the empty string, or if via is not set, then the TLV is not sent.

BACKEND xdirector.service(STRING service)

Description

Return a backend from the director for a service name (DNS SRV record).

This Method is only supported when a dynamic.resolver() object has been passed to the dynamic.director() constructor.

SRV records contain host (called target) and port information. Dynamic backends are automatically added and maintained based on this information as if xdirector.backend() had been called.

SRV records also contain priority and weight information. The xdirector.service() method returns a target (backend) from the lowest priority found healthy. If there are multiple healthy targets for the lowest priority, one is chosen randomly based on the probabilities defined by the weight attributes.

If there are no healthy targets at all, the last target from the lowest priority is returned.

Note that it highly recommended to use a probe argument to the dynamic.director() to avoid returning unresponsive backends.

VOID xdirector.debug(BOOL)

Description: Enable or disable debugging for a dynamic director, logging background operations related to backends management.

new xresolver = dynamic.resolver(BOOL set_from_os, INT parallel)

new xresolver = dynamic.resolver(
   BOOL set_from_os=1,
   INT parallel=16
)

Create parallel getdns contexts to be used with the dynamic.director() constructor - see xresolver.use()

Parameters:

set_from_os: whether defaults are taken from the operating system,

see https://getdnsapi.net/documentation/spec/#8-dns-contexts for details
parallel: the number of contexts allocated. Each name resolution

in progress requires one context.

Additional configuration of the resolver contexts is possible through the methods documented below. Attempts to call these methods from outside vcl_init{} will trigger a VCL failure.