Nuntio (balenae): Docker containers as consul services

Nuntio is a bridge between Docker containers and the Consul service catalog. It automatically registers/deregisters labelled Docker containers with Consul as they appear.

Using Nuntio

Nuntio watches for new Docker containers and inspects them to determine what services they provide. Any services are added to a service registry.

Nuntio is highly extensible and configurable. It is possible to configure different platforms as service sources and to configure different service registries as targets. The default application is a Docker platform and Consul service registry.

Concepts

Nuntio uses a couple of abstract concepts throughout its code and documentation.

Service

A service is anything that listens on a TCP or UDP port.

(Service) Platform

A platform is the thing that manages/runs processes that have services and can provide metadata about the process state (IP/port that it listens on, running or not, health status, ...).

(Service) Registry

A registry is the thing where services are stored in, so they can be queried by other applications.

Platform Service

A platform service is the application as seen by the Service Platform. Since a process can listen on multiple ports, it can contain multiple individual Services.

Registry Service

A registry service is the service as stored in the Service Registry. It maps to exactly one Service.

Check

A check is a component that checks the health of a Service. Checks can be handled by the Service Registry itself, or in Nuntio if the Service Registry does not support a check type.

Configuration options

Nuntio is a Spring Boot application. This means there are many ways to supply configuration parameters:

• Java system properties: use -D<name>=<value>
• Environment variables: replace all . with _
• Command-line arguments: use --<name>=<value>

Additional ways to supply configuration options is documented in the Spring Boot documentation

Most specific configuration options are dependent on the selected platform and registry and are documented there.

Only global configuration options are documented here.

Option	Default	Description
nuntio.engine.serviceAddress.[ipv4|ipv6]	true	Which type of service addresses will be registered in the service registry.
nuntio.engine.forcedTags	{}	Force-add comma-separated tags on all services
nuntio.engine.live.enabled	true	Enables watching the platform eventstream to immediately react to services changing state.
nuntio.engine.live.blocking	true	Enables blocking-mode watching of the eventstream. This option can be disabled to fall back to polling-mode operation in case there are problems with waking up blocked threads.
nuntio.engine.live.delay	1s	Time between checks in polling-mode watching.
nuntio.engine.antiEntropy.enabled	true	Enables anti-entropy scanning.
nuntio.engine.antiEntropy.delay	1m	Time between anti-entropy scans.
nuntio.engine.checks.heartbeat	true	Enable registration of heartbeat check.
nuntio.engine.checks.healthcheck	true	Enable registration of healthcheck check.
nuntio.engine.shutdownMode	UNREGISTER_CHECKS	What actions nuntio should take when its process is terminated. UNREGISTER_SERVICES: Unregister all services that were registered by this nuntio instance. UNREGISTER_CHECKS: Unregister all checks for all services that were registered by this nuntio instance. UNREGISTER_HEARTBEAT: Unregister only the heartbeat check. NOTHING: Do not perform any special operation when shutting down.

Docker platform

The Docker platform registers services for containers with certain labels (or environment variables).

Docker platform configuration

Option	Default	Description
nuntio.docker.enabled	true	Enable Docker platform
nuntio.docker.daemon.host	Platform-dependent	Connection string to the docker daemon (e.g.: unix:///var/run/docker.sock, tcp://some-host:2375)
nuntio.docker.daemon.tlsVerify	false	Enable/disable TLS verification (switch between http & https protocols)
nuntio.docker.daemon.certPath	null	Path to certificates needed for TLS verification
nuntio.docker.bind	PUBLISHED	Values: PUBLISHED/INTERNAL. Which IP address/port to publish for containers. PUBLISHED: Only published ports are used. They are used with the host IP and bound port. INTERNAL: All exposed ports are used. They are used with the internal container IP and exposed port. If there are multiple internal container IPs (when the container is attached to multiple networks), no service will be published. To select a specific (e.g. overlay) network, a network filter can be specified with nuntio.docker.bind.filter.
nuntio.docker.bind.filter		Only applicable for INTERNAL binds. Docker filters to determine which network(s) to use IP addresses for. Filters are comma-separated and the same format as in docker network ls. Only one network may match per-container.
nuntio.docker.nuntioLabel.enabled	true	Enables configuring services on Docker containers with labels following the [label specification](#docker-labels-for-nuntio)
nuntio.docker.nuntioLabel.prefix	nuntio.xenit.eu	Prefix for Nuntio labels.
nuntio.docker.registratorCompat.enabled	false	Enable compatibility with some registrator labels/environment variables. See [Registrator compatibility](#registrator-compatibility-mode) for details on which parts are implemented.
nuntio.docker.registratorCompat.explicit	false	Enables -explicit mode registrator compatibility, only registering containers that have service configurations present.

Docker labels for Nuntio

Nuntio needs to know which services to register with which name, tags and metadata. This information is specified with labels on the docker container.

To avoid collisions with other uses for labels, they are namespaced under a configurable prefix (see nuntio.docker.nuntioLabel.prefix).

There are two options to specify labels:

• Global: these labels apply to all services (published/exposed ports) on the Docker container.
• Per-port: these labels apply only to the service with the matching exposed port on the Docker container. Port-specific labels are applied by appending the port number to the service prefix: <prefix>/<port>. If you have an UDP service, you need to specify it on the service prefix: <prefix>/udp:<port>

The labels follow these formats:

• <prefix>/service: Comma-separated service names to use for this service. Required to be able to register a service.
• <prefix>/tags: Comma-separated tag names to register with this service.
• <prefix>/metadata/<key>: Metadata value to register with this service.

Label examples

• nuntio.xenit.eu/service=my-service,other-service: All ports of the container are registered under both my-service and other-service
• nuntio.xenit.eu/80/service=lb-front: Port 80/tcp of the container is registered under lb-front
• nuntio.xenit.eu/udp:53/tags=authoritative: The services for port 53/udp of the container will have the authoritative tag
• nuntio.xenit.eu/metadata/my-key=interesting-value: The services for all ports of the container will have metadata value my-key=interesting-value

Some concrete examples:

services:
   db1:
      image: postgres
      ports:
        - 1234:5432
      labels:
        nuntio.xenit.eu/service: db
        nuntio.xenit.eu/tags: primary
        nuntio.xenit.eu/metadata/hosted-dbs: metrics,users
   db2:
     image: postgres
     ports:
       - 1235:5432
     labels:
       nuntio.xenit.eu/5432/service: db,db-analytics
       nuntio.xenit.eu/5432/tags: secondary
       nuntio.xenit.eu/5432/metadata/hosted-dbs: metrics,users
   app:
     image: my-app:latest
     ports:
       - 8080
       - 8081
       - 8082
     labels:
       nuntio.xenit.eu/8080/service: app-public
       nuntio.xenit.eu/8080/metadata/published-domain: my-awesome-app.example
       nuntio.xenit.eu/8081/service: app-admin
       nuntio.xenit.eu/metadata/prometheus-scrape: /metrics/prometheus
   custom-dns:
      image: my-dns-server:latest
      ports:
        - 10.5.2.1:53:5300/udp 
      labels:
        nuntio.xenit.eu/udp:5300/service: dns

This will register the following services:

• db
  • instance db1: ServicePort=1234 Tags=primary Meta[hosted-dbs]=metrics,users
  • instance db2: ServicePort=1235 Tags=secondary Meta[hosted-dbs]=metrics,users
• db-analytics
  • instance db2: ServicePort=1235 Tags=secondary Meta[hosted-dbs]=metrics,users
• app-public
  • instance app: ServicePort=[mapped port for 8080] Meta[published-domain]=my-awesome-app.example Meta[prometheus-scrape]=/metrics/prometheus
• app-admin
  • instance app: ServicePort=[mapped port for 8081] Meta[prometheus-scrape]=/metrics/prometheus
• dns
  • instance custom-dns: ServiceIp=10.5.2.1 ServicePort=53

Registrator compatibility mode

To ease migration from registrator to Nuntio, parts of the registrator model are supported.

Compatibility can be enabled with the nuntio.docker.registratorCompat.enabled=true.

Following registrator features are supported for both labels and environment variables:

• SERVICE_IGNORE: If this variable is present, no services on this container will be registered.
• SERVICE_<port>_IGNORE: If this variable is present, the service on the port is not registered.
• SERVICE_NAME, SERVICE_<port>_NAME: Setting a service name. If SERVICE_NAME is used and multiple services are present, the service name will be suffixed with the internal port.
• SERVICE_TAGS, SERVICE_<port>_TAGS: Setting service tags.
• SERVICE_<meta>, SERVICE_<port>_<meta>: Setting service metadata values.

Following registrator configuration options are supported:

• -internal: Use nuntio.docker.bind=INTERNAL to register internal IP and port instead of the host mapped ones.
• -explicit: Use nuntio.docker.registratorCompat.explicit=true
• -tags: Use nuntio.engine.forcedTags to force-enabled tags on services

Consul registry

The Consul registry supports registering services to Consul.

Consul registry configuration

Option	Default	Description
nuntio.consul.enabled	true	Enable Consul registry
nuntio.consul.host	localhost	Hostname of the local Consul agent
nuntio.consul.port	8500	Port of the Consul HTTP API
nuntio.consul.token	null	ACL token for the Consul HTTP API
nuntio.consul.checks.[heartbeat|pause|healthcheck].ttl	heartbeat: 24h pause: 5m healthcheck: 5m	TTL before a check type expires and is considered critical.
nuntio.consul.checks.[heartbeat|pause|healthcheck].deregisterCriticalServiceAfter	heartbeat: 72h pause: null healthcheck: null	Time before a critical check deregisters the service.