✨ Support provisioning with docker-compose.yml

[linkdd]

Netbox version	Plugin Version	Agent version	Docker API Version
3.7.*	1.6.1	-	-

Actual Behavior

In order to provision a complete stack, one has to create manually the registries, images, volumes, networks, and containers, 1 by 1.

This is a bit tedious, and error-prone (you could go too fast and create the container when the image has not been pulled yet 😱 ).

This is usually orchestrated via an Ansible playbook, which will make the API calls to Netbox, and has to poll the journal entries to determine when i can move on to the next step, or fail.

Such Ansible playbooks are tedious to write, maintain and debug.

Desired Behavior

A possible solution would be to support a subset of the docker-compose.yml file format.

An API endpoint would expect via POST:

• the compose file (as YAML or JSON) as the request body
• the docker host id as either URL Path parameter, or Query String parameter

The endpoint handler would have to:

• create or update the Netbox entities
• poll their journal entries to determine when to move to the next step

The "create or update" part is essential in case of failure so that already successful operations are a "No Op" in the retry.

I considered relying on Django transactions, but that would not work, as the entities would not be committed until the transaction is done, which would not trigger the Docker Agent until the end, which would not allow us to poll journal entries for the required orchestration.

---

To be defined:

• the docker-compose.yml file format does not support specifying registry credentials, should we extend the file format to include them or expect registries to be already present in the Netbox?
• how do we infer the correct registry in Netbox to use from the image field of the compose file?

[linkdd]

How do we infer the correct registry to use in Netbox from the image field of the compose file

https://docs.docker.com/reference/cli/docker/image/tag/#extended-description

An image tag is [HOST[:PORT]/]PATH.

When HOST is not present, it defaults to the public registry registry-1.docker.io with docker.io being the canonical name.

In Netbox, dockerhub has the following server address:

https://registry.hub.docker.com/v2

So, that would be a special case, is it the only one?

When HOST (and PORT) is present, the registry server address is:

http(s)://HOST[:PORT]/v2

For example:

Image	Registry Server Address
myregistry:8080/myimage:latest	https://myregistry:8080/v2/
myimage:latest	https://registry.hub.docker.com/v2/
docker.io/myimage:latest	https://registry.hub.docker.com/v2/

Is that correct?

[lvenier]

I would not consider registries for this feature in the initial version.

I would first modify the image entity in the plugin to make the registry as optional and defaulting a "defaulted registry" to it within the plugin through the API.
Then you would be able to implement this feature with the defaulted registry in the first version.

[linkdd]

Bookmark:

Today, I discovered https://schemastore.org

The docker-compose.yml file format spec is hosted on this website (via Github) and can be found here: https://raw.githubusercontent.com/compose-spec/compose-spec/master/schema/compose-spec.json

We can use this in the new API endpoint to validate the received document reliably.