DockerFinder: Multi-attribute search of Docker images

Antonio Brogi, Davide Neri, Jacopo Soldani
IEEE International Conference on Cloud Engineering (IC2E), 2017 

Download the citation as .bib

@inproceedings{BrogiNeriSoldani:dockerfinder,
  author    = {Antonio Brogi and
               Davide Neri and
               Jacopo Soldani},
  title     = {DockerFinder: Multi-attribute Search of Docker Images},
  booktitle = {2017 {IEEE} International Conference on Cloud Engineering, {IC2E}
               2017, Vancouver, BC, Canada, April 4-7, 2017},
  pages     = {273--278},
  publisher = {{IEEE}},
  year      = {2017},
  url       = {https://doi.org/10.1109/IC2E.2017.41},
  doi       = {10.1109/IC2E.2017.41},
  timestamp = {Wed, 17 May 2017 10:11:45 +0200},
  biburl    = {http://dblp.uni-trier.de/rec/bib/conf/ic2e/BrogiNS17},
  bibsource = {dblp computer science bibliography, http://dblp.org}
}

Why DockerFinder ?

Docker only permits looking for images by specifying a term , which is then exploited to return all images where such term occurs in the name, in the description or in the name of the user that built the image.

For example, if we submit the query with the term java

$ docker search java

the result is:

NAME                   DESCRIPTION                                     STARS     OFFICIAL   AUTOMATED
java                   Java is a concurrent, class-based, and obj...   1264      [OK]       
anapsix/alpine-java    Oracle Java 8 (and 7) with GLIBC 2.23 over...   177                  [OK]
develar/java                                                           52                   [OK]
isuper/java-oracle     This repository contains all java releases...   48                   [OK]

As a consequence, users cannot specify more complex queries, e.g., by imposing requirements on the software distributions an image must support

What is DockerFinder ?

DockerFinder is a microservice-based prototype that permits searching for images
based on multiple attributes.

The attributes for which is possible to search an image are:

1. Software versions supported (e.g. python 2.7 or java 1.8).
2. Size.
3. Stars.
4. Pulls.

Try-it now

You can try DockerFinder by using:

• GUI: web-based UI that permits searching for images with a graphical environment.
• CLI: a command line interface client. Credits to: lucarin91

GUI of Docker Finder

The GUI of DockerFinder is running on http://black.di.unipi.it/dockerfinder

An example of a multi-attribute query submitted to DockerFinder by using the GUI is shown in the gif below. It search the images that support:

• Java 1.8,
• Python 2.7,
• pulls >= 20.

CLI of DockerFinder

Install the Dockerfinder client using pip by the following command:

pip install dfinder

After the installation just invok the dfinder software as following:

dfinder java:1.7 --sort -size --sort +stars

Docker Finder main steps

1. DockerFinder crawls images from a remote Docker registry,
2. It automatically analyses such images to produce multi-attribute descriptions to be stored in a local repository,
3. It permits searching for images by querying the local repository through a GUI or a RESTful API.

The microservice-based architecture DockerFinder

The figure below details the microservice-based architecture of Docker Finder. The microservice (represented as rectangles) are divided in the three three main functionalities carried out by Docker Finder:

1. Analysis: the analysis of each image consists in retrieving all the metadata already available in the registry, and in running a container to au- tomatically extract its runtime features (e.g., the software distributions it support).
2. Storage: DockerFinder stores all produced image descriptions in a local repository.
3. Discovery: DokcerFinder allows users to search for images by submit multi-attribute queries thorugh a GUI or RESTful APIs (Search API, Software service API).

Getting started

Docker Finder can be deployed as a multi-container Docker application. The microservice-based architecture of Docker Finder is deployed as a multi-container Docker application (figure).

In order to deploy Docker Finder (locally) the requirements are the following:

• Docker engine >= 1.12
• **Docker Compose >= 1.9.0 **

Each service is shippend within a Docker image (represented as boxes) and the protocol communications are represented as dashed lines (e.g. HTTP, AMQP, mongodb).

Docker Compose (Single-host deployment)

Docker Finder can be runned locally as a multi-container Docker application using Docker Compose.

In order to run DockerFinder into your local host, copy, paste, and tun the following command.

$ git clone https://github.com/di-unipi-socc/DockerFinder.git && cd DockerFinder &&
docker-compose up -d

It starts all the services of DockerFinder into your local host 127.0.0.1.

Every service can be reached:

• GUI (port 80)
• Images API (port 3000)
• Software API (port 3001)
• RabbitMQ managment (port 8082)

In order to stop all the containers:

$ docker-compose stop

Docker Swarm - Multiple-host deployment

The requirements for deploying DockeFinder as a swarm are:

• docker >= 1.13
• docker-compose >= 1.10
• docker-machine >= 0.9
• Virtualbox > 5

DockerFinder is deployed in 3 VMs using virtualBox, where :

• swarm-manger is the Vm where the core services of Docker Finder are executed : crawler, rabbitmq,images_server, images_db, software_server, software_db, webapp.
• worker-1: is the VM where the scanner s are executed.
• worker-2: is the VM here the scanner s are executed.

The script init-all.sh:

• creates 3 virtualbox VMs with the docker-machine tool and initialize a swarm with the docker swarm command, composed by three nodes: - swarm-manager: is the manager of the swarm - worker-1: is the first worker - worker-2: is the second worker.

How to run

./init-all.sh

Enter in the swarm-manager machine:

eval $(docker-machine env swarm-manager)

Deploy the services:

docker stack deploy --compose-file=docker-compose.yml df

Monitor the services:

docker stack ps df

Stop the services:

docker stack rm df