Dysnomia is a tool and plug-in system that can be used to automatically deploy mutable components. It is primarily designed to be used in combination with Disnix for activation and deactivation of services, but it can also be used as a seperate utility.
When deploying software systems, most of its components are static or immutable, such as end-user programs, development tools, and servers. These components can be deployed (apart from their state files) on a read-only filesystem and never change after they have been built.
For example, the Nix package manager, which is used as a basis for local deployment in Disnix, achieves many of its quality attributes from immutability, such as reliable and reproducible deployment. Moreover, each time Nix deploys a new version or variant of a component it is stored next to an older version or variant. After a component has deployed, it is usually sufficient to launch it from the command-line or program launcher menu from the desktop.
However, to fully automate deployment procedures for certain kinds of systems, we also need to deploy components that cannot be managed in such a deployment model, such as databases and source code repositories, because it is too costly to store multiple generations of them next to each other.
Moreover, mutable components may also have to be activated (or deactivated) in so-called containers, such as application servers, managing the resources of an application. These procedures cannot be executed generically, as they are specific to the type of container that is used.
Mutable components are components with the following characteristics:
- Their state may change imperatively over time.
- They may have to be activated or deactivated in a container before they can be used. To do this, the state of a container must be modified.
- The have a physical state and logical representation of the state, which is typically a dump taken at a certain moment in a portable/consistent way.
Installation of Dysnomia is very straight forward by running the standard Autotools build procedure:
$ ./configure
$ make
$ make install
When building from the Git repository, you should run the bootstrap script first:
$ ./bootstrap
The Dysnomia package contains a collection of Dysnomia modules capable of
executing the deployment steps for certain types of mutable components. The
configure
script tries to automatically detect which ones the system is able
to support, by looking at the presence of certain command-line utilities and
files.
It is also possible to disable certain Dysnomia modules or to tune the configuration parameters. More information on this can be found by running:
$ ./configure --help
To deploy mutable components, Dysnomia uses a plugin system and delegates the reposibility of executing deployment activities for a certain component to Dysnomia modules.
Dysnomia provides the following modules that have no external service dependencies:
echo
. Mereley echos the parameters and environment variables used during activation or deactivation. Useful for debugging purposes.process
starts a process that daemonizes and terminates a process by using its corresponding PID filewrapper
delegates the execution of activities to thebin/wrapper
executable that is bundled with a componentfileset
. Deploys a directory on the filesystem that is populated with static/immutable and dynamic/mutable files.
To deploy running processes (e.g. system or application services, such microservices) you Dysnomia offers a number of plugins that use a variety of process managers to manage the lifecycle of a process:
bsdrc-script
activates or deactivates a BSD rc script.cygrunsrv-service
activates or deactivates a Windows service via cygrunsrvdocker-container
activates or deactivates a Docker container.launchd-daemon
activates or deactivates a launchd daemons6-rc-service
activates or deactivates a s6-rc service.supervisord-program
activates or deactivates a supervisord program configuration section.systemd-unit
activates or deactivates a systemd unitsysvinit-script
activates or deactivates a sysvinit script (also known as LSB Init compliant scripts)
Dysnomia will try to autodetect which process managers are available and will automatically install the appropriate plugins.
It is also possible to work with a process manager-agnostic configuration file and target any of the process managers listed above with one single configuration file:
managed-process
translates a process manager-agnostic JSON configuration to a desired process manament-specific configuration and activates or deactivates the process using a process manager listed above.
The Dysnomia package contains the following application-specific modules, that can be optionally enabled/disabled:
apache-webapplication
. Deploys a web application in a document root folder of the Apache HTTP server.axis2-webservice
. Deploys an Axis2 ARchive (AAR) file inside an Axis2 container.ejabberd-dump
. Deploys an Ejabberd configuration database.iis-webapplication
. Deploys a web application in a document root folder of the Internet Information Services (IIS) server.mongo-database
. Deploys a MongoDB database inside a MongoDB DBMS instance.mssql-database
. Deploys a database to a SQL Server DBMS instance.mysql-database
. Deploys a database to a MySQL DBMS instance.nginx-webapplication
. Deploys a web application in a document root folder of the Nginx web server.influx-database
. Deploys a timeseries database to a InfluxDB server instance.nixos-configuration
. Deploys a specific NixOS configuration.postgresql-database
. Deploys a database to a PostgreSQL DBMS instance.subversion-repository
. Deploys Subversion repository dump into a Subversion working directory.tomcat-webapplication
. Deploys a Java Web Application ARchive (WAR) file inside an Apache Tomcat servlet container.xinetd-service
. Deploys an xinetd service configuration part into axinetd.d
configuration directory.
systemd
units deployed by Dysnomia are wanted by the dysnomia.target
, if
this file exists. However, this target file is not created by default. You need
to do this yourself first. The following command typically suffices:
$ cat > /etc/systemd-mutable/system/dysnomia.target <<EOF
[Unit]
Description=Services that are activated and deactivated by Dysnomia
After=final.target
EOF
A dysnomia.target
file makes it possible to automatically start all Dysnomia
deployed services after a reboot.
In order to use Dysnomia to deploy mutable components, we require two kinds of dependencies:
- A component containing a logical snapshot of the initial state of a mutable component
- A configuration file capturing properties of the container in which the component must be deployed
The following file could be stored in ~/testdb/mysql-database/createdb.sql
representing the logical state of a MySQL database. In this particular case, this
file is a collection of SQL statements setting up the initial schema of the
database:
create table author
( AUTHOR_ID INTEGER NOT NULL,
FirstName VARCHAR(255) NOT NULL,
LastName VARCHAR(255) NOT NULL,
PRIMARY KEY(AUTHOR_ID)
);
create table books
( ISBN VARCHAR(255) NOT NULL,
Title VARCHAR(255) NOT NULL,
AUTHOR_ID INTEGER NOT NULL,
PRIMARY KEY(ISBN),
FOREIGN KEY(AUTHOR_ID) references author(AUTHOR_ID) on update cascade on delete cascade
);
The folder ~/testdb
represents a logical state dump that we can deploy through
a Dysnomia module.
Besides specifying the state of the database, we also need to know to which DBMS
instance (a.k.a. container) we have to deploy a component. The container
settings are captured in a separate container configuration file, such as
~/mysql-production
:
type=mysql-database
mysqlUsername=root
mysqlPassword=verysecret
The above file is a very simple textual configuration files consisting of
key=value pairs. The type
property is the only setting that is mandatory,
because it is used to invoke the corresponding Dysnomia module that takes care
of the deployment operations for that container. The remaining properties are
used by the particular Dysnomia module.
With those two files, we can perform a deployment activity, such as activating a MySQL database inside a MySQL DBMS instance:
$ dysnomia --operation activate --component ~/testdb --container ~/mysql-production
Every component has its own way of representing its logical state and each of them require different container settings. For databases, these are typically SQL dumps and authentication settings.
Web applications have archive files (WAR/AAR) or a collection of web related files (HTML, CSS etc.) as a representation of their logical state. Consult the actual Dysnomia modules for more information.
Dysnomia can also be used to manage snapshots of mutable components. Running the following operation captures the state of a deployed MySQL database:
$ dysnomia --operation snapshot --component ~/testdb --container ~/mysql-production
Restoring the last taken snapshot can be done by running:
$ dysnomia --operation restore --component ~/testdb --container ~/mysql-production
Snapshots taken by Dysnomia are stored in a so-called Dysnomia snapshot store
(stored by default in /var/state/dysnomia
, but can be changed by setting the
DYSNOMIA_STATEDIR
environment variable), a special purpose directory that
stores multiple generations of snapshots according to some naming convention
that uniquely identifies each snapshot.
The following command can be used to query all snapshots taken for the component
testdb
deployed to the MySQL container.
$ dysnomia-snapshots --query-all --container mysql-database --component testdb
mysql-production/testdb/9b0c3562b57dafd00e480c6b3a67d29146179775b67dfff5aa7a138b2699b241
mysql-production/testdb/1df326254d596dd31d9d9db30ea178d05eb220ae51d093a2cbffeaa13f45b21c
mysql-production/testdb/330232eda02b77c3629a4623b498855c168986e0a214ec44f38e7e0447a3f7ef
In most cases, only the latest snapshot is useful. The following query only shows the last generation snapshot:
$ dysnomia-snapshots --query-latest --container mysql-production --component testdb
mysql-production/testdb/330232eda02b77c3629a4623b498855c168986e0a214ec44f38e7e0447a3f7ef
The query operations show the relative paths of the snapshot directories so that their names are consistent among multiple machines. Their absolute paths can be resolved by running:
$ dysnomia-snapshots --resolve mysql-database/testdb/330232eda02b77c3629a4623b498855c168986e0a214ec44f38e7e0447a3f7ef
/var/state/dysnomia/snapshots/mysql-production/testdb/330232eda02b77c3629a4623b498855c168986e0a214ec44f38e7e0447a3f7ef
Every container type follows its own naming convention that uniquely identifies a
snapshot. For example, for MySQL databases a snapshot is identified by its
output hash, such as 9b0c3562b57dafd00e480c6b3a67d29146179775b67dfff5aa7a138b2699b241
.
Using a specific naming convention (e.g. computing an output hash) has all kinds of advantanges. For example, if we take a snapshot twice and they happen to be the same (which is reflected in the output hash), we only have to store the result once.
Not all component types use output hashes as a naming convention. For example, for Subversion repositories the revision number is used. Besides reducing storage redundancy this convention has another advantage -- when restoring a snapshot, we can first check whether the repository is at the right revision. There is no need to restore a snapshot if the revision number equals the revision number of a snapshot.
In addition to querying the available snapshots, it is also possible to check their integrity to detect whether they have been corrupted or not.
By running a query operation and adding the --check
parameter, the integrity
of the corresponding snapshots can be checked. For example, the following
command checks the integrity of all MySQL database snapshots in the store:
$ dysnomia-snapshots --query-all --check --container mysql-database
Apart from snapshotting and restoring the state of mutable components, it may also be desirable to delete state, such as removing a database.
To remove state, first a component must be deactivated:
$ dysnomia --operation deactivate --component ~/testdb --container ~/mysql-production
The above operation does not delete the database. Instead, it simply marks it as garbage, but otherwise keeps it. Actually deleting the database can be done by invoking the garbage collect operation:
$ dysnomia --operation collect-garbage --component ~/testdb --container ~/mysql-production
The above command first checks whether the database has been marked as garbage. If this is the case (because it has been deactivated) it is dropped. Otherwise, this command does nothing (because we do not want to delete stuff that is actually in use).
Dysnomia stores multiple generations of snapshots next to each other and also never automatically deletes them. Instead, it must be done explicitly by the user.
Clearing up older generation of snapshots can be done by invoking the garbage collect operation. The following command deletes all but the latest snapshot generation from the Dysnomia snapshots store:
$ dysnomia-snapshots --gc
The amount of snapshots that must be kept can be adjusted by providing the
--keep
parameter:
$ dysnomia-snapshots --gc --keep 3
The above command states that the last 3 generations of snapshots should be kept.
When incidents occur and it is desired to debug or execute arbitrary maintenance tasks, it can be somewhat annoying to manually configure all properties so that we connect to a component deployed to a container.
The Dysnomia shell can be used to spawn a session in which the environment variables are configured to contain the container's configuration properties:
$ dysnomia --shell --component ~/testdb --container ~/mysql-production
In addition to a shell session that contains a container configuration properties, a Dysnomia module also typically displays command-line tool suggestions to the user executing common housekeeping tasks.
Besides executing operations on individual mutable components, we can also
manage sets of containers (and their corresponding mutable components) in one go
through the dysnomia-containers
utility.
The following command shows all the available containers to deploy to:
$ dysnomia-containers --query-containers
mysql-database
postgresql-database
The above command searches for container configuration files in the directories
provided by the DYSNOMIA_CONTAINERS_PATH
environment variable (which defaults
to: /etc/dysnomia/containers
).
We can also display all the available mutable components:
$ dysnomia-containers --query-available-components
mysql-database/testdb
postgresql-database/testdb
The above command searches for component configuration files in the directories
provided by the DYSNOMIA_COMPONENTS_PATH
environment variable (which defaults
to: /etc/dysnomia/components
). Optionally, you can filter the output per
container by providing the --container
parameter.
The following command shows all components that have been activated in a container:
$ dysnomia-containers --query-activated-components
The most useful operation is probably the deploy function:
$ dysnomia-containers --deploy
The above command will automatically deploy all available mutable components that have not been activated yet, and will undeploy all activated components that are not available anymore. This command automates the deactivation and activation steps of collections of components.
We can also snapshot the state of all activated components:
$ dysnomia-containers --snapshot
and restore the state of them:
$ dysnomia-containers --restore
The following command removes the state of all components that have been marked as garbage:
$ dysnomia-containers --collect-garbage
We can also directly execute any Dysnomia operation on all activated components:
$ dysnomia-containers --operation snapshot
It is also possible to generate a Nix expression capturing the properties of all the container configurations, by running:
$ dysnomia-containers --generate-expr
The above command shows a generated Nix expression that may look as follows:
{
properties = {
"hostname" = "test1";
"mem" = "1023096";
"supportedTypes" = [
"mysql-database"
"process"
"tomcat-webapplication"
];
"system" = "x86_64-linux";
};
containers = {
mysql-database = {
"mysqlPassword" = "admin";
"mysqlPort" = "3306";
"mysqlUsername" = "root";
};
tomcat-webapplication = {
"tomcatPort" = "8080";
};
};
}
The generated expression is an attribute set exposing two attributes. The
containers
attribute is composed of all container configuration files in the
DYSNOMIA_CONTAINERS_PATH
environment variable.
The properties
attribute contains non-functional machine-level properties
that can be freely chosen. These are takes from the /etc/dysnomia/properties
configuration file or the file to which the DYSNOMIA_PROPERTIES
environment
variable refers.
For example, the above machine properties are generated from the following configuration file:
hostname="$(hostname)"
mem=$(grep 'MemTotal:' /proc/meminfo | sed -e 's/kB//' -e 's/MemTotal://' -e 's/ //g')
supportedTypes=("mysql-database" "process" "tomcat-webapplication")
system="x86_64-linux"
The Nix expression output generated by dysnomia-containers --generate-expr
makes it convenient to integrate Dysnomia with various Nix-driven utilities,
such as disnix-capture-infra
(part of Disnix) and the
Dynamic Disnix Avahi publisher.
The Dysnomia toolset also contains utilities to declaratively create users and groups on a variety of operating systems from the same declarative specification.
The following configuration file can be added to a Dysnomia component configuration, to specify that a group needs to be created:
$ mkdir -p mycomponent/dysnomia-support/groups
$ cat > mycomponent/dysnomia-support/mygroup <<EOF
gid=2000
EOF
The above specification states that we should create a group named: mygroup
with GID: 2000
. The gid
property is optional.
With the following command the group gets created:
$ dysnomia-addgroups mycomponent
we can delete the group with:
$ dysnomia-delgroups mycomponent
Similarly, we can create users with a user configuration file, such as:
$ mkdir -p mycomponent/dysnomia-support/users
$ cat > mycomponent/dysnomia-support/myuser <<EOF
uid=2000
password="secret"
group="mygroup"
description="My user"
homeDir="/home/myuser"
createHomeDir=1
shell="/bin/sh"
EOF
The above configuration specifies the following properties:
- The name of the user is:
myuser
- The UID of the user is:
2000
(optional) - The password of the user is:
secret
(optional). Passwords only need to be configured for interactive users. It is also possible to use command substitution (e.g.$(cat /etc/mysecret)
) to separate the secrets from the configuration file. - The description of the user (optional)
- The home directory of the user (optional). Default value is
/dev/null
. - Whether the home directory should be created or not (
createHomeDir
) - The shell that the user should use (
/bin/sh
), optional.
With the following command, we can create the user:
$ dysnomia-addusers mycomponent
and remove it as follows:
$ dysnomia-delusers mycomponent
In some deployment scenarios, you may want to flexibly switch to a deployment in which user accounts get created (for production deployments) and a deployment in which everything runs as an unprivileged user (for testing).
For services that use home directories as state directories, this means that extra work needs to be done, because these are only created when a user is created.
It is also possible to only create a home directory from a user specification:
$ mkdir -p mycomponent/dysnomia-support/users
$ cat > mycomponent/dysnomia-support/myuser <<EOF
homeDir="/home/myuser"
createHomeDir=1
createHomeDirOnly=1
EOF
and running:
$ dysnomia-addusers mycomponent
In the above example configuration file, the createHomeDirOnly
property
specifies that only the home directory should be created, but not the user
itself.
In most usage scenarios, you never directly use the Dysnomia user management
or group management tools -- they are typically indirectly used by a variety
of Dysnomia modules. Most notably, the process management modules, e.g.
process
, managed-process
, sysvinit-script
, systemd-unit
etc.
automatically invoke these utilities when the above configurations are included
so that any managed executable can conveniently run as an unprivileged user.
In addition to Disnix, it is also possible to use Dysnomia on NixOS-level to automatically manage mutable components belonging to a system configuration:
{pkgs, ...}:
{
# Import the Dysnomia NixOS module to make its functionality available
imports = [ ./dysnomia-module.nix ];
services = {
# Enabling MySQL in the NixOS configuration implies creating a Dysnomia
# container configuration file for it
mysql = {
enable = true;
package = pkgs.mysql;
rootPassword = pkgs.writeTextFile {
name = "mysqlpw";
text = "verysecret";
};
};
# Enabling PostgreSQL in the NixOS configuration implies creating a
# Dysnomia container configuration file for it
postgresql = {
enable = true;
package = pkgs.postgresql;
};
dysnomia = {
enable = true;
# Here, we deploy databases to the corresponding DBMSes with Dysnomia
components = {
mysql-database = {
testdb = pkgs.writeTextFile {
name = "testdb";
text = ''
create table author
( AUTHOR_ID INTEGER NOT NULL,
FirstName VARCHAR(255) NOT NULL,
LastName VARCHAR(255) NOT NULL,
PRIMARY KEY(AUTHOR_ID)
);
'';
};
};
postgresql-database = {
testdb = pkgs.writeTextFile {
name = "testdb";
text = ''
create table author
( AUTHOR_ID INTEGER NOT NULL,
FirstName VARCHAR(255) NOT NULL,
LastName VARCHAR(255) NOT NULL,
PRIMARY KEY(AUTHOR_ID)
);
'';
};
};
};
};
...
}
The above code block shows an example NixOS configuration, in which we do the following:
- We import the Dysnomia module from the source package to make its features available.
- We enable the Dysnomia NixOS service
- We enable some system services, such as MySQL and PostgreSQL. The Dysnomia
NixOS module automatically generates Dysnomia container configuration files
for them (and puts them in
/etc/dysnomia/containers
of the corresponding NixOS deployment) - We define the available mutable components. In this particular example, a
MySQL database named
testdb
and PostgreSQL database namedtestdb
which both have one table named:author
are created.
After deploying the NixOS configuration with the following command-line instruction:
$ nixos-rebuild switch
We can deploy the mutable components as follows:
$ dysnomia-containers --deploy
And (for example) snapshot the state of the mutable components as follows:
$ dysnomia-containers --snapshot
To prevent the state of the mutable components to conflict with those deployed
by Disnix, the Dysnomia module sets DYSNOMIA_STATEDIR
to
/var/state/dysnomia-nixos
so that they are managed separately.
Custom Dysnomia modules are relatively easy to implement. Every Dysnomia module is a process in which the first command-line parameter represents the activity to execute and the second parameter represents the path to a component containing a logical state snapshot. The container properties are made available through environment variables.
The following code fragment shows the source code of the echo
module, that
simply echoes what it is doing:
#!/bin/bash
set -e
set -o pipefail
# Dysnomia module that simply echos the activity that is being executed.
case "$1" in
# Executes all steps necessary to activate a service. It returns a zero
# exit status in case of success.
activate)
echo "Echo module: Activate service: $2"
;;
# Executes all steps necessary to deactivate a service. It returns a zero
# exit status in case of success.
deactivate)
echo "Echo module: Deactivate service: $2"
;;
# Notifies a service that an upgrade is performed. A service can use this to
# take precautions or to reach quiescence. It can also reject the upgrade by
# returning a non-zero exit status.
lock)
echo "Echo module: Lock service: $2"
;;
# Notifies a service that an upgrade has finished. A service can use this
# to resume its normal operations.
unlock)
echo "Echo module: Unlock service: $2"
;;
# Snapshots the corresponding state of the service in a preferably consistent
# and portable manner in a special purpose folder with a naming strategy.
snapshot)
echo "Echo module: Snapshot state of service: $2"
;;
# Restores the state of the service from the special purpose folder with a
# naming strategy.
restore)
echo "Echo module: Restore state of service: $2"
;;
# Collects the garbage of the service by permanently removing it
collect-garbage)
echo "Echo module: Collect garbage of service: $2"
;;
# Script that gets executed when spawning a shell session. It is typically
# used to provide usage instructions to the user.
shell)
echo "This is the echo shell session"
;;
esac
# Print the environment variables
echo "Environment variables:"
set
Currently, Dysnomia supports the following types of operations:
activate
is used to activate the component in a container.deactivate
is used to deactivate the component in a container.lock
is invoked by Disnix before the upgrade transition starts. This operation can be used to consult a deployed component to determine whether it is safe to upgrade and to take precautions before the upgrade starts (such as queing incoming connections).unlock
is invoked by Disnix after the upgrade transition is over. This can be used to notify the component to resume its normal operations.snapshot
is used to snapshot the logical state of a component in a container. This operation is optionally executed by Disnix to move data from one machine to another.restore
is used to restore the logical state of a component in a container. This operation is optionally executed by Disnix to move data from one machine to another.collect-garbage
is used to remove the state of a component in a container.
The above code examples are written in bash, but any lanugage can be used as long as the tool provides the same command-line interface and properly uses the environment variables from the container specification.
The implementation of each operation is completely the responsible of the implementer. However, for mutable components with persistent state, such as databases, we typically follow a convention for many of the operations:
#!/bin/bash
set -e
set -o pipefail
# Autoconf settings
export prefix=@prefix@
# Import utility functions
source @datadir@/@PACKAGE@/util
# Sets a number of common utility environment variables
composeUtilityVariables $0 $2 $3
case "$1" in
activate)
# Initalize the given schema if the database does not exists
if ! exampleStateInitialized
then
exampleInitializeState
fi
markComponentAsActive
;;
deactivate)
markComponentAsGarbage
;;
snapshot)
# Dump the state of the component in a temp dir
tmpdir=$(mktemp -d)
cd $tmpdir
exampleSnapshotState | xz > dump.xz
# Compose a unique name for the snapshot
hash=$(cat dump.xz | sha256sum)
hash=${hash:0:64}
snapshotsPath=$(composeSnapshotsPath)
if [ -d $snapshotsPath/$hash ]
then
# If the snapshot exists in the store already, discard it
rm -Rf $tmpdir
else
# Import the snapshot into the snapshot store
mkdir -p $snapshotsPath/$hash
mv dump.xz $snapshotsPath/$hash
rmdir $tmpdir
fi
# Create a generation symlink for the snapshot
createGenerationSymlink $hash
;;
restore)
lastSnapshot=$(determineLastSnapshot)
if [ "$lastSnapshot" != "" ]
then
exampleRestoreState $lastSnapshot
fi
;;
collect-garbage)
if componentMarkedAsGarbage
then
exampleDeleteState
unmarkComponentAsGarbage
fi
;;
shell)
cat >&2 <<EOF
This is a shell session that can be used to control the '$componentName' database.
EOF
;;
esac
The above code fragment outlines an example module implementing deployment operations of a database:
activate
: The activate operation checks whether the database exists in the DBMS. If the database does not exists, it gets created and an initial static dump (typically a schema) is imported. It also marks the database as active so that it will not be removed by the garbage collector.deactivate
: Marks the mutable component (database) as garbage so that it will be removed by the garbage collector.snapshot
: Snapshots the database and composes generation symlink determining the order of the snapshots. As an optimisation, the module also tries to store a snapshot only once. If it has been taken once before, the earlier result is reused. To make the optimisation work, a naming convention must be chosen. In the above example, the output hash of the snapshot is used.restore
: Determines the last generation snapshot and restores it. If no snapshot is in the store, it does nothing.collect-garbage
: Checks if the component is not deployed to a container and deletes it if this the case. Otherwise, it does nothing.
Dynomia includes a set of utility functions to make implementing these operations more convenient.
Each module takes its own container and component configuration properties. Both
are exposed as environment variables. Consult the documentation inside the
modules (stored in the dysnomia-modules/
sub folder of this package) for more
information.
This package is released under the MIT license.