Skip to content
This repository has been archived by the owner on Oct 15, 2024. It is now read-only.

0.9.12 Release
Compare
Choose a tag to compare
@mpranj mpranj released this 03 Mar 13:05
· 961 commits to master since this release
0.9.12
45b0eb9
  • guid: F2193578-1773-43A9-85CA-79EA8CE48D7B
  • author: Mihael Pranjić
  • pubDate: Fri, 03 Mar 2023 08:07:28 +0100
  • shortDesc: New Backend Logic, Copy-on-Write, FLOSS Course

We are proud to release Elektra 0.9.12.

What is Elektra?

Elektra serves as a universal and secure framework to access configuration settings in a global, hierarchical key database.
For more information, visit https://libelektra.org.

You can also read the news on our website.

You can try out the latest Elektra release using our docker image elektra/elektra.
This is the quickest way to get started with Elektra without compiling and other obstacles, simply run:

docker pull elektra/elektra
docker run -it elektra/elektra

Highlights

  • New Backend
  • Copy-on-Write
  • FLOSS

New Backend

The entire logic for backends has been rewritten, to allow for more flexibility und an unlimited number of plugins.
Instead of calling plugins directly, libelektra-kdb now only calls so-called backend plugins and special hook plugins.
There is a contract between libelektra-kdb and the backend plugins.
All backend plugins must adhere to this contract.
To achieve this goal, most backend plugins will call other plugins (like libelektra-kdb did previously).

The logic previously implemented in libelektra-kdb was moved to the new default backend plugin backend.
It works like the old system, but now also allows an unlimited number of plugins in positions where that makes sense.
For example, you can have unlimited postgetstorage plugins, but only a single getresolver.

There have also been slight changes to kdbGet and kdbSet.
Please read their API docs to find out, if you rely on any behavior that has been altered.
You can also read the new low-level docs to find out all the intricate details.

The structure of system:/elektra/mountpoints changed as well.
Take a look at the new docs, if you need to know details.

Updating config

The mountpoint configuration format contains breaking changes and a manual upgrade process is needed.
Follow these steps to upgrade the old mountpoint configuration to the new format:

Warning: BACK UP YOUR CONFIG FILES BEFORE UPDATING!
We recommend making a backup of the file printed by kdb file system:/elektra/mountpoints before updating your installation.
In the unlikely case that the migration script fails, you can still use the information from the backup to manually recreate your mountpoints.

To update your existing system:/elektra/mountpoints data you can use the migration script.

Note: To run the script you must have Elektra, Python (>= 3.7) and the Python binding installed.
The script uses the Python binding to manipulate Keys and KeySets, but it does not use the kdb CLI tool, or the KDB API.
It is safe to run this script before, or after you update your Elektra installation.

By default, the script loads the file /etc/kdb/elektra.ecf.
If you changed where system:/elektra/mountpoints is stored, you can provide an alternative path:

./migrate-mountpoints.py /path/to/your/mountpoints/config.file

Note: Because the script does not use the KDB API it only works, if the mountpoints config file uses the default dump format.

If your mountpoints config file is not using the dump format, you may still be able to use the migration script.
However, in that case, you will have to use the script before updating your Elektra installation:

  1. Run kdb export system:/elektra/mountpoints dump to get a copy of your mountpoints config in dump format
  2. Write this data to a file and run the migration script on the file.
  3. To get the data back in your original format you can use
./migrate-mountpoints.py /path/to/file/from/step2 | kdb convert dump your-format > /path/to/converted/file
  1. Run kdb file system:/elektra/mountpoints to find out where your mountpoint config is stored.
    Make sure to back up this file, before upgrading your installation.
  2. Now upgrade your Elektra installation.
  3. Copy the file /path/to/converted/file from step 3 to the location you got in step 4.

The script will read the old mountpoint configuration from the given file.
It will convert the configuration and print the new version to stdout.

You can inspect the output to make sure, everything is in order.
When you are ready to commit the changes, you can manually edit the config file, or use:

./migrate-mountpoints.py --output=/etc/kdb/elektra.ecf /etc/kdb/elektra.ecf

Individual changes

Copy-on-Write

Thanks to (Maximilian Irlinger @atmaxinger) our Key and KeySet datastructures are now fully copy-on-write!
This means noticeably reduced memory usage for cases where keys and keysets are copied and/or duplicated!

We ran some very promising benchmarks, each were performed with 400,000 keys.
All benchmarks were executed using valgrind --tool=massif --time-unit=B --max-snapshots=200 --threshold=0.1.

Benchmark Old Implementation Copy-on-Write Size Reduction Remarks
createkeys.c 5.3 MiB 6.5 MiB -22 %
deepdup.c 10.5 MiB 8.2 MiB 22 %
large.c 18.9 MiB 15.3 MiB 19 %
kdb.c 23.5 MiB 17.8 MiB 24 %
kdbget.c 11.0 MiB 8.8 MiB 20 %
kdbmodify.c 11.0 MiB 8.8 MiB 20 % Same results as kdbget.c

First, it should be noted that a single key, without counting payload, is about 50% larger with the copy-on-write implementation.
This explains why the createkeys.c benchmark yields a negative reduction result.
This benchmark only allocates keys, so not much improvement can be expected there.
Still, as other stuff also uses heap memory, the overall memory consumption only increased by 22%, which is far less than 50%.

All other benchmarks saw meaningful reductions of heap memory used.
One interesting observation is that kdbget.c and kdbmodify.c used exactly the same memory.
This can most likely be explained by internal caching within the memory allocator of glibc.

We also performed runtime tests on the same benchmarks using perf stat --repeat 13 to ensure no major performance regressions occur.

Benchmark Old Implementation Deviation Copy-on-Write Deviation Runtime Increase
createkeys.c 0.209572 s 0.36 % 0.21987 s 0.77 % 4.9 %
deepdup.c 0.23025 s 0.47 % 0.231804 s 0.32 % 0.6 %
large.c 1.14038 s 0.21 % 1.14837 s 0.21 % 0.7 %
kdb.c 1.9270 s 2.63 % 1.93354 s 0.17 % 0.3 %
kdbget.c 0.145663 s 0.17 % 0.15763 s 0.70 % 8.2 %
kdbmodify.c 0.146506 s 0.19 % 0.156347 s 0.15 % 6.7 %

Overall, the runtime performance hit is less than 10%.
The more a program does, the less the additional overhead of the copy-on-write algorithms matter.
One interesting detail is that keyCopy and keyDup have become quite a bit faster.
This can be seen by comparing the differences between createkeys.c and deepdup.c.
The differences are 21 ms for the old implementation and 12 ms for the copy-on-write implementation.

FLOSS

A Free/Libre and Open Source Software (FLOSS) Initiative couldn't survive with many small contributions fixing annoying problems.
This release also contains all contributions done via one term of the FLOSS course.
The success was tremendous, as shown in the rest of the release notes.

A big thanks to the students for their contributions!

Plugins

The following text lists news about the plugins we updated in this release.

yajl

  • Fix an issue where trying to set invalid meta-keys won't show an error. (Juri Schreib @Bujuhu)

list

  • Removed the outdated list plugin. (Maximilian Irlinger @atmaxinger)
    (Was only needed for global plugins, which are now replaced by hooks.)

logchange

  • Made logchange a notification-send hook plugin. (Maximilian Irlinger @atmaxinger)

toml

  • Fix bug, where meta-keys that cannot be inserted don't report an error. (@Bujuhu)

uname

  • Add error handling if uname call fails. (Richard Stöckl @eiskasten)

quickdump

  • elektraQuickdumpSet: don't fclose if stdout. (@hannes99)

blockresolver

  • Add encoding test for blockresolver read. (@dtdirect)
  • Refactor and restructure blockresolver. (@dtdirect)

desktop

mini

  • Fix a bug where writing meta-keys will fail silently. (Juri Schreib @Bujuhu)

mmapstorage

  • Remove code duplication in the data block calculation. (Richard Stöckl @eiskasten)

network

  • Add a retry mechanism. (Richard Stöckl @eiskasten)

xfconf

  • Add xfconf storage plugin with the ability to read and write to xfconf channels. (Richard Stöckl @eiskasten)
  • Make xfconf valgrind suppressions more flexible to lib updates. (Mihael Pranjić @mpranj)

date

csvstorage

  • Fix a bug where writing unkown meta-keys will fail silently. (Juri Schreib @Bujuhu)

Libraries

The text below summarizes updates to the C (and C++)-based libraries of Elektra.

Compatibility

  • Global plugins do not work anymore, use hooks instead.

Core

  • The Key and KeySet datastructures are now fully copy-on-write. (Maximilian Irlinger @atmaxinger)
  • keyCopy now only allocates additional memory if KEY_CP_META or KEY_CP_ALL is used. (Maximilian Irlinger @atmaxinger)
  • Check for circular links (overrides). (@0x6178656c)

io

  • Check file flags for elektraIoFdSetFlags: file flags must be exactly one of: read only, write only or read write. (Richard Stöckl @eiskasten)

Merge

  • Add methods elektraMergeGetConflictingKeys and elektraMergeIsKeyConflicting to check which keys were causing a merge conflict. (Maximilian Irlinger @atmaxinger)

Bindings

Bindings allow you to utilize Elektra using various programming languages.
This section keeps you up-to-date with the multi-language support provided by Elektra.

intercept/env

  • Remove fallback code. (Markus Raab)
  • Command-line functionality is broken due to new-backend differences.

intercept/fs

  • Use KDB_MAX_PATH_LENGTH for better portability. (Markus Raab)

jna

  • Documentation: Improved build instructions. (@Bujuhu)
  • Add validation on get for whitelist plugin. (@Bujuhu)
  • Upgrade Gradle to 8.0.1. (Mihael Pranjić @mpranj)

rust

  • Fix "feature resolver is required" error. (Markus Raab)

elixir

Tools

kdb

  • Removed global-mount and global-umount commands. (Maximilian Irlinger @atmaxinger)
  • Fixed SIGSEGV when using kdb find without argument. (Christian Jonak-Moechel @joni1993)

elektrad

  • Removed leftover package-lock.json file. (stefnotch)

Scripts

Documentation

  • Restructured contrib/api. (Markus Raab).
  • Improve page on compilation. (@0x6178656c)
  • Improve page for bindings. (@0x6178656c)
  • Improve page for getting started. (@stefnotch)
  • Remove version number from docker README and replace it with latest. (@joni1993)
  • Fix grammar for elektra-granularity.md. (@dtdirect)
  • Rephrase sections in doc/dev/error-\*. (@dtdirect)
  • Improve Git.md. (Juri Schreib @Bujuhu) (Nikola Prvulovic @Dynamichost96)
  • Unify spelling of man pages. (@stefnotch) (@Janldeboer)
  • Extend consistency check check_doc.sh to work for contrib, dev and tutorials. (@joni1993)
  • Fix internal links. (@0x6178656c)
  • Update AUR Link from elektra to libelektra package. (@Bujuhu)
  • Update example Ansible playbook in VISION.md. (@Bujuhu)
  • Harmonize spelling of Git. (@joni1993)
  • Update packaging instructions for Fedora. (@0x6178656c)
  • Improve use of gender. (@0x6178656c)
  • Fix some minor mistakes in CONTRIBUTING.md. (@joni1993)
  • Fix various spelling errors. (@joni1993)
  • Denoted package names & global variable names in INSTALL.md as Code. (@Janldeboer)
  • Improve readability of doc/tutorials/highlevel.md. (@deoknats861)
  • Improve reference to Podman documentation. (@0x6178656c)
  • Unify spelling. (@joni1993)
  • Fix typo in dev/hooks.md. (@dtdirect)
  • Remove unused images from doc/images. (@dtdirect)
  • Fixed Coverage Badge Link. (@Janldeboer)
  • Improve CONTRIBUTING doc. (Juri Schreib @Bujuhu) and (Nikola Prvulovic @Dynamichost96)
  • Update Doxyfile with Doxygen 1.9.4. (@0x6178656c)
  • Add project logo to Doxygen in Doxyfile. (@dtdirect)
  • Add mermaid.js to the project using doxygen-mermaid. (@dtdirect)
  • Create diagrams in mermaid.js to use in doxygen. (@dtdirect)
  • Create README for Doxygen and Mermaid JS. (@dtdirect)
  • Tutorial: Add automatic validation to Docker tutorial (Schreib @Bujuhu) (Nikola Prvulovic @Dynamichost96)
  • Add mention of audit-dependencies script in doc/todo/RELEASE.md. (@Bujuhu)
  • Move note in GETSTARTED.md. (@joni1993)
  • Use code blocks to prevent Markdown from falsy rendering LaTeX. (@stefnotch), (@Janldeboer)
  • Fix broken links in use cases for KDB after files were renamed. (Florian Lindner @flo91)
  • Replace http links with https. (Richard Stöckl @eiskasten)
  • Enhance notifications.md in doc/tutorial. (@dtdirect)
  • Add tutorial how to suppress memleaks in plugins from dependencies. (Richard Stöckl @eiskasten)
  • Write about history to make plans of Elektra's adoption more clear. (Markus Raab)

Use Cases

Decisions

Tutorials

  • Add tutorial for manual installation from the AUR on Arch Linux. (@Bujuhu)
  • Add Markdown shell recorder validation to install.webui.md. (@deoknats861)
  • Fix the outdated array tutorial. (Juri Schreib @Bujuhu) (Nikola Prvulovic @Dynamichost96)
  • Reinstate mounting tutorial. (@Bujuhu)
  • Make namespaces tutorial verifiable. (@0x6178656c)
  • Move Podman-related information to a dedicated page. (@0x6178656c)

Man Pages

  • Update man page (patch) as suggested by the CI to fix CI error on master. (Florian Lindner @flo91)
  • Added links to the website & webui after further reading. (Philipp Nirnberger @Nirnberger)
  • Upgrade ronn-ng to 0.10.1.pre3. (Mihael Pranjić @mpranj)

Tests

Shell Recorder

Packaging

  • Add missing new backend plugin to components of libelektra package. (Mihael Pranjić @mpranj)

Build

CMake

  • Fix warning for CMP0115. (0x6178656c)
  • Change Doxygen configuration for LaTeX. (0x6178656c)
  • Fix developer warning for package DISCOUNT. (Dennis Toth @dtdirect)
  • Pass --stacktrace to gradle for the JNA builds. (Maximilian Irlinger @atmaxinger)
  • Adapt npm build flags to remove reproducability issues. (Juri Schreib @Bujuhu) (Nikola Prvulovic @Dynamichost96)
  • Fix creation of shell recorder tests. (@0x6178656c)

Docker

  • Update packagename libpcrec++-dev to libpcrecpp0v5 in Debian Sid. (Richard Stöckl @eiskasten)
  • Add shellcheck to Debian containers. (@0x6178656c)
  • Use openjdk-17-jdk in Debian Sid. (Maximilian Irlinger @atmaxinger)
  • Add Fedora 37 images. (Mihael Pranjić @mpranj)
  • Update Debian Sid image to use repository Python modules instead of installing with pip3 due to upstream debian changes. (Mihael Pranjić @mpranj)
  • Debian Bullsye: use clang 13. (Mihael Pranjić @mpranj)
  • Update Alpine Linux to 3.17.2. (Mihael Pranjić @mpranj)

Gradle

  • Use Gradle 7.5.1. (Mihael Pranjić @mpranj)
  • Update java-library.gradle to use archiveClassifier (Maximilian Irlinger @atmaxinger)

Infrastructure

Jenkins

  • Add Fedora 37 builds, drop Fedora 35 builds. (Mihael Pranjić @mpranj)
  • Run more tests also on Master. (Markus Raab)
  • Move doc to main build stage. (Markus Raab)
  • Disable parallel test runs. (Maximilian Irlinger @atmaxinger)
  • Upgrade Jenkins node container to Debian bullseye. (@0x6178656c)
  • Undo previous change that added automatic ctest --rerun-failed to Jenkins CI. (@kodebach)

Cirrus

  • Use Fedora 37. (Mihael Pranjić @mpranj)
  • Fix macos_instance reference, upgrade to macOS Ventura (by default), use Python 3.11 and Ruby 3.x. (Mihael Pranjić @mpranj)
  • Automatically rerun testmod_dbus* tests on macOS. (@kodebach)
  • Fix dbus not starting on macOS. (Maximilian Irlinger @atmaxinger)

GitHub Actions

  • Add auto-cancellation-running action. (Tomislav Makar @tmakar)
  • Automatically rerun testmod_dbus* tests on macOS. (@kodebach)
  • Fix dbus not starting on macOS. (Maximilian Irlinger @atmaxinger)
  • Change stale issue/PR checking to GitHub action. (@0x6178656c)
  • Update configuration of stale issue/PR action. (@0x6178656c)
  • Upgrade actions to recent versions and remove deprecated ruby-setup action. (Mihael Pranjić @mpranj)

Website

The website is generated from the repository, so all information about plugins, bindings and tools are always up-to-date. Furthermore, we changed:

Outlook

We are currently working on following topics:

  • 1.0 API (Klemens Böswirth @kodebach) and (Stefan Hanreich)
  • Session recording and better Ansible integration (Maximilian Irlinger @atmaxinger)
  • Change tracking (Maximilian Irlinger @atmaxinger)
  • Rewriting tools in C (@hannes99)
  • Elektrify KDE and GNOME (Mihael Pranjić @mpranj)
  • Elektrify XFCE (Richard Stöckl @eiskasten)
  • Mounting SQL databases (Florian Lindner @flo91)
  • Recording Configuration (Maximilian Irlinger)
  • Ansible-Elektra (Lukas Hartl) and (Maximilian Irlinger)
  • Configure Olimex Base Images (Maximilian Irlinger)
  • Improving Build Server Infrastructure (Lukas Hartl) and (Maximilian Irlinger)
  • Improve Java Development Experience (Michael Tucek)

Statistics

We closed about 150 issues for this release.

About 28 authors changed 960 files with 29400 insertions(+) and 20927 deletions(-) in 1421 commits.

Thanks to all authors for making this release possible!

Join the Initiative!

We welcome new contributors!
Read here about how to get started.

As first step, you could give us feedback about these release notes.
Contact us via our issue tracker.

Get the Release!

You can download the release from

The hashsums are:

  • name: elektra-0.9.12.tar.gz
  • size: 9297913
  • md5sum: a6de9401709283b69ec211681f2a7757
  • sha1: cb4e282d1346fda771de7510663652555f8e6c7d
  • sha256: 38238ba4a5318f999dc3045da06467abf529344dc46ad3fdf42bdca0155e149c

The release tarball is also available signed using GnuPG from

The following GPG Key was used to sign this release: 12CC44541E1B8AD9B66AFAD55262E7353324914A

Already built API documentation can be found

Stay tuned!

Subscribe to the RSS feed to always get the release notifications.

If you also want to participate, or for any questions and comments, please contact us via our issue tracker on GitHub.

Permalink to this NEWS entry

For more information, see https://libelektra.org.

Best regards,
Elektra Initiative

Contributors

joni1993, Nirnberger, and 14 other contributors
Assets 5
Loading