Skip to content

Commit

Permalink
docs: add and enhance documentation for gui, setup and usage (#34)
Browse files Browse the repository at this point in the history
  • Loading branch information
tepene authored May 10, 2024
1 parent 4f2130b commit 4df5d16
Show file tree
Hide file tree
Showing 8 changed files with 224 additions and 78 deletions.
70 changes: 35 additions & 35 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Universal Blue - Forge

On-premises Universal Blue. This repository is intended to provide the service units
On-premises Universal Blue. This projects intended is to provide the service units
necessary to set up a self-hosted OS forge for custom images.

> **Warning**
Expand Down Expand Up @@ -30,65 +30,65 @@ then made available to all other components and are valid for 2 years and 30 day

### Reverse Proxy

As an entry point for all components we use [Traefik](https://doc.traefik.io/traefik/) as
a reverse proxy. Based on URL routing it will redirect the traffic to the
As an entry point for all web components we use [Traefik](https://doc.traefik.io/traefik/)
as a reverse proxy. Based on URL routing it will redirect the traffic to the
right container instance.

The reverse proxy dashboard is available at <https://traefik.ublue.local>

### Container Registry

As container registry we make use of the [Docker Registry 2.0](https://hub.docker.com/_/registry/)
implementation for storing and distributing container images
implementation for storing and distributing container images.

The container registry API is available at <https://registry.ublue.local/v2>

### Anvil

The blacksmith's work is done with [Ansible](https://docs.ansible.com/ansible/latest/index.html).

The shiny GUI is missing but this should not shy us away. See [usage](#usage) for instructions.
There are two methods of operating the forge, either via a [GUI](./docs/gui.md) available
at <https://forge.ublue.local> or via [just](./docs/just.md) command runner.

## Handling the forge
Details about the project and usage instructions are available in the [documentation](./docs/index.md)
section.

You can use the `forge.sh` to **setup**, **heat-up** and **cool-down** the forge.
## Installation

<!-- markdownlint-disable MD013 -->
### Pre-requisites

| Command | Description |
| ---------------------- | ----------------------------------------------------------- |
| `./forge.sh setup` | Setup the forge for the first time or update existing setup |
| `./forge.sh heat-up` | Start the forge |
| `./forge.sh cool-down` | Stop the forge |
As many tools as possible are built-in but still we rely on some pre-requisites.
These tools and service are necessary to get started:

<!-- markdownlint-enable MD013 -->
- [Podman](https://podman.io/)
Must be installed and a [podman socket](https://github.com/containers/podman/blob/main/docs/tutorials/socket_activation.md)
in the user space must be active.

### Usage
- [jq](https://jqlang.github.io/jq/)
Must be installed. It it currently needed in the setup process to parse certain parameters
automatically for you

Once the forge has been setup the following recipes are available via [just command runner](https://github.com/casey/just).
- The kernel parameter `net.ipv4.ip_unprivileged_port_start` must be set to `80`
This is because of podman's [shortcoming](https://github.com/containers/podman/blob/main/rootless.md#shortcomings-of-rootless-podman)
to bind to ports `< 1024`. Our reverse-proxy is listening on port `80` and `443`
for incoming traffic.

<!-- markdownlint-disable MD013 -->
- [OpenSSH](https://www.openssh.com/)
Must be installed and the service activated. Ansible needs this to execute all the fancy
commands on your host for you.

| Just recipe | Input argument | Default argument value | Description |
| --------------------- | ----------------------- | ------------------------------------------- | -------------------------------------------- |
| `forge_project-clone` | `forge_config_var_file` | $HOME/ublue-os_forge/forge_default_vars.env | Clone git project repository |
| `forge_project-build` | `forge_config_var_file` | $HOME/ublue-os_forge/forge_default_vars.env | Build container image and upload to registry |
### Setup / Heat-Up / Cool-Down

<!-- markdownlint-enable MD013 -->
For the initial setup and maintenance of the forge you can use the [forge.sh](forge.sh) script:

All available settings for the `forge_config_var_file` are documented in the [variables.md](./docs/variables.md)
file. To launch a recipe you simple run:

```sh
just -f forge.just {{ recipe_name }} {{ forge_config_var_file }}
```
<!-- markdownlint-disable MD013 -->

**_Example:_**
| Command | Description |
| ---------------------- | ----------------------------------------------------------- |
| `./forge.sh setup` | Setup the forge for the first time or update existing setup |
| `./forge.sh heat-up` | Start the forge |
| `./forge.sh cool-down` | Stop the forge |

```sh
just -f forge.just forge_project-clone ~/ublue-os_forge/my-forge-project.env
```
<!-- markdownlint-enable MD013 -->

In case you don't have [just command runner](https://github.com/casey/just) available.
Have a look at the [forge.just](./forge.just) file. It easy enough to understand which commands
are executed via the just recipes.
Details about what the setup does can be found [here](./docs/setup.md).
Binary file added docs/assets/gui_ansible.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/assets/gui_navigation.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
36 changes: 36 additions & 0 deletions docs/gui.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
# Usage with GUI

We have simple UI available at <https://forge.ublue.local>.

## Navigation

For navigation you can use the navigation bar on the top left.

![navigation](./assets/gui_navigation.png)

### Home

Currently nothing to see here. The goal is to display some
general information here. Links to documentation and important
news etc.

### Ansible

Here you can load your forge [configuration](./index.md#configuration) file from the data store,
clone projects and build container images. The buttons should be self explanatory.
The "Deploy VM" button is currently a mock-up, sorry :wink:
we are [working](https://github.com/ublue-os/forge/issues/35) on it.

![ansible](./assets/gui_ansible.png)

### Registry

Currently nothing to see here. The goal is to display some
information about the docker registry <registry.ublue.local> here.
Like a list of all available images you have uploaded etc.

### About

Currently nothing to see here. The goal is to display some
information here about the software and versions used in this project
and maybe some acknowledgments etc.
65 changes: 65 additions & 0 deletions docs/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
# Universal Blue - Forge Documentation

On-premises Universal Blue provides the service units necessary to set up a self-hosted
OS forge for custom images.

## Configuration

Since we are using ansible in the background, all user configurations are loaded via
[`extra-vars`](https://docs.ansible.com/ansible/latest/playbook_guide/playbooks_variables.html#defining-variables-at-runtime)
at runtime. No worries on how things are plugged together, you only have to know where
to create your configuration files. That's all.

So, during forge setup a podman volume `ublue-os_forge-data` is created. All user configuration
etc. is stored there. You can inspect the volume with:

```sh
❯ podman volume inspect ublue-os_forge-data
[
{
"Name": "ublue-os_forge-data",
"Driver": "local",
"Mountpoint": "/var/home/stephan/.local/share/containers/storage/volumes/ublue-os_forge-data/_data",
"CreatedAt": "2024-05-06T21:25:28.818751853+02:00",
"Labels": {},
"Scope": "local",
"Options": {},
"MountCount": 0,
"NeedsCopyUp": true,
"LockNumber": 98
}
]
```

Under the `"Mountpoint:"` you find the actual path on your file system. I would recommend you
symlink this folder to a path more rememberable. For example you can do the following:

```sh
FORGE_POD_DATA_DIR="$(podman volume inspect ublue-os_forge-data | jq -r '.[0].Mountpoint')"
ln -s $FORGE_POD_DATA_DIR $HOME/ublue-os_forge-data
```

With this the data folder would be available in your home directory under `~/ublue-os_forge-data`

In that folder you will find an example configuration similar to this:

```yaml
## ublue-os forge extra-vars example configuration
## For more details got to https://github.com/ublue-os/forge/blob/main/docs/variables.md
---
forge_git_repository_url: https://github.com/ublue-os/bluefin.git
forge_git_repository_destination: /var/home/stephan/.local/share/containers/storage/volumes/ublue-os_forge-data/_data/data/bluefin
forge_git_repository_version: main
forge_registry_url: registry.ublue.local
```
This file acts as a template. It has all available settings outlined for you. Simple copy
it and modify it to your liking. For each project you want to handle with the forge you can
create a dedicated file.
Details about the available variables are documented [here](./variables.md).
## Usage
There are two methods of operating the forge, either via a [GUI](./gui.md) available
at <https://forge.ublue.local> or via [just](./just.md) command runner.
31 changes: 31 additions & 0 deletions docs/just.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
# Usage with just command runner

If you don't want to use the [GUI](./gui.md) we provide the following recipes
via [just command runner](https://github.com/casey/just).

<!-- markdownlint-disable MD013 -->

| Just recipe | Input argument | Description |
| --------------------- | ----------------------- | -------------------------------------------- |
| `forge_project-clone` | `forge_config_var_file` | Clone git project repository |
| `forge_project-build` | `forge_config_var_file` | Build container image and upload to registry |

<!-- markdownlint-enable MD013 -->

The input argument expects a [configuration](./index.md#configuration) file from the data store.

To launch a recipe you simple run:

```sh
just -f forge.just {{ recipe_name }} {{ forge_config_var_file }}
```

**_Example:_**

```sh
just -f forge.just forge_project-clone ~/ublue-os_forge/my-forge-project.env
```

In case you don't have [just command runner](https://github.com/casey/just) available.
Have a look at the [forge.just](../forge.just) file. It easy enough to understand which commands
are executed via the just recipes.
43 changes: 43 additions & 0 deletions docs/setup.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
# Forge setup

The initial setup and maintenance of the forge you can use the `forge.sh` script:

<!-- markdownlint-disable MD013 -->

| Command | Description |
| ---------------------- | ----------------------------------------------------------- |
| `./forge.sh setup` | Setup the forge for the first time or update existing setup |
| `./forge.sh heat-up` | Start the forge |
| `./forge.sh cool-down` | Stop the forge |

<!-- markdownlint-enable MD013 -->

## What happens during setup

Just in case you haven't have a look at the [forge.sh](../forge.sh) script. The following
things happen during the setup:

1. The [pre-requisites](../README.md#pre-requisites) are checked

2. You will be asked for your user credentials
This is not to spy on you. We need them to execute the [configure_host.yml](../anvil/ansible/playbooks/configure_host.yml).
It will configure the necessary settings so you don't have to do that manually.
See step 7.

3. A podman network `ublue-os_forge` is created. All containers from this setup will be
attached to it in the next step

4. All containers are built and spun up according to the [forge-pod.yml](../forge-pod.yml)
specification.

5. The forge SSH key is added to your `~/.ssh/authorized_keys` file.
The root certificate and key are created during container build mentioned in the
previous step.

6. The forge root certificate is copied to the [data](index.md#configuration) directory.

7. The [configure_host.yml](../anvil/ansible/playbooks/configure_host.yml) playbook mentioned
in step 2 is executed.
This will created the necessary host entries in `/etc/hosts`, copy the forge root certificate
to `/etc/pki/ca-trust/source/anchors/`, `update-ca-trust` and create the example configuration
in the [data](index.md#configuration) directory.
57 changes: 14 additions & 43 deletions docs/variables.md
Original file line number Diff line number Diff line change
@@ -1,52 +1,23 @@
# Variables

The following sections contains all important variables defined for daily usage.
All variables mentioned here can be declared in a line-delimited file of environment variables.
On this page all important variables defined for daily usage are documented.
All variables mentioned here can be declared in a `yaml` configuration file.

An example file on the host system with all variables available will be created on setup
for you. By default it can be found under `$HOME/ublue-os_forge/forge_default_vars.env`.
Have a look at the [configuration](./index.md#configuration) chapter for details
on where to find the configuration directory.

On playbook launch the variable file will be imported into the ansible container so that
the settings are available during playbook execution.

## group_vars/all/data.yml

In the [data.yml](../ansible/group_vars/all/data.yml) all variables are defined
which are used in the context of the data handling.
The following configuration variables are available and can be set to your liking:

<!-- markdownlint-disable MD013 -->

| name | type | environment variable | default value | description |
| ---------------------------------------- | ---- | -------------------- | ------------------------------------------- | --------------------------------------------- |
| `forge_data_path` | str | `FORGE_DATA_PATH` | $HOME/ublue-os_forge | Path where forge will store files per default |
| `forge_data_default_variables_file_path` | str | | $HOME/ublue-os_forge/forge_default_vars.env | Path to default configuration file |

<!-- markdownlint-enable MD013 -->

## group_vars/all/git.yml

In the [git.yml](../ansible/group_vars/all/git.yml/) all variables are defined which are
used in the context of the git repositories.

<!-- markdownlint-disable MD013 -->

| name | type | environment variable | default value | description |
| ---------------------------------- | ---- | ---------------------------------- | ----------------------------------------- | ---------------------------------------------- |
| `forge_git_repository_url` | str | `FORGE_GIT_REPOSITORY_URL` | <https://github.com/ublue-os/bluefin.git> | Git repository url |
| `forge_git_repository_destination` | str | `FORGE_GIT_REPOSITORY_DESTINATION` | $HOME/ublue-os/forge/bluefin | Git destination where repository is cloned to |
| `forge_git_repository_version` | str | `FORGE_GIT_REPOSITORY_VERSION` | main | Git repository branch or tag or commit version |

<!-- markdownlint-enable MD013 -->

## group_vars/all/registry.yml

In the [registry.yml](../ansible/group_vars/all/registry.yml) all variables are defined
which are used in the context of the container registry.

<!-- markdownlint-disable MD013 -->
| Name | Type | Default value | Description |
| ---------------------------------- | ---- | ------------------------------------------------- | -------------------------------------------------------------------------------- |
| `forge_git_repository_url` | str | <https://github.com/ublue-os/bluefin.git> | Git repository url |
| `forge_git_repository_destination` | str | `{{ forge_data_volume_mountpoint }}`/data/bluefin | Git destination where repository is cloned to. Can be any directory on your host |
| `forge_git_repository_version` | str | main | Git repository branch or tag or commit version |
| `forge_registry_url` | str | registry.ublue.local | Container registry url |

| name | type | environment variable | default value | description |
| -------------------- | ---- | -------------------- | -------------------- | ---------------------- |
| `forge_registry_url` | str | `FORGE_REGISTRY_URL` | registry.ublue.local | Container registry url |
<!-- markdownlint-enable MD013-->

<!-- markdownlint-enable MD013 -->
**_Note:_** The `{{ forge_data_volume_mountpoint }}` points to your ublue-os_forge-data
podman volume.

0 comments on commit 4df5d16

Please sign in to comment.