TMS Configuration Service Documentation

Introduction

This is the Tazama configuration service. The Configuration Service is a tool to allow a non developer to modify the various artefacts within a Tazama deployment. We provide more information on the target implementers in the Target Customers Document.

It is assumed that you have already reviewed the main Tazama documentation. Once you are ready to deploy, please ensure you follow the machine setup instructions at Section 3.2.1 of contribution Guide.

The sequence of a standalone deployment of the configuration service is as follows:

Tazama Stack - if you wish to streamline your service you need the following shared components:
1. Authentication Service
2. KeyCloak
3. ArangoDB
4. NATS
5. ValKey
6. LumberJack
7. Event-sidecar
8. Elasticsearch
9. Kibana
Config Service Backend Deployment
1. Auth Lib Dependency
Config Service Frontend Deployment

Tazama Auth Service and Keycloak Deployment

Keycloak is deployed alongside the Tazama Authentication Service, which acts as an abstraction layer for Keycloak. The authentication service integrates Keycloak into the Configuration Service Backend, ensuring secure and seamless authentication across the platform, but with the flexibility to replace Keycloak with another service if desired.

Clone Full Stack Tazama

Do a git clone of the full stack tazama git clone https://github.com/tazama-lf/Full-Stack-Docker-Tazama

If you have setup your Token, you can jump to Docker Compose

Set GH Token (if not done already)

you have to set your git token so as to be able to deploy the auth service correctly

macOS & Linux (Ubuntu/Debian)

For Bash (bash shell users):

echo 'export GH_TOKEN="your_personal_access_token"' >> ~/.bashrc
source ~/.bashrc

For Zsh (zsh shell users, default for macOS):

echo 'export GH_TOKEN="your_personal_access_token"' >> ~/.zshrc
source ~/.zshrc

For Fish (fish shell users):

set -Ux GH_TOKEN "your_personal_access_token"

Windows

Option 1: Using Command Prompt

Run this in cmd:

setx GH_TOKEN "your_personal_access_token"

Option 2: Using PowerShell

Run:

[System.Environment]::SetEnvironmentVariable("GH_TOKEN", "your_personal_access_token", "User")

Option 3: Through the GUI

Press Win + R, type sysdm.cpl, and hit Enter.
Go to the Advanced tab and click Environment Variables.
Under User variables, click New.
Set Variable name as GH_TOKEN and Variable value as your_personal_access_token.
Click OK, then restart your terminal or system.

To confirm that the variable is set correctly:

macOS & Linux:

echo $GH_TOKEN

Windows (cmd):

echo %GH_TOKEN%

Windows (PowerShell):

$env:GH_TOKEN

Docker Compose

Make sure you have docker properly setup.

cd into Full-Stack-Docker-Tazama

add ports 18529:8529 to docker-compose.infrastructure.yaml

WARNING ❗ Currently the Tazama Lumberjack has a dependency on NATS, it might not deploy if you do not remove the dependence. Inside docker-compose.dev.logs-elastic.yaml file, look for - nats and remove it.

WARNING ❗ Currently docker-compose.dev.logs-elastic.yaml has a dependency on - nats It will not deploy if you don't remove the dependency. Inside docker-compose.dev.logs-elastic.yaml look for - nats and remove it

then run this command in the terminal to deploy auth service and keycloak, and the necessary infrastructure

docker compose -f docker-compose.infrastructure.yaml -p tazama up -d
docker compose -f docker-compose.auth.yaml -p tazama up -d
docker compose -f docker-compose.dev.logs-elastic.yaml -p tazama up -d
docker compose -f docker-compose.dev.elastic.yaml -p tazama up -d

Access Keycloak Admin Console

Open http://localhost:8080/admin
Login using: Username: admin | Password: password

Confirm The Auth Service is Up

We use postman to check the service, with the following request

We will check we can Get a token using a New Postman request

Type: POST
URL: http://localhost:3020/v1/auth/login

With the following in the body (raw)

{
  "username": "tazama-user",
  "password": "password"
}

If all is configured with Tazama - you should receive a token.

Keycloak Setup for Config Service Users, Roles and Privileges

Login to the Keycloak Portal http://localhost:8080/ - user name admin and password password

A realm called tazama was already created during the deployment for the Tazama service

WARNING: MAKE SURE YOU SELECT THE TAZAMA REALM - IT MAY HAVE DEFAULTED TO MASTER

Create Groups

You should see tazama_admin and tazama_tms

Go to Groups. Click on create group
Enter the following one at a time:
- config_svc_admin
- config_svc_editor
- config_svc_approver
- config_svc_viewer

Create Realm Roles

You should see GET_V1_EVENT_FLOW_CONTROL_ACCOUNT and more

Go to Realm roles
Click on Create role
Create the following roles one at a time:
- SECURITY_APPROVE_NETWORK_MAP
- SECURITY_CREATE_NETWORK_MAP
- SECURITY_CREATE_RULE
- SECURITY_CREATE_RULE_CONFIG
- SECURITY_CREATE_TYPOLOGY
- SECURITY_CREATE_TYPOLOGY_RULE_CONFIG
- SECURITY_DISABLE_RULE
- SECURITY_DISABLE_RULE_CONFIG
- SECURITY_DELETE_NETWORK_MAP
- SECURITY_DELETE_RULE
- SECURITY_DELETE_RULE_CONFIG
- SECURITY_DELETE_TYPOLOGY
- SECURITY_DISABLE_NETWORK_MAP
- SECURITY_DISABLE_TYPOLOGY
- SECURITY_EXPORT_NETWORK_MAP
- SECURITY_GET_NETWORK_MAP
- SECURITY_GET_RULE
- SECURITY_GET_RULES
- SECURITY_GET_RULE_CONFIG
- SECURITY_GET_RULE_CONFIGS
- SECURITY_GET_RULE_RULE_CONFIG
- SECURITY_GET_TYPOLOGIES
- SECURITY_GET_TYPOLOGY
- SECURITY_GET_TYPOLOGY_RULE_CONFIG
- SECURITY_GET_TYPOLOGY_RULE_CONFIGS
- SECURITY_IMPORT_NETWORK_MAP
- SECURITY_UPDATE_NETWORK_MAP
- SECURITY_UPDATE_RULE
- SECURITY_UPDATE_RULE_CONFIG
- SECURITY_UPDATE_TYPOLOGY
- SECURITY_UPDATE_TYPOLOGY_RULE_CONFIG
- SECURITY_APPROVE_RULE
- SECURITY_REJECT_RULE
- SECURITY_WITHDRAW_RULE
- SECURITY_DEPLOY_RULE
- SECURITY_RETIRE_RULE
- SECURITY_ARCHIVE_RULE
- SECURITY_ABANDON_RULE
- SECURITY_APPROVE_RULE_CONFIG
- SECURITY_REJECT_RULE_CONFIG
- SECURITY_WITHDRAW_RULE_CONFIG
- SECURITY_DEPLOY_RULE_CONFIG
- SECURITY_RETIRE_RULE_CONFIG
- SECURITY_ARCHIVE_RULE_CONFIG
- SECURITY_ABANDON_RULE_CONFIG
- SECURITY_APPROVE_TYPOLOGY
- SECURITY_REJECT_TYPOLOGY
- SECURITY_WITHDRAW_TYPOLOGY
- SECURITY_DEPLOY_TYPOLOGY
- SECURITY_RETIRE_TYPOLOGY
- SECURITY_ARCHIVE_TYPOLOGY
- SECURITY_ABANDON_TYPOLOGY
- SECURITY_REJECT_NETWORK_MAP
- SECURITY_WITHDRAW_NETWORK_MAP
- SECURITY_DEPLOY_NETWORK_MAP
- SECURITY_RETIRE_NETWORK_MAP
- SECURITY_ARCHIVE_NETWORK_MAP
- SECURITY_ABANDON_NETWORK_MAP
- EXIT_COND_GET_ALL
- EXIT_COND_GET_BY_ID
- EXIT_COND_GET_USER_DEFAULT
- EXIT_COND_CREATE_USER
- EXIT_COND_SET_USER_DEFAULT
- EXIT_COND_DELETE_USER

Assign Realm Roles to Groups

We have created 58 realm roles.

config_svc_admin

Go to Groups
Click on config_svc_admin
Click on Role mapping
Click on Assign role
Select the following roles. all 58 of them (expand the modal so you can see all of the roles in needed):
- SECURITY_APPROVE_NETWORK_MAP
- SECURITY_CREATE_NETWORK_MAP
- SECURITY_CREATE_RULE
- SECURITY_CREATE_RULE_CONFIG
- SECURITY_CREATE_TYPOLOGY
- SECURITY_CREATE_TYPOLOGY_RULE_CONFIG
- SECURITY_DISABLE_RULE
- SECURITY_DISABLE_RULE_CONFIG
- SECURITY_DELETE_NETWORK_MAP
- SECURITY_DELETE_RULE
- SECURITY_DELETE_RULE_CONFIG
- SECURITY_DELETE_TYPOLOGY
- SECURITY_DISABLE_NETWORK_MAP
- SECURITY_DISABLE_TYPOLOGY
- SECURITY_EXPORT_NETWORK_MAP
- SECURITY_GET_NETWORK_MAP
- SECURITY_GET_RULE
- SECURITY_GET_RULES
- SECURITY_GET_RULE_CONFIG
- SECURITY_GET_RULE_CONFIGS
- SECURITY_GET_RULE_RULE_CONFIG
- SECURITY_GET_TYPOLOGIES
- SECURITY_GET_TYPOLOGY
- SECURITY_GET_TYPOLOGY_RULE_CONFIG
- SECURITY_GET_TYPOLOGY_RULE_CONFIGS
- SECURITY_IMPORT_NETWORK_MAP
- SECURITY_UPDATE_NETWORK_MAP
- SECURITY_UPDATE_RULE
- SECURITY_UPDATE_RULE_CONFIG
- SECURITY_UPDATE_TYPOLOGY
- SECURITY_UPDATE_TYPOLOGY_RULE_CONFIG
- SECURITY_APPROVE_RULE
- SECURITY_REJECT_RULE
- SECURITY_WITHDRAW_RULE
- SECURITY_DEPLOY_RULE
- SECURITY_RETIRE_RULE
- SECURITY_ARCHIVE_RULE
- SECURITY_ABANDON_RULE
- SECURITY_APPROVE_RULE_CONFIG
- SECURITY_REJECT_RULE_CONFIG
- SECURITY_WITHDRAW_RULE_CONFIG
- SECURITY_DEPLOY_RULE_CONFIG
- SECURITY_RETIRE_RULE_CONFIG
- SECURITY_ARCHIVE_RULE_CONFIG
- SECURITY_ABANDON_RULE_CONFIG
- SECURITY_APPROVE_TYPOLOGY
- SECURITY_REJECT_TYPOLOGY
- SECURITY_WITHDRAW_TYPOLOGY
- SECURITY_DEPLOY_TYPOLOGY
- SECURITY_RETIRE_TYPOLOGY
- SECURITY_ARCHIVE_TYPOLOGY
- SECURITY_ABANDON_TYPOLOGY
- SECURITY_REJECT_NETWORK_MAP
- SECURITY_WITHDRAW_NETWORK_MAP
- SECURITY_DEPLOY_NETWORK_MAP
- SECURITY_RETIRE_NETWORK_MAP
- SECURITY_ARCHIVE_NETWORK_MAP
- SECURITY_ABANDON_NETWORK_MAP
- EXIT_COND_GET_ALL
- EXIT_COND_GET_BY_ID
- EXIT_COND_GET_USER_DEFAULT
- EXIT_COND_CREATE_USER
- EXIT_COND_SET_USER_DEFAULT
- EXIT_COND_DELETE_USER
The total roles should be 58 in number

config_svc_approver

Click on config_svc_approver
Click on Role mapping
Click on Assign role
Select the below roles:
- SECURITY_APPROVE_RULE
- SECURITY_REJECT_RULE
- SECURITY_APPROVE_RULE_CONFIG
- SECURITY_REJECT_RULE_CONFIG
- SECURITY_APPROVE_TYPOLOGY
- SECURITY_REJECT_TYPOLOGY
- SECURITY_APPROVE_NETWORK_MAP
- SECURITY_REJECT_NETWORK_MAP
- SECURITY_DEPLOY_RULE
- SECURITY_RETIRE_RULE
- SECURITY_ARCHIVE_RULE
- SECURITY_DEPLOY_RULE_CONFIG
- SECURITY_RETIRE_RULE_CONFIG
- SECURITY_ARCHIVE_RULE_CONFIG
- SECURITY_DEPLOY_TYPOLOGY
- SECURITY_RETIRE_TYPOLOGY
- SECURITY_ARCHIVE_TYPOLOGY
- SECURITY_DEPLOY_NETWORK_MAP
- SECURITY_RETIRE_NETWORK_MAP
- SECURITY_ARCHIVE_NETWORK_MAP
- SECURITY_GET_RULE
- SECURITY_GET_RULES
- SECURITY_GET_RULE_CONFIG
- SECURITY_GET_RULE_CONFIGS
- SECURITY_GET_RULE_RULE_CONFIG
- SECURITY_GET_TYPOLOGY
- SECURITY_GET_TYPOLOGIES
- SECURITY_GET_TYPOLOGY_RULE_CONFIG
- SECURITY_GET_TYPOLOGY_RULE_CONFIGS
- SECURITY_GET_NETWORK_MAP
- EXIT_COND_GET_ALL
- EXIT_COND_GET_BY_ID
- EXIT_COND_GET_USER_DEFAULT
- EXIT_COND_CREATE_USER
- EXIT_COND_SET_USER_DEFAULT
- EXIT_COND_DELETE_USER
The Total Roles should be 36 in Number

config_svc_editor

Click on config_svc_editor
Click on Role mapping
Click on Assign role
Select the below roles:
- SECURITY_CREATE_RULE
- SECURITY_CREATE_RULE_CONFIG
- SECURITY_CREATE_TYPOLOGY
- SECURITY_CREATE_NETWORK_MAP
- SECURITY_CREATE_TYPOLOGY_RULE_CONFIG
- SECURITY_UPDATE_RULE
- SECURITY_UPDATE_RULE_CONFIG
- SECURITY_UPDATE_TYPOLOGY
- SECURITY_UPDATE_NETWORK_MAP
- SECURITY_UPDATE_TYPOLOGY_RULE_CONFIG
- SECURITY_GET_RULE
- SECURITY_GET_RULES
- SECURITY_GET_RULE_CONFIG
- SECURITY_GET_RULE_CONFIGS
- SECURITY_GET_RULE_RULE_CONFIG
- SECURITY_GET_TYPOLOGY
- SECURITY_GET_TYPOLOGIES
- SECURITY_GET_TYPOLOGY_RULE_CONFIG
- SECURITY_GET_TYPOLOGY_RULE_CONFIGS
- SECURITY_GET_NETWORK_MAP
- SECURITY_WITHDRAW_RULE
- SECURITY_WITHDRAW_RULE_CONFIG
- SECURITY_WITHDRAW_TYPOLOGY
- SECURITY_WITHDRAW_NETWORK_MAP
- SECURITY_ABANDON_RULE
- SECURITY_ABANDON_RULE_CONFIG
- SECURITY_ABANDON_TYPOLOGY
- SECURITY_ABANDON_NETWORK_MAP
- EXIT_COND_GET_ALL
- EXIT_COND_GET_BY_ID
- EXIT_COND_GET_USER_DEFAULT
- EXIT_COND_CREATE_USER
- EXIT_COND_SET_USER_DEFAULT
- EXIT_COND_DELETE_USER
The Total Roles should be 34 in Number

config_svc_viewer

Click on config_svc_viewer
Click on Role mapping
Click on Assign role
Select the below roles:
- SECURITY_GET_RULE
- SECURITY_GET_RULE_CONFIG
- SECURITY_GET_RULE_CONFIGS
- SECURITY_GET_RULE_RULE_CONFIG
- SECURITY_GET_RULES
- SECURITY_GET_TYPOLOGIES
- SECURITY_GET_TYPOLOGY
- SECURITY_GET_TYPOLOGY_RULE_CONFIG
- SECURITY_GET_TYPOLOGY_RULE_CONFIGS
- SECURITY_GET_NETWORK_MAP
- EXIT_COND_GET_ALL
- EXIT_COND_GET_BY_ID
- EXIT_COND_GET_USER_DEFAULT
- EXIT_COND_CREATE_USER
- EXIT_COND_SET_USER_DEFAULT
- EXIT_COND_DELETE_USER
The Total Roles should be 16 in Number

Create User

We need to create test users so we can evaluate the service

User Details

Go to user
Click on add user
Check the Email Verified
Fill out the Username Email First Name Last Name NOTE - User name must be the same as email for now
Click on Join group then select the group for the user. Click join
Then click on create

User Password

Go to user
Click on the user
Click on Credentials
Then create a password. Remember to uncheck Temporary if it is checked

Assign Groups to Existing User

You can assign groups or change the groups of an exitsing user

Go to user
Click on the specific user eg tazama-user
Click on Groups
Click on Join Group
Select the required group
Then click on Join

Config Service Packages

Clone the config service frontend and backend using this command git clone --branch dev https://github.com/frmscoe/config-service.git

or git clone --branch dev [email protected]:frmscoe/config-service.git if you have setup SSH

Config Service Backend Deployment

Navigate to config-service/packages/config-svc-be

Install All The Required Dependencies

nvm use
npm install

Auth Lib Dependency - NPM Package Installation

Ensure that a .npmrc file exists at the same location with the package.json file of config-svc-be. It should contain the below content:

@tazama-lf:registry=https://npm.pkg.github.com
//npm.pkg.github.com/:_authToken=${GH_TOKEN}

Then run the below command to install auth-lib

npm install @tazama-lf/auth-lib

Audit and Logging

navigate to config-svc-be and install this library

npm i @tazama-lf/frms-coe-lib

confirm that Elastic search is running

http://localhost:9200/

confirm that kibana is running

http://localhost:5601

NB: The ELK will use the environment variables that are set in Full-Stack-Docker-Tazama. The same environment variables have to be repeated on .env of the config-svc-be. Check the env.sample for all the required environment variables

Copy and paste this environment variable on .env file of config-svc-be.

# Audit and Logging
SIDECAR_HOST=localhost:15000
NODE_ENV=dev
FUNCTION_NAME=relay-service
MAX_CPU=1

EVENT_SIDECAR_PORT=15000                  # Must match SIDECAR_HOST port


NATS_SERVER=nats://nats:4222              # Must match event-sidecar's NATS_SERVER
NATS_SUBJECT=Lumberjack                   # Must match event-sidecar's NATS_SUBJECT
ELASTIC_SEARCH_VERSION=8.15.1             # Match your Elasticsearch version
ELASTIC_HOST=http://elasticsearch:9200
FLUSHBYTES=1024                           # Can tune based on volume
ES_PORT=9200

Create a folder inside config-svc-be called env inside it, create a file called lumberjack.env

Paste the following environment variables inside the file

ELASTIC_HOST=http://elasticsearch:9200
ELASTIC_STACK_VERSION=8.15.1
ELASTIC=true
NATS_SERVER=nats://nats:4222
NATS_SUBJECT=Lumberjack
FLUSHBYTES=1024
ES_PORT=9200

Set Environment Variables

There is a .env file created at the root of the folder and setup the environment variables below.

# Arango Database Connection.
DATABASE_HOST=http://localhost:18529/ # ArangoDB URL
DATABASE_NAME=config-svc-db # The main database
SYSTEM_DATABASE_NAME=_system # System database (pre-installed)
DATABASE_USERNAME=root # Default username
DATABASE_PASSWORD=password # Default password
DATABASE_NAME_TEST=config-svc-db-test # Test database

# config-svc-be application port
PORT=3007

Set Auth Lib Environment Variable

Add the following to the .env you created above

# Auth Lib
AUTH_URL=http://localhost:8080 
KEYCLOAK_REALM=tazama
CLIENT_ID=auth-lib-client 
CLIENT_SECRET=auth-lib-client-test-secret 
CERT_PATH_PRIVATE=/absolute/path/to/test-private-key.pem 
CERT_PATH_PUBLIC=/absolute/path/to/test-public-key.pem 
AUTH_SERVICE_URL=http://localhost:3020

Private Key (private-key.pem) - CERT_PATH_PRIVATE:

A test-private-key.pem has been provided for you. You should find it in config-service/packages/test-private-key.pem

Public Key (public-key.pem) - CERT_PATH_PUBLIC:

A test-public-key.pem has been provided for you. You should find it in config-service/packages/test-private-key.pem

Store Paths in the `.env` File

Now that you have your keys, store the file paths in config-svc-be/.env:

CERT_PATH_PRIVATE=/absolute/path/to/config-service/packages/test-private-key.pem
CERT_PATH_PUBLIC=/absolute/path/to/config-service/packages/test-private-key.pem

Replace /absolute/path/to/ with the actual location where you saved the files.

Troubleshooting

If auth-lib throws an error like "Missing or Corrupted Private Key", ensure that:
- The .env file is loaded before auth-lib is initialized.
- The private key file exists and has the correct path.
If permissions issues arise, change file permissions:
- chmod 600 test-private-key.pem test-public-key.pem

Start The Server

npm run start

Check Server open http://localhost:3007

API Endpoints swagger http://localhost:3007/swagger

NB: for the environment variables, you can copy the contents in .env.sample file into .env file and then modify them to suit your setup.

Config Service Frontend Deployment

Navigate to config-service/packages/config-svc-fe

NPM Package Installation

nvm use
npm install

Set Environment Variables

create a .env and paste the following environment variables:

PORT=4000 # this specifies the port the frontend is running

NEXT_PUBLIC_CONFIG_SVC_BE_URL=http://localhost:3007 # this is the config_svc_be URL
NEXT_PUBLIC_SECURITY_BC_URL=http://localhost:8080/realms/tazama   # this is the keycloak realm URL
NEXT_PUBLIC_SECURITY_BC_CLIENT_ID=auth-lib-client # this is the keycloak client ID
NEXT_PUBLIC_SECURITY_BC_SECRET=auth-lib-client-test-secret # this is the keycloak client secret

NEXT_PUBLIC_SECURITY_FETCH_LIMIT=100 #this is for setting the total number of artifates that are fetched

Warning ❗ You must set your IP instead of local host, if you are running a remote machine.

SSL Certificate

You need an SSL certificate to run the server.

At the project root config-svc-fe, run this command mkdir certificates
At the project root config-svc-fe, run this command openssl req -x509 -newkey rsa:2048 -nodes -sha256 -days 365 -keyout certificates/localhost.key -out certificates/localhost.crt -subj "/CN=localhost"

NB: This uses OpenSSL and they come preinstalled in macOS and Linux.

Start The Server

npm run dev

Check Server http://localhost:4000 or https://localhost:4000

NB: for the environment variables, you can copy the contents in .env.sample file into .env file and then modify them to suit your setup.

Automated Testing

Now the service is deployed, we can run the following automated tests

Frontend - Login Page Test

EmailInputForm Test
PasswordInputForm Test
LoginProcess Test
ImportRuleTests

The automated test was built using Jest Test Framework. Use the below command to run the test:

npm run test

To generate a coverage report for the test use this command (NB: this is optional):

npx jest --coverage

Name		Name	Last commit message	Last commit date
Latest commit History 49 Commits
.github		.github
docs		docs
packages		packages
.gitignore		.gitignore
LICENSE		LICENSE
README.md		README.md

License

frmscoe/config-service

Folders and files

Latest commit

History

Repository files navigation

TMS Configuration Service Documentation

Introduction

Tazama Auth Service and Keycloak Deployment

Clone Full Stack Tazama

Set GH Token (if not done already)

Docker Compose

Access Keycloak Admin Console

Confirm The Auth Service is Up

Keycloak Setup for Config Service Users, Roles and Privileges

Create Groups

Create Realm Roles

Assign Realm Roles to Groups

config_svc_admin

config_svc_approver

config_svc_editor

config_svc_viewer

Create User

User Details

User Password

Assign Groups to Existing User

Config Service Packages

Config Service Backend Deployment

Install All The Required Dependencies

Auth Lib Dependency - NPM Package Installation

Audit and Logging

Set Environment Variables

Set Auth Lib Environment Variable

Private Key (private-key.pem) - CERT_PATH_PRIVATE:

Public Key (public-key.pem) - CERT_PATH_PUBLIC:

Store Paths in the .env File

Troubleshooting

Start The Server

Config Service Frontend Deployment

NPM Package Installation

Set Environment Variables

SSL Certificate

Start The Server

Automated Testing

About

Resources

License

Code of conduct

Contributing

Security policy

Uh oh!

Stars

Watchers

Forks

Releases

Packages 0

Uh oh!

Contributors 6

Uh oh!

Languages

Store Paths in the `.env` File

Packages