feat(compose): Configure Keycloak with OpenTofu

[haikoschol]

This PR sets up automatic execution of OpenTofu code for configuring Keycloak in the docker-compose environment.

The actual configuration needs some tweaking as currently authentication of ort-server fails. I couldn't tell from the realm export which settings were important changes and which were default values. I picked a few, but apparently missed some. Maybe someone with a better understanding of the config applied in master-realm.json can point those out.

The login in the UI fails as well, probably as a consequence of the former failure.

This is the log from the keycloak container:

keycloak-1          | 2024-05-17 04:45:10,958 WARN  [org.keycloak.events] (executor-thread-2) type="LOGIN_ERROR", realmId="515b1d67-d038-45b3-b05d-18ed10bf318e", clientId="ort-server", userId="null", ipAddress="192.168.237.13", error="invalid_client_credentials", grant_type="password"
core-1              | 2024-05-17 04:45:11.063 [DefaultDispatcher-worker-1] level=INFO  o.e.a.o.s.DefaultAuthorizationService - Creating 'superuser' role.
keycloak-1          | 2024-05-17 04:45:11,076 WARN  [org.keycloak.events] (executor-thread-2) type="LOGIN_ERROR", realmId="515b1d67-d038-45b3-b05d-18ed10bf318e", clientId="ort-server", userId="null", ipAddress="192.168.237.13", error="invalid_client_credentials", grant_type="password"

[haikoschol]

I'll try to implement the same functionality with Pulumi in Kotlin to have both alternatives suggested in #20 available.

[haikoschol]

Somewhat unrelated, but it might also make sense to create a separate realm instead of making changes to master. This is recommended for production deployments in the Keycloak documentation. I think it makes sense to do the same in the dev environment to catch issues with hard-coded realm names early.

[haikoschol]

I tweaked the config a bit more and managed to get rid of the error. This is the log output from ORT Server core regarding Keycloak that I see now:

core-1              | 2024-05-21 06:39:42.028 [DefaultDispatcher-worker-2] level=INFO  i.k.server.application.Application - Ensuring superuser role and group.
core-1              | 2024-05-21 06:39:44.275 [DefaultDispatcher-worker-4] level=INFO  o.e.a.o.s.DefaultAuthorizationService - Adding role 'superuser' to group 'SUPERUSERS'.
core-1              | 2024-05-21 06:39:44.344 [DefaultDispatcher-worker-8] level=INFO  i.k.server.application.Application - Synchronizing Keycloak permissions.
core-1              | 2024-05-21 06:39:44.403 [DefaultDispatcher-worker-10] level=INFO  o.e.a.o.s.DefaultAuthorizationService - Synchronizing Keycloak roles for organization permissions.
core-1              | 2024-05-21 06:39:44.516 [DefaultDispatcher-worker-10] level=DEBUG Exposed - SELECT organizations.id, organizations."name", organizations.description FROM organizations
core-1              | 2024-05-21 06:39:44.524 [DefaultDispatcher-worker-10] level=INFO  o.e.a.o.s.DefaultAuthorizationService - Synchronizing Keycloak roles for product permissions.
core-1              | 2024-05-21 06:39:44.623 [DefaultDispatcher-worker-10] level=DEBUG Exposed - SELECT products.id, products.organization_id, products."name", products.description FROM products
core-1              | 2024-05-21 06:39:44.626 [DefaultDispatcher-worker-10] level=INFO  o.e.a.o.s.DefaultAuthorizationService - Synchronizing Keycloak roles for repository permissions.
core-1              | 2024-05-21 06:39:44.698 [DefaultDispatcher-worker-10] level=DEBUG Exposed - SELECT repositories.id, repositories.product_id, repositories."type", repositories.url FROM repositories
core-1              | 2024-05-21 06:39:44.740 [DefaultDispatcher-worker-10] level=INFO  i.k.server.application.Application - Synchronizing Keycloak roles.
core-1              | 2024-05-21 06:39:44.851 [DefaultDispatcher-worker-10] level=INFO  o.e.a.o.s.DefaultAuthorizationService - Synchronizing Keycloak roles for organization roles.
core-1              | 2024-05-21 06:39:44.856 [DefaultDispatcher-worker-10] level=DEBUG Exposed - SELECT organizations.id, organizations."name", organizations.description FROM organizations
core-1              | 2024-05-21 06:39:44.859 [DefaultDispatcher-worker-10] level=INFO  o.e.a.o.s.DefaultAuthorizationService - Synchronizing Keycloak groups for organization roles.
core-1              | 2024-05-21 06:39:44.861 [DefaultDispatcher-worker-10] level=DEBUG Exposed - SELECT organizations.id, organizations."name", organizations.description FROM organizations
core-1              | 2024-05-21 06:39:44.865 [DefaultDispatcher-worker-10] level=INFO  o.e.a.o.s.DefaultAuthorizationService - Synchronizing Keycloak roles for product roles.
core-1              | 2024-05-21 06:39:44.867 [DefaultDispatcher-worker-10] level=DEBUG Exposed - SELECT products.id, products.organization_id, products."name", products.description FROM products
core-1              | 2024-05-21 06:39:44.868 [DefaultDispatcher-worker-10] level=INFO  o.e.a.o.s.DefaultAuthorizationService - Synchronizing Keycloak groups for product roles.
core-1              | 2024-05-21 06:39:44.870 [DefaultDispatcher-worker-10] level=DEBUG Exposed - SELECT products.id, products.organization_id, products."name", products.description FROM products
core-1              | 2024-05-21 06:39:44.870 [DefaultDispatcher-worker-10] level=INFO  o.e.a.o.s.DefaultAuthorizationService - Synchronizing Keycloak roles for repository roles.
core-1              | 2024-05-21 06:39:44.872 [DefaultDispatcher-worker-10] level=DEBUG Exposed - SELECT repositories.id, repositories.product_id, repositories."type", repositories.url FROM repositories
core-1              | 2024-05-21 06:39:44.873 [DefaultDispatcher-worker-10] level=INFO  o.e.a.o.s.DefaultAuthorizationService - Synchronizing Keycloak groups for repository roles.
core-1              | 2024-05-21 06:39:44.875 [DefaultDispatcher-worker-10] level=DEBUG Exposed - SELECT repositories.id, repositories.product_id, repositories."type", repositories.url FROM repositories
core-1              | 2024-05-21 06:39:44.875 [DefaultDispatcher-worker-10] level=INFO  i.k.server.application.Application - Synchronized Keycloak permissions and roles.

However, log in via the UI still fails with a 401 response.

[mnonnenmacher]

However, log in via the UI still fails with a 401 response.

I think you forgot to add the "react" client that was added here:
33504d4

[haikoschol]

However, log in via the UI still fails with a 401 response.

I think you forgot to add the "react" client that was added here: 33504d4

It's at the bottom of the keycloak.tf file.

[sschuberth]

How relevant is this still for us, @mmurto, also in the context of #895?

[mmurto]

How relevant is this still for us, @mmurto, also in the context of #895?

I'd keep this around either for someone to continue or to use as reference.

[sschuberth]

I'd keep this around either for someone to continue or to use as reference.

Ok, but can we still close it to clean up the list of open PRs? Even for closed PRs the code is still maintained for reference.

[mnonnenmacher]

I'd keep this around either for someone to continue or to use as reference.

Ok, but can we still close it to clean up the list of open PRs? Even for closed PRs the code is still maintained for reference.

That's right, but a draft PR is more visible and could maybe even motivate someone to finish the work.

[sschuberth]

could maybe even motivate someone to finish the work.

High hopes; it didn't work for ORT, though.

[mmurto]

could maybe even motivate someone to finish the work.

High hopes; it didn't work for ORT, though.

Shouldn't you look at closed PRs to determine whether someone has been motivated to close one? ;)

[sschuberth]

Shouldn't you look at closed PRs to determine whether someone has been motivated to close one? ;)

Not necessarily. Looking at the number of long-pending draft PRs also tells you something.

[mmurto]

Shouldn't you look at closed PRs to determine whether someone has been motivated to close one? ;)

Not necessarily. Looking at the number of long-pending draft PRs also tells you something.

Absolutely! I have no strong opinions in any direction here - the open PR doesn't really bother me, but closing doesn't likely make anything worse either.