Skip to content

Commit

Permalink
Remove references to the map store (#593)
Browse files Browse the repository at this point in the history
Signed-off-by: Alexander Schwartz <[email protected]>
Co-authored-by: Kamesh Akella <[email protected]>
  • Loading branch information
ahus1 and kami619 authored Nov 20, 2023
1 parent d196593 commit 1405b51
Show file tree
Hide file tree
Showing 42 changed files with 24 additions and 3,225 deletions.
82 changes: 6 additions & 76 deletions .github/workflows/provision-minikube.yml
Original file line number Diff line number Diff line change
Expand Up @@ -59,90 +59,20 @@ jobs:
github token: ${{ secrets.GITHUB_TOKEN }}
driver: docker
start args: --memory 4096
- name: Provision Legacy store on PostgreSQL DB
- name: Provision on PostgreSQL DB
working-directory: provision/minikube
run: |
PATH=$PATH:$GITHUB_WORKSPACE/bin
./rebuild.sh
task KC_DATABASE=postgres KC_STORAGE=
task KC_DATABASE=postgres
kubectl get pods -A
kubectl get events -A
- name: Create Dataset on Postgres Legacy Store
- name: Create Dataset on Postgres DB
working-directory: provision/minikube
run: |
task KC_DATABASE=postgres KC_STORAGE= dataset-import -- -a clear-status-completed
task KC_DATABASE=postgres KC_STORAGE= dataset-import -- -a create-realms -r 1 -c 5 -u 5 -i 1000
task KC_DATABASE=postgres KC_STORAGE= dataset-import -- -a status-completed
- name: Provision JPA Map store on PostgreSQL DB
working-directory: provision/minikube
run: |
PATH=$PATH:$GITHUB_WORKSPACE/bin
task KC_DATABASE=postgres KC_STORAGE=jpa
- name: Create Dataset on Postgres JPA Map Store
working-directory: provision/minikube
run: |
task KC_DATABASE=postgres KC_STORAGE=jpa KC_OTEL=true dataset-import -- -a clear-status-completed
task KC_DATABASE=postgres KC_STORAGE=jpa KC_OTEL=true dataset-import -- -a create-realms -r 1 -c 5 -u 5 -i 1000
task KC_DATABASE=postgres KC_STORAGE=jpa KC_OTEL=true dataset-import -- -a status-completed
- name: Provision JPA Map store on Cockroach DB Single Node
working-directory: provision/minikube
run: |
PATH=$PATH:$GITHUB_WORKSPACE/bin
task KC_DATABASE=cockroach-single KC_STORAGE=jpa
kubectl get pods -A
kubectl get events -A
- name: Create Dataset on Cockroach JPA Map Store
working-directory: provision/minikube
run: |
task KC_DATABASE=cockroach-single KC_STORAGE=jpa dataset-import -- -a clear-status-completed
task KC_DATABASE=cockroach-single KC_STORAGE=jpa dataset-import -- -a create-realms -r 1 -c 5 -u 5 -i 1000
task KC_DATABASE=cockroach-single KC_STORAGE=jpa dataset-import -- -a status-completed
- name: Provision Hot Rod Map store on Infinispan (In-Memory store)
working-directory: provision/minikube
run: |
PATH=$PATH:$GITHUB_WORKSPACE/bin
task KC_DATABASE=infinispan KC_STORAGE=hotrod
kubectl get pods -A
kubectl get events -A
- name: Create Dataset on Infinispan Hot Rod Store
working-directory: provision/minikube
run: |
task KC_DATABASE=infinispan KC_STORAGE=hotrod dataset-import -- -a clear-status-completed
task KC_DATABASE=infinispan KC_STORAGE=hotrod dataset-import -- -a create-realms -r 1 -c 5 -u 5 -i 1000
task KC_DATABASE=infinispan KC_STORAGE=hotrod dataset-import -- -a status-completed
- name: Provision JPA and Hot Rod store on PostgreSQL and Infinispan (In-Memory store)
working-directory: provision/minikube
run: |
PATH=$PATH:$GITHUB_WORKSPACE/bin
task KC_DATABASE=postgres+infinispan KC_STORAGE=jpa+hotrod
kubectl get pods -A
kubectl get events -A
- name: Create Dataset on Infinispan Hot Rod and PostgreSQL JPA Store
working-directory: provision/minikube
run: |
task KC_DATABASE=postgres+infinispan KC_STORAGE=jpa+hotrod dataset-import -- -a clear-status-completed
task KC_DATABASE=postgres+infinispan KC_STORAGE=jpa+hotrod dataset-import -- -a create-realms -r 1 -c 5 -u 5 -i 1000
task KC_DATABASE=postgres+infinispan KC_STORAGE=jpa+hotrod dataset-import -- -a status-completed
- name: Provision JPA Map store on CHM (File store)
working-directory: provision/minikube
run: |
PATH=$PATH:$GITHUB_WORKSPACE/bin
task KC_DATABASE=none KC_STORAGE=chm
kubectl get pods -A
kubectl get events -A
- name: Create Dataset on CHM Store
working-directory: provision/minikube
run: |
task KC_DATABASE=none KC_STORAGE=chm dataset-import -- -a clear-status-completed
task KC_DATABASE=none KC_STORAGE=chm dataset-import -- -a create-realms -r 1 -c 5 -u 5 -i 1000
task KC_DATABASE=none KC_STORAGE=chm dataset-import -- -a status-completed
- name: Provision Cockroach Operator minimally
working-directory: provision/minikube
run: |
# minimal testing for storage cockroach-operator, as GitHub actions doesn't have enough resources for this
helm template --debug keycloak --set hostname=minikube.nip.io --set storage=cockroach-operator
kubectl get pods -A
kubectl get events -A
task KC_DATABASE=postgres dataset-import -- -a clear-status-completed
task KC_DATABASE=postgres dataset-import -- -a create-realms -r 1 -c 5 -u 5 -i 1000
task KC_DATABASE=postgres dataset-import -- -a status-completed
- name: Show logs on failure
if: ${{ failure() }}
working-directory: provision/minikube
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -84,15 +84,10 @@ public void setup() throws IOException {
}

@Test
public void deployWithLegacyStore() throws IOException, ExecutionException, InterruptedException, URISyntaxException {
public void deployWithCurrentStore() throws IOException, ExecutionException, InterruptedException, URISyntaxException {
runTestWithParameter();
}

@Test
public void deployWithMapStore() throws IOException, ExecutionException, InterruptedException, URISyntaxException {
runTestWithParameter("--storage", "chm");
}

private void runTestWithParameter(String... args) throws IOException, URISyntaxException, InterruptedException, ExecutionException {
Path keycloakProvidersFolder = getKeycloakProvidersFolder();
Path keycloakDatasetProviderJar = getKeycloakDatasetProviderJar();
Expand Down
5 changes: 0 additions & 5 deletions doc/kubernetes/modules/ROOT/nav.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -34,11 +34,6 @@
** xref:storage/postgres.adoc[]
** xref:storage/aurora-regional-postgres.adoc[]
** xref:storage/aurora-global-postgres.adoc[]
** xref:storage/cockroach-single.adoc[]
** xref:storage/cockroach-operator.adoc[]
** xref:storage/infinispan.adoc[]
** xref:storage/postgres-infinispan.adoc[]
** xref:storage/concurrent-hash-map.adoc[]
* xref:error-messages.adoc[]
* xref:utils.adoc[]
+
Expand Down
59 changes: 3 additions & 56 deletions doc/kubernetes/modules/ROOT/pages/customizing-deployment.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -14,12 +14,11 @@ See xref:storage-configurations.adoc[] to see which storage options are availabl

To change the default, create an `.env` in the `provision/minikube` folder.

An example configuration as below enables CockroachDB single node and the map storage with JPA.
An example configuration as below enables PostgreSQL.

[source]
----
KC_DATABASE=cockroach-single
KC_STORAGE=jpa
KC_DATABASE=postgres
----

include::partial$rerun-task-after-changes.adoc[]
Expand All @@ -28,7 +27,7 @@ As an alternative way the parameters can be passed on the command line of the ta

[source,bash]
----
task KC_STORAGE="jpa" KC_DATABASE="cockroach-single"
task KC_DATABASE="postgres"
----

== Available Keycloak configuration options
Expand All @@ -51,65 +50,13 @@ See xref:storage/postgres.adoc[] for more information.
`aurora-postgres`:: Connect to an AWS Aurora PostgreSQL cluster.
+
See xref:storage/aurora-regional-postgres.adoc[] and xref:storage/aurora-global-postgres.adoc[] for more information.

`cockroach-single`:: Deploy a single-node CockroachDB instance.
+
See xref:storage/cockroach-single.adoc[] for more information.

`cockroach-operator`:: Deploy a multi-node CockroachDB instance using the CockroachDB operator.
+
See xref:storage/cockroach-operator.adoc[] for more information.

`infinispan`:: Deploy a single-node Infinispan instance.
+
See xref:storage/infinispan.adoc[] for more information.

`postgres+infinispan`:: Deploy mixed setup with session data stored in Infinispan and all other entities stored in PostgreSQL.
+
See xref:storage/postgres-infinispan.adoc[] for more information.

`none`:: Deploy no data store.
+
See xref:storage/concurrent-hash-map.adoc[] for more information.
--

KC_DATABASE_URL::
Define the external endpoint of databases that are deployed external to the Kubernetes cluster.
+
Default value: empty


[[KC_STORAGE,KC_STORAGE]]
KC_STORAGE::
Set the storage configuration parameter for Keycloak.
+
Default value: empty
+
Available options:
+
--
(empty):: Deploys Keycloak with the legacy store.
This requires `KC_DATABASE` to be set to `postgres`.
`jpa`:: Deploys Keycloak with the new map store configured for JPA.
This requires `KC_DATABASE` to be set to `postgres` or `cockroach`.
+
See xref:storage/postgres.adoc[], xref:storage/cockroach-single.adoc[] or xref:storage/cockroach-operator.adoc[] for more information.

`hotrod`:: Deploys Keycloak with the new map store configured for Hot Rod.
This requires `KC_DATABASE` to be set to `infinispan`.
+
See xref:storage/infinispan.adoc[] for more information.

`jpa+hotrod`:: Deploys Keycloak with the new map store configured for Hot Rod for sessions and JPA for all other entities.
This requires `KC_DATABASE` to be set to `postgres+infinispan`.
+
See xref:storage/postgres-infinispan.adoc[] for more information.

`chm`:: Deploys Keycloak with the new map store configured with Concurrent Hash Map, which is an in-memory store that is not shared between Keycloak instances.
+
See xref:storage/concurrent-hash-map.adoc[] for more information.
--

[[KC_DB_POOL_INITIAL_SIZE,KC_DB_POOL_INITIAL_SIZE]]
KC_DB_POOL_INITIAL_SIZE::
Set the initial database size of the DB connection pool.
Expand Down
33 changes: 0 additions & 33 deletions doc/kubernetes/modules/ROOT/pages/error-messages.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -35,39 +35,6 @@ Caution::
Having Keycloak running into a "`Sorry, acquisition timeout`" and returning an HTTP 5xx code to the caller is a sensible load shedding mechanism.
See xref:running/concepts/database-connections.adoc[] for details.

[[retry-serializable]]
== Keycloak message `RETRY_SERIALIZABLE`

Full message::
+
[.wrap]
----
org.postgresql.util.PSQLException, ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_SERIALIZABLE - failed preemptive refresh due to a conflict: intent on key /Table/137/... See: https://www.cockroachlabs.com/docs/v22.1/transaction-retry-error-reference.html#retry_serializable
----

Context::
This error message can appear when running Keycloak with CockroachDB, both xref:storage/cockroach-single.adoc[single node] or xref:storage/cockroach-operator.adoc[multi node with the operator].
+
This might happen during the load test while Keycloak processes requests.

Cause::
Some transactions are not fully serializable as data has been modified in parallel transactions.

Effect::
The database rolls back the transaction and asks the caller to repeat the request.
Some users see error messages.

Remedy::
* Analyze the request or URL where this happens by looking at the log, and discuss this with engineers.
* Use the following SQL to find the table that is causing the problems. In the preceding example it would be `137`:
+
[source,sql]
----
SELECT DISTINCT ti.descriptor_name as table_name, us.table_id
FROM crdb_internal.index_usage_statistics us, crdb_internal.table_indexes ti
WHERE us.table_id = ti.descriptor_id ORDER BY us.table_id ASC;
----

== Keycloak message `prepared transactions are disabled`

Full message::
Expand Down
2 changes: 1 addition & 1 deletion doc/kubernetes/modules/ROOT/pages/other.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -95,7 +95,7 @@ Add local customizations via `keycloak/values.yaml`:

* Set `monitoring` to `false` to install Keycloak without monitoring options is.

* Set `disableCaches` to `true` to disable caches in the Keycloak's legacy store.
* Set `disableCaches` to `true` to disable caches in the Keycloak's store.

== Pause/Resume setup

Expand Down
30 changes: 3 additions & 27 deletions doc/kubernetes/modules/ROOT/pages/storage-configurations.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,7 @@

{description}

Keycloak provides two storage implementations:

* Legacy Storage
* New Storage (also called Map Storage)
This setup supports different databases.

== Default setup

Expand All @@ -18,36 +15,15 @@ See xref:storage/postgres.adoc[Accessing the PostgreSQL database inside minikube

The following table lists the different storages:

[%autowidth,cols=",^,^"]
[%autowidth,cols=",^"]
|===
|Storage |Legacy Storage |Map Storage
|Storage |Current Storage

|xref:storage/postgres.adoc[PostgreSQL]
|✅
|✅

|xref:storage/aurora-regional-postgres.adoc[Aurora Regional PostgreSQL] or +
xref:storage/aurora-global-postgres.adoc[Aurora Global PostgreSQL]
|✅
|✅

|xref:storage/cockroach-single.adoc[CockroachDB Single]
|
|✅

|xref:storage/cockroach-single.adoc[CockroachDB Operator]
|
|✅

|xref:storage/infinispan.adoc[Infinispan]
|
|✅

|xref:storage/postgres-infinispan.adoc[PostgreSQL + Infinispan]
|
|✅

|xref:storage/concurrent-hash-map.adoc[Concurrent Hash Map]
|
|✅
|===
Loading

0 comments on commit 1405b51

Please sign in to comment.