Skip to content

Commit

Permalink
chore(docs): Document WAL window configuration options (#1050)
Browse files Browse the repository at this point in the history
  • Loading branch information
alco committed May 7, 2024
1 parent 6027c2d commit 6840f94
Show file tree
Hide file tree
Showing 3 changed files with 30 additions and 0 deletions.
7 changes: 7 additions & 0 deletions docs/api/_ELECTRIC_RESUMABLE_WAL_WINDOW.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
Maximum size that the sync service is allowed to use for keeping around old WAL records in Postgres. Measured in bytes unless one of the following suffixes is used: `k` for KB; `K` for KiB; `m` for MB; `M` for MiB; `g` for GB; `G` for GiB.

Normally, Postgres discards WAL records as soon as they are acknowledged by the replica. However, for the sync service to be able to restore its caches after a restart, it needs to hold on to old WAL records since those may contain transactions that affect electrified tables.

Setting this to a low value may lead to clients having to discard their local copy of the server state and restart their replication stream from scratch.

_In the future Electric may store change diffs or employ techniques for compacting the retained WAL records with the goal of extending the time frame during which a client may catch up to the latest server state without discarding its local state._
3 changes: 3 additions & 0 deletions docs/api/_ELECTRIC_TXN_CACHE_SIZE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
Size of the in-memory cache for transactions the sync service consumes from Postgres over the logical replication stream. Measured in bytes unless one of the following suffixes is used: `k` for KB; `K` for KiB; `m` for MB; `M` for MiB; `g` for GB; `G` for GiB.

This cache is used to quickly catch up newly connected clients to the current state in Postgres. When the sync service restarts, it will repopulate this in-memory cache by streaming already committed transactions from retained Postgres' WAL records, going as far back in time as the size of the WAL window configured with `ELECTRIC_RESUMABLE_WAL_WINDOW` allows.
20 changes: 20 additions & 0 deletions docs/api/service.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,8 @@ import DatabaseUrl from './_DATABASE_URL.md'
import DatabaseUseIpv6 from './_DATABASE_USE_IPV6.md'
import ElectricUseIpv6 from './_ELECTRIC_USE_IPV6.md'
import HttpPort from './_HTTP_PORT.md'
import ElectricResumableWalWindow from './_ELECTRIC_RESUMABLE_WAL_WINDOW.md'
import ElectricTxnCacheSize from './_ELECTRIC_TXN_CACHE_SIZE.md'
import LogLevel from './_LOG_LEVEL.md'


Expand Down Expand Up @@ -103,6 +105,24 @@ Configure how Electric connects to Postgres and exposes its HTTP/WebSocket API.
<HttpPort />
</EnvVarConfig>

### ELECTRIC_RESUMABLE_WAL_WINDOW

<EnvVarConfig
name="ELECTRIC_RESUMABLE_WAL_WINDOW"
defaultValue="2G"
example="800M">
<ElectricResumableWalWindow />
</EnvVarConfig>

### ELECTRIC_TXN_CACHE_SIZE

<EnvVarConfig
name="ELECTRIC_TXN_CACHE_SIZE"
defaultValue="256M"
example="2g">
<ElectricTxnCacheSize />
</EnvVarConfig>

### LOG_LEVEL

<EnvVarConfig
Expand Down

0 comments on commit 6840f94

Please sign in to comment.