Add a how-to guide for configuring high availability and failover

[bgravenorst]

Summary

Web3Signer has no documentation on running more than one instance. Add a how-to guide covering high availability and failover, so operators can keep validators signing through the loss of a single instance without risking a double-signing penalty.

Suggested location: docs/how-to/configure-high-availability.md.

Why this is needed

Running a single Web3Signer instance makes it a single point of failure. If it stops, validators miss attestations and block proposals until it recovers.

The obvious fix, running more instances, is also the dangerous one. With most services extra replicas only add resilience. With validator signing, two instances that sign the same attestation or block cause a slashable offense. Web3Signer prevents this when instances share one PostgreSQL slashing protection database, because database locking ensures only one instance signs for a given key.

That relationship is the single most important operational concept for anyone running Web3Signer on mainnet, and it is currently spread thinly across the architecture, slashing protection, and run-at-scale pages rather than explained in one place. Operators looking for the answer today have to find it in third-party blog posts and cloud vendor reference architectures.

Getting this wrong causes real financial loss, so the guidance should come from the official documentation.

What the page should cover

• Why you would run more than one instance, and honest guidance on who does not need to. Solo and home stakers running a single validator are usually better served by a single instance.
• How Web3Signer stays safe with multiple instances: all instances load the same keys and connect to the same PostgreSQL slashing protection database, and database locking prevents a conflicting message from being signed.
• A clear warning never to run multiple instances without a shared slashing protection database, and never to disable slashing protection to improve performance.
• The recommended topology, including whether instances run active-active behind a load balancer or active-passive.
• A topology diagram. Store new images in static/img/ and reference them with a site path such as /img/<file-name>.
• Setup steps: provision the shared database, start each instance with the same keys and database connection details, and place the instances behind a load balancer.
• Load balancer health checks using the /upcheck and /healthcheck endpoints, and pointing the validator client at the load balancer rather than an individual instance.
• Monitoring guidance, including the slashing protection database health check interval and the relevant metrics.
• How to perform a rolling upgrade or maintenance without downtime, including draining one instance at a time and keeping at least one healthy instance signing.
• A note for Kubernetes users about using a PodDisruptionBudget so node drains and cluster upgrades cannot remove all signing instances at once.
• Securing and backing up the shared database, since it is both the safety boundary and a critical dependency.
• Links to related pages: slashing protection, configure slashing protection, run Web3Signer at scale, metrics, and private key management best practices.