Merge branch 'testnet-3' into validator-deadlines

.gitignore

@@ -12,3 +12,4 @@ GIT_SHA
 build
 database
 *.db
+.DS_Store

Dockerfile

@@ -35,7 +35,7 @@ ENV PATH="$POETRY_HOME/bin:$VENV_PATH/bin:/root/.cargo/bin:$PATH"
 FROM python-base as builder-base
 
 RUN apk upgrade --no-cache
-RUN apk add --no-cache gcc musl-dev python3-dev libffi-dev openssl-dev curl libgcc libstdc++
+RUN apk add --no-cache gcc musl-dev python3-dev libffi-dev openssl-dev curl libgcc libstdc++ postgresql-libs postgresql-dev
 RUN curl https://sh.rustup.rs -sSf | \
     sh -s -- --default-toolchain stable -y
 RUN rm -rf /var/cache/apt/*

README.md

@@ -352,6 +352,80 @@ Keystores for vault {vault} successfully recovered to {keystores_dir}
 > Note: For security purposes, make sure to protect your mnemonic as it can be used to generate your validator keys.
 > Always verify the network and endpoints before running the command.
 
+### Web3Signer infrastructure commands
+
+#### 1. Update database
+
+The command encrypts and loads validator keys from keystore files into the database
+
+```bash
+./v3-operator update-db --db-url postgresql://postgres:postgres@localhost:5432/web3signer --keystores-dir ./data/keystores --keystores-password-file ./data/keystores/password.txt
+Loading keystores...              [####################################]  10/10
+Encrypting database keys...
+Generated 10 validator keys, upload them to the database? [Y/n]: Y
+The database contains 10 validator keys.
+Save decryption key: '<DECRYPTION KEYS>'
+```
+
+##### update-db options
+
+- `--keystores-dir` - The directory with validator keys in the EIP-2335 standard. Defaults to ./data/keystores.
+- `--keystores-password-file` - The path to file with password for encrypting the keystores. Defaults to ./data/keystores/password.txt.
+- `--db-url` - The database connection address.
+- `--encryption-key` - The key for encrypting database record. If you are upload new keystores use the same encryption key.
+- `--no-confirm` - Skips confirmation messages when provided.
+
+**NB! You must store the decryption key in a secure place.
+It will allow you to upload new keystores in the existing database**
+
+#### 2. Sync validator configs
+
+Creates validator configuration files for Lighthouse, Prysm, and Teku clients to sign data using keys form database.
+
+```bash
+./v3-operator sync-validator
+Enter the recipient address for MEV & priority fees: 0xB31...1
+Enter the endpoint of the web3signer service: https://web3signer-example.com
+Enter the database connection string, ex. 'postgresql://username:pass@hostname/dbname': postgresql://postgres:postgres@localhost/web3signer
+Enter the total number of validators connected to the web3signer: 30
+Enter the validator index to generate the configuration files: 5
+
+Done. Generated configs with 50 keys for validator #5.
+Validator definitions for Lighthouse saved to data/configs/validator_definitions.yml file.
+Signer keys for Teku\Prysm saved to data/configs/signer_keys.yml file.
+Proposer config for Teku\Prysm saved to data/configs/proposer_config.json file.
+```
+
+##### sync-validator options
+
+- `--validator-index` - The validator index to generate the configuration files.
+- `--total-validators` - The total number of validators connected to the web3signer.
+- `--db-url` - The database connection address.
+- `--web3signer-endpoint` - The endpoint of the web3signer service.
+- `--fee-recipient` - The recipient address for MEV & priority fees.
+- `--disable-proposal-builder` - Disable proposal builder for Teku and Prysm clients.
+- `--output-dir` - The directory to save configuration files. Defaults to ./data/configs.
+
+#### 3. Sync Web3Signer config
+
+The command is running by the init container in web3signer pods.
+Fetch and decrypt keys for web3signer and store them as keypairs in the output_dir.
+
+Set `DECRYPTION_KEY` env, use value generated by `update-db` command
+
+```bash
+./v3-operator sync-web3signer
+Enter the folder where web3signer keystores will be saved: /data/web3signer
+Enter the database connection string, ex. 'postgresql://username:pass@hostname/dbname': postgresql://postgres:postgres@localhost/web3signer
+Web3Signer now uses 7 private keys.
+```
+
+##### sync-web3signer options
+
+- `--db-url` - The database connection address.
+- `--output-dir` - The folder where Web3Signer keystores will be saved.
+- `--decryption-key-env` - The environment variable with the decryption key for private keys in the database.
+
 ## Monitoring Operator with Prometheus
 
 Operator supports monitoring using Prometheus by providing a `/metrics` endpoint that Prometheus can scrape to gather