Vault provider implementation

[ramizpolic]

Summary

To add Vault provider in a way that is backwards compatible between secret-init and vault-env, we need to consider the way they handle configuration flows.

Vault-env

1. Solution specific configs use VAULT_ENV_* env format with the exceptions of VAULT_LOG_LEVEL and VAULT_LOG_JSON

2. Provider specific configs use VAULT_* env format (note that the only supported provider is Vault)

Secret-init

1. Solution specific configs should use SECRET_INIT_* env format

Backwards compatibility will not be respected for these configs between vault-env due to standard differences.
For example, using VAULT_ENV_* (or even VAULT_LOG_LEVEL) to configure secret-init does not make much sense.
We can easily compile a list of different vars between the two solutions for tracking.

2. Provider specific configs should use {PROVIDER}_ env format. In addition, {provider}: prefix in a given env variable value indicates which provider needs to perform the injection/env substituion.

File provider can use FILE_ env name prefix for its configs and file: env value prefix to indicate its usage. For example:

• Configuration: FILE_MOUNT_PATH=/secrets
• Usage: INJECTED_VIA_FILE=file:example (note that the actual host path is /secrets/example)

Vault provider can use VAULT_ env name prefix for its configs and vault: env value prefix to indicate its usage. For example:

• Configuration: VAULT_ADDR=http://vault, VAULT_TOKEN_FILE=/path/to/token
• Usage: INJECTED_VIA_VAULT=vault:secret/data/example#KEY

With this approach, we preserve backwards compatibility between vault-env and secret-init related to the actual Vault provider.
However, please note that other env value prefixes are supported by vault-sdk/injector (e.g. >>vault:) which needs to be further checked.

Notes

• How to handle blacklistable envs per provider? For example, Vault blacklists certain evs from injection.
  • We can keep track of the blacklisted envs as part of provider configs. Each provider can then define its own list of blacklisted envs. Since we will keep track of all the created/used providers in one place, we can easily get the overall list of all blacklisted envs.
• Only one instance of the same provider can be used (i.e. we can have both file and vault provider in a single run, but not two different vault providers).
  • Not feasible to support multiple instance of the same provider since we use explicit envs to control per-provider configuration, same as in vault-env. Changing it would require redesigning the whole flow.

Implementation

1. Change the solution specific envs to use the new format (SECRET_INIT_)
2. Make the providers take control of their configs. For example:

// file: provider/file/config.go

type Config struct {
	MountPath string `json:"mountPath"` // loaded from FILE_MOUNT_PATH env
}

func LoadConfig() (*Config, error) {} // load

// file: provider/file/file.go

func NewProvider(cfg *Config) (provider.Provider, error) {} // create

3. TODO: @ramizpolic add steps on how to handle creation of multiple providers and how to handle blacklisted envs for each provider

[csatib02]

I also found 2 environment variable that uses solution-specific formatting, but it is provider-specific:

VAULT_ENV_FROM_PATH:

"Comma-delimited list of vault paths to pull in all secrets as environment variables, which also supports versioning (since vault-env v1.21.1)."

VAULT_ENV_PASSTHROUGH:

"Comma separated list of VAULT_* related environment variables to pass through to vault-env to the main process. E.g. VAULT_ADDR,VAULT_ROLE."

Source: BV-Docs

It should be changed to provider-specific formatting, therefore used with VAULT_ prefix.