Vault executing revocation statements before postgresql credentials expiry

[rusanki]

Describe the bug
We have a database secret with default and max ttl of 72000h but vault is running revocation statements after 32 days.

To Reproduce

Not able to reproduce this anywhere else. We had default ttl and max ttl kept at default 768h but a few months back we updated it to 72000h to avoid expiring leases. Now we have faced this behaviour for 3rd time. We verified the database user as well which is having extended 72000h of expiry but can see a recovacation statement which alters the user property to NOLOGIN ran after 32 days period.

Expected behavior
We would expect REVOCATION STATEMENTS to have never run because of extended lease expiry.

Environment:

• Vault Server Version (retrieve with vault status): 1.12.1
• Vault CLI Version (retrieve with vault version): 1.14.1
• Server Operating System/Architecture: Linux amd64

Vault server configuration file(s):

# Paste your Vault config here.
# Be sure to scrub any sensitive values
ui = true
log_level = "Debug"

retry_join {
  leader_tls_servername = "vault-server-tls"
}

listener "tcp" {
  tls_disable = 0
  address = "[::]:8200"
  cluster_address = "[::]:8201"
  tls_cert_file = "/vault/userconfig/vault-server-tls/tls.crt"
  tls_key_file  = "/vault/userconfig/vault-server-tls/tls.key"
  tls_client_ca_file = "/vault/userconfig/vault-server-tls/ca.crt"

}
seal "awskms" {
  region     = "ap-south-1"
  kms_key_id = "<id>"
}
storage "postgresql" {
  ha_enabled = "true"
}

Additional context
We now will remove the revocation statement to avoid the recurrence but wanted to highlight the behaviour and a possible bug.

[miagilepner]

Hi, I can't seem to reproduce this with the information given. TTLs take into account both the mount TTLs and the role TTLs. Did you update both to 72000h? Additionally, if you try to read the credentials, what value do you see for lease_duration?

[rusanki]

@miagilepner ,

mount ttl is 720000h (corrected)
role ttl is is 120000h

This also happened a day later with another vault deployment where lease reached the 32 days period. Thankfully we were able to update the revocation statements to not disable login of the user bcs of the outage that happened earlier and avoided it for this other env.
Similar behaviour where revocation statement was run even when the lease duration was 120000h, postgres credentials is also showing the similar expiry for 2038. same configuration is applied there as well, i.e. with extended mount and role ttl

[rusanki]

Hi @miagilepner , anything else that can help on this issue ? we checked everything e2e multiple times but could not figure out why this would happen. And pls note that we have been using this configuration in production for 2 years but kept ignoring this behaviour for sometime and now completely relying on dummy revocation statements to remain unaffected from this behaviour but this is very scary in general to witness and proceed any further or resume back on relying for revocation statements

[miagilepner]

@rusanki What is the TTL of the token that is used to create the database credentials? If the token that was used to create the credentials has expired, then the credentials will be revoked because it is assumed that the client to whom that token belonged no longer has access, so the credentials that they have received should also no longer be valid. See the leases concepts page:

When a token is revoked, Vault will revoke all leases that were created using that token.

[rusanki]

We have an extended expiry of 13 years for the database role and also verified all the leases as well for the role before and after the issue multiple times.

[rusanki]

can I do anything further here ?