JWT Signature verification failed

[TyranVdMerwePaytec]

I am running into the following issue when trying to run DAGs after the API server pod has restarted (OOM, Evicted, etc):

`
Failed to validate JWT [airflow.api_fastapi.execution_api.deps] token=redacted
/home/airflow/.local/lib/python3.12/site-packages/airflow/api_fastapi/execution_api/deps.py:86 in call
/home/airflow/.local/lib/python3.12/site-packages/airflow/api_fastapi/auth/tokens.py:327 in avalidated_claims
/home/airflow/.local/lib/python3.12/site-packages/jwt/api_jwt.py:222 in decode
/home/airflow/.local/lib/python3.12/site-packages/jwt/api_jwt.py:156 in decode_complete
/home/airflow/.local/lib/python3.12/site-packages/jwt/api_jws.py:220 in decode_complete
/home/airflow/.local/lib/python3.12/site-packages/jwt/api_jws.py:328 in _verify_signature

InvalidSignatureError: Signature verification failed
`

The jwt secret (key) is set by the Helm install and does not change.

The only way to resolve the issue is to restart the scheduler pod.

Any ideas or advice would be appreciated.

[Blarc]

I have the same problem and I think I found the answer. Whenever you do helm upgrade the secret with the api-secret-key is re-created because the helm chart uses:
api-secret-key: {{ (.Values.apiSecretKey) | default (randAlphaNum 32) | b64enc | quote }}

But this value is not propagated to the pods until the pods are restarted. The helm upgrade command restarts only the pods that were changed.

I think the solution is to set the apiSecretKey in values.yaml so it doesn't change on each helm upgrade.