GHA runner suddenly not authorized to perform sts:AssumeRoleWithWebIdentity

[davidristov]

Describe the bug

All workflows suddenly started failing on the aws-actions/configure-aws-credentials step with the following error:
Not authorized to perform sts:AssumeRoleWithWebIdentity

From the workflows history, we had everything running up until Wednesday last week (Oct 18) and started seeing the issue from Friday (Oct 20). In between there are no relevant configuration changes that affect these resources (IAM role, OIDC provider).

Expected Behavior

IAM role gets assumed successfully by the GitHub Actions runner.

Current Behavior

GitHub Actions runner is not authorized to assume the defined IAM role.

Reproduction Steps

GitHub workflow:

name: Development deploy

on:
  workflow_dispatch:

env:
  AWS_REGION: 'eu-central-1'

permissions:
  id-token: write
  contents: read

jobs:
  my-job:
    runs-on: ubuntu-latest
    environment:
      name: dev

    steps:
    - name: Checkout Code
      uses: actions/checkout@v3
      with:
        submodules: recursive
        token: ${{ secrets.SUBMODULES_PAT }}

    - name: Configure AWS Credentials
      uses: aws-actions/configure-aws-credentials@v3
      with:
        role-to-assume: ${{ vars.GHA_ROLE }}
        aws-region: ${{ env.AWS_REGION }}
        role-session-name: DevSession

Both vars.GHA_ROLE and env.AWS_REGION have the correct values.

Trust policy on IAM role:

{
	"Version": "2012-10-17",
	"Statement": [
		{
			"Effect": "Allow",
			"Principal": {
				"Federated": "arn:aws:iam::<AWS_ACCOUNT>:oidc-provider/token.actions.githubusercontent.com"
			},
			"Action": "sts:AssumeRoleWithWebIdentity",
			"Condition": {
				"StringEquals": {
					"token.actions.githubusercontent.com:aud": "sts.amazonaws.com"
				},
				"ForAnyValue:StringLike": {
					"token.actions.githubusercontent.com:sub": [
						"repo:<GITHUB_ORG>/<GITHUB_REPO>:*",
						"repo:<GITHUB_ORG>/<GITHUB_REPO>:*",
						"repo:<GITHUB_ORG>/<GITHUB_REPO>:*"
					]
				}
			}
		}
	]
}

AWS OIDC provider URL: https://token.actions.githubusercontent.com
Audience: sts.amazonaws.com
Thumbprints: 6938fd4d98bab03faadb97b34396831e3780aea1, 1c58a3a8518e8759bf075b76b750d4f2df264fcd, 1b511abead59c6ce207077c0bf0e0043b1382612

Possible Solution

No response

Additional Information/Context

No response

[altaurog]

I have a similar problem. The CloudTrail event reports "An unknown error occurred"

    "eventVersion": "1.08",
    "userIdentity": {
        "type": "WebIdentityUser",
        "principalId": "arn:aws:iam::xxxxxxxxxxxx:oidc-provider/token.actions.githubusercontent.com:sts.amazonaws.com:repo:myorg/myrepo:environment:dev",
        "userName": "repo:myorg/myrepo:environment:dev",
        "identityProvider": "arn:aws:iam::xxxxxxxxxxxx:oidc-provider/token.actions.githubusercontent.com"
    },
    "eventTime": "2023-10-24T11:23:38Z",
    "eventSource": "sts.amazonaws.com",
    "eventName": "AssumeRoleWithWebIdentity",
    "awsRegion": "us-east-1",
    "sourceIPAddress": "13.91.166.0",
    "userAgent": "aws-sdk-js/3.423.0 ua/2.0 os/linux#6.2.0-1014-azure lang/js md/nodejs#20.5.0 api/sts#3.423.0 configure-aws-credentials-for-github-actions",
    "errorCode": "AccessDenied",
    "errorMessage": "An unknown error occurred",
    "requestParameters": {
        "roleArn": "arn:aws:iam::xxxxxxxxxxxx:role/myrole",
        "roleSessionName": "GitHubActions",
        "durationSeconds": 3600
    },
    "responseElements": null,
    "requestID": "599caa41-968b-4bb9-adbd-bd856f606fe1",
    "eventID": "70b61a4d-2ec7-4ca1-87af-f36572d6dd2a",
    "readOnly": true,
    "resources": [
        {
            "accountId": "xxxxxxxxxxxx",
            "type": "AWS::IAM::Role",
            "ARN": "arn:aws:iam::xxxxxxxxxxxx:role/myrole"
        }
    ],
    "eventType": "AwsApiCall",
    "managementEvent": true,
    "recipientAccountId": "xxxxxxxxxxxx",
    "eventCategory": "Management",
    "tlsDetails": {
        "tlsVersion": "TLSv1.2",
        "cipherSuite": "ECDHE-RSA-AES128-GCM-SHA256",
        "clientProvidedHostHeader": "sts.us-east-1.amazonaws.com"
    }
}

[davidristov]

I have the same CloudTrail event.

[altaurog]

My bad, I’d recreated the roles via cloudformation and the names had changed. @davidristov may have a real issue, but mine was a configuration error, now resolved.

[davidristov]

I just got it resolved and it was a configuration error as well. GitHub organization name had changed from capital first letter to lowercase making the IAM role trust policy invalid.

Initially, I checked for new thumbprints which was the case but after adding it the issue still persisted. I assumed it was somewhat related similarly like the issue a few months ago. Closing issue.

[github-actions]

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.