Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Cannot access IAM via CLI in GovCloud with short-term credentials. #8918

Closed
ranok opened this issue Sep 12, 2024 · 2 comments
Closed

Cannot access IAM via CLI in GovCloud with short-term credentials. #8918

ranok opened this issue Sep 12, 2024 · 2 comments
Labels
bug This issue is a bug. needs-triage This issue or PR still needs to be triaged.

Comments

@ranok
Copy link

ranok commented Sep 12, 2024

Describe the bug

I am trying to get an IAM role details for a role in GovCloud (specified region as us-gov-west-1) while using a cli that's configured with short-term credentials (ASIA...) and an aws_session_token set. I can perform API queries to other services (e.g., STS, S3, Lambda), but IAM throws the following error (also tried via boto3):
botocore.exceptions.ClientError: An error occurred (InvalidClientTokenId) when calling the GetRole operation: The security token included in the request is invalid

I have verified that the endpoint is correct (iam.us-gov.amazonaws.com), and when I created the short term credentials with sts get-session-token I specified the region.

When I use permanent access credentials, this works fine, but the short term credentials fail for IAM specifically.

Expected Behavior

Like when aws cli is configured with static, long-term creds, I expect to see the role details for the specified role.

Current Behavior

aws iam get-role --role-name ROLE_NAME --debug --region us-gov-west-1
2024-09-12 14:08:40,953 - MainThread - awscli.clidriver - DEBUG - CLI version: aws-cli/2.15.25 Python/3.11.8 Darwin/23.6.0 exe/x86_64
2024-09-12 14:08:40,955 - MainThread - awscli.clidriver - DEBUG - Arguments entered to CLI: ['iam', 'get-role', '--role-name', 'ROLE_NAME, '--debug', '--region', 'us-gov-west-1']
2024-09-12 14:08:40,972 - MainThread - botocore.hooks - DEBUG - Event building-command-table.main: calling handler <function add_s3 at 0x11067eca0>
2024-09-12 14:08:40,972 - MainThread - botocore.hooks - DEBUG - Event building-command-table.main: calling handler <function add_ddb at 0x1104cac00>
2024-09-12 14:08:40,972 - MainThread - botocore.hooks - DEBUG - Event building-command-table.main: calling handler <bound method BasicCommand.add_command of <class 'awscli.customizations.configure.configure.ConfigureCommand'>>
2024-09-12 14:08:40,972 - MainThread - botocore.hooks - DEBUG - Event building-command-table.main: calling handler <function change_name at 0x110426840>
2024-09-12 14:08:40,972 - MainThread - botocore.hooks - DEBUG - Event building-command-table.main: calling handler <function change_name at 0x1104562a0>
2024-09-12 14:08:40,972 - MainThread - botocore.hooks - DEBUG - Event building-command-table.main: calling handler <function alias_opsworks_cm at 0x1106b1760>
2024-09-12 14:08:40,972 - MainThread - botocore.hooks - DEBUG - Event building-command-table.main: calling handler <function add_history_commands at 0x110531620>
2024-09-12 14:08:40,972 - MainThread - botocore.hooks - DEBUG - Event building-command-table.main: calling handler <bound method BasicCommand.add_command of <class 'awscli.customizations.devcommands.CLIDevCommand'>>
2024-09-12 14:08:40,973 - MainThread - botocore.hooks - DEBUG - Event building-command-table.main: calling handler <function add_waiters at 0x1106b1620>
2024-09-12 14:08:40,973 - MainThread - botocore.hooks - DEBUG - Event building-command-table.main: calling handler <bound method AliasSubCommandInjector.on_building_command_table of <awscli.alias.AliasSubCommandInjector object at 0x110774410>>
2024-09-12 14:08:40,973 - MainThread - botocore.loaders - DEBUG - Loading JSON file: /usr/local/aws-cli/awscli/data/cli.json
2024-09-12 14:08:40,974 - MainThread - botocore.hooks - DEBUG - Event top-level-args-parsed: calling handler <function resolve_types at 0x1105dafc0>
2024-09-12 14:08:40,974 - MainThread - botocore.hooks - DEBUG - Event top-level-args-parsed: calling handler <function no_sign_request at 0x1105db2e0>
2024-09-12 14:08:40,974 - MainThread - botocore.hooks - DEBUG - Event top-level-args-parsed: calling handler <function resolve_verify_ssl at 0x1105db240>
2024-09-12 14:08:40,974 - MainThread - botocore.hooks - DEBUG - Event top-level-args-parsed: calling handler <function resolve_cli_read_timeout at 0x1105db420>
2024-09-12 14:08:40,974 - MainThread - botocore.hooks - DEBUG - Event top-level-args-parsed: calling handler <function resolve_cli_connect_timeout at 0x1105db380>
2024-09-12 14:08:40,974 - MainThread - botocore.hooks - DEBUG - Event top-level-args-parsed: calling handler <built-in method update of dict object at 0x110771700>
2024-09-12 14:08:40,974 - MainThread - botocore.session - DEBUG - Setting config variable for region to 'us-gov-west-1'
2024-09-12 14:08:40,974 - MainThread - awscli.clidriver - DEBUG - CLI version: aws-cli/2.15.25 Python/3.11.8 Darwin/23.6.0 exe/x86_64 prompt/off
2024-09-12 14:08:40,974 - MainThread - awscli.clidriver - DEBUG - Arguments entered to CLI: ['iam', 'get-role', '--role-name', 'ROLE_NAME', '--debug', '--region', 'us-gov-west-1']
2024-09-12 14:08:40,974 - MainThread - botocore.hooks - DEBUG - Event session-initialized: calling handler <function add_timestamp_parser at 0x11067f600>
2024-09-12 14:08:40,974 - MainThread - botocore.hooks - DEBUG - Event session-initialized: calling handler <function register_uri_param_handler at 0x10fd3ed40>
2024-09-12 14:08:40,974 - MainThread - botocore.hooks - DEBUG - Event session-initialized: calling handler <function add_binary_formatter at 0x110732e80>
2024-09-12 14:08:40,974 - MainThread - botocore.hooks - DEBUG - Event session-initialized: calling handler <function no_pager_handler at 0x10fc30fe0>
2024-09-12 14:08:40,974 - MainThread - botocore.hooks - DEBUG - Event session-initialized: calling handler <function inject_assume_role_provider_cache at 0x10fd5a660>
2024-09-12 14:08:40,976 - MainThread - botocore.utils - DEBUG - IMDS ENDPOINT: http://169.254.169.254/
2024-09-12 14:08:40,984 - MainThread - botocore.hooks - DEBUG - Event session-initialized: calling handler <function attach_history_handler at 0x110509ee0>
2024-09-12 14:08:40,984 - MainThread - botocore.hooks - DEBUG - Event session-initialized: calling handler <function inject_json_file_cache at 0x1104b2e80>
2024-09-12 14:08:40,999 - MainThread - botocore.loaders - DEBUG - Loading JSON file: /usr/local/aws-cli/awscli/botocore/data/iam/2010-05-08/service-2.json
2024-09-12 14:08:41,007 - MainThread - botocore.hooks - DEBUG - Event building-command-table.iam: calling handler <function _add_wizard_command at 0x110731bc0>
2024-09-12 14:08:41,007 - MainThread - botocore.hooks - DEBUG - Event building-command-table.iam: calling handler <function add_waiters at 0x1106b1620>
2024-09-12 14:08:41,020 - MainThread - botocore.loaders - DEBUG - Loading JSON file: /usr/local/aws-cli/awscli/botocore/data/iam/2010-05-08/waiters-2.json
2024-09-12 14:08:41,020 - MainThread - botocore.hooks - DEBUG - Event building-command-table.iam: calling handler <bound method AliasSubCommandInjector.on_building_command_table of <awscli.alias.AliasSubCommandInjector object at 0x110774410>>
2024-09-12 14:08:41,020 - MainThread - awscli.clidriver - DEBUG - OrderedDict([('role-name', <awscli.arguments.CLIArgument object at 0x110a3fed0>)])
2024-09-12 14:08:41,020 - MainThread - botocore.hooks - DEBUG - Event building-argument-table.iam.get-role: calling handler <function add_streaming_output_arg at 0x11067fb00>
2024-09-12 14:08:41,020 - MainThread - botocore.hooks - DEBUG - Event building-argument-table.iam.get-role: calling handler <function add_cli_input_json at 0x10fd5afc0>
2024-09-12 14:08:41,020 - MainThread - botocore.hooks - DEBUG - Event building-argument-table.iam.get-role: calling handler <function add_cli_input_yaml at 0x10fd5b060>
2024-09-12 14:08:41,020 - MainThread - botocore.hooks - DEBUG - Event building-argument-table.iam.get-role: calling handler <function unify_paging_params at 0x1104cb240>
2024-09-12 14:08:41,034 - MainThread - botocore.loaders - DEBUG - Loading JSON file: /usr/local/aws-cli/awscli/botocore/data/iam/2010-05-08/paginators-1.json
2024-09-12 14:08:41,034 - MainThread - botocore.hooks - DEBUG - Event building-argument-table.iam.get-role: calling handler <function add_generate_skeleton at 0x1105d9800>
2024-09-12 14:08:41,034 - MainThread - botocore.hooks - DEBUG - Event before-building-argument-table-parser.iam.get-role: calling handler <bound method OverrideRequiredArgsArgument.override_required_args of <awscli.customizations.cliinput.CliInputJSONArgument object at 0x110a3c4d0>>
2024-09-12 14:08:41,034 - MainThread - botocore.hooks - DEBUG - Event before-building-argument-table-parser.iam.get-role: calling handler <bound method OverrideRequiredArgsArgument.override_required_args of <awscli.customizations.cliinput.CliInputYAMLArgument object at 0x110998c10>>
2024-09-12 14:08:41,034 - MainThread - botocore.hooks - DEBUG - Event before-building-argument-table-parser.iam.get-role: calling handler <bound method GenerateCliSkeletonArgument.override_required_args of <awscli.customizations.generatecliskeleton.GenerateCliSkeletonArgument object at 0x110a3cc90>>
2024-09-12 14:08:41,034 - MainThread - botocore.hooks - DEBUG - Event building-command-table.iam_get-role: calling handler <function add_waiters at 0x1106b1620>
2024-09-12 14:08:41,034 - MainThread - botocore.hooks - DEBUG - Event building-command-table.iam_get-role: calling handler <bound method AliasSubCommandInjector.on_building_command_table of <awscli.alias.AliasSubCommandInjector object at 0x110774410>>
2024-09-12 14:08:41,034 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.iam.get-role.role-name: calling handler <awscli.paramfile.URIArgumentHandler object at 0x1107a7750>
2024-09-12 14:08:41,034 - MainThread - botocore.hooks - DEBUG - Event process-cli-arg.iam.get-role: calling handler <awscli.argprocess.ParamShorthandParser object at 0x10fc5e350>
2024-09-12 14:08:41,034 - MainThread - awscli.arguments - DEBUG - Unpacked value of 'ROLE_NAME' for parameter "role_name": 'ROLE_NAME'
2024-09-12 14:08:41,034 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.iam.get-role.cli-input-json: calling handler <awscli.paramfile.URIArgumentHandler object at 0x1107a7750>
2024-09-12 14:08:41,034 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.iam.get-role.cli-input-yaml: calling handler <awscli.paramfile.URIArgumentHandler object at 0x1107a7750>
2024-09-12 14:08:41,034 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.iam.get-role.generate-cli-skeleton: calling handler <awscli.paramfile.URIArgumentHandler object at 0x1107a7750>
2024-09-12 14:08:41,034 - MainThread - botocore.hooks - DEBUG - Event calling-command.iam.get-role: calling handler <bound method CliInputArgument.add_to_call_parameters of <awscli.customizations.cliinput.CliInputJSONArgument object at 0x110a3c4d0>>
2024-09-12 14:08:41,034 - MainThread - botocore.hooks - DEBUG - Event calling-command.iam.get-role: calling handler <bound method CliInputArgument.add_to_call_parameters of <awscli.customizations.cliinput.CliInputYAMLArgument object at 0x110998c10>>
2024-09-12 14:08:41,034 - MainThread - botocore.hooks - DEBUG - Event calling-command.iam.get-role: calling handler <bound method GenerateCliSkeletonArgument.generate_skeleton of <awscli.customizations.generatecliskeleton.GenerateCliSkeletonArgument object at 0x110a3cc90>>
2024-09-12 14:08:41,035 - MainThread - botocore.credentials - DEBUG - Looking for credentials via: env
2024-09-12 14:08:41,035 - MainThread - botocore.credentials - DEBUG - Looking for credentials via: assume-role
2024-09-12 14:08:41,035 - MainThread - botocore.credentials - DEBUG - Looking for credentials via: assume-role-with-web-identity
2024-09-12 14:08:41,035 - MainThread - botocore.credentials - DEBUG - Looking for credentials via: sso
2024-09-12 14:08:41,035 - MainThread - botocore.credentials - DEBUG - Looking for credentials via: shared-credentials-file
2024-09-12 14:08:41,035 - MainThread - botocore.credentials - INFO - Found credentials in shared credentials file: ~/.aws/credentials
2024-09-12 14:08:41,036 - MainThread - botocore.loaders - DEBUG - Loading JSON file: /usr/local/aws-cli/awscli/botocore/data/endpoints.json
2024-09-12 14:08:41,044 - MainThread - botocore.hooks - DEBUG - Event choose-service-name: calling handler <function handle_service_name_alias at 0x10e8bea20>
2024-09-12 14:08:41,058 - MainThread - botocore.loaders - DEBUG - Loading JSON file: /usr/local/aws-cli/awscli/botocore/data/iam/2010-05-08/endpoint-rule-set-1.json
2024-09-12 14:08:41,059 - MainThread - botocore.loaders - DEBUG - Loading JSON file: /usr/local/aws-cli/awscli/botocore/data/partitions.json
2024-09-12 14:08:41,060 - MainThread - botocore.hooks - DEBUG - Event creating-client-class.iam: calling handler <function add_generate_presigned_url at 0x10e80ca40>
2024-09-12 14:08:41,060 - MainThread - botocore.configprovider - DEBUG - Looking for endpoint for iam via: environment_service
2024-09-12 14:08:41,060 - MainThread - botocore.configprovider - DEBUG - Looking for endpoint for iam via: environment_global
2024-09-12 14:08:41,060 - MainThread - botocore.configprovider - DEBUG - Looking for endpoint for iam via: config_service
2024-09-12 14:08:41,060 - MainThread - botocore.configprovider - DEBUG - Looking for endpoint for iam via: config_global
2024-09-12 14:08:41,060 - MainThread - botocore.configprovider - DEBUG - No configured endpoint found.
2024-09-12 14:08:41,061 - MainThread - botocore.regions - DEBUG - Using partition endpoint for iam, us-gov-west-1: aws-us-gov-global
2024-09-12 14:08:41,062 - MainThread - botocore.endpoint - DEBUG - Setting iam timeout as (60, 60)
2024-09-12 14:08:41,062 - MainThread - botocore.regions - DEBUG - Calling endpoint provider with parameters: {'Region': 'us-gov-west-1', 'UseDualStack': False, 'UseFIPS': False}
2024-09-12 14:08:41,063 - MainThread - botocore.regions - DEBUG - Endpoint provider result: https://iam.us-gov.amazonaws.com
2024-09-12 14:08:41,063 - MainThread - botocore.regions - DEBUG - Selecting from endpoint provider's list of auth schemes: "sigv4". User selected auth scheme is: "None"
2024-09-12 14:08:41,063 - MainThread - botocore.regions - DEBUG - Selected auth type "v4" as "v4" with signing context params: {'region': 'us-gov-west-1', 'signing_name': 'iam'}
2024-09-12 14:08:41,063 - MainThread - botocore.hooks - DEBUG - Event provide-client-params.iam.GetRole: calling handler <function base64_decode_input_blobs at 0x110732f20>
2024-09-12 14:08:41,063 - MainThread - botocore.hooks - DEBUG - Event before-parameter-build.iam.GetRole: calling handler <function generate_idempotent_uuid at 0x10e8e4d60>
2024-09-12 14:08:41,063 - MainThread - botocore.hooks - DEBUG - Event before-call.iam.GetRole: calling handler <function inject_api_version_header_if_needed at 0x10e8e6840>
2024-09-12 14:08:41,063 - MainThread - botocore.endpoint - DEBUG - Making request for OperationModel(name=GetRole) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'aws-cli/2.15.25 Python/3.11.8 Darwin/23.6.0 exe/x86_64 prompt/off command/iam.get-role'}, 'body': {'Action': 'GetRole', 'Version': '2010-05-08', 'RoleName': 'lambda-test-session-create-role'}, 'url': 'https://iam.us-gov.amazonaws.com/', 'context': {'client_region': 'aws-us-gov-global', 'client_config': <botocore.config.Config object at 0x110f506d0>, 'has_streaming_input': False, 'auth_type': 'v4', 'signing': {'region': 'us-gov-west-1', 'signing_name': 'iam'}, 'endpoint_properties': {'authSchemes': [{'name': 'sigv4', 'signingName': 'iam', 'signingRegion': 'us-gov-west-1'}]}}}
2024-09-12 14:08:41,063 - MainThread - botocore.hooks - DEBUG - Event request-created.iam.GetRole: calling handler <bound method RequestSigner.handler of <botocore.signers.RequestSigner object at 0x110e41990>>
2024-09-12 14:08:41,063 - MainThread - botocore.hooks - DEBUG - Event choose-signer.iam.GetRole: calling handler <function set_operation_specific_signer at 0x10e8e4c20>
2024-09-12 14:08:41,064 - MainThread - botocore.auth - DEBUG - Calculating signature using v4 auth.
2024-09-12 14:08:41,064 - MainThread - botocore.auth - DEBUG - CanonicalRequest:
POST
/

content-type:application/x-www-form-urlencoded; charset=utf-8
host:iam.us-gov.amazonaws.com
x-amz-date:20240912T200841Z
x-amz-security-token:REDACTED

content-type;host;x-amz-date;x-amz-security-token
74b72a3883b5c8a56c4470e82868fa93b8859579248627bf8dae9f2d63f3677c
2024-09-12 14:08:41,064 - MainThread - botocore.auth - DEBUG - StringToSign:
AWS4-HMAC-SHA256
20240912T200841Z
20240912/us-gov-west-1/iam/aws4_request
c55a81c0ff4e38601b5899b00a0aff945f0928a592270b5f4f983ab58af6a33d
2024-09-12 14:08:41,064 - MainThread - botocore.auth - DEBUG - Signature:
fec59f4db720406899d6b3844d15387136d527ff5bee49a38547c0d22893fc8b
2024-09-12 14:08:41,064 - MainThread - botocore.endpoint - DEBUG - Sending http request: <AWSPreparedRequest stream_output=False, method=POST, url=https://iam.us-gov.amazonaws.com/, headers={'Content-Type': b'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': b'aws-cli/2.15.25 Python/3.11.8 Darwin/23.6.0 exe/x86_64 prompt/off command/iam.get-role', 'X-Amz-Date': b'20240912T200841Z', 'X-Amz-Security-Token': b'REDACTED', 'Authorization': b'AWS4-HMAC-SHA256 Credential=ASIAREDACTED/20240912/us-gov-west-1/iam/aws4_request, SignedHeaders=content-type;host;x-amz-date;x-amz-security-token, Signature=fec59f4db720406899d6b3844d15387136d527ff5...', 'Content-Length': '74'}>
2024-09-12 14:08:41,065 - MainThread - botocore.httpsession - DEBUG - Certificate path: /usr/local/aws-cli/awscli/botocore/cacert.pem
2024-09-12 14:08:41,065 - MainThread - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): iam.us-gov.amazonaws.com:443
2024-09-12 14:08:41,484 - MainThread - urllib3.connectionpool - DEBUG - https://iam.us-gov.amazonaws.com:443 "POST / HTTP/1.1" 403 305
2024-09-12 14:08:41,484 - MainThread - botocore.parsers - DEBUG - Response headers: {'Date': 'Thu, 12 Sep 2024 20:08:40 GMT', 'x-amzn-RequestId': '6ca66a39-87b0-4d36-aab9-4bff707642a2', 'Content-Type': 'text/xml', 'Content-Length': '305'}
2024-09-12 14:08:41,484 - MainThread - botocore.parsers - DEBUG - Response body:
b'<ErrorResponse xmlns="https://iam.amazonaws.com/doc/2010-05-08/">\n  <Error>\n    <Type>Sender</Type>\n    <Code>InvalidClientTokenId</Code>\n    <Message>The security token included in the request is invalid</Message>\n  </Error>\n  <RequestId>6ca66a39-87b0-4d36-aab9-4bff707642a2</RequestId>\n</ErrorResponse>\n'
2024-09-12 14:08:41,489 - MainThread - botocore.hooks - DEBUG - Event needs-retry.iam.GetRole: calling handler <bound method RetryHandler.needs_retry of <botocore.retries.standard.RetryHandler object at 0x1109aa5d0>>
2024-09-12 14:08:41,490 - MainThread - botocore.retries.standard - DEBUG - Not retrying request.
2024-09-12 14:08:41,490 - MainThread - botocore.hooks - DEBUG - Event after-call.iam.GetRole: calling handler <function json_decode_policies at 0x10e8e59e0>
2024-09-12 14:08:41,490 - MainThread - botocore.hooks - DEBUG - Event after-call.iam.GetRole: calling handler <bound method RetryQuotaChecker.release_retry_quota of <botocore.retries.standard.RetryQuotaChecker object at 0x110f0b450>>
2024-09-12 14:08:41,490 - MainThread - awscli.clidriver - DEBUG - Exception caught in main()
Traceback (most recent call last):
  File "awscli/clidriver.py", line 460, in main
  File "awscli/clidriver.py", line 595, in __call__
  File "awscli/clidriver.py", line 798, in __call__
  File "awscli/clidriver.py", line 929, in invoke
  File "awscli/clidriver.py", line 941, in _make_client_call
  File "awscli/botocore/client.py", line 357, in _api_call
  File "awscli/botocore/client.py", line 724, in _make_api_call
botocore.exceptions.ClientError: An error occurred (InvalidClientTokenId) when calling the GetRole operation: The security token included in the request is invalid

Reproduction Steps

For a GovCloud account, get a temporary session token with aws sts get-session-token, then configure the AWS CLI with those values, use aws configure set aws_session_token TOKEN to set the session token. Then try to perform an IAM get-role call.

Possible Solution

No response

Additional Information/Context

No response

CLI version used

aws-cli/2.17.49

Environment details (OS name and version, etc.)

aws-cli/2.17.49 Python/3.11.9 Darwin/23.6.0 exe/x86_64
@ranok ranok added bug This issue is a bug. needs-triage This issue or PR still needs to be triaged. labels Sep 12, 2024
@ranok
Copy link
Author

ranok commented Sep 13, 2024

Apparently this is a limitation with GetSessionToken without MFA: https://docs.aws.amazon.com/STS/latest/APIReference/API_GetSessionToken.html

@ranok ranok closed this as completed Sep 13, 2024
@github-actions GitHub Actions
Copy link

This issue is now closed. Comments on closed issues are hard for our team to see.
If you need more assistance, please open a new issue that references this one.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug This issue is a bug. needs-triage This issue or PR still needs to be triaged.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@ranok