Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support IMDSv2 #5

Merged
merged 6 commits into from
Jun 7, 2024
Merged

Support IMDSv2 #5

merged 6 commits into from
Jun 7, 2024

Conversation

pingyu
Copy link
Collaborator

@pingyu pingyu commented Jun 6, 2024

Please help keep the CHANGELOG up to date by providing a one sentence summary of your change:

Access to S3 meet the following on an EC2 instance with IAM role having permission of S3 actions:

[2024/06/05 07:57:52.996 +00:00] [WARN] [util.rs:90] ["aws request meet error."] [uuid=10ed33a3-a245-40d0-b4ff-40bf02b8c7dc] [context=get_cred_on_premise] [retry?=true] [err="Couldn't find AWS credentials in environment, credentials file, or IAM role;No (or empty) AWS_ACCESS_KEY_ID in environment;Couldn't stat credentials file: [ \"/home/ec2-user/.aws/credentials\" ]. Non existant, or no permission.;Could not get request from environment: Neither environment variable 'AWS_CONTAINER_CREDENTIALS_FULL_URI' nor 'AWS_CONTAINER_CREDENTIALS_RELATIVE_URI' is set;Could not get credentials from iam: invalid uri character"]

There are 2 issues:

  1. The error message of invalid uri charactor was misleadering. The security-credentials API actually returned the 401 error, but we do not check the status code, and consider error message as role name:
<?xml version="1.0" encoding="iso-8859-1"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
	"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
 <head>
  <title>401 - Unauthorized</title>
 </head>
 <body>
  <h1>401 - Unauthorized</h1>
 </body>
  1. EC2 instance was configured as "Required" IMDSv2, but we do not support IMDSv2.

In this PR:

  • Check HTTP status code, and return error if it's not a successful code.
  • Support IMDSv2.

Manual Test

  • Show S3 file by code of baseline
./show --id 450254793210069006
Jun 06 10:33:35.447 WARN aws request meet error., uuid: 0ea65fe7-45da-4963-8b71-d0c3553476cd, context: get_cred_on_premise, retry?: true, err: Couldn't find AWS credentials in environment, credentials file, or IAM role;No (or empty) AWS_ACCESS_KEY_ID in environment;Couldn't stat credentials file: [ "/home/ec2-user/.aws/credentials" ]. Non existant, or no permission.;Could not get request from environment: Neither environment variable 'AWS_CONTAINER_CREDENTIALS_FULL_URI' nor 'AWS_CONTAINER_CREDENTIALS_RELATIVE_URI' is set;Could not get credentials from iam: invalid uri character
Jun 06 10:33:36.557 WARN aws request meet error., uuid: 0ea65fe7-45da-4963-8b71-d0c3553476cd, context: get_cred_on_premise, retry?: true, err: Couldn't find AWS credentials in environment, credentials file, or IAM role;No (or empty) AWS_ACCESS_KEY_ID in environment;Couldn't stat credentials file: [ "/home/ec2-user/.aws/credentials" ]. Non existant, or no permission.;Could not get request from environment: Neither environment variable 'AWS_CONTAINER_CREDENTIALS_FULL_URI' nor 'AWS_CONTAINER_CREDENTIALS_RELATIVE_URI' is set;Could not get credentials from iam: invalid uri character
  • Show S3 file with new codes of this PR
./show-new --id 450254793210069006
Jun 06 10:33:41.122 INFO read file 450254793210069006, size 128411, takes 80ms, retry 0
...
...

@pingyu pingyu closed this Jun 6, 2024
@pingyu pingyu reopened this Jun 6, 2024
@coocood coocood merged commit d6ffce8 into tikv:cse Jun 7, 2024
9 checks passed
pingyu added a commit to pingyu/rusoto that referenced this pull request Jun 7, 2024
@pingyu pingyu mentioned this pull request Jun 7, 2024
pingyu added a commit to pingyu/rusoto that referenced this pull request Aug 8, 2024
Signed-off-by: Ping Yu <[email protected]>
Signed-off-by: Ping Yu <[email protected]>
pingyu added a commit that referenced this pull request Aug 8, 2024
* Support IMDSv2 (#5)

Signed-off-by: Ping Yu <[email protected]>
Signed-off-by: Ping Yu <[email protected]>

* Apply suggestions from code review

Signed-off-by: Ping Yu <[email protected]>

Co-authored-by: kennytm <[email protected]>
Signed-off-by: Ping Yu <[email protected]>

---------

Signed-off-by: Ping Yu <[email protected]>
Signed-off-by: Ping Yu <[email protected]>
Co-authored-by: kennytm <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants