Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Redis Signer for connecting to Redis 7 using IAM authentication #6611

Open
1 of 2 tasks
meenar-se opened this issue Jun 19, 2023 · 9 comments
Open
1 of 2 tasks

Redis Signer for connecting to Redis 7 using IAM authentication #6611

meenar-se opened this issue Jun 19, 2023 · 9 comments
Assignees
Labels
cross-sdk feature-request New feature or enhancement. May require GitHub community feedback. p2 This is a standard priority issue

Comments

@meenar-se
Copy link

Describe the feature

Need an implementation for Redis signer so that its useful to connect to Elastic cache redis version 7 or above using IAM Authentication.

Use Case

Recently AWS introduced an option to connect to Redis 7 using IAM authentication. But its not directly supported by the library. So we have slightly tweaked the RDS signer to support Redis as well. It will be useful if we add the Redis Signer as well in library itself.

Proposed Solution

No response

Other Information

No response

Acknowledgements

  • I may be able to implement this feature request
  • This feature might incur a breaking change

SDK version used

3.350.0

Environment details (OS name and version, etc.)

Macos 12.6.3

@meenar-se meenar-se added feature-request New feature or enhancement. May require GitHub community feedback. needs-triage This issue or PR still needs to be triaged. labels Jun 19, 2023
@RanVaknin RanVaknin self-assigned this Jun 19, 2023
@RanVaknin
Copy link
Contributor

Hi @meenar-se ,

Thanks for opening this feature request.

I think your request is reasonable, but like RDS signer, this would need to be a handwritten utility (whereas most of the SDK is code generated from the API models of the each service), this means that it will have to be properly designed and implemented in a uniform fashion across all SDKs.

Additionally, feature requests are accepted and implemented based on community engagement (comments, upvotes, or duplicate FRs). This helps us prioritize features in the most impactful way, and use the teams resources wisely.

I will transfer this FR to the cross SDK repo for it to gain traction there. This unfortunately means that it will not get prioritized right away, but it doesn't mean it wont in the future.

Since you marked the "I may be able to implement this feature request" checkbox, I'd encourage you to write your implementation here on this ticket for two reasons:

  1. It will allow other community members that are facing the same issue to use your solution as a workaround.
  2. When time comes to execute on this feature request, it will be a good starting points for one of the devs to refer to and test against.

Thanks again 😄
Ran

@RanVaknin RanVaknin removed the needs-triage This issue or PR still needs to be triaged. label Jun 20, 2023
@RanVaknin RanVaknin transferred this issue from aws/aws-sdk-js-v3 Jun 20, 2023
@meenar-se
Copy link
Author

Implementation

Configuration file: runtimeConfig.ts

import { fromNodeProviderChain } from '@aws-sdk/credential-providers';
import { Hash } from '@aws-sdk/hash-node';
import { loadConfig } from '@aws-sdk/node-config-provider';
import { SignerConfig } from './Signer';

/**
 * @internal
 */
export const getRuntimeConfig = (config: SignerConfig) : SignerConfig => ({
  runtime: 'node',
  sha256: config?.sha256 ?? Hash.bind(null, 'sha256'),
  credentials: config?.credentials ?? fromNodeProviderChain(),
  region: config?.region ?? loadConfig(NODE_REGION_CONFIG_OPTIONS, NODE_REGION_CONFIG_FILE_OPTIONS),
  expiresIn: 900,
  ...config,
} as SignerConfig);

Signer Implementation: Signer.ts

import { SignatureV4 } from '@aws-sdk/signature-v4';
import {
  AwsCredentialIdentity,
  AwsCredentialIdentityProvider,
  ChecksumConstructor,
} from '@aws-sdk/types';
import { formatUrl } from '@aws-sdk/util-format-url';

import { getRuntimeConfig as __getRuntimeConfig } from './runtimeConfig';
import { defaultProvider } from '@aws-sdk/credential-provider-node';

export interface SignerConfig {
  /**
   * The AWS credentials to sign requests with. Uses the default credential provider chain if not specified.
   */
  credentials?: AwsCredentialIdentity | AwsCredentialIdentityProvider
  /**
   * The hostname of the database to connect to.
   */
  hostname: string
  /**
   * The port number the database is listening on.
   */
  port?: number
  /**
   * The region the database is located in. Uses the region inferred from the runtime if omitted.
   */
  region?: string
  /**
   * The SHA256 hasher constructor to sign the request.
   */
  sha256?: ChecksumConstructor
  /**
   * The username to login as.
   */
  username: string
  runtime?: string
  expiresIn?: number
}

/**
 * The signer class that generates an auth token to a database.
 */
export class Signer {

  private readonly protocol: string = 'http:';
  private readonly service: string = 'elasticache';

  public constructor(public configuration: SignerConfig) {
    this.configuration = __getRuntimeConfig(configuration);
    
  }

  public async getAuthToken(): Promise<string> {
    const signer = new SignatureV4({
      service: this.service,
      region: this.configuration.region!,
      credentials: this.configuration.credentials ?? defaultProvider(),
      sha256: this.configuration.sha256!,
    });

    const request = new HttpRequest({
      method: 'GET',
      protocol: this.protocol,
      hostname: this.configuration.hostname,
      port: this.configuration.port,
      query: {
        Action: 'connect',
        User: this.configuration.username,   
      },
      headers: {
        host: `${this.configuration.hostname}`,
      },
    });

    const presigned = await signer.presign(request, {
      expiresIn: this.configuration.expiresIn,
    });
    const format = formatUrl(presigned).replace(`${this.protocol}//`, '');
    console.log(format);
    
    return format;
  }
}

Usage example

import { createClient } from "redis";
import { Signer } from "./Signer";


export const redisConnect = async () => {
  console.log("calling redis connect");
  const credentials = await generateAssumeRoleCreds();
  const sign = new Signer({
    region: region,
    hostname: `${replicationGroupId}`,
    username: userId,
    credentials: credentials,
  });
  const presignedUrl = await sign.getAuthToken();
  const redisConfig = {
    url: `redis://master.xxx.xxx.xxxx.use1.cache.amazonaws.com:6379`,
    password: presignedUrl,
    username: userId,
    socket: {
      tls: true,
      rejectUnauthorized: false,
    },
  };
  const redisClient = await createClient(redisConfig);
  try {
    await redisClient.connect();
    console.log(await redisClient.get("key"));
  } catch (error) {
    console.log("Error catched " + error);
  }
};

@meenar-se
Copy link
Author

I can also work on the implementing this feature in SDK. We can have the signer as common utility and use it for REDIS and RDS. Please suggest.

@RanVaknin RanVaknin added the p2 This is a standard priority issue label Aug 29, 2024
@RanVaknin RanVaknin transferred this issue from aws/aws-sdk Oct 30, 2024
@X-Guardian
Copy link

HI @meenar-se, did you ever get your sample code to work? I've tried it on Elasticache redis/valkey serverless and non-severless and always get ErrorReply: WRONGPASS invalid username-password pair or user is disabled.

For serverless, it looks like an additional ResourceType=ServerlessCache query parameter is needed, but I've tried it with and without. Ref: https://docs.aws.amazon.com/AmazonElastiCache/latest/dg/auth-iam.html#auth-iam-Connecting

@X-Guardian
Copy link

X-Guardian commented Nov 6, 2024

I got it working using https://github.com/aws-samples/elasticache-iam-auth-demo-app as reference. Here is my updated code including serverless support:

Configuration file: runtimeConfig.ts

import { fromNodeProviderChain } from '@aws-sdk/credential-providers';
import { NODE_REGION_CONFIG_FILE_OPTIONS, NODE_REGION_CONFIG_OPTIONS } from '@smithy/config-resolver';
import { Hash } from '@smithy/hash-node';
import { loadConfig } from '@smithy/node-config-provider';
import { SignerConfig } from './signer';

/**
 * @internal
 */
export const getRuntimeConfig = (config: SignerConfig) => {
  return {
    runtime: 'node',
    sha256: config?.sha256 ?? Hash.bind(null, 'sha256'),
    credentials: config?.credentials ?? fromNodeProviderChain(),
    region: config?.region ?? loadConfig(NODE_REGION_CONFIG_OPTIONS, NODE_REGION_CONFIG_FILE_OPTIONS),
    ...config,
  };
};

Signer Implementation: signer.ts

import { formatUrl } from '@aws-sdk/util-format-url';
import { HttpRequest } from '@smithy/protocol-http';
import { SignatureV4 } from '@smithy/signature-v4';
import {
  AwsCredentialIdentity,
  AwsCredentialIdentityProvider,
  ChecksumConstructor,
  HashConstructor,
  Provider,
} from '@smithy/types';

import { getRuntimeConfig as __getRuntimeConfig } from './runtimeConfig';

export enum ClusterType {
  normal = 'normal',
  serverless = 'serverless',
}

export interface SignerConfig {
  /**
   * The AWS credentials to sign requests with. Uses the default credential provider chain if not specified.
   */
  credentials?: AwsCredentialIdentity | AwsCredentialIdentityProvider;
  /**
   * The hostname of the cache to connect to.
   */
  cacheName: string;
  /**
   * The resource type of the cluster.
   */
  resourceType: string;
  /**
   * The region the database is located in. Uses the region inferred from the runtime if omitted.
   */
  region?: string;
  /**
   * The SHA256 hasher constructor to sign the request.
   */
  sha256?: ChecksumConstructor | HashConstructor;
  /**
   * The username to login as.
   */
  username: string;
}

/**
 * The signer class that generates an auth token to a database.
 */
export class Signer {
  private readonly credentials: AwsCredentialIdentity | AwsCredentialIdentityProvider;
  private readonly cacheName: string;
  private readonly resourceType: string;
  private readonly protocol: string = 'https:';
  private readonly region: string | Provider<string>;
  private readonly service: string = 'elasticache';
  private readonly sha256: ChecksumConstructor | HashConstructor;
  private readonly username: string;

  constructor(configuration: SignerConfig) {
    const runtimeConfiguration = __getRuntimeConfig(configuration);

    this.credentials = runtimeConfiguration.credentials;
    this.cacheName = runtimeConfiguration.cacheName;
    this.resourceType = runtimeConfiguration.resourceType;
    this.region = runtimeConfiguration.region;
    this.sha256 = runtimeConfiguration.sha256;
    this.username = runtimeConfiguration.username;
  }

  public async getAuthToken(): Promise<string> {
    const signer = new SignatureV4({
      service: this.service,
      region: this.region,
      credentials: this.credentials,
      sha256: this.sha256,
    });

    const request = new HttpRequest({
      method: 'GET',
      protocol: this.protocol,
      hostname: this.cacheName,
      query: {
        Action: 'connect',
        User: this.username,
        // add ResourceType property if serverless
        ...(this.resourceType === 'serverless' ? { ResourceType: 'ServerlessCache' } : {}),
      },
      headers: {
        host: this.cacheName,
      },
    });

    const presigned = await signer.presign(request, {
      expiresIn: 900,
    });

    return formatUrl(presigned).replace(`${this.protocol}//`, '');
  }
}

Usage example

import { fromNodeProviderChain } from '@aws-sdk/credential-providers';
import { createClient } from 'redis';
import { Signer, ClusterType } from './signer';

const options = {
  hostname: 'valkey-serverless.id.serverless.euw2.cache.amazonaws.com',
  cacheName: 'valkey-serverless',
  region: 'eu-west-2',
  clusterType: ClusterType.serverless,
  username: 'valkey-serverless-user',
};

export const redisConnect = async () => {
  let credentials = fromNodeProviderChain();

  console.log('calling redis connect');
  const sign = new Signer({
    credentials: credentials,
    cacheName: options.cacheName,
    region: options.region,
    resourceType: options.clusterType,
    username: options.username,
  });
  const presignedUrl = await sign.getAuthToken();
  const redisConfig = {
    url: options.hostname,
    password: presignedUrl,
    username: options.username,
    socket: {
      tls: true,
      rejectUnauthorized: false,
    },
  };
  const redisClient = createClient(redisConfig);
  try {
    await redisClient.connect();
    console.log(await redisClient.ping());
  } catch (error) {
    console.log('Error catched ' + error);
  }
};

@zachary-blackbird
Copy link

Note that IAM creds expire (I think every 15 minutes), so something needs to refresh them. The redis library does not have a refresh built in (but one is proposed). I guess the application layer just needs to reinitialize the redis client on credential timeout.

@lpavliuk
Copy link

Just in case someone is trying to implement IAM auth to AWS Elasticache in NodeJS with auto-reconnect when the token expires, here is the implementation that works for me.

IAMAuthTokenRequest.js

const { CrtSignerV4 } = require('@aws-sdk/signature-v4-crt');
const { fromEnv } = require('@aws-sdk/credential-providers');
const url = require('url');

class IAMAuthTokenRequest {
    static REQUEST_METHOD = 'GET';
    static REQUEST_PROTOCOL = 'http:';
    static PARAM_ACTION = 'Action';
    static PARAM_USER = 'User';
    static PARAM_RESOURCE_TYPE = 'ResourceType';
    static RESOURCE_TYPE_SERVERLESS_CACHE = 'ServerlessCache';
    static ACTION_NAME = 'connect';
    static SERVICE_NAME = 'elasticache';
    static TOKEN_EXPIRY_SECONDS = 900;

    constructor({userId, cacheName, region, isServerless}) {
        this.userId = userId;
        this.cacheName = cacheName;
        this.region = region || process.env.AWS_REGION;
        this.isServerless = isServerless || false;
    }

    async getAuthToken(credentials = fromEnv()) {
        let request = {
            method: IAMAuthTokenRequest.REQUEST_METHOD,
            protocol: IAMAuthTokenRequest.REQUEST_PROTOCOL,
            hostname: this.cacheName,
            headers: {
                host: this.cacheName
            },
            query: {
                [IAMAuthTokenRequest.PARAM_ACTION]: IAMAuthTokenRequest.ACTION_NAME,
                [IAMAuthTokenRequest.PARAM_USER]: this.userId,
                ...this.isServerless ? {
                    [IAMAuthTokenRequest.PARAM_RESOURCE_TYPE]: IAMAuthTokenRequest.RESOURCE_TYPE_SERVERLESS_CACHE
                } : {}
            }
        };

        let signed_request = await this.sign(request, credentials);

        return url
            .format(signed_request)
            .toString()
            .replace(`${request.protocol}//`, '');
    }

    async sign(request, credentials) {
        const signer = new CrtSignerV4({
            credentials,
            service: IAMAuthTokenRequest.SERVICE_NAME,
            region: this.region
        });

        return await signer.presign(request, {
            expiresIn: IAMAuthTokenRequest.TOKEN_EXPIRY_SECONDS,
        });
    }
}

module.exports = IAMAuthTokenRequest;

RedisTokenConnector.js

const Redis = require("ioredis");
const StandaloneConnector = require("ioredis/built/connectors/StandaloneConnector").default;

/**
 * RedisTokenConnector is a Redis connector that fetches the password
 * dynamically on each connect() call, and hacks it in the Redis instance.
 */
class RedisTokenConnector extends StandaloneConnector {
    /**
     * @param {Object} options - The options for the RedisTokenConnector.
     * @param {Object} options.tokenConnector - The token connector options.
     * @param {Object} options.tokenConnector.redisRef - Reference to an existing RedisRef instance.
     * @param {Function} options.tokenConnector.getToken - Function used to retrieve the connection password.
     */
    constructor(options) {
        super(options);

        this.redisRef = options.tokenConnector.redisRef;
        this.getToken = options.tokenConnector.getToken;
    }

    /**
     * Connects to the Redis instance, fetching the password dynamically.
     * @param {Function} _ - Error emitter function.
     * @returns {Promise<NetStream>} - The network stream.
     */
    async connect(_) {
        const token = await this.getToken();

        const condition = this.redisRef.current?.condition;
        if (!condition) throw new Error("expected redis.condition to be set at this point");

        if (condition.auth === undefined || typeof condition.auth === "string") {
            condition.auth = token; // password only
        } else if (Array.isArray(condition.auth)) {
            condition.auth = [condition.auth[0], token]; // [username, password]
        }

        return super.connect(_);
    }
}

module.exports = RedisTokenConnector;

Usage example:

const Redis = require('ioredis');
const RedisTokenConnector = require('./RedisTokenConnector');
const IAMAuthTokenRequest = require('./IAMAuthTokenRequest');

let iamAuthTokenRequest = new IAMAuthTokenRequest({
    userId: '<CACHE_USERNAME>',
    cacheName: '<CACHE_NAME>',
    region: 'ap-southeast-2',
    isServerless: false, // "false" in case you use Replication Group
});

const redisRef = { current: null };
const redisClient = new Redis({
    host: '<CACHE_ENDPOINT>',
    port: 6379,
    username: '<CACHE_USERNAME>',
    tls: {}, // Required if TLS encryption is enabled
    tokenConnector: {
        redisRef,
        getToken: async () => await iamAuthTokenRequest.getAuthToken()
    },
    Connector: RedisTokenConnector
    // ...other options
});
// IMPORTANT, if you do not set redisRef.current you'll get an error when connecting
redisRef.current = redisClient;

Also, a big thank you to the answers that helped me understand and implement it in NodeJS:

@taraspos
Copy link

taraspos commented Feb 6, 2025

@taraspos
Copy link

taraspos commented Feb 7, 2025

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cross-sdk feature-request New feature or enhancement. May require GitHub community feedback. p2 This is a standard priority issue
Projects
None yet
Development

No branches or pull requests

7 participants