-
Notifications
You must be signed in to change notification settings - Fork 601
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Redis Signer for connecting to Redis 7 using IAM authentication #6611
Comments
Hi @meenar-se , Thanks for opening this feature request. I think your request is reasonable, but like RDS signer, this would need to be a handwritten utility (whereas most of the SDK is code generated from the API models of the each service), this means that it will have to be properly designed and implemented in a uniform fashion across all SDKs. Additionally, feature requests are accepted and implemented based on community engagement (comments, upvotes, or duplicate FRs). This helps us prioritize features in the most impactful way, and use the teams resources wisely. I will transfer this FR to the cross SDK repo for it to gain traction there. This unfortunately means that it will not get prioritized right away, but it doesn't mean it wont in the future. Since you marked the "I may be able to implement this feature request" checkbox, I'd encourage you to write your implementation here on this ticket for two reasons:
Thanks again 😄 |
ImplementationConfiguration file: runtimeConfig.ts
Signer Implementation: Signer.ts
Usage example
|
I can also work on the implementing this feature in SDK. We can have the signer as common utility and use it for REDIS and RDS. Please suggest. |
HI @meenar-se, did you ever get your sample code to work? I've tried it on Elasticache redis/valkey serverless and non-severless and always get For serverless, it looks like an additional |
I got it working using https://github.com/aws-samples/elasticache-iam-auth-demo-app as reference. Here is my updated code including serverless support: Configuration file: runtimeConfig.tsimport { fromNodeProviderChain } from '@aws-sdk/credential-providers';
import { NODE_REGION_CONFIG_FILE_OPTIONS, NODE_REGION_CONFIG_OPTIONS } from '@smithy/config-resolver';
import { Hash } from '@smithy/hash-node';
import { loadConfig } from '@smithy/node-config-provider';
import { SignerConfig } from './signer';
/**
* @internal
*/
export const getRuntimeConfig = (config: SignerConfig) => {
return {
runtime: 'node',
sha256: config?.sha256 ?? Hash.bind(null, 'sha256'),
credentials: config?.credentials ?? fromNodeProviderChain(),
region: config?.region ?? loadConfig(NODE_REGION_CONFIG_OPTIONS, NODE_REGION_CONFIG_FILE_OPTIONS),
...config,
};
}; Signer Implementation: signer.tsimport { formatUrl } from '@aws-sdk/util-format-url';
import { HttpRequest } from '@smithy/protocol-http';
import { SignatureV4 } from '@smithy/signature-v4';
import {
AwsCredentialIdentity,
AwsCredentialIdentityProvider,
ChecksumConstructor,
HashConstructor,
Provider,
} from '@smithy/types';
import { getRuntimeConfig as __getRuntimeConfig } from './runtimeConfig';
export enum ClusterType {
normal = 'normal',
serverless = 'serverless',
}
export interface SignerConfig {
/**
* The AWS credentials to sign requests with. Uses the default credential provider chain if not specified.
*/
credentials?: AwsCredentialIdentity | AwsCredentialIdentityProvider;
/**
* The hostname of the cache to connect to.
*/
cacheName: string;
/**
* The resource type of the cluster.
*/
resourceType: string;
/**
* The region the database is located in. Uses the region inferred from the runtime if omitted.
*/
region?: string;
/**
* The SHA256 hasher constructor to sign the request.
*/
sha256?: ChecksumConstructor | HashConstructor;
/**
* The username to login as.
*/
username: string;
}
/**
* The signer class that generates an auth token to a database.
*/
export class Signer {
private readonly credentials: AwsCredentialIdentity | AwsCredentialIdentityProvider;
private readonly cacheName: string;
private readonly resourceType: string;
private readonly protocol: string = 'https:';
private readonly region: string | Provider<string>;
private readonly service: string = 'elasticache';
private readonly sha256: ChecksumConstructor | HashConstructor;
private readonly username: string;
constructor(configuration: SignerConfig) {
const runtimeConfiguration = __getRuntimeConfig(configuration);
this.credentials = runtimeConfiguration.credentials;
this.cacheName = runtimeConfiguration.cacheName;
this.resourceType = runtimeConfiguration.resourceType;
this.region = runtimeConfiguration.region;
this.sha256 = runtimeConfiguration.sha256;
this.username = runtimeConfiguration.username;
}
public async getAuthToken(): Promise<string> {
const signer = new SignatureV4({
service: this.service,
region: this.region,
credentials: this.credentials,
sha256: this.sha256,
});
const request = new HttpRequest({
method: 'GET',
protocol: this.protocol,
hostname: this.cacheName,
query: {
Action: 'connect',
User: this.username,
// add ResourceType property if serverless
...(this.resourceType === 'serverless' ? { ResourceType: 'ServerlessCache' } : {}),
},
headers: {
host: this.cacheName,
},
});
const presigned = await signer.presign(request, {
expiresIn: 900,
});
return formatUrl(presigned).replace(`${this.protocol}//`, '');
}
} Usage exampleimport { fromNodeProviderChain } from '@aws-sdk/credential-providers';
import { createClient } from 'redis';
import { Signer, ClusterType } from './signer';
const options = {
hostname: 'valkey-serverless.id.serverless.euw2.cache.amazonaws.com',
cacheName: 'valkey-serverless',
region: 'eu-west-2',
clusterType: ClusterType.serverless,
username: 'valkey-serverless-user',
};
export const redisConnect = async () => {
let credentials = fromNodeProviderChain();
console.log('calling redis connect');
const sign = new Signer({
credentials: credentials,
cacheName: options.cacheName,
region: options.region,
resourceType: options.clusterType,
username: options.username,
});
const presignedUrl = await sign.getAuthToken();
const redisConfig = {
url: options.hostname,
password: presignedUrl,
username: options.username,
socket: {
tls: true,
rejectUnauthorized: false,
},
};
const redisClient = createClient(redisConfig);
try {
await redisClient.connect();
console.log(await redisClient.ping());
} catch (error) {
console.log('Error catched ' + error);
}
}; |
Note that IAM creds expire (I think every 15 minutes), so something needs to refresh them. The redis library does not have a refresh built in (but one is proposed). I guess the application layer just needs to reinitialize the redis client on credential timeout. |
Just in case someone is trying to implement IAM auth to AWS Elasticache in NodeJS with auto-reconnect when the token expires, here is the implementation that works for me. IAMAuthTokenRequest.js const { CrtSignerV4 } = require('@aws-sdk/signature-v4-crt');
const { fromEnv } = require('@aws-sdk/credential-providers');
const url = require('url');
class IAMAuthTokenRequest {
static REQUEST_METHOD = 'GET';
static REQUEST_PROTOCOL = 'http:';
static PARAM_ACTION = 'Action';
static PARAM_USER = 'User';
static PARAM_RESOURCE_TYPE = 'ResourceType';
static RESOURCE_TYPE_SERVERLESS_CACHE = 'ServerlessCache';
static ACTION_NAME = 'connect';
static SERVICE_NAME = 'elasticache';
static TOKEN_EXPIRY_SECONDS = 900;
constructor({userId, cacheName, region, isServerless}) {
this.userId = userId;
this.cacheName = cacheName;
this.region = region || process.env.AWS_REGION;
this.isServerless = isServerless || false;
}
async getAuthToken(credentials = fromEnv()) {
let request = {
method: IAMAuthTokenRequest.REQUEST_METHOD,
protocol: IAMAuthTokenRequest.REQUEST_PROTOCOL,
hostname: this.cacheName,
headers: {
host: this.cacheName
},
query: {
[IAMAuthTokenRequest.PARAM_ACTION]: IAMAuthTokenRequest.ACTION_NAME,
[IAMAuthTokenRequest.PARAM_USER]: this.userId,
...this.isServerless ? {
[IAMAuthTokenRequest.PARAM_RESOURCE_TYPE]: IAMAuthTokenRequest.RESOURCE_TYPE_SERVERLESS_CACHE
} : {}
}
};
let signed_request = await this.sign(request, credentials);
return url
.format(signed_request)
.toString()
.replace(`${request.protocol}//`, '');
}
async sign(request, credentials) {
const signer = new CrtSignerV4({
credentials,
service: IAMAuthTokenRequest.SERVICE_NAME,
region: this.region
});
return await signer.presign(request, {
expiresIn: IAMAuthTokenRequest.TOKEN_EXPIRY_SECONDS,
});
}
}
module.exports = IAMAuthTokenRequest; RedisTokenConnector.js const Redis = require("ioredis");
const StandaloneConnector = require("ioredis/built/connectors/StandaloneConnector").default;
/**
* RedisTokenConnector is a Redis connector that fetches the password
* dynamically on each connect() call, and hacks it in the Redis instance.
*/
class RedisTokenConnector extends StandaloneConnector {
/**
* @param {Object} options - The options for the RedisTokenConnector.
* @param {Object} options.tokenConnector - The token connector options.
* @param {Object} options.tokenConnector.redisRef - Reference to an existing RedisRef instance.
* @param {Function} options.tokenConnector.getToken - Function used to retrieve the connection password.
*/
constructor(options) {
super(options);
this.redisRef = options.tokenConnector.redisRef;
this.getToken = options.tokenConnector.getToken;
}
/**
* Connects to the Redis instance, fetching the password dynamically.
* @param {Function} _ - Error emitter function.
* @returns {Promise<NetStream>} - The network stream.
*/
async connect(_) {
const token = await this.getToken();
const condition = this.redisRef.current?.condition;
if (!condition) throw new Error("expected redis.condition to be set at this point");
if (condition.auth === undefined || typeof condition.auth === "string") {
condition.auth = token; // password only
} else if (Array.isArray(condition.auth)) {
condition.auth = [condition.auth[0], token]; // [username, password]
}
return super.connect(_);
}
}
module.exports = RedisTokenConnector; Usage example: const Redis = require('ioredis');
const RedisTokenConnector = require('./RedisTokenConnector');
const IAMAuthTokenRequest = require('./IAMAuthTokenRequest');
let iamAuthTokenRequest = new IAMAuthTokenRequest({
userId: '<CACHE_USERNAME>',
cacheName: '<CACHE_NAME>',
region: 'ap-southeast-2',
isServerless: false, // "false" in case you use Replication Group
});
const redisRef = { current: null };
const redisClient = new Redis({
host: '<CACHE_ENDPOINT>',
port: 6379,
username: '<CACHE_USERNAME>',
tls: {}, // Required if TLS encryption is enabled
tokenConnector: {
redisRef,
getToken: async () => await iamAuthTokenRequest.getAuthToken()
},
Connector: RedisTokenConnector
// ...other options
});
// IMPORTANT, if you do not set redisRef.current you'll get an error when connecting
redisRef.current = redisClient; Also, a big thank you to the answers that helped me understand and implement it in NodeJS: |
Potentially |
Seems like official redis client starting from 5.0.0-next.6 has support of credentialProvider: |
Describe the feature
Need an implementation for Redis signer so that its useful to connect to Elastic cache redis version 7 or above using IAM Authentication.
Use Case
Recently AWS introduced an option to connect to Redis 7 using IAM authentication. But its not directly supported by the library. So we have slightly tweaked the RDS signer to support Redis as well. It will be useful if we add the Redis Signer as well in library itself.
Proposed Solution
No response
Other Information
No response
Acknowledgements
SDK version used
3.350.0
Environment details (OS name and version, etc.)
Macos 12.6.3
The text was updated successfully, but these errors were encountered: