Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

client-cloudwatch-logs doesn't return some logs #6753

Open
3 of 4 tasks
mike-bridgefy opened this issue Dec 20, 2024 · 5 comments
Open
3 of 4 tasks

client-cloudwatch-logs doesn't return some logs #6753

mike-bridgefy opened this issue Dec 20, 2024 · 5 comments
Assignees
Labels
bug This issue is a bug. p2 This is a standard priority issue

Comments

@mike-bridgefy
Copy link

Checkboxes for prior research

Describe the bug

When using the node library, data stored in cloudwatch is obtained, however it does not return the required information
package.json
... "dependencies": { "@aws-sdk/client-cloudwatch-logs": "^3.716.0", ...
We use log levels and when we try to return the logs of type "ERROR" the script returns an empty array, however if I do the same with type "INFO" it does return the information
Script

    const logs = await this._clientLogs.send(
      new FilterLogEventsCommand({
        logGroupName,
        logStreamNames,
        // filterPattern,
        filterPattern: '{$.logLevel="E*"}',
        startTime,
        endTime,
      }),
    );

If the same filter is performed in the AWS cloudwatch console, the information is returned correctly

fields @timestamp, @message, @logStream as username
| filter logLevel="ERROR"
| sort @timestamp desc
| limit 10000

Regression Issue

  • Select this option if this issue appears to be a regression.

SDK version number

@aws-sdk/[email protected]

Which JavaScript Runtime is this issue in?

Node.js

Details of the browser/Node.js/ReactNative version

20.12.1

Reproduction Steps

Try to make similar filters using the library and you will see that it does not return complete information

Observed Behavior

Using the package returns different information than in the AWS console when it should be the same

Expected Behavior

We expect the same behavior by doing the filters in the library and in the AWS console

Possible Solution

No response

Additional Information/Context

No response

@mike-bridgefy mike-bridgefy added bug This issue is a bug. needs-triage This issue or PR still needs to be triaged. labels Dec 20, 2024
@aBurmeseDev
Copy link
Member

Hi @mike-bridgefy - thanks for reaching out.

Before we further investigate, could you try these expressions below?

// This filter pattern uses the JSON notation to specify the field name logLevel and checks if its value is exactly equal to the string "ERROR". 
filterPattern: '{ $.logLevel = "ERROR" }'

// This checks if the logLevel field starts with the letter "E" and is followed by any other characters. This pattern will match log events with log levels like "ERROR", "EMERGENCY", "EWARN", etc.
filterPattern: "?logLevel = /E.*/"

// This checks if the logLevel field exactly matches the string "ERROR". This pattern will only match log events with the log level "ERROR" and will not match log levels like "ERRORWARN" or "EMERROR".
filterPattern: "?logLevel = /ERROR/"

@aBurmeseDev aBurmeseDev self-assigned this Dec 21, 2024
@aBurmeseDev aBurmeseDev added response-requested Waiting on additional info and feedback. Will move to \"closing-soon\" in 7 days. p2 This is a standard priority issue and removed needs-triage This issue or PR still needs to be triaged. labels Dec 21, 2024
Copy link

github-actions bot commented Jan 1, 2025

This issue has not received a response in 1 week. If you still think there is a problem, please leave a comment to avoid the issue from automatically closing.

@github-actions github-actions bot added the closing-soon This issue will automatically close in 4 days unless further comments are made. label Jan 1, 2025
@mike-bridgefy
Copy link
Author

Hi @aBurmeseDev,
I apologize, we were out of the office at the end of the year.
I have tried the options that you mentioned and the behavior is the same as what I have reported and not what it should be.

@github-actions github-actions bot removed closing-soon This issue will automatically close in 4 days unless further comments are made. response-requested Waiting on additional info and feedback. Will move to \"closing-soon\" in 7 days. labels Jan 3, 2025
@aBurmeseDev
Copy link
Member

@mike-bridgefy - could you add the middlewareStack to the client in your code and share the output with us to further investigate?

client.middlewareStack.add(
  (next, context) => async (args) => {
    args.request.headers["x-amz-meta-foo"] = "bar";
    console.log("AWS SDK context", context.clientName, context.commandName);
    console.log("AWS SDK request input", args.input);
    const result = await next(args);
    console.log("AWS SDK request output:", result.output);
    return result;
  },
  {
    step: "build",
    name: "addFooMetadataMiddleware",
    tags: ["METADATA", "FOO"],
    override: true,
  }
);

@aBurmeseDev aBurmeseDev added the response-requested Waiting on additional info and feedback. Will move to \"closing-soon\" in 7 days. label Jan 4, 2025
@mike-bridgefy
Copy link
Author

Hi @aBurmeseDev
I am using Nest and I put the code in a middleware but it does not return the information from the console, the middleware code does run but does not show the middlewareStack logs. Do you know if what I am doing is correct? Or how should the middlewareStack be integrated?

@Injectable()
export class LoggerMiddleware implements NestMiddleware {
  use(req: Request, res: Response, next: NextFunction) {
    const client = new CloudWatchLogsClient({region: 'us-east-1'});
    client.middlewareStack.add(
      (next, context) => async (args) => {
        args.request.headers["x-amz-meta-foo"] = "bar";
        console.log(args.request);
        console.log("AWS SDK context", context.clientName, context.commandName);
        console.log("AWS SDK request input", args.input);
        const result = await next(args);
        console.log("AWS SDK request output:", result.output);
        return result;
      },
      {
        step: "build",
        name: "addFooMetadataMiddleware",
        tags: ["METADATA", "FOO"],
        override: true,
      }
    );
    next();
  }
}

I really appreciate your support

@github-actions github-actions bot removed the response-requested Waiting on additional info and feedback. Will move to \"closing-soon\" in 7 days. label Jan 7, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug This issue is a bug. p2 This is a standard priority issue
Projects
None yet
Development

No branches or pull requests

2 participants