From 250c59a7be153abac44c978e523730bad0095025 Mon Sep 17 00:00:00 2001 From: Ry Biesemeyer Date: Wed, 24 Aug 2022 20:40:34 +0000 Subject: [PATCH] fix: avoid `message` in manticore exception logging context When Logstash is configured to use JSON-formatted logs, the Log4j2 JSONLayout will serialize both the provided message and the context map's `message` in the same key/value collection. This is _syntactically_ valid, because JSON does not impose constraints on the uniqueness of keys in a key/value collection, but considered _semantically_ invalid by many parsers including Beats that attempt to deserialize into a structure that _do_ impose uniqueness constraints. Here we ensure that we provide the _contents_ of the exception's message inside a single message, without also providing a `message` key to the logging context. --- CHANGELOG.md | 3 +++ .../outputs/elasticsearch/http_client/manticore_adapter.rb | 4 ++-- logstash-output-elasticsearch.gemspec | 2 +- 3 files changed, 6 insertions(+), 3 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 51961c8c..b83787c3 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,3 +1,6 @@ +## 11.6.1 + - Fixed issue where exceptions logged by the Manticore adapter could be passed both a `message` and a context map including a `message`, resulting in log events that could not be reasonably be parsed when formatted with Log4j2's JSONLayout + ## 11.6.0 - Added support for `ca_trusted_fingerprint` when run on Logstash 8.3+ [#1074](https://github.com/logstash-plugins/logstash-output-elasticsearch/pull/1074) diff --git a/lib/logstash/outputs/elasticsearch/http_client/manticore_adapter.rb b/lib/logstash/outputs/elasticsearch/http_client/manticore_adapter.rb index ba128007..2c2b8e1f 100644 --- a/lib/logstash/outputs/elasticsearch/http_client/manticore_adapter.rb +++ b/lib/logstash/outputs/elasticsearch/http_client/manticore_adapter.rb @@ -88,7 +88,7 @@ def perform_request(url, method, path, params={}, body=nil) end def log_request_error(e) - details = { message: e.message, exception: e.class } + details = { exception: e.class } details[:cause] = e.cause if e.respond_to?(:cause) details[:backtrace] = e.backtrace if @logger.debug? @@ -101,7 +101,7 @@ def log_request_error(e) :info end - @logger.send level, "Failed to perform request", details + @logger.send level, "Failed to perform request: #{e.message}", details log_java_exception(details[:cause], :debug) if details[:cause] && @logger.debug? end diff --git a/logstash-output-elasticsearch.gemspec b/logstash-output-elasticsearch.gemspec index 92813f5d..745548c5 100644 --- a/logstash-output-elasticsearch.gemspec +++ b/logstash-output-elasticsearch.gemspec @@ -1,6 +1,6 @@ Gem::Specification.new do |s| s.name = 'logstash-output-elasticsearch' - s.version = '11.6.0' + s.version = '11.6.1' s.licenses = ['apache-2.0'] s.summary = "Stores logs in Elasticsearch" s.description = "This gem is a Logstash plugin required to be installed on top of the Logstash core pipeline using $LS_HOME/bin/logstash-plugin install gemname. This gem is not a stand-alone program"