Constant heap growth after implementing jaeger tracing

[iKeysa]

Library included: opentracing-spring-jaeger-web-starter
version: 3.1.1/3.2.2
implementation: Extended ServletFilterSpanDecorator & RestTemplateSpanDecorator to set tags and span details to log details related to our API calls.
Issue: Our API gets called twice in a business flow and we call 3 downstream systems to aggregate data. It was observed that data that was set to tags is being added to span as a hashmap. As a results, 2 calls to api is resulting in adding 12 hashmaps to heap. Since these hashmaps are still being referenced in the memory, minor/mixed GC is not collecting them. Below is histo of our heap with and without jaeger tracing. It's a 2-3 hour test.

with jaeger

num #instances #bytes class name

1: 6209579 198706528 java.util.HashMap$Node
2: 6275334 150608016 java.lang.String
3: 490368 69699240 [Ljava.util.HashMap$Node;
4: 747753 47571272 [C
5: 1185323 37930336 io.opentracing.util.ThreadLocalScope
6: 474978 34198416 io.jaegertracing.internal.JaegerSpanContext
7: 474978 30398592 io.jaegertracing.internal.JaegerSpan
8: 480967 23086416 java.util.HashMap
9: 490463 11771112 java.util.ArrayList

without jaeger

num #instances #bytes class name

1: 148201 14553584 [C
2: 52875 11382616 [B
3: 262144 6291456 org.apache.logging.log4j.core.async.AsyncLoggerConfigDisruptor$Log4jEventWrapper
4: 44951 4929096 [Ljava.lang.Object;
5: 147255 3534120 java.lang.String
6: 20508 3117216 sun.security.ssl.SSLSessionImpl
7: 13168 2704688 [I
8: 20646 2307752 java.lang.Class
9: 66099 2115168 java.util.concurrent.ConcurrentHashMap$Node
10: 51668 2066720 java.util.LinkedHashMap$Entry
11: 31884 1783368 [Ljava.lang.String;
12: 21543 1378752 java.util.concurrent.ConcurrentHashMap
13: 14062 1275808 [Ljava.util.HashMap$Node;
14: 52835 1268040 java.util.ArrayList