Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unclosed Span leads to OOM and breaks references #34

Open
ledor473 opened this issue Jun 12, 2020 · 2 comments
Open

Unclosed Span leads to OOM and breaks references #34

ledor473 opened this issue Jun 12, 2020 · 2 comments
Labels

Comments

@ledor473
Copy link

Using the OpenTracingChannelInterceptor class may cause the application to go in OOM because the Span gets started but are not always closed. Also, since the Scope remains active, the references (wrong parent).

In the code below, the Span is created and is made active in the ScopeManager.

It's supposed to be closed here:

Object scopeValue = message.getHeaders().get(SCOPE_HEADER);
if (scopeValue instanceof Scope) {
Span span = tracer.scopeManager().activeSpan();
closeResources(ex, (Scope) scopeValue, span);
}

That logic only works if the application is using a DirectChannel but it's unlikely to work with something like the ExecutorChannel.
Furthermore, it appears that Spring Cloud Stream has a DirectChannel implementation that is not actually "direct" and needs to be excluded according to the code in Sleuth.

Our issue was with the DirectWithAttributesChannel specifically, but the fix we are testing is to only activate the Span if it's a DirectChannel, pass both the Span and Scope in the headers and close both of them in the afterSendCompletion.

@pavolloffay
Copy link
Contributor

thanks for reporting @ledor473, would you like to submit a PR to fix it?

@ledor473
Copy link
Author

@pavolloffay yes we are still testing the change but I'll send something as soon as it's fixed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants