Cleanup after tests to avoid OOME: Metaspace in fast-run testOnly sequence #254
Conversation
|
To validate this pull request against other sbt modules, please visit this link. |
…uence After these changes, this project no longer exhibits the problems discussed in sbt/sbt#2056
retronym
force-pushed
the
topic/metaspace
branch
from
0215226
to
dd7b74a
Compare
|
I am guessing you mean sbt/sbt#4166. |
|
Yeah, this is really just a demo of how to do the cleanup in the current
version of SBT. The log4j shutdown part is arguably project specific,
though I guess SBT could do that in the users behalf by default
…On Sat, 14 Jul 2018 at 13:30, eugene yokota ***@***.***> wrote:
I am guessing you mean sbt/sbt#4166
<sbt/sbt#4166>.
I was using LM test as an example I had handy, but the OP likely had
another build exhibiting the issue, so I think we should fix them either in
ScalaTest or sbt.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#254 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AAEAD4VmEU5xBIxkdY_UVnGvgJn9RfYBks5uGWXigaJpZM4VOUWr>
.
|
build.sbt
Outdated
| // shutdown Log4J to avoid classloader leaks | ||
| try { | ||
| val logManager = Class.forName("org.apache.logging.log4j.LogManager") | ||
| logManager.getMethod("shutdown").invoke(null) |
There was a problem hiding this comment.
Since I am using async logging, I am guessing calling shutdown right away would risk some logs not showing up. But I think it would make sense to remove loggers that are no longer needed at some point.
There was a problem hiding this comment.
Log4J might block until these are written:
/**
* Shutdown using the LoggerContext appropriate for the caller of this method.
* This is equivalent to calling {@code LogManager.shutdown(false)}.
*
* This call is synchronous and will block until shut down is complete.
* This may include flushing pending log events over network connections.
*
* @since 2.6
*/
public static void shutdown() {
shutdown(false);
}I haven't tested this.
I don't think it matters much for tests in any case.
There was a problem hiding this comment.
Another problem is that some appenders are long-living (relay appender used by server, backing appender, I think), so I can't stop them. If the loggers are leaking the loaders then I'd need to find out how to prevent that in general, I think.
There was a problem hiding this comment.
To be clear, this is shutting down Log4J that is in the classloader of the tests themselves, not Log4J used in the SBT build that is calling the tests.
If the test were forked, you get away without explicitly shutting down because Log4J registers a shutdown hook to clean up. But that shutdown hook becomes a classloader leak in a "container" environment like a webapp or SBT.
There was a problem hiding this comment.
https://logging.apache.org/log4j/2.x/manual/webapp.html
Because of the nature of class loaders within web applications, Log4j resources cannot be cleaned up through normal means. Log4j must be "started" when the web application deploys and "shut down" when the web application undeploys.
There was a problem hiding this comment.
I see. Wouldn't val logManager = Class.forName("org.apache.logging.log4j.LogManager") load log4j in the context of the build tho?
There was a problem hiding this comment.
Oh, you're right. I meant to use loader. I'll push a fix.
|
I locally still see metaspace climb to 250MB
|
After these changes, this project no longer exhibits the problems
discussed in sbt/sbt#4166.