Fix unnecessary mmap'ing of compass .dat files #648
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes #595.
The access of the
compass/*.dat
files is currently broken. During chunk generation we easily mmap the same.dat
file (1 MB each) thousands of times, leading to OOMs if the VM/OS doesn't unmap fast enough during GCs.The access to the
compass/*.dat
files is synchronized via a single "Compass Service" thread. TheCompassService
class posts job tasks to this thread. The current idea is that when each task finishes, it checks whether more tasks are queued up, and if not, unmap the.dat
file. There are two main issues:CompassServide.#jobSize
grows only into the negative (due to some missingjobSize++
). This means that we un-map all.dat
files after each job even if more jobs are queued up..dat
file.This PR fixes this by removing the the broken job accounting all-together and replaces it with a per dimension timeout: If a
.dat
file goes unused for 60 seconds, we'll unmap it.Local testing shows that we go from around ~10k map/unmap operations to 4 when generation a new SP world (with boosted meteor generation chances so I don't have to go looking).
Implementation details
The PR achieves this by adding a new
AutoClosingCompassReader
class. It is associated with a singleCompassReader
that needs to be retrieved viaAutoClosingCompassReader#get
. Ever timeget
is called, we schedule a delayed "closing task". If one is already schedule whenget
is called, we'll cancel it.