Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(cu): remove unused drained process memory #1013

Open
TillaTheHun0 opened this issue Sep 11, 2024 · 0 comments
Open

chore(cu): remove unused drained process memory #1013

TillaTheHun0 opened this issue Sep 11, 2024 · 0 comments
Labels
cu ao Compute Unit enhancement New feature or request

Comments

@TillaTheHun0
Copy link
Member

TillaTheHun0 commented Sep 11, 2024

Background

During operation, the CU will drain LRU wasm memory into files, to free up space in its heap. If the corresponding ao process receives traffic, the CU will then "rehydrate" the cache entry for that process, by reading the memory in from the file.

When the CU shuts down, they files are effectively orphaned.

Problem

These WASM memory files can be quite large, and over time consume a nontrivial amount of disk. They are never cleaned up and are basically a one time use.

Solution

When the process wasm memory read from the file and subsequently is saved back into the LRU cache, the file that contained the drained memory should be deleted. Great care needs to be taken to ensure the files is not being used elsewhere. A lock can be used to ensure this is the case

Developer Notes

This deletion should be performed asynchronously, and fail gracefully.

@TillaTheHun0 TillaTheHun0 added enhancement New feature or request cu ao Compute Unit labels Sep 11, 2024
@TillaTheHun0 TillaTheHun0 changed the title chore(cu): remove old state-* files on startup to free up disk chore(cu): remove old state-* files on shutdown to free up disk Sep 13, 2024
@TillaTheHun0 TillaTheHun0 changed the title chore(cu): remove old state-* files on shutdown to free up disk chore(cu): remove unused drained process memory Sep 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cu ao Compute Unit enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant