Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add FUSE Passthrough Support in Stargz-Snapshotter #1867

Open
wswsmao opened this issue Nov 19, 2024 · 3 comments
Open

Add FUSE Passthrough Support in Stargz-Snapshotter #1867

wswsmao opened this issue Nov 19, 2024 · 3 comments

Comments

@wswsmao
Copy link
Contributor

wswsmao commented Nov 19, 2024

Hi,

FUSE Passthrough has been introduced in the Linux kernel version 6.9. This feature has shown significant performance improvements, as detailed in the following articles:

Phoronix Article on FUSE Passthrough
Linux Kernel Commit

Additionally, the go-fuse package, which Stargz-Snapshotter depends on, has also added support for this passthrough feature:

go-fuse Commit 1
go-fuse Commit 2
go-fuse Commit 3

When a user-defined file implements the FilePassthroughFder interface, go-fuse will attempt to register the file fd from the file with the kernel.

I believe it would be beneficial for Stargz-Snapshotter to consider supporting this feature as well. Here’s a proposed implementation plan for FUSE passthrough:

  1. During the Open phase, attempt to pre-read the entire file instead of reading it in chunks.
  2. Utilize the existing cache's Get method to retrieve the fd of the cached file that has been written to local storage.
  3. Implement the FilePassthroughFder interface in node.file, allowing the fd from step 2 to be registered with the kernel via go-fuse.

By following this approach, subsequent Read operations would not need to return to user space, and go-fuse would release the registered information when necessary.

Thank you for considering this enhancement!

Best regards,
abush wang

@wswsmao wswsmao changed the title Add Supporting FUSE Passthrough in Stargz-Snapshotter Add FUSE Passthrough Support in Stargz-Snapshotter Nov 19, 2024
@wswsmao
Copy link
Contributor Author

wswsmao commented Nov 19, 2024

The lattest commit is the implementation for FUSE Passthrough support in stargz-snapshotter
https://github.com/wswsmao/stargz-snapshotter/tree/passthrough

@ktock
Copy link
Member

ktock commented Nov 19, 2024

SGTM. Could you open a PR?

@wswsmao
Copy link
Contributor Author

wswsmao commented Nov 19, 2024

SGTM. Could you open a PR?

OK, this is pr
#1868

ktock added a commit that referenced this issue Nov 21, 2024
Add FUSE Passthrough Support in Stargz-Snapshotter #1867
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants