Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Call cat_ranges in blockcache for async filesystems #1336

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

monken
Copy link

@monken monken commented Aug 19, 2023

The CachingFileSystem can benefit from calling cat_ranges for asynchronous filesystems to concurrently download blocks. This speeds up filling the cache significantly for filesystems like HTTP and S3FS.

@monken monken force-pushed the blockcache-cat-ranges branch 3 times, most recently from 1c95789 to 09d06d8 Compare August 19, 2023 20:12
@martindurant
Copy link
Member

Sorry I let this one slide - I still think it's useful. I have just the one comment above.

@martindurant
Copy link
Member

Can you please solve the conflict, and I can have another look?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants