Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding support for onerror parameter in walk #1303

Merged
merged 3 commits into from
Jul 26, 2023
Merged

Conversation

epizut
Copy link
Contributor

@epizut epizut commented Jul 4, 2023

Fixes #1297, I was not sure how to test the async version since I haven't found any suitable existing test to start with.

The current walk() implementation is missing the onerror parameter.

See os.walk

@martindurant
Copy link
Member

I quite like onerror being a function, but in the existing API, it's a string like "omit", "raise"... Can you please revisit other usage and try to achieve consistency?

For async, the tricky part can be to figure out when to consider errors. Most of the coroutines will return error instances, so we can check for that. You will need to look over in s3fs for probably the most complete application.

@epizut
Copy link
Contributor Author

epizut commented Jul 17, 2023

Sorry, I miss the existing on_error, I planned to use a function to mimic os.walk().
I defaulted on omit to avoid any behavior change, this differs from the existing on_error usage that raises by default. Unfortunately, find() expects walk() to not raise when called with a file path. I will be happy to default on raise, but it will need a few more changes.

@epizut epizut force-pushed the master branch 2 times, most recently from a8baaf4 to c1e0469 Compare July 17, 2023 13:33
@martindurant martindurant merged commit 166f462 into fsspec:master Jul 26, 2023
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Adding support for onerror parameter in walk
2 participants