Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Speedup time parser #461

Merged
merged 3 commits into from
Nov 30, 2023
Merged

Speedup time parser #461

merged 3 commits into from
Nov 30, 2023

Conversation

sergei-encord
Copy link
Collaborator

We're using dateutil parse method, which is cool, as it can correctly guess the data format and pretty error-tolerant.
But this also makes parsing quite expensive, because in most of the cases date time is of correct known format.
So introducing a wrapper, that tries to parse known formats first, and fallbacks to the old behaviour only if necessary.
That provides up to 10x parsing speed improvement for usual case, while introducing only 1-2% overhead for cases that really need the more fault tolerant parsing.

Copy link

Unit test report

114 tests   113 ✔️  3s ⏱️
    1 suites      1 💤
    1 files        0

Results for commit 1e047a8.

@sergei-encord sergei-encord merged commit 680b82e into master Nov 30, 2023
3 checks passed
@sergei-encord sergei-encord deleted the speedup-time-parser branch November 30, 2023 11:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant