Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Improve shorts remove key and using tqdm for progress #12

Open
wants to merge 10 commits into
base: feat-async-runner
Choose a base branch
from

Conversation

indrastorms
Copy link
Contributor

@indrastorms indrastorms commented Mar 9, 2024

Removing already marked watched beforehand can increase the perf. Changing "#short" to "short" because some title has #ytshorts

@indrastorms indrastorms changed the title Feat async runner improve shorts remove key Mar 9, 2024
@indrastorms indrastorms changed the title improve shorts remove key improve shorts remove key and using tqdm for progress Mar 10, 2024
main.py Outdated Show resolved Hide resolved
main.py Outdated Show resolved Hide resolved
main.py Outdated Show resolved Hide resolved
@oSumAtrIX
Copy link
Member

@alexandreteles I am not sure if opening a file handle everytime is done correctly here.

@alexandreteles
Copy link
Member

@alexandreteles I am not sure if opening a file handle everytime is done correctly here.

The async file handler deals with that. We could also open a single handler and share between the workers but that will require writing extra logic which aiofiles already implements.

@indrastorms indrastorms changed the title improve shorts remove key and using tqdm for progress feat: Improve shorts remove key and using tqdm for progress Mar 10, 2024
Copy link
Member

@alexandreteles alexandreteles left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@oSumAtrIX Can you test this once last time?

@oSumAtrIX oSumAtrIX self-requested a review March 12, 2024 14:28
@indrastorms
Copy link
Contributor Author

Please update the requirement file with tqdm. I don't have poetry installed 😅

@alexandreteles
Copy link
Member

Please update the requirement file with tqdm. I don't have poetry installed 😅

I will first merge the other changes and then investigate this one again.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants