-
Notifications
You must be signed in to change notification settings - Fork 136
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Eliminate Redundant Fetches in RSS Controller #1442
Eliminate Redundant Fetches in RSS Controller #1442
Conversation
/ok to test |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looking over the changes, I'm getting the feeling that the RSS Controller is getting a bit confusing. Lets refactor the class so the steps are clear and we dont need to check whether we are using a cache or if its a URL so often.
For each item in the input list:
- Call a function to turn the input item into text
- If this is a file, read the file from disk into a string
- If this is a URL, use the session to download the text from the URL
- It should not be required at this stage to check whether or not the cache is enabled
- With the returned text, try to parse it with feedparser
- If feedparser fails, try to parse with BeautifulSoup
That should be the only steps necessary to process the feed and should clean up the code a bit.
/ok to test |
/ok to test |
1 similar comment
/ok to test |
…scontroller-performs-redundant-fetches
/ok to test |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for adding the test
/merge |
Description
Addressed redundant feed fetches in the RSS Controller when parsing manually with BeautifulSoup and cache is enabled.
Closes #1419
By Submitting this PR I confirm: