Automatic re-crawling of RSS feeds

[aecio]

Support (re)crawling of RSS feeds similarly to sitemaps.xml.

[thawkins]

I would like to upvote this item, if there are any pointers as to where to add this item, we could contribute some resources to get it done.

[aecio]

I'd be happy to accept a Pull Request. We won't be able to work on this on the next few weeks (maybe months).
The relevant code to start to look at are these:

• How the SitemapsRecrawlSelector works: https://github.com/ViDA-NYU/ache/blob/master/src/main/java/focusedCrawler/link/frontier/selector/SitemapsRecrawlSelector.java
• Where it is used to select which links should be loaded into the crawl queue: https://github.com/ViDA-NYU/ache/blob/17577ccc9a43121f722843ce914ab02f0538be41/src/main/java/focusedCrawler/link/frontier/CrawlScheduler.java#L91-L150

Similar code would need to be written for the RSS feeds.

[thawkins]

We will look at this and see if we can submit a PR, I can assign resources internaly from in my org to work on this.