Ability to manually stop crawling

[rushi]

I'd like to be able to stop the crawling process when certain conditions have been met. It would be useful to have:

const crawler = new Crawler('example.com');
crawler.crawl();

setTimeout(() => {
  crawler.stop(); // Emits 'end' event
}, 3000);

[rushi]

Thinking of a way to implement this - Maybe just have a status property within the crawler with three possible values: 'new', 'started', 'complete' and you only crawl to the next page if status != 'complete'

I can submit a PR if you like

[rushi]

Upon a little debugging, I found instead of setting a status to stop the crawling one can set the URL filter this.config.urlFilter = () => false;

[safonovpro]

Hey, @rushi. Sorry for the long answer. Thank you for a good idea! Should be easy to do. I will do it within a week.
P.S. If you want, send your PR.