Skip to content

Commit

Permalink
Fix more current_url unrecoverable exception cases
Browse files Browse the repository at this point in the history
Following up on 5211f67
  • Loading branch information
ghostwords committed Jan 6, 2025
1 parent 43aec21 commit 0620644
Showing 1 changed file with 6 additions and 3 deletions.
9 changes: 6 additions & 3 deletions crawler.py
Original file line number Diff line number Diff line change
Expand Up @@ -1069,11 +1069,14 @@ def crawl(self):
self.get_current_url() or domain)

except WebDriverException as ex:
self.logger.error("%s on %s: %s", type(ex).__name__,
self.get_current_url() or domain,
ex.msg)
if should_restart(ex):
self.logger.error("%s on %s: %s", type(ex).__name__,
domain, ex.msg)
self.restart_browser()
else:
self.logger.error("%s on %s: %s", type(ex).__name__,
self.get_current_url() or domain,
ex.msg)

num_total = len(domains)
if num_total:
Expand Down

0 comments on commit 0620644

Please sign in to comment.