Skip to content

Patty-OFurniture/icrawler

 
 

Repository files navigation

iCrawlerTK

Introduction

  • See "iCrawlerTK.*" in /examples/
  • UI in tkinter for icrawler https://github.com/hellock/icrawler/
  • Overloading, MonkeyPatching, core code patches, logging, and of course TK because it comes with python (Tcl/Tk options)
  • Offers the basic, common options for hellock/icrawler, based on the provided example crawl.py
  • This also includes minor tweaks if needed to the original icrawler library, Same license (currently MIT) as the iCrawler package
  • Name files as Content-Disposition has them, or from the URL otherwise
  • File type detection
  • Language selector (based on Google, since icrawler has it as a param already), TODO: Baidu, which solves a common error
  • config file

Code (C) 2024 Patty O'Furniture Same license (currently MIT) as the iCrawler package

> [!TIP] > For more advanced usage about built-in crawlers, please refer to the [documentation](http://icrawler.readthedocs.io/en/latest/builtin.html).

> Writing your own crawlers with this framework is also convenient, see the [tutorials](http://icrawler.readthedocs.io/en/latest/extend.html).

About

A multi-thread crawler framework with many builtin image crawlers provided.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Python 100.0%