Skip to content
@scrapy

Scrapy project

An open source and collaborative framework for extracting the data you need from websites. In a fast, simple, yet extensible way.

Pinned Loading

  1. scrapy scrapy Public

    Scrapy, a fast high-level web crawling & scraping framework for Python.

    Python 51.6k 10.4k

  2. scrapy.org scrapy.org Public

    The scrapy.org website

    HTML 60 138

Repositories

Showing 10 of 27 repositories
  • scrapy Public

    Scrapy, a fast high-level web crawling & scraping framework for Python.

    scrapy/scrapy’s past year of commit activity
    Python 51,553 BSD-3-Clause 10,403 441 (21 issues need help) 220 Updated Jun 27, 2024
  • scrapy.org Public

    The scrapy.org website

    scrapy/scrapy.org’s past year of commit activity
    HTML 60 138 1 1 Updated Jun 21, 2024
  • scrapyd Public

    A service daemon to run Scrapy spiders

    scrapy/scrapyd’s past year of commit activity
    Python 2,881 BSD-3-Clause 569 21 5 Updated Jun 18, 2024
  • form2request Public

    AI-powered Python 3.8+ library to build HTTP requests out of HTML forms.

    scrapy/form2request’s past year of commit activity
    Python 2 BSD-3-Clause 0 0 0 Updated Jun 18, 2024
  • parsel Public

    Parsel lets you extract data from XML/HTML documents using XPath or CSS selectors

    scrapy/parsel’s past year of commit activity
    Python 1,098 BSD-3-Clause 136 29 (1 issue needs help) 12 Updated Jun 14, 2024
  • w3lib Public

    Python library of web-related functions

    scrapy/w3lib’s past year of commit activity
    Python 385 BSD-3-Clause 104 11 (1 issue needs help) 5 Updated Jun 12, 2024
  • itemloaders Public

    Library to populate items using XPath and CSS with a convenient API

    scrapy/itemloaders’s past year of commit activity
    Python 44 BSD-3-Clause 15 17 4 Updated Jun 4, 2024
  • itemadapter Public

    Common interface for data container classes

    scrapy/itemadapter’s past year of commit activity
    Python 60 BSD-3-Clause 10 6 3 Updated Jun 3, 2024
  • protego Public

    A pure-Python robots.txt parser with support for modern conventions.

    scrapy/protego’s past year of commit activity
    DIGITAL Command Language 52 BSD-3-Clause 26 6 (1 issue needs help) 1 Updated May 28, 2024
  • queuelib Public

    Collection of persistent (disk-based) and non-persistent (memory-based) queues for Python

    scrapy/queuelib’s past year of commit activity
    Python 264 BSD-3-Clause 54 3 2 Updated May 4, 2024
View all repositories

Top languages

Loading…