Skip to content
/ scrapy Public
forked from scrapy/scrapy

Scrapy, a fast high-level web crawling & scraping framework for Python.

License

Notifications You must be signed in to change notification settings

s-pace/scrapy

This branch is 4259 commits behind scrapy/scrapy:master.

Folders and files

NameName
Last commit message
Last commit date

Latest commit

f729d74 · Oct 5, 2017
Jan 23, 2017
Jul 30, 2015
Oct 5, 2017
Jul 24, 2017
Oct 5, 2017
Jan 23, 2017
Oct 2, 2017
May 18, 2017
Aug 13, 2015
Feb 13, 2017
Sep 20, 2017
Mar 14, 2013
Dec 27, 2016
Jul 24, 2015
Oct 2, 2012
Apr 21, 2017
Nov 8, 2016
Jun 16, 2015
Apr 29, 2012
May 29, 2017
May 18, 2017
Jan 18, 2016
May 4, 2015
Feb 14, 2017
Feb 14, 2017
Mar 1, 2016
Jun 19, 2017
Sep 13, 2017

Scrapy

PyPI Version Build Status Wheel Status Coverage report Conda Version

Overview

Scrapy is a fast high-level web crawling and web scraping framework, used to crawl websites and extract structured data from their pages. It can be used for a wide range of purposes, from data mining to monitoring and automated testing.

For more information including a list of features check the Scrapy homepage at: http://scrapy.org

Requirements

  • Python 2.7 or Python 3.3+
  • Works on Linux, Windows, Mac OSX, BSD

Install

The quick way:

pip install scrapy

For more details see the install section in the documentation: http://doc.scrapy.org/en/latest/intro/install.html

Documentation

Documentation is available online at http://doc.scrapy.org/ and in the docs directory.

Releases

You can find release notes at https://doc.scrapy.org/en/latest/news.html

Community (blog, twitter, mail list, IRC)

See http://scrapy.org/community/

Contributing

See http://doc.scrapy.org/en/master/contributing.html

Code of Conduct

Please note that this project is released with a Contributor Code of Conduct (see https://github.com/scrapy/scrapy/blob/master/CODE_OF_CONDUCT.md).

By participating in this project you agree to abide by its terms. Please report unacceptable behavior to opensource@scrapinghub.com.

Companies using Scrapy

See http://scrapy.org/companies/

Commercial Support

See http://scrapy.org/support/

About

Scrapy, a fast high-level web crawling & scraping framework for Python.

Resources

License

Code of conduct

Stars

Watchers

Forks

Packages

No packages published

Languages

  • Python 99.6%
  • Other 0.4%