A Python web scraper that extracts book information from books.toscrape.com, including titles, prices, and availability status. The scraped data is saved to a CSV file for further analysis.
This project demonstrates practical web scraping techniques using Python. It scrapes a multi-page website, handles pagination, and exports the collected data into a structured CSV format.
- Multi-page scraping: Automatically follows pagination links to scrape all pages
- Data extraction: Collects book titles, prices, and availability information
- CSV export: Saves data to
books.csvfor easy analysis - Error handling: Gracefully handles network errors and exceptions
- Rate limiting: Includes polite delays between requests (0.5 seconds)
- Character encoding: Properly handles UTF-8 encoding
pracwebscraping/
├── README.md # Project documentation
├── scraper.py # Main web scraper script
└── books.csv # Output file with scraped book data
- Python 3.6+
requests- HTTP library for fetching web pagesbeautifulsoup4- HTML/XML parsing library
- Clone the repository:
git clone https://github.com/TxcWb/pracwebscraping.git
cd pracwebscraping- Install dependencies:
pip install requests beautifulsoup4Run the scraper:
python scraper.pyThe script will:
- Start scraping from
https://books.toscrape.com/index.html - Parse each page to extract book information
- Follow pagination links automatically
- Display progress messages for each page
- Save all collected books to
books.csv - Display the total number of books found
The books.csv file will contain three columns:
- title: The book title
- price: The book price (including currency symbol)
- availability: Stock availability status
The scraper uses BeautifulSoup to parse HTML and locate:
- Product information in
<article class="product_pod">elements - Book titles in
<h3>→<a>tags - Prices in
<p class="price_color">elements - Availability in
<p class="instock availability">elements - Next page links in
<li class="next">elements
It implements proper URL resolution using urljoin() to handle relative links and includes rate limiting (0.5-second delay between requests) to be respectful to the server.
title,price,availability
A Light in the Attic,£51.77,In stock
Tipping the Velvet,£53.74,In stock
Soumission,£50.10,In stock
...This project teaches:
- HTTP requests and response handling
- HTML parsing with BeautifulSoup
- CSS selector usage for data extraction
- Pagination handling
- CSV file writing
- Exception handling for network errors
- Web scraping best practices (rate limiting, polite delays)
This project is provided as-is for educational purposes.
- The target website (books.toscrape.com) is specifically designed for web scraping practice
- Always check a website's
robots.txtand terms of service before scraping - The 0.5-second delay between requests is a courtesy to avoid overloading servers