Skip to content

Scraped a website to extract valuable information. Did this for a company. I used Python, BeautifulSoup, Selenium to achieve my objectives.

License

Notifications You must be signed in to change notification settings

mikael19/PythonWebScraping

Repository files navigation

How to run

OS constraints:

  • Ubuntu 16
  • Python 3.6.0
  • chromedriver

Create a directory

This directory will be the virtualenv for this project You could name it webscrapper_env

cd to the directory

cd webscrapper_env

create the virtualenv

virtualenv .

clone this repository

git clone https://github.com/mikael19/py_web_scrapper.git

cd to the cloned repo

cd py_web_scrapper

Install the scrapper app

pip install -e .

Install the app's requirements

pip install -r requirements.txt

activate the virtualenv

source ../bin/activate

Scrappe the website

webscraper_cli --url a The excel sheet will be in the current directory, open it to see your results !

About

Scraped a website to extract valuable information. Did this for a company. I used Python, BeautifulSoup, Selenium to achieve my objectives.

Topics

Resources

License

Stars

Watchers

Forks

Languages