Indeed Job Scrapper

This Python-based web scraping project utilizes Selenium and BeautifulSoup to extract job listings for Python Developer positions in Gurgaon, Haryana, from Indeed.

Table of Contents

• Technologies Used
• Features
• Database Structure
• Getting Started
• Overview
• Contributing

Technologies Used

Make sure you have the following installed before running the application:

• Beatiful soup
• Selenium
• Django
• Numpy
• MongoDB

Features

• Extracting Utilizes Selenium and BeautifulSoup to extract job listings for Python Developer positions in Gurgaon, Haryana
• Cleaning The scraped data is then cleaned, including converting job salaries to per annum values and handling missing values.
• Storage The cleaned data is stored in a MongoDB database using MongoDB Compass and Atlas.
• Admin Panel An admin panel is implemented using Django to perform CRUD operations on the job data.

Database Structure

The application uses a database table with the following structure:

Field	Type	Description
job_title	VARCHAR(255)	Title of the job position
company_name	VARCHAR(255)	Name of the hiring company
company_location	VARCHAR(255)	Location of the company
total_salary	INT	Annual salary for the job
City	VARCHAR(255)	City where the job is located
State	VARCHAR(255)	State where the job is located
link	VARCHAR(255)	Link to the job listing

Getting Started

1. Clone the Repository:

   git clone https://github.com/hardik2207/AST-Consulting.git
   cd ASTCONSULTING

2. Run the Scrapper

    run Job scrapper.py

3. Run Admin Panel

   python manage.py runserver

Overview

• Scrapped Jobs

  

• Cleaned Jobs

  

• Data stored in Mongodb

  

• Admin Panel using Django

  1. Admin Login

  2. Add Job

  3. Search Job

  4. Edit Job

  5. Delete Job

• Mean salary of Python developers in Delhi

Contributing

Contributions are welcome! Feel free to open issues or submit pull requests to improve the project.