|
1 |
| -# Project requirements |
| 1 | +# Fullstack Price Aggregator |
| 2 | +UK supermarket price aggregator via web scraping. This was developed for my university project as an MVP so there are aspects which don't function correctly. I may update it in the future. |
2 | 3 |
|
3 |
| -* Docker |
4 | 4 |
|
5 |
| -Install docker for your platform, open a terminal and cd into the project root, from here run the command "docker-compose up --build". |
6 |
| -This will build the image |
7 |
| -demo |
| 5 | +## Features |
| 6 | +* Python REST backend via FastAPI |
| 7 | +* VueJS Frontend SPA with TypeScript using [this template](https://github.com/Armour/vue-typescript-admin-template). |
| 8 | +* JWT Authentication between backend and frontend |
| 9 | +* CLI |
| 10 | +* [Docker](https://github.com/docker) |
| 11 | +* [Poetry](https://github.com/python-poetry/poetry) |
| 12 | +* Web Scraped prices via [selectorlib](https://selectorlib.com/) |
| 13 | +* Scrape JavaScript enabled pages using [splash](https://github.com/scrapinghub/splash) |
| 14 | + |
| 15 | + |
| 16 | +## Setup |
| 17 | +Ensure you have Docker and Node.js installed. |
| 18 | + |
| 19 | +1. Clone this repository |
| 20 | +2. cd into project root and run `docker-compose up --build -d`. The starts the backend and on first run will build all the necessary containers which can take a few minutes. |
| 21 | +3. Once the backend has finished building, obtain a shell with `docker-compose exec backend bash`. |
| 22 | +4. To initialise the database run `python manage.py createdb` from within the bash shell. |
| 23 | + * (Optional) To seed the db with dummy data run `python manage.py seeddb` |
| 24 | + Note all passwords are set to **password** |
| 25 | +5. `cd` into the client directory run `npm install` which will install the required node modules. |
| 26 | +6. run `npm run serve` which will serve the client frontend. |
| 27 | + |
| 28 | + |
| 29 | +## Project URLs |
| 30 | + |
| 31 | +| URL | Description | |
| 32 | +|:----------------------------------------:|:--------------------------------------------------------------------------------------------| |
| 33 | +| [0.0.0.0:8000/api](0.0.0.0:8000/api) | Backend JSON API | |
| 34 | +| [0.0.0.0:8000/docs](0.0.0.0:8000/docs) | Backend OpenAPI/Swagger-generated API Reference Documentation | |
| 35 | +| [0.0.0.0:8000/redoc](0.0.0.0:8000/redoc) | Alternative interactive documentation provided by [ReDoc](https://github.com/Redocly/redoc) | |
| 36 | +| [localhost:9527](localhost:9527) | Frontend VueJS Single Page Application | |
| 37 | + |
| 38 | + |
| 39 | +## CLI |
| 40 | +The backend includes a CLI which is heavily inspired by [Netflix's Dispatch](https://github.com/Netflix/dispatch). |
| 41 | + |
| 42 | +[`typer`](https://github.com/tiangolo/typer) (same author as FastAPI) was used to create a CLI for the project and is accessed via [manage.py](./server/manage.py). |
| 43 | + |
| 44 | +To run commands you will need to a shell running inside the backend container with: |
| 45 | + |
| 46 | + docker-compose run backend bash |
| 47 | + |
| 48 | +To see all the available commands: |
| 49 | + |
| 50 | + root@72293bee6b37:/app# python manage.py |
| 51 | + Usage: manage.py [OPTIONS] COMMAND [ARGS]... |
| 52 | + |
| 53 | + Options: |
| 54 | + --help Show this message and exit. |
| 55 | + |
| 56 | + Commands: |
| 57 | + config Display application configuration. |
| 58 | + createdb Creates an empty database. |
| 59 | + createrole Add role to database. |
| 60 | + createuser Create new user in the database. |
| 61 | + develop Start a development server with reload. |
| 62 | + dropdb Drop the existing database. |
| 63 | + routes Display application routes and dependencies. |
| 64 | + seeddb Add fake data to database. |
| 65 | + shell Starts an interactive shell with app object imported. |
| 66 | + |
| 67 | +#### Routes |
| 68 | + |
| 69 | + root@72293bee6b37:/app# python manage.py routes |
| 70 | + |
| 71 | + Application Endpoints |
| 72 | + Path Methods Dependencies |
| 73 | + ----------------------------- --------- -------------------------------------------------------- |
| 74 | + /api/v1/users/ GET ['RoleChecker: Roles: admin,user'] |
| 75 | + /api/v1/users/ POST ['RoleChecker: Roles: admin,user'] |
| 76 | + /api/v1/users/{id} GET ['RoleChecker: Roles: admin,user'] |
| 77 | + /api/v1/users/{id} PUT ['RoleChecker: Roles: admin,user'] |
| 78 | + /api/v1/users/{id} DELETE ['RoleChecker: Roles: admin,user'] |
| 79 | + /api/v1/users/{id}/roles GET ['RoleChecker: Roles: admin,user'] |
| 80 | + /api/v1/users/{id}/roles PUT ['RoleChecker: Roles: admin,user'] |
| 81 | + /api/v1/users/{id}/shops GET ['RoleChecker: Roles: admin,user'] |
| 82 | + /api/v1/users/{id}/shops PUT ['RoleChecker: Roles: admin,user'] |
| 83 | + /api/v1/roles/ GET ['RoleChecker: Roles: admin,user'] |
| 84 | + /api/v1/roles/ POST ['RoleChecker: Roles: admin,user'] |
| 85 | + /api/v1/roles/{id} GET ['RoleChecker: Roles: admin,user'] |
| 86 | + /api/v1/roles/{id} PUT ['RoleChecker: Roles: admin,user'] |
| 87 | + /api/v1/roles/{id} DELETE ['RoleChecker: Roles: admin,user'] |
| 88 | + /api/v1/shops/ GET ['RoleChecker: Roles: admin,user'] |
| 89 | + /api/v1/shops/ POST ['RoleChecker: Roles: admin,user'] |
| 90 | + /api/v1/shops/{id} PUT ['RoleChecker: Roles: admin,user'] |
| 91 | + /api/v1/shops/{id} DELETE ['RoleChecker: Roles: admin,user'] |
| 92 | + /api/v1/shops/listings/ GET ['RoleChecker: Roles: admin,user'] |
| 93 | + /api/healthcheck GET [] |
| 94 | + |
| 95 | +#### Config |
| 96 | + |
| 97 | + python manage.py config |
| 98 | + |
| 99 | + Application Configuration |
| 100 | + Setting Value(s) |
| 101 | + -------------------------- -------------------------------------------------------------------- |
| 102 | + APP_DIR /app/app |
| 103 | + STATIC_DIR /app/app/static |
| 104 | + EMAIL_TEMPLATES_DIR /app/app/static/email-templates/html |
| 105 | + PROJECT_NAME Fastapi Backend |
| 106 | + SERVER_HOST 0.0.0.0 |
| 107 | + CORS_WHITELIST ['http://localhost', 'http://localhost:8000', 'http://0.0.0.0:8000'] |
| 108 | + FASTAPI_ENV development |
| 109 | + DEBUG False |
| 110 | + LOG_LEVEL debug |
| 111 | + |
| 112 | + FIRST_SUPERUSER_PASSWORD a5dbf43e07f4d19e5b73bc89a8f74 |
| 113 | + USERS_OPEN_REGISTRATION True |
| 114 | + SECRET_KEY ********** |
| 115 | + JWT_AUTH_LIFETIME_SECONDS 604800 |
| 116 | + JWT_EMAIL_LIFETIME_SECONDS 3600 |
| 117 | + |
| 118 | + SMTP_PASSWORD ********** |
| 119 | + SMTP_TLS False |
| 120 | + SMTP_SSL False |
| 121 | + SMTP_HOST mailhog |
| 122 | + SMTP_PORT 1025 |
| 123 | + POSTGRES_USER postgres |
| 124 | + POSTGRES_PASSWORD ********** |
| 125 | + POSTGRES_HOST postgres |
| 126 | + POSTGRES_PORT 5432 |
| 127 | + POSTGRES_DB fastapi_backend |
| 128 | + |
| 129 | +## Frontend views |
| 130 | + |
| 131 | +### Login view |
| 132 | + |
| 133 | + |
| 134 | +### Admin dashboard view |
| 135 | + |
| 136 | + |
| 137 | +### User CRUD view |
| 138 | + |
| 139 | + |
| 140 | +### User create view |
| 141 | + |
| 142 | + |
| 143 | +### User profile view |
| 144 | + |
| 145 | + |
| 146 | +### Shop CRUD view |
| 147 | + |
| 148 | + |
| 149 | +### Shop select view |
| 150 | + |
| 151 | + |
| 152 | +### Scraped listings view |
| 153 | + |
| 154 | + |
0 commit comments