diff --git a/.dockerignore b/.dockerignore new file mode 100644 index 0000000..1d55356 --- /dev/null +++ b/.dockerignore @@ -0,0 +1,8 @@ +node_modules +dist +.env +.git +.gitignore +__pycache__/ +db.sqlite3 +.venv/ \ No newline at end of file diff --git a/.gitignore b/.gitignore index 7354677..8a673a7 100644 Binary files a/.gitignore and b/.gitignore differ diff --git a/Dockerfile b/Dockerfile new file mode 100644 index 0000000..9a2c915 --- /dev/null +++ b/Dockerfile @@ -0,0 +1,16 @@ +FROM python:3.11.9-slim + +ENV PYTHONDONTWRITEBYTECODE=1 +ENV PYTHONUNBUFFERED=1 + +WORKDIR /app + +COPY requirements.txt . + +RUN pip install --no-cache-dir -r requirements.txt + +COPY . . + +EXPOSE 8000 + +CMD ["python","backend/manage.py","runserver","0.0.0.0:8000"] \ No newline at end of file diff --git a/README.md b/README.md index 59f8c28..3ff4aad 100644 --- a/README.md +++ b/README.md @@ -15,8 +15,7 @@ This README is based on the current codebase, not the older planning documents i - [What Works Today](#what-works-today) - [Stack](#stack) - [Repo Layout](#repo-layout) -- [Local Setup](#local-setup) -- [Background Jobs](#background-jobs) +- [Local Setup (Docker)](#local-setup-docker) - [Frontend Behavior](#frontend-behavior) - [API Surface](#api-surface) - [Testing](#testing) @@ -82,260 +81,25 @@ outreach_frontend/ # currently empty *.md # product and architecture docs from earlier phases ``` -## Local Setup +## Local Setup (Docker) +We use Docker to orchestrate the entire LeadOrbit stack locally, eliminating the need for manual virtual environments and background process management. -### 1. Create a virtual environment and install dependencies +*Prerequisites:* -```sh -python -m venv .venv - -# Windows -.venv\Scripts\activate - -# macOS/Linux -source .venv/bin/activate - -pip install -r requirements.txt -``` - -### 2. Create `backend/.env` - -The backend reads environment variables from `backend/.env`. - -```env -DEBUG=True -SECRET_KEY=change-me -BACKEND_BASE_URL=http://127.0.0.1:8000 -FRONTEND_BASE_URL=http://127.0.0.1:8080 - -CELERY_TASK_ALWAYS_EAGER=true -CELERY_BROKER_URL=redis://localhost:6379/0 -ENABLE_AUTO_REPLY_DETECTION=false -LAUNCH_IMMEDIATE_PASSES=1 - -OPENROUTER_API_KEY= -OPENROUTER_MODEL=mistralai/mistral-nemo -OPENROUTER_APP_URL=http://127.0.0.1:8080 -OPENROUTER_APP_NAME=LeadOrbit Campaign Builder - -GEMINI_API_KEY= - -GOOGLE_CLIENT_ID= -GOOGLE_CLIENT_SECRET= -GOOGLE_REDIRECT_URI=http://127.0.0.1:8000/api/v1/auth/google/callback - -TWILIO_ACCOUNT_SID= -TWILIO_AUTH_TOKEN= -TWILIO_PHONE_NUMBER= -``` - -### 3. Run migrations - -```sh -python backend/manage.py migrate -``` - -### 4. Start the backend - -```sh -cd backend -python manage.py runserver 8000 -``` - -### 5. Serve the frontend - -In a second terminal: - -```sh -cd frontend -python -m http.server 8080 -``` - -Open `http://127.0.0.1:8080/login.html`. - -## Background Jobs - -In local development, campaign processing and CSV import work without Redis because `CELERY_TASK_ALWAYS_EAGER=true` by default when `DEBUG=True`. - -If you want true async/background execution: - -1. Start Redis. -2. Set `CELERY_TASK_ALWAYS_EAGER=false` in `backend/.env`. -3. Run the Celery worker and beat scheduler: - -```sh -cd backend -celery -A backend worker -l info -celery -A backend beat -l info -``` - -Reply polling only runs when `ENABLE_AUTO_REPLY_DETECTION=true`. - -## Frontend Behavior - -- The active frontend is the static `frontend/*.html` app. There is no active Node build step for the current UI. -- [`frontend/api.js`](frontend/api.js) defaults to `http://127.0.0.1:8000/api/v1` on localhost and the deployed Render API elsewhere. -- You can override the API base URL in the browser: - -```js -localStorage.setItem('api_base_url', 'http://127.0.0.1:8000/api/v1'); -``` - -Main screens: - -- `login.html` -- `register.html` -- `dashboard.html` -- `leads.html` -- `campaigns.html` -- `campaign-builder.html` -- `analytics.html` -- `settings.html` - -## API Surface - -Core endpoints exposed by the current backend: - -- `POST /api/v1/auth/register/` -- `GET/PATCH /api/v1/auth/me/` -- `POST /api/v1/token/` -- `POST /api/v1/token/refresh/` -- `GET/POST /api/v1/leads/` -- `POST /api/v1/leads/import_csv/` -- `GET/POST/PATCH/DELETE /api/v1/campaigns/` -- `POST /api/v1/campaigns/{id}/enroll/` -- `POST /api/v1/campaigns/{id}/launch/` -- `POST /api/v1/campaigns/ai-generate/` -- `GET /api/v1/analytics/dashboard/` -- `POST /api/v1/webhooks/email/` -- `GET /api/v1/connected-accounts/` -- `GET /api/v1/auth/google/login` -- `GET /api/v1/auth/google/callback` - -## Testing - -Run the backend test suite from `backend/`: - -```sh -python manage.py test -``` - -Current repo state: `27` backend tests pass. The suite covers auth/profile updates, lead import, tenant isolation, campaign creation, campaign launch, non-email flow handling, conditional branching, connected-account ownership rules, reply polling, and AI fallback behavior. + - Docker and Docker Compose installed on your machine. -## Current Caveats - -- Some older root planning documents still mention the original `Lime` name, but the active README, backend, and frontend runtime paths use `LeadOrbit`. -- The settings page shows a Gemini API key field, but it is not persisted from the UI. AI credentials are read from `backend/.env`. -- The danger-zone buttons in `settings.html` are presentational only right now. -- [`frontend/src`](frontend/src) and [`backend/config`](backend/config) look like leftover scaffold code and are not part of the main runtime path. -- Some endpoints are still MVP-grade and should be hardened before a production multi-tenant deployment. In particular, analytics currently uses unscoped aggregate queries, and a few utility endpoints are intentionally permissive. -- The root markdown docs describe product intent and planning. The codebase is the source of truth for current behavior. - -## Integration Notes - -- Gmail sending requires valid Google OAuth client credentials and a connected account from the Settings page. -- SMS and call steps require Twilio credentials and leads with phone numbers. -- If no AI credentials are configured, the campaign builder AI composer still returns a deterministic fallback draft instead of failing hard. - -# LeadOrbit - -Current branding note: the active app is branded as `LeadOrbit`; older planning documents in the repo may still mention the original `Lime` name. - -LeadOrbit is a multi-tenant outbound outreach MVP built with Django REST Framework and a static HTML/JavaScript frontend. The implemented code supports organization signup, JWT auth, CSV lead import, campaign building, lead enrollment, Gmail sender connection, AI-assisted email drafting, webhook-based engagement tracking, and analytics pages. - -This README is based on the current codebase, not the older planning documents in the repo root. - -## 📑 Table of Contents -- [LeadOrbit](#leadorbit) -- [What Works Today](#what-works-today) -- [Stack](#stack) -- [Repo Layout](#repo-layout) -- [Local Setup](#local-setup) -- [Background Jobs](#background-jobs) -- [Frontend Behavior](#frontend-behavior) -- [API Surface](#api-surface) -- [Testing](#testing) -- [Current Caveats](#current-caveats) -- [Integration Notes](#integration-notes) - -## What Works Today - -- JWT auth with organization creation during signup -- Tenant-scoped users, leads, campaigns, and connected sender accounts -- CSV lead import with flexible column aliases and background task support -- Campaign builder with sequence editing, lead selection, sender selection, and launch flow -- Email sending through Gmail API when a Google account is connected -- SMS and call execution through Twilio when credentials are configured -- Open/reply/click/bounce tracking through the webhook endpoint -- AI draft generation through OpenRouter, with fallback draft generation when no API key is set -- Per-lead merge tag replacement, with Gemini-based send-time personalization when configured -- Dashboard and analytics pages backed by API data -- Celery tasks for CSV import, campaign processing, and reply polling - -Execution status by step type: - -- Fully implemented: `EMAIL`, `SMS`, `CALL`, `WAIT`, `CONDITION_OPEN`, `CONDITION_REPLY`, `CONDITION_CLICK` -- Builder-visible but currently placeholder/auto-advance steps: `WHATSAPP`, `LINKEDIN`, `MANUAL` - -## Stack - -- Backend: Django 5, Django REST Framework, Simple JWT, Celery, `django-cors-headers` -- Frontend: static HTML pages, ES modules, Bootstrap 5, Chart.js -- Database: SQLite by default at `backend/db.sqlite3` -- Integrations: Google OAuth/Gmail API, OpenRouter, Gemini, Twilio -- Python target: `3.11.9` via [`runtime.txt`](runtime.txt) - -## Repo Layout - -```text -backend/ - manage.py - backend/ # active Django project package used by manage.py - campaigns/ # campaign models, serializers, tasks, Gmail/Twilio/AI integrations - leads/ # lead models, CSV import task, lead/tag API - tenants/ # organizations, tenant middleware, security middleware - users/ # custom user model, registration, profile/JWT auth - config/ # older scaffold package; not the active settings module - -frontend/ - *.html # active static product screens - api.js # API base URL + auth token helpers - main.js # shared auth bootstrapping/logout handling - theme.css # shared dashboard styling - src/ # leftover Vite starter files; not used by the live UI - -outreach_frontend/ # currently empty -*.md # product and architecture docs from earlier phases -``` - -## Local Setup - -### 1. Create a virtual environment and install dependencies +### 1. Configure Environment Variables +Create a .env file in the backend/ directory: ```sh -python -m venv .venv - -# Windows -.venv\Scripts\activate - -# macOS/Linux -source .venv/bin/activate - -pip install -r requirements.txt -``` - -### 2. Create `backend/.env` - -The backend reads environment variables from `backend/.env`. - -```env DEBUG=True SECRET_KEY=change-me BACKEND_BASE_URL=http://127.0.0.1:8000 FRONTEND_BASE_URL=http://127.0.0.1:8080 -CELERY_TASK_ALWAYS_EAGER=true -CELERY_BROKER_URL=redis://localhost:6379/0 +CELERY_BROKER_URL=redis://redis:6379/0 +CELERY_RESULT_BACKEND=redis://redis:6379/1 +CELERY_TASK_ALWAYS_EAGER=false ENABLE_AUTO_REPLY_DETECTION=false LAUNCH_IMMEDIATE_PASSES=1 @@ -354,48 +118,27 @@ TWILIO_ACCOUNT_SID= TWILIO_AUTH_TOKEN= TWILIO_PHONE_NUMBER= ``` - -### 3. Run migrations - +### 2. Start the Stack +Run the following command from the root of the repository to build and start the database, backend, frontend, Redis, and Celery workers: ```sh -python backend/manage.py migrate +docker compose up --build ``` -### 4. Start the backend - +### 3. Apply Migrations +Once the containers are running cleanly, open a new terminal window and apply the database migrations: ```sh -cd backend -python manage.py runserver 8000 +docker compose exec backend python backend/manage.py migrate ``` +### 4. Access the Application -### 5. Serve the frontend - -In a second terminal: - -```sh -cd frontend -python -m http.server 8080 -``` - -Open `http://127.0.0.1:8080/login.html`. - -## Background Jobs - -In local development, campaign processing and CSV import work without Redis because `CELERY_TASK_ALWAYS_EAGER=true` by default when `DEBUG=True`. - -If you want true async/background execution: - -1. Start Redis. -2. Set `CELERY_TASK_ALWAYS_EAGER=false` in `backend/.env`. -3. Run the Celery worker and beat scheduler: +1. Frontend UI: http://localhost:8080/login.html +2. Backend API: http://localhost:8000 +To shut down the stack, use-- ```sh -cd backend -celery -A backend worker -l info -celery -A backend beat -l info +docker compose down ``` -Reply polling only runs when `ENABLE_AUTO_REPLY_DETECTION=true`. ## Frontend Behavior diff --git a/backend/celerybeat-schedule b/backend/celerybeat-schedule deleted file mode 100644 index 64aa8be..0000000 Binary files a/backend/celerybeat-schedule and /dev/null differ diff --git a/backend/db.sqlite3 b/backend/db.sqlite3 deleted file mode 100644 index e43ec6b..0000000 Binary files a/backend/db.sqlite3 and /dev/null differ diff --git a/docker-compose.yml b/docker-compose.yml new file mode 100644 index 0000000..41c4974 --- /dev/null +++ b/docker-compose.yml @@ -0,0 +1,64 @@ +services: + frontend: + image: nginx:latest + volumes: + - ./frontend:/usr/share/nginx/html + ports: + - "8082:80" + networks: + - My_Network_new + + backend: + build: + context: . + dockerfile: Dockerfile + command: python backend/manage.py runserver 0.0.0.0:8000 + env_file: + - backend/.env + ports: + - "8000:8000" + networks: + - My_Network_new + volumes: + - .:/app + depends_on: + - redis + + redis: + image: redis:7.2-alpine + ports: + - "6379:6379" + networks: + - My_Network_new + + celery_worker: + build: . + networks: + - My_Network_new + volumes: + - .:/app + env_file: + - backend/.env + working_dir: /app/backend + command: celery -A backend worker -l info + depends_on: + - redis + - backend + + beat_scheduler: + build: . + networks: + - My_Network_new + volumes: + - .:/app + env_file: + - backend/.env + working_dir: /app/backend + command: celery -A backend beat -l info + depends_on: + - redis + - backend + + +networks: + My_Network_new: diff --git a/requirements.txt b/requirements.txt index 1598dc3..20d98f7 100644 --- a/requirements.txt +++ b/requirements.txt @@ -14,3 +14,4 @@ cryptography>=48.0.1 firebase-admin>=6.5,<7.0 twilio>=9.0,<10.0 gunicorn>=20.1,<21.0 +beautifulsoup4 \ No newline at end of file