Enterprise Grounded RAG Dashboard 🧠💼

A production-ready, clean-architecture Retrieval-Augmented Generation (RAG) dashboard designed for deep document intelligence. The system ingests materials (PDF, DOCX, TXT, MD), splits and embeds them in a persistent vector database, executes high-recall hybrid semantic & keyword retrieval, and performs cross-encoder reranking. Grounded answers are generated with strict citations and streamed in real-time over Server-Sent Events (SSE).

---

🏗️ Architecture Design & RAG Flow

The application is structured using a clean, modular service-oriented architecture:

graph TD
    Client[React Frontend] -->|HTTP/SSE| API[FastAPI Backend]
    API --> Auth[Auth Service]
    API --> DocSvc[Document Service]
    API --> ChatSvc[Chat & RAG Service]
    
    DocSvc -->|Parser/Chunker| Parser[Advanced Chunker]
    Parser -->|Embeddings| Emb[OpenAI text-embedding-3-small]
    Emb -->|Store| DB[(Chroma Vector DB)]
    
    ChatSvc -->|Multi-Query Expansion| LLM_Exp[GPT-4o Query Generator]
    LLM_Exp -->|Query Variants| ChromaRetriever[Chroma Semantic Search]
    LLM_Exp -->|Query Variants| BM25[BM25 Keyword Search]
    
    ChromaRetriever & BM25 -->|Merge Rankings| RRF[Reciprocal Rank Fusion]
    RRF -->|Rerank Chunks| Reranker[Flashrank MiniLM-L-12]
    Reranker -->|Top-N Context + Query| LLM_Gen[OpenAI GPT-4o Generation]
    LLM_Gen -->|Typewriter Stream| Client
    
    DB_SQL[(SQLite Metadata Store)] <--> DocSvc & ChatSvc & Auth

Loading

1. Ingestion & Advanced Chunking Pipeline

• Parsing: Extracts text page-by-page from PDFs, paragraphs from Word files (.docx), and direct content from Markdown/text files.
• Chunking with Context Enrichment: Splits documents using LangChain's RecursiveCharacterTextSplitter. To enhance semantic precision, we perform Metadata Prefixing: each chunk is prefixed with its document name and page coordinates (e.g. [Source: employee_handbook.pdf | Page: 4]) before embedding. This prevents LLM context loss.
• Vector Storage: Generates 1536-dim embeddings via OpenAI's text-embedding-3-small and indexes them in a persistent Chroma DB collection.

2. Multi-Query Hybrid Retrieval & Fusion

• Multi-Query Expansion: GPT-4o dynamically expands the user query into 3 search variants to overcome vocabulary mismatch and improve recall.
• Hybrid Retrieval: Runs a dual-path search for all query variants:
  1. Semantic Path: Cosine similarity search over ChromaDB.
  2. Lexical Path: BM25 keyword matching over all text chunks belonging to the user's uploaded documents (multi-tenant isolation).
• Reciprocal Rank Fusion (RRF): Merges the semantic and keyword candidate lists. A reciprocal rank calculation assigns high scoring weights to chunks appearing near the top of either list.
• Cross-Encoder Reranking: Uses Flashrank (ms-marco-MiniLM-L-12-v2) to re-score the fused candidate pool, selecting the absolute top-N context chunks to send to the generator.

3. Generation, Memory & Streaming

• Hallucination Prevention: Prompts OpenAI's GPT-4o with strict instructions to base answers solely on the retrieved chunks, state "I cannot find the answer..." if information is missing, and output detailed inline source citations (e.g. [Source: report.pdf | Page: 3]).
• SSE Streaming: Uses async generators to stream answer tokens to the frontend in real-time, yielding a typewriter typing effect.
• SQL Conversational History: Chat history is persisted in SQLite (via SQLAlchemy) to maintain short-term dialogue context.

---

🛠️ Technology Stack

• Backend: Python, FastAPI, SQLAlchemy, Pydantic v2
• RAG & Orchestration: LangChain, LlamaIndex, Rank-BM25, Flashrank
• Embeddings & LLMs: OpenAI GPT-4o, OpenAI text-embedding-3-small
• Vector Database: ChromaDB (persistent local format)
• Security: PyJWT (JSON Web Tokens), BCrypt Hashing
• Frontend: Vite, React, Vanilla CSS (Glassmorphic dark theme)
• Deployment: Docker, Docker Compose, Nginx (SSE Reverse Proxy)

---

🚀 Setup & Execution Instructions

Option A: Quickstart via Docker Compose (Recommended)

This spins up the FastAPI backend, the React frontend (behind an Nginx reverse proxy on port 80), and a Redis caching server.

1. Clone the Repository and navigate to the project directory.
2. Create the Environment File: Copy backend/.env.example to backend/.env and edit it to add your OpenAI key:

   cp backend/.env.example backend/.env
   # Add your OPENAI_API_KEY

3. Start Containers:

   docker compose up -d --build

4. Access the Application:
   • Dashboard Interface: http://localhost/
   • Backend OpenAPI (Swagger): http://localhost:8000/docs
   • Backend health check: http://localhost:8000/api/health

---

Option B: Local Manual Setup (Development Mode)

1. Backend Setup

1. Navigate to backend and create virtual env:

   cd backend
   python3 -m venv venv
   source venv/bin/activate

2. Install Dependencies:

   pip install -r requirements.txt

3. Configure Environment: Ensure backend/.env is set up with a valid OPENAI_API_KEY.
4. Boot the Backend Server:

   python -m backend.app.main
   # or
   uvicorn backend.app.main:app --reload --port 8000

2. Frontend Setup

1. Navigate to frontend:

   cd ../frontend

2. Install Node Packages:

   npm install

3. Run Vite Development Server:

   npm run dev

   Open http://localhost:3000/ in your browser. All API requests are automatically proxied to the backend at port 8000.

---

🧪 Testing and Verification

The backend includes automated tests covering chunking logic, RRF fusion calculations, and API endpoint routing.

To execute tests:

cd backend
pytest -v

---

🔗 Key API Endpoint Documentation

Method	Endpoint	Description	Auth Required
POST	/api/auth/register	Create user account	No
POST	/api/auth/login	Authenticate and obtain JWT token	No
GET	/api/auth/me	Fetch user profile	Yes (Bearer)
POST	/api/documents/upload	Upload document (PDF/DOCX/TXT/MD)	Yes (Bearer)
GET	/api/documents	List uploaded documents	Yes (Bearer)
DELETE	/api/documents/{doc_id}	Purge document and chunks from DB	Yes (Bearer)
POST	/api/chat/ask	Normal JSON Question & Answer	Yes (Bearer)
POST	/api/chat/stream	Server-Sent Events (SSE) Streaming QA	Yes (Bearer)
GET	/api/chat/sessions	Fetch user conversation history list	Yes (Bearer)
GET	/api/health	System health check (DB connectivity)	No

API cURL Usage Examples

1. Register a User

curl -X POST http://localhost:8000/api/auth/register \
  -H "Content-Type: application/json" \
  -d '{"username": "johndoe", "email": "john@company.com", "password": "securepassword123"}'

2. Login to Get Token

curl -X POST http://localhost:8000/api/auth/login \
  -H "Content-Type: application/json" \
  -d '{"username": "johndoe", "password": "securepassword123"}'

Response:

{
  "access_token": "eyJhbGciOiJIUzI1NiIsIn...",
  "token_type": "bearer"
}

3. Upload a Document

curl -X POST http://localhost:8000/api/documents/upload \
  -H "Authorization: Bearer <YOUR_ACCESS_TOKEN>" \
  -F "file=@/path/to/document.pdf"

4. Ask a Question (Normal JSON response)

curl -X POST http://localhost:8000/api/chat/ask \
  -H "Authorization: Bearer <YOUR_ACCESS_TOKEN>" \
  -H "Content-Type: application/json" \
  -d '{"question": "What is the net revenue for Q4?", "top_k": 4}'

---

📈 Future System Improvements

1. Graph RAG / Entity Extraction: Incorporate Neo4j knowledge graphs to support multi-hop reasoning over document schemas.
2. Evaluation Framework (Ragas / TruLens): Add automated evaluations to monitor faithfulness, answer relevance, and context recall in CI/CD pipelines.
3. Advanced Semantic Caching: Implement semantic caching via Redis with threshold similarity filters (e.g. $0.96+$ cosine similarity) to instantly serve cached answers.
4. Local LLM Support: Support Ollama / vLLM backends to allow fully on-premise, offline vector computations and inference.