Skip to content

PrinceThummar011/Docuchat

BranchesTags

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

23 Commits
.devcontainer
.devcontainer
 
 
docuchat
docuchat
 
 
results
results
 
 
tests
tests
 
 
.gitignore
.gitignore
 
 
README.md
README.md
 
 
pyproject.toml
pyproject.toml
 
 
requirements.txt
requirements.txt
 
 
uv.lock
uv.lock
 
 

Repository files navigation

🤖 DocuChat

RAG-powered document assistant — ask questions, get accurate answers from your own files

Python Streamlit LangChain Groq FAISS Tests Hit Rate uv License: MIT

🚀 Live Demo

📌 What is DocuChat?

DocuChat lets you upload any document (PDF, DOCX, TXT) and chat with it using a full RAG pipeline. Instead of dumping the whole document into a prompt, it semantically retrieves only the most relevant chunks and sends them to Groq's LLM — giving precise, grounded answers from your exact content.

✨ Features

Feature Description
📄 Multi-format support Upload PDF, DOCX, and TXT files
🔍 MMR semantic search FAISS + Maximal Marginal Relevance finds diverse, relevant passages
🤖 Accurate answers Strict document-grounded responses, no outside hallucination
📚 Multi-document Query across multiple documents at once
💬 Conversation memory Follow-up questions use the last 3 turns as context
🏷️ Source citations Answers reference which document and section they came from
⚡ Fast inference Groq's llama-3.3-70b-versatile at ~12ms retrieval latency
🔒 Private Embedding model runs 100% locally; only top chunks leave your machine

🏗️ RAG Architecture

  ┌─────────────────────────────────────────────────────────────────┐
  │                    INDEXING  (on upload)                        │
  │                                                                 │
  │  PDF/DOCX/TXT ──► Text Extraction ──► _clean_text()            │
  │                   + Page Labels        (unicode, whitespace)    │
  │                   + Table Extraction                            │
  │                           │                                     │
  │                           ▼                                     │
  │               RecursiveCharacterTextSplitter                    │
  │               chunk_size=1000 | overlap=200                     │
  │                           │                                     │
  │                           ▼                                     │
  │               HuggingFace Embeddings                            │
  │               (all-MiniLM-L6-v2, normalized)                    │
  │                           │                                     │
  │                           ▼                                     │
  │                   FAISS Vector Store                            │
  └─────────────────────────────────────────────────────────────────┘

  ┌─────────────────────────────────────────────────────────────────┐
  │                 RETRIEVAL + GENERATION                          │
  │                                                                 │
  │  User Question ──► Embed Question                               │
  │                         │                                       │
  │                         ▼                                       │
  │           MMR Search (fetch_k=20, select k=6)                   │
  │           + Relevance Score Filter (≥ 0.25)                     │
  │           + [Source N: filename] labels                         │
  │                         │                                       │
  │                         ▼                                       │
  │        System Prompt + Chat History (last 3 turns)              │
  │        + Document Context + Question                            │
  │                         │                                       │
  │                         ▼                                       │
  │         Groq LLM (llama-3.3-70b-versatile, temp=0.1)           │
  │                         │                                       │
  │                         ▼                                       │
  │              Grounded Answer with Source References ✅          │
  └─────────────────────────────────────────────────────────────────┘

🗂️ Project Structure

Docuchat/
├── docuchat/                   # Main Python package
│   ├── __init__.py
│   ├── core/
│   │   ├── document.py         # PDF / DOCX / TXT extraction + cleaning
│   │   ├── rag.py              # FAISS store, MMR retrieval, RAG pipeline
│   │   └── validator.py        # GROQ API key validation
│   └── ui/
│       └── app.py              # Streamlit chat UI
├── tests/
│   ├── evaluate_rag.py         # Retrieval accuracy evaluation (no API key needed)
│   ├── test_unit.py            # 44 pytest unit tests
│   └── fixtures/               # Sample documents for testing
│       ├── company_policy.txt  # HR / policy document
│       ├── product_spec.txt    # Technical specification
│       └── research_paper.txt  # Academic paper
├── results/
│   └── eval_report.json        # Latest evaluation results (auto-generated)
├── uploads/                    # Temporary uploaded files (gitignored)
├── pyproject.toml
└── README.md

🛠️ Tech Stack

Layer Technology
UI Streamlit
LLM Framework LangChain (langchain-groq, langchain-core)
LLM Provider Groq API — llama-3.3-70b-versatile
Vector Store FAISS (faiss-cpu)
Embeddings HuggingFace — all-MiniLM-L6-v2 (local)
Text Splitting langchain-text-splitters — RecursiveCharacterTextSplitter
Doc Parsing PyPDF2, python-docx
Package Manager uv
Testing pytest + custom retrieval evaluator

⚙️ Setup & Installation

Prerequisites

  • Python 3.11+
  • uv — install with: 
    curl -LsSf https://astral.sh/uv/install.sh | sh

Install dependencies

git clone https://github.com/PrinceThummar011/Docuchat.git
cd Docuchat
uv sync

Run the app

uv run streamlit run docuchat/ui/app.py

Open http://localhost:8501 in your browser.

🧪 Testing & Evaluation

All tests run without a Groq API key — only the embedding model (local) is required.

Run unit tests

uv run pytest tests/test_unit.py -v

Run retrieval accuracy evaluation

uv run python tests/evaluate_rag.py        # print report
uv run python tests/evaluate_rag.py --json # also save results/eval_report.json

📊 Evaluation Results

Last evaluated: March 11, 2026 · Embedding model: all-MiniLM-L6-v2 · Chunk size: 1000 · Overlap: 200

Unit Test Suite — pytest tests/test_unit.py

Test Class Tests Result
TestCleanText 7 ✅ 7 / 7 passed
TestTextExtraction 6 ✅ 6 / 6 passed
TestVectorStore 7 ✅ 7 / 7 passed
TestRetrievalAccuracy 15 ✅ 15 / 15 passed
TestApiKeyValidation 9 ✅ 9 / 9 passed
Total 44 ✅ 44 / 44 passed

Retrieval Accuracy Evaluation — evaluate_rag.py

Methodology: 30 factual QA pairs were manually created across 3 different test documents (HR policy, technical specification, research paper). For each question, the pipeline retrieves the top-6 chunks from a combined FAISS index. A question is marked a "hit" if any retrieved chunk contains the expected answer keyword(s). No LLM call is made — this is a pure retrieval quality test.

Overall Metrics

Metric Score What it means
Hit Rate @1 80.0% Correct answer in the very first retrieved chunk
Hit Rate @3 86.7% Correct answer found within top 3 chunks
Hit Rate @6 96.7% Correct answer found within top 6 chunks
MRR (Mean Reciprocal Rank) 0.847 Average quality of ranking (1.0 = always rank-1)
Precision @6 26.1% Fraction of retrieved chunks that are truly relevant
Avg Retrieval Latency 12.2 ms Time to retrieve top-6 chunks per query

Per-Document Breakdown

Document Type Questions @1 @3 @6 MRR
company_policy.txt HR / Policy 10 80.0% 80.0% 100% 0.850
product_spec.txt Technical Spec 10 80.0% 90.0% 100% 0.858
research_paper.txt Academic Paper 10 80.0% 90.0% 90.0% 0.833

Per-Question Results

ID Question (summarised) @1 @3 @6
CP-01 Remote work days per week ✅ ✅ ✅
CP-02 PTO accrual rate — year 1 ❌ ❌ ✅
CP-03 Sick days per year ✅ ✅ ✅
CP-04 Primary caregiver parental leave ✅ ✅ ✅
CP-05 Health insurance premium coverage % ❌ ❌ ✅
CP-06 Annual professional development budget ✅ ✅ ✅
CP-07 401k plan administrator ✅ ✅ ✅
CP-08 Duration of a PIP ✅ ✅ ✅
CP-09 Screen lock timeout requirement ✅ ✅ ✅
CP-10 Bereavement days — immediate family ✅ ✅ ✅
PS-01 Peak CEC efficiency ✅ ✅ ✅
PS-02 Maximum DC input power ✅ ✅ ✅
PS-03 Rated AC output power ✅ ✅ ✅
PS-04 Number of MPPT inputs ✅ ✅ ✅
PS-05 Ingress protection rating ✅ ✅ ✅
PS-06 Inverter weight ❌ ❌ ✅
PS-07 Standard warranty period ✅ ✅ ✅
PS-08 Communication protocols ❌ ✅ ✅
PS-09 Operating temperature range ✅ ✅ ✅
PS-10 Safety certifications ✅ ✅ ✅
RP-01 Executive function reduction % ✅ ✅ ✅
RP-02 Number of study participants ✅ ✅ ✅
RP-03 Device used to measure sleep ✅ ✅ ✅
RP-04 Decision-making error increase % ✅ ✅ ✅
RP-05 Performance overestimation gap ❌ ✅ ✅
RP-06 Study duration ✅ ✅ ✅
RP-07 Recommended sleep hours (NSF) ✅ ✅ ✅
RP-08 Does caffeine offset severe CPSD? ✅ ✅ ✅
RP-09 Institution that conducted the study ❌ ❌ ❌
RP-10 Cognitive test battery used ✅ ✅ ✅

Only 1 question missed at @6: RP-09 ("Which institution?") — the word "Stanford" appears only in the document header/author affiliation, which FAISS does not rank highly for abstract institution-name queries. This is a known limitation of dense retrieval on metadata-style facts.

📝 Important Notes

For Reviewers / Interviewers

  • All evaluation metrics are real — measured by running tests/evaluate_rag.py locally. No numbers were fabricated. You can reproduce them with uv run python tests/evaluate_rag.py.
  • No Groq API key is required to run the evaluation or unit tests. The embedding model (all-MiniLM-L6-v2) runs locally.
  • The 1 missed question (RP-09) is documented honestly. It reflects a genuine limitation of dense retrieval: when the answer is in a document header rather than the body text, the embedding similarity may not rank it highly.

Design Decisions

Decision Rationale
Chunk size 1000 (not 256–500) Smaller chunks cut answers mid-sentence; larger chunks provide full context
MMR retrieval (not top-k cosine) Pure cosine returns near-duplicate chunks; MMR ensures diversity
llama-3.3-70b-versatile The 8b-instant model gave shorter, less detailed answers
Temperature 0.1 Lower temperature = more deterministic, factual answers
Conversation history (last 3 turns) Enables follow-up questions like "what about the second point?"
Score filter ≥ 0.25 Removes noise chunks that confuse the LLM into hallucinating

Known Limitations

  • Scanned PDFs (image-only): PyPDF2 cannot extract text from image-based PDFs. Use OCR tools (Tesseract) as a pre-processing step.
  • Very large documents (>50 pages): Indexing is fast, but the FAISS store is rebuilt in-memory on every upload. For production use, persist the index to disk.
  • Tables in PDFs: PDF table extraction is limited. DOCX tables are fully extracted.
  • Dense retrieval blind spot: Rare named entities that appear only in document metadata (author names, institution headers) may not retrieve correctly, as seen in RP-09.

🚀 How To Use

Step 1 ──► Get a free GROQ API key at https://console.groq.com/keys
           Paste it in the sidebar (starts with gsk_)

Step 2 ──► Upload your documents (PDF / DOCX / TXT)
           Knowledge base builds automatically in the background

Step 3 ──► Ask any question in the chat box
           e.g. "What are the key responsibilities?"
                "Summarize the contract terms"
                "What is the project deadline?"
                "What did you mean in your previous answer?"  ← follow-ups work!

Step 4 ──► Get accurate, source-grounded answers ✅
           e.g. "According to [Source: contract.pdf], the deadline is March 31."

🔑 Get a Free GROQ API Key

  1. Go to console.groq.com/keys
  2. Sign up / Log in (free)
  3. Click Create API Key
  4. Copy the key (starts with gsk_)
  5. Paste it in the DocuChat sidebar

Groq offers a generous free tier — no credit card required.

🔒 Privacy & Security

  • ✅ No API keys are stored or hardcoded in the repo
  • ✅ Uploaded documents are stored locally only in uploads/
  • ✅ Your key is used only to call the Groq API on your behalf
  • ✅ The embedding model (all-MiniLM-L6-v2) runs 100% locally
  • ✅ Only the top-6 most relevant text chunks leave your machine (to Groq)

About

Chat with your PDF, DOCX, and TXT files using AI. Upload a document and start asking questions. 💬📄

docuchat-by-prince.streamlit.app/

Resources

Readme
Activity

Stars

0 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases

Packages

Contributors

Languages