Skip to content

Commit 0ce7882

Browse files
kutcodeclaude
andcommitted
Update documentation to reflect current features and file support
- README: add XLSX support, source traceability, contradiction detection, SME routing, environment variables reference, supported file formats table, quick-start script - README: fix broken local file paths to use relative links, update architecture table to include PostgreSQL/Supabase and openpyxl, remove explicit AI provider branding - CONTRIBUTING: add XLSX, contradiction detection, and SME routing to welcome list - USE_CASES: add Excel-based questionnaire section, update shared ops with new features - .env.example: add SME routing config variables - frontend/README: fix broken relative link Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
1 parent 7fba558 commit 0ce7882

5 files changed

Lines changed: 124 additions & 57 deletions

File tree

.env.example

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -65,6 +65,10 @@ QF_AGENT_MAX_QUESTIONS_PER_CALL=40 # Max questions sent per LLM call
6565
# QF_AGENT_ANTHROPIC_API_KEY=sk-ant-...
6666
# QF_AGENT_ANTHROPIC_MODEL=claude-sonnet-4-6
6767

68+
# ─── SME Routing (optional) ──────────────────────────────────────
69+
# QF_SME_ROUTING_ENABLED=false
70+
# QF_CATEGORY_SME_MAP={"Security": "security@company.com", "Privacy": "privacy@company.com"}
71+
6872
# ─── Feedback / Corrections ──────────────────────────────────────
6973
# QF_FEEDBACK_AUTO_ADD_TO_KB=true # Auto-add user corrections to KB on finalize
7074
# QF_FEEDBACK_MIN_CONFIDENCE=0.0 # Only auto-add if original confidence was above this

CONTRIBUTING.md

Lines changed: 11 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,9 +6,12 @@ Thanks for helping improve TrustReply.
66

77
Contributions are especially useful in these areas:
88

9-
- parser support for additional document layouts
9+
- parser support for additional document layouts (DOCX, PDF, XLSX, CSV)
1010
- PDF extraction and OCR improvements
11+
- Excel format preservation and edge-case handling
1112
- knowledge-base management improvements
13+
- contradiction detection and duplicate classification enhancements
14+
- SME routing and workflow improvements
1215
- batch workflow and review UX
1316
- tests, fixtures, and reproducible bug cases
1417
- documentation and onboarding improvements
@@ -39,6 +42,12 @@ npm install
3942
npm run build
4043
```
4144

45+
### Quick Start (Both Services)
46+
47+
```bash
48+
./start-backend.sh
49+
```
50+
4251
## Pull Requests
4352

4453
Please keep pull requests focused and include:
@@ -58,6 +67,7 @@ This repository is open source, so developers may use and modify it under the pr
5867
- Add tests for parser, matcher, generator, or API behavior when possible.
5968
- Keep user-facing changes clear and practical.
6069
- Prefer deterministic behavior for sensitive document workflows.
70+
- When adding Excel support, ensure format preservation (dropdowns, merged cells, data validation, styles).
6171

6272
## Reporting Security Issues
6373

README.md

Lines changed: 86 additions & 48 deletions
Original file line numberDiff line numberDiff line change
@@ -11,22 +11,26 @@ The product approach is:
1111
- reuse trusted answers from a curated knowledge base
1212
- fill documents automatically where confidence is high
1313
- flag missing or uncertain questions for review
14+
- detect contradictions and duplicates in your knowledge base
15+
- route flagged questions to the right subject-matter experts
1416
- learn from resolved questions over time
1517

1618
## What It Does
1719

18-
- Upload `.docx`, `.pdf`, and `.csv` questionnaires
19-
- Parse tables, row-block layouts, paragraphs, CSV grids, and several document profile variants
20+
- Upload `.docx`, `.pdf`, `.xlsx`, and `.csv` questionnaires
21+
- Parse tables, row-block layouts, paragraphs, Excel worksheets, CSV grids, and several document profile variants
2022
- Match questions against a Q&A knowledge base using sentence-transformer embeddings
21-
- Optionally run a two-stage AI agent workflow (Research Agent + Fill Agent) for context-aware answers
22-
- Write answers back into supported documents while preserving formatting where possible
23+
- Optionally run a two-stage agent workflow (Research Agent + Fill Agent) for context-aware answers
24+
- Write answers back into supported documents while preserving formatting (including Excel dropdowns, merged cells, and styles)
25+
- Track answer sources with full traceability back to the originating KB entry
26+
- Detect contradictions between knowledge-base entries automatically
27+
- Route flagged questions to subject-matter experts by category
2328
- Group repeated flagged questions so teams answer them once instead of many times
2429
- Export unresolved flagged questions as a simple `category,question,answer` CSV
2530
- Import completed CSVs back into the knowledge base
2631
- Sync flagged questions against newly imported knowledge-base answers
2732
- Run bulk uploads with batch summaries and downloadable batch ZIP outputs
2833
- Troubleshoot difficult files by comparing parser profiles before retrying
29-
- Run optional AI troubleshooting analysis with root-cause hints and trace logs
3034

3135
## Primary Use Cases
3236

@@ -39,33 +43,35 @@ TrustReply is useful anywhere a team repeatedly answers structured questionnaire
3943
- Procurement and due-diligence packets
4044
- Internal operations and compliance forms
4145

42-
More examples are documented in [docs/USE_CASES.md](/Users/kutluhanbayram/Desktop/AI%20QUESTIONNAIRE%20FILLER%20-%20DOCS/docs/USE_CASES.md).
46+
More examples are documented in [docs/USE_CASES.md](docs/USE_CASES.md).
4347

4448
## Product Workflow
4549

4650
1. Add approved Q&A pairs to the Knowledge Base.
47-
2. Upload one file or a batch of questionnaire documents or CSV questionnaires.
51+
2. Upload one file or a batch of questionnaire documents (.docx, .pdf, .xlsx, or .csv).
4852
3. TrustReply parses the document and matches questions to known answers.
4953
4. Optional agent mode can research context, fill unresolved answers, and flag uncertain prompts.
50-
5. Review auto-filled answers in the inline review queue with confidence scores.
54+
5. Review auto-filled answers in the inline review queue with confidence scores and source traceability.
5155
6. Edit or approve answers, then finalize and download the completed document.
52-
7. Unresolved questions are grouped in the Flagged Questions queue.
56+
7. Unresolved questions are grouped in the Flagged Questions queue, with optional SME routing.
5357
8. Export missing questions as CSV, fill in answers, and re-import them.
5458
9. Sync flagged questions with the updated knowledge base.
5559

5660
## Key Features
5761

58-
- **Knowledge Base Management**: CRUD, categories, search, CSV/JSON import/export, duplicate detection
62+
- **Knowledge Base Management**: CRUD, categories, search, CSV/JSON import/export, duplicate detection, contradiction detection
5963
- **Semantic Matching**: embedding-based question matching for paraphrased prompts
64+
- **Source Traceability**: every answer links back to the KB entry it was matched from, with category, similarity score, and source Q&A visible in review
65+
- **Contradiction Detection**: automatically flags KB entries with conflicting answers on the same topic
66+
- **SME Routing**: route flagged questions to category-specific subject-matter experts via configurable email mappings
6067
- **Confidence Score Visibility**: per-answer confidence badges (green/yellow/red) so reviewers focus on low-confidence answers
61-
- **Answer Review Queue**: inline review table after processing approve, edit, or override any answer before downloading
68+
- **Answer Review Queue**: inline review table after processing -- approve, edit, or override any answer before downloading
6269
- **Finalize & Download**: regenerate the output document with edited answers after review
63-
- **AI Agent Mode**: default `agent` mode with contextual research/fill workflows
64-
- **Provider Model Discovery**: Settings can pull model dropdown options directly from OpenAI and Claude APIs
65-
- **Test Connection**: verify AI provider credentials from the Settings page
70+
- **Excel Support**: full .xlsx round-trip -- parse questions from Excel workbooks and write answers back preserving dropdowns, merged cells, data validation, and cell styles
71+
- **Agent Mode**: optional agent mode with contextual research/fill workflows
6672
- **Agent Instruction Presets**: built-in and custom instruction presets for common answering styles
67-
- **Parser Profiles**: multiple layout strategies for document and CSV questionnaire structures
68-
- **Troubleshooting**: compare parser profiles plus optional AI diagnostics and trace output
73+
- **Parser Profiles**: multiple layout strategies for document, Excel, and CSV questionnaire structures
74+
- **Troubleshooting**: compare parser profiles plus optional advanced diagnostics and trace output
6975
- **Human-in-the-loop Review**: grouped flagged questions, resolution flow, and KB sync
7076
- **Batch Processing**: upload up to 50 files in one batch, track per-file results, and download ZIP outputs
7177
- **Confirmation Dialogs**: styled confirmation modals for destructive actions (delete, bulk dismiss)
@@ -77,18 +83,19 @@ More examples are documented in [docs/USE_CASES.md](/Users/kutluhanbayram/Deskto
7783
|---|---|
7884
| Frontend | Next.js / React |
7985
| Backend | FastAPI |
80-
| Database | SQLite + SQLAlchemy |
81-
| Document Parsing | `python-docx`, `pdfplumber` |
86+
| Database | PostgreSQL (Supabase) or SQLite |
87+
| Document Parsing | `python-docx`, `pdfplumber`, `openpyxl` |
8288
| Semantic Matching | `sentence-transformers` (`all-MiniLM-L6-v2`) |
8389
| Styling | Vanilla CSS |
8490

8591
## Repository Layout
8692

8793
```text
88-
backend/ FastAPI app, parser/matcher/generator services, tests, scripts
89-
frontend/ Next.js app
90-
test-data/ sample documents and generated stress-test corpora
91-
docs/ product and contributor documentation
94+
backend/ FastAPI app, parser/matcher/generator services, tests, scripts
95+
frontend/ Next.js app
96+
test-data/ 34 sample questionnaires across xlsx, docx, pdf, and csv formats
97+
docs/ product and contributor documentation
98+
start-backend.sh one-command startup script for local development
9299
```
93100

94101
## Local Development
@@ -115,6 +122,14 @@ Open [http://localhost:3000](http://localhost:3000).
115122

116123
Note: the frontend `.env.local` is configured for `http://127.0.0.1:8001`, so run backend on port `8001` for local development.
117124

125+
### Quick Start (Both Services)
126+
127+
```bash
128+
./start-backend.sh
129+
```
130+
131+
This kills stale processes, starts both backend and frontend via `nohup`, and verifies health.
132+
118133
## Docker
119134

120135
```bash
@@ -151,56 +166,79 @@ QF_AGENT_MAX_CONTEXT_CHARS=6000
151166

152167
Notes:
153168

154-
- Supported providers in Settings: OpenAI API and Claude API (Anthropic).
169+
- Two providers are supported and configurable in Settings.
155170
- Parser profiles are still used to anchor exact question/answer placement in output documents.
156171
- Configure provider/base URL/model/key in the Settings page (keys are persisted in backend env settings).
157172

158-
### Claude API (Anthropic)
159-
160-
TrustReply also supports native Claude API settings:
173+
Docker quick-start without saving your key in source:
161174

162175
```bash
163-
QF_AGENT_PROVIDER=anthropic
164-
QF_AGENT_API_BASE=https://api.anthropic.com/v1
165-
QF_AGENT_API_KEY=your_anthropic_api_key
166-
QF_AGENT_MODEL=claude-3-5-haiku-latest
167-
QF_AGENT_DEFAULT_MODE=agent
176+
export QF_AGENT_API_KEY=your_api_key
177+
export QF_AGENT_ENABLED=true
178+
docker compose up --build
168179
```
169180

170-
Docker quick-start without saving your key in source:
181+
## SME Routing Setup
182+
183+
SME routing is optional and disabled by default. Enable it via Settings or environment variables:
171184

172185
```bash
173-
export OPENAI_API_KEY=your_api_key
174-
export QF_AGENT_ENABLED=true
175-
docker compose up --build
186+
QF_SME_ROUTING_ENABLED=true
187+
QF_CATEGORY_SME_MAP='{"Security": "security-team@company.com", "Privacy": "privacy@company.com"}'
176188
```
177189

190+
When enabled, flagged questions are automatically assigned to the configured SME email for their category upon resolution.
191+
178192
## Example Data
179193

180-
The repository includes example content under [test-data](/Users/kutluhanbayram/Desktop/AI%20QUESTIONNAIRE%20FILLER%20-%20DOCS/test-data), including:
194+
The repository includes 34 realistic GRC questionnaires under [test-data](test-data/), including:
195+
196+
- 11 Excel (.xlsx) files with dropdowns, merged cells, and color themes
197+
- 11 Word (.docx) files with styled tables and section headings
198+
- 10 PDF files with numbered questions and section dividers
199+
- 1 CSV file
200+
- A seed script (`backend/scripts/seed_demo_data.py`) to populate the knowledge base with 43 Q&A pairs across 10 GRC categories
201+
202+
## Supported File Formats
181203

182-
- starter knowledge-base CSVs
183-
- example questionnaire files
184-
- generated parser stress-test corpora for DOCX and CSV uploads
204+
| Format | Upload | Output |
205+
|---|---|---|
206+
| `.docx` | Full table/paragraph/row-block parsing | Answers written back preserving styles |
207+
| `.pdf` | Question extraction via pdfplumber | Answers written to new PDF |
208+
| `.xlsx` | Auto-detect question/answer columns across sheets | Answers written back preserving dropdowns, merged cells, validation, and styles |
209+
| `.csv` | Tabular questionnaire parsing | Processed and matched |
185210

186211
## Current Limitations
187212

188-
- PDF handling is more limited than DOCX write-back
189-
- CSV support is designed for tabular questionnaire layouts rather than arbitrary spreadsheet-style workbooks
190-
- scanned PDFs still need OCR support for best results
191-
- parser coverage is good for many common layouts, but not every possible enterprise form
192-
- the current default database is SQLite, which is ideal for local use and prototypes but not long-term multi-tenant production
213+
- PDF write-back is more limited than DOCX and XLSX round-trip
214+
- Scanned PDFs still need OCR support for best results
215+
- Parser coverage is good for many common layouts, but not every possible enterprise form
216+
217+
## Environment Variables Reference
218+
219+
| Variable | Description | Default |
220+
|---|---|---|
221+
| `QF_DATABASE_URL` | Database connection string | SQLite local |
222+
| `QF_SUPABASE_URL` | Supabase project URL (optional) ||
223+
| `QF_SUPABASE_ANON_KEY` | Supabase anonymous key (optional) ||
224+
| `QF_SUPABASE_JWT_SECRET` | JWT secret for auth (optional) ||
225+
| `QF_AGENT_ENABLED` | Enable agent mode | `false` |
226+
| `QF_AGENT_PROVIDER` | Provider name | `openai` |
227+
| `QF_AGENT_API_KEY` | Provider API key ||
228+
| `QF_AGENT_MODEL` | Model identifier | `gpt-4.1-nano` |
229+
| `QF_SME_ROUTING_ENABLED` | Enable SME routing | `false` |
230+
| `QF_CATEGORY_SME_MAP` | JSON map of category to SME email | `{}` |
193231

194232
## Open Source
195233

196234
TrustReply is released under the MIT License. That means other developers can use, modify, and distribute the software under the terms of that license.
197235

198-
Maintainers still control what gets merged into the official upstream project. If you want to contribute improvements back to the main repository, please follow [CONTRIBUTING.md](/Users/kutluhanbayram/Desktop/AI%20QUESTIONNAIRE%20FILLER%20-%20DOCS/CONTRIBUTING.md).
236+
Maintainers still control what gets merged into the official upstream project. If you want to contribute improvements back to the main repository, please follow [CONTRIBUTING.md](CONTRIBUTING.md).
199237

200238
## Contributing
201239

202240
Issues, fixes, parser improvements, UX improvements, and new document-layout support are all welcome.
203241

204-
- Contribution guide: [CONTRIBUTING.md](/Users/kutluhanbayram/Desktop/AI%20QUESTIONNAIRE%20FILLER%20-%20DOCS/CONTRIBUTING.md)
205-
- Code of conduct: [CODE_OF_CONDUCT.md](/Users/kutluhanbayram/Desktop/AI%20QUESTIONNAIRE%20FILLER%20-%20DOCS/CODE_OF_CONDUCT.md)
206-
- License: [LICENSE](/Users/kutluhanbayram/Desktop/AI%20QUESTIONNAIRE%20FILLER%20-%20DOCS/LICENSE)
242+
- Contribution guide: [CONTRIBUTING.md](CONTRIBUTING.md)
243+
- Code of conduct: [CODE_OF_CONDUCT.md](CODE_OF_CONDUCT.md)
244+
- License: [LICENSE](LICENSE)

docs/USE_CASES.md

Lines changed: 22 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -11,17 +11,18 @@ Security teams and revenue teams regularly receive questionnaires asking about:
1111
- incident response
1212
- secure SDLC
1313

14-
TrustReply helps answer these using an approved answer library so teams do not rewrite the same response for every prospect or partner.
14+
TrustReply helps answer these using an approved answer library so teams do not rewrite the same response for every prospect or partner. Source traceability links each answer back to the KB entry it came from for audit purposes.
1515

1616
## Vendor Risk Assessments
1717

1818
Procurement and third-party risk teams often send very similar forms in different document formats. TrustReply can:
1919

20-
- parse common questionnaire layouts
21-
- handle tabular CSV questionnaires alongside document uploads
20+
- parse common questionnaire layouts across DOCX, PDF, XLSX, and CSV
21+
- handle Excel workbooks with dropdowns and merged cells
2222
- reuse approved vendor-risk answers
23-
- flag missing questions for follow-up
23+
- flag missing questions for follow-up and route them to category-specific SMEs
2424
- group repeated unresolved prompts so they are answered once
25+
- detect contradictions between existing KB entries to maintain consistency
2526

2627
## Privacy Reviews
2728

@@ -56,21 +57,35 @@ During procurement or partnership reviews, teams are often asked for:
5657

5758
TrustReply can fill the common parts and surface the exceptions.
5859

60+
## Excel-Based Questionnaires
61+
62+
Many enterprises distribute questionnaires as Excel workbooks with:
63+
64+
- dropdown validation cells (Yes/No/N/A)
65+
- merged header rows
66+
- color-coded sections
67+
- multiple sheets
68+
69+
TrustReply parses these natively and writes answers back while preserving all formatting, data validation, and cell styles.
70+
5971
## Shared Answer Operations
6072

6173
TrustReply is not only a document filler. It is also useful as a team workflow around institutional knowledge:
6274

6375
- maintain a categorized answer base
64-
- review flagged questions in one queue
76+
- review flagged questions in one queue with SME routing
77+
- detect and resolve contradictions across KB entries
6578
- export unresolved items for SME completion in CSV form
6679
- sync newly answered questions back into the platform
80+
- track answer provenance with source traceability
6781

6882
## Stress Testing and Parser Validation
6983

70-
The repository includes generated document corpora so maintainers can test:
84+
The repository includes 34 generated questionnaire files across XLSX, DOCX, PDF, and CSV formats so maintainers can test:
7185

7286
- common layouts already expected to work
7387
- mixed-coverage documents with some known and some unknown answers
74-
- parser behavior across row-block, multi-column, paragraph-based, and CSV-based layouts
88+
- parser behavior across row-block, multi-column, paragraph-based, Excel, and CSV-based layouts
89+
- Excel format preservation including dropdowns, merged cells, and styles
7590

7691
This makes TrustReply useful both as an end-user product and as a platform contributors can improve.

frontend/README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22

33
This directory contains the Next.js frontend for TrustReply.
44

5-
For full project setup, product overview, and use cases, start with the root [README.md](/Users/kutluhanbayram/Desktop/AI%20QUESTIONNAIRE%20FILLER%20-%20DOCS/README.md).
5+
For full project setup, product overview, and use cases, start with the root [README.md](../README.md).
66

77
## Local Development
88

0 commit comments

Comments
 (0)