Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
51 commits
Select commit Hold shift + click to select a range
bdd418e
feat: implement isolated configuration system and AutoRAG optimization
spyrchat Sep 23, 2025
230506b
feat: Remove data leakage issue because question and answer both got …
spyrchat Sep 23, 2025
729f22b
feat: Add SOSUM dataset analysis tool with publication-quality plotting
spyrchat Sep 23, 2025
1b118d6
feat: Implement StratifiedRAGDatasetSplitter for balanced dataset spl…
spyrchat Sep 24, 2025
6733d35
Refactor retrieval pipeline and embedding components for improved con…
spyrchat Sep 25, 2025
7446e8a
feat: Update evaluation metrics and k-values across benchmark scenari…
spyrchat Sep 25, 2025
17c07b9
feat: Enhance StackOverflowBenchmarkAdapter to support Qdrant integra…
spyrchat Sep 26, 2025
304b0eb
feat: Update benchmark scenarios and dataset configurations for impro…
spyrchat Sep 26, 2025
a93b024
feat: Enhance retrieval configurations and metrics handling; update a…
spyrchat Sep 26, 2025
a67c9ab
feat: Update fusion method configurations and enhance embedding provi…
spyrchat Sep 27, 2025
103b603
feat: Refactor sparse embedder classes; replace SparseEmbedder with B…
spyrchat Sep 27, 2025
c2bab73
feat: Refactor benchmark adapter and enhance embedding classes; imple…
spyrchat Sep 27, 2025
1701159
To be fixed in hybrid retrieval sparse score is 0
spyrchat Sep 27, 2025
ecf1132
Fixed an issue where the thybrid retriever would not return sparse re…
spyrchat Sep 28, 2025
f26f034
feat: Update YAML configuration for Splade embedding provider; change…
spyrchat Sep 28, 2025
19d2a57
feat: Add retrieval time statistics and summary printing to benchmark…
spyrchat Sep 28, 2025
497d39b
Refactor benchmark scenarios and implement new statistical analysis f…
spyrchat Sep 28, 2025
57da761
feat: Enhance benchmark configuration validation and update dataset l…
spyrchat Sep 30, 2025
dc785b6
feat: Add hybrid Splade + BGE-M3 benchmark configuration; implement g…
spyrchat Sep 30, 2025
267a3bc
refactor: Clean up import statements and improve error handling in ut…
spyrchat Sep 30, 2025
a2ddfd5
Remove deprecated dataset adapters and configurations for energy pape…
spyrchat Oct 1, 2025
a1c37db
Refactor hybrid retrieval configurations and optimize alpha parameter
spyrchat Oct 3, 2025
6e0a0f8
feat: implement dynamic adapter loading system
spyrchat Oct 5, 2025
bffd093
Refactor experiment configurations and remove deprecated example
spyrchat Oct 5, 2025
93137a6
Update experiment plots with new performance metrics and visualizations
spyrchat Oct 5, 2025
19f2554
Update experiment 1 plots with new visualizations
spyrchat Oct 6, 2025
d265a91
Add stratification sanity check documentation and new analysis results
spyrchat Oct 7, 2025
f3c2549
Refactor config.yml to streamline agent retrieval settings and update…
spyrchat Oct 7, 2025
7937601
Add scripts and testing framework documentation
spyrchat Oct 8, 2025
cde848c
Revise system requirements and remove support details
spyrchat Oct 8, 2025
ae1b557
Refactor agent schema and retrieval configurations for refined RAG pi…
spyrchat Oct 8, 2025
19378f4
Enhance fast dense retrieval configuration for BGE-M3 with detailed e…
spyrchat Oct 8, 2025
285ee4a
Merge branch 'loaders' of github.com:spyrchat/Thesis into loaders
spyrchat Oct 8, 2025
7b24981
Refactor retrieval configuration and retriever classes
spyrchat Oct 8, 2025
a3f260f
Enhance generator functionality with configurable prompt styles and a…
spyrchat Oct 8, 2025
bb353ca
Add LLM-as-a-Judge evaluation script and implement LLMJudge class for…
spyrchat Oct 9, 2025
2ce03ba
Add graph visualization scripts for RAG agent
spyrchat Oct 9, 2025
4344d88
Refactor stratified splitting approach in StratifiedRAGDatasetSplitter
spyrchat Oct 9, 2025
7579a6c
feat: Update environment configuration and add API keys
spyrchat Oct 10, 2025
5f64a0e
Refactor code structure for improved readability and maintainability
spyrchat Oct 10, 2025
1ccd9c3
Update experiment plots, enhance retrieval configuration, and remove …
spyrchat Oct 12, 2025
0bd3a56
Refactor code structure for improved readability and maintainability
spyrchat Oct 13, 2025
572ebbe
fix: Mark subproject as dirty to indicate uncommitted changes
spyrchat Oct 13, 2025
5475359
feat: Add integration tests for Self-RAG graph with detailed logging
spyrchat Oct 14, 2025
f4169c9
Remove retrieval demo instructions from README
spyrchat Oct 14, 2025
c61b6ab
Update readme.md
spyrchat Oct 14, 2025
49314b5
Revise README to remove troubleshooting and integration details
spyrchat Oct 14, 2025
7cee06e
Update README to remove obsolete benchmark scripts
spyrchat Oct 14, 2025
9fb1e22
Merge branch 'loaders' of github.com:spyrchat/Thesis into loaders
spyrchat Oct 14, 2025
b2c647f
Enhance CLI with config file support for ingest, status, and cleanup …
spyrchat Oct 14, 2025
6466bbb
Remove test coverage section and related notes from README
spyrchat Oct 14, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 4 additions & 7 deletions .env_example
Original file line number Diff line number Diff line change
@@ -1,7 +1,4 @@
QDRANT_HOST = "..."
QDRANT_PORT = ...
QDRANT_COLLECTION="..."

TITAN_S3_BUCKET="..."
TITAN_S3_OUTPUT_URI="..."
TITAN_ROLE_ARN="..."
# === API Keys ===
OPENAI_API_KEY=""
GOOGLE_API_KEY=""
VOYAGE_API_KEY=""
55 changes: 55 additions & 0 deletions CLI_REFERENCE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
# Actual CLI Reference - Verified Against Codebase

**Last Updated:** 2025-10-08
**Purpose:** Single source of truth for all CLI commands that actually exist

## ✅ Scripts WITH CLI Support

### 1. bin/ingest.py
**Subcommands:**
- `ingest` - Ingest a dataset
- `status` - Show pipeline status
- `cleanup` - Clean up canary collections

**Global Flags:**
- `--config`, `-c` - Configuration file path
- `--verbose`, `-v` - Verbose logging

**Ingest Subcommand Flags:**
- `adapter_type` (positional, optional) - Adapter type
- `dataset_path` (positional, optional) - Path to dataset
- `--version` - Dataset version (default: "1.0.0")
- `--split` - Dataset split (choices: train, val, test, all; default: all)
- `--dry-run` - Don't upload to vector store
- `--max-docs` - Maximum documents to process
- `--canary` - Use canary collection
- `--verify` - Run verification after ingestion

### 2. main.py
**Flags:**
- `--query` - Single query to process (enables non-interactive mode)

### 3. benchmarks/experiment1.py
**Flags:**
- `--test` - Run in test mode
- `--output-dir` - Output directory (default: 'results/experiment_1')

### 4. benchmarks/experiment3.py
**Flags:**
- `--test` - Run in test mode
- `--output-dir` - Output directory (default: 'results/experiment_3')

### 5. benchmarks/optimize_2d_grid_alpha_rrfk.py
**Flags:**
- `--scenario-yaml` (required) - Path to scenario YAML
- `--dataset-path` (required) - Path to dataset
- `--n-folds` - Number of folds (default: 5)
- `--max-queries-dev` - Max queries for dev set
- `--max-queries-test` - Max queries for test set
- `--output-dir` - Output directory (default: "results/")

### 6. benchmarks/stratification.py
**Flags:**
- `--dataset-path` (required) - Path to dataset root
- `--fold` - Fold number (default: 0)
- `--split` - Split type (choices: train, dev, test; default: test)
Loading
Loading