fix: Resolve reranking variable mismatch and enable config overrides (Issue #465)

[manavgup]

Summary

This PR fixes Issue #465 by resolving three critical issues that prevented reranking from working:

1. Variable name mismatch in reranker (PRIMARY FIX)
2. Backend startup error (BLOCKING BUG)
3. Per-request reranking configuration (FEATURE ENHANCEMENT)

Changes

1. Fix Variable Name Mismatch in Reranker

• Problem: Reranking scores were always 0.0 because LLM prompts had unfilled {document} placeholder
• Root cause: Reranker used "document" variable, WatsonX provider expected "context"
• Fix: Changed variable name from "document" to "context" in reranker.py
• Impact: Reranking now generates valid scores in 0.0-1.0 range
• Files: backend/rag_solution/retrieval/reranker.py:140,174

2. Fix Backend Startup Error

• Problem: Backend failed to start with 'dict' object has no attribute 'model_dump'
• Root cause: system_initialization_service.py passed dict instead of Pydantic model
• Fix: Convert LLMProviderInput to LLMProviderUpdate before passing to update_provider()
• Impact: Backend starts successfully, hot-reload works
• Files: backend/rag_solution/services/system_initialization_service.py:106-107

3. Enable Per-Request Reranking Config

• Problem: No way to enable/disable reranking per search request
• Root cause: Config metadata not passed to _apply_reranking method
• Fix: Added config_metadata parameter to enable per-request reranking control
• Impact: Tests can now compare with/without reranking using config_metadata
• Files: backend/rag_solution/services/search_service.py:239-262,695-700,939-944

Test Results

Before Fix:

• Reranking scores: always 0.0
• Backend startup: failed with Pydantic error
• Config override: not supported

After Fix:

• Reranking scores: 0.0-1.0 range (e.g., top score changed from 0.6278 to 0.8000)
• Backend startup: successful with hot-reload
• Config override: enabled via config_metadata.enable_reranking

Evidence:

WITHOUT reranking: Top score: 0.6278
WITH reranking:    Top score: 0.8000

Test Artifacts

Added two test scripts:

1. test_reranking_impact.py: Demonstrates reranking effect by comparing scores with/without reranking
2. create_reranking_template.py: Creates RERANKING template in database (updated to use correct API endpoint)

Database Change

Created reranking template directly in database:

INSERT INTO prompt_templates (id, user_id, name, template_type, template_format, input_variables, max_context_length, is_default)
VALUES (gen_random_uuid(), 'ee76317f-3b6f-4fea-8b74-56483731f58c', 'Default Reranking Template', 'RERANKING', ..., 4000, true);

Known Limitation

Reranking can only improve the ranking of chunks that are already retrieved. If the embedding model doesn't retrieve relevant chunks in the initial top_k, reranking cannot bring them in. This is a separate embedding model quality issue that will be addressed in subsequent issues.

Related Issues

Closes #465

Checklist

• Code follows project style guidelines
• All linting checks pass (Ruff, MyPy)
• Backend starts successfully
• Reranking functionality verified with test script
• Commit message follows conventional commits format
• Database template created

🤖 Generated with Claude Code

[github-actions]

🚀 Development Environment Options

This repository supports Dev Containers for a consistent development environment.

Option 1: GitHub Codespaces (Recommended)

Create a cloud-based development environment:

1. Click the green Code button above
2. Select the Codespaces tab
3. Click Create codespace on fix/issue-465-reranking-fix
4. Wait 2-3 minutes for environment setup
5. Start coding with all tools pre-configured!

Option 2: VS Code Dev Containers (Local)

Use Dev Containers on your local machine:

1. Install Docker Desktop
2. Install VS Code
3. Install the Dev Containers extension
4. Clone this PR branch locally
5. Open in VS Code and click "Reopen in Container" when prompted

Option 3: Traditional Local Setup

Set up the development environment manually:

# Clone the repository
git clone https://github.com/manavgup/rag_modulo.git
cd rag_modulo
git checkout fix/issue-465-reranking-fix

# Initialize development environment
make dev-init
make dev-build
make dev-up
make dev-validate

Available Commands

Once in your development environment:

make help           # Show all available commands
make dev-validate   # Validate environment setup
make test-atomic    # Run atomic tests
make test-unit      # Run unit tests
make lint          # Run linting

Services Available

When running make dev-up:

• Backend API: http://localhost:8000
• Frontend: http://localhost:3000
• MLflow: http://localhost:5001

---

This automated message helps reviewers quickly set up the development environment.

[github-actions]

Code Review for PR #476

Thank you for this well-documented PR! The fixes address critical issues in the reranking functionality.

Strengths

1. Root Cause Analysis - Excellent identification of the variable name mismatch (document to context)
2. Code Quality - Minimal, surgical changes that follow project conventions
3. Comprehensive Documentation - Clear before/after test results showing improvement (0.6278 to 0.8000)

Issues & Recommendations

1. Critical: Hardcoded User Path in Test Script

File: backend/dev_tests/test_reranking_impact.py:20

Hardcoded personal directory path that won't work for other developers. Fix: Make path relative or remove since TEST_FILE is unused.

2. Missing Unit Tests for New Functionality

File: backend/rag_solution/services/search_service.py:239-262

The new config_metadata parameter in _apply_reranking() is not covered by unit tests. Add tests for:

• Config override enabling/disabling reranking
• Fallback to settings when config_metadata is None
• Fallback when config_metadata doesn't contain enable_reranking

3. Import Placement Anti-Pattern

File: backend/rag_solution/services/system_initialization_service.py:106

Import statement inside a function rather than at module level. Violates PEP 8 and impacts performance. Move import to top of file with other schema imports.

4. Hardcoded User UUID in Test Scripts

Files: backend/dev_tests/create_reranking_template.py:7, backend/dev_tests/test_reranking_impact.py:26

Use environment variable with fallback or add comment explaining this is a specific test user.

Architecture & Design

Positive:

• Per-request config override is well-designed using existing config_metadata pattern
• Graceful fallback logic
• Non-breaking signature change (config_metadata has default None)

Security

No security concerns identified - no credential leaks, proper UUID validation, localhost only.

Performance

No performance issues - minimal overhead, simple dict.get() lookup.

Action Items Before Merge (Required)

1. Fix hardcoded path in test_reranking_impact.py:20 (or remove unused variable)
2. Move import to module level in system_initialization_service.py:106
3. Add unit tests for _apply_reranking config override behavior

Summary

Overall Assessment: Good work with minor issues

This PR successfully fixes the critical reranking bug. The code changes are correct and minimal. Fix the 3 items above, then approve and merge. The fixes are solid and address the root cause effectively!

🤖 Review by Claude Code