fix: Resolve all configuration issues and duplicates

Critical Fixes:
- Fix duplicate /api/market-data prefix: Changed market_data_workflow to /api/market-data/workflow
- Fix Celery app name: Changed from 'quantum_trading_matrix' to 'octopus_trading_platform'
- Fix Celery worker queues: Added all missing queues (ml_training, risk, strategies, analytics, generative, llm)
- Fix agents_v2.py prefix: Changed to /api/agents/v2 to avoid conflicts
- Standardize queue naming: Use data_processing (underscore) consistently

Documentation:
- Add API_ROUTES_DOCUMENTATION.md: Comprehensive route documentation
- Add CONFIGURATION_ISSUES.md: Detailed issue tracking and fixes
- Add MARKET_DATA_WORKFLOW.md: Market data workflow documentation

All critical route conflicts and misconfigurations resolved.

Author: massoudsh

API_ROUTES_DOCUMENTATION.md

@@ -0,0 +1,178 @@
+# API Routes Documentation
+
+## Overview
+
+This document provides a comprehensive guide to all API routes in the Octopus Trading Platform.
+
+## Route Organization
+
+### Authentication & Authorization
+- `/api/auth/*` - Professional Authentication (`professional_auth.py`)
+- `/auth/*` - Legacy Authentication (`auth.py`)
+
+### Market Data APIs
+
+#### Professional Market Data
+- **Prefix**: `/api/market-data`
+- **Router**: `professional_market_data.py`
+- **Purpose**: Professional-grade real-time and historical market data
+- **Features**: Real-time quotes, historical data, technical indicators
+
+#### Market Data Workflow
+- **Prefix**: `/api/market-data/workflow`
+- **Router**: `market_data_workflow.py`
+- **Purpose**: Workflow management for fetching market data from free APIs
+- **Features**: Batch fetching, async tasks, source status, historical data
+
+#### Legacy Market Data (V1)
+- **Prefix**: `/api/v1/market-data`
+- **Router**: `market_data.py` (from routes)
+- **Purpose**: Legacy API for backward compatibility
+
+#### Simple Real Market Data
+- **Prefix**: `/api` (specific endpoints)
+- **Router**: `simple_real_data.py`
+- **Purpose**: Simple, lightweight market data endpoints
+
+### Real-Time Data
+- **Prefix**: `/api/v1/realtime`
+- **Router**: `realtime.py` (from routes)
+- **Purpose**: Real-time data streaming and updates
+
+### Machine Learning
+- **Prefix**: `/api/v1/ml`
+- **Router**: `ml_models.py` (from routes)
+- **Purpose**: ML model management and predictions
+
+### Portfolio Management
+- **Prefix**: `/portfolios`
+- **Router**: `portfolios.py` (from routes)
+- **Purpose**: Portfolio CRUD operations
+
+### Agent Monitoring
+- **Prefix**: `/api/agents`
+- **Router**: `agents.py`
+- **Purpose**: AI agent status, logs, and decisions
+
+- **Prefix**: `/api/agents/v2`
+- **Router**: `agents_v2.py`
+- **Purpose**: Alternative/backup agent monitoring API
+
+### Wallet & Funding
+- **Prefix**: `/api/wallet`
+- **Router**: `wallet.py`
+- **Purpose**: Multi-currency balances, transactions, bank accounts
+
+### Security & Access Control
+- **Prefix**: `/api/security`
+- **Router**: `security.py`
+- **Purpose**: API keys, sessions, trading permissions
+
+### Market Scenarios
+- **Prefix**: `/api/scenarios`
+- **Router**: `scenarios.py`
+- **Purpose**: Market regime scenario analysis
+
+### Risk Management
+- **Prefix**: `/api/risk`
+- **Router**: `risk.py`
+- **Purpose**: Risk metrics and analysis
+
+### LLM & AI Analytics
+- **Prefix**: `/llm`
+- **Router**: `llm_simple.py`
+- **Purpose**: Language model operations
+
+### Real Data Sources
+- **Prefix**: `/api` (specific endpoints)
+- **Routers**: 
+  - `macro_data.py` - Macro economic data
+  - `onchain_data.py` - On-chain crypto data
+  - `social_data.py` - Social sentiment data
+
+### WebSocket Endpoints
+
+#### Main WebSocket
+- **Path**: `/ws`
+- **Type**: Direct WebSocket endpoint
+- **Purpose**: Real-time market data streaming
+- **Implementation**: `main_refactored.py` (direct endpoint)
+
+#### WebSocket API (V1)
+- **Prefix**: `/api/v1/websocket`
+- **Router**: `websocket.py` (from routes)
+- **Purpose**: Legacy WebSocket API
+
+#### WebSocket Real-time
+- **Prefix**: `/api/ws`
+- **Router**: `websocket_realtime.py`
+- **Purpose**: Real-time agent status and wallet transaction updates
+
+### Comprehensive API
+- **Prefix**: `/api`
+- **Router**: `comprehensive_api.py`
+- **Purpose**: Comprehensive trading operations and analytics
+
+## Route Conflicts Resolution
+
+### Market Data Routes
+All market data routes are now properly separated:
+- `/api/market-data` - Professional API
+- `/api/market-data/workflow` - Workflow management
+- `/api/v1/market-data` - Legacy V1 API
+- `/api/real-market-data` - Simple endpoints
+
+### Agent Routes
+- `/api/agents` - Main agent monitoring
+- `/api/agents/v2` - Alternative/backup API
+
+### WebSocket Routes
+- `/ws` - Main WebSocket (direct)
+- `/api/v1/websocket` - Legacy API
+- `/api/ws` - Real-time updates
+
+## Best Practices
+
+1. **Use Professional APIs**: Prefer `/api/market-data` over legacy routes
+2. **Workflow API**: Use `/api/market-data/workflow` for batch operations
+3. **WebSocket**: Use `/ws` for main real-time streaming
+4. **Versioning**: V1 routes are for backward compatibility only
+
+## Migration Guide
+
+### From Legacy to Professional APIs
+
+**Old**: `/api/v1/market-data/current/{symbol}`
+**New**: `/api/market-data/quote/{symbol}`
+
+**Old**: `/api/v1/market-data/historical/{symbol}`
+**New**: `/api/market-data/historical/{symbol}`
+
+### From Simple to Workflow API
+
+**Old**: `/api/real-market-data`
+**New**: `/api/market-data/workflow/fetch/{symbol}`
+
+## Authentication
+
+Most endpoints require JWT authentication:
+```
+Authorization: Bearer <token>
+```
+
+Exceptions:
+- `/health` - Public health check
+- `/docs` - API documentation
+- `/` - Root endpoint
+
+## Rate Limiting
+
+All authenticated endpoints have rate limiting:
+- Default: 30 requests per minute
+- Some endpoints may have custom limits
+
+Check response headers:
+- `X-RateLimit-Limit`: Maximum requests
+- `X-RateLimit-Remaining`: Remaining requests
+- `X-RateLimit-Reset`: Reset time
+

CONFIGURATION_ISSUES.md

@@ -0,0 +1,138 @@
+# Configuration Issues & Fixes
+
+## 🔴 Critical Issues Found
+
+### 1. **DUPLICATE API PREFIX: `/api/market-data`**
+
+**Problem:**
+- `professional_market_data.py` has prefix: `/api/market-data`
+- `market_data_workflow.py` has prefix: `/api/market-data`
+- Both are registered in `main_refactored.py`, causing route conflicts!
+
+**Location:**
+- `Modules/src/api/endpoints/professional_market_data.py:20`
+- `Modules/src/api/endpoints/market_data_workflow.py:27`
+- `Modules/src/main_refactored.py:135, 159`
+
+**Fix:**
+Change one of the prefixes. Recommended:
+- Keep `professional_market_data.py` as `/api/market-data`
+- Change `market_data_workflow.py` to `/api/market-data/workflow`
+
+### 2. **DUPLICATE AGENTS ROUTER**
+
+**Problem:**
+- `agents.py` has prefix: `/api/agents`
+- `agents_v2.py` has prefix: `/api/agents` (duplicate!)
+- Only `agents.py` is registered, but `agents_v2.py` exists with same prefix
+
+**Location:**
+- `Modules/src/api/endpoints/agents.py:35`
+- `Modules/src/api/endpoints/agents_v2.py:23`
+
+**Fix:**
+- Remove or rename `agents_v2.py` if not needed
+- Or change prefix to `/api/agents/v2` if it's a versioned API
+
+### 3. **Celery Worker Missing Queues**
+
+**Problem:**
+Worker command only listens to: `default,data-processing,prediction,backtesting,portfolio`
+
+But `task_routes` defines additional queues:
+- `ml_training`
+- `risk`
+- `strategies`
+- `analytics`
+- `generative`
+- `llm`
+
+**Location:**
+- `Modules/docker-compose-core.yml:111`
+- `Modules/src/core/celery_app.py:49-60`
+
+**Fix:**
+Update celery-worker command to include all queues:
+```yaml
+command: celery -A src.core.celery_app worker -l info -Q default,data-processing,market_data,ml_training,prediction,backtesting,portfolio,risk,strategies,analytics,generative,llm
+```
+
+### 4. **Celery App Name Mismatch**
+
+**Problem:**
+- Celery app name: `"quantum_trading_matrix"`
+- Platform name: `"Octopus Trading Platform"`
+
+**Location:**
+- `Modules/src/core/celery_app.py:13`
+
+**Fix:**
+Change to: `"octopus_trading_platform"` for consistency
+
+### 5. **Multiple Market Data Routers**
+
+**Problem:**
+Multiple market data routers with overlapping functionality:
+1. `market_data.router` (from routes) - prefix `/api/v1/market-data`
+2. `market_router` (professional_market_data) - prefix `/api/market-data`
+3. `market_data_workflow_router` - prefix `/api/market-data` (CONFLICTS!)
+4. `simple_data_router` - prefix `/api`
+
+**Location:**
+- `Modules/src/main_refactored.py:136, 135, 159, 144`
+
+**Fix:**
+Consolidate or clearly separate:
+- `/api/v1/market-data` - Legacy/v1 API
+- `/api/market-data` - Professional API
+- `/api/market-data/workflow` - Workflow API (rename)
+- `/api/real-market-data` - Simple real data (rename)
+
+## ⚠️ Medium Priority Issues
+
+### 6. **WebSocket Endpoints**
+
+**Problem:**
+Multiple WebSocket implementations:
+- `websocket.router` - prefix `/api/v1/websocket`
+- `ws_realtime_router` - prefix `/api/ws`
+- Direct endpoint `/ws`
+
+**Location:**
+- `Modules/src/main_refactored.py:139, 156, 162`
+
+**Fix:**
+Consolidate or document which to use:
+- `/ws` - Main WebSocket endpoint
+- `/api/v1/websocket` - Legacy API
+- `/api/ws` - Real-time updates
+
+### 7. **Missing Queue in Task Routes**
+
+**Problem:**
+`market_data.*` tasks route to `data_processing` queue, but worker command uses `data-processing` (with hyphen)
+
+**Location:**
+- `Modules/src/core/celery_app.py:51`
+- `Modules/docker-compose-core.yml:111`
+
+**Fix:**
+Ensure consistency: Use `data_processing` (underscore) everywhere
+
+## 📝 Recommended Fixes
+
+### Priority 1 (Critical - Route Conflicts)
+1. Fix duplicate `/api/market-data` prefix
+2. Remove or rename `agents_v2.py`
+3. Update Celery worker queues
+
+### Priority 2 (Consistency)
+4. Rename Celery app to match platform
+5. Consolidate market data routers
+6. Document WebSocket endpoints
+
+### Priority 3 (Cleanup)
+7. Remove unused routers if any
+8. Standardize naming conventions
+9. Add route documentation
+