Add support for other vector stores

[ashrobertsdragon]

Description

Extend the system to support vector stores beyond PostgreSQL + pgvector.

Current Implementation

The system currently uses PostgreSQL with the pgvector extension as the only vector store backend.

Proposed Enhancement

Add support for additional vector store providers, such as:

• Pinecone: Managed vector database
• Weaviate: Open-source vector search engine
• Qdrant: Vector similarity search engine
• Milvus: Open-source vector database
• Chroma: Embedded vector store
• FAISS: Facebook AI Similarity Search (local option)

Implementation Approach

1. Create a vector store abstraction/interface
2. Implement provider-specific adapters
3. Add configuration option to select vector store
4. Ensure consistent query interface across providers
5. Update documentation with provider-specific setup

Configuration Example

VECTOR_STORE=pgvector
POSTGRES_HOST=localhost
# or
VECTOR_STORE=pinecone
PINECONE_API_KEY=your_key
PINECONE_ENVIRONMENT=us-west1-gcp

Benefits

• Flexibility: Choose the right store for deployment requirements
• Scalability: Use managed services for production workloads
• Cost optimization: Select based on budget constraints
• Local development: Use embedded stores like FAISS or Chroma
• Cloud-native: Deploy with cloud-managed vector databases

Considerations

• Each provider has different query capabilities
• Migration between stores requires re-embedding
• Performance characteristics vary significantly
• Cost models differ across providers