Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor(LLM config): Restructure LLM configuration and management #276

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

mthezi
Copy link
Contributor

@mthezi mthezi commented Nov 20, 2024

WHAT:

  • Support customizable LLM selection for each processing stage
  • Optimize model selection based on task requirements

WHY:

  • Enable cost optimization by using appropriate models for different tasks
  • Improve translation quality with high-end models where needed

DETAILS:

  • Recommend high-quality models (e.g. claude-3-5-sonnet) for translation tasks
  • Suggest cost-effective models (e.g. Yi-Lightning) with JSON output support for simple operations
  • Refactor config structure to support per-stage LLM configuration
  • Update core modules to respect stage-specific LLM settings

AFFECTED:

  • config.yaml: Add per-stage LLM configuration
  • core/: Update LLM handling in processing modules
  • st_components/: Update UI for LLM selection

Breaking Changes:

  • Configuration file structure updated for LLM settings
  • LLM initialization process modified

WHAT:
- Support customizable LLM selection for each processing stage
- Optimize model selection based on task requirements

WHY:
- Enable cost optimization by using appropriate models for different tasks
- Improve translation quality with high-end models where needed

DETAILS:
- Recommend high-quality models (e.g. claude-3-5-sonnet) for translation tasks
- Suggest cost-effective models (e.g. Yi-Lightning) with JSON output support for simple operations
- Refactor config structure to support per-stage LLM configuration
- Update core modules to respect stage-specific LLM settings

AFFECTED:
- config.yaml: Add per-stage LLM configuration
- core/: Update LLM handling in processing modules
- st_components/: Update UI for LLM selection

Breaking Changes:
- Configuration file structure updated for LLM settings
- LLM initialization process modified
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant