-
Notifications
You must be signed in to change notification settings - Fork 9
Add dedicated VLM model configuration and wire it through multi-tenant VLM creation #10
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 2 commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -15,6 +15,8 @@ LLM_API_KEY="your_llm_api_key_here" | |
| LLM_BASE_URL="https://ark.ap-southeast.bytepluses.com/api/v3" | ||
| # 使用的模型名称 | ||
| LLM_MODEL=seed-1-6-250615 | ||
| # 用于多模态图片理解的 VLM 模型(与 LLM_MODEL 解耦) | ||
| VLM_MODEL=seed-1-6-250615 | ||
| # LLM 供应商标识(ark/openai/claude) | ||
| LLM_PROVIDER=ark | ||
| # VLM 图片理解 API 超时时间(秒,默认 120 秒) | ||
|
Comment on lines
15
to
25
Owner
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. LLM和VLM难道不可以换供应商吗?难道一定要用LLM的Base URL吗?
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 已支持 VLM 独立的 API Key/Base URL(env 示例已添加 VLM_API_KEY/VLM_BASE_URL),不再强制复用 LLM Base URL(e5548d2)。 |
||
|
|
@@ -318,4 +320,3 @@ TZ=Asia/Shanghai | |
|
|
||
| # --- Python 配置 --- | ||
| PYTHONUNBUFFERED=1 # 禁用输出缓冲,实时查看日志 | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -10,7 +10,7 @@ | |
|
|
||
| import os | ||
| from typing import Optional | ||
| from pydantic import Field | ||
| from pydantic import Field, field_validator | ||
| from pydantic_settings import BaseSettings | ||
|
|
||
|
|
||
|
|
@@ -22,6 +22,7 @@ class LLMConfig(BaseSettings): | |
| api_key: str = Field(..., description="LLM API Key") | ||
| base_url: str = Field(..., description="LLM API Base URL") | ||
| model: str = Field(default="seed-1-6-250615", description="LLM Model Name") | ||
| vlm_model: str = Field(default="seed-1-6-250615", description="VLM Model Name") | ||
|
Owner
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 这个还是默认回退到seed了,user不设置不一定用这个.直接去掉默认行为,不设置就报错
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 已改为必填且仅使用 VLM_MODEL 配置,不再回退默认值(e5548d2)。 |
||
| vlm_timeout: int = Field(default=120, description="VLM Image Understanding Timeout (seconds)") | ||
| timeout: int = Field(default=60, description="General LLM Timeout (seconds)") | ||
|
|
||
|
|
@@ -35,6 +36,19 @@ class Config: | |
| env_file = ".env" | ||
| extra = "ignore" | ||
|
|
||
| @field_validator("vlm_model", mode="before") | ||
| @classmethod | ||
| def _load_vlm_model(cls, v): | ||
| """ | ||
| 优先读取独立的 VLM_MODEL(保持与 LLM 文本模型解耦) | ||
| 回退顺序: | ||
| 1) 显式传入值 | ||
| 2) 环境变量 VLM_MODEL(独立配置) | ||
| 3) 环境变量 LLM_VLM_MODEL(前缀形式) | ||
| 4) 默认值 | ||
| """ | ||
| return v or os.getenv("VLM_MODEL") or os.getenv("LLM_VLM_MODEL") or "seed-1-6-250615" | ||
|
Owner
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 为什么要多级回退?我们只兼容一个配置,不引入额外的复杂性
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 已去掉多级回退,仅接受独立的 VLM_MODEL(e5548d2)。 |
||
|
|
||
|
|
||
| # ==================== Embedding Configuration ==================== | ||
|
|
||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -51,6 +51,7 @@ def __init__( | |
| self.ark_api_key = config.llm.api_key | ||
| self.ark_base_url = config.llm.base_url | ||
| self.ark_model = config.llm.model | ||
| self.ark_vlm_model = config.llm.vlm_model | ||
|
|
||
| self.sf_api_key = config.embedding.api_key | ||
| self.sf_base_url = config.embedding.base_url | ||
|
|
@@ -278,7 +279,7 @@ def _create_vision_model_func(self, llm_config: Dict): | |
| import aiohttp | ||
|
|
||
| # 从配置中提取参数(支持租户覆盖) | ||
| model = llm_config.get("model", self.ark_model) | ||
| model = llm_config.get("vlm_model", self.ark_vlm_model) | ||
|
Owner
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. ark_model是什么?你还在用火山的命名方式吗?
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 已移除 ark 命名,统一使用 llm/vlm 前缀并保持含义清晰(e5548d2)。 |
||
| api_key = llm_config.get("api_key", self.ark_api_key) | ||
| base_url = llm_config.get("base_url", self.ark_base_url) | ||
| vlm_timeout = llm_config.get("vlm_timeout", self.vlm_timeout) | ||
|
|
@@ -552,4 +553,4 @@ async def get_tenant_lightrag(tenant_id: str) -> LightRAG: | |
| LightRAG: 该租户的实例 | ||
| """ | ||
| manager = get_multi_tenant_manager() | ||
| return await manager.get_instance(tenant_id) | ||
| return await manager.get_instance(tenant_id) | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The documentation says VLM_MODEL is required (必填), but in the code at src/config.py:24, vlm_model uses Field(...) which makes it required at the pydantic validation level. However, the comment also says it should be "independent from LLM_MODEL" (独立于 LLM_MODEL), but the default value for LLM_MODEL and the example value for VLM_MODEL are both the same: seed-1-6-250615. This creates confusion about whether they should actually be different models. Consider clarifying whether VLM_MODEL can use the same model as LLM_MODEL (for models that support both text and vision), or if they must be different models.