-
Notifications
You must be signed in to change notification settings - Fork 9
feat: Add strict grounding mode to prevent AI from fabricating answers when context is insufficient #4
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
feat: Add strict grounding mode to prevent AI from fabricating answers when context is insufficient #4
Changes from 3 commits
Commits
Show all changes
8 commits
Select commit
Hold shift + click to select a range
4d741ae
Initial plan
Copilot 7033584
feat: 添加严格 Grounding 模式,防止 AI 在知识库信息不足时编造答案
Copilot a64b77a
fix: 修复代码审查发现的问题 - 统一引用格式和提取辅助函数
Copilot 4cc8df8
docs: 添加查询模式与 Prompt 对应关系说明
Copilot fe23a33
fix: 处理 strict_grounding 配置的布尔值类型,改进 env.example 文档
Copilot 58bc7be
更新 prompt_manager.py
BukeLy e24f8dc
更新 prompt_manager.py
BukeLy 0fdf1af
更新 prompt_manager.py
BukeLy File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,148 @@ | ||
| # 严格 Grounding 模式 | ||
|
|
||
| ## 问题背景 | ||
|
|
||
| 默认情况下,当知识库中的 chunks 不足以回答用户问题时,AI 可能会"强行编造"一个答案,即使这个答案没有依据。这是因为 LLM 有强大的生成能力,即使在没有足够上下文的情况下也会尝试给出"看起来合理"的回答。 | ||
|
|
||
| ## 解决方案 | ||
|
|
||
| 通过启用 **严格 Grounding 模式**,系统会使用增强版的 prompt,明确要求 AI 在信息不足时: | ||
|
|
||
| 1. **评估上下文充分性**:在生成答案前,先判断上下文是否足够回答问题 | ||
| 2. **明确拒绝回答**:如果信息不足,使用标准化的拒绝回答格式 | ||
| 3. **禁止编造内容**:严格禁止使用 AI 的通用知识来填补知识库的空白 | ||
|
|
||
| ## 使用方法 | ||
|
|
||
| ### 方式 1:环境变量(全局配置) | ||
|
|
||
| 在 `.env` 文件中添加: | ||
|
|
||
| ```bash | ||
| # 启用严格 Grounding 模式 | ||
| LIGHTRAG_STRICT_GROUNDING=true | ||
| ``` | ||
|
|
||
| ### 方式 2:租户配置(租户级覆盖) | ||
|
|
||
| 通过 API 更新租户配置: | ||
|
|
||
| ```bash | ||
| curl -X PUT "http://localhost:8000/tenants/your_tenant/config" \ | ||
| -H "Content-Type: application/json" \ | ||
| -d '{ | ||
| "custom_prompts": { | ||
| "strict_grounding": "true" | ||
| } | ||
| }' | ||
| ``` | ||
|
|
||
| ### 方式 3:完全自定义 Prompt | ||
|
|
||
| 如果需要更精细的控制,可以完全自定义 RAG 响应 prompt: | ||
|
|
||
| ```bash | ||
| # 环境变量方式 | ||
| LIGHTRAG_RAG_RESPONSE_PROMPT="你的自定义 prompt..." | ||
| LIGHTRAG_NAIVE_RAG_RESPONSE_PROMPT="你的自定义 naive 模式 prompt..." | ||
| ``` | ||
|
|
||
| 或通过租户配置 API: | ||
|
|
||
| ```bash | ||
| curl -X PUT "http://localhost:8000/tenants/your_tenant/config" \ | ||
| -H "Content-Type: application/json" \ | ||
| -d '{ | ||
| "custom_prompts": { | ||
| "rag_response": "你的自定义 KG 模式 prompt...", | ||
| "naive_rag_response": "你的自定义 naive 模式 prompt..." | ||
| } | ||
| }' | ||
| ``` | ||
|
|
||
| ## 配置优先级 | ||
|
|
||
| 从高到低: | ||
|
|
||
| 1. **租户配置的 `rag_response`/`naive_rag_response`**(完全自定义) | ||
| 2. **环境变量 `LIGHTRAG_RAG_RESPONSE_PROMPT`/`LIGHTRAG_NAIVE_RAG_RESPONSE_PROMPT`** | ||
| 3. **`strict_grounding=true`**(使用增强版默认 prompt) | ||
| 4. **LightRAG 原生 prompt**(默认行为) | ||
|
|
||
| ## 增强版 Prompt 的关键指令 | ||
|
|
||
| 启用严格 Grounding 模式后,prompt 会包含以下关键指令: | ||
|
|
||
| ```markdown | ||
| ---Critical Grounding Rules (MUST FOLLOW)--- | ||
|
|
||
| ⚠️ **ABSOLUTE REQUIREMENT**: You must ONLY use information explicitly stated in the **Context**. | ||
|
|
||
| **Before generating any answer, you MUST evaluate:** | ||
| 1. Does the Context contain information that DIRECTLY answers the user's question? | ||
| 2. Is the information in the Context SUFFICIENT and RELEVANT to provide a complete answer? | ||
|
|
||
| **If the answer is NO to either question, you MUST respond with:** | ||
| > 抱歉,根据当前知识库中的内容,我无法找到与您问题直接相关的信息。请尝试: | ||
| > - 重新表述您的问题 | ||
| > - 提供更多上下文信息 | ||
| > - 确认相关文档是否已上传到知识库 | ||
|
|
||
| **DO NOT:** | ||
| - ❌ Make up or fabricate information not in the Context | ||
| - ❌ Use your general knowledge to fill gaps | ||
| - ❌ Provide speculative or assumed answers | ||
| - ❌ Say "based on my knowledge" or similar phrases | ||
| - ❌ Combine partial information to create misleading answers | ||
|
|
||
| **DO:** | ||
| - ✅ Explicitly state when information is not available | ||
| - ✅ Only cite facts that appear in the Context | ||
| - ✅ Be honest about the limitations of the provided information | ||
| ``` | ||
|
|
||
| ## 效果对比 | ||
|
|
||
| ### 未启用严格 Grounding 模式 | ||
|
|
||
| 用户问题:`公司的年度收入是多少?` | ||
|
|
||
| (假设知识库中没有收入数据) | ||
|
|
||
| AI 可能回答: | ||
| > 根据相关文档,该公司是一家成熟的企业...虽然具体年度收入数据未在文档中明确提及,但从其业务规模来看,估计年收入应该在... | ||
|
|
||
| ### 启用严格 Grounding 模式 | ||
|
|
||
| 同样的问题,AI 会回答: | ||
| > 抱歉,根据当前知识库中的内容,我无法找到与您问题直接相关的信息。请尝试: | ||
| > - 重新表述您的问题 | ||
| > - 提供更多上下文信息 | ||
| > - 确认相关文档是否已上传到知识库 | ||
|
|
||
| ## 刷新配置 | ||
|
|
||
| 修改配置后,需要刷新租户实例缓存: | ||
|
|
||
| ```bash | ||
| # 刷新特定租户 | ||
| curl -X POST "http://localhost:8000/tenants/your_tenant/config/refresh" | ||
|
|
||
| # 或重启服务(全局生效) | ||
| docker compose restart rag-api | ||
| ``` | ||
|
|
||
| ## 相关配置 | ||
|
|
||
| | 配置项 | 类型 | 描述 | | ||
| |-------|------|------| | ||
| | `LIGHTRAG_STRICT_GROUNDING` | 环境变量 | 全局启用严格 Grounding 模式 | | ||
| | `strict_grounding` | 租户配置 | 租户级启用严格 Grounding 模式 | | ||
| | `rag_response` | 租户配置 | 完全自定义 KG 模式响应 prompt | | ||
| | `naive_rag_response` | 租户配置 | 完全自定义 naive 模式响应 prompt | | ||
|
|
||
| ## 注意事项 | ||
|
|
||
| 1. **语言适配**:当前默认拒绝回答消息是中文,如需英文或其他语言,请使用完全自定义 prompt | ||
| 2. **性能影响**:严格 Grounding 模式不会影响性能,仅修改 prompt 内容 | ||
| 3. **兼容性**:此功能与所有查询模式(naive、local、global、hybrid、mix)兼容 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
[nitpick] Documentation inconsistency: The documentation states the default behavior is
falsewhen strict grounding is not set (line 263: "默认 false,不改变 LightRAG 原有行为"), and line 264 shows the example asLIGHTRAG_STRICT_GROUNDING=true.However, looking at the code in prompt_manager.py, when
use_strict_groundingisFalse, the code doesn't set any prompt at all (lines 285-287, 298-300), which means it will use whatever LightRAG's default prompt is. This is correct behavior, but the env.example should make it clearer that:Consider updating line 264 to show the disabled state as the example:
# LIGHTRAG_STRICT_GROUNDING=true # Uncomment to enableOr add a comment clarifying the default:
# 默认:注释掉或设置为 false(使用 LightRAG 原生行为) LIGHTRAG_STRICT_GROUNDING=true