Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] 对于固定到预设提示词的消息,出现如下问题:①发送时遗漏固定的系统提示词有所遗漏;②无法自动删除固定的系统提示词消息,导致重复发送。 #5485

Open
jean-raffe opened this issue Sep 21, 2024 · 5 comments
Labels
bug Something isn't working help wanted Extra attention is needed

Comments

@jean-raffe
Copy link

📦 部署方式

Vercel

📌 软件版本

v2.15.2

💻 系统环境

Windows

📌 系统版本

Windows 11 Home

🌐 浏览器

Chrome

📌 浏览器版本

128.0.6613.138 (Stable)

🐛 问题描述

  1. 系统提示词遗漏:对于多条系统提示词,在实际请求中发现只保留了最后两条。
  2. 系统提示词固定至预设提示词后,不会自动删除,并且会出现请求中,导致没有意义的token占用。

📷 复现步骤

1. 系统提示词遗漏

对于多条系统提示词,在实际请求中发现只保留了最后两条,示例如下:

【编辑消息记录界面】

system : 接下来我会要求你分析文章并回答我提出的问题。
system : 请你根据文章用中文回答我的问题。
system : 回答问题时,应当尽可能以英文原文的文本作为依据。
system : 回答问题时,应当尽可能给出英文原文的出处。

【请求 - messages 部分】

{'role': 'system', 'content': '回答问题时,应当尽可能以英文原文的文本作为依据。'},
{'role': 'system', 'content': '回答问题时,应当尽可能给出英文原文的出处。'},
...

2. 系统提示词固定后,存在没有意义的token占用

系统提示词固定至预设提示词后,不会自动删除,并且会出现请求中,示例:

【聊天界面 - 固定前】

system : 接下来我会要求你分析文章并回答我提出的问题。
system : 请你根据文章用中文回答我的问题。
system : 回答问题时,应当尽可能以英文原文的文本作为依据。
system : 回答问题时,应当尽可能给出英文原文的出处。

【请求 - messages 部分】

{'role': 'system', 'content': '回答问题时,应当尽可能以英文原文的文本作为依据。'},
{'role': 'system', 'content': '回答问题时,应当尽可能给出英文原文的出处。'},
...


【聊天界面 - 固定后】

system : 接下来我会要求你分析文章并回答我提出的问题。
system : 请你根据文章用中文回答我的问题。
system : 回答问题时,应当尽可能以英文原文的文本作为依据。
system : 回答问题时,应当尽可能给出英文原文的出处。
system : 接下来我会要求你分析文章并回答我提出的问题。
system : 请你根据文章用中文回答我的问题。
system : 回答问题时,应当尽可能以英文原文的文本作为依据。
system : 回答问题时,应当尽可能给出英文原文的出处。

【请求 - messages 部分】

{'role': 'system', 'content': '接下来我会要求你分析文章并回答我提出的问题。'},
{'role': 'system', 'content': '请你根据文章用中文回答我的问题。'},
{'role': 'system', 'content': '回答问题时,应当尽可能以英文原文的文本作为依据。'},
{'role': 'system', 'content': '回答问题时,应当尽可能给出英文原文的出处。'},
{'role': 'system', 'content': '回答问题时,应当尽可能以英文原文的文本作为依据。'},
{'role': 'system', 'content': '回答问题时,应当尽可能给出英文原文的出处。'},
...

🚦 期望结果

情形一、避免出现漏发情况:不论有几条系统提示词,均保证在请求中发送。
情形二、在固定系统提示词后能自动从聊天中删除,避免重复。

📝 补充信息

理由

系统提示词在实际使用中可能会被分次、不断地补充、完善,但是没有在上下文与系统提示词中反复出现的必要。
当系统提示词数目较多时,Next Chat 不会发送所有的系统提示词,尝试数次请求之后发现保留的是最末两条系统提示词。这会使得系统提示词在补充之后不够理想,尤其是当前面的提示词段落较长,不方便寻找需要修改的位置的时候。
如果发送所有的系统提示词被认为是一个可选而非必要的手段,建议实现在固定后自动删除固定的系统提示词,更充分地利用有限地上下文token,另外也能简化操作,避免手动删除固定后的系统提示词,尤其是当系统提示词较长或较多时。

@jean-raffe jean-raffe added the bug Something isn't working label Sep 21, 2024
@Issues-translate-bot
Copy link

Bot detected the issue body's language is not English, translate it automatically.


Title: [Bug] For messages fixed to preset prompt words, the following problems occur: ① Fixed system prompt words are omitted when sending; ② Fixed system prompt words cannot be automatically deleted, resulting in repeated sending .

📦 Deployment method

Vercel

📌 Software version

v2.15.2

💻 System environment

Windows

📌 System version

Windows 11 Home

🌐 Browser

Chrome

📌 Browser version

128.0.6613.138 (Stable)

🐛 Problem description

  1. Missing system prompt words: For multiple system prompt words, it was found that only the last two were retained in the actual request.
  2. After the system prompt word is fixed to the default prompt word, it will not be automatically deleted and will appear in the request, resulting in meaningless token occupation.

📷 Steps to reproduce

1. System prompt word missing

For multiple system prompt words, it was found in the actual request that only the last two were retained. The example is as follows:

[Edit message record interface]

system : Next I will ask you to analyze the article and answer the questions I pose.
system: Please answer my question in Chinese based on the article.
system: When answering questions, you should use the original English text as a basis as much as possible.
system: When answering questions, the source of the original English text should be given as much as possible.

[Request - messages part]

{'role': 'system', 'content': 'When answering questions, you should use the original English text as a basis as much as possible. '},
{'role': 'system', 'content': 'When answering questions, the source of the original English text should be given as much as possible. '},
...

2. After the system prompt word is fixed, there are meaningless token occupations.

After the system prompt word is fixed to the default prompt word, it will not be automatically deleted and will appear in the request. Example:

[Chat Interface - Before Fixed]

system : Next I will ask you to analyze the article and answer the questions I pose.
system: Please answer my question in Chinese based on the article.
system: When answering questions, you should use the original English text as a basis as much as possible.
system: When answering questions, the source of the original English text should be given as much as possible.

[Request - messages part]

{'role': 'system', 'content': 'When answering questions, you should use the original English text as a basis as much as possible. '},
{'role': 'system', 'content': 'When answering questions, the source of the original English text should be given as much as possible. '},
...


[Chat Interface - After Fixed]

system : Next I will ask you to analyze the article and answer the questions I pose.
system: Please answer my question in Chinese based on the article.
system: When answering questions, you should use the original English text as a basis as much as possible.
system: When answering questions, the source of the original English text should be given as much as possible.
system : Next I will ask you to analyze the article and answer the questions I pose.
system: Please answer my question in Chinese based on the article.
system: When answering questions, you should use the original English text as a basis as much as possible.
system: When answering questions, the source of the original English text should be given as much as possible.

[Request - messages part]

{'role': 'system', 'content': 'Next I will ask you to analyze the article and answer the questions I raised. '},
{'role': 'system', 'content': 'Please answer my question in Chinese based on the article. '},
{'role': 'system', 'content': 'When answering questions, you should use the original English text as a basis as much as possible. '},
{'role': 'system', 'content': 'When answering questions, the source of the original English text should be given as much as possible. '},
{'role': 'system', 'content': 'When answering questions, you should use the original English text as a basis as much as possible. '},
{'role': 'system', 'content': 'When answering questions, the source of the original English text should be given as much as possible. '},
...

🚦 Expected results

Scenario 1: Avoid missed transmissions: No matter how many system prompt words there are, they are guaranteed to be sent in the request.
Scenario 2: After fixing the system prompt word, it can be automatically deleted from the chat to avoid duplication.

📝 Supplementary information

Reason

System prompt words may be supplemented and improved step by step in actual use, but there is no need to appear repeatedly in the context and system prompt words.
When the number of system prompt words is large, Next Chat will not send all system prompt words. After trying several requests, it is found that the last two system prompt words are retained. This will make the system prompt words less than ideal after being supplemented, especially when the previous prompt word paragraph is long and it is inconvenient to find the location that needs to be modified.
If sending all system prompt words is considered an optional but not necessary means, it is recommended to implement automatic deletion of fixed system prompt words after fixing to make full use of limited context tokens and to simplify operations. , avoid manually deleting fixed system prompt words, especially when the system prompt words are long or numerous.

@H0llyW00dzZ
Copy link
Contributor

📦 部署方式

Vercel

📌 软件版本

v2.15.2

💻 系统环境

Windows

📌 系统版本

Windows 11 Home

🌐 浏览器

Chrome

📌 浏览器版本

128.0.6613.138 (Stable)

🐛 问题描述

  1. 系统提示词遗漏:对于多条系统提示词,在实际请求中发现只保留了最后两条。
  2. 系统提示词固定至预设提示词后,不会自动删除,并且会出现请求中,导致没有意义的token占用。

📷 复现步骤

1. 系统提示词遗漏

对于多条系统提示词,在实际请求中发现只保留了最后两条,示例如下:

【编辑消息记录界面】

system : 接下来我会要求你分析文章并回答我提出的问题。 system : 请你根据文章用中文回答我的问题。 system : 回答问题时,应当尽可能以英文原文的文本作为依据。 system : 回答问题时,应当尽可能给出英文原文的出处。

【请求 - messages 部分】

{'role': 'system', 'content': '回答问题时,应当尽可能以英文原文的文本作为依据。'}, {'role': 'system', 'content': '回答问题时,应当尽可能给出英文原文的出处。'}, ...

2. 系统提示词固定后,存在没有意义的token占用

系统提示词固定至预设提示词后,不会自动删除,并且会出现请求中,示例:

【聊天界面 - 固定前】

system : 接下来我会要求你分析文章并回答我提出的问题。 system : 请你根据文章用中文回答我的问题。 system : 回答问题时,应当尽可能以英文原文的文本作为依据。 system : 回答问题时,应当尽可能给出英文原文的出处。

【请求 - messages 部分】

{'role': 'system', 'content': '回答问题时,应当尽可能以英文原文的文本作为依据。'}, {'role': 'system', 'content': '回答问题时,应当尽可能给出英文原文的出处。'}, ...

【聊天界面 - 固定后】

system : 接下来我会要求你分析文章并回答我提出的问题。 system : 请你根据文章用中文回答我的问题。 system : 回答问题时,应当尽可能以英文原文的文本作为依据。 system : 回答问题时,应当尽可能给出英文原文的出处。 system : 接下来我会要求你分析文章并回答我提出的问题。 system : 请你根据文章用中文回答我的问题。 system : 回答问题时,应当尽可能以英文原文的文本作为依据。 system : 回答问题时,应当尽可能给出英文原文的出处。

【请求 - messages 部分】

{'role': 'system', 'content': '接下来我会要求你分析文章并回答我提出的问题。'}, {'role': 'system', 'content': '请你根据文章用中文回答我的问题。'}, {'role': 'system', 'content': '回答问题时,应当尽可能以英文原文的文本作为依据。'}, {'role': 'system', 'content': '回答问题时,应当尽可能给出英文原文的出处。'}, {'role': 'system', 'content': '回答问题时,应当尽可能以英文原文的文本作为依据。'}, {'role': 'system', 'content': '回答问题时,应当尽可能给出英文原文的出处。'}, ...

🚦 期望结果

情形一、避免出现漏发情况:不论有几条系统提示词,均保证在请求中发送。 情形二、在固定系统提示词后能自动从聊天中删除,避免重复。

📝 补充信息

理由

系统提示词在实际使用中可能会被分次、不断地补充、完善,但是没有在上下文与系统提示词中反复出现的必要。 当系统提示词数目较多时,Next Chat 不会发送所有的系统提示词,尝试数次请求之后发现保留的是最末两条系统提示词。这会使得系统提示词在补充之后不够理想,尤其是当前面的提示词段落较长,不方便寻找需要修改的位置的时候。 如果发送所有的系统提示词被认为是一个可选而非必要的手段,建议实现在固定后自动删除固定的系统提示词,更充分地利用有限地上下文token,另外也能简化操作,避免手动删除固定后的系统提示词,尤其是当系统提示词较长或较多时。

Are you using OpenAI or another provider? Anyway, not all AI providers support system roles, such as OpenAI, which can have multiple system roles. For example, in Claude, there is only one system role, and it must be at the first (very top).

@Issues-translate-bot
Copy link

Bot detected the issue body's language is not English, translate it automatically.


📦 Deployment method

Vercel

📌 Software version

v2.15.2

💻 System environment

Windows

📌 System version

Windows 11 Home

🌐 Browser

Chrome

📌 Browser version

128.0.6613.138 (Stable)

🐛 Problem description

  1. Missing system prompt words: For multiple system prompt words, it was found that only the last two were retained in the actual request.
  2. After the system prompt word is fixed to the default prompt word, it will not be automatically deleted and will appear in the request, resulting in meaningless token occupation.

📷 Steps to reproduce

1. System prompt word missing

For multiple system prompt words, it was found that only the last two were retained in the actual request. The example is as follows:

[Edit message record interface]

system : Next I will ask you to analyze the article and answer my questions. system: Please answer my question in Chinese based on the article. system: When answering questions, you should use the original English text as a basis as much as possible. system: When answering questions, the source of the original English text should be given as much as possible.

[Request - messages part]

{'role': 'system', 'content': 'When answering questions, you should use the original English text as a basis as much as possible. '}, {'role': 'system', 'content': 'When answering questions, the source of the original English text should be given as much as possible. '}, ...

2. After the system prompt word is fixed, there are meaningless token occupations.

After the system prompt word is fixed to the default prompt word, it will not be automatically deleted and will appear in the request. Example:

[Chat Interface - Before Fixed]

system : Next I will ask you to analyze the article and answer my questions. system: Please answer my question in Chinese based on the article. system: When answering questions, you should use the original English text as a basis as much as possible. system: When answering questions, the source of the original English text should be given as much as possible.

[Request - messages part]

{'role': 'system', 'content': 'When answering questions, you should use the original English text as a basis as much as possible. '}, {'role': 'system', 'content': 'When answering questions, the source of the original English text should be given as much as possible. '}, ...

[Chat Interface - After Fixed]

system : Next I will ask you to analyze the article and answer my questions. system: Please answer my question in Chinese based on the article. system: When answering questions, you should use the original English text as a basis as much as possible. system: When answering questions, the source of the original English text should be given as much as possible. system : Next I will ask you to analyze the article and answer the questions I pose. system: Please answer my question in Chinese based on the article. system: When answering questions, you should use the original English text as a basis as much as possible. system: When answering questions, the source of the original English text should be given as much as possible.

[Request - messages part]

{'role': 'system', 'content': 'Next I will ask you to analyze the article and answer the questions I raised. '}, {'role': 'system', 'content': 'Please answer my question in Chinese based on the article. '}, {'role': 'system', 'content': 'When answering questions, you should use the original English text as a basis as much as possible. '}, {'role': 'system', 'content': 'When answering questions, the source of the original English text should be given as much as possible. '}, {'role': 'system', 'content': 'When answering questions, you should use the original English text as a basis as much as possible. '}, {'role': 'system', 'content': 'When answering questions, the source of the original English text should be given as much as possible. '}, ...

🚦 Expected results

Scenario 1: Avoid missed transmissions: No matter how many system prompt words there are, they are guaranteed to be sent in the request. Scenario 2: After fixing the system prompt word, it can be automatically deleted from the chat to avoid duplication.

📝 Supplementary information

Reason

System prompt words may be supplemented and improved in stages in actual use, but there is no need to appear repeatedly in the context and system prompt words. When the number of system prompt words is large, Next Chat will not send all system prompt words. After trying several requests, it is found that the last two system prompt words are retained. This will make the system prompt words less than ideal after being supplemented, especially when the previous prompt word paragraph is long and it is inconvenient to find the location that needs to be modified. If sending all system prompt words is considered an optional but not necessary means, it is recommended to implement automatic deletion of fixed system prompt words after fixing to make full use of limited context tokens and to simplify operations. , avoid manually deleting fixed system prompt words, especially when the system prompt words are long or numerous.

Are you using OpenAI or another provider? Anyway, not all AI providers support system roles, such as OpenAI, which can have multiple system roles. For example, in Claude, there is only one system role, and it must be at the first (very top).

@QAbot-zh
Copy link

我的建议是进入对话设置里根据自己的需要调整提示词,这样自由度最高,也不容易出错,靠默认的对话管理很难满足所有人对系统提示词的管理需求

@Issues-translate-bot
Copy link

Bot detected the issue body's language is not English, translate it automatically.


My suggestion is to enter the dialogue settings and adjust the prompt words according to your own needs. This has the highest degree of freedom and is less error-prone. It is difficult to rely on the default dialogue management to meet everyone's management needs for system prompt words.

@Dogtiti Dogtiti added the help wanted Extra attention is needed label Sep 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working help wanted Extra attention is needed
Projects
None yet
Development

No branches or pull requests

5 participants