-
Notifications
You must be signed in to change notification settings - Fork 30.4k
Closed
Labels
Description
System Info
In transformers/src/transformers/utils/chat_template_utils.py@render_jinja_template()
, if continue_final_message
is set to True
, then the content of the final message will be used to split the rendered_chat
string.
final_msg_loc = rendered_chat.rindex(final_message.strip())
...
rendered_chat = rendered_chat[: final_msg_loc + len(final_message.strip())]
So if the content of the final message is empty or matches some substrings of the chat template, there will be issues.
Who can help?
Information
- The official example scripts
- My own modified scripts
Tasks
- An officially supported task in the
examples
folder (such as GLUE/SQuAD, ...) - My own task or dataset (give details below)
Reproduction
from transformers import AutoTokenizer
m = [
{'role': 'user', 'content': 'user-input'},
{'role': 'assistant', 'content': 'end'}
]
tokenizer = AutoTokenizer.from_pretrained("Qwen3-0.6B")
print (tokenizer.apply_chat_template(m, continue_final_message=True, tokenize=False))
will output
<|im_start|>user
user-input<|im_end|>
<|im_start|>assistant
<think>
</think>
end<|im_end
Expected behavior
Expected output
<|im_start|>user
user-input<|im_end|>
<|im_start|>assistant
<think>
</think>
end