fix: Improve model compatibility in parsing and logging#7
fix: Improve model compatibility in parsing and logging#7abinashkarki wants to merge 3 commits intoalexzhang13:mainfrom
Conversation
|
So I don't enable The None call was handled in a separate PR, thanks though! I can merge this if you revert the above 2, "Handle special tokens around FINAL statements (parsing.py)" is fine. I will think on it though if it's necessary. Thanks! |
Improves compatibility with models that wrap FINAL/FINAL_VAR in special tokens like <|begin_of_box|>FINAL(...)<|end_of_box|>.
6b1416a to
6a38945
Compare
|
Thanks for the feedback! I've reverted the python code block tag support and the None model handling (didn't realize it was already fixed in #5). The updated PR now only includes the special token handling for FINAL statements, which helps with models that wrap output in tokens like <|begin_of_box|>FINAL(...)<|end_of_box|>. |
|
@abinashkarki Returning to this, the only problem now is if the model is thinking / doing some kind of CoT and is saying "I will then do FINAL(...)...". We don't want to accept these cases -- I think we should just be stricter about what exactly we accept, e.g. maybe just some trivial special tokens in case this is something people are observing. |
- Only accept FINAL at start of line (with optional whitespace) - Or immediately after special tokens like <|begin_of_box|> - Rejects mid-sentence usage like 'I will then do FINAL(...)' - Preserves GLM-4 compatibility which wraps FINAL in special tokens
|
Thanks for the feedback @alexzhang13 Updated the regex to be stricter. Why this matters: What changed: FINAL is now only accepted at:
This rejects CoT false positives like |
These changes improve compatibility with various LLM backends that use different output formats (e.g., Gemini, some OpenRouter models).