Skip to content

Conversation

Columpio
Copy link

@Columpio Columpio commented Nov 27, 2024

What is done?

A produced command is extracted from markdown backticks in --shell mode

Why?

Because LLMs are usually taught to produce markdown output. So, even if you ask gently in a prompt, you get markdown backticks most of the time. You will get it especially often on small models, e.g., when running Ollama.

What I tested

  • If LLM generates no backticks, it's output is returned as is
  • LLM generates multiline command
  • LLM generates a command which itself contains backticks for some reason (e.g., you ask to grep -rnI "```")
  • LLM generates a surrounding comment text with backticks
  • Windows/Linux/Mac-style newlines

P.S.

Hope other Ollama guys like me will now be happy. Please like this PR if you are.

With this PR I can finally do DEFAULT_MODEL=ollama/qwen2.5-coder:3b-instruct-q6_K.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant