Ollama and qwen models

[qubit0]

Here's my .env file:

# LLM API Configuration - pointing to local Ollama
LLM_API_KEY=ollama
LLM_BASE_URL=http://localhost:11434/api
LLM_MODEL_NAME=qwen3.5:0.8b

I get the following in backend/logs/

[2026-04-01 21:26:03] DEBUG [mirofish.api.generate_ontology:158] 模拟需求: What would happen if they can't deliver 50% of their promise in 100 days?...
[2026-04-01 21:26:03] INFO [mirofish.api.generate_ontology:177] 创建项目: proj_450fa2d55c88
[2026-04-01 21:26:03] INFO [mirofish.api.generate_ontology:212] 文本提取完成，共 44882 字符
[2026-04-01 21:26:03] INFO [mirofish.api.generate_ontology:215] 调用 LLM 生成本体定义...
[2026-04-01 21:26:03] DEBUG [mirofish.request.log_response:62] 响应: 500

[dosubot]

Hi @qubit0! I'm Dosu and I’m helping the MiroFish team.

This is a known Ollama compatibility issue that's come up before (e.g., issue #330). There are two problems with your setup:

1. Wrong base URL — MiroFish uses the OpenAI Python SDK, so it needs Ollama's OpenAI-compatible endpoint. Change /api to /v1:

LLM_BASE_URL=http://localhost:11434/v1

[1] [2]

2. response_format incompatibility — Even after fixing the URL, you'll likely still hit 500 errors because MiroFish's chat_json() method unconditionally sends response_format={"type": "json_object"} [3], which Ollama doesn't support by default — especially with smaller models like qwen3.5:0.8b [4].

To work around this, PR #186 added fallback retry logic that automatically retries without response_format when the first attempt fails. If you're running via Docker, rebuild with docker compose up --build to pick up this fix.

Summary of recommended .env changes:

LLM_API_KEY=ollama
LLM_BASE_URL=http://localhost:11434/v1
LLM_MODEL_NAME=qwen3.5:0.8b

Note that MiroFish is officially tested only with Alibaba's qwen-plus model [5], so smaller local models may have additional output quality issues even if the API calls succeed.

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}