🐞 Bug Report: OpenRouter AI

[cvladan]

Please confirm the following:

• Be sure to check out FAQ
• I have carefully read the README
• I have searched through the issues page but couldn't find a solution.(Including issue that has been closed)
• Easydict has been upgraded to the latest version
• I understand and agree to the above, and understand that the project maintainer has limited energy, issues that do not follow the rules may be ignored or closed directly

Problem description

It is not possible to set the OpenRouter AI as a custom provider, I don't know why.

Is the issue consistently reproducible?

Reproducible

Steps to Reproduce

Explained

Screenshot

I did

Runtime log(zip file)

none

Expected result

To work

Proposed solution (optional)

Fix

Device type and OS version

Apple M1 Pro

Are you willing to submit a PR to fix this issue?

• I'm willing to submit a PR to fix this issue

[github-actions]

Hello cvladan, Thank you for your first issue contribution 🎉

[tisfeng]

Because OpenRouter's stream response has some special data that is different from the standard OpenAI stream API, Easydict parses the data with an error.

We are using a third-party OpenAI library to parse the data, which is a bit more strict about the data format.

If you can, try using another OpenAI standard API service.

[cvladan]

"another OpenAI standard API service"
... I am not aware of any

[cvladan]

Ah, you mean - you won't make it work with OpenRouter?
Ok. Thanks

[tisfeng]

It may not be that easy to fix this, so if someone can come PR I'd be happy to take it.

GitHub and Groq provide standard OpenAI APIs that work well with Easydict, and they both have free credits you can try.

[tisfeng]

Because OpenRouter's streaming response includes comment which causes OpenAI to fail to parse the data correctly, I've just fixed it and submitted a PR upstream MacPaw/OpenAI#243 .

We've fixed this issue, and the next version of Easydict will support OpenRouter.

: OPENROUTER PROCESSING

: OPENROUTER PROCESSING

data: {"id":"gen-1737464219-EoV242VsxqQk6IZCKHkC","provider":"SambaNova","model":"meta-llama/llama-3.1-405b-instruct","object":"chat.completion.chunk","created":1737464219,"choices":[{"index":0,"delta":{"role":"assistant","content":""},"finish_reason":null,"native_finish_reason":null,"logprobs":null}],"system_fingerprint":"fastcoe"}

data: {"id":"gen-1737464219-EoV242VsxqQk6IZCKHkC","provider":"SambaNova","model":"meta-llama/llama-3.1-405b-instruct","object":"chat.completion.chunk","created":1737464219,"choices":[{"index":0,"delta":{"role":"assistant","content":""},"finish_reason":null,"native_finish_reason":null,"logprobs":null}],"system_fingerprint":"fastcoe"}

data: {"id":"gen-1737464219-EoV242VsxqQk6IZCKHkC","provider":"SambaNova","model":"meta-llama/llama-3.1-405b-instruct","object":"chat.completion.chunk","created":1737464219,"choices":[{"index":0,"delta":{"role":"assistant","content":""},"finish_reason":null,"native_finish_reason":null,"logprobs":null}],"system_fingerprint":"fastcoe"}

data: {"id":"gen-1737464219-EoV242VsxqQk6IZCKHkC","provider":"SambaNova","model":"meta-llama/llama-3.1-405b-instruct","object":"chat.completion.chunk","created":1737464219,"choices":[{"index":0,"delta":{"role":"assistant","content":"How's it going? Is there something I can help you with "},"finish_reason":null,"native_finish_reason":null,"logprobs":null}],"system_fingerprint":"fastcoe"}

data: {"id":"gen-1737464219-EoV242VsxqQk6IZCKHkC","provider":"SambaNova","model":"meta-llama/llama-3.1-405b-instruct","object":"chat.completion.chunk","created":1737464219,"choices":[{"index":0,"delta":{"role":"assistant","content":"or would you like "},"finish_reason":null,"native_finish_reason":null,"logprobs":null}],"system_fingerprint":"fastcoe"}

data: {"id":"gen-1737464219-EoV242VsxqQk6IZCKHkC","provider":"SambaNova","model":"meta-llama/llama-3.1-405b-instruct","object":"chat.completion.chunk","created":1737464219,"choices":[{"index":0,"delta":{"role":"assistant","content":"to chat?"},"finish_reason":null,"native_finish_reason":null,"logprobs":null}],"system_fingerprint":"fastcoe"}

data: {"id":"gen-1737464219-EoV242VsxqQk6IZCKHkC","provider":"SambaNova","model":"meta-llama/llama-3.1-405b-instruct","object":"chat.completion.chunk","created":1737464219,"choices":[{"index":0,"delta":{"role":"assistant","content":""},"finish_reason":null,"native_finish_reason":null,"logprobs":null}],"system_fingerprint":"fastcoe"}

data: {"id":"gen-1737464219-EoV242VsxqQk6IZCKHkC","provider":"SambaNova","model":"meta-llama/llama-3.1-405b-instruct","object":"chat.completion.chunk","created":1737464219,"choices":[{"index":0,"delta":{"role":"assistant","content":""},"finish_reason":"stop","native_finish_reason":"stop","logprobs":null}],"system_fingerprint":"fastcoe"}

: OPENROUTER PROCESSING

data: {"id":"gen-1737464219-EoV242VsxqQk6IZCKHkC","provider":"SambaNova","model":"meta-llama/llama-3.1-405b-instruct","object":"chat.completion.chunk","created":1737464219,"choices":[{"index":0,"delta":{"role":"assistant","content":""},"finish_reason":null,"native_finish_reason":null,"logprobs":null}],"usage":{"prompt_tokens":36,"completion_tokens":20,"total_tokens":56}}

data: [DONE]

[tisfeng]

2.11.1 version has fixed this issue.