Skip to content

Support Lemonade as a local AI provider (text, image, TTS, ASR) #506

@cosarah

Description

@cosarah

Background

Lemonade is a local AI server (optimized for AMD Ryzen AI NPU/GPU, also runs on CPU) that exposes a unified OpenAI-compatible HTTP API for chat, image generation, TTS, and ASR. Adding it as a first-class provider lets users run OpenMAIC fully locally on a single endpoint, similar to how Ollama is supported today — but with image/TTS/ASR coverage that Ollama does not provide.

Reference docs:

Lemonade endpoints relevant to OpenMAIC

Default base URL: http://localhost:13305/v1 (no API key required).

Capability Endpoint Status in Lemonade
Chat / text LLM POST /v1/chat/completions (streaming, tools, vision, temperature/top_p/top_k/max_tokens) available
Image generation POST /v1/images/generations (sd-cpp recipe; steps, cfg_scale, width, height configurable via /v1/load) available
Text-to-speech POST /v1/audio/speech available
Speech-to-text POST /v1/audio/transcriptions (whisper.cpp backend; currently wav + json only) partial
Model listing GET /v1/models available
Health / loaded models GET /v1/health available

OpenAI-compatible payloads, so most of OpenMAIC's existing transport layers can be reused.

Scope of this issue

Add a new built-in provider lemonade that covers all four modalities.

1. Text LLM provider

  • Add 'lemonade' to BuiltInProviderId in lib/types/provider.ts.
  • Register provider in lib/ai/providers.ts — mirror the ollama entry:
    • type: 'openai'
    • defaultBaseUrl: 'http://localhost:13305/v1'
    • requiresApiKey: false
    • icon /logos/lemonade.svg (mono logo set in MONO_LOGO_PROVIDERS if appropriate)
  • Update lib/server/provider-config.ts keyless providers set to include lemonade.
  • Models list: leave empty / dynamic (like Ollama) or seed with a few common Lemonade Server registry names (e.g. Qwen3-0.6B-GGUF, Llama-3.2-1B-Instruct-Hybrid, Qwen2.5-VL-7B-Instruct). Recommend dynamic enumeration via GET /v1/models.

2. Image generation provider

  • Add 'lemonade' to ImageProviderId in lib/media/types.ts.
  • Implement provider in lib/media/image-providers.ts calling POST /v1/images/generations with OpenAI-style payload.
  • Surface in app/api/generate/image/route.ts resolution path (resolveImageApiKey / resolveImageBaseUrl).

3. TTS provider

Per the playbook in lib/audio/tts-providers.ts header:

  • Add 'lemonade-tts' to TTSProviderId in lib/audio/types.ts.
  • Add config to lib/audio/constants.ts (no api key, default base URL http://localhost:13305/v1).
  • Implement generateLemonadeTTS(config, text) calling POST /v1/audio/speech.
  • Wire into generateTTS() switch.
  • Add i18n strings.

4. ASR provider

Per the playbook in lib/audio/asr-providers.ts:

  • Add 'lemonade-asr' to ASRProviderId.
  • Add config to lib/audio/constants.ts.
  • Implement transcribeLemonadeASR(config, audioBuffer) posting to POST /v1/audio/transcriptions as multipart form.
  • Note Lemonade limitation: only wav + json response format are supported today — convert input or document the constraint.
  • Wire into transcribeASR() switch.

Suggested implementation order

  1. Text LLM provider (smallest change, validates plumbing).
  2. Image generation.
  3. TTS.
  4. ASR (requires audio format handling).

Each step can ship as an independent PR.

Out of scope

  • Lemonade-specific lifecycle endpoints (/v1/pull, /v1/load, /v1/unload, /v1/health) — interesting for a future enhancement (auto model installation / pre-warm) but not required for parity with current providers.
  • Embeddings (/v1/embeddings) — track separately if/when OpenMAIC needs an embeddings abstraction.
  • Realtime ASR over WebSocket — separate issue.

Acceptance criteria

  • User can select Lemonade in Settings for chat, image, TTS, and ASR provider pickers.
  • Default base URL points at http://localhost:13305/v1; no API key required.
  • End-to-end: with a running local Lemonade Server, generating a classroom that uses chat + image + TTS + ASR succeeds without hitting any cloud provider.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions