feat(models): support provider-owned image models - #164
Conversation
MagicCube
left a comment
There was a problem hiding this comment.
Found two issues with image protocol handling. The 33 focused tests I ran passed, but do not cover these cases.
| return_base64: true, | ||
| extra_body: { response_format: "b64_json" }, | ||
| } | ||
| : { response_format: "b64_json" }), |
There was a problem hiding this comment.
[P1] Build a compatible payload for standard OpenAI Images
The new openai-images mode still sends metadata.size verbatim, while configuration and runtime validation only permit 1K/2K/3K/4K. Standard OpenAI Images requires pixel dimensions such as 1024x1024. This branch also always sends response_format, which GPT Image models do not support. Consequently, configuring a custom provider against the standard OpenAI endpoint cannot generate an image with the available settings. Please support protocol/model-appropriate sizes and output parameters, and add coverage for the standard mode alongside the extra-body fixture. Official parameter definitions.
There was a problem hiding this comment.
Fixed in 9a93982. Standard OpenAI Images now uses the official pixel-size options, maps the legacy 1K value to 1024x1024, rejects ambiguous larger presets before fetch, and only sends response_format for DALL-E models (GPT Image omits it). The settings editor and generate_image contract now expose protocol-compatible sizes. Added focused GPT Image, DALL-E, and invalid-preset coverage.
| ] | ||
| : []; | ||
| return { | ||
| ...(api !== "ark-images" ? { api } : {}), |
There was a problem hiding this comment.
[P2] Preserve an explicit Ark protocol on custom providers
Normalization unconditionally drops api: "ark-images", but a custom provider defaults to openai-images when api is absent. I reproduced this by adding a custom provider, saving an image configuration with api: "ark-images" through updateProvider, and constructing a new ModelManager with the same settings directory: the API changes from ark-images to undefined. After restart, generation therefore uses a different payload and omits Ark's watermark/stream fields. Please only omit the API when it equals defaultApi, or preserve the normalized API explicitly.
There was a problem hiding this comment.
Fixed in 9a93982. Image protocol normalization now omits api only when it equals the owning provider default, so a custom provider explicitly configured with ark-images survives persistence and reload. Added a restart regression test and exposed Ark Images in the custom-provider image API selector.
|
Pushed follow-up fixes in 814298d:
Verification: 72/72 top-level focused tests, including an isolated suite of 8/8 mounted React regressions; changed-file lint/typecheck, Prettier, and desktop/web production builds passed. The PR description records build warnings and the live-provider/native-CEF checks that were not rerun. |
|
Resolved the current Resolution details:
Local verification:
GitHub now reports the PR as mergeable; checks may still need to settle. |
Summary
extra_bodycompatibility modesgenerate_imagebehaviorReview Fixes
response_formatby default for GPT Image; canonical DALL-E models explicitly request base64.ark-imagesprotocol on custom providers through save/reload.responseFormat: "b64_json"setting for gateway aliases, wired through the editor, persistence, and request payload. No image-URL download path is introduced.1Kto1024x1024only for standard OpenAI Images, across the settings editor, inventory validation, and thread tools. Editing a model no longer replaces its size set with all options or resets its default toauto. Ambiguous legacy sizes require explicit correction; Ark and extra-body presets remain unchanged.1792x1024option while DALL-E/GPT Image 1 restrictions remain enforced.main(f2bb156) and preserve its desktop internationalization. New image settings and provider selection labels are available in English and Chinese.Latest integration:
afa506bcbb9613cd0935f1cf958297d626d92a09.Verification
git diff --check: passed.trae/evaluation assets or credential matchesThe prior live Agnes smoke at
9a93982generated a 1K PNG. Live OpenAI/Agnes requests and native CEF interaction were not rerun for this integration commit; provider behavior is covered with injected responses and mounted React tests.No Agnes-specific identifiers or endpoints are hard-coded in the implementation. Evaluation assets and credentials are excluded from this PR.