feat: support multimodal image input - #9
Merged
Merged
Conversation
ruanzhengxin-zhuxing
force-pushed
the
feat_multi_modal
branch
2 times, most recently
from
May 18, 2026 05:56
0f49f00 to
f13764e
Compare
binxin-wbx
reviewed
May 18, 2026
binxin-wbx
reviewed
May 18, 2026
ruanzhengxin-zhuxing
force-pushed
the
feat_multi_modal
branch
5 times, most recently
from
May 18, 2026 08:58
04528e8 to
72611f5
Compare
Collaborator
|
LGTM |
- Add image paste support in the REPL: Ctrl+V inserts `[Image #N]` placeholders that are assembled into a multimodal message at submit time, with text and multiple images interleaved by position. - Add per-model multimodal capability resolution (settings.yml override → built-in registry → openapi_compatible auto-detect via `/v1/models` `architecture.input_modalities` → default off), with a disk cache at `~/.iac-code/.multimodal-cache.yml`. - Add image utilities: clipboard readers (macOS osascript, Wayland `wl-paste`, X11 `xclip`, Windows PowerShell) with 2s timeouts and Wayland→X11 fallback, format detection, Pillow-based resizing to the 5 MB Base64 budget, session-scoped cache under `~/.iac-code/image-cache/<session>/`, and background cleanup of old session directories. - Wire image blocks into Anthropic and OpenAI provider message serialization; extend agent message types accordingly. - Show a capability-gated hint when the active model does not support images so placeholders are not inserted. - Update README with usage, supported clipboard sources, settings.yml schema, and auto-detect limitations; refresh translations. - Add tests covering clipboard adapters, image processing/resizing, capability detection, provider image-block serialization, REPL bracketed-paste handling, and the capability gate. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
ruanzhengxin-zhuxing
force-pushed
the
feat_multi_modal
branch
from
May 18, 2026 10:36
72611f5 to
753570f
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
[Image #N]placeholders that are assembled into a multimodal message at submit time, with text and multiple images interleaved by position./v1/modelsarchitecture.input_modalities→ default off), with a disk cache at~/.iac-code/.multimodal-cache.yml.wl-paste, X11xclip, Windows PowerShell) with 2s timeouts and Wayland→X11 fallback, format detection, Pillow-based resizing to the 5 MB Base64 budget, session-scoped cache under~/.iac-code/image-cache/<session>/, and background cleanup of old session directories.