Give Agents access to Terminal output

[maskdotdev]

sorry this is so close to the 1k limit outline in the contributing.md., some of the code here could be reused by the diff pr at #1003 (could make smaller if I remove the timeline and composer pills, but I think those provide value to the end user)

What Changed

Added terminal selection as chat context, surfaced pending terminal-context chips in the composer, and rendered sent terminal context as compact metadata in the timeline.

Why

resolves #26

UI Changes

Composer now shows selected terminal snippets as removable chips, the terminal drawer adds an “Add to chat” action for selections, and sent messages show a compact terminal-context indicator with tooltip preview.

Checklist

• This PR is small and focused
• I explained what changed and why
• I included before/after screenshots for any UI changes
• I included a video for animation/interaction changes

t3code-terminal-context-optimized.mp4

Note

Give agents access to terminal output by adding inline terminal context to the composer

• Adds a terminal text selection flow: selecting text in the embedded terminal and clicking 'Add to chat' captures the selection as a TerminalContextDraft and inserts an inline placeholder chip into the composer prompt.
• Extends the Lexical editor in ComposerPromptEditor.tsx with a ComposerTerminalContextNode decorator node that renders an interactive chip; backspacing over it removes the context from the store.
• Adds deriveComposerSendState and buildExpiredTerminalContextToastCopy in ChatView.logic.ts to block sending when only expired (empty-text) contexts are present, and to show warning toasts when expired contexts are omitted from a send.
• Appends terminal context blocks to the outgoing message text via appendTerminalContextsToPrompt, materializing inline labels and a structured <terminal_context> block.
• Persists terminal context metadata (without snapshot text) in composerDraftStore.ts; draft hydration restores metadata with blank in-memory text snapshots.
• Renders terminal context chips inline in sent user messages in MessagesTimeline.tsx, with tooltips and correct copy text.
• Risk: the send button disabled state and slash-command parsing now depend on composerSendState.hasSendableContent rather than simple text/image checks, which changes behavior whenever terminal contexts are present.

^{Macroscope summarized dc8bf63.}

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: 30806f51-974e-4573-92e0-298bd5493397

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

📝 Coding Plan

• Generate coding plan for human review comments

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[juliusmarminge]

this looks amazing - definitely will try and get this merged soon. thank you! don't worry about the diff limit for this one

one nit: i think this indicator is a bit small and discrete, how would it look to keep the pill-like thing on top of the message similar to the composer input (or even inline like file taggings?)

another "feature" I originally wanted from this is the ability to tag a terminal using look at the logs in @terminal-1 and fix the build issue or whatever (autocompletes similar to the workspace entries search) and we'd transform that in the user message to include a reference to the terminal log file (in ~/.t3/userdata/logs/terminals/.log` that the agent can go look at

[maskdotdev]

I know you mentiond to not worry about the diff size on this one, but would you prefer if I create another issue and separate PR for the @terminal feature, or do it on this PR as well?

[juliusmarminge]

I know you mentiond to not worry about the diff size on this one, but would you prefer if I create another issue and separate PR for the @terminal feature, or do it on this PR as well?

i don't mind either. whatever is easiest!

[maskdotdev]

this looks amazing - definitely will try and get this merged soon. thank you! don't worry about the diff limit for this one

one nit: i think this indicator is a bit small and discrete, how would it look to keep the pill-like thing on top of the message similar to the composer input (or even inline like file taggings?)

another "feature" I originally wanted from this is the ability to tag a terminal using look at the logs in @terminal-1 and fix the build issue or whatever (autocompletes similar to the workspace entries search) and we'd transform that in the user message to include a reference to the terminal log file (in ~/.t3/userdata/logs/terminals/.log` that the agent can go look at

I'm leaning towards the inlined ones, possibly the basic one, which one do you prefer? Could be a combination of them too.

[juliusmarminge]

inline pills i think looks the best - will also match the existing file pills the best

[maskdotdev]

Style now reflects inline in a similar fashion to file mentions

Created a separate issue for @mention terminals feature at #1042

Will create a separate PR for that one 🫡

[juliusmarminge]

mind attaching a new recording of the current state?

[maskdotdev]

of course! here you go

t3code-term-context-inline.mp4

[brrock]

Nice PR, just make sure to not spam julius with reviews as it is the weekend!

[maskdotdev]

I think I found a better way to enable terminal/s context, that's not doing manual selection... I'll ping you tomorrow Julius, or feel free to reach out.

t3-code-term-context.mov

[juliusmarminge]

I think I found a better way to enable terminal/s context, that's not doing manual selection... I'll ping you tomorrow Julius, or feel free to reach out.

t3-code-term-context.mov

is this adjusting the system prompt to include the path to the terminals? or how do you inject the context?

[maskdotdev]

I think I found a better way to enable terminal/s context, that's not doing manual selection... I'll ping you tomorrow Julius, or feel free to reach out.
t3-code-term-context.mov

is this adjusting the system prompt to include the path to the terminals? or how do you inject the context?

I used tools instead, I gave these terminal tools to the app server via dynamicTools, OpenAI Dynamic Tools. We dont pass the terminal output in the prompt up front. If the llm decides it needs terminal context (didnt want to bloat the context window), it calls eitherlist_thread_terminals or read_thread_terminal. The server receives that tool call and automatically attaches the current thread threadId, we then asks the local TerminalManager for the matching terminal rendered contents, then the server sends the result back to Codex as the tool response.

LMK if you need more details... one thing I dont like about this approach is the architecture for tools, I think it'd be worth having like an internal tool registry, but for now it just a terminal tools file... also, I didnt push on this PR, because I was not sure what you would prefer, and I think this new approach may require some discussion around tools architecture.

[juliusmarminge]

what's the effort required to go from the current explicit-tagging method to the tool based one? and is line-based context inclusion (the original PR goal) still supported?

[maskdotdev]

is line-based context inclusion (the original PR goal) still supported?

Line based context inclusion is currently the only feature supported here in this PR. This would be unaffected even if we do explicit tags or auto term context.

what's the effort required to go from the current explicit-tagging method to the tool based one?
These are kinda different, or at least the approach I took is different. explicit tagging, is mostly UI work with terminal log path reading, although those logs need to be cleaned up of terminal escape codes, otherwise its wasted context (imo)...

While the tool based approach we do 99% of the work in the server (still do cleanup of terminal escape code), and we either add a new lib like xterm/headless, or grabbing the render output from the rendered terminal dom.

both of these are about a couple 100s loc, implementations

hopefully this answers your questions.

Note: Tool based approach is currently on a fully separate local branch

[juliusmarminge]

@macroscope-app review this

[juliusmarminge]

LGTM.

what do you think of having both taggin and tool based? tagging feels more discoverable for people?

having tagging would also be good if we add agent providers which doesn't support custom tools

[maskdotdev]

LGTM.

what do you think of having both taggin and tool based? tagging feels more discoverable for people?

having tagging would also be good if we add agent providers which doesn't support custom tools

Sounds good I can both on the same PR, ill be setting that up then.