Give Agents access to Terminal output by maskdotdev · Pull Request #1032 · pingdotgg/t3code

maskdotdev · 2026-03-13T16:23:33Z

sorry this is so close to the 1k limit outline in the contributing.md., some of the code here could be reused by the diff pr at #1003 (could make smaller if I remove the timeline and composer pills, but I think those provide value to the end user)

What Changed

Added terminal selection as chat context, surfaced pending terminal-context chips in the composer, and rendered sent terminal context as compact metadata in the timeline.

Why

resolves #26

UI Changes

Composer now shows selected terminal snippets as removable chips, the terminal drawer adds an “Add to chat” action for selections, and sent messages show a compact terminal-context indicator with tooltip preview.

Checklist

This PR is small and focused
I explained what changed and why
I included before/after screenshots for any UI changes
I included a video for animation/interaction changes

t3code-terminal-context-optimized.mp4

Note

Give agents access to terminal output by attaching selected text as context in chat

Adds a floating 'Add to chat' button in TerminalViewport that appears when the user selects text; clicking it appends a <terminal_context> block to the composer draft via composerDraftStore.
Pending terminal contexts are shown as removable pill chips above the composer via a new ComposerPendingTerminalContexts component.
On send, the terminal context block is appended to the message text; user messages render compact inline labels (e.g. @terminal-1:3-7) with tooltips instead of raw block text.
The composer draft store persists and restores terminal contexts per thread, deduplicates identical selections, and clears them with other composer content.
Timeline height estimation and the empty-send guard are updated to account for terminal context blocks correctly.

^{Macroscope summarized 050a3bc.}

coderabbitai · 2026-03-13T16:23:42Z

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: d615eb2d-78b7-4a55-be1f-70a3d03f5d47

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests
Post copyable unit tests in a comment

📝 Coding Plan

Generate coding plan for human review comments

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

Tip

CodeRabbit can use OpenGrep to find security vulnerabilities and bugs across 17+ programming languages.

OpenGrep is compatible with Semgrep configurations. Add an opengrep.yml or semgrep.yml configuration file to your project to enable OpenGrep analysis.

apps/web/src/lib/terminalContext.ts

apps/web/src/components/ThreadTerminalDrawer.tsx

juliusmarminge

this looks amazing - definitely will try and get this merged soon. thank you! don't worry about the diff limit for this one

one nit: i think this indicator is a bit small and discrete, how would it look to keep the pill-like thing on top of the message similar to the composer input (or even inline like file taggings?)

another "feature" I originally wanted from this is the ability to tag a terminal using look at the logs in @terminal-1 and fix the build issue or whatever (autocompletes similar to the workspace entries search) and we'd transform that in the user message to include a reference to the terminal log file (in ~/.t3/userdata/logs/terminals/.log` that the agent can go look at

maskdotdev · 2026-03-13T16:47:03Z

I know you mentiond to not worry about the diff size on this one, but would you prefer if I create another issue and separate PR for the @terminal feature, or do it on this PR as well?

juliusmarminge · 2026-03-13T16:51:27Z

I know you mentiond to not worry about the diff size on this one, but would you prefer if I create another issue and separate PR for the @terminal feature, or do it on this PR as well?

i don't mind either. whatever is easiest!

maskdotdev · 2026-03-13T17:55:43Z

this looks amazing - definitely will try and get this merged soon. thank you! don't worry about the diff limit for this one

one nit: i think this indicator is a bit small and discrete, how would it look to keep the pill-like thing on top of the message similar to the composer input (or even inline like file taggings?)

another "feature" I originally wanted from this is the ability to tag a terminal using look at the logs in @terminal-1 and fix the build issue or whatever (autocompletes similar to the workspace entries search) and we'd transform that in the user message to include a reference to the terminal log file (in ~/.t3/userdata/logs/terminals/.log` that the agent can go look at

I'm leaning towards the inlined ones, possibly the basic one, which one do you prefer? Could be a combination of them too.

juliusmarminge · 2026-03-13T18:07:04Z

inline pills i think looks the best - will also match the existing file pills the best

maskdotdev · 2026-03-13T18:18:01Z

Style now reflects inline in a similar fashion to file mentions

Created a separate issue for @mention terminals feature at #1042

Will create a separate PR for that one 🫡

juliusmarminge · 2026-03-13T18:21:58Z

mind attaching a new recording of the current state?

maskdotdev · 2026-03-13T18:33:35Z

of course! here you go

t3code-term-context-inline.mp4

brrock · 2026-03-14T17:37:34Z

Nice PR, just make sure to not spam julius with reviews as it is the weekend!

maskdotdev · 2026-03-15T16:43:05Z

I think I found a better way to enable terminal/s context, that's not doing manual selection... I'll ping you tomorrow Julius, or feel free to reach out.

t3-code-term-context.mov

juliusmarminge · 2026-03-16T16:28:40Z

I think I found a better way to enable terminal/s context, that's not doing manual selection... I'll ping you tomorrow Julius, or feel free to reach out.

t3-code-term-context.mov

is this adjusting the system prompt to include the path to the terminals? or how do you inject the context?

maskdotdev · 2026-03-16T16:46:49Z

I think I found a better way to enable terminal/s context, that's not doing manual selection... I'll ping you tomorrow Julius, or feel free to reach out.
t3-code-term-context.mov

is this adjusting the system prompt to include the path to the terminals? or how do you inject the context?

I used tools instead, I gave these terminal tools to the app server via dynamicTools, OpenAI Dynamic Tools. We dont pass the terminal output in the prompt up front. If the llm decides it needs terminal context (didnt want to bloat the context window), it calls eitherlist_thread_terminals or read_thread_terminal. The server receives that tool call and automatically attaches the current thread threadId, we then asks the local TerminalManager for the matching terminal rendered contents, then the server sends the result back to Codex as the tool response.

LMK if you need more details... one thing I dont like about this approach is the architecture for tools, I think it'd be worth having like an internal tool registry, but for now it just a terminal tools file... also, I didnt push on this PR, because I was not sure what you would prefer, and I think this new approach may require some discussion around tools architecture.

juliusmarminge · 2026-03-16T16:57:26Z

what's the effort required to go from the current explicit-tagging method to the tool based one? and is line-based context inclusion (the original PR goal) still supported?

maskdotdev · 2026-03-16T17:10:27Z

is line-based context inclusion (the original PR goal) still supported?

Line based context inclusion is currently the only feature supported here in this PR. This would be unaffected even if we do explicit tags or auto term context.

what's the effort required to go from the current explicit-tagging method to the tool based one?
These are kinda different, or at least the approach I took is different. explicit tagging, is mostly UI work with terminal log path reading, although those logs need to be cleaned up of terminal escape codes, otherwise its wasted context (imo)...

While the tool based approach we do 99% of the work in the server (still do cleanup of terminal escape code), and we either add a new lib like xterm/headless, or grabbing the render output from the rendered terminal dom.

both of these are about a couple 100s loc, implementations

hopefully this answers your questions.

Note: Tool based approach is currently on a fully separate local branch

maskdotdev added 4 commits March 13, 2026 08:53

Add terminal context selections to chat

c875b6f

Fix terminal add-to-chat click handling

aab9382

Fix terminal context chat timeline regressions

8e7031c

Refactor terminal context message display state

17bc503

github-actions bot added size:XXL 1,000+ changed lines (additions + deletions). vouch:unvouched PR author is not yet trusted in the VOUCHED list. labels Mar 13, 2026

macroscopeapp bot reviewed Mar 13, 2026

View reviewed changes

apps/web/src/lib/terminalContext.ts Show resolved Hide resolved

apps/web/src/components/ThreadTerminalDrawer.tsx Show resolved Hide resolved

Fix terminal context review feedback

9822e65

juliusmarminge reviewed Mar 13, 2026

View reviewed changes

Render terminal contexts inline in chat timeline

1bee2ab

Merge branch 'main' into feat/terminal-context

b708f38

maskdotdev requested a review from juliusmarminge March 13, 2026 23:29

Merge branch 'main' into feat/terminal-context

f3527cf

maskdotdev changed the title ~~Give Agents access to Terminal output~~ draft: Give Agents access to Terminal output Mar 15, 2026

maskdotdev changed the title ~~draft: Give Agents access to Terminal output~~ Give Agents access to Terminal output Mar 15, 2026

Merge branch 'main' into feat/terminal-context

3e23c8d

Merge branch 'main' into feat/terminal-context

050a3bc

Conversation

maskdotdev commented Mar 13, 2026 • edited by macroscopeapp bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What Changed

Why

UI Changes

Checklist

Give agents access to terminal output by attaching selected text as context in chat

Uh oh!

coderabbitai bot commented Mar 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Review skipped

Uh oh!

Uh oh!

Uh oh!

juliusmarminge left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

maskdotdev commented Mar 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

juliusmarminge commented Mar 13, 2026

Uh oh!

maskdotdev commented Mar 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

juliusmarminge commented Mar 13, 2026

Uh oh!

maskdotdev commented Mar 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

juliusmarminge commented Mar 13, 2026

Uh oh!

maskdotdev commented Mar 13, 2026

Uh oh!

brrock commented Mar 14, 2026

Uh oh!

maskdotdev commented Mar 15, 2026

Uh oh!

juliusmarminge commented Mar 16, 2026

Uh oh!

maskdotdev commented Mar 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

juliusmarminge commented Mar 16, 2026

Uh oh!

maskdotdev commented Mar 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

maskdotdev commented Mar 13, 2026 •

edited by macroscopeapp bot

Loading

coderabbitai bot commented Mar 13, 2026 •

edited

Loading

juliusmarminge left a comment •

edited

Loading

maskdotdev commented Mar 13, 2026 •

edited

Loading

maskdotdev commented Mar 13, 2026 •

edited

Loading

maskdotdev commented Mar 13, 2026 •

edited

Loading

maskdotdev commented Mar 16, 2026 •

edited

Loading

maskdotdev commented Mar 16, 2026 •

edited

Loading