Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions THIRD_PARTY_NOTICES.md
Original file line number Diff line number Diff line change
Expand Up @@ -473,8 +473,10 @@ when multiple packages use the same license.
- `clsx@2.1.1` — MIT
- `cookie@1.1.1` — MIT
- `dompurify@3.4.12` — MPL-2.0 or Apache-2.0
- `fancy-canvas@2.1.0` — MIT
- `framer-motion@12.42.2` — MIT
- `highlight.js@11.11.1` — BSD-3-Clause
- `lightweight-charts@5.2.0` — Apache-2.0
- `lucide-react@1.25.0` — ISC
- `marked@18.0.7` — MIT
- `motion@12.42.2` — MIT
Expand All @@ -494,6 +496,16 @@ The type-only `@types/*` and `csstype` packages installed to compile the
TypeScript source do not enter the browser bundle and are intentionally
omitted from this production list.

### Vendored dependency licenses

`fancy-canvas@2.1.0` (a dependency of `lightweight-charts`) publishes a `files`
allowlist covering only its compiled JavaScript and type declarations, so its
MIT text is absent from the npm tarball. The upstream license from
https://github.com/tradingview/fancy-canvas is kept verbatim at
`frontend/vendor-licenses/fancy-canvas-LICENSE.txt`, and the Control UI builder
appends it to the generated ledger marked as vendored. Packages that ship no
license and have no vendored copy still fail the build.

### React Control UI fonts

- Inter Variable — Copyright (c) 2016 The Inter Project Authors
Expand Down
61 changes: 61 additions & 0 deletions docs/artifacts-and-media.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,67 @@ Use artifacts for:

Use chat text for short answers, decisions, and next steps.

## Inline Charts

Most artifacts render in Web UI chat as a download chip; images and audio render
inline. One more mime renders inline as an interactive chart:

| Mime | Rendered as |
|------|-------------|
| `application/vnd.agentos.chart+json` | Candlestick chart with a volume histogram |

There is nothing to register: any skill gets a chart by publishing a file with
that mime. Two steps, and both have a failure mode worth knowing.

**1. Write the JSON inside the workspace.** Scripts run with the workspace as
their working directory, so a bare filename lands in the right place.
`publish_artifact` rejects anything outside the workspace, so an absolute path
like `/tmp/chart.json` fails *after* the data has already been fetched.

**2. Publish it with the mime spelled out.**

```
publish_artifact path=bonk-1h.chart.json mime=application/vnd.agentos.chart+json
```

Passing `mime` is what makes the chart appear. Omit it and the filename guess
returns `application/json`, which classifies as a data artifact and renders as
an ordinary download chip — no error, just no chart. Tell the model to pass it
explicitly in your SKILL.md.

The body is:

```json
{
"type": "candlestick",
"title": "BONK · 1h",
"subtitle": "SOL · 1h",
"candles": [
{ "time": 1754380800, "open": 1.2e-6, "high": 1.5e-6, "low": 1.1e-6,
"close": 1.4e-6, "volume": 12500.5 }
]
}
```

`time` is a Unix timestamp in **seconds** and `volume` is optional; every other
candle field is required. Rows may arrive in any order and may repeat a
timestamp — the renderer sorts them and keeps the last entry per timestamp.
`title` and `subtitle` are display-only text, never markup.

Hovering a candle reveals its open, high, low, close, volume, and the move from
open to close as a percentage, so there is no need to repeat those numbers in
the reply.

The `gmgn-market` and `gmgn-token` skills each ship `scripts/kline_chart.py`,
which converts GMGN kline output into this shape; use it as a worked example
when adding charts to another skill. Note that they carry a copy each rather
than sharing one: `{baseDir}` resolves to the skill that is running, so a skill
cannot reach into another skill's `scripts/` directory — and the skill it
pointed at may not even be enabled. Give your skill its own copy.

Charts render in the Web UI only. Other surfaces receive the artifact as a
normal JSON file, so keep a short text summary in the reply alongside the chart.

## Document Skills

AgentOS includes skills for common document formats:
Expand Down
19 changes: 19 additions & 0 deletions docs/features/skills.md
Original file line number Diff line number Diff line change
Expand Up @@ -190,6 +190,25 @@ agentos skills tap remove <owner/repo>

Use taps when your team maintains its own skill catalog.

## What a Skill Can Put in the Chat

Besides text, a skill hands files to the surface with `publish_artifact`. Most
arrive in Web UI chat as a download chip, but some mimes render inline instead:

| Mime | Rendered as |
| --- | --- |
| `image/*` | Inline preview |
| `audio/*` | Inline player |
| `application/vnd.agentos.chart+json` | Interactive candlestick chart |

Nothing has to be registered for this: the mime a skill publishes decides how
its output is drawn, so any skill can render a chart without a frontend change.

[`artifacts-and-media.md`](../artifacts-and-media.md#inline-charts) carries the
payload shape, the workspace rule `publish_artifact` enforces, and the two
mistakes that produce no error and no chart — omitting the mime, and writing the
file outside the workspace.

## Publish and Inspect

Publish a skill directory:
Expand Down
2 changes: 1 addition & 1 deletion docs/tools-and-sandbox.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ For a focused permissions guide, see
| Web | `web_search`, `web_fetch`, `http_request`. |
| Memory | `memory_search`, `memory_save`, `memory_get`, `memory_delete`, `memory`. |
| Sessions | `sessions_send`, `sessions_spawn`, `sessions_list`, `sessions_history`, `session_status`. |
| Artifacts | `publish_artifact`. |
| Artifacts | `publish_artifact`. The mime decides the rendering: some draw inline in chat rather than as a download chip — see [artifacts-and-media.md](artifacts-and-media.md#inline-charts). |
| Media | image generation, PDF, TTS, and media helpers. |
| Skills | `skill_list`, `skill_view`, `skill_create`, `skill_edit`, `install_skill_deps`, `meta_invoke`. |
| Control | cron scheduling and gateway control operations. |
Expand Down
7 changes: 6 additions & 1 deletion docs/web-ui.md
Original file line number Diff line number Diff line change
Expand Up @@ -196,7 +196,12 @@ cards for:
- exported data files;
- PDFs, slide decks, images, and other generated outputs.

For channel delivery limits and artifact recovery, see
Images and audio play inline in the card. An artifact published as
`application/vnd.agentos.chart+json` renders as an interactive candlestick
chart instead of a download chip — the `gmgn-market` and `gmgn-token` skills use
this for token price charts.

For the chart payload shape, channel delivery limits, and artifact recovery, see
[`artifacts-and-media.md`](artifacts-and-media.md).

## Approvals
Expand Down
16 changes: 16 additions & 0 deletions frontend/package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions frontend/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@
"clsx": "^2.1.1",
"dompurify": "^3.4.12",
"highlight.js": "^11.11.1",
"lightweight-charts": "^5.2.0",
"lucide-react": "^1.25.0",
"marked": "^18.0.7",
"motion": "^12.42.2",
Expand Down
28 changes: 28 additions & 0 deletions frontend/src/views/chat/chat-unified-css.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,34 @@ describe('unified Chat CSS contract', () => {
)
})

it('reserves chart geometry and overlays the status so a drawn chart cannot shift the transcript', () => {
// The canvas is filled after a lazy import plus a payload fetch, so its
// height must exist before either resolves.
expect(css).toMatch(/\.chat-surface \.msg-artifact-chart__canvas \{[\s\S]*?height: 20rem;/)
// Canvas and status share one grid cell — hiding the status must not
// collapse a row.
expect(css).toMatch(
/\.chat-surface \.msg-artifact-chart__canvas,[\s\S]*?\.chat-surface \.msg-artifact-chart__status \{[\s\S]*?grid-row: 3;[\s\S]*?grid-column: 1;/,
)
expect(css).toMatch(/\.chat-surface \.msg-artifact-charts \{[\s\S]*?max-width: 100%;/)
})

it('reserves the crosshair readout its own row so the first hover cannot shift the chart', () => {
// The strip is empty until a chart draws, so its height has to exist
// before the first crosshair move fills it.
expect(css).toMatch(
/\.chat-surface \.msg-artifact-chart__readout \{[\s\S]*?grid-row: 2;[\s\S]*?min-height: 1\.125rem;/,
)
// Three rows: header, readout, then the canvas cell.
expect(css).toMatch(
/\.chat-surface \.msg-artifact-chart \{[\s\S]*?grid-template-rows: auto auto 1fr;/,
)
// The strip must not swallow the crosshair it is reporting on.
expect(css).toMatch(
/\.chat-surface \.msg-artifact-chart__readout \{[\s\S]*?pointer-events: none;/,
)
})

it('reserves portalled header geometry before reactive controls mount', () => {
expect(css).toMatch(
/\.shell-chat-header__context \{[\s\S]*?min-height: 2\.5rem;[\s\S]*?overflow: visible;/,
Expand Down
139 changes: 138 additions & 1 deletion frontend/src/views/chat/chat-unified.css
Original file line number Diff line number Diff line change
Expand Up @@ -228,7 +228,8 @@
}

.chat-surface .msg-artifact-gallery,
.chat-surface .msg-artifact-files {
.chat-surface .msg-artifact-files,
.chat-surface .msg-artifact-charts {
display: grid;
gap: 0.625rem;
}
Expand All @@ -241,6 +242,12 @@
max-width: min(36rem, 100%);
}

/* Charts take the full transcript column — a candlestick series is unreadable
at gallery-card width. */
.chat-surface .msg-artifact-charts {
max-width: 100%;
}

.chat-surface .msg-artifact-card {
display: flex;
flex-direction: column;
Expand Down Expand Up @@ -285,6 +292,136 @@
font-weight: 600;
}

/* Chart artifacts (chart.ts). The canvas is drawn asynchronously by
lightweight-charts after a lazy import plus a payload fetch, so the card
reserves its full height up front — same no-layout-shift contract the image
preview above follows. */
.chat-surface .msg-artifact-chart {
display: grid;
grid-template-rows: auto auto 1fr;
gap: 0.5rem;
width: 100%;
padding: 0.75rem;
overflow: hidden;
border: 1px solid var(--border);
border-radius: var(--radius-surface);
background: color-mix(in srgb, var(--card) 92%, transparent);
color: var(--foreground);
}

.chat-surface .msg-artifact-chart__header {
display: flex;
align-items: center;
justify-content: space-between;
gap: 0.75rem;
min-width: 0;
}

.chat-surface .msg-artifact-chart__name {
overflow: hidden;
font-size: 0.8125rem;
font-weight: 600;
text-overflow: ellipsis;
white-space: nowrap;
}

.chat-surface .msg-artifact-chart__download {
flex-shrink: 0;
color: var(--muted-foreground);
font-family: var(--font-mono);
font-size: 0.6875rem;
text-decoration: none;
}

.chat-surface .msg-artifact-chart__download:hover {
color: var(--foreground);
}

/* Canvas and status share one grid cell so the status can disappear on a
successful draw without collapsing a row and shifting the transcript. */
.chat-surface .msg-artifact-chart__canvas,
.chat-surface .msg-artifact-chart__status {
grid-row: 3;
grid-column: 1;
}

.chat-surface .msg-artifact-chart__canvas {
width: 100%;
height: 20rem;
}

.chat-surface .msg-artifact-chart__status {
place-self: center;
margin: 0;
color: var(--muted-foreground);
font-family: var(--font-mono);
font-size: 0.6875rem;
pointer-events: none;
}

.chat-surface .msg-artifact-chart__status[hidden] {
display: none;
}

/* Crosshair readout. lightweight-charts has no tooltip, so this strip carries
the hovered candle's OHLC and its move. It takes its own row above the
canvas rather than floating over it, so it can neither be clipped by the
card nor cover a candle, and it reserves its height up front so the first
hover does not shift the chart. */
.chat-surface .msg-artifact-chart__readout {
display: flex;
flex-wrap: wrap;
gap: 0.0625rem 0.625rem;
grid-row: 2;
grid-column: 1;
min-height: 1.125rem;
padding: 0 0.125rem;
color: var(--muted-foreground);
font: 500 0.6875rem/1.125rem var(--font-mono);
pointer-events: none;
}

.chat-surface .msg-artifact-chart__readout-cell[hidden] {
display: none;
}

.chat-surface .msg-artifact-chart__readout-label {
margin-right: 0.1875rem;
opacity: 0.65;
}

.chat-surface .msg-artifact-chart__readout-value {
color: var(--foreground);
}

.chat-surface
.msg-artifact-chart__readout-cell[data-direction='up']
.msg-artifact-chart__readout-value {
color: #26a69a;
}

.chat-surface
.msg-artifact-chart__readout-cell[data-direction='down']
.msg-artifact-chart__readout-value {
color: #ef5350;
}

/* Mirror chartTheme()'s light palette so the move reads the same colour as the
candle body it describes. */
:root[data-theme='light']
.chat-surface
.msg-artifact-chart__readout-cell[data-direction='up']
.msg-artifact-chart__readout-value {
color: #0f9d81;
}

:root[data-theme='light']
.chat-surface
.msg-artifact-chart__readout-cell[data-direction='down']
.msg-artifact-chart__readout-value {
color: #e03131;
}

.chat-surface .msg-artifact-card__meta {
color: var(--muted-foreground);
font: 500 0.6875rem/1.4 var(--font-mono);
Expand Down
Loading
Loading