Skip to content

feat(chat): render token price charts inline in Web UI chat - #224

Merged
keyKQ merged 7 commits into
use-agent-os:mainfrom
keyKQ:feat/chat-inline-charts
Aug 7, 2026
Merged

feat(chat): render token price charts inline in Web UI chat#224
keyKQ merged 7 commits into
use-agent-os:mainfrom
keyKQ:feat/chat-inline-charts

Conversation

@keyKQ

@keyKQ keyKQ commented Aug 5, 2026

Copy link
Copy Markdown
Collaborator

What

Web UI chat renders artifacts as download chips, with images and audio playing inline. This adds one more inline form: an artifact published as application/vnd.agentos.chart+json draws as an interactive candlestick chart with a volume histogram.

The gmgn-market and gmgn-token skills use it, so a token price question comes back with a real chart next to the text summary instead of a table of numbers.

How

One new mime, routed through the existing artifact seam. artifactCategory gains a chart category; renderArtifacts emits a mount placeholder instead of a chip, and a mounter fetches the payload and draws into it. Going through the artifact seam rather than a markdown fence means history replay redraws charts with no separate code path.

Payload shape (docs/artifacts-and-media.md documents it for other skill authors):

{ "type": "candlestick", "title": "BONK · 1h", "subtitle": "SOL · 1h",
  "candles": [{ "time": 1754380800, "open": 1.2e-6, "high": 1.5e-6,
                "low": 1.1e-6, "close": 1.4e-6, "volume": 12500.5 }] }

Normalization is defensive on purpose. lightweight-charts asserts on unordered or repeated timestamps and would take the whole transcript render down, so normalizeChartPayload coerces numeric strings, drops rows missing OHLC fields, converts millisecond timestamps to seconds, sorts, and de-duplicates by timestamp (last entry wins, matching how an exchange restates a candle).

Bundle cost is deferred. lightweight-charts is imported dynamically, so it lands in its own 63.5 KB gz chunk and never loads for a chat with no chart. The largest chunk sits at 132.5 KB against the 180 KB budget. All charts on a transcript share one import rather than each racing its own.

Price precision scales to the token. A meme token at 0.0000017 would render as 0.00 under the default 2-decimal formatter, so decimals are derived from the smallest close. This is applied per series rather than via localization.priceFormatter, which is chart-global and would otherwise print volume as 2739.0000000000 instead of 2.74K.

A crosshair readout, because lightweight-charts ships no tooltip. Without one the chart shows a shape and nothing else — hovering a candle says neither what it opened at nor how far it moved. A strip above the canvas carries the hovered candle's time, OHLC, volume, and close-against-open as a signed percentage: the move the body itself draws, not the move since the previous close. It rests on the newest candle until the cursor arrives and returns there when the cursor leaves. It takes its own grid row rather than floating over the canvas, so it cannot cover the candles it describes or be clipped by the card, its height is reserved so the first hover cannot shift the transcript, and it is pointer-events: none so it cannot swallow the crosshair it reports on.

Each skill publishes a chart on its own

{baseDir} resolves per skill and the loader has no skill-to-skill dependency mechanism — requires covers bins and env only. A gmgn-token that deferred to gmgn-market for the chart step would therefore render nothing at all whenever only gmgn-token was enabled, silently. Both skills now ship scripts/kline_chart.py and the guidance needed to publish a chart alone; a test pins the two copies byte-for-byte so they cannot drift.

Both skill docs also state that --output must stay a bare filename: scripts run with the workspace as their working directory and publish_artifact only accepts files inside it, so an absolute path outside the workspace fails the publish after the candles have already been fetched.

Interaction and lifecycle

A click on the chart must not download it. The transcript binds one delegated handler that downloads any non-anchor element resolving to [data-artifact-download]. Stamping that attribute on the chart host made every pan, zoom, and crosshair click fetch the JSON instead — the chart drew but could not be touched. The audio card already had the answer: it is the other non-anchor card and keeps the attribute only on its Download anchor, which the handler steps aside for. The chart card now matches, and the transcript export still resolves the URL through the child-anchor fallback written for audio.

Charts are disposed when their row goes away. The transcript rebuilds its rows wholesale on a session switch and on "load earlier". Disposing only on route unmount stranded a canvas, a ResizeObserver, and a theme callback per rebuild. Live charts are keyed by host so detached ones can be swept, and both rebuild sites sweep.

Third-party origin

Package Version License Upstream
lightweight-charts 5.2.0 Apache-2.0 https://github.com/tradingview/lightweight-charts
fancy-canvas 2.1.0 MIT transitive dependency

Both are recorded in THIRD_PARTY_NOTICES.md. layout.attributionLogo is left at its default true, so the TradingView attribution mark stays visible.

fancy-canvas publishes a files allowlist covering only its compiled JavaScript and type declarations, so its MIT text is absent from the npm tarball and the license bundler had no text to embed — which failed the Control UI build on every runner. The upstream license is vendored at frontend/vendor-licenses/fancy-canvas-LICENSE.txt and used only when an installed package ships none; the generated ledger marks it as vendored rather than presenting it as something the tarball carried. A package with neither a license nor a vendored copy still fails the build, so a new dependency without attribution stays a deliberate decision.

Using this from another skill

Nothing has to be registered: the mime a skill publishes decides how its output
is drawn, so any skill renders a chart without a frontend change. That is only
useful if a skill author can find out, and the contract previously lived in a
document nothing on the authoring path linked to — features/skills.md, the
canonical skills reference, did not mention artifacts at all.

  • docs/features/skills.md now lists the mimes that render inline and links to
    the contract.
  • docs/tools-and-sandbox.md says the same on the publish_artifact row.
  • docs/artifacts-and-media.md documents the two steps that actually produce a
    chart, both of which fail quietly: the file must be inside the workspace, and
    publish_artifact needs the mime spelled out or the filename guess yields
    application/json and an ordinary download chip.
  • skill_create points at that document in its result rather than its
    description, which would spend tokens on every turn to say something that only
    matters at the moment a skill is written.

Notes for reviewers

  • Resolution is chosen by the model from the token's creation_timestamp against a documented age table, targeting 30–100 candles.
  • Charts are Web UI only. Other surfaces receive the artifact as ordinary JSON, which is why the skills keep their text summary.
  • The readout formats times in UTC because lightweight-charts plots UTC unless a timezone is configured; a local-time readout would disagree with the axis directly beneath it.

Security

The gmgn skills warn that token name, symbol, and description are fully attacker-controlled — anyone can mint a token with arbitrary text in them. Every payload-derived string in this change reaches the DOM through textContent, never innerHTML. Card markup built as an HTML string continues to use the existing esc/escAttr helpers.

Test plan

Full gate per AGENTS.md, all green:

python scripts/build_control_ui.py build   # bundle budget: 132.5 / 180 KB gz
npm --prefix frontend run check            # tsc + eslint + prettier + 1738 vitest
uv run ruff check src tests                # All checks passed
uv run mypy src/agentos                    # no issues in 591 source files
uv run pytest tests -q                     # 7323 passed, 27 skipped
uv build --wheel                           # both kline_chart.py copies confirmed in the wheel

Coverage added beyond the payload helpers, which were the only part previously tested:

  • chart.test.ts — 51 cases. The mounter runs against a stubbed lightweight-charts: mount, idempotence under repeated stream calls, error and empty-payload states, detached-row disposal, re-mount after a rebuild, theme re-colour, teardown, and the crosshair readout including unsubscribe-before-remove ordering.
  • artifacts.test.ts — the placeholder's hooks, the handoff to the mounter on stream append and flush, and that the canvas resolves to no download target. That last one fails if the attribute ever returns to the host.
  • history.test.ts — a replayed row reaches the mounter, so a reload redraws charts.
  • chart.test.ts also pins the renderer→mounter contract end to end: markup from the real artifact renderer, drawn by the real mounter. The two modules meet through class names and a data attribute, which nothing else would catch a rename in.
  • tests/test_skill_gmgn_charts.py — 11 cases over the converter now that two skills carry it: candle discovery across every response shape the CLI has shipped, ms→s, ordering and dedup, dropped partial rows, and USD volume rather than token-count amount.
  • tests/test_scripts/test_build_control_ui.py — the vendored-license fallback, that a package with no license and no vendored entry still fails, that a declared-but-missing vendored file fails, and that the checked-in vendored files carry a real copyright.

Verified against a live gateway with real GMGN data, not only fixtures: the chart draws, the crosshair readout tracks the hovered candle, and clicking the chart no longer downloads it.

keyKQ added 5 commits August 6, 2026 14:13
Artifacts published as `application/vnd.agentos.chart+json` now draw as an
interactive candlestick chart in the chat transcript instead of a download
chip, and the gmgn skills emit one alongside their text summaries.

The chart goes through the existing artifact seam, so history replay redraws
it without a separate code path. lightweight-charts is imported dynamically:
it lands in its own 63.5 KB gz chunk and never loads for a chat with no
chart, leaving the Chat route at 93.8 KB gz against a 180 KB budget.

Payload strings (token names and symbols) are attacker-controlled on-chain
metadata, so every one of them reaches the DOM through textContent.

- chart.ts: payload normalization (ms->s, sort, dedupe by timestamp) plus a
  lazy, theme-aware mounter
- artifacts.ts: new `chart` category rendering a mount placeholder
- gmgn-market: scripts/kline_chart.py converts kline output to the payload
- gmgn-token: fetches a chart during research via the gmgn-market skill
Four gaps in the inline-chart feature, found by re-reading it against
what it was supposed to do rather than against its own description.

The Control UI build failed on every CI runner, so nothing downstream of
it ever ran — not ruff, not mypy, not pytest, not vitest. `fancy-canvas`,
a dependency of lightweight-charts, publishes a `files` allowlist
covering only its compiled JavaScript, so its MIT text never leaves the
source repository and the license bundler found nothing to embed. The
upstream license is now vendored under frontend/vendor-licenses/ and used
only when an installed package ships none; a package with neither a
license nor a vendored copy still fails the build, so a new dependency
without attribution stays a deliberate decision.

A chart was supposed to appear when the gmgn-token skill researched a
token. Instead gmgn-token deferred to gmgn-market — which owns the
converter script and the resolution table — and the loader has no
skill-to-skill dependency mechanism (`requires` covers bins and env
only). With only gmgn-token enabled the chart silently never appeared.
Each skill now ships the converter and the guidance it needs to publish a
chart alone, pinned byte-for-byte against each other.

Charts leaked on every session switch and every "load earlier": the
mounter disposed them only on route unmount, but the transcript rebuilds
its rows in place, stranding a canvas, a ResizeObserver and a theme
callback per rebuild. Live charts are now keyed by their host so a
detached row can be swept, and the two rebuild sites sweep.

None of the drawing or wiring had tests — the pure payload helpers did,
but a rename on either side of the placeholder contract would have shown
the user nothing at all and failed no test. The mounter, the placeholder
markup, the stream and history handoffs, and the renderer-to-mounter
contract are covered against a stubbed lightweight-charts, and the
converter gained pytest coverage now that two skills carry it.

Charts also share one library import instead of racing one per chart.
publish_artifact only accepts files under the active workspace, and a
model told to build a chart payload will reach for /tmp unless told
otherwise — which fails the publish after the candles were already
fetched. Both chart sections now say to keep --output a bare filename.
The transcript binds one delegated click handler that downloads any
non-anchor element resolving to `[data-artifact-download]`, and the chart
placeholder stamped that attribute on its outer host. Every click that
landed anywhere inside the card — including a pan, a zoom, or a crosshair
move on the canvas itself — matched it, called preventDefault, and
fetched the JSON. The chart drew, but it could not be touched.

The audio card already had the answer: it is the other non-anchor card,
and it keeps the attribute off the host and only on its Download anchor,
which the handler steps aside for because it is an anchor. The chart card
now does the same, and the transcript export still resolves the URL
through the child-anchor fallback written for audio.

The test asserts the canvas resolves to no download target, which fails
if the attribute ever returns to the host.
lightweight-charts ships no tooltip, so the chart could show a shape and
nothing else: hovering a candle told you neither what it opened at nor
how far it moved, which is most of what a candle is for.

A readout strip above the canvas now carries the hovered candle's time,
OHLC and volume, plus close against open as a signed percentage — the
move the body itself draws, not the move since the previous close. It
rests on the newest candle until the cursor arrives and returns there
when the cursor leaves, so the strip is never blank.

The strip takes its own grid row rather than floating over the canvas:
an overlay would cover the candles it describes and could be clipped by
the card. Its height is reserved up front so the first hover cannot shift
the transcript, and it is pointer-events: none so it cannot swallow the
crosshair it reports on.

Prices use the payload's own precision, so a token trading at 0.0000123
reads at full resolution instead of flattening to 0.00, and the
percentage takes its colour from the same palette as the candle body in
both themes.
@keyKQ
keyKQ force-pushed the feat/chat-inline-charts branch from e3b914c to 8e81201 Compare August 6, 2026 08:54
keyKQ added 2 commits August 6, 2026 16:12
The section documented the payload but not the two steps that actually
produce a chart, which are also the two that fail quietly: the file has
to be inside the workspace, and publish_artifact needs the mime spelled
out or the filename guess makes it a plain download chip.

Also corrects the converter's home now that both gmgn skills carry a
copy, and says why a skill cannot borrow another skill's script.
The chart contract was documented only in artifacts-and-media.md, which
nothing on the skill-authoring path links to. features/skills.md — the
canonical skills reference — never mentioned artifacts at all, so a skill
author had no way to learn that publishing one mime rather than another
is the whole difference between a chart and a download chip.

features/skills.md now lists the mimes that render inline and links to
the contract; the tools reference says the same on the publish_artifact
row. skill_create points at it in its result rather than its description,
which would spend tokens on every turn to say something that only matters
at the moment a skill is written.
@keyKQ
keyKQ merged commit 72b551a into use-agent-os:main Aug 7, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant