Skip to content
Merged
Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
52 commits
Select commit Hold shift + click to select a range
651bfe5
Migrating to Golem 1.5, WIP
vigoo Mar 21, 2026
83f4138
Base wasm fix
vigoo Mar 21, 2026
1ed4ce7
Remove kebab case
vigoo Mar 21, 2026
cc02e06
Examples converted into integration tests
vigoo Mar 21, 2026
917a287
Switched to zio-test, fixed tests
vigoo Mar 22, 2026
0d8a0df
Removed huge dependency and switched to fullLinkJS
vigoo Mar 22, 2026
a94d50f
All integration tests passing
vigoo Mar 22, 2026
2aceb58
Initial support for code-first routes
vigoo Mar 22, 2026
69e5c29
Schema fixes and inlining case class constructor parameters
vigoo Mar 22, 2026
54ac0d8
Doc comment update
vigoo Mar 22, 2026
398f68c
Webhook support
vigoo Mar 22, 2026
711ef71
Integration test skill, and scala version cleanup (zio-golem needs 3.…
vigoo Mar 23, 2026
b65ff1a
Initial version of config and secret support
vigoo Mar 23, 2026
bae230b
Config and secret improvements
vigoo Mar 23, 2026
45eee2e
Principal injection
vigoo Mar 23, 2026
6a702ab
Snapshotting
vigoo Mar 23, 2026
106a145
More rpc constructor variants
vigoo Mar 23, 2026
709b7ad
Merge branch 'main' into golem-1.5
vigoo Mar 23, 2026
2daf9e3
zio-golem specific agents file
vigoo Mar 24, 2026
92d693e
zio-http/fetch example
vigoo Mar 24, 2026
d7c0983
Value encoding and scala2 fixes
vigoo Mar 24, 2026
0ceb3e9
Better numeric type support
vigoo Mar 24, 2026
2e0c3cd
Enum and result support and more fixes
vigoo Mar 24, 2026
044291e
Better char and map support
vigoo Mar 24, 2026
7debcb1
Experiment with @constructor
vigoo Mar 24, 2026
31b52d5
Unified the code generator code
vigoo Mar 25, 2026
b65aa92
RPC client code generator
vigoo Mar 25, 2026
f787fdf
Merge branch 'main' into golem-1.5
vigoo Mar 25, 2026
a219e58
private constructors
vigoo Mar 26, 2026
ef85ff8
Better config override in RPC clients
vigoo Mar 26, 2026
3ebef35
Abstract remote method base
vigoo Mar 26, 2026
f95f00e
constructor method -> constructor class
vigoo Mar 26, 2026
7c28c8b
Better name for agent identity spec
vigoo Mar 26, 2026
5e0b3ac
Using ZIO schema for configs, wip
vigoo Mar 26, 2026
1fc90a1
Using ZIO schema for configs completely
vigoo Mar 26, 2026
82b8d19
Cleaned up snapshotting
vigoo Mar 26, 2026
8ec2b03
Persisting principal in snapshots
vigoo Mar 26, 2026
fef82a4
More principal validations
vigoo Mar 26, 2026
e34b455
Type names
vigoo Mar 26, 2026
8859f7f
Renames
vigoo Mar 26, 2026
cd34fbe
Renames and reorganizations
vigoo Mar 26, 2026
9ffa5be
Docs/skills fixes
vigoo Mar 26, 2026
b8ba2d1
More docs
vigoo Mar 26, 2026
75ca03b
Merge branch 'main' into golem-1.5
vigoo Mar 26, 2026
2ad76ae
Merge branch 'main' into golem-1.5
vigoo Mar 26, 2026
1356156
Got rid of auth: Byte
vigoo Mar 26, 2026
7b6cfe2
Addressed some review comments
vigoo Mar 26, 2026
2aac774
Multimodal over RPC fixes
vigoo Mar 26, 2026
72b2c46
fmt
vigoo Mar 26, 2026
dac40ef
More fmt
vigoo Mar 26, 2026
c15aa03
CI change
vigoo Mar 26, 2026
637d573
Fix
vigoo Mar 26, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
110 changes: 110 additions & 0 deletions .claude/skills/zio-golem-base-image/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,110 @@
---
name: zio-golem-base-image
description: Explains the zio-golem WIT folder structure and how to regenerate the agent_guest.wasm base image. Use when working with WIT definitions, upgrading Golem versions, or regenerating the guest runtime WASM.
---

# zio-golem Base Image (agent_guest.wasm)

The base image `agent_guest.wasm` is a QuickJS-based WASM component that serves as the guest runtime for Scala.js agents on Golem. It must be regenerated whenever WIT definitions change.

## WIT Folder Structure

```
golem/wit/
├── main.wit # Hand-maintained world definition (golem:agent-guest)
├── deps.toml # wit-deps manifest — points to golemcloud/golem main branch
├── deps.lock # Auto-generated lock file (gitignored)
└── deps/ # Auto-populated by wit-deps (gitignored)
├── golem-core/
├── golem-agent/
├── golem-1.x/
├── golem-rdbms/
├── golem-durability/
├── blobstore/
├── cli/
├── clocks/
├── config/
├── ...
└── sockets/
```

- **`main.wit`** defines the `golem:agent-guest` world — the set of imports/exports the agent component uses. This file is checked in and maintained manually.
- **`deps.toml`** declares a single dependency source: the golem repo's main branch tarball. `wit-deps` downloads and extracts the WIT packages from it.
- **`deps/`** and **`deps.lock`** are gitignored — they are populated by running `wit-deps` from `golem/`.

## When to Regenerate

The base image **must be regenerated** whenever:

1. **`wit/main.wit` changes** — adding/removing imports or exports
2. **WIT dependencies update** — e.g., upgrading from Golem v1.5.0 to a newer version (update `deps.toml` then regenerate)
3. **`wasm-rquickjs` updates** — a new version of the wrapper generator may produce different output

The generated `agent_guest.wasm` is checked in at two locations (embedded in the sbt and mill plugins):
- `golem/sbt/src/main/resources/golem/wasm/agent_guest.wasm`
- `golem/mill/resources/golem/wasm/agent_guest.wasm`

## Prerequisites

### 1. Rust toolchain

```bash
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
rustup target add wasm32-wasip1
```

### 2. cargo-component

```bash
cargo install cargo-component
```

### 3. wit-deps

```bash
cargo install wit-deps-cli
```

### 4. wasm-rquickjs (pinned to 0.1.0)

The script enforces a specific version of `wasm-rquickjs` and will refuse to run if the installed version does not match. The required version is defined by `REQUIRED_WASM_RQUICKJS_VERSION` in `generate-agent-guest-wasm.sh`.

```bash
cargo install wasm-rquickjs-cli@0.1.0
```

## How to Regenerate

From the **repository root** (`zio-blocks/`):

```bash
./golem/tools/generate-agent-guest-wasm.sh
```

The script performs these steps:

1. Runs `wit-deps update` in `golem/` to fetch the latest WIT dependencies into `golem/wit/deps/`
2. Stages a clean WIT package in `.generated/agent-wit-root/` (copies `main.wit` + `deps/`)
3. Runs `wasm-rquickjs generate-wrapper-crate` to produce a Rust crate from the WIT
4. Builds with `cargo component build --release` targeting `wasm32-wasip1`
5. Installs the resulting `agent_guest.wasm` into both plugin resource directories

## Fetching Dependencies Only

To update the WIT dependencies without regenerating the WASM:

```bash
cd golem && wit-deps
```

The generation script always uses `wit-deps update` to ensure deps are fresh. To update deps without a full regeneration:

```bash
cd golem && wit-deps update
```

## How It Fits Together

At build time, the sbt/mill `GolemPlugin` extracts the embedded `agent_guest.wasm` from plugin resources and writes it to the user project's `.generated/agent_guest.wasm`. Then `golem-cli` uses this base runtime to compose the final component: it injects the user's Scala.js bundle into the QuickJS runtime and wraps it as a proper Golem agent component.

The Scala SDK does **not** parse WIT to generate Scala bindings. Instead, Scala macros + ZIO Schema produce `AgentMetadata` at compile time, and `WitTypeBuilder` maps schema types to WIT-compatible JS representations at runtime. The WIT definitions only flow through the WASM guest runtime.
197 changes: 197 additions & 0 deletions .claude/skills/zio-golem-development/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,197 @@
---
name: zio-golem-development
description: "Compile, publish, and test the ZIO Golem Scala.js SDK. Use when working on the golem/ subtree: building the SDK, publishing locally, compiling/running the gettingStarted demo, regenerating the agent_guest.wasm, or debugging end-to-end deployment."
---

# ZIO Golem SDK Development

The Golem SDK for Scala.js lives under `golem/` in the zio-blocks monorepo. It targets the Golem WIT API v1.5.0 and produces WASM components that run on the Golem platform via a QuickJS-based guest runtime.

## Repository Layout

```
golem/
├── core/ # zio-golem-core (Scala.js facades, agent framework)
├── model/ # zio-golem-model (WIT value types, RPC types)
├── macros/ # zio-golem-macros (Scala 3 macros, JVM-only)
├── sbt/ # zio-golem-sbt (SBT plugin, Scala 2.12)
├── mill/ # Mill plugin
├── wit/ # WIT definitions (main.wit + deps/)
│ ├── main.wit # Primary WIT — package golem:agent-guest, world agent-guest
│ ├── deps/ # WIT dependencies (copied from golem repo)
│ └── dts/ # Generated TypeScript d.ts (source of truth for JS exports)
├── tools/ # generate-agent-guest-wasm.sh and agent-wit/
├── examples/ # zioGolemExamples project
├── gettingStarted/ # Standalone demo project (separate sbt build)
├── quickstart/ # Quickstart template
└── docs/ # Documentation
```

## Scala Versions

- **Scala 3.3.7** — All Golem JS/JVM projects. Must always prefix sbt commands with `++3.3.7`.
- **Scala 2.12.21** — The SBT plugin (`zioGolemSbt`) only. Use `++2.12.21!` (the `!` forces override).

> **Important**: `sbt --client` mode preserves Scala version across invocations. Always specify the version explicitly to avoid version drift.

## SBT Project Names

| Project | Description |
|---------|-------------|
| `zioGolemCoreJS` / `zioGolemCoreJVM` | Core agent framework, Scala.js facades |
| `zioGolemModelJS` / `zioGolemModelJVM` | WIT value types, RPC types |
| `zioGolemMacros` | Scala 3 macros (JVM only, cross-used at compile time) |
| `zioGolemSbt` | SBT plugin (Scala 2.12) |
| `zioGolemExamples` | Example agents |

## Compiling

From the monorepo root (`/home/vigoo/projects/zio-blocks`):

```bash
# Compile examples (good smoke test)
sbt --client "++3.3.7; zioGolemExamples/fastLinkJS"

# Compile core
sbt --client "++3.3.7; zioGolemCoreJS/compile"

# Compile model
sbt --client "++3.3.7; zioGolemModelJS/compile"
```

Use the standard AGENTS.md sbt logging pattern:
```bash
ROOT="$(git rev-parse --show-toplevel)" && mkdir -p "$ROOT/.git/agent-logs"
LOG="$ROOT/.git/agent-logs/sbt-$(date +%s)-$$.log"
sbt --client -Dsbt.color=false "++3.3.7; zioGolemExamples/fastLinkJS" >"$LOG" 2>&1
echo "Exit: $? | Log: $LOG"
# Query: tail -50 "$LOG" or grep -i error "$LOG"
```

## Publishing Locally

The `gettingStarted` project depends on `0.0.0-SNAPSHOT` artifacts. All golem projects have `publish / skip := true` by default, so you must override it.

### Step 1: Publish Dependencies + Golem Libraries (Scala 3.3.7)

```bash
sbt --client '++3.3.7; set ThisBuild / version := "0.0.0-SNAPSHOT"; set ThisBuild / packageDoc / publishArtifact := false; set every (publish / skip) := false; typeidJVM/publishLocal; typeidJS/publishLocal; chunkJVM/publishLocal; chunkJS/publishLocal; markdownJVM/publishLocal; markdownJS/publishLocal; schemaJVM/publishLocal; schemaJS/publishLocal; zioGolemModelJVM/publishLocal; zioGolemModelJS/publishLocal; zioGolemMacros/publishLocal; zioGolemCoreJS/publishLocal; zioGolemCoreJVM/publishLocal'
```

### Step 2: Publish SBT Plugin (Scala 2.12.21)

```bash
sbt --client '++2.12.21!; set ThisBuild / version := "0.0.0-SNAPSHOT"; set ThisBuild / packageDoc / publishArtifact := false; set every (publish / skip) := false; zioGolemSbt/publishLocal'
```

> **Gotcha**: The `golemPublishLocal` alias in `build.sbt` does NOT work correctly — it doesn't override `publish / skip` and uses the wrong Scala 2.12 version. Always use the explicit commands above.

## Building the gettingStarted Project

The `gettingStarted` project at `golem/gettingStarted/` is a standalone sbt project (its own `build.sbt`, `project/plugins.sbt`). It depends on the SDK at `0.0.0-SNAPSHOT`.

### Prerequisites
1. Publish the SDK locally (both steps above).

### Clean Build
```bash
cd golem/gettingStarted
rm -rf target project/target .bsp .generated .golem
sbt -batch -no-colors -Dsbt.supershell=false compile
```

### Key SBT Tasks
- `sbt golemPrepare` — Generates `.generated/agent_guest.wasm` (extracted from plugin resources) and `.generated/scala-js-template.yaml` (component manifest template).
- `sbt compile` — Compiles the Scala agent code.
- `sbt fastLinkJS` — Links the Scala.js bundle (produces the JS that QuickJS will run).

### Project Structure
- `build.sbt` — Enables `ScalaJSPlugin` + `GolemPlugin`, sets `scalaJSUseMainModuleInitializer := false`, ESModule output.
- `project/plugins.sbt` — Adds `zio-golem-sbt` and `sbt-scalajs`.
- `golem.yaml` — Declares app name, includes `.generated/scala-js-template.yaml`, defines component `scala:demo`.
- `repl-counter.rib` — Rib script for end-to-end testing via `golem-cli repl`.

## End-to-End Testing

### Start the Local Golem Server
```bash
golem server run --clean
```
This starts the all-in-one Golem server on `localhost:9881`.

### Using run.sh
```bash
cd golem/gettingStarted
bash run.sh
```

The script does:
1. `sbt golemPrepare` — Generate wasm + manifest template
2. `golem-cli build --yes` — Build the WASM component (links QuickJS runtime + Scala.js bundle)
3. `golem-cli deploy --yes` — Deploy to local Golem server
4. `golem-cli repl scala:demo --script-file repl-counter.rib` — Run the demo

### Manual Steps
```bash
cd golem/gettingStarted
sbt golemPrepare
golem-cli build --yes
golem-cli deploy --yes --local
golem-cli repl scala:demo --script-file repl-counter.rib --local
```

> **Note**: `golem-cli` v1.5.0-dev is at `/home/vigoo/.cargo/bin/golem-cli`. The `--local` flag targets the local server.

## Regenerating agent_guest.wasm

The agent_guest.wasm is the QuickJS-based WASM runtime that wraps the Scala.js bundle. Regenerate it when WIT definitions change.

### Script
```bash
./golem/tools/generate-agent-guest-wasm.sh
```

### What It Does
1. Resolves WIT deps via `wit-deps update`.
2. Stages WIT package from `golem/wit/` (skipping the `all/` dep directory).
3. Generates TypeScript d.ts definitions via `wasm-rquickjs generate-dts` → saved to `golem/wit/dts/`.
4. Generates QuickJS wrapper crate via `wasm-rquickjs generate-wrapper-crate`.
5. Builds with `cargo component build --release`.
6. Installs the wasm into `golem/sbt/src/main/resources/golem/wasm/agent_guest.wasm` and `golem/mill/resources/golem/wasm/agent_guest.wasm`.
7. Copies d.ts files to `golem/wit/dts/`.

### Requirements
- `wit-deps` (`cargo install wit-deps-cli`)
- `wasm-rquickjs` v0.1.0 (`cargo install wasm-rquickjs-cli@0.1.0`)
- Rust toolchain + `cargo-component` (`cargo install cargo-component`)

## WIT Management

### Files
- **Primary**: `golem/wit/main.wit` — The `golem:agent-guest` package definition.
- **Dependencies**: `golem/wit/deps/` — Copied from the main Golem repo at `/home/vigoo/projects/golem/wit/deps/`.
- **TypeScript reference**: `golem/wit/dts/` — Generated d.ts files showing exact JS types expected by the wasm runtime. `exports.d.ts` is the source of truth for what the JS module must export.
- **Legacy**: `golem/tools/agent-wit/main.wit` — May still be referenced but `golem/wit/main.wit` is authoritative.

### Updating WIT Dependencies
1. Delete `golem/wit/deps.lock` (prevents `wit-deps` from restoring stale versions).
2. Copy fresh deps from the Golem repo: `cp -r /home/vigoo/projects/golem/wit/deps/* golem/wit/deps/`.
3. Run `wit-deps update` from the `golem/` directory.
4. Regenerate `agent_guest.wasm` using the generate script.

> **Gotcha**: `wit-deps` with a stale `deps.lock` can silently overwrite correct deps with old versions.

### TypeScript SDK Reference
The TypeScript SDK at `/home/vigoo/projects/golem/sdks/ts/wit/` is the reference for correct WIT definitions when in doubt.

## Common Errors and Solutions

| Error | Cause | Solution |
|-------|-------|----------|
| `Function discover-agent-types not found in interface golem:agent/guest@1.5.0` | Stale `agent_guest.wasm` built from old WIT | Regenerate wasm with `generate-agent-guest-wasm.sh` |
| `Cannot find exported JS function guest.discoverAgentTypes` | Scala.js Guest object doesn't match WIT signature | Update `Guest.scala` to export all 4 functions with correct v1.5.0 signatures (including `principal` param) |
| `YAML deserialization error` in `golem.yaml` about `BuildCommand` | Old GolemPlugin manifest format | Update `GolemPlugin.scala` to use v1.5.0 format (`componentWasm`/`outputWasm`) |
| `wit-deps` restoring stale 1.1.7 dependencies | Stale `deps.lock` | Delete `golem/wit/deps.lock`, copy fresh deps from Golem repo |
| `Provided exports: (empty)` after deploy | QuickJS fails to evaluate the JS module silently | JS crashes during initialization — check for ESM strict-mode issues, bundle size limits, or import path mismatches |
| `publish / skip` preventing local publish | Default setting in `build.sbt` | Use `set every (publish / skip) := false` in the sbt command |
| Wrong Scala 2.12 version for plugin | `golemPublishLocal` alias uses `2.12.20` instead of `2.12.21` | Use the explicit `++2.12.21!` command instead of the alias |
Loading