Skip to content

Latest commit

 

History

History
163 lines (111 loc) · 14.4 KB

run-protocol.md

File metadata and controls

163 lines (111 loc) · 14.4 KB

OMK Run Protocol v1

The OMK Run Protocol defines one versioned contract for task execution and evaluation:

TaskSpec -> ExecutionAttempt -> Observation -> EvaluationResult -> RuntimeDecision

omk-protocol owns these records and the pure reducers that connect them. Tool execution, persistence, runtime scheduling, routing, and topology selection remain outside the package.

Implemented scope

The first v1 slice is available under packages/protocol with schema version omk.run.v1.

Contract Purpose
TaskSpec Goal and required or advisory ClaimPredicate records
ExecutionAttempt One completed initial, retry, failover, or resumed execution
Observation Immutable facts tied to a task and attempt
ClaimEvaluation Derived satisfied, violated, or inconclusive claim result
EvaluationResult Claim evaluations plus one semantic pass, fail, or inconclusive verdict
RuntimeDecision Pure continue, retry, failover, or stop decision
WaiverRecord Explicit, scoped, attributable, and optionally expiring exception

Every top-level record carries schemaVersion. Parsers reject unsupported versions, malformed timestamps, duplicate claim IDs, invalid JSON facts, and empty logical conditions.

Isolated command-run profile

RunContract and RunStartCommand add strict, immutable parsers for the opt-in linux-command-v1 and linux-scripted-agent-v1 profiles. They pin the input digest, write scope, commands, stdout assertions and finite phase/snapshot limits. The scripted profile also pins 1–16 approved steps and a 1–32 logical request cap for an offline AgentSession reference adapter. Unknown authority/provider fields are rejected; the host must separately authorize the exact contract digest. RunResumeCommand / parseRunResumeCommand() pin the contract, candidate and exact expected revision/generation. The host may acquire at most three generations under MAX_VERIFIED_RUN_GENERATIONS; a resume command does not authorize replay of a writer. The separate RunWriterRestartCommand / parseRunWriterRestartCommand() binds baseDigest to a durable input checkpoint for an explicit local writer restart. Recovery commands share command IDs and the generation cap; none resets budgets.

linux-command-dag-v1 adds RunDagWriter / RunDagTask with 1–16 nodes, explicit artifact dependencies, disjoint write scopes and 1–2 preapproved command attempts per node. Optional maxConcurrentTasks accepts only 1 or 2; omission preserves legacy serialization and serial execution. orderRunDag() provides deterministic FIFO topological order and runDagAncestors() includes the full transitive input closure. RunTaskRetryCommand / parseRunTaskRetryCommand() pins a task selection to the original input and exact run revision/generation. These pure contracts never authorize dispatch or authenticate cached output; the Coordinator checks stored material and records adoption before reuse.

The coding-agent's RunCoordinator owns execution and its separate v2 journal. It binds native v3 receipt cores to the candidate through a supervisor attestation, then supplies authenticated checks to the existing claim-closure reducer. This does not replace the task/attempt/evaluation contracts or add execution to this package. See Verified Run for the implemented CLI/SDK path, candidate recovery and checkpoint-based local writer restart, plus the bounded command DAG, eager frontier and selective retry, plus remaining live-model, verification-edge, plan amendment and control-surface work.

Durable goal lifecycle

A durable goal is working-directory state, not a session-file field or a TaskSpec. /goal <objective> creates or edits .omk/goals/current.json; /goal without arguments shows its status and round count.

Goals created by /goal use an eight-round cap. After a turn settles, the controller queues the next round as a follow-up while the goal is active, no message is pending, and the cap has not been reached. A turn settles when no automatic retry follows it, so an attempt that is about to be retried does not use up a round. Reaching the cap stops automatic continuation. Without an acceptance check the controller does not infer or mark completion.

Through 1.2.4 the controller advanced the round at every attempt's agent_end and sent the next turn while that run still owned the session. The session rejected the message (Agent is already processing), so the round was spent and the goal never continued.

For programmatic lifecycle control, import createDurableGoal, parseDurableGoalSnapshot, applyDurableGoalCommand, and DurableGoalStore from open-multi-agent-kit. The reducer supports edit, pause, resume, block, round advancement, evidence attachment, completion, and clear transitions.

Every mutation consumes the current revisioned GoalRef; stale revisions are rejected. Editing the objective or round limit, or advancing a round, starts a new semantic generation and invalidates earlier completion evidence. Completion requires lowercase SHA-256 evidence captured during the current generation. The built-in controller dates each transition no earlier than the journal's last timestamp, so a wall clock that steps back does not fail it. Programmatic callers get the same rule from nextDurableGoalTimestamp(goal): pass its result as the now argument of applyDurableGoalCommand() or DurableGoalStore.transition(). A raw new Date().toISOString() fails with goal timestamps must be monotonic once the clock has stepped back. The journal lock is reclaimed only after 30 s without a refresh, so a forward clock step shorter than that cannot hand a live writer's lock to another session. A writer that loses its lock anyway fails with ECOMPROMISED before publishing the journal, and the process keeps running.

Acceptance checks

/goal verify <command> approves one command as the goal's acceptance check and runs it now. The goal then completes only on evidence from that check:

Step Behavior
Approval Command safety runs first; a blocked or declined command is not approved. A command that cannot run at all (see limits) is not approved either.
Execution Receipt-bound local bash under the default bash sandbox (OMK_BASH_SANDBOX), with the built-in bash timeout of 5 minutes. Receipts and their replay ledger go to .omk/goals/evidence/<goal key>/.
Pass Exit code 0 and an open strict evidence gate. The receipt is attached as goal evidence, and the workspace state right after the check is recorded.
After each settled turn The approved check runs again. A pass completes the goal. A failure queues the next round with the command and its exit code; the output goes to the operator, never to the model. At the round limit, or when the check cannot run, the goal is blocked with the reason. Esc during the check stops it without continuing.
/goal complete Requires a receipt from this session that passed in the current generation and still matches the workspace.
/goal Shows the approved check and whether a passing receipt still matches the workspace.

The workspace state is the manifest digest of HEAD, the git dirty set and the dirty files' contents, without .omk/goals, .omk/metrics, or .omk/runs. Host-owned state is excluded before the 512-path cap is applied. A tracked edit, a new untracked file or a HEAD move after the check makes the evidence stale, and /goal complete asks for /goal verify again. Each check writes a goal_verification session entry with the receipt ID, digest, status and exit code.

Limits:

  • The approval and the trusted receipts live in the OMK process. The agent can write the workspace, the goal journal and the receipt files, so nothing read back from them approves a command. After a restart, approve the check again with /goal verify <command>. Until then /goal complete uses the reducer rule above.
  • The receipt path accepts static command lines only. Variable expansion and command substitution ($HOME, $(...)) fail closed; put them in a script and verify sh ./check.sh.
  • Outside a git work tree the workspace state binds nothing (unavailable). More than 512 goal-relevant dirty paths are bound partially (partial_truncated). Acceptance-gated completion requires complete scope; unavailable or partial evidence is never promoted to whole-workspace verification.
  • The check proves what the command checks. A weak command completes a goal early.

Seam checkpoints

A durable goal can carry one bounded Goal / Core / Verified / Open / Next seam checkpoint. The checkpoint is another revision in .omk/goals/current.json, not a separate state tree. It is bound to the goal generation and referenced evidence IDs, forced-redacted before persistence, and correlated by a content digest. The digest is unkeyed: it detects accidental mismatch but does not authenticate workspace state against a same-user editor.

Use the interactive command with strict JSON:

/goal checkpoint {"core":["Keep deterministic gates authoritative"],"verified":["focused-tests"],"open":["Historical calibration"],"next":"Run the package checks"}

verified may name only fresh evidence already attached to the current goal generation. A round advance carries the checkpoint record forward but does not make its evidence current-round proof. The built-in controller injects checkpoint prose only when the user recorded it through /goal checkpoint in the current process. A checkpoint loaded from mutable workspace state is noted by digest and its prose is not promoted to user authority. Editing the objective or round limit clears it. The session stores only a goal_checkpoint entry containing the goal ID, revision, and checkpoint digest; complete content remains in the existing durable-goal journal.

Programmatic callers use the record-checkpoint command with applyDurableGoalCommand() or DurableGoalStore.transition(). parseDurableGoalCheckpointCommand() parses the interactive JSON shape; formatDurableGoalCheckpoint() renders the five fields.

Evaluation model

evaluateTask() is a pure TaskSpec + ExecutionAttempt + Observation[] + WaiverRecord[] -> EvaluationResult reducer. It does not mutate its inputs or stored evidence.

An observation condition selects facts by observation kind and task or attempt scope. Its expected facts are a recursive object subset; arrays match exactly.

  • no candidate observation: inconclusive
  • candidate with matching facts: satisfied
  • candidates present but none match: violated
  • all, any, and not compose conditions without adding evaluator state

Required, unwaived violations reduce to fail. Required, unwaived missing observations reduce to inconclusive. Otherwise the semantic verdict is pass. Advisory claims are reported but do not block. A task with no required claims is inconclusive.

reduceRuntimeDecision() then maps the semantic verdict through an explicit runtime policy. pass always stops successfully; fail and inconclusive behavior is supplied as onFail and onInconclusive. Retry and failover counters are not fields: consumers derive them from ExecutionAttempt records.

Advisory best-of-N judge

chooseWithAdvisoryJudge() is an optional selection sidecar, not a protocol verdict producer. It validates each EvaluationResult, admits only candidates whose semantic verdict is pass, and lets a judge score only that eligible set. Zero eligible candidates produce no selection; one skips the model; multiple candidates use weighted 0–4 rubric scores.

Candidate material, the task goal, and rubric descriptions are forced-redacted and bounded before the judge receives them. The model adapter uses a tool-free prompt, treats candidate text as untrusted data, disables retries and cache retention, and accepts only a complete matrix of known candidate and criterion IDs. A provider error, timeout, malformed JSON, unknown ID, or incomplete score matrix selects the existing deterministic first choice and reports a sanitized fallback reason.

Since v0.98.3, the first-party model adapter requires a normal stop, cancellation discards late advice, and top-score ties report their deterministic rank source. Diagnostic counts preserve intake and missingness; they are not independent verification. See Advisory selection integrity.

The sidecar never creates an Observation, changes EvaluationResult.semanticVerdict, supplies independent-verifier evidence, waives a claim, or changes RuntimeDecision. Parsing the evaluation checks structure, not the truth of its evidence. Run fresh tests, integrity checks, security gates, and evidence freshness checks after applying the selected candidate.

Waivers

A waiver names one task and claim, the approver, approval time, rationale, and evidence references. It may be limited to one attempt and may expire. Evaluation fails closed for cross-task, unknown-claim, future-approved, expired, duplicate, or advisory-claim waivers. The underlying claim result remains visible; waiverId records why it did not block.

EvidenceReceipt v3 bridge

EvidenceReceipt v3 remains the integrity layer. evidenceReceiptToObservation() from open-multi-agent-kit validates the immutable core digest, then projects only execution facts into an Observation:

  • receipt schema version and claim text
  • exit code, timeout flag, and abort flag
  • duration and executor
  • a digest-bound receipt reference

The adapter deliberately omits the legacy mutable evidence status. Receipt digest validation does not prove ledger membership, trusted attestation, runner honesty, freshness, or OS isolation; apply those checks separately before trusting the observation.

import { evaluateTask, reduceRuntimeDecision } from "omk-protocol";
import { evidenceReceiptToObservation } from "open-multi-agent-kit";

The legacy TaskContract, EvidenceStatus, TaskContractBuilder.setVerdict(), and updateEvidenceStatus() remain for compatibility and are deprecated. New code should append observations and recompute evaluation.

Authority boundaries

This slice does not change runtime ownership:

  • the coding-agent still owns provider retry and failover execution;
  • AdaptOrch WPL still owns its existing work-packet state machine and adjudication types;
  • scheduler and router separation, topology validation, background-task durability, and algorithm-isolation work remain follow-up migrations.

Those components should consume omk-protocol rather than define new task, attempt, observation, or semantic-verdict types.