release: v0.2.24 - #199
Merged
Merged
release: v0.2.24#199
Conversation
…s API
Replace the hand-coded `accelerator::model::{gpt4_1, deepseek_*}` builders
with a declarative provider table plus a `<provider>/<model>` resolver.
Adding a new provider is now a one-row table edit. Resource API panics on
missing or unknown active models are replaced with Option/Result so a typo or
a missing credential produces a clear error or hitch fragment instead of a
silent 401 against the wrong endpoint.
- Add `accelerator::provider` with PROVIDERS and MODEL_PRESETS const tables
- Add `resolve_model(spec)` parsing `provider/model`, provider-only, and
auto-detect (first env_var-set provider in priority order)
- Preserve Cost/Limit/Modalities via a separate MODEL_PRESETS table joined at
resolve time; unknown models still resolve, just without preset metadata
- Stash the bare model name in `Model.extra["wire_name"]` so qualified
registry keys (`sii/gpt-4.1`) do not leak onto the wire
- Delete `accelerator::model` (gpt4_1 / deepseek_v4_flash / deepseek_v4_pro)
- Drop hardcoded model registration from `State::default()`; CLI injects
the resolved model via `with_model` after parsing `--model`
- Add `--model <provider>/<model>` CLI flag (optional; auto-detects when omitted)
- Resources::active_model now returns Option<&Model>; Resources::use_model
returns Result<String, ModelNotRegistered>
- completion::complete emits a hitch when no model is active, no panic
- Machine::apply Action::Model translates use_model errors into a hitch
pushed onto the inbox so Policy can recover instead of crashing
- Tests cover all resolve branches (explicit, provider-only, auto-detect,
bare name rejection, unknown provider, missing credential, no credential
at all, unknown model under known provider, error message content) plus
Resources active_model/use_model edge cases
Refs: #27
Refs: #5, #16
* fix(machine): parallel tool execution in reactor via join_all
Replace the serial for-await loop in reactor::react() with a
partition-then-join_all pattern:
Phase 1: Text/hitch fragments → inbox immediately
ToolCall fragments → collect futures (without awaiting)
Phase 2: join_all executes all tool calls concurrently
Phase 3: Push tool_call + result pairs to inbox by index
Before: for frag in fragments { tool.execute().await }
After: join_all(tool_futures).await
Behavioral guarantees preserved:
- Text fragments always reach inbox before tool results
- Each ToolResult is paired with its ToolCall by call_id
- Tool not found → hitch pushed immediately (no spawning)
- No Tool trait changes needed (join_all works with non-'static lifetimes)
Performance: N tools × latency → max(latency) instead of sum(latency)
Closes: #3
* fix(machine): isolate tool panics via tokio::spawn in reactor
Each tool execution is now wrapped inside tokio::spawn. A panic inside
any tool's async block becomes a JoinError instead of unwinding through
join_all and crashing the entire process.
Before: 1 tool panics (e.g. &command[..60] on UTF-8 boundary) →
all tools killed, process exits.
After: 1 tool panics → JoinError caught → hitch pushed to inbox →
other 2 tools complete normally.
This requires cloning tool_arc (already Arc<dyn Tool>) and env (into
Arc<Environment>) before spawning, both cheap operations.
Two test additions:
- one_tool_panic_kills_all_join_all_tools: sync #[should_panic] repro
- spawn_isolates_tool_panic: tokio::spawn with PanicTool fails
isolatedly while healthy tools succeed
Fixes: #83
* fix(python-sdk): decouple destroy() from channel close
RCMClient.destroy() was calling self._channel.close(), which made the
entire client unusable after a single destroy(). Consecutive Open calls
would crash with "Cannot invoke RPC on closed channel".
Fix: destroy() only destroys the machine on the server. New close()
method for explicit channel shutdown.
* docs(examples,readme): add gRPC Python demo and SDK documentation
Adds the Paper Digest gRPC demo (examples/research-assistant/grpc_demo.py)
that mirrors the arxiv_pipeline.rcm graph lifecycle through Python gRPC:
Open → Setup (Append/Model/Activate) → Policy Loop → Destroy
SDK README (sdks/python/README.md) with setup instructions, demo usage,
and API reference table.
README.md updated with gRPC demo section + example table entries for
Python SDK and demo files.
* fix(machine): use first context message as initial prompt, fall back to system role placeholder
send() used Message::user('.') as the initial prompt for rig's
completion_request builder API. rig includes this prompt in the chat
history sent to the LLM. When the LLM saw a bare '.' with user role
after its own assistant reply (no tool call), it interpreted this as a
user accidentally sending a period and started hallucinating responses
like 'looks like you just sent a period', which entered context via
Take and created a perpetual loop.
Fix: use the first actual context message as the initial prompt
(always the real first message in practice). For the degenerate empty
case, fall back to Message::system('_') — system role carries no
conversational intent and won't trigger spurious LLM responses.
Rewrote the remaining-messages slicing from 5-line if/else to 1-line
.get(1..).unwrap_or_default().
* fix(machine): restore .messages() call for context beyond first fragment
PR #86 (commit b0a32fd) split messages into (initial_prompt, remaining)
but the .messages(remaining_messages) call was lost during the merge.
Every LLM call only received the first context fragment — all other
fragments (system prompts, purpose, tool results, assistant replies)
were silently dropped.
This fix: a4e6189
- split_messages() extracted as helper function + 3 inline tests
- .messages(remaining_messages) restored after completion_request()
- Removes inline first()/get(1..) logic in favor of helper
Refs: #87
* refactor(accelerator): remove illegal inline tests, move LSP client test to tests/
- Remove client_key_hashes_root_and_server (tested derive macro on
private helper struct, not real crate API behaviour)
- Move fake_lsp_server_reports_versioned_diagnostics from src/ to
tests/ as lsp_client.rs (mock infrastructure + Python subprocess
dependency, belongs in integration tests)
- Export LspClient and ServerSpec from lsp module for test access
- Make LspClient::start_with_command public for integration testing
* fix(machine): construct CompletionRequest directly to eliminate prompt placeholder and message rotation
`send()` had been using rig's `completion_request(prompt).messages(history)`
builder, where `prompt` is appended to the END of `chat_history` at
`build()` time. Picking which message is the "prompt" produced two
historical regressions:
- #43 family: `Message::user(".")` stub was interpreted by the LLM as
a user reply, causing "looks like you sent a period" hallucination
loops.
- #86 / #88: switching the stub to `messages[0]` rotated the leading
system instruction to the END of the request — `[Sys, User]` became
`[User, Sys]` on the wire.
Switch to direct `CompletionRequest` construction:
- `chat_history: OneOrMany<Message>` is literally what gets serialized.
No prompt slot, no rotation, no fabricated stub.
- `OneOrMany::many(empty)` errors, so the degenerate empty-context case
surfaces as a `Fragment::hitch(role=System)` Result — consistent with
the existing `"no active model set"` path in `complete()`.
- All `CompletionRequest` fields are spelled out explicitly; the absence
of `Default` for that struct turns future field additions into a
compile error rather than a silent behavior change.
The helper `build_request()` is testable in isolation. New unit tests
guard against the regressions above: `build_request_preserves_message_order`,
`build_request_empty_messages_returns_hitch`, plus coverage for
single-message and temperature/max_tokens passthrough.
Replaces the `split_messages` helper and its three placeholder-centric
tests.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* fix(machine): preserve reasoning content from thinking-mode responses and share across parallel tool_calls
DeepSeek and Kimi Coding return `reasoning_content` on assistant turns
in thinking mode and require it to be echoed back on subsequent requests
that include that turn as history. The previous code dropped the
reasoning in `decode()` (the catch-all `_ => {}` arm) and emitted a
`Reasoning::new(".")` stub in `encode()` when `thinking=true`. Kimi
tolerated the stub; DeepSeek did not, producing HTTP 400:
The `reasoning_content` in the thinking mode must be passed back to the API.
End-to-end this caused the paper_search captain loop to die after the
first tool-call turn, repeatedly hitching with status 400.
Fix:
- Add `ToolCall.reasoning: Option<String>` (`#[serde(default)]`, so old
persisted contexts keep deserializing). Captured by `decode`,
re-emitted by `encode`.
- `decode()` now buffers `AssistantContent::Reasoning` blocks and attaches
them to the *next* `AssistantContent::ToolCall`. Critically, the buffer
is **not** cleared on ToolCall — a single reasoning block can be
emitted by the model alongside multiple parallel tool_calls (paper
search produced exactly this shape against DeepSeek), and every
fragment in that turn must carry the reasoning when re-encoded. The
buffer is cleared only on Text or Image, which mark a new logical turn.
- `encode()` Role::Assistant emits `tc.reasoning` verbatim when present.
The `Reasoning::new(".")` stub is retained only as a fallback for
`thinking=true` AND `reasoning.is_none()` (legacy Kimi path for
fragments constructed without a captured reasoning string).
- The `_ => {}` non-exhaustive arm in `decode()` is removed; rig's
`AssistantContent` is now fully matched (Text/Reasoning/ToolCall/Image),
so future variants become a compile error rather than silent drops.
- `Fragment::with_reasoning()` is a no-op on non-ToolCall content; used
by the decoder.
New regression tests in `completion::tests`:
- `decode_attaches_reasoning_to_following_tool_call`
- `decode_concatenates_multi_block_reasoning_before_tool_call`
- `decode_discards_reasoning_before_text_turn`
- `decode_tool_call_without_reasoning_has_none`
- `decode_parallel_tool_calls_share_one_reasoning` — direct regression
for the paper_digest 400 loop observed at 2026-05-27T14:10:36.
Plus in `tests/completion.rs`:
- `encode_tool_call_emits_stored_reasoning_not_placeholder` — captured
reasoning wins over the `.` stub regardless of `thinking` flag.
- `encode_assistant_text_with_thinking_does_not_synthesize_reasoning` —
the stub remains tool-call-only.
Verified against the live demo: `paper_digest.rcm` against
`deepseek-v4-flash` now completes 5 LLM calls (including a 2-parallel
tool_call turn) with zero hitches.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* style(machine): apply rustfmt to completion.rs
CI's rustfmt check flagged four sites that exceeded the line-length
budget:
- `endpoint.completion(...)` timeout block reformatted to a multi-line
match.
- `Fragment::tool_call(...)` arguments collapsed onto one line.
- `assert!(matches!(...))` in `build_request_empty_messages_returns_hitch`
wrapped to a multi-line `assert!`.
- Stray double-blank line in `tests/completion.rs`.
Pure formatting; no behavior change.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* refactor(machine): expose completion::{build_request, decode} and move tests to integration file
AGENTS.md requires tests in `tests/`, not inline `#[cfg(test)] mod`
modules. The 9 tests added in da0bf5f (build_request) and 1a2457b
(decode reasoning) lived inline because they exercised private helpers.
Promote `build_request` and `decode` to `pub`, then move the tests to
`tests/completion.rs`. The two helpers are now part of the crate API,
symmetric with the already-public `encode`. Test helper functions
(`dummy_model`, `assistant_reasoning`, `assistant_tool_call`) move
alongside.
`tests/completion.rs` grows from 12 to 21 tests; `src/completion.rs`
loses its `mod tests` block entirely. No behavior change; all 21 tests
pass against the moved code.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* fix(python-sdk): add termination gate to ReactPolicy to prevent infinite loop
ReactPolicy had no mechanism to stop the agent loop. Halt was always
prioritized over Done, so the agent would call the LLM indefinitely
without ever terminating.
Fix: mirror the Rust Captain's termination heuristic:
- First call -> Halt (LLM hasn't spoken yet)
- Last consumed fragment was tool_result -> Halt (LLM needs to read output)
- Inbox empty + no tool result -> Done (natural termination)
Code cleanup:
- Loop variable 'a' -> 'action' (AGENTS.md compliance)
- _last_was_tool_result avoids list() copy (uses len() + [-1] directly)
Tests: 7 unit tests covering all termination branches
* fix(machine): Resources::lookup returns LookupResult tri-state instead of Option
lookup() previously returned Option<&dyn Tool>, conflating 'tool is
registered but disabled' with 'tool is not registered at all'. The
reactor treated both as 'not found' with the same hitch message.
Fix:
- New LookupResult enum: Active / Inactive / NotFound
- lookup() returns LookupResult (no reference)
- New get() method returns Option<&dyn Tool> for Active path only
- reactor.rs match on LookupResult::Inactive emits a distinct message
('tool is disabled — activate it before use')
- reactor.rs Active branch uses expect() with invariant annotation
Tests (8 total, zero getter tests):
- enable/disable transition tests (behavior)
- lookup_active/Inactive/NotFound implies get returns Some/None (invariant)
- hitch message text verification for Inactive and NotFound
Closes: #16
* Add autoresearch survey scaffold
* Add modular autoresearch survey pipeline
* Constrain autoresearch survey context handoffs
* Tighten autoresearch-survey handoffs and tool grants
Make the survey pipeline's inter-node context smaller and more uniform,
and give each node only the tools it needs.
- Add schema/handoff.md: one minimal handoff envelope (run_dir first,
then artifact/status, plus optional counts/ids/verdict/risks/next,
~15 lines max). All 19 prompts now reference it instead of each
describing an ad-hoc final-message format.
- Harden the run_dir invariant: standalone runs may fall back to the
newest runs/* directory, but must surface that recovery as a risk
rather than switching run_dir silently. handoff.md documents the
fs-list lookup (list runs/ directly; its parent hides it via
gitignore; timestamp names sort chronologically).
- Minimize tools: drop shell from the 13 pure read/write nodes
(scouts keep arxiv_search, mergers/judges/rank/map/brief are fs-only).
shell remains only where genuinely needed: anchor (timestamp/env),
query_plan (mint run_dir), reference_expander (pdftotext/strings).
Verified with accelerate parse on all edited graphs and inventory on
the whole project.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Fix handoff.md path in survey prompts (cwd-relative bug)
The previous commit referenced the new handoff contract as the bare
relative path `schema/handoff.md`. The CLI runs from the repo root, so
the fs tool resolved that to `<root>/schema/handoff.md`, which does not
exist — every node hit `fs error: File not found: './schema/handoff.md'`.
That fs error became an orphan tool fragment which `flux mode=last`
forwarded downstream, so the next node opened its context with a
role=tool message that had no preceding tool_calls, and DeepSeek
rejected the request with HTTP 400.
Use the repo-root path `examples/autoresearch-survey/schema/handoff.md`,
matching how every other schema file is already referenced.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* List available tools when a tool name is not found
When a model calls a tool that does not exist (e.g. `write` instead of
`fs` with action="write"), the reactor pushed a bare "tool 'X' not
found" hitch with no code. The captain policy treats code-less hitches
as transient and retries, but the terse message gave the model nothing
to correct against, so it repeated the same bad call until the retry
budget exhausted and the node halted with no output.
Include the active tool names in the not-found message so the retry is
actionable and the model can self-correct within budget.
Observed in the autoresearch-survey survey_brief node, which is granted
only `fs`: it intermittently called a hallucinated `write` tool and the
run ended before 07_survey_brief.md was written.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Add survey_writer node for the final long-form survey
The pipeline produced only the compact 07_survey_brief.md and stopped,
so a run ended with no readable survey for the user — the brief is an
audit summary by design ("do not write a long survey article").
Add survey_writer as a new terminal node after survey_brief. It reads
the spec, research map, judge panel, and brief from run_dir and projects
them into a section-structured narrative survey (08_survey.md): abstract,
introduction, one section per method family, benchmarks, comparison,
open problems, conclusion, references. It then prints the full survey as
its final message so the user sees the report directly.
The writer is evidence-constrained: it obeys the JudgePanel
forbidden_overclaims, only makes comparisons the BenchmarkJudge marked
ready, never invents citations, and scopes itself honestly on partial
runs. Tools stay fs-only; it can be run standalone against an existing
run_dir.
Wiring: survey_brief.context -> brief_handoff -> survey_writer.context,
survey_brief.done -> survey_writer.trigger, survey_writer -> output.
Docs and the survey_brief prompt updated to reflect the map-then-narrative
two-product design.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* List valid actions when fs gets an unknown action
The fs tool's catch-all returned a bare "unknown action 'find'" with no
hint at the valid set. A model that hallucinated an action (e.g. `find`)
got nothing to correct against, so the captain policy retried the same
bad call until the budget exhausted.
Include the valid actions (read, write, edit, list) in the error, same
remedy as the tool-not-found message, so the retry is actionable.
Observed in the survey_writer node when run standalone: it guessed an
fs action `find` and the run halted with no output.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Make run_dir fallback list runs/ directly in all prompts
When a node runs standalone with no run_dir in context, it must recover
the newest run directory from disk. The prompts said only "fall back to
the newest run directory", so the model listed the parent
(examples/autoresearch-survey), where fs hides runs/ because it is
gitignored — the model never saw the run dirs and flailed into an
invalid fs action.
Spell out the lookup in all 19 prompts: list
examples/autoresearch-survey/runs *directly* (its parent hides it),
take the last timestamped entry, and surface the recovery as a risk.
This matches the rule already documented in schema/handoff.md.
Latent in the 16 chain nodes (full runs always carry run_dir, so the
fallback never fired) and first surfaced by running survey_writer alone.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Drop survey_brief from the end-to-end pipeline
The brief sat between the judge panel and the writer producing a second,
overlapping summary. Wire judge_panel straight into survey_writer and
let the writer read the spec, research map, and judge panel directly.
survey_brief.rcm and its prompt are kept as a standalone unit for runs
that want a compact audit summary instead of the full article; the
prompt no longer assumes a writer runs after it.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Widen retrieval budgets so the pool is larger
Runs produced too few papers and admitted as much in their own scope
limitations. Roughly double the discovery budget end to end: scouts go
to 4-6 queries at topK 10, semantic expansion to 5-8 queries at topK 10,
citation seeds to 6-10, and the query plan requires more queries per
type. rank_pool now deduplicates without pruning for size, so the wider
pool survives into the research map.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Rewrite survey_writer to produce a real narrative survey
The survey read as a classified list of retrieved papers, leaked internal
object names (BenchmarkJudge, RelationGraph, etc.), and was assembled from
the now-removed brief.
Rewrite the writer prompt and survey schema around three demands:
- Self-contained document: written for an external reader, with zero
references to the machinery (no judge names, matrices, graphs, run_dir,
or "pipeline"). State substance, not provenance.
- Narrative, not catalog: an explicit research arc (early approaches,
bottlenecks, why each wave followed, current frontier), each method
family explained by why it exists and how it trades off against the
others, and cross-family synthesis with reasoning. The reader should
finish with a mental model of the field.
- Read from spec, research map, and judge panel directly; write via the
fs tool with action="write"; print the full survey to the user.
Docs updated for the brief-free pipeline and the 08_survey.md artifact.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(accelerator): restore purpose injection dropped in Captain refactor
Captain setup step 2 was a no-op that claimed the purpose was "already injected from outside", but nothing injected it after the Phase abstraction was inlined (49a0b69). Agents therefore received only system scaffolding (captain prompt + AGENTS.md + env) and no task, so they fell into ask-mode and merely greeted instead of executing.
Restore the behavior of the deleted InjectPurpose phase: during setup, append the purpose as a User fragment (tag="purpose"), guarded against re-injection. This fixes the project-maintainer pipelines, whose per-step tasks live entirely in `purpose`. Examples that carry their task in the captain system prompt (autoresearch-survey) never relied on this channel and are unaffected.
Add regression tests for purpose injection and the empty-purpose skip.
* Remove deprecated research-assistant example files
* feat(accelerator): add ContextFlux::Fold mode and context reordering for last/append/digest
Introduce three changes to the context delivery pipeline that together enable
cleaner, more predictable context structures when an accelerator receives
upstream handoff content.
Fold mode (ContextFlux::Fold) extracts the last assistant text from each
upstream slot and stores it in fold_payload on the output State. The downstream
accelerator then merges fold_payload into its purpose before Captain injects
purpose_initial, so upstream information is folded directly into the task
description. No upstream assistant fragments appear in the context — the LLM
sees a single coherent user message.
Context reordering (last/append/digest modes): on the first Halt after Captain
setup, fire() moves non-scaffolding upstream fragments to after the env
fragment and injects a purpose_b fragment at the end. Final order:
scaffolding → upstream content → purpose_b
Captain purpose tag changed from "purpose" to "purpose_initial" to avoid tag
collision with purpose_b injected later.
- crates/accelerator/src/flux.rs: add Fold variant to ContextFlux, extract
last assistant text from each slot during apply()
- crates/accelerator/src/state.rs: add fold_payload: String field
- crates/accelerator/src/graph.rs: propagate fold_payload through context channel
- crates/accelerator/src/accelerator.rs: fire() reordering logic on first Halt
- crates/accelerator/src/policy/captain.rs: tag "purpose" → "purpose_initial"
- crates/accelerator/tests/graph_names.rs: 5 new tests (fold extraction,
single assistant, ignores system, multi-slot, reordering)
- crates/accelerator/tests/captain.rs: update existing tests for purpose_initial
Refs: #97
* Add 5 single-agent RCM examples: folder-concierge, repo-snapshot, paper-scout, cook-tonight, city-halfday
Each example is a one-shot, single-.rcm workflow that collects context
autonomously from the runtime purpose. folder-concierge and repo-snapshot
use only built-in tools (fs/find/git/shell). paper-scout, cook-tonight,
and city-halfday use AnySearch MCP + Fetch MCP.
Structure per example:
README.md, prompts/*.txt, outputs/.gitkeep
* Fix compiler warnings: remove unused trace import, allow dead_code on arxiv deser structs
* Refactor Captain setup: extract agent/instruction/purpose injection into reusable modules
- agent.rs: ensure_agent_prompt — inject system prompt (tag=agent)
- instruction.rs: ensure_instructions — inject AGENTS.md/CLAUDE.md/CONTEXT.md (tag=instruction)
- purpose.rs: ensure_purpose — inject runtime --purpose (tag=purpose), fixing the bug where purpose was silently skipped
* Redesign Captain as typed phase machine with Step{Emit,Ready}
- Phase: Agent → Instruction → Purpose → Resources → Respond → Running
- agent::prepare: normalizes tag=agent to unique context[0]
- instruction::load: injects AGENTS.md/CLAUDE.md/CONTEXT.md once
- purpose::append: append-only, User role, allows multiple instances
- env::refresh: updates env timestamp before each Halt
- resources::activate: activates model + all tools
- Step{Emit,Ready} replaces Action::Done as within-helper signal
- Removed unused first_call, setup_action, halt_after_env
* Refine Captain: clean purpose, separate setup from respond
- purpose::append stores purpose.text directly (no ## Purpose wrapper)
- Captain::setup only handles Agent/Instruction/Purpose/Resources
- Respond and Running are handled directly in decide(), not via setup
- enter(phase) replaces set_phase for cleaner naming
* Extract reusable moves into policy::moves submodule
- agent/env/instruction/purpose/resources moved under policy::moves/
- Captaine references updated to moves::agent/instruction/purpose/resources/env
- policy::moves created from scratch (new mod.rs)
* Fix MCP HTTP connection reuse: disable idle pool
Disables reqwest HTTP/2 idle connection pooling for MCP HTTP
transport via pool_max_idle_per_host(0). Previous behaviour
reused the compile-stage idle connection for tools/call, but
the server-side (AnySearch via CloudFront) sends GoAway before
the tool call arrives, resulting in a silent 120s timeout.
This ensures each MCP HTTP request opens a fresh connection,
avoiding the stale-connection problem entirely.
* Strip rig meta-crate dependency: use rig-core directly instead
- Replace rig meta-crate (16 unused sub-crates) with rig-core + minimal features
- Upgrade accelerator's reqwest 0.12 → 0.13 to eliminate dual-version build
- Cargo.lock shrinks from 7323 to 914 lines (~87% reduction in resolved deps)
* Remove fetch MCP from paper-scout, cook-tonight, city-halfday
- Deleted mcp fetch block and mcps reference from all three .rcm files
- Updated prompts to reference AnySearch only
- Updated READMEs to remove Fetch MCP setup and description
* Fix phase order and context display semantics
- Insert Environment phase between Instruction and Purpose, ensuring
system env fragment is injected before the user purpose fragment.
Result: [system][system][system] then [user] then [assistant/tool_call].
- Update text.rs print_fragment to use content-type labels instead of
role labels: ToolCall → [tool_call], ToolResult → [tool_result].
Context output now reads [tool_call] → arxiv_search {...} and
[tool_result] (45 lines) instead of [assistant] and [tool].
* Add built-in webfetch tool: fetch URL → readable text via scraper
- Single dependency: scraper 0.27 (html5ever-based HTML parser)
- Fetches URL via existing reqwest client
- Extracts readable text via scraper::Html text nodes (strips HTML chrome)
- Extracts <title> and prepends as heading
- Respects max_length parameter (default 5K, max 100K)
- 1MB body limit, 30s timeout
- Graceful fallback for non-HTML content types
* Move webfetch tests from inline to tests/webfetch.rs
* Add dependency hygiene tooling and update development guide
- Install cargo-nextest, cargo-machete, cargo-deny
- Add deny.toml with license allowlist, advisory policy, banned TLS stacks
- Remove unused tracing-subscriber from accelerator (library crate rule)
- Add publish = false to all workspace crates
- Add version fields to internal path dependencies
- Fix nextest process-per-test isolation issue in compile.rs temp file naming
- Document known machete false positives (prost, tonic-build) via metadata
- Update AGENTS.md: add Dependency Hygiene section, add nextest rule,
remove old auto-commit constraint in favor of atomic commits
* Upgrade webfetch: browser-grade headers, rate limiting, HTTP/2 + Brotli
- Browser-like User-Agent (Chrome 131 on macOS)
- Full browser header set: Accept, Accept-Language, Accept-Encoding (br/gzip/deflate),
Cache-Control, Sec-Fetch-{Dest,Mode,Site,User}, Upgrade-Insecure-Requests
- Domain-based rate limiting (≥2s between requests to same host)
- reqwest features: http2, brotli, gzip, deflate, cookies
- Static LazyLock client built once with all default headers
* fix: resolve clippy warnings across workspace
- captain.rs: remove never_loop (all branches return), collapse nested if-lets
- flux.rs: replace needless_range_loop with enums, collapse nested if-lets
- All other warnings (result_large_err, option_map_unit_fn, type_complexity,
collapsible_if, useless_format, unnecessary_filter_map) are pre-existing
in the codebase and not addressed here
clippy --workspace --all-targets: 0 errors
* Fix clippy warnings, add rustfmt.toml, upgrade CI to nextest + deny + machete
- Fix clippy error: remove dead 'loop' wrapper in captain.rs decide()
- Add #[allow] annotations for intentionally accepted lints:
result_large_err (tonic::Status, Fragment as error type),
type_complexity (boxed future type), large_enum_variant (AcceleratorBody)
- Add impl Default for MachineManager
- Add rustfmt.toml with project formatting conventions
- Set clippy -- -D warnings in CI
- Switch CI test step to cargo-nextest + cargo test --doc (doctests)
- Add CI dependency hygiene job: cargo-machete + cargo-deny
* Add process-safe test rule and clippy allow discipline to AGENTS.md
* Audit-driven rewrite: SSRF protection, streaming, safe truncation, script/style filtering
- SSRF: validate_url rejects non-http/https schemes, private IP ranges (10.x,
127.x, 192.168.x, 169.254.x, etc.), loopback hostnames, and .local/.internal
TLDs. Redirect policy re-validates every redirect target.
- Streaming body read: bytes_stream() checks MAX_BYTES incrementally instead
of buffering the entire response before checking size.
- Safe Unicode truncation: char_indices() avoids byte-split panics on non-ASCII
pages.
- Script/style filtering: recursive ElementRef walk skips script/style/noscript/
iframe subtrees. Block elements (p, h1-h6, li, pre, etc.) insert newlines for
readable paragraph structure.
- Mutex: std::sync::Mutex replaced with tokio::sync::Mutex to avoid blocking
async runtime.
- Rate limiter eviction: map trimmed when it exceeds 1000 entries (oldest >60s
entries removed).
- Title selector static: LazyLock avoids unwrap() on every call.
- Description corrected: no longer claims to strip navigation/sidebars.
* Add general-purpose example with webfetch, arxiv, AnySearch
- general.rcm with tools: fs, find, git, shell, webfetch, arxiv_search
- MCP: AnySearch for web search
- Prompt: one-shot, no follow-up questions, writes to outputs/
- README with run examples for different scenarios
* Remove tool enumeration from general prompt — tools auto-activate
* Simplify general prompt: remove output path and summary requirement
* Reject known binary content types before downloading body
- Added is_binary() check for image/, audio/, video/, application/pdf,
application/zip, application/gzip, application/x-tar, etc.
- Returns structured result 'binary content (image/png), not readable as text'
instead of dumping raw binary bytes into model context
* Fix clippy warnings: collapsible-if in webfetch, dead code in graph_names
* Drive survey topic from purpose instead of an env var
The anchor node read the topic from the AUTORESEARCH_TOPIC environment
variable. Now that purpose flows through the graph as a first-class
channel, the topic is just the graph's purpose.
- Wire input.purpose -> anchor.purpose so the CLI --purpose flag (or the
graph's declared purpose) reaches the entry node, which the captain
injects into context as a purpose-tagged message.
- anchor.rcm declares a default topic purpose; --purpose overrides it.
- Rewrite the anchor prompt to take the topic from the injected purpose,
falling back to topic.md only when no purpose is present. Drop the
"do not rely on the RCM purpose field" instruction and the env lookup.
- Remove AUTORESEARCH_TOPIC from rcm.toml; document --purpose in README.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Add image_gen built-in tool (text-to-image via OpenAI images API)
Generates an image from a prompt and writes it to disk, returning the
saved path. Calls the OpenAI images API (gpt-image-2) using OPENAI_API_KEY
from the environment, decodes the b64 result, and writes a PNG under the
given filePath (typically inside run_dir). Modeled on the webfetch and
arxiv_download tools; registered as a built-in so any .rcm node can grant it.
Enables pipelines to produce a generated figure as a real artifact — e.g.
a survey's opening global picture — rather than inline data.
* Add image_planner node for the survey's opening global picture
Insert image_planner between judge_panel and survey_writer:
judge_panel -> image_planner -> survey_writer -> output.
image_planner reads the research map and judge panel, composes a
self-contained "research landscape" image prompt, and calls the image_gen
tool to render run_dir/08_global_picture.png. It is granted only fs and
image_gen. If image_gen fails (e.g. no OPENAI_API_KEY) it reports blocked
so the run still completes.
survey_writer now opens the article with that figure when present —
embedded right after the abstract with an orienting sentence — and omits
it cleanly when absent (no broken link). Schema and docs updated; README
notes OPENAI_API_KEY is optional and the figure is skipped without it.
* Drop the topic.md fallback; topic comes only from purpose
The anchor node had three topic sources: the injected purpose, a
topic.md file, and (earlier) an env var. With the topic now driven by
the graph's purpose — set via --purpose or anchor.rcm's default — the
topic.md layer is redundant.
Remove the topic.md fallback from the anchor prompt, the README input
section, and .gitignore, and delete the unused topic.md.example. The
input.purpose -> anchor.purpose wire is kept: it is the only path for a
top-level --purpose to reach the entry node in this composite graph.
* Stop instruction search at the nearest AGENTS.md; add example guide
The captain's instruction loader walked from cwd to the filesystem root
and collected every AGENTS.md / CLAUDE.md / CONTEXT.md it found. Running
the survey from the repo root injected the host repo's development guide
(Rust conventions, git workflow, test rules) into every survey node —
irrelevant context that pollutes a literature-research task.
Stop at the nearest directory that has instruction files: a local guide
overrides ancestors instead of stacking on top of them. Add an
AGENTS.md under examples/autoresearch-survey so the survey nodes get
research-task guidance and no longer inherit the repo's dev guide.
Run the survey with cwd at examples/autoresearch-survey to pick up the
local guide; user-global config (~/.synergy, ~/.claude) is unaffected.
* Add fragment tag and action hook events for richer tape animation
- Emit tag field on appended/inserted/replaced/taken hook events
- Emit model/activate/deactivate hook events (were silent before)
- Add after field to inserted event for insert-position tracking
* Redesign tape animation: tag-aware colors, long-jump pointer, fixed-viewport layout
- Map cell colors from kind + tag + role with dedicated scaffolding/purpose hues
- Long-distance pointer jumps in at most 6 frames instead of per-cell walking
- Insert/swap/remove/replace ops resolve by fragment id (survives intermediate inserts)
- Fixed viewport with overflow hidden tape count instead of dynamic row expansion
- Expose snapshot_events() for mock testing without a real terminal
- Add cell tone and glyph to snapshot for assertion coverage
* Use example-relative paths in prompts; run from the example dir
Running from the example directory (to pick up the local AGENTS.md)
broke every fs path: the prompts hardcoded repo-root paths like
examples/autoresearch-survey/schema/X, which fs resolved against the new
cwd as examples/autoresearch-survey/examples/autoresearch-survey/schema/X
— not found. Each node had to fail, list ., and guess the real path.
Make all prompt paths relative to the example directory: schema/X,
runs/<ts>, etc. The .rcm `file "../prompts/..."` refs are compile-time
and unaffected. handoff.md's run_dir example and the README run commands
now use the example dir as cwd. Also drop the now-dead topic.md ignore.
Run with `cd examples/autoresearch-survey` first.
* Add a Chinese edition of the survey
Add survey_writer_zh after survey_writer: it reads the finished English
08_survey.md and writes a faithful Chinese translation to
08_survey.zh.md. It is a downstream leaf — the graph's printed output
stays the English survey; the Chinese edition lands on disk alongside it.
The translator preserves structure, citations, identifiers, and the
global-picture image link; it translates prose and headings into academic
Chinese without re-researching or adding claims. Tools are fs-only.
Wiring: survey_writer.context -> survey_handoff -> survey_writer_zh,
triggered by survey_writer.done. Docs and the index manifest updated.
* Carry run_dir in the handoff so the global picture lands in run_dir
End-to-end runs generated the global picture but wrote it to a nested
path (examples/autoresearch-survey/examples/autoresearch-survey/runs/.../
08_global_picture.png), so survey_writer never found it and the survey
had no figure.
Root cause: nodes pass context with flux mode=last, which forwards only
the last fragment. When a node's last message is a tool receipt or text
without run_dir, the next node gets no run_dir and falls back to scanning
runs/ — every node was falling back. image_planner then built the image
path from a re-prefixed run_dir and wrote outside the run.
Fix the contract so run_dir actually flows: handoff.md now requires the
handoff to be the node's LAST message (after all writes, no trailing tool
call), with run_dir as the first line, verbatim and un-prefixed.
image_planner is told to write exactly <run_dir>/08_global_picture.png
without re-prefixing. image_gen and the apicz endpoint themselves work —
the generated file is a valid 1536x1024 PNG.
* Point image_gen at the apicz endpoint; tidy formatting
Switch the image generation endpoint to the apicz gateway used for
end-to-end testing, drop the repo-root example from the filePath
description (it invited the nested-path mistake; use a relative
runs/<ts>/... example and say not to add prefixes), and apply rustfmt.
The endpoint and model stay hard-coded for now; making the provider
configurable remains follow-up work.
* Tape animation: in-place scaffolding flash, wider tag palette, per-action verbs
Three refinements on top of the tape redesign:
- Scaffolding (env/agent/instruction/purpose) re-emits now flash in place
via a new TapeOp::Flash that does not move the main pointer. The per-round
env timestamp tick no longer drags the pointer back across long tapes.
Classification is by CellKind::is_scaffolding().
- tag_color palette widened 8 -> 16 hues so distinct tags read as distinct,
stable colors; non-special tags remain hash-mapped (Tagged).
- Each action has its own intermediate state/verb: Take settles through a
CellState::Taking "intake" (distinct from a plain append), Flash uses
CellState::Flashing "refresh", and model/activate/deactivate surface a
transient status-row badge (held BADGE_TICKS ticks). Swap keeps its ◆ hold.
Adds a sticky `last_action` to TapeState/TapeSnapshotTape so the transient
verbs are observable to tests after the animation drains. 4 new snapshot
tests cover flash-without-pointer-move, distinct tag tones, intake, and the
resource-action verb. All terminal-free via snapshot_events.
* Tape animation polish: slower flash, solid head trail, red for errors only
Minor tweaks on the existing crawling-head model (deeper redesign deferred):
- Scaffolding flash now breathes for FLASH_TICKS ticks instead of a single
one-frame blink, so the per-round env refresh is readable in place.
- The head fills its trail: any pending cell behind the pointer is committed
as it moves, so a long jump no longer leaves a "filled/hollow/filled"
patchwork of skipped cells.
- Red is reserved for errors (hitch). Removal now fades in dim grey rather
than reusing the error colour.
Tests: scaffolding flash holds the pointer across many env refreshes (not
just one); no hollow cell remains behind the head after a long run; hitch is
the only error-toned cell. 9 tape tests pass; fmt + clippy clean.
Note: the underlying ping-pong/lag is inherent to the single traveling head
and is not fixed here by design choice — a fuller redesign (separate
scaffolding strip, reality-first cells) remains the longer-term option.
* fix(maintainer): post comments via quoted heredoc + verify they land
The PR-review / issue-triage / mention pipelines posted comments with inline
`gh ... -b "<body>"`. The shell tool runs `sh -c`, so any backtick / $ / quote
in a review body — code identifiers in markdown, i.e. almost always — was
interpreted by the shell, mangling the comment or executing embedded commands.
The respond nodes only carry the shell tool, so they had no body-file fallback.
- Templates (pr_review, issue_triage, mention_handler): post every comment and
the code_change PR body via a quoted heredoc piped to `--body-file -`. The
quoted delimiter disables all expansion, so backticks/$/quotes go verbatim.
Proven at the sh level: a body with `cmd`, $(...), $HOME and quotes is sent
literally with nothing executed.
- Templates: after posting, re-read the latest comment to confirm it landed,
retry once on failure, and report status: ok+comment_url or status: blocked —
never a silent fake success.
- maintainer.yml: snapshot the bot-comment count before the run and fail the
job if pr_review / issue_triage finishes without it rising (model skipped the
post, or a fork PR's read-only GITHUB_TOKEN returned 403). Turns the previous
silent green no-op into a visible failure.
* refactor(maintainer): split pr_review into meta/diff/intent nodes with budgeted diff
The PR review was one analyze node fed the entire `gh pr diff`. digest doesn't
truncate tool results, so a large diff overflowed the model context, and a single
node conflated "is the code sound?" with "does the PR meet its stated goal?".
Restructure pr_review.rcm.tpl into a finer graph:
meta -> diff_review -> intent_check -> respond
- meta: pull PR metadata + per-file size inventory (the diff budget input).
- diff_review (shell+fs): pull the diff only within a ~1500-line budget; over
budget, fetch only the riskiest files' patches and read any file on demand via
fs to ground each hunk. The diff no longer has to fit in context — the node
retrieves detail agentically instead of being firehosed the whole thing.
- intent_check (shell+fs): judge whether the PR's stated goal holds and is
actually achieved; flag scope drift and missing companion changes.
- respond: fold both into one comment, posted via the quoted-heredoc +
self-verify path from the previous commit.
maintainer.yml: after rendering (templates come from base), check out the PR head
for pull_request events so diff_review/intent_check read the PR's code. Files are
only read, never built or executed, so PR code on disk is not run; the rendered
.rcm in the gitignored .rcm-cache survives the switch. Internal/branch PRs only.
* feat(accelerator): add a `map` fan-out node (runtime sub-graph over the existing GraphRun)
The graph engine could only express a static DAG (flux modes Append/Last/Digest/
Thread/Fold) — no way to fan a node out over a runtime-sized list (per-paper or
per-section work).
A `map` is a single node whose children are added dynamically: at run time it
scatters its input into k items and **builds an inner graph — k seeded worker
clones → a k-arity merge Flux → output — then runs it with the ordinary
`GraphRun`**. Externally it is one node; internally it is `input → [(B1..Bk) →
merge] → output`. This reuses the whole graph layer — `Graph`, `GraphRun` (incl.
its per-frontier concurrency, so the k workers run in parallel for free), the
`Flux` merge, wiring, validation — instead of a bespoke executor.
Engine (crates/accelerator):
- AcceleratorBody::Map(MapAccelerator { inner, scatter, gather }); run() builds the
inner graph and calls Graph::run_seeded.
- Graph::run_seeded(input, seeds): seed each dynamically-added worker's input by
component id, then run via GraphRun (boundary wiring still honored).
- ScatterSpec::Json: parse the input's last assistant message as a JSON array;
each element becomes one worker, injected as an `item`-tagged fragment on a
clone of the shared context. Non-array/empty → one worker (never hard-fails).
- GatherSpec::Digest maps to a k-arity ContextFlux::Digest merge.
- map_start / map_done hooks; per-worker progress shows via the inner graph's
own component hooks.
DSL (crates/cli/src/rcm): `map NAME { accelerator = Alias; scatter = json;
gather = digest }` compiles to an accelerator-kind component, wired like any node.
Tests: scatter units; a model-free runtime fan-out test (3 elements → a 3-worker
inner graph → merge) via a Done policy; parser tests; and an examples/map-smoke
demo whose stream shows the nested inner graph.
* feat(autoresearch-survey): full-text, fan-out pipeline via the map primitive
Rebuild the writing half of the survey pipeline so depth comes from reading full
text (not abstracts) and from per-section expansion, using the new `map` fan-out
primitive twice.
New stages (rank_pool → … unchanged up to here):
- card_plan: pick up to ~30 papers worth reading in full (prefers abstract-only
core/method papers), emit a JSON work list.
- paper_cards (map): per paper, download the PDF, read the full text *through the
research anchor's lens*, and write a compact card to run_dir/cards/<id>.md —
problem/method/results-with-caveats, anchor-relevance, and a cross-domain
transfer note. Replaces abstract-only paper notes.
- research_map now builds from those full-text cards.
- survey_outline (was survey_writer): design the macro skeleton + an explicit
through-line, emit a JSON section list.
- section_expand (map): per section, write a detailed, card-grounded section to
run_dir/sections/<n>_<slug>.md.
- survey_assembler: stitch the sections, embed the figure, and build one deduped
reference list → 08_survey.md + index.md. survey_writer_zh translates as before.
Each map's upstream emits a JSON array (one element per item, carrying run_dir);
scatter=json fans the inner accelerator out per element, gather=digest rolls the
per-item handoffs up. Per-item artifacts live on disk, so no node holds the whole
corpus or the whole survey.
Adds schema/paper_card.md and schema/section.md; new prompts + rcm nodes; rewires
autoresearch_survey.rcm; updates README/CONTEXT_FLOW and the example-compile test.
survey_writer.{rcm,txt} are superseded by outline/expand/assembler.
Requires the `map` engine primitive (separate PR) — and a new accelerate release
before the example runs end-to-end off the published binary.
* feat(autoresearch-survey): cross-domain transfer as a first-class lane
Cross-domain transfer used to live only in the per-paper card's `transfer` note —
an afterthought over papers that happened to be in-pool. Move it to retrieval time
and carry it through the pipeline as a distinct lane (option B).
Expansion gains a third parallel branch, `cross_domain_expander`: it abstracts the
methods we already found into domain-agnostic patterns, searches OTHER fields for
work that shares those patterns (neighbouring fields first), filters to genuine
cross-domain candidates with a transfer hypothesis, and writes 03d_cross_domain.md
with role: cross_domain. expansion_merge becomes arity 4.
The role is carried through so it actually surfaces instead of being washed out by
scope discipline:
- rank_pool: a separate `transfer_set` ranked by `transfer_potential`; low
topic_fit is expected and never a reason to drop, never mixed into in-domain sets.
- scope_judge: treats the transfer lane as intentional inspiration, not leakage —
ideas may be discussed as hypotheses, but a cross-domain result is never reported
as in-field.
- research_map: a `cross_domain_transfer` section (pattern + transfer hypothesis).
- survey_outline: reserves a dedicated cross-domain section when the lane is
non-empty.
- card_plan: also reads a few transfer candidates in full, so the transfer
hypothesis is grounded in full text, not abstracts.
Schemas (expansion/ranked_pool/research_map), prompts, the expansion graph, and
the README/CONTEXT_FLOW docs updated accordingly. Compiles (example-compile test +
full-pipeline compile-check).
* fix(autoresearch-survey): card-plan budget in schema (100) + assembler index timing
- Add schema/card_plan.md defining the contract, with the budget as a visible
constant max_fulltext_papers = 100 (raised from the prompt-buried 30). card_plan
now reads the schema and references the constant instead of a hardcoded number,
so the cap lives at the contract layer.
- survey_assembler index.md no longer hard-links 08_survey.zh.md as if it exists:
it runs before the downstream translator, so the manifest now lists the Chinese
edition with a note that it is written by the downstream translation step and
may not exist yet — no dead link presented as live.
* add storage crate: import WAL engine from Axiom project
Copy segment-based WAL with CRC32C integrity, sparse indexing, and
crash-safe atomic writes into RCM as the foundation for optional
session persistence. Rename crate from 'wal' to 'storage', register
in workspace members.
93 tests passed, 0 skipped.
* refactor(catalog): unified registration layer for tools, models, prompts, policies, environments, and MCP servers
- Catalog fields are now private with typed register_*() APIs and duplicate name
validation
- tools/models/prompts/mcps all belong to ResourceCatalog; stale resources/
preset module removed
- MCP server definition is decoupled from activation: register_mcp_server()
only saves config, resources_for() starts selected MCPs on demand
- .rcm and gRPC semantics unified: top-level definitions register into
Catalog, accelerator selection picks by name only
- Rust external extension: register_tool/register_policy/register_environment
on Catalog, with_catalog() on RcmService, compile_file_with_catalog()
- Proto OpenRequest split into model_definitions/mcp_definitions (define) and
models/mcps (select)
* cleanup(catalog): remove dead gRPC policy field, cache MCP tools, drop unstable external web tests
- Remove OpenRequest.policy from proto and server — gRPC has no policy
execution path; keep it in .rcm/CLI only
- Rename Catalog::resources_for to build_runtime_resources to clarify the
method starts MCP servers as a side effect
- Cache selected MCP tools in Catalog via OnceCell so the same server is
started at most once rather than per-accelerator
- Make ResourceCatalog private (it had no public methods)
- Rename ResourceSelection.prompts to prompt_texts and drop the implicit
'inject all registered prompts' behaviour — only explicitly selected
prompts appear in Resources
- Remove three unstable external-network tests (example.com, httpbin.org)
that fail when offline
* refactor(policy, sdk): extract Rust ReAct move, align Python SDK with definitions/selections proto, replace stale examples
- Extract Captain's running-phase ReAct logic into policy/moves/react.rs;
Captain now only handles setup phase and delegates running to react::decide()
- Regenerate Python protobuf stubs from updated proto
- Update RCMClient.open() to {model,mcp}_definitions + {models,mcps,tools}
selection semantics, with keyword-only keyword arguments
- Update Python ReactPolicy: support user/system→Halt, tool_result→Halt,
hitch retry, idle→Done
- Delete three stale examples (arxiv_research, weather, system_info)
- Add echo_agent.py: minimal Python controller, prompt-only agent
- Add math_mcp_server.py + mcp_math.py: local stdio MCP server with
add/multiply tools, Python controller drives the loop
- Update README and __init__.py for new SDK shape
* refactor(machine, storage): serialize machine runtime, split ToolRuntime from Resources, implement Store with full action replay
- Resources now holds only serializable state: ToolDefinition (name/desc/
params) instead of Arc<dyn Tool>, plus models/prompts/active_tools.
All fields derive Serialize/Deserialize.
- ToolRuntime added as separate non-serializable container for executable
Arc<dyn Tool> instances, passed explicitly to Machine::apply().
- Action, Context, Inbox, Usage derive Serialize/Deserialize.
- Machine::apply() returns ApplyResult { done, event } where event is a
MachineEvent (action + outcome) that can be recorded to WAL.
- Halt records its reactor output (fragments + usage) in the event, enabling
deterministic replay without re-calling LLM/tool during restore.
- storage::Store: high-level API on top of Wal that records/checkpoints/
restores MachineState (context + environment + resources + inbox + step +
done). restore() loads latest checkpoint then replays subsequent events.
- 5 new Store tests: empty restore, context actions, resource actions, halt
output, checkpoint + incremental replay.
- Remove unused SegmentReader::find and clean up dead with_tool/replace_tools
methods.
* cleanup(machine, storage): clarify tool definitions vs executors, record non-Halt inbox outcomes
- Rename Resources.tools to tool_definitions so serializable tool schemas are
not confused with executable tool handles
- Rename ToolRuntime internal map to executors and remove unused with_tool()
- Centralize Catalog's dual registration of ToolDefinition + ToolRuntime
executor in add_tool_to_runtime_resources()
- Replace ActionOutcome::StateOnly with ActionOutcome::State { inbox } so
failed non-Halt actions can replay recorded inbox side effects
- Remove unused MachineEvent::reactor(), Store state_from_parts(),
inbox_from_fragments(), and Resources replace_* helpers
- Add Store coverage for replaying failed-action hitch output
Verification: cargo check && cargo nextest run (400 passed)
* test(accelerator): adapt map tests to explicit ToolRuntime
* fix(ci): group machine runtime args and satisfy clippy
* feat(accelerator): fs read extracts PDF text via a safe pdftotext wrapper
Previously `fs read` refused PDFs ("require external extraction"), so paper_card
fell back to the raw `shell` tool running `pdftotext` — handing the model an
arbitrary shell. Replace that with a safe, encapsulated path.
- New tools/fs/pdf.rs: extract_pdf_text() invokes `pdftotext` with parameterized
args via Command (never a shell, so the path is never interpreted), and as
defense in depth rejects any path with a shell metacharacter, requires a .pdf
extension, and bounds runtime (60s) and output (500 KB).
- fs read now routes .pdf through that wrapper and returns the extracted text
(offset/limit paginated like any text file); other document types still error.
- paper_card.rcm drops `shell` (tools = arxiv_download, fs); paper_card.txt just
does `fs read <pdf>` and falls back to abstract_only if extraction fails.
The model no longer needs raw shell to read papers. Unit-tested the metacharacter
guard; verified real extraction with poppler's pdftotext on a downloaded PDF.
(reference_expander still uses shell for citation scraping — left for a separate
change.)
* refactor(autoresearch-survey): reference_expander reads PDFs via fs, not shell
Migrate the second raw-shell PDF user to the safe path. reference_expander now
reads each downloaded PDF with `fs read` (which extracts text via the safe
pdftotext wrapper) and finds the References section in that text, instead of
shelling out to `pdftotext`/`strings`. Drops `shell` from its tools
(arxiv_download, arxiv_search, fs).
Both PDF-reading nodes (paper_card, reference_expander) are now off raw shell.
anchor / query_plan still use shell for run_dir/timestamp creation — a separate
concern, left untouched.
* feat(accelerator): add FluxMode::Bridge for cross-channel data transfer, remove fold_payload
Replace ContextFlux::Fold (which used a side-channel fold_payload on State)
with FluxMode::Bridge — a first-class cross-channel transfer mechanism that
reads from one channel, transforms data, and writes to another.
- Add BridgeKind::ContextLastTextToPurpose: extracts last assistant text from
context slots and writes to purpose
- Split channel() into input_channel() / output_channel() so Bridge can
have different input and output channels
- Delete State::fold_payload — no more side-channel
- merge_input() now concatenates wired + base purpose when both are present
- Update compile.rs, ast.rs, parser.rs for dual-channel ComponentTag and
bridge syntax (channel=bridge, from=context, to=purpose, mode=last_text)
- Rewrite 4 Fold tests as Bridge tests
180 tests pass, clippy clean.
* refactor(accelerator): make Bridge purely mechanical, move extraction to upstream Flux
BridgeKind::ContextToPurpose replaces ContextLastTextToPurpose.
Bridge no longer applies any extraction/filtering logic — it purely
flattens all text fragments from context into a purpose string.
Upstream ContextFlux (Last/Append/Digest/Thread) controls what
fragments reach the Bridge.
RCM syntax: channel=bridge from=context to=purpose mode=flatten
Tests now demonstrate composition: Flux(Last|Append|Digest) → Bridge
rather than Bridge acting standalone.
* chore(accelerator): remove redundant doc comments on flux variants and apply helpers
Variant names are self-explanatory (Append, Last, Digest, Thread, Bridge).
Helper functions are clear from their signatures. No behavioral changes.
* fix(map): file-based scatter so fan-out no longer depends on a chat JSON message
A full run fanned both maps out to a single item: the planners (card_plan,
survey_outline) ended with a handoff, not a bare JSON array, and ContextFlux::Last
forwards only the final fragment — so scatter saw no array and fell back to 1.
Result: sections/ and cards/ each had one file and the survey was a stub. Models
write files reliably (the outline was perfect) but don't reliably end with bare
JSON in chat, so move the work list to disk.
Engine:
- ScatterSpec::File(name): recover run_dir from the incoming handoff, read
<run_dir>/<name>, parse the JSON array; any failure falls back to one item.
- ScatterSpec is now Clone (not Copy); scatter takes &ScatterSpec.
- run_dir_from_context() parses the last `run_dir:` line from the handoff.
- Tests: run_dir recovery, file-scatter fallbacks, and an end-to-end (no-model)
test that a 3-element JSON file fans out to 3 workers.
DSL: `map { scatter = file "<name>" }` (ast scatter_file + parser + compile +
parse test).
Pipeline:
- card_plan writes run_dir/00_card_plan.json + handoff; paper_cards scatter=file.
- survey_outline writes run_dir/00_sections.json + handoff; section_expand
scatter=file. Schemas updated to the file contract.
- survey_assembler: write 08_survey.md + index.md then STOP (it was looping on
re-list/re-read); use fs list (not the nonexistent `stat`) for the figure check.
Pipeline review: every node now takes run_dir via the handoff and reads its inputs
from disk; the maps fan out via a disk file. No node depends on parsing structured
data out of a chat message anymore.
* refactor(machine,storage): eliminate apply_event dual-logic via single apply with replay mode
Replace the separate apply_event implementation in Store with a single
Machine::apply that supports both live execution and replay. This
eliminates the dual state-transition logic that was already diverging
(Take missing fragment_ids, Halt missing usages in replay).
Key changes:
- Move usages/counts from Machine into MachineState for lossless resume
- Introduce ApplyContext struct (ctx, env, resources, inbox, usages, counts)
- Introduce ApplyMode enum: Live { tool_runtime } | Replay { cached_outcome }
- Machine::apply takes ApplyContext + ApplyMode instead of scattered params
- Machine::apply_state wraps apply for convenient MachineState usage
- Delete apply_event and recover_state_outcome from store.rs entirely
- Store::restore now calls Machine::apply_state in ApplyMode::Replay
- Store::restore is now async (calls Machine::apply which is async)
- Add Action::is_done() helper for replay validation
- Update all callers: accelerator, server, tests
- Add tokio to storage dev-dependencies for async store tests
- Add halt_output_replays_through_inbox_and_take test verifying usages survive
* fix(ci): disable cancel-in-progress for maintainer workflow
Long-running review runs (e.g. kimi-k2-6 diff_review taking 15+ minutes
with 17 tool-call rounds) were being killed whenever a new push or comment
landed on the same PR, because cancel-in-progress: true immediately
terminates the in-flight run. Switching to false makes runs queue instead,
so a review that already started always finishes.
Refs: run 72341150628 on PR #106
* fix: cap frontier concurrency + per-model timeout (map fan-out + long zh)
A 100-paper run exposed two issues a full run hit:
1. paper_cards fanned out ~100 workers in one frontier with no bound — each doing
arxiv_download + full-text read + LLM. Under that load most workers never
finished, so only 1 card was written (the survey fell back to abstracts).
Fix: GraphRun runs each frontier in bounded waves (FRONTIER_CONCURRENCY = 6).
Sequential pipelines and the small parallel scout/judge frontiers are under the
cap and unaffected; only wide `map` fan-outs are throttled. Worker rendezvous
is not a thing here (workers are independent), so no deadlock.
2. survey_writer_zh timed out (6× at the 180s default) translating the full
~650-line survey in one generation, then exhausted retries — no zh output.
Fix: add a per-model `timeout` field to the model DSL (ast + parser + compile)
and set the zh model to 600s. The field is general — any node can tune it.
Verified: full accelerator + cli suites, fmt, clippy. Concurrency cap covered by
the existing barrier-based graph tests (they rendezvous within the cap and still
pass); timeout parse/compile covered by the example-compile test (zh uses it).
* fix(cli): validate bridge from/to channels at compile time
flux_mode_from_def accepted any from/to channel pair but always
produced BridgeKind::ContextToPurpose, silently ignoring mismatched
directions (e.g. from=purpose to=environment would compile but produce
wrong output at runtime). Now only context→purpose is accepted; other
combinations are rejected with a clear error message.
* feat(autoresearch-survey): raise fan-out concurrency + framework-first zh translation
- Raise FRONTIER_CONCURRENCY 6 -> 16: paper_cards (and any map) read in parallel
with healthy width, each paper still its own worker (dynamic fan-out), without
the 100-wide overload that starved workers.
- Replace the single-shot zh translator with a framework-first, per-section chain
(the map again):
- zh_frame: translate title/abstract and build a glossary — a "keep in English"
list (Transformer, ImageNet, Forward-Forward, BP, arXiv ids, code idents…) and
agreed Chinese renderings for translatable terms.
- zh_sections (map over 00_sections.json): translate each section in parallel,
obeying the glossary so terminology doesn't drift, into sections_zh/<n>_<slug>.md.
- zh_assemble: copy-stitch the Chinese sections into 08_survey.zh.md (no
re-translation -> a fast, bounded step that won't time out).
This fixes both the one-shot 180s timeout (sections are bounded) and the
"over-Chinese" drift (shared glossary keeps proper nouns in English, reads more
fluently). survey_writer_zh.rcm stays as a standalone single-shot unit.
Docs + example-compile test updated; full suites / fmt / clippy green.
* feat(autoresearch-survey): prefer arXiv HTML over pdftotext for paper cards
pdftotext extracts text from PDFs with known issues: double-column
text reordered, formulas lost, no section structure preserved.
arXiv renders most papers as HTML (arxiv.org/html/<id>) which
preserves headings, paragraphs, and tables — far more readable for
LLM consumption.
Changes:
- paper_card.rcm: add webfetch to the tool list
- paper_card.txt: try webfetch of the HTML version first (max_length
200K), fall back to arxiv_download + fs read (pdftotext) only when
HTML is unavailable; add note about LaTeX residue in formulas
- paper_card.md: add html evidence level, update rules to cover both
HTML and PDF fallback paths
* fix(accelerator): sanitize scatter_file path to prevent traversal
scatter_file constructed the file path by naively joining
cwd + run_dir + name without canonicalizing. Since run_dir comes
from model-generated handoff text and name from .rcm config,
a value like ../../etc could escape the working directory.
Use resolve_path (shared with the fs tool) to normalize the path,
then verify the result stays under cwd. On escape, fall back to a
single item — consistent with all other scatter failure modes.
* feat(tools): add SpawnTool — LLM-driven fan-out via tool calls
Replace the implicit MapAccelerator (ScatterSpec/File/Gather) with a
SpawnTool that planner LLMs call explicitly. This eliminates the
engine-level fan-out infrastructure (MapAccelerator, scatter_file,
run_dir recovery, ScatterSpec, GatherSpec, map DSL node) and replaces
it with a clean Tool boundary — the fan-out list arrives as a
structured tool parameter, workers run concurrently with bounded
parallelism, and the LLM receives a summary it can use to retry
failed items.
Changes:
- SpawnTool (crates/accelerator/src/tools/spawn.rs): new Tool that
takes items[] + max_parallel, runs one worker accelerator per item,
returns per-item outcome summary. Uses Futur…
Expose content-free request-shape and provider classifications. Redact HTTP response bodies before failures enter machine state. Add the 0.2.21 release contract and incident guardrails.
Preserve whether RCM model declarations explicitly enable or disable thinking. Emit the OpenAI-compatible request extension only for explicit OpenAI declarations.
Contributor
Author
|
Review 结论:可合并。 本次发布 PR 的目标(v0.2.24 快照,核心为 DeepSeek provider-native transport 修复)已经达成。 代码质量
目标 / 范围
修改建议
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This promotion PR publishes the complete v0.2.24 release snapshot, including the generated Homebrew Formula. It is created and merged only by the release workflow after required checks pass.