Move the wire vocabulary into tinymemory-bus, beneath tinymemory-api

Cargo.toml

@@ -21,6 +21,7 @@ members = ["crates/*"]
 default-members = [
     "crates/tinymemory",
     "crates/tinymemory-api",
+    "crates/tinymemory-bus",
     "crates/tinymemory-conformance",
     "crates/tinymemory-core",
     "crates/tinymemory-remote",

README.md

@@ -23,6 +23,10 @@ crates/
 ├── tinymemory-api/     the contract. Dependency-light on purpose: depending on
 │                       it never drags in SQLite, git2, reqwest, or an async
 │                       runtime
+├── tinymemory-bus/     the wire contract for the loadable module: member names,
+│                       the payload types, and one typed call per member. What a
+│                       *host* links to talk to `tinymemory-module`, which ships
+│                       as a `cdylib` and exports no Rust surface of its own
 ├── tinymemory-core/    the substance: ingestion, the summary tree, chunk
 │                       storage, entities, the graph, the diff ledger, goals,
 │                       tool-memory, and the Composio sync layer. The largest

crates/tinymemory-bus/Cargo.toml

@@ -0,0 +1,62 @@
+[package]
+name = "tinymemory-bus"
+# Not published, for the same reason `tinymemory-api` is not: the graph below it
+# reaches crates that are not on crates.io. A host takes this by git or by path.
+publish = false
+version = "0.1.0"
+edition = "2021"
+rust-version = "1.96"
+license = "MIT"
+repository = "https://github.com/tinyhumansai/tinymemory"
+description = "The TinyBus wire contract for the TinyMemory module: member names, payload types, and typed calls"
+
+# Three dependencies, and the ceiling is low on purpose.
+#
+# This crate is what a *host* compiles against to talk to the loaded module, so
+# it must cost that host almost nothing: no engine, no storage, no async
+# runtime, and — importantly — no `tinybus`. See `src/lib.rs` for why the
+# transport is deliberately absent, and the root manifest's note on
+# `crates/tinymemory-module` for what depending on the vendored `tinybus` from a
+# workspace member would do to this workspace.
+#
+# Nothing here may pull in `rusqlite`, `git2`, `reqwest`, `regex`, an async
+# runtime, or `tinybus`. Guard with the FORWARD form, which is scoped to this
+# package — `cargo tree -i` discards the `-p` scope and exits clean even when
+# this crate is the one pulling the dependency in:
+#
+#   cargo tree -p tinymemory-bus -e normal,build --prefix none \
+#     | grep -Ei 'rusqlite|libsqlite|git2|reqwest|regex|tokio|tinybus'   # expect no match
+[dependencies]
+# The single definition of every type on the wire. Re-exported, never
+# redefined — see `src/types/mod.rs`.
+tinymemory-api = { path = "../tinymemory-api" }
+# The call structs derive both halves: `Serialize` to build an argument array,
+# `Deserialize` so a module-side test can decode one back.
+serde = { version = "1", features = ["derive"] }
+# A tinybus frame body is JSON, so an encoded argument list is a
+# `serde_json::Value` and nothing here needs a different representation.
+serde_json = "1"
+thiserror = "2"
+
+[lints.rust]
+unsafe_code = "forbid"
+missing_docs = "warn"
+missing_debug_implementations = "warn"
+unreachable_pub = "warn"
+rust_2018_idioms = { level = "warn", priority = -1 }
+
+[lints.clippy]
+all = { level = "warn", priority = -1 }
+pedantic = { level = "warn", priority = -1 }
+unwrap_used = "warn"
+expect_used = "warn"
+panic = "warn"
+todo = "warn"
+unimplemented = "warn"
+missing_errors_doc = "warn"
+missing_panics_doc = "warn"
+doc_markdown = "warn"
+
+[lints.rustdoc]
+broken_intra_doc_links = "warn"
+private_intra_doc_links = "warn"

crates/tinymemory-bus/README.md

@@ -0,0 +1,135 @@
+# tinymemory-bus
+
+The wire contract for the TinyMemory `TinyBus` module, as a library a host
+links.
+
+TinyMemory ships as a loadable module so a host does not compile the engine.
+`crates/tinymemory-module` exports one object with 89 members on it, and it
+ships as a `cdylib` — a host can load it, but it cannot `use` anything out of
+it. This crate is what the host compiles against instead:
+
+| module   | what it holds                                                  |
+| -------- | -------------------------------------------------------------- |
+| `names`  | the bus name, the object path, one constant per member          |
+| `types`  | every value type that crosses a frame                           |
+| `calls`  | one struct per member: arguments in wire order, plus reply type |
+| `wire`   | the error names, and the mapping back to `MemoryError`          |
+
+Four dependencies, none of them heavy: `tinymemory-api` for the types, `serde`
+and `serde_json` for the encoding, `thiserror` for one small error enum. No
+engine, no storage, no async runtime — and no `tinybus`.
+
+## Why the types are re-exported, not defined
+
+The obvious reading of "a crate that holds the bus types" is a crate that
+*defines* them. That would be a mistake, and the repository has already made
+the equivalent one once: when `tinymemory-api` was resolved twice, by git and
+by path, `MemoryCategory` from one copy was not the same type as
+`MemoryCategory` from the other, and the mismatch only surfaced at the seam.
+The root `Cargo.toml`'s `[patch]` table exists to prevent exactly that.
+
+Defining structurally identical types here would reproduce it deliberately: the
+module would serve `tinymemory_api::` types, the host would hold
+`tinymemory_bus::` ones, and every call site would need a conversion whose
+correctness nothing checks. So there is one definition, in `tinymemory-api`,
+surfaced here. A host gets the types the module serves — the same types, not
+equivalents.
+
+## Why not just depend on `tinymemory-api`
+
+It would compile. But `tinymemory-api` is the **driver** contract: it also
+carries `MemoryProvider` and its eighteen capability traits, the
+mandatory-family composition, the null driver, and the `host::` config sections
+a host persists in `config.toml`. A host that loads the module implements none
+of that — it makes calls.
+
+This crate is the subset that crosses a frame. What a host compiles against is
+what it can actually send and receive, and a trait method that is not exported
+on the bus is absent here rather than tempting.
+
+## Why arguments get a struct
+
+`#[tinybus::interface]` puts a method's arguments on the wire as a positional
+JSON array, decoded into a tuple on the far side. That is a fine encoding and a
+bad thing to write by hand. `Store` takes six arguments:
+
+```json
+["work", "standup", "…", "core", null, "internal"]
+```
+
+Two are `Option`s, two are enums that serialize as strings, and swapping
+`namespace` with `key` produces a call that succeeds and writes the entry to the
+wrong place. Nothing on the module side can catch it — both are `String`, in
+the right position count, and the engine has no way to know which one the caller
+meant.
+
+So a caller fills in named fields and `BusCall::into_args` does the positioning.
+The reply type travels with the call for the same reason: `Get` answers
+`Option<MemoryEntry>` and `Forget` answers `bool`, both are perfectly good JSON,
+and decoding one as the other fails somewhere far from the call.
+
+## There is no client here
+
+This crate holds no connection and no `call()` that sends anything. Two reasons.
+
+A host already owns its connection — its reconnect policy, its timeouts, its
+tracing, its own idea of what a memory call costs it. A client here would either
+duplicate that or fight it, and the useful part is already in `calls` and
+`types`.
+
+And structurally it could not work anyway: `tinybus` is a vendored submodule
+whose manifest inherits fields from its own nested `[workspace.package]`, so a
+member of this workspace that depends on it makes cargo resolve that inheritance
+against the wrong root and fail. That is why `crates/tinymemory-module` is its
+own workspace root — see the note on `exclude` in the root `Cargo.toml`. A
+contract crate a host links has no business being a separate workspace, so it
+stays transport-free.
+
+Wiring it up host-side is small:
+
+```rust,ignore
+use tinymemory_bus::calls::BusCall;
+use tinymemory_bus::names::{BUS_NAME, OBJECT_PATH};
+use tinymemory_bus::{types::MemoryError, wire};
+
+/// Make one call, and give a failure back as the driver's own error type.
+async fn call<C: BusCall>(
+    connection: &tinybus::Connection,
+    call: C,
+) -> Result<C::Response, MemoryError> {
+    let args = call
+        .into_args()
+        .map_err(|e| MemoryError::Invalid(e.to_string()))?;
+
+    match connection
+        .call(BUS_NAME, OBJECT_PATH, C::METHOD, args)
+        .await
+    {
+        Ok(body) => C::decode_response(body).map_err(|e| MemoryError::Other(e.into())),
+        // The name is the contract; `from_wire` is the same table the module
+        // mapped out through, so the variant survives the round trip.
+        Err(tinybus::Error::MethodFailed { name, message }) => {
+            Err(wire::from_wire(&name, &message))
+        }
+        Err(other) => Err(MemoryError::Other(other.into())),
+    }
+}
+```
+
+`OpenStore` is the one member that needs more than that: it returns an object
+*path*, not a value, and calls against that path use the same `BUS_NAME` and the
+same member names. Treat `OBJECT_PATH` as the root object rather than the only
+one.
+
+## Staying in step with the module
+
+`names::METHODS` lists every member. `crates/tinymemory-module` asserts its
+served members against that list, in order, in
+`the_served_members_are_exactly_the_published_contract`. Nothing else links the
+two — this crate lists members by hand, the module derives them from its
+`#[tinybus::interface]` block — so that test is what turns a drift into a
+`cargo test` failure instead of an `UnknownMethod` in a host at runtime.
+
+Adding a member is therefore three edits in this crate: a constant in
+`names::methods`, an entry in `names::METHODS`, and a call struct in the
+matching `calls` family (which `calls::test::COVERED` also lists).

crates/tinymemory-bus/src/calls/chunks.rs

@@ -0,0 +1,121 @@
+//! The persisted chunk model and its embeddings.
+//!
+//! One [`BusCall`] per member; see [`crate::calls`] for how they are used.
+
+use serde::{Deserialize, Serialize};
+use serde_json::Value;
+
+use crate::calls::BusCall;
+use crate::error::Error;
+use crate::names::methods;
+use crate::types;
+
+/// Arguments for `ListChunks`.
+///
+/// Chunks matching the query, size-checked.
+///
+/// `ChunkQuery::limit` bounds rows, not bytes, and a chunk carries full
+/// content — so this is one of the methods where the ceiling matters most.
+#[derive(Debug, Clone, Serialize, Deserialize)]
+pub struct ListChunks {
+    /// The `query` argument — wire position 0.
+    pub query: types::ChunkQuery,
+    /// The `scope` argument — wire position 1.
+    pub scope: Option<types::SourceScope>,
+}
+
+impl BusCall for ListChunks {
+    const METHOD: &'static str = methods::LIST_CHUNKS;
+
+    type Response = Vec<types::Chunk>;
+
+    fn into_args(self) -> crate::Result<Value> {
+        serde_json::to_value((self.query, self.scope)).map_err(Error::Encode)
+    }
+}
+
+/// Arguments for `GetChunk`.
+///
+/// One chunk, size-checked.
+///
+/// A single object is checked for the same reason a list is: the ceiling is
+/// a property of the frame, not of the row count, and one chunk carries
+/// full content with no bound of its own. A list of one that is refused
+/// while the singular read of the same chunk succeeds would be an odd
+/// contract to explain.
+#[derive(Debug, Clone, Serialize, Deserialize)]
+pub struct GetChunk {
+    /// The `chunk_id` argument — wire position 0.
+    pub chunk_id: String,
+}
+
+impl BusCall for GetChunk {
+    const METHOD: &'static str = methods::GET_CHUNK;
+
+    type Response = Option<types::Chunk>;
+
+    fn into_args(self) -> crate::Result<Value> {
+        serde_json::to_value((self.chunk_id,)).map_err(Error::Encode)
+    }
+}
+
+/// Arguments for `ChunkDetail`.
+///
+/// One chunk plus its metadata, size-checked.
+#[derive(Debug, Clone, Serialize, Deserialize)]
+pub struct ChunkDetail {
+    /// The `chunk_id` argument — wire position 0.
+    pub chunk_id: String,
+}
+
+impl BusCall for ChunkDetail {
+    const METHOD: &'static str = methods::CHUNK_DETAIL;
+
+    type Response = Option<types::ChunkDetail>;
+
+    fn into_args(self) -> crate::Result<Value> {
+        serde_json::to_value((self.chunk_id,)).map_err(Error::Encode)
+    }
+}
+
+/// Arguments for `StorageKinds`.
+///
+/// Takes no arguments, so it encodes as an empty positional array.
+#[derive(Debug, Clone, Serialize, Deserialize)]
+pub struct StorageKinds;
+
+impl BusCall for StorageKinds {
+    const METHOD: &'static str = methods::STORAGE_KINDS;
+
+    type Response = Vec<String>;
+
+    fn into_args(self) -> crate::Result<Value> {
+        Ok(Value::Array(Vec::new()))
+    }
+}
+
+/// Arguments for `ChunkEmbeddings`.
+///
+/// Embedding vectors are the largest thing this interface returns.
+///
+/// A 1536-dimension vector encodes to roughly 10 KiB of JSON, so a few
+/// hundred chunks reach the frame ceiling on their own. Checked for the same
+/// reason `List` is, and refused by name rather than truncated — a short
+/// batch is indistinguishable from "those chunks have no vector".
+#[derive(Debug, Clone, Serialize, Deserialize)]
+pub struct ChunkEmbeddings {
+    /// The `chunk_ids` argument — wire position 0.
+    pub chunk_ids: Vec<String>,
+    /// The `model_signature` argument — wire position 1.
+    pub model_signature: String,
+}
+
+impl BusCall for ChunkEmbeddings {
+    const METHOD: &'static str = methods::CHUNK_EMBEDDINGS;
+
+    type Response = Vec<types::ChunkEmbedding>;
+
+    fn into_args(self) -> crate::Result<Value> {
+        serde_json::to_value((self.chunk_ids, self.model_signature)).map_err(Error::Encode)
+    }
+}