diff --git a/AGENTS.md b/AGENTS.md index 3cef8465b6..7b8f4ff558 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -412,7 +412,23 @@ Per-domain Cargo features drop whole domains **at compile time** (smaller binary | Set | Where it lives | What it is | | --- | --- | --- | -| **Contributor** | `[features] default` in `Cargo.toml` | What a bare `cargo check`, `cargo test` and rust-analyzer compile. 9 cheap gates. **353 packages / 3 native builds** (`libsqlite3-sys`, `lzma-sys`, `ring`). | +| **Contributor** | `[features] default` in `Cargo.toml` | What a bare `cargo check`, `cargo test` and rust-analyzer compile. 10 cheap gates. **~353 packages / 3 native builds** (`libsqlite3-sys`, `lzma-sys`, `ring`). | + +> **`modules` is in `default`, and it is the one gate here that is not optional.** +> The table below has documented it as Contrib=ON since it landed and +> `scripts/ci/product-features.txt` has always listed it, but it was missing from +> `[features] default` — so a bare `cargo test --lib -- memory::` failed **26** +> tests (582 passed / 26 failed), every one a "null vs module" assertion, because +> `memory::binding::module_provider` took its `#[cfg(not(feature = "modules"))]` +> arm and bound `NullMemoryProvider`. A further 15 module-gated tests did not +> exist at all. With the gate on: **623 passed, 0 failed.** A default set that +> cannot run its own test suite is not an inner loop, so this one stays. +> It is also the cheapest gate in the list — **+9 packages / +5 unique names** +> (`ureq`, `ureq-proto`, `utf8-zero`, `toml_edit`, `toml_write`) and **zero** new +> native builds; the native list is identical with it on and off. Nothing like +> the cohorts that motivated splitting `default` from the product set. It does +> **not** move the kernel floor — that profile is `--no-default-features +> --features flows` and never reads this list. | **Product** | `scripts/ci/product-features.txt` | What the shipped desktop app has. 16 gates. **540 packages / 7 native builds** (adds `bzip2-sys`, `libgit2-sys`, `libz-sys`, `zstd-sys`). | `default` used to be the product set, which made the inner loop pay for the whole product on every edit — web3's ethers/secp256k1 cohort, `documents`' zstd/bzip2 native builds (since removed from the graph entirely — the codecs run in a module now), the cpal/hound/arboard/enigo/rdev stack behind `voice`+`inference`, `contacts`' macOS objc2 cohort, `crash-reporting`'s sentry tree, `tui`'s ratatui. Those are default-OFF now. **This did not change what ships**: the shell has set `default-features = false` since #1061 and never inherited `default` anyway. @@ -660,6 +676,80 @@ always-on kernel surface, so `features = ["modules"]` there puts a loader plus one — 305 → 308 packages, which the kernel-floor ratchet caught. It is forwarded from this crate's own `modules` feature instead. +#### The memory seam — one contract, two live paths (#5560) + +Memory is the second module consumer, and it is **half migrated**. Read this +before touching `src/openhuman/memory/`. + +**The contract is `tinymemory-api`, and `crate::openhuman::memory::api` is a +re-export of it — not a copy.** `3ee5a3cad` inlined that crate as 10,894 lines +under `src/openhuman/memory/api/`, every file byte-identical to +`vendor/tinymemory/api/src/` apart from doc-comment paths. Nothing behaved +differently, which is what made it worth undoing: the contract is the vocabulary +the host, `ModuleMemoryProvider`, and the separately compiled module all speak, +and the module compiles against the **crate**. A verbatim copy made the host's +`MemoryError`, `Chunk`, `Capabilities` and `MemoryProvider` distinct types from +the ones on the wire. `api::wire` is where that bit hardest — its own docs, and +`modules/memory.rs`, both justify sharing the error table because +reimplementing it "is what would let a `PathEscape` arrive as an `Invalid`" — +and while the host held a private copy of that table the sentence described an +intention rather than the build. `memory/api.rs` is a short `pub use` now; +`memory/api_identity_tests.rs` pins the identity with type equalities, so a +re-inlining fails to compile rather than passing silently. + +**`memory::api` is the contract surface, not an alias for the crate.** It +exports only what actually crosses the bus, derived from both directions — +outbound from `modules/memory.rs`, inbound from `modules/memory_host.rs`. Whole +namespaces where the namespace *is* wire vocabulary (`capabilities`, `chunks`, +`error`, `goals`, `health`, `provider` with its `provider::types` payloads, +`recall`, `tool_memory`, `tree`, `types`, `wire`), plus `CONTRACT_VERSION` for +version negotiation. Three exclusions are deliberate and each has a reason: + +- **`host`** is re-exported as **two types, not the namespace** — only + `MemoryEvent` and `SpacyResponse` cross the bus. The rest of + `tinymemory_api::host` is the *in-process engine-embedding* seam (the + persisted `MemoryConfig` sections, `MemoryHostConfig`, `EmbeddingProvider`, + `MemoryEventSink`), which the host hands to `tinymemory-core` directly and + which never touches a module. +- **`null`** is the fallback driver `memory::binding` installs when no module is + available — what runs when nothing crosses the bus, so the opposite of + contract. Name `tinymemory_api::null` at the call site. +- **`traits`**, **`version`** and **`is_compatible`** had zero uses in `src/`; + they were alias surface only. + +That is the point of the split: `tinymemory-api` is *also* the crate this host +embeds the engine through, and "the module contract" and "the host's own use of +the crate" are different surfaces. Reaching the second one by naming +`tinymemory_api::` directly keeps the difference visible in the source rather +than in someone's memory. **Do not widen `memory::api` back out to the whole +crate** — if a new path needs something not exported there, the question to +answer first is whether it crosses the bus. + +**`tinymemory-api` stays; `tinymemory-core` has not left yet.** The API crate is +the host-owned contract and is meant to be a dependency. The *engine* crate is +still linked (1.44 MB of `.text`) because ~71 lines across 38 production files +name `tinymemory_core::` directly, and ~687 more paths reach it through the +twenty-five module re-exports in `memory/mod.rs`. `memory/direct_engine_refs_tests.rs` +is the ratchet over the first number, with every file classified as a re-export +shim, a host-seam installation, or a call that needs a wider bus surface. + +**Most of what remains is blocked upstream, not here.** `modules::registry` pins +the TinyMemory module to a released, SHA-256-verified artifact, so a new bus +method is a `tinymemory` release plus a registry re-pin before it is a host +change. Adding a `MemoryProvider` method without that produces a driver that +answers `Unsupported` — strictly worse than the direct call, because the failure +moves from compile time to run time. The concrete gaps (retrieval filters, chunk +reads, an entity-kind filter, source listing, the people domain, and the +`source_scope` task-local) are enumerated in that lint's module docs. + +**One trap worth naming: `tinymemory-api` and `tinycortex-api` are two crates +with near-identical types.** The engine's +`tinymemory_core::store::chunks::types::SourceKind` resolves to +`tinycortex_api::chunks::SourceKind`, which is **not** the contract's +`memory::api::chunks::SourceKind`. Swapping one import for the other looks like +a free type carve-out and is a type error — the module does that conversion at +its own boundary. + #### The `tui` gate The tabbed terminal UI (`openhuman`, or explicitly `openhuman tui` / alias `chat`) lives in `src/openhuman/tui/` and follows the **`mcp`/`voice` facade+stub** pattern: `pub mod tui;` is always compiled; the behavioural submodules (`app`, `render`, `state`, `terminal`, `runner`) are `#[cfg(feature = "tui")]`; and `#[cfg(not(feature = "tui"))] mod stub;` re-exposes the one symbol an always-compiled caller reaches — `run_from_cli` — with a build-fact error body (`"tui feature disabled at compile time … --features tui"`). Bare-command auto-launch requires terminal stdin/stdout and `HostKind::Cli`; Docker, CI, pipes, and `--no-tui` retain the non-TUI CLI path. diff --git a/Cargo.toml b/Cargo.toml index cd8bb7852d..e607f974d6 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -635,7 +635,22 @@ proptest = "1" # gated-off domains, so every CI lane that builds or tests "the product" now # passes `--features "$(scripts/ci/product-features.sh)"`. If you add a lane, # decide which of the two sets it is testing and say so. -default = ["media", "skills", "flows", "mcp", "channels", "medulla", "http-server", "scheduler-gate", "file-logging"] +# `modules` is in this list because the memory seam is not optional at test +# time. `memory::binding::module_provider` has a `#[cfg(not(feature = +# "modules"))]` arm that binds `NullMemoryProvider`, so without the gate a bare +# `cargo test --lib -- memory::` fails 26 tests, every one of them a "null vs +# module" assertion, and 15 further module-gated tests do not exist at all +# (582 passed / 26 failed, versus 623 passed / 0 failed with the gate on). +# A default set that cannot run its own test suite is not a usable inner loop. +# The cost is the smallest of any gate here: +9 packages / +5 unique names +# (`ureq`, `ureq-proto`, `utf8-zero`, `toml_edit`, `toml_write`) and **zero** +# new native builds — the native list is byte-identical with the gate on and +# off. That is nothing like the cohorts #4919 moved out of `default`, and it +# also brings the build in line with what AGENTS.md has always documented +# (`modules`: Contrib=ON, Product=ON) and with `scripts/ci/product-features.txt`, +# which already lists it. This does NOT move the kernel floor: that profile is +# `--no-default-features --features flows` and never reads this list. +default = ["media", "skills", "flows", "mcp", "channels", "medulla", "http-server", "scheduler-gate", "file-logging", "modules"] # HTTP + Socket.IO server transport (#5048): the `/rpc` JSON-RPC endpoint and # its auth middleware/CORS layer (`core::jsonrpc`, `core::auth`), the `/v1` # OpenAI-compatible router (`inference::http`), the ad-hoc static-dir file diff --git a/src/openhuman/memory/api.rs b/src/openhuman/memory/api.rs new file mode 100644 index 0000000000..10039faa09 --- /dev/null +++ b/src/openhuman/memory/api.rs @@ -0,0 +1,90 @@ +//! The **tinybus module contract** for memory — the vocabulary that crosses the +//! bus between this host and the separately compiled TinyMemory module, named +//! as a re-export of the [`tinymemory_api`] crate and never as a copy of it. +//! +//! # Why this file is nine lines and not ten thousand +//! +//! This module used to be a directory holding 10,894 lines that were, file for +//! file, byte-identical to `vendor/tinymemory/api/src/` apart from the paths +//! written inside doc comments. Commit `3ee5a3cad` ("run tiny domains as +//! TinyBus modules") put them there. +//! +//! Nothing behaved differently the day that landed, which is exactly what made +//! it worth undoing. The contract is the vocabulary **three** parties speak: +//! host call sites, [`crate::openhuman::modules::memory::ModuleMemoryProvider`] +//! serialising onto the bus, and the separately compiled TinyMemory module on +//! the far end — and that module compiles against the crate. A verbatim copy +//! made the host's `MemoryError`, `Chunk`, `Capabilities` and `MemoryProvider` +//! *distinct types* from the ones on the wire, kept in step by nothing but +//! whoever remembered to edit both. +//! +//! [`wire`] is where that mattered most. Its module docs, and +//! `modules/memory.rs`, both say the error table is shared by both ends +//! precisely because reimplementing it "is what would let a `PathEscape` arrive +//! as an `Invalid`, silently reclassifying a sandbox escape as a caller +//! mistake". While the host held its own copy of that table the sentence +//! described an intention rather than the build. +//! +//! # What this module exports, and what it deliberately does not +//! +//! It is the contract surface, **not a convenience alias for the crate**. The +//! set below is derived from what actually crosses the bus, in both directions: +//! outbound from `modules/memory.rs`, where `ModuleMemoryProvider` serialises +//! each capability family onto the wire, and inbound from +//! `modules/memory_host.rs`, the host callbacks the module calls back into. +//! +//! The distinction is the point. `tinymemory-api` is also the crate this host +//! embeds the memory *engine* through, and those two roles are not the same +//! surface. Anything the host uses for its own purposes — engine config +//! sections, the driver-fallback provider, trait scaffolding — is reached by +//! naming [`tinymemory_api`] directly at the call site, so that "the module +//! contract" and "the host's own use of the crate" are told apart in the +//! source rather than in someone's memory. +//! +//! Excluded, with the reason: +//! +//! - **`host`** is re-exported as **two types, not the namespace.** It is the +//! engine-embedding seam — `MemoryConfig` and the other persisted config +//! sections, `EmbeddingProvider`, `MemoryHostConfig`, `MemoryEventSink` — +//! which the host hands to `tinymemory-core` in-process and which never +//! touch the bus. Only [`host::MemoryEvent`] and [`host::SpacyResponse`] +//! cross it, inbound, and only those two are exported here. +//! - **`null`** is `NullMemoryProvider`, the fallback bound by +//! `memory::binding` when no module driver is available. It is what runs +//! when nothing crosses the bus, which makes it the opposite of contract. +//! - **`traits`**, **`version`** (as a module path) and **`is_compatible`** had +//! no use anywhere in `src/`; they were alias surface only. Version +//! negotiation is done through [`CONTRACT_VERSION`], which `memory::binding` +//! and `memory::ops::provider` compare against the driver's reported +//! contract, so that constant stays. +//! +//! Everything else is re-exported as a whole namespace on purpose: each of +//! `capabilities`, `chunks`, `error`, `goals`, `health`, `provider` (with its +//! `provider::types` payload vocabulary), `recall`, `tool_memory`, `tree`, +//! `types` and `wire` is wire vocabulary end to end — `provider` alone is the +//! fourteen capability-family traits plus eleven payload types, and naming +//! those twenty-five individually would restate the crate's own module +//! structure without narrowing anything. +//! +//! `memory/api_identity_tests.rs` pins the survivors with type identities, so a +//! future re-inlining fails to compile instead of passing silently. +//! +//! Prefer naming [`tinymemory_api`] directly in new code. These paths exist so +//! the several hundred existing `memory::api::…` call sites did not have to +//! move in the same change that removed the duplicate. + +pub use tinymemory_api::{ + capabilities, chunks, error, goals, health, provider, recall, tool_memory, tree, types, wire, + CONTRACT_VERSION, +}; + +/// The inbound half of the seam: the only two `tinymemory_api::host` types that +/// cross the bus, rather than the whole engine-embedding namespace. +/// +/// `modules/memory_host.rs` serves both — [`MemoryEvent`] is what the module +/// publishes back into the host's event bus, and [`SpacyResponse`] answers the +/// module's NLP callback. The rest of `tinymemory_api::host` is the in-process +/// engine seam and must be named on the crate. +pub mod host { + pub use tinymemory_api::host::{MemoryEvent, SpacyResponse}; +} diff --git a/src/openhuman/memory/api/capabilities.rs b/src/openhuman/memory/api/capabilities.rs deleted file mode 100644 index 4dd3aef7c9..0000000000 --- a/src/openhuman/memory/api/capabilities.rs +++ /dev/null @@ -1,397 +0,0 @@ -//! Capability families a memory driver may advertise, and the set type used to -//! negotiate them. -//! -//! ## Why capabilities exist -//! -//! A memory driver is not required to implement the whole surface. The kernel -//! asks a driver which families it supports **once**, at bind time, caches the -//! answer, and then unregisters the RPC methods and omits the agent tools that -//! belong to an unadvertised family. Absence beats a registered handler that -//! returns "not implemented": a present-but-failing method teaches a model that -//! the capability exists and makes it retry. -//! -//! Calling an unadvertised capability is therefore a *kernel* bug, not a driver -//! error. [`crate::openhuman::memory::api::error::MemoryError::Unsupported`] exists for the one case the -//! kernel cannot pre-empt: an out-of-process driver that answers `501` for a -//! family its handshake claimed. -//! -//! ## Mandatory families -//! -//! [`Capability::Core`], [`Capability::Recall`], and [`Capability::Portability`] -//! are mandatory. Without core and recall a driver is not a memory backend at -//! all; without portability a user cannot leave it, which makes the binding a -//! one-way door. [`Capabilities::validate`] is the single place that rule is -//! encoded — call it at bind time and refuse the bind on `Err`. -//! -//! ## Wire stability -//! -//! The set crosses the process boundary in the driver handshake -//! (`POST /v1/handshake` → `{ contract_version, driver_id, capabilities[] }`), -//! so the serialized form is a JSON **array of stable snake_case strings**, not -//! discriminant integers — inserting a variant in the middle of the enum must -//! not silently re-map an already-deployed driver's advertised set. -//! [`Capability::as_str`] is the authority for those strings and is pinned -//! against the serde derive by a test. -//! -//! ## Deliberately not `#[non_exhaustive]` -//! -//! Adding a family is a [`crate::openhuman::memory::api::CONTRACT_VERSION`] **minor** bump and should -//! break every exhaustive `match` in every host that filters registration by -//! family — that compile error is the mechanism which guarantees the new family -//! is actually wired somewhere. Marking this enum `#[non_exhaustive]` would -//! convert that compile-time guarantee into a silent fall-through at the crate -//! boundary (the failure mode recorded for `DataSource` during the M0 -//! carve-out). If a future family must be added without breaking downstream -//! matches, bump the **major** version instead. - -use serde::{Deserialize, Serialize}; -use thiserror::Error; - -use crate::openhuman::memory::api::error::MemoryError; - -/// One capability family a memory driver may advertise. -/// -/// The variants are exactly the thirteen families of the memory contract. Each -/// maps to a trait family in the contract, a group of RPC methods, and a group -/// of agent tools; a driver that does not advertise a family simply has that -/// surface absent. -#[derive(Clone, Copy, Debug, Eq, PartialEq, Ord, PartialOrd, Hash, Serialize, Deserialize)] -#[serde(rename_all = "snake_case")] -pub enum Capability { - /// Store / get / forget / list / namespaces. **Mandatory.** - Core, - /// Ranked retrieval for a query. **Mandatory.** - Recall, - /// Document and chat ingestion — the driver owns chunking and embedding. - Ingest, - /// The namespace-document tier: put / get / query documents. - Documents, - /// Summary-tree query, drill-down, seal, and cascade. - Tree, - /// Entity index, entity edges, and hotness. - Entities, - /// Key/value graph read and write. - Graph, - /// Snapshot capture and change computation. - Diff, - /// Goal extraction and goal records. - Goals, - /// Per-tool learned memory. - ToolMemory, - /// Accepting synced source items; the host still owns credentials and - /// scheduling. - Sources, - /// Re-embed, compact, consolidate ("dream"), and doctor. - Maintenance, - /// Export and import of the whole store as a stream. **Mandatory.** - Portability, -} - -impl Capability { - /// Every family, in declaration order. - /// - /// Declaration order is also bit order in [`Capabilities`] and iteration - /// order in its serialized form, so this slice is the single ordering - /// authority for the whole module. - pub const ALL: [Capability; 13] = [ - Capability::Core, - Capability::Recall, - Capability::Ingest, - Capability::Documents, - Capability::Tree, - Capability::Entities, - Capability::Graph, - Capability::Diff, - Capability::Goals, - Capability::ToolMemory, - Capability::Sources, - Capability::Maintenance, - Capability::Portability, - ]; - - /// The families a driver must advertise to be bindable at all. - /// - /// See the module docs for why these three and not others. - pub const MANDATORY: [Capability; 3] = [ - Capability::Core, - Capability::Recall, - Capability::Portability, - ]; - - /// Every family, in declaration order. Slice form of [`Self::ALL`], for - /// callers that want to iterate without naming the array length. - pub fn all() -> &'static [Capability] { - &Self::ALL - } - - /// Stable snake_case identifier used on the wire, in config, and in logs. - /// - /// This is the authority for the serialized form; the serde derive is - /// pinned against it by `capability_as_str_matches_serde_representation`. - /// Changing a string here is a breaking change for every already-deployed - /// driver and requires a [`crate::openhuman::memory::api::CONTRACT_VERSION`] major bump. - pub fn as_str(self) -> &'static str { - match self { - Self::Core => "core", - Self::Recall => "recall", - Self::Ingest => "ingest", - Self::Documents => "documents", - Self::Tree => "tree", - Self::Entities => "entities", - Self::Graph => "graph", - Self::Diff => "diff", - Self::Goals => "goals", - Self::ToolMemory => "tool_memory", - Self::Sources => "sources", - Self::Maintenance => "maintenance", - Self::Portability => "portability", - } - } - - /// Parse back from the on-wire form. - /// - /// # Errors - /// - /// Returns the unrecognised input in an error message. An unknown string is - /// expected in practice: a driver speaking a newer minor contract version - /// may advertise a family this build has never heard of. Callers - /// negotiating a handshake should **skip** unknown families rather than - /// fail the bind — an unknown family is one this kernel would never call. - pub fn parse(raw: &str) -> Result { - Self::ALL - .iter() - .copied() - .find(|cap| cap.as_str() == raw) - .ok_or_else(|| format!("unknown memory capability: {raw}")) - } - - /// Whether this family is mandatory for every driver. - pub fn is_mandatory(self) -> bool { - Self::MANDATORY.contains(&self) - } - - /// Position of this family in [`Self::ALL`]; also its bit index in - /// [`Capabilities`]. - fn index(self) -> u16 { - match self { - Self::Core => 0, - Self::Recall => 1, - Self::Ingest => 2, - Self::Documents => 3, - Self::Tree => 4, - Self::Entities => 5, - Self::Graph => 6, - Self::Diff => 7, - Self::Goals => 8, - Self::ToolMemory => 9, - Self::Sources => 10, - Self::Maintenance => 11, - Self::Portability => 12, - } - } - - /// Single-bit mask for this family within a [`Capabilities`] set. - fn bit(self) -> u64 { - 1u64 << self.index() - } -} - -impl std::fmt::Display for Capability { - fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { - f.write_str(self.as_str()) - } -} - -impl std::str::FromStr for Capability { - type Err = String; - - fn from_str(raw: &str) -> Result { - Self::parse(raw) - } -} - -/// A driver's advertised capability set. -/// -/// Internally a bitset, so `contains` is a single mask test on the hot path and -/// the type is `Copy`. Externally it serializes as a JSON array of -/// [`Capability::as_str`] strings in [`Capability::ALL`] order — duplicates in -/// the input collapse, and ordering in the input is not preserved, because a -/// set has neither. -#[derive(Clone, Copy, Debug, Default, PartialEq, Eq, Hash)] -pub struct Capabilities { - bits: u64, -} - -impl Capabilities { - /// The empty default capability set. The `null` driver advertises - /// [`Self::mandatory`] via its [`MemoryProvider::capabilities`](crate::openhuman::memory::api::provider::MemoryProvider::capabilities) - /// implementation, not this. - pub const fn empty() -> Self { - Self { bits: 0 } - } - - /// Every family. Advertised by the embedded `tinycortex` driver. - pub fn all() -> Self { - Capability::ALL.into_iter().collect() - } - - /// Exactly the mandatory families — the minimum bindable set. - pub fn mandatory() -> Self { - Capability::MANDATORY.into_iter().collect() - } - - /// Whether `capability` is advertised. - pub fn contains(&self, capability: Capability) -> bool { - self.bits & capability.bit() != 0 - } - - /// Whether every family in `other` is advertised here. - pub fn contains_all(&self, other: Capabilities) -> bool { - self.bits & other.bits == other.bits - } - - /// Adds `capability` in place. Idempotent. - pub fn insert(&mut self, capability: Capability) { - self.bits |= capability.bit(); - } - - /// Removes `capability` in place. Idempotent. - pub fn remove(&mut self, capability: Capability) { - self.bits &= !capability.bit(); - } - - /// Builder form of [`Self::insert`]. - pub fn with(mut self, capability: Capability) -> Self { - self.insert(capability); - self - } - - /// Builder form of [`Self::remove`]. - pub fn without(mut self, capability: Capability) -> Self { - self.remove(capability); - self - } - - /// Advertised families in [`Capability::ALL`] order. - pub fn iter(&self) -> impl Iterator + '_ { - Capability::ALL - .into_iter() - .filter(move |cap| self.contains(*cap)) - } - - /// Number of advertised families. - pub fn len(&self) -> usize { - self.bits.count_ones() as usize - } - - /// Whether no family is advertised. - pub fn is_empty(&self) -> bool { - self.bits == 0 - } - - /// Mandatory families this set is missing, in [`Capability::ALL`] order. - /// Empty when the set is bindable. - pub fn missing_mandatory(&self) -> Vec { - Capability::MANDATORY - .into_iter() - .filter(|cap| !self.contains(*cap)) - .collect() - } - - /// Rejects a set that is missing any mandatory family. - /// - /// Call this at bind time; on `Err` refuse the bind and fall back to the - /// embedded default rather than binding a driver a user could not leave. - /// - /// # Errors - /// - /// Returns [`MissingMandatoryCapabilities`] listing **every** missing - /// mandatory family, not just the first, so the operator sees the whole gap - /// in one message. - pub fn validate(&self) -> Result<(), MissingMandatoryCapabilities> { - let missing = self.missing_mandatory(); - if missing.is_empty() { - Ok(()) - } else { - Err(MissingMandatoryCapabilities { missing }) - } - } -} - -impl FromIterator for Capabilities { - fn from_iter>(iter: I) -> Self { - let mut set = Self::empty(); - for capability in iter { - set.insert(capability); - } - set - } -} - -impl Extend for Capabilities { - fn extend>(&mut self, iter: I) { - for capability in iter { - self.insert(capability); - } - } -} - -impl Serialize for Capabilities { - fn serialize(&self, serializer: S) -> Result - where - S: serde::Serializer, - { - serializer.collect_seq(self.iter()) - } -} - -impl<'de> Deserialize<'de> for Capabilities { - /// Skips any family string this build does not recognise, rather than - /// failing the whole deserialize. - /// - /// A remote driver speaking a newer minor contract version may advertise a - /// family this build has never heard of — see [`Capability::parse`] and the - /// module-level "wire stability" docs. Rejecting the whole handshake on one - /// unknown string would refuse an otherwise-compatible driver; the correct - /// behaviour is to drop the family this kernel could never call anyway. - fn deserialize(deserializer: D) -> Result - where - D: serde::Deserializer<'de>, - { - let raw = Vec::::deserialize(deserializer)?; - let families = raw - .into_iter() - .filter_map(|family| Capability::parse(&family).ok()); - Ok(families.collect()) - } -} - -/// A driver advertised a capability set missing at least one mandatory family. -/// -/// Carries the missing families rather than a formatted string so the caller -/// can report them structurally (status RPC, bind-failure event) as well as in -/// a log line. -#[derive(Debug, Clone, PartialEq, Eq, Error)] -#[error( - "memory driver advertises an incomplete capability set; missing mandatory families: {}", - .missing.iter().map(|c| c.as_str()).collect::>().join(", ") -)] -pub struct MissingMandatoryCapabilities { - /// Mandatory families absent from the advertised set, in - /// [`Capability::ALL`] order. Never empty. - pub missing: Vec, -} - -impl From for MemoryError { - /// An incomplete advertised set is a caller/config error, not an - /// unsupported call: the driver said something invalid about itself, which - /// is why this maps to [`MemoryError::Invalid`] and not - /// [`MemoryError::Unsupported`]. - fn from(value: MissingMandatoryCapabilities) -> Self { - MemoryError::Invalid(value.to_string()) - } -} - -#[cfg(test)] -#[path = "capabilities_tests.rs"] -mod tests; diff --git a/src/openhuman/memory/api/capabilities_tests.rs b/src/openhuman/memory/api/capabilities_tests.rs deleted file mode 100644 index 1e5e550edd..0000000000 --- a/src/openhuman/memory/api/capabilities_tests.rs +++ /dev/null @@ -1,320 +0,0 @@ -//! Unit tests for the capability vocabulary in [`super`]. -//! -//! Three properties are load-bearing and each has its own test: -//! -//! 1. the enum has exactly the thirteen contract families and no more; -//! 2. the serialized form is stable snake_case **strings**, never discriminant -//! integers — a driver deployed against an older build must keep advertising -//! the same set after a variant is inserted mid-enum; -//! 3. [`super::Capabilities::validate`] rejects a set missing **any** of the -//! three mandatory families, checked one family at a time. - -use super::*; -use serde_json::json; - -#[test] -fn capability_has_exactly_the_thirteen_contract_families() { - assert_eq!(Capability::ALL.len(), 13); - assert_eq!(Capability::all().len(), 13); - - let names: Vec<&str> = Capability::ALL.iter().map(|c| c.as_str()).collect(); - assert_eq!( - names, - vec![ - "core", - "recall", - "ingest", - "documents", - "tree", - "entities", - "graph", - "diff", - "goals", - "tool_memory", - "sources", - "maintenance", - "portability", - ] - ); -} - -#[test] -fn capability_all_has_no_duplicates() { - let mut seen = std::collections::BTreeSet::new(); - for capability in Capability::ALL { - assert!( - seen.insert(capability.as_str()), - "duplicate capability in ALL: {capability}" - ); - } -} - -#[test] -fn capability_as_str_matches_serde_representation() { - // The wire form is the stable contract; `as_str` is the authority and the - // derive must agree with it for every variant. - for capability in Capability::ALL { - assert_eq!( - serde_json::to_value(capability).unwrap(), - json!(capability.as_str()), - "serde form drifted from as_str for {capability}" - ); - } -} - -#[test] -fn capability_serializes_as_a_string_not_an_integer() { - // Guards the specific regression the string form exists to prevent: - // inserting a variant must not re-map an already-deployed driver's set. - for capability in Capability::ALL { - assert!( - serde_json::to_value(capability).unwrap().is_string(), - "{capability} did not serialize as a string" - ); - } -} - -#[test] -fn capability_parse_round_trips_every_variant() { - for capability in Capability::ALL { - assert_eq!(Capability::parse(capability.as_str()), Ok(capability)); - assert_eq!( - capability.as_str().parse::(), - Ok(capability), - "FromStr disagreed with parse for {capability}" - ); - let decoded: Capability = - serde_json::from_value(json!(capability.as_str())).expect("known family decodes"); - assert_eq!(decoded, capability); - } -} - -#[test] -fn capability_parse_rejects_unknown_family() { - let err = Capability::parse("quantum_recall").expect_err("unknown family must not parse"); - assert!(err.contains("quantum_recall"), "unhelpful error: {err}"); -} - -#[test] -fn mandatory_families_are_core_recall_and_portability() { - assert_eq!( - Capability::MANDATORY, - [ - Capability::Core, - Capability::Recall, - Capability::Portability - ] - ); - for capability in Capability::ALL { - assert_eq!( - capability.is_mandatory(), - matches!( - capability, - Capability::Core | Capability::Recall | Capability::Portability - ), - "wrong mandatory classification for {capability}" - ); - } -} - -#[test] -fn capabilities_all_contains_every_family() { - let all = Capabilities::all(); - assert_eq!(all.len(), Capability::ALL.len()); - for capability in Capability::ALL { - assert!(all.contains(capability), "all() is missing {capability}"); - } - assert!(!all.is_empty()); -} - -#[test] -fn capabilities_empty_contains_nothing() { - let none = Capabilities::empty(); - assert!(none.is_empty()); - assert_eq!(none.len(), 0); - for capability in Capability::ALL { - assert!(!none.contains(capability)); - } - // The default capability set is empty (the null driver itself advertises - // `Capabilities::mandatory()`, not the default). - assert_eq!(Capabilities::default(), none); -} - -#[test] -fn capabilities_bit_width_has_room_well_beyond_the_current_thirteen_families() { - // A `u16` bitset (the original representation) has exactly 16 bit - // positions, leaving room for only 3 more families before a family's - // `1 << index` bit-shift overflows. Pin the wider `u64` representation so - // a future family addition doesn't have to rediscover that ceiling. - assert!(std::mem::size_of::() * 8 >= 64); -} - -#[test] -fn capabilities_insert_and_remove_are_idempotent() { - let mut set = Capabilities::empty(); - set.insert(Capability::Tree); - set.insert(Capability::Tree); - assert_eq!(set.len(), 1); - assert!(set.contains(Capability::Tree)); - assert!(!set.contains(Capability::Graph)); - - set.remove(Capability::Tree); - set.remove(Capability::Tree); - assert!(set.is_empty()); -} - -#[test] -fn capabilities_builder_forms_mirror_insert_and_remove() { - let set = Capabilities::empty() - .with(Capability::Core) - .with(Capability::Recall) - .without(Capability::Recall); - assert!(set.contains(Capability::Core)); - assert!(!set.contains(Capability::Recall)); -} - -#[test] -fn capabilities_contains_all_checks_subsets() { - let full = Capabilities::all(); - let mandatory = Capabilities::mandatory(); - - assert!(full.contains_all(mandatory)); - assert!(!mandatory.contains_all(full)); - assert!(mandatory.contains_all(mandatory)); - assert!(full.contains_all(Capabilities::empty())); -} - -#[test] -fn capabilities_iterates_in_declaration_order() { - let set: Capabilities = [ - Capability::Portability, - Capability::Core, - Capability::Tree, - Capability::Recall, - ] - .into_iter() - .collect(); - - assert_eq!( - set.iter().collect::>(), - vec![ - Capability::Core, - Capability::Recall, - Capability::Tree, - Capability::Portability - ] - ); -} - -#[test] -fn capabilities_serde_round_trips_and_uses_a_string_array() { - let set = Capabilities::mandatory().with(Capability::ToolMemory); - let encoded = serde_json::to_value(set).unwrap(); - - // Declaration order, snake_case strings — the `capabilities[]` handshake - // field. - assert_eq!( - encoded, - json!(["core", "recall", "tool_memory", "portability"]) - ); - - let decoded: Capabilities = serde_json::from_value(encoded).unwrap(); - assert_eq!(decoded, set); -} - -#[test] -fn capabilities_full_set_serde_round_trips() { - let all = Capabilities::all(); - let encoded = serde_json::to_string(&all).unwrap(); - let decoded: Capabilities = serde_json::from_str(&encoded).unwrap(); - assert_eq!(decoded, all); -} - -#[test] -fn capabilities_deserialization_collapses_duplicates_and_ignores_order() { - let decoded: Capabilities = - serde_json::from_value(json!(["portability", "core", "core", "recall"])).unwrap(); - assert_eq!(decoded, Capabilities::mandatory()); - assert_eq!(decoded.len(), 3); -} - -#[test] -fn capabilities_deserialization_skips_an_unknown_family() { - // A remote driver speaking a newer minor contract version may advertise a - // family this build has never heard of (see the module docs' "wire - // stability" section and `Capability::parse`). The handshake must still - // decode — with the unknown family dropped — rather than failing the bind - // outright. - let decoded: Capabilities = - serde_json::from_value(json!(["core", "warp_drive", "recall"])).unwrap(); - assert_eq!( - decoded, - Capabilities::empty() - .with(Capability::Core) - .with(Capability::Recall) - ); -} - -#[test] -fn validate_accepts_the_minimum_bindable_set() { - assert_eq!(Capabilities::mandatory().validate(), Ok(())); - assert_eq!(Capabilities::all().validate(), Ok(())); -} - -#[test] -fn validate_rejects_a_set_missing_core() { - let set = Capabilities::all().without(Capability::Core); - let err = set.validate().expect_err("missing core must be rejected"); - assert_eq!(err.missing, vec![Capability::Core]); - assert!(err.to_string().contains("core"), "{err}"); -} - -#[test] -fn validate_rejects_a_set_missing_recall() { - let set = Capabilities::all().without(Capability::Recall); - let err = set.validate().expect_err("missing recall must be rejected"); - assert_eq!(err.missing, vec![Capability::Recall]); - assert!(err.to_string().contains("recall"), "{err}"); -} - -#[test] -fn validate_rejects_a_set_missing_portability() { - // Portability is mandatory because without it a bind is a one-way door. - let set = Capabilities::all().without(Capability::Portability); - let err = set - .validate() - .expect_err("missing portability must be rejected"); - assert_eq!(err.missing, vec![Capability::Portability]); - assert!(err.to_string().contains("portability"), "{err}"); -} - -#[test] -fn validate_reports_every_missing_mandatory_family_at_once() { - let err = Capabilities::empty() - .validate() - .expect_err("the null set must be rejected"); - assert_eq!( - err.missing, - vec![ - Capability::Core, - Capability::Recall, - Capability::Portability - ] - ); -} - -#[test] -fn missing_mandatory_converts_to_an_invalid_memory_error() { - let err = Capabilities::empty().validate().unwrap_err(); - let message = err.to_string(); - let converted: MemoryError = err.into(); - // An incomplete advertised set is a bad claim about the driver, not an - // unsupported call. - assert!(matches!(converted, MemoryError::Invalid(ref m) if *m == message)); -} - -#[test] -fn missing_mandatory_is_empty_for_a_valid_set() { - assert!(Capabilities::mandatory().missing_mandatory().is_empty()); - assert!(Capabilities::all().missing_mandatory().is_empty()); -} diff --git a/src/openhuman/memory/api/chunks.rs b/src/openhuman/memory/api/chunks.rs deleted file mode 100644 index 789bc1be2b..0000000000 --- a/src/openhuman/memory/api/chunks.rs +++ /dev/null @@ -1,424 +0,0 @@ -//! Core types for the memory chunk layer. -//! -//! This module defines the canonical [`Chunk`] representation produced by the -//! ingestion pipeline along with its provenance [`Metadata`] and back-pointer -//! [`SourceRef`]. -//! -//! All chunk IDs are deterministic: `sha256(source_kind | "\0" | source_id | -//! "\0" | seq | "\0" | content)` truncated to 32 hex chars so re-ingest of the -//! same source material yields stable IDs and idempotent upserts. - -use chrono::{DateTime, Utc}; -use serde::{Deserialize, Serialize}; -use sha2::{Digest, Sha256}; - -/// Which kind of upstream source produced a chunk. -/// -/// Used both as a metadata discriminator and as the routing key for the -/// canonicaliser dispatch in the ingest pipeline. -#[derive(Clone, Copy, Debug, Eq, PartialEq, Hash, Serialize, Deserialize)] -#[serde(rename_all = "snake_case")] -pub enum SourceKind { - /// Chat transcript scoped by channel or group (Slack, Discord, Telegram, WhatsApp…). - Chat, - /// Email thread (Gmail and generic IMAP). - Email, - /// Standalone document (Notion page, Drive doc, meeting note, uploaded file…). - Document, -} - -impl SourceKind { - /// Stable string representation for DB storage and RPC surfaces. - pub fn as_str(self) -> &'static str { - match self { - SourceKind::Chat => "chat", - SourceKind::Email => "email", - SourceKind::Document => "document", - } - } - - /// Parse back from the on-wire / on-disk string form. - /// - /// # Errors - /// - /// Returns an error when `s` is not a supported source kind. - pub fn parse(s: &str) -> Result { - match s { - "chat" => Ok(SourceKind::Chat), - "email" => Ok(SourceKind::Email), - "document" => Ok(SourceKind::Document), - other => Err(format!("unknown source kind: {other}")), - } - } -} - -/// Concrete upstream provider the content came from. -/// -/// Each variant maps to exactly one [`SourceKind`] via [`Self::kind`]. Wire -/// form is snake_case (see [`Self::as_str`] / [`Self::parse`]) so it is stable -/// across DB rows, JSON-RPC payloads, and logs. -/// -/// Marked `#[non_exhaustive]` so new providers can be added in later phases -/// without breaking downstream pattern matches. -#[derive(Clone, Copy, Debug, Eq, PartialEq, Hash, Serialize, Deserialize)] -#[serde(rename_all = "snake_case")] -#[non_exhaustive] -pub enum DataSource { - // ── Chat transcripts (grouped by channel/group) ──────────────────── - /// Discord channel/server messages. Feeds [`SourceKind::Chat`]. - Discord, - /// Telegram chat/group messages. Feeds [`SourceKind::Chat`]. - Telegram, - /// WhatsApp chat/group messages. Feeds [`SourceKind::Chat`]. - Whatsapp, - - // ── Agent conversations (stored as durable memory) ──────────────── - /// Agent conversation transcripts persisted as durable memory. Feeds [`SourceKind::Chat`]. - Conversation, - - // ── Email threads (grouped by thread) ────────────────────────────── - /// Gmail thread. Feeds [`SourceKind::Email`]. - Gmail, - /// Catch-all for non-Gmail providers (Outlook, FastMail, generic IMAP, …). - OtherEmail, - - // ── Documents (no grouping) ──────────────────────────────────────── - /// Notion page. Feeds [`SourceKind::Document`]. - Notion, - /// Meeting notes document. Feeds [`SourceKind::Document`]. - MeetingNotes, - /// Google Drive document. Feeds [`SourceKind::Document`]. - DriveDocs, -} - -impl DataSource { - /// Which [`SourceKind`] this provider feeds into. - pub fn kind(self) -> SourceKind { - match self { - Self::Discord | Self::Telegram | Self::Whatsapp | Self::Conversation => { - SourceKind::Chat - } - Self::Gmail | Self::OtherEmail => SourceKind::Email, - Self::Notion | Self::MeetingNotes | Self::DriveDocs => SourceKind::Document, - } - } - - /// Stable snake_case identifier for DB storage, RPC payloads, and logs. - pub fn as_str(self) -> &'static str { - match self { - Self::Discord => "discord", - Self::Telegram => "telegram", - Self::Whatsapp => "whatsapp", - Self::Conversation => "conversation", - Self::Gmail => "gmail", - Self::OtherEmail => "other_email", - Self::Notion => "notion", - Self::MeetingNotes => "meeting_notes", - Self::DriveDocs => "drive_docs", - } - } - - /// Parse back from the on-wire / on-disk string form. - /// - /// # Errors - /// - /// Returns an error when `s` is not a supported data source. - pub fn parse(s: &str) -> Result { - match s { - "discord" => Ok(Self::Discord), - "telegram" => Ok(Self::Telegram), - "whatsapp" => Ok(Self::Whatsapp), - "conversation" => Ok(Self::Conversation), - "gmail" => Ok(Self::Gmail), - "other_email" => Ok(Self::OtherEmail), - "notion" => Ok(Self::Notion), - "meeting_notes" => Ok(Self::MeetingNotes), - "drive_docs" => Ok(Self::DriveDocs), - other => Err(format!("unknown data source: {other}")), - } - } - - /// Every known variant, in declaration order. Useful for tests, CLI - /// completion, and enumerating supported providers in diagnostic output. - pub fn all() -> &'static [DataSource] { - &[ - Self::Discord, - Self::Telegram, - Self::Whatsapp, - Self::Conversation, - Self::Gmail, - Self::OtherEmail, - Self::Notion, - Self::MeetingNotes, - Self::DriveDocs, - ] - } -} - -/// A concrete pointer back to where a chunk originated — used for citation, -/// drill-down, and deduplication at re-ingest time. -/// -/// Consumers should treat this as an opaque, source-specific reference. The -/// shape depends on [`SourceKind`]: -/// - **Chat**: `{platform}://{channel}/{message_id}` or `{permalink}` -/// - **Email**: message-id header (``) or provider URL -/// - **Document**: file path, Notion page URL, Drive file id -#[derive(Clone, Debug, Eq, PartialEq, Serialize, Deserialize)] -pub struct SourceRef { - /// Opaque provider-specific identifier for the exact source record. - pub value: String, -} - -impl SourceRef { - /// Wrap an opaque provider-specific identifier as a [`SourceRef`]. - pub fn new(value: impl Into) -> Self { - Self { - value: value.into(), - } - } -} - -/// Provenance metadata captured per chunk at ingest time. -/// -/// Captures at minimum: source type, source identifier, owner/account, -/// timestamps, and tags/labels when available. -#[derive(Clone, Debug, Eq, PartialEq, Serialize, Deserialize)] -pub struct Metadata { - /// Which upstream source kind produced this chunk. - pub source_kind: SourceKind, - /// Stable logical id for the ingestion group (channel id, thread id, doc id). - /// - /// Chat: channel/group id. Email: thread id. Document: doc id. - pub source_id: String, - /// Account or user the content belongs to. Empty string for anonymous / system sources. - pub owner: String, - /// Point-in-time timestamp for ordering within a source. - /// - /// For chats = message time; for emails = message sent time; - /// for documents = last-modified or ingest time. - #[serde(with = "chrono::serde::ts_milliseconds")] - pub timestamp: DateTime, - /// Covering time range the chunk spans. For a single leaf it usually equals - /// `(timestamp, timestamp)`; for later summary nodes it widens to cover all - /// children. - #[serde(with = "time_range_serde")] - pub time_range: (DateTime, DateTime), - /// Arbitrary labels / tags carried through from the source (e.g. Gmail labels, - /// Slack reactions, Notion tags). Ingest does not interpret these. - #[serde(default)] - pub tags: Vec, - /// Opaque pointer back to the raw source record for drill-down / citation. - pub source_ref: Option, - /// When set, overrides `source_id` for the chunk file path so multiple - /// items share one directory. `source_id` remains the dedup key. - #[serde(default, skip_serializing_if = "Option::is_none")] - pub path_scope: Option, -} - -impl Metadata { - /// Convenience constructor used by canonicalisers: point timestamp, - /// `time_range = (timestamp, timestamp)`. - pub fn point_in_time( - source_kind: SourceKind, - source_id: impl Into, - owner: impl Into, - timestamp: DateTime, - ) -> Self { - Self { - source_kind, - source_id: source_id.into(), - owner: owner.into(), - timestamp, - time_range: (timestamp, timestamp), - tags: Vec::new(), - source_ref: None, - path_scope: None, - } - } -} - -/// A single ingested chunk — the atomic persistence unit. -/// -/// In the design this is the leaf of a source tree. Later phases build summary -/// nodes on top of these leaves; here they live standalone. -#[derive(Clone, Debug, Eq, PartialEq, Serialize, Deserialize)] -pub struct Chunk { - /// Deterministic id derived from (source_kind, source_id, seq_in_source, content). - pub id: String, - /// Canonical Markdown content. - pub content: String, - /// Provenance metadata. - pub metadata: Metadata, - /// Token count (rough heuristic — 1 token ≈ 4 chars). - pub token_count: u32, - /// Sequence number of this chunk inside its logical source. Stable and - /// starts at 0 for the first chunk of a source. - pub seq_in_source: u32, - /// When this chunk was persisted to the local store. - #[serde(with = "chrono::serde::ts_milliseconds")] - pub created_at: DateTime, - /// True when this chunk is a sub-split of a single logical unit (e.g. a - /// chat message or email body that exceeded `max_tokens`). Each piece - /// carries this flag so downstream scorers can lower its weight relative to - /// whole-unit chunks. - #[serde(default)] - pub partial_message: bool, -} - -/// A chunk staged for the MD-content write path: a [`Chunk`] whose full body -/// lives on disk at `content_path` (with `content_sha256` for integrity), while -/// the SQLite `content` column carries only a ≤500-char preview. -#[derive(Clone, Debug, Eq, PartialEq)] -pub struct StagedChunk { - /// The chunk being persisted. - pub chunk: Chunk, - /// Forward-slash relative path (under the content root) where the full body lives. - pub content_path: String, - /// Hex SHA-256 of the on-disk body, recorded for integrity checks. - pub content_sha256: String, -} - -/// Deterministic chunk id. -/// -/// `sha256(source_kind | "\0" | source_id | "\0" | seq | "\0" | content)` -/// hex-encoded, first 32 chars (128 bits of collision resistance). -/// -/// Content is included so multiple ingest calls that share a `source_id` don't -/// collide on `seq=0,1,2,…`. Re-ingesting the same canonical content under the -/// same `(source_id, seq)` still produces the same id, so upserts stay -/// idempotent. -pub fn chunk_id( - source_kind: SourceKind, - source_id: &str, - seq_in_source: u32, - content: &str, -) -> String { - let mut hasher = Sha256::new(); - hasher.update(source_kind.as_str().as_bytes()); - hasher.update([0u8]); - hasher.update(source_id.as_bytes()); - hasher.update([0u8]); - hasher.update(seq_in_source.to_be_bytes()); - hasher.update([0u8]); - hasher.update(content.as_bytes()); - let digest = hasher.finalize(); - let hex = digest.iter().fold(String::with_capacity(64), |mut acc, b| { - use std::fmt::Write; - let _ = write!(acc, "{b:02x}"); - acc - }); - hex[..32].to_string() -} - -/// Approximate token count (GPT-family heuristic: 1 token ≈ 4 chars). -pub fn approx_token_count(text: &str) -> u32 { - // saturating_add guards against absurdly long inputs - let chars = text.chars().count() as u32; - chars.saturating_add(3) / 4 -} - -/// Per-character weight in **quarter-token** units for -/// [`conservative_token_estimate`]. Deliberately pessimistic so the chunker and -/// the embed backstop never under-split: real SentencePiece/WordPiece output for -/// hash-, code-, and markdown-dense text approaches ~1 token/char — far above -/// the `chars/4` GPT heuristic in [`approx_token_count`]. -fn char_token_quarters(ch: char) -> u32 { - if ch.is_ascii_alphanumeric() { - 2 // 0.50 token/char — alphanumeric runs pack ~2-4 chars per token - } else if ch.is_whitespace() { - 1 // 0.25 token/char — whitespace usually merges into adjacent pieces - } else { - 4 // 1.00 token/char — ASCII punctuation/symbols AND all non-ASCII - // (Hebrew/CJK/emoji), which tokenise ~1 piece per char or worse - } -} - -/// Conservative (over-estimating) token count, for embed-safety decisions only. -/// -/// [`approx_token_count`] (`chars/4`) under-counts dense markdown/hash/code by -/// ~5×. This weights characters by class so the result is an upper-ish bound on -/// real tokeniser output. It does **not** replace `approx_token_count`, which -/// still drives summariser/seal token budgeting. -pub fn conservative_token_estimate(text: &str) -> u32 { - let quarters: u64 = text - .chars() - .map(|c| u64::from(char_token_quarters(c))) - .sum(); - let tokens = quarters.div_ceil(4); // ceil(quarters / 4) - tokens.min(u64::from(u32::MAX)) as u32 -} - -/// Largest leading slice of `text` whose [`conservative_token_estimate`] is -/// ≤ `budget`, ending on a UTF-8 char boundary. Returns the whole string when -/// already within budget. Used as the embed-path backstop so an over-long body -/// can never be sent to the embedder above its input limit. -pub fn truncate_to_conservative_tokens(text: &str, budget: u32) -> &str { - if conservative_token_estimate(text) <= budget { - return text; - } - let cap = u64::from(budget).saturating_mul(4); // quarter-tokens - let mut acc: u64 = 0; - for (idx, ch) in text.char_indices() { - let q = u64::from(char_token_quarters(ch)); - if acc + q > cap { - return &text[..idx]; - } - acc += q; - } - text -} - -/// `serde(with = ...)` shim for `(DateTime, DateTime)`. -/// -/// Chrono has no built-in serde helper for a *pair* of timestamps, so this -/// mirrors `chrono::serde::ts_milliseconds` but for a 2-tuple: each endpoint -/// round-trips through millisecond-since-epoch integers under the field -/// names `start_ms` / `end_ms`. -mod time_range_serde { - use chrono::{DateTime, TimeZone, Utc}; - use serde::{Deserialize, Deserializer, Serialize, Serializer}; - - /// On-wire shape: millisecond-since-epoch pair. - #[derive(Serialize, Deserialize)] - struct Wire { - start_ms: i64, - end_ms: i64, - } - - /// Serialize a `(start, end)` UTC timestamp pair as `{start_ms, end_ms}`. - pub fn serialize( - value: &(DateTime, DateTime), - serializer: S, - ) -> Result { - Wire { - start_ms: value.0.timestamp_millis(), - end_ms: value.1.timestamp_millis(), - } - .serialize(serializer) - } - - /// Deserialize a `{start_ms, end_ms}` pair back into UTC timestamps. - /// - /// # Errors - /// Returns a `serde` custom error if either millisecond value does not - /// map to a valid `DateTime` (chrono's `timestamp_millis_opt` fails, - /// e.g. out-of-range values). - pub fn deserialize<'de, D: Deserializer<'de>>( - deserializer: D, - ) -> Result<(DateTime, DateTime), D::Error> { - let wire = Wire::deserialize(deserializer)?; - let start = Utc - .timestamp_millis_opt(wire.start_ms) - .single() - .ok_or_else(|| serde::de::Error::custom("invalid start_ms"))?; - let end = Utc - .timestamp_millis_opt(wire.end_ms) - .single() - .ok_or_else(|| serde::de::Error::custom("invalid end_ms"))?; - Ok((start, end)) - } -} - -#[cfg(test)] -#[path = "chunks_tests.rs"] -mod tests; diff --git a/src/openhuman/memory/api/chunks_tests.rs b/src/openhuman/memory/api/chunks_tests.rs deleted file mode 100644 index 3d9c89f4dc..0000000000 --- a/src/openhuman/memory/api/chunks_tests.rs +++ /dev/null @@ -1,209 +0,0 @@ -//! Unit tests for the chunk model (`super`). - -use super::*; -use chrono::TimeZone; - -#[test] -fn chunk_id_is_deterministic() { - let a = chunk_id(SourceKind::Chat, "slack:#eng", 0, "hello"); - let b = chunk_id(SourceKind::Chat, "slack:#eng", 0, "hello"); - assert_eq!(a, b); - assert_eq!(a, "95785e45df3ff65599a71866e0412993"); - assert_eq!(a.len(), 32); -} - -#[test] -fn conservative_estimate_weights_by_char_class() { - assert_eq!(conservative_token_estimate("abcd"), 2); // 4 alnum × 2q / 4 - assert_eq!(conservative_token_estimate(" "), 1); // 4 ws × 1q / 4 - assert_eq!(conservative_token_estimate("....,,,,"), 8); // 8 punct × 4q / 4 - assert_eq!(conservative_token_estimate("שלום"), 4); // 4 non-ascii × 4q / 4 - assert_eq!(conservative_token_estimate(""), 0); -} - -#[test] -fn conservative_estimate_exceeds_approx_for_dense_content() { - let dense = "claude-memory:openhuman:MEMORY.md:67d6fe2727d431b16d41630babfdcf1cdf61bda7b9ba\n" - .repeat(40); - assert!( - conservative_token_estimate(&dense) > approx_token_count(&dense), - "conservative estimate must exceed chars/4 on dense content", - ); -} - -#[test] -fn truncate_respects_budget_and_char_boundaries() { - let text = "שלום עולם ".repeat(100); // Hebrew, ~1 token/char - let out = truncate_to_conservative_tokens(&text, 10); - assert!(conservative_token_estimate(out) <= 10); - assert!(text.starts_with(out)); // valid prefix on a char boundary - assert!(out.len() < text.len()); -} - -#[test] -fn truncate_is_noop_within_budget() { - let text = "short and sweet"; - assert_eq!(truncate_to_conservative_tokens(text, 1000), text); -} - -#[test] -fn chunk_id_varies_with_seq() { - let a = chunk_id(SourceKind::Chat, "slack:#eng", 0, "hello"); - let b = chunk_id(SourceKind::Chat, "slack:#eng", 1, "hello"); - assert_ne!(a, b); -} - -#[test] -fn chunk_id_varies_with_source_kind() { - let a = chunk_id(SourceKind::Chat, "foo", 0, "hello"); - let b = chunk_id(SourceKind::Email, "foo", 0, "hello"); - assert_ne!(a, b); -} - -#[test] -fn chunk_id_varies_with_source_id() { - let a = chunk_id(SourceKind::Chat, "x", 0, "hello"); - let b = chunk_id(SourceKind::Chat, "y", 0, "hello"); - assert_ne!(a, b); -} - -#[test] -fn chunk_id_varies_with_content() { - let a = chunk_id(SourceKind::Chat, "slack:c1", 0, "bucket A content"); - let b = chunk_id(SourceKind::Chat, "slack:c1", 0, "bucket B content"); - assert_ne!(a, b); -} - -#[test] -fn source_kind_round_trip() { - for kind in [SourceKind::Chat, SourceKind::Email, SourceKind::Document] { - assert_eq!(SourceKind::parse(kind.as_str()).unwrap(), kind); - } -} - -#[test] -fn data_source_round_trip() { - for ds in DataSource::all() { - assert_eq!(DataSource::parse(ds.as_str()).unwrap(), *ds); - } -} - -#[test] -fn data_source_has_all_variants() { - assert_eq!(DataSource::all().len(), 9); -} - -#[test] -fn data_source_kind_mapping() { - use DataSource::*; - for ds in [Discord, Telegram, Whatsapp, Conversation] { - assert_eq!(ds.kind(), SourceKind::Chat); - } - for ds in [Gmail, OtherEmail] { - assert_eq!(ds.kind(), SourceKind::Email); - } - for ds in [Notion, MeetingNotes, DriveDocs] { - assert_eq!(ds.kind(), SourceKind::Document); - } -} - -#[test] -fn data_source_parse_rejects_unknown() { - assert!(DataSource::parse("nope").is_err()); - assert!(DataSource::parse("Discord").is_err()); // case-sensitive - assert!(DataSource::parse("drive docs").is_err()); // no spaces -} - -#[test] -fn data_source_serde_is_snake_case() { - let ds = DataSource::MeetingNotes; - let json = serde_json::to_string(&ds).unwrap(); - assert_eq!(json, "\"meeting_notes\""); - let parsed: DataSource = serde_json::from_str("\"meeting_notes\"").unwrap(); - assert_eq!(parsed, ds); -} - -#[test] -fn approx_token_count_scales_linearly() { - assert_eq!(approx_token_count(""), 0); - assert_eq!(approx_token_count("a"), 1); // 1→1 - assert_eq!(approx_token_count("abcd"), 1); // 4→1 - assert_eq!(approx_token_count("abcde"), 2); // 5→2 - assert_eq!(approx_token_count(&"x".repeat(400)), 100); -} - -#[test] -fn source_kind_parse_rejects_unknown_wire_values() { - assert_eq!( - SourceKind::parse("video").unwrap_err(), - "unknown source kind: video" - ); -} - -#[test] -fn metadata_constructor_and_source_ref_fill_documented_defaults() { - let timestamp = Utc.timestamp_millis_opt(1_700_000_000_123).unwrap(); - let mut metadata = Metadata::point_in_time(SourceKind::Document, "doc-1", "alice", timestamp); - metadata.source_ref = Some(SourceRef::new("notion://doc-1")); - - assert_eq!(metadata.source_id, "doc-1"); - assert_eq!(metadata.owner, "alice"); - assert_eq!(metadata.time_range, (timestamp, timestamp)); - assert!(metadata.tags.is_empty()); - assert_eq!(metadata.source_ref.unwrap().value, "notion://doc-1"); -} - -#[test] -fn chunk_json_round_trips_millisecond_time_range_and_partial_default() { - let timestamp = Utc.timestamp_millis_opt(1_700_000_000_123).unwrap(); - let chunk = Chunk { - id: "chunk".into(), - content: "body".into(), - metadata: Metadata::point_in_time(SourceKind::Chat, "channel", "alice", timestamp), - token_count: 1, - seq_in_source: 0, - created_at: timestamp, - partial_message: true, - }; - let encoded = serde_json::to_value(&chunk).unwrap(); - assert_eq!( - encoded["metadata"]["time_range"]["start_ms"], - timestamp.timestamp_millis() - ); - assert_eq!(serde_json::from_value::(encoded).unwrap(), chunk); - - let mut legacy = serde_json::to_value(&chunk).unwrap(); - legacy.as_object_mut().unwrap().remove("partial_message"); - assert!( - !serde_json::from_value::(legacy) - .unwrap() - .partial_message - ); -} - -#[test] -fn chunk_json_rejects_out_of_range_time_range_endpoints() { - let timestamp = Utc.timestamp_millis_opt(1_700_000_000_123).unwrap(); - let chunk = Chunk { - id: "chunk".into(), - content: "body".into(), - metadata: Metadata::point_in_time(SourceKind::Chat, "channel", "alice", timestamp), - token_count: 1, - seq_in_source: 0, - created_at: timestamp, - partial_message: false, - }; - let mut encoded = serde_json::to_value(chunk).unwrap(); - encoded["metadata"]["time_range"]["start_ms"] = serde_json::json!(i64::MAX); - assert!(serde_json::from_value::(encoded.clone()) - .unwrap_err() - .to_string() - .contains("invalid start_ms")); - - encoded["metadata"]["time_range"]["start_ms"] = serde_json::json!(0); - encoded["metadata"]["time_range"]["end_ms"] = serde_json::json!(i64::MAX); - assert!(serde_json::from_value::(encoded) - .unwrap_err() - .to_string() - .contains("invalid end_ms")); -} diff --git a/src/openhuman/memory/api/error.rs b/src/openhuman/memory/api/error.rs deleted file mode 100644 index 0c493f166c..0000000000 --- a/src/openhuman/memory/api/error.rs +++ /dev/null @@ -1,104 +0,0 @@ -//! Engine-level error type shared by ported modules that want a typed error -//! surface. Modules that mirror OpenHuman's `anyhow`-based signatures may keep -//! using `anyhow::Result`; this enum is for contracts that benefit from -//! matchable variants (validation, not-found, taint, IO). -//! -//! `?` converts `std::io::Error` and `serde_json::Error` into -//! [`MemoryError::Io`] / [`MemoryError::Serde`] automatically via the derived -//! `#[from]` impls, and any `anyhow::Error` (including one produced by `?` on -//! a foreign error type inside an `anyhow`-returning function) into -//! [`MemoryError::Other`]. The purpose-built variants ([`MemoryError::NotFound`], -//! [`MemoryError::Invalid`], [`MemoryError::BudgetExceeded`], -//! [`MemoryError::PathEscape`]) are constructed explicitly by callers that want -//! matchable, typed failure — they are never inferred from a foreign error. -//! -//! [`MemoryError::Unsupported`] is the one variant that belongs to the *driver -//! contract* rather than the engine: it is what a caller gets when a bound -//! driver does not implement the capability family a call needs. See its docs -//! for why that should be rare. - -use thiserror::Error; - -use crate::openhuman::memory::api::capabilities::Capability; - -/// Errors surfaced by the memory engine. -#[derive(Debug, Error)] -pub enum MemoryError { - /// A requested record / source / node was not found. - #[error("not found: {0}")] - NotFound(String), - /// Caller-supplied input failed validation. - #[error("invalid input: {0}")] - Invalid(String), - /// A configured budget (tokens, cost, depth) was exceeded. - #[error("budget exceeded: {0}")] - BudgetExceeded(String), - /// A path escaped the workspace sandbox (symlink / traversal). - #[error("path escapes workspace: {0}")] - PathEscape(String), - /// Underlying IO failure. - #[error("io error: {0}")] - Io(#[from] std::io::Error), - /// Serialization / deserialization failure. - #[error("serde error: {0}")] - Serde(#[from] serde_json::Error), - /// The bound driver does not implement the named capability family. - /// - /// This should be **rare**, because capabilities are negotiated once at - /// bind time and the kernel unregisters the RPC methods and omits the agent - /// tools of every unadvertised family. Reaching this variant means one of: - /// - /// - an out-of-process driver answered `501` for a family its handshake - /// claimed (the case [`crate::openhuman::memory::api::capabilities`] cannot pre-empt); - /// - a caller bypassed the capability filter — a kernel bug. - /// - /// ## Why the payload is an owned `String` and not a [`Capability`] - /// - /// The transport adapter constructs this from a wire response, where the - /// family is a runtime string that may not be a known [`Capability`] at all - /// — a driver speaking a newer minor contract version, a vendor extension, - /// or simply a typo in a third-party backend. A `Capability` field would - /// force the adapter to drop that information or fail parsing, and a - /// `&'static str` cannot be produced from a runtime value without leaking - /// memory. An owned `String` is the only representation that round-trips - /// every case. - /// - /// Construct it with [`MemoryError::unsupported`] when the family is known - /// (that path yields the canonical [`Capability::as_str`] spelling) and - /// with [`MemoryError::unsupported_raw`] when it came off the wire. - #[error("unsupported capability: {capability}")] - Unsupported { - /// Wire name of the capability family that is not supported — - /// [`Capability::as_str`] when known, otherwise the raw string the - /// driver reported. - capability: String, - }, - /// Catch-all wrapping an opaque lower-level error. - #[error(transparent)] - Other(#[from] anyhow::Error), -} - -impl MemoryError { - /// Builds [`MemoryError::Unsupported`] for a family this build knows, - /// using its canonical [`Capability::as_str`] spelling. - pub fn unsupported(capability: Capability) -> Self { - Self::Unsupported { - capability: capability.as_str().to_string(), - } - } - - /// Builds [`MemoryError::Unsupported`] from a family name that came off the - /// wire and may not correspond to any known [`Capability`]. - pub fn unsupported_raw(capability: impl Into) -> Self { - Self::Unsupported { - capability: capability.into(), - } - } -} - -/// Convenience result alias for engine-level fallible operations. -pub type MemoryEngineResult = Result; - -#[cfg(test)] -#[path = "error_tests.rs"] -mod tests; diff --git a/src/openhuman/memory/api/error_tests.rs b/src/openhuman/memory/api/error_tests.rs deleted file mode 100644 index 0a8d7ab06f..0000000000 --- a/src/openhuman/memory/api/error_tests.rs +++ /dev/null @@ -1,59 +0,0 @@ -//! Unit tests for [`super::MemoryError`], focused on the `Unsupported` variant -//! added for the driver contract. The older variants are exercised where they -//! are constructed, in the engine crate. - -use super::*; -use crate::openhuman::memory::api::capabilities::Capability; - -#[test] -fn unsupported_from_a_known_capability_uses_the_canonical_wire_name() { - for capability in Capability::ALL { - let err = MemoryError::unsupported(capability); - match err { - MemoryError::Unsupported { - capability: ref got, - } => { - assert_eq!(got, capability.as_str()); - } - other => panic!("expected Unsupported, got {other:?}"), - } - } -} - -#[test] -fn unsupported_raw_preserves_a_family_this_build_does_not_know() { - // The reason the payload is an owned `String`: a driver speaking a newer - // minor contract version can name a family that is not a `Capability` here, - // and the adapter must be able to report it verbatim. - let err = MemoryError::unsupported_raw("holographic_recall"); - match err { - MemoryError::Unsupported { ref capability } => { - assert_eq!(capability, "holographic_recall"); - assert!(Capability::parse(capability).is_err()); - } - other => panic!("expected Unsupported, got {other:?}"), - } -} - -#[test] -fn unsupported_display_names_the_capability() { - assert_eq!( - MemoryError::unsupported(Capability::Tree).to_string(), - "unsupported capability: tree" - ); - assert_eq!( - MemoryError::unsupported(Capability::ToolMemory).to_string(), - "unsupported capability: tool_memory" - ); -} - -#[test] -fn unsupported_is_distinguishable_from_the_other_variants() { - // A transport adapter maps `501` to `Unsupported` and everything else - // elsewhere, so the variant must not collide with `Invalid` / `NotFound`. - let unsupported = MemoryError::unsupported(Capability::Diff); - assert!(matches!(unsupported, MemoryError::Unsupported { .. })); - - let invalid = MemoryError::Invalid("diff".to_string()); - assert!(!matches!(invalid, MemoryError::Unsupported { .. })); -} diff --git a/src/openhuman/memory/api/goals.rs b/src/openhuman/memory/api/goals.rs deleted file mode 100644 index 697a86784b..0000000000 --- a/src/openhuman/memory/api/goals.rs +++ /dev/null @@ -1,131 +0,0 @@ -//! Domain types for the agent's long-term goals list. -//! -//! Goals are a small, ordered list of durable objectives the agent holds when -//! interacting with the user. They are persisted as a compact markdown document -//! (`MEMORY_GOALS.md`) by the engine crate's `memory::goals::store` and -//! surfaced over RPC + agent tools. Each item carries a stable short id so -//! edit/delete operations can address a specific line without depending on -//! ordering. -//! -//! This module is **pure data**: it owns the shape, parse, and render only. -//! The validating mutation surface (`add` / `edit` / `delete`) lives next to -//! the `regex`-backed PII/secret predicates it calls, in the engine crate's -//! `memory::goals::store::GoalsDocMutations` trait, so the value types stay -//! free of the safety machinery and of `regex`. The cap-enforcing persistence -//! layer and the reflection apply/dedupe logic live in the engine crate too. - -use serde::{Deserialize, Serialize}; - -/// Markdown header rendered at the top of `MEMORY_GOALS.md`. -pub(crate) const HEADER: &str = "# Long-term Goals"; - -/// A single long-term goal item. -#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] -pub struct GoalItem { - /// Stable short id (e.g. `g1`). Used as the dedupe/address key for - /// `edit`/`delete`. Rendered inline in the markdown as `- [g1] …`. - pub id: String, - /// The goal text — one concise sentence. - pub text: String, -} - -impl GoalItem { - /// Construct a goal item from an id + text, trimming surrounding - /// whitespace from the text. - pub fn new(id: impl Into, text: impl Into) -> Self { - Self { - id: id.into(), - text: text.into().trim().to_string(), - } - } -} - -/// The full goals document — an ordered list of [`GoalItem`]s. -#[derive(Debug, Clone, Default, PartialEq, Eq, Serialize, Deserialize)] -pub struct GoalsDoc { - /// Ordered goal items. Order is meaningful for rendering and cap trimming - /// (oldest = front). - pub items: Vec, -} - -impl GoalsDoc { - /// Parse a `MEMORY_GOALS.md` body into a [`GoalsDoc`]. - /// - /// Recognised item lines look like `- [g1] do the thing`. Lines that don't - /// match (the header, blank lines, free prose) are ignored so a - /// hand-edited file degrades gracefully rather than erroring. - pub fn parse(body: &str) -> Self { - let mut items = Vec::new(); - for line in body.lines() { - let trimmed = line.trim(); - // Strip the leading list marker, if present. - let rest = match trimmed.strip_prefix("- ") { - Some(r) => r.trim(), - None => continue, - }; - // Expect `[id] text`. - let Some(after_open) = rest.strip_prefix('[') else { - continue; - }; - let Some(close_idx) = after_open.find(']') else { - continue; - }; - let id = after_open[..close_idx].trim(); - let text = after_open[close_idx + 1..].trim(); - if id.is_empty() || text.is_empty() { - continue; - } - items.push(GoalItem::new(id, text)); - } - Self { items } - } - - /// Render the document back to markdown suitable for `MEMORY_GOALS.md`. - /// - /// NOTE: this emits only the header and the recognised `- [id] text` - /// item lines — any free prose, sub-bullets, or other hand-added content - /// a user wrote into the file is not represented in [`GoalsDoc`] and is - /// therefore dropped on the next `parse` → mutate → `render` round-trip - /// (e.g. via `add`/`edit`/`delete`/reflection). Treat this file as - /// machine-owned rather than freely hand-editable. - pub fn render(&self) -> String { - let mut out = String::from(HEADER); - out.push_str("\n\n"); - for item in &self.items { - out.push_str(&format!("- [{}] {}\n", item.id, item.text)); - } - out - } - - /// Whether the list currently has no items. Used to drive the - /// "first run / initial population" reflection behaviour. - pub fn is_empty(&self) -> bool { - self.items.is_empty() - } - - /// Number of goal items currently held. - pub fn len(&self) -> usize { - self.items.len() - } - - /// Allocate the next free `g` id not already used in the list. - pub fn next_id(&self) -> String { - let mut n = self.items.len() + 1; - loop { - let candidate = format!("g{n}"); - if !self.items.iter().any(|i| i.id == candidate) { - return candidate; - } - n += 1; - } - } - - /// Whether the list already holds `id`. - pub fn contains_id(&self, id: &str) -> bool { - self.items.iter().any(|i| i.id == id) - } -} - -#[cfg(test)] -#[path = "goals_tests.rs"] -mod tests; diff --git a/src/openhuman/memory/api/goals_tests.rs b/src/openhuman/memory/api/goals_tests.rs deleted file mode 100644 index e67410ce40..0000000000 --- a/src/openhuman/memory/api/goals_tests.rs +++ /dev/null @@ -1,22 +0,0 @@ -//! Unit tests for [`super::GoalsDoc`] parse/render — the pure-data half. -//! -//! The validating mutation tests (`add` / `edit` / `delete`, including the -//! secret/PII rejection cases) live in the engine crate next to the -//! `GoalsDocMutations` trait that owns them: `memory::goals::mutations_tests`. - -use super::*; - -#[test] -fn render_starts_with_header() { - let doc = GoalsDoc::default(); - assert!(doc.render().starts_with("# Long-term Goals")); -} - -#[test] -fn parse_ignores_non_item_lines() { - let body = "# Long-term Goals\n\nsome stray prose\n- [g1] real goal\n- malformed line\n"; - let doc = GoalsDoc::parse(body); - assert_eq!(doc.items.len(), 1); - assert_eq!(doc.items[0].id, "g1"); - assert_eq!(doc.items[0].text, "real goal"); -} diff --git a/src/openhuman/memory/api/health.rs b/src/openhuman/memory/api/health.rs deleted file mode 100644 index e98c3242d1..0000000000 --- a/src/openhuman/memory/api/health.rs +++ /dev/null @@ -1,119 +0,0 @@ -//! Liveness state a memory driver reports about itself. -//! -//! ## Why this lives in the contract crate and not in the host -//! -//! The OpenHuman kernel has (or will have) a *generic* subsystem-agnostic -//! `DriverHealth` shared by memory, inference, channels, and sandbox. This crate -//! cannot name that type: `tinymemory-api` is the contract a third-party driver -//! compiles against, and a driver must be able to depend on it without pulling -//! in the OpenHuman host — nor should the next subsystem cut over inherit -//! generic kernel vocabulary from a *memory* crate. -//! -//! So the contract carries its own [`MemoryHealth`], and the host's memory -//! adapter converts. The conversion is deliberately trivial and lossless: this -//! is a **small closed enum with a reason string**, shaped one-for-one against -//! the kernel's `Ready | Degraded { reason } | Down { reason }`, not a -//! free-form struct that would need field-by-field mapping and would drift. -//! Keep it that way — if a driver needs to report something richer, it belongs -//! in a driver-specific status payload, not here. -//! -//! ## Wire form -//! -//! Serializes as an internally-tagged object with a stable snake_case `status` -//! discriminant, which is also the shape of the transport adapter's -//! `GET /v1/health` → `{ status, reason }` response: -//! -//! ```json -//! { "status": "ready" } -//! { "status": "degraded", "reason": "vector index rebuilding" } -//! { "status": "down", "reason": "connection refused" } -//! ``` - -use serde::{Deserialize, Serialize}; - -/// Health of a bound memory driver, as the driver reports it. -/// -/// The three states are ordered by severity and mean different things to the -/// kernel: -/// -/// - [`MemoryHealth::Ready`] — serve traffic normally. -/// - [`MemoryHealth::Degraded`] — still serve traffic, but surface the reason -/// in status output; results may be incomplete or slow. -/// - [`MemoryHealth::Down`] — do not serve traffic; the bind should be surfaced -/// as failed and, per the fallback rule, the embedded default rebound. -#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] -#[serde(tag = "status", rename_all = "snake_case")] -pub enum MemoryHealth { - /// The driver is reachable and serving requests normally. - Ready, - /// The driver is serving requests, but something is wrong and the caller - /// should surface it. Results may be incomplete, stale, or slow. - Degraded { - /// Operator-facing explanation. Must not contain credentials, tokens, - /// or user memory content — this string is logged and shown in status - /// output. - reason: String, - }, - /// The driver cannot serve requests at all. - Down { - /// Operator-facing explanation, subject to the same redaction rule as - /// [`MemoryHealth::Degraded::reason`]. - reason: String, - }, -} - -impl MemoryHealth { - /// Convenience constructor for [`MemoryHealth::Degraded`]. - pub fn degraded(reason: impl Into) -> Self { - Self::Degraded { - reason: reason.into(), - } - } - - /// Convenience constructor for [`MemoryHealth::Down`]. - pub fn down(reason: impl Into) -> Self { - Self::Down { - reason: reason.into(), - } - } - - /// Stable snake_case discriminant, matching the serialized `status` field. - pub fn as_str(&self) -> &'static str { - match self { - Self::Ready => "ready", - Self::Degraded { .. } => "degraded", - Self::Down { .. } => "down", - } - } - - /// The operator-facing reason, when there is one. `None` for - /// [`MemoryHealth::Ready`]. - pub fn reason(&self) -> Option<&str> { - match self { - Self::Ready => None, - Self::Degraded { reason } | Self::Down { reason } => Some(reason.as_str()), - } - } - - /// Whether the kernel should route traffic to this driver. - /// - /// True for [`MemoryHealth::Ready`] and [`MemoryHealth::Degraded`] — a - /// degraded driver is still the bound driver — and false for - /// [`MemoryHealth::Down`]. - pub fn is_usable(&self) -> bool { - !matches!(self, Self::Down { .. }) - } -} - -impl std::fmt::Display for MemoryHealth { - fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { - match self.reason() { - Some(reason) => write!(f, "{}: {reason}", self.as_str()), - None => f.write_str(self.as_str()), - } - } -} - -#[cfg(test)] -#[path = "health_tests.rs"] -mod tests; diff --git a/src/openhuman/memory/api/health_tests.rs b/src/openhuman/memory/api/health_tests.rs deleted file mode 100644 index a31c96516c..0000000000 --- a/src/openhuman/memory/api/health_tests.rs +++ /dev/null @@ -1,90 +0,0 @@ -//! Unit tests for [`super::MemoryHealth`]. -//! -//! These pin the two properties the host's memory adapter depends on: the -//! variant set is closed and small enough for a lossless `match` into the -//! kernel's generic `DriverHealth`, and the wire form carries a stable -//! `status` discriminant plus a `reason`. - -use super::*; -use serde_json::json; - -#[test] -fn ready_has_no_reason_and_is_usable() { - let health = MemoryHealth::Ready; - assert_eq!(health.as_str(), "ready"); - assert_eq!(health.reason(), None); - assert!(health.is_usable()); - assert_eq!(health.to_string(), "ready"); -} - -#[test] -fn degraded_carries_a_reason_and_is_still_usable() { - let health = MemoryHealth::degraded("vector index rebuilding"); - assert_eq!(health.as_str(), "degraded"); - assert_eq!(health.reason(), Some("vector index rebuilding")); - // A degraded driver is still the bound driver. - assert!(health.is_usable()); - assert_eq!(health.to_string(), "degraded: vector index rebuilding"); -} - -#[test] -fn down_carries_a_reason_and_is_not_usable() { - let health = MemoryHealth::down("connection refused"); - assert_eq!(health.as_str(), "down"); - assert_eq!(health.reason(), Some("connection refused")); - assert!(!health.is_usable()); - assert_eq!(health.to_string(), "down: connection refused"); -} - -#[test] -fn health_serializes_with_a_stable_status_discriminant() { - assert_eq!( - serde_json::to_value(MemoryHealth::Ready).unwrap(), - json!({ "status": "ready" }) - ); - assert_eq!( - serde_json::to_value(MemoryHealth::degraded("slow")).unwrap(), - json!({ "status": "degraded", "reason": "slow" }) - ); - assert_eq!( - serde_json::to_value(MemoryHealth::down("gone")).unwrap(), - json!({ "status": "down", "reason": "gone" }) - ); -} - -#[test] -fn health_round_trips_through_serde() { - for health in [ - MemoryHealth::Ready, - MemoryHealth::degraded("reindexing"), - MemoryHealth::down("auth expired"), - ] { - let encoded = serde_json::to_string(&health).unwrap(); - let decoded: MemoryHealth = serde_json::from_str(&encoded).unwrap(); - assert_eq!(decoded, health); - } -} - -#[test] -fn health_constructors_match_their_variants() { - assert_eq!( - MemoryHealth::degraded("x"), - MemoryHealth::Degraded { - reason: "x".to_string() - } - ); - assert_eq!( - MemoryHealth::down("y"), - MemoryHealth::Down { - reason: "y".to_string() - } - ); -} - -#[test] -fn degraded_without_a_reason_is_rejected_on_the_wire() { - // `reason` is mandatory: a degraded/down driver that explains nothing is - // useless in status output, so the contract refuses to decode it. - assert!(serde_json::from_value::(json!({ "status": "degraded" })).is_err()); - assert!(serde_json::from_value::(json!({ "status": "down" })).is_err()); -} diff --git a/src/openhuman/memory/api/host/cloud_providers.rs b/src/openhuman/memory/api/host/cloud_providers.rs deleted file mode 100644 index dfdd6c3f73..0000000000 --- a/src/openhuman/memory/api/host/cloud_providers.rs +++ /dev/null @@ -1,855 +0,0 @@ -//! Cloud provider credential schema. -//! -//! Each entry in `Config::cloud_providers` represents one configured LLM -//! backend. Providers are keyed by a user-chosen `slug` (e.g. `"openai"`, -//! `"my-deepseek"`). The factory in `inference::provider::factory` -//! resolves workload-to-provider strings against this list at runtime using -//! the grammar `":"`. -//! -//! Legacy configs that use `type`/`default_model` are migrated in-memory on -//! load via `migrate_legacy_fields()`. - -use schemars::JsonSchema; -use serde::{Deserialize, Serialize}; - -#[derive(Debug, Clone, Copy, PartialEq, Eq)] -pub struct BuiltinCloudProvider { - pub slug: &'static str, - pub label: &'static str, - pub endpoint: &'static str, - pub auth_style: AuthStyle, -} - -pub const BUILTIN_CLOUD_PROVIDERS: &[BuiltinCloudProvider] = &[ - BuiltinCloudProvider { - slug: "openhuman", - label: "OpenHuman", - endpoint: "https://api.openhuman.ai/v1", - auth_style: AuthStyle::OpenhumanJwt, - }, - BuiltinCloudProvider { - slug: "openai", - label: "OpenAI", - endpoint: "https://api.openai.com/v1", - auth_style: AuthStyle::Bearer, - }, - BuiltinCloudProvider { - slug: "anthropic", - label: "Anthropic", - endpoint: "https://api.anthropic.com/v1", - auth_style: AuthStyle::Anthropic, - }, - BuiltinCloudProvider { - slug: "openrouter", - label: "OpenRouter", - endpoint: "https://openrouter.ai/api/v1", - auth_style: AuthStyle::Bearer, - }, - BuiltinCloudProvider { - slug: "orcarouter", - label: "OrcaRouter", - endpoint: "https://api.orcarouter.ai/v1", - auth_style: AuthStyle::Bearer, - }, - BuiltinCloudProvider { - slug: "gmi", - label: "GMI", - endpoint: "https://api.gmi-serving.com/v1", - auth_style: AuthStyle::Bearer, - }, - BuiltinCloudProvider { - slug: "fireworks", - label: "Fireworks", - endpoint: "https://api.fireworks.ai/inference/v1", - auth_style: AuthStyle::Bearer, - }, - BuiltinCloudProvider { - slug: "moonshot", - label: "Kimi (Moonshot)", - endpoint: "https://api.moonshot.ai/v1", - auth_style: AuthStyle::Bearer, - }, - BuiltinCloudProvider { - slug: "groq", - label: "Groq", - endpoint: "https://api.groq.com/openai/v1", - auth_style: AuthStyle::Bearer, - }, - BuiltinCloudProvider { - slug: "mistral", - label: "Mistral", - endpoint: "https://api.mistral.ai/v1", - auth_style: AuthStyle::Bearer, - }, - BuiltinCloudProvider { - slug: "deepseek", - label: "DeepSeek", - endpoint: "https://api.deepseek.com/v1", - auth_style: AuthStyle::Bearer, - }, - BuiltinCloudProvider { - slug: "together", - label: "Together AI", - endpoint: "https://api.together.xyz/v1", - auth_style: AuthStyle::Bearer, - }, - BuiltinCloudProvider { - slug: "google", - label: "Google Gemini", - endpoint: "https://generativelanguage.googleapis.com/v1beta/openai", - auth_style: AuthStyle::Bearer, - }, - BuiltinCloudProvider { - slug: "cerebras", - label: "Cerebras", - endpoint: "https://api.cerebras.ai/v1", - auth_style: AuthStyle::Bearer, - }, - BuiltinCloudProvider { - slug: "xai", - label: "xAI", - endpoint: "https://api.x.ai/v1", - auth_style: AuthStyle::Bearer, - }, - BuiltinCloudProvider { - slug: "huggingface", - label: "Hugging Face", - endpoint: "https://router.huggingface.co/v1", - auth_style: AuthStyle::Bearer, - }, - BuiltinCloudProvider { - slug: "nvidia", - label: "NVIDIA", - endpoint: "https://integrate.api.nvidia.com/v1", - auth_style: AuthStyle::Bearer, - }, - BuiltinCloudProvider { - slug: "zai", - label: "Z.AI", - endpoint: "https://api.z.ai/api/paas/v4", - auth_style: AuthStyle::Bearer, - }, - BuiltinCloudProvider { - slug: "minimax", - label: "MiniMax", - // MiniMax exposes a full OpenAI-compatible surface at `/v1` - // (`/v1/chat/completions`, `/v1/models`). The previous `/anthropic` - // base + Anthropic auth pointed at MiniMax's Messages-protocol API, - // which OpenHuman does not speak — it only builds OpenAI-style - // `/chat/completions` and `/models` — so both chat and model-listing - // 404'd (`/anthropic/chat/completions`, `/anthropic/models`). The - // 404 on model-listing was Sentry TAURI-RUST-8X3. Use the `/v1` - // OpenAI surface with Bearer auth so both paths resolve. - endpoint: "https://api.minimax.io/v1", - auth_style: AuthStyle::Bearer, - }, - BuiltinCloudProvider { - slug: "stepfun", - label: "StepFun", - endpoint: "https://api.stepfun.ai/step_plan/v1", - auth_style: AuthStyle::Bearer, - }, - BuiltinCloudProvider { - slug: "kilocode", - label: "Kilo Code", - endpoint: "https://api.kilo.ai/api/gateway", - auth_style: AuthStyle::Bearer, - }, - BuiltinCloudProvider { - slug: "deepinfra", - label: "DeepInfra", - endpoint: "https://api.deepinfra.com/v1/openai", - auth_style: AuthStyle::Bearer, - }, - BuiltinCloudProvider { - slug: "novita", - label: "Novita", - endpoint: "https://api.novita.ai/v3/openai", - auth_style: AuthStyle::Bearer, - }, - BuiltinCloudProvider { - slug: "venice", - label: "Venice", - endpoint: "https://api.venice.ai/api/v1", - auth_style: AuthStyle::Bearer, - }, - BuiltinCloudProvider { - slug: "vercel-ai-gateway", - label: "Vercel AI Gateway", - endpoint: "https://ai-gateway.vercel.sh/v1", - auth_style: AuthStyle::Bearer, - }, - BuiltinCloudProvider { - slug: "sumopod", - label: "SumoPod", - endpoint: "https://ai.sumopod.com/v1", - auth_style: AuthStyle::Bearer, - }, - BuiltinCloudProvider { - slug: "modelscope", - label: "ModelScope", - endpoint: "https://api-inference.modelscope.cn/v1", - auth_style: AuthStyle::Bearer, - }, -]; - -fn builtin_cloud_provider(type_str: &str) -> Option<&'static BuiltinCloudProvider> { - BUILTIN_CLOUD_PROVIDERS - .iter() - .find(|provider| provider.slug == type_str) -} - -/// Whether `slug` matches a built-in cloud provider preset. -/// -/// The chat factory uses this to decide capability defaults (e.g. whether the -/// provider exposes the OpenAI Responses API) only for providers we ship and -/// therefore know the API surface of. Custom / user-defined slugs are treated -/// as unknown and keep the permissive defaults. -pub fn is_builtin_cloud_slug(slug: &str) -> bool { - builtin_cloud_provider(slug).is_some() -} - -/// Whether a built-in cloud provider exposes the OpenAI **Responses API** -/// (`/v1/responses`). -/// -/// Only OpenAI's first-party endpoint serves `/responses`; every other built-in -/// preset (DeepSeek, Groq, Mistral, Fireworks, …) is chat-completions-only. -/// Enabling the chat-completions-404 → `/responses` fallback for those -/// guarantees a second 404 against an endpoint that does not exist, which floods -/// Sentry with an empty-body `" Responses API error:"` event -/// (TAURI-RUST-5EN — same class as the local-provider TAURI-RUST-59Y fix). The -/// factory consults this to build chat-completions-only built-ins with -/// `new_no_responses_fallback`. -/// -/// Custom / unknown slugs are intentionally NOT covered here (see -/// [`is_builtin_cloud_slug`]): a user-defined OpenAI-compatible endpoint may be -/// a genuine OpenAI proxy that does support `/responses`, so the factory keeps -/// the fallback for those. -pub fn builtin_cloud_supports_responses_api(slug: &str) -> bool { - matches!(slug, "openai") -} - -/// Extract the lowercased authority host from an endpoint URL, dropping the -/// scheme, any userinfo, the port, and the path. Returns `None` when no host -/// can be parsed. Tolerant of a missing scheme and of IPv6 literals. -pub fn endpoint_host(endpoint: &str) -> Option { - let s = endpoint.trim(); - // Drop the scheme (`https://…`); tolerate a bare `host/path` form. - let after_scheme = s.split_once("://").map(|(_, rest)| rest).unwrap_or(s); - // The authority ends at the first path / query / fragment delimiter. - let authority = after_scheme - .split(['/', '?', '#']) - .next() - .unwrap_or(after_scheme); - // Strip any `user:pass@` userinfo prefix. - let host_port = authority - .rsplit_once('@') - .map(|(_, host)| host) - .unwrap_or(authority); - // Strip the port, handling bracketed IPv6 literals (`[::1]:8080`). - let host = if let Some(rest) = host_port.strip_prefix('[') { - rest.split_once(']').map(|(h, _)| h).unwrap_or(rest) - } else { - host_port - .rsplit_once(':') - .map(|(h, _)| h) - .unwrap_or(host_port) - }; - let host = host.trim().to_ascii_lowercase(); - (!host.is_empty()).then_some(host) -} - -/// Whether `host` is the authority host of any built-in cloud **inference** -/// provider (e.g. `openrouter.ai`, `api.openai.com`, `api.groq.com`). -/// -/// Derived entirely from [`BUILTIN_CLOUD_PROVIDERS`] so the set stays in sync -/// with the provider registry. `host` is compared case-insensitively against -/// each preset's [`endpoint_host`]. -/// -/// # Why this exists -/// -/// `config.api_url` is overloaded: it is the chat/inference endpoint, but -/// `api::config::effective_backend_api_url` also reuses it as the -/// OpenHuman **backend** base for team/billing/auth calls. A BYO user who -/// points `api_url` at a provider's canonical base (`https://openrouter.ai/api/v1`) -/// would otherwise have every backend domain call routed to the inference host -/// → 400/404 (TAURI-RUST-HW1: 4932 `GET /teams/me/usage` 400s from `openrouter.ai`). -/// The backend-URL resolver uses this to treat such hosts as non-backend and -/// fall back to the default backend chain — the cloud analogue of the local-AI -/// guard that fixed the Ollama case (OPENHUMAN-TAURI-51/-80/-7Z). -pub fn host_is_builtin_cloud_provider(host: &str) -> bool { - let host = host.trim().to_ascii_lowercase(); - if host.is_empty() { - return false; - } - BUILTIN_CLOUD_PROVIDERS - .iter() - .any(|p| endpoint_host(p.endpoint).as_deref() == Some(host.as_str())) -} - -/// Whether an endpoint **host** is a known cloud host that does NOT serve the -/// OpenAI Responses API (`/v1/responses`) — i.e. it is chat-completions-only, -/// regardless of which user slug points at it. -/// -/// Derived entirely from [`BUILTIN_CLOUD_PROVIDERS`]: a host is chat-only when -/// some built-in preset uses it AND no preset at that host advertises the -/// Responses API (only OpenAI's `api.openai.com` does). This closes the -/// custom-slug gap behind the builtin-slug gate -/// ([`builtin_cloud_supports_responses_api`]): a user slug pointed at, e.g., -/// `integrate.api.nvidia.com` must never attempt `/responses` (TAURI-RUST-5A1), -/// while a genuinely unknown proxy host keeps the permissive fallback so a real -/// OpenAI proxy still gets `/responses`. -pub fn endpoint_host_is_chat_completions_only(endpoint: &str) -> bool { - let Some(host) = endpoint_host(endpoint) else { - return false; - }; - let mut matched_chat_only = false; - for provider in BUILTIN_CLOUD_PROVIDERS { - if endpoint_host(provider.endpoint).as_deref() == Some(host.as_str()) { - if builtin_cloud_supports_responses_api(provider.slug) { - // A Responses-capable built-in lives at this host → not chat-only. - return false; - } - matched_chat_only = true; - } - } - matched_chat_only -} - -/// Authentication header style for a cloud provider. -/// -/// Wire format is lowercase (e.g. `"bearer"`). Determines which HTTP headers -/// are attached when calling the provider's API. -#[derive(Debug, Clone, Copy, Serialize, Deserialize, JsonSchema, PartialEq, Eq, Default)] -#[serde(rename_all = "lowercase")] -pub enum AuthStyle { - /// OpenAI-compatible: `Authorization: Bearer ` - #[default] - Bearer, - /// Anthropic: `x-api-key: ` + `anthropic-version: 2023-06-01` - Anthropic, - /// OpenHuman session JWT (injected by the backend provider, not stored here). - OpenhumanJwt, - /// No auth header — e.g. local Ollama. - None, -} - -impl AuthStyle { - pub fn as_str(&self) -> &'static str { - match self { - Self::Bearer => "bearer", - Self::Anthropic => "anthropic", - Self::OpenhumanJwt => "openhuman_jwt", - Self::None => "none", - } - } -} - -/// Endpoint config for one cloud LLM provider. -/// -/// **Note on secrets**: API keys are NOT stored on this struct. They live in -/// `auth-profiles.json` via `security::credentials::AuthService`, -/// keyed by `provider:` (falling back to bare `` for legacy -/// entries). The factory looks up the token at call time via -/// `inference::provider::factory::auth_key_for_slug`. -/// -/// ## Back-compat -/// -/// Old configs may have `type` and `default_model` fields. These are -/// tolerated on read (via `legacy_type` / `default_model`) but never written. -/// Call `migrate_legacy_fields()` after deserialising. -#[derive(Debug, Clone, Serialize, Deserialize, JsonSchema, PartialEq, Eq)] -#[serde(default)] -pub struct CloudProviderCreds { - /// Opaque stable id, e.g. `"p_openai_a8c3f"`. Never shown in the UI. - /// Generated once by [`generate_provider_id`] and never changes. - pub id: String, - /// Routing key chosen by the user or seeded from the legacy type. - /// Lower-case alphanumeric + `-`. Must be unique per config and not in the - /// reserved list (see [`is_slug_reserved`]). The factory resolves - /// `":"` strings against this field. - pub slug: String, - /// Human-readable display label, supplied by the frontend. Not used in routing. - pub label: String, - /// OpenAI-compatible base URL (`/models`, `/chat/completions` etc. are appended). - pub endpoint: String, - /// Authentication header style. - pub auth_style: AuthStyle, - - // ── Back-compat: old `type` field ─────────────────────────────────────── - /// Legacy discriminator written by older builds. Read-only; never emitted. - #[serde(rename = "type", default, skip_serializing)] - pub legacy_type: Option, - - // ── Back-compat: old `default_model` field ────────────────────────────── - /// Legacy default model written by older builds. Read-only; never emitted. - #[serde(default, skip_serializing)] - pub default_model: Option, -} - -impl Default for CloudProviderCreds { - fn default() -> Self { - Self { - id: String::new(), - slug: String::new(), - label: String::new(), - endpoint: String::new(), - auth_style: AuthStyle::Bearer, - legacy_type: None, - default_model: None, - } - } -} - -/// Reserved slugs that may not be used for user-configured providers. -/// These are sentinels in the factory's routing grammar. -/// -/// `ollama` is deliberately NOT reserved: the AI settings panel registers an -/// `ollama` `cloud_providers` entry so `list_configured_models` can resolve -/// the user's chosen base_url for the model dropdown. The factory's chat -/// routing is unaffected — the `ollama:` prefix branch in -/// `factory::create_chat_provider_from_string` fires before the -/// `:` cloud-provider lookup, so a synthetic `ollama` entry -/// never reaches `make_cloud_provider_by_slug`. When no `cloud_providers` -/// row exists (config drift, upgrade from a build that only persisted -/// `config.local_ai.base_url`, flush-vs-probe race), -/// `inference::provider::ops::list_configured_models` -/// falls back to a synthetic entry via `synthesize_local_runtime_entry` -/// (Sentry TAURI-RUST-28Z fix). The same fallback applies to `lmstudio`. -pub fn is_slug_reserved(s: &str) -> bool { - matches!(s.trim(), "" | "cloud" | "openhuman" | "pid") -} - -/// Apply legacy field migration in-place. -/// -/// Idempotent: only fills in empty fields from the legacy `type`/`default_model` -/// values. Safe to call on already-migrated entries. -pub fn migrate_legacy_fields(entry: &mut CloudProviderCreds) { - let legacy_type = entry.legacy_type.clone().unwrap_or_default(); - let lt = legacy_type.trim(); - - // Slug from legacy type when missing. - if entry.slug.is_empty() && !lt.is_empty() { - entry.slug = lt.to_string(); - log::debug!( - "[config][cloud_providers] migrated slug from legacy type='{}' id={}", - lt, - entry.id - ); - } - - // Label from static map when missing. - if entry.label.is_empty() { - entry.label = legacy_label_for(if entry.slug.is_empty() { - lt - } else { - &entry.slug - }) - .to_string(); - log::debug!( - "[config][cloud_providers] migrated label='{}' for slug='{}' id={}", - entry.label, - entry.slug, - entry.id - ); - } - - // Endpoint from legacy defaults when missing. - if entry.endpoint.is_empty() { - let ep = legacy_default_endpoint(lt); - if !ep.is_empty() { - entry.endpoint = ep.to_string(); - } - } - - // Auth style from legacy type when still at default Bearer. - if entry.auth_style == AuthStyle::Bearer { - if let Some(provider) = builtin_cloud_provider(lt) { - entry.auth_style = provider.auth_style; - } - } -} - -/// Map a legacy type string (or slug) to a human-readable label. -fn legacy_label_for(type_str: &str) -> &'static str { - builtin_cloud_provider(type_str) - .map(|provider| provider.label) - .unwrap_or("Custom") -} - -/// Map a legacy type string to its well-known default endpoint. -fn legacy_default_endpoint(type_str: &str) -> &'static str { - builtin_cloud_provider(type_str) - .map(|provider| provider.endpoint) - .unwrap_or("") -} - -/// Generate a short opaque id for a new provider entry. -/// -/// Format: `"p__<5 random alphanumerics>"`, e.g. `"p_openai_a8c3f"`. -/// The random suffix is not cryptographically strong — it only needs to be -/// unique within a single user's config file. -pub fn generate_provider_id(slug: &str) -> String { - use std::time::{SystemTime, UNIX_EPOCH}; - // Cheap pseudo-random from timestamp nanoseconds — adequate for local - // config uniqueness without pulling in a PRNG crate. - let nanos = SystemTime::now() - .duration_since(UNIX_EPOCH) - .unwrap_or_default() - .subsec_nanos(); - let chars: &[u8] = b"abcdefghijklmnopqrstuvwxyz0123456789"; - let mut suffix = String::with_capacity(5); - let mut seed = nanos as usize; - for _ in 0..5 { - suffix.push(chars[seed % chars.len()] as char); - seed = seed - .wrapping_mul(6364136223846793005) - .wrapping_add(1442695040888963407); - seed = (seed >> 33) ^ seed; - } - // Sanitise slug to only alphanumeric + '-' for the id prefix. - let safe_slug: String = slug - .chars() - .map(|c| { - if c.is_ascii_alphanumeric() || c == '-' { - c - } else { - '_' - } - }) - .take(20) - .collect(); - format!("p_{}_{}", safe_slug, suffix) -} - -// ── Back-compat type alias ────────────────────────────────────────────────── -// Kept so existing code that imports `CloudProviderType` compiles without -// sweeping changes. New code should use `AuthStyle` directly. - -/// Legacy discriminator enum. **Deprecated**: use `AuthStyle` on new entries. -/// Retained only to satisfy callers that still pattern-match on -/// `CloudProviderType` (e.g. the migration module). Will be removed once all -/// call sites are updated to slug-keyed lookups. -#[derive(Debug, Clone, Serialize, Deserialize, JsonSchema, PartialEq, Eq)] -#[serde(rename_all = "lowercase")] -pub enum CloudProviderType { - Openhuman, - Openai, - Anthropic, - Openrouter, - Orcarouter, - Custom, -} - -impl CloudProviderType { - /// Well-known default base URL for each provider type. - pub fn default_endpoint(&self) -> &'static str { - match self { - Self::Openhuman => "https://api.openhuman.ai/v1", - Self::Openai => "https://api.openai.com/v1", - Self::Anthropic => "https://api.anthropic.com/v1", - Self::Openrouter => "https://openrouter.ai/api/v1", - Self::Orcarouter => "https://api.orcarouter.ai/v1", - Self::Custom => "", - } - } - - /// Human-readable label used in logs and error messages. - pub fn label(&self) -> &'static str { - match self { - Self::Openhuman => "OpenHuman", - Self::Openai => "OpenAI", - Self::Anthropic => "Anthropic", - Self::Openrouter => "OpenRouter", - Self::Orcarouter => "OrcaRouter", - Self::Custom => "Custom", - } - } - - /// Lowercase wire-format string (matches JSON serialisation). - pub fn as_str(&self) -> &'static str { - match self { - Self::Openhuman => "openhuman", - Self::Openai => "openai", - Self::Anthropic => "anthropic", - Self::Openrouter => "openrouter", - Self::Orcarouter => "orcarouter", - Self::Custom => "custom", - } - } - - /// Corresponding `AuthStyle`. - pub fn auth_style(&self) -> AuthStyle { - match self { - Self::Openhuman => AuthStyle::OpenhumanJwt, - Self::Anthropic => AuthStyle::Anthropic, - _ => AuthStyle::Bearer, - } - } -} - -#[cfg(test)] -mod tests { - use super::{ - builtin_cloud_supports_responses_api, endpoint_host, - endpoint_host_is_chat_completions_only, host_is_builtin_cloud_provider, - is_builtin_cloud_slug, is_slug_reserved, migrate_legacy_fields, AuthStyle, - CloudProviderCreds, BUILTIN_CLOUD_PROVIDERS, - }; - - #[test] - fn reserved_slugs() { - for s in ["", " ", "cloud", "openhuman", "pid"] { - assert!(is_slug_reserved(s), "{s:?} must stay reserved"); - } - } - - // Regression: `ollama` was previously reserved, which made the AI settings - // panel unable to persist an `ollama` cloud_providers entry — so the - // model-list dropdown failed with "no cloud provider with id or slug - // 'ollama' found". The factory's chat routing is unaffected by this - // change because the `ollama:` prefix branch fires before any - // cloud_providers lookup. - #[test] - fn ollama_and_lmstudio_are_not_reserved() { - assert!( - !is_slug_reserved("ollama"), - "ollama must be usable as a cloud_providers slug for the /models probe" - ); - assert!( - !is_slug_reserved("lmstudio"), - "lmstudio is a free-form OpenAI-compatible slug" - ); - } - - #[test] - fn builtin_cloud_provider_defaults_cover_phase_one_presets() { - for (slug, label, endpoint, auth_style) in [ - ( - "groq", - "Groq", - "https://api.groq.com/openai/v1", - AuthStyle::Bearer, - ), - ( - "deepseek", - "DeepSeek", - "https://api.deepseek.com/v1", - AuthStyle::Bearer, - ), - ( - "minimax", - "MiniMax", - "https://api.minimax.io/v1", - AuthStyle::Bearer, - ), - ( - "sumopod", - "SumoPod", - "https://ai.sumopod.com/v1", - AuthStyle::Bearer, - ), - ( - "modelscope", - "ModelScope", - "https://api-inference.modelscope.cn/v1", - AuthStyle::Bearer, - ), - ] { - let mut entry = CloudProviderCreds { - id: format!("p_{slug}"), - legacy_type: Some(slug.to_string()), - ..Default::default() - }; - migrate_legacy_fields(&mut entry); - - assert_eq!(entry.slug, slug); - assert_eq!(entry.label, label); - assert_eq!(entry.endpoint, endpoint); - assert_eq!(entry.auth_style, auth_style); - } - } - - #[test] - fn builtin_cloud_provider_slugs_are_unique() { - let mut slugs = std::collections::HashSet::new(); - for provider in BUILTIN_CLOUD_PROVIDERS { - assert!( - slugs.insert(provider.slug), - "duplicate built-in cloud provider slug {}", - provider.slug - ); - } - } - - #[test] - fn is_builtin_cloud_slug_matches_presets_only() { - for slug in ["openai", "deepseek", "groq", "mistral"] { - assert!(is_builtin_cloud_slug(slug), "{slug} is a built-in preset"); - } - for slug in ["my-proxy", "custom-openai", "totally-unknown", ""] { - assert!( - !is_builtin_cloud_slug(slug), - "{slug:?} is not a built-in preset" - ); - } - } - - #[test] - fn only_openai_builtin_exposes_responses_api() { - assert!(builtin_cloud_supports_responses_api("openai")); - for slug in ["deepseek", "groq", "mistral", "fireworks", "together"] { - assert!( - !builtin_cloud_supports_responses_api(slug), - "{slug} is chat-completions-only and must not advertise the Responses API" - ); - } - } - - /// Drift guard (TAURI-RUST-5EN): couple the capability helper to the - /// preset list so adding a new built-in that wrongly claims the Responses - /// API — or renaming `openai` — fails CI rather than silently re-enabling - /// the guaranteed-404 `/responses` fallback. OpenAI's first-party endpoint - /// is the only built-in that serves `/v1/responses`. - #[test] - fn responses_api_capability_is_coupled_to_the_preset_list() { - for provider in BUILTIN_CLOUD_PROVIDERS { - let expected = provider.slug == "openai"; - assert_eq!( - builtin_cloud_supports_responses_api(provider.slug), - expected, - "built-in {} Responses-API capability drifted from the openai-only invariant", - provider.slug - ); - } - } - - #[test] - fn endpoint_host_parses_scheme_path_and_port() { - assert_eq!( - endpoint_host("https://integrate.api.nvidia.com/v1").as_deref(), - Some("integrate.api.nvidia.com") - ); - // Missing scheme, mixed case, trailing path. - assert_eq!( - endpoint_host("API.OpenAI.com/v1/chat").as_deref(), - Some("api.openai.com") - ); - // Userinfo + explicit port are stripped. - assert_eq!( - endpoint_host("https://user:pass@api.groq.com:443/openai/v1").as_deref(), - Some("api.groq.com") - ); - // Bracketed IPv6 literal with port. - assert_eq!( - endpoint_host("http://[::1]:8080/v1").as_deref(), - Some("::1") - ); - assert_eq!(endpoint_host(" ").as_deref(), None); - } - - /// TAURI-RUST-HW1: the backend-URL resolver uses this to reroute backend - /// domain calls away from a BYO inference host. Every built-in provider host - /// must be recognised; OpenHuman backend hosts and unknown proxies must not. - #[test] - fn host_is_builtin_cloud_provider_recognises_inference_hosts() { - for host in [ - "openrouter.ai", - "api.openai.com", - "api.anthropic.com", - "api.groq.com", - "generativelanguage.googleapis.com", - "API.OPENAI.COM", // case-insensitive - ] { - assert!( - host_is_builtin_cloud_provider(host), - "{host} is a built-in cloud inference host" - ); - } - for host in [ - "api.tinyhumans.ai", - "staging-api.tinyhumans.ai", - "my-backend.example", - "", - ] { - assert!( - !host_is_builtin_cloud_provider(host), - "{host:?} is not a built-in cloud inference host" - ); - } - // Every registry endpoint's own host must classify as builtin. - for provider in BUILTIN_CLOUD_PROVIDERS { - let host = endpoint_host(provider.endpoint).expect("preset endpoint has a host"); - assert!( - host_is_builtin_cloud_provider(&host), - "{} ({host}) must be recognised", - provider.slug - ); - } - } - - /// TAURI-RUST-5A1: a *custom* slug pointed at a known chat-only host (NVIDIA) - /// must be classified chat-only so the factory disables the guaranteed-404 - /// `/responses` fallback — the builtin-slug gate alone misses this because - /// the slug is not builtin. - #[test] - fn nvidia_host_is_chat_completions_only_regardless_of_slug() { - assert!(endpoint_host_is_chat_completions_only( - "https://integrate.api.nvidia.com/v1" - )); - // Other chat-only built-in hosts too. - for endpoint in [ - "https://api.deepseek.com/v1", - "https://api.groq.com/openai/v1", - "https://api.mistral.ai/v1", - ] { - assert!( - endpoint_host_is_chat_completions_only(endpoint), - "{endpoint} is a chat-completions-only built-in host" - ); - } - } - - #[test] - fn openai_host_and_unknown_proxies_keep_the_responses_fallback() { - // OpenAI's first-party host serves /responses — must NOT be gated off, - // even via a custom proxy slug pointed at it. - assert!(!endpoint_host_is_chat_completions_only( - "https://api.openai.com/v1" - )); - // Genuinely unknown proxy hosts keep the permissive default (they may be - // real OpenAI proxies that implement /responses). - for endpoint in [ - "https://my-llm-proxy.internal.example/v1", - "https://litellm.mycorp.dev/v1", - "", - ] { - assert!( - !endpoint_host_is_chat_completions_only(endpoint), - "{endpoint:?} is an unknown host and must keep the fallback" - ); - } - } - - /// Drift guard: the host-based gate must agree with the slug-based - /// capability for every built-in preset's own endpoint, so adding a preset - /// can't silently desync the two gates. - #[test] - fn host_gate_agrees_with_slug_capability_for_every_builtin() { - for provider in BUILTIN_CLOUD_PROVIDERS { - // OpenhumanJwt / Anthropic presets never route through the - // OpenAI-compatible Responses fallback; the gate only matters for - // the Bearer OpenAI-compatible hosts. - if provider.auth_style != AuthStyle::Bearer { - continue; - } - let host_chat_only = endpoint_host_is_chat_completions_only(provider.endpoint); - let slug_supports = builtin_cloud_supports_responses_api(provider.slug); - assert_eq!( - host_chat_only, !slug_supports, - "host gate for built-in {} disagrees with its slug capability", - provider.slug - ); - } - } -} diff --git a/src/openhuman/memory/api/host/composio.rs b/src/openhuman/memory/api/host/composio.rs deleted file mode 100644 index 2cbec2a958..0000000000 --- a/src/openhuman/memory/api/host/composio.rs +++ /dev/null @@ -1,878 +0,0 @@ -//! Composio value types — connections, capabilities, execute responses. -//! -//! Moved here from the host's `integrations::composio::types` because the -//! extracted memory sync pipelines read these fields directly on every run, and -//! a trait accessor per field would be absurd. They are inert serde data with -//! no behaviour and no dependencies beyond `serde`, so the contract crate's -//! dependency-light guarantee is unaffected. -//! -//! The Composio *client* deliberately did not come with them — see -//! `crate::openhuman::memory::core_impl::composio_host`. Its `Direct` variant wraps a host agent -//! tool, and mode dispatch is host policy. -//! -//! Domain types for the Composio integration. -//! -//! These mirror the response envelopes emitted by the openhuman backend under -//! `/agent-integrations/composio/*`. See: -//! - `src/routes/agentIntegrations/composio.ts` -//! - `src/controllers/agentIntegrations/composio/*.ts` -//! in the backend repo for the authoritative shapes. - -use serde::{Deserialize, Deserializer, Serialize}; - -/// Accepts either a JSON string or an object whose first matching field -/// (`slug`/`id`/`name`/`key`) is a string. Lets us tolerate upstream -/// shape drift where a previously-stringy field is now nested in an -/// object — e.g. `"toolkit": {"slug": "gmail", "logo": "…"}`. -fn de_string_or_object<'de, D: Deserializer<'de>>(d: D) -> Result { - use serde::de::Error; - let v = serde_json::Value::deserialize(d)?; - match v { - serde_json::Value::String(s) => Ok(s), - serde_json::Value::Object(map) => { - for key in ["slug", "id", "name", "key"] { - if let Some(serde_json::Value::String(s)) = map.get(key) { - return Ok(s.clone()); - } - } - Err(D::Error::custom( - "expected string or object with slug/id/name/key field", - )) - } - other => Err(D::Error::custom(format!( - "expected string, got {}", - match other { - serde_json::Value::Null => "null", - serde_json::Value::Bool(_) => "bool", - serde_json::Value::Number(_) => "number", - serde_json::Value::Array(_) => "array", - _ => "unknown", - } - ))), - } -} - -/// Like [`de_string_or_object`] but optional and resilient: missing / -/// null / unrecognized object shapes return `None` instead of erroring. -fn de_opt_string_or_object<'de, D: Deserializer<'de>>(d: D) -> Result, D::Error> { - let v = Option::::deserialize(d)?; - Ok(match v { - None | Some(serde_json::Value::Null) => None, - Some(serde_json::Value::String(s)) => Some(s), - Some(serde_json::Value::Object(map)) => { - let mut found = None; - for key in ["state", "value", "slug", "id", "name", "key"] { - if let Some(serde_json::Value::String(s)) = map.get(key) { - found = Some(s.clone()); - break; - } - } - found - } - _ => None, - }) -} - -// ── Toolkits ──────────────────────────────────────────────────────── - -/// One toolkit from the live Composio catalog, forwarded verbatim from the -/// backend (`GET /agent-integrations/composio/toolkits`). -/// -/// The core does not interpret these fields — it passes them straight through -/// to the desktop UI so the app no longer hardcodes toolkit display metadata -/// (see the workspace `COMPOSIO_DYNAMIC_CATALOG_PLAN.md`). Everything except -/// `slug` is best-effort; backends predating the dynamic catalog omit the -/// whole `catalog` array, in which case the UI falls back to local metadata. -#[derive(Debug, Clone, Default, Serialize, Deserialize)] -pub struct ComposioToolkitCatalogEntry { - /// Toolkit slug as Composio emits it, e.g. `"googlecalendar"`. - pub slug: String, - /// Human-readable name, e.g. `"Google Calendar"`. - #[serde(default)] - pub name: String, - /// Composio-hosted logo URL (`meta.logo`). - #[serde(default, skip_serializing_if = "Option::is_none")] - pub logo: Option, - /// Short description (`meta.description`). - #[serde(default, skip_serializing_if = "Option::is_none")] - pub description: Option, - /// Composio category names (`meta.categories`). - #[serde(default, skip_serializing_if = "Vec::is_empty")] - pub categories: Vec, - /// Whether the user can connect/use this toolkit (passed the backend gate). - #[serde(default, skip_serializing_if = "Option::is_none")] - pub enabled: Option, -} - -/// Response body of `GET /agent-integrations/composio/toolkits`. -#[derive(Debug, Clone, Default, Serialize, Deserialize)] -pub struct ComposioToolkitsResponse { - /// Server-enforced toolkit allowlist, e.g. `["gmail", "notion"]`. - #[serde(default)] - pub toolkits: Vec, - /// Rich render model from the live Composio catalog. Optional — empty when - /// the backend predates the dynamic catalog. Forwarded as-is to the UI. - #[serde(default, skip_serializing_if = "Vec::is_empty")] - pub catalog: Vec, -} - -/// One row in OpenHuman's local Composio capability matrix. -/// -/// Unlike `ComposioToolkitsResponse`, this is not tied to a signed-in -/// backend/direct Composio session. It describes what this core build knows -/// how to do for each toolkit: whether the toolkit has a native provider -/// implementation, a curated tool catalog, profile/sync hooks, and memory -/// ingestion support. -#[derive(Debug, Clone, Serialize, Deserialize)] -pub struct ComposioCapability { - pub toolkit: String, - pub description: String, - pub native_provider: bool, - pub curated_tools: bool, - pub curated_tool_count: usize, - pub tool_execution: bool, - pub user_profile: bool, - pub initial_sync: bool, - pub periodic_sync: bool, - pub sync_interval_secs: Option, - pub trigger_webhooks: bool, - pub memory_ingest: bool, -} - -/// Response body of `composio.list_capabilities`. -#[derive(Debug, Clone, Default, Serialize, Deserialize)] -pub struct ComposioCapabilitiesResponse { - #[serde(default)] - pub capabilities: Vec, -} - -/// Response body of `composio.list_agent_ready_toolkits`. -/// -/// Sorted slugs that have a curated agent catalog — the frontend -/// uses this to decide whether to label a connected toolkit as -/// "preview / agent integration coming soon". See #2283. -#[derive(Debug, Clone, Default, Serialize, Deserialize)] -pub struct ComposioAgentReadyToolkitsResponse { - #[serde(default)] - pub toolkits: Vec, -} - -// ── Connections ───────────────────────────────────────────────────── - -/// One connected Composio account (OAuth integration instance). -#[derive(Debug, Clone, Serialize, Deserialize)] -pub struct ComposioConnection { - /// Composio connection id (what you DELETE to disconnect). - pub id: String, - /// Toolkit slug, e.g. `"gmail"`. - pub toolkit: String, - /// Connection status — `"ACTIVE"`, `"CONNECTED"`, `"PENDING"`, … - pub status: String, - /// ISO timestamp (backend passes this through from Composio). - #[serde(rename = "createdAt", default, skip_serializing_if = "Option::is_none")] - pub created_at: Option, - /// Account email — populated from the cached provider profile when - /// the toolkit reports an email address (e.g. Gmail, Google Calendar, - /// Google Sheets). Lets the UI picker show "Gmail · user@example.com" - /// instead of a generic "Account N" label. - #[serde( - rename = "accountEmail", - default, - skip_serializing_if = "Option::is_none" - )] - pub account_email: Option, - /// Workspace or team display name — populated for workspace-based - /// services (e.g. Slack: user display name / team name, Notion: workspace - /// name). Used by the picker when no email is available. - #[serde(default, skip_serializing_if = "Option::is_none")] - pub workspace: Option, - /// Screen name or handle — populated for username-based services - /// (e.g. GitHub login, Twitter handle). Used by the picker as a - /// last-resort identity hint after email and workspace. - #[serde(default, skip_serializing_if = "Option::is_none")] - pub username: Option, -} - -impl ComposioConnection { - /// Return the toolkit slug in the canonical form used by provider - /// lookup, prompt injection, and tool-action prefix matching. - pub fn normalized_toolkit(&self) -> String { - self.toolkit.trim().to_ascii_lowercase() - } - - /// Whether this row represents a usable connection. - /// - /// The web UI already treats status case-insensitively. Keep the - /// core-side chat/runtime filters aligned so a backend spelling such - /// as `connected` cannot display as connected in Settings while - /// disappearing from the agent's integration surface. - pub fn is_active(&self) -> bool { - let status = self.status.trim(); - status.eq_ignore_ascii_case("ACTIVE") || status.eq_ignore_ascii_case("CONNECTED") - } -} - -/// Response body of `GET /agent-integrations/composio/connections`. -#[derive(Debug, Clone, Default, Serialize, Deserialize)] -pub struct ComposioConnectionsResponse { - #[serde(default)] - pub connections: Vec, -} - -/// Response body of `POST /agent-integrations/composio/authorize`. -#[derive(Debug, Clone, Serialize, Deserialize)] -pub struct ComposioAuthorizeResponse { - /// Composio-hosted OAuth URL the user opens in a browser. - #[serde(rename = "connectUrl")] - pub connect_url: String, - /// Composio connection id created by this authorize call. - #[serde(rename = "connectionId")] - pub connection_id: String, -} - -/// Response body of `DELETE /agent-integrations/composio/connections/:id`. -#[derive(Debug, Clone, Serialize, Deserialize)] -pub struct ComposioDeleteResponse { - #[serde(default)] - pub deleted: bool, - #[serde(default)] - pub memory_chunks_deleted: usize, -} - -// ── Tools ─────────────────────────────────────────────────────────── - -/// OpenAI function-calling schema returned by the backend for each tool. -/// -/// The backend wraps Composio's upstream shape; we keep the `type` + -/// `function` envelope so callers can forward directly into an LLM. -#[derive(Debug, Clone, Serialize, Deserialize)] -pub struct ComposioToolSchema { - #[serde(rename = "type", default = "default_function_type")] - pub kind: String, - pub function: ComposioToolFunction, -} - -fn default_function_type() -> String { - "function".to_string() -} - -#[derive(Debug, Clone, Serialize, Deserialize)] -pub struct ComposioToolFunction { - /// Composio action slug, e.g. `"GMAIL_SEND_EMAIL"`. - pub name: String, - /// Human-readable description shown to the model. - #[serde(default)] - pub description: Option, - /// JSON schema for the tool's INPUT parameters. - #[serde(default)] - pub parameters: Option, - /// JSON schema describing the tool's OUTPUT/return-value shape, when the - /// upstream listing publishes one. Composio's v3 `/tools` endpoint calls - /// this `output_parameters` — documented as "Schema definition of return - /// values from the tool" - /// () — - /// alongside `input_parameters`. `None` means "unknown" (not "empty"): - /// the backend-proxied `/agent-integrations/composio/tools` path is - /// opaque to this crate and may not forward it, and not every Composio - /// action publishes an output schema. - #[serde(default)] - pub output_parameters: Option, -} - -/// Response body of `GET /agent-integrations/composio/tools`. -#[derive(Debug, Clone, Default, Serialize, Deserialize)] -pub struct ComposioToolsResponse { - #[serde(default)] - pub tools: Vec, -} - -// ── Execute ───────────────────────────────────────────────────────── - -/// Response body of `POST /agent-integrations/composio/execute`. -#[derive(Debug, Clone, Serialize, Deserialize)] -pub struct ComposioExecuteResponse { - /// Raw result from the upstream provider. - #[serde(default)] - pub data: serde_json::Value, - /// Did the provider report success? - #[serde(default)] - pub successful: bool, - /// Provider error message if any. - #[serde(default)] - pub error: Option, - /// Amount charged to the caller (base + margin) in USD. - #[serde(rename = "costUsd", default)] - pub cost_usd: f64, - /// Backend-rendered compact markdown for known tools (set by - /// backend PR tinyhumansai/backend#683). When present and non-empty - /// callers should prefer this over `data` for LLM/CLI consumption. - #[serde(rename = "markdownFormatted", default)] - pub markdown_formatted: Option, -} - -// ── GitHub repos + triggers ───────────────────────────────────────── - -/// One repository returned by `GET /agent-integrations/composio/github/repos`. -#[derive(Debug, Clone, Serialize, Deserialize)] -pub struct ComposioGithubRepo { - pub owner: String, - pub repo: String, - #[serde(rename = "fullName")] - pub full_name: String, - #[serde(default)] - pub private: Option, - #[serde(rename = "defaultBranch", default)] - pub default_branch: Option, - #[serde(rename = "htmlUrl", default)] - pub html_url: Option, -} - -/// Response body of `GET /agent-integrations/composio/github/repos`. -#[derive(Debug, Clone, Serialize, Deserialize)] -pub struct ComposioGithubReposResponse { - #[serde(rename = "connectionId")] - pub connection_id: String, - #[serde(default, rename = "repositories")] - pub repositories: Vec, -} - -/// Response body of `POST /agent-integrations/composio/triggers`. -#[derive(Debug, Clone, Serialize, Deserialize)] -pub struct ComposioCreateTriggerResponse { - #[serde(rename = "triggerId")] - pub trigger_id: String, - #[serde(default)] - pub status: Option, -} - -// ── Trigger management (catalog + active list + enable/disable) ───── - -/// Per-repo descriptor used by GitHub-scoped available triggers. -#[derive(Debug, Clone, Serialize, Deserialize)] -pub struct ComposioAvailableTriggerRepo { - pub owner: String, - pub repo: String, -} - -/// One entry in `GET /agent-integrations/composio/triggers/available`. -#[derive(Debug, Clone, Serialize, Deserialize)] -pub struct ComposioAvailableTrigger { - pub slug: String, - /// `"static"` or `"github_repo"`. - pub scope: String, - #[serde( - rename = "defaultConfig", - default, - skip_serializing_if = "Option::is_none" - )] - pub default_config: Option, - #[serde( - rename = "requiredConfigKeys", - default, - skip_serializing_if = "Option::is_none" - )] - pub required_config_keys: Option>, - #[serde(default, skip_serializing_if = "Option::is_none")] - pub repo: Option, -} - -#[derive(Debug, Clone, Default, Serialize, Deserialize)] -pub struct ComposioAvailableTriggersResponse { - #[serde(default)] - pub triggers: Vec, -} - -/// One entry in `GET /agent-integrations/composio/triggers`. -#[derive(Debug, Clone, Serialize, Deserialize)] -pub struct ComposioActiveTrigger { - #[serde(deserialize_with = "de_string_or_object")] - pub id: String, - #[serde(deserialize_with = "de_string_or_object")] - pub slug: String, - #[serde(deserialize_with = "de_string_or_object")] - pub toolkit: String, - #[serde(rename = "connectionId", deserialize_with = "de_string_or_object")] - pub connection_id: String, - #[serde( - rename = "triggerConfig", - default, - skip_serializing_if = "Option::is_none" - )] - pub trigger_config: Option, - #[serde( - default, - skip_serializing_if = "Option::is_none", - deserialize_with = "de_opt_string_or_object" - )] - pub state: Option, -} - -#[derive(Debug, Clone, Default, Serialize, Deserialize)] -pub struct ComposioActiveTriggersResponse { - #[serde(default)] - pub triggers: Vec, -} - -/// Response body of `POST /agent-integrations/composio/triggers` (enable). -#[derive(Debug, Clone, Serialize, Deserialize)] -pub struct ComposioEnableTriggerResponse { - #[serde(rename = "triggerId")] - pub trigger_id: String, - pub slug: String, - #[serde(rename = "connectionId")] - pub connection_id: String, -} - -/// Response body of `DELETE /agent-integrations/composio/triggers/:id`. -#[derive(Debug, Clone, Serialize, Deserialize)] -pub struct ComposioDisableTriggerResponse { - #[serde(default)] - pub deleted: bool, -} - -// ── Triggers ──────────────────────────────────────────────────────── - -/// Payload of the `composio:trigger` Socket.IO event emitted by the backend -/// when a Composio webhook is received, HMAC-verified, and delivered to the -/// user's active sockets. -/// -/// See `src/controllers/agentIntegrations/composio/handleWebhook.ts` in the -/// backend repo. -#[derive(Debug, Clone, Serialize, Deserialize)] -pub struct ComposioTriggerEvent { - /// Toolkit slug, e.g. `"gmail"`. - #[serde(default)] - pub toolkit: String, - /// Trigger slug, e.g. `"GMAIL_NEW_GMAIL_MESSAGE"`. - #[serde(default)] - pub trigger: String, - /// Trigger-specific payload (provider-defined shape). - #[serde(default)] - pub payload: serde_json::Value, - /// Metadata the backend attaches: `{ id, uuid }`. - #[serde(default)] - pub metadata: ComposioTriggerMetadata, -} - -#[derive(Debug, Clone, Default, Serialize, Deserialize)] -pub struct ComposioTriggerMetadata { - #[serde(default)] - pub id: String, - #[serde(default)] - pub uuid: String, -} - -#[derive(Debug, Clone, Serialize, Deserialize)] -pub struct ComposioTriggerHistoryEntry { - /// Unix timestamp in milliseconds when the trigger reached the core. - pub received_at_ms: u64, - /// Toolkit slug, e.g. `"gmail"`. - pub toolkit: String, - /// Trigger slug, e.g. `"GMAIL_NEW_GMAIL_MESSAGE"`. - pub trigger: String, - /// Backend metadata id for this event. - pub metadata_id: String, - /// Backend metadata UUID for this event. - pub metadata_uuid: String, - /// Raw provider payload as forwarded by the backend socket event. - pub payload: serde_json::Value, -} - -#[derive(Debug, Clone, Serialize, Deserialize)] -pub struct ComposioTriggerHistoryResult { - /// Directory containing daily JSONL archives. - pub archive_dir: String, - /// Today's JSONL file path. - pub current_day_file: String, - /// Recent triggers, newest first. - pub entries: Vec, -} - -#[cfg(test)] -mod tests { - use super::*; - use serde_json::json; - - #[test] - fn connection_is_active_matches_ui_status_normalization() { - for status in ["ACTIVE", "CONNECTED", "active", "connected", " connected "] { - let conn = ComposioConnection { - id: "c1".into(), - toolkit: "slack".into(), - status: status.into(), - created_at: None, - account_email: None, - workspace: None, - username: None, - }; - assert!(conn.is_active(), "status {status:?} should be active"); - } - - for status in ["PENDING", "INITIATED", "FAILED", ""] { - let conn = ComposioConnection { - id: "c1".into(), - toolkit: "slack".into(), - status: status.into(), - created_at: None, - account_email: None, - workspace: None, - username: None, - }; - assert!(!conn.is_active(), "status {status:?} should not be active"); - } - } - - #[test] - fn connection_normalizes_toolkit_for_runtime_matching() { - let conn = ComposioConnection { - id: "c1".into(), - toolkit: " Slack ".into(), - status: "ACTIVE".into(), - created_at: None, - account_email: None, - workspace: None, - username: None, - }; - assert_eq!(conn.normalized_toolkit(), "slack"); - } - - #[test] - fn toolkits_response_defaults_to_empty() { - let resp: ComposioToolkitsResponse = serde_json::from_str("{}").unwrap(); - assert!(resp.toolkits.is_empty()); - } - - #[test] - fn toolkits_response_roundtrips() { - let resp = ComposioToolkitsResponse { - toolkits: vec!["gmail".into(), "notion".into()], - ..Default::default() - }; - let value = serde_json::to_value(&resp).unwrap(); - // Empty catalog is skipped on the wire — back-compat with old cores. - assert_eq!(value, json!({ "toolkits": ["gmail", "notion"] })); - let back: ComposioToolkitsResponse = serde_json::from_value(value).unwrap(); - assert_eq!(back.toolkits, vec!["gmail", "notion"]); - assert!(back.catalog.is_empty()); - } - - #[test] - fn toolkits_response_forwards_catalog() { - // A backend that sends the dynamic catalog must deserialize and - // re-serialize verbatim so the field reaches the desktop UI. - let raw = json!({ - "toolkits": ["gmail"], - "catalog": [ - { - "slug": "gmail", - "name": "Gmail", - "logo": "https://logos.composio.dev/api/gmail", - "description": "Send and read email", - "categories": ["productivity"], - "enabled": true - } - ] - }); - let resp: ComposioToolkitsResponse = serde_json::from_value(raw).unwrap(); - assert_eq!(resp.catalog.len(), 1); - let entry = &resp.catalog[0]; - assert_eq!(entry.slug, "gmail"); - assert_eq!(entry.name, "Gmail"); - assert_eq!(entry.enabled, Some(true)); - assert_eq!(entry.categories, vec!["productivity".to_string()]); - - // Round-trips back out with the catalog intact. - let value = serde_json::to_value(&resp).unwrap(); - assert_eq!(value["catalog"][0]["slug"], "gmail"); - assert_eq!(value["catalog"][0]["enabled"], true); - } - - #[test] - fn connection_parses_and_serializes_camelcase_created_at() { - let raw = json!({ - "id": "conn_1", - "toolkit": "gmail", - "status": "ACTIVE", - "createdAt": "2026-02-01T00:00:00Z" - }); - let conn: ComposioConnection = serde_json::from_value(raw.clone()).unwrap(); - assert_eq!(conn.id, "conn_1"); - assert_eq!(conn.toolkit, "gmail"); - assert_eq!(conn.status, "ACTIVE"); - assert_eq!(conn.created_at.as_deref(), Some("2026-02-01T00:00:00Z")); - - // Round-trip must use camelCase too. - let serialized = serde_json::to_value(&conn).unwrap(); - assert!(serialized.get("createdAt").is_some()); - } - - #[test] - fn connection_without_created_at_omits_field_when_serialized() { - let conn = ComposioConnection { - id: "x".into(), - toolkit: "notion".into(), - status: "PENDING".into(), - created_at: None, - account_email: None, - workspace: None, - username: None, - }; - let s = serde_json::to_value(&conn).unwrap(); - assert!( - s.get("createdAt").is_none(), - "createdAt must be skipped when None" - ); - } - - #[test] - fn authorize_response_uses_camelcase_keys() { - let raw = json!({ - "connectUrl": "https://composio.dev/oauth/abc", - "connectionId": "conn_2" - }); - let resp: ComposioAuthorizeResponse = serde_json::from_value(raw).unwrap(); - assert_eq!(resp.connect_url, "https://composio.dev/oauth/abc"); - assert_eq!(resp.connection_id, "conn_2"); - - let s = serde_json::to_value(&resp).unwrap(); - assert!(s.get("connectUrl").is_some()); - assert!(s.get("connectionId").is_some()); - } - - #[test] - fn tool_schema_defaults_type_field_to_function() { - let raw = json!({ - "function": { - "name": "GMAIL_SEND_EMAIL", - "description": "Send an email", - "parameters": { "type": "object" } - } - }); - let tool: ComposioToolSchema = serde_json::from_value(raw).unwrap(); - assert_eq!(tool.kind, "function"); - assert_eq!(tool.function.name, "GMAIL_SEND_EMAIL"); - assert_eq!(tool.function.description.as_deref(), Some("Send an email")); - assert!(tool.function.parameters.is_some()); - } - - #[test] - fn tool_function_tolerates_missing_description_and_parameters() { - let raw = json!({ "function": { "name": "SLUG_ONLY" } }); - let tool: ComposioToolSchema = serde_json::from_value(raw).unwrap(); - assert_eq!(tool.function.name, "SLUG_ONLY"); - assert!(tool.function.description.is_none()); - assert!(tool.function.parameters.is_none()); - } - - #[test] - fn execute_response_parses_cost_and_error() { - let raw = json!({ - "data": { "messageId": "m-1" }, - "successful": true, - "error": null, - "costUsd": 0.0025 - }); - let resp: ComposioExecuteResponse = serde_json::from_value(raw).unwrap(); - assert!(resp.successful); - assert!(resp.error.is_none()); - assert!((resp.cost_usd - 0.0025).abs() < f64::EPSILON); - } - - #[test] - fn execute_response_defaults_when_fields_missing() { - let resp: ComposioExecuteResponse = serde_json::from_str("{}").unwrap(); - assert!(!resp.successful); - assert!(resp.error.is_none()); - assert_eq!(resp.cost_usd, 0.0); - assert!(resp.data.is_null()); - } - - #[test] - fn available_trigger_deserializes_and_serializes_camelcase_fields() { - let raw = json!({ - "slug": "GMAIL_NEW_GMAIL_MESSAGE", - "scope": "static", - "defaultConfig": { "labelIds": ["INBOX"] }, - "requiredConfigKeys": ["labelIds"], - "repo": { "owner": "acme", "repo": "inbox" } - }); - let trigger: ComposioAvailableTrigger = serde_json::from_value(raw).unwrap(); - assert_eq!(trigger.slug, "GMAIL_NEW_GMAIL_MESSAGE"); - assert_eq!(trigger.scope, "static"); - assert_eq!( - trigger.default_config, - Some(json!({ "labelIds": ["INBOX"] })) - ); - assert_eq!( - trigger.required_config_keys, - Some(vec!["labelIds".to_string()]) - ); - let repo = trigger.repo.as_ref().expect("repo"); - assert_eq!(repo.owner, "acme"); - assert_eq!(repo.repo, "inbox"); - - let value = serde_json::to_value(&trigger).unwrap(); - assert!(value.get("defaultConfig").is_some()); - assert!(value.get("requiredConfigKeys").is_some()); - } - - #[test] - fn active_trigger_parses_connection_id_and_optional_fields() { - let raw = json!({ - "id": "ti_1", - "slug": "GMAIL_NEW_GMAIL_MESSAGE", - "toolkit": "gmail", - "connectionId": "c-1", - "triggerConfig": { "labelIds": "INBOX" }, - "state": "active" - }); - let trigger: ComposioActiveTrigger = serde_json::from_value(raw).unwrap(); - assert_eq!(trigger.id, "ti_1"); - assert_eq!(trigger.slug, "GMAIL_NEW_GMAIL_MESSAGE"); - assert_eq!(trigger.connection_id, "c-1"); - assert_eq!(trigger.trigger_config, Some(json!({"labelIds":"INBOX"}))); - assert_eq!(trigger.state.as_deref(), Some("active")); - - let value = serde_json::to_value(&trigger).unwrap(); - assert!(value.get("connectionId").is_some()); - assert!(value.get("triggerConfig").is_some()); - assert!(value.get("state").is_some()); - } - - #[test] - fn trigger_enable_response_uses_camelcase_and_optional_defaults() { - let raw = json!({ - "triggerId": "ti_9", - "slug": "GMAIL_NEW_GMAIL_MESSAGE", - "connectionId": "c-9" - }); - let resp: ComposioEnableTriggerResponse = serde_json::from_value(raw).unwrap(); - assert_eq!(resp.trigger_id, "ti_9"); - assert_eq!(resp.slug, "GMAIL_NEW_GMAIL_MESSAGE"); - assert_eq!(resp.connection_id, "c-9"); - - let serialized = serde_json::to_value(&resp).unwrap(); - assert_eq!(serialized.get("triggerId").unwrap(), "ti_9"); - assert_eq!(serialized.get("connectionId").unwrap(), "c-9"); - } - - #[test] - fn delete_trigger_response_defaults_deleted_to_false() { - let raw = json!({}); - let resp: ComposioDisableTriggerResponse = serde_json::from_value(raw).unwrap(); - assert!(!resp.deleted); - } - - #[test] - fn trigger_event_defaults_empty_fields_to_empty_strings() { - let ev: ComposioTriggerEvent = serde_json::from_str("{}").unwrap(); - assert_eq!(ev.toolkit, ""); - assert_eq!(ev.trigger, ""); - assert_eq!(ev.metadata.id, ""); - assert_eq!(ev.metadata.uuid, ""); - assert!(ev.payload.is_null()); - } - - #[test] - fn trigger_event_parses_full_payload() { - let raw = json!({ - "toolkit": "gmail", - "trigger": "GMAIL_NEW_GMAIL_MESSAGE", - "payload": { "subject": "hi" }, - "metadata": { "id": "evt-1", "uuid": "uuid-1" } - }); - let ev: ComposioTriggerEvent = serde_json::from_value(raw).unwrap(); - assert_eq!(ev.toolkit, "gmail"); - assert_eq!(ev.trigger, "GMAIL_NEW_GMAIL_MESSAGE"); - assert_eq!(ev.metadata.id, "evt-1"); - assert_eq!(ev.metadata.uuid, "uuid-1"); - assert_eq!(ev.payload["subject"], "hi"); - } - - #[test] - fn active_trigger_accepts_string_fields() { - let v = json!({ - "id": "t1", - "slug": "GMAIL_NEW_MAIL", - "toolkit": "gmail", - "connectionId": "c1", - "state": "ACTIVE", - }); - let trig: ComposioActiveTrigger = serde_json::from_value(v).unwrap(); - assert_eq!(trig.id, "t1"); - assert_eq!(trig.slug, "GMAIL_NEW_MAIL"); - assert_eq!(trig.toolkit, "gmail"); - assert_eq!(trig.connection_id, "c1"); - assert_eq!(trig.state.as_deref(), Some("ACTIVE")); - } - - #[test] - fn active_trigger_accepts_object_fields() { - // Mirrors upstream API drift where these fields arrive as objects - // rather than plain strings. - let v = json!({ - "id": {"id": "t1"}, - "slug": {"slug": "GMAIL_NEW_MAIL"}, - "toolkit": {"slug": "gmail", "logo": "https://…"}, - "connectionId": {"id": "c1"}, - "state": {"state": "ACTIVE", "slug": "should-be-ignored"}, - }); - let trig: ComposioActiveTrigger = serde_json::from_value(v).unwrap(); - assert_eq!(trig.id, "t1"); - assert_eq!(trig.slug, "GMAIL_NEW_MAIL"); - assert_eq!(trig.toolkit, "gmail"); - assert_eq!(trig.connection_id, "c1"); - // `state` priority must prefer the literal `state` key over metadata. - assert_eq!(trig.state.as_deref(), Some("ACTIVE")); - } - - #[test] - fn active_trigger_state_falls_back_to_value() { - let v = json!({ - "id": "t1", - "slug": "X", - "toolkit": "gmail", - "connectionId": "c1", - "state": {"value": "PENDING"}, - }); - let trig: ComposioActiveTrigger = serde_json::from_value(v).unwrap(); - assert_eq!(trig.state.as_deref(), Some("PENDING")); - } - - #[test] - fn active_trigger_state_missing_or_unknown_returns_none() { - let v = json!({ - "id": "t1", - "slug": "X", - "toolkit": "gmail", - "connectionId": "c1", - }); - let trig: ComposioActiveTrigger = serde_json::from_value(v).unwrap(); - assert!(trig.state.is_none()); - - let v = json!({ - "id": "t1", - "slug": "X", - "toolkit": "gmail", - "connectionId": "c1", - "state": {"unrelated": 42}, - }); - let trig: ComposioActiveTrigger = serde_json::from_value(v).unwrap(); - assert!(trig.state.is_none()); - } - - #[test] - fn active_trigger_required_field_rejects_unsupported_object() { - // Object without any of slug/id/name/key must fail loudly so we - // notice further upstream shape drift instead of silently dropping - // the trigger. - let v = json!({ - "id": {"unrelated": 42}, - "slug": "X", - "toolkit": "gmail", - "connectionId": "c1", - }); - let err = serde_json::from_value::(v).unwrap_err(); - assert!(err.to_string().contains("expected string or object")); - } -} diff --git a/src/openhuman/memory/api/host/config.rs b/src/openhuman/memory/api/host/config.rs deleted file mode 100644 index 7ee21b8104..0000000000 --- a/src/openhuman/memory/api/host/config.rs +++ /dev/null @@ -1,256 +0,0 @@ -//! [`MemoryHostConfig`] — the memory subsystem's view of the host's config. -//! -//! # Why a trait and not a struct -//! -//! The host's `Config` is one giant serde struct covering voice, channels, -//! sandboxing, inference routing, the agent harness — the lot. The memory -//! subsystem reads about two dozen of its fields. Moving the whole struct into -//! this crate would drag the host's entire configuration vocabulary into a -//! contract crate that is meant to stay dependency-light; leaving it behind and -//! passing individual values would mean rewriting every function signature in -//! the extracted code. -//! -//! A trait threads the needle. `crate::openhuman::memory::core_impl::Config` is the alias -//! `dyn MemoryHostConfig`, so a function that took `config: &Config` before the -//! extraction still takes `config: &Config` after it, and the host's concrete -//! `Config` unsize-coerces at the call site with no edit at all. Only the field -//! *accesses* inside the extracted code change, from `config.workspace_dir` to -//! `config.workspace_dir()`. -//! -//! # Accessor shapes are chosen for zero churn, not for elegance -//! -//! Several accessors return `&PathBuf` / `&Vec` where `&Path` / `&[T]` would -//! be the idiomatic choice. That is deliberate: the extracted code calls -//! `.clone()` on these values in dozens of places, and `&Path`/`&[T]` would -//! silently resolve `.clone()` to the *reference*'s `Clone` impl and fail at the -//! use site with a confusing type error. Returning the owning type keeps every -//! one of those sites compiling unchanged. -//! -//! # Mutation -//! -//! Three methods take `&mut self`. They exist because the extracted code owns -//! two write paths the host does not: the composio source-caps migration and -//! the CLI's env-override re-application. Everything else is read-only. - -use std::path::PathBuf; - -use super::cloud_providers::CloudProviderCreds; -use super::local_ai::LocalAiConfig; -use super::scheduler_gate::SchedulerGateConfig; -use super::storage_memory::{MemoryConfig, MemoryTreeConfig}; - -/// Composio routing mode: proxied through the host's cloud backend. -pub const COMPOSIO_MODE_BACKEND: &str = "backend"; -/// Composio routing mode: BYO API key, calling `backend.composio.dev` directly. -pub const COMPOSIO_MODE_DIRECT: &str = "direct"; - -/// The subset of a host's Composio configuration the memory sync pipelines read. -/// -/// Passed by value rather than by reference because the host's own -/// `ComposioConfig` carries fields (toolkit triage opt-outs, the enabled flag) -/// that have nothing to do with memory, and because borrowing it would pin the -/// host's type into this contract. -#[derive(Debug, Clone, Default, PartialEq, Eq)] -pub struct ComposioMode { - /// [`COMPOSIO_MODE_BACKEND`] or [`COMPOSIO_MODE_DIRECT`]. - pub mode: String, - /// The Composio entity the host authenticates as. - pub entity_id: String, - /// Direct-mode API key, when the user hand-wrote one into `config.toml`. - /// The keychain-backed value takes precedence and is resolved host-side. - pub api_key: Option, - /// Whether the LLM triage turn is switched off for all triggers. - pub triage_disabled: bool, -} - -impl ComposioMode { - /// True when the host routes Composio calls directly rather than through - /// its cloud backend. - #[must_use] - pub fn is_direct(&self) -> bool { - self.mode.eq_ignore_ascii_case(COMPOSIO_MODE_DIRECT) - } -} - -/// The host's configuration, as the memory subsystem sees it. -/// -/// Implemented by the embedding application for its own root config type. See -/// the module docs for why this is a trait and why the accessor return types -/// are shaped the way they are. -#[async_trait::async_trait] -pub trait MemoryHostConfig: Send + Sync + std::fmt::Debug { - // ── Paths ─────────────────────────────────────────────────────────────── - - /// Root of the host's internal per-user state. Every memory database, - /// summary-tree directory and queue file is resolved beneath this. - fn workspace_dir(&self) -> &PathBuf; - - /// Absolute path of the `config.toml` this config was loaded from. - fn config_path(&self) -> &PathBuf; - - /// Where chunk `.md` files are written. Either the explicit - /// `memory_tree.content_dir` or `/memory_tree/content`. - fn memory_tree_content_root(&self) -> PathBuf; - - // ── Memory-owned sections ─────────────────────────────────────────────── - - /// The `[memory]` block — backend selection, embedding provider/model/dims, - /// relevance floor, SQLite timeouts. - fn memory(&self) -> &MemoryConfig; - - /// The `[memory_tree]` block — summary-tree embedder, extractor and - /// summariser wiring. - fn memory_tree(&self) -> &MemoryTreeConfig; - - /// The `[scheduler_gate]` block — when background LLM-bound work may run. - fn scheduler_gate(&self) -> &SchedulerGateConfig; - - // ── Host-owned sections the memory subsystem still reads ──────────────── - // - // These are the seam's rough edge (see the module docs on `host`): they are - // read only to *construct* embedding providers, which is work that belongs - // in the host. Moving the embedding factory back out of the core would let - // all four of these accessors go away. - - /// The `[local_ai]` block — whether a local runtime is enabled and which - /// model it serves. - fn local_ai(&self) -> &LocalAiConfig; - - /// Configured cloud LLM/embedding backends, keyed by user-chosen slug. - fn cloud_providers(&self) -> &Vec; - - /// `provider:model` routing string for the embeddings workload, if pinned. - fn embeddings_provider(&self) -> Option<&str>; - - /// `provider:model` routing string for the memory workload, if pinned. - fn memory_provider(&self) -> Option<&str>; - - /// The local model id for a workload, when that workload is routed to - /// Ollama (`"ollama:"`). `None` for cloud or unset workloads. - /// - /// This is the single source of truth for "is this workload local?" — - /// callers must not consult the deprecated `local_ai.usage.*` booleans or - /// `memory_tree.llm_backend`. - fn workload_local_model(&self, workload: &str) -> Option; - - // ── Scalars ───────────────────────────────────────────────────────────── - - /// The concrete config behind this trait object, for host code that needs - /// its own type back. - /// - /// The seam deliberately hands the core a `dyn MemoryHostConfig`, and that - /// is the right shape for everything the core does. But a *host* - /// implementation of one of the behavioural seams — chat-model routing, - /// Composio mode dispatch — is handed the same trait object and has to get - /// its own `Config` back: routing reads BYOK fallbacks, per-role routes and - /// credentials, none of which are on this trait and none of which should - /// be. - /// - /// Implementations return `self`. A host downcasts and, on failure, falls - /// back to whatever it was configured with — a failure means the config is - /// somebody else's type (a test double), not that something is wrong. - fn as_any(&self) -> &dyn std::any::Any; - - /// An owned, shareable handle to this config. - /// - /// `crate::openhuman::memory::core_impl::Config` is the *unsized* `dyn MemoryHostConfig`, which - /// makes `&Config` free at every call site — the host's concrete `Config` - /// unsize-coerces with no edit. The cost is that a borrow cannot be turned - /// into an owned value: background loops that outlive their caller, structs - /// that hold a config, and `spawn_blocking` bodies all need one. - /// - /// This is that escape hatch. Implementations return - /// `Arc::new(self.clone())`; callers that only read should keep taking - /// `&Config` rather than reaching for this. - fn to_arc(&self) -> std::sync::Arc; - - /// Backend base URL, used to recognise first-party endpoints. - fn api_url(&self) -> Option<&str>; - - /// The backend API URL this host actually talks to, with the host's own - /// environment and default resolution already applied. - /// - /// Distinct from [`Self::api_url`], which is the raw configured value — - /// resolution (env override, staging/prod default, trailing-slash - /// normalisation) is host logic and must not be re-derived here. - fn effective_backend_api_url(&self) -> String; - - /// The current backend session bearer, or `None` when signed out. - /// - /// Read through the trait rather than from a config field because the host - /// keeps it in its credential store, not in `config.toml`. - /// - /// # Errors - /// - /// Returns `Err` when the credential store cannot be read — distinct from - /// `Ok(None)`, which means "read fine, not signed in". - fn session_token(&self) -> Result, String>; - - /// Default chat model id. - fn default_model(&self) -> Option<&str>; - - /// Default sampling temperature for background LLM calls. - fn default_temperature(&self) -> f64; - - /// Optional language for background LLM artifacts — tree summaries, - /// extraction reasons, learning reflections. `None` keeps the default. - fn output_language(&self) -> Option<&str>; - - /// Global memory-sync cadence in seconds. `None` means "no explicit choice" - /// and callers fall back to [`super::DEFAULT_MEMORY_SYNC_INTERVAL_SECS`]; - /// `Some(0)` means manual-only. - fn memory_sync_interval_secs(&self) -> Option; - - /// Whether the user has finished onboarding. Background ingestion holds off - /// until they have. - fn onboarding_completed(&self) -> bool; - - /// Whether at-rest secret encryption is switched on for this workspace. - fn secrets_encrypt(&self) -> bool; - - /// Composio routing mode + credentials, as the sync pipelines need them. - fn composio(&self) -> ComposioMode; - - // ── Memory sources ────────────────────────────────────────────────────── - // - // Serde-mediated on purpose. `MemorySourceEntry` is defined by the *engine* - // crate (`tinycortex`), which this contract crate must not depend on — it - // would drag SQLite in and break the dependency-light guarantee this crate - // exists to hold. JSON is the narrowest waist that keeps the type where it - // belongs. - - /// The persisted `[[memory_sources]]` registry, as JSON. - /// - /// # Errors - /// Propagates a serialization failure from the host's own entry type. - fn memory_sources_json(&self) -> anyhow::Result; - - /// Replace the persisted `[[memory_sources]]` registry. Does not save to - /// disk — call [`Self::save`] afterwards. - /// - /// # Errors - /// Returns an error when `value` does not deserialize into the host's entry - /// type, in which case the registry is left untouched. - fn set_memory_sources_json(&mut self, value: serde_json::Value) -> anyhow::Result<()>; - - // ── Migration bookkeeping ─────────────────────────────────────────────── - - /// Version of the composio source-capabilities migration already applied. - fn composio_source_caps_migration_version(&self) -> u32; - - /// Record that the composio source-capabilities migration has run. - fn set_composio_source_caps_migration_version(&mut self, version: u32); - - // ── Lifecycle ─────────────────────────────────────────────────────────── - - /// Re-apply the host's environment-variable overlay over this config. - /// Used by the CLI entry points, which build a config before the host's - /// normal load path has run. - fn apply_env_overrides(&mut self); - - /// Persist this config back to [`Self::config_path`] atomically. - /// - /// # Errors - /// Propagates the host's own write/serialize failure. - async fn save(&self) -> anyhow::Result<()>; -} diff --git a/src/openhuman/memory/api/host/embedding_host.rs b/src/openhuman/memory/api/host/embedding_host.rs deleted file mode 100644 index 55451a762c..0000000000 --- a/src/openhuman/memory/api/host/embedding_host.rs +++ /dev/null @@ -1,101 +0,0 @@ -//! [`EmbeddingHost`] — provider *construction*, which the host owns. -//! -//! [`super::EmbeddingProvider`] is the contract for a provider that already -//! exists. This trait is the other half: how one comes into being. Resolving an -//! API key from the credential store, knowing which managed cloud endpoint the -//! signed-in user is entitled to, knowing where the local Ollama server is -//! listening — all of that is host policy, and none of it belongs in a memory -//! engine. -//! -//! The core reaches this through a process-global installed at startup, for the -//! same reason [`super::MemoryEventSink`] is a global: the construction sites -//! sit deep inside retrieval and sealing call stacks that already thread a -//! config and a store handle. -//! -//! # Default is failure, not silence -//! -//! Unlike the event sink, an unwired [`EmbeddingHost`] must **not** degrade -//! quietly. A missing sink drops a notification about work that already -//! happened; a missing embedding provider means vectors would be written into -//! the wrong embedding space, or a query would silently return lexical-only -//! results. Both are data corruption with a delayed fuse, so the unwired -//! accessors return `Err`/`None` and every call site is written to propagate. - -use std::sync::Arc; - -use super::EmbeddingProvider; - -/// Builds [`EmbeddingProvider`]s on the core's behalf. -/// -/// Object-safe: the core holds one as `Arc`. -pub trait EmbeddingHost: Send + Sync + std::fmt::Debug { - /// The API key for `provider`, from the host's credential store. - /// - /// Returns `None` when the provider has no stored credential — which is not - /// an error: a local provider needs none, and an unconfigured cloud one is - /// a state the caller reports rather than a failure. - fn resolve_api_key(&self, provider: &str) -> Option; - - /// Base URL of the local Ollama server, honouring the host's env override - /// and config before falling back to the default. - fn ollama_base_url(&self) -> String; - - /// The host's default provider — the managed cloud embedder. - /// - /// Constructed lazily with respect to authentication: this may be called - /// before login completes, and the first `embed()` is what fails if the - /// user is unauthenticated. - fn default_embedding_provider(&self) -> Arc; - - /// Builds a provider from an explicit provider/model/credential triple. - /// - /// # Errors - /// - /// Returns `Err` when `provider` is not one the host knows how to build, or - /// when the supplied credentials are unusable for it. - fn create_embedding_provider_with_credentials( - &self, - provider: &str, - model: &str, - dims: usize, - api_key: &str, - custom_endpoint: Option<&str>, - ) -> Result, String>; - - /// Whether `model` accepts a caller-chosen output dimensionality. - /// - /// Asking for dimensions a model does not support is rejected by the - /// provider at request time, so the core checks first rather than writing a - /// batch that will fail halfway. - fn model_supports_dimensions(&self, model: &str) -> bool; - - /// The managed cloud embedder at an explicit model and dimensionality. - /// - /// # Errors - /// - /// Returns `Err` when the host cannot reach its managed endpoint - /// configuration. - fn cloud_embedding_provider( - &self, - model: &str, - dims: usize, - ) -> Result, String>; - - /// The default model id the managed cloud embedder uses. - fn default_cloud_embedding_model(&self) -> &str; - - /// The dimensionality [`Self::default_cloud_embedding_model`] emits. - fn default_cloud_embedding_dimensions(&self) -> usize; - - /// An Ollama-backed provider at `base_url`. - /// - /// # Errors - /// - /// Returns `Err` when the host cannot construct one for `model`. - fn ollama_embedding_provider( - &self, - base_url: &str, - model: &str, - dims: usize, - ) -> Result, String>; -} diff --git a/src/openhuman/memory/api/host/embeddings.rs b/src/openhuman/memory/api/host/embeddings.rs deleted file mode 100644 index d5cc55bcf4..0000000000 --- a/src/openhuman/memory/api/host/embeddings.rs +++ /dev/null @@ -1,110 +0,0 @@ -//! [`EmbeddingProvider`] — text → vector, supplied by the host. -//! -//! The memory subsystem embeds chunks, summaries and queries, but it does not -//! decide *how*: which provider, which credentials, which rate limit and which -//! fallback are host policy. So the core takes an `Arc` -//! and never constructs one. -//! -//! This trait deliberately lives in the contract crate rather than in -//! `tinymemory-core`, so that a host implementing it does not have to depend on -//! the engine. It carries nothing heavier than `async-trait` and `anyhow`. - -use async_trait::async_trait; - -/// Formats the canonical embedding-space signature string. -/// -/// This is the **single source of truth** for the signature format. Both the -/// live-provider [`EmbeddingProvider::signature`] and any config-derived -/// signature must route through here, so a signature computed from -/// configuration is byte-identical to one computed from an instantiated -/// provider. Drift between the two silently splits one embedding space into -/// two, and every vector written on the wrong side of the split becomes -/// unsearchable without a re-embed. -#[must_use] -pub fn format_embedding_signature(name: &str, model_id: &str, dims: usize) -> String { - format!( - "provider={}:{};model={}:{};dims={dims}", - name.len(), - name, - model_id.len(), - model_id - ) -} - -#[cfg(test)] -mod tests { - use super::format_embedding_signature; - - #[test] - fn delimiter_characters_cannot_make_distinct_spaces_collide() { - let first = format_embedding_signature("a;model=b", "c", 3); - let second = format_embedding_signature("a", "b;model=c", 3); - assert_ne!(first, second); - } -} - -/// Converts text into numerical vectors. -#[async_trait] -pub trait EmbeddingProvider: Send + Sync { - /// Provider name, e.g. `"ollama"`, `"openai"`. - fn name(&self) -> &str; - - /// Stable model identifier used to generate embeddings. - fn model_id(&self) -> &str; - - /// Number of dimensions in the generated embeddings. - fn dimensions(&self) -> usize; - - /// Stable signature for the embedding space. - /// - /// Changing any component means existing vectors are no longer comparable - /// with newly generated ones and must be stored and queried separately - /// until a migration re-embeds them. - fn signature(&self) -> String { - format_embedding_signature(self.name(), self.model_id(), self.dimensions()) - } - - /// Generates embeddings for a batch of strings. - /// - /// # Errors - /// Propagates transport, authentication and quota failures from the - /// underlying provider. - async fn embed(&self, texts: &[&str]) -> anyhow::Result>>; - - /// Generates an embedding for a single string. - /// - /// # Errors - /// As [`Self::embed`], plus an error when the provider returns no vector. - async fn embed_one(&self, text: &str) -> anyhow::Result> { - let mut results = self.embed(&[text]).await?; - results - .pop() - .ok_or_else(|| anyhow::anyhow!("Empty embedding result")) - } -} - -/// The inert provider bound when semantic search is switched off or no -/// embedding backend is configured. Reports zero dimensions and returns one -/// empty vector per input, so keyword-only retrieval keeps working while -/// vector rerank degrades to a no-op rather than an error. -#[derive(Debug, Clone, Copy, Default)] -pub struct NoopEmbedding; - -#[async_trait] -impl EmbeddingProvider for NoopEmbedding { - fn name(&self) -> &str { - "none" - } - - fn model_id(&self) -> &str { - "none" - } - - fn dimensions(&self) -> usize { - 0 - } - - async fn embed(&self, texts: &[&str]) -> anyhow::Result>> { - Ok(vec![Vec::new(); texts.len()]) - } -} diff --git a/src/openhuman/memory/api/host/error_reporter.rs b/src/openhuman/memory/api/host/error_reporter.rs deleted file mode 100644 index 02c874ba8e..0000000000 --- a/src/openhuman/memory/api/host/error_reporter.rs +++ /dev/null @@ -1,43 +0,0 @@ -//! [`ErrorReporter`] — the host's crash/error telemetry, as the core sees it. -//! -//! The memory subsystem reports a handful of failures that are worth a -//! developer's attention: a corrupt SQLite database, host filesystem I/O -//! errors, a sync run that failed for a non-user reason. *Where* those go — -//! Sentry, a log sink, nowhere — and which of them count as expected rather -//! than exceptional is host policy, so the core states the fact and the host -//! decides what to do with it. -//! -//! # The two methods are not interchangeable -//! -//! [`ErrorReporter::report_error`] is unconditional: the caller has already -//! decided this is a real defect. [`ErrorReporter::report_error_or_expected`] -//! asks the host to classify first, so routine user- and config-caused failures -//! (an unreachable local runtime, a revoked OAuth token) do not page anyone. -//! Collapsing them into one would either spam the error channel or hide real -//! bugs, which is why both exist. - -/// Receives error reports from the memory subsystem. -/// -/// Takes the **already-rendered** message rather than a concrete error type: -/// the trait has to be object-safe, so it cannot be generic over `E: Display` -/// the way the host's own `report_error` is. The core's free functions keep -/// that generic signature and render with `{:#}` — the alternate specifier that -/// makes `anyhow::Error` print its full context chain — before crossing. -pub trait ErrorReporter: Send + Sync + std::fmt::Debug { - /// Report `error` as a defect worth investigating. - /// - /// `domain` and `operation` are stable, low-cardinality strings used for - /// grouping (`"memory"` / `"tree_jobs_worker_corrupt"`); `tags` carries - /// additional non-sensitive key/value context. - fn report_error(&self, rendered: &str, domain: &str, operation: &str, tags: &[(&str, &str)]); - - /// Report `error`, letting the host classify it as a defect or an expected - /// user/config failure and route it accordingly. - fn report_error_or_expected( - &self, - rendered: &str, - domain: &str, - operation: &str, - tags: &[(&str, &str)], - ); -} diff --git a/src/openhuman/memory/api/host/events.rs b/src/openhuman/memory/api/host/events.rs deleted file mode 100644 index a843db43bd..0000000000 --- a/src/openhuman/memory/api/host/events.rs +++ /dev/null @@ -1,228 +0,0 @@ -//! [`MemoryEventSink`] — the events the memory subsystem announces. -//! -//! # Why the host's event enum does not move -//! -//! The host's `DomainEvent` is a single flat enum covering agents, channels, -//! cron, tools, webhooks and the system domain as well as memory. It is the -//! host's own vocabulary; a *memory* crate must not own it, and importing it -//! would make every other subsystem's events a transitive dependency of memory. -//! -//! So the seam runs the other way. This module defines the ~15 memory-domain -//! events the extracted code emits, as a small enum of plain data. The host -//! implements [`MemoryEventSink`] by mapping each variant onto the matching -//! `DomainEvent` and publishing it on its own bus. The core publishes into the -//! sink and never learns that a bus exists. -//! -//! # Subscribing is not part of this seam -//! -//! Several extracted modules used to *subscribe* as well as publish -//! (`sync_events.rs`, `sync/composio/bus.rs`, `conversations/bus.rs`). Those are -//! host wiring by the repository README's split — event-bus subscribers belong -//! in the host, next to the registration site that installs them. They move back -//! rather than growing a subscribe method here. - -/// Why an embedding model was reported unhealthy, and what took over. -#[derive(Debug, Clone, PartialEq, Eq, serde::Serialize, serde::Deserialize)] -pub struct EmbeddingHealthReason { - /// The provider that failed. - pub provider: String, - /// The model that failed. - pub model: String, - /// The provider bound in its place. - pub fallback_provider: String, - /// Operator-facing explanation. Never carries credentials. - pub message: String, -} - -/// What kicked off a sync run — a schedule, a user action, a webhook. -pub type SyncTrigger = String; - -/// A memory-domain event, as announced by `tinymemory-core`. -/// -/// Field names and types mirror the host's own event payloads exactly, so the -/// host's [`MemoryEventSink`] impl is a straight structural mapping with no -/// judgement calls in it. -/// -/// Deliberately **not** `#[non_exhaustive]`: the host's mapping impl matches -/// exhaustively on purpose, so adding a variant here is a compile error at the -/// mapping site rather than an event that silently never reaches the bus. -#[derive(Debug, Clone, serde::Serialize, serde::Deserialize)] -pub enum MemoryEvent { - /// A sync run moved to a new stage. - SyncStageChanged { - /// What started this run. - trigger: SyncTrigger, - /// The stage just entered. - stage: String, - /// Provider slug, when the stage is provider-scoped. - provider: Option, - /// Connection id, when the stage is connection-scoped. - connection_id: Option, - /// Free-form operator detail. - detail: Option, - /// Memory-source id, when the stage is source-scoped. - source_id: Option, - }, - /// A document entered the ingestion pipeline. - IngestionStarted { - /// Document being ingested. - document_id: String, - /// Human-readable document title. - title: String, - /// Target namespace. - namespace: String, - /// Items still queued behind this one. - queue_depth: usize, - }, - /// A document left the ingestion pipeline. - IngestionCompleted { - /// Document that was ingested. - document_id: String, - /// Target namespace. - namespace: String, - /// Whether ingestion succeeded. - success: bool, - /// Wall-clock duration. - elapsed_ms: u64, - /// Items still queued afterwards. - queue_depth: usize, - }, - /// A source document was canonicalized into chunks. - DocumentCanonicalized { - /// Source the document came from. - source_id: String, - /// Source kind (`gmail`, `slack`, `file`, …). - source_kind: String, - /// How many chunks were written. - chunks_written: usize, - /// Ids of the written chunks. - chunk_ids: Vec, - /// Unix timestamp, seconds with fraction. - canonicalized_at: f64, - /// Truncated body preview for operator UIs. - body_preview: Option, - }, - /// An hour bucket was sealed and summarized. - TreeSummarizerHourCompleted { - /// Tree namespace. - namespace: String, - /// Node that was sealed. - node_id: String, - /// Tokens in the produced summary. - token_count: u32, - }, - /// A summary was propagated up a level. - TreeSummarizerPropagated { - /// Tree namespace. - namespace: String, - /// Node that received the propagated summary. - node_id: String, - /// Level name. - level: String, - /// Tokens in the produced summary. - token_count: u32, - }, - /// A full tree rebuild finished. - TreeSummarizerRebuildCompleted { - /// Tree namespace. - namespace: String, - /// Nodes in the rebuilt tree. - total_nodes: u64, - }, - /// Progress ticks during a tree build, for the operator UI. - TreeBuildProgress { - /// Coarse phase name. - phase: String, - /// Fine step name. - step: String, - /// Which tree, when scoped. - tree_scope: Option, - /// Tree level, when levelled. - level: Option, - /// Items processed in this step. - item_count: Option, - /// Free-form operator detail. - detail: Option, - }, - /// An embedding model failed health checks and a fallback was bound. - EmbeddingModelUnhealthy(EmbeddingHealthReason), - /// The configured memory driver could not be bound, and another was used. - DriverBindFailed { - /// Driver named in config. - configured_driver: String, - /// Driver actually bound. - bound_driver: String, - /// Why the configured driver was rejected. - reason: String, - }, - /// A diff snapshot was captured for a source. - DiffSnapshotTaken { - /// The new snapshot. - snapshot_id: String, - /// Source the snapshot covers. - source_id: String, - /// Source kind. - source_kind: String, - /// Items in the snapshot. - item_count: usize, - /// What triggered the snapshot. - trigger: String, - }, - /// Diffs were acknowledged by the user. - DiffMarkedRead { - /// Sources marked read. - source_ids: Vec, - /// Snapshots marked read. - snapshot_ids: Vec, - }, - /// The set of connected Composio toolkits changed. - ComposioIntegrationsChanged { - /// Toolkit slugs now connected. - toolkits: Vec, - }, - /// The memory subsystem is asking for a sync run. - SyncRequested { - /// Channel to report progress back on, when the request came from one. - channel_id: Option, - }, - /// The local embedding runtime is unusable and the user must act outside - /// the app (start Ollama, pull the model). - /// - /// The host surfaces this in its durable user-error centre. Carries no - /// provider text, model id or endpoint — see [`LOCAL_MODEL_UNAVAILABLE_KIND`]. - LocalModelUnavailable { - /// Short, non-sensitive tag naming which producer fired - /// (`health_gate` / `embed_classify`), so the two paths stay - /// distinguishable in the log without a correlation id. - origin: String, - }, -} - -/// Stable `error_type` token for the local-embedding-runtime user error. -/// -/// Mirrors the frontend `UserErrorKind` discriminator of the same name. It is -/// defined in the contract crate because both sides name it: the host builds -/// the wire payload from it, and the core's tests assert on it. A drift on -/// either side drops the signal silently. -pub const LOCAL_MODEL_UNAVAILABLE_KIND: &str = "local_model_unavailable"; - -/// `error_source` for the memory subsystem's user errors. Drives the panel's -/// scope grouping (`socketService` maps it to the `memory` `UserErrorScope`). -pub const MEMORY_USER_ERROR_SOURCE: &str = "memory"; - -/// Receives [`MemoryEvent`]s and does something host-shaped with them. -pub trait MemoryEventSink: Send + Sync + std::fmt::Debug { - /// Announce an event. Implementations must not block and must not fail — - /// an event bus that can reject a publish turns every emit site into an - /// error path, which is not what any of the call sites want. - fn publish(&self, event: MemoryEvent); -} - -/// The sink bound when no host has installed one — in unit tests, in the -/// standalone engine build, and before startup wiring runs. Drops everything. -#[derive(Debug, Clone, Copy, Default)] -pub struct NoopEventSink; - -impl MemoryEventSink for NoopEventSink { - fn publish(&self, _event: MemoryEvent) {} -} diff --git a/src/openhuman/memory/api/host/evidence.rs b/src/openhuman/memory/api/host/evidence.rs deleted file mode 100644 index 7134b6d6a0..0000000000 --- a/src/openhuman/memory/api/host/evidence.rs +++ /dev/null @@ -1,49 +0,0 @@ -//! [`EvidenceRef`] — a pointer to the thing a learned fact was learned from. -//! -//! Moved here from the host's `agent::learning::candidate` because it is -//! persisted *in the memory store*: `store::namespace_store::profile` writes it -//! into profile rows, and the Composio provider-profile sync reads it back. Two -//! structurally identical enums either side of the seam would round-trip -//! through serde and silently diverge on the first added variant. -//! -//! Inert serde data; the contract crate's dependency-light guarantee is -//! unaffected. **Its serde form is persisted**, so the `#[serde(tag = "type")]` -//! representation and every variant name are a compatibility surface. - -use serde::{Deserialize, Serialize}; - -/// A typed pointer back into the memory substrate from which a candidate was -/// derived. Used for provenance tracking, citation, and the `evidence_ids` -/// column in `user_profile_facets` (Phase 3+). -/// -/// Serialised with a `"type"` discriminator in snake_case so the JSON is -/// human-readable: `{"type":"episodic","episodic_id":42}`. -#[derive(Debug, Clone, PartialEq, Eq, Hash, Serialize, Deserialize)] -#[serde(tag = "type", rename_all = "snake_case")] -pub enum EvidenceRef { - /// A single row in `episodic_log`. - Episodic { episodic_id: i64 }, - /// A contiguous window of rows in `episodic_log`. - EpisodicWindow { from_id: i64, to_id: i64 }, - /// A row in the tree-source summary table. - SourceSummary { summary_id: String }, - /// A node in `tree_topic`. - TreeTopic { topic_id: String }, - /// A chunk in `vector_chunks` associated with a document source. - DocumentChunk { source_id: String, chunk_id: String }, - /// A specific message in an email source. - EmailMessage { - source_id: String, - message_id: String, - }, - /// A field value from a connected provider (Composio toolkit). - Provider { - toolkit: String, - connection_id: String, - field: String, - }, - /// A tool call record within an episodic entry. - ToolCall { tool_name: String, episodic_id: i64 }, - /// A per-window weight from `tree_source`. - TreeSourceWeight { window_label: String }, -} diff --git a/src/openhuman/memory/api/host/local_ai.rs b/src/openhuman/memory/api/host/local_ai.rs deleted file mode 100644 index f5cee1e7f6..0000000000 --- a/src/openhuman/memory/api/host/local_ai.rs +++ /dev/null @@ -1,285 +0,0 @@ -//! Local AI runtime configuration. - -use schemars::JsonSchema; -use serde::{Deserialize, Serialize}; - -/// Per-feature flags controlling which subsystems route through the selected -/// local runtime. All default to `false` (use cloud instead). Guarded by -/// `LocalAiConfig::runtime_enabled` — when that is `false` every helper -/// method below returns `false` regardless of these values. -#[derive(Debug, Clone, Serialize, Deserialize, JsonSchema)] -#[serde(default)] -#[derive(Default)] -pub struct LocalAiUsage { - /// When true (and `runtime_enabled`), use the local model for embedding - /// generation instead of the cloud backend. - #[serde(default)] - pub embeddings: bool, - /// When true (and `runtime_enabled`), use the local model inside the - /// heartbeat loop. - #[serde(default)] - pub heartbeat: bool, - /// When true (and `runtime_enabled`), use the local model for - /// learning/reflection passes. - #[serde(default)] - pub learning_reflection: bool, - /// When true (and `runtime_enabled`), use the local model for - /// subconscious evaluation and execution. - #[serde(default)] - pub subconscious: bool, -} - -#[derive(Debug, Clone, Serialize, Deserialize, JsonSchema)] -#[serde(default)] -pub struct LocalAiConfig { - /// Master runtime switch. Defaults to `false` — local AI is OFF by default. - /// Note: the old on-disk field was `enabled`; that key is now unknown to - /// serde and will be silently ignored on load (intentional forced reset). - #[serde(default = "default_runtime_enabled")] - pub runtime_enabled: bool, - /// Local provider identifier. Supported values are `ollama`, `lm_studio`, - /// and `omlx`; unknown values normalize to `ollama` at runtime. - #[serde(default = "default_provider")] - pub provider: String, - /// Optional provider base URL. For LM Studio this defaults to - /// `http://localhost:1234/v1`. - #[serde(default)] - pub base_url: Option, - #[serde(default)] - pub api_key: Option, - #[serde(default = "default_model_id")] - pub model_id: String, - #[serde(default = "default_chat_model_id")] - pub chat_model_id: String, - #[serde(default = "default_vision_model_id")] - pub vision_model_id: String, - #[serde(default = "default_embedding_model_id")] - pub embedding_model_id: String, - #[serde(default = "default_stt_model_id")] - pub stt_model_id: String, - #[serde(default = "default_stt_download_url")] - pub stt_download_url: Option, - /// Legacy voice STT routing string. `"cloud"` (the default) means "use - /// `voice_server.stt_engine`"; a third-party `"[:]"` overrides - /// the engine outright. The local `"whisper"` value it once accepted is - /// dead — `config::migrations` rewrites it back to `"cloud"`. - #[serde(default = "default_stt_provider")] - pub stt_provider: String, - #[serde(default = "default_tts_voice_id")] - pub tts_voice_id: String, - /// Voice TTS provider selector. `"cloud"` (default) routes through the - /// backend ElevenLabs proxy and returns rich visemes; `"piper"` runs - /// local Piper via the `PIPER_BIN` env var. - #[serde(default = "default_tts_provider")] - pub tts_provider: String, - #[serde(default = "default_tts_download_url")] - pub tts_download_url: Option, - #[serde(default = "default_tts_config_download_url")] - pub tts_config_download_url: Option, - #[serde(default = "default_quantization")] - pub quantization: String, - #[serde(default = "default_preload_vision_model")] - pub preload_vision_model: bool, - #[serde(default = "default_preload_embedding_model")] - pub preload_embedding_model: bool, - #[serde(default = "default_preload_stt_model")] - pub preload_stt_model: bool, - #[serde(default = "default_preload_tts_voice")] - pub preload_tts_voice: bool, - #[serde(default = "default_download_url")] - pub download_url: Option, - #[serde(default = "default_autosummary_debounce_ms")] - pub autosummary_debounce_ms: u64, - #[serde(default)] - pub selected_tier: Option, - /// Explicit MVP opt-in marker. Bootstrap disables local AI unless this is - /// `true`, regardless of any prior `selected_tier` value. Existing installs - /// (upgrading from pre-MVP) default to `false` and must re-opt-in from - /// Settings. Set by `apply_preset` on any non-disabled tier. - #[serde(default)] - pub opt_in_confirmed: bool, - /// Optional path to a manually-installed Ollama binary. - #[serde(default)] - pub ollama_binary_path: Option, - /// When true and Ollama is available, pass raw transcription through a - /// local LLM to fix grammar/punctuation using conversation context. - #[serde(default = "default_voice_llm_cleanup_enabled")] - pub voice_llm_cleanup_enabled: bool, - /// Ollama `options.num_ctx` override. When set, every chat request to - /// an Ollama provider includes `"options": {"num_ctx": }` so - /// the model allocates at least this much KV-cache. Ollama defaults - /// to 2048 for many models which is too small for agentic use. - #[serde(default)] - pub num_ctx: Option, - /// Per-feature flags. Each gate is AND-ed with `runtime_enabled`. - /// All default to `false` (cloud path). - #[serde(default)] - pub usage: LocalAiUsage, -} - -fn default_runtime_enabled() -> bool { - false -} - -fn default_provider() -> String { - "ollama".to_string() -} - -fn default_model_id() -> String { - "gemma3:1b-it-qat".to_string() -} - -fn default_chat_model_id() -> String { - "gemma3:1b-it-qat".to_string() -} - -fn default_vision_model_id() -> String { - String::new() -} - -fn default_embedding_model_id() -> String { - // bge-m3 (1024 dims, 8192-token context). Required by the memory tree's - // fixed on-disk embedding format (EMBEDDING_DIM=1024) — `all-minilm` - // (384 dims) and `nomic-embed-text` (768 dims) would fail the - // post-call dim validator at `memory::tree::score::embed::mod::embed`. - "bge-m3".to_string() -} - -fn default_stt_model_id() -> String { - "ggml-base-q5_1.bin".to_string() -} - -fn default_tts_voice_id() -> String { - "en_US-lessac-medium".to_string() -} - -fn default_stt_provider() -> String { - "cloud".to_string() -} - -fn default_tts_provider() -> String { - "cloud".to_string() -} - -fn default_stt_download_url() -> Option { - Some( - "https://huggingface.co/ggerganov/whisper.cpp/resolve/main/ggml-base-q5_1.bin?download=true" - .to_string(), - ) -} - -fn default_tts_download_url() -> Option { - Some( - "https://huggingface.co/rhasspy/piper-voices/resolve/main/en/en_US/lessac/medium/en_US-lessac-medium.onnx?download=true" - .to_string(), - ) -} - -fn default_tts_config_download_url() -> Option { - Some( - "https://huggingface.co/rhasspy/piper-voices/resolve/main/en/en_US/lessac/medium/en_US-lessac-medium.onnx.json?download=true" - .to_string(), - ) -} - -fn default_quantization() -> String { - "q4".to_string() -} - -fn default_preload_vision_model() -> bool { - false -} - -fn default_preload_embedding_model() -> bool { - true -} - -fn default_preload_stt_model() -> bool { - false -} - -fn default_preload_tts_voice() -> bool { - false -} - -fn default_download_url() -> Option { - None -} - -fn default_autosummary_debounce_ms() -> u64 { - 2500 -} - -fn default_voice_llm_cleanup_enabled() -> bool { - true -} - -impl LocalAiConfig { - /// Returns `true` when the local Ollama runtime is active. - /// This is the primary gate; all per-feature helpers below AND with this. - pub fn is_active(&self) -> bool { - self.runtime_enabled - } - - /// **Deprecated** — read from `Config::workload_uses_local("embeddings")` - /// instead. This helper only consults the legacy `usage.*` booleans, which - /// are no longer the source of truth after the unified AI settings - /// migration (schema_version >= 2). - #[deprecated(note = "Use Config::workload_uses_local(\"embeddings\")")] - pub fn use_local_for_embeddings(&self) -> bool { - self.runtime_enabled && self.usage.embeddings - } - - /// **Deprecated** — read from `Config::workload_uses_local("heartbeat")`. - #[deprecated(note = "Use Config::workload_uses_local(\"heartbeat\")")] - pub fn use_local_for_heartbeat(&self) -> bool { - self.runtime_enabled && self.usage.heartbeat - } - - /// **Deprecated** — read from `Config::workload_uses_local("learning")`. - #[deprecated(note = "Use Config::workload_uses_local(\"learning\")")] - pub fn use_local_for_learning(&self) -> bool { - self.runtime_enabled && self.usage.learning_reflection - } - - /// **Deprecated** — read from `Config::workload_uses_local("subconscious")`. - #[deprecated(note = "Use Config::workload_uses_local(\"subconscious\")")] - pub fn use_local_for_subconscious(&self) -> bool { - self.runtime_enabled && self.usage.subconscious - } -} - -impl Default for LocalAiConfig { - fn default() -> Self { - Self { - runtime_enabled: default_runtime_enabled(), - provider: default_provider(), - base_url: None, - api_key: None, - model_id: default_model_id(), - chat_model_id: default_chat_model_id(), - vision_model_id: default_vision_model_id(), - embedding_model_id: default_embedding_model_id(), - stt_model_id: default_stt_model_id(), - stt_download_url: default_stt_download_url(), - stt_provider: default_stt_provider(), - tts_voice_id: default_tts_voice_id(), - tts_provider: default_tts_provider(), - tts_download_url: default_tts_download_url(), - tts_config_download_url: default_tts_config_download_url(), - quantization: default_quantization(), - preload_vision_model: default_preload_vision_model(), - preload_embedding_model: default_preload_embedding_model(), - preload_stt_model: default_preload_stt_model(), - preload_tts_voice: default_preload_tts_voice(), - download_url: default_download_url(), - autosummary_debounce_ms: default_autosummary_debounce_ms(), - selected_tier: None, - opt_in_confirmed: false, - ollama_binary_path: None, - voice_llm_cleanup_enabled: default_voice_llm_cleanup_enabled(), - num_ctx: None, - usage: LocalAiUsage::default(), - } - } -} diff --git a/src/openhuman/memory/api/host/mod.rs b/src/openhuman/memory/api/host/mod.rs deleted file mode 100644 index 5d3e1d6f9b..0000000000 --- a/src/openhuman/memory/api/host/mod.rs +++ /dev/null @@ -1,93 +0,0 @@ -//! The **host seam** — everything `tinymemory-core` needs from the application -//! that embeds it, expressed as object-safe traits plus the plain serde config -//! structs the memory subsystem owns. -//! -//! # Why this module exists -//! -//! `tinymemory-core` holds the substance of a memory subsystem: the store, the -//! summary tree, the sync pipelines, ingestion, recall. Per the repository -//! README's split, the *host* keeps the RPC surface, the agent tools, the -//! security policy, the schedulers, the event bus, and config loading. That -//! split only works if the core can name what it needs from the host without -//! naming the host itself — which is what these traits are. -//! -//! # The three seams -//! -//! - [`MemoryHostConfig`] — the host's configuration, read through accessor -//! methods rather than public fields. `crate::openhuman::memory::core_impl::Config` is the type -//! alias `dyn MemoryHostConfig`, so code moved out of the host keeps writing -//! `config: &Config` and the host's concrete `Config` unsize-coerces at every -//! call site. -//! - [`EmbeddingProvider`] — text → vector. The core never builds one; the host -//! resolves provider credentials, rate limits and routing and hands an -//! `Arc` down. -//! - [`MemoryEventSink`] — the handful of domain events the memory subsystem -//! publishes. The host implements it by publishing its own event enum onto -//! its own bus; the core never learns that enum exists. -//! -//! # Config *sections* live here, config *loading* does not -//! -//! [`MemoryConfig`], [`MemoryTreeConfig`], [`MemorySubsystemConfig`] and friends -//! moved here from the host because the core reads their fields directly and a -//! trait accessor per field would be absurd. They are inert serde/`schemars` -//! data with no behaviour, and **their serde representation is persisted in -//! users' `config.toml`** — field names, defaults, and `#[serde(...)]` -//! attributes are a compatibility surface, not an implementation detail. -//! -//! Sections that are *not* memory-owned but that the core still reads -//! ([`LocalAiConfig`], [`cloud_providers`]) are here for the same mechanical -//! reason. They are the seam's rough edge: the honest fix is to move embedding -//! *construction* back into the host, at which point the core stops reading -//! them and they can go home. - -pub mod cloud_providers; -pub mod composio; -pub mod local_ai; -pub mod scheduler_gate; -pub mod storage_memory; -pub mod subsystems; - -mod config; -mod embedding_host; -mod embeddings; -mod error_reporter; -mod events; -mod evidence; -mod nlp; -mod routes; -mod usage; - -#[cfg(test)] -pub mod test_support; - -pub use cloud_providers::{ - endpoint_host, generate_provider_id, is_slug_reserved, migrate_legacy_fields, AuthStyle, - CloudProviderCreds, CloudProviderType, -}; -pub use config::{ComposioMode, MemoryHostConfig, COMPOSIO_MODE_BACKEND, COMPOSIO_MODE_DIRECT}; -pub use embedding_host::EmbeddingHost; -pub use embeddings::{format_embedding_signature, EmbeddingProvider, NoopEmbedding}; -pub use error_reporter::ErrorReporter; -pub use events::{ - EmbeddingHealthReason, MemoryEvent, MemoryEventSink, NoopEventSink, SyncTrigger, - LOCAL_MODEL_UNAVAILABLE_KIND, MEMORY_USER_ERROR_SOURCE, -}; -pub use evidence::EvidenceRef; -pub use local_ai::{LocalAiConfig, LocalAiUsage}; -pub use nlp::{SpacyEntity, SpacyResponse}; -pub use routes::EmbeddingRouteConfig; -pub use scheduler_gate::{PauseReason, Policy, SchedulerGateConfig, SchedulerGateMode}; -pub use storage_memory::{ - LlmBackend, MemoryConfig, MemoryTreeConfig, StorageConfig, StorageProviderConfig, - StorageProviderSection, DEFAULT_CLOUD_LLM_MODEL, -}; -pub use subsystems::{ - MemoryDriverConfig, MemoryHooksConfig, MemorySubsystemConfig, SubsystemsConfig, -}; -pub use usage::UsageInfo; - -/// Effective default global memory-sync cadence (seconds) used when -/// [`MemoryHostConfig::memory_sync_interval_secs`] is `None` — i.e. the user has -/// not explicitly picked a schedule. 24h, matching the "Sync every 24h" preset -/// surfaced in the Memory Sources UI. -pub const DEFAULT_MEMORY_SYNC_INTERVAL_SECS: u64 = 86_400; diff --git a/src/openhuman/memory/api/host/nlp.rs b/src/openhuman/memory/api/host/nlp.rs deleted file mode 100644 index 46e66605c4..0000000000 --- a/src/openhuman/memory/api/host/nlp.rs +++ /dev/null @@ -1,30 +0,0 @@ -//! spaCy extraction results — the wire shape of the host's Python NLP server. -//! -//! Moved here from the host's `runtime::python_server::spacy` because the -//! summary tree's query-entity extractor consumes them directly, canonicalising -//! each entity into the same `:` namespace the indexed chunks use. -//! Inert serde data. -//! -//! Provisioning the runtime (`ensure_spacy`, `spacy_provisioned`, the model id) -//! deliberately stayed in the host: downloading and launching a Python server -//! is not something a memory engine should do. - -use serde::{Deserialize, Serialize}; - -#[derive(Debug, Clone, Serialize, Deserialize)] -pub struct SpacyEntity { - pub text: String, - pub label: String, - #[serde(default)] - pub start: u32, - #[serde(default)] - pub end: u32, -} - -#[derive(Debug, Clone, Serialize, Deserialize)] -pub struct SpacyResponse { - #[serde(default)] - pub entities: Vec, - #[serde(default)] - pub nouns: Vec, -} diff --git a/src/openhuman/memory/api/host/routes.rs b/src/openhuman/memory/api/host/routes.rs deleted file mode 100644 index f5204e5736..0000000000 --- a/src/openhuman/memory/api/host/routes.rs +++ /dev/null @@ -1,18 +0,0 @@ -//! [`EmbeddingRouteConfig`] — a per-workload embedding provider override. -//! -//! Moved here from the host's `config::schema::routes` because the memory -//! store's factory reads its fields directly when resolving which embedder -//! backs a workload. Inert serde data; **its serde form is persisted** in -//! users' `config.toml`. - -use schemars::JsonSchema; -use serde::{Deserialize, Serialize}; - -#[derive(Debug, Clone, Serialize, Deserialize, JsonSchema)] -pub struct EmbeddingRouteConfig { - pub hint: String, - pub provider: String, - pub model: String, - #[serde(default)] - pub dimensions: Option, -} diff --git a/src/openhuman/memory/api/host/scheduler_gate.rs b/src/openhuman/memory/api/host/scheduler_gate.rs deleted file mode 100644 index 9d3b4e3c12..0000000000 --- a/src/openhuman/memory/api/host/scheduler_gate.rs +++ /dev/null @@ -1,188 +0,0 @@ -//! Scheduler-gate configuration — controls when background AI work runs. -//! -//! Consumed by `openhuman::cron::scheduler_gate`. - -use schemars::JsonSchema; -use serde::{Deserialize, Serialize}; - -#[derive(Debug, Clone, Copy, Serialize, Deserialize, JsonSchema, PartialEq, Eq)] -#[serde(rename_all = "snake_case")] -#[derive(Default)] -pub enum SchedulerGateMode { - /// Decide based on power + CPU + deployment-mode signals. - #[default] - Auto, - /// Always run background AI flat-out (server / power-user setting). - AlwaysOn, - /// Never run background AI. User can still trigger work explicitly. - Off, -} - -impl SchedulerGateMode { - pub fn as_str(self) -> &'static str { - match self { - Self::Auto => "auto", - Self::AlwaysOn => "always_on", - Self::Off => "off", - } - } -} - -#[derive(Debug, Clone, Serialize, Deserialize, JsonSchema)] -#[serde(default)] -pub struct SchedulerGateConfig { - /// Top-level mode — `auto` (default), `always_on`, or `off`. - #[serde(default)] - pub mode: SchedulerGateMode, - - /// Battery charge floor in `auto` mode, 0.0..=1.0. Below this and not on - /// AC, the gate throttles. Default: 0.80. - #[serde(default = "default_battery_floor")] - pub battery_floor: f32, - - /// CPU busy threshold (recent global usage, 0..100). Above this, the gate - /// throttles even when plugged in. Default: 70.0 (i.e. <30% headroom). - #[serde(default = "default_cpu_busy_threshold")] - pub cpu_busy_threshold_pct: f32, - - /// In `Throttled` mode, sleep this many ms before each LLM-bound job to - /// serialise workers and let the host catch up. Default: 30_000 (30s). - #[serde(default = "default_throttled_backoff_ms")] - pub throttled_backoff_ms: u64, - - /// In `Paused` mode, re-check the policy every this many ms so workers - /// resume promptly when the user toggles the gate back on. Default: - /// 60_000 (60s). - #[serde(default = "default_paused_poll_ms")] - pub paused_poll_ms: u64, - - /// Hard CPU ceiling (recent global usage, 0..100). When the host CPU - /// climbs above this in `auto` mode, the gate flips to - /// `Paused { CpuPressure }` rather than just `Throttled` — every - /// background LLM call is held until the host calms down. Distinct - /// from `cpu_busy_threshold_pct`, which only triggers `Throttled`. - /// Default: 95.0. - #[serde(default = "default_cpu_severe_pct")] - pub cpu_severe_pct: f32, - - /// When `true`, `auto` mode only runs background LLM work while the - /// laptop is on AC power. On battery the gate flips to - /// `Paused { OnBattery }` — no background inference at all, - /// regardless of charge level. - /// - /// Default `false` to preserve the prior behavior (battery-floor - /// based throttling). Power-conscious users who never want - /// background inference on battery can flip this on. - #[serde(default)] - pub require_ac_power: bool, -} - -fn default_battery_floor() -> f32 { - 0.80 -} -fn default_cpu_busy_threshold() -> f32 { - 70.0 -} -fn default_throttled_backoff_ms() -> u64 { - 30_000 -} -fn default_paused_poll_ms() -> u64 { - 60_000 -} -fn default_cpu_severe_pct() -> f32 { - 95.0 -} - -impl Default for SchedulerGateConfig { - fn default() -> Self { - Self { - mode: SchedulerGateMode::default(), - battery_floor: default_battery_floor(), - cpu_busy_threshold_pct: default_cpu_busy_threshold(), - throttled_backoff_ms: default_throttled_backoff_ms(), - paused_poll_ms: default_paused_poll_ms(), - cpu_severe_pct: default_cpu_severe_pct(), - require_ac_power: false, - } - } -} - -// ── Gate decision vocabulary ──────────────────────────────────────────────── -// -// `Policy` and `PauseReason` moved here from the host's -// `cron::scheduler_gate::policy` because the extracted sync loops read them on -// every tick to decide whether to back off. They are inert `Copy` enums with no -// dependencies; the *decision function* that produces a `Policy` from sampled -// signals stays in the host, where the signals are. - -/// Why the gate is currently paused. Carried by [`Policy::Paused`] so -/// downstream consumers (UI, logging, observability) can surface a -/// specific user-facing reason instead of a generic "paused" label. -/// -/// New variants will land alongside #1073's full power-aware work -/// (`OnBattery`, `CpuPressure`); `UserDisabled` covers the existing -/// `SchedulerGateMode::Off` path and `Unknown` is the safe fallback for -/// callers that don't have specific context yet. -#[derive(Debug, Clone, Copy, PartialEq, Eq)] -pub enum PauseReason { - /// User explicitly turned the gate off in config. - UserDisabled, - /// Host on battery and gate's power-aware mode kicked in (#1073). - OnBattery, - /// CPU pressure exceeded the gate threshold (#1073). - CpuPressure, - /// No active app session — background AI work is suspended until the - /// user signs in again. Trumps every other signal: while signed out - /// the host should do *no* LLM-bound work, period. Set by - /// `gate::set_signed_out(true)` from the credentials lifecycle and - /// from 401-detection sites. - SignedOut, - /// Pause reason not yet classified — placeholder while #1073 is in flight. - Unknown, -} - -impl PauseReason { - pub fn as_str(self) -> &'static str { - match self { - Self::UserDisabled => "user_disabled", - Self::OnBattery => "on_battery", - Self::CpuPressure => "cpu_pressure", - Self::SignedOut => "signed_out", - Self::Unknown => "unknown", - } - } -} - -/// Background-AI scheduling tier. See module docs in `mod.rs` for semantics. -#[derive(Debug, Clone, Copy, PartialEq, Eq)] -pub enum Policy { - Aggressive, - Normal, - Throttled, - /// Gate paused. The `reason` is rendered to users in the memory-sync - /// status UI (#1136) and recorded in observability. - Paused { - reason: PauseReason, - }, -} - -impl Policy { - pub fn as_str(self) -> &'static str { - match self { - Self::Aggressive => "aggressive", - Self::Normal => "normal", - Self::Throttled => "throttled", - Self::Paused { .. } => "paused", - } - } - - /// `Some(reason)` when paused, `None` otherwise. Convenience for - /// callers that only need the reason and don't want to pattern-match - /// the whole enum (UI badges, log line construction). - pub fn pause_reason(self) -> Option { - match self { - Self::Paused { reason } => Some(reason), - _ => None, - } - } -} diff --git a/src/openhuman/memory/api/host/storage_memory.rs b/src/openhuman/memory/api/host/storage_memory.rs deleted file mode 100644 index 928f1da038..0000000000 --- a/src/openhuman/memory/api/host/storage_memory.rs +++ /dev/null @@ -1,561 +0,0 @@ -//! Storage provider and memory configuration. - -use schemars::JsonSchema; -use serde::{Deserialize, Serialize}; -use std::path::PathBuf; - -#[derive(Debug, Clone, Serialize, Deserialize, Default, JsonSchema)] -#[serde(default)] -pub struct StorageConfig { - #[serde(default)] - pub provider: StorageProviderSection, -} - -#[derive(Debug, Clone, Serialize, Deserialize, Default, JsonSchema)] -#[serde(default)] -pub struct StorageProviderSection { - #[serde(default)] - pub config: StorageProviderConfig, -} - -#[derive(Debug, Clone, Serialize, Deserialize, JsonSchema)] -#[serde(default)] -#[derive(Default)] -pub struct StorageProviderConfig { - #[serde(default)] - pub provider: String, -} - -#[derive(Clone, Serialize, Deserialize, JsonSchema)] -#[allow(clippy::struct_excessive_bools)] -#[serde(default)] -pub struct MemoryConfig { - #[serde(default = "default_memory_backend")] - pub backend: String, - #[serde(default = "default_true")] - pub auto_save: bool, - #[serde(default = "default_embedding_provider")] - pub embedding_provider: String, - #[serde(default = "default_embedding_model")] - pub embedding_model: String, - #[serde(default = "default_embedding_dims")] - pub embedding_dimensions: usize, - /// Outbound embedding-request budget for cloud providers, in requests per - /// minute. Cloud backends (OpenHuman/Voyage, OpenAI, remote `custom:` - /// endpoints) cap requests per account; the client throttles to stay under - /// that quota rather than tripping 429s. `0` disables throttling. Loopback - /// endpoints are always exempt. Env override: - /// `OPENHUMAN_MEMORY_EMBED_RATE_LIMIT`. - #[serde(default = "default_embedding_rate_limit_per_min")] - pub embedding_rate_limit_per_min: u32, - #[serde(default = "default_min_relevance_score")] - pub min_relevance_score: f64, - #[serde(default)] - pub sqlite_open_timeout_secs: Option, - - /// Base URL for the `agentmemory` REST server. Honored only when - /// `backend = "agentmemory"`. Defaults to `http://localhost:3111` - /// (the agentmemory loopback default). - #[serde(default)] - pub agentmemory_url: Option, - - /// Optional bearer token sent as `Authorization: Bearer ` - /// to the agentmemory REST server. When unset, the backend speaks - /// to a local agentmemory daemon without authentication. Setting a - /// secret + a non-loopback host enables the v0.9.12 plaintext-bearer - /// guard semantics on the client side: the backend refuses to send - /// the token over plaintext HTTP when the host is not loopback. - #[serde(default)] - pub agentmemory_secret: Option, - - /// Per-request timeout for the agentmemory REST client, in - /// milliseconds. Defaults to 5000 ms. - #[serde(default)] - pub agentmemory_timeout_ms: Option, -} - -fn default_memory_backend() -> String { - "sqlite".into() -} - -fn default_true() -> bool { - true -} - -fn default_embedding_provider() -> String { - // Default to the OpenHuman backend (Voyage-backed `embedding-v1`) so a - // fresh install works without requiring a local Ollama daemon. Users - // who want fully-local embeddings can flip this to "ollama" in - // `config.toml` or enable `local_ai.usage.embeddings = true`, which is - // wired into the memory factory via `LocalAiConfig::use_local_for_embeddings`. - "cloud".into() -} -fn default_embedding_model() -> String { - // Keep this in sync with `embeddings::cloud::DEFAULT_CLOUD_EMBEDDING_MODEL`. - "embedding-v1".into() -} -fn default_embedding_dims() -> usize { - // Keep this in sync with `embeddings::cloud::DEFAULT_CLOUD_EMBEDDING_DIMENSIONS`. - 1024 -} -fn default_embedding_rate_limit_per_min() -> u32 { - // Cloud embedding backends cap requests at ~60/min per account. Keep in - // sync with `embeddings::rate_limit::DEFAULT_EMBEDDING_RATE_LIMIT_PER_MIN`. - 60 -} -fn default_min_relevance_score() -> f64 { - 0.4 -} - -impl Default for MemoryConfig { - fn default() -> Self { - Self { - backend: default_memory_backend(), - auto_save: default_true(), - embedding_provider: default_embedding_provider(), - embedding_model: default_embedding_model(), - embedding_dimensions: default_embedding_dims(), - embedding_rate_limit_per_min: default_embedding_rate_limit_per_min(), - min_relevance_score: default_min_relevance_score(), - sqlite_open_timeout_secs: None, - agentmemory_url: None, - agentmemory_secret: None, - agentmemory_timeout_ms: None, - } - } -} - -// Manual `Debug` implementation that redacts `agentmemory_secret`. Without -// this, any `format!("{cfg:?}")` / `tracing::debug!(?cfg, ...)` / panic -// message capturing a `MemoryConfig` would dump the bearer token in -// plaintext — directly against the repo rule "Never log secrets, raw -// JWTs, API keys, credentials, or full PII in debug logs". -impl std::fmt::Debug for MemoryConfig { - fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { - f.debug_struct("MemoryConfig") - .field("backend", &self.backend) - .field("auto_save", &self.auto_save) - .field("embedding_provider", &self.embedding_provider) - .field("embedding_model", &self.embedding_model) - .field("embedding_dimensions", &self.embedding_dimensions) - .field( - "embedding_rate_limit_per_min", - &self.embedding_rate_limit_per_min, - ) - .field("min_relevance_score", &self.min_relevance_score) - .field("sqlite_open_timeout_secs", &self.sqlite_open_timeout_secs) - .field("agentmemory_url", &self.agentmemory_url) - .field( - "agentmemory_secret", - &self.agentmemory_secret.as_ref().map(|_| ""), - ) - .field("agentmemory_timeout_ms", &self.agentmemory_timeout_ms) - .finish() - } -} - -/// Which inference backend the memory_tree's LLM calls (extractor + -/// summariser) should use. -/// -/// - `Cloud` (default): route through `providers::router` against the -/// OpenHuman backend with the `summarization-v1` model. No local Ollama -/// required. -/// - `Local`: keep using the legacy Ollama-direct path (the -/// `llm_extractor_endpoint` / `llm_summariser_endpoint` config). Useful -/// for offline development and CI smoke tests. -/// -/// Embedder selection is unchanged — `OllamaEmbedder` (bge-m3) stays -/// local-only and isn't governed by this enum. -#[derive(Debug, Clone, Copy, Eq, PartialEq, Serialize, Deserialize, JsonSchema)] -#[serde(rename_all = "lowercase")] -#[derive(Default)] -pub enum LlmBackend { - /// Route through the OpenHuman backend (default). - #[default] - Cloud, - /// Use the local Ollama path configured via `llm_extractor_*` / - /// `llm_summariser_*`. - Local, -} - -impl LlmBackend { - /// Stable wire string for env vars / RPCs / logs. - pub fn as_str(self) -> &'static str { - match self { - Self::Cloud => "cloud", - Self::Local => "local", - } - } - - /// Inverse of [`Self::as_str`]; case-insensitive parse. - pub fn parse(s: &str) -> Result { - match s.trim().to_ascii_lowercase().as_str() { - "cloud" => Ok(Self::Cloud), - "local" => Ok(Self::Local), - other => Err(format!("unknown llm (expected cloud|local): {other}")), - } - } -} - -fn default_llm_backend() -> LlmBackend { - LlmBackend::default() -} - -/// Default model identifier to use when `llm_backend = "cloud"`. Routed -/// through the OpenHuman backend; keep in sync with the backend's -/// summariser model registry. -pub const DEFAULT_CLOUD_LLM_MODEL: &str = "summarization-v1"; - -fn default_cloud_llm_model() -> Option { - Some(DEFAULT_CLOUD_LLM_MODEL.to_string()) -} - -/// Phase 4 memory-tree configuration — embedding provider wiring for the -/// hierarchical memory (#710). -/// -/// When `embedding_endpoint` and `embedding_model` are both set, ingest -/// and bucket-seal route every new chunk/summary through the Ollama -/// embedder before writing. When unset, behaviour depends on -/// `embedding_strict`: -/// - `true` (default): ingest/seal bail with a clear config error. -/// - `false`: fall back to the inert zero-vector embedder and warn. -/// -/// Env overrides apply in `openhuman::config::schema::load`: -/// - `OPENHUMAN_MEMORY_EMBED_ENDPOINT` -/// - `OPENHUMAN_MEMORY_EMBED_MODEL` -/// - `OPENHUMAN_MEMORY_EMBED_TIMEOUT_MS` -/// - `OPENHUMAN_MEMORY_EXTRACT_ENDPOINT` -/// - `OPENHUMAN_MEMORY_EXTRACT_MODEL` -/// - `OPENHUMAN_MEMORY_EXTRACT_TIMEOUT_MS` -/// - `OPENHUMAN_MEMORY_SUMMARISE_ENDPOINT` -/// - `OPENHUMAN_MEMORY_SUMMARISE_MODEL` -/// - `OPENHUMAN_MEMORY_SUMMARISE_TIMEOUT_MS` -/// - `OPENHUMAN_MEMORY_TREE_CONTENT_DIR` (Phase MD-content) -/// - `OPENHUMAN_MEMORY_TREE_LLM_BACKEND` (cloud|local) -/// - `OPENHUMAN_MEMORY_TREE_CLOUD_LLM_MODEL` -#[derive(Debug, Clone, Serialize, Deserialize, JsonSchema)] -#[serde(default)] -pub struct MemoryTreeConfig { - /// Ollama endpoint for the embedder (e.g. `http://localhost:11434`). - /// `None` disables the Ollama path — see `embedding_strict` for the - /// resulting behaviour. - #[serde(default = "default_memory_tree_embedding_endpoint")] - pub embedding_endpoint: Option, - - /// Embedding model name. Must produce 768-dim vectors (see - /// `memory::tree::score::embed::EMBEDDING_DIM`). `None` disables - /// the Ollama path. - #[serde(default = "default_memory_tree_embedding_model")] - pub embedding_model: Option, - - /// Per-request timeout for the embedder, in milliseconds. - #[serde(default = "default_memory_tree_embedding_timeout_ms")] - pub embedding_timeout_ms: Option, - - /// When true, ingest/seal refuse to run with embeddings disabled. - /// When false, an inert zero-vector embedder is used and retrieval - /// rerank falls back to scope + recency ordering only. - #[serde(default = "default_memory_tree_embedding_strict")] - pub embedding_strict: bool, - - /// Ollama endpoint for the LLM entity extractor - /// (`memory::tree::score::extract::llm::LlmEntityExtractor`). - /// Defaults to `Some("http://localhost:11434")` — the standard - /// Ollama listener — see `default_memory_tree_llm_endpoint`. - /// Soft failures in the LLM path fall back to regex-only for - /// that chunk. - #[serde(default = "default_memory_tree_llm_endpoint")] - pub llm_extractor_endpoint: Option, - - /// Model name for the entity extractor. Defaults to `gemma3:4b` - /// (see `default_memory_tree_llm_model` for the rationale); - /// override to a smaller model on resource-constrained hosts. - #[serde(default = "default_memory_tree_llm_model")] - pub llm_extractor_model: Option, - - /// Per-request timeout for the LLM extractor, in milliseconds. - #[serde(default = "default_memory_tree_llm_extractor_timeout_ms")] - pub llm_extractor_timeout_ms: Option, - - /// Ollama endpoint for the summariser - /// (`memory::tree::tree_source::summariser::llm::LlmSummariser`). - /// Defaults to `Some("http://localhost:11434")` — see - /// `default_memory_tree_llm_endpoint`. Soft failures fall back - /// to `InertSummariser` per seal. - #[serde(default = "default_memory_tree_llm_endpoint")] - pub llm_summariser_endpoint: Option, - - /// Model name for the summariser. Defaults to `gemma3:4b` — - /// larger Gemma tiers (`gemma3:12b-it-qat`, `gemma3:27b`) produce - /// more coherent abstractive summaries at higher latency. See - /// `default_memory_tree_llm_model`. - #[serde(default = "default_memory_tree_llm_model")] - pub llm_summariser_model: Option, - - /// Per-request timeout for the summariser, in milliseconds. Default - /// is higher than the extractor because summarisation uses more - /// tokens and therefore takes longer to generate. - #[serde(default = "default_memory_tree_llm_summariser_timeout_ms")] - pub llm_summariser_timeout_ms: Option, - - /// Phase MD-content: root directory where chunk `.md` files are stored. - /// - /// Resolved at runtime via `MemoryHostConfig::memory_tree_content_root`: - /// - `Some(path)` → use that path verbatim. - /// - `None` → default `/memory_tree/content/`. - /// - /// Env override: `OPENHUMAN_MEMORY_TREE_CONTENT_DIR` (empty string = fall - /// back to default, consistent with other memory_tree env vars). - #[serde(default = "default_memory_tree_content_dir")] - pub content_dir: Option, - - /// Backend selector for the memory_tree's LLM calls (extractor + - /// summariser). Defaults to [`LlmBackend::Cloud`] so a fresh install - /// works without requiring a local Ollama daemon. Set to - /// [`LlmBackend::Local`] (or `OPENHUMAN_MEMORY_TREE_LLM_BACKEND=local`) to - /// keep the legacy Ollama-direct path. - /// - /// The embedder is unaffected by this setting — `OllamaEmbedder` (bge-m3) - /// stays local-only. - #[serde(default = "default_llm_backend")] - pub llm_backend: LlmBackend, - - /// **Deprecated / inert.** Formerly the model identifier for managed - /// (`llm_backend = "cloud"`) summarization. The managed summarization tier is - /// now fixed at `summarization-v1` - /// (`inference::provider::factory::summarization_tier_model`) - /// and this field is no longer consumed — the hosted backend serves exactly - /// one tier for this workload. Kept for config back-compat (existing - /// `config.toml` / `OPENHUMAN_MEMORY_TREE_CLOUD_LLM_MODEL` still parse without - /// error). To run summarization on a different model, point `memory_provider` - /// at a BYOK/local provider instead, where the model rides in the provider - /// string. - /// - /// Defaults to [`DEFAULT_CLOUD_LLM_MODEL`] (`summarization-v1`). - #[serde(default = "default_cloud_llm_model")] - pub cloud_llm_model: Option, - - /// Provider:model string for the smart_walk retrieval agent (e.g. - /// `"deepseek:deepseek-chat"`). When set, the smart walk loop uses this - /// model instead of the general memory/chat provider. Fast, cheap models - /// work best here since the walker makes many short-turn calls. - /// - /// Env override: `OPENHUMAN_MEMORY_TREE_SMART_WALK_MODEL`. - #[serde(default)] - pub smart_walk_model: Option, - - /// Explicit opt-in to cloud-based summarization when local AI is disabled. - /// - /// Default `false` — "Build Summary Trees" was local-only before #002. - /// Enabling this routes workspace memory summaries to the configured cloud - /// provider. Set `memory_tree.cloud_summarization_opt_in = true` or - /// `OPENHUMAN_MEMORY_TREE_CLOUD_SUMMARIZATION=true` to acknowledge that memory - /// content will be sent to an external service. - #[serde(default)] - pub cloud_summarization_opt_in: bool, - - /// Enable the spaCy NER sidecar used by the deterministic (E2GraphRAG) - /// retriever to extract entities from a query. When `true` (default), the - /// managed Python runtime provisions spaCy on first use and serves entity - /// extraction over stdio. When `false` — or whenever Python/spaCy is - /// unavailable — query-entity extraction falls back to the in-Rust - /// regex+LLM extractor (`score::extract`). Env override: - /// `OPENHUMAN_MEMORY_TREE_SPACY_ENABLED`. - #[serde(default = "default_memory_tree_spacy_enabled")] - pub spacy_enabled: bool, -} - -fn default_memory_tree_spacy_enabled() -> bool { - // Opt-in (#5056). Default OFF so a fresh install never provisions the spaCy - // venv + `en_core_web_sm` model on first launch, and the runtime Python - // server is not spawned on every boot when no local NLP is configured. - // Query-entity extraction degrades to the in-Rust regex+LLM extractor - // (`score::extract`); operators opt in via config or - // `OPENHUMAN_MEMORY_TREE_SPACY_ENABLED=1`. - false -} - -/// Returns `None` so that existing installs that never opted into Phase 4 -/// embeddings stay on the inert zero-vector path rather than suddenly -/// attempting to reach a local Ollama daemon they haven't configured. -/// Operators enable the Ollama path by setting either `embedding_endpoint` -/// in TOML or the `OPENHUMAN_MEMORY_EMBED_ENDPOINT` env var. -fn default_memory_tree_embedding_endpoint() -> Option { - None -} - -fn default_memory_tree_embedding_model() -> Option { - None -} - -fn default_memory_tree_embedding_timeout_ms() -> Option { - Some(10_000) -} - -/// Defaults to `false` so installs without an embedding endpoint fall back -/// to the inert zero-vector embedder (with a warn log) instead of refusing -/// to run. Set to `true` in production configs that require embeddings. -fn default_memory_tree_embedding_strict() -> bool { - false -} - -/// Shared `None` default for the LLM-path fields (extractor + summariser -/// endpoints + models). Keeping the same function for all of them makes -/// the intent explicit. -/// -/// Default points at the standard Ollama localhost listener. A user -/// who sets `llm_backend = "local"` plus a `_model` is clearly opting -/// into Ollama, and forcing them to also specify the endpoint just to -/// hit `localhost:11434` was a stealth foot-gun: the -/// `OllamaChatProvider` returned an error on an empty endpoint, which -/// the summariser silently swallowed into its `InertSummariser` -/// fallback — producing concat-and-truncate "summaries" that looked -/// correct but didn't run any LLM at all. With a default endpoint in -/// place, the only signal needed to enable a local LLM seal is a -/// non-empty `_model`. Override via TOML or -/// `OPENHUMAN_MEMORY_TREE_LLM_*_ENDPOINT` to point at a different -/// Ollama host. -fn default_memory_tree_llm_endpoint() -> Option { - Some("http://localhost:11434".to_string()) -} - -fn default_memory_tree_llm_extractor_timeout_ms() -> Option { - Some(15_000) -} - -fn default_memory_tree_llm_summariser_timeout_ms() -> Option { - // 120s — large enough for small/medium local models to finish a - // seal-budget summary on a cold-loaded weight cache. Tighter - // values cause the LlmSummariser to time out and silently fall - // back to InertSummariser (no LLM signal in the resulting node). - Some(120_000) -} - -/// Returns `None` so the default `/memory_tree/content/` path is -/// used unless explicitly overridden via TOML or env var. -fn default_memory_tree_content_dir() -> Option { - None -} - -/// Default Ollama model for the memory-tree LLMs (extractor + summariser). -/// -/// `gemma3:4b` is in the Gemma 3 family (Gemma 4 isn't released yet) -/// and sits between the 1B compact tier and the 12B/27B large tiers. -/// At ~3 GB on disk and ~8 GB RAM at inference it stays inside the -/// envelope of a typical laptop and produces coherent abstractive -/// summaries on real Gmail inboxes — smaller models (≤1.5B) regress -/// to "the email says X, the email says Y" enumeration that's barely -/// better than the InertSummariser concat fallback. -/// -/// Override via `memory_tree.llm_summariser_model` / -/// `llm_extractor_model` in TOML (or `OPENHUMAN_MEMORY_TREE_LLM_*_MODEL` -/// env vars) to scale up (`gemma3:12b-it-qat`, `llama3.1:8b`) or down -/// (`gemma3:1b-it-qat`) for the host's headroom. The frontend -/// `ModelCatalog` lists the curated picks the UI offers as -/// downloadable presets. -fn default_memory_tree_llm_model() -> Option { - Some("gemma3:4b".to_string()) -} - -impl Default for MemoryTreeConfig { - fn default() -> Self { - Self { - embedding_endpoint: default_memory_tree_embedding_endpoint(), - embedding_model: default_memory_tree_embedding_model(), - embedding_timeout_ms: default_memory_tree_embedding_timeout_ms(), - embedding_strict: default_memory_tree_embedding_strict(), - llm_extractor_endpoint: default_memory_tree_llm_endpoint(), - llm_extractor_model: default_memory_tree_llm_model(), - llm_extractor_timeout_ms: default_memory_tree_llm_extractor_timeout_ms(), - llm_summariser_endpoint: default_memory_tree_llm_endpoint(), - llm_summariser_model: default_memory_tree_llm_model(), - llm_summariser_timeout_ms: default_memory_tree_llm_summariser_timeout_ms(), - content_dir: default_memory_tree_content_dir(), - llm_backend: default_llm_backend(), - cloud_llm_model: default_cloud_llm_model(), - smart_walk_model: None, - cloud_summarization_opt_in: false, - spacy_enabled: default_memory_tree_spacy_enabled(), - } - } -} - -#[cfg(test)] -mod tests { - use super::*; - - #[test] - fn llm_default_is_cloud() { - assert_eq!(LlmBackend::default(), LlmBackend::Cloud); - assert_eq!(MemoryTreeConfig::default().llm_backend, LlmBackend::Cloud); - } - - #[test] - fn llm_round_trip() { - for v in [LlmBackend::Cloud, LlmBackend::Local] { - assert_eq!(LlmBackend::parse(v.as_str()).unwrap(), v); - } - } - - #[test] - fn llm_parse_is_case_insensitive() { - assert_eq!(LlmBackend::parse("CLOUD").unwrap(), LlmBackend::Cloud); - assert_eq!(LlmBackend::parse(" Local ").unwrap(), LlmBackend::Local); - } - - #[test] - fn llm_parse_rejects_unknown() { - assert!(LlmBackend::parse("hybrid").is_err()); - assert!(LlmBackend::parse("").is_err()); - } - - #[test] - fn cloud_llm_model_default_is_summarizer_v1() { - let cfg = MemoryTreeConfig::default(); - assert_eq!( - cfg.cloud_llm_model.as_deref(), - Some(DEFAULT_CLOUD_LLM_MODEL) - ); - assert_eq!(DEFAULT_CLOUD_LLM_MODEL, "summarization-v1"); - } - - /// #5056: spaCy is opt-in — a fresh install must never provision the - /// spaCy venv / `en_core_web_sm` model, nor spawn the runtime Python - /// server, without an explicit config or env-var opt-in. - #[test] - fn spacy_enabled_defaults_to_false() { - assert!(!MemoryTreeConfig::default().spacy_enabled); - assert!(!default_memory_tree_spacy_enabled()); - } - - #[test] - fn memory_tree_config_default_content_dir_is_none() { - let cfg = MemoryTreeConfig::default(); - assert!( - cfg.content_dir.is_none(), - "default content_dir must be None so workspace default path is used" - ); - } - - /// Verify that the env-var override logic correctly maps non-empty strings - /// to `Some(PathBuf)` and empty/blank strings to `None`. We test the - /// logic inline (not via `apply_env_overrides`) to avoid mutating the - /// process environment in a way that could race with parallel tests. - #[test] - fn content_dir_env_override_logic() { - // Simulate the load.rs overlay logic. - let apply = |raw: &str| -> Option { - let trimmed = raw.trim(); - if trimmed.is_empty() { - None - } else { - Some(PathBuf::from(trimmed)) - } - }; - - assert_eq!(apply("/tmp/foo"), Some(PathBuf::from("/tmp/foo"))); - assert_eq!(apply(" /tmp/foo "), Some(PathBuf::from("/tmp/foo"))); - assert_eq!(apply(""), None); - assert_eq!(apply(" "), None); - } -} diff --git a/src/openhuman/memory/api/host/subsystems.rs b/src/openhuman/memory/api/host/subsystems.rs deleted file mode 100644 index c68a9241df..0000000000 --- a/src/openhuman/memory/api/host/subsystems.rs +++ /dev/null @@ -1,261 +0,0 @@ -//! `[subsystems.*]` config section — the uniform cross-subsystem driver-binding -//! shape defined in `docs/specs/kernel.md` §3.6 and `docs/specs/plan-memory.md` §4.5. -//! -//! GREENFIELD / ZERO BEHAVIOUR CHANGE: nothing reads this config yet. It exists -//! so `[subsystems.memory]` can be authored today and so `inference`, -//! `channels`, `sandbox`, … can slot in later as sibling fields on -//! [`SubsystemsConfig`] without reshaping this type. -//! -//! Shape (kernel.md §3.6 / plan-memory.md §4.5): -//! -//! ```toml -//! [subsystems.memory] -//! driver = "tinymemory" -//! -//! [subsystems.memory.hooks] -//! auto_recall = true -//! auto_capture = true -//! max_context_tokens = 2000 -//! recall_max_chars = 1000 -//! capture_max_chars = 500 -//! -//! [subsystems.memory.drivers.supermemory] -//! class = "external" -//! transport = "http" -//! endpoint = "https://api.supermemory.ai" -//! credential_ref = "keychain:supermemory" -//! trust_state = "untrusted" -//! ``` - -use schemars::JsonSchema; -use serde::{Deserialize, Serialize}; -use std::collections::BTreeMap; - -/// Top-level `[subsystems]` config block. Currently carries only `memory`; -/// future subsystems (`inference`, `channels`, `sandbox`, …) are added here -/// as sibling fields — see kernel.md §3.6. -#[derive(Debug, Clone, Serialize, Deserialize, Default, JsonSchema)] -#[serde(default)] -pub struct SubsystemsConfig { - #[serde(default)] - pub memory: MemorySubsystemConfig, -} - -/// `[subsystems.memory]` — which driver is bound for the memory subsystem, -/// its hook budgets, and the per-driver option table. -/// -/// `PartialEq`/`Eq` let `CoreContext::rebind_workspace` short-circuit a -/// no-op rebind by comparing the config it was handed against the one already -/// held — equality is value comparison only, so it never prints or leaks the -/// credential fields the way `Debug` would. `Hash` lets `binding` -/// key its per-workspace cache on the whole config, so a changed driver/hooks/ -/// trust for an already-bound workspace yields a fresh binding rather than a -/// stale cache hit. -#[derive(Debug, Clone, PartialEq, Eq, Hash, Serialize, Deserialize, JsonSchema)] -#[serde(default)] -pub struct MemorySubsystemConfig { - /// The bound driver id (e.g. `"tinymemory"`, `"supermemory"`, `"null"`). - /// Must match a key under `drivers` when that driver needs options. - #[serde(default = "default_memory_driver")] - pub driver: String, - - #[serde(default)] - pub hooks: MemoryHooksConfig, - - /// Per-driver option tables, keyed by driver id. The module default - /// (`tinymemory`) needs no entry here — its options continue to live in - /// the existing `[memory]` / `[memory_tree]` / `[[memory_sources]]` - /// blocks (plan-memory.md §4.5: "no user-visible config break"). - #[serde(default)] - pub drivers: BTreeMap, -} - -fn default_memory_driver() -> String { - "tinymemory".into() -} - -impl Default for MemorySubsystemConfig { - fn default() -> Self { - Self { - driver: default_memory_driver(), - hooks: MemoryHooksConfig::default(), - drivers: BTreeMap::new(), - } - } -} - -/// Memory-hook budgets — the auto-recall / auto-capture behavior gating -/// values. Defaults reproduce today's (pre-`[subsystems]`) behavior exactly; -/// nothing reads these yet. -#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, Serialize, Deserialize, JsonSchema)] -#[serde(default)] -pub struct MemoryHooksConfig { - #[serde(default = "default_true")] - pub auto_recall: bool, - #[serde(default = "default_true")] - pub auto_capture: bool, - #[serde(default = "default_max_context_tokens")] - pub max_context_tokens: usize, - #[serde(default = "default_recall_max_chars")] - pub recall_max_chars: usize, - #[serde(default = "default_capture_max_chars")] - pub capture_max_chars: usize, -} - -fn default_true() -> bool { - true -} -fn default_max_context_tokens() -> usize { - 2000 -} -fn default_recall_max_chars() -> usize { - 1000 -} -fn default_capture_max_chars() -> usize { - 500 -} - -impl Default for MemoryHooksConfig { - fn default() -> Self { - Self { - auto_recall: default_true(), - auto_capture: default_true(), - max_context_tokens: default_max_context_tokens(), - recall_max_chars: default_recall_max_chars(), - capture_max_chars: default_capture_max_chars(), - } - } -} - -/// One entry under `[subsystems.memory.drivers.]`. Describes an -/// external/embedded driver binding — class, transport, endpoint, and a -/// *reference* to a credential resolved via the keychain (never an inline -/// secret; plan-memory.md §4.5, kernel.md §3.6). -/// -/// `trust_state` is fail-closed `"untrusted"` per kernel.md §3.4: an external -/// driver must have its trust explicitly raised before bind succeeds. -/// -/// MUST NOT derive `Debug` — see the manual impl below. `credential_ref` is a -/// secret handle and plan-memory.md §7 Tier-3 conformance requires "credential never -/// in `Debug`/error output", mirroring `storage_memory::MemoryConfig`'s -/// manual redacting `Debug` impl for `agentmemory_secret`. -/// -/// `PartialEq`/`Eq` are safe to derive: they compare values for equality and -/// never render them, so `credential_ref` stays out of any output. -#[derive(Clone, PartialEq, Eq, Hash, Serialize, Deserialize, JsonSchema)] -#[serde(default)] -pub struct MemoryDriverConfig { - /// Driver class: `"embedded"` | `"external"` | `"null"`. See kernel.md §3.1. - #[serde(default)] - pub class: Option, - - /// Wire transport for external drivers, e.g. `"http"`. See plan-memory.md §4.2. - #[serde(default)] - pub transport: Option, - - /// Base endpoint URL for external/http drivers. - #[serde(default)] - pub endpoint: Option, - - /// A *reference* to a credential (e.g. `"keychain:supermemory"`), - /// resolved kernel-side through the existing keychain — never an inline - /// secret. Redacted in `Debug`/error output; see the manual `Debug` impl. - #[serde(default)] - pub credential_ref: Option, - - /// Fail-closed trust state for this driver binding. Defaults to - /// `"untrusted"`; must be explicitly raised before an external driver's - /// bind succeeds (kernel.md §3.4). - #[serde(default = "default_trust_state")] - pub trust_state: String, -} - -fn default_trust_state() -> String { - "untrusted".into() -} - -impl Default for MemoryDriverConfig { - fn default() -> Self { - Self { - class: None, - transport: None, - endpoint: None, - credential_ref: None, - trust_state: default_trust_state(), - } - } -} - -// Manual `Debug` implementation that redacts `credential_ref`. Without this, -// any `format!("{cfg:?}")` / `tracing::debug!(?cfg, ...)` / panic message -// capturing a `MemoryDriverConfig` would dump the credential reference -// verbatim. The value itself (e.g. `"keychain:supermemory"`) is only a -// *reference*, not the secret — but plan-memory.md §7 Tier-3 conformance requires it -// never appear in Debug/error output regardless, so this mirrors -// `MemoryConfig`'s `agentmemory_secret` treatment exactly. NEVER derive -// `Debug` on this struct. -impl std::fmt::Debug for MemoryDriverConfig { - fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { - f.debug_struct("MemoryDriverConfig") - .field("class", &self.class) - .field("transport", &self.transport) - .field("endpoint", &self.endpoint) - .field( - "credential_ref", - &self.credential_ref.as_ref().map(|_| ""), - ) - .field("trust_state", &self.trust_state) - .finish() - } -} - -#[cfg(test)] -mod tests { - use super::*; - - #[test] - fn subsystems_config_defaults_reproduce_today_behavior() { - let cfg = SubsystemsConfig::default(); - assert_eq!(cfg.memory.driver, "tinymemory"); - assert!(cfg.memory.hooks.auto_recall); - assert!(cfg.memory.hooks.auto_capture); - assert_eq!(cfg.memory.hooks.max_context_tokens, 2000); - assert_eq!(cfg.memory.hooks.recall_max_chars, 1000); - assert_eq!(cfg.memory.hooks.capture_max_chars, 500); - assert!(cfg.memory.drivers.is_empty()); - } - - #[test] - fn absent_subsystems_block_deserializes_to_default() { - let cfg: SubsystemsConfig = toml::from_str("").expect("empty toml parses"); - assert_eq!( - serde_json::to_value(&cfg).unwrap(), - serde_json::to_value(SubsystemsConfig::default()).unwrap() - ); - } - - #[test] - fn memory_driver_config_debug_never_leaks_credential_ref() { - let driver = MemoryDriverConfig { - class: Some("external".into()), - transport: Some("http".into()), - endpoint: Some("https://api.supermemory.ai".into()), - credential_ref: Some("keychain:supermemory-super-secret-value".into()), - trust_state: "untrusted".into(), - }; - let debug_output = format!("{driver:?}"); - assert!( - !debug_output.contains("keychain:supermemory-super-secret-value"), - "Debug output must never contain the credential_ref value: {debug_output}" - ); - assert!( - debug_output.contains(""), - "Debug output should show a redaction marker: {debug_output}" - ); - } - - #[test] - fn memory_driver_config_default_trust_state_is_untrusted() { - assert_eq!(MemoryDriverConfig::default().trust_state, "untrusted"); - } -} diff --git a/src/openhuman/memory/api/host/test_support.rs b/src/openhuman/memory/api/host/test_support.rs deleted file mode 100644 index 63038141fc..0000000000 --- a/src/openhuman/memory/api/host/test_support.rs +++ /dev/null @@ -1,209 +0,0 @@ -//! [`TestHostConfig`] — a concrete, `Default`-able [`MemoryHostConfig`] for -//! tests. -//! -//! `crate::openhuman::memory::core_impl::Config` is `dyn MemoryHostConfig`, which cannot be -//! `Default::default()`ed. The extracted test suites build a config, tweak two -//! or three fields, and pass `&config` into the code under test — a pattern -//! that needs a real struct. This is that struct. -//! -//! It is behind the `test-support` feature and enabled from -//! `tinymemory-core`'s dev-dependencies, so it never enters a shipped build. -//! It is deliberately *not* a mock: the fields are the real config sections -//! with their real serde defaults, so a test that asserts on default behaviour -//! is asserting on the same values production loads. - -use std::path::PathBuf; - -use super::cloud_providers::CloudProviderCreds; -use super::config::{ComposioMode, MemoryHostConfig}; -use super::local_ai::LocalAiConfig; -use super::scheduler_gate::SchedulerGateConfig; -use super::storage_memory::{MemoryConfig, MemoryTreeConfig}; - -/// A concrete host config for tests. Fields are public — mutate them directly -/// rather than reaching for a builder. -#[derive(Debug, Clone, Default)] -#[non_exhaustive] -pub struct TestHostConfig { - /// See [`MemoryHostConfig::workspace_dir`]. - pub workspace_dir: PathBuf, - /// See [`MemoryHostConfig::config_path`]. - pub config_path: PathBuf, - /// See [`MemoryHostConfig::memory`]. - pub memory: MemoryConfig, - /// See [`MemoryHostConfig::session_token`]. `None` is signed-out. - pub session_token: Option, - /// See [`MemoryHostConfig::memory_tree`]. - pub memory_tree: MemoryTreeConfig, - /// See [`MemoryHostConfig::scheduler_gate`]. - pub scheduler_gate: SchedulerGateConfig, - /// See [`MemoryHostConfig::local_ai`]. - pub local_ai: LocalAiConfig, - /// See [`MemoryHostConfig::cloud_providers`]. - pub cloud_providers: Vec, - /// See [`MemoryHostConfig::embeddings_provider`]. - pub embeddings_provider: Option, - /// See [`MemoryHostConfig::memory_provider`]. - pub memory_provider: Option, - /// See [`MemoryHostConfig::api_url`]. - pub api_url: Option, - /// See [`MemoryHostConfig::default_model`]. - pub default_model: Option, - /// See [`MemoryHostConfig::default_temperature`]. - pub default_temperature: f64, - /// See [`MemoryHostConfig::output_language`]. - pub output_language: Option, - /// See [`MemoryHostConfig::memory_sync_interval_secs`]. - pub memory_sync_interval_secs: Option, - /// See [`MemoryHostConfig::onboarding_completed`]. - pub onboarding_completed: bool, - /// See [`MemoryHostConfig::secrets_encrypt`]. - pub secrets_encrypt: bool, - /// See [`MemoryHostConfig::composio`]. - pub composio: ComposioMode, - /// See [`MemoryHostConfig::memory_sources_json`]. Defaults to an empty - /// array so a test that never touches sources behaves like a fresh install. - pub memory_sources: Option, - /// See [`MemoryHostConfig::composio_source_caps_migration_version`]. - pub composio_source_caps_migration_version: u32, -} - -#[async_trait::async_trait] -impl MemoryHostConfig for TestHostConfig { - fn workspace_dir(&self) -> &PathBuf { - &self.workspace_dir - } - - fn config_path(&self) -> &PathBuf { - &self.config_path - } - - fn memory_tree_content_root(&self) -> PathBuf { - self.memory_tree - .content_dir - .clone() - .unwrap_or_else(|| self.workspace_dir.join("memory_tree").join("content")) - } - - fn memory(&self) -> &MemoryConfig { - &self.memory - } - - fn memory_tree(&self) -> &MemoryTreeConfig { - &self.memory_tree - } - - fn scheduler_gate(&self) -> &SchedulerGateConfig { - &self.scheduler_gate - } - - fn local_ai(&self) -> &LocalAiConfig { - &self.local_ai - } - - fn cloud_providers(&self) -> &Vec { - &self.cloud_providers - } - - fn embeddings_provider(&self) -> Option<&str> { - self.embeddings_provider.as_deref() - } - - fn memory_provider(&self) -> Option<&str> { - self.memory_provider.as_deref() - } - - fn workload_local_model(&self, workload: &str) -> Option { - let raw = match workload { - "memory" => self.memory_provider.as_deref(), - "embeddings" => self.embeddings_provider.as_deref(), - _ => None, - }?; - let model = raw.trim().strip_prefix("ollama:")?.trim(); - if model.is_empty() { - None - } else { - Some(model.to_string()) - } - } - - fn as_any(&self) -> &dyn std::any::Any { - self - } - - fn to_arc(&self) -> std::sync::Arc { - std::sync::Arc::new(self.clone()) - } - - fn api_url(&self) -> Option<&str> { - self.api_url.as_deref() - } - - fn effective_backend_api_url(&self) -> String { - // No resolution to do: a test config states its backend URL outright, - // and the host's env/default ladder is not something to reimplement - // here. - self.api_url.clone().unwrap_or_default() - } - - fn session_token(&self) -> Result, String> { - // `Ok(None)` — "read fine, not signed in" — rather than `Err`, so a - // test that never sets a token exercises the signed-out path instead of - // a credential-store failure. - Ok(self.session_token.clone()) - } - - fn default_model(&self) -> Option<&str> { - self.default_model.as_deref() - } - - fn default_temperature(&self) -> f64 { - self.default_temperature - } - - fn output_language(&self) -> Option<&str> { - self.output_language.as_deref() - } - - fn memory_sync_interval_secs(&self) -> Option { - self.memory_sync_interval_secs - } - - fn onboarding_completed(&self) -> bool { - self.onboarding_completed - } - - fn secrets_encrypt(&self) -> bool { - self.secrets_encrypt - } - - fn composio(&self) -> ComposioMode { - self.composio.clone() - } - - fn memory_sources_json(&self) -> anyhow::Result { - Ok(self - .memory_sources - .clone() - .unwrap_or_else(|| serde_json::Value::Array(Vec::new()))) - } - - fn set_memory_sources_json(&mut self, value: serde_json::Value) -> anyhow::Result<()> { - self.memory_sources = Some(value); - Ok(()) - } - - fn composio_source_caps_migration_version(&self) -> u32 { - self.composio_source_caps_migration_version - } - - fn set_composio_source_caps_migration_version(&mut self, version: u32) { - self.composio_source_caps_migration_version = version; - } - - fn apply_env_overrides(&mut self) {} - - async fn save(&self) -> anyhow::Result<()> { - Ok(()) - } -} diff --git a/src/openhuman/memory/api/host/usage.rs b/src/openhuman/memory/api/host/usage.rs deleted file mode 100644 index 11d9c45627..0000000000 --- a/src/openhuman/memory/api/host/usage.rs +++ /dev/null @@ -1,33 +0,0 @@ -//! [`UsageInfo`] — token accounting returned by an inference provider. -//! -//! Lives in the contract crate because both sides name it: the host's chat -//! providers produce it, and the memory subsystem's summariser threads it back -//! out so callers can attribute cost to a summarisation run. It is inert data -//! with no dependencies, so it costs the contract crate nothing. - -/// Token usage information returned by the provider after an inference call. -#[derive(Debug, Clone, Default)] -pub struct UsageInfo { - /// Number of tokens in the input/prompt. - pub input_tokens: u64, - /// Number of tokens in the output/completion. - pub output_tokens: u64, - /// Total context window size for the model (0 if unknown). - pub context_window: u64, - /// Number of input tokens that were served from the KV cache - /// (returned by backends that support prompt caching, e.g. via - /// `openhuman.usage.cached_input_tokens` or - /// `prompt_tokens_details.cached_tokens`). - pub cached_input_tokens: u64, - /// Number of input tokens written into a provider prompt/KV cache on this - /// request (cache-creation / cache-write tokens). Distinct from - /// `cached_input_tokens` (cache reads). Zero when the provider does not - /// report a cache-write breakdown. - pub cache_creation_tokens: u64, - /// Number of reasoning/thinking output tokens when the provider exposes - /// them separately from `output_tokens`. Zero when unavailable. - pub reasoning_tokens: u64, - /// Amount billed for this request in USD (from - /// `openhuman.billing.charged_amount_usd`). Zero when unavailable. - pub charged_amount_usd: f64, -} diff --git a/src/openhuman/memory/api/mod.rs b/src/openhuman/memory/api/mod.rs deleted file mode 100644 index 05061a10c5..0000000000 --- a/src/openhuman/memory/api/mod.rs +++ /dev/null @@ -1,83 +0,0 @@ -//! Stable public contracts for the TinyMemory memory system. -//! -//! This crate holds the value types, error enum, capability vocabulary, and -//! storage trait that memory engines and their embedding hosts compile -//! against. It is engine-neutral on purpose: `tinycortex` is the default -//! embedded engine, not the owner of the contract, and a second engine -//! (`supermemory`, `mem0`, a self-hosted HTTP backend) implements the same -//! traits without either engine learning about the other. -//! It is deliberately dependency-light (serde / serde_json / -//! chrono / sha2 / anyhow / thiserror / async-trait / uuid only) so depending on -//! the contract never drags in SQLite, git2, reqwest, regex, or an async -//! runtime. -//! -//! ## Self-contained by design -//! -//! Nothing here names a host type. A third-party memory driver must be able to -//! depend on this crate alone, and the *generic* subsystem/driver vocabulary of -//! the OpenHuman kernel (`Driver`, `DriverClass`, `SubsystemRegistry`, the -//! policy `Guard`) must not be inherited from a *memory* crate by whichever -//! subsystem is cut over next. So the contract carries its own identity, -//! capability, and health vocabulary, and the host's memory adapter converts at -//! the boundary — see [`health`] for the shape that conversion relies on. -//! -//! Driver *class* (embedded / external / null) is deliberately **absent**: that -//! is a host configuration fact about how a driver was bound, not something a -//! driver reports about itself. -//! -//! ## The TinyCortex engine's historical paths still resolve -//! -//! This contract used to live in the TinyCortex repository as `tinycortex-api`. -//! That crate is now a deprecated re-export of this one, and the engine crate -//! aliases these modules back into their historical paths -//! (`crate::openhuman::memory::engine::{types, error, traits}`, -//! `crate::openhuman::memory::engine::chunks::types`, `crate::openhuman::memory::engine::tree::runtime::types`, -//! `crate::openhuman::memory::engine::tool_memory::types`, `crate::openhuman::memory::engine::goals::types`), -//! so every existing path keeps resolving unchanged. -//! -//! ## Module map -//! -//! - [`types`]: pure data contracts (entries, hits, taint, namespaces). -//! - [`recall`]: the borrowed [`recall::RecallOpts`] and owned, serde-derived -//! [`recall::OwnedRecallOpts`] recall filters (both re-exported from -//! [`types`]). -//! - [`capabilities`]: the thirteen [`capabilities::Capability`] families and -//! the [`capabilities::Capabilities`] set negotiated at bind time. -//! - [`provider`]: the driver contract — [`provider::MemoryProvider`] plus the -//! thirteen capability family traits and the value types they need. -//! - [`null`]: [`null::NullMemoryProvider`], the reference driver a -//! compiled-out or unconfigured memory subsystem binds to. -//! - [`health`]: [`health::MemoryHealth`], the liveness state a driver reports. -//! - [`version`]: [`CONTRACT_VERSION`] and the [`is_compatible`] bind rule. -//! - [`error`]: the typed [`error::MemoryError`] enum and its result alias. -//! - [`traits`]: the [`traits::Memory`] storage-backend trait. -//! - [`chunks`]: the persisted chunk model ([`chunks::Chunk`], [`chunks::Metadata`], -//! [`chunks::SourceRef`], …) and the deterministic [`chunks::chunk_id`]. -//! - [`tree`]: the markdown summary-tree node model ([`tree::TreeNode`], -//! [`tree::NodeLevel`], [`tree::TreeStatus`], …). -//! - [`tool_memory`]: tool-scoped rule contracts ([`tool_memory::ToolMemoryRule`], …). -//! - [`goals`]: the long-term goals document ([`goals::GoalsDoc`], [`goals::GoalItem`]). -//! - [`host`]: the **host seam** — [`host::MemoryHostConfig`], -//! [`host::EmbeddingProvider`], [`host::MemoryEventSink`], and the memory -//! config sections whose serde form is persisted in a host's `config.toml`. -//! - [`wire`]: the error-name table a driver reached over a bus or a socket -//! round-trips [`error::MemoryError`] through. Shared by both ends of every -//! such transport, so the names cannot drift apart. - -pub mod capabilities; -pub mod chunks; -pub mod error; -pub mod goals; -pub mod health; -pub mod host; -pub mod null; -pub mod provider; -pub mod recall; -pub mod tool_memory; -pub mod traits; -pub mod tree; -pub mod types; -pub mod version; -pub mod wire; - -pub use version::{is_compatible, CONTRACT_VERSION}; diff --git a/src/openhuman/memory/api/null.rs b/src/openhuman/memory/api/null.rs deleted file mode 100644 index adc020f57b..0000000000 --- a/src/openhuman/memory/api/null.rs +++ /dev/null @@ -1,480 +0,0 @@ -//! [`NullMemoryProvider`] — the reference driver that stores nothing. -//! -//! ## What it is for -//! -//! A memory subsystem that is compiled out, disabled by configuration, or -//! explicitly bound to `driver = "null"` still has to bind *something*: the -//! kernel's registry holds exactly one driver per slot, and code that reaches -//! the slot must find a value rather than an `Option` it has to unwrap at every -//! call site. This is that value. It replaces the hand-written per-domain -//! `stub.rs` files with one generic answer. -//! -//! It is also the fixture the capability-degradation tests bind: with it in the -//! slot, the ten optional families are unadvertised, so their RPC methods are -//! unregistered and their agent tools are absent — and the core still boots. -//! -//! And it is the existence proof for the mandatory set: if -//! [`crate::openhuman::memory::api::provider::MemoryCore`], [`crate::openhuman::memory::api::provider::MemoryRecall`], and -//! [`crate::openhuman::memory::api::provider::MemoryPortability`] could not be implemented without a -//! storage engine, they would be the wrong three to have made mandatory. -//! -//! ## `/dev/null` semantics, and what that costs -//! -//! Writes are **accepted and discarded**; reads return empty. This mirrors the -//! Unix device the driver is named after, and it is the only behaviour that -//! lets the mandatory three be advertised honestly: a `store` that returned -//! [`crate::openhuman::memory::api::error::MemoryError::Unsupported`] would contradict advertising -//! [`crate::openhuman::memory::api::capabilities::Capability::Core`], and one that returned a hard -//! error would turn every optional auto-capture into a user-visible failure. -//! -//! The cost is real: content written here is gone. That is acceptable for a -//! subsystem the operator turned off, and unacceptable as a fallback for a -//! driver that failed to bind — **that** case falls back to the embedded -//! default, never to this. Do not wire it as a general-purpose failure mode. -//! -//! ## Why it implements all thirteen families but advertises three -//! -//! The ten optional families are implemented and every method returns -//! [`crate::openhuman::memory::api::error::MemoryError::Unsupported`] naming its family, but the -//! `as_*` accessors return `None` and -//! [`crate::openhuman::memory::api::provider::MemoryProvider::capabilities`] lists only the mandatory -//! three. So: -//! -//! - through `&dyn MemoryProvider` — the only way product code sees a driver — -//! an unadvertised family is simply **unreachable**, which is the intended -//! degradation; -//! - through the concrete type, a direct call yields a typed, *named* -//! `Unsupported` error, which is what makes the contract's error mapping -//! testable without writing a second mock. -//! -//! [`crate::openhuman::memory::api::provider::audit_provider`] confirms the two views agree. - -use async_trait::async_trait; - -use crate::openhuman::memory::api::capabilities::{Capabilities, Capability}; -use crate::openhuman::memory::api::error::MemoryError; -use crate::openhuman::memory::api::goals::GoalsDoc; -use crate::openhuman::memory::api::health::MemoryHealth; -use crate::openhuman::memory::api::provider::types::{ - DiffReport, EntityHit, ExportPage, ExportRecord, ImportOutcome, IngestItem, IngestOutcome, - MaintenanceReport, SnapshotRef, SourceItem, SourceScope, -}; -use crate::openhuman::memory::api::provider::{ - MemoryCore, MemoryDiff, MemoryDocuments, MemoryEntities, MemoryGoals, MemoryGraph, - MemoryIngest, MemoryMaintenance, MemoryPortability, MemoryProvider, MemoryRecall, - MemorySourceSink, MemoryToolMemory, MemoryTree, -}; -use crate::openhuman::memory::api::recall::OwnedRecallOpts; -use crate::openhuman::memory::api::tool_memory::ToolMemoryRule; -use crate::openhuman::memory::api::tree::{IngestRequest, QueryResult, TreeStatus}; -use crate::openhuman::memory::api::types::{ - GraphRelationRecord, MemoryCategory, MemoryEntry, MemoryKvRecord, MemoryTaint, - NamespaceDocumentInput, NamespaceRetrievalContext, NamespaceSummary, StoredMemoryDocument, -}; - -/// The [`driver_id`](MemoryProvider::driver_id) this driver reports. -pub const NULL_DRIVER_ID: &str = "null"; - -/// Shorthand for the `Unsupported` error every unadvertised family returns. -fn unsupported(capability: Capability) -> Result { - Err(MemoryError::unsupported(capability)) -} - -/// A driver that accepts every write, discards it, and returns nothing. -/// -/// See the module documentation for what it is for, why writes are silently -/// dropped, and why it implements ten families it does not advertise. -#[derive(Debug, Clone, Copy, Default)] -pub struct NullMemoryProvider; - -impl NullMemoryProvider { - /// Construct the null driver. It holds no state, so every instance is - /// interchangeable. - pub const fn new() -> Self { - Self - } -} - -#[async_trait] -impl MemoryProvider for NullMemoryProvider { - fn driver_id(&self) -> &str { - NULL_DRIVER_ID - } - - /// Exactly the mandatory three. The ten optional families are implemented - /// below but deliberately not advertised, so they stay unreachable through - /// the trait object. - fn capabilities(&self) -> Capabilities { - Capabilities::mandatory() - } - - /// Always [`MemoryHealth::Ready`]: a driver with no backing store has - /// nothing that can be unreachable, and reporting `Degraded` would make - /// every status view of a deliberately-disabled subsystem look broken. - async fn health(&self) -> MemoryHealth { - MemoryHealth::Ready - } - - // The `as_*` accessors are all left at their `None` defaults: nothing - // optional is reachable through the trait object. That absence is the whole - // point of this driver, so overriding any of them would be the bug. -} - -#[async_trait] -impl MemoryCore for NullMemoryProvider { - /// Accepts and discards. See the module docs on `/dev/null` semantics. - async fn store( - &self, - _namespace: &str, - _key: &str, - _content: &str, - _category: MemoryCategory, - _session_id: Option<&str>, - _taint: MemoryTaint, - ) -> Result<(), MemoryError> { - Ok(()) - } - - async fn get(&self, _namespace: &str, _key: &str) -> Result, MemoryError> { - Ok(None) - } - - /// Always `Ok(false)`: nothing was ever stored, so nothing existed to - /// forget. Consistent with the idempotence the family requires. - async fn forget(&self, _namespace: &str, _key: &str) -> Result { - Ok(false) - } - - async fn list( - &self, - _namespace: Option<&str>, - _category: Option<&MemoryCategory>, - _session_id: Option<&str>, - ) -> Result, MemoryError> { - Ok(Vec::new()) - } - - async fn namespaces(&self) -> Result, MemoryError> { - Ok(Vec::new()) - } -} - -#[async_trait] -impl MemoryRecall for NullMemoryProvider { - async fn recall( - &self, - _query: &str, - _limit: usize, - _opts: &OwnedRecallOpts, - _scope: Option<&SourceScope>, - ) -> Result, MemoryError> { - Ok(Vec::new()) - } -} - -#[async_trait] -impl MemoryPortability for NullMemoryProvider { - /// One empty, terminal page: no records and no continuation cursor, so a - /// caller's export loop terminates on the first iteration. - /// - /// This driver never issues a cursor (every page is the first and only - /// page), so any `Some(_)` cursor a caller passes back is necessarily one - /// this driver did not hand out — reject it rather than silently treating - /// it as a valid terminal page. - async fn export_page( - &self, - cursor: Option<&str>, - _limit: usize, - ) -> Result { - if cursor.is_some() { - return Err(MemoryError::Invalid( - "null provider does not issue export cursors".into(), - )); - } - - Ok(ExportPage::default()) - } - - /// Counts every record as skipped rather than imported. Reporting them as - /// imported would tell a migration its data landed somewhere it did not. - async fn import_records( - &self, - records: Vec, - ) -> Result { - Ok(ImportOutcome { - imported: 0, - skipped: u32::try_from(records.len()).unwrap_or(u32::MAX), - failed: 0, - errors: Vec::new(), - }) - } -} - -#[async_trait] -impl MemoryIngest for NullMemoryProvider { - async fn ingest_document(&self, _item: IngestItem) -> Result { - unsupported(Capability::Ingest) - } - - async fn ingest_chat(&self, _messages: Vec) -> Result { - unsupported(Capability::Ingest) - } -} - -#[async_trait] -impl MemoryDocuments for NullMemoryProvider { - async fn put_document(&self, _input: NamespaceDocumentInput) -> Result { - unsupported(Capability::Documents) - } - - async fn get_document( - &self, - _namespace: &str, - _key: &str, - ) -> Result, MemoryError> { - unsupported(Capability::Documents) - } - - async fn list_documents( - &self, - _namespace: Option<&str>, - ) -> Result { - unsupported(Capability::Documents) - } - - async fn list_namespaces(&self) -> Result, MemoryError> { - unsupported(Capability::Documents) - } - - async fn delete_document( - &self, - _namespace: &str, - _document_id: &str, - ) -> Result { - unsupported(Capability::Documents) - } - - async fn clear_namespace(&self, _namespace: &str) -> Result<(), MemoryError> { - unsupported(Capability::Documents) - } - - async fn query_documents( - &self, - _namespace: &str, - _query: &str, - _limit: usize, - ) -> Result { - unsupported(Capability::Documents) - } - - async fn recall_documents( - &self, - _namespace: &str, - _limit: usize, - ) -> Result { - unsupported(Capability::Documents) - } -} - -#[async_trait] -impl MemoryTree for NullMemoryProvider { - async fn append(&self, _request: IngestRequest) -> Result<(), MemoryError> { - unsupported(Capability::Tree) - } - - async fn query_source( - &self, - _namespace: &str, - _source_id: &str, - _limit: usize, - _scope: Option<&SourceScope>, - ) -> Result, MemoryError> { - unsupported(Capability::Tree) - } - - async fn drill_down( - &self, - _namespace: &str, - _node_id: &str, - ) -> Result { - unsupported(Capability::Tree) - } - - async fn seal(&self, _namespace: &str) -> Result { - unsupported(Capability::Tree) - } - - async fn cascade(&self, _namespace: &str) -> Result { - unsupported(Capability::Tree) - } -} - -#[async_trait] -impl MemoryEntities for NullMemoryProvider { - async fn entities( - &self, - _namespace: &str, - _query: Option<&str>, - _limit: usize, - ) -> Result, MemoryError> { - unsupported(Capability::Entities) - } - - async fn entity_edges( - &self, - _namespace: &str, - _entity_id: &str, - _limit: usize, - ) -> Result, MemoryError> { - unsupported(Capability::Entities) - } - - async fn touch_entities( - &self, - _namespace: &str, - _entity_ids: &[String], - ) -> Result<(), MemoryError> { - unsupported(Capability::Entities) - } -} - -#[async_trait] -impl MemoryGraph for NullMemoryProvider { - async fn kv_get( - &self, - _namespace: Option<&str>, - _key: &str, - ) -> Result, MemoryError> { - unsupported(Capability::Graph) - } - - async fn kv_put( - &self, - _namespace: Option<&str>, - _key: &str, - _value: serde_json::Value, - ) -> Result<(), MemoryError> { - unsupported(Capability::Graph) - } - - async fn kv_delete(&self, _namespace: Option<&str>, _key: &str) -> Result { - unsupported(Capability::Graph) - } - - async fn kv_list( - &self, - _namespace: Option<&str>, - _prefix: Option<&str>, - _limit: usize, - ) -> Result, MemoryError> { - unsupported(Capability::Graph) - } - - async fn relations( - &self, - _namespace: Option<&str>, - _subject: Option<&str>, - _predicate: Option<&str>, - _limit: usize, - ) -> Result, MemoryError> { - unsupported(Capability::Graph) - } - - async fn put_relation(&self, _relation: GraphRelationRecord) -> Result<(), MemoryError> { - unsupported(Capability::Graph) - } -} - -#[async_trait] -impl MemoryDiff for NullMemoryProvider { - async fn capture_snapshot(&self, _source_id: &str) -> Result { - unsupported(Capability::Diff) - } - - async fn snapshots( - &self, - _source_id: &str, - _limit: usize, - ) -> Result, MemoryError> { - unsupported(Capability::Diff) - } - - async fn diff( - &self, - _source_id: &str, - _from: Option<&str>, - _to: &str, - ) -> Result { - unsupported(Capability::Diff) - } -} - -#[async_trait] -impl MemoryGoals for NullMemoryProvider { - async fn goals(&self) -> Result { - unsupported(Capability::Goals) - } - - async fn set_goals(&self, _goals: GoalsDoc) -> Result<(), MemoryError> { - unsupported(Capability::Goals) - } -} - -#[async_trait] -impl MemoryToolMemory for NullMemoryProvider { - async fn tool_rules(&self, _tool_name: &str) -> Result, MemoryError> { - unsupported(Capability::ToolMemory) - } - - async fn put_tool_rule(&self, _rule: ToolMemoryRule) -> Result<(), MemoryError> { - unsupported(Capability::ToolMemory) - } - - async fn delete_tool_rule( - &self, - _tool_name: &str, - _rule_id: &str, - ) -> Result { - unsupported(Capability::ToolMemory) - } -} - -#[async_trait] -impl MemorySourceSink for NullMemoryProvider { - async fn accept_source_items( - &self, - _source_id: &str, - _source_kind: &str, - _items: Vec, - _taint: MemoryTaint, - ) -> Result { - unsupported(Capability::Sources) - } - - async fn forget_source(&self, _source_id: &str) -> Result { - unsupported(Capability::Sources) - } -} - -#[async_trait] -impl MemoryMaintenance for NullMemoryProvider { - async fn reembed(&self) -> Result { - unsupported(Capability::Maintenance) - } - - async fn compact(&self) -> Result { - unsupported(Capability::Maintenance) - } - - async fn consolidate(&self) -> Result { - unsupported(Capability::Maintenance) - } - - async fn doctor(&self) -> Result { - unsupported(Capability::Maintenance) - } -} - -#[cfg(test)] -#[path = "null_tests.rs"] -mod tests; diff --git a/src/openhuman/memory/api/null_tests.rs b/src/openhuman/memory/api/null_tests.rs deleted file mode 100644 index bd65e8a27c..0000000000 --- a/src/openhuman/memory/api/null_tests.rs +++ /dev/null @@ -1,242 +0,0 @@ -//! Tests for the reference null driver. -//! -//! These pin three separate contracts: -//! -//! 1. the mandatory-three set is genuinely implementable without a store; -//! 2. an unadvertised family is **unreachable** through the trait object, which -//! is the degradation behaviour the kernel relies on; -//! 3. a direct call to an unadvertised family yields a typed `Unsupported` -//! error that **names** the family, which is what the transport adapter's -//! `501` mapping is checked against. -//! -//! ## No async runtime here, on purpose -//! -//! `tinymemory-api` must not depend on tokio (or any executor) — that is the -//! whole point of the crate. Every future in this module completes on its first -//! poll, so a six-line std-only [`block_on`] is sufficient and adds no -//! dependency. - -use std::future::Future; -use std::pin::pin; -use std::task::{Context, Poll}; - -use super::*; -use crate::openhuman::memory::api::provider::audit_provider; -use crate::openhuman::memory::api::types::MemoryCategory; - -/// Drive a future that is ready on first poll to completion, without an -/// executor. Panics rather than spinning if a future ever returns `Pending`, -/// because in this module that would mean a supposedly-inert implementation -/// started doing real work. -fn block_on(future: F) -> F::Output { - let mut future = pin!(future); - let mut context = Context::from_waker(std::task::Waker::noop()); - match future.as_mut().poll(&mut context) { - Poll::Ready(value) => value, - Poll::Pending => panic!("null driver future must complete on first poll"), - } -} - -#[test] -fn null_driver_advertises_exactly_the_mandatory_families() { - let driver = NullMemoryProvider::new(); - let capabilities = driver.capabilities(); - - assert_eq!(driver.driver_id(), NULL_DRIVER_ID); - assert_eq!(capabilities.len(), 3); - for capability in Capability::MANDATORY { - assert!( - capabilities.contains(capability), - "{capability} must be advertised" - ); - } -} - -#[test] -fn null_driver_passes_capability_validation() { - // The mandatory-three set is the minimum bindable set, so the reference - // driver must be bindable. If this ever fails, either the mandatory list - // grew or the null driver stopped implementing it. - let driver = NullMemoryProvider::new(); - assert_eq!(driver.capabilities().validate(), Ok(())); -} - -#[test] -fn null_driver_is_self_consistent() { - assert_eq!(audit_provider(&NullMemoryProvider::new()), Ok(())); -} - -#[test] -fn null_driver_reports_ready() { - let health = block_on(NullMemoryProvider::new().health()); - assert_eq!(health, MemoryHealth::Ready); - assert!(health.is_usable()); -} - -#[test] -fn null_driver_shutdown_is_an_idempotent_no_op() { - let driver = NullMemoryProvider::new(); - assert!(block_on(driver.shutdown()).is_ok()); - assert!(block_on(driver.shutdown()).is_ok()); -} - -#[test] -fn mandatory_core_accepts_writes_and_reads_back_empty() { - let driver = NullMemoryProvider::new(); - - block_on(driver.store( - "global", - "k", - "v", - MemoryCategory::Core, - None, - MemoryTaint::ExternalSync, - )) - .expect("null store must accept the write"); - - assert!(block_on(driver.get("global", "k")) - .expect("get must succeed") - .is_none()); - assert!(!block_on(driver.forget("global", "k")).expect("forget must succeed")); - assert!(block_on(driver.list(None, None, None)) - .expect("list must succeed") - .is_empty()); - assert!(block_on(driver.namespaces()) - .expect("namespaces must succeed") - .is_empty()); -} - -#[test] -fn mandatory_recall_returns_no_hits() { - let driver = NullMemoryProvider::new(); - let hits = block_on(driver.recall("anything", 10, &OwnedRecallOpts::default(), None)) - .expect("recall must succeed"); - assert!(hits.is_empty()); -} - -#[test] -fn mandatory_portability_round_trips_as_an_empty_store() { - let driver = NullMemoryProvider::new(); - - let page = block_on(driver.export_page(None, 100)).expect("export must succeed"); - assert!(page.records.is_empty()); - assert!( - page.next_cursor.is_none(), - "the absent cursor is what terminates the caller's export loop" - ); - - let outcome = block_on(driver.import_records(vec![ExportRecord { - kind: "entry".to_string(), - id: "rec-1".to_string(), - namespace: None, - taint: MemoryTaint::Internal, - payload: serde_json::Value::Null, - }])) - .expect("import must succeed"); - - // Skipped, never imported: reporting an import would tell a migration its - // data landed somewhere it did not. - assert_eq!(outcome.imported, 0); - assert_eq!(outcome.skipped, 1); - assert_eq!(outcome.failed, 0); -} - -#[test] -fn export_page_rejects_a_cursor_it_never_issued() { - let driver = NullMemoryProvider::new(); - - let err = block_on(driver.export_page(Some("unexpected"), 100)) - .expect_err("a cursor this driver never issued must be rejected, not silently accepted"); - assert!( - matches!(err, MemoryError::Invalid(_)), - "expected MemoryError::Invalid, got {err:?}" - ); -} - -#[test] -fn every_unadvertised_family_is_unreachable_through_the_trait_object() { - let driver = NullMemoryProvider::new(); - let provider: &dyn MemoryProvider = &driver; - - assert!(provider.as_ingest().is_none()); - assert!(provider.as_documents().is_none()); - assert!(provider.as_tree().is_none()); - assert!(provider.as_entities().is_none()); - assert!(provider.as_graph().is_none()); - assert!(provider.as_diff().is_none()); - assert!(provider.as_goals().is_none()); - assert!(provider.as_tool_memory().is_none()); - assert!(provider.as_sources().is_none()); - assert!(provider.as_maintenance().is_none()); -} - -#[test] -fn advertised_and_reachable_agree_for_every_family() { - // The invariant that keeps the capability set honest, checked family by - // family rather than only through the aggregate audit. - let driver = NullMemoryProvider::new(); - let provider: &dyn MemoryProvider = &driver; - let advertised = provider.capabilities(); - - for capability in Capability::ALL { - assert_eq!( - advertised.contains(capability), - provider.provides(capability), - "{capability}: advertised and reachable must agree" - ); - } -} - -/// Assert a result is `Unsupported` and names the expected family. -fn assert_unsupported(result: Result, expected: Capability) { - match result { - Err(MemoryError::Unsupported { capability }) => { - assert_eq!(capability, expected.as_str()); - } - other => panic!("expected Unsupported({expected}), got {other:?}"), - } -} - -#[test] -fn unadvertised_families_return_unsupported_naming_their_capability() { - let driver = NullMemoryProvider::new(); - - assert_unsupported(block_on(driver.ingest_chat(Vec::new())), Capability::Ingest); - assert_unsupported( - block_on(driver.get_document("global", "k")), - Capability::Documents, - ); - assert_unsupported(block_on(driver.seal("global")), Capability::Tree); - assert_unsupported( - block_on(driver.entities("global", None, 10)), - Capability::Entities, - ); - assert_unsupported(block_on(driver.kv_get(None, "k")), Capability::Graph); - assert_unsupported( - block_on(driver.capture_snapshot("src-abc")), - Capability::Diff, - ); - assert_unsupported(block_on(driver.goals()), Capability::Goals); - assert_unsupported(block_on(driver.tool_rules("shell")), Capability::ToolMemory); - assert_unsupported( - block_on(driver.forget_source("src-abc")), - Capability::Sources, - ); - assert_unsupported(block_on(driver.doctor()), Capability::Maintenance); -} - -#[test] -fn provider_is_usable_as_a_shared_trait_object() { - // The registry binds `Arc`, so the trait object must be - // `Send + Sync` and every family trait must be object-safe. This test fails - // to *compile* rather than to run if that ever regresses. - fn assert_send_sync(_value: &T) {} - - let provider: std::sync::Arc = - std::sync::Arc::new(NullMemoryProvider::new()); - assert_send_sync(&provider); - assert_eq!(provider.driver_id(), NULL_DRIVER_ID); - assert!(block_on(provider.list(None, None, None)) - .expect("list through the trait object") - .is_empty()); -} diff --git a/src/openhuman/memory/api/provider/audit.rs b/src/openhuman/memory/api/provider/audit.rs deleted file mode 100644 index 125e2f4d27..0000000000 --- a/src/openhuman/memory/api/provider/audit.rs +++ /dev/null @@ -1,132 +0,0 @@ -//! The honesty check: does a driver's advertised capability set match the -//! surface it actually exposes? -//! -//! [`MemoryProvider::capabilities`] is a *claim*, and the kernel acts on it — -//! it registers RPC methods and assembles agent tools from the advertised set -//! and never re-checks. A driver that advertises a family it does not implement -//! therefore produces a surface that exists in `/schema`, appears in the agent's -//! tool list, and fails on first use. That is precisely the -//! "registered-but-failing" outcome the degradation design exists to avoid. -//! -//! [`audit_provider`] compares the claim against -//! [`MemoryProvider::provides`] — which is derived from the accessors, so it -//! cannot drift from reality — and reports both directions of mismatch. Run it -//! at bind time next to [`crate::openhuman::memory::api::capabilities::Capabilities::validate`], and in -//! every driver's own test suite. -//! -//! The two directions mean different things: -//! -//! - **Advertised but absent** is a bug that will surface as a failing call. It -//! should refuse the bind. -//! - **Present but unadvertised** is dead surface: the family works but the -//! kernel unregistered it, so nothing can reach it. Usually a forgotten -//! entry in the driver's `capabilities()` list. - -use std::fmt; - -use crate::openhuman::memory::api::capabilities::Capability; -use crate::openhuman::memory::api::error::MemoryError; -use crate::openhuman::memory::api::provider::driver::MemoryProvider; - -/// A disagreement between what a driver advertises and what it implements. -/// -/// Carries the families structurally rather than as a formatted string so a -/// caller can report them in a status payload or a bind-failure event as well -/// as in a log line. At least one of the two vectors is non-empty. -#[derive(Debug, Clone, PartialEq, Eq)] -pub struct CapabilityAudit { - /// Families the driver advertises but does not expose. These will fail on - /// first call; refuse the bind. - pub advertised_but_absent: Vec, - /// Families the driver exposes but does not advertise. These are - /// unreachable, because the kernel filters from the advertised set. - pub present_but_unadvertised: Vec, -} - -impl fmt::Display for CapabilityAudit { - fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { - let mut parts = Vec::new(); - if !self.advertised_but_absent.is_empty() { - parts.push(format!( - "advertised but not implemented: {}", - join(&self.advertised_but_absent) - )); - } - if !self.present_but_unadvertised.is_empty() { - parts.push(format!( - "implemented but not advertised: {}", - join(&self.present_but_unadvertised) - )); - } - write!(f, "memory driver capability mismatch; {}", parts.join("; ")) - } -} - -impl std::error::Error for CapabilityAudit {} - -impl From for MemoryError { - /// A mismatch is the driver saying something untrue about itself, which is - /// a configuration/implementation error rather than an unsupported call — - /// hence [`MemoryError::Invalid`] and not - /// [`MemoryError::Unsupported`]. Same reasoning as - /// [`crate::openhuman::memory::api::capabilities::MissingMandatoryCapabilities`]. - fn from(value: CapabilityAudit) -> Self { - MemoryError::Invalid(value.to_string()) - } -} - -fn join(families: &[Capability]) -> String { - families - .iter() - .map(|cap| cap.as_str()) - .collect::>() - .join(", ") -} - -/// Compare a driver's advertised capability set against its reachable surface. -/// -/// Walks every [`Capability`] in declaration order, so the returned vectors are -/// in that order too. -/// -/// # Errors -/// -/// Returns [`CapabilityAudit`] when the two disagree in either direction. A -/// driver that agrees with itself returns `Ok(())`. -/// -/// # Examples -/// -/// ``` -/// # use openhuman_core::openhuman::memory::api::null::NullMemoryProvider; -/// # use openhuman_core::openhuman::memory::api::provider::audit_provider; -/// // The reference null driver is self-consistent. -/// assert!(audit_provider(&NullMemoryProvider::new()).is_ok()); -/// ``` -pub fn audit_provider(provider: &dyn MemoryProvider) -> Result<(), CapabilityAudit> { - let advertised = provider.capabilities(); - let mut advertised_but_absent = Vec::new(); - let mut present_but_unadvertised = Vec::new(); - - for capability in Capability::ALL { - match ( - advertised.contains(capability), - provider.provides(capability), - ) { - (true, false) => advertised_but_absent.push(capability), - (false, true) => present_but_unadvertised.push(capability), - _ => {} - } - } - - if advertised_but_absent.is_empty() && present_but_unadvertised.is_empty() { - Ok(()) - } else { - Err(CapabilityAudit { - advertised_but_absent, - present_but_unadvertised, - }) - } -} - -#[cfg(test)] -#[path = "audit_tests.rs"] -mod tests; diff --git a/src/openhuman/memory/api/provider/audit_tests.rs b/src/openhuman/memory/api/provider/audit_tests.rs deleted file mode 100644 index 814f1597c5..0000000000 --- a/src/openhuman/memory/api/provider/audit_tests.rs +++ /dev/null @@ -1,201 +0,0 @@ -//! Tests for the advertised-vs-implemented honesty check. -//! -//! Two deliberately dishonest fixtures sit here — one that over-claims and one -//! that under-claims — because the whole value of [`audit_provider`] is -//! catching drivers that disagree with themselves, and neither direction is -//! reachable from an honest driver. - -use async_trait::async_trait; - -use super::*; -use crate::openhuman::memory::api::capabilities::Capabilities; -use crate::openhuman::memory::api::health::MemoryHealth; -use crate::openhuman::memory::api::null::NullMemoryProvider; -use crate::openhuman::memory::api::provider::types::{ - ExportPage, ExportRecord, ImportOutcome, SourceScope, -}; -use crate::openhuman::memory::api::provider::{ - MemoryCore, MemoryPortability, MemoryRecall, MemoryTree, -}; -use crate::openhuman::memory::api::recall::OwnedRecallOpts; -use crate::openhuman::memory::api::types::{ - MemoryCategory, MemoryEntry, MemoryTaint, NamespaceSummary, -}; - -/// A provider that forwards the mandatory three to [`NullMemoryProvider`] so -/// each fixture below only has to describe the thing it is lying about. -struct Fixture { - inner: NullMemoryProvider, - advertised: Capabilities, - expose_tree: bool, -} - -impl Fixture { - fn new(advertised: Capabilities, expose_tree: bool) -> Self { - Self { - inner: NullMemoryProvider::new(), - advertised, - expose_tree, - } - } -} - -#[async_trait] -impl MemoryCore for Fixture { - async fn store( - &self, - namespace: &str, - key: &str, - content: &str, - category: MemoryCategory, - session_id: Option<&str>, - taint: MemoryTaint, - ) -> Result<(), MemoryError> { - self.inner - .store(namespace, key, content, category, session_id, taint) - .await - } - - async fn get(&self, namespace: &str, key: &str) -> Result, MemoryError> { - self.inner.get(namespace, key).await - } - - async fn forget(&self, namespace: &str, key: &str) -> Result { - self.inner.forget(namespace, key).await - } - - async fn list( - &self, - namespace: Option<&str>, - category: Option<&MemoryCategory>, - session_id: Option<&str>, - ) -> Result, MemoryError> { - self.inner.list(namespace, category, session_id).await - } - - async fn namespaces(&self) -> Result, MemoryError> { - self.inner.namespaces().await - } -} - -#[async_trait] -impl MemoryRecall for Fixture { - async fn recall( - &self, - query: &str, - limit: usize, - opts: &OwnedRecallOpts, - scope: Option<&SourceScope>, - ) -> Result, MemoryError> { - self.inner.recall(query, limit, opts, scope).await - } -} - -#[async_trait] -impl MemoryPortability for Fixture { - async fn export_page( - &self, - cursor: Option<&str>, - limit: usize, - ) -> Result { - self.inner.export_page(cursor, limit).await - } - - async fn import_records( - &self, - records: Vec, - ) -> Result { - self.inner.import_records(records).await - } -} - -#[async_trait] -impl MemoryProvider for Fixture { - fn driver_id(&self) -> &str { - "fixture" - } - - fn capabilities(&self) -> Capabilities { - self.advertised - } - - async fn health(&self) -> MemoryHealth { - MemoryHealth::Ready - } - - fn as_tree(&self) -> Option<&dyn MemoryTree> { - if self.expose_tree { - Some(&self.inner) - } else { - None - } - } -} - -#[test] -fn honest_driver_passes_the_audit() { - let honest = Fixture::new(Capabilities::mandatory().with(Capability::Tree), true); - assert_eq!(audit_provider(&honest), Ok(())); -} - -#[test] -fn over_claiming_driver_is_reported_as_advertised_but_absent() { - // Advertises everything, exposes no optional accessor. Every one of the ten - // optional families would fail on first call — the exact - // registered-but-failing outcome the capability filter exists to prevent. - let liar = Fixture::new(Capabilities::all(), false); - - let audit = audit_provider(&liar).expect_err("over-claiming driver must fail the audit"); - assert_eq!(audit.present_but_unadvertised, Vec::new()); - assert_eq!(audit.advertised_but_absent.len(), 10); - assert!(audit.advertised_but_absent.contains(&Capability::Tree)); - // The mandatory three are supertraits, so they can never be missing. - assert!(!audit.advertised_but_absent.contains(&Capability::Core)); - assert!(!audit.advertised_but_absent.contains(&Capability::Recall)); - assert!(!audit - .advertised_but_absent - .contains(&Capability::Portability)); -} - -#[test] -fn under_claiming_driver_is_reported_as_present_but_unadvertised() { - // Implements the tree but forgot to list it: the family works and is - // completely unreachable, because the kernel filters from the advertised - // set. - let shy = Fixture::new(Capabilities::mandatory(), true); - - let audit = audit_provider(­).expect_err("under-claiming driver must fail the audit"); - assert_eq!(audit.advertised_but_absent, Vec::new()); - assert_eq!(audit.present_but_unadvertised, vec![Capability::Tree]); -} - -#[test] -fn audit_findings_are_reported_in_declaration_order() { - let liar = Fixture::new(Capabilities::all(), false); - let audit = audit_provider(&liar).expect_err("expected a mismatch"); - - let declaration_order: Vec = Capability::ALL - .into_iter() - .filter(|cap| audit.advertised_but_absent.contains(cap)) - .collect(); - assert_eq!(audit.advertised_but_absent, declaration_order); -} - -#[test] -fn audit_error_names_every_mismatched_family_and_maps_to_invalid() { - let liar = Fixture::new(Capabilities::all(), false); - let audit = audit_provider(&liar).expect_err("expected a mismatch"); - - let rendered = audit.to_string(); - for capability in &audit.advertised_but_absent { - assert!( - rendered.contains(capability.as_str()), - "audit message must name {capability}: {rendered}" - ); - } - - // A driver lying about itself is a config/implementation error, not an - // unsupported call. - let error: MemoryError = audit.into(); - assert!(matches!(error, MemoryError::Invalid(_))); -} diff --git a/src/openhuman/memory/api/provider/content.rs b/src/openhuman/memory/api/provider/content.rs deleted file mode 100644 index 1c1974367e..0000000000 --- a/src/openhuman/memory/api/provider/content.rs +++ /dev/null @@ -1,208 +0,0 @@ -//! Optional families that put content *into* memory and navigate it: -//! [`MemoryIngest`], [`MemoryDocuments`], and [`MemoryTree`]. -//! -//! All three are optional. A driver that advertises none of them is still a -//! memory backend — it just accepts entries only through -//! [`crate::openhuman::memory::api::provider::MemoryCore::store`] and has no document tier and no -//! summary tree. The kernel unregisters the matching RPC methods and omits the -//! matching agent tools rather than registering handlers that fail. -//! -//! ## No configuration crosses this boundary -//! -//! Chunk sizes, embedding models, summariser prompts, seal thresholds, and -//! cascade policy are all *driver* concerns. None of them appear in these -//! signatures: the embedded driver reads them from the `MemoryConfig` it -//! already holds, and an external driver has its own. This was the sharpest -//! test of whether the M0 crate carve-out drew the line in the right place — -//! the families that looked most config-dependent turned out not to need any. - -use async_trait::async_trait; - -use crate::openhuman::memory::api::capabilities::Capability; -use crate::openhuman::memory::api::chunks::Chunk; -use crate::openhuman::memory::api::error::MemoryError; -use crate::openhuman::memory::api::provider::types::{IngestItem, IngestOutcome, SourceScope}; -use crate::openhuman::memory::api::tree::{IngestRequest, QueryResult, TreeStatus}; -use crate::openhuman::memory::api::types::{ - NamespaceDocumentInput, NamespaceRetrievalContext, StoredMemoryDocument, -}; - -/// Bulk content ingestion — the driver owns chunking and embedding. -/// -/// The distinction from [`crate::openhuman::memory::api::provider::MemoryCore::store`] is ownership of -/// the pipeline: `store` persists exactly one entry the caller has already -/// shaped, whereas ingest hands over raw source material and lets the driver -/// decide how to split, embed, and index it. -#[async_trait] -pub trait MemoryIngest: Send + Sync { - /// Ingest one standalone document. - /// - /// # Errors - /// - /// [`MemoryError::Invalid`] for content the driver refuses (empty body, - /// unsupported MIME), otherwise backend failures. - async fn ingest_document(&self, item: IngestItem) -> Result; - - /// Ingest a run of chat messages that share a conversation. - /// - /// Taken as a batch rather than one call per message because chat chunking - /// is inherently cross-message: a driver needs neighbouring turns to decide - /// where a chunk boundary belongs. Ordering within `messages` is - /// significant and must be preserved by the caller. - /// - /// # Errors - /// - /// As [`Self::ingest_document`]. Partial success is reported through the - /// counts in [`IngestOutcome`], not as an error. - async fn ingest_chat(&self, messages: Vec) -> Result; -} - -/// The namespace-document tier: whole documents addressed by `(namespace, key)`. -/// -/// Distinct from [`crate::openhuman::memory::api::provider::MemoryCore`] in granularity and in what is -/// stored: entries are short facts, documents are bodies with titles, tags, -/// source types, and structured metadata, and they carry their own ranked query -/// surface. -#[async_trait] -pub trait MemoryDocuments: Send + Sync { - /// Upsert a document, returning its driver-assigned id. - /// - /// Keyed by `(namespace, key)` from the input: reusing a key replaces the - /// existing document rather than creating a second one. - /// - /// # Errors - /// - /// [`MemoryError::Invalid`] for a rejected input, otherwise backend - /// failures. - async fn put_document(&self, input: NamespaceDocumentInput) -> Result; - - /// Fetch a document by `(namespace, key)`. - /// - /// # Errors - /// - /// A missing document is `Ok(None)`; `Err` is reserved for backend - /// failures. - async fn get_document( - &self, - namespace: &str, - key: &str, - ) -> Result, MemoryError>; - - /// List document summaries, optionally restricted to one namespace. - async fn list_documents( - &self, - namespace: Option<&str>, - ) -> Result; - - /// List every namespace containing documents. - async fn list_namespaces(&self) -> Result, MemoryError>; - - /// Delete a document by its driver-assigned id. - async fn delete_document( - &self, - namespace: &str, - document_id: &str, - ) -> Result; - - /// Delete all data belonging to one namespace. - async fn clear_namespace(&self, namespace: &str) -> Result<(), MemoryError>; - - /// Run a ranked query over one namespace's documents. - /// - /// Returns both the ranked hits and the driver's rendered context text, so - /// a caller that only wants something injectable does not have to - /// re-assemble it (and re-assemble it differently from every other caller). - /// - /// # Errors - /// - /// Backend failures only; a query that matches nothing returns an empty - /// hit list. - async fn query_documents( - &self, - namespace: &str, - query: &str, - limit: usize, - ) -> Result; - - /// Recall the highest-ranked context from a namespace without a query. - /// - /// This is a distinct engine operation rather than a query with an empty - /// string: query-less recall applies the namespace's freshness and - /// priority ranking without introducing a synthetic search term. - /// - /// # Errors - /// - /// [`MemoryError::Unsupported`] when a provider predating this optional - /// operation does not implement it, otherwise backend failures. An empty - /// namespace returns empty context. - async fn recall_documents( - &self, - _namespace: &str, - _limit: usize, - ) -> Result { - Err(MemoryError::unsupported(Capability::Documents)) - } -} - -/// The time-ordered summary tree: buffered leaves rolled up into hour → day → -/// month → year → root summaries. -/// -/// Sealing and cascading are exposed as explicit calls rather than happening -/// implicitly on ingest because the **host** owns scheduling. A driver runs one -/// step when asked; it does not get to install its own background loop. This is -/// the same rule as the engine's `queue::run_once`. -#[async_trait] -pub trait MemoryTree: Send + Sync { - /// Append raw content to the ingestion buffer for later sealing. - /// - /// # Errors - /// - /// [`MemoryError::Invalid`] for a rejected request, otherwise backend - /// failures. - async fn append(&self, request: IngestRequest) -> Result<(), MemoryError>; - - /// Retrieve the chunks a single logical source contributed, newest first. - /// - /// `scope` is the per-turn allowlist and must be applied **inside** the - /// driver's query, for the reasons in [`SourceScope`]. `None` means - /// unrestricted. - /// - /// # Errors - /// - /// Backend failures only; an unknown `source_id` yields an empty vector. - async fn query_source( - &self, - namespace: &str, - source_id: &str, - limit: usize, - scope: Option<&SourceScope>, - ) -> Result, MemoryError>; - - /// Fetch one node together with its direct children, for navigation. - /// - /// # Errors - /// - /// [`MemoryError::NotFound`] when `node_id` does not exist in `namespace`. - async fn drill_down(&self, namespace: &str, node_id: &str) -> Result; - - /// Convert buffered content into leaf nodes, returning the resulting tree - /// state. - /// - /// Idempotent when the buffer is empty: sealing nothing is a successful - /// no-op, not an error, so a scheduler may call it unconditionally. - /// - /// # Errors - /// - /// Backend failures only. - async fn seal(&self, namespace: &str) -> Result; - - /// Roll sealed leaves up through the parent levels, returning the resulting - /// tree state. - /// - /// Idempotent for the same reason as [`Self::seal`]. - /// - /// # Errors - /// - /// Backend failures only. - async fn cascade(&self, namespace: &str) -> Result; -} diff --git a/src/openhuman/memory/api/provider/driver.rs b/src/openhuman/memory/api/provider/driver.rs deleted file mode 100644 index f099012bc6..0000000000 --- a/src/openhuman/memory/api/provider/driver.rs +++ /dev/null @@ -1,199 +0,0 @@ -//! [`MemoryProvider`] — the single trait a memory driver implements, and the -//! object the kernel binds. -//! -//! ## Self-contained on purpose -//! -//! `MemoryProvider` does **not** extend a host `Driver` trait and names no host -//! type. `tinymemory-api` is what a third-party driver compiles against, so it -//! must not drag in the OpenHuman host; and the generic subsystem vocabulary -//! (`Driver`, `DriverClass`, `SubsystemRegistry`, the policy `Guard`) belongs -//! kernel-side, where inference and channels can share it without importing a -//! *memory* crate. -//! -//! The bridge is the host's memory adapter, which implements the host `Driver` -//! for an `Arc` and converts [`MemoryHealth`] into the -//! kernel's `DriverHealth`. That conversion is trivial by construction — see -//! [`crate::openhuman::memory::api::health`]. -//! -//! Driver **class** (embedded / external / null) is deliberately absent from -//! this trait. Class is a fact about how the host bound a driver, recorded in -//! host configuration; a driver self-reporting it would let a misconfigured -//! external backend claim to be embedded and skip the egress and trust checks -//! that class gates. -//! -//! ## The accessor form, and why not `Any` -//! -//! The kernel binds `Arc` and needs per-family access. Two -//! designs were available: downcast through [`std::any::Any`], or one -//! `Option`-returning accessor per optional family. The accessors win: -//! -//! - **No unchecked downcast.** `Any` would require the caller to name a -//! concrete driver type, which defeats the point of binding behind a trait -//! object, or to register type ids, which is the same table with worse -//! ergonomics. -//! - **The capability set and the reachable surface stay provably in sync.** -//! [`crate::openhuman::memory::api::provider::audit_provider`] compares [`MemoryProvider::capabilities`] -//! against what the accessors actually return, so "advertised but not -//! implemented" is a detectable, testable mistake instead of a runtime -//! surprise on the first call. -//! - **[`MemoryProvider::provides`] is an exhaustive `match`** over -//! [`Capability`], so adding a family without wiring an accessor fails to -//! compile. -//! -//! The three mandatory families are supertraits rather than accessors, so they -//! are callable directly on the trait object and cannot be absent. -//! -//! ## Object safety -//! -//! Every method here and in every family trait is object-safe: no generic -//! parameters, no `Self` in return position, no associated constants. The -//! `#[async_trait]` attribute rewrites the `async fn`s into boxed futures, -//! which is what makes them dyn-compatible at all. - -use async_trait::async_trait; - -use crate::openhuman::memory::api::capabilities::{Capabilities, Capability}; -use crate::openhuman::memory::api::error::MemoryError; -use crate::openhuman::memory::api::health::MemoryHealth; -use crate::openhuman::memory::api::provider::content::{MemoryDocuments, MemoryIngest, MemoryTree}; -use crate::openhuman::memory::api::provider::knowledge::{MemoryDiff, MemoryEntities, MemoryGraph}; -use crate::openhuman::memory::api::provider::mandatory::{ - MemoryCore, MemoryPortability, MemoryRecall, -}; -use crate::openhuman::memory::api::provider::records::{ - MemoryGoals, MemoryMaintenance, MemorySourceSink, MemoryToolMemory, -}; - -/// A bound memory driver. -/// -/// Implementors must also implement the three mandatory families -/// ([`MemoryCore`], [`MemoryRecall`], [`MemoryPortability`]) — they are -/// supertraits, so a driver missing any of them cannot be constructed as a -/// provider at all. -/// -/// The ten optional families are reached through the `as_*` accessors below. -/// Each defaults to `None`, so a minimal driver implements only what it -/// supports and inherits correct absence for everything else. -#[async_trait] -pub trait MemoryProvider: MemoryCore + MemoryRecall + MemoryPortability + 'static { - /// Stable identifier for this driver (`tinycortex`, `supermemory`, `null`). - /// - /// Appears in status output, log lines, tracing spans, and audit events, so - /// it must be stable across restarts and must not embed a URL, a token, or - /// anything else user- or deployment-specific. - fn driver_id(&self) -> &str; - - /// The families this driver implements. - /// - /// Asked **once** at bind time and cached: the kernel filters RPC - /// registration and agent-tool assembly from the cached answer, so a set - /// that changes after binding will not be noticed. A driver whose surface - /// genuinely varies must report the union and answer - /// [`MemoryError::Unsupported`] for the gaps. - /// - /// Must be honest: every advertised family must be reachable through its - /// accessor. [`crate::openhuman::memory::api::provider::audit_provider`] checks exactly that. - fn capabilities(&self) -> Capabilities; - - /// Current liveness, as the driver reports it. - /// - /// Called on bind and on demand for status output. Implementations should - /// be cheap and must not block indefinitely — a health probe that hangs is - /// indistinguishable from a subsystem that is down, but takes a timeout to - /// find out. - async fn health(&self) -> MemoryHealth; - - /// Release resources ahead of process exit or a rebind. - /// - /// Defaults to a successful no-op, because most drivers have nothing to - /// release; a driver holding a connection pool or a background task should - /// override it. The host's adapter forwards its `Driver::shutdown` here. - /// - /// Must be idempotent: a rebind followed by process exit calls it twice. - /// - /// # Errors - /// - /// Backend failures during teardown. The caller logs and continues — - /// shutdown failure never blocks exit. - async fn shutdown(&self) -> Result<(), MemoryError> { - Ok(()) - } - - /// Bulk ingestion, when advertised. - fn as_ingest(&self) -> Option<&dyn MemoryIngest> { - None - } - - /// The namespace-document tier, when advertised. - fn as_documents(&self) -> Option<&dyn MemoryDocuments> { - None - } - - /// The summary tree, when advertised. - fn as_tree(&self) -> Option<&dyn MemoryTree> { - None - } - - /// The entity index, when advertised. - fn as_entities(&self) -> Option<&dyn MemoryEntities> { - None - } - - /// The key/value and relation graph, when advertised. - fn as_graph(&self) -> Option<&dyn MemoryGraph> { - None - } - - /// Snapshot and change tracking, when advertised. - fn as_diff(&self) -> Option<&dyn MemoryDiff> { - None - } - - /// The long-term goals document, when advertised. - fn as_goals(&self) -> Option<&dyn MemoryGoals> { - None - } - - /// Per-tool learned rules, when advertised. - fn as_tool_memory(&self) -> Option<&dyn MemoryToolMemory> { - None - } - - /// The host-sync write seam, when advertised. - fn as_sources(&self) -> Option<&dyn MemorySourceSink> { - None - } - - /// Scheduler-driven upkeep, when advertised. - fn as_maintenance(&self) -> Option<&dyn MemoryMaintenance> { - None - } - - /// Whether `capability` is actually **reachable** on this driver. - /// - /// This is the implementation-side truth, as opposed to - /// [`Self::capabilities`], which is the advertised claim. The two should - /// agree; [`crate::openhuman::memory::api::provider::audit_provider`] is where they are compared. - /// - /// The mandatory three are always `true` because they are supertraits. The - /// remaining ten delegate to their accessor. - /// - /// The `match` is deliberately exhaustive: [`Capability`] is not - /// `#[non_exhaustive]`, so adding a family without adding an accessor and - /// an arm here is a compile error rather than a silent `false`. - fn provides(&self, capability: Capability) -> bool { - match capability { - Capability::Core | Capability::Recall | Capability::Portability => true, - Capability::Ingest => self.as_ingest().is_some(), - Capability::Documents => self.as_documents().is_some(), - Capability::Tree => self.as_tree().is_some(), - Capability::Entities => self.as_entities().is_some(), - Capability::Graph => self.as_graph().is_some(), - Capability::Diff => self.as_diff().is_some(), - Capability::Goals => self.as_goals().is_some(), - Capability::ToolMemory => self.as_tool_memory().is_some(), - Capability::Sources => self.as_sources().is_some(), - Capability::Maintenance => self.as_maintenance().is_some(), - } - } -} diff --git a/src/openhuman/memory/api/provider/knowledge.rs b/src/openhuman/memory/api/provider/knowledge.rs deleted file mode 100644 index 2a49a3a9a8..0000000000 --- a/src/openhuman/memory/api/provider/knowledge.rs +++ /dev/null @@ -1,179 +0,0 @@ -//! Optional families that expose *derived structure* over stored memory: -//! [`MemoryEntities`], [`MemoryGraph`], and [`MemoryDiff`]. -//! -//! Each is independently optional. A driver may have a key/value graph but no -//! entity index, or track source snapshots without either. The kernel filters -//! RPC registration and agent-tool assembly per family, so an absent family is -//! invisible rather than present-and-failing. -//! -//! As in [`crate::openhuman::memory::api::provider::content`], no configuration crosses this boundary: -//! extraction models, hotness decay curves, and snapshot retention are driver -//! concerns and appear in none of these signatures. - -use async_trait::async_trait; - -use crate::openhuman::memory::api::error::MemoryError; -use crate::openhuman::memory::api::provider::types::{DiffReport, EntityHit, SnapshotRef}; -use crate::openhuman::memory::api::types::{GraphRelationRecord, MemoryKvRecord}; - -/// The entity index: who and what the stored memory is about. -#[async_trait] -pub trait MemoryEntities: Send + Sync { - /// List entities in a namespace, ranked by hotness when `query` is `None` - /// and by match quality otherwise. - /// - /// # Errors - /// - /// Backend failures only; an unknown namespace yields an empty vector. - async fn entities( - &self, - namespace: &str, - query: Option<&str>, - limit: usize, - ) -> Result, MemoryError>; - - /// Edges incident to one entity, most relevant first. - /// - /// Returns [`GraphRelationRecord`] — the same shape [`MemoryGraph`] uses — - /// so a caller that has both families does not have to reconcile two edge - /// representations. - /// - /// # Errors - /// - /// Backend failures only; an unknown `entity_id` yields an empty vector - /// rather than [`MemoryError::NotFound`], because "no edges" and "no such - /// entity" are the same answer to this question. - async fn entity_edges( - &self, - namespace: &str, - entity_id: &str, - limit: usize, - ) -> Result, MemoryError>; - - /// Record that these entities were just observed, updating hotness. - /// - /// Separate from the read path because hotness is a *write* the host - /// triggers at known moments (a turn referenced these entities), not - /// something a driver should infer from being queried — otherwise merely - /// browsing the index would reshape ranking. - /// - /// # Errors - /// - /// Backend failures only. Unknown ids are ignored, not rejected. - async fn touch_entities( - &self, - namespace: &str, - entity_ids: &[String], - ) -> Result<(), MemoryError>; -} - -/// The key/value and relation graph tier. -/// -/// `namespace` is `Option<&str>` throughout: `None` addresses the global, -/// namespace-less slice, matching the storage shape of -/// [`MemoryKvRecord::namespace`] and [`GraphRelationRecord::namespace`]. -#[async_trait] -pub trait MemoryGraph: Send + Sync { - /// Read one key/value record. - /// - /// # Errors - /// - /// A missing key is `Ok(None)`; `Err` is reserved for backend failures. - async fn kv_get( - &self, - namespace: Option<&str>, - key: &str, - ) -> Result, MemoryError>; - - /// Upsert one key/value record. - /// - /// # Errors - /// - /// [`MemoryError::Invalid`] for a rejected key, otherwise backend failures. - async fn kv_put( - &self, - namespace: Option<&str>, - key: &str, - value: serde_json::Value, - ) -> Result<(), MemoryError>; - - /// Delete one key/value record, reporting whether it existed. - async fn kv_delete(&self, namespace: Option<&str>, key: &str) -> Result; - - /// List key/value records, optionally restricted to a key prefix. - /// - /// # Errors - /// - /// Backend failures only. - async fn kv_list( - &self, - namespace: Option<&str>, - prefix: Option<&str>, - limit: usize, - ) -> Result, MemoryError>; - - /// Query relations, narrowing by subject and/or predicate. - /// - /// Both filters are `None`-able so one method covers "everything about this - /// subject", "every edge of this type", and "the whole slice", instead of - /// three near-identical methods. - /// - /// # Errors - /// - /// Backend failures only. - async fn relations( - &self, - namespace: Option<&str>, - subject: Option<&str>, - predicate: Option<&str>, - limit: usize, - ) -> Result, MemoryError>; - - /// Upsert one relation, keyed by `(namespace, subject, predicate, object)`. - /// - /// # Errors - /// - /// [`MemoryError::Invalid`] for a malformed edge, otherwise backend - /// failures. - async fn put_relation(&self, relation: GraphRelationRecord) -> Result<(), MemoryError>; -} - -/// Snapshot capture and change computation over synced sources. -#[async_trait] -pub trait MemoryDiff: Send + Sync { - /// Capture a snapshot of one source's current items. - /// - /// # Errors - /// - /// [`MemoryError::NotFound`] for an unknown `source_id`, otherwise backend - /// failures. - async fn capture_snapshot(&self, source_id: &str) -> Result; - - /// List snapshots for one source, newest first. - /// - /// # Errors - /// - /// Backend failures only; an unknown `source_id` yields an empty vector. - async fn snapshots( - &self, - source_id: &str, - limit: usize, - ) -> Result, MemoryError>; - - /// Compute the change set between two snapshots of one source. - /// - /// `from` is `Option<&str>` so the first-ever diff — where there is no - /// baseline and every item is an addition — is expressible without a - /// separate method or a sentinel id. - /// - /// # Errors - /// - /// [`MemoryError::NotFound`] when either snapshot id is unknown, otherwise - /// backend failures. - async fn diff( - &self, - source_id: &str, - from: Option<&str>, - to: &str, - ) -> Result; -} diff --git a/src/openhuman/memory/api/provider/mandatory.rs b/src/openhuman/memory/api/provider/mandatory.rs deleted file mode 100644 index 4699bcea2a..0000000000 --- a/src/openhuman/memory/api/provider/mandatory.rs +++ /dev/null @@ -1,188 +0,0 @@ -//! The three mandatory capability families: [`MemoryCore`], [`MemoryRecall`], -//! and [`MemoryPortability`]. -//! -//! These are supertraits of [`crate::openhuman::memory::api::provider::MemoryProvider`], which is what -//! makes "mandatory" a *compile-time* fact rather than a runtime check: a type -//! that does not implement all three cannot be a provider at all, so there is -//! no way to bind a driver that is missing them. -//! -//! The other ten families are reached through `Option`-returning accessors on -//! the provider, so their absence is representable and their presence is not -//! assumed. See [`crate::openhuman::memory::api::provider::MemoryProvider`] for that half. -//! -//! ## Why every method returns [`MemoryError`] and not `anyhow::Error` -//! -//! The transport adapter must be able to turn a `501` from an out-of-process -//! driver into [`MemoryError::Unsupported`], and the kernel must be able to -//! tell "this driver cannot do that" apart from "this driver failed". An -//! `anyhow::Error` erases exactly that distinction. The engine's own -//! [`crate::openhuman::memory::api::traits::Memory`] trait keeps `anyhow::Result` — it is an internal -//! storage abstraction with existing implementors, not the driver contract. - -use async_trait::async_trait; - -use crate::openhuman::memory::api::error::MemoryError; -use crate::openhuman::memory::api::provider::types::{ - ExportPage, ExportRecord, ImportOutcome, SourceScope, -}; -use crate::openhuman::memory::api::recall::OwnedRecallOpts; -use crate::openhuman::memory::api::types::{ - MemoryCategory, MemoryEntry, MemoryTaint, NamespaceSummary, -}; - -/// Store, read, and delete individual memory entries. **Mandatory.** -/// -/// This is the smallest surface that still makes something a memory backend: -/// without it there is nothing to recall from and nothing to export. -#[async_trait] -pub trait MemoryCore: Send + Sync { - /// Upsert an entry, keyed by `(namespace, key)`. - /// - /// ## Taint is an argument, never a decision - /// - /// Unlike the engine's [`crate::openhuman::memory::api::traits::Memory`], which has a `store` and a - /// separate `store_with_taint` whose default implementation silently drops - /// the taint, the contract has **one** store and it always takes a - /// [`MemoryTaint`]. Provenance is stamped by the host policy guard before - /// the call; a driver that could default it would be able to launder - /// externally-sourced content into internal-trust content, which is the - /// single failure mode the guard exists to prevent. - /// - /// # Errors - /// - /// [`MemoryError::Invalid`] for caller input the driver rejects, - /// [`MemoryError::Io`] or [`MemoryError::Other`] for backend failures. - async fn store( - &self, - namespace: &str, - key: &str, - content: &str, - category: MemoryCategory, - session_id: Option<&str>, - taint: MemoryTaint, - ) -> Result<(), MemoryError>; - - /// Fetch the entry for an exact `(namespace, key)`. - /// - /// # Errors - /// - /// A missing entry is `Ok(None)`, never an error; `Err` is reserved for - /// backend failures. - async fn get(&self, namespace: &str, key: &str) -> Result, MemoryError>; - - /// Delete the entry for `(namespace, key)`, reporting whether it existed. - /// - /// Idempotent: forgetting an absent key is `Ok(false)`, so callers may call - /// it unconditionally. - /// - /// # Errors - /// - /// Backend failures only. - async fn forget(&self, namespace: &str, key: &str) -> Result; - - /// List entries, narrowing by namespace, category, and session. - /// - /// Each `Some` filter narrows the result; all `None` lists everything the - /// driver holds. An empty result is `Ok(vec![])`. - /// - /// # Errors - /// - /// Backend failures only. - async fn list( - &self, - namespace: Option<&str>, - category: Option<&MemoryCategory>, - session_id: Option<&str>, - ) -> Result, MemoryError>; - - /// Enumerate namespaces with their aggregate counts, for discovery. - /// - /// # Errors - /// - /// Backend failures only. - async fn namespaces(&self) -> Result, MemoryError>; -} - -/// Ranked retrieval. **Mandatory.** -#[async_trait] -pub trait MemoryRecall: Send + Sync { - /// Return up to `limit` entries relevant to `query`, most relevant first. - /// - /// `opts` is the **owned** [`OwnedRecallOpts`], never the borrowed - /// `RecallOpts<'a>`: a lifetime parameter cannot travel through an - /// object-safe `#[async_trait]` method, and the borrowed form derives no - /// serde impls so it could never be a request body. An embedded driver - /// converts to the borrowed form at its own boundary, which is zero-copy. - /// - /// `scope` is the per-turn source allowlist and is a **query predicate the - /// driver must apply internally** — see [`SourceScope`] for why applying it - /// after the fact is wrong. `None` means unrestricted. - /// - /// An empty or non-matching `query` yields `Ok(vec![])`, not an error. - /// - /// # Errors - /// - /// [`MemoryError::Invalid`] for a malformed filter, otherwise backend - /// failures. - async fn recall( - &self, - query: &str, - limit: usize, - opts: &OwnedRecallOpts, - scope: Option<&SourceScope>, - ) -> Result, MemoryError>; -} - -/// Export and import the whole store. **Mandatory.** -/// -/// Mandatory because binding a memory backend without it is a one-way door: a -/// user who cannot export cannot leave. It is the capability that makes every -/// other binding reversible, which is also why the `mirror` migration driver is -/// expressible at all. -#[async_trait] -pub trait MemoryPortability: Send + Sync { - /// Read one page of the export, continuing from `cursor`. - /// - /// Pass `None` to start. The export is complete when the returned - /// [`ExportPage::next_cursor`] is `None` — an empty `records` vector is - /// **not** a terminator, because a driver may legitimately return an empty - /// page while skipping a range. - /// - /// `limit` is a request, not a guarantee; a driver may return fewer. - /// - /// ## Why pages and not a stream - /// - /// A `Stream` return type would either make the trait non-object-safe or - /// drag an async runtime into a crate that deliberately has none. Paging - /// keeps both properties and still bounds memory, with the caller choosing - /// the bound. - /// - /// # Errors - /// - /// [`MemoryError::Invalid`] for a cursor this driver did not issue, - /// otherwise backend failures. - async fn export_page( - &self, - cursor: Option<&str>, - limit: usize, - ) -> Result; - - /// Write a batch of previously-exported records. - /// - /// Records carry their own [`crate::openhuman::memory::api::types::MemoryTaint`]; an importing - /// driver must persist what it is given and must not re-stamp provenance. - /// - /// Partial success is normal and is reported in [`ImportOutcome`] rather - /// than as an error: a migration should not abort a million-record restore - /// because one record was malformed. - /// - /// # Errors - /// - /// Reserved for failures that make the whole batch meaningless (backend - /// unavailable, transaction aborted). Per-record rejection belongs in - /// [`ImportOutcome::failed`]. - async fn import_records( - &self, - records: Vec, - ) -> Result; -} diff --git a/src/openhuman/memory/api/provider/mod.rs b/src/openhuman/memory/api/provider/mod.rs deleted file mode 100644 index 8026d1fb9b..0000000000 --- a/src/openhuman/memory/api/provider/mod.rs +++ /dev/null @@ -1,73 +0,0 @@ -//! The memory driver contract: [`MemoryProvider`] plus the thirteen capability -//! family traits a driver may implement. -//! -//! ## Shape -//! -//! ```text -//! MemoryProvider ── identity, capabilities, health, shutdown -//! : MemoryCore (mandatory — supertrait, always callable) -//! : MemoryRecall (mandatory — supertrait, always callable) -//! : MemoryPortability (mandatory — supertrait, always callable) -//! ├─ as_ingest() -> Option<&dyn MemoryIngest> -//! ├─ as_documents() -> Option<&dyn MemoryDocuments> -//! ├─ as_tree() -> Option<&dyn MemoryTree> -//! ├─ as_entities() -> Option<&dyn MemoryEntities> -//! ├─ as_graph() -> Option<&dyn MemoryGraph> -//! ├─ as_diff() -> Option<&dyn MemoryDiff> -//! ├─ as_goals() -> Option<&dyn MemoryGoals> -//! ├─ as_tool_memory() -> Option<&dyn MemoryToolMemory> -//! ├─ as_sources() -> Option<&dyn MemorySourceSink> -//! └─ as_maintenance() -> Option<&dyn MemoryMaintenance> -//! ``` -//! -//! The mandatory three are supertraits, so "mandatory" is enforced by the type -//! system rather than by a runtime check. The optional ten are accessors that -//! default to `None`, so absence is the default and presence is opt-in. -//! -//! ## Rules that bind every family -//! -//! 1. **Typed errors, always.** Every method returns -//! `Result<_, MemoryError>`. The transport adapter maps an out-of-process -//! `501` onto [`crate::openhuman::memory::api::error::MemoryError::Unsupported`], and the kernel -//! distinguishes "cannot" from "failed". `anyhow::Error` would erase that. -//! 2. **No configuration crosses the boundary.** Not one signature names a -//! config type. A driver holds its own configuration; the contract passes -//! domain arguments only. -//! 3. **No host types.** Nothing here names an OpenHuman type, so a -//! third-party driver depends on this crate alone. -//! 4. **The driver never assigns provenance.** [`crate::openhuman::memory::api::types::MemoryTaint`] is -//! an argument on every write path and a preserved field on every import. -//! 5. **The host owns the loop.** Sealing, cascading, maintenance, and source -//! sync are all "run one step when asked"; no driver installs a background -//! task or hooks the agent turn. -//! 6. **Object safety throughout.** No generics, no `Self` returns, no -//! associated constants — every family is usable as `&dyn`. -//! -//! ## Reference implementation -//! -//! [`crate::openhuman::memory::api::null::NullMemoryProvider`] implements all thirteen families: -//! `/dev/null` semantics for the mandatory three, and -//! [`crate::openhuman::memory::api::error::MemoryError::Unsupported`] for the other ten, which it does -//! not advertise. It is what a compiled-out or unconfigured memory subsystem -//! binds to, and it doubles as the proof that the mandatory set is -//! implementable without a storage engine. - -pub mod audit; -pub mod content; -pub mod driver; -pub mod knowledge; -pub mod mandatory; -pub mod records; -pub mod types; - -pub use audit::{audit_provider, CapabilityAudit}; -pub use content::{MemoryDocuments, MemoryIngest, MemoryTree}; -pub use driver::MemoryProvider; -pub use knowledge::{MemoryDiff, MemoryEntities, MemoryGraph}; -pub use mandatory::{MemoryCore, MemoryPortability, MemoryRecall}; -pub use records::{MemoryGoals, MemoryMaintenance, MemorySourceSink, MemoryToolMemory}; -pub use types::{ - ChangeKind, DiffReport, EntityHit, EntityRef, ExportPage, ExportRecord, ImportOutcome, - IngestItem, IngestOutcome, MaintenanceReport, SnapshotRef, SourceChange, SourceItem, - SourceScope, -}; diff --git a/src/openhuman/memory/api/provider/records.rs b/src/openhuman/memory/api/provider/records.rs deleted file mode 100644 index 5b8ee13a67..0000000000 --- a/src/openhuman/memory/api/provider/records.rs +++ /dev/null @@ -1,169 +0,0 @@ -//! The remaining optional families: [`MemoryGoals`], [`MemoryToolMemory`], -//! [`MemorySourceSink`], and [`MemoryMaintenance`]. -//! -//! Goals and tool memory are small curated record sets the agent reads on -//! nearly every turn. The source sink is the seam the host's sync machinery -//! writes through. Maintenance is the seam the host's scheduler drives. -//! -//! ## The host keeps the loop; the driver runs one step -//! -//! [`MemorySourceSink`] receives already-fetched items — the host owns -//! credentials, OAuth, rate limits, and the schedule. [`MemoryMaintenance`] -//! exposes four operations the host's existing scheduler calls; no driver -//! installs a background task of its own. Both follow the same rule as the -//! engine's `queue::run_once`, and both are why a driver never needs to see -//! configuration or a keychain. - -use async_trait::async_trait; - -use crate::openhuman::memory::api::error::MemoryError; -use crate::openhuman::memory::api::goals::GoalsDoc; -use crate::openhuman::memory::api::provider::types::{ - IngestOutcome, MaintenanceReport, SourceItem, -}; -use crate::openhuman::memory::api::tool_memory::ToolMemoryRule; -use crate::openhuman::memory::api::types::MemoryTaint; - -/// The agent's long-term goals document. -#[async_trait] -pub trait MemoryGoals: Send + Sync { - /// Read the current goals document. - /// - /// A driver with no goals yet returns an empty [`GoalsDoc`], not - /// [`MemoryError::NotFound`] — "no goals" is a valid state, not a missing - /// record. - /// - /// # Errors - /// - /// Backend failures only. - async fn goals(&self) -> Result; - - /// Replace the goals document wholesale. - /// - /// Whole-document replacement rather than per-item add/edit/delete because - /// the validating mutation surface (PII and secret predicates) is **host** - /// policy: the host parses, validates, mutates, and hands back the result. - /// Exposing per-item mutation here would put that policy behind a trait a - /// third-party driver implements, where it could be skipped. - /// - /// # Errors - /// - /// [`MemoryError::Invalid`] for a document the driver refuses (e.g. over - /// its own item cap), otherwise backend failures. - async fn set_goals(&self, goals: GoalsDoc) -> Result<(), MemoryError>; -} - -/// Per-tool learned rules — durable guidance attached to a specific tool. -#[async_trait] -pub trait MemoryToolMemory: Send + Sync { - /// Rules for one tool, highest priority first. - /// - /// # Errors - /// - /// Backend failures only; a tool with no rules yields an empty vector. - async fn tool_rules(&self, tool_name: &str) -> Result, MemoryError>; - - /// Upsert one rule, keyed by [`ToolMemoryRule::id`]. - /// - /// # Errors - /// - /// [`MemoryError::Invalid`] for a malformed rule, otherwise backend - /// failures. - async fn put_tool_rule(&self, rule: ToolMemoryRule) -> Result<(), MemoryError>; - - /// Delete one rule, reporting whether it existed. - /// - /// Idempotent, like [`crate::openhuman::memory::api::provider::MemoryCore::forget`]. - /// - /// # Errors - /// - /// Backend failures only. - async fn delete_tool_rule(&self, tool_name: &str, rule_id: &str) -> Result; -} - -/// The write seam for host-driven source sync. -#[async_trait] -pub trait MemorySourceSink: Send + Sync { - /// Accept a batch of items the host fetched from one logical source. - /// - /// `taint` applies to the whole batch and is stamped by the host. Sync - /// paths ingesting third-party content pass - /// [`MemoryTaint::ExternalSync`]; the driver persists what it is given and - /// never assigns provenance itself. - /// - /// `source_kind` is a wire string (`folder`, `composio`, …) rather than an - /// enum because the set of source kinds is owned by the host's sync - /// machinery and grows without a contract change. - /// - /// # Errors - /// - /// [`MemoryError::Invalid`] for a rejected batch, otherwise backend - /// failures. Per-item outcomes are counted in [`IngestOutcome`]. - async fn accept_source_items( - &self, - source_id: &str, - source_kind: &str, - items: Vec, - taint: MemoryTaint, - ) -> Result; - - /// Drop everything the driver holds for one logical source, returning how - /// many units were removed. - /// - /// This is the disconnect path: when a user removes a source, its content - /// must leave memory. Idempotent — an unknown `source_id` returns `Ok(0)`. - /// - /// # Errors - /// - /// Backend failures only. - async fn forget_source(&self, source_id: &str) -> Result; -} - -/// Periodic upkeep the host's scheduler drives. -/// -/// All four operations must be safe to call repeatedly and safe to interrupt: -/// the scheduler may invoke them on a timer, and a desktop process can exit at -/// any point. A driver that cannot bound the work should do a slice per call -/// and report progress in [`MaintenanceReport`]. -#[async_trait] -pub trait MemoryMaintenance: Send + Sync { - /// Recompute embeddings for content whose embedding is missing or stale. - /// - /// # Errors - /// - /// Backend failures, or [`MemoryError::BudgetExceeded`] when an embedding - /// budget is exhausted mid-run. - async fn reembed(&self) -> Result; - - /// Reclaim space: vacuum indexes, drop tombstones, prune dead references. - /// - /// # Errors - /// - /// Backend failures only. - async fn compact(&self) -> Result; - - /// Merge and summarise accumulated memory — the "dream" pass. - /// - /// The embedded driver maps this onto its seal/cascade/reembed cycle; an - /// external driver maps it onto whatever it calls the same idea. The - /// contract deliberately does not specify the mechanism, only that it is - /// the operation a scheduler runs when the system is idle. - /// - /// # Errors - /// - /// Backend failures only. - async fn consolidate(&self) -> Result; - - /// Read-only integrity check. - /// - /// Reports findings in [`MaintenanceReport::findings`] and must change - /// nothing — [`MaintenanceReport::changed`] is always `0`. A driver that - /// repairs as it inspects should expose that as [`Self::compact`] instead, - /// so an operator can diagnose without mutating. - /// - /// # Errors - /// - /// Backend failures only. A *finding* is not an error: a store with - /// problems still returns `Ok` with the problems listed. - async fn doctor(&self) -> Result; -} diff --git a/src/openhuman/memory/api/provider/types.rs b/src/openhuman/memory/api/provider/types.rs deleted file mode 100644 index 830d0af3c3..0000000000 --- a/src/openhuman/memory/api/provider/types.rs +++ /dev/null @@ -1,393 +0,0 @@ -//! Value types that exist only because the *driver contract* needs them. -//! -//! Everything here is inert data: serde-derived, dependency-light, and free of -//! any engine or host type. They are separated from [`crate::openhuman::memory::api::types`] because -//! that module carries the historical engine value types (which the engine -//! crate aliases back into `crate::openhuman::memory::engine::types`), whereas these are new -//! shapes introduced by the provider contract itself. -//! -//! ## Why these types and not the engine's -//! -//! Several families the contract exposes (diff, entities, sources, -//! maintenance) have richer types inside the `tinycortex` engine — for example -//! `memory::diff::types::DiffResult`. Those types are *implementation* shapes: -//! they carry git commit SHAs, ledger paths, and engine-specific enums. A -//! third-party driver cannot produce them and must not be required to. -//! -//! So the contract defines the narrower shape a *caller* actually needs, with -//! wire strings deliberately identical to the engine's where they overlap -//! (`added`/`removed`/`modified`), so the embedded driver's conversion is a -//! field-for-field map rather than a translation. -//! -//! ## What is deliberately absent -//! -//! No type here names a configuration struct. `MemoryConfig` stayed engine-side -//! in the M0 carve-out and stays there: a driver holds its own configuration -//! and the contract passes only domain arguments. If a future method cannot be -//! expressed without configuration, that is a signal the family was designed -//! wrong, not that the contract should widen. - -use chrono::{DateTime, Utc}; -use serde::{Deserialize, Serialize}; - -use crate::openhuman::memory::api::chunks::{DataSource, SourceRef}; -use crate::openhuman::memory::api::types::MemoryTaint; - -/// A per-turn allowlist of memory sources, passed **into** the driver as a -/// query predicate. -/// -/// ## Why this is a parameter and not a post-filter -/// -/// The host computes a per-turn source allowlist from product policy. If that -/// allowlist were applied after the driver returned rows, a `limit` would be -/// consumed by rows the caller is not allowed to see — so a scoped query could -/// return fewer results than it should, or none at all, purely as an artefact -/// of filtering order. Worse, an out-of-process driver would have already been -/// handed a query it should never have answered in full. -/// -/// The predicate therefore travels with the call. `None` means unrestricted; -/// `Some(scope)` means the driver must apply it *inside* its query. -/// -/// ## Matching rule (fail-closed) -/// -/// [`SourceScope::allows_source_id`] encodes the embedded engine's SQL -/// semantics verbatim: a source-attributed id is in scope when it either equals -/// an allowed id outright, or begins with `mem_src:{allowed}:`. An **empty** -/// allow list therefore matches nothing — a scope that lists no sources denies -/// all source-attributed content rather than waving it through. -/// -/// Content that is not attributed to a memory source at all (no -/// `memory_sources` provenance) is outside this predicate's remit; the driver -/// decides that, exactly as the engine's SQL does today. -#[derive(Clone, Debug, Default, PartialEq, Eq, Serialize, Deserialize)] -pub struct SourceScope { - /// Allowed memory-source identifiers. Empty denies all source-attributed - /// content. - pub allow: Vec, -} - -impl SourceScope { - /// Builds a scope from any iterator of source identifiers. - pub fn new(allow: impl IntoIterator>) -> Self { - Self { - allow: allow.into_iter().map(Into::into).collect(), - } - } - - /// Whether this scope lists no sources — in which case it denies all - /// source-attributed content. See the type docs for why that is the - /// fail-closed reading and not "unrestricted". - pub fn is_empty(&self) -> bool { - self.allow.is_empty() - } - - /// Whether `source_id` is in scope, using the engine's equality-or-prefix - /// rule. - /// - /// ``` - /// use openhuman_core::openhuman::memory::api::provider::types::SourceScope; - /// - /// let scope = SourceScope::new(["src-abc"]); - /// assert!(scope.allows_source_id("src-abc")); - /// assert!(scope.allows_source_id("mem_src:src-abc:item-1")); - /// assert!(!scope.allows_source_id("src-xyz")); - /// - /// // An empty scope denies everything. - /// assert!(!SourceScope::default().allows_source_id("src-abc")); - /// ``` - pub fn allows_source_id(&self, source_id: &str) -> bool { - self.allow.iter().any(|allowed| { - source_id == allowed || source_id.starts_with(&format!("mem_src:{allowed}:")) - }) - } -} - -/// One unit of content handed to [`crate::openhuman::memory::api::provider::MemoryIngest`]. -/// -/// The driver owns chunking, embedding, and persistence — this type carries -/// only what the driver cannot know: where the content came from, when, who it -/// belongs to, and how far it may be trusted. -#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)] -pub struct IngestItem { - /// Target namespace; `None` means the driver's default namespace. - #[serde(default)] - pub namespace: Option, - /// Concrete upstream provider the content came from. - pub source: DataSource, - /// Stable logical id for the ingestion group (channel id, thread id, doc - /// id). This is the dedupe key, not a display value. - pub source_id: String, - /// Account or user the content belongs to; empty for anonymous/system - /// sources. - #[serde(default)] - pub owner: String, - /// Opaque pointer back to the raw source record, for citation and - /// drill-down. - #[serde(default)] - pub source_ref: Option, - /// The content itself, already decoded to text. - pub content: String, - /// MIME type of [`Self::content`] when the caller knows it. - #[serde(default)] - pub mime: Option, - /// Event time used for ordering and tree placement; the driver substitutes - /// ingest time when absent. - #[serde(default)] - pub timestamp: Option>, - /// Labels carried through from the source. Ingest does not interpret them. - #[serde(default)] - pub tags: Vec, - /// Provenance taint. The **host** stamps this; a driver must persist what it - /// is given and must never assign or upgrade it. - #[serde(default)] - pub taint: MemoryTaint, - /// Overrides `source_id` for on-disk path grouping only; `source_id` - /// remains the dedupe key. - #[serde(default)] - pub path_scope: Option, -} - -/// What an ingest call actually persisted. -/// -/// Counts rather than content, so the caller can report progress and detect a -/// silently-dropping driver without holding the written material in memory. -#[derive(Clone, Debug, Default, PartialEq, Eq, Serialize, Deserialize)] -pub struct IngestOutcome { - /// Units the driver newly persisted. - pub written: u32, - /// Units the driver recognised as already present and skipped. - pub skipped: u32, - /// Driver-assigned ids for the written units, when the driver exposes them. - /// May be empty even when [`Self::written`] is non-zero — an external - /// backend is not obliged to surface its internal ids. - #[serde(default)] - pub ids: Vec, -} - -/// One line of the portability stream. -/// -/// Export and import are defined over records rather than bytes so the contract -/// stays free of an async runtime and of any streaming abstraction: the host -/// adapter turns a page of records into NDJSON (and back) at the transport -/// boundary. -/// -/// [`Self::kind`] is a driver-defined string rather than an enum. A backend has -/// record kinds this crate has never heard of, and a migration between two -/// backends must round-trip them untouched rather than drop what it cannot -/// classify. -#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)] -pub struct ExportRecord { - /// Driver-defined record kind (e.g. `entry`, `document`, `chunk`). - pub kind: String, - /// Driver-assigned id, unique within [`Self::kind`]. - pub id: String, - /// Owning namespace, when the record has one. - #[serde(default)] - pub namespace: Option, - /// Provenance taint of the record's content. Preserved across - /// export → import; an importing driver must not re-stamp it. - #[serde(default)] - pub taint: MemoryTaint, - /// The record body, in the exporting driver's own shape. - pub payload: serde_json::Value, -} - -/// One page of an export, plus the cursor that continues it. -/// -/// Paging (rather than a stream) keeps [`crate::openhuman::memory::api::provider::MemoryPortability`] -/// object-safe and runtime-agnostic while still bounding memory: the caller -/// decides the page size and drives the loop. -#[derive(Clone, Debug, Default, PartialEq, Serialize, Deserialize)] -pub struct ExportPage { - /// Records in this page. May be empty on the final page. - pub records: Vec, - /// Opaque cursor to pass to the next call. `None` means the export is - /// complete — this, not an empty [`Self::records`], is the terminator. - #[serde(default)] - pub next_cursor: Option, -} - -/// What an import call actually accepted. -#[derive(Clone, Debug, Default, PartialEq, Eq, Serialize, Deserialize)] -pub struct ImportOutcome { - /// Records written. - pub imported: u32, - /// Records recognised as already present and skipped. - pub skipped: u32, - /// Records rejected. A non-zero value with an empty [`Self::errors`] is a - /// driver bug: a rejection the operator cannot diagnose. - pub failed: u32, - /// Operator-facing reasons for the failures, bounded by the driver. Must - /// not contain record content or credentials — this is logged. - #[serde(default)] - pub errors: Vec, -} - -/// Identity of an entity in the driver's index. -#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)] -pub struct EntityRef { - /// Canonical, driver-stable entity id. - pub id: String, - /// Entity kind as a wire string (`person`, `organization`, `topic`, …). - /// A string rather than an enum because the taxonomy is the driver's, and a - /// kind this build does not recognise must still round-trip. - pub kind: String, - /// Display name. - pub name: String, -} - -/// An entity together with its recency/frequency signals. -#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)] -pub struct EntityHit { - /// The entity itself. - pub entity: EntityRef, - /// Driver-computed hotness, higher is hotter. Not normalised across - /// drivers — compare within one driver's results only. - pub hotness: f64, - /// Number of times the entity was observed. - pub mentions: u32, -} - -/// Identity of a captured snapshot. -/// -/// The engine's own snapshot type additionally carries the git commit SHA and -/// ledger trailers that back it; those are implementation, so the contract -/// exposes only the identity and the counts a caller can act on. -#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)] -pub struct SnapshotRef { - /// Driver-stable snapshot id. - pub id: String, - /// Logical source this snapshot covers. - pub source_id: String, - /// Human-readable source label at capture time. - #[serde(default)] - pub label: String, - /// Number of items materialised into the snapshot. - pub item_count: u32, - /// Capture time in milliseconds since the Unix epoch. - pub taken_at_ms: i64, -} - -/// What happened to one item between two snapshots. -/// -/// Wire strings are identical to the engine's `memory::diff::types::ChangeKind` -/// so the embedded adapter maps rather than translates. -#[derive(Clone, Copy, Debug, PartialEq, Eq, Serialize, Deserialize)] -#[serde(rename_all = "snake_case")] -pub enum ChangeKind { - /// Present in the later snapshot only. - Added, - /// Present in the earlier snapshot only. - Removed, - /// Present in both, with differing content. - Modified, -} - -impl ChangeKind { - /// Stable wire string. - pub fn as_str(self) -> &'static str { - match self { - Self::Added => "added", - Self::Removed => "removed", - Self::Modified => "modified", - } - } -} - -/// A single item-level change inside a [`DiffReport`]. -/// -/// Item identity is the item id, never the title, so a rename reports as a -/// removal plus an addition rather than a modification. -#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)] -pub struct SourceChange { - /// Stable item id. - pub item_id: String, - /// Display title, or the id when the driver has no better label. - #[serde(default)] - pub title: String, - /// What kind of change occurred. - pub kind: ChangeKind, - /// Content hash on the earlier side; absent for an addition. - #[serde(default)] - pub old_content_hash: Option, - /// Content hash on the later side; absent for a removal. - #[serde(default)] - pub new_content_hash: Option, -} - -/// The result of diffing one source between two snapshots. -#[derive(Clone, Debug, Default, PartialEq, Eq, Serialize, Deserialize)] -pub struct DiffReport { - /// Source this diff covers. - pub source_id: String, - /// Baseline snapshot id; `None` for a first-ever diff, where everything is - /// an addition. - #[serde(default)] - pub from_snapshot_id: Option, - /// Target snapshot id. - pub to_snapshot_id: String, - /// Items added. - pub added: u32, - /// Items removed. - pub removed: u32, - /// Items modified. - pub modified: u32, - /// Items present and unchanged. - pub unchanged: u32, - /// Per-item changes. May be truncated by the driver; the counts above are - /// authoritative. - #[serde(default)] - pub changes: Vec, -} - -/// One item handed to [`crate::openhuman::memory::api::provider::MemorySourceSink`] by the host's sync -/// machinery. -/// -/// The host owns credentials, scheduling, and fetching; the driver owns storage -/// and indexing. This type is the whole of what crosses that line. -#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)] -pub struct SourceItem { - /// Stable per-source item id. Dedupe key; not a display value. - pub item_id: String, - /// Display title. - #[serde(default)] - pub title: String, - /// Item body, already decoded to text. - pub content: String, - /// MIME type of [`Self::content`] when known. - #[serde(default)] - pub mime: Option, - /// Canonical URL back to the item, when it has one. - #[serde(default)] - pub url: Option, - /// Upstream last-modified time in milliseconds since the Unix epoch. - #[serde(default)] - pub updated_at_ms: Option, - /// Labels carried through from the source. - #[serde(default)] - pub tags: Vec, -} - -/// Outcome of one maintenance operation. -/// -/// A single shape covers reembed, compact, consolidate, and doctor because the -/// caller does the same thing with all four: report progress and surface -/// findings. A per-operation result type would multiply the contract surface -/// without giving any caller more to act on. -#[derive(Clone, Debug, Default, PartialEq, Eq, Serialize, Deserialize)] -pub struct MaintenanceReport { - /// Which operation ran (`reembed`, `compact`, `consolidate`, `doctor`). - pub operation: String, - /// Units the driver examined. - pub examined: u64, - /// Units the driver changed. Always `0` for `doctor`, which is read-only. - pub changed: u64, - /// Operator-facing findings and notes. Must not contain memory content or - /// credentials — this is logged and shown in status output. - #[serde(default)] - pub findings: Vec, -} - -#[cfg(test)] -#[path = "types_tests.rs"] -mod tests; diff --git a/src/openhuman/memory/api/provider/types_tests.rs b/src/openhuman/memory/api/provider/types_tests.rs deleted file mode 100644 index 390e59a24a..0000000000 --- a/src/openhuman/memory/api/provider/types_tests.rs +++ /dev/null @@ -1,133 +0,0 @@ -//! Tests for the contract-only value types. -//! -//! The focus is the two things a later slice can silently break: the -//! fail-closed reading of an empty [`SourceScope`], and the wire strings / -//! serde defaults that an out-of-process driver depends on. - -use super::*; - -#[test] -fn empty_source_scope_denies_every_source() { - let scope = SourceScope::default(); - assert!(scope.is_empty()); - assert!(!scope.allows_source_id("src-abc")); - assert!(!scope.allows_source_id("mem_src:src-abc:item")); -} - -#[test] -fn source_scope_matches_exact_id_and_mem_src_prefix() { - let scope = SourceScope::new(["src-abc", "src-def"]); - - assert!(scope.allows_source_id("src-abc")); - assert!(scope.allows_source_id("src-def")); - assert!(scope.allows_source_id("mem_src:src-abc:item-1")); - assert!(scope.allows_source_id("mem_src:src-def:nested:item")); - - assert!(!scope.allows_source_id("src-xyz")); - assert!(!scope.allows_source_id("mem_src:src-xyz:item-1")); -} - -#[test] -fn source_scope_prefix_requires_the_trailing_separator() { - // `src-abc` must not smear onto `src-abcdef`: the engine's SQL binds - // `mem_src:{id}:` including the trailing colon, so a longer id that merely - // starts with an allowed one is out of scope. - let scope = SourceScope::new(["src-abc"]); - assert!(!scope.allows_source_id("mem_src:src-abcdef:item")); - assert!(!scope.allows_source_id("src-abcdef")); -} - -#[test] -fn change_kind_wire_strings_match_the_engine() { - // These strings are shared with `memory::diff::types::ChangeKind`, so the - // embedded adapter maps rather than translates. Changing one is a contract - // major bump. - for (kind, expected) in [ - (ChangeKind::Added, "added"), - (ChangeKind::Removed, "removed"), - (ChangeKind::Modified, "modified"), - ] { - assert_eq!(kind.as_str(), expected); - assert_eq!( - serde_json::to_value(kind).expect("serialize change kind"), - serde_json::Value::String(expected.to_string()), - ); - } -} - -#[test] -fn export_page_terminates_on_absent_cursor_not_empty_records() { - let page = ExportPage::default(); - assert!(page.records.is_empty()); - assert!(page.next_cursor.is_none()); - - // An empty page with a cursor is a legitimate mid-export state, so callers - // must not treat "no records" as the terminator. - let midway = ExportPage { - records: Vec::new(), - next_cursor: Some("cursor-2".to_string()), - }; - assert!(midway.next_cursor.is_some()); -} - -#[test] -fn export_record_round_trips_taint_and_opaque_payload() { - let record = ExportRecord { - kind: "vendor_specific_kind".to_string(), - id: "rec-1".to_string(), - namespace: Some("global".to_string()), - taint: MemoryTaint::ExternalSync, - payload: serde_json::json!({ "anything": [1, 2, 3] }), - }; - - let json = serde_json::to_string(&record).expect("serialize record"); - let back: ExportRecord = serde_json::from_str(&json).expect("deserialize record"); - - assert_eq!(back, record); - assert_eq!(back.taint, MemoryTaint::ExternalSync); -} - -#[test] -fn ingest_item_deserializes_from_the_minimal_body() { - // Every optional field carries `#[serde(default)]`, so a caller that knows - // only source, id, and content can still build a valid request. - let item: IngestItem = serde_json::from_value(serde_json::json!({ - "source": "notion", - "source_id": "page-1", - "content": "hello", - })) - .expect("deserialize minimal ingest item"); - - assert_eq!(item.source, DataSource::Notion); - assert_eq!(item.namespace, None); - assert_eq!(item.owner, ""); - assert!(item.tags.is_empty()); - // Provenance defaults to the conservative-for-writes `Internal`; the host - // guard overrides it explicitly on every sync path. - assert_eq!(item.taint, MemoryTaint::Internal); -} - -#[test] -fn maintenance_report_defaults_to_a_clean_read_only_run() { - let report = MaintenanceReport { - operation: "doctor".to_string(), - ..MaintenanceReport::default() - }; - assert_eq!(report.changed, 0); - assert!(report.findings.is_empty()); -} - -#[test] -fn diff_report_expresses_a_first_ever_diff_without_a_sentinel() { - let report = DiffReport { - source_id: "src-abc".to_string(), - from_snapshot_id: None, - to_snapshot_id: "snap-1".to_string(), - added: 3, - ..DiffReport::default() - }; - - let json = serde_json::to_value(&report).expect("serialize diff report"); - assert_eq!(json["from_snapshot_id"], serde_json::Value::Null); - assert_eq!(json["added"], 3); -} diff --git a/src/openhuman/memory/api/recall.rs b/src/openhuman/memory/api/recall.rs deleted file mode 100644 index 20ef1ad6c2..0000000000 --- a/src/openhuman/memory/api/recall.rs +++ /dev/null @@ -1,157 +0,0 @@ -//! Recall filter contracts — the borrowed engine form and the owned -//! contract/wire form, kept side by side so they cannot drift. -//! -//! ## Why there are two -//! -//! [`RecallOpts`] is the historical, engine-facing shape: it borrows its string -//! filters so a hot retrieval path allocates nothing. That makes it unusable as -//! a contract type in two independent ways — it derives no serde impls, so it -//! cannot be a `POST /v1/memory/recall` request body, and its lifetime -//! parameter would have to be threaded through every `#[async_trait]` recall -//! method, which destroys the object safety the whole driver model rests on. -//! -//! [`OwnedRecallOpts`] is the answer: the same five fields, owned, serde- -//! derived. Contract and wire paths use the owned form; the engine path keeps -//! the borrowed one and converts at the boundary via -//! `RecallOpts::from(&owned)`, which is zero-copy for the string fields. -//! -//! ## Field parity is the contract -//! -//! A field added to one form and not the other is a silent contract hole: the -//! wire would accept a filter the engine never applies, or the engine would -//! offer a filter no remote driver can be told about. Two defences are in -//! place, and both must stay: -//! -//! 1. Both [`From`] impls **exhaustively destructure** their source, so adding -//! a field to either struct without handling it fails to compile. -//! 2. `owned_and_borrowed_recall_opts_have_identical_fields` in -//! `recall_tests.rs` round-trips a fully non-default value through both -//! directions, so a field that is merely *dropped* during conversion fails -//! the test. -//! -//! Both types live in this module (rather than in `types.rs`) precisely so the -//! pair is read and edited together. They are re-exported from -//! [`crate::openhuman::memory::api::types`], so every historical `types::RecallOpts` path — including -//! the engine crate's `crate::openhuman::memory::engine::types::` alias — keeps resolving. - -use serde::{Deserialize, Serialize}; - -use crate::openhuman::memory::api::types::MemoryCategory; - -/// Optional filters for recall — the **borrowed, engine-facing** form. -/// -/// Borrows its string filters so an engine call path can pass slices of a -/// caller-owned request without allocating. It is deliberately *not* -/// serializable and deliberately *not* used in the driver contract: a lifetime -/// parameter cannot travel through an object-safe `#[async_trait]` method, and -/// a borrowed struct cannot be a request body. -/// -/// Use [`OwnedRecallOpts`] for anything that crosses a trait object or the -/// wire, and convert at the boundary with the [`From`] impl below. The two -/// types carry the same fields; a field added to one and not the other is a -/// silent contract hole, which -/// `owned_and_borrowed_recall_opts_have_identical_fields` exists to catch. -#[derive(Debug, Default, Clone)] -pub struct RecallOpts<'a> { - /// Restrict recall to this namespace; `None` falls back to [`crate::openhuman::memory::api::types::GLOBAL_NAMESPACE`]. - pub namespace: Option<&'a str>, - /// Restrict recall to entries of this category. - pub category: Option, - /// Restrict recall to entries scoped to this session. - pub session_id: Option<&'a str>, - /// Drop hits scoring below this threshold (typically 0.0–1.0). - pub min_score: Option, - /// When `true`, include conversational hits from other sessions in the same - /// workspace alongside the namespace recall. - pub cross_session: bool, -} - -/// Optional filters for recall — the **owned, contract-facing** form. -/// -/// This is the type the driver contract and the JSON wire protocol use. It -/// exists because [`RecallOpts`] cannot serve either role: -/// -/// - it derives no `Serialize`/`Deserialize`, so it cannot be a -/// `POST /v1/memory/recall` request body; -/// - it carries a borrow lifetime, which would have to be threaded through -/// every `#[async_trait]` recall method and destroys object safety at the -/// `dyn` boundary the whole driver model rests on. -/// -/// The borrowed form stays for engine-internal use so the embedded driver's -/// hot path allocates nothing: build the owned value once at the contract -/// boundary, then hand `RecallOpts::from(&owned)` down. -/// -/// Every field is `#[serde(default)]` so a minimal request body — even `{}` — -/// deserializes to the same value as [`Default::default`]. -#[derive(Debug, Default, Clone, PartialEq, Serialize, Deserialize)] -pub struct OwnedRecallOpts { - /// Restrict recall to this namespace; `None` falls back to [`crate::openhuman::memory::api::types::GLOBAL_NAMESPACE`]. - #[serde(default, skip_serializing_if = "Option::is_none")] - pub namespace: Option, - /// Restrict recall to entries of this category. - #[serde(default, skip_serializing_if = "Option::is_none")] - pub category: Option, - /// Restrict recall to entries scoped to this session. - #[serde(default, skip_serializing_if = "Option::is_none")] - pub session_id: Option, - /// Drop hits scoring below this threshold (typically 0.0–1.0). - #[serde(default, skip_serializing_if = "Option::is_none")] - pub min_score: Option, - /// When `true`, include conversational hits from other sessions in the same - /// workspace alongside the namespace recall. - #[serde(default)] - pub cross_session: bool, -} - -impl<'a> From<&'a OwnedRecallOpts> for RecallOpts<'a> { - /// Borrows the owned form for an engine call. Zero-copy for the two string - /// fields; [`MemoryCategory`] is cloned because it owns a `String` in its - /// [`MemoryCategory::Custom`] variant and [`RecallOpts`] holds it by value. - /// - /// Exhaustively destructures the source so adding a field to - /// [`OwnedRecallOpts`] without handling it here is a compile error. - fn from(owned: &'a OwnedRecallOpts) -> Self { - let OwnedRecallOpts { - namespace, - category, - session_id, - min_score, - cross_session, - } = owned; - RecallOpts { - namespace: namespace.as_deref(), - category: category.clone(), - session_id: session_id.as_deref(), - min_score: *min_score, - cross_session: *cross_session, - } - } -} - -impl From> for OwnedRecallOpts { - /// Takes ownership of a borrowed form — the direction a transport adapter - /// needs when turning an engine-shaped call into a request body. - /// - /// Exhaustively destructures the source for the same reason as the inverse - /// impl. - fn from(borrowed: RecallOpts<'_>) -> Self { - let RecallOpts { - namespace, - category, - session_id, - min_score, - cross_session, - } = borrowed; - OwnedRecallOpts { - namespace: namespace.map(str::to_string), - category, - session_id: session_id.map(str::to_string), - min_score, - cross_session, - } - } -} - -#[cfg(test)] -#[path = "recall_tests.rs"] -mod tests; diff --git a/src/openhuman/memory/api/recall_tests.rs b/src/openhuman/memory/api/recall_tests.rs deleted file mode 100644 index 5adf31754c..0000000000 --- a/src/openhuman/memory/api/recall_tests.rs +++ /dev/null @@ -1,158 +0,0 @@ -//! Unit tests for the recall filter contracts in [`super`]. -//! -//! The load-bearing test here is -//! `owned_and_borrowed_recall_opts_have_identical_fields`: it is the runtime -//! half of the field-parity defence described in the module docs (the compile -//! half being the exhaustive destructuring inside both `From` impls). - -use super::*; -use serde_json::json; - -/// Every field set to a non-default value, so a conversion that silently drops -/// one is visible. -fn fully_populated_owned() -> OwnedRecallOpts { - OwnedRecallOpts { - namespace: Some("projects".to_string()), - category: Some(MemoryCategory::Custom("field_notes".to_string())), - session_id: Some("session-42".to_string()), - min_score: Some(0.75), - cross_session: true, - } -} - -#[test] -fn owned_and_borrowed_recall_opts_have_identical_fields() { - let owned = fully_populated_owned(); - - // Owned → borrowed. Destructured exhaustively so a new field on - // `RecallOpts` fails to compile here rather than silently going unchecked. - let borrowed = RecallOpts::from(&owned); - let RecallOpts { - namespace, - category, - session_id, - min_score, - cross_session, - } = borrowed.clone(); - assert_eq!(namespace, Some("projects")); - assert_eq!(category, Some(MemoryCategory::Custom("field_notes".into()))); - assert_eq!(session_id, Some("session-42")); - assert_eq!(min_score, Some(0.75)); - assert!(cross_session); - - // Borrowed → owned, and back to the value we started from. A field dropped - // in either direction fails this equality. - let round_tripped = OwnedRecallOpts::from(borrowed); - assert_eq!(round_tripped, owned); -} - -#[test] -fn borrowed_view_is_zero_copy_over_the_owned_strings() { - let owned = fully_populated_owned(); - let borrowed = RecallOpts::from(&owned); - - // The borrowed form points *into* the owned value rather than at a copy; - // that is the whole reason the borrowed form survives. - assert_eq!( - borrowed.namespace.unwrap().as_ptr(), - owned.namespace.as_deref().unwrap().as_ptr() - ); - assert_eq!( - borrowed.session_id.unwrap().as_ptr(), - owned.session_id.as_deref().unwrap().as_ptr() - ); -} - -#[test] -fn owned_recall_opts_defaults_match_borrowed_defaults() { - let owned = OwnedRecallOpts::default(); - let borrowed = RecallOpts::from(&owned); - - assert!(borrowed.namespace.is_none()); - assert!(borrowed.category.is_none()); - assert!(borrowed.session_id.is_none()); - assert!(borrowed.min_score.is_none()); - assert!(!borrowed.cross_session); - - // And the borrowed default converts back to the owned default. - assert_eq!(OwnedRecallOpts::from(RecallOpts::default()), owned); -} - -#[test] -fn owned_recall_opts_serde_round_trips_every_field() { - let owned = fully_populated_owned(); - let encoded = serde_json::to_value(&owned).unwrap(); - - assert_eq!( - encoded, - json!({ - "namespace": "projects", - "category": "custom:field_notes", - "session_id": "session-42", - "min_score": 0.75, - "cross_session": true - }) - ); - - let decoded: OwnedRecallOpts = serde_json::from_value(encoded).unwrap(); - assert_eq!(decoded, owned); -} - -#[test] -fn empty_recall_body_deserializes_to_the_default() { - // A minimal `POST /v1/memory/recall` body must be accepted: every field is - // `#[serde(default)]`. - let decoded: OwnedRecallOpts = serde_json::from_value(json!({})).unwrap(); - assert_eq!(decoded, OwnedRecallOpts::default()); -} - -#[test] -fn partial_recall_body_leaves_unmentioned_fields_at_default() { - let decoded: OwnedRecallOpts = - serde_json::from_value(json!({ "namespace": "global" })).unwrap(); - assert_eq!(decoded.namespace.as_deref(), Some("global")); - assert!(decoded.category.is_none()); - assert!(decoded.session_id.is_none()); - assert!(decoded.min_score.is_none()); - assert!(!decoded.cross_session); -} - -/// The wire form omits absent filters rather than emitting explicit nulls. -/// -/// `OwnedRecallOpts` is the body of `POST /v1/memory/recall`, which the spec -/// describes as an optional-filters bag. Emitting `"namespace": null` for every -/// unset filter is valid JSON but forces a backend to distinguish "absent" from -/// "explicitly null" for no gain. Pinned here because changing the emitted shape -/// after a driver has shipped is observable to any backend that draws that -/// distinction. -#[test] -fn absent_recall_filters_are_omitted_from_the_wire_form() { - let json = serde_json::to_value(OwnedRecallOpts::default()).expect("serialize"); - assert_eq!( - json, - serde_json::json!({ "cross_session": false }), - "unset optional filters must be omitted, not serialized as null" - ); - - let populated = OwnedRecallOpts { - namespace: Some("work".into()), - ..Default::default() - }; - let json = serde_json::to_value(&populated).expect("serialize"); - assert_eq!( - json, - serde_json::json!({ "namespace": "work", "cross_session": false }) - ); -} - -/// Omitting a filter and sending it as `null` must both decode to `None`, so a -/// backend built against either spelling keeps working. -#[test] -fn omitted_and_explicit_null_recall_filters_both_decode_to_none() { - let omitted: OwnedRecallOpts = serde_json::from_str("{}").expect("decode {}"); - let explicit: OwnedRecallOpts = - serde_json::from_str(r#"{"namespace":null,"category":null,"session_id":null,"min_score":null,"cross_session":false}"#) - .expect("decode explicit nulls"); - assert_eq!(omitted, explicit); - assert_eq!(omitted, OwnedRecallOpts::default()); -} diff --git a/src/openhuman/memory/api/tool_memory.rs b/src/openhuman/memory/api/tool_memory.rs deleted file mode 100644 index 8699d6d9f1..0000000000 --- a/src/openhuman/memory/api/tool_memory.rs +++ /dev/null @@ -1,162 +0,0 @@ -//! Domain types for the tool-scoped memory layer. -//! -//! A [`ToolMemoryRule`] is a durable, actionable instruction attached to a -//! specific tool (e.g. `email`, `shell`, `web_search`). Unlike per-tool -//! effectiveness statistics, these rules capture **guidance** — corrections, -//! safety constraints, and learned operational rules that the agent should -//! obey when considering or invoking that tool. -//! -//! Rules carry a [`ToolMemoryPriority`] level so the retrieval pipeline can -//! distinguish safety-critical instructions from soft suggestions: -//! -//! - [`ToolMemoryPriority::Critical`] — pinned into the system prompt and -//! therefore not subject to mid-session context compression. -//! - [`ToolMemoryPriority::High`] — surfaced alongside critical rules at -//! tool-selection time. -//! - [`ToolMemoryPriority::Normal`] — available on demand via the recall -//! APIs, but not eagerly injected. -//! -//! These are pure data contracts: the snake_case wire strings -//! (`normal`/`high`/`critical`, `user_explicit`/`post_turn`/`programmatic`) -//! are preserved verbatim from OpenHuman so serialized rules stay -//! byte-compatible across the boundary. - -use serde::{Deserialize, Serialize}; - -/// Priority/criticality of a [`ToolMemoryRule`]. -/// -/// Used by both storage (to filter what is pinned into the system prompt) -/// and retrieval (to sort high-priority guidance ahead of advisory notes). -#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Serialize, Deserialize)] -#[serde(rename_all = "snake_case")] -#[derive(Default)] -pub enum ToolMemoryPriority { - /// Soft suggestion — surfaced on demand, not eagerly injected. - #[default] - Normal, - /// Important guidance — eagerly injected at tool-selection time. - High, - /// Safety-critical rule — pinned into the (compression-resistant) - /// system prompt so it survives the agent's full session. - Critical, -} - -impl ToolMemoryPriority { - /// True for priorities that must be eagerly surfaced to the agent - /// (Critical/High rules are both pinned into the system prompt and - /// prefetched at session start, so they survive context compression). - pub fn is_eager(self) -> bool { - matches!(self, Self::Critical | Self::High) - } -} - -/// Where a [`ToolMemoryRule`] originated from. -/// -/// Recorded for provenance and so consumers (UI / debugging) can tell user -/// edicts apart from auto-captured observations. -#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] -#[serde(rename_all = "snake_case")] -#[derive(Default)] -pub enum ToolMemorySource { - /// User explicitly asked the agent to remember this rule. - UserExplicit, - /// Captured automatically from a post-turn observation (tool failure, - /// repeated correction, etc.). - PostTurn, - /// Written by another subsystem (e.g. an integration provisioner). - #[default] - Programmatic, -} - -/// A single tool-scoped memory rule. -/// -/// Stored under the `tool-{tool_name}` namespace as an entry keyed by -/// `rule/{rule_id}`. The id is stable across updates so callers can -/// upsert by replaying the same id. -#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] -pub struct ToolMemoryRule { - /// Stable identifier within `(tool_name)`. Generated by callers via - /// [`ToolMemoryRule::generate_id`] when one is not supplied. - pub id: String, - /// Tool this rule applies to (e.g. `email`, `shell`). - pub tool_name: String, - /// Natural-language guidance that should reach the agent. - pub rule: String, - /// Criticality level for retrieval and compression behaviour. - #[serde(default)] - pub priority: ToolMemoryPriority, - /// Where this rule came from. - #[serde(default)] - pub source: ToolMemorySource, - /// Optional free-form tags for filtering (e.g. `safety`, `permission`). - #[serde(default)] - pub tags: Vec, - /// RFC3339 timestamp of when the rule was first written. - pub created_at: String, - /// RFC3339 timestamp of the last update. - pub updated_at: String, -} - -impl ToolMemoryRule { - /// Build a new rule with a freshly generated id and `created_at` / - /// `updated_at` set to "now". - pub fn new( - tool_name: impl Into, - rule: impl Into, - priority: ToolMemoryPriority, - source: ToolMemorySource, - ) -> Self { - let now = chrono::Utc::now().to_rfc3339(); - Self { - id: Self::generate_id(), - tool_name: tool_name.into(), - rule: rule.into(), - priority, - source, - tags: Vec::new(), - created_at: now.clone(), - updated_at: now, - } - } - - /// Generate a fresh, opaque rule id. - /// - /// Each byte of a v4 UUID is encoded as two lowercase ASCII letters in - /// the `a..=p` range (one per nibble). The result is a separator-free, - /// digit-free token — deliberately shaped so it never trips a PII - /// boundary check when used as a storage key. - pub fn generate_id() -> String { - let mut id = String::with_capacity(33); - id.push('r'); - for byte in uuid::Uuid::new_v4().as_bytes() { - id.push((b'a' + (byte >> 4)) as char); - id.push((b'a' + (byte & 0x0f)) as char); - } - id - } - - /// Storage key used inside the tool namespace. - pub fn storage_key(id: &str) -> String { - format!("rule/{id}") - } -} - -/// Namespace string for a given tool. Trimmed and lower-cased so callers -/// can pass user-supplied tool names without leaking whitespace into -/// downstream queries. -/// -/// The `tool-` prefix is intentionally distinct from `global`, `skill-…` -/// and `tool_effectiveness` so retrieval and clearing operations can -/// reason about the namespace without ambiguity. Always build the -/// namespace through this helper — never hard-code the `tool-` format. -/// -/// The engine crate's `ToolMemoryStore::put_rule` applies the same -/// normalization to the stored rule so namespace and display/grouping identity -/// cannot diverge. -pub fn tool_memory_namespace(tool_name: &str) -> String { - format!("tool-{}", tool_name.trim().to_lowercase()) -} - -#[cfg(test)] -#[path = "tool_memory_tests.rs"] -mod tests; diff --git a/src/openhuman/memory/api/tool_memory_tests.rs b/src/openhuman/memory/api/tool_memory_tests.rs deleted file mode 100644 index 821369eaed..0000000000 --- a/src/openhuman/memory/api/tool_memory_tests.rs +++ /dev/null @@ -1,128 +0,0 @@ -//! Tests for the tool-scoped memory domain types. - -use super::*; - -#[test] -fn priority_default_is_normal() { - assert_eq!(ToolMemoryPriority::default(), ToolMemoryPriority::Normal); -} - -#[test] -fn priority_ordering_puts_critical_above_high() { - assert!(ToolMemoryPriority::Critical > ToolMemoryPriority::High); - assert!(ToolMemoryPriority::High > ToolMemoryPriority::Normal); -} - -#[test] -fn priority_is_eager_for_high_and_critical_only() { - assert!(ToolMemoryPriority::Critical.is_eager()); - assert!(ToolMemoryPriority::High.is_eager()); - assert!(!ToolMemoryPriority::Normal.is_eager()); -} - -#[test] -fn priority_snake_case_serde() { - assert_eq!( - serde_json::to_string(&ToolMemoryPriority::Critical).unwrap(), - "\"critical\"" - ); - assert_eq!( - serde_json::to_string(&ToolMemoryPriority::Normal).unwrap(), - "\"normal\"" - ); -} - -#[test] -fn source_snake_case_serde() { - assert_eq!( - serde_json::to_string(&ToolMemorySource::UserExplicit).unwrap(), - "\"user_explicit\"" - ); - assert_eq!( - serde_json::to_string(&ToolMemorySource::PostTurn).unwrap(), - "\"post_turn\"" - ); - assert_eq!( - serde_json::to_string(&ToolMemorySource::Programmatic).unwrap(), - "\"programmatic\"" - ); -} - -#[test] -fn source_default_is_programmatic() { - assert_eq!(ToolMemorySource::default(), ToolMemorySource::Programmatic); -} - -#[test] -fn rule_new_fills_id_and_timestamps() { - let rule = ToolMemoryRule::new( - "email", - "never email Sarah", - ToolMemoryPriority::Critical, - ToolMemorySource::UserExplicit, - ); - assert!(!rule.id.is_empty()); - assert_eq!(rule.tool_name, "email"); - assert_eq!(rule.rule, "never email Sarah"); - assert_eq!(rule.priority, ToolMemoryPriority::Critical); - assert_eq!(rule.source, ToolMemorySource::UserExplicit); - assert!(rule.created_at == rule.updated_at); -} - -#[test] -fn rule_generate_id_produces_unique_values() { - let a = ToolMemoryRule::generate_id(); - let b = ToolMemoryRule::generate_id(); - assert_ne!(a, b); - assert!(a.starts_with('r')); - assert!(a[1..].chars().all(|c| matches!(c, 'a'..='p'))); -} - -#[test] -fn generated_rule_ids_are_safe_memory_document_keys() { - // Generated ids must be free of digits and separators so the resulting - // storage key never resembles PII (phone numbers, ids, etc.) to a - // boundary check downstream. - for _ in 0..128 { - let id = ToolMemoryRule::generate_id(); - assert!( - id.chars().all(|ch| ch.is_ascii_lowercase()), - "generated id should avoid PII-shaped digits and separators: {id}" - ); - let key = ToolMemoryRule::storage_key(&id); - assert!( - key.bytes().all(|b| b == b'/' || b.is_ascii_lowercase()), - "generated storage key should not contain PII-shaped bytes: {key}" - ); - } -} - -#[test] -fn rule_storage_key_uses_rule_prefix() { - assert_eq!(ToolMemoryRule::storage_key("abc"), "rule/abc"); -} - -#[test] -fn rule_serde_roundtrip_preserves_fields() { - let rule = ToolMemoryRule { - id: "id-1".into(), - tool_name: "shell".into(), - rule: "never run sudo".into(), - priority: ToolMemoryPriority::High, - source: ToolMemorySource::PostTurn, - tags: vec!["safety".into()], - created_at: "2026-05-11T00:00:00Z".into(), - updated_at: "2026-05-11T00:00:01Z".into(), - }; - let json = serde_json::to_string(&rule).unwrap(); - let back: ToolMemoryRule = serde_json::from_str(&json).unwrap(); - assert_eq!(back, rule); -} - -#[test] -fn namespace_uses_tool_prefix_and_trims_whitespace() { - assert_eq!(tool_memory_namespace("email"), "tool-email"); - assert_eq!(tool_memory_namespace(" shell "), "tool-shell"); - assert_eq!(tool_memory_namespace("Send_Email"), "tool-send_email"); - assert_eq!(tool_memory_namespace("WebSearch"), "tool-websearch"); -} diff --git a/src/openhuman/memory/api/traits.rs b/src/openhuman/memory/api/traits.rs deleted file mode 100644 index c96e0b9bd8..0000000000 --- a/src/openhuman/memory/api/traits.rs +++ /dev/null @@ -1,157 +0,0 @@ -//! The high-level [`Memory`] trait every storage backend implements. -//! -//! Ported from OpenHuman's `memory::traits`. Backend-specific escape hatches -//! (e.g. raw SQLite connection access) are intentionally omitted here so the -//! trait stays storage-agnostic; concrete backends expose those via their own -//! inherent methods. -//! -//! ## Contract notes -//! -//! - Every method returns `anyhow::Result<_>` rather than a typed error: this -//! trait is a stable abstraction boundary over heterogeneous backends -//! (SQLite, vector DB, in-memory, …), each with its own error domain, so -//! callers should treat a returned `Err` as opaque and log/propagate it -//! rather than match on its variant. Concrete backends document their own -//! failure modes (e.g. IO errors, malformed persisted rows) alongside their -//! inherent methods. -//! - None of these methods are specified to panic; a conforming implementation -//! should convert failures (invalid input, backend errors, poisoned locks) -//! into `Err` instead. -//! - [`Memory::store`] and [`Memory::store_with_taint`] are upserts keyed by -//! `(namespace, key)`: calling them again with the same key replaces the -//! prior entry rather than erroring or duplicating it. - -use async_trait::async_trait; - -use super::types::{MemoryCategory, MemoryEntry, MemoryTaint, NamespaceSummary, RecallOpts}; - -/// The core trait for memory storage and retrieval. -/// -/// Any persistence backend (SQLite, Postgres, vector DB, in-memory, …) should -/// implement this to participate in a TinyMemory-backed memory engine. -#[async_trait] -pub trait Memory: Send + Sync { - /// Returns the backend name (e.g. `"sqlite"`, `"vector"`, `"in_memory"`). - fn name(&self) -> &str; - - /// Stores a new memory entry or updates an existing one. - /// - /// Idempotent upsert keyed by `(namespace, key)`: calling this again with - /// the same `namespace`/`key` replaces the previous `content`, `category`, - /// and `session_id` rather than erroring or creating a duplicate. Entries - /// stored this way carry [`MemoryTaint::Internal`] (the default); use - /// [`Self::store_with_taint`] to persist content from an external source. - /// - /// # Errors - /// - /// Returns `Err` on any backend failure (IO, serialization, connection - /// loss); implementations must not panic on caller-controlled input. - async fn store( - &self, - namespace: &str, - key: &str, - content: &str, - category: MemoryCategory, - session_id: Option<&str>, - ) -> anyhow::Result<()>; - - /// Store an entry with explicit provenance taint. - /// - /// Sync paths ingesting third-party text MUST use this with - /// [`MemoryTaint::ExternalSync`]. The default implementation degrades to - /// [`Self::store`] for backends that do not yet persist taint — meaning it - /// silently drops the `taint` argument for any backend that has not - /// overridden this method. Backends whose durability/policy story depends - /// on taint being recorded MUST override this method rather than rely on - /// the default. - async fn store_with_taint( - &self, - namespace: &str, - key: &str, - content: &str, - category: MemoryCategory, - session_id: Option<&str>, - taint: MemoryTaint, - ) -> anyhow::Result<()> { - if taint != MemoryTaint::Internal { - anyhow::bail!("backend does not support taint-preserving storage"); - } - self.store(namespace, key, content, category, session_id) - .await - } - - /// Recalls memories matching a query using keyword or semantic search. - /// - /// `limit` caps the number of returned entries; `opts` narrows the search - /// by namespace, category, session, minimum score, and cross-session - /// inclusion (see [`RecallOpts`]). An empty or non-matching `query` should - /// yield `Ok(vec![])`, not an error. Result ordering is backend-defined - /// (typically most-relevant first) but callers must not assume a stable - /// order across backends. - async fn recall( - &self, - query: &str, - limit: usize, - opts: RecallOpts<'_>, - ) -> anyhow::Result>; - - /// Recall documents whose *vector* similarity alone meets a threshold. - /// - /// Returns `(key, content)` pairs, most-relevant first. Defaults to empty so - /// keyword-only / mock backends opt out; a backend that overrides this - /// should treat `min_vector_similarity` as an inclusive floor (hits scoring - /// strictly below it are dropped) and `limit` as a hard cap on the - /// returned count. - async fn recall_relevant_by_vector( - &self, - namespace: &str, - query: &str, - limit: usize, - min_vector_similarity: f64, - ) -> anyhow::Result> { - let _ = (namespace, query, limit, min_vector_similarity); - Ok(Vec::new()) - } - - /// Retrieves a specific entry by exact `(namespace, key)`. - /// - /// Returns `Ok(None)` — not `Err` — when no entry exists for the pair; - /// `Err` is reserved for backend failures. - async fn get(&self, namespace: &str, key: &str) -> anyhow::Result>; - - /// Lists entries, optionally scoped by namespace, category, and session. - /// - /// Each `Option` filter narrows the result set when `Some`; passing all - /// three as `None` lists every entry the backend holds. An empty result - /// set is `Ok(vec![])`, never an error. - async fn list( - &self, - namespace: Option<&str>, - category: Option<&MemoryCategory>, - session_id: Option<&str>, - ) -> anyhow::Result>; - - /// Deletes the entry for `(namespace, key)`. Returns whether it existed. - /// - /// Idempotent: forgetting an already-absent `(namespace, key)` returns - /// `Ok(false)` rather than erroring, so callers may call this - /// unconditionally without checking existence first. - async fn forget(&self, namespace: &str, key: &str) -> anyhow::Result; - - /// Lists all namespaces with aggregate stats for agent-side discovery. - /// - /// See [`NamespaceSummary`] for the per-namespace count and - /// last-updated timestamp returned. - async fn namespace_summaries(&self) -> anyhow::Result>; - - /// Total count of all entries in the backend, across all namespaces. - async fn count(&self) -> anyhow::Result; - - /// Health check on the underlying storage system. - /// - /// Returns `true` when the backend is reachable and able to serve - /// requests. Unlike the other methods this reports failure as `false` - /// rather than `Err`, so it is safe to call from a liveness probe without - /// error-handling boilerplate. - async fn health_check(&self) -> bool; -} diff --git a/src/openhuman/memory/api/tree.rs b/src/openhuman/memory/api/tree.rs deleted file mode 100644 index b6c6bd5576..0000000000 --- a/src/openhuman/memory/api/tree.rs +++ /dev/null @@ -1,212 +0,0 @@ -//! Domain types for the markdown time-based summary tree. -//! -//! Organises summaries as a time hierarchy: root → year → month → day → hour -//! (leaf). Ported from OpenHuman's `memory_tree/tree_runtime/types.rs`. - -use chrono::{DateTime, Datelike, Timelike, Utc}; -use serde::{Deserialize, Serialize}; -use std::path::PathBuf; - -/// Hierarchical level of a tree node. -#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, Serialize, Deserialize)] -#[serde(rename_all = "snake_case")] -pub enum NodeLevel { - /// Single tree root; aggregates all years. Wire string `"root"`. - Root, - /// One node per calendar year. Wire string `"year"`. - Year, - /// One node per calendar month. Wire string `"month"`. - Month, - /// One node per calendar day. Wire string `"day"`. - Day, - /// Leaf level; one node per hour, where raw content lands. Wire string `"hour"`. - Hour, -} - -impl NodeLevel { - /// Maximum number of tokens allowed at this level. - pub fn max_tokens(&self) -> u32 { - match self { - Self::Hour => 1_000, - Self::Day => 2_000, - Self::Month => 4_000, - Self::Year => 8_000, - Self::Root => 20_000, - } - } - - /// The level above this one in the hierarchy (`None` for root). - pub fn parent_level(&self) -> Option { - match self { - Self::Hour => Some(Self::Day), - Self::Day => Some(Self::Month), - Self::Month => Some(Self::Year), - Self::Year => Some(Self::Root), - Self::Root => None, - } - } - - /// True only for the leaf level (hour). - pub fn is_leaf(&self) -> bool { - matches!(self, Self::Hour) - } - - /// Parse a level string from YAML frontmatter. - pub fn from_str_label(s: &str) -> Option { - match s.trim().to_ascii_lowercase().as_str() { - "root" => Some(Self::Root), - "year" => Some(Self::Year), - "month" => Some(Self::Month), - "day" => Some(Self::Day), - "hour" => Some(Self::Hour), - _ => None, - } - } - - /// Label for display / frontmatter. - pub fn as_str(&self) -> &'static str { - match self { - Self::Root => "root", - Self::Year => "year", - Self::Month => "month", - Self::Day => "day", - Self::Hour => "hour", - } - } -} - -/// A single node in the summary tree. -#[derive(Debug, Clone, Serialize, Deserialize)] -pub struct TreeNode { - /// Path-style hierarchical id, e.g. `"2024/03/15/09"` or `"root"`. - pub node_id: String, - /// Namespace owning this tree (isolates independent trees). - pub namespace: String, - /// Hierarchical level this node sits at. - pub level: NodeLevel, - /// Id of the parent node; `None` only for the root. - pub parent_id: Option, - /// Rolled-up summary text for this node. - pub summary: String, - /// Estimated token count of [`Self::summary`]; bounded by [`NodeLevel::max_tokens`]. - pub token_count: u32, - /// Number of direct children rolled into this node. - pub child_count: u32, - /// Creation timestamp (UTC). - pub created_at: DateTime, - /// Last-update timestamp (UTC). - pub updated_at: DateTime, - /// Optional opaque metadata blob; omitted from serialization when absent. - #[serde(default, skip_serializing_if = "Option::is_none")] - pub metadata: Option, -} - -/// Metadata about an entire tree within a namespace. -#[derive(Debug, Clone, Serialize, Deserialize)] -pub struct TreeStatus { - /// Namespace the tree belongs to. - pub namespace: String, - /// Total number of nodes across all levels. - pub total_nodes: u64, - /// Number of populated levels (tree height). - pub depth: u32, - /// Timestamp of the earliest ingested entry, if any. - pub oldest_entry: Option>, - /// Timestamp of the most recent ingested entry, if any. - pub newest_entry: Option>, - /// When the tree was last (re)built or sealed. - pub last_run_at: Option>, -} - -/// Input for appending raw content to the ingestion buffer. -#[derive(Debug, Clone, Serialize, Deserialize)] -pub struct IngestRequest { - /// Target namespace to append content into. - pub namespace: String, - /// Raw content to buffer for summarization. - pub content: String, - /// Event time used to derive the hour leaf; defaults to ingestion time when absent. - #[serde(default)] - pub timestamp: Option>, - /// Optional structured metadata carried alongside the content. - #[serde(default, skip_serializing_if = "Option::is_none")] - pub metadata: Option, -} - -/// Result of a tree query at a specific node. -#[derive(Debug, Clone, Serialize, Deserialize)] -pub struct QueryResult { - /// The node addressed by the query. - pub node: TreeNode, - /// Direct children of [`Self::node`], for drill-down navigation. - pub children: Vec, -} - -/// Rough token estimate: ~4 characters per token. -pub fn estimate_tokens(text: &str) -> u32 { - u32::try_from(text.len().div_ceil(4)).unwrap_or(u32::MAX) -} - -/// Derive the parent node ID from a node ID. -pub fn derive_parent_id(node_id: &str) -> Option { - if node_id == "root" { - return None; - } - match node_id.rfind('/') { - Some(pos) => Some(node_id[..pos].to_string()), - None => Some("root".to_string()), - } -} - -/// Determine the `NodeLevel` from a node ID string. -pub fn level_from_node_id(node_id: &str) -> NodeLevel { - if node_id == "root" { - return NodeLevel::Root; - } - match node_id.matches('/').count() { - 0 => NodeLevel::Year, - 1 => NodeLevel::Month, - 2 => NodeLevel::Day, - _ => NodeLevel::Hour, - } -} - -/// Derive all ancestor node IDs from a timestamp (hour through root). -/// Returns `(hour_id, day_id, month_id, year_id, root_id)`. -pub fn derive_node_ids(ts: &DateTime) -> (String, String, String, String, String) { - let year = format!("{}", ts.year()); - let month = format!("{}/{:02}", ts.year(), ts.month()); - let day = format!("{}/{:02}/{:02}", ts.year(), ts.month(), ts.day()); - let hour = format!( - "{}/{:02}/{:02}/{:02}", - ts.year(), - ts.month(), - ts.day(), - ts.hour() - ); - (hour, day, month, year, "root".to_string()) -} - -/// Convert a node ID to a relative file path within the tree directory. -pub fn node_id_to_path(node_id: &str) -> PathBuf { - if node_id == "root" { - return PathBuf::from("root.md"); - } - if node_id.starts_with('/') - || node_id - .split('/') - .any(|part| part.is_empty() || !part.chars().all(|c| c.is_ascii_digit())) - { - return PathBuf::from("invalid"); - } - let level = level_from_node_id(node_id); - if level.is_leaf() { - PathBuf::from(format!("{node_id}.md")) - } else { - PathBuf::from(node_id).join("summary.md") - } -} - -#[cfg(test)] -#[path = "tree_tests.rs"] -mod tests; diff --git a/src/openhuman/memory/api/tree_tests.rs b/src/openhuman/memory/api/tree_tests.rs deleted file mode 100644 index bb9965fa46..0000000000 --- a/src/openhuman/memory/api/tree_tests.rs +++ /dev/null @@ -1,86 +0,0 @@ -//! Tests for the markdown time-tree node types. - -use super::*; -use chrono::TimeZone; -use std::path::PathBuf; - -#[test] -fn node_level_max_tokens() { - assert_eq!(NodeLevel::Hour.max_tokens(), 1_000); - assert_eq!(NodeLevel::Day.max_tokens(), 2_000); - assert_eq!(NodeLevel::Month.max_tokens(), 4_000); - assert_eq!(NodeLevel::Year.max_tokens(), 8_000); - assert_eq!(NodeLevel::Root.max_tokens(), 20_000); -} - -#[test] -fn node_level_parent_chain() { - assert_eq!(NodeLevel::Hour.parent_level(), Some(NodeLevel::Day)); - assert_eq!(NodeLevel::Day.parent_level(), Some(NodeLevel::Month)); - assert_eq!(NodeLevel::Month.parent_level(), Some(NodeLevel::Year)); - assert_eq!(NodeLevel::Year.parent_level(), Some(NodeLevel::Root)); - assert_eq!(NodeLevel::Root.parent_level(), None); -} - -#[test] -fn derive_parent_id_chain() { - assert_eq!(derive_parent_id("2024/03/15/14"), Some("2024/03/15".into())); - assert_eq!(derive_parent_id("2024/03/15"), Some("2024/03".into())); - assert_eq!(derive_parent_id("2024/03"), Some("2024".into())); - assert_eq!(derive_parent_id("2024"), Some("root".into())); - assert_eq!(derive_parent_id("root"), None); -} - -#[test] -fn level_from_node_id_all_levels() { - assert_eq!(level_from_node_id("root"), NodeLevel::Root); - assert_eq!(level_from_node_id("2024"), NodeLevel::Year); - assert_eq!(level_from_node_id("2024/03"), NodeLevel::Month); - assert_eq!(level_from_node_id("2024/03/15"), NodeLevel::Day); - assert_eq!(level_from_node_id("2024/03/15/14"), NodeLevel::Hour); -} - -#[test] -fn derive_node_ids_from_timestamp() { - let ts = Utc.with_ymd_and_hms(2024, 3, 15, 14, 30, 0).unwrap(); - let (hour, day, month, year, root) = derive_node_ids(&ts); - assert_eq!(hour, "2024/03/15/14"); - assert_eq!(day, "2024/03/15"); - assert_eq!(month, "2024/03"); - assert_eq!(year, "2024"); - assert_eq!(root, "root"); -} - -#[test] -fn node_id_to_path_mapping() { - assert_eq!(node_id_to_path("root"), PathBuf::from("root.md")); - assert_eq!(node_id_to_path("2024"), PathBuf::from("2024/summary.md")); - assert_eq!( - node_id_to_path("2024/03"), - PathBuf::from("2024/03/summary.md") - ); - assert_eq!( - node_id_to_path("2024/03/15/14"), - PathBuf::from("2024/03/15/14.md") - ); -} - -#[test] -fn estimate_tokens_rough() { - assert_eq!(estimate_tokens(""), 0); - assert_eq!(estimate_tokens("abcd"), 1); - assert_eq!(estimate_tokens(&"a".repeat(4000)), 1000); -} - -#[test] -fn node_level_roundtrip() { - for level in [ - NodeLevel::Root, - NodeLevel::Year, - NodeLevel::Month, - NodeLevel::Day, - NodeLevel::Hour, - ] { - assert_eq!(NodeLevel::from_str_label(level.as_str()), Some(level)); - } -} diff --git a/src/openhuman/memory/api/types.rs b/src/openhuman/memory/api/types.rs deleted file mode 100644 index f08da47ba9..0000000000 --- a/src/openhuman/memory/api/types.rs +++ /dev/null @@ -1,436 +0,0 @@ -//! Core public data contracts for the TinyMemory memory contract. -//! -//! These types are the stable surface shared across every layer (storage, -//! ingestion, retrieval, RPC). They are pure data — no storage side effects, -//! no interior mutability, freely `Clone`/`Send`/`Sync` — and are ported -//! faithfully from OpenHuman's `memory` and `memory_store` modules so wire -//! formats (snake_case enum strings, serde defaults) stay byte-compatible when -//! OpenHuman imports this crate. -//! -//! ## Wire-compatibility contract -//! -//! Every `#[serde(rename_all = "snake_case")]` enum here has its variant -//! strings persisted in on-disk indexes (SQLite columns, markdown frontmatter) -//! and/or sent over the RPC boundary. Renaming a variant, or a struct field -//! that lacks `#[serde(default)]`, is a breaking change for any host reading -//! previously-written data. When adding a field, prefer `#[serde(default)]` so -//! older persisted rows continue to deserialize. -//! -//! ## Fail-closed provenance -//! -//! [`MemoryTaint`] is the one field in this module with a safety-relevant -//! default: it decodes unknown/corrupt persisted strings as -//! [`MemoryTaint::ExternalSync`] rather than [`MemoryTaint::Internal`], so a -//! caller that forgets to persist taint, or an index that has drifted, fails -//! toward *more* restrictive tool-use policy rather than less. - -use serde::{Deserialize, Serialize}; - -/// The recall filter contracts live in [`crate::openhuman::memory::api::recall`] so the borrowed and -/// owned forms sit next to each other and cannot drift, and are re-exported -/// here so every historical `types::RecallOpts` path — including the engine -/// crate's `crate::openhuman::memory::engine::types::` alias — keeps resolving unchanged. -pub use crate::openhuman::memory::api::recall::{OwnedRecallOpts, RecallOpts}; - -/// Default namespace used when a caller passes no explicit namespace. -pub const GLOBAL_NAMESPACE: &str = "global"; - -/// Provenance / trust signal attached to a memory entry. -/// -/// Drives downstream policy — most importantly whether automation whose context -/// contains this content may invoke external-effect tools. Defaults to -/// [`MemoryTaint::Internal`] so legacy rows (no persisted taint column) and all -/// in-memory defaults are conservatively trusted as user-driven content. -/// -/// Sync paths that ingest text from third-party services (Gmail / Slack / -/// Notion / Composio / MCP / …) MUST set this to [`MemoryTaint::ExternalSync`] -/// at write time so callers can refuse external-effect tools on tainted context. -#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)] -pub enum MemoryTaint { - /// User-driven memory (chat, manual remember, internal heuristics). - #[default] - Internal, - /// Content ingested from an external sync source. - ExternalSync, -} - -impl Serialize for MemoryTaint { - fn serialize(&self, serializer: S) -> Result - where - S: serde::Serializer, - { - serializer.serialize_str(self.as_db_str()) - } -} - -impl<'de> Deserialize<'de> for MemoryTaint { - fn deserialize(deserializer: D) -> Result - where - D: serde::Deserializer<'de>, - { - let raw = String::deserialize(deserializer)?; - Ok(Self::from_db_str(&raw)) - } -} - -impl MemoryTaint { - /// Serialised form used by the SQLite `memory_docs.taint` column. - /// - /// # Examples - /// - /// ``` - /// use openhuman_core::openhuman::memory::api::types::MemoryTaint; - /// - /// assert_eq!(MemoryTaint::Internal.as_db_str(), "internal"); - /// assert_eq!(MemoryTaint::ExternalSync.as_db_str(), "external_sync"); - /// ``` - pub fn as_db_str(&self) -> &'static str { - match self { - Self::Internal => "internal", - Self::ExternalSync => "external_sync", - } - } - - /// Reverse of [`Self::as_db_str`]. Unknown values fail closed to the more - /// restrictive [`MemoryTaint::ExternalSync`] so policy gates refuse - /// external-effect tools on content of unknown provenance. - /// - /// Note this is *not* a strict inverse of [`Self::as_db_str`]: it never - /// errors, so a malformed or unexpected `raw` string (empty, wrong case, - /// truncated by a partial write, …) silently maps to - /// [`MemoryTaint::ExternalSync`] rather than surfacing as a parse failure. - /// - /// # Examples - /// - /// ``` - /// use openhuman_core::openhuman::memory::api::types::MemoryTaint; - /// - /// assert_eq!(MemoryTaint::from_db_str("internal"), MemoryTaint::Internal); - /// assert_eq!(MemoryTaint::from_db_str("external_sync"), MemoryTaint::ExternalSync); - /// // Unrecognised input fails closed rather than erroring. - /// assert_eq!(MemoryTaint::from_db_str("garbage"), MemoryTaint::ExternalSync); - /// ``` - pub fn from_db_str(raw: &str) -> Self { - match raw { - "internal" => Self::Internal, - "external_sync" => Self::ExternalSync, - _ => Self::ExternalSync, - } - } -} - -/// Categories used to organize and filter memories by nature and lifecycle. -#[derive(Debug, Clone, PartialEq, Eq)] -pub enum MemoryCategory { - /// Long-term foundational facts, user preferences, permanent decisions. - Core, - /// Temporal logs reflecting daily activities or ephemeral state. - Daily, - /// Contextual information derived from active conversations. - Conversation, - /// A user- or system-defined custom category. - Custom(String), -} - -/// The stable wire/display representation uses the built-in labels directly -/// and prefixes custom values with `custom:`. The prefix keeps -/// `Custom("core")` distinct from [`MemoryCategory::Core`] and makes Display, -/// serde, and [`std::str::FromStr`] true inverses. -impl std::fmt::Display for MemoryCategory { - fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { - match self { - Self::Core => write!(f, "core"), - Self::Daily => write!(f, "daily"), - Self::Conversation => write!(f, "conversation"), - Self::Custom(name) => write!(f, "custom:{name}"), - } - } -} - -impl std::str::FromStr for MemoryCategory { - type Err = String; - - fn from_str(value: &str) -> Result { - match value { - "core" => Ok(Self::Core), - "daily" => Ok(Self::Daily), - "conversation" => Ok(Self::Conversation), - "custom:" => Ok(Self::Custom(String::new())), - value if value.starts_with("custom:") && value.len() > "custom:".len() => { - Ok(Self::Custom(value["custom:".len()..].to_string())) - } - value if !value.is_empty() => Ok(Self::Custom(value.to_string())), - _ => Err(format!("unknown memory category: {value}")), - } - } -} - -impl Serialize for MemoryCategory { - fn serialize(&self, serializer: S) -> Result - where - S: serde::Serializer, - { - serializer.serialize_str(&self.to_string()) - } -} - -impl<'de> Deserialize<'de> for MemoryCategory { - fn deserialize(deserializer: D) -> Result - where - D: serde::Deserializer<'de>, - { - let value = String::deserialize(deserializer)?; - value.parse().map_err(serde::de::Error::custom) - } -} - -/// A single stored memory entry with associated metadata. -#[derive(Debug, Clone, Serialize, Deserialize)] -pub struct MemoryEntry { - /// Unique identifier (usually a UUID). - pub id: String, - /// Key or title associated with this memory. - pub key: String, - /// Actual content / value of the memory. - pub content: String, - /// Optional namespace for logical separation. - #[serde(default)] - pub namespace: Option, - /// Organizational category. - pub category: MemoryCategory, - /// ISO 8601 timestamp of create / last-update. - pub timestamp: String, - /// Optional session scope. - pub session_id: Option, - /// Optional relevance / confidence score (typically 0.0–1.0). - pub score: Option, - /// Provenance taint (see [`MemoryTaint`]). Absent on legacy JSON, in which - /// case it defaults to [`MemoryTaint::Internal`]; unknown persisted string - /// values decode as [`MemoryTaint::ExternalSync`]. - #[serde(default)] - pub taint: MemoryTaint, -} - -/// Summary row for agent-side namespace discovery. -#[derive(Debug, Clone, Serialize, Deserialize)] -pub struct NamespaceSummary { - /// Namespace identifier. - pub namespace: String, - /// Number of memory entries currently stored in the namespace. - pub count: usize, - /// RFC3339 timestamp of the most recent update in the namespace, if any. - pub last_updated: Option, -} - -/// Input payload for upserting a namespace-scoped memory document. -#[derive(Debug, Clone, Serialize, Deserialize)] -pub struct NamespaceDocumentInput { - /// Target namespace for the document. - pub namespace: String, - /// Stable upsert key; reusing a key updates the existing document. - pub key: String, - /// Human-readable title. - pub title: String, - /// Document body. - pub content: String, - /// Origin of the content (e.g. `chat`, `gmail`, `notion`). - pub source_type: String, - /// Caller-defined priority label. - pub priority: String, - /// Free-form tags for filtering. - #[serde(default)] - pub tags: Vec, - /// Arbitrary structured metadata carried alongside the document. - #[serde(default)] - pub metadata: serde_json::Value, - /// Category label (see [`MemoryCategory`] wire strings). - pub category: String, - /// Optional session scope. - #[serde(default)] - pub session_id: Option, - /// Explicit document id; generated when absent. - #[serde(default)] - pub document_id: Option, - /// Provenance taint; defaults to [`MemoryTaint::Internal`] for legacy JSON - /// missing this field. Unknown persisted string values decode as - /// [`MemoryTaint::ExternalSync`]. - #[serde(default)] - pub taint: MemoryTaint, -} - -/// One ranked retrieval result for a namespace text query. -#[derive(Debug, Clone, Serialize, Deserialize)] -pub struct NamespaceQueryResult { - /// Upsert key of the matched document. - pub key: String, - /// Matched content. - pub content: String, - /// Relevance score for this hit. - pub score: f64, - /// Category label of the matched document. - pub category: String, - /// Provenance taint; unknown persisted values decode as `external_sync`. - #[serde(default)] - pub taint: MemoryTaint, -} - -/// Discriminator for the kind of stored memory item a hit refers to. -#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)] -#[serde(rename_all = "snake_case")] -pub enum MemoryItemKind { - /// A namespace-scoped memory document (`memory_docs` row). - Document, - /// A key/value record. - Kv, - /// An episodic / conversational memory. - Episodic, - /// A discrete event entry. - Event, -} - -/// Persisted form of a memory document as stored in `memory_docs`. -#[derive(Debug, Clone, Serialize, Deserialize)] -pub struct StoredMemoryDocument { - /// Unique document id. - pub document_id: String, - /// Owning namespace. - pub namespace: String, - /// Stable upsert key. - pub key: String, - /// Human-readable title. - pub title: String, - /// Document body. - pub content: String, - /// Origin of the content (e.g. `chat`, `gmail`). - pub source_type: String, - /// Caller-defined priority label. - pub priority: String, - /// Free-form tags. - pub tags: Vec, - /// Arbitrary structured metadata. - pub metadata: serde_json::Value, - /// Category label. - pub category: String, - /// Optional session scope. - pub session_id: Option, - /// Creation time as a Unix timestamp (seconds). - pub created_at: f64, - /// Last-update time as a Unix timestamp (seconds). - pub updated_at: f64, - /// Path, relative to the vault root, of the authoritative markdown file. - pub markdown_rel_path: String, - /// Provenance taint; unknown persisted values decode as `external_sync`. - #[serde(default)] - pub taint: MemoryTaint, -} - -/// A single KV row, namespace-scoped or global (when `namespace` is `None`). -#[derive(Debug, Clone, Serialize, Deserialize)] -pub struct MemoryKvRecord { - /// Owning namespace, or `None` for a global row. - pub namespace: Option, - /// KV key. - pub key: String, - /// Stored JSON value. - pub value: serde_json::Value, - /// Last-update time as a Unix timestamp (seconds). - pub updated_at: f64, -} - -/// A graph edge (subject — predicate → object) plus accumulated evidence. -#[derive(Debug, Clone, Serialize, Deserialize)] -pub struct GraphRelationRecord { - /// Owning namespace, or `None` for a global relation. - pub namespace: Option, - /// Edge subject (head entity). - pub subject: String, - /// Relation type linking subject to object. - pub predicate: String, - /// Edge object (tail entity). - pub object: String, - /// Arbitrary structured attributes attached to the edge. - pub attrs: serde_json::Value, - /// Last-update time as a Unix timestamp (seconds). - pub updated_at: f64, - /// Number of independent observations supporting this edge. - pub evidence_count: u32, - /// Optional ordering hint among sibling relations. - pub order_index: Option, - /// Documents that contributed evidence for this edge. - pub document_ids: Vec, - /// Chunks that contributed evidence for this edge. - pub chunk_ids: Vec, -} - -/// Per-signal contribution to a hit's final score, for ranking explainers. -#[derive(Debug, Clone, Serialize, Deserialize, Default)] -pub struct RetrievalScoreBreakdown { - /// Lexical / keyword match contribution. - pub keyword_relevance: f64, - /// Vector (cosine) similarity contribution. - pub vector_similarity: f64, - /// Graph-proximity contribution. - pub graph_relevance: f64, - /// Episodic-recall contribution. - pub episodic_relevance: f64, - /// Recency contribution. - pub freshness: f64, - /// Weighted combination of the above signals; the value used for ranking. - pub final_score: f64, -} - -/// A single ranked retrieval hit. -#[derive(Debug, Clone, Serialize, Deserialize)] -pub struct NamespaceMemoryHit { - /// Identifier of the matched item (interpretation depends on [`Self::kind`]). - pub id: String, - /// Which kind of stored item this hit refers to. - pub kind: MemoryItemKind, - /// Owning namespace. - pub namespace: String, - /// Upsert key of the matched item. - pub key: String, - /// Title, when the item has one. - pub title: Option, - /// Matched content. - pub content: String, - /// Category label. - pub category: String, - /// Origin of the content, when known. - pub source_type: Option, - /// Last-update time as a Unix timestamp (seconds). - pub updated_at: f64, - /// Final ranking score; mirrors [`RetrievalScoreBreakdown::final_score`]. - pub score: f64, - /// Per-signal explanation of how [`Self::score`] was derived. - pub score_breakdown: RetrievalScoreBreakdown, - /// Source document id, when the hit resolves to a document. - #[serde(default)] - pub document_id: Option, - /// Source chunk id, when the hit resolves to a chunk. - #[serde(default)] - pub chunk_id: Option, - /// Graph relations that reinforced this hit's ranking. - #[serde(default)] - pub supporting_relations: Vec, - /// Provenance taint; unknown persisted values decode as `external_sync`. - #[serde(default)] - pub taint: MemoryTaint, -} - -/// Aggregated retrieval result for a namespace: rendered context plus hits. -#[derive(Debug, Clone, Serialize, Deserialize)] -pub struct NamespaceRetrievalContext { - /// Namespace the retrieval ran against. - pub namespace: String, - /// Originating query text, if any. - pub query: Option, - /// Rendered, ready-to-inject context assembled from [`Self::hits`]. - pub context_text: String, - /// Ranked hits backing the rendered context. - pub hits: Vec, -} - -#[cfg(test)] -#[path = "types_tests.rs"] -mod tests; diff --git a/src/openhuman/memory/api/types_tests.rs b/src/openhuman/memory/api/types_tests.rs deleted file mode 100644 index 5ee61b5ac5..0000000000 --- a/src/openhuman/memory/api/types_tests.rs +++ /dev/null @@ -1,234 +0,0 @@ -//! Unit tests for the core memory data contracts in [`super`]. - -use super::*; -use serde_json::json; - -#[test] -fn global_namespace_constant_is_stable() { - assert_eq!(GLOBAL_NAMESPACE, "global"); -} - -#[test] -fn memory_category_display_outputs_expected_values() { - assert_eq!(MemoryCategory::Core.to_string(), "core"); - assert_eq!(MemoryCategory::Daily.to_string(), "daily"); - assert_eq!(MemoryCategory::Conversation.to_string(), "conversation"); - assert_eq!( - MemoryCategory::Custom("project_notes".into()).to_string(), - "custom:project_notes" - ); -} - -#[test] -fn memory_category_serde_uses_snake_case() { - assert_eq!( - serde_json::to_string(&MemoryCategory::Core).unwrap(), - "\"core\"" - ); - assert_eq!( - serde_json::to_string(&MemoryCategory::Daily).unwrap(), - "\"daily\"" - ); - assert_eq!( - serde_json::to_string(&MemoryCategory::Conversation).unwrap(), - "\"conversation\"" - ); - assert_eq!( - serde_json::to_string(&MemoryCategory::Custom("core".into())).unwrap(), - "\"custom:core\"" - ); - for category in [ - MemoryCategory::Core, - MemoryCategory::Daily, - MemoryCategory::Conversation, - MemoryCategory::Custom("core".into()), - MemoryCategory::Custom("tool_memory".into()), - ] { - assert_eq!( - category.to_string().parse::().unwrap(), - category - ); - let json = serde_json::to_string(&category).unwrap(); - assert_eq!( - serde_json::from_str::(&json).unwrap(), - category - ); - } - assert_eq!( - "project_notes".parse::().unwrap(), - MemoryCategory::Custom("project_notes".into()) - ); -} - -#[test] -fn memory_entry_roundtrip_preserves_optional_fields() { - let entry = MemoryEntry { - id: "id-1".into(), - key: "favorite_language".into(), - content: "Rust".into(), - namespace: Some("global".into()), - category: MemoryCategory::Core, - timestamp: "2026-02-16T00:00:00Z".into(), - session_id: Some("session-abc".into()), - score: Some(0.98), - taint: MemoryTaint::Internal, - }; - let json = serde_json::to_string(&entry).unwrap(); - let parsed: MemoryEntry = serde_json::from_str(&json).unwrap(); - assert_eq!(parsed.id, "id-1"); - assert_eq!(parsed.namespace.as_deref(), Some("global")); - assert_eq!(parsed.category, MemoryCategory::Core); - assert_eq!(parsed.session_id.as_deref(), Some("session-abc")); - assert_eq!(parsed.score, Some(0.98)); - assert_eq!(parsed.taint, MemoryTaint::Internal); -} - -#[test] -fn memory_taint_defaults_to_internal_for_legacy_rows() { - let legacy = r#"{ - "id":"x","key":"k","content":"c","namespace":null, - "category":"core","timestamp":"2026-01-01T00:00:00Z", - "session_id":null,"score":null - }"#; - let parsed: MemoryEntry = serde_json::from_str(legacy).unwrap(); - assert_eq!(parsed.taint, MemoryTaint::Internal); -} - -#[test] -fn memory_taint_db_str_roundtrip_and_fails_closed() { - assert_eq!(MemoryTaint::Internal.as_db_str(), "internal"); - assert_eq!(MemoryTaint::ExternalSync.as_db_str(), "external_sync"); - assert_eq!(MemoryTaint::from_db_str("internal"), MemoryTaint::Internal); - assert_eq!( - MemoryTaint::from_db_str("external_sync"), - MemoryTaint::ExternalSync - ); - // Unknown / corrupt values fail closed to the restrictive variant. - assert_eq!(MemoryTaint::from_db_str(""), MemoryTaint::ExternalSync); - assert_eq!( - MemoryTaint::from_db_str("EXTERNAL_SYNC"), - MemoryTaint::ExternalSync - ); - assert_eq!( - MemoryTaint::from_db_str("future"), - MemoryTaint::ExternalSync - ); -} - -#[test] -fn memory_taint_serde_unknown_values_fail_closed() { - assert_eq!( - serde_json::from_str::("\"unexpected\"").unwrap(), - MemoryTaint::ExternalSync - ); - assert_eq!( - serde_json::to_string(&MemoryTaint::ExternalSync).unwrap(), - "\"external_sync\"" - ); -} - -#[test] -fn memory_item_kind_serde_uses_snake_case() { - assert_eq!( - serde_json::to_string(&MemoryItemKind::Document).unwrap(), - "\"document\"" - ); - let decoded: MemoryItemKind = serde_json::from_str("\"episodic\"").unwrap(); - assert_eq!(decoded, MemoryItemKind::Episodic); -} - -#[test] -fn namespace_document_input_defaults_optional_fields() { - let value = json!({ - "namespace": "global", "key": "note-1", "title": "Title", - "content": "Body", "source_type": "manual", "priority": "normal", - "metadata": {}, "category": "core" - }); - let parsed: NamespaceDocumentInput = serde_json::from_value(value).unwrap(); - assert!(parsed.tags.is_empty()); - assert!(parsed.session_id.is_none()); - assert!(parsed.document_id.is_none()); - assert_eq!(parsed.taint, MemoryTaint::Internal); -} - -#[test] -fn namespace_document_input_taint_roundtrips_external_sync() { - let input = NamespaceDocumentInput { - namespace: "skill-gmail".into(), - key: "thread-1".into(), - title: "Subject".into(), - content: "Body".into(), - source_type: "composio-sync".into(), - priority: "medium".into(), - tags: Vec::new(), - metadata: json!({}), - category: "core".into(), - session_id: None, - document_id: None, - taint: MemoryTaint::ExternalSync, - }; - let value = serde_json::to_value(&input).unwrap(); - assert_eq!( - value.get("taint").and_then(|v| v.as_str()), - Some("external_sync") - ); - let parsed: NamespaceDocumentInput = serde_json::from_value(value).unwrap(); - assert_eq!(parsed.taint, MemoryTaint::ExternalSync); -} - -#[test] -fn retrieval_score_breakdown_default_is_zeroed() { - let b = RetrievalScoreBreakdown::default(); - assert_eq!(b.keyword_relevance, 0.0); - assert_eq!(b.vector_similarity, 0.0); - assert_eq!(b.graph_relevance, 0.0); - assert_eq!(b.episodic_relevance, 0.0); - assert_eq!(b.freshness, 0.0); - assert_eq!(b.final_score, 0.0); -} - -#[test] -fn memory_kv_record_roundtrips_with_optional_namespace() { - for record in [ - MemoryKvRecord { - namespace: None, - key: "theme".into(), - value: json!("dark"), - updated_at: 1.5, - }, - MemoryKvRecord { - namespace: Some("project".into()), - key: "state".into(), - value: json!({"open": true}), - updated_at: 2.5, - }, - ] { - let value = serde_json::to_value(&record).unwrap(); - let decoded: MemoryKvRecord = serde_json::from_value(value).unwrap(); - assert_eq!(decoded.namespace, record.namespace); - assert_eq!(decoded.key, record.key); - assert_eq!(decoded.value, record.value); - assert_eq!(decoded.updated_at, record.updated_at); - } -} - -#[test] -fn namespace_memory_hit_defaults_optional_fields_and_taint() { - let hit: NamespaceMemoryHit = serde_json::from_value(json!({ - "id": "hit-1", "kind": "document", "namespace": "global", - "key": "note-1", "title": "Title", "content": "Body", - "category": "core", "source_type": "manual", "updated_at": 3.5, - "score": 0.8, - "score_breakdown": { - "keyword_relevance": 0.5, "vector_similarity": 0.2, - "graph_relevance": 0.0, "episodic_relevance": 0.0, - "freshness": 0.1, "final_score": 0.8 - } - })) - .unwrap(); - assert!(hit.document_id.is_none()); - assert!(hit.chunk_id.is_none()); - assert!(hit.supporting_relations.is_empty()); - assert_eq!(hit.kind, MemoryItemKind::Document); - assert_eq!(hit.taint, MemoryTaint::Internal); -} diff --git a/src/openhuman/memory/api/version.rs b/src/openhuman/memory/api/version.rs deleted file mode 100644 index fc8da38c1f..0000000000 --- a/src/openhuman/memory/api/version.rs +++ /dev/null @@ -1,91 +0,0 @@ -//! The memory contract version and the compatibility rule that governs it. -//! -//! Re-exported at the crate root, so the canonical paths are -//! [`crate::openhuman::memory::api::CONTRACT_VERSION`] and [`crate::openhuman::memory::api::is_compatible`]. -//! -//! ## The rule -//! -//! `CONTRACT_VERSION` is `(major, minor)`: -//! -//! - **Minor bump — an addition that capability negotiation alone makes safe.** -//! A new [`crate::openhuman::memory::api::capabilities::Capability`] family is the canonical case: an -//! older driver simply never advertises it, the corresponding RPC methods are -//! unregistered, and the kernel never calls in. A new optional field on an -//! existing wire type, or a new error variant an older kernel can treat as -//! opaque, are the same shape — nothing that already compiled stops -//! compiling, and there is no way for an old driver to be asked for -//! something it never claimed to support. -//! - **Major bump — an existing signature changed, OR a method was added to an -//! already-advertised family.** A method's parameters or return type moved, a -//! mandatory family was added or removed, a wire string changed — or a driver -//! advertising an existing family (say [`crate::openhuman::memory::api::capabilities::Capability::Core`]) -//! now has to implement one more method on it. That last case looks additive -//! but is not: capability negotiation has **family granularity only** — there -//! is no way to advertise "`Core`, but without the new method" — so an older -//! driver that still advertises `Core` can be called into a method it does -//! not implement. Bump the major half instead, which forces every driver -//! claiming that family to actually implement the new surface before it can -//! bind again. -//! -//! ## Why only the major half gates the bind -//! -//! An out-of-process driver reports the version it speaks in its handshake -//! (`POST /v1/handshake` → `{ contract_version, driver_id, capabilities[] }`). -//! **A major mismatch refuses the bind**; a minor difference in either -//! direction is accepted, because capability negotiation already covers it: -//! -//! - remote minor > local minor — the driver advertises families this build has -//! never heard of. Unknown family strings are skipped during handshake -//! parsing, so this kernel simply never calls them. -//! - remote minor < local minor — the driver is missing families this build -//! knows about. It does not advertise them, so the corresponding RPC methods -//! are unregistered and the agent tools are absent. That is the ordinary -//! degradation path, not an error. -//! -//! Refusing on a minor difference would therefore reject a driver that is -//! perfectly usable, and would make adding a family a fleet-wide breaking -//! change — which is exactly what the major/minor split exists to avoid. -//! -//! Encoding the rule here rather than in prose means a caller cannot get it -//! subtly wrong: the bind path calls [`is_compatible`], never compares tuples -//! by hand. - -/// Version of the memory contract this crate defines, as `(major, minor)`. -/// -/// See the module docs for the bump rule. Bump the **minor** half only for an -/// addition capability negotiation alone makes safe — a new capability family, -/// a new optional wire field, a new opaque-to-old-kernels error variant. Bump -/// the **major** half — and reset the minor to `0` — for an existing signature -/// change, a mandatory family change, a wire string change, **or a new method -/// added to a family a driver may already advertise** (negotiation is -/// family-granular, not method-granular, so that case cannot be made minor-safe -/// by negotiation alone). -pub const CONTRACT_VERSION: (u16, u16) = (2, 0); - -/// Whether a driver speaking `remote` can be bound against this build. -/// -/// Compatible exactly when the major halves match. See the module docs for why -/// the minor half is informational. -/// -/// # Examples -/// -/// ``` -/// use openhuman_core::openhuman::memory::api::{is_compatible, CONTRACT_VERSION}; -/// -/// // The version this build speaks is always compatible with itself. -/// assert!(is_compatible(CONTRACT_VERSION)); -/// -/// // A minor difference in either direction is fine — capability negotiation -/// // covers the delta. -/// assert!(is_compatible((CONTRACT_VERSION.0, CONTRACT_VERSION.1 + 7))); -/// -/// // A major mismatch refuses the bind. -/// assert!(!is_compatible((CONTRACT_VERSION.0 + 1, 0))); -/// ``` -pub fn is_compatible(remote: (u16, u16)) -> bool { - remote.0 == CONTRACT_VERSION.0 -} - -#[cfg(test)] -#[path = "version_tests.rs"] -mod tests; diff --git a/src/openhuman/memory/api/version_tests.rs b/src/openhuman/memory/api/version_tests.rs deleted file mode 100644 index b6baf3935c..0000000000 --- a/src/openhuman/memory/api/version_tests.rs +++ /dev/null @@ -1,83 +0,0 @@ -//! Unit tests for the contract version rule in [`super`]. -//! -//! The rule these pin is the one from the kernel design: a **minor** bump means -//! a capability was added and stays compatible; a **major** mismatch refuses -//! the bind. - -use super::*; - -#[test] -fn contract_version_starts_at_one_zero() { - assert_eq!(CONTRACT_VERSION, (2, 0)); -} - -#[test] -fn own_version_is_compatible_with_itself() { - assert!(is_compatible(CONTRACT_VERSION)); -} - -#[test] -fn a_minor_bump_stays_compatible_in_both_directions() { - let (major, minor) = CONTRACT_VERSION; - - // Remote ahead: it advertises families this build does not know. Unknown - // family strings are skipped during handshake parsing. - assert!(is_compatible((major, minor + 1))); - assert!(is_compatible((major, minor + 25))); - assert!(is_compatible((major, u16::MAX))); - - // Remote behind: it lacks families this build knows. Those simply are not - // advertised, so the surface degrades — the ordinary path, not an error. - assert!(is_compatible((major, minor.saturating_sub(1)))); - assert!(is_compatible((major, 0))); -} - -#[test] -fn a_major_mismatch_refuses_the_bind() { - let (major, minor) = CONTRACT_VERSION; - - // Remote ahead by a major: an existing signature changed under us. - assert!(!is_compatible((major + 1, 0))); - assert!(!is_compatible((major + 1, minor))); - assert!(!is_compatible((major + 1, u16::MAX))); - - // Remote behind by a major: same reasoning, other direction. A newer minor - // does not rescue an older major. - assert!(!is_compatible((major - 1, u16::MAX))); - assert!(!is_compatible((0, 0))); -} - -#[test] -fn adding_a_method_to_an_already_advertised_family_requires_a_major_bump() { - // Capability negotiation has family granularity, not method granularity: - // there is no way to advertise "Core, but without the new method". So a - // method added to a family a driver may already advertise (e.g. Core, - // Recall) cannot be made minor-safe by negotiation the way a brand-new - // capability family can — an older driver still advertising that family - // would be called into a method it never implemented. This is why the - // module docs classify that addition as a MAJOR bump, not minor, even - // though it looks additive. This test exists so the rule cannot be - // re-derived from `is_compatible`'s code alone, which only encodes "major - // halves must match" and says nothing about *why* a same-family method - // addition belongs on the major side of that line. - assert!( - !is_compatible((CONTRACT_VERSION.0 + 1, 0)), - "a method added to an existing family must ship as a major bump, \ - which this asserts refuses the bind against an old build" - ); -} - -#[test] -fn compatibility_depends_only_on_the_major_half() { - let (major, _) = CONTRACT_VERSION; - for minor in [0u16, 1, 2, 7, 999, u16::MAX] { - assert!( - is_compatible((major, minor)), - "minor {minor} should not affect compatibility" - ); - assert!( - !is_compatible((major + 1, minor)), - "minor {minor} must not rescue a major mismatch" - ); - } -} diff --git a/src/openhuman/memory/api/wire.rs b/src/openhuman/memory/api/wire.rs deleted file mode 100644 index df6e701ac4..0000000000 --- a/src/openhuman/memory/api/wire.rs +++ /dev/null @@ -1,133 +0,0 @@ -//! Error names for a driver reached over a wire, and the mapping both ends use. -//! -//! # Why this is here and not in the transport -//! -//! A driver can be in-process, in a loadable module, or behind a socket. The -//! last two need [`MemoryError`] to survive a round trip through a -//! `(name, message)` pair, because that is all a bus or an HTTP status gives -//! you. -//! -//! The mapping could have lived in whichever adapter needed it first. It lives -//! here instead because there will be more than one adapter, and two copies of -//! a name table drift: the module side starts answering -//! `…Error.PathEscape` while the host side still only recognises -//! `…Error.Invalid`, and the symptom is a security-relevant error -//! silently reclassified as a caller mistake. One table, used by both ends, with -//! [`round_trips_every_variant`](self) pinning it. -//! -//! # One name per variant, not one per outcome class -//! -//! An earlier sketch collapsed these onto three names — "the caller can fix it", -//! "the capability is absent", "something broke" — on the grounds that a host has -//! only those three responses. That is wrong for two reasons. -//! -//! A host does not merely *react* to a driver error; it **is** a -//! [`MemoryProvider`](crate::openhuman::memory::api::provider::MemoryProvider) to everything above it, -//! so it has to hand its own callers a `MemoryError`. Collapsing on the way out -//! and guessing on the way back in would turn a `NotFound` into an `Invalid`, -//! and `get`'s contract says a missing entry is `Ok(None)` while an `Invalid` is -//! a real failure — so the guess is observable. -//! -//! And `PathEscape` is not interchangeable with `Invalid`. It reports a symlink -//! or traversal attempt that left the workspace sandbox, which a host may want -//! to log, report or refuse to retry differently from a malformed argument. -//! -//! # Unrecognised names are backend failures -//! -//! [`from_wire`] maps anything it does not know to [`MemoryError::Other`], never -//! to [`MemoryError::Invalid`]. A driver newer than this build may name an error -//! this table has no variant for, and telling a caller its input was wrong when -//! it was not sends it into a rewrite loop over something already correct. -//! -//! # Messages, and what must not be in them -//! -//! The name is the contract; the message is for a human. Neither may carry a -//! namespace key, an entry's content, a recall query, a credential or an -//! absolute path — memory content is user data, and an error string is not a -//! place for it. `Io` and `Serde` are deliberately flattened into a message -//! here, because reconstructing a live `std::io::Error` or -//! `serde_json::Error` on the far side is not possible and not useful. - -use crate::openhuman::memory::api::error::MemoryError; - -/// A requested record, source or node was not found. -pub const NOT_FOUND: &str = "ai.tinyhumans.tinymemory.Error.NotFound"; -/// Caller-supplied input failed validation. -pub const INVALID: &str = "ai.tinyhumans.tinymemory.Error.Invalid"; -/// A configured budget was exceeded. -pub const BUDGET_EXCEEDED: &str = "ai.tinyhumans.tinymemory.Error.BudgetExceeded"; -/// A path escaped the workspace sandbox. -pub const PATH_ESCAPE: &str = "ai.tinyhumans.tinymemory.Error.PathEscape"; -/// An underlying IO failure. -pub const IO: &str = "ai.tinyhumans.tinymemory.Error.Io"; -/// A serialization or deserialization failure. -pub const SERDE: &str = "ai.tinyhumans.tinymemory.Error.Serde"; -/// The driver does not implement the named capability family. -pub const UNSUPPORTED: &str = "ai.tinyhumans.tinymemory.Error.Unsupported"; -/// An opaque lower-level failure. -pub const OTHER: &str = "ai.tinyhumans.tinymemory.Error.Other"; - -/// The wire name for `error`. -/// -/// Total by construction: the `match` is exhaustive, so a variant added to -/// [`MemoryError`] is a compile error here rather than a silent fallthrough onto -/// [`OTHER`]. -#[must_use] -pub fn wire_name(error: &MemoryError) -> &'static str { - match error { - MemoryError::NotFound(_) => NOT_FOUND, - MemoryError::Invalid(_) => INVALID, - MemoryError::BudgetExceeded(_) => BUDGET_EXCEEDED, - MemoryError::PathEscape(_) => PATH_ESCAPE, - MemoryError::Io(_) => IO, - MemoryError::Serde(_) => SERDE, - MemoryError::Unsupported { .. } => UNSUPPORTED, - MemoryError::Other(_) => OTHER, - } -} - -/// The message to send alongside [`wire_name`]. -/// -/// For most variants this is the inner string rather than the `Display` output, -/// so the receiving side can rebuild the variant without the prefix -/// (`"invalid input: "`, …) being baked into the payload twice. -#[must_use] -pub fn wire_message(error: &MemoryError) -> String { - match error { - MemoryError::NotFound(message) - | MemoryError::Invalid(message) - | MemoryError::BudgetExceeded(message) - | MemoryError::PathEscape(message) => message.clone(), - MemoryError::Unsupported { capability } => capability.clone(), - // No inner string to lift: these carry a foreign error type, so the - // rendered form is all there is. - MemoryError::Io(inner) => inner.to_string(), - MemoryError::Serde(inner) => inner.to_string(), - MemoryError::Other(inner) => inner.to_string(), - } -} - -/// Rebuild a [`MemoryError`] from a `(name, message)` pair. -/// -/// An unrecognised `name` becomes [`MemoryError::Other`] — see the module docs -/// on why it must not become [`MemoryError::Invalid`]. -#[must_use] -pub fn from_wire(name: &str, message: &str) -> MemoryError { - match name { - NOT_FOUND => MemoryError::NotFound(message.to_string()), - INVALID => MemoryError::Invalid(message.to_string()), - BUDGET_EXCEEDED => MemoryError::BudgetExceeded(message.to_string()), - PATH_ESCAPE => MemoryError::PathEscape(message.to_string()), - // `std::io::Error` cannot be reconstructed with its original kind from a - // string, and inventing one would be worse than being honest that this - // crossed a wire. The message is preserved. - IO => MemoryError::Other(anyhow::anyhow!("io error: {message}")), - SERDE => MemoryError::Other(anyhow::anyhow!("serde error: {message}")), - UNSUPPORTED => MemoryError::unsupported_raw(message), - _ => MemoryError::Other(anyhow::anyhow!("{message}")), - } -} - -#[cfg(test)] -#[path = "wire_tests.rs"] -mod tests; diff --git a/src/openhuman/memory/api/wire_tests.rs b/src/openhuman/memory/api/wire_tests.rs deleted file mode 100644 index 7fdba08c13..0000000000 --- a/src/openhuman/memory/api/wire_tests.rs +++ /dev/null @@ -1,104 +0,0 @@ -//! The name table is a contract, so these tests pin it rather than exercise it. - -use super::{from_wire, wire_message, wire_name}; -use crate::openhuman::memory::api::capabilities::Capability; -use crate::openhuman::memory::api::error::MemoryError; - -/// Every variant, so a new one fails to compile in `wire_name` and fails here. -fn every_variant() -> Vec { - vec![ - MemoryError::NotFound("thread-7".to_string()), - MemoryError::Invalid("limit must be positive".to_string()), - MemoryError::BudgetExceeded("depth 12 exceeds 8".to_string()), - MemoryError::PathEscape("symlink leaves workspace".to_string()), - MemoryError::Io(std::io::Error::other("disk gone")), - MemoryError::Serde(serde_json::from_str::("nope").unwrap_err()), - MemoryError::unsupported(Capability::Tree), - MemoryError::Other(anyhow::anyhow!("engine stopped")), - ] -} - -#[test] -fn round_trips_every_variant() { - for error in every_variant() { - let name = wire_name(&error); - let message = wire_message(&error); - let rebuilt = from_wire(name, &message); - - // Io and Serde deliberately degrade to `Other`: neither foreign error - // type can be reconstructed from a string. Everything else must come - // back as the same variant, because a host re-raises it to its own - // callers and the variant is what they match on. - match (&error, &rebuilt) { - (MemoryError::Io(_) | MemoryError::Serde(_), MemoryError::Other(_)) => {} - _ => assert_eq!( - std::mem::discriminant(&error), - std::mem::discriminant(&rebuilt), - "{name} did not round-trip to the same variant" - ), - } - assert!( - rebuilt.to_string().contains(message.trim()) || message.is_empty(), - "{name} lost its message: {rebuilt}" - ); - } -} - -#[test] -fn every_name_is_distinct() { - let mut names: Vec<&str> = every_variant().iter().map(wire_name).collect(); - let before = names.len(); - names.sort_unstable(); - names.dedup(); - assert_eq!(before, names.len(), "two variants share a wire name"); -} - -#[test] -fn an_unrecognised_name_is_a_backend_failure_not_an_input_error() { - // The load-bearing case. A driver newer than this build names something we - // have no variant for; classifying it as `Invalid` would tell a caller its - // request was wrong and send it into a rewrite loop. - let rebuilt = from_wire("ai.tinyhumans.tinymemory.Error.SomethingNewer", "hmm"); - assert!(matches!(rebuilt, MemoryError::Other(_)), "{rebuilt:?}"); -} - -#[test] -fn a_path_escape_does_not_collapse_onto_invalid() { - // These were nearly given one shared name. A sandbox escape is not a - // malformed argument, and a host may log or refuse to retry it differently. - assert_ne!( - wire_name(&MemoryError::PathEscape("x".to_string())), - wire_name(&MemoryError::Invalid("x".to_string())) - ); -} - -#[test] -fn a_missing_entry_stays_not_found() { - // `get`'s contract makes a missing entry `Ok(None)` and an `Invalid` a real - // failure, so conflating the two is observable to a caller. - let rebuilt = from_wire(super::NOT_FOUND, "absent"); - assert!(matches!(rebuilt, MemoryError::NotFound(_)), "{rebuilt:?}"); -} - -#[test] -fn an_unsupported_capability_keeps_its_family_name() { - let error = MemoryError::unsupported(Capability::Diff); - let rebuilt = from_wire(wire_name(&error), &wire_message(&error)); - match rebuilt { - MemoryError::Unsupported { capability } => { - assert_eq!(capability, Capability::Diff.as_str()); - } - other => panic!("expected Unsupported, got {other:?}"), - } -} - -#[test] -fn an_unknown_capability_name_off_the_wire_survives() { - // A driver on a newer minor contract may name a family this build has no - // `Capability` for. It must not be dropped or fail to parse. - let rebuilt = from_wire(super::UNSUPPORTED, "vendor_extension"); - match rebuilt { - MemoryError::Unsupported { capability } => assert_eq!(capability, "vendor_extension"), - other => panic!("expected Unsupported, got {other:?}"), - } -} diff --git a/src/openhuman/memory/api_identity_tests.rs b/src/openhuman/memory/api_identity_tests.rs new file mode 100644 index 0000000000..abcf087b3c --- /dev/null +++ b/src/openhuman/memory/api_identity_tests.rs @@ -0,0 +1,139 @@ +//! The host's memory contract must **be** `tinymemory-api`, not a copy of it. +//! +//! `crate::openhuman::memory::api` is the vocabulary three parties speak: +//! the host call sites, [`crate::openhuman::modules::memory::ModuleMemoryProvider`] +//! (which serialises it onto the bus), and the separately compiled TinyMemory +//! module on the far end — which compiles against the `tinymemory-api` **crate**. +//! +//! Commit `3ee5a3cad` ("run tiny domains as TinyBus modules") inlined that +//! crate into `src/openhuman/memory/api/` as 10,894 lines of verbatim copy. +//! Every file was byte-identical to `vendor/tinymemory/api/src/` apart from +//! doc-comment paths, so nothing behaved differently on the day — but the two +//! were now *distinct types* that only a human comparing files could keep in +//! step. +//! +//! That is precisely the failure mode `modules/memory.rs` says the shared error +//! table exists to prevent: "reimplementing the mapping here is what would let +//! a `PathEscape` arrive as an `Invalid`, silently reclassifying a sandbox +//! escape as a caller mistake." `api::wire` is the table, and while the host +//! held its own copy of it that sentence described an *aspiration*, not the +//! build — the host end and the module end were two independent definitions +//! free to drift the moment either side was edited. +//! +//! These assertions are **type identities**, so they fail at compile time +//! rather than at run time. Against the inlined copy this file does not build: +//! `expected 'tinymemory_api::chunks::SourceKind', found +//! 'openhuman::openhuman::memory::api::chunks::SourceKind'`. That is the point +//! — a future re-inlining cannot land quietly, it breaks the build. + +/// The chunk vocabulary the tree families pass across the seam. +#[test] +fn chunk_types_are_the_contract_crates() { + fn source_kind( + k: crate::openhuman::memory::api::chunks::SourceKind, + ) -> tinymemory_api::chunks::SourceKind { + k + } + fn chunk(c: crate::openhuman::memory::api::chunks::Chunk) -> tinymemory_api::chunks::Chunk { + c + } + let _ = source_kind as fn(_) -> _; + let _ = chunk as fn(_) -> _; +} + +/// The error enum and the wire table that round-trips it. If these two ever +/// separate, a driver error can be reclassified in transit with nothing failing. +#[test] +fn error_and_wire_table_are_the_contract_crates() { + fn error( + e: crate::openhuman::memory::api::error::MemoryError, + ) -> tinymemory_api::error::MemoryError { + e + } + let _ = error as fn(_) -> _; + + // Same input, same name, from both ends of the seam. + let host = crate::openhuman::memory::api::wire::wire_name( + &crate::openhuman::memory::api::error::MemoryError::PathEscape("/etc/passwd".to_string()), + ); + let crate_side = tinymemory_api::wire::wire_name( + &tinymemory_api::error::MemoryError::PathEscape("/etc/passwd".to_string()), + ); + assert_eq!(host, crate_side, "the wire error table must be one table"); +} + +/// The driver contract itself: a `MemoryProvider` built against the crate must +/// satisfy the host's trait object, or the seam has two provider vocabularies. +#[test] +fn provider_contract_is_the_contract_crates() { + fn provider( + p: std::sync::Arc, + ) -> std::sync::Arc { + p + } + let _ = provider as fn(_) -> _; + + fn capabilities( + c: crate::openhuman::memory::api::capabilities::Capabilities, + ) -> tinymemory_api::capabilities::Capabilities { + c + } + let _ = capabilities as fn(_) -> _; +} + +/// The **inbound** half of the seam. `memory::api::host` deliberately re-exports +/// two types rather than the whole `tinymemory_api::host` namespace: that +/// namespace is the in-process engine-embedding seam (the persisted +/// `MemoryConfig` sections, `MemoryHostConfig`, `EmbeddingProvider`, +/// `MemoryEventSink`), none of which touches the bus. These two do — +/// `modules/memory_host.rs` publishes `MemoryEvent` back onto the host's event +/// bus and answers the module's NLP callback with `SpacyResponse` — so these +/// two are contract, and must be the crate's types rather than lookalikes. +#[test] +fn host_seam_types_are_the_contract_crates() { + fn event( + e: crate::openhuman::memory::api::host::MemoryEvent, + ) -> tinymemory_api::host::MemoryEvent { + e + } + let _ = event as fn(_) -> _; + + fn spacy( + s: crate::openhuman::memory::api::host::SpacyResponse, + ) -> tinymemory_api::host::SpacyResponse { + s + } + let _ = spacy as fn(_) -> _; +} + +/// Version negotiation is contract. `memory::binding` and `memory::ops::provider` +/// compare a driver's reported contract against this constant before binding it, +/// so a host copy that drifted from the crate's value would admit a driver the +/// module end considers incompatible — or reject one it does not. +#[test] +fn contract_version_is_the_contract_crates() { + assert_eq!( + crate::openhuman::memory::api::CONTRACT_VERSION, + tinymemory_api::CONTRACT_VERSION, + "the contract version must be one value, not two" + ); +} + +/// `null` is **not** contract surface, and this pins the boundary from the other +/// side. `NullMemoryProvider` is the fallback `memory::binding` installs when no +/// module driver is available — it is what runs when nothing crosses the bus. +/// It is still the crate's type (the host does not get to invent a second +/// provider vocabulary), but it is reached by naming `tinymemory_api::null` at +/// the call site rather than through `memory::api`, which is what keeps "the +/// module contract" and "the host's own use of the crate" distinguishable in +/// the source. If `null` is ever re-added to `memory::api`, delete this test +/// deliberately rather than letting the distinction erode. +#[test] +fn the_fallback_driver_is_the_contract_crates_but_not_contract_surface() { + fn provider( + p: std::sync::Arc, + ) -> std::sync::Arc { + p + } + let _ = provider as fn(_) -> _; +} diff --git a/src/openhuman/memory/binding.rs b/src/openhuman/memory/binding.rs index 25a1b89851..8de68766d4 100644 --- a/src/openhuman/memory/binding.rs +++ b/src/openhuman/memory/binding.rs @@ -43,10 +43,10 @@ use std::sync::{Arc, OnceLock, RwLock}; use crate::openhuman::memory::api::capabilities::Capabilities; use crate::openhuman::memory::api::health::MemoryHealth; -use crate::openhuman::memory::api::null::{NullMemoryProvider, NULL_DRIVER_ID}; use crate::openhuman::memory::api::provider::MemoryProvider; use crate::openhuman::memory::api::CONTRACT_VERSION; use crate::openhuman::memory::guard::{GuardPolicy, MemoryGuard}; +use tinymemory_api::null::{NullMemoryProvider, NULL_DRIVER_ID}; use crate::core::subsystem::{ BoundDriver, DriverCapabilities, DriverClass, DriverHealth, SubsystemSlot, diff --git a/src/openhuman/memory/binding_tests.rs b/src/openhuman/memory/binding_tests.rs index b8d9949c35..940e09f2ce 100644 --- a/src/openhuman/memory/binding_tests.rs +++ b/src/openhuman/memory/binding_tests.rs @@ -19,9 +19,9 @@ use std::sync::Arc; use crate::core::subsystem::{DriverHealth, SubsystemSlot}; use crate::openhuman::memory::api::capabilities::Capabilities; use crate::openhuman::memory::api::health::MemoryHealth; -use crate::openhuman::memory::api::null::{NullMemoryProvider, NULL_DRIVER_ID}; use crate::openhuman::memory::api::provider::MemoryProvider; use crate::openhuman::memory::api::CONTRACT_VERSION; +use tinymemory_api::null::{NullMemoryProvider, NULL_DRIVER_ID}; use crate::openhuman::memory::api::capabilities::Capability; use crate::openhuman::memory::api::error::MemoryError; diff --git a/src/openhuman/memory/direct_engine_refs_tests.rs b/src/openhuman/memory/direct_engine_refs_tests.rs new file mode 100644 index 0000000000..e156d1c10c --- /dev/null +++ b/src/openhuman/memory/direct_engine_refs_tests.rs @@ -0,0 +1,522 @@ +//! Enforcement lint: the set of production files that call `tinymemory-core` +//! **directly**, around the module seam, must not grow. +//! +//! # Why a second path to memory is a correctness problem, not a size problem +//! +//! `memory::binding` says it plainly: "the built-in driver is the compiled +//! TinyMemory TinyBus module. The host no longer exposes an embedded engine +//! class for memory." Every call that goes over that bus is round-tripped +//! through [`crate::openhuman::memory::api::wire`]'s error table — the one +//! `modules/memory.rs` keeps shared "because reimplementing the mapping here is +//! what would let a `PathEscape` arrive as an `Invalid`, silently reclassifying +//! a sandbox escape as a caller mistake" — and is filtered by the capability +//! set `ModuleMemoryProvider::verify` cross-checks against the module's own +//! answer. +//! +//! A direct `tinymemory_core::…` call gets neither. It is a second, unpoliced +//! door into the same subsystem, and two doors into one capability is a +//! capability whose behaviour can diverge. `tinymemory-core` also stays linked +//! into the shipped binary — 1.44 MB of `.text`, the 7th largest crate — for as +//! long as any of these remain, which is the visible symptom rather than the +//! disease (#5560). +//! +//! # This lint is a ratchet, not an invariant +//! +//! Same shape and same reasoning as [`super::bypass_allowlist_tests`], and as +//! `INTENTIONALLY_NOT_FORWARDED` in `scripts/lib/feature-forwarding.mjs`: the +//! current direct callers are enumerated in [`ALLOWED`] with a classification +//! and a reason each, and **that list may shrink but must never grow**. A lint +//! that was red on day one would be `#[ignore]`d within a week; a green ratchet +//! converges. +//! +//! # The classification, and why most of the list cannot move yet +//! +//! Each entry carries a [`Verdict`], which is the inventory the migration is +//! driven from: +//! +//! - [`Verdict::SeamExpressible`] — the existing `MemoryProvider` surface +//! already covers this. These are the ones to migrate; a non-empty set here +//! is a to-do list, not a steady state. +//! - [`Verdict::NeedsWiderSeam`] — the call wants something the thirteen +//! capability families do not expose. **These are blocked upstream, not +//! here.** `modules::registry` pins the TinyMemory module to a released, +//! SHA-256-verified artifact (v1.0.1 at the time of writing), so a new bus +//! method is a `tinymemory` release plus a registry re-pin before it is a +//! host change. Adding the trait method alone would produce a driver that +//! answers `Unsupported` — strictly worse than the direct call it replaced, +//! because the failure moves from compile time to run time. +//! - [`Verdict::HostSide`] — not a driver call at all. Re-export shims, +//! host-seam installation, and inert type imports. These are correct as they +//! stand and are counted only so "deliberate" stays distinguishable from +//! "forgotten". +//! +//! ## The concrete gaps, for whoever picks the upstream work up +//! +//! The seam's tree family is `query_source(namespace, source_id, limit, scope) +//! -> Vec`, `drill_down(namespace, node_id) -> QueryResult`, `append`, +//! `seal`, `cascade`. What the host actually calls is richer in four ways, and +//! each is a distinct upstream ask: +//! +//! 1. **Retrieval takes filters the seam has no room for** — a time window, a +//! free-text query, a `SourceKind`, a depth and a limit +//! (`query::backend`, `query::cover_window`, `query::fast_walk`). +//! 2. **Chunk reads have no family at all.** `store::chunks::store::{get_chunk, +//! list_chunks}` with a nine-field `ListChunksQuery` backs three agent tools +//! (`memory_chunk_context`, `raw_chunks`, `vector_search`); the seam's only +//! chunk door is `query_source`, keyed on a single source id. +//! 3. **Entity search has no kind filter.** `MemoryEntities::entities` takes +//! `(namespace, query, limit)`; `memory_tree_search_entities` additionally +//! filters on `Vec`. +//! 4. **Sources cannot be listed.** `MemorySourceSink` is +//! `accept_source_items` + `forget_source`; `memory_diff` needs +//! `sources::{get_source, list_sources}`. +//! +//! Two more sit outside the tree families entirely: the `people` domain has no +//! capability family (`PeopleStore`, `Handle`, `PersonId`), and +//! `source_scope::{current_source_scope, chunk_source_allowed}` is a +//! task-local the host sets and the engine reads — it is host policy that +//! happens to live in the engine crate, and it should probably move to +//! `tinymemory-api` rather than gain a bus method. +//! +//! # Known weaknesses, stated rather than hidden +//! +//! - **The lint sees text, not types.** A reference reached through a +//! re-export under another name is invisible to it — and the memory tree is +//! full of those on purpose: `memory/mod.rs` re-exports twenty-five engine +//! modules, and ~687 `memory::store::…` / `memory::tree::…` paths elsewhere +//! resolve into the crate through them. **This lint deliberately does not +//! count those.** It counts the sites that *name* the crate, because those +//! are the ones a migration edits. The re-export surface is a separate, +//! larger problem tracked in the issue, and pretending this number covers it +//! would be the worst outcome. +//! - **By-path test files are out of scope** (`*_tests.rs`, `tests.rs`, +//! `test_support/`), matching the sibling lint. Several inline +//! `#[cfg(test)]` modules do name the crate (`query::drill_down`, +//! `query::fetch_leaves`, `query::query_source` each assert a tool's result +//! against a direct engine call); those files are listed, and the entry says +//! so. +//! - **Comment lines are skipped**, so the many doc comments that reference +//! `tinymemory_core::…` by path do not inflate the count. + +use std::collections::BTreeSet; +use std::path::{Path, PathBuf}; + +/// Why a file may name the engine crate today. +#[derive(Clone, Copy, PartialEq, Eq, Debug)] +pub(crate) enum Verdict { + /// The existing `MemoryProvider` surface covers this. Migrate it. + SeamExpressible, + /// Blocked on a wider bus surface, which means an upstream `tinymemory` + /// release and a `modules::registry` re-pin. + NeedsWiderSeam, + /// Not a driver call: a re-export shim, host-seam installation, or an + /// inert type import. + HostSide, +} + +/// The literal this lint searches for. A single needle, deliberately: the +/// question is "does this file name the engine crate", not "which item". +const NEEDLE: &str = "tinymemory_core::"; + +/// `(repo-relative path, verdict, why it names the engine today)`. +/// +/// Adding an entry is a decision, not a way to silence the lint. Sorted by +/// path — [`scan`] returns a `BTreeSet`, so keeping the literal in the same +/// order makes diffs readable. +const ALLOWED: &[(&str, Verdict, &str)] = &[ + // ── Re-export shims: `pub use tinymemory_core::::*;` ──────────── + // + // These are the historical-path aliases `memory/mod.rs` documents. They + // name the crate once each and call nothing. Removing them is the + // re-export problem, not the direct-call problem. + ( + "src/openhuman/agent/learning/candidate.rs", + Verdict::HostSide, + "re-export shim for learning_candidate types", + ), + ( + "src/openhuman/agent/tinyagents/thread_context.rs", + Verdict::HostSide, + "re-export shim for the thread-id task-local", + ), + ( + "src/openhuman/inference/embeddings/provider_trait.rs", + Verdict::HostSide, + "re-export shim for TinyAgentsEmbeddingProvider, which cannot live host-side", + ), + ( + "src/openhuman/memory/conversations/mod.rs", + Verdict::HostSide, + "re-export shim: pub use tinymemory_core::conversations::*", + ), + ( + "src/openhuman/memory/diff/mod.rs", + Verdict::HostSide, + "re-export shim: pub use tinymemory_core::diff::*", + ), + ( + "src/openhuman/memory/mod.rs", + Verdict::HostSide, + "the re-export block itself — twenty-five engine modules under their historical paths", + ), + ( + "src/openhuman/memory/people/mod.rs", + Verdict::HostSide, + "re-export shim: pub use tinymemory_core::people::*", + ), + ( + "src/openhuman/memory/sources/mod.rs", + Verdict::HostSide, + "re-export shim: pub use tinymemory_core::sources::*", + ), + ( + "src/openhuman/memory/sync/composio/mod.rs", + Verdict::HostSide, + "re-export shim: pub use tinymemory_core::sync::composio::*", + ), + ( + "src/openhuman/memory/sync/composio/providers/mod.rs", + Verdict::HostSide, + "re-export shim: pub use tinymemory_core::sync::composio::providers::*", + ), + ( + "src/openhuman/memory/sync/composio/providers/slack/mod.rs", + Verdict::HostSide, + "re-export shim: pub use tinymemory_core::sync::composio::providers::slack::*", + ), + ( + "src/openhuman/memory/sync/mod.rs", + Verdict::HostSide, + "re-export shim: pub use tinymemory_core::sync::*", + ), + ( + "src/openhuman/memory/sync/sync_status/mod.rs", + Verdict::HostSide, + "re-export shim: pub use tinymemory_core::sync::sync_status::*", + ), + ( + "src/openhuman/memory/tool_memory/mod.rs", + Verdict::HostSide, + "re-export shim: pub use tinymemory_core::tool_memory::*", + ), + ( + "src/openhuman/memory/tree/health/mod.rs", + Verdict::HostSide, + "re-export shim: pub use tinymemory_core::tree::health::*", + ), + ( + "src/openhuman/memory/tree/mod.rs", + Verdict::HostSide, + "re-export shim: pub use tinymemory_core::tree::*", + ), + ( + "src/openhuman/memory/tree/retrieval/mod.rs", + Verdict::HostSide, + "re-export shim: pub use tinymemory_core::tree::retrieval::*", + ), + ( + "src/openhuman/memory/tree/tree/mod.rs", + Verdict::HostSide, + "re-export shim: pub use tinymemory_core::tree::tree::*", + ), + ( + "src/openhuman/memory/tree/tree_runtime/mod.rs", + Verdict::HostSide, + "re-export shim: pub use tinymemory_core::tree::tree_runtime::*", + ), + // ── Host-seam installation: the host handing itself TO the engine ─────── + // + // The direction of these is inbound, not outbound: they install embedding / + // chat / config / NLP / scheduler / shutdown / error-reporting callbacks + // into the in-process engine. `modules/memory_host.rs` is the same seam + // served over the bus. They are what an embedded engine needs, and they + // are the last thing to remove, not the first. + ( + "src/openhuman/memory/host.rs", + Verdict::HostSide, + "installs the event sink into the in-process engine", + ), + ( + "src/openhuman/memory/host_impls.rs", + Verdict::HostSide, + "installs the eight host seams (embedding, chat, composio, config, nlp, scheduler gate, shutdown, error reporter); mirrored over the bus by modules/memory_host.rs", + ), + // ── Retrieval: filters the seam's tree family has no room for ─────────── + ( + "src/openhuman/memory/query/backend.rs", + Verdict::NeedsWiderSeam, + "retrieval::source::query_source / drill_down / fetch_leaves take a time window, a free-text query, a SourceKind and a depth; MemoryTree::query_source takes (namespace, source_id, limit, scope) and drill_down takes (namespace, node_id)", + ), + ( + "src/openhuman/memory/query/cover_window.rs", + Verdict::NeedsWiderSeam, + "retrieval::cover::cover_window has no seam equivalent", + ), + ( + "src/openhuman/memory/query/drill_down.rs", + Verdict::NeedsWiderSeam, + "inline #[cfg(test)] module only — asserts the tool result against a direct engine drill_down; the production path goes through query/backend.rs", + ), + ( + "src/openhuman/memory/query/fast_walk.rs", + Verdict::NeedsWiderSeam, + "retrieval::fast_retrieve (the E2GraphRAG retriever) has no seam equivalent", + ), + ( + "src/openhuman/memory/query/fetch_leaves.rs", + Verdict::NeedsWiderSeam, + "inline #[cfg(test)] module only — asserts the tool result against a direct engine fetch_leaves", + ), + ( + "src/openhuman/memory/query/ingest_document.rs", + Verdict::NeedsWiderSeam, + "names SourceKind / SourceRef, which are tinycortex-api types the engine re-exports — NOT the same type as the contract's api::chunks::SourceKind, so this is not a type carve-out", + ), + ( + "src/openhuman/memory/query/query_source.rs", + Verdict::NeedsWiderSeam, + "SourceKind in production, plus an inline #[cfg(test)] assertion against a direct engine query_source", + ), + ( + "src/openhuman/memory/query/search_entities.rs", + Verdict::NeedsWiderSeam, + "retrieval::search_entities filters on Vec; MemoryEntities::entities takes (namespace, query, limit) only", + ), + // ── Agent tools: chunk reads, source listing, people, source scope ────── + ( + "src/openhuman/memory/sync/composio/providers/context_ext.rs", + Verdict::NeedsWiderSeam, + "extends the engine's ProviderContext; the sync pipeline is engine-internal and has no capability family", + ), + ( + "src/openhuman/memory/tools/diff.rs", + Verdict::NeedsWiderSeam, + "sources::{get_source, list_sources}; MemorySourceSink is accept_source_items + forget_source, with no list door", + ), + ( + "src/openhuman/memory/tools/people.rs", + Verdict::NeedsWiderSeam, + "people::store::PeopleStore and the Handle/Interaction/PersonId vocabulary; there is no people capability family", + ), + ( + "src/openhuman/memory/tools/raw_store/kinds.rs", + Verdict::NeedsWiderSeam, + "store::MemoryKind — an engine type with no contract counterpart", + ), + ( + "src/openhuman/memory/tools/raw_store/raw_chunks.rs", + Verdict::NeedsWiderSeam, + "store::chunks::store::list_chunks with a nine-field ListChunksQuery, plus the source_scope task-local", + ), + ( + "src/openhuman/memory/tools/raw_store/raw_search.rs", + Verdict::NeedsWiderSeam, + "retrieval::search::search_entities with an EntityKind filter", + ), + ( + "src/openhuman/memory/tools/search/chunk_context.rs", + Verdict::NeedsWiderSeam, + "get_chunk / list_chunks by id and source, plus source_scope::chunk_source_allowed", + ), + ( + "src/openhuman/memory/tools/search/hybrid_search.rs", + Verdict::NeedsWiderSeam, + "constructs a UnifiedMemory directly and reads MemoryItemKind; the seam has no constructor door", + ), + ( + "src/openhuman/memory/tools/search/vector_search.rs", + Verdict::NeedsWiderSeam, + "vector chunk search over ListChunksQuery, plus the source_scope task-local", + ), +]; + +/// True for source files the lint deliberately does not scan. +/// +/// By-path only, matching [`super::bypass_allowlist_tests`] — see that module +/// for why inline `#[cfg(test)]` blocks are left in scope rather than +/// brace-tracked. +fn is_test_path(path: &Path) -> bool { + if path.components().any(|c| c.as_os_str() == "test_support") { + return true; + } + match path.file_name().and_then(|n| n.to_str()) { + Some(name) => name == "tests.rs" || name.ends_with("_tests.rs"), + None => false, + } +} + +fn collect_rs_files(dir: &Path, out: &mut Vec) { + let Ok(entries) = std::fs::read_dir(dir) else { + return; + }; + for entry in entries.flatten() { + let path = entry.path(); + if path.is_dir() { + collect_rs_files(&path, out); + } else if path.extension().is_some_and(|ext| ext == "rs") && !is_test_path(&path) { + out.push(path); + } + } +} + +/// Every repo-relative path in this crate's `src` that names the engine crate +/// outside a comment. +fn scan() -> BTreeSet { + let root = Path::new(env!("CARGO_MANIFEST_DIR")); + let mut files = Vec::new(); + collect_rs_files(&root.join("src"), &mut files); + + let mut found = BTreeSet::new(); + for path in &files { + let Ok(text) = std::fs::read_to_string(path) else { + continue; + }; + let rel = path + .strip_prefix(root) + .unwrap_or(path) + .to_string_lossy() + .replace('\\', "/"); + for line in text.lines() { + if line.trim_start().starts_with("//") { + continue; + } + if line.contains(NEEDLE) { + found.insert(rel.clone()); + break; + } + } + } + found +} + +fn allowed_set() -> BTreeSet { + ALLOWED + .iter() + .map(|(path, _, _)| (*path).to_string()) + .collect() +} + +fn render(paths: impl IntoIterator) -> String { + paths.into_iter().map(|p| format!("\n {p}")).collect() +} + +/// A scanner that silently found nothing would turn every other test here into +/// a rubber stamp, so refuse to pass vacuously. +/// +/// `memory/mod.rs` is the most stable pin available: it is the re-export block +/// itself, so it names the crate by construction. If the scanner stops seeing +/// it, the scanner is broken — fix it, do not relax this assertion. +#[test] +fn direct_reference_scanner_is_not_vacuous() { + let found = scan(); + assert!( + found.contains("src/openhuman/memory/mod.rs"), + "scanner found no direct engine reference in memory/mod.rs, which is the re-export block; \ + the scanner is broken" + ); + assert!( + found.len() > 20, + "scanner found only {} files; expected the full direct-reference surface", + found.len() + ); +} + +/// **The ratchet.** A new file naming `tinymemory_core::` fails here. +/// +/// If the new call is genuinely unavoidable, add it to [`ALLOWED`] with a +/// [`Verdict`] and a reason. If it is not, route it through +/// `CoreContext::memory()` and the `MemoryProvider` seam. +#[test] +fn no_new_files_call_the_engine_directly() { + let found = scan(); + let allowed = allowed_set(); + let unexpected: Vec = found.difference(&allowed).cloned().collect(); + assert!( + unexpected.is_empty(), + "new direct `tinymemory_core::` reference(s) — route these through the MemoryProvider seam, \ + or add them to ALLOWED with a Verdict and a reason:{}", + render(unexpected) + ); +} + +/// The staleness half. An allowlist that outlives its entries rots into dead +/// strings that document nothing — the same failure `INTENTIONALLY_NOT_FORWARDED` +/// guards against. A migrated file must be *removed* from the list, so the +/// count is always the real one. +#[test] +fn allowlist_has_no_stale_entries() { + let found = scan(); + let allowed = allowed_set(); + let stale: Vec = allowed.difference(&found).cloned().collect(); + assert!( + stale.is_empty(), + "ALLOWED names file(s) that no longer reference the engine — delete these entries so the \ + ratchet reflects reality:{}", + render(stale) + ); +} + +/// Every entry carries a reason, and no path is listed twice. A blank reason is +/// an allowlist entry that documents nothing, which is what the list exists to +/// prevent. +#[test] +fn allowlist_entries_are_well_formed() { + let mut seen = BTreeSet::new(); + for (path, _, reason) in ALLOWED { + assert!( + !reason.trim().is_empty(), + "{path} is allowlisted with no reason" + ); + assert!( + seen.insert(*path), + "{path} is listed twice; one entry per file" + ); + } +} + +/// The migration to-do list must be empty, and stay empty by being *worked* +/// rather than re-labelled. +/// +/// A [`Verdict::SeamExpressible`] entry says "the seam already covers this and +/// nobody moved it". That is a bug with a known fix, so it fails here rather +/// than sitting in a list nobody reads. Downgrading an entry to +/// [`Verdict::NeedsWiderSeam`] to silence this is the one edit that would make +/// the lint lie — [`no_new_files_call_the_engine_directly`] would still pass, +/// and the gap would vanish from view. +#[test] +fn nothing_is_left_migratable() { + let pending: Vec<&str> = ALLOWED + .iter() + .filter(|(_, verdict, _)| *verdict == Verdict::SeamExpressible) + .map(|(path, _, _)| *path) + .collect(); + assert!( + pending.is_empty(), + "these files can already be expressed through MemoryProvider and should be migrated: {pending:?}" + ); +} + +/// The blocked set is the upstream ask, so it must be non-empty for as long as +/// the engine is still linked — and empty when it is not. +/// +/// This is the test that makes "`tinymemory-core` left the build" self-proving: +/// on the day the crate is dropped, [`scan`] returns nothing, `ALLOWED` empties, +/// and this assertion is what forces the module docs above to be rewritten +/// rather than left describing a world that no longer exists. +#[test] +fn the_blocked_set_matches_the_engine_still_being_linked() { + let blocked = ALLOWED + .iter() + .filter(|(_, verdict, _)| *verdict == Verdict::NeedsWiderSeam) + .count(); + let host_side = ALLOWED + .iter() + .filter(|(_, verdict, _)| *verdict == Verdict::HostSide) + .count(); + assert!( + blocked > 0 || host_side > 0, + "nothing references tinymemory-core any more — drop the path dependency from Cargo.toml, \ + remove its cargo-machete `ignored` entry, ratchet scripts/kernel-floor.limits, and rewrite \ + this module's docs (#5560)" + ); +} diff --git a/src/openhuman/memory/guard/provider_tests.rs b/src/openhuman/memory/guard/provider_tests.rs index 16852aa182..53e053b604 100644 --- a/src/openhuman/memory/guard/provider_tests.rs +++ b/src/openhuman/memory/guard/provider_tests.rs @@ -5,13 +5,13 @@ use super::*; use std::sync::Arc; use crate::openhuman::memory::api::capabilities::{Capabilities, Capability}; -use crate::openhuman::memory::api::null::NullMemoryProvider; use crate::openhuman::memory::api::provider::types::SourceScope; use crate::openhuman::memory::api::provider::{ audit_provider, MemoryCore, MemoryPortability, MemoryProvider, MemoryRecall, }; use crate::openhuman::memory::api::recall::OwnedRecallOpts; use crate::openhuman::memory::api::types::{MemoryCategory, MemoryTaint}; +use tinymemory_api::null::NullMemoryProvider; use crate::core::bus::BUS; use crate::core::events::DomainEvent; diff --git a/src/openhuman/memory/mod.rs b/src/openhuman/memory/mod.rs index d176ac3c0a..cc66c2226e 100644 --- a/src/openhuman/memory/mod.rs +++ b/src/openhuman/memory/mod.rs @@ -57,9 +57,13 @@ pub mod sync; pub mod tool_memory; pub mod tree; +#[cfg(test)] +mod api_identity_tests; #[cfg(test)] mod bypass_allowlist_tests; #[cfg(test)] +mod direct_engine_refs_tests; +#[cfg(test)] mod profile_conn_guard_tests; #[cfg(test)] mod seam_integration_tests;