Skip to content

Commit ab638cc

Browse files
Serve the Composio connection bootstrap over the bus
`ComposioProvider::on_connection_created` is the last thing in the memory seam that a host can only reach by naming the engine. OpenHuman's connection-created subscriber builds a `tinymemory_core` `ProviderContext` purely to hand it to that hook, which is why `memory/sync/composio/ providers/context_ext.rs` still imports the engine crate — and why the crate cannot leave the product dependency graph (openhuman#5560). `BootstrapConnection(toolkit, connection_id)` closes it. It sits in `MemorySourceSync` beside `RunConnectionSync`, and deliberately not inside it: a sync moves items and runs many times, a bootstrap establishes what a sync then assumes and runs once. They also fail differently, and a caller can only decline to stop syncing over a failed bootstrap if it can tell the two apart. Caps are left at their defaults rather than accepted as arguments. No provider overrides the hook today and the default impl reads neither `max_items` nor `sync_depth_days` — they cap how much a *sync* walks, and giving a bootstrap a walk budget would imply it walks. The registry init is the part worth reviewing. `get_provider` resolves through a process-global that the *host's* boot used to fill; a `cdylib` has its own statics, so inside the module that registry is empty and `get_provider` answers `None` rather than erroring — every bootstrap would have reported "no composio provider registered" over a working connection, with nothing in a build or a type check saying so. The module now calls `init_default_providers` beside the other seams it installs for exactly this reason. The existing sync pipeline resolves its provider a different way and never depended on it, so this is a new requirement rather than a fix. Two tests, because the failure is invisible to compilation: a unit test that the registry really registers what it claims, and an E2E against the dlopen'd artifact asserting `BootstrapConnection` fails because no client resolves and never because the registry is empty. Those two are one line apart in the engine and worlds apart in meaning. Note for reviewers: `cargo clippy --workspace --all-targets -- -D warnings` is already red on main with six `unwrap`/`unwrap_err` findings in `crates/tinymemory-documents/src/convert/test.rs`. Untouched here.
1 parent 413b07c commit ab638cc

9 files changed

Lines changed: 196 additions & 2 deletions

File tree

crates/tinymemory-api/src/null_tests.rs

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -514,6 +514,10 @@ fn every_optional_method_fails_with_its_advertised_family_name() {
514514
block_on(driver.run_connection_sync("gmail", "conn-1")),
515515
Capability::SourceSync,
516516
);
517+
assert_unsupported(
518+
block_on(driver.bootstrap_connection("gmail", "conn-1")),
519+
Capability::SourceSync,
520+
);
517521
assert_unsupported(
518522
block_on(driver.source_sync_state("gmail", "conn-1")),
519523
Capability::SourceSync,

crates/tinymemory-api/src/provider/sync.rs

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -155,6 +155,50 @@ pub trait MemorySourceSync: Send + Sync {
155155
Err(MemoryError::unsupported(Capability::SourceSync))
156156
}
157157

158+
/// Run one connection's first-time bootstrap.
159+
///
160+
/// What a host's "this connection was just authorised" event reaches. The
161+
/// driver resolves the provider for `toolkit` and runs its bootstrap: at
162+
/// minimum fetching and persisting the account profile, and for providers
163+
/// that override it, registering triggers or seeding labels as well.
164+
///
165+
/// # Why this is not part of [`Self::run_connection_sync`]
166+
///
167+
/// A sync moves items and is expected to run many times; a bootstrap
168+
/// establishes the things a sync then assumes and is expected to run once.
169+
/// Folding them together would either re-register triggers on every sync
170+
/// or leave a connection whose first sync silently has no profile behind
171+
/// it — and the two also fail differently, which is the more practical
172+
/// reason: a bootstrap that fails should not stop items from syncing, and
173+
/// a caller can only make that choice if it can tell the two apart.
174+
///
175+
/// # Not idempotent, and the caller owns that
176+
///
177+
/// Calling it twice runs the provider's bootstrap twice. Providers whose
178+
/// bootstrap is a trigger registration should make that registration
179+
/// idempotent themselves; the contract does not promise it, because a
180+
/// driver cannot know whether a second call means "retry the one that
181+
/// failed" or "the connection was re-authorised".
182+
///
183+
/// # Errors
184+
///
185+
/// [`MemoryError::Invalid`] for a toolkit the driver has no provider for,
186+
/// or a connection it cannot resolve — the same rule
187+
/// [`Self::run_connection_sync`] follows, and for the same reason: a
188+
/// silent success over a connection that can never bootstrap is worse than
189+
/// an error.
190+
///
191+
/// [`MemoryError::Unsupported`] from a driver that serves this family but
192+
/// not this member. Otherwise the provider's own failure.
193+
async fn bootstrap_connection(
194+
&self,
195+
toolkit: &str,
196+
connection_id: &str,
197+
) -> Result<(), MemoryError> {
198+
let _ = (toolkit, connection_id);
199+
Err(MemoryError::unsupported(Capability::SourceSync))
200+
}
201+
158202
async fn source_sync_state(
159203
&self,
160204
toolkit: &str,

crates/tinymemory-bus/src/lib.rs

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22
//! the members that carry them.
33
//!
44
//! TinyMemory ships as a loadable `TinyBus` module: `crates/tinymemory-module`
5-
//! exports one object with 121 members on it, built as a `cdylib`. A host that
5+
//! exports one object with 122 members on it, built as a `cdylib`. A host that
66
//! loads it — OpenHuman — can call into it but cannot `use` anything out of it,
77
//! so the payload vocabulary has to be published as an ordinary library. This
88
//! is that library.

crates/tinymemory-bus/src/names.rs

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -294,6 +294,8 @@ pub mod methods {
294294
/// `RunSourceSync` — run one configured memory source's sync now,
295295
/// whatever kind it is.
296296
pub const RUN_SOURCE_SYNC: &str = "RunSourceSync";
297+
/// `BootstrapConnection` — run one connection's first-time bootstrap.
298+
pub const BOOTSTRAP_CONNECTION: &str = "BootstrapConnection";
297299
/// `SourceSyncState` — the persisted cursor and budget for one connection.
298300
pub const SOURCE_SYNC_STATE: &str = "SourceSyncState";
299301
/// `SyncAuditLog` — past sync runs, newest first.
@@ -319,7 +321,7 @@ pub mod methods {
319321
/// The order matters: `tinybus`'s `Interface::members()` returns declaration
320322
/// order, and the module compares the two sequences directly rather than as
321323
/// sets, so a reordering is caught alongside an addition or a removal.
322-
pub const METHODS: [&str; 121] = [
324+
pub const METHODS: [&str; 122] = [
323325
methods::DRIVER_ID,
324326
methods::CAPABILITIES,
325327
methods::HEALTH,
@@ -433,6 +435,7 @@ pub const METHODS: [&str; 121] = [
433435
methods::DIAGNOSE,
434436
methods::RUN_CONNECTION_SYNC,
435437
methods::RUN_SOURCE_SYNC,
438+
methods::BOOTSTRAP_CONNECTION,
436439
methods::SOURCE_SYNC_STATE,
437440
methods::SYNC_AUDIT_LOG,
438441
methods::ESTIMATE_SYNC_COST_USD,

crates/tinymemory-module/src/lib.rs

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -170,6 +170,15 @@ async fn setup(connection: Connection, mut config: ModuleConfig) -> BusResult<()
170170
// credential strip above, because this is the seam that hands the config
171171
// back out to the engine repeatedly.
172172
tinymemory_core::config_loader::set_config_loader(Arc::new(ModuleConfigLoader::new(&config)));
173+
// The Composio provider registry is a process-global too, and it is the one
174+
// the host used to fill on its own boot. This process has its own statics,
175+
// so without this line `get_provider` answers `None` for every toolkit
176+
// inside the module — and it answers `None` rather than failing to build,
177+
// which is why nothing above catches it. `BootstrapConnection` is the
178+
// member that reads it; the sync pipeline resolves its provider a different
179+
// way and is unaffected either way. Idempotent by the registry's own
180+
// contract, so a second call from a host that also inits is harmless.
181+
tinymemory_core::sync::composio::providers::init_default_providers();
173182
host::install(connection.clone());
174183
// The two seams no bus interface serves, and no local answer can honestly
175184
// stand in for. Both degraded in silence rather than with a named cause;
@@ -763,6 +772,7 @@ mod exports {
763772
// live here; these are the on-demand half plus what past runs cost.
764773
"RunConnectionSync",
765774
"RunSourceSync",
775+
"BootstrapConnection",
766776
"SourceSyncState",
767777
"SyncAuditLog",
768778
"EstimateSyncCostUsd",

crates/tinymemory-module/src/service/mod.rs

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -65,6 +65,7 @@
6565
//! Diagnose() -> Diagnosis
6666
//!
6767
//! RunConnectionSync(toolkit, connection_id) -> SyncRunOutcome
68+
//! BootstrapConnection(toolkit, connection_id) -> ()
6869
//! RunSourceSync(source_id) -> SyncRunOutcome
6970
//! SourceSyncState(toolkit, connection_id) -> Option<SourceSyncState>
7071
//! SyncAuditLog(limit) -> [SyncAuditEntry]
@@ -1750,6 +1751,19 @@ impl MemoryService {
17501751
.map_err(|error| into_bus_error(&error))
17511752
}
17521753

1754+
/// Run one connection's first-time bootstrap.
1755+
///
1756+
/// Beside `RunConnectionSync` rather than inside it: a sync moves items and
1757+
/// runs many times, a bootstrap establishes what a sync then assumes and
1758+
/// runs once. They also fail differently, and a caller can only decline to
1759+
/// stop syncing over a failed bootstrap if it can tell the two apart.
1760+
async fn bootstrap_connection(&self, toolkit: String, connection_id: String) -> BusResult<()> {
1761+
require_family!(self, as_source_sync, Capability::SourceSync)
1762+
.bootstrap_connection(&toolkit, &connection_id)
1763+
.await
1764+
.map_err(|error| into_bus_error(&error))
1765+
}
1766+
17531767
/// The persisted cursor, dedup and budget state for one connection.
17541768
///
17551769
/// `None` is "never synced", which is a state and not an error — a status

crates/tinymemory-module/src/service/test.rs

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -743,6 +743,12 @@ async fn the_two_new_families_are_gated_on_their_own_capability() {
743743
.expect_err("a driver without the source-sync family must refuse");
744744
assert_eq!(refusal(error), wire::UNSUPPORTED);
745745

746+
let error = service
747+
.bootstrap_connection("gmail".to_string(), "conn-1".to_string())
748+
.await
749+
.expect_err("a driver without the source-sync family must refuse");
750+
assert_eq!(refusal(error), wire::UNSUPPORTED);
751+
746752
let error = service
747753
.coding_session_status()
748754
.await
@@ -763,3 +769,35 @@ async fn the_two_new_families_are_gated_on_their_own_capability() {
763769
.expect_err("a driver without the maintenance family must refuse");
764770
assert_eq!(refusal(error), wire::UNSUPPORTED);
765771
}
772+
773+
/// The Composio provider registry is filled by this process, not by the host.
774+
///
775+
/// It is a process-global, and before the memory engine moved into a module the
776+
/// host's own boot was what called `init_default_providers`. A `cdylib` has its
777+
/// own statics, so that call does nothing for this process — and the failure is
778+
/// silent in the worst way: `get_provider` answers `None` rather than erroring,
779+
/// so `BootstrapConnection` would report "no composio provider registered for
780+
/// 'gmail'" on a perfectly good connection, and nothing in a build or a type
781+
/// check would have said so.
782+
///
783+
/// This pins the call the module's startup makes. It is deliberately asserting
784+
/// a toolkit the registry's own `init_default_providers` registers rather than
785+
/// an arbitrary string, so that a rename upstream fails here instead of in the
786+
/// field.
787+
#[test]
788+
fn the_default_composio_providers_populate_the_registry() {
789+
use tinymemory_core::sync::composio::providers::{get_provider, init_default_providers};
790+
791+
init_default_providers();
792+
793+
assert!(
794+
get_provider("gmail").is_some(),
795+
"init_default_providers must register the gmail provider; BootstrapConnection \
796+
resolves through this registry and answers Invalid when it is empty"
797+
);
798+
assert!(
799+
get_provider("__definitely_not_a_real_toolkit__").is_none(),
800+
"an unregistered toolkit must stay unregistered — otherwise the assertion above \
801+
would pass against a registry that returns something for everything"
802+
);
803+
}

crates/tinymemory-module/tests/module_e2e.rs

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -695,6 +695,7 @@ const EXPECTED_METHODS: &[&str] = &[
695695
"Diagnose",
696696
"RunConnectionSync",
697697
"RunSourceSync",
698+
"BootstrapConnection",
698699
"SourceSyncState",
699700
"SyncAuditLog",
700701
"EstimateSyncCostUsd",
@@ -1671,3 +1672,39 @@ async fn portability_and_lifecycle_round_trip(bus: &tinybus::Proxy) {
16711672
.expect("Shutdown timed out")
16721673
.expect("Shutdown");
16731674
}
1675+
1676+
#[tokio::test]
1677+
#[ignore = "drives a real dlopen'ed module; must be the only such test in the process — see the module docs"]
1678+
async fn bootstrap_connection_finds_its_provider_registry_inside_the_module() {
1679+
// The Composio provider registry is a process-global that the *host's* boot
1680+
// used to fill. This module is a `cdylib` with its own statics, so unless
1681+
// its startup calls `init_default_providers` the registry here is empty —
1682+
// and `get_provider` answers `None` rather than erroring, so every
1683+
// `BootstrapConnection` would report "no composio provider registered" over
1684+
// a perfectly good connection. Nothing in a build, a type check or a unit
1685+
// test in the module's own workspace sees that, because they all run in a
1686+
// process the host has already initialised.
1687+
//
1688+
// So this asserts against the *loaded artifact*, and it asserts the
1689+
// distinction rather than success: with no Composio configured in a temp
1690+
// workspace the call is expected to fail, but it must fail because no
1691+
// client resolves, never because the registry is empty. Those two are one
1692+
// line apart in the engine and worlds apart in what they mean.
1693+
let workspace = tempfile::tempdir().expect("tempdir");
1694+
let (client, _host, _task) = admit_module(workspace.path()).await;
1695+
1696+
let result: Result<(), _> = proxy(&client)
1697+
.call(
1698+
"BootstrapConnection",
1699+
("gmail".to_string(), "conn-1".to_string()),
1700+
)
1701+
.await;
1702+
1703+
let error = result.expect_err("no Composio is configured in a temp workspace");
1704+
let rendered = format!("{error:?}");
1705+
assert!(
1706+
!rendered.contains("no composio provider registered"),
1707+
"the module's provider registry is empty — its startup did not call \
1708+
init_default_providers. Error was: {rendered}"
1709+
);
1710+
}

crates/tinymemory-tinycortex/src/engine/mod.rs

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2502,6 +2502,50 @@ impl MemorySourceSync for TinycortexProvider {
25022502
})
25032503
}
25042504

2505+
async fn bootstrap_connection(
2506+
&self,
2507+
toolkit: &str,
2508+
connection_id: &str,
2509+
) -> Result<(), MemoryError> {
2510+
use tinymemory_core::sync::composio::providers::{get_provider, ProviderContext};
2511+
2512+
// Same gate as `run_connection_sync`, and deliberately before the
2513+
// provider lookup: a toolkit with no pipeline cannot bootstrap into
2514+
// anything a later sync would read, so reporting it here names the
2515+
// real problem rather than "no provider".
2516+
ensure_syncable_toolkit(toolkit)?;
2517+
2518+
let provider = get_provider(toolkit).ok_or_else(|| {
2519+
MemoryError::Invalid(format!("no composio provider registered for '{toolkit}'"))
2520+
})?;
2521+
2522+
// `from_config` answers `None` when no Composio client resolves in
2523+
// either mode — the not-signed-in case. That is `Invalid` rather than a
2524+
// silent `Ok`: a caller that just authorised a connection and gets a
2525+
// success back would believe the profile was fetched.
2526+
let ctx = ProviderContext::from_config(
2527+
self.config.to_arc(),
2528+
toolkit,
2529+
Some(connection_id.to_string()),
2530+
)
2531+
.ok_or_else(|| {
2532+
MemoryError::Invalid(format!(
2533+
"no viable composio client for '{toolkit}'; connection {connection_id} \
2534+
cannot bootstrap"
2535+
))
2536+
})?;
2537+
2538+
// `max_items` / `sync_depth_days` are left at their defaults on
2539+
// purpose. They cap how much a *sync* walks; a bootstrap fetches one
2540+
// profile and registers what the provider needs, and giving it a walk
2541+
// budget would imply it walks.
2542+
provider.on_connection_created(&ctx).await.map_err(|error| {
2543+
MemoryError::Other(anyhow::anyhow!(
2544+
"bootstrap {toolkit} connection {connection_id}: {error}"
2545+
))
2546+
})
2547+
}
2548+
25052549
async fn source_sync_state(
25062550
&self,
25072551
toolkit: &str,

0 commit comments

Comments
 (0)