Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
78 changes: 78 additions & 0 deletions docs/adr/0106-source-bound-connected-provider-loading.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
---
status: proposed
date: 2026-10-30
related: 0078, 0099
---

# 0106: Bind connected providers to source-only loading

## Context and Problem Statement

Connected-read reviews bind a selected provider and source, but ordinary discovery imports plugin entry points before
recording their origin. An in-place update can therefore make disk contents differ from code already executing. Python's
normal timestamp bytecode cache can also load older code after a same-size, same-timestamp source replacement. A plugin
cannot establish this boundary by checking its own source after its initializer has already run.

## Considered Options

- **Host-controlled source-only provider loading with a closed graph** (proposed).
- **Plugin self-reported file hashes**: the verifier itself may already be stale.
- **Assume installed files or version metadata identify loaded code**: neither proves which code was executed.
- **Rely on managed immutable deployments**: desirable, but not yet enforced for ordinary installed plugins.

## Decision Outcome

Chosen option: **host-controlled source-only provider loading with a closed graph**, because a trusted host must capture
and compile the bytes before provider code runs. Ordinary plugin discovery remains unchanged. A connected provider is
eligible only through a separately reviewed admission boundary that binds the exact owner-selected provider, source
graph, account and operation. Installed `heinzel.connected_sources` metadata may request capture only when it matches
the same distribution's ordinary module entry point; it does not assert trust, grant access, or select additional runtime
dependencies. The host checks the installed manifest and owns the trusted runtime baseline and the process-lifetime
capture catalog. Ordinary boot enables the guard; providers lacking a verifiable declaration remain available to ordinary
heads but cannot use the guarded connected-read path. Unknown dependencies, preloaded code, native extensions, unsupported loaders, graph
changes and ambiguous import ownership fail closed. The host rechecks the graph and grant before each operation and
fences later in-flight calls when changes are observed. A running request cannot be retroactively undone.

Captured plugin code may to import only already-loaded host runtime modules and attributes sealed
after builtin discovery. It checks standard-library origins and loaders, the installed Pydantic manifest, and the
trusted running core's source root. A scoped import function returns the exact sealed objects or loads the plugin's
captured package; missing runtime dependencies are never loaded on the plugin's behalf. Sealed objects and specs are
rechecked before use. An ineligible plugin can still use ordinary discovery, but cannot use the source-guarded
connected-read path. An embedder constructing `Kernel` without a source catalog retains responsibility for provider
trust and does not obtain this guard.

The public `captured_source_active()` predicate reports whether its exact calling module is executing under the host's
source loader with a verified snapshot and sealed runtime. It is false outside that initialization context and does
not grant permission or establish publisher trust. A plugin may use it to omit its captured-read service on older
cores or ordinary loading; current source binding and human approval remain separate requirements.

The fixed host initialization imports `email.policy` for general MIME parsing before external plugin discovery. That
bootstrap and the Python interpreter's own imports are part of the trusted host runtime, not verified plugin code;
the source guard does not claim to prevent shadowing before that trusted initialization. No plugin declaration can
extend this bootstrap. A missing runtime module or attribute makes the plugin ineligible for guarded reads.

A source digest establishes local continuity, not publisher authentication, immunity to same-user interference, or
provider account authority. Isolated provider processes or verified immutable deployments may be needed to preserve
ordinary module identity while applying this boundary. Do not reload or replace live plugin modules to obtain a digest.

### Consequences

This adds startup, compatibility and dependency-inventory work. Existing connected providers must add a matching
metadata declaration and use a verifiable non-editable installed distribution before guarded connected reads work;
unsupported plugins continue to work through ordinary heads but cannot receive this authority. Default boot enables
source capture for matching installed declarations; manually constructed kernels without the catalog remain an explicit
embedding trust boundary. The loader supports bounded pure-Python package trees and an already-loaded host runtime.
Native plugin extensions and unloaded runtime dependencies remain ineligible for guarded reads.

This is a continuity mechanism for owner-selected plugin code, not a Python sandbox or publisher attestation. Static
import inspection and the scoped importer bind ordinary imports to captured bytes or sealed runtime objects. Python
reflection and dynamic imports remain capabilities of the already trusted installed code; the regression demonstrating
a dynamic helper import records that limit. Do not present the digest or capture predicate as proof that arbitrary
plugin code is safe, immutable against a same-user adversary, or authorized to use an account. Exact human-approved
account, operation and source grants remain necessary before a connected read.

### Confirmation

Focused tests exercise source-only compilation despite stale bytecode, refusal of already loaded packages and detection
of source changes. Independent review must establish closed dependency graphs, verified provider identity, current
binding checks and cancellation semantics before any real provider is admitted.
1 change: 1 addition & 0 deletions docs/adr/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -196,3 +196,4 @@ of inventing them. Two factual corrections came with it: ADR 0004 names the plug
| [0098](0098-pages-showcases-the-product-and-discovers-public-plugin-metadata.md) | Pages showcases the product and discovers public plugin metadata | proposed | 2026-10-30 |
| [0099](0099-reviewed-connected-reads-and-local-agent-notifications.md) | Review connected reads and local agent notifications separately | proposed | 2026-10-30 |
| [0104](0104-reviewed-item-evidence-for-local-notifications.md) | Bind notification deduplication to observed source items | proposed | 2026-10-30 |
| [0106](0106-source-bound-connected-provider-loading.md) | Bind connected providers to source-only loading | proposed | 2026-10-30 |
2 changes: 2 additions & 0 deletions src/heinzel/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -184,6 +184,7 @@
WidgetLet,
Window,
capability_read_schema,
captured_source_active,
clean_text,
draft_reporting_lines,
free_busy_people,
Expand Down Expand Up @@ -366,6 +367,7 @@
"Window",
"__version__",
"capability_read_schema",
"captured_source_active",
"clean_text",
"draft_reporting_lines",
"free_busy_people",
Expand Down
4 changes: 4 additions & 0 deletions src/heinzel/kernel/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@
from heinzel.kernel._bounded_process import ProcessExecute, ProcessOutcome, run_bounded_process, signal_process_group
from heinzel.kernel._capabilities import CAPABILITIES, MODES, NO_ADAPTER, Capability, Mode, Requirement
from heinzel.kernel._capability_schema import capability_read_schema
from heinzel.kernel._captured_source import captured_source_active
from heinzel.kernel._chat import ActionGate, ProposedCall
from heinzel.kernel._chat_services import (
CheckInChatContext,
Expand Down Expand Up @@ -241,6 +242,7 @@
StreamProvider,
ThemeChoice,
)
from heinzel.kernel._source_loading import SourceGuardedConnectedService
from heinzel.kernel._telemetry import (
HEINZEL_ID,
VERBS,
Expand Down Expand Up @@ -499,6 +501,7 @@
"SoundStream",
"Soundscape",
"SoundscapeLet",
"SourceGuardedConnectedService",
"Span",
"Status",
"StatusItem",
Expand Down Expand Up @@ -545,6 +548,7 @@
"Window",
"boot",
"capability_read_schema",
"captured_source_active",
"check_repository_url",
"check_target",
"clean_text",
Expand Down
14 changes: 11 additions & 3 deletions src/heinzel/kernel/_boot.py
Original file line number Diff line number Diff line change
@@ -1,14 +1,22 @@
"""Boot: load the configuration, discover every module and build the kernel. What heads call on start-up."""

import email.policy

from heinzel.kernel._config import KERNEL_SECTION, Config
from heinzel.kernel._discovery import discover
from heinzel.kernel._kernel import Kernel
from heinzel.kernel._settings import Settings
from heinzel.kernel._source_loading import SourceDiscovery

_installed_sources = SourceDiscovery(installed=True)
# Fixed host bootstrap for MIME parsing, independent of any plugin declaration.
_host_runtime = (email.policy,)


def boot(settings: Settings | None = None) -> Kernel:
def boot(settings: Settings | None = None, *, source_discovery: "SourceDiscovery | None" = None) -> Kernel:
"""Discover every module and return a ready kernel."""
source_discovery = source_discovery or _installed_sources
config = Config.load()
settings = settings or config.settings(Settings, KERNEL_SECTION)
records, errors = discover(plugins=settings.plugins_enabled)
return Kernel(settings, records, errors, config)
records, errors = discover(plugins=settings.plugins_enabled, source_discovery=source_discovery)
return Kernel(settings, records, errors, config, source_discovery=source_discovery)
36 changes: 36 additions & 0 deletions src/heinzel/kernel/_captured_source.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
"""Host execution marker for source-captured plugin initialization."""

import inspect
from collections.abc import Callable, Generator
from contextlib import contextmanager
from contextvars import ContextVar

_active: ContextVar[tuple[dict[str, object], Callable[[], bool]] | None] = ContextVar("captured_source", default=None)


@contextmanager
def executing_captured_source(module_globals: dict[str, object], verify: Callable[[], bool]) -> Generator[None]:
"""Set the host-owned marker only for one captured module execution."""
token = _active.set((module_globals, verify))
try:
yield
finally:
_active.reset(token)


def captured_source_active() -> bool:
"""Whether the calling module is currently executing under the host's captured source loader.

This reports source-loading context only. It does not grant access, establish publisher trust, or approve a read.
"""
marker = _active.get()
frame = inspect.currentframe()
try:
if marker is None or frame is None or frame.f_back is None or frame.f_back.f_globals is not marker[0]:
return False
try:
return marker[1]()
except Exception: # noqa: BLE001 - an unavailable source must not claim capture
return False
finally:
del frame
17 changes: 16 additions & 1 deletion src/heinzel/kernel/_discovery.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@
from collections.abc import Iterator
from importlib.metadata import Distribution, PackageNotFoundError, distribution, entry_points
from types import ModuleType
from typing import TYPE_CHECKING

from heinzel import __version__
from heinzel.kernel._installation import CORE_DISTRIBUTION
Expand All @@ -22,10 +23,15 @@
"""Project-URL labels naming the source repository, normalized as in PEP 753, most specific first."""
ENTRY_POINT_GROUP = "heinzel.modules"

if TYPE_CHECKING:
from heinzel.kernel._source_loading import SourceDiscovery

logger = logging.getLogger(__name__)


def discover(*, plugins: bool = True) -> tuple[list[ModuleRecord], dict[str, str]]:
def discover(
*, plugins: bool = True, source_discovery: "SourceDiscovery | None" = None
) -> tuple[list[ModuleRecord], dict[str, str]]:
"""Find built-in modules and, unless disabled, plugin modules.

A plugin that fails to import is reported in the returned errors and skipped, so one broken plugin never takes
Expand All @@ -47,6 +53,15 @@ def discover(*, plugins: bool = True) -> tuple[list[ModuleRecord], dict[str, str
builtin=False,
url=repository_of(dist) if dist else None,
)
if source_discovery is not None:
try:
captured = source_discovery.try_capture(entry_point, origin)
except Exception as exc: # noqa: BLE001 - never reexecute a failed captured initializer
_skip(errors, entry_point.name, f"{type(exc).__name__}: {exc}")
continue
if captured is not None:
records.extend(captured.records)
continue
try:
target = entry_point.load()
except Exception as exc: # noqa: BLE001 - isolate broken plugins
Expand Down
8 changes: 7 additions & 1 deletion src/heinzel/kernel/_kernel.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
from dataclasses import dataclass, replace
from functools import partial
from pathlib import Path
from typing import Annotated, Any, cast, get_args, get_origin
from typing import TYPE_CHECKING, Annotated, Any, cast, get_args, get_origin

import pluggy
from pydantic import ConfigDict, ValidationError, validate_call
Expand Down Expand Up @@ -51,6 +51,9 @@
from heinzel.kernel._settings import Settings
from heinzel.kernel._spec import PROJECT_NAME, Spec

if TYPE_CHECKING:
from heinzel.kernel._source_loading import SourceDiscovery

logger = logging.getLogger(__name__)

ADAPTER_ONLY_LETS = (ProviderLet, RequirementLet, SettingsLet, CredentialLet, LayerLet)
Expand Down Expand Up @@ -106,6 +109,7 @@ def __init__(
records: Iterable[ModuleRecord] = (),
errors: Mapping[str, str] | None = None,
config: Config | None = None,
source_discovery: "SourceDiscovery | None" = None,
) -> None:
"""Register modules. Disabled modules (and their submodules) are blocked, not registered.

Expand All @@ -114,6 +118,7 @@ def __init__(
records: Modules to register, typically from `heinzel.kernel.discover`.
errors: Plugins that failed to load, by entry point name, for reporting.
config: Configuration files and their content; none (defaults and environment only) if not given.
source_discovery: Optional host-owned source continuity catalog.

A plugin module that collides with a registered one — same qualified name, a submodule named like one of its
parent's operations, or a service type provided twice — is skipped and reported in `errors`, so one bad
Expand All @@ -123,6 +128,7 @@ def __init__(
HeinzelError: If built-in modules collide.
"""
self.config = config or Config.empty()
self.source_discovery = source_discovery
self.settings = settings or self.config.settings(Settings, KERNEL_SECTION)
self.errors: dict[str, str] = dict(errors or {})
self._manager = pluggy.PluginManager(PROJECT_NAME)
Expand Down
Loading
Loading