Skip to content

MeshJob: Stream subscription mode — subscribe_events async iterator (Python) #1046

Description

@dhyansraj

Summary

Extend the MeshJob event-injection surface (#1032) with Stream[T] subscription mode for the Python SDK: mesh.jobs.subscribe_events(job_id, types=, after=, long_poll_secs=) -> AsyncIterator[dict].

Mirrors the recv_event/send_event vertical (#1041 Python, #1043 TS, #1045 Java). This issue covers the Python runtime; TypeScript and Java verticals will follow as separate issues to keep PRs reviewable.

Motivation

recv_event (consumer side, drained once-per-message via JobController.recv_event) and send_event (producer side, fire-and-forget via JobProxy.send_event) cover the point-to-point event injection contract. They do not cover observer/fan-out patterns where a third party (or the commissioner itself, after the original commission/call returned) wants to watch a job's event stream without consuming from it.

Stream subscription mode adds a non-destructive read API:

  • Reads the registry's per-job event log (JobEvent ent table, (job_id, seq) indexed).
  • Resumes from a caller-supplied after cursor — late subscribers replay history; reconnecting subscribers resume from their last seq.
  • Long-polls the registry (wait param on list_job_events) so idle subscribers don't busy-loop.
  • Yields dicts shaped {seq, type, payload, ts}.
  • Terminates naturally when the job reaches a terminal state (registry signals end-of-stream once no more events can be appended).

This is the symmetric counterpart to send_event on the producer side: producers push, observers read.

Surface

async for event in mesh.jobs.subscribe_events(job_id, types=["progress"], after=0):
    print(event["seq"], event["payload"])
  • types: optional filter list (server-side filter via existing ListJobEvents query param).
  • after: cursor; subscribers persist this to resume across restarts.
  • long_poll_secs: controls per-request wait parameter.

Scope

  • Rust core: JobProxy::list_events(after, types, wait) -> Vec<JobEvent> — thin pass-through to backend.list_job_events. No new TaskBackend trait method.
  • pyo3: PyJobProxy.list_events binding with parse_timeout_secs guard (NaN/Inf/negative reject, try_from_secs_f64 overflow).
  • Python SDK: mesh.jobs.subscribe_events async generator; uses the LRU JobProxy cache from #1041 (no new cache).
  • Integration test: tc27 — producer running run_until_done pushes 3 events; commissioner observer subscribes via subscribe_events from after=0 and asserts each event seq/payload is observed end-to-end. Reuses existing long-task-provider / long-task-consumer fixtures (extends with run_until_done + commission_subscribe_observer tools).

Out of scope for this issue (separate verticals):

Acceptance

  • tsuite run --suite-path tests/integration --uc uc21_meshjob --parallel 4 → 21/21 pass (20 baseline + 1 new tc27).
  • Python unit tests cover: server filter pass-through, after cursor semantics, terminal-state stream end, JobProxy cache reuse.
  • Rust unit tests cover JobProxy::list_events against the in-memory backend.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions