Skip to content

Add observation hierarchies with groups and multi-part payloads#52

Closed
Dig-Doug wants to merge 1 commit intomainfrom
groups
Closed

Add observation hierarchies with groups and multi-part payloads#52
Dig-Doug wants to merge 1 commit intomainfrom
groups

Conversation

@Dig-Doug
Copy link
Owner

Summary

  • Replace flat string labels with first-class Group types that form hierarchical containers for observations. Groups are observations themselves (ObservationType::Group) with parent pointers, enabling tree structures like request/response hierarchies.
  • Add multi-part payload support via ObservationPayloadHandle, allowing observations to accumulate named payloads over time (e.g., request then response).
  • Add group! macro, GroupBuilder, GroupHandle, SendGroup, and GroupId types across shared model, macros, and client crates.

API Changes

  • ObservationBuilder: .label()/.labels() removed, replaced with .group(&GroupHandle)
  • New named payload terminal methods: .named_payload(), .named_debug(), .named_raw_payload()
  • New AdditionalPayload uploader message for streaming named payloads to the server
  • Tracing layer uses GroupHandle::from_id with deterministic span ID → GroupId mapping
  • Axum request observer creates request/response group hierarchies per-request
  • Server templates updated from labels to group_ids

Test plan

  • cargo build --all-features — clean build, no warnings
  • cargo test --all-features — all 69 tests pass (unit + integration)
  • Axum layer tests updated to account for group observations in counts
  • Tracing layer tests updated to use group-based assertions
  • Manual smoke test with group! + observe!().named_payload() against live server

🤖 Generated with Claude Code

Replace flat string labels with first-class Group types that form hierarchical
containers for observations. Groups are observations themselves (ObservationType::Group)
with parent pointers, enabling tree structures like request/response hierarchies.

Add multi-part payload support via ObservationPayloadHandle, allowing observations
to accumulate named payloads over time (e.g., request then response).

Key changes:
- GroupId type, GroupBuilder, GroupHandle, SendGroup in shared/client
- group! macro for creating groups
- ObservationBuilder: .label()/.labels() replaced with .group(&GroupHandle)
- Named payload methods: .named_payload(), .named_debug(), .named_raw_payload()
- AdditionalPayload uploader message for streaming named payloads
- Tracing layer uses GroupHandle::from_id with deterministic span ID mapping
- Axum request observer creates group hierarchies per-request
- All tests updated, server templates updated

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Multi-payload server support with protobuf storage

- Add PayloadId newtype to shared crate for uniquely identifying payloads
- Add prost-based protobuf types for server storage layer
- Rewrite storage traits to support multiple named payloads per observation
- Use sled prefix scan with metadata + inline payload key structure
- Update server API: list returns metadata-only, get returns inline data
- Add new blob route with payload_id param plus legacy route
- Update client to generate and send PayloadId with each payload
- Change multipart field format to {obs_id}:{payload_id}:{name}
- Update all tests for new multi-payload observation model

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Unify payload handling with PayloadBuilder and payload manifest

Remove mime_type/payload_size from Observation struct since these are
per-payload concerns, not observation-level metadata. All payloads now
flow through a single unified path instead of separate primary and
additional payload code paths.

Key changes:
- Add PayloadBuilder for creating named payloads in shared crate
- Replace ObservationWithPayload with separate Observation + Payload
  messages in the uploader channel
- Send payload_manifest JSON in multipart uploads so the server gets
  authoritative MIME types (falls back to heuristic for old clients)
- Add mime_type/size to GetPayload API response for per-payload metadata
- Fix named_payload sending payload twice (as both primary and additional)
- Update templates to derive type/size from per-payload data

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@Dig-Doug
Copy link
Owner Author

Superseded by stacked PRs #57, #63, #59, #60, #61, #62

@Dig-Doug Dig-Doug closed this Feb 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant