docs: evaluate Pi observability packages#401
Conversation
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 0b640b55ff
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
|
|
||
| ### 第三阶段：只有出现团队需求才接 SaaS/自建平台 | ||
|
|
||
| 若以后需要团队共享、跨机器搜索、评估和长期趋势，再从 `pi-langfuse`、Braintrust、Latitude 或 Grafana 中选择。个人本机调试阶段先用本地两款，部署和隐私成本最低。 |
There was a problem hiding this comment.
For the team/SaaS path, this recommendation is incomplete: I checked the Pi package catalog and it lists @posthog/pi (pi install npm:@posthog/pi, captures LLM generations, tool executions, and traces to PostHog with privacy mode) and pi-sentry-monitor (captures sessions, tool executions, token usage, and errors as Sentry traces). Since the doc says the search covered Pi registry, npm, and vendor docs, omitting these Pi-native packages can steer maintainers to Langfuse/Braintrust/Latitude/Grafana without considering available alternatives; please include or explicitly reject them.
Useful? React with 👍 / 👎.
| | [`@braintrust/pi-extension`](https://github.com/braintrustdata/braintrust-pi-extension) | Braintrust 官方仓库；npm `0.10.0`，2026-07-17；采集 session/turn/LLM/tool、compaction、token/cost/TTFT，提供 Web trace | 需要 Braintrust 账号并上传数据；刻意不记录完整 provider payload，适合指标与评估，不适合深挖上下文 | | ||
| | [`@latitude-data/pi-telemetry`](https://github.com/latitude-dev/latitude-llm/tree/development/packages/telemetry/pi) | Latitude 官方 monorepo；采集 prompt、assistant、system/context、provider 参数、tool 定义/参数/结果、token/cost，并发到 Latitude Web | 需要账号/项目/API key；不是本地 dashboard | | ||
| | [`@amaster.ai/pi-telemetry`](https://github.com/TGYD-helige/pi/tree/master/packages/pi-telemetry) | npm `0.1.6`，2026-07-20；支持 Langfuse 和通用 OTLP/HTTP，peer 要求当前 `@earendil-works/pi-coding-agent>=0.74.0` | 只是 exporter，没有自带 Web UI；还要选择并运行观测后端 | | ||
| | [`@grafana/agento11y-pi`](https://github.com/grafana/agento11y/tree/main/plugins/pi) | Grafana 官方 Agento11y Pi 插件，可把 conversation/model/token/tool/timing 发到 Grafana Cloud AI Observability | 当前 npm manifest 仍声明旧 `@mariozechner/pi-*` peers，项目又处在 Sigil → Agento11y 包名迁移期；对当前 `@earendil-works` Pi 0.81 需额外兼容性验证 | |
There was a problem hiding this comment.
In this OTel/Grafana comparison, the doc skips the current pi-otel package even though I checked its GitHub package metadata and it declares @earendil-works/pi-coding-agent:*, installs with pi install npm:pi-otel, and /otel start launches a local Aspire web dashboard with Grafana LGTM support. When someone wants a local or self-hosted OTel UI rather than a SaaS exporter, omitting this option makes the Grafana/OTel recommendation look more constrained than it is; please include it or document why it was rejected.
Useful? React with 👍 / 👎.
1 participant
Summary
Purpose
Scope
.agents/docs/research/2026-07-22-pi-observability-ecosystem.mdValidation
uv lock --checkuv run ruff check .uv run ruff format --check .uv run pytest tests/ -x --timeout=60(996 passed, 26 skipped)uv run python scripts/check_legibility.pyNotes