Skip to content

fix(telemetry): replace fire-and-forget uvx with bounded queue-based … - #28

Open
caihe-ch wants to merge 1 commit into
mainfrom
autowonder/12469/attempt-1/alibabacloud-agent-toolkit
Open

fix(telemetry): replace fire-and-forget uvx with bounded queue-based …#28
caihe-ch wants to merge 1 commit into
mainfrom
autowonder/12469/attempt-1/alibabacloud-agent-toolkit

Conversation

@caihe-ch

Copy link
Copy Markdown
Collaborator

…upload

The telemetry hooks spawned one uvx alibabacloud.mcp-proxy@latest plugin-telemetry process per event with no concurrency limit, timeout, or orphan reaping. Each uvx resolved @latest on every invocation and was detached via ( ... & ) + disown, causing processes to reparent to PID 1. Under sustained load this accumulated thousands of orphan processes, exhausting inodes and disk via UV cache proliferation.

Replace the fire-and-forget pattern with a queue-based bounded worker:

  • telemetry_enqueue.py: hook scripts write event args as JSON to a per-client queue directory, then start the worker if not running
  • telemetry_worker.py: single-instance worker (flock), uses fixed venv with pinned package version instead of uvx @latest, hard timeout per upload with process group kill (no orphans), finite retry with dead-letter logging, configurable queue size cap
  • Modified 3 shell scripts (post-tool-trace.sh, prompt-trace.sh, stop-turn-increment.sh) to pipe handler output to enqueue helper
  • Modified stop_handler.py _spawn_upload() to queue events instead of Popen fire-and-forget

Applied consistently across alibabacloud-core, alibabacloud-spec-ops, and alibabacloud-ecs-ops plugins. Bumped core version to 1.0.34.

Added 30 automated tests covering: off switch, queue operations, single-instance lock, retry/dead-letter, timeout, process cleanup, and code safety (no uvx @latest / disown remaining).

Fixes: #50646

Description

Type of Change

  • New plugin
  • New skill
  • Bug fix
  • Documentation update
  • CI/CD change
  • Other

Checklist

  • I have read the CONTRIBUTING guide
  • My changes pass python3 tools/validate.py
  • I have updated relevant documentation

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

…upload

The telemetry hooks spawned one `uvx alibabacloud.mcp-proxy@latest
plugin-telemetry` process per event with no concurrency limit, timeout,
or orphan reaping. Each uvx resolved @latest on every invocation and
was detached via `( ... & )` + `disown`, causing processes to reparent
to PID 1. Under sustained load this accumulated thousands of orphan
processes, exhausting inodes and disk via UV cache proliferation.

Replace the fire-and-forget pattern with a queue-based bounded worker:

- telemetry_enqueue.py: hook scripts write event args as JSON to a
  per-client queue directory, then start the worker if not running
- telemetry_worker.py: single-instance worker (flock), uses fixed
  venv with pinned package version instead of uvx @latest, hard
  timeout per upload with process group kill (no orphans), finite
  retry with dead-letter logging, configurable queue size cap
- Modified 3 shell scripts (post-tool-trace.sh, prompt-trace.sh,
  stop-turn-increment.sh) to pipe handler output to enqueue helper
- Modified stop_handler.py _spawn_upload() to queue events instead
  of Popen fire-and-forget

Applied consistently across alibabacloud-core, alibabacloud-spec-ops,
and alibabacloud-ecs-ops plugins. Bumped core version to 1.0.34.

Added 30 automated tests covering: off switch, queue operations,
single-instance lock, retry/dead-letter, timeout, process cleanup,
and code safety (no uvx @latest / disown remaining).

Fixes: #50646
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant