-
Notifications
You must be signed in to change notification settings - Fork 20
Description
Bug Description
When a cron job fires, two CLI processes start simultaneously instead of one. Both complete independently with different outputs.
Root Cause
In ductor_bot/cron/observer.py, method _schedule_job() overwrites self._scheduled[job_id] with a new Task without canceling the existing one. The old Task becomes orphaned but remains alive in the asyncio event loop and fires at its original scheduled time.
# Current (buggy):
task = asyncio.create_task(self._run_at(delay, scheduled_job))
self._scheduled[job_id] = task # old task NOT cancelled, still fires# Fix:
if job_id in self._scheduled:
old_task = self._scheduled[job_id]
if not old_task.done():
old_task.cancel()
task = asyncio.create_task(self._run_at(delay, scheduled_job))
self._scheduled[job_id] = taskEvidence
Logs show 50+ occurrences of double execution with ~1 second gap between starts and different stdout sizes (e.g. 942 vs 1016 bytes), confirming two independent executions:
2026-03-15 20:00:00 [INFO] Cron job starting job=Gmail Support Check
2026-03-15 20:00:00 [INFO] Cron job starting job=Gmail Support Check
2026-03-15 20:00:23 [INFO] Cron job completed ... stdout=942 result=47
2026-03-15 20:00:24 [INFO] Cron job completed ... stdout=1016 result=101
Why It Happens
A reschedule event (triggered by _on_file_change() when update_run_status() writes to JSON, or from request_reschedule()) fires close to the scheduled execution time. This creates a new Task while the old one is still pending. Both then fire.
Impact
- Tasks with side effects (sending Telegram messages, updating memory, calling external APIs) execute twice
- Non-idempotent tasks produce duplicate output
- The
_executing: set[str]guard exists but only protects_reschedule_all(), not_schedule_job()
Version
ductor 0.15.0