Skip to content

CronObserver: double execution due to orphaned asyncio Task in _schedule_job() #75

@alexeymorozua

Description

@alexeymorozua

Bug Description

When a cron job fires, two CLI processes start simultaneously instead of one. Both complete independently with different outputs.

Root Cause

In ductor_bot/cron/observer.py, method _schedule_job() overwrites self._scheduled[job_id] with a new Task without canceling the existing one. The old Task becomes orphaned but remains alive in the asyncio event loop and fires at its original scheduled time.

# Current (buggy):
task = asyncio.create_task(self._run_at(delay, scheduled_job))
self._scheduled[job_id] = task  # old task NOT cancelled, still fires
# Fix:
if job_id in self._scheduled:
    old_task = self._scheduled[job_id]
    if not old_task.done():
        old_task.cancel()
task = asyncio.create_task(self._run_at(delay, scheduled_job))
self._scheduled[job_id] = task

Evidence

Logs show 50+ occurrences of double execution with ~1 second gap between starts and different stdout sizes (e.g. 942 vs 1016 bytes), confirming two independent executions:

2026-03-15 20:00:00 [INFO] Cron job starting job=Gmail Support Check
2026-03-15 20:00:00 [INFO] Cron job starting job=Gmail Support Check
2026-03-15 20:00:23 [INFO] Cron job completed ... stdout=942 result=47
2026-03-15 20:00:24 [INFO] Cron job completed ... stdout=1016 result=101

Why It Happens

A reschedule event (triggered by _on_file_change() when update_run_status() writes to JSON, or from request_reschedule()) fires close to the scheduled execution time. This creates a new Task while the old one is still pending. Both then fire.

Impact

  • Tasks with side effects (sending Telegram messages, updating memory, calling external APIs) execute twice
  • Non-idempotent tasks produce duplicate output
  • The _executing: set[str] guard exists but only protects _reschedule_all(), not _schedule_job()

Version

ductor 0.15.0

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions