fix: answer callers queued behind a macrostep that raises (#688)

Author: fgmacedo

docs/async.md

@@ -163,9 +163,11 @@ model: they are enqueued and processed by the running loop.
 ```
 
 If an exception occurs during processing (with `catch_errors_as_events=False`),
-the exception is routed to the caller whose event caused it. Other callers
+the exception is routed to the caller whose event raised it. Other callers
 whose events were still pending will also receive the exception, since the
-processing loop clears the queue on failure.
+processing loop clears the queue on failure. An exception raised by the
+internal events or eventless transitions that follow an event goes only to
+the callers still pending: the caller of that event already has its result.
 
 
 (syncengine)=

docs/releases/3.2.2.md

@@ -67,6 +67,13 @@ fail after upgrading:
   awaiting a delayed one, which was never answered, now gets `None`, or `TransitionNotAllowed`
   on a `StateMachine`. See
   {ref}`release-3.2.2-macrostep`.
+- When an exception escapes an internal event or eventless transition enabled by an external
+  event (any exception on a `StateMachine`, one from a validator or `prepare_event` on a
+  `StateChart`), the events still queued are dropped, and a coroutine awaiting one gets the
+  exception. The queued external events used to run before the internal event or eventless
+  transition. Internal events raised before a callback fails, and events sent before a failed
+  lazy initial activation on the async engine, are dropped too, instead of running at the next
+  event. See {ref}`release-3.2.2-macrostep`.
 - An attribute named with the `error_`, `done_state_` or `done_invoke_` prefix now registers
   two events, so `events`, `allowed_events` and `enabled_events()` list both ids. See
   {ref}`release-3.2.2-prefixed-events`.
@@ -363,8 +370,23 @@ machine terminates because every region of a top-level parallel state reached a
 On 3.2.1, a coroutine awaiting a delayed event still pending when the machine terminated was
 never answered.
 
+An exception that escapes the internal events or eventless transitions following an external
+event is now handled like one escaping the external event itself: every event still queued,
+internal or external, delayed or not, is dropped, and a coroutine awaiting the `send()` of a
+dropped event gets that exception. Any exception escapes with `catch_errors_as_events=False`,
+as on a `StateMachine`; on a `StateChart`, one from a validator or `prepare_event` does. On
+3.2.1 the queued external events ran before those internal events and eventless transitions,
+and their callers got their own results; the exception was then raised by the sync `send()`
+that started processing, and on the async engine it reached no caller. On the async engine, as
+on 3.2.1, the coroutine whose event enabled the failing transitions gets its own event's
+result. Internal events raised by a callback before it fails are dropped too: they used to run
+at the next event. The same applies when the lazy initial activation fails on the async engine,
+where a second coroutine awaiting an event sent before it used to wait forever, and the events
+sent before it, which used to run at the next `send()`, are dropped.
+
 [#668](https://github.com/fgmacedo/python-statemachine/pull/668),
 [#678](https://github.com/fgmacedo/python-statemachine/issues/678),
+[#686](https://github.com/fgmacedo/python-statemachine/issues/686),
 [#685](https://github.com/fgmacedo/python-statemachine/pull/685).
 
 (release-3.2.2-prefixed-events)=

statemachine/engines/async_.py

@@ -373,6 +373,8 @@ async def processing_loop(self, caller_future: "asyncio.Future[object] | None" =
                     self._drop_external_events()
 
         except Exception as exc:
+            self._reject_pending_futures(exc)
+            self.clear()
             if caller_future is not None:
                 # Route the exception to the caller's future if still pending.
                 # If already resolved (caller's own event succeeded before a
@@ -478,17 +480,12 @@ async def _take_external_event(self, external_event: TriggerData):
                 self._resolve_future(event_future, result)
                 return result
             if not self.sm.allow_event_without_transition:
-                tna = TransitionNotAllowed(external_event.event, self.sm.configuration)
-                self._reject_future(event_future, tna)
-                self._reject_pending_futures(tna)
-                raise tna
+                raise TransitionNotAllowed(external_event.event, self.sm.configuration)
             # Event allowed but no transition — resolve with None
             self._resolve_future(event_future, None)
             return self._sentinel
         except Exception as exc:
             self._reject_future(event_future, exc)
-            self._reject_pending_futures(exc)
-            self.clear()
             raise
 
     async def enabled_events(self, *args, **kwargs):

statemachine/engines/base.py

@@ -185,6 +185,7 @@ def put(self, trigger_data: TriggerData, internal: bool = False):
         )
 
     def clear(self):
+        self.internal_queue.clear()
         self.external_queue.clear()
 
     def _drop_external_events(self):

statemachine/engines/sync.py

@@ -70,10 +70,13 @@ def processing_loop(self):
                 took_events = True
                 while took_events and self.running:
                     self.clear_cache()
-                    took_events = self._run_macrostep()
-
-                    # Spawn invoke handlers for states entered during this macrostep.
-                    self._invoke_manager.spawn_pending_sync()
+                    try:
+                        took_events = self._run_macrostep()
+                        # Spawn invoke handlers for states entered during this macrostep.
+                        self._invoke_manager.spawn_pending_sync()
+                    except Exception:
+                        self.clear()
+                        raise
                     self._check_root_final_state()
                     if not self.running:
                         break