Skip to content

Commit 1e93efe

Browse files
committed
docs: add v1 to v2 migration guide
Add docs/migration-1.x-to-2.x.md covering the 2.x breaking changes: typed per-operation error hierarchy, first-run serialize/deserialize round trip, removed config types and fields, construction-time config validation, and wait_for_condition changes; plus the optional should_complete feature. Also document the initial_state serdes requirement on WaitForConditionConfig.
1 parent d4090f1 commit 1e93efe

2 files changed

Lines changed: 220 additions & 1 deletion

File tree

docs/migration-1.x-to-2.x.md

Lines changed: 209 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,209 @@
1+
# Migrating from 1.x to 2.x
2+
3+
`2.x` is a breaking major release. Every change is a bug fix or brings Python to
4+
parity with the JavaScript and Java SDKs. The two changes most likely to touch
5+
your code are the typed, per-operation **error hierarchy** and the
6+
**serialize/deserialize round trip on the first run**.
7+
8+
There is no compatibility shim: removed names (for example `CallableRuntimeError`)
9+
are gone with no alias. If you are not ready to migrate, stay on `1.x`.
10+
11+
> Instrumentation plugins are out of scope here. The experimental `plugins=` hook
12+
> already existed in `1.x`; `2.x` adds opt-in auto-discovery and `PluginLoadError`.
13+
> Because plugins are opt-in and still evolving, they are documented with the
14+
> plugin/OpenTelemetry feature rather than in this guide.
15+
16+
## What Changed and What to Do
17+
18+
| Change | What you must do |
19+
| --- | --- |
20+
| `CallableRuntimeError`, `UserlandError`, `CallableRuntimeErrorSerializableDetails` removed; typed per-operation errors added | Catch `StepError`, `InvokeError`, `ChildContextError`, or `WaitForConditionError` (or the base `DurableOperationError`) instead of `CallableRuntimeError`. |
21+
| `CallbackError` moved out of the termination tree; graded subtypes added | Remove any `termination_reason == TerminationReason.CALLBACK_ERROR` check (the enum member is gone). Optionally catch `CallbackTimeoutError` / `CallbackExternalError` / `CallbackSubmitterError`. |
22+
| `BatchResult.throw_if_error()` now raises a typed error | Replace `except CallableRuntimeError` with `ChildContextError` (ordinary item failure), `SerDesError` (item serialize/deserialize failure), and `BatchCompletionError` (custom `should_complete` failed the batch with no item error). `ChildContextError` and `BatchCompletionError` share the base `DurableOperationError`, but `SerDesError` does not, so catch `(DurableOperationError, SerDesError)` or list all three. |
23+
| Serialization/deserialization failures surface as `SerDesError`, a direct child of `DurableExecutionsError` (not `ExecutionError`) | If you caught serdes failures with `except ExecutionError`, catch `SerDesError` (or `DurableExecutionsError`) instead. |
24+
| First-run serialize/deserialize round trip for `step`, child contexts, `map`/`parallel`, and `wait_for_condition` | On the first run these now return `deserialize(serialize(x))`, the same canonical value replay returns. If you relied on the raw pre-serialization object, use the deserialized shape instead (or make your `SerDes` round-trip identity). Ensure `wait_for_condition` `initial_state` is serializable by the configured serdes. For a transient serdes failure, raise the new `RetryableSerDesError` (retries) instead of `SerDesError` (permanent). |
25+
| `InvokeConfig.timeout` and `InvokeConfig.timeout_seconds` removed | Remove them. Enforce any timeout inside the invoked function or as a separate timer. |
26+
| Removed `ItemBatcher`, `ItemsPerBatchUnit`, `BatchedInput`, `TerminationMode`, `StepFuture`, `MapConfig.item_batcher`, `ChildConfig.item_serdes`; also `ChainedInvokeFailedToStartType`, `ChainedInvokeTimeoutType`, `ChainedInvokeStopType` (from `lambda_service`) | Remove all uses. Replace `ChildConfig.item_serdes` with `ChildConfig.serdes`. |
27+
| `MapConfig` / `ParallelConfig` / `CompletionConfig` validate arguments at construction (e.g. `max_concurrency=0` or `min_successful=0` now raise `ValidationError`); `min_successful > total` is validated at the `map()`/`parallel()` call, not at construction | Wrap config construction, and the `map()`/`parallel()` call, in `try/except ValidationError` when inputs are external. |
28+
| `CompletionConfig.all_completed()` now actually tolerates all failures | If you hand-built the old all-`None` config, use the factory instead. |
29+
| `BatchResult.all` now omits never-started branches, so `total_count` and positional iteration differ for early-completed batches | If you index `.all` by original position or expect `total_count` to include unstarted branches, update that logic. |
30+
| For `map`/`parallel`, a custom `summary_generator` output is now stored under a `"summary"` key in an SDK-owned envelope (it no longer replaces the checkpoint payload). `ChildConfig.summary_generator` is unchanged: its output is still checkpointed verbatim | If you parse `map`/`parallel` summary payloads from execution history, read the `"summary"` key from the envelope. Child-context summary consumers need no change. |
31+
| `WaitDecision` removed; `WaitStrategyConfig.timeout` / `timeout_seconds` removed | Use `WaitForConditionDecision` (`stop_polling()` / `continue_waiting(delay)`). |
32+
| `wait_for_condition` raises `WaitForConditionError` when it exhausts `max_attempts` | Catch `WaitForConditionError` instead of inspecting the returned state. |
33+
34+
Find affected code before upgrading:
35+
36+
```bash
37+
rg -n "CallableRuntimeError|UserlandError|CallableRuntimeErrorSerializableDetails" .
38+
rg -n "CallbackError|CALLBACK_ERROR" .
39+
rg -n "InvokeConfig\(|\.timeout_seconds" .
40+
rg -n "WaitDecision|WaitStrategyConfig\(|item_batcher|ItemBatcher|ItemsPerBatchUnit" .
41+
rg -n "TerminationMode|BatchedInput|StepFuture|ChildConfig\(" .
42+
rg -n "ChainedInvoke|except ExecutionError" .
43+
```
44+
45+
## Error Handling (the biggest change)
46+
47+
In `1.x` nearly every user-land failure surfaced as one `CallableRuntimeError`,
48+
so a failed step was indistinguishable from a failed invoke or child branch. `2.x`
49+
raises a specific type per operation, all under a new base `DurableOperationError`.
50+
Inspect the failure through its fields, not `__cause__`: `error_type`, `message`,
51+
`data`, and `stack_trace`. Do not rely on `__cause__` being the original
52+
exception - the SDK reconstructs a `DurableOperationError` stand-in carrying
53+
those same fields (on both the first run and replay, for determinism), so the
54+
original type is not preserved (a `ValueError` does not stay a `ValueError`) and
55+
custom attributes are lost.
56+
57+
For `StepError`, `InvokeError`, `ChildContextError`, and `WaitForConditionError`,
58+
`error_type` is the name of the error that escaped your code (e.g. `"ValueError"`).
59+
The graded callback errors are different: `CallbackTimeoutError`,
60+
`CallbackExternalError`, and `CallbackSubmitterError` are constructed without the
61+
originating `error_type`, so `error_type` is the callback class name, not the
62+
underlying cause. Use the specific callback exception type (and `message` /
63+
`data` / `stack_trace`) to distinguish those.
64+
65+
```python
66+
# 1.x
67+
from aws_durable_execution_sdk_python.exceptions import CallableRuntimeError
68+
try:
69+
result = context.step(charge_card, name="charge")
70+
except CallableRuntimeError as e:
71+
context.logger.error("something failed: %s", e.message)
72+
73+
# 2.x
74+
from aws_durable_execution_sdk_python import StepError, DurableOperationError
75+
try:
76+
result = context.step(charge_card, name="charge")
77+
except StepError as e: # or `except DurableOperationError` to catch any operation
78+
context.logger.error("charge step failed: %s", e.message)
79+
```
80+
81+
New types, all exported from the package root: `DurableOperationError` (base),
82+
`StepError`, `InvokeError`, `ChildContextError`, `WaitForConditionError`,
83+
`CallbackError` (+ `CallbackExternalError`, `CallbackTimeoutError`,
84+
`CallbackSubmitterError`), plus `SerDesError` (now exported) and
85+
`RetryableSerDesError`. `SerDesError` stays a direct child of
86+
`DurableExecutionsError`; `RetryableSerDesError` is a retryable `InvocationError`.
87+
88+
### Callbacks
89+
90+
`context.wait_for_callback(...)` returns the payload directly and raises the
91+
callback error from the call itself (there is no `callback.result()`):
92+
93+
```python
94+
from aws_durable_execution_sdk_python import (
95+
CallbackError, CallbackTimeoutError, CallbackSubmitterError,
96+
)
97+
try:
98+
payload = context.wait_for_callback(submit_approval, name="approval")
99+
except CallbackTimeoutError:
100+
... # timeout / heartbeat expiry
101+
except CallbackSubmitterError:
102+
... # the submitter step failed
103+
except CallbackError as e: # external + internal
104+
context.logger.error("callback failed: %s", e.message)
105+
```
106+
107+
### map / parallel
108+
109+
`throw_if_error()` can raise three types: `ChildContextError` for the ordinary
110+
item/branch failure, `SerDesError` if an item result failed to serialize or
111+
deserialize, and `BatchCompletionError` when a custom `should_complete` predicate
112+
marked the batch failed with no failed item (see the completion-predicate section
113+
below). `ChildContextError` and `BatchCompletionError` share the base
114+
`DurableOperationError`, but `SerDesError` does not, so catch
115+
`(DurableOperationError, SerDesError)` or list all three explicitly.
116+
117+
```python
118+
from aws_durable_execution_sdk_python import (
119+
ChildContextError, SerDesError, BatchCompletionError,
120+
)
121+
122+
result = context.map(items, process_item)
123+
try:
124+
result.throw_if_error()
125+
except (ChildContextError, SerDesError, BatchCompletionError):
126+
for err in result.get_errors(): # every failed item's ErrorObject
127+
context.logger.error("%s: %s", err.type, err.message)
128+
```
129+
130+
## Serialize/Deserialize Round Trip
131+
132+
`1.x` returned the raw in-memory result on the first run but the deserialized
133+
result on replay, so a non-identity custom `SerDes` produced different values.
134+
`2.x` round-trips (`serialize` then `deserialize`) on the first run for `step`,
135+
child contexts, `map`/`parallel`, and `wait_for_condition` (which also feeds the
136+
deserialized state to the wait strategy). Non-identity serdes (for example
137+
canonicalizing) is fully supported: the first run now returns the canonical
138+
`deserialize(serialize(x))` value, matching replay. If your code depended on the
139+
raw pre-serialization object on the first run, switch to the deserialized shape
140+
(or make the serdes round-trip identity). This also surfaces genuine
141+
serialization bugs on the first run instead of later on replay.
142+
143+
`invoke` and `wait` are unaffected. `wait_for_callback` is implemented via a
144+
child context, so its result is serialized and deserialized before it returns:
145+
do not rely on callback-result object identity. The enclosing child context uses
146+
the default (extended-type) serdes, not `WaitForCallbackConfig.serdes`, so the
147+
value your callback deserializer returns must itself be serializable by the
148+
default serdes; otherwise the child raises `SerDesError`.
149+
150+
`wait_for_condition` also round-trips `initial_state` through the serdes before
151+
the first check, so `initial_state` must now be serializable by the configured
152+
serdes.
153+
154+
## New in 2.x: Custom Completion Predicate (Optional)
155+
156+
`2.x` adds a `should_complete` predicate to `CompletionConfig`, giving `map` and
157+
`parallel` full control over when a batch completes early. This is a new feature,
158+
not a breaking change - no action is required unless you adopt it.
159+
160+
```python
161+
from aws_durable_execution_sdk_python import complete_batch, continue_batch
162+
from aws_durable_execution_sdk_python.config import CompletionConfig
163+
164+
config = CompletionConfig(
165+
should_complete=lambda status: (
166+
complete_batch() if status.success_count >= 2 else continue_batch()
167+
)
168+
)
169+
```
170+
171+
The predicate receives a `CompletionStatus` snapshot (counts plus per-item
172+
statuses) and returns a `CompletionDecision` - `continue_batch()`, or
173+
`complete_batch(CompletionOutcome.SUCCEEDED)` / `complete_batch(CompletionOutcome.FAILED)`
174+
(the outcome defaults to `SUCCEEDED`). A `FAILED` outcome marks the whole batch
175+
failed; `throw_if_error()` then raises `BatchCompletionError` (a
176+
`DurableOperationError` subtype) even when no individual item failed. Individual
177+
item/branch failures still surface as `ChildContextError`. Notes:
178+
179+
- It cannot be combined with `min_successful` or the `tolerated_failure_*`
180+
fields; doing so raises `ValidationError` at construction.
181+
- The predicate must be deterministic, side-effect-free, and monotonic (once a
182+
level of progress would complete the batch, more progress must not flip it back
183+
to continue). Replaying an already-completed batch uses the checkpointed
184+
decision, but a mid-run resume re-runs the batch live and re-evaluates the
185+
predicate as completed branches replay, possibly in a different order.
186+
- New exports: `complete_batch`, `continue_batch`, `CompletionStatus`,
187+
`CompletionDecision`, `CompletionOutcome`, `CompletionItemStatus`,
188+
`BatchItemStatus`, `BatchCompletionError`.
189+
190+
## New in 2.x: Attempt Number in Contexts (Optional)
191+
192+
`StepContext` and `WaitForConditionCheckContext` now expose an `attempt` field
193+
(the current attempt number, starting at 1). Read it inside a step or a
194+
`wait_for_condition` check to branch on the retry count. The SDK injects these
195+
contexts, so normal usage needs no change. But `attempt` is a required
196+
dataclass field with no default: if you construct these contexts directly (in
197+
tests, fixtures, or wrappers), you must now pass `attempt` or construction fails
198+
with `TypeError`.
199+
200+
## Recommended Validation After Upgrading
201+
202+
1. Build and run your test suite against `2.x`, and grep for the removed names above.
203+
2. Trigger a failure in a `step`, an `invoke`, and a `map`/`parallel` branch;
204+
confirm you catch `StepError`, `InvokeError`, and `ChildContextError`.
205+
3. Exercise a `wait_for_callback` timeout and a submitter-step failure
206+
(`CallbackTimeoutError`, `CallbackSubmitterError`).
207+
4. Exercise a `wait_for_condition` that exhausts its attempts (`WaitForConditionError`).
208+
5. If you use a custom `SerDes`, run a workflow that checkpoints both a result and
209+
an error payload and confirm first-run output equals replay output.

packages/aws-durable-execution-sdk-python/src/aws_durable_execution_sdk_python/waits.py

Lines changed: 11 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -99,7 +99,17 @@ def wait_strategy(result: T, attempts_made: int) -> WaitForConditionDecision:
9999

100100
@dataclass(frozen=True)
101101
class WaitForConditionConfig(Generic[T]):
102-
"""Configuration for wait_for_condition."""
102+
"""Configuration for wait_for_condition.
103+
104+
Attributes:
105+
wait_strategy: Called after each poll with (state, attempts_made) and
106+
returns a WaitForConditionDecision (continue_waiting or stop_polling).
107+
initial_state: State passed to the first poll. It is round-tripped
108+
through serdes (serialize then deserialize) before the first check,
109+
so it must be serializable by the configured serdes.
110+
serdes: SerDes used to serialize and deserialize the polled state at
111+
each checkpoint. Defaults to the SDK's extended-type serdes when None.
112+
"""
103113

104114
wait_strategy: Callable[[T, int], WaitForConditionDecision]
105115
initial_state: T

0 commit comments

Comments
 (0)