|
| 1 | +# Migrating from 1.x to 2.x |
| 2 | + |
| 3 | +`2.x` is a breaking major release. Every change is a bug fix or brings Python to |
| 4 | +parity with the JavaScript and Java SDKs. The two changes most likely to touch |
| 5 | +your code are the typed, per-operation **error hierarchy** and the |
| 6 | +**serialize/deserialize round trip on the first run**. |
| 7 | + |
| 8 | +There is no compatibility shim: removed names (for example `CallableRuntimeError`) |
| 9 | +are gone with no alias. If you are not ready to migrate, stay on `1.x`. |
| 10 | + |
| 11 | +> Instrumentation plugins are out of scope here. The experimental `plugins=` hook |
| 12 | +> already existed in `1.x`; `2.x` adds opt-in auto-discovery and `PluginLoadError`. |
| 13 | +> Because plugins are opt-in and still evolving, they are documented with the |
| 14 | +> plugin/OpenTelemetry feature rather than in this guide. |
| 15 | +
|
| 16 | +## What Changed and What to Do |
| 17 | + |
| 18 | +| Change | What you must do | |
| 19 | +| --- | --- | |
| 20 | +| `CallableRuntimeError`, `UserlandError`, `CallableRuntimeErrorSerializableDetails` removed; typed per-operation errors added | Catch `StepError`, `InvokeError`, `ChildContextError`, or `WaitForConditionError` (or the base `DurableOperationError`) instead of `CallableRuntimeError`. | |
| 21 | +| `CallbackError` moved out of the termination tree; graded subtypes added | Remove any `termination_reason == TerminationReason.CALLBACK_ERROR` check (the enum member is gone). Optionally catch `CallbackTimeoutError` / `CallbackExternalError` / `CallbackSubmitterError`. | |
| 22 | +| `BatchResult.throw_if_error()` now raises a typed error | Replace `except CallableRuntimeError` with `ChildContextError` (ordinary item failure), `SerDesError` (item serialize/deserialize failure), and `BatchCompletionError` (custom `should_complete` failed the batch with no item error). `ChildContextError` and `BatchCompletionError` share the base `DurableOperationError`, but `SerDesError` does not, so catch `(DurableOperationError, SerDesError)` or list all three. | |
| 23 | +| Serialization/deserialization failures surface as `SerDesError`, a direct child of `DurableExecutionsError` (not `ExecutionError`) | If you caught serdes failures with `except ExecutionError`, catch `SerDesError` (or `DurableExecutionsError`) instead. | |
| 24 | +| First-run serialize/deserialize round trip for `step`, child contexts, `map`/`parallel`, and `wait_for_condition` | On the first run these now return `deserialize(serialize(x))`, the same canonical value replay returns. If you relied on the raw pre-serialization object, use the deserialized shape instead (or make your `SerDes` round-trip identity). Ensure `wait_for_condition` `initial_state` is serializable by the configured serdes. For a transient serdes failure, raise the new `RetryableSerDesError` (retries) instead of `SerDesError` (permanent). | |
| 25 | +| `InvokeConfig.timeout` and `InvokeConfig.timeout_seconds` removed | Remove them. Enforce any timeout inside the invoked function or as a separate timer. | |
| 26 | +| Removed `ItemBatcher`, `ItemsPerBatchUnit`, `BatchedInput`, `TerminationMode`, `StepFuture`, `MapConfig.item_batcher`, `ChildConfig.item_serdes`; also `ChainedInvokeFailedToStartType`, `ChainedInvokeTimeoutType`, `ChainedInvokeStopType` (from `lambda_service`) | Remove all uses. Replace `ChildConfig.item_serdes` with `ChildConfig.serdes`. | |
| 27 | +| `MapConfig` / `ParallelConfig` / `CompletionConfig` validate arguments at construction (e.g. `max_concurrency=0` or `min_successful=0` now raise `ValidationError`); `min_successful > total` is validated at the `map()`/`parallel()` call, not at construction | Wrap config construction, and the `map()`/`parallel()` call, in `try/except ValidationError` when inputs are external. | |
| 28 | +| `CompletionConfig.all_completed()` now actually tolerates all failures | If you hand-built the old all-`None` config, use the factory instead. | |
| 29 | +| `BatchResult.all` now omits never-started branches, so `total_count` and positional iteration differ for early-completed batches | If you index `.all` by original position or expect `total_count` to include unstarted branches, update that logic. | |
| 30 | +| For `map`/`parallel`, a custom `summary_generator` output is now stored under a `"summary"` key in an SDK-owned envelope (it no longer replaces the checkpoint payload). `ChildConfig.summary_generator` is unchanged: its output is still checkpointed verbatim | If you parse `map`/`parallel` summary payloads from execution history, read the `"summary"` key from the envelope. Child-context summary consumers need no change. | |
| 31 | +| `WaitDecision` removed; `WaitStrategyConfig.timeout` / `timeout_seconds` removed | Use `WaitForConditionDecision` (`stop_polling()` / `continue_waiting(delay)`). | |
| 32 | +| `wait_for_condition` raises `WaitForConditionError` when it exhausts `max_attempts` | Catch `WaitForConditionError` instead of inspecting the returned state. | |
| 33 | + |
| 34 | +Find affected code before upgrading: |
| 35 | + |
| 36 | +```bash |
| 37 | +rg -n "CallableRuntimeError|UserlandError|CallableRuntimeErrorSerializableDetails" . |
| 38 | +rg -n "CallbackError|CALLBACK_ERROR" . |
| 39 | +rg -n "InvokeConfig\(|\.timeout_seconds" . |
| 40 | +rg -n "WaitDecision|WaitStrategyConfig\(|item_batcher|ItemBatcher|ItemsPerBatchUnit" . |
| 41 | +rg -n "TerminationMode|BatchedInput|StepFuture|ChildConfig\(" . |
| 42 | +rg -n "ChainedInvoke|except ExecutionError" . |
| 43 | +``` |
| 44 | + |
| 45 | +## Error Handling (the biggest change) |
| 46 | + |
| 47 | +In `1.x` nearly every user-land failure surfaced as one `CallableRuntimeError`, |
| 48 | +so a failed step was indistinguishable from a failed invoke or child branch. `2.x` |
| 49 | +raises a specific type per operation, all under a new base `DurableOperationError`. |
| 50 | +Inspect the failure through its fields, not `__cause__`: `error_type`, `message`, |
| 51 | +`data`, and `stack_trace`. Do not rely on `__cause__` being the original |
| 52 | +exception - the SDK reconstructs a `DurableOperationError` stand-in carrying |
| 53 | +those same fields (on both the first run and replay, for determinism), so the |
| 54 | +original type is not preserved (a `ValueError` does not stay a `ValueError`) and |
| 55 | +custom attributes are lost. |
| 56 | + |
| 57 | +For `StepError`, `InvokeError`, `ChildContextError`, and `WaitForConditionError`, |
| 58 | +`error_type` is the name of the error that escaped your code (e.g. `"ValueError"`). |
| 59 | +The graded callback errors are different: `CallbackTimeoutError`, |
| 60 | +`CallbackExternalError`, and `CallbackSubmitterError` are constructed without the |
| 61 | +originating `error_type`, so `error_type` is the callback class name, not the |
| 62 | +underlying cause. Use the specific callback exception type (and `message` / |
| 63 | +`data` / `stack_trace`) to distinguish those. |
| 64 | + |
| 65 | +```python |
| 66 | +# 1.x |
| 67 | +from aws_durable_execution_sdk_python.exceptions import CallableRuntimeError |
| 68 | +try: |
| 69 | + result = context.step(charge_card, name="charge") |
| 70 | +except CallableRuntimeError as e: |
| 71 | + context.logger.error("something failed: %s", e.message) |
| 72 | + |
| 73 | +# 2.x |
| 74 | +from aws_durable_execution_sdk_python import StepError, DurableOperationError |
| 75 | +try: |
| 76 | + result = context.step(charge_card, name="charge") |
| 77 | +except StepError as e: # or `except DurableOperationError` to catch any operation |
| 78 | + context.logger.error("charge step failed: %s", e.message) |
| 79 | +``` |
| 80 | + |
| 81 | +New types, all exported from the package root: `DurableOperationError` (base), |
| 82 | +`StepError`, `InvokeError`, `ChildContextError`, `WaitForConditionError`, |
| 83 | +`CallbackError` (+ `CallbackExternalError`, `CallbackTimeoutError`, |
| 84 | +`CallbackSubmitterError`), plus `SerDesError` (now exported) and |
| 85 | +`RetryableSerDesError`. `SerDesError` stays a direct child of |
| 86 | +`DurableExecutionsError`; `RetryableSerDesError` is a retryable `InvocationError`. |
| 87 | + |
| 88 | +### Callbacks |
| 89 | + |
| 90 | +`context.wait_for_callback(...)` returns the payload directly and raises the |
| 91 | +callback error from the call itself (there is no `callback.result()`): |
| 92 | + |
| 93 | +```python |
| 94 | +from aws_durable_execution_sdk_python import ( |
| 95 | + CallbackError, CallbackTimeoutError, CallbackSubmitterError, |
| 96 | +) |
| 97 | +try: |
| 98 | + payload = context.wait_for_callback(submit_approval, name="approval") |
| 99 | +except CallbackTimeoutError: |
| 100 | + ... # timeout / heartbeat expiry |
| 101 | +except CallbackSubmitterError: |
| 102 | + ... # the submitter step failed |
| 103 | +except CallbackError as e: # external + internal |
| 104 | + context.logger.error("callback failed: %s", e.message) |
| 105 | +``` |
| 106 | + |
| 107 | +### map / parallel |
| 108 | + |
| 109 | +`throw_if_error()` can raise three types: `ChildContextError` for the ordinary |
| 110 | +item/branch failure, `SerDesError` if an item result failed to serialize or |
| 111 | +deserialize, and `BatchCompletionError` when a custom `should_complete` predicate |
| 112 | +marked the batch failed with no failed item (see the completion-predicate section |
| 113 | +below). `ChildContextError` and `BatchCompletionError` share the base |
| 114 | +`DurableOperationError`, but `SerDesError` does not, so catch |
| 115 | +`(DurableOperationError, SerDesError)` or list all three explicitly. |
| 116 | + |
| 117 | +```python |
| 118 | +from aws_durable_execution_sdk_python import ( |
| 119 | + ChildContextError, SerDesError, BatchCompletionError, |
| 120 | +) |
| 121 | + |
| 122 | +result = context.map(items, process_item) |
| 123 | +try: |
| 124 | + result.throw_if_error() |
| 125 | +except (ChildContextError, SerDesError, BatchCompletionError): |
| 126 | + for err in result.get_errors(): # every failed item's ErrorObject |
| 127 | + context.logger.error("%s: %s", err.type, err.message) |
| 128 | +``` |
| 129 | + |
| 130 | +## Serialize/Deserialize Round Trip |
| 131 | + |
| 132 | +`1.x` returned the raw in-memory result on the first run but the deserialized |
| 133 | +result on replay, so a non-identity custom `SerDes` produced different values. |
| 134 | +`2.x` round-trips (`serialize` then `deserialize`) on the first run for `step`, |
| 135 | +child contexts, `map`/`parallel`, and `wait_for_condition` (which also feeds the |
| 136 | +deserialized state to the wait strategy). Non-identity serdes (for example |
| 137 | +canonicalizing) is fully supported: the first run now returns the canonical |
| 138 | +`deserialize(serialize(x))` value, matching replay. If your code depended on the |
| 139 | +raw pre-serialization object on the first run, switch to the deserialized shape |
| 140 | +(or make the serdes round-trip identity). This also surfaces genuine |
| 141 | +serialization bugs on the first run instead of later on replay. |
| 142 | + |
| 143 | +`invoke` and `wait` are unaffected. `wait_for_callback` is implemented via a |
| 144 | +child context, so its result is serialized and deserialized before it returns: |
| 145 | +do not rely on callback-result object identity. The enclosing child context uses |
| 146 | +the default (extended-type) serdes, not `WaitForCallbackConfig.serdes`, so the |
| 147 | +value your callback deserializer returns must itself be serializable by the |
| 148 | +default serdes; otherwise the child raises `SerDesError`. |
| 149 | + |
| 150 | +`wait_for_condition` also round-trips `initial_state` through the serdes before |
| 151 | +the first check, so `initial_state` must now be serializable by the configured |
| 152 | +serdes. |
| 153 | + |
| 154 | +## New in 2.x: Custom Completion Predicate (Optional) |
| 155 | + |
| 156 | +`2.x` adds a `should_complete` predicate to `CompletionConfig`, giving `map` and |
| 157 | +`parallel` full control over when a batch completes early. This is a new feature, |
| 158 | +not a breaking change - no action is required unless you adopt it. |
| 159 | + |
| 160 | +```python |
| 161 | +from aws_durable_execution_sdk_python import complete_batch, continue_batch |
| 162 | +from aws_durable_execution_sdk_python.config import CompletionConfig |
| 163 | + |
| 164 | +config = CompletionConfig( |
| 165 | + should_complete=lambda status: ( |
| 166 | + complete_batch() if status.success_count >= 2 else continue_batch() |
| 167 | + ) |
| 168 | +) |
| 169 | +``` |
| 170 | + |
| 171 | +The predicate receives a `CompletionStatus` snapshot (counts plus per-item |
| 172 | +statuses) and returns a `CompletionDecision` - `continue_batch()`, or |
| 173 | +`complete_batch(CompletionOutcome.SUCCEEDED)` / `complete_batch(CompletionOutcome.FAILED)` |
| 174 | +(the outcome defaults to `SUCCEEDED`). A `FAILED` outcome marks the whole batch |
| 175 | +failed; `throw_if_error()` then raises `BatchCompletionError` (a |
| 176 | +`DurableOperationError` subtype) even when no individual item failed. Individual |
| 177 | +item/branch failures still surface as `ChildContextError`. Notes: |
| 178 | + |
| 179 | +- It cannot be combined with `min_successful` or the `tolerated_failure_*` |
| 180 | + fields; doing so raises `ValidationError` at construction. |
| 181 | +- The predicate must be deterministic, side-effect-free, and monotonic (once a |
| 182 | + level of progress would complete the batch, more progress must not flip it back |
| 183 | + to continue). Replaying an already-completed batch uses the checkpointed |
| 184 | + decision, but a mid-run resume re-runs the batch live and re-evaluates the |
| 185 | + predicate as completed branches replay, possibly in a different order. |
| 186 | +- New exports: `complete_batch`, `continue_batch`, `CompletionStatus`, |
| 187 | + `CompletionDecision`, `CompletionOutcome`, `CompletionItemStatus`, |
| 188 | + `BatchItemStatus`, `BatchCompletionError`. |
| 189 | + |
| 190 | +## New in 2.x: Attempt Number in Contexts (Optional) |
| 191 | + |
| 192 | +`StepContext` and `WaitForConditionCheckContext` now expose an `attempt` field |
| 193 | +(the current attempt number, starting at 1). Read it inside a step or a |
| 194 | +`wait_for_condition` check to branch on the retry count. The SDK injects these |
| 195 | +contexts, so normal usage needs no change. But `attempt` is a required |
| 196 | +dataclass field with no default: if you construct these contexts directly (in |
| 197 | +tests, fixtures, or wrappers), you must now pass `attempt` or construction fails |
| 198 | +with `TypeError`. |
| 199 | + |
| 200 | +## Recommended Validation After Upgrading |
| 201 | + |
| 202 | +1. Build and run your test suite against `2.x`, and grep for the removed names above. |
| 203 | +2. Trigger a failure in a `step`, an `invoke`, and a `map`/`parallel` branch; |
| 204 | + confirm you catch `StepError`, `InvokeError`, and `ChildContextError`. |
| 205 | +3. Exercise a `wait_for_callback` timeout and a submitter-step failure |
| 206 | + (`CallbackTimeoutError`, `CallbackSubmitterError`). |
| 207 | +4. Exercise a `wait_for_condition` that exhausts its attempts (`WaitForConditionError`). |
| 208 | +5. If you use a custom `SerDes`, run a workflow that checkpoints both a result and |
| 209 | + an error payload and confirm first-run output equals replay output. |
0 commit comments