feat: v0.3.0 — full Java framework parity

CHANGELOG.md

@@ -6,6 +6,96 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
 
 ---
 
+## v0.3.0-M1 (2026-05-07)
+
+### Java framework parity push — major release
+
+This milestone closes the parity gap with the Java Firefly Framework
+(``fireflyframework-orchestration`` and the surrounding modules). It rewrites
+the transactional engine from scratch and adds nine missing modules.
+
+### Added — additional adapters & meta-packages
+- **IDP adapters**: ``KeycloakIdpAdapter``, ``AwsCognitoIdpAdapter``,
+  ``AzureAdIdpAdapter`` (alongside the existing ``InternalDbIdpAdapter``).
+- **ECM storage adapters**: ``AwsS3StorageAdapter``, ``AzureBlobStorageAdapter``.
+- **ECM e-signature adapters**: ``DocuSignESignatureAdapter``,
+  ``AdobeSignESignatureAdapter``, ``LogaltyESignatureAdapter``.
+- **Notification provider adapters**: ``SendGridEmailProvider``,
+  ``TwilioSmsProvider``, ``FirebasePushProvider``, ``ResendEmailProvider``.
+- **Client protocols**: ``SoapClient``/``SoapClientBuilder``,
+  ``GrpcClientBuilder``, ``GraphQLClient``/``GraphQLClientBuilder``,
+  ``WebSocketClient``/``WebSocketClientBuilder``.
+- **Config server**: ``pyfly.config_server`` with
+  ``ConfigServer``/``ConfigClient`` and filesystem + in-memory backends.
+- **Starter meta-packages**: ``pyfly.starters`` exposing
+  ``enable_core_stack``, ``enable_application_stack``, ``enable_data_stack``,
+  ``enable_domain_stack`` mirroring the Java starter modules.
+- **Extra domain validators**: ``is_valid_date``, ``is_valid_datetime``,
+  ``is_valid_national_id``, ``is_valid_sort_code``, ``is_valid_interest_rate``.
+
+### Added — transactional engine, complete rewrite
+- **`pyfly.transactional.core`** — new shared foundation: `ExecutionContext`,
+  `ExecutionStatus`, `ExecutionPattern`, `RetryPolicy`, `TopologyBuilder`,
+  `ArgumentResolver`, `StepInvoker`, `BackpressureStrategy` (adaptive,
+  batched, circuit-breaker), `OrchestrationEvents` /
+  `CompositeOrchestrationEvents` / `LoggerOrchestrationEvents`,
+  `OrchestrationMetrics`, `OrchestrationTracer`, `DeadLetterService`,
+  `RecoveryService`, `OrchestrationScheduler`, `OrchestrationValidator`,
+  `EventGateway`, `ExecutionReport`, `InMemoryPersistenceProvider`.
+- **`pyfly.transactional.workflow`** — entirely new pattern: `@workflow`,
+  `@workflow_step`, `@wait_for_signal`, `@wait_for_timer`,
+  `@wait_for_all`/`@wait_for_any`, `@child_workflow`, `@compensation_step`,
+  `@workflow_query`, `@on_workflow_complete`/`@on_workflow_error`,
+  `@scheduled_workflow`, plus `WorkflowEngine`, `WorkflowExecutor`,
+  `WorkflowRegistry`, `SignalService`, `TimerService`,
+  `ChildWorkflowService`, `ContinueAsNewService`, `WorkflowQueryService`,
+  `WorkflowBuilder`.
+- **Persistence adapters** — `pyfly.transactional.persistence.RedisPersistenceProvider`,
+  `CachePersistenceProvider`, `SqlAlchemyPersistenceProvider`.
+- **REST controllers** — `OrchestrationController`, `DeadLetterController`,
+  `WorkflowController` exposing list/start/signal/retry endpoints.
+- **HealthIndicator** + composite `OrchestrationHealthIndicator`.
+- `OrchestrationBuilder` root + `SagaBuilder` + `TccBuilder` for programmatic
+  pattern definition.
+- `@scheduled_saga`, `@scheduled_tcc`, `@step_event`, `@tcc_event`
+  annotations.
+
+### Added — new modules
+- **`pyfly.eventsourcing`** — `AggregateRoot`, `EventStore` (in-memory and
+  SQLAlchemy), `SnapshotStore`, `TransactionalOutbox`, `Projection` /
+  `ProjectionRunner`, `EventUpcaster`, `EventSourcedRepository`.
+- **`pyfly.callbacks`** — outbound callback dispatcher with HMAC signing,
+  retries, configurable subscriptions and execution tracking.
+- **`pyfly.webhooks`** — inbound webhook ingestion with signature validation,
+  idempotency dedup, and pluggable listeners.
+- **`pyfly.notifications`** — email / SMS / push abstractions with port
+  pattern and dummy + SMTP adapters.
+- **`pyfly.idp`** — identity provider port + internal-DB adapter with bcrypt
+  password hashing, user / session / role management.
+- **`pyfly.ecm`** — document storage, metadata, folders, e-signature ports
+  with local-filesystem and no-op adapters.
+- **`pyfly.plugins`** — pluggable module system: `@plugin`, `@extension`,
+  `@extension_point`, `PluginManager`, `PluginDependencyResolver`,
+  `ExtensionRegistry`.
+- **`pyfly.rule_engine`** — YAML-based business rules with logical
+  composition, batch evaluation and an in-memory rule-set repository.
+
+### Added — EDA enhancements
+- `EventCircuitBreaker`, `InMemoryEdaDeadLetterStore`,
+  `JsonEventSerializer` / `AvroEventSerializer` / `ProtobufEventSerializer`,
+  `HeaderEventFilter` / `PredicateEventFilter`.
+
+### Added — domain validators (`pyfly.validation.domain`)
+- `is_valid_iban` / `valid_iban`, `is_valid_bic` / `valid_bic`,
+  `is_valid_phone_number`, `is_valid_credit_card`, `is_valid_cvv`,
+  `is_valid_currency_code`, `is_valid_amount`, `is_valid_account_number`,
+  `is_valid_tax_id`, `is_valid_pin`, `is_strong_password`.
+
+### Tests
+- 2700+ tests passing, ~135 new tests for the new modules.
+
+---
+
 ## v0.2.0-M11 (2026-03-01)
 
 ### Fixed

README.md

@@ -487,7 +487,7 @@ See the full [CLI Reference](docs/cli.md) for details.
 
 ## Modules
 
-PyFly currently implements **27 modules** organized into four layers:
+PyFly currently implements **36 modules** organized into five layers:
 
 ### Foundation Layer
 
@@ -524,7 +524,22 @@ PyFly currently implements **27 modules** organized into four layers:
 | **Scheduling** | Cron jobs, fixed-rate tasks | Spring Scheduling |
 | **Resilience** | Rate limiter, bulkhead, timeout, fallback | Resilience4j (in `fireflyframework-client`) |
 | **Shell** | CLI commands, interactive REPL, runners | Spring Shell |
-| **Transactional** | Distributed Saga and TCC transaction orchestration with compensation and recovery | `fireflyframework-transactional-engine` |
+| **Transactional** | Saga + Workflow + TCC orchestration: signal-driven, DAG, compensation, multi-backend persistence, DLQ, recovery | `fireflyframework-orchestration` |
+| **Event Sourcing** | AggregateRoot, EventStore, snapshots, transactional outbox, projections, upcasting | `fireflyframework-eventsourcing` |
+| **Plugins** | `@plugin` / `@extension_point` / `@extension`, dependency-ordered lifecycle | `fireflyframework-plugins` |
+| **Rule Engine** | YAML DSL, AST evaluator, batch evaluation, rule-set repository | `fireflyframework-rule-engine` |
+| **Config Server** | Spring Cloud Config Server analogue + client | `fireflyframework-config-server` |
+
+### Integration Layer
+
+| Module | Description | Firefly Java Equivalent |
+|--------|-------------|------------------------|
+| **IDP** | Identity-provider port + Keycloak / AWS Cognito / Azure AD / internal-DB adapters | `fireflyframework-idp` + adapters |
+| **ECM** | Document storage / metadata / folders / e-signature ports + AWS S3 / Azure Blob / DocuSign / Adobe Sign / Logalty / local-fs adapters | `fireflyframework-ecm` + adapters |
+| **Notifications** | Email / SMS / push ports + SendGrid / Twilio / Firebase / Resend / SMTP / dummy adapters | `fireflyframework-notifications` + adapters |
+| **Callbacks** | Outbound webhook dispatcher with HMAC signing, retries, execution tracking | `fireflyframework-callbacks` |
+| **Webhooks** | Inbound webhook ingestion with signature validation, idempotency, listener pattern | `fireflyframework-webhooks` |
+| **Starters** | Meta-packages (`enable_core_stack` / `application` / `data` / `domain`) | `fireflyframework-starter-*` |
 
 ### Cross-Cutting Layer
 
@@ -562,7 +577,15 @@ Browse all guides in the [Module Guides Index](docs/modules/README.md):
 - [WebFilters](docs/modules/web-filters.md) — Request/response filter chain
 - [Actuator](docs/modules/actuator.md) — Health checks, extensible endpoints
 - [Custom Actuator Endpoints](docs/modules/custom-actuator-endpoints.md) — Build your own actuator endpoints
-- [Transactional Engine](docs/modules/transactional.md) — SAGA and TCC distributed transaction patterns
+- [Transactional Engine](docs/modules/transactional.md) — Saga, Workflow, and TCC distributed transaction patterns
+- [Event Sourcing](docs/modules/eventsourcing.md) — Aggregates, event store, snapshots, outbox, projections
+- [Plugins](docs/modules/plugins.md) — Plugin SPI, extension points, lifecycle
+- [Rule Engine](docs/modules/rule-engine.md) — YAML DSL, AST evaluator, batch evaluation
+- [Callbacks (outbound)](docs/modules/callbacks.md) — Dispatch domain events to external HTTP endpoints
+- [Webhooks (inbound)](docs/modules/webhooks.md) — Receive, verify, dedupe, dispatch
+- [Notifications](docs/modules/notifications.md) — Email / SMS / push abstractions
+- [IDP (Identity Provider)](docs/modules/idp.md) — Keycloak / AWS Cognito / Azure AD / internal-DB adapters
+- [ECM (Content Management)](docs/modules/ecm.md) — Documents, folders, e-signature workflows
 - [Admin Dashboard](docs/modules/admin.md) — Embedded management dashboard, server mode, custom views
 
 ### Adapter Reference

docs/modules/README.md

@@ -103,7 +103,24 @@ Coordinate multi-step operations across services with automatic compensation and
 
 | Guide | What You'll Learn |
 |-------|-------------------|
-| [Transactional Engine](transactional.md) | `@saga`, `@saga_step`, `@tcc`, `@tcc_participant`, `@try_method`, `@confirm_method`, `@cancel_method`, parameter injection (`Input`, `FromStep`, `Header`, `Variable`), 5 compensation policies, DAG-based execution, saga composition, backpressure strategies, persistence and recovery |
+| [Transactional Engine](transactional.md) | Saga (`@saga`, `@saga_step`), Workflow (`@workflow`, `@workflow_step`, `@wait_for_signal`, `@wait_for_timer`, `@child_workflow`), TCC (`@tcc`, `@tcc_participant`, `@try_method`, `@confirm_method`, `@cancel_method`), parameter injection (`Input`, `FromStep`, `Header`, `Variable`), 5 compensation policies, DAG execution, persistence (in-memory / Redis / SQLAlchemy / cache), recovery, dead-letter queue, scheduling, REST endpoints |
+| [Event Sourcing](eventsourcing.md) | `AggregateRoot`, `EventStore` (in-memory + SQLAlchemy), `SnapshotStore`, `TransactionalOutbox`, `Projection` / `ProjectionRunner`, `EventUpcaster`, `EventSourcedRepository` |
+
+---
+
+## Integration
+
+Bridge to the rest of your stack — outbound webhooks, inbound webhooks, identity, content management, notifications, plugins and rules.
+
+| Guide | What You'll Learn |
+|-------|-------------------|
+| [Callbacks (outbound webhooks)](callbacks.md) | Subscriptions, HMAC signing, retry, execution tracking |
+| [Webhooks (inbound)](webhooks.md) | Signature validation, idempotency, listener pattern |
+| [Notifications](notifications.md) | Email / SMS / push ports, dummy + SMTP adapters |
+| [IDP (Identity Provider)](idp.md) | `IdpAdapter` port + internal-DB reference adapter, login, MFA, roles |
+| [ECM (Content Management)](ecm.md) | Document storage, metadata, folders, e-signature ports + adapters |
+| [Plugins](plugins.md) | `@plugin`, `@extension`, `@extension_point`, `PluginManager`, dependency resolution |
+| [Rule Engine](rule-engine.md) | YAML DSL, AST, batch evaluation, repository |
 
 ---
 
@@ -179,7 +196,15 @@ pyfly/
 ├── observability/   Metrics, tracing, health checks
 ├── actuator/        Monitoring endpoints, extensible registry
 ├── admin/           Embedded management dashboard, SSE, server mode
-├── transactional/   SAGA and TCC distributed transactions
+├── transactional/   Saga, Workflow, TCC; persistence adapters; DLQ; scheduler
+├── eventsourcing/   Aggregates, event store, snapshots, outbox, projections
+├── callbacks/       Outbound webhooks (event → external HTTP)
+├── webhooks/        Inbound webhooks with signature validation + idempotency
+├── notifications/   Email / SMS / push provider abstraction
+├── idp/             Identity-provider adapter (auth, users, roles, MFA)
+├── ecm/             Documents, folders, e-signature ports + adapters
+├── plugins/         Plugin system with extension points + lifecycle
+├── rule_engine/     YAML-based business rules (AST + evaluator)
 ├── shell/           CLI commands, runners, Click adapter
 ├── testing/         Test fixtures and assertions
 └── cli/             Project scaffolding and tooling

docs/modules/callbacks.md

@@ -0,0 +1,41 @@
+# Callbacks (outbound webhooks)
+
+`pyfly.callbacks` ships outbound notifications to configured external URLs
+when domain events fire — the symmetric pair to `pyfly.webhooks`.
+
+## Configure subscriptions
+
+```python
+from pyfly.callbacks import (
+    CallbackConfig, CallbackSubscription, InMemoryCallbackConfigRepository,
+)
+
+config = CallbackConfig(
+    tenant_id="acme",
+    name="webhook-suite",
+    secret="topsecret",
+    subscriptions=[
+        CallbackSubscription(
+            event_type="OrderPlaced",
+            target_url="https://customer.example.com/hooks/orders",
+        ),
+        CallbackSubscription(
+            event_type="*",
+            target_url="https://audit.example.com/all",
+        ),
+    ],
+)
+await configs.save(config)
+```
+
+## Dispatch an event
+
+```python
+results = await callback_dispatcher.dispatch(
+    "acme", "OrderPlaced", {"id": 1, "amount": 99}
+)
+```
+
+Each match retries on failure (`max_attempts`, `backoff_ms`), records an
+``X-Pyfly-Signature`` HMAC header when ``secret`` is set, and persists a
+``CallbackExecution`` for observability.

docs/modules/ecm.md

@@ -0,0 +1,42 @@
+# Enterprise Content Management (ECM)
+
+`pyfly.ecm` provides hexagonal abstractions for document storage,
+metadata, folders and e-signature workflows.
+
+## Document upload / download
+
+```python
+from pyfly.ecm import (
+    DocumentService, LocalFilesystemStorageAdapter,
+)
+from pyfly.ecm.in_memory import InMemoryFolderRepository, InMemoryMetadataStorage
+
+service = DocumentService(
+    storage=LocalFilesystemStorageAdapter("/var/firefly/docs"),
+    metadata=InMemoryMetadataStorage(),
+    folders=InMemoryFolderRepository(),
+)
+
+doc = await service.upload(name="contract.pdf", content=pdf_bytes, content_type="application/pdf")
+content = await service.download(doc.id)
+```
+
+## E-signature
+
+```python
+from pyfly.ecm import (
+    ESignatureService, NoOpESignatureAdapter, Recipient, SignatureRequest,
+)
+
+service = ESignatureService(adapter=NoOpESignatureAdapter())
+envelope = await service.request(
+    SignatureRequest(
+        document_id=doc.id,
+        recipients=[Recipient(name="Alice", email="alice@example.com")],
+        subject="Please sign your contract",
+    ),
+)
+```
+
+Replace ``NoOpESignatureAdapter`` with a real provider (DocuSign, Adobe
+Sign, Logalty) by implementing the ``ESignatureAdapter`` protocol.

docs/modules/eventsourcing.md

@@ -0,0 +1,72 @@
+# Event Sourcing
+
+`pyfly.eventsourcing` is a port of `org.fireflyframework.eventsourcing`.
+Aggregates emit `DomainEvent`s; an `EventStore` persists them; a
+repository replays the stream to reconstruct state; a snapshot store
+truncates replay cost; a `TransactionalOutbox` provides at-least-once
+delivery to a broker; `ProjectionRunner` updates read models.
+
+## Defining an aggregate
+
+```python
+from dataclasses import dataclass
+from pyfly.eventsourcing import AggregateRoot, DomainEvent
+
+@dataclass
+class OrderPlaced(DomainEvent):
+    order_id: str = ""
+    amount: int = 0
+
+class Order(AggregateRoot):
+    def __init__(self) -> None:
+        super().__init__()
+        self.amount = 0
+        self.when(OrderPlaced, lambda agg, evt: setattr(agg, "amount", evt.amount))
+```
+
+## Saving and loading
+
+```python
+from pyfly.eventsourcing import (
+    InMemoryEventStore, InMemorySnapshotStore,
+)
+from pyfly.eventsourcing.repository import EventSourcedRepository
+
+store = InMemoryEventStore()
+snapshots = InMemorySnapshotStore()
+repo = EventSourcedRepository(store, factory=Order, snapshots=snapshots)
+
+order = Order()
+order.id = "o-1"
+order.apply(OrderPlaced(order_id="o-1", amount=99))
+await repo.save(order)
+
+# Restart, then:
+recovered = await repo.load("o-1")
+assert recovered.amount == 99
+```
+
+## Outbox pattern
+
+```python
+from pyfly.eventsourcing import TransactionalOutbox
+
+async def publish(envelope):
+    await broker.publish(envelope)
+
+outbox = TransactionalOutbox(publish=publish, max_attempts=5)
+await outbox.start()
+await outbox.enqueue(envelope_for_event)
+```
+
+## Projections
+
+```python
+from pyfly.eventsourcing.projection import FunctionProjection, ProjectionRunner
+
+async def read_model(envelope):
+    await db.upsert("orders_view", envelope)
+
+runner = ProjectionRunner(FunctionProjection("orders_view", read_model), store)
+await runner.start()
+```

docs/modules/idp.md

@@ -0,0 +1,25 @@
+# Identity Provider (IDP)
+
+`pyfly.idp` bridges to an external identity provider behind a single
+``IdpAdapter`` protocol — covering user management, authentication,
+session introspection, MFA challenges and role assignment.
+
+## Built-in adapter
+
+`pyfly.idp.adapters.internal_db.InternalDbIdpAdapter` is a reference
+implementation backed by an in-memory map and bcrypt-hashed passwords.
+
+```python
+from pyfly.idp import IdpUser, LoginRequest, InternalDbIdpAdapter
+
+adapter = InternalDbIdpAdapter()
+user = await adapter.create_user(IdpUser(username="alice", email="a@x.com"), "secret123")
+auth = await adapter.login(LoginRequest(username="alice", password="secret123"))
+print(auth.access_token)
+```
+
+## Implementing your own
+
+Satisfy the ``IdpAdapter`` Protocol — every method is async; password and
+token storage are entirely up to the adapter. Wire your adapter as the
+``IdpAdapter`` bean and the framework picks it up.