refactor(core): deprecate raising_dispatch (#17941)

## Description

Deprecates and removes `core.raising_dispatch` in favor of an opt-in `allow_raise=True` flag on the existing `core.dispatch` API. Tightens the in-tree blocking-exception hierarchy and aligns LangChain non-stream wrappers to tag the LLM span on AI Guard block.

### What changed

**1. `core.dispatch` gains `allow_raise=True`**

`ddtrace/internal/core/event_hub.py` — `dispatch()` and `dispatch_event()` accept a keyword-only `allow_raise: bool = False`. When set, listener `Exception`s propagate to the caller (first raiser wins; subsequent listeners are skipped). `BaseException`-derived exceptions (including `DDBlockException`) always propagate regardless of the flag — matching the existing default-deny semantics for blocks.

This collapses two near-duplicate dispatch entry points into one and lets call sites be explicit about whether listener exceptions should surface.

**2. `core.raising_dispatch` removed**

The symbol is removed from `ddtrace.internal.core` and `event_hub.py`. All in-tree call sites are migrated:
- `ddtrace/contrib/internal/langchain/patch.py` — six `before` events (chat/llm × generate/agenerate/stream).
- `ddtrace/contrib/internal/openai/patch.py` — four sites (sync `before`/`after`, async `before`/`after`).

The `.sg` rule and snapshot tests for `core-raising-dispatch` are removed since the symbol no longer exists.

**3. New `DDBlockException` base class**

`ddtrace/internal/_exceptions.py` introduces `DDBlockException(BaseException)` as the umbrella for any in-tree decision to abort the current operation (web-request blocking, AI Guard policy abort, future product blocks). `BlockingException` (ASM) and `AIGuardAbortError` (AI Guard) now derive from it.

This gives integrations a single, semantically-correct catch target for "the platform decided to block this call" — distinct from arbitrary user/library `Exception`s. It still inherits from `BaseException` so a generic `except Exception:` cannot silently swallow a block decision.

**4. LangChain non-stream wrappers: tag the LLM span on block**

The four non-stream wrappers (`traced_llm_generate`, `traced_llm_agenerate`, `traced_chat_model_generate`, `traced_chat_model_agenerate`) now have an explicit `except DDBlockException:` arm that calls `span.set_exc_info(*sys.exc_info())` before re-raising, in addition to the existing `except Exception:` arm.

Required because `AIGuardAbortError` derives from `BaseException` (via `DDBlockException`) and is therefore *not* caught by `except Exception:`. Without the explicit arm, blocked LangChain calls would emit an LLMObs span with no error info — leaving a hole between the AI Guard span (block decision) and the LLM span (no link back to the abort). Aligns LangChain's non-stream behavior with the contract from #17913 for openai (`_patched_endpoint` uses `except BaseException as e: err = e; raise`, then `_traced_endpoint`'s `finally` calls `set_exc_info`).

**5. `OpenAIAIGuardAbortError`: documented catchability asymmetry**

The compound class inherits from both `openai.UnprocessableEntityError` (`Exception`-derived) and `AIGuardAbortError` (now `BaseException`-derived). Via MRO it remains `Exception`-derived — so `except openai.APIError:` blocks still work for users migrating from non-AI-Guard error handling. The asymmetry vs plain `AIGuardAbortError` is now spelled out in an AIDEV-NOTE on the class. Code that wants uniform block detection across providers should branch on `isinstance(e, AIGuardAbortError)`.

## Testing

- `tests/internal/test_context_events_api.py` — new coverage for `dispatch(..., allow_raise=True)` semantics: `Exception`s propagate when set, are swallowed when not, listeners short-circuit on first raise, `BaseException`-derived listener exceptions (`DDBlockException` subclasses) always propagate regardless of the flag, the `dispatch_event` variant behaves identically, and the new exception inheritance is asserted.
- `tests/appsec/ai_guard/langchain/test_langchain.py` — two new tests pin the `set_exc_info`-on-block contract on the LLM span (sync `chat.invoke` + async `chat.ainvoke`). A regression that swaps the explicit `except DDBlockException` arm back to `except Exception` would surface immediately.
- Local runs:
  - `appsec::ai_guard_langchain` venv `5484ca0` (Python 3.13) — passes.
  - `appsec::ai_guard_openai` venv `1224d93` (Python 3.12) — passes (this was the suite that initially exposed the missing openai-contrib migration).
- `hatch run lint:fmt` clean on all modified files.

## Risks

- **`AIGuardAbortError` inheritance change is user-visible.** It now derives from `DDBlockException(BaseException)` instead of `Exception`. User code today doing `try: model.invoke(...) except Exception: ...` to handle aborts **will silently stop catching them**. This is intentional (the `BaseException` derivation is what prevents accidental swallowing of blocks) but it does break code that relied on the prior behavior. Spelled out in the release note's `upgrade` section. The OpenAI-compatible variant (`OpenAIAIGuardAbortError`) remains catchable by `except Exception:` via MRO for SDK compatibility.
- **`core.raising_dispatch` removal.** Internal helper under `ddtrace.internal.core` — not part of the public API. All in-tree call sites are migrated in this PR. The release note's `deprecations` section documents the migration path for any out-of-tree consumer that imported it.
- **Trace shape change on blocked LangChain non-stream calls.** The LLM span is now finished with `error == 1` and `error.type` containing `AIGuardAbortError`. Downstream consumers that filter LLM spans by `error == 0` will see blocked LangChain calls as errored — desired observability change per #17913 but worth flagging.
- **No public API breakage on `core.dispatch`.** `allow_raise` is keyword-only, defaults to `False`, preserves existing non-raising behavior for every existing caller.

## Additional Notes

Release note: `releasenotes/notes/aiguard-abort-baseexception-deprecate-raising-dispatch-3a5c7f61ee4d2e1b.yaml` covers the `AIGuardAbortError` inheritance change (`upgrade` section) and the `raising_dispatch` removal (`deprecations` section).

Co-authored-by: alberto.vara <alberto.vara@datadoghq.com>

Author: avara1986

.github/workflows/system-tests.yml

@@ -95,7 +95,7 @@ jobs:
           persist-credentials: false
           repository: 'DataDog/system-tests'
           # Automatically managed, use scripts/update-system-tests-version to update
-          ref: '0526d7eb6ad321c14f7d4c7574cf8970089757c6'
+          ref: '24c2e96e7703a8128d5a29cae2e95776d86ef790'
 
       - name: Download wheels to binaries directory
         uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
@@ -148,7 +148,7 @@ jobs:
           persist-credentials: false
           repository: 'DataDog/system-tests'
           # Automatically managed, use scripts/update-system-tests-version to update
-          ref: '0526d7eb6ad321c14f7d4c7574cf8970089757c6'
+          ref: '24c2e96e7703a8128d5a29cae2e95776d86ef790'
 
       - name: Build runner
         uses: ./.github/actions/install_runner
@@ -458,4 +458,4 @@ jobs:
           needs.integration-frameworks-system-tests.result == 'cancelled'||
           needs.tracer-release.result == 'failure' ||
           needs.tracer-release.result == 'cancelled'
-        run: exit 1
+        run: exit 1

.gitlab-ci.yml

@@ -16,7 +16,7 @@ variables:
   DD_VPA_TEMPLATE: "vpa-template-cpu-p70-10percent-2x-oom-min-cap"
   # CI_DEBUG_SERVICES: "true"
   # Automatically managed, use scripts/update-system-tests-version to update
-  SYSTEM_TESTS_REF: "0526d7eb6ad321c14f7d4c7574cf8970089757c6"
+  SYSTEM_TESTS_REF: "24c2e96e7703a8128d5a29cae2e95776d86ef790"
 
   # Profiling native build image (built from dd/images/dd-trace-py/profiling_native)
   PROFILING_NATIVE_IMAGE: "registry.ddbuild.io/dd-trace-py:v103334885-be1888c-profiling_native"

.sg/rules/core-raising-dispatch.yml

@@ -224,8 +224,8 @@ def _langchain_generate_finally(*args, **kwargs):
 
     Releases the AI Guard active counter that the matching ``.before``
     listener bumped. Dispatched from the contrib's ``finally`` block so it
-    fires on every exit path — success, block (``_raising_dispatch`` raises
-    out of ``.before``), or exception inside the underlying LLM call. The
+    fires on every exit path — success, block (``core.dispatch(..., allow_raise=True)``
+    raises out of ``.before``), or exception inside the underlying LLM call. The
     counter reset is a no-op when the counter is already zero, so listener
     invocations that don't pair with a ``.before`` set are safe.
     """
@@ -247,21 +247,21 @@ def _langchain_llm_stream_before(client: AIGuardClient, instance, args, kwargs):
 
 
 def _evaluate_langchain_messages(client: AIGuardClient, messages):
-    """Evaluate the prompt and surface a block as a returned ``AIGuardAbortError``.
+    """Evaluate the prompt and re-raise ``AIGuardAbortError`` on a block.
 
-    Returns the abort error so the contrib's dispatcher can re-raise it; the
-    ``AIGuardClient`` already gates on ``ai_guard_config._ai_guard_block``,
-    so a returned error always represents a blocking decision.
-    Allow / skip paths return ``None``.
+    Re-raises so the contrib's ``core.dispatch(..., allow_raise=True)``
+    propagates the abort. The ``AIGuardClient`` already gates on
+    ``ai_guard_config._ai_guard_block``, so a raised error always represents
+    a blocking decision. Allow / skip paths return ``None``.
     """
     from langchain_core.messages import HumanMessage
 
     # only call evaluator when the last message is an actual user prompt
     if len(messages) > 0 and isinstance(messages[-1], HumanMessage):
         try:
             client.evaluate(_convert_messages(messages), Options(block=ai_guard_config._ai_guard_block))
-        except AIGuardAbortError as e:
-            return e
+        except AIGuardAbortError:
+            raise
         except Exception:
             logger.debug("Failed to evaluate chat model prompt", exc_info=True)
     return None

.sg/tests/__snapshots__/core-raising-dispatch-snapshot.yml

@@ -66,6 +66,19 @@ class OpenAIAIGuardAbortError(openai.UnprocessableEntityError, AIGuardAbortError
             Catchable as either ``openai.APIError`` / ``openai.UnprocessableEntityError``
             (idiomatic OpenAI error handling, no retry on 422) or
             ``AIGuardAbortError`` (Datadog-specific, exposes ``action`` / ``reason``).
+
+            AIDEV-NOTE: catchability asymmetry vs plain ``AIGuardAbortError``.
+            ``AIGuardAbortError`` derives from ``DDBlockException(BaseException)``
+            so a generic ``except Exception:`` does NOT catch it (by design — a
+            block decision must not be silently swallowed). However,
+            ``OpenAIAIGuardAbortError`` *also* inherits from
+            ``openai.UnprocessableEntityError``, which is ``Exception``-derived,
+            so via MRO this subclass IS catchable by ``except Exception:``.
+            That asymmetry is intentional: the OpenAI contrib must keep
+            ``except openai.APIError:`` blocks working unchanged for users
+            migrating from non-AI-Guard error handling. Code that wants
+            uniform block detection across providers should branch on
+            ``isinstance(e, AIGuardAbortError)``.
             """
 
             def __init__(self, action, reason, tags=None, sds=None, tag_probs=None):
@@ -285,7 +298,7 @@ def _openai_chat_completion_before(client, kwargs):
     try:
         client.evaluate(ai_guard_messages, Options(block=ai_guard_config._ai_guard_block))
     except AIGuardAbortError as e:
-        return _wrap_abort_error(e)
+        raise _wrap_abort_error(e)
     except Exception:
         logger.debug("Failed to evaluate OpenAI chat completion request", exc_info=True)
     return None
@@ -298,7 +311,7 @@ def _openai_chat_completion_after(client, kwargs, resp):
     returns.  Skips streaming responses (handled separately) and when a
     framework evaluation is already active.
 
-    On block: returns an ``OpenAIAIGuardAbortError`` (or plain
+    On block: raises an ``OpenAIAIGuardAbortError`` (or plain
     ``AIGuardAbortError`` when the OpenAI SDK is not importable).  Allow /
     skip paths return ``None``.
     """
@@ -316,7 +329,7 @@ def _openai_chat_completion_after(client, kwargs, resp):
     try:
         client.evaluate(all_messages, Options(block=ai_guard_config._ai_guard_block))
     except AIGuardAbortError as e:
-        return _wrap_abort_error(e)
+        raise _wrap_abort_error(e)
     except Exception:
         logger.debug("Failed to evaluate OpenAI chat completion response", exc_info=True)
     return None

.sg/tests/core-raising-dispatch-test.yml

@@ -14,6 +14,7 @@
 from ddtrace.ext import http
 from ddtrace.internal import core
 from ddtrace.internal import telemetry
+from ddtrace.internal._exceptions import DDBlockException
 import ddtrace.internal.logger as ddlogger
 from ddtrace.internal.settings.asm import ai_guard_config
 from ddtrace.internal.telemetry import TELEMETRY_NAMESPACE
@@ -94,8 +95,13 @@ def __init__(self, message: Optional[str], status: int = 0, errors: Optional[lis
         super().__init__(message)
 
 
-class AIGuardAbortError(Exception):
-    """Exception to abort current execution due to security policy."""
+class AIGuardAbortError(DDBlockException):
+    """Exception to abort current execution due to security policy.
+
+    Inherits from ``DDBlockException`` (which is ``BaseException``-derived) so
+    that a generic ``except Exception:`` in user code does not accidentally
+    swallow an AI Guard block decision.
+    """
 
     def __init__(
         self,