feat(workflow_engine): Add in hook for producing occurrences from the stateful detector

[everettbu]

Test [new PR number opened]

[greptile-apps]

Greptile Summary

This PR implements a significant architectural enhancement to Sentry's workflow engine by adding the ability for stateful detectors to produce issue occurrences. The changes span across four main files:

Core Model Enhancement (detector.py): A new group_type property is added to the Detector model, providing typed access to the GroupType instance associated with each detector. This refactors the existing detector_handler property to eliminate code duplication and improve type safety by centralizing the group type lookup logic.

Handler Migration (grouptype.py): The MetricAlertDetectorHandler is refactored to inherit from StatefulDetectorHandler instead of DetectorHandler, removing its concrete evaluate() method implementation. This aligns with the broader migration towards stateful detection capabilities.

Processor Core Logic (detector.py): The most substantial changes occur in the processor, where the detector evaluation pipeline is enhanced to support occurrence production. Key modifications include:

• Changing return types from lists to dictionaries keyed by DetectorGroupKey to eliminate duplicate handling
• Adding a new abstract method build_occurrence_and_event_data that subclasses must implement
• Integrating Kafka occurrence production when detectors transition to non-OK status
• Removing redundant duplicate detection logic in favor of dictionary-based deduplication

Test Infrastructure (test_detector.py): Comprehensive test updates validate the new occurrence production functionality, including mock implementations of the abstract method and verification of Kafka integration.

These changes enable the workflow engine to create and emit proper IssueOccurrence objects to Kafka when detectors identify problems, completing the integration between the detection system and Sentry's issue tracking infrastructure. The architecture now supports both issue creation (via occurrences) and resolution (via status change messages).

Confidence score: 2/5

• This PR introduces significant architectural changes but has several implementation gaps that make it risky to merge
• The score reflects incomplete abstract method implementations and potential runtime failures in the MetricAlertDetectorHandler
• Files needing attention: src/sentry/incidents/grouptype.py (incomplete implementation), src/sentry/workflow_engine/processors/detector.py (verify abstract method contracts)

_{4 files reviewed, 3 comments}

_{Edit Code Review Bot Settings | Greptile}

[greptile-apps]

logic: StatefulDetectorHandler is an abstract base class that requires implementing several abstract methods: counter_names, get_dedupe_value, get_group_key_values, and build_occurrence_and_event_data. This class will not be instantiable without these implementations.

[greptile-apps]

logic: The hardcoded value 6 for group_2 seems inconsistent - the test data shows group_2 has value 10, but the expected occurrence is built with value 6

Suggested change

	occurrence_2, event_data_2 = build_mock_occurrence_and_event(
	detector.detector_handler, "group_2", 6, PriorityLevel.HIGH
	occurrence_2, event_data_2 = build_mock_occurrence_and_event(
	detector.detector_handler, "group_2", 10, PriorityLevel.HIGH

[greptile-apps]

logic: The test builds an expected result using build_mock_occurrence_and_event with 'val1' and value 6, but this doesn't match the test's group_key parameter 'group_key' - potential test logic error

[GitHoobar]

Review Summary

🏷️ Draft Comments (1)

Skipped posting 1 draft comments that were valid but scored below your review threshold (>=13/15). Feel free to update them here.

src/sentry/workflow_engine/models/detector.py (1)

59-61: group_type property calls grouptype.registry.get_by_slug(self.type) on every access, causing repeated registry lookups for the same detector and wasting CPU if accessed frequently.

📊 Impact Scores:

• Production Impact: 2/5
• Fix Specificity: 4/5
• Urgency Impact: 2/5
• Total Score: 8/15

🤖 AI Agent Prompt (Copy & Paste Ready):

In src/sentry/workflow_engine/models/detector.py, lines 59-61, the `group_type` property performs a registry lookup on every access, which is inefficient if accessed frequently. Decorate the property with `@functools.lru_cache(maxsize=1)` to cache the result per instance and avoid repeated lookups. Ensure you import `functools` at the top of the file if not already present.

---

[ron-x5labs]

Code Review: feat(workflow_engine): Add in hook for producing occurrences from the stateful detector

Problem

Stateful detectors could evaluate conditions and track state, but firing transitions produced no issue occurrence — evaluate_group_key_value left result = None with a TODO: Add hook here for generating occurrence. This PR adds that hook and switches detector evaluation results from lists to dicts keyed by DetectorGroupKey. (The PR body is content-free; intent inferred from the title and diff.)

Solution Reviewed

• StatefulDetectorHandler gains an abstract build_occurrence_and_event_data(group_key, value, new_status); evaluate_group_key_value calls it on every non-OK status transition and forwards occurrence + event_data through process_detectors → create_issue_occurrence_from_result → produce_occurrence_to_kafka. The PriorityLevel(new_status) conversion is safe (DetectorPriorityLevel mirrors PriorityLevel values; OK is handled by the resolution branch).
• process_detectors / DetectorHandler.evaluate / StatefulDetectorHandler.evaluate now return dict[DetectorGroupKey, DetectorEvaluationResult]; the duplicate-group-key guard + error log in process_detectors were removed — correct, dict keying makes duplicates structurally impossible.
• Detector.group_type property extracted and reused; tests restructured around the new hook.

Summary

Solid refactor of the plumbing, and the list→dict migration breaks no in-repo callers (process_detectors has no production caller yet; handler.evaluate is only called from process_detectors). Three issues keep this short of mergeable: the new tests cannot actually detect wrong group_key/value/priority being passed into the hook (id-only IssueOccurrence equality + one shared hardcoded id), the only non-test handler became uninstantiable (a registered grouptype now raises TypeError on instantiation instead of no-opping), and the newly-activated fire path silently swallows Kafka produce failures before committing state. All are non-blocking today but become expensive once the pipeline is wired up.

Files Reviewed

• src/sentry/workflow_engine/processors/detector.py — deeply reviewed
• src/sentry/incidents/grouptype.py — deeply reviewed
• tests/sentry/workflow_engine/processors/test_detector.py — deeply reviewed
• src/sentry/workflow_engine/models/detector.py — lightly reviewed (small property extraction)

Verification

• python3 -m py_compile on all four changed files — passed
• pytest — skipped: this environment has no Python 3.12 / sentry dependency stack or database; findings are grounded in direct source reads and cross-checked against upstream (the worktree is byte-identical to getsentry/sentry commit de60b7fb01a for the core files)

Issues Found

🔴 Blocking

None.

🟡 Non-blocking

• src/sentry/incidents/grouptype.py:11 — MetricAlertDetectorHandler is now uninstantiable; see inline comment.
• tests/sentry/workflow_engine/processors/test_detector.py:252 — hook-argument assertions are vacuous and the mock's event_data shape is not JSON-serializable; see inline comment.
• src/sentry/workflow_engine/processors/detector.py:62 — produce-then-commit ordering silently loses alerts on Kafka failure; see inline comment.

💡 Suggestions

• src/sentry/workflow_engine/processors/detector.py:230 — stale docstring; see inline comment.

Verdict

Recommend changes before merge — nothing is broken in production today (the pipeline has no production caller yet), but the hook-argument tests are vacuous, the placeholder handler crashes on instantiation, and the fire path loses alerts silently; all three are cheap to fix now and expensive to debug later.

[ron-x5labs]

Latent crash: this handler is now uninstantiable. StatefulDetectorHandler declares four abstract members (counter_names, get_dedupe_value, get_group_key_values, and the new build_occurrence_and_event_data) and this pass subclass implements none of them, so Detector.detector_handler (models/detector.py:86, group_type.detector_handler(self)) now raises TypeError for any metric_alert_fire detector instead of the previous instantiable no-op (evaluate -> []). That property handles its other two failure modes with logger.error + return None but has no guard for this one, so one such detector would abort the whole process_detectors batch. No production path creates these detectors yet, but the registered MetricAlertFire grouptype now points at a class that can never be constructed. Suggest keeping the placeholder instantiable until the real implementation lands:

class MetricAlertDetectorHandler(StatefulDetectorHandler[QuerySubscriptionUpdate]):
    counter_names: list[str] = []

    def get_dedupe_value(self, data_packet: DataPacket[QuerySubscriptionUpdate]) -> int:
        raise NotImplementedError

    def get_group_key_values(self, data_packet: DataPacket[QuerySubscriptionUpdate]) -> dict[str, int]:
        raise NotImplementedError

    def build_occurrence_and_event_data(self, group_key, value, new_status):
        raise NotImplementedError

(The # TODO: This will be a stateful detector when we build that abstraction comment above is also stale now — this PR is that abstraction.)

[ron-x5labs]

The new hook's argument propagation is untestable as mocked — and the mock's event_data shape would crash in production. Two issues with build_mock_occurrence_and_event:

1. IssueOccurrence.__eq__ compares only self.id (issues/issue_occurrence.py:181-184), and every occurrence built here shares the hardcoded id eb4b0acf…. So assertions pass vacuously: TestEvaluateGroupKeyValue.test_dedupe builds the expected occurrence with ("val1", 6) but calls evaluate_group_key_value("group_key", 10, …), and test_state_results_multi_group builds group_2's expected occurrence with value 6 while the packet carries 10. A handler that passed the wrong group_key, value, or new_status into the hook would keep every assertion green — the core behavior this PR introduces is untested. Derive the id from the hook args and/or assert the fields directly:

id=uuid.uuid5(uuid.NAMESPACE_X500, f"{group_key}:{value}:{new_status.name}").hex
# and/or:
assert results[0][1]["group_2"].result.fingerprint == handler.build_fingerprint("group_2")

2. event_data embeds raw datetime objects for "timestamp"/"received", but produce_occurrence_to_kafka runs plain json.dumps on the payload (issues/producer.py:72) — a real implementation following this shape would raise TypeError. Cf. uptime/issue_platform.py, which uses detection_time.isoformat(). The shape is never exercised because the tests mock the producer.

[ron-x5labs]

Fire-path delivery semantics: produce failures are silently swallowed, then state is committed anyway. create_issue_occurrence_from_result → produce_occurrence_to_kafka catches KafkaException and only logs it (issues/producer.py:82-90), so a dropped produce is indistinguishable from success here; handler.commit_state_updates() (line 68) then persists the new status/dedupe value, state_data.status == new_status on the next packet, and the occurrence is permanently lost — a silent missed alert with no metric. Conversely, a non-KafkaException raise (e.g. the event-id ValueError at producer.py:100-101) aborts before commit, so group keys already produced duplicate on the next packet. This ordering is now exercised for every occurrence the new hook produces. At minimum, make the at-least-once vs at-most-once choice explicit and emit a failure metric:

# make create_issue_occurrence_from_result return success (or re-raise)
produced = all(
    create_issue_occurrence_from_result(result)
    for result in detector_results.values()
    if result.result is not None
)
if detector_results:
    results.append((detector, detector_results))
if produced:
    handler.commit_state_updates()

[ron-x5labs]

nit (docs): this docstring still says "returns a list of DetectorEvaluationResult" — the signature and body now return a dict keyed by DetectorGroupKey:

Evaluates a given data packet and returns a dict of `DetectorEvaluationResult`
keyed by `DetectorGroupKey`.