Skip to content

perf(stream): extract reportStageError out of GraphInterpreter.execute hot loop#3119

Merged
He-Pin merged 4 commits into
apache:mainfrom
He-Pin:perf/stream/extract-reportStageError
Jun 22, 2026
Merged

perf(stream): extract reportStageError out of GraphInterpreter.execute hot loop#3119
He-Pin merged 4 commits into
apache:mainfrom
He-Pin:perf/stream/extract-reportStageError

Conversation

@He-Pin

@He-Pin He-Pin commented Jun 22, 2026

Copy link
Copy Markdown
Member

Motivation

GraphInterpreter.execute is on the hottest path of the stream engine — it runs once per queued event. Inside the main event loop it declares a local def reportStageError(e: Throwable), which the Scala 2.13 compiler lifts into a private final synthetic method on the enclosing class (not a per-iteration closure allocation, since the body only references class-level fields and methods). Lifting it explicitly out of the loop makes the intent obvious, shrinks the body of execute, and gives the JIT a tighter compile target for inlining the genuinely hot processPush/processPull paths that follow each catch block.

Modification

Move the nested reportStageError definition out of execute into a private instance method on GraphInterpreter, placed between shutdownCounters and execute. The three call sites in execute (normal dispatch, chased PUSH, chased PULL) now invoke the instance method directly. The method body and the catch-bound locals at the call sites are unchanged.

Visibility is private (not private[stream]) because the only call sites live in GraphInterpreter.execute itself — ActorGraphInterpreter does not reference this method, and there is no planned external caller. If one materialises, widening later is a one-line change.

Because activeStage, chasedPush, chasedPull, chaseCounter, log, logics, defaultErrorReportingLogLevel, enqueue, and afterStageHasRun are all fields/methods of the enclosing GraphInterpreter, the lifted method observes exactly the same state as the previous nested def, and the chasing abort semantics (reset chaseCounter, re-enqueue pending chased push/pull) are preserved character-for-character.

Result

  • Smaller execute bytecode → friendlier to JIT inlining of the hot processPush/processPull calls that follow each try/catch.
  • Identical error-path semantics: log via the configured LogLevels attribute, call activeStage.failStage, reset chaseCounter, and re-enqueue any pending chased push/pull.
  • No allocation reduction — a previous revision of this PR claimed per-iteration closure allocation; that was incorrect. Scala 2.13 already compiles the nested def as a synthetic method when only class fields are referenced.

Benchmarks

Initial JMH data with InterpreterBenchmark (graph_interpreter_100k_elements, 100k elements per invocation) showed improvement in the shallow-pipeline case:

numberOfIds Baseline (main) With this PR Δ
1 45,474.58 ± 7,621.99 50,295.10 ± 1,221.59 +10.6%
5 11,731.50 ± 1,012.70 12,224.65 ± 866.05 +4.2%
10 5,879.32 ± 148.51 5,795.71 ± 233.77 ~0%

Caveats, flagged by the reviewer:

  • The baseline's error band on numberOfIds=1 (±7,622, ~17% relative) is far wider than the PR run (±1,222, ~2.4%), and the two confidence intervals overlap. This suggests the baseline run was noisy (GC, thermal, background processes) and the +10.6% should be treated as indicative rather than statistically decisive.
  • The effect shrinks with pipeline depth, as expected — per-event overhead matters less when stage work dominates.

The honest takeaway is: this change produces smaller, cleaner bytecode for execute, which is genuinely beneficial for JIT compilation regardless of the exact throughput delta. Reviewers should treat any performance claim here as "likely small, not regressive, and hard to measure at this level."

Tests

  • sbt "stream-tests/Test/testOnly org.apache.pekko.stream.impl.fusing.GraphInterpreterSpec" — 11/11 passed.
  • sbt "stream-tests/Test/testOnly org.apache.pekko.stream.impl.fusing.GraphInterpreterFailureModesSpec" — 9/9 passed. All 9 failure scenarios (failure in onPull, onPush, onUpstreamFinish, onUpstreamFailure, onDownstreamFinish, preStart, postStop, plus pending cancel/complete) exercise reportStageError through execute's catch blocks.
  • sbt "bench-jmh/Jmh/run ... .*InterpreterBenchmark.*" — results above.

No new test was added. The existing GraphInterpreterFailureModesSpec already provides directional coverage of the refactored method through every failure hook; a separate test targeting reportStageError in isolation would duplicate that coverage without improving regression detection (the method is private and reachable only through execute, which is precisely what the existing specs invoke).

Reviewer-Driven Revisions

Following an independent review, the following were corrected from the initial submission:

  1. Removed the incorrect "per-iteration closure allocation" claim. Scala 2.13 compiles the nested def as a synthetic method on the enclosing class when only class fields are referenced, so no closure is allocated per iteration.
  2. Narrowed private[stream] to private (the only callers are in this class; ActorGraphInterpreter does not reference the method).
  3. Removed the incorrect ActorGraphInterpreter usage claim from the description.
  4. Rewrote the scaladoc to describe the real benefit (smaller execute bytecode for JIT) rather than allocation reduction.
  5. Reverted a cosmetic rename of the catch-bound local from e to ex — per maintainer nit, there is no real shadowing since the catch-bound e and the method parameter e live in completely different scopes.

References

None.

…e hot loop

Motivation:
GraphInterpreter.execute is invoked once per event in the stream engine and shows up as
a ~10% CPU hotspot in flamegraphs of high-throughput gRPC workloads. The nested
`def reportStageError` closure inside the while-loop forced the Scala compiler to emit
closure-capture bytecode (loading activeStage, chasedPush, chasedPull, chaseCounter,
log, logics) on every iteration, even though every captured name is already a class field
and the closure is only invoked on the error path.

Modification:
Lift `reportStageError` to a private[stream] instance method of GraphInterpreter, placed
between `shutdownCounters` and `execute`. The three try/catch sites in execute (normal
dispatch, chased PUSH, chased PULL) now call the method directly. Rename the outer
`catch case NonFatal(e)` local to `ex` to remove name-shadowing with the method parameter.

Result:
- The hot while-loop body in `execute` shrinks by ~30 lines of bytecode, giving the JIT
  a tighter compile target and a better chance of inlining `processEvent`/`processPush`.
- Zero per-iteration closure-capture overhead (previously the nested `def` produced a
  fresh closure allocation each loop pass before JIT escape analysis could eliminate it).
- Behavior is preserved: `GraphInterpreterSpec` 11/11 tests pass (identity/chained/
  detacher/zip/broadcast/merge/balance/cycle).
- Incidental benefit: error-path logging is now declared once and easier to audit.

Tests:
- sbt "stream-tests/Test/testOnly org.apache.pekko.stream.impl.fusing.GraphInterpreterSpec"
  -> 11 passed, 0 failed.

References:
None - internal stream engine refactor.
@He-Pin He-Pin added the t:stream Pekko Streams label Jun 22, 2026
@He-Pin He-Pin added this to the 2.0.0-M4 milestone Jun 22, 2026
@He-Pin He-Pin requested a review from pjfanning June 22, 2026 07:02
Comment thread stream/src/main/scala/org/apache/pekko/stream/impl/fusing/GraphInterpreter.scala Outdated

@pjfanning pjfanning left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

A good candidate for backporting to 1.7.x branch

Motivation:
An independent review of apache#3119 identified three inaccuracies in the original
submission:

1. The claim that the nested `def reportStageError` inside `execute` allocated
   a fresh closure per loop iteration was incorrect. Scala 2.13 compiles the
   nested `def` as a private synthetic method on the enclosing class when only
   class fields are referenced — verified via `javap` against the baseline
   bytecode (`private final void reportStageError$1(java.lang.Throwable)`).
2. The claim that `ActorGraphInterpreter` referenced the lifted method was
   false — a search of the file yields zero references.
3. The `private[stream]` visibility granted package-wide access that no caller
   actually uses.

Modification:
- Rewrite the scaladoc on `reportStageError` to describe the real benefit:
  smaller `execute` bytecode for a tighter JIT compile target on the hot
  `processPush`/`processPull` paths that follow each try/catch. No allocation
  reduction is claimed.
- Narrow visibility from `private[stream]` to `private` (all three call sites
  live in `GraphInterpreter.execute` itself).
- Rename the catch-bound local `e` to `ex` at the remaining two call sites
  (chased PUSH and chased PULL) so all three try/catch blocks use the same
  naming convention. The main-dispatch call site was already renamed in the
  previous commit.

Result:
- Semantics preserved character-for-character (same log-level dispatch, same
  `failStage`, same chasing-abort branch).
- `GraphInterpreterSpec` 11/11 and `GraphInterpreterFailureModesSpec` 9/9
  pass locally; the existing failure spec already provides directional
  coverage of the refactored method through every failure hook.
- PR description updated in apache#3119 to reflect the corrected justification
  and to flag the JMH variance caveat.

Tests:
- sbt "stream-tests/Test/testOnly org.apache.pekko.stream.impl.fusing.GraphInterpreterSpec
  org.apache.pekko.stream.impl.fusing.GraphInterpreterFailureModesSpec"
  -> 20 passed, 0 failed.

References:
Refs apache#3119
@He-Pin

He-Pin commented Jun 22, 2026

Copy link
Copy Markdown
Member Author

I think we can have better numbers in Pekko GRPC 2.0.0. https://github.com/LesnyRumcajs/grpc_bench

Comment thread stream/src/main/scala/org/apache/pekko/stream/impl/fusing/GraphInterpreter.scala Outdated
He-Pin added 2 commits June 22, 2026 16:31
Motivation:
A maintainer nit on apache#3119 flagged the catch-bound local rename `e` -> `ex`
introduced in the previous commit as unnecessary. There is no real shadowing:
the catch-bound `e` and the method parameter `e` on `reportStageError` live
in completely different lexical scopes, so the rename adds no clarity and
diverges from the surrounding codebase convention.

Modification:
Revert `case NonFatal(ex) => reportStageError(ex)` back to
`case NonFatal(e) => reportStageError(e)` at all three call sites in
`GraphInterpreter.execute` (normal dispatch, chased PUSH, chased PULL).
No other changes.

Result:
- Code matches the surrounding convention in the stream engine.
- Semantics identical (catch-bound `e` still bound per call site, method
  parameter `e` still bound per invocation).
- `GraphInterpreterSpec` 11/11 and `GraphInterpreterFailureModesSpec` 9/9
  pass locally.
- PR description updated to record the revert as reviewer-driven revision apache#5.

Tests:
- sbt "stream-tests/Test/testOnly org.apache.pekko.stream.impl.fusing.GraphInterpreterSpec
  org.apache.pekko.stream.impl.fusing.GraphInterpreterFailureModesSpec"
  -> 20 passed, 0 failed.

References:
Refs apache#3119
Motivation:
A maintainer comment on apache#3119 noted that `@InternalApi` on a class-scoped
`private def` is meaningless — the annotation documents that a *non-private*
method is part of Pekko's internal API and may change without notice. When
the method is already `private` to the enclosing class, no external caller
can see it regardless, so the annotation adds no information.

Modification:
Remove `@InternalApi` from `reportStageError`. The `InternalApi` import is
retained because it is still used on the enclosing `GraphInterpreter`
companion object, the class itself, and on `activeStage` / `nonNull` which
are `private[stream]` (package-visible) and therefore still warrant the
annotation.

Result:
- No behavioral change; the method remains class-private and its semantics
  are preserved character-for-character.
- `GraphInterpreterSpec` 11/11 and `GraphInterpreterFailureModesSpec` 9/9
  pass locally.

Tests:
- sbt "stream-tests/Test/testOnly org.apache.pekko.stream.impl.fusing.GraphInterpreterSpec
  org.apache.pekko.stream.impl.fusing.GraphInterpreterFailureModesSpec"
  -> 20 passed, 0 failed.

References:
Refs apache#3119
@He-Pin

He-Pin commented Jun 22, 2026

Copy link
Copy Markdown
Member Author

Addressed both maintainer nits in two follow-up commits on this branch:

  • 1913c46 — Reverted the eex rename at all three call sites. Confirmed no real shadowing: the catch-bound e and the method parameter e live in different lexical scopes.
  • 3963499 — Removed the redundant @InternalApi from reportStageError. The InternalApi import is retained because it is still used on the enclosing GraphInterpreter companion object, the class itself, and the private[stream] activeStage/nonNull members (where the annotation remains meaningful since those members are package-visible).

Both commits verified locally: GraphInterpreterSpec 11/11 and GraphInterpreterFailureModesSpec 9/9 pass.

PR description updated to record these as reviewer-driven revisions. Thanks for the careful review.

@He-Pin He-Pin closed this Jun 22, 2026
@He-Pin He-Pin reopened this Jun 22, 2026
@He-Pin He-Pin merged commit 85bc2aa into apache:main Jun 22, 2026
9 checks passed
@He-Pin He-Pin deleted the perf/stream/extract-reportStageError branch June 22, 2026 09:38
@pjfanning

Copy link
Copy Markdown
Member

@He-Pin is this worth backporting to 1.7.x?

@He-Pin

He-Pin commented Jun 22, 2026

Copy link
Copy Markdown
Member Author

I don't think so, this is just a micro optimization, wdyt, otherwise we have much to backport.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

t:stream Pekko Streams

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants