fix(windows): redraw Flutter only when the captured WebView frame changed - #19
Merged
Y-PLONI merged 2 commits intoSep 7, 2026
Merged
Conversation
…nged Windows.Graphics.Capture delivers a frame for the captured composition visual on every compositor tick, whether or not the WebView repainted. The texture bridge forwarded each one with MarkTextureFrameAvailable, so Flutter re-rasterized at the display refresh rate for as long as any WebView was on screen (measured in Otzaria: ~430 texture redraws per 5 s with the page idle, ~340 with even the caret blink paused), and the raster thread copied the frame again on every one of those frames. The GPU bridge now keeps the frame Flutter samples (`surface_`) as the reference and compares every incoming frame against it with a small compute shader (one dispatch, 4-byte readback; no CPU copy of pixels). Only a frame with at least one differing pixel is copied into `surface_` and announced to Flutter. `GetSurfaceDescriptor` no longer copies on the raster thread; it just hands out the shared handle. With the page idle Flutter now draws only when the caret blinks (18 redraws per 5 s) and nothing at all when nothing changes; otzaria.exe idle CPU went from ~20% of a core to ~4%. The immediate context is shared by every WebView of the plugin, so the compare/copy sequence is serialized with a mutex on GraphicsContext. Falls back to the previous forward-every-frame behaviour when the compute shader cannot be built (feature level below 11_0, or d3dcompiler_47 missing; it is loaded on demand, no new link dependency). The frame pool stays dispatcher-bound: a free-threaded pool was tried and delivered no frames at all with this plugin's WRL event handlers.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Connection with issue(s)
Issues are disabled on this repository, so there is no issue to resolve. Downstream context: Y-PLONI/otzaria-word-editor#53 (typing latency investigation in the Word plugin, where the host-side measurements below were made).
What
On Windows the WebView is shown through a Flutter texture: WebView2 draws into a composition visual,
Windows.Graphics.Capturecaptures that visual, and the captured frame is copied into a DXGI shared texture that Flutter samples.Capturing a composition visual yields a frame on every compositor tick, whether or not the WebView repainted.
TextureBridge::OnFrameArrivedforwarded each of them withMarkTextureFrameAvailable, so Flutter re-rasterized at the display refresh rate for as long as any WebView was on screen, andGetSurfaceDescriptorcopied the frame again on the raster thread each time.This PR:
GetSurfaceDescriptorjust hands out the shared handle);GraphicsContext(several WebViews share it).It falls back to the previous forward-every-frame behaviour when the compute shader cannot be built (feature level < 11_0, or
d3dcompiler_47.dllmissing — loaded on demand, no new link dependency). The frame pool stays dispatcher-bound: a free-threaded pool was tried and delivered no frames at all with the plugin's WRL event handlers.Measurements
Otzaria, Debug build of the same commit, same machine, same 5-page document open in the same WebView; the only difference is this plugin. Flutter frames counted from the Dart VM timeline (
Animator::DrawLastLayerTrees= texture-driven redraws); latency measured with realSendInputkeystrokes into the WebView and screen polling of the caret line until its pixels change.otzaria.exeCPU while idle (percent of one core, 10 s)Latency to the screen is unchanged within noise: the hop count (Chromium → DWM → capture → Flutter → DWM) is the same. What changes is that an idle WebView no longer keeps Flutter's raster thread and the GPU busy at 60 fps.
Verification
flutter build windows --debugof Otzaria against this branch; the WebView renders, updates on typing and caret blink, survives tab switches (pause/resume recreates the surface), and the dev-plugin reload./W4,_HAS_EXCEPTIONS=0), no warnings.Testing and Review Notes
Build Otzaria (Debug) against this branch and open any plugin tab.
Record the Dart VM timeline for 5 s with the page idle:
Animator::DrawLastLayerTreesshould appear only when the page changes (caret blink, typing), not ~60 times per second.Type into the page, switch tabs away and back (pause/resume), resize the window, open two plugin tabs: the WebView must render and update in all cases.
On a machine without D3D feature level 11_0 or without
d3dcompiler_47.dllthe log printsWebView frame compare unavailableand behaviour is the same as before this PR.