linux_util: negotiate explicit DRM modifiers in dmabuf probe + guard NVIDIA OSR - #633
Draft
mysticalsoap wants to merge 2 commits into
Draft
linux_util: negotiate explicit DRM modifiers in dmabuf probe + guard NVIDIA OSR#633mysticalsoap wants to merge 2 commits into
mysticalsoap wants to merge 2 commits into
Conversation
NVIDIA's EGL/GBM stack doesn't support implicit-modifier dmabuf import
the way Mesa does, so glEGLImageTargetTexture2DOES failed with
GL_INVALID_OPERATION (0x502) and the whole session fell back to
software CEF rendering — even though the runtime shared-texture import
path (gpu_paint's Vulkan-based dmabuf_import.rs) already negotiates
explicit modifiers correctly and is unaffected.
Query eglQueryDmaBufModifiersEXT for ARGB8888, filtered to non-
external-only entries, and allocate the test GBM buffer via
gbm_bo_create_with_modifiers when both the extension and symbol are
available, passing the chosen modifier explicitly via
EGL_DMA_BUF_PLANE0_MODIFIER_{LO,HI}_EXT. Missing symbols or an empty
modifier list fall through to the existing implicit gbm_bo_create path
unchanged, so Mesa drivers are unaffected.
Verified against RTX 4090 + KWin: the probe now negotiates an explicit
modifier (0x300000000606013) and reports GBM -> EGL -> GL import OK.
Note: enabling the shared-texture path this way surfaces a separate,
still-unresolved failure in CEF's own GPU process (repeated "Unable to
initialize SkSurface" in shared_image_representation.cc, blank
window) that was previously masked by the software-rendering fallback.
See PR description for details — not yet safe to merge on its own.
Assisted-by: claude-sonnet-5
The modifier fix in the previous commit is real and correct — NVIDIA can genuinely import a dmabuf via the transport probe. But enabling shared textures on that basis surfaces a separate, independent failure: CEF's OSR compositor can't wrap the imported buffer as a writable render target (SkSurfaces::WrapBackendTexture fails, "Unable to initialize SkSurface" in Chromium's shared_image_representation.cc), producing a blank window every frame instead of a UI. This is not jellium-desktop's bug to fix. Electron hits the identical failure — same file, same message, same trigger (OSR + shared texture), same NVIDIA+Wayland scope, AMD/Mesa and non-OSR unaffected — and closed their own tracking issue as not planned with no root cause or workaround (electron/electron#49247). Both CEF and Electron only wrap Chromium's own GPU-compositing code; neither project controls this code path. Query EGL_VENDOR once the display is available and block shared textures on NVIDIA regardless of what the transport probe reports, trading a working (if slow) software-rendered UI for a correctly understood limitation instead of an unexplained blank window. Verified live on RTX 4090 + KWin: transport still negotiates the modifier and passes, the vendor guard overrides it, zero SkSurface errors, UI renders as before (software path, same as pre-fix baseline). Assisted-by: claude-sonnet-5
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
On NVIDIA/Wayland the dmabuf capability probe (
src/linux_util/src/dmabuf_probe.rs) allocates its test buffer with an implicit modifier and imports it via EGL without negotiating an explicit one. NVIDIA rejects implicit-modifier import, soglEGLImageTargetTexture2DOESfails withGL_INVALID_OPERATION(0x502) and shared textures latch off (same signature as #157). The Vulkan import path ingpu_paint/dmabuf_import.rsalready negotiates explicit modifiers; the EGL probe didn't.Fixing that alone regresses NVIDIA, though: once the probe passes, CEF's OSR compositor can't wrap the buffer as a writable render target ("Unable to initialize SkSurface" in Chromium's
shared_image_representation.cc), blanking the window. That's an upstream Chromium bug CEF and Electron both inherit — Electron closed the identical failure not-planned (electron/electron#49247). Full write-up and two-layer diagnosis in #632.Fix
Two commits:
eglQueryDmaBufModifiersEXTfor ARGB8888 (non-external-only), allocate viagbm_bo_create_with_modifiers, pass it viaEGL_DMA_BUF_PLANE0_MODIFIER_{LO,HI}_EXT. Missing symbols or no advertised modifier fall through to the existing implicit path, so Mesa is unchanged.EGL_VENDORand block shared textures on NVIDIA regardless of the probe result, so the corrected probe can't hand CEF the OSR path that blanks the window.Net-neutral by design: NVIDIA already fell back to software before this (via the 0x502 failure) and still does; Mesa already worked and is untouched. The value is a correct, Vulkan-path-consistent probe plus a guard that's ready for whenever the upstream OSR bug is fixed. Happy to drop the code and keep just the issue if you'd prefer.
Verification
just fmt/just lint/just buildpass. On an RTX 4090 + KWin (Plasma 6.7.3, driver 610.43.03): commit 1 alone makes the probe negotiate0x300000000606013and pass, then blanks with repeated SkSurface errors; commit 2 keeps the probe passing but blocks shared textures (one warn log), zero SkSurface errors, UI renders on the software path as before.Status
Draft — this doesn't improve NVIDIA performance (blocked upstream), it replaces an unexplained transport failure with a diagnosed one and lands groundwork. Flagging the
EGL_VENDORstring check as the main thing to sanity-check before this is ready.Assisted-by: claude-sonnet-5