Skip to content

Document that the dmabuf import path bypasses the vaMapBuffer2 guard #534

Description

@EtienneLescot

The dmabuf zero-copy import path never consults vaapi_is_safe_to_probe(). This is
safe today, but only for a non-obvious reason that is written down nowhere — so a
future change can silently reintroduce a core dump.

The asymmetry

vaapi_is_safe_to_probe() is called from exactly one place,
VideoEncoder::open (electron/native/pipewire-capture/src/encoder.rs:247):

if backend == Backend::Vaapi && forced.is_none() && !vaapi_is_safe_to_probe() {

The dmabuf route does not go through VideoEncoder::open at all. Capture::start
(src/capture.rs:262) builds DmabufImporter::new and then
VideoEncoder::open_importing, and dmabuf_import::available() constructs a full VAAPI
device on its own. Neither checks the guard.

Why it is currently harmless

The guard exists because the vendored ffmpeg 8.1 calls vaMapBuffer2, which libva only
gained in 2.22. On an older libva the implib trampoline does not fail cleanly — it
assert(0)s and dumps core.

But vaMapBuffer2 is only reached when a buffer is mapped for the CPU, i.e. the
hwupload / staging route. The zero-copy import never maps anything CPU-side, so it
never reaches that symbol.

Measured (Ubuntu 24.04.4, libva 2.20.0, AMD Mendocino / Radeon 610M, Mesa 25.2.8)

Vendored ffmpeg n8.1.2, CPU-upload path — aborts:

$ ffmpeg -init_hw_device vaapi=va:/dev/dri/renderD128 -filter_hw_device va \
    -f lavfi -i testsrc=size=320x240:rate=30:duration=1 \
    -vf 'format=nv12,hwupload' -c:v h264_vaapi -f mp4 -y out.mp4
implib-gen: libva.so.2: failed to resolve symbol 'vaMapBuffer2' via dlsym
ffmpeg: _libva_so_2_tramp_resolve: Assertion `0 && "Assertion in generated code"' failed.
exit=134   (SIGABRT, core dumped)

Creating the VAAPI device alone succeeds (exit 0), so the abort is specific to the map,
not to VAAPI initialisation.

The PR #508 helper on the same machine, whole-monitor capture, no env overrides:

[osc-dmabuf] negotiated 1920x1080 uses_dmabuf=1 modifier=0x200000000201b03
[osc-dmabuf] mmap fd=26 len=8294400 FAILED errno=1 (Operation not permitted)
[osc-dmabuf] mmap failed (this driver does not allow CPU mapping) — using GPU import path
{"event":"encoder-selection","rejected":[],"video":"vaapi"}
{"convertMs":0.0,"uploadMs":0.0000392,"encodeMs":0.467,"frames":7258,"dropped":0,"durationMs":120966}

No abort, dropped: 0, valid MP4.

Corollary worth recording

On a libva < 2.22 machine the ladder's guard removes VAAPI from the shm path, so the
dmabuf path is the only route to hardware H.264 there. Anything that disables the
dmabuf import on such a machine silently downgrades it to software encoding.

Ask

Documentation only, no behaviour change:

  • A comment at Capture::start's dmabuf branch and/or on open_importing stating the
    invariant: this path must never map a surface CPU-side, because it deliberately
    bypasses the vaMapBuffer2 guard.
  • A line in dmabuf_import.rs's module header saying the same, since that is where a
    CPU-side fallback would most plausibly be added.
  • Optionally note the libva 2.22 interaction in docs/dmabuf-vaapi-plan.md, which does
    not mention it.

Found while hardware-validating #508 on a Mendocino die (a different die from the Renoir
the existing measurements came from). The validation itself passed: 59.4 distinct
frames/s over a 2-minute capture, convertMs 0.0, and a stable fd count (35 → 35, max 36)
across 7258 imported frames.

Metadata

Metadata

Assignees

No one assigned

    Labels

    documentationImprovements or additions to documentation

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions