Skip to content

fix(BACKEND-VULKAN): correct three false records about the B60; VK-I's staging half is answered by ReBAR - #3334

Open
phantomic12 wants to merge 2 commits into
mudler:mainfrom
phantomic12:row/BACKEND-VULKAN-VKI-REBAR
Open

phantomic12 wants to merge 2 commits into
mudler:mainfrom
phantomic12:row/BACKEND-VULKAN-VKI-REBAR

Conversation

@phantomic12

@phantomic12 phantomic12 commented Sep 27, 2026 •

Copy link
Copy Markdown
Contributor

An Intel Arc Pro B60 (garlic-clove, 8086:e211, ASUS subsys 1849:6023, xe driver) is on the estate, and it is the hardware VK-I was scoped against — vulkan-full-support.md recorded the 2026-08-06 decision as "GB10 first, acquire later", and the Arc half of that acquisition has now happened.

Three records about it are wrong, and one planned deliverable turns out to be answered rather than buildable.

1. The staging-path half needs no code

VK-I's deliverable is "the staging path for non-host-visible memory". Measured 2026-09-27 with vulkaninfo: the device is PHYSICAL_DEVICE_TYPE_DISCRETE_GPU with two heaps (20.91 GiB DEVICE_LOCAL, 23.44 GiB host) — but memoryTypes[3] and [6] expose DEVICE_LOCAL | HOST_VISIBLE | HOST_COHERENT (propertyFlags 0x0007) on the device-local heap. Resizable BAR maps the VRAM into the host address space, so the condition VK-I was built to survive does not arise on this card, and VulkanContext's existing preference for a device-local host-visible type (vulkan_context.cpp:873) already selects it.

Writing a staging path here would be code for a condition this hardware does not have.

This retires a risk; it does not delete a requirement. The property is ReBAR's, not Arc's: a discrete card without ReBAR exposes device-local VRAM the host may not dereference, and there the GGUF keep-quant CPU fall-through and the portable reference tier are corruption, not slowness. The ordered-fallback requirement stands, and is now written down.

2. A false code comment

The BACKEND-VULKAN-KEEPQUANT block in src/vt/vulkan/vulkan_ops.cpp asserted "The B60 is integrated". vulkaninfo says DISCRETE. The code was right for the wrong reason — VulkanBackend::DeviceMemoryIsHostAddressable() returns true unconditionally and the allocation really is host-dereferenceable — but the comment was a trap rather than a typo: a non-ReBAR Arc card would be discrete and non-host-visible, and the comment would tell the next reader it was fine. Comment only; no code path changes.

3. A false registry line

.agents/environment.md asserted "No Intel GPU exists on any box here, so BACKEND-XPU end-to-end work is HW-BLOCKED". garlic-clove is an Intel GPU.

BACKEND-XPU nonetheless stays SPIKE for a different and still-true reason: the SYCL toolchain is absent, not the hardware — no icpx, no sycl-ls, no /dev/accel nodes. The Level Zero runtime libraries are installed and are not sufficient. Closing BACKEND-XPU now needs a oneAPI install, not hardware acquisition.

Also stale — and the reason none of this was caught

The BACKEND-VULKAN matrix cell still described the 2026-07-22 skeleton: 8 native ops, "no model runs on Vulkan". The tree registers 35 Vulkan ops including the full GDN/SSM set, GGUF keep-quant/TQ1_0, EXL3 and MoE, and a model does run: test_opt_paged_engine is 6/6 prompts token-exact (96/96) with 0 declines, test_vulkan_backend 35/35 (2650), test_backend_cross_device 11/11 (132). Corrected by a dated append rather than a rewrite, per the keyed-record rule.

VK-I is partially answered; the half that matters is still owed

"The gate re-run where Vulkan actually matters." On GB10, as §0 already says, llama.cpp's own CUDA backend beats both of us and Vulkan is nobody's fastest path — every Vulkan speed number so far is either llvmpipe or the wrong chip. 20.91 GiB of device-local memory is the binding constraint, so 27B does not fit.

check-agent-record and check-device-leakage both stay green. Other red gates on this base are pre-existing at 1de097c46.

Note for review: src/vt/vulkan/vulkan_ops.cpp is a comment-only change and was not compiled here — the gate evidence is the record checkers, not a build.

ISSUE-LOCAL-01M3JGPW1FN5SG506ANGBT3C0Q
ISSUE-LOCAL-01M3JGXN1QTRE1D7WX3JMV0KE2

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:codebuff/buffy [freebuff]

@localai-org-maint-bot localai-org-maint-bot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Source review at a07535c641f378b67ebe613b3bae480533eefc55: the new non-ReBAR conclusion contradicts the allocator.

src/vt/vulkan/vulkan_context.cpp:872-875 first tries DEVICE_LOCAL | HOST_VISIBLE | HOST_COHERENT, then falls back to HOST_VISIBLE | HOST_COHERENT without DEVICE_LOCAL. It refuses initialization if neither exists. AllocBuffer uses that selected type and maps it with vkMapMemory; vulkan_backend.cpp:131-135 documents why every allocation is host-addressable. Therefore failure to find the first, device-local type does not imply the host vec_dot path reads unmappable VRAM: the second type is a valid host-visible fallback.

Please correct the new comment at vulkan_ops.cpp:2144-2153 and the matching spec/issue statements that call this memory corruption and require refusing both reference paths whenever the first lookup fails. Distinguish host visibility (required for correctness) from device locality (preferred for performance). The B60 being discrete is a useful correction, but its ReBAR configuration is not the only way this allocator guarantees host addressability.

@mudler this needs correction before merge. This is a source-backed finding, not device acceptance; Python/CMake/compiler tools are unavailable here, and I have not independently run the reported B60 measurements.

@phantomic12
phantomic12 force-pushed the row/BACKEND-VULKAN-VKI-REBAR branch 2 times, most recently from 657cb64 to 8367cad Compare September 28, 2026 19:51
@phantomic12

Copy link
Copy Markdown
Contributor Author

Corrected, and you were right — this was a real defect in the branch, not a wording preference.

The claim rested on a discrete card without ReBAR exposes DEVICE_LOCAL VRAM the host cannot address, so the host vec_dot kernel reads memory it cannot. But the allocator never depends on DEVICE_LOCAL. vulkan_context.cpp:872-875:

int type = FindMemoryType(mem, ~0u, kHostFlags | VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT);
unified_memory_ = type >= 0;
if (type < 0) type = FindMemoryType(mem, ~0u, kHostFlags);
VT_CHECK(type >= 0, "vulkan: no HOST_VISIBLE|HOST_COHERENT memory type");

It prefers device-local host-visible and falls back to plain HOST_VISIBLE | HOST_COHERENT, refusing to initialize only if that fails too. AllocBuffer allocates from whichever type was selected and vkMapMemory-maps it, and DeviceMemoryIsHostAddressable() returns true unconditionally (vulkan_backend.cpp:130-135) because every allocation is mapped. So failing the first lookup does not imply unmappable VRAM — exactly as you said.

The distinction now written down: host visibility is what correctness requires and the ordered fallback supplies it on any board; device locality is a performance preference whose absence costs the GPU a bus crossing instead of a local VRAM hit. That is slowness, not unsafety. unified_memory_ records which happened, and it is the performance branch, not a safety one.

Withdrawn: the claim that the keep-quant fall-through and the portable reference tier must both be refused on such a board. They are correct there and slow. The ordered-fallback requirement itself is unchanged and is the one already in the code.

Corrected at all three sites you named, plus the one you did not:

  • src/vt/vulkan/vulkan_ops.cpp, the BACKEND-VULKAN-KEEPQUANT block
  • .agents/specs/vulkan-full-support.md §6.2
  • .agents/environment.md, the garlic-clove board cell
  • the commit message, which repeated the claim

The two issue records did not carry it, so nothing to fix there.

Amended onto 8367cad35 rather than stacked, so the branch stays one commit and the wrong claim is never in its history. check-agent-record, check-symbol-anchors (5 stale, identical to main, repaired by the record-anchor work and not introduced here), check-conflict-markers, and both commit-range checkers are rc=0.

Still unrun, and unchanged by this: no vulkan build and no device. The allocator behaviour is read from source and the B60 figures come from the recorded vulkaninfo capture on garlic-clove; neither was re-measured here.

@mudler-agent mudler-agent left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes requested: the PR body and canonical issue still say non-ReBAR means corruption, while the allocator, updated spec, and source comment establish host-visible fallback and therefore slowness rather than unsafety. Make the landing message and issue agree with the implementation.

…eparate host visibility from device locality

vulkaninfo says the Intel Arc Pro B60 on garlic-clove is DISCRETE, and
three records here said "integrated". They were wrong about the board
and right about the consequence, for a reason none of them gave. This
corrects the board claim and, on review, corrects the consequence too:
an earlier revision of this branch called the no-ReBAR case MEMORY
CORRUPTION, and that was wrong in a way that mattered.

WHAT WAS WRONG, precisely. The claim rested on "a discrete card
without ReBAR exposes DEVICE_LOCAL VRAM the host cannot address, so the
host vec_dot kernel reads memory it cannot". But the allocator never
depends on DEVICE_LOCAL. vulkan_context.cpp:872-875:

  int type = FindMemoryType(mem, ~0u, kHostFlags | DEVICE_LOCAL_BIT);
  unified_memory_ = type >= 0;
  if (type < 0) type = FindMemoryType(mem, ~0u, kHostFlags);
  VT_CHECK(type >= 0, "vulkan: no HOST_VISIBLE|HOST_COHERENT memory type");

It PREFERS device-local host-visible, and when that lookup fails it
FALLS BACK to plain HOST_VISIBLE | HOST_COHERENT, refusing to initialize
only if that fails too. AllocBuffer allocates from whichever type was
selected and vkMapMemory-maps it, and DeviceMemoryIsHostAddressable()
returns true unconditionally (vulkan_backend.cpp:130-135) because every
allocation is mapped. So failing to find the device-local type does NOT
imply unmappable VRAM: the fallback is host-visible, and the host kernel
keeps reading memory it can address.

THE DISTINCTION THAT REPLACES IT. Host visibility is what correctness
requires, and the ordered fallback supplies it on any board.
Device locality is a performance preference, and its absence costs the
GPU a trip across the bus instead of a local VRAM hit. That is
SLOWNESS, not unsafety. unified_memory_ is the flag recording which of
the two happened, and it is the performance branch, not a safety one.

So the standing requirement is unchanged and is the one already in the
code: keep PREFERRING a device-local host-visible type, and let the
fallback carry correctness where none exists. What is withdrawn is the
claim that both the keep-quant fall-through and the portable reference
tier must be REFUSED on such a board. They are correct there and slow.

Corrected at all three sites that carried it: the
BACKEND-VULKAN-KEEPQUANT block in src/vt/vulkan/vulkan_ops.cpp, spec
vulkan-full-support.md §6.2, and the garlic-clove board cell in
.agents/environment.md.

(1) THE BOARD. Intel(R) Arc(tm) Pro B60 Graphics (BMG G21),
8086:e211, ASUS subsys 1849:6023, xe driver. 20.91 GiB DEVICE_LOCAL
plus 23.44 GiB host heap, and memoryTypes[3]/[6] expose DEVICE_LOCAL |
HOST_VISIBLE | HOST_COHERENT (propertyFlags 0x0007) on the
device-local heap, so the condition VK-I was built to survive does not
arise on this card. VK-I's staging half is answered by ReBAR; its
gate-re-run half is not, and remains the whole value of VK-I.

(2) A FALSE CODE COMMENT. The BACKEND-VULKAN-KEEPQUANT block asserted
"The B60 is integrated". vulkaninfo says DISCRETE. The code was right
for the wrong reason -- DeviceMemoryIsHostAddressable() returns true
unconditionally and the allocation really is host-dereferenceable --
but the comment named a board property that does not exist, and would
have taught a reader that a discrete card is safe here, which is the
inference that does not hold.

(3) A STALE OWE. backend-matrix.md recorded VK-I as fully answered by
acquisition. Acquisition answers the staging half only.

GATES rc=0: check-agent-record, check-symbol-anchors (5 stale, identical
to main and repaired by the record-anchor work, not introduced here),
check-conflict-markers, check-commit-trailers --range,
check-commit-style --range.

NOT RUN: no vulkan build and no device. The allocation behaviour is
read from the allocator source and the B60 figures from the recorded
vulkaninfo capture on garlic-clove; neither was re-measured here.

ISSUE-LOCAL-01M3JGPW1FN5SG506ANGBT3C0Q
ISSUE-LOCAL-01M3JGXN1QTRE1D7WX3JMV0KE2

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:codebuff/buffy [freebuff]
@phantomic12
phantomic12 force-pushed the row/BACKEND-VULKAN-VKI-REBAR branch from 8367cad to b49cd6a Compare October 1, 2026 19:17
…llback model

The review of b49cd6a noted the canonical issue still read the
no-ReBAR case as corruption after the code comment, spec and board cell
were corrected to the ordered-fallback model. The issue now states the
same model the code carries: host addressability comes from the
allocator's HOST_VISIBLE fallback, not from the card being integrated,
so a non-ReBAR board pays a bus crossing -- slowness, not unsafety.

ISSUE-LOCAL-01M3JGPW1FN5SG506ANGBT3C0Q

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:anthropic/devin [devin]

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants