Skip to content

vulkan: extend PTQ1_0 fast integer-dot path to INTEL_XE1 iGPUs (Arrow Lake-H) - #274

Open
rriley-quorum wants to merge 1 commit into
PrismML-Eng:prismfrom
rriley-quorum:vulkan-arc-140t-ptq1-fix
Open

rriley-quorum wants to merge 1 commit into
PrismML-Eng:prismfrom
rriley-quorum:vulkan-arc-140t-ptq1-fix

Conversation

@rriley-quorum

Copy link
Copy Markdown

Overview

PR #238 gated the PTQ1_0 fast integer-dot shader on INTEL_XE2 classification (minSubgroupSize == 16). Intel Arc 140T (Arrow Lake-H, Xe-LPG+) reports minSubgroupSize == 8 and classifies as INTEL_XE1 instead, so it falls through to the blanket Intel-Windows opt-out and stays on the slow dequant path, even though the shader only needs integerDotProduct4x8BitPackedSignedAccelerated, which INTEL_XE1 classification already guarantees.

Extends the existing check to also allow PTQ1_0 through under INTEL_XE1 when integer_dot_product is set.

Measured on Intel Arc 140T (Arrow Lake-H, driver 101.8860), Ternary-Bonsai-2-27B-PTQ1_0: 1.3 -> 8.8 tok/s.

Additional information

Closes #270

Requirements

  • I have read and agree with the contributing guidelines
  • AI usage disclosure: YES, used Claude Sonnet 5 to investigate tok/s and identify solution

… Lake-H)

PR PrismML-Eng#238 gated the PTQ1_0 fast integer-dot shader on INTEL_XE2
classification (minSubgroupSize == 16). Intel Arc 140T (Arrow Lake-H,
Xe-LPG+) reports minSubgroupSize == 8 and classifies as INTEL_XE1
instead, so it falls through to the blanket Intel-Windows opt-out and
stays on the slow dequant path, even though the shader only needs
integerDotProduct4x8BitPackedSignedAccelerated, which INTEL_XE1
classification already guarantees.

Extends the existing check to also allow PTQ1_0 through under
INTEL_XE1 when integer_dot_product is set.

Measured on Intel Arc 140T (Arrow Lake-H, driver 101.8860),
Ternary-Bonsai-2-27B-PTQ1_0: 1.3 -> 8.8 tok/s.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@bri-prism

Copy link
Copy Markdown

Code review from the Xe2/Xe3 side (Intel Arc B390, Panther Lake, classifies as INTEL_XE2 with minSubgroupSize = 16; I don't have an Xe1 part):

  • The change is minimal and correct as far as I can see. The new condition is INTEL_XE2 || (INTEL_XE1 && integer_dot_product && src0_type == PTQ1_0), so:
    • the INTEL_XE2 branch, which the B390 and presumably Lunar Lake take, is unchanged;
    • INTEL_XE1 is admitted only for PTQ1_0;
    • that relies on the accelerated 4x8 integer dot, which ggml_vk_get_device_architecture() already requires for the INTEL_XE1 class.
  • Other quant types on Xe1 keep their existing path, so nothing else on older Intel iGPUs should move.
  • It merges cleanly onto current prism (adfffbe41), although the branch is behind.

The one open item is correctness of the new path on an Xe1 part. test-backend-ops test -b Vulkan0 -o MUL_MAT -p ptq1_0 on the 140T would cover it. Run it with GGML_VK_DISABLE_MMVQ unset: the variable is presence-checked, so =0 still disables MMVQ.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

PTQ1_0 Vulkan fast path (#238) misses Xe3 iGPUs reporting minSubgroupSize == 8 (Intel Arc 140T / Panther Lake)

2 participants