Conversation
## Row `BACKEND-VULKAN`, branch `row/BACKEND-VULKAN-CONVFWD`; the same sub-row naming as `row/BACKEND-VULKAN-GDN` (mudler#145) and `row/BACKEND-VULKAN-GDN-CORE` (mudler#152). ## Before starting - Issue/PR search and existing claim: no open PR, claim or issue names `kCausalConv1dFwd` on Vulkan. Local issue filed and closed in this PR: `.agents/issues/BACKEND-VULKAN/ISSUE-LOCAL-01M3V01R90QQTQ08BWSBM7AGN2.md` (Kind: gap). - Pull request shape selected at row claim: one op, one shader, its tests and docs; helper PR. - Roadmap or matrix row: `.agents/backend-matrix.md` `BACKEND-VULKAN`; VK-G in `.agents/specs/vulkan-full-support.md`. `scripts/ready-for-helper.py --check-local`: structure OK, live claims not verified locally. - Exact current-code and test/evidence anchors inspected: the GDN-family block comment in `src/vt/vulkan/vulkan_ops.cpp` that left this op "for a follow-up rather than guessed at here" and named the two safe dispatch shapes; `src/vt/cpu/cpu_ops.cpp` `CausalConv1dFwdKernel`; `src/vt/ops.cpp` `CausalConv1dFwd` / `CheckConvCommon`; the decode-update and compressed-state cases in `tests/vt/test_vulkan_backend.cpp`; §6.0a of the spec. ## What changed A native Vulkan kernel for `vt::CausalConv1dFwd`, the GDN PREFILL depthwise causal conv, which ran on the portable CPU reference tier once per GDN layer. The default mapping is the first shape the existing comment named: one invocation per (sequence, channel), serial over its tokens, with the old state window copied to a private array before the write-back, as the CPU kernel's `old_row` does. An opt-in token split (`VT_VULKAN_CONV_TARGET_GROUPS`) stays in one dispatch by giving every token that reads the carried state (`t < width`) to block 0, which is also the only block that reads or writes `conv_state`. Shapes the shader does not serve decline to the reference tier through `GetOpFallback`: K = 1, K - 1 > 8, `conv_state` rows wider than K - 1, storage dtypes outside f32/f16/bf16, flags neither i8 nor i32, a grid past `maxComputeWorkGroupCount[0]`, or any index past the shader's uint32 arithmetic. `VT_VULKAN_CONV_FWD=0` keeps the op on the reference tier for a same-binary A/B. Both knobs are documented in `docs/ENVIRONMENT.md`; the spec gains §6.0b. ## Evidence New cases in `tests/vt/test_vulkan_backend.cpp`: - `the prefill causal conv1d runs NATIVELY on Vulkan, in both token mappings`: default and a 5-block split on lengths 5, 1, 9, against the CPU oracle; the specialization value (`|1`, `|5`) is asserted from `PipelineKeysFor`. - `the prefill causal conv1d matches the CPU oracle on its edge shapes`: i8 flags at byte offsets 0-3, no bias, silu off, bf16 and f16 operands with bf16 or f32 output, a zero-length sequence, a length-1 sequence with initial state, a padded `x` row stride. - `the prefill causal conv1d keeps a bf16 conv_state IN PLACE, bit-exact vs the f32 arm`. - `the prefill causal conv1d DECLINES a width past its window, and stays correct` and `... DECLINES a conv_state row wider than K-1`: decline counted, bytes equal to the reference. Outputs to the GDN NMSE tolerance; rolled state bit-exact, and spelled out independently of the oracle for the main case. Red-before / green-after on llvmpipe (LLVM 20.1.2, 256 bits), the device class this repository's Vulkan CI uses; five runs per arm of `-tc='*prefill causal conv1d*,*cooperative-matrix capability*'`: ``` this PR 5/5 6 passed 798/798 assertions pre-fix behaviour (every block copies the state row, plain blk*tb split) 5/5 1 failed 790/798 assertions CHECK( nmse <= kGdnNmseTol ) 0.0140122 <= 0.0005 CHECK( memcmp(split, default) == 0 ) 115 == 0 ``` Production call site removed (the `RegisterOp` disabled): 5 of the 6 cases fail (`RanNative`, decline counters). Discrete NVIDIA RTX PRO 6000 (Windows, MSVC 14.44.35207, `/W4 /WX`, 0 warnings): all new cases pass. `test_vulkan_backend` 47/51; the four failures are on main and unchanged by this PR (three TQ2 NMSE cases, the `vt_matmul_tiled` pipeline check), same values with and without it. ``` scripts/gen-vulkan-spirv.py --check committed SPIR-V is up to date scripts/check-agent-record.py agent record OK scripts/check-env-doc.py no finding from this PR ``` - [ ] `scripts/agent-preflight.sh` passes: not run in full, see Honest gaps. - [x] tests that cover this change: listed above. - [x] public docs changed only for facts this PR owns: two rows in `docs/ENVIRONMENT.md`. ## Speed claims - [x] This PR makes NO speed claim. ## Honest gaps - The race test is a red-before on llvmpipe only. On the discrete NVIDIA card the pre-fix behaviour PASSES, most likely because the blocks of one (sequence, channel) are adjacent invocations that run together there (not separately demonstrated). It also stops failing on llvmpipe if the split uses 4 blocks instead of 5, which is why the test uses 5 and the block count is capped rather than rounded to a power of two. - The split and the decision to leave it off by default were looked at on one discrete NVIDIA card only. No other device informed them. - The shader's `query_start_loc` guard bounds memory but is not validation: a malformed table whose intervals overlap gives unspecified output here, where the CPU reference throws. Device-side all-or-nothing validation would need a second dispatch and is not attempted. - Observed, not changed: the CPU reference addresses `conv_state` rows with stride K - 1, while `CheckConvCommon` admits wider rows (speculative-decode taps) and describes them as using the physical stride. This kernel declines those shapes, so it reproduces the reference exactly, including that addressing. - `VT_VULKAN_CONV_FWD=0` is read once at registration and has no unit test. - `scripts/agent-preflight.sh` was not run in full: on Windows one of its tests shells out to WSL and hangs. The platform-neutral checkers it calls were run individually (above); CI is the full run. - Built and run on Windows with local patches that are NOT in this diff: the tev1 registration's cross-TU static-init order, and three MSVC C4244 narrowings in existing lines (`test_tev1.cpp`, two in `vulkan_ops.cpp`). The Linux llvmpipe build (GCC 13.3) used the same patched tree; whether it needs them was not tested. - `check-env-doc.py` fails on main for three existing variables (`VT_VK_DISABLE`, `VT_VK_DISABLE_PAGED_ATTN`, `VT_VK_FENCE_TIMEOUT_MS`); not touched here. - `vulkan_spirv.h` records glslang 16.5.0 (the version `gen-vulkan-spirv.py` names as CI's pin) where main recorded 16.4.0. `--check` with 16.5.0 matches main's blobs, so the existing 43 modules are byte-identical; only that line and the new module change. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5-5 [claude-code]
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Row
BACKEND-VULKAN, branchrow/BACKEND-VULKAN-CONVFWD; the same sub-row namingas
row/BACKEND-VULKAN-GDN(#145) androw/BACKEND-VULKAN-GDN-CORE(#152).Before starting
kCausalConv1dFwdon Vulkan. Local issue filed and closed in this PR:.agents/issues/BACKEND-VULKAN/ISSUE-LOCAL-01M3V01R90QQTQ08BWSBM7AGN2.md(Kind: gap).
docs; helper PR.
.agents/backend-matrix.mdBACKEND-VULKAN; VK-G in.agents/specs/vulkan-full-support.md.scripts/ready-for-helper.py --check-local: structure OK, live claims not verified locally.comment in
src/vt/vulkan/vulkan_ops.cppthat left this op "for a follow-uprather than guessed at here" and named the two safe dispatch shapes;
src/vt/cpu/cpu_ops.cppCausalConv1dFwdKernel;src/vt/ops.cppCausalConv1dFwd/CheckConvCommon; the decode-update and compressed-statecases in
tests/vt/test_vulkan_backend.cpp; §6.0a of the spec.What changed
A native Vulkan kernel for
vt::CausalConv1dFwd, the GDN PREFILL depthwisecausal conv, which ran on the portable CPU reference tier once per GDN layer.
The default mapping is the first shape the existing comment named: one
invocation per (sequence, channel), serial over its tokens, with the old state
window copied to a private array before the write-back, as the CPU kernel's
old_rowdoes. An opt-in token split (VT_VULKAN_CONV_TARGET_GROUPS) stays inone dispatch by giving every token that reads the carried state (
t < width)to block 0, which is also the only block that reads or writes
conv_state.Shapes the shader does not serve decline to the reference tier through
GetOpFallback: K = 1, K - 1 > 8,conv_staterows wider than K - 1, storagedtypes outside f32/f16/bf16, flags neither i8 nor i32, a grid past
maxComputeWorkGroupCount[0], or any index past the shader's uint32arithmetic.
VT_VULKAN_CONV_FWD=0keeps the op on the reference tier for asame-binary A/B. Both knobs are documented in
docs/ENVIRONMENT.md; the specgains §6.0b.
Evidence
New cases in
tests/vt/test_vulkan_backend.cpp:the prefill causal conv1d runs NATIVELY on Vulkan, in both token mappings:default and a 5-block split on lengths 5, 1, 9, against the CPU oracle; the
specialization value (
|1,|5) is asserted fromPipelineKeysFor.the prefill causal conv1d matches the CPU oracle on its edge shapes: i8flags at byte offsets 0-3, no bias, silu off, bf16 and f16 operands with bf16
or f32 output, a zero-length sequence, a length-1 sequence with initial state,
a padded
xrow stride.the prefill causal conv1d keeps a bf16 conv_state IN PLACE, bit-exact vs the f32 arm.the prefill causal conv1d DECLINES a width past its window, and stays correctand... DECLINES a conv_state row wider than K-1: decline counted,bytes equal to the reference.
Outputs to the GDN NMSE tolerance; rolled state bit-exact, and spelled out
independently of the oracle for the main case.
Red-before / green-after on llvmpipe (LLVM 20.1.2, 256 bits), the device class
this repository's Vulkan CI uses; five runs per arm of
-tc='*prefill causal conv1d*,*cooperative-matrix capability*':Production call site removed (the
RegisterOpdisabled): 5 of the 6 cases fail(
RanNative, decline counters).Discrete NVIDIA RTX PRO 6000 (Windows, MSVC 14.44.35207,
/W4 /WX, 0warnings): all new cases pass.
test_vulkan_backend47/51; the four failuresare on main and unchanged by this PR (three TQ2 NMSE cases, the
vt_matmul_tiledpipeline check), same values with and without it.scripts/agent-preflight.shpasses: not run in full, see Honest gaps.docs/ENVIRONMENT.md.Speed claims
Honest gaps
the pre-fix behaviour PASSES, most likely because the blocks of one
(sequence, channel) are adjacent invocations that run together there (not
separately demonstrated). It also stops failing on llvmpipe if the split uses
4 blocks instead of 5, which is why the test uses 5 and the block count is
capped rather than rounded to a power of two.
discrete NVIDIA card only. No other device informed them.
query_start_locguard bounds memory but is not validation: amalformed table whose intervals overlap gives unspecified output here, where
the CPU reference throws. Device-side all-or-nothing validation would need a
second dispatch and is not attempted.
conv_staterows withstride K - 1, while
CheckConvCommonadmits wider rows (speculative-decodetaps) and describes them as using the physical stride. This kernel declines
those shapes, so it reproduces the reference exactly, including that
addressing.
VT_VULKAN_CONV_FWD=0is read once at registration and has no unit test.scripts/agent-preflight.shwas not run in full: on Windows one of its testsshells out to WSL and hangs. The platform-neutral checkers it calls were run
individually (above); CI is the full run.
tev1 registration's cross-TU static-init order, and three MSVC C4244
narrowings in existing lines (
test_tev1.cpp, two invulkan_ops.cpp). TheLinux llvmpipe build (GCC 13.3) used the same patched tree; whether it needs
them was not tested.
check-env-doc.pyfails on main for three existing variables(
VT_VK_DISABLE,VT_VK_DISABLE_PAGED_ATTN,VT_VK_FENCE_TIMEOUT_MS); nottouched here.
vulkan_spirv.hrecords glslang 16.5.0 (the versiongen-vulkan-spirv.pynames as CI's pin) where main recorded 16.4.0.
--checkwith 16.5.0 matchesmain's blobs, so the existing 43 modules are byte-identical; only that line
and the new module change.
FOLLOWING_AGENTS_PROTOCOL
Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5-5 [claude-code]