Skip to content

build: generate gl and hlsl shader variants with spirv-cross - #207

Open
CanReader wants to merge 6 commits into
mainfrom
feat/shader-cross-compile
Open

CanReader wants to merge 6 commits into
mainfrom
feat/shader-cross-compile

Conversation

@CanReader

Copy link
Copy Markdown
Owner

What this changes

Shaders can now be written once as Vulkan GLSL and the build generates the OpenGL GLSL and SM 5.0 HLSL from the SPIR-V. SPIRV-Cross is vendored as a submodule (vendors/SPIRV-Cross, pinned to vulkan-sdk-1.4.363.0) and its CLI is built as a host tool. sleak_cross_compile_shader() writes name.vert.glsl / name.frag.glsl and name.vert.hlsl / name.frag.hlsl into <build>/Engine/shaders, and they are staged next to the .spv with the same mechanism as #196. OpenGLShader now loads the generated name.<stage>.glsl when it exists and falls back to name_gl.<stage> per stage.

First migrations: debug_line, ibl_irradiance, ibl_prefilter, ibl_brdf_lut and the vertex stage of lighting_pass. Their hand-written _gl files are deleted.

Stacked on #196, review that one first.

Why

Every shader has up to four hand-written copies and they have drifted (default_shader is 543/467/186/134 lines across its variants). Most of the GL copies are the Vulkan file with layout(location) stripped and gl_VertexIndex renamed, and nothing checks them until a driver compiles them at runtime.

Only glslc is assumed on the machine (no dxc, slangc or spirv-cross), so SPIRV-Cross comes in as source and builds with the engine. It only needs SPIR-V as input, so on a machine without glslc it runs on the committed .spv and the GL shaders still get generated. That is what keeps Windows CI and Windows builds without the Vulkan SDK working after the _gl files are gone. When cross compiling (USE_MINGW) the vendored tool can not run, so configure looks for a host spirv-cross and fails with a message if there is none.

Some choices that are not obvious from the diff:

  • GL output is GLSL 4.00 with GL_ARB_shading_language_420pack. At 4.10 and up SPIRV-Cross puts layout(location) on varyings, and a generated stage with locations does not link against a hand-written stage without them (GL only matches by name when neither side has one). At 4.00 varyings match by name, so a generated lighting_pass.vert links with any lighting_pass_gl.frag, including SleakCraft's own. binding still comes through via 420pack.
  • Push constant blocks become a std140 uniform block with no binding (--glsl-emit-push-constant-as-ubo), which GL puts at binding 0. That is exactly where the GL backend already binds DebugLineUBO, FaceUBO and PrefilterUBO.
  • The GL binding of everything is the Vulkan binding with the set dropped. That is why generation is opt in through ENGINE_CROSS_SHADERS and not every shader: it is only correct when the Vulkan layout already matches what the GL backend binds. I checked each listed shader against its GL call site (OpenGLIBL.cpp, the debug line pass, ExecuteDeferredLightingPass).
  • debug_line.vert negates gl_Position.y for Vulkan, so it is listed under ENGINE_CROSS_SHADERS_FLIP_Y and SPIRV-Cross negates it back (--flip-vert-y). The other migrated vertex shaders do not flip.

The HLSL is generated but nothing loads it. The DX11 and DX12 loaders still read the hand-written name.hlsl / name_dx12.hlsl. The generated files use main as the entry point, TEXCOORDn input semantics and compiler assigned registers, while the DX backends expect VS_Main/PS_Main, POSITION/NORMAL/... semantics and their own register and root signature layout. Switching them needs per shader semantic and register mapping I can not run on Linux, so I left the DX loaders alone rather than ship something only CI has compiled. CI now compiles every generated HLSL file with fxc on Windows, so that side is at least known to be valid HLSL.

Not covered here:

  • lighting_pass.frag is not generated for GL. The GL and Vulkan lighting passes have different data contracts (GL reads LightUBO at binding 2 with up to 16 point and directional lights and tone maps in the pass, Vulkan reads a single directional light plus three fill lights from ShadowLightUBO and leaves tone mapping to bloom composite). Unifying that means changing what one backend feeds the shader, which is a behavior change, not a port. The GL fragment shader that is missing from the engine (OpenGL deferred rendering can never initialize: lighting_pass_gl.frag does not exist #144) is a separate PR on top of this one.
  • ssao, ssao_blur, gbuffer, skybox, default_shader, skinned_shader stay hand-written for the same reason: the GL and Vulkan versions bind different things (or the GL skybox tone maps and the Vulkan one does not), so generating GL from the Vulkan file would change what GL renders.
  • tonemap_gl.* is not loaded by anything (only DX11 uses tonemap.hlsl), it is just dead. Left for a cleanup PR.
  • Game shaders: the function is usable from a game after add_subdirectory(Engine), but SleakCraft's shaders are not wired to it here.
  • A game that overrides one of the migrated engine shaders now has to ship a file under the generated name, since the loader prefers name.<stage>.glsl over name_gl.<stage>.
  • Building SPIRV-Cross adds about 75 s of CPU time to a clean build at -j4. It is EXCLUDE_FROM_ALL and only the CLI and the libraries it needs get built.

How it was verified

  • Builds on Linux
  • Ran SleakCraft on Vulkan
  • Ran SleakCraft on OpenGL
  • Builds on Windows (left to Windows CI, including the new fxc step)

Clean SleakCraft debug build against this branch (-j4, Ninja). All nine generated pairs were written to build/Engine/shaders and staged into bin/assets/shaders, the spirv-cross binary and its static libs stay in the build tree, and the engine checkout had no changes besides mine. Every generated GL file also compiles with glslangValidator and with glslc --target-env=opengl, which is what the new Linux CI step runs.

Without glslc (-DGLSLC=): configure warns as in #196, and the generation runs on the committed .spv and produces byte identical GL files.

OpenGL on llvmpipe under xvfb, from SleakCraft's bin: ran until the 90 s timeout, created world citest, OpenGL GBuffer created ... deferred rendering active, OpenGLIBL: precompute complete, no errors. strace shows it opened debug_line.{vert,frag}.glsl, all six ibl_*.glsl and lighting_pass.vert.glsl, and the hand-written lighting_pass_gl.frag (SleakCraft's copy), so the mixed generated/hand-written program links on Mesa. I also took a 640x360 frame at 45 s with the generated shaders and again with the old hand-written _gl files put back in bin. The pictures are the same apart from the HUD numbers (FPS, vertex counts).

Vulkan on the real GPU under xvfb: ran until the 60 s timeout, loaded the world, no new validation output. The Vulkan .spv are not touched by this PR.

Checklist

  • Shader changes: no shader source changed, migrated shaders lose their _gl copy and get generated ones instead
  • Public headers documented with /// comments (no public header changed)
  • No new includes from src/ or include/private/ in public headers
  • No game-specific vocabulary added to engine API

SPIRV-Cross is pinned to vulkan-sdk-1.4.363.0 and built as a host tool. sleak_cross_compile_shader() turns a shader's SPIR-V into GL GLSL and SM 5.0 HLSL in the build tree, staged next to the .spv. Only the shaders listed in ENGINE_CROSS_SHADERS are generated.
@CanReader
CanReader changed the base branch from build/spirv-out-of-source to main October 8, 2026 08:25

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant