Skip to content

Make packaging more solid#557

Merged
freezy merged 55 commits into
masterfrom
dev
Jun 26, 2026
Merged

Make packaging more solid#557
freezy merged 55 commits into
masterfrom
dev

Conversation

@freezy

@freezy freezy commented Jun 25, 2026

Copy link
Copy Markdown
Owner

This PR expands the VPE packaging pipeline into a more complete GLB-based .vpe format with runtime loading support, cooked texture payloads, richer material/light metadata, screenshots, and table metadata.

It also adds documentation for the new packaging flow, improves runtime/editor diagnostics, and tightens input handling around paused or unfocused player states.

  • Added a formal .vpe package structure with manifest data, stable node IDs, metadata, screenshots, and sidecar payloads.
  • Added async package export/import flows with progress reporting and cancellation support.
  • Added runtime package loading, including GLB image replacement, material resolution, cooked texture loading, and load progress reporting.
  • Added cooked texture support, including source texture preservation, optional zstd compression, BC7 GPU texture encoding, normal-map repacking, and cache handling.
  • Expanded material export/import coverage for HDRP-oriented table materials, including rubber, metal, DMD, fabric, silk, and additional material hints.
  • Added screenshot generation for table, cabinet, and backbox views, including light-state handling during capture.
  • Added backbox/cabinet marker components and additional packables for lights, sounds, primitives, slingshots, teleporters, and simulation state.
  • Added runtime graphics/SRP bridge helpers and improved SRP-compatible light shadow access.
  • Improved native input behavior by dropping input while the app is paused or unfocused.
  • Added developer documentation for packaging format, export/import flow, materials, shader variants, and benchmarks.

freezy added 30 commits April 19, 2026 15:53
Mapping-resolved lamps miss ~25% of LightComponents, so enumerate all
of them and identify flashers via the mapping plus the "F_" name
fallback. Suppress bulbs structurally by deactivating GameObjects
under each Light source transform, which survives HDRP render-graph
state caching across offscreen render sessions; restore active state
and clear property blocks on Restore. Let LightComponent toggle
faux-bulb GameObjects directly in edit mode and lazily re-collect
them when the cache is empty, so the LampManager edit-mode buttons
keep working without the runtime path.
The `Enabled` setter used the runtime fade value `_value`, which is only
initialized in `Awake()` and therefore always 0 in edit mode. Toggling
"on" was a no-op visually and zeroed the emissive via the property
block, leaving the bulb dark until the next off→on cycle.

Use 1.0/0.0 for the editor toggle and lazily collect faux bulbs in
`SetMaterialIntensity` so a single "on" restores emission without
requiring an off→on cycle.
Replace the bespoke forceRenderingOff + MaterialPropertyBlock + Source
GameObject deactivation with the same mechanism LampManager uses: flip
Light.enabled, then set LightComponent.Enabled, which hides the faux-bulb
meshes via SetActive in edit mode. Also cover LightGroupComponent
sub-lamps that have no LightComponent of their own (e.g. L51's skull
eyes) by collecting their orphan lights and emissive meshes separately.
Drop the temp diagnostic log.
Each shot toggles bulb/lamp state, but HDRP only reconciles culling
and GPU-resident-drawer instance data on PlayerLoop ticks - not
within a synchronous SubmitRenderRequest loop. Rendering in the same
frame the state was applied captured the previously settled state,
causing bulbs to leak between "on" and "off" shots.

Drive the capture via EditorApplication.update so we can yield a few
frames between applying a shot's lamp state and rendering it. Generate
is now asynchronous and delivers its result through an onComplete
callback (with an onError callback for failures).
freezy added 24 commits May 29, 2026 16:08
Add CabinetComponent and BackboxComponent as authoring-only markers so
tooling can locate the cabinet and backbox. The screenshot generator
hides any GameObject carrying these markers (saving and restoring its
active state) to keep them out of the top-down playfield shots.

The package writer now activates marker-tagged objects for the
active-only export and restores them afterwards, while skipping the
marker components themselves from serialization. Generated screenshots
are encoded to webp via NetVips and stored under a top-level
"screenshots/" folder in the .vpe; the source folder is passed in by
the table inspector (Assets/Screenshots).
Write a `table-bounds.json` describing the table's pixel rect within
the screenshots (shared across all shots) and package it alongside
the webp files, so the player can crop screenshots to the table and
drop the surrounding background.
- Add Abbreviation and BackglassImage fields to TableMetadata; the
  backglass texture is excluded from JSON and packaged separately.
- Encode generated screenshots as jpg (q=90) instead of webp and
  write the user-supplied backglass to screenshots/backglass.jpg.
- Surface Abbreviation in VpeTableMetadataSummary.
Cook every captured texture into its final GPU format at export (BC7
for color/mask data, DXT5 in HDRP AG packing for normals, mips baked)
and store it raw in `meta/textures.bin`. The GLB carries no images
for captured materials, and runtime upload is a single
`LoadRawTextureData` through a pinned pointer — no PNG decode, no
runtime `Texture2D.Compress`, no normal repack. On Terminator 2 this
takes in-editor import from ~23.3s to ~1.7-2.2s.

Supporting runtime changes:

- Introduce `PackageCompression` and write `table.glb`,
  `colliders.glb`, and `textures.bin` as stored zip entries; deflate
  on block-compressed and already-packed glTF data is pure load-time
  waste.
- Skip the runtime-compression toggle for cooked payloads and for
  non-RGB-packed normals (they don't go through the repack path).
- Cache the `PackagedRefs` PackAsAttribute type scan per domain (was
  ~450ms per load) and expose `WarmUpTypeScan` for worker-thread
  priming.
- Prefetch `materials.v1.json` (parsed off-thread), `textures.bin`,
  and all item/ref entries on worker threads with their own storage
  instances; SharpZipLib readers aren't thread-safe to share.
- Scan the GLB for the embedded-materials extension and missing
  tangents on a worker, overlapped with `gltf.Load`.
- Import the GLB with an uninterrupted defer agent (the loader
  already shows a loading screen).
- Yield by ~25ms time budget instead of every 16 items while
  restoring packables, and cap GPU upload work at ~160MB per frame to
  avoid `DXGI_ERROR_DEVICE_HUNG` on D3D12.

Reader keeps the legacy PNG sidecar path as a fallback for older
packages and for materials with unsupported shaders. Updates the
Packaging README and the benchmarks doc with the cooked-format
section and measured numbers.
The previous format shipped pre-cooked BC7/DXT5 mip chains in
`meta/textures.bin`, which made packages engine-coupled, balloon to
689 MB and lose authoring fidelity on editor re-import. The new format
ships the original PNG/JPEG asset bytes, and the player derives the
GPU-ready payload locally on first load.

- `VpeTextureCook` decodes sources in parallel (StbImageSharp on
  workers, native `LoadImage` for huge files), repacks normals into
  HDRP's AG layout on the GPU via `VpeCookNormalRepack.shader`,
  generates mips, and BC7-encodes every mip with the DirectXTex
  compute shaders bundled in `Resources/VpeBc7Encode.compute`.
- `VpeTextureCache` persists the cook output under
  `persistentDataPath/TextureCache`, keyed by package size, write time
  and `VpeTextureCookSettings` (resolution divisor for low-spec).
- `RuntimePackageReader` uses the cache when available and falls back
  to the source PNG decode path for platforms without compute support
  or uncaptured materials.
- `PackageReader.ImportMaterials` writes the source bytes back as
  texture assets (normal-map flag, sRGB, wrap/filter/aniso, max-size
  restored from the payload) and delegates material reconstruction to
  the registered `IVpeMaterialV1EditorImporter`, restoring masks,
  thickness, transmission and refraction state that the GLB-based
  import previously dropped.
- `VpeTextureImportPreprocessor` configures the re-imported texture
  assets so HDRP picks them up correctly.

Terminator 2 benchmark (editor play mode): 477 MB package (lossless),
~9.5 s first load including the cook, ~1.1 s on every load after.
Add an editor `ImportMaterialsOnly` entry point that rebuilds materials
and textures from the v1 payload without restoring items, references,
globals or collider meshes, for iterating on the material/texture path.

Rebalance the runtime load-stage progress ranges so the texture cook
(which reports fine-grained progress inside `RestoringMaterials`) and
the scene import dominate the bar, matching the post-cook load profile.

Expose `VpeTextureCookSettings.CompressTextures` to toggle BC7
compression at cook time and fold it into the cache hash so cooked
outputs are invalidated when the setting changes.
Adds a normative FORMAT.md spec alongside the implementation README,
introduces a root manifest.json (format/schema versions, root node id,
component types) and replaces ad-hoc bindings with stable per-node
glTF `extras.vpeId` ids referenced by items, refs, mappings and
material/light payloads.

Drops the V1 suffix from material and light schemas (now versioned via
the manifest), adds a GLB image-swap step to preserve lossless source
bytes through glTFast export, and refactors the package reader/writer,
PackagedFiles/PackagedRefs and mapping payloads onto the new id scheme.
The GPU repack path read _MainTex through an sRGB-decoding SRV
which gamma-distorted normals, and refreshing the compiled shader
variant from a package Resources shader was unreliable. Both
worker- and main-thread decode paths now AG-pack on the CPU, so
the cook blit is always a plain copy and VpeCookNormalRepack can
be removed.
Adds VpeNormalRepackCompute and a small RepackNormal compute kernel
that reads the source through a plain SRV (no sRGB decode) and writes
the (1, y, 1, x) AG layout directly into mip 0 of the cook target.
Falls back to the existing CPU repack when compute is unsupported or
the shader fails to load.
Wraps the cooked payload in independent ~16 MB zstd frames so both
write and load parallelize across cores, keeping managed zstd's
per-frame cost negligible. Compression is opt-in via
CompressPayloadOnWrite and the v3 header carries a flag so existing
caches keep loading without a re-cook.
Run the .vpe export off the main thread: yield between stages,
load texture bytes (and 16-bit PNG downconvert via libvips) in
parallel on worker threads, and surface a cancelable editor
progress bar. Defer texture blob materialization until after
material capture so the heavy disk + decode work no longer
freezes Unity. Pre-warm libvips and serialize its managed API
calls to avoid a cold GObject init race that hard-crashed Unity.
libvips operations are thread-safe once vips_init has completed; only
the lazy cold init races. With EnsureVipsInitialized already called
on the main thread before LoadAll fans out, the per-call VipsLock
around NewFromBuffer/Cast/PngsaveBuffer was redundant and serialized
the 16-bit PNG downconvert path for no reason.

Drop the lock, tighten the surrounding comments, and move
EnsureVipsInitialized below LoadAll so the precondition reads in
order.
Read cropWidth/cropHeight from screenshots/table-bounds.json during
metadata extraction and surface it as ScreenshotAspectRatio so the
player can lay out its table carousel without decoding the screenshot
image.
Expose ConfiguredBindings as a static override and BuildDefaultBindings
as a public static helper so the player can supply custom key bindings
to the native input layer. StartPolling now sends the host-supplied
bindings when set, falling back to the built-in defaults otherwise.
Add a comment in VpeNodeIds documenting that glTF 2.1's native per-object
uid field is the natural successor to the current extras.vpeId scheme,
along with why it can't be adopted yet (spec still provisional, glTFast
6.19.0 is glTF 2.0-only).
Prevents held/pressed keys from accumulating in the ring buffer and
flushing into the game on resume.
Adds an AppFocused flag on NativeInputManager that the host sets each
frame from Application.isFocused. Native input events are discarded
while the window lacks focus, so background key presses in other apps
don't reach the game.
Introduces VpeGraphics, a host-facing API that lets the player app push
plain-data render-quality settings (camera AA + Volume effects) to the
active render pipeline without referencing pipeline types. The SRP
assembly registers an IVpeGraphicsApplier via RuntimeInitializeOnLoad,
mirroring the VpeMaterialResolver pattern.

Also exposes SimulationThreadComponent.GetCurrentSnapshot() so a
performance overlay can read the latest timing/latency snapshot from
the main thread.
The `Light.shadowRadius` and `Light.shadowAngle` properties aren't
available on all Unity versions / render pipelines. Route reads and
writes through a reflection helper with safe fallbacks so packing
keeps working when the properties are missing.
Introduces the `vpe.fabric.silk` material type with portable `Lit`
fallback and HDRP fabric controls (thread map, fuzz, UV scales).
Extends `Lit.Hdrp` hints with specular occlusion/color, clear-coat
mask, transparent preserve-specular blending, and Z-test overrides.
Adds a `RuntimeGltfMaterialGenerator` factory so the runtime package
reader can plug in an SRP-specific glTFast material generator.

display: harden material creation in `DisplayComponent.RegenerateMesh`
by validating the shared material before reuse and throwing a clear
error (with the component path) when `CreateMaterial` fails.

lamp: log mapping summary and sample lamp events in `LampPlayer` to
help diagnose unmapped or misrouted lamp IDs at runtime.
Wrap the glTFast ConsoleLogger with GltfShaderMissingFilterLogger so
the ShaderMissing errors for the intentionally stripped glTF-pbr
shadergraphs are dropped. The HDRP material resolver replaces every
imported glTF material at runtime, so the fallback shadergraph is
never used and its absence is not an error. All other log messages
pass through unchanged.
Add a packaging guide explaining why runtime-resolved HDRP materials
need a preloaded ShaderVariantCollection in standalone builds, and how
to refresh the capture when adding new tables. Cross-link the new page
from the packaging index, import, and materials docs.
Reflect the move from packed `textures.bin` to per-file PNG/JPEG
sources under `table/textures/`, the rename of `materials.v1.json`
to `materials.json` (versioned by `FormatVersion`), and the dropping
of the `V1` suffix from the material API types.

Document the player-side texture cook plus per-table cache as the
shipping load path: GPU mip/BC7 encode and dxt5nm normal repack
during the cook, raw upload on cached loads, with the non-cooking
fallback still honoring `GenerateMipMaps` and `RuntimeCompress`.

Capture the new material types (`vpe.dmd`, `vpe.fabric.silk`),
revise texture ownership so all captured-material textures move out
of the GLB, mark `textures.bin` and the resolver's GPU normal repack
as superseded in the benchmarks, and replace the "next targets"
section with a status note since the load-time work is done.
Invisible collider primitives (renderer disabled) were omitted from
table.glb, leaving them mesh-less at runtime and producing no
colliders. Route their meshes through colliders.glb via IColliderMesh
and re-attach a MeshFilter on unpack, while visible primitives keep
reusing the table.glb mesh to avoid duplication.
@freezy freezy self-assigned this Jun 25, 2026
@greptile-apps

greptile-apps Bot commented Jun 25, 2026

Copy link
Copy Markdown

Too many files changed for review. (175 files found, 100 file limit)

Bump the test project's editor version from 2022.3.11f1 to
6000.5.0f1 and update package dependencies accordingly
(ai.navigation, test-framework, ugui).

Migrate code to the new APIs:
- Adopt the generic UnityEditor.IMGUI.Controls.TreeView<int> in the
  editor tree view and manager UI.
- Add UnityObjectId helper that maps EntityId to a stable int on
  6000.5+ and falls back to GetInstanceID on older versions, then
  route all component id lookups through it.
- Update the NLog UnityTarget to the current Target base class API.
@freezy freezy merged commit ba5746b into master Jun 26, 2026
14 checks passed
@freezy freezy deleted the dev branch June 26, 2026 06:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant