Repository navigation
mesh export: cut define-phase cost and memory on fragment-heavy meshes - #1118
Merged
Merged
Conversation
Three changes, measured on HALL_BENCH (46 objects, 3589 submeshes): export wall 22.1s -> 12.7s, peak RSS 12.6GB -> 4.7GB, define phase 12.6s -> 1.2s. 1. Bake the object transform into the welded points and pre-rotate the loop normals in numpy during the parallel compute phase, instead of passing 'transformation' to DefineMeshExt. Previously the C++ side deep-copied + rotated the normals of EVERY submesh sharing the point buffer in the serial define phase (~3.1ms per 157k-point submesh). The new 'normals_pretransformed' flag (LuxCore) skips that work. Negative-determinant (mirror) transforms keep the C++ path. 2. Compact each submesh to the welded points it actually uses (local point array + remapped triangles + compact attribute arrays) for objects without a subdivision shape. Subdivided objects keep the shared point buffer, which the C++ subdiv envelope needs. This removes the full-object-size per-submesh fallback attribute arrays (~2.2GB of numpy allocations on the 736-submesh mint object). 3. Replace the 16-object chunk pipeline with a per-object in-flight window (BLC_EXPORT_INFLIGHT, default 4), bounding the memory held by extracted-but-not-yet-defined meshes. Also: set crease_pairs only on the first submesh of an object (they are identical for all submeshes sharing the point buffer, and the subdiv envelope does not read per-submesh creases). Correctness: PLY dumps of all 736 mint submeshes are identical between the compact and forced-shared paths (triangles, normals, UVs); DanishMood raw film vs the 2.11.2 wheel matches at noise level with PhotonGI off (mean|d|=0.009) and at reference brightness with it on. Generated by Mistral Vibe. Co-Authored-By: Mistral Vibe <vibe@mistral.ai>
…t color arrays - Triangles are grouped by (material, island) with a single argsort + boundary split when there are many groups (previously one full scan of tri_group per group, O(groups x tris) on fragment-heavy meshes). - Corner color rgb/alpha arrays are made contiguous once per object and POINT-domain colors are indexed from the pre-welded arrays, instead of re-slicing per submesh. - BLC_TIME_STAGES=1 prints per-stage timings of compute_submeshes. Verified: PLY dumps of all 736 submeshes of the fragment-heavy mint object are triangle-identical between the compact and forced-shared paths. Generated by Mistral Vibe. Co-Authored-By: Mistral Vibe <vibe@mistral.ai>
…lementation compute_submeshes() now calls pyluxcore.compute_submeshes_c() when the loaded pyluxcore build provides it (the whole weld/islands/grouping/ gather pipeline runs in C++ with the GIL released, returning zero-copy numpy arrays), and falls back to the previous numpy implementation (_compute_submeshes_py) when the symbol is missing or BLC_COMPUTE=py. The Python orchestration (extract -> compute -> define pipeline, payload dict shape, info stats) is unchanged. Generated by Mistral Vibe. Co-Authored-By: Mistral Vibe <vibe@mistral.ai>
…E_LOCK pyluxcore now serializes scene mesh mutations internally (recursive mesh-def mutex in slg::Scene) and releases the GIL around the C++ calls, so DefineMeshExt/SetMeshTriangleAOV are safe to call from worker threads. convert_objects() now runs compute_submeshes() + define_submeshes() fused in the thread pool: the calling thread only does the Blender-API extraction and collects results, so the serial define phase in the main thread is gone. The in-flight window (BLC_EXPORT_INFLIGHT) now bounds extract+compute+define. _Scene_lock is removed; define_submeshes() no longer takes it. Generated by Mistral Vibe. Co-Authored-By: Mistral Vibe <vibe@mistral.ai>
…g one per occurrence Packed images used by many textures/materials produced one log line per reuse (e.g. 110 lines for the same file in DanishMood). The skips are now counted per image and reported once at the end of the export (and in cleanup), with the number of times each image was skipped. Generated by Mistral Vibe. Co-Authored-By: Mistral Vibe <vibe@mistral.ai>
The 'Exporting <mesh> - N submesh(es)' block was printed from the worker threads, so the lines of different meshes interleaved. Workers now buffer their mesh's log lines and the main thread prints them at collect time, in job order, so each mesh's block stays together. Generated by Mistral Vibe. Co-Authored-By: Mistral Vibe <vibe@mistral.ai>
The separator now shows the elapsed wall time since the start of the current export pass (e.g. '[BLC] (wall 3.456s)'), so the per-mesh blocks form a readable timeline. convert_objects resets the reference at each export pass; the serial convert() path sets it if unset. Generated by Mistral Vibe. Co-Authored-By: Mistral Vibe <vibe@mistral.ai>
Add log()/log_str() helpers in utils/log.py (wall time in seconds since the addon was loaded, same idea as the [SDL][t] timestamps of LuxCore) and route all [BLC] print() calls through them: luxloader, utils register/unregister, viewport view_update timings, image export (incl. the already-exported summary) and mesh export. In the parallel exporter, worker threads buffer their mesh's messages already formatted (via log_str) so the per-mesh blocks keep their timestamps while staying grouped when printed from the main thread. The buffer parameter was renamed to log_lines to not shadow the log() function. Generated by Mistral Vibe. Co-Authored-By: Mistral Vibe <vibe@mistral.ai>
LuxCore timestamps its [SDL] messages as WallClockTime() - lcInitTime, where lcInitTime is the wall-clock time of the luxcore::Init() call (src/luxcore/luxcoreinit.cpp). utils.log now exposes reset_t0(), called in register() right before pyluxcore.Init(), so [BLC][t] and [SDL][t] share the same reference and interleaved log lines line up (previously [BLC] used the addon-import time, drifting by the wheel-install time). Generated by Mistral Vibe. Co-Authored-By: Mistral Vibe <vibe@mistral.ai>
first_run() now logs 'Exporting meshes' when the mesh export begins and 'Mesh export finished in <t>s' (wall time, [BLC][t] format) when it completes, so the per-mesh blocks are framed by clear boundaries. Generated by Mistral Vibe. Co-Authored-By: Mistral Vibe <vibe@mistral.ai>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Measured on HALL_BENCH (46 objects, 3589 submeshes): export wall 22.1s -> 12.7s, peak RSS 12.6GB -> 4.7GB, define phase 12.6s -> 1.2s.
transformationtoDefineMeshExt. Previously the C++ side deep-copied + rotated the normals of every submesh sharing the point buffer in the serial define phase (~3.1ms per 157k-point submesh; 736 submeshes = ~7s + ~1.4GB). The newnormals_pretransformedflag (LuxCore PR Blender 3.1.2 linux (Ubuntu 22.10) crashes when entering material or render view #766) skips that work. Mirror transforms (det < 0) keep the C++ path.BLC_EXPORT_INFLIGHT, default 4) instead of 16-object chunks, bounding memory of extracted-but-undefined meshes.Also sets crease_pairs only on the first submesh of an object (identical for all submeshes; the subdiv envelope does not read per-submesh creases).
Correctness: PLY dumps of all 736 mint submeshes identical between compact and forced-shared paths (triangles/normals/UVs); DanishMood raw film vs the 2.11.2 PyPI wheel matches at noise level with PhotonGI off (mean|d|=0.009) and at reference brightness with it on.
Companion LuxCore change: PR LuxCoreRender/LuxCore#766 (
normals_pretransformedflag).