Repository navigation
Establish repeatable plot benchmarks and retain optimization evidence - #30
Merged
Merged
Conversation
Recover the completed benchmark baseline from the original local work. Source tree is byte-identical to original commit 88d459dbbad9cff1a292b3ffa87be97970d371e6, including the initial workload suite and unavailable-counter handling. Original commits are preserved in vnm_plot-benchmarks-2026-11-04.zip; this publication uses a new commit identity because direct Git push authentication was unavailable.
Recover the final 14-case suite, report and compact measurements without applying the experimental production optimization. Source tree is byte-identical to original foundation head 15eb736a897d147afa799470318312d913f06f13. Retained Linux QRhi Null measurements establish CPU preparation-path cost and allocation traffic only; the separately retained production experiment has an unresolved conservative-workload slowdown. Original history remains in vnm_plot-benchmarks-2026-11-04.zip.
…ty [skip ci] Preserve 80 fixed initial follow-up observations, including identical-binary controls. Conservative no-text is slower in all eight retained-binary pairs (+5.45% median), while its identical-binary control is +0.71% with mixed signs. Record the measured boundary defect and the native-display blocker before the bounded same-executable diagnostic. Production optimization remains separate.
Guard the remaining preparation lookup and finish scope, then store harness observations after closing frame timing and allocation accounting. Reject internal renderer scopes and observations in frame-only reports. Advance the workload contract to v3 so historical v2 timings are not silently compared with the corrected boundary. The existing Python validator now passes ten focused tests and rejects the retained report's unexpected renderer.frame.finish scope. Native C++ verification is continuing on this draft.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Merged: benchmark foundation and workload-v3 correction. Final continuation evidence: PR #31.
Why
The existing offscreen benchmark could miss settled layout reuse and rescan immutable fixture data, conflating fixture behavior with production work. Its frame-only mode also retained internal profiler bookkeeping inside the frame measurement. This PR establishes repeatable workloads and corrects the measurement boundary so later optimizations can be assessed against explicit controls.
Changes
Production files under
src/andinclude/are unchanged from the base. The production label-membership experiment and temporary same-executable diagnostic remain on separate branches.Evidence and decision
The text workloads repeatedly save 38 allocations and 896 requested bytes per frame. In the same-executable comparison, settled text changes from 190 to 152 allocations/frame; conservative text changes from 206 to 168. No-text allocation counts remain unchanged.
The retained two-binary conservative no-text comparison is slower in all eight balanced pairs, with a +5.45% median paired change. In the corrected same-executable diagnostic, the corresponding control is evenly split, with a −0.71% median. Settled-text mean time favors the candidate by −4.24% median, but variation overlaps the identical-mode control and the paired p95 median is +0.47%.
This confirms the allocation reduction without establishing a general timing benefit. Code placement remains a hypothesis for the retained-binary penalty: the same-executable experiment also changes the harness from v2 to v3, so the cause is not isolated. The production optimization remains outside this PR.
Validation and limits
Measurements use explicit QRhi Null. They cover CPU-preparation-path wall times and observed render-thread C++ allocation traffic. Hardware GPU throughput, application FPS, presentation latency and pixel equivalence remain unverified.
A bounded native display attempt failed because this environment cannot create the required AF_UNIX sockets.
FontRendererDrawRangesandSampleUploadReuseremain unrun here; Null is not substituted for those raster gates. Hosted native CI was not dispatched. This PR merged on 4 November 2026 at 01:53 UTC. The final report was completed after the merge and is proposed separately in PR #31.Source identity and retained artifacts
Publication used the connected GitHub API because shell Git push credentials were unavailable. API commit identifiers differ from local identifiers; corresponding Git tree IDs were verified equal.
1ecbeab5f1a4f876e9a30a8e9697990b226d2e3a88d459dbbad9cff1a292b3ffa87be97970d371e6a15293f39b52bbee54e129d59e6739e74a2970e615eb736a897d147afa799470318312d913f06f1371df3e83b56d80b856c2bad73faa231c1200fff0e9cf091df613709fc8013def2b47062cfbf97c06ca15d8cea331fb70a923bcb1a8c75ede3493882f043e0a61d6e0a4127399aa341c397249634e479194a3d43388f91ea5213c2057c3f5f20fad2091a241a0088fe1faf4a85b21638631e32b80501c6c700ba0a410e07efa11ec5620496c808f2bab02d2fc7600d01848f56da988246562bd91e307954b5255Diagnostic reproduction branch:
diagnostic/label-membership-same-image-20261104. It is not a proposed production configuration.The original
vnm_plot-benchmarks-2026-11-04.zippreserves original Git objects, patches, all 284 original measured processes and logs. The newvnm_plot-benchmark-followup-2026-11-04.zippreserves the 144 continuation processes, fixed plans, build/test logs, assembly evidence, independent review and checksums. Source repositories, SDKs and duplicated binaries are excluded from the new raw-evidence archive.