Skip to content

Make recorded terminal demos reproducible #3049

Description

@AlexSkrypnyk

Summary

Regenerating the recorded terminal demos rewrites the committed artefacts even when nothing about the recording changed. A real rendering change is therefore indistinguishable from the recorder's own jitter, and any two branches that both regenerate collide on the same files.

The same defect was found and fixed in AlexSkrypnyk/prompty, where regeneration is now reproducible: recording the same session twice produces the same bytes. This issue proposes the same approach here.

Details

.vortex/docs/.utils/update-videos.php drives VideoRecorder.php to record each demo with asciinema and an expect script, then renders it, writing .vortex/docs/static/img/<name>.{json,svg,png,gif}. The committed .json cast, the SVG, and the GIF all carry the recording's timing, so all three are rewritten on every regeneration.

Three properties of the recorder make its output vary between runs that record identical sessions. All three are present here:

  1. Typing is humanised. The expect scripts set send_human and type with send -h, which varies its inter-character delays at random by design. That alone changes where the terminal splits its writes.
  2. Frames follow the terminal's chunking. A frame boundary lands wherever the terminal happened to split a write, which depends on the scheduler rather than on the session.
  3. Timestamps are wall-clock. Nothing rewrites the recorded gaps before rendering, so the machine's timing reaches the artefacts directly.

With six demos recorded on one machine, load makes the third worse: a stalled session shifts its gaps, and the stalls land differently every run.

How to confirm

Regenerate twice with no source change between the runs, and compare:

php .vortex/docs/.utils/update-videos.php
git stash
php .vortex/docs/.utils/update-videos.php
git diff --stat

Anything other than an empty diff is the recorder's own noise.

Worth knowing before reaching for a narrower fix: a checker that ignores the timing attributes and compares the rest does not work. It was prototyped in prompty and still reported 12 of 28 assets as differing, because the runs captured different numbers of frames - one filmstrip was 1840px wide, the next 1680px.

Suggested direction

Four changes made regeneration reproducible in prompty, and they carry over here:

  • Type at a fixed rate rather than with send -h, so the session drives the terminal identically every time.
  • Treat a recording's output as one stream and cut it into frames where the session redrew - the cursor-up sequence a terminal application emits when it repaints - so the terminal's chunking can no longer invent a frame.
  • Rewrite every gap to one of two canonical durations before rendering: one for the repaints within a step, one between steps. The wall clock then never reaches the artefacts, and playback pacing becomes a deliberate choice rather than a recording of how slow the machine was that day.
  • Bound how many recordings run at once. That load is what stalls a session enough to blur a deliberate pause against the gaps inside a step, which is what the classification above depends on.

With those in place, git status after a regeneration becomes the check: a clean tree means the rendering did not change, and any diff is a real change worth reading.

Evidence from prompty: before the change, two back-to-back regenerations differed in 12 of 28 assets; after it, four consecutive regenerations produced byte-identical output for all 28.

Two notes specific to this setup:

  • The demos here record real command runs rather than a scripted prompt library, so some output is genuinely variable - timestamps, durations, container ids, paths. Those need masking for the artefacts to settle, and the exercise is worth doing on its own: anything that changes between two runs of the same command is also something a reader cannot rely on.
  • The .gif and .png outputs go through a separate rendering path, so they need checking against the same test rather than assuming the cast fix settles them.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    A1Board worker 1

    Type

    No type

    Projects

    • Status
      Released in 1.41.0

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions