Skip to content

Build the tool from the checkout when no install provides it - #33

Merged
dcadolph merged 3 commits into
mainfrom
feat/checkout-build
Nov 16, 2026
Merged

dcadolph merged 3 commits into
mainfrom
feat/checkout-build

Conversation

@dcadolph

@dcadolph dcadolph commented Nov 16, 2026 •

Copy link
Copy Markdown
Owner

When no documented install gave the example session a binary, kibble skipped every example in the repository. Fifteen of the 97 pinned clones lost all their examples this way. This change builds the tool from the checkout instead, so one missing install route no longer costs every example, and it adds the rules that keep the newly reached lines honest.

Checkout build

When every install route is skipped, refused, or yields no binary, kibble builds the tool with its language's standard build and puts it on PATH. It builds only a tool it can name. Go uses go install of the root or of the cmd/ directory named for the repository or the only one there. Rust uses cargo install --path . of the one binary or the one named for the repository, with --locked when a lock file exists. Python uses pip install . for the one console script or the one named for it, into a virtual environment of its own with the script linked onto PATH. A checkout that offers no such name builds nothing.

Every example result says it was built from the checkout and the documented install was not tested. The install rows keep their own verdicts. A checkout that does not build reads BLOCKED with its error line or a timeout, since the failure belongs to kibble's stand-in and not to the document.

What the session receives

The session tree leaves out directories by name, such as dist and build, and the name is only a guess. A directory that holds files git tracks is now streamed whatever it is called, which goreleaser and k6 need to compile. A session that builds from the checkout also takes every tracked file whatever its size, so a vendored table over the usual 2 MB limit is not dropped. If that pushes the tree past the stream cap, the size limit comes back. A checkout without a git index falls back to the names alone.

Reading the lines it now reaches

  • A line that fails after an earlier line of the same document named its input and was skipped or blocked reads as the same hole, not a new failure. Names are compared as paths, and a redirect target counts as naming the file.
  • A command in a block the plan left out, such as a make test that tells the reader to run a development shell the README shows, reads as depending on that block.
  • A scaffold line the checkout stopped, such as goreleaser init refusing because the repository's own configuration is there, taints the same tool's later lines in that document. Their failures, and any nonzero exit the documents bless, read checkout-state. A usage error or a crash still stands.
  • A flag that sets the exit code for a finding, such as --exit-code 1, asks for that exit and is read as documented behavior.
  • A crash is read only from lines the tool printed bare. A crash word inside a table row or a quoted block is the tool reporting something it read.
  • A request whose URL host did not resolve, where an earlier line of the same document named that host and did not pass, reads as depending on that line. A Kubernetes guide's curl http://nginx after its skipped kubectl lines is the cluster missing, not the guide. A dead domain no such line names still fails.
  • Each collapsed <details> element is a section of its own. A README that offers a template setup and a manual setup side by side under one heading ran both in one shell, so the manual setup met the template's directory. It now reads inherited-state, as a later heading already did.

Environment changes

The session asks each documented binary for --version once at the start. After a line that ran one of them exits nonzero, it asks again, only for a binary that passed the first time. A crash followed by a failed probe reads BLOCKED with the new reason environment-changed: something the session ran changed the environment the tool runs from, so the crash says nothing about the line. A crash the tool survives still reads FAIL, and a plain nonzero exit is never read this way.

Verification

  • Every new rule has a paired case that must not match, and each rule switched off fails its test. The probe test runs the generated shell in bash against a stub tool that breaks on cue.
  • Fifteen of the 97 pinned clones gain a checkout build, and every binary pick is right. None of the six corpus repositories uses it.
  • All fifteen of those repositories, which tested nothing in the last sweep, were run in Docker. Thirteen now test something. The other two read BLOCKED for the right reasons: one needs generated documentation the repository does not commit, and one build timed out.
  • Three FAIL lines remain across the fifteen, all in one repository, and each was read by hand.
  • The corpus pins, the recall corpus, and the two known failures in the earlier sweeps hold. Run locally on arm64, fd reads one pass short of its pin on both this branch and main, because its npm binary does not start on that platform. Plans over the 30 clones in the run are unchanged apart from the new details field.
  • The silent-repository count from the sweep script matches kibble's exit 3 on the same runs.
  • Build, vet, race tests, and lint are clean.

The jq recall row flips to caught in its own commit. The full reconciliation lane over the corpus and the earlier sweeps runs before merge.

When every documented install is skipped or yields no binary, kibble now
builds the tool from the checkout with its language's standard build and
runs the examples against it. A failed build reads BLOCKED. Crashes that
come from the session's own environment read BLOCKED instead of FAIL.
Each collapsed details element is now a section of its own, so a setup
that runs in a directory a sibling setup entered reads inherited-state. A
URL host that did not resolve, named by an earlier line that did not pass,
reads depends-on-skipped.
@dcadolph
dcadolph merged commit 135971f into main Nov 16, 2026
3 checks passed
@dcadolph
dcadolph deleted the feat/checkout-build branch November 16, 2026 23:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant