Skip to content

Hook cleanup waits for jobs and bounds failure details - #1090

Open
zlareb1 wants to merge 6 commits into
mainfrom
fix/hook-contract-sandbox-drain
Open

zlareb1 wants to merge 6 commits into
mainfrom
fix/hook-contract-sandbox-drain

Conversation

@zlareb1

@zlareb1 zlareb1 commented Nov 20, 2026 •

Copy link
Copy Markdown
Member

Related: #1089

Related: #995

Changelog category

  • CI Fix or Improvement

Changelog entry

Not required.

Why

Hook contract tests remove their sandbox after the immediate hook exits. A detached flusher can still be writing there. The fixture waits for its owned job files to disappear before removing the sandbox; the flusher removes each job after its cursor and recording writes.

A hosted Windows case still reached the separate cleanup deadline with an unfinished job. The failure named no remaining job, so it could not distinguish an empty dispatch file from a job containing a delivery. Cleanup now reports the pending count and at most eight generated job filenames and byte sizes. It reads no job contents. The Windows failure's cause remains unknown.

Scope and risk

The hook cleanup concern changes the contract test's six cleanup sites, a test helper and its controls. The composition also includes the two canonical capture-observation test files. A controlled detached writer demonstrates premature removal, and the real flusher runs behind a response barrier. An unfinished job fails cleanup after the unchanged 45-second teardown bound. Failure details have a fixed shape and retain no payload, token or session identifier.

The helper still refuses to remove unfinished jobs. There is no removal retry, disabled delivery or production hook change. All original hook cases, inputs, concurrency, output assertions and measured deadlines remain unchanged.

The current-main composition also consumes the private-query fixture's request-scoped capture observer. Unrelated public tasks keep their original sink while the request and its children retain the private-capture assertion. All original native-log and permission checks remain. The hosted failure reports a capture file but omits its contents and writer; it establishes no private-query leak or unique cause.

Compatibility, rollout and rollback

No wire, settings, plugin version, shipped hook or operator changes. Reverting changes only these test fixtures and controls.

Tests

On this composition, seven targeted Node cleanup controls and six capture-isolation controls pass. Scoped source, formatting, syntax, prose and whitespace checks pass. The two capture files are exact to the canonical fixture repair. Full new exact-head hosted CI and a fresh independent entire review remain mandatory.

  • The new pending-job detail assertion fails against the prior helper because its error contains no metadata.
  • Seven selected cleanup, bounded-detail privacy, unreachable-server and isolated-home import cases pass on the rebased source tree.
  • Current Node syntax, scoped Python lint and formatting, public prose and committed-range whitespace checks pass. The Biome result remains a prior-source receipt for the byte-identical hook files.
  • Current main is composed. The three hook entries retain their exact Git blobs and modes, and the two capture entries match the canonical source. Every other tree entry matches main.
  • Full hosted CI on the new head, including Windows, and a fresh independent whole review remain required. Local results do not qualify Windows or establish its failure's cause.

Prior-source evidence

The original cleanup fails the synthetic-writer and unfinished-job refusal controls and passes the empty-job control. The waiting helper passes both writer cases and the unfinished-job refusal. Those receipts remain prior-source evidence; the current seven-case run checks the final source.

AI assistance in the code

  • AI-generated: Codex wrote the fixture helper, bounded failure details and regression tests.
  • I can explain every line of this diff and how it interacts with the rest of Loom.

Checklist

  • Full hosted CI passes on the final head.
  • Fresh independent whole review is complete on the final head.
  • Production deadlines and all original contract assertions remain unchanged.
  • No generated files, migrations, shipped plugin or frontend changes.
  • Commits are signed off.

@vercel

vercel Bot commented Nov 20, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
loom-docs Ready Ready Preview Nov 21, 2026 5:49am UTC

Request Review

@zlareb1
zlareb1 force-pushed the fix/hook-contract-sandbox-drain branch from f1a5ac7 to 334a924 Compare November 20, 2026 23:43
@zlareb1
zlareb1 force-pushed the fix/hook-contract-sandbox-drain branch 2 times, most recently from e9a7dd2 to 4415724 Compare November 21, 2026 01:24
Signed-off-by: zlareb1 <lareb.zafar@clickhouse.com>
@zlareb1
zlareb1 force-pushed the fix/hook-contract-sandbox-drain branch from 4415724 to 8cf4840 Compare November 21, 2026 02:14
@zlareb1 zlareb1 changed the title Hook contract cleanup waits for detached jobs Hook cleanup waits for jobs and bounds failure details Nov 21, 2026
@zlareb1
zlareb1 force-pushed the fix/hook-contract-sandbox-drain branch from 5f566db to 3f95c54 Compare November 21, 2026 03:58
Keep each test sandbox until its detached flusher removes its final job marker. Preserve hook deadlines and verify unfinished jobs refuse cleanup.

Signed-off-by: zlareb1 <lareb.zafar@clickhouse.com>
Signed-off-by: zlareb1 <lareb.zafar@clickhouse.com>
Keep unfinished jobs intact and report only generated filenames and byte sizes when their existing cleanup deadline expires.

Signed-off-by: zlareb1 <lareb.zafar@clickhouse.com>
@zlareb1
zlareb1 force-pushed the fix/hook-contract-sandbox-drain branch from 3f95c54 to b0d37ff Compare November 21, 2026 05:02
Signed-off-by: zlareb1 <lareb.zafar@clickhouse.com>
Signed-off-by: zlareb1 <lareb.zafar@clickhouse.com>
zlareb1 added a commit that referenced this pull request Nov 21, 2026
Related: #1090
Related: #1096

### Changelog category

- CI Fix or Improvement (changelog entry is not required)

### Changelog entry

Not required.

### Why

Cost assertions need the process state around each original sample as well as the selected minima. Ingestion fixture failures need the worker attempt's outcome when a dataset read returns the wrong number of datasets.

Record numeric process, wall and thread clocks, allocation counts, GC state, and tracing/profiling state around the original cost calls. Scalar checks, registered lanes, the independent lane-walk child and the SQL ratio check retain their sample values and order. Failure output includes that context.

The workbook fixture's two-dataset assertion reports the actual worker return and observed dataset count. The metadata-edit fixture reports the same fields when its one-dataset assertion fails. A worker return describes that attempt; it does not establish persisted state at the later dataset read. The retrieval-floor fixture reads fixed persisted status fields only when its original readiness assertion fails.

### Scope and risk

Only test code changes. The hook-contract prerequisite waits for its owned detached flusher before removing the sandbox, using the existing isolated-home and child-environment helpers. Its hook deadlines and assertions stay unchanged.

Sample observation runs outside the original timed calls. Boundary deltas include observer work and collection or tracer restoration. Observation can perturb later samples; its fields are diagnostic context, not a cost measure or acceptance criterion. Input text and exception messages are omitted. Context errors record their class without changing the judge.

The lane-walk child emits its original two CPU minima and all six ordered samples. Input sizes, operations, call order, sample counts, minima, warmup, subprocess deadlines, comparison, floor and retry policy stay unchanged. The causes of the Python, Lua, SQL and setup.cfg cost failures remain unknown. This change claims no parser performance fix.

Dataset diagnostics use the existing worker return and dataset response, with no additional I/O, connection, status read, worker attempt or retry. The workbook still requires exactly two datasets, then runs its original unpacking and every sheet, count, rejected-row and served-row assertion. The metadata-edit fixture retains its exact cardinality, metadata, deletion and served-snapshot checks. Only fixed worker-state strings and dataset counts enter the added messages.

The retrieval diagnostic performs one same-scope status lookup with a one-second timeout. It emits whitelisted state, stage and error-code values plus a bounded integer attempt count. Missing or unavailable observations use fixed labels. Stored error messages, source content, metadata and identifiers are excluded from the added messages.

The preserved workbook failure reaches an empty dataset result after successful upload, ingest, extraction and dataset-read HTTP responses. Those responses do not establish the ingestion cause. This change adds failure context and changes no production ingestion behavior.

The retrieval fixture also observes the existing inline retry helper through one test-scoped dispatcher. Per-attempt context matches the trusted scope and originating task, delegates the same objects and awaitable exactly once, and resets before cleanup. Failure messages contain closed exception labels and bounded native numeric codes through at most four cause links. No new I/O, worker attempt, retry or deadline change is introduced. Original ingestion causation remains unknown.

### Compatibility, rollout and rollback

- Wire changes and operator steps: none.
- Rollback: revert the test diagnostics and fixture prerequisites.

### Tests

The current-main composition includes the reviewed scanner action pin from #960. Its workflow file matches main exactly; every preceding concern, generated entry and unchanged support remains byte-identical. Current scoped commands below are new-head results. Earlier CI results and review verdicts do not transfer.

On this current-main composition, all 64 directly related ingestion-observer controls pass. Scoped Ruff, formatting, public prose and committed-range whitespace pass. Three canonical diagnostic files match the independently reviewed ingestion-context source exactly.

Targeted parser-context controls also pass on this head. The prior 47-case aggregate and six Node controls remain retained receipts, not repeated aggregates. Seven cost and Node concern entries remain exact; the shared lane helper combines the unchanged diagnostic sampler nodes with current main's Thrift selectors and samples. A whole-module AST proof verifies that exact composition. The full tree equals current main plus the retained cost diagnostics and three canonical ingestion files. The CPU sampler, judges, floors, thresholds and workloads remain unchanged. Earlier failure-only sample-context and Node deadline controls retain their original evidence; no cause or performance repair is inferred from them.

The retrieval controls preserve strict readiness, scope/task isolation, delegate identity, cancellation, bounded numeric codes, privacy, the ready path and cleanup. Dataset and workbook assertions remain strict. Original hosted ingestion and cost failures are not rerun locally; their causes remain unknown. Earlier CI and review verdicts do not transfer. Fresh independent entire review and full new exact-head hosted CI remain required.

### AI assistance in the code

- AI-generated: Codex wrote the diagnostics and checked the unchanged assertion and timing contracts.

- [x] I can explain every line of this diff and how it interacts with the rest of Loom.

### Checklist

- [x] Focused qualification of the final candidate is complete.
- [ ] Full required CI passes on the eventual public head.
- [ ] A fresh independent whole review is complete on the eventual public head.
- [x] Commits are signed off.

Signed-off-by: zlareb1 <lareb.zafar@clickhouse.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant