Skip to content

Bound inference admission by documents and source pages - #75

Merged
xujustinj merged 3 commits into
mainfrom
feat/page-budget-admission
Oct 12, 2026
Merged

xujustinj merged 3 commits into
mainfrom
feat/page-budget-admission

Conversation

@xujustinj

@xujustinj xujustinj commented Oct 12, 2026 •

Copy link
Copy Markdown

Add optional --max_concurrent_pages alongside the document limit. Count readable source PDF pages before submission and admit in FIFO order; oversized PDFs run alone. Based on main, including the merged seeded shuffle (#74).

Hold capacity through provider processing, polling, and retries. Cached results consume no capacity. Document errors remain recorded while subsequent documents continue. Page-limited attempts use independent workers, so a thread stuck beyond its deadline cannot starve healthy documents or retries. Python cannot kill a running thread: a timed-out attempt or remote job can outlive its local reservation. These limits bound local admission, not all surviving remote work.

Save the page-count plan and timestamped admission/release events. Capacity rollback and waiter notification survive log-write errors; the I/O error still propagates. On resume, archive the previous plan and event log under _admission_history/ before creating the current session's files, preserving interrupted records too.

The default remains unlimited pages. PDF counting uses the existing local or runners extra and is imported only when enabled; no dependency-lock changes.

Validation:

  • Full local suite: 1,040 passed; one existing numba-dependent test skipped.
  • 29 page-admission cases, including 101/123-page PDFs under a 100-page budget, FIFO/no backfill, cached skips, cancellation, Rich on/off, and CLI forwarding.
  • A stuck first document leaves two healthy documents able to execute (2 successes / 1 timeout); a healthy retry also executes while the first attempt remains stuck. Covered with 1/2 workers and both file/test-case entrypoints.
  • Admission and release log-write failures leave no capacity leak and wake waiters.
  • Resume preserves prior logs/plans byte-for-byte, including a truncated trailing record and an all-cached run.
  • Repository-wide Ruff lint/format and git diff --check.
  • No provider requests or paid inference.

@xujustinj
xujustinj changed the base branch from feat/reproducible-shuffle to main October 12, 2026 06:59
@xujustinj
xujustinj merged commit b9bf92a into main Oct 12, 2026
3 checks passed
@xujustinj
xujustinj deleted the feat/page-budget-admission branch October 12, 2026 07:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant