Skip to content

perf(tables): Speed up dense table postprocessing - #535

Merged
CyMule merged 3 commits into
mainfrom
perf/faster-table-postprocessing
Nov 10, 2026
Merged

CyMule merged 3 commits into
mainfrom
perf/faster-table-postprocessing

Conversation

@CyMule

@CyMule CyMule commented Nov 5, 2026 •

Copy link
Copy Markdown
Contributor

What

Speed up dense table postprocessing while preserving extracted cells, text, ordering, HTML, and confidence values.

Full-job example: generated dense ledger

A two-page fictional ledger was processed through real table detection, OCR, and Table Transformer inference. Each crop contained approximately 950 inferred cells and 970 OCR tokens. These are controlled CPU test results, not production traffic measurements.

Measurement Before After Speedup
Table geometry processing, summed across both crops 13.68 s 0.33 s 41×
Complete job, submission through completion 90.07 s 76.77 s 1.17× — 13.30 s / 14.8% less elapsed time

Values are means across five matched baseline/candidate pairs with fixed worker counts and resources, interleaved run order, and identical instrumentation. All five dense-job pairs improved, saving 4.92–23.18 seconds. Model inference and OCR timings remained approximately unchanged.

The 41× result applies to the geometry step, not the model or whole job. Ordinary table PDFs and a medium-density ledger did not show a reliable whole-job improvement. Unchanged-baseline and no-table controls showed substantial job-lifecycle noise, so these results establish a promising workload-specific benefit, not a universal speedup. Step spans can overlap and should not be added to reconstruct job duration.

Reproducible geometry-only benchmark

Run python -m benchmarks.table_geometry. Medians of seven interleaved runs on a shared arm64 Mac, Python 3.12.13 / NumPy 2.4.4:

Cells / tokens Before After Speedup
10 / 40 0.574 ms 0.506 ms 1.13×
100 / 400 56.498 ms 26.161 ms 2.16×
500 / 2,000 1,399.155 ms 575.815 ms 2.43×

This separate synthetic benchmark measures assignment and text selection; it excludes OCR, model inference, and HTML generation. Speedup is before-time divided by after-time.

Why

Dense tables require many repeated rectangle comparisons when assigning words to cells. Batching those comparisons reduces Python overhead. The benefit grows when this geometry work consumes a meaningful share of the job.

Thresholds, first-match ties, token order, and object identity are preserved. Temporary pairwise arrays are bounded; unsupported coordinates and unusual rectangles retain the scalar path. Winning scores retain scalar arithmetic to preserve exact confidence rounding. No model weights or dependencies change.

Validation

On 18754ba: 431 library tests passed, with one pre-existing skip. All 13 distinct real table inputs matched exact cells and confidence, including the previous rounding regressions; 114 file comparisons in the timing campaign matched. Additional randomized checks cover score values, scalar types, assignments, and confidence without new tolerances.

CI passes on Python 3.11–3.13; independent review and Cubic found no issues. Ruff lint/format and git diff --check pass. Human approval is still required.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 5 files

Shadow auto-approve: would not auto-approve. This PR does not meet the repository auto-approval settings.
Tip: Get a Slack DM when cubic finishes reviewing your PRs. Connect Slack

View guided diff | Re-trigger cubic

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

0 issues found across 2 files (changes from recent commits).

Shadow auto-approve: would not auto-approve. This PR does not meet the repository auto-approval settings.

View guided diff | Re-trigger cubic

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

0 issues found across 2 files (changes from recent commits).

Shadow auto-approve: would not auto-approve. This PR does not meet the repository auto-approval settings.

View guided diff | Re-trigger cubic

@paulkarayan

Copy link
Copy Markdown

Automated review, forwarded by @paulkarayan. Findings are code-verified where they name a file and line; the strong-review gpt-pro leg and cubic did not run.

GO -- 0 blocker(s), 0 concern(s), 3 note(s), 1 nit(s).

Vectorized table-geometry fast path matches the scalar path exactly (0 mismatches in 64k fuzz cases); no defects found.

Notes (3)

  1. Four open PRs (527, 530, 534, 535) rewrite slot_into_containers: 530/534 conflict textually with 535; 527 merges clean but then only changes 535's scalar fallback.
  2. Fallback cost (unmeasured): when one token forces the scalar fallback, each per-row/column get_bbox_span_subset call now pays Python validation on top of the scalar loop; output unchanged.
  3. CI all green on 3.11-3.13 at 2026-11-09T21:10Z; body proof SHA matches the reviewed head 18754ba.

Nits (1)

  1. Overlap-fraction math is duplicated at table_postprocess.py:207-218 and :323-331; a shared helper would keep the two fast paths from drifting.
Full review
# Unstructured-IO/unstructured-inference#535 -- perf(tables): Speed up dense table postprocessing

Asked by: pk, via a forward to Thorn (https://unstructuredai.slack.com/archives/D0BFPFGDNP7/p1794258460768609). Author: CyMule. Reviewed head: 18754ba11c8030d236eda847fb1505e759b3abbb (base main, 0 commits behind).

**Verdict:** GO
**Blast radius:** 3/5

The PR adds a NumPy-batched fast path for `slot_into_containers` and `get_bbox_span_subset`. The original loops stay as `_scalar` fallbacks for any box the fast path cannot match exactly. No leg found a defect. Two independent fuzz runs (4,000 cases here, 60,000 in fable) found zero differences from the scalar path in assignments, score values, score types, and span identity, and CI is green on 3.11 to 3.13. To trust it, accept that the real-document parity claim (13 table inputs) is the author's own evidence, and pick one of the four open PRs rewriting this function (see Collides with).

## Collides with

Three other open PRs rewrite the same function, `slot_into_containers` in `unstructured_inference/models/table_postprocess.py`. Each was confirmed by opening that function on both heads. None carries an `order` signal, so there is no merge-order line.

- https://github.com/Unstructured-IO/unstructured-inference/pull/530 -- "perf(tables): Vectorize slot_into_containers token-to-cell scoring" (KRRT7, head 9930e716). Rewrites the `slot_into_containers` body (hunk `@@ -167,34 +172,43`). Textual conflict with 535 in `table_postprocess.py`, `CHANGELOG.md` and `__version__.py` [obs: `git merge-tree`].
- https://github.com/Unstructured-IO/unstructured-inference/pull/534 -- "perf(tables): Vectorize remove_objects_without_content span matching" (KRRT7, head f218032c). The same `slot_into_containers` rewrite as 530, plus a batched `_span_subsets_inside_bboxes`. That helper covers the same ground as 535's `get_bbox_span_subset` fast path. Textual conflict in the same three files [obs].
- https://github.com/Unstructured-IO/unstructured-inference/pull/527 -- "perf(tables): stream stable token assignment for native coordinates" (cragwolfe, head 604e068b). Replaces the sort with a streaming maximum inside the original `slot_into_containers` loop. It merges cleanly with 535 [obs], but the merged tree puts 527's streaming loop inside 535's `_slot_into_containers_scalar` (merged file line 247 onward) [obs]. The result is that 527 only changes the fallback path, and it also changes the oracle that 535's parity tests compare against.

## Notes

- Four open PRs (527, 530, 534, 535) take different approaches to speeding up the same table-geometry hot path. Whichever lands first makes the other three either rebase through a conflict (530, 534) or apply to only the scalar fallback (527). No ordering is forced.
- Fallback cost [inf, unmeasured]: `remove_objects_without_content` (table_postprocess.py:298) calls `get_bbox_span_subset` once per row or column. Each call now validates every token in Python before it decides to fall back. When one token in the table is unsupported, every call pays that validation and then also runs the full scalar loop. The output stays correct; such a table can only be somewhat slower than on main. Raised by fable; not benchmarked.
- CI at 2026-11-09T21:10:56Z [obs, `gh pr checks`]: everything passes (test 3.11/3.12/3.13, lint, changelog, CodeQL, Socket, cubic), except the `claude` job, which was skipped. The body's proof SHA (18754ba) equals the reviewed head. `reviewDecision` is REVIEW_REQUIRED.
- When any box in a call is outside the fast path (zero or negative area, `np.float32` or `np.int64` coordinates, `abs >= 2**24`, NaN or inf), the WHOLE call falls back to the scalar path. The output stays correct; that call just gets no speedup. [obs: `_positive_boxes` at table_postprocess.py:14-34]

## Findings

No BLOCKER or CONCERN survived verification.

- NIT: the width/height/area overlap-fraction math is written out twice, at table_postprocess.py:207-218 (`slot_into_containers`) and :323-331 (`get_bbox_span_subset`). A shared `_overlap_fractions` helper would keep the two fast paths from drifting apart. Today the parity tests pin both copies.

## Dismissed

- blast-radius residual risk, "float rounding in the vectorized argmax could pick a different container than the scalar sort on near-ties": dismissed. The vectorized path does the same IEEE float64 operations in the same order as `Rect.intersect` plus `get_area` (min/max, subtract, multiply, divide). Integer coordinates below 2**24 give products below 2**50, so they are exact in float64, and Python int/int true division is correctly rounded. The winner's score is then recomputed through `Rect` (table_postprocess.py:224-227), so the stored score is scalar. A local fuzz (4000 random cases: int, float, np.float64 and near-2**24 coordinates, duplicated containers for ties, zero-width packages, thresholds 0 to 1, forced on and off) found 0 mismatches in assignments, score values, score types, and subset identity [obs].
- cheshire Slop flag 1, "slot_into_containers does too much; split out the chunked argmax": dismissed as style. The function is about 50 lines and has one job.
- cheshire Slop flag 2, "the overlap-fraction math is written twice (slot_into_containers and get_bbox_span_subset)": true [obs], kept as a NIT, not a defect. Both copies are pinned against their scalar twins by `test_geometry_matches_scalar_with_chunking`.

## What each leg said

- **blast-radius:** 3/5. Shared table postprocessing that every table extraction calls. Reversibility 1 (a plain revert works; the scalar path stays in the code), wire contract 1, test coverage 2.
- **cheshire:** 1 flag (Slop, LLM judgment), two style points. One was dismissed and one is kept as a NIT. It ran with no repo config and no AGENTS.md (ungrounded).
- **collisions (review-graph):** complete=true, 3 pairs, all ranked "likely duplicate work". All 3 were confirmed (see Collides with). Note from the tool: "excluded 11 open PR(s) as candidates (exclude hook)".
- **fable (strong-review, report-only):** 171s, exit 0: SAFE TO MERGE, no actionable findings. Its own 60,000-case fuzz (17,303 cases on the fast path) found 0 mismatches. It reasoned through argmax-versus-stable-sort tie order, IEEE operation order, and the `2**24` exactness bound. It also raised one unmeasured efficiency note (see Notes). It did not run the PR's tests because they need torch.
- **codex:** `codex exec review --base origin/main`, 89s, exit 0: "No actionable regression was found in the diff." Codex did not run the tests.
- **ruff:** `uvx ruff check` on the 4 changed .py files: All checks passed.
- **Not run:** gpt-pro / oracle leg (needs a browser on a shared CDP port; unattended run). cubic (disabled for this run; the GitHub cubic bot check passed and left 3 COMMENTED reviews, with no unresolved threads [obs]).

## Not covered

- The PR's own test file and the benchmark were not run locally (`tables.py` needs torch). CI ran them green.
- The real-document parity claims (13 table inputs, 114 file comparisons) and the full-job timings are the author's evidence. They were not reproduced here.
- test-critic, marmalade and vorpal were outside this unattended run's scope and did not run.
- gpt-pro / oracle leg: not run (browser leg). cubic CLI: disabled for this run.
- Cheshire verdicts were not logged to the check-feedback file, because this run may write only the packet files.
- The 11 open PRs that the review-graph exclude hook dropped were not checked for collisions.

@CyMule
CyMule merged commit ebcf4d3 into main Nov 10, 2026
19 checks passed
@CyMule
CyMule deleted the perf/faster-table-postprocessing branch November 10, 2026 14:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants