[felixdittrich92]

What

Hooks added with add_hook() now run before the word crops are extracted, in both OCRPredictor and KIEPredictor. The recognition model reads exactly the boxes the hooks return.

Why

Hooks used to run after _prepare_crops, when the crops had already been cut from the original detections:

• changing a box in a hook only changed the reported geometry, not what was read;
• removing or reordering boxes paired the recognized text with the wrong boxes, without any error. On a sample invoice, dropping one of 71 boxes gave 70 words the wrong text.

Changes

• _OCRPredictor._apply_hooks runs the hooks on the detected boxes before cropping.
• _OCRPredictor._align_hook_output:
  • checks the hook output: one array per page, same box format ((N, 4) or (N, 4, 2));
  • gives each returned box the objectness score of the closest detected box, so moved, filtered or reordered boxes keep their own score. A box added on a page without detections gets a score of 1.
• KIEPredictor follows the same flow per class. Its hooks still receive and return the {class_name: [...]} dict.
• Hooks receive a copy of the list, so editing it in place doesn't affect the score matching.
• The add_hook docstring documents the contract.

# Skip the top and bottom 5% of the page: these boxes are no longer read
predictor.add_hook(lambda preds: [b[((b[:, 1] + b[:, 3]) / 2 > 0.05) & ((b[:, 1] + b[:, 3]) / 2 < 0.95)] for b in preds])

Compatibility

Hooks that return the boxes unchanged behave exactly as before. A hook that returns something other than one array per page, or boxes of another shape, now raises a ValueError.

Tests

• tests/common/test_models.py::test_align_hook_output: score matching, added and removed boxes, polygons, invalid outputs.
• tests/pytorch/test_models_zoo_pt.py::test_predictors_hooks_run_before_cropping: OCR and KIE, straight and rotated boxes. It fails on main and passes with this change.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 97.74%. Comparing base (323de1f) to head (6dcb313).

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2184      +/-   ##
==========================================
- Coverage   97.74%   97.74%   -0.01%     
==========================================
  Files         170      170              
  Lines       10892    10932      +40     
==========================================
+ Hits        10646    10685      +39     
- Misses        246      247       +1     

Flag	Coverage Δ
unittests	97.74% <100.00%> (-0.01%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.