Skip to content

fix: Skip an ABI function that cannot be indexed - #351

Merged
Uxío (Uxio0) merged 3 commits into
mainfrom
uxio/pla-2002-decoder-service-stop-one-malformed-abi-row-from-blocking
Sep 18, 2026
Merged

Uxío (Uxio0) merged 3 commits into
mainfrom
uxio/pla-2002-decoder-service-stop-one-malformed-abi-row-from-blocking

Conversation

@Uxio0

Copy link
Copy Markdown
Member

Make sure these boxes are checked! 📦✅

  • You ran ./run_tests.sh (98 passed; ran alembic upgrade head + pytest against local containers on alternative ports because the compose ports were taken)
  • You ran pre-commit run -a (clean on the changed files; insert-license also wants to add SPDX headers to 51 unrelated files that are already missing them on main, left alone)

What was wrong? 👾

Closes https://linear.app/safe-global/issue/PLA-2002

Rows in the abi table come from third party sources, so an element can be missing the fields a selector needs. selectors_from_abis read fn_abi["type"] and let the exception out, so one bad row aborted the whole batch.

At startup that exception reaches the background loader in app/main.py, which logs it and retries every DECODER_LOAD_RETRY_SECONDS forever. /health/ready stays at 503 and the pod never joins the Service. It is silent until the next restart, which may be days after the row arrived, and nothing alerts on it.

The reload path already tolerated a bad row since PLA-1954. Startup did not, because it hands a whole batch to a single thread call.

Raised by Felipe Alvarado (@falvaradorodriguez) while reviewing #350.

How was it fixed? 🎯

The guard is now per element and catches everything:

  • Per element, because a per row guard drops every function of a contract when one input is malformed. Losing one method is better. It also meant get_contract_abi_selectors_with_functions returned {} for that contract, so it lost FULL_MATCH and PARTIAL_MATCH decoding entirely.
  • Catching everything, because which exception a malformed element raises is a detail of the libraries that parse it, not a property of the data. This PR proves the point: the old malformed_abi mock stopped raising once type was defaulted, so it had to be rewritten to keep failing.

Two related fixes in the same filter:

  • A missing type means "function", as the ABI spec says. It has to be filled in rather than just accepted, because function_abi_to_4byte_selector reads the key and raises without it.
  • Elements with no name are left out. abi_to_signature invents a signature for them (function(uint256)), and decoding one later raises KeyError outside DataDecoderException, so the request returns 500. That path already existed for elements with an explicit "type": "function" and no name; defaulting the type would have widened it.

The per row handler in load_new_abis is gone. selectors_from_abis is the single funnel for startup, reload and the per address lookup, so the guard belongs there and only there.

Behaviour change worth noting: ABIs whose elements omit type used to be dropped, and are now indexed. That is why the malformed_abi mock changed shape: [{"name": "buyDroid"}] is a valid ABI under the spec default.

Numbers

The restructuring was benchmarked against the previous shape on 500 production-shaped ABIs:

ms/batch
before 9.387
after 9.411

+0.25%, about +5ms at 100k rows. The per element try costs ~0.2ns per element thanks to zero-cost exceptions in 3.13, and the dict copy only happens when type is absent, which real ABIs never are.

The short-circuit in _generate_selectors_with_abis_from_abi is kept and now goes through function_abis. It saves a 68us thread hop for an ABI with no functions, against 0.29us to check. It can sit outside the guard because function_abis cannot raise: it ignores an abi_json that is not a list and skips any element that is not a dict.

Follow-ups, not in this PR

  • load_new_abis calls add_abi per row, so each row pays a ~68us thread hop for ~18us of work, and the previous_last_abi_id is None branch feeds the entire table through it. At 300k rows that is ~20s of thread overhead against ~5.7s of real work. Routing it through the batched path would make it ~41ms. Same optimisation that took startup from 14.4s to 1.9s.
  • Normalising type at write time instead of read time. abi_hash hashes the raw json, so [{"name":"f"}] and [{"type":"function","name":"f"}] are stored as two rows for the same ABI.
  • Streaming the row id so a skipped element can be traced back to a row and fixed, instead of only logging the element.
  • Counting skipped elements and putting the number in the "Contract ABIs for decoding were loaded" line, so it can be alerted on. Today the service starts green with a hole in the selector map and only a log line says so.

Rows in the abi table come from third party sources, so an element can be
missing the fields a selector needs. selectors_from_abis read fn_abi["type"]
and let the exception out, so one bad row aborted the whole batch. At startup
that reached the background loader in main.py, which retries forever, so
/health/ready stayed at 503 and the pod never joined the Service. The reload
path already tolerated a bad row since PLA-1954, startup did not.

The guard is now per element and catches everything. Which exception a
malformed element raises is a detail of the libraries that parse it, and the
invariant is that no single element can leave the decoder without a selector
map. Per element also means losing one method instead of every function of a
contract, which is what a per row guard would drop.

A missing type now means "function", as the ABI spec says. It has to be filled
in because function_abi_to_4byte_selector reads the key and raises without it.
Elements with no name are left out: abi_to_signature invents a signature for
them, and decoding one raises KeyError later, outside DataDecoderException, so
the request returns 500.

The per row handler in load_new_abis is gone. selectors_from_abis is the single
funnel for startup, reload and the per address lookup, so the guard belongs
there and only there.

The malformed_abi mock was [{"name": "buyDroid"}], which is now a valid ABI, so
it moves to a tuple input without components.

Refs PLA-2002
@Uxio0
Uxío (Uxio0) requested a review from a team as a code owner September 17, 2026 16:13
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 17, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-09-17T16:15:52.436773Z 3df5cf2 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

Comment thread app/services/data_decoder.py
Co-authored-by: Felipe Alvarado <felipe@safe.global>
@Uxio0
Uxío (Uxio0) merged commit bb43518 into main Sep 18, 2026
8 checks passed
@Uxio0
Uxío (Uxio0) deleted the uxio/pla-2002-decoder-service-stop-one-malformed-abi-row-from-blocking branch September 18, 2026 08:58
@github-actions github-actions Bot locked and limited conversation to collaborators Sep 18, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants