test: Check why a blockchain test block was rejected - #1623
Conversation
19d0bc2 to
3d555c5
Compare
3bc40bc to
a5099a1
Compare
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #1623 +/- ##
==========================================
- Coverage 97.73% 97.70% -0.04%
==========================================
Files 171 171
Lines 15565 15593 +28
Branches 3604 3606 +2
==========================================
+ Hits 15213 15235 +22
- Misses 263 269 +6
Partials 89 89
Flags with carried forward coverage won't be shown. Click here to find out more.
🚀 New features to boost your workflow:
|
3d555c5 to
3ddecd8
Compare
a5099a1 to
85903da
Compare
3ddecd8 to
13146b5
Compare
85903da to
80f22df
Compare
13146b5 to
528d969
Compare
80f22df to
65c90ed
Compare
3271aac to
29faa6c
Compare
65c90ed to
582fff4
Compare
29faa6c to
04fdfcb
Compare
582fff4 to
0b87e1b
Compare
0b87e1b to
84f53a7
Compare
There was a problem hiding this comment.
Pull request overview
Tightens blockchain-test rejection checks so invalid blocks must fail for the fixture’s expected reason.
Changes:
- Preserves structured transaction errors for matching.
- Validates transaction, request, block, and header-mismatch rejection reasons.
- Documents signature-recovery and ommer-validation gaps.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
test/utils/t8n.cpp |
Renders preserved transaction errors. |
test/utils/block_transition.hpp |
Stores rejection error codes. |
test/utils/block_transition.cpp |
Propagates transaction error codes. |
test/blockchaintest/blockchaintest_runner.cpp |
Adds rejection-reason verification. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
84f53a7 to
4b40369
Compare
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 6 out of 6 changed files in this pull request and generated 1 comment.
Suppressed comments (2)
test/blockchaintest/blockchaintest_runner.cpp:396
- This disables reason checking for every executed block that contains an ommer, not only fixtures whose expected failure is an unvalidated ommer-header rule. Consequently, an unrelated block with an ommer can fail any header check above for the wrong reason and still be counted as correctly rejected. Gate the exemption on the specific unsupported ommer exception(s), rather than merely on a nonempty ommer list.
const auto expect_fixture_names = [&](std::string_view names) {
if (!names_spec_exception || ommers_not_validated)
return;
test/blockchaintest/blockchaintest_runner.cpp:334
- The new rejection-reason behavior has no regression coverage: the existing
test/integration/blockchaintest/*.jsoncases contain no invalid block withexpectException. Add invalid fixtures that verify both a matching reason and a deliberately different reason (including validation, rejected-transaction, and computed-header exits), so future broad matching or silent exits cannot reintroduce false positives.
EXPECT_TRUE(
is_expected_block_exception(block_error, test_block.expected_exception))
| {state::INCORRECT_BLOCK_FORMAT, | ||
| "BlockException.GAS_USED_OVERFLOW|" | ||
| "BlockException.IMPORT_IMPOSSIBLE_UNCLES_OVER_PARIS|" | ||
| "BlockException.RLP_STRUCTURES_ENCODING|" | ||
| "BlockException.RLP_INVALID_FIELD_OVERFLOW_64"}, |
There was a problem hiding this comment.
Agreed on the coarseness, and I have added a comment stating it: nine validate_block() branches share INCORRECT_BLOCK_FORMAT, so to the matcher these four names are interchangeable, and separating them needs a distinct code per branch.
Worth noting it is not new behaviour introduced here. Before this commit the loader's map_legacy_exception rewrote those same three spec names to INCORRECT_BLOCK_FORMAT outright, so the interchangeability already existed — it was just hidden in the loader, and it destroyed the fixture's stated expectation on the way. This change removes those rewrites and puts the equivalence in one visible table, which is what makes the granularity gap reviewable at all. Giving the branches distinct codes is a change to state/errors.hpp plus validate_block(), deliberately outside this PR.
A block expected to be invalid only had to differ from its fixture somewhere: every header mismatch counted as "correctly rejected", so a block that failed the state root of a test about the receipts root passed just as well. A rejected transaction only had to be rejected for some reason, with "TransactionException." appearing anywhere in the expectation. Each header mismatch is now matched against the BlockException it is the symptom of, and the transaction reason goes through is_expected_tx_exception(), the same mapping the state test runner uses. That one takes the error code rather than its message, so apply_block() reports the code. Two rules evmone does not implement keep their old verdict, each marked with a TODO: transaction senders come from the fixture instead of being recovered from the signature, and ommer headers are not validated.
4b40369 to
8068e29
Compare
Follow-up to #1623. The unrecovered-sender gap was patched only in the state-root check's names, but the blob-gas check runs before it, so a block whose bad signature the runner cannot see was judged against a rule it does not break. Skip the whole ladder instead, as already done for the unvalidated ommer headers. No existing fixture reaches that path, so the regression test is a new one: a Cancun block carrying a blob transaction with a broken signature and a `blobGasUsed` the execution contradicts. Without the fix it reports ``` the block failed the check for BlockException.INCORRECT_BLOB_GAS_USED|BlockException.BLOB_GAS_USED_ABOVE_LIMIT, expected TransactionException.INVALID_SIGNATURE_VRS ```
A blockchain test block expected to be invalid counted as correctly rejected however evmone rejected it, so a block failing a rule other than the one the test is about still passed. The header-mismatch branches were the weakest: any difference from the claimed header ended the case silently.
Compare the rejection against the fixture's exception at every exit: block validation, a rejected transaction, requests collection, and each header mismatch, which now names the
BlockExceptionit is the symptom of.Two known gaps are documented rather than checked: transaction senders come from the fixture instead of the signature, so a broken signature surfaces only as a state-root mismatch, and ommer headers are not validated.
🤖 Generated with Claude Code