Skip to content

feat(tests): add Leios endorser block log test - #3642

Merged
mkoura merged 2 commits into
masterfrom
leios_eb_logs
Aug 20, 2026
Merged

feat(tests): add Leios endorser block log test#3642
mkoura merged 2 commits into
masterfrom
leios_eb_logs

Conversation

@mkoura

@mkoura mkoura commented Aug 20, 2026

Copy link
Copy Markdown
Collaborator

Add test_leios_blocks.py with a test that checks the node reports the expected
Leios endorser block activity in its logs. The traces live in two namespaces,
Consensus.LeiosKernel.* and Consensus.LeiosPeer.*, and the test requires them
in two tiers. Every pool must report the messages that all block producers take
part in - EB announcements from peers and accepted announcements, acquired EB
bodies, txs and votes, its own vote and the assembled certificate. Forging,
announcing, storing and certifying an EB is reported only by the pool that won
the EB election, so those are required in the log of at least one pool. The
namespaces are anchored with \] so that Certified does not also match
CertifiedAndAnnounced.

The test is skipped outside the Dijkstra era, where the traces don't exist, and
when no Tx load generator is enabled. An EB is forged from exactly the mempool
backlog that doesn't fit into a Praos block, so with no load there is no EB, no
announcement, no votes and no certificate. For the same reason the block
production rate is checked once a cluster instance is assigned, which cannot be
a skipif marker: leios_fast produces a block every 20 seconds and gets EBs,
while the 10x faster block rate of local_fast drains the mempool.

The search waits for new blocks in rounds. Each round searches only the part of
a log that was appended since the previous round, and only for the messages that
are still missing there, so waiting for many blocks doesn't mean reading the
whole trace heavy log over and over, and a pool whose messages are all found is
not searched at all. The new search position is recorded before the search, so
that lines appended while it runs are not skipped in the next round. A stalled
chain and a log file that could not be searched are reported together with the
messages that were missing, so a failure is not just "no EB messages found".

The test cannot be marked testnets - it needs the node logs of a local cluster
and waits for many blocks - so the Leios regression run, which selects
testnets, would not run it. Add a leios marker, mark the module with it and
select it as well in scripts/test_leios.sh and in the Leios workflow.

ENABLE_TX_CENTRIFUGE and ENABLE_TX_FIREHOSE are added to the configuration,
so far they were read only by runner/regression.sh.

The expected messages were verified against the cluster logs of the 2026-08-18
leios_fast regression run - all of them are present in each of the three pool
logs. The skip paths were verified against a running cluster. The test itself has
not been run against a live Dijkstra cluster with a load generator.

mkoura added 2 commits August 20, 2026 22:07
Check that the node reports the expected Leios endorser block
activity in its logs: EB announcements, votes and certificates on
every pool, and forging, announcing, storing and certifying an EB on
at least one pool. The producer side messages are reported only by
the pool that won the EB election, so they cannot be required on
every pool log.

The search waits for new blocks in rounds. Each round searches only
the part of a log that was appended since the previous round, and
only for the messages that are still missing there, so waiting for
many blocks doesn't mean reading the whole log over and over. A
stalled chain and a log file that could not be searched are reported
together with the messages that were missing, so that a failure is
not just "no EB messages found".

The test is skipped outside the Dijkstra era and when no Tx load
generator is enabled - an EB is forged from exactly the mempool
backlog that doesn't fit into a Praos block, so without load there is
no EB, no announcement, no votes and no certificate. For the same
reason the block production rate is checked once a cluster instance
is assigned: on variants that produce a block every few seconds the
mempool never builds up a backlog.

Add `ENABLE_TX_CENTRIFUGE` and `ENABLE_TX_FIREHOSE` to the
configuration, so far they were read only by `runner/regression.sh`.
The Leios tests cannot be marked `testnets` - they need the node logs
of a local cluster and wait for many blocks - so the Leios regression
run, which selects `testnets`, would not run them. Mark them `leios`
and select that marker as well, in the launcher script and in the
workflow.
@mkoura
mkoura requested a review from saratomaz as a code owner August 20, 2026 20:17
@mkoura
mkoura merged commit 58887e2 into master Aug 20, 2026
3 checks passed
@mkoura
mkoura deleted the leios_eb_logs branch August 20, 2026 20:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant