Repository navigation
Conversation
…ce condition by waiting for post-warmboot stats sync
gang-tao
force-pushed
the
m4061clsc_agent_fix1
branch
from
October 30, 2026 12:09
e7033f5 to
333bae7
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Pre-submission checklist
pip install -r requirements-dev.txt && pre-commit installpre-commit runSummary
In Split-Agent / Multi-Switch test environments,
AgentAclTableGroupTrafficTesttest cases (specificallyVerifyDscpMarkingAndTtlAclTableandVerifyQueuePerHostAclTableAndTtlAclTable) occasionally failed during warmboot with assertion timeouts:Root Cause
In Split-Agent architecture, hardware counters are asynchronously published by HwAgent to SwSwitch via the
syncHwStatsThrift stream every second.During cold boot, the test executes a multi-second
setup()routine, allowing ample time for initial hardware statistics to synchronize.However, during warm boot,
setup()is skipped and the test runner immediately entersverify(), callingpktCounterHelper()within milliseconds of SwSwitch initialization. At this instant, SwSwitch has not yet received its first post-warmbootsyncHwStatssnapshot from Agent, causinggetAclInOutPacketsto return a default baseline of 0.When HwAgent subsequently pushes the first statistics event (~100ms later), it carries the cumulative ASIC counters inherited across warm boot (DSCP: 240, TTL: 80) plus newly injected packets (DSCP: 40, TTL: 20), resulting in
dscpAclPkts - beforeAclPkts.first = 280 - 0 = 280expected 40), triggering a test timeout failure.Solution
In
AgentAclTableGroupTrafficTests.cpp, invokethis->getNextUpdatedPortStats(this->masterLogicalPortIds()[0])before querying initial ACL counters in both_verifyHelperDscpTtlAclTablesand_verifyHelperMultipleAclTables.This ensures the test runner blocks until HwAgent publishes a fresh post-warmboot statistics snapshot with updated timestamps, guaranteeing the baseline counter accurately reflects hardware state before packet transmission begins.
This follows the exact same established pattern as commit 186ac01 : Wait for HW stats before ACL baseline in AgentQueuePerHostTest) and AgentMPLSTtlTests, ensuring the test runner blocks until HwAgent publishes a fresh post-warmboot statistics snapshot before packet transmission begins.
Test Plan
Both test cases were verified on icephoton unit with 100% pass rate across multiple Coldboot + Warmboot iterations:
AgentAclTableGroupTrafficTest.VerifyDscpMarkingAndTtlAclTable (3 consecutive runs):
• Run 1:
[ PASSED ] cold_boot.AgentAclTableGroupTrafficTest.VerifyDscpMarkingAndTtlAclTable (23668 ms)
[ PASSED ] warm_boot.AgentAclTableGroupTrafficTest.VerifyDscpMarkingAndTtlAclTable (12130 ms)
• Run 2:
[ PASSED ] cold_boot.AgentAclTableGroupTrafficTest.VerifyDscpMarkingAndTtlAclTable (23580 ms)
[ PASSED ] warm_boot.AgentAclTableGroupTrafficTest.VerifyDscpMarkingAndTtlAclTable (12127 ms)
• Run 3:
[ PASSED ] cold_boot.AgentAclTableGroupTrafficTest.VerifyDscpMarkingAndTtlAclTable (23857 ms)
[ PASSED ] warm_boot.AgentAclTableGroupTrafficTest.VerifyDscpMarkingAndTtlAclTable (12130 ms)
AgentAclTableGroupTrafficTest.VerifyQueuePerHostAclTableAndTtlAclTable (2 consecutive runs):
• Run 1:
[ PASSED ] cold_boot.AgentAclTableGroupTrafficTest.VerifyQueuePerHostAclTableAndTtlAclTable (25331 ms)
[ PASSED ] warm_boot.AgentAclTableGroupTrafficTest.VerifyQueuePerHostAclTableAndTtlAclTable (14132 ms)
• Run 2:
[ PASSED ] cold_boot.AgentAclTableGroupTrafficTest.VerifyQueuePerHostAclTableAndTtlAclTable (25865 ms)
[ PASSED ] warm_boot.AgentAclTableGroupTrafficTest.VerifyQueuePerHostAclTableAndTtlAclTable (14133 ms)