os:Scenario-based system stability verification. - #7499
os:Scenario-based system stability verification.#7499prathamrajbhatt-456 wants to merge 1 commit into
Conversation
…lock issues. Configs need to be enabled: CONFIG_EXAMPLES_TESTCASE=y CONFIG_EXAMPLES_TESTCASE_KERNEL=y CONFIG_DRIVERS_OS_API_TEST=y CONFIG_TC_KERNEL_SEM_SCENARIO=y CONFIG_STC_KERNEL_SCHED=y CONFIG_SMP=y It includes scenarios for affinity, priority, RR, SMP and monitoring.
| int stc_sched_rr_main(void); | ||
| int stc_sched_smp_main(void); | ||
| int stc_sched_priority_main(void); | ||
| int stc_sched_affinity_main(void); |
There was a problem hiding this comment.
Could you please explain what is difference itc and stc?
itc means Integrated test
There was a problem hiding this comment.
stc stands for scenario test case
The key difference is ITC validates API return values, whereas STC validates actual runtime scenarios.
Like for example:
Scenario-Round-Robin-01: Verify that round-robin (SCHED_RR) scheduling fairly rotates CPU time among equal-priority tasks when the system is oversubscribed (more runnable tasks than available CPUs).
hyunjongkim123
left a comment
hyunjongkim123
left a comment
There was a problem hiding this comment.
Thanks for adding these scenario-based scheduler test cases. They make the RR/SMP behavior much easier to validate and debug.
| int ncpus = stc_ncpus(); | ||
| int nworkers = ncpus + 1; | ||
|
|
||
| stc_workload_init(&wl, nworkers, SCHED_PRIORITY_DEFAULT, SCHED_RR); |
There was a problem hiding this comment.
There are tasks such as LWIP_TCP/IP (priority 105) that repeatedly wake up and block.
When the worker and harness priorities are 100 and 110, respectively, those higher-priority tasks can introduce additional scheduling points. This can mask an RR failure by giving otherwise-starved workers incidental execution opportunities.
With the original priorities (worker: 100, harness: 110), RR-01 fails the fairness assertion at line 176. With the raised priorities (worker: 180, harness: 190), RR-01 instead fails the starvation assertion at line 172.
I believe the failure at line 172 is the expected and more direct result, because it shows that a runnable worker was never scheduled during the measurement window.
Would it make sense to raise the test priority band? We could use 180 for the workers and 190 for the harness, since kernel_tc is originally created with priority 180.
| * | ||
| ****************************************************************************/ | ||
|
|
||
| static void stc_sched_rr02_fairness_single_cpu(void) |
There was a problem hiding this comment.
How about consolidating RR-02 and RR-03? Rather than testing CPU0 and the last CPU in separate test cases, RR-02 could iterate over every CPU, pin same-priority RR workers to the target CPU, and verify RR fairness. If it also enables sample_cpu and verifies seen_mask == target_cpu and violations == 0, it can cover both RR-02/RR-03 and the actual affinity-enforcement coverage of AFF-01.
AFF-03 also appears largely covered by this approach in terms of RR behavior and static affinity enforcement. Its only unique aspect is running workers with different affinity masks concurrently. If that coverage is not required, AFF-03 could be removed; otherwise, the mixed-affinity setup could be added as a subcase of the consolidated RR-02.
|
|
||
| /* Once promoted it outranks every peer, so it must run. */ | ||
|
|
||
| TC_ASSERT_GT("pri03_promoted_ran", after, 0); |
There was a problem hiding this comment.
PRI-03 currently only verifies that the target worker runs at least once after its priority is raised. It would be more meaningful to compare the target worker's relative work share before and after the priority change, and verify that its CPU share increases after promotion.
| case TESTIOC_GET_TCB_ADJ_STACK_SIZE: | ||
| case TESTIOC_SCHED_FOREACH: | ||
| #ifdef CONFIG_SMP | ||
| case TESTIOC_SCHED_CPUSTATE: |
There was a problem hiding this comment.
I may be missing something, but are the changes to test_sched.c, os_api_test_drv.c, and os_api_test_drv.h needed for this patch?
I could not find any of the new scheduler scenarios using TESTIOC_SCHED_CPUSTATE; AFF-05 and SMP-01 appear to use the existing TESTIOC_SCHED_FOREACH interface instead. If there is no planned consumer for TESTIOC_SCHED_CPUSTATE, would it make sense to keep these changes for the patch that introduces its usage?
There was a problem hiding this comment.
I agree with @hyunjongkim123.
TESTIOC_SCHED_CPUSTATE is not used in any test case that added.
| printf("[SCN-SMP] structural invariant scenarios, ncpus=%d\n", | ||
| stc_ncpus()); | ||
|
|
||
| stc_sched_smp01_starvation_invariant(); |
There was a problem hiding this comment.
Since RR-03 directly verifies RR fairness on a secondary CPU, could SMP-01 be removed if RR-01 through RR-03 are retained?
4 participants
We have created scenarios for system stability verification for better detection of product issues, currently made for deadlock and hang issues as reported in product testing issues. these scenario-based cases have been verified on current TizenRT public code on which 6 scenarios are failed and 7 scenarios are passed.
Note: The complete scenario-based system stability plan will include to cover other issues as well. such as system drivers, arm specific, locking and reboot cases.