-
Notifications
You must be signed in to change notification settings - Fork 36
DR-008 Option 4: two-stage test-execution workflow (PR 2 of 2) #280
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
fce6a51
a8dcbb7
6f83e03
e6ddcc9
e131bdd
529351d
4311861
d14e228
2959000
f9b4afe
fb20ff4
92fed52
eb0c90e
0f8ea37
9b9c9a5
0b08e04
7be8e62
df1aebb
8bbe458
33f49cd
a5466d0
4b06e8e
9268f85
e26c47d
3d06a2a
e816620
c44af1f
66b4dc1
2761dcc
f56cf80
104e15b
c992cdf
913bc97
17a4a91
e6e13e9
89e6e8b
5d9963e
b4af1d1
a992474
0b6b496
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Large diffs are not rendered by default.
Large diffs are not rendered by default.
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,42 @@ | ||
| # ******************************************************************************* | ||
| # Copyright (c) 2026 Contributors to the Eclipse Foundation | ||
| # | ||
| # See the NOTICE file(s) distributed with this work for additional | ||
| # information regarding copyright ownership. | ||
| # | ||
| # This program and the accompanying materials are made available under the | ||
| # terms of the Apache License Version 2.0 which is available at | ||
| # https://www.apache.org/licenses/LICENSE-2.0 | ||
| # | ||
| # SPDX-License-Identifier: Apache-2.0 | ||
| # ******************************************************************************* | ||
| # | ||
| # Deliberate ceilings for the deps Stage 2 collects test artifacts from. | ||
| # | ||
| # ref_int's resolved set is imposed on every module under test whether a dependency is named here or | ||
| # not; this file does not decide *which* deps get pinned. It decides which of those pins are a | ||
| # decision rather than an inheritance. `bazel_dep(version = ...)` is only a floor that MVS raises | ||
| # silently, while `single_version_override` is also a ceiling, so the versions below are ones Stage 1 | ||
| # reports as `asserted` rather than `incidental`. | ||
| # | ||
| # Every version here equals what MVS resolves today, so this changes no build now -- only what | ||
| # happens the day something in the graph asks for more. | ||
|
|
||
| # C++ test binaries and the coverage .dat files genhtml reads. | ||
| single_version_override( | ||
| module_name = "googletest", | ||
| version = "1.17.0.bcr.2", | ||
| ) | ||
|
|
||
| # Rust test rules that build the .profraw-emitting binaries. | ||
| single_version_override( | ||
| module_name = "rules_rust", | ||
| version = "0.68.2-score", | ||
| ) | ||
|
|
||
| # Ferrocene coverage tooling behind those .profraw files. Must move in the same commit as the | ||
| # bazel_dep in score_rust_toolchains.MODULE.bazel, or the bump is overruled back to this value. | ||
| single_version_override( | ||
| module_name = "score_toolchains_rust", | ||
| version = "0.9.1", | ||
| ) |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,87 @@ | ||
| # ******************************************************************************* | ||
| # Copyright (c) 2026 Contributors to the Eclipse Foundation | ||
| # | ||
| # See the NOTICE file(s) distributed with this work for additional | ||
| # information regarding copyright ownership. | ||
| # | ||
| # This program and the accompanying materials are made available under the | ||
| # terms of the Apache License Version 2.0 which is available at | ||
| # https://www.apache.org/licenses/LICENSE-2.0 | ||
| # | ||
| # SPDX-License-Identifier: Apache-2.0 | ||
| # ******************************************************************************* | ||
| # | ||
| # DR-008 Stage 2 configuration, owned by reference_integration. Layered ON TOP of the | ||
| # module's own .bazelrc via --bazelrc, so ref_int defines the configs it names in | ||
| # known_good.json instead of dangling a name whose meaning lives downstream. | ||
| # | ||
| # Read last, so single-valued flags here win; --extra_toolchains and other accumulating | ||
| # flags add to what the module already registers. Do NOT add --noworkspace_rc: it was tried and | ||
| # reverted because it discarded module settings unrelated to the configs ref_int names (stub | ||
| # trace-library selection, sandbox settings, module-owned libclang/cc toolchains, Rust coverage | ||
| # instrumentation, clippy aspects), leaving modules configured silently wrong instead of failing. | ||
| # The "3 of 8 modules" figure behind that reversal is a single earlier observation, not re-measured. | ||
|
|
||
| # rules_android is pulled in transitively (grpc-java -> rules_jvm_external) and evaluates | ||
| # android_sdk_repository, which fails when ANDROID_HOME points at an incomplete SDK, as on | ||
| # CI runners after the disk-cleanup step. Only score_baselibs guards against this itself. | ||
| common --repo_env=ANDROID_HOME= | ||
|
|
||
| # ─── stage2-linux-x86_64: emitted unconditionally by quality_runners.py ────── | ||
| build:stage2-linux-x86_64 --host_platform=@score_bazel_platforms//:x86_64-linux-gcc_12.2.0-posix | ||
| build:stage2-linux-x86_64 --platforms=@score_bazel_platforms//:x86_64-linux-gcc_12.2.0-posix | ||
|
|
||
| # Test selection and coverage policy. `coverage` inherits `test` inherits `build`. | ||
| # -miri: ref_int registers no miri toolchain. -no-coverage: gcov-instrumenting a TSAN | ||
| # binary reports false races on the non-atomic __gcov* counters. | ||
| test:stage2-linux-x86_64 --build_tests_only | ||
| test:stage2-linux-x86_64 --test_tag_filters=-manual,-miri,-no-coverage | ||
| test:stage2-linux-x86_64 --test_output=errors | ||
| test:stage2-linux-x86_64 --test_summary=testcase | ||
| test:stage2-linux-x86_64 --test_verbose_timeout_warnings | ||
| test:stage2-linux-x86_64 --test_timeout=1200 | ||
| test:stage2-linux-x86_64 --nocache_test_results | ||
|
|
||
| coverage:stage2-linux-x86_64 --features=coverage | ||
| coverage:stage2-linux-x86_64 --combined_report=lcov | ||
| # Make gcov counter updates atomic so a multithreaded coverage test is race-free. | ||
| coverage:stage2-linux-x86_64 --copt=-fprofile-update=atomic | ||
| coverage:stage2-linux-x86_64 --linkopt=-fprofile-update=atomic | ||
|
|
||
| # ─── stage2-gcc: score's gcc x86_64 toolchain ──────────────────────────────── | ||
| # Opt-in because the target name is generated by each module's own gcc.toolchain() call: | ||
| # score_communication passes use_base_constraints_only = True, which yields :x86_64-linux | ||
| # instead, and it registers its own cc toolchain unconditionally — so it omits this. | ||
| build:stage2-gcc --extra_toolchains=@score_gcc_x86_64_toolchain//:x86_64-linux-gcc_12.2.0 | ||
|
|
||
| # ─── stage2-rust: ferrocene Rust toolchain ─────────────────────────────────── | ||
| # Opt-in because score_time declares no score_toolchains_rust, so the apparent repo name | ||
| # does not resolve in its graph. Folds into the base once Phase 1 injects a bazel_dep stub | ||
| # for every module in the resolved set (PR #278). | ||
| build:stage2-rust --extra_toolchains=@score_toolchains_rust//toolchains/ferrocene:ferrocene_x86_64_unknown_linux_gnu | ||
|
|
||
| # ─── ferrocene-coverage: Rust coverage instrumentation ─────────────────────── | ||
| # Added in code by stage2_config_flags, never opted into via known_good.json: rustc must emit | ||
| # .profraw during the same run ferrocene_report later reads. kyron/persistency/lifecycle_health | ||
| # define this name identically themselves (layering repeats the same values, a no-op); | ||
| # score_logging has no Rust instrumentation config, so this is its only source. | ||
| build:ferrocene-coverage --@rules_rust//rust/settings:extra_rustc_flag=-Cinstrument-coverage | ||
| build:ferrocene-coverage --@rules_rust//rust/settings:extra_rustc_flag=-Clink-dead-code | ||
| build:ferrocene-coverage --@rules_rust//rust/settings:extra_rustc_flag=-Ccodegen-units=1 | ||
| build:ferrocene-coverage --@rules_rust//rust/settings:extra_rustc_flag=-Cdebuginfo=2 | ||
| build:ferrocene-coverage --@rules_rust//rust/settings:extra_exec_rustc_flag=-Cinstrument-coverage | ||
| build:ferrocene-coverage --@rules_rust//rust/settings:extra_exec_rustc_flag=-Clink-dead-code | ||
| build:ferrocene-coverage --@rules_rust//rust/settings:extra_exec_rustc_flag=-Ccodegen-units=1 | ||
| build:ferrocene-coverage --@rules_rust//rust/settings:extra_exec_rustc_flag=-Cdebuginfo=2 | ||
| test:ferrocene-coverage --run_under=@score_tooling//coverage:llvm_profile_wrapper | ||
|
|
||
| # Coverage needs to have all intermediate .rlibs to be able to proceed | ||
| build:ferrocene-coverage --remote_download_all | ||
|
|
||
| # score_persistency's rust_coverage_config; its own .bazelrc does not define this name, so this | ||
| # is the sole source. Only the two score_baselibs settings are ported -- deps persistency itself | ||
| # declares, so they resolve in its checkout. ref_int's root .bazelrc has a third, | ||
| # @score_logging-relative one, which persistency's `extra_test_config` already passes directly. | ||
| build:ferrocene-coverage-per --config=ferrocene-coverage | ||
| build:ferrocene-coverage-per --@score_baselibs//src/log:safety_level=qm | ||
| build:ferrocene-coverage-per --@score_baselibs//score/json:base_library=nlohmann |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -12,15 +12,18 @@ | |
| "@score_baselibs//score/json:base_library=nlohmann" | ||
| ], | ||
| "exclude_test_targets": [ | ||
| "//score/language/safecpp/aborts_upon_exception:abortsuponexception_toolchain_test", | ||
| "//score/containers:dynamic_array_test", | ||
| "//score/mw/log/configuration:*", | ||
| "//score/json/examples:*", | ||
| "//score/flatbuffers:version_reader_test" | ||
| "//score/language/safecpp/aborts_upon_exception:abortsuponexception_toolchain_test" | ||
| ], | ||
| "langs": [ | ||
| "cpp" | ||
| ] | ||
| ], | ||
| "bazel_config": [ | ||
| "stage2-gcc", | ||
| "stage2-rust" | ||
|
Comment on lines
+20
to
+22
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. it feels like it doesnt need to be placed in this json as its common for every module
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. score_time and score_communication are exceptions for the above. |
||
| ], | ||
| "exclude_test_target_reasons": { | ||
| "//score/language/safecpp/aborts_upon_exception:abortsuponexception_toolchain_test": "Stage 2's toolchain does not implement the aborts_upon_exception feature." | ||
| } | ||
| } | ||
| }, | ||
| "score_communication": { | ||
|
|
@@ -36,15 +39,9 @@ | |
| "extra_test_config": [ | ||
| "@score_communication//score/memory/shared/flags:use_typedshmd=False" | ||
| ], | ||
| "exclude_test_targets": [ | ||
| "//score/mw/com/impl:unit_test_runtime_single_exec", | ||
| "//score/mw/com/impl:runtime_test", | ||
| "//score/mw/com/impl/configuration:config_parser_test", | ||
| "//score/mw/com/impl/configuration:configuration_test", | ||
| "//score/mw/com/impl/configuration:configuration_json_parsing_strategy_test", | ||
| "//score/mw/com/impl/tracing/configuration:tracing_filter_config_parser_test", | ||
| "//score/mw/com/impl/tracing:tracing_runtime_test", | ||
| "//score/mw/com/impl/bindings/lola/tracing:tracing_runtime_test" | ||
| "exclude_test_targets": [], | ||
| "bazel_config": [ | ||
| "stage2-rust" | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. why com has only rust?
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. score_communication passes use_base_constraints_only=True to its own gcc.toolchain(), which changes its generated toolchain target name leading to referencing a target that doesn't exist. |
||
| ] | ||
| } | ||
| }, | ||
|
|
@@ -66,7 +63,14 @@ | |
| "exclude_test_targets": [ | ||
| "//src/cpp/tests:bm_kvs_cpp" | ||
| ], | ||
| "rust_coverage_config": "ferrocene-coverage-per" | ||
| "rust_coverage_config": "ferrocene-coverage-per", | ||
| "bazel_config": [ | ||
| "stage2-gcc", | ||
| "stage2-rust" | ||
| ], | ||
| "exclude_test_target_reasons": { | ||
| "//src/cpp/tests:bm_kvs_cpp": "google_benchmark benchmark declared as a cc_test; a timing measurement, not a correctness test, so it is not meaningful in a UT/coverage run." | ||
| } | ||
| } | ||
| }, | ||
| "score_orchestrator": { | ||
|
|
@@ -76,6 +80,10 @@ | |
| "code_root_path": "//src/...", | ||
| "langs": [ | ||
| "rust" | ||
| ], | ||
| "bazel_config": [ | ||
| "stage2-gcc", | ||
| "stage2-rust" | ||
| ] | ||
| } | ||
| }, | ||
|
|
@@ -86,6 +94,10 @@ | |
| "code_root_path": "//src/...", | ||
| "langs": [ | ||
| "rust" | ||
| ], | ||
| "bazel_config": [ | ||
| "stage2-gcc", | ||
| "stage2-rust" | ||
| ] | ||
| } | ||
| }, | ||
|
|
@@ -97,9 +109,16 @@ | |
| "code_root_path": "//score/...", | ||
| "exclude_test_targets": [ | ||
| "//score/health_monitor/src/rust:miri_tests", | ||
| "//score/launch_manager/src/daemon/src/common/concurrency:mpmc_concurrent_queue_tsan_test", | ||
| "//score/launch_manager/src/daemon/src/common/concurrency:mpsc_bounded_queue_tsan_test" | ||
| ] | ||
| "//score/launch_manager/src/daemon/src/common/concurrency:mpmc_concurrent_queue_tsan_test" | ||
| ], | ||
| "bazel_config": [ | ||
| "stage2-gcc", | ||
| "stage2-rust" | ||
| ], | ||
| "exclude_test_target_reasons": { | ||
| "//score/health_monitor/src/rust:miri_tests": "miri_test rule (tags=[\"miri\"]); needs the miri interpreter toolchain, which Stage 2's configs do not provide.", | ||
| "//score/launch_manager/src/daemon/src/common/concurrency:mpmc_concurrent_queue_tsan_test": "built with -fsanitize=thread (tags=[\"tsan\"]); needs the TSan runtime and a sanitizer build, incompatible with the coverage configuration." | ||
| } | ||
| } | ||
| }, | ||
| "score_logging": { | ||
|
|
@@ -121,8 +140,10 @@ | |
| "@score_logging//score/datarouter/build_configuration_flags:file_transfer=False", | ||
| "@score_logging//score/datarouter/build_configuration_flags:use_local_vlan=True" | ||
| ], | ||
| "exclude_test_targets": [ | ||
| "//score/mw/log/legacy_non_verbose_api:unit_test" | ||
| "exclude_test_targets": [], | ||
| "bazel_config": [ | ||
| "stage2-gcc", | ||
| "stage2-rust" | ||
| ] | ||
| } | ||
| }, | ||
|
|
@@ -135,6 +156,9 @@ | |
| "metadata": { | ||
| "langs": [ | ||
| "cpp" | ||
| ], | ||
| "bazel_config": [ | ||
| "stage2-gcc" | ||
| ] | ||
| } | ||
| }, | ||
|
|
@@ -158,7 +182,14 @@ | |
| ], | ||
| "langs": [ | ||
| "cpp" | ||
| ] | ||
| ], | ||
| "exclude_test_target_reasons": { | ||
| "//score/config_management/config_daemon/code/factory/details:unit_test_mw_com": "Excluded when the module joined ref_int (#299): depends on mw_com/SOCAL targets not present in the OSS build, which ref_int's patches/config_management/001-adapt-proxy-api-and-remove-internal-targets.patch removes. Carried over unchanged; not yet re-audited for Stage 2.", | ||
| "//score/config_management/config_daemon/code/services/details/mw_com:unit_test": "Excluded when the module joined ref_int (#299): depends on mw_com/SOCAL targets not present in the OSS build, which ref_int's patches/config_management/001-adapt-proxy-api-and-remove-internal-targets.patch removes. Carried over unchanged; not yet re-audited for Stage 2.", | ||
| "//score/config_management/config_provider/code/config_provider/factory:unit_tests_mw_com": "Excluded when the module joined ref_int (#299): depends on mw_com/SOCAL targets not present in the OSS build, which ref_int's patches/config_management/001-adapt-proxy-api-and-remove-internal-targets.patch removes. Carried over unchanged; not yet re-audited for Stage 2.", | ||
| "//score/config_management/config_provider/code/proxies/details:unit_test_mw": "Excluded when the module joined ref_int (#299): depends on mw_com/SOCAL targets not present in the OSS build, which ref_int's patches/config_management/001-adapt-proxy-api-and-remove-internal-targets.patch removes. Carried over unchanged; not yet re-audited for Stage 2.", | ||
| "//score/config_management/dependability/...": "Excluded when the module joined ref_int (#299): depends on mw_com/SOCAL targets not present in the OSS build, which ref_int's patches/config_management/001-adapt-proxy-api-and-remove-internal-targets.patch removes. Carried over unchanged; not yet re-audited for Stage 2." | ||
| } | ||
| } | ||
| } | ||
| }, | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
where is this file generated from? All modules should be from known_good
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
not generated — known_good.json pins first-party repos by commit, these are registry deps pinned by version (confirmed via bazel mod graph --verbose: 2 of the 3 pins do move MVS's selection)