Skip to content

chore: add just test-integrations to run every integration against the checkout - #659

Closed
lesnik512 wants to merge 1 commit into
mainfrom
chore/test-integrations-recipe
Closed

lesnik512 wants to merge 1 commit into
mainfrom
chore/test-integrations-recipe

Conversation

@lesnik512

Copy link
Copy Markdown
Member

Adds just test-integrations, which runs each integration's own test suite and ty check against this checkout of modern-di. docs/agents/release.md now says to run it on main before tagging.

modern-python/modern-di-grpc#42 is why. 4.1.0 changed FinalizerError's message, two grpc tests matched the old text, and we only found out by running the integrations by hand after the release. This recipe makes that run one command, before the tag.

What it does

For each page under docs/integrations/ except writing-integrations, the same list test-docs-integrations uses, it:

  1. shallow-clones modern-python/modern-di-<page> from GitHub into a temp dir,
  2. runs that repo's own just install,
  3. installs this checkout over the released modern-di with uv pip install --overrides, as test-docs-integrations does,
  4. runs pytest and ty check.

It prints ok or FAIL per repo, with the last 30 lines of the log for a failure, and exits 1 if any repo failed. Page names pick a subset: just test-integrations grpc flask. The temp dir is removed on exit.

CI doesn't run it. It clones 14 repos and takes about a minute and a half, and a red integration is a release decision, not a reason to block an unrelated PR. Adding it to CI later is easy if you want that.

Checked

  • All 14 repos pass against current main (1m19s).
  • The clones really use this checkout: with a raise added to modern_di/__init__.py, just test-integrations grpc flask reported both as FAIL with that error in the log.
  • A failure exits 1 (checked with a name that can't be cloned).
  • just lint passes.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Benchmark

Details
Benchmark suite Current: a6e9bd3 Previous: 06a4ad3 Ratio
benchmarks/test_guard_by_type.py::test_g16_resolve_by_type 3800726.2503737006 iter/sec (stddev: 2.3433297617884394e-8) 5697958.620842745 iter/sec (stddev: 1.3570675084797464e-8) 1.50
benchmarks/test_guard_by_type.py::test_g17_resolve_by_type_large_registry 3832535.2270466564 iter/sec (stddev: 1.1832039138476998e-8) 5507230.222262393 iter/sec (stddev: 1.473790074985332e-8) 1.44
benchmarks/test_guard_cold.py::test_g8_cold_first_resolve 21966.90204369706 iter/sec (stddev: 0.00004748861655289963) 28796.035400829132 iter/sec (stddev: 0.00003477620348274097) 1.31
benchmarks/test_guard_cold.py::test_g8b_cold_first_resolve_cached 17346.21921148182 iter/sec (stddev: 0.00016445385610787996) 23789.329295981275 iter/sec (stddev: 0.0001364890390419508) 1.37
benchmarks/test_guard_concurrency.py::test_g14_concurrent_cached_hit[1] 397.41459954605716 iter/sec (stddev: 0.00009754413118840224) 772.5884051722902 iter/sec (stddev: 0.00005957745996070279) 1.94
benchmarks/test_guard_concurrency.py::test_g14_concurrent_cached_hit[2] 365.11170162905586 iter/sec (stddev: 0.00010069653547796728) 726.2491960237369 iter/sec (stddev: 0.00005256946017327482) 1.99
benchmarks/test_guard_concurrency.py::test_g14_concurrent_cached_hit[4] 385.86599505189645 iter/sec (stddev: 0.00009174472369198083) 644.9401154384699 iter/sec (stddev: 0.00006905929844057822) 1.67
benchmarks/test_guard_concurrency.py::test_g15_concurrent_first_resolve[1] 13088.652607161783 iter/sec (stddev: 0.000008898067487238017) 10367.531586629619 iter/sec (stddev: 0.000011787870112262922) 0.79
benchmarks/test_guard_concurrency.py::test_g15_concurrent_first_resolve[2] 6408.332883407202 iter/sec (stddev: 0.000022504397563323764) 7895.09887395877 iter/sec (stddev: 0.000016271910172012886) 1.23
benchmarks/test_guard_concurrency.py::test_g15_concurrent_first_resolve[4] 3316.459463946121 iter/sec (stddev: 0.00004390571160984311) 2841.7375273641196 iter/sec (stddev: 0.000059076005376613964) 0.86
benchmarks/test_guard_concurrency.py::test_g15b_concurrent_first_resolve_sibling_children[1] 9153.356478614664 iter/sec (stddev: 0.000008724415732086717) 8728.95122525976 iter/sec (stddev: 0.000011740064282683766) 0.95
benchmarks/test_guard_concurrency.py::test_g15b_concurrent_first_resolve_sibling_children[2] 3999.303321367087 iter/sec (stddev: 0.000021139151573911502) 3817.609870811879 iter/sec (stddev: 0.00004637032929711864) 0.95
benchmarks/test_guard_concurrency.py::test_g15b_concurrent_first_resolve_sibling_children[4] 1844.9588237467353 iter/sec (stddev: 0.00003949568377896021) 1832.393641153777 iter/sec (stddev: 0.00007382606875497562) 0.99
benchmarks/test_guard_concurrency.py::test_g15c_worker_pool_floor_control[1] 39404.26012644559 iter/sec (stddev: 0.000005786448913028672) 22290.87812686603 iter/sec (stddev: 0.00001527182704554603) 0.57
benchmarks/test_guard_concurrency.py::test_g15c_worker_pool_floor_control[2] 15566.267742197371 iter/sec (stddev: 0.000009941429818939364) 13493.624093917231 iter/sec (stddev: 0.000006389400203434426) 0.87
benchmarks/test_guard_concurrency.py::test_g15c_worker_pool_floor_control[4] 5723.667077170503 iter/sec (stddev: 0.00003551743821946883) 5097.403739356766 iter/sec (stddev: 0.00004571893196013458) 0.89
benchmarks/test_guard_lifecycle.py::test_g6_build_child_container 1571855.1167650167 iter/sec (stddev: 3.0296281757756405e-8) 2371454.1424857685 iter/sec (stddev: 2.6211170365587407e-8) 1.51
benchmarks/test_guard_lifecycle.py::test_g6b_build_child_container_auto_scope 1659872.4628646974 iter/sec (stddev: 2.830373379092392e-8) 2538845.929497744 iter/sec (stddev: 1.9117327860890257e-8) 1.53
benchmarks/test_guard_lifecycle.py::test_g7_request_lifecycle_batch 2970.1592524156686 iter/sec (stddev: 0.0000123177617375346) 3567.7475644970327 iter/sec (stddev: 0.000013772813876831617) 1.20
benchmarks/test_guard_lifecycle.py::test_g7c_event_loop_floor_control 55120.44668449177 iter/sec (stddev: 0.000004039394043048347) 85610.81182513075 iter/sec (stddev: 0.0000012788754374561593) 1.55
benchmarks/test_guard_lifecycle.py::test_g7b_request_cycle_sync 380784.79937539395 iter/sec (stddev: 1.0216745313813754e-7) 548537.3101780373 iter/sec (stddev: 7.407490454055349e-8) 1.44
benchmarks/test_guard_lifecycle.py::test_g13_teardown_at_scale 51642.51044624719 iter/sec (stddev: 0.0000044979478964133706) 78282.71903368316 iter/sec (stddev: 0.0000015417757250000493) 1.52
benchmarks/test_guard_lifecycle.py::test_g13b_teardown_at_scale_async_no_finalizers 439.74542050807247 iter/sec (stddev: 0.001846188427590344) 676.2998591549633 iter/sec (stddev: 0.0012432667241365112) 1.54
benchmarks/test_guard_resolve.py::test_g1_transient_resolve 2136790.1469494114 iter/sec (stddev: 4.127213406254551e-8) 3228350.8607419287 iter/sec (stddev: 2.4345694374559554e-8) 1.51
benchmarks/test_guard_resolve.py::test_g2_cached_resolve 3376332.959373851 iter/sec (stddev: 2.7890488277849425e-8) 5597120.706795254 iter/sec (stddev: 1.640099525564007e-8) 1.66
benchmarks/test_guard_resolve.py::test_g3_deep_chain 701587.2851058841 iter/sec (stddev: 6.270457149706357e-8) 1034598.8477457863 iter/sec (stddev: 5.307044947317676e-8) 1.47
benchmarks/test_guard_resolve.py::test_g4_wide_resolve 352339.9415216004 iter/sec (stddev: 6.821887875370687e-7) 588705.2540590164 iter/sec (stddev: 4.142006868953795e-7) 1.67
benchmarks/test_guard_resolve.py::test_g5_cross_scope 1427864.6865479506 iter/sec (stddev: 1.6435594812763764e-7) 2886963.74919021 iter/sec (stddev: 2.461600147524541e-8) 2.02
benchmarks/test_guard_resolve.py::test_g9_context_resolve 1358833.7510357772 iter/sec (stddev: 1.8451356675236155e-7) 1914023.0419625533 iter/sec (stddev: 1.4276067933761925e-7) 1.41
benchmarks/test_guard_resolve.py::test_g12_override_active_resolve 684819.2387094403 iter/sec (stddev: 2.535525454447545e-7) 929223.8100579204 iter/sec (stddev: 1.3445317172632747e-7) 1.36
benchmarks/test_guard_resolve.py::test_g18_alias_hop 3568980.8878593915 iter/sec (stddev: 2.8048796158771167e-8) 5418806.553362775 iter/sec (stddev: 1.6285366623237592e-8) 1.52
benchmarks/test_guard_validate.py::test_g10_validate_deep_chain 34912.563579351925 iter/sec (stddev: 0.00002115885427016495) 34407.38193391391 iter/sec (stddev: 0.0002676948715150353) 0.99
benchmarks/test_guard_validate.py::test_g11_validate_wide 20819.726918859877 iter/sec (stddev: 0.000028931664918310425) 25055.079625458882 iter/sec (stddev: 0.000023298813867826017) 1.20

This comment was automatically generated by workflow using github-action-benchmark.

@lesnik512

Copy link
Copy Markdown
Member Author

Dropped: not worth a recipe.

@lesnik512 lesnik512 closed this Oct 10, 2026
@lesnik512
lesnik512 deleted the chore/test-integrations-recipe branch October 10, 2026 19:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant