Repository navigation
Add code duplication analysis to the dev workflow - #450
yarikoptic wants to merge 13 commits into
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #450 +/- ##
==========================================
- Coverage 92.41% 92.26% -0.15%
==========================================
Files 15 15
Lines 1344 1344
Branches 182 182
==========================================
- Hits 1242 1240 -2
- Misses 68 69 +1
- Partials 34 35 +1 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
A critical test assertion is stale, and CI/tooling depend on unpinned or implicit environments.
Review effort: Lite
Findings: 1
Open (2)
What changed in this PR
Adds zero-threshold jscpd duplication checks to tox and GitHub Actions, with refactoring of duplicated tests.
Changes:
- Adds jscpd configuration and checking tooling.
- Integrates duplication analysis into tox and CI.
- Refactors repetitive tests into parametrized tests and shared helpers.
| File | Summary | Review notes |
|---|---|---|
tox.ini |
Adds the duplication tox environment. | — |
tools/check-duplication.sh |
Runs jscpd analysis. | Pin @latest to a tested version (moderate, 3 votes). |
test/test_plot.py |
Extracts shared plotting helpers. | — |
test/test_ls.py |
Extracts shared prefix parsing helpers. | — |
test/duct_main/test_report.py |
Consolidates report test setup. | — |
test/duct_main/test_prepare_outputs.py |
Parametrizes repetitive output tests. | — |
test/duct_main/test_aggregation.py |
Extracts aggregation test helpers. | Refresh current_sample after updates (critical, 1 vote). |
.jscpd.json |
Configures duplication detection and exclusions. | — |
.gitignore |
Ignores temporary analysis files. | — |
.github/workflows/test.yaml |
Runs duplication checks in CI. | Provision Node.js explicitly (moderate, 1 vote). |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
|
@asmacdo please review and potentially accept this one since it is likely to breed conflicts very easily due to massive pass through the tests |
Add .jscpd.json (threshold 4%, matching the current measured baseline in test/) and a tox env backed by tools/check-duplication.sh, so duplication checking becomes a standing part of `tox`. CHANGELOG.md is excluded from scanning since its duplication is auto-generated release-note boilerplate (intuit/auto), not code to refactor. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Add the `duplication` toxenv to the existing include-matrix job in test.yaml, alongside lint/typing/minimal. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Replace 16 near-identical capture/output combination tests with a single test parametrized over itertools.product(Outputs, Outputs), using Outputs.has_stdout/has_stderr to compute expected behavior per case. Removes the jscpd-flagged duplication in this file. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Extract _make_report, _current_sample, _assert_current_matches_full_run_averages, and _report_with_three_samples to remove the repeated Report-construction and averages-assertion boilerplate jscpd flagged across several test functions. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Merge test_averages_two_samples/test_averages_three_samples into one test parametrized over the number of updates, and extract _make_process_stats to share the ProcessStats construction between the green/red parametrized cases. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Replace six repeated "parse Prefix: lines out of ls() output" blocks with a shared _extract_prefixes staticmethod. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Extract a _plot_args staticmethod for the argparse.Namespace (usage.json, min_ratio=3.0, cpu=ps-pcpu) that the plotting tests rebuilt inline: the two output="outfile.png" sanity tests and all the backend-selection tests that plot with no output file, including those added on main by the interactive-backend probing work (#449). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01D9HgYZ2Q66KTvBD4cLZeEZ
The mitigation commits above brought the repo to 0% duplication; enforce zero-tolerance going forward. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
- tools/check-duplication.sh: pin jscpd to 5.3.3 instead of @latest, so a new jscpd release can't change parsing/config or fail CI on an otherwise-unchanged commit (Copilot review). - test_aggregation.py: re-fetch current_sample via _current_sample() after the extra update_from_sample() calls in test_aggregation_averages, instead of relying on Sample.aggregate() happening to carry the same Averages object forward across reassignment. Correct either way today, but the previous form relied on an implementation detail a future refactor of aggregate() could break silently (Copilot review). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
test_aggregation_single_sample_sanity compared each average to itself (`averages.rss == averages.rss`), so it could never fail -- carried over verbatim from main by the dedup refactor. Compare against the totals as the comment says, sharing an _assert_averages_match_totals() helper with test_aggregation_single_stat_multiple_samples_sanity, which already did the intended check. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01D9HgYZ2Q66KTvBD4cLZeEZ
jscpd is run via npx, so the job silently depended on whatever Node the runner image happens to ship. Set it up explicitly (jscpd needs >= 18), as Copilot's review overview suggested. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01D9HgYZ2Q66KTvBD4cLZeEZ
a6516b2 to
fdb6a92
Compare
The parametrized test computed what to expect via Outputs.has_stdout / has_stderr -- the same predicates prepare_outputs branches on -- so a bug in them moved expected and actual together and nothing failed (e.g. has_stdout() true for every value but NONE passed the whole suite; the 16 original hard-coded tests caught it in 7 cases). Spell out which Outputs enable each stream instead. Also assert TailPipe.start() is called once per tee'd stream: dropping stdout.start() went unnoticed both before and after the refactor. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01D9HgYZ2Q66KTvBD4cLZeEZ
The branch that swaps a kernel-abbreviated "[name]" cmd for the full command line had no dedicated test; it was hit only when a live process sampled during e2e runs happened to show bracketed, so codecov/project flapped by ~0.2% between otherwise identical runs. Test it directly. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01D9HgYZ2Q66KTvBD4cLZeEZ
|
Ill review and ill probably cause conflicts with my other tests, which ill fixup here |
|
I'll re-run the failed job once the workflow run finishes; GitHub refuses a re-run while the run is still going. The sampling and signal-handling e2e tests depend on timing, and making them robust on slow macOS runners would be a separate change. Generated by Claude Code |
|
Update on CI for edba47e: the macOS 3.14 job passed on re-run, so every test job is now green.
A while ago the same kind of drift hit Generated by Claude Code |


Add code duplication analysis (jscpd) to the development workflow. It should help keeping my alergy under control! ;)
Changes
.jscpd.json(threshold 0%), ignoring caches/build artifacts andCHANGELOG.md(its "duplication" is auto-generated release-noteboilerplate from intuit/auto, not code to refactor)
tox -e duplicationcheck backed bytools/check-duplication.shduplicationtoxenv to CI (.github/workflows/test.yaml)was in
test/, none insrc/):test_prepare_outputs.py: 16 near-identical tests → 1 parametrized testtest_aggregation.py: extracted sharedReport-construction/assertion helperstest_report.py: parametrized/extracted shared setuptest_ls.py,test_plot.py: extracted small shared helpersTesting
✅
tox -e duplicationpasses at 0% duplication✅ Full test suite passes (471 tests)
✅
tox -e lint/tox -e typingpass on the touched files🤖 Generated with Claude Code using about to be submitted
/introduce-duplicates-analysisCON/skill