"Run tests at cursor" takes exponentially longer with more pytest tests

[farmio]

Type: Bug

When using Python pytest to parametrize tests, the test explorer takes exponentially longer, while CPU is at ~100%, the more tests are created using parametize when using "Run tests at cursor" to start the tests (starting from the "Run tests" button in test explorer works fine).

Here is a minimal example:

import pytest


@pytest.mark.parametrize("value", range(10))
def test_test(value):
    assert True

With range(10) my system (Apple M1 Pro 16 GB) takes ~1 second (measured by hand with stopwatch).
With range(100) it takes ~8 seconds
With range(150) it takes ~50 seconds

I'd expect it to take roughly as long as running it from test explorer or pytest <file> from command line - which is about 0.1 seconds (<1 second for range(1500)).

VS Code version: Code 1.76.2 (Universal) (ee2b180d582a7f601fa6ecfdad8d9fd269ab1884, 2023-03-14T17:54:09.061Z)
OS version: Darwin arm64 22.3.0
Modes:
Sandboxed: No

System Info

Item	Value
CPUs	Apple M1 Pro (10 x 24)
GPU Status	2d_canvas: enabled canvas_oop_rasterization: disabled_off direct_rendering_display_compositor: disabled_off_ok gpu_compositing: enabled metal: disabled_off multiple_raster_threads: enabled_on opengl: enabled_on rasterization: enabled raw_draw: disabled_off_ok skia_renderer: enabled_on video_decode: enabled video_encode: enabled vulkan: disabled_off webgl: enabled webgl2: enabled webgpu: disabled_off
Load (avg)	5, 43, 74
Memory (System)	16.00GB (0.07GB free)
Process Argv	--crash-reporter-id 3f533b57-abc7-4565-8781-b3146cb3a6b8
Screen Reader	no
VM	0%

Extensions (33)

Extension	Author (truncated)	Version
vscode-gcode-syntax	app	0.7.7
better-toml	bun	0.3.2
vscode-markdownlint	Dav	0.49.0
vscode-eslint	dba	2.4.0
EditorConfig	Edi	0.16.4
elm-ls-vscode	elm	2.6.0
json-tools	eri	1.0.2
prettier-vscode	esb	9.10.4
copilot	Git	1.78.9758
vscode-test-explorer	hbe	2.21.1
flux	inf	1.0.4
elixir-ls	Jak	0.13.0
vscode-edit-csv	jan	0.7.4
vscode-home-assistant	kee	1.35.0
vscode-sshfs	Kel	1.26.0
vscode-docker	ms-	1.24.0
vscode-language-pack-de	MS-	1.76.2023030809
flake8	ms-	2023.4.0
isort	ms-	2022.8.0
python	ms-	2023.4.1
vscode-pylance	ms-	2023.3.40
jupyter	ms-	2023.2.1200692131
jupyter-keymap	ms-	1.1.0
jupyter-renderers	ms-	1.0.15
vscode-jupyter-cell-tags	ms-	0.1.8
vscode-jupyter-slideshow	ms-	0.1.5
remote-containers	ms-	0.288.0
test-adapter-converter	ms-	0.1.7
vscode-typescript-tslint-plugin	ms-	1.3.4
vscode-commons	red	0.0.6
vscode-yaml	red	1.12.2
rust-analyzer	rus	0.3.1451
markdown-all-in-one	yzh	3.5.1

(6 theme extensions excluded)

A/B Experiments

vsliv368:30146709
vsreu685:30147344
python383:30185418
vspor879:30202332
vspor708:30202333
vspor363:30204092
vswsl492cf:30256860
vslsvsres303:30308271
vserr242cf:30382550
pythontb:30283811
vsjup518:30340749
pythonptprofiler:30281270
vshan820:30294714
vstes263:30335439
vscoreces:30445986
pythondataviewer:30285071
vscod805:30301674
binariesv615:30325510
bridge0708:30335490
bridge0723:30353136
cmake_vspar411:30581797
vsaa593cf:30376535
pythonvs932:30410667
cppdebug:30492333
vsclangdc:30486549
c4g48928:30535728
dsvsc012cf:30540253
pynewext54:30695312
azure-dev_surveyone:30548225
vsccc:30610678
nodejswelcome1:30587005
3biah626:30602489
pyind779:30671433
f6dab269:30613381
pythonsymbol12:30671437
2i9eh265:30646982
6233i204:30672705
vsccsb:30677849
pythonb192:30669360
functionswalk:30687959
pythonms35:30701012

[vscodenpa]

Thanks for creating this issue! It looks like you may be using an old version of VS Code, the latest stable release is 1.77.0. Please try upgrading to the latest version and checking whether this issue remains.

Happy Coding!

[farmio]

I just did the update to 1.77.0 and the issue persists.

Version: 1.77.0 (Universal)
Commit: 7f329fe6c66b0f86ae1574c2911b681ad5a45d63
Date: 2023-03-29T09:57:11.797Z
Electron: 19.1.11
Chromium: 102.0.5005.196
Node.js: 16.14.2
V8: 10.2.154.26-electron.0
OS: Darwin arm64 22.3.0
Sandboxed: No

[connor4312]

This takes a while for me, but I think this is on the Python side. Looking at the Output window, they run one process per parameterized "variant" (not sure of the correct term.) So that's 150 processes for range(150). VS Code core is fine.

[connor4312]

oh, actually, this is because we "include" all of them individually. Hmm. We could probably optimize that in VS Code. I'll put in a change there.