From 2ee6c3bc4418857822e66b965b84a3633c9b9f62 Mon Sep 17 00:00:00 2001 From: Vasu Jaganath Date: Fri, 9 Oct 2026 16:44:56 -0400 Subject: [PATCH] fix: move the image assembler and precompute slide adapters to the image-tools images The image-assembler-plugin 1.4.0-dev0 and precompute-slide-plugin 1.7.0-dev0 images pin preadator 0.3.0.dev1. Its worker initializer reads each worker's name with a zero-timeout get on a shared queue, so a worker that loses the queue's lock to a sibling gets queue.Empty and the process pool breaks. Under load this failed 10/30 assembler and 12/30 precompute runs on the BBBC001 data. image-assembler-tool 1.4.3 uses the stdlib ProcessPoolExecutor and no preadator: 0/150 under the same load. precompute-slide-tool 1.7.3-dev0 uses preadator 0.4.0.dev2, which still has the zero-timeout get in a nested ProcessManager, so it fails far less often (2/90) but not never. 1.7.2 cannot be used: it fails on the deflate-compressed TIFF the new assembler writes (ADOBE_DEFLATE needs imagecodecs). The new precompute-slide's imageType values are Intensity and Segmentation instead of image and segmentation, so the bbbc workflows pass Intensity. The doc and manifest links point at the image-tools repository the tools moved to. --- cwl_adapters/image_assembler.cwl | 6 +++--- cwl_adapters/precompute_slide.cwl | 8 ++++---- workflows/bbbc.py | 2 +- workflows/bbbc.wic | 2 +- 4 files changed, 9 insertions(+), 9 deletions(-) diff --git a/cwl_adapters/image_assembler.cwl b/cwl_adapters/image_assembler.cwl index f6d179f..1b4b1f0 100644 --- a/cwl_adapters/image_assembler.cwl +++ b/cwl_adapters/image_assembler.cwl @@ -6,11 +6,11 @@ label: Image Assembler doc: |- This plugin assembles images into a stitched image using an image stitching vector. - https://github.com/PolusAI/polus-plugins/tree/master/transforms/images/image-assembler-plugin + https://github.com/NovagenResearch/image-tools/tree/master/transforms/images/image-assembler-tool requirements: DockerRequirement: - dockerPull: polusai/image-assembler-plugin:1.4.0-dev0 + dockerPull: polusai/image-assembler-tool:1.4.3 # See https://www.commonwl.org/v1.0/CommandLineTool.html#InitialWorkDirRequirement InitialWorkDirRequirement: listing: @@ -94,4 +94,4 @@ $namespaces: $schemas: - https://raw.githubusercontent.com/edamontology/edamontology/master/EDAM_dev.owl -# manifest: https://raw.githubusercontent.com/PolusAI/polus-plugins/master/transforms/images/image-assembler-plugin/plugin.json \ No newline at end of file +# manifest: https://raw.githubusercontent.com/NovagenResearch/image-tools/master/transforms/images/image-assembler-tool/plugin.json \ No newline at end of file diff --git a/cwl_adapters/precompute_slide.cwl b/cwl_adapters/precompute_slide.cwl index 2419170..c90ad4d 100644 --- a/cwl_adapters/precompute_slide.cwl +++ b/cwl_adapters/precompute_slide.cwl @@ -6,11 +6,11 @@ label: Precompute Slide doc: |- This plugin generates image pyramids in multiple viewing formats. - https://github.com/PolusAI/polus-plugins/tree/master/visualization/polus-precompute-slide-plugin + https://github.com/NovagenResearch/image-tools/tree/master/visualization/precompute-slide-tool requirements: DockerRequirement: - dockerPull: polusai/precompute-slide-plugin:1.7.0-dev0 + dockerPull: polusai/precompute-slide-tool:1.7.3-dev0 # See https://www.commonwl.org/v1.0/CommandLineTool.html#InitialWorkDirRequirement InitialWorkDirRequirement: listing: @@ -36,9 +36,9 @@ inputs: prefix: --pyramidType imageType: - label: Image is either Segmentation or Image + label: Image is either Intensity or Segmentation doc: |- - Image is either Segmentation or Image + Image is either Intensity or Segmentation inputBinding: prefix: --imageType type: string diff --git a/workflows/bbbc.py b/workflows/bbbc.py index dd7a31e..f33f3c4 100644 --- a/workflows/bbbc.py +++ b/workflows/bbbc.py @@ -39,7 +39,7 @@ def workflow() -> Workflow: precompute_slide = Step(clt_path='cwl_adapters/precompute_slide.cwl') precompute_slide.inpDir = image_assembler.outDir precompute_slide.pyramidType = 'Zarr' - precompute_slide.imageType = 'image' + precompute_slide.imageType = 'Intensity' precompute_slide.outDir = Path('precompute_slide.outDir') steps = [bbbcdownload, diff --git a/workflows/bbbc.wic b/workflows/bbbc.wic index 67cde4b..4a032e2 100644 --- a/workflows/bbbc.wic +++ b/workflows/bbbc.wic @@ -43,7 +43,7 @@ steps: - outDir: !& image_assembler.outDir precompute_slide: in: - imageType: !ii image + imageType: !ii Intensity inpDir: !* image_assembler.outDir outDir: !ii precompute_slide.outDir pyramidType: !ii Zarr