Skip to content

feat(laparoscopy): assisted segmentation, guided tutorial, per-user label colours - #112

Merged
homarone00 merged 3 commits into
mainfrom
feat/assisted-segmentation
Oct 16, 2026
Merged

homarone00 merged 3 commits into
mainfrom
feat/assisted-segmentation

Conversation

@Morelli-01

Copy link
Copy Markdown
Member

What and why

Assisted segmentation (Assist). In laparoscopy Annotation Mode, Segment by clicking turns positive/negative clicks into a mask from a SurgiSAM2 model running on the DjangoWorker GPU worker. The mask is applied as one undoable edit and can be tracked onto the next N frames (0–50), but only where the label is still empty. Tracked frames are saved through the existing autosaver, so they can't conflict with brush saves.

  • Privacy and access. The browser never talks to the worker. A new WebSocket relay (ws/assist/<domain>/<patient_id>/, common/consumers.py) checks write access and the assisted_segmentation method against the patient's own project. It finds the video from the patient, never from the browser, and opens the worker session itself.
  • No storage credentials on the worker. It gets a 10-minute signed link to that one video (presign_get, the same function the runner uses for cluster jobs).
  • When it shows. Only with the method enabled on the project, ASSIST_WORKER_URL/ASSIST_WORKER_TOKEN set (empty hides it), and write access.
  • Migrations. 0063 enables the method on laparoscopy projects; setup_laparoscopy_modalities does the same for new ones.

Tutorial. A guided tour of quadrant marking and Annotation Mode (driver.js 1.9.0 from jsDelivr, pinned with SRI; its stylesheet is vendored because the CSP style-src is 'self'). CONTRIBUTING's CDN section is updated.

Labels. Adding, renaming and retiring segmentation labels and quadrants is now project-admin only (it was write access). Colours become personal: each user's choice is stored in UserPreference.label_colors (migration 0062) and only they see it. Behaviour change: annotators can no longer manage labels, and a label's shared default colour is fixed once it's created.

Annotation Mode, smaller changes.

  • Ctrl+Z undoes; Ctrl+Y or Ctrl+Shift+Z redoes.
  • The toolbar label dropdown is gone; the label is picked in the Labels card.
  • The timeline ticks for annotated frames are removed.

Deployment. New optional settings ASSIST_WORKER_URL, ASSIST_WORKER_TOKEN and ASSIST_CONNECT_TIMEOUT, documented in .env.example and docs/setup.md §8. The worker must be able to reach this deployment's OBJECT_STORAGE_ENDPOINT_URL and must list its origin in its own VIDEO_SOURCE_ORIGINS.

Impact on other domains (reviewed):

  • Bundle: after normalising the renamed chunk names, every non-laparoscopy viewer entry and every shared chunk is byte-identical; only image-segment.js changed.
  • Label/quadrant APIs: they are also mounted under /maxillo/ (maxillo/app_urls.py), where the rules are now stricter. No maxillo page calls them.
  • UserPreference: gets one new column only (label_colors).
  • The one site-wide change: bs-compat.css gives .btn-group > .btn position: relative and a raised z-index on hover/focus/active, matching Bootstrap's own rule. That covers 9 templates across domains.

How it was checked

  • Tests added or updated for the change
    • common/tests_assist.py: the relay refuses anonymous users, viewers, projects without the method, unconfigured deployments and other files; it frees the worker session on leave; relays worker errors; the worker gets a signed link, never the key.
    • Laparoscopy page flags and label/quadrant permissions.
    • Per-user colours.
    • frontend/tests/segmentAssist.test.js.
  • python manage.py test passes against MySQL: the full suite (1416 tests) passed before the signed-link change; common.tests_assist, laparoscopy and annotation were re-run after it. makemigrations --check is clean.
  • ruff check . and lint-imports pass (no new exceptions in pyproject.toml).
  • npm run build leaves no diff (the bundle is up to date); npm test passes (891); npm run verify is OK.
  • Also checked end to end on a dev stack: a real Assist session went from preparing to ready, and the worker downloaded the video through the signed link.

Core paths

  • This touches a path in .github/CODEOWNERS (common/models.py, common/migrations/0062–0063, yggdrasil/settings.py). A maintainer has signed off: @maintainer
  • New URLs that name an object authorize against that object's own project. The new route is a WebSocket, so the HTTP cross-project matrix doesn't cover it. It authorizes write access on the patient's own project and has its own refusal tests. The cross-project matrix passes.

Data and privacy

  • No real patient data, names or identifiers in code, fixtures, screenshots or this description.

@homarone00 homarone00 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

great work, nice features

@homarone00
homarone00 merged commit 0843c3f into main Oct 16, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants