Skip to content

ngmix: config-selectable UberSeg neighbour masking (BLEND_HANDLING)#770

Draft
cailmdaley wants to merge 1 commit into
ngmix_v2.0from
ngmix-uberseg
Draft

ngmix: config-selectable UberSeg neighbour masking (BLEND_HANDLING)#770
cailmdaley wants to merge 1 commit into
ngmix_v2.0from
ngmix-uberseg

Conversation

@cailmdaley

@cailmdaley cailmdaley commented Jun 30, 2026

Copy link
Copy Markdown
Contributor

Closes #776.

What

Adds a config-selectable UberSeg neighbour treatment to the ngmix module, alongside the current noise-fill. Additive and default-unchanged: a new option BLEND_HANDLING = {noisefill, uberseg} (default noisefill) selects how a neighbour sharing a galaxy's postage stamp is handled.

  • noisefill (default) — the historical behaviour, byte-for-byte unchanged: flagged pixels are replaced by a noise realisation and kept at inverse-variance weight.
  • uberseg — hard-masks (weight → 0) every stamp pixel closer to a neighbour's segmentation footprint than to the central object's, leaving the image untouched.

Why

The 2026-06-23 tomography call flagged the current noise-fill as suspect: including blended (FLAGS=2) objects produces B-modes, while discarding them is a strong, non-metacalibratable selection bias. UberSeg (Sheldon's MEDS/ngmix treatment) is the principled alternative the group has long wanted. The deliverable is the option plus the A/B evidence, so the production default can be chosen on data, not priors — this PR is the option; the validation and the default flip are downstream (see #769-adjacent discussion and the linked issue).

How

  • uberseg_weight(weight, seg, object_number) reimplements Erin Sheldon's meds._uberseg.uberseg_tree nearest-segment Voronoi partition with a scipy cKDTree. Reimplemented, not depended on: the astronomy MEDS is esheldon/meds (the PyPI meds is an unrelated package), and import meds drags the full fitsio/esutil stack for a single C tree. The surviving central core is a single connected, roughly circular region — emergent geometry of the partition, faithful to MEDS get_uberseg (no separate aperture).
  • Plumbed ngmix_runnerNgmixdo_ngmix_metacalmake_ngmix_observationprepare_ngmix_weights; Postage_stamp gains a per-epoch segs list. Unknown BLEND_HANDLING fails fast at construction.
  • Tests (tests/module/test_ngmix_uberseg.py): synthetic two-object stamp asserts the mask geometry (neighbour-side zeroed, central core single-connected, matches brute-force nearest-segment); the noisefill default is byte-identical to the legacy noise-fill on a shared RNG; uberseg leaves the image untouched while zeroing neighbour/flagged weight. Existing test_ngmix / test_ngmix_weight_validation pass unchanged.

Not in scope (deliberately)

  • The segmentation-map source is gated — see the linked issue. UberSeg needs the seg map with object IDs at each stamp; ShapePipe carries only a per-epoch binary flag today, and the seg map with NUMBERs is the coadd's while ngmix fits per-epoch stamps (MEDS interpolates the coadd seg per-epoch). The two options (extend vignetmaker to cut + reproject a coadd SEGMENTATION vignet, vs approximate from the binary flag) are posed there for the experts. This PR implements everything not gated; selecting uberseg without a seg map raises with a pointer to the issue. This PR is therefore not production-complete until the seg source is decided and wired.
  • Image treatment under metacal — per the design intent (and Axel's "leave neighbour objects on the image"), uberseg zeroes the weight without noise-filling the image. metacal shears in Fourier space using all pixels regardless of weight, so residual neighbour flux can leak through the shear; the noise-fill-vs-uberseg A/B B-mode comparison is what measures this, and Axel's weight-symmetrisation follow-on (out of scope) is what would address it. Fabian validates the science before any default flip.
  • No change to the noise-fill path, the production default, or the get_noise-vs-sigma_mad noise-estimate question (ngmix: the reduced-χ² anomaly on stars (error model) #769).

Part of the ngmix v2.0 epic (#762).

— Claude, on behalf of Cail

A new ngmix-module option BLEND_HANDLING = {noisefill, uberseg} selects how
a neighbour sharing a galaxy's stamp is treated. The default `noisefill` is
byte-for-byte the historical behaviour (flagged pixels replaced by a noise
realisation, kept at inverse-variance weight); `uberseg` instead hard-masks
(weight -> 0) every pixel closer to a neighbour's segmentation footprint than
to the central object's, leaving the image untouched.

`uberseg_weight()` reimplements Erin Sheldon's MEDS uberseg (the
`meds._uberseg.uberseg_tree` nearest-segment Voronoi partition) with a
scipy cKDTree, rather than depending on `meds` whose import drags the full
fitsio/esutil stack. The surviving central core is a single connected,
roughly circular region — emergent geometry of the partition, not a separate
aperture, faithful to MEDS `get_uberseg`.

Plumbed runner -> Ngmix -> do_ngmix_metacal -> make_ngmix_observation ->
prepare_ngmix_weights; Postage_stamp gains a per-epoch `segs` list. The
segmentation-map *source* is not wired here (it is plumbing-gated: the seg
map is the coadd's, ngmix fits per-epoch stamps); selecting `uberseg`
without a seg map raises with a pointer to the gating issue.

Tests: synthetic two-object stamp asserts the mask geometry (neighbour-side
zeroed, central core single-connected, matches brute-force nearest-segment),
the noisefill default is byte-identical to the legacy noise-fill, and uberseg
leaves the image untouched while zeroing neighbour/flagged weight.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018DLfuwQzavQQ9GbcKZQPc5
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

UberSeg neighbour masking (Erin Sheldon) — config-selectable, validated against noise-fill

1 participant