ngmix: config-selectable UberSeg neighbour masking (BLEND_HANDLING)#770
Draft
cailmdaley wants to merge 1 commit into
Draft
ngmix: config-selectable UberSeg neighbour masking (BLEND_HANDLING)#770cailmdaley wants to merge 1 commit into
cailmdaley wants to merge 1 commit into
Conversation
A new ngmix-module option BLEND_HANDLING = {noisefill, uberseg} selects how
a neighbour sharing a galaxy's stamp is treated. The default `noisefill` is
byte-for-byte the historical behaviour (flagged pixels replaced by a noise
realisation, kept at inverse-variance weight); `uberseg` instead hard-masks
(weight -> 0) every pixel closer to a neighbour's segmentation footprint than
to the central object's, leaving the image untouched.
`uberseg_weight()` reimplements Erin Sheldon's MEDS uberseg (the
`meds._uberseg.uberseg_tree` nearest-segment Voronoi partition) with a
scipy cKDTree, rather than depending on `meds` whose import drags the full
fitsio/esutil stack. The surviving central core is a single connected,
roughly circular region — emergent geometry of the partition, not a separate
aperture, faithful to MEDS `get_uberseg`.
Plumbed runner -> Ngmix -> do_ngmix_metacal -> make_ngmix_observation ->
prepare_ngmix_weights; Postage_stamp gains a per-epoch `segs` list. The
segmentation-map *source* is not wired here (it is plumbing-gated: the seg
map is the coadd's, ngmix fits per-epoch stamps); selecting `uberseg`
without a seg map raises with a pointer to the gating issue.
Tests: synthetic two-object stamp asserts the mask geometry (neighbour-side
zeroed, central core single-connected, matches brute-force nearest-segment),
the noisefill default is byte-identical to the legacy noise-fill, and uberseg
leaves the image untouched while zeroing neighbour/flagged weight.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018DLfuwQzavQQ9GbcKZQPc5
19fd652 to
0534daf
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #776.
What
Adds a config-selectable UberSeg neighbour treatment to the ngmix module, alongside the current noise-fill. Additive and default-unchanged: a new option
BLEND_HANDLING = {noisefill, uberseg}(defaultnoisefill) selects how a neighbour sharing a galaxy's postage stamp is handled.noisefill(default) — the historical behaviour, byte-for-byte unchanged: flagged pixels are replaced by a noise realisation and kept at inverse-variance weight.uberseg— hard-masks (weight → 0) every stamp pixel closer to a neighbour's segmentation footprint than to the central object's, leaving the image untouched.Why
The 2026-06-23 tomography call flagged the current noise-fill as suspect: including blended (
FLAGS=2) objects produces B-modes, while discarding them is a strong, non-metacalibratable selection bias. UberSeg (Sheldon's MEDS/ngmix treatment) is the principled alternative the group has long wanted. The deliverable is the option plus the A/B evidence, so the production default can be chosen on data, not priors — this PR is the option; the validation and the default flip are downstream (see #769-adjacent discussion and the linked issue).How
uberseg_weight(weight, seg, object_number)reimplements Erin Sheldon'smeds._uberseg.uberseg_treenearest-segment Voronoi partition with a scipycKDTree. Reimplemented, not depended on: the astronomy MEDS isesheldon/meds(the PyPImedsis an unrelated package), andimport medsdrags the fullfitsio/esutilstack for a single C tree. The surviving central core is a single connected, roughly circular region — emergent geometry of the partition, faithful to MEDSget_uberseg(no separate aperture).ngmix_runner→Ngmix→do_ngmix_metacal→make_ngmix_observation→prepare_ngmix_weights;Postage_stampgains a per-epochsegslist. UnknownBLEND_HANDLINGfails fast at construction.tests/module/test_ngmix_uberseg.py): synthetic two-object stamp asserts the mask geometry (neighbour-side zeroed, central core single-connected, matches brute-force nearest-segment); thenoisefilldefault is byte-identical to the legacy noise-fill on a shared RNG;ubersegleaves the image untouched while zeroing neighbour/flagged weight. Existingtest_ngmix/test_ngmix_weight_validationpass unchanged.Not in scope (deliberately)
vignetmakerto cut + reproject a coaddSEGMENTATIONvignet, vs approximate from the binary flag) are posed there for the experts. This PR implements everything not gated; selectingubersegwithout a seg map raises with a pointer to the issue. This PR is therefore not production-complete until the seg source is decided and wired.ubersegzeroes the weight without noise-filling the image. metacal shears in Fourier space using all pixels regardless of weight, so residual neighbour flux can leak through the shear; the noise-fill-vs-uberseg A/B B-mode comparison is what measures this, and Axel's weight-symmetrisation follow-on (out of scope) is what would address it. Fabian validates the science before any default flip.get_noise-vs-sigma_madnoise-estimate question (ngmix: the reduced-χ² anomaly on stars (error model) #769).Part of the ngmix v2.0 epic (#762).
— Claude, on behalf of Cail