Merge in Prerelease - #65
Open
g7gpr wants to merge 78 commits into
Open
Conversation
When the watchdog force-kills a hung BufferedCapture (SIGKILL on bc.pid only), its RawFrameSaver child is orphaned and loops forever on a shared exit Event that is never set, holding its inherited ~450MB frame buffer. Across watchdog restarts on a multicam box this accumulated hundreds of orphaned processes (291 vs ~35 baseline) -> ~14 GiB anon -> OOM. Arm PR_SET_PDEATHSIG=SIGKILL in RawFrameSaver.run() so an orphaned saver is killed the instant its parent dies, with a getppid()==1 race guard. Adds Misc.setParentDeathSignal.
releaseResources() calls pipeline.set_state(Gst.State.NULL), which blocks indefinitely when rtspsrc is wedged on a connected-but-unresponsive camera (RTSP TEARDOWN never completes). That froze the capture loop's heartbeat for 180s until the watchdog force-killed the process - the event that orphans RawFrameSaver. Wrap both set_state(NULL) calls in runWithTimeout (GST_TEARDOWN_TIMEOUT=10s). A stuck teardown is abandoned (pipeline left to GC) so capture keeps its heartbeat and is never force-killed for a hang.
Two compatibility gaps in the PDEATHSIG fix: - setParentDeathSignal default was signal.SIGKILL, evaluated at import - absent on Windows, breaking Misc.py import there. Use numeric 9. - Under the forkserver start method (Python 3.14's new Linux default) the OS parent is the fork-server, not BufferedCapture, so PR_SET_PDEATHSIG fires on the wrong death. Capture the logical parent PID in RawFrameSaver.__init__ (runs in the parent under fork/spawn/forkserver alike) and self-terminate in the idle loop if it dies. This is start-method- and platform-agnostic; PDEATHSIG remains the instant fast-path under fork/spawn.
Add Kht.pyxbld so the KHT wrapper can rebuild via pyximport
Fix compile args
Auto levels are now turned on when SkyFit2 starts, so images appear properly stretched immediately instead of using the full bit range. The histogram region is no longer locked while auto levels are active: dragging the region (or Ctrl+clicking to set a level) switches to manual mode, keeping the auto-determined levels as the starting point. Ctrl+A still toggles auto levels on and off. Full-range setLevels calls on station change and state load now only apply in manual mode so they don't override the auto stretch. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
img_zoom.changeHandle() reloads the zoom image via setImage() without levels, so pyqtgraph assigns it independent min/max levels. With auto levels on, the histogram was updated before the zoom reload and never re-emitted, leaving the zoom window stretched differently from the main image. The manual path had the same latent issue: setLevels() with an unchanged full range early-returns in setRegion() without emitting. Explicitly reapply the histogram levels to both image items after the reload, matching the existing pattern in updateImage handling. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Promote the exception handlers around night product generation in Reprocess (calibration report, observation summary, timelapse, flat, shower association, FOV KML, flux files, field sums, FT archiving, config audit, timestamp plot) from log.debug to log.warning so failures reach the uploaded night logs instead of being silently swallowed. Route ConfigReader validation warnings (FPS clamp, FOV check, jpg/png ranges, binning factor/method, band ratios, upload disabled on the default station code) through logging instead of print(), so they also land in the night logs with a level and timestamp. Add an on-screen key legend to FRbinViewer (toggled with 'h') - the keys were previously only documented in the --help epilog. Document the runnable RMS/ pipeline modules in Utils/README.md, which previously only cataloged the Utils/ scripts. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
ArchiveDetections, UploadManager, DownloadPlatepar, and CaptureModeSwitcher have hardcoded test stubs in their __main__ blocks rather than user CLIs, so following the README could trigger unintended test file/network operations. Mark them as internal instead. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…arden prctl helper - createGstreamDevice cleanup set_state(NULL) now time-bounded (blocking finding) - os.kill(pid, 0) probe gated to POSIX, catches ProcessLookupError only - setParentDeathSignal: document thread-death and subreaper semantics, resolve libc via find_library (musl), log instead of silent failure - Replace local _timedCall with runWithTimeout; log late teardown unwinds
RMS/ClearSkyDetector.py was swept into PR #762 unannounced. It is a standalone work-in-progress tool: nothing in RMS imports it, and it cannot currently be imported since it depends on RMS.ConfigReader.findConfigInDir, which does not exist in this repo. Development continues out of tree.
…-default Enable auto levels by default in SkyFit2
bugfix for #929 malformed kml file in some cases
…amesaver-oom Fix OOM caused by orphaned RawFrameSaver processes after watchdog force-kills
…tector Remove ClearSkyDetector, accidentally included in a SkyFit refactor PR
Reject a recalibration fit that matched fewer than a minimum fraction of the detected stars (default 0.6, configurable via recalibration_min_match_fraction). checkFitGoodness only inspects the residuals of the stars that did match, so a sparse spurious subset at a wrong pointing could pass, get stamped auto_recalibrated, and chain forward as the seed for the next FF - walking the pointing away over a cloudy night. The gate applies even with force_platepar_save, so the last-resort coarse-radius fallback can no longer save such fits either. The gate and the photometry fit now use the matched stars from the last successful fit iteration instead of whatever the last matchStarsResiduals call produced - a failed tighter-radius iteration used to overwrite matched_stars with matches of a platepar that was never accepted (and could leave it unbound under force_platepar_save). SkyFit2's manual pointing-only recalibration disables the gate since the result is user-inspected.
Comment out stale rmem_max/wmem_max lines in /etc/sysctl.conf that load after the drop-in at boot and override it.
…age-gate Add star coverage gate to per-FF recalibration
Make UDP buffer sizes persist across reboots
pyqtgraph <= 0.11 HistogramLUTItem.setImageItem() always passes its getLookupTable method to ImageItem.setLookupTable(), and newer versions still do so for non-grayscale gradients. _composeDisplayLut() assumed an array and crashed with IndexError on np.asarray(callable). Resolve a callable base LUT at render time, as pyqtgraph does, and compose gamma/inversion on its result.
…e-lut Fix SkyFit2 crash on older pyqtgraph (callable LUT)
The comments added with the catalog LM fix were far longer than anything else in SkyFit2 and cited file:line numbers that will go stale. Cut them down to the usual one to three lines. Also drop the claim that the post-save sync stores "the value actually written": config_catalog_lm is unrounded, while the file holds one decimal. The 0.05 tolerance in isConfigModified() covers the difference. The R and F keys change cat_lim_mag too, but did not refresh the save state, so the Save Config button stayed disabled and the File Manager button stayed grey after a keyboard LM change. Refresh it there as well. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…nfig-offset # Conflicts: # Utils/SkyFit2.py
Saving the config from SkyFit2 wrote [Calibration] catalog_mag_limit, which it had no business touching. The LM is a working value in the tuner, not something the user asked to persist. Worse, it was written as cat_lim_mag - 1.0, so it ratcheted: a config with LM 6.5 was loaded into the LM spinbox, saved back as 5.5, loaded as 5.5 on the next start, saved as 4.5, and so on down to the 3.0 floor. Every open and save cycle cost recalibration another magnitude. Drop the Calibration section from the surgical config writer, so only the [StarExtraction] keys the Save Config button advertises are touched. The LM the tuner finds stays a session value, as before. This also removes the modified/save-cycle plumbing for the LM, which only existed to support writing it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The catalog limiting magnitude serves two roles in SkyFit2. It is a viewing knob, nudged with the spinboxes and the R/F keys to show more or fewer stars, and it is the result of Phase 3 of the star detection tuner, which finds the LM that best matches the confirmed detections. Saving the config wrote the knob, so glancing at fainter stars and hitting save silently changed the LM that the nightly recalibration matches at. Write the tuned value instead, and only that. It is offered in the save dialog as a checkbox, on by default, showing the old and the new value. Unchecking it leaves the config LM alone. The dialog now lists every value the save will write as old -> new, so the star extraction parameters are no longer applied sight unseen either. The tuned LM also counts as an unsaved change on its own, so a tune that only moves the LM still enables the Save Config button. The working LM never does, which is what keeps the value from drifting across repeated save cycles. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The star detection tuner optimizes for interactive work, where a few hundred stars are plenty, but the nightly processing needs a deeper list. Saving the tuned value could therefore leave a station worse off than the shipped default of 800. Floor the saved value at MIN_CONFIG_MAX_STARS. The working value in the GUI is left alone, so the detection view does not change under the user. The save dialog says when the floor, rather than the tuning, set the value that gets written. The modified check compares the floored value as well, so a config that already holds 800 is not reported as modified forever by a smaller working value. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The preview was a stack of sentences, which made it hard to see at a glance which values a save would actually change. Put it in a grid of parameter, current value and new value, and bold the new value wherever it differs. Values that stay the same are shown once, without an arrow. The catalog LM checkbox is now the parameter cell of its own row, so it lines up with the rest instead of trailing its values in the label. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…g-offset Save only the tuned catalog LM, and show what a config save writes
…d-artifacts Ignore Cython-generated C sources by pattern and QueuedPool backup pickles
…lity Make night product failures visible and improve tool discoverability
The previous version held F_scale in the parameter vector but missed two sites on the NN/RANSAC path, where Nelder-Mead (unlike LM) drifts the flat coordinate: - the final "apply best RANSAC params" extraction persisted the drifted best_res.x[3] into self.F_scale; - the recursive final matched-pair fit was called without forwarding fixed_scale, so it re-fit the scale freely. Now every extraction, scoring temporary, warm-start temporary and the iteration-callback copy reads the held f_scale_fixed instead of the parameter, and fixed_scale is forwarded through the recursive fitAstrometry call. Since the residual functions already make the objective independent of the scale parameter, the drifting coordinate has no effect on the result. Verified on AU000A (radial9-odd) synthetic recovery for BOTH cost paths: use_nn_cost=False: fixed_scale=False recovers (0.000 px), True holds use_nn_cost=True : fixed_scale=False recovers (0.001 px), True holds (no drift) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The radial fixed_scale fix gated every NN/RANSAC extraction and residual site, but left the two matched-pair (use_nn_cost=False) extractions bare: the fwd-rev loop and the final joint refinement wrote self.F_scale from the fitted vector. They held the seed only because the patched residual zeroes the F_scale Jacobian column and least_squares leaves that coordinate put -- an implicit reliance on optimizer null-space behavior, on the primary path the "Fixed scale" GUI checkbox actually exercises (fitPickedStars -> use_nn_cost=False). Gate both with the same `f_scale_fixed if fixed_scale else abs(...)` pattern used everywhere else so the hold is explicit. No behavior change: verified on a radial7-odd synthetic recovery -- fixed_scale=True holds the seed, fixed_scale=False recovers truth at 0.000 px. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Honor fixed_scale in the radial fitAstrometry paths
tryQuickAlignment and getInitialParamsAstrometryNet save and restore self.fixed_scale around their work, but never modify it in between and never forward it into their fitAstrometry(use_nn_cost=True) calls -- so the pair is dead code that reads like unfinished "Fixed scale" wiring. Not forwarding it is correct, not a bug: both methods (and the AutoPlatepar NN callers) fit from scratch with first_platepar_fit=True, i.e. initial acquisition, where the scale must be free to be determined. Pinning it there would degrade acquisition when the box is checked. The "Fixed scale" checkbox is honored where it belongs -- refinement via fitPickedStars, which already passes fixed_scale=self.fixed_scale. Remove the dangling save/restore so the code matches the intent. No behavior change (fixed_scale was never used in these blocks). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…estore Remove dead user_fixed_scale save/restore in SkyFit2 acquisition paths
Handing each extractor its own ~236 MB snapshot copy per block (added to avoid a refill race under forkserver/spawn) had no concurrency bound: on noisy nights slow extractions piled up, each pinning a copy, and the OS OOM-killed capture mid-night on memory-limited Raspberry Pis. Revert to the original design where the extractor reads the LIVE ping-pong buffer, so memory stays fixed at the two existing frame buffers. Pass the mp.Array base (not a numpy view) so it crosses to the child by handle under fork and forkserver/spawn alike instead of being pickled by value; the extractor rebuilds its view via .get_obj() since the live base is a locked Array. With two buffers the grace window is about one block (less in practice, since the handshake is released and the FF/JPG saved before the extractor starts), so a rare extraction that outruns it lets capture overwrite the block mid-read and corrupts that one FR clip - the accepted trade-off for bounded memory; FF files and detection are unaffected. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
NumPy 2.0 removed the implicit conversion of a size-1 array to a Python scalar via int()/float(). np.fromfile(..., count=1) returns a length-1 array, so int(np.fromfile(...)) raises "only 0-dimensional arrays can be converted to Python scalars" on NumPy >= 2.0. This surfaced when Trixie (Python 3.13) pulled numpy>=2.3, breaking FRbinViewer via FRbin.read. Index the size-1 array before converting (int(np.fromfile(...)[0])), matching the form already used at FRbin.py:176 and in StarCatalog.py. Applied to all 39 affected sites across the binary readers: Vid.py, FFbin.py, FTfile.py, FieldIntensities.py, and FRbin.py. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…-fix Fix mid-night OOM (fixed-memory live-buffer extractor) and NumPy 2.x binary readers (#959)
checkWhiteRatio() aborts all line finding when too many pixels pass the detection threshold, with only a debug-level line that reaches the nightly log file but never the console. On stations where the video path smooths per-pixel temporal noise (e.g. software re-encoded RTSP streams), stdpixel collapses while the maxpixel noise floor stays, the k1*stdpixel + j1 threshold lands below that floor, and every image of a clear night gets rejected - the station is effectively blind to meteors while the console shows nothing but 'detected meteors: 0'. Promote the first rejection (and every 100th) to a warning with the measured ratio and a hint that k1/j1 may be too low for the camera, so affected operators can notice and correct the thresholds. The rejection counter is per worker process, so it is used only to rate limit the warning and is not printed as a run-wide total.
Warn when the white-ratio check silently skips meteor detection
Cython 3.3.0 compatibility
…candidates Add star candidate and PSF-fit visualization
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.