feat(ba): activate robust kernel infrastructure + add Tukey and GNC-TLS - #74
Merged
heiwang1997 merged 2 commits intoMay 11, 2026
Merged
Conversation
…Term The `RobustKernel` / `HuberRobustKernel` classes exist in `vipe/slam/ba/kernel.py` and the `Solver` already knows how to apply a per-term kernel via `add_term(term, kernel=...)` + `ConcreteTermEvalReturn.apply_robust_kernel`, but no caller ever passes a kernel: `GraphBuffer.bundle_adjustment` adds its `DenseDepthFlowTerm` instances without one, so the IRLS machinery is effectively dead code. This commit finishes the wiring: * `vipe/slam/ba/kernel.py`: `HuberRobustKernel.__init__` now takes a `threshold` argument (previously the cutoff was hard-coded to 1, which is too small for DenseDepthFlowTerm's /8 feature-grid residuals). Add a `build_robust_kernel(name, threshold)` factory so callers can opt in via config without importing kernel classes directly. * `vipe/slam/components/buffer.py`: `GraphBuffer.bundle_adjustment` reads `ba.robust_kernel` / `ba.robust_kernel_threshold` from the slam config, builds the kernel, and passes it to both `DenseDepthFlowTerm` `solver.add_term` calls (the main dense-flow term and, if enabled, the sparse-tracks term, which share the same class). * `configs/slam/default.yaml`: expose the two new `ba.*` keys with `robust_kernel: null` default, so the pre-PR L2 bundle adjustment is reproduced bit-for-bit. No behaviour change on existing configs. Turning it on (`ba.robust_kernel: huber`) enables Huber IRLS reweighting of the dense-flow residual, which down-weights dynamic pixels and flow-mismatch outliers. Made-with: Cursor
…er schedule
Extends the robust kernel machinery activated in the previous commit
with two additional kernels commonly used in robust BA:
* `TukeyRobustKernel(c)`: biweight loss, `w(r) = (1 - (r/c)^2)^2` for
`|r| <= c` and 0 otherwise. Harsher than Huber (fully rejects rather
than linearly down-weighting extreme residuals).
* `TLSGncRobustKernel(c_bar, mu_step, mu_init, mu_max)`: Graduated
Non-Convexity with Truncated Least Squares, following
Yang, Antonante, Tzoumas, Carlone, "Graduated Non-Convexity for
Robust Spatial Perception: From Non-Minimal Solvers to Global
Outlier Rejection", ICRA/RAL 2020,
and GTSAM's `GncOptimizer.h`
https://github.com/borglab/gtsam/blob/develop/gtsam/nonlinear/GncOptimizer.h
The GNC mu schedule is driven externally (`set_mu` / `update_mu`) rather
than advanced inside `apply`, so that the BA loop can hold mu fixed
across a small Gauss-Newton sub-problem at each level. `RobustKernel`
grows a minimal set of GNC hooks (`is_gnc` / `set_mu` / `update_mu` /
`mu_max`) with no-op defaults; plain IRLS kernels (Huber / Tukey) ignore
them transparently.
`GraphBuffer.bundle_adjustment` now picks between two BA loop structures
based on `robust_kernel.is_gnc()`:
* Non-GNC (L2 / Huber / Tukey): unchanged flat `n_iters` GN loop.
* GNC: nested `mu_outer(n_mu_steps) x GN_inner(n_iters // n_mu_steps)`
with mu frozen inside each inner loop. `ba.gnc_n_mu_steps` (default
4) controls how many mu levels split the `n_iters` BA budget.
All new knobs live under `ba.*` in `configs/slam/default.yaml` and
default to values that reproduce the pre-commit-1 L2 behaviour when
`robust_kernel == null`.
Made-with: Cursor
heiwang1997
self-requested a review
May 11, 2026 21:43
Collaborator
|
Looks good to me, thanks for the contribution @qiaozhijian. I will be keeping the robust kernel by default |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Activate robust kernel in BA; add Tukey and GNC-TLS
Summary
VIPE's bundle-adjustment solver already supports per-term robust kernels
(
Solver.add_term(term, kernel=...)+ConcreteTermEvalReturn.apply_robust_kernel)and ships a
HuberRobustKernel, but no caller ever passes one — the IRLSmachinery is effectively unused. This PR:
ba.robust_kernel/ba.robust_kernel_thresholdfrom the SLAMconfig into
GraphBuffer.bundle_adjustmentsoDenseDepthFlowTerm(dense flow and, if enabled, sparse tracks) receives the kernel.
HuberRobustKernel.threshold(was hard-coded to 1,which is too tight for the /8 feature-grid residuals).
TukeyRobustKernel(c)— biweight, hard-rejects|r| > c.TLSGncRobustKernel(c_bar, mu_init, mu_step, mu_max)— GraduatedNon-Convexity with Truncated Least Squares
(Yang-Antonante-Tzoumas-Carlone, RA-L 2020;
weights / schedule follow GTSAM's
GncOptimizer.h).The BA loop gains a μ-outer / GN-inner nesting: each μ level runs
n_iters // gnc_n_mu_stepsGauss-Newton iterations at frozen μ,then μ advances multiplicatively toward the hard-TLS limit.
Default
robust_kernel: nullreproduces the pre-PR L2 behaviour bit-for-bit.Changes
Two atomic commits:
feat(ba): parameterize HuberRobustKernel and wire into DenseDepthFlowTerm— activates the existing kernel infra with Huber only. (+76 / −8)
feat(ba): add Tukey and GNC-TLS robust kernels with mu-outer / GN-inner schedule— adds the two kernels and the GNC schedule loop. (+196 / −10)
No changes to
solver.py,terms.py, or any consumer outsideGraphBuffer.bundle_adjustment. No new dependencies.Benchmark
Protocol: 5 datasets × 2 clips × 201 frames @ 10 Hz (20 s each) = 10 clips.
Config:
configs/slam/default.yamldefaults (keyframe_depth=metric3d-small,backend_iters=24), varying onlyba.robust_kernel. For GNC-TLS the BAloop runs
gnc_n_mu_steps × gnc_gn_iters_per_mu = 4 × 6 = 24GN iterationsper call (matches the default
backend_iters=24used here).Metrics (lower-is-better except mAA):
ATE_se3: SE(3)-aligned absolute trajectory RMSE normalised by GT pathlength. Alignment leaves scale unchanged, so this exposes monocular
scale drift.
RTE/RRE°: median relative translation / rotation error acrossstrides {10, 50, 100} frames.
mAA@τ: AUC of consecutive-frame rotation accuracy under τ°.Overall (mean over 10 clips)
Relative vs L2
Per-dataset (2 clips each)
Takeaways:
never worse than L2 by more than noise floor on any dataset.
ATE_se3 −21 %, GrandTour ATE_se3 −28 %. They hurt on low-parallax
static outdoor (KITTI ATE_se3 +169 % for Tukey) where aggressive
outlier rejection discards valid long-tail residuals. Recommended as
opt-in for domains known to contain dynamic content.
Kernel
applycosts < 1 % of BA iteration time; the 0–6 % ms/framevariation is within per-clip noise.
Backward compatibility
ba.robust_kernel: null(default) →build_robust_kernelreturnsNone→
Solver.run_inplaceskipsapply_robust_kernel→ identical linearsystem as before.