3-D bubble surfaces from a few camera silhouettes, seen through refractive tank walls
A DNS bubble in homogeneous shear with gravity, starting nearly spherical at t = 96.0. It deforms over the next 2.5 time units as the real rig's four cameras see it. Left: each camera's silhouette with the fitted outline. Middle: the 16-parameter support-function fit. Right: a horizontal cut through the fit (blue), the DNS truth (grey) and the visual hull (orange), with each frame's volume error.
How it works · Results · Quick start · Repository layout · Reports
Cameras record a bubble's outline, not its surface. The classic way to recover it is the visual hull: it keeps every point that all cameras see inside the silhouette. With four cameras, what remains is a faceted body that is always too large, by about 17–18 % in volume on DNS bubbles. This repository holds two reconstructions:
- The production pipeline used on the four-camera water-tank experiments (
visual_hull). It is a Python port of the MATLAB visual hull, followed by surface refinement and a spherical-harmonic surface fit. Bubbles are tracked across frames, their surfaces smoothed in time, and the results shown in interactive HTML viewers. - The support-function fit (
visual_hull.support_function, new in October 2026). It fits a smooth convex body directly to the refractive rays through every outline pixel. It has been validated on DNS bubbles against six published methods. Its own production stages (A-SF to C-SF) handle many bubbles per frame, overlapping silhouettes and tracks; they have been tested on a synthetic eight-bubble movie, not yet on real frames.
| mean volume error on strongly deformed bubbles visual hull: 17 % |
the surface error of the best published method rms 1.9 % of R, against 3.5 % |
error in the oscillation strength σ(v_major) virtual cameras: 1.1–12.5 times too large |
parameters per bubble, about 1 s per fit no voxels, no marching cubes |
Each outline pixel of each camera defines a water-side ray (OpenLPT's refractive lineOfSight), and that ray must
just graze the bubble. Describe the bubble by its support function h(n): the distance from the origin to the
bubble's tangent plane with outward normal n. The gap between a ray (p, d) and the body is then
g is a signed distance in mm and is zero on the outline. Its derivative with respect to the parameters is −∂h/∂θ at the maximising normal (envelope theorem), so a Gauss–Newton fit needs no rendering and no projections. The default body is an ellipsoid plus the seven ℓ = 3 modes in the ellipsoid's own frame. That makes 16 parameters (c: 3, M: 6, b: 7), and four cameras see every one of them.
flowchart LR
M["binary masks<br/>+ OpenLPT cameras"]
subgraph SFIT ["support-function fit"]
O["outline pixels<br/>(sub-pixel)"] --> R["refractive rays<br/>(lineOfSight)"] --> E["ellipsoid<br/>(dual quadric, linear)"] --> GN["Gauss–Newton<br/>on the ray gaps"]
end
subgraph PROD ["production pipeline"]
H["visual hull"] --> RF["surface<br/>refinement"] --> SH["spherical-harmonic<br/>surface fit"] --> TR["tracking +<br/>temporal smoothing"]
end
M --> O
M --> H
GN --> P["volume · area · axes<br/>r_major · surface mesh"]
TR --> V["interactive<br/>HTML viewers"]
The derivation, the symbolic checks (visual_hull/derivations/) and the per-bubble recipe are in
support_function/DESIGN.md and the
theory report.
The test set is 150 DNS bubbles rendered through the real four-camera rig, including its geometry and refraction. Every method gets the same masks. The published methods were re-implemented from their papers. The choices the papers leave open are listed in the comparison report.
HST + gravity, frame 9635, aspect ratio 2.6. ΔV is the volume error on this frame. rms is the rms distance to the true surface divided by the equivalent radius R.
All 201 frames of the window t = 96.5–98.5. r_major is half the bubble's largest width. v_major = dr_major/dt is computed as in clocksim (fourth-order differences). "rms" is the error of r_major relative to the truth, and the error of v_major in units of the truth's standard deviation.
| method | volume error, mean |ΔV| | surface rms / R | σ(v_major) / truth |
|---|---|---|---|
| this work: support function, 16 parameters | 0.71 % / 0.25 % | 1.86 % / 0.37 % | 0.94 / 1.04 |
| this work: support-function ellipsoid, 9 parameters | 0.99 % / 0.24 % | 3.50 % / 0.74 % | 0.94 / 1.04 |
| ellipse per view → ellipsoid (Cross & Zisserman 1998; Rubino et al. 2018) | 1.05 % / 0.23 % | 3.50 % / 0.74 % | 0.94 / 1.04 |
| two-view elliptic slices (Fujiwara et al. 2004; Honkanen 2009; Saito & Toriu 2015) | 2.52 % / 0.32 % | 4.18 % / 1.06 % | 0.73 / 0.99 |
| Bézier virtual cameras (Huang et al. 2025) | 6.75 % / 3.25 % | 4.85 % / 2.10 % | 1.13 / 1.73 |
| virtual cameras (Masuk, Salibindla & Ni 2019) | 7.42 % / 1.77 % | 6.07 % / 2.27 % | 5.26 / 12.5 |
| single-view spheroid (standard practice; mean of the four cameras) | 10.7 % / 4.05 % | — | 0.72 / 1.05 |
| visual hull (Laurentini 1994) | 17.1 % / 18.1 % | 9.22 % / 9.35 % | 0.79 / 1.38 |
Each cell gives two values: HST + gravity, then HST without gravity.
- Volume and surface: the gravity case is 50 strongly deformed bubbles (aspect ratio up to 2.6). The no-gravity case is 100 nearly spherical ones.
- σ(v_major) / truth: every frame of t = 96.5–98.5 with gravity, and of the burst t = 102–107 without. 1 is perfect.
Note
The masks here are noise-free renders of DNS bubbles, so every number is a best case. This work's fit has also been tested against blur, threshold bias, noise, calibration error and overlapping silhouettes (theory report, section V7). The re-implemented published methods have not been tested against these yet.
A synthetic movie: 8 deforming DNS bubbles of 0.7–2.0 mm (35–100 px) over 40 frames, through the four-camera rig. The events are designed in: bubbles stacked in depth, a small bubble hidden behind a large one, a fast crossing, and bubbles entering and leaving the view. Outlines are coloured by track and dashed when flagged.
Each bubble is fitted to its own outline rays, and outlines merged in a camera are split between bubbles by their ray gaps (stage A-SF). Stage B's matcher, with velocity prediction, links the bubbles into tracks (B-SF). The tracks then correct what the detector got wrong (C-SF).
| synthetic movie, 183 fully visible bubble-frames | per frame (A-SF + B-SF) | after the track refit (C-SF) |
|---|---|---|
| bubbles found | 181 | 183 |
| volume error, median / worst | 0.92 % / 10.4 % | 0.88 % / 4.3 % |
| track volume (median of its unflagged frames), median / worst error | 0.09 % / 0.93 % | 0.08 % / 0.93 % |
| identity switches · bubbles split over two tracks | 0 · 1 | 0 · 0 |
The movie and its truth are in demo/: python demo/run_demo.py runs the three stages on it and prints these numbers.
-
The detector caused the large errors. A coarse visual hull finds the bubbles, and it fails in two ways:
- A bubble gets no seed. This happens when the bubble is smaller than about two detector voxels, or has left one camera's image. Its outline is then fitted as part of the bubble it touches: −10 % in volume.
- A bubble gets a seed that is too small. The image edge cuts the bubble, and the overlap split then drops its own outline: −24 %.
Seeds predicted along the tracks fix both: −2.9 % and −0.9 %.
-
Track history fixes the volume, not the shape.
- A bubble's volume is constant along its track, so the track's median volume is ten times more accurate than one frame's.
- The neighbouring frames do not improve r_major, r_minor or the shape: the shape changes between frames by about as much as each frame's error. Even forcing the true volume into the fit leaves the axes where they were.
- The shape is therefore never smoothed in time.
-
Overlaps: the tracks supply the seeds, not the shapes.
- Every overlap failure on the movie came from a missing or undersized seed, and the refit fixes those.
- Starting the split of a merged outline from the shape the track predicts instead of a sphere (
--track-shapes) moved only two of 183 bubble-frames, one closer to the truth and one further away, so it stays off. - Where a camera sees a third of an outline, several shapes fit the rays about equally well.
Details: theory report, "Many bubbles and tracks".
git clone https://github.com/clockj/bubble-shape-reconstruction.git
cd bubble-shape-reconstruction
# conda-forge environment; OpenLPT supplies the refractive camera model (pyopenlpt)
conda env create -f visual_hull/environment.yml
conda activate bubble-visual-hull
pip install -e visual_hull
# demo: the three support-function stages on a synthetic 40-frame movie of eight bubbles, then their errors
# against the movie's truth (about 1 min on 32 cores; writes demo/out/)
python demo/run_demo.py
# unit tests on the same movie: the carved detector, the fit, the matcher (about a minute)
python -m unittest discover -s visual_hull/tests -vProduction pipeline (four refractive cameras, TIFF masks, OpenLPT camera files), run from visual_hull/:
# A. per frame: TIFF masks -> visual hull -> refined surface -> bubble properties -> SH surface fit
python scripts/reconstruct_tiff_data.py --working-dir <data> --frames 0 49 --voxel-size 0.5 0.5 0.5 \
--sh-degree 4 --sh-inscribed --sh-silhouette --max-aspect-ratio 5
# B. tracking + temporal smoothing; every bubble keeps one sh_track_id across frames
python scripts/temporal_smooth_sh.py --frames 0 49 --recon-dir <Results/run> --output-dir <smoothed>
# C. interactive HTML viewers (Plotly from its CDN)
python scripts/visualize_smoothed_html.py --smoothed-dir <smoothed> --working-dir <data> --out <html>Support-function pipeline with tracking, on the same inputs, run from visual_hull/:
# A-SF. every bubble per frame: coarse visual hull (detector only) -> outline rays (OpenLPT) -> 16-parameter fit.
# Silhouettes merged in a camera are split between the bubbles by their ray gaps. The hull is carved coarse to
# fine: the same seeds as projecting every voxel (--detector hull), about 300x faster. --skip-existing resumes.
python scripts/reconstruct_support_function.py --working-dir <data> --frames 0 12604 --max-workers -1 --skip-existing
# B-SF. tracks (stage B's matcher; --predict-velocity matches each bubble from its last velocity), smoothed
# velocity, r_major(t) and v_major -> <data>/Results/recon_sf/sf_tracks.csv and sf_tracks.mat
python scripts/track_support_function.py --recon-dir <data>/Results/recon_sf --fps 20 --predict-velocity
# C-SF. the tracks, then a refit of the frames where they know better than the detector: missed bubbles, seeds cut
# by the image edge, track pieces joined across missed frames -> <data>/Results/recon_sf_refit/
python scripts/refit_tracks_support_function.py --recon-dir <data>/Results/recon_sf --fps 20 --predict-velocityEach frame's Bubble_Frame_<frame>_sf.mat holds every bubble's centroid, volume, r_major, r_minor, axes, the 16
parameters and quality columns. sf_flag marks the bubbles to check; the bits are listed in the script's docstring.
For sizes, use each track's volume (track_volume_median, d_eq_track in sf_tracks.csv), not a single frame's.
Spherical-harmonic fit safeguards and filters
--sh-min-points-per-coeff(default 3) caps each bubble's degree. An undersampled fit otherwise oscillates into negative-radius "petals".--sh-inscribedfits the hull's outer envelope with a one-sided overshoot penalty, so the surface hugs the inside of the hull.--sh-silhouetteadjusts the coefficients to match each bubble's re-projected silhouette in every camera. Mean IoU rose from 0.60 to 0.86 on the reference frame.--size-range MIN MAXkeeps bubbles by equivalent diameter.--max-aspect-ratiodrops elongated visual-hull ghosts: real bubbles have an aspect ratio of about 1.5–3, phantoms above 6.
The four-view visual hull over-estimates by about +6 % in diameter and +19 % in volume (synthetic spheres). Temporal smoothing reduces jitter about threefold but does not remove this bias.
Support-function fit, given the rays through every outline pixel:
from visual_hull.support_function import sfrecon as sf
# P, D: (J, 3) origins and unit directions of the water-side rays through every outline pixel of every
# camera (OpenLPT lineOfSight), with the origins shifted to a seed point c0
# N: contact normals from each contour's tangent (silhouette.tangent_normals, one camera at a time)
th0 = sf.dual_quadric(N, P) # initial ellipsoid, linear in its 9 unknowns (None if degenerate)
th9, *_ = sf.fit(sf.SupportModel(0), th0, P, D) # 9-parameter ellipsoid
model = sf.AffineSupportModel(3)
th, gap, *_ = sf.fit(model, th9, P, D) # 16 parameters; gap = ray-body distance per ray (mm)
shape = sf.shape_summary(model, th) # V, A, centroid (relative to c0), surface mesh X, FDESIGN.md covers the full per-bubble recipe: outline extraction, flagging cameras with a large rms gap, and the
3-camera fallback. It also shows how to import the subpackage without pyopenlpt.
visual_hull/ Python package (pip install -e visual_hull)
├── src/visual_hull/
│ ├── hull.py, refinement.py, properties.py, … visual hull -> surface -> bubble properties (MATLAB port)
│ ├── improved/ soft visual hull, mesh smoothing, spherical-harmonic surface fit
│ ├── support_function/ support-function fit from outline rays; pipeline.py: every bubble in a frame;
│ │ tracks.py: track tables, joining, prediction (DESIGN.md)
│ ├── tracking.py frame-to-frame matching shared by stages B, B-SF and C-SF
│ ├── virtual_camera/ virtual-camera rounding of the four-view hull
│ └── mesh_surface/ free-form deformable-mesh surfaces
├── scripts/ production stages A–C and A-SF, B-SF, C-SF; validation and benchmarks
├── tests/ unit tests on the demo movie
├── derivations/ SymPy checks of the support-function geometry
└── environment.yml conda environment (bubble-visual-hull)
demo/ a synthetic 40-frame movie with its truth; run_demo.py, evaluate.py
report/ reports (Markdown, with self-contained HTML versions)
docs/figures/ the figures on this page
The validation runs are in visual_hull/test/<YYYYMMDD-HHMMSS>-<name>/, each with its scripts, report.md,
figures and small results; Slurm jobs, logs and per-frame outputs are left out. Most of them read DNS bubble
fields, which are not included: open an issue to ask for them. The figures on this page were made in the run 20261024-155928-readme-figures, and the tracking
movie in 20261024-231122-track-history.
| report | contents |
|---|---|
| Bubble reconstruction methods compared (2026-10-24) | this work against six published methods on the same DNS bubbles: shape, volume, axes, r_major / v_major time series and their spectra |
| Compact bubble reconstruction from 3–4 refractive cameras (2026-10-23, revised 2026-10-24) | support-function theory, observability, derivation checks, DNS validation, limits (blur, calibration, overlap), tracer losses; many bubbles in a frame, tracks, and the track refit |
MIT © 2026 clockj