Repository navigation
Tuning keys and a picture stage (vignette, colour, sharpen) - #13
m0d16l14n1 wants to merge 2 commits into
Conversation
22c338b to
68b1882
Compare
|
Rebased onto current
Assisted-by: Claude Opus 5.5 |
|
Blockers: 1. Checked the new keys and their range checks in config.c, the CPU/GPU parity of
HW row: J815 with this package on the default config, then with Assisted-by: LLM (Claude:Opus 5.5) |
68b1882 to
f6cdfdd
Compare
|
@RyanTheTide fixed.
Rebased onto main 03cd679, and the title is updated. In a GPU-less arm64 container, Assisted-by: LLM (Claude:Opus 5.5) |
|
Re-review of f6cdfdd: No blockers. From the last review:
I diffed the old head 68b1882 against f6cdfdd: apart from the dropped board section, the tree is identical. The strength/spatial/temporal-frames keys and the picture stage are the code I already reviewed, so I have nothing new on them. HW row: J815 with this package on the default config, then with Assisted-by: LLM (Claude:Opus 5.5) |
DjDeveloperr
left a comment
There was a problem hiding this comment.
Adversarial review at head f6cdfdd7affdcf353a0424d3fdba4288e2257da2: Request changes (1 ABI-safety blocker + owed GPU/HW evidence on current head).
Blockers
- Plugin ABI break without version bump (
src/backend.h:21,35-60,96,src/gpu/denoise_gpu.c:395-402,414,479): This PR extendsstruct cdn_frame_infofrom 16 bytes to 24 bytes (appendingtuned,temporal_frames, andspatial_q8) and has thedlopen-loadedcdn-backend-gpu.soplugin dereference those new fields inparameters()viacdn_info_spatial_q8(info)andcdn_info_age_max(info), while keepingCDN_BACKEND_ABI == 1andCDN_DMABUF_ABI == 1. Notesrc/backend.hexplicitly promises that ABI v1 keeps the original ops/frame layouts intact (andtests/test_loader.ctests ABI gating across plugins). If an ABI v1 daemon/caller passing the original 16-bytestruct cdn_frame_infoloads the newcdn-backend-gpu.so,ops->abi == CDN_BACKEND_ABIsucceeds andcdn-backend-gpu.soperforms an out-of-bounds stack read pastpts_ns. Fix: either bumpCDN_BACKEND_ABI(andCDN_DMABUF_ABI) so mismatched binaries are cleanly rejected, or preserve the ABI v1struct cdn_frame_infolayout and deliver tuning via an optional companion symbol (matching the existingCDN_DMABUF_ENTRY/CDN_QUERY_ENTRYpattern). - GPU and J815 hardware evidence still owed on
f6cdfdd: On the current headf6cdfdd,test_gpuwas skipped (GPU-less container) and the owed J815 hardware row (default config,strength=0.5 spatial=0.5 temporal-frames=4, andvignette=0.5 sharpen=0.5reloaded live; CPU/GPU bit-exactness on a recorded clip; per-frame timing within 30 fps budget with the new CPUpicture.cpass; 0 dropped frames in 5 min; 0 AVC / 0 failed units / 0 new kernel errors;cdn_config_printmatchingsensor:imx958-uw) has not been posted yet.
Checked & non-blocking notes
- Compiled & tested locally (aarch64,
-Wall -Wextra -Werror):test_picture(ok),test_cpu(PASS, tuned 9.47 dB vs default 18.30 dB),test_config(PASS),src/main.c, and SPIR-V compilation of bothsrc/gpu/filter.compandsrc/gpu/noise.compall succeeded cleanly. CPU/NEON/GPU fixed-point arithmetic parity forspatial_q8andage_max, and the 3-row ring buffer insrc/picture.c, are sound. - Attribution & model gating: Clean. Both commits (
ef0ef81a,f6cdfdd7) carry humanSigned-off-byonly, PR body ends withAssisted-by, and[apple,j815]was properly dropped in favor of[sensor:imx958-uw]. - Non-blocking nits:
src/picture.c:160: vignette luma scaling hardcodes limited-range black level 16 (16 + (((v - 16) * g[x]) >> 12)), which darkens corner pixels< 16on full-range NV12.tests/test_config.c: worth adding parser/range-rejection tests for the 10 new keys insrc/config.c:88-98.
Assisted-by: LLM (Google:Gemini 4 Neon)
The default filter is the strongest it can be: the filtered picture only, the full 3x3 smoothing after the temporal stage and an 8-frame running mean. On the J815 that read as soft, with smear on motion. Three [denoise] keys, live on reload: - strength 0..1: the filtered picture mixed with the camera's (1 = filtered only, the default; 0.5 = half the grain comes back); - spatial 0..1: scales the 3x3 smoothing (0 = none, 1 = default); - temporal-frames 2..8: caps the running mean (8 = default). The CPU and GPU backends take them through cdn_frame_info and stay bit-exact with each other (test_gpu); test_cpu checks that a lighter setting keeps more of the input. Out-of-range values are refused like the other keys. The plugins read the new struct cdn_frame_info fields, so CDN_BACKEND_ABI and CDN_DMABUF_ABI move to 2: a plugin and a caller built for different layouts now refuse each other instead of reading past the end of an older caller's info. The loader test's bad fixture is a plugin built for ABI 1. Signed-off-by: Nick <nekreshen@yandex.ru>
…th, tint, sharpen A last pass over each output frame, after the denoiser and also on the frames it passes through: - vignette 0..1: lens-shading compensation, gain 1 + v * r^2 (r = 1 in the corners) on luma and on the chroma's distance from neutral, for cameras whose ISP tuning leaves the corners dark; - brightness -1..1, contrast 0.5..2 (around mid grey): one luma LUT; - saturation 0..2, warmth -1..1 (Cb/Cr toward orange or blue), tint -1..1 (toward magenta or green); - sharpen 0..1: a 3x3 unsharp mask on luma. All neutral by default, and the pass is skipped then. The gain maps and the LUT are rebuilt only when the size or the settings change. On the CPU, NV12 in place; test_picture covers neutral, the vignette's centre, corners and symmetry, each control's direction and an edge for sharpen. Signed-off-by: Nick <nekreshen@yandex.ru>
f6cdfdd to
f78464c
Compare
|
@DjDeveloperr thanks, the ABI break was real. Fixed in
Vignette nit: the luma path assumes limited range, which is what the ISP delivers here. I've left it as it is, and will make it range-aware if a full-range source appears. The GPU and J815 hardware row on this head is still owed (the set in the description). #16 carries older copies of these two commits and takes the bump when it is rebased. Assisted-by: LLM (Claude:Opus 5.5) |
|
Re-review at f78464c. Nothing new from me. Both of my earlier points are still resolved: the two commits carry only your HW row, still owed: J815 on f78464c with the default config, then with Assisted-by: LLM (Claude:Opus 5.5) |
|
Closing this: I'm taking the J815 camera work out of the upstream trees for now and continuing it on my own machine. Thanks for the reviews. Assisted-by: LLM (Claude:Opus 5.5) |
Two commits for the MacBook Air 15" (M5, J815), each usable on any board:
strength / spatial / temporal-frames. The default filter is as strong as it gets: the filtered picture only, full 3x3 smoothing and an 8-frame mean. On the J815 that looked soft, with smear on motion. These keys scale each part, live on reload:
test_gpu);test_cpuchecks that a lighter setting keeps more of the input.Picture stage. A last pass after the denoiser, also on frames it passes through:
vignette: lens-shading compensation, for an ISP tuning that leaves the corners dark;brightness,contrast,saturation,warmth,tint;sharpen: a 3x3 unsharp mask.Everything is neutral by default, and the pass is skipped then.
test_picturecovers each control.Changed on this head (review):
[apple,j815]board section is dropped. The J815 kernel reportsimx958-uw, so[sensor:imx958-uw]from config, main, data: key camera profiles on sysfs sensor name ([sensor:imx958-uw]) #14 already covers it, and a board-keyed copy would only mask a sensor-keyed change.main03cd679.Signed-off-byand no agent trailer.Changed on this head (DjDeveloperr's review):
CDN_BACKEND_ABIandCDN_DMABUF_ABIare now 2. The plugins read the three newstruct cdn_frame_infofields, so an ABI 1 caller's shorter info would have been read past its end. A plugin and a caller built for different layouts now refuse each other. The loader test's bad fixture is a plugin built for ABI 1 (CDN_BACKEND_ABI - 1). The daemon and the plugins ship in one package, so nothing installed changes. Note for Camera effects on the Neural Engine: Portrait blur, Studio Light, Center Stage #16, which carries older copies of these two commits: its rebase takes the bump.test_configchecks the 10 new keys: accepted values, and out-of-range or non-numeric ones refused.Tested
meson teston this head (Arch Linux ARM container,-Dwerror=true, no GPU): 11 OK,gpuskipped;loader,config,pictureandcpupass.gpu-dmabuffails there the same way onmain(10 OK, the same skip and the same failure): the container has no Vulkan device.CDN_TEST_REQUIRE_GPU=1: 12 OK, 1 skipped.Still owed (HW row): J815 with this package on the default config, then with
strength=0.5 spatial=0.5 temporal-frames=4and withvignette=0.5 sharpen=0.5, each set and reloaded. Pass: the stream runs, the GPU and CPU backends stay bit-identical on a recorded clip, per-frame time stays inside the 30 fps budget, 0 dropped frames in 5 min, and 0 new journal errors.All defaults are unchanged, so nothing changes for existing boards unless a key is set.
Part of the J815 bring-up; board results are summarised in aurora-silicon/linux discussion #135.
Assisted-by: LLM (Claude:Opus 5.5)