Skip to content

CI on every toolchain, 100% line coverage of the library, and the bugs it found - #15

Merged
ShadobaDev merged 18 commits into
mainfrom
test/coverage-and-ci-improvement
Nov 6, 2026
Merged

ShadobaDev merged 18 commits into
mainfrom
test/coverage-and-ci-improvement

Conversation

@ShadobaDev

Copy link
Copy Markdown
Owner

Summary

CI now builds and tests the project on MSVC, MinGW and Linux, and measures the library's test coverage on Linux. Covering the library found real bugs, fixed here. The coverage job now fails if a line goes uncovered or branches drop below 90%.

Library coverage (lib/src, lib/include, gcovr as CI runs it): lines 100% (3376), functions 100% (342), branches 95.1% (3727/3920). 429 tests pass on Linux, 393 on MinGW (one skipped: a Windows-only demonstration).

Bugs fixed

  • render() with sliceHeight == 0 killed the caller's process. The slice count divided by zero (SIGFPE) before the height was checked. It now returns a typed SlicingFailed.
  • An unpack that filled the disk reported success with the last file cut short: the stream was checked before it was closed. It is closed first now.
  • WorkspacePackager::unpack() lost non-ASCII names in a process left in the "C" locale. It now unpacks in a UTF-8 locale on its own thread.
  • A greyscale or grey+alpha overlay failed the whole render. It is converted to sRGB first, as pages are. A 16-bit overlay is scaled to 8 bits, not clipped.
  • computeFileSha256() returned a hash of part of a file when a read stopped short, and an empty-data hash for a folder. Both now return empty ("cannot be read").
  • Unchecked inputs:
    • MarginCropper refuses a negative margin and computes the safe area in 64 bits.
    • ScaledStrip::append() refuses a page of another width, a page with no image, and a strip taller than an int can count.
    • Scaler bounds targetWidth by VIPS_MAX_COORD.
  • Failed saves left their temporary file behind. Five copies of "write a temporary file, then move it over the destination" are now one function, publishTempFile(), which always cleans up. The serializers also check for an error when closing the file.
  • Grade neutrality compared doubles exactly. isNeutral() and the grade itself now accept a value within k_neutralTolerance (1e-6) of its neutral value. That is too little to move a 16-bit pixel. A slider value left a hair off its default, for example by arithmetic or a JSON round trip, now counts as untouched.

How coverage is held

  • Failure paths: tests/lib-fault-tests/ (Linux only) makes any libvips, libarchive or C library function the library calls fail on demand, or return an image libvips never would. Every error path the library checks is tested this way.
  • Dead code: code that cannot run was deleted, not excluded. Examples:
    • a self-check of a lookup that was already resolved;
    • preset filters on palettes that every setter keeps preset-free;
    • an empty-sink check on a sink that is never empty.
      Code that cannot be reached but has to stay carries a GCOVR_EXCL marker with its reason, and is listed in the wiki's Platform page.
  • Why branches stop at 95%: what remains is what gcc counts and no test can take:
    • cleanup of temporaries on an exception's way out;
    • matching an exception's type in a catch;
    • guards of function-local statics;
    • the default of a switch that covers every enum value;
    • two states only another platform or a race reaches.
      The floor sits at 90% and is only raised.
  • Docs: docs/CODING_STYLE.md says what new library code needs: a test, a fault wrapper, deletion, or a marker. docs/CHEATSHEET.md shows how to measure coverage locally, the same way CI does.

Also

  • CI sends Linux coverage to Coveralls. The README shows CI, coverage, release and licence badges.
  • The Release job takes its notes from the tag's section of docs/CHANGELOG.md.
  • minipsd 0.4.1, whose plugin links into a shared library on Linux.

ShadobaDev and others added 18 commits November 3, 2026 23:44
…x showed

CI builds and tests on MSVC, MinGW and Linux, and sends Linux's
coverage to Coveralls, on pushes to main and on pull requests; the README
carries its badges, the release's and the licences'. The Release job takes
its notes from the tag's section of docs/CHANGELOG.md.

What the first Linux run found:
- WorkspacePackager::unpack() lost non-ASCII names in a process left in the
  "C" locale: libarchive gives names in the current locale. It now unpacks
  with its thread alone in a UTF-8 locale.
- The PSD template tests read the layer as RLE; it has been ZIP since the
  templates switched. The pixels are now read back through psdload.
- minipsd 0.4.1: its plugin links into a shared library on Linux.
- The in-place write race is a Windows demonstration (POSIX: skipped), the
  CLI suite's timeout covers an instrumented build, and the memory test's
  chapter outlasts libvips' operation cache on 8.15 as on 8.18.

Docs: GITHUB-RELEASE, TODO and the wiki describe 0.6.0 as released.
Extend the existing GoogleTest files (no parallel fixtures) to cover
reachable error, boundary and size-limit branches that integration runs
never hit: null/empty inputs, zero/negative/oversize dimensions, bit-depth
full-scale, and the first/last-slice geometry.

Coverage (gcovr --filter lib/ --filter cli/ --exclude-throw-branches):
  before: lines 79.6%  functions 93.7%  branches 61.2% (5334/8715)  272 tests
  after:  lines 80.6%  functions 94.2%  branches 62.0% (5406/8715)  298 tests

Branch coverage rose most in (file: before -> after):
  margin_cropper          22.6% ->  83.9%  (+19)  zero/negative safe area, happy crop
  strip_overlay_compositor 54.0% -> 64.6%  (+17)  all blend modes, RGBA slice,
                                                  opaque-PNG addalpha, 16-bit cast,
                                                  rasterizeOverlayAtWidth probe
  template_generator      74.4% ->  80.6%  (+14)  MINIPSD_MAX_SIDE (width side too),
                                                  zero/negative canvas, margins >
                                                  canvas, non-positive dpi
  scaler                  41.2% ->  52.9%  (+10)  targetWidth<=0 (both overloads),
                                                  invalid buffer, buffer-overload scale
  pixel_buffer            47.4% ->  68.4%   (+4)  release(), clone() (empty + valid)
  scaled_strip            71.7% ->  73.2%   (+4)  grayscale -> sRGB fold (mixed strip)
  colour_corrector        56.7% ->  58.6%   (+4)  invalid-buffer grade, 16-/32-bit
                                                  full-scale brightness

Bugs found: none. Every new test asserts a concrete result (pixels,
dimensions, exception type) and all 298 tests pass.

Still uncovered, with the reason:
- cli/main.cpp and infrastructure/model_json: the bulk of the remaining
  untaken branches are compiler-generated exception-cleanup landing pads and
  nlohmann get/contains paths; flipping them would need malformed input per
  field and would only execute lines, not assert behaviour.
- libvips/libarchive failure paths (e.g. vips_resize/vips_join/vips_embed/
  vips_cast failing mid-pipeline, colour_corrector's failVips, slice_writer
  and workspace_packager I/O errors): not reachable without injecting a
  failing libvips/libarchive.
- scaled_strip buildSlice's "buffer already released" and "join padded the
  slice" guards: internal-invariant assertions unreachable through the public
  API with valid input.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019egXEqmTekGhA8Z974HrBX
Drop --filter cli/ from the gcovr step: the coverage report is meant for
library code. --filter lib/ matches only lib/src and lib/include; the
tests, the fetched dependencies (build/_deps) and cmake helper files stay
outside it, and the repository has no examples directory.

Library-only coverage (gcovr --filter lib/ --exclude-throw-branches):
  before the new tests: lines 86.6%  functions 97.0%  branches 62.9% (3723/5916)
  after:                lines 88.0%  functions 97.5%  branches 64.1% (3795/5916)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019egXEqmTekGhA8Z974HrBX
expectedSliceCount() divided the strip height by outputProfile.sliceHeight
before ScaledStrip::sliceAll() validated it, so render() with
sliceHeight == 0 raised SIGFPE and terminated the caller's process (render's
safety net catches C++ exceptions, not signals). A non-positive height now
counts as no slices, and sliceAll() reports the same typed SlicingFailed
error a negative height already got.

Reproduction: ProcessingPipeline::render() with any readable input and
RenderRequest::outputProfile.sliceHeight = 0 -> "Floating point exception",
exit status 136. Covered by RenderGuardTest.NonPositiveSliceHeightIsATypedFailure.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019egXEqmTekGhA8Z974HrBX
Library-only coverage (gcovr --filter lib/ --exclude-throw-branches):
  before: lines 88.0%  functions  97.5%  branches 64.1%
  after:  lines 95.9%  functions 100.0%  branches 73.0%  (388 tests)

New tests drive real inputs, not fault injection: missing/occupied/blocked
paths, corrupt and hostile zips, files older versions wrote without optional
fields, unknown enum names, libvips-native (.v) images whose pixel format or
coding libvips cannot convert, size limits (resize factor > 1e7, MINIPSD_MAX_SIDE)
and boundaries (cancel on the last full slice, Crop tail, partial renders).

Two bugs are pinned by skipped tests (GTEST_SKIP names the cause):
- a greyscale or grey+alpha overlay fails the whole render
  (SliceOutputTest.GreyscaleOverlayIsComposited);
- rasterizeOverlayRgba() returns an empty raster for a grey+alpha asset
  (StripOverlayCompositorTest.GreyAlphaOverlayIsPromotedToRgba).
SanitizeInputsTest.ErrorInputStaysErrorWhileStillUnreadable skips when file
permissions cannot stop the current user (running as root).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019egXEqmTekGhA8Z974HrBX
What the coverage run found, fixed:
- A greyscale or grey+alpha overlay failed the whole render, and came back
  empty as RGBA: rasterizeOverlay() turns it into sRGB before its alpha,
  as ScaledStrip does with pages; a 16-bit one is scaled to 8 bits, not
  clipped.
- computeFileSha256() hashed part of a file a read cut short, and a folder
  as no bytes; both now come back empty, "cannot be read".
- MarginCropper refuses a negative margin, and computes the safe area in
  64 bits so two huge margins cannot wrap to a positive one.
- ScaledStrip::append() refuses a page of another width, one with no image,
  and a strip taller than an int counts.
- WorkspaceSerializer and ProfileBundleSerializer leave no temp file behind
  after a failed save, and no longer ignore an error on closing it.
- Scaler bounds targetWidth by VIPS_MAX_COORD, in one check for both
  overloads.

CI measures the library only, without the branches gcc gives lines that
hold no code. Tests that assumed libvips 8.15 now hold on 8.18 too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The workspace, profile bundles, packages, output slices and thumbnails
each wrote into a temporary sibling and moved it over the destination,
in five copies; two of them left the temporary file behind on a failure.
publishTempFile() moves it (with an optional copy fallback) and always
cleans up; writeTextFileAtomically() is the two serializers' whole save.
Callers keep their messages and exception types. The copy fallback's
comment now gives its real use: a destination another program holds
writable but not replaceable (Windows); a sibling never crosses devices.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A second test executable, Linux only, defines the libvips, libarchive
and libc functions a test needs to fail. libplatemaker.so binds its
calls to them (the executable exports its symbols), and each passes the
call on unless a test armed it: Fault::failCall("vips_copy", n) fails
the n-th call from now, once. A test whose armed fault never fired
fails, and arming a function with no wrapper throws. One test per kind
of wrapper: a libvips operation, a libvips function that is not one,
a C function.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each libvips call the library checks for failure now fails once in a
test: ColourCorrector, PixelBuffer, ScaledStrip, Scaler, the overlay
compositor, TemplateGenerator, decodePageToRgba and ImageIO. A test
checks what the caller is told (an exception naming the step, or the
documented empty result) and that no libvips object outlives the
failure. Fault::returnImage() hands the library an image libvips never
returns (no width, no alpha) for the guards against that.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
WorkspacePackager::unpack() checked each entry's stream before closing
it, so a write that failed on the last flush (a full disk) went unseen:
the unpack reported success with the file cut short. It closes first now.
Tests for the archive and file-system failures: a package entry that
cannot be written, a package that cannot be opened, every write of an
unpack failing in turn, paths that cannot be resolved (a symlink loop),
a temporary file on /dev/full, and a read that fails mid-hash.

The 0.6.0 changelog gains the fixes of this round that 0.5.1 users would
notice; the cheatsheet shows how to measure coverage locally, as CI does.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Code that could never run is deleted: evalCurve's return after a loop
that always returns, a catch around a hash that no longer throws, key
checks on JSON the serializer has just written itself, and the slice
writer's copy of outputFormatExtension().

Two guards against libvips results are now tested, not excluded: a
slice built at another height than asked, and a grade that comes back
without its alpha band.

What no test can reach carries a GCOVR_EXCL marker with its reason:
returns after exhaustive switches, VIPS_INIT failing, eight identifier
collisions, ScaledStrip's own invariants, and nlohmann's enum macros.
gcovr gains --exclude-noncode-lines in CI and the cheatsheet, dropping
the exception cleanup it counts at a lone closing brace.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Conditions the converted JSON always met are gone: clear() no longer
asks whether the key is there, the templates loop no longer checks
their shape, has_root_path() stands for root name or root directory,
and migrate() sets the version it always set.

Tests now take the other side of the rest: an overlay or template
naming no file, an empty or unnamed target, a token never cancelled,
odd manifest parts, the compression each format gets, keys missing
from a workspace file, a project uid given twice, provenance naming a
page twice or one that is no input; and, by injection, no UTF-8 locale,
an entry name only in the locale's form or in none, and a libarchive
error without a reason.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Delete three checks that can never fail: detectCanvasConfigChange's
lookup of an id effectiveCanvasProfileIds already resolved, the preset
filter in snapshotMeta (every setter strips presets) and the empty-id
guard in replaceCanvasProfiles (every setter mints ids).

Extend the existing tests to the cases they missed: detectStaleness's
output-profile and format axes, a page with one known dimension, a
render that could not measure a page, a rename in place, moving a
project onto itself, Skipped and Pending in inputsAllProcessed,
duplicate output uids, an overlay without a path, a same-width
profile, an untouched id during relink, a project added after a
snapshot and an edited legacy op-<name> profile.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Replace code that can never take its other branch: headerGeometry reads
the orientation through libvips (vips_image_get_orientation and
_swap), applyToRgba's neutral check calls Models::isNeutral, and the
impossible checks go - a zero image side, an empty candidate list, a
released buffer during band normalisation, a null graded image, the
hash loop's break, the serializer's preset filter and an empty log
sink.

A second sliceAll() does reach the released-buffer refusal, so it loses
its exclusion and gains a test.

Fix tests that checked nothing: the case-and-slash path case never
reached samePath (the twin is skipped as identical), and the alpha test
used an opaque alpha, which libwebp drops.

Extend the tests to: one-margin pages, a five-band page, bad decode
sizes, a join failing after the first, generic RGB pages and overlays,
a template with zero margins, an unrecognised template fingerprint,
each staleness axis alone, every curve, a grade returned at another
size or depth, an empty thumbnail, a removed source, self-move of a
PixelBuffer, an output-only repair and orphans without a report.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
isNeutral() and the grade compared brightness, contrast and saturation
with ==, so a slider value a hair off its default (0.1 + 0.2 - 0.3 is
5.6e-17) counted as a grade: the project re-rendered for nothing, and
the grade ran a pass that changed no pixel. Both now compare within
k_neutralTolerance (1e-6), which is under a tenth of one 16-bit level.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The library now covers every line and 95.1% of its branches. The job
holds that with gcovr's floors: lines at 100, branches at 90. The
report still reaches the job summary when a floor fails the job.

CODING_STYLE says what new library code needs: a test, a fault
wrapper, deletion, or a marker with its reason. It also says the
branch floor only goes up.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
gcc 15 counts the exception cleanup of a multi-line returned list at the
line of its first element, and nothing else there, so CI's coverage job
saw that line as never run. Each component is now built on its own, with
code on its first line, and the list is returned in one line.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… the folder

The planner made a relative path absolute against the canonical workspace
folder. MSVC's weakly_canonical expands a short 8.3 name (RUNNER~1 to
runneradmin), so a template named relative to the workspace came out spelled
differently from every other file, and the test failed on the MSVC runner.
The canonical folder now only identifies files; their source path is built
on the folder as given, as load() reads it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@ShadobaDev
ShadobaDev merged commit 6e333db into main Nov 6, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants