Repository navigation
minipsd 0.4.0: the libvips plugin, and layers without transparency - #13
Merged
Merged
Conversation
…7.3, 1) libvips-plugin/ holds psdload, psdload_buffer and psdload_source, laid out as libvips' own loaders, with minipsd compiled in privately (MINIPSD_STATIC). minipsd_vips_init() registers them after VIPS_INIT(): libvips' prebuilt Windows builds load no modules. The header is read from the mapped file: size, bands, format, interpretation, resolution, the ICC profile, EXIF, XMP, IPTC and psd-* fields; pixels come next. libvips' fetch moves to cmake/minipsd_libvips.cmake, shared with examples/vips_composite.c; MINIPSD_VIPS_PLUGIN builds the plugin and its test, vips.plugin, which CI's msvc job runs.
psdload's pixels: vips_image_generate, then vips_sequential, as libvips' own spngload; each band is one minipsd_decode_composite straight into the region, from libvips' threads at once on the one reader. Bitmap comes as B_W (black 0, white 255); bands stop at MINIPSD_PLANES. vips.plugin compares every pixel with minipsd's own decode: documents of 301 x 203 in RGB 8 and 16 (PSD and PSB), grey 16, CMYK 8 and bitmap, loaded randomly, sequentially, from a buffer and from a source, then tests/data, tests/fixtures and psd-tools' set when configured (318 of 319 the same, one truncated file refused by both).
layer=N loads one layer (bottom first; -1, the default, the composite): its rectangle, placed by xoffset and yoffset, with psd-layer-name, -blend-mode, -opacity and -hidden; every image says psd-layer-count. An empty layer is refused. ZIP layers inflate through the zlib libvips links (zlib 1.3.1's headers fetched and pinned on Windows), each thread keeping its channels inflated between bands. Indexed comes through its palette as sRGB, RGBA with a transparent index (1047), as libvips' own loaders give palettes; bits-per-sample is the file's depth. vips.plugin checks each document's layers against the pixels it gave, ZIP and ZIP-predicted ones among them, indexed against its palette, and every layer of the listed files against minipsd's decode.
psdsave, psdsave_buffer and psdsave_target write an image as one layer that is also the composite: grey, RGB or CMYK at 8 or 16 bits, grey and RGB as floats, alpha kept or made opaque; the resolution, ICC profile, EXIF, XMP, IPTC and pixel aspect ratio kept. Options: psb (also a file named .psb, read from the target libvips writes files through), zip and layer_name. The image is held whole while it is written; streaming is S7.5. VipsForeignSaveable (8.17) and VipsSaveable are both handled. vips.plugin saves each kind as a file, into a buffer and into a target, the same bytes each time, and reads it back through psdload: the layer as the image, the composite too where alpha is full (floats within 1e-6); ZIP, PSB, the layer name, the profile, and six bands refused.
GIMP takes the composite's first extra channel for transparency only by
a negative layer count or the alpha name "Transparency", and reads no
'Mt*' block, so a 32-bit document's transparency (its layers in the
writer's 'Lr32', the count positive) showed as an extra channel. Where
the count cannot say it, the writer now names the channel as
Photoshop's transparentbg.psd does: 1006 "Transparency", 1045 the same,
1053 id 0, unless the document keeps names of its own. A document with
no layers also gets an empty 'Mtrn' ('Mt16', 'Mt32'), which the reader
already honours: its transparency was lost before. Files with layers at
8 and 16 bits are written byte for byte as before.
…4, 5) CMYK reached libvips as PSD stores it, the ink inverted; libvips holds 0 as no ink (as tiffload gives it, as jpegload and ImageMagick make it), so psdload inverts the four colour channels and psdsave inverts them back. MINIPSD_VIPS_PLUGIN moves to the top CMakeLists, so the plugin builds without the tests; a program may bring its own libvips as the targets minipsd_libvips and minipsd_libvips_zlib. vips.plugin also reads a 1024 x 30000 PSD sequentially, libvips holding a few rows' worth, and checks the saver's CMYK in the file without the plugin. examples/vips_composite.c builds on Linux (three casts). MANUAL, SPECIFICATION, CODE-ARCH, CHEATSHEET, README and the changelog describe the plugin; the ROADMAP ticks S7.2 to S7.4.
minipsd_vips.c had grown to about a thousand lines. The loader's classes go to minipsd_vips_psdload.c, the saver's to minipsd_vips_psdsave.c, each with a header declaring its types; what both share (constants, PSD_ARGS_*, the suffixes, class_properties, invert_cmyk) to minipsd_vips_util.c/.h, the helpers extern as minipsd_vips__*. minipsd_vips.c keeps only minipsd_vips_init. Each side compiles in only its half of minipsd: the loader with MINIPSD_NO_WRITER, the saver with MINIPSD_NO_READER. Functions and types take the plugin's prefix: minipsd_vips_psdload_* and minipsd_vips_psdsave_*, statics included, and the GTypes MinipsdVipsPsdload* and MinipsdVipsPsdsave*, so neither the type names nor the get_type symbols can meet a loader libvips may add itself. The operations keep their nicknames, psdload and psdsave.
Version 0.3.2 in minipsd.h, the changelog section dated, COMPATIBILITY.md regenerated (no change but the version), the README's release line. INSTALL.md's examples take 0.3.2 and say how a program adds the libvips plugin (MINIPSD_VIPS_PLUGIN, minipsd::vips, its own libvips targets); the manual says sequential loading's memory is a few bands per thread, growing with the width, as measured against magickload. The roadmap ticks S4.11 and adds S4.12, this release.
composite_alpha now says, for every composite (supplied, composed from the layers, or stored), whether it has transparency; 0 writes the mode's colour channels only, the layer count positive, no transparency names or 'Mtrn', so an image without layers is flat, as Photoshop and psd-tools save one. A composed composite without it is matted against white. minipsd_layer.no_alpha writes a layer without a transparency channel, its pixels the colour channels only; the composer takes it as opaque. minipsd_read sets both from the file: such a layer and such a composite decode without transparency, so a round trip keeps them. BREAKING: a program that supplies or composes a composite with transparency sets composite_alpha = 1. The libvips plugin: psdsave writes an image without alpha flat (the option layered: one layer without transparency) instead of adding an opaque band; psdload gives a layer without its channel no alpha band.
minipsd's writer, keeping nothing between pieces, encodes a layer row three times (its channel's size, its own size, the row) and builds and mattes a composite row five times. psdsave now encodes each channel once, into its own buffer, and hands them to minipsd as stored channels and composite_stored, which it copies: an 8000 x 8000 RGBA image saves in 0.7 s instead of 2.4, the file byte for byte the same; without alpha, flat, in 0.3 s. minipsd gains the two pieces that needs, the writer's own: minipsd_encode_row, a channel's row as the file stores it, PackBits or raw; and minipsd_matte_row, a composite's row matted against white as the writer mattes it (RGB; every other mode as given). The writer itself is unchanged. The plugin's test compares every file psdsave writes with what minipsd_write makes from the pixels: the same bytes.
…ption minipsd: MINIPSD_COMPRESSION_DEFAULT is 0 and the other constants move up one (RLE 1, RAW 2, ZIP 3, ZIP_PREDICTED 4), so that zero means the default everywhere: a document's RLE, a layer's the document's. minipsd_layer.compression gives a layer's channels, its mask's too, a compression of their own; the writer asks for the deflate when any layer deflates. The composite is unchanged: raw in a raw document, else RLE. BREAKING: the constants' numbers change; code that names them compiles as it was. The libvips plugin: psdsave's compression (rle, zip, none; the enum MinipsdVipsCompression in minipsd_vips.h, by nick in option strings) replaces zip; none writes the composite raw too. Raw is measured no faster than RLE now that each channel is encoded once.
…w a PSD cannot count The reader refuses a channel with a side above either format's before sizing its plane: a 2^31 x 2^31 mask at 32 bits made a plane of 2^64 bytes, which wrapped to 0 and passed. minipsd_read bounds a mask's sides as a layer's. The writer refuses (MINIPSD_ERR_SIZE) a layer's or the composite's RLE row above 65535 bytes in a PSD, where it sizes rows: 32-bit rows wider than about 16 380 px may not fit the two-byte count, and the file came out with wrong counts and no error. A PSB takes them. minipsd_write's buffer holds the widest row: a row cut by the buffer's end is encoded again from its start, so 4096 bytes cut a long row many times. 30000 x 400 RGBA noise: 1406 ms to 796 ms, the same bytes.
minipsd: minipsd_predict_row, a row as minipsd_write hands it to the deflate for MINIPSD_ZIP_PREDICTED (big-endian, shuffled at 32 bits, each unit its difference from its left neighbour); the writer's own ZIP rows go through the same code. psdsave reads the image once, top to bottom, in the bands libvips hands it, and encodes each channel's rows as they come, so the raw image is never held: at 20000 x 20000 RGB, 1994 MB to 1215 MB for an 813 MB file, 2.1 s to 1.5 s. Each channel keeps room ahead of its rows for its marker and row sizes, so the block a PSD or a PSB stores is written in place. ZIP is deflated as the rows come, a stream a channel; zlib's stream depends on how its input is cut, so these bytes differ from a one-call deflate's. compression=zip-predicted (MINIPSD_VIPS_COMPRESSION_ZIP_PREDICTED). psdsave saves a PSB by itself when a PSD cannot hold the file: a row longer than its two-byte count, a layer section past 4 GiB, or a file past the 2 GB Photoshop opens as a PSD.
A CMake project that builds as it is: the plugin's sources, minipsd.h amalgamated and private to the plugin (a program's own minipsd, of any version, is left alone), cmake/minipsd_libvips.cmake to find libvips, a README and example.c, which saves a PSD through the plugin, reads it back, and is the zip's test. tools/package_vips.cmake packs it from libvips-plugin/release/.
The version is 0.4.0: breaking, so a MINOR. The changelog says what breaks and how to migrate: composite_alpha for every composite, the compression constants renumbered, minipsd_read giving transparency as the file has it. The manual, the spec, the header and INSTALL describe layers and composites without transparency, a layer's own compression, channels encoded by the program (the row functions), the writer's buffers, untrusted files, and the libvips plugin as it saves now and as released, with how a program that fetches libvips itself keeps it to one (FETCHCONTENT_SOURCE_DIR_MINIPSD_LIBVIPS). cmake/minipsd_libvips.cmake fetches libvips as minipsd_libvips, its target's name: as minipsd_vips, a program's FetchContent of the plugin's zip under that natural name would have won, and the plugin got its own zip for libvips. ROADMAP: S7.5 done, S4.12 releases 0.4.0.
The plugin's formats move to libvips-plugin/foreign/, as libvips keeps its own: psdload.c, psdsave.c, and psd.c and psd.h for what both share (as foreign/tiff.c and tiff.h; the three internal headers become one). minipsd_vips.h and minipsd_vips.c, the plugin's API and registration, stay beside. The zip mirrors it, minipsd's single header in foreign/minipsd/, as libvips carries libnsgif and qoi, so a pull request to libvips copies foreign/ as it is. Types and functions are named as libvips names its own (VipsForeignSavePsd*, vips_foreign_save_psd_*, VipsForeignLoadPsd*, vips_foreign_load_psd_*; helpers vips__foreign_psd_*); the compression enum is VipsForeignPsdCompression, VIPS_FOREIGN_PSD_COMPRESSION_*. psdsave's encoder is split: VipsForeignSavePsdChannels holds one image's channels (a layer's, or the composite's in one block), and a pass reads one image once, in bands, encoding each row into any number of them. psdsave is one pass into the layer and the composite; the bytes are unchanged. ROADMAP: S7.6, a layered document saved through libvips.
A saver takes one image; psdsave_layers (and _buffer, _target, with vips_psdsave_layers in minipsd_vips.h) takes an array of images, one a layer, bottom first, as vips_composite does. Each layer's name, opacity, visibility, blend mode and clipping come from the image's psd-layer-* fields, the ones psdload sets, its position from xoffset and yoffset; psd-layer-empty makes an empty layer, as Photoshop writes one. Each image is read once, in bands, its channels encoded as its rows come. The composite is given, or made by vips_composite from the visible layers (blend modes by the spec's table, opacity in the alpha), flattened against white when no layer has alpha. The canvas is width and height, else the composite's size, else every layer's from (left, top). psdload loads an empty layer as one transparent pixel with psd-layer-empty, and sets psd-layer-clipping, so a file's layers loaded and saved come back the same bytes: Platemaker's template, and a document with a placed, half-opaque, multiplied, clipped layer, are byte for byte minipsd_write's. All 2280 layers of psd-tools' set now load as minipsd decodes them.
A layer's mask travels as an image in its psd-layer-mask field: one band of the layers' depth, its own rectangle in xoffset and yoffset, with psd-layer-mask-default-color and psd-layer-mask-disabled. A group travels as records in the list, as the file has them: below its layers an image with psd-layer-section "end", above them one with "open" or "closed" carrying the group's name, visibility, opacity and mode. psdload gives both so, the top record's mode from its 'lsct' block, and psdsave_layers writes them back: a document with a group, a clipped layer with a mask and an empty layer comes back the same bytes. The composite psdsave_layers makes now hides a hidden group's layers, multiplies their opacity by the group's, and multiplies a layer's alpha by its mask. Clipping, a group's own mode and its mask are not drawn.
libvips accepts no contribution made with generative AI (its CONTRIBUTING.md), so the plugin never moves into libvips, and libvips' own names would only claim its namespace in every program that links the plugin. Types are MinipsdVipsLoad* and MinipsdVipsSave* (so are their GType names), functions minipsd_vips_*, the enum MinipsdVipsCompression (MINIPSD_VIPS_COMPRESSION_*), the calls minipsd_vips_psdsave_layers, _target and _buffer. The operations keep their nicknames: psdload, psdsave, psdsave_layers. The comments and docs that promised a move into libvips say it stays a plugin.
minipsd-vips-X.Y.Z.zip now holds minipsd-vips.spdx.json, an SPDX 2.3 fragment listing every other file of the zip with its checksums, and its CMakeLists sets MINIPSD_SBOM_FILE on minipsd::vips, as minipsd sets it on minipsd::minipsd: one download, one hash, code and SBOM. Both fragments come from cmake/minipsd_spdx.cmake; minipsd's own is unchanged. tools/check_spdx.cmake checks a fragment against its files, and CI and the release run it on the unpacked zip. The Windows build's zlib headers stay zlib 1.3.1's, now with the reason: libvips' libz1 is zlib-ng built zlib-compatible and reports "1.3.1.zlib-ng", and zlib-ng's own headers exist only once it is built.
The manual, the spec, INSTALL, the README and the zip's README now say plainly what the plugin does: it reads a file's composite or any layer, with its look, mask and groups, and writes one image, or several as the layers of one document. psdsave_layers is documented: its fields, the composite it composes and what it leaves out, and what a record loses on the way (text, effects, smart objects, adjustments' settings). INSTALL's ABI note now rests on a comparison: VipsForeignLoad and VipsForeignSave are the same in libvips 8.15.1 and 8.18.7, and a field added to either breaks a compiled plugin. CHANGELOG 0.4.0 lists the additions; ROADMAP marks S7.6 done.
minipsd_write cuts the file into pieces of 4 KiB, or of the widest row's bound; the widest can be a mask's, which no test had, so CI's line coverage stopped at 99.9%. WriteFailureIsReported now checks the pieces: at most 4 KiB without a wide row, more for a 5000-wide mask, within its row's bound.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The libvips plugin (
libvips-plugin/,minipsd::vips)layer=N. Every record loads, a recordwithout pixels as one transparent pixel with
psd-layer-empty. Each layer comes with its look inpsd-layer-*fields (name, opacity, visibility, blend mode, clipping), its mask as an image inpsd-layer-mask, and its group records marked bypsd-layer-section. Supports grey, RGB, CMYK,bitmap, indexed, Lab and multichannel. Rows are decoded in bands from libvips' threads.
psd-tools' set: 2280 of 2280 layers decode as minipsd decodes them.
image once, in bands, so memory stays near the file's size. Compression
rle,zip,zip-predictedornone; it switches to PSB by itself when a PSD cannot hold the file.minipsd_vips_psdsave_layers): several images saved as the layers of onedocument, masks and groups included. The composite is either given or composed by
vips_composite.minipsd_write's.MinipsdVips*,minipsd_vips_*). libvips' CONTRIBUTING.mdaccepts no contribution made with generative AI, so the plugin stays a separate project.
minipsd-vips-X.Y.Z.zip: a CMake project with minipsd's single header inside andits own SBOM fragment (
MINIPSD_SBOM_FILEonminipsd::vips). Before publishing, CI and therelease check the zip against that fragment and build it from the zip alone.
minipsd
composite_alphanow governs every composite.minipsd_readgives transparency as the file has it.no_alphalayers;minipsd_encode_row,minipsd_predict_rowandminipsd_matte_row.minipsd_writere-encoded long rows;Checked
psdsave_layers.