Repository navigation
Description
⚠️ This issue respects the following points: ⚠️
- This is not a troubleshooting question, general support matter, or webserver/proxy problem, but likely a bug (if unsure, ask the Community Help Forum).
- This issue is not already reported on Github OR solved at the Community Help Forum (I've searched!).
- I'm using a maintained major version of Nextcloud Server and tested against the latest patch level. (Supported major versions and current patch levels).
- I agree to follow Nextcloud's Code of Conduct.
- I've tried my best to provide clear reproduction steps that someone unfamiliar with this bug could use to reproduce it.
Bug description
After upgrading to 34.0.6 (from [PREVIOUS VERSION]), previews of JPEG files generated through the Imaginary provider are broken: a thin strip of the picture at the top, flat gray below. Nothing is written to nextcloud.log; the broken image is accepted and cached as a valid preview.
The cause appears to be in lib/private/Preview/Imaginary.php. For JPEGs, the code reads the start of the file (to check EXIF orientation) and then builds the request body as
$body = new AppendStream([Utils::streamFor($head), Utils::streamFor($stream)]);(line 120 in 34.0.6). When the file stream is seekable, the body that reaches Imaginary is "head + the whole file from offset 0" instead of "head + the rest of the file". Imaginary decodes the first chunk, runs into a second JPEG header, logs Corrupt JPEG data: premature end of data segment and fills the rest of the image with gray.
HEIC and PNG files are not affected; they do not go through this code path.
Measured on one test file (960×1280 JPEG, 400,472 bytes):
| Size | |
|---|---|
| File on disk | 400,472 bytes |
| Body built by the AppendStream construction | 408,664 bytes (= 8,192 + 400,472) |
Preview 960-1280-max.jpg, unmodified 34.0.6 |
9,396 bytes, mostly gray |
| Same file POSTed directly to Imaginary | 176,751 bytes, correct |
Preview 960-1280-max.jpg after the workaround below |
175,995 bytes, correct |
Steps to reproduce
- Nextcloud 34.0.6 with
OC\Preview\ImaginaryinenabledPreviewProvidersandpreview_imaginary_urlpointing to a running Imaginary instance. - Put a JPEG on storage that returns a seekable stream (here: a "Local" external storage mount).
- Generate a preview: open the file in Files or Talk, or run
occ preview:generate --size=1024x1024 <fileid>. - Look at the generated preview in
appdata_*/preview/.../<fileid>/and at the Imaginary log.
The duplication can be shown without Imaginary, using the same construction as the provider:
sudo -u www-data php -r '
require "/var/www/html/nextcloud/lib/base.php";
$f = \OCP\Server::get(\OCP\Files\IRootFolder::class)->getUserFolder("USER")->get("path/to/photo.jpg");
$s = $f->fopen("r");
$head = fread($s, 256 * 1024);
$body = new \GuzzleHttp\Psr7\AppendStream([\GuzzleHttp\Psr7\Utils::streamFor($head), \GuzzleHttp\Psr7\Utils::streamFor($s)]);
$d = (string)$body;
echo "head: ", strlen($head), " bytes\n";
echo "body: ", strlen($d), " bytes, file: ", $f->getSize(), " bytes\n";
echo "file start: ", bin2hex(substr($d, 0, 4)), ", after head: ", bin2hex(substr($d, strlen($head), 4)), "\n";'Output on the affected server:
head: 8192 bytes
body: 408664 bytes, file: 400472 bytes
file start: ffd8ffe1, after head: ffd8ffe1
The body is larger than the file by exactly the head size, and a new JPEG header (ffd8ffe1) follows directly after the head.
Likely mechanism: AppendStream::rewind() / seek(0) rewinds every underlying stream, including the file stream that has already been read past the head. Guzzle rewinds a seekable body before sending, and the string cast in the snippet above does the same. I confirmed this with the snippet and with the results in the table, not by capturing the actual HTTP request body.
Expected behavior
Imaginary receives the file exactly once and the preview shows the complete picture.
Nextcloud Server version
34
Operating system
Debian/Ubuntu
PHP engine version
PHP 8.3
Web server
Apache (supported)
Database engine version
MariaDB
Is this bug present after an update or on a fresh install?
Updated from a MINOR version (ex. 32.0.1 to 32.0.2)
Are you using the Nextcloud Server Encryption module?
Encryption is Disabled
What user-backends are you using?
- Default user-backend (database)
- LDAP/ Active Directory
- SSO - SAML
- Other
Configuration report
{
"system": {
"serverid": "1",
"instanceid": "***REMOVED SENSITIVE VALUE***",
"passwordsalt": "***REMOVED SENSITIVE VALUE***",
"secret": "***REMOVED SENSITIVE VALUE***",
"allow_local_remote_servers": true,
"trusted_domains": [
"**********",
"**********",
"localhost",
"**********",
"localhost",
"**********"
],
"push_server_url": "https:*******",
"datadirectory": "***REMOVED SENSITIVE VALUE***",
"dbtype": "mysql",
"version": "34.0.6.2",
"overwrite.cli.url": "https:*******",
"dbname": "***REMOVED SENSITIVE VALUE***",
"dbhost": "***REMOVED SENSITIVE VALUE***",
"dbport": "",
"dbtableprefix": "oc_",
"mysql.utf8mb4": true,
"dbuser": "***REMOVED SENSITIVE VALUE***",
"dbpassword": "***REMOVED SENSITIVE VALUE***",
"installed": true,
"maintenance": false,
"memcache.local": "\\OC\\Memcache\\APCu",
"memcache.distributed": "\\OC\\Memcache\\Redis",
"redis": {
"host": "***REMOVED SENSITIVE VALUE***",
"port": 0
},
"memcache.locking": "\\OC\\Memcache\\Redis",
"theme": "",
"loglevel": 2,
"overwriteprotocol": "https",
"mail_smtpmode": "smtp",
"mail_sendmailmode": "smtp",
"mail_from_address": "***REMOVED SENSITIVE VALUE***",
"mail_domain": "***REMOVED SENSITIVE VALUE***",
"mail_smtphost": "***REMOVED SENSITIVE VALUE***",
"mail_smtpport": "587",
"default_locale": "ru_RU",
"default_phone_region": "RU",
"app_install_overwrite": {
"0": "documentserver_community",
"2": "files_rightclick",
"3": "metadata",
"4": "files_archive"
},
"enable_previews": true,
"preview_libreoffice_path": "\/usr\/bin\/libreoffice",
"apps_paths": [
{
"path": "\/var\/www\/html\/nextcloud\/apps",
"url": "\/apps",
"writable": false
},
{
"path": "\/var\/www\/html\/nextcloud\/custom_apps",
"url": "\/custom_apps",
"writable": true
}
],
"mail_smtpauth": true,
"mail_smtpname": "***REMOVED SENSITIVE VALUE***",
"mail_smtppassword": "***REMOVED SENSITIVE VALUE***",
"maintenance_window_start": 22,
"collabora\/code": "https:*******",
"fileuploadnew.chunk_size": "1G",
"fileuploadnew.max_filesize": "25G",
"memory_limit": "26G",
"timeout": 36000,
"enable_push": true,
"has_internet_connection": true,
"notify_push_base_endpoint": "https:********",
"overwritewebroot": "",
"mail_smtpauthtype": "LOGIN",
"mail_smtpstreamoptions": {
"ssl": {
"allow_self_signed": false,
"verify_peer": true,
"verify_peer_name": true
}
},
"logtimezone": "Europe\/Moscow",
"mail_smtptimeout": 60,
"trusted_proxies": "***REMOVED SENSITIVE VALUE***",
"overwritehost": "cd.m-pr.tv",
"preview_imaginary_url": "***REMOVED SENSITIVE VALUE***",
"enabledPreviewProviders": [
"OC\\Preview\\Imaginary",
"OC\\Preview\\ImaginaryPDF",
"OC\\Preview\\TXT",
"OC\\Preview\\MarkDown",
"OC\\Preview\\OpenDocument",
"OC\\Preview\\MSOffice2003",
"OC\\Preview\\MSOfficeDoc",
"OC\\Preview\\MSOffice2007",
"OC\\Preview\\Photoshop",
"OC\\Preview\\Font",
"OC\\Preview\\MP3",
"OC\\Preview\\Movie"
],
"preview_max_x": "2048",
"preview_max_y": "2048",
"preview_concurrency_all": 8,
"preview_concurrency_new": 4,
"preview_max_memory": "512",
"preview_max_filesize_image": "128",
"filelocking.ttl": 600,
"files.chunked_upload.max_size": 5242880,
"forwarded_for_headers": [
"HTTP_X_FORWARDED_FOR"
],
"lost_password_link": "disabled"
}
}List of activated Apps
Enabled:
- activity: 7.0.0
- admin_audit: 1.24.0
- admincockpit: 1.4.6
- analytics: 7.2.0
- announcementbanner: 2.8.1
- appstore: 1.0.0
- bruteforcesettings: 7.0.0
- calendar: 6.6.5
- cloud_federation_api: 1.18.0
- comments: 1.24.0
- contacts: 8.10.0
- contactsinteraction: 1.15.0
- dashboard: 7.14.0
- dav: 1.40.0
- deck: 1.18.6
- diskmap: 0.5.0
- external: 9.0.1
- federatedfilesharing: 1.24.0
- federation: 1.24.0
- files: 2.6.0
- files_accesscontrol: 5.0.1
- files_automatedtagging: 5.0.1
- files_downloadlimit: 5.2.0
- files_external: 1.26.0
- files_lock: 34.0.1
- files_pdfviewer: 7.0.1
- files_reminders: 1.7.0
- files_retention: 5.0.0
- files_sharing: 1.26.1
- files_trashbin: 1.24.0
- files_versions: 1.27.0
- flow_notifications: 5.0.0
- forms: 5.4.2
- formvox: 1.6.0
- guests: 4.10.1
- logcleaner: 1.6.5
- logreader: 7.0.0
- lookup_server_connector: 1.22.0
- mail: 5.13.2
- nextcloud_announcements: 6.0.0
- notes: 6.1.1
- notifications: 7.0.0-dev.1
- notify_push: 1.4.2
- oauth2: 1.22.0
- onlyoffice: 10.2.1
- password_policy: 6.0.0-dev.0
- previewgenerator: 5.15.0
- privacy: 6.0.0-dev.1
- profile: 1.3.0
- provisioning_api: 1.24.0
- recommendations: 7.0.0
- related_resources: 5.0.0-dev.0
- serverinfo: 6.0.0
- settings: 1.17.0
- share_audit_dashboard: 0.8.2
- sharebymail: 1.24.0
- spreed: 24.0.8
- support: 6.0.0
- survey_client: 6.0.0-dev.0
- systemtags: 1.24.0
- tables: 2.4.0
- tasks: 0.18.1
- text: 8.0.1
- theming: 2.9.0
- thesearchpage: 2.5.1
- twofactor_backupcodes: 1.23.0
- twofactor_totp: 16.0.0
- updatenotification: 1.24.0
- user_status: 1.14.0
- viewer: 7.0.0-dev.0
- weather_status: 1.14.0
- webhook_listeners: 1.6.0
- workflow_script: 5.1.0
- workflowengine: 2.16.0
Disabled:
- app_api: 34.0.0 (installed 32.0.0)
- circles: 34.0.0 (installed 28.0.0)
- encryption: 2.22.0
- firstrunwizard: 7.0.0-dev.0 (installed 2.18.0)
- office: 1.0.0 (installed 1.0.0)
- photos: 7.0.0 (installed 2.4.0)
- suspicious_login: 12.0.0-dev.0
- testing: 1.23.0
- twofactor_nextcloud_notification: 8.0.0
- user_ldap: 1.25.0Nextcloud Signing status
Nextcloud Logs
Nothing is logged in `nextcloud.log` for the broken previews.Additional info
Workaround
Changing line 120 so that a seekable stream is rewound and sent once fixed it here:
$body = rewind($stream) ? $stream : new AppendStream([Utils::streamFor($head), Utils::streamFor($stream)]);After this change, restarting PHP-FPM and running occ preview:cleanup, previews are correct again. I am posting this as a workaround, not as a proposed fix. I assume the AppendStream construction exists for streams that cannot be rewound, and there may be a cleaner solution (for example, preventing the rewind of the second stream).
Scope and limitations of this report
- Confirmed only with JPEGs sent from the Talk app on iOS, and analysed in depth on one file. JPEGs from other sources were not tested.
- The files are on a "Local" external storage mount. Primary storage and object storage were not tested.
- In the reproduction,
fread()returned only 8,192 bytes although more was requested. This may depend on the storage wrapper. occ integrity:check-corepassed forImaginary.phpbefore I applied the workaround, so this is the released code. The check does list one other locally modified file,lib/private/SystemTag/SystemTagManager.php, which is unrelated to previews.
Additional info
Imaginary container log for each affected preview request:
Corrupt JPEG data: premature end of data segment
Metadata
Metadata
Assignees
Labels
Type
Projects
- StatusShow more project fieldsTo triage
Activity