Skip to content

[Bug]: Imaginary preview provider sends the start of a JPEG twice (AppendStream is rewound), resulting in mostly gray previews #66693

Description

@kaynemo

⚠️ This issue respects the following points: ⚠️

  • This is not a troubleshooting question, general support matter, or webserver/proxy problem, but likely a bug (if unsure, ask the Community Help Forum).
  • This issue is not already reported on Github OR solved at the Community Help Forum (I've searched!).
  • I'm using a maintained major version of Nextcloud Server and tested against the latest patch level. (Supported major versions and current patch levels).
  • I agree to follow Nextcloud's Code of Conduct.
  • I've tried my best to provide clear reproduction steps that someone unfamiliar with this bug could use to reproduce it.

Bug description

After upgrading to 34.0.6 (from [PREVIOUS VERSION]), previews of JPEG files generated through the Imaginary provider are broken: a thin strip of the picture at the top, flat gray below. Nothing is written to nextcloud.log; the broken image is accepted and cached as a valid preview.

The cause appears to be in lib/private/Preview/Imaginary.php. For JPEGs, the code reads the start of the file (to check EXIF orientation) and then builds the request body as

$body = new AppendStream([Utils::streamFor($head), Utils::streamFor($stream)]);

(line 120 in 34.0.6). When the file stream is seekable, the body that reaches Imaginary is "head + the whole file from offset 0" instead of "head + the rest of the file". Imaginary decodes the first chunk, runs into a second JPEG header, logs Corrupt JPEG data: premature end of data segment and fills the rest of the image with gray.

HEIC and PNG files are not affected; they do not go through this code path.

Measured on one test file (960×1280 JPEG, 400,472 bytes):

Size
File on disk 400,472 bytes
Body built by the AppendStream construction 408,664 bytes (= 8,192 + 400,472)
Preview 960-1280-max.jpg, unmodified 34.0.6 9,396 bytes, mostly gray
Same file POSTed directly to Imaginary 176,751 bytes, correct
Preview 960-1280-max.jpg after the workaround below 175,995 bytes, correct

Steps to reproduce

  1. Nextcloud 34.0.6 with OC\Preview\Imaginary in enabledPreviewProviders and preview_imaginary_url pointing to a running Imaginary instance.
  2. Put a JPEG on storage that returns a seekable stream (here: a "Local" external storage mount).
  3. Generate a preview: open the file in Files or Talk, or run occ preview:generate --size=1024x1024 <fileid>.
  4. Look at the generated preview in appdata_*/preview/.../<fileid>/ and at the Imaginary log.

The duplication can be shown without Imaginary, using the same construction as the provider:

sudo -u www-data php -r '
require "/var/www/html/nextcloud/lib/base.php";
$f = \OCP\Server::get(\OCP\Files\IRootFolder::class)-&gt;getUserFolder("USER")-&gt;get("path/to/photo.jpg");
$s = $f-&gt;fopen("r");
$head = fread($s, 256 * 1024);
$body = new \GuzzleHttp\Psr7\AppendStream([\GuzzleHttp\Psr7\Utils::streamFor($head), \GuzzleHttp\Psr7\Utils::streamFor($s)]);
$d = (string)$body;
echo "head: ", strlen($head), " bytes\n";
echo "body: ", strlen($d), " bytes, file: ", $f-&gt;getSize(), " bytes\n";
echo "file start: ", bin2hex(substr($d, 0, 4)), ", after head: ", bin2hex(substr($d, strlen($head), 4)), "\n";'

Output on the affected server:

head: 8192 bytes
body: 408664 bytes, file: 400472 bytes
file start: ffd8ffe1, after head: ffd8ffe1

The body is larger than the file by exactly the head size, and a new JPEG header (ffd8ffe1) follows directly after the head.

Likely mechanism: AppendStream::rewind() / seek(0) rewinds every underlying stream, including the file stream that has already been read past the head. Guzzle rewinds a seekable body before sending, and the string cast in the snippet above does the same. I confirmed this with the snippet and with the results in the table, not by capturing the actual HTTP request body.

Expected behavior

Imaginary receives the file exactly once and the preview shows the complete picture.

Nextcloud Server version

34

Operating system

Debian/Ubuntu

PHP engine version

PHP 8.3

Web server

Apache (supported)

Database engine version

MariaDB

Is this bug present after an update or on a fresh install?

Updated from a MINOR version (ex. 32.0.1 to 32.0.2)

Are you using the Nextcloud Server Encryption module?

Encryption is Disabled

What user-backends are you using?

  • Default user-backend (database)
  • LDAP/ Active Directory
  • SSO - SAML
  • Other

Configuration report

{
    "system": {
        "serverid": "1",
        "instanceid": "***REMOVED SENSITIVE VALUE***",
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "allow_local_remote_servers": true,
        "trusted_domains": [
            "**********",
            "**********",
            "localhost",
            "**********",
            "localhost",
            "**********"
        ],
        "push_server_url": "https:*******",
        "datadirectory": "***REMOVED SENSITIVE VALUE***",
        "dbtype": "mysql",
        "version": "34.0.6.2",
        "overwrite.cli.url": "https:*******",
        "dbname": "***REMOVED SENSITIVE VALUE***",
        "dbhost": "***REMOVED SENSITIVE VALUE***",
        "dbport": "",
        "dbtableprefix": "oc_",
        "mysql.utf8mb4": true,
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "installed": true,
        "maintenance": false,
        "memcache.local": "\\OC\\Memcache\\APCu",
        "memcache.distributed": "\\OC\\Memcache\\Redis",
        "redis": {
            "host": "***REMOVED SENSITIVE VALUE***",
            "port": 0
        },
        "memcache.locking": "\\OC\\Memcache\\Redis",
        "theme": "",
        "loglevel": 2,
        "overwriteprotocol": "https",
        "mail_smtpmode": "smtp",
        "mail_sendmailmode": "smtp",
        "mail_from_address": "***REMOVED SENSITIVE VALUE***",
        "mail_domain": "***REMOVED SENSITIVE VALUE***",
        "mail_smtphost": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpport": "587",
        "default_locale": "ru_RU",
        "default_phone_region": "RU",
        "app_install_overwrite": {
            "0": "documentserver_community",
            "2": "files_rightclick",
            "3": "metadata",
            "4": "files_archive"
        },
        "enable_previews": true,
        "preview_libreoffice_path": "\/usr\/bin\/libreoffice",
        "apps_paths": [
            {
                "path": "\/var\/www\/html\/nextcloud\/apps",
                "url": "\/apps",
                "writable": false
            },
            {
                "path": "\/var\/www\/html\/nextcloud\/custom_apps",
                "url": "\/custom_apps",
                "writable": true
            }
        ],
        "mail_smtpauth": true,
        "mail_smtpname": "***REMOVED SENSITIVE VALUE***",
        "mail_smtppassword": "***REMOVED SENSITIVE VALUE***",
        "maintenance_window_start": 22,
        "collabora\/code": "https:*******",
        "fileuploadnew.chunk_size": "1G",
        "fileuploadnew.max_filesize": "25G",
        "memory_limit": "26G",
        "timeout": 36000,
        "enable_push": true,
        "has_internet_connection": true,
        "notify_push_base_endpoint": "https:********",
        "overwritewebroot": "",
        "mail_smtpauthtype": "LOGIN",
        "mail_smtpstreamoptions": {
            "ssl": {
                "allow_self_signed": false,
                "verify_peer": true,
                "verify_peer_name": true
            }
        },
        "logtimezone": "Europe\/Moscow",
        "mail_smtptimeout": 60,
        "trusted_proxies": "***REMOVED SENSITIVE VALUE***",
        "overwritehost": "cd.m-pr.tv",
        "preview_imaginary_url": "***REMOVED SENSITIVE VALUE***",
        "enabledPreviewProviders": [
            "OC\\Preview\\Imaginary",
            "OC\\Preview\\ImaginaryPDF",
            "OC\\Preview\\TXT",
            "OC\\Preview\\MarkDown",
            "OC\\Preview\\OpenDocument",
            "OC\\Preview\\MSOffice2003",
            "OC\\Preview\\MSOfficeDoc",
            "OC\\Preview\\MSOffice2007",
            "OC\\Preview\\Photoshop",
            "OC\\Preview\\Font",
            "OC\\Preview\\MP3",
            "OC\\Preview\\Movie"
        ],
        "preview_max_x": "2048",
        "preview_max_y": "2048",
        "preview_concurrency_all": 8,
        "preview_concurrency_new": 4,
        "preview_max_memory": "512",
        "preview_max_filesize_image": "128",
        "filelocking.ttl": 600,
        "files.chunked_upload.max_size": 5242880,
        "forwarded_for_headers": [
            "HTTP_X_FORWARDED_FOR"
        ],
        "lost_password_link": "disabled"
    }
}

List of activated Apps

Enabled:
  - activity: 7.0.0
  - admin_audit: 1.24.0
  - admincockpit: 1.4.6
  - analytics: 7.2.0
  - announcementbanner: 2.8.1
  - appstore: 1.0.0
  - bruteforcesettings: 7.0.0
  - calendar: 6.6.5
  - cloud_federation_api: 1.18.0
  - comments: 1.24.0
  - contacts: 8.10.0
  - contactsinteraction: 1.15.0
  - dashboard: 7.14.0
  - dav: 1.40.0
  - deck: 1.18.6
  - diskmap: 0.5.0
  - external: 9.0.1
  - federatedfilesharing: 1.24.0
  - federation: 1.24.0
  - files: 2.6.0
  - files_accesscontrol: 5.0.1
  - files_automatedtagging: 5.0.1
  - files_downloadlimit: 5.2.0
  - files_external: 1.26.0
  - files_lock: 34.0.1
  - files_pdfviewer: 7.0.1
  - files_reminders: 1.7.0
  - files_retention: 5.0.0
  - files_sharing: 1.26.1
  - files_trashbin: 1.24.0
  - files_versions: 1.27.0
  - flow_notifications: 5.0.0
  - forms: 5.4.2
  - formvox: 1.6.0
  - guests: 4.10.1
  - logcleaner: 1.6.5
  - logreader: 7.0.0
  - lookup_server_connector: 1.22.0
  - mail: 5.13.2
  - nextcloud_announcements: 6.0.0
  - notes: 6.1.1
  - notifications: 7.0.0-dev.1
  - notify_push: 1.4.2
  - oauth2: 1.22.0
  - onlyoffice: 10.2.1
  - password_policy: 6.0.0-dev.0
  - previewgenerator: 5.15.0
  - privacy: 6.0.0-dev.1
  - profile: 1.3.0
  - provisioning_api: 1.24.0
  - recommendations: 7.0.0
  - related_resources: 5.0.0-dev.0
  - serverinfo: 6.0.0
  - settings: 1.17.0
  - share_audit_dashboard: 0.8.2
  - sharebymail: 1.24.0
  - spreed: 24.0.8
  - support: 6.0.0
  - survey_client: 6.0.0-dev.0
  - systemtags: 1.24.0
  - tables: 2.4.0
  - tasks: 0.18.1
  - text: 8.0.1
  - theming: 2.9.0
  - thesearchpage: 2.5.1
  - twofactor_backupcodes: 1.23.0
  - twofactor_totp: 16.0.0
  - updatenotification: 1.24.0
  - user_status: 1.14.0
  - viewer: 7.0.0-dev.0
  - weather_status: 1.14.0
  - webhook_listeners: 1.6.0
  - workflow_script: 5.1.0
  - workflowengine: 2.16.0
Disabled:
  - app_api: 34.0.0 (installed 32.0.0)
  - circles: 34.0.0 (installed 28.0.0)
  - encryption: 2.22.0
  - firstrunwizard: 7.0.0-dev.0 (installed 2.18.0)
  - office: 1.0.0 (installed 1.0.0)
  - photos: 7.0.0 (installed 2.4.0)
  - suspicious_login: 12.0.0-dev.0
  - testing: 1.23.0
  - twofactor_nextcloud_notification: 8.0.0
  - user_ldap: 1.25.0

Nextcloud Signing status

Nextcloud Logs

Nothing is logged in `nextcloud.log` for the broken previews.

Additional info

Workaround

Changing line 120 so that a seekable stream is rewound and sent once fixed it here:

$body = rewind($stream) ? $stream : new AppendStream([Utils::streamFor($head), Utils::streamFor($stream)]);

After this change, restarting PHP-FPM and running occ preview:cleanup, previews are correct again. I am posting this as a workaround, not as a proposed fix. I assume the AppendStream construction exists for streams that cannot be rewound, and there may be a cleaner solution (for example, preventing the rewind of the second stream).

Scope and limitations of this report

  • Confirmed only with JPEGs sent from the Talk app on iOS, and analysed in depth on one file. JPEGs from other sources were not tested.
  • The files are on a "Local" external storage mount. Primary storage and object storage were not tested.
  • In the reproduction, fread() returned only 8,192 bytes although more was requested. This may depend on the storage wrapper.
  • occ integrity:check-core passed for Imaginary.php before I applied the workaround, so this is the released code. The check does list one other locally modified file, lib/private/SystemTag/SystemTagManager.php, which is unrelated to previews.

Additional info

Imaginary container log for each affected preview request:

Corrupt JPEG data: premature end of data segment

Metadata

Metadata

Assignees

No one assigned

    Labels

    0. Needs triagePending check for reproducibility or if it fits our roadmap34-feedbackbug

    Type

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions