diff --git a/README.md b/README.md index f37fe30..338142b 100644 --- a/README.md +++ b/README.md @@ -45,6 +45,7 @@ Pure decode speed with one thread, bytes pre-loaded to memory. **Bold** = best p | `turbojpeg` | 640 | 818 | 708 | 426 | 613 | | `jpeg4py` | 636 | 760 | 699 | 423 | 611 | | `kornia-rs` | 642 | 761 | 664 | 391 | 629 | +| `ajpegli` | 400 | 501 | 323 | 273 | 337 | | `opencv` | 664 | 841 | 721 | 445 | 645 | | `imagecodecs` | 677 | 775 | 723 | **457** | 661 | | `pyvips` | 420 | 586 | 462 | 261 | 413 | @@ -68,6 +69,7 @@ Best `images_per_second` across `num_workers ∈ {0, 2, 4, 8}` for each library | `turbojpeg` | 1,535 @ 4w | 2,800 @ 8w | 1,710 @ 8w | 1,347 @ 4w | 2,389 @ 8w | | `jpeg4py` | 1,443 @ 4w | 2,453 @ 8w | 1,651 @ 8w | 1,411 @ 8w | 2,312 @ 8w | | `kornia-rs` | 1,327 @ 8w | 2,394 @ 8w | 1,422 @ 8w | 1,260 @ 8w | 1,951 @ 8w | +| `ajpegli` | 1,458 @ 8w | 2,882 @ 8w | 1,498 @ 8w | 1,448 @ 8w | 2,397 @ 8w | | `opencv` | 1,457 @ 4w | 2,814 @ 8w | 1,707 @ 8w | 1,419 @ 8w | 2,414 @ 8w | | `imagecodecs` | 1,543 @ 4w | 2,476 @ 8w | 1,677 @ 8w | 1,443 @ 8w | 2,242 @ 8w | | `pillow` | 1,283 @ 4w | 2,465 @ 8w | 1,565 @ 8w | 1,387 @ 8w | 2,350 @ 8w | @@ -79,7 +81,7 @@ Best `images_per_second` across `num_workers ∈ {0, 2, 4, 8}` for each library -_**5 platforms** · 50,000 images · 5 runs each · latest run 2026-04-22_ +_**5 platforms** · 50,000 images · 5 runs each · latest run 2026-05-20_ @@ -93,6 +95,7 @@ Current headline patterns: - `torchvision` wins both AMD platforms at peak `DataLoader` throughput and is effectively tied for first on Neoverse V2. - `imageio` is not a single-thread leader, but wins peak `DataLoader` throughput on Neoverse V2 in the current GCP runs. - OpenCV is rarely the absolute winner, but is consistently close to the local winner and has successful `DataLoader` results on every platform. +- `ajpegli` has valid single-thread and `DataLoader` results, but skips one uncommon ImageNet JPEG in this matrix, so it is reported as a strict decoder rather than a zero-skip default. - PyVips is reported for single-thread decode only; it is skipped in fork-based `DataLoader` benchmarks because of libvips threadpool deadlocks in this harness. ## GitAds Sponsored @@ -219,6 +222,8 @@ output/ ### Comprehensive codec libraries +- **ajpegli** — Python wrapper for Google's JPEGli decoder; supports both + path-based reads and in-memory byte decoding. - **imagecodecs** — uses libjpeg-turbo 3.x; prebuilt ARM64 wheels - **pyvips** — libvips bindings (bundled in wheels). Single-thread only; the libvips threadpool deadlocks under fork-based PyTorch DataLoader, @@ -254,7 +259,7 @@ output/ - Use the DataLoader benchmark for final decoder and `num_workers` selection. - Start with OpenCV when you need a robust default that runs everywhere. - Try `torchvision` when your pipeline already wants tensors and you can benchmark the target CPU. -- Try `simplejpeg` / `turbojpeg` / `jpeg4py` when maximum libjpeg-turbo-backed speed matters and your dataset policy handles uncommon JPEG modes. +- Try lower-level native decoders such as `simplejpeg`, `turbojpeg`, `jpeg4py`, `kornia-rs`, or `ajpegli` when maximum decode speed matters and your dataset policy handles uncommon JPEG modes. ### Choosing for pure decode speed diff --git a/docs/assets/benchmarks/fig01_protocol_rank_change.png b/docs/assets/benchmarks/fig01_protocol_rank_change.png index 164df7d..549cff3 100644 Binary files a/docs/assets/benchmarks/fig01_protocol_rank_change.png and b/docs/assets/benchmarks/fig01_protocol_rank_change.png differ diff --git a/docs/assets/benchmarks/fig02_amd_worker_delta.png b/docs/assets/benchmarks/fig02_amd_worker_delta.png index 798c4dc..17fbcf0 100644 Binary files a/docs/assets/benchmarks/fig02_amd_worker_delta.png and b/docs/assets/benchmarks/fig02_amd_worker_delta.png differ diff --git a/docs/assets/benchmarks/fig03_tensorflow_arm_penalty.png b/docs/assets/benchmarks/fig03_tensorflow_arm_penalty.png index a42fa24..382b1ae 100644 Binary files a/docs/assets/benchmarks/fig03_tensorflow_arm_penalty.png and b/docs/assets/benchmarks/fig03_tensorflow_arm_penalty.png differ diff --git a/docs/assets/benchmarks/fig04_decoder_recommendation_summary.png b/docs/assets/benchmarks/fig04_decoder_recommendation_summary.png index 8b8fe66..352467c 100644 Binary files a/docs/assets/benchmarks/fig04_decoder_recommendation_summary.png and b/docs/assets/benchmarks/fig04_decoder_recommendation_summary.png differ diff --git a/docs/gcp_benchmarks.md b/docs/gcp_benchmarks.md index f468eb0..5765912 100644 --- a/docs/gcp_benchmarks.md +++ b/docs/gcp_benchmarks.md @@ -60,8 +60,8 @@ tarball wins. To force the slow path for testing, rename or delete the tarball. ```bash ./gcp/run.sh \ - --imagenet-bucket gs://my-bucket/imagenet/val \ - --results-bucket gs://my-bucket/imread-results \ + --imagenet-bucket gs://YOUR_BUCKET/imagenet/val \ + --results-bucket gs://YOUR_BUCKET/imread-results \ --no-wait ``` @@ -72,19 +72,19 @@ What happens while you sleep: 1. A `c3-standard-8` VM boots on GCP (Intel Sapphire Rapids, 8 vCPU) 2. Downloads ImageNet from GCS to local disk (~30s within GCP, tarball fast path) -3. Runs all 11 library benchmarks in memory mode (1-thread + default-thread) — ~15-25 min -4. Runs all DataLoader benchmarks at 50k images, workers ∈ {0, 2, 4, 8} — ~25-45 min -5. Uploads results to `gs://my-bucket/imread-results//output/` +3. Runs all supported library benchmarks in memory mode (1-thread + default-thread) — ~15-25 min +4. Runs requested DataLoader benchmarks at 50k images, workers ∈ {0, 2, 4, 8} — ~25-45 min +5. Uploads results to `gs://YOUR_BUCKET/imread-results//output/` 6. **Deletes itself** — no orphaned VM, no ongoing billing ### In the morning ```bash # Check if done (exit 0 = object exists) -gcloud storage objects describe gs://my-bucket/imread-results//DONE +gcloud storage objects describe gs://YOUR_BUCKET/imread-results//DONE # Fetch results -gcloud storage cp --recursive gs://my-bucket/imread-results//output/ ./output/ +gcloud storage cp --recursive gs://YOUR_BUCKET/imread-results//output/ ./output/ ``` The exact commands are printed by `--no-wait` when you start the run. @@ -92,7 +92,7 @@ The exact commands are printed by `--no-wait` when you start the run. ### Watch the log mid-run (optional) ```bash -gcloud storage cat gs://my-bucket/imread-results//startup.log +gcloud storage cat gs://YOUR_BUCKET/imread-results//startup.log ``` This streams the full VM stdout, including per-library progress, without SSH. @@ -105,13 +105,37 @@ If you want the script to block, download results, and exit all in one command: ```bash ./gcp/run.sh \ - --imagenet-bucket gs://my-bucket/imagenet/val \ - --results-bucket gs://my-bucket/imread-results + --imagenet-bucket gs://YOUR_BUCKET/imagenet/val \ + --results-bucket gs://YOUR_BUCKET/imread-results ``` Progress is printed every 30 seconds. Results land in `./output/` when done. Press Ctrl+C at any point — the VM is deleted immediately via a cleanup trap. +### Run one decoder + +Use `--libs` when you only need new data for one decoder. For example, to add +`ajpegli` results without rerunning every library: + +```bash +./gcp/run.sh \ + --machine-type c4-standard-16 \ + --smoke \ + --libs ajpegli \ + --imagenet-bucket gs://YOUR_BUCKET/imagenet/val \ + --results-bucket gs://YOUR_BUCKET/imread-results +``` + +For the full paper platform matrix: + +```bash +./gcp/run-many.sh \ + --machine-types "c4-standard-16 c3d-standard-16 c4d-standard-16 c4a-standard-16 t2a-standard-16" \ + --libs ajpegli \ + --imagenet-bucket gs://YOUR_BUCKET/imagenet/val \ + --results-bucket gs://YOUR_BUCKET/imread-results +``` + --- ## Options @@ -123,6 +147,7 @@ Press Ctrl+C at any point — the VM is deleted immediately via a cleanup trap. | `--results-bucket` | (required) | GCS bucket for results and logs | | `--zone` | `us-central1-a` | GCP zone | | `--machine-type` | `c3-standard-8` | VM type (Intel Sapphire Rapids, 8 vCPU, 32 GB) | +| `--libs` | `all` | Comma-separated decoder names, or `all` | | `--num-images` | `50000` | Number of images per benchmark run (full ImageNet val) | | `--num-runs` | `5` | Single-thread timed runs per library (see [Sample size](#sample-size))| | `--dl-runs` | `3` | DataLoader timed runs per worker config | @@ -141,7 +166,7 @@ All flags can also be set as environment variables (`IMAGENET_BUCKET`, `RESULTS_ ## Venv cache -Cold installing 11 decoder libraries on the VM takes ~25 minutes. To avoid paying that on +Cold installing decoder libraries on the VM takes ~25 minutes. To avoid paying that on every run, the first run on a given `(os, arch, deps)` combo tars `venvs/` and uploads it to `gs:///imread-cache/`. Subsequent runs pull and extract it in seconds. @@ -282,6 +307,12 @@ ignored `_internal/papers/` workspace: uv run --extra plot python -m tools.paper_assets --all ``` +For the public arXiv preprint workspace specifically: + +```bash +uv run --extra plot python -m tools.paper_assets --all --paper-dir _internal/papers/arxiv_preprint +``` + Do not commit `_internal/` outputs. They are local manuscript artifacts; the tracked source of truth is the benchmark JSON plus the generator code. diff --git a/docs/plotting_and_statistics.md b/docs/plotting_and_statistics.md index 4ecf752..f45c82c 100644 --- a/docs/plotting_and_statistics.md +++ b/docs/plotting_and_statistics.md @@ -118,6 +118,6 @@ throughput. - "TensorFlow is always slow." - "OpenCV is fastest." - "PyVips is broken." Use "unsupported in this fork-based PyTorch DataLoader harness." -- "Strict libjpeg-turbo wrappers are wrong." Use "they require an explicit CMYK fallback policy." +- "Strict native JPEG decoders are wrong." Use "they require an explicit fallback policy for uncommon JPEG modes." - "Rank 1 beats rank 2" when their uncertainty or practical gap makes them a top tier. - Cross-platform absolute throughput comparisons without explaining CPU speed and microarchitecture. diff --git a/gcp/run.sh b/gcp/run.sh index 0b8bc54..d1789d2 100755 --- a/gcp/run.sh +++ b/gcp/run.sh @@ -11,6 +11,7 @@ # Optional: # --zone GCP zone (default: us-central1-a) # --machine-type GCP machine type (default: c3-standard-8) +# --libs comma-separated decoder names, or all (default: all) # --num-images images to benchmark (default: 50000 = full ImageNet val) # --num-runs single-thread timed runs per library (default: 5) # --dl-runs DataLoader timed runs per worker config (default: 3) @@ -32,14 +33,14 @@ # ./gcp/run.sh --upload-imagenet ~/imagenet/val --imagenet-bucket gs://my-bucket/imagenet/val # # # Smoke test on a new machine type (~10 min, ~$0.10) -# ./gcp/run.sh --machine-type c4-standard-16 --smoke \ -# --imagenet-bucket gs://my-bucket/imagenet/val \ -# --results-bucket gs://my-bucket/imread-results --no-wait +# ./gcp/run.sh --machine-type c4-standard-16 --smoke --libs ajpegli \ +# --imagenet-bucket gs://YOUR_BUCKET/imagenet/val \ +# --results-bucket gs://YOUR_BUCKET/imread-results --no-wait # # # Full run (blocks ~45-90 min depending on machine, fetches results when done) # ./gcp/run.sh \ -# --imagenet-bucket gs://my-bucket/imagenet/val \ -# --results-bucket gs://my-bucket/imread-results +# --imagenet-bucket gs://YOUR_BUCKET/imagenet/val \ +# --results-bucket gs://YOUR_BUCKET/imread-results # # # Fire and forget # ./gcp/run.sh --imagenet-bucket gs://... --results-bucket gs://... --no-wait @@ -68,6 +69,7 @@ NUM_IMAGES="${NUM_IMAGES:-50000}" NUM_RUNS="${NUM_RUNS:-5}" DL_RUNS="${DL_RUNS:-3}" WORKERS="${WORKERS:-0 2 4 8}" +LIBS="${LIBS:-all}" IMAGENET_BUCKET="${IMAGENET_BUCKET:-}" RESULTS_BUCKET="${RESULTS_BUCKET:-}" NO_WAIT=false @@ -84,6 +86,7 @@ while [[ $# -gt 0 ]]; do --results-bucket) RESULTS_BUCKET="$2"; shift 2 ;; --zone) ZONE="$2"; shift 2 ;; --machine-type) MACHINE_TYPE="$2"; shift 2 ;; + --libs) LIBS="$2"; shift 2 ;; --num-images) NUM_IMAGES="$2"; shift 2 ;; --num-runs) NUM_RUNS="$2"; shift 2 ;; --dl-runs) DL_RUNS="$2"; shift 2 ;; @@ -169,6 +172,7 @@ echo " Run ID : $RUN_NAME" echo " Machine : $MACHINE_TYPE ($ZONE)" echo " ImageNet : $IMAGENET_BUCKET" echo " Results : $RUN_GCS" +echo " Libraries : $LIBS" echo " Num images : $NUM_IMAGES" echo " Single runs : $NUM_RUNS" echo " DataLoader : $DL_RUNS runs × workers=[$WORKERS]" @@ -257,7 +261,7 @@ gcloud compute instances create "$RUN_NAME" \ --image-project=ubuntu-os-cloud \ --boot-disk-size=60GB \ --boot-disk-type="$BOOT_DISK_TYPE" \ - --metadata="results-bucket=$RUN_GCS,imagenet-bucket=$IMAGENET_BUCKET,num-images=$NUM_IMAGES,num-runs=$NUM_RUNS,dl-runs=$DL_RUNS,workers=$WORKERS,repo-tarball=$RUN_GCS/repo.tar.gz,cache-bucket=$CACHE_BUCKET,force-rebuild=$FORCE_REBUILD,keep-on-failure=$([[ $KEEP_ON_FAILURE == true ]] && echo 1 || echo 0)" \ + --metadata="results-bucket=$RUN_GCS,imagenet-bucket=$IMAGENET_BUCKET,libs=$LIBS,num-images=$NUM_IMAGES,num-runs=$NUM_RUNS,dl-runs=$DL_RUNS,workers=$WORKERS,repo-tarball=$RUN_GCS/repo.tar.gz,cache-bucket=$CACHE_BUCKET,force-rebuild=$FORCE_REBUILD,keep-on-failure=$([[ $KEEP_ON_FAILURE == true ]] && echo 1 || echo 0)" \ --metadata-from-file=startup-script="$STARTUP_SCRIPT" \ --scopes=storage-rw,compute-rw \ --maintenance-policy=TERMINATE \ diff --git a/gcp/vm_startup.sh b/gcp/vm_startup.sh index 0cf5802..09a1e15 100755 --- a/gcp/vm_startup.sh +++ b/gcp/vm_startup.sh @@ -73,6 +73,7 @@ meta() { # ── Read metadata ────────────────────────────────────────────────────────────── RESULTS_BUCKET=$(meta results-bucket) IMAGENET_BUCKET=$(meta imagenet-bucket) +LIBS=$(meta libs 2>/dev/null || echo "all") NUM_IMAGES=$(meta num-images) REPO_TARBALL=$(meta repo-tarball) # Optional metadata with sensible defaults so older callers still work. @@ -137,6 +138,7 @@ echo " imread benchmark — VM startup" echo " $(date -u)" echo " Results bucket : $RESULTS_BUCKET" echo " ImageNet : $IMAGENET_BUCKET" +echo " Libraries : $LIBS" echo " Num images : $NUM_IMAGES" echo " Repo tarball : $REPO_TARBALL" echo "═══════════════════════════════════════════════════" @@ -365,7 +367,7 @@ echo "[step 5] Done." # encoded on the decoder classes themselves, no shell `case` games. echo echo "[step 6] Running benchmarks..." -echo " NUM_IMAGES=$NUM_IMAGES NUM_RUNS=$NUM_RUNS DL_RUNS=$DL_RUNS" +echo " LIBS=$LIBS NUM_IMAGES=$NUM_IMAGES NUM_RUNS=$NUM_RUNS DL_RUNS=$DL_RUNS" echo " WORKERS=$WORKERS" WORKERS_CSV=$(echo "$WORKERS" | tr ' ' ',') # Partial-failure policy: per-decoder failures (e.g. turbojpeg crashing on a @@ -380,7 +382,7 @@ WORKERS_CSV=$(echo "$WORKERS" | tr ' ' ',') imread-benchmark run \ --data-dir "$IMAGENET_DIR" \ --output-dir output \ - --libs all \ + --libs "$LIBS" \ --mode both \ --num-images "$NUM_IMAGES" \ --num-runs "$NUM_RUNS" \ diff --git a/imread_benchmark/decoders/ajpegli_decoder.py b/imread_benchmark/decoders/ajpegli_decoder.py new file mode 100644 index 0000000..b63039c --- /dev/null +++ b/imread_benchmark/decoders/ajpegli_decoder.py @@ -0,0 +1,22 @@ +from __future__ import annotations + +import numpy as np + +from imread_benchmark.decoders import BaseDecoder + + +class AJpegliDecoder(BaseDecoder): + """ajpegli — Python bindings for Google's JPEGli decoder.""" + + name = "ajpegli" + package_name = "ajpegli" + + def decode(self, data: bytes) -> np.ndarray: + import ajpegli + + return ajpegli.imdecode(data, mode="RGB") + + def decode_path(self, path: str) -> np.ndarray: + import ajpegli + + return ajpegli.imread(path, mode="RGB") diff --git a/output/linux_AMD-EPYC-9B14/ajpegli_1t_results.json b/output/linux_AMD-EPYC-9B14/ajpegli_1t_results.json new file mode 100644 index 0000000..5f23e40 --- /dev/null +++ b/output/linux_AMD-EPYC-9B14/ajpegli_1t_results.json @@ -0,0 +1,57 @@ +{ + "library": "ajpegli", + "mode": "memory", + "run_tag": "1t", + "requested_threads": 1, + "effective_threads": 1, + "system_info": { + "Python": "3.12.3", + "OS": "Linux", + "OS Version": "#17~24.04.1-Ubuntu SMP Thu May 7 15:12:38 UTC 2026", + "Machine": "x86_64", + "CPU": { + "brand_raw": "AMD EPYC 9B14", + "arch": "X86_64", + "hz_advertised_raw": "Unknown", + "count": 16 + }, + "ajpegli": "1.0.0" + }, + "benchmark_results": { + "images_per_second_mean": 400.17318636034486, + "images_per_second_std": 0.0418143320963244, + "images_per_second_p50": 400.1734736537274, + "images_per_second_p90": 400.22015594016983, + "images_per_second_p99": 400.2213877410496, + "us_per_image_mean": 2498.91808097762, + "us_per_image_p50": 2498.9162596651927, + "us_per_image_p90": 2499.213334074683, + "us_per_image_p99": 2499.224782008442, + "raw_times_s": [ + 124.929381337, + 124.95721351500003, + 124.94331406699996, + 124.95880347400009, + 124.92831326099997 + ], + "raw_throughput_ips": [ + 400.21810293870345, + 400.1289608942668, + 400.1734736537274, + 400.12386970721263, + 400.22152460781405 + ], + "num_images_total": 50000, + "num_images_decoded": 49999, + "num_images_skipped": 1, + "skip_rate": 2e-05, + "skip_indices": [ + 19876 + ], + "skip_examples": [ + "idx=19876: DecodeError: jpegli decode failed: ./lib/jpegli/color_transform.cc:549: Unsupported color transform 4 -> 2" + ] + }, + "num_images": 50000, + "num_runs": 5 +} \ No newline at end of file diff --git a/output/linux_AMD-EPYC-9B14/ajpegli_dataloader_results.json b/output/linux_AMD-EPYC-9B14/ajpegli_dataloader_results.json new file mode 100644 index 0000000..a7f585d --- /dev/null +++ b/output/linux_AMD-EPYC-9B14/ajpegli_dataloader_results.json @@ -0,0 +1,125 @@ +{ + "library": "ajpegli", + "benchmark_type": "dataloader", + "batch_size": 32, + "num_threads": 1, + "system_info": { + "Python": "3.12.3", + "OS": "Linux", + "OS Version": "#17~24.04.1-Ubuntu SMP Thu May 7 15:12:38 UTC 2026", + "Machine": "x86_64", + "CPU": { + "brand_raw": "AMD EPYC 9B14", + "arch": "X86_64", + "hz_advertised_raw": "Unknown", + "count": 16 + }, + "ajpegli": "1.0.0" + }, + "worker_results": [ + { + "num_workers": 0, + "num_warmup": 1, + "images_per_second_mean": 395.540106984955, + "images_per_second_std": 0.08917249239482852, + "images_per_second_p50": 395.5006375171751, + "images_per_second_p90": 395.6309822101358, + "images_per_second_p99": 395.66030976605197, + "us_per_image_mean": 2528.188755475108, + "us_per_image_p50": 2528.4409306586103, + "us_per_image_p90": 2528.668662461249, + "us_per_image_p99": 2528.7199021168426, + "raw_times_s": [ + 126.43375104500001, + 126.41951809199986, + 126.36745961799988 + ], + "raw_throughput_ips": [ + 395.4561150543139, + 395.5006375171751, + 395.663568383376 + ] + }, + { + "num_workers": 2, + "num_warmup": 1, + "images_per_second_mean": 704.4197781175443, + "images_per_second_std": 0.09802237689406454, + "images_per_second_p50": 704.4847162356363, + "images_per_second_p90": 704.491642780342, + "images_per_second_p99": 704.4932012529007, + "us_per_image_mean": 1419.608095101902, + "us_per_image_p50": 1419.4772107242134, + "us_per_image_p90": 1419.8052894617902, + "us_per_image_p99": 1419.879107177745, + "raw_times_s": [ + 70.99294557000007, + 70.97156881199999, + 70.97244105899995 + ], + "raw_throughput_ips": [ + 704.2812437004781, + 704.4933744165183, + 704.4847162356363 + ] + }, + { + "num_workers": 4, + "num_warmup": 1, + "images_per_second_mean": 1258.8811499873761, + "images_per_second_std": 4.412095443803712, + "images_per_second_p50": 1261.3187124413855, + "images_per_second_p90": 1262.3730723324231, + "images_per_second_p99": 1262.6103033079066, + "us_per_image_mean": 794.3659432055289, + "us_per_image_p50": 792.8210294005852, + "us_per_image_p90": 797.1908646852931, + "us_per_image_p99": 798.1740776243524, + "raw_times_s": [ + 39.913367892, + 39.64025864899986, + 39.598881841999855 + ], + "raw_throughput_ips": [ + 1252.6880752155598, + 1261.3187124413855, + 1262.6366623051827 + ] + }, + { + "num_workers": 8, + "num_warmup": 1, + "images_per_second_mean": 1458.2945506521744, + "images_per_second_std": 67.87134268899736, + "images_per_second_p50": 1505.1164847873945, + "images_per_second_p90": 1506.9814989029114, + "images_per_second_p99": 1507.4011270789026, + "us_per_image_mean": 687.2718519970394, + "us_per_image_p50": 664.4004036280655, + "us_per_image_p90": 720.1138757855215, + "us_per_image_p99": 732.6494070209492, + "raw_times_s": [ + 36.70137814900045, + 33.21935578099965, + 33.16798205399982 + ], + "raw_throughput_ips": [ + 1362.3194147373376, + 1505.1164847873945, + 1507.4477524317906 + ] + } + ], + "num_images": 49999, + "num_images_total": 50000, + "num_images_decoded": 49999, + "num_images_skipped": 1, + "skip_rate": 2e-05, + "skip_indices": [ + 19876 + ], + "skip_examples": [ + "idx=19876: DecodeError: jpegli decode failed: ./lib/jpegli/color_transform.cc:549: Unsupported color transform 4 -> 2" + ], + "num_runs": 3 +} \ No newline at end of file diff --git a/output/linux_AMD-EPYC-9B14/run_summary.json b/output/linux_AMD-EPYC-9B14/run_summary.json index a9daf25..c0c276f 100644 --- a/output/linux_AMD-EPYC-9B14/run_summary.json +++ b/output/linux_AMD-EPYC-9B14/run_summary.json @@ -1,5 +1,5 @@ { - "timestamp_utc": "2026-04-20T09:56:20.148943+00:00", + "timestamp_utc": "2026-05-20T07:26:36.975587+00:00", "system": "linux_AMD-EPYC-9B14", "mode": "both", "num_images": 50000, @@ -12,33 +12,11 @@ 8 ], "libs_requested": [ - "imagecodecs", - "imageio", - "jpeg4py", - "kornia", - "opencv", - "pillow", - "pyvips", - "simplejpeg", - "skimage", - "tensorflow", - "torchvision", - "turbojpeg" + "ajpegli" ], "libs_skipped_platform": [], "libs_run": [ - "imagecodecs", - "imageio", - "jpeg4py", - "kornia", - "opencv", - "pillow", - "pyvips", - "simplejpeg", - "skimage", - "tensorflow", - "torchvision", - "turbojpeg" + "ajpegli" ], "failures": [], "exit_status": "ok" diff --git a/output/linux_AMD-EPYC-9B45/ajpegli_1t_results.json b/output/linux_AMD-EPYC-9B45/ajpegli_1t_results.json new file mode 100644 index 0000000..6a00d49 --- /dev/null +++ b/output/linux_AMD-EPYC-9B45/ajpegli_1t_results.json @@ -0,0 +1,57 @@ +{ + "library": "ajpegli", + "mode": "memory", + "run_tag": "1t", + "requested_threads": 1, + "effective_threads": 1, + "system_info": { + "Python": "3.12.3", + "OS": "Linux", + "OS Version": "#17~24.04.1-Ubuntu SMP Thu May 7 15:12:38 UTC 2026", + "Machine": "x86_64", + "CPU": { + "brand_raw": "AMD EPYC 9B45", + "arch": "X86_64", + "hz_advertised_raw": "Unknown", + "count": 16 + }, + "ajpegli": "1.0.0" + }, + "benchmark_results": { + "images_per_second_mean": 501.40607004730794, + "images_per_second_std": 1.4483472687335015, + "images_per_second_p50": 501.92741517795105, + "images_per_second_p90": 502.8401842340531, + "images_per_second_p99": 503.0168942122083, + "us_per_image_mean": 1994.408155383107, + "us_per_image_p50": 1992.319944598891, + "us_per_image_p90": 2001.4822450649003, + "us_per_image_p99": 2003.353835268704, + "raw_times_s": [ + 99.49145564899999, + 99.39437228099996, + 99.91614804, + 99.61400490999995, + 100.17608592499994 + ], + "raw_throughput_ips": [ + 502.5456676037944, + 503.03652865422555, + 500.4096032603621, + 501.92741517795105, + 499.1111355402063 + ], + "num_images_total": 50000, + "num_images_decoded": 49999, + "num_images_skipped": 1, + "skip_rate": 2e-05, + "skip_indices": [ + 19876 + ], + "skip_examples": [ + "idx=19876: DecodeError: jpegli decode failed: ./lib/jpegli/color_transform.cc:549: Unsupported color transform 4 -> 2" + ] + }, + "num_images": 50000, + "num_runs": 5 +} \ No newline at end of file diff --git a/output/linux_AMD-EPYC-9B45/ajpegli_dataloader_results.json b/output/linux_AMD-EPYC-9B45/ajpegli_dataloader_results.json new file mode 100644 index 0000000..87d5df0 --- /dev/null +++ b/output/linux_AMD-EPYC-9B45/ajpegli_dataloader_results.json @@ -0,0 +1,125 @@ +{ + "library": "ajpegli", + "benchmark_type": "dataloader", + "batch_size": 32, + "num_threads": 1, + "system_info": { + "Python": "3.12.3", + "OS": "Linux", + "OS Version": "#17~24.04.1-Ubuntu SMP Thu May 7 15:12:38 UTC 2026", + "Machine": "x86_64", + "CPU": { + "brand_raw": "AMD EPYC 9B45", + "arch": "X86_64", + "hz_advertised_raw": "Unknown", + "count": 16 + }, + "ajpegli": "1.0.0" + }, + "worker_results": [ + { + "num_workers": 0, + "num_warmup": 1, + "images_per_second_mean": 495.0257758269601, + "images_per_second_std": 0.9000896938219153, + "images_per_second_p50": 494.9755224640577, + "images_per_second_p90": 495.91704316212446, + "images_per_second_p99": 496.1288853191895, + "us_per_image_mean": 2020.1035066168, + "us_per_image_p50": 2020.3019232584666, + "us_per_image_p90": 2023.6595372587476, + "us_per_image_p99": 2024.415000408811, + "raw_times_s": [ + 100.77346728199996, + 101.22292253900014, + 101.01307586100006 + ], + "raw_throughput_ips": [ + 496.15242333664116, + 493.94938168018126, + 494.9755224640577 + ] + }, + { + "num_workers": 2, + "num_warmup": 1, + "images_per_second_mean": 917.8155570285107, + "images_per_second_std": 2.5808088732191092, + "images_per_second_p50": 916.7596429170272, + "images_per_second_p90": 920.4472714554936, + "images_per_second_p99": 921.2769878766486, + "us_per_image_mean": 1089.5521296359254, + "us_per_image_p50": 1090.798452709056, + "us_per_image_p90": 1092.1730186643738, + "us_per_image_p99": 1092.4822960043202, + "raw_times_s": [ + 54.62474049100001, + 54.53883183700009, + 54.265978460999804 + ], + "raw_throughput_ips": [ + 915.3178495783948, + 916.7596429170272, + 921.3691785901102 + ] + }, + { + "num_workers": 4, + "num_warmup": 1, + "images_per_second_mean": 1765.2533957377761, + "images_per_second_std": 10.440613074055774, + "images_per_second_p50": 1771.3857780950389, + "images_per_second_p90": 1773.332574046474, + "images_per_second_p99": 1773.770603135547, + "us_per_image_mean": 566.5108058027846, + "us_per_image_p50": 564.5297666753355, + "us_per_image_p90": 569.90384132483, + "us_per_image_p99": 571.1130081209662, + "raw_times_s": [ + 28.18720078199999, + 28.225923804000104, + 28.56179675200019 + ], + "raw_throughput_ips": [ + 1773.8192730343328, + 1771.3857780950389, + 1750.5551360839565 + ] + }, + { + "num_workers": 8, + "num_warmup": 1, + "images_per_second_mean": 2882.0378647531375, + "images_per_second_std": 26.94656360657679, + "images_per_second_p50": 2894.974301880136, + "images_per_second_p90": 2904.284906872454, + "images_per_second_p99": 2906.3797929957254, + "us_per_image_mean": 347.0072136376057, + "us_per_image_p50": 345.426209604193, + "us_per_image_p90": 350.3271055901117, + "us_per_image_p99": 351.4298071869435, + "raw_times_s": [ + 17.201811042999907, + 17.270965054000044, + 17.577264926999987 + ], + "raw_throughput_ips": [ + 2906.6125581205333, + 2894.974301880136, + 2844.526734258742 + ] + } + ], + "num_images": 49999, + "num_images_total": 50000, + "num_images_decoded": 49999, + "num_images_skipped": 1, + "skip_rate": 2e-05, + "skip_indices": [ + 19876 + ], + "skip_examples": [ + "idx=19876: DecodeError: jpegli decode failed: ./lib/jpegli/color_transform.cc:549: Unsupported color transform 4 -> 2" + ], + "num_runs": 3 +} \ No newline at end of file diff --git a/output/linux_AMD-EPYC-9B45/run_summary.json b/output/linux_AMD-EPYC-9B45/run_summary.json index d771286..f5768c9 100644 --- a/output/linux_AMD-EPYC-9B45/run_summary.json +++ b/output/linux_AMD-EPYC-9B45/run_summary.json @@ -1,5 +1,5 @@ { - "timestamp_utc": "2026-04-22T04:23:59.184727+00:00", + "timestamp_utc": "2026-05-20T07:56:38.744173+00:00", "system": "linux_AMD-EPYC-9B45", "mode": "both", "num_images": 50000, @@ -12,33 +12,11 @@ 8 ], "libs_requested": [ - "imagecodecs", - "imageio", - "jpeg4py", - "kornia", - "opencv", - "pillow", - "pyvips", - "simplejpeg", - "skimage", - "tensorflow", - "torchvision", - "turbojpeg" + "ajpegli" ], "libs_skipped_platform": [], "libs_run": [ - "imagecodecs", - "imageio", - "jpeg4py", - "kornia", - "opencv", - "pillow", - "pyvips", - "simplejpeg", - "skimage", - "tensorflow", - "torchvision", - "turbojpeg" + "ajpegli" ], "failures": [], "exit_status": "ok" diff --git a/output/linux_INTEL(R)-XEON(R)-PLATINUM-8581C-CPU-@-2.30GHz/ajpegli_1t_results.json b/output/linux_INTEL(R)-XEON(R)-PLATINUM-8581C-CPU-@-2.30GHz/ajpegli_1t_results.json new file mode 100644 index 0000000..c9164b2 --- /dev/null +++ b/output/linux_INTEL(R)-XEON(R)-PLATINUM-8581C-CPU-@-2.30GHz/ajpegli_1t_results.json @@ -0,0 +1,57 @@ +{ + "library": "ajpegli", + "mode": "memory", + "run_tag": "1t", + "requested_threads": 1, + "effective_threads": 1, + "system_info": { + "Python": "3.12.3", + "OS": "Linux", + "OS Version": "#17~24.04.1-Ubuntu SMP Thu May 7 15:12:38 UTC 2026", + "Machine": "x86_64", + "CPU": { + "brand_raw": "INTEL(R) XEON(R) PLATINUM 8581C CPU @ 2.30GHz", + "arch": "X86_64", + "hz_advertised_raw": "Unknown", + "count": 16 + }, + "ajpegli": "1.0.0" + }, + "benchmark_results": { + "images_per_second_mean": 322.64026591760773, + "images_per_second_std": 0.18115753668035064, + "images_per_second_p50": 322.5440754147659, + "images_per_second_p90": 322.8610102027509, + "images_per_second_p99": 322.8680490550159, + "us_per_image_mean": 3099.4281273665474, + "us_per_image_p50": 3100.351475109502, + "us_per_image_p90": 3101.1020993979873, + "us_per_image_p99": 3101.2256165579306, + "raw_times_s": [ + 155.04171097699998, + 155.05886579499997, + 154.867931527, + 154.85855299800005, + 155.014473404 + ], + "raw_throughput_ips": [ + 322.48741119360596, + 322.45173304764535, + 322.8492787823092, + 322.868831149712, + 322.5440754147659 + ], + "num_images_total": 50000, + "num_images_decoded": 49999, + "num_images_skipped": 1, + "skip_rate": 2e-05, + "skip_indices": [ + 19876 + ], + "skip_examples": [ + "idx=19876: DecodeError: jpegli decode failed: ./lib/jpegli/color_transform.cc:549: Unsupported color transform 4 -> 2" + ] + }, + "num_images": 50000, + "num_runs": 5 +} \ No newline at end of file diff --git a/output/linux_INTEL(R)-XEON(R)-PLATINUM-8581C-CPU-@-2.30GHz/ajpegli_dataloader_results.json b/output/linux_INTEL(R)-XEON(R)-PLATINUM-8581C-CPU-@-2.30GHz/ajpegli_dataloader_results.json new file mode 100644 index 0000000..c04e3f4 --- /dev/null +++ b/output/linux_INTEL(R)-XEON(R)-PLATINUM-8581C-CPU-@-2.30GHz/ajpegli_dataloader_results.json @@ -0,0 +1,125 @@ +{ + "library": "ajpegli", + "benchmark_type": "dataloader", + "batch_size": 32, + "num_threads": 1, + "system_info": { + "Python": "3.12.3", + "OS": "Linux", + "OS Version": "#17~24.04.1-Ubuntu SMP Thu May 7 15:12:38 UTC 2026", + "Machine": "x86_64", + "CPU": { + "brand_raw": "INTEL(R) XEON(R) PLATINUM 8581C CPU @ 2.30GHz", + "arch": "X86_64", + "hz_advertised_raw": "Unknown", + "count": 16 + }, + "ajpegli": "1.0.0" + }, + "worker_results": [ + { + "num_workers": 0, + "num_warmup": 1, + "images_per_second_mean": 316.0411979979685, + "images_per_second_std": 0.6903132920984998, + "images_per_second_p50": 316.4459478420736, + "images_per_second_p90": 316.5757448806198, + "images_per_second_p99": 316.60494921429273, + "us_per_image_mean": 3164.159560164537, + "us_per_image_p50": 3160.0973462269226, + "us_per_image_p90": 3171.142176611533, + "us_per_image_p99": 3173.62726344807, + "raw_times_s": [ + 157.9207390250001, + 158.69199530700007, + 158.0017072139999 + ], + "raw_throughput_ips": [ + 316.6081941402564, + 315.0694520115754, + 316.4459478420736 + ] + }, + { + "num_workers": 2, + "num_warmup": 1, + "images_per_second_mean": 564.2642895922646, + "images_per_second_std": 0.7479586645605707, + "images_per_second_p50": 564.1352699319291, + "images_per_second_p90": 565.0174685049394, + "images_per_second_p99": 565.2159631838667, + "us_per_image_mean": 1772.222298899311, + "us_per_image_p50": 1772.6244985899643, + "us_per_image_p90": 1774.425853481073, + "us_per_image_p99": 1774.8311583315724, + "raw_times_s": [ + 88.7420347340003, + 88.62945230499963, + 88.45654112900002 + ], + "raw_throughput_ips": [ + 563.4195806966726, + 564.1352699319291, + 565.2380181481919 + ] + }, + { + "num_workers": 4, + "num_warmup": 1, + "images_per_second_mean": 1051.6507789544892, + "images_per_second_std": 1.4857629378390995, + "images_per_second_p50": 1052.559594375203, + "images_per_second_p90": 1052.7815470212427, + "images_per_second_p99": 1052.8314863666014, + "us_per_image_mean": 950.8878973312795, + "us_per_image_p50": 950.0649705194105, + "us_per_image_p90": 952.2402817376299, + "us_per_image_p99": 952.7297267617291, + "raw_times_s": [ + 47.50229846100001, + 47.48978078200025, + 47.638252692999686 + ], + "raw_throughput_ips": [ + 1052.559594375203, + 1052.8370351827525, + 1049.555707305512 + ] + }, + { + "num_workers": 8, + "num_warmup": 1, + "images_per_second_mean": 1497.8681551051213, + "images_per_second_std": 15.895543024374971, + "images_per_second_p50": 1507.2954898289122, + "images_per_second_p90": 1510.1213489640195, + "images_per_second_p99": 1510.7571672694187, + "us_per_image_mean": 667.6912427981895, + "us_per_image_p50": 663.4399205384118, + "us_per_image_p90": 674.8839949718999, + "us_per_image_p99": 677.4589117194347, + "raw_times_s": [ + 33.886572934000014, + 33.09377782499996, + 33.17133258700005 + ], + "raw_throughput_ips": [ + 1475.481161738655, + 1510.8278137477964, + 1507.2954898289122 + ] + } + ], + "num_images": 49999, + "num_images_total": 50000, + "num_images_decoded": 49999, + "num_images_skipped": 1, + "skip_rate": 2e-05, + "skip_indices": [ + 19876 + ], + "skip_examples": [ + "idx=19876: DecodeError: jpegli decode failed: ./lib/jpegli/color_transform.cc:549: Unsupported color transform 4 -> 2" + ], + "num_runs": 3 +} \ No newline at end of file diff --git a/output/linux_INTEL(R)-XEON(R)-PLATINUM-8581C-CPU-@-2.30GHz/run_summary.json b/output/linux_INTEL(R)-XEON(R)-PLATINUM-8581C-CPU-@-2.30GHz/run_summary.json index 053139c..a4ce078 100644 --- a/output/linux_INTEL(R)-XEON(R)-PLATINUM-8581C-CPU-@-2.30GHz/run_summary.json +++ b/output/linux_INTEL(R)-XEON(R)-PLATINUM-8581C-CPU-@-2.30GHz/run_summary.json @@ -1,5 +1,5 @@ { - "timestamp_utc": "2026-04-20T04:42:35.048967+00:00", + "timestamp_utc": "2026-05-20T06:48:41.391423+00:00", "system": "linux_INTEL(R)-XEON(R)-PLATINUM-8581C-CPU-@-2.30GHz", "mode": "both", "num_images": 50000, @@ -12,33 +12,11 @@ 8 ], "libs_requested": [ - "imagecodecs", - "imageio", - "jpeg4py", - "kornia", - "opencv", - "pillow", - "pyvips", - "simplejpeg", - "skimage", - "tensorflow", - "torchvision", - "turbojpeg" + "ajpegli" ], "libs_skipped_platform": [], "libs_run": [ - "imagecodecs", - "imageio", - "jpeg4py", - "kornia", - "opencv", - "pillow", - "pyvips", - "simplejpeg", - "skimage", - "tensorflow", - "torchvision", - "turbojpeg" + "ajpegli" ], "failures": [], "exit_status": "ok" diff --git a/output/linux_Neoverse-N1/ajpegli_1t_results.json b/output/linux_Neoverse-N1/ajpegli_1t_results.json new file mode 100644 index 0000000..546ed0f --- /dev/null +++ b/output/linux_Neoverse-N1/ajpegli_1t_results.json @@ -0,0 +1,57 @@ +{ + "library": "ajpegli", + "mode": "memory", + "run_tag": "1t", + "requested_threads": 1, + "effective_threads": 1, + "system_info": { + "Python": "3.12.3", + "OS": "Linux", + "OS Version": "#17~24.04.1-Ubuntu SMP Fri May 8 07:28:18 UTC 2026", + "Machine": "aarch64", + "CPU": { + "brand_raw": "Neoverse-N1", + "arch": "ARM_8", + "hz_advertised_raw": "Unknown", + "count": 16 + }, + "ajpegli": "1.0.0" + }, + "benchmark_results": { + "images_per_second_mean": 273.3220241829019, + "images_per_second_std": 0.12797635904768148, + "images_per_second_p50": 273.2632747988807, + "images_per_second_p90": 273.48103471605333, + "images_per_second_p99": 273.52786375416207, + "us_per_image_mean": 3658.6887652433043, + "us_per_image_p50": 3659.4745515510313, + "us_per_image_p90": 3660.2722502770057, + "us_per_image_p99": 3660.3530267557358, + "raw_times_s": [ + 182.97006810300002, + 183.00322100300002, + 183.014439734, + 182.78960036499984, + 182.87656866199995 + ], + "raw_throughput_ips": [ + 273.2632747988807, + 273.21377036954095, + 273.1970224462638, + 273.53306698061857, + 273.4029863192054 + ], + "num_images_total": 50000, + "num_images_decoded": 49999, + "num_images_skipped": 1, + "skip_rate": 2e-05, + "skip_indices": [ + 19876 + ], + "skip_examples": [ + "idx=19876: DecodeError: jpegli decode failed: ./lib/jpegli/color_transform.cc:549: Unsupported color transform 4 -> 2" + ] + }, + "num_images": 50000, + "num_runs": 5 +} \ No newline at end of file diff --git a/output/linux_Neoverse-N1/ajpegli_dataloader_results.json b/output/linux_Neoverse-N1/ajpegli_dataloader_results.json new file mode 100644 index 0000000..9031d67 --- /dev/null +++ b/output/linux_Neoverse-N1/ajpegli_dataloader_results.json @@ -0,0 +1,125 @@ +{ + "library": "ajpegli", + "benchmark_type": "dataloader", + "batch_size": 32, + "num_threads": 1, + "system_info": { + "Python": "3.12.3", + "OS": "Linux", + "OS Version": "#17~24.04.1-Ubuntu SMP Fri May 8 07:28:18 UTC 2026", + "Machine": "aarch64", + "CPU": { + "brand_raw": "Neoverse-N1", + "arch": "ARM_8", + "hz_advertised_raw": "Unknown", + "count": 16 + }, + "ajpegli": "1.0.0" + }, + "worker_results": [ + { + "num_workers": 0, + "num_warmup": 1, + "images_per_second_mean": 268.16233327336806, + "images_per_second_std": 0.05438074389495371, + "images_per_second_p50": 268.16100735263853, + "images_per_second_p90": 268.2158725677621, + "images_per_second_p99": 268.2282172411649, + "us_per_image_mean": 3729.084651513033, + "us_per_image_p50": 3729.102936598737, + "us_per_image_p90": 3729.8218244484906, + "us_per_image_p99": 3729.983574214685, + "raw_times_s": [ + 186.49634731900005, + 186.45141772700026, + 186.4037454270001 + ], + "raw_throughput_ips": [ + 268.09640359592265, + 268.16100735263853, + 268.229588871543 + ] + }, + { + "num_workers": 2, + "num_warmup": 1, + "images_per_second_mean": 495.7600850572162, + "images_per_second_std": 0.37217675871723255, + "images_per_second_p50": 495.57713252151075, + "images_per_second_p90": 496.1385944664174, + "images_per_second_p99": 496.2649234040214, + "us_per_image_mean": 2017.105841030155, + "us_per_image_p50": 2017.8493606272168, + "us_per_image_p90": 2018.347794227883, + "us_per_image_p99": 2018.459941788033, + "raw_times_s": [ + 100.89045018200022, + 100.92160165899986, + 100.7477729960001 + ], + "raw_throughput_ips": [ + 495.57713252151075, + 495.4241626974937, + 496.278959952644 + ] + }, + { + "num_workers": 4, + "num_warmup": 1, + "images_per_second_mean": 963.3546117125278, + "images_per_second_std": 1.4435598236964764, + "images_per_second_p50": 962.3768259681572, + "images_per_second_p90": 964.7917803885301, + "images_per_second_p99": 965.335145133114, + "us_per_image_mean": 1038.0416836070056, + "us_per_image_p50": 1039.0940149602975, + "us_per_image_p90": 1039.1677322586443, + "us_per_image_p99": 1039.1843186507724, + "raw_times_s": [ + 51.95826889299997, + 51.953661653999916, + 51.79120786900012 + ], + "raw_throughput_ips": [ + 962.2914901758028, + 962.3768259681572, + 965.3955189936233 + ] + }, + { + "num_workers": 8, + "num_warmup": 1, + "images_per_second_mean": 1447.7777900748115, + "images_per_second_std": 14.357776807600311, + "images_per_second_p50": 1451.8981853524733, + "images_per_second_p90": 1460.7287118971515, + "images_per_second_p99": 1462.715580369704, + "us_per_image_mean": 690.7819469322732, + "us_per_image_p50": 688.7535297505952, + "us_per_image_p90": 697.7791615912336, + "us_per_image_p99": 699.8099287553772, + "raw_times_s": [ + 34.43698773400001, + 35.00107844200011, + 34.17715351800007 + ], + "raw_throughput_ips": [ + 1451.8981853524733, + 1428.49884133864, + 1462.936343533321 + ] + } + ], + "num_images": 49999, + "num_images_total": 50000, + "num_images_decoded": 49999, + "num_images_skipped": 1, + "skip_rate": 2e-05, + "skip_indices": [ + 19876 + ], + "skip_examples": [ + "idx=19876: DecodeError: jpegli decode failed: ./lib/jpegli/color_transform.cc:549: Unsupported color transform 4 -> 2" + ], + "num_runs": 3 +} \ No newline at end of file diff --git a/output/linux_Neoverse-N1/run_summary.json b/output/linux_Neoverse-N1/run_summary.json index 4a23434..25e84e5 100644 --- a/output/linux_Neoverse-N1/run_summary.json +++ b/output/linux_Neoverse-N1/run_summary.json @@ -1,5 +1,5 @@ { - "timestamp_utc": "2026-04-22T11:54:57.234170+00:00", + "timestamp_utc": "2026-05-20T09:31:31.728687+00:00", "system": "linux_Neoverse-N1", "mode": "both", "num_images": 50000, @@ -12,33 +12,11 @@ 8 ], "libs_requested": [ - "imagecodecs", - "imageio", - "jpeg4py", - "kornia", - "opencv", - "pillow", - "pyvips", - "simplejpeg", - "skimage", - "tensorflow", - "torchvision", - "turbojpeg" + "ajpegli" ], "libs_skipped_platform": [], "libs_run": [ - "imagecodecs", - "imageio", - "jpeg4py", - "kornia", - "opencv", - "pillow", - "pyvips", - "simplejpeg", - "skimage", - "tensorflow", - "torchvision", - "turbojpeg" + "ajpegli" ], "failures": [], "exit_status": "ok" diff --git a/output/linux_Neoverse-V2/ajpegli_1t_results.json b/output/linux_Neoverse-V2/ajpegli_1t_results.json new file mode 100644 index 0000000..23de595 --- /dev/null +++ b/output/linux_Neoverse-V2/ajpegli_1t_results.json @@ -0,0 +1,57 @@ +{ + "library": "ajpegli", + "mode": "memory", + "run_tag": "1t", + "requested_threads": 1, + "effective_threads": 1, + "system_info": { + "Python": "3.12.3", + "OS": "Linux", + "OS Version": "#17~24.04.1-Ubuntu SMP Fri May 8 07:28:18 UTC 2026", + "Machine": "aarch64", + "CPU": { + "brand_raw": "Neoverse-V2", + "arch": "ARM_8", + "hz_advertised_raw": "Unknown", + "count": 16 + }, + "ajpegli": "1.0.0" + }, + "benchmark_results": { + "images_per_second_mean": 336.5096555686261, + "images_per_second_std": 0.053186774900635385, + "images_per_second_p50": 336.543795657525, + "images_per_second_p90": 336.5522303497023, + "images_per_second_p99": 336.5539952989524, + "us_per_image_mean": 2971.6830065241297, + "us_per_image_p50": 2971.381475169502, + "us_per_image_p90": 2972.2669766435324, + "us_per_image_p99": 2972.488494308286, + "raw_times_s": [ + 148.56610237699994, + 148.62268285599998, + 148.56151335200002, + 148.59191712899997, + 148.56367750200002 + ], + "raw_throughput_ips": [ + 336.543795657525, + 336.41567383387815, + 336.5541914044246, + 336.48532817968425, + 336.5492887676188 + ], + "num_images_total": 50000, + "num_images_decoded": 49999, + "num_images_skipped": 1, + "skip_rate": 2e-05, + "skip_indices": [ + 19876 + ], + "skip_examples": [ + "idx=19876: DecodeError: jpegli decode failed: ./lib/jpegli/color_transform.cc:549: Unsupported color transform 4 -> 2" + ] + }, + "num_images": 50000, + "num_runs": 5 +} \ No newline at end of file diff --git a/output/linux_Neoverse-V2/ajpegli_dataloader_results.json b/output/linux_Neoverse-V2/ajpegli_dataloader_results.json new file mode 100644 index 0000000..5362fbd --- /dev/null +++ b/output/linux_Neoverse-V2/ajpegli_dataloader_results.json @@ -0,0 +1,125 @@ +{ + "library": "ajpegli", + "benchmark_type": "dataloader", + "batch_size": 32, + "num_threads": 1, + "system_info": { + "Python": "3.12.3", + "OS": "Linux", + "OS Version": "#17~24.04.1-Ubuntu SMP Fri May 8 07:28:18 UTC 2026", + "Machine": "aarch64", + "CPU": { + "brand_raw": "Neoverse-V2", + "arch": "ARM_8", + "hz_advertised_raw": "Unknown", + "count": 16 + }, + "ajpegli": "1.0.0" + }, + "worker_results": [ + { + "num_workers": 0, + "num_warmup": 1, + "images_per_second_mean": 337.17803670826714, + "images_per_second_std": 0.09635851932368043, + "images_per_second_p50": 337.152652797203, + "images_per_second_p90": 337.2758727545173, + "images_per_second_p99": 337.3035972449131, + "us_per_image_mean": 2965.792468049361, + "us_per_image_p50": 2966.015517610354, + "us_per_image_p90": 2966.5637002060053, + "us_per_image_p99": 2966.6870412900266, + "raw_times_s": [ + 148.2978098650001, + 148.23009237299993, + 148.33207059200004 + ], + "raw_throughput_ips": [ + 337.152652797203, + 337.3066777438459, + 337.07477958375233 + ] + }, + { + "num_workers": 2, + "num_warmup": 1, + "images_per_second_mean": 633.8903019941181, + "images_per_second_std": 0.5335495211347878, + "images_per_second_p50": 633.8048328463738, + "images_per_second_p90": 634.4268009639374, + "images_per_second_p99": 634.5667437903892, + "us_per_image_mean": 1577.5611413894949, + "us_per_image_p50": 1577.772759335187, + "us_per_image_p90": 1578.811286185724, + "us_per_image_p99": 1579.0449547270948, + "raw_times_s": [ + 78.79041150700004, + 78.88706019400001, + 78.95196682400001 + ], + "raw_throughput_ips": [ + 634.5822929933282, + 633.8048328463738, + 633.2837801426523 + ] + }, + { + "num_workers": 4, + "num_warmup": 1, + "images_per_second_mean": 1258.9206679021615, + "images_per_second_std": 0.6530444296624627, + "images_per_second_p50": 1258.989665265139, + "images_per_second_p90": 1259.5449305274076, + "images_per_second_p99": 1259.669865211418, + "us_per_image_mean": 794.331441688836, + "us_per_image_p50": 794.2876955939137, + "us_per_image_p90": 794.7428061361231, + "us_per_image_p99": 794.8452060081202, + "raw_times_s": [ + 39.713590492000094, + 39.69170843500024, + 39.742034332 + ], + "raw_throughput_ips": [ + 1258.989665265139, + 1259.6837468429746, + 1258.0885915983713 + ] + }, + { + "num_workers": 8, + "num_warmup": 1, + "images_per_second_mean": 2396.6259315465195, + "images_per_second_std": 3.531830395243404, + "images_per_second_p50": 2396.544663160484, + "images_per_second_p90": 2400.1022000674607, + "images_per_second_p99": 2400.9026458715307, + "us_per_image_mean": 417.2541732234615, + "us_per_image_p50": 417.26741644832646, + "us_per_image_p90": 417.8538956979106, + "us_per_image_p99": 417.98585352906707, + "raw_times_s": [ + 20.824312890999863, + 20.862953554999876, + 20.899607774999822 + ], + "raw_throughput_ips": [ + 2400.991584294205, + 2396.544663160484, + 2392.3415471848693 + ] + } + ], + "num_images": 49999, + "num_images_total": 50000, + "num_images_decoded": 49999, + "num_images_skipped": 1, + "skip_rate": 2e-05, + "skip_indices": [ + 19876 + ], + "skip_examples": [ + "idx=19876: DecodeError: jpegli decode failed: ./lib/jpegli/color_transform.cc:549: Unsupported color transform 4 -> 2" + ], + "num_runs": 3 +} \ No newline at end of file diff --git a/output/linux_Neoverse-V2/run_summary.json b/output/linux_Neoverse-V2/run_summary.json index 5f178ed..01a26e3 100644 --- a/output/linux_Neoverse-V2/run_summary.json +++ b/output/linux_Neoverse-V2/run_summary.json @@ -1,5 +1,5 @@ { - "timestamp_utc": "2026-04-20T14:55:02.246242+00:00", + "timestamp_utc": "2026-05-20T08:38:10.022642+00:00", "system": "linux_Neoverse-V2", "mode": "both", "num_images": 50000, @@ -12,33 +12,11 @@ 8 ], "libs_requested": [ - "imagecodecs", - "imageio", - "jpeg4py", - "kornia", - "opencv", - "pillow", - "pyvips", - "simplejpeg", - "skimage", - "tensorflow", - "torchvision", - "turbojpeg" + "ajpegli" ], "libs_skipped_platform": [], "libs_run": [ - "imagecodecs", - "imageio", - "jpeg4py", - "kornia", - "opencv", - "pillow", - "pyvips", - "simplejpeg", - "skimage", - "tensorflow", - "torchvision", - "turbojpeg" + "ajpegli" ], "failures": [], "exit_status": "ok" diff --git a/pyproject.toml b/pyproject.toml index 4b31a8a..2b4b797 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -22,12 +22,13 @@ dependencies = [ ] # Two groups, not thirteen. Real conflicts only exist between these two: # tensorflow vs torch — numpy/protobuf pin fights -# Everything else (opencv, skimage, imageio, kornia-rs, simplejpeg, turbojpeg, -# imagecodecs, pyvips, jpeg4py, torch, torchvision, Pillow) coexists in one venv. +# Everything else (ajpegli, opencv, skimage, imageio, kornia-rs, simplejpeg, +# turbojpeg, imagecodecs, pyvips, jpeg4py, torch, torchvision, Pillow) coexists in one venv. # # Pillow-SIMD WAS a third group (incompatible with mainstream Pillow). Dropped # 2026-04 — see docs/gcp_benchmarks.md "Why no Pillow-SIMD?" for rationale. optional-dependencies.mainstream = [ + "ajpegli", "imagecodecs", "imageio", "jpeg4py; sys_platform=='linux'", @@ -48,6 +49,7 @@ optional-dependencies.mainstream = [ optional-dependencies.plot = [ "matplotlib>=3.8", "seaborn>=0.13" ] optional-dependencies.tensorflow = [ "tensorflow" ] scripts.imread-benchmark = "imread_benchmark.cli:app" +entry-points."imread_benchmark.decoders".ajpegli = "imread_benchmark.decoders.ajpegli_decoder:AJpegliDecoder" entry-points."imread_benchmark.decoders".imagecodecs = "imread_benchmark.decoders.imagecodecs_decoder:ImageCodecsDecoder" entry-points."imread_benchmark.decoders".imageio = "imread_benchmark.decoders.imageio_decoder:ImageIODecoder" entry-points."imread_benchmark.decoders".jpeg4py = "imread_benchmark.decoders.jpeg4py_decoder:Jpeg4pyDecoder" diff --git a/tests/conftest.py b/tests/conftest.py index 273da2e..42c95c1 100644 --- a/tests/conftest.py +++ b/tests/conftest.py @@ -15,6 +15,7 @@ # Maps PyPI distribution name → top-level importable module. Drives test # skipping for decoders whose third-party library isn't installed in this venv. PKG_TO_MODULE = { + "ajpegli": "ajpegli", "opencv-python-headless": "cv2", "pillow": "PIL", "scikit-image": "skimage", diff --git a/tests/test_decoders.py b/tests/test_decoders.py index 03353d2..3aae90f 100644 --- a/tests/test_decoders.py +++ b/tests/test_decoders.py @@ -69,6 +69,7 @@ def test_decoder_class_contract(name, cls): def test_registry_has_known_decoders(): """Entry-point discovery in pyproject.toml shouldn't silently drop decoders.""" expected = { + "ajpegli", "opencv", "pillow", "skimage", diff --git a/tests/test_paper_assets.py b/tests/test_paper_assets.py index f400db9..cb73cdf 100644 --- a/tests/test_paper_assets.py +++ b/tests/test_paper_assets.py @@ -65,10 +65,10 @@ def test_paper_data_matrix_matches_claimed_scope() -> None: summary = validate_paper_data(load_results(root), load_dataloader_results(root)) assert summary["platforms"] == 5 - assert summary["single_thread_decoders"] == 12 - assert summary["dataloader_decoders"] == 10 - assert summary["single_thread_rows"] == 60 - assert summary["dataloader_worker_rows"] == 200 + assert summary["single_thread_decoders"] == 13 + assert summary["dataloader_decoders"] == 11 + assert summary["single_thread_rows"] == 65 + assert summary["dataloader_worker_rows"] == 220 def test_validate_paper_data_missing_platform_raises() -> None: diff --git a/tools/_results.py b/tools/_results.py index 20150db..3985f7a 100644 --- a/tools/_results.py +++ b/tools/_results.py @@ -17,6 +17,7 @@ "turbojpeg", "jpeg4py", "kornia-rs", + "ajpegli", "opencv", "imagecodecs", "pyvips", diff --git a/tools/paper_assets.py b/tools/paper_assets.py index e00ffec..3ec8c6f 100644 --- a/tools/paper_assets.py +++ b/tools/paper_assets.py @@ -89,8 +89,8 @@ def _require_plotting() -> tuple[ModuleType, ModuleType, type]: WORKERS_ORDER = (0, 2, 4, 8) EXPECTED_SINGLE_DECODERS = set(LIBRARY_ORDER) EXPECTED_DATALOADER_DECODERS = set(LIBRARY_ORDER) - {"pyvips", "tensorflow"} -EXPECTED_SKIP_DECODERS = {"jpeg4py", "kornia-rs", "turbojpeg"} -ROBUSTNESS_DECODERS = ("jpeg4py", "kornia-rs", "turbojpeg", "pyvips", "tensorflow") +EXPECTED_SKIP_DECODERS = {"ajpegli", "jpeg4py", "kornia-rs", "turbojpeg"} +ROBUSTNESS_DECODERS = ("ajpegli", "jpeg4py", "kornia-rs", "turbojpeg", "pyvips", "tensorflow") FIG_DPI = 300 CLAIM_FIGURE_BASENAMES: tuple[str, ...] = ( @@ -482,7 +482,7 @@ def generate_platform_recommendation_table(dl: pd.DataFrame, dest: Path) -> None r"universal recommendation.}" ) note = ( - r" {\footnotesize Strict libjpeg-turbo-family wrappers may be fast but skipped one ImageNet JPEG; " + r" {\footnotesize Strict native JPEG decoders/wrappers may be fast but skipped one ImageNet JPEG; " r"PyVips and TensorFlow are not PyTorch \texttt{DataLoader} choices in this harness.\par}" ) dest.write_text( @@ -560,6 +560,7 @@ def _dataloader_eligibility(input_dir: Path, library: str) -> str: def generate_robustness_table(input_dir: Path, df_1t: pd.DataFrame, dest: Path) -> None: headers = ["Decoder", "DataLoader eligibility", "Skipped images", "Example failure", "Interpretation"] interpretations = { + "ajpegli": "Fast path, but rejects the same uncommon ImageNet image.", "jpeg4py": "Fast path, but needs an explicit CMYK fallback policy.", "kornia-rs": "Fast path, but rejects the same uncommon ImageNet image.", "turbojpeg": "Fast path, but needs an explicit CMYK fallback policy.", @@ -680,18 +681,26 @@ def plot_fig02_amd_worker_delta(dl: pd.DataFrame, out_dir: Path, formats: tuple[ _require_plotting() dl = _paper_scope(dl) libs = [lib for lib in _ordered_libs_present(dl) if lib not in {"pyvips", "tensorflow"}] + delta_by_platform: dict[str, list[float]] = {} + for plat in (ZEN4_PLATFORM, ZEN5_PLATFORM): + deltas = [] + for lib in libs: + w4 = _ips_at_workers(dl, plat, lib, 4) + w8 = _ips_at_workers(dl, plat, lib, 8) + deltas.append(np.nan if w4 is None or w4 == 0 or w8 is None else 100.0 * (w8 / w4 - 1.0)) + delta_by_platform[plat] = deltas + finite_deltas = [d for deltas in delta_by_platform.values() for d in deltas if not pd.isna(d)] + x_min = min(-18.0, min(finite_deltas) - 5.0) + x_max = max(32.0, max(finite_deltas) + 8.0) + fig, axes = plt.subplots(1, 2, figsize=(11, 5.6), sharey=True, constrained_layout=True) labels = {ZEN4_PLATFORM: "AMD Zen 4", ZEN5_PLATFORM: "AMD Zen 5"} y = np.arange(len(libs)) for ax, plat in zip(axes, (ZEN4_PLATFORM, ZEN5_PLATFORM), strict=True): - deltas = [] + deltas = delta_by_platform[plat] colors = [] - for lib in libs: - w4 = _ips_at_workers(dl, plat, lib, 4) - w8 = _ips_at_workers(dl, plat, lib, 8) - delta = np.nan if w4 is None or w4 == 0 or w8 is None else 100.0 * (w8 / w4 - 1.0) - deltas.append(delta) + for delta in deltas: if pd.isna(delta): colors.append("0.68") else: @@ -707,7 +716,7 @@ def plot_fig02_amd_worker_delta(dl: pd.DataFrame, out_dir: Path, formats: tuple[ ax.set_title(labels[plat], fontsize=10, fontweight="bold") ax.set_xlabel("Throughput change from w=4 to w=8") ax.grid(True, axis="x", alpha=0.25) - ax.set_xlim(-18, 32) + ax.set_xlim(x_min, x_max) axes[0].set_yticks(y) axes[0].set_yticklabels(libs) diff --git a/uv.lock b/uv.lock index 615e7b7..c0eb332 100644 --- a/uv.lock +++ b/uv.lock @@ -29,6 +29,32 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/18/a6/907a406bb7d359e6a63f99c313846d9eec4f7e6f7437809e03aa00fa3074/absl_py-2.4.0-py3-none-any.whl", hash = "sha256:88476fd881ca8aab94ffa78b7b6c632a782ab3ba1cd19c9bd423abc4fb4cd28d", size = 135750 }, ] +[[package]] +name = "ajpegli" +version = "1.0.0" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "numpy" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/30/d9/9e1ebfed347fa500fe027a19e112f5a252ecf935cfccd6ded0a7e601fbad/ajpegli-1.0.0.tar.gz", hash = "sha256:58e52b7a51f3bf1631d574869c0123c8d525edbd2893cfa44e26759cd784713d", size = 97407485 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/a5/72/39ada8422379778f8d64444b2ffe3f552deebd328b3370391535d7301e36/ajpegli-1.0.0-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:95341aab9701da44561aab86b938ad7cb9a214b3eaed41893ab52c159f7aaa83", size = 273768 }, + { url = "https://files.pythonhosted.org/packages/ab/57/1cec79331aeec49a67ea656c78059886cc0a367c514c66bc9195e9e90d40/ajpegli-1.0.0-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:d8b8f3f21baa47fb539701ffbad50287a4e54c5d7658096bf43f321f0050feae", size = 195503 }, + { url = "https://files.pythonhosted.org/packages/8d/09/5836b713f16dc1ba90c3b5d09a895d3eaf8c2f0a8b962de5a7697504625e/ajpegli-1.0.0-cp311-cp311-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:7e6e034a181494d149328c02c07c3e246dac09d6717b45e7d56f9f763e9a8791", size = 285550 }, + { url = "https://files.pythonhosted.org/packages/2a/90/49e973b99957da3a56b2e485d4b6d68fd9f906f6c47ef42026fd11ed897b/ajpegli-1.0.0-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:9638b916e236900e099528e30cda109cfb26e3a6a58b29734fd9791b6f965fe0", size = 342508 }, + { url = "https://files.pythonhosted.org/packages/70/10/5bf1eeaf54466f18765518e3da804471cf4488d916fd242c5298397c3bea/ajpegli-1.0.0-cp311-cp311-win_amd64.whl", hash = "sha256:a014c356e2c0bb1684e148d876aecce7909655873d527f0b7531799903b5f6a7", size = 257437 }, + { url = "https://files.pythonhosted.org/packages/b4/06/a0e3c74972771db6212001c06ab86430bd721c19ecd4b5e6c2b4fd55d1e8/ajpegli-1.0.0-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:669e1d0e991f6597e816026cf79a1f7c4dc6825f5494d0ed1e5b803255d26df2", size = 277841 }, + { url = "https://files.pythonhosted.org/packages/d6/4d/a16c18aa5e893be388d15fa6994c5ef2d18ac645d8955d6d973b855560e1/ajpegli-1.0.0-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:3bb23026e4098ad3089a0fc5a15da39816c5e4b8cc18e2601c5177bc7ac7a3ef", size = 197174 }, + { url = "https://files.pythonhosted.org/packages/cd/e9/994546434e06f82f72d98bf6902ade62241f18c455f2303c2c5a77569cca/ajpegli-1.0.0-cp312-cp312-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:bc6fc8ae251517fac4a0996194bb65fa884079b23ed8cafb2a928d3fdcb3f333", size = 287717 }, + { url = "https://files.pythonhosted.org/packages/8b/7a/9c001b02ba80c4de629bc63038929ad898dafe05eb4df5dc7a17d8896b64/ajpegli-1.0.0-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:66edffabdaf5e84034793db4bbcb4f462aca332244e2a92c6878621481fce245", size = 344521 }, + { url = "https://files.pythonhosted.org/packages/0b/19/97b491e3c51a39648775393a55196d129153abfa5f62105302b0cef0afe8/ajpegli-1.0.0-cp312-cp312-win_amd64.whl", hash = "sha256:3b9b7b7b3ce9fd042da068d06ba2fb08c64595b2e0aba4420dcf0f49ece12bcd", size = 259641 }, + { url = "https://files.pythonhosted.org/packages/61/9a/86e0cd693cf77290a587dafd2fc7942eb9384eecf5e196b6c3c9f36f3e83/ajpegli-1.0.0-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:7681ed14c9cc89a844de21c4feb824f504fca5dad08c89a264429dd35aea4c41", size = 277902 }, + { url = "https://files.pythonhosted.org/packages/d9/22/f124f98e63da211e0b38dc9d155f8ddd84ffc2c64ddff2b5115456dbd80b/ajpegli-1.0.0-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:d51799aa8786128738e864eb7c8770714864eb7840a6061cf6e90e9461f371d3", size = 197259 }, + { url = "https://files.pythonhosted.org/packages/77/21/b4eb5a4b9a147829a8923d69f8dcd9c15dea972f025276773bae0d26f454/ajpegli-1.0.0-cp313-cp313-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:1435aea7cb962eddd6e73206b7b3e49b2e7aa1a0c5daadcb9c8d42cecbf30df1", size = 287732 }, + { url = "https://files.pythonhosted.org/packages/f2/53/1fa8bb7ce2a01e554bf905ae4fd2bd1ad99d4de33211e01c0401f91ce0a3/ajpegli-1.0.0-cp313-cp313-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:797f2deeaf71d43d4aab64eab2cdef93126d3cfdc7fadfdc16c78e01af5cba05", size = 344707 }, + { url = "https://files.pythonhosted.org/packages/d6/ac/a825b22dcc8e7278773b1dc7c3d8aa0ddb3d04d3bb891176dd464becf4f6/ajpegli-1.0.0-cp313-cp313-win_amd64.whl", hash = "sha256:d78157fb08e5718eb96d5438e28ca90a047d0fc4b0b32d6ca6b0b10a45c13811", size = 259717 }, +] + [[package]] name = "annotated-doc" version = "0.0.4" @@ -592,6 +618,7 @@ dependencies = [ [package.optional-dependencies] mainstream = [ + { name = "ajpegli" }, { name = "imagecodecs" }, { name = "imageio" }, { name = "jpeg4py", marker = "sys_platform == 'linux'" }, @@ -625,6 +652,7 @@ dev = [ [package.metadata] requires-dist = [ + { name = "ajpegli", marker = "extra == 'mainstream'" }, { name = "imagecodecs", marker = "extra == 'mainstream'" }, { name = "imageio", marker = "extra == 'mainstream'" }, { name = "jpeg4py", marker = "sys_platform == 'linux' and extra == 'mainstream'" },