From bb8819817dd1707e7844a36e5b7766384fd4e0cc Mon Sep 17 00:00:00 2001 From: "Jonathan J. Helmus" Date: Fri, 24 Jul 2026 08:12:27 -0500 Subject: [PATCH 1/3] Add information about pbs performance --- BENCHMARKS.md | 47 +++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 47 insertions(+) create mode 100644 BENCHMARKS.md diff --git a/BENCHMARKS.md b/BENCHMARKS.md new file mode 100644 index 000000000..72c2b8c6a --- /dev/null +++ b/BENCHMARKS.md @@ -0,0 +1,47 @@ +## python-build-standalone performance + +Shows a chart with violin plots with benchmark results + +python-build-standalone provides high-performance CPython distributions designed for compatibility across a broad range of Linux distributions, macOS releases, and Windows versions. It's builds incorporate compiler optimizations such as profile-guided optimization (PGO), link-time optimization (LTO), and, where appropriate, BOLT post-link binary optimization. Together, these techniques improve runtime performance while preserving portability. +Benchmarks using the pyperformance suite show that python-build-standalone performs competitively with, and frequently outperforms, other widely used CPython distributions. + + +### Figure details + +The figure above compares CPython 3.14.6 performance across several distributions using pyperformance. Each violin represents the distribution of per-benchmark mean runtime ratios between an alternative CPython distribution and python-build-standalone on the same platform and architecture. Ratios greater than 1 indicate that python-build-standalone was faster; ratios less than 1 indicate that the alternative distribution was faster. + +The horizontal axis uses a logarithmic scale. The vertical marker within each violin indicates the geometric mean of the runtime ratios, which is also expressed as a percentage beside each distribution. + +From top to bottom, the distributions shown are: +* The Docker `python:3.14` image for x86-64, providing CPython 3.14.6. +* A conda-forge Python 3.14.6 environment for `linux-64`. +* The system Python 3.14.6 in a `fedora:44` x86-64 Docker container. +* The system Python 3.14.6 in a debian:forky x86-64 Docker container. +* CPython 3.14.6 from the Python.org macOS installer on an arm64 Mac. +* A conda-forge Python 3.14.6 environment for `osx-arm64`. +* CPython 3.14.6 installed through Homebrew on an arm64 Mac. +* CPython 3.14.6 from the Python.org Windows installer for x86-64. +* A conda-forge Python 3.14.6 environment for `win-64`. + +The reference interpreter for each comparison was the corresponding platform- and architecture-matched python-build-standalone CPython 3.14.6 distribution from the [`20260623` release](https://github.com/astral-sh/python-build-standalone/releases#release-20260623), installed using `uv`. + +Benchmarks were run in early to mid-July 2026 and reflect the distributions and packages available during that period. + +### Benchmark methodology + +Benchmarks were executed from a virtual environment created with the reference interpreter into which pyperformance 1.14.0 was installed. Results were collected using: +``` shell +pyperformance run --rigorous --warmups 2 --output +``` + +The complete benchmark suite was run at least twice to assess consistency. + +Linux benchmarks were run inside Docker containers on an Ubuntu 24.04 host with an Intel Core i9-9900K processor. Hyper-Threading and Intel SpeedStep were disabled. + +macOS benchmarks were run on a MacBook Pro with an Apple M5 Max processor. + +Windows benchmarks were run on a Windows 11 host with an Intel Core i5-9500 processor. Intel Turbo Boost was disabled; this processor does not support Hyper-Threading. From 071e9e7a8f71262de0a115008c059bebb205cd08 Mon Sep 17 00:00:00 2001 From: "Jonathan J. Helmus" Date: Mon, 17 Aug 2026 13:14:45 -0500 Subject: [PATCH 2/3] update BENCHMARKS.md based on review --- BENCHMARKS.md | 12 ++++-------- 1 file changed, 4 insertions(+), 8 deletions(-) diff --git a/BENCHMARKS.md b/BENCHMARKS.md index 72c2b8c6a..ae554c234 100644 --- a/BENCHMARKS.md +++ b/BENCHMARKS.md @@ -6,13 +6,7 @@ src="https://github.com/user-attachments/assets/5499f9ce-ee02-4485-baa2-9d982fe457e6" /> -python-build-standalone provides high-performance CPython distributions designed for compatibility across a broad range of Linux distributions, macOS releases, and Windows versions. It's builds incorporate compiler optimizations such as profile-guided optimization (PGO), link-time optimization (LTO), and, where appropriate, BOLT post-link binary optimization. Together, these techniques improve runtime performance while preserving portability. -Benchmarks using the pyperformance suite show that python-build-standalone performs competitively with, and frequently outperforms, other widely used CPython distributions. - - -### Figure details - -The figure above compares CPython 3.14.6 performance across several distributions using pyperformance. Each violin represents the distribution of per-benchmark mean runtime ratios between an alternative CPython distribution and python-build-standalone on the same platform and architecture. Ratios greater than 1 indicate that python-build-standalone was faster; ratios less than 1 indicate that the alternative distribution was faster. +The figure above compares CPython 3.14.6 performance across several distributions using pyperformance. Each [violin](https://en.wikipedia.org/wiki/Violin_plot) represents the distribution of per-benchmark mean runtime ratios between an alternative CPython distribution and python-build-standalone on the same platform and architecture. Ratios greater than 1 indicate that python-build-standalone was faster; ratios less than 1 indicate that the alternative distribution was faster. The horizontal axis uses a logarithmic scale. The vertical marker within each violin indicates the geometric mean of the runtime ratios, which is also expressed as a percentage beside each distribution. @@ -21,7 +15,7 @@ From top to bottom, the distributions shown are: * A conda-forge Python 3.14.6 environment for `linux-64`. * The system Python 3.14.6 in a `fedora:44` x86-64 Docker container. * The system Python 3.14.6 in a debian:forky x86-64 Docker container. -* CPython 3.14.6 from the Python.org macOS installer on an arm64 Mac. +* CPython 3.14.6 from the [Python.org macOS installer](https://www.python.org/ftp/python/3.14.6/python-3.14.6-macos11.pkg) on an arm64 Mac. * A conda-forge Python 3.14.6 environment for `osx-arm64`. * CPython 3.14.6 installed through Homebrew on an arm64 Mac. * CPython 3.14.6 from the Python.org Windows installer for x86-64. @@ -31,6 +25,8 @@ The reference interpreter for each comparison was the corresponding platform- an Benchmarks were run in early to mid-July 2026 and reflect the distributions and packages available during that period. +Supporting material, including [Dockerfiles](https://github.com/jjhelmus/cpython-benchmarks/tree/main/containers), commands for creating conda environments, and scripts for running the benchmarks, can be found in [jjhelmus/cpython-benchmarks](https://github.com/jjhelmus/cpython-benchmarks). This repository also contains the raw data and the [script](https://github.com/jjhelmus/cpython-benchmarks/blob/main/plot_pbs_314_comparison.py) used to produce the figure. + ### Benchmark methodology Benchmarks were executed from a virtual environment created with the reference interpreter into which pyperformance 1.14.0 was installed. Results were collected using: From ea86f1b12d0b0ded1ca315ebe1146d866690f52b Mon Sep 17 00:00:00 2001 From: "Jonathan J. Helmus" Date: Tue, 18 Aug 2026 12:41:08 -0500 Subject: [PATCH 3/3] Update BENCHMARKS.md Co-authored-by: Brent Westbrook <36778786+ntBre@users.noreply.github.com> --- BENCHMARKS.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/BENCHMARKS.md b/BENCHMARKS.md index ae554c234..74da21e03 100644 --- a/BENCHMARKS.md +++ b/BENCHMARKS.md @@ -14,7 +14,7 @@ From top to bottom, the distributions shown are: * The Docker `python:3.14` image for x86-64, providing CPython 3.14.6. * A conda-forge Python 3.14.6 environment for `linux-64`. * The system Python 3.14.6 in a `fedora:44` x86-64 Docker container. -* The system Python 3.14.6 in a debian:forky x86-64 Docker container. +* The system Python 3.14.6 in a `debian:forky` x86-64 Docker container. * CPython 3.14.6 from the [Python.org macOS installer](https://www.python.org/ftp/python/3.14.6/python-3.14.6-macos11.pkg) on an arm64 Mac. * A conda-forge Python 3.14.6 environment for `osx-arm64`. * CPython 3.14.6 installed through Homebrew on an arm64 Mac.