Verify any claim · lenz.io
“Python 3.12 runs about 20% faster than Python 3.11 on the official Python benchmark suite.”
The conclusion
Official benchmarks do not support a 20% speedup. Python 3.12 was reported as roughly 4–5% faster overall than Python 3.11, with results varying by workload and platform. The larger figure appears to confuse this comparison with Python 3.11's performance gain over Python 3.10.
Caveats
- Benchmark performance varies by workload, hardware, operating system, and build configuration.
- The 20% figure likely conflates Python 3.12's results with Python 3.11's larger gain over Python 3.10.
- An overall benchmark average does not mean every program runs faster.
Fact-check inside the tools you already use
Connect Lenz to ChatGPT, Claude, or WhatsApp and check a claim mid-conversation.
Or create a free account to bookmark this verification and run your own checks.
Sources
Sources used in the analysis
Many large and small performance improvements(like PEP 709 and support for the BOLT binary optimizer), delivering an estimated 5% overall performance improvementcitation needed.
Many large and small performance improvements <https://docs.python.org/3.12/whatsnew/3.12.html#optimizations> (like PEP 709 <https://peps.python.org/pep-0709/> and support for the BOLT binary optimizer), delivering an estimated 5% overall performance improvement.
Python 3.11 is between 10-60% faster than Python 3.10. On average, we measured a 1.25x speedup on the standard benchmark suite.
Overall, according to the Faster CPython benchmarks for the latest release, Python 3.12 outperforms 3.11 by about 1.04x.
For the results above, the "faster/slower" result is a geometric mean of each of the benchmarks.
The release notes talk about a 5% gain in general, an underwhelming number, but I wanted to check by myself.
In the first test, 3.12 and 3.13 were slightly faster than 3.11, but then on this new test the results reversed, with 3.11 being the fastest. … For this reason I'm also seeing 3.11, 3.12 and 3.13 as a group with similar performance characteristics.
The pyperformance benchmark suite has multiple goals: - Help to detect performance regression in a Python implementation - Validate that an optimization change makes Python faster and don’t performance regressions, or only minor regressions - Compare two implementations of Python, for example CPython and PyPy
The Python Performance Benchmark suite is the standard set of benchmarks used in open-source Python optimization work. During the 3.12 development cycle, we contributed several new benchmarks to it so that it more accurately represents workload characteristics we see at Meta.
All tests were run on Windows 11 computers using the pyperformance 1.0.9 library in Python 3.11.6 and Python 3.12.0 (both 64-bit versions). … | Result (geometric mean) | | 1.01x slower | … | Result (geometric mean) | | 1.05x faster |
P.S.: I have come across this more holistic comparison of 3.12 to 3.11. It had run `pyperformance 1.0.9` on AMD Ryzen 9 7900 and Intel Core i3-1315U, both are x86 CPUs. What is curious is that on average 3.12 was ~10% slower on AMD while ~5% faster on Intel.
Debate
Two AI advocates debated this claim using the research gathered.
Argument for
Official Faster CPython tracking and release materials establish that 3.12 delivers measurable geometric-mean gains over 3.11 on the standard pyperformance suite (Sources 1, 2, 4, 5, 8, 9), continuing the same benchmark-driven optimization program that produced large suite-wide speedups in the prior cycle (Source 3). Independent runs and Meta's expanded official benchmarks further confirm net faster 3.12 results on representative workloads, so the claim of a substantial ~20%-class improvement on the official suite is the correct reading of that cumulative evidence.
The Proponent commits a non sequitur by treating Python 3.11's separate 1.25× gain over 3.10 (Source 3, docs.python.org) as evidence for a 20% 3.12-over-3.11 gain; the directly relevant official materials instead estimate 5% overall (Sources 1 and 2, python.org and Python-announce), and Faster CPython reports about 1.04× (Source 4, LWN). Meta's statement that it added benchmarks to better reflect its own workloads (Source 9, Meta Engineering) neither reports a 20% result nor establishes the asserted official-suite average, while the cited geometric-mean methodology (Source 5, Faster CPython) cannot transform the reported 4–5% gains into 20%.
Argument against
The official Python 3.12 release material estimates only a 5% overall performance improvement, not 20% (Source 1, python.org; Source 2, Python-announce), while Faster CPython benchmarks reported approximately 1.04× performance versus 3.11—about 4% faster (Source 4, LWN). Independent pyperformance testing found results ranging from 1% slower to 5% faster (Source 10), and another comparison found 3.12 roughly 10% slower on AMD and 5% faster on Intel (Source 11), decisively contradicting an asserted official-suite gain of about 20%.
The Opponent fixates on the conservative 5% release-note estimate in Sources 1 and 2 while downplaying the Faster CPython geometric-mean result of roughly 1.04× on the official suite (Source 4) and the same benchmark methodology that documented large suite-wide gains in the prior cycle (Sources 3, 5, 8, 9), thereby treating a single rounded figure as dispositive rather than part of cumulative measured improvement. Independent runs in Sources 10 and 11 show platform variance yet still include net-faster outcomes, so they do not refute the official-suite evidence that 3.12 is faster than 3.11 at a scale the claim correctly characterizes as substantial.
Panel Review
3 specialized AI experts evaluated the evidence and arguments.
Reviewer 1 — The Logic Examiner
The direct evidence from python.org's own release notes (Sources 1, 2), the Faster CPython benchmark tracker (Source 4), and Meta's benchmark contributions all converge on a roughly 4-5% improvement, not 20%; independent third-party runs (Sources 10, 11) show even more mixed results, some showing 3.12 slower than 3.11 on certain platforms. The Proponent's case relies on conflating Python 3.11's separate 1.25x speedup over 3.10 (Source 3, a different comparison entirely) with the 3.12-vs-3.11 comparison, which is a non sequitur/hasty generalization that inflates the actual measured gain; the Opponent's rebuttal correctly identifies this fallacy and the direct evidence overwhelmingly refutes the 20% figure.
Reviewer 2 — The Source Auditor
The most reliable sources, including official release notes from python.org (Source 1) and LWN.net (Source 4), state that Python 3.12 delivers an estimated 4% to 5% performance improvement over Python 3.11, not 20%. The 20% figure appears to mistakenly conflate the performance gains of Python 3.11 over 3.10 (Source 3) with the much smaller gains of 3.12 over 3.11.
Reviewer 3 — The Precision Analyst
The directly relevant evidence reports an estimated 5% overall improvement for Python 3.12 (Sources 1 and 2) and about 1.04× performance versus 3.11 on Faster CPython benchmarks (Source 4), which is roughly 4% faster rather than 20%. The claim is false as worded because its approximately 20% figure materially overstates the reported 3.12-over-3.11 benchmark gain.
Panel summary
Official Python release materials and the Faster CPython benchmark data report an overall improvement of roughly 4–5% for Python 3.12 over 3.11. Independent tests show workload- and platform-dependent results, including occasional regressions, but provide no support for a suite-wide 20% gain. The apparent basis for 20% is a conflation with Python 3.11's substantially larger improvement over Python 3.10. The claim therefore fails on source support, inferential validity, and quantitative accuracy.