PyPy v7.3.4: release of 2.7 and 3.7¶
The PyPy team is proud to release the version 7.3.4 of PyPy, which includes two different interpreters:
- PyPy2.7, which is an interpreter supporting the syntax and the features of Python 2.7 including the stdlib for CPython 2.7.18+ (the
+
is for backported security updates)- PyPy3.7, which is an interpreter supporting the syntax and the features of Python 3.7, including the stdlib for CPython 3.7.10. We no longer refer to this as beta-quality as the last incompatibilities with CPython (in the
re
module) have been fixed.
We are no longer releasing a Python3.6 version, as we focus on updating to Python 3.8. We have begun streaming the advances towards this goal on Saturday evenings European time on https://www.twitch.tv/pypyproject. If Python3.6 is important to you, please reach out as we could offer sponsored longer term support.
The two interpreters are based on much the same codebase, thus the multiple release. This is a micro release, all APIs are compatible with the other 7.3 releases. Highlights of the release include binary Windows 64 support, faster numerical instance fields, and a preliminary HPy backend.
A new contributor (Ondrej Baranovič - thanks!) took us up on the challenge to get windows 64-bit support. The work has been merged and for the first time we are releasing a 64-bit Windows binary package.
The release contains the biggest change to PyPy’s implementation of the
instances of user-defined classes in many years. The optimization was
motivated by the report of performance problems running a numerical particle
emulation. We implemented an optimization that stores int
and float
instance fields in an unboxed way, as long as these fields are type-stable
(meaning that the same field always stores the same type, using the principle
of type freezing). This gives significant performance improvements on
numerical pure-Python code, and other code where instances store many integers
or floating point numbers.
There were also a number of optimizations for methods around strings and bytes, following user reported performance problems. If you are unhappy with PyPy’s performance on some code of yours, please report an issue!
A major new feature is prelminary support for the Universal mode of HPy: a
new way of writing c-extension modules to totally encapsulate PyObject*
.
The goal, as laid out in the HPy documentation and recent HPy blog post,
is to enable a migration path
for c-extension authors who wish their code to be performant on alternative
interpreters like GraalPython (written on top of the Java virtual machine),
RustPython, and PyPy. Thanks to Oracle and IBM for sponsoring work on HPy.
Support for the vmprof statistical profiler has been extended to ARM64 via a built-in backend.
Several issues exposed in the 7.3.3 release were fixed. Many of them came from the great work ongoing to ship PyPy-compatible binary packages in conda-forge. A big shout out to them for taking this on.
Development of PyPy takes place on https://foss.heptapod.net/pypy/pypy. We have seen an increase in the number of drive-by contributors who are able to use gitlab + mercurial to create merge requests.
The CFFI backend has been updated to version 1.14.5 and the cppyy backend to 1.14.2. We recommend using CFFI rather than C-extensions to interact with C, and using cppyy for performant wrapping of C++ code for Python.
As always, we strongly recommend updating to the latest versions. Many fixes are the direct result of end-user bug reports, so please continue reporting issues as they crop up.
You can find links to download the v7.3.4 releases here:
We would like to thank our donors for the continued support of the PyPy project. If PyPy is not quite good enough for your needs, we are available for direct consulting work. If PyPy is helping you out, we would love to hear about it and encourage submissions to our renovated blog site via a pull request to https://github.com/pypy/pypy.org
We would also like to thank our contributors and encourage new people to join the project. PyPy has many layers and we need help with all of them: PyPy and RPython documentation improvements, tweaking popular modules to run on PyPy, or general help with making RPython’s JIT even better. Since the previous release, we have accepted contributions from 10 new contributors, thanks for pitching in, and welcome to the project!
If you are a python library maintainer and use C-extensions, please consider making a cffi / cppyy version of your library that would be performant on PyPy. In any case both cibuildwheel and the multibuild system support building wheels for PyPy.
What is PyPy?¶
PyPy is a Python interpreter, a drop-in replacement for CPython 2.7, 3.7, and soon 3.8. It’s fast (PyPy and CPython 3.7.4 performance comparison) due to its integrated tracing JIT compiler.
We also welcome developers of other dynamic languages to see what RPython can do for them.
This PyPy release supports:
- x86 machines on most common operating systems (Linux 32/64 bits, Mac OS X 64 bits, Windows 32/64 bits, OpenBSD, FreeBSD)
- big- and little-endian variants of PPC64 running Linux,
- s390x running Linux
- 64-bit ARM machines running Linux.
PyPy does support ARM 32 bit processors, but does not release binaries.
Changelog¶
Python 3.7+¶
- Update the
re
module to the Python 3.7 implementation - Fix the
crypt
thread lock (issue 3395) and fix input encoding (issue 3378) - Fixes
utf_8_decode
forfinal=False
(issue 3348) - Test, fix for
time.strftime(u'%y\ud800%m', time.localtime(192039127))
CALL_FUNCTION_KW
with keyword arguments is now much faster, because the data structure storing the arguments can be removed by the JIT- Fix the
repr
of subclasses - Better error message for
object.__init__
with too many parameters - Fix bug in
codecs
where using a function from the parser turns warnings into SyntaxErrors a bit too eagerly - Produce proper deprecation warnings from the compiler, with the right filename and line number
- Fixes for circular imports (bpo 30024) and stack usage (bpo 31286)
- A type annotated assignment was incorrectly handled in the scoping rules, leading to a crash in complex situations (issue 3355)
- Fix a segfault in nonblocking bufferio reads (issue 3172)
- Use correct slot for
sni_callback
attribute in_ssl
(issue 3359) - Hang on to
servername_callback
handle in_ssl
so it will not be deleted until the context is deleted (issue 3396) - Implement
set_wakeup_fd(warn_on_full_buffer)
(issue 3227) - Round-trip invalid UTF-16 data in
winreg
without aUnicodeDecodeError
(issue 3342) - Truncate
REG_SZ
at firstNULL
inwinreg
to matchreg.exe
behaviour (bpo 25778) - Fix for surrogates in
winreg
input value (issue 3345) - In
sysconfig
,INCLUDEPY
andINCLUDEDIR
should point to the original directory even in a virtualenv (issue 3364) - Add
LDLIBRARY
tosysconfig
for posgresql - Prevent overflow in
_hash_long
on win64 using method from CPython - Raise
ValueError
whenargv[0]
ofexecv
and friends is empty (bpo 28732) - Allow compiler to inherit flags from
__future__.annotations
(issue 3371) - Provide a PyPy
BytesBuilder
alternative toio.BytesIO
in pure-pythonpickle
- Generalize venv to copy all
*.exe
and*.dll
for windows - The evaluation order of keys and values of large dict literals was wrong in 3.7 (in lower versions it was the same way, but in 3.7 the evaluation order of small dicts changed), issue 3380
- Cache the imported
re
module in_sre
(going through__import__
is unfortunately quite expensive on 3.x) - Mention a repeated keyword argument in the error message
- Stop emitting the
STORE_ANNOTATION
andBINARY_DIVIDE
bytecodes, update pyc magic number - Fix
site.py
to be closer to upstream to enablepip install --user
andpip install --local
- No longer call
eval()
on content received via HTTP in CJK codec tests (bpo 41944) - Add missing c_/f_/contiguous flags on memoryview
- Fix
xml.ElementTree.extend
not working on iterators (issue 3181, bpo 43399) - Python -m now adds starting directory to sys.path (bpo 33053)
- Reimplement
heapq.merge()
using a linked tournament tree (bpo 38938) - Fix shring of cursors in
sqllite3
(issues 3351 and 3403) - Fix remaining
sqllite3
incompatibilities - Fix
CALL_METHOD_KW
to not lose the immutability of the keyword name tuple
Python 3.7 C-API¶
- Change
char *
toconst char *
inPyStructSequence_Field
,PyStructSequence_Desc
,PyGetSetDef
,wrapperbase
- Implement
METH_FASTCALL
(issue 3357) - Add
pystrtod.h
and expose constants - Clean up some
char *
->const char *
misnaming (issue 3362) - Accept
NULL
input toPyLong_AsUnsignedLongLongMask
- Add
PyImport_GetModule
(issue 3385) - Converting utf-8 to 1-byte buffers must consider latin-1 encoding (issue 3413)
- Fix value of
.__module__
and.__name__
on the result ofPyType_FromSpec
- Add missing
PyFile_FromFd