#78 Support for Python PEP 384 - a stable API for extensions

open
nobody
None
5
2014-06-10
2010-12-22
Noel O'Boyle
No

The forthcoming Python 3.2 implements PEP 384. The short description is at http://docs.python.org/dev/whatsnew/3.2.html:

"In the past, extension modules built for one Python version were often not usable with other Python versions. Particularly on Windows, every feature release of Python required rebuilding all extension modules that one wanted to use. This requirement was the result of the free access to Python interpreter internals that extension modules could use.

With Python 3.2, an alternative approach becomes available: extension modules which restrict themselves to a limited API (by defining Py_LIMITED_API) cannot use many of the internals, but are constrained to a set of API functions that are promised to be stable for several releases. As a consequence, extension modules built for 3.2 in that mode will also work with 3.3, 3.4, and so on. Extension modules that make use of details of memory structures can still be built, but will need to be recompiled for every feature release."

Could SWIG look into defining Py_LIMITED_API in future releases?

Naturally I have no clue whether this is feasible, but if it were, it would sure save me a lot of work (in about 4 years time!). Right now, I have to create 4 distinct installers for 4 versions of Python (2.5, 2.6, 2.7, 3.1). If the 3.2 installer worked with 3.3, 3.4, etc. that'd be great.

Discussion

  • Olly Betts
    Olly Betts
    2010-12-23

    We don't really have an active SWIG/Python maintainer currently, so realistically if you want something like this to happen any time soon, you probably need to drive it forwards yourself.

    What happens if you just try defining Py_LIMITED_API while building the currently generated code with Python 3.2?

     
  • Noel O'Boyle
    Noel O'Boyle
    2010-12-30

    I've just checked and right now, I can't compile the bindings in the first place with Python 3.2. Seems to be already reported - see related bug report http://sourceforge.net/tracker/?func=detail&aid=3057804&group_id=1645&atid=101645.

    I understand the heads up, but I'm not familiar with either the Python or SWIG internals I'm afraid.

     
  • Vadim Zeitlin
    Vadim Zeitlin
    2014-06-10

    Just a not very helpful update: I tried building with Py_LIMITED_API and there are many compilation problems in SWIG-generated code: some examples of functions/macros that are unavailable in the limited API are PyInstanceMethod_New(), PyTuple_GET_SIZE(), PyTuple_GET_ITEM(), PyCFunction_GET_FLAGS() and so on... Some of them have relatively obvious replacements but not all.