pyobjc-dev — CVS commit notices go here

[Pyobjc-dev] [ANN] PyObjC 5.1.1

[Ronald O. <ron...@ma...>]

Hi,

I’ve pushed PyObjC 5.1.1 to PyPI. This is a very minor feature update: it adds a small number of symbols introduced in the macOS 10.14.1 SDK included with Xcode 10.1.

Ronald

Re: [Pyobjc-dev] notarization works! (was Re: not that)

[Ronald O. <ron...@ma...>]

> On 29 Oct 2018, at 18:22, Glyph <gl...@tw...> wrote:
>>> 
>> 
>> Which package needs _cffi_backend? I can add a recipe for that to py2app to do this automagically.
> 
> This may sound obvious, but: cffi :-).  In my case, pyOpenSSL -> cryptography -> cffi.

I’ll look into adding a recipe for this to py2app.  

Ronald

Re: [Pyobjc-dev] notarization works! (was Re: not that)

[Glyph <gl...@tw...>]

On Sun, Oct 28, 2018, at 11:58 PM, Ronald Oussoren wrote:
> 
> 
>> On 29 Oct 2018, at 00:56, Glyph <gl...@tw...> wrote:
>> 
>> 
>> 
>>> On Oct 28, 2018, at 2:57 PM, Glyph <gl...@tw...> wrote:>>> 
>>>> I wonder what the “hardened runtime” option actually does and
>>>> enforces.   In 3.7 the line in ctypes/__init__.py that causes the
>>>> exception is a call that creates a dummy C function, and likely
>>>> triggers the first allocation for storing a libffi closure which
>>>> could be something the hardened runtime doesn’t like (being
>>>> writeable + executable memory).>>> 
>>> Interesting. Perhaps what I want is simply
>>> https://developer.apple.com/documentation/security/com_apple_security_cs_allow-unsigned-executable-memory
>>> then?  Any chance you know how to jam that into a `codesign` command
>>> line somehow? :-)>>> 
>> 
>> Thank you so much for this tip, Ronald!  This was much easier than I
>> anticipated, and things are working now!> 
> Great.
> 
>> 
>> The relevant entitlements file is literally just:
>> 
>>> <?xml version="1.0" encoding="UTF-8"?>
>>> <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "
>>> http://www.apple.com/DTDs/PropertyList-1.0.dtd">>>> <plist version="1.0">
>>> <dict>
>>> <key>com.apple.security.cs.allow-unsigned-executable-memory</key>
>>> <true/>
>>> </dict>
>>> </plist>
>> 
>> I dropped that in a file, added `--entitlements=$THAT_FILE.plist` to
>> my codesign invocations, removed all my workarounds for ctypes et.
>> al. (except for the hard-coded 'import _cffi_backend' still necessary
>> to convince modulegraph to include enough code for SSL to work), and
>> then tried launching my app.> 
> Which package needs _cffi_backend? I can add a recipe for that to
> py2app to do this automagically.
This may sound obvious, but: cffi :-).  In my case, pyOpenSSL ->
cryptography -> cffi.
> 
>> Success!  Then I tried notarizing it: also success!  Time permitting,
>> I'll be updating my blog post at
>> https://glyph.twistedmatrix.com/2018/01/shipping-pygame-mac-app.html
>> with this information, and possibly publishing the now unfortunately
>> somewhat complex tooling I use to do signing now.>> 
>> So I don't know if I'm the first to *do* this, but looking at the
>> archives for these lists I seem to be the first to *report* it: you
>> can successfully codesign and notarize apps created with py2app and
>> python 3.6!>> 
>> It seems to me that whatever "MAP_JIT" is (an mmap flag, I'm
>> guessing?) libffi needs to be using it for the memory it places
>> synthetic closures into, so that this entitlement won't be necessary
>> with some future version of Python.  But it looks like Apple is not
>> pushing particularly hard to deprecate this one right now, thank
>> goodness :-).> 
> MAP_JIT is a mmap flag that’s apparently introduced in 10.14. The
> slides at https://developer.apple.com/videos/play/wwdc2018/702/
> mention this flag and the hardened runtime.> 
> I guess we should add this flag to the code in
> Modules/_ctypes/malloc_closure.c CPython) and in the similar code in
> PyObjC.  The annoying bit is that the flag is new in 10.14, and
> CPython installers are created on 10.9 which means those won’t include
> the new flag for a long time.> 
> I’ll have to check if using MAP_JIT is ok when deploying on older
> macOS versions, or if the code should do a runtime version check.
I can't shed any light on this, but I suspect the cffi folks will also
have to figure this out, and may already have some sense of how this
works.  I filed an issue with them here:
https://bitbucket.org/cffi/cffi/issues/391/cffi-doesnt-work-inside-a-macos-app-bundle

Re: [Pyobjc-dev] notarization works! (was Re: not that)

[Ronald O. <ron...@ma...>]

> On 29 Oct 2018, at 07:58, Ronald Oussoren via Pyobjc-dev <pyo...@li...> wrote:
> 
> MAP_JIT is a mmap flag that’s apparently introduced in 10.14. The slides at https://developer.apple.com/videos/play/wwdc2018/702/ <https://developer.apple.com/videos/play/wwdc2018/702/> mention this flag and the hardened runtime.  
> 
> I guess we should add this flag to the code in Modules/_ctypes/malloc_closure.c CPython) and in the similar code in PyObjC.  The annoying bit is that the flag is new in 10.14, and CPython installers are created on 10.9 which means those won’t include the new flag for a long time. 
> 
> I’ll have to check if using MAP_JIT is ok when deploying on older macOS versions, or if the code should do a runtime version check. 

I filed an issue with PyObjC to ensure I don’t forget to look into this: https://bitbucket.org/ronaldoussoren/pyobjc/issues/253/use-map_fixed-on-macos-1014 <https://bitbucket.org/ronaldoussoren/pyobjc/issues/253/use-map_fixed-on-macos-1014>.

I’ll look into ctypes when I have a good solution for PyObjC.

Ronald

Re: [Pyobjc-dev] notarization works! (was Re: not that)

[Ronald O. <ron...@ma...>]

> On 29 Oct 2018, at 00:56, Glyph <gl...@tw...> wrote:
> 
> 
> 
>> On Oct 28, 2018, at 2:57 PM, Glyph <gl...@tw... <mailto:gl...@tw...>> wrote:
>> 
>>> I wonder what the “hardened runtime” option actually does and enforces.   In 3.7 the line in ctypes/__init__.py that causes the exception is a call that creates a dummy C function, and likely triggers the first allocation for storing a libffi closure which could be something the hardened runtime doesn’t like (being writeable + executable memory). 
>> 
>> Interesting. Perhaps what I want is simply https://developer.apple.com/documentation/security/com_apple_security_cs_allow-unsigned-executable-memory <https://developer.apple.com/documentation/security/com_apple_security_cs_allow-unsigned-executable-memory> then?  Any chance you know how to jam that into a `codesign` command line somehow? :-)
>> 
> 
> Thank you so much for this tip, Ronald!  This was much easier than I anticipated, and things are working now!

Great.

> 
> The relevant entitlements file is literally just:
> 
> <?xml version="1.0" encoding="UTF-8"?>
> <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd <http://www.apple.com/DTDs/PropertyList-1.0.dtd>">
> <plist version="1.0">
> <dict>
> 	<key>com.apple.security.cs.allow-unsigned-executable-memory</key>
> 	<true/>
> </dict>
> </plist>
> 
> I dropped that in a file, added `--entitlements=$THAT_FILE.plist` to my codesign invocations, removed all my workarounds for ctypes et. al. (except for the hard-coded 'import _cffi_backend' still necessary to convince modulegraph to include enough code for SSL to work), and then tried launching my app.  

Which package needs _cffi_backend? I can add a recipe for that to py2app to do this automagically.


> Success!  Then I tried notarizing it: also success!  Time permitting, I'll be updating my blog post at https://glyph.twistedmatrix.com/2018/01/shipping-pygame-mac-app.html <https://glyph.twistedmatrix.com/2018/01/shipping-pygame-mac-app.html> with this information, and possibly publishing the now unfortunately somewhat complex tooling I use to do signing now.
> 
> So I don't know if I'm the first to do this, but looking at the archives for these lists I seem to be the first to report it: you can successfully codesign and notarize apps created with py2app and python 3.6!
> 
> It seems to me that whatever "MAP_JIT" is (an mmap flag, I'm guessing?) libffi needs to be using it for the memory it places synthetic closures into, so that this entitlement won't be necessary with some future version of Python.  But it looks like Apple is not pushing particularly hard to deprecate this one right now, thank goodness :-).

MAP_JIT is a mmap flag that’s apparently introduced in 10.14. The slides at https://developer.apple.com/videos/play/wwdc2018/702/ <https://developer.apple.com/videos/play/wwdc2018/702/> mention this flag and the hardened runtime.  

I guess we should add this flag to the code in Modules/_ctypes/malloc_closure.c CPython) and in the similar code in PyObjC.  The annoying bit is that the flag is new in 10.14, and CPython installers are created on 10.9 which means those won’t include the new flag for a long time. 

I’ll have to check if using MAP_JIT is ok when deploying on older macOS versions, or if the code should do a runtime version check. 

Ronald


> 
> -glyph
> _______________________________________________
> Pyobjc-dev mailing list
> Pyo...@li...
> https://lists.sourceforge.net/lists/listinfo/pyobjc-dev

Re: [Pyobjc-dev] [Pythonmac-SIG] [ANN] PyObjC 5.0

[Glyph <gl...@tw...>]

> On Sep 21, 2018, at 4:50 AM, Ronald Oussoren <ron...@ma...> wrote:
> 
>> P.S.: speaking of build gymnastics, does anyone on this list happen to have a way to use CocoaPods or something similar to pull in open source Cocoa frameworks to a PyObjC application?
> 
> I haven’t looked into this yet.  I guess the hardest part is creating the metadata for constants and pointer arguments. Could you file and issue for this?  
> 

Sorry it took a minute to get to this :).  Filed here: https://bitbucket.org/ronaldoussoren/py2app/issues/255/tooling-and-or-documentation-to-integrate <https://bitbucket.org/ronaldoussoren/py2app/issues/255/tooling-and-or-documentation-to-integrate>

-glyph

[Pyobjc-dev] notarization works! (was Re: not that)

[Glyph <gl...@tw...>]

> On Oct 28, 2018, at 2:57 PM, Glyph <gl...@tw...> wrote:
> 
>> I wonder what the “hardened runtime” option actually does and enforces.   In 3.7 the line in ctypes/__init__.py that causes the exception is a call that creates a dummy C function, and likely triggers the first allocation for storing a libffi closure which could be something the hardened runtime doesn’t like (being writeable + executable memory). 
> 
> Interesting. Perhaps what I want is simply https://developer.apple.com/documentation/security/com_apple_security_cs_allow-unsigned-executable-memory <https://developer.apple.com/documentation/security/com_apple_security_cs_allow-unsigned-executable-memory> then?  Any chance you know how to jam that into a `codesign` command line somehow? :-)
> 

Thank you so much for this tip, Ronald!  This was much easier than I anticipated, and things are working now!

The relevant entitlements file is literally just:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
	<key>com.apple.security.cs.allow-unsigned-executable-memory</key>
	<true/>
</dict>
</plist>

I dropped that in a file, added `--entitlements=$THAT_FILE.plist` to my codesign invocations, removed all my workarounds for ctypes et. al. (except for the hard-coded 'import _cffi_backend' still necessary to convince modulegraph to include enough code for SSL to work), and then tried launching my app.  Success!  Then I tried notarizing it: also success!  Time permitting, I'll be updating my blog post at https://glyph.twistedmatrix.com/2018/01/shipping-pygame-mac-app.html <https://glyph.twistedmatrix.com/2018/01/shipping-pygame-mac-app.html> with this information, and possibly publishing the now unfortunately somewhat complex tooling I use to do signing now.

So I don't know if I'm the first to do this, but looking at the archives for these lists I seem to be the first to report it: you can successfully codesign and notarize apps created with py2app and python 3.6!

It seems to me that whatever "MAP_JIT" is (an mmap flag, I'm guessing?) libffi needs to be using it for the memory it places synthetic closures into, so that this entitlement won't be necessary with some future version of Python.  But it looks like Apple is not pushing particularly hard to deprecate this one right now, thank goodness :-).

-glyph

Re: [Pyobjc-dev] [Pythonmac-SIG] py2app apps codesigned with "--options runtime", i.e. any notorizable app, fails to launch with ctypes error (was Re: ctypes MemoryError when attempting to launch notarized app)

[Glyph <gl...@tw...>]

> On Oct 28, 2018, at 1:48 PM, Ronald Oussoren <ron...@ma...> wrote:
> 
> 
> 
>> On 28 Oct 2018, at 19:47, Glyph <gl...@tw...> wrote:
>> 
>> 
>> 
>>> On Oct 28, 2018, at 11:20 AM, Glyph <gl...@tw...> wrote:
>>> 
>>> 
>>> 
>>>> On Oct 28, 2018, at 2:27 AM, Ronald Oussoren <ron...@ma...> wrote:
>>> 
>>>>> 
>>>>> Curiously, this is the same traceback that comes from https://forum.kodi.tv/showthread.php?tid=329171, which suggests it's something fundamental to strict shared-library sandboxing that ctypes trips over when trying to initialize itself.
>>>>> 
>>>>> Does anyone have experience with this, or ideas about what to do?
>>>> 
>>>> I’m afraid not. I currently get away with not signing apps at all, although properly supporting signing is on my way too long wish list for py2app.  
>>> 
>>> The ability to distribute unsigned apps is not-so-slowly going away; even the ability to distribute non-notarized apps has a very limited shelf-life at this point.  So this ought to be an alarming development for everyone - having Python apps effectively banned from macOS distribution is a big potential problem :-\.
>>> 
>>> The good news here is that aside from having to write a little for loop in shell (shown below) getting the app codesigned previously was easy, and my app *did* pass notarization, so nothing that py2app is doing is breaking things on apple's end.  It's just a matter of a ctypes bug.
>> 
>> On that note: more good news.  While I haven't round-tripped through notarization again yet, this is a bit less dire than it first appeared.  If I prevent the import of ctypes with an `import sys; sys.modules['ctypes'] = None`, and add a 'sed' script to my build process to prevent _setup_ctypes from running in __boot__, then the app launches again.
>> 
>> Apparently my app doesn't actually need ctypes.
> 
> Good to hear that. 
> 
>> 
>> The problem seems to be that Twisted includes a ctypes import; modulegraph sees this and thinks there is a hard dependency, and inserts the ctypes setup blob into __boot__.  However, this is a conditional import, and it's for Windows support anyway.
> 
> Hmm…. I wonder what’s the best way forward here. I could add on option to disable ctypes support, but that is a kludge.  A weak importing hook (something like the never withdrawn PEP 369) could execute this code only when actually needed, but I have no idea how hard it would be to implement this.
> 
> 
>> 
>> (There also seem to be problems with cffi-using libraries, but not other shared objects, so maybe this is a bug in libffi; however, these don't interfere with py2app itself starting up.)
> 
> Interesting…  I haven’t had complaints about PyObjC yet, and that also uses libffi.  
> 
> I wonder what the “hardened runtime” option actually does and enforces.   In 3.7 the line in ctypes/__init__.py that causes the exception is a call that creates a dummy C function, and likely triggers the first allocation for storing a libffi closure which could be something the hardened runtime doesn’t like (being writeable + executable memory). 

Interesting. Perhaps what I want is simply https://developer.apple.com/documentation/security/com_apple_security_cs_allow-unsigned-executable-memory then?  Any chance you know how to jam that into a `codesign` command line somehow? :-)

> P.S. I just noticed that the traceback in your initial message doesn’t include the actual exception, just the traceback. 

Oh; it’s “MemoryError”, no exception message.

> Ronald
> 

Re: [Pyobjc-dev] py2app apps codesigned with "--options runtime", i.e. any notorizable app, fails to launch with ctypes error (was Re: ctypes MemoryError when attempting to launch notarized app)

[Ronald O. <ron...@ma...>]

> On 28 Oct 2018, at 19:20, Glyph <gl...@tw...> wrote:
> 
> 
> 
>>> 
>>> Curiously, this is the same traceback that comes from https://forum.kodi.tv/showthread.php?tid=329171 <https://forum.kodi.tv/showthread.php?tid=329171>, which suggests it's something fundamental to strict shared-library sandboxing that ctypes trips over when trying to initialize itself.
>>> 
>>> Does anyone have experience with this, or ideas about what to do?
>> 
>> I’m afraid not. I currently get away with not signing apps at all, although properly supporting signing is on my way too long wish list for py2app.  
> 
> The ability to distribute unsigned apps is not-so-slowly going away; even the ability to distribute non-notarized apps has a very limited shelf-life at this point.  So this ought to be an alarming development for everyone - having Python apps effectively banned from macOS distribution is a big potential problem :-\.

Yup. That’s something that worries me as well, and not just for Python apps.  Not being able to run your own code without paying Apple for the privilege is not something I look forward to.  I’m still hoping that the option to run unsigned code doesn’t go away.


> 
> The good news here is that aside from having to write a little for loop in shell (shown below) getting the app codesigned previously was easy, and my app *did* pass notarization, so nothing that py2app is doing is breaking things on apple's end.  It's just a matter of a ctypes bug.
> 
> As I see it, there's 2 problems here:
> 
> py2app's __boot__.py should fail more gracefully if initializing ctypes doesn't work, since not everybody needs ctypes.  Shall I file this on the tracker?

Yes please.  I’d prefer a solution that doesn’t involve ignoring errors, but that’s probably the easiest fix for now.  What’s the exception you’re getting? Tweaking the code in py2app/bootstrap/ctypes_setup.py to ignore that exception would be trivial. 


> ctypes itself should address whatever eldritch hideousness is causing this; in addition to the windows security layer stuff I found, grsecurity TPE causes the same traceback: https://bugs.python.org/issue28429 <https://bugs.python.org/issue28429>
>> With some luck there’s some entitlement or code signing option that causes this problem.  What is the output of "codesign --display --verbose=4” for the application? Both with and without notarisation? 
> 
> Sorry, my original message was not clear.  App notarization itself is not the problem, it's the "stricter requirements" that I ambiguously referenced.  The requirement in question is the '--options runtime' flag passed to 'codesign'.  So you can just codesign an app (even with an ad-hoc identity, you technically could do this without even having a valid cert, although the way one generates one of those escapes me) with the 'runtime' option, you can reproduce this.
> 
> So if I sign my app like this:
> 
> #!/bin/bash
> find "${NAME}.app" -iname '*.so' -or -iname '*.dylib' |
>     while read libfile; do
>           codesign --sign "${IDENTITY}" \
>                    --deep "${libfile}" \
>                    --force \
>                    --options runtime;
>     done;
> 
> codesign --sign "${IDENTITY}" \
>          --deep "${NAME}.app" \
>          --force \
>          --options runtime;
> 
> and then run it as "./${NAME}.app/Contents/MacOS/${NAME}".  I immediately get the traceback given above.

Great. That should make it easier for me to reproduce the issue. 

Ronald

Re: [Pyobjc-dev] [Pythonmac-SIG] py2app apps codesigned with "--options runtime", i.e. any notorizable app, fails to launch with ctypes error (was Re: ctypes MemoryError when attempting to launch notarized app)

[Ronald O. <ron...@ma...>]

> On 28 Oct 2018, at 19:47, Glyph <gl...@tw...> wrote:
> 
> 
> 
>> On Oct 28, 2018, at 11:20 AM, Glyph <gl...@tw... <mailto:gl...@tw...>> wrote:
>> 
>> 
>> 
>>> On Oct 28, 2018, at 2:27 AM, Ronald Oussoren <ron...@ma... <mailto:ron...@ma...>> wrote:
>> 
>>>> 
>>>> Curiously, this is the same traceback that comes from https://forum.kodi.tv/showthread.php?tid=329171 <https://forum.kodi.tv/showthread.php?tid=329171>, which suggests it's something fundamental to strict shared-library sandboxing that ctypes trips over when trying to initialize itself.
>>>> 
>>>> Does anyone have experience with this, or ideas about what to do?
>>> 
>>> I’m afraid not. I currently get away with not signing apps at all, although properly supporting signing is on my way too long wish list for py2app.  
>> 
>> The ability to distribute unsigned apps is not-so-slowly going away; even the ability to distribute non-notarized apps has a very limited shelf-life at this point.  So this ought to be an alarming development for everyone - having Python apps effectively banned from macOS distribution is a big potential problem :-\.
>> 
>> The good news here is that aside from having to write a little for loop in shell (shown below) getting the app codesigned previously was easy, and my app *did* pass notarization, so nothing that py2app is doing is breaking things on apple's end.  It's just a matter of a ctypes bug.
> 
> On that note: more good news.  While I haven't round-tripped through notarization again yet, this is a bit less dire than it first appeared.  If I prevent the import of ctypes with an `import sys; sys.modules['ctypes'] = None`, and add a 'sed' script to my build process to prevent _setup_ctypes from running in __boot__, then the app launches again.
> 
> Apparently my app doesn't actually need ctypes.

Good to hear that. 

> 
> The problem seems to be that Twisted includes a ctypes import; modulegraph sees this and thinks there is a hard dependency, and inserts the ctypes setup blob into __boot__.  However, this is a conditional import, and it's for Windows support anyway.

Hmm…. I wonder what’s the best way forward here. I could add on option to disable ctypes support, but that is a kludge.  A weak importing hook (something like the never withdrawn PEP 369) could execute this code only when actually needed, but I have no idea how hard it would be to implement this.


> 
> (There also seem to be problems with cffi-using libraries, but not other shared objects, so maybe this is a bug in libffi; however, these don't interfere with py2app itself starting up.)

Interesting…  I haven’t had complaints about PyObjC yet, and that also uses libffi.  

I wonder what the “hardened runtime” option actually does and enforces.   In 3.7 the line in ctypes/__init__.py that causes the exception is a call that creates a dummy C function, and likely triggers the first allocation for storing a libffi closure which could be something the hardened runtime doesn’t like (being writeable + executable memory). 

P.S. I just noticed that the traceback in your initial message doesn’t include the actual exception, just the traceback. 


Ronald

Re: [Pyobjc-dev] [Pythonmac-SIG] py2app apps codesigned with "--options runtime", i.e. any notorizable app, fails to launch with ctypes error (was Re: ctypes MemoryError when attempting to launch notarized app)

[Glyph <gl...@tw...>]

> On Oct 28, 2018, at 11:20 AM, Glyph <gl...@tw...> wrote:
> 
> 
> 
>> On Oct 28, 2018, at 2:27 AM, Ronald Oussoren <ron...@ma... <mailto:ron...@ma...>> wrote:
> 
>>> 
>>> Curiously, this is the same traceback that comes from https://forum.kodi.tv/showthread.php?tid=329171 <https://forum.kodi.tv/showthread.php?tid=329171>, which suggests it's something fundamental to strict shared-library sandboxing that ctypes trips over when trying to initialize itself.
>>> 
>>> Does anyone have experience with this, or ideas about what to do?
>> 
>> I’m afraid not. I currently get away with not signing apps at all, although properly supporting signing is on my way too long wish list for py2app.  
> 
> The ability to distribute unsigned apps is not-so-slowly going away; even the ability to distribute non-notarized apps has a very limited shelf-life at this point.  So this ought to be an alarming development for everyone - having Python apps effectively banned from macOS distribution is a big potential problem :-\.
> 
> The good news here is that aside from having to write a little for loop in shell (shown below) getting the app codesigned previously was easy, and my app *did* pass notarization, so nothing that py2app is doing is breaking things on apple's end.  It's just a matter of a ctypes bug.

On that note: more good news.  While I haven't round-tripped through notarization again yet, this is a bit less dire than it first appeared.  If I prevent the import of ctypes with an `import sys; sys.modules['ctypes'] = None`, and add a 'sed' script to my build process to prevent _setup_ctypes from running in __boot__, then the app launches again.

Apparently my app doesn't actually need ctypes.

The problem seems to be that Twisted includes a ctypes import; modulegraph sees this and thinks there is a hard dependency, and inserts the ctypes setup blob into __boot__.  However, this is a conditional import, and it's for Windows support anyway.

(There also seem to be problems with cffi-using libraries, but not other shared objects, so maybe this is a bug in libffi; however, these don't interfere with py2app itself starting up.)

-glyph

[Pyobjc-dev] py2app apps codesigned with "--options runtime", i.e. any notorizable app, fails to launch with ctypes error (was Re: ctypes MemoryError when attempting to launch notarized app)

[Glyph <gl...@tw...>]

> On Oct 28, 2018, at 2:27 AM, Ronald Oussoren <ron...@ma...> wrote:
> 
> 
> 
>> On 28 Oct 2018, at 06:27, Glyph <gl...@tw...> wrote:
>> 
>> I adjusted my code-signing to use the new, stricter requirements imposed by app notarization. I managed to get it successfully notarized, but the app is now non-functional as a result: at startup, I get:
>> 
>> Traceback (most recent call last):
>>  File "my.app/Contents/Resources/__boot__.py", line 93, in <module>
>>    _setup_ctypes()
>>  File "my.app/Contents/Resources/__boot__.py", line 86, in _setup_ctypes
>>    from ctypes.macholib import dyld
>>  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
>>  File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
>>  File "<frozen importlib._bootstrap>", line 656, in _load_unlocked
>>  File "<frozen importlib._bootstrap>", line 626, in _load_backward_compatible
>>  File "ctypes/__init__.pyc", line 538, in <module>
>>  File "ctypes/__init__.pyc", line 273, in _reset_cache
>> 
>> (If anyone wants to follow along in the traceback, this is using python.org 3.6.6.)
> 
> On what version of macOS? I expect 10.14 because that’s the only release that actually knows about notarization, but enabling this feature might also affect how the app is signed.

ProductName:	Mac OS X
ProductVersion:	10.14
BuildVersion:	18A391

>> This happens before any of my code even runs, so I can't just try to avoid ctypes.
> 
> You could patch the __boot__.py file before signing to see if that helps. Although this should cause problems further on, the call to _setup_ctypes should only be created when some code in your app bundle has a dependency on ctypes. 

I'll give that a shot.

>> 
>> Curiously, this is the same traceback that comes from https://forum.kodi.tv/showthread.php?tid=329171 <https://forum.kodi.tv/showthread.php?tid=329171>, which suggests it's something fundamental to strict shared-library sandboxing that ctypes trips over when trying to initialize itself.
>> 
>> Does anyone have experience with this, or ideas about what to do?
> 
> I’m afraid not. I currently get away with not signing apps at all, although properly supporting signing is on my way too long wish list for py2app.  

The ability to distribute unsigned apps is not-so-slowly going away; even the ability to distribute non-notarized apps has a very limited shelf-life at this point.  So this ought to be an alarming development for everyone - having Python apps effectively banned from macOS distribution is a big potential problem :-\.

The good news here is that aside from having to write a little for loop in shell (shown below) getting the app codesigned previously was easy, and my app *did* pass notarization, so nothing that py2app is doing is breaking things on apple's end.  It's just a matter of a ctypes bug.

As I see it, there's 2 problems here:

py2app's __boot__.py should fail more gracefully if initializing ctypes doesn't work, since not everybody needs ctypes.  Shall I file this on the tracker?
ctypes itself should address whatever eldritch hideousness is causing this; in addition to the windows security layer stuff I found, grsecurity TPE causes the same traceback: https://bugs.python.org/issue28429

> With some luck there’s some entitlement or code signing option that causes this problem.  What is the output of "codesign --display --verbose=4” for the application? Both with and without notarisation? 

Sorry, my original message was not clear.  App notarization itself is not the problem, it's the "stricter requirements" that I ambiguously referenced.  The requirement in question is the '--options runtime' flag passed to 'codesign'.  So you can just codesign an app (even with an ad-hoc identity, you technically could do this without even having a valid cert, although the way one generates one of those escapes me) with the 'runtime' option, you can reproduce this.

So if I sign my app like this:

#!/bin/bash
find "${NAME}.app" -iname '*.so' -or -iname '*.dylib' |
    while read libfile; do
          codesign --sign "${IDENTITY}" \
                   --deep "${libfile}" \
                   --force \
                   --options runtime;
    done;

codesign --sign "${IDENTITY}" \
         --deep "${NAME}.app" \
         --force \
         --options runtime;

and then run it as "./${NAME}.app/Contents/MacOS/${NAME}".  I immediately get the traceback given above.

-glyph

Re: [Pyobjc-dev] ctypes MemoryError when attempting to launch notarized app

[Ronald O. <ron...@ma...>]

> On 28 Oct 2018, at 06:27, Glyph <gl...@tw...> wrote:
> 
> I adjusted my code-signing to use the new, stricter requirements imposed by app notarization.  I managed to get it successfully notarized, but the app is now non-functional as a result: at startup, I get:
> 
> Traceback (most recent call last):
>   File "my.app/Contents/Resources/__boot__.py", line 93, in <module>
>     _setup_ctypes()
>   File "my.app/Contents/Resources/__boot__.py", line 86, in _setup_ctypes
>     from ctypes.macholib import dyld
>   File "<frozen importlib._bootstrap>", line 971, in _find_and_load
>   File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
>   File "<frozen importlib._bootstrap>", line 656, in _load_unlocked
>   File "<frozen importlib._bootstrap>", line 626, in _load_backward_compatible
>   File "ctypes/__init__.pyc", line 538, in <module>
>   File "ctypes/__init__.pyc", line 273, in _reset_cache
> 
> (If anyone wants to follow along in the traceback, this is using python.org 3.6.6.)

On what version of macOS? I expect 10.14 because that’s the only release that actually knows about notarization, but enabling this feature might also affect how the app is signed.
> 
> This happens before any of my code even runs, so I can't just try to avoid ctypes.

You could patch the __boot__.py file before signing to see if that helps. Although this should cause problems further on, the call to _setup_ctypes should only be created when some code in your app bundle has a dependency on ctypes. 

> 
> Curiously, this is the same traceback that comes from https://forum.kodi.tv/showthread.php?tid=329171, which suggests it's something fundamental to strict shared-library sandboxing that ctypes trips over when trying to initialize itself.
> 
> Does anyone have experience with this, or ideas about what to do?

I’m afraid not. I currently get away with not signing apps at all, although properly supporting signing is on my way too long wish list for py2app.  


With some luck there’s some entitlement or code signing option that causes this problem.  What is the output of "codesign --display --verbose=4” for the application? Both with and without notarisation? 

Ronald

> 
> -glyph
> _______________________________________________
> Pyobjc-dev mailing list
> Pyo...@li...
> https://lists.sourceforge.net/lists/listinfo/pyobjc-dev

[Pyobjc-dev] ctypes MemoryError when attempting to launch notarized app

[Glyph <gl...@tw...>]

I adjusted my code-signing to use the new, stricter requirements imposed by app notarization.  I managed to get it successfully notarized, but the app is now non-functional as a result: at startup, I get:

Traceback (most recent call last):
  File "my.app/Contents/Resources/__boot__.py", line 93, in <module>
    _setup_ctypes()
  File "my.app/Contents/Resources/__boot__.py", line 86, in _setup_ctypes
    from ctypes.macholib import dyld
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 656, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 626, in _load_backward_compatible
  File "ctypes/__init__.pyc", line 538, in <module>
  File "ctypes/__init__.pyc", line 273, in _reset_cache

(If anyone wants to follow along in the traceback, this is using python.org <http://python.org/> 3.6.6.)

This happens before any of my code even runs, so I can't just try to avoid ctypes.

Curiously, this is the same traceback that comes from https://forum.kodi.tv/showthread.php?tid=329171 <https://forum.kodi.tv/showthread.php?tid=329171>, which suggests it's something fundamental to strict shared-library sandboxing that ctypes trips over when trying to initialize itself.

Does anyone have experience with this, or ideas about what to do?

-glyph

[Pyobjc-dev] [ANN] PyObjC 5.1

[Ronald O. <ron...@ma...>]

Hi,

I’ve pushed PyObjC 5.1 to PyPI. This is a minor feature release.  The most interesting change is that the Objective-C proxies for builtin Python types now support NSSecureCoding.


The full list of changes:

Xcode 10 “GM” contains one difference from the last beta: the constant MLComputeUnitsCPUAndGPU in the CoreML bindings.

#222: Add a proxy for C’s “FILE*” type on Python 3. This is not necessary on Python 2 because the default IO stack on Python 2 already uses FILE* internally.

This proxy type is very minimal and shouldn’t not be used for general I/O.

Bindings are up-to-date w.r.t. Xcode 10.1 (beta)

Updated the support code for framework wrappers to be able to emit deprecation warnings on the first import of a deprecated constants (functions and methods will only raise a deprecation warning when called).

This is just an infrastructure change, the actual framework bindings do not yet contain the information used to emit deprecation warnings.

Add metadata for deprecation warnings to the “Contacts” framework

#252: Import ABCs from collections.abc instead of collections because the latter is deprecated.

#180, #251: Instances of most builtin value types and sequences (int, float, str, unicode, tuple, list, set, frozenset and dict) can now be written to archives that require secureCoding.

Ronald

[Pyobjc-dev] [ANN] py2app 0.18

[Ronald O. <ron...@ma...>]

Hi,

I’ve released py2app 0.18 on PyPI. The only change w.r.t. 0.17 is a recipe for “six.moves”, that should also work when the “six” library has been vendored by some other package.

Ronald

Re: [Pyobjc-dev] [Pythonmac-SIG] [ANN] PyObjC 5.0

[Ronald O. <ron...@ma...>]

> On 21 Sep 2018, at 23:24, Glyph <gl...@tw...> wrote:
> 
> 
> 
>> On Sep 21, 2018, at 4:50 AM, Ronald Oussoren <ron...@ma... <mailto:ron...@ma...>> wrote:
>> 
>> 
>> 
>>> On 21 Sep 2018, at 05:59, Glyph <gl...@tw... <mailto:gl...@tw...>> wrote:
>>> 
>>> On Sep 18, 2018, at 1:22 PM, Ronald Oussoren via Pythonmac-SIG <pyt...@py... <mailto:pyt...@py...>> wrote:
>>>> 
>>>> PyObjC 5.0 is out
>>> 
>>> Thanks again for your tireless (or at least apparently tireless) maintenance, Ronald!
>>> 
>>>> * Added bindings for the “CarbonCore” subframework of the “CoreServices” framework.
>>>> 
>>>> Most APIs in this subframework are not available to Python, only those APIs that are not deprecated and seem interesting are exposed.
>>> 
>>> One thing I wondered about when I saw "carbon" here is that I wonder if the APIs that could make https://pythonhosted.org/pyobjc/examples/Cocoa/AppKit/HotKeyPython/index.html <https://pythonhosted.org/pyobjc/examples/Cocoa/AppKit/HotKeyPython/index.html> are exposed again?  
>> 
>> The API used in that example is not exposed through PyObjC at this time (AFAIK it is part of “Carbon.framework”, not the CarbonCore sub framework of “CoreServices.framework”. 
>> 
>> I’m open to adding bindings to interesting APIs, but note that “RegisterEventHotKey” is not documented on Apple’s website and might not be the right solution for modern code. I don’t know yet what the right solution is though (a number of APIs that are exposed through PyObjC can be used to implement similar functionality).
> 
> If there were a more modern solution I'd gladly use it rather than asking you to expose gross and ancient Carbon APIs :).  Which APIs that are already exposed can do this?

Good question :-).  I thought CGEventTapCreate and related APIs could be used for this, but now that I’ve reread the documentation I no longer think so. MASShortCut still uses the Carbon APIs in its implementation, which seems to confirm that there is no good modern alternative for the RegisterEventHotKey API.

> 
>>> The cautions that the relevant Carbon APIs require 32-bit mode are inaccurate; https://blog.shpakovski.com/2012/07/global-keyboard-shortcuts-in-cocoa.html <https://blog.shpakovski.com/2012/07/global-keyboard-shortcuts-in-cocoa.html> explains that they're still supported in sandboxed Mac App Store apps, so they're just poorly documented, not deprecated.  (It seems that there's never been a proper Cocoa replacement for this functionality, just a couple of popular wrappers floating around out there…)
>>> 
>>> This is something I have wanted to do in Python many times, but never quite badly enough to figure out all the build gymnastics required to get the bindings for https://github.com/shpakovski/MASShortcut <https://github.com/shpakovski/MASShortcut> built and bundled with my application.
>> 
>> I’ll look into exposing this API for PyObjC 5.1, even if that means adding a very limited set of bindings for Carbon.framework.
> 
> Thanks!
> 
>>> -glyph
>>> 
>>> P.S.: speaking of build gymnastics, does anyone on this list happen to have a way to use CocoaPods or something similar to pull in open source Cocoa frameworks to a PyObjC application?
>> 
>> I haven’t looked into this yet.  I guess the hardest part is creating the metadata for constants and pointer arguments. Could you file and issue for this? 
> 
> Sure, sometime this weekend I'll try to write it up.

I have created an experimental binding to MASShortcut on my bitbucket account: https://bitbucket.org/ronaldoussoren/pyobjc-masshortcut <https://bitbucket.org/ronaldoussoren/pyobjc-masshortcut>. This library can be installed using “pip install pyobjc-MASShortcut==1.0a0” (and has a wheel if you use a 64-bit installation of Python 3.7)

This “works” in the sense that I ran my unit tests with Python 3.6 (64-bit), but I have done no testing beyond that.

Creating the bindings was fairly easy, but harder than it should be because the tooling I use to create bindings is primarily created for Apple’s system frameworks and has some issues with 3th-party frameworks. I do want to improve the tooling for this, but don’t know when I’ll get around to actually doing this.

Ronald

Re: [Pyobjc-dev] [Pythonmac-SIG] [ANN] PyObjC 5.0

[Glyph <gl...@tw...>]

> On Sep 21, 2018, at 4:50 AM, Ronald Oussoren <ron...@ma...> wrote:
> 
> 
> 
>> On 21 Sep 2018, at 05:59, Glyph <gl...@tw... <mailto:gl...@tw...>> wrote:
>> 
>> On Sep 18, 2018, at 1:22 PM, Ronald Oussoren via Pythonmac-SIG <pyt...@py... <mailto:pyt...@py...>> wrote:
>>> 
>>> PyObjC 5.0 is out
>> 
>> Thanks again for your tireless (or at least apparently tireless) maintenance, Ronald!
>> 
>>> * Added bindings for the “CarbonCore” subframework of the “CoreServices” framework.
>>> 
>>> Most APIs in this subframework are not available to Python, only those APIs that are not deprecated and seem interesting are exposed.
>> 
>> One thing I wondered about when I saw "carbon" here is that I wonder if the APIs that could make https://pythonhosted.org/pyobjc/examples/Cocoa/AppKit/HotKeyPython/index.html <https://pythonhosted.org/pyobjc/examples/Cocoa/AppKit/HotKeyPython/index.html> are exposed again?  
> 
> The API used in that example is not exposed through PyObjC at this time (AFAIK it is part of “Carbon.framework”, not the CarbonCore sub framework of “CoreServices.framework”. 
> 
> I’m open to adding bindings to interesting APIs, but note that “RegisterEventHotKey” is not documented on Apple’s website and might not be the right solution for modern code. I don’t know yet what the right solution is though (a number of APIs that are exposed through PyObjC can be used to implement similar functionality).

If there were a more modern solution I'd gladly use it rather than asking you to expose gross and ancient Carbon APIs :).  Which APIs that are already exposed can do this?

>> The cautions that the relevant Carbon APIs require 32-bit mode are inaccurate; https://blog.shpakovski.com/2012/07/global-keyboard-shortcuts-in-cocoa.html <https://blog.shpakovski.com/2012/07/global-keyboard-shortcuts-in-cocoa.html> explains that they're still supported in sandboxed Mac App Store apps, so they're just poorly documented, not deprecated.  (It seems that there's never been a proper Cocoa replacement for this functionality, just a couple of popular wrappers floating around out there…)
>> 
>> This is something I have wanted to do in Python many times, but never quite badly enough to figure out all the build gymnastics required to get the bindings for https://github.com/shpakovski/MASShortcut <https://github.com/shpakovski/MASShortcut> built and bundled with my application.
> 
> I’ll look into exposing this API for PyObjC 5.1, even if that means adding a very limited set of bindings for Carbon.framework.

Thanks!

>> -glyph
>> 
>> P.S.: speaking of build gymnastics, does anyone on this list happen to have a way to use CocoaPods or something similar to pull in open source Cocoa frameworks to a PyObjC application?
> 
> I haven’t looked into this yet.  I guess the hardest part is creating the metadata for constants and pointer arguments. Could you file and issue for this? 

Sure, sometime this weekend I'll try to write it up.

Re: [Pyobjc-dev] [Pythonmac-SIG] [ANN] PyObjC 5.0

[Ronald O. <ron...@ma...>]

> On 21 Sep 2018, at 05:59, Glyph <gl...@tw...> wrote:
> 
> On Sep 18, 2018, at 1:22 PM, Ronald Oussoren via Pythonmac-SIG <pyt...@py... <mailto:pyt...@py...>> wrote:
>> 
>> PyObjC 5.0 is out
> 
> Thanks again for your tireless (or at least apparently tireless) maintenance, Ronald!
> 
>> * Added bindings for the “CarbonCore” subframework of the “CoreServices” framework.
>> 
>> Most APIs in this subframework are not available to Python, only those APIs that are not deprecated and seem interesting are exposed.
> 
> One thing I wondered about when I saw "carbon" here is that I wonder if the APIs that could make https://pythonhosted.org/pyobjc/examples/Cocoa/AppKit/HotKeyPython/index.html <https://pythonhosted.org/pyobjc/examples/Cocoa/AppKit/HotKeyPython/index.html> are exposed again?  

The API used in that example is not exposed through PyObjC at this time (AFAIK it is part of “Carbon.framework”, not the CarbonCore sub framework of “CoreServices.framework”. 

I’m open to adding bindings to interesting APIs, but note that “RegisterEventHotKey” is not documented on Apple’s website and might not be the right solution for modern code. I don’t know yet what the right solution is though (a number of APIs that are exposed through PyObjC can be used to implement similar functionality).

> The cautions that the relevant Carbon APIs require 32-bit mode are inaccurate; https://blog.shpakovski.com/2012/07/global-keyboard-shortcuts-in-cocoa.html <https://blog.shpakovski.com/2012/07/global-keyboard-shortcuts-in-cocoa.html> explains that they're still supported in sandboxed Mac App Store apps, so they're just poorly documented, not deprecated.  (It seems that there's never been a proper Cocoa replacement for this functionality, just a couple of popular wrappers floating around out there…)
> 
> This is something I have wanted to do in Python many times, but never quite badly enough to figure out all the build gymnastics required to get the bindings for https://github.com/shpakovski/MASShortcut <https://github.com/shpakovski/MASShortcut> built and bundled with my application.

I’ll look into exposing this API for PyObjC 5.1, even if that means adding a very limited set of bindings for Carbon.framework.

> 
> -glyph
> 
> P.S.: speaking of build gymnastics, does anyone on this list happen to have a way to use CocoaPods or something similar to pull in open source Cocoa frameworks to a PyObjC application?

I haven’t looked into this yet.  I guess the hardest part is creating the metadata for constants and pointer arguments. Could you file and issue for this?  

Ronald

Re: [Pyobjc-dev] [Pythonmac-SIG] [ANN] PyObjC 5.0

[Glyph <gl...@tw...>]

On Sep 18, 2018, at 1:22 PM, Ronald Oussoren via Pythonmac-SIG <pyt...@py...> wrote:
> 
> PyObjC 5.0 is out

Thanks again for your tireless (or at least apparently tireless) maintenance, Ronald!

> * Added bindings for the “CarbonCore” subframework of the “CoreServices” framework.
> 
> Most APIs in this subframework are not available to Python, only those APIs that are not deprecated and seem interesting are exposed.

One thing I wondered about when I saw "carbon" here is that I wonder if the APIs that could make https://pythonhosted.org/pyobjc/examples/Cocoa/AppKit/HotKeyPython/index.html <https://pythonhosted.org/pyobjc/examples/Cocoa/AppKit/HotKeyPython/index.html> are exposed again?  The cautions that the relevant Carbon APIs require 32-bit mode are inaccurate; https://blog.shpakovski.com/2012/07/global-keyboard-shortcuts-in-cocoa.html <https://blog.shpakovski.com/2012/07/global-keyboard-shortcuts-in-cocoa.html> explains that they're still supported in sandboxed Mac App Store apps, so they're just poorly documented, not deprecated.  (It seems that there's never been a proper Cocoa replacement for this functionality, just a couple of popular wrappers floating around out there...)

This is something I have wanted to do in Python many times, but never quite badly enough to figure out all the build gymnastics required to get the bindings for https://github.com/shpakovski/MASShortcut <https://github.com/shpakovski/MASShortcut> built and bundled with my application.

-glyph

P.S.: speaking of build gymnastics, does anyone on this list happen to have a way to use CocoaPods or something similar to pull in open source Cocoa frameworks to a PyObjC application?

[Pyobjc-dev] [ANN] PyObjC 5.0

[Ronald O. <ron...@ma...>]

PyObjC 5.0 is out

The release of macOS 10.14 is near, it is therefore time to release a new major version of PyObjC. I’ve uploaded PyObjC to PyPI, it can be installed using “python3 -m pip install -U pyobjc”.

What is PyObjC

The PyObjC project provides bindings to most of Apple’s higher-level APIs (frameworks). More information about these bindings and how to use PyObjC can be found on the PyObjC website.

What is new

The main feature of this release is the addition of support for APIs introduced in macOS 10.14 (Mojave).

In particular:
* Adds support for macOS 10.14 (Mojave)
* This release updates the framework wrappers with support for new APIs in macOS 10.14 and adds bindings for the following new frameworks:

		•  AdSupport
		•  CoreAudio (new in macOS 10.0)
		•  CoreAudioKit (new in macOS 10.4)
		•  CoreMedia (new in macOS 10.7)
		•  CoreMediaIO (new in macOS 10.7)
		•  DiscRecording (new in macOS 10.2)
		•  DiscRecordingUI (new in macOS 10.2)
		•  DVDPlayback (new in macOS 10.3)
		•  MediaToolbox
		•  NaturalLanguage
		•  Network
		•  OSAKit (new in macOS 10.4)
		•  UserNotifications
		•  VideoSubscriberAccount
		•  VideoToolbox (new in macOS 10.8)


* Added two features that can help with gating code on the version of macos:

1) The constants “objc.MAC_OS_X_VERSION_CURRENT” can be compared with one of the “objc.MAC_OS_X_VERSION_…” contants.

2) The function “objc.macos_avaiable(major, minor[, patch])” returns true if the current macOS version is at least the specified version, comparable with “@available” in Swift.
	
* PR19: Fix deprecation warning in bridgesupport support module
Patch by: Mickaël Schoentgen

* Creating objc.ObjCPointer instances now results in a Python warning, instead of an unconditional message on stdout.

* System bridgesupport XML files (normally not used by PyObjC) can contain constant numbers with value “inf”, PyObjC now knows how to handle those.

* Added bindings for the “Metadata” subframework of the “CoreServices” framework.

* Added bindings for the “CarbonCore” subframework of the “CoreServices” framework.

Most APIs in this subframework are not available to Python, only those APIs that are not deprecated and seem interesting are exposed.

* The separate framework wrappers DictionaryServices, LaunchServices and SearchKit are deprecated, use the CoreServices bindings instead.

These framework wrappers still exists, but are effectively aliases for CoreServices with this release. Because of this these bindings can expose more symbols than previously.

* Fix unexpected exception when trying to call getattr on a framework wrapped with a name that isn’t a valid identifier.

* Issue 244: Bad metadata for CGPDFOperatorTableSetCallback

* Issue 247: Fix crash in regression test case

* One specific test in pyobjc-core crashed the interpreter when run separately. Because of this I’ve disabled an optimization that uses alloca instead of PyMem_Malloc to allocate memory for now.

Supporting development

I do all development on PyObjC in my spare time. Please consider donating if you use PyObjC professionally. This will help me to improve PyObjC and related projects. See my website for more information.


P.S. This is a cross-post from http://blog.ronaldoussoren.net/2018/09/18/pyobjc-is-out.html, and I’ve also tweeted about this from @RonaldOussoren.

[Pyobjc-dev] [ANN] PyObjC 5.0b1

[Ronald O. <ron...@ma...>]

Hi,

I just uploaded PyObjC 5.0b1 to PyPI. This is the first beta of the version of PyObjC that includes support for macOS 10.14 (Mojave). I expect to release 5.0 soon after the GM release of Xcode 10. 

The primary new “feature” as compared with the 5.0a1 is that this release includes binary wheels for all supported versions of Python (2.7, 3.4, 3.5, 3.6 and 3.7, and for 2.7, 3.6 and 3.7 there are wheels for both the older “intel” installer and the newer 64-bit installer on Python.org) 

Changes w.r.t. PyObjC 5.0a1:

* Bindings updated for Xcode 10 beta 6.

* Add a custom binding for a number of structure types in
  CoreAudio:

  - AudioBuffer
  - AudioBufferList
  - AudioChannelDescription
  - AudioChannelLayout
  - AudioValueTranslation

  With this patch using APIs with these types should actually
  work.

* PR19: Fix deprecation warning in bridgesupport support module

  Patch by: Mickaël Schoentgen

* Creating objc.ObjCPointer instances now results in a
  Python warning, instead of an unconditional message on
  stdout.

  .. note::

     The creation of these objects is a sign that APIs are
     not wrapped correctly, these objects are created for
     pointers where the bridge doesn't know how to handle
     them properly.

* System bridgesupport XML files (normally not used by PyObjC)
  can contain constant numbers with value "inf", PyObjC now
  knows how to handle those.

* Added bindings for the "Metadata" subframework of the
  "CoreServices" framework.

* Added bindings for the "CarbonCore" subframework of the
  "CoreServices" framework.

  Most APIs in this subframework are not available to Python,
  only those APIs that are not deprecated and seem interesting
  are exposed.

* The separate framework wrappers DictionaryServices,
  LaunchServices and SearchKit are deprecated, use
  the CoreServices bindings instead.

  These framework wrappers still exists, but are effectively
  aliases for CoreServices with this release. Because of this
  these bindings can expose more symbols than previously.

* Fix unexpected exception when trying to call getattr
  on a framework wrapped with a name that isn't a valid
  identifier.

* #244: Bad metadata for CGPDFOperatorTableSetCallback

* #247: Fix crash in regression test case

  One specific test in pyobjc-core crashed the interpreter
  when run separately. Because of this I've disabled an
  optimization that uses alloca instead of PyMem_Malloc to
  allocate memory for now.


Regards,

  Ronald

[Pyobjc-dev] [ANN] py2app 0.16 and macholib 1.11

[Ronald O. <ron...@ma...>]

Hi,

I’ve released py2app 0.16 and macholib 1.11 with a couple of small changes.

The most important fix is in macholib and avoids the "New Mach-O header is too large to relocate in …” error when using Python wheels containing shared libraries (such as Pillow).



A full list of changes in py2app 0.16:

* #244: Copy the Tcl/Tk support libraries into the application bundle for
  Python builds using a classic unix install of Tcl/Tk instead of a framework
  build.

  This results in working app bundles when a Python.org installation that
  includes Tcl/Tk (such as Python 3.7).

* Don't copy numpy into application just because the application uses
  Pillow.

* Add recipe for Pyside

  Patch by Alberto Sottile

And macholib 1.11:

* Add very hacky limited support for @loader_path. This is just
  enough to deal with extensions and dylibs found in Python
  binary wheels.


Regards,

Ronald

[Pyobjc-dev] 10.14 branch merged into default

[Ronald O. <ron...@ma...>]

Hi,

I’ve merged the 10.14 branch into the default branch and will continue the work towards PyObjC 5.0 there.

My focus for near future w.r.t. PyObjC will be on testing the bindings with various python versions and macOS releases and adding some manual wrappers (C code) for a couple of APIs in the CoreMedia, CoreMediaIO and MediaToolbox frameworks.

Ronald

P.S. readthedocs.com <http://readthedocs.com/> is doing maintenance, hence https://pyobjc.readthedocs.io/ <https://pyobjc.readthedocs.io/> does not yet reflect my merge at this time. That should resolve itself automatically. 

[Pyobjc-dev] [ANN] First alpha release for PyObjC 5.0

[Ronald O. <ron...@ma...>]

[This is a cross-post from http://blog.ronaldoussoren.net]

I pushed a first alpha release for PyObjC 5.0 to PyPI, it can be installed with “pip install –pre pyobc”.

The major change in the 5.0 release is the addition of API bindings for macOS 10.14. This release is mostly up-to-date w.r.t. developer beta 3 of that release.

Other than updating existing API bindings this release adds new framework bindings for the following frameworks:

	• AdSupport
	• CoreAudio (new in macOS 10.0)
	• CoreAudioKit (new in macOS 10.4)
	• CoreMedia (new in macOS 10.7)
	• CoreMediaIO (new in macOS 10.7)
	• DiscRecording (new in macOS 10.2)
	• DiscRecordingUI (new in macOS 10.2)
	• DVDPlayback (new in macOS 10.3)
	• MediaToolbox
	• NaturalLanguage
	• Network
	• OSAKit (new in macOS 10.4)
	• UserNotifications
	• VideoSubscriberAccount

The bindings for CoreAudio, CoreMedia and MediaToolbox aren’t fully usable in this release, I have to write C code for a number of APIs and data structures that cannot be accessed using the generic FFI in pyobjc-core.

This alpha release only included wheels for the 64-bit installer of Python 3.7 (the default download on Python.org), the final release will include the full set of wheels.

Footnote

The release is a week later and less complete than I had hoped earlier. The reason for that is primarily that I was too optimistic on the amount of work I’d be able to do before and during EuroPython. In the end I barely touched my computer for PyObjC work at EuroPython, and not at all during the trip around Scotland beforehand (both of which were good for me, but less so for making progress)

Regard,

  Ronald