[Pyobjc-dev] py2app apps codesigned with "--options runtime", i.e. any notorizable app, fails to launch with ctypes error (was Re: ctypes MemoryError when attempting to launch notarized app)

[Glyph <gl...@tw...>]

> On Oct 28, 2018, at 2:27 AM, Ronald Oussoren <ron...@ma...> wrote:
> 
> 
> 
>> On 28 Oct 2018, at 06:27, Glyph <gl...@tw...> wrote:
>> 
>> I adjusted my code-signing to use the new, stricter requirements imposed by app notarization. I managed to get it successfully notarized, but the app is now non-functional as a result: at startup, I get:
>> 
>> Traceback (most recent call last):
>>  File "my.app/Contents/Resources/__boot__.py", line 93, in <module>
>>    _setup_ctypes()
>>  File "my.app/Contents/Resources/__boot__.py", line 86, in _setup_ctypes
>>    from ctypes.macholib import dyld
>>  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
>>  File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
>>  File "<frozen importlib._bootstrap>", line 656, in _load_unlocked
>>  File "<frozen importlib._bootstrap>", line 626, in _load_backward_compatible
>>  File "ctypes/__init__.pyc", line 538, in <module>
>>  File "ctypes/__init__.pyc", line 273, in _reset_cache
>> 
>> (If anyone wants to follow along in the traceback, this is using python.org 3.6.6.)
> 
> On what version of macOS? I expect 10.14 because that’s the only release that actually knows about notarization, but enabling this feature might also affect how the app is signed.

ProductName:	Mac OS X
ProductVersion:	10.14
BuildVersion:	18A391

>> This happens before any of my code even runs, so I can't just try to avoid ctypes.
> 
> You could patch the __boot__.py file before signing to see if that helps. Although this should cause problems further on, the call to _setup_ctypes should only be created when some code in your app bundle has a dependency on ctypes. 

I'll give that a shot.

>> 
>> Curiously, this is the same traceback that comes from https://forum.kodi.tv/showthread.php?tid=329171 <https://forum.kodi.tv/showthread.php?tid=329171>, which suggests it's something fundamental to strict shared-library sandboxing that ctypes trips over when trying to initialize itself.
>> 
>> Does anyone have experience with this, or ideas about what to do?
> 
> I’m afraid not. I currently get away with not signing apps at all, although properly supporting signing is on my way too long wish list for py2app.  

The ability to distribute unsigned apps is not-so-slowly going away; even the ability to distribute non-notarized apps has a very limited shelf-life at this point.  So this ought to be an alarming development for everyone - having Python apps effectively banned from macOS distribution is a big potential problem :-\.

The good news here is that aside from having to write a little for loop in shell (shown below) getting the app codesigned previously was easy, and my app *did* pass notarization, so nothing that py2app is doing is breaking things on apple's end.  It's just a matter of a ctypes bug.

As I see it, there's 2 problems here:

py2app's __boot__.py should fail more gracefully if initializing ctypes doesn't work, since not everybody needs ctypes.  Shall I file this on the tracker?
ctypes itself should address whatever eldritch hideousness is causing this; in addition to the windows security layer stuff I found, grsecurity TPE causes the same traceback: https://bugs.python.org/issue28429

> With some luck there’s some entitlement or code signing option that causes this problem.  What is the output of "codesign --display --verbose=4” for the application? Both with and without notarisation? 

Sorry, my original message was not clear.  App notarization itself is not the problem, it's the "stricter requirements" that I ambiguously referenced.  The requirement in question is the '--options runtime' flag passed to 'codesign'.  So you can just codesign an app (even with an ad-hoc identity, you technically could do this without even having a valid cert, although the way one generates one of those escapes me) with the 'runtime' option, you can reproduce this.

So if I sign my app like this:

#!/bin/bash
find "${NAME}.app" -iname '*.so' -or -iname '*.dylib' |
    while read libfile; do
          codesign --sign "${IDENTITY}" \
                   --deep "${libfile}" \
                   --force \
                   --options runtime;
    done;

codesign --sign "${IDENTITY}" \
         --deep "${NAME}.app" \
         --force \
         --options runtime;

and then run it as "./${NAME}.app/Contents/MacOS/${NAME}".  I immediately get the traceback given above.

-glyph