Re: [Pyobjc-dev] notarization works! (was Re: not that)

[Glyph <gl...@tw...>]

On Sun, Oct 28, 2018, at 11:58 PM, Ronald Oussoren wrote:
> 
> 
>> On 29 Oct 2018, at 00:56, Glyph <gl...@tw...> wrote:
>> 
>> 
>> 
>>> On Oct 28, 2018, at 2:57 PM, Glyph <gl...@tw...> wrote:>>> 
>>>> I wonder what the “hardened runtime” option actually does and
>>>> enforces.   In 3.7 the line in ctypes/__init__.py that causes the
>>>> exception is a call that creates a dummy C function, and likely
>>>> triggers the first allocation for storing a libffi closure which
>>>> could be something the hardened runtime doesn’t like (being
>>>> writeable + executable memory).>>> 
>>> Interesting. Perhaps what I want is simply
>>> https://developer.apple.com/documentation/security/com_apple_security_cs_allow-unsigned-executable-memory
>>> then?  Any chance you know how to jam that into a `codesign` command
>>> line somehow? :-)>>> 
>> 
>> Thank you so much for this tip, Ronald!  This was much easier than I
>> anticipated, and things are working now!> 
> Great.
> 
>> 
>> The relevant entitlements file is literally just:
>> 
>>> <?xml version="1.0" encoding="UTF-8"?>
>>> <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "
>>> http://www.apple.com/DTDs/PropertyList-1.0.dtd">>>> <plist version="1.0">
>>> <dict>
>>> <key>com.apple.security.cs.allow-unsigned-executable-memory</key>
>>> <true/>
>>> </dict>
>>> </plist>
>> 
>> I dropped that in a file, added `--entitlements=$THAT_FILE.plist` to
>> my codesign invocations, removed all my workarounds for ctypes et.
>> al. (except for the hard-coded 'import _cffi_backend' still necessary
>> to convince modulegraph to include enough code for SSL to work), and
>> then tried launching my app.> 
> Which package needs _cffi_backend? I can add a recipe for that to
> py2app to do this automagically.
This may sound obvious, but: cffi :-).  In my case, pyOpenSSL ->
cryptography -> cffi.
> 
>> Success!  Then I tried notarizing it: also success!  Time permitting,
>> I'll be updating my blog post at
>> https://glyph.twistedmatrix.com/2018/01/shipping-pygame-mac-app.html
>> with this information, and possibly publishing the now unfortunately
>> somewhat complex tooling I use to do signing now.>> 
>> So I don't know if I'm the first to *do* this, but looking at the
>> archives for these lists I seem to be the first to *report* it: you
>> can successfully codesign and notarize apps created with py2app and
>> python 3.6!>> 
>> It seems to me that whatever "MAP_JIT" is (an mmap flag, I'm
>> guessing?) libffi needs to be using it for the memory it places
>> synthetic closures into, so that this entitlement won't be necessary
>> with some future version of Python.  But it looks like Apple is not
>> pushing particularly hard to deprecate this one right now, thank
>> goodness :-).> 
> MAP_JIT is a mmap flag that’s apparently introduced in 10.14. The
> slides at https://developer.apple.com/videos/play/wwdc2018/702/
> mention this flag and the hardened runtime.> 
> I guess we should add this flag to the code in
> Modules/_ctypes/malloc_closure.c CPython) and in the similar code in
> PyObjC.  The annoying bit is that the flag is new in 10.14, and
> CPython installers are created on 10.9 which means those won’t include
> the new flag for a long time.> 
> I’ll have to check if using MAP_JIT is ok when deploying on older
> macOS versions, or if the code should do a runtime version check.
I can't shed any light on this, but I suspect the cffi folks will also
have to figure this out, and may already have some sense of how this
works.  I filed an issue with them here:
https://bitbucket.org/cffi/cffi/issues/391/cffi-doesnt-work-inside-a-macos-app-bundle