pyobjc-dev — CVS commit notices go here

Re: [Pyobjc-dev] [Pythonmac-SIG] PyObjC 3.2 released

[Glyph L. <gl...@tw...>]

> On Dec 8, 2016, at 3:37 AM, Ronald Oussoren <ron...@ma...> wrote:
> 
> 
>> On 8 Dec 2016, at 00:40, Glyph Lefkowitz <gl...@tw... <mailto:gl...@tw...>> wrote:
>>> 
>>> 
>>> Last time I checked systems like Homebrew only build 64-bit binaries (on anything resembling modern hardware). It should be possible to support both i386/x86_64 python.org <http://python.org/> and homebrew using the same wheel file by tweaking the platform tag. The filename of the wheels will get annoyingly long, but that’s something users won’t see anyway.   If that doesn’t work out I’ll install homebrew on my build machine and have the build script generate wheels for that as well. 
>> 
>> You shouldn't need to do this; and in fact you shouldn't do it :).  If you tweak the platform tag, you'll break Pip's ability to discover the wheel as valid-to-install for the current platform.  Platform tags are a descriptive mechanism for Python, Pip, and Setuptools, not a point for user extensibility.
> 
> I know how platform tags work, I’ve read the relevant PEPs when the were proposed. The “tweak” I wanted to use is documented as “Compressed Tag Sets” in PEP 425, and is how the python version is added to the wheel filename for wheels supporting both python2 and python3, as in:
> 
>    pyobjc_framework_XgridFoundation-3.3a0-py2.py3-none-any.whl
> 
> This should work in the platform tag as well, and is already used by wheels for matplotlib (see https://pypi.python.org/pypi/matplotlib <https://pypi.python.org/pypi/matplotlib>).

Aah, I was reading you backwards.  I thought you were saying something about inventing a platform tag for Homebrew (since they don't have one of their own; they build specifically to be compatible with system python).  This makes sense.

>> There are a lot of fiddly nuances here, but basically, "intel" is the architecture for "fat binary, intel", and "x86_64" is the architecture for "64-bit".  (I have no idea what the story is on 10.5 and previous but IMHO you can skip bothering to build wheels for them for now).  If you want to build release wheels, just build "intel" (fat binary) and any Python will be happy with them.
> 
> I wrote the distutils code that sets the platform to a sane value for fat binaries on OSX ;-).  

Heh. I think we both know a little too much about this for our own good :-).

> That’s cool, this means pip did implement the logic that’s needed to understand that “intel” is an acceptable architecture when looking for a wheel for an “x86_64” build of CPython.

I believe it understands the tags pretty thoroughly; I haven't tested, but I believe it also knows that an x86_64 wheel won't cut it on an 'intel' python because it might blow up later.

>> 
>> This is probably worth reading, as it explains the above in a lot more detail: https://github.com/MacPython/wiki/wiki/Spinning-wheels <https://github.com/MacPython/wiki/wiki/Spinning-wheels>.  But the lede is: just use the default behavior of a fat binary Python, it will do the right thing :).
> 
> Looking at that page the relevant logic was added in pip 6, that’s old enough to not worry about using a compressed tag set for the platform. 

Cool.  I'm really looking forward to never building pyobjc again :-).

-glyph

Re: [Pyobjc-dev] [Pythonmac-SIG] PyObjC 3.2 released

[Ronald O. <ron...@ma...>]

> On 8 Dec 2016, at 00:40, Glyph Lefkowitz <gl...@tw...> wrote:
>> 
>> 
>> Last time I checked systems like Homebrew only build 64-bit binaries (on anything resembling modern hardware). It should be possible to support both i386/x86_64 python.org <http://python.org/> and homebrew using the same wheel file by tweaking the platform tag. The filename of the wheels will get annoyingly long, but that’s something users won’t see anyway.   If that doesn’t work out I’ll install homebrew on my build machine and have the build script generate wheels for that as well. 
> 
> You shouldn't need to do this; and in fact you shouldn't do it :).  If you tweak the platform tag, you'll break Pip's ability to discover the wheel as valid-to-install for the current platform.  Platform tags are a descriptive mechanism for Python, Pip, and Setuptools, not a point for user extensibility.

I know how platform tags work, I’ve read the relevant PEPs when the were proposed. The “tweak” I wanted to use is documented as “Compressed Tag Sets” in PEP 425, and is how the python version is added to the wheel filename for wheels supporting both python2 and python3, as in:

   pyobjc_framework_XgridFoundation-3.3a0-py2.py3-none-any.whl

This should work in the platform tag as well, and is already used by wheels for matplotlib (see https://pypi.python.org/pypi/matplotlib <https://pypi.python.org/pypi/matplotlib>).

> 
> 
> There are a lot of fiddly nuances here, but basically, "intel" is the architecture for "fat binary, intel", and "x86_64" is the architecture for "64-bit".  (I have no idea what the story is on 10.5 and previous but IMHO you can skip bothering to build wheels for them for now).  If you want to build release wheels, just build "intel" (fat binary) and any Python will be happy with them.

I wrote the distutils code that sets the platform to a sane value for fat binaries on OSX ;-).  

> 
> Regardless of what architecture you run it under - and I did test this before sending this message :) - system python will produce _intel wheels; Homebrew produces _x86-64 wheels.  Cracking one open, this is what both 32- and 64-bit system python built:
> 
> $ file CoreFoundation/_CoreFoundation.so 
> CoreFoundation/_CoreFoundation.so: Mach-O universal binary with 2 architectures: [i386: Mach-O bundle i386] [x86_64: Mach-O 64-bit bundle x86_64]
> CoreFoundation/_CoreFoundation.so (for architecture i386):	Mach-O bundle i386
> CoreFoundation/_CoreFoundation.so (for architecture x86_64):	Mach-O 64-bit bundle x86_64
> 
> and this is what Homebrew built:
> 
> $ file CoreFoundation/_CoreFoundation.so 
> CoreFoundation/_CoreFoundation.so: Mach-O 64-bit bundle x86_64
> 
> Homebrew could happily 'pip install *.whl' in the directory of _intel wheels built by system Python, and I imported the module and made a couple of objc method calls, which worked fine, just to make double extra sure my understanding is correct.

That’s cool, this means pip did implement the logic that’s needed to understand that “intel” is an acceptable architecture when looking for a wheel for an “x86_64” build of CPython.

> 
> This is probably worth reading, as it explains the above in a lot more detail: https://github.com/MacPython/wiki/wiki/Spinning-wheels <https://github.com/MacPython/wiki/wiki/Spinning-wheels>.  But the lede is: just use the default behavior of a fat binary Python, it will do the right thing :).

Looking at that page the relevant logic was added in pip 6, that’s old enough to not worry about using a compressed tag set for the platform. 

Ronald

Re: [Pyobjc-dev] [Pythonmac-SIG] PyObjC 3.2 released

[Glyph L. <gl...@tw...>]

> On Dec 7, 2016, at 12:21 AM, Ronald Oussoren <ron...@ma...> wrote:
> 
> 
>> On 7 Dec 2016, at 00:28, Glyph Lefkowitz <gl...@tw... <mailto:gl...@tw...>> wrote:
>> 
>> Was this intentionally off-list?  (Feel free to forward my reply back to the list if not).
> 
> It wasn’t and I’ll continue the conversation on-list.
> 
>> 
>>> On Dec 6, 2016, at 2:31 PM, Ronald Oussoren <ron...@ma... <mailto:ron...@ma...>> wrote:
>>> 
>>>> 
>>>> On 6 Dec 2016, at 21:43, Glyph Lefkowitz <gl...@tw... <mailto:gl...@tw...>> wrote:
>>>> 
>>>>> 
>>>>> On Dec 6, 2016, at 12:17 PM, Ronald Oussoren <ron...@ma... <mailto:ron...@ma...>> wrote:
>>>>> 
>>>>>> 
>>>>>> On 6 Dec 2016, at 20:30, Glyph Lefkowitz <gl...@tw... <mailto:gl...@tw...>> wrote:
>>>>>> 
>>>>>> 
>>>>>>> On Dec 6, 2016, at 10:36 AM, Ronald Oussoren <ron...@ma... <mailto:ron...@ma...>> wrote:
>>>>>>> 
>>>>>>> 
>>>>>>>> On 6 Dec 2016, at 19:19, Glyph Lefkowitz <gl...@tw... <mailto:gl...@tw...>> wrote:
>>>>>>>> 
>>>>>>>> 
>>>>>>>>> On Dec 6, 2016, at 8:59 AM, Ronald Oussoren <ron...@ma... <mailto:ron...@ma...>> wrote:
>>>>>>>>> 
>>>>>>>>> Hi,
>>>>>>>>> 
>>>>>>>>> I’ve pushed PyObjC 3.2 to PyPI (finally…). The major new feature in this version is support for OSX 10.12 (Sierra) and the new APIs introduced in this version of OSX.
>>>>>>>> 
>>>>>>>> 🎉😃🎉
>>>>>>>> 
>>>>>>>>> There is also a backward incompatible change: the default method signature is calculated differently than before. This will primarily affect code that adds python-only method to a class, like so:
>>>>>>>>> 
>>>>>>>>> class MyObject (NSObject):
>>>>>>>>>     def amethod(self, a, b): pass
>>>>>>>> 
>>>>>>>> Huh, I didn't even realize that was possible before.  Hopefully none of my code will be affected as a result :-).
>>>>>>> 
>>>>>>> It wasn’t really documented, but I wouldn’t be surprised if someone relies on the previous behavior…
>>>>>>> 
>>>>>>>> 
>>>>>>>> Is there any chance that now this release is done, wheel builds might be on the horizon? :)
>>>>>>> 
>>>>>>> Maybe.  The important bit w.r.t. wheel builds is that they must be automated, otherwise releases take too much time and I end up not doing releases at all.  That said, automating this shouldn’t be too hard and should be able to save me some time in the end (I currently also test manually…)
>>>>>> 
>>>>>> If manual source releasing is `python setup.py sdist && twine upload dist/*`, then "manual" wheel releasing is `python setup.py sdist bdist_wheel && twine upload dist/*`.  It doesn't seem like that should be a crushing maintenance burden :-).
>>>>>> 
>>>>>> Although perhaps there's something I'm missing?  The subtleties of binary compatibility often escape me.
>>>>> 
>>>>> That’s about it, although I don’t use twine (see <https://bitbucket.org/ronaldoussoren/pyobjc/src/3e6dce3dcfa0a3a72f9a204855d83644eaf6afc0/development-support/upload-release?at=default&fileviewer=file-view-default <https://bitbucket.org/ronaldoussoren/pyobjc/src/3e6dce3dcfa0a3a72f9a204855d83644eaf6afc0/development-support/upload-release?at=default&fileviewer=file-view-default>>).  I’m slightly worried
>>>>> about binary compatibility and want to have some way to do smoke tests of the generated wheels on various platforms.
>>>> 
>>>> Twine is strongly recommended by the packaging community these days.  Mostly the security problems with `setup.py upload` have been fixed, but there's no easy way to check, especially on the command line, so it remains somewhat risky to use, especially in a scripted context.
>>>> 
>>>> That said, the process is mostly the same; just stick `bdist_wheel` in on that command line somewhere before `upload` and you're good to go.
>>> 
>>> Building wheels isn’t as bad as I thought, turns out I can build wheels for all subprojects of PyObjC on 10.12 with a small tweak of the setup.py file, even for the subprojects that won’t work on 10.12 (luckily none of those contain C extensions).
>> 
>> Excellent!
>> 
>>> I now have a script that builds and collects sdists as well as wheels for Python 2.7, 3.4, 3.5 and 3.6.  There now definitely will be a 3.2.1 release later this week, even if it only contains wheels (and possibly the 2.7 issue I ran into). I’ll probably only provide wheels for the 32/64-bit Intel installer on python.org <http://python.org/> (which is the default these days), possibly with some fine-tuning of the platform tag in the wheel name to support Homebrew as well (which AFAIK installs a 64-bit Python).
>> 
>> Oh, is the system python not 64-bit?  sys.maxint suggests that it is…
> 
> The system python is both 32-bit and 64-bit, depending on how you start it:
> 
> ronald@Menegroth[0]$ file /usr/bin/python
> /usr/bin/python: Mach-O universal binary with 2 architectures: [i386: Mach-O executable i386] [x86_64: Mach-O 64-bit executable x86_64]
> /usr/bin/python (for architecture i386):	Mach-O executable i386
> /usr/bin/python (for architecture x86_64):	Mach-O 64-bit executable x86_64
> 
> IIRC you have to use some special command-line option to specify the architecture to use with the system python, with python.org <http://python.org/> binaries you can use “arch -i386 python” to select the 32-bit binary.
> 
> There are two installers on python.org <http://python.org/>: one supports i386 and x86_64 on OSX 10.6 or later, the other supports i386 and ppc on OSX 10.5 (and possibly 10.4 as well).  The latter is no longer relevant.
> 
> Last time I checked systems like Homebrew only build 64-bit binaries (on anything resembling modern hardware). It should be possible to support both i386/x86_64 python.org <http://python.org/> and homebrew using the same wheel file by tweaking the platform tag. The filename of the wheels will get annoyingly long, but that’s something users won’t see anyway.   If that doesn’t work out I’ll install homebrew on my build machine and have the build script generate wheels for that as well. 

You shouldn't need to do this; and in fact you shouldn't do it :).  If you tweak the platform tag, you'll break Pip's ability to discover the wheel as valid-to-install for the current platform.  Platform tags are a descriptive mechanism for Python, Pip, and Setuptools, not a point for user extensibility.

There are a lot of fiddly nuances here, but basically, "intel" is the architecture for "fat binary, intel", and "x86_64" is the architecture for "64-bit".  (I have no idea what the story is on 10.5 and previous but IMHO you can skip bothering to build wheels for them for now).  If you want to build release wheels, just build "intel" (fat binary) and any Python will be happy with them.

Regardless of what architecture you run it under - and I did test this before sending this message :) - system python will produce _intel wheels; Homebrew produces _x86-64 wheels.  Cracking one open, this is what both 32- and 64-bit system python built:

$ file CoreFoundation/_CoreFoundation.so 
CoreFoundation/_CoreFoundation.so: Mach-O universal binary with 2 architectures: [i386: Mach-O bundle i386] [x86_64: Mach-O 64-bit bundle x86_64]
CoreFoundation/_CoreFoundation.so (for architecture i386):	Mach-O bundle i386
CoreFoundation/_CoreFoundation.so (for architecture x86_64):	Mach-O 64-bit bundle x86_64

and this is what Homebrew built:

$ file CoreFoundation/_CoreFoundation.so 
CoreFoundation/_CoreFoundation.so: Mach-O 64-bit bundle x86_64

Homebrew could happily 'pip install *.whl' in the directory of _intel wheels built by system Python, and I imported the module and made a couple of objc method calls, which worked fine, just to make double extra sure my understanding is correct.

This is probably worth reading, as it explains the above in a lot more detail: https://github.com/MacPython/wiki/wiki/Spinning-wheels <https://github.com/MacPython/wiki/wiki/Spinning-wheels>.  But the lede is: just use the default behavior of a fat binary Python, it will do the right thing :).

-glyph

Re: [Pyobjc-dev] [Pythonmac-SIG] PyObjC 3.2 released puObjC

[Ronald O. <ron...@ma...>]

> On 7 Dec 2016, at 01:49, Christopher Barker <pyt...@gm...> wrote:
> 
> I haven't been involved for a while, but this gitHub org:
> 
> https://github.com/MacPython <https://github.com/MacPython>
> 
> Is doing a lot of wheel building, and I think they are using CIs to build and test.
> 
> Maybe they could add pyObjC
> 
> Or you could see what they have set up.


They appear to use travis to build, with some script to sanitise binary dependencies (that is, copy non-system shared libraries into the wheel).  Might be interesting to look into for PyObjC as well, but for now manual building will do.

Ronald


> 
> -CHB
> 
> 
> 
> On Tue, Dec 6, 2016 at 12:56 PM Ronald Oussoren <ron...@ma... <mailto:ron...@ma...>> wrote:
>> On 6 Dec 2016, at 21:43, Glyph Lefkowitz <gl...@tw... <mailto:gl...@tw...>> wrote:
> 
>> 
>> 
>>> Anyways, there may be a 3.2.1 release by the end of the week, I just noticed that building PyObjC on 10.12 doesn’t work when using python 2.7, I apparently only tested python 2.7 on older OSX releases :-(
>> 
> 
>> Thanks for saving me some time - I was just about to test that configuration :).  It works fine on 10.11, which is what this machine is running!
> 
> What version of Xcode do you use?  And do you use a framework build of Python?  The issue I saw earlier should be present on 10.11 as well when using Xcode 8.1 (which includes the 10.12 SDK)
> 
> Ronald
> 
> _______________________________________________
> 
> Pythonmac-SIG maillist  -  Pyt...@py... <mailto:Pyt...@py...>
> 
> https://mail.python.org/mailman/listinfo/pythonmac-sig <https://mail.python.org/mailman/listinfo/pythonmac-sig>
> 
> unsubscribe: https://mail.python.org/mailman/options/Pythonmac-SIG <https://mail.python.org/mailman/options/Pythonmac-SIG>
> 

Re: [Pyobjc-dev] [Pythonmac-SIG] PyObjC 3.2 released

[Ronald O. <ron...@ma...>]

> On 7 Dec 2016, at 00:28, Glyph Lefkowitz <gl...@tw...> wrote:
> 
> Was this intentionally off-list?  (Feel free to forward my reply back to the list if not).

It wasn’t and I’ll continue the conversation on-list.

> 
>> On Dec 6, 2016, at 2:31 PM, Ronald Oussoren <ron...@ma... <mailto:ron...@ma...>> wrote:
>> 
>>> 
>>> On 6 Dec 2016, at 21:43, Glyph Lefkowitz <gl...@tw... <mailto:gl...@tw...>> wrote:
>>> 
>>>> 
>>>> On Dec 6, 2016, at 12:17 PM, Ronald Oussoren <ron...@ma... <mailto:ron...@ma...>> wrote:
>>>> 
>>>>> 
>>>>> On 6 Dec 2016, at 20:30, Glyph Lefkowitz <gl...@tw... <mailto:gl...@tw...>> wrote:
>>>>> 
>>>>> 
>>>>>> On Dec 6, 2016, at 10:36 AM, Ronald Oussoren <ron...@ma... <mailto:ron...@ma...>> wrote:
>>>>>> 
>>>>>> 
>>>>>>> On 6 Dec 2016, at 19:19, Glyph Lefkowitz <gl...@tw... <mailto:gl...@tw...>> wrote:
>>>>>>> 
>>>>>>> 
>>>>>>>> On Dec 6, 2016, at 8:59 AM, Ronald Oussoren <ron...@ma... <mailto:ron...@ma...>> wrote:
>>>>>>>> 
>>>>>>>> Hi,
>>>>>>>> 
>>>>>>>> I’ve pushed PyObjC 3.2 to PyPI (finally…). The major new feature in this version is support for OSX 10.12 (Sierra) and the new APIs introduced in this version of OSX.
>>>>>>> 
>>>>>>> 🎉😃🎉
>>>>>>> 
>>>>>>>> There is also a backward incompatible change: the default method signature is calculated differently than before. This will primarily affect code that adds python-only method to a class, like so:
>>>>>>>> 
>>>>>>>> class MyObject (NSObject):
>>>>>>>>     def amethod(self, a, b): pass
>>>>>>> 
>>>>>>> Huh, I didn't even realize that was possible before.  Hopefully none of my code will be affected as a result :-).
>>>>>> 
>>>>>> It wasn’t really documented, but I wouldn’t be surprised if someone relies on the previous behavior…
>>>>>> 
>>>>>>> 
>>>>>>> Is there any chance that now this release is done, wheel builds might be on the horizon? :)
>>>>>> 
>>>>>> Maybe.  The important bit w.r.t. wheel builds is that they must be automated, otherwise releases take too much time and I end up not doing releases at all.  That said, automating this shouldn’t be too hard and should be able to save me some time in the end (I currently also test manually…)
>>>>> 
>>>>> If manual source releasing is `python setup.py sdist && twine upload dist/*`, then "manual" wheel releasing is `python setup.py sdist bdist_wheel && twine upload dist/*`.  It doesn't seem like that should be a crushing maintenance burden :-).
>>>>> 
>>>>> Although perhaps there's something I'm missing?  The subtleties of binary compatibility often escape me.
>>>> 
>>>> That’s about it, although I don’t use twine (see <https://bitbucket.org/ronaldoussoren/pyobjc/src/3e6dce3dcfa0a3a72f9a204855d83644eaf6afc0/development-support/upload-release?at=default&fileviewer=file-view-default <https://bitbucket.org/ronaldoussoren/pyobjc/src/3e6dce3dcfa0a3a72f9a204855d83644eaf6afc0/development-support/upload-release?at=default&fileviewer=file-view-default>>).  I’m slightly worried
>>>> about binary compatibility and want to have some way to do smoke tests of the generated wheels on various platforms.
>>> 
>>> Twine is strongly recommended by the packaging community these days.  Mostly the security problems with `setup.py upload` have been fixed, but there's no easy way to check, especially on the command line, so it remains somewhat risky to use, especially in a scripted context.
>>> 
>>> That said, the process is mostly the same; just stick `bdist_wheel` in on that command line somewhere before `upload` and you're good to go.
>> 
>> Building wheels isn’t as bad as I thought, turns out I can build wheels for all subprojects of PyObjC on 10.12 with a small tweak of the setup.py file, even for the subprojects that won’t work on 10.12 (luckily none of those contain C extensions).
> 
> Excellent!
> 
>> I now have a script that builds and collects sdists as well as wheels for Python 2.7, 3.4, 3.5 and 3.6.  There now definitely will be a 3.2.1 release later this week, even if it only contains wheels (and possibly the 2.7 issue I ran into). I’ll probably only provide wheels for the 32/64-bit Intel installer on python.org <http://python.org/> (which is the default these days), possibly with some fine-tuning of the platform tag in the wheel name to support Homebrew as well (which AFAIK installs a 64-bit Python).
> 
> Oh, is the system python not 64-bit?  sys.maxint suggests that it is…

The system python is both 32-bit and 64-bit, depending on how you start it:

ronald@Menegroth[0]$ file /usr/bin/python
/usr/bin/python: Mach-O universal binary with 2 architectures: [i386: Mach-O executable i386] [x86_64: Mach-O 64-bit executable x86_64]
/usr/bin/python (for architecture i386):	Mach-O executable i386
/usr/bin/python (for architecture x86_64):	Mach-O 64-bit executable x86_64

IIRC you have to use some special command-line option to specify the architecture to use with the system python, with python.org <http://python.org/> binaries you can use “arch -i386 python” to select the 32-bit binary.

There are two installers on python.org <http://python.org/>: one supports i386 and x86_64 on OSX 10.6 or later, the other supports i386 and ppc on OSX 10.5 (and possibly 10.4 as well).  The latter is no longer relevant.

Last time I checked systems like Homebrew only build 64-bit binaries (on anything resembling modern hardware). It should be possible to support both i386/x86_64 python.org <http://python.org/> and homebrew using the same wheel file by tweaking the platform tag. The filename of the wheels will get annoyingly long, but that’s something users won’t see anyway.   If that doesn’t work out I’ll install homebrew on my build machine and have the build script generate wheels for that as well. 

Ronald

Re: [Pyobjc-dev] [Pythonmac-SIG] PyObjC 3.2 released puObjC

[Christopher B. <pyt...@gm...>]

I haven't been involved for a while, but this gitHub org:

https://github.com/MacPython

Is doing a lot of wheel building, and I think they are using CIs to build
and test.

Maybe they could add pyObjC

Or you could see what they have set up.

-CHB



On Tue, Dec 6, 2016 at 12:56 PM Ronald Oussoren <ron...@ma...>
wrote:

> On 6 Dec 2016, at 21:43, Glyph Lefkowitz <gl...@tw...> wrote:
>
>
>
> Anyways, there may be a 3.2.1 release by the end of the week, I just
> noticed that building PyObjC on 10.12 doesn’t work when using python 2.7, I
> apparently only tested python 2.7 on older OSX releases :-(
>
>
> Thanks for saving me some time - I was just about to test that
> configuration :).  It works fine on 10.11, which is what this machine is
> running!
>
>
> What version of Xcode do you use?  And do you use a framework build of
> Python?  The issue I saw earlier should be present on 10.11 as well when
> using Xcode 8.1 (which includes the 10.12 SDK)
>
> Ronald
>
> _______________________________________________
>
> Pythonmac-SIG maillist  -  Pyt...@py...
>
> https://mail.python.org/mailman/listinfo/pythonmac-sig
>
> unsubscribe: https://mail.python.org/mailman/options/Pythonmac-SIG
>
>

Re: [Pyobjc-dev] [Pythonmac-SIG] PyObjC 3.2 released

[Glyph L. <gl...@tw...>]

> On Dec 6, 2016, at 12:55 PM, Ronald Oussoren <ron...@ma...> wrote:
> 
> 
>> On 6 Dec 2016, at 21:43, Glyph Lefkowitz <gl...@tw... <mailto:gl...@tw...>> wrote:
>> 
>> 
>>> Anyways, there may be a 3.2.1 release by the end of the week, I just noticed that building PyObjC on 10.12 doesn’t work when using python 2.7, I apparently only tested python 2.7 on older OSX releases :-(
>> 
>> Thanks for saving me some time - I was just about to test that configuration :).  It works fine on 10.11, which is what this machine is running!
> 
> What version of Xcode do you use?  And do you use a framework build of Python?  The issue I saw earlier should be present on 10.11 as well when using Xcode 8.1 (which includes the 10.12 SDK)

So... technically the answer for Xcode is "8.1" but when I launched it to find out it asked me to "install required components"... but even after installing them it builds fine (with pip --no-use-wheel to avoid using my previously-cached wheels from the earlier attempt).  I also tried pip wheel --no-cache-dir just in case; it works that way as well.

I use a framework build of 2.7, from Homebrew.

Given the confusion you've expressed, I went ahead and tried to build this for my Sierra machine as well, and it worked there too.  So as far as I can tell there is no bug, but I haven't tried with the system-provided Python.

Happy to report any other salient details though; let me know.

-glyph

Re: [Pyobjc-dev] [Pythonmac-SIG] PyObjC 3.2 released

[Ronald O. <ron...@ma...>]

> On 6 Dec 2016, at 21:43, Glyph Lefkowitz <gl...@tw...> wrote:
> 
> 
>> Anyways, there may be a 3.2.1 release by the end of the week, I just noticed that building PyObjC on 10.12 doesn’t work when using python 2.7, I apparently only tested python 2.7 on older OSX releases :-(
> 
> Thanks for saving me some time - I was just about to test that configuration :).  It works fine on 10.11, which is what this machine is running!

What version of Xcode do you use?  And do you use a framework build of Python?  The issue I saw earlier should be present on 10.11 as well when using Xcode 8.1 (which includes the 10.12 SDK)

Ronald

Re: [Pyobjc-dev] [Pythonmac-SIG] PyObjC 3.2 released

[Glyph L. <gl...@tw...>]

> On Dec 6, 2016, at 12:17 PM, Ronald Oussoren <ron...@ma...> wrote:
> 
>> 
>> On 6 Dec 2016, at 20:30, Glyph Lefkowitz <gl...@tw... <mailto:gl...@tw...>> wrote:
>> 
>> 
>>> On Dec 6, 2016, at 10:36 AM, Ronald Oussoren <ron...@ma... <mailto:ron...@ma...>> wrote:
>>> 
>>> 
>>>> On 6 Dec 2016, at 19:19, Glyph Lefkowitz <gl...@tw... <mailto:gl...@tw...>> wrote:
>>>> 
>>>> 
>>>>> On Dec 6, 2016, at 8:59 AM, Ronald Oussoren <ron...@ma... <mailto:ron...@ma...>> wrote:
>>>>> 
>>>>> Hi,
>>>>> 
>>>>> I’ve pushed PyObjC 3.2 to PyPI (finally…). The major new feature in this version is support for OSX 10.12 (Sierra) and the new APIs introduced in this version of OSX.
>>>> 
>>>> 🎉😃🎉
>>>> 
>>>>> There is also a backward incompatible change: the default method signature is calculated differently than before. This will primarily affect code that adds python-only method to a class, like so:
>>>>> 
>>>>> class MyObject (NSObject):
>>>>>     def amethod(self, a, b): pass
>>>> 
>>>> Huh, I didn't even realize that was possible before.  Hopefully none of my code will be affected as a result :-).
>>> 
>>> It wasn’t really documented, but I wouldn’t be surprised if someone relies on the previous behavior…
>>> 
>>>> 
>>>> Is there any chance that now this release is done, wheel builds might be on the horizon? :)
>>> 
>>> Maybe.  The important bit w.r.t. wheel builds is that they must be automated, otherwise releases take too much time and I end up not doing releases at all.  That said, automating this shouldn’t be too hard and should be able to save me some time in the end (I currently also test manually…)
>> 
>> If manual source releasing is `python setup.py sdist && twine upload dist/*`, then "manual" wheel releasing is `python setup.py sdist bdist_wheel && twine upload dist/*`.  It doesn't seem like that should be a crushing maintenance burden :-).
>> 
>> Although perhaps there's something I'm missing?  The subtleties of binary compatibility often escape me.
> 
> That’s about it, although I don’t use twine (see <https://bitbucket.org/ronaldoussoren/pyobjc/src/3e6dce3dcfa0a3a72f9a204855d83644eaf6afc0/development-support/upload-release?at=default&fileviewer=file-view-default <https://bitbucket.org/ronaldoussoren/pyobjc/src/3e6dce3dcfa0a3a72f9a204855d83644eaf6afc0/development-support/upload-release?at=default&fileviewer=file-view-default>>).  I’m slightly worried
> about binary compatibility and want to have some way to do smoke tests of the generated wheels on various platforms.

Twine is strongly recommended by the packaging community these days.  Mostly the security problems with `setup.py upload` have been fixed, but there's no easy way to check, especially on the command line, so it remains somewhat risky to use, especially in a scripted context.

That said, the process is mostly the same; just stick `bdist_wheel` in on that command line somewhere before `upload` and you're good to go.

> In theory binary compatibility shouldn’t be a problem, I used to do builds on newer OSX releases when the system where the application was running but that was a couple of releases ago (both of PyObjC and OSX).
> 
> Binary compatibility is likely a larger problem for CPython itself, due to configure picking up some definitions that aren’t available on older OSX releases (IIRC functions like openat(2) are new in 10.10, that causes problems with CPython 3.5 that uses there functions when avaiable.  Nothing too hard, but also not something I have a real need for at the moment (and adding weak-linking code-paths could result in patches that won’t be accepted…).  
> 
> BTW. The reason I’m avoiding this work is more that I’d like to be able to kick of a script that runs a long time and reports on test results on a number of OSX and Python versions, instead of me doing that work. I think I have a plan to get there, and that should make it fairly trivial to generate wheels (and sdists) as a side effect of the test run.

IMHO the pareto principle applies here.  Even with potentially "bad" (untested) wheels, for most people, using recent OS X, `pip instal pyobjc` will go a zillion times faster (which is useful even for expert developers), and will work even on computers without a compiler installed.  This is especially useful for e.g. kids trying out programming for the first time on their Mac and trying to run some project from PyPI; it can make the difference between "works perfectly" and "totally inscrutable failure".  For a lot of projects you can just say "oh, just run xcode-select --install", but for pyobjc you actually need all the SDK headers, so you need an apple ID and at that point we've already lost the game for such casual users.  (To be clear, this is not a hypothetical concern; I have had several potentially interested students give up on Python and instead learn AppleScript due to this problem.)

For those on very old OSes (10.9 is not supported by Apple any more, is it?) pip install --no-binary :all: will helpfully use sdists from PyPI and do local builds as before; and given the distribution landscape, those users probably have to do that anyway.  So I think doing the dumb, untested wheel release will still be a huge benefit overall, and does not break things irreparably even if it works very poorly.  In the absolute nightmare worst-case scenario where the uploaded wheels just cause impossible to diagnose segfaults for some users, a +.1 release bump with only sdists will restore things back to normal.  (But if this were the case, it would not be possible to build apps that users could run with pyobjc, and... that seems to work fine :) so it seems likely to work.)

Basically, wheels will definitely make the situation a lot better for a really big audience, while possibly making it a little worse for a very small audience.

I will happily volunteer to do wheel uploads myself if you would add me to the project on PyPI.  I'd rather do the builds straight from the same environment doing the testing and sdist (i.e. yours) but I am fairly confident that I can build working pyobjc wheels since I do it every couple of days for my own use :).

> Anyways, there may be a 3.2.1 release by the end of the week, I just noticed that building PyObjC on 10.12 doesn’t work when using python 2.7, I apparently only tested python 2.7 on older OSX releases :-(

Thanks for saving me some time - I was just about to test that configuration :).  It works fine on 10.11, which is what this machine is running!

-glyph

Re: [Pyobjc-dev] [Pythonmac-SIG] PyObjC 3.2 released

[Ronald O. <ron...@ma...>]

> On 6 Dec 2016, at 20:30, Glyph Lefkowitz <gl...@tw...> wrote:
> 
> 
>> On Dec 6, 2016, at 10:36 AM, Ronald Oussoren <ron...@ma...> wrote:
>> 
>> 
>>> On 6 Dec 2016, at 19:19, Glyph Lefkowitz <gl...@tw...> wrote:
>>> 
>>> 
>>>> On Dec 6, 2016, at 8:59 AM, Ronald Oussoren <ron...@ma...> wrote:
>>>> 
>>>> Hi,
>>>> 
>>>> I’ve pushed PyObjC 3.2 to PyPI (finally…). The major new feature in this version is support for OSX 10.12 (Sierra) and the new APIs introduced in this version of OSX.
>>> 
>>> 🎉😃🎉
>>> 
>>>> There is also a backward incompatible change: the default method signature is calculated differently than before. This will primarily affect code that adds python-only method to a class, like so:
>>>> 
>>>> class MyObject (NSObject):
>>>>     def amethod(self, a, b): pass
>>> 
>>> Huh, I didn't even realize that was possible before.  Hopefully none of my code will be affected as a result :-).
>> 
>> It wasn’t really documented, but I wouldn’t be surprised if someone relies on the previous behavior…
>> 
>>> 
>>> Is there any chance that now this release is done, wheel builds might be on the horizon? :)
>> 
>> Maybe.  The important bit w.r.t. wheel builds is that they must be automated, otherwise releases take too much time and I end up not doing releases at all.  That said, automating this shouldn’t be too hard and should be able to save me some time in the end (I currently also test manually…)
> 
> If manual source releasing is `python setup.py sdist && twine upload dist/*`, then "manual" wheel releasing is `python setup.py sdist bdist_wheel && twine upload dist/*`.  It doesn't seem like that should be a crushing maintenance burden :-).
> 
> Although perhaps there's something I'm missing?  The subtleties of binary compatibility often escape me.

That’s about it, although I don’t use twine (see <https://bitbucket.org/ronaldoussoren/pyobjc/src/3e6dce3dcfa0a3a72f9a204855d83644eaf6afc0/development-support/upload-release?at=default&fileviewer=file-view-default <https://bitbucket.org/ronaldoussoren/pyobjc/src/3e6dce3dcfa0a3a72f9a204855d83644eaf6afc0/development-support/upload-release?at=default&fileviewer=file-view-default>>).  I’m slightly worried
about binary compatibility and want to have some way to do smoke tests of the generated wheels on various platforms.

In theory binary compatibility shouldn’t be a problem, I used to do builds on newer OSX releases when the system where the application was running but that was a couple of releases ago (both of PyObjC and OSX).  

Binary compatibility is likely a larger problem for CPython itself, due to configure picking up some definitions that aren’t available on older OSX releases (IIRC functions like openat(2) are new in 10.10, that causes problems with CPython 3.5 that uses there functions when avaiable.  Nothing too hard, but also not something I have a real need for at the moment (and adding weak-linking code-paths could result in patches that won’t be accepted…).  

BTW. The reason I’m avoiding this work is more that I’d like to be able to kick of a script that runs a long time and reports on test results on a number of OSX and Python versions, instead of me doing that work. I think I have a plan to get there, and that should make it fairly trivial to generate wheels (and sdists) as a side effect of the test run.

Anyways, there may be a 3.2.1 release by the end of the week, I just noticed that building PyObjC on 10.12 doesn’t work when using python 2.7, I apparently only tested python 2.7 on older OSX releases :-(

Ronald

> 
> -glyph
> 

Re: [Pyobjc-dev] [Pythonmac-SIG] PyObjC 3.2 released

[Glyph L. <gl...@tw...>]

> On Dec 6, 2016, at 10:36 AM, Ronald Oussoren <ron...@ma...> wrote:
> 
> 
>> On 6 Dec 2016, at 19:19, Glyph Lefkowitz <gl...@tw...> wrote:
>> 
>> 
>>> On Dec 6, 2016, at 8:59 AM, Ronald Oussoren <ron...@ma...> wrote:
>>> 
>>> Hi,
>>> 
>>> I’ve pushed PyObjC 3.2 to PyPI (finally…). The major new feature in this version is support for OSX 10.12 (Sierra) and the new APIs introduced in this version of OSX.
>> 
>> 🎉😃🎉
>> 
>>> There is also a backward incompatible change: the default method signature is calculated differently than before. This will primarily affect code that adds python-only method to a class, like so:
>>> 
>>> class MyObject (NSObject):
>>>      def amethod(self, a, b): pass
>> 
>> Huh, I didn't even realize that was possible before.  Hopefully none of my code will be affected as a result :-).
> 
> It wasn’t really documented, but I wouldn’t be surprised if someone relies on the previous behavior…
> 
>> 
>> Is there any chance that now this release is done, wheel builds might be on the horizon? :)
> 
> Maybe.  The important bit w.r.t. wheel builds is that they must be automated, otherwise releases take too much time and I end up not doing releases at all.  That said, automating this shouldn’t be too hard and should be able to save me some time in the end (I currently also test manually…)

If manual source releasing is `python setup.py sdist && twine upload dist/*`, then "manual" wheel releasing is `python setup.py sdist bdist_wheel && twine upload dist/*`.  It doesn't seem like that should be a crushing maintenance burden :-).

Although perhaps there's something I'm missing?  The subtleties of binary compatibility often escape me.

-glyph

Re: [Pyobjc-dev] [Pythonmac-SIG] PyObjC 3.2 released

[Ronald O. <ron...@ma...>]

> On 6 Dec 2016, at 19:19, Glyph Lefkowitz <gl...@tw...> wrote:
> 
> 
>> On Dec 6, 2016, at 8:59 AM, Ronald Oussoren <ron...@ma...> wrote:
>> 
>> Hi,
>> 
>> I’ve pushed PyObjC 3.2 to PyPI (finally…). The major new feature in this version is support for OSX 10.12 (Sierra) and the new APIs introduced in this version of OSX.
> 
> 🎉😃🎉
> 
>> There is also a backward incompatible change: the default method signature is calculated differently than before. This will primarily affect code that adds python-only method to a class, like so:
>> 
>>  class MyObject (NSObject):
>>       def amethod(self, a, b): pass
> 
> Huh, I didn't even realize that was possible before.  Hopefully none of my code will be affected as a result :-).

It wasn’t really documented, but I wouldn’t be surprised if someone relies on the previous behavior…

> 
> Is there any chance that now this release is done, wheel builds might be on the horizon? :)

Maybe.  The important bit w.r.t. wheel builds is that they must be automated, otherwise releases take too much time and I end up not doing releases at all.  That said, automating this shouldn’t be too hard and should be able to save me some time in the end (I currently also test manually…)

Ronald

Re: [Pyobjc-dev] [Pythonmac-SIG] PyObjC 3.2 released

[Glyph L. <gl...@tw...>]

> On Dec 6, 2016, at 8:59 AM, Ronald Oussoren <ron...@ma...> wrote:
> 
> Hi,
> 
> I’ve pushed PyObjC 3.2 to PyPI (finally…). The major new feature in this version is support for OSX 10.12 (Sierra) and the new APIs introduced in this version of OSX.

🎉😃🎉

> There is also a backward incompatible change: the default method signature is calculated differently than before. This will primarily affect code that adds python-only method to a class, like so:
> 
>   class MyObject (NSObject):
>        def amethod(self, a, b): pass

Huh, I didn't even realize that was possible before.  Hopefully none of my code will be affected as a result :-).

Is there any chance that now this release is done, wheel builds might be on the horizon? :)

-glyph

[Pyobjc-dev] PyObjC 3.2 released

[Ronald O. <ron...@ma...>]

Hi,

I’ve pushed PyObjC 3.2 to PyPI (finally…). The major new feature in this version is support for OSX 10.12 (Sierra) and the new APIs introduced in this version of OSX.

There is also a backward incompatible change: the default method signature is calculated differently than before. This will primarily affect code that adds python-only method to a class, like so:

   class MyObject (NSObject):
        def amethod(self, a, b): pass

This would work in previous versions, but no longer works because the method name suggests that this is a method without arguments (the translation to an Objective-C selector contains no colons). The workaround for this is easy enough: use the decorator “objc.python_method” on methods that won’t be called from Objective-C.

This change was necessary to make it possible to use generic decorators on selectors. 

Ronald

Re: [Pyobjc-dev] [Pythonmac-SIG] PyObjC and macOS 10.12 (Sierra)

[Ronald O. <ron...@ma...>]

> On 24 Oct 2016, at 19:37, Ronald Oussoren <ron...@ma...> wrote:
> […]
> I can now reproduce the problem using a small extension that doesn’t even touch PyObjC. See <https://bitbucket.org/ronaldoussoren/pyobjc/issues/174/crash-using-dictionaryservices-on-1012 <https://bitbucket.org/ronaldoussoren/pyobjc/issues/174/crash-using-dictionaryservices-on-1012>>. There is some material difference between /usr/bin/python and a python.org <http://python.org/> binary installation of Python 3.5.  This demonstrates the same crash using small C extension that exports a function that calls DCSGetTermInRange.
> 
> The crash is in a background thread that appears to do work for DCSGetTermInRange.
> 
> Next steps:
> * Try to reproduce using python.org <http://python.org/> installer for 2.7
> * Try to reproduce using a local build of 3.5.2
> 
> Now where did I leave my Yak shaving kit?

The 2.7 framework build on python.org causes the same crash, a local unix build of 3.5.2 doesn’t. As this isn’t a PyObjC issue I won’t investigate this until after the next release.

The good news w.r.t. that release: I just finished my first pass through the entire testsuite on 10.12 and fixed all issues w.r.t. metadata. The next step toward a release is to perform testing on older OSX release to ensure I didn’t introduce issues there.

Ronald

Re: [Pyobjc-dev] [Pythonmac-SIG] PyObjC and macOS 10.12 (Sierra)

[Ronald O. <ron...@ma...>]

> On 25 Oct 2016, at 10:36, Andrew Jaffe <a.h...@gm...> wrote:
> 
> On 12/10/2016 21:02, Ronald Oussoren wrote:
> >
> >> On 12 Oct 2016, at 21:09, Glyph Lefkowitz <gl...@tw...
> >> <mailto:gl...@tw...>> wrote:
> >>
> >>> On Oct 12, 2016, at 11:49 AM, Andrew Jaffe <a.h...@gm...
> >>> <mailto:a.h...@gm...>> wrote:
> >>>
> >>> Not me. If I understand correctly, Glyph -- who undoubtedly
> >>> understand the situation better than I do -- still thinks that
> >>> there's no actual bug here, since we shouldn't be using the framework
> >>> build this way, but I'm not sure I understand/agree...
> >>
> >> To be honest, I'm not 100% sure this is a good idea either; it's just
> >> that I know Donald Stufft has had a terrible time with Apple system
> >> python for several years, and he regards this as a positive change.
> >>  From my personal perspective, there's a good case to be made that a
> >> python in /System should just load from /System and one in /Library
> >> should load only from /Library, similar to the way --prefix works on
> >> "regular" UNIX.  But, this is what we've got :).
> >
> > I don’t mind if the /System python looks in /Library for stuff that the
> > user installed there, but I do consider it a bug that Apple installs
> > system files in /Library because that affects all installations of
> > Python 2.7.
> >
> > That’s what we get for playing nice with OSX conventions for where to
> > locate files :-(. Luckily this isn’t a problem for Python3 as Apple
> > doesn’t ship that (and I’d be surprised if they ever unless they start
> > shipping Python3 code as part of the OS).
> 
> Well, I did submit a radar, and although I'm not sure how far the NDA extends, I hope I can say that, unfortunately, I got a "behaves as intended" response…

That’s too bad. But I wouldn’t expect this to change anytime soon even if this would have been classified as a bug.

Ronald

Re: [Pyobjc-dev] [Pythonmac-SIG] PyObjC and macOS 10.12 (Sierra)

[Andrew J. <a.h...@gm...>]

On 12/10/2016 21:02, Ronald Oussoren wrote:
 >
 >> On 12 Oct 2016, at 21:09, Glyph Lefkowitz <gl...@tw...
 >> <mailto:gl...@tw...>> wrote:
 >>
 >>> On Oct 12, 2016, at 11:49 AM, Andrew Jaffe <a.h...@gm...
 >>> <mailto:a.h...@gm...>> wrote:
 >>>
 >>> Not me. If I understand correctly, Glyph -- who undoubtedly
 >>> understand the situation better than I do -- still thinks that
 >>> there's no actual bug here, since we shouldn't be using the framework
 >>> build this way, but I'm not sure I understand/agree...
 >>
 >> To be honest, I'm not 100% sure this is a good idea either; it's just
 >> that I know Donald Stufft has had a terrible time with Apple system
 >> python for several years, and he regards this as a positive change.
 >>  From my personal perspective, there's a good case to be made that a
 >> python in /System should just load from /System and one in /Library
 >> should load only from /Library, similar to the way --prefix works on
 >> "regular" UNIX.  But, this is what we've got :).
 >
 > I don’t mind if the /System python looks in /Library for stuff that the
 > user installed there, but I do consider it a bug that Apple installs
 > system files in /Library because that affects all installations of
 > Python 2.7.
 >
 > That’s what we get for playing nice with OSX conventions for where to
 > locate files :-(. Luckily this isn’t a problem for Python3 as Apple
 > doesn’t ship that (and I’d be surprised if they ever unless they start
 > shipping Python3 code as part of the OS).

Well, I did submit a radar, and although I'm not sure how far the NDA 
extends, I hope I can say that, unfortunately, I got a "behaves as 
intended" response...

Andrew

Re: [Pyobjc-dev] [Pythonmac-SIG] PyObjC and macOS 10.12 (Sierra)

[Ronald O. <ron...@ma...>]

> On 13 Oct 2016, at 12:47, Ronald Oussoren <ron...@ma...> wrote:
> 
> 
>> On 12 Oct 2016, at 23:06, Nicholas Riley <nj...@il... <mailto:nj...@il...>> wrote:
>> 
>> Have you tried DevToolsSecurity?  Sounds like it might be a
>> security/codesigning issue.
>> 
>> https://developer.apple.com/legacy/library/documentation/Darwin/Reference/ManPages/man8/DevToolsSecurity.8.html <https://developer.apple.com/legacy/library/documentation/Darwin/Reference/ManPages/man8/DevToolsSecurity.8.html>
>> 
> 
> I hadn’t, but changing this doesn’t help.  Debugging on the console isn’t too bad, although I am stuck at the moment while debugging one of the crashes I get running the testsuite on 10.12.
> 
> The two programs below both call DCSGetTermRangeInString, both work on 10.11, but the python version crashes on 10.12. Sadly enough it crashes in a secondary thread that’s doing some work for the DictionaryServices framework. I’m not yet sure why the python script causes a crash of the interpreter.
> 
> # Python
> from DictionaryServices import DCSGetTermRangeInString
> 
> text = "the hello world program"
> rng = DCSGetTermRangeInString(None, text, 5)
> print(rng)
> #end
> 
> // Objective-C
> #import <CoreServices/CoreServices.h>
> #import <Foundation/Foundation.h>
> 
> int main(void)
> {
>    CFRange rng;
> 
>    rng = DCSGetTermRangeInString(NULL, CFSTR("the hello world program"), 5);
>    NSLog(@"%ld %ld", rng.location, rng.length);
>    return 0;
> }
> // end


I can now reproduce the problem using a small extension that doesn’t even touch PyObjC. See <https://bitbucket.org/ronaldoussoren/pyobjc/issues/174/crash-using-dictionaryservices-on-1012>. There is some material difference between /usr/bin/python and a python.org binary installation of Python 3.5.  This demonstrates the same crash using small C extension that exports a function that calls DCSGetTermInRange.

The crash is in a background thread that appears to do work for DCSGetTermInRange.

Next steps:
* Try to reproduce using python.org installer for 2.7
* Try to reproduce using a local build of 3.5.2

Now where did I leave my Yak shaving kit?

Ronald

[Pyobjc-dev] PyObjC on macOS 10.12

[Ronald O. <ron...@ma...>]

Hi,

A release of PyObjC with proper support for 10.12 is getting closer. I’ve finished the metadata updates using the beta SDK that was current when I started the work, and I’m currently debugging two crashing bugs: The tests for both pyobjc-framework-DictionaryServices and pyobjc-framework-SpriteKit cause hard crashes on OSX 10.12.

When those issues are resolved I’ll start testing on older OSX releases, followed by a release of PyObjC 3.2 (still without wheels).  With some luck the 3.2 release will be sometime next week.

After that I’d like to work on:
* Update framework bindings to the latest OSX SDK (using the current Xcode 8.1 beta)
* Somehow automate running tests on various OSX releases
* Building wheels (and automate doing so using the previous item)
* Add asyncio support to PyObjC (probably using new subproject), by implementing an asyncio eventloop using CFRunLoop
* Experiment with adding utility functions and methods that make it easier to integrate asyncio(-style) code in a PyObjC GUI, for example by adding helper methods that enable using coroutines instead of completion handler callbacks.  That will definitely be a new subproject initially, adding this is technically feasible but I don’t know yet if this will actually be useful.

As always I have no idea when I’ll actually work on this, my time to work on this is fairly limited. 

Ronald

Re: [Pyobjc-dev] [Pythonmac-SIG] PyObjC and macOS 10.12 (Sierra)

[Ronald O. <ron...@ma...>]

> On 12 Oct 2016, at 23:06, Nicholas Riley <nj...@il...> wrote:
> 
> Have you tried DevToolsSecurity?  Sounds like it might be a
> security/codesigning issue.
> 
> https://developer.apple.com/legacy/library/documentation/Darwin/Reference/ManPages/man8/DevToolsSecurity.8.html
> 

I hadn’t, but changing this doesn’t help.  Debugging on the console isn’t too bad, although I am stuck at the moment while debugging one of the crashes I get running the testsuite on 10.12.

The two programs below both call DCSGetTermRangeInString, both work on 10.11, but the python version crashes on 10.12. Sadly enough it crashes in a secondary thread that’s doing some work for the DictionaryServices framework. I’m not yet sure why the python script causes a crash of the interpreter.

# Python
from DictionaryServices import DCSGetTermRangeInString

text = "the hello world program"
rng = DCSGetTermRangeInString(None, text, 5)
print(rng)
#end

// Objective-C
#import <CoreServices/CoreServices.h>
#import <Foundation/Foundation.h>

int main(void)
{
   CFRange rng;

   rng = DCSGetTermRangeInString(NULL, CFSTR("the hello world program"), 5);
   NSLog(@"%ld %ld", rng.location, rng.length);
   return 0;
}
// end

Ronald

Re: [Pyobjc-dev] [Pythonmac-SIG] PyObjC and macOS 10.12 (Sierra)

[Ronald O. <ron...@ma...>]

> On 12 Oct 2016, at 08:08, Glyph Lefkowitz <gl...@tw...> wrote:
> 
> 
>> On Oct 11, 2016, at 2:26 PM, Ronald Oussoren <ron...@ma... <mailto:ron...@ma...>> wrote:
>> 
>> BTW. Has anyone experience with using LLDB on Sierra? I’m currently running Sierra in a VM and for some reason LLDB doesn’t appear to work, in an SSH session I cannot start programs at all and on the console a crashing bug in the interpreter exists the interpreter instead of breaking into the debugger. That’s rather annoying when you’re trying to a debug a crash on 10.12 that doesn’t happen on 10.11.
> 
> Ever since the introduction of SIP, system-level debugging tools like LLDB have worked very poorly for me, even on self-built executables.  I don't see a difference in its behavior on 10.11 vs. 10.12 though.  I can run it on `curl` from Homebrew, but not `python`; my guess being that Python is trying to dlopen() some SIP-protected thing whereas `curl` is loading only things from Homebrew?
> 
> If it's a VM, then system integrity is less of a concern; have you tried just blanket disabling SIP to see if that improves the situation?  I've had some luck with other tools (dtrace, mostly) working in that configuration.  (I had to totally disable SIP though, disabling one flag at a time didn't work.)

I haven’t had problems with SIP on 10.11. This also doesn’t seem to be a SIP issue, I got the same behavior after disabling SIP.

I did manage to debug on the VM’s console by explicitly running the Python interpreter inside the framework (Python.app/Contents/MacOS/Python) instead of the launcher in {sys.prefix}/bin.  This doesn’t work in an SSH session though. That’s a bit annoying, but is workable until I get around to installing sierra on my main machine.   

Ronald

P.S. I’m debugging using a Python.org <http://python.org/> install of python3.5, not using the /System installation of Python2.7. Using the latter is asking for problems when working on a package that Apple also ships. 

> 
> -glyph
> 
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most 
> engaging tech sites, SlashDot.org! http://sdm.link/slashdot_______________________________________________
> Pyobjc-dev mailing list
> Pyo...@li...
> https://lists.sourceforge.net/lists/listinfo/pyobjc-dev

Re: [Pyobjc-dev] [Pythonmac-SIG] PyObjC and macOS 10.12 (Sierra)

[Ronald O. <ron...@ma...>]

> On 12 Oct 2016, at 21:09, Glyph Lefkowitz <gl...@tw...> wrote:
> 
> 
>> On Oct 12, 2016, at 11:49 AM, Andrew Jaffe <a.h...@gm... <mailto:a.h...@gm...>> wrote:
>> 
>> Not me. If I understand correctly, Glyph -- who undoubtedly understand the situation better than I do -- still thinks that there's no actual bug here, since we shouldn't be using the framework build this way, but I'm not sure I understand/agree...
> 
> To be honest, I'm not 100% sure this is a good idea either; it's just that I know Donald Stufft has had a terrible time with Apple system python for several years, and he regards this as a positive change.  From my personal perspective, there's a good case to be made that a python in /System should just load from /System and one in /Library should load only from /Library, similar to the way --prefix works on "regular" UNIX.  But, this is what we've got :).

I don’t mind if the /System python looks in /Library for stuff that the user installed there, but I do consider it a bug that Apple installs system files in /Library because that affects all installations of Python 2.7.

That’s what we get for playing nice with OSX conventions for where to locate files :-(. Luckily this isn’t a problem for Python3 as Apple doesn’t ship that (and I’d be surprised if they ever unless they start shipping Python3 code as part of the OS).

Ronald

> 
> -glyph
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most 
> engaging tech sites, SlashDot.org! http://sdm.link/slashdot_______________________________________________
> Pyobjc-dev mailing list
> Pyo...@li...
> https://lists.sourceforge.net/lists/listinfo/pyobjc-dev

Re: [Pyobjc-dev] [Pythonmac-SIG] PyObjC and macOS 10.12 (Sierra)

[Glyph L. <gl...@tw...>]

> On Oct 12, 2016, at 11:49 AM, Andrew Jaffe <a.h...@gm...> wrote:
> 
> Not me. If I understand correctly, Glyph -- who undoubtedly understand the situation better than I do -- still thinks that there's no actual bug here, since we shouldn't be using the framework build this way, but I'm not sure I understand/agree...

To be honest, I'm not 100% sure this is a good idea either; it's just that I know Donald Stufft has had a terrible time with Apple system python for several years, and he regards this as a positive change.  From my personal perspective, there's a good case to be made that a python in /System should just load from /System and one in /Library should load only from /Library, similar to the way --prefix works on "regular" UNIX.  But, this is what we've got :).

-glyph

Re: [Pyobjc-dev] [Pythonmac-SIG] PyObjC and macOS 10.12 (Sierra)

[Andrew J. <a.h...@gm...>]

On 11/10/2016 22:26, Ronald Oussoren wrote:
>
>> On 17 Sep 2016, at 19:59, Glyph Lefkowitz <gl...@tw...
>> <mailto:gl...@tw...>> wrote:
>>
>>>
>>> On Sep 17, 2016, at 9:27 AM, Ned Deily <na...@py...
>>> <mailto:na...@py...>> wrote:
>>>
>>> On 2016-09-13 19:33, Glyph Lefkowitz wrote:
>>>>> On Sep 13, 2016, at 3:35 PM, Andrew Jaffe <a.h...@gm...
>>>>> <mailto:a.h...@gm...>
>>>>> <mailto:a.h...@gm...>> wrote:
>>>>>
>>>>> Aha!
>>>>>
>>>>> $ ls -lt /Library/Python/2.7/site-packages/
>>>>> total 0
>>>>> -rwxr-xr-x  1 root  wheel  157 31 Jul 02:36 Extras.pth*
>>>>> -rw-r--r--  1 root  wheel  119 31 Jul 02:36 README
>>>>> $ more /Library/Python/2.7/site-packages/Extras.pth
>>>>> /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python
>>>>> /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/PyObjC
>>>>>
>>>>> Now I wonder how those got there?!
>>>>>
>>>>
>>>> Hah!  Thanks for sharing.  Very satisfying to actually make a *correct*
>>>> prediction about setuptools' behavior :)
>>>
>>> This seems to be Apple's doing.  AFAICT, 10.12 is shipping with this
>>> Extras.pth file in /Library/Python/2.7; it's something new.  And,
>>> unfortunately, due to https://bugs.python.org/issue4865, the
>>> site-packages directory for the system Python 2.7 is included in
>>> sys.path along with the non-system framework Python site-packages.
>>
>> Hrm.  I guess everyone I knew on the beta was using homebrew python :(.
>>
>> I'm surprised that Apple is putting stuff in /Library.  I don't have a
>> Sierra box handy - /Library isn't SIP-protected now, is it?
>>
>> This seems wrong; someone should file a radar (and probably share
>> on http://www.openradar.me <http://www.openradar.me/> for further
>> discussion).
>
> Did someone file a radar for this? Otherwise I can do so.

Not me. If I understand correctly, Glyph -- who undoubtedly understand 
the situation better than I do -- still thinks that there's no actual 
bug here, since we shouldn't be using the framework build this way, but 
I'm not sure I understand/agree...

> I can affirm that Extras.pth is now in /Library/Python/2.7 instead of in
> a system location. This sucks as bits of the system install now infect
> other installs of Python 2.7. One more reason to stop using 2.7 I guess ;-)

Andrew

Re: [Pyobjc-dev] [Pythonmac-SIG] PyObjC and macOS 10.12 (Sierra)

[Glyph L. <gl...@tw...>]

> On Oct 11, 2016, at 2:26 PM, Ronald Oussoren <ron...@ma...> wrote:
> 
> BTW. Has anyone experience with using LLDB on Sierra? I’m currently running Sierra in a VM and for some reason LLDB doesn’t appear to work, in an SSH session I cannot start programs at all and on the console a crashing bug in the interpreter exists the interpreter instead of breaking into the debugger. That’s rather annoying when you’re trying to a debug a crash on 10.12 that doesn’t happen on 10.11.

Ever since the introduction of SIP, system-level debugging tools like LLDB have worked very poorly for me, even on self-built executables.  I don't see a difference in its behavior on 10.11 vs. 10.12 though.  I can run it on `curl` from Homebrew, but not `python`; my guess being that Python is trying to dlopen() some SIP-protected thing whereas `curl` is loading only things from Homebrew?

If it's a VM, then system integrity is less of a concern; have you tried just blanket disabling SIP to see if that improves the situation?  I've had some luck with other tools (dtrace, mostly) working in that configuration.  (I had to totally disable SIP though, disabling one flag at a time didn't work.)

-glyph