Thread: [Pyobjc-dev] ANN: py2app 0.5
Brought to you by:
ronaldoussoren
From: Ronald O. <ron...@ma...> - 2010-07-27 19:13:45
Attachments:
smime.p7s
|
Hi, I've just uploaded py2app 0.5 and new versions of altgraph, modulegraph and macholib. I hope this solves most issues with py2app. "easy_install-X.Y -U py2app" should install the software for you (where X.Y is your favorite version of Python) There is one new feature in this release: experimental support for python 3. This basicly means that I managed to build a single application as a standalone application bundle, without much testing. Alias builds and plugin bundles almost certainly don't work (the first because alias builds use the Carbon module which isn't available in python 3, the latter because I had to rewrite the C code in the application bundles and probably have to do the same for plugin bundles). Ronald |
From: Virgil D. <hs...@ha...> - 2010-07-30 10:06:59
|
On Tue, Jul 27, 2010 at 9:13 PM, Ronald Oussoren <ron...@ma...> wrote: > Hi, > > I've just uploaded py2app 0.5 and new versions of altgraph, modulegraph and macholib. I hope this solves most issues with py2app. "easy_install-X.Y -U py2app" should install the software for you (where X.Y is your favorite version of Python) > > There is one new feature in this release: experimental support for python 3. This basicly means that I managed to build a single application as a standalone application bundle, without much testing. Alias builds and plugin bundles almost certainly don't work (the first because alias builds use the Carbon module which isn't available in python 3, the latter because I had to rewrite the C code in the application bundles and probably have to do the same for plugin bundles). > > Ronald > ------------------------------------------------------------------------------ > The Palm PDK Hot Apps Program offers developers who use the > Plug-In Development Kit to bring their C/C++ apps to Palm for a share > of $1 Million in cash or HP Products. Visit us here for more details: > http://ad.doubleclick.net/clk;226879339;13503038;l? > http://clk.atdmt.com/CRS/go/247765532/direct/01/ > _______________________________________________ > Pyobjc-dev mailing list > Pyo...@li... > https://lists.sourceforge.net/lists/listinfo/pyobjc-dev > > Thanks a bunch Ronald, your work is very very much appreciated. I tried to build my app on a whole spanking new stack (Python 2.7, pyobjc 2.3 and py2app 0.5.2) and it worked (I use the plugin feature). I had, however, to erase the prebuilt files and re-build them. This brings a question: I noticed that you added several new prebuilt for each architecture combination. I ask: why using prebuilt at all? Why not simply have the setup build the thing on the user machine upon install? This way the user gets binaries that exactly correspond to the cflags he used when building python. Are there cases where a py2app user would want any other cflags? I only recently discovered the pythonmac-sig and I saw your call for help. It's sad that nobody answered yet, but I can't say I'm surprised. Like I told you at EuroPython, I'd like to help but I have a very hard time understanding py2app's convoluted code base (distutils is probably to blame for that, god I hate distutils/setuptools). I guess it's the same thing for other users. I also had an interesting discussion with Alex Morega. He was asking me why I used py2app at all. He gave me a link (http://github.com/alex-morega/tinymail and http://github.com/alex-morega/WhizbangPlugin ) to his Python/Cocoa apps which don't use any packaging. It turns out the reason he can get away without py2app or bundlebuilder is because he uses the system Python and doesn't have other dependencies, but it still raises questions: Is it possible that py2app is a little too complex for what it does? I was thinking maybe it would be a good idea to go toward code simplification in py2app and that a good way to start would be to get rid of distutils. In any case, if nobody's on it, I'd gladly take care of the documentation modernization. Virgil Dupras |
From: Greg E. <gre...@ca...> - 2010-07-31 03:09:00
|
Virgil Dupras wrote: > Is it possible that py2app is a little too complex for what > it does? I think a lot of the complexity of py2app and py2exe come from trying to automatically figure out what modules and libraries need to be included. I've been thinking for a while about creating something simpler that doesn't attempt any automatic module discovery at all. You would be required to construct a project file that explicitly lists all the required modules and libraries, including standard library modules. This may sound onerous, but I don't think it would be if you did it as part of the process of developing the code. There would be a testing mode which would run the application in an isolated environment using the current project settings. If you did all your development that way, you would immediately discover when something was missing and needed to be added to the project. Furthermore, you would be assured that if it works during testing, it will also work when bundled -- something that currently requires a separate testing step when using py2app or py2exe, because you can never trust them to find all the dependencies correctly. Does anyone else think such a tool would be useful? -- Greg |
From: Christopher B. <Chr...@no...> - 2010-08-01 18:29:27
|
Greg Ewing wrote: > I've been thinking for a while about creating something > simpler that doesn't attempt any automatic module discovery > at all. You would be required to construct a project file > that explicitly lists all the required modules and libraries, > including standard library modules. I've thought for a while that there is way too much re-invention of the wheel with the various stand-alone builders. I'd love to see more flexibility, and ideally code sharing, by breaking down the process into the various parts: 1) the API for specifying what you need built -- py2app and py2exe at least share much )but not all) of this, though they are slightly incompatible. AFAIK, the rest are all different 2) Figuring out what needs to be included. py2app and bbfreeze both use modulegraph, though bb-freeze apparently forked it. 3) Actually building the bundle -- by definition this will be different on different platforms, and there are multiple ways of doing it on one platform Anyway, ideally, each of these steps could share a common API, and so each bundle builder could mix and match the parts as they saw fit. Getting everyone to cooperate is a big social, rather then technical problem, but py2app at least could be designed to allow each of these pieces to be replaceable. (maybe it already is -- I haven't poked into the code enough to know) So, aside from allowing more code sharing, this approach would make it easier to plug in different pieces, like Greg's proposed manual specification of modules. As for that proposal -- I agree with other posters that that really would be onerous. However, a hybrid would be great -- run some sort of automated tool that outputs an easy-to-read-and-edit text file that could be edited, and have the bundle builder use that. Then you culd e3dit it, write it frm scratch, whatever you like. If nothing else, it would really help speed to be able to re-build an app without having to go through the process of determining what to include each time. I often find myself modifying the source a bit, knowing for sure that I haven't changed the dependencies, and then having to wait for py2app (or py2exe) to do the full analysis again. Another idea I have for determining what to include is to take the opposite tack of source-code analysis -- run-time analysis: You would run your app (or better yet, a test suite), and the tool would do a run-time examination of what's imported (would that be as simple as looking in sys.modules?). That way you would capture the dynamic imports that the source-code analysis misses. You would miss the conditional imports, but if you have a good test suite, you'd catch those too. (and if you don't, you'd have identified a hole in your tests). And you would miss the conditional imports that you really don't want ( I never want Tk just because I'm using PIL, for instance) If you really wanted a I-don't-care-how-big-it-is-I-just-want-it-to-work version, you could use a superset of runt-time and source-code analysis. I do want to be clear that py2app has been the best bundle builder I've used, and I really appreciate Ronald's effort to bring it up to speed again. -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chr...@no... |
From: Matthias B. <mat...@gm...> - 2010-08-02 21:55:47
|
Christopher Barker wrote: > Greg Ewing wrote: >> I've been thinking for a while about creating something >> simpler that doesn't attempt any automatic module discovery >> at all. You would be required to construct a project file >> that explicitly lists all the required modules and libraries, >> including standard library modules. > > I've thought for a while that there is way too much re-invention of the > wheel with the various stand-alone builders. I'd love to see more > flexibility, and ideally code sharing, by breaking down the process into > the various parts: > > [...] > So, aside from allowing more code sharing, this approach would make it > easier to plug in different pieces, like Greg's proposed manual > specification of modules. > > As for that proposal -- I agree with other posters that that really > would be onerous. However, a hybrid would be great -- run some sort of > automated tool that outputs an easy-to-read-and-edit text file that > could be edited, and have the bundle builder use that. Then you culd > e3dit it, write it frm scratch, whatever you like. Is anyone here familiar with Py++ [1]? What Christopher describes reminded me very much of that tool. Py++ is not a tool to produce standalone packages like py2app does, instead it's a tool to generate Python bindings for C/C++ libraries, but it seems their inner workings share similarities. Just like py2app has to find out what modules get imported by an application, Py++ needs to find out what functions/classes are in a library and which of them need to be turned into Python functions/classes. Py++ does that by running gccxml on the header files to extract the contents of the files which is then further processed. For simple libraries it might be enough to let Py++ just convert everything it finds in the headers, but for more complex libraries you want to customize that and leave out some stuff. Now the thing is, Py++ is not implemented as a standalone tool you just run on some input files, but instead, it's a framework for writing wrapper generators. This means, the user writes the main script (in Python) which invokes Py++ functionality and this means the actual control of the entire process is in the user's hands. In simple cases, it's enough to leave everything up to Py++ which means your main script will be very short. But if you want to, you can intercept any step in the processing pipeline and modify the data. For example, as mentioned above, the first step is to get the declarations from the header files which builds a tree of declarations that's stored in memory. Before the tree is passed on, your main script can inspect and modify it. You could even choose to skip that gccxml step entirely and produce the declaration tree by some other means. Subsequent Py++ steps just work on that tree, it doesn't matter anymore who created it. After that, the wrapper source code is generated which is again a tree of text blocks. Before this tree is written to disk to generate the source files, you can again intercept this and modify the tree. This design makes Py++ very flexible because you can inject your own logic into the pipeline to tailer the process to your needs. Now it sounds like exactly the same thing could be done in py2app (which I have never used so far, by the way, but from reading the mails here it sounds like it doesn't already provide this sort of flexibilty). Its first step is to find out what modules need to be included. If this produces a well-defined data structure with an API to modify/create it and the user gets the chance to modify that data before it's passed on (or even recreate it from scratch), then all use-cases that have been mentioned here could be handled by the same tool. Sorry if this all doesn't make any sense or py2app already has different means of achieving this flexibilty. As I said, I haven't used py2app yet (but it's on my todo list to try it out), it's just because Christopher's mail sounded so much like Py++ that I thought I'd chime in for a moment. Cheers, - Matthias - [1] http://www.language-binding.net/ |
From: Ronald O. <ron...@ma...> - 2010-07-31 08:26:28
Attachments:
smime.p7s
|
On 31 Jul, 2010, at 3:09, Greg Ewing wrote: > Virgil Dupras wrote: >> Is it possible that py2app is a little too complex for what >> it does? > > I think a lot of the complexity of py2app and py2exe come > from trying to automatically figure out what modules and > libraries need to be included. I object to calling py2app complex, the code base is fairly easy to understand. Have you even looked at the code base? > > I've been thinking for a while about creating something > simpler that doesn't attempt any automatic module discovery > at all. You would be required to construct a project file > that explicitly lists all the required modules and libraries, > including standard library modules. That is not simpler, it just shifts the complexity to the user. It is better to shift the complexity away from the user, that way the user does not have to reinvent the weel. Ronald |
From: Virgil D. <hs...@ha...> - 2010-07-31 09:26:48
|
On Sat, Jul 31, 2010 at 10:26 AM, Ronald Oussoren <ron...@ma...> wrote: > > On 31 Jul, 2010, at 3:09, Greg Ewing wrote: > >> Virgil Dupras wrote: >>> Is it possible that py2app is a little too complex for what >>> it does? >> >> I think a lot of the complexity of py2app and py2exe come >> from trying to automatically figure out what modules and >> libraries need to be included. > > I object to calling py2app complex, the code base is fairly easy to understand. Have you even looked at the code base? > >> >> I've been thinking for a while about creating something >> simpler that doesn't attempt any automatic module discovery >> at all. You would be required to construct a project file >> that explicitly lists all the required modules and libraries, >> including standard library modules. > > That is not simpler, it just shifts the complexity to the user. It is better to shift the complexity away from the user, that way the user does not have to reinvent the weel. > > Ronald > ------------------------------------------------------------------------------ > The Palm PDK Hot Apps Program offers developers who use the > Plug-In Development Kit to bring their C/C++ apps to Palm for a share > of $1 Million in cash or HP Products. Visit us here for more details: > http://p.sf.net/sfu/dev2dev-palm > _______________________________________________ > Pyobjc-dev mailing list > Pyo...@li... > https://lists.sourceforge.net/lists/listinfo/pyobjc-dev > > I'm not sure I agree that no dependency finding would be a good thing. Importing a stdlib modules often imports tons of other ones you don't know about and manually figuring these dependencies out can be really tedious. However, I think that using modulegraph instead of the built-in modulefinder is a mistake. For one thing, modulegraph doesn't support relative imports, yet. Yeah, we could work on it, but why bother when modulefinder already does it? It's just more maintenance. I know modulegraph is supposed to be better, but I would *gladly* list my non-stdlib dependencies manually rather than pray that py2app finds them correctly. The way I work with py2app now is that whenever a dependency isn't found by py2app, I put an explicit import of it in my "main py file" in a section documented for being py2app workaround imports. It works, but I'm sure there's a more elegant way... I think distutils is supposed to let you list dependencies manually in the setup() call, but when I tried it I found it extremely non-working, at all. So yes, a reliable way to manually list dependencies would be a great thing. Virgil Dupras |
From: Ronald O. <ron...@ma...> - 2010-08-01 12:00:04
Attachments:
smime.p7s
|
On 31 Jul, 2010, at 10:56, Virgil Dupras wrote: > On Sat, Jul 31, 2010 at 10:26 AM, Ronald Oussoren > <ron...@ma...> wrote: >> >> On 31 Jul, 2010, at 3:09, Greg Ewing wrote: >> >>> Virgil Dupras wrote: >>>> Is it possible that py2app is a little too complex for what >>>> it does? >>> >>> I think a lot of the complexity of py2app and py2exe come >>> from trying to automatically figure out what modules and >>> libraries need to be included. >> >> I object to calling py2app complex, the code base is fairly easy to understand. Have you even looked at the code base? >> >>> >>> I've been thinking for a while about creating something >>> simpler that doesn't attempt any automatic module discovery >>> at all. You would be required to construct a project file >>> that explicitly lists all the required modules and libraries, >>> including standard library modules. >> >> That is not simpler, it just shifts the complexity to the user. It is better to shift the complexity away from the user, that way the user does not have to reinvent the weel. >> >> Ronald >> ------------------------------------------------------------------------------ >> The Palm PDK Hot Apps Program offers developers who use the >> Plug-In Development Kit to bring their C/C++ apps to Palm for a share >> of $1 Million in cash or HP Products. Visit us here for more details: >> http://p.sf.net/sfu/dev2dev-palm >> _______________________________________________ >> Pyobjc-dev mailing list >> Pyo...@li... >> https://lists.sourceforge.net/lists/listinfo/pyobjc-dev >> >> > > I'm not sure I agree that no dependency finding would be a good thing. > Importing a stdlib modules often imports tons of other ones you don't > know about and manually figuring these dependencies out can be really > tedious. However, I think that using modulegraph instead of the > built-in modulefinder is a mistake. For one thing, modulegraph doesn't > support relative imports, yet. Adding is possible, and will be done. I'm working on that right now to be honest, and that code should be done by the end of the day (including unittests). I don't agree that using modulefinder would be better. For one, modulefinder will never support setuptools trickery (see the zope.interface discussion on pythonmac-sig), while that can and will be added to modulegraph. > Yeah, we could work on it, but why > bother when modulefinder already does it? It's just more maintenance. > I know modulegraph is supposed to be better, but I would *gladly* list > my non-stdlib dependencies manually rather than pray that py2app finds > them correctly. The way I work with py2app now is that whenever a > dependency isn't found by py2app, I put an explicit import of it in my > "main py file" in a section documented for being py2app workaround > imports. It works, but I'm sure there's a more elegant way... You can add an includes section to your setup.py file (see the includes argument on the command-line). What would be better is to research why py2app doesn't find modules you use, or at least try to create a small sample that demonstrates the problem I cannot fix issues if you don't report them. > > I think distutils is supposed to let you list dependencies manually in > the setup() call, but when I tried it I found it extremely > non-working, at all. So yes, a reliable way to manually list > dependencies would be a great thing. setuptools has install_requires/setup_requires, but py2app does not have support for that yet (although setup_requires should ensure that the eggs your require are installed before py2app gets run). You can also list includes/excludes in the py2app options dictionary: setup( options=dict(py2app=dict( include=['mymodule'], )) ) Ronald Ronald |
From: Ronald O. <ron...@ma...> - 2010-08-01 12:50:50
Attachments:
smime.p7s
|
On 1 Aug, 2010, at 13:59, Ronald Oussoren wrote: >> >> I'm not sure I agree that no dependency finding would be a good thing. >> Importing a stdlib modules often imports tons of other ones you don't >> know about and manually figuring these dependencies out can be really >> tedious. However, I think that using modulegraph instead of the >> built-in modulefinder is a mistake. For one thing, modulegraph doesn't >> support relative imports, yet. > > Adding is possible, and will be done. I'm working on that right now to be honest, and that code should be done by the end of the day (including unittests). The version of modulegraph in the repository now supports absolute and relative imports (with unittests) Ronald |
From: Tom M. <to...@de...> - 2010-07-28 14:55:40
|
Hey, I'd just like to say thanks - it's incredibly great to have some movement and leadership here (plus this'll make the installation process of py2app way less annoying - I got used to installing macholib, altgraph, modulegraph, and py2app separately in their dependency order) Tom On Tue, Jul 27, 2010 at 3:13 PM, Ronald Oussoren <ron...@ma...>wrote: > Hi, > > I've just uploaded py2app 0.5 and new versions of altgraph, modulegraph and > macholib. I hope this solves most issues with py2app. "easy_install-X.Y -U > py2app" should install the software for you (where X.Y is your favorite > version of Python) > > There is one new feature in this release: experimental support for python > 3. This basicly means that I managed to build a single application as a > standalone application bundle, without much testing. Alias builds and > plugin bundles almost certainly don't work (the first because alias builds > use the Carbon module which isn't available in python 3, the latter because > I had to rewrite the C code in the application bundles and probably have to > do the same for plugin bundles). > > Ronald > _______________________________________________ > Pythonmac-SIG maillist - Pyt...@py... > http://mail.python.org/mailman/listinfo/pythonmac-sig > unsubscribe: http://mail.python.org/mailman/options/Pythonmac-SIG > > |
From: Ronald O. <ron...@ma...> - 2010-08-03 07:53:19
Attachments:
smime.p7s
|
On 1 Aug, 2010, at 19:29, Christopher Barker wrote: > Greg Ewing wrote: >> I've been thinking for a while about creating something >> simpler that doesn't attempt any automatic module discovery >> at all. You would be required to construct a project file >> that explicitly lists all the required modules and libraries, >> including standard library modules. > > I've thought for a while that there is way too much re-invention of the > wheel with the various stand-alone builders. I'd love to see more > flexibility, and ideally code sharing, by breaking down the process into > the various parts: > 1) the API for specifying what you need built -- py2app and py2exe at > least share much )but not all) of this, though they are slightly > incompatible. AFAIK, the rest are all different AFAIK Bob intented py2app to be compatible with py2exe when specifying what to build. That won't always work though due to differences between the two platforms. It might be useful to think about a cleaner way to specify what to build, I don't particularly like the current way and especially not when building more complex applications. > > 2) Figuring out what needs to be included. py2app and bbfreeze both use > modulegraph, though bb-freeze apparently forked it. Correct. I don't know what py2exe uses, it might use the stdlib modulefinder. > > 3) Actually building the bundle -- by definition this will be different > on different platforms, and there are multiple ways of doing it on one > platform > > Anyway, ideally, each of these steps could share a common API, and so > each bundle builder could mix and match the parts as they saw fit. > > Getting everyone to cooperate is a big social, rather then technical > problem, but py2app at least could be designed to allow each of these > pieces to be replaceable. (maybe it already is -- I haven't poked into > the code enough to know) > > So, aside from allowing more code sharing, this approach would make it > easier to plug in different pieces, like Greg's proposed manual > specification of modules. I totally agree that is would be better to try to get the various stand-alone builders to at least share code for the common functionality. A common API for the various components would help with that, but it would IMHO be better to actually merge the various components. Ronald |
From: Ronald O. <ron...@ma...> - 2010-08-03 08:27:18
Attachments:
smime.p7s
|
On 2 Aug, 2010, at 23:55, Russell E. Owen wrote: > > > P.S. I'm curious about the virtualenv improvements. I use virtualenv to > develop my code, but when I've got the appropriate environment set up if > I build the application using py2app I always get the appended traceback > (even witih py2app 0.5.2). > > To work around it I have my setup.py script add a few necessary bits to > sys.path and I build from a fresh login with no virtual environment set > up. > > No big deal, but if there is a way to make the build work even when the > required packages are setup using virutalenv, I'd like to know about it. That's strange, I have build a couple of applications using virtualenvs. I don't use pip though and have patches to virtualenv that might change its behavior enough the explain the difference. Another set of patches to push upstream... Ronald |
From: Christopher B. <Chr...@no...> - 2010-08-03 16:24:17
|
Ronald Oussoren wrote: > AFAIK Bob intented py2app to be compatible with py2exe when > specifying what to build. That won't always work though due > to differences between the two platforms. There are differences that are because of platform. I think datafiles are handles differently at least. If you want to consider those bugs, I could try to dig into some of my setups and identify the issues I've found -- but that won't be for a couple weeks at least (vacation!) > It might be useful to think about a cleaner way to specify what to > build, I don't particularly like the current way and especially not when > building more complex applications. agreed -- though it's no horrible. > I totally agree that is would be better to try to get the various > stand-alone builders to at least share code for the common > functionality. A common API for the various components would help > with that, but it would IMHO be better to actually merge the various > components. Me too, but that requires cooperation, which may be hard to come by. I'd at least contact the ebb-freeze guy (Ralf?) -- he seemed a little confused about how to share code -- for instance, he said he couldn't use py2exe's code for including the icon because of licensing, but it looked like the license was compatible to me. With a little encouragement, maybe he'd be glad to share more. -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chr...@no... |
From: Ronald O. <ron...@ma...> - 2010-08-04 06:19:25
Attachments:
smime.p7s
|
On 3 Aug, 2010, at 18:24, Christopher Barker wrote: > Ronald Oussoren wrote: >> AFAIK Bob intented py2app to be compatible with py2exe when >> specifying what to build. That won't always work though due >> to differences between the two platforms. > > There are differences that are because of platform. I think datafiles > are handles differently at least. > > If you want to consider those bugs, I could try to dig into some of my > setups and identify the issues I've found -- but that won't be for a > couple weeks at least (vacation!) This has no hurry, but I at least want to know about differences and document them. And all differences that don't have a good reason should probably be fixed. > > >> It might be useful to think about a cleaner way to specify what to >> build, I don't particularly like the current way and especially not when >> building more complex applications. > > agreed -- though it's no horrible. > >> I totally agree that is would be better to try to get the various >> stand-alone builders to at least share code for the common >> functionality. A common API for the various components would help >> with that, but it would IMHO be better to actually merge the various >> components. > > Me too, but that requires cooperation, which may be hard to come by. I'd > at least contact the ebb-freeze guy (Ralf?) -- he seemed a little > confused about how to share code -- for instance, he said he couldn't > use py2exe's code for including the icon because of licensing, but it > looked like the license was compatible to me. With a little > encouragement, maybe he'd be glad to share more. My first goal is to go through the py2app stack to fix small issues and add proper testing, after that I'm open for cooperation. Ronald |