Menu
▾
▴
bitpim-devel
|
From: Tom P. <mlp...@ea...> - 2004-03-15 14:14:38
Attachments:
BitPim
|
On Mar 15, 2004, at 12:00 AM, Roger Binns wrote: > On Mar 15, Tom Pollard wrote: >> Personally, I think that the startup script for BitPim should be a >> shell script, rather than a Python script. > > You should discuss this on bitpim-devel. On the other platforms, and > I had believed Mac as well, the binary distribution of BitPim includes > the Python interpretter and all modules needed (both ones written in > Python and the necessary shared libraries). I'm not sure how things are done on the other platforms, but I would have expected you to use the 'freeze' system for packaging a script as a standalone executable, similarly to 'perl2exe' and it's ilk. These systems package up the scripts, modules and dlls needed to run a script in a zip archive with a binary loader prepended. The loader is responsible for unpacking things, setting up the environment and executing your script using the bundled interpreter. This lets you run your script even if perl or python isn't installed at all on the target machine. The way BitPim's packaged for the Mac is simpler; MacOS X makes it easy to do something along the lines of perl2exe without actually using a single zip archive with a binary loader. But, you still need something like a loader to kick things off. For BitPim, that's a small Python script. This seems a little inelegant; since, this first script is the guy who sets things up so that subsequent python scripts can use the bundled python interpreter and modules, and so it requires the user to have a separate, working Python installation. From the recent problem report on the bitpim-user list, we saw that Python is an optional install on MacOS X. The BitPim startup script doesn't do anything that you can't do just as easily with a shell script, and a shell script would remove the need for a working Python on the user's machine. Anyway, this is just a suggestion. Most Mac BitPim users do seem to have Python installed. In any case, I've attached a shell script that could be used to replace the current BitPim python startup script. It uses a small shell function to extract the parent directory name from a filepath because the 'dirname' shell utility (like Python) is only there in MacOS X if the user installs the optional BSD package. Otherwise, it's just about line-for-line equivalent with the python version. Cheers, Tom |
|
From: Steven P. <n9...@n9...> - 2004-03-15 15:59:26
|
On Mar 15, 2004, at 8:14 AM, Tom Pollard wrote: > I'm not sure how things are done on the other platforms, but I would > have expected you to use the 'freeze' system for packaging a script > as a standalone executable, similarly to 'perl2exe' and it's ilk. Except, as far as I know, that does not work properly on MacOS X. Prove me wrong, I won't mind. ;^) > the user to have a separate, working Python installation. From the > recent problem report on the bitpim-user list, we saw that Python is > an optional install on MacOS X. The BitPim startup script doesn't do > anything that you can't do just as easily with a shell script, and a > shell script would remove the need for a working Python on the user's > machine. An optional install, yes, but one that is marked by default on every installation. So, unless a user intentionally goes out of their way when doing the install or upgrade to make it NOT install, it will be there. > I've attached a shell script that could be used to replace the current > BitPim python startup script. It uses a small shell function to > extract the parent directory name from a filepath because the > 'dirname' shell utility (like Python) is only there in MacOS X if the > user installs the optional BSD package. Otherwise, it's just about > line-for-line equivalent with the python version. Honestly, I'm not all that concerned about watching out for people who intentionally go into their installation process and remove things without knowing what they are doing. Also, it was not my design to build the bundle this way, I am using the standard "bundlebuilder.py" routine that is part of the Python distribtution and is, as far as I can tell, the accepted and "approved" way to build distributable bundles on MacOS X. To wedge in a different script would require gutting the work done by bundlebuilder.py and making it a manual process each time. Just my $0.02. If you think that your approach should be integrated into the bundlebuilder.py module as an alternative launch mechanism, perhaps it would be worthwhile joining the MacPython list (if you aren't already on it) and throwing it out there for discussion. It might be a useful option. |
|
From: Tom P. <mlp...@ea...> - 2004-03-15 16:41:40
|
On Mar 15, 2004, at 10:59 AM, Steven Palm wrote: >> I'm not sure how things are done on the other platforms, but I would >> have expected you to use the 'freeze' system for packaging a script >> as a standalone executable, similarly to 'perl2exe' and it's ilk. > > Except, as far as I know, that does not work properly on MacOS X. > Prove me wrong, I won't mind. ;^ I have no idea what the state of 'freeze' on Macs is, to be honest - I'm a python newbie. (I do use ActiveState's 'PerlApp' under Windows and the open-source 'PAR' package under Unix (including MacOS X) for packaging Perl scripts as executables.) >> the user to have a separate, working Python installation. From the >> recent problem report on the bitpim-user list, we saw that Python is >> an optional install on MacOS X. The BitPim startup script doesn't do >> anything that you can't do just as easily with a shell script, and a >> shell script would remove the need for a working Python on the user's >> machine. > > An optional install, yes, but one that is marked by default on every > installation. So, unless a user intentionally goes out of their way > when doing the install or upgrade to make it NOT install, it will be > there. If it's optional, it's optional. Running ordinary apps under MacOS X doesn't require its presence. My understanding is that it's mainly there to support Terminal users and Unix geeks. If it was my company's product, I'd insist on taking this simple step to make sure my app ran on a minimal install of the OS. > Also, it was not my design to build the bundle this way, I am using > the standard "bundlebuilder.py" routine that is part of the Python > distribtution and is, as far as I can tell, the accepted and > "approved" way to build distributable bundles on MacOS X. To wedge in > a different script would require gutting the work done by > bundlebuilder.py and making it a manual process each time. I didn't realize that. I'd also be disinclined to muck with bundlebuilder.py unless there was some stronger reason than this. Maybe if the issue comes up again I'll lobby the bundlebuilder maintainer to accept this change. Just on technical grounds, it seems clearly the right way to go. Anyway, thanks for being willing to discuss it. Cheers, Tom |
|
From: Michael C. <ma...@ca...> - 2004-03-15 17:33:11
|
On 3/15/04, Tom Pollard posited: > >On Mar 15, 2004, at 10:59 AM, Steven Palm wrote: >> >>An optional install, yes, but one that is marked by default on every >>installation. So, unless a user intentionally goes out of their way >>when doing the install or upgrade to make it NOT install, it will be >>there. > >If it's optional, it's optional. Running ordinary apps under MacOS X >doesn't require its presence. My understanding is that it's mainly >there to support Terminal users and Unix geeks. If it was my >company's product, I'd insist on taking this simple step to make sure >my app ran on a minimal install of the OS. Ah, if only more 'software professionals' were this professional about their work! Back when I had a software company, I could have used as many of you as I could get! >I didn't realize that. I'd also be disinclined to muck with >bundlebuilder.py unless there was some stronger reason than this. >Maybe if the issue comes up again I'll lobby the bundlebuilder >maintainer to accept this change. Just on technical grounds, it seems Dang, where were you when I was hiring? Probably still in grade school ;-) --=20 Mike Casteel ma...@ca... Seattle, WA |
|
From: Roger B. <ro...@ro...> - 2004-03-15 17:46:42
|
> I have no idea what the state of 'freeze' on Macs is, to be honest - > I'm a python newbie. (I do use ActiveState's 'PerlApp' under Windows > and the open-source 'PAR' package under Unix (including MacOS X) for > packaging Perl scripts as executables.) For BitPim we do use different tools on each platform, as they all have different strengths and weaknesses. It is definitely a goal that there should be no need for anything to be already installed by the user. That goal is met 100% on Windows and Linux. On Windows Python itself is a DLL. py2exe produces a zip of all your .py code, and then has a stub executable that loads the DLL, and sets the zip as the code path. (See python-dev group for some proposals for Python automatically finding the zip, removing the need for even the stub executable). Since zip files as an import source were only introduced in Python 2.3, that approach doesn't work with earlier versions. On Linux we use Python 2.2, and the cx_Freeze tool. That packages up the Python binary with the code appended to the file and stub code to set things up right (ie similar to zip but using custom code and archive format) I believe that all the mechanisms will move over to the zip method but that will set the minimum supported Python version as 2.3. We also make sure we use "standard" stuff, and don't modify any other components. For example I could install Python 2.3 on Linux and base everything off that, but that would be a non-standard installation. Similarly we could benefit from moving up to the developer version of wxPython+wxWidgets. As Steven can tell you, I don't do that either :-) If in your opinion something can be done better, then please contribute that to the necessary place (in this case whoever makes bundlebuilder). Also be very aware that there may be good reasons why they did things the way they did. Also be aware that they may be concerned about backwards compatibility (eg working with both Jaguar and Panther). But this is what open source is about - you get the best from everyone rather than one what secretive arbitrary team of people make. Roger |
|
From: Steven P. <n9...@n9...> - 2004-03-15 18:38:10
|
On Mar 15, 2004, at 10:41 AM, Tom Pollard wrote: >>> the user to have a separate, working Python installation. From the >>> recent problem report on the bitpim-user list, we saw that Python is >>> an optional install on MacOS X. The BitPim startup script doesn't >>> do anything that you can't do just as easily with a shell script, >>> and a shell script would remove the need for a working Python on the >>> user's machine. >> >> An optional install, yes, but one that is marked by default on every >> installation. So, unless a user intentionally goes out of their way >> when doing the install or upgrade to make it NOT install, it will be >> there. > > If it's optional, it's optional. Running ordinary apps under MacOS X > doesn't require its presence. My understanding is that it's mainly > there to support Terminal users and Unix geeks. If it was my > company's product, I'd insist on taking this simple step to make sure > my app ran on a minimal install of the OS. Well, look at it this way... If you intentionally choose to not install that piece, then you choose to break things that might depend on it. So yes, you have that choice, but there are consequences for it. I don't know what else you would also potentially break, but it's easy enough to simply put in the requirements of your software program that you require the BSD subsystem be installed. I'd liken the 'optional' but here to also say that printer drivers are optional, and you don't have to install them... Now, is it up to the application developer to figure out that you didn't and somehow work around that? Maybe a far stretch, but similar in the vein that the user took some intentional action to remove a part of the standard install. Now, if this were a company product where real money was being charged for it and there were a compelling reason to make it as idiot proof as possible, despite what the users may have done to their system, then I'd agree with you and would be compelled to try to make it work. However, that is not the case, and there are design decisions to use the base or common denominator of a standard set of components on top of a standard install of the operating system with it's default options. Just my opinion, anyway. I'm not intimately familiar with Python, this is the first time I've worked with it. However, after discussing things on the PythonMac mailing list, it came out that bundlebuilder.py was the way to generate a bundle. It doesn't mean it's the best, or the most fool-proof, etc... So the options are possibly open for an alternative, nobody can be aware of every alternative way to do something, no more than they can know every conceivable end-user configuration they'll be running against. If they could, I suppose we'd have no bugs or errors in shipping code. :-) > Anyway, thanks for being willing to discuss it. Always open here, I don't claim to have any lock on the best way to do things (if there is one, opinions abound...) and can always learn something new and useful. |
|
From: Steven P. <n9...@n9...> - 2004-03-15 19:00:05
|
On Mar 15, 2004, at 10:41 AM, Tom Pollard wrote:
> I didn't realize that. I'd also be disinclined to muck with
> bundlebuilder.py unless there was some stronger reason than this.
> Maybe if the issue comes up again I'll lobby the bundlebuilder
> maintainer to accept this change. Just on technical grounds, it seems
> clearly the right way to go. Anyway, thanks for being willing to
> discuss it.
Just an aside.... I was looking through bundlebuilder.py just now to
see if this could be remedied easily enough, and it appears that it
might be do-able... But you have to love this bit of code
(particularly the comment):
if self.standalone or self.semi_standalone:
# XXX we're screwed when the end user has deleted
# /usr/bin/python
hashbang = "/usr/bin/python"
... yes we are. :-)
|
|
From: Roger B. <ro...@ro...> - 2004-03-15 20:01:37
|
> if self.standalone or self.semi_standalone: > # XXX we're screwed when the end user has deleted > # /usr/bin/python > hashbang = "/usr/bin/python" Gives new meaning to "standalone". Maybe they need a "reallystandaloneireallymeanitandnooptionaldependenciesthistimeplease" :-) (I have been known to name variables/functions like that sometimes when experiencing a lot of frustration with other components). Roger |
|
From: Tom P. <mlp...@ea...> - 2004-03-15 20:12:03
|
On Mar 15, 2004, at 3:02 PM, Roger Binns wrote: >> if self.standalone or self.semi_standalone: >> # XXX we're screwed when the end user has deleted >> # /usr/bin/python >> hashbang = "/usr/bin/python" > > Gives new meaning to "standalone". Maybe they need a > "reallystandaloneireallymeanitandnooptionaldependenciesthistimeplease" > :-) > (I have been known to name variables/functions like that sometimes when > experiencing a lot of frustration with other components). Yes, I noticed. MakeTheDamnThingRedraw() put a smile on my face when I ran across it. It made me remember how glad I am not to be doing X11/Motif programming anymore... Tom |
|
From: Steven P. <n9...@n9...> - 2004-03-15 19:16:47
|
On Mar 15, 2004, at 8:14 AM, Tom Pollard wrote: > In any case, I've attached a shell script that could be used to > replace the current BitPim python startup script. It uses a small > shell function to extract the parent directory name from a filepath > because the 'dirname' shell utility (like Python) is only there in > MacOS X if the user installs the optional BSD package. Otherwise, it's > just about line-for-line equivalent with the python version. I'm not interested in removing the BSD subsystem from my computer to test, but I see you state that `dirname` is not available unless you install the BSD subsystem. You are using `sed`, however, so that one *is* installed? Given this kind of thinking, it's probably somewhat arbitrary what commands Apple puts on your system as a CORE package versus the BSD package, because I would land `sed` as a part of the BSD subsystem as well, but it apparently is not. This makes it hard to be foolproof for future OS revisions as well if you write a shell script that depends on certain commands being there (albeit minimal in this case. |
|
From: Tom P. <mlp...@ea...> - 2004-03-15 19:40:40
|
On Mar 15, 2004, at 2:16 PM, Steven Palm wrote: >> In any case, I've attached a shell script that could be used to >> replace the current BitPim python startup script. It uses a small >> shell function to extract the parent directory name from a filepath >> because the 'dirname' shell utility (like Python) is only there in >> MacOS X if the user installs the optional BSD package. Otherwise, >> it's just about line-for-line equivalent with the python version. > > I'm not interested in removing the BSD subsystem from my computer to > test, but I see you state that `dirname` is not available unless you > install the BSD subsystem. You are using `sed`, however, so that one > *is* installed? Yes, sed and awk are always installed on a minimal unix system. > Given this kind of thinking, it's probably somewhat arbitrary what > commands Apple puts on your system as a CORE package versus the BSD > package, because I would land `sed` as a part of the BSD subsystem as > well, but it apparently is not. It's understandable that this looks arbitrary to you, but it's not, really. I was surprised that there was a 'dirname' utility at all, as soon as I saw it, because I know it's not standard. That prompted me to check the Receipts directory to find which package it was installed with, and I found it was part of the BSD install. (I have a perl script that lets you search your Receipts to see who installed what, if that sounds useful. The Linux rpm system lets you ask questions like that easily, but I couldn't find anything for MacOS X that was comparable. My perl script isn't really suitable for release, IMO, because it's kind of slow - it has to reread all of the receipts each time you run the script and that's an annoying delay. It ought to cache that stuff somewhere, since it only rarely changes.) > This makes it hard to be foolproof for future OS revisions as well if > you write a shell script that depends on certain commands being there > (albeit minimal in this case. Not really. I've been dealing with cross-platform Unix issues for most of my career, and know how to write a portable Bourne shell script that you can expect to run pretty much anywhere. (Not a terribly vital job skill nowadays, unfortunately.) Sed and awk are always safe. The main challenge is recognizing and avoiding shell features that are bash- or ksh-specific. You might be surprised how much you can still do within these limitations. Cheers, Tom |
|
From: Roger B. <ro...@ro...> - 2004-03-15 19:59:50
|
> Not really. I've been dealing with cross-platform Unix issues for most > of my career, and know how to write a portable Bourne shell script that > you can expect to run pretty much anywhere. Me too! I used to work on machines that had multiple universes (as they called it). You could have a sysv environment or a bsd environment with commands scattered between them, and lots of other fun issues. > Sed and awk are always safe. They may always exist, but they don't necessarily always run with non-trivial data. Things like arbitrary limits on line length (such as 512 or 1024 chars), breaking on any files not having UNIX end of lines, limits on the number of variables etc. > The main > challenge is recognizing and avoiding shell features that are bash- or > ksh-specific. You might be surprised how much you can still do within > these limitations. Heck, I still don't use shell functions except when I occassionally remember they exist :-) I ended up writing a macro system in TCL that had a library of shell code, and then expanded it for the main shell scripts. It turned out to be very useful as there is so much stuff that doesn't work between machines, even modern ones (for example try killing one of your own processes named foo, or finding out how much free disk space there is for a proposed directory name). > (Not a terribly vital job skill nowadays, unfortunately.) It isn't the skill that matters, but more the way of thinking that does. I am glad that a lot of these lower level issues are non-issues any more, while the thinking is still very relevant. For example here is another great programmer complaining about the very same issue: http://www.joelonsoftware.com/articles/PleaseLinker.html Roger |
×
Want the latest updates on software, tech news, and AI?
Get latest updates about software, tech news, and AI from SourceForge directly in your inbox once a month.
Thanks for helping keep SourceForge clean.
X