From: techtonik <tec...@us...> - 2009-07-22 21:16:00
|
Hello, How can it happen that application compiled with py2exe can not see PYTHONPATH set in system environment? --anatoly t. |
From: Daniel P. <da...@pr...> - 2009-07-23 20:36:28
|
techtonik wrote: > On Thu, Jul 23, 2009 at 2:35 PM, Daniel Pryden<da...@pr...> wrote: >> Is there a reason why you can't just run "python setup.py py2exe" again? >> That will rebuild library.zip for you with whatever updated modules >> you need. > > Because Mercurial for windows is distributed in compiled py2exe form > and I do not have compiler to compile myself. I need "convert" > extension bundled with Mercurial that is operational, but to convert > Bazaar repositories it uses "bzrlib" python module that of course > isn't included. > Ah. It sounds like a much easier solution to your problem is this: 1. Download and install Python from python.org 2. Download and install a "pure python" Mercurial from http://mercurial.selenic.com/wiki/WindowsInstall (matching your Python version) 3. Download and install a Python installer for Bzr from https://launchpad.net/bzr/+download (also matching your Python version) 4. Use the tools in Python script form instead of compiled py2exe form. >> If you're looking for a "plugin" type system, where you have certain >> modules distributed with your program (in the library.zip) and others >> loaded from the filesystem at runtime, then you'll probably need to >> implement that yourself. As Daryl Tester pointed out, you're free to >> manipulate sys.path in the frozen program to import modules from >> locations besides library.zip. You may also want to read up on the >> __import__() function for more details. > > What about making this (module overrides) a feature of py2exe? For > example, compiled module may look into the directory with library.zip > for "dropins/" subdirectory (like in Eclipse) to be added into > PYTHONPATH. In this case any py2exe program that uses dropins=true > will allow other people manually add optional modules not included in > distribution or it may even update itself from internet/svn/mercurial. > Doesn't seem likely to me. Py2exe is used by lots of different projects with lots of very different packaging requirements, and a feature like you describe is highly unlikely to be useful to all of them. However, it's not my opinion that matters -- it's Thomas Heller and Jimmy Retzlaff who you need to convince. A much better solution would be to bug the Mercurial developers to distribute Mercurial for Windows in a way that allows for this kind of extensibility. I have no idea how the Mercurial code is organized, so I have no idea what level of effort would be involved, but supporting a "dropins/" directory should really be Mercurial's job, not py2exe's. Hopefully this helps! - Daniel. |
From: Thomas H. <th...@ct...> - 2009-07-23 20:50:16
|
Very useful advise! Daniel Pryden schrieb: > techtonik wrote: >> On Thu, Jul 23, 2009 at 2:35 PM, Daniel Pryden<da...@pr...> wrote: >>> Is there a reason why you can't just run "python setup.py py2exe" again? >>> That will rebuild library.zip for you with whatever updated modules >>> you need. >> >> Because Mercurial for windows is distributed in compiled py2exe form >> and I do not have compiler to compile myself. I need "convert" >> extension bundled with Mercurial that is operational, but to convert >> Bazaar repositories it uses "bzrlib" python module that of course >> isn't included. >> > > Ah. It sounds like a much easier solution to your problem is this: > 1. Download and install Python from python.org > 2. Download and install a "pure python" Mercurial from > http://mercurial.selenic.com/wiki/WindowsInstall (matching your Python > version) > 3. Download and install a Python installer for Bzr from > https://launchpad.net/bzr/+download (also matching your Python version) > 4. Use the tools in Python script form instead of compiled py2exe form. > >>> If you're looking for a "plugin" type system, where you have certain >>> modules distributed with your program (in the library.zip) and others >>> loaded from the filesystem at runtime, then you'll probably need to >>> implement that yourself. As Daryl Tester pointed out, you're free to >>> manipulate sys.path in the frozen program to import modules from >>> locations besides library.zip. You may also want to read up on the >>> __import__() function for more details. >> >> What about making this (module overrides) a feature of py2exe? For >> example, compiled module may look into the directory with library.zip >> for "dropins/" subdirectory (like in Eclipse) to be added into >> PYTHONPATH. In this case any py2exe program that uses dropins=true >> will allow other people manually add optional modules not included in >> distribution or it may even update itself from internet/svn/mercurial. >> > > Doesn't seem likely to me. Py2exe is used by lots of different projects > with lots of very different packaging requirements, and a feature like > you describe is highly unlikely to be useful to all of them. However, > it's not my opinion that matters -- it's Thomas Heller and Jimmy > Retzlaff who you need to convince. > > A much better solution would be to bug the Mercurial developers to > distribute Mercurial for Windows in a way that allows for this kind of > extensibility. I have no idea how the Mercurial code is organized, so I > have no idea what level of effort would be involved, but supporting a > "dropins/" directory should really be Mercurial's job, not py2exe's. > > Hopefully this helps! > > - Daniel. > > > ------------------------------------------------------------------------------ -- Thanks, Thomas |
From: techtonik <tec...@us...> - 2009-07-23 21:12:50
|
On Thu, Jul 23, 2009 at 4:36 PM, Daniel Pryden<da...@pr...> wrote: > > Ah. It sounds like a much easier solution to your problem is this: > 1. Download and install Python from python.org > 2. Download and install a "pure python" Mercurial from > http://mercurial.selenic.com/wiki/WindowsInstall (matching your Python > version) > 3. Download and install a Python installer for Bzr from > https://launchpad.net/bzr/+download (also matching your Python version) > 4. Use the tools in Python script form instead of compiled py2exe form. Have already done that thanks to invaluable input from #mercurial. Great they've chosen to mirror C code with Pure version. Funny thing is that I've updated WindowsInstall with information about how to build Pure version while waiting for replies in this thread. =) Thanks for investigation anyway. >>> If you're looking for a "plugin" type system, where you have certain >>> modules distributed with your program (in the library.zip) and others >>> loaded from the filesystem at runtime, then you'll probably need to >>> implement that yourself. As Daryl Tester pointed out, you're free to >>> manipulate sys.path in the frozen program to import modules from >>> locations besides library.zip. You may also want to read up on the >>> __import__() function for more details. >> >> What about making this (module overrides) a feature of py2exe? For >> example, compiled module may look into the directory with library.zip >> for "dropins/" subdirectory (like in Eclipse) to be added into >> PYTHONPATH. In this case any py2exe program that uses dropins=true >> will allow other people manually add optional modules not included in >> distribution or it may even update itself from internet/svn/mercurial. >> > > Doesn't seem likely to me. Py2exe is used by lots of different projects > with lots of very different packaging requirements, and a feature like > you describe is highly unlikely to be useful to all of them. However, > it's not my opinion that matters -- it's Thomas Heller and Jimmy > Retzlaff who you need to convince. If this feature is to be implemented, it would be of course optional. But first it would be nice to document special handling of PYTHONPATH somewhere in py2exe docs, because not many realize that the problem exists at all. > A much better solution would be to bug the Mercurial developers to > distribute Mercurial for Windows in a way that allows for this kind of > extensibility. I have no idea how the Mercurial code is organized, so I > have no idea what level of effort would be involved, but supporting a > "dropins/" directory should really be Mercurial's job, not py2exe's. Will try to make a proposal, but this logic is only applicable to py2exe distribution, because in other cases there is easy_install, --prefix options and other rules for installing python modules. -- anatoly t. |
From: Daniel P. <da...@pr...> - 2009-07-23 22:51:59
|
techtonik wrote: > But first it would be nice to document special handling of PYTHONPATH > somewhere in py2exe docs, because not many realize that the problem > exists at all. I disagree that "the problem exists at all". The intent of using py2exe is to package an executable such that the users of the program can use it like they would any other binary executable. Users may be (and probably should be) oblivious to the fact that py2exe was used in the process. I think the Mercurial developers should document the fact that the py2exe distribution of Mercurial cannot be extended the way that the pure Python version can be. That's not py2exe's fault -- it's an inherent complication of trying to distribute the same program in multiple, mutually-incompatible forms. (And it's not like py2exe is alone in this regard -- I don't know of ANY freeze program that recommends mixing frozen modules and external filesystem modules, unless you really, really, know what you're doing.) I think the point is this: if all you have is an .exe file (or even an .exe file together with some .zip and .pyd files), think of it as an .exe file, not as a Python program. If you try to act like it's still a .py file you're going to be sadly disappointed. However, I would assume that most users would expect this behavior, not be surprised by it. >> A much better solution would be to bug the Mercurial developers to >> distribute Mercurial for Windows in a way that allows for this kind of >> extensibility. I have no idea how the Mercurial code is organized, so I >> have no idea what level of effort would be involved, but supporting a >> "dropins/" directory should really be Mercurial's job, not py2exe's. > > Will try to make a proposal, but this logic is only applicable to > py2exe distribution, because in other cases there is easy_install, > --prefix options and other rules for installing python modules. > Right. So, in other words, what you're saying is that the Mercurial binary distribution (which, as an implementation detail, they happened to make with py2exe) doesn't have the same features as are provided by the distutils (--prefix option) and setuptools (easy_install) packages that are available when you're installing normal Python modules. This still doesn't sound like py2exe's problem. (If I may be permitted to be slightly more snide: it sounds like it's YOUR problem. I don't know of anyone else running into this same issue, and there is an easy workaround, which you've already discovered yourself. So if you're the only one with the problem, and you have the tools to fix it, then what was the question again?) I'll tell you what: if you're not 100% satisfied with py2exe, I will personally refund the $0.00 that you paid for it. In fact, I'm sure the Mercurial developers will give you the same refund for the Mercurial binary distribution as well... :-) - Daniel. |
From: Daryl T. <dt-...@ha...> - 2009-07-23 00:18:34
|
techtonik wrote: > How can it happen that application compiled with py2exe can not see > PYTHONPATH set in system environment? Because then the executable would behave differently if Python were installed on the computer or not, and self-contained executables aren't supposed to do that? I believe it's by design. If you require a "work around", then perhaps within the py2exified you can program check if PYTHONPATH is set, and modify sys.path accordingly. -- Regards, Daryl Tester "Crush! Kill! Deploy!" -- Lost In Space I.T. Dept. |
From: techtonik <tec...@us...> - 2009-07-23 05:21:01
|
On Wed, Jul 22, 2009 at 8:02 PM, Daryl Tester<dt-...@ha...> wrote: > techtonik wrote: > >> How can it happen that application compiled with py2exe can not see >> PYTHONPATH set in system environment? > > Because then the executable would behave differently if Python were installed > on the computer or not, and self-contained executables aren't supposed to do > that? Ok. But how can I supply additional optional module to the py2exe bundled program? How can I upgrade a version of module inside the library? I tried to repack library.zip, but failed. > I believe it's by design. If you require a "work around", then perhaps within > the py2exified you can program check if PYTHONPATH is set, and modify sys.path > accordingly. Is that means patching original code and repacking? Where to read about this py2exified? Regards, -- anatoly t. |
From: Daniel P. <da...@pr...> - 2009-07-23 18:35:09
|
techtonik wrote: > Ok. But how can I supply additional optional module to the py2exe > bundled program? How can I upgrade a version of module inside the > library? I tried to repack library.zip, but failed. Is there a reason why you can't just run "python setup.py py2exe" again? That will rebuild library.zip for you with whatever updated modules you need. If you're looking for a "plugin" type system, where you have certain modules distributed with your program (in the library.zip) and others loaded from the filesystem at runtime, then you'll probably need to implement that yourself. As Daryl Tester pointed out, you're free to manipulate sys.path in the frozen program to import modules from locations besides library.zip. You may also want to read up on the __import__() function for more details. > Is that means patching original code and repacking? Yes. If you want to change the behavior of the program, you need to change the source code. No other way about it. The way you ask this question indicates that you're dealing with a program that was already developed by someone else, and you are simply trying to adapt it into a frozen executable with py2exe. Unfortunately, this is not always a simple task. In many cases, especially when the program is written to provide special functionality like plugins or optional modules, the program needs to be modified (sometimes substantially) to work correctly in a frozen environment. > Where to read > about this py2exified? I believe Daryl was trying to say py2exe-ified -- that is, a Python program that py2exe has modIFIED into a frozen executable. The point is that if you want to do any special sys.path modifications (using PYTHONPATH or any other mechanism) then you will need add that code into the program that you use as input into py2exe. Py2exe won't do it for you automatically. Hopefully this helps! - Daniel. |
From: techtonik <tec...@us...> - 2009-07-23 20:17:28
|
On Thu, Jul 23, 2009 at 2:35 PM, Daniel Pryden<da...@pr...> wrote: > >> Ok. But how can I supply additional optional module to the py2exe >> bundled program? How can I upgrade a version of module inside the >> library? I tried to repack library.zip, but failed. > > Is there a reason why you can't just run "python setup.py py2exe" again? > That will rebuild library.zip for you with whatever updated modules > you need. Because Mercurial for windows is distributed in compiled py2exe form and I do not have compiler to compile myself. I need "convert" extension bundled with Mercurial that is operational, but to convert Bazaar repositories it uses "bzrlib" python module that of course isn't included. > If you're looking for a "plugin" type system, where you have certain > modules distributed with your program (in the library.zip) and others > loaded from the filesystem at runtime, then you'll probably need to > implement that yourself. As Daryl Tester pointed out, you're free to > manipulate sys.path in the frozen program to import modules from > locations besides library.zip. You may also want to read up on the > __import__() function for more details. What about making this (module overrides) a feature of py2exe? For example, compiled module may look into the directory with library.zip for "dropins/" subdirectory (like in Eclipse) to be added into PYTHONPATH. In this case any py2exe program that uses dropins=true will allow other people manually add optional modules not included in distribution or it may even update itself from internet/svn/mercurial. -- anatoly t. |
From: Thomas H. <th...@ct...> - 2009-07-23 20:46:30
|
Daryl Tester schrieb: > techtonik wrote: > >> How can it happen that application compiled with py2exe can not see >> PYTHONPATH set in system environment? > > Because then the executable would behave differently if Python were installed > on the computer or not, and self-contained executables aren't supposed to do > that? > > I believe it's by design. If you require a "work around", then perhaps within > the py2exified you can program check if PYTHONPATH is set, and modify sys.path > accordingly. > Yes, it is by design. py2exe's programs are very careful to ignore all the usual environment variables that Python normally uses. This is to make sure that the the exe is not influenced in any way by an installed Python interpreter. You can, of course, invent your own mechanism by extending sys.path inside your program. -- Thanks, Thomas |