From: Werner F. B. <wer...@fr...> - 2008-02-21 17:06:20
|
I am in the midst of converting my application over to sqlalchemy from another ORM package and wanted to get going in packaging it with py2exe for further testing. However I do run into problems. I get that model "pkg_resources" is not found when I try to run the executable. I am using py2exe 0.6.6 is it supposed to support eggs? If not has anyone figured out a way to deal with eggs and py2exe. Any hints would be much appreciated. Werner |
From: Thomas H. <th...@ct...> - 2008-02-21 17:45:48
|
Werner F. Bruhin schrieb: > I am in the midst of converting my application over to sqlalchemy from > another ORM package and wanted to get going in packaging it with py2exe > for further testing. > > However I do run into problems. I get that model "pkg_resources" is not > found when I try to run the executable. > > I am using py2exe 0.6.6 is it supposed to support eggs? No, eggs are not yet supported. > If not has > anyone figured out a way to deal with eggs and py2exe. There was a thread on this mailing list in feb. 2007, titled 'supporting egg files in py2exe', which may help. And if someone (hint, hint) wants to implement egg-support that would be great. Thomas |
From: Werner F. B. <wer...@fr...> - 2008-02-21 18:42:05
|
Thomas, Thomas Heller wrote: > Werner F. Bruhin schrieb: >> I am in the midst of converting my application over to sqlalchemy from >> another ORM package and wanted to get going in packaging it with py2exe >> for further testing. >> >> However I do run into problems. I get that model "pkg_resources" is not >> found when I try to run the executable. >> >> I am using py2exe 0.6.6 is it supposed to support eggs? > > No, eggs are not yet supported. > >> If not has >> anyone figured out a way to deal with eggs and py2exe. > > There was a thread on this mailing list in feb. 2007, titled > 'supporting egg files in py2exe', which may help. No this didn't help. But I found this: http://py2exe.org/index.cgi/ExeWithEggs Which nearly gets me there. Only problem I have (I think) is to copy the package into the zip file. Is there any method available to copy a whole folder with sub-folders to a zip file or does one have to walk the folders and copy file by file? > > And if someone (hint, hint) wants to implement egg-support that would be great. Would love to help on this but this is way over my head. Werner |
From: Ralf S. <ra...@br...> - 2008-02-25 12:49:25
|
Thomas Heller wrote: > Werner F. Bruhin schrieb: >> I am in the midst of converting my application over to sqlalchemy from >> another ORM package and wanted to get going in packaging it with py2exe >> for further testing. >> >> However I do run into problems. I get that model "pkg_resources" is not >> found when I try to run the executable. bbfreeze does support egg files. Take a look at http://pypi.python.org/pypi/bbfreeze/ (Note that it doesn't support some stuff that py2exe does) >> >> I am using py2exe 0.6.6 is it supposed to support eggs? > > No, eggs are not yet supported. > >> If not has >> anyone figured out a way to deal with eggs and py2exe. > > There was a thread on this mailing list in feb. 2007, titled > 'supporting egg files in py2exe', which may help. > > And if someone (hint, hint) wants to implement egg-support that would be great. > Whoever does it, feel free to take a look at bbfreeze's implementation. All it basically does, is check if module is used from some egg file and then copy the whole egg. It was rather easy to implement. |
From: King Simon-N. <sim...@mo...> - 2008-02-22 12:58:06
|
> -----Original Message----- > From: py2...@li... > [mailto:py2...@li...] On Behalf > Of Werner F. Bruhin > Sent: 21 February 2008 17:06 > To: py2...@li... > Subject: [Py2exe-users] sqlalchemy 0.4.x and py2exe > > I am in the midst of converting my application over to > sqlalchemy from > another ORM package and wanted to get going in packaging it > with py2exe > for further testing. > > However I do run into problems. I get that model > "pkg_resources" is not > found when I try to run the executable. > > I am using py2exe 0.6.6 is it supposed to support eggs? If not has > anyone figured out a way to deal with eggs and py2exe. > > Any hints would be much appreciated. > > Werner > Hi Werner, I'm currently using py2exe with eggs, but it is definitely a hack at the moment. I started with the instructions at http://www.py2exe.org/index.cgi/ExeWithEggs, but my final implementation is a little different. Also, I haven't tried it with SQLAlchemy, so there may be other issues that I'm unaware of. I'm currently using FormEncode, Genshi, matplotlib, numpy and wxPython, all with python 2.5. Firstly, I don't use the zipfile or bundle options (ie skip_archive=True) - it may be possible to get them to work, but I haven't tried it. After py2exe has run, in my dist directory alongside the executables and various .pyc files I have a <name>.egg directory for each egg that I use. There is also a file called 'eggs.pth' that names each of the eggs. In my setup.py, I have a function, get_eggs, that returns a list of pathnames to each of the eggs that I require (I keep them in a dependencies directory). It would probably be possible to use setuptools to do this based on install_requires entries in setup.py, but I haven't tried that. I subclass py2exe to do these things: A) unpack the eggs into the library directory B) prevent files that are in the eggs getting copied to the dist directory directly Here are the relevant parts of my py2exe subclass (I've removed the stuff that isn't relevant - I hope it still works): ------------------------------------------------------------------------ --- import os from os.path import join as pjoin, abspath, dirname, basename, isdir from py2exe.build_exe import py2exe import pkg_resources from setuptools.command import easy_install from setuptools import setup as setuptools_setup def get_eggs(): # Return a list of paths to egg files that you require # ... return [egg1, egg2, egg3, ...] class ExeBuilder(py2exe): def find_needed_modules(self, *args, **kwargs): # Hook find_needed_modules to install all the eggs that we # know about. To install them, run: # # easy_install -Zmaxd <lib_directory> <eggs...> # eggs = get_eggs() pathfile = open(pjoin(self.lib_dir, 'eggs.pth'), 'w') for egg in eggs: eggbase = basename(egg) print "Installing %s" % eggbase easy_install.main(['--quiet', '-Zmaxd', self.lib_dir, egg]) print "Getting top level packages for %s" % basename(egg) new_path = pjoin(self.lib_dir, eggbase) egg_info_dir = pjoin(new_path, 'EGG-INFO') metadata = pkg_resources.PathMetadata(new_path, egg_info_dir) pathfile.write(basename(egg) + '\n') if metadata.has_metadata('top_level.txt'): toplevels = metadata.get_metadata_lines('top_level.txt') self.toplevels.update(toplevels) if metadata.has_metadata('native_libs.txt'): libs = list(metadata.get_metadata_lines('native_libs.txt')) self.native_libs.update(basename(lib) for lib in libs) # Get rid of all .py files from the egg - they just waste space print "Removing .py files from %s" % basename(egg) for basedir, dirs, files in os.walk(new_path): py_files = [f for f in files if f.endswith('.py')] for py_file in py_files: os.unlink(pjoin(basedir, py_file)) pathfile.close() # Call the base class to find the rest of the modules py2exe.find_needed_modules(self, *args, **kwargs) def parse_mf_results(self, mf): # Remove any files that were found by modulefinder but are # included in the eggs that we've installed. py_files, extensions, builtins = py2exe.parse_mf_results(self, mf) toplevels = self.toplevels def module_in_eggs(module): package = module.__name__.split('.')[0] return package in toplevels py_files = [m for m in py_files if not module_in_eggs(m)] # Can't filter out extensions here - it breaks the find_dlls # method. Instead we filter them in copy_dlls and # copy_extensions return py_files, extensions, builtins def _remove_egg_items(self, items): filtered = [] for item in items: if isinstance(item, basestring): filename = item else: filename = item.__file__ if basename(filename) not in self.native_libs: filtered.append(item) else: self.skipped.add(filename) return filtered def copy_extensions(self, extensions): py2exe.copy_extensions(self, self._remove_egg_items(extensions)) def copy_dlls(self, dlls): py2exe.copy_dlls(self, self._remove_egg_items(dlls)) def run(self): self.toplevels = set() self.native_libs = set() self.skipped = set() py2exe.run(self) setuptools_setup( cmdclass = {'py2exe': ExeBuilder}, options = ..., windows = ... ) ------------------------------------------------------------------------ --- I did warn you that it was a hack! There is one more change that is necessary - the eggs.pth file only gets processed at runtime if python considers the dist directory to be a 'site' directory. So I have this snippet at the very beginning of my main program, before I import anything else: ------------------------------------------------------------------------ --- # When built using py2exe, we need to ensure that the directory # containing our eggs is defined as a site directory (so that the # eggs.pth file works) import sys if hasattr(sys, 'frozen'): from os.path import dirname import site; site.addsitedir(dirname(sys.executable)) ------------------------------------------------------------------------ --- In my 'includes', I specifically mention 'setuptools', 'pkg_resources' and 'site' - I can't remember if that was absolutely necessary. I hope this helps. If it works for you, I should probably update the wiki page. If we could do away with the get_eggs function and use setuptools metadata directly, that would be even better. Cheers, Simon |
From: Werner F. B. <wer...@fr...> - 2008-02-24 09:43:12
|
Simon, King Simon-NFHD78 wrote: ... > Hi Werner, > > I'm currently using py2exe with eggs, but it is definitely a hack at the > moment. I started with the instructions at > http://www.py2exe.org/index.cgi/ExeWithEggs, but my final implementation > is a little different. Also, I haven't tried it with SQLAlchemy, so > there may be other issues that I'm unaware of. I'm currently using > FormEncode, Genshi, matplotlib, numpy and wxPython, all with python 2.5. > Any chance you could send me your complete setup.py? I don't use FormEncode nor Genshi but the rest is the same plus some other stuff for XML and Firebird SQL. I am not sure how to integrate your solution with what I use, so seeing the whole file might help me. Werner |
From: King Simon-N. <sim...@mo...> - 2008-02-25 10:37:42
Attachments:
setup.py
|
> -----Original Message----- > From: Werner F. Bruhin [mailto:wer...@fr...] > Sent: 24 February 2008 09:43 > To: King Simon-NFHD78 > Cc: py2...@li... > Subject: Re: sqlalchemy 0.4.x and py2exe > > Simon, > > King Simon-NFHD78 wrote: > ... > > > Hi Werner, > > > > I'm currently using py2exe with eggs, but it is definitely > a hack at the > > moment. I started with the instructions at > > http://www.py2exe.org/index.cgi/ExeWithEggs, but my final > implementation > > is a little different. Also, I haven't tried it with SQLAlchemy, so > > there may be other issues that I'm unaware of. I'm currently using > > FormEncode, Genshi, matplotlib, numpy and wxPython, all > with python 2.5. > > > Any chance you could send me your complete setup.py? > > I don't use FormEncode nor Genshi but the rest is the same plus some > other stuff for XML and Firebird SQL. > > I am not sure how to integrate your solution with what I use, > so seeing > the whole file might help me. > > Werner > The attached script works for me. You would need to change at least the get_eggs and get_data_files functions for something that makes sense for your project. My get_eggs function assumes that I have directories called dependencies/runtime/crossplatform and dependencies/runtime/windows containing the egg files. Hope that helps, Simon |
From: Werner F. B. <wer...@fr...> - 2008-02-26 18:18:53
Attachments:
setup.py
|
Simon, In am finally starting to work on this based on your script. I get the following exception: Traceback (most recent call last): File "setup.py", line 426, in <module> run_setup() File "setup.py", line 421, in run_setup ("prog\\Share\\Schemata", amaraschemata), File "C:\Python25\lib\distutils\core.py", line 151, in setup dist.run_commands() File "C:\Python25\lib\distutils\dist.py", line 974, in run_commands self.run_command(cmd) File "C:\Python25\lib\distutils\dist.py", line 994, in run_command cmd_obj.run() File "setup.py", line 264, in run py2exe.run(self) File "C:\Python25\lib\site-packages\py2exe\build_exe.py", line 223, in run self._run() File "C:\Python25\lib\site-packages\py2exe\build_exe.py", line 274, in _run self.find_needed_modules(mf, required_files, required_modules) File "setup.py", line 220, in find_needed_modules py2exe.find_needed_modules(self, *args, **kwargs) File "C:\Python25\lib\site-packages\py2exe\build_exe.py", line 1189, in find_needed_modules mf.import_hook(mod) File "C:\Python25\lib\site-packages\py2exe\mf.py", line 649, in import_hook return Base.import_hook(self,name,caller,fromlist) File "C:\Python25\lib\site-packages\py2exe\mf.py", line 130, in import_hook q, tail = self.find_head_package(parent, name) File "C:\Python25\lib\site-packages\py2exe\mf.py", line 184, in find_head_package raise ImportError, "No module named " + qname ImportError: No module named setuptools Which confused me, as I can do: import setuptools >>> import pkg_resources >>> dir() ['__builtins__', 'glob', 'head', 'item', 'mygif', 'os', 'pkg_resources', 'setuptools', 'tail'] If I comment out the "includes" close then this goes away but when I run the exe I get an error that pkg_resources is not there (obviously correct). I also noted that I get both the sqlalchemy folder (with a subset of the egg) and the sqlalchemy egg into the dist folder. I basically used your setup.py, removed some of the excludes and added some includes I need. Do you have an idea what causes the exception? In case it might help I attach my setup.py. Werner |
From: Werner F. B. <wer...@fr...> - 2008-02-26 21:05:59
|
A further update on my problem with the egg stuff. When leaving the includes section I get an exception. *** searching for required modules *** Installing sqlalchemy Getting top level packages for sqlalchemy Removing .py files from sqlalchemy Installing setuptools Getting top level packages for setuptools Removing .py files from setuptools Above is the py2exe stdout output and here is the exception: Traceback (most recent call last): File "setup.py", line 427, in <module> run_setup() File "setup.py", line 422, in run_setup ("Share\\Schemata", amaraschemata), File "C:\Python25\lib\distutils\core.py", line 151, in setup dist.run_commands() File "C:\Python25\lib\distutils\dist.py", line 974, in run_commands self.run_command(cmd) File "C:\Python25\lib\distutils\dist.py", line 994, in run_command cmd_obj.run() File "setup.py", line 264, in run py2exe.run(self) File "C:\Python25\lib\site-packages\py2exe\build_exe.py", line 223, in run self._run() File "C:\Python25\lib\site-packages\py2exe\build_exe.py", line 274, in _run self.find_needed_modules(mf, required_files, required_modules) File "setup.py", line 219, in find_needed_modules py2exe.find_needed_modules(self, *args, **kwargs) File "C:\Python25\lib\site-packages\py2exe\build_exe.py", line 1189, in find_needed_modules mf.import_hook(mod) File "C:\Python25\lib\site-packages\py2exe\mf.py", line 649, in import_hook return Base.import_hook(self,name,caller,fromlist) File "C:\Python25\lib\site-packages\py2exe\mf.py", line 130, in import_hook q, tail = self.find_head_package(parent, name) File "C:\Python25\lib\site-packages\py2exe\mf.py", line 184, in find_head_package raise ImportError, "No module named " + qname ImportError: No module named setuptools If I comment the "includes" section I get a successful build but when I run the application it can not find the module pkg_resources. In get_eggs I have specified: eggs = ['sqlalchemy', 'setuptools'] I think it has to do with sqlalchemy, it tries to import pkg_resources when it failes to find the database backend. Checking my dist folder I see that I have a folder sqlalchemy and sqlalchemy-0.4.0-py2.5.egg. Now the first one does not contain the db backend I use. If I add sqlalchemy to the packages list for py2exe then the sqlalchemy folder contains all the db backends and the application works. This is obviously just a work around, but it at least pin points the problem. Werner |
From: Werner F. B. <wer...@fr...> - 2008-02-27 10:40:57
Attachments:
setup.py
|
Hi all, "solved", at least for my use for the moment. The problem with pkg_resources and setuptools not found was easy to correct. Just needed to add: pkg_resources.require("setuptools") to the setup.py, this is necessary if an egg is installed in multiversion. I then also specified setuptools and sqlalchemy in the packages option, but did not specify it in get_eggs. If I did specify then I would get the egg and a package folder into my dist folder. I also have to use skip_archive as otherwise amara can't find its data files. I attached my current version as it might be helpful for others. Werner |