From: Johannes B. <buc...@gm...> - 2011-09-16 13:29:45
|
Hi Thomas, If you look at http://mavenjython.sourceforge.net/compile/usage.html you will see how in a demo project, the nose python library is included. Instead, you should put "Pygments" there, as this is the way pypi [1] has it in its catalogue. If you want a specific version, use Pygments==1.4, like easy_install demands it. You probably did that already -- if you run mvn package, what is the output? maven-jython-compile-plugin is pretty verbose anyways. If you use a HTTP proxy, you have to set the http(s)_proxy environment variables, as the proxy settings of maven are not forwarded to easy_install. I think putting all python stuff in Lib/ can be a good convention. If you don't like it for some reason, during your jython initialization (InitJython), add your folder to the python path. You can do this by overriding newInterpreter or createEnterpreter to call PySystemState.path.append() or you can, in run(), append your folder to the completed console. c.eval("sys.path.append('myfolder')") If you don't know/use the (Abstract)InitJython I'm talking about here, look at the demo project in http://mavenjython.sourceforge.net/ and its code [2]. Well, please get back to this list to let us know what the exact issue you are struggling with is, and in case you were able to solve it, how you did. All the best, Johannes [1] http://pypi.python.org/pypi/Pygments [2] https://sourceforge.net/p/mavenjython/code/ci/8a27a826b5e77825e02611263e34ac84e233f824/tree/jython-compile-maven-plugin-test/src/main/java/net/sf/mavenjython/ On 09/16/2011 12:39 PM, Thomas Mortagne wrote: > Hi, > > The project seems great but I have some issue which is probably more > related to Jython itself actually but will will probably be able to > help ;) > > I'm packaging Pygments as a library jar used by some other one, at > startup Jythong scan jars and find it. You can see it on > http://maven.xwiki.org/externals/pygments/pygments/1.4-xwiki/. > > I would love to use your maven plugin instead of downloading and > putting myself the Pygments files in my maven project but looks like > Jython is unable to find Pygments when built with it. The difference I > see is that your plugin put the python files in a Lib/ subfolder and > I'm packing then directly at the root of the jar in > http://maven.xwiki.org/externals/pygments/pygments/1.4-xwiki/. > > Is it me not properly using Jython ? Any idea what could I do to use > your plugin in my use case ? > > On Wed, Aug 17, 2011 at 01:09, Johannes Buchner <buc...@gm...> wrote: > Dear Jython Users! > > I have released version 1.0 of two Maven plugins that make the > integration of Jython into Java projects easier. > > The first one (jython-compile-maven-plugin) allows packaging of jython > including python libraries. > The second (python-test-maven-plugin) allows testing of python code. > They are hosted on http://mavenjython.sourceforge.net/ and I put some > information on them at > http://wiki.python.org/jython/JythonFaq/DistributingJythonScripts#Using_Maven > > > > I would welcome any feedback on what can be improved or what is needed. > > Cheers, > Johannes |
From: Johannes B. <buc...@gm...> - 2011-09-16 21:31:45
|
On 09/16/2011 04:43 PM, Thomas Mortagne wrote: > On Fri, Sep 16, 2011 at 15:29, Johannes Buchner <buc...@gm...> wrote: >> Hi Thomas, >> >> If you look at http://mavenjython.sourceforge.net/compile/usage.html >> you will see how in a demo project, the nose python library is included. >> Instead, you should put "Pygments" there, as this is the way pypi [1] has it in its catalogue. If you want a specific version, use Pygments==1.4, like easy_install demands it. >> >> You probably did that already -- if you run mvn package, what is the output? maven-jython-compile-plugin is pretty verbose anyways. >> >> If you use a HTTP proxy, you have to set the http(s)_proxy environment variables, as the proxy settings of maven are not forwarded to easy_install. >> >> I think putting all python stuff in Lib/ can be a good convention. If you don't like it for some reason, during your jython initialization (InitJython), add your folder to the python path. > > As I said I don't have any issue with the plugin and it properly > generate a jar file with pygments in it. I'm just talking about the > fact that it's putting pygments in a Lib/ subfolder in the jar. > > It's not that I don't like it, it's that it does not work by default > with Jython as far as I can see in my pretty simple code: Jython is > able to find pygments when it's at the root of the jar file but not > when it's in a Lib/ subfolder of the jar file. > > Here is my java code: > > PythonInterpreter pythonInterpreter = new PythonInterpreter(); > pythonInterpreter.exec("import pygments\nfrom pygments.lexers import > guess_lexer"); Aha! The keyword then is "import site". easy_install probably installs a egg in site-packages? (at least it always does that for me). I recommend comparing your initialization code with the AbstractInitJython code from https://sourceforge.net/p/mavenjython/code/ci/8a27a826b5e77825e02611263e34ac84e233f824/tree/jython-compile-maven-plugin-test/src/main/java/net/sf/mavenjython/AbstractJythonInit.java Around line 38, you will see imp.load("site"); If you still have issues, start with the demo project, and add Pygments as a python library. Then drop in the console and see if it lets you import it. I just tested this, and it works for me. Again, my answer would be to approximate your init code with the demo project's. But I also have to say that here, Pygments-1.4-py2.5.egg/ is put into Lib/site-packages, there is no Lib/pygments/ folder. There is something different at your end that I can't reproduce. Also try mvn clean (or deleting the target/jython-plugins-tmp/ folder. You can debug this problem better by dropping into the jython console (instead of running your code, run c.interact()). Then take a look at the python-path variables (e.g. sys.path). I have sys.path = ['my-jar-with-dependencies.jar/Lib', '__classpath__', '__pyclasspath__/'] If you insist on using your packaged jar, there should be another entry there, 'pygments-1.4.jar/pygments'. Which solution you choose also depends a lot on how you want to deploy to users. Best of luck, Johannes > > with both pygments-1.4.jar and jython-standalone-2.5.2.jar in my classpath. > > If pygments-1.4.jar contains /Lib/pygments/... it does not work but if > it directly have /pygments/... it's working well. I even tested in my > working jar to move things to a Lib/ folder in the jar and it does not > work so it's really not related to the maven plugin except for the > fact that it put things in /Lib/ folder. > > By not working I mean getting "ImportError: No module named pygments". > >> You can do this by overriding newInterpreter or createEnterpreter to >> call PySystemState.path.append() >> or you can, in run(), append your folder to the completed console. >> c.eval("sys.path.append('myfolder')") >> If you don't know/use the (Abstract)InitJython I'm talking about here, look at the demo project in http://mavenjython.sourceforge.net/ and its code [2]. >> >> Well, please get back to this list to let us know what the exact issue you are struggling with is, and in case you were able to solve it, how you did. >> >> All the best, >> Johannes >> >> [1] http://pypi.python.org/pypi/Pygments >> [2] https://sourceforge.net/p/mavenjython/code/ci/8a27a826b5e77825e02611263e34ac84e233f824/tree/jython-compile-maven-plugin-test/src/main/java/net/sf/mavenjython/ >> >> On 09/16/2011 12:39 PM, Thomas Mortagne wrote: >>> Hi, >>> >>> The project seems great but I have some issue which is probably more >>> related to Jython itself actually but will will probably be able to >>> help ;) >>> >>> I'm packaging Pygments as a library jar used by some other one, at >>> startup Jythong scan jars and find it. You can see it on >>> http://maven.xwiki.org/externals/pygments/pygments/1.4-xwiki/. >>> >>> I would love to use your maven plugin instead of downloading and >>> putting myself the Pygments files in my maven project but looks like >>> Jython is unable to find Pygments when built with it. The difference I >>> see is that your plugin put the python files in a Lib/ subfolder and >>> I'm packing then directly at the root of the jar in >>> http://maven.xwiki.org/externals/pygments/pygments/1.4-xwiki/. >>> >>> Is it me not properly using Jython ? Any idea what could I do to use >>> your plugin in my use case ? >>> >>> On Wed, Aug 17, 2011 at 01:09, Johannes Buchner <buc...@gm...> wrote: >>> Dear Jython Users! >>> >>> I have released version 1.0 of two Maven plugins that make the >>> integration of Jython into Java projects easier. >>> >>> The first one (jython-compile-maven-plugin) allows packaging of jython >>> including python libraries. >>> The second (python-test-maven-plugin) allows testing of python code. >>> They are hosted on http://mavenjython.sourceforge.net/ and I put some >>> information on them at >>> http://wiki.python.org/jython/JythonFaq/DistributingJythonScripts#Using_Maven >>> >>> >>> >>> I would welcome any feedback on what can be improved or what is needed. >>> >>> Cheers, >>> Johannes >> |
From: Thomas M. <tho...@gm...> - 2011-09-19 08:14:27
|
On Fri, Sep 16, 2011 at 23:31, Johannes Buchner <buc...@gm...> wrote: > On 09/16/2011 04:43 PM, Thomas Mortagne wrote: >> On Fri, Sep 16, 2011 at 15:29, Johannes Buchner <buc...@gm...> wrote: >>> Hi Thomas, >>> >>> If you look at http://mavenjython.sourceforge.net/compile/usage.html >>> you will see how in a demo project, the nose python library is included. >>> Instead, you should put "Pygments" there, as this is the way pypi [1] has it in its catalogue. If you want a specific version, use Pygments==1.4, like easy_install demands it. >>> >>> You probably did that already -- if you run mvn package, what is the output? maven-jython-compile-plugin is pretty verbose anyways. >>> >>> If you use a HTTP proxy, you have to set the http(s)_proxy environment variables, as the proxy settings of maven are not forwarded to easy_install. >>> >>> I think putting all python stuff in Lib/ can be a good convention. If you don't like it for some reason, during your jython initialization (InitJython), add your folder to the python path. >> >> As I said I don't have any issue with the plugin and it properly >> generate a jar file with pygments in it. I'm just talking about the >> fact that it's putting pygments in a Lib/ subfolder in the jar. >> >> It's not that I don't like it, it's that it does not work by default >> with Jython as far as I can see in my pretty simple code: Jython is >> able to find pygments when it's at the root of the jar file but not >> when it's in a Lib/ subfolder of the jar file. >> >> Here is my java code: >> >> PythonInterpreter pythonInterpreter = new PythonInterpreter(); >> pythonInterpreter.exec("import pygments\nfrom pygments.lexers import >> guess_lexer"); > > Aha! The keyword then is "import site". easy_install probably installs a egg in site-packages? (at least it always does that for me). > I recommend comparing your initialization code with the AbstractInitJython code from > https://sourceforge.net/p/mavenjython/code/ci/8a27a826b5e77825e02611263e34ac84e233f824/tree/jython-compile-maven-plugin-test/src/main/java/net/sf/mavenjython/AbstractJythonInit.java > > Around line 38, you will see imp.load("site"); > > If you still have issues, start with the demo project, and add Pygments as a python library. Then drop in the console and see if it lets you import it. > I just tested this, and it works for me. Again, my answer would be to approximate your init code with the demo project's. > > But I also have to say that here, Pygments-1.4-py2.5.egg/ is put into Lib/site-packages, there is no Lib/pygments/ folder. There is something different at your end that I can't reproduce. Also try mvn clean (or deleting the target/jython-plugins-tmp/ folder. Here is the pom.xml I'm using: xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd"> <modelVersion>4.0.0</modelVersion> <groupId>pygments</groupId> <artifactId>pygments</artifactId> <version>1.4-xwiki</version> <name>Pygments</name> <packaging>jar</packaging> <dependencies> <dependency> <groupId>org.python</groupId> <artifactId>jython-standalone</artifactId> <version>2.5.2</version> </dependency> </dependencies> <build> <plugins> <plugin> <groupId>net.sf.mavenjython</groupId> <artifactId>jython-compile-maven-plugin</artifactId> <version>1.2</version> <executions> <execution> <phase>compile</phase> <goals> <goal>jython</goal> </goals> </execution> </executions> <configuration> <libraries> <param>pygments</param> </libraries> </configuration> </plugin> </plugins> </build> </project> And that definitely produce me a jar containing: META-INF/... Lib -- EGG-INFO/... -- pygments/... -- setuptools/... -- easy_install$py.class -- easy_install.py -- pkg_resources$py.class -- pkg_resources.py I'm using "mvn clean install" and I'm executing that on Ubuntu 11.4 64b with Java 6 and Maven 3.0.3. I can send you the produced jar I you want (it's a bit big to attach it to the mailing list I guess). BTW it produces a lot of things that are useless for me, would be nice to have some property allowing to package only the library and not all the easy-install stuff. > > You can debug this problem better by dropping into the jython console (instead of running your code, run c.interact()). Then take a look at the python-path variables (e.g. sys.path). > I have sys.path = ['my-jar-with-dependencies.jar/Lib', '__classpath__', '__pyclasspath__/'] > > If you insist on using your packaged jar, there should be another entry there, 'pygments-1.4.jar/pygments'. > > Which solution you choose also depends a lot on how you want to deploy to users. > > Best of luck, > Johannes > >> >> with both pygments-1.4.jar and jython-standalone-2.5.2.jar in my classpath. >> >> If pygments-1.4.jar contains /Lib/pygments/... it does not work but if >> it directly have /pygments/... it's working well. I even tested in my >> working jar to move things to a Lib/ folder in the jar and it does not >> work so it's really not related to the maven plugin except for the >> fact that it put things in /Lib/ folder. >> >> By not working I mean getting "ImportError: No module named pygments". >> >>> You can do this by overriding newInterpreter or createEnterpreter to >>> call PySystemState.path.append() >>> or you can, in run(), append your folder to the completed console. >>> c.eval("sys.path.append('myfolder')") >>> If you don't know/use the (Abstract)InitJython I'm talking about here, look at the demo project in http://mavenjython.sourceforge.net/ and its code [2]. >>> >>> Well, please get back to this list to let us know what the exact issue you are struggling with is, and in case you were able to solve it, how you did. >>> >>> All the best, >>> Johannes >>> >>> [1] http://pypi.python.org/pypi/Pygments >>> [2] https://sourceforge.net/p/mavenjython/code/ci/8a27a826b5e77825e02611263e34ac84e233f824/tree/jython-compile-maven-plugin-test/src/main/java/net/sf/mavenjython/ >>> >>> On 09/16/2011 12:39 PM, Thomas Mortagne wrote: >>>> Hi, >>>> >>>> The project seems great but I have some issue which is probably more >>>> related to Jython itself actually but will will probably be able to >>>> help ;) >>>> >>>> I'm packaging Pygments as a library jar used by some other one, at >>>> startup Jythong scan jars and find it. You can see it on >>>> http://maven.xwiki.org/externals/pygments/pygments/1.4-xwiki/. >>>> >>>> I would love to use your maven plugin instead of downloading and >>>> putting myself the Pygments files in my maven project but looks like >>>> Jython is unable to find Pygments when built with it. The difference I >>>> see is that your plugin put the python files in a Lib/ subfolder and >>>> I'm packing then directly at the root of the jar in >>>> http://maven.xwiki.org/externals/pygments/pygments/1.4-xwiki/. >>>> >>>> Is it me not properly using Jython ? Any idea what could I do to use >>>> your plugin in my use case ? >>>> >>>> On Wed, Aug 17, 2011 at 01:09, Johannes Buchner <buc...@gm...> wrote: >>>> Dear Jython Users! >>>> >>>> I have released version 1.0 of two Maven plugins that make the >>>> integration of Jython into Java projects easier. >>>> >>>> The first one (jython-compile-maven-plugin) allows packaging of jython >>>> including python libraries. >>>> The second (python-test-maven-plugin) allows testing of python code. >>>> They are hosted on http://mavenjython.sourceforge.net/ and I put some >>>> information on them at >>>> http://wiki.python.org/jython/JythonFaq/DistributingJythonScripts#Using_Maven >>>> >>>> >>>> >>>> I would welcome any feedback on what can be improved or what is needed. >>>> >>>> Cheers, >>>> Johannes >>> > -- Thomas |
From: Pierre T. <pie...@gm...> - 2011-09-19 14:14:51
|
2011/9/19 Thomas Mortagne <tho...@gm...> > Here is the pom.xml I'm using: > > xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 > http://maven.apache.org/maven-v4_0_0.xsd"> > <modelVersion>4.0.0</modelVersion> > <groupId>pygments</groupId> > <artifactId>pygments</artifactId>... > OK. I was able to reproduce the problem. It does not work when the files are located in the Lib directory of the archive. It works when they are at the root. This is not fun. It should have an option for that. Pierre Thibault Python Developer/Développeur Python Montréal, QC *pie...@gm...* |
From: Johannes B. <buc...@gm...> - 2011-09-18 00:07:40
|
Von Pierre Thibault am 18.09.11 00:05 Uhr 2011/8/16 Johannes Buchner < buc...@gm... > information on them at http://wiki.python.org/jython/JythonFaq/DistributingJythonScripts#Using_Maven The demo project link does not work. Thanks, fixed. Johannes |
From: Johannes B. <buc...@gm...> - 2011-09-19 11:17:41
|
Hi Thomas! Thank you for your email. I see it now and can reproduce the problem. However, importing works fine if you do: [user@localhost pygments-test]$ jython Jython 2.5.2 (Release_2_5_2:7206, Mar 2 2011, 23:12:06) [OpenJDK Server VM (Sun Microsystems Inc.)] on java1.6.0_22 Type "help", "copyright", "credits" or "license" for more information. >>> import sys >>> sys.path.append("target/pygments-1.4-xwiki.jar/Lib") >>> import pygments >>> or $ java -Dpython.path=target/pygments-1.4-xwiki.jar/Lib -jar ~/bin/jython/jython.jar Jython 2.5.2 (Release_2_5_2:7206, Mar 2 2011, 23:12:06) [OpenJDK Server VM (Sun Microsystems Inc.)] on java1.6.0_22 Type "help", "copyright", "credits" or "license" for more information. >>> import pygments >>> or modifying PySystemState.path in a similar manner. Now you have the jar with pygments in Lib/, how are you deploying to users? If you use jar-with-dependencies to create a fat jar including jython, the Lib/ folder will be merged and the library will load fine. This is the case I always have, as I deploy jython with the application. How is your application (xwiki being a web application, I presume), being launched? Which jython do you use? The one installed on the system? Do you want to end up with one big jar or many small ones? If the latter, in some form jython has to be informed to look into those jars' Lib folder. Please expand on how your application is finally launched. One solution would be, as you already discovered, to put the libraries in / instead of Lib, as they will be discovered when jython searches the classpath. However, I am hesitant to this, because it would mean that we are in fact repackaging all pypi packages as jars, which is pointless work. I am sorry for the confusion before, I forgot that I specifically programmed the plugin to take all the site-package eggs (from buttarget/jython-plugins-tmp/Lib/site-packages/) and copy them to Lib/. Yes, I am considering allowing to remove easy_install; So far I only had the use case where the plugin is used on the final user deployable jar-with-dependencies. Cheers, Johannes On 09/19/2011 10:14 AM, Thomas Mortagne wrote: > On Fri, Sep 16, 2011 at 23:31, Johannes Buchner <buc...@gm...> wrote: >> On 09/16/2011 04:43 PM, Thomas Mortagne wrote: >>> On Fri, Sep 16, 2011 at 15:29, Johannes Buchner <buc...@gm...> wrote: >>>> Hi Thomas, >>>> >>>> If you look at http://mavenjython.sourceforge.net/compile/usage.html >>>> you will see how in a demo project, the nose python library is included. >>>> Instead, you should put "Pygments" there, as this is the way pypi [1] has it in its catalogue. If you want a specific version, use Pygments==1.4, like easy_install demands it. >>>> >>>> You probably did that already -- if you run mvn package, what is the output? maven-jython-compile-plugin is pretty verbose anyways. >>>> >>>> If you use a HTTP proxy, you have to set the http(s)_proxy environment variables, as the proxy settings of maven are not forwarded to easy_install. >>>> >>>> I think putting all python stuff in Lib/ can be a good convention. If you don't like it for some reason, during your jython initialization (InitJython), add your folder to the python path. >>> >>> As I said I don't have any issue with the plugin and it properly >>> generate a jar file with pygments in it. I'm just talking about the >>> fact that it's putting pygments in a Lib/ subfolder in the jar. >>> >>> It's not that I don't like it, it's that it does not work by default >>> with Jython as far as I can see in my pretty simple code: Jython is >>> able to find pygments when it's at the root of the jar file but not >>> when it's in a Lib/ subfolder of the jar file. >>> >>> Here is my java code: >>> >>> PythonInterpreter pythonInterpreter = new PythonInterpreter(); >>> pythonInterpreter.exec("import pygments\nfrom pygments.lexers import >>> guess_lexer"); >> >> Aha! The keyword then is "import site". easy_install probably installs a egg in site-packages? (at least it always does that for me). >> I recommend comparing your initialization code with the AbstractInitJython code from >> https://sourceforge.net/p/mavenjython/code/ci/8a27a826b5e77825e02611263e34ac84e233f824/tree/jython-compile-maven-plugin-test/src/main/java/net/sf/mavenjython/AbstractJythonInit.java >> >> Around line 38, you will see imp.load("site"); >> >> If you still have issues, start with the demo project, and add Pygments as a python library. Then drop in the console and see if it lets you import it. >> I just tested this, and it works for me. Again, my answer would be to approximate your init code with the demo project's. >> >> But I also have to say that here, Pygments-1.4-py2.5.egg/ is put into Lib/site-packages, there is no Lib/pygments/ folder. There is something different at your end that I can't reproduce. Also try mvn clean (or deleting the target/jython-plugins-tmp/ folder. > > Here is the pom.xml I'm using: > > xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 > http://maven.apache.org/maven-v4_0_0.xsd"> > <modelVersion>4.0.0</modelVersion> > <groupId>pygments</groupId> > <artifactId>pygments</artifactId> > <version>1.4-xwiki</version> > <name>Pygments</name> > <packaging>jar</packaging> > <dependencies> > <dependency> > <groupId>org.python</groupId> > <artifactId>jython-standalone</artifactId> > <version>2.5.2</version> > </dependency> > </dependencies> > <build> > <plugins> > <plugin> > <groupId>net.sf.mavenjython</groupId> > <artifactId>jython-compile-maven-plugin</artifactId> > <version>1.2</version> > <executions> > <execution> > <phase>compile</phase> > <goals> > <goal>jython</goal> > </goals> > </execution> > </executions> > <configuration> > <libraries> > <param>pygments</param> > </libraries> > </configuration> > </plugin> > </plugins> > </build> > </project> > > And that definitely produce me a jar containing: > > META-INF/... > Lib > -- EGG-INFO/... > -- pygments/... > -- setuptools/... > -- easy_install$py.class > -- easy_install.py > -- pkg_resources$py.class > -- pkg_resources.py > > I'm using "mvn clean install" and I'm executing that on Ubuntu 11.4 > 64b with Java 6 and Maven 3.0.3. I can send you the produced jar I you > want (it's a bit big to attach it to the mailing list I guess). > > BTW it produces a lot of things that are useless for me, would be nice > to have some property allowing to package only the library and not all > the easy-install stuff. > >> >> You can debug this problem better by dropping into the jython console (instead of running your code, run c.interact()). Then take a look at the python-path variables (e.g. sys.path). >> I have sys.path = ['my-jar-with-dependencies.jar/Lib', '__classpath__', '__pyclasspath__/'] >> >> If you insist on using your packaged jar, there should be another entry there, 'pygments-1.4.jar/pygments'. >> >> Which solution you choose also depends a lot on how you want to deploy to users. >> >> Best of luck, >> Johannes >> >>> >>> with both pygments-1.4.jar and jython-standalone-2.5.2.jar in my classpath. >>> >>> If pygments-1.4.jar contains /Lib/pygments/... it does not work but if >>> it directly have /pygments/... it's working well. I even tested in my >>> working jar to move things to a Lib/ folder in the jar and it does not >>> work so it's really not related to the maven plugin except for the >>> fact that it put things in /Lib/ folder. >>> >>> By not working I mean getting "ImportError: No module named pygments". >>> >>>> You can do this by overriding newInterpreter or createEnterpreter to >>>> call PySystemState.path.append() >>>> or you can, in run(), append your folder to the completed console. >>>> c.eval("sys.path.append('myfolder')") >>>> If you don't know/use the (Abstract)InitJython I'm talking about here, look at the demo project in http://mavenjython.sourceforge.net/ and its code [2]. >>>> >>>> Well, please get back to this list to let us know what the exact issue you are struggling with is, and in case you were able to solve it, how you did. >>>> >>>> All the best, >>>> Johannes >>>> >>>> [1] http://pypi.python.org/pypi/Pygments >>>> [2] https://sourceforge.net/p/mavenjython/code/ci/8a27a826b5e77825e02611263e34ac84e233f824/tree/jython-compile-maven-plugin-test/src/main/java/net/sf/mavenjython/ >>>> >>>> On 09/16/2011 12:39 PM, Thomas Mortagne wrote: >>>>> Hi, >>>>> >>>>> The project seems great but I have some issue which is probably more >>>>> related to Jython itself actually but will will probably be able to >>>>> help ;) >>>>> >>>>> I'm packaging Pygments as a library jar used by some other one, at >>>>> startup Jythong scan jars and find it. You can see it on >>>>> http://maven.xwiki.org/externals/pygments/pygments/1.4-xwiki/. >>>>> >>>>> I would love to use your maven plugin instead of downloading and >>>>> putting myself the Pygments files in my maven project but looks like >>>>> Jython is unable to find Pygments when built with it. The difference I >>>>> see is that your plugin put the python files in a Lib/ subfolder and >>>>> I'm packing then directly at the root of the jar in >>>>> http://maven.xwiki.org/externals/pygments/pygments/1.4-xwiki/. >>>>> >>>>> Is it me not properly using Jython ? Any idea what could I do to use >>>>> your plugin in my use case ? >>>>> >>>>> On Wed, Aug 17, 2011 at 01:09, Johannes Buchner <buc...@gm...> wrote: >>>>> Dear Jython Users! >>>>> >>>>> I have released version 1.0 of two Maven plugins that make the >>>>> integration of Jython into Java projects easier. >>>>> >>>>> The first one (jython-compile-maven-plugin) allows packaging of jython >>>>> including python libraries. >>>>> The second (python-test-maven-plugin) allows testing of python code. >>>>> They are hosted on http://mavenjython.sourceforge.net/ and I put some >>>>> information on them at >>>>> http://wiki.python.org/jython/JythonFaq/DistributingJythonScripts#Using_Maven >>>>> >>>>> >>>>> >>>>> I would welcome any feedback on what can be improved or what is needed. >>>>> >>>>> Cheers, >>>>> Johannes >>>> |
From: Thomas M. <tho...@gm...> - 2011-09-19 13:02:09
|
On Mon, Sep 19, 2011 at 13:17, Johannes Buchner <buc...@gm...> wrote: > Hi Thomas! > > Thank you for your email. I see it now and can reproduce the problem. > > However, importing works fine if you do: > > [user@localhost pygments-test]$ jython > Jython 2.5.2 (Release_2_5_2:7206, Mar 2 2011, 23:12:06) > [OpenJDK Server VM (Sun Microsystems Inc.)] on java1.6.0_22 > Type "help", "copyright", "credits" or "license" for more information. >>>> import sys >>>> sys.path.append("target/pygments-1.4-xwiki.jar/Lib") >>>> import pygments >>>> > > or > $ java -Dpython.path=target/pygments-1.4-xwiki.jar/Lib -jar ~/bin/jython/jython.jar > Jython 2.5.2 (Release_2_5_2:7206, Mar 2 2011, 23:12:06) > [OpenJDK Server VM (Sun Microsystems Inc.)] on java1.6.0_22 > Type "help", "copyright", "credits" or "license" for more information. >>>> import pygments >>>> > > or modifying PySystemState.path in a similar manner. > > Now you have the jar with pygments in Lib/, how are you deploying to users? > > If you use jar-with-dependencies to create a fat jar including jython, the Lib/ folder will be merged and the library will load fine. This is the case I always have, as I deploy jython with the application. I'm using it in a webapp so as a jar loaded by Tomcat for example. It's distributed in a jar packaged in a war generally. > > How is your application (xwiki being a web application, I presume), being launched? Which jython do you use? The one installed on the system? I'm using jython-standalone jar in the webapp along with the pygments jar (and potentially others later if someone require a python library for another component) and it seems cleaner to me to keep them separated since for example XWiki python macro require jython but not Pygments but if Pygments is provided in the classpath it's nice to be able to use it in your python code (same for any other Python lib packaged as a jar). Said another way there is a lot of possible final distributions packages depending of what components (and their dependencies) you peak in the XWiki platform. > Do you want to end up with one big jar or many small ones? If the latter, in some form jython has to be informed to look into those jars' Lib folder. Many jars. Maven style war packages. > Please expand on how your application is finally launched. Either as a war or expanded war but from classloader point of view it's pretty much the same thing: different jars. > > One solution would be, as you already discovered, to put the libraries in / instead of Lib, as they will be discovered when jython searches the classpath. However, I am hesitant to this, because it would mean that we are in fact repackaging all pypi packages as jars, which is pointless work. Well transforming a Python packaging in jar is after all what I want here. Or more precisely transforming it in something that is automatically discovered by Jython. Wondering if that would be doable to make Jython search for egg packaging in the classpath, sounds the more natural thing to me. Not sure how Java would react when hitting file like that in its classpath. I will keep my current root folder in jar strategy for now and try to find some time to explorer the above when Pygments 1.5 is out. > > I am sorry for the confusion before, I forgot that I specifically programmed the plugin to take all the site-package eggs (from buttarget/jython-plugins-tmp/Lib/site-packages/) and copy them to Lib/. > > Yes, I am considering allowing to remove easy_install; So far I only had the use case where the plugin is used on the final user deployable jar-with-dependencies. Ok that would be nice, easy_install is not that big but the less the better ;) > > Cheers, > Johannes > > > > On 09/19/2011 10:14 AM, Thomas Mortagne wrote: >> On Fri, Sep 16, 2011 at 23:31, Johannes Buchner <buc...@gm...> wrote: >>> On 09/16/2011 04:43 PM, Thomas Mortagne wrote: >>>> On Fri, Sep 16, 2011 at 15:29, Johannes Buchner <buc...@gm...> wrote: >>>>> Hi Thomas, >>>>> >>>>> If you look at http://mavenjython.sourceforge.net/compile/usage.html >>>>> you will see how in a demo project, the nose python library is included. >>>>> Instead, you should put "Pygments" there, as this is the way pypi [1] has it in its catalogue. If you want a specific version, use Pygments==1.4, like easy_install demands it. >>>>> >>>>> You probably did that already -- if you run mvn package, what is the output? maven-jython-compile-plugin is pretty verbose anyways. >>>>> >>>>> If you use a HTTP proxy, you have to set the http(s)_proxy environment variables, as the proxy settings of maven are not forwarded to easy_install. >>>>> >>>>> I think putting all python stuff in Lib/ can be a good convention. If you don't like it for some reason, during your jython initialization (InitJython), add your folder to the python path. >>>> >>>> As I said I don't have any issue with the plugin and it properly >>>> generate a jar file with pygments in it. I'm just talking about the >>>> fact that it's putting pygments in a Lib/ subfolder in the jar. >>>> >>>> It's not that I don't like it, it's that it does not work by default >>>> with Jython as far as I can see in my pretty simple code: Jython is >>>> able to find pygments when it's at the root of the jar file but not >>>> when it's in a Lib/ subfolder of the jar file. >>>> >>>> Here is my java code: >>>> >>>> PythonInterpreter pythonInterpreter = new PythonInterpreter(); >>>> pythonInterpreter.exec("import pygments\nfrom pygments.lexers import >>>> guess_lexer"); >>> >>> Aha! The keyword then is "import site". easy_install probably installs a egg in site-packages? (at least it always does that for me). >>> I recommend comparing your initialization code with the AbstractInitJython code from >>> https://sourceforge.net/p/mavenjython/code/ci/8a27a826b5e77825e02611263e34ac84e233f824/tree/jython-compile-maven-plugin-test/src/main/java/net/sf/mavenjython/AbstractJythonInit.java >>> >>> Around line 38, you will see imp.load("site"); >>> >>> If you still have issues, start with the demo project, and add Pygments as a python library. Then drop in the console and see if it lets you import it. >>> I just tested this, and it works for me. Again, my answer would be to approximate your init code with the demo project's. >>> >>> But I also have to say that here, Pygments-1.4-py2.5.egg/ is put into Lib/site-packages, there is no Lib/pygments/ folder. There is something different at your end that I can't reproduce. Also try mvn clean (or deleting the target/jython-plugins-tmp/ folder. >> >> Here is the pom.xml I'm using: >> >> xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 >> http://maven.apache.org/maven-v4_0_0.xsd"> >> <modelVersion>4.0.0</modelVersion> >> <groupId>pygments</groupId> >> <artifactId>pygments</artifactId> >> <version>1.4-xwiki</version> >> <name>Pygments</name> >> <packaging>jar</packaging> >> <dependencies> >> <dependency> >> <groupId>org.python</groupId> >> <artifactId>jython-standalone</artifactId> >> <version>2.5.2</version> >> </dependency> >> </dependencies> >> <build> >> <plugins> >> <plugin> >> <groupId>net.sf.mavenjython</groupId> >> <artifactId>jython-compile-maven-plugin</artifactId> >> <version>1.2</version> >> <executions> >> <execution> >> <phase>compile</phase> >> <goals> >> <goal>jython</goal> >> </goals> >> </execution> >> </executions> >> <configuration> >> <libraries> >> <param>pygments</param> >> </libraries> >> </configuration> >> </plugin> >> </plugins> >> </build> >> </project> >> >> And that definitely produce me a jar containing: >> >> META-INF/... >> Lib >> -- EGG-INFO/... >> -- pygments/... >> -- setuptools/... >> -- easy_install$py.class >> -- easy_install.py >> -- pkg_resources$py.class >> -- pkg_resources.py >> >> I'm using "mvn clean install" and I'm executing that on Ubuntu 11.4 >> 64b with Java 6 and Maven 3.0.3. I can send you the produced jar I you >> want (it's a bit big to attach it to the mailing list I guess). >> >> BTW it produces a lot of things that are useless for me, would be nice >> to have some property allowing to package only the library and not all >> the easy-install stuff. >> >>> >>> You can debug this problem better by dropping into the jython console (instead of running your code, run c.interact()). Then take a look at the python-path variables (e.g. sys.path). >>> I have sys.path = ['my-jar-with-dependencies.jar/Lib', '__classpath__', '__pyclasspath__/'] >>> >>> If you insist on using your packaged jar, there should be another entry there, 'pygments-1.4.jar/pygments'. >>> >>> Which solution you choose also depends a lot on how you want to deploy to users. >>> >>> Best of luck, >>> Johannes >>> >>>> >>>> with both pygments-1.4.jar and jython-standalone-2.5.2.jar in my classpath. >>>> >>>> If pygments-1.4.jar contains /Lib/pygments/... it does not work but if >>>> it directly have /pygments/... it's working well. I even tested in my >>>> working jar to move things to a Lib/ folder in the jar and it does not >>>> work so it's really not related to the maven plugin except for the >>>> fact that it put things in /Lib/ folder. >>>> >>>> By not working I mean getting "ImportError: No module named pygments". >>>> >>>>> You can do this by overriding newInterpreter or createEnterpreter to >>>>> call PySystemState.path.append() >>>>> or you can, in run(), append your folder to the completed console. >>>>> c.eval("sys.path.append('myfolder')") >>>>> If you don't know/use the (Abstract)InitJython I'm talking about here, look at the demo project in http://mavenjython.sourceforge.net/ and its code [2]. >>>>> >>>>> Well, please get back to this list to let us know what the exact issue you are struggling with is, and in case you were able to solve it, how you did. >>>>> >>>>> All the best, >>>>> Johannes >>>>> >>>>> [1] http://pypi.python.org/pypi/Pygments >>>>> [2] https://sourceforge.net/p/mavenjython/code/ci/8a27a826b5e77825e02611263e34ac84e233f824/tree/jython-compile-maven-plugin-test/src/main/java/net/sf/mavenjython/ >>>>> >>>>> On 09/16/2011 12:39 PM, Thomas Mortagne wrote: >>>>>> Hi, >>>>>> >>>>>> The project seems great but I have some issue which is probably more >>>>>> related to Jython itself actually but will will probably be able to >>>>>> help ;) >>>>>> >>>>>> I'm packaging Pygments as a library jar used by some other one, at >>>>>> startup Jythong scan jars and find it. You can see it on >>>>>> http://maven.xwiki.org/externals/pygments/pygments/1.4-xwiki/. >>>>>> >>>>>> I would love to use your maven plugin instead of downloading and >>>>>> putting myself the Pygments files in my maven project but looks like >>>>>> Jython is unable to find Pygments when built with it. The difference I >>>>>> see is that your plugin put the python files in a Lib/ subfolder and >>>>>> I'm packing then directly at the root of the jar in >>>>>> http://maven.xwiki.org/externals/pygments/pygments/1.4-xwiki/. >>>>>> >>>>>> Is it me not properly using Jython ? Any idea what could I do to use >>>>>> your plugin in my use case ? >>>>>> >>>>>> On Wed, Aug 17, 2011 at 01:09, Johannes Buchner <buc...@gm...> wrote: >>>>>> Dear Jython Users! >>>>>> >>>>>> I have released version 1.0 of two Maven plugins that make the >>>>>> integration of Jython into Java projects easier. >>>>>> >>>>>> The first one (jython-compile-maven-plugin) allows packaging of jython >>>>>> including python libraries. >>>>>> The second (python-test-maven-plugin) allows testing of python code. >>>>>> They are hosted on http://mavenjython.sourceforge.net/ and I put some >>>>>> information on them at >>>>>> http://wiki.python.org/jython/JythonFaq/DistributingJythonScripts#Using_Maven >>>>>> >>>>>> >>>>>> >>>>>> I would welcome any feedback on what can be improved or what is needed. >>>>>> >>>>>> Cheers, >>>>>> Johannes >>>>> > -- Thomas |
From: Thomas M. <tho...@gm...> - 2011-09-16 14:43:48
|
On Fri, Sep 16, 2011 at 15:29, Johannes Buchner <buc...@gm...> wrote: > Hi Thomas, > > If you look at http://mavenjython.sourceforge.net/compile/usage.html > you will see how in a demo project, the nose python library is included. > Instead, you should put "Pygments" there, as this is the way pypi [1] has it in its catalogue. If you want a specific version, use Pygments==1.4, like easy_install demands it. > > You probably did that already -- if you run mvn package, what is the output? maven-jython-compile-plugin is pretty verbose anyways. > > If you use a HTTP proxy, you have to set the http(s)_proxy environment variables, as the proxy settings of maven are not forwarded to easy_install. > > I think putting all python stuff in Lib/ can be a good convention. If you don't like it for some reason, during your jython initialization (InitJython), add your folder to the python path. As I said I don't have any issue with the plugin and it properly generate a jar file with pygments in it. I'm just talking about the fact that it's putting pygments in a Lib/ subfolder in the jar. It's not that I don't like it, it's that it does not work by default with Jython as far as I can see in my pretty simple code: Jython is able to find pygments when it's at the root of the jar file but not when it's in a Lib/ subfolder of the jar file. Here is my java code: PythonInterpreter pythonInterpreter = new PythonInterpreter(); pythonInterpreter.exec("import pygments\nfrom pygments.lexers import guess_lexer"); with both pygments-1.4.jar and jython-standalone-2.5.2.jar in my classpath. If pygments-1.4.jar contains /Lib/pygments/... it does not work but if it directly have /pygments/... it's working well. I even tested in my working jar to move things to a Lib/ folder in the jar and it does not work so it's really not related to the maven plugin except for the fact that it put things in /Lib/ folder. By not working I mean getting "ImportError: No module named pygments". > You can do this by overriding newInterpreter or createEnterpreter to > call PySystemState.path.append() > or you can, in run(), append your folder to the completed console. > c.eval("sys.path.append('myfolder')") > If you don't know/use the (Abstract)InitJython I'm talking about here, look at the demo project in http://mavenjython.sourceforge.net/ and its code [2]. > > Well, please get back to this list to let us know what the exact issue you are struggling with is, and in case you were able to solve it, how you did. > > All the best, > Johannes > > [1] http://pypi.python.org/pypi/Pygments > [2] https://sourceforge.net/p/mavenjython/code/ci/8a27a826b5e77825e02611263e34ac84e233f824/tree/jython-compile-maven-plugin-test/src/main/java/net/sf/mavenjython/ > > On 09/16/2011 12:39 PM, Thomas Mortagne wrote: >> Hi, >> >> The project seems great but I have some issue which is probably more >> related to Jython itself actually but will will probably be able to >> help ;) >> >> I'm packaging Pygments as a library jar used by some other one, at >> startup Jythong scan jars and find it. You can see it on >> http://maven.xwiki.org/externals/pygments/pygments/1.4-xwiki/. >> >> I would love to use your maven plugin instead of downloading and >> putting myself the Pygments files in my maven project but looks like >> Jython is unable to find Pygments when built with it. The difference I >> see is that your plugin put the python files in a Lib/ subfolder and >> I'm packing then directly at the root of the jar in >> http://maven.xwiki.org/externals/pygments/pygments/1.4-xwiki/. >> >> Is it me not properly using Jython ? Any idea what could I do to use >> your plugin in my use case ? >> >> On Wed, Aug 17, 2011 at 01:09, Johannes Buchner <buc...@gm...> wrote: >> Dear Jython Users! >> >> I have released version 1.0 of two Maven plugins that make the >> integration of Jython into Java projects easier. >> >> The first one (jython-compile-maven-plugin) allows packaging of jython >> including python libraries. >> The second (python-test-maven-plugin) allows testing of python code. >> They are hosted on http://mavenjython.sourceforge.net/ and I put some >> information on them at >> http://wiki.python.org/jython/JythonFaq/DistributingJythonScripts#Using_Maven >> >> >> >> I would welcome any feedback on what can be improved or what is needed. >> >> Cheers, >> Johannes > -- Thomas |
From: Thomas M. <tho...@gm...> - 2011-09-16 14:51:42
|
On Fri, Sep 16, 2011 at 16:43, Thomas Mortagne <tho...@gm...> wrote: > On Fri, Sep 16, 2011 at 15:29, Johannes Buchner <buc...@gm...> wrote: >> Hi Thomas, >> >> If you look at http://mavenjython.sourceforge.net/compile/usage.html >> you will see how in a demo project, the nose python library is included. >> Instead, you should put "Pygments" there, as this is the way pypi [1] has it in its catalogue. If you want a specific version, use Pygments==1.4, like easy_install demands it. >> >> You probably did that already -- if you run mvn package, what is the output? maven-jython-compile-plugin is pretty verbose anyways. >> >> If you use a HTTP proxy, you have to set the http(s)_proxy environment variables, as the proxy settings of maven are not forwarded to easy_install. >> >> I think putting all python stuff in Lib/ can be a good convention. If you don't like it for some reason, during your jython initialization (InitJython), add your folder to the python path. > > As I said I don't have any issue with the plugin and it properly > generate a jar file with pygments in it. I'm just talking about the > fact that it's putting pygments in a Lib/ subfolder in the jar. > > It's not that I don't like it, it's that it does not work by default > with Jython as far as I can see in my pretty simple code: Jython is > able to find pygments when it's at the root of the jar file but not > when it's in a Lib/ subfolder of the jar file. > > Here is my java code: > > PythonInterpreter pythonInterpreter = new PythonInterpreter(); > pythonInterpreter.exec("import pygments\nfrom pygments.lexers import > guess_lexer"); > > with both pygments-1.4.jar and jython-standalone-2.5.2.jar in my classpath. > > If pygments-1.4.jar contains /Lib/pygments/... it does not work but if > it directly have /pygments/... it's working well. I even tested in my > working jar to move things to a Lib/ folder in the jar and it does not > work so it's really not related to the maven plugin except for the > fact that it put things in /Lib/ folder. > > By not working I mean getting "ImportError: No module named pygments". The project can be found on https://github.com/xwiki/xwiki-platform/tree/master/xwiki-platform-core/xwiki-platform-rendering/xwiki-platform-rendering-macros/xwiki-platform-rendering-macro-code > >> You can do this by overriding newInterpreter or createEnterpreter to >> call PySystemState.path.append() >> or you can, in run(), append your folder to the completed console. >> c.eval("sys.path.append('myfolder')") >> If you don't know/use the (Abstract)InitJython I'm talking about here, look at the demo project in http://mavenjython.sourceforge.net/ and its code [2]. >> >> Well, please get back to this list to let us know what the exact issue you are struggling with is, and in case you were able to solve it, how you did. >> >> All the best, >> Johannes >> >> [1] http://pypi.python.org/pypi/Pygments >> [2] https://sourceforge.net/p/mavenjython/code/ci/8a27a826b5e77825e02611263e34ac84e233f824/tree/jython-compile-maven-plugin-test/src/main/java/net/sf/mavenjython/ >> >> On 09/16/2011 12:39 PM, Thomas Mortagne wrote: >>> Hi, >>> >>> The project seems great but I have some issue which is probably more >>> related to Jython itself actually but will will probably be able to >>> help ;) >>> >>> I'm packaging Pygments as a library jar used by some other one, at >>> startup Jythong scan jars and find it. You can see it on >>> http://maven.xwiki.org/externals/pygments/pygments/1.4-xwiki/. >>> >>> I would love to use your maven plugin instead of downloading and >>> putting myself the Pygments files in my maven project but looks like >>> Jython is unable to find Pygments when built with it. The difference I >>> see is that your plugin put the python files in a Lib/ subfolder and >>> I'm packing then directly at the root of the jar in >>> http://maven.xwiki.org/externals/pygments/pygments/1.4-xwiki/. >>> >>> Is it me not properly using Jython ? Any idea what could I do to use >>> your plugin in my use case ? >>> >>> On Wed, Aug 17, 2011 at 01:09, Johannes Buchner <buc...@gm...> wrote: >>> Dear Jython Users! >>> >>> I have released version 1.0 of two Maven plugins that make the >>> integration of Jython into Java projects easier. >>> >>> The first one (jython-compile-maven-plugin) allows packaging of jython >>> including python libraries. >>> The second (python-test-maven-plugin) allows testing of python code. >>> They are hosted on http://mavenjython.sourceforge.net/ and I put some >>> information on them at >>> http://wiki.python.org/jython/JythonFaq/DistributingJythonScripts#Using_Maven >>> >>> >>> >>> I would welcome any feedback on what can be improved or what is needed. >>> >>> Cheers, >>> Johannes >> > > > > -- > Thomas > -- Thomas |