Re: [Pydev-code] configuring Python Interpreter for more unusual setups

[Fabio Z. <fa...@es...>]

Hi Jonah,

I'm answering inline...

On Sat, Aug 10, 2013 at 9:46 PM, Jonah Graham <jo...@ki...>wrote:

> I have a follow-up questions related to configuring the Python
> Interpreter for what may be considered more "unusual" setups.
>
> The state of play today:
> 1) When creating the interpreter info, the Python reported
> sys.executable is used as the exeOrJar value. (i.e. <executable> in
> interpreterInfo.py output). This means if you select /usr/bin/python
> in the GUI, but that actually a link to /usr/bin/python2.7, the
> exeOrJar added is /usr/bin/python2.7.
> 2) The default selected Python Path is based on whether the folder is
> in sys.prefix (i.e. <lib path="ins">/<lib path="out"> in
> interpreterInfo.py output)
> 3) Jython interpreter must be a jar file[1]
>
> Both of these issues affect configuring PyDev for an interpreter that
> has a custom launch script around it. Two examples are jython on
> Ubuntu and cctbx.python (http://cci.lbl.gov/cctbx_build/). The wrapper
> scripts set up the interpreter properly (e.g. adding -Dpython.home or
> setting LD_LIBRARY_PATH). However, because PyDev uses the advertised
> sys.executable, these interpreters don't work after creation.
>
> What I would like to propose is:
> 1) that the sys.executable is no longer used, and instead PyDev uses
> the path supplied by the user. [2]
>

I think that the major issue I remember in this approach is that java had
problems running files which weren't the actual interpreter (link or shell
script)... but if this is well tested, I guess it's something to check (on
windows/linux I can check here, but I don't have access to mac os to check
that there...). Also, PyDev passes a number of different arguments, so, the
target does have to work as a Python replacement, otherwise you may end up
with the wrong PYTHONPATH when running a project.


> 2) the default paths included are all the paths in sys.path (apart
> from the ones PyDev is inserting, e.g. org.python.pydev/pysrc)
>

I think by default we could add all but the ones from PyDev and the ones
that map to some source folder for a project in the workspace (there's a
button on the config window which does that, but it should probably be the
default).


> 3) Allow Jython to be configured as a wrapper script (i.e. launch as
> exe) (this is actually an orthogonal change, but somewhat related. I
> see this has been an issue before:
>
> http://stackoverflow.com/questions/10671446/error-configuring-jython-2-5-1-on-eclipse-3-7-2-on-ubuntu-12-04


This one is probably a bit trickier as you'd have to mess with doing the
actual launch differently, and the wrapper script would need to accept the
parameters that PyDev uses to configure the paths -- and maintain
compatibility with configuring through the .jar (it's still doable, but a
bit more tricky than the others).


>
> [3])
>
> Frankly, I fear that there are many unknown (to me) issues with making
> this kind of change.
>
> If this sounds broadly acceptable, I will formulate this as an actual
> pull request for proper review. However, if this is unlikely to be
> accepted in any way, I would much appreciate the heads up so that I
> can start looking to permanently solve my interpreter info issues
> another way.
>

I think this is doable and a request that does come up from time to time,
the only thing is checking issues for how java itself interacts with those
launches, possible buffering issues, etc. If you can come up with a patch
running on the different OSes properly, I see no reason not to apply it.


>
> [1] /usr/bin/jython on Ubuntu can be configured as a Python
> interpreter but not a Jython interpreter. However I don't know how
> much real mileage you can really get on that.
>

It should work partially, mostly because you can't get completions from JDT
when configuring Jython with a dependency on a java project as specified in
http://pydev.org/manual_101_project_conf2.html (actually, any executable
which is a 'perfect' Python replacement could be configured as Python, but
Jython/IronPython have special configurations, so, other places may not
work very well -- which is not the case for PyPy, where the recommendation
is setting it up as a Python interpreter).


> [2] As mitigation, if this seems too much of a change, is to add a
> checkbox on the New... or alternatively an addition step in the wizard
> that lets the user select sys.executable value or user entered value
>

If it does work properly this shouldn't be needed...


> [3] I don't know if there are other places in PyDev that rely on
> Jython.jar being run with the same java as is known to Eclipse. For
> example, on Ubuntu it is fairly easy to have /usr/bin/jython use a
> different java than Eclipse is run with.
>

This shouldn't be a problem. As before, the main thing here is the
code-completion on things from the java core.


>
> Thanks for taking time to consider this,
> Jonah
>
>
> On 1 August 2013 21:59, Jonah Graham <jo...@ki...> wrote:
> > Hi Andrew and Fabio,
> >
> > I would like to add my two-cents on this topic. Echoing what Fabio
> > said, I think that PyDev should not include a pre-configured Python
> > (or other environment), however if redhat is providing pydev packages
> > ready to use, it should include pydev configured for the python
> > packages. (Same applies to other distributions, where possible.)
> >
> > I think this should be an external plug-in to pydev, using the
> > extensions available.
> >
> > One option is to fully configure the interpreter. Dawn has a similar
> > use case where jython is packaged to allow scripting of the Dawn suite
> > of tools. At Dawn we use:
> >
> https://github.com/DawnScience/scisoft-ui/blob/master/uk.ac.diamond.scisoft/src/uk/ac/diamond/scisoft/JythonCreator.java
> > to create a Jython Interpreter info automatically. I assume that
> > froglogic's squish does something similar as it includes an Eclipse
> > based IDE using PyDev and a  Python build in its install tree and the
> > Python is all configured in PyDev.
> >
> > Another option is to use/extend my recently contributed changes to
> > PyDev to allow the "Auto" button to be extended. For example, the
> > default implementation of Auto just searches for python, an improved
> > version could look for python2 or python3.2 or whatever else is
> > installed by the package manager. In this situation, a new plug-in
> > that uses package info can contribute to PyDev's Auto by using
> > org.python.pydev.pydev_interpreter_provider to provide a list of all
> > installed Python's known. This new extension point has the option to
> > run an install process too when selected, i.e. it could run yum or
> > whatever to install python.
> >
> > I originally implemented this new feature to make it easier for
> > scientists (who may have limited previous knowledge of
> > installing/configuring Python) to get started with Dawn. It gives the
> > option during Auto to install EPD Free's distribution of Python
> > https://www.enthought.com/products/epd/free/. The advantage of EPD is
> > that it is already preconfigured with all the packages the scientists
> > are likely to need.
> >
> > See https://github.com/fabioz/Pydev/pull/38 for the pull request that
> > added the feature.
> >
> > One final thought. I am about to do some work to make it easier to use
> > cctbx python with PyDev (for Dawn again). cctbx has some extra
> > settings that need to be set-up correctly. I haven't investigated this
> > much yet, but I may extend the above mentioned extension point to
> > allow further configuring of the interpreter info after creation. (The
> > current extension point stops at providing an IInterpreterProvider
> > which provides install option and path/name of installed python).
> > There has also been some discussion of making Sage work with Dawn and
> > PyDev, but I don't know how much demand there is for that yet.
> >
> > BTW Dawn's website in case you haven't come across it:
> > http://www.dawnsci.org/ Much of the work I am currently doing for
> > PyDev is funded by Diamond Light Source http://diamond.ac.uk/ to work
> > on improving Dawn.
> >
> > Jonah
> >
> >
> > On 1 August 2013 19:20, Fabio Zadrozny <fa...@gm...> wrote:
> >> Hi Andrew,
> >>
> >> Actually, it *should* be much better now than at that bug report:
> >>
> >> Right now, when PyDev is activated (i.e.: a Python editor is opened or
> >> something that requires the interpreter), PyDev will already try to
> >> auto-configure an interpreter, searching on different locations
> depending on
> >> your platform.
> >>
> >> After doing that, it'll still ask the user to confirm if that's correct,
> >> which I think is something good to have so that he can at least check
> we got
> >> the correct interpreter.
> >>
> >> This happens at
> >>
> org.python.pydev.ui.interpreters.AbstractInterpreterManager.ConfigureInterpreterJob
> >> (you can test on a brand new workspace opening a .py file).
> >>
> >> So, I think the issue is mostly dealt with...
> >>
> >> What can still be done to improve the situation is that when the new
> project
> >> wizard (which you're familiar with) is opened, if the user still has no
> >> interpreter configured, we'll try to configure it automatically there
> too
> >> (because right now, if he goes there without a configured interpreter,
> he
> >> has to manually click the link to choose the interpreter and press
> >> 'auto-config': this could be done for him as we know he'll have to click
> >> that link before proceeding because he has no interpreter configured --
> you
> >> can check ConfigureInterpreterJob as a reference for how to do that).
> >>
> >> Cheers,
> >>
> >> Fabio
> >>
> >>
> >> On Thu, Aug 1, 2013 at 2:38 PM, Andrew Ferrazzutti <afe...@re...
> >
> >> wrote:
> >>>
> >>> Hi Fabio,
> >>>
> >>> Alex recently assigned me to the bug where PyDev doesn't come a
> >>> pre-configured Python environment. Considering this bug was reported
> six
> >>> years ago, I'm assuming there's some reason why it hasn't been
> addressed. Is
> >>> there a certain way you wanted this to be fixed? Alex suggested using
> an
> >>> extension point to automatically search for /usr/bin/python or
> somewhere
> >>> similar for a usable binary. If there's some other method you'd rather
> use
> >>> I'm all ears.
> >>>
> >>> Andrew
> >>
> >>
> >>
> >>
> ------------------------------------------------------------------------------
> >> Get your SQL database under version control now!
> >> Version control is standard for application code, but databases havent
> >> caught up. So what steps can you take to put your SQL databases under
> >> version control? Why should you start doing it? Read more to find out.
> >>
> http://pubads.g.doubleclick.net/gampad/clk?id=49501711&iu=/4140/ostg.clktrk
> >> _______________________________________________
> >> pydev-code mailing list
> >> pyd...@li...
> >> https://lists.sourceforge.net/lists/listinfo/pydev-code
> >>
>
>
> ------------------------------------------------------------------------------
> Get 100% visibility into Java/.NET code with AppDynamics Lite!
> It's a free troubleshooting tool designed for production.
> Get down to code-level detail for bottlenecks, with <2% overhead.
> Download for free and get started troubleshooting in minutes.
> http://pubads.g.doubleclick.net/gampad/clk?id=48897031&iu=/4140/ostg.clktrk
> _______________________________________________
> pydev-code mailing list
> pyd...@li...
> https://lists.sourceforge.net/lists/listinfo/pydev-code
>