On Fri, Aug 9, 2013 at 8:19 PM, Jonah Graham <jonah@kichwacoders.com> wrote:
Hi,

I need to solve a similar problem. I have some scripts that are user
editable, but are run by an execfile with some other magic wrapped
around it. In that magic some new globals are added, which seem in a
similar way to how web2py is described in
http://sourceforge.net/p/pydev/feature-requests/534/

What I would like to do is add a new extension point to extend the
behaviour of CompiledModule:setTokens() and have it essentially add
new values to "array" within this if statement?

            //as we will use it for code completion on sources that
map to modules, the __file__ should also
            //be added...
            if (array.size() > 0 && (name.equals("__builtin__") ||
name.equals("builtins"))) {
                array.add(new CompiledToken("__file__", "", "", name,
IToken.TYPE_BUILTIN));
                array.add(new CompiledToken("__name__", "", "", name,
IToken.TYPE_BUILTIN));
                array.add(new CompiledToken("__builtins__", "", "",
name, IToken.TYPE_BUILTIN));
                array.add(new CompiledToken("__dict__", "", "", name,
IToken.TYPE_BUILTIN));
// extension point data insertion would be here[1]
            }

Does that sound like a correct solution? Does that seem generally
useful? If so I can put an initial pull request together for review.

[1] Should the extension point users return AbstractToken, or is there
a subset of that API that should be exposed (e.g. just name). My
preference would be AbstractToken, to allow full range of options for
extension point user.

I think it sounds like a proper solution... when getting that, it may be nice providing the interpreter itself, the shell and the result of manager.getCompletePythonPath(                    nature.getProjectInterpreter(), nature.getRelatedInterpreterManager()) which are in that same context, as PyDev itself could check if web2py is installed when asked for the builtins and if it is it could provide the web2py tokens to support it properly.

The extension point users should probably return a List<IToken>.

Cheers,

Fabio
 

Jonah


On 23 July 2013 12:39, Fabio Zadrozny <fabioz@esss.com.br> wrote:
> On Tue, Jul 23, 2013 at 4:46 AM, Andreas Pakulat <andreas@froglogic.com>
> wrote:
>>
>> Hi Fabio,
>>
>> thanks for the hint about the pycompletionserver.py. Currently we
>> already have a change or two to PyDev and roll our own but we would
>> rather use the official version.
>>
>> Unfortunately our interpreter is embedded into a commandline tool that
>> does not support being used like a normal python interpreter. So thats
>> why  choosing it as an interpreter is not suitable at the moment. What
>> it does is basically load the main script file (i.e. parse it but does
>> not run it immediately), then add a few modules to the python
>> interpreter via C API, import those modules into the global namespace of
>> the main script file and also import several functions into the global
>> namespace. This was done as a convenience for the users but now clearly
>> bites back at us when trying to teach PyDev about these specialties.
>
>
> Ok, so, it's not really a custom interpreter, just a different main... Not
> sure if that's the best approach (I'd probably go toward a more common
> approach of having a different main entry which then would run the script
> after configuring the environment, such as most web frameworks do or maybe a
> custom sitecustomize).
>
> Still, if you bundle it as an executable, it shouldn't be all that hard to
> make it look like a real python interpreter (i.e.: one that could really be
> used as an interpreter -- that way you wouldn't need to change PyDev at all
> and everything should just work). This is probably the preferred approach.
>
> Now, if that's difficult for you, the pycompletionserver.py/importsTipper.py
> adjustment probably requires less fiddling to integrate.
>
> Regarding a better approach within PyDev itself, please create a feature
> request for that in https://sw-brainwy.rhcloud.com/tracker/PyDev/ (this is
> also a request web2py users have -- for which there's a discussed workaround
> detailed at the old tracker:
> http://sourceforge.net/p/pydev/feature-requests/534/).
>
>>
>>
>> Its of course also possible to import those registered modules in the
>> script file or in one that is being imported into the main script.
>>
>> What I ended up doing now which seems to work with 2 caveats (see
>> further down) is that I'm using the extension points
>> pydev_interpreter_new_custom_entries, pydev_modules_observer and the
>> completion extension point. The first one registers the names of the
>> modules we add via the C API as 'builtins', the second one adds tokens
>> to the __builtin__ module for those modules and the functions we import
>> into the global namespace. It also populates our modules with their
>> functions when the PyDev code needs them (a script importing our squish
>> module for example). All this seems to avoid errors being flagged for
>> our Squish API, but does not yet provide the modules and functions for
>> completion. Hence using the completion extension point to support that.
>>
>> The two minor issues with this are:
>>
>> 1. For the builtins registration using the
>> pydev_interpreter_new_custom_entries extension point the users need to
>> remove and re-add the interpreter so that PyDev fetches the builtins
>> again.
>>
>> Maybe there's a way to do the above programatically upon startup of a
>> plugin?
>>
>> 2. When using 'import squish' in a script now a warning is flagged
>> about this shadowing a builtin module which is correct from what PyDev
>> knows but not really happening in the actual interpreter that executes
>> the tests.
>>
>> The second point is not much of a problem as its just a warning and
>> hence does not jump out at the user as much as an error (in particular
>> an error in script code that Squish just generated).
>>
>> Andreas
>>
>>
>> On 2013-07-22 22:39, Fabio Zadrozny wrote:
>> > Hi Andreas,
>> >
>> > Actually, if you really have a custom interpreter and this is added
>> > in its startup, it's strange that it doesn't work already (PyDev
>> > always spawns a shell for the completion and upon startup it'll load
>> > the __builtin__ -- or builtins -- module and do a dir() in it).
>> >
>> > So, if it's custom interpreter, it's mostly a matter of filling the
>> > __builtins__ at the startup... If that's not the case, you could
>> > change PyDev to add some custom code at the module scope at
>> > /org.python.pydev/pysrc/pycompletionserver.py (but you'll have to
>> > maintain your own fork if you choose this path).
>> >
>> > Cheers,
>> >
>> > Fabio
>> >
>> > On Fri, Jul 19, 2013 at 12:15 PM, Andreas Pakulat
>> > <andreas@froglogic.com> wrote:
>> >
>> >> Hi,
>> >>
>> >> I'm currently working on improving our usage of PyDev, in particular
>> >> starting to let PyDev not only syntax highlight but also analyze the
>> >> code our users write. Unfortunately this has one drawback, we use a
>> >> custom interpreter and add several global functions to it (this is
>> >> done
>> >> by having a module for these functions and then importing everything
>> >> from this module by default into the global namespace).
>> >>
>> >> I know about the setting for the PyDev analyzer to not show errors
>> >> on
>> >> certain undefined functions/variables, but the number of functions
>> >> makes
>> >> this single lineedit/preference pretty cumbersome to extend.
>> >>
>> >> I'm wondering wether there's a different way (via code maybe?) to
>> >> teach
>> >> PyDev about a list of functions it should consider to be
>> >> 'predefined',
>> >> similar to the builtin modules?
>> >>
>> >> I'm currently using PyDev 2.6, but could upgrade to 2.7.5 if that
>> >> makes
>> >> it easier to achieve what I want.
>> >>
>> >> Andreas
>> >>
>> >> --
>> >> Andreas Pakulat squish@froglogic.com
>> >> froglogic GmbH - Automated UI and Web Testing
>> >>
>> >>
>> >>
>> >> ------------------------------------------------------------------------------
>> >> See everything from the browser to the database with AppDynamics
>> >> Get end-to-end visibility with application monitoring from
>> >> AppDynamics
>> >> Isolate bottlenecks and diagnose root cause in seconds.
>> >> Start your free trial of AppDynamics Pro today!
>> >>
>> >>
>> >> http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140/ostg.clktrk
>> >> [1]
>> >> _______________________________________________
>> >> pydev-code mailing list
>> >> pydev-code@lists.sourceforge.net
>> >> https://lists.sourceforge.net/lists/listinfo/pydev-code [2]
>> >
>> >
>> >
>> > Links:
>> > ------
>> > [1]
>> >
>> >
>> > http://pubads.g.doubleclick.net/gampad/clk?id=48808831&amp;iu=/4140/ostg.clktrk
>> > [2] https://lists.sourceforge.net/lists/listinfo/pydev-code
>> >
>> >
>> >
>> > ------------------------------------------------------------------------------
>> > See everything from the browser to the database with AppDynamics
>> > Get end-to-end visibility with application monitoring from
>> > AppDynamics
>> > Isolate bottlenecks and diagnose root cause in seconds.
>> > Start your free trial of AppDynamics Pro today!
>> >
>> >
>> > http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140/ostg.clktrk
>> >
>> > _______________________________________________
>> > pydev-code mailing list
>> > pydev-code@lists.sourceforge.net
>> > https://lists.sourceforge.net/lists/listinfo/pydev-code
>>
>> --
>> Andreas Pakulat squish@froglogic.com
>> froglogic GmbH - Automated UI and Web Testing
>>
>>
>> ------------------------------------------------------------------------------
>> See everything from the browser to the database with AppDynamics
>> Get end-to-end visibility with application monitoring from AppDynamics
>> Isolate bottlenecks and diagnose root cause in seconds.
>> Start your free trial of AppDynamics Pro today!
>>
>> http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140/ostg.clktrk
>> _______________________________________________
>> pydev-code mailing list
>> pydev-code@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/pydev-code
>
>
>
> ------------------------------------------------------------------------------
> See everything from the browser to the database with AppDynamics
> Get end-to-end visibility with application monitoring from AppDynamics
> Isolate bottlenecks and diagnose root cause in seconds.
> Start your free trial of AppDynamics Pro today!
> http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140/ostg.clktrk
> _______________________________________________
> pydev-code mailing list
> pydev-code@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/pydev-code
>

------------------------------------------------------------------------------
Get 100% visibility into Java/.NET code with AppDynamics Lite!
It's a free troubleshooting tool designed for production.
Get down to code-level detail for bottlenecks, with <2% overhead.
Download for free and get started troubleshooting in minutes.
http://pubads.g.doubleclick.net/gampad/clk?id=48897031&iu=/4140/ostg.clktrk
_______________________________________________
pydev-code mailing list
pydev-code@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/pydev-code