Re: [Pydev-code] Web2py helper: Adding imports and parsing models with jython scripting

SourceForge Headquarters 1320 Columbia Street Suite 310 San Diego, CA 92101 +1 (858) 422-6466

Hi Fabio, I don't know if you're familiar with web2py, I'll try to
give you an example.

Project in Eclipse: web2py_example

Structure:
---------------------------------------
web2py_example
  src/
    cache/
    controllers/
      default.py
    cron/
    databases/
    errors/
    languages/
    models/
      config.py
      db.py
      db_functions.py
      db_models.py
      db_others.py
      mail.py
      menu.py
      version.py
    modules/
    ...
  .project
  .pydevproject
---------------------------------------

The 'src' folder is linked to the web2py application folder, for
example, src-> c:\web2py\applications\my_application. Some examples of
files:

models/db.py:
------------------------------------------------
# coding: utf8
db = DAL('sqlite://storage.sqlite')
...
------------------------------------------------

models/db_functions.py:
------------------------------------------------
# coding: utf8

DATETIME_FORMAT = "%d/%m/%Y %H:%M:%S"

def get_latest_sample_date():
    maxID = db.sensor_samples.id.max()
...
------------------------------------------------

models/menu.py:
------------------------------------------------
response.title = request.application

if request.function == 'somefunction':
  response.menu = ...
else:
  response.menu = ...
...
------------------------------------------------

controllers/default.py:
------------------------------------------------
@auth.requires_login()
def index():
    time = datetime.now().strftime(DATETIME_FORMAT)
    latest = get_latest_sample_date()
    response.flash = 'Sample message'

    return dict(
        time=time,
        latestSample=latest)
...
------------------------------------------------

On web2py, the code for function 'index' on controllers/default.py is
run when you navigate to http://server/my_application/deafult/index.

Web2py "magically" populates an environment, so some instances like
'request', 'response', 'session', and some modules and classes are
imported. Then, web2py evals all the files in the models/ folder.
Finally, it evals the controllers/default.py file, and calls the
'index' function (the returned dictionary is passed to the
corresponding view, but that doesn't matter right now).

So, what do I want to achieve?

When editing a file in models/ (for example models/db.py), I would
like pydev to know about that DAL was imported from gluon.dal.DAL
(web2py does it automatically).

Then, in models/db_functions.py I would like pydev to know about the
'db' variable, which was created in modles/db.py (web2py evaluates
these files in alphabetical order, so db.py comes befure
db_functions.py).

Also in models/menu.py, I would like pydev to know that request and
response are instances of gluon.globals.Request and
gluon.globals.Response respectively, etc.

When editing the file controlles/default.py, I'd like web2py to know
about 'request', 'response' (and some others), as well as about the
declarations in all the models/ file (in the code example, I'm calling
the 'get_latest_sample_date' function declared in
models/db_functions.py)

Basically, I want to setup the environment in a similar way to what is
done by web2py in gluon.compilyapp, functions 'build_environment' and
'run_models':

-------------------------------------
def build_environment(request, response, session):
    """
    Build the environment dictionary into which web2py files are executed.
    """

    environment = {}
    for key in html.__all__:
        environment[key] = getattr(html, key)

    # Overwrite the URL function with a proxy
    # url function which contains this request.
    environment['URL'] = html._gURL(request)

    for key in validators.__all__:
        environment[key] = getattr(validators, key)
    if not request.env:
        request.env = Storage()
    environment['T'] = translator(request)
    environment['HTTP'] = HTTP
    environment['redirect'] = redirect
    environment['request'] = request
    environment['response'] = response
    environment['session'] = session
    environment['cache'] = Cache(request)
    environment['DAL'] = DAL
    environment['Field'] = Field
    environment['SQLDB'] = SQLDB        # for backward compatibility
    environment['SQLField'] = SQLField  # for backward compatibility
    environment['SQLFORM'] = SQLFORM
    environment['SQLTABLE'] = SQLTABLE
    environment['LOAD'] = LoadFactory(environment)
    environment['local_import'] = \
        lambda name, reload=False, app=request.application:\
        local_import_aux(name,reload,app)
    BaseAdapter.set_folder(os.path.join(request.folder, 'databases'))
    response._view_environment = copy.copy(environment)
    return environment

def run_models_in(environment):
    """
    Runs all models (in the app specified by the current folder)
    It tries pre-compiled models first before compiling them.
    """

    folder = environment['request'].folder
    path = os.path.join(folder, 'compiled')
    if os.path.exists(path):
        for model in listdir(path, '^models_.+\.pyc$', 0):
            restricted(read_pyc(model), environment, layer=model)
    else:
        models = listdir(os.path.join(folder, 'models'), '^\w+\.py$',
                         0)
        for model in models:
            layer = model
            if is_gae:
                code = getcfs(model, model,
                              lambda: compile2(open(model, 'r').read(),layer))
            else:
                code = getcfs(model, model, None)
            restricted(code, environment, layer)

-------------------------------------

So, do you think this can be done just with scripting, or will it
require extending pydev?

Suggestions are welcome, and I think web2py developers would find this
very useful.

Greets.

On Thu, Apr 28, 2011 at 3:32 PM, Fabio Zadrozny <fa...@es...> wrote:
> On Thu, Apr 28, 2011 at 10:18 AM, Álvaro J. Iradier <air...@gm...> wrote:
>> That might work for predefined classes or items, like "request",
>> "response", "session"..., but I want to go further. I want to
>> dinamically parse the files in models/, which define some variables
>> and functions, and I want to make those available too.
>>
>> Is it possible to get this far using the jython scripting, or should I
>> develop a plugin or a patch for pydev?
>>
>
> Can you give an actual example with code on what you want to achieve?
>
> Cheers,
>
> Fabio
>
>
>> Greets.
>>
>> On Thu, Apr 28, 2011 at 3:07 PM, Fabio Zadrozny <fab...@gm...> wrote:
>>> Might be easier providing predefined completions (i.e.: generating
>>> python stub code for web2py -- you can create a Python script that
>>> loads the environment and extracts info as needed).
>>>
>>> See: http://pydev.org/manual_101_interpreter.html#id2 for details
>>>
>>> An example script that converts QScintilla .api files to the
>>> predefined completions is available at:
>>> https://github.com/aptana/Pydev/blob/master/plugins/org.python.pydev.jython/jysrc/convert_api_to_pypredef.py
>>> (this file should also be in your local PyDev)
>>>
>>> Cheers,
>>>
>>> Fabio
>>>
>>> On Thu, Apr 28, 2011 at 9:36 AM, Álvaro J. Iradier <air...@gm...> wrote:
>>>> Hi all,
>>>>
>>>> I'm trying to make a Jython Script for Pydev for better integration
>>>> with web2py development.
>>>>
>>>> Basically, I need some way to access the python interpreter that is
>>>> used for syntax check and code completion from the jython script.
>>>>
>>>> What I would like to do is, when a .py file for a web2py project is
>>>> loaded in the editor, fist, make some imports into the current
>>>> interpreter, for example:
>>>>
>>>> from gluon.globals import Request
>>>> from gluon.globals import Session
>>>> from gluon.globals import Response
>>>>
>>>> then instantiate some of this classes into global variables:
>>>>
>>>> request=Request()
>>>> session=Session()
>>>> response=Response()
>>>>
>>>> to simulate a real request environment.
>>>>
>>>> Finally, I want to "exec" all the files in the models/ folder, in
>>>> order to have the globals declared in there into de interpreter.
>>>>
>>>> Can you guide me into the right path? I'm totally lost looking at the
>>>> code in com.python.pydev.codecompletion... I don't know how to get
>>>> there from the jython script (or even if it's possible).
>>>>
>>>> Thanks very much.
>>>>
>>>> --
>>>>
>>>> (:=================================:)
>>>>  Alvaro J. Iradier Muro - air...@gm...
>>>>
>>>> ------------------------------------------------------------------------------
>>>> WhatsUp Gold - Download Free Network Management Software
>>>> The most intuitive, comprehensive, and cost-effective network
>>>> management toolset available today.  Delivers lowest initial
>>>> acquisition cost and overall TCO of any competing solution.
>>>> http://p.sf.net/sfu/whatsupgold-sd
>>>> _______________________________________________
>>>> pydev-code mailing list
>>>> pyd...@li...
>>>> https://lists.sourceforge.net/lists/listinfo/pydev-code
>>>>
>>>
>>> ------------------------------------------------------------------------------
>>> WhatsUp Gold - Download Free Network Management Software
>>> The most intuitive, comprehensive, and cost-effective network
>>> management toolset available today.  Delivers lowest initial
>>> acquisition cost and overall TCO of any competing solution.
>>> http://p.sf.net/sfu/whatsupgold-sd
>>> _______________________________________________
>>> pydev-code mailing list
>>> pyd...@li...
>>> https://lists.sourceforge.net/lists/listinfo/pydev-code
>>>
>>
>>
>>
>> --
>> (:=================================:)
>>  Alvaro J. Iradier Muro - air...@gm...
>>
>> ------------------------------------------------------------------------------
>> WhatsUp Gold - Download Free Network Management Software
>> The most intuitive, comprehensive, and cost-effective network
>> management toolset available today.  Delivers lowest initial
>> acquisition cost and overall TCO of any competing solution.
>> http://p.sf.net/sfu/whatsupgold-sd
>> _______________________________________________
>> pydev-code mailing list
>> pyd...@li...
>> https://lists.sourceforge.net/lists/listinfo/pydev-code
>>
>
> ------------------------------------------------------------------------------
> WhatsUp Gold - Download Free Network Management Software
> The most intuitive, comprehensive, and cost-effective network
> management toolset available today.  Delivers lowest initial
> acquisition cost and overall TCO of any competing solution.
> http://p.sf.net/sfu/whatsupgold-sd
> _______________________________________________
> pydev-code mailing list
> pyd...@li...
> https://lists.sourceforge.net/lists/listinfo/pydev-code
>

-- 
(:=================================:)
 Alvaro J. Iradier Muro - air...@gm...