#534 Improved support for web2py autocompletion

[Alvaro J. Iradier Muro]

Attached web2py_example.zip is a minimal web2py project.

Project in Eclipse: web2py_example

The 'src' folder is linked to the web2py application folder, for
example, src-> c:\web2py\applications\web2py_example.

Structure:
---------------------------------------
web2py_example
src/
cache/
controllers/
default.py
cron/
databases/
errors/
languages/
models/
config.py
db.py
db_functions.py
menu.py
modules/
...
.project
.pydevproject
---------------------------------------

models/db.py:
------------------------------------------------
# -*- coding: utf-8 -*-

#########################################################################
## This scaffolding model makes your app work on Google App Engine too
#########################################################################
db = DAL('sqlite://storage.sqlite')

from gluon.tools import *
mail = Mail() # mailer
auth = Auth(globals(),db) # authentication/authorization
crud = Crud(globals(),db) # for CRUD helpers using auth
service = Service(globals()) # for json, xml, jsonrpc, xmlrpc, amfrpc
plugins = PluginManager()

------------------------------------------------

models/db_functions.py:
------------------------------------------------
# coding: utf8
DATETIME_FORMAT = "%d/%m/%Y %H:%M:%S"

def web2py_sample_function():
return "Hello"
------------------------------------------------

models/menu.py:
------------------------------------------------
# -*- coding: utf-8 -*-

response.title = request.application

response.menu = [
(T('Index'), False, URL(request.application,'default','index'), [])
]

if request.function == 'index':

response.menu+=[
...
------------------------------------------------

controllers/default.py:
------------------------------------------------
# -*- coding: utf-8 -*-
from datetime import datetime

def index():
time = datetime.now().strftime(DATETIME_FORMAT)
return web2py_sample_function() + " " + time
------------------------------------------------

On web2py, the code for function 'index' on controllers/default.py is
run when you navigate to http://server/web2py_example/deafult/index.

Web2py "magically" populates an environment, so some instances like
'request', 'response', 'session', and some modules and classes are
imported. Then, web2py evals all the files in the models/ folder.
Finally, it evals the controllers/default.py file, and calls the
'index' function (the returned dictionary is passed to the
corresponding view, but that doesn't matter right now).

So, what do I want to achieve?

When editing a file in models/ (for example models/db.py), I would
like pydev to know about that DAL was imported from gluon.dal.DAL
(web2py does it automatically).

Then, in models/db_functions.py I would like pydev to know about the
'db' variable, which was created in modles/db.py (web2py evaluates
these files in alphabetical order, so db.py comes befure
db_functions.py).

Also in models/menu.py, I would like pydev to know that request and
response are instances of gluon.globals.Request and
gluon.globals.Response respectively, etc.

When editing the file controlles/default.py, I'd like web2py to know
about 'request', 'response' (and some others), as well as about the
declarations in all the models/ file (in the code example, I'm calling
the 'web2py_sample_function' function declared in
models/db_functions.py)

Basically, I want to setup the environment in a similar way to what is
done by web2py in gluon.compilyapp, functions 'build_environment' and
'run_models':

-------------------------------------
def build_environment(request, response, session):
"""
Build the environment dictionary into which web2py files are executed.
"""

environment = {}
for key in html.__all__:
environment[key] = getattr(html, key)

# Overwrite the URL function with a proxy
# url function which contains this request.
environment['URL'] = html._gURL(request)

for key in validators.__all__:
environment[key] = getattr(validators, key)
if not request.env:
request.env = Storage()
environment['T'] = translator(request)
environment['HTTP'] = HTTP
environment['redirect'] = redirect
environment['request'] = request
environment['response'] = response
environment['session'] = session
environment['cache'] = Cache(request)
environment['DAL'] = DAL
environment['Field'] = Field
environment['SQLDB'] = SQLDB # for backward compatibility
environment['SQLField'] = SQLField # for backward compatibility
environment['SQLFORM'] = SQLFORM
environment['SQLTABLE'] = SQLTABLE
environment['LOAD'] = LoadFactory(environment)
environment['local_import'] = \ lambda name, reload=False, app=request.application:\ local_import_aux(name,reload,app)
BaseAdapter.set_folder(os.path.join(request.folder, 'databases'))
response._view_environment = copy.copy(environment)
return environment

def run_models_in(environment):
"""
Runs all models (in the app specified by the current folder)
It tries pre-compiled models first before compiling them.
"""

folder = environment['request'].folder
path = os.path.join(folder, 'compiled')
if os.path.exists(path):
for model in listdir(path, '^models_.+\.pyc$', 0):
restricted(read_pyc(model), environment, layer=model)
else:
models = listdir(os.path.join(folder, 'models'), '^\w+\.py$',
0)
for model in models:
layer = model
if is_gae:
code = getcfs(model, model,
lambda: compile2(open(model, 'r').read(),layer))
else:
code = getcfs(model, model, None)
restricted(code, environment, layer)

-------------------------------------

Obviously, the web2py source folder must be somewhere in the PYTHONPATH, in order to do the propder imports.

What do you think it's the better approach? Predefined completions? Jython scripting? Or is it harder?

Thanks.

[Alvaro J. Iradier Muro]

Minimal web2py example project

[Fabio Zadrozny]

Ok, took a quick look and found a quick hack :)

You can do the following:

Edit (in your local copy of PyDev) the file eclipse\plugins\org.python.pydev__xxxx\PySrc\importsTipper.py the function GenerateTip.

Search for the comment: #print_ >> open('temp.txt', 'w'), f

and right after it, add the code below... (changing the path to r.folder to be a valid path in your machine as well as the D:\temp\err.txt).

....if data == '__builtin__': #Note that for Python 3.x, it'd need to use builtins instead.
........try:
............import __builtin__
............from gluon.compileapp import build_environment
............from gluon.globals import Request, Response, Session
............r = Request()
............r.folder = r'd:\temp' #To build the environment a valid folder must be passed!
............env = build_environment(r, Response(), Session())
............for key, v in env.items():
................setattr(__builtin__, key, v)
........except:
............import traceback;traceback.print_exc(file=open(r'D:\temp\err.txt', 'w'))

Then, restart eclipse and check if all the 'global' tokens are there for web2py :)

If it's not, please take a look if there's something in err.txt and add it to this feature request.

[SourceForge Robot]

This Tracker item was closed automatically by the system. It was
previously set to a Pending status, and the original submitter
did not respond within 15 days (the time period specified by
the administrator of this Tracker).

[Alvaro J. Iradier Muro]

I've slightly modified the proposed code. It was working well, but I was missing the definitions in the models/ folder, which in web2py are also "automagically" imported into the environment.

So, this is the current code. In imports tipper, after the comment: #print_ >> open('temp.txt', 'w'), f, I add:

GenerateWeb2pyCompletions(data)

and define this function as:

def GenerateWeb2pyCompletions(data):
if data == '__builtin__': #Note that for Python 3.x, it'd need to use builtins instead.
env = dict()
try:
import __builtin__
from gluon.compileapp import build_environment, run_models_in
from gluon.globals import Request, Response, Session

r = Request()

#To build the environment a valid folder must be passed!
r.folder = r'/path/to/my/web2py/project/folder'

#Necessary for run_models_in
r.env.http_host = 'localhost'
r.application = 'init'
r.controller = 'default'
r.function = 'index'

env = build_environment(r, Response(), Session())
run_models_in(env)
except:
import traceback
traceback.print_exc(file=open(r'/tmp/err.txt', 'w'))
for key, v in env.items():
setattr(__builtin__, key, v)

Notice I've added a call to "run_models_in" function from web2py. It needs that r.env.http_host, r.application, r.controller, and r.function are set, as well as the real folder for the environment (in the original proposal, /tmp was used instead of /path/to/my/web2py/project/folder).

For the application, controller, and function, it looks like 'init', 'default', and 'index' work perfect, as well as using 'localhost' for r.env.http_host.

What I would need to make this perfect is a way to configure the application folder in a project independent way. Is this code executed globally, for all projects, or on a project basis? If it's executed for every project, what about adding an option to select a source folder as "web2py project folder", and run this code on that folder?

Thanks very much.

[Alvaro J. Iradier Muro]

Sorry for the identation, I'm including the contents of the GenerateWeb2pyCompletions again:

def GenerateWeb2pyCompletions(data):
....if data == '__builtin__': #Note that for Python 3.x, it'd need to use builtins instead.
........env = dict()
........try:
............import __builtin__
............from gluon.compileapp import build_environment, run_models_in
............from gluon.globals import Request, Response, Session

............r = Request()

............#To build the environment a valid folder must be passed!
............r.folder = r'/home/airadier/klnetcenter/klnetcenter'

............#Necessary for run_models_in
............r.env.http_host = 'localhost'
............r.application = 'init'
............r.controller = 'default'
............r.function = 'index'

............env = build_environment(r, Response(), Session())
............run_models_in(env)
........except:
............import traceback
............traceback.print_exc(file=open(r'/tmp/err.txt', 'w'))
........for key, v in env.items():
............setattr(__builtin__, key, v)

[liturgist]

Is this still the best solution for Pydev/web2py developers?

Would it make sense to integrate this into the Pydev release package? Should this be a parameter preferences setting?

[Fabio Zadrozny]

Yes', I believe this is still the best solution... I still have to integrate it better in PyDev (just didn't have the time to do it).

[Anonymous]

Is there a way to set the web2py project folder on a per project basis as requested by airadier?

[Anonymous]

^ tried that, but I still have undefined variables on response, T, auth, request, db, service and crud on default.py in the welcome project.

Tried the suggested solution on Liclipse 1.2.1 (win64). The full file path seems to be plugins\org.python.pydev_xxx\pysrc \ _pydev_imports_tipper.py

I have done what Alvaro suggested but no success.
I use the web2py source package, and created the pydev project in the root directory of web2py. It understands the gluon explicit imports but not the "automagical" ones.