pydev-cvs Mailing List for PyDev for Eclipse (Page 304)
Brought to you by:
fabioz
You can subscribe to this list here.
2004 |
Jan
|
Feb
(4) |
Mar
(48) |
Apr
(56) |
May
(64) |
Jun
(27) |
Jul
(66) |
Aug
(81) |
Sep
(148) |
Oct
(194) |
Nov
(78) |
Dec
(46) |
---|---|---|---|---|---|---|---|---|---|---|---|---|
2005 |
Jan
(125) |
Feb
(126) |
Mar
(163) |
Apr
(133) |
May
(115) |
Jun
(307) |
Jul
(387) |
Aug
(417) |
Sep
(283) |
Oct
(148) |
Nov
(45) |
Dec
(53) |
2006 |
Jan
(240) |
Feb
(200) |
Mar
(267) |
Apr
(231) |
May
(245) |
Jun
(361) |
Jul
(142) |
Aug
(12) |
Sep
(210) |
Oct
(99) |
Nov
(7) |
Dec
(30) |
2007 |
Jan
(161) |
Feb
(511) |
Mar
(265) |
Apr
(74) |
May
(147) |
Jun
(151) |
Jul
(94) |
Aug
(68) |
Sep
(98) |
Oct
(144) |
Nov
(26) |
Dec
(36) |
2008 |
Jan
(98) |
Feb
(107) |
Mar
(199) |
Apr
(113) |
May
(119) |
Jun
(112) |
Jul
(92) |
Aug
(71) |
Sep
(101) |
Oct
(16) |
Nov
|
Dec
|
From: Fabio Z. <fa...@us...> - 2004-09-14 17:42:27
|
Update of /cvsroot/pydev/org.python.pydev/src/org/python/pydev/editor/refactoring In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv9579/src/org/python/pydev/editor/refactoring Added Files: PyRefactoring.java Log Message: Code completion improvements. Starting refactoring integration with bicycle repair man. --- NEW FILE: PyRefactoring.java --- /* * Created on Sep 14, 2004 * * @author Fabio Zadrozny */ package org.python.pydev.editor.refactoring; /** * This class is used to make the refactorings. * * The design is basically: handle the actions and pass them to the * python server (that should be using bicycle repair man). * * Later, this might be changed as long as the interface provided is * the same. * * @author Fabio Zadrozny */ public class PyRefactoring { } |
From: Fabio Z. <fa...@us...> - 2004-09-14 17:42:27
|
Update of /cvsroot/pydev/org.python.pydev/PySrc/ThirdParty/brm In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv9579/PySrc/ThirdParty/brm Added Files: README DESIGN.html ChangeLog AUTHORS COPYING __init__.py NEWS Log Message: Code completion improvements. Starting refactoring integration with bicycle repair man. --- NEW FILE: DESIGN.html --- <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> <html> <head> <title>Bicyclerepairman Design</title> </head> <body> <h1>Bicyclerepairman Design</h1> This information is correct at 27th Jan 2004. <h2> Bike Component Interface </h2> <pre> import bike ctx = bike.init() ctx.renameByCoordinates("/home/joe/src/foo/bah.py",52,8,"bah") ctx.save() ctx.undo() </pre> <h2>File System Dependency:</h2> Bicyclerepair man operates off of the file system. Modules are loaded from the file system, and saved back after a refactoring. Integration with IDEs currently relies on the IDE ensuring that the files on disk are up-to-date before performing a query or refactoring, and is responsible for reloading the modified files after the operation. <!-- <h2>Example: - findDefinition</h2> Client code calls findDefinition via the context object in bikefacade.py. The client passes in a filename and coordinates of a reference she wants to find the definition of. The bikefacade module sets a search path (based on PYTHONPATH and the position of the file in a package hierarchy) and then calls the findDefinition module in the query package. The findDefinition module loads the file into a Module object via the getModule() call in the parsing package. --> <h2>Parsing and Fastparsing - the <em>parser</em> package</h2> <p> BRM originally did all its parsing using the Python compiler package (see python docs). This parser generates very accurate AST. Unfortunately this proved to be far too slow for interactive use, and so I built a seperate parser which does a basic job of parsing, but very fast. BRM uses this basic parser to locate and navigate round the code, and then applies the compiler package to small bits of code where necessary. </p> <p> The fastparser module in the parsing package operates on a single module at a time. It generates a simple tree containing Class and Function nodes, with a Module node at the root. This tree is called the 'fastparserast', and it's nodes (Module,Class,Function) are defined in the fastparserast module. Each fastparserast node contains a pointer to the start and end lines of the node in the source text, and basic information (e.g. name). It also contains methods for navigating between nodes (e.g. getChildNodes(), getParent() </p> <h3>BRM Tricks - masking the code</h3> <p> The fastparser is very fast because it leverages the power of the regex package, and has been carefully profiled and optimised. Before parsing a file it applies a number of regexes over the source to mask out all the comments and strings. This then allows it to scan the source very quickly for functions and classes using python keywords ('class', 'def') without fear of getting confused by occurences of keywords in strings. It builds up a tree of functions and classes in a module, storing their start and end positions (lines) in the source text. </p> <h3> BRM Tricks2 - logical lines and doctored lines</h3> <p> As mentioned above, the compiler module is very accurate but very slow. Because of this, BRM uses it sparingly, choosing only to parse lines of code when it is reasonably confident that the line contains something important. </p> <p> In order to parse a single line of source code, the line must sometimes be doctored to make it parsable. Take for example the following: <pre> def foo(a, b, c): </pre> This line cannot be parsed on its own, but may contain some interesting code. BRM overcomes this constraint via the makeLineParseable() function in the parserutils module. This function decorates the line with noops that make it parsable. For the above example it generates the following code: <pre> def foo(a, b, c): pass </pre> Which parses fine. </p> <p> The other problem with parsing single lines of code is that python statements often span multiple lines. e.g. <pre> myref = foo(arg1, arg2, arg3, arg4) </pre> To combat this problem, BRM has a neat function called generateLogicalLines() (also on the parserutils module). This function takes a list of (masked) physical lines as input, and generates complete 'logical' lines as output. <h2>Querying the code - The <em>query</em> Package</h2> <p> As mentioned above, the relative slow speed of the compiler module prohibits BRM from building up an AST of the entire file and then navigating it. Instead the query package uses text searches to narrow down the search and then parses single 'logical lines' using the compiler. </p> <h3>logical lines and doctored lines</h3> <p> In order to parse a single line of source code, the line must sometimes be doctored to make it parsable. Take for example the following: <pre> a = myfunc(arg1, arg2, arg3, arg4) </pre> <pre> def foo(a, b, c): </pre> The first problem is multiline lines. </p> <h3>Example - findDefinition</h3> <p> For example, when searching for the definition of a reference (see findDefinition.py), BRM first takes the input coordinates and locates the fastparserast node denoting the class/function scope containing the reference (by checking the coords against the start/end coords of the fastparserast nodes). This provides the initial search scope for the query. </p> <p> BRM then obtains the sourcecode of the scope with strings/comments masked out. It searches </p> <p> </p> <p> </p> <p> </p> <hr> <address><a href="mailto:pd...@us...">Phil Dawes</a></address> <!-- Created: Mon Jan 26 17:43:32 GMT 2004 --> <!-- hhmts start --> Last modified: Mon Jan 26 18:40:51 GMT 2004 <!-- hhmts end --> </body> </html> --- NEW FILE: COPYING --- COPYRIGHT AND PERMISSION NOTICE Copyright (c) 2000 by Shae Erisson <sh...@la...> Copyright (c) 2001 by Phil Dawes <pd...@us...> All rights reserved. Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, provided that the above copyright notice(s) and this permission notice appear in all copies of the Software and that both the above copyright notice(s) and this permission notice appear in supporting documentation. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT OF THIRD PARTY RIGHTS. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR HOLDERS INCLUDED IN THIS NOTICE BE LIABLE FOR ANY CLAIM, OR ANY SPECIAL INDIRECT OR CONSEQUENTIAL DAMAGES, OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. Except as contained in this notice, the name of a copyright holder shall not be used in advertising or otherwise to promote the sale, use or other dealings in this Software without prior written authorization of the copyright holder. --- NEW FILE: ChangeLog --- 2004-02-11 Phil Dawes <pd...@us...> * bike/query/getTypeOf.py: rewrote resolveImportedModuleOrPackage to use purely getModuleOrPackageUsingFQN searches. (doesnt use getTypeOf Root searching any more) * bike/query/common.py: rewrote getLogicalLine to handle multilines that parse seperately. (But was submitted by Peter Astrand) 2004-02-10 Phil Dawes <pd...@us...> * bike/query/getTypeOf.py: Added functionality to search from the current directory if looking for an import and scope is a module 2004-02-10 Phil Dawes <pd...@us...> * bike/refactor/moveToModule.py: Added handling of individual 'from a.foo import theFunction'. [...1989 lines suppressed...] 2000-09-01 18:22 jhermann * README: Additions by shae 2000-08-19 01:35 eris * README: new dir structure by snibril aka Jürgen Herrman first stab at a tree matcher in common.py 2000-08-01 01:44 jhermann * README: Added a readme dummy --- NEW FILE: README --- Bicycle Repair Man - a Python Refactoring Browser ================================================= Copyright (c) 2000 by Shae Erisson <sh...@la...> Copyright (c) 2001-3 by Phil Dawes <pd...@us...> All rights reserved, see COPYING for details. $Id: README,v 1.7 2003/08/24 19:48:43 pdawes Exp $ ----------------------------------------------------------------------------- Bicycle Repair Man is the Python Refactoring Browser, helping Pythonistas everywhere glide over the gory details of refactoring their code. Watch him extract jumbled code into well ordered classes. Gasp, as he renames all occurrences of a method. Thank You, Bicycle Repair Man! execute ./testall.py to run all the tests see INSTALL for installation instructions (uses distutils). see README.idle, README.emacs etc.. for instructions on how to integrate bicyclerepairman into supported IDEs. ----------------------------------------------------------------------------- What's Python? Python is a programming language. To find out more about it, go to the Python Homepage at http://www.python.org/ What's a Refactoring Browser? A Refactoring Browser is an editor that automates Refactorings. The first Refactoring Browser was written by Dr. Don Roberts and Dr. John Brant at the University of Illinois in Urbana-Champagne. Dr. Don Roberts wrote his Ph.D. thesis on the design and implementation of the Refactoring Browser. For more detail, read the aforementioned thesis at http://st-www.cs.uiuc.edu/~droberts/thesis.pdf What's a Refactoring? A Refactoring is a behaviour preserving change to source code. Some Refactorings are RenameVariable, RenameClass, RenameMethod, PullUpMethod, and PushDownVariable. Lots of people say it's very easy to just type a different name in where your class, method, or variable is defined. That's not always a refactoring though. The Refactoring Browser is smart enough to rename every reference to your class, method or variable. If you've ever renamed a variable and broken classes in widely scattered parts of your system, you might be happier using a Refactoring Browser. A Refactoring Browser operates on any of method, function, class, or variable. It can add, delete, rename, move up down or sideways, inline and abstract. There are some operations that are specific to one of the three types, such as abstracting a variable into accessors, or turing several lines of code into a separate method. For more information on Refactoring, check out the websites of Martin Fowler at http://www.martinfowler.com/ and his Refactoring Site at http://www.refactoring.com/ . Why Bicycle Repair Man? The Bicycle Repair Man was a superhero in a Monty Python skit, his special power was repairing bicycles. --- NEW FILE: NEWS --- Main highlights of each new version. See ChangeLog for a more detailed description of changes. Version 0.9 ----------- This version removes the requirement to load files into bicyclerepairman before refactoring. Instead, it searches the PYTHONPATH (and the path in which the file being queried is in). This allows 'findDefinition' queries to be made on references (e.g. classes, methods) where the definition is in the python library. Version 0.8 ----------- This release improves on the internal type-deduction engine to handle variables and attributes. To reflect this, 'rename' now works on variables and attributes, in addition to methods, functions and classes. This release also adds vim support (thanks to Marius Gedminas and Matt Yeates for this) Version 0.7 ----------- This release includes a totally re-written type querying engine, which is much faster and paves the way for new refactorings. It also adds the 'FindReferences' and 'FindDefinition' query to emacs and idle. Version 0.6 ----------- This release adds undo functionality to the mix, and beefs up the idle and emacs integration so that code is automatically imported into brm when you load a file into a buffer. Version 0.5 ----------- This release adds the ExtractMethod refactoring Version 0.4 ----------- This release adds support for IDLE (see README.idle), and fixes a few bugs. The CLI and GUI interfaces are now deprecated and have been removed from this release. Version 0.3 ----------- This release adds the RenameClass and RenameFunction refactorings. It also contains the initial xemacs integration functionality (see README.xemacs). Version 0.2 ----------- This release adds a simple GUI for renaming methods - run bikegui.py after installation. There's also some upgrades to pyxmi. It should now be able to generate xmi to model all generalizations (including cross-package ones). N.B. pyxmi.py is now called py2xmi.py. See changelog for reasons! Version 0.1 ----------- This is the first release of Bicycle Repair Man. It requires python version 2.2 and above. This version supports a partial implementation of the RenameMethod refactoring through a command line interface. It automatically renames the method and references to the method that it can deduce. It asks you about method references it can't deduce the instance type of. This software should be considered alpha, and may damage your source files - backup your sources before use! See INSTALL for installation and usage instructions. N.B. This package also contains pyxmi - a little python -> xmi tool I cobbled together out of the bicycle repair man parsing package. It generates an xmi file out of a source-file or package-structure, suitable for loading into the argouml tool. See http://argouml.tigris.org. --- NEW FILE: AUTHORS --- Authors ------- Phil Dawes <pd...@us...> - Current Maintainer and Main Author Shae Erisson <sh...@la...> - Original Maintainer The following people have contributed code, bugfixes and patches: Jürgen Hermann <jh...@we...> Canis Lupus Syver Enstad <syv...@on...> Windows emacs patches Mathew Yeates <ma...@co...> VIM support and bug fixes Marius Gedminas <mg...@de...> More VIM support François Pinard <pi...@ir...> Pymacs + help with emacs integration Ender <en...@ob...> Jonathan <jon...@in...> Steve <the...@ya...> Peter Astrand <pe...@ce...> See ChangeLog for more details. --- NEW FILE: __init__.py --- |
From: Fabio Z. <fa...@us...> - 2004-09-14 17:42:21
|
Update of /cvsroot/pydev/org.python.pydev/PySrc/Refactoring In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv9579/PySrc/Refactoring Added Files: Refactoring.py MatchesDialog.py __init__.py MatchesForm.py MatchesForm.ui Log Message: Code completion improvements. Starting refactoring integration with bicycle repair man. --- NEW FILE: MatchesForm.py --- # -*- coding: utf-8 -*- # Form implementation generated from reading ui file '/home/detlev/Development/Python/Eric/eric3/Refactoring/MatchesForm.ui' # # Created: Mit Aug 13 21:23:54 2003 # by: The PyQt User Interface Compiler (pyuic) 3.7 # # WARNING! All changes made in this file will be lost! import sys from qt import * class MatchesForm(QDialog): def __init__(self,parent = None,name = None,modal = 0,fl = 0): QDialog.__init__(self,parent,name,modal,fl) if not name: self.setName("MatchesForm") self.setSizeGripEnabled(1) MatchesFormLayout = QVBoxLayout(self,6,6,"MatchesFormLayout") self.matchesList = QListView(self,"matchesList") self.matchesList.addColumn(self.__tr("Filename")) self.matchesList.addColumn(self.__tr("Line")) self.matchesList.addColumn(self.__tr("Confidence [%]")) self.matchesList.setAllColumnsShowFocus(1) self.matchesList.setShowSortIndicator(1) MatchesFormLayout.addWidget(self.matchesList) layout1 = QHBoxLayout(None,0,6,"layout1") spacer = QSpacerItem(40,20,QSizePolicy.Expanding,QSizePolicy.Minimum) layout1.addItem(spacer) self.closeButton = QPushButton(self,"closeButton") layout1.addWidget(self.closeButton) spacer_2 = QSpacerItem(40,20,QSizePolicy.Expanding,QSizePolicy.Minimum) layout1.addItem(spacer_2) MatchesFormLayout.addLayout(layout1) self.languageChange() self.resize(QSize(649,480).expandedTo(self.minimumSizeHint())) self.clearWState(Qt.WState_Polished) self.connect(self.closeButton,SIGNAL("clicked()"),self,SLOT("close()")) self.connect(self.matchesList,SIGNAL("doubleClicked(QListViewItem*)"),self.handleDoubleClicked) def languageChange(self): self.setCaption(self.__tr("Refactoring Matches")) self.matchesList.header().setLabel(0,self.__tr("Filename")) self.matchesList.header().setLabel(1,self.__tr("Line")) self.matchesList.header().setLabel(2,self.__tr("Confidence [%]")) self.closeButton.setText(self.__tr("&Close")) def handleDoubleClicked(self,a0): print "MatchesForm.handleDoubleClicked(QListViewItem*): Not implemented yet" def __tr(self,s,c = None): return qApp.translate("MatchesForm",s,c) if __name__ == "__main__": a = QApplication(sys.argv) QObject.connect(a,SIGNAL("lastWindowClosed()"),a,SLOT("quit()")) w = MatchesForm() a.setMainWidget(w) w.show() a.exec_loop() --- NEW FILE: MatchesDialog.py --- # -*- coding: utf-8 -*- # Copyright (c) 2003 Detlev Offenbach <de...@di...> # from qt import * from MatchesForm import MatchesForm """ Module implementing a dialog to show matching references/definitions. """ class MatchesDialog(MatchesForm): """ Class implementing a dialog to show matching references/definitions. """ def __init__(self,ui,parent = None,name = None,modal = 0,fl = 0): """ Constructor @param ui reference to the UI object @param parent parent of this dialog (QWidget) @param name name of this dialog (string or QString) @param modal flag indicating a modal window (boolean) @param fl window flags """ MatchesForm.__init__(self,parent,name,modal,fl) self.ui = ui self.matchesList.setSorting(0) self.matchesList.setColumnAlignment(1, Qt.AlignRight) self.matchesList.setColumnAlignment(2, Qt.AlignRight) dummy = self.trUtf8("Dummy") def handleDoubleClicked(self, itm): """ Private slot to handle the DoubleClicked signal of the list. """ lineno = int(str(itm.text(1))) fn = str(itm.text(0)) self.ui.getViewManager().displayPythonFile(fn, lineno) def addEntry(self, ref): """ Public slot to add a reference to the listview. """ itm = QListViewItem(self.matchesList, ref.filename, " %5d" % ref.lineno, " %5d" % ref.confidence) --- NEW FILE: MatchesForm.ui --- <!DOCTYPE UI><UI version="3.1" stdsetdef="1"> <class>MatchesForm</class> <widget class="QDialog"> <property name="name"> <cstring>MatchesForm</cstring> </property> <property name="geometry"> <rect> <x>0</x> <y>0</y> <width>649</width> <height>480</height> </rect> </property> <property name="caption"> <string>Refactoring Matches</string> </property> <property name="sizeGripEnabled"> <bool>true</bool> </property> <vbox> <property name="name"> <cstring>unnamed</cstring> </property> <widget class="QListView"> <column> <property name="text"> <string>Filename</string> </property> <property name="clickable"> <bool>true</bool> </property> <property name="resizable"> <bool>true</bool> </property> </column> <column> <property name="text"> <string>Line</string> </property> <property name="clickable"> <bool>true</bool> </property> <property name="resizable"> <bool>true</bool> </property> </column> <column> <property name="text"> <string>Confidence [%]</string> </property> <property name="clickable"> <bool>true</bool> </property> <property name="resizable"> <bool>true</bool> </property> </column> <property name="name"> <cstring>matchesList</cstring> </property> <property name="allColumnsShowFocus"> <bool>true</bool> </property> <property name="showSortIndicator"> <bool>true</bool> </property> </widget> <widget class="QLayoutWidget"> <property name="name"> <cstring>layout1</cstring> </property> <hbox> <property name="name"> <cstring>unnamed</cstring> </property> <spacer> <property name="name"> <cstring>spacer1</cstring> </property> <property name="orientation"> <enum>Horizontal</enum> </property> <property name="sizeType"> <enum>Expanding</enum> </property> <property name="sizeHint"> <size> <width>40</width> <height>20</height> </size> </property> </spacer> <widget class="QPushButton"> <property name="name"> <cstring>closeButton</cstring> </property> <property name="text"> <string>&Close</string> </property> </widget> <spacer> <property name="name"> <cstring>spacer2</cstring> </property> <property name="orientation"> <enum>Horizontal</enum> </property> <property name="sizeType"> <enum>Expanding</enum> </property> <property name="sizeHint"> <size> <width>40</width> <height>20</height> </size> </property> </spacer> </hbox> </widget> </vbox> </widget> <connections> <connection> <sender>closeButton</sender> <signal>clicked()</signal> <receiver>MatchesForm</receiver> <slot>close()</slot> </connection> <connection> <sender>matchesList</sender> <signal>doubleClicked(QListViewItem*)</signal> <receiver>MatchesForm</receiver> <slot>handleDoubleClicked(QListViewItem*)</slot> </connection> </connections> <slots> <slot>handleDoubleClicked(QListViewItem*)</slot> </slots> <layoutdefaults spacing="6" margin="6"/> </UI> --- NEW FILE: __init__.py --- # -*- coding: utf-8 -*- # Copyright (c) 2003 Detlev Offenbach <de...@di...> # """ Package implementing the refactoring interface module of eric3. This package implements just the interface for the eric3 IDE. The refactoring is done using the Bicyle Repair Man (brm) software (<a href="http://bicyclerepair.sourceforge.net">http://bicyclerepair.sourceforge.net</a>). """ --- NEW FILE: Refactoring.py --- # -*- coding: utf-8 -*- # Copyright (c) 2003 Detlev Offenbach <de...@di...> # """ Module implementing the refactoring interface to brm. """ import os import sys sys.path.insert(1, os.path.join(os.path.dirname(sys.argv[0]), "ThirdParty", "brm")) import ThirdParty.brm.bike as bike from qt import * from MatchesDialog import MatchesDialog import Utilities import Preferences BRM_VERSION_STR = '0.9 cvs20040211' class SilentLogger: """ Class implementing a logger that doesn't log anything. """ def write(*args): """ Public method to write the arguments. """ pass class Refactoring(QObject): """ Class implementing the refactoring interface to brm. """ def __init__(self, parent = None, *args): """ Constructor @param parent parent (QObject) @param *args arguments passed on to QObject """ QObject.__init__(*(self, parent) + args) self.ui = parent self.projectpath = '' self.projectopen = 0 self.refreshing = 0 self.matchesDialog = None self.init() def initActions(self): """ Public method to define the refactoring actions. """ self.resetBRMAct = QAction(self.trUtf8('Reset'), self.trUtf8('Re&set'),0, self) self.resetBRMAct.setStatusTip(self.trUtf8('Reset the refactoring machine')) self.resetBRMAct.setWhatsThis(self.trUtf8( """<b>Reset</b>""" """<p>Reset the refactoring machine.</p>""" )) self.connect(self.resetBRMAct,SIGNAL('activated()'),self.handleReset) self.queryReferencesAct = QAction(self.trUtf8('Find references'), self.trUtf8('Find &References'),0, self) self.queryReferencesAct.setStatusTip(self.trUtf8('Find references of the highlighted item')) self.queryReferencesAct.setWhatsThis(self.trUtf8( """<b>Find references</b>""" """<p>Find references to the highlighted class, method, function or variable.</p>""" )) self.connect(self.queryReferencesAct,SIGNAL('activated()'),self.handleQueryReferences) self.queryDefinitionAct = QAction(self.trUtf8('Find definition'), self.trUtf8('Find &Definition'),0, self) self.queryDefinitionAct.setStatusTip(self.trUtf8('Find definition of the highlighted item')) self.queryDefinitionAct.setWhatsThis(self.trUtf8( """<b>Find definition</b>""" """<p>Find the definition to the highlighted class, method, function or variable.</p>""" )) self.connect(self.queryDefinitionAct,SIGNAL('activated()'),self.handleQueryDefinition) self.refactoringRenameAct = QAction(self.trUtf8('Rename'), self.trUtf8('&Rename'),0, self) self.refactoringRenameAct.setStatusTip(self.trUtf8('Rename the highlighted item')) self.refactoringRenameAct.setWhatsThis(self.trUtf8( """<b>Rename</b>""" """<p>Rename the highlighted class, method, function or variable.</p>""" )) self.connect(self.refactoringRenameAct,SIGNAL('activated()'),self.handleRename) self.refactoringExtractMethodAct = QAction(self.trUtf8('Extract method'), self.trUtf8('Extract &Method'),0, self) self.refactoringExtractMethodAct.setStatusTip(self.trUtf8('Extract the highlighted area as a method')) self.refactoringExtractMethodAct.setWhatsThis(self.trUtf8( """<b>Extract method</b>""" """<p>Extract the highlighted area as a method or function.</p>""" )) self.connect(self.refactoringExtractMethodAct,SIGNAL('activated()'),self.handleExtractMethod) self.refactoringInlineLocalVariableAct = QAction(self.trUtf8('Inline local variable'), self.trUtf8('&Inline Local Variable'),0, self) self.refactoringInlineLocalVariableAct.setStatusTip(self.trUtf8('Inlines the selected local variable')) self.refactoringInlineLocalVariableAct.setWhatsThis(self.trUtf8( """<b>Inline local variable</b>""" """<p>Inlines the selected local variable.</p>""" )) self.connect(self.refactoringInlineLocalVariableAct,SIGNAL('activated()'),self.handleInlineLocalVariable) self.refactoringExtractLocalVariableAct = QAction(self.trUtf8('Extract local variable'), self.trUtf8('Extract Local &Variable'),0, self) self.refactoringExtractLocalVariableAct.setStatusTip(self.trUtf8('Extract the highlighted area as a local variable')) self.refactoringExtractLocalVariableAct.setWhatsThis(self.trUtf8( """<b>Extract local variable</b>""" """<p>Extract the highlighted area as a local variable.</p>""" )) self.connect(self.refactoringExtractLocalVariableAct,SIGNAL('activated()'),self.handleExtractLocalVariable) self.refactoringUndoAct = QAction(self.trUtf8('Undo'), self.trUtf8('&Undo'),0, self) self.refactoringUndoAct.setStatusTip(self.trUtf8('Undo refactorings')) self.refactoringUndoAct.setWhatsThis(self.trUtf8( """<b>Undo</b>""" """<p>Undo the performed refactorings.</p>""" )) self.connect(self.refactoringUndoAct,SIGNAL('activated()'),self.handleUndo) self.refactoringMoveClassAct = QAction(self.trUtf8('Move Class'), self.trUtf8('Move &Class'),0, self) self.refactoringMoveClassAct.setStatusTip(self.trUtf8('Move class to new module')) self.refactoringMoveClassAct.setWhatsThis(self.trUtf8( """<b>Move class</b>""" """<p>Moves the class pointed to by the cursor to a new module.</p>""" )) self.connect(self.refactoringMoveClassAct,SIGNAL('activated()'),self.handleMoveClass) self.refactoringMoveFunctionAct = QAction(self.trUtf8('Move Function'), self.trUtf8('Move &Function'),0, self) self.refactoringMoveFunctionAct.setStatusTip(self.trUtf8('Move function to new module')) self.refactoringMoveFunctionAct.setWhatsThis(self.trUtf8( """<b>Move function</b>""" """<p>Moves the function pointed to by the cursor to a new module.</p>""" )) self.connect(self.refactoringMoveFunctionAct,SIGNAL('activated()'),self.handleMoveFunction) # disable some action until the code behind them works reliably self.refactoringMoveClassAct.setEnabled(0) self.refactoringMoveFunctionAct.setEnabled(0) def initMenu(self): """ Public slot to initialize the refactoring menu. @return the menu generated (QPopupMenu) """ self.menuitems = [] menu = QPopupMenu(self.ui) lbl = QLabel('Bicycle Repair Man', menu) lbl.setFrameStyle( QFrame.Panel | QFrame.Sunken ) lbl.setAlignment(Qt.AlignHCenter) font = lbl.font() font.setBold(1) lbl.setFont(font) menu.insertItem(lbl) menu.insertTearOffHandle() smenu = QPopupMenu(self.ui) self.queryReferencesAct.addTo(smenu) self.queryDefinitionAct.addTo(smenu) self.menuitems.append(menu.insertItem(self.trUtf8("&Query"), smenu)) smenu = QPopupMenu(self.ui) self.refactoringRenameAct.addTo(smenu) smenu.insertSeparator() self.refactoringExtractMethodAct.addTo(smenu) smenu.insertSeparator() self.refactoringInlineLocalVariableAct.addTo(smenu) self.refactoringExtractLocalVariableAct.addTo(smenu) smenu.insertSeparator() self.refactoringMoveClassAct.addTo(smenu) self.refactoringMoveFunctionAct.addTo(smenu) smenu.insertSeparator() self.refactoringUndoAct.addTo(smenu) self.menuitems.append(menu.insertItem(self.trUtf8("&Refactoring"), smenu)) menu.insertSeparator() self.resetBRMAct.addTo(menu) self.menu = menu return menu def init(self): """ Private slot to handle the Reset action. """ self.brmctx = bike.init() self.logging = Preferences.getRefactoring("Logging") if self.logging: self.brmctx.setProgressLogger(self.ui.stdout) else: self.brmctx.setProgressLogger(SilentLogger()) self.brmctx.setWarningLogger(self.ui.stderr) def handleReset(self): """ Private slot to handle the Reset action. """ self.init() ################################################################## ## methods below are handling the various Query actions ################################################################## def handleQueryReferences(self): """ Private slot to handle the Query References action. """ self.showOutput() aw = self.ui.getViewManager().activeWindow() if aw is None: return if not aw.hasSelectedText(): # no selection available QMessageBox.warning(self.ui, self.trUtf8("Query References"), self.trUtf8("Highlight the name of a Function, Class or Method and try again."), self.trUtf8("&OK")) return if not self.confirmAllBuffersSaved(): return filename = aw.getFileName() line, column, dummy1, dummy2 = aw.getSelection() qApp.setOverrideCursor(Qt.waitCursor) qApp.processEvents() self.matchesDialog = None if self.projectopen: sys.path.insert(0, self.projectpath) try: try: for ref in self.brmctx.findReferencesByCoordinates(filename, line + 1, column): if self.matchesDialog is None: self.matchesDialog = MatchesDialog(self.ui) self.matchesDialog.show() self.matchesDialog.addEntry(ref) except AssertionError: pass except Exception, e: self.__unhandledException(e) finally: if self.projectopen: del sys.path[0] qApp.restoreOverrideCursor() if self.matchesDialog is None: QMessageBox.information(self.ui, self.trUtf8("Query References"), self.trUtf8("No matches were found."), self.trUtf8("&OK")) def handleQueryDefinition(self): """ Private slot to handle the Query Definition action """ self.showOutput() aw = self.ui.getViewManager().activeWindow() if aw is None: return if not self.confirmAllBuffersSaved(): return filename = aw.getFileName() if aw.hasSelectedText(): line, column, dummy1, dummy2 = aw.getSelection() else: line, column = aw.getCursorPosition() qApp.setOverrideCursor(Qt.waitCursor) qApp.processEvents() if self.projectopen: sys.path.insert(0, self.projectpath) try: try: defns = self.brmctx.findDefinitionByCoordinates(filename, line + 1, column) try: firstref = defns.next() self.ui.getViewManager().displayPythonFile(firstref.filename, firstref.lineno) except StopIteration: QMessageBox.information(self.ui, self.trUtf8("Query Definition"), self.trUtf8("Definition wasn't found."), self.trUtf8("&OK")) else: self.matchesDialog = None for ref in defns: if self.matchesDialog is None: self.matchesDialog = MatchesDialog(self.ui) self.matchesDialog.show() self.matchesDialog.addEntry(firstref) self.matchesDialog.addEntry(ref) except AssertionError: pass except Exception, e: self.__unhandledException(e) finally: if self.projectopen: del sys.path[0] qApp.restoreOverrideCursor() ################################################################## ## methods below are handling the various Refactoring actions ################################################################## def renameMethodPromptCallback(self, filename, line, start, stop): """ Private slot called by the refactoring machine to ask the user for confirmation. @param filename the name of the file (string) @param line the line of the object (int) @param start beginning column of the object (int) @param stop end column of the object (int) @return flag indicating renaming wanted (boolean) """ vm = self.ui.getViewManager() # display the file and select the method call vm.displayPythonFile(filename) aw = vm.activeWindow() cline, cindex = aw.getCursorPosition() aw.ensureLineVisible(line-1) aw.gotoLine(line-1) aw.setSelection(line-1, start, line-1, stop) ans = QMessageBox.information(self.ui, self.trUtf8("Rename"), self.trUtf8("Cannot deduce the type of the highlighted object reference.<br>" "Rename this declaration?"), self.trUtf8("&Yes"), self.trUtf8("&No")) aw.setCursorPosition(cline, cindex) aw.ensureCursorVisible() return ans == 0 def refreshEditors(self, savedfiles, filename, line): """ Private method to refresh modified editors. @param savedfiles list of filenames of modified files (list of strings) @param filename filename of the active editor (string) @param line line to place cursor at (int) """ vm = self.ui.getViewManager() openFiles = vm.getOpenFilenames() self.refreshing = 1 for file in savedfiles: normfile = Utilities.normcasepath(file) if normfile in openFiles: dummy, editor = vm.getEditor(normfile) editor.refresh() if filename is not None: vm.displayPythonFile(filename, line) self.refreshing = 0 def handleRename(self): """ Private slot to handle the Rename action. """ self.showOutput() aw = self.ui.getViewManager().activeWindow() if aw is None: return if not aw.hasSelectedText(): # no selection available QMessageBox.warning(self.ui, self.trUtf8("Rename"), self.trUtf8("Highlight the declaration you want to rename and try again."), self.trUtf8("&OK")) return if not self.confirmAllBuffersSaved(): return filename = aw.getFileName() line, column, dummy1, dummy2 = aw.getSelection() if self.projectopen: sys.path.insert(0, self.projectpath) try: try: self.brmctx.setRenameMethodPromptCallback(self.renameMethodPromptCallback) newname, ok = QInputDialog.getText(self.trUtf8("Rename"), self.trUtf8("Rename to:")) if not ok or newname.isEmpty(): return # user pressed cancel or didn't enter anything self.brmctx.renameByCoordinates(filename, line+1, column, str(newname)) savedfiles = self.brmctx.save() self.refreshEditors(savedfiles, filename, line+1) except AssertionError: pass except Exception, e: self.__unhandledException(e) finally: if self.projectopen: del sys.path[0] def handleExtractMethod(self): """ Private slot to handle the Extract Method action. """ self.showOutput() aw = self.ui.getViewManager().activeWindow() if aw is None: return if not aw.hasSelectedText(): # no selection available QMessageBox.warning(self.ui, self.trUtf8("Extract Method"), self.trUtf8("Highlight the region of code you want to extract and try again."), self.trUtf8("&OK")) return if not self.confirmAllBuffersSaved(): return filename = aw.getFileName() startline, startcolumn, endline, endcolumn = aw.getSelection() newname, ok = QInputDialog.getText(self.trUtf8("Extract Method"), self.trUtf8("New Name:")) if not ok or newname.isEmpty(): return # user pressed cancel or didn't enter anything if self.projectopen: sys.path.insert(0, self.projectpath) try: try: self.brmctx.extractMethod(filename, startline+1, startcolumn, endline+1, endcolumn, str(newname)) savedfiles = self.brmctx.save() self.refreshEditors(savedfiles, filename, startline+1) except AssertionError: pass except Exception, e: self.__unhandledException(e) finally: if self.projectopen: del sys.path[0] def handleInlineLocalVariable(self): """ Private slot to handle the Inline Local Variable action. """ self.showOutput() aw = self.ui.getViewManager().activeWindow() if aw is None: return if not self.confirmAllBuffersSaved(): return filename = aw.getFileName() if aw.hasSelectedText(): line, column, dummy1, dummy2 = aw.getSelection() else: line, column = aw.getCursorPosition() if self.projectopen: sys.path.insert(0, self.projectpath) try: try: self.brmctx.inlineLocalVariable(filename, line+1, column) savedfiles = self.brmctx.save() self.refreshEditors(savedfiles, filename, line+1) except AssertionError: pass except Exception, e: self.__unhandledException(e) finally: if self.projectopen: del sys.path[0] def handleExtractLocalVariable(self): """ Private slot to handle the Extract Local Variable action. """ self.showOutput() aw = self.ui.getViewManager().activeWindow() if aw is None: return if not aw.hasSelectedText(): # no selection available QMessageBox.warning(self.ui, self.trUtf8("Extract Local Variable"), self.trUtf8("Highlight the region of code you want to extract and try again."), self.trUtf8("&OK")) return if not self.confirmAllBuffersSaved(): return filename = aw.getFileName() startline, startcolumn, endline, endcolumn = aw.getSelection() newname, ok = QInputDialog.getText(self.trUtf8("Extract Local Variable"), self.trUtf8("New Name:")) if not ok or newname.isEmpty(): return # user pressed cancel or didn't enter anything if self.projectopen: sys.path.insert(0, self.projectpath) try: try: self.brmctx.extractLocalVariable(filename, startline+1, startcolumn, endline+1, endcolumn, str(newname)) savedfiles = self.brmctx.save() self.refreshEditors(savedfiles, filename, startline+1) except AssertionError: pass except Exception, e: self.__unhandledException(e) finally: if self.projectopen: del sys.path[0] def handleMoveClass(self): """ Private slot to handle the Move Class action. """ self.showOutput() aw = self.ui.getViewManager().activeWindow() if aw is None: return if not self.confirmAllBuffersSaved(): return filename = aw.getFileName() line, column = aw.getCursorPosition() newname = QFileDialog.getSaveFileName(\ None, self.trUtf8("Python Files (*.py)"), None, None, self.trUtf8("New Module"), None, 1) if newname.isEmpty(): return # user pressed cancel or didn't enter anything newname = str(newname) if not newname.endswith('.py'): newname = "%s.py" % newname if not os.path.exists(newname): try: f = open(newname, "w") f.close() except Exception, e: self.__unhandledException(e) return if self.projectopen: sys.path.insert(0, self.projectpath) try: try: self.brmctx.moveClassToNewModule(filename, line+1, str(newname)) savedfiles = self.brmctx.save() self.refreshEditors(savedfiles, filename, line+1) except AssertionError: pass except Exception, e: self.__unhandledException(e) finally: if self.projectopen: del sys.path[0] def handleMoveFunction(self): """ Private slot to handle the Move Function action. """ self.showOutput() aw = self.ui.getViewManager().activeWindow() if aw is None: return if not self.confirmAllBuffersSaved(): return filename = aw.getFileName() line, column = aw.getCursorPosition() newname = QFileDialog.getSaveFileName(\ None, self.trUtf8("Python Files (*.py)"), None, None, self.trUtf8("New Module"), None, 1) if newname.isEmpty(): return # user pressed cancel or didn't enter anything newname = str(newname) if not newname.endswith('.py'): newname = "%s.py" % newname if not os.path.exists(newname): try: f = open(newname, "w") f.close() except Exception, e: self.__unhandledException(e) return if self.projectopen: sys.path.insert(0, self.projectpath) try: try: self.brmctx.moveFunctionToNewModule(filename, line+1, str(newname)) savedfiles = self.brmctx.save() self.refreshEditors(savedfiles, filename, line+1) except AssertionError: pass except Exception, e: self.__unhandledException(e) finally: if self.projectopen: del sys.path[0] def handleUndo(self): """ Private slot to handle the Undo action. """ self.showOutput() aw = self.ui.getViewManager().activeWindow() if aw is not None: filename = aw.getFileName() line, column = aw.getCursorPosition() else: filename = None line = 0 try: try: self.brmctx.undo() except bike.UndoStackEmptyException: QMessageBox.information(self.ui, self.trUtf8("Undo"), self.trUtf8("The Undo stack is empty."), self.trUtf8("&OK")) savedfiles = self.brmctx.save() self.refreshEditors(savedfiles, filename, line+1) except AssertionError: pass except Exception, e: self.__unhandledException(e) ################################################################## ## methods below are private utility methods ################################################################## def confirmBufferIsSaved(self, editor): """ Private method to check, if an editor has unsaved changes. @param editor Reference to the editor to be checked. """ return editor.checkDirty() def confirmAllBuffersSaved(self): """ Private method to check, if any editor has unsaved changes. """ return self.ui.getViewManager().checkAllDirty() def setMenuItemsEnabled(self, enabled): """ Private method to enable/disable menu items. @param enabled Flag indicating enabled or disabled status. (boolean) """ for itm in self.menuitems: self.menu.setItemEnabled(itm, enabled) def __unhandledException(self, msg): """ Private method handling not specifically handled exceptions. @param msg message describing the exception (string) """ if not str(msg): msg = str(sys.exc_info()[0]) QMessageBox.critical(None, self.trUtf8("Refactoring"), self.trUtf8("""Caught exception <b>%1</b>""") .arg(str(msg)), self.trUtf8("&OK"), None, None, 0, -1) def showOutput(self): """ Private method to switch to the relevant output tab. """ if self.logging: qApp.mainWidget().showLogTab("stdout") else: qApp.mainWidget().showLogTab("stderr") ################################################################## ## slots below are public utility methods ################################################################## def handlePreferencesChanged(self): """ Public slot called when the preferences have been changed. """ self.logging = Preferences.getRefactoring("Logging") if self.logging: self.brmctx.setProgressLogger(self.ui.stdout) else: self.brmctx.setProgressLogger(SilentLogger()) def getActions(self): """ Public method to get a list of all actions. @return list of all actions (list of QAction) """ actionList = [] for act in self.queryList("QAction"): if not isinstance(act, QActionGroup): actionList.append(act) return actionList def handleProjectOpened(self): """ Public slot to handle the projectOpened signal. """ self.projectopen = 1 self.projectpath = self.ui.getProject().ppath def handleProjectClosed(self): """ Public slot to handle the projectClosed signal. """ self.projectopen = 0 self.projectpath = '' |
From: Fabio Z. <fa...@us...> - 2004-09-14 17:42:21
|
Update of /cvsroot/pydev/org.python.pydev/PySrc/ThirdParty/brm/bike In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv9579/PySrc/ThirdParty/brm/bike Added Files: log.py __init__.py globals.py logging.py bikefacade.py Log Message: Code completion improvements. Starting refactoring integration with bicycle repair man. --- NEW FILE: globals.py --- try: True = 1 False = 0 except: pass --- NEW FILE: log.py --- import sys class SilentLogger: def write(*args): pass progress = SilentLogger() warning = SilentLogger() #warning = sys.stderr --- NEW FILE: logging.py --- #! /usr/bin/env python # # Copyright 2001-2002 by Vinay Sajip. All Rights Reserved. # # Permission to use, copy, modify, and distribute this software and its # documentation for any purpose and without fee is hereby granted, # provided that the above copyright notice appear in all copies and that # both that copyright notice and this permission notice appear in # supporting documentation, and that the name of Vinay Sajip # not be used in advertising or publicity pertaining to distribution # of the software without specific, written prior permission. # VINAY SAJIP DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE, INCLUDING # ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL # VINAY SAJIP BE LIABLE FOR ANY SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES OR # ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER # IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT # OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. # # For the change history, see README.txt in the distribution. [...1956 lines suppressed...] _listener = None _releaseLock() # bicycle repair man stuff bike_logger_initialised = 0 def init(): global bike_logger_initialised if not bike_logger_initialised: log = getLogger("bike") h = StreamHandler() h.setFormatter(Formatter( fmt="%(pathname)s:%(lineno)s:%(levelname)s %(message)s")) log.addHandler(h) bike_logger_initialised = 1 if __name__ == "__main__": print __doc__ --- NEW FILE: __init__.py --- # The root bicyclerepairman package # do: # --------------------- # import bike # ctx = bike.load() # --------------------- # to instantiate a bicyclerepairman context object from bikefacade import init, NotAPythonModuleOrPackageException, CouldntLocateASTNodeFromCoordinatesException, UndoStackEmptyException --- NEW FILE: bikefacade.py --- import os import sys import compiler from parser import ParserError from bike.parsing.pathutils import getRootDirectory from bike.refactor import extractMethod from bike.refactor.rename import rename from bike.refactor.extractMethod import coords from bike.transformer.save import save as saveUpdates from bike.parsing.utils import fqn_rcar, fqn_rcdr from bike.parsing import visitor from bike.transformer.undo import getUndoStack, UndoStackEmptyException from bike.parsing.fastparserast import getRoot, Class, Function from bike.query.common import getScopeForLine from bike.query.getTypeOf import getTypeOfExpr, UnfoundType from bike.query.findReferences import findReferences from bike.query.findDefinition import findAllPossibleDefinitionsByCoords from bike.refactor import inlineVariable, extractVariable, moveToModule from bike.parsing.load import Cache from bike import log def init(): #context = BRMContext_impl() context = BRMContext_wrapper() return context # the context object public interface class BRMContext(object): def save(self): """ save the changed files out to disk """ def setRenameMethodPromptCallback(self, callback): """ sets a callback to ask the user about method refs which brm can't deduce the type of. The callback must be callable, and take the following parameters: - filename - linenumber - begin column - end column (begin and end columns enclose the problematic method call) """ def renameByCoordinates(self, filename_path, line, col, newname): """ an ide friendly method which renames a class/fn/method pointed to by the coords and filename""" def extract(self, filename_path, begin_line, begin_col, end_line, end_col, name): """ extracts the region into the named method/function based on context""" def inlineLocalVariable(self,filename_path, line, col): """ Inlines the variable pointed to by line:col. (N.B. line:col can also point to a reference to the variable as well as the definition) """ def extractLocalVariable(self,filename_path, begin_line, begin_col, end_line, end_col, variablename): """ Extracts the region into a variable """ def setProgressLogger(self,logger): """ Sets the progress logger to an object with a write method """ def setWarningLogger(self,logger): """ Sets the warning logger to an object with a write method """ def undo(self): """ undoes the last refactoring. WARNING: this is dangerous if the user has modified files since the last refactoring. Raises UndoStackEmptyException""" def findReferencesByCoordinates(self, filename_path, line, column): """ given the coords of a function, class, method or variable returns a generator which finds references to it. """ def findDefinitionByCoordinates(self,filename_path,line,col): """ given the coordates to a reference, tries to find the definition of that reference """ def moveClassToNewModule(self,filename_path, line, newfilename): """ moves the class pointed to by (filename_path, line) to a new module """ def moveFunctionToNewModule(self,filename_path, line, newfilename): """ moves the function pointed to by (filename_path, line) to a new module """ class NotAPythonModuleOrPackageException: pass class CouldntLocateASTNodeFromCoordinatesException: pass # Wrapper to ensure that caches are purged on each request class BRMContext_wrapper: def __init__(self): self.brmctx = BRMContext_impl() def __getattr__(self,name): return BRMContext_callWrapper(self.brmctx,name) class BRMContext_callWrapper: def __init__(self,brmctx,methodname): self.name = methodname self.brmctx = brmctx def __call__(self,*args): Cache.instance.reset() try: return getattr(self.brmctx,self.name)(*args) finally: Cache.instance.reset() class BRMContext_impl(BRMContext): def __init__(self): self.ast = getRoot() # Used because some refactorings delegate back to the user. # this flag ensures that code isnt imported during those times self.readyToLoadNewCode = 1 self.paths = [] getUndoStack(1) # force new undo stack if not getRoot().unittestmode: log.warning = sys.stderr self.promptUserClientCallback = None def _getAST(self): return self.ast # returns a list of saved filenames def save(self): savedfiles = saveUpdates() return savedfiles def setRenameMethodPromptCallback(self, callback): self.promptUserClientCallback = callback def normalizeFilename(self,filename): filename = os.path.expanduser(filename) filename = os.path.normpath(os.path.abspath(filename)) return filename def extractMethod(self, filename_path, begin_line, begin_column, end_line, end_column, methodname): self.extract(filename_path, begin_line, begin_column, end_line, end_column,methodname) def extractFunction(self, filename_path, begin_line, begin_column, end_line, end_column, methodname): self.extract(filename_path, begin_line, begin_column, end_line, end_column,methodname) # does it based on context def extract(self, filename_path, begin_line, begin_col, end_line, end_col, name): filename_path = self.normalizeFilename(filename_path) extractMethod.extractMethod(filename_path, coords(begin_line, begin_col), coords(end_line, end_col), name) def inlineLocalVariable(self,filename_path, line, col): filename_path = self.normalizeFilename(filename_path) inlineVariable.inlineLocalVariable(filename_path,line,col) def extractLocalVariable(self,filename_path, begin_line, begin_col, end_line, end_col, variablename): filename_path = self.normalizeFilename(filename_path) extractVariable.extractLocalVariable(filename_path, coords(begin_line, begin_col), coords(end_line, end_col), variablename) def moveClassToNewModule(self,filename_path, line, newfilename): filename_path = self.normalizeFilename(filename_path) newfilename = self.normalizeFilename(newfilename) moveToModule.moveClassToNewModule(filename_path, line, newfilename) def moveFunctionToNewModule(self,filename_path, line, newfilename): filename_path = self.normalizeFilename(filename_path) newfilename = self.normalizeFilename(newfilename) moveToModule.moveFunctionToNewModule(filename_path, line, newfilename) def undo(self): getUndoStack().undo() def _promptUser(self, filename, lineno, colbegin, colend): return self.promptUserClientCallback(filename, lineno, colbegin, colend) # must be an object with a write method def setProgressLogger(self,logger): log.progress = logger # must be an object with a write method def setWarningLogger(self,logger): log.warning = logger # filename_path must be absolute def renameByCoordinates(self, filename_path, line, col, newname): filename_path = self.normalizeFilename(filename_path) Cache.instance.reset() try: self._setNonLibPythonPath(filename_path) rename(filename_path,line,col,newname, self.promptUserClientCallback) finally: Cache.instance.reset() def _reverseCoordsIfWrongWayRound(self, colbegin, colend): if(colbegin > colend): colbegin,colend = colend,colbegin return colbegin,colend def findDefinitionByCoordinates(self,filename_path,line,col): filename_path = self.normalizeFilename(filename_path) self._setCompletePythonPath(filename_path) return findAllPossibleDefinitionsByCoords(filename_path,line,col) # filename_path must be absolute def findReferencesByCoordinates(self, filename_path, line, column): filename_path = self.normalizeFilename(filename_path) self._setNonLibPythonPath(filename_path) return findReferences(filename_path,line,column) def refreshASTFromFileSystem(self): for path in self.paths: self.ast = loadast(path, self.ast) def _setCompletePythonPath(self,filename): pythonpath = [] + sys.path # make a copy self.ast.pythonpath = pythonpath def _setNonLibPythonPath(self,filename): if getRoot().unittestmode: return pythonpath = self._removeLibdirsFromPath(sys.path) pythonpath = [os.path.abspath(p) for p in pythonpath] self.ast.pythonpath = pythonpath def _getCurrentSearchPath(self): return self.ast.pythonpath def _removeLibdirsFromPath(self, pythonpath): libdir = os.path.join(sys.prefix,"lib").lower() pythonpath = [p for p in pythonpath if not p.lower().startswith(libdir)] return pythonpath def _deducePackageOfFile(filename): package = "" dot = "" dir = os.path.dirname(filename) while dir != ""and \ os.path.exists(os.path.join(dir, "__init__.py")): dir, dirname = os.path.split(dir) package = dirname+dot+package dot = "." return package |
From: Fabio Z. <fa...@us...> - 2004-09-14 17:42:21
|
Update of /cvsroot/pydev/org.python.pydev/PySrc/ThirdParty/brm/bike/parsing In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv9579/PySrc/ThirdParty/brm/bike/parsing Added Files: pathutils.py __init__.py constants.py load.py newstuff.py visitor.py fastparser.py fastparserast.py parserutils.py utils.py Log Message: Code completion improvements. Starting refactoring integration with bicycle repair man. --- NEW FILE: fastparserast.py --- from __future__ import generators from parserutils import generateLogicalLines, maskStringsAndComments, maskStringsAndRemoveComments import re import os import compiler from bike.transformer.save import resetOutputQueue TABWIDTH = 4 classNameRE = re.compile("^\s*class\s+(\w+)") fnNameRE = re.compile("^\s*def\s+(\w+)") _root = None def getRoot(): global _root if _root is None: resetRoot() return _root def resetRoot(root = None): global _root _root = root or Root() _root.unittestmode = False resetOutputQueue() def getModule(filename_path): from bike.parsing.load import CantLocateSourceNodeException, getSourceNode try: sourcenode = getSourceNode(filename_path) return sourcenode.fastparseroot except CantLocateSourceNodeException: return None def getPackage(directory_path): from bike.parsing.pathutils import getRootDirectory rootdir = getRootDirectory(directory_path) if rootdir == directory_path: return getRoot() else: return Package(directory_path, os.path.basename(directory_path)) class Root: def __init__(self, pythonpath = None): # singleton hack to allow functions in query package to appear # 'stateless' resetRoot(self) # this is to get round a python optimisation which reuses an # empty list as a default arg. unfortunately the client of # this method may fill that list, so it's not empty if not pythonpath: pythonpath = [] self.pythonpath = pythonpath def __repr__(self): return "Root()" #return "Root(%s)"%(self.getChildNodes()) # dummy method def getChild(self,name): return None class Package: def __init__(self, path, name): self.path = path self.name = name def getChild(self,name): from bike.parsing.newstuff import getModule return getModule(os.path.join(self.path,name+".py")) def __repr__(self): return "Package(%s,%s)"%(self.path, self.name) # used so that linenum can be an attribute class Line(str): pass class StructuralNode: def __init__(self, filename, srclines, modulesrc): self.childNodes = [] self.filename = filename self._parent = None self._modulesrc = modulesrc self._srclines = srclines self._maskedLines = None def addChild(self, node): self.childNodes.append(node) node.setParent(self) def setParent(self, parent): self._parent = parent def getParent(self): return self._parent def getChildNodes(self): return self.childNodes def getChild(self,name): matches = [c for c in self.getChildNodes() if c.name == name] if matches != []: return matches[0] def getLogicalLine(self,physicalLineno): return generateLogicalLines(self._srclines[physicalLineno-1:]).next() # badly named: actually returns line numbers of import statements def getImportLineNumbers(self): try: return self.importlines except AttributeError: return[] def getLinesNotIncludingThoseBelongingToChildScopes(self): srclines = self.getMaskedModuleLines() lines = [] lineno = self.getStartLine() for child in self.getChildNodes(): lines+=srclines[lineno-1: child.getStartLine()-1] lineno = child.getEndLine() lines+=srclines[lineno-1: self.getEndLine()-1] return lines def generateLinesNotIncludingThoseBelongingToChildScopes(self): srclines = self.getMaskedModuleLines() lines = [] lineno = self.getStartLine() for child in self.getChildNodes(): for line in srclines[lineno-1: child.getStartLine()-1]: yield self.attachLinenum(line,lineno) lineno +=1 lineno = child.getEndLine() for line in srclines[lineno-1: self.getEndLine()-1]: yield self.attachLinenum(line,lineno) lineno +=1 def generateLinesWithLineNumbers(self,startline=1): srclines = self.getMaskedModuleLines() for lineno in range(startline,len(srclines)+1): yield self.attachLinenum(srclines[lineno-1],lineno) def attachLinenum(self,line,lineno): line = Line(line) line.linenum = lineno return line def getMaskedModuleLines(self): from bike.parsing.load import Cache try: maskedlines = Cache.instance.maskedlinescache[self.filename] except: # make sure src is actually masked # (could just have keywords masked) maskedsrc = maskStringsAndComments(self._modulesrc) maskedlines = maskedsrc.splitlines(1) Cache.instance.maskedlinescache[self.filename] = maskedlines return maskedlines class Module(StructuralNode): def __init__(self, filename, name, srclines, maskedsrc): StructuralNode.__init__(self, filename, srclines, maskedsrc) self.name = name self.indent = -TABWIDTH self.flattenedNodes = [] self.module = self def getMaskedLines(self): return self.getMaskedModuleLines() def getFlattenedListOfChildNodes(self): return self.flattenedNodes def getStartLine(self): return 1 def getEndLine(self): return len(self.getMaskedModuleLines())+1 def getSourceNode(self): return self.sourcenode def setSourceNode(self, sourcenode): self.sourcenode = sourcenode def matchesCompilerNode(self,node): return isinstance(node,compiler.ast.Module) and \ node.name == self.name def getParent(self): if self._parent is not None: return self._parent else: from newstuff import getPackage return getPackage(os.path.dirname(self.filename)) def __str__(self): return "bike:Module:"+self.filename indentRE = re.compile("^(\s*)\S") class Node: # module = the module node # linenum = starting line number def __init__(self, name, module, linenum, indent): self.name = name self.module = module self.linenum = linenum self.endline = None self.indent = indent def getMaskedLines(self): return self.getMaskedModuleLines()[self.getStartLine()-1:self.getEndLine()-1] def getStartLine(self): return self.linenum def getEndLine(self): if self.endline is None: physicallines = self.getMaskedModuleLines() lineno = self.linenum logicallines = generateLogicalLines(physicallines[lineno-1:]) # skip the first line, because it's the declaration line = logicallines.next() lineno+=line.count("\n") # scan to the end of the fn for line in logicallines: #print lineno,":",line, match = indentRE.match(line) if match and match.end()-1 <= self.indent: break lineno+=line.count("\n") self.endline = lineno return self.endline # linenum starts at 0 def getLine(self, linenum): return self._srclines[(self.getStartLine()-1) + linenum] baseClassesRE = re.compile("class\s+[^(]+\(([^)]+)\):") class Class(StructuralNode, Node): def __init__(self, name, filename, module, linenum, indent, srclines, maskedmodulesrc): StructuralNode.__init__(self, filename, srclines, maskedmodulesrc) Node.__init__(self, name, module, linenum, indent) self.type = "Class" def getBaseClassNames(self): #line = self.getLine(0) line = self.getLogicalLine(self.getStartLine()) match = baseClassesRE.search(line) if match: return [s.strip()for s in match.group(1).split(",")] else: return [] def getColumnOfName(self): match = classNameRE.match(self.getLine(0)) return match.start(1) def __repr__(self): return "<bike:Class:%s>" % self.name def __str__(self): return "bike:Class:"+self.filename+":"+\ str(self.getStartLine())+":"+self.name def matchesCompilerNode(self,node): return isinstance(node,compiler.ast.Class) and \ node.name == self.name def __eq__(self,other): return isinstance(other,Class) and \ self.filename == other.filename and \ self.getStartLine() == other.getStartLine() # describes an instance of a class class Instance: def __init__(self, type): assert type is not None self._type = type def getType(self): return self._type def __str__(self): return "Instance(%s)"%(self.getType()) class Function(StructuralNode, Node): def __init__(self, name, filename, module, linenum, indent, srclines, maskedsrc): StructuralNode.__init__(self, filename, srclines, maskedsrc) Node.__init__(self, name, module, linenum, indent) self.type = "Function" def getColumnOfName(self): match = fnNameRE.match(self.getLine(0)) return match.start(1) def __repr__(self): return "<bike:Function:%s>" % self.name def __str__(self): return "bike:Function:"+self.filename+":"+\ str(self.getStartLine())+":"+self.name def matchesCompilerNode(self,node): return isinstance(node,compiler.ast.Function) and \ node.name == self.name --- NEW FILE: fastparser.py --- #!/usr/bin/env python from bike.parsing.fastparserast import * from bike.parsing.parserutils import * from parser import ParserError #import exceptions indentRE = re.compile("^\s*(\w+)") # returns a tree of objects representing nested classes and functions # in the source def fastparser(src,modulename="",filename=""): try: return fastparser_impl(src,modulename,filename) except RuntimeError, ex: # if recursive call exceeds maximum depth if str(ex) == "maximum recursion limit exceeded": raise ParserError,"maximum recursion depth exceeded when fast-parsing src "+filename else: raise def fastparser_impl(src,modulename,filename): lines = src.splitlines(1) maskedSrc = maskPythonKeywordsInStringsAndComments(src) maskedLines = maskedSrc.splitlines(1) root = Module(filename,modulename,lines,maskedSrc) parentnode = root lineno = 0 for line in maskedLines: lineno+=1 #print "line",lineno,":",line m = indentRE.match(line) if m: indent = m.start(1) tokenstr = m.group(1) if tokenstr == "import" or tokenstr == "from": while indent <= parentnode.indent: # root indent is -TABWIDTH parentnode = parentnode.getParent() try: parentnode.importlines.append(lineno) except AttributeError: parentnode.importlines = [lineno] elif tokenstr == "class": m2 = classNameRE.match(line) if m2: n = Class(m2.group(1), filename, root, lineno, indent, lines, maskedSrc) root.flattenedNodes.append(n) while indent <= parentnode.indent: parentnode = parentnode.getParent() parentnode.addChild(n) parentnode = n elif tokenstr == "def": m2 = fnNameRE.match(line) if m2: n = Function(m2.group(1), filename, root, lineno, indent, lines, maskedSrc) root.flattenedNodes.append(n) while indent <= parentnode.indent: parentnode = parentnode.getParent() parentnode.addChild(n) parentnode = n elif indent <= parentnode.indent and \ tokenstr in ['if','for','while','try']: parentnode = parentnode.getParent() while indent <= parentnode.indent: parentnode = parentnode.getParent() return root --- NEW FILE: constants.py --- messages={"PARSING":"Parsing","ADDTYPEINFO":"Deducing Type Information"} MAXCALLDEPTH = 10 --- NEW FILE: load.py --- from bike.globals import * import os from bike.parsing.fastparser import fastparser class Cache: def __init__(self): self.reset() def reset(self): self.srcnodecache = {} self.typecache = {} self.maskedlinescache = {} instance = None Cache.instance = Cache() class CantLocateSourceNodeException(Exception): pass def getSourceNode(filename_path): #print "getSourceNode:",filename_path sourcenode = None try: sourcenode = Cache.instance.srcnodecache[filename_path] except KeyError: pass if sourcenode is None: from bike.parsing.newstuff import translateFnameToModuleName sourcenode = SourceFile.createFromFile(filename_path, translateFnameToModuleName(filename_path)) if sourcenode is None: raise CantLocateSourceNodeException(filename_path) Cache.instance.srcnodecache[filename_path]=sourcenode return sourcenode class SourceFile: def createFromString(filename, modulename, src): return SourceFile(filename,modulename,src) createFromString = staticmethod(createFromString) def createFromFile(filename,modulename): try: f = file(filename) src = f.read() f.close() except IOError: return None else: return SourceFile(filename,modulename,src) createFromFile = staticmethod(createFromFile) def __init__(self, filename, modulename, src): if os.path.isabs(filename): self.filename = filename else: self.filename = os.path.abspath(filename) self.modulename = modulename self.resetWithSource(src) def resetWithSource(self, source): # fastparser ast self.fastparseroot = fastparser(source,self.modulename,self.filename) self.fastparseroot.setSourceNode(self) self._lines = source.splitlines(1) self.sourcenode = self def __repr__(self): return "Source(%s,%s)"%('source', self.filename) def getChildNodes(self): return self.fastparseroot.getChildNodes() def getSource(self): return "".join(self.getLines()) def getLine(self,linenum): return self.getLines()[linenum-1] # TODO: rename me! def getFlattenedListOfFastParserASTNodes(self): return self.fastparseroot.getFlattenedListOfChildNodes() def getLines(self): return self._lines --- NEW FILE: parserutils.py --- from __future__ import generators import re escapedQuotesRE = re.compile(r"(\\\\|\\\"|\\\')") # changess \" \' and \\ into ** so that text searches # for " and ' won't hit escaped ones def maskEscapedQuotes(src): return escapedQuotesRE.sub("**", src) stringsAndCommentsRE = \ re.compile("(\"\"\".*?\"\"\"|'''.*?'''|\"[^\"]*\"|\'[^\']*\'|#.*?\n)", re.DOTALL) import string #transtable = string.maketrans('classdefifforwhiletry', "*********************") # performs a transformation on all of the comments and strings so that # text searches for python keywords won't accidently find a keyword in # a string or comment def maskPythonKeywordsInStringsAndComments(src): src = escapedQuotesRE.sub("**", src) allstrings = stringsAndCommentsRE.split(src) # every odd element is a string or comment for i in xrange(1, len(allstrings), 2): allstrings[i] = allstrings[i].upper() #allstrings[i] = allstrings[i].translate(transtable) return "".join(allstrings) allchars = string.maketrans("", "") allcharsExceptNewline = allchars[: allchars.index('\n')]+allchars[allchars.index('\n')+1:] allcharsExceptNewlineTranstable = string.maketrans(allcharsExceptNewline, '*'*len(allcharsExceptNewline)) # replaces all chars in a string or a comment with * (except newlines). # this ensures that text searches don't mistake comments for keywords, and that all # matches are in the same line/comment as the original def maskStringsAndComments(src): src = escapedQuotesRE.sub("**", src) allstrings = stringsAndCommentsRE.split(src) # every odd element is a string or comment for i in xrange(1, len(allstrings), 2): if allstrings[i].startswith("'''")or allstrings[i].startswith('"""'): allstrings[i] = allstrings[i][:3]+ \ allstrings[i][3:-3].translate(allcharsExceptNewlineTranstable)+ \ allstrings[i][-3:] else: allstrings[i] = allstrings[i][0]+ \ allstrings[i][1:-1].translate(allcharsExceptNewlineTranstable)+ \ allstrings[i][-1] return "".join(allstrings) # replaces all chars in a string or a comment with * (except newlines). # this ensures that text searches don't mistake comments for keywords, and that all # matches are in the same line/comment as the original def maskStringsAndRemoveComments(src): src = escapedQuotesRE.sub("**", src) allstrings = stringsAndCommentsRE.split(src) # every odd element is a string or comment for i in xrange(1, len(allstrings), 2): if allstrings[i].startswith("'''")or allstrings[i].startswith('"""'): allstrings[i] = allstrings[i][:3]+ \ allstrings[i][3:-3].translate(allcharsExceptNewlineTranstable)+ \ allstrings[i][-3:] elif allstrings[i].startswith("#"): allstrings[i] = '\n' else: allstrings[i] = allstrings[i][0]+ \ allstrings[i][1:-1].translate(allcharsExceptNewlineTranstable)+ \ allstrings[i][-1] return "".join(allstrings) implicitContinuationChars = (('(', ')'), ('[', ']'), ('{', '}')) emptyHangingBraces = [0,0,0,0,0] linecontinueRE = re.compile(r"\\\s*(#.*)?$") multiLineStringsRE = \ re.compile("(^.*?\"\"\".*?\"\"\".*?$|^.*?'''.*?'''.*?$)", re.DOTALL) #def splitLogicalLines(src): # src = multiLineStringsRE.split(src) # splits the string into logical lines. This requires the comments to # be removed, and strings masked (see other fns in this module) def splitLogicalLines(src): physicallines = src.splitlines(1) return [x for x in generateLogicalLines(physicallines)] class UnbalancedBracesException: pass # splits the string into logical lines. This requires the strings # masked (see other fns in this module) # Physical Lines *Must* start on a non-continued non-in-a-comment line # (although detects unbalanced braces) def generateLogicalLines(physicallines): tmp = [] hangingBraces = list(emptyHangingBraces) hangingComments = 0 for line in physicallines: # update hanging braces for i in range(len(implicitContinuationChars)): contchar = implicitContinuationChars[i] numHanging = hangingBraces[i] hangingBraces[i] = numHanging+line.count(contchar[0]) - \ line.count(contchar[1]) hangingComments ^= line.count('"""') % 2 hangingComments ^= line.count("'''") % 2 if hangingBraces[0] < 0 or \ hangingBraces[1] < 0 or \ hangingBraces[2] < 0: raise UnbalancedBracesException() if linecontinueRE.search(line): tmp.append(line) elif hangingBraces != emptyHangingBraces: tmp.append(line) elif hangingComments: tmp.append(line) else: tmp.append(line) yield "".join(tmp) tmp = [] # see above but yields (line,linenum) # needs physicallines to have linenum attribute # TODO: refactor with previous function def generateLogicalLinesAndLineNumbers(physicallines): tmp = [] hangingBraces = list(emptyHangingBraces) hangingComments = 0 linenum = None for line in physicallines: if tmp == []: linenum = line.linenum # update hanging braces for i in range(len(implicitContinuationChars)): contchar = implicitContinuationChars[i] numHanging = hangingBraces[i] hangingBraces[i] = numHanging+line.count(contchar[0]) - \ line.count(contchar[1]) hangingComments ^= line.count('"""') % 2 hangingComments ^= line.count("'''") % 2 if linecontinueRE.search(line): tmp.append(line) elif hangingBraces != emptyHangingBraces: tmp.append(line) elif hangingComments: tmp.append(line) else: tmp.append(line) yield "".join(tmp),linenum tmp = [] # takes a line of code, and decorates it with noops so that it can be # parsed by the python compiler. # e.g. "if foo:" -> "if foo: pass" # returns the line, and the adjustment made to the column pos of the first char # line must have strings and comments masked # # N.B. it only inserts keywords whitespace and 0's notSpaceRE = re.compile("\s*(\S)") commentRE = re.compile("#.*$") def makeLineParseable(line): return makeLineParseableWhenCommentsRemoved(commentRE.sub("",line)) def makeLineParseableWhenCommentsRemoved(line): line = line.strip() if ":" in line: if line.endswith(":"): line += " pass" if line.startswith("try"): line += "\nexcept: pass" elif line.startswith("except") or line.startswith("finally"): line = "try: pass\n" + line return line elif line.startswith("else") or line.startswith("elif"): line = "if 0: pass\n" + line return line elif line.startswith("yield"): return ("return"+line[5:]) return line --- NEW FILE: utils.py --- # get the first element of a fully qualified python path #(e.g. _car('a.b.c.d') = 'a') def fqn_car(fqn): try: return fqn[:fqn.index(".")] except ValueError: # i.e. no dots in fqn return fqn # get the other elements of a fully qualified python path #(e.g. _cdr('a.b.c.d') = 'b.c.d') def fqn_cdr(fqn): try: return fqn[fqn.index(".")+1:] except ValueError: # i.e. no dots in fqn return "" # reverse of above _rcar("a.b.c.d") = "d" def fqn_rcar(fqn): try: return fqn[fqn.rindex(".")+1:] except ValueError: # i.e. no dots in fqn return fqn # reverse of above _rcdr("a.b.c.d") = "a.b.c" def fqn_rcdr(fqn): try: return fqn[:fqn.rindex(".")] except ValueError: # i.e. no dots in fqn return "" --- NEW FILE: pathutils.py --- # A some of this code is take from Pythius - # Copyright (GPL) 2001 Jurgen Hermann <jh...@we...> from bike.globals import * import os def containsAny(str, set): """ Check whether 'str' contains ANY of the chars in 'set' """ return 1 in [c in str for c in set] def getPathOfModuleOrPackage(dotted_name, pathlist = None): """ Get the filesystem path for a module or a package. Return the file system path to a file for a module, and to a directory for a package. Return None if the name is not found, or is a builtin or extension module. """ from bike.parsing.newstuff import getPythonPath if pathlist is None: pathlist = getPythonPath() import imp # split off top-most name parts = dotted_name.split('.', 1) if len(parts) > 1: # we have a dotted path, import top-level package try: file, pathname, description = imp.find_module(parts[0], pathlist) if file: file.close() except ImportError: return None # check if it's indeed a package if description[2] == imp.PKG_DIRECTORY: # recursively handle the remaining name parts pathname = getPathOfModuleOrPackage(parts[1], [pathname]) else: pathname = None else: # plain name try: file, pathname, description = imp.find_module(dotted_name, pathlist) if file: file.close() if description[2]not in[imp.PY_SOURCE, imp.PKG_DIRECTORY]: pathname = None except ImportError: pathname = None return pathname def getFilesForName(name): """ Get a list of module files for a filename, a module or package name, or a directory. """ import imp if not os.path.exists(name): # check for glob chars if containsAny(name, "*?[]"): import glob files = glob.glob(name) list = [] for file in files: list.extend(getFilesForName(file)) return list # try to find module or package name = getPathOfModuleOrPackage(name) if not name: return[] if os.path.isdir(name): # find all python files in directory list = [] os.path.walk(name, _visit_pyfiles, list) return list elif os.path.exists(name) and not name.startswith("."): # a single file return [name] return [] def _visit_pyfiles(list, dirname, names): """ Helper for getFilesForName(). """ # get extension for python source files if not globals().has_key('_py_ext'): import imp global _py_ext _py_ext = [triple[0]for triple in imp.get_suffixes()if triple[2] == imp.PY_SOURCE][0] # don't recurse into CVS or Subversion directories if 'CVS'in names: names.remove('CVS') if '.svn'in names: names.remove('.svn') names_copy = [] + names for n in names_copy: if os.path.isdir(os.path.join(dirname, n))and \ not os.path.exists(os.path.join(dirname, n, "__init__.py")): names.remove(n) # add all *.py files to list list.extend( [os.path.join(dirname, file) for file in names if os.path.splitext(file)[1] == _py_ext and not file.startswith(".")]) # returns the directory which holds the first package of the package # hierarchy under which 'filename' belongs def getRootDirectory(filename): if os.path.isdir(filename): dir = filename else: dir = os.path.dirname(filename) while dir != "" and \ os.path.exists(os.path.join(dir, "__init__.py")): dir = os.path.dirname(dir) return dir # Returns the root package directoryname of the package hierarchy # under which 'filename' belongs def getPackageBaseDirectory(filename): if os.path.isdir(filename): dir = filename else: dir = os.path.dirname(filename) if not os.path.exists(os.path.join(dir, "__init__.py")): # parent dir is not a package return dir while dir != "" and \ os.path.exists(os.path.join(os.path.dirname(dir), "__init__.py")): dir = os.path.dirname(dir) return dir def filenameToModulePath(fname): directoriesPreceedingRoot = getRootDirectory(fname) import os # strip off directories preceeding root package directory if directoriesPreceedingRoot != "": mpath = fname.replace(directoriesPreceedingRoot, "") else: mpath = fname if(mpath[0] == os.path.normpath("/")): mpath = mpath[1:] mpath, ext = os.path.splitext(mpath) mpath = mpath.replace(os.path.normpath("/"), ".") return mpath --- NEW FILE: newstuff.py --- from __future__ import generators # Holding module for scaffolding needed to transition parsing package # into stateless design import os import re from bike.parsing.pathutils import getRootDirectory, getPackageBaseDirectory, \ filenameToModulePath, getPathOfModuleOrPackage, getFilesForName from bike.parsing.fastparserast import Module, Package, getRoot, getPackage, getModule import sys from bike.parsing.load import getSourceNode, CantLocateSourceNodeException def translateFnameToModuleName(filename_path): return filenameToModulePath(filename_path) # scope is the scope to search from def getModuleOrPackageUsingFQN(fqn, dirpath=None): pythonpath = getPythonPath() #print "getModuleOrPackageUsingFQN",pythonpath,fqn if dirpath is not None: assert os.path.isdir(dirpath) pythonpath = [dirpath] + pythonpath filename = getPathOfModuleOrPackage(fqn,pythonpath) #print "getModuleOrPackageUsingFQN - filename",filename if filename is not None: if os.path.isdir(filename): return getPackage(filename) else: return getModule(filename) else: return None def getPythonPath(): return getRoot().pythonpath def generateModuleFilenamesInPythonPath(contextFilename): files = [] rootdir = getRootDirectory(contextFilename) if rootdir in getPythonPath(): # just search the pythonpath for path in getPythonPath(): for file in getFilesForName(path): if file not in files: # check for duplicates files.append(file) yield file else: # search the package hierarchy containing contextFilename # in addition to pythonpath basedir = getPackageBaseDirectory(contextFilename) for path in [basedir] + getPythonPath(): for file in getFilesForName(path): if file not in files: # check for duplicates files.append(file) yield file # and search the files immediately above the package hierarchy for file in getFilesForName(os.path.join(rootdir,"*.py")): if file not in files: # check for duplicates files.append(file) yield file def generateModuleFilenamesInPackage(filenameInPackage): basedir = getPackageBaseDirectory(filenameInPackage) for file in getFilesForName(basedir): yield file # search all sourcenodes globally from the perspective of file 'contextFilename' def getSourceNodesContainingRegex(regexstr,contextFilename): regex = re.compile(regexstr) for fname in generateModuleFilenamesInPythonPath(contextFilename): try: f = file(fname) src = f.read() finally: f.close() if regex.search(src) is not None: yield getSourceNode(fname) fromRegex = re.compile("^\s*from\s+(\w+)\s+import") importregex = re.compile("^\s*import\s+(\w+)") # fileInPackage is the filename of a file in the package hierarchy # generates file and directory paths def generatePackageDependencies(fileInPackage): rejectPackagePaths = [getPackageBaseDirectory(fileInPackage)] for fname in generateModuleFilenamesInPackage(fileInPackage): try: f = file(fname) src = f.read() finally: f.close() packagepath = None for line in src.splitlines(): match = fromRegex.search(line) or importregex.search(line) if match is not None: modulepath = match.group(1) packagename = modulepath.split('.')[0] packagepath = getPathOfModuleOrPackage(packagename, getPythonPath()) if packagepath is not None and \ packagepath not in rejectPackagePaths: rejectPackagePaths.append(packagepath) # avoid duplicates yield packagepath --- NEW FILE: visitor.py --- from __future__ import generators class TreeWalker(object): VERBOSE = 0 def __init__(self): self.node = None self._cache = {} def default(self, node, *args): for child in node.getChildNodes(): self.dispatch(child, *args) def dispatch(self, node, *args): self.node = node klass = node.__class__ meth = self._cache.get(klass, None) if meth is None: className = klass.__name__ meth = getattr(self.visitor, 'visit' + className, self.default) self._cache[klass] = meth return meth(node, *args) def preorder(self, tree, visitor, *args): """Do preorder walk of tree using visitor""" self.visitor = visitor visitor.visit = self.dispatch visitor.visitChildren = self.default return self.dispatch(tree, *args) class GeneratingTreeWalker(TreeWalker): def default(self, node, *args): for child in node.getChildNodes(): for i in self.dispatch(child, *args): yield i def walk(tree, visitor): walker = TreeWalker() walker.preorder(tree, visitor) return walker.visitor def walkAndGenerate(tree,visitor): walker = GeneratingTreeWalker() return walker.preorder(tree, visitor) --- NEW FILE: __init__.py --- #from addtypeinfo import addtypeinfo #from load import load #from brmtransformer import parse |
From: Fabio Z. <fa...@us...> - 2004-09-14 17:42:21
|
Update of /cvsroot/pydev/org.python.pydev/PySrc/ThirdParty/brm/bike/query In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv9579/PySrc/ThirdParty/brm/bike/query Added Files: getTypeOf.py __init__.py common.py findReferences.py relationships.py getAllRelatedClasses.py getReferencesToModule.py findDefinition.py Log Message: Code completion improvements. Starting refactoring integration with bicycle repair man. --- NEW FILE: getReferencesToModule.py --- from __future__ import generators from bike.query.common import Match, globalScanForMatches, getScopeForLine, MatchFinder from getTypeOf import getTypeOf, getTypeOfExpr import compiler import re def getReferencesToModule(root, fqn): modulename = fqn.split(".")[-1] moduleobj = getTypeOf(root, fqn) moduleRefFinder = ModuleRefFinder(moduleobj) for ref in globalScanForMatches(moduleRefFinder, modulename): yield ref class ModuleRefFinder(MatchFinder): def __init__(self, targetmodule): self.targetmodule = targetmodule def visitName(self, node): if node.name == self.targetmodule.name: if getTypeOfExpr(self.scope, node) == self.targetmodule: self.appendMatch(node.name) self.popWordsUpTo(node.name) def visitImport(self, node): for name, alias in node.names: if name.split(".")[-1] == self.targetmodule.name: if getTypeOf(self.scope, name) == self.targetmodule: self.appendMatch(self.targetmodule.name) for nameelem in name.split("."): self.popWordsUpTo(nameelem) if alias is not None: self.popWordsUpTo(alias) def visitGetattr(self, node): for c in node.getChildNodes(): self.visit(c) if node.attrname == self.targetmodule.name: if getTypeOfExpr(self.scope, node) == self.targetmodule: self.appendMatch(self.targetmodule.name) self.popWordsUpTo(node.attrname) def visitFrom(self, node): for elem in node.modname.split("."): if elem == self.targetmodule.name: getTypeOf(self.scope, elem) == self.targetmodule self.appendMatch(self.targetmodule.name) self.popWordsUpTo(elem) for name, alias in node.names: if name == self.targetmodule.name: if alias and \ getTypeOf(self.scope, alias) == self.targetmodule: self.appendMatch(self.targetmodule.name) elif getTypeOf(self.scope, name) == self.targetmodule: self.appendMatch(self.targetmodule.name) if name != "*": self.popWordsUpTo(name) if alias is not None: self.popWordsUpTo(alias) --- NEW FILE: findDefinition.py --- from __future__ import generators from bike.query.common import Match, MatchFinder, \ getScopeForLine, indexToCoordinates, \ translateSourceCoordsIntoASTNode, scanScopeForMatches, \ isAMethod, convertNodeToMatchObject, walkLinesContainingStrings from bike.parsing.parserutils import generateLogicalLines,\ generateLogicalLinesAndLineNumbers, \ splitLogicalLines, makeLineParseable import compiler from compiler.ast import Getattr, Name, AssName, AssAttr from bike.parsing.fastparserast import getRoot, Package, Class, \ Module, Function, Instance import re from bike.query.getTypeOf import getTypeOfExpr, UnfoundType, \ isWordInLine, resolveImportedModuleOrPackage from bike.parsing import visitor from bike.parsing.visitor import walkAndGenerate from bike.parsing.parserutils import makeLineParseable,splitLogicalLines from bike.parsing.newstuff import getSourceNodesContainingRegex from bike.parsing.load import getSourceNode from bike import log class CantFindDefinitionException: pass def findAllPossibleDefinitionsByCoords(filepath,lineno,col): #try: node = translateSourceCoordsIntoASTNode(filepath,lineno,col) #except: # import traceback # traceback.print_exc() if node is None: raise "selected node type not supported" scope = getScopeForLine(getSourceNode(filepath),lineno) match = findDefinitionFromASTNode(scope,node) if match is not None: yield match if isinstance(node,Getattr) and (match is None or match.confidence != 100): root = getRoot() name = node.attrname for match in scanPythonPathForMatchingMethodNames(name,filepath): yield match print >>log.progress,"done" def findDefinitionFromASTNode(scope,node): assert node is not None if isinstance(node,Name) or isinstance(node,AssName): while 1: # try scope children childscope = scope.getChild(node.name) if childscope is not None: return convertNodeToMatchObject(childscope,100) if isinstance(scope,Package): scope = scope.getChild("__init__") # try arguments and assignments match = scanScopeAST(scope,node.name, AssignmentAndFnArgsSearcher(node.name)) if match is not None: return match # try imports match = searchImportedModulesForDefinition(scope,node) if match is not None: return match if not isinstance(scope,Module): # try parent scope scope = scope.getParent() else: break assert isinstance(scope,Module) elif isinstance(node,Getattr) or isinstance(node,AssAttr): exprtype = getTypeOfExpr(scope,node.expr) if not (exprtype is None or isinstance(exprtype,UnfoundType)): if isinstance(exprtype,Instance): exprtype = exprtype.getType() match = findDefinitionOfAttributeFromASTNode(exprtype, node.attrname) else: match = findDefinitionFromASTNode(exprtype, Name(node.attrname)) if match is not None: return match elif isinstance(node,compiler.ast.Function) or \ isinstance(node,compiler.ast.Class): if isAMethod(scope,node): match = findDefinitionOfAttributeFromASTNode(scope, node.name) else: match = findDefinitionFromASTNode(scope,Name(node.name)) if match is not None: return match type = getTypeOfExpr(scope,node) if type is not None and (not isinstance(type,UnfoundType)) and \ (not isinstance(type,Instance)): return convertNodeToMatchObject(type,100) else: return None def findDefinitionOfAttributeFromASTNode(type,name): assert isinstance(type,Class) attrfinder = AttrbuteDefnFinder([type],name) # first scan the method names: for child in type.getChildNodes(): if child.name == name: return convertNodeToMatchObject(child,100) # then scan the method source for attribues for child in type.getChildNodes(): if isinstance(child,Function): try: return scanScopeForMatches(child.module.getSourceNode(), child, attrfinder, name).next() except StopIteration: continue class AttrbuteDefnFinder(MatchFinder): def __init__(self,targetClasses,targetAttribute): self.targetClasses = targetClasses self.targetAttributeName = targetAttribute def visitAssAttr(self, node): for c in node.getChildNodes(): self.visit(c) if node.attrname == self.targetAttributeName: exprtype = getTypeOfExpr(self.scope,node.expr) if isinstance(exprtype,Instance) and \ exprtype.getType() in self.targetClasses: self.appendMatch(self.targetAttributeName) #else: # self.appendMatch(self.targetAttributeName,50) self.popWordsUpTo(node.attrname) def searchImportedModulesForDefinition(scope,node): lines = scope.module.getSourceNode().getLines() for lineno in scope.getImportLineNumbers(): logicalline = getLogicalLine(lines,lineno) logicalline = makeLineParseable(logicalline) ast = compiler.parse(logicalline) class ImportVisitor: def __init__(self,node): self.target = node self.match = None assert isinstance(self.target,Name), \ "Getattr not supported" def visitFrom(self, node): module = resolveImportedModuleOrPackage(scope,node.modname) if module is None: # couldn't find module return if node.names[0][0] == '*': # e.g. from foo import * match = findDefinitionFromASTNode(module,self.target) if match is not None: self.match = match return for name, alias in node.names: if alias is None and name == self.target.name: match = findDefinitionFromASTNode(module,self.target) if match is not None: self.match = match return match = visitor.walk(ast, ImportVisitor(node)).match if match: return match # loop def getLogicalLine(lines,lineno): return generateLogicalLines(lines[lineno-1:]).next() class AssignmentAndFnArgsSearcher(MatchFinder): def __init__(self,name): self.targetname = name self.match = None def visitAssName(self, node): if node.name == self.targetname: idx = self.getNextIndexOfWord(self.targetname) self.match = idx return def visitFunction(self, node): self.popWordsUpTo(node.name) for arg, default in self.zipArgs(node.argnames, node.defaults): if arg == self.targetname: idx = self.getNextIndexOfWord(self.targetname) self.match = idx return self.popWordsUpTo(arg) if default is not None: self.visit(default) self.visit(node.code) def getMatch(self): return self.match # scans for lines containing keyword, and then runs the visitor over # the parsed AST for that line def scanScopeAST(scope,keyword,matchfinder): lines = scope.generateLinesNotIncludingThoseBelongingToChildScopes() match = None for line,linenum in generateLogicalLinesAndLineNumbers(lines): if isWordInLine(keyword, line): doctoredline = makeLineParseable(line) ast = compiler.parse(doctoredline) matchfinder.reset(line) match = visitor.walk(ast,matchfinder).getMatch() if match is not None: column,yoffset = indexToCoordinates(line,match) m = createMatch(scope,linenum + yoffset,column) return m return None def createMatch(scope,lineno,x): m = Match() m.sourcenode = scope.module.getSourceNode() m.filename = m.sourcenode.filename m.lineno = lineno m.colno = x m.confidence = 100 return m # scan for methods globally (from perspective of 'perspectiveFilename') def scanPythonPathForMatchingMethodNames(name, contextFilename): class MethodFinder: def __init__(self,srcnode): self.matches = [] self.srcnode = srcnode def visitFunction(self,node): node = getScopeForLine(self.srcnode, self.lineno) if isinstance(node.getParent(),Class): if node.name == name: self.matches.append(convertNodeToMatchObject(node,50)) for srcnode in getSourceNodesContainingRegex(name,contextFilename): m = MethodFinder(srcnode) walkLinesContainingStrings(srcnode.fastparseroot,m,[name]) for match in m.matches: yield match def getIndexOfWord(line,targetword): words = re.split("(\w+)", line) idx = 0 for word in words: if word == targetword: break idx += len(word) return idx --- NEW FILE: getAllRelatedClasses.py --- """ def getAllRelatedClasses(root,classfqn): classobj = getTypeOf(root,classfqn) rootClasses = _getRootClasses(classobj) #print rootClasses relatedClasses = [] + rootClasses for rootClass in rootClasses: relatedClasses += _getAllSubClasses(rootClass,root) return relatedClasses def _getRootClasses(klass): if klass is None: # i.e. dont have base class in our ast return None if klass.getBaseClassNames() == []: # i.e. is a root class return[klass] else: rootclasses = [] for base in klass.getBaseClassNames(): baseclass = getTypeOf(klass,base) rootclass = _getRootClasses(baseclass) if rootclass is None: # base class not in our ast rootclass = [klass] rootclasses+=rootclass return rootclasses def _getAllSubClasses(baseclass, root, subclasses = []): class ClassVisitor: def visitSource(self,node): self.visit(node.fastparseroot) def visitClass(self, node): for basename in node.getBaseClassNames(): if basename.find(baseclass.name) != -1 and \ getTypeOf(node,basename) == baseclass: subclasses.append(node) _getAllSubClasses(node,root,subclasses) for child in node.getChildNodes(): self.visit(child) walk(root, ClassVisitor()) return subclasses """ --- NEW FILE: getTypeOf.py --- # getTypeOf(scope,fqn) and getTypeOfExpr(scope,ast) from bike.parsing.fastparserast import Class, Function, Module, Root, getRoot, Package, Instance, getModule from bike.parsing.parserutils import generateLogicalLines, makeLineParseable,splitLogicalLines, makeLineParseable from bike.parsing import visitor from bike import log from bike.parsing.newstuff import getModuleOrPackageUsingFQN from bike.parsing.load import Cache import os import re import compiler # used if an assignment exists, but cant find type # e.g. a = SomeFunctionNotLoaded() # (as opposed to 'None' if cant find an assignment) class UnfoundType: pass getTypeOfStack = [] # name is the fqn of the reference, scope is the scope ast object from # which the question is being asked. # returns an fastparser-ast object representing the type # or None if type not found def getTypeOf(scope, fqn): if isinstance(scope, Root): assert False, "Can't use getTypeOf to resolve from Root. Use getModuleOrPackageUsingFQN instead" #print "getTypeOf:"+fqn+" -- "+str(scope) #print #print str(getTypeOfStack) #print if (fqn,scope) in getTypeOfStack: # loop protection return None # this is crap! hashcode = str(scope)+fqn try: getTypeOfStack.append((fqn,scope)) try: type = Cache.instance.typecache[hashcode] except KeyError: type = getTypeOf_impl(scope, fqn) Cache.instance.typecache[hashcode] = type return type finally: del getTypeOfStack[-1] def getTypeOf_impl(scope, fqn): #print "getTypeOf_impl",scope,fqn if fqn == "None": return None if "."in fqn: rcdr = ".".join(fqn.split(".")[:-1]) rcar = fqn.split(".")[-1] newscope = getTypeOf(scope,rcdr) if newscope is not None: return getTypeOf(newscope, rcar) else: #print "couldnt find "+rcdr+" in "+str(scope) pass assert scope is not None #assert not ("." in fqn) if isinstance(scope,UnfoundType): return UnfoundType() if isinstance(scope, Package): #assert 0,scope return handlePackageScope(scope, fqn) elif isinstance(scope,Instance): return handleClassInstanceAttribute(scope, fqn) else: return handleModuleClassOrFunctionScope(scope,fqn) def handleModuleClassOrFunctionScope(scope,name): if name == "self" and isinstance(scope,Function) and \ isinstance(scope.getParent(),Class): return Instance(scope.getParent()) matches = [c for c in scope.getChildNodes()if c.name == name] if matches != []: return matches[0] type = scanScopeSourceForType(scope, name) if type != None: return type #print "name = ",name,"scope = ",scope type = getImportedType(scope, name) # try imported types #print "type=",type if type != None: return type parentScope = scope.getParent() while isinstance(parentScope,Class): # don't search class scope, since this is not accessible except # through self (is this true?) parentScope = parentScope.getParent() if not (isinstance(parentScope,Package) or isinstance(parentScope,Root)): return getTypeOf(parentScope, name) def handleClassInstanceAttribute(instance, attrname): theClass = instance.getType() # search methods and inner classes match = theClass.getChild(attrname) if match: return match #search methods for assignments with self.foo getattrs for child in theClass.getChildNodes(): if not isinstance(child,Function): continue res = scanScopeAST(child,attrname, SelfAttributeAssignmentVisitor(child,attrname)) if res is not None: return res def handlePackageScope(package, fqn): #print "handlePackageScope",package,fqn child = package.getChild(fqn) if child: return child if isinstance(package,Root): return getModuleOrPackageUsingFQN(fqn) # try searching the fs node = getModuleOrPackageUsingFQN(fqn,package.path) if node: return node # try the package init module initmod = package.getChild("__init__") if initmod is not None: type = getImportedType(initmod, fqn) if type: return type # maybe fqn is absolute return getTypeOf(getRoot(), fqn) wordRE = re.compile("\w+") def isWordInLine(word, line): if line.find(word) != -1: words = wordRE.findall(line) if word in words: return 1 return 0 def getImportedType(scope, fqn): lines = scope.module.getSourceNode().getLines() for lineno in scope.getImportLineNumbers(): logicalline = generateLogicalLines(lines[lineno-1:]).next() logicalline = makeLineParseable(logicalline) ast = compiler.parse(logicalline) match = visitor.walk(ast, ImportVisitor(scope,fqn)).match if match: return match #else loop class ImportVisitor: def __init__(self,scope,fqn): self.match = None self.targetfqn = fqn self.scope = scope def visitImport(self, node): # if target fqn is an import, then it must be a module or package for name, alias in node.names: if name == self.targetfqn: self.match = resolveImportedModuleOrPackage(self.scope,name) elif alias is not None and alias == self.targetfqn: self.match = resolveImportedModuleOrPackage(self.scope,name) def visitFrom(self, node): if node.names[0][0] == '*': # e.g. from foo import * if not "."in self.targetfqn: module = resolveImportedModuleOrPackage(self.scope, node.modname) if module: self.match = getTypeOf(module, self.targetfqn) else: for name, alias in node.names: if alias == self.targetfqn or \ (alias is None and name == self.targetfqn): scope = resolveImportedModuleOrPackage(self.scope, node.modname) if scope is not None: if isinstance(scope,Package): self.match = getModuleOrPackageUsingFQN(name,scope.path) else: assert isinstance(scope,Module) self.match = getTypeOf(scope, name) class TypeNotSupportedException: def __init__(self,msg): self.msg = msg def __str__(self): return self.msg # attempts to evaluate the type of the expression def getTypeOfExpr(scope, ast): if isinstance(ast, compiler.ast.Name): return getTypeOf(scope, ast.name) elif isinstance(ast, compiler.ast.Getattr) or \ isinstance(ast, compiler.ast.AssAttr): # need to do this in order to match foo.bah.baz as # a string in import statements fqn = attemptToConvertGetattrToFqn(ast) if fqn is not None: return getTypeOf(scope,fqn) expr = getTypeOfExpr(scope, ast.expr) if expr is not None: attrnametype = getTypeOf(expr, ast.attrname) return attrnametype return None elif isinstance(ast, compiler.ast.CallFunc): node = getTypeOfExpr(scope,ast.node) if isinstance(node,Class): return Instance(node) elif isinstance(node,Function): return getReturnTypeOfFunction(node) else: #raise TypeNotSupportedException, \ # "Evaluation of "+str(ast)+" not supported. scope="+str(scope) print >> log.warning, "Evaluation of "+str(ast)+" not supported. scope="+str(scope) return None def attemptToConvertGetattrToFqn(ast): fqn = ast.attrname ast = ast.expr while isinstance(ast,compiler.ast.Getattr): fqn = ast.attrname + "." + fqn ast = ast.expr if isinstance(ast,compiler.ast.Name): return ast.name + "." + fqn else: return None getReturnTypeOfFunction_stack = [] def getReturnTypeOfFunction(function): if function in getReturnTypeOfFunction_stack: # loop protection return None try: getReturnTypeOfFunction_stack.append(function) return getReturnTypeOfFunction_impl(function) finally: del getReturnTypeOfFunction_stack[-1] def getReturnTypeOfFunction_impl(function): return scanScopeAST(function,"return",ReturnTypeVisitor(function)) # does parse of scope sourcecode to deduce type def scanScopeSourceForType(scope, name): return scanScopeAST(scope,name,AssignmentVisitor(scope,name)) # scans for lines containing keyword, and then runs the visitor over # the parsed AST for that line def scanScopeAST(scope,keyword,astvisitor): lines = scope.getLinesNotIncludingThoseBelongingToChildScopes() src = ''.join(lines) match = None #print "scanScopeAST:"+str(scope) for line in splitLogicalLines(src): if isWordInLine(keyword, line): #print "scanning for "+keyword+" in line:"+line[:-1] doctoredline = makeLineParseable(line) ast = compiler.parse(doctoredline) match = visitor.walk(ast,astvisitor).getMatch() if match: return match return match class AssignmentVisitor: def __init__(self,scope,targetName): self.match=None self.scope = scope self.targetName = targetName def getMatch(self): return self.match def visitAssign(self,node): if isinstance(node.expr,compiler.ast.CallFunc): for assnode in node.nodes: if isinstance(assnode,compiler.ast.AssName) and \ assnode.name == self.targetName: self.match = getTypeOfExpr(self.scope,node.expr) if self.match is None: self.match = UnfoundType() class SelfAttributeAssignmentVisitor: def __init__(self,scope,targetName): self.match=None self.scope = scope self.targetName = targetName def getMatch(self): return self.match def visitAssign(self,node): if isinstance(node.expr,compiler.ast.CallFunc): for assnode in node.nodes: if isinstance(assnode,compiler.ast.AssAttr) and \ isinstance(assnode.expr,compiler.ast.Name) and \ assnode.expr.name == "self" and \ assnode.attrname == self.targetName: self.match = getTypeOfExpr(self.scope,node.expr) #print "here!",self.match.getType().fqn class ReturnTypeVisitor: def __init__(self,fn): self.match=None self.fn = fn def getMatch(self): return self.match def visitReturn(self,node): try: self.match = getTypeOfExpr(self.fn,node.value) except TypeNotSupportedException, ex: pass def resolveImportedModuleOrPackage(scope,fqn): # try searching from directory containing scope module path = os.path.dirname(scope.module.filename) node = getModuleOrPackageUsingFQN(fqn,path) if node is not None: return node # try searching from the root node = getModuleOrPackageUsingFQN(fqn) if node is not None: return node --- NEW FILE: findReferences.py --- from __future__ import generators from bike.globals import * from bike.parsing.fastparserast import Module, Class, Function, getRoot, Instance from bike.query.common import Match, MatchFinder,\ getScopeForLine, indexToCoordinates, \ translateSourceCoordsIntoASTNode, scanScopeForMatches,\ globalScanForMatches, isAMethod, convertNodeToMatchObject from compiler.ast import AssName,Name,Getattr,AssAttr import compiler from findDefinition import findDefinitionFromASTNode from bike.query.getTypeOf import getTypeOfExpr, UnfoundType from bike.query.relationships import getRootClassesOfHierarchy from bike import log from bike.parsing.load import getSourceNode class CouldntFindDefinitionException(Exception): pass def findReferencesIncludingDefn(filename,lineno,col): return findReferences(filename,lineno,col,1) def findReferences(filename,lineno,col,includeDefn=0): sourcenode = getSourceNode(filename) node = translateSourceCoordsIntoASTNode(filename,lineno,col) assert node is not None scope,defnmatch = getDefinitionAndScope(sourcenode,lineno,node) try: for match in findReferencesIncludingDefn_impl(sourcenode,node, scope,defnmatch): if not includeDefn and match == defnmatch: continue # don't return definition else: yield match except CouldntFindDefinitionException: raise CouldntFindDefinitionException("Could not find definition. Please locate manually (maybe using find definition) and find references from that") def findReferencesIncludingDefn_impl(sourcenode,node,scope,defnmatch): if isinstance(node,Name) or isinstance(node,AssName): return generateRefsToName(node.name,scope,sourcenode,defnmatch) elif isinstance(node,Getattr) or isinstance(node,AssAttr): exprtype = getTypeOfExpr(scope,node.expr) if exprtype is None or isinstance(exprtype,UnfoundType): raise CouldntFindDefinitionException() if isinstance(exprtype,Instance): exprtype = exprtype.getType() return generateRefsToAttribute(exprtype,node.attrname) else: targetname = node.attrname return globalScanForMatches(sourcenode.filename, NameRefFinder(targetname, defnmatch), targetname, ) if match is not None: return match elif isinstance(node,compiler.ast.Function) or \ isinstance(node,compiler.ast.Class): return handleClassOrFunctionRefs(scope, node, defnmatch) else: assert 0,"Seed to references must be Name,Getattr,Function or Class" def handleClassOrFunctionRefs(scope, node, defnmatch): if isAMethod(scope,node): for ref in generateRefsToAttribute(scope,node.name): yield ref else: #yield convertNodeToMatchObject(node,100) yield defnmatch for ref in generateRefsToName(node.name,scope, scope.module.getSourceNode(), defnmatch): yield ref def getDefinitionAndScope(sourcenode,lineno,node): scope = getScopeForLine(sourcenode,lineno) if scope.getStartLine() == lineno and \ scope.matchesCompilerNode(node): # scope is the node return scope.getParent(), convertNodeToMatchObject(scope,100) defnmatch = findDefinitionFromASTNode(scope,node) if defnmatch is None: raise CouldntFindDefinitionException() scope = getScopeForLine(sourcenode,defnmatch.lineno) return scope,defnmatch def generateRefsToName(name,scope,sourcenode,defnmatch): assert scope is not None if isinstance(scope,Function): # search can be limited to scope return scanScopeForMatches(sourcenode,scope, NameRefFinder(name,defnmatch), name) else: return globalScanForMatches(sourcenode.filename, NameRefFinder(name,defnmatch), name) class NameRefFinder(MatchFinder): def __init__(self, targetstr,targetMatch): self.targetstr = targetstr self.targetMatch = targetMatch def visitName(self, node): if node.name == self.targetstr: potentualMatch = findDefinitionFromASTNode(self.scope, node) if potentualMatch is not None and \ potentualMatch == self.targetMatch: self.appendMatch(node.name) self.popWordsUpTo(node.name) visitAssName = visitName def visitFunction(self, node): self.popWordsUpTo(node.name) for arg, default in self.zipArgs(node.argnames, node.defaults): if arg == self.targetstr: self.appendMatch(arg) self.popWordsUpTo(arg) if default is not None: self.visit(default) self.visit(node.code) def visitFrom(self, node): for elem in node.modname.split("."): self.popWordsUpTo(elem) for name, alias in node.names: if name == self.targetstr: if alias is not None: pretendNode = Name(alias) else: pretendNode = Name(name) if findDefinitionFromASTNode(self.scope, pretendNode) \ == self.targetMatch: self.appendMatch(name) self.popWordsUpTo(name) if alias is not None: self.popWordsUpTo(alias) def visitGetattr(self, node): for c in node.getChildNodes(): self.visit(c) if node.attrname == self.targetstr: defn = findDefinitionFromASTNode(self.scope, node) if defn is not None and defn == self.targetMatch: self.appendMatch(node.attrname) self.popWordsUpTo(node.attrname) def visitImport(self, node): for name, alias in node.names: if name.split(".")[-1] == self.targetstr: getattr = self.createGetattr(name) if findDefinitionFromASTNode(self.scope, getattr) == self.targetMatch: self.appendMatch(self.targetstr) for nameelem in name.split("."): self.popWordsUpTo(nameelem) if alias is not None: self.popWordsUpTo(alias) def createGetattr(self,fqn): node = Name(fqn[0]) for name in fqn.split(".")[1:]: node = Getattr(node,name) return node def generateRefsToAttribute(classobj,attrname): rootClasses = getRootClassesOfHierarchy(classobj) attrRefFinder = AttrbuteRefFinder(rootClasses,attrname) for ref in globalScanForMatches(classobj.filename, attrRefFinder, attrname): yield ref print >>log.progress,"Done" class AttrbuteRefFinder(MatchFinder): def __init__(self,rootClasses,targetAttribute): self.rootClasses = rootClasses self.targetAttributeName = targetAttribute def visitGetattr(self, node): for c in node.getChildNodes(): self.visit(c) if node.attrname == self.targetAttributeName: exprtype = getTypeOfExpr(self.scope,node.expr) if isinstance(exprtype,Instance) and \ self._isAClassInTheSameHierarchy(exprtype.getType()): self.appendMatch(self.targetAttributeName) elif isinstance(exprtype,UnfoundType) or \ exprtype is None: # couldn't find type, so not sure self.appendMatch(self.targetAttributeName,50) else: pass # definately not a match self.popWordsUpTo(node.attrname) visitAssAttr = visitGetattr def visitFunction(self,node): # visit methods if node.name == self.targetAttributeName: parentScope = self.scope.getParent() #print parentScope #print self.targetClasses if isinstance(parentScope,Class) and \ self._isAClassInTheSameHierarchy(parentScope): self.appendMatch(node.name) for c in node.getChildNodes(): self.visit(c) def _isAClassInTheSameHierarchy(self,classobj): #return classobj in self.targetClasses targetRootClasses = getRootClassesOfHierarchy(classobj) for rootclass in self.rootClasses: if rootclass in targetRootClasses: return True return False --- NEW FILE: relationships.py --- # queries to do with module/class/function relationships from __future__ import generators from bike.globals import * from getTypeOf import getTypeOf, getTypeOfExpr from bike.parsing.newstuff import generateModuleFilenamesInPythonPath, generateModuleFilenamesInPackage, getPythonPath from bike.parsing.pathutils import getPackageBaseDirectory from bike.query.common import MatchFinder, walkLinesContainingStrings, getScopeForLine from bike import log from bike.parsing.fastparserast import Module import re def getRootClassesOfHierarchy(klass): if klass is None: # i.e. dont have base class in our ast return None if klass.getBaseClassNames() == []: # i.e. is a root class return [klass] else: rootclasses = [] for base in klass.getBaseClassNames(): baseclass = getTypeOf(klass,base) rootclass = getRootClassesOfHierarchy(baseclass) if rootclass is None: # base class not in our ast rootclass = [klass] rootclasses+=rootclass return rootclasses --- NEW FILE: __init__.py --- # --- NEW FILE: common.py --- from __future__ import generators from bike.globals import * from bike.parsing.fastparserast import getRoot, Function, Class, Module, getModule from bike.parsing.parserutils import generateLogicalLines, makeLineParseable, UnbalancedBracesException, generateLogicalLinesAndLineNumbers from bike.parsing.newstuff import getSourceNodesContainingRegex from bike.parsing import visitor from bike import log import compiler from compiler.ast import Getattr, Name import re class Match: def __repr__(self): return ",".join([self.filename, str(self.lineno), str(self.colno), str(self.confidence)]) def __eq__(self,other): if self is None or other is None: return False return self.filename == other.filename and \ self.lineno == other.lineno and \ self.colno == other.colno def getScopeForLine(sourceNode, lineno): scope = None childnodes = sourceNode.getFlattenedListOfFastParserASTNodes() if childnodes == []: return sourceNode.fastparseroot #module node scope = sourceNode.fastparseroot for node in childnodes: if node.linenum > lineno: break scope = node if scope.getStartLine() != scope.getEndLine(): # is inline while scope.getEndLine() <= lineno: scope = scope.getParent() return scope # global from the perspective of 'contextFilename' def globalScanForMatches(contextFilename, matchFinder, targetname): for sourcenode in getSourceNodesContainingRegex(targetname, contextFilename): print >> log.progress, "Scanning", sourcenode.filename searchscope = sourcenode.fastparseroot for match in scanScopeForMatches(sourcenode,searchscope, matchFinder,targetname): yield match def scanScopeForMatches(sourcenode,scope,matchFinder,targetname): lineno = scope.getStartLine() for line in generateLogicalLines(scope.getMaskedLines()): if line.find(targetname) != -1: doctoredline = makeLineParseable(line) ast = compiler.parse(doctoredline) scope = getScopeForLine(sourcenode, lineno) matchFinder.reset(line) matchFinder.setScope(scope) matches = visitor.walk(ast, matchFinder).getMatches() for index, confidence in matches: match = Match() match.filename = sourcenode.filename match.sourcenode = sourcenode x, y = indexToCoordinates(line, index) match.lineno = lineno+y match.colno = x match.colend = match.colno+len(targetname) match.confidence = confidence yield match lineno+=line.count("\n") def walkLinesContainingStrings(scope,astWalker,targetnames): lineno = scope.getStartLine() for line in generateLogicalLines(scope.getMaskedLines()): if lineContainsOneOf(line,targetnames): doctoredline = makeLineParseable(line) ast = compiler.parse(doctoredline) astWalker.lineno = lineno matches = visitor.walk(ast, astWalker) lineno+=line.count("\n") def lineContainsOneOf(line,targetnames): for name in targetnames: if line.find(name) != -1: return True return False # translates an idx in a logical line into physical line coordinates # returns x and y coords def indexToCoordinates(src, index): y = src[: index].count("\n") startOfLineIdx = src.rfind("\n", 0, index)+1 x = index-startOfLineIdx return x, y # interface for MatchFinder classes # implement the visit methods class MatchFinder: def setScope(self, scope): self.scope = scope def reset(self, line): self.matches = [] self.words = re.split("(\w+)", line) # every other one is a non word self.positions = [] i = 0 for word in self.words: self.positions.append(i) #if '\n' in word: # handle newlines # i = len(word[word.index('\n')+1:]) #else: i+=len(word) self.index = 0 def getMatches(self): return self.matches # need to visit childnodes in same order as they appear def visitPrintnl(self,node): if node.dest: self.visit(node.dest) for n in node.nodes: self.visit(n) def visitName(self, node): self.popWordsUpTo(node.name) def visitClass(self, node): self.popWordsUpTo(node.name) for base in node.bases: self.visit(base) def zipArgs(self, argnames, defaults): """Takes a list of argument names and (possibly a shorter) list of default values and zips them into a list of pairs (argname, default). Defaults are aligned so that the last len(defaults) arguments have them, and the first len(argnames) - len(defaults) pairs have None as a default. """ fixed_args = len(argnames) - len(defaults) defaults = [None] * fixed_args + list(defaults) return zip(argnames, defaults) def visitFunction(self, node): self.popWordsUpTo(node.name) for arg, default in self.zipArgs(node.argnames, node.defaults): self.popWordsUpTo(arg) if default is not None: self.visit(default) self.visit(node.code) def visitGetattr(self,node): self.visit(node.expr) self.popWordsUpTo(node.attrname) def visitAssName(self, node): self.popWordsUpTo(node.name) def visitAssAttr(self, node): self.visit(node.expr) self.popWordsUpTo(node.attrname) def visitImport(self, node): for name, alias in node.names: for nameelem in name.split("."): self.popWordsUpTo(nameelem) if alias is not None: self.popWordsUpTo(alias) def visitFrom(self, node): for elem in node.modname.split("."): self.popWordsUpTo(elem) for name, alias in node.names: self.popWordsUpTo(name) if alias is not None: self.popWordsUpTo(alias) def visitLambda(self, node): for arg, default in self.zipArgs(node.argnames, node.defaults): self.popWordsUpTo(arg) if default is not None: self.visit(default) self.visit(node.code) def visitGlobal(self, node): for name in node.names: self.popWordsUpTo(name) def popWordsUpTo(self, word): if word == "*": return # won't be able to find this posInWords = self.words.index(word) idx = self.positions[posInWords] self.words = self.words[posInWords+1:] self.positions = self.positions[posInWords+1:] def appendMatch(self,name,confidence=100): idx = self.getNextIndexOfWord(name) self.matches.append((idx, confidence)) def getNextIndexOfWord(self,name): return self.positions[self.words.index(name)] class CouldNotLocateNodeException(Exception): pass def translateSourceCoordsIntoASTNode(filename,lineno,col): module = getModule(filename) maskedlines = module.getMaskedModuleLines() lline,backtrackchars = getLogicalLine(module, lineno) doctoredline = makeLineParseable(lline) ast = compiler.parse(doctoredline) idx = backtrackchars+col nodefinder = ASTNodeFinder(lline,idx) node = compiler.walk(ast, nodefinder).node if node is None: raise CouldNotLocateNodeException("Could not translate editor coordinates into source node") return node def getLogicalLine(module, lineno): # we know that the scope is the start of a logical line, so # we search from there scope = getScopeForLine(module.getSourceNode(), lineno) linegenerator = \ module.generateLinesWithLineNumbers(scope.getStartLine()) for lline,llinenum in \ generateLogicalLinesAndLineNumbers(linegenerator): if llinenum > lineno: break prevline = lline prevlinenum = llinenum backtrackchars = 0 for i in range(prevlinenum,lineno): backtrackchars += len(module.getSourceNode().getLines()[i-1]) return prevline, backtrackchars class ASTNodeFinder(MatchFinder): # line is a masked line of text # lineno and col are coords def __init__(self,line,col): self.line = line self.col = col self.reset(line) self.node = None def visitName(self,node): if self.checkIfNameMatchesColumn(node.name): self.node = node self.popWordsUpTo(node.name) def visitGetattr(self,node): self.visit(node.expr) if self.checkIfNameMatchesColumn(node.attrname): self.node = node self.popWordsUpTo(node.attrname) def visitFunction(self, node): if self.checkIfNameMatchesColumn(node.name): self.node = node self.popWordsUpTo(node.name) for arg, default in self.zipArgs(node.argnames, node.defaults): if self.checkIfNameMatchesColumn(arg): self.node = Name(arg) self.popWordsUpTo(arg) if default is not None: self.visit(default) self.visit(node.code) visitAssName = visitName visitAssAttr = visitGetattr def visitClass(self, node): if self.checkIfNameMatchesColumn(node.name): self.node = node self.popWordsUpTo(node.name) for base in node.bases: self.visit(base) def checkIfNameMatchesColumn(self,name): idx = self.getNextIndexOfWord(name) #print "name",name,"idx",idx,"self.col",self.col if idx <= self.col and idx+len(name) > self.col: return 1 return 0 def visitFrom(self, node): for elem in node.modname.split("."): self.popWordsUpTo(elem) for name, alias in node.names: if self.checkIfNameMatchesColumn(name): self.node = self._manufactureASTNodeFromFQN(name) return self.popWordsUpTo(name) if alias is not None: self.popWordsUpTo(alias) # gets round the fact that imports etc dont contain nested getattr # nodes for fqns (e.g. import a.b.bah) by converting the fqn # string into a getattr instance def _manufactureASTNodeFromFQN(self,fqn): if "." in fqn: assert 0, "getattr not supported yet" else: return Name(fqn) def isAMethod(scope,node): return isinstance(node,compiler.ast.Function) and \ isinstance(scope,Class) def convertNodeToMatchObject(node,confidence=100): m = Match() m.sourcenode = node.module.getSourceNode() m.filename = node.filename if isinstance(node,Module): m.lineno = 1 m.colno = 0 elif isinstance(node,Class) or isinstance(node,Function): m.lineno = node.getStartLine() m.colno = node.getColumnOfName() m.confidence = confidence return m |
Update of /cvsroot/pydev/org.python.pydev/PySrc/ThirdParty/brm/bike/refactor In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv9579/PySrc/ThirdParty/brm/bike/refactor Added Files: moveToModule.py utils.py extractVariable.py extractMethod.py inlineVariable.py __init__.py rename.py Log Message: Code completion improvements. Starting refactoring integration with bicycle repair man. --- NEW FILE: inlineVariable.py --- from bike.query.findDefinition import findAllPossibleDefinitionsByCoords from bike.query.findReferences import findReferences from bike.parsing.parserutils import maskStringsAndRemoveComments, linecontinueRE from bike.transformer.undo import getUndoStack from bike.transformer.save import queueFileToSave from parser import ParserError from bike.parsing.load import getSourceNode import compiler import re def inlineLocalVariable(filename, lineno,col): sourceobj = getSourceNode(filename) return inlineLocalVariable_old(sourceobj, lineno,col) def inlineLocalVariable_old(sourcenode,lineno,col): definition, region, regionlinecount = getLocalVariableInfo(sourcenode, lineno, col) addUndo(sourcenode) replaceReferences(sourcenode, findReferences(sourcenode.filename, definition.lineno, definition.colno), region) delLines(sourcenode, definition.lineno-1, regionlinecount) updateSource(sourcenode) def getLocalVariableInfo(sourcenode, lineno, col): definition = findDefinition(sourcenode, lineno, col) region, linecount = getRegionToInline(sourcenode, definition) return definition, region, linecount def findDefinition(sourcenode, lineno, col): definition = findAllPossibleDefinitionsByCoords(sourcenode.filename, lineno,col).next() assert definition.confidence == 100 return definition def getRegionToInline(sourcenode, defn): line, linecount = getLineAndContinues(sourcenode, defn.lineno) start, end = findRegionToInline(maskStringsAndRemoveComments(line)) return line[start:end], linecount def findRegionToInline(maskedline): match = re.compile("[^=]+=\s*(.+)$\n", re.DOTALL).match(maskedline) assert match return match.start(1), match.end(1) # Possible refactoring: move to class of sourcenode def getLineAndContinues(sourcenode, lineno): line = sourcenode.getLine(lineno) linecount = 1 while linecontinueRE.search(line): line += sourcenode.getLine(lineno + linecount) linecount += 1 return line, linecount def addUndo(sourcenode): getUndoStack().addSource(sourcenode.filename,sourcenode.getSource()) def replaceReferences(sourcenode, references, replacement): for reference in safeReplaceOrder( references ): replaceReference(sourcenode, reference, replacement) def safeReplaceOrder( references ): """ When inlining a variable, if multiple instances occur on the line, then the last reference must be replaced first. Otherwise the remaining intra-line references will be incorrect. """ def safeReplaceOrderCmp(self, other): return -cmp(self.colno, other.colno) result = list(references) result.sort(safeReplaceOrderCmp) return result def replaceReference(sourcenode, ref, replacement): """ sourcenode.getLines()[ref.lineno-1][ref.colno:ref.colend] = replacement But strings don't support slice assignment as they are immutable. :( """ sourcenode.getLines()[ref.lineno-1] = \ replaceSubStr(sourcenode.getLines()[ref.lineno-1], ref.colno, ref.colend, replacement) def replaceSubStr(str, start, end, replacement): return str[:start] + replacement + str[end:] # Possible refactoring: move to class of sourcenode def delLines(sourcenode, lineno, linecount=1): del sourcenode.getLines()[lineno:lineno+linecount] def updateSource(sourcenode): queueFileToSave(sourcenode.filename,"".join(sourcenode.getLines())) --- NEW FILE: extractVariable.py --- from bike.parsing.parserutils import maskStringsAndRemoveComments from bike.transformer.undo import getUndoStack from parser import ParserError import compiler from bike.refactor.extractMethod import coords from bike.refactor.utils import getTabWidthOfLine, getLineSeperator,\ reverseCoordsIfWrongWayRound from bike.transformer.save import queueFileToSave from bike.parsing.load import getSourceNode def extractLocalVariable(filename, startcoords, endcoords, varname): sourceobj = getSourceNode(filename) if startcoords.line != endcoords.line: raise "Can't do multi-line extracts yet" startcoords, endcoords = \ reverseCoordsIfWrongWayRound(startcoords,endcoords) line = sourceobj.getLine(startcoords.line) tabwidth = getTabWidthOfLine(line) linesep = getLineSeperator(line) region = line[startcoords.column:endcoords.column] getUndoStack().addSource(sourceobj.filename,sourceobj.getSource()) sourceobj.getLines()[startcoords.line-1] = \ line[:startcoords.column] + varname + line[endcoords.column:] defnline = tabwidth*" " + varname + " = " + region + linesep sourceobj.getLines().insert(startcoords.line-1,defnline) queueFileToSave(sourceobj.filename,"".join(sourceobj.getLines())) --- NEW FILE: extractMethod.py --- import re import compiler from bike.parsing import visitor from bike.query.common import getScopeForLine from bike.parsing.parserutils import generateLogicalLines, \ makeLineParseable, maskStringsAndRemoveComments from parser import ParserError from bike.parsing.fastparserast import Class from bike.transformer.undo import getUndoStack from bike.refactor.utils import getTabWidthOfLine, getLineSeperator, \ reverseCoordsIfWrongWayRound from bike.transformer.save import queueFileToSave from bike.parsing.load import getSourceNode TABSIZE = 4 class coords: def __init__(self, line, column): self.column = column self.line = line def __str__(self): return "("+str(self.column)+","+str(self.line)+")" commentRE = re.compile(r"#.*?$") class ParserException(Exception): pass def extractMethod(filename, startcoords, endcoords, newname): ExtractMethod(getSourceNode(filename), startcoords, endcoords, newname).execute() class ExtractMethod(object): def __init__(self,sourcenode, startcoords, endcoords, newname): self.sourcenode = sourcenode startcoords, endcoords = \ reverseCoordsIfWrongWayRound(startcoords,endcoords) self.startline = startcoords.line self.endline = endcoords.line self.startcol = startcoords.column self.endcol= endcoords.column self.newfn = NewFunction(newname) self.getLineSeperator() self.adjustStartColumnIfLessThanTabwidth() self.adjustEndColumnIfStartsANewLine() self.fn = self.getFunctionObject() self.getRegionToBuffer() #print "-"*80 #print self.extractedLines #print "-"*80 self.deduceIfIsMethodOrFunction() def execute(self): self.deduceArguments() getUndoStack().addSource(self.sourcenode.filename, self.sourcenode.getSource()) srclines = self.sourcenode.getLines() newFnInsertPosition = self.fn.getEndLine()-1 self.insertNewFunctionIntoSrcLines(srclines, self.newfn, newFnInsertPosition) self.writeCallToNewFunction(srclines) src = "".join(srclines) queueFileToSave(self.sourcenode.filename,src) def getLineSeperator(self): line = self.sourcenode.getLines()[self.startline-1] linesep = getLineSeperator(line) self.linesep = linesep def adjustStartColumnIfLessThanTabwidth(self): tabwidth = getTabWidthOfLine(self.sourcenode.getLines()[self.startline-1]) if self.startcol < tabwidth: self.startcol = tabwidth def adjustEndColumnIfStartsANewLine(self): if self.endcol == 0: self.endline -=1 nlSize = len(self.linesep) self.endcol = len(self.sourcenode.getLines()[self.endline-1])-nlSize def getFunctionObject(self): return getScopeForLine(self.sourcenode,self.startline) def getTabwidthOfParentFunction(self): line = self.sourcenode.getLines()[self.fn.getStartLine()-1] match = re.match("\s+",line) if match is None: return 0 else: return match.end(0) # should be in the transformer module def insertNewFunctionIntoSrcLines(self,srclines,newfn,insertpos): tabwidth = self.getTabwidthOfParentFunction() while re.match("\s*"+self.linesep,srclines[insertpos-1]): insertpos -= 1 srclines.insert(insertpos, self.linesep) insertpos +=1 fndefn = "def "+newfn.name+"(" if self.isAMethod: fndefn += "self" if newfn.args != []: fndefn += ", "+", ".join(newfn.args) else: fndefn += ", ".join(newfn.args) fndefn += "):"+self.linesep srclines.insert(insertpos,tabwidth*" "+fndefn) insertpos +=1 tabwidth += TABSIZE if self.extractedCodeIsAnExpression(srclines): assert len(self.extractedLines) == 1 fnbody = [tabwidth*" "+ "return "+self.extractedLines[0]] else: fnbody = [tabwidth*" "+line for line in self.extractedLines] if newfn.retvals != []: fnbody.append(tabwidth*" "+"return "+ ", ".join(newfn.retvals) + self.linesep) for line in fnbody: srclines.insert(insertpos,line) insertpos +=1 def writeCallToNewFunction(self, srclines): startline = self.startline endline = self.endline startcol = self.startcol endcol= self.endcol fncall = self.constructFunctionCallString(self.newfn.name, self.newfn.args, self.newfn.retvals) self.replaceCodeWithFunctionCall(srclines, fncall, startline, endline, startcol, endcol) def replaceCodeWithFunctionCall(self, srclines, fncall, startline, endline, startcol, endcol): if startline == endline: # i.e. extracted code part of existing line line = srclines[startline-1] srclines[startline-1] = self.replaceSectionOfLineWithFunctionCall(line, startcol, endcol, fncall) else: self.replaceLinesWithFunctionCall(srclines, startline, endline, fncall) def replaceLinesWithFunctionCall(self, srclines, startline, endline, fncall): tabwidth = getTabWidthOfLine(srclines[startline-1]) line = tabwidth*" " + fncall + self.linesep srclines[startline-1:endline] = [line] def replaceSectionOfLineWithFunctionCall(self, line, startcol, endcol, fncall): line = line[:startcol] + fncall + line[endcol:] if not line.endswith(self.linesep): line+=self.linesep return line def constructFunctionCallString(self, fnname, fnargs, retvals): fncall = fnname + "("+", ".join(fnargs)+")" if self.isAMethod: fncall = "self." + fncall if retvals != []: fncall = ", ".join(retvals) + " = "+fncall return fncall def deduceArguments(self): lines = self.fn.getLinesNotIncludingThoseBelongingToChildScopes() # strip off comments lines = [commentRE.sub(self.linesep,line) for line in lines] extractedLines = maskStringsAndRemoveComments("".join(self.extractedLines)).splitlines(1) linesbefore = lines[:(self.startline - self.fn.getStartLine())] linesafter = lines[(self.endline - self.fn.getStartLine()) + 1:] # split into logical lines linesbefore = [line for line in generateLogicalLines(linesbefore)] extractedLines = [line for line in generateLogicalLines(extractedLines)] linesafter = [line for line in generateLogicalLines(linesafter)] if self.startline == self.endline: # need to include the line code is extracted from line = generateLogicalLines(lines[self.startline - self.fn.getStartLine():]).next() linesbefore.append(line[:self.startcol] + "dummyFn()" + line[self.endcol:]) assigns = getAssignments(linesbefore) fnargs = getFunctionArgs(linesbefore) candidateArgs = assigns + fnargs refs = getVariableReferencesInLines(extractedLines) self.newfn.args = [ref for ref in refs if ref in candidateArgs] assignsInExtractedBlock = getAssignments(extractedLines) usesAfterNewFunctionCall = getVariableReferencesInLines(linesafter) usesInPreceedingLoop = getVariableReferencesInLines( self.getPreceedingLinesInLoop(linesbefore,line)) self.newfn.retvals = [ref for ref in usesInPreceedingLoop+usesAfterNewFunctionCall if ref in assignsInExtractedBlock] def getPreceedingLinesInLoop(self,linesbefore,firstLineToExtract): if linesbefore == []: return [] tabwidth = getTabWidthOfLine(firstLineToExtract) rootTabwidth = getTabWidthOfLine(linesbefore[0]) llines = [line for line in generateLogicalLines(linesbefore)] startpos = len(llines)-1 loopTabwidth = tabwidth for idx in range(startpos,0,-1): line = llines[idx] if re.match("(\s+)for",line) is not None or \ re.match("(\s+)while",line) is not None: candidateLoopTabwidth = getTabWidthOfLine(line) if candidateLoopTabwidth < loopTabwidth: startpos = idx return llines[startpos:] def getRegionToBuffer(self): startline = self.startline endline = self.endline startcol = self.startcol endcol= self.endcol self.extractedLines = self.sourcenode.getLines()[startline-1:endline] match = re.match("\s*",self.extractedLines[0]) tabwidth = match.end(0) self.extractedLines = [line[startcol:] for line in self.extractedLines] # above cropping can take a blank line's newline off. # this puts it back for idx in range(len(self.extractedLines)): if self.extractedLines[idx] == '': self.extractedLines[idx] = self.linesep if startline == endline: # need to crop the end # (n.b. if region is multiple lines, then whole lines are taken) self.extractedLines[-1] = self.extractedLines[-1][:endcol-startcol] if self.extractedLines[-1][-1] != '\n': self.extractedLines[-1] += self.linesep def extractedCodeIsAnExpression(self,lines): if len(self.extractedLines) == 1: charsBeforeSelection = lines[self.startline-1][:self.startcol] if re.match("^\s*$",charsBeforeSelection) is not None: return 0 if re.search(":\s*$",charsBeforeSelection) is not None: return 0 return 1 return 0 def deduceIfIsMethodOrFunction(self): if isinstance(self.fn.getParent(),Class): self.isAMethod = 1 else: self.isAMethod = 0 # holds information about the new function class NewFunction: def __init__(self,name): self.name = name # lines = list of lines. # Have to have strings masked and comments removed def getAssignments(lines): class AssignVisitor: def __init__(self): self.assigns = [] def visitAssTuple(self, node): for a in node.nodes: if a.name not in self.assigns: self.assigns.append(a.name) def visitAssName(self, node): if node.name not in self.assigns: self.assigns.append(node.name) def visitAugAssign(self, node): if isinstance(node.node, compiler.ast.Name): if node.node.name not in self.assigns: self.assigns.append(node.node.name) assignfinder = AssignVisitor() for line in lines: doctoredline = makeLineParseable(line) try: ast = compiler.parse(doctoredline) except ParserError: raise ParserException("couldnt parse:"+doctoredline) visitor.walk(ast, assignfinder) return assignfinder.assigns # lines = list of lines. # Have to have strings masked and comments removed def getFunctionArgs(lines): if lines == []: return [] class FunctionVisitor: def __init__(self): self.result = [] def visitFunction(self, node): for n in node.argnames: if n != "self": self.result.append(n) fndef = generateLogicalLines(lines).next() doctoredline = makeLineParseable(fndef) try: ast = compiler.parse(doctoredline) except ParserError: raise ParserException("couldnt parse:"+doctoredline) return visitor.walk(ast, FunctionVisitor()).result # lines = list of lines. Have to have strings masked and comments removed def getVariableReferencesInLines(lines): class NameVisitor: def __init__(self): self.result = [] def visitName(self, node): if node.name not in self.result: self.result.append(node.name) reffinder = NameVisitor() for line in lines: doctoredline = makeLineParseable(line) try: ast = compiler.parse(doctoredline) except ParserError: raise ParserException("couldnt parse:"+doctoredline) visitor.walk(ast, reffinder) return reffinder.result --- NEW FILE: rename.py --- from bike.transformer.WordRewriter import WordRewriter from bike.query.findReferences import findReferencesIncludingDefn from bike.transformer.save import save def rename(filename,lineno,col,newname,promptcallback=None): strrewrite = WordRewriter() for match in findReferencesIncludingDefn(filename,lineno,col): #print "rename match ",match if match.confidence == 100 or promptUser(promptcallback,match): strrewrite.rewriteString(match.sourcenode, match.lineno,match.colno,newname) strrewrite.commit() def promptUser(promptCallback,match): if promptCallback is not None and \ promptCallback(match.filename, match.lineno, match.colno, match.colend): return 1 return 0 --- NEW FILE: utils.py --- import re def getLineSeperator(line): if line.endswith("\r\n"): linesep = "\r\n" # windoze else: linesep = line[-1] # mac or unix return linesep def getTabWidthOfLine(line): match = re.match("\s+",line) if match is None: return 0 else: return match.end(0) def reverseCoordsIfWrongWayRound(startcoords,endcoords): if(startcoords.line > endcoords.line) or \ (startcoords.line == endcoords.line and \ startcoords.column > endcoords.column): return endcoords,startcoords else: return startcoords,endcoords --- NEW FILE: moveToModule.py --- import bike.globals from bike.parsing.load import getSourceNode from bike.parsing.fastparserast import Module from bike.query.common import getScopeForLine, convertNodeToMatchObject from bike.transformer.save import queueFileToSave, save from bike.transformer.undo import getUndoStack from bike.refactor.extractMethod import getVariableReferencesInLines from bike.refactor.utils import getLineSeperator from bike.query.findDefinition import findDefinitionFromASTNode from bike.query.findReferences import findReferences from bike.parsing.pathutils import filenameToModulePath from compiler.ast import Name import re def moveClassToNewModule(origfile,line,newfile): srcnode = getSourceNode(origfile) targetsrcnode = getSourceNode(newfile) classnode = getScopeForLine(srcnode,line) classlines = srcnode.getLines()[classnode.getStartLine()-1: classnode.getEndLine()-1] getUndoStack().addSource(srcnode.filename, srcnode.getSource()) getUndoStack().addSource(targetsrcnode.filename, targetsrcnode.getSource()) srcnode.getLines()[classnode.getStartLine()-1: classnode.getEndLine()-1] = [] targetsrcnode.getLines().extend(classlines) queueFileToSave(srcnode.filename,srcnode.getSource()) queueFileToSave(targetsrcnode.filename,targetsrcnode.getSource()) exactFromRE = "(from\s+\S+\s+import\s+%s)(.*)" fromRE = "from\s+\S+\s+import\s+(.*)" def moveFunctionToNewModule(origfile,line,newfile): srcnode = getSourceNode(origfile) targetsrcnode = getSourceNode(newfile) scope = getScopeForLine(srcnode,line) linesep = getLineSeperator(srcnode.getLines()[0]) matches = findReferences(origfile, line, scope.getColumnOfName()) origFileImport = [] fromline = 'from %s import %s'%(filenameToModulePath(newfile),scope.name) for match in matches: if match.filename == origfile: origFileImport = fromline + linesep else: s = getSourceNode(match.filename) m = s.fastparseroot if match.lineno in m.getImportLineNumbers(): getUndoStack().addSource(s.filename, s.getSource()) maskedline = m.getLogicalLine(match.lineno) origline = s.getLines()[match.lineno-1] reMatch = re.match(exactFromRE%(scope.name),maskedline) if reMatch and not (',' in reMatch.group(2) or \ '\\' in reMatch.group(2)): restOfOrigLine = origline[len(reMatch.group(1)):] s.getLines()[match.lineno-1] = fromline + restOfOrigLine elif re.match(fromRE,maskedline): #remove the element from the import stmt line = re.sub('%s\s*?,'%(scope.name),'',origline) s.getLines()[match.lineno-1] = line #and add a new line nextline = match.lineno + maskedline.count('\\') + 1 s.getLines()[nextline-1:nextline-1] = [fromline+linesep] queueFileToSave(s.filename,s.getSource()) refs = getVariableReferencesInLines(scope.getMaskedLines()) scopeLines = srcnode.getLines()[scope.getStartLine()-1: scope.getEndLine()-1] importModules = deduceImportsForNewFile(refs, scope) importlines = composeNewFileImportLines(importModules, linesep) getUndoStack().addSource(srcnode.filename, srcnode.getSource()) getUndoStack().addSource(targetsrcnode.filename, targetsrcnode.getSource()) srcnode.getLines()[scope.getStartLine()-1: scope.getEndLine()-1] = origFileImport targetsrcnode.getLines().extend(importlines+scopeLines) queueFileToSave(srcnode.filename,srcnode.getSource()) queueFileToSave(targetsrcnode.filename,targetsrcnode.getSource()) def composeNewFileImportLines(importModules, linesep): importlines = [] for mpath in importModules: importlines += "from %s import %s"%(mpath, ', '.join(importModules[mpath])) importlines += linesep return importlines def deduceImportsForNewFile(refs, scope): importModules = {} for ref in refs: match = findDefinitionFromASTNode(scope,Name(ref)) if match.filename == scope.module.filename: tgtscope = getScopeForLine(getSourceNode(match.filename), match.lineno) while tgtscope != scope and not isinstance(tgtscope,Module): tgtscope = tgtscope.getParent() if not isinstance(tgtscope,Module): continue # was defined in this function mpath = filenameToModulePath(match.filename) if mpath in importModules: importModules[mpath].append(ref) else: importModules[mpath] = [ref] return importModules --- NEW FILE: __init__.py --- |
From: Fabio Z. <fa...@us...> - 2004-09-14 17:41:45
|
Update of /cvsroot/pydev/org.python.pydev/PySrc/ThirdParty/brm/bike/transformer In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv9473/PySrc/ThirdParty/brm/bike/transformer Log Message: Directory /cvsroot/pydev/org.python.pydev/PySrc/ThirdParty/brm/bike/transformer added to the repository |
From: Fabio Z. <fa...@us...> - 2004-09-14 17:41:45
|
Update of /cvsroot/pydev/org.python.pydev/src/org/python/pydev/editor/refactoring In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv9473/src/org/python/pydev/editor/refactoring Log Message: Directory /cvsroot/pydev/org.python.pydev/src/org/python/pydev/editor/refactoring added to the repository |
From: Fabio Z. <fa...@us...> - 2004-09-14 17:41:44
|
Update of /cvsroot/pydev/org.python.pydev/PySrc/ThirdParty/brm/bike/refactor In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv9473/PySrc/ThirdParty/brm/bike/refactor Log Message: Directory /cvsroot/pydev/org.python.pydev/PySrc/ThirdParty/brm/bike/refactor added to the repository |
From: Fabio Z. <fa...@us...> - 2004-09-14 17:41:44
|
Update of /cvsroot/pydev/org.python.pydev/PySrc/Refactoring In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv9473/PySrc/Refactoring Log Message: Directory /cvsroot/pydev/org.python.pydev/PySrc/Refactoring added to the repository |
From: Fabio Z. <fa...@us...> - 2004-09-14 17:41:44
|
Update of /cvsroot/pydev/org.python.pydev/PySrc/ThirdParty/brm/bike In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv9473/PySrc/ThirdParty/brm/bike Log Message: Directory /cvsroot/pydev/org.python.pydev/PySrc/ThirdParty/brm/bike added to the repository |
From: Fabio Z. <fa...@us...> - 2004-09-14 17:41:44
|
Update of /cvsroot/pydev/org.python.pydev/PySrc/ThirdParty In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv9473/PySrc/ThirdParty Log Message: Directory /cvsroot/pydev/org.python.pydev/PySrc/ThirdParty added to the repository |
From: Fabio Z. <fa...@us...> - 2004-09-14 17:41:43
|
Update of /cvsroot/pydev/org.python.pydev/PySrc/ThirdParty/brm/bike/parsing In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv9473/PySrc/ThirdParty/brm/bike/parsing Log Message: Directory /cvsroot/pydev/org.python.pydev/PySrc/ThirdParty/brm/bike/parsing added to the repository |
From: Fabio Z. <fa...@us...> - 2004-09-14 17:41:43
|
Update of /cvsroot/pydev/org.python.pydev/PySrc/ThirdParty/brm/bike/query In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv9473/PySrc/ThirdParty/brm/bike/query Log Message: Directory /cvsroot/pydev/org.python.pydev/PySrc/ThirdParty/brm/bike/query added to the repository |
From: Fabio Z. <fa...@us...> - 2004-09-14 17:41:43
|
Update of /cvsroot/pydev/org.python.pydev/PySrc/ThirdParty/brm In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv9473/PySrc/ThirdParty/brm Log Message: Directory /cvsroot/pydev/org.python.pydev/PySrc/ThirdParty/brm added to the repository |
From: Fabio Z. <fa...@us...> - 2004-09-14 17:41:37
|
Update of /cvsroot/pydev/org.python.pydev.debug/src/org/python/pydev/debug/ui/launching In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv9450/src/org/python/pydev/debug/ui/launching Modified Files: PythonRunnerConfig.java Removed Files: SocketUtil.java Log Message: Code completion improvements. Starting refactoring integration with bicycle repair man. Index: PythonRunnerConfig.java =================================================================== RCS file: /cvsroot/pydev/org.python.pydev.debug/src/org/python/pydev/debug/ui/launching/PythonRunnerConfig.java,v retrieving revision 1.8 retrieving revision 1.9 diff -C2 -d -r1.8 -r1.9 *** PythonRunnerConfig.java 25 Jul 2004 13:53:51 -0000 1.8 --- PythonRunnerConfig.java 14 Sep 2004 17:41:27 -0000 1.9 *************** *** 24,27 **** --- 24,28 ---- import org.python.pydev.debug.core.PydevDebugPlugin; import org.python.pydev.plugin.PydevPrefs; + import org.python.pydev.plugin.SocketUtil; /** --- SocketUtil.java DELETED --- |
From: Fabio Z. <fa...@us...> - 2004-09-13 19:48:04
|
Update of /cvsroot/pydev/org.python.pydev/tests/org/python/pydev/editor/codecompletion In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv4324/tests/org/python/pydev/editor/codecompletion Modified Files: PythonShellTest.java Log Message: self. getting instance variables too. Index: PythonShellTest.java =================================================================== RCS file: /cvsroot/pydev/org.python.pydev/tests/org/python/pydev/editor/codecompletion/PythonShellTest.java,v retrieving revision 1.1 retrieving revision 1.2 diff -C2 -d -r1.1 -r1.2 *** PythonShellTest.java 13 Sep 2004 17:11:49 -0000 1.1 --- PythonShellTest.java 13 Sep 2004 19:47:54 -0000 1.2 *************** *** 78,86 **** public void testOther(){ ! String str = "class C(object): \n"+ " \n"+ " def __init__(self): \n"+ - " \n"+ " print dir(self) \n"+ " \n"+ --- 78,106 ---- public void testOther(){ ! String str = getTestStr(); ! ! List list = shell.getClassCompletions("C",str); ! assertEquals(18, list.size()); ! ! list = shell.getTokenCompletions("C",str); ! assertEquals(17, list.size()); ! // for (Iterator iter = list.iterator(); iter.hasNext();) { ! // Object[] element = (Object[]) iter.next(); ! // System.out.println(element[0]); ! // System.out.println(element[1]); ! // } ! ! ! ! } ! ! /** ! * @return ! */ ! private String getTestStr() { ! String str = "class C(object): \n"+ " \n"+ " def __init__(self): \n"+ " print dir(self) \n"+ " \n"+ *************** *** 90,104 **** " \n"+ " def b(self): \n"+ ! " self.a \n"+ ! " \n"+ " pass \n"; ! ! List list = shell.getTokenCompletions("C",str); ! assertEquals(17, list.size()); ! // for (Iterator iter = list.iterator(); iter.hasNext();) { ! // Object[] element = (Object[]) iter.next(); ! // System.out.println(element[0]); ! // System.out.println(element[1]); ! // } ! } } --- 110,116 ---- " \n"+ " def b(self): \n"+ ! " self.c=1 \n"+ " pass \n"; ! return str; ! } } |
From: Fabio Z. <fa...@us...> - 2004-09-13 19:48:04
|
Update of /cvsroot/pydev/org.python.pydev/PySrc In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv4324/PySrc Modified Files: simpleinspect.py test_pyserver.py simpleTipper.py pycompletionserver.py test_simpleTipper.py Log Message: self. getting instance variables too. Index: test_simpleTipper.py =================================================================== RCS file: /cvsroot/pydev/org.python.pydev/PySrc/test_simpleTipper.py,v retrieving revision 1.1 retrieving revision 1.2 diff -C2 -d -r1.1 -r1.2 *** test_simpleTipper.py 10 Sep 2004 19:42:23 -0000 1.1 --- test_simpleTipper.py 13 Sep 2004 19:47:54 -0000 1.2 *************** *** 42,46 **** def testEnv1(self): ! comps = simpleTipper.GenerateTip(self.getDoc1(), None) import math, inspect --- 42,46 ---- def testEnv1(self): ! comps = simpleTipper.GenerateTip(self.getDoc1(), None, True) import math, inspect *************** *** 61,65 **** def testEnv1CToken(self): ! comps = simpleTipper.GenerateTip(self.getDoc1(), 'C') checkedA = False for tup in comps: --- 61,65 ---- def testEnv1CToken(self): ! comps = simpleTipper.GenerateTip(self.getDoc1(), 'C', True) checkedA = False for tup in comps: *************** *** 69,72 **** --- 69,98 ---- self.assert_(checkedA) + + + def getDoc2(self): + s = \ + ''' + class C(object): + def __init__(self): + self.a = 1 + self.b = 2 + ''' + return s + + def testEnv2(self): + ''' + Now, check completion for C - should return object methods, 'a' and 'b' + ''' + comps = simpleTipper.GenerateTip(self.getDoc2(), 'C', True) + # print comps + checkedA = False + for tup in comps: + if tup[0] == 'a': + checkedA = True + self.assert_(checkedA) + # self.assert_(('a',' ') in comps) + + if __name__ == '__main__': Index: simpleinspect.py =================================================================== RCS file: /cvsroot/pydev/org.python.pydev/PySrc/simpleinspect.py,v retrieving revision 1.1 retrieving revision 1.2 diff -C2 -d -r1.1 -r1.2 *** simpleinspect.py 10 Sep 2004 19:42:23 -0000 1.1 --- simpleinspect.py 13 Sep 2004 19:47:54 -0000 1.2 *************** *** 1,7 **** ''' @author Fabio Zadrozny ''' ! def GenerateTip (__eraseThisV): exec(__eraseThisV) --- 1,9 ---- ''' + Do not pollute this namespace! + @author Fabio Zadrozny ''' ! def GenerateTip (__eraseThisV, __eraseThisToken): exec(__eraseThisV) Index: simpleTipper.py =================================================================== RCS file: /cvsroot/pydev/org.python.pydev/PySrc/simpleTipper.py,v retrieving revision 1.2 retrieving revision 1.3 diff -C2 -d -r1.2 -r1.3 *** simpleTipper.py 13 Sep 2004 17:11:51 -0000 1.2 --- simpleTipper.py 13 Sep 2004 19:47:54 -0000 1.3 *************** *** 2,11 **** @author Fabio Zadrozny ''' ! def GenerateTip (theDoc, token): ''' Put in the doc the code so that we get the locals. ''' if token is None: --- 2,13 ---- @author Fabio Zadrozny ''' + import compiler ! def GenerateTip (theDoc, token, checkForSelf): ''' Put in the doc the code so that we get the locals. ''' + originalDoc = theDoc if token is None: *************** *** 40,57 **** import simpleinspect - import compiler try: ! __eraseThis = compiler.compile(theDoc, 'temporary', 'exec') ! simpleinspect.__eraseThisTips = [] ! simpleinspect.GenerateTip (__eraseThis) toReturn = simpleinspect.__eraseThisTips simpleinspect.__eraseThisTips = [] return toReturn except : import sys s = str(sys.exc_info()[1]) return [('ERROR_COMPLETING',s)] \ No newline at end of file --- 42,109 ---- import simpleinspect try: ! __eraseThis = compiler.compile(theDoc, 'temporary_file_completion.py', 'exec') ! simpleinspect.__eraseThisTips = [] ! simpleinspect.GenerateTip (__eraseThis, token) toReturn = simpleinspect.__eraseThisTips simpleinspect.__eraseThisTips = [] + + if checkForSelf: + toReturn += GetSelfVariables(originalDoc, token) + return toReturn except : import sys s = str(sys.exc_info()[1]) + print s return [('ERROR_COMPLETING',s)] + class Visitor(compiler.visitor.ASTVisitor): + + def __init__(self, classToVisit): + self.classToVisit = classToVisit + self.selfAttribs = [] + + def visitClass(self, node): + # print node.name + if node.name == self.classToVisit: + for n in node.getChildNodes(): + self.visit(n) + + def visitAssign(self, node): + + for n in node.getChildNodes(): + if isinstance(n,compiler.ast.AssAttr): + if n.expr.name == 'self': + self.selfAttribs.append((n.attrname,'Instance attribute')) + + + def GetSelfVariables(theDoc, classToVisit): + ast = compiler.parse(theDoc) + + visitor = Visitor(classToVisit) + compiler.walk(ast, visitor) + + return visitor.selfAttribs + + + + + + + + + + + + + + + + + + + \ No newline at end of file Index: test_pyserver.py =================================================================== RCS file: /cvsroot/pydev/org.python.pydev/PySrc/test_pyserver.py,v retrieving revision 1.2 retrieving revision 1.3 diff -C2 -d -r1.2 -r1.3 *** test_pyserver.py 13 Sep 2004 17:11:50 -0000 1.2 --- test_pyserver.py 13 Sep 2004 19:47:54 -0000 1.3 *************** *** 76,80 **** def b(self): ! self.a pass --- 76,80 ---- def b(self): ! self.c=1 pass *************** *** 84,87 **** --- 84,91 ---- completions = connToRead.recv(4086) + sToWrite.send('@@CLASS_GLOBALS(C):%s\nEND@@'%s) + completions2 = connToRead.recv(4086) + self.assert_(len(completions) != len(completions2)) + self.sendKillMsg(sToWrite) Index: pycompletionserver.py =================================================================== RCS file: /cvsroot/pydev/org.python.pydev/PySrc/pycompletionserver.py,v retrieving revision 1.4 retrieving revision 1.5 diff -C2 -d -r1.4 -r1.5 *** pycompletionserver.py 13 Sep 2004 17:11:50 -0000 1.4 --- pycompletionserver.py 13 Sep 2004 19:47:54 -0000 1.5 *************** *** 14,17 **** --- 14,18 ---- MSG_GLOBALS = '@@GLOBALS:' MSG_TOKEN_GLOBALS = '@@TOKEN_GLOBALS(' + MSG_CLASS_GLOBALS = '@@CLASS_GLOBALS(' MSG_INVALID_REQUEST = '@@INVALID_REQUEST' *************** *** 108,112 **** if MSG_GLOBALS in data: data = data.replace(MSG_GLOBALS, '') ! comps = simpleTipper.GenerateTip(data, None) self.sendCompletionsMessage(comps) --- 109,113 ---- if MSG_GLOBALS in data: data = data.replace(MSG_GLOBALS, '') ! comps = simpleTipper.GenerateTip(data, None, False) self.sendCompletionsMessage(comps) *************** *** 114,118 **** data = data.replace(MSG_TOKEN_GLOBALS, '') token, data = self.getTokenAndData(data) ! comps = simpleTipper.GenerateTip(data, token) self.sendCompletionsMessage(comps) --- 115,125 ---- data = data.replace(MSG_TOKEN_GLOBALS, '') token, data = self.getTokenAndData(data) ! comps = simpleTipper.GenerateTip(data, token, False) ! self.sendCompletionsMessage(comps) ! ! elif MSG_CLASS_GLOBALS in data: ! data = data.replace(MSG_CLASS_GLOBALS, '') ! token, data = self.getTokenAndData(data) ! comps = simpleTipper.GenerateTip(data, token, True) self.sendCompletionsMessage(comps) |
From: Fabio Z. <fa...@us...> - 2004-09-13 19:48:04
|
Update of /cvsroot/pydev/org.python.pydev/src/org/python/pydev/editor/codecompletion In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv4324/src/org/python/pydev/editor/codecompletion Modified Files: PythonShell.java PyCodeCompletion.java Log Message: self. getting instance variables too. Index: PythonShell.java =================================================================== RCS file: /cvsroot/pydev/org.python.pydev/src/org/python/pydev/editor/codecompletion/PythonShell.java,v retrieving revision 1.2 retrieving revision 1.3 diff -C2 -d -r1.2 -r1.3 *** PythonShell.java 13 Sep 2004 17:11:49 -0000 1.2 --- PythonShell.java 13 Sep 2004 19:47:54 -0000 1.3 *************** *** 269,272 **** --- 269,301 ---- /** + * @param token + * @param docToParse + * @return + */ + public List getClassCompletions(String token, String str) { + String s = "@@CLASS_GLOBALS("+token+"):"+str+"\nEND@@"; + try { + this.write(s); + + return getCompletions(); + } catch (IOException e) { + e.printStackTrace(); + try { + this.endIt(); + } catch (IOException e1) { + // TODO Auto-generated catch block + e1.printStackTrace(); + } + try { + this.startIt(); + } catch (IOException e2) { + // TODO Auto-generated catch block + e2.printStackTrace(); + } + return getInvalidCompletion(); + } + } + + /** * @return */ *************** *** 294,296 **** --- 323,327 ---- } + + } \ No newline at end of file Index: PyCodeCompletion.java =================================================================== RCS file: /cvsroot/pydev/org.python.pydev/src/org/python/pydev/editor/codecompletion/PyCodeCompletion.java,v retrieving revision 1.4 retrieving revision 1.5 diff -C2 -d -r1.4 -r1.5 *** PyCodeCompletion.java 13 Sep 2004 17:11:50 -0000 1.4 --- PyCodeCompletion.java 13 Sep 2004 19:47:54 -0000 1.5 *************** *** 97,110 **** String docToParse = getDocToParse(doc, documentOffset); ! if (theActivationToken.replace('.',' ').trim().equals("self")){ ! // System.out.println("serverCompletion - self."); ! Location loc = Location.offsetToLocation(doc, documentOffset); ! AbstractNode closest = ModelUtils.getLessOrEqualNode(edit.getPythonModel(),loc); ! Scope scope = closest.getScope().findContainingClass(); ! String token = scope.getStartNode().getName(); ! ! List completions = serverShell.getTokenCompletions(token, docToParse); theList.addAll(completions); --- 97,114 ---- String docToParse = getDocToParse(doc, documentOffset); ! String trimmed = theActivationToken.replace('.',' ').trim(); ! if (trimmed.equals("") == false){ ! List completions; ! if (trimmed.equals("self")){ ! Location loc = Location.offsetToLocation(doc, documentOffset); ! AbstractNode closest = ModelUtils.getLessOrEqualNode(edit.getPythonModel(),loc); ! Scope scope = closest.getScope().findContainingClass(); ! String token = scope.getStartNode().getName(); ! completions = serverShell.getClassCompletions(token, docToParse); ! }else{ ! completions = serverShell.getTokenCompletions(trimmed, docToParse); ! } theList.addAll(completions); *************** *** 147,151 **** newDoc += wholeDoc.substring(lineInformation.getOffset() + lineInformation.getLength(), docLength); ! System.out.println("DOC:"+newDoc); } catch (BadLocationException e1) { --- 151,155 ---- newDoc += wholeDoc.substring(lineInformation.getOffset() + lineInformation.getLength(), docLength); ! // System.out.println("DOC:"+newDoc); } catch (BadLocationException e1) { |
From: Fabio Z. <fa...@us...> - 2004-09-13 18:23:19
|
Update of /cvsroot/pydev/org.python.pydev/src/org/python/pydev/editor/codecompletion In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv18177/src/org/python/pydev/editor/codecompletion Modified Files: PyCodeCompletionPreferencesPage.java Log Message: Index: PyCodeCompletionPreferencesPage.java =================================================================== RCS file: /cvsroot/pydev/org.python.pydev/src/org/python/pydev/editor/codecompletion/PyCodeCompletionPreferencesPage.java,v retrieving revision 1.2 retrieving revision 1.3 diff -C2 -d -r1.2 -r1.3 *** PyCodeCompletionPreferencesPage.java 13 Sep 2004 17:11:49 -0000 1.2 --- PyCodeCompletionPreferencesPage.java 13 Sep 2004 18:23:09 -0000 1.3 *************** *** 68,72 **** addField(new BooleanFieldEditor( USE_SERVER_TIP, "Use server tipper enviroment (otherwise console is used)?", p)); ! } /* --- 68,72 ---- addField(new BooleanFieldEditor( USE_SERVER_TIP, "Use server tipper enviroment (otherwise console is used)?", p)); ! } /* |
From: Fabio Z. <fa...@us...> - 2004-09-13 17:12:34
|
Update of /cvsroot/pydev/org.python.pydev/src/org/python/pydev/editor/codecompletion In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv1125/src/org/python/pydev/editor/codecompletion Modified Files: PyTemplateCompletion.java PythonShell.java PyCodeCompletionPreferencesPage.java PythonCompletionProcessor.java CompletionCache.java PyCodeCompletion.java Log Message: New code completion. Index: PyCodeCompletionPreferencesPage.java =================================================================== RCS file: /cvsroot/pydev/org.python.pydev/src/org/python/pydev/editor/codecompletion/PyCodeCompletionPreferencesPage.java,v retrieving revision 1.1 retrieving revision 1.2 diff -C2 -d -r1.1 -r1.2 *** PyCodeCompletionPreferencesPage.java 25 Aug 2004 17:25:23 -0000 1.1 --- PyCodeCompletionPreferencesPage.java 13 Sep 2004 17:11:49 -0000 1.2 *************** *** 36,39 **** --- 36,42 ---- public static final boolean DEFAULT_AUTOCOMPLETE_ON_PAR = false; + public static final String USE_SERVER_TIP = "USE_SERVER_TIP"; + public static final boolean DEFAULT_USE_SERVER_TIP = false; + /** */ *************** *** 62,65 **** --- 65,71 ---- addField(new BooleanFieldEditor( AUTOCOMPLETE_ON_PAR, "Autocomplete on '('?", p)); + + addField(new BooleanFieldEditor( + USE_SERVER_TIP, "Use server tipper enviroment (otherwise console is used)?", p)); } *************** *** 81,84 **** --- 87,91 ---- prefs.setDefault(AUTOCOMPLETE_DELAY, DEFAULT_AUTOCOMPLETE_DELAY); prefs.setDefault(AUTOCOMPLETE_ON_PAR, DEFAULT_AUTOCOMPLETE_ON_PAR); + prefs.setDefault(USE_SERVER_TIP, DEFAULT_USE_SERVER_TIP); } *************** *** 99,102 **** --- 106,113 ---- } + public static boolean useServerTipEnviroment() { + return PydevPrefs.getPreferences().getBoolean(PyCodeCompletionPreferencesPage.USE_SERVER_TIP); + } + /* (non-Javadoc) * @see org.eclipse.core.runtime.Preferences.IPropertyChangeListener#propertyChange(org.eclipse.core.runtime.Preferences.PropertyChangeEvent) Index: PyTemplateCompletion.java =================================================================== RCS file: /cvsroot/pydev/org.python.pydev/src/org/python/pydev/editor/codecompletion/PyTemplateCompletion.java,v retrieving revision 1.1 retrieving revision 1.2 diff -C2 -d -r1.1 -r1.2 *** PyTemplateCompletion.java 11 Aug 2004 13:04:13 -0000 1.1 --- PyTemplateCompletion.java 13 Sep 2004 17:11:49 -0000 1.2 *************** *** 6,11 **** --- 6,13 ---- package org.python.pydev.editor.codecompletion; + import java.io.File; import java.util.List; + import org.eclipse.core.runtime.CoreException; import org.eclipse.jface.text.BadLocationException; import org.eclipse.jface.text.IRegion; *************** *** 51,55 **** */ protected Image getImage(Template template) { ! // TODO Auto-generated method stub return null; } --- 53,62 ---- */ protected Image getImage(Template template) { ! try { ! File file = PyCodeCompletion.getImageWithinIcons("template.gif"); ! return new Image(null, file.getAbsolutePath()); ! } catch (CoreException e) { ! e.printStackTrace(); ! } return null; } Index: PythonCompletionProcessor.java =================================================================== RCS file: /cvsroot/pydev/org.python.pydev/src/org/python/pydev/editor/codecompletion/PythonCompletionProcessor.java,v retrieving revision 1.3 retrieving revision 1.4 diff -C2 -d -r1.3 -r1.4 *** PythonCompletionProcessor.java 25 Aug 2004 17:25:41 -0000 1.3 --- PythonCompletionProcessor.java 13 Sep 2004 17:11:50 -0000 1.4 *************** *** 16,19 **** --- 16,20 ---- import org.eclipse.jface.text.contentassist.IContextInformationValidator; import org.eclipse.swt.graphics.Point; + import org.python.pydev.editor.PyEdit; /** *************** *** 31,34 **** --- 32,44 ---- private CompletionCache completionCache = new CompletionCache(); + private PyEdit edit; + + /** + * @param edit + */ + public PythonCompletionProcessor(PyEdit edit) { + this.edit = edit; + } + private boolean endsWithSomeChar(char cs[], String activationToken) { for (int i = 0; i < cs.length; i++) { *************** *** 43,48 **** public ICompletionProposal[] computeCompletionProposals(ITextViewer viewer, int documentOffset) { ! List propList = new ArrayList(); IDocument doc = viewer.getDocument(); Point selectedRange = viewer.getSelectedRange(); --- 53,59 ---- public ICompletionProposal[] computeCompletionProposals(ITextViewer viewer, int documentOffset) { ! IDocument doc = viewer.getDocument(); + Point selectedRange = viewer.getSelectedRange(); *************** *** 55,67 **** java.lang.String qualifier = ""; ! char[] cs = getCompletionProposalAutoActivationCharacters(); ! ! while (endsWithSomeChar(cs, activationToken) == false ! && activationToken.length() > 0) { ! qualifier = activationToken.charAt(activationToken.length() - 1) ! + qualifier; ! activationToken = activationToken.substring(0, activationToken ! .length() - 1); } --- 66,85 ---- java.lang.String qualifier = ""; ! char[] cs = new char[]{'.', '(', ' '}; ! ! //we complete on '.' and '('. ! //' ' gets globals ! //and any other char gets globals on token and templates. ! //we have to get the qualifier. e.g. bla.foo = foo is the qualifier. ! if(activationToken.indexOf('.')!= -1){ ! while (endsWithSomeChar(cs, activationToken) == false ! && activationToken.length() > 0) { ! ! qualifier = activationToken.charAt(activationToken.length() - 1) ! + qualifier; ! activationToken = activationToken.substring(0, activationToken ! .length() - 1); ! } } *************** *** 71,92 **** theDoc += "\n" + activationToken; ! List allProposals = this.completionCache.getAllProposals(theDoc, ! activationToken, documentOffset, qlen, codeCompletion); ! //templates proposals are added here. ! this.templatesCompletion.addTemplateProposals(viewer, documentOffset, ! propList); ! for (Iterator iter = allProposals.iterator(); iter.hasNext();) { ICompletionProposal proposal = (ICompletionProposal) iter.next(); if (proposal.getDisplayString().startsWith(qualifier)) { ! propList.add(proposal); } } ! ICompletionProposal[] proposals = new ICompletionProposal[propList ! .size()]; // and fill with list elements ! propList.toArray(proposals); // Return the proposals return proposals; --- 89,113 ---- theDoc += "\n" + activationToken; ! List pythonProposals = getPythonProposals(documentOffset, doc, theDoc, activationToken, qlen); ! List templateProposals = getTemplateProposals(viewer, documentOffset, activationToken, qualifier, pythonProposals); ! ArrayList pythonAndTemplateProposals = new ArrayList(); ! pythonAndTemplateProposals.addAll(pythonProposals); ! pythonAndTemplateProposals.addAll(templateProposals); ! ! ArrayList returnProposals = new ArrayList(); ! ! for (Iterator iter = pythonAndTemplateProposals.iterator(); iter.hasNext();) { ICompletionProposal proposal = (ICompletionProposal) iter.next(); if (proposal.getDisplayString().startsWith(qualifier)) { ! returnProposals.add(proposal); } } ! ICompletionProposal[] proposals = new ICompletionProposal[returnProposals.size()]; ! // and fill with list elements ! returnProposals.toArray(proposals); // Return the proposals return proposals; *************** *** 94,97 **** --- 115,150 ---- } + /** + * @param documentOffset + * @param doc + * @param theDoc + * @param activationToken + * @param qlen + * @return + */ + private List getPythonProposals(int documentOffset, IDocument doc, java.lang.String theDoc, String activationToken, int qlen) { + List allProposals = this.completionCache.getAllProposals(edit, doc, theDoc, + activationToken, documentOffset, qlen, codeCompletion); + return allProposals; + } + + /** + * @param viewer + * @param documentOffset + * @param activationToken + * @param qualifier + * @param allProposals + */ + private List getTemplateProposals(ITextViewer viewer, int documentOffset, String activationToken, java.lang.String qualifier, List allProposals) { + List propList = new ArrayList(); + if(activationToken.trim().equals("") == false){ + //templates proposals are added here. + this.templatesCompletion.addTemplateProposals(viewer, documentOffset, + propList); + + } + return propList; + } + /* * (non-Javadoc) Index: PythonShell.java =================================================================== RCS file: /cvsroot/pydev/org.python.pydev/src/org/python/pydev/editor/codecompletion/PythonShell.java,v retrieving revision 1.1 retrieving revision 1.2 diff -C2 -d -r1.1 -r1.2 *** PythonShell.java 27 Aug 2004 17:12:00 -0000 1.1 --- PythonShell.java 13 Sep 2004 17:11:49 -0000 1.2 *************** *** 6,60 **** package org.python.pydev.editor.codecompletion; - import java.io.BufferedInputStream; import java.io.File; import java.io.IOException; import java.net.Socket; import org.eclipse.core.runtime.CoreException; /** ! * ! * TODO: THIS IS STILL ONLY A TEST!! ! * * @author Fabio Zadrozny */ public class PythonShell { ! public Process p; ! private Socket socket; ! public static final String END_MSG = "@END@"; - public PythonShell() throws IOException, CoreException { - startIt(); - } /** * @throws CoreException */ ! public void startIt() throws IOException, CoreException { ! ! File serverFile = new File("D:\\dev_programs\\eclipse_3\\eclipse\\workspace\\org.python.pydev\\PySrc\\pycompletionserver.py"); ! //PyCodeCompletion.getScriptWithinPySrc("pycompletionserver.py"); ! ! p = Runtime.getRuntime().exec("python "+serverFile.getAbsolutePath()); ! ! sleepALittle(); ! ! socket = new Socket("127.0.0.1",50007); ! ! write("TESTE"+END_MSG); ! String b = read(); ! System.out.println(b); } ! /** * */ private void sleepALittle() { try { synchronized(this){ ! wait(100); } } catch (InterruptedException e) { --- 6,94 ---- package org.python.pydev.editor.codecompletion; import java.io.File; import java.io.IOException; + import java.net.ServerSocket; import java.net.Socket; + import java.util.ArrayList; + import java.util.List; + import java.util.StringTokenizer; import org.eclipse.core.runtime.CoreException; /** ! * TODO: Still only a test!! * @author Fabio Zadrozny */ public class PythonShell { ! ! public static final int BUFFER_SIZE = 1024; ! /** ! * Python server process. ! */ ! public Process process; ! ! /** ! * We should write in this socket. ! */ ! private Socket socketToWrite; ! ! /** ! * We should read this socket. ! */ ! private Socket socketToRead; ! ! /** ! * Python file that works as the server. ! */ ! private File serverFile; + /** + * Server socket (accept connections). + */ + private ServerSocket serverSocket; /** + * Initialize given the file that points to the python server (execute it + * with python). + * + * @param f file pointing to the python server + * + * @throws IOException * @throws CoreException */ ! public PythonShell(File f) throws IOException, CoreException { ! serverFile = f; ! if(!serverFile.exists()){ ! throw new RuntimeException("Can't find python server file"); ! } } ! /** + * Initialize with the default python server file. * + * @throws IOException + * @throws CoreException + */ + public PythonShell() throws IOException, CoreException { + this(PyCodeCompletion.getScriptWithinPySrc("pycompletionserver.py")); + } + + /** + * Just wait a little... */ private void sleepALittle() { + sleepALittle(25); //25 millis + } + + /** + * Just wait a little... + */ + private void sleepALittle(int t) { try { synchronized(this){ ! wait(t); //millis } } catch (InterruptedException e) { *************** *** 64,92 **** /** ! * @throws IOException */ ! private void closeConn() throws IOException { ! if(socket != null){ ! socket.getOutputStream().write("".getBytes()); ! socket.close(); } - socket = null; } /** ! * @return * @throws IOException */ public String read() throws IOException { String str = ""; ! while(str.endsWith(END_MSG)){ ! BufferedInputStream inputStream = new BufferedInputStream(socket.getInputStream(), 3); ! int size = inputStream.available(); ! byte b[] = new byte[size]; ! inputStream.read(b); ! str += new String(b); } ! return str; } /** --- 98,148 ---- /** ! * This method creates the python server process and starts the sockets, so that we ! * can talk with the server. ! * ! * @throws IOException is some error happens creating the sockets - the process is terminated. */ ! public void startIt() throws IOException{ ! try { ! process = Runtime.getRuntime().exec("python "+serverFile.getAbsolutePath()+" 50007 50008"); ! ! sleepALittle(); ! ! socketToWrite = new Socket("127.0.0.1",50007); //we should write in port 50007 ! serverSocket = new ServerSocket(50008); //and read in port 50008 ! socketToRead = serverSocket.accept(); ! } catch (IOException e) { ! ! if(process!=null){ ! process.destroy(); ! } ! e.printStackTrace(); ! throw e; } } + /** ! * @return s string with the contents read. * @throws IOException */ public String read() throws IOException { String str = ""; ! ! while(str.indexOf("END@@") == -1 ){ ! byte[] b = new byte[PythonShell.BUFFER_SIZE]; ! ! this.socketToRead.getInputStream().read(b); ! String s = new String(b); ! ! str += s; } ! ! //remove @@COMPLETIONS ! str = str.replaceFirst("@@COMPLETIONS",""); ! //remove END@@ ! return str.substring(0, str.indexOf("END@@")); } + /** *************** *** 95,105 **** */ public void write(String str) throws IOException { ! socket.getOutputStream().write(str.getBytes()); } /** * Kill our sub-process. */ ! private void endIt() { try { closeConn(); --- 151,186 ---- */ public void write(String str) throws IOException { ! this.socketToWrite.getOutputStream().write(str.getBytes()); ! } ! ! ! /** ! * @throws IOException ! */ ! private void closeConn() throws IOException { ! write("@@KILL_SERVER_END@@"); ! if(socketToWrite != null){ ! socketToWrite.close(); ! } ! socketToWrite = null; ! ! if(socketToRead != null){ ! socketToRead.close(); ! } ! socketToRead = null; ! ! if(serverSocket != null){ ! serverSocket.close(); ! } ! serverSocket = null; } + /** * Kill our sub-process. + * @throws IOException */ ! void endIt() throws IOException { ! try { closeConn(); *************** *** 107,120 **** e.printStackTrace(); } ! if (p!= null){ ! p.destroy(); ! p = null; } } ! ! public static void main(String[] args) throws IOException, CoreException{ ! PythonShell shell = new PythonShell(); ! shell.endIt(); } --- 188,295 ---- e.printStackTrace(); } ! if (process!= null){ ! int i = process.getErrorStream().available(); ! byte b[] = new byte[i]; ! process.getErrorStream().read(b); ! System.out.println(new String(b)); ! ! i = process.getInputStream().available(); ! b = new byte[i]; ! process.getErrorStream().read(b); ! System.out.println(new String(b)); ! ! process.destroy(); ! try { ! process.waitFor(); ! } catch (InterruptedException e1) { ! e1.printStackTrace(); ! } ! process = null; } } ! ! ! ! /** ! * @param str ! * @throws IOException ! */ ! public List getGlobalCompletions(String str) { ! try { ! this.write("@@GLOBALS:"+str+"\nEND@@"); ! ! return getCompletions(); ! } catch (IOException e) { ! e.printStackTrace(); ! try { ! this.endIt(); ! } catch (IOException e1) { ! // TODO Auto-generated catch block ! e1.printStackTrace(); ! } ! try { ! this.startIt(); ! } catch (IOException e2) { ! // TODO Auto-generated catch block ! e2.printStackTrace(); ! } ! return getInvalidCompletion(); ! } ! ! } ! ! /** ! * @param str ! * @throws IOException ! */ ! public List getTokenCompletions(String token, String str) { ! String s = "@@TOKEN_GLOBALS("+token+"):"+str+"\nEND@@"; ! try { ! this.write(s); ! ! return getCompletions(); ! } catch (IOException e) { ! e.printStackTrace(); ! try { ! this.endIt(); ! } catch (IOException e1) { ! // TODO Auto-generated catch block ! e1.printStackTrace(); ! } ! try { ! this.startIt(); ! } catch (IOException e2) { ! // TODO Auto-generated catch block ! e2.printStackTrace(); ! } ! return getInvalidCompletion(); ! } ! } ! ! /** ! * @return ! */ ! private List getInvalidCompletion() { ! List l = new ArrayList(); ! l.add(new String[]{"SERVER_ERROR","please try again."}); ! return l; ! } ! ! ! /** ! * @throws IOException ! */ ! private List getCompletions() throws IOException { ! ArrayList list = new ArrayList(); ! String string = this.read().replaceAll("\\(","").replaceAll("\\)",""); ! StringTokenizer tokenizer = new StringTokenizer(string, ","); ! ! while(tokenizer.hasMoreTokens()){ ! String token = tokenizer.nextToken(); ! String description = tokenizer.nextToken(); ! list.add(new String[]{token, description}); ! } ! return list; } Index: PyCodeCompletion.java =================================================================== RCS file: /cvsroot/pydev/org.python.pydev/src/org/python/pydev/editor/codecompletion/PyCodeCompletion.java,v retrieving revision 1.3 retrieving revision 1.4 diff -C2 -d -r1.3 -r1.4 *** PyCodeCompletion.java 16 Aug 2004 20:55:31 -0000 1.3 --- PyCodeCompletion.java 13 Sep 2004 17:11:50 -0000 1.4 *************** *** 13,16 **** --- 13,17 ---- import java.net.URL; import java.util.ArrayList; + import java.util.Iterator; import java.util.List; *************** *** 20,24 **** --- 21,33 ---- import org.eclipse.core.runtime.Path; import org.eclipse.core.runtime.Platform; + import org.eclipse.jface.text.BadLocationException; + import org.eclipse.jface.text.IDocument; + import org.eclipse.jface.text.IRegion; import org.osgi.framework.Bundle; + import org.python.pydev.editor.PyEdit; + import org.python.pydev.editor.model.AbstractNode; + import org.python.pydev.editor.model.Location; + import org.python.pydev.editor.model.ModelUtils; + import org.python.pydev.editor.model.Scope; import org.python.pydev.plugin.PydevPlugin; *************** *** 28,38 **** */ public class PyCodeCompletion { ! /* ! * (non-Javadoc) ! * ! * @see org.eclipse.jface.text.contentassist.IContentAssistProcessor#computeCompletionProposals(org.eclipse.jface.text.ITextViewer, ! * int) ! */ int docBoundary = -1; // the document prior to the activation token /** --- 37,44 ---- */ public class PyCodeCompletion { ! ! int docBoundary = -1; // the document prior to the activation token + private PythonShell pytonShell; /** *************** *** 54,57 **** --- 60,66 ---- /** * Returns a list with the tokens to use for autocompletion. + * @param edit + * @param doc + * @param documentOffset * * @param theCode *************** *** 59,69 **** * @return */ ! public List autoComplete(java.lang.String theCode, java.lang.String theActivationToken) { List theList = new ArrayList(); ! File tipperFile=null; try { ! tipperFile = getAutoCompleteScript(); } catch (CoreException e) { --- 68,182 ---- * @return */ ! public List autoComplete(PyEdit edit, IDocument doc, int documentOffset, java.lang.String theCode, java.lang.String theActivationToken) { + if(PyCodeCompletionPreferencesPage.useServerTipEnviroment()){ + return serverCompletion(theActivationToken, edit, doc, documentOffset, theCode); + }else{ + return consoleCompletion(theCode); + } + } + + + /** + * @param edit + * @param doc + * @param documentOffset + * @param theCode + * @param theCode + */ + private List serverCompletion(String theActivationToken, PyEdit edit, IDocument doc, int documentOffset, String theCode) { List theList = new ArrayList(); ! PythonShell serverShell = null; ! try { ! serverShell = getServerShell(); ! } catch (Exception e) { ! throw new RuntimeException(e); ! } ! ! ! String docToParse = getDocToParse(doc, documentOffset); ! ! if (theActivationToken.replace('.',' ').trim().equals("self")){ ! // System.out.println("serverCompletion - self."); ! ! Location loc = Location.offsetToLocation(doc, documentOffset); ! AbstractNode closest = ModelUtils.getLessOrEqualNode(edit.getPythonModel(),loc); + Scope scope = closest.getScope().findContainingClass(); + String token = scope.getStartNode().getName(); + + List completions = serverShell.getTokenCompletions(token, docToParse); + theList.addAll(completions); + + } + else{ //go to globals + // System.out.println("simpleCompletion - ''"); + List completions = serverShell.getGlobalCompletions(docToParse); + theList.addAll(completions); + + } + return theList; + + } + + /** + * @param doc + * @param documentOffset + * @return + */ + private String getDocToParse(IDocument doc, int documentOffset) { + String wholeDoc = doc.get(); + String newDoc = ""; try { ! int lineOfOffset = doc.getLineOfOffset(documentOffset); ! IRegion lineInformation = doc.getLineInformation(lineOfOffset); ! ! int docLength = doc.getLength(); ! String src = doc.get(lineInformation.getOffset(), documentOffset-lineInformation.getOffset()); ! ! String spaces=""; ! for (int i = 0; i < src.length(); i++) { ! if(src.charAt(i) != ' '){ ! break; ! } ! spaces += ' '; ! } ! ! newDoc = wholeDoc.substring(0, lineInformation.getOffset()); ! newDoc += spaces+"pass\n"; ! newDoc += wholeDoc.substring(lineInformation.getOffset() + lineInformation.getLength(), docLength); ! ! System.out.println("DOC:"+newDoc); ! ! } catch (BadLocationException e1) { ! e1.printStackTrace(); ! } ! return newDoc; ! } ! ! /** ! * @return ! * @throws CoreException ! * @throws IOException ! * ! */ ! private PythonShell getServerShell() throws IOException, CoreException { ! if(pytonShell == null){ ! pytonShell = new PythonShell(); ! pytonShell.startIt(); ! } ! return pytonShell; ! ! } ! ! /** ! * @param theCode ! */ ! private List consoleCompletion(java.lang.String theCode) { ! List theList = new ArrayList(); ! File tipperFile=null; ! try { ! ! tipperFile = getAutoCompleteScript(false); } catch (CoreException e) { *************** *** 86,90 **** writer.write("sys.path.insert(0,r'"+tipperFile.getParent()+"')\n"); - //now we have it in the modules... import and call tipper with the code... writer.write("from tipper import GenerateTip\n"); --- 199,202 ---- *************** *** 98,102 **** writer.write("GenerateTip(s)\n\n\n"); - writer.flush(); writer.close(); --- 210,213 ---- *************** *** 105,108 **** --- 216,220 ---- while ((str = in.readLine()) != null) { if (!str.startsWith("tip: ")){ + // System.out.println("std output: " + str); continue; } *************** *** 110,114 **** str = str.substring(5); ! theList.add(str); } in.close(); --- 222,226 ---- str = str.substring(5); ! theList.add(new String[]{str,""}); } in.close(); *************** *** 129,132 **** --- 241,258 ---- } + /** + * + * @param useSimpleTipper + * @return the script to get the variables. + * + * @throws CoreException + */ + public static File getAutoCompleteScript(boolean useSimpleTipper) throws CoreException { + if(useSimpleTipper){ + return getScriptWithinPySrc("simpleTipper.py"); + }else{ + return getScriptWithinPySrc("tipper.py"); + } + } /** *************** *** 136,141 **** * @throws CoreException */ ! public static File getAutoCompleteScript() throws CoreException { ! String targetExec = "tipper.py"; IPath relative = new Path("PySrc").addTrailingSeparator().append( --- 262,266 ---- * @throws CoreException */ ! public static File getScriptWithinPySrc(String targetExec) throws CoreException { IPath relative = new Path("PySrc").addTrailingSeparator().append( *************** *** 157,160 **** --- 282,305 ---- } + public static File getImageWithinIcons(String icon) throws CoreException { + + IPath relative = new Path("icons").addTrailingSeparator().append( + icon); + + Bundle bundle = PydevPlugin.getDefault().getBundle(); + + URL bundleURL = Platform.find(bundle, relative); + URL fileURL; + try { + fileURL = Platform.asLocalURL(bundleURL); + File f = new File(fileURL.getPath()); + + return f; + } catch (IOException e) { + throw new CoreException(PydevPlugin.makeStatus(IStatus.ERROR, + "Can't find image", null)); + } + } + /** * The docBoundary should get until the last line before the one *************** *** 181,185 **** calcDocBoundary(theDoc, documentOffset); } ! return theDoc.substring(this.docBoundary + 1, documentOffset); } --- 326,334 ---- calcDocBoundary(theDoc, documentOffset); } ! String str = theDoc.substring(this.docBoundary + 1, documentOffset); ! if (str.endsWith(" ")){ ! str = " "; ! } ! return str; } Index: CompletionCache.java =================================================================== RCS file: /cvsroot/pydev/org.python.pydev/src/org/python/pydev/editor/codecompletion/CompletionCache.java,v retrieving revision 1.1 retrieving revision 1.2 diff -C2 -d -r1.1 -r1.2 *** CompletionCache.java 11 Aug 2004 13:04:13 -0000 1.1 --- CompletionCache.java 13 Sep 2004 17:11:50 -0000 1.2 *************** *** 12,16 **** --- 12,18 ---- import java.util.Map; + import org.eclipse.jface.text.IDocument; import org.eclipse.jface.text.contentassist.CompletionProposal; + import org.python.pydev.editor.PyEdit; /** *************** *** 18,67 **** */ public class CompletionCache { ! private Map cache = new HashMap(); private List cacheEntries = new ArrayList(); - ! public List getAllProposals(String theDoc, String activationToken, ! int documentOffset, int qlen, PyCodeCompletion codeCompletion) { - - List allProposals = null; ! if (cache.containsKey(theDoc)) { ! //if it is in the cache, we can just get the proposals, ! //the only thing here, is that we have to change its size depending ! //on the new qlen. ! ! ! allProposals = new ArrayList(); ! List proposals = (List) cache.get(theDoc); ! ! for (Iterator iter = proposals.iterator(); iter.hasNext();) { ! CompletionProposal prop = (CompletionProposal) iter.next(); ! String displayString = prop.getDisplayString(); ! allProposals.add(new CompletionProposal(displayString, ! documentOffset - qlen, qlen, displayString.length())); ! } ! ! ! ! } else { ! List theList = codeCompletion.autoComplete(theDoc, ! activationToken); allProposals = new ArrayList(); for (Iterator iter = theList.iterator(); iter.hasNext();) { ! String element = (String) iter.next(); ! CompletionProposal proposal = new CompletionProposal(element, ! documentOffset - qlen, qlen, element.length()); allProposals.add(proposal); } ! cacheEntries.add(theDoc); ! cache.put(theDoc, allProposals); ! //we don't want this the get huge... ! if (cacheEntries.size() > 20) { Object entry = cacheEntries.remove(0); cache.remove(entry); --- 20,63 ---- */ public class CompletionCache { ! private Map cache = new HashMap(); private List cacheEntries = new ArrayList(); ! /** ! * Returns all the completions ! * ! * @param edit ! * @param doc ! * @param partialDoc ! * @param activationToken ! * @param documentOffset ! * @param qlen ! * @param codeCompletion ! * @return ! */ ! public List getAllProposals(PyEdit edit, IDocument doc, String partialDoc, ! String activationToken, int documentOffset, int qlen, PyCodeCompletion codeCompletion) { ! List allProposals = getCacheProposals(partialDoc, documentOffset, qlen); ! ! if(allProposals == null){ //no cache proposals ! List theList = codeCompletion.autoComplete(edit, doc, ! documentOffset, partialDoc, activationToken); allProposals = new ArrayList(); for (Iterator iter = theList.iterator(); iter.hasNext();) { ! String element[] = (String[]) iter.next(); ! ! CompletionProposal proposal = new CompletionProposal(element[0], ! documentOffset - qlen, qlen, element[0].length(),null, null, null, element[1]); allProposals.add(proposal); } ! cacheEntries.add(partialDoc); ! cache.put(partialDoc, allProposals); ! //we don't want this to get huge... ! if (cacheEntries.size() > 2) { Object entry = cacheEntries.remove(0); cache.remove(entry); *************** *** 71,73 **** --- 67,102 ---- } + + /** + * @param partialDoc + * @param documentOffset + * @param qlen + * @return + */ + private List getCacheProposals(String partialDoc, int documentOffset, int qlen) { + List allProposals = null; + if (cache.containsKey(partialDoc)) { + //if it is in the cache, we can just get the proposals, + //the only thing here, is that we have to change its size depending + //on the new qlen. + + allProposals = new ArrayList(); + List proposals = (List) cache.get(partialDoc); + + for (Iterator iter = proposals.iterator(); iter.hasNext();) { + CompletionProposal prop = (CompletionProposal) iter.next(); + String displayString = prop.getDisplayString(); + allProposals.add(new CompletionProposal( + displayString, // + documentOffset - qlen, // + qlen, // + displayString.length(), // + prop.getImage(), // + prop.getDisplayString(), // + prop.getContextInformation(), // + prop.getAdditionalProposalInfo()));// + } + } + return allProposals; + } } \ No newline at end of file |
From: Fabio Z. <fa...@us...> - 2004-09-13 17:12:33
|
Update of /cvsroot/pydev/org.python.pydev/PySrc In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv1125/PySrc Modified Files: test_pyserver.py pycompletionserver.py simpleTipper.py Log Message: New code completion. Index: pycompletionserver.py =================================================================== RCS file: /cvsroot/pydev/org.python.pydev/PySrc/pycompletionserver.py,v retrieving revision 1.3 retrieving revision 1.4 diff -C2 -d -r1.3 -r1.4 *** pycompletionserver.py 10 Sep 2004 19:42:23 -0000 1.3 --- pycompletionserver.py 13 Sep 2004 17:11:50 -0000 1.4 *************** *** 34,38 **** def removeInvalidChars(self, msg): ! return msg.replace(',','').replace('(','').replace(')','') def formatCompletionMessage(self, completionsList): --- 34,40 ---- def removeInvalidChars(self, msg): ! if msg: ! return msg.replace(',','').replace('(','').replace(')','') ! return ' ' def formatCompletionMessage(self, completionsList): Index: simpleTipper.py =================================================================== RCS file: /cvsroot/pydev/org.python.pydev/PySrc/simpleTipper.py,v retrieving revision 1.1 retrieving revision 1.2 diff -C2 -d -r1.1 -r1.2 *** simpleTipper.py 10 Sep 2004 19:42:23 -0000 1.1 --- simpleTipper.py 13 Sep 2004 17:11:51 -0000 1.2 *************** *** 37,51 **** ''' % (token,token) ! import simpleinspect import compiler ! __eraseThis = compiler.compile(theDoc, 'temporary', 'exec') ! ! simpleinspect.__eraseThisTips = [] ! simpleinspect.GenerateTip (__eraseThis) ! toReturn = simpleinspect.__eraseThisTips ! simpleinspect.__eraseThisTips = [] ! return toReturn --- 37,56 ---- ''' % (token,token) ! import simpleinspect import compiler ! try: ! __eraseThis = compiler.compile(theDoc, 'temporary', 'exec') ! ! simpleinspect.__eraseThisTips = [] ! simpleinspect.GenerateTip (__eraseThis) ! toReturn = simpleinspect.__eraseThisTips ! simpleinspect.__eraseThisTips = [] ! return toReturn ! except : ! import sys ! s = str(sys.exc_info()[1]) ! return [('ERROR_COMPLETING',s)] Index: test_pyserver.py =================================================================== RCS file: /cvsroot/pydev/org.python.pydev/PySrc/test_pyserver.py,v retrieving revision 1.1 retrieving revision 1.2 diff -C2 -d -r1.1 -r1.2 *** test_pyserver.py 10 Sep 2004 19:42:23 -0000 1.1 --- test_pyserver.py 13 Sep 2004 17:11:50 -0000 1.2 *************** *** 46,50 **** connToRead, addr = sToRead.accept() - # print 'test connected addr', addr --- 46,49 ---- *************** *** 64,68 **** self.assert_('END@@' in completions) ! self.sendKillMsg(sToWrite) --- 63,87 ---- self.assert_('END@@' in completions) ! s = \ ! ''' ! class C(object): ! ! def __init__(self): ! ! print dir(self) ! ! def a(self): ! pass ! ! ! def b(self): ! self.a ! ! pass ! ''' ! ! sToWrite.send('@@TOKEN_GLOBALS(C):%s\nEND@@'%s) ! completions = connToRead.recv(4086) ! self.sendKillMsg(sToWrite) |
From: Fabio Z. <fa...@us...> - 2004-09-13 17:12:04
|
Update of /cvsroot/pydev/org.python.pydev/src/org/python/pydev/editor In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv1125/src/org/python/pydev/editor Modified Files: PyEditConfiguration.java PyEdit.java Log Message: New code completion. Index: PyEditConfiguration.java =================================================================== RCS file: /cvsroot/pydev/org.python.pydev/src/org/python/pydev/editor/PyEditConfiguration.java,v retrieving revision 1.18 retrieving revision 1.19 diff -C2 -d -r1.18 -r1.19 *** PyEditConfiguration.java 25 Aug 2004 17:26:08 -0000 1.18 --- PyEditConfiguration.java 13 Sep 2004 17:11:53 -0000 1.19 *************** *** 63,69 **** // private ColorManager colorManager; private String[] indentPrefixes = { " ", "\t", ""}; ! public PyEditConfiguration(ColorCache colorManager) { colorCache = colorManager; } --- 63,71 ---- // private ColorManager colorManager; private String[] indentPrefixes = { " ", "\t", ""}; + private PyEdit edit; ! public PyEditConfiguration(ColorCache colorManager, PyEdit edit) { colorCache = colorManager; + this.setEdit(edit); } *************** *** 291,295 **** // next create a content assistant processor to populate the completions window ! IContentAssistProcessor processor = new PythonCompletionProcessor(); // No code completion in strings --- 293,297 ---- // next create a content assistant processor to populate the completions window ! IContentAssistProcessor processor = new PythonCompletionProcessor(this.getEdit()); // No code completion in strings *************** *** 359,362 **** --- 361,378 ---- } + /** + * @param edit The edit to set. + */ + private void setEdit(PyEdit edit) { + this.edit = edit; + } + + /** + * @return Returns the edit. + */ + private PyEdit getEdit() { + return edit; + } + Index: PyEdit.java =================================================================== RCS file: /cvsroot/pydev/org.python.pydev/src/org/python/pydev/editor/PyEdit.java,v retrieving revision 1.20 retrieving revision 1.21 diff -C2 -d -r1.20 -r1.21 *** PyEdit.java 6 Aug 2004 17:20:02 -0000 1.20 --- PyEdit.java 13 Sep 2004 17:11:53 -0000 1.21 *************** *** 99,103 **** setDocumentProvider(new PyDocumentProvider()); } ! editConfiguration = new PyEditConfiguration(colorCache); setSourceViewerConfiguration(editConfiguration); indentStrategy = (PyAutoIndentStrategy)editConfiguration.getAutoIndentStrategy(null, IDocument.DEFAULT_CONTENT_TYPE); --- 99,103 ---- setDocumentProvider(new PyDocumentProvider()); } ! editConfiguration = new PyEditConfiguration(colorCache,this); setSourceViewerConfiguration(editConfiguration); indentStrategy = (PyAutoIndentStrategy)editConfiguration.getAutoIndentStrategy(null, IDocument.DEFAULT_CONTENT_TYPE); |
From: Fabio Z. <fa...@us...> - 2004-09-13 17:12:02
|
Update of /cvsroot/pydev/org.python.pydev In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv1125 Modified Files: .classpath Log Message: New code completion. Index: .classpath =================================================================== RCS file: /cvsroot/pydev/org.python.pydev/.classpath,v retrieving revision 1.9 retrieving revision 1.10 diff -C2 -d -r1.9 -r1.10 *** .classpath 6 Aug 2004 17:20:03 -0000 1.9 --- .classpath 13 Sep 2004 17:11:53 -0000 1.10 *************** *** 4,7 **** --- 4,9 ---- <classpathentry kind="con" path="org.eclipse.pde.core.requiredPlugins"/> <classpathentry kind="con" path="org.eclipse.jdt.launching.JRE_CONTAINER"/> + <classpathentry kind="src" path="tests"/> + <classpathentry sourcepath="ECLIPSE_HOME/plugins/org.eclipse.jdt.source_3.0.0/src/org.junit_3.8.1/junitsrc.zip" kind="var" path="JUNIT_HOME/junit.jar"/> <classpathentry kind="output" path="bin"/> </classpath> |