Thread: [Pydev-code] Fwd: Viewing ASTs in PyDev
Brought to you by:
fabioz
From: Neil <nei...@ut...> - 2007-05-03 23:24:23
|
Hi Fabio, I'm following along with the mailing list advice stated here: http://www.nabble.com/PyDev-Extension-Points-Quick-Fix-%28copied-from-pydev-forum%29-tf503666.html#a1365501 where you advise extending from pydev_builder to walk the AST. However, I'm new to Eclipse plugin programming so I wonder if you could be a bit more specific. Currently my plugin does something like: public class PyGoal extends PyDevBuilderVisitor { SourceModule sm = (SourceModule) getSourceModule(resource, document, nature); SimpleNode sn = sm.getAst(); EasyASTIteratorVisitor vis = new EasyASTIteratorVisitor(); try{ sn.accept(vis); } catch(Exception e) {} } What I'd like to do is create a simplified syntax tree structure that just stores functions, variables, control structures, and calls. I'd then do various specific things to that tree for my own plugin. Can you illustrate how I would visit a particular document's AST, putting the elements of interest in my own data structure? How do I tell the PyDevBuilderVisitor what document/resource I'm working with? thanks neil ernst On 4/25/07, Neil Ernst <ne...@cs...> wrote: > I think it would look something like the JDT AST viewer plugin: > http://www.eclipse.org/articles/Article-JavaCodeManipulation_AST/index.html > > I think I should probably create a separate plugin project that calls > on PyDev to do the actual AST generation, then load that into my > plugin. > > thanks for the advice > neil > > On 4/25/07, Ueli Kistler <uki...@hs...> wrote: > > Hi, > > > > i would not recommend using JGraph as I did... it would make more sense > > to extend the current outline view or provide a new "outline"-like view > > (i think there can be only one "true" outline view actually because its > > bound to an editor implementation? Not sure about this..) > > > > This could of course be done in a separate project... it would require > > similiar dependencies from PyDev as PEPTIC. > > > > Cheers, > > Ueli > > > > -- > > > > > > > > > > > > Fabio Zadrozny wrote: > > > > My first challenge is to view the ASTs for the entire project (I'm > > > > looking at the Trac source code). I notice in your code you have a > > > > class for viewing the AST using JGraph. Would you be able to point me > > > > to a way to have this done in PyDev? I've been exploring the Jython > > > > scripting elements of PyDev but I can't get your viewer to start that > > > > way. > > > > > > The viewer that uses JGraph is not bundled in the pydev distribution > > > and thus cannot be accessed with the scripting... If you have the > > > source code, you can change those files from the 'contrib' source > > > folder to the actual 'src' source folder so that they can be accessed > > > (and add the related jars as dependencies for the plugin... or maybe a > > > better idea would be creating another plugin to hold those things, > > > which you can use and extend with your own code). > > > > > > Also, I think I would need more info on what you actually want to > > > access to help you better... > > > > > > Cheers, > > > > > > Fabio > > > > > -- Neil A. Ernst Ph.D. Candidate University of Toronto http://neilernst.net |
From: Neil <nei...@ut...> - 2007-05-04 22:06:38
|
That makes sense... what I'm stuck on is trying to get my own plugin activated. I'm currently extending org.python.pydev.pydev_builder, but I'm not sure what this provides me... do I need to implement a particular interface, which is then added to the listener pool by Eclipse by default? It would be great if someone had some sample code on how to use these extension points... I'd rather not hack into the PyDev source directly. thanks neil On 5/4/07, Ueli Kistler <uki...@hs...> wrote: > Hi, > > basically you need to write your own visitor class. > This is easily done extending VisitorBase. VisitorBase will traverse > "automatically" all nodes you don't want to handle yourself. > Override the traverse-method and call "node.traverse(this)". > > For the nodes you are interested in: > This class will traverse all nodes of the AST and use a "callback" > method when traversing a node. > E.g. if it traverses a ClassDef Node it will call visitClassDef(ClassDef > node). > So actually the visitor can construct your treeview-model, if you just > override all the visit-Methods for AST nodes you require. > E.g. visitClassDef, visitFunctionDef, visitCall, etc. etc. .. > > Note also that there are different AST nodes for "variables" and also > for "control structures"... > you should be able to figure out what is what for by having a look at > the AST Rewriter i wrote for PEPTIC (especiall visit-methods). > Have a look at > http://pydev.cvs.sourceforge.net/pydev/org.python.pydev.refactoring/src/org/python/pydev/refactoring/ast/rewriter/RewriterVisitor.java?revision=1.2&view=markup > > Cheers, > Ueli > > -- > > Neil wrote: > > Hi Fabio, I'm following along with the mailing list advice stated here: > > http://www.nabble.com/PyDev-Extension-Points-Quick-Fix-%28copied-from-pydev-forum%29-tf503666.html#a1365501 > > > > where you advise extending from pydev_builder to walk the AST. > > > > However, I'm new to Eclipse plugin programming so I wonder if you > > could be a bit more specific. > > > > Currently my plugin does something like: > > > > public class PyGoal extends PyDevBuilderVisitor { > > SourceModule sm = (SourceModule) getSourceModule(resource, document, nature); > > SimpleNode sn = sm.getAst(); > > EasyASTIteratorVisitor vis = new EasyASTIteratorVisitor(); > > try{ > > sn.accept(vis); > > } catch(Exception e) {} > > } > > What I'd like to do is create a simplified syntax tree structure that > > just stores functions, variables, control structures, and calls. I'd > > then do various specific things to that tree for my own plugin. > > > > Can you illustrate how I would visit a particular document's AST, > > putting the elements of interest in my own data structure? How do I > > tell the PyDevBuilderVisitor what document/resource I'm working with? > > > > thanks > > neil ernst > > > > On 4/25/07, Neil Ernst <ne...@cs...> wrote: > > > >> I think it would look something like the JDT AST viewer plugin: > >> http://www.eclipse.org/articles/Article-JavaCodeManipulation_AST/index.html > >> > >> I think I should probably create a separate plugin project that calls > >> on PyDev to do the actual AST generation, then load that into my > >> plugin. > >> > >> thanks for the advice > >> neil > >> > >> On 4/25/07, Ueli Kistler <uki...@hs...> wrote: > >> > >>> Hi, > >>> > >>> i would not recommend using JGraph as I did... it would make more sense > >>> to extend the current outline view or provide a new "outline"-like view > >>> (i think there can be only one "true" outline view actually because its > >>> bound to an editor implementation? Not sure about this..) > >>> > >>> This could of course be done in a separate project... it would require > >>> similiar dependencies from PyDev as PEPTIC. > >>> > >>> Cheers, > >>> Ueli > >>> > >>> -- > >>> > >>> > >>> > >>> > >>> > >>> Fabio Zadrozny wrote: > >>> > >>>>> My first challenge is to view the ASTs for the entire project (I'm > >>>>> looking at the Trac source code). I notice in your code you have a > >>>>> class for viewing the AST using JGraph. Would you be able to point me > >>>>> to a way to have this done in PyDev? I've been exploring the Jython > >>>>> scripting elements of PyDev but I can't get your viewer to start that > >>>>> way. > >>>>> > >>>> The viewer that uses JGraph is not bundled in the pydev distribution > >>>> and thus cannot be accessed with the scripting... If you have the > >>>> source code, you can change those files from the 'contrib' source > >>>> folder to the actual 'src' source folder so that they can be accessed > >>>> (and add the related jars as dependencies for the plugin... or maybe a > >>>> better idea would be creating another plugin to hold those things, > >>>> which you can use and extend with your own code). > >>>> > >>>> Also, I think I would need more info on what you actually want to > >>>> access to help you better... > >>>> > >>>> Cheers, > >>>> > >>>> Fabio > >>>> > >>> > > > > > > > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by DB2 Express > Download DB2 Express C - the FREE version of DB2 express and take > control of your XML. No limits. Just data. Click to get it now. > http://sourceforge.net/powerbar/db2/ > _______________________________________________ > pydev-code mailing list > pyd...@li... > https://lists.sourceforge.net/lists/listinfo/pydev-code > -- Neil A. Ernst Ph.D. Candidate University of Toronto http://neilernst.net -- Neil A. Ernst Ph.D. Candidate University of Toronto http://neilernst.net |
From: Fabio Z. <fa...@gm...> - 2007-05-05 23:01:39
|
On 5/4/07, Neil <nei...@ut...> wrote: > > That makes sense... > > what I'm stuck on is trying to get my own plugin activated. I'm > currently extending org.python.pydev.pydev_builder, but I'm not sure > what this provides me... do I need to implement a particular > interface, which is then added to the listener pool by Eclipse by > default? > > It would be great if someone had some sample code on how to use these > extension points... I'd rather not hack into the PyDev source > directly. You'd need to add to your plugin: <extension point="org.python.pydev.pydev_builder"> <builder_participant class="my.pydev.Builder"/> </extension> and it needs to extend org.python.pydev.builder.PyDevBuilderVisitor I'll take a look at the current schemas and give a better documentation for the next release. Cheers, Fabio thanks > neil > > On 5/4/07, Ueli Kistler <uki...@hs...> wrote: > > Hi, > > > > basically you need to write your own visitor class. > > This is easily done extending VisitorBase. VisitorBase will traverse > > "automatically" all nodes you don't want to handle yourself. > > Override the traverse-method and call "node.traverse(this)". > > > > For the nodes you are interested in: > > This class will traverse all nodes of the AST and use a "callback" > > method when traversing a node. > > E.g. if it traverses a ClassDef Node it will call visitClassDef(ClassDef > > node). > > So actually the visitor can construct your treeview-model, if you just > > override all the visit-Methods for AST nodes you require. > > E.g. visitClassDef, visitFunctionDef, visitCall, etc. etc. .. > > > > Note also that there are different AST nodes for "variables" and also > > for "control structures"... > > you should be able to figure out what is what for by having a look at > > the AST Rewriter i wrote for PEPTIC (especiall visit-methods). > > Have a look at > > > http://pydev.cvs.sourceforge.net/pydev/org.python.pydev.refactoring/src/org/python/pydev/refactoring/ast/rewriter/RewriterVisitor.java?revision=1.2&view=markup > > > > Cheers, > > Ueli > > > > -- > > > > Neil wrote: > > > Hi Fabio, I'm following along with the mailing list advice stated > here: > > > > http://www.nabble.com/PyDev-Extension-Points-Quick-Fix-%28copied-from-pydev-forum%29-tf503666.html#a1365501 > > > > > > where you advise extending from pydev_builder to walk the AST. > > > > > > However, I'm new to Eclipse plugin programming so I wonder if you > > > could be a bit more specific. > > > > > > Currently my plugin does something like: > > > > > > public class PyGoal extends PyDevBuilderVisitor { > > > SourceModule sm = (SourceModule) getSourceModule(resource, document, > nature); > > > SimpleNode sn = sm.getAst(); > > > EasyASTIteratorVisitor vis = new EasyASTIteratorVisitor(); > > > try{ > > > sn.accept(vis); > > > } catch(Exception e) {} > > > } > > > What I'd like to do is create a simplified syntax tree structure that > > > just stores functions, variables, control structures, and calls. I'd > > > then do various specific things to that tree for my own plugin. > > > > > > Can you illustrate how I would visit a particular document's AST, > > > putting the elements of interest in my own data structure? How do I > > > tell the PyDevBuilderVisitor what document/resource I'm working with? > > > > > > thanks > > > neil ernst > > > > > > On 4/25/07, Neil Ernst <ne...@cs...> wrote: > > > > > >> I think it would look something like the JDT AST viewer plugin: > > >> > http://www.eclipse.org/articles/Article-JavaCodeManipulation_AST/index.html > > >> > > >> I think I should probably create a separate plugin project that calls > > >> on PyDev to do the actual AST generation, then load that into my > > >> plugin. > > >> > > >> thanks for the advice > > >> neil > > >> > > >> On 4/25/07, Ueli Kistler <uki...@hs...> wrote: > > >> > > >>> Hi, > > >>> > > >>> i would not recommend using JGraph as I did... it would make more > sense > > >>> to extend the current outline view or provide a new "outline"-like > view > > >>> (i think there can be only one "true" outline view actually because > its > > >>> bound to an editor implementation? Not sure about this..) > > >>> > > >>> This could of course be done in a separate project... it would > require > > >>> similiar dependencies from PyDev as PEPTIC. > > >>> > > >>> Cheers, > > >>> Ueli > > >>> > > >>> -- > > >>> > > >>> > > >>> > > >>> > > >>> > > >>> Fabio Zadrozny wrote: > > >>> > > >>>>> My first challenge is to view the ASTs for the entire project (I'm > > >>>>> looking at the Trac source code). I notice in your code you have a > > >>>>> class for viewing the AST using JGraph. Would you be able to > point me > > >>>>> to a way to have this done in PyDev? I've been exploring the > Jython > > >>>>> scripting elements of PyDev but I can't get your viewer to start > that > > >>>>> way. > > >>>>> > > >>>> The viewer that uses JGraph is not bundled in the pydev > distribution > > >>>> and thus cannot be accessed with the scripting... If you have the > > >>>> source code, you can change those files from the 'contrib' source > > >>>> folder to the actual 'src' source folder so that they can be > accessed > > >>>> (and add the related jars as dependencies for the plugin... or > maybe a > > >>>> better idea would be creating another plugin to hold those things, > > >>>> which you can use and extend with your own code). > > >>>> > > >>>> Also, I think I would need more info on what you actually want to > > >>>> access to help you better... > > >>>> > > >>>> Cheers, > > >>>> > > >>>> Fabio > > >>>> > > >>> > > > > > > > > > > > > > > > > ------------------------------------------------------------------------- > > This SF.net email is sponsored by DB2 Express > > Download DB2 Express C - the FREE version of DB2 express and take > > control of your XML. No limits. Just data. Click to get it now. > > http://sourceforge.net/powerbar/db2/ > > _______________________________________________ > > pydev-code mailing list > > pyd...@li... > > https://lists.sourceforge.net/lists/listinfo/pydev-code > > > > > -- > Neil A. Ernst > Ph.D. Candidate > University of Toronto > http://neilernst.net > > > -- > Neil A. Ernst > Ph.D. Candidate > University of Toronto > http://neilernst.net > > ------------------------------------------------------------------------- > This SF.net email is sponsored by DB2 Express > Download DB2 Express C - the FREE version of DB2 express and take > control of your XML. No limits. Just data. Click to get it now. > http://sourceforge.net/powerbar/db2/ > _______________________________________________ > pydev-code mailing list > pyd...@li... > https://lists.sourceforge.net/lists/listinfo/pydev-code > |
From: Fabio Z. <fa...@gm...> - 2007-05-06 00:00:33
|
On 5/4/07, Neil <nei...@ut...> wrote: > > That makes sense... > > what I'm stuck on is trying to get my own plugin activated. I'm > currently extending org.python.pydev.pydev_builder, but I'm not sure > what this provides me... do I need to implement a particular > interface, which is then added to the listener pool by Eclipse by > default? > > It would be great if someone had some sample code on how to use these > extension points... I'd rather not hack into the PyDev source > directly. Actually, I think the extension point org.python.pydev.parser.pydev_parser_observer is more what you want: <extension point="org.python.pydev.parser.pydev_parser_observer"> <parser_observer class="my.parser.Observer"/> </extension> and the Observer class must implement org.python.pydev.core.parser.IParserObserver (it already gives you the generated ast -- the outline is also based on this extension). Cheers, Fabio |
From: Neil <nei...@ut...> - 2007-05-08 03:10:23
|
Thanks, that's exactly the pointer I needed! neil On 5/5/07, Fabio Zadrozny <fa...@gm...> wrote: > > > On 5/4/07, Neil <nei...@ut...> wrote: > > That makes sense... > > > > what I'm stuck on is trying to get my own plugin activated. I'm > > currently extending org.python.pydev.pydev_builder, but I'm not sure > > what this provides me... do I need to implement a particular > > interface, which is then added to the listener pool by Eclipse by > > default? > > > > It would be great if someone had some sample code on how to use these > > extension points... I'd rather not hack into the PyDev source > > directly. > > > Actually, I think the extension point > org.python.pydev.parser.pydev_parser_observer is more what > you want: > > <extension > point="org.python.pydev.parser.pydev_parser_observer "> > <parser_observer class="my.parser.Observer"/> > </extension> > > and the Observer class must implement > org.python.pydev.core.parser.IParserObserver (it already > gives you the generated ast -- the outline is also based on this extension). > > Cheers, > > Fabio > > ------------------------------------------------------------------------- > This SF.net email is sponsored by DB2 Express > Download DB2 Express C - the FREE version of DB2 express and take > control of your XML. No limits. Just data. Click to get it now. > http://sourceforge.net/powerbar/db2/ > _______________________________________________ > pydev-code mailing list > pyd...@li... > https://lists.sourceforge.net/lists/listinfo/pydev-code > > -- Neil A. Ernst Ph.D. Candidate University of Toronto http://neilernst.net |
From: Ueli K. <uki...@hs...> - 2007-05-04 08:32:01
|
Hi, basically you need to write your own visitor class. This is easily done extending VisitorBase. VisitorBase will traverse "automatically" all nodes you don't want to handle yourself. Override the traverse-method and call "node.traverse(this)". For the nodes you are interested in: This class will traverse all nodes of the AST and use a "callback" method when traversing a node. E.g. if it traverses a ClassDef Node it will call visitClassDef(ClassDef node). So actually the visitor can construct your treeview-model, if you just override all the visit-Methods for AST nodes you require. E.g. visitClassDef, visitFunctionDef, visitCall, etc. etc. .. Note also that there are different AST nodes for "variables" and also for "control structures"... you should be able to figure out what is what for by having a look at the AST Rewriter i wrote for PEPTIC (especiall visit-methods). Have a look at http://pydev.cvs.sourceforge.net/pydev/org.python.pydev.refactoring/src/org/python/pydev/refactoring/ast/rewriter/RewriterVisitor.java?revision=1.2&view=markup Cheers, Ueli -- Neil wrote: > Hi Fabio, I'm following along with the mailing list advice stated here: > http://www.nabble.com/PyDev-Extension-Points-Quick-Fix-%28copied-from-pydev-forum%29-tf503666.html#a1365501 > > where you advise extending from pydev_builder to walk the AST. > > However, I'm new to Eclipse plugin programming so I wonder if you > could be a bit more specific. > > Currently my plugin does something like: > > public class PyGoal extends PyDevBuilderVisitor { > SourceModule sm = (SourceModule) getSourceModule(resource, document, nature); > SimpleNode sn = sm.getAst(); > EasyASTIteratorVisitor vis = new EasyASTIteratorVisitor(); > try{ > sn.accept(vis); > } catch(Exception e) {} > } > What I'd like to do is create a simplified syntax tree structure that > just stores functions, variables, control structures, and calls. I'd > then do various specific things to that tree for my own plugin. > > Can you illustrate how I would visit a particular document's AST, > putting the elements of interest in my own data structure? How do I > tell the PyDevBuilderVisitor what document/resource I'm working with? > > thanks > neil ernst > > On 4/25/07, Neil Ernst <ne...@cs...> wrote: > >> I think it would look something like the JDT AST viewer plugin: >> http://www.eclipse.org/articles/Article-JavaCodeManipulation_AST/index.html >> >> I think I should probably create a separate plugin project that calls >> on PyDev to do the actual AST generation, then load that into my >> plugin. >> >> thanks for the advice >> neil >> >> On 4/25/07, Ueli Kistler <uki...@hs...> wrote: >> >>> Hi, >>> >>> i would not recommend using JGraph as I did... it would make more sense >>> to extend the current outline view or provide a new "outline"-like view >>> (i think there can be only one "true" outline view actually because its >>> bound to an editor implementation? Not sure about this..) >>> >>> This could of course be done in a separate project... it would require >>> similiar dependencies from PyDev as PEPTIC. >>> >>> Cheers, >>> Ueli >>> >>> -- >>> >>> >>> >>> >>> >>> Fabio Zadrozny wrote: >>> >>>>> My first challenge is to view the ASTs for the entire project (I'm >>>>> looking at the Trac source code). I notice in your code you have a >>>>> class for viewing the AST using JGraph. Would you be able to point me >>>>> to a way to have this done in PyDev? I've been exploring the Jython >>>>> scripting elements of PyDev but I can't get your viewer to start that >>>>> way. >>>>> >>>> The viewer that uses JGraph is not bundled in the pydev distribution >>>> and thus cannot be accessed with the scripting... If you have the >>>> source code, you can change those files from the 'contrib' source >>>> folder to the actual 'src' source folder so that they can be accessed >>>> (and add the related jars as dependencies for the plugin... or maybe a >>>> better idea would be creating another plugin to hold those things, >>>> which you can use and extend with your own code). >>>> >>>> Also, I think I would need more info on what you actually want to >>>> access to help you better... >>>> >>>> Cheers, >>>> >>>> Fabio >>>> >>> > > > |