Thread: [cedet-semantic] Constructing the semantic parse tree
Brought to you by:
zappo
From: Thomas J. <tja...@gm...> - 2013-06-21 08:50:36
|
Hi all, I'm quite new at this and I'm trying to wrap my head around how to construct a semantic parse tree. Please bare with me since I have never taken a compiler class or similar. According to the semantic manual the parse tree is a tree of semantic tags. What I don't understand is how I would go about constructing tags that have children. For example, I'm working with the Erlang language right now. I have bovine grammar with the rule (I'm aware the rule is in no way complete or probably even correct, I'm still just trying to understand how things work): function-clause : function-name LPAREN arguments RPAREN ARROW body PERIOD (FUNCTION-TAG $1 'function $3) What do I do with the body ($6)? How do I get it parsed and included in the parse tree? Also, I would like to ask if there's any way of ensuring that buffer lexing is done at the correct depth so that the token stream is complete and contains no semantic-list's. Thank you! |
From: Eric M. L. <er...@si...> - 2013-06-21 11:45:15
|
On 06/21/2013 04:50 AM, Thomas Järvstrand wrote: > Hi all, > > I'm quite new at this and I'm trying to wrap my head around how to > construct a semantic parse tree. Please bare with me since I have never > taken a compiler class or similar. > > According to the semantic manual the parse tree is a tree of semantic > tags. What I don't understand is how I would go about constructing tags > that have children. For example, I'm working with the Erlang language > right now. I have bovine grammar with the rule (I'm aware the rule is in > no way complete or probably even correct, I'm still just trying to > understand how things work): > > function-clause > : function-name LPAREN arguments RPAREN ARROW body PERIOD > (FUNCTION-TAG $1 'function $3) > > What do I do with the body ($6)? How do I get it parsed and included in > the parse tree? > > Also, I would like to ask if there's any way of ensuring that buffer > lexing is done at the correct depth so that the token stream is complete > and contains no semantic-list's. Hi Thomas, Unlike other parsers, like bison, yacc, etc, the parsers in Semantic purposely doesn't parse out all the lists in your buffer, instead allowing the parser to choose when to descend into a list. Thus, if there is something useful in 'body' above, you would use EXPAND or EXPANDALL feature in your action to dive into that list. Semantic parsers are also about TAGS such as you might get with ETAGS, not about a full parse tree that you might be thinking of. You could generate one, of course, but the infrastructure is really setup for the high level tags like functions. As such, the 'body' of a function is usually completely ignored. If you choose to save it, you just need to add some attribute to your FUNCTION-TAG call, perhaps like this: (FUNCTION-TAG $1 nil $3 :code $6) and in this case, it will just save a pair of numbers. I suppose you could also use: (FUNCTION-TAG $1 nil $3 :code (CODE-TAG "" $6)) but then you need to write a support function to identify :code as a list of child tags so it gets expanded back into the buffer correctly. Good Luck Eric |
From: Stephen L. <ste...@st...> - 2013-06-21 20:46:47
|
Thomas Järvstrand <tja...@gm...> writes: > What I don't understand is how I would go about constructing tags that have > children. For example, I'm working with the Erlang language right now. I > have bovine grammar with the rule (I'm aware the rule is in no way complete > or probably even correct, I'm still just trying to understand how things > work): > > function-clause > : function-name LPAREN arguments RPAREN ARROW body PERIOD > (FUNCTION-TAG $1 'function $3) > > What do I do with the body ($6)? How do I get it parsed and included in the > parse tree? What do you plan to do with the parse tree? That strongly influences how you build it. -- -- Stephe |
From: Thomas J. <tja...@gm...> - 2013-06-22 11:23:47
|
Thank you, that explains a lot, I will have to think on this for a bit then. The idea was to use semantic to provide refactoring features and context-sensitive auto-completion for an Erlang dev environment package that I'm working on. T On 21 Jun 2013 21:46, "Stephen Leake" <ste...@st...> wrote: > Thomas Järvstrand <tja...@gm...> writes: > > > What I don't understand is how I would go about constructing tags that > have > > children. For example, I'm working with the Erlang language right now. I > > have bovine grammar with the rule (I'm aware the rule is in no way > complete > > or probably even correct, I'm still just trying to understand how things > > work): > > > > function-clause > > : function-name LPAREN arguments RPAREN ARROW body PERIOD > > (FUNCTION-TAG $1 'function $3) > > > > What do I do with the body ($6)? How do I get it parsed and included in > the > > parse tree? > > What do you plan to do with the parse tree? > > That strongly influences how you build it. > -- > -- Stephe > |
From: Eric M. L. <er...@si...> - 2013-06-22 12:43:43
|
Hi Thomas, So you know, there is an erlang parser already in the semantic/bovine/erlang.by parser. As far as I know, it parses, but we don't have much in the way of tests, etc. To do what you are looking to work on, check: http://cedet.sourceforge.net/addlang.shtml and I think you would need Step 2, 4, and perhaps 5. Step 5 helps get the context analyzer up which is useful for refactoring. Since there are pretty good defaults already for 2,4, and 5, the work there might be easy. (ie - a small # of things to overload.) After then, look at semantic/symref.el and semantic/symref/* which is a simple framework from which some kinds of refactoring can take place. At the moment, just renames is done. If you make some big changes and would like to contribute to CEDET, you will need to sign papers with the FSF for contributions to Emacs and CEDET. If you are interested, I can send you the form. Good Luck Eric On 06/22/2013 07:23 AM, Thomas Järvstrand wrote: > Thank you, that explains a lot, I will have to think on this for a bit > then. The idea was to use semantic to provide refactoring features and > context-sensitive auto-completion for an Erlang dev environment package > that I'm working on. > > T > > On 21 Jun 2013 21:46, "Stephen Leake" <ste...@st... > <mailto:ste...@st...>> wrote: > > Thomas Järvstrand <tja...@gm... > <mailto:tja...@gm...>> writes: > > > What I don't understand is how I would go about constructing tags > that have > > children. For example, I'm working with the Erlang language right > now. I > > have bovine grammar with the rule (I'm aware the rule is in no > way complete > > or probably even correct, I'm still just trying to understand how > things > > work): > > > > function-clause > > : function-name LPAREN arguments RPAREN ARROW body PERIOD > > (FUNCTION-TAG $1 'function $3) > > > > What do I do with the body ($6)? How do I get it parsed and > included in the > > parse tree? > > What do you plan to do with the parse tree? > > That strongly influences how you build it. > -- > -- Stephe > > > > ------------------------------------------------------------------------------ > This SF.net email is sponsored by Windows: > > Build for Windows Store. > > http://p.sf.net/sfu/windows-dev2dev > > > > _______________________________________________ > cedet-semantic mailing list > ced...@li... > https://lists.sourceforge.net/lists/listinfo/cedet-semantic |
From: Thomas J. <tja...@gm...> - 2013-06-22 22:56:25
|
Thanks, I will look into it. I'm aware of the Erlang parser, I just there where some oddities in it and would like to understand how it all works together. Thomas 2013/6/22 Eric M. Ludlam <er...@si...> > Hi Thomas, > > So you know, there is an erlang parser already in the semantic/bovine/ > erlang.by parser. As far as I know, it parses, but we don't have much in > the way of tests, etc. > > To do what you are looking to work on, check: > http://cedet.sourceforge.net/**addlang.shtml<http://cedet.sourceforge.net/addlang.shtml> > > and I think you would need Step 2, 4, and perhaps 5. Step 5 helps get the > context analyzer up which is useful for refactoring. > > Since there are pretty good defaults already for 2,4, and 5, the work > there might be easy. (ie - a small # of things to overload.) > > After then, look at semantic/symref.el and semantic/symref/* which is a > simple framework from which some kinds of refactoring can take place. At > the moment, just renames is done. > > If you make some big changes and would like to contribute to CEDET, you > will need to sign papers with the FSF for contributions to Emacs and CEDET. > If you are interested, I can send you the form. > > Good Luck > Eric > > > > > On 06/22/2013 07:23 AM, Thomas Järvstrand wrote: > >> Thank you, that explains a lot, I will have to think on this for a bit >> then. The idea was to use semantic to provide refactoring features and >> context-sensitive auto-completion for an Erlang dev environment package >> that I'm working on. >> >> T >> >> On 21 Jun 2013 21:46, "Stephen Leake" <stephen_leake@stephe-leake.**org<ste...@st...> >> <mailto:stephen_leake@stephe-**leake.org <ste...@st...>>> >> wrote: >> >> Thomas Järvstrand <tja...@gm... >> <mailto:tja...@gm...>**> writes: >> >> > What I don't understand is how I would go about constructing tags >> that have >> > children. For example, I'm working with the Erlang language right >> now. I >> > have bovine grammar with the rule (I'm aware the rule is in no >> way complete >> > or probably even correct, I'm still just trying to understand how >> things >> > work): >> > >> > function-clause >> > : function-name LPAREN arguments RPAREN ARROW body PERIOD >> > (FUNCTION-TAG $1 'function $3) >> > >> > What do I do with the body ($6)? How do I get it parsed and >> included in the >> > parse tree? >> >> What do you plan to do with the parse tree? >> >> That strongly influences how you build it. >> -- >> -- Stephe >> >> >> >> ------------------------------**------------------------------** >> ------------------ >> This SF.net email is sponsored by Windows: >> >> Build for Windows Store. >> >> http://p.sf.net/sfu/windows-**dev2dev<http://p.sf.net/sfu/windows-dev2dev> >> >> >> >> ______________________________**_________________ >> cedet-semantic mailing list >> cedet-semantic@lists.**sourceforge.net<ced...@li...> >> https://lists.sourceforge.net/**lists/listinfo/cedet-semantic<https://lists.sourceforge.net/lists/listinfo/cedet-semantic> >> > |
From: Thomas J. <tja...@gm...> - 2013-10-12 08:14:18
|
Hi again, I got sidetracked for a while, but I would still be very interested in getting this to work. H ave I understood it correctly that semantic/bovine is not really built to be able to extract arbitrary data about a source file? Eg. for navigation and refactoring purposes I would like to store information about include files, external function calls, usage of record types and probably other things in the future. I would have to build most of that extraction myself even if I do it using semantic right? Would I use compound tags<http://www.randomsample.de/cedetdocs/semantic-langdev/Tag-Expansion.html#Tag-Expansion>for that? Once I have the information, I'm guessing I can store the resulting tags in semanticdb? Thanks Thomas 2013/6/23 Thomas Järvstrand <tja...@gm...> > Thanks, I will look into it. I'm aware of the Erlang parser, I just there > where some oddities in it and would like to understand how it all works > together. > > Thomas > > > 2013/6/22 Eric M. Ludlam <er...@si...> > > Hi Thomas, >> >> So you know, there is an erlang parser already in the semantic/bovine/ >> erlang.by parser. As far as I know, it parses, but we don't have much >> in the way of tests, etc. >> >> To do what you are looking to work on, check: >> http://cedet.sourceforge.net/**addlang.shtml<http://cedet.sourceforge.net/addlang.shtml> >> >> and I think you would need Step 2, 4, and perhaps 5. Step 5 helps get >> the context analyzer up which is useful for refactoring. >> >> Since there are pretty good defaults already for 2,4, and 5, the work >> there might be easy. (ie - a small # of things to overload.) >> >> After then, look at semantic/symref.el and semantic/symref/* which is a >> simple framework from which some kinds of refactoring can take place. At >> the moment, just renames is done. >> >> If you make some big changes and would like to contribute to CEDET, you >> will need to sign papers with the FSF for contributions to Emacs and CEDET. >> If you are interested, I can send you the form. >> >> Good Luck >> Eric >> >> >> >> >> On 06/22/2013 07:23 AM, Thomas Järvstrand wrote: >> >>> Thank you, that explains a lot, I will have to think on this for a bit >>> then. The idea was to use semantic to provide refactoring features and >>> context-sensitive auto-completion for an Erlang dev environment package >>> that I'm working on. >>> >>> T >>> >>> On 21 Jun 2013 21:46, "Stephen Leake" <stephen_leake@stephe-leake.**org<ste...@st...> >>> <mailto:stephen_leake@stephe-**leake.org<ste...@st...>>> >>> wrote: >>> >>> Thomas Järvstrand <tja...@gm... >>> <mailto:tja...@gm...>**> writes: >>> >>> > What I don't understand is how I would go about constructing tags >>> that have >>> > children. For example, I'm working with the Erlang language right >>> now. I >>> > have bovine grammar with the rule (I'm aware the rule is in no >>> way complete >>> > or probably even correct, I'm still just trying to understand how >>> things >>> > work): >>> > >>> > function-clause >>> > : function-name LPAREN arguments RPAREN ARROW body PERIOD >>> > (FUNCTION-TAG $1 'function $3) >>> > >>> > What do I do with the body ($6)? How do I get it parsed and >>> included in the >>> > parse tree? >>> >>> What do you plan to do with the parse tree? >>> >>> That strongly influences how you build it. >>> -- >>> -- Stephe >>> >>> >>> >>> ------------------------------**------------------------------** >>> ------------------ >>> This SF.net email is sponsored by Windows: >>> >>> Build for Windows Store. >>> >>> http://p.sf.net/sfu/windows-**dev2dev<http://p.sf.net/sfu/windows-dev2dev> >>> >>> >>> >>> ______________________________**_________________ >>> cedet-semantic mailing list >>> cedet-semantic@lists.**sourceforge.net<ced...@li...> >>> https://lists.sourceforge.net/**lists/listinfo/cedet-semantic<https://lists.sourceforge.net/lists/listinfo/cedet-semantic> >>> >> > |