Thread: [cedet-semantic] multiple conditions in #ifdef not parsed correctly
Brought to you by:
zappo
From: Brian J. C. <gma...@xo...> - 2009-11-17 04:20:25
|
Thank you for such a wonderful set of tools. I have recently started using CEDET with boost and have narrowed down a parsing "problem." The problem is that #ifdef pre-parsing is not evaluating correctly. An example of this type of statement is as follows: #if defined(BOOST_NO_MEMBER_TEMPLATES) && !defined(BOOST_MSVC6_MEMBER_TEMPLATES) I am using emacs 23: "GNU Emacs 23.1.50.1 (x86_64-pc-linux-gnu, GTK+ Version 2.18.0) of 2009-09-27 on crested, modified by Debian" with CEDET from the cvs repository at cedet.cvs.sourceforge.net) I executed the following emacs-lisp after starting emacs via 'emacs --no-init'): ,---- | (if (not (featurep 'cedet)) | (load-file "~/.elisp/cedet/common/cedet.elc")) | (global-ede-mode t) | (require 'ede-cpp-root) | (require 'semantic-gcc) | (ede-cpp-root-project "cedet-test" | :file "~/src/cedet-test/test.cpp") | (semantic-load-enable-excessive-code-helpers) | (require 'semantic-ia) `---- Here is the contents of test.cpp: ,---- | #undef BOOST_NO_MEMBER_TEMPLATES | #undef BOOST_MSVC6_MEMBER_TEMPLATES | | | #if defined(BOOST_NO_MEMBER_TEMPLATES) | int mymethod(int myint) { | return myint++; | } | #else | // This is what should be recognized and it is | int mymethod(int myint) { | return myint + 2; | } | #endif | | | #if !defined(BOOST_MSVC6_MEMBER_TEMPLATES) | // This is what should be recognized an it is | void mysecondmethod(int) { | return; | } | #else | void mysecondmethod(unsigned int) { | return; | } | #endif | | // So BOOST_NO_MEMBER_TEMPLATES is NOT defined | // and BOOST_MSVC6_MEMBER_TEMPLATES is NOT defined | | // so this is #if false && true --> | // should yield false, but it yields true | #if defined(BOOST_NO_MEMBER_TEMPLATES) && !defined(BOOST_MSVC6_MEMBER_TEMPLATES) | // This should not be the value that is parsed but it is... | void definedBOOST_NO_MEMBER_TEMPLATES_AND_bang_BOOST_MSVC6() | { | return; | } | #else | // This is what should be parsed but it is not | void not_definedBOOST_NO_MEMBER_TEMPLATES_AND_bang_BOOST_MSVC6() | { | return; | } | #endif | | // (senator-force-refresh) `---- At first glance (and I mean glance), it looks like the parser is evaluating if "BOOST_MSVC6_MEMBER_TEMPLATES && !definedBOOST_NO_MEMBER_TEMPLATES_AND_bang_BOOST_MSVC6)" is defined rather than seeing this as two separate conditions Any help with this would be greatly appreciated. Thanks, ;-Brian -- Brian J. Carlson |
From: Brian J. C. <gma...@xo...> - 2009-11-17 05:51:35
|
Thank you for such a wonderful set of tools. I have recently started using CEDET with boost and have narrowed down a parsing "problem." The problem is that #ifdef pre-parsing is not evaluating correctly. An example of this type of statement is as follows: #if defined(BOOST_NO_MEMBER_TEMPLATES) && !defined(BOOST_MSVC6_MEMBER_TEMPLATES) I am using emacs 23: "GNU Emacs 23.1.50.1 (x86_64-pc-linux-gnu, GTK+ Version 2.18.0) of 2009-09-27 on crested, modified by Debian" with CEDET from the cvs repository at cedet.cvs.sourceforge.net) I executed the following emacs-lisp after starting emacs via 'emacs --no-init'): ,---- | (if (not (featurep 'cedet)) | (load-file "~/.elisp/cedet/common/cedet.elc")) | (global-ede-mode t) | (require 'ede-cpp-root) | (require 'semantic-gcc) | (ede-cpp-root-project "cedet-test" | :file "~/src/cedet-test/test.cpp") | (semantic-load-enable-excessive-code-helpers) | (require 'semantic-ia) `---- Here is the contents of test.cpp: ,---- | #undef BOOST_NO_MEMBER_TEMPLATES | #undef BOOST_MSVC6_MEMBER_TEMPLATES | | | #if defined(BOOST_NO_MEMBER_TEMPLATES) | int mymethod(int myint) { | return myint++; | } | #else | // This is what should be recognized and it is | int mymethod(int myint) { | return myint + 2; | } | #endif | | | #if !defined(BOOST_MSVC6_MEMBER_TEMPLATES) | // This is what should be recognized an it is | void mysecondmethod(int) { | return; | } | #else | void mysecondmethod(unsigned int) { | return; | } | #endif | | // So BOOST_NO_MEMBER_TEMPLATES is NOT defined | // and BOOST_MSVC6_MEMBER_TEMPLATES is NOT defined | | // so this is #if false && true --> | // should yield false, but it yields true | #if defined(BOOST_NO_MEMBER_TEMPLATES) && !defined(BOOST_MSVC6_MEMBER_TEMPLATES) | // This should not be the value that is parsed but it is... | void definedBOOST_NO_MEMBER_TEMPLATES_AND_bang_BOOST_MSVC6() | { | return; | } | #else | // This is what should be parsed but it is not | void not_definedBOOST_NO_MEMBER_TEMPLATES_AND_bang_BOOST_MSVC6() | { | return; | } | #endif | | // (senator-force-refresh) `---- At first glance (and I mean glance), it looks like the parser is evaluating if "BOOST_MSVC6_MEMBER_TEMPLATES && !definedBOOST_NO_MEMBER_TEMPLATES_AND_bang_BOOST_MSVC6)" is defined rather than seeing this as two separate conditions. Or the lexer is not dealing with the && or || in the #if xxxxxxxxxx && xxxxxxxx type of pre-processing directive. Any help with this would be greatly appreciated. Thanks, ;-Brian -- Brian J. Carlson |
From: Eric M. L. <er...@si...> - 2009-11-18 23:57:30
|
Hi Brian, Your assessment is correct. The preprocessor in CEDET for C doesn't do much for complex expressions. It's been in there for a couple years, and you're the first person to notice. :) The reason is that it isn't really a very simple problem. We are deep in the lexical analyzer, and suddenly we need a lexical analyzer to solve this problem. Right now we just have a bit of a short cut for the simple case of a single expression. Ideally, we'd use the lexical analyzer and a parser to solve the problem, except we are trying to do a single pass lexical analysis, instead of the tradition 2 pass that gcc and most other compilers employ. If you'd like to take a crack at this problem in semantic-c-do-lex-if, I expect that the solution is to change the regex in semantic-lex-c-if to exclude the expression, and to extract it 'by hand' in semantic-c-do-lex-if where it could then be evaluated. If you don't have time, hopefully I can examine this more after Thanksgiving. If your patch is over 10 lines long or so, I'll also need a release so your changes can be included in Emacs. Also, if it helps, you can disable semantic-c-obey-conditional-section-parsing-flag to just go with the 'whatever' approach. :) Eric Brian J. Carlson wrote: > Thank you for such a wonderful set of tools. > > I have recently started using CEDET with boost and have narrowed down a > parsing "problem." > > The problem is that #ifdef pre-parsing is not evaluating correctly. > > An example of this type of statement is as follows: > #if defined(BOOST_NO_MEMBER_TEMPLATES) && !defined(BOOST_MSVC6_MEMBER_TEMPLATES) > > > I am using emacs 23: > "GNU Emacs 23.1.50.1 (x86_64-pc-linux-gnu, GTK+ Version 2.18.0) > of 2009-09-27 on crested, modified by Debian" > with CEDET from the cvs repository at cedet.cvs.sourceforge.net) > > I executed the following emacs-lisp after starting > emacs via 'emacs --no-init'): > > ,---- > | (if (not (featurep 'cedet)) > | (load-file "~/.elisp/cedet/common/cedet.elc")) > | (global-ede-mode t) > | (require 'ede-cpp-root) > | (require 'semantic-gcc) > | (ede-cpp-root-project "cedet-test" > | :file "~/src/cedet-test/test.cpp") > | (semantic-load-enable-excessive-code-helpers) > | (require 'semantic-ia) > `---- > > Here is the contents of test.cpp: > ,---- > | #undef BOOST_NO_MEMBER_TEMPLATES > | #undef BOOST_MSVC6_MEMBER_TEMPLATES > | > | > | #if defined(BOOST_NO_MEMBER_TEMPLATES) > | int mymethod(int myint) { > | return myint++; > | } > | #else > | // This is what should be recognized and it is > | int mymethod(int myint) { > | return myint + 2; > | } > | #endif > | > | > | #if !defined(BOOST_MSVC6_MEMBER_TEMPLATES) > | // This is what should be recognized an it is > | void mysecondmethod(int) { > | return; > | } > | #else > | void mysecondmethod(unsigned int) { > | return; > | } > | #endif > | > | // So BOOST_NO_MEMBER_TEMPLATES is NOT defined > | // and BOOST_MSVC6_MEMBER_TEMPLATES is NOT defined > | > | // so this is #if false && true --> > | // should yield false, but it yields true > | #if defined(BOOST_NO_MEMBER_TEMPLATES) && !defined(BOOST_MSVC6_MEMBER_TEMPLATES) > | // This should not be the value that is parsed but it is... > | void definedBOOST_NO_MEMBER_TEMPLATES_AND_bang_BOOST_MSVC6() > | { > | return; > | } > | #else > | // This is what should be parsed but it is not > | void not_definedBOOST_NO_MEMBER_TEMPLATES_AND_bang_BOOST_MSVC6() > | { > | return; > | } > | #endif > | > | // (senator-force-refresh) > `---- > > > At first glance (and I mean glance), it looks like the parser is evaluating > if "BOOST_MSVC6_MEMBER_TEMPLATES && !definedBOOST_NO_MEMBER_TEMPLATES_AND_bang_BOOST_MSVC6)" is defined > rather than seeing this as two separate conditions. Or the lexer is not dealing with the && or || in the > #if xxxxxxxxxx && xxxxxxxx type of pre-processing directive. > > Any help with this would be greatly appreciated. > > Thanks, > ;-Brian > |
From: Brian J. C. <gma...@xo...> - 2009-11-19 19:35:13
|
"Eric M. Ludlam" <er...@si...> on Wed, 18 November 2009 (18:59 EST)] wrote: >> >> I have recently started using CEDET with boost and have narrowed down a >> parsing "problem." >> >> The problem is that #ifdef pre-parsing is not evaluating correctly. >> >> An example of this type of statement is as follows: >> #if defined(BOOST_NO_MEMBER_TEMPLATES) && !defined(BOOST_MSVC6_MEMBER_TEMPLATES) >> > Hi Brian, > > Your assessment is correct. The preprocessor in CEDET for C doesn't > do much for complex expressions. It's been in there for a couple years, > and you're the first person to notice. :) Yeah. Not often that I'm the first to notice stuff. > The reason is that it isn't really a very simple problem. We are > deep in the lexical analyzer, and suddenly we need a lexical analyzer to > solve this problem. Right now we just have a bit of a short cut for the > simple case of a single expression. Indeed it is not a very simple problem. > Ideally, we'd use the lexical analyzer and a parser to solve the > problem, except we are trying to do a single pass lexical analysis, > instead of the tradition 2 pass that gcc and most other compilers employ. > > If you'd like to take a crack at this problem in > semantic-c-do-lex-if, I expect that the solution is to change the regex > in semantic-lex-c-if to exclude the expression, and to extract it 'by > hand' in semantic-c-do-lex-if where it could then be evaluated. If you > don't have time, hopefully I can examine this more after Thanksgiving. I started by looking at that function into how to handle this and realized that it was anything but trivial. I'm not exactly a fluent (e)lisp developer but I've been looking for something to sink my teeth into for some time. I also looked at hideif.el (in emacs/lisp/progmodes) which gave another good start, but then started noticing that it would take some time to deal with anything besides the #if (defined(X) && defined(Y)) || defined(Z). But even that required some parsing. I'll give it a go and let you know if I find anything that will work. (of course, if I _actually_ do (which may be a stretch) you'll be welcome to use/throw away as you see fit. > > > If your patch is over 10 lines long or so, I'll also need a release > so your changes can be included in Emacs. No doubt the solution that I was thinking about would have been way over 10 lines. But maybe I'm not thinking about the correct approach. If I get something working, than I'll send you what I do and we'll figure out the paperwork if/when I do. > > > Also, if it helps, you can disable > semantic-c-obey-conditional-section-parsing-flag to just go with the > 'whatever' approach. :) I may have to do just that in the short term. Of course I also noticed that the following is also not quite right, but this is a result of my looking at the first problem and is not really that much of a problem for me right now: #define A #define B #if defined(A) void a(); #elif defined(B) void b(); #else void c(); #endif In this case both a(), and b() are parsed. Again not an issue for me and is obviously a sibling of the first issue. I'll keep working in my spare time to see if I can find a way to work with more complex preprocessor statements. Thanks, ;-b PS: Thanks again for your awesome set of tools. PPS: How'd the pumpkin toss go? Thanks, ;-Brian |
From: Eric M. L. <er...@si...> - 2009-11-19 22:09:45
|
Brian J. Carlson wrote: > > PPS: How'd the pumpkin toss go? > We threw a 9 lb pumpkin 693 feet. Woot! That was great for us, but 1st place in our division went 2087 ft, so we have room for improvement. http://www.siege-engine.com I'll dig into the ifdef part of the message perhaps this weekend. Eric |
From: Eric M. L. <er...@si...> - 2009-11-27 18:18:24
|
Hi, Thanks for the patch. You are right, this is a pretty big patch. Your comments indicate that you borrowed a lot from hideif. Since hideif is a part of Emacs, perhaps that code can be used directly? If not, since CEDET is also becoming a part of Emacs, the shared pieces could be extracted. If this level of complexity is really needed and there is not shared part with hideif, perhaps it is time to buckle down and write a real lexical pre-processor for Semantic. Though I tend to think that every time I have to pile a new hack on top of the existing preprocessor. As for your need of a release, I'm guessing from the comments that the code you wrote is relatively small vs what was borrowed from hideif? I counted 13 lines, plus a few lines in the original lexical fcn you modified. That might be ok, though if you enjoy hacking Emacs, it might be wise for you to provide them a release anyway. That would let you implement changes in hideif as necessary. Thanks Eric Brian J. Carlson wrote: > "Brian J. Carlson" <gma...@xo...> [Thu, 19 November 2009 (14:32 EST)] wrote: > >>>> I have recently started using CEDET with boost and have narrowed down a >>>> parsing "problem." >>>> >>>> The problem is that #ifdef pre-parsing is not evaluating correctly. >>>> >>>> An example of this type of statement is as follows: >>>> #if defined(BOOST_NO_MEMBER_TEMPLATES) && !defined(BOOST_MSVC6_MEMBER_TEMPLATES) >>>> >>> Hi Brian, >>> >>> Your assessment is correct. The preprocessor in CEDET for C doesn't >>> do much for complex expressions. > >>> If you'd like to take a crack at this problem in >>> semantic-c-do-lex-if, I expect that the solution is to change the regex >>> in semantic-lex-c-if to exclude the expression, and to extract it 'by >>> hand' in semantic-c-do-lex-if where it could then be evaluated. If you >>> don't have time, hopefully I can examine this more after Thanksgiving. > > Eric, > > I did take a quick crack at this. I borrowed from hideif.el and do some > parsing. It's not perfect (in that it doesn't evaluate C-Macros in an > #if pre-processor directive among other things), but it does a very > large number of the things I needed. It's extremely "hackish" and > probably doesn't follow the paradigm (ooooooh BS Bingo word) you were > thinking about. > > I'd be interested in working with this some more if you'd like. > > > > ------------------------------------------------------------------------ > > > >>> >>> If your patch is over 10 lines long or so, I'll also need a release >>> so your changes can be included in Emacs. > > Ummm it is well over 10 lines, so if you decide to include my changes, > let me know and I'll fill out the FSF (I'm assuming it's FSF since I > noticed that CEDET is going into the next emacs YEAH!!!) paperwork for > your changes. > >> Of course I also noticed that the following is also not quite right, but this >> is a result of my looking at the first problem and is not really that much of a >> problem for me right now: >> >> #define A >> #define B >> #if defined(A) >> void a(); >> #elif defined(B) >> void b(); >> #else >> void c(); >> #endif > > My changes still doesn't address this, but this was a very tricky piece, > to evaluate the first #if then know to skip over all the next > #elif statement. (i.e. difficult to know whether the first #if was > evaluated as true and the #elif at (point) was to be tested or not. > At least in the very short term. > > > Thanks, > > ;-Brian J. Carlson > Leominster, MA > > > ------------------------------------------------------------------------ > > ------------------------------------------------------------------------------ > Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day > trial. Simplify your report design, integration and deployment - and focus on > what you do best, core application coding. Discover what's new with > Crystal Reports now. http://p.sf.net/sfu/bobj-july > > > ------------------------------------------------------------------------ > > _______________________________________________ > Cedet-devel mailing list > Ced...@li... > https://lists.sourceforge.net/lists/listinfo/cedet-devel |
From: Brian J. C. <gma...@xo...> - 2009-11-25 19:05:32
|
> > Eric, > > I did take a quick crack at this. Sorry about the huge file attachment. I meant to send the cvs diff output. -- Thanks, ;-Brian J. Carlson Leominster, MA |
From: Eric M. L. <er...@si...> - 2010-01-20 04:21:03
|
Hi Brian, It's been a while, but I finally got to integrating your patch. I've been able to figure out what is going on, and can now use the hideif code directly without copying it. I also have a better idea of how many lines of code you wrote vs copied from hideif. If you can get a release to the FSF that would be great. You can just write to cop...@fs.... Tell them you have contributions to both EMACS and CEDET, and you will be sent the forms by email which you will need to send back via paper. Despite all that, I ran all my test suites and they pass, and I've checked this code into CVS. I can pull it out later if you have issues getting paper work signed and ready for the FSF. For all those watching at home, I don't know how the hideif API stability has been over the past few releases of Emacs/XEmacs. It would be great if you could test this on various Emacs flavors and let me know if there are issues. Thanks! Lastly, there is now a new side-effect feature! If you use CEDET/Semantic, and HIDEIF, you can set this variable: (setq semantic-c-takeover-hideif t) and hideif will now automatically use all the Semantic information for evaluating what to hide. Nifty! Except that I haven't tested it as I'm not a hideif user. If you are, try it at let me know how it goes, and we can add it to the manual as another augmented Emacs feature. Thanks! Eric On 11/25/2009 12:15 PM, Brian J. Carlson wrote: > "Brian J. Carlson"<gma...@xo...> [Thu, 19 November 2009 (14:32 EST)] wrote: > >>>> >>>> I have recently started using CEDET with boost and have narrowed down a >>>> parsing "problem." >>>> >>>> The problem is that #ifdef pre-parsing is not evaluating correctly. >>>> >>>> An example of this type of statement is as follows: >>>> #if defined(BOOST_NO_MEMBER_TEMPLATES)&& !defined(BOOST_MSVC6_MEMBER_TEMPLATES) >>>> >> >>> Hi Brian, >>> >>> Your assessment is correct. The preprocessor in CEDET for C doesn't >>> do much for complex expressions. > >>> If you'd like to take a crack at this problem in >>> semantic-c-do-lex-if, I expect that the solution is to change the regex >>> in semantic-lex-c-if to exclude the expression, and to extract it 'by >>> hand' in semantic-c-do-lex-if where it could then be evaluated. If you >>> don't have time, hopefully I can examine this more after Thanksgiving. > > Eric, > > I did take a quick crack at this. I borrowed from hideif.el and do some > parsing. It's not perfect (in that it doesn't evaluate C-Macros in an > #if pre-processor directive among other things), but it does a very > large number of the things I needed. It's extremely "hackish" and > probably doesn't follow the paradigm (ooooooh BS Bingo word) you were > thinking about. > > I'd be interested in working with this some more if you'd like. > > > > > > >>> >>> >>> If your patch is over 10 lines long or so, I'll also need a release >>> so your changes can be included in Emacs. > > Ummm it is well over 10 lines, so if you decide to include my changes, > let me know and I'll fill out the FSF (I'm assuming it's FSF since I > noticed that CEDET is going into the next emacs YEAH!!!) paperwork for > your changes. > >> Of course I also noticed that the following is also not quite right, but this >> is a result of my looking at the first problem and is not really that much of a >> problem for me right now: >> >> #define A >> #define B >> #if defined(A) >> void a(); >> #elif defined(B) >> void b(); >> #else >> void c(); >> #endif > > My changes still doesn't address this, but this was a very tricky piece, > to evaluate the first #if then know to skip over all the next > #elif statement. (i.e. difficult to know whether the first #if was > evaluated as true and the #elif at (point) was to be tested or not. > At least in the very short term. > > > Thanks, > > ;-Brian J. Carlson > Leominster, MA > > > > ------------------------------------------------------------------------------ > Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day > trial. Simplify your report design, integration and deployment - and focus on > what you do best, core application coding. Discover what's new with > Crystal Reports now. http://p.sf.net/sfu/bobj-july > > > > _______________________________________________ > Cedet-devel mailing list > Ced...@li... > https://lists.sourceforge.net/lists/listinfo/cedet-devel |
From: Eric M. L. <er...@si...> - 2010-01-23 02:46:38
|
To reply to my own message... I found some bugs in hideif, so I've updated the parse to be robust to hideif catastrophically failing. There is also a rather complex new test that exercises some of those nifty new features. Enjoy. Eric On 01/19/2010 11:21 PM, Eric M. Ludlam wrote: > Hi Brian, > > It's been a while, but I finally got to integrating your patch. > > I've been able to figure out what is going on, and can now use the > hideif code directly without copying it. I also have a better idea of > how many lines of code you wrote vs copied from hideif. If you can get > a release to the FSF that would be great. > > You can just write to cop...@fs.... Tell them you have > contributions to both EMACS and CEDET, and you will be sent the forms by > email which you will need to send back via paper. > > > Despite all that, I ran all my test suites and they pass, and I've > checked this code into CVS. I can pull it out later if you have issues > getting paper work signed and ready for the FSF. > > For all those watching at home, I don't know how the hideif API > stability has been over the past few releases of Emacs/XEmacs. It would > be great if you could test this on various Emacs flavors and let me know > if there are issues. Thanks! > > Lastly, there is now a new side-effect feature! If you use > CEDET/Semantic, and HIDEIF, you can set this variable: > > (setq semantic-c-takeover-hideif t) > > and hideif will now automatically use all the Semantic information for > evaluating what to hide. Nifty! Except that I haven't tested it as I'm > not a hideif user. If you are, try it at let me know how it goes, and > we can add it to the manual as another augmented Emacs feature. > > Thanks! > Eric > > > |