Re: [Cpptool-develop] Declaration parsing started...

[Baptiste L. <gai...@fr...>]

----- Original Message ----- 
From: "Andre Baresel" <and...@gm...>
To: "CppTool Mailing List" <Cpp...@li...>
Sent: Saturday, April 26, 2003 4:06 PM
Subject: Re: [Cpptool-develop] Declaration parsing started...


> Baptiste Lepilleur wrote:
>
> >Notes that concerning the function pointer, the most important place to
> >support them is typedef. In most of the code I have seen, function
pointers
> >were nearly always 'typedefed'.
> >
> Function pointers and even the declaration of 'operator'
> overloading/type-conversion can be detected
> now. The parser does recognize this and parses it correctly (hopefully!?
> see Tests), but no information
> is stored yet. We get Type-Specifiers and Unparsed-Declarators at the
> moment.

I skimmed over the test, and this looks very promising. Nice works !

> Well typedef itselfe is no problem, since it is just one keyword.
> Note that it can even stay in the middle of a declaration:
>
>    int typedef mydef;
>
> Looks strange ? yes, but the EBNF allows this.

What it the comon equivalent expression ? (That's the kind of stuff that
will probably never be used, just like the fact you can put braces around
parameter name).

> Tests took really long (see UnparsedDeclarationMutatorTests), but after
> all I managed a very clean
> solution. I first played arround with the VariableDeclMutator which does
> nearly the same job, but was
> not able to fullfill all tests and was to complex after adding
> functionality. I'll try to replace the variable-
> declaration-mutator as soon as the 'variable-detection' is handled by
> the new Parser.

Local variable declaration is very close to class attribute declaration, so
reuse should occur.

> Durring tests I also recognized a problem in parsing declaration which
> can not be solved with parser
> information only (I believe): (Has been added to the Wiki)
>
>       typedef int myint;
>
>       class x {
>       public:
>          static myint y;
>       };
>
>       myint ::x::y;        // <--- how to parse ?
> (variant a)  type "myint::x::y"  + no variable
> (variant b)  type "myint::x"     + variable "::y"
> (variant c)  type "myint"        + variable "::x::y"
>
>       btw: spaces are allowed between '::' and identifier ("x<space>::y"
> equals "x::y")
>       "class x ::z::y" has three meanings !

> I wrote a test on this (deactivated), but this problem is not high
priority.

Could you provide some examples of variable declaration ? For what I've
tried, it is not possible to prefix the name of the declared variable by ::.

int ::x;        // failed to compile => can not force x to be in global
namespace

The only place I could see this work is when you are referering to variable
x in some expression.

> >Concerning code clean-up, you might want to reuse the SourceBuilder class
we
> >use in the rfta library (or implement a similar class, more dedicated to
ast
> >node range). This would makes the tests easier to write, read and
maintain.
> >
> I was thinking about that, but than decide a different solution.
> For class body parsing I will check this test solution again.
> I more and more understand your test routines ;-)...

I've looked at some of the tests, and there is some 'magical' constants
appearing from nowhere for ast node range test. This makes it hard to
understand what is the range of a given node. How did you figure out their
values anyway ?

As for the test routines, they call it test driven design, or test first
design. I just made the environment to make it possible for us ;-). It was
actually the reason why I renounced to use Boost.Spirit (a generic
programming parser framework). Compile time were huge (just including the
headers in a cpp resulted in a compile time of around a minute !).

> Open tasks for me, at the moment:
>    - class body parsing, supporting additional declaration elements
> (e.g. public, protected keywords)

You may want to add QT extension too:

public slots:        // (any access modifier is ok before slots)
signals:

slots is a macro that expands to nothing, and signals expands to protected
(I think). More info: http://doc.trolltech.com/3.1/signalsandslots.html.

They should not be expanded, but memorized as additional access modifiers.

>         "UnparsedClassSpecifierMutator" will be created
>    - simple declarator parsing for detecting variables
>         "UnparsedDeclaratorMutator" will be created
>    - function header parsing, to detect parameters
>         "UnparsedFunctionHeaderMutator" will be created
>    - I'm thinking about a new Mutator to replace the MaxLODMutator.
>       This Mutator only accesses Mutator for more details at the
> specified Source Position.
>          It would allow us for Refactoring Operations to start parsing
> allways at SourceLevel and
>           than step down to the position where the user has activated
> some refactoring request.
>       How to name it ? what about             "PositionMaxLODMutator"
>     - An additional Mutator could be written which only parses source
> positions where some
>        identifier does appear. (reuse of the PositionMaxLODMutator)

I was thinking of something along the same lines. I've added a CPPParser
class, which should ultimately become the main interface to the rftaparser
library (at the current time, it's just a squeleton).

The only service we need at the current time is:
- find and parse the function body which encompass a specified location
(usualy, the user selection).

A service we will need later:
- parse all the source, but don't parse struct/class/function bodies

As for the implementation, I would go the other way: keep the MaxLODMutator
expanding everything (if given at top-level, everything get expanded). And I
would add another LOD mutator to implements second service (up to function
level). May be name it BodyLessLODMutator (we parse everything but the
bodies) or something like that.

The first service would be implemented roughly like this:
- parse at bodyless level (DeclarationListParser + BodyLessLODMutator)
- walk down the ast node tree until we find a body that includes the
specified position (error if not found)
- mutate the body with the MaxLODMutator
- return the function declaration node (will need to updated Refactoring to
start at this level, and obtains the compound statement node from that one).

We will probably need a service concerning identifiers, but will study that
when we get there (global renaming).

> For the future:
>     - to access identifier:
>       identifier resolver strategy for class/namespace/function header
>         --> we need named scopes, and methods to resolve scoped identifier

Yes, and that promise to be tough. We'll definitively use TDD for that one.
Hopyfully, we'll have a nice code model to work with at the time.

I'll also try to get something out so that we can run the new parser over a
large number of sources and study the output (much like astdumper).

Baptiste.
>
> -- André