Re: [Opencxx-users] parse error on valid code

[Grzegorz J. <ja...@ac...>]

Stefan Seefeld wrote:
> Grzegorz Jakacki wrote:
> 
>>> Ah, now I understand again ! So, the solution is to add a type 
>>> dictionary
>>> ('Environment' ?) to the Parser and then let the 'isTypeSpecifier' 
>>> use that
>>> instead of just detecting built-in types. Right ?
>>
>>
>>
>> For many cases this should work. However in general it will not.
> 
> 
> understood. Putting a type dictionary right into the parser means
> that the type recovery can't be done in a second pass, but instead
> has to be done as soon as a new type declaration has been detected.
> 
> The separation between parser and mop libraries doesn't make sense
> in that context any more...

I think it still does. First of all, Parser should not depend on 
Environment, just on an interface that would allow to get what it needs 
from environment:

                     (iface)
+---------+     +--------------------+
| Parser  |---->| TemplatesResolver  |
+---------+     +--------------------+
                         |
                        /_\
                         |
                     +---------------+
                     | Environment   |
                     +---------------+

It would allow at least to test parser in separation from the MOP.
Moreover, observe, that this interface needs to cover only a part of 
Environment functionality, in particular it does not have to deal with 
variable declarations at all.

Even if we move all processing to one-pass model, I would strongly 
recommend that parser do not depend on concrete MOP classes, but only on 
interfaces. It gives a chance to reuse parser in another context, also 
to test it without touching the rest of the system. Good example of such 
parser design is XercesC++ (XML parser from Apache Foundation).

> 
>> The top-level environment is maintained by ClassWalker. Driver in the 
>> main loop runs Parser::rProgram() and feeds the obtained AST into 
>> ClassWalker. Parser::rProgram() returns AST representing one top-level 
>> construct (definition or declaration), so one iteration analyzes one 
>> top-level construct. Only when ClassWalker traverses the AST, the 
>> information about types defined in it are stuffed into Environment.
> 
> 
> what does 'ClassWalker' do ? The name suggests it is only inspecting
> class definitions, but not you are suggesting it deals with arbitrary
> type recovery.

AFAIK it traverses the AST and registers all definitions and 
declarations in the Environment it maintains.

>> This is fair enough when there is no namespaces or nested templates. 
>> However, if top-level construct is a namespace, then Environment does 
>> not have a clue about anything defined in it until it is fully parsed.
> 
> 
> hmm, I don't understand the design of Environment et al. yet. Why doesn't
> it 'have a clue' until it is 'fully parsed' ? In the example
> 
> namespace Foo
> {
> typedef int Int;
> typedef std::vecor<Int> IntVector;
> }
> 
> Shouldn't it know 'Int' when it sees the typedef for IntVector ?

The top-level loop will call parser. Parser will return AST representing 
all the above text. At this moment the text is already parsed, but 
Environment object still does not know anything about Int or IntVector. 
Only now the main loop calls ClassWalker, which begins to traverse the 
AST. The traversal reaches first typedef and registers Int in top-level 
Environment, then second typedef and registers IntVector.

So, Environment does not know about Int or IntVector until the parsing 
of the whole namespace ends.

> 
>> I don't think there is an easy and complete solution. I have one 
>> solution in mind, but it is neither cheap, nor easy.
> 
> 
> Please document the current design and the solutions you envision. It will
> help a lot to work out the remaining issues to make opencxx a very powerful
> tool that supports all standard C++.

I am putting it really high on my TODO list, I will try to post it this 
week. As for existing design, however, my knowledge is limited to what I 
absorbed or figured out, so I am afraid there are going to be some dark 
corners.

> More and more people use synopsis, and the bug reports about parse errors
> I get become more and more sophisticated, indicating that it becomes more
> and more important to make the parser work correct not only for the most
> used C++ features, but really covering the full standard.

That's great news. There is still a lot of work, but let's keep moving 
forward.

BR
Grzegorz