opencxx-users

[Opencxx-users] Reverse-engineered architecture docs

[Grzegorz J. <ja...@ac...>]

Hi,

I checked in my attempt at reverse-engineering docs of OpenC++ 
architecture on the class level. The document is incomplete and most 
likely in flux, since architecture is changing now, but at least it 
describes status quo. Please review, fix, give feedback.

The docs are in opencxx/doc/architecture.html

BR
Grzegorz

Re: [Opencxx-users] Reverse-engineered architecture docs

[Raphael Y. de C. <rca...@im...>]

How can I find this document?

I used the CVS browser from SF, but I didn't find it there.
http://cvs.sourceforge.net/viewcvs.py/opencxx/opencxx/doc/

Am I looking in the correct place?

thanks,
Raphael

OBS: Grzegorz Jakacki, sorry for double email. The first was wrongly sent only
to you...

Citando Grzegorz Jakacki <ja...@ac...>:

> Hi,
> 
> I checked in my attempt at reverse-engineering docs of OpenC++ 
> architecture on the class level. The document is incomplete and most 
> likely in flux, since architecture is changing now, but at least it 
> describes status quo. Please review, fix, give feedback.
> 
> The docs are in opencxx/doc/architecture.html
> 
> BR
> Grzegorz
> 
> 
> 
> -------------------------------------------------------
> This SF.Net email is sponsored by BEA Weblogic Workshop
> FREE Java Enterprise J2EE developer tools!
> Get your free copy of BEA WebLogic Workshop 8.1 today.
> http://ads.osdn.com/?ad_id=4721&alloc_id=10040&op=click
> _______________________________________________
> Opencxx-users mailing list
> Ope...@li...
> https://lists.sourceforge.net/lists/listinfo/opencxx-users
> 

Raphael Yokoingawa de Camargo <rca...@im...>

Re: [Opencxx-users] Reverse-engineered architecture docs

[Grzegorz J. <ja...@ac...>]

Raphael Yokoingawa de Camargo wrote:
> How can I find this document?
> 
> I used the CVS browser from SF, but I didn't find it there.
> http://cvs.sourceforge.net/viewcvs.py/opencxx/opencxx/doc/
> 
> Am I looking in the correct place?

Yes. On SF anonymous CVS (and webcvs) have a propagation delay of ~6 
hours wrt. to the main CVS. The files are visible already, but most 
likely were not when you first checked.

> 
> thanks,
> Raphael
> 
> OBS: Grzegorz Jakacki, sorry for double email. The first was wrongly sent only
> to you...

No problem.

Best regardz
Grzegorz

> 
> Citando Grzegorz Jakacki <ja...@ac...>:
> 
> 
>>Hi,
>>
>>I checked in my attempt at reverse-engineering docs of OpenC++ 
>>architecture on the class level. The document is incomplete and most 
>>likely in flux, since architecture is changing now, but at least it 
>>describes status quo. Please review, fix, give feedback.
>>
>>The docs are in opencxx/doc/architecture.html
>>
>>BR
>>Grzegorz
>>
>>
>>
>>-------------------------------------------------------
>>This SF.Net email is sponsored by BEA Weblogic Workshop
>>FREE Java Enterprise J2EE developer tools!
>>Get your free copy of BEA WebLogic Workshop 8.1 today.
>>http://ads.osdn.com/?ad_id=4721&alloc_id=10040&op=click
>>_______________________________________________
>>Opencxx-users mailing list
>>Ope...@li...
>>https://lists.sourceforge.net/lists/listinfo/opencxx-users
>>
> 
> 
> Raphael Yokoingawa de Camargo <rca...@im...>
> 
> 
> -------------------------------------------------------
> This SF.Net email is sponsored by BEA Weblogic Workshop
> FREE Java Enterprise J2EE developer tools!
> Get your free copy of BEA WebLogic Workshop 8.1 today.
> http://ads.osdn.com/?ad_id=4721&alloc_id=10040&op=click
> _______________________________________________
> Opencxx-users mailing list
> Ope...@li...
> https://lists.sourceforge.net/lists/listinfo/opencxx-users

Re: [Opencxx-users] Reverse-engineered architecture docs

[Stefan S. <se...@sy...>]

Grzegorz Jakacki wrote:
> Hi,
> 
> I checked in my attempt at reverse-engineering docs of OpenC++ 
> architecture on the class level. The document is incomplete and most 
> likely in flux, since architecture is changing now, but at least it 
> describes status quo. Please review, fix, give feedback.
> 
> The docs are in opencxx/doc/architecture.html

nice work, thanks a lot !

Stefan

Re: [Opencxx-users] Reverse-engineered architecture docs

[Stefan S. <se...@sy...>]

Grzegorz Jakacki wrote:
> Hi,
> 
> I checked in my attempt at reverse-engineering docs of OpenC++ 
> architecture on the class level. The document is incomplete and most 
> likely in flux, since architecture is changing now, but at least it 
> describes status quo. Please review, fix, give feedback.
> 
> The docs are in opencxx/doc/architecture.html

wouldn't it be a good idea to publish these documents on the website ?

I'm working my way through the code as I'm cleaning up the 'synopsis branch'.
Also, our latest discussions with Chiba have helped me a lot to understand
the current approach...

Something that has to be discussed quite in detail for anybody to understand
the design of the parser (notably the Walker stuff) is the two-stage parsing
that is done, i.e. the Parser building up a preliminary parse tree and then
the Walker running over it creating the various 'Environments'. (Up to our
discussion a while ago I just couldn't figure out why the Walker modified
the ptree it was traversing. Now I see and understand that it needs to to
fill in the things the parser's first pass couldn't figure out yet.)

A simple example would help to illustrate this:

-----
struct Foo {};
void bar(const Foo &);
-----

creating a parse / syntax tree for this using occ is done in two steps:

* the first builds a preliminary ptree via Lexer -> Parser
* the second refines the ptree using the Walker to build an 'Environment'
   containing the declaration of 'Foo', which is needed to properly encode
   the name/type of 'bar' (in the first pass the parser has to *guess* some
   details about the function arguments, as no Environment object is available
   for it to inspect).

I hope this makes some sense...

Now, I believe it would be important to try to move this to a one-step
parse, i.e. integrate the Environment building into the parser such that
the parser can correctly deal with corner cases that used to be interpreted
incorrectly.

Regards,
		Stefan

Re: [Opencxx-users] Reverse-engineered architecture docs

[Grzegorz J. <ja...@ac...>]

Stefan Seefeld wrote:
> Grzegorz Jakacki wrote:
> 
>> Hi,
>>
>> I checked in my attempt at reverse-engineering docs of OpenC++ 
>> architecture on the class level. The document is incomplete and most 
>> likely in flux, since architecture is changing now, but at least it 
>> describes status quo. Please review, fix, give feedback.
>>
>> The docs are in opencxx/doc/architecture.html
> 
> 
> wouldn't it be a good idea to publish these documents on the website ?

I am putting it on my TODO list. However, anybody with developer's 
access is capable of doing it too, so if anybody feels like it, please 
do (check out module

> I'm working my way through the code as I'm cleaning up the 'synopsis 
> branch'.
> Also, our latest discussions with Chiba have helped me a lot to understand
> the current approach...
> 
> Something that has to be discussed quite in detail for anybody to 
> understand
> the design of the parser (notably the Walker stuff) is the two-stage 
> parsing
> that is done, i.e. the Parser building up a preliminary parse tree and then
> the Walker running over it creating the various 'Environments'. (Up to our
> discussion a while ago I just couldn't figure out why the Walker modified
> the ptree it was traversing. Now I see and understand that it needs to to
> fill in the things the parser's first pass couldn't figure out yet.)
> A simple example would help to illustrate this:
> 
> -----
> struct Foo {};
> void bar(const Foo &);
> -----
> 
> creating a parse / syntax tree for this using occ is done in two steps:
> 
> * the first builds a preliminary ptree via Lexer -> Parser
> * the second refines the ptree using the Walker to build an 'Environment'
>   containing the declaration of 'Foo', which is needed to properly encode
>   the name/type of 'bar' (in the first pass the parser has to *guess* some
>   details about the function arguments, as no Environment object is 
> available
>   for it to inspect).
> 
> I hope this makes some sense...

This is classical way of doing analysis in compilation, see "Compilers. 
Principles, Techniques and Tools" by Aho, Seti and Ullman, chapter 1.2.
I don't have English version at hand, so I cannot quote, sorry.

The only catch is that C++ grammar (as per ISO Standard) is not 
context-free, so the AST resulting from parsing is indeed an AST with 
"gray areas", which are clarified later during context-dependant analysis.

One factor that makes this pattern messy in OpenC++ is that Walker 
(ClassWalker, strictly speaking) performs not only context-dependant 
analysis, but also source-to-source translation.

> Now, I believe it would be important to try to move this to a one-step
> parse, i.e. integrate the Environment building into the parser such that
> the parser can correctly deal with corner cases that used to be interpreted
> incorrectly.

This would be great, but I am afraid it is impossible. Consider:

class A
{
     void f() { g(); }
     void g() {};
};

So we are doomed to perform some form of two-pass analysis, in 
particular binding cannot be performed together with parsing.

Apart of that, I still oppose integrating too much into parser. I 
suggest functionally equivalent solution of making higher-level services
(e.g. environment creation, identifiers lookup) available to parser 
through an abstract interface. That way parser is not coupled to the 
rest of the code. (Perhaps this is what you meant when you wrote about 
"integrating the Environment building into parser" and my comment is 
unnecessary, but I wanted to be certain we have a closure on that.)

Best regards
Grzegorz


> Regards,
>         Stefan
> 
> 
> -------------------------------------------------------
> SF.Net email is sponsored by Shop4tech.com-Lowest price on Blank Media
> 100pk Sonic DVD-R 4x for only $29 -100pk Sonic DVD+R for only $33
> Save 50% off Retail on Ink & Toner - Free Shipping and Free Gift.
> http://www.shop4tech.com/z/Inkjet_Cartridges/9_108_r285
> _______________________________________________
> Opencxx-users mailing list
> Ope...@li...
> https://lists.sourceforge.net/lists/listinfo/opencxx-users

Re: [Opencxx-users] Reverse-engineered architecture docs

[Stefan S. <se...@sy...>]

Grzegorz Jakacki wrote:

> One factor that makes this pattern messy in OpenC++ is that Walker 
> (ClassWalker, strictly speaking) performs not only context-dependant 
> analysis, but also source-to-source translation.

yeah, and this is indeed where my confusion started. It would already
be less confusing if there were multiple Walkers, one to finish the
parsing, and others to do the mop stuff.

>> Now, I believe it would be important to try to move this to a one-step
>> parse, i.e. integrate the Environment building into the parser such that
>> the parser can correctly deal with corner cases that used to be 
>> interpreted
>> incorrectly.
> 
> 
> This would be great, but I am afraid it is impossible. Consider:
> 
> class A
> {
>     void f() { g(); }
>     void g() {};
> };
> 
> So we are doomed to perform some form of two-pass analysis, in 
> particular binding cannot be performed together with parsing.

Hmm, you are right. I'll have to look again into the gcc sources
to see how they do it...

> Apart of that, I still oppose integrating too much into parser. I 
> suggest functionally equivalent solution of making higher-level services
> (e.g. environment creation, identifiers lookup) available to parser 
> through an abstract interface.

That's what I'm thinking about indeed. The parser would have a reference
to a 'SymbolTable', and each time it enters a scope it calls table->new_scope(),
each time it runs into a declaration it pushes it into the current scope, etc.

I'm still considering what it takes to make occ accept C code.
The lexer is easy, as we can at runtime mask some keywords (i.e. report them
as identifiers). The parser may be relatively easy, too, as we only have to
exclude a number of productions and may be add one or two (C is almost a
subset of C++). The SymbolTable finally would need some flexibility, too,
as name lookup doesn't follow the same rules (well, there are far less rules
to start with :-)

Regards,
		Stefan