opencxx-users

[Opencxx-users] Re: [Synopsis-devel] The Array type

[Stefan S. <se...@sy...>]

Hi Gilles,

Gilles J. Seguin wrote:
> On Fri, 2004-07-30 at 22:09, Stefan Seefeld wrote:
> 
>>Thanks. In fact, I'm working on the C++ parser right now,
>>to be able to expose a much more rich AST to python to allow
>>real code inspection and following that code generation.
>>Drop in if you have ideas or even want to help :-)
> 
> 
> Hello Stefan,
> 
> First a little introduction.
> I have offered Grzegorz Jakacki, from the opencxx project
> to update the configuration of opencxx.  That step is done.
> The change are visible on the branch rel_2_8.
> Grzegorz is targetting third week of august for releasing 2.8 .

yeah, I know :-)

> Next project in opencxx, increase test coverage.
> Any idea are welcome.

I'm doing that inside synopsis right now, i.e. I'm adding tests
to the opencxx backend inside synopsis. My tests are all organized
using the qmtest framework (www.qmtest.com), and I offered Grzegorz
to put something similar into place for his project.
That being said, I still hope (as you may know) that we can at some
point merge both projects. Right now I'm doing most work on synopsis,
offering from time to time to merge back fixes into the OpenC++ project.

> The main point of this email, how can I see the work that
> you are doing on the C++ parser.
> I have done 'svn checkout', so I have revision 1332.

I'm just starting my work on the backend. My plans are (in that order):

* rewrite some low level classes (Buffer, Lexer, basically)

* make the existing API more clean (const-correct, more typed, etc.)

* try to understand how the Parser needs to be changed in order to
   correctly understand the tokens, i.e. build a correct parse tree / AST
   (see recent posts on the opencxx list).

* build a true high-level AST API on top (no more Car() / Cdr() calls !)

* export that AST API publicly through C++ as well as python

* get rid of the occ executable, provide a python script instead (for
   backward compatibility only)

* solidify the C++ / python APIs to let users build their domain-specific
   applications either as python scripts or C++ programs

I realize this is a lot of work. I'm not sure what the timeframe is to get
all of this done. I'v almost finished the first point (the Buffer is committed,
the Lexer will follow one of these days).
If you are interested in getting involved, let me know.

Kind regards,
		Stefan

Re: [Opencxx-users] testsuite

[<se...@in...>]

On Fri, 2004-08-13 at 06:08, Gilles J. Seguin wrote:
> Tests for the occ binary on the PATH environment variable
>  <http://www.infonet.ca/segg/occ.tgz>
> will unpack in occ directory.
> $ cd occ

for attachment, untar here.
will add directories rtti.qms and rtti_src
$ qmtest run rtti

> $ qmtest run	# will run everything.
> $ qmtest run parse  # will do the same as 'qmtest run'
> since we have only one directory.
> $ qmtest run parse.access1  # will run only one test.
> 
> The tests examine results from 'occ -p -s myfile.cc'

Re: [Opencxx-users] testsuite

[<se...@in...>]

On Fri, 2004-08-13 at 06:47, Gilles J. Seguin wrote:
> On Fri, 2004-08-13 at 06:08, Gilles J. Seguin wrote:
> > Tests for the occ binary on the PATH environment variable
> >  <http://www.infonet.ca/segg/occ.tgz>
> > will unpack in occ directory.
> > $ cd occ
> 
> for attachment, untar here.

we need the attachment,
and do not reply to myself, duh.

> will add directories rtti.qms and rtti_src
> $ qmtest run rtti
> 
> > $ qmtest run	# will run everything.
> > $ qmtest run parse  # will do the same as 'qmtest run'
> > since we have only one directory.
> > $ qmtest run parse.access1  # will run only one test.
> > 
> > The tests examine results from 'occ -p -s myfile.cc'

Re: [Opencxx-users] Re: [Synopsis-devel] The Array type

[<se...@in...>]

On Sun, 2004-08-01 at 14:17, Stefan Seefeld wrote:
> Hi Gilles,
> 
> Gilles J. Seguin wrote:

[...]

> > Next project in opencxx, increase test coverage.
> > Any idea are welcome.
> 
> I'm doing that inside synopsis right now, i.e. I'm adding
> tests to the opencxx backend inside synopsis. My tests are
> all organized using the qmtest framework (www.qmtest.com),
> and I offered Grzegorz to put something similar into place
> for his project.

QMtest is a very god choice. QMtest is a general-purpose,
cross-platform software testing tools. It also have a gui
for creating, managing, and executing tests through a browser,
provides support for parallel test execution.

The first goal will be to provide a 'make test' target, that
is intended for developer, which increase the level of
confidence that changes are not affecting others functionalities
in an non intended way.  We will still provide a 'make check'
target that is generally done after ./configure && 'make',
to ensure functionality before performing 'make install'.

Stephan could you give me access to what you have already done.
Or things, sorry for my English, that can be done now.

We need to discuss what needs to be part of opencxx or synopsis.
And prioritization of work step.

> That being said, I still hope (as you may know) that we can at some
> point merge both projects. Right now I'm doing most work on synopsis,
> offering from time to time to merge back fixes into the OpenC++ project.
> 
> > The main point of this email, how can I see the work that
> > you are doing on the C++ parser.
> > I have done 'svn checkout', so I have revision 1332.
> 
> I'm just starting my work on the backend. My plans are (in that order):
> 
> * rewrite some low level classes (Buffer, Lexer, basically)
> 
> * make the existing API more clean (const-correct, more typed, etc.)
> 
> * try to understand how the Parser needs to be changed in order to
>    correctly understand the tokens, i.e. build a correct parse tree / AST
>    (see recent posts on the opencxx list).
> 
> * build a true high-level AST API on top (no more Car() / Cdr() calls !)
> 
> * export that AST API publicly through C++ as well as python
> 
> * get rid of the occ executable, provide a python script instead (for
>    backward compatibility only)
> 
> * solidify the C++ / python APIs to let users build their domain-specific
>    applications either as python scripts or C++ programs
> 
> I realize this is a lot of work. I'm not sure what the timeframe is to get
> all of this done. I'v almost finished the first point (the Buffer is committed,
> the Lexer will follow one of these days).
> If you are interested in getting involved, let me know.
> 
> Kind regards,
> 		Stefan

Re: [Opencxx-users] Re: [Synopsis-devel] The Array type

[Stefan S. <se...@sy...>]

Gilles J. Seguin wrote:

> QMtest is a very god choice. QMtest is a general-purpose,
> cross-platform software testing tools. It also have a gui
> for creating, managing, and executing tests through a browser,
> provides support for parallel test execution.
> 
> The first goal will be to provide a 'make test' target, that
> is intended for developer, which increase the level of
> confidence that changes are not affecting others functionalities
> in an non intended way.  We will still provide a 'make check'
> target that is generally done after ./configure && 'make',
> to ensure functionality before performing 'make install'.
> 
> Stephan could you give me access to what you have already done.
> Or things, sorry for my English, that can be done now.

Have a look into synopsis. To run the tests just follow the
instructions in the toplevel README file (executing
'python setup.py test' will run qmtest underneath). The code
I wrote to customize qmtest to my needs is in the tests/classes

http://synopsis.fresco.org/viewsvn/synopsis-Synopsis/trunk/tests/classes/

I currently only test opencxx through its single python entry point
('occ.parse'). I'll add C++ API tests for the Buffer and Lexer classes
once I'v finished them.

> We need to discuss what needs to be part of opencxx or synopsis.
> And prioritization of work step.

The list in my last mail reflects my own priorities. Feel free to
help out.

Regards,
		Stefan

Re: [Opencxx-users] Re: [Synopsis-devel] The Array type

[Grzegorz J. <ja...@ac...>]

Stefan Seefeld wrote:
> Hi Gilles,
> 
> Gilles J. Seguin wrote:
> 
>> On Fri, 2004-07-30 at 22:09, Stefan Seefeld wrote:
>>
>>> Thanks. In fact, I'm working on the C++ parser right now,
>>> to be able to expose a much more rich AST to python to allow
>>> real code inspection and following that code generation.
>>> Drop in if you have ideas or even want to help :-)
>>
>>
>>
>> Hello Stefan,
>>
>> First a little introduction.
>> I have offered Grzegorz Jakacki, from the opencxx project
>> to update the configuration of opencxx.  That step is done.
>> The change are visible on the branch rel_2_8.
>> Grzegorz is targetting third week of august for releasing 2.8 .
> 
> 
> yeah, I know :-)
> 
>> Next project in opencxx, increase test coverage.
>> Any idea are welcome.
> 
> 
> I'm doing that inside synopsis right now, i.e. I'm adding tests
> to the opencxx backend inside synopsis. My tests are all organized
> using the qmtest framework (www.qmtest.com), and I offered Grzegorz
> to put something similar into place for his project.

Just to clarify: I have never rejected it and this is still open. I 
tried to understand how Synopsis uses qmtest, but for misc. reasons I 
failed. Stefan, if you have running examples that do qmtesting of 
OpenC++, I would love to have a look. And generally, could you post 
instructions how to check out your OpenC++ "branch" from Synopsis repo?

And last but not least: I think that the first thing that should be 
established if we get down to increasing coverage is a mechanism to 
measure line coverage conveniently (using e.g. gcov). Otherwise speaking 
about increasing coverage is a little bit hand waving. If any of you 
guys want to take it, please do, I will be busy with other things at 
least till the end of August (samples, 2.8, merging to main, docs).

> That being said, I still hope (as you may know) that we can at some
> point merge both projects.  Right now I'm doing most work on synopsis,
> offering from time to time to merge back fixes into the OpenC++ project.

This is certainly suboptimal. However, I don't see any straightforward 
solution at the moment. I would suggest that we go through options 
again, so that we can have more convenient setup, say, in mid-September.

BR
Grzegorz

>> The main point of this email, how can I see the work that
>> you are doing on the C++ parser.
>> I have done 'svn checkout', so I have revision 1332.
> 
> 
> I'm just starting my work on the backend. My plans are (in that order):
> 
> * rewrite some low level classes (Buffer, Lexer, basically)
> 
> * make the existing API more clean (const-correct, more typed, etc.)
> 
> * try to understand how the Parser needs to be changed in order to
>   correctly understand the tokens, i.e. build a correct parse tree / AST
>   (see recent posts on the opencxx list).
> 
> * build a true high-level AST API on top (no more Car() / Cdr() calls !)
> 
> * export that AST API publicly through C++ as well as python
> 
> * get rid of the occ executable, provide a python script instead (for
>   backward compatibility only)
> 
> * solidify the C++ / python APIs to let users build their domain-specific
>   applications either as python scripts or C++ programs
> 
> I realize this is a lot of work. I'm not sure what the timeframe is to get
> all of this done. I'v almost finished the first point (the Buffer is 
> committed,
> the Lexer will follow one of these days).
> If you are interested in getting involved, let me know.
> 
> Kind regards,
>         Stefan
> 
> 
> -------------------------------------------------------
> This SF.Net email is sponsored by OSTG. Have you noticed the changes on
> Linux.com, ITManagersJournal and NewsForge in the past few weeks? Now,
> one more big change to announce. We are now OSTG- Open Source Technology
> Group. Come see the changes on the new OSTG site. www.ostg.com
> _______________________________________________
> Opencxx-users mailing list
> Ope...@li...
> https://lists.sourceforge.net/lists/listinfo/opencxx-users

Re: [Opencxx-users] Re: [Synopsis-devel] The Array type

[Stefan S. <se...@sy...>]

Grzegorz Jakacki wrote:

> Just to clarify: I have never rejected it and this is still open. I 
> tried to understand how Synopsis uses qmtest, but for misc. reasons I 
> failed. Stefan, if you have running examples that do qmtesting of 
> OpenC++, I would love to have a look. And generally, could you post 
> instructions how to check out your OpenC++ "branch" from Synopsis repo?

There are no OpenC++ tests checked in yet, I hope to check in something
useful this week. I did write some code to test the Buffer, Lexer, Parser,
and Encoding classes, though.

As to the code in general, just download a Synopsis snapshot from
http://synopsis.fresco.org/download/, the OpenC++ stuff is in
Synopsis/Parsers/Cxx (may be a bit unstable these days as I'm heavily
refactoring / rewriting).

> And last but not least: I think that the first thing that should be 
> established if we get down to increasing coverage is a mechanism to 
> measure line coverage conveniently (using e.g. gcov). Otherwise speaking 
> about increasing coverage is a little bit hand waving. If any of you 
> guys want to take it, please do, I will be busy with other things at 
> least till the end of August (samples, 2.8, merging to main, docs).

I don't quite agree. There are currently no unit tests, so 'coverage'
could just mean to write them starting with the most central classes
first (Buffer, Lexer, PTree classes, Encoding, Environment, Parser).

Regards,
		Stefan

Re: [Opencxx-users] testsuit (was: Re: [Synopsis-devel] The Array type)

[<se...@in...>]

On Tue, 2004-08-10 at 04:58, Grzegorz Jakacki wrote:
> Stefan Seefeld wrote:
> > Hi Gilles,
> > 
> > Gilles J. Seguin wrote:
> > 
> >> On Fri, 2004-07-30 at 22:09, Stefan Seefeld wrote:
> >>
> >>> Thanks. In fact, I'm working on the C++ parser right now,
> >>> to be able to expose a much more rich AST to python to allow
> >>> real code inspection and following that code generation.
> >>> Drop in if you have ideas or even want to help :-)
> >>
> >>
> >>
> >> Hello Stefan,
> >>
> >> First a little introduction.
> >> I have offered Grzegorz Jakacki, from the opencxx project
> >> to update the configuration of opencxx.  That step is done.
> >> The change are visible on the branch rel_2_8.
> >> Grzegorz is targetting third week of august for releasing 2.8 .
> > 
> > 
> > yeah, I know :-)
> > 
> >> Next project in opencxx, increase test coverage.
> >> Any idea are welcome.
> > 
> > 
> > I'm doing that inside synopsis right now, i.e. I'm adding tests
> > to the opencxx backend inside synopsis. My tests are all organized
> > using the qmtest framework (www.qmtest.com), and I offered Grzegorz
> > to put something similar into place for his project.
> 
> Just to clarify: I have never rejected it and this is still open. I 
> tried to understand how Synopsis uses qmtest, but for misc. reasons I 
> failed.

Tests for the occ binary on the PATH environment variable
 <http://www.infonet.ca/segg/occ.tgz>
will unpack in occ directory.
$ cd occ
$ qmtest run	# will run everything.
$ qmtest run parse  # will do the same as 'qmtest run'
since we have only one directory.
$ qmtest run parse.access1  # will run only one test.

The tests examine results from 'occ -p -s myfile.cc'

> Stefan, if you have running examples that do qmtesting of 
> OpenC++, I would love to have a look. And generally, could you post 
> instructions how to check out your OpenC++ "branch" from Synopsis repo?
> 
> And last but not least: I think that the first thing that should be 
> established if we get down to increasing coverage is a mechanism to 
> measure line coverage conveniently (using e.g. gcov). Otherwise speaking 
> about increasing coverage is a little bit hand waving. If any of you 
> guys want to take it, please do, I will be busy with other things at 
> least till the end of August (samples, 2.8, merging to main, docs).
> 
> > That being said, I still hope (as you may know) that we can at some
> > point merge both projects.  Right now I'm doing most work on synopsis,
> > offering from time to time to merge back fixes into the OpenC++ project.
> 
> This is certainly suboptimal. However, I don't see any straightforward 
> solution at the moment. I would suggest that we go through options 
> again, so that we can have more convenient setup, say, in mid-September.
> 
> BR
> Grzegorz
> 
> >> The main point of this email, how can I see the work that
> >> you are doing on the C++ parser.
> >> I have done 'svn checkout', so I have revision 1332.
> > 
> > 
> > I'm just starting my work on the backend. My plans are (in that order):
> > 
> > * rewrite some low level classes (Buffer, Lexer, basically)
> > 
> > * make the existing API more clean (const-correct, more typed, etc.)
> > 
> > * try to understand how the Parser needs to be changed in order to
> >   correctly understand the tokens, i.e. build a correct parse tree / AST
> >   (see recent posts on the opencxx list).
> > 
> > * build a true high-level AST API on top (no more Car() / Cdr() calls !)
> > 
> > * export that AST API publicly through C++ as well as python
> > 
> > * get rid of the occ executable, provide a python script instead (for
> >   backward compatibility only)
> > 
> > * solidify the C++ / python APIs to let users build their domain-specific
> >   applications either as python scripts or C++ programs
> > 
> > I realize this is a lot of work. I'm not sure what the timeframe is to get
> > all of this done. I'v almost finished the first point (the Buffer is 
> > committed,
> > the Lexer will follow one of these days).
> > If you are interested in getting involved, let me know.
> > 
> > Kind regards,
> >         Stefan

Re: [Opencxx-users] testsuit

[Stefan S. <se...@sy...>]

Gilles J. Seguin wrote:

> Tests for the occ binary on the PATH environment variable
>  <http://www.infonet.ca/segg/occ.tgz>
> will unpack in occ directory.
> $ cd occ
> $ qmtest run	# will run everything.
> $ qmtest run parse  # will do the same as 'qmtest run'
> since we have only one directory.
> $ qmtest run parse.access1  # will run only one test.
> 
> The tests examine results from 'occ -p -s myfile.cc'

nice ! Some suggestions, though:

you hardcode the path '/usr/bin/occ' in each single test descriptor.
I'v just changed one to see that it works, and I'm not going to edit
all these files :-)

What I'd suggest to do instead is defining a new test (python) class
that is then registered with the database and used for all tests.
Then you can configure paths and other stuff right into this class
*once*.

Second I think it would be good to substructure the tests. In fact
I'd suggest to write a new 'test database', and then use the subdivision
into test suites to set up test types for each suite separately.

That will allow to tailor the test types, for example one suite/folder
to test the Encoding class, one to test ptree generation / manipulation,
etc., etc.

Note that this is exactly the approach I'v taken in synopsis, i.e.
I wrote my own database and a handfull of test classes to do various
things (one test class compiles a C++ file and executes it, another
runs a particular synopsis processor on a specific input file, you
get the idea...)

That said, I think what you'v done is a nice starting point ! May
I integrate it into my existing testing harness ?

Regards,
		Stefan

Re: [Opencxx-users] testsuit

[<se...@in...>]

On Tue, 2004-08-10 at 19:16, Stefan Seefeld wrote:
> Gilles J. Seguin wrote:
> 
> > Tests for the occ binary on the PATH environment variable
> >  <http://www.infonet.ca/segg/occ.tgz>
> > will unpack in occ directory.
> > $ cd occ
> > $ qmtest run	# will run everything.
> > $ qmtest run parse  # will do the same as 'qmtest run'
> > since we have only one directory.
> > $ qmtest run parse.access1  # will run only one test.
> > 
> > The tests examine results from 'occ -p -s myfile.cc'
> 
> nice ! Some suggestions, though:
> 
> you hardcode the path '/usr/bin/occ' in each single test descriptor.
> I'v just changed one to see that it works, and I'm not going to
> edit all these files :-)
> 
> What I'd suggest to do instead is defining a new test (python) class
> that is then registered with the database and used for all tests.
> Then you can configure paths and other stuff right into this class
> *once*.
> 
> Second I think it would be good to substructure the tests. In fact
> I'd suggest to write a new 'test database', and then use the subdivision
> into test suites to set up test types for each suite separately.

tests are stolen from gcc/gcc/testsuite/g++.dg with branch tag
gcc-3_4-rhl-branch.
And follow the same structure.
The tar file was a copy of a test directory executing occ on
uninstall binary and library.  On which I have apply sed command on
all file.

> That will allow to tailor the test types, for example one suite/folder
> to test the Encoding class, one to test ptree generation / manipulation,
> etc., etc.

What I am trying to do now, is to generate source files base on
rules(rProgram, rDeclaration, rClassSpec).  Are they producing
valid ASTs ?
First step, document the Grammer. 

> Note that this is exactly the approach I'v taken in synopsis, i.e.
> I wrote my own database and a handfull of test classes to do various
> things (one test class compiles a C++ file and executes it, another
> runs a particular synopsis processor on a specific input file, you
> get the idea...)
> 
> That said, I think what you'v done is a nice starting point ! May
> I integrate it into my existing testing harness ?
> 
> Regards,
> 		Stefan

Re: [Opencxx-users] testsuit

[Grzegorz J. <ja...@ac...>]

Stefan Seefeld wrote:
> Gilles J. Seguin wrote:
> 
>> Tests for the occ binary on the PATH environment variable
>>  <http://www.infonet.ca/segg/occ.tgz>
>> will unpack in occ directory.
>> $ cd occ
>> $ qmtest run    # will run everything.
>> $ qmtest run parse  # will do the same as 'qmtest run'
>> since we have only one directory.
>> $ qmtest run parse.access1  # will run only one test.
>>
>> The tests examine results from 'occ -p -s myfile.cc'
> 
> 
> nice ! Some suggestions, though:
> 
> you hardcode the path '/usr/bin/occ' in each single test descriptor.
> I'v just changed one to see that it works, and I'm not going to edit
> all these files :-)
> 
> What I'd suggest to do instead is defining a new test (python) class
> that is then registered with the database and used for all tests.
> Then you can configure paths and other stuff right into this class
> *once*.
> 
> Second I think it would be good to substructure the tests. In fact
> I'd suggest to write a new 'test database', and then use the subdivision
> into test suites to set up test types for each suite separately.
> 
> That will allow to tailor the test types, for example one suite/folder
> to test the Encoding class, one to test ptree generation / manipulation,
> etc., etc.
> 
> Note that this is exactly the approach I'v taken in synopsis, i.e.
> I wrote my own database and a handfull of test classes to do various
> things (one test class compiles a C++ file and executes it, another
> runs a particular synopsis processor on a specific input file, you
> get the idea...)
> 
> That said, I think what you'v done is a nice starting point ! 

Absolutely.

 > May
> I integrate it into my existing testing harness ?

Stefan, could we steer toward integration of what you have in Synopsis 
with what is currently in opencxx.sf.net ? I.e. could you integrate 
Gilles'  development it in a way that can be backported to (or merged 
with) the opencxx from SF CVS?

Thanks
Grzegorz



> 
> Regards,
>         Stefan
> 
> 
> -------------------------------------------------------
> SF.Net email is sponsored by Shop4tech.com-Lowest price on Blank Media
> 100pk Sonic DVD-R 4x for only $29 -100pk Sonic DVD+R for only $33
> Save 50% off Retail on Ink & Toner - Free Shipping and Free Gift.
> http://www.shop4tech.com/z/Inkjet_Cartridges/9_108_r285
> _______________________________________________
> Opencxx-users mailing list
> Ope...@li...
> https://lists.sourceforge.net/lists/listinfo/opencxx-users

Re: [Opencxx-users] testsuit

[Stefan S. <se...@sy...>]

Grzegorz Jakacki wrote:

> Stefan, could we steer toward integration of what you have in Synopsis 
> with what is currently in opencxx.sf.net ? I.e. could you integrate 
> Gilles'  development it in a way that can be backported to (or merged 
> with) the opencxx from SF CVS?

sure,
		Stefan