opencxx-users

[Opencxx-users] scriptable code generation

[Stefan S. <se...@sy...>]

hi there,

many moons ago we have discussed whether or not it would
be useful to expose the OpenC++ API to python and thus
to allow to drive the process of parsing and code generation
to be scripted.

As part of synopsis I'v done some experimentation, and
attached to this mail are some first results. The file
is a python script that uses python modules 'PTree',
'SymbolLookup', and 'Processor' to access the respective
APIs directly (this implements the design I posted earlier
last summer).
This particular example looks for function definitions
and inserts comments containing the function's name
at the beginning of each function block.

Beside the advantage we had discussed earlier to remove
the need for (portable) management of sub-processes, dynamic
modules, etc., there are also other advantages I can see:

With such a scripting approach, I'm seriously wondering
whether there is still a need to 'augment' the code in-place,
essentially changing the programming language.

Why not store the rules that drive the code manipulation
outside the code, as part of the script that drives the
processor ?

I have got quite positive comments from users who expressed
concern that the form of metaprogramming promoted by OpenC++
would require users not only to be expert in C++, but also
to know the additional dialects to use to drive the code
generation.

What do other OpenC++ users think ? What is the advantage
of this form of metaprogramming as opposed to scriptable
code manipulation driven by external rules ?

I'd appreciate any feedback,

		Stefan

Re: [Opencxx-users] scriptable code generation

[Grzegorz J. <ja...@ac...>]

Stefan Seefeld wrote:
> hi there,
> 
> many moons ago we have discussed whether or not it would
> be useful to expose the OpenC++ API to python and thus
> to allow to drive the process of parsing and code generation
> to be scripted.
> 
> As part of synopsis I'v done some experimentation, and
> attached to this mail are some first results. The file
> is a python script that uses python modules 'PTree',
> 'SymbolLookup', and 'Processor' to access the respective
> APIs directly (this implements the design I posted earlier
> last summer).
> This particular example looks for function definitions
> and inserts comments containing the function's name
> at the beginning of each function block.
> 
> Beside the advantage we had discussed earlier to remove
> the need for (portable) management of sub-processes, dynamic
> modules, etc., there are also other advantages I can see:
> 
> With such a scripting approach, I'm seriously wondering
> whether there is still a need to 'augment' the code in-place,
> essentially changing the programming language.

I see a lot of confusion here. There is no need to use any language 
extensions in order to use OpenC++ framework or OpenC++Core library.

> Why not store the rules that drive the code manipulation
> outside the code, as part of the script that drives the
> processor ?

This is what OpenC++ framework does today, except that it uses (vanila) 
C++ instead of scripting languages.

> I have got quite positive comments from users who expressed
> concern that the form of metaprogramming promoted by OpenC++
> would require users not only to be expert in C++, but also
> to know the additional dialects to use to drive the code
> generation.

It is a pity that you do not give names and quotes. I also wonder why I 
am not seeing these concerns on this list. Chiba designed really sound 
model that serves many people well over many years, perhaps your users 
are missing something. I am expressing the concern that you are 
spreading FUD about OpenC++ model.

> What is the advantage
> of this form of metaprogramming as opposed to scriptable
> code manipulation driven by external rules ?

Very nice idea. If you want to go ahead and add it to OpenC++ that would 
be great. But the code you are posting is not usable with OpenC++ as far 
as I can see, only with Synopsis. And Synopsis has locked its 
OpenC++-derived code under the license that makes it impossible for 
OpenC++ to leverage the code etc. etc. This issue has been beaten to 
death on this forum. Your posting is Synopsis-specific material (not to 
mention the fact that by spreading FUD clearly harms OpenC++ project). I 
asked you several times not to post messages of this kind here, but you 
seem not to understand. This is the last time I am asking you.

Regards
Grzegorz

Re: [Opencxx-users] scriptable code generation

[Stefan S. <se...@sy...>]

Grzegorz,

>> With such a scripting approach, I'm seriously wondering
>> whether there is still a need to 'augment' the code in-place,
>> essentially changing the programming language.
> 
> 
> I see a lot of confusion here. There is no need to use any language 
> extensions in order to use OpenC++ framework or OpenC++Core library.

I'm not talking about using OpenC++ in general, I'm talking about
code generation.

Users have to know both, C++ as well as certain syntax extensions
(http://opencxx.sourceforge.net/opencxx/html/base.html) in order
to manipulate the code.

>> I have got quite positive comments from users who expressed
>> concern that the form of metaprogramming promoted by OpenC++
>> would require users not only to be expert in C++, but also
>> to know the additional dialects to use to drive the code
>> generation.
> 
> 
> It is a pity that you do not give names and quotes. I also wonder why I 
> am not seeing these concerns on this list. Chiba designed really sound 
> model that serves many people well over many years, perhaps your users 
> are missing something. I am expressing the concern that you are 
> spreading FUD about OpenC++ model.

I'm not bashing the OpenC++ programming model in any way. As you say,
it has served its useres very well over the years. However, when it
was first developed, alternative techniques such as hybrid programming
(http://www.boost-consulting.com/writing/bpl.html) weren't really available.
Now they are, and so I'm comparing these new possibilities.

I don't understand your hostility and aggression. You seem to be
annoyed by me working on Synopsis instead of OpenC++.
While I find the incompatibilities of our visions quite unfortunate,
I still think there is room for conceptual / design discussions that
doesn't lock any of us into either project specifics. You are
asking for collaboration, but you are the first one to shut down any
attempt I make to get a constructive discussion going. I think you are
painting yourself into a corner.

Regards,
		Stefan

Re: [Opencxx-users] scriptable code generation

[Grzegorz J. <ja...@ac...>]

Stefan Seefeld wrote:
> Grzegorz,
> 
>>> With such a scripting approach, I'm seriously wondering
>>> whether there is still a need to 'augment' the code in-place,
>>> essentially changing the programming language.
>>
>>
>>
>> I see a lot of confusion here. There is no need to use any language 
>> extensions in order to use OpenC++ framework or OpenC++Core library.
> 
> 
> I'm not talking about using OpenC++ in general, I'm talking about
> code generation.
> 
> Users have to know both, C++ as well as certain syntax extensions
> (http://opencxx.sourceforge.net/opencxx/html/base.html) in order
> to manipulate the code.

This is false. Cf. Class::ChangeDefaultMetaclass() or see occ-core.

>>> I have got quite positive comments from users who expressed
>>> concern that the form of metaprogramming promoted by OpenC++
>>> would require users not only to be expert in C++, but also
>>> to know the additional dialects to use to drive the code
>>> generation.
>>
>>
>>
>> It is a pity that you do not give names and quotes. I also wonder why 
>> I am not seeing these concerns on this list. Chiba designed really 
>> sound model that serves many people well over many years, perhaps your 
>> users are missing something. I am expressing the concern that you are 
>> spreading FUD about OpenC++ model.
> 
> 
> I'm not bashing the OpenC++ programming model in any way. As you say,
> it has served its useres very well over the years. However, when it
> was first developed, alternative techniques such as hybrid programming
> (http://www.boost-consulting.com/writing/bpl.html) weren't really 
> available.
> Now they are, and so I'm comparing these new possibilities.

You do, by suggesting that existing model has disadvantages, which it in 
fact does not have. It is orthogonal to the question if pythonic API 
would be useful (it would, IMHO).

> I don't understand your hostility and aggression. You seem to be
> annoyed by me working on Synopsis instead of OpenC++.

Of course not. Work on what you want to work, just play fair.

You took OpenC++ code and you put it in your project. You solicited 
substantial time and work from OpenC++ community (e.g. I wrote 
architecture docs per your request). This helped you to improve 
Synopsis's branch of OpenC++. However, despite many requests, you refuse 
to share your improvements under Open++ license -- you locked it under 
LGPL, which is much more restrictive.

Your license lock resulted in a situation, where the improved version of 
Synopsis's OpenC++ is more attractive than the original version, yet the 
improvements cannot be backported. This attracts developers away from 
OpenC++ to Synopsis.

Also for some reason you refused to build both Synopsis and 
OpenC++framework on an independent common core library, being an 
intersection of Synopsis and OpenC++. In particular you were opposing 
the idea of making such a library an independently buildable package. 
Clearly such core library would get most attention, since it would be in 
a focal point of both Synopsis and OpenC++ developers. Perhaps you were 
afraid that as a standalone package it will distract manpower from 
Synopsis proper.

> While I find the incompatibilities of our visions quite unfortunate,
> I still think there is room for conceptual / design discussions that
> doesn't lock any of us into either project specifics. You are
> asking for collaboration, but you are the first one to shut down any
> attempt I make to get a constructive discussion going. I think you are
> painting yourself into a corner.

I don't trust in your honest intentions any more. I think you just want 
to attract more manpower to your project, period. Let me remind you that 
when I was pointing out that your LGPL-lockin is unfair to OpenC++ you 
were moderating my posts out, to prevent them from appearing on Synopsis 
mailing list. How can I trust you now? Why should I invest time in 
discussing architecture with you, just to find out that my project 
cannot reap any benefits from it?

Having said that, I view your Synopsis-centered postings on this list 
just as a way of attracting manpower to your LGPL-locked project.

Regards,
Grzegorz

Re: [Opencxx-users] scriptable code generation

[Stefan S. <se...@sy...>]

Grzegorz Jakacki wrote:
> Stefan Seefeld wrote:

>> I'm not bashing the OpenC++ programming model in any way. As you say,
>> it has served its useres very well over the years. However, when it
>> was first developed, alternative techniques such as hybrid programming
>> (http://www.boost-consulting.com/writing/bpl.html) weren't really 
>> available.
>> Now they are, and so I'm comparing these new possibilities.
> 
> 
> You do, by suggesting that existing model has disadvantages, which it in 
> fact does not have. It is orthogonal to the question if pythonic API 
> would be useful (it would, IMHO).

Suggesting that I bash OpenC++ and that I spread 'FUD' just because I
ask you to take a fresh look at metaprogramming is quite a
misrepresentation.
My question was not about whether or not to export an API to python,
but to compare advantages and disadvantates of using language extensions
for metaprogramming as opposed to scripting.

I appreciate you replying you don't find the existing model having
disadvantages. What are the reasons for your thinking ? If you think
the scripting approach I suggest is a good alternative, how would you
compare both ?

> You took OpenC++ code and you put it in your project. You solicited 
> substantial time and work from OpenC++ community (e.g. I wrote 
> architecture docs per your request). This helped you to improve 
> Synopsis's branch of OpenC++.

Indeed it did. Any good discussion will trigger new development. I'm
quite sorry that you see this very one-sided, though, as I would hope
that you, too, would have enjoyed the exchange. The may be single most
valuable aspect in Free Software is that by construction you can't prohibit
anybody from using the work of others. It thus stresses the collective
aspect of knowledge. I totally fail to understand how you experienced
this as if I had taken away something from you ('substantial time and work').

> Your license lock resulted in a situation, where the improved version of 
> Synopsis's OpenC++ is more attractive than the original version, yet the 
> improvements cannot be backported. This attracts developers away from 
> OpenC++ to Synopsis.

If that is indeed the case it may be a good time to rethink your licensing
terms then.

> Also for some reason you refused to build both Synopsis and 
> OpenC++framework on an independent common core library, being an 
> intersection of Synopsis and OpenC++. In particular you were opposing 
> the idea of making such a library an independently buildable package. 
> Clearly such core library would get most attention, since it would be in 
> a focal point of both Synopsis and OpenC++ developers. Perhaps you were 
> afraid that as a standalone package it will distract manpower from 
> Synopsis proper. 

Certainly not, as I had to do the work anyways. It was, as you may remember,
a matter of project management.

I'm still trying to get this discussion back to where I started it, i.e.
to compare various approaches to metaprogramming. The fact that I draw
my current design ideas from my experience with synopsis does by no means
imply that this is a 'synopsis-centric' discussion.

In particular, if you find the ideas I bring forward valuable, I'd certainly
be more than happy to help apply them to OpenC++.

What do other OpenC++ developers and users think ?

Regards,
		Stefan

Re: [Opencxx-users] scriptable code generation

[Brian K. <bk...@fr...>]

Hi,

I've been using OpenC++ for about a year to implement some extensions to C++ to 
enable parallel programming.  My usage model has been to link my metaclass 
directly with libocc.  So, the user never uses the `metaclass` syntax, just the 
syntax extensions that I've added.

 From my point of view, the scripting extensions you're suggesting sound 
interesting, but I'm a little concerned with performance.  After all, a 
seemingly small C++ file that includes a few STL headers quickly grows to be 
tens of thousands of lines long.

One other question I had was whether you're using SWIG for generating the 
interface.  Python isn't my favorite language, but if you're using SWIG then I 
could conceivably use Bigloo or Ocaml as my main implementation language while 
using occ as the parser.

Brian Kahne
PowerPC Design Center
Freescale Semiconductor Inc.


Stefan Seefeld wrote:
> Grzegorz Jakacki wrote:
> 
>> Stefan Seefeld wrote:
> 
> 
>>> I'm not bashing the OpenC++ programming model in any way. As you say,
>>> it has served its useres very well over the years. However, when it
>>> was first developed, alternative techniques such as hybrid programming
>>> (http://www.boost-consulting.com/writing/bpl.html) weren't really 
>>> available.
>>> Now they are, and so I'm comparing these new possibilities.
>>
>>
>>
>> You do, by suggesting that existing model has disadvantages, which it 
>> in fact does not have. It is orthogonal to the question if pythonic 
>> API would be useful (it would, IMHO).
> 
> 
> Suggesting that I bash OpenC++ and that I spread 'FUD' just because I
> ask you to take a fresh look at metaprogramming is quite a
> misrepresentation.
> My question was not about whether or not to export an API to python,
> but to compare advantages and disadvantates of using language extensions
> for metaprogramming as opposed to scripting.
> 
> I appreciate you replying you don't find the existing model having
> disadvantages. What are the reasons for your thinking ? If you think
> the scripting approach I suggest is a good alternative, how would you
> compare both ?
> 
>> You took OpenC++ code and you put it in your project. You solicited 
>> substantial time and work from OpenC++ community (e.g. I wrote 
>> architecture docs per your request). This helped you to improve 
>> Synopsis's branch of OpenC++.
> 
> 
> Indeed it did. Any good discussion will trigger new development. I'm
> quite sorry that you see this very one-sided, though, as I would hope
> that you, too, would have enjoyed the exchange. The may be single most
> valuable aspect in Free Software is that by construction you can't prohibit
> anybody from using the work of others. It thus stresses the collective
> aspect of knowledge. I totally fail to understand how you experienced
> this as if I had taken away something from you ('substantial time and 
> work').
> 
>> Your license lock resulted in a situation, where the improved version 
>> of Synopsis's OpenC++ is more attractive than the original version, 
>> yet the improvements cannot be backported. This attracts developers 
>> away from OpenC++ to Synopsis.
> 
> 
> If that is indeed the case it may be a good time to rethink your licensing
> terms then.
> 
>> Also for some reason you refused to build both Synopsis and 
>> OpenC++framework on an independent common core library, being an 
>> intersection of Synopsis and OpenC++. In particular you were opposing 
>> the idea of making such a library an independently buildable package. 
>> Clearly such core library would get most attention, since it would be 
>> in a focal point of both Synopsis and OpenC++ developers. Perhaps you 
>> were afraid that as a standalone package it will distract manpower 
>> from Synopsis proper. 
> 
> 
> Certainly not, as I had to do the work anyways. It was, as you may 
> remember,
> a matter of project management.
> 
> I'm still trying to get this discussion back to where I started it, i.e.
> to compare various approaches to metaprogramming. The fact that I draw
> my current design ideas from my experience with synopsis does by no means
> imply that this is a 'synopsis-centric' discussion.
> 
> In particular, if you find the ideas I bring forward valuable, I'd 
> certainly
> be more than happy to help apply them to OpenC++.
> 
> What do other OpenC++ developers and users think ?
> 
> Regards,
>         Stefan
> 
> 
> -------------------------------------------------------
> SF email is sponsored by - The IT Product Guide
> Read honest & candid reviews on hundreds of IT Products from real users.
> Discover which products truly live up to the hype. Start reading now.
> http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click
> _______________________________________________
> Opencxx-users mailing list
> Ope...@li...
> https://lists.sourceforge.net/lists/listinfo/opencxx-users

Re: [Opencxx-users] scriptable code generation

[Stefan S. <se...@sy...>]

Brian Kahne wrote:

 >  From my point of view, the scripting extensions you're suggesting sound
 > interesting, but I'm a little concerned with performance.  After all, a
 > seemingly small C++ file that includes a few STL headers quickly grows
 > to be tens of thousands of lines long.

Right. Well, there is more than one answer to this. It is true that walking
over a huge parse tree calling back and forth between C++ and python on each
node may be costly. I haven't done any profiling to measure / compare the
performance.
An alternative approach not requiring a walker implemented entirely in python
would be to code a new walker in C++ that is configured flexibly (again, using
some form of scripting). What I have in mind here is some kind of pattern
matcher akin to XPath/DOM, so you can express what nodes in the parse tree
to apply your callback on. Doing this 'pattern matching' in C++ may be more
efficient. This pattern matching is unrelated to the actual callbacks being
invoked, so the callbacks could still be written in python (say).

 > One other question I had was whether you're using SWIG for generating
 > the interface.  Python isn't my favorite language, but if you're using
 > SWIG then I could conceivably use Bigloo or Ocaml as my main
 > implementation language while using occ as the parser.

That's an interesting question. I do use C++ and python a lot, and the
prototype I have right now uses boost.python, as it makes the reflection
of C++ into python a very straight forward and easy exercise.

However, as the parser can certainly inspect its own code, it could also
be used to generate its own language bindings, whether that's python,
ocaml, or any other language compatible with a C++ backend.

Grzegorz, have you tried to run OpenC++ over its own code ? As OpenC++
doesn't make much use of templates it should even work as it is
now, shouldn't it ?

Regards,
		Stefan