pygccxml-development

[pygccxml-development] declarations string, stability

[Gustavo C. <gjc...@gm...>]

  Hi Roman and all,

  Suppose I wanted to:

     1- Scan header files once;
     2- Store the definitions found in my own XML format, including for each
definition a string that uniquely identifies it;
     3- Later, I run the same program that scans a new set of definitions
     4- At this point the program should be able to detect: a) newly added
definitions, b) changed definitions (only changed parameter names), c)
removed definitions;

For step 4 I need to get a definitions identifier that is stable across
current and future (py)gccxml versions.  Is decl_string the identifier I
want?

Thanks in advance.

-- 
Gustavo J. A. M. Carneiro
INESC Porto, Telecommunications and Multimedia Unit
"The universe is always one step beyond logic." -- Frank Herbert

Re: [pygccxml-development] declarations string, stability

[Roman Y. <rom...@gm...>]

On Dec 8, 2007 3:02 PM, Gustavo Carneiro <gjc...@gm...> wrote:
>   Hi Roman and all,
>
>   Suppose I wanted to:
>
>      1- Scan header files once;
>      2- Store the definitions found in my own XML format, including for each
> definition a string that uniquely identifies it;
>       3- Later, I run the same program that scans a new set of definitions
>      4- At this point the program should be able to detect: a) newly added
> definitions, b) changed definitions (only changed parameter names), c)
> removed definitions;
>
> For step 4 I need to get a definitions identifier that is stable across
> current and future (py)gccxml versions.  Is decl_string the identifier I
> want?

No, what you want is to use declarations.full_name function.

P.S. Be careful with unnamed declarations.

-- 
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/

Re: [pygccxml-development] declarations string, stability

[Gustavo C. <gjc...@gm...>]

On 08/12/2007, Roman Yakovenko <rom...@gm...> wrote:
>
> On Dec 8, 2007 3:02 PM, Gustavo Carneiro <gjc...@gm...> wrote:
> >   Hi Roman and all,
> >
> >   Suppose I wanted to:
> >
> >      1- Scan header files once;
> >      2- Store the definitions found in my own XML format, including for
> each
> > definition a string that uniquely identifies it;
> >       3- Later, I run the same program that scans a new set of
> definitions
> >      4- At this point the program should be able to detect: a) newly
> added
> > definitions, b) changed definitions (only changed parameter names), c)
> > removed definitions;
> >
> > For step 4 I need to get a definitions identifier that is stable across
> > current and future (py)gccxml versions.  Is decl_string the identifier I
> > want?
>
> No, what you want is to use declarations.full_name function.


Are you sure?  It does not seem to include the full signature. Example, for
the methods:

    int get_int (const char *from_string);
    int get_int (double from_float);

full_name prints:

'::SomeObject::get_int'
'::SomeObject::get_int'

P.S. Be careful with unnamed declarations.


I am not sure when I get unnamed declarations.  But in any case, one step at
a time... :)

-- 
Gustavo J. A. M. Carneiro
INESC Porto, Telecommunications and Multimedia Unit
"The universe is always one step beyond logic." -- Frank Herbert

Re: [pygccxml-development] declarations string, stability

[Ben S. <bsc...@lu...>]

=20



=09
	No, what you want is to use declarations.full_name function.
=09
	Are you sure?  It does not seem to include the full signature.
Example, for the methods:=20
=09
	    int get_int (const char *from_string);
	    int get_int (double from_float);
=09
	full_name prints:
=09
	'::SomeObject::get_int'
	'::SomeObject::get_int'
=09
	=20

You could use str(declaration). That'll print out the full signature of
the member functions.
Otherwise, you'll probably have to write your own formatting function.
=20
Cheers
Ben

Re: [pygccxml-development] declarations string, stability

[Gustavo C. <gjc...@gm...>]

On 08/12/2007, Ben Schleimer <bsc...@lu...> wrote:
>
>
>
>
> No, what you want is to use declarations.full_name function.
>
> Are you sure?  It does not seem to include the full signature. Example,
> for the methods:
>
>     int get_int (const char *from_string);
>     int get_int (double from_float);
>
> full_name prints:
>
> '::SomeObject::get_int'
> '::SomeObject::get_int'
>
>
> You could use str(declaration). That'll print out the full signature of
> the member functions.
>

str(declaration) is not good because:
  1. it prints parameter names (which clearly are not part of the
signature);
  2. it adds a redundant "[member function]" description which I do not need
and am afraid might change one day;
  3. I don't think there are any guarantees the string format won't change
in the future.

Otherwise, you'll probably have to write your own formatting function.
>

I guess I could do it.  I was just hoping pygccxml had it already, to save
me some work... :P

-- 
Gustavo J. A. M. Carneiro
INESC Porto, Telecommunications and Multimedia Unit
"The universe is always one step beyond logic." -- Frank Herbert

Re: [pygccxml-development] declarations string, stability

[Roman Y. <rom...@gm...>]

On Dec 8, 2007 10:49 PM, Gustavo Carneiro <gjc...@gm...> wrote:
> On 08/12/2007, Ben Schleimer <bsc...@lu...> wrote:
> >
> > No, what you want is to use declarations.full_name function.
> >
> > Are you sure?  It does not seem to include the full signature. Example,
> for the methods:
> >
> >     int get_int (const char *from_string);
> >     int get_int (double from_float);
> >
> > full_name prints:
> >
> > '::SomeObject::get_int'
> > '::SomeObject::get_int'

My mistake, I forgot about arguments

> >
> > You could use str(declaration). That'll print out the full signature of
> the member functions.
>
> str(declaration) is not good because:
>   1. it prints parameter names (which clearly are not part of the
> signature);
>   2. it adds a redundant "[member function]" description which I do not need
> and am afraid might change one day;

You are right

>   3. I don't think there are any guarantees the string format won't change
> in the future.

You are right

> >
> >
> > Otherwise, you'll probably have to write your own formatting function.
>
> I guess I could do it.  I was just hoping pygccxml had it already, to save
> me some work... :P

Not in this case :-(. I think you will have to introduce your own format.

I guess you should consider:
* function default values
* function exceptions
* static or not
* virtuality
* access type
* who knows what else

May be you should take a look on "cache" functionality (
http://language-binding.net/pygccxml/apidocs/pygccxml.parser.declarations_cache-module.html
). The only problem that it has is that I don't promise to be backward
compatible. This is a hard to meet requirement, so I just ask from
users to clean the cache every time they upgrade pygccxml.

I also found out that this cache is pretty useless, because it doesn't
give the desired performance boost.

I don't know what is the size of your project,  but may be instead of
creating some format that will describe the headers content, you will
save the whole XML and next time you will compare between 2 XML files?
Of course this will not work if we talk about huge projects.

Another idea is to take a look on Py++. Recently I implemented
functionality that allows user  to manage dependencies in multi-module
project with a single line of code. Take a look on next file:
http://pygccxml.svn.sourceforge.net/viewvc/pygccxml/pyplusplus_dev/pyplusplus/utils/__init__.py?revision=1173&view=markup
The class, you should look at, is exposed_decls_db_t. As in other
cases I will not promise backward compatibility.

HTH

-- 
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/

Re: [pygccxml-development] declarations string, stability

[Ben S. <bsc...@lu...>]

Hi,

	=20

		=20
		You could use str(declaration). That'll print out the
full signature of the member functions.


	str(declaration) is not good because:
	  1. it prints parameter names (which clearly are not part of
the signature);
	  2. it adds a redundant "[member function]" description which I
do not need and am afraid might change one day;=20
	  3. I don't think there are any guarantees the string format
won't change in the future.
=09


		Otherwise, you'll probably have to write your own
formatting function.


	I guess I could do it.  I was just hoping pygccxml had it
already, to save me some work... :P=20
	=20

 Nope, I guess everyone wants their own version of __str__() ;).
=20
As a sidenote (for Roman), maybe it would be good to have hooks for a
visiter pattern in declaration_t.
Then it would obvious that if the user programmer wants to traverse the
decl tree, there's a standard interface for doing so instead of the user
having to manually traverse himself.
Then the scopedef functionality wouldn't be needed. Instead, we could
have a standard set of visitors which collect different types of decls
and return them as a list or mdecl_wrapper_t like scopedef.*s() does
now...
I know this would be useful because I had to search for decls based on
their attributes which wasn't a param in scopedef.member_functions(..)
so I had to manually test the attributes anyway.
=20
Cheers
Ben
=20

Re: [pygccxml-development] declarations string, stability

[Roman Y. <rom...@gm...>]

On Dec 8, 2007 11:07 PM, Ben Schleimer <bsc...@lu...> wrote:
> As a sidenote (for Roman), maybe it would be good to have hooks for a
> visiter pattern in declaration_t.
> Then it would obvious that if the user programmer wants to traverse the decl
> tree, there's a standard interface for doing so instead of the user having
> to manually traverse himself.

I almost agree with you. I could improve the interface, meanwhile you
can use decl_visitor_t (
http://language-binding.net/pygccxml/apidocs/pygccxml.declarations.decl_visitor.decl_visitor_t-class.html
)
class and apply_visitor function (
http://language-binding.net/pygccxml/apidocs/pygccxml.declarations.algorithm-module.html#apply_visitor
)

> Then the scopedef functionality wouldn't be needed. Instead, we could have a
> standard set of visitors which collect different types of decls and return
> them as a list or mdecl_wrapper_t like scopedef.*s() does now...

Not completely true. Behind the scene, scopedef_t class introduces
very powerful 2-level hash optimization

declaration type : declaration name : declaration ( this scope only )
and
declaration type : declaration name : declaration ( all declarations
from this and internal scopes recursively )

On huge projects you feel the difference. I don't have the numbers,
but you can play with scopedef_t.clear_optimizer and
scopedef_t.init_optimizer functions.

> I know this would be useful because I had to search for decls based on their
> attributes which wasn't a param in scopedef.member_functions(..) so I had to
> manually test the attributes anyway.

So why do you need visitor? You already know that you need member
functions, the only thing left is to specialized the query. This is
easily done by passing a callable to the member_functions function:

ns.mem_funs( lambda x: len( x.arguments ) > 19 )

By the way, in this case scopedef_t understands that you are looking
for member function, so it cuts the number of comparisons to member
functions only. It also selects the shorter list based on "recursive"
argument value.

P.S. What functionality is missing from the member_functions interface?

-- 
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/

Re: [pygccxml-development] declarations string, stability

[Ben S. <bsc...@lu...>]

>=20
> I almost agree with you. I could improve the interface,=20
> meanwhile you can use decl_visitor_t (=20
> http://language-binding.net/pygccxml/apidocs/pygccxml.declarat
> ions.decl_visitor.decl_visitor_t-class.html
> )
> class and apply_visitor function (
> http://language-binding.net/pygccxml/apidocs/pygccxml.declarat
> ions.algorithm-module.html#apply_visitor
> )

Oh, I didn't know about that functionality. Well, I'll definitely try
using it next time...

>=20
> > I know this would be useful because I had to search for=20
> decls based on=20
> > their attributes which wasn't a param in=20
> scopedef.member_functions(..)=20
> > so I had to manually test the attributes anyway.
>=20
> So why do you need visitor? You already know that you need=20
> member functions, the only thing left is to specialized the=20
> query. This is easily done by passing a callable to the=20
> member_functions function:
>=20
> ns.mem_funs( lambda x: len( x.arguments ) > 19 )
>=20

Argg, again, I didn't know about it because its not mentioned anywhere
in the documentation.
Maybe again a paragraph about decl_visitor_t and callable would make
them more clear to the
End users.


>=20
> P.S. What functionality is missing from the member_functions=20
> interface?

The ability to search for decls based on their attributes.
This really should be a visitor or callable function because it's
entirely string matching and project specific.

Cheers
Ben

Re: [pygccxml-development] declarations string, stability

[Roman Y. <rom...@gm...>]

On Dec 10, 2007 8:09 PM, Ben Schleimer <bsc...@lu...> wrote:
>
> >
> > I almost agree with you. I could improve the interface,
> > meanwhile you can use decl_visitor_t (
> > http://language-binding.net/pygccxml/apidocs/pygccxml.declarat
> > ions.decl_visitor.decl_visitor_t-class.html
> > )
> > class and apply_visitor function (
> > http://language-binding.net/pygccxml/apidocs/pygccxml.declarat
> > ions.algorithm-module.html#apply_visitor
> > )
>
> Oh, I didn't know about that functionality. Well, I'll definitely try
> using it next time...

Where would you look for such thing?

> >
> > > I know this would be useful because I had to search for
> > decls based on
> > > their attributes which wasn't a param in
> > scopedef.member_functions(..)
> > > so I had to manually test the attributes anyway.
> >
> > So why do you need visitor? You already know that you need
> > member functions, the only thing left is to specialized the
> > query. This is easily done by passing a callable to the
> > member_functions function:
> >
> > ns.mem_funs( lambda x: len( x.arguments ) > 19 )
> >
>
> Argg, again, I didn't know about it because its not mentioned anywhere
> in the documentation.
> Maybe again a paragraph about decl_visitor_t and callable would make
> them more clear to the
> End users.

It is described here:
http://language-binding.net/pygccxml/query_interface.html#usage-examples

-- 
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/