[pygccxml-development] Another way to speed up the generation process...

[Matthias B. <ba...@ir...>]

Hi,

I just thought I'd share a little code snippet that can be useful to 
speed up the bindings generation process. The below function simply 
removes all declaration nodes that do not originate from the specified 
directories (i.e. it removes all the system chunk that is not going to 
be wrapped anyway). So you should call that function right after the 
parsing step.
I noticed that these extra nodes can eat up a lot of time as I was 
porting my script over to OSX where a lot more system headers are read.
On Linux the running time was reduced from about 4:40 minutes to around 
3:20 minutes and on OSX it went down from 7:20 minutes to 4:10 minutes.

On OSX the top-level namespace contained more than 20000 nodes whereas 
on Linux it was only about 4700 nodes. What I still have to investigate 
is how the cache can be modified so that it stores the data *after* 
pruning. It's a bit odd that reading the cache on OSX (14MB) takes more 
than a minute whereas on Linux (6MB) it only takes 7 seconds. Obviously, 
the algorithms used in the pickle module are far from being O(n).... ;)

- Matthias -


def pruneByHeader(rootdecl, includedirs):
     """Remove all nodes that don't originate from the specified 
directories.

     rootdecl must be the root node of the declaration tree (as a pygccxml
     namespace_t object). All nodes right beneath the root who do not
     originate from any directory in the list includedirs are removed
     from the tree.
     This operation is just meant to speed up subsequent queries
     by removing anything that will not get wrapped anyway.
     """
     includedirs = map(lambda x: os.path.abspath(x), includedirs)
     num_decls = len(rootdecl.declarations)
     num_removed = 0
     for decl in rootdecl.declarations[:]:
         header = getattr(decl.location, "file_name", None)
         # Check if the decl should be kept. If it should, then continue
         # the loop...
         if header!=None:
             header = os.path.abspath(header)
             keep_it = False
             for dir in includedirs:
                 if header.startswith(dir):
                     keep_it = True
                     break
             if keep_it:
                 continue
         # Remove the declaration
         decl.parent.remove_declaration(decl)
         num_removed += 1

     print "%d out of %d top-level declarations removed (%d 
kept)."%(num_removed, num_decls, num_decls-num_removed)