Re: [pygccxml-development] Comments on module_builder cache functionality

[Allen B. <al...@vr...>]

Roman Yakovenko wrote:

> On 8/29/06, Allen Bierbaum <al...@vr...> wrote:
>
>> So now I have to figure out how to get this to work with module builder
>> cache and there is a problem.  Namely, in module builder the call to
>> parse_declarations does not have access to the list of files that gccxml
>> had to include.  The only place where these files seem to be available
>> is in source_reader.py on line 237.  It does not appear that this
>> information remains in the pygccxml tree but I could be wrong.  Also,
>> the files list is only used to update the decl cache so it is not
>> available later during execution.
>>
>> Does anybody know a way to either a) get the computed file list from
>> source_reader back into the module builder
>
>
> You don't need this
>
>>  or b) get a list of files
>> that were included from a pygccxml decl tree?
>
>
> pygccxml.declarations.declarations_files
> But you also don't need this functionality

How would I call update then in the interface you propose below?  I need 
the list of include files if that part of the key check is going to work.

>
> My idea was next:
>
> class mbcache_t( pygccxml.parser.base_cache_t ):
>    def __init__( self, *args ):
>         pygccxml.parser.base_cache_t.__init__( self )
>         test whether there is a need to load from cache on disk
>         self.decls_cache = pygccxml.parser.file_cache_t( file_name )
>
>   def update( self, .... ):
>         self.decls_cache.update( self,  .... )
>
>   def flush( self, ... )
>        self.decls_cache.flush()
>        write to disk files passed to module_builder_t
>
> Thus, mbcache_t only needs to warry about files list that was passed to
> module_builder_t
>
> I think this is pretty simple idea, that will take few hours to 
> implement.

Maybe I am missing something, but what I don't see is how the interface 
provides the capabilities I need.  It would not be difficult for me to 
write a little helper class that wrapped the cache file and loading if 
that is the issue, but I don't see a good way to reuse the file_cache_t 
code or really a good reason to do so.  What I need is different and 
smaller.

I need something like:

class mbcache_t(object):
   def __init__(self, cache_file)
      self.cache_file = file(cache_file)

   def getCachedTree(self, sourceFiles, configuration):
      # Read and test prefix signature against sourceFiles and configuration
      # If no match, early exit
      # Read decl tree
      # Read signature and list of files used to create cache
      # If one of those files has changed, then return None
      # Else return tree

   def updateCachedTree(self, sourceFiles, configuration, includedFiles, 
declTree):
      # Open cache file for writing
      # Compute prefix sig and dump it
      # Dump the decl tree
      # Compute and dump signature for and includeFile list
      # Save and close the file

I can easily create a class like this and use it if you think that will 
make the code more modular.  But I hope you can see this is quite a bit 
different from the cache map that we use in pygccxml.  There is no map, 
there is no need for dirty/used flag, it works for multiple files, it is 
smart enough to only load if needed.  I would reuse the signature code, 
but the rest would be unique.

But, back to the question at hand.  How would I get the "includedFiles" 
for the updateCacheTree call?  Note that this call would need to happen 
builder.__init__ where all I have access to is the self.__global_ns decl 
tree that I am caching.  The gccxml scanner that is used to pass the 
parameters to the file_cache_t.update() method is not available at this 
level in the code.

Any ideas?

-Allen

>
> What do you think?
>