Re: [pygccxml-development] Py++ Feature request - cache file compression

[<tho...@fr...>]

>On Fri, Dec 19, 2008 at 7:15 AM, Thomas Paviot <tho...@fr...> wrote:
>> Hello,
>>
>> I use the Py++ 'cache' feature in order to speed up module_build_t
>> creation. Unfortunately, generated xml files are huge. For instance, in
>> the project I'm currently working on, I have about 200 xml files with an
>> approximative size of 2Mb for each one. As a result, total size of
>> cached gccxml files are about 800Mb.
>
>400 Mb, right?

Well, if 2*2 still equals 4, then yes: 400Mb! (sorry for the mistake).

>>
>> Wouldn't it be possible to zip/unzip these files before they are used by
>> pygccxml? Speed wouldn't really be affected, and the size improvement
>> would be about 80% (new size of my folder would then be only 160Mb). The
>> cache filename could be passed to module_builder_t either with .xml
>> extension (current behaviour) or with .zip extension (xml file would be
>> stored as zip archives).
>
>I think, if you add such functionality the cache advantage will almost gone.

I made another mistake in my message: it seems that cached files are not xml formatted(that's what I thought but I realized it was wrong after I opened these files in my text editor).

Regarding time advantage, zipfile python module provides a very fast compression/extraction algorithm. I used it in a other project and was really amazed about the performances. But you're right, it have to be measured.

>You can implement something outside of Py++/pygccxml
>    - "unzip" before passing to py++
>    - zip after code generation was done.
>
>Thus you will get the time measures. If they are good I will be glad
>to work with you to integrate this functionality to Py++/pygccxml.

It's a good way to check the idea. I send you back the results as soon as I can.

>>I do not have enough free time now to implement
>> this feature on my own, but I can provide a patch in the next few weeks.
>
>It could be nice if you describe in few words how you are going to do
>this, so we could close the interface only once :-).

I don't know yet about the 'how' and just expressed a 'why'.

>Thanks

Regards,

Thomas