current gzip/inflate code in pavuk does not allow huge files
Brought to you by:
stoecker
Menu
▾
▴
#33 current gzip/inflate code in pavuk does not allow huge files
Can't download a transfer-compressed HD DVD ISO, because the current gzip/inflate code will load the complete compressed AND decompressed file in memory (in-mem decompressing!).
This should and CAN be fixed for non-parsable files (== anything that's not HTML, CSS or JavaScript).
pavuk internals do not cope well with overlarge files which do not fit in memory: the code will keep the complete file content in memory for parsing/filtering tasks as well.
Hence, there are two ways about this: (1) either give pavuk a way to 'stream' overlarge files to disc instead of keeping them in memory, OR (2) deciding this is not a bug but a feature == list this behaviour as 'known and by design'.
I'm two ways about it. Might have big impact in certain locations and how many people mirror complete DVD images/distros + web pages?