Re: [pclasses-devel] adding zlib/bzip streams support

[Christian P. <cp...@se...>]

Am Samstag 01 Januar 2005 15:25 schrieb stephan beal:
> Hi, Christian!

Happy new year !

> i'm looking over the IOD/IOS code, and i would like to port in 2 LGPL
> classes a couple other guys wrote (don't remember their names) which
> provide transparent gzip/bzip2 support.  Would you mind if i add an
> optional dependency on zlib/bzip2 to port these classes in?

transparent compression support sounds good ...

> i'm not yet certain how to fit them best into the architecture. They are
> implemented as i/stream[buffer] implementations and only support files
> (not, zB, stringstream). There is no FileStream class any longer - only
> a File device which gets passed to an IOStream. That means i can't
> extend/modify FileStream. Perhaps a (FileStream : public IOStream)
> convenience type would be a solution? We could then add member funcs to
> it like:

I would add "I/O Filters" to IODevice. This would allow all IODevice clients 
to also use transparent compression, which would be really cool. (Remember: 
Socket is also an IODevice).

> setCompressionPolicy( None|Gzip|Bzip );
> CompressionPolicy getCompressionPolicy() const;

class IOFilter {
public:
  virtual size_t read(char*, size_t);
  virtual size_t write(const char*, size_t);
};

void IODevice::setFilter(IOFilter*);

class GZipIOFilter : public IOFilter { ... };
class BZip2IOFilter : public IOFilter { ... };

really cool!

>
> For reading files we can auto-determine the decompression technique
> based on the first few bytes (i don't yet know how to dispatch this
> type of logic via a factory, though).

this could be transparently done by IODevice::read(). But IODevice must then 
know all IOFilter's. I think we should keep out automatic filter detection 
for now.

> The problem with this approach is that FileStream would need to proxy
> all IOS-related API calls to the appropriate de/compressor stream.
> That's possible to implement, i guess, but tedious.

proxying to the IOFilter is then done internally by IODevice.

> So, i guess my question is: if you were going to add GZip file support,
> where would you put it?
>
> PS: i think the IOS/IOD architecture will easily support the addition of
> Zip archive support, since we can treat a zip as a virtual filesystem.
> With s11n support on top of that we've got some really flexible object
> persistency options. :)

Keep in mind, IOS/IOD architecture is completly unrelated to the 
"Network-Protocol-Transparent I/O" arch (IOManager, IOHandler, ...). 
But you're right, zip, tar, what-ever-archiver support should be added also.


Shall i add IOFilter's so you can start with the GZip/BZip2 Filters?