Re: [TCLCORE] zlib support in 8.6 and the SPDY protocol

[<L....@su...>]

Lars,

I believe your thinking is right. For some time it has struck me that gzip -9 isn't that efficient
- it's just a larger window to pick the initial dictionary from, but still stream based, with the
limits of streaming.

Preseeded compression dictionaries crop up a lot in DVB standards etc. (thinking of
ETSI TS 102 539 V1.3.1 (2010-04) Technical Specification Digital Video Broadcasting (DVB);
Carriage of Broadband Content Guide (BCG) information over Internet Protocol (IP) and
the like.)

A compressor aiming for the best compression would attempt to compress an entire file
multiple times, just to build a better starting dictionary, and the final compressed
file would be smaller as a result, but still uncompressible by gunzip/zlib streaming.

So, if your hack works, can you build a better compressor tool?

Lloyd Wood
http://sat-net.com/L.Wood/


________________________________________
From: Lars Hellström [Lar...@re...]
Sent: 21 March 2012 17:16
To: tcl...@li...
Subject: Re: [TCLCORE] zlib support in 8.6 and the SPDY protocol

Michael Schlenker skrev 2012-03-20 21.07:
> Hi all,
>
> i just studied the Google proposal for the SPDY protocol out of curiosity, which is a multiplexed stream inside a TLS secured connection.

Not just for TLS, it seems (although secure connections would obviously
benefit from this kind of multiplexing mechanism).

> It is being implemented in a lot of clients/web servers recently and so i was curious, as it would probably fit Tcl channels really well.

There are indeed things in the specification that makes one think about
reflected channels...

> But while reading the specification at http://tools.ietf.org/html/draft-mbelshe-httpbis-spdy-00 i found a part that seems to be not doable
> with the zlib support in Tcl.
>
> The critical part is listed in section 2.6.10.1, where it talks about
> header compression with a pre-seeded zlib dictionary, which probably should
> be done with zlib deflateSetDictionary() function (see
> http://zlib.net/manual.html#Advanced) which is not exposed by Tcls zlib
> support currently.

It occurs to me that (given the sliding window principle underlying zlib) a
work-around could (in the stream interface) be to compress the pre-seed
data, flush, and then compress what you actually want to send; something like

proc SPDY_compress {data} {
    variable preseed ; # Constant
    set S [zlib stream deflate]
    set res \x78\xBB
    $S add -flush $preseed  ; # Compress and throw away the result
    append res [binary format I [$S checksum]]
    $S put -finalize $data
    append res [$S get] [binary format I [$S checksum]]
    $S close
    return $res
}

However, even if this works (I haven't tested it), it runs the risk of being
classified as a horrible hack; essentially it performs block-level surgery
on the raw compressed stream (throw first block away, keep the rest), and
then manually constructs the zlib wrapper required by the SPDY spec.

> Should this function be exposed in 8.6?

In view of the above "alternative", the answer is probably yes.

Lars Hellström


------------------------------------------------------------------------------
This SF email is sponsosred by:
Try Windows Azure free for 90 days Click Here
http://p.sf.net/sfu/sfd2d-msazure
_______________________________________________
Tcl-Core mailing list
Tcl...@li...
https://lists.sourceforge.net/lists/listinfo/tcl-core