From: Nicolas C. <war...@fr...> - 2004-04-09 11:44:46
|
(getting back to some old thread) > > - base64 encode/decode > > - abstract high level I/O with support for C basic types ( read_i16 , > > write_f16 ..... ) > > - zlib deflate/inflate written in pure OCaml. > > One of the things I've been nooddling around with is a radicaly new I/O > arhitecture. The basic idea is to make i/o more like java and less like > C. The core idea is that there would be four classes of objects: > > 1) sinks and sources. These would be the fundamental sources/destinations > of the data- files, sockets, and strings would all be sinks and/or > sources. Most likely these will be built on top of the old-style Ocaml > I/O routines. > > 2) filters. These look like a sink or source, but wrap either a > sink/source or another filter. Examples of filters would > compression/decompression, encryption/decryption, mime or uu > encode/decode, eoln conversion, etc. Also, a filter might just collect > statistics on the stuff as it goes by, not changing the data- so line > counters, word counters, and check summers would be filters. Theses two are now reality in the IO module. For example, I managed to wrap CamlZip module on an IO and - without changing a line of my code - I can now read/write over normal or compressed files. Here's the code that's converting a standard IO into a compressed/uncompressed one. I think it's worth including into the ExtLib but the problem is it rely on another module (CamlZip, by Xavier Leroy) which itself contains C stubs using the ZLib. I think we need an additional folder into the ExtLib called "tools" that will be useful parts of code - yet not installed because of external dependencies. What do you think about the idea ? Here's the code - the inflate method is still broken since it requires to know the available data size, and is not "inflating on demand". Nicolas Cannasse ---- let inflate i = let available = ref (match IO.available i with None -> assert false | Some size -> size) in let buf = Buffer.create 0 in let refill str = let len = String.length str in if !available >= len then begin let data = IO.nread i len in String.blit data 0 str 0 len; available := !available - len; len end else begin let size = !available in let data = IO.nread i size in String.blit data 0 str 0 size; available := 0; size end in let flush str n = Buffer.add_substring buf str 0 n in Zlib.uncompress ~header:true refill flush; IO.input_string (Buffer.contents buf) let deflate o = let buf = Buffer.create 0 in let flush() = let data = Buffer.contents buf in let pos = ref 0 in let available = ref (String.length data) in let refill str = let len = String.length str in if !available >= len then begin String.blit data !pos str 0 len; available := !available - len; pos := !pos + len; len end else begin let size = !available in String.blit data !pos str 0 size; available := 0; pos := !pos + size; size end in let flush str n = if n = String.length str then IO.nwrite o str else IO.nwrite o (String.sub str 0 n) in Zlib.compress ~level:9 ~header:true refill flush; IO.flush o; Buffer.reset buf; in IO.create_out (Buffer.add_char buf) (Buffer.add_string buf) (fun () -> Buffer.length buf) (fun () -> flush(); IO.flush o) (fun () -> flush(); IO.close_out o) |