From: Bardur A. <oca...@sc...> - 2004-12-16 13:51:14
|
On Thu, Dec 16, 2004 at 01:38:06PM +0000, Richard Jones wrote: > On Thu, Dec 16, 2004 at 01:48:04PM +0100, Bardur Arantsson wrote: > > > From this you can have trimming functions: > > > > > > let triml ?(test = isspace) str = ... > > > let trimr ?(test = isspace) str = ... > > > let trim ?test str = ... > > > > ... or lstrip/rstrip/strip as they would be known > > according to the current naming conventions. :) > > > > The fact that trim takes a "test" closure instead of > > string with characters should IMHO be regarded as a > > feature and the 'old' strip should just be removed... Of > > course it may mean that the compiler cannot inline the > > "test" calls, but that is, again IMHO, a compiler issue. > > Or have both ... > It's not that I'm dead set against having both, but it just seems like interface bloat... but the functionality is certainly needed. :) -- Bardur Arantsson <ba...@im...> <ba...@sc...> - Homer no function beer well without. Homer Simpson | The Simpsons |
From: Richard J. <ri...@an...> - 2004-12-16 16:38:48
|
Make the interface as big as it needs to be. Rich. --=20 Richard Jones. http://www.annexia.org/ http://www.j-london.com/ >>> http://www.team-notepad.com/ - collaboration tools for teams <<< Merjis Ltd. http://www.merjis.com/ - improving website return on investment Write Apache modules in OCaml - http://www.merjis.com/developers/mod_caml |
From: Richard J. <ri...@an...> - 2004-12-16 13:46:18
|
On Thu, Dec 16, 2004 at 12:03:24PM +0000, Richard Jones wrote: > let rec uniq ?(cmp =3D Pervasives.compare) =3D function > [] -> [] > | [x] -> [x] > | x :: y :: xs when compare x y =3D 0 -> > uniq (x :: xs) > | x :: y :: xs -> > x :: uniq (y :: xs) This function is somewhat broken w.r.t its handling of the optional ?cmp argument, but I hope you understood what I meant to do ... Rich. --=20 Richard Jones. http://www.annexia.org/ http://www.j-london.com/ >>> http://www.team-notepad.com/ - collaboration tools for teams <<< Merjis Ltd. http://www.merjis.com/ - improving website return on investment Write Apache modules in OCaml - http://www.merjis.com/developers/mod_caml |
From: Bardur A. <oca...@sc...> - 2004-12-18 00:30:29
|
On Sat, Dec 18, 2004 at 11:16:53AM +1100, skaller wrote: > On Sat, 2004-12-18 at 00:40, Bardur Arantsson wrote: > > > I think we may be discussing different things... I'm only > > talking about something which ExtLib developers can use > > exclusively for testing ExtLib itself, not about making a > > test harness available to ExtLib users. > > No, we're talking about the same thing. > > > In general, yes, but it doesn't mean that we can't agree > > on an *internal* method for testing ExtLib itself. > > Well, can you could write a specification? > > Assume a directory containing tests, people can dump > tests in it. The tests need names -- what names? > it must be possible to specify expected result, > the harness must report which test(s) (if any) fail. > > You can use an 'assert' test (if the code runs at > all it passes). The problem with that is that > library function are hard to test like that.. > > How do you test 'string.split'? This is not so easy! > Try: > > assert (x = combine "/" (split x "/")) > > Which suggests making the split char vary. Uh, and...? If you want to try different splitter characters, just do let l = [ ("/", "a/b", ["a";"b"]); ("/", "a/b/c", ["a";"b"]); ("#", "a#b", ...) ] in List.map (fun sep str exp_values -> assert (exp_values = split sep str)); List.map (fun sep str exp_values -> assert (str = combine sep exp_values)) in your test code. This also buys separate tests for split and combine (and thus tests more than whether the two are "inverses" of each other). If you want more flexibility you can always generate the l list as a cartesian product of separators and field values/strings. I think you may be overdesigning/overthinking this... > How do you test ExtSet? You need stuff to put in it, > where does it come from? Same idea as above, you just hardcode the test data into your test_*.ml files... -- Bardur Arantsson <ba...@im...> <ba...@sc...> A morning without coffee is like something without something else. |
From: skaller <sk...@us...> - 2004-12-18 01:35:39
|
On Sat, 2004-12-18 at 11:30, Bardur Arantsson wrote: > I think you may be overdesigning/overthinking this... Yes I am. Devils Advocately... :) -- John Skaller, mailto:sk...@us... voice: 061-2-9660-0850, snail: PO BOX 401 Glebe NSW 2037 Australia Checkout the Felix programming language http://felix.sf.net |
From: Brian H. <bh...@sp...> - 2004-03-19 18:35:04
|
On Fri, 19 Mar 2004, Nicolas Cannasse wrote: > - base64 encode/decode > - abstract high level I/O with support for C basic types ( read_i16 , > write_f16 ..... ) > - zlib deflate/inflate written in pure OCaml. One of the things I've been nooddling around with is a radicaly new I/O arhitecture. The basic idea is to make i/o more like java and less like C. The core idea is that there would be four classes of objects: 1) sinks and sources. These would be the fundamental sources/destinations of the data- files, sockets, and strings would all be sinks and/or sources. Most likely these will be built on top of the old-style Ocaml I/O routines. 2) filters. These look like a sink or source, but wrap either a sink/source or another filter. Examples of filters would compression/decompression, encryption/decryption, mime or uu encode/decode, eoln conversion, etc. Also, a filter might just collect statistics on the stuff as it goes by, not changing the data- so line counters, word counters, and check summers would be filters. 3) formatters and parsers. Formatters take the fundamental datatypes of ocaml (int, float, etc) and convert them into strings and write the strings to a filter or source/sink. Parsers take strings and parse them back into the fundamental types of ocaml. 4) Directories- collections of one or more sources or sinks or subdirectories. This is so we can deal with "multi-file" streams, like tar or zip files. This might also be a way to generically handle file system issues. This might also be the way to deal with file forks on MacOS. The questions I haven't answered yet are: 1) Unicode or no unicode? It's not that converting to/from unicode into one of the utf- formats is so hard, it's just that now a stream of characters is a different concept than a stream of bytes, with different filters working on different types streams. And a conversion object stuck in between. All pipes would now be at least three objects deep (formatter, unicode conversion, source/sink), instead of just two. 2) What are the semantics of directories? Do we want to include file system fun? 3) How to deal with the lack of polymorphism on formatters. All of the implementations of this structure I've seen (ab)use polymorphism to implement the formatters- they have a print function which is overloaded for the specific types- print(int), print(float), etc. Often they have an operator overloaded or defined as well. I'd love to be able to have a print function with a signature filter -> 'a -> filter, but I can't think of how to do it. -- "Usenet is like a herd of performing elephants with diarrhea -- massive, difficult to redirect, awe-inspiring, entertaining, and a source of mind-boggling amounts of excrement when you least expect it." - Gene Spafford Brian |
From: Nicolas C. <war...@fr...> - 2004-03-20 09:44:41
|
> > - base64 encode/decode > > - abstract high level I/O with support for C basic types ( read_i16 , > > write_f16 ..... ) > > - zlib deflate/inflate written in pure OCaml. > > One of the things I've been nooddling around with is a radicaly new I/O > arhitecture. The basic idea is to make i/o more like java and less like > C. The core idea is that there would be four classes of objects: [...] Hi Brian, What we need is an high-level IO abstraction that will offer a wide API for most of the common usages. The IO module I posted earlier on this list was a beginning of answer to this. It didn't made his way to the CVS because I lacked time, but looks like there was no people against it. In this IO module, I made the following choices regarding polymorphism : (quoting myself) For efficiency reason each input/output have two ways of reading/writing data : for example a channel input can write char by char or directly strings. it's then a (char,string) input. An ouput takes a third parameter which is the value returned by a call to close. (end-of-quote) So we have here : * ('a,'b) input : an input to a file is a (char,string) input * ('a,'b,'c) output : an output to a buffer is a (char,string,string) output since "closing" the buffer returns the total string written into we can provide several conversion "parsers" functions such as * val in_ord : (char,string) input -> (int,int list) input * val in_chr : (int,int list) input -> (char,string) input or "filters" such as : * val in_b64 : (char,string) input -> (char,string) input * val out_zlib : (char,string,'a) output -> (char,string,'a) output Concerning the "Directories", I think its even more high level and should be handled in a different module that will be built on top of IO. Please remember that the "file" concept have no meaning in an abstract IO. Concerning Unicode, adding a filter to handle this looks okay (not by default). For the "polymorphism" of formatters, this is not really possible in the OCaml type system. One choice is to use printf/scanf on an output : that's exactly what's I'm doing in IO module. Regards, Nicolas Cannasse |
From: Richard J. <ri...@an...> - 2004-03-20 10:20:29
|
On Fri, Mar 19, 2004 at 01:37:14PM -0600, Brian Hurt wrote: > 1) Unicode or no unicode? It's not that converting to/from unicode into > one of the utf- formats is so hard, it's just that now a stream of > characters is a different concept than a stream of bytes, with different > filters working on different types streams. And a conversion object stuck > in between. All pipes would now be at least three objects deep > (formatter, unicode conversion, source/sink), instead of just two. UTF-8 support should be there, and should be driven from the LANG / LC_* / locale. It's a non-trivial amount of work to get this to work of course. Rich. -- Richard Jones. http://www.annexia.org/ http://www.j-london.com/ Merjis Ltd. http://www.merjis.com/ - improving website return on investment PTHRLIB is a library for writing small, efficient and fast servers in C. HTTP, CGI, DBI, lightweight threads: http://www.annexia.org/freeware/pthrlib/ |
From: Yamagata Y. <yor...@mb...> - 2004-03-20 23:54:11
|
From: Richard Jones <ri...@an...> Subject: Re: [Ocaml-lib-devel] Future of ExtLib Date: Sat, 20 Mar 2004 10:20:16 +0000 > UTF-8 support should be there, and should be driven from the LANG / > LC_* / locale. It's a non-trivial amount of work to get this to work > of course. I could contribute this. I have working code for Camomile. But, it requires small C code to call ISO-C functions. What do you think, Nocolas? Windows (proper) supports all ISO-C functions? and (for the future reference) how about strcol? In addition, I'm thinking of UTF-16 support (for Windows and Java) A major problem is lack of 16-bit interger array in OCaml. We could use a string and do type-cast when it is passed to C functions. This would involve detecting endianness in installtion time and providing some C-macros. Alternatively, we could use Bigarray, but Bigarray is not efficient. I'm interested in other people's opinions. -- Yamagata Yoriyuki |
From: Nicolas C. <war...@fr...> - 2004-03-21 09:57:24
|
> > UTF-8 support should be there, and should be driven from the LANG / > > LC_* / locale. It's a non-trivial amount of work to get this to work > > of course. > > I could contribute this. I have working code for Camomile. But, it > requires small C code to call ISO-C functions. What do you think, > N(i)colas? That's little troublesome. Even if we have only one C little file, the installation, compilation --- for a lot of different kind of systems --- is troublesome. Oh ! just's got a good idea. If we have to add a C part, then what we should do is a C emulator. How does it works ? - it's a C program that can resolve and call dynlinked C functions - it can also manipulate basic C data structures (convert from and to OCaml data, and also "raw" access , a little like Obj module) - it can alloc and free some C memory Then when we need to add some C parts, we just need to write some Caml code that will use Emu-C, and which will handle the system particularities... at runtime. exemple (far from being correct, just give an idea) : let strstr = EmuC.resolve "libc" "strstr" in let found = EmuC.call strstr ["my_string";"_s"] in if found = EmuC.null then None else Some (EmuC.alloc_string found) > Windows (proper) supports all ISO-C functions? and (for the future > reference) how about strcol? I don't know about that. There is strcoll ANSI compatible function. ( and wcscoll for Unicode strings ) > In addition, I'm thinking of UTF-16 support (for Windows and Java) A > major problem is lack of 16-bit interger array in OCaml. We could use > a string and do type-cast when it is passed to C functions. This > would involve detecting endianness in installtion time and providing > some C-macros. > > Alternatively, we could use Bigarray, but Bigarray is not efficient. > > I'm interested in other people's opinions. Use (future) EmuC :-) let block = EmuC.alloc size in and then EmuC.setw / EmuC.getw to access 16-bits values into block. Nicolas Cannasse |
From: Nicolas C. <war...@fr...> - 2004-04-09 11:44:46
|
(getting back to some old thread) > > - base64 encode/decode > > - abstract high level I/O with support for C basic types ( read_i16 , > > write_f16 ..... ) > > - zlib deflate/inflate written in pure OCaml. > > One of the things I've been nooddling around with is a radicaly new I/O > arhitecture. The basic idea is to make i/o more like java and less like > C. The core idea is that there would be four classes of objects: > > 1) sinks and sources. These would be the fundamental sources/destinations > of the data- files, sockets, and strings would all be sinks and/or > sources. Most likely these will be built on top of the old-style Ocaml > I/O routines. > > 2) filters. These look like a sink or source, but wrap either a > sink/source or another filter. Examples of filters would > compression/decompression, encryption/decryption, mime or uu > encode/decode, eoln conversion, etc. Also, a filter might just collect > statistics on the stuff as it goes by, not changing the data- so line > counters, word counters, and check summers would be filters. Theses two are now reality in the IO module. For example, I managed to wrap CamlZip module on an IO and - without changing a line of my code - I can now read/write over normal or compressed files. Here's the code that's converting a standard IO into a compressed/uncompressed one. I think it's worth including into the ExtLib but the problem is it rely on another module (CamlZip, by Xavier Leroy) which itself contains C stubs using the ZLib. I think we need an additional folder into the ExtLib called "tools" that will be useful parts of code - yet not installed because of external dependencies. What do you think about the idea ? Here's the code - the inflate method is still broken since it requires to know the available data size, and is not "inflating on demand". Nicolas Cannasse ---- let inflate i = let available = ref (match IO.available i with None -> assert false | Some size -> size) in let buf = Buffer.create 0 in let refill str = let len = String.length str in if !available >= len then begin let data = IO.nread i len in String.blit data 0 str 0 len; available := !available - len; len end else begin let size = !available in let data = IO.nread i size in String.blit data 0 str 0 size; available := 0; size end in let flush str n = Buffer.add_substring buf str 0 n in Zlib.uncompress ~header:true refill flush; IO.input_string (Buffer.contents buf) let deflate o = let buf = Buffer.create 0 in let flush() = let data = Buffer.contents buf in let pos = ref 0 in let available = ref (String.length data) in let refill str = let len = String.length str in if !available >= len then begin String.blit data !pos str 0 len; available := !available - len; pos := !pos + len; len end else begin let size = !available in String.blit data !pos str 0 size; available := 0; pos := !pos + size; size end in let flush str n = if n = String.length str then IO.nwrite o str else IO.nwrite o (String.sub str 0 n) in Zlib.compress ~level:9 ~header:true refill flush; IO.flush o; Buffer.reset buf; in IO.create_out (Buffer.add_char buf) (Buffer.add_string buf) (fun () -> Buffer.length buf) (fun () -> flush(); IO.flush o) (fun () -> flush(); IO.close_out o) |
From: Brian H. <bh...@sp...> - 2004-04-09 15:35:25
|
On Fri, 9 Apr 2004, Nicolas Cannasse wrote: > > 1) sinks and sources. > > 2) filters. > Theses two are now reality in the IO module. This is good- these two were the "core" of the idea, the rest was decoration. > For example, I managed to wrap CamlZip module on an IO and - without > changing a line of my code - I can now read/write over normal or compressed > files. Here's the code that's converting a standard IO into a > compressed/uncompressed one. I think it's worth including into the ExtLib > but the problem is it rely on another module (CamlZip, by Xavier Leroy) > which itself contains C stubs using the ZLib. I think we need an additional > folder into the ExtLib called "tools" that will be useful parts of code - > yet not installed because of external dependencies. > What do you think about the idea ? Actually, this is a deep problem, and not limited to just compression. I could see reimplementing zlib in pure Ocaml. But the #2 thing to do is encryption/decryption. For which you should not only call out to C, but have the C call hand tuned assembly. 20 cycles/byte encryption speed (approx. the speed of a good symmetric key crypto system like AES or Twofish) sounds real good, until you realize that this means 10 milliseconds to encrypt a megabyte of data on a 2GHz machine. Similiarly, I'm willing to be that RSA encryption using GMP will be signifigantly faster than using the Ocaml bignum library. Every cycle counts in this special case- even C is too slow. Compression we might want to reimplement in Ocaml. Reimplementing the crypto libraries in pure Ocaml, while theoretically possible, would impose severe performance limits. So, there are three possibilities: - Stick with implementing everything in Ocaml. This gives us portability and library independence (you don't need zlib nor gmp installed) at the cost of performance - Allow C/ASM callouts for performance - Provide both, and some mechanism for choosing between them (with the attendent problems) -- "Usenet is like a herd of performing elephants with diarrhea -- massive, difficult to redirect, awe-inspiring, entertaining, and a source of mind-boggling amounts of excrement when you least expect it." - Gene Spafford Brian |
From: Nicolas C. <war...@fr...> - 2004-04-09 16:05:52
|
> Actually, this is a deep problem, and not limited to just compression. I > could see reimplementing zlib in pure Ocaml. But the #2 thing to do is > encryption/decryption. For which you should not only call out to C, but > have the C call hand tuned assembly. [...] > So, there are three possibilities: > > - Stick with implementing everything in Ocaml. This gives us > portability and library independence (you don't need zlib nor gmp > installed) at the cost of performance > > - Allow C/ASM callouts for performance > > - Provide both, and some mechanism for choosing between them (with > the attendent problems) The problem is also the cost of reimplemeting. It was little painful for me to have to use the zlib, since I know it will requires all my sub-projects to compile and deploy some C code : the zlib itself - but also the CamlZip stubs (for which an Win32 Makefile was not given). But after looking at the ZLib format RFC, I somehow thought that it would take me quite a lot of time to reimplement it, and in the end maybe get a lower quality result (since zlib itself is quite a nice piece of code, highly suppported and optimized). If I had enough time, this is what I will do : - write a small C library that can alloc/access C memory , load dynlinked libraries, retreive and call functions from it - add Ocaml interface to this C library - port this C library to all architectures were Ocaml is working (including the good compilation parameters) - add this C library to ExtLib and install it by default and then we can write all the stubs to external C librairies in pure OCaml using this small C library ( let's call it EmuC ). We can then provide a lot of stubs for a lot of different C libraries according that : - theses libraries are dynlinkable - they can be accessed using EmuC Good points : - theses stubs will be pure ocaml and depends only on EmuC so we are minimizing the dependencies - the user can compile the stubs without actually having the library compiled/installed. - we only have to maintain and port one file of C code : emuC Bad points : - we have to correctly write the stubs in order to support several versions of the dynlinked library (while only a recompilation would have been needed for C stubs) - a little speed tradeoff since we're adding an C layer over every memory access we're doing Regards, Nicolas Cannasse |
From: Brian H. <bh...@sp...> - 2004-04-09 16:27:11
|
On Fri, 9 Apr 2004, Nicolas Cannasse wrote: > The problem is also the cost of reimplemeting. > It was little painful for me to have to use the zlib, since I know it will > requires all my sub-projects to compile and deploy some C code : the zlib > itself - but also the CamlZip stubs (for which an Win32 Makefile was not > given). But after looking at the ZLib format RFC, I somehow thought that it > would take me quite a lot of time to reimplement it, and in the end maybe > get a lower quality result (since zlib itself is quite a nice piece of code, > highly suppported and optimized). Hmm. Poking at a little bit, it doesn't look that bad. Actually, Ocaml might be a better language to implement them in, as several of the core data structures are already implemented. > > If I had enough time, this is what I will do : [ write EmuC ] The performance I'm not worried about- I don't think it'd be that bad. Nor the writting of stubs. EmuC would just be a library that shouldn't be used except by knowledgable people. Rather like the tricks we do in the optimized List module. We're already saying "do as we say, not as we do." This is one of the purposes of libraries, IMHO- the library gets the tricky stuff right, and everyone else uses the library. No, the problems I have with this idea are: 1) You'd still have C dependencies, breaking the Ocaml-only nature of ExtLib. 2) You would still have to deal with impedence mismatches- C API's that don't translate well into Ocaml. For example, routines which take pointers to variables to return more than one value. Or APIs that depend upon the "shape" of variables (for example, using unions). Or that want to use Macros to inline code. I'm not sure what all the impedence problems might be. -- "Usenet is like a herd of performing elephants with diarrhea -- massive, difficult to redirect, awe-inspiring, entertaining, and a source of mind-boggling amounts of excrement when you least expect it." - Gene Spafford Brian |
From: <syl...@po...> - 2004-04-09 18:01:43
|
Hello, On Fri, Apr 09, 2004 at 12:33:12PM -0500, Brian Hurt wrote: > On Fri, 9 Apr 2004, Nicolas Cannasse wrote: > > No, the problems I have with this idea are: > > 1) You'd still have C dependencies, breaking the Ocaml-only nature of > ExtLib. > > 2) You would still have to deal with impedence mismatches- C API's that > don't translate well into Ocaml. For example, routines which take > pointers to variables to return more than one value. Or APIs that depend > upon the "shape" of variables (for example, using unions). Or that want > to use Macros to inline code. I'm not sure what all the impedence > problems might be. > Even if i am not an Extlib developper, i permit myself to raise my voice... Sorry if you don't agree. I think extlib is a great project by the 100% Pure Ocaml coding style. I think putting 1 byte of C in it is not a good idea. It will break a lot of things, need to be maintained over a lot of arch ( including Ms Win, Linux, Unix, Irix et al ). I think it is far too complicated to be interessant. My opinion, is that you can create a new repository especially for project related to extlib... It will be far more simple, and will gives dependency on C only on this external module. Just to give you an example : CVSROOT/ extlib/ extlib-io-zip/ extlib-io-aes/ ... I think it is the less complicated you can do... Off course, the compilation will require to load other module than extlib, but i think it is worth the effort. Thank you for reading Kind regard Sylvain Le Gall |
From: Nicolas C. <war...@fr...> - 2004-04-09 19:02:38
|
> > No, the problems I have with this idea are: > > > > 1) You'd still have C dependencies, breaking the Ocaml-only nature of > > ExtLib. > > > > 2) You would still have to deal with impedence mismatches- C API's that > > don't translate well into Ocaml. For example, routines which take > > pointers to variables to return more than one value. Or APIs that depend > > upon the "shape" of variables (for example, using unions). Or that want > > to use Macros to inline code. I'm not sure what all the impedence > > problems might be. > > > > Even if i am not an Extlib developper, i permit myself to raise my > voice... Sorry if you don't agree. > > I think extlib is a great project by the 100% Pure Ocaml coding style. I > think putting 1 byte of C in it is not a good idea. It will break a lot > of things, need to be maintained over a lot of arch ( including Ms Win, > Linux, Unix, Irix et al ). I think it is far too complicated to be > interessant. > > My opinion, is that you can create a new repository especially for > project related to extlib... It will be far more simple, and will gives > dependency on C only on this external module. Just to give you an > example : > CVSROOT/ > extlib/ > extlib-io-zip/ > extlib-io-aes/ > ... > > I think it is the less complicated you can do... > Off course, the compilation will require to load other module than > extlib, but i think it is worth the effort. The point about EmuC is : - do we write and maintain C stubs for each of the external librairies we might need or - do we use EmuC only and write stubs in OCaml using EmuC If we choose the EmuC way, then we have only one - quite small - C file to maintain. That's might still be quite big, so maybe we don't have to put EmuC in the default install at the beginning, but after we have stubs for let's say 5 or more librairies, it will be worth it. Other choice is to keep away from C, but there is some times (zlib, cryptography, ipv6 support, and many others... ) that we can't or that the effort to write it again in ocaml is simply to big compare to writing stubs and maintaining EmuC. I don't want also to put C code into ExtLib, but maybe one day we'll have to :-) Regards, Nicolas Cannasse |
From: <syl...@po...> - 2004-04-09 19:30:00
|
Hello, On Fri, Apr 09, 2004 at 08:59:51PM +0200, Nicolas Cannasse wrote: > > The point about EmuC is : > - do we write and maintain C stubs for each of the external librairies we > might need > or > - do we use EmuC only and write stubs in OCaml using EmuC > If we choose the EmuC way, then we have only one - quite small - C file to > maintain. That's might still be quite big, so maybe we don't have to put > EmuC in the default install at the beginning, but after we have stubs for > let's say 5 or more librairies, it will be worth it. > > Other choice is to keep away from C, but there is some times (zlib, > cryptography, ipv6 support, and many others... ) that we can't or that the > effort to write it again in ocaml is simply to big compare to writing stubs > and maintaining EmuC. I don't want also to put C code into ExtLib, but maybe > one day we'll have to :-) > Well, i have nothing against emuc... I think it can come along extlib, but i don't think it is good to have it in extlib ( just my point of view ). However, have you considered some tools like camlidl ? I use it to wrap some C stuff, and it is very simple and powerful ( but it is not very flexible... ). It allows you to cp zlib.h camlzlib.idl vim camlzlib.idl -> explain that this structure, should this one in ocaml, this function returns, if the return is this, then it is an error etc camlidl camlzlib.idl -> camlzlib.c camlzlib.ml camlzlib.mli compile It is very easy to write stubs with it... I think it is a good tool to write easily C stub in Ocaml ( moreover it is written by a guy you should know : X. Leroy ) http://caml.inria.fr/camlidl/ I don't say the solution is perfect, but i have so many problem using C on different architecture comparing to ocaml portability ( which is sometime partial -- essentially when it depends on C code ). Kind regard Sylvain Le Gall |
From: Brian H. <bh...@sp...> - 2004-04-09 19:54:37
|
On Fri, 9 Apr 2004 syl...@po... wrote: > Hello, > > Even if i am not an Extlib developper, i permit myself to raise my > voice... Sorry if you don't agree. IMHO, If you're an Extlib user, you have the right to a voice here. > > I think extlib is a great project by the 100% Pure Ocaml coding style. I > think putting 1 byte of C in it is not a good idea. It will break a lot > of things, need to be maintained over a lot of arch ( including Ms Win, > Linux, Unix, Irix et al ). I think it is far too complicated to be > interessant. Actually, writting portable C isn't the problem. I routinely write C code which is much more widely portable than Ocaml is. But the more I think about it, the more I think linking to external libraries has other problems. For example, zlib and gmp both are standard installs on Linux systems. They both exist for Windows, but are not part of the standard installs. So now, to use extlib on windows, you now not only need to get ocaml and extlib, but also zlib and gmp. Plus, the code I'm thinking of grabbing for symmetric key crypto aren't standardly packaged in any library I'm aware of. How we connect to this code is almost irrelevent, the code still needs to be there. -- "Usenet is like a herd of performing elephants with diarrhea -- massive, difficult to redirect, awe-inspiring, entertaining, and a source of mind-boggling amounts of excrement when you least expect it." - Gene Spafford Brian |
From: Sylvain LE G. <syl...@po...> - 2004-04-09 20:18:23
|
Hello, On Fri, Apr 09, 2004 at 04:00:37PM -0500, Brian Hurt wrote: > On Fri, 9 Apr 2004 syl...@po... wrote: > > > Hello, > > > > Even if i am not an Extlib developper, i permit myself to raise my > > voice... Sorry if you don't agree. > > IMHO, If you're an Extlib user, you have the right to a voice here. > Thanks ;-) > > > > I think extlib is a great project by the 100% Pure Ocaml coding style. I > > think putting 1 byte of C in it is not a good idea. It will break a lot > > of things, need to be maintained over a lot of arch ( including Ms Win, > > Linux, Unix, Irix et al ). I think it is far too complicated to be > > interessant. > > Actually, writting portable C isn't the problem. I routinely write C code > which is much more widely portable than Ocaml is. > Well, most of the time, porting from BE to LEndian and trading with different int size is a great problem ( for me at least ). You need a lot of #define #ifdef... I don't like this... Test coverage for code is enough difficult. Adding compile time condition give you more code to test... > But the more I think about it, the more I think linking to external > libraries has other problems. For example, zlib and gmp both are > standard installs on Linux systems. They both exist for Windows, but are > not part of the standard installs. So now, to use extlib on windows, you > now not only need to get ocaml and extlib, but also zlib and gmp. Plus, > the code I'm thinking of grabbing for symmetric key crypto aren't > standardly packaged in any library I'm aware of. How we connect to this > code is almost irrelevent, the code still needs to be there. > I agree... The idea of having zlib, encryption is good but i think it should be not part of extlib. Kind regard Sylvain Le Gall ps : do you plan to use cryptokit and cryptgps for your library ? |
From: Brian H. <bh...@sp...> - 2004-04-09 21:18:36
|
On Fri, 9 Apr 2004, Sylvain LE GALL wrote: > > Actually, writting portable C isn't the problem. I routinely write C code > > which is much more widely portable than Ocaml is. > > > > Well, most of the time, porting from BE to LEndian and trading with > different int size is a great problem ( for me at least ). You need a > lot of #define #ifdef... I don't like this... Test coverage for code is > enough difficult. Adding compile time condition give you more code to > test... Worse yet, I've worked on systems which were *neither* BE nor LE (hint: DEC tried to switch horses in midstream). BE/LE only come into play when you're reading/writting binary data, or playing games with unions. In the first case, I tend to construct my ints by hand to control endianess. If you're doing I/O, the extra shift/ors aren't that big of a problem. As for playing games with unions, these are bad ideas anyways, don't do them. As for int sizes, all this means is you need to think about what types things are. One of the worst habits C coders get into is not thinking about what types variables should be. I'd argue this is more important in C than it is in Ocaml. An example: what type should the variable i be in the following code? for (i = 0; i < sizeof(arr)/sizeof(arr[0]); ++i) { arr[i] = 0; } My answer: i should have the type size_t. Thinking about what types variables should be solves most of the problems (and eliminates the vast majority of pointless typecasts). I hate ifdefs, except for providing imdepotence for header files (at which point I insist on them). But it's surprising how portable you can get without them. > I agree... > > The idea of having zlib, encryption is good but i think it should be not > part of extlib. That's where I'm ending up as well. Or at least not C-based encryption and compression. Ocaml code only. Note that there might be a sister project spawned to use the C code for higher performance. > ps : do you plan to use cryptokit and cryptgps for your library ? I hadn't thought about it much. Last time I took a spin through the crypto libraries, I was more than a little surprised that not one used GMP for RSA. Which I found quite surprising, as I'm willing to bet that GMP would give you the best possible RSA performance with the least work from the implementor. IIRC, GMP already has an "exponentiate in a modular field" operation, implemented with tuned assembly code. This is the core operation of RSA. For symmetric key and hashing, I was thinking of rounding up the hand-tunned assembly versions kicking around. I haven't decided what to do with Elliptic curve yet. Thinking about it a second, there's a problem with providing Ocaml implementations of various symmetric key crypto systems- most of them assume you have access to efficient 32-bit integers. Which means you either use Int32 at a serious performance and memory hit. -- "Usenet is like a herd of performing elephants with diarrhea -- massive, difficult to redirect, awe-inspiring, entertaining, and a source of mind-boggling amounts of excrement when you least expect it." - Gene Spafford Brian |
From: William N. <wne...@cs...> - 2004-04-10 01:16:19
|
On Apr 9, 2004, at 4:24 PM, Brian Hurt wrote: > I hadn't thought about it much. Last time I took a spin through the > crypto libraries, I was more than a little surprised that not one used > GMP > for RSA. Which I found quite surprising, as I'm willing to bet that > GMP > would give you the best possible RSA performance with the least work > from > the implementor. IIRC, GMP already has an "exponentiate in a modular > field" operation, implemented with tuned assembly code. This is the > core > operation of RSA. I was a bit surprised by this as well, so I modified cryptokit to use GMP, and I also added in a number of additional features, like DSA, SHA-{265, 384, 512}, a number of random number and prime generation routines, hash chains, etc. I've been meaning to package it up for quite a while now, but I'm an idiot when it comes to things like makefiles, and I was barely able to cobble one together for my own needs... And oh yeah... my ocamldoc code is a bit farkled. Plus there's the whole issue of getting GMP and MLGMP up and running. Anyway, if you have a need for this, I'd be happy to sent you what I have. > For symmetric key and hashing, I was thinking of > rounding up the hand-tunned assembly versions kicking around. I > haven't > decided what to do with Elliptic curve yet. As always, the problem with hand tuned assembly is the portability issue, that's why I try to stay away from it where I can -- plus, I'm not sure you really get significant enough benefits from assembly coding the modern symmetric ciphers and hashes. And I've been meaning to add some EC stuff to cryptokit-gmp for a long time. Laziness, it'll get you every time. > Thinking about it a second, there's a problem with providing Ocaml > implementations of various symmetric key crypto systems- most of them > assume you have access to efficient 32-bit integers. Which means you > either use Int32 at a serious performance and memory hit. Yep. And this is one of the biggest pains-in-the-ass for me when it comes to OCaml (I do crypto research for a living). There are times I would kill for an efficient word32/64 datatype... Thank goodness we at least have string_unsafe_get and set. William D. Neumann "Here come the bunnies with the sugar water, Do a little dance with the farmer's daughter." -- The Halo Benders |
From: Nicolas C. <war...@fr...> - 2004-04-09 21:03:21
|
> > Hello, > > > > Even if i am not an Extlib developper, i permit myself to raise my > > voice... Sorry if you don't agree. > > IMHO, If you're an Extlib user, you have the right to a voice here. Same opinion here. > > I think extlib is a great project by the 100% Pure Ocaml coding style. I > > think putting 1 byte of C in it is not a good idea. It will break a lot > > of things, need to be maintained over a lot of arch ( including Ms Win, > > Linux, Unix, Irix et al ). I think it is far too complicated to be > > interessant. > > Actually, writting portable C isn't the problem. I routinely write C code > which is much more widely portable than Ocaml is. > > But the more I think about it, the more I think linking to external > libraries has other problems. For example, zlib and gmp both are > standard installs on Linux systems. They both exist for Windows, but are > not part of the standard installs. So now, to use extlib on windows, you > now not only need to get ocaml and extlib, but also zlib and gmp. Plus, > the code I'm thinking of grabbing for symmetric key crypto aren't > standardly packaged in any library I'm aware of. How we connect to this > code is almost irrelevent, the code still needs to be there. As I told you : you don't need to have the libs installed to compile stubs. Now if you want to use the features that are needind the libs, you only need to have the dynamic library in your path so that EmuC can find it. All linking is done dynamicly at runtime. As for the zlib - and many other libraries - there is always some binary available for windows, because we - windows programmers - are known to be not clever enough to rebuild from a Makefile :-) To answer to Sylvain, crypto was just an example of external library, I'm not sure it should be part of ExtLib (or maybe only the MD5 - useful enough to be standard : anyone contributing ? ). Regards, Nicolas Cannasse |
From: Sylvain LE G. <syl...@po...> - 2004-04-09 21:15:16
|
Hello, On Fri, Apr 09, 2004 at 11:00:34PM +0200, Nicolas Cannasse wrote: > > > Hello, > > > > > > Even if i am not an Extlib developper, i permit myself to raise my > > > voice... Sorry if you don't agree. > > > > IMHO, If you're an Extlib user, you have the right to a voice here. > > Same opinion here. > > > > I think extlib is a great project by the 100% Pure Ocaml coding style. I > > > think putting 1 byte of C in it is not a good idea. It will break a lot > > > of things, need to be maintained over a lot of arch ( including Ms Win, > > > Linux, Unix, Irix et al ). I think it is far too complicated to be > > > interessant. > > > > Actually, writting portable C isn't the problem. I routinely write C code > > which is much more widely portable than Ocaml is. > > > > But the more I think about it, the more I think linking to external > > libraries has other problems. For example, zlib and gmp both are > > standard installs on Linux systems. They both exist for Windows, but are > > not part of the standard installs. So now, to use extlib on windows, you > > now not only need to get ocaml and extlib, but also zlib and gmp. Plus, > > the code I'm thinking of grabbing for symmetric key crypto aren't > > standardly packaged in any library I'm aware of. How we connect to this > > code is almost irrelevent, the code still needs to be there. > > As I told you : you don't need to have the libs installed to compile stubs. > Now if you want to use the features that are needind the libs, you only need > to have the dynamic library in your path so that EmuC can find it. All > linking is done dynamicly at runtime. As for the zlib - and many other > libraries - there is always some binary available for windows, because we - > windows programmers - are known to be not clever enough to rebuild from a > Makefile :-) > Yep... But i think it could be achieve by doing an external module... It has three benefits : - don't imply C code in extlib - don't imply dependency on external library - is a good example to know if you can let people build their own external module for extlib ( ie by this way you can test the completeness of the interface given to external people to build things with extlib ). > To answer to Sylvain, crypto was just an example of external library, I'm > not sure it should be part of ExtLib (or maybe only the MD5 - useful enough > to be standard : anyone contributing ? ). > INRIA has already contribut to this ;-) ( Hash of string are MD5 sums, cryptokit use it directly, cryptokit has already a MD5 module ) Regard Sylvain Le Gall |
From: Yamagata Y. <yor...@mb...> - 2004-03-20 03:33:50
|
Before anything, I'd like to mention the recent exchange about another "extlib" in the caml-list and others. I'm little embarrassed in the current situation. If I remember correctly, another "extlib" is older than ours, so, in my opinion, we have to change the name. Whatever reasons there are, we have to give a due credit to the previous work. If we are going to change the nature of our library, then it is a good opportunity to change the name. From: "Nicolas Cannasse" <war...@fr...> Subject: [Ocaml-lib-devel] Future of ExtLib Date: Fri, 19 Mar 2004 09:29:01 +0100 > That's convenient, but in fact since we're keeping 99% compability > with stdlib, having the user to port the small parts of his code and > then he can use all extensions provided as if he was using stdlib > seems to be a fair trade. Of course this means some changes to > current project, and that's why I would like to get your comments > and advices about this - I'm especialy interested in current > ExtLib's contributors opinions. I'm not clear about what you want to do. But if you mean, say, we provide List module in extlib, so that override List of the stdlib, then I think it is definitely a bad idea. In addition to forcing to use the new stdlib in order to access some module in extlib, it will cause incomprehensive "inconsistent assumption" error if modules using old stdlib and using new stdlib are mixed up. If we use -pack option, then the change would be more bearable, but I prefer to split extlib to several packages, one for replacing stdlib, another for providing new features. Maybe we could split the packages for their purpose, like providing new data structure, I/O, etc, too. From: "Nicolas Cannasse" <war...@fr...> Subject: [Ocaml-lib-devel] Future of ExtLib Date: Fri, 19 Mar 2004 09:29:01 +0100 > Concerning additionnals modules here's what's on my wish-list (which is > somehow also on my current todo-list) : > - base64 encode/decode > - abstract high level I/O with support for C basic types ( read_i16 , > write_f16 ..... ) > - zlib deflate/inflate written in pure OCaml. Do you interested in diet set and map? (set and map over integer, internally represented as sets and maps over intervals.) I could contribute them. Also, I'd like to change install.ml so that it ignores findlib if the destination directory is explicitly given by the arguments. -- Yamagata Yoriyuki |
From: <syl...@po...> - 2004-03-20 10:04:41
|
Hello, On Sat, Mar 20, 2004 at 12:33:07PM +0900, Yamagata Yoriyuki wrote: > Before anything, I'd like to mention the recent exchange about another > "extlib" in the caml-list and others. I'm little embarrassed in the > current situation. If I remember correctly, another "extlib" is older > than ours, so, in my opinion, we have to change the name. Whatever > reasons there are, we have to give a due credit to the previous work. > If we are going to change the nature of our library, then it is a good > opportunity to change the name. > I don't agree with your point of view. The so called "older" version of extlib is only "older" because before it was named "stew". And one day, the upstream decided to recall it extlib. I see no reason to change the name of extlib because one guy thinks it was a prettier name. It is a personnal point of view, but i keep considering that extlib is "tm" to sf library ( and not the other one ). Regard Sylvain Le Gall |