From: Nicolas C. <war...@fr...> - 2003-11-14 05:34:20
Attachments:
path.ml
|
Hi list, I came up very recently with a typical problem. I had a file A and its relative path, I had a file B and its absolute path, I wanted to get the file B path relative to A.... not so easy actually. I wrote this small module Path : Path.make "my/../path/./" build a relative path Path.absolute give the absolute path Path.relative a b give the path of a relative to b Path.up goes up one directory I did some tests under windows , it looks like it works well. If people can test it under *nix... Note : this does not actually do any system checking ( the path may not exists or contains invalid characters ). Only special paths . .. and / ( and windows drive letters ) are recognized as special. Nicolas Cannasse |
From: <syl...@po...> - 2003-11-14 06:46:33
|
On Fri, Nov 14, 2003 at 02:33:34PM +0900, Nicolas Cannasse wrote: > Hi list, > I came up very recently with a typical problem. > I had a file A and its relative path, I had a file B and its absolute path, > I wanted to get the file B path relative to A.... not so easy actually. I > wrote this small module Path : > > Path.make "my/../path/./" build a relative path > Path.absolute give the absolute path > Path.relative a b give the path of a relative to b > Path.up goes up one directory > > I did some tests under windows , it looks like it works well. If people can > test it under *nix... > > Note : this does not actually do any system checking ( the path may not > exists or contains invalid characters ). Only special paths . .. and / ( and > windows drive letters ) are recognized as special. > > Nicolas Cannasse Hello, In fact, i have exactly the same module for Unix ( but i need to call it SysPath ). Are you interested in fusionning your module with mine ? I had also SysUtil which try to implement some common unix function ( find, which, mkdir, test... ) in SysUtil. I am really interested in fusioning your module with mine, because i want fileutils ( my module ) to be able to work on every platform. Kind regard Sylvain LE GALL |
From: Nicolas C. <war...@fr...> - 2003-11-14 07:01:57
|
[...] > In fact, i have exactly the same module for Unix ( but i need to call it > SysPath ). > > Are you interested in fusionning your module with mine ? > > I had also SysUtil which try to implement some common unix function ( > find, which, mkdir, test... ) in SysUtil. > > I am really interested in fusioning your module with mine, because i > want fileutils ( my module ) to be able to work on every platform. I agree. Such a nice module including a lot of features for working with filesystem and handling incompatibilites would definitly be useful. I'm ok for merging, as long as you don't append a reference to the Unix module : actually Path is manipulating "virtual" paths and then does not need to rely on the filesystem function. Having some functions in other module relying on path is ok. You can proceed the merge and post it on this list (including a clear MLI interface), I'm sure there is other people here that might be interested. Nicolas Cannasse |
From: Sylvain LE G. <syl...@po...> - 2003-11-14 18:38:33
|
On Fri, Nov 14, 2003 at 04:01:51PM +0900, Nicolas Cannasse wrote: > [...] > > In fact, i have exactly the same module for Unix ( but i need to call it > > SysPath ). > > > > Are you interested in fusionning your module with mine ? > > > > I had also SysUtil which try to implement some common unix function ( > > find, which, mkdir, test... ) in SysUtil. > > > > I am really interested in fusioning your module with mine, because i > > want fileutils ( my module ) to be able to work on every platform. > > I agree. > Such a nice module including a lot of features for working with filesystem > and handling incompatibilites would definitly be useful. > I'm ok for merging, as long as you don't append a reference to the Unix > module : actually Path is manipulating "virtual" paths and then does not > need to rely on the filesystem function. Having some functions in other > module relying on path is ok. > > You can proceed the merge and post it on this list (including a clear MLI > interface), I'm sure there is other people here that might be interested. > > Nicolas Cannasse > > Hello, Well, all the idea i just read from this thread will be included in such a module. For now, it is very simple ( ie very simple algo ). But it works very well for Unix. I want to do Win32SysPath, MacSysPath and UnixSysPath and SysPath which will be a bind to the current platforme SysPath. For this i need an abstraction layer to compute some specific things ( path separator, drive letter et al ). I think the next version will come with a parser/lexer for each OS which will produce something like describe in the thread ( but i prefer type path = Root of string | Component of string | ParentDir | CurrentDir | Empty | Link of string * path... I don't know yet for Link because it is system dependent. I separate the module in two : - SysPath : includes all things which doesn't touch to filesystem ( that is the reason why i don't know if it is interessant to have Link ). - SysUtil : includes all things which touch the filesystem, so you have to be consistent with FS, it is also a try to imitate a lot of Unix command like find, which... For now, it has been tested and it works on Linux ( and the abstract SysPath has been tested with Linux FS syntax ). Here are the prototype : SysPath.mli : (** You cannot pass a base path which is relative *) exception Base_path_relative (** One of the path you have passed is relative and cannot be reduce *) exception Path_relative_unreducable (** All operation as defined in module Filename *) val current_dir_name : string val parent_dir_name : string val concat : string -> string -> string val is_relative : string -> bool val is_implicit : string -> bool val check_suffix : string -> string -> bool val chop_suffix : string -> string -> string val chop_extension : string -> string val basename : string -> string val dirname : string -> string val temp_file : string -> string -> string val open_temp_file : ?mode:open_flag list -> string -> string -> string * out_channel val quote : string -> string (** Take a list of path component and return the string corresponding to this path *) val implode : string list -> string (** Take a string corresponding to a path a return the component of this path *) val explode : string -> string list (** Remove all path component which are relative inside the path For example : /a/../b -> /b/. Path must not be relative *) val reduce : string -> string (** Create an absolute path from a path relative to the base path *) val make_absolute : string -> string -> string (** Create a path which is relative to the base path *) val make_relative : string -> string -> string (** Create an environnement PATH like string from different path *) val make_path : string list -> string (** Return the different component of an environnement PATH like string *) val explode_path : string -> string list SysUtil.mli : (** Pattern you can use to test file *) type test_file = Is_dev_block | Is_dev_char | Is_dir | Exists | Is_file | Is_set_group_ID | Has_sticky_bit | Is_link | Is_pipe | Is_readable | Is_writeable | Size_not_null | Is_socket | Has_set_user_ID | Is_exec | Is_owned_by_user_ID | Is_owned_by_group_ID | Is_newer_than of string * string | Is_older_than of string * string | Has_same_device_and_inode of string * string | And of test_file * test_file | Or of test_file * test_file | Not of test_file | Match of string | True | False val list_dir : string -> string list (** Apply a filtering pattern to a filename *) val filter : test_file -> string list -> string list (** Test the existence of the file... *) val test : test_file -> string -> bool (** Try to find the executable in the PATH. Use environement variable PATH if none is provided *) val which : ?path:string list -> string -> string The first functions of SysPath are simple binding to Filename. In SysUtils i have a reference to Unix ( sorry ). If you have any suggestion... This library is under developpement and has no public release yet. But if some of you are interested, i will speed up the dev in order to do a public release. Kind regard Sylvain LE GALL |
From: Nicolas C. <war...@fr...> - 2003-11-15 01:37:44
|
> I separate the module in two : > - SysPath : includes all things which doesn't touch to filesystem ( that > is the reason why i don't know if it is interessant to have Link ). > - SysUtil : includes all things which touch the filesystem, so you have to > be consistent with FS, it is also a try to imitate a lot of Unix command > like find, which... > > For now, it has been tested and it works on Linux ( and the abstract > SysPath has been tested with Linux FS syntax ). [...] > If you have any suggestion... > This library is under developpement and has no public release yet. But > if some of you are interested, i will speed up the dev in order to do a > public release. The test_file patterns does not seem very powerful to me : - for the most obvious cases, like filtering by extension, it can't be used - for complex cases, you are limited by the number of test cases. I think it could be easily remplaced by : val filter : (Unix.stats -> bool) -> string list -> string list (** we are clearly showing the dependency to unix here *) Other things I would like to get added to the Filename module : val extension : string -> string (* returns the extension of the file or "" if no extension *) val switch_extension : file:string -> ext:string -> string (* change the extension of a filename *) Nicolas Cannasse |
From: <syl...@po...> - 2003-11-15 09:50:57
|
Hello, On Sat, Nov 15, 2003 at 10:34:29AM +0900, Nicolas Cannasse wrote: > > I separate the module in two : > > - SysPath : includes all things which doesn't touch to filesystem ( that > > is the reason why i don't know if it is interessant to have Link ). > > - SysUtil : includes all things which touch the filesystem, so you have to > > be consistent with FS, it is also a try to imitate a lot of Unix command > > like find, which... > > > > For now, it has been tested and it works on Linux ( and the abstract > > SysPath has been tested with Linux FS syntax ). > [...] > > If you have any suggestion... > > This library is under developpement and has no public release yet. But > > if some of you are interested, i will speed up the dev in order to do a > > public release. > > The test_file patterns does not seem very powerful to me : > - for the most obvious cases, like filtering by extension, it can't be used ??? and what about Match(".*\\.cmi") for test ( it really works since i use it ). > - for complex cases, you are limited by the number of test cases. > I think it could be easily remplaced by : > ??? there is no limit about the number of test : And(Match(".*\\.cmi"),Or(Is_file,Is_dir))... But you could go to over thousand of test like this. Anyway the test_file pattern is a direct copy of the command test in Unix, which is one of the most used in Unix, so i think it is enough powerfull ( i just have removed all the test concerning strings ). > val filter : (Unix.stats -> bool) -> string list -> string list (** we are > clearly showing the dependency to unix here *) > > Other things I would like to get added to the Filename module : > > val extension : string -> string (* returns the extension of the file or "" > if no extension *) > val switch_extension : file:string -> ext:string -> string (* change the > extension of a filename *) > I agree on this, just raising an exception if extension is not found ( raise Not_found ), better than returning "". Kind regard Sylvain LE GALL |
From: Remi V. <van...@la...> - 2003-11-16 11:35:35
|
"Sylvain LE GALL" <syl...@po...> writes: Hello, [...] firstly, for both SysUtil.mli and SysPath.mli, I would begin them by a type filename = string and replace string by filename where a file name (or directory name) is waited. It would make everything clearer. > SysUtil.mli : > > (** Pattern you can use to test file *) > type test_file = > Is_dev_block > | Is_dev_char > | Is_dir > | Exists > | Is_file > | Is_set_group_ID > | Has_sticky_bit > | Is_link > | Is_pipe > | Is_readable > | Is_writeable > | Size_not_null > | Is_socket > | Has_set_user_ID > | Is_exec > | Is_owned_by_user_ID > | Is_owned_by_group_ID > | Is_newer_than of string * string > | Is_older_than of string * string > | Has_same_device_and_inode of string * string > | And of test_file * test_file > | Or of test_file * test_file > | Not of test_file > | Match of string > | True > | False It's very interesting. Some possible amelioration : - What Is_newer_than mean exactly, what are those 2 string? - a Is_bigger_than of int and a Is_smaller_than of int could be useful; - for the newer and older, It could be useful to give a date directly; - I would use : Or of test_file list and And of test_file list; -- Rémi Vanicat |
From: <syl...@po...> - 2003-11-16 11:57:21
|
Hello, On Sun, Nov 16, 2003 at 12:34:48PM +0100, Remi Vanicat wrote: > "Sylvain LE GALL" <syl...@po...> writes: > > Hello, > > [...] > > firstly, for both SysUtil.mli and SysPath.mli, I would begin them by > a type filename = string > and replace string by filename where a file name (or directory name) > is waited. > You are right, i will add it to my todo list. > It would make everything clearer. > > > SysUtil.mli : > > > > (** Pattern you can use to test file *) > > type test_file = > > Is_dev_block > > | Is_dev_char > > | Is_dir > > | Exists > > | Is_file > > | Is_set_group_ID > > | Has_sticky_bit > > | Is_link > > | Is_pipe > > | Is_readable > > | Is_writeable > > | Size_not_null > > | Is_socket > > | Has_set_user_ID > > | Is_exec > > | Is_owned_by_user_ID > > | Is_owned_by_group_ID > > | Is_newer_than of string * string > > | Is_older_than of string * string > > | Has_same_device_and_inode of string * string > > | And of test_file * test_file > > | Or of test_file * test_file > > | Not of test_file > > | Match of string > > | True > > | False > > It's very interesting. > Some possible amelioration : > - What Is_newer_than mean exactly, what are those 2 string? The two strings are filename > - a Is_bigger_than of int and a Is_smaller_than of int could be > useful; I treat of file not of int, i don't want to get <, >, <=, >= in my library ( but if you insist, it is not very complicated to add this test ). > - for the newer and older, It could be useful to give a date > directly; And the date problem arise : yes but which date ( actually ocaml std library only provide timestamp conversion... ). > - I would use : Or of test_file list and And of test_file list; > Yes.... Think, i'll keep And/Or and add AndList/OrList Regard Sylvain LE GALL |
From: Remi V. <van...@la...> - 2003-11-17 10:12:21
|
<syl...@po...> writes: > Hello, > Hello, >> > SysUtil.mli : >> > >> > (** Pattern you can use to test file *) >> > type test_file = >> > Is_dev_block >> > | Is_dev_char >> > | Is_dir >> > | Exists >> > | Is_file >> > | Is_set_group_ID >> > | Has_sticky_bit >> > | Is_link >> > | Is_pipe >> > | Is_readable >> > | Is_writeable >> > | Size_not_null >> > | Is_socket >> > | Has_set_user_ID >> > | Is_exec >> > | Is_owned_by_user_ID >> > | Is_owned_by_group_ID >> > | Is_newer_than of string * string >> > | Is_older_than of string * string >> > | Has_same_device_and_inode of string * string >> > | And of test_file * test_file >> > | Or of test_file * test_file >> > | Not of test_file >> > | Match of string >> > | True >> > | False >> >> It's very interesting. >> Some possible amelioration : >> - What Is_newer_than mean exactly, what are those 2 string? > > The two strings are filename > I mean, I understood what is the result of the test Exists apply to the filename "foo". I don't understood what is the result of the test (Is_newer_than ("bar", "quux")) apply to the filename "foo". >> - a Is_bigger_than of int and a Is_smaller_than of int could be >> useful; > I treat of file not of int, i don't want to get <, >, <=, >= in my > library ( but if you insist, it is not very complicated to add this test > ). The idea was to compare the size of the file with an number of octet (or any other size measurement that could be useful). >> - for the newer and older, It could be useful to give a date >> directly; > > And the date problem arise : yes but which date ( actually ocaml std > library only provide timestamp conversion... ). I agree there. As I've understood, you use the Unix.stat function for your implementation, so I would use the same format than the st_?time field of the stats type. > >> - I would use : Or of test_file list and And of test_file list; >> > > Yes.... Think, i'll keep And/Or and add AndList/OrList Well, it is not so important, just a preference of mine. -- Rémi Vanicat |
From: <syl...@po...> - 2003-11-17 18:54:49
|
Hello, On Mon, Nov 17, 2003 at 11:11:36AM +0100, Remi Vanicat wrote: > <syl...@po...> writes: > > > Hello, > > > > Hello, > > >> It's very interesting. > >> Some possible amelioration : > >> - What Is_newer_than mean exactly, what are those 2 string? > > > > The two strings are filename > > > > I mean, I understood what is the result of the test Exists apply to > the filename "foo". I don't understood what is the result of the test > (Is_newer_than ("bar", "quux")) apply to the filename "foo". > Ok, i see your point... In fact, it is not a real API of mine, the test is here because it is in the exec "test", that you can found of unix. Basically, i took the man of "test" and reimplement it as it is... But i can extend it... For example : Is_newer_than_date(x) can be a good test also. > >> - a Is_bigger_than of int and a Is_smaller_than of int could be > >> useful; > > I treat of file not of int, i don't want to get <, >, <=, >= in my > > library ( but if you insist, it is not very complicated to add this test > > ). > > The idea was to compare the size of the file with an number of octet > (or any other size measurement that could be useful). > Well perfect, what about Is_bigger_than_size(x), Is_smaller_than_size(x), Is_equal_to_size(x) with x of type type file_size = TO of int | GO of int | MO of int | KO of int | O of int And rounding to the the same unit. > > >> - for the newer and older, It could be useful to give a date > >> directly; > > > > And the date problem arise : yes but which date ( actually ocaml std > > library only provide timestamp conversion... ). > > I agree there. As I've understood, you use the Unix.stat function for > your implementation, so I would use the same format than the st_?time > field of the stats type. > I prefer not to implement directly the size field, which doesn't speak too much too me... But it is an option > > > >> - I would use : Or of test_file list and And of test_file list; > >> > > > > Yes.... Think, i'll keep And/Or and add AndList/OrList > > Well, it is not so important, just a preference of mine. > Preference is very interesting... Because i would to have some people using my lib ;-> The discussion is totally open, my lib is at an early stage of dev and i would like to have suggestion ( i don't defend the actual layout of the lib which was at first and adhoc lib for some dev of mine ). Kind regard Sylvain LE GALL |
From: Michal M. <mal...@pl...> - 2003-11-14 11:25:28
|
On Fri, Nov 14, 2003 at 02:33:34PM +0900, Nicolas Cannasse wrote: > Note : this does not actually do any system checking ( the path may not > exists or contains invalid characters ). Only special paths . .. and / ( and > windows drive letters ) are recognized as special. No characters are special in unix paths beside '/'. Special paths are only "." and ".." (and maybe "/"). Of course this also depends on filesystem, but most native FSes allow anything. But windows drive letters IMHO should be optional (for example support them if OS is windows). -- : Michal Moskal :: http://www.kernel.pl/~malekith : GCS {C,UL}++++$ a? !tv : When in doubt, use brute force. -- Ken Thompson : {E-,w}-- {b++,e}>+++ h |
From: Eric C. C. <ec...@cm...> - 2003-11-14 13:56:59
|
On Fri, Nov 14, 2003 at 12:23:17PM +0100, Michal Moskal wrote: > On Fri, Nov 14, 2003 at 02:33:34PM +0900, Nicolas Cannasse wrote: > > Note : this does not actually do any system checking ( the path may not > > exists or contains invalid characters ). Only special paths . .. and / ( and > > windows drive letters ) are recognized as special. > > No characters are special in unix paths beside '/'. Special paths are > only "." and ".." (and maybe "/"). Of course this also depends on > filesystem, but most native FSes allow anything. It's not clear that you can do this independently of the OS interface. For example, to handle a/b/../c correctly, you need to check whether "b" is a symbolic link and interpret it accordingly. -- Eric C. Cooper e c c @ c m u . e d u |
From: Nicolas C. <war...@fr...> - 2003-11-15 01:08:21
|
> > > Note : this does not actually do any system checking ( the path may not > > > exists or contains invalid characters ). Only special paths . .. and / ( and > > > windows drive letters ) are recognized as special. > > > > No characters are special in unix paths beside '/'. Special paths are > > only "." and ".." (and maybe "/"). Of course this also depends on > > filesystem, but most native FSes allow anything. > > It's not clear that you can do this independently of the OS interface. > For example, to handle a/b/../c correctly, you need to check whether > "b" is a symbolic link and interpret it accordingly. It's true. But when you're doing a/b/../c , do you actually really now what you're doing ? :) Theses kind of things should be marked as "features", not "bug". Nicolas Cannasse |
From: Martin J. <mar...@em...> - 2003-11-14 12:57:07
|
On Fri, 14 Nov 2003, Nicolas Cannasse wrote: > Hi list, > I came up very recently with a typical problem. > I had a file A and its relative path, I had a file B and its absolute path, > I wanted to get the file B path relative to A.... not so easy actually. I > wrote this small module Path : > > Path.make "my/../path/./" build a relative path > Path.absolute give the absolute path > Path.relative a b give the path of a relative to b > Path.up goes up one directory > > I did some tests under windows , it looks like it works well. If people can > test it under *nix... > > Note : this does not actually do any system checking ( the path may not > exists or contains invalid characters ). Only special paths . .. and / ( and > windows drive letters ) are recognized as special. Hi, Such a module seems very important to me too. I've got some suggestions too. It seems important to me to be able to choose input/output style: Unix, Windows, MacOS, and others. Maybe as an option passed to functions "make" and "to_string". type style = Unix | Windows | Cygwin | ... let default_style = match Sys.os_type with "Win32" -> Windows | "Unix" -> Unix | ... let of_string ?(style = default_style) s = ... let to_string ?(style = default_style) x = ... This is less clear to me, but I feel that it would be convenient to split the type t into a root part and a virtual path part: type root = Windows of string | Unix | Current_dir | Path of t and t = { root : root; virtual_path : string list } ... and implement functions that change either the root or the virtual path. Some problems concerning the current implementation under Linux: 1. //tmp///toto is valid and equivalent to /tmp/toto (because any filename must have at least one character) 2. /../../.. is equivalent to / (which is dangerous anyway) 3. toto/ is more restrictive than toto (trailing slash indicates that toto must be a directory), so don't output trailing slashes except for the root directory, or introduce a distinction between files and directories. For solving points 1 and 2, maybe it would be nice to introduce 2 input styles: Unix and Abstract, where Unix conforms to Unix (Linux?) standards and Abstract is something more abstract, that makes "/.." invalid and allows 0-length filenames (would it be useful?). Martin |
From: skaller <sk...@oz...> - 2003-11-15 16:50:18
|
On Fri, 2003-11-14 at 23:58, Martin Jambon wrote: > On Fri, 14 Nov 2003, Nicolas Cannasse wrote: > > type root = Windows of string | Unix | Current_dir | Path of t > > and t = { root : root; > virtual_path : string list } > > ... and implement functions that change either the root or the > virtual path. I have had this concept for some time, however it is slightly different. I use it in interscript. The concept is of a "mount point" which is an arbitrary atomic operating system dependent prefix (usually specifying a directory), plus a universal relative pathname convention following Unix: name / name / .... / name The universal filename is constrained -- you can NOT represent all possible filenames on every system, instead, every such filename is representable on all systems. The way this works is: the mount point is obtained from the command line or some environment variable, by some portable means, but it is never analysed, so the system dependency is irrelevant to portability. The string form of the universal filename can be used, or an unpacked form as with 'virtual_path : string list' can be constructed. Either form can be used for manipulation of subdirectories or files, but it is all indepdendent of the mount point. Thus: in interscript you can write: @h = tangler('src/xxx.ml') [blah bla code to tangle out to xxx .. ] and on the command line you write: iscr --tangler_directory="d:\program files\silly windows\" and the function 'make_os_filename' combines the mount point and the universal filename to make (in this case) a Windows filename. Thus, all filenames inside interscript work on all platforms, even though relative Unix filenames must be used -- precisely *because* they're used as the universal format. You will note you CANNOT specify an absolute path -- it is not permitted in the suffix part. In particular no portable program is allowed to know which filesystem it is working with. The client program must act as if each mount point is precisely that: the root of a file system. With this concept, issues like: -----------Martin---------------------> Some problems concerning the current implementation under Linux: 1. //tmp///toto is valid and equivalent to /tmp/toto (because any filename must have at least one character) 2. /../../.. is equivalent to / (which is dangerous anyway) 3. toto/ is more restrictive than toto (trailing slash indicates that toto <---------------------------------------- ... simply cannot arise. We dont care about the mount point string at all, since the program never sees it and isn't permitted to synthesise it. We care about the universal part, but what Linux does is irrelevant. We can define our own rules (and I think we just ban consecutive slashes for example). Finally: this mechanism requires a bit of a paradigm shift. You need to see that splitting the notion of heirarchical directories and filesystems/mount points always works when there is a portability requirement. And post-finally <g>: one can extend the concept to include access methods. Whence there is already a Standard portable format, namely that used for a URL. |
From: Yamagata Y. <yo...@us...> - 2003-11-14 13:26:26
|
From: "Nicolas Cannasse" <war...@fr...> Subject: [Ocaml-lib-devel] Proposal : Path Date: Fri, 14 Nov 2003 14:33:34 +0900 > let def_sep =3D (if windows then "\\" else "/") > > exception Cannot_up > exception Invalid_path of string > > let to_string =3D function > | [] -> "" > | [x] when x =3D=3D def_sep -> def_sep > | p -> > String.concat def_sep (List.rev p) ^ def_sep Is there a reason not using Filename module? -- Yamagata Yoriyuki |
From: Nicolas C. <war...@fr...> - 2003-11-15 01:06:33
|
> > let def_sep =3D (if windows then "\\" else "/") > > > > exception Cannot_up > > exception Invalid_path of string > > > > let to_string =3D function > > | [] -> "" > > | [x] when x =3D=3D def_sep -> def_sep > > | p -> > > String.concat def_sep (List.rev p) ^ def_sep > > Is there a reason not using Filename module? > The path is actually not using any filename. it's only the directory part which is manipulated. The only thing is that once the path is converted into a string, you can safely add a filename at its end. Nicolas Cannasse |