[MinGW-notify] [ mingw-Patches-3046195 ] Native symlinks

SourceForge Headquarters 225 Broadway Suite 1600 San Diego, CA 92101 +1 (858) 454-5900

Patches item #3046195, was opened at 2010-08-16 07:29
Message generated for change (Comment added) made by earnie
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=302435&aid=3046195&group_id=2435

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: msys
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Ladislav Michl (ladis)
Assigned to: Cesar Strauss (cstrauss)
Summary: Native symlinks

Initial Comment:
This premiliary patch contains support for native symlinks. See this thread: http://thread.gmane.org/gmane.comp.gnu.mingw.user/32432/

As I do not know which Windows versions are officially supported by MSYS, create_symlink function is not implemented as it is available since Vista. Original patch had CreateSymbolicLink dynamically loaded based on Windows version, but since CreateHardLink is used directly (and IsWow54Process elsewhere as well), lets leave actual implementation once this question is resolved.

msys_symlink was called from path.cc with arguments swapped, so fix both caller and callee. At least this part of patch would deserve merging.

Also it turned out, that symlinks are actually easy to implement, but learning the rest of MSYS about their existence is much harder. There is some code trying to resolve .lnk files as well as code which stores symlink info into NTFS extended attributes and also code implementing links with BackupWrite. Everything probably inherited from Cygwin. How should it be done in MSYS?

Interestingly enough BackupWrite approach uses fallback semantics. If 'symlink' cannot be created, file copy is performed. On Linux link syscall returns -EPERM on FAT filesystem (and I would expect -ENOTSUPP). I'd rather avoid silent fallback, or make that optional.

More comments later, there is already enough questions asked.

Thank you,
ladis

----------------------------------------------------------------------

>Comment By: Earnie Boyd (earnie)
Date: 2010-08-17 08:33

Message:
>Given this, is there any longer a need for the symlink-as-copy fallback to
be on by default on MSYS?

Testing is the only way to know but I would guess it would still be
needed.

----------------------------------------------------------------------

Comment By: Ladislav Michl (ladis)
Date: 2010-08-17 07:29

Message:
> Please see the source in autoload.cc. If you add a CreateSymbolicLink
> declaration in there, you can call it in the code directly. You are
> actually calling a stub code that will load the appropriate library by
> demand, and either forward your call to the real function, or return
FALSE
> if it doesn't exists in your version of Windows.

Very interesting, usefull and explains my questions. I added
CreateSymbolicLink to the w32api (patch attached) and modified
autoload.cc accordingly.

> Please swap the order of the arguments of the symlink function itself as
> well. According to the POSIX standard, the contents of the symbolic link
> (frompath) is the first argument. To see it, type "symlink" on the
search
> box in the following page:
> http://www.opengroup.org/onlinepubs/009695399/

http://www.opengroup.org/onlinepubs/009695399/functions/symlink.html

    int symlink(const char *path1, const char *path2);

    The symlink() function shall create a symbolic link called path2 that
    contains the string pointed to by path1 ( path2 is the name of the
    symbolic link created, path1 is the string contained in the symbolic
link).

Well, my understanding is quite opposite and as implemented in
patch - content of the symbolic link (topath) is the first argument.
As symlink name points to symlink content.

$ touch file
$ ln -s file link
$ ls -l
-rw-r--r--   1 ladis ladis    0 2010-08-17 14:52 file
lrwxrwxrwx   1 ladis ladis    4 2010-08-17 14:52 link -> file

in patched MSYS with debug enabled, you get:
$ ln -s file link
msys_symlink (file, link)
create_copy_file (C:\MSYS\test\file, C:\MSYS\test\link)

as you can see, symlink points from 'link' (second argument) to 'file'
(first argument).

> It would be nice to display symlink info in "ls -l".
> Maybe a start would be updating the symlink_info::check method.

I read symlink_info::check method yesterday as well as the rest of that
file
and now after more than 12 hours I'm able to comment it without showing
too much emotions. Ummm... So, the only sane way how to update path.cc
is to downwrite its desired function and then implement it as it keeps all
cygwin logic which has no use for us and make code very hard to
understand and modify (someone already tried to do so). I would
very much appreciate if this could be done by someone more familiar with
MSYS, but in the worst case I can do it myself (it will take me quite some
time).

> I guess you should ensure the following system calls are aware of
> symlinks: stat, lstat and readlink (you can read about them in the POSIX
> standard link I gave above).

Actually the most important is unlink and remove as they are not junctions
aware.

> Autoconf currently has an AC_PROG_LN_S macro which will set the
> $(LN_S) variable to either "ln -s", "ln" or "cp -p", depending on the
> host capabilities. Given this, is there any longer a need for the
> symlink-as-copy fallback to be on by default on MSYS?

I modified the patch to provide some fallback logic, which is currently
disabled and copy method used. Easy to fix once we came to a
conclusion.

----------------------------------------------------------------------

Comment By: Cesar Strauss (cstrauss)
Date: 2010-08-16 23:12

Message:
> As I do not know which Windows versions are officially supported by
> MSYS, create_symlink function is not implemented as it is available
> since Vista. Original patch had CreateSymbolicLink dynamically loaded 
> based on Windows version, but since CreateHardLink is used directly 
> (and IsWow54Process elsewhere as well), lets leave actual
> implementation once this question is resolved.

Please see the source in autoload.cc. If you add a CreateSymbolicLink
declaration in there, you can call it in the code directly. You are
actually calling a stub code that will load the appropriate library by
demand, and either forward your call to the real function, or return FALSE
if it doesn't exists in your version of Windows.

> msys_symlink was called from path.cc with arguments swapped, so fix
both
> caller and callee. At least this part of patch would deserve merging.

Please swap the order of the arguments of the symlink function itself as
well. According to the POSIX standard, the contents of the symbolic link
(frompath) is the first argument. To see it, type "symlink" on the search
box in the following page:
http://www.opengroup.org/onlinepubs/009695399/

> Also it turned out, that symlinks are actually easy to implement, but
> learning the rest of MSYS about their existence is much harder. There
> is some code trying to resolve .lnk files as well as code which stores
> symlink info into NTFS extended attributes and also code implementing
> links with BackupWrite. Everything probably inherited from Cygwin. How
> should it be done in MSYS?

It would be nice to display symlink info in "ls -l".
Maybe a start would be updating the symlink_info::check method.
I guess you should  ensure the following system calls are aware of
symlinks: stat, lstat and readlink (you can read about them in the POSIX
standard link I gave above).

Earnie writes:
> If the OS and/or file system does not support CreateSymbolicLink then a
> copy of the file is correct for what we need. Too many configure
scripts
> use ln -s to not have the fallback of copy by default.

Autoconf currently has an AC_PROG_LN_S macro which will set the $(LN_S)
variable to either "ln -s", "ln" or "cp -p", depending on the host
capabilities. Given this, is there any longer a need for the
symlink-as-copy fallback to be on by default on MSYS?

Regards,
Cesar

----------------------------------------------------------------------

Comment By: Earnie Boyd (earnie)
Date: 2010-08-16 08:09

Message:
> Also it turned out, that symlinks are actually easy to implement, but
> learning the rest of MSYS about their existence is much harder. There
> is some code trying to resolve .lnk files as well as code which
> stores symlink info into NTFS extended attributes and also code
> implementing links with BackupWrite. Everything probably inherited
> from Cygwin. How should it be done in MSYS?

Yes, the .lnk code was inherited from Cygwin.  I disabled it because
Windows native programs do not understand .lnk files.  For directories I
would suggest using Reparse (Junction) Points.  For files I would suggest
that if the OS and file system supports it we use the CreateSymbolicLink. 
If the OS and/or file system does not support CreateSymbolicLink then a
copy of the file is correct for what we need.  Too many configure scripts
use ln -s to not have the fallback of copy by default.

Cesar Strauss is the current maintainer.  He may have more to say.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=302435&aid=3046195&group_id=2435

[MinGW-notify] [ mingw-Patches-3046195 ] Native symlinks

A native Windows port of the GNU Compiler Collection (GCC)

[MinGW-notify] [ mingw-Patches-3046195 ] Native symlinks