From: Vincent R. <fo...@sm...> - 2010-03-22 23:08:34
Attachments:
msys_symlink.h
msys_symlink.cc
|
Hi, Instead of always criticizing mingw I have decided to contribute and to implement the first step to get support for symbolic link. So you will find attached two files msys_symlink.{h,c} in which I rewrote the symbolic link logic. I made a distinction between a symbolic link to a file and to a folder, I can sump new behavior as shown below : On system not using NTFS filesystem we use the old copy logic symlink_to_file = create_copy_dir; symlink_to_dir = create_copy_file; On system using NTFS and before Windows Vista we use hardlinks for file and recursive copy for folder except if you define an env. var MSYS_ALLOW_JUNCTION. In this case we use junction(also called reparse point). I plan to write a shell extension inspired by TortoiseSVN that is able to display junctions correctly on Windows 2000/XP and after it will be possible to enable junction by default. Finally on system >= Windows Vista if we have admin permissions we use native symbolic link (through the use of CreateSymbolicLinkA loaded dynamically) and if not the case we go back to junction for dir and hard link for file. If you want to test native symbolic link, you can start a shell by choosing "Running as administrator". As I told it's a first step and now there is a lot of work to make msys aware of symbolic link(reparse point and native symlink). If people wants to contribute they should have a look at source\winsup\cygwin\path.cc and source\winsup\cygwin\syscalls.cc. Anyway this patch fix a memory leak in msys_symlink.cc:230 return RecursiveCopy (w_frompath.get_win32 (), w_topath.get_win32 (), origpath); ... done: #if DO_CPP_NEW delete sb_frompath; delete sb_topath; #else free (sb_frompath); free (sb_topath); #endif return res; } where it was returning without deallocating sb_frompath and sb_topath. You can find new msysCore dll(based on last msysCore) here : http://www.smartmobili.com/Downloads/msysCore-1.0.14-2-symlink-bin.tar.lzma REMARK : to implement junctions I had to store path as unicode but msys didn't allow to use wcslen, wcscpy, wcscat so finally I have provided implementations. TODO : - see if we can get rid of admin permissions to create native symlink by playing with NTFS ioctl as we already do with junctions. - write some comments that will allow to optimize this whole code - Write a shell extension on Windows 2000/XP |
From: Zouzou <int...@12...> - 2010-03-23 00:28:49
|
i believe it is MSYS you are trying to improve, not MinGW, right? do you know of Cygwin <http://www.cygwin.com/>? it seems like you are basically trying to turn your MSYS into a pseudo-Cygwin environment, in which case it might be better to just go with Cygwin. they have symlinks and friends pretty well implemented. Zouzou |
From: Cesar S. <ces...@gm...> - 2010-03-24 01:48:54
|
Zouzou wrote: > it might be better to just go with Cygwin. they have symlinks and > friends pretty well implemented. I think there is an important difference: by using native symlinks and junctions, standard Windows applications are able to open and traverse these symlinks. Cygwin symlinks are visible only to Cygwin applications. Regards, Cesar |
From: Cesar S. <ces...@gm...> - 2010-03-24 01:42:11
|
Vincent Richomme wrote: > I have decided to contribute and to > implement the first step to get support for symbolic link. Interesting work. I will try and see if I can download a trial version of newer Windows to try it out. Regards, Cesar |
From: Vincent R. <fo...@sm...> - 2010-03-24 02:37:09
|
wrote: > Vincent Richomme wrote: > >> I have decided to contribute and to >> implement the first step to get support for symbolic link. > > Interesting work. I will try and see if I can download a trial version > of newer Windows to try it out. > I modified the patch a bit and I am fixing some stuff. Will send the new patch soon. |
From: Vincent R. <fo...@sm...> - 2010-03-24 17:20:01
Attachments:
msysCore-symlinks.patch
|
On Wed, 24 Mar 2010 03:36:59 +0100, Vincent Richomme <fo...@sm...> wrote: > wrote: >> Vincent Richomme wrote: >> >>> I have decided to contribute and to >>> implement the first step to get support for symbolic link. >> >> Interesting work. I will try and see if I can download a trial version >> of newer Windows to try it out. >> > > I modified the patch a bit and I am fixing some stuff. > Will send the new patch soon. > Hi, please find a better version of patch. I will comment later because I don't have time right now. Just to sump up now symlinks respect relative path (that was not the case with my first patch). I recommend to test on Windows Vista/7 with admin priviledges and to use cygwin 1.7 or standard command line to see results. |
From: Keith M. <kei...@us...> - 2010-03-24 21:19:09
|
On Wednesday 24 March 2010 02:36:59 Vincent Richomme wrote: > >> I have decided to contribute and to > >> implement the first step to get support for symbolic link. > > > > Interesting work. I will try and see if I can download a trial > > version of newer Windows to try it out. > > I modified the patch a bit and I am fixing some stuff. > Will send the new patch soon. Would it not be useful to open a feature request ticket, (on the SF tracker)? Then you would have a central location for depositing and discussing patches, without them being swamped in general mail list traffic, (where they may be overlooked, or worse still, lost). -- Regards, Keith. |
From: Ladislav M. <la...@li...> - 2010-08-13 20:50:24
|
Vincent Richomme <forumer@...> writes: > Hi, > > please find a better version of patch. Hi, patch is somehow hard to read, comments inline, patch shrunk to make it more readable. A bit consolidated and a bit fixed patch follows. > I will comment later because I don't have time right now. Hmm, several months of silence... Nobody interested in such a cool feature? I'll work on it if there is a chance to get it merged as I pretty much need it. > Just to sump up now symlinks respect relative path (that was not the case > with my first patch). I was unable to make junction work at all and had to use absolute paths. > I recommend to test on Windows Vista/7 with admin priviledges and to use > cygwin 1.7 or standard command line to see results. > > > +/* msys_symlink.c [snip] > +/* > +* MSYS DOESN'T SEEM TO ALLOW TO USE WCHAR FUNCTIONS > +* SO WE REDECLARE SIMPLE ONES > +*/ lstrlenW and friends work for me, changed to use them. ... > + hFile = CreateFileW (wszNewPath, GENERIC_READ | GENERIC_WRITE, 0, 0, > + OPEN_ALWAYS, here we probably want to use FILE_EXISTING ... > +static int > +symlink_env_init (symlink_env_t *symlink_env) ... > + memset(symlink_env, 0, sizeof(symlink_env_t)); > + symlink_env->symlink_to_file = create_copy_dir; > + symlink_env->symlink_to_dir = create_copy_file; I wonder if this code ever worked... ... > + if ((strncmp ((const char*)fsname_buff, "NTFS", 4) == 0)) Here we probably do not want to check for NTFS filesystem type, but rather FILE_SUPPORTS_REPARSE_POINTS flag. Also we do not want to query current directory, but one we are going to make symlink on. > + { > + /* Before Windows Vista */ > + if ((dwMajVer < 6)) > + { > + symlink_env->symlink_to_file = create_hard_link; > + symlink_env->symlink_to_dir = create_junction; > + symlink_env->symlink_to_dir = create_copy_dir; > + } > + else > + { > + /* > + * Check if we have admin permissions > + * TODO : check if we can implement native symlink > + * without admin perms > + */ > + if (has_admin_perms () == TRUE) Here we do not want to check for admin permitions, but rather for SeCreateSymbolicLinkPrivilege, which is assigned only to administrator by default. Anyway, thank you for very nice work. Hopefully one day it will get merged. Best regards, ladis |
From: Ladislav M. <la...@li...> - 2010-08-13 21:00:10
|
Here is updated version of the patch. Some code got dropped as it should be implemented different way and smaller patch makes reviewing easier. Questions: - where to hook code supposed to run only once at dll startup? - what logic should be used to determine whenever native symlinks are allowed or not? (fail or fallback should be an option) - to which header file CreateSymbolicLink prototype and friends belong? Comments and suggestions welcome (especially from Vincent :-)) ladis cvs -z3 -q diff -u -- src\winsup\cygwin\msys_symlink.cc src\winsup\cygwin\msys_symlink.h Index: src/winsup/cygwin/msys_symlink.cc =================================================================== RCS file: /cvsroot/mingw/msys/rt/src/winsup/cygwin/msys_symlink.cc,v retrieving revision 1.7 diff -u -r1.7 msys_symlink.cc --- src/winsup/cygwin/msys_symlink.cc 5 Jan 2010 20:07:20 -0000 1.7 +++ src/winsup/cygwin/msys_symlink.cc 13 Aug 2010 20:18:15 -0000 @@ -48,12 +48,163 @@ #include <stdlib.h> #endif +#if 1 +#include <stdio.h> +#undef debug_printf +#define debug_printf(str,...) fprintf(stderr, str "\n", ##__VA_ARGS__) +#endif + +static int +create_copy_file (char *topath, char *frompath, symlink_env_t *symlink_env) +{ + debug_printf ("create_copy_file (%s, %s)", topath, frompath); + + if (CopyFileA (frompath, topath, FALSE)) + return 0; + + __seterrno (); + return -1; +} + +static int +create_hard_link (char *topath, char *frompath, symlink_env_t *symlink_env) +{ + debug_printf ("create_hard_link (%s, %s)", topath, frompath); + + if (CreateHardLinkA (topath, frompath, NULL)) + return 0; + + __seterrno (); + return -1; +} + +#define FSCTL_SET_REPARSE_POINT 0x900a4 + +static int +create_junction (char *topath, char *frompath, symlink_env_t *symlink_env) +{ + size_t nBytes, Len; + char Buf[MAX_PATH*3]; + wchar_t wszNewPath[MAX_PATH+1]; + wchar_t wszOldPath[MAX_PATH+1]; + wchar_t targetNativeFileName[MAX_PATH+1]; + HANDLE h; + DWORD dwBytes; + PREPARSE_DATA_BUFFER rBuf = (PREPARSE_DATA_BUFFER) Buf; + int res = -1; + + debug_printf ("create_junction (%s, %s)", topath, frompath); + debug_printf ("target path=[%s]", symlink_env->path); + + /* Convert paths into wchar_t */ + MultiByteToWideChar (CP_UTF8, 0, topath, -1, wszNewPath, MAX_PATH); + MultiByteToWideChar (CP_UTF8, 0, frompath, -1, wszOldPath, MAX_PATH); + + /* Build the native target name */ + lstrcpyW (targetNativeFileName, L"\\??\\"); + lstrcatW (targetNativeFileName, wszOldPath); + + Len = lstrlenW (targetNativeFileName); + if ((targetNativeFileName[Len-1] == L'\\') && + (targetNativeFileName[Len-2] != L':')) + { + Len--; + targetNativeFileName[Len] = 0; + } + + if (!CreateDirectoryW (wszNewPath, NULL)) + { + debug_printf ("create_junction: CreateDirectoryW failed"); + __seterrno (); + goto done; + } + + h = CreateFileW (wszNewPath, GENERIC_READ | GENERIC_WRITE, 0, 0, + OPEN_EXISTING, + FILE_FLAG_BACKUP_SEMANTICS | FILE_FLAG_OPEN_REPARSE_POINT, + 0); + if (h == INVALID_HANDLE_VALUE) + { + debug_printf ("create_junction: CreateFileW failed"); + __seterrno (); + goto remove; + } + + memset (Buf, 0, sizeof(*Buf)); + nBytes = Len * sizeof(wchar_t); + /* Build the reparse info */ + rBuf->ReparseTag = IO_REPARSE_TAG_MOUNT_POINT; + rBuf->ReparseDataLength = nBytes + 12; + rBuf->SymbolicLinkReparseBuffer.PrintNameOffset = 0; + rBuf->SymbolicLinkReparseBuffer.SubstituteNameLength = nBytes; + rBuf->SymbolicLinkReparseBuffer.PrintNameOffset = nBytes + sizeof(wchar_t); + rBuf->SymbolicLinkReparseBuffer.PrintNameLength = 0; + lstrcpyW(rBuf->SymbolicLinkReparseBuffer.PathBuffer, targetNativeFileName); + + /* Set the link */ + if (!DeviceIoControl (h, FSCTL_SET_REPARSE_POINT, rBuf, + rBuf->ReparseDataLength + + REPARSE_DATA_BUFFER_HEADER_SIZE, + NULL, 0, &dwBytes, NULL)) + { + __seterrno (); + debug_printf ("create_junction: Error setting junction"); + goto close; + } + + res = 0; +close: + CloseHandle (h); +remove: + if (res) + RemoveDirectoryW (wszNewPath); +done: + return res; +} + +static int +create_symlink (char *topath, char *frompath, symlink_env_t *symlink_env) +{ + HINSTANCE h; + CREATESYMBOLICLINKA pfnCreateSymbolicLinkA; + int res = -1; + + debug_printf ("create_native_symlink (%s, %s)", topath, frompath); + + h = LoadLibraryA ("kernel32.dll"); + if (h) + { + debug_printf ("kernel32.dll loaded"); + pfnCreateSymbolicLinkA = + (CREATESYMBOLICLINKA)GetProcAddress (h, "CreateSymbolicLinkA"); + + if (pfnCreateSymbolicLinkA) + { + if (!pfnCreateSymbolicLinkA ((LPSTR)topath, + (LPSTR)symlink_env->path, + symlink_env->symlink_type)) + { + debug_printf("CreateSymbolicLinkA(%s, %s, %d) failed", + topath, symlink_env->path, + symlink_env->symlink_type); + __seterrno (); + goto done; + } + } + } + + res = 0; +done: + FreeLibrary (h); + return res; +} + /* Create a deep copy of frompath as topath, while avoiding descending in origpath. */ static int -RecursiveCopy (char * frompath, char * topath, const char * origpath) +create_copy_dir (char *topath, char *frompath, symlink_env_t *symlink_env) { #if DO_CPP_NEW struct _WIN32_FIND_DATAA *dHfile = new struct _WIN32_FIND_DATAA; @@ -66,7 +217,7 @@ int topos = strlen (topath); int res = -1; - debug_printf("RecursiveCopy (%s, %s)", frompath, topath); + debug_printf("create_copy_dir (%s, %s)", topath, frompath); /* Create the destination directory */ if (!CreateDirectoryEx (frompath, topath, NULL)) @@ -105,9 +256,10 @@ if (dHfile->dwFileAttributes & FILE_ATTRIBUTE_DIRECTORY) { /* Recurse into the child directory */ - debug_printf("%s <-> %s", frompath, origpath); - if (strcmp (frompath, origpath)) // avoids endless recursion - if (RecursiveCopy (frompath, topath, origpath)) + debug_printf("%s <-> %s", frompath, symlink_env->fullpath); + /* avoid endless recursion */ + if (strcmp (frompath, symlink_env->fullpath)) + if (create_copy_dir (topath, frompath, symlink_env)) goto done; } else @@ -137,6 +289,44 @@ return res; } +static int +symlink_env_init (symlink_env_t *symlink_env) +{ + DWORD dwVer; + int res = -1; + + if (!symlink_env) + goto done; + + memset(symlink_env, 0, sizeof(symlink_env_t)); + symlink_env->symlink_to_file = create_copy_file; + symlink_env->symlink_to_dir = create_copy_dir; + + /* Get OS version */ + dwVer = GetVersion(); + + if (1) + { + if (((DWORD)(LOBYTE(LOWORD(dwVer))) < 6)) /* Before Vista */ + { + debug_printf("symlink_env_init: hardlink/junction"); + symlink_env->symlink_to_file = create_hard_link; + symlink_env->symlink_to_dir = create_junction; + } + else + { + debug_printf("symlink_env_init: symlink"); + symlink_env->has_native_support = 1; + symlink_env->symlink_to_file = create_symlink; + symlink_env->symlink_to_dir = create_symlink; + } + } + + res = 0; +done: + return res; +} + extern "C" int msys_symlink (const char * topath, const char * frompath) @@ -149,6 +339,7 @@ struct stat *sb_topath = (struct stat *) malloc (sizeof (struct stat)); #endif char real_frompath[MAX_PATH]; + symlink_env_t symlink_env; int res = -1; debug_printf("msys_symlink (%s, %s)", frompath, topath); @@ -220,23 +411,34 @@ goto done; } - debug_printf("w32 paths: %s , %s",w_frompath.get_win32 (),w_topath.get_win32 ()); + debug_printf("w32 paths: %s , %s", + w_frompath.get_win32 (),w_topath.get_win32 ()); + + /* From here path have been validated */ + if (symlink_env_init(&symlink_env) != 0) + goto done; + + /* Save original path in case we will use junctions and symlinks + * because then we use the path as it is without getting the + * full normalized path - thus it respects relative path */ + strncpy (symlink_env.path, frompath, MAX_PATH); + strncpy (symlink_env.fullpath, w_topath.get_win32 (), MAX_PATH); if (S_ISDIR (sb_frompath->st_mode)) - /* Start a deep recursive directory copy */ { - char origpath[MAX_PATH]; - strcpy (origpath, w_topath.get_win32 ()); - return RecursiveCopy (w_frompath.get_win32 (), w_topath.get_win32 (), origpath); + symlink_env.symlink_type = kSymlinkDir; + res = symlink_env.symlink_to_dir (w_topath.get_win32 (), + w_frompath.get_win32 (), + &symlink_env); + goto done; } else - /* Just copy the file */ { - if (!CopyFile (w_frompath, w_topath, FALSE)) - { - __seterrno (); - goto done; - } + symlink_env.symlink_type = kSymlinkFile; + res = symlink_env.symlink_to_file (w_topath.get_win32 (), + w_frompath.get_win32 (), + &symlink_env); + goto done; } } res = 0; @@ -249,5 +451,6 @@ free (sb_frompath); free (sb_topath); #endif + return res; } Index: src/winsup/cygwin/msys_symlink.h =================================================================== RCS file: /cvsroot/mingw/msys/rt/src/winsup/cygwin/msys_symlink.h,v retrieving revision 1.1 diff -u -r1.1 msys_symlink.h --- src/winsup/cygwin/msys_symlink.h 13 Jun 2002 21:54:21 -0000 1.1 +++ src/winsup/cygwin/msys_symlink.h 13 Aug 2010 08:54:45 -0000 @@ -11,3 +11,33 @@ #include <sys/cygwin.h> #include "cygerrno.h" +typedef enum _SYMLINK_TYPE +{ + kSymlinkFile = 0, + kSymlinkDir = 1, +} SYMLINK_TYPE; + +struct symlink_env_s; +typedef int (*SYMLINK_TO) (char *topath, char *frompath, struct symlink_env_s *symlink_env); + +typedef struct symlink_env_s +{ + int has_native_support; + void *handle; /* native handle to kernel32.dll */ + void *symbol_addr; /* address of CreateSymbolicLinkA */ + char path[MAX_PATH+1]; /* original path used by junctions/symlinks */ + char fullpath[MAX_PATH+1]; + SYMLINK_TYPE symlink_type; + SYMLINK_TO symlink_to_file; + SYMLINK_TO symlink_to_dir; +} symlink_env_t; + + +/* !!! Should be declared in w32api !!! */ +BOOL WINAPI CreateSymbolicLinkA( + LPSTR lpSymlinkFileName, + LPSTR lpTargetFileName, + DWORD dwFlags +); + +typedef BOOL (WINAPI *CREATESYMBOLICLINKA) (LPSTR, LPSTR, DWORD); |
From: Keith M. <kei...@us...> - 2010-08-13 21:29:09
|
On Friday 13 August 2010 23:59:59 Ladislav Michl wrote: > Here is updated version of the patch. Some code got dropped as it > should be implemented different way and smaller patch makes reviewing > easier. > > Questions: > - where to hook code supposed to run only once at dll startup? > - what logic should be used to determine whenever native symlinks are > allowed or not? (fail or fallback should be an option) > - to which header file CreateSymbolicLink prototype and friends > belong? > > Comments and suggestions welcome (especially from Vincent :-)) Well, I am not Vincent, and the technical lead WRT MSYS integration must come from Cesar Strauss, but I do have one comment... This work is much too valuable, and deserves a deeper level of technical discussion than should be entrusted to this general mailing list; please open a ticket on the patch tracker; see: http://mingw.org/wiki/SubmitPatches so that it doesn't slip under the radar again. -- Regards, Keith. |
From: Earnie <ea...@us...> - 2010-08-15 19:22:22
|
Keith Marshall wrote: > On Friday 13 August 2010 23:59:59 Ladislav Michl wrote: >> Here is updated version of the patch. Some code got dropped as it >> should be implemented different way and smaller patch makes reviewing >> easier. >> >> Questions: >> - where to hook code supposed to run only once at dll startup? >> - what logic should be used to determine whenever native symlinks are >> allowed or not? (fail or fallback should be an option) >> - to which header file CreateSymbolicLink prototype and friends >> belong? >> >> Comments and suggestions welcome (especially from Vincent :-)) > > Well, I am not Vincent, and the technical lead WRT MSYS integration must > come from Cesar Strauss, but I do have one comment... > > This work is much too valuable, and deserves a deeper level of technical > discussion than should be entrusted to this general mailing list; please > open a ticket on the patch tracker; see: > > http://mingw.org/wiki/SubmitPatches > > so that it doesn't slip under the radar again. > Yes, please do. We are definitely interested in the work. -- Earnie -- http://www.for-my-kids.com |
From: Vincent R. <fo...@sm...> - 2010-08-13 22:34:25
|
> Here is updated version of the patch. Some code got dropped as it should be > implemented different way and smaller patch makes reviewing easier. > > Questions: > - where to hook code supposed to run only once at dll startup? > - what logic should be used to determine whenever native symlinks are > allowed or not? (fail or fallback should be an option) > - to which header file CreateSymbolicLink prototype and friends belong? > > Comments and suggestions welcome (especially from Vincent :-)) > > ladis > Hi, I am going on vacation in a few hours so it's not the right time for me to comment. Some credits should go to LRN because he is the one who first gave me the idea to work on this subject. All I can say for now is I tried to contribute a bit but I won't fight for its integration because I am not using msys anymore. Vincent |
From: Ladislav M. <la...@li...> - 2010-08-16 11:34:18
|
On Sat, Aug 14, 2010 at 12:15:32AM +0200, Vincent Richomme wrote: > Hi, > > I am going on vacation in a few hours so it's not the right time for me > to comment. > Some credits should go to LRN because he is the one who first gave me > the idea to work on this subject. > All I can say for now is I tried to contribute a bit but I won't fight > for its integration because I am not using msys anymore. Hi, I hope you won't mind if I try to push towards its merging... Thank you for your work. Submitted to the patch tracker as suggested (ID: 3046195) ladis |