From: SF/projects/mingw n. l. <min...@li...> - 2011-04-26 11:57:24
|
Bugs item #3072100, was opened at 2010-09-20 19:13 Message generated for change (Comment added) made by keithmarshall You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=102435&aid=3072100&group_id=2435 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: MSYS Group: Known Feature Status: Open Resolution: None Priority: 5 Private: No Submitted By: Bob Friesenhahn (bfriesen) Assigned to: Nobody/Anonymous (nobody) Summary: MSYS make is 60X slower than before Initial Comment: MSYS make 3.82 is now 60X slower than earlier make programs. When building GraphicsMagick, it takes 2.5 minutes before make does any work or decides there is nothing to be done. After make has decided what to do, the build proceeds normally. It normally takes a couple of seconds before make decides what needs to be done. With a Cygwin installation on the same machine using the same source files and same build parameters, make decides there is nothing to be done in about two seconds. The build is done with the source files accessed via CIFS to a fast server and object files written locally. Using 'make -d' shows that there are about 340K rules which are tested, without any noticeable pauses or slowness due to server access. Charles Wilson tells me that make now tests various permutations of upper and lower case in order to handle filesystems which don't preserve case. This approach would certainly dramatically multiply the number of filesystem acceses and may explain the dramatic slowdown. I also notice that make 3.82 tests about twice as many implicit rules than make 3.81 does. ---------------------------------------------------------------------- >Comment By: Keith Marshall (keithmarshall) Date: 2011-04-26 11:57 Message: case preserving != case sensitive The windows file system is case insensitive, pretending otherwise can have serious, (and for all practical purposes, insoluble), consequences. Pattern rules such as %.$(OBJEXT): %.c will fail dramatically, if the compiler doesn't strictly honour the case preserving concept; even for an 8.3 file name, the number of permutations of this rule which must be written, if you try to pretend that case preserving == case sensitive becomes prohibitive. YOU may not have encountered this issue, but I have; indeed, it is the fundamental reason why cygwin was rejected out of hand, for porting a legacy DJGPP application, in my day job; (cygwin's make promotes the case preserving == case sensitive fiction, as did early MSYS builds). Granted, problems are more likely to arise with legacy MS-DOS applications, than with anything developed from scratch, for more modern WinNT based platforms. When I first raised the issue, several years ago, it was a serious problem for me; the solution was to ensure that make was built with its CASE_INSENSITIVE_FILESYSTEM option enabled. It would be less of an issue for me today, because I no longer have to maintain that legacy application. However I still believe it is important that we continue to offer a make which does respect the true case insensitive nature of the file system. When we first addressed the issue, we offered two variants of make; the default was built with CASE_INSENSITIVE_FILESYSTEM, (which I still believe to be the safest option), and the alternative called csmake was offered for those who wished to promulgate the case preserving == case sensitive fiction. Later, Cesar devised the current hybrid build, which attempts to be all things to all men. It seemed like a good idea, at the time. Maybe the speed issue is just too serious a detraction, and we should revert to the two alternative choices, leaving users to decide which is most suitable for them to install as make.exe ---------------------------------------------------------------------- Comment By: Earnie Boyd (earnie) Date: 2011-04-25 17:26 Message: Case preserving case insensitive file name operation is the issue. Yes, it preserves case but files named FOO, Foo, fOO, fOo and foo will all be found regardless of the reference. The patch allows for non-matching targets to be considered so that the target FOO doesn't build if foo is found and is up-to-date. Frankly, I have never had an issue with make without the patch. ---------------------------------------------------------------------- Comment By: Bob Friesenhahn (bfriesen) Date: 2011-04-25 14:22 Message: Since Windows is case-preserving, I tend to wonder about the decision for 'make' to be case insensitive by default. It seems like a bad idea to me since those developing projects primarily under MinGW but with a desire to be portable to other systems (e.g. Linux, or even Cygwin) will potentially produce projects which don't work at all due to case issues. My project (GraphicsMagick) is open sourced and soon I will be encouraging users to build it from source on MinGW rather than MSVC. If 'make' seems incredibly slow, I will soon hear complaints. Regardless, I did already add a shell alias in order to improve my personal efficiency. ---------------------------------------------------------------------- Comment By: Earnie Boyd (earnie) Date: 2011-04-25 11:39 Message: I was just trying to eliminate possibilities. I agree with you, IIRC csmake was added as a result of conversations I had made on one of the lists (mingw-users or mingw-dvlpr). Solution to your problem is to set in your ~/.profile file ``alias make=csmake'' and let's be done with the conversation. I would encourage you though to create a wiki post of your findings and the solution to the issue I've given you. Keith and Cesar, do you think we should close as won't fix or do you think we should reverse the decision to include the case insensitive patch as the default make for MSYS and instead of csmake have one that is named cimake? ---------------------------------------------------------------------- Comment By: Bob Friesenhahn (bfriesen) Date: 2011-04-23 17:05 Message: Earnie, I relish data caching, and quite rarely reboot computers. Caching is good. My timings are done with anything that can be cached already cached on the client system The file server has 20GB of RAM so it also does lots of caching. Since the software builds fine using 'csmake' there is no need for any case-insensitive code to be used. ---------------------------------------------------------------------- Comment By: Bob Friesenhahn (bfriesen) Date: 2011-04-23 16:57 Message: To put things in perspective, here is the total package build time from 'csmake -j 4' on the same system: real 0m46.391s user 1m15.990s sys 0m41.554s With csmake, the whole package could build from scratch in the time that it takes for 'make' to visibly start doing anything at all. ---------------------------------------------------------------------- Comment By: Earnie Boyd (earnie) Date: 2011-04-23 16:54 Message: To eliminate the possibility of data caching affecting the timings you need to reboot your computer and enter csmake command first. $ time ls /bin real 0m0.469s user 0m0.030s sys 0m0.154s $ time ls /bin real 0m0.391s user 0m0.030s sys 0m0.093s As you can see the second occurrence uses much less system time. Although I doubt you'll see a big improvement over 2m for make. ---------------------------------------------------------------------- Comment By: Bob Friesenhahn (bfriesen) Date: 2011-04-23 16:41 Message: I should mention that GraphicsMagick uses a non-recursive Automake build configuration so that source file name specifications including a directory component when building. The source files reside on a fast Samba server but the build directory is hosted by the Windows system. ---------------------------------------------------------------------- Comment By: Bob Friesenhahn (bfriesen) Date: 2011-04-23 16:38 Message: I finally got back to this since today I am using MinGW and it is driving me crazy. The 'csmake' program runs vastly faster than the standard MinGW 'make' to decide to do nothing for a GraphicsMagick build: bfriesen@optiplex755 ~/mingw/GM-16-static $ time make make all-am make[1]: Entering directory `/home/bfriesen/mingw/GM-16-static' make[1]: Leaving directory `/home/bfriesen/mingw/GM-16-static' real 2m14.689s user 0m5.295s sys 0m40.890s bfriesen@optiplex755 ~/mingw/GM-16-static $ time csmake csmake all-am csmake[1]: Entering directory `/home/bfriesen/mingw/GM-16-static' csmake[1]: Leaving directory `/home/bfriesen/mingw/GM-16-static' real 0m2.562s user 0m0.858s sys 0m0.921s ---------------------------------------------------------------------- Comment By: Cesar Strauss (cstrauss) Date: 2010-09-22 23:04 Message: Please try the following version of make (csmake) which does not have the case preserving patch: http://prdownloads.sourceforge.net/mingw/csmake-3.81-MSYS-1.0.11-2.tar.bz2?download Does csmake run at the normal speed? Regards, Cesar ---------------------------------------------------------------------- Comment By: Bob Friesenhahn (bfriesen) Date: 2010-09-21 15:01 Message: My error. It is indeed 3.81: $ make --version GNU Make 3.81 Copyright (C) 2006 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. This program built for i686-pc-msys ---------------------------------------------------------------------- Comment By: Cesar Strauss (cstrauss) Date: 2010-09-21 00:30 Message: Dear Bob, Could you send the output of make --version, please? I'm a bit confused, since the latest MSYS version available is still 3.81. See https://sourceforge.net/projects/mingw/files_beta/MSYS/make/ Regards, Cesar ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=102435&aid=3072100&group_id=2435 |