Menu
▾
▴
[Mingw-notify] [ mingw-Bugs-1791494 ] find not reporting all expected matches
|
From: SourceForge.net <no...@so...> - 2007-09-14 11:53:05
|
Bugs item #1791494, was opened at 2007-09-10 10:31 Message generated for change (Comment added) made by keithmarshall You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=102435&aid=1791494&group_id=2435 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: msys Group: None Status: Open Resolution: None Priority: 1 Private: No Submitted By: Keith Marshall (keithmarshall) Assigned to: Cesar Strauss (cstrauss) Summary: find not reporting all expected matches Initial Comment: Using up-to-date snapshots (I think), for MSYS-1.0.11: $ cat /etc/fstab d:/usr/mingw /mingw d:/usr/local /usr/local x:/control /db d:/opt /opt e:/ /usb /dev /dev $ mount D:\usr\MSYS-1.0.11 on / type user (binmode,noumount) D:\usr\MSYS-1.0.11 on /usr type user (binmode,noumount) d:\usr\local on /usr/local type user (binmode) d:\usr\mingw on /mingw type user (binmode) x:\control on /db type user (binmode) C:\TEMP on /tmp type user (binmode,noumount) d:\opt on /opt type user (binmode) \dev on /dev type user (binmode) c: on /c type user (binmode,noumount) d: on /d type user (binmode,noumount) e: on /usb type user (binmode) [...] $ (cd /; pwd -W) D:/usr/MSYS-1.0.11 $ find /d -iname msys.bat 2>/dev/null /d/MSYS/1.0/msys.bat $ find /d/usr -iname msys.bat 2>/dev/null /d/usr/MSYS-1.0.11/home/keith/sandbox/msys/msys/store/noarch/bin/msys.bat /d/usr/MSYS-1.0.11/msys.bat /d/usr/tmp/msys/msys.bat The first find command reports only the msys.bat from an original MSYS-1.0.10 installation. Why does it not also search into the D:/usr directory, (a real directory), to locate the three additional files? FWIW: $ msysinfo msysinfo-1.3: Send this to the MSYS support list: MSYS 1.0.11(0.46/3/2) 2006-08-07 12:54 i686 unknown; targ=MINGW32 GNU bash, version 3.1.0(3)-release (i686-pc-msys); ENV=.profile GNU Make version 3.79.1,Built for i686-pc-msys; MAKE_MODE=unix gcc.exe (GCC) 3.4.5 (mingw special); targ=MINGW32 GNU ld version 2.17.50 20060824 2488 2006-08-08 18:33:24.000000000 +0100 /bin/msys-1.0.dll 2064 2003-01-02 08:05:27.000000000 +0000 /bin/msysltdl-3.dll 5680 2004-04-30 23:15:05.000000000 +0100 /bin/make.exe 600 2006-01-18 20:06:00.000000000 +0000 /mingw/bin/gcc.exe 6237 2006-08-25 06:35:47.000000000 +0100 /mingw/bin/ld.exe HOME=/home/keith Sysname=MINGW32_NT-5.0 OSTYPE=msys TERM=cygwin PATH=.:/usr/local/bin:/mingw/bin:/bin:/c/WINNT/system32:/c/WINNT :/c/WINNT/System32/Wbem:/c/Scripts/DLLs:/c/Program Files/WinZip/ :/c/Program Files/Microsoft Office/Office:/c/Program Files/Micro soft Office/Office10:/c/Program Files/ManageSoft/Common $ ls -tx /home/keith .bash_history .viminfo approot/ sandbox/ gdbtk.ini junk/ .profile src/ dumpargs.c cvsroot/ xx.exe* xx.f kits/ foo/ scancode.txt foobar/ acbug-2.61 scandir.c mail.txt top-post.txt junk20070523/ brendon/ hello/ tmp/ user_number_input.o user_number_input.c user_number_input.h myfloat.o myfloat.c myfloat.h sbox/ .mingwPORT/ noinst/ .ssh/ foo.exe* hang.exe* battest.bat* access.exe* printenv.exe* .profile~ ---------------------------------------------------------------------- >Comment By: Keith Marshall (keithmarshall) Date: 2007-09-14 11:53 Message: Logged In: YES user_id=823908 Originator: YES $ find /d 2>find.err >find.log $ $ grep msys.bat find.log /d/MSYS/1.0/msys.bat $ tail find.log /d/RECYCLER/S-1-5-21-117609710-764733703-839522115-39808/Dd7.htm /d/RECYCLER/S-1-5-21-117609710-764733703-839522115-39808/Dd8.kf /d/RECYCLER/S-1-5-21-117609710-764733703-839522115-39808/Dd9.kf /d/RECYCLER/S-1-5-21-117609710-764733703-839522115-39808/desktop.ini /d/RECYCLER/S-1-5-21-117609710-764733703-839522115-39808/INFO2 /d/SYSTEM /d/System Volume Information /d/Total /d/usr Hmm. Doesn't seem to be traversing into /d/SYSTEM, "/d/System Volume Information" or /d/Total; let's see... $ ls -a /d/SYSTEM . .. Ok, this one's empty... $ ls -a /d/System\ Volume\ Information/ ls: reading directory /d/System Volume Information/: Permission denied ...this one, I cannot access... $ ls -a /d/Total/ . .. ENG_SAPGUI62 HelpDeskInfo PIPC Voloview ...but for this one, I should see a traversal! Let's add something elsewhere... $ touch /opt/packages/msys.bat /d/SYSTEM/msys.bat /d/Total/msys.bat $ find /d 2>find.err >find.log $ grep msys.bat find.log /d/MSYS/1.0/msys.bat /d/opt/packages/msys.bat /d/SYSTEM/msys.bat Ah ha! Now we are seeing something interesting; we've found the additions in /d/opt/packages, (recall that /d/opt is mounted on /opt), and in /d/SYSTEM, but still nothing in /d/Total or in /d/usr... $ tail find.log /d/RECYCLER/S-1-5-21-117609710-764733703-839522115-39808/Dd7.htm /d/RECYCLER/S-1-5-21-117609710-764733703-839522115-39808/Dd8.kf /d/RECYCLER/S-1-5-21-117609710-764733703-839522115-39808/Dd9.kf /d/RECYCLER/S-1-5-21-117609710-764733703-839522115-39808/desktop.ini /d/RECYCLER/S-1-5-21-117609710-764733703-839522115-39808/INFO2 /d/SYSTEM /d/SYSTEM/msys.bat /d/System Volume Information /d/Total /d/usr ...and indeed, we now see a traversal into /d/SYSTEM, but still not into "/d/System Volume Information", (which we wouldn't expect anyway, because of its permissions), nor into /d/Total or /d/usr. I did wonder if, maybe, the name `usr' might be significant, (because `/usr' is special). I think that's another red herring, (it wouldn't explain the failure to traverse /d/Total), but pursuing it did turn up something interesting: If I rename it to `kdm'... $ find /d 2>/dev/null >find.log $ grep msys.bat find.log /d/kdm/MSYS-1.0.11/home/rgbl0264/sandbox/msys/msys/store/noarch/bin/msys.bat /d/kdm/MSYS-1.0.11/msys.bat /d/kdm/tmp/msys/msys.bat /d/MSYS/1.0/msys.bat /d/opt/packages/msys.bat /d/SYSTEM/msys.bat Eureka! Those expected files have miraculously appeared. Even if I rename as `opu', (moving it lexically below the /opt mount point... $ find /d 2>/dev/null >find.log $ grep msys.bat find.log /d/MSYS/1.0/msys.bat /d/opt/packages/msys.bat /d/opu/MSYS-1.0.11/home/rgbl0264/sandbox/msys/msys/store/noarch/bin/msys.bat /d/opu/MSYS-1.0.11/msys.bat /d/opu/tmp/msys/msys.bat /d/SYSTEM/msys.bat ...but, if I move it to, say `tsr'... $ find /d 2>/dev/null >find.log $ grep msys.bat find.log /d/MSYS/1.0/msys.bat /d/opt/packages/msys.bat /d/SYSTEM/msys.bat ...they vanish again! It seems that the pivotal entity is that "/d/System Volume Information" directory, which I can see, but can't see into; any directory appearing lexically above it is traversed; anything below is not. So what's special about it? Access permissions? Let's see: $ cat find.err find: /d/8957ac83a281329dbadd: Permission denied find: /d/csc: Permission denied find: /d/System Volume Information: Permission denied If it were permissions alone, then I'd expect no traversal of any directory appearing lexically below /d/8957ac83a281329dbadd, (which is first of all present, so I'd expect no hits at all). $ head find.log /d /d/8957ac83a281329dbadd /d/csc /d/DOCUMENT /d/lordoc /d/lordoc/.anin-man.ms.swp /d/lordoc/81pc251.ms /d/lordoc/81pc251.pdf /d/lordoc/91F1-firing.html /d/lordoc/anin-man.pdf So, not permissions alone then; what else stands out? Spaces in the path name, perhaps? $ mkdir "/d/Path With Spaces" $ touch !$/msys.bat touch "/d/Path With Spaces"/msys.bat $ find /d 2>/dev/null > find.log $ grep msys.bat find.log /d/MSYS/1.0/msys.bat /d/opt/packages/msys.bat /d/Path With Spaces/msys.bat /d/SYSTEM/msys.bat Nope. It finds "/d/Path With Spaces/msys.bat" ok. Maybe a combination of the two? I don't know. I'm out of ideas. Over to you. ---------------------------------------------------------------------- Comment By: Cesar Strauss (cstrauss) Date: 2007-09-13 16:39 Message: Logged In: YES user_id=1369729 Originator: NO Please try: find /d > find.txt grep msys.bat find.txt tail find.txt Do not send find.txt, just the output from the commands above. Let's see where find stops... ---------------------------------------------------------------------- Comment By: Keith Marshall (keithmarshall) Date: 2007-09-13 13:27 Message: Logged In: YES user_id=823908 Originator: YES Lest we become distracted by the /usr/local issue, it too appears to be a red herring: $ umount /usr/local $ find /d -iname msys.bat 2>/dev/null /d/MSYS/1.0/msys.bat $ find /d/usr -iname msys.bat 2>/dev/null /d/usr/MSYS-1.0.11/home/keith/sandbox/msys/msys/store/noarch/bin/msys.bat /d/usr/MSYS-1.0.11/msys.bat /d/usr/tmp/msys/msys.bat "`umount'?", did I hear you ask? I'll open a feature request to discuss it in more depth, but I've attached it for completeness; it's a simple script to filter entries out of /etc/fstab, so /usr/local isn't mounted now: $ mount D:\usr\MSYS-1.0.11 on / type user (binmode,noumount) D:\usr\MSYS-1.0.11 on /usr type user (binmode,noumount) d:\usr\mingw on /mingw type user (binmode) x:\control on /db type user (binmode) C:\TEMP on /tmp type user (binmode,noumount) d:\opt on /opt type user (binmode) \dev on /dev type user (binmode) c: on /c type user (binmode,noumount) d: on /d type user (binmode,noumount) e: on /usb type user (binmode) f: on /f type user (binmode,noumount) g: on /g type user (binmode,noumount) i: on /i type user (binmode,noumount) j: on /j type user (binmode,noumount) l: on /l type user (binmode,noumount) t: on /t type user (binmode,noumount) u: on /u type user (binmode,noumount) v: on /v type user (binmode,noumount) w: on /w type user (binmode,noumount) x: on /x type user (binmode,noumount) z: on /z type user (binmode,noumount) (And, yes, I do have a corresponding `mount' script, but that's for the feature request). File Added: umount ---------------------------------------------------------------------- Comment By: Keith Marshall (keithmarshall) Date: 2007-09-13 12:43 Message: Logged In: YES user_id=823908 Originator: YES > There may be another introduced bug but I do remember that older > versions of find would exhibit this situation when the physical > directory didn't exist for the mount point. Sorry Earnie, but I think this is a red herring. My MSYS installation root is: $ (cd /; pwd -W) D:/usr/MSYS-1.0.11 Note that D: is a real partition, (probably extended; I didn't set the box up, and it's too much hassle to check at the moment), on a local hard drive. D:\usr\MSYS-1.0.11 (obviously) exists as a physical path, in the Woe32 domain. D:\usr\local also exists as a physical path, and is mapped to /usr/local via an /etc/fstab reference, as an MSYS virtual path; (the intention here was to share /usr/local between MSYS-1.0.10 and MSYS-1.0.11 -- not concurrently of course; that did appear to work at one time, but now MSYS-1.0.10 dies on start-up, so it's become rather redundant). But back on topic: if I now start up cmd.exe, and D:\> cd \usr\msys-1.0.11 D:\usr\MSYS-1.0.11> mkdir d D:\usr\MSYS-1.0.11> mkdir d\usr D:\usr\MSYS-1.0.11> mkdir d\usr\local D:\usr\MSYS-1.0.11> mkdir usr D:\usr\MSYS-1.0.11> mkdir usr\local and back in MSYS, after closing all MSYS sessions and starting anew, it makes not a blind bit of difference: $ find /d -iname msys.bat 2>/dev/null /d/MSYS/1.0/msys.bat $ find /d/usr -iname msys.bat 2>/dev/null /d/usr/MSYS-1.0.11/home/keith/sandbox/msys/msys/store/noarch/bin/msys.bat /d/usr/MSYS-1.0.11/msys.bat /d/usr/tmp/msys/msys.bat Or, am I missing something? ---------------------------------------------------------------------- Comment By: Earnie Boyd (earnie) Date: 2007-09-13 11:35 Message: Logged In: YES user_id=15438 Originator: NO There may be another introduced bug but I do remember that older versions of find would exhibit this situation when the physical directory didn't exist for the mount point. ---------------------------------------------------------------------- Comment By: Keith Marshall (keithmarshall) Date: 2007-09-13 11:09 Message: Logged In: YES user_id=823908 Originator: YES Thanks for following up, Cesar. > Please locate an "oldfind.exe" program in your /bin directory. Hmm. It's not there; guess I missed that update :-( ... > If you do not have it, grab it from the findutils package in the > Snapshot download category on SourceForge. Ok, got it now. > Then: > 1) rename "find.exe" to "ftsfind.exe" > 2) rename "oldfind.exe" to "find.exe" > 3) retry the failed command. > > Does it work? 'Fraid not. Seems to search for about 50% longer, but still only finds the one file in /d/MSYS/1.0 Not a rigorous benchmark, I know, but... $ cp findutils-4.3.0/bin/find.exe /bin $ find --version GNU find version 4.3.0 Features enabled: LEAF_OPTIMISATION FTS $ date +%T-%N; find /d -iname msys.bat 2>/dev/null; date +%T-%N 11:48:56-095000000 /d/MSYS/1.0/msys.bat 11:49:01-048000000 (Repeated 4 times; consistent at ~4.9 sec). $ cp findutils-4.3.0/bin/oldfind.exe /bin/find.exe $ find --version GNU find version 4.3.0 Features enabled: LEAF_OPTIMISATION $ date +%T-%N; find /d -iname msys.bat 2>/dev/null; date +%T-%N 11:51:20-534000000 /d/MSYS/1.0/msys.bat 11:51:28-097000000 (Repeated 4 times; consistent at ~7.5 sec). ---------------------------------------------------------------------- Comment By: Cesar Strauss (cstrauss) Date: 2007-09-12 23:41 Message: Logged In: YES user_id=1369729 Originator: NO Hi Keith, Please locate an "oldfind.exe" program in your /bin directory. If you do not have it, grab it from the findutils package in the Snapshot download category on SourceForge. Then: 1) rename "find.exe" to "ftsfind.exe" 2) rename "oldfind.exe" to "find.exe" 3) retry the failed command. Does it work? In findutils-4.3, two versions of "find" are always built and installed. "find.exe" uses a "new" file hierarchy traversal method (FTS), which appears to be incompatible with MSYS. "oldfind.exe" retains the "old" method, which works in MSYS. The simplest workaround is using the --without-fts at configure time. It causes "find.exe" to use the old traversal method. ---------------------------------------------------------------------- Comment By: Cesar Strauss (cstrauss) Date: 2007-09-11 00:39 Message: Logged In: YES user_id=1369729 Originator: NO The test case should read: $ touch /c/aardvark $ mkdir /c/aatest $ mkdir /c/aatest/a # <- forgot this line $ echo "c:/aatest /xyz" >> /etc/fstab $ find /c /c /c/aardvark /c/aatest /c/aatest/a [stops right here] ---------------------------------------------------------------------- Comment By: Cesar Strauss (cstrauss) Date: 2007-09-11 00:34 Message: Logged In: YES user_id=1369729 Originator: NO I was able to reproduce it, somewhat. For some reason, ``find'' is stopping right after traversing a path listed in the mount table. For instance: $ touch /c/aardvark $ mkdir /c/aatest $ echo "c:/aatest /xyz" >> /etc/fstab $ find /c /c /c/aardvark /c/aatest /c/aatest/a [ends right here] Earnie's suggestion of creating a physical mount point directory didn't change the result. In the following, note that ``find'' appears to traverse the file system in case-insensitive alphabetical order. In Keith's first case (find /d), it must be stopping after traversing d:/opt (it's in his mount table). Before stopping, it traverses d:/MSYS and finds the msys.bat in there (MSYS comes earlier than opt in traversal order). It doesn't reach d:/usr because d:/opt comes before d:/usr in search order. I can't explain why Keith's second case succeeds, however. It should stop in d:/usr/local before reaching d:/usr/MSYS-1.0.11, if my theory was correct. In any case, I think what I have is enough to start debugging. ---------------------------------------------------------------------- Comment By: Earnie Boyd (earnie) Date: 2007-09-10 21:08 Message: Logged In: YES user_id=15438 Originator: NO I believe this goes back to the issue of the mount point directory not physically existing but will allow the mount to succeed because Windows users are not used to UNIX semantics. If you ``mkdir /d/usr/MSYS-1.0.11/d'' the ``find /d -iname ...'' command will be much happier about finding the things in /d/usr. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=102435&aid=1791494&group_id=2435 |