#2062 glob() may fold case in command args with no globbing tokens.

[Jan Nijtmans]

Compile the following little program:

#include <stdio.h>
int main(int argc, char **argv){
  printf("argv1: %s\n", argv[1]);
}

run it:

>.\a.exe version
argv1: VERSION

This effects fossil when compiled with mingw:

>.\fossil version
.\fossil: unknown command: VERSION
.\fossil: use "help" for more information

[Jan Nijtmans]

Additional information: It turns out that the current directory contained a file named "VERSION". Apparently this is a feature: If argv[1] looks like an existing filename, but it differs from it in case only, the argument is changed to match the file name exactly.

Is there any compiler option or another way to switch this feature off? Not all
programs have a filename as first commandline argument.......

[Keith Marshall]

This is a consequence of file name globbing, which applies by default for all unquoted arguments on the command line, coupled with the case-insensitive nature of Windows file system, which causes a glob expansion of the unquoted keyword "version" to match the existing file named "VERSION".

You can disable globbing for any single argument by enclosing it in quotes; (double quotes by default, but _mingw.h describes an option to also accept single quotes). Alternatively, you can define the public symbol _CRT_glob, (an integer with C binding), with a value of zero, to disable globbing entirely. There are also other options, described in _mingw.h, which you may wish to consider; of particular interest in this case, may be:

int _CRT_glob = __CRT_GLOB_USE_MINGW__ + __CRT_GLOB_CASE_SENSITIVE__;

which would still allow globbing, using the MinGW algorithm, but would require file names to match case-sensitively, so that unquoted "version" would not match a file named "VERSION".

[Jan Nijtmans]

If I compile fossil with mingw-w64 or msc (with globbing enabled),
this behavior is different: globbing only makes sense when
the argument at least contains the '*' character. My
suggestion would be to let mingw do command line
globbing the same as MSVC does it (even though I
cannot find documentation on it). Experimenting,
however, with a MSVC build of fossil:

.\fossil version
This is fossil version 1.27......
.\fossil ver*ion
.\fossil unknown command: VERSION
.\fossil use "help" for more information

See: [http://msdn.microsoft.com/en-us/library/8bch7bkk.aspx]
(This describes how globbing is enabled in a MSVC build, it
doesn't document the exact globbing algorithm).

It appears that the mingw command line globbing is different
from how Windows "operating system commands" do it.

[Keith Marshall]

Sometime during the lifetime of of WinXP or Vista, Microsoft changed their globbing algorithm. In the process, they broke it, beyond any hope of redemption. (Try to get a globbing token into a command line, with globbing enabled. You can't. Quoting, which should work, doesn't do the trick. Microsoft seem to think this broken crap is actually desirable behaviour, and they have refused to fix it).

At the request of some of our users, we provided our own algorithm to correct the Microsoft breakage. Other GCC distributors for Windows may not use our runtime, so will not benefit from our improved globbing algorithm; that isn't any justification for us to not offer the enhancement. If you don't want to use it in your application, set _CRT_glob to one to revert to Microsoft's broken globbing, or to zero to disable it altogether, (which IIRC is the default for MSVC).

That said, our new algorithm does have an internal (static) function to test for the presence of globbing tokens in the pattern it is evaluating. The behaviour you report suggests that this function may be bypassed in some circumstance; this is why I'd like you to at least try the

int _CRT_glob = __CRT_GLOB_USE_MINGW__ + __CRT_GLOB_CASE_SENSITIVE__;

work around, to confirm my diagnosis before I embark on what may turn out to be a wild goose chase.

[Jan Nijtmans]

Thanks! I tried your suggested:

int _CRT_glob = __CRT_GLOB_USE_MINGW__ + __CRT_GLOB_CASE_SENSITIVE__;

and it indeed helps. However, for compatibility with earlier releases,
"int _CRT_glob = 1" gives the least surprises, no matter how broken it might be.

I would prefer a _CRT_glob option which only uses globbing when the argument
contains at least a single globbing character. Without any globbing character,
whether the argument matches an available filename or not, it doesn't make sense
to modify the argument and skipping the globbing will result in a faster startup
normally. Doing globbing always is what caused the surprising behavior here.
Anyway I understand what's going on now.

Many thanks!

[Keith Marshall]

I tried your suggested:

int _CRT_glob = __CRT_GLOB_USE_MINGW__ + __CRT_GLOB_CASE_SENSITIVE__;

and it indeed helps.

Thanks. That confirms that the problem is that the globbing algorithm is indeed bypassing a call to its static is_glob_pattern() function, at some point.

However, for compatibility with earlier releases, "int _CRT_glob = 1" gives the least surprises, no matter how broken it might be.

Of course, because that's exactly what previous versions did. The problem is that, from Vista onwards, that option denies you any chance of injecting a literal globbing token into any command line argument, because Microsoft broke the effect that quoting should deliver.

I would prefer a _CRT_glob option which only uses globbing when the argument contains at least a single globbing character.

It shouldn't need any _CRT_glob option; it should work that way regardless. This is a bug; it requires investigation. Earnie, since I wrote glob.c, feel free to reassign this to me.

Without any globbing character, whether the argument matches an available filename or not, it doesn't make sense to modify the argument ...

Except when the modification results from matching a globbing token, (but that's pretty much what you've said).

and skipping the globbing will result in a faster startup normally.

To some extent, probably; each argument still needs to be parsed character by character, through is_glob_pattern(), to check for unquoted globbing tokens, but we should be able to avoid calls to readdir(), (implemented using _findfirst() and _findnext() on Windows).

[Keith Marshall]

The attached patch seems to fix this issue for me. Can you test it, please, with _CRT_glob set to its default value of 2?

[Jan Nijtmans]

Thanks for fixing it. I don't have a build environment for MinGW, but I'll check as soon as there is a new release or snapshot.

In fossil, the problem is solved already:
http://fossil-scm.org/index.html/info/f2ce2f80f43edd4e

[Keith Marshall]

Earnie,

Since the OP prefers to wait for a release, shall I go ahead and commit? (On 4.1-dev, presumably)?

[Earnie Boyd]

Yes, go ahead with the 4.1-dev commit.

[Keith Marshall]

Done.

[Earnie Boyd]

Pending release of 4.1.

[Keith Marshall]

Technically, this is a bug fix. Why should it's application be deferred until 4.1? Surely, it would be better to get it into 4.0, ASAP.

[Keith Marshall]

Irrespective of any discussion regarding inclusion in mingwrt-4.0 vs. mingwrt-4.1, all mingwrt-4.x releases are dead and buried; this is fixed in all current mingwrt-3.x and mingwrt-5.x releases.