#1621 MSYS bash: quoting does not prevent globbing

MSYS
pending
Earnie Boyd
WSL (5)
Bug
later
Unknown
False
2013-02-11
2012-02-01
Keith Marshall
No

Originally reported by Eli Zaretskii; see http://thread.gmane.org/gmane.comp.gnu.mingw.user/38302

When a globbing character appears within a quoted shell command argument, then glob expansion should not occur. This behaviour may be correctly observed when passing such arguments to shell built-in commands, or MSYS-aware applications, (i.e. those linked with msys-1.0.dll). However, when passing such arguments to MinGW applications, the protective quotes are not preserved, and the globbing characters are inappropriately expanded by the MSVCRT startup code.

There are two issues identified in the original mail thread:

1) When passing arguments to MinGW applications, quoting is correctly applied to ensure that embedded white-space, escapes and double quotes are preserved. The same courtesy needs to be accorded to globbing characters which have not been expanded by the shell itself; the current implementation neglects this requirement. The remedy for this defect is entirely within MSYS remit.

2) Recent versions of MSVCRT itself, apparently by Microsoft design, may not honour quoting as protection from glob expansion. It further appears that Microsoft have refused to correct this regression; thus it may be necessary to develop an augmented MinGW startup capability, eschewing the MSVCRT startup code in favour of our own -- perhaps as a user selectable option -- which will DTRT.

Discussion

1 2 > >> (Page 1 of 2)
  • Earnie Boyd
    Earnie Boyd
    2012-02-01

    A decision of which method should be default needs to be made. I would prefer the MinGW method to be the default. This effort needs coordinated between Cesar and Chris as well as Cygwin and the MinGW-64 projects. We want to ensure consistent behavior regardless of which project is creating the native MSVCRT program. Cygwin should be easy enough because of their use of cross compiling but if we change something in MSYS to enforce this change then Cygwin may need to do a change as well.

     
  • Keith Marshall
    Keith Marshall
    2012-02-02

    Earnie,

    There are two effectively separate issues, BOTH of which must be addressed to achieve an acceptable solution. It isn't essential that the two issues be tackled by the same development team.

    Issue (1) needs to be addressed within MSYS. The current code which prepares the command line, which MSYS will ultimately pass to CreateProcess to invoke the MinGW application, appears to DTRT when any single argument contains embedded white-space or double quote characters. It does not DTRT, when such an argument contains any globbing metacharacter which the shell did not expand, (because it was protected by quoting); such embedded metacharacters need to be handled similarly to to white-space or double quotes.

    Since this correction MUST occur within MSYS itself, I'll leave it to Cesar or Chuck to expedite the necessary changes to the implementation. If a similar change is appropriate in Cygwin, then that becomes the remit of the Cygwin folks, but we should not allow them to interfere in our own decision to DTRT.

    Issue (2) is not peculiar to MSYS. It needs to be addressed within the mingwrt start-up code; I already have some experimental code of my own, which I would be willing to publish under a MIT licence, and which may fit the bill.

    BTW, I have been doing some digging around on MSDN, and experimenting on the basis of my findings, in conjunction with MinGW. According to:
    http://msdn.microsoft.com/en-us/library/ah6y4f3z.aspx

    argument parsing is handled by _setargv(). That's certainly consistent with how it was when I last used a Microsoft compiler, (MSC-6 on MS-DOS -- i.e. the last of the MS-DOS compilers, predating even MSVC-1; not MSVC-6), and when I completely replaced the setargv() implementation in the standard library, to get a much more unix-like behaviour for the command line parsing in my own applications.

    However, MinGW's runtime does NOT provide this MSDN documented capability. It uses __getmainargs():
    http://msdn.microsoft.com/en-us/library/ff770599.aspx

    to parse the command line. AFAICT, this function NEVER calls _setargv(), (nor indeed any variant of the name setargv(), prefixed by any number of underscores from zero to three); thus, for any application using __getmainargs() as its command line parser, the _setargv() capability documented by MSDN does not seem to be available. Thus, it would seem that we need to provide our own replacement for __getmainargs(), in its entirety, or, if we believe it may do some important initialisation beyond command line parsing, at least switch off its argv globbing feature, and reparse subject to appropriate globbing rules, after any __getmainargs() initialisation has been completed.

     
  • Keith Marshall
    Keith Marshall
    2012-10-30

    prerequisite makefile.in correction

     
    Attachments
  • Keith Marshall
    Keith Marshall
    2012-10-30

    correction for msvcrt broken globbing

     
    Attachments
  • Keith Marshall
    Keith Marshall
    2012-10-30

    Aspect #2 of this bug is addressed by the pair of attached patches, both of which are against 4.0-dev branch of mingw.org-wsl, as of 29-Oct-2012 (a.m. GMT); see http://thread.gmane.org/gmane.comp.gnu.mingw.devel/4954/focus=4990 for background discussion.

    wsl-20121029-3.patch actually address an unrelated, but critical, issue; however, it touches Makefile.in, which is also touched by wsl-20121029-4.patch, and is thus a prerequisite for clean application of the latter.

    wsl-20121029-4.patch provides the implementation for the proposed resolution of aspect #2 of this bug.

    Aspect #1 remains unresolved, pending appropriate intervention by Cesar or Chuck.

     
  • Keith Marshall
    Keith Marshall
    2012-10-30

    • priority: 5 --> 7
     
  • Cesar Strauss
    Cesar Strauss
    2012-11-07

    > 1) When passing arguments to MinGW applications, quoting is correctly
    > applied to ensure that embedded white-space, escapes and double
    > quotes are preserved. The same courtesy needs to be accorded to
    > globbing characters which have not been expanded by the shell itself;
    > the current implementation neglects this requirement. The remedy for
    > this defect is entirely within MSYS remit.

    I'll include this in the next MSYS release (which is long overdue, BTW).

    Regards,

    Cesar

     
  • Cesar Strauss
    Cesar Strauss
    2012-11-29

    The fix for the MSYS side of this issue was released with MSYS runtime 1.0.18.

     
  • Earnie Boyd
    Earnie Boyd
    2012-11-30

    Thanks, Cesar. I'm reassigning to myself for the WSL piece.

     
  • Earnie Boyd
    Earnie Boyd
    2012-11-30

    • assigned_to: cstrauss --> earnie
     
1 2 > >> (Page 1 of 2)