Command argument parsing bugs vs regular win32 tools

The MSYS2 runtime does some amount of interpretation of its own of the incoming command line string (which mostly is a feature, I guess). However the MSYS2 (or cygwin I presume) specific argument parsing breaks a number of details regarding how generic win32 processes parse their command line string.

The LLVM testsuite runs a large number of test cases, that involve invoking executables such as sed and grep, where the arguments may involve tricky strings with regexes and similar, where the exact escaping/unescaping of tricky characters is cruicial. The LLVM testsuite doesn't know if the sed or grep (or other shell utils) invoked are regular win32 executables (e.g. built with mingw, e.g. provided by the GnuWin tool suite) or MSYS2 based ones (in particular, LLVM's testsuite can automatically locate the set of tools installed as part of Git for Windows, as they generally fit the bill perfectly and often can be found installed on users' systems).

The LLVM tests have run into a number of cases where the regular win32 argument quoting/escaping rules don't work with MSYS based tools; some of the issues can be avoided by choosing a different quoting/escaping strategy, but overall there doesn't seem to be a single strategy that works consistently with both MSYS2 based tools and regular win32 based tools.

1. If one wants to pass the argument a"b to a regular win32 process, it is escaped as a\"b, which after argument parsing/unescaping gets parsed back into a"b. If a MSYS2 based process is passed a\"b, it receives a\b instead. This particular bug can be avoided by quoting the whole string instead; both win32 and MSYS2 based tools parse "a\"b" as a"b.

2. If one wants to pass the argument a[b\c to a regular win32 process, it doesn't need any extra escaping. If a MSYS2 based process is passed the argument a[b\c, the executable receives the argument a[bc. In this case, quoting the whole argument string, into "a[b\c" makes both win32 and MSYS2 based tools receive the same, expected argument.

3. If one wants to pass the argument a\b\\c\\\\d to a regular win32 process, it doesn't need any extra escaping either, and both win32 and MSYS2 based tools receive the same, the expected argument. However, if one quotes the whole argument (as a workaround to 1. or 2. above), the MSYS2 based tool receives a\b\c\\d instead, which wasn't what was intended.

4. As a separate workaround to 2. and 3. above, one can choose to instead set the env variable MSYS=noglob. That works around both issues 2. and 3., but breaks issue 1. above again, making both a\"b and "a\"b" be parsed as a\b instead of a\"b.

Or more consistently, here are the sets of inputs and parsed outputs:

So there doesn't seem to be any combination of quoting or setting MSYS=noglob that consistently reproduces what's expected (what the win32 equivalent app receives). By quoting case 1 and 3 into 4 and 6, but not quoting case 2, one can get something that works, but that's extremely brittle, as if case 2 requires quoting the argument for other reasons, it breaks again.

To reproduce, build a small test app like this with both the MSYS2 runtime and mingw, and try invoking it (e.g. from cmd.exe) with the various test input arguments:

#include <stdio.h>
int main(int argc, char **argv) {
    for (int i = 0; i < argc; i++)
        printf("%d: %s\n", i, argv[i]);
    return 0;
}

I presume this is mostly an upstream cygwin issue and there's not much you want to try to do about it at the MSYS2 level, but I thought I'd file it here first, for visibility and input, before proceeding further upstream.

Start with the provided small C argv-printing test app; build it once with the MSYS2 runtime and once with mingw, then invoke both from cmd.exe using the listed inputs. Compare outputs under default settings and MSYS=noglob against the win32 expectations. Done means the parsing discrepancy is reproduced and its fix or upstream disposition is established.