DomainKey-Signature: a=rsa-sha1; c=nofws; d=sourceware.org; h=list-id
	:list-unsubscribe:list-subscribe:list-archive:list-post
	:list-help:sender:from:reply-to:to:subject:date:message-id
	:references:in-reply-to:content-type:content-transfer-encoding
	:mime-version; q=dns; s=default; b=BUiUGVVehivUC1nt4YS2XNY0kD45T
	9uIBPEv2BXCGBNCEBo+CNQDDbVeGhqwp0t8V/pxb+7XEK6DTvdrCjlJdwWxYvU4l
	mUopY+fTT3NXSlKuoxzUVQdVq3RmXjemaxbzAG2Urpl5IRima3iSiH1mgwbKRhu9
	mhxERtk2LBonFQ=
Mailing-List: contact cygwin-help@cygwin.com; run by ezmlm
Sender: cygwin-owner@cygwin.com
Mail-Followup-To: cygwin@cygwin.com
From: "Stephen Provine via cygwin" <cygwin@cygwin.com>
Reply-To: Stephen Provine <stephpr@microsoft.com>
To: "cygwin@cygwin.com" <cygwin@cygwin.com>,
        "anrdaemon@yandex.ru"	<anrdaemon@yandex.ru>
Subject: RE: Command line processing in dcrt0.cc does not match Microsoft parsing rules
Date: Mon, 9 Sep 2019 16:41:37 +0000
Message-ID: <CY4PR21MB0838765A912890C93D4F223DB9B70@CY4PR21MB0838.namprd21.prod.outlook.com>
References: <MWHPR21MB08452919F35B1B0C5F0EB4DCB9BD0@MWHPR21MB0845.namprd21.prod.outlook.com>   <MWHPR21MB0845F78385792965A94E0CD9B9BD0@MWHPR21MB0845.namprd21.prod.outlook.com>  <MWHPR21MB084508155AB621C7AD81309CB9B90@MWHPR21MB0845.namprd21.prod.outlook.com>  <MWHPR21MB08456D9F03AF8BD450E6AB2EB9B80@MWHPR21MB0845.namprd21.prod.outlook.com>  <MWHPR21MB0845282E7582DC95ADF0F140B9BB0@MWHPR21MB0845.namprd21.prod.outlook.com> <135817606.20190906233445@yandex.ru>
In-Reply-To: <135817606.20190906233445@yandex.ru>
received-spf: None (protection.outlook.com: microsoft.com does not designate permitted sender hosts)
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-userprincipalname: Dlg3WBAQH0jJhuw5vAjHfXgPLNLROytFlNlnRhIrwltabRn1hHCv07C84GIV9WWh+jx5iJXW7QUmmSLhI5q3AQ==
Content-Transfer-Encoding: 8bit
X-MIME-Autoconverted: from quoted-printable to 8bit by delorie.com id x89GgH5U032446

On 2019-09-06 13:35, Andrey Repin wrote:
> CMD escape character is ^, not \

You are correct about the cmd.exe interpretation, so my test cases were
buggy, but Go invokes other executables using CreateProcess directly and
is not subject to the additional set of command line processing rules that
are used by cmd.exe.

If you see the last exchange with Eric, I think it is clear that there is a case
missing in the Cygwin processing rules that becomes a problem when a
calling process directly reverses the rules, specifically when an argument
value does not itself need to be quoted but it has a double quote in the
value. This is rule 4 in what I found to be the most definitive reference:

http://daviddeley.com/autohotkey/parameters/parameters.htm#WINCRULESCHANGE

And see the fourth example in section 5.4.

However, the *safest* way to construct a command line is to avoid this
case and make sure to always double quote an argument that contains
double quotes. The official algorithm from a Microsoft source was
previously posted by Eric:

https://blogs.msdn.microsoft.com/twistylittlepassagesallalike/2011/04/23/everyone-quotes-command-line-arguments-the-wrong-way/

Interesting that there's actually nothing in this article that specifically
means it *shouldn't* be ok to do what the Go algorithm does, it just
happens to be simpler if you don't worry about that case.

FWIW, .NET Core uses this algorithm:

https://github.com/dotnet/corefx/blob/master/src/Common/src/CoreLib/System/PasteArguments.cs

Which I think is probably pretty good validation that it's the right one to use.

So, the outcome of all of this is that Go should probably update their logic
as it's based on the wrong official source. I plan to follow up there. If there
is any interest in the future to correct the parsing behavior in Cygwin, the
information needed to do that is in this thread. Personally, I think that if
Cygwin fixes the problem it's easier to recompile all those binaries than try
to locate all potential source calling processes to make sure they follow
the right algorithm (Go isn't right, what about Node, Python, etc...) But
I'm not going to push on this point as I can work around it for my case.

Thanks,
Stephen

-----Original Message-----
From: Andrey Repin <anrdaemon@yandex.ru> 
Sent: Friday, September 6, 2019 1:35 PM
To: Stephen Provine <stephpr@microsoft.com>; cygwin@cygwin.com
Subject: Re: Command line processing in dcrt0.cc does not match Microsoft parsing rules

Greetings, Stephen Provine!

> On 2019-09-04 23:29, Brian Inglis wrote:
>> As standard on Unix systems, just add another level of quoting for 
>> each level of interpretation, as bash will process that command line, 
>> then bash will process the script command line.

> My mistake - I'm very aware of the quoting rules, yet in my test 
> script for this scenario I forgot to quote the arguments. However, if 
> POSIX rules are being implemented, there is still something I didn't expect. Here's my bash script:

> #!/bin/bash
> echo "$1"
> echo "$2" 
> echo "$3"

> And I invoke it like this from a Windows command prompt:

> C:\> bash -x script.sh foo bar\"baz bat
> + echo foo
> foo
> + echo 'bar\baz bat'
> bar\baz bat
> + echo ''

> Not expected. Called from within Cygwin, the behavior is correct:

Again, fully expected.

> $ bash -x script.sh foo bar\"baz bat
> + echo foo
> foo
> + echo 'bar"baz'
> bar"baz
> + echo bat
> bat

> Can you explain this difference?

CMD escape character is ^, not \

> The reason I ask is that if this worked, the way Go constructs the 
> command line string would be just fine.

No.


--
With best regards,
Andrey Repin
Friday, September 6, 2019 23:33:46

Sorry for my terrible english...


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple