X-Recipient:	archive-cygwin AT delorie DOT com
DomainKey-Signature:	a=rsa-sha1; c=nofws; d=sourceware.org; h=list-id
	:list-unsubscribe:list-subscribe:list-archive:list-post
	:list-help:sender:from:reply-to:to:subject:date:message-id
	:references:in-reply-to:content-type:content-transfer-encoding
	:mime-version; q=dns; s=default; b=BUiUGVVehivUC1nt4YS2XNY0kD45T
	9uIBPEv2BXCGBNCEBo+CNQDDbVeGhqwp0t8V/pxb+7XEK6DTvdrCjlJdwWxYvU4l
	mUopY+fTT3NXSlKuoxzUVQdVq3RmXjemaxbzAG2Urpl5IRima3iSiH1mgwbKRhu9
	mhxERtk2LBonFQ=
DKIM-Signature:	v=1; a=rsa-sha1; c=relaxed; d=sourceware.org; h=list-id
	:list-unsubscribe:list-subscribe:list-archive:list-post
	:list-help:sender:from:reply-to:to:subject:date:message-id
	:references:in-reply-to:content-type:content-transfer-encoding
	:mime-version; s=default; bh=CgBk9zRbONSTfkmODPlsLAdEVuw=; b=ce+
	e/6oZqBZ8ou7sKo3KSWHF4fpAA39lB7xd1Elzl4IssvU7gbZyNHzGCnLStnCmfHR
	U37yocsgiB2m9lxTGx04MAk0LQxrT3DB92uAnWhrcjPjzsJrNZHim+unpE1vTrZ0
	6s6IHidrbmxqFzz8QzrpB25Vr88GNw7pEZWKxI8A=
Mailing-List:	contact cygwin-help AT cygwin DOT com; run by ezmlm
List-Id:	<cygwin.cygwin.com>
List-Subscribe:	<mailto:cygwin-subscribe AT cygwin DOT com>
List-Archive:	<http://sourceware.org/ml/cygwin/>
List-Post:	<mailto:cygwin AT cygwin DOT com>
List-Help:	<mailto:cygwin-help AT cygwin DOT com>, <http://sourceware.org/ml/#faqs>
Sender:	cygwin-owner AT cygwin DOT com
Mail-Followup-To:	cygwin AT cygwin DOT com
Delivered-To:	mailing list cygwin AT cygwin DOT com
Authentication-Results:	sourceware.org; auth=none
X-Spam-SWARE-Status:	No, score=-6.9 required=5.0 tests=BAYES_00,GIT_PATCH_2,RCVD_IN_DNSWL_NONE,SPF_HELO_PASS,SPF_PASS autolearn=ham version=3.3.1 spammy=Friday, stephen, heres, friday
X-HELO:	NAM04-BN3-obe.outbound.protection.outlook.com
ARC-Seal:	i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=j9LH5MIrPzUaIvqujd+hkeZo89bQ3kIJW1SlUszFMtvgxADGgwxbDeLvNgka2g+gYgLhxUg3yNjpurqZj8iSnSEwa0ZM1Eifi7kNjtuhHiPA3M7k4HTJxfQmWklaZ3zenO9VSS/IHm5VDPRpu4KmNnrXwHBPDRcy36O9s0q8VVQdCWXWO5l1gPqk2cXh+mtegaxLNNwMjjbLsku6E5NkfFzOztLwNfz4uPNPODTX3TU90a3Jg+c9Ncmn0IA4JWq6tRj+LXgG+YzG/XfbSzoyoHFl0bFnX3KpLxvPWfeQ/L+KMOghqFtiTwmHMGuQVysWDhX9S5j5JBD2G/NcpPtnPQ==
ARC-Message-Signature:	i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=TC9+/s+2Y1vckKKS+ur1H+c/NCHBrO0l+bOKFRhC1eI=; b=W5UGAkcW7seHZ3cjRkRBkZKX6CEPEqwrLo5OCEGf9h3xjd/HFX+yEKkzQ52uXkcQY6pmrLFEWwpsEiwltpsmennQkH1EUCgRdywrA6yKWJ+xSI5SXu5J1IuhrE+WICOKAF8BJLKOrboo1Fc3jvyXN/hKPJ8VMTEqE3CU0Foszc0Yi5JMa3DAwmU7gqEZVOoS1lsXZo5ZnrbQHx5/M/8mEjMozX3SNlAw8z5woIIYBqa0YVhkblI1g/L8rZtXF3V/sEiWXCWEMcmX+XARVrqsvWUubYtfd0v7kEsDj4K4+BiidqlDWMF+b3mLZs9cu8EZxpEq1rcOWg3VJwdG8X/g6Q==
ARC-Authentication-Results:	i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=microsoft.com; dmarc=pass action=none header.from=microsoft.com; dkim=pass header.d=microsoft.com; arc=none
DKIM-Signature:	v=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=TC9+/s+2Y1vckKKS+ur1H+c/NCHBrO0l+bOKFRhC1eI=; b=JjCqkUVW/oczG5fjk2CMQasQfcRB/1uiRBvnYrylURBV7vIT/BxWjI2tctRgqRsI9Ec540Z64D7tEvpv2UUv//A5cn3ttpX5crFVoqgbfhDNUQgkk0VNl4iUA5af17eRf5RneL1k5lpSVqTv8q7doDpyn4UHKQlf1PU5g37Afd8=
From:	"Stephen Provine via cygwin" <cygwin AT cygwin DOT com>
Reply-To:	Stephen Provine <stephpr AT microsoft DOT com>
To:	"cygwin AT cygwin DOT com" <cygwin AT cygwin DOT com>,
	"anrdaemon AT yandex DOT ru" <anrdaemon AT yandex DOT ru>
Subject:	RE: Command line processing in dcrt0.cc does not match Microsoft parsing rules
Date:	Mon, 9 Sep 2019 16:41:37 +0000
Message-ID:	<CY4PR21MB0838765A912890C93D4F223DB9B70@CY4PR21MB0838.namprd21.prod.outlook.com>
References:	<MWHPR21MB08452919F35B1B0C5F0EB4DCB9BD0 AT MWHPR21MB0845 DOT namprd21 DOT prod DOT outlook DOT com> <MWHPR21MB0845F78385792965A94E0CD9B9BD0 AT MWHPR21MB0845 DOT namprd21 DOT prod DOT outlook DOT com> <MWHPR21MB084508155AB621C7AD81309CB9B90 AT MWHPR21MB0845 DOT namprd21 DOT prod DOT outlook DOT com> <MWHPR21MB08456D9F03AF8BD450E6AB2EB9B80 AT MWHPR21MB0845 DOT namprd21 DOT prod DOT outlook DOT com> <MWHPR21MB0845282E7582DC95ADF0F140B9BB0 AT MWHPR21MB0845 DOT namprd21 DOT prod DOT outlook DOT com> <135817606 DOT 20190906233445 AT yandex DOT ru>
In-Reply-To:	<135817606.20190906233445@yandex.ru>
authentication-results:	spf=none (sender IP is ) smtp.mailfrom=stephpr AT microsoft DOT com;
x-ms-exchange-purlcount:	3
x-ms-oob-tlc-oobclassifiers:	OLM:10000;
received-spf:	None (protection.outlook.com: microsoft.com does not designate permitted sender hosts)
x-ms-exchange-senderadcheck:	1
x-ms-exchange-transport-forked:	True
MIME-Version:	1.0
X-MS-Exchange-CrossTenant-mailboxtype:	HOSTED
X-MS-Exchange-CrossTenant-userprincipalname:	Dlg3WBAQH0jJhuw5vAjHfXgPLNLROytFlNlnRhIrwltabRn1hHCv07C84GIV9WWh+jx5iJXW7QUmmSLhI5q3AQ==
X-MIME-Autoconverted:	from quoted-printable to 8bit by delorie.com id x89GgH5U032446

On 2019-09-06 13:35, Andrey Repin wrote:
> CMD escape character is ^, not \

You are correct about the cmd.exe interpretation, so my test cases were
buggy, but Go invokes other executables using CreateProcess directly and
is not subject to the additional set of command line processing rules that
are used by cmd.exe.

If you see the last exchange with Eric, I think it is clear that there is a case
missing in the Cygwin processing rules that becomes a problem when a
calling process directly reverses the rules, specifically when an argument
value does not itself need to be quoted but it has a double quote in the
value. This is rule 4 in what I found to be the most definitive reference:

http://daviddeley.com/autohotkey/parameters/parameters.htm#WINCRULESCHANGE

And see the fourth example in section 5.4.

However, the *safest* way to construct a command line is to avoid this
case and make sure to always double quote an argument that contains
double quotes. The official algorithm from a Microsoft source was
previously posted by Eric:

https://blogs.msdn.microsoft.com/twistylittlepassagesallalike/2011/04/23/everyone-quotes-command-line-arguments-the-wrong-way/

Interesting that there's actually nothing in this article that specifically
means it *shouldn't* be ok to do what the Go algorithm does, it just
happens to be simpler if you don't worry about that case.

FWIW, .NET Core uses this algorithm:

https://github.com/dotnet/corefx/blob/master/src/Common/src/CoreLib/System/PasteArguments.cs

Which I think is probably pretty good validation that it's the right one to use.

So, the outcome of all of this is that Go should probably update their logic
as it's based on the wrong official source. I plan to follow up there. If there
is any interest in the future to correct the parsing behavior in Cygwin, the
information needed to do that is in this thread. Personally, I think that if
Cygwin fixes the problem it's easier to recompile all those binaries than try
to locate all potential source calling processes to make sure they follow
the right algorithm (Go isn't right, what about Node, Python, etc...) But
I'm not going to push on this point as I can work around it for my case.

Thanks,
Stephen

-----Original Message-----
From: Andrey Repin <anrdaemon AT yandex DOT ru> 
Sent: Friday, September 6, 2019 1:35 PM
To: Stephen Provine <stephpr AT microsoft DOT com>; cygwin AT cygwin DOT com
Subject: Re: Command line processing in dcrt0.cc does not match Microsoft parsing rules

Greetings, Stephen Provine!

> On 2019-09-04 23:29, Brian Inglis wrote:
>> As standard on Unix systems, just add another level of quoting for 
>> each level of interpretation, as bash will process that command line, 
>> then bash will process the script command line.

> My mistake - I'm very aware of the quoting rules, yet in my test 
> script for this scenario I forgot to quote the arguments. However, if 
> POSIX rules are being implemented, there is still something I didn't expect. Here's my bash script:

> #!/bin/bash
> echo "$1"
> echo "$2" 
> echo "$3"

> And I invoke it like this from a Windows command prompt:

> C:\> bash -x script.sh foo bar\"baz bat
> + echo foo
> foo
> + echo 'bar\baz bat'
> bar\baz bat
> + echo ''

> Not expected. Called from within Cygwin, the behavior is correct:

Again, fully expected.

> $ bash -x script.sh foo bar\"baz bat
> + echo foo
> foo
> + echo 'bar"baz'
> bar"baz
> + echo bat
> bat

> Can you explain this difference?

CMD escape character is ^, not \

> The reason I ask is that if this worked, the way Go constructs the 
> command line string would be just fine.

No.


--
With best regards,
Andrey Repin
Friday, September 6, 2019 23:33:46

Sorry for my terrible english...


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

- Raw text -