X-Recipient:	archive-cygwin AT delorie DOT com
X-Original-To:	cygwin AT cygwin DOT com
Delivered-To:	cygwin AT cygwin DOT com
DMARC-Filter:	OpenDMARC Filter v1.3.2 sourceware.org C5628385B834
Authentication-Results:	sourceware.org;
	dmarc=none (p=none dis=none) header.from=towo.net
Authentication-Results:	sourceware.org; spf=none smtp.mailfrom=towo AT towo DOT net
Subject:	Re: bug report: shell expansion in argv[] processing sensitive to
	LANG, e.g. "ls: cannot access '*.pdf': No such file or directory", but works
	okay in bash
To:	cygwin AT cygwin DOT com
References:	<DM5PR13MB0876FCF9357D4881E9A4476FCBF10 AT DM5PR13MB0876 DOT namprd13 DOT prod DOT outlook DOT com>
From:	Thomas Wolff <towo AT towo DOT net>
X-Tagtoolbar-Keys:	D20200324100722391
Message-ID:	<dccd5d06-0edf-7a28-7640-b33e8864e464@towo.net>
Date:	Tue, 24 Mar 2020 10:07:22 +0100
User-Agent:	Mozilla/5.0 (Windows NT 10.0; WOW64; rv:68.0) Gecko/20100101
	Thunderbird/68.6.0
MIME-Version:	1.0
In-Reply-To:	<DM5PR13MB0876FCF9357D4881E9A4476FCBF10@DM5PR13MB0876.namprd13.prod.outlook.com>
X-Provags-ID:	V03:K1:K1KvxFU6dgC9y26UtvVF+5I3gY3d2v7ek3s3G07kYFqop4AXe8O
	L6anyQzikdzpOUF80joYcIa6KVpNoxDmMy8BSu7ZgGKjAZ2lilDdt8ej5YiU9J4d01SAGhg
	wMD0LlEJsl5k/+RapEagSRDyU5g03ewhQd4Oisx5o6CGz/uLmx2e8mf5Fczq9WRrkwQrgsy
	pWhSFpoM8lEFppKYBI2Uw==
X-UI-Out-Filterresults:	notjunk:1;V03:K0:ktZiQCcHjFI=:NvJmTHfheQT6TfTk7S+QSa
	IQM451ZGhRDR0MLyVg4wASM8Mq7pGCbRswB4RXOxGYyjOAA1fMp+C1+pzC5GLhwRv6BjKSdG7
	NYW7tdUvnyx4mYxGEHaTHixNstjibkc+sjOCIGTTdk7jfwxdZeWBAF7tCwwzO5qpH01Di/g7K
	DGcAzSw0AJIQYGIyECLSl1SucADFhLaRALq1XVVI2Z7ppKNADmPJtm91iUse1Q6s8zEuJF7un
	/858tdJbXj/wx4w9IW0861zLBA36io4oOnLcaC+cJSmb59VxXaab/N+yyWIx2CE6jmqZkmRZH
	bMjdBY98Oofo0/P8fn/zcLUJCLlgsIgtQBoF/9MZMWlT4+AtXCLZv3gzvJcp1HOn+IQZEgYJO
	GKRKyvvvskKERToT1v3tmRUWnFKEgWr9QYx1AQEP6T6W0nPyLKrmuTlOv+TMWNl+1/w4Z7D1y
	IUrOtrsuC5rr3PNwzPRUk+ZmO+GC1jJYDcU6cxP03Eth8QvgK1SsWKmpCrru7As54hbyC1ccW
	mSfwFgwMZpy8DCIsILnTCRHuEuOSTchqt1g/WIM+uzIiIhM57V1726J2YQC26wMrDBTfnsltL
	TUfcixzJHrCCzuW/SCojT/ZkfnaqoJe2n2FF9tP5oVxoGg/BVrY3mjzCQf4frdUgVEJddaZl3
	AqBT/DzjVP17vDiJdZCpzjFiLu5ATe2KJR3zKKqxeqxM949XowXAvZe9JlpaCO07mIKpgmH1r
	33MFM2p+GGn3NYQ+aMEsaaTJQeFmwm+vUXn2K9dlTBmKWLMnQ1aJrEEeG+F7U3sovuq1XRELx
	+mDUwAsv1c0RJy7AuNARbvKoOVCqbKJknn7ksvNGbgkYVawLSU=
X-Spam-Status:	No, score=1.4 required=5.0 tests=BAYES_00, KAM_DMARC_STATUS,
	KAM_LAZY_DOMAIN_SECURITY, RCVD_IN_BARRACUDACENTRAL, RCVD_IN_DNSWL_NONE,
	RCVD_IN_MSPIKE_H2, SPF_HELO_NONE, SPF_NONE,
	TXREP autolearn=no autolearn_force=no version=3.4.2
X-Spam-Level:	*
X-Spam-Checker-Version:	SpamAssassin 3.4.2 (2018-09-13) on
	server2.sourceware.org
X-BeenThere:	cygwin AT cygwin DOT com
X-Mailman-Version:	2.1.29
List-Id:	General Cygwin discussions and problem reports <cygwin.cygwin.com>
List-Unsubscribe:	<http://cygwin.com/mailman/options/cygwin>,
	<mailto:cygwin-request AT cygwin DOT com?subject=unsubscribe>
List-Archive:	<http://cygwin.com/pipermail/cygwin/>
List-Post:	<mailto:cygwin AT cygwin DOT com>
List-Help:	<mailto:cygwin-request AT cygwin DOT com?subject=help>
List-Subscribe:	<http://cygwin.com/mailman/listinfo/cygwin>,
	<mailto:cygwin-request AT cygwin DOT com?subject=subscribe>
Errors-To:	cygwin-bounces AT cygwin DOT com
Sender:	"Cygwin" <cygwin-bounces AT cygwin DOT com>
X-MIME-Autoconverted:	from quoted-printable to 8bit by delorie.com id 02OAPDCr016016

Am 24.03.2020 um 08:18 schrieb Jay Libove via Cygwin:
> Hi Cygwin team,
> Here is a consolidated bug report based on the discussion in recent days which I'd started under the subject " shell expansion produces e.g. "ls: cannot access '*.pdf': No such file or directory" in Windows CMD shell, but works okay in bash " (thread starter https://cygwin.com/pipermail/cygwin/2020-March/244161.html )
> Many thanks to Paul, Andrey, and others for helping me nail down where and how it seems to be happening.
> My apologies in advance that my coding days are long behind me, so I'm not in a position to include a proposed code fix.
>
> cygcheck output attached (lightly modified to redact a couple of personal items).
>
> Problem:
> Under certain circumstances (see Steps to Reproduce, below) Cygwin programs' built-in argv[] globbing will produce unexpected:
> "{programName}: cannot access '{glob pattern}: No such file or directory"
> e.g.
> "ls: cannot access '*.pdf': No such file or directory"
> .. despite the fact that e.g. *.pdf definitely exists.
>
> Steps to Reproduce:
> * Have some files in the local director with accented characters in the names, e.g.:
> C:> mkdir c:\temp\test
> C:> cd c:\temp\test
> C:> touch héllo.pdf
> C:> touch gòodbye.pdf
> C:> touch normal.pdf
> * DON'T have the LANG= environment variable set to anything
> * NOT in bash or Cygwin Terminal, but rather within Windows CMD.exe, execute a Cygwin command which needs to do file name globbing because the Windows CMD.exe shells does not do so for it, e.g.
> C:> ls *.pdf
> C:> cat *.pdf
> These will produce "ls: cannot access '*.pdf': No such file or directory"
> Although, curiously,
> C:> ls *or*
> does correctly produce:
> normal.pdf
>
> Also, display output of the áccènted characters is incomplete:
> C:> ls
> 'g'$'\303\262''odbye.pdf'  'h'$'\303\251''llo.pdf'   normal.pdf
> C:> bash
> jay_l AT DESKTOP-I9MRIE3 /cygdrive/c/Temp
> $ ls
> 'g'$'\303\262''odbye.pdf'  'h'$'\303\251''llo.pdf'   normal.pdf
>
>
> Analysis:
> I've verified that it's not about case sensitivity. That is, it's not a matter of ls *.pdf vs. ls *.PDF.
> If these test commands are run either under bash.exe or within a Cygwin Terminal window, the problem does not occur.
> I've verified that the Windows system locale (per Windows' Region setting) actually doesn't matter. (I've reproduced this both on systems in Region Spain with language English-International and English-Ireland, and in a VM with a bog standard vanilla US English Windows).
>
> Credits to Paul for suggesting deleting files one by one until the problem goes away, and to Andrey for pointing out `locale` and the LANG= setting.
>
> Set LANG=en_US.UTF-8, e.g.
> C:> set LANG=en_US.UTF-8
> .. and the problem goes away.
> C:> ls *.pdf
> gòodbye.pdf
> héllo.pdf
> normal.pdf
> C:> ls
> gòodbye.pdf
> héllo.pdf
> normal.pdf
>
> Interestingly, Andrey mentioned that he sets LANG=ru_RU.CP866 and he doesn't see the problem. When I tried that exact setting, I still had the problem.
> So it's maybe not just that LANG must be set to *something*, but that somehow LANG must be set to something that matches something in Windows? (Sorry, I know that's nearly uselessly vague).
>
>
> In summary, it appears that the way that the argv[] globbing code which gets compiled in to Cygwin programs functions a bit differently than the way the shell globbing code works within bash.exe.
> And this produces unexpected globbing failures.
(As commented in the other thread already:)
Maybe it can simply be fixed by changing the order of setting up locale 
stuff and applying the expansion in cygwin?
(I would look into the code if I had a clue where to find the respective 
things.)
Thomas


> Thanks to all the Cygwin maintainers for this amazing software, for so many years!
> -Jay
--
Problem reports:      https://cygwin.com/problems.html
FAQ:                  https://cygwin.com/faq/
Documentation:        https://cygwin.com/docs.html
Unsubscribe info:     https://cygwin.com/ml/#unsubscribe-simple

- Raw text -