DMARC-Filter: OpenDMARC Filter v1.4.2 delorie.com 55RF0q3l1477820
Authentication-Results: delorie.com; dmarc=pass (p=none dis=none) header.from=cygwin.com
Authentication-Results: delorie.com; spf=pass smtp.mailfrom=cygwin.com
DKIM-Filter: OpenDKIM Filter v2.11.0 delorie.com 55RF0q3l1477820
Authentication-Results: delorie.com;
	dkim=pass (1024-bit key, unprotected) header.d=cygwin.com header.i=@cygwin.com header.a=rsa-sha256 header.s=default header.b=Yc3ZClgM
X-Recipient: archive-cygwin AT delorie DOT com
DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org BD2E4385B538
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cygwin.com;
	s=default; t=1751036451;
	bh=mRN+B/v/FJi+f7rMu4zBicc+yvVM4ekSwLBwsIHFN7k=;
	h=Date:To:Subject:References:In-Reply-To:List-Id:List-Unsubscribe:
	 List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc:
	 From;
	b=Yc3ZClgMldntlQ3ysdBbBmxvgYhJkcbwAJeZYxiFWjSm7nm/vKvC5dmAMBFxP09Pc
	 g2xWBsHXhOl0sSXslbP3vnoeF4LxD8LU396kKjTyIRbVhHlh477dMz3p3YK4r04AOv
	 SXJ778MwvQpndOaTBMz2zqr4qdFj/JKXRuWvqP3A=
X-Original-To: cygwin AT cygwin DOT com
Delivered-To: cygwin AT cygwin DOT com
DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 71A753858408
Date: Fri, 27 Jun 2025 16:59:51 +0200
To: cygwin AT cygwin DOT com
Subject: Re: readdir() returns inaccessible name if file was created with
 invalid UTF-8
Message-ID: <aF6x55WXIS1t655i@calimero.vinschen.de>
Mail-Followup-To: cygwin AT cygwin DOT com
References: <96f2253b-791b-b8a0-97dd-8d257eefb9b1 AT t-online DOT de>
 <03c4fae7-7322-572c-ae72-52e300f0b438 AT t-online DOT de>
 <aFxRfI4NdZ8y5IlK AT calimero DOT vinschen DOT de>
 <f78c615c-aefe-b3d0-aada-5f9d0cf73a0a AT t-online DOT de>
 <aF5y15iQ840LxLYJ AT calimero DOT vinschen DOT de>
 <3295c8bd-2c09-76c7-8b5f-0106dc39dd96 AT t-online DOT de>
MIME-Version: 1.0
Content-Disposition: inline
In-Reply-To: <3295c8bd-2c09-76c7-8b5f-0106dc39dd96@t-online.de>
X-BeenThere: cygwin AT cygwin DOT com
X-Mailman-Version: 2.1.30
Precedence: list
List-Id: General Cygwin discussions and problem reports <cygwin.cygwin.com>
List-Unsubscribe: <https://cygwin.com/mailman/options/cygwin>,
 <mailto:cygwin-request AT cygwin DOT com?subject=unsubscribe>
List-Archive: <https://cygwin.com/pipermail/cygwin/>
List-Post: <mailto:cygwin AT cygwin DOT com>
List-Help: <mailto:cygwin-request AT cygwin DOT com?subject=help>
List-Subscribe: <https://cygwin.com/mailman/listinfo/cygwin>,
 <mailto:cygwin-request AT cygwin DOT com?subject=subscribe>
From: Corinna Vinschen via Cygwin <cygwin AT cygwin DOT com>
Reply-To: cygwin AT cygwin DOT com
Cc: Corinna Vinschen <corinna-cygwin AT cygwin DOT com>
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Errors-To: cygwin-bounces~archive-cygwin=delorie DOT com AT cygwin DOT com
Sender: "Cygwin" <cygwin-bounces~archive-cygwin=delorie DOT com AT cygwin DOT com>

On Jun 27 15:32, Christian Franke via Cygwin wrote:
> $ touch $'t-\xef\x80\x80'
> The name mapping is:
> "t-\xEF\x80\x80" -(open, ...)-> L"t-\xDB59" -(readdir)-> "t-"

Did you copy/paste this from the old mail, by any chance?

Using the latest test DLL the mapping is

  "t-\xEF\x80\x80" -(open, ...)-> L"t-\xF000" 

And that's basically correct, albeit it leads to problems.

You know that we defined the area from 0xf000 to 0xf0ff as our private
use area to create filenames with characters invalid in DOS filenames
by transposing these chars into the private use area.  When converting
the filenames back, the 0xf0XX chars are transposed back to 0xXX.

But yeah, I found the bug here.  The problem is that the transpose table
incorrectly contains NUL as transposable character.  So if you create
L"t-\xF000", that's fine.  However, when converting this name back to
UTF-8, the filename becomes L"t-\0".  Oops.

I dropped the ASCII NUL from the list of transposable characters and
now what you get is this:

  $ touch $'t-\xef\x80\x80'
  $ touch $'t-\xef\x80\x81'
  $ ls -l
  total 0
  -rw-r--r-- 1 corinna vinschen 0 Jun 27 16:49 't-'$'\001'
  -rw-r--r-- 1 corinna vinschen 0 Jun 27 16:49 't-'$'\357\200\200'

Apart from the incorrect transposition of ASCII NUL, the transposition 
works transparently:

  $ echo foo > $'t-\xef\x80\x81'
  $ cat $'t-\xef\x80\x81'
  foo
  $ cat $'t-\x01'
  foo

I'll apply the patch shortly.


Thanks,
Corinna

-- 
Problem reports:      https://cygwin.com/problems.html
FAQ:                  https://cygwin.com/faq/
Documentation:        https://cygwin.com/docs.html
Unsubscribe info:     https://cygwin.com/ml/#unsubscribe-simple