DMARC-Filter:	OpenDMARC Filter v1.4.2 delorie.com 4APBfr3O2150136
Authentication-Results:	delorie.com; dmarc=pass (p=none dis=none) header.from=cygwin.com
Authentication-Results:	delorie.com; spf=pass smtp.mailfrom=cygwin.com
DKIM-Filter:	OpenDKIM Filter v2.11.0 delorie.com 4APBfr3O2150136
Authentication-Results:	delorie.com;
	dkim=pass (1024-bit key, unprotected) header.d=cygwin.com header.i=@cygwin.com header.a=rsa-sha256 header.s=default header.b=wWS2pdT+
X-Recipient:	archive-cygwin AT delorie DOT com
DKIM-Filter:	OpenDKIM Filter v2.11.0 sourceware.org AFD9A385770A
DKIM-Signature:	v=1; a=rsa-sha256; c=relaxed/relaxed; d=cygwin.com;
	s=default; t=1732534911;
	bh=p2eLLI7USyT5MagA2bn9NueyqJKUvVI29xfOCtILWJM=;
	h=References:In-Reply-To:Date:Subject:To:List-Id:List-Unsubscribe:
	List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:
	From;
	b=wWS2pdT+CbmwoXzXvBIFPWfTGWpRnnNe+CHpMzIW8Diqz9X6nSSeVG5CbMPSfw+Qb
	r8jpEBFDu8CC50ReDu+gqusQPhYJDSLYxciRBkpysTaXWpqSJ/6+rV4SjlDotFz6W7
	IlVBSatU0gktjW6EFKY5E35juPqrfteYrrFjq6GI=
X-Original-To:	cygwin AT cygwin DOT com
Delivered-To:	cygwin AT cygwin DOT com
DMARC-Filter:	OpenDMARC Filter v1.4.2 sourceware.org A5E373857B98
ARC-Filter:	OpenARC Filter v1.0.0 sourceware.org A5E373857B98
ARC-Seal:	i=2; a=rsa-sha256; d=sourceware.org; s=key; t=1732534876; cv=pass;
	b=w46aPa70QALxsq77whEccf3ZtNrGBIWH2hKq5Ax5MdhDCiCANm0CzK6ABfF888vZVeoy5D10On/tnsUC50Eim7Syy0Xv93xl+wCqGlDSkA8GnqLordIGEubSldicOOmqT2s/YjXsApY3HN7EjDKbvwawnUbflLQeJ9BO6PaU4WE=
ARC-Message-Signature:	i=2; a=rsa-sha256; d=sourceware.org; s=key;
	t=1732534876; c=relaxed/simple;
	bh=9IRxmTy+Tv75htNlAKrs17av+uhkxETAKrQYQ36uIc8=;
	h=DKIM-Signature:MIME-Version:From:Date:Message-ID:Subject:To;
	b=ChffbVHhNGR53T5NK/BuogsjtY5kHEYbYgmo56zeRH1/Ck7BjbdpZgd5sSoZAWoGd2EeSbHv0K2H97LdHXjKC2PsdnG7JBrUAhtZoDY8lVO1i8QvJIivs1Fc1CHRY6oiyO2wHU2AheBm0ERohUcyYlzwxbtC2mNSptFH8xxi9qU=
ARC-Authentication-Results:	i=2; server2.sourceware.org
DKIM-Filter:	OpenDKIM Filter v2.11.0 sourceware.org A5E373857B98
X-Sender-Id:	dreamhost|x-authsender|gisburn AT nrubsig DOT org
ARC-Seal:	i=1; s=arc-2022; d=mailchannels.net; t=1732534874; a=rsa-sha256;
	cv=none;
	b=G88Yi/cBjWmK8dNjShmXxnciqw4nEfOdSsfFWUbeZpErXD3ZeHi6SX/kL48I/duX91pXWf
	LDGe7O5ZRtY+IQKljXJtLSBrZFlmI7l/JoLEpZlVV2xMtXmzavP45rq+9hvrxmaVz96t09
	Fgh8g5iDaiIXhV+ip6jSmG3Zk3xomAUigz0f7h0MtqvFcPEOIpx4aBB0PlPkT1RTWzeysA
	PifqcHxMAimRR5GLgSZI23ymnEKrStRyJhTSPMeOziID/mjELQzxiBYb5qg/4XWyRJ6ujB
	VpxRONcwYTk5m5pMe/jbNk10EM7reWcm9Wuoj8LPXHyVNEqniOYHke7jteipRg==
ARC-Message-Signature:	i=1; a=rsa-sha256; c=relaxed/relaxed;
	d=mailchannels.net; s=arc-2022; t=1732534874;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	to:to:cc:mime-version:mime-version:content-type:content-type:
	content-transfer-encoding:content-transfer-encoding:
	in-reply-to:in-reply-to:references:references:dkim-signature;
	bh=MqmdeqpD293CW0S76IFYBruRXH2RvPFJq3ZLgdkj//A=;
	b=f5WD2xT75z/V7S/2AvC35tHzOoxW1m7+c1LfNd2S+Baes63052JA89kbnLsWx9CGZ7s4Ar
	Uys5O0vAOfVxAlTAFufxuCd/xsTRFjDF/ocDPBWUIkZL0NN14JCbgHDiCrmPDpR87UQQPl
	iFL/v4Utq5Drdu6GuJEUBuAhrHj25yc8kPKt1tUeEwQ7Khlg/GKTpzgZ+wM9jILkp/g3iS
	GqJLzR5u0g2hxD94zntNrWWTDawvV1JbHKLh+oZM/rUsYl66SN5qvf1ZrPiQlKfA9DXd41
	AFjsPCLfNy/z7PZHaAlHqN9QtrzYTWaVmGCFq6baCXWSgPGsA13/yYFdtKtl4g==
ARC-Authentication-Results:	i=1; rspamd-868968d99d-9jqkh;
	auth=pass smtp.auth=dreamhost smtp.mailfrom=roland DOT mainz AT nrubsig DOT org
X-Sender-Id:	dreamhost|x-authsender|gisburn AT nrubsig DOT org
X-MC-Relay:	Neutral
X-MailChannels-SenderId:	dreamhost|x-authsender|gisburn AT nrubsig DOT org
X-MailChannels-Auth-Id:	dreamhost
X-Cure-Society:	6351cd4d62e7e7e0_1732534874341_2504694308
X-MC-Loop-Signature:	1732534874341:1372893042
X-MC-Ingress-Time:	1732534874341
X-Gm-Message-State:	AOJu0YxY+0rhTv+2dMxYbCs4hJJiTnD4l0l7Uue2dxAhNG7cxHbrkDLZ
	/M+cQh+/hIyfmit3pTNdV0vJ3S19qMNqKbOpUYjy5nnCwRFj6l6ZpSRKT3fuo9CmxT100wo1i+J
	FVbi22JA840tP4298nZ6qge+4Bfk=
X-Google-Smtp-Source:	AGHT+IGiZx5ilyNnftydYtSA2ZfoXDaDDteieY8i7xUi/PKSszqgjRBDkVAqKShSylx211MukfiBAQ4LfNGxqZ5aagA=
X-Received:	by 2002:a5d:64e4:0:b0:382:4aae:68c7 with SMTP id
	ffacd0b85a97d-38260be4002mr9754955f8f.53.1732534872400; Mon, 25 Nov 2024
	03:41:12 -0800 (PST)
MIME-Version:	1.0
References:	<CALXu0Uc=wOUbEaTzpfrYdn3Q8rXcaR-if84ZWXPgFJ1qxQbizQ AT mail DOT gmail DOT com>
	<08d6a039-215c-c8e8-0280-15bc0ce28a43 AT jdrake DOT com>
	<CALXu0Ud1ne0F-K5fkHh_Ujp0BvZc6GUMmH3JTSAY_V+sCq8DsQ AT mail DOT gmail DOT com>
In-Reply-To:	<CALXu0Ud1ne0F-K5fkHh_Ujp0BvZc6GUMmH3JTSAY_V+sCq8DsQ@mail.gmail.com>
Date:	Mon, 25 Nov 2024 12:40:45 +0100
X-Gmail-Original-Message-ID:	<CAKAoaQnJzpoL0xvi5R9qbmAJV-Q_QtwYaWHCDhTCFzPRoKp+mA AT mail DOT gmail DOT com>
Message-ID:	<CAKAoaQnJzpoL0xvi5R9qbmAJV-Q_QtwYaWHCDhTCFzPRoKp+mA@mail.gmail.com>
Subject:	Re: [Ms-nfs41-client-devel] Corrupted file name in Cygwin - does
	Cygwin do a silly rename if a file is open?
To:	cygwin AT cygwin DOT com, ms-nfs41-client-devel AT lists DOT sourceforge DOT net
X-BeenThere:	cygwin AT cygwin DOT com
X-Mailman-Version:	2.1.30
List-Id:	General Cygwin discussions and problem reports <cygwin.cygwin.com>
List-Archive:	<https://cygwin.com/pipermail/cygwin/>
List-Post:	<mailto:cygwin AT cygwin DOT com>
List-Help:	<mailto:cygwin-request AT cygwin DOT com?subject=help>
List-Subscribe:	<https://cygwin.com/mailman/listinfo/cygwin>,
	<mailto:cygwin-request AT cygwin DOT com?subject=subscribe>
From:	Roland Mainz via Cygwin <cygwin AT cygwin DOT com>
Reply-To:	Roland Mainz <roland DOT mainz AT nrubsig DOT org>
Sender:	"Cygwin" <cygwin-bounces~archive-cygwin=delorie DOT com AT cygwin DOT com>
X-MIME-Autoconverted:	from base64 to 8bit by delorie.com id 4APBfr3O2150136

On Sun, Nov 24, 2024 at 8:32 AM Cedric Blancher
<cedric DOT blancher AT gmail DOT com> wrote:
>
> On Sat, 23 Nov 2024 at 17:47, Jeremy Drake <cygwin AT jdrake DOT com> wrote:
> >
> > On Sat, 23 Nov 2024, Cedric Blancher via Cygwin wrote:
> >
> > > Good afternoon!
> > >
> > > Does Cygwin do a silly rename if a Cygwin file is open but gets
> > > /bin/rm at the same time?
> >
> > Yes!  See function try_to_bin in winsup/cygwin/syscalls.cc:
> >       /* Create unique filename.  Start with a dot, followed by "cyg"
> >          transposed into the Unicode low surrogate area (U+dc00) on file
> >          systems supporting Unicode (except Samba), followed by the inode
> >          number in hex, followed by a path hash in hex.  The combination
> >          allows to remove multiple hardlinks to the same file. */
>
> That code is wrong.
>
> bash -c 'printf ".\udc63\udc79\udc67#\n"' | iconv -f UTF-8
> .iconv: illegal input sequence at position 1
>
> 334 RtlAppendUnicodeToString (&recycler,
> 335 (pc.fs_flags () & FILE_UNICODE_ON_DISK
> 336 && !pc.fs_is_samba ())
> 337 ? L".\xdc63\xdc79\xdc67" : L".cyg");
>
> SAMBA is right to reject L".\xdc63\xdc79\xdc67", because it is not a
> valid UTF-16 sequence. ReFS with validation, OpenZFS and so on will
> all REJECT such file names, and neither can NFSv4 because file names
> must be valid Unicode (even if nfsd would not validate then filesystem
> being shared via nfsd will reject that).
> So this can only work on ntfs, and only if it is not validating the
> input UTF.16 sequence.
>
> AFAIK FILE_UNICODE_ON_DISK means that the wchar_t sequences must be
> valid UTF-16, and not just be a random sequence of 16bit values.
>
> @Corinna Vinschen Could this sequence please be changed to a VALID
> UTF-8 sequence, such as \u[fffc]\u[fffc]\u[fffc]? That might work with
> SAMBA, ReFS, OpenZFS NFSv4, ...

That does not help with existing Cygwin installations and Cygwin
32bit, which is stuck at Cygwin 3.3.x ... ;-(

I agree that the L".\xdc63\xdc79\xdc67" prefix will backfire on
something like ReFS, OpenZFS etc (SAMBA uses the prefix for
filesystems which do NOT have |FILE_UNICODE_ON_DISK| set), but for
ms-nfs41-client I just stomp over the issues with this patch (wording
still needs to be improved):
---- snip ----
diff --git a/daemon/setattr.c b/daemon/setattr.c
index 9eaafb5..6e9729e 100644
--- a/daemon/setattr.c
+++ b/daemon/setattr.c
@@ -284,6 +284,46 @@ static int handle_nfs41_rename(void
*daemon_context, setattr_upcall_args *args)

    EASSERT((rename->FileNameLength%sizeof(WCHAR)) == 0);

+#define CYGWIN_STOMP_SILLY_RENAME_INVALID_UTF16_SEQUENCE 1
+
+#ifdef CYGWIN_STOMP_SILLY_RENAME_INVALID_UTF16_SEQUENCE
+    /*
+     * Stomp Cygwin "silly rename" invalid Unicode sequence
+     *
+     * Cygwin has it's own variation of "silly rename" (i.e. if
+     * someone deletes a file while someone else still has
+     * a valid fd to that file it first renames that file with a
+     * special prefix, see
+     * newlib-cygwin/winsup/cygwin/syscalls.cc, function
+     * |try_to_bin()|).
+     *
+     * Unfortunately on filesystems supporting Unicode
+     * (i.e. |FILE_UNICODE_ON_DISK|) Cygwin adds the prefix
+     * L".\xdc63\xdc79\xdc67", which is NOT a valid UTF-16 sequence,
+     * and will be rejected by a filesystem validating the
+     * UTF-16 sequence (e.g. SAMBA, ReFS, OpenZFS, ...).
+     * In our case the NFSv4.1 protocol requires valid UTF-8
+     * sequences, and the NFS server will reject filenames if either
+     * the server or the exported filesystem will validate the UTF-8
+     * sequence.
+     *
+     * Since Cygwin only does a |rename()| and never a lookup by
+     * that filename we just stomp the prefix with the prefix used
+     * for non-|FILE_UNICODE_ON_DISK| filesystems.
+     * We ignore the side-effects here, e.g. that Win32 will still
+     * "remember" the original filename in the file name cache.
+     */
+    if ((rename->FileNameLength > (4*sizeof(wchar_t))) &&
+        (!memcmp(rename->FileName,
+            L".\xdc63\xdc79\xdc67", (4*sizeof(wchar_t))))) {
+        DPRINTF(1, ("handle_nfs41_rename(args->path='%s'): "
+            "Cygwin sillyrename prefix \".\\xdc63\\xdc79\\xdc67\" "
+            "detected, squishing prefix to \".cyg\"\n",
+            args->path));
+        (void)memcpy(rename->FileName, L".cyg", 4*sizeof(wchar_t));
+    }
+#endif /* CYGWIN_STOMP_SILLY_RENAME_INVALID_UTF16_SEQUENCE */
+
    dst_path.len = (unsigned short)WideCharToMultiByte(CP_UTF8,
        WC_ERR_INVALID_CHARS|WC_NO_BEST_FIT_CHARS,
        rename->FileName, rename->FileNameLength/sizeof(WCHAR),
---- snip ----

----

Bye,
Roland
-- 
  __ .  . __
 (o.\ \/ /.o) roland DOT mainz AT nrubsig DOT org
  \__\/\/__/  MPEG specialist, C&&JAVA&&Sun&&Unix programmer
  /O /==\ O\  TEL +49 641 3992797
 (;O/ \/ \O;)

-- 
Problem reports:      https://cygwin.com/problems.html
FAQ:                  https://cygwin.com/faq/
Documentation:        https://cygwin.com/docs.html
Unsubscribe info:     https://cygwin.com/ml/#unsubscribe-simple

- Raw text -