DKIM-Filter: OpenDKIM Filter v2.11.0 delorie.com 457LAu911230213
Authentication-Results: delorie.com;
	dkim=pass (1024-bit key, unprotected) header.d=cygwin.com header.i=@cygwin.com header.a=rsa-sha256 header.s=default header.b=vIfFwICl
X-Recipient: archive-cygwin AT delorie DOT com
DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 1635F3A2685B
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cygwin.com;
	s=default; t=1717766653;
	bh=ZN8XGtGQp+ze9MFeSnY+T1P2r7NRH+8bMNpeA68ckro=;
	h=References:In-Reply-To:Date:Subject:To:List-Id:List-Unsubscribe:
	 List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:
	 From;
	b=vIfFwIClQF2if4GsoXlPNGr/NTT6jSYOJDlX4TwZ989W3R2KohWsVaJw5gXPEUIG/
	 /ZfMiScx0gQf/DM0JpJv6rUpboY7eueRaFiW8fOYHMByazUrOZ1h51C/pgmBZkmDZA
	 VdZRF63o0qzkBjWKBoOchs3Q5LyhrA/oY7g2DO0o=
X-Original-To: cygwin AT cygwin DOT com
Delivered-To: cygwin AT cygwin DOT com
DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 8D3AE3A27375
ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 8D3AE3A27375
ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1717766588; cv=none;
 b=MOoaevodIJ/gGPyEcidpVtXIIH2kzkNZzvNt3fCDr9mLU+VaU94gJZq8btLPM54EdkKZZS7Xsp/wRwO3r2/AoeVhmXGb68J34fV0IVTAm2DWRba4a9lehiRN2GbgFuxCIC28jjIcOSNJhIF5wvDZTNZvLlqE5GsV9Ol35aM2p+c=
ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key;
 t=1717766588; c=relaxed/simple;
 bh=hanMxNqnqw5VzXhwfsakDFJfOZ3dAT2vKanhm1o5PQA=;
 h=MIME-Version:From:Date:Message-ID:Subject:To;
 b=BYpyq2xxW4KufSLuV/IisFK4wIpCQRWAledSsuFqFBBtmqQ+XGyWH8aGGHk5iqcGoGVT/OIlGYFBBmIHSGgTJOr2MpctzadE2ugacxWmrbR5jikCEvVCTrvkzNCcFf4QQqoG5Id15VgnZSoN6FnRSlEoWV5lAao3L91/4LlKqzs=
ARC-Authentication-Results: i=1; server2.sourceware.org
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20230601; t=1717766576; x=1718371376;
 h=content-transfer-encoding:to:subject:message-id:date:from
 :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
 :subject:date:message-id:reply-to;
 bh=xzNhhL28DBeZoayPnXeF6a0HBEdkVRGdMkl20knJa+U=;
 b=MvlyhYBY87EHd3lcEewtWurzU3R2WbISk5q/0lK9SqQ3x3FNPlUQr4c4UmzbZNEqiU
 Bnf8YzKrQv5KaJomOeyiVcNm9tm2uL47HzuqE3+N9PrNRBOfjCcD3sAYzCeHJqkkHvRm
 ESxVipMPuUh7gtSZeBye/k+0heVY0MK06Pmf0tCMyo3jhht/v2kAvarXTMEiuZ6ZHG2G
 sf/e4eib2QUJoTQo9CPhjzI+rw1G6TJd3y0xsFgfJIl7xt5FHfKOtvjW3Ek2mhvSUSVW
 9FStNfCfGOZrJyoWHTeo3W999gCDsA+Yit60e/JQ1Fo16uyX76mhbSf2N9vzip1CPaQ9
 K0rA==
X-Gm-Message-State: AOJu0YyhkeB4jrK5z9wnvBadZAC9PPC0TjjdUtYz74tL0FGnEBAS5O1H
 xK+KeG1nng0AwWMPIGbRQVjd1pJ2QYWkzix1i7xgqJrjAv1kM21CQl4+LRW0Ue4OnazpAzdL1K8
 pbXDOkDDJS4f50maggI0uEsX+7qpD6T52
X-Google-Smtp-Source: AGHT+IGW+6fGnDGkfK5+rR4qHvb2E4q1hynt3TqpyL1EjJ8R1jw+GtwDMRJv9bnuf1A3c/PSOYnwaIL6qSPR2zcJsR4=
X-Received: by 2002:a05:6602:164b:b0:7ea:f9ee:58ff with SMTP id
 ca18e2360f4ac-7eb5724ac13mr281016439f.14.1717766576141; Fri, 07 Jun 2024
 06:22:56 -0700 (PDT)
MIME-Version: 1.0
References: <CALXu0UfRxk5G=3OjbwoNFW7xSj1rHTNw5giS6YZb9p1gF5ceTA AT mail DOT gmail DOT com>
 <ZPHDrz7VedOBROtT AT calimero DOT vinschen DOT de>
 <CALXu0Ue9SyJod+0k24pQzs3KPg1RPquRfhN3tw3GYG-qMt_+DQ AT mail DOT gmail DOT com>
 <ZZv_K1NyxE-btqQt AT calimero DOT vinschen DOT de>
In-Reply-To: <ZZv_K1NyxE-btqQt@calimero.vinschen.de>
Date: Fri, 7 Jun 2024 15:22:29 +0200
Message-ID: <CAKAoaQ=-tRtO3TRWhYwsGJyp+XwgqwieXNomETTzTh_eNR=o_w@mail.gmail.com>
Subject: Re: Cygwin generates syscalls for *.lnk files on filesystems with
 native symlink support?
To: cygwin AT cygwin DOT com
X-Spam-Status: No, score=-1.6 required=5.0 tests=BAYES_00,
 FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,
 KAM_DMARC_STATUS, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2, SPF_HELO_NONE,
 SPF_PASS, TXREP,
 T_SCC_BODY_TEXT_LINE autolearn=no autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
 server2.sourceware.org
X-BeenThere: cygwin AT cygwin DOT com
X-Mailman-Version: 2.1.30
List-Id: General Cygwin discussions and problem reports <cygwin.cygwin.com>
List-Archive: <https://cygwin.com/pipermail/cygwin/>
List-Post: <mailto:cygwin AT cygwin DOT com>
List-Help: <mailto:cygwin-request AT cygwin DOT com?subject=help>
List-Subscribe: <https://cygwin.com/mailman/listinfo/cygwin>,
 <mailto:cygwin-request AT cygwin DOT com?subject=subscribe>
From: Roland Mainz via Cygwin <cygwin AT cygwin DOT com>
Reply-To: Roland Mainz <roland DOT mainz AT nrubsig DOT org>
Content-Type: text/plain; charset="utf-8"
Sender: "Cygwin" <cygwin-bounces+archive-cygwin=delorie DOT com AT cygwin DOT com>
Content-Transfer-Encoding: 8bit
X-MIME-Autoconverted: from base64 to 8bit by delorie.com id 457LAu911230213

On Mon, Jan 8, 2024 at 2:56 PM Corinna Vinschen via Cygwin
<cygwin AT cygwin DOT com> wrote:
>
> On Dec 18 13:04, Cedric Blancher via Cygwin wrote:
> > On Fri, 1 Sept 2023 at 13:00, Corinna Vinschen via Cygwin
> > <cygwin AT cygwin DOT com> wrote:
> > >
> > > On Sep  1 06:23, Cedric Blancher via Cygwin wrote:
> > > > Good morning!
> > > >
> > > > During a Cygwin 3.4.8-1.x86_64 debugging session I noticed something
> > > > odd when I looked at the network traffic generated by one of our
> > > > cluster nodes:
> > > > It seems that for each call to a tool (i.e. starting "sed" from
> > > > "bash") Cygwin searches for *.lnk files.
> > > >
> > > > Is this correct even when the filesystem in question has native
> > > > symlink support (e.g. NFS)?
> > >
> > > Yes.  During file handling, Cygwin doesn't know what filesystem a
> > > file is on until it could actually open the file and request file
> > > and filesystem info from the open handle.
> >
> > Why? If you have the path name you could lookup the (cached) mount
> > points, and determine the filesystem type. Same solution applies for
> > UNC paths, where you can easily lookup the filesystem type, and cache
> > it per-process or in Cygserver.
>
> No can do.
>
> We *do* filesystem info caching, but this requires to be able to
> open the file in the first place.  If the file exists, we're done, but
> if not, we have to evaluate all symbolic links in the path first.
> Simply reducing the path to the mount point and then translate it into
> a Windows path doesn't cut it.
>
> After the file (or its parent dir) could be opened, so we know the path
> is valid, we can fetch the filesystem info right from the open file
> handle.  At this point, we can shortcut the whole thing, reducing
> the three necessary calls to kernel functions to only the first one
> and skipping all the filesystem evaluation code.
>
> > > So if Cygwin couldn't open
> > > "foo" because the NtCreateFile call returned with status
> > > STATUS_OBJECT_PATH_NOT_FOUND or STATUS_OBJECT_NAME_NOT_FOUND, or
> > > STATUS_NO_SUCH_FILE, or one of the countless other status codes the
> > > kernel (or the driver) might return in case a file doesn't exist,
> > > it will tack on .lnk and .exe and, for historical reasons, .exe.lnk,
> > > and try again.
> >
> > Can this machinery please be turned off via CYGWIN env var option? As
> > discussed in https://www.mail-archive.com/cygwin AT cygwin DOT com/msg174547.html
> > this machinery causes very bad filesystem lookup performance, and it
> > would IMO a good idea to have an option to turn this off, and just
> > allow and expect native links (for NTFS, ReFS and NFS). Maybe CYGWIN
> > env var option winsymlinks_expect:native? winsymlinks_expect takes a :
> > seperated list which symlink types are to be expected.
>
> We won't add any more options to the CYGWIN env variable if it's not
> necessary, and there's no proof at all that this is necessary in this
> case.
>
> If the file exists as specified, there will be no further check for
> .exe or .lnk.  If it doesn't exist, *and* the return code from
> the kernel is STATUS_OBJECT_PATH_NOT_FOUND (or any one of a
> number of equivalent return values) we're done here and are going
> to fall back to checking for symlinks in the leading path components.
>
> If we got over that, the check for .exe is unavoidable.
[snip]

What about doing lookups in parallel, e.g. use |NtCreateFile()| with
an |IO_STATUS_BLOCK|, and do lookups for *.exe, *.lnk etc. in parallel
?
That might not help for all filesystem drivers, but some of them are
multithreaded (e.g. SMB, ms-nfs41-client, ...) and would greatly
benefit from that since the requests would go async&&parallel over the
network, instead of sequentially like it is now.

----

Bye,
Roland
-- 
  __ .  . __
 (o.\ \/ /.o) roland DOT mainz AT nrubsig DOT org
  \__\/\/__/  MPEG specialist, C&&JAVA&&Sun&&Unix programmer
  /O /==\ O\  TEL +49 641 3992797
 (;O/ \/ \O;)

-- 
Problem reports:      https://cygwin.com/problems.html
FAQ:                  https://cygwin.com/faq/
Documentation:        https://cygwin.com/docs.html
Unsubscribe info:     https://cygwin.com/ml/#unsubscribe-simple