X-Recipient: archive-cygwin AT delorie DOT com
DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 855BC38618AD
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cygwin.com;
	s=default; t=1703190790;
	bh=1H6h+ypaS+fqodUl39yUbQwopy19sKb+x3Kp7B77pDc=;
	h=Date:To:Cc:Subject:In-Reply-To:References:List-Id:
	 List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe:
	 From:Reply-To:From;
	b=gsaC8UmQJrCJeSeia2SLhNiV8fYm7+q5LPOkuEbJLUOs3aHo36CTW9mlFKapNwgFd
	 7l4JFVOoUWdzfFE+RcR5Xa+Pmj9QX/Vezm04DMrSZ+otOgEf9ka0ZG5VgoK2IJs276
	 1EpX+wHvDrH/T5ixCpujnBm+FMOxPGcF9iTRWlMg=
X-Original-To: cygwin AT cygwin DOT com
Delivered-To: cygwin AT cygwin DOT com
DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 9D2D23858C56
ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 9D2D23858C56
ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1703190776; cv=none;
 b=LEi2jojP5qTF236NhhnZpb985rHqfPXjkyhFliRY7Gw28rU6/pAyzTANPRM8eSSjWu/5dI1Do7gKmKd1ic1sYgZaIat3Zl/VfgZIecRQ2dfusVxpHtBUfiei3NQXm0OLd6AS7VEb2mCU4YpqG6XTdgqUzUc6E9gNzqZRKSN8lLg=
ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key;
 t=1703190776; c=relaxed/simple;
 bh=yQTG3SABJp8esGLrtB0PlIY3kFtODAdJAXbjsT2f4BY=;
 h=DKIM-Signature:MIME-Version:Date:From:To:Subject:Message-ID;
 b=lEZ+EfkF/OMA+AvTmopRPSAyCHYJlDrGw0lT78aRKXDILzVstwDJsyqslq1aHp9lTNVaPDY6rMhP+e1UNvLpoXqjYZAusuq4S3vpiJRbrED5nh1CzrpJfGLHbNI6vdLzgdqGOnQ647QPV1Ncy/x2DYyVZKr6nxfvgYdOft4h160=
ARC-Authentication-Results: i=1; server2.sourceware.org
X-Authority-Analysis: v=2.4 cv=Cousz10D c=1 sm=1 tr=0 ts=6584a0f6
 a=pMSlDXUwMa7SJ1EIez8PdQ==:117 a=pMSlDXUwMa7SJ1EIez8PdQ==:17
 a=xqWC_Br6kY4A:10 a=IkcTkHD0fZMA:10 a=e2cXIFwxEfEA:10 a=w_pzkKWiAAAA:8
 a=eZNnFSucP04RNPoSW7UA:9 a=QEXdDO2ut3YA:10 a=sRI3_1zDfAgwuvI8zelB:22
MIME-Version: 1.0
Date: Thu, 21 Dec 2023 12:32:51 -0800
To: Martin Wege <martin DOT l DOT wege AT gmail DOT com>
Cc: cygwin AT cygwin DOT com
Subject: Re: rfe: CYGWIN fslinktypes option? Re: Catastrophic Cygwin find .
 -ls, grep performance on samba share compared to WSL&Linux
In-Reply-To: <CANH4o6OjJJZQkbELt+H3WdAxQbLGZ1DL0ytevknRpbTO9sVUig@mail.gmail.com>
References: <CAAvCNcBZGepZMP9Q0D5ua+6ACftDOQEriqnuCbwg6umBPUA72Q AT mail DOT gmail DOT com>
 <CAAvCNcB0_0ZeujP23QZFZaDvVTh5rxbXJw4FP6uXNPErCgdZ2w AT mail DOT gmail DOT com>
 <07c7379e983c9f436ebf86e3818ca843 AT kylheku DOT com>
 <CANH4o6OjJJZQkbELt+H3WdAxQbLGZ1DL0ytevknRpbTO9sVUig AT mail DOT gmail DOT com>
User-Agent: Roundcube Webmail/1.4.15
Message-ID: <4723aab7e2b331cb81946eff0fb4e862@kylheku.com>
X-Sender: kaz AT kylheku DOT com
X-CMAE-Envelope: MS4xfDUSa6PrHD1PN+vfJHKvbEGzRUUMpbncIvaqzClTgMNMlts7IREelzTDr8m6gTkvZmfHHDHKf72k3640Vs23HDBbxPdT80z2XkRuhOjerhzBHGkMynJI
 QucdsefXcir3tXb2BifEVBv8V/beBuVYXLFv7bfgU6GmCMXogccuhymOZLJoaFeyOO6HLXau/F7CZhRefZP9XQYQRL5rWeqAl1k=
X-Spam-Status: No, score=-3.0 required=5.0 tests=BAYES_00, DKIM_SIGNED,
 DKIM_VALID, DKIM_VALID_EF, HEADER_FROM_DIFFERENT_DOMAINS, RCVD_IN_DNSWL_LOW,
 RCVD_IN_MSPIKE_H4, RCVD_IN_MSPIKE_WL, SPF_HELO_NONE, SPF_PASS, TXREP,
 T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
 server2.sourceware.org
X-BeenThere: cygwin AT cygwin DOT com
X-Mailman-Version: 2.1.30
Precedence: list
List-Id: General Cygwin discussions and problem reports <cygwin.cygwin.com>
List-Unsubscribe: <https://cygwin.com/mailman/options/cygwin>,
 <mailto:cygwin-request AT cygwin DOT com?subject=unsubscribe>
List-Archive: <https://cygwin.com/pipermail/cygwin/>
List-Post: <mailto:cygwin AT cygwin DOT com>
List-Help: <mailto:cygwin-request AT cygwin DOT com?subject=help>
List-Subscribe: <https://cygwin.com/mailman/listinfo/cygwin>,
 <mailto:cygwin-request AT cygwin DOT com?subject=subscribe>
From: Kaz Kylheku via Cygwin <cygwin AT cygwin DOT com>
Reply-To: Kaz Kylheku <kaz AT kylheku DOT com>
Content-Type: text/plain; charset="utf-8"
Errors-To: cygwin-bounces+archive-cygwin=delorie DOT com AT cygwin DOT com
Sender: "Cygwin" <cygwin-bounces+archive-cygwin=delorie DOT com AT cygwin DOT com>
Content-Transfer-Encoding: 8bit
X-MIME-Autoconverted: from base64 to 8bit by delorie.com id 3BLKXBYx005216

On 2023-12-21 04:16, Martin Wege via Cygwin wrote:
> On Wed, Dec 20, 2023 at 6:21 PM Kaz Kylheku via Cygwin
> <cygwin AT cygwin DOT com> wrote:
>>
>> On 2023-12-17 22:22, Dan Shelton via Cygwin wrote:
>> > It would be nice if someone from the Cygwin authors could assist me in
>> > figuring out why this happens.
>>
>> Cygwin is famously slow; this is nothing new. We are grateful
>> for Cygwin because it makes stuff work at all; if it were blazing
>> fast that would be a bonus.
>>
>> E.g. git operations (clone, rebase, ...); ./configure scripts; ...: all
>> run like molasses.
>>
>> The following is just my fast and loose opinion, shot from the hip,
>> and possibly off or wrong, but it likely has to do with the layering.
>> Cygwin's core API is based on a C library called Newlib. Cygwin bolts
>> Newlib to Windows by means of an additional shim below Newlib that
>> is based on C++ objects, where there is path munging going on and such,
>> and that's where the Win32 calls get made. It's an additional abstraction.
> 
> I disagree with that. Ok, part of that is that the layering causes
> more memory allocations and copies, but this is not the root cause.

I seem to recall that most operations that take a path argument have
to convert the path from Cygwin to Win32, and I think that also involves
going from 8 bit to UTF-16 also. That's gotta hurt a bit.
 
> The root cause is IMO the extra Win32 syscalls (>= 3 per file lookup,
> compared to 1 on Linux) to lookup the *.lnk and *.exe.lnk files on
> filesystems which have native link support (NTFS, ReFS, SMBFS, NFS).
> On SMBFS and NFS it hurts the most, because access latency is the
> highest for networked filesystems.

Could some intelligent caching be added there? (Discussion of
associated invalidation problem in 3... 2.... 1... )

Can you discuss more details, so people don't have to dive into code
to understand it? If we are accessing some file "foo", the application
or user may actually be referring to a "foo.lnk" link. But in the
happy case that "foo" exists, why would we bother looking for "foo.lnk"?

If "foo" does not exist, but "foo.lnk" does, that could probably be
cached, so that next time "foo" is accessed, we go straight for "foo.lnk",
and keep using that while it exists.

If someone has both "foo" and "foo.lnk" in the same directory,
that's a bit of a degenerate case; how important is it to be "correct",
anyway.

> So my proposal would be to add an option ('fslinktypes') to the CYGWIN
> environment variable to define which types of links are supported:
> default 'all'. which is an shortcut for 'native,lnk,lnkexe'.
> So in case people do not want 'lnk' link support they just add
> CYGWIN+=' fslinktypes:native' to env, to turn off support for
> lnk/lnk.exe style links, and be happy.

So this complements the winsymlinks option? winsymlinks has to do
with how the Cygwin DLL creates symbolic links, whereas this has to do
with what objects are recognized as links.

The implementation would probably want to compare fslinktypes
and winsymlinks to make sure they are harmonized together;
if winsymlinks tells Cygwin to make .lnk files, but then fslinktypes
banishes them, that's something diagnosable somehow.

-- 
Problem reports:      https://cygwin.com/problems.html
FAQ:                  https://cygwin.com/faq/
Documentation:        https://cygwin.com/docs.html
Unsubscribe info:     https://cygwin.com/ml/#unsubscribe-simple