DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org EE5853858C52
MIME-Version: 1.0
References: <CAAvCNcBZGepZMP9Q0D5ua+6ACftDOQEriqnuCbwg6umBPUA72Q@mail.gmail.com>
 <CAAvCNcB0_0ZeujP23QZFZaDvVTh5rxbXJw4FP6uXNPErCgdZ2w@mail.gmail.com>
 <07c7379e983c9f436ebf86e3818ca843@kylheku.com>
In-Reply-To: <07c7379e983c9f436ebf86e3818ca843@kylheku.com>
Date: Thu, 21 Dec 2023 13:16:40 +0100
Message-ID: <CANH4o6OjJJZQkbELt+H3WdAxQbLGZ1DL0ytevknRpbTO9sVUig@mail.gmail.com>
Subject: rfe: CYGWIN fslinktypes option? Re: Catastrophic Cygwin find . -ls,
 grep performance on samba share compared to WSL&Linux
To: cygwin@cygwin.com
From: Martin Wege via Cygwin <cygwin@cygwin.com>
Reply-To: Martin Wege <martin.l.wege@gmail.com>
Content-Type: text/plain; charset="utf-8"
Sender: "Cygwin" <cygwin-bounces+archive-cygwin=delorie.com@cygwin.com>
Content-Transfer-Encoding: 8bit

On Wed, Dec 20, 2023 at 6:21 PM Kaz Kylheku via Cygwin
<cygwin@cygwin.com> wrote:
>
> On 2023-12-17 22:22, Dan Shelton via Cygwin wrote:
> > It would be nice if someone from the Cygwin authors could assist me in
> > figuring out why this happens.
>
> Cygwin is famously slow; this is nothing new. We are grateful
> for Cygwin because it makes stuff work at all; if it were blazing
> fast that would be a bonus.
>
> E.g. git operations (clone, rebase, ...); ./configure scripts; ...: all
> run like molasses.
>
> The following is just my fast and loose opinion, shot from the hip,
> and possibly off or wrong, but it likely has to do with the layering.
> Cygwin's core API is based on a C library called Newlib. Cygwin bolts
> Newlib to Windows by means of an additional shim below Newlib that
> is based on C++ objects, where there is path munging going on and such,
> and that's where the Win32 calls get made. It's an additional abstraction.

I disagree with that. Ok, part of that is that the layering causes
more memory allocations and copies, but this is not the root cause.

The root cause is IMO the extra Win32 syscalls (>= 3 per file lookup,
compared to 1 on Linux) to lookup the *.lnk and *.exe.lnk files on
filesystems which have native link support (NTFS, ReFS, SMBFS, NFS).
On SMBFS and NFS it hurts the most, because access latency is the
highest for networked filesystems.

So my proposal would be to add an option ('fslinktypes') to the CYGWIN
environment variable to define which types of links are supported:
default 'all'. which is an shortcut for 'native,lnk,lnkexe'.
So in case people do not want 'lnk' link support they just add
CYGWIN+=' fslinktypes:native' to env, to turn off support for
lnk/lnk.exe style links, and be happy.

@Corinna Vinschen Would that be acceptable?

Thanks,
Martin

-- 
Problem reports:      https://cygwin.com/problems.html
FAQ:                  https://cygwin.com/faq/
Documentation:        https://cygwin.com/docs.html
Unsubscribe info:     https://cygwin.com/ml/#unsubscribe-simple