DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 4613938582A6
MIME-Version: 1.0
References: <CAAvCNcBZGepZMP9Q0D5ua+6ACftDOQEriqnuCbwg6umBPUA72Q AT mail DOT gmail DOT com>
 <CAAvCNcB0_0ZeujP23QZFZaDvVTh5rxbXJw4FP6uXNPErCgdZ2w AT mail DOT gmail DOT com>
 <07c7379e983c9f436ebf86e3818ca843 AT kylheku DOT com>
 <CANH4o6OjJJZQkbELt+H3WdAxQbLGZ1DL0ytevknRpbTO9sVUig AT mail DOT gmail DOT com>
 <4723aab7e2b331cb81946eff0fb4e862 AT kylheku DOT com>
In-Reply-To: <4723aab7e2b331cb81946eff0fb4e862@kylheku.com>
Date: Sun, 24 Dec 2023 01:47:09 +0100
Message-ID: <CAKAoaQnQ2eL9JJfn=CeJ06WujqgLdLVeXS7ojf7GvvmkB-KYoA@mail.gmail.com>
Subject: Re: rfe: CYGWIN fslinktypes option? Re: Catastrophic Cygwin find .
 -ls, grep performance on samba share compared to WSL&Linux
To: cygwin AT cygwin DOT com
From: Roland Mainz via Cygwin <cygwin AT cygwin DOT com>
Reply-To: Roland Mainz <roland DOT mainz AT nrubsig DOT org>
Content-Type: text/plain; charset="utf-8"
Sender: "Cygwin" <cygwin-bounces+archive-cygwin=delorie DOT com AT cygwin DOT com>
Content-Transfer-Encoding: 8bit

On Thu, Dec 21, 2023 at 9:32 PM Kaz Kylheku via Cygwin
<cygwin AT cygwin DOT com> wrote:
> On 2023-12-21 04:16, Martin Wege via Cygwin wrote:
> > On Wed, Dec 20, 2023 at 6:21 PM Kaz Kylheku via Cygwin
> > <cygwin AT cygwin DOT com> wrote:
[snip]
> > The root cause is IMO the extra Win32 syscalls (>= 3 per file lookup,
> > compared to 1 on Linux) to lookup the *.lnk and *.exe.lnk files on
> > filesystems which have native link support (NTFS, ReFS, SMBFS, NFS).
> > On SMBFS and NFS it hurts the most, because access latency is the
> > highest for networked filesystems.
>
> Could some intelligent caching be added there? (Discussion of
> associated invalidation problem in 3... 2.... 1... )

See below, basically a short-lived cache which is only valid for the
lifetime of the one POSIX function call would be OK...

> Can you discuss more details, so people don't have to dive into code
> to understand it? If we are accessing some file "foo", the application
> or user may actually be referring to a "foo.lnk" link. But in the
> happy case that "foo" exists, why would we bother looking for "foo.lnk"?
>
> If "foo" does not exist, but "foo.lnk" does, that could probably be
> cached, so that next time "foo" is accessed, we go straight for "foo.lnk",
> and keep using that while it exists.
>
> If someone has both "foo" and "foo.lnk" in the same directory,
> that's a bit of a degenerate case; how important is it to be "correct",
> anyway.

Question, mainly for Corinna:
Could the code be modified to use one |NtQueryDirectoryFile()| call
with a SINGLE pattern testing for { "foo", "foo.lnk", "foo.lnk.exe",
... } (instead of calling the kernel for each suffix independently)
and cache that information for the lifetime of the matching POSIX
function call ?
The idea is to reduce the number of userland<--->kernel roundstrips
from <n> to <1>, and filesystem drivers could be optimized even
further (for example if the network filesystem protocol supports file
name globbing...)

----

Bye,
Roland
-- 
  __ .  . __
 (o.\ \/ /.o) roland DOT mainz AT nrubsig DOT org
  \__\/\/__/  MPEG specialist, C&&JAVA&&Sun&&Unix programmer
  /O /==\ O\  TEL +49 641 3992797
 (;O/ \/ \O;)

-- 
Problem reports:      https://cygwin.com/problems.html
FAQ:                  https://cygwin.com/faq/
Documentation:        https://cygwin.com/docs.html
Unsubscribe info:     https://cygwin.com/ml/#unsubscribe-simple