DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org BE99E3858C39
Subject: Re: Curiosity about file access performance
To: Adam Dinwoodie <adam@dinwoodie.org>, cygwin <cygwin@cygwin.com>
References: <00895c47-8df9-1f17-baac-0b3560de9d1c@cs.umass.edu>
 <CA+kUOamNpqUSnFOnnd0SdSZJj7uohG6LaDL4jz95cttbM8XDVQ@mail.gmail.com>
From: Eliot Moss <moss@cs.umass.edu>
Message-ID: <a68b69d2-9624-281e-a561-816bef366c2f@cs.umass.edu>
Date: Fri, 29 Oct 2021 11:58:45 +0100
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101
 Thunderbird/68.12.1
MIME-Version: 1.0
In-Reply-To: <CA+kUOamNpqUSnFOnnd0SdSZJj7uohG6LaDL4jz95cttbM8XDVQ@mail.gmail.com>
Content-Language: en-US
Precedence: list
Reply-To: moss@cs.umass.edu
Content-Transfer-Encoding: 7bit
Content-Type: text/plain; charset="us-ascii"; Format="flowed"
Errors-To: cygwin-bounces+archive-cygwin=delorie.com@cygwin.com
Sender: "Cygwin" <cygwin-bounces+archive-cygwin=delorie.com@cygwin.com>

On 10/29/2021 11:44 AM, Adam Dinwoodie wrote:

> AIUI it's a fundamental part of the trade-offs that NTFS makes:
> compared to common Linux file systems like ext4, NTFS is much slower
> at things like parsing directory structures (which is a necessary part
> of opening any given file). In the same way that native Windows
> programs tend to use threading implementations that work differently
> to fork(), native Windows applications will also often much prefer
> large monolithic data files, where native *nix applications are much
> more likely to have lots of small files. As a result, for things that
> require opening lots of files, WSL (at least if you're using the
> native WSL disk, which will be a *nix disk image stored in a file,
> rather than files under /mnt/c or similar) will likely be quicker than
> a similar operation through Cygwin, as Cygwin will always be affected
> by those NTFS overheads.

Ah, that's interesting.  The files in question, that seem to be opened
(and *maybe* read) faster are in the *nix hierarchy, while my book files
are all in Windows (/mnt/c on WSL1).  So the huge speedup reading those
makes sense.  The speedup processing the rest still doesn't quite make
sense, unless maybe WSL1's parsed-directory caching is more effective
than Cygwin's or something.  (I assume something like that is going on,
to reduce conversions of directories to *nix format.)

Regards - Eliot

-- 
Problem reports:      https://cygwin.com/problems.html
FAQ:                  https://cygwin.com/faq/
Documentation:        https://cygwin.com/docs.html
Unsubscribe info:     https://cygwin.com/ml/#unsubscribe-simple