DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org C963A3858D28
MIME-Version: 1.0
References: <CANH4o6Mxai4_C=d1P93Prrimb8_H=trTwm-Eg+WBwpomN3tNJw@mail.gmail.com>
 <ZgwFNde2z804koS_@calimero.vinschen.de>
In-Reply-To: <ZgwFNde2z804koS_@calimero.vinschen.de>
Date: Wed, 3 Apr 2024 00:35:27 +0200
Message-ID: <CANH4o6P8cts9TJgpdjR4mi+sj2YvuDa=d49XLcEVvYnRB81KRw@mail.gmail.com>
Subject: Re: Cygwin&Win32 file prefetch, block sizes?
To: cygwin@cygwin.com
From: Martin Wege via Cygwin <cygwin@cygwin.com>
Reply-To: Martin Wege <martin.l.wege@gmail.com>
Content-Type: text/plain; charset="utf-8"
Sender: "Cygwin" <cygwin-bounces+archive-cygwin=delorie.com@cygwin.com>
Content-Transfer-Encoding: 8bit

On Tue, Apr 2, 2024 at 3:17 PM Corinna Vinschen via Cygwin
<cygwin@cygwin.com> wrote:
>
> On Apr  2 02:04, Martin Wege via Cygwin wrote:
> > Hello,
> >
> > Is there any document which describes how Cygwin and Win32 file
> > prefetch and readahead work, and which sizes are used (e.g. always
> > read one full page even if only 16 bytes are requested?)?
>
> I'm not aware of any docs, but again, keep in mind that Cygwin is a
> usersapce DLL. We basically do what Windows does for low-level file
> access.
>
> > Quick /usr/bin/stat /etc/profile returns "IO Block: 65536". Does that
> > mean the file's block size is really 64k? Is this info per filesystem,
> > or hardcoded in Cygwin?
>
> Hardcoded in Cygwin since 2017, based on a discussion in terms of
> file access performance, especially when using stdio.h functions:
>
>   https://cygwin.com/cgit/newlib-cygwin/commit/?id=7bef7db5ccd9c

OUCH.

While I can understand the motivation, FAT32 on multi-GB-devices
having 64k block size, and Win32 API on Win95/98/ME/Win7 being
optimized to that insane block size, it is absolutely WRONG with
today's NTFS and even more so with ReFS. This only works if you stream
files, but as soon as you are doing random read/writes the performance
is terrible due to cache thrashing. That could explain the many
complaints about Cygwin's IO performance.

So, what can be done? I'm not a benchmarking guru, so I'd like to
propose to add a tunable called EXPERIMENTAL_PREFERRED_IO_BLKSIZE to
the CYGWIN env variable (marked as "experimental"), so the
benchmarking guys can do performance testing without recompiling
everything, get perf results for Cygwin 3.6, and decide what to do for
Cygwin 3.7.

Thanks,
Martin

-- 
Problem reports:      https://cygwin.com/problems.html
FAQ:                  https://cygwin.com/faq/
Documentation:        https://cygwin.com/docs.html
Unsubscribe info:     https://cygwin.com/ml/#unsubscribe-simple