X-Recipient: archive-cygwin AT delorie DOT com
DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 969033858CDB
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cygwin.com;
	s=default; t=1712106642;
	bh=YoVBAEYUNYlZkWTQLVI/V0SlCYN/mibMvy8+lCUDoAQ=;
	h=Date:Subject:To:References:In-Reply-To:List-Id:List-Unsubscribe:
	 List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:
	 From;
	b=Q86A7h38JFtwrxsDGSCI0xoolTysIjveRNi6QdV98hI1bdkd40j7l/im4efE6mBRR
	 lWhqJMYZGR7F8yY3afdNau1QEtJAmW/Z3zdogUuLHiyERzxpph5Sx6E+juqQMkFrry
	 TLGunB1x7DSE7hC4zHFbE7Vu1srIpilCniwo82+c=
X-Original-To: cygwin AT cygwin DOT com
Delivered-To: cygwin AT cygwin DOT com
DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org C97443858CD1
ARC-Filter: OpenARC Filter v1.0.0 sourceware.org C97443858CD1
ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1712106620; cv=none;
 b=kTnCrP/DOm9uRAbCtYbDpD8L8YMWHiHk2wlNWDZJYR+ZsQYfcs/YH20BWrL1K7O8T5roQF3TlvVisSi2BYelOP5hTzbmLeiOlnGOgtMC8IagTLb/OWAjYbsicBDmqq2yO2VGmSKlyMmrLMegwpHUpQUmWCSZbvj1dcNr+7Qzgx0=
ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key;
 t=1712106620; c=relaxed/simple;
 bh=P924+SIVYAKgTMnizAvmDhELjx1aVME4f1wVsmSQ0VQ=;
 h=Message-ID:Date:MIME-Version:Subject:To:From;
 b=XaYGVgjWkeAaKv4SgkDmdJz4xTZGkABms1oZiecFHPMsIAt0CmcWyt8k3wXdF6Ko0nzUudg0vyUR9zju0xMQKKXVMS51PyBK93hGPBR/CIwCPtIrclfkiJJfH2AOvXxi4J+as1bV8W5riDAGNMx5GC3YlFLlXOAnmVcZ/XozHYg=
ARC-Authentication-Results: i=1; server2.sourceware.org
Message-ID: <af940815-81af-4ca5-8198-072d09dad23e@maxrnd.com>
Date: Tue, 2 Apr 2024 18:10:16 -0700
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: Cygwin&Win32 file prefetch, block sizes?
To: cygwin AT cygwin DOT com
References: <CANH4o6Mxai4_C=d1P93Prrimb8_H=trTwm-Eg+WBwpomN3tNJw AT mail DOT gmail DOT com>
 <ZgwFNde2z804koS_ AT calimero DOT vinschen DOT de>
 <CANH4o6P8cts9TJgpdjR4mi+sj2YvuDa=d49XLcEVvYnRB81KRw AT mail DOT gmail DOT com>
Content-Language: en-US
In-Reply-To: <CANH4o6P8cts9TJgpdjR4mi+sj2YvuDa=d49XLcEVvYnRB81KRw@mail.gmail.com>
X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00, KAM_DMARC_STATUS,
 SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
 server2.sourceware.org
X-BeenThere: cygwin AT cygwin DOT com
X-Mailman-Version: 2.1.30
Precedence: list
List-Id: General Cygwin discussions and problem reports <cygwin.cygwin.com>
List-Unsubscribe: <https://cygwin.com/mailman/options/cygwin>,
 <mailto:cygwin-request AT cygwin DOT com?subject=unsubscribe>
List-Archive: <https://cygwin.com/pipermail/cygwin/>
List-Post: <mailto:cygwin AT cygwin DOT com>
List-Help: <mailto:cygwin-request AT cygwin DOT com?subject=help>
List-Subscribe: <https://cygwin.com/mailman/listinfo/cygwin>,
 <mailto:cygwin-request AT cygwin DOT com?subject=subscribe>
From: Mark Geisert via Cygwin <cygwin AT cygwin DOT com>
Reply-To: Mark Geisert <mark AT maxrnd DOT com>
Content-Type: text/plain; charset="utf-8"; Format="flowed"
Errors-To: cygwin-bounces+archive-cygwin=delorie DOT com AT cygwin DOT com
Sender: "Cygwin" <cygwin-bounces+archive-cygwin=delorie DOT com AT cygwin DOT com>
Content-Transfer-Encoding: 8bit
X-MIME-Autoconverted: from base64 to 8bit by delorie.com id 4331AiwX2050545

On 4/2/2024 3:35 PM, Martin Wege via Cygwin wrote:
> On Tue, Apr 2, 2024 at 3:17 PM Corinna Vinschen via Cygwin
> <cygwin AT cygwin DOT com> wrote:
>>
>> On Apr  2 02:04, Martin Wege via Cygwin wrote:
>>> Hello,
>>>
>>> Is there any document which describes how Cygwin and Win32 file
>>> prefetch and readahead work, and which sizes are used (e.g. always
>>> read one full page even if only 16 bytes are requested?)?
>>
>> I'm not aware of any docs, but again, keep in mind that Cygwin is a
>> usersapce DLL. We basically do what Windows does for low-level file
>> access.
>>
>>> Quick /usr/bin/stat /etc/profile returns "IO Block: 65536". Does that
>>> mean the file's block size is really 64k? Is this info per filesystem,
>>> or hardcoded in Cygwin?
>>
>> Hardcoded in Cygwin since 2017, based on a discussion in terms of
>> file access performance, especially when using stdio.h functions:
>>
>>    https://cygwin.com/cgit/newlib-cygwin/commit/?id=7bef7db5ccd9c
> 
> OUCH.
> 
> While I can understand the motivation, FAT32 on multi-GB-devices
> having 64k block size, and Win32 API on Win95/98/ME/Win7 being
> optimized to that insane block size, it is absolutely WRONG with
> today's NTFS and even more so with ReFS. This only works if you stream
> files, but as soon as you are doing random read/writes the performance
> is terrible due to cache thrashing. That could explain the many
> complaints about Cygwin's IO performance.

No comment.

> So, what can be done? I'm not a benchmarking guru, so I'd like to
> propose to add a tunable called EXPERIMENTAL_PREFERRED_IO_BLKSIZE to
> the CYGWIN env variable (marked as "experimental"), so the
> benchmarking guys can do performance testing without recompiling
> everything, get perf results for Cygwin 3.6, and decide what to do for
> Cygwin 3.7.

That kind of experiment is what folks who can build their own 
cygwin1.dll might do.  I doubt we'd want to make a run-time global disk 
I/O strategy changer available like this, even temporarily.

What could make sense is enhancing Cygwin's posix_fadvise() to support 
POSIX_FADV_RANDOM getting mapped to Windows' FILE_RANDOM_ACCESS flag.
Something like this is currently done for POSIX_FADV_SEQUENTIAL -> 
FILE_SEQUENTIAL_ONLY.  These are per-filedescriptor adjustments and due 
to Windows limitations would apply to a whole file rather than having 
the POSIX behavior of being settable for a byte range within a file.

SHTDI, PTC, and all that :-).

..mark


-- 
Problem reports:      https://cygwin.com/problems.html
FAQ:                  https://cygwin.com/faq/
Documentation:        https://cygwin.com/docs.html
Unsubscribe info:     https://cygwin.com/ml/#unsubscribe-simple