X-Recipient: archive-cygwin AT delorie DOT com
DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org B3A133858413
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cygwin.com;
	s=default; t=1712097365;
	bh=Sg/A3udbWXq8I2Xct805yXMExB04sNUDTXW8j0W3u1I=;
	h=References:In-Reply-To:Date:Subject:To:List-Id:List-Unsubscribe:
	 List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:
	 From;
	b=u66h6m9A2dVLTXAlUPDWMznIK2iwISYuXeRUzSbTDeNXlHbaQjbsUzeuhKjYhhGCo
	 c8da8+9PwHe2CJYd1EBrNSX5KHvoVSFLvWjYSrMr82hoaX+4z5f/IDT8cpRbr4Bj78
	 qR3O51CX6N3BOj1jFnvs0z6dqJeAZkryjY36WfW8=
X-Original-To: cygwin AT cygwin DOT com
Delivered-To: cygwin AT cygwin DOT com
DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org C963A3858D28
ARC-Filter: OpenARC Filter v1.0.0 sourceware.org C963A3858D28
ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1712097343; cv=none;
 b=kFRPq/6vF8dbjzbSY0N5fg5yk/jzM9kWPhgC53+cIHcHeftin4pLwCfBQsOJxCCDCCiL9uFzIz5yMCEIlJhQUYKTsoKtQJJnZySxaiIwBU3eXfGK3EUIP0bj9aRzGlDcc6KHOcARWoszMMwXo7Vl4/n8NekGIdOlXDgmqFv43cE=
ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key;
 t=1712097343; c=relaxed/simple;
 bh=PqdJa5uPCmFlpobUK/Xdrav7DcQ3oVnH6ja8mJb//9s=;
 h=DKIM-Signature:MIME-Version:From:Date:Message-ID:Subject:To;
 b=fLTG3f2Fd/14oda8+lMQItAfIesYbf9xoV1OCgR5C3Q2mrMCWwq1ZOag4ZcN7ocB/xcdNXk5LxPxDCKI8gDy9duNrc7sPif2wbzXpD2QD1ELMxRe1pBlSE5MZdBk/I2O+wgGmz1xwWvb1RpPvAt0a9kyqGFBT24MqxPRCKN1qrY=
ARC-Authentication-Results: i=1; server2.sourceware.org
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20230601; t=1712097339; x=1712702139;
 h=content-transfer-encoding:to:subject:message-id:date:from
 :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
 :subject:date:message-id:reply-to;
 bh=mQV/NmotOyZWkRd+yY1Ur6ekZAN70VRA6oI14xBGtRQ=;
 b=Wp5ZauAW7ks2/6az0dJ5YbffxBxyaaeyPrS8sujk1w/7q+CDquG51TQLy7jGUBW7mu
 UIqg1+W1Px777fbgkS0mX8eDq/T+kph6Xk1SdQUqisxADCujKpIKrwFixzoGbccKFH6c
 gnRv0TS/fn6Kn5iRQlTM5jC6mI24z8G4q/xD1ZF5fFsGDYLuXbqQnHDBSRCXI+UwKS/u
 unF4C2DzBaZgQQduV/od5DOo6BvdAjzO16/ab09sKdFdOxc9VDZ7AGFynyKEjP0iTlPh
 T39KiA8rDH1lTTx+PAxYCxcP6a0c7yS5ATuMknl3m3r/YScbSGAhlyasPe8EL9qHE9+k
 Yihw==
X-Gm-Message-State: AOJu0YwdIlakdcDf6YFpRegPDeNNo9IrWfLpPe/bMpgZt0GHfsr4bW1x
 7Favjl66aB8FL922pR6cJ45T1UK/dUBu1XeH3iBuyQlBK62LCR20bo+35kUaDPzh89Q4pMakxqG
 TPbJsfU85aV5+xt+yWHKSOOSOMxIs7x8fkg4=
X-Google-Smtp-Source: AGHT+IEpmwYKre2w5V094IMDI8lkP5zRKoDzniaC8Ltecm8bCm5Bv24W0aC6wemj+wvP7ezR88MRQ0la+FOIhXtqQAs=
X-Received: by 2002:a05:6512:3f28:b0:515:a9d5:aab6 with SMTP id
 y40-20020a0565123f2800b00515a9d5aab6mr2797517lfa.18.1712097338678; Tue, 02
 Apr 2024 15:35:38 -0700 (PDT)
MIME-Version: 1.0
References: <CANH4o6Mxai4_C=d1P93Prrimb8_H=trTwm-Eg+WBwpomN3tNJw AT mail DOT gmail DOT com>
 <ZgwFNde2z804koS_ AT calimero DOT vinschen DOT de>
In-Reply-To: <ZgwFNde2z804koS_@calimero.vinschen.de>
Date: Wed, 3 Apr 2024 00:35:27 +0200
Message-ID: <CANH4o6P8cts9TJgpdjR4mi+sj2YvuDa=d49XLcEVvYnRB81KRw@mail.gmail.com>
Subject: Re: Cygwin&Win32 file prefetch, block sizes?
To: cygwin AT cygwin DOT com
X-Spam-Status: No, score=0.3 required=5.0 tests=BAYES_00, DKIM_SIGNED,
 DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,
 SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
 server2.sourceware.org
X-BeenThere: cygwin AT cygwin DOT com
X-Mailman-Version: 2.1.30
List-Id: General Cygwin discussions and problem reports <cygwin.cygwin.com>
List-Archive: <https://cygwin.com/pipermail/cygwin/>
List-Post: <mailto:cygwin AT cygwin DOT com>
List-Help: <mailto:cygwin-request AT cygwin DOT com?subject=help>
List-Subscribe: <https://cygwin.com/mailman/listinfo/cygwin>,
 <mailto:cygwin-request AT cygwin DOT com?subject=subscribe>
From: Martin Wege via Cygwin <cygwin AT cygwin DOT com>
Reply-To: Martin Wege <martin DOT l DOT wege AT gmail DOT com>
Content-Type: text/plain; charset="utf-8"
Sender: "Cygwin" <cygwin-bounces+archive-cygwin=delorie DOT com AT cygwin DOT com>
Content-Transfer-Encoding: 8bit
X-MIME-Autoconverted: from base64 to 8bit by delorie.com id 432Ma7Df2003516

On Tue, Apr 2, 2024 at 3:17 PM Corinna Vinschen via Cygwin
<cygwin AT cygwin DOT com> wrote:
>
> On Apr  2 02:04, Martin Wege via Cygwin wrote:
> > Hello,
> >
> > Is there any document which describes how Cygwin and Win32 file
> > prefetch and readahead work, and which sizes are used (e.g. always
> > read one full page even if only 16 bytes are requested?)?
>
> I'm not aware of any docs, but again, keep in mind that Cygwin is a
> usersapce DLL. We basically do what Windows does for low-level file
> access.
>
> > Quick /usr/bin/stat /etc/profile returns "IO Block: 65536". Does that
> > mean the file's block size is really 64k? Is this info per filesystem,
> > or hardcoded in Cygwin?
>
> Hardcoded in Cygwin since 2017, based on a discussion in terms of
> file access performance, especially when using stdio.h functions:
>
>   https://cygwin.com/cgit/newlib-cygwin/commit/?id=7bef7db5ccd9c

OUCH.

While I can understand the motivation, FAT32 on multi-GB-devices
having 64k block size, and Win32 API on Win95/98/ME/Win7 being
optimized to that insane block size, it is absolutely WRONG with
today's NTFS and even more so with ReFS. This only works if you stream
files, but as soon as you are doing random read/writes the performance
is terrible due to cache thrashing. That could explain the many
complaints about Cygwin's IO performance.

So, what can be done? I'm not a benchmarking guru, so I'd like to
propose to add a tunable called EXPERIMENTAL_PREFERRED_IO_BLKSIZE to
the CYGWIN env variable (marked as "experimental"), so the
benchmarking guys can do performance testing without recompiling
everything, get perf results for Cygwin 3.6, and decide what to do for
Cygwin 3.7.

Thanks,
Martin

-- 
Problem reports:      https://cygwin.com/problems.html
FAQ:                  https://cygwin.com/faq/
Documentation:        https://cygwin.com/docs.html
Unsubscribe info:     https://cygwin.com/ml/#unsubscribe-simple