DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 3037B3858D38
MIME-Version: 1.0
References: <CALXu0Udckh5XVXr23yaz1Fg1DoJ_12jE+XKo-jukDPoeL_vdZA@mail.gmail.com>
 <ZWm4xf0yfNjYqL-V@calimero.vinschen.de>
In-Reply-To: <ZWm4xf0yfNjYqL-V@calimero.vinschen.de>
Date: Wed, 6 Dec 2023 09:49:00 +0100
Message-ID: <CALXu0Ue3LjqdZKgK46=E-TgXBunUn6=LHqKi1Y5-N1hoAA_Fcg@mail.gmail.com>
Subject: Sparse file support for SMB by default? Re: Comment about "Cygwin:
 sparse support: enable automatic sparsifying of files on SSDs", extend
 feature to VMware/qemu disks?
To: cygwin@cygwin.com
From: Cedric Blancher via Cygwin <cygwin@cygwin.com>
Reply-To: Cedric Blancher <cedric.blancher@gmail.com>
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Sender: "Cygwin" <cygwin-bounces+archive-cygwin=delorie.com@cygwin.com>

On Fri, 1 Dec 2023 at 11:43, Corinna Vinschen via Cygwin
<cygwin@cygwin.com> wrote:
>
> On Dec  1 11:12, Cedric Blancher via Cygwin wrote:
> > Good morning!
> >
> > Small comment about 3.5.0 commit "Cygwin: sparse support: enable
> > automatic sparsifying of files on SSDs"
> > https://cygwin.com/git/?p=newlib-cygwin.git;a=commit;h=832e91422c4ae9b2dcd0c307779f3dd53ee9c0ac
> >
> > Can this functionality be extended to VMware and QEMU virtual disks, please?
>
> Not without a complete rewrite and lots of slowing down code.
>
>   VMware and QEMU virtual disks are not exposed to the OS as VMware and
>   QEMU virtual disks as far as the volume and filesystem information is
>   going.  They are just some arbitrary harddisks managed by some
>   arbitrary driver.

VMware emulates NVME SSD these days, so this should work, yes?

Primary concern is how to detect whether sparse file support is ON in
Cygwin for a specific filesystem? Maybe /sbin/mount without options
should have a sparse/nosparse mount flag to reflect what is used?

>
>   What Cygwin gets to see is that you want to access a file on some
>   disk.  It opens the file and fetches volume information.  So it finds:
>
>   - The filesystem returns the FILE_SUPPORTS_SPARSE_FILES
>
>     So we can create sparse files.
>
>   - The volume returns the SSINFO_FLAGS_NO_SEEK_PENALTY flag.
>
>     So we're on a drive having no seek penalty.  This is *probably*
>     an SSD, but may be some other type of drive.

What about SMB? SMB supports sparse files, and should certainly use
this by DEFAULT, as it makes many HPC apps operate much faster.
BUT: SSINFO_FLAGS_NO_SEEK_PENALTY is only for filesystems with a
drive, and not for network filesystem.

Same applies to the NFSv4.1 filesystem driver, I'm going to pester
Roland Mainz to add FILE_SUPPORTS_SPARSE_FILES support, so same issue
as SMB here.

Ced

-- 
Problem reports:      https://cygwin.com/problems.html
FAQ:                  https://cygwin.com/faq/
Documentation:        https://cygwin.com/docs.html
Unsubscribe info:     https://cygwin.com/ml/#unsubscribe-simple