X-Recipient:	archive-cygwin AT delorie DOT com
DKIM-Filter:	OpenDKIM Filter v2.11.0 sourceware.org E1B70385AE75
DKIM-Signature:	v=1; a=rsa-sha256; c=relaxed/relaxed; d=cygwin.com;
	s=default; t=1701855541;
	bh=XOu8lxBqOVHfDruOMrdukfK75nyH3+ZpxtASQQxqQCA=;
	h=Date:To:Subject:References:In-Reply-To:List-Id:List-Unsubscribe:
	List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc:
	From;
	b=tX015Kn27r5mbCi53NSdqQ6iUTDSb9ptPwU/ZuZRaVp6QlO1OE0W3A/X0cAO86dtI
	3WQzlid7FYjq+NSf7tJGaQBmlS5XcR04nlD95zYftTWeRZ9/VhUFCwXMtCqoKeZm5L
	/7uQWFKoNYYPuk+Rw6Y7Ed8wxArQeAjYmp7zE7eE=
X-Original-To:	cygwin AT cygwin DOT com
Delivered-To:	cygwin AT cygwin DOT com
DKIM-Filter:	OpenDKIM Filter v2.11.0 sourceware.org 5EA9D3858C54
Date:	Wed, 6 Dec 2023 10:38:44 +0100
To:	cygwin AT cygwin DOT com
Subject:	Re: Sparse file support for SMB by default? Re: Comment about
	"Cygwin: sparse support: enable automatic sparsifying of files on SSDs",
	extend feature to VMware/qemu disks?
Message-ID:	<ZXBBJJ18Q3y4CSBx@calimero.vinschen.de>
Mail-Followup-To:	cygwin AT cygwin DOT com
References:	<CALXu0Udckh5XVXr23yaz1Fg1DoJ_12jE+XKo-jukDPoeL_vdZA AT mail DOT gmail DOT com>
	<ZWm4xf0yfNjYqL-V AT calimero DOT vinschen DOT de>
	<CALXu0Ue3LjqdZKgK46=E-TgXBunUn6=LHqKi1Y5-N1hoAA_Fcg AT mail DOT gmail DOT com>
MIME-Version:	1.0
In-Reply-To:	<CALXu0Ue3LjqdZKgK46=E-TgXBunUn6=LHqKi1Y5-N1hoAA_Fcg@mail.gmail.com>
X-BeenThere:	cygwin AT cygwin DOT com
X-Mailman-Version:	2.1.30
List-Id:	General Cygwin discussions and problem reports <cygwin.cygwin.com>
List-Unsubscribe:	<https://cygwin.com/mailman/options/cygwin>,
	<mailto:cygwin-request AT cygwin DOT com?subject=unsubscribe>
List-Archive:	<https://cygwin.com/pipermail/cygwin/>
List-Post:	<mailto:cygwin AT cygwin DOT com>
List-Help:	<mailto:cygwin-request AT cygwin DOT com?subject=help>
List-Subscribe:	<https://cygwin.com/mailman/listinfo/cygwin>,
	<mailto:cygwin-request AT cygwin DOT com?subject=subscribe>
From:	Corinna Vinschen via Cygwin <cygwin AT cygwin DOT com>
Reply-To:	cygwin AT cygwin DOT com
Cc:	Corinna Vinschen <corinna-cygwin AT cygwin DOT com>
Errors-To:	cygwin-bounces+archive-cygwin=delorie DOT com AT cygwin DOT com
Sender:	"Cygwin" <cygwin-bounces+archive-cygwin=delorie DOT com AT cygwin DOT com>

On Dec  6 09:49, Cedric Blancher via Cygwin wrote:
> On Fri, 1 Dec 2023 at 11:43, Corinna Vinschen via Cygwin
> VMware emulates NVME SSD these days, so this should work, yes?

It doesn't matter, see below.

> Primary concern is how to detect whether sparse file support is ON in
> Cygwin for a specific filesystem? Maybe /sbin/mount without options
> should have a sparse/nosparse mount flag to reflect what is used?

What concern?  I explained exactly what happens, and it all revolves
around the fact that a filesystem sets the FILE_SUPPORTS_SPARSE_FILES
flag or not.  This is the only interesting fact.  Nothing else
matters.

If you want to see if a filesystem supports that flag, just use the
/usr/lib/csih/getVolInfo tool:

  $ /usr/lib/csih/getVolInfo .
  Device Type        : 7
  Characteristics    : 20020
  Volume Name        : <root>
  Serial Number      : 2292001085
  Max Filenamelength : 255
  Filesystemname     : <NTFS>
  Flags              : 3e72eff
    FILE_CASE_SENSITIVE_SEARCH        : TRUE
    FILE_CASE_PRESERVED_NAMES         : TRUE
    FILE_UNICODE_ON_DISK              : TRUE
    FILE_PERSISTENT_ACLS              : TRUE
    FILE_FILE_COMPRESSION             : TRUE
    FILE_VOLUME_QUOTAS                : TRUE
    FILE_SUPPORTS_SPARSE_FILES        : TRUE
    FILE_SUPPORTS_REPARSE_POINTS      : TRUE
    FILE_SUPPORTS_REMOTE_STORAGE      : FALSE
    FILE_RETURNS_CLEANUP_RESULT_INFO  : TRUE
    FILE_SUPPORTS_POSIX_UNLINK_RENAME : TRUE
    FILE_VOLUME_IS_COMPRESSED         : FALSE
    FILE_SUPPORTS_OBJECT_IDS          : TRUE
    FILE_SUPPORTS_ENCRYPTION          : TRUE
    FILE_NAMED_STREAMS                : TRUE
    FILE_READ_ONLY_VOLUME             : FALSE
    FILE_SEQUENTIAL_WRITE_ONCE        : FALSE
    FILE_SUPPORTS_TRANSACTIONS        : TRUE
    FILE_SUPPORTS_HARD_LINKS          : TRUE
    FILE_SUPPORTS_EXTENDED_ATTRIBUTES : TRUE
    FILE_SUPPORTS_OPEN_BY_FILE_ID     : TRUE
    FILE_SUPPORTS_USN_JOURNAL         : TRUE
    FILE_SUPPORTS_INTEGRITY_STREAMS   : FALSE
    FILE_SUPPORTS_BLOCK_REFCOUNTING   : FALSE
    FILE_SUPPORTS_SPARSE_VDL          : FALSE
    FILE_DAX_VOLUME                   : FALSE
    FILE_SUPPORTS_GHOSTING            : FALSE

> >   What Cygwin gets to see is that you want to access a file on some
> >   disk.  It opens the file and fetches volume information.  So it finds:
> >
> >   - The filesystem returns the FILE_SUPPORTS_SPARSE_FILES
> >
> >     So we can create sparse files.
> >
> >   - The volume returns the SSINFO_FLAGS_NO_SEEK_PENALTY flag.
> >
> >     So we're on a drive having no seek penalty.  This is *probably*
> >     an SSD, but may be some other type of drive.
> 
> What about SMB? SMB supports sparse files, and should certainly use
> this by DEFAULT, as it makes many HPC apps operate much faster.
> BUT: SSINFO_FLAGS_NO_SEEK_PENALTY is only for filesystems with a
> drive, and not for network filesystem.
> 
> Same applies to the NFSv4.1 filesystem driver, I'm going to pester
> Roland Mainz to add FILE_SUPPORTS_SPARSE_FILES support, so same issue
> as SMB here.

Again, the filesystem doesn't matter.  It either sets the
FILE_SUPPORTS_SPARSE_FILES flag or not, as simple as that.

If it does, you can create sparse files with chattr +S, or you can rely
on the lseek/ftruncate/posix_fallocate automatism, or you stamp a hole
into the file with fallocate(FALLOC_FL_PUNCH_HOLE).

The *only* difference is if you have to use the "sparse" mount option or
not.

Basically, with 3.4, you always have to set the "sparse" mount option,
with 3.5, on local SSDs you don't.  I don't see a problem here.


Corinna

-- 
Problem reports:      https://cygwin.com/problems.html
FAQ:                  https://cygwin.com/faq/
Documentation:        https://cygwin.com/docs.html
Unsubscribe info:     https://cygwin.com/ml/#unsubscribe-simple

- Raw text -