X-Recipient: archive-cygwin AT delorie DOT com
DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 0E7BE385AC2E
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cygwin.com;
	s=default; t=1701426699;
	bh=4oouO3NIK3Mtn/LAEl6UgJP38GU25AfmK9PUUGvqve0=;
	h=References:In-Reply-To:Date:Subject:To:List-Id:List-Unsubscribe:
	 List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:
	 From;
	b=jhitbzHdV8ERB+HvLobeW0vZAwtDeUcrF2YesPjS6URN/LhpDdf53MDc5f0wPX0WQ
	 ns3D9qzk9ivIyXokM6PmYRvQe/pdDqoQXSJ3coNKPI/6B3ddBEcGAAEa7cG90EuTUW
	 WaZmCP4bPvxLHlnsjQf0VQ3/E6y7QBTJTdi94gT0=
X-Original-To: cygwin AT cygwin DOT com
Delivered-To: cygwin AT cygwin DOT com
DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 662403858C20
ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 662403858C20
ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1701426657; cv=none;
 b=pHqkMjNokTxtLgVp8OafdzuYOo6ilL8kSclN0LJIsZCOmcOMm4g7QzUfM4DGZ6vYli8baZBj8nKC58ST20Pj3ygFeXNkcFlYjJ90M1/Ur8hsFgVCTHOr4J3uwkV3jk7aR/Rfb+foPMu0NU/Km+NNQMsrBYA3aIke0iGhZAUC2jw=
ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key;
 t=1701426657; c=relaxed/simple;
 bh=dGp/Byl3q3wVK/cLmudCwNR9a4JdufA9HpUzElWtDcw=;
 h=DKIM-Signature:MIME-Version:From:Date:Message-ID:Subject:To;
 b=t+BvVeGQfpmMFo3HIjtDqYGoh97QF1u+j1gWLgX/diKKnJwN3IJSw9nXG6NRqyc2MqzUIsOUsxbc4RKb2U10CwNyh8E+1CZ1e9v7TmXZfTlnzhA3URpUuqPD6DWCFXlCzHkxwkP+js2wyPmMOz1dNeAmGlmdux4wYAunWW//RHo=
ARC-Authentication-Results: i=1; server2.sourceware.org
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20230601; t=1701426653; x=1702031453;
 h=content-transfer-encoding:to:subject:message-id:date:from
 :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
 :subject:date:message-id:reply-to;
 bh=yPtID/V0EmN3ZjyO4ZTY8okEOO7q8qU64Mc1dtQi0go=;
 b=FNmIYAFQ0ULoBO2Jsom82Gk4RgCXFIExOBPBZoIL3QB8hBKuKvcdKZSz2BA028qrtg
 4RtcQaljPav9Iqy4Sk4omeA2Rf8Divn7ukMvW9ceBajofNBM0nGdJnhrueMM3rUxOZg/
 AQg53ZQueBI+Bd86JDVpXA7x1rueb+XQZ+LBSpgf0wP/4TmPMjrQEwTg7aflwl6d3pF5
 IBirQuNjCMz7Efd45R0Xs+Ew/AI6PF3DGjeEdWIt0bOnTgo2mCWo9U5MghgLDYqR4ktI
 VgpnXvBDz4zaI2PUTCk9oEVL7BOHOJocul0UUGbfQyTk0IGYRGxo6XlceIuLoMhZBZyr
 Z6vw==
X-Gm-Message-State: AOJu0YyHZEOiXxFaF7cYVLf5cLBktdsu2+J7FiSl3v5XKP7r+OZA3jpQ
 zRpMhaFfT/JAkiIrQBK4CnSfc70gYMbqPYZsPP2jfl8GFsA=
X-Google-Smtp-Source: AGHT+IEhn4MCWabJXHgXiG+fQ35ql1sJTJ+DWMZDoKa9KoKeLeywVxi8i7rXk9ly3olsMmvdumzjjS2+r+URVOHgaPA=
X-Received: by 2002:a50:f602:0:b0:54c:676b:aa30 with SMTP id
 c2-20020a50f602000000b0054c676baa30mr137578edn.52.1701426177041; Fri, 01 Dec
 2023 02:22:57 -0800 (PST)
MIME-Version: 1.0
References: <CALXu0Uee2CHeCpthOKFmct+6HnEsKHbSzdK3Jyx8GsnYyt7Lbw AT mail DOT gmail DOT com>
 <ZWCCfqny4lC11bBJ AT calimero DOT vinschen DOT de>
 <ZWXBH_viPFuSF6be AT calimero DOT vinschen DOT de>
In-Reply-To: <ZWXBH_viPFuSF6be@calimero.vinschen.de>
Date: Fri, 1 Dec 2023 11:22:20 +0100
Message-ID: <CALXu0UcLK5AvrYDOg-9L2wFVhvjJuJre6X6GvL+5SyL744eygA@mail.gmail.com>
Subject: Re: Cygwin api to punch a hole into a file?
To: cygwin AT cygwin DOT com
X-Spam-Status: No, score=-0.4 required=5.0 tests=BAYES_00, DKIM_SIGNED,
 DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,
 SPF_HELO_NONE, SPF_PASS, TXREP,
 T_SCC_BODY_TEXT_LINE autolearn=unavailable autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
 server2.sourceware.org
X-BeenThere: cygwin AT cygwin DOT com
X-Mailman-Version: 2.1.30
List-Id: General Cygwin discussions and problem reports <cygwin.cygwin.com>
List-Archive: <https://cygwin.com/pipermail/cygwin/>
List-Post: <mailto:cygwin AT cygwin DOT com>
List-Help: <mailto:cygwin-request AT cygwin DOT com?subject=help>
List-Subscribe: <https://cygwin.com/mailman/listinfo/cygwin>,
 <mailto:cygwin-request AT cygwin DOT com?subject=subscribe>
From: Cedric Blancher via Cygwin <cygwin AT cygwin DOT com>
Reply-To: Cedric Blancher <cedric DOT blancher AT gmail DOT com>
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Sender: "Cygwin" <cygwin-bounces+archive-cygwin=delorie DOT com AT cygwin DOT com>

On Tue, 28 Nov 2023 at 11:30, Corinna Vinschen via Cygwin
<cygwin AT cygwin DOT com> wrote:
>
> On Nov 24 12:01, Corinna Vinschen via Cygwin wrote:
> > On Nov 23 23:36, Cedric Blancher via Cygwin wrote:
> > > Linux has fallocate(fd, FALLOC_FL_PUNCH_HOLE|FALLOC_FL_KEEP_SIZE, ...)
> > > to punch a hole into a file, i.e. deallocate the blocks given and make
> > > the file a "sparse file".
> >
> > We don't support the Linux-specific fallocate(2) call, only ftruncate(2)
> > and posix_fallocate(3).  Patches, as usual, thoughtfully considered.
>
> The next test release cygwin-3.5.0-0.485.g65831f88d6c4 introduces
> the Linux-specific fallocate(2) call.  Naturally we can't support
> all flags, but the following flag combinations are allowed:
>
> - 0                             same as posix_fallocate(3)
> - FALLOC_FL_KEEP_SIZE
> - FALLOC_FL_PUNCH_HOLE | FALLOC_FL_KEEP_SIZE
> - FALLOC_FL_ZERO_RANGE
> - FALLOC_FL_ZERO_RANGE | FALLOC_FL_KEEP_SIZE
>
> A few comments:
>
> - With 0 and FALLOC_FL_KEEP_SIZE, sparse blocks in the given range
>   are re-allocated as per the POSIX requirements.  For that, it uses
>   the same code as FALLOC_FL_ZERO_RANGE, but only for the holes within
>   the range.
>
> - With FALLOC_FL_KEEP_SIZE, over-allocation is done by setting
>   the allocation size of a file while keeping EOF the same.
>   However, in contrast to your typical Linux filesystem, over-
>   allocation on Windows filesystems is only temporary.  The
>   over-allocated blocks are returned to free blocks as soon as
>   the last HANDLE to the file is closed.
>
> - With FALLOC_FL_KEEP_SIZE, no over-allocation is performed on sparse
>   files.  The reason is that over-allocation on sparse files has no
>   effect on Windows.
>
> - FALLOC_FL_ZERO_RANGE is implemented as writing zeros to the
>   given range.  For parts of the range which are already allocated
>   data blocks, as much zeros are written as necessary.  For holes
>   in sparse files, only a single zero byte is written to the
>   hole per 64K chunk, which is the allocation granularity of
>   sparse files.
>
> - FALLOC_FL_ZERO_RANGE is NOT atomic.
>
> Please give it a try.

Corinna, THANK you!

Related question about commit "Cygwin: pwrite(2): sparsify file"
https://cygwin.com/git/?p=newlib-cygwin.git;a=blobdiff;f=winsup/cygwin/fhandler/disk_file.cc;h=c70afed49f1ecdb11812d31a9663d18c0a5f03f7;hp=b49b25c71ad030a9391db89a157cadcd095d4f36;hb=f64f3eced8e0f51753bc199f3ba96fab06a54848;hpb=114f89caff7b9b62b0b12bc2c6d143daf47b8042

I see the value of 128k (128*1024 bytes) quite often in your sparse
file commits. Can you please make this value a per filesystem tunable?
Not all filesystems have a 128k block/stripe size, and certainly most
filesystems have smaller minimum hole sizes than 128k (e.g. 512bytes
is common, ref pathconf _PC_MIN_HOLE_SIZE).

Ced
-- 
Cedric Blancher <cedric DOT blancher AT gmail DOT com>
[https://plus.google.com/u/0/+CedricBlancher/]
Institute Pasteur

-- 
Problem reports:      https://cygwin.com/problems.html
FAQ:                  https://cygwin.com/faq/
Documentation:        https://cygwin.com/docs.html
Unsubscribe info:     https://cygwin.com/ml/#unsubscribe-simple