DMARC-Filter: OpenDMARC Filter v1.4.2 delorie.com 50G0FwvN4109687
Authentication-Results: delorie.com; dmarc=pass (p=none dis=none) header.from=cygwin.com
Authentication-Results: delorie.com; spf=pass smtp.mailfrom=cygwin.com
DKIM-Filter: OpenDKIM Filter v2.11.0 delorie.com 50G0FwvN4109687
Authentication-Results: delorie.com;
	dkim=pass (1024-bit key, unprotected) header.d=cygwin.com header.i=@cygwin.com header.a=rsa-sha256 header.s=default header.b=O2lbWAwn
X-Recipient: archive-cygwin AT delorie DOT com
DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org C3DD13857BA5
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cygwin.com;
	s=default; t=1736986557;
	bh=yncslWERqkzqDoOSXOnzuObNi0LP29a2Yj+vsNNyPY0=;
	h=Date:Subject:To:References:In-Reply-To:List-Id:List-Unsubscribe:
	 List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc:
	 From;
	b=O2lbWAwnaXMf6bxKgZNDbiAv8hxNmxp1CqNrSWLzZBopJdV7aXdN8TZlUt9HQxjVa
	 e8ZI3aAN83ct3R5rUJJ4QRLXjPjkTQjqyWE+2NFnxEB6pHhfsWeStvGLRs4wqV6JU9
	 qaVERDvViK3TNAI+MaxM9dDYfb5gsIH88BtqIx78=
X-Original-To: cygwin AT cygwin DOT com
Delivered-To: cygwin AT cygwin DOT com
DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 912D63857C68
ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 912D63857C68
ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1736986514; cv=none;
 b=kAsLims2KqUPmPyrQjZTtw3Nb+kFTlwAZJpDHfz0VQqSfblDtUidPZWX/F5Z8Jeg4Spj1UV27CL9GhnIB74mgRE+R3H0wvF7ReUxUwqHioZ77Na1o12zCbsVBogPaRgTzoCgRBcoVOBupycLX+v72NbzEzzUclPwt2UJccsj+Tk=
ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key;
 t=1736986514; c=relaxed/simple;
 bh=bFWHGa81Tc3715rypCfiRpDwD04xJ/BzP4oDPq0zScE=;
 h=Message-ID:Date:MIME-Version:From:Subject:To:DKIM-Signature;
 b=aeOrjqCR++Jtq7GZrqSGfzlA7vC/+4QsEy0Es/As0m9beafIyrMsyZnvg7U3kBZCp2xDRtH0SrLWjpIm4q9sliHX/prug3wpy3L7CGORiR1N+49Hl5rbKD5YMzrY/ec3cbNEx7X7zJVWrlbbbmY+KPfBuSIBE3OblUDtcqO/jBc=
ARC-Authentication-Results: i=1; server2.sourceware.org
DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 912D63857C68
Message-ID: <44405d41-b945-4dd2-bbee-8dd37733f40b@systematicsw.ab.ca>
Date: Wed, 15 Jan 2025 17:15:11 -0700
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: Cygwin 3.6 possible issue handling compressed .pdb files on SSD?
Content-Language: en-CA
To: cygwin AT cygwin DOT com
References: <CAKAoaQmwna7sSfpYFn0OUsqpdXAgJ5FyaNkhKv6K+otr0Oa5QA AT mail DOT gmail DOT com>
 <4712dcf7-1d4d-4d27-b7c1-b705d3a0a553 AT SystematicSW DOT ab DOT ca>
 <CAKAoaQ=aj+xB2J65Rgv1tV4oSK9fq0tk8Bq6ZoXAoOG_hBR5eA AT mail DOT gmail DOT com>
 <bf9d2afd-68e4-46b3-a9e6-37de7ffb8fb1 AT SystematicSW DOT ab DOT ca>
 <Z4fvtuywpUttG-66 AT calimero DOT vinschen DOT de>
Organization: Systematic Software
In-Reply-To: <Z4fvtuywpUttG-66@calimero.vinschen.de>
X-Rspamd-Queue-Id: 5C8C132
X-Rspamd-Server: rspamout07
X-Stat-Signature: ipajnmwrs71m1psyqc5iwmjtugom5m7t
X-Session-Marker: 427269616E2E496E676C69734053797374656D6174696353572E61622E6361
X-Session-ID: U2FsdGVkX1835Tk/8kUK1es6nRAkUfrSwbWRBDWFojA=
X-HE-Tag: 1736986512-388224
X-HE-Meta: U2FsdGVkX1/QNJXUaD0gIHd4MVfZmKWsbg48PE8xbO5PyAMaKmXvDDEmvGgwMC+IvQFOebWovwtJCPj5MJQWNZayzQ/enQWmHC0zbTVZPra6hac2VUQd8LdkOhVHNVOiLGbEpcKD0uL8UpWXA8zeN56wl+2Ts1/Ab1uc7N696qKtSELkHozKYfpW1PizZv7OcCKFI9r1F3VhyXQM6cGM8BXhSOrb4DKNaImhC26uMCqo86/7lIzBC3JKKMaQahuTe0RYJx5+UD3BTCQet6RVVWhLQDewNscOyUmpKu38ea1q8Eq6R1FSRNJDsPlr/mriXfMMSAR9pffbgVbmqxkRvQMQEkXGHwA4ZqlpxrQ9ru5iwBoKTgtIaQuVibuwhF7Xy7X37xITkRzUTjVaEKVgQQCH+c95HlYdRJNiLIEgwIIFswx/k3EYEDg996+uHbEl9UcGW2I3NFYG7lPyZ6RkP3Q+SNHF5NT03vO8QtsvkiY=
X-BeenThere: cygwin AT cygwin DOT com
X-Mailman-Version: 2.1.30
Precedence: list
List-Id: General Cygwin discussions and problem reports <cygwin.cygwin.com>
List-Unsubscribe: <https://cygwin.com/mailman/options/cygwin>,
 <mailto:cygwin-request AT cygwin DOT com?subject=unsubscribe>
List-Archive: <https://cygwin.com/pipermail/cygwin/>
List-Post: <mailto:cygwin AT cygwin DOT com>
List-Help: <mailto:cygwin-request AT cygwin DOT com?subject=help>
List-Subscribe: <https://cygwin.com/mailman/listinfo/cygwin>,
 <mailto:cygwin-request AT cygwin DOT com?subject=subscribe>
From: Brian Inglis via Cygwin <cygwin AT cygwin DOT com>
Reply-To: cygwin AT cygwin DOT com
Cc: brian DOT inglis AT systematicsw DOT ab DOT ca
Content-Type: text/plain; charset="utf-8"; Format="flowed"
Errors-To: cygwin-bounces~archive-cygwin=delorie DOT com AT cygwin DOT com
Sender: "Cygwin" <cygwin-bounces~archive-cygwin=delorie DOT com AT cygwin DOT com>
Content-Transfer-Encoding: 8bit
X-MIME-Autoconverted: from base64 to 8bit by delorie.com id 50G0FwvN4109687

On 2025-01-15 10:26, Corinna Vinschen via Cygwin wrote:
> On Jan 15 09:12, Brian Inglis via Cygwin wrote:
>> On 2025-01-14 03:13, Roland Mainz via Cygwin wrote:
>>> On Tue, Jan 14, 2025 at 7:19 AM Brian Inglis via Cygwin
>>> <cygwin AT cygwin DOT com> wrote:
>>>> On 2025-01-13 13:10, Roland Mainz via Cygwin wrote:
>>>>> I just hit an endless loop with /usr/bin/cp from "coreutils" version
>>>>> 9.5-1 copying a larger *.pdb file (it seems that only this specific
>>>>> file is affected...) from Visual Studio 19.
>>>>> [...]
>>> I think I found the problem:
>>> The *.pdb file uses NTFS compression:
>>> ---- snip ----
>>> /bin/winfsinfo filebasicinfo "$(cygpath -w
>>> $PWD/../build.vc19/x64/Debug/nfs41_driver.pdb)"
>>> (
>>>          filename='C:\cygwin64\home\roland_mainz\work\msnfs41_uidmapping\ms-nfs41-client-kofemannvacation\build.vc19\x64\Debug\nfs41_driver.pdb'
>>>          CreationTime=133812707624654816
>>>          LastAccessTime=133813220892976366
>>>          LastWriteTime=133812707639811081
>>>          ChangeTime=133812707639811081
>>>          typeset -a FileAttributes=(
>>>                  FILE_ATTRIBUTE_ARCHIVE
>>>                  FILE_ATTRIBUTE_COMPRESSED
>>>          )
>>> )
>>> ---- snip ----
>>>
>>> If I remove the "FILE_ATTRIBUTE_COMPRESSED" flag /bin/cp works without problems.
>>> I think the issues here are:
>>> 1. Coreutils 9.5-1 /bin/cp erroneously assumes that a file is sparse
>>> if the number of blocks is smaller than $((filesize / fs_blocksize)) -
>>> but in this case the file is NOT sparse, just compressed.
>>> 2. The loop to copy a sparse file is faulty because there are no holes
>>> in that file. That itself is IMHO already a bad idea to have a
>>> separate codepath for sparse files, just the normal codepath should
>>> use SEEK_HOLE and just skip those in the destination
>>
>> A possible issue is that Cygwin assumes sparse files on SSD
> 
> No, that's not the problem, because SPARSE handling requires that
> the FILE_ATTRIBUTE_SPARSE_FILE flag of the file is actually set.
> 
> For instance, see lseek w/ SEEK_DATA/SEEK_HOLE, which handles
> non-sparse files as documented by the Linux man page
> https://man7.org/linux/man-pages/man2/lseek.2.html:
> 
>          if (!has_attribute (FILE_ATTRIBUTE_SPARSE_FILE))
>            {
>              /* Default behaviour if sparse files are not supported:
>                 SEEK_DATA: seek to offset
>                 SEEK_HOLE: seek to EOF */
>              fpi.CurrentByteOffset.QuadPart = (whence == SEEK_DATA)
>                                               ? offset
>                                               : fsi.EndOfFile.QuadPart;
>              break;
>            }
> 
>> so we need
>> fhandler/disk_file:fstat_helper to allow cp to handle compressed files
>> normally.
> 
> Say again?  What exactly are you expecting from stat()?  Fibbing about
> the number of blocks taken by a FS-compressed file?

Just a suggestion to work around the issue transparently?

> For the time being, it seems the assumption that #blocks * blocksize <
> filesize is not really correct.
> 
> Actually cp(1) should test if lseek(SEEK_HOLE) returns EOF.  If so,
> the file isn't sparse.
> 
> On Cygwin it could also check with
> 
>    (ioctl (fd, FS_IOC_GETFLAGS, &flags) & FS_SPARSE_FL) != 0
> 
> that the file is sparse, but the lseek test is target-independent.
> 
> On Cygwin and Linux you could also test with
> 
>    (ioctl (fd, FS_IOC_GETFLAGS, &flags) & FS_COMPR_FL) != 0
> 
> that the file is compressed, but there's a problem so far.  The
> Linux flag is called FS_COMPR_FL but the Cygwin flag is called
> FS_COMPRESSED_FL.  THis flag is only supported on btrfs and I'm
> not sure it already existed when I added FS_IOC_GETFLAGS...
> 
> Anyway, I'll change FS_COMPR_FL to FS_COMPRESSED_FL and make
> FS_COMPRESSED_FL an alias for backward compat...

Thanks - I'll submit a bug report and ask what approach they would prefer?

-- 
Take care. Thanks, Brian Inglis              Calgary, Alberta, Canada

La perfection est atteinte                   Perfection is achieved
non pas lorsqu'il n'y a plus rien à ajouter  not when there is no more to add
mais lorsqu'il n'y a plus rien à retrancher  but when there is no more to cut
                                 -- Antoine de Saint-Exupéry

-- 
Problem reports:      https://cygwin.com/problems.html
FAQ:                  https://cygwin.com/faq/
Documentation:        https://cygwin.com/docs.html
Unsubscribe info:     https://cygwin.com/ml/#unsubscribe-simple