DMARC-Filter: OpenDMARC Filter v1.4.2 delorie.com 50FFxUpa3930300
Authentication-Results: delorie.com; dmarc=pass (p=none dis=none) header.from=cygwin.com
Authentication-Results: delorie.com; spf=pass smtp.mailfrom=cygwin.com
DKIM-Filter: OpenDKIM Filter v2.11.0 delorie.com 50FFxUpa3930300
Authentication-Results: delorie.com;
	dkim=pass (1024-bit key, unprotected) header.d=cygwin.com header.i=@cygwin.com header.a=rsa-sha256 header.s=default header.b=WAE/xWXB
X-Recipient: archive-cygwin AT delorie DOT com
DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org C9DFB3851C0C
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cygwin.com;
	s=default; t=1736956768;
	bh=vnoRjNO/t8XgTchUGkMqZE9Eq+/LLg/UTHHQxZ7PQrw=;
	h=Date:Subject:To:References:In-Reply-To:List-Id:List-Unsubscribe:
	 List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc:
	 From;
	b=WAE/xWXBCQGMTOV9baI5TyLuEx/MEK+5w+pn81LSrTmuAJQ2g7NWTkx8eRs7Rh7n/
	 jV6S4aW9tMFaWXb/2Xnhg7eYmDAFQtBrekmcAma1wL4dgEzl+sgLij5R7u10uiub3p
	 XPOxnL3/Q8Rabyp74ZFNjQ2SVP1kbu7OI7V8/SAA=
X-Original-To: cygwin AT cygwin DOT com
Delivered-To: cygwin AT cygwin DOT com
DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 38C1B3852761
ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 38C1B3852761
ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1736956743; cv=none;
 b=RAMKyCCNOVaBTWJleeBMmBV+K2f3FOikd4SDP54G0H7TmywhxAj6vkpY/0N7rrNYSWBFMEeOlSGuVDl7is93uFH8XeRAsattGLM3/xv4jrvPztk42SL/jYS+Kss9GZlDLVVmAwLnTBVva5x677thiDrk+6MK85AETHW1mn0ikhw=
ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key;
 t=1736956743; c=relaxed/simple;
 bh=tAPBsMk58DFg8C57jgoANv/0tcT2+EmaN+xPiZtMoZM=;
 h=Message-ID:Date:MIME-Version:From:Subject:To:DKIM-Signature;
 b=H3yuLUCwo9LPEGEsz0bbvB9IrBJxRwJmN67/ZJ4aFKcvox7HwjFC5Capq/cNBqV/w7mMIF/0NOvW5vSiwfuQM5mRcYl9iMWfWSm6urcp8sTY2AJVIV0MubXX16Z8XfOU8NxCKJVqVkCzEFFbKx5ay4GXnTt+CpOYm1wcGUgOqKo=
ARC-Authentication-Results: i=1; server2.sourceware.org
DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 38C1B3852761
Message-ID: <4d1caa29-9027-433d-aa3e-d0b55670526e@SystematicSW.ab.ca>
Date: Wed, 15 Jan 2025 08:58:59 -0700
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: Cygwin 3.6 /usr/bin/cp from "coreutils" version 9.5-1 stuck in an
 endless loop...
Content-Language: en-CA
To: cygwin AT cygwin DOT com
References: <CAKAoaQmwna7sSfpYFn0OUsqpdXAgJ5FyaNkhKv6K+otr0Oa5QA AT mail DOT gmail DOT com>
 <4712dcf7-1d4d-4d27-b7c1-b705d3a0a553 AT SystematicSW DOT ab DOT ca>
 <CAKAoaQ=aj+xB2J65Rgv1tV4oSK9fq0tk8Bq6ZoXAoOG_hBR5eA AT mail DOT gmail DOT com>
Organization: Systematic Software
In-Reply-To: <CAKAoaQ=aj+xB2J65Rgv1tV4oSK9fq0tk8Bq6ZoXAoOG_hBR5eA@mail.gmail.com>
X-Rspamd-Queue-Id: CCF5317
X-Stat-Signature: 5piyuaahi1c1w6azma4jtifuu6qj9qni
X-Rspamd-Server: rspamout02
X-Session-Marker: 427269616E2E496E676C69734053797374656D6174696353572E61622E6361
X-Session-ID: U2FsdGVkX18uC9svMs+r1aFPfmIX3LG6bP3hlEmb0Lk=
X-HE-Tag: 1736956740-208867
X-HE-Meta: U2FsdGVkX1/qsCS4v9nUiqBRTlwGayIdifeLF0NRZBP0cjjtP4rUN94PONAV5D9sMJiiFoW8KV9YLQQ/7qAZ6HTPCJZzAcuG2/Cd56/b874vCZt9I9wwQgr5ddt0OuO/fId1g7A3mlMwSLGCEsyhNMaFnYMgd0k1XoncRqAFlPCyG4HVC72ZJRYkEm1P5euKvVP7q29wsgtolqL9A97QGiOKW8fHzvUVUXfJ7rAp+UvNUZo7RYEx9cJZLYfDaOEy7000E5tmymRA+YnV7l56CcjVUEDSWqd5Up6uZGVZ6MLL4YuYia/BHewqZppH53+4QAWSLldMWoDbpW0BAMv52dmuQ4sGtCDeK1AC9oIi8j8evbxRQUWDydCMDUaNEB+pkUVQTzsmS0btWQBrLGZW3e6WHKu5kM1DXRbKA4DCqsYwrx9Y0jQX4wkmoNINmyRjQsB50w2NbVbDcw3xGyoaUA==
X-BeenThere: cygwin AT cygwin DOT com
X-Mailman-Version: 2.1.30
Precedence: list
List-Id: General Cygwin discussions and problem reports <cygwin.cygwin.com>
List-Unsubscribe: <https://cygwin.com/mailman/options/cygwin>,
 <mailto:cygwin-request AT cygwin DOT com?subject=unsubscribe>
List-Archive: <https://cygwin.com/pipermail/cygwin/>
List-Post: <mailto:cygwin AT cygwin DOT com>
List-Help: <mailto:cygwin-request AT cygwin DOT com?subject=help>
List-Subscribe: <https://cygwin.com/mailman/listinfo/cygwin>,
 <mailto:cygwin-request AT cygwin DOT com?subject=subscribe>
From: Brian Inglis via Cygwin <cygwin AT cygwin DOT com>
Reply-To: cygwin AT cygwin DOT com
Cc: Brian Inglis <Brian DOT Inglis AT SystematicSW DOT ab DOT ca>
Content-Type: text/plain; charset="utf-8"; Format="flowed"
Errors-To: cygwin-bounces~archive-cygwin=delorie DOT com AT cygwin DOT com
Sender: "Cygwin" <cygwin-bounces~archive-cygwin=delorie DOT com AT cygwin DOT com>
Content-Transfer-Encoding: 8bit
X-MIME-Autoconverted: from base64 to 8bit by delorie.com id 50FFxUpa3930300

On 2025-01-14 03:13, Roland Mainz via Cygwin wrote:
> On Tue, Jan 14, 2025 at 7:19 AM Brian Inglis via Cygwin
> <cygwin AT cygwin DOT com> wrote:
>> On 2025-01-13 13:10, Roland Mainz via Cygwin wrote:
>>> I just hit an endless loop with /usr/bin/cp from "coreutils" version
>>> 9.5-1 copying a larger *.pdb file (it seems that only this specific
>>> file is affected...) from Visual Studio 19.
>>>
>>> Using strace -p $pid_of_cp I get this output:
>>> ---- snip ----
>>> ...
>>>     212 11917852 [main] cp 1319 fhandler_base::lseek: setting file
>>> pointer to 1708032
>>>     200 11918052 [main] cp 1319 lseek: 1708032 = lseek(4, 1708032, 3)
>>>     239 11918291 [main] cp 1319 fhandler_base::lseek: setting file
>>> pointer to 1708032
>>>     266 11918557 [main] cp 1319 lseek: 1708032 = lseek(4, 1708032, 4)
>>>     160 11918717 [main] cp 1319 fhandler_base::lseek: setting file
>>> pointer to 1708032
> [snip]
>>> ...
>>>    ---- snip ----
>>> This never stops, even after a couple of hours, but cp(1) can be
>>> killed with <CTRL-C>
>>>
>>> Downgrading to "coreutils" version 9.0-1 fixes the problem.
>>>
>>> Cygwin version itself is
>>> "CYGWIN_NT-10.0-19045 chickenmonster 3.6.0-0.304.g264544bf72f6.x86_64 2025-01-13 10:15 UTC x86_64 Cygwin"
>>
>> The command is not simply looping, it is repeating 4 SEEK_HOLE, 0 SEEK_SET, 3
>> SEEK_DATA, at the same file offset, which looks like some kind of retry cycle,
>> but each of the operations are succeeding.
>>
>> What is the exact command you are running and what are the source and target
>> filesystems?
> 
> See https://nrubsig.kpaste.net/70f1c8
> 
>> What is the exact size of the file and what device type is it on: SSD or HDD?
> 
> "Virtual SSD", which is VMware's NVME emulation
> 
>> What is the allocation size of the file and how many 4KB holes (zeroed blocks)
>> are in the file?
>>
>> Could you please try running the command under strace to see what it is doing
>> before it gets in to that cycle?
> 
> See https://nrubsig.kpaste.net/70f1c8 for the strace log.
> 
> I think I found the problem:
> The *.pdb file uses NTFS compression:
> ---- snip ----
> /bin/winfsinfo filebasicinfo "$(cygpath -w
> $PWD/../build.vc19/x64/Debug/nfs41_driver.pdb)"
> (
>         filename='C:\cygwin64\home\roland_mainz\work\msnfs41_uidmapping\ms-nfs41-client-kofemannvacation\build.vc19\x64\Debug\nfs41_driver.pdb'
>         CreationTime=133812707624654816
>         LastAccessTime=133813220892976366
>         LastWriteTime=133812707639811081
>         ChangeTime=133812707639811081
>         typeset -a FileAttributes=(
>                 FILE_ATTRIBUTE_ARCHIVE
>                 FILE_ATTRIBUTE_COMPRESSED
>         )
> )
> ---- snip ----
> 
> If I remove the "FILE_ATTRIBUTE_COMPRESSED" flag /bin/cp works without problems.
> I think the issues here are:
> 1. Coreutils 9.5-1 /bin/cp erroneously assumes that a file is sparse
> if the number of blocks is smaller than $((filesize / fs_blocksize)) -
> but in this case the file is NOT sparse, just compressed.
> 2. The loop to copy a sparse file is faulty because there are no holes
> in that file. That itself is IMHO already a bad idea to have a
> separate codepath for sparse files, just the normal codepath should
> use SEEK_HOLE and just skip those in the destination

The other issue is that Cygwin assumes sparse files on SSD, so we need 
fhandler/disk_file:fstat_helper to allow cp to handle compressed files normally.

Separate codepath is probably because most files are not sparse on most 
filesystems, and copying the range is faster, especially if it can mmap input 
and output, and let the pager/swapper do the work.

-- 
Take care. Thanks, Brian Inglis              Calgary, Alberta, Canada

La perfection est atteinte                   Perfection is achieved
non pas lorsqu'il n'y a plus rien à ajouter  not when there is no more to add
mais lorsqu'il n'y a plus rien à retrancher  but when there is no more to cut
                                 -- Antoine de Saint-Exupéry

-- 
Problem reports:      https://cygwin.com/problems.html
FAQ:                  https://cygwin.com/faq/
Documentation:        https://cygwin.com/docs.html
Unsubscribe info:     https://cygwin.com/ml/#unsubscribe-simple