X-Recipient: archive-cygwin@delorie.com
X-Original-To: cygwin@cygwin.com
Delivered-To: cygwin@cygwin.com
DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 1FB773858C20
Authentication-Results: sourceware.org;
 dmarc=none (p=none dis=none) header.from=cs.umass.edu
Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=cs.umass.edu
Subject: Re: Renaming (with 'mv') very large files is SLOW
To: cygwin@kosowsky.org, cygwin@cygwin.com
References: <25079.63376.349893.86395@consult.pretender>
From: Eliot Moss <moss@cs.umass.edu>
Message-ID: <c1e4529a-d119-1662-6eeb-2d42950802c7@cs.umass.edu>
Date: Mon, 31 Jan 2022 09:59:17 -0500
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101
 Thunderbird/68.12.1
MIME-Version: 1.0
In-Reply-To: <25079.63376.349893.86395@consult.pretender>
Content-Language: en-US
X-Spam-Status: No, score=-2.2 required=5.0 tests=BAYES_00, KAM_DMARC_STATUS,
 NICE_REPLY_A, SPF_HELO_NONE, SPF_PASS, TXREP,
 T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.4
X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on
 server2.sourceware.org
X-BeenThere: cygwin@cygwin.com
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: General Cygwin discussions and problem reports <cygwin.cygwin.com>
List-Unsubscribe: <https://cygwin.com/mailman/options/cygwin>,
 <mailto:cygwin-request@cygwin.com?subject=unsubscribe>
List-Archive: <https://cygwin.com/pipermail/cygwin/>
List-Post: <mailto:cygwin@cygwin.com>
List-Help: <mailto:cygwin-request@cygwin.com?subject=help>
List-Subscribe: <https://cygwin.com/mailman/listinfo/cygwin>,
 <mailto:cygwin-request@cygwin.com?subject=subscribe>
Reply-To: moss@cs.umass.edu
Content-Transfer-Encoding: 7bit
Content-Type: text/plain; charset="us-ascii"; Format="flowed"
Errors-To: cygwin-bounces+archive-cygwin=delorie.com@cygwin.com
Sender: "Cygwin" <cygwin-bounces+archive-cygwin=delorie.com@cygwin.com>

On 1/31/2022 9:52 AM, cygwin@kosowsky.org wrote:
 > I tried renaming some very large files (20-40 GB) using:
 >     mv <oldname> <newname>
 > without changing the directory of course.
 >
 > The process took about 10-20 minutes with Task Manager showing disk
 > activity of 100+ MB/s.
 >
 > Is there something about such large 'renaming' that actually results
 > in the file being really moved (aka copied) rather than just renamed?

The two places are probably on different volumes (loosely, different disks).
That requires a physical move, even under Linux.  Your volumes seem a bit slow
to access - is one perhaps across a slow network?  The rates you cite suggest
movement of 50Mb/s (50Mb read + 50Mb write = 100Mb overall).  For 40 Gb that
should take 40Gb / 50Mb = about 820 secs = a little under 14 mins.

(When I say your volumes are slow, I speak from the luxury of having a 2Tb
solid state drive!  Actually, those speeds may be reasonable depending on the
nature of your system.)

If the two locations are on different drives, there's no real avoiding this.

Regards - Eliot Moss

-- 
Problem reports:      https://cygwin.com/problems.html
FAQ:                  https://cygwin.com/faq/
Documentation:        https://cygwin.com/docs.html
Unsubscribe info:     https://cygwin.com/ml/#unsubscribe-simple