X-Recipient: archive-cygwin AT delorie DOT com
X-Original-To: cygwin AT cygwin DOT com
Delivered-To: cygwin AT cygwin DOT com
DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 1FB773858C20
Authentication-Results: sourceware.org;
 dmarc=none (p=none dis=none) header.from=cs.umass.edu
Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=cs.umass.edu
Subject: Re: Renaming (with 'mv') very large files is SLOW
To: cygwin AT kosowsky DOT org, cygwin AT cygwin DOT com
References: <25079 DOT 63376 DOT 349893 DOT 86395 AT consult DOT pretender>
From: Eliot Moss <moss AT cs DOT umass DOT edu>
Message-ID: <c1e4529a-d119-1662-6eeb-2d42950802c7@cs.umass.edu>
Date: Mon, 31 Jan 2022 09:59:17 -0500
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101
 Thunderbird/68.12.1
MIME-Version: 1.0
In-Reply-To: <25079.63376.349893.86395@consult.pretender>
Content-Language: en-US
X-Spam-Status: No, score=-2.2 required=5.0 tests=BAYES_00, KAM_DMARC_STATUS,
 NICE_REPLY_A, SPF_HELO_NONE, SPF_PASS, TXREP,
 T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.4
X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on
 server2.sourceware.org
X-BeenThere: cygwin AT cygwin DOT com
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: General Cygwin discussions and problem reports <cygwin.cygwin.com>
List-Unsubscribe: <https://cygwin.com/mailman/options/cygwin>,
 <mailto:cygwin-request AT cygwin DOT com?subject=unsubscribe>
List-Archive: <https://cygwin.com/pipermail/cygwin/>
List-Post: <mailto:cygwin AT cygwin DOT com>
List-Help: <mailto:cygwin-request AT cygwin DOT com?subject=help>
List-Subscribe: <https://cygwin.com/mailman/listinfo/cygwin>,
 <mailto:cygwin-request AT cygwin DOT com?subject=subscribe>
Reply-To: moss AT cs DOT umass DOT edu
Content-Transfer-Encoding: 7bit
Content-Type: text/plain; charset="us-ascii"; Format="flowed"
Errors-To: cygwin-bounces+archive-cygwin=delorie DOT com AT cygwin DOT com
Sender: "Cygwin" <cygwin-bounces+archive-cygwin=delorie DOT com AT cygwin DOT com>

On 1/31/2022 9:52 AM, cygwin AT kosowsky DOT org wrote:
 > I tried renaming some very large files (20-40 GB) using:
 >     mv <oldname> <newname>
 > without changing the directory of course.
 >
 > The process took about 10-20 minutes with Task Manager showing disk
 > activity of 100+ MB/s.
 >
 > Is there something about such large 'renaming' that actually results
 > in the file being really moved (aka copied) rather than just renamed?

The two places are probably on different volumes (loosely, different disks).
That requires a physical move, even under Linux.  Your volumes seem a bit slow
to access - is one perhaps across a slow network?  The rates you cite suggest
movement of 50Mb/s (50Mb read + 50Mb write = 100Mb overall).  For 40 Gb that
should take 40Gb / 50Mb = about 820 secs = a little under 14 mins.

(When I say your volumes are slow, I speak from the luxury of having a 2Tb
solid state drive!  Actually, those speeds may be reasonable depending on the
nature of your system.)

If the two locations are on different drives, there's no real avoiding this.

Regards - Eliot Moss

-- 
Problem reports:      https://cygwin.com/problems.html
FAQ:                  https://cygwin.com/faq/
Documentation:        https://cygwin.com/docs.html
Unsubscribe info:     https://cygwin.com/ml/#unsubscribe-simple