X-Recipient:	archive-cygwin AT delorie DOT com
X-Spam-Check-By:	sourceware.org
Date:	Wed, 31 Oct 2007 17:27:12 -0500
From:	Brian Ford <Brian DOT Ford AT FlightSafety DOT com>
Reply-To:	cygwin AT cygwin DOT com
To:	cygwin AT cygwin DOT com
Subject:	Re: copying a million tiny files?
In-Reply-To:	<6a42eec70710311438s273b63dcxc6c741dde4593afc@mail.gmail.com>
Message-ID:	<Pine.CYG.4.58.0710311724580.2628@PC1163-8460-XP.flightsafety.com>
References:	<6a42eec70710311438s273b63dcxc6c741dde4593afc AT mail DOT gmail DOT com>
MIME-Version:	1.0
X-IsSubscribed:	yes
Mailing-List:	contact cygwin-help AT cygwin DOT com; run by ezmlm
List-Id:	<cygwin.cygwin.com>
List-Unsubscribe:	<mailto:cygwin-unsubscribe-archive-cygwin=delorie DOT com AT cygwin DOT com>
List-Subscribe:	<mailto:cygwin-subscribe AT cygwin DOT com>
List-Archive:	<http://sourceware.org/ml/cygwin/>
List-Post:	<mailto:cygwin AT cygwin DOT com>
List-Help:	<mailto:cygwin-help AT cygwin DOT com>, <http://sourceware.org/ml/#faqs>
Sender:	cygwin-owner AT cygwin DOT com
Mail-Followup-To:	cygwin AT cygwin DOT com
Delivered-To:	mailing list cygwin AT cygwin DOT com

On Wed, 31 Oct 2007, sam reckoner wrote:

> I'm not exaggerating. I have over one million small files that like to
> move between disks. The problem is that even getting a directory
> listing takes forever.
>
> Is there a best practice for this?
>
> I don't really need the directory listing, I just need to move all the
> files. I have been using rsync, but that takes a very long time to
> generate the list of files to be moved.

That usually is the best method if you are doing an incremental move.  A
first time move is usually faster this way:

tar -cf - srcdir | tar -C dstdir -xf -

-- 
Brian Ford
Lead Realtime Software Engineer
VITAL - Visual Simulation Systems
FlightSafety International
the best safety device in any aircraft is a well-trained crew...

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

- Raw text -