X-Recipient: archive-cygwin AT delorie DOT com
X-Spam-Check-By: sourceware.org
Date: Wed, 31 Oct 2007 17:27:12 -0500
From: Brian Ford <Brian DOT Ford AT FlightSafety DOT com>
Reply-To: cygwin AT cygwin DOT com
To: cygwin AT cygwin DOT com
Subject: Re: copying a million tiny files?
In-Reply-To: <6a42eec70710311438s273b63dcxc6c741dde4593afc@mail.gmail.com>
Message-ID: <Pine.CYG.4.58.0710311724580.2628@PC1163-8460-XP.flightsafety.com>
References: <6a42eec70710311438s273b63dcxc6c741dde4593afc AT mail DOT gmail DOT com>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII
X-IsSubscribed: yes
Mailing-List: contact cygwin-help AT cygwin DOT com; run by ezmlm
Precedence: bulk
List-Id: <cygwin.cygwin.com>
List-Unsubscribe: <mailto:cygwin-unsubscribe-archive-cygwin=delorie DOT com AT cygwin DOT com>
List-Subscribe: <mailto:cygwin-subscribe AT cygwin DOT com>
List-Archive: <http://sourceware.org/ml/cygwin/>
List-Post: <mailto:cygwin AT cygwin DOT com>
List-Help: <mailto:cygwin-help AT cygwin DOT com>, <http://sourceware.org/ml/#faqs>
Sender: cygwin-owner AT cygwin DOT com
Mail-Followup-To: cygwin AT cygwin DOT com
Delivered-To: mailing list cygwin AT cygwin DOT com

On Wed, 31 Oct 2007, sam reckoner wrote:

> I'm not exaggerating. I have over one million small files that like to
> move between disks. The problem is that even getting a directory
> listing takes forever.
>
> Is there a best practice for this?
>
> I don't really need the directory listing, I just need to move all the
> files. I have been using rsync, but that takes a very long time to
> generate the list of files to be moved.

That usually is the best method if you are doing an incremental move.  A
first time move is usually faster this way:

tar -cf - srcdir | tar -C dstdir -xf -

-- 
Brian Ford
Lead Realtime Software Engineer
VITAL - Visual Simulation Systems
FlightSafety International
the best safety device in any aircraft is a well-trained crew...

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/