X-Recipient: archive-cygwin AT delorie DOT com
X-Spam-Check-By: sourceware.org
To: cygwin AT cygwin DOT com
From: Andrew DeFaria <Andrew AT DeFaria DOT com>
Subject:  Re: copying a million tiny files?
Date:  Wed, 31 Oct 2007 23:52:38 -0700
Lines: 29
Message-ID: <fgbt3o$srn$1@ger.gmane.org>
References:  <6a42eec70710311438s273b63dcxc6c741dde4593afc AT mail DOT gmail DOT com> <47294BE7 DOT E195E110 AT dessent DOT net> <001c01c81c3d$992990c0$020aa8c0 AT DFW5RB41>
Mime-Version:  1.0
Content-Type:  text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding:  7bit
User-Agent: Thunderbird 2.0.0.6 (Windows/20070728)
In-Reply-To: <001c01c81c3d$992990c0$020aa8c0@DFW5RB41>
X-IsSubscribed: yes
Mailing-List: contact cygwin-help AT cygwin DOT com; run by ezmlm
Precedence: bulk
List-Id: <cygwin.cygwin.com>
List-Unsubscribe: <mailto:cygwin-unsubscribe-archive-cygwin=delorie DOT com AT cygwin DOT com>
List-Subscribe: <mailto:cygwin-subscribe AT cygwin DOT com>
List-Archive: <http://sourceware.org/ml/cygwin/>
List-Post: <mailto:cygwin AT cygwin DOT com>
List-Help: <mailto:cygwin-help AT cygwin DOT com>, <http://sourceware.org/ml/#faqs>
Sender: cygwin-owner AT cygwin DOT com
Mail-Followup-To: cygwin AT cygwin DOT com
Delivered-To: mailing list cygwin AT cygwin DOT com

Gary R. Van Sickle wrote:
> I have a similar situation to the OP (copying many thousands of small 
> files over a fairly slow link), and actually timed using XCOPY vs. 
> Cygwin methods (cp in my case). It didn't make a significant 
> difference. Ultimately what I think you run into in these sorts of 
> situations is that you bump up against the slowness of the link (or 
> physical disk) because, POSIX emulation or not, all your caches do is 
> thrash.
I think your situation is vastly different. Granted copying over a 
network connection will be soo much slower that the additional overhead 
of Cygwin emulation fades away. But the OP was talking about copying 
between to supposedly local disks. With the network link out of the way 
the overhead of Cygwin emulation could become significant.

As for your cp across a link... At one time I was using cp to copy our 
release images (about 9 files of about 50-150 meg mind you) across a WAN 
link (from Santa Clara, California -> Shanghai, China). The link was 
real slow and it was talking ~30-45 minutes per image IIRC!

Then I changed it to an ncftp and the time per image dropped down to ~5 
minutes.

Moral of the story? SMB is perhaps the worse way to transfer data across 
network links.

YMMV.
-- 
Andrew DeFaria <http://defaria.com>
I used to have an open mind but my brains kept falling out.


--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/