Mailing-List: contact cygwin-help AT cygwin DOT com; run by ezmlm
List-Subscribe: <mailto:cygwin-subscribe AT cygwin DOT com>
List-Archive: <http://sourceware.org/ml/cygwin/>
List-Post: <mailto:cygwin AT cygwin DOT com>
List-Help: <mailto:cygwin-help AT cygwin DOT com>, <http://sourceware.org/ml/#faqs>
Sender: cygwin-owner AT cygwin DOT com
Mail-Followup-To: cygwin AT cygwin DOT com
Delivered-To: mailing list cygwin AT cygwin DOT com
From: "Bob Brown" <bob AT smsolutions DOT co DOT nz>
To: Eric Blake <ebb9 AT byu DOT net>, cygwin AT cygwin DOT com
Date: Wed, 10 Aug 2005 16:25:14 +1200
MIME-Version: 1.0
Subject: Re: Rsync over SSH dies before it is able to send with lots of files
Message-ID: <42FA2A6A.29370.257AEDF0@localhost>
In-reply-to: <42F9777B.5080206@byu.net>
References: <42FA081C DOT 25957 DOT 24F4EADF AT localhost>
Content-type: text/plain; charset=US-ASCII
Content-transfer-encoding: 7BIT
Content-description: Mail message body

On 9 Aug 2005 at 21:41, Eric Blake wrote:

> For starters, since you are using rsync recursion, get rid of the * to
> give rsync just a single filename at startup, rather than making bash
> try to expand 45000 names all at once on the command prompt:

Bingo!  Of course now that you mention it, it's as clear as day!  I omitted 
the * and the rsync works perfectly.

> Hmm, you're making bash expand a wildcard into 45000 files.  This
> shouldn't crash bash, but I'm guessing there is some of memory
> corruption or malloc failure when consuming so much memory all before
> forking(). I'll have to see if I can try and reproduce the failure.

Do you think the win32 path length will be included for every file?  If this is 
the case you can probably safely assume 30-40 bytes per filename.  If this 
were the case it would quickly add up to about 1.7MB in just passed 
filenames alone.  Considering that this will ultimately process a directory 
with 500,000 files in it it's not hard to work out that things are going to go 
pear shaped in a hurry.

Thanks for the quick spotting!

Cheers,

- Bob -

--
Bob Brown, bob AT smsolutions DOT co DOT nz
SmartMedia Solutions, Dunedin NZ


--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/