X-Recipient:	archive-cygwin AT delorie DOT com
X-SWARE-Spam-Status:	No, hits=-1.8 required=5.0 tests=BAYES_00,MSGID_FROM_MTA_HEADER
X-Spam-Check-By:	sourceware.org
To:	KHMan <keinhong AT gmail DOT com>, <cygwin AT cygwin DOT com>
Date:	Tue, 24 Mar 2009 06:59:18 -0000
Subject:	RE: rsync under cygwin too slow
Envelope-To:	keinhong AT gmail DOT com,cygwin AT cygwin DOT com
Message-ID:	<0000094450@timbuctoo.liddicott.com>
From:	"Sam Liddicott" <sam AT liddicott DOT com>
MIME-Version:	1.0
content-class:
X-IsSubscribed:	yes
Mailing-List:	contact cygwin-help AT cygwin DOT com; run by ezmlm
List-Id:	<cygwin.cygwin.com>
List-Subscribe:	<mailto:cygwin-subscribe AT cygwin DOT com>
List-Archive:	<http://sourceware.org/ml/cygwin/>
List-Post:	<mailto:cygwin AT cygwin DOT com>
List-Help:	<mailto:cygwin-help AT cygwin DOT com>, <http://sourceware.org/ml/#faqs>
Sender:	cygwin-owner AT cygwin DOT com
Mail-Followup-To:	cygwin AT cygwin DOT com
Delivered-To:	mailing list cygwin AT cygwin DOT com

Its because that's not how rsync should be used over the network.

There should be a copy of rsync running on the remote machine that reads th=
e remote files and sends only the checksums.

The setup shown here (as guessed) has the loca rsync read the remote files =
over the lan in order to get the checksums.

Disabling checksums and using timestamps can help but is much less certain.

Better to use rsyncover ssh.
Sam



-----Original Message-----
From: KHMan <keinhong AT gmail DOT com>
Sent: Tuesday, March 24, 2009 3:16 AM
To: cygwin AT cygwin DOT com
Subject: Re: rsync under cygwin too slow


Alfred von Campe wrote:
>> rsync -rtuplg --stats --verbose --progress --modify-window=3D2=20
>> /cygdrive/c/localfolder /cygdrive/z/backups/
>> ...
>> ...
>> The local and remote folders were already synchronized, that is why 0=20
>> files were transferred. File list generation did take only 3 minutes=20
>> but the whole process took more than 1.5 hours to finish and consumed=20
>> a whopping 1.9 GB of downloaded bytes and 0.4 GB of uploaded bytes as=20
>> per network counters. For what? Zero files, Zero bytes of file data.
>=20
> I'm pretty sure this is because rsync has to copy all the files locally=
=20
> to compare them to see if they need to be copied.  Try running the rsync=
=20
> as a service on the system that hosts the Z: drive (or run rsync over=20
> ssh to the system that hosts the Z drive) and you will see large=20
> performance improvement.

Looks to me there is no --checksum option, so rsync skips based on=20
size and modtime. I do something similar to Vinod. Here is an=20
extra data point to compare against (y and u are on separate=20
drives on same machine):

$ time rsync -a --progress --stats --delete /cygdrive/y/Installs/=20
/cygdrive/u/Installs/
sending incremental file list

Number of files: 284
Number of files transferred: 0
Total file size: 2284961277 bytes
Total transferred file size: 0 bytes
Literal data: 0 bytes
Matched data: 0 bytes
File list size: 8583
File list generation time: 0.030 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 8606
Total bytes received: 22

sent 8606 bytes  received 22 bytes  17256.00 bytes/sec
total size is 2284961277  speedup is 264830.93

real    0m0.678s
user    0m0.155s
sys     0m0.217s

With --checksum:

$ time rsync -ac --progress --stats --delete /cygdrive/y/Installs/=20
/cygdrive/u/Installs/
sending incremental file list

Number of files: 284
Number of files transferred: 0
Total file size: 2284961277 bytes
Total transferred file size: 0 bytes
Literal data: 0 bytes
Matched data: 0 bytes
File list size: 12951
File list=20

[The entire original message is not included]

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

- Raw text -