X-Recipient: archive-cygwin AT delorie DOT com X-SWARE-Spam-Status: No, hits=-1.8 required=5.0 tests=BAYES_00,MSGID_FROM_MTA_HEADER X-Spam-Check-By: sourceware.org To: KHMan , Date: Tue, 24 Mar 2009 06:59:18 -0000 Subject: RE: rsync under cygwin too slow Envelope-To: keinhong AT gmail DOT com,cygwin AT cygwin DOT com Message-ID: <0000094450@timbuctoo.liddicott.com> From: "Sam Liddicott" MIME-Version: 1.0 content-class: Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="iso-8859-1" X-IsSubscribed: yes Mailing-List: contact cygwin-help AT cygwin DOT com; run by ezmlm List-Id: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: cygwin-owner AT cygwin DOT com Mail-Followup-To: cygwin AT cygwin DOT com Delivered-To: mailing list cygwin AT cygwin DOT com Its because that's not how rsync should be used over the network. There should be a copy of rsync running on the remote machine that reads th= e remote files and sends only the checksums. The setup shown here (as guessed) has the loca rsync read the remote files = over the lan in order to get the checksums. Disabling checksums and using timestamps can help but is much less certain. Better to use rsyncover ssh. Sam -----Original Message----- From: KHMan Sent: Tuesday, March 24, 2009 3:16 AM To: cygwin AT cygwin DOT com Subject: Re: rsync under cygwin too slow Alfred von Campe wrote: >> rsync -rtuplg --stats --verbose --progress --modify-window=3D2=20 >> /cygdrive/c/localfolder /cygdrive/z/backups/ >> ... >> ... >> The local and remote folders were already synchronized, that is why 0=20 >> files were transferred. File list generation did take only 3 minutes=20 >> but the whole process took more than 1.5 hours to finish and consumed=20 >> a whopping 1.9 GB of downloaded bytes and 0.4 GB of uploaded bytes as=20 >> per network counters. For what? Zero files, Zero bytes of file data. >=20 > I'm pretty sure this is because rsync has to copy all the files locally= =20 > to compare them to see if they need to be copied. Try running the rsync= =20 > as a service on the system that hosts the Z: drive (or run rsync over=20 > ssh to the system that hosts the Z drive) and you will see large=20 > performance improvement. Looks to me there is no --checksum option, so rsync skips based on=20 size and modtime. I do something similar to Vinod. Here is an=20 extra data point to compare against (y and u are on separate=20 drives on same machine): $ time rsync -a --progress --stats --delete /cygdrive/y/Installs/=20 /cygdrive/u/Installs/ sending incremental file list Number of files: 284 Number of files transferred: 0 Total file size: 2284961277 bytes Total transferred file size: 0 bytes Literal data: 0 bytes Matched data: 0 bytes File list size: 8583 File list generation time: 0.030 seconds File list transfer time: 0.000 seconds Total bytes sent: 8606 Total bytes received: 22 sent 8606 bytes received 22 bytes 17256.00 bytes/sec total size is 2284961277 speedup is 264830.93 real 0m0.678s user 0m0.155s sys 0m0.217s With --checksum: $ time rsync -ac --progress --stats --delete /cygdrive/y/Installs/=20 /cygdrive/u/Installs/ sending incremental file list Number of files: 284 Number of files transferred: 0 Total file size: 2284961277 bytes Total transferred file size: 0 bytes Literal data: 0 bytes Matched data: 0 bytes File list size: 12951 File list=20 [The entire original message is not included] -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/