Mailing-List: contact cygwin-help AT cygwin DOT com; run by ezmlm List-Subscribe: List-Archive: List-Post: List-Help: , Sender: cygwin-owner AT cygwin DOT com Mail-Followup-To: cygwin AT cygwin DOT com Delivered-To: mailing list cygwin AT cygwin DOT com Message-ID: <40F3C4E4.5050600@x-ray.at> Date: Tue, 13 Jul 2004 13:17:56 +0200 From: Reini Urban User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; de-AT; rv:1.7) Gecko/20040616 MIME-Version: 1.0 To: cygwin AT cygwin DOT com Subject: Re: rsync very slow, but not a network issue References: <40F2CE78 DOT 7030109 AT alexisgallagher DOT com> <40F2DD8B DOT 5080406 AT x-ray DOT at> <40F36FD4 DOT B3C75A45 AT dessent DOT net> In-Reply-To: <40F36FD4.B3C75A45@dessent.net> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-IsSubscribed: yes Brian Dessent schrieb: > Reini Urban wrote: >>>When the file is alredy there, rsync reports a speedup of about 70. >>>(When the file is not already there, the speed up is 1, of course.) I am >>>running rsync over ssh with pre-generated keys installed in my .ssh >>>directories. >> >>This is a binary MP3. >>rsync (as diff) is not good in checking binary diffs. >>Please try it with a typical text file, where the patch >>is smaller than the source. > > That is not correct at all, and rsync would be much less useful if it > were true. rsync works with arbitrary binary data just as well as it > does with text. It has no conception of 'lines' or any other type of > delimited text. All it does is split up the file into fixed-length > chunks and calculates a rolling checksum of each of these. It does not > use anything resembling 'diff' or 'patch'. > > http://rsync.samba.org/tech_report/node2.html Thanks for clarification. Makes sense. So as punishment I made another bench. What is the maximum overhead? It cannot be the reported difference (factor 500). http://rsync.samba.org/tech_report/node6.html has about a similar filesize (2.1MB binary). In this example with a hit:miss ration of 64247:948 the overhead is 5312200:5629158+1632284 (if read and write is synchronous) compared to rcp. So about max 2.5 overhead, and not 500. So check if the net overhead in the cygwin version is broken? Could be easily tested out with two local files: head -c 2100000 < /dev/random >test1.mp3 head -c 2100000 < /dev/random >test2.mp3 cp test1.mp3 test1-same.mp3 cp test2.mp3 test2.bak time rsync test1.mp3 test2.mp3 real 0m0.518s user 0m0.201s sys 0m0.171s time rsync test1.mp3 test1-same.mp3 real 0m0.524s user 0m0.154s sys 0m0.263s RSYNC_RSH=ssh TEST=othermachine cp test2.bak test2.mp3 time scp test2.mp3 $TEST: 63.2KB/s real 0m36.619s user 0m0.170s sys 0m0.233s # 100% different $ time rsync test1.mp3 $TEST:test2.mp3 Server is very old version of rsync, upgrade recommended. real 0m37.162s user 0m0.388s sys 0m0.202s # 100% same $ time rsync test2.mp3 $TEST:test2.bak Server is very old version of rsync, upgrade recommended. real 0m7.298s user 0m1.201s sys 0m2.217s 6x faster than scp, with same data. 0.5s (1.4%) slower than scp, with complete random data. $ rsync --version rsync version 2.6.2 protocol version 28 $ ssh $TEST rsync --version rsync version 2.4.6 protocol version 24 $ ssh -V OpenSSH_3.8.1p1, OpenSSL 0.9.7d 17 Mar 2004 $ ssh $TEST ssh -V OpenSSH_3.7.1p2, SSH protocols 1.5/2.0, OpenSSL 0.9.6c 21 dec 2001 $ uname -r 1.5.10(0.116/4/2) -- Reini Urban http://xarch.tu-graz.ac.at/home/rurban/ -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/