Mailing-List: contact cygwin-help AT cygwin DOT com; run by ezmlm
List-Subscribe: <mailto:cygwin-subscribe AT cygwin DOT com>
List-Archive: <http://sourceware.org/ml/cygwin/>
List-Post: <mailto:cygwin AT cygwin DOT com>
List-Help: <mailto:cygwin-help AT cygwin DOT com>, <http://sourceware.org/ml/#faqs>
Sender: cygwin-owner AT cygwin DOT com
Mail-Followup-To: cygwin AT cygwin DOT com
Delivered-To: mailing list cygwin AT cygwin DOT com
Message-ID: <40F3C4E4.5050600@x-ray.at>
Date: Tue, 13 Jul 2004 13:17:56 +0200
From: Reini Urban <rurban AT x-ray DOT at>
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; de-AT; rv:1.7) Gecko/20040616
MIME-Version: 1.0
To: cygwin AT cygwin DOT com
Subject: Re: rsync very slow, but not a network issue
References: <40F2CE78 DOT 7030109 AT alexisgallagher DOT com> <40F2DD8B DOT 5080406 AT x-ray DOT at> <40F36FD4 DOT B3C75A45 AT dessent DOT net>
In-Reply-To: <40F36FD4.B3C75A45@dessent.net>
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: 7bit
X-IsSubscribed: yes

Brian Dessent schrieb:
> Reini Urban wrote:
>>>When the file is alredy there, rsync reports a speedup of about 70.
>>>(When the file is not already there, the speed up is 1, of course.) I am
>>>running rsync over ssh with pre-generated keys installed in my .ssh
>>>directories.
>>
>>This is a binary MP3.
>>rsync (as diff) is not good in checking binary diffs.
>>Please try it with a typical text file, where the patch
>>is smaller than the source.
> 
> That is not correct at all, and rsync would be much less useful if it
> were true.  rsync works with arbitrary binary data just as well as it
> does with text.  It has no conception of 'lines' or any other type of
> delimited text.  All it does is split up the file into fixed-length
> chunks and calculates a rolling checksum of each of these.  It does not
> use anything resembling 'diff' or 'patch'. 
> 
> http://rsync.samba.org/tech_report/node2.html

Thanks for clarification. Makes sense.
So as punishment I made another bench.

What is the maximum overhead?
It cannot be the reported difference (factor 500).

http://rsync.samba.org/tech_report/node6.html has about a similar 
filesize (2.1MB binary).
In this example with a hit:miss ration of 64247:948 the overhead is 
5312200:5629158+1632284 (if read and write is synchronous) compared to rcp.
So about max 2.5 overhead, and not 500.

So check if the net overhead in the cygwin version is broken?
Could be easily tested out with two local files:

head -c 2100000 < /dev/random >test1.mp3
head -c 2100000 < /dev/random >test2.mp3
cp test1.mp3 test1-same.mp3
cp test2.mp3 test2.bak
time rsync test1.mp3 test2.mp3

   real    0m0.518s
   user    0m0.201s
   sys     0m0.171s

time rsync test1.mp3 test1-same.mp3

   real    0m0.524s
   user    0m0.154s
   sys     0m0.263s

RSYNC_RSH=ssh
TEST=othermachine
cp test2.bak test2.mp3
time scp test2.mp3 $TEST:

   63.2KB/s
real    0m36.619s
user    0m0.170s
sys     0m0.233s

# 100% different
$ time rsync test1.mp3 $TEST:test2.mp3
Server is very old version of rsync, upgrade recommended.

real    0m37.162s
user    0m0.388s
sys     0m0.202s

# 100% same
$ time rsync test2.mp3 $TEST:test2.bak
Server is very old version of rsync, upgrade recommended.

real    0m7.298s
user    0m1.201s
sys     0m2.217s

6x faster than scp, with same data.
0.5s (1.4%) slower than scp, with complete random data.

$ rsync --version
rsync  version 2.6.2  protocol version 28
$ ssh $TEST rsync --version
rsync version 2.4.6  protocol version 24
$ ssh -V
OpenSSH_3.8.1p1, OpenSSL 0.9.7d 17 Mar 2004
$ ssh $TEST ssh -V
OpenSSH_3.7.1p2, SSH protocols 1.5/2.0, OpenSSL 0.9.6c 21 dec 2001
$ uname -r
1.5.10(0.116/4/2)
-- 
Reini Urban
http://xarch.tu-graz.ac.at/home/rurban/


--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/