Mailing-List: contact cygwin-help AT cygwin DOT com; run by ezmlm List-Subscribe: List-Archive: List-Post: List-Help: , Sender: cygwin-owner AT cygwin DOT com Mail-Followup-To: cygwin AT cygwin DOT com Delivered-To: mailing list cygwin AT cygwin DOT com Message-ID: <3DF8E3C4.7000901@kleckner.net> Date: Thu, 12 Dec 2002 11:30:12 -0800 From: Jim Kleckner User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:0.9.4.1) Gecko/20020508 Netscape6/6.2.3 X-Accept-Language: en-us MIME-Version: 1.0 To: cygwin AT cygwin DOT com Subject: Re: Rsync performance increase through buffering (cygwin/ssh solution?) Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Below is a description of a buffering speed improvement. The original message with complete patch for performance can be found here: http://groups.google.com/groups?hl=en&lr=&ie=UTF-8&oe=UTF-8&newwindow=1&selm=at1i0e%241tj3%241%40FreeBSD.csie.NCTU.edu.tw This may explain the horrendous performance of rsync with ssh under cygwin. Perhaps the way that cygwin emulates these calls. It is especially bad under the circumstances mentioned here - with a large sync of data where the files are identical (-I). The hang fix has been identified elsewhere in cygwin: Check out these links: http://www.cygwin.com/ml/cygwin/2002-10/msg00308.html http://sources.redhat.com/ml/cygwin/2002-09/msg01155.html patched rsync for hang under cygwin: http:// www.addr.com/~dmitry/rsync.exe Jim craig AT atheros DOT com (Craig Barratt) wrote in message news:... > I've been studying the read and write buffering in rsync and it turns > out most I/O is done just a couple of bytes at a time. This means there > are lots of system calls, and also most network traffic comprises lots > of small packets. The behavior is most extreme when sending/receiving > file deltas of identical files. > > The main case where I/O is buffered is writes from the server (when > io multiplexing is on). These are usually buffered in 4092 byte > chunks with a 4 byte header. However, reading of these packets is > usually unbuffered, and writes from the client are generally not > buffered. For example: when receiving 1st phase checksums (6 bytes > per block), 2 reads are done: one of 4 bytes and one of 2 bytes, > meaning there are 4 system calls (select/read/select/read) per 6 > bytes of checksum data). > > One cost of this is some performance, but a significant issue is that > unbuffered writes generate very short (and very many) ethernet packets, > which means the overhead is quite large on slow network connections. ... see this link for the rest: http://groups.google.com/groups?hl=en&lr=&ie=UTF-8&oe=UTF-8&newwindow=1&selm=at1i0e%241tj3%241%40FreeBSD.csie.NCTU.edu.tw -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Bug reporting: http://cygwin.com/bugs.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/