X-Spam-Check-By: sourceware.org
Date: Tue, 28 Mar 2006 16:39:52 +0200
From: Corinna Vinschen <corinna-cygwin AT cygwin DOT com>
To: cygwin AT cygwin DOT com
Subject: Re: very poor cygwin scp performance in some situations
Message-ID: <20060328143952.GN20907@calimero.vinschen.de>
Reply-To: cygwin AT cygwin DOT com
Mail-Followup-To: cygwin AT cygwin DOT com
References: <04b601c651b4$a1e9e020$b3db87d4 AT multiplay DOT co DOT uk> <20060328074041 DOT GJ20907 AT calimero DOT vinschen DOT de> <01f501c65254$796572e0$b3db87d4 AT multiplay DOT co DOT uk>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <01f501c65254$796572e0$b3db87d4@multiplay.co.uk>
User-Agent: Mutt/1.4.2i
Mailing-List: contact cygwin-help AT cygwin DOT com; run by ezmlm
Precedence: bulk
List-Unsubscribe: <mailto:cygwin-unsubscribe-archive-cygwin=delorie DOT com AT cygwin DOT com>
List-Subscribe: <mailto:cygwin-subscribe AT cygwin DOT com>
List-Archive: <http://sourceware.org/ml/cygwin/>
List-Post: <mailto:cygwin AT cygwin DOT com>
List-Help: <mailto:cygwin-help AT cygwin DOT com>, <http://sourceware.org/ml/#faqs>
Sender: cygwin-owner AT cygwin DOT com
Mail-Followup-To: cygwin AT cygwin DOT com
Delivered-To: mailing list cygwin AT cygwin DOT com

On Mar 28 11:43, Steven Hartland wrote:
> ----- Original Message ----- 
> From: "Corinna Vinschen" 
> >On Mar 27 16:39, Steven Hartland wrote:
> >>I've done quite a bit of digging and not found any real answers
> >>on why cygwin's scp performance is so poor.
> >
> >Cygwin is just emulating POSIX by calling Windows functions.  It's not
> >an operating system which runs natively on the machine, so what do
> >you expect?  If you're unhappy with scp's performance you can try
> >using:
> >
> >- sftp
> >
> >- tar over ssh:
> >
> >   tar cf - dir-to-copy | ssh remotemachine 'cd targetdir && tar xf -'
> >
> >- http://www.psc.edu/networking/projects/hpn-ssh/
> 
> Sorry to sound rude but did you even look at the figures I posted?

Yeah, I'm just slightly annoyed about the "Cygwin is slow" routine on
this mailing list.

However, the performance of ssh/scp is not a Cygwin problem per se.
Note that the poor performance of ssh/scp is a well-known fact, which
has been discussed pretty often on the ssh developers list, and which
has different impact on different OSes.  The above hpn patch was
supposed to tackle exactly this problem.  It's not integrated into
mainline so far, because security is of a bigger concern than speed for
the OpenSSH developer team, which should be immediately understandable.

I made some tests locally and my default numbers (118 Meg file) are

  From Linux to Cygwin, started on Linux:  9.1MB/s
  From Linux to Cygwin, started on Cygwin: 1.2MB/s
 
It turns out that three factors limit the speed on the client side:

- The worst of all is ssh itself.  When reading blocks from the network,
  it uses an 8K buffer.  Whatever you try with setting different socket
  buffer sizes or disabling the Nagle algorithm, nothing has any
  interesting impact if not ssh's own buffer is raised in size.  By just
  setting the buffer to 64K (in the ssh source file clientloop.c,
  function client_process_net_input, line 575), the above copy started on
  Cygwin suddenly races along with 4.2MB/s.

- The default WinSock send/recv buffer size is apparently 8K.  This is a
  quite small buffer for bulk data transfer.  Raising this buffer to 64K
  (setsockopt's SOL_SOCKET/SO_RCVBUF option), too, results in a
  throughput of 5.9MB/s.

  Note that the performance suffers again, if the socket buffer is
  smaller than the application buffer.

- Disabling the Nagle algorithm (setsockopt's IPPROTO_TCP/TCP_NODELAY
  option) doesn't really have a lot of impact, but it's slightly
  noticable.  The throughput goes up to 6.1MB/s.

And no, I won't release an OpenSSH package which differs from the
vanilla sources, unless it's really really necessary to get it working
on Cygwin.  If the performance is a problem for you, build your own
package.  You should have enough details now to get this done.


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Project Co-Leader          cygwin AT cygwin DOT com
Red Hat

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/