X-Recipient: archive-cygwin AT delorie DOT com
X-Spam-Check-By: sourceware.org
Date: Thu, 25 Jun 2009 16:58:33 +0200
From: Corinna Vinschen <corinna-cygwin AT cygwin DOT com>
To: cygwin AT cygwin DOT com
Subject: Re: [1.7.0-50] scp progress counter flies through first 175 MB or 	so
Message-ID: <20090625145833.GS7289@calimero.vinschen.de>
Reply-To: cygwin AT cygwin DOT com
Mail-Followup-To: cygwin AT cygwin DOT com
References: <2ul445d2cfjj1q2t2viropiikoj70slglb AT 4ax DOT com> <20090625101039 DOT GP7289 AT calimero DOT vinschen DOT de> <20090625140508 DOT GA17040 AT ednor DOT casa DOT cgf DOT cx> <20090625143651 DOT GR7289 AT calimero DOT vinschen DOT de>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20090625143651.GR7289@calimero.vinschen.de>
User-Agent: Mutt/1.5.19 (2009-02-20)
Mailing-List: contact cygwin-help AT cygwin DOT com; run by ezmlm
Precedence: bulk
List-Id: <cygwin.cygwin.com>
List-Unsubscribe: <mailto:cygwin-unsubscribe-archive-cygwin=delorie DOT com AT cygwin DOT com>
List-Subscribe: <mailto:cygwin-subscribe AT cygwin DOT com>
List-Archive: <http://sourceware.org/ml/cygwin/>
List-Post: <mailto:cygwin AT cygwin DOT com>
List-Help: <mailto:cygwin-help AT cygwin DOT com>, <http://sourceware.org/ml/#faqs>
Sender: cygwin-owner AT cygwin DOT com
Mail-Followup-To: cygwin AT cygwin DOT com
Delivered-To: mailing list cygwin AT cygwin DOT com

On Jun 25 16:36, Corinna Vinschen wrote:
> On Jun 25 10:05, Christopher Faylor wrote:
> > Is ssh using non-blocking pipes opened for write?  Until a week or two
> > ago, Cygwin didn't support those and treated the non-blocking write as a
> > blocking write.
> 
> scp switches the pipes to non-blocking and then tries to do blocking io
> on its own, using the poll() function.  It calls a function called scpio
> which basically work like this:
> 
>   scpio (io_function, fd, buf, size)
>   {
>     for (offset = 0; offset < size;) {
>       r = io_function (fd, buf + offset, size - offset);
>       [...]
>       if (r < 0) {
> 	if (errno == EINTR)
> 	  continue;
> 	if (errno == EAGAIN || errno == EWOULDBLOCK) {
> 	  poll (fd, 1, -1) /* Use poll() for blocking */
> 	  continue;
> 	}
> 	[...]
>       }
>       offset += r;
>     }
>   }
> 
> Looks like scp now stumbles over the pipe select() implementation.

I re-activated the old experimental pipe select code using
NtQueryInformationFile and the result is tha scp is more or less back to
normal.  It's still using a good amount of kernel memory, about 64 Megs,
but it appears to be working much better with this select implementation.
I'm just not sure if that's now really useful as a generic write select on
pipes.

Btw., while playing with this I also tried something like this:

$ cmd
C:\cygwin\home\corinna> type file | ssh linux-box 'cat > file'

This *seems* to do something and it finishes writing at one point without
error message, but only the first 192K of the file are written to the
server.  The rest just disappears.  This is independent of the aforementioned
select method.

However, this works fine:

$ cmd /c type file | ssh linux-box 'cat > file'

AFAICS the difference is just who created the pipe.  In the working case
the pipe got created by Cygwin, in the non-working case it has been
created by cmd.


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Project Co-Leader          cygwin AT cygwin DOT com
Red Hat

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple