X-Recipient: archive-cygwin AT delorie DOT com
X-Spam-Check-By: sourceware.org
Cc: Herb Maeder <maeder-cygml AT maeder DOT org>
Message-Id: <4D2EF43C-A36F-4EE2-B389-C089A8BDFB3A@reading.ac.uk>
From: Fred Kemp <c DOT f DOT kemp AT reading DOT ac DOT uk>
To: cygwin AT cygwin DOT com
In-Reply-To: <36141.1227145681@maeder.org>
Content-Type: text/plain; charset=WINDOWS-1252; format=flowed; delsp=yes
Mime-Version: 1.0 (Apple Message framework v929.2)
Subject: Re: rsync 3.0.4 over ssh hanging on cygwin 1.7
Date: Thu, 20 Nov 2008 16:32:04 +0000
References: <36141 DOT 1227145681 AT maeder DOT org>
X-Mailer: Apple Mail (2.929.2)
X-IsSubscribed: yes
Mailing-List: contact cygwin-help AT cygwin DOT com; run by ezmlm
Precedence: bulk
List-Id: <cygwin.cygwin.com>
List-Unsubscribe: <mailto:cygwin-unsubscribe-archive-cygwin=delorie DOT com AT cygwin DOT com>
List-Subscribe: <mailto:cygwin-subscribe AT cygwin DOT com>
List-Archive: <http://sourceware.org/ml/cygwin/>
List-Post: <mailto:cygwin AT cygwin DOT com>
List-Help: <mailto:cygwin-help AT cygwin DOT com>, <http://sourceware.org/ml/#faqs>
Sender: cygwin-owner AT cygwin DOT com
Mail-Followup-To: cygwin AT cygwin DOT com
Delivered-To: mailing list cygwin AT cygwin DOT com
Content-Transfer-Encoding: 8bit
X-MIME-Autoconverted: from quoted-printable to 8bit by delorie.com id mAKGX6Fd032156

Thanks to all who have replied so far - as Herb says, the cygwin+sshd 
+rsync combo is extremely useful, not least in that it allows my  
client PC's to simply run sshd as a very efficient (ie resource  
unintensive) background service, requiring no further user  
intervention (A Good Thing™).  Workarounds such as running OpenVPN or  
VirtualBox seem like the proverbial  sledgehammer to crack a nut and  
additionally add a further layer to configure/go wrong.

 From my own point of view, I'm going to try playing around with a few  
options such as creating an ssh tunnel as described here: http://backuppc.wiki.sourceforge.net/Workaround+BackupPC+Windows+2003+Hang

to see if that helps and beyond that see if I can come up with some  
combination of rsync options and shell scripting that might allow me  
to atomise the transfer somewhat, possibly using --write-batch -- 
timeout and some batch vs logfile crunching to restart transfer at  
last file transferred at timeout point. It won't be as quick or as  
simple, but if it works and is robust, that'll do for me for now (I'm  
at least a week late on rolling this out as it is!) I won't however  
hold my breath as a test of recursively running rsync with --timeout  
on one user directory is showing only about another 5 files each time...

As ever all tips and pointers gratefully received, and if I have any  
blinding glimpses of the obvious, I'll certainly share with the  
list. :-)

Cheers,

Fred.



On Nov 20, 2008, at 1:48 AM, Herb Maeder wrote:

> On 19 Nov 2008 09:54:41 EST, Christopher Faylor wrote:
>> On Wed, Nov 19, 2008 at 07:24:33AM -0500, Brett Serkez wrote:
>>> I spent considerable time on this and reported were the problem is
>>> occurring to no avail, don't waste your time.  In a nutshell the  
>>> issue
>>> is with Cygwin's bi-directional pipe emulation, this is a  
>>> fundamental
>>> feature of all UNIXies.  Secure Shell "forks and execs" rsync,
>>> connecting standard out and in so that data flows over the internet
>>> to/from SSH and then locally to/from rsync.  The problem is that
>>> eventually a "signal" is missed and SSH and rsync deadlock, the  
>>> local
>>> pipe emulation is imperfect, and the rsync protocol has no  
>>> provision to
>>> recover from this dead lock.
>>>
>>> A fix would require a change to this fundamental feature of  
>>> Cygwin, it
>>> is not clear to me that Windows has the necessary functionality to
>>> properly implement, such a fix would require extensive retesting.
>>
>> Your analysis of the problem is likely incorrect.  The problem has  
>> been
>> reported to be due to the fact that there is no foolproof way in  
>> Windows
>> to tell when a pipe can be written to in a non-blocking fashion.   
>> Since
>> that fact hasn't changed, I doubt that this has anything to do with
>> missed "signal"s and it certainly doesn't have anything to do with
>> bidirectional pipes.
>>
>> Nevertheless, the problem is still there and as always PGA.  I have
>> spent a considerable amount of time staring at the code and  
>> googling for
>> a solution but to no avail.
>
> Sounds like solving the root of problem may be beyond our control  
> for now.
> But, as an alternative, do you think that a cygwin specific  
> workaround to
> this problem in rsync (and/or sshd) might be feasible?  If so, would  
> you
> have any suggestions on how to approach that task?
>
> Obviously the ideal solution would be to get the underlying problem  
> fixed
> in windows, especially since there's no reason other programs won't  
> run
> into the same issue.  But since the cygwin+sshd+rsync combo is quite
> useful, it shows this problem regularly, and the alternatives aren't
> great, having a specific workaround would be nice.  Even if it means
> trading off performance and/or convenience for correct functionality.
>
> Herb.
>
> --
> Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
> Problem reports:       http://cygwin.com/problems.html
> Documentation:         http://cygwin.com/docs.html
> FAQ:                   http://cygwin.com/faq/
>


--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/