Subject: Re: rsync 3.0.4 over ssh hanging on cygwin 1.7
From: Fred Kemp <[email-masked]>
To: [email-masked]
Cc: Herb Maeder <[email-masked]>
Date: Thu, 20 Nov 2008 16:32:04 +0000
List-Id: <cygwin.cygwin.com>

Thanks to all who have replied so far - as Herb says, the cygwin+sshd 
+rsync combo is extremely useful, not least in that it allows my  
client PC's to simply run sshd as a very efficient (ie resource  
unintensive) background service, requiring no further user  
intervention (A Good Thing™).  Workarounds such as running OpenVPN or  
VirtualBox seem like the proverbial  sledgehammer to crack a nut and  
additionally add a further layer to configure/go wrong.

 From my own point of view, I'm going to try playing around with a few  
options such as creating an ssh tunnel as described here: http://backuppc.wiki.sourceforge.net/Workaround+BackupPC+Windows+2003+Hang

to see if that helps and beyond that see if I can come up with some  
combination of rsync options and shell scripting that might allow me  
to atomise the transfer somewhat, possibly using --write-batch -- 
timeout and some batch vs logfile crunching to restart transfer at  
last file transferred at timeout point. It won't be as quick or as  
simple, but if it works and is robust, that'll do for me for now (I'm  
at least a week late on rolling this out as it is!) I won't however  
hold my breath as a test of recursively running rsync with --timeout  
on one user directory is showing only about another 5 files each time...

As ever all tips and pointers gratefully received, and if I have any  
blinding glimpses of the obvious, I'll certainly share with the  
list. :-)

Cheers,

Fred.

On Nov 20, 2008, at 1:48 AM, Herb Maeder wrote:

> On 19 Nov 2008 09:54:41 EST, Christopher Faylor wrote:
>> On Wed, Nov 19, 2008 at 07:24:33AM -0500, Brett Serkez wrote:
>>> I spent considerable time on this and reported were the problem is
>>> occurring to no avail, don't waste your time.  In a nutshell the  
>>> issue
>>> is with Cygwin's bi-directional pipe emulation, this is a  
>>> fundamental
>>> feature of all UNIXies.  Secure Shell "forks and execs" rsync,
>>> connecting standard out and in so that data flows over the internet
>>> to/from SSH and then locally to/from rsync.  The problem is that
>>> eventually a "signal" is missed and SSH and rsync deadlock, the  
>>> local
>>> pipe emulation is imperfect, and the rsync protocol has no  
>>> provision to
>>> recover from this dead lock.
>>>
>>> A fix would require a change to this fundamental feature of  
>>> Cygwin, it
>>> is not clear to me that Windows has the necessary functionality to
>>> properly implement, such a fix would require extensive retesting.
>>
>> Your analysis of the problem is likely incorrect.  The problem has  
>> been
>> reported to be due to the fact that there is no foolproof way in  
>> Windows
>> to tell when a pipe can be written to in a non-blocking fashion.   
>> Since
>> that fact hasn't changed, I doubt that this has anything to do with
>> missed "signal"s and it certainly doesn't have anything to do with
>> bidirectional pipes.
>>
>> Nevertheless, the problem is still there and as always PGA.  I have
>> spent a considerable amount of time staring at the code and  
>> googling for
>> a solution but to no avail.
>
> Sounds like solving the root of problem may be beyond our control  
> for now.
> But, as an alternative, do you think that a cygwin specific  
> workaround to
> this problem in rsync (and/or sshd) might be feasible?  If so, would  
> you
> have any suggestions on how to approach that task?
>
> Obviously the ideal solution would be to get the underlying problem  
> fixed
> in windows, especially since there's no reason other programs won't  
> run
> into the same issue.  But since the cygwin+sshd+rsync combo is quite
> useful, it shows this problem regularly, and the alternatives aren't
> great, having a specific workaround would be nice.  Even if it means
> trading off performance and/or convenience for correct functionality.
>
> Herb.
>
> --
> Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
> Problem reports:       http://cygwin.com/problems.html
> Documentation:         http://cygwin.com/docs.html
> FAQ:                   http://cygwin.com/faq/
>

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/