X-Spam-Check-By: sourceware.org Message-ID: <43BDCD93.1060009@gmail.com> Date: Thu, 05 Jan 2006 20:53:23 -0500 From: Ken Senior User-Agent: Mozilla Thunderbird 1.0.7 (Windows/20050923) MIME-Version: 1.0 To: Brett Serkez CC: cygwin AT cygwin DOT com Subject: Re: cygwin and rsync References: <43BDAFBC DOT 9080401 AT gmail DOT com> <1136506661 DOT 2833 DOT 251196190 AT webmail DOT messagingengine DOT com> <43BDBBC2 DOT 1040100 AT gmail DOT com> <1136509322 DOT 5881 DOT 251198868 AT webmail DOT messagingengine DOT com> In-Reply-To: <1136509322.5881.251198868@webmail.messagingengine.com> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-IsSubscribed: yes Mailing-List: contact cygwin-help AT cygwin DOT com; run by ezmlm List-Subscribe: List-Archive: List-Post: List-Help: , Sender: cygwin-owner AT cygwin DOT com Mail-Followup-To: cygwin AT cygwin DOT com Delivered-To: mailing list cygwin AT cygwin DOT com Let me say again, THANKS for helping! Well, the screen has stayed frozen for some 10 minutes. Some observations: 1. Killing the rsync process (CTRL-C) on [LOCAL/Linux] does not kill the process on [REMOTE/Windows]. I have to kill the process manually on [REMOTE]. 2. The process seems to stop at one or two of the same files each time: run1 : hung at file A bytes transferred: 2,392,064 run2 : hung at file B bytes transferred: 2,408,448 run3 : hung at file B bytes transferred: 2,408,448 run4 : hung back at file A bytes transferred: 2,408,448 run5 : hung at file A bytes transferred: 2,408,448 Deleted all previously-transferred files and then: run6 : hung at file A bytes transferred: 2,359,296 run7 : hung at file A bytes transferred: 2,359,296 run8 : hung at file A bytes transferred: 2,359,296 Deleted all previously-transferred files and then: run9 : hung at file A bytes transferred: 2,359,296 Deleted all previously-transferred files and then: run10: hung at file A bytes transferred: 2,359,296 Notice that some of these have interesting differences: 2,359,296 - 2,392,064=32,768=2^15 2,408,448 - 2,392,064 = 16,384 = 2^14. Files A and B are each small files as are files adjacent to them. 3. To answer your earlier question, I installed cygwin and added the rsync and openssh which were part of the distribution but were not installed by default. I did not compile my own. Brett Serkez wrote: >Ken, > >The rsync protocol actually does check-suming of blocks to efficiently >detect and transfer files. While it may look like it is hung, it may >actually be just transfering check sums on each file. When I perform >long transfers it looks hung from time to time, but my use of the extra >v switches helped me better understand the protocol. > >When it stops at a different directory and file, is it always >further along? > >It may well be hung, but be sure. Interesting that it works the other >way without having to build a custom rsync. I've found this behavior >consistent, but then again, I always install Cygwin with the same set of >packages. Do you do a full install of Cygwin? I usually perform a >default install and just add vi, tcsh, openssh and rsync. > >Brett > >On Thu, 05 Jan 2006 19:37:22 -0500, "Ken Senior" > said: > > >>Thanks Brett for the quick reply. The multiple -v is a handy thing to >>remember. >> >>In fact this is not my problem though. It looks like the listing of >>files just hangs midstream. Some local directories are created and >>sometimes a few files make it too, but it just hangs. For example: >> >> . . . >> recv_generator(MATLAB/wavelab/Papers/SpinCycle/cspinfo03.m,8425) >> recv_generator(MATLAB/wavelab/Papers/SpinCycle/cspinfo04.m,8425) >> recv_generator(MATLAB/wavelab/Papers/SpinCycle/cspinfo05.m,8425) >> >>None of these files were actually transferred and there are plenty >>more of these similarly-named files well beyond cspinfo05.m, leaving >>me no additional info. Moreover, there are no special characters or >>spaces in these files to suggest a problem in the file or directory >>name. Also, each launch seems to stop at a different directory and >>file. Bizarre. I don't know how to use strace or I'd try that. -K >> >> >>Brett Serkez wrote: >> >> >> >>>Ken, >>> >>>I run rsync both Windows and Linux to Linux, Linux always being the >>>'server'. In my case I found the hang to be up front, before secure >>>shell even attempted to access the network and also in my case >>>changing from a local socket to a local pipe resolved the issue: >>> >>>http://cygwin.com/ml/cygwin/2005-12/msg01096.html >>> >>>Have you tried adding more v switches to your command line, like: >>> >>>rsync -avvz user@[REMOTE/Windows]:/path/to/stuff/ dest/on/local/ >>>rsync -avvvz user@[REMOTE/Windows]:/path/to/stuff/ dest/on/local/ >>>rsync -avvvvvz user@[REMOTE/Windows]:/path/to/stuff/ dest/on/local/ >>> >>>Each time you a a v switch, it increases the debug output, I think up >>>to 4 or 5. This would help to narrow the issue as you'll see how far >>>your getting in the protocol which might help narrow the problem. >>> >>>Is it at all possible to go the other way? I know if you are >>>willing to build your own rsync with the socketpair() called >>>disabled it will work. >>> >>>Brett >>> >>> >>> >>>On Thu, 05 Jan 2006 18:46:04 -0500, "Ken Senior" >>> said: >>> >>> >>> >>> >>>>Hi. >>>> >>>>I posted the message listed below to the rsync list, but one of the >>>>readers there suggested I post it here indicating that there's a >>>>known/common problem in cygwin of data loss in local pipes (whatever >>>>that means I am not sure). I find my rsync command hanging, whether >>>>or not I run it over SSH, that is with our without the --rsh='ssh -l >>>>username'. I searched and read the archived mail on the cygwin list >>>>archives and though there are six pages of "rsync hanging" issues, I >>>>didn't find much help. Unfortunately, the logs are saying nothing >>>>and I couldn't get the strace business to work---no doubt because >>>>I've never used it. Can anyone here suggest anything? -Thanks >>>> >>>> >>>> >>>>---- >>>> >>>>I just installed the latest version of cygwin (1.5.18-1) from >>>>www.cygwin.com on my Windows XP machine. I have had a lot of >>>>success with rsync between Linux boxes but after many months of >>>>mixed results I have had much less than perfect luck with rsync from >>>>a Linux box [LOCAL] to a windows box [REMOTE]---that is, constant >>>>hanging. >>>> >>>>I have tried to follow the rsync FAQ on using strace to figure out >>>>why things are hanging, but I'm not understanding what gets launched >>>>where and in what order. I created the rsync-debug script (below) >>>>as suggested, but it's unclear how to use it. Would you guys mind >>>>giving a step-by-step on how to get this strace info? >>>> >>>>For example, let's say on [LOCAL/Linux] I want to issue the command: >>>> >>>>rsync -avz user@[REMOTE/Windows]:/path/to/stuff/ dest/on/local/ >>>> >>>>Do I first launch the rsync-debug on [REMOTE/Windows]? Do I modify >>>>the above command in order to get things rolling? >>>> >>>>Cheers and thanks in advance. >>>> >>>>-Ken >>>> >>>>---- >>>> >>>>rsync-debug script: >>>> >>>>ulimit -c unlimited strace -f rsync --daemon --no-detach 2>/tmp/rsync- >>>>$$.out >>>> >>>> >>>> >>>> >>>> >>>>-- >>>>Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple >>>>Problem reports: http://cygwin.com/problems.html >>>>Documentation: http://cygwin.com/docs.html FAQ: >>>>http://cygwin.com/faq/ >>>> >>>> >>>> >>>> >>>> >>>---------------------------------------------------------------- >>>Brett C. Serkez, Techie >>> >>> >>> >>> >>> >>> >---------------------------------------------------------------- >Brett C. Serkez, Techie > > > > -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/