delorie.com/archives/browse.cgi   search  
Mail Archives: cygwin/2009/12/16/09:35:20

X-Recipient: archive-cygwin AT delorie DOT com
X-SWARE-Spam-Status: No, hits=-2.3 required=5.0 tests=AWL,BAYES_00
X-Spam-Check-By: sourceware.org
Message-ID: <4B28EFEE.9050005@cs.umass.edu>
Date: Wed, 16 Dec 2009 09:34:22 -0500
From: Eliot Moss <moss AT cs DOT umass DOT edu>
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.1.5) Gecko/20091204 Thunderbird/3.0
MIME-Version: 1.0
To: rsync AT lists DOT samba DOT org, cygwin <cygwin AT cygwin DOT com>
Subject: rsync hang on large file in a set
X-IsSubscribed: yes
Mailing-List: contact cygwin-help AT cygwin DOT com; run by ezmlm
List-Id: <cygwin.cygwin.com>
List-Subscribe: <mailto:cygwin-subscribe AT cygwin DOT com>
List-Archive: <http://sourceware.org/ml/cygwin/>
List-Post: <mailto:cygwin AT cygwin DOT com>
List-Help: <mailto:cygwin-help AT cygwin DOT com>, <http://sourceware.org/ml/#faqs>
Sender: cygwin-owner AT cygwin DOT com
Mail-Followup-To: cygwin AT cygwin DOT com
Delivered-To: mailing list cygwin AT cygwin DOT com

This is to both the rsync and cygwin lists, and I
regret that it is vaguer than I would like, but ...

With cygwin 1.7 (latest beta) and the rsync released
with it (3.0.6-1) I try to sync a couple of Gb of
Thunderbird mail files. The flag settings were
-avzuP plus a --chmod and some includes/excludes
to control which files to move.

I would repeatably get "hangs" on particular files,
which are generally modified by appending (mail
added at the end). These files (two in particular)
are approximately 100Mb or a little more in size,
and that seemed to have something to do with it.

The "hangs" I speak of are where rsync and ssh spin,
consuming cpu time (fan running fast!) but making
no progress.

Now here's the odd thing. If I kill of the rsync and
ssh processes and rsync the one file by itself, it
will go through. (Of course the --partial implied
by -P means that I may have to transfer a lot of
data :-( ).

Curiously, I got the same behavior with a recent rsync
head (20091129).

Last night I made two changes and have not seen a hang
since (though I am monitoring this, of course). One
was to substitute --progress and --partial-dir for
-P.

The other -- which is the cygwin relevant part -- was
the compile with socketpair turned off (manually in the
config file). I did this because Corinna and I had found
issues between socketpair and BLODA on my (Windows 7) box,
which she appeared to have fixed in revisions to 1.7 The
problems had to do with fork, dup, and such, not with
actual data transfer AFAIK. At her request I still have
the BLODA installed, and other cygwin things generally
work right.

I will also play with putting socketpair back in and
keeping --partial-dir. But the various reports of rsync
hangs, and a particular report a little while ago of
busy looping in buffer transfer (and I do have -z on,
might that impact?) struck me as possibly related. I
am not sure how to debug this when it happens, so if
there are steps I can take upon such a hang to get you
better info, do enlighten me!

Best wishes -- Eliot Moss
==============================================================================
J. Eliot B. Moss, Professor               http://www.cs.umass.edu/~moss    www
Director, Arch. and Lang. Impl. Lab.      +1-413-545-4206                voice
Department of Computer Science            +1-413-695-4226                 cell
140 Governor's Drive, Room 372            +1-413-545-1249                  fax
University of Massachusetts at Amherst    moss AT cs DOT umass DOT edu              email
Amherst, MA  01003-9264  USA              +1-413-545-2746 Laurie Downey  sec'y
==============================================================================

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

- Raw text -


  webmaster     delorie software   privacy  
  Copyright © 2019   by DJ Delorie     Updated Jul 2019