X-Recipient: archive-cygwin AT delorie DOT com
X-SWARE-Spam-Status: No, hits=-1.0 required=5.0	tests=BAYES_00,DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED,SPF_HELO_PASS,T_RP_MATCHES_RCVD
X-Spam-Check-By: sourceware.org
To: cygwin AT cygwin DOT com
From: Richard Ivarson <RiIvarson AT gmail DOT com>
Subject: Re: Rsync stops inmid of synchronisation
Date: Tue, 06 Mar 2012 11:02:26 +0100
Lines: 65
Message-ID: <jj4nen$d04$1@dough.gmane.org>
References: <jiku3n$fg6$1 AT dough DOT gmane DOT org> <4F4E5478 DOT 1080100 AT molconn DOT com>
Mime-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-15; format=flowed
Content-Transfer-Encoding: 7bit
User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:10.0.2) Gecko/20120216 Thunderbird/10.0.2
In-Reply-To: <4F4E5478.1080100@molconn.com>
X-IsSubscribed: yes
Mailing-List: contact cygwin-help AT cygwin DOT com; run by ezmlm
Precedence: bulk
List-Id: <cygwin.cygwin.com>
List-Unsubscribe: <mailto:cygwin-unsubscribe-archive-cygwin=delorie DOT com AT cygwin DOT com>
List-Subscribe: <mailto:cygwin-subscribe AT cygwin DOT com>
List-Archive: <http://sourceware.org/ml/cygwin/>
List-Post: <mailto:cygwin AT cygwin DOT com>
List-Help: <mailto:cygwin-help AT cygwin DOT com>, <http://sourceware.org/ml/#faqs>
Sender: cygwin-owner AT cygwin DOT com
Mail-Followup-To: cygwin AT cygwin DOT com
Delivered-To: mailing list cygwin AT cygwin DOT com

LMH, thank you for your hints.

So I checked all the drives and files and they're good, as well as the other 
hardware.
Also the file's permissions are good.

The idea to sync just one folder after the other is good. It turned out that 
those few folders with many smaller files (up to 35000 files, mainly man 
pages) cause the hang of rsync often, but not always because with a certain 
chance these folders sync fine. That makes it very hard to find the real error.

My syncing script now uses separate rsync commands (i.e. lines in the script) 
for all the "problematic" folders in contrast to before where all the folders 
have been separated by spaces in one rsync command. In combination with 
rsync's "timeout" parameter (see below please) only those rsync commands with 
an "offending" folder are being skipped when the timeout happens, so that the 
remaining rsync commands can continue to run. This is an improvement. :-)

This is what I found out in detail, so far:

- If I rsync the problematic folder to a local device, rsync always runs fine!

- If I rsync the same problematic folder to the remote computer (on a local 
network), the problematic folder makes the sending rsync to hang often. So it 
can't be a problem with the files, it must be some rsync network stuff 
causing the hang.

- If I add the "--dry-run" parameter, then interestingly rsync hangs too at 
about the same position in the problematic folder, i.e. without actually 
having send content data (just information data?) to the receiver.

- So is it the sender or receiver which causes the hang? I think the sender, 
because when I add the rsync parameter "--timeout=30" the rsync command is 
being terminated/skipped at the offending folder and continues with the next 
rsync command in the script. When however I use the parameter 
"--contimeout=30" instead, the scripts waits "forever" -- as it happens with 
no timeout parameter.


So in the end I still don't see what's going on, unfortunately.

-Richard




LMH wrote:
> One thing to check is the disk drives. I have had rsync stop when it reaches
> corrupted sectors, especially if those sectors corrupt part of the file
> system. I don't remember anything helpful in the log files, I just noticed
> that it wasn't finishing. Try running disk diagnostic software and make sure
> your hardware is good. What kind of hard drives do you have and what OS?
>
> If that doesn't show up anything, I would start by synching just one folder.
> Create a test folder on both computers and add a few files. Make sure your
> permissions are correct. Make a change in one file and then run rsync. If it
> finishes and the logs look good, add more to the test folder and see if you
> can find where it brakes. You can test things like very large files, and
> folders with a large number of files. If you can't get it to fail, then the
> issue may be with the data on one of the computers.
>
> The problem may also be with the rsync software, but I can't advise well on
> that end.
>
> LMH


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple