DomainKey-Signature: a=rsa-sha1; c=nofws; d=sourceware.org; h=list-id
	:list-unsubscribe:list-subscribe:list-archive:list-post
	:list-help:sender:mime-version:in-reply-to:references:from:date
	:message-id:subject:to:content-type; q=dns; s=default; b=L4uYpZT
	jemQESZGClMGScG2d4owcqodFm1eqYlBZipzRz7ehDlKjwU0omRY670K0xo/W0Pl
	C50lhFhHAU4Sq6IQHz6u0wDm0wxuLtKzuby/iIGH80SVQzhlkfDVHrshEcCrNLUt
	5RVwmBhcgzFFMpKQ8X8adFrBlOc4oGnUA5J8=
Mailing-List: contact cygwin-help@cygwin.com; run by ezmlm
Sender: cygwin-owner@cygwin.com
Mail-Followup-To: cygwin@cygwin.com
MIME-Version: 1.0
In-Reply-To: <CAGpXXZJV-nsB=xyO7=EBUb-JCk5KqMPS=bvg2aWgw_jm4OZFsQ@mail.gmail.com>
References: <CAGpXXZJV-nsB=xyO7=EBUb-JCk5KqMPS=bvg2aWgw_jm4OZFsQ@mail.gmail.com>
From: Greg Freemyer <greg.freemyer@gmail.com>
Date: Mon, 11 Apr 2016 12:57:50 -0400
Message-ID: <CAGpXXZLJ=O7y-6YEcODv4r32X96zGSU5PX9KwzwF3QedSV4iHA@mail.gmail.com>
Subject: Re: rsync performance degradation? Could be a windows issue?
To: cygwin@cygwin.com
Content-Type: text/plain; charset=UTF-8

On Thu, Mar 31, 2016 at 4:48 PM, Greg Freemyer <greg.freemyer@gmail.com> wrote:
> All,
>
> I'm not sure this is on-topic, but at least I'm in cygwin when I
> notice the below:
>
> I do a lot of large data transfers between USB drives.  Often I get
> great speeds (70MB/sec or more).
>
> Sometimes it falls way off to closer to 20MB/sec with the same class
> of hardware.
>
> I experienced the "slow" transfer speed today, so I thought I'd ask if
> anyone knew a way to resolve it?

I'm trying rsync of a bunch of large files again today.

Initially I saw the same slow copy speed (about 20 MB/sec).  My
destination was connected to a USB 3.1 port, but it seemed to be the
bottleneck so I moved it to a USB 3 port (3.1 should be 2x the speed
of 3.0, so it should not have been the bottleneck).

After getting my throughput up above 50 MB/sec I notice in the windows
"performance monitor" that my source disk is hitting 100% utilization,
then a few seconds later my destination disk is.  And back and forth.
It seems I'm only reading or writing for a few seconds, then
alternating.

I assume the issue is that too much data is being read / cached by
rsync prior to it being written out so I'm getting no advantage of
reading and writing in parallel.

(I gather iostat isn't available for cygwin?)

Is there some rsync (or cygwin) option / feature that would encourage
parallel reading/writing?

fyi: I did some linux testing with "dd" over the weekend and I hit 140
MB/sec if I used a 100MB blocksize.  That was to / from the raw disk
(/dev/sda => /dev/sdb).

If I bumped my blocksize to 1GB for dd in linux, my throughput dropped
to 70 MB/sec just as I see right now with rsync in cygwin.

Thanks
Greg

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple