X-Recipient: archive-cygwin AT delorie DOT com
X-Original-To: cygwin AT cygwin DOT com
Delivered-To: cygwin AT cygwin DOT com
DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org E22293858401
Authentication-Results: sourceware.org;
 dmarc=none (p=none dis=none) header.from=house.org
Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=house.org
X-Virus-Scanned: Debian amavisd-new at emo02-pco.easydns.vpn
Mime-Version: 1.0 (Mac OS X Mail 6.2 \(1499\))
Subject: Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?
From: Chris Roehrig <croehrig AT house DOT org>
In-Reply-To: <a446765a-1a6c-ad1a-08bb-af69bd6a614c@maxrnd.com>
Date: Tue, 24 Aug 2021 17:35:15 -0700
Message-Id: <28B175CC-A653-4279-A683-A4759029DDDD@house.org>
References: <41A583E1-C8E7-42AB-9F24-EEC33A41EC60 AT house DOT org>
 <a446765a-1a6c-ad1a-08bb-af69bd6a614c AT maxrnd DOT com>
To: Mark Geisert <mark AT maxrnd DOT com>
X-Mailer: Apple Mail (2.1499)
X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00, KAM_DMARC_STATUS,
 SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.4
X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on
 server2.sourceware.org
X-BeenThere: cygwin AT cygwin DOT com
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: General Cygwin discussions and problem reports <cygwin.cygwin.com>
List-Unsubscribe: <https://cygwin.com/mailman/options/cygwin>,
 <mailto:cygwin-request AT cygwin DOT com?subject=unsubscribe>
List-Archive: <https://cygwin.com/pipermail/cygwin/>
List-Post: <mailto:cygwin AT cygwin DOT com>
List-Help: <mailto:cygwin-request AT cygwin DOT com?subject=help>
List-Subscribe: <https://cygwin.com/mailman/listinfo/cygwin>,
 <mailto:cygwin-request AT cygwin DOT com?subject=subscribe>
Cc: cygwin AT cygwin DOT com
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Errors-To: cygwin-bounces+archive-cygwin=delorie DOT com AT cygwin DOT com
Sender: "Cygwin" <cygwin-bounces+archive-cygwin=delorie DOT com AT cygwin DOT com>

Here's my test in a nutshell:

# ON WINDOWS: install Cygwin and enable cygsshd
	ssh-host-config -y
	# set up authorized keys, etc to make things easier

# LINUX: Create a 2GB random file on Linux:
	dd if=/dev/urandom of=/tmp/bigfile.bin bs=1M count=2000

# WINDOWS: rsync "pull" to cygwin
	LINUXHOST=mylinuxhost
	rsync -Pva $LINUXHOST:/tmp/bigfile.bin /tmp/            # 100MB/s full speed
	rm /tmp/bigfile.bin

# LINUX: rsync "push" to cygwin (from Linux machine)
	WINHOST=mywindowshost
	rsync -Pva /tmp/bigfile.bin $WINHOST:/tmp/              # slow, dropping to 4MB/s


I get the same results transferring the other direction (WIN -> LINUX):   The Cygwin remote endpoint is always slow.





On Tue Aug 24 2021, at 4:43 PM, Mark Geisert <mark AT maxrnd DOT com> wrote:

> Chris Roehrig wrote:
>> I have a network of Windows, Linux and Mac machines and I use rsync to synchronize various directories between them.
>> I'm trying to figure out why my rsync transfers are so slow (<4 MB/s) only when the remote endpoint is Cygwin rsync over sshd (with both a Linux or Cygwin rsync client).   In all other scenarios, I get the full 100MB/s as expected from gigabit ethernet.  This has been an ongoing problem for me for a couple of years over several Windows and Cygwin versions, and I'd like to try to fix it.
>> If I run rsync --daemon --no-detach under mintty in the foreground on the remote Windows endpoint,  I get the full 100 MB/s transfers, so it seems like it has something to do with rsync.exe running in the background under the cygrunsrv+sshd service (which was installed normally using ssh-host-config).
>> If I do:
>> 	pv /dev/zero | ssh $WINHOST "cat > /dev/null"
>> or even
>> 	pv /dev/urandom | ssh $WINHOST md5sum
>> I also get the full 100 MB/s transfers, so it doesn't look like sshd itself is being throttled by bandwidth or CPU.
>> The machines have less than 15% CPU utilization while transferring, with each of the 4 cores less than 30%, so it doesn't look to be CPU issue.
>> In Task Manager, sshd.exe and rsync.exe seem to be running normally using only few percent CPU, and show Power Throttling=Disabled, Priority=Normal.   Setting their Priority to High doesn't seem to change things.
>> Looking in Resource Monitor on the remote endpoint, the network usage is pretty much a flat horizontal line at about 18 Mbps (2.5 MB/s), so it sure looks to me as if rsync is somehow being bandwidth-throttled when run in the background under cygsshd.
>> It's almost as if rsync has an implicit --bwlimit override when it is run from cygrunsrv+sshd (I've tried --bwlimit=0 on the client which makes no difference).
>> Any ideas?    Not sure where to go from here.
> 
> You're not the first to report this.  I don't have any quick answer.  But can you give one or two simple examples of commands that give slow transfers in your environment?  Simple like your 'pv' examples, if possible, using whatever method that works.
> 
> ..mark
> 
> -- 
> Problem reports:      https://cygwin.com/problems.html
> FAQ:                  https://cygwin.com/faq/
> Documentation:        https://cygwin.com/docs.html
> Unsubscribe info:     https://cygwin.com/ml/#unsubscribe-simple


-- 
Problem reports:      https://cygwin.com/problems.html
FAQ:                  https://cygwin.com/faq/
Documentation:        https://cygwin.com/docs.html
Unsubscribe info:     https://cygwin.com/ml/#unsubscribe-simple