DomainKey-Signature: a=rsa-sha1; c=nofws; d=sourceware.org; h=list-id
	:list-unsubscribe:list-subscribe:list-archive:list-post
	:list-help:sender:mime-version:in-reply-to:references:from:date
	:message-id:subject:to:content-type; q=dns; s=default; b=UpKHGDj
	qQDPxjOK0GCdOFLeG9OmXJgdVMNlA78MG9f7be3kmA9w0ioKNm5kljAotPMg8r4o
	IqhiI0g6Q7yjodaprRP59gh3tL/nu5GP76uqAXc7ubOhSrDcfcnwOg7NuAR80O2G
	HIvecda6hTsYlI7UGRxGMJzle4UhE6+Zl1MY=
Mailing-List: contact cygwin-help AT cygwin DOT com; run by ezmlm
Sender: cygwin-owner AT cygwin DOT com
Mail-Followup-To: cygwin AT cygwin DOT com
MIME-Version: 1.0
In-Reply-To: <5adc37f5-608b-6c1f-6d14-83343c82dc9f@SystematicSw.ab.ca>
References: <CAO1c0AT52ZzCKc3yD_7eX6ri4VubrTb5L+X-4vHWoXBAX2jsfw AT mail DOT gmail DOT com> <5adc37f5-608b-6c1f-6d14-83343c82dc9f AT SystematicSw DOT ab DOT ca>
From: Daniel Havey <dhavey AT gmail DOT com>
Date: Tue, 29 Nov 2016 17:08:07 -0800
Message-ID: <CAO1c0ARyXv76_7WO3mvZFjmZ1yYw8Q8bcz5e37YpuVmqhJwejg@mail.gmail.com>
Subject: Re: Cygwin TCP slow
To: Brian DOT Inglis AT systematicsw DOT ab DOT ca, cygwin AT cygwin DOT com
Content-Type: text/plain; charset=UTF-8

Okay, I will find some time to produce a patch.  It might take a while
though because I have a day job :).  BTW, what the heck is an STC?
Here is an experiment with three machines like this: O----O----O
The one in the middle has a 50ms of delay (25ms in each direction).

Here are the results from Cygwin on top of normal Windows:
[send side perf]
f:\home>iperf3 -c 10.178.204.101
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-10.00  sec  47.5 MBytes  39.8 Mbits/sec                  sender
[  4]   0.00-10.00  sec  47.5 MBytes  39.8 Mbits/sec                  receiver

[receive side perf]
f:\home>iperf3 -c 10.178.204.101 -R
[ ID] Interval           Transfer     Bandwidth       Retr
[  4]   0.00-10.00  sec  41.2 MBytes  34.5 Mbits/sec    0             sender
[  4]   0.00-10.00  sec  39.7 MBytes  33.3 Mbits/sec                  receiver

This matches the calculated performance.  Then we made a private build
of Windows that ignores SO_RCVBUF and SO_SNDBUF and just always uses
autotuning no matter what the app does.
[send side perf]
C:\testbox\tests>iperf3 -c 10.178.204.101
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-10.00  sec   540 MBytes   453 Mbits/sec                  sender
[  4]   0.00-10.00  sec   540 MBytes   453 Mbits/sec                  receiver

[receive side perf]
C:\testbox\tests>iperf3 -c 10.178.204.101 -R
[ ID] Interval           Transfer     Bandwidth       Retr
[  4]   0.00-10.00  sec   553 MBytes   464 Mbits/sec    0             sender
[  4]   0.00-10.00  sec   553 MBytes   464 Mbits/sec                  receiver

If you set SO_RCVBUF and SO_SNDBUF then performance will be limited
according to the calculated values.  If you don't set those values and
let Windows autotuning do its thing then you will always get the
maximum available throughput.

I'll email again when I have the patch.  If you would like more
testing let me know and we can have our test people run some more
experiments.

thanxs :)
...Daniel


On Mon, Nov 28, 2016 at 12:51 PM, Brian Inglis
<Brian DOT Inglis AT systematicsw DOT ab DOT ca> wrote:
> On 2016-11-28 12:54, Daniel Havey wrote:
>> We have had complaints from several large hardware vendors that
>> Windows networking is slow for apps like iperf that are used to
>> measure throughput.  Iperf on Windows is compiled against the
>> cygwin1.dll.  We have root caused the problem to a couple of lines of
>> code in net.cc that set SO_RCVBUF and SO_SNDBUF to about 200KB.
>>
>> The theoretical window/RTT plot for the buffer size set by Cygwin
>> (0x34000 = 200KB) gives us:
>> 1ms -> 1703Mbps
>> 2ms -> 851Mbps
>> 3ms -> 567Mbps
>> 4ms -> 425Mbps
>> 5ms -> 340Mbps
>> 6ms -> 283Mbps
>> 7ms -> 243Mbps
>> 8ms -> 212Mbps
>> 9ms -> 189Mbps
>> 10ms -> 170Mbps
>> 20ms -> 85Mbps
>> 40ms -> 42Mbps
>> 60ms -> 28Mbps
>> 80ms -> 21Mbps
>>
>> We have confirmed this by experiment and also confirmed that the
>> limitation goes away if the buffers are not manually set.  Windows has
>> autotuning and when the buffers are set manually the autotuning is
>> disabled.  This is causing the throughput limitation.  So we would
>> like to formally ask that you please not manually set SO_RCVBUF or
>> SO_SNDBUF.
>
> See problem reports: http://cygwin.com/problems.html
> Provide STC, patches, attach cygcheck -svr output?
> Links to downstream bug reports, testing, results?
> Note that Cygwin iperf is year old 2.0.5.
>
> --
> Take care. Thanks, Brian Inglis, Calgary, Alberta, Canada
>
> --
> Problem reports:       http://cygwin.com/problems.html
> FAQ:                   http://cygwin.com/faq/
> Documentation:         http://cygwin.com/docs.html
> Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
>

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple