X-Recipient: archive-cygwin AT delorie DOT com
DomainKey-Signature: a=rsa-sha1; c=nofws; d=sourceware.org; h=list-id
	:list-unsubscribe:list-subscribe:list-archive:list-post
	:list-help:sender:mime-version:in-reply-to:references:date
	:message-id:subject:from:to:content-type; q=dns; s=default; b=Oe
	/kNfOhagHVircs0+HDaLK8mhXStP2LC7/ZIabw5MQvbxZQlL0eoX8F+gBVt8Wf62
	8fDVg5laIy6LUnMqsgKRwEZ6CLEK5neSNZB0D7gOLMqXuOTK2RsNZNb3bDUqsSOd
	F2SdxUSwjtESkN3GMwQfigQrgBJDIpYB4Kc6tJzx8=
DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=sourceware.org; h=list-id
	:list-unsubscribe:list-subscribe:list-archive:list-post
	:list-help:sender:mime-version:in-reply-to:references:date
	:message-id:subject:from:to:content-type; s=default; bh=1LwQiojW
	gGzGEy4XOJQT2LyOv0c=; b=oEMIlUR6fFtqI9vxYkQrtu7UlEU5Kqp9aOPwnqMm
	6MePq0SxktsfvdbyzzQrwL4W581u9Q3w7BpJ4t2Y4S5pCT9jGpnCGncZpOIK71Q8
	Zo4hhX5nXg8rJetIEMg0Kkl+wMS9yk9Y84D83oMaUI4tLRTmSmYXy1GGIKQbSrzj
	xJ0=
Mailing-List: contact cygwin-help AT cygwin DOT com; run by ezmlm
List-Id: <cygwin.cygwin.com>
List-Subscribe: <mailto:cygwin-subscribe AT cygwin DOT com>
List-Archive: <http://sourceware.org/ml/cygwin/>
List-Post: <mailto:cygwin AT cygwin DOT com>
List-Help: <mailto:cygwin-help AT cygwin DOT com>, <http://sourceware.org/ml/#faqs>
Sender: cygwin-owner AT cygwin DOT com
Mail-Followup-To: cygwin AT cygwin DOT com
Delivered-To: mailing list cygwin AT cygwin DOT com
Authentication-Results: sourceware.org; auth=none
X-Virus-Found: No
X-Spam-SWARE-Status: No, score=0.2 required=5.0 tests=AWL,BAYES_50,FREEMAIL_ENVFROM_END_DIGIT,FREEMAIL_FROM,RCVD_IN_DNSWL_LOW,SPF_PASS autolearn=ham version=3.3.2
X-HELO: mail-wm0-f53.google.com
MIME-Version: 1.0
X-Received: by 10.194.79.72 with SMTP id h8mr32262415wjx.136.1449435386153; Sun, 06 Dec 2015 12:56:26 -0800 (PST)
In-Reply-To: <5663EB9A.40002@maxrnd.com>
References: <CABPLASTtRK4mNxh0M_AnZgjJQ15kWPx+L=U=VCU3Wwi7jV_57A AT mail DOT gmail DOT com>	<564E3017 DOT 90205 AT maxrnd DOT com>	<CABPLASTLrH_udLuu2F-m5P6dkENW1Z4YHEudp4NG0-FGLJgPMg AT mail DOT gmail DOT com>	<5650379B DOT 4030405 AT maxrnd DOT com>	<20151121105301 DOT GE2755 AT calimero DOT vinschen DOT de>	<5652C402 DOT 7040006 AT maxrnd DOT com>	<24780-1448274431-7444 AT sneakemail DOT com>	<5653B52B DOT 5000804 AT maxrnd DOT com>	<20151126093427 DOT GJ2755 AT calimero DOT vinschen DOT de>	<5656DDEF DOT 9070603 AT maxrnd DOT com>	<5662C199 DOT 7040906 AT maxrnd DOT com>	<CABPLAST5EnifrAQ2xKZmohKhyxQHh=K3x3DeCL+BTdHN8nN98w AT mail DOT gmail DOT com>	<566367C8 DOT 5020703 AT maxrnd DOT com>	<CABPLASSY3WWpHAeh=5gqRKdg85M8Wzkrq9qMaDhzhk2zvxgcOw AT mail DOT gmail DOT com>	<5663EB9A DOT 40002 AT maxrnd DOT com>
Date: Sun, 6 Dec 2015 21:56:26 +0100
Message-ID: <CABPLASQZrDMnN32GG3-qRsnHhjsoroaY7ti1wx5uASDqdU7M+g@mail.gmail.com>
Subject: Re: Cygwin multithreading performance
From: Kacper Michajlow <kasper93 AT gmail DOT com>
To: cygwin AT cygwin DOT com
Content-Type: text/plain; charset=UTF-8
X-IsSubscribed: yes

2015-12-06 9:02 GMT+01:00 Mark Geisert <mark AT maxrnd DOT com>:
> Kacper Michajlow wrote:
>>
>> 2015-12-05 23:40 GMT+01:00 Mark Geisert <mark AT maxrnd DOT com>:
>>>
>>> It looks like we're going to have to compare actual pthread_mutex_lock()
>>> implementations.  Inspecting source is nice but I don't want to be
>>> chasing a
>>> mirage so I really hope there's a pthread_mutex_lock() function inside
>>> the
>>> MinGW git you are running.  gdb could easily answer that question.  Could
>>> you please do an 'info func pthread_mutex_lock' after starting MinGW git
>>> under MinGW gdb with a breakpoint at main() (so libraries are loaded).
>
> [...]
>>
>> Hmm, thinking about it mingw doesn't have pthread implementation or
>> any wrapper for it. If someone needs pthread they would probably go
>> for pthreads-w32 implementation.
>>
>> I started to wonder because I don't recall git would need pthreads to
>> compile on Windows. And indeed they have a wrapper for Windows API...
>> https://github.com/git/git/blob/master/compat/win32/pthread.h
>> https://github.com/git/git/blob/master/compat/win32/pthread.c
>
>
> OK, so git has its own pthread_mutex_lock/unlock ops which map to very
> light-weight critical section operations.
>
>> Though it is not really a matter that "native" git build is fast and
>> all, but that Cygwin's one really struggles if it comes to MT workload.
>
>
> In the worst cases I see using your testcase, about half the time the
> busiest locks are processed within 1 usec but there's a spectrum of longer
> latencies for the other half of the time.  I don't know (yet) if that can be
> improved in Cygwin's more general implementation but at least the matter has
> now been brought to our attention :).
,
Yes, I can imagine, git's objects are very small so threading overhead
is very noticeable.

>> And this not only issue with git unfortunately. Download speeds are
>> also limited on Cygwin. I know POSIX compatibility layers comes with a
>> price but I would love to see improvements in those areas.
>> Cygwin:
>> Receiving objects: 100% (230458/230458), 78.41 MiB | 1.53 MiB/s, done.
>> "native" git:
>> Receiving objects: 100% (230458/230458), 78.41 MiB | 18.54 MiB/s, done.
>
>
> You're asserting this additional testcase has the same cause.  What is
> telling you that?  And FTR what is the git command you are issuing?  I can
> then do the lock latency analysis on this new testcase if warranted.

No, sorry, I mixed different things. It is just that I'm ruining both
git build lately and I wanted to share another issue before I forget
about it.

This was git clone command for some random repository from github.
There is a lot factors at hand here but the fact is with cygwin speed
is capped on 1.5MB/s and this is reproducible. This is probably also
related to the fact that git operates on large amount small object.
But this time it is single thread workload. I tried strace this, but
frankly I am not sure what to look for.

All in all I just want to bring those issues to your attention.
Whether it is fixable or not is another story. But we will not know
unless someone with required knowledge analyze it.

-Kacper

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple