X-Recipient: archive-cygwin AT delorie DOT com
DomainKey-Signature: a=rsa-sha1; c=nofws; d=sourceware.org; h=list-id
	:list-unsubscribe:list-subscribe:list-archive:list-post
	:list-help:sender:mime-version:in-reply-to:references:date
	:message-id:subject:from:to:content-type; q=dns; s=default; b=W3
	wgsbEvykVcFW/rtyDzy+HmCQdBjB2ZSlOVatkLIxSBGONQZEkBtL7HFuW28EjrQ3
	rs/J30fYlFuAnESyaApqkc8UaFM/hNjHFPKM4pZQV2E48j/i4ZgFgh8abcqXs2J6
	XXJOZFL9E3p0ztXEYbD6keLQLCZ2wO2kbCHyg51ww=
DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=sourceware.org; h=list-id
	:list-unsubscribe:list-subscribe:list-archive:list-post
	:list-help:sender:mime-version:in-reply-to:references:date
	:message-id:subject:from:to:content-type; s=default; bh=V/V0nVgM
	aWEWHpXJNdzHMKtN7ZY=; b=VLKyWknGzAlpshtPyKwWof0iubR89Uwxx1Mz90b/
	EG9UIqElmsq1vIGxblkBbZRaKucIZ4UcB0WpP/OMqcJhmfuN8fG59dBgG+363lX9
	Uz3vHnvOZkpocvpnC9h2xlXn8/vg6E+/nGfzOJV82WrFVJpRHlF4GC9uHKrk2O8R
	yAk=
Mailing-List: contact cygwin-help AT cygwin DOT com; run by ezmlm
List-Id: <cygwin.cygwin.com>
List-Subscribe: <mailto:cygwin-subscribe AT cygwin DOT com>
List-Archive: <http://sourceware.org/ml/cygwin/>
List-Post: <mailto:cygwin AT cygwin DOT com>
List-Help: <mailto:cygwin-help AT cygwin DOT com>, <http://sourceware.org/ml/#faqs>
Sender: cygwin-owner AT cygwin DOT com
Mail-Followup-To: cygwin AT cygwin DOT com
Delivered-To: mailing list cygwin AT cygwin DOT com
Authentication-Results: sourceware.org; auth=none
X-Virus-Found: No
X-Spam-SWARE-Status: No, score=-0.6 required=5.0 tests=AWL,BAYES_05,FREEMAIL_ENVFROM_END_DIGIT,FREEMAIL_FROM,RCVD_IN_DNSWL_LOW,SPF_PASS autolearn=ham version=3.3.2
X-HELO: mail-wm0-f45.google.com
MIME-Version: 1.0
X-Received: by 10.28.214.143 with SMTP id n137mr13834806wmg.2.1449369282971; Sat, 05 Dec 2015 18:34:42 -0800 (PST)
In-Reply-To: <566367C8.5020703@maxrnd.com>
References: <CABPLASTtRK4mNxh0M_AnZgjJQ15kWPx+L=U=VCU3Wwi7jV_57A AT mail DOT gmail DOT com>	<564E3017 DOT 90205 AT maxrnd DOT com>	<CABPLASTLrH_udLuu2F-m5P6dkENW1Z4YHEudp4NG0-FGLJgPMg AT mail DOT gmail DOT com>	<5650379B DOT 4030405 AT maxrnd DOT com>	<20151121105301 DOT GE2755 AT calimero DOT vinschen DOT de>	<5652C402 DOT 7040006 AT maxrnd DOT com>	<24780-1448274431-7444 AT sneakemail DOT com>	<5653B52B DOT 5000804 AT maxrnd DOT com>	<20151126093427 DOT GJ2755 AT calimero DOT vinschen DOT de>	<5656DDEF DOT 9070603 AT maxrnd DOT com>	<5662C199 DOT 7040906 AT maxrnd DOT com>	<CABPLAST5EnifrAQ2xKZmohKhyxQHh=K3x3DeCL+BTdHN8nN98w AT mail DOT gmail DOT com>	<566367C8 DOT 5020703 AT maxrnd DOT com>
Date: Sun, 6 Dec 2015 03:34:42 +0100
Message-ID: <CABPLASSY3WWpHAeh=5gqRKdg85M8Wzkrq9qMaDhzhk2zvxgcOw@mail.gmail.com>
Subject: Re: Cygwin multithreading performance
From: Kacper Michajlow <kasper93 AT gmail DOT com>
To: cygwin AT cygwin DOT com
Content-Type: text/plain; charset=UTF-8
X-IsSubscribed: yes

2015-12-05 23:40 GMT+01:00 Mark Geisert <mark AT maxrnd DOT com>:
> Kacper Michajlow wrote:
>>
>> 2015-12-05 11:51 GMT+01:00 Mark Geisert <mark AT maxrnd DOT com>:
>>>
>>> Mark Geisert wrote:
>>> In the OP's very good testcase the most heavily contended locks, by far,
>>> are
>>> those internal to git's builtin/pack-objects.c.  I plan to show actual
>>> stats
>>> after some more cleanup, but I did notice something in that git source
>>> file
>>> that might explain the difference between Cygwin and MinGW when running
>>> this
>>> testcase...
>>>
>>> #ifndef NO_PTHREADS
>>>
>>> static pthread_mutex_t read_mutex;
>>> #define read_lock()             pthread_mutex_lock(&read_mutex)
>>> #define read_unlock()           pthread_mutex_unlock(&read_mutex)
>>>
>>> static pthread_mutex_t cache_mutex;
>>> #define cache_lock()            pthread_mutex_lock(&cache_mutex)
>>> #define cache_unlock()          pthread_mutex_unlock(&cache_mutex)
>>>
>>> static pthread_mutex_t progress_mutex;
>>> #define progress_lock()         pthread_mutex_lock(&progress_mutex)
>>> #define progress_unlock()       pthread_mutex_unlock(&progress_mutex)
>>>
>>> #else
>>>
>>> #define read_lock()             (void)0
>>> #define read_unlock()           (void)0
>>> #define cache_lock()            (void)0
>>> #define cache_unlock()          (void)0
>>> #define progress_lock()         (void)0
>>> #define progress_unlock()       (void)0
>>>
>>> #endif
>>>
>>> Is it possible the MinGW version of git is compiled with NO_PTHREADS
>>> #defined?  If so, it would mean there's no locking being done at all and
>>> would explain the faster execution and near 100% CPU utilization when
>>> running under MinGW.
>>
>>
>> Nah, there is no threading enabled when there is no pthreads. How
>> would that work? :D See thread-utils.h
>>
>> #ifndef NO_PTHREADS
>> #include <pthread.h>
>>
>> extern int online_cpus(void);
>> extern int init_recursive_mutex(pthread_mutex_t*);
>>
>> #else
>>
>> #define online_cpus() 1
>>
>> #endif
>
>
> We're not familiar at all with MinGW.  Could you locate the source for
> MinGW's pthread_mutex_lock() online and give us a link to it?  And BTW,
> which Windows are you running and on what kind of hardware (bitness and
> #CPUS/threads)?
>
> It looks like we're going to have to compare actual pthread_mutex_lock()
> implementations.  Inspecting source is nice but I don't want to be chasing a
> mirage so I really hope there's a pthread_mutex_lock() function inside the
> MinGW git you are running.  gdb could easily answer that question.  Could
> you please do an 'info func pthread_mutex_lock' after starting MinGW git
> under MinGW gdb with a breakpoint at main() (so libraries are loaded).
>
>
> ..mark
>
>
> --
> Problem reports:       http://cygwin.com/problems.html
> FAQ:                   http://cygwin.com/faq/
> Documentation:         http://cygwin.com/docs.html
> Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
>

Hmm, thinking about it mingw doesn't have pthread implementation or
any wrapper for it. If someone needs pthread they would probably go
for pthreads-w32 implementation.

I started to wonder because I don't recall git would need pthreads to
compile on Windows. And indeed they have a wrapper for Windows API...
https://github.com/git/git/blob/master/compat/win32/pthread.h
https://github.com/git/git/blob/master/compat/win32/pthread.c

Though it is not really a matter that "native" git build is fast and
all, but that Cygwin's one really struggles if it comes to MT workload
.

And this not only issue with git unfortunately. Download speeds are
also limited on Cygwin. I know POSIX compatibility layers comes with a
price but I would love to see improvements in those areas.
Cygwin:
Receiving objects: 100% (230458/230458), 78.41 MiB | 1.53 MiB/s, done.
"native" git:
Receiving objects: 100% (230458/230458), 78.41 MiB | 18.54 MiB/s, done.

I'm on Windows 10 x64 and i7 5820K (6C/12T).

-Kacper

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple