X-Recipient:	archive-cygwin AT delorie DOT com
X-Original-To:	cygwin AT cygwin DOT com
Delivered-To:	cygwin AT cygwin DOT com
DMARC-Filter:	OpenDMARC Filter v1.4.1 sourceware.org 345FD385842B
Authentication-Results:	sourceware.org;
	dmarc=fail (p=none dis=none) header.from=nifty.ne.jp
Authentication-Results:	sourceware.org; spf=fail smtp.mailfrom=nifty.ne.jp
DKIM-Filter:	OpenDKIM Filter v2.10.3 conssluserg-03.nifty.com 23REDbqS005934
DKIM-Signature:	v=1; a=rsa-sha256; c=relaxed/relaxed; d=nifty.ne.jp;
	s=dec2015msa; t=1651068818;
	bh=YM3hJKGPFLli0v1IZtaUVAENJBFZwIemplzS/wZhsYQ=;
	h=Date:From:To:Subject:In-Reply-To:References:From;
	b=qU7DMR2OQZypUBTuTdcXPSSJO+vDxQkUI8yf8Z2TrrjPK0YwFXNlivAQKQDPgEwHl
	QjRdfRndRtm7fvrJhNN7hQnvlmwxjI1WGHmkk1xWKXCNaG3K8AN573zxq/md+D8SAj
	ojCAJhmmtSkEIsn4qNUU45jpASHLfeg8GZEdYO1+JXMws/iWIbt5We24IWknEJNJpm
	FlwOWHOQ2gPYH7uPWWvvEjFp361TGAzu0vf65Gh7mj2W1wAIzoVGlVOPHExIhujmMu
	K0un7LY7gsZKayTULpn7PltC3WVTXZHNWaN+wyCY6fnN7XEV25aHTZ/OUPwPBgA6kB
	SLYJAuOoYfGIg==
X-Nifty-SrcIP:	[119.150.44.95]
Date:	Wed, 27 Apr 2022 23:13:40 +0900
From:	Takashi Yano <takashi DOT yano AT nifty DOT ne DOT jp>
To:	cygwin AT cygwin DOT com
Subject:	Re: GNU make losing jobserver tokens
Message-Id:	<20220427231340.0bdab2d6d903dd4fb44bd12f@nifty.ne.jp>
In-Reply-To:	<20220401174551.820cbc148852554108397e03@nifty.ne.jp>
References:	<9b9da583-124d-9d5f-4c10-6622602ca8dc AT oracle DOT com>
	<20220401174551 DOT 820cbc148852554108397e03 AT nifty DOT ne DOT jp>
X-Mailer:	Sylpheed 3.7.0 (GTK+ 2.24.30; i686-pc-mingw32)
Mime-Version:	1.0
X-Spam-Status:	No, score=-11.6 required=5.0 tests=BAYES_00, DKIM_SIGNED,
	DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, NICE_REPLY_A,
	RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS,
	TXREP autolearn=ham autolearn_force=no version=3.4.4
X-Spam-Checker-Version:	SpamAssassin 3.4.4 (2020-01-24) on
	server2.sourceware.org
X-BeenThere:	cygwin AT cygwin DOT com
X-Mailman-Version:	2.1.29
List-Id:	General Cygwin discussions and problem reports <cygwin.cygwin.com>
List-Unsubscribe:	<https://cygwin.com/mailman/options/cygwin>,
	<mailto:cygwin-request AT cygwin DOT com?subject=unsubscribe>
List-Archive:	<https://cygwin.com/pipermail/cygwin/>
List-Post:	<mailto:cygwin AT cygwin DOT com>
List-Help:	<mailto:cygwin-request AT cygwin DOT com?subject=help>
List-Subscribe:	<https://cygwin.com/mailman/listinfo/cygwin>,
	<mailto:cygwin-request AT cygwin DOT com?subject=subscribe>
Errors-To:	cygwin-bounces+archive-cygwin=delorie DOT com AT cygwin DOT com
Sender:	"Cygwin" <cygwin-bounces+archive-cygwin=delorie DOT com AT cygwin DOT com>

On Fri, 1 Apr 2022 17:45:51 +0900
Takashi Yano wrote:
> On Mon, 21 Mar 2022 15:28:17 +0100
> Magnus Ihse Bursie wrote:
> > Hi,
> > 
> > I'm working for Oracle on the OpenJDK build team. We're using GNU make 
> > to build the JDK on all supported platforms. For Windows, we use Cygwin 
> > as our build environment, including the Cygwin version of GNU make.
> > 
> > We have had a long-standing issue with make losing jobserver tokens. 
> > ("long-standing" here means for years, and years, at least since GNU 
> > make 4.0, up to and including the current latest version in Cygwin.)
> > 
> > Most runs end with something like:
> > 
> > make[2]: INTERNAL: Exiting with 11 jobserver tokens available; should be 
> > 12!
> > 
> > Since the build still succeeds, and it just affects performance (and 
> > typically not that much), we have not spend too much time getting to the 
> > bottom of this.
> > 
> > Now, however, I've come across a machine where this happens repeatedly, 
> > and on a much worse scale:
> > 
> > make[2]: INTERNAL: Exiting with 1 jobserver tokens available; should be 24!
> > 
> > This effectively turns the highly parallelized builds into 
> > single-threaded builds, and is absolutely detrimental for performance. 
> > On the flip side, this also makes for the perfect testing environment to 
> > really get to the bottom of this issue.
> > 
> > I started out by sending a question to bug-make AT gnu DOT org. The folks over 
> > there reported that this was not a known problem with GNU make on 
> > Windows in general, and that as far as they knew, the mingw port did not 
> > suffer from this problem.
> > 
> > Instead, they suggested that it was a Cygwin-specific problem, possibly 
> > related to issues with emulating Posix pipes and/or signals in Cygwin.
> > 
> > So, my first question is: Is this a known problem in Cygwin GNU make? 
> > Are there any workarounds/fixes to get around it?
> > 
> > Otherwise: Any suggestions on how to go on and debug this? I am willing 
> > to build and test an instrumented debug build of make, but I will need 
> > assistance to find my way around the source and spot likely candidates 
> > for the source of the problem.
> 
> I have tried to reproduce the issue by building OpenJDK
> from source, however, I could not.
> 
> Instead, I encountered another issue.
> 
> Building OpenJDK sometimes (rarely) failed with error such as:
> 
>       0 [sig] make 5484 sig_send: error sending signal 11, pid 5484, pipe handle 0x118, nb 0, packsize 176, Win32 error 0
>  124917 [main] make 5484 sig_send: error sending signal -72, pid 5484, pipe handle 0x118, nb 0, packsize 176, Win32 error 0
> common/modules/GensrcModuleInfo.gmk:77: *** open: /home/yano/jdk/build/windows-x86-server-release/make-support/vardeps/make/common/modules/GensrcModuleInfo.gmk/jdk.accessibility/ALL_MODULES.vardeps: No such file or directory.  Stop.
> make[2]: *** [make/Main.gmk:141: jdk.accessibility-gensrc-moduleinfo] Error 2
> make[2]: *** Waiting for unfinished jobs....
> 
> 
> I looked into this new problem and found that wait_sig() thread
> crashes with segfault. It seems that accessing _main_tls causes
> access violation if a signal is sent just after the process is
> started.
> 
> static void WINAPI
> wait_sig (VOID *)
> {
>   [...]
>       if (!pack.mask)
> 	{
> 	  tl_entry = cygheap->find_tls (_main_tls);
> 	  dummy_mask = _main_tls->sigmask;       // <--- Segfault here
> 	  cygheap->unlock_tls (tl_entry);
> 	  pack.mask = &dummy_mask;
> 	}
> 
> I also found the following patch resolves the issue.
> 
> diff --git a/winsup/cygwin/sigproc.cc b/winsup/cygwin/sigproc.cc
> index 62df96652..3824af199 100644
> --- a/winsup/cygwin/sigproc.cc
> +++ b/winsup/cygwin/sigproc.cc
> @@ -1325,6 +1325,10 @@ wait_sig (VOID *)
>    _sig_tls = &_my_tls;
>    bool sig_held = false;
>  
> +  /* Wait for _main_tls initialization. */
> +  while (!cygwin_finished_initializing)
> +    Sleep (10);
> +
>    sigproc_printf ("entering ReadFile loop, my_readsig %p, my_sendsig %p",
>  		  my_readsig, my_sendsig);
>  
> 
> I guess _main_tls may not be initialized correctly until
> cygwin_finished_initializing is set.
> 
> Any comments would be appreciated.

Ping?

-- 
Takashi Yano <takashi DOT yano AT nifty DOT ne DOT jp>

-- 
Problem reports:      https://cygwin.com/problems.html
FAQ:                  https://cygwin.com/faq/
Documentation:        https://cygwin.com/docs.html
Unsubscribe info:     https://cygwin.com/ml/#unsubscribe-simple

- Raw text -