Mail Archives: cygwin/2022/03/21/10:29:02

Magnus Ihse Bursie <magnus DOT ihse DOT bursie AT oracle DOT com> · Mon, 21 Mar 2022 15:28:17 +0100

Hi,

I'm working for Oracle on the OpenJDK build team. We're using GNU make 
to build the JDK on all supported platforms. For Windows, we use Cygwin 
as our build environment, including the Cygwin version of GNU make.

We have had a long-standing issue with make losing jobserver tokens. 
("long-standing" here means for years, and years, at least since GNU 
make 4.0, up to and including the current latest version in Cygwin.)

Most runs end with something like:

make[2]: INTERNAL: Exiting with 11 jobserver tokens available; should be 
12!

Since the build still succeeds, and it just affects performance (and 
typically not that much), we have not spend too much time getting to the 
bottom of this.

Now, however, I've come across a machine where this happens repeatedly, 
and on a much worse scale:

make[2]: INTERNAL: Exiting with 1 jobserver tokens available; should be 24!

This effectively turns the highly parallelized builds into 
single-threaded builds, and is absolutely detrimental for performance. 
On the flip side, this also makes for the perfect testing environment to 
really get to the bottom of this issue.

I started out by sending a question to bug-make AT gnu DOT org. The folks over 
there reported that this was not a known problem with GNU make on 
Windows in general, and that as far as they knew, the mingw port did not 
suffer from this problem.

Instead, they suggested that it was a Cygwin-specific problem, possibly 
related to issues with emulating Posix pipes and/or signals in Cygwin.

So, my first question is: Is this a known problem in Cygwin GNU make? 
Are there any workarounds/fixes to get around it?

Otherwise: Any suggestions on how to go on and debug this? I am willing 
to build and test an instrumented debug build of make, but I will need 
assistance to find my way around the source and spot likely candidates 
for the source of the problem.

/Magnus

-- 
Problem reports:      https://cygwin.com/problems.html
FAQ:                  https://cygwin.com/faq/
Documentation:        https://cygwin.com/docs.html
Unsubscribe info:     https://cygwin.com/ml/#unsubscribe-simple

X-Recipient:	archive-cygwin AT delorie DOT com
X-Original-To:	cygwin AT cygwin DOT com
Delivered-To:	cygwin AT cygwin DOT com
DMARC-Filter:	OpenDMARC Filter v1.4.1 sourceware.org 07AB63858403
Authentication-Results:	sourceware.org;
	dmarc=pass (p=none dis=none) header.from=oracle.com
Authentication-Results:	sourceware.org; spf=pass smtp.mailfrom=oracle.com
DKIM-Signature:	v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com;
	h=message-id : date :
	to : from : subject : content-type : content-transfer-encoding :
	mime-version; s=corp-2021-07-09;
	bh=6EwqCZQ9qaffTFEj6b/8jFwinOyOq9o0zwcyTzA+Ypw=;
	b=mwZsI4HkFG+ml/SXKsMFnXliNnsW8hd1tjqlicDer9j6l4wfOA7dHdLfKL+zi78O8OuM
	sTAjkYs7Q4Pi39tSgjGO64u+ylXdl2cSV9UdWXG7NCoBMVtHGLzRE1M+w+K5XcB90Ocs
	BxfbP12hDhtCwXZiXGj52HixLra7pHyW4jQBFYQfqPThdFSk7SA49WD/JoxT7F9yhbbP
	VKN6s9RE3Z0AkEfJy71yV56J9FsVm4eM0L6COVLpRt3zPvbe2ZvxNtqlwFLeyL+ckPcE
	Gaxa3ZZIw9t+7VMVJKoJ0equowNOop8WAKFGOBc2gAIwjqtj9bosGeNPHg7sYlu5dBcM ig==
ARC-Seal:	i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
	b=fWqTikyexd3sIoEkMr01EzL6hd6eN+xwSYxAYyXAxe5Y+unZRnNVEHaPmjNmOtax1eI2hs3ANwopJ3u2+KkJBDqBhBVvuugDkal4KUbckEeteQar8GXJOXuJwFxN5hkJbeCuMEaPDwF+b5BMnj/Osogzm1XgT5c0dLjrO+pWalOoCMnAoF00aZfpYFgNtJk/wRZre21IPajwnVGVPJHf66bedqJp7FmhYCsFaoV7qb+gFs/xDM9rIQC5Kym5Ah0R0m3Ak3yPAJ0DjwF5BW40FDAMaBYy87CJGrFbGT1I5kPMQfW1wEH0xIjDDgdCY7Pl0yKirCwZ/otNjP4UQFC0HQ==
ARC-Message-Signature:	i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
	s=arcselector9901;
	h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
	bh=6EwqCZQ9qaffTFEj6b/8jFwinOyOq9o0zwcyTzA+Ypw=;
	b=DUV2/jNoAg4MuaLYng6SgmoyoB2qmkBKPrWXYySyXuGzTB7f4TeOt2MyqGQgkdJrJpeIUhZPZiCsky5CFiJeEghia/3wkNfc49lfWje4YSCn/M+hXTI3A9UlVc0enuIRpRHeP+f1vJo4S0tnXbeThSqztHn0ePNNcd6sQyktwMZSGOAh4kE3SCQC7Iqvx5sXO+iAcFGQkhKqfsOa+e6uYFGozDUlEkiwsx1/LkQRXNsKtCIfwQxKqGSFVGCuSIu2QwY1EXAQvgazPu4T76eYCbwMeMBcjYxDxj66kDLjvTRYtqwS1w3Lmv1qloWqqhqtpvv6gULTSaazUQw5xg735Q==
ARC-Authentication-Results:	i=1; mx.microsoft.com 1; spf=pass
	smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com;
	dkim=pass header.d=oracle.com; arc=none
DKIM-Signature:	v=1; a=rsa-sha256; c=relaxed/relaxed;
	d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com;
	h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
	bh=6EwqCZQ9qaffTFEj6b/8jFwinOyOq9o0zwcyTzA+Ypw=;
	b=KHI0/OnGIaVnsSEEngQhr8+kSO9HGliphZaf6F1NP3+VcY2XTOLZY2EEHMLewNHChcemTO07kZ/P+GYjuaAGuUPczGz1lL8ZyFfR1Vig9MbOUQ6Du8G/hsbMjsFgD+R8SugxDVCZydRuy1jkQoCuLnYXEIh+Ei8JMG/Fh2Sw/og=
Message-ID:	<9b9da583-124d-9d5f-4c10-6622602ca8dc@oracle.com>
Date:	Mon, 21 Mar 2022 15:28:17 +0100
User-Agent:	Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101
	Thunderbird/91.5.0
To:	cygwin AT cygwin DOT com
From:	Magnus Ihse Bursie <magnus DOT ihse DOT bursie AT oracle DOT com>
Subject:	GNU make losing jobserver tokens
X-ClientProxiedBy:	LO2P265CA0499.GBRP265.PROD.OUTLOOK.COM
	(2603:10a6:600:13b::6) To DM6PR10MB4089.namprd10.prod.outlook.com
	(2603:10b6:5:159::27)
MIME-Version:	1.0
X-MS-PublicTrafficType:	Email
X-MS-Office365-Filtering-Correlation-Id:	17c98f9b-61c5-45c0-db47-08da0b47058f
X-MS-TrafficTypeDiagnostic:	BY5PR10MB4066:EE_
X-Microsoft-Antispam-PRVS:	<BY5PR10MB4066D412D7CF70383BF6265DAD169 AT BY5PR10MB4066 DOT namprd10 DOT prod DOT outlook DOT com>
X-MS-Exchange-SenderADCheck:	1
X-MS-Exchange-AntiSpam-Relay:	0
X-Microsoft-Antispam:	BCL:0;
X-Microsoft-Antispam-Message-Info:	oVfhiBJQKav3vIOv/itWJr2WTkF9XBD3iUGaayLoCdwNeYhNjYnbYeaq8ojzvy/IO4EOvsmvu8OXVlxe1MJtABUCjGjtWZ/dUu864QsuBZDfOmgO45q8MIZw+0HaOv/HC5bSxtouESHGhJk+/k45Wj5j/LNB7+06ug3mK76uuSUipZ/j9EcOhx8wi6gG1FRAa1HragE1iKksd+uvUg+Ewau47zL5YBuSrMOi1Vy77DpMnZeMnKgJIqpYINrZ2Dp8RTu148ZUJ0UdLgGtR0AOcYFYPYZgOTkECQpiQZyQpqDBONr9nY0S98bXd+oZVTpfwnHFWcDMyTRHGQ8L+3gjPEY6tfN0xDEQszEyiLvMdkO1hrs1doEGha78Xu4ae4Lh16x05K3qwOToDRkrfhdAdhTHmdCAo0kKWy+dRWjXQS2f3tKbqWhF9eiJHyyG4dosXgwoDj98/HghPQzCt68fEY8MQSZ8WDBnTsCpsOQo5LCZ/R5rmlSgC++5ZjlTr/DQRv9wMZI61TFSrm5bUZd2jeoQh0ztdkqoRJEJQXF2lD9+iq7wyx1XAR75EEfTBO+PgscYwFQsLM7mevehFnfbvfik0ifRl8CSvfczOn8bJPMbY8H4tH0QwhL410I8xpE78B+x4WG0A+WrskdHAaXxpGYOSl4CsKv3aLO/4KmB5URwxFIIcIEmticRaUUMLDyg+TCEdvHTAbC4r47+8W9A5qi+s0GCwqj7SIorQGtR1og=
X-Forefront-Antispam-Report:	CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:;
	IPV:NLI; SFV:NSPM; H:DM6PR10MB4089.namprd10.prod.outlook.com; PTR:; CAT:NONE;
	SFS:(13230001)(366004)(8936002)(186003)(6512007)(2906002)(6506007)(8676002)(6916009)(31696002)(5660300002)(86362001)(6666004)(6486002)(66946007)(66556008)(38100700002)(2616005)(66476007)(316002)(31686004)(508600001)(36756003)(45980500001)(43740500002);
	DIR:OUT; SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount:	1
X-MS-Exchange-AntiSpam-MessageData-0:	=?utf-8?B?MDRFV0JtK1MzMnNpbDB6bzVPUmhObWFiZFYwdEV4UUQxeG5QT1gyM29ucS9n?=
	=?utf-8?B?M2l3UWJocVZMSVM0QWFEWERTbDA0SDAwL2ExaFdLWEFDOURtOFpudWVPRmR2?=
	=?utf-8?B?RHg0L3lQR2tLMG9nMVJzdFlueC84cmRySkQyUzE5Y2RCaE9Yc1FZUllldW1h?=
	=?utf-8?B?VmV4SkpMT0ZSd1lBeUt5bVhpM3FFdjFmR3dTMmtmdHRCaGVyNTJROTRNYWx1?=
	=?utf-8?B?VVJyUDBvYWw1azdwT3YwUDlXMEhLeXhPa09DcjlOWFg3aVlhNEhMNkR0VTNi?=
	=?utf-8?B?ejZWS1hDdWJ5dytSa3BPYUc1T3JZOVdUbDlrWHo0bEJtWUs2OTZnT1BwS2Zz?=
	=?utf-8?B?aklzdklBdHJzZ1d1U1Fubks4eEcwL3luNUg5KzdDZ0gxZEpMZ25GVWh5UU54?=
	=?utf-8?B?bEhLT011Q1F0Vkpvb3lsbWVNbDFIN2tRM3dDamwvcGg4RHNndEk1Mmx0U3hV?=
	=?utf-8?B?SnZTeDN2MzBtQ2xiWDFrb3B0K0JZRDhzdjlPZkhyNVpjYjZnMjJRcWlrT2JF?=
	=?utf-8?B?OUVqWlhCblJZSTlTNHhKaXNTdkhoNVh6dlVIN3pib0p5UkIwL3pQVXVTLyt4?=
	=?utf-8?B?RVpQZ3ZmZzhZQ1A3c3JRUHM2VWU0V3VzMUsxZjIvall6QWpKdnRSdHU3b0Yz?=
	=?utf-8?B?L1VpZEFoTDFXL1RKTE5tU003OGNZbzhEOHZ1bFFVdDN6K3RiTE52RkRjeG4w?=
	=?utf-8?B?KzlVK3pFWDNDM3ZmVk9uUWVDUS9PYStrQUpBTklBY0k0NFFnTHBuVWpzMWg1?=
	=?utf-8?B?TkVpaDFUdW1CL0xqN0czTXlyVGF2Tlc2R0t5bGpjMGxLKy84blFnMEhKVkwv?=
	=?utf-8?B?RUNmcW1BV2loeGxVOWV5MHE3Q0RUT1FXbGljWUxGTkdRNVVXaDM5R1ZzdlpK?=
	=?utf-8?B?NW9ha08yODNPaFhjbTgzNEMxKzhEbGgxK2NVK1pzR2h2TU5PMTlrVlRxRTJL?=
	=?utf-8?B?T2NDRVNNRTBUeTl4QkE1SmhQZkI0N0kxQnpqRWZoTmxQQlVTeFpnb08vMEUv?=
	=?utf-8?B?OU1nVUh4QUNUaTBVaTJyUVBuK0F5Qk1DOThzeGV5Y0dYcUFRaCtFK3E4YkJT?=
	=?utf-8?B?UEhMOFVpZHlmakdjVnUwNFVzMlNmZFUzTnc3bStGZHBGRGU3ZldwVWxtdUlY?=
	=?utf-8?B?ak9EeUlDV2NYWEhyWXNsd0dXYWpIblpySUNPQjVKZkg2MGh3VWw4ZGNSUzRk?=
	=?utf-8?B?S0ZNUzdJREVzK2hDODFNektxd2I4KzJZbGxJemxRUEtuQ292cjZHSWZBMmNL?=
	=?utf-8?B?VzBoNmpObno2dmVZd1Y0bTFuSFd2SWk2dFZrWGszVTdpTmJTTTBxY3orSVBK?=
	=?utf-8?B?TktiU0NJU0NxalIwSHErVlQzams2MW1BcDdTamR2a2hvLzArK1pVSWFqaUNB?=
	=?utf-8?B?QitjTGV6R0RjMmt5d1FFc29qRmlTdFo2aGhEWjBjcG9PY3dLbW9oRU0vV3pQ?=
	=?utf-8?B?OFRHWmdtUEt0WXhXblRJeGtlVERSU09QMUw5VGpkOENSZVg3ME04eXRsVDFK?=
	=?utf-8?B?NWc2cUN3eFJuYUtxdUxIM0NrbVllbkt6eU44SmlsZE5iWEFnU2xCTWY5UlNt?=
	=?utf-8?B?d1dGQTZrMHZKdkR0THVJNHhGd1V2UDljWlNtSU13SGNhZ3dPMk9VNjU0cEE0?=
	=?utf-8?B?VCtyS0FwYTBGYWhqTHhLbUFVZHlsbzBCMUpKUzJMTlFkK2U5S2lKUlQvU25t?=
	=?utf-8?B?eGpKZlVzRE9FUFdCTVBHalR1UWNpbkdlbmx4T21seFhuWHg5czJCNURwSFJD?=
	=?utf-8?B?Wlpxc28xVVB2ajQ3L3FYQXkrc3pzZVhNd0hPemk5NDhLSkQybEJubHhwZ1BB?=
	=?utf-8?B?QWxvdnVnV3ZzN3Q5YlpsemJTSDJPaEYyelpMR2srR1hiUFlDQ1NoZWd3WjB3?=
	=?utf-8?B?MC9lMVpGRUNNaElwWUw1YjhFT08vanlaUkhrb2hmM1NPV1B3cCtlTHdDTnZV?=
	=?utf-8?B?TE1ROUR0bEVzL1dBcngxMjVaOUJlUUJNWTZEYWNxeWQ2VVhGaURISUJtOGhj?=
	=?utf-8?B?VFlpdnZqWkcvRjY5QjNaWDZNeXUvSWpjc0FLek00ODRrZGZBL2hEc1Z4ZzEw?=
	=?utf-8?Q?irUGaY?=
X-OriginatorOrg:	oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id:	17c98f9b-61c5-45c0-db47-08da0b47058f
X-MS-Exchange-CrossTenant-AuthSource:	DM6PR10MB4089.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs:	Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime:	21 Mar 2022 14:28:09.2900 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader:	Hosted
X-MS-Exchange-CrossTenant-Id:	4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType:	HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName:	ZYZcqrKDDOSMEYUjapYMCSU7yILr3JksG+0rCpzfMXvUHRJgHj28Oo07XE0gefXjBVDVxss/ujdje68S3PZ8JjLAj0m7Oqwt36V7cuXyQws=
X-MS-Exchange-Transport-CrossTenantHeadersStamped:	BY5PR10MB4066
X-Proofpoint-Virus-Version:	vendor=nai engine=6300 definitions=10292
	signatures=694221
X-Proofpoint-Spam-Details:	rule=notspam policy=default score=0 adultscore=0
	malwarescore=0 mlxscore=0
	bulkscore=0 phishscore=0 spamscore=0 mlxlogscore=847 suspectscore=0
	classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2202240000
	definitions=main-2203210092
X-Proofpoint-GUID:	xuGw5MdM63XQO-_r88BvBeV3ZYEo2ngD
X-Proofpoint-ORIG-GUID:	xuGw5MdM63XQO-_r88BvBeV3ZYEo2ngD
X-Spam-Status:	No, score=-3.2 required=5.0 tests=BAYES_00, DKIM_SIGNED,
	DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, RCVD_IN_DNSWL_LOW, RCVD_IN_MSPIKE_H5,
	RCVD_IN_MSPIKE_WL, SPF_HELO_NONE, SPF_NONE, TXREP,
	T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.4
X-Spam-Checker-Version:	SpamAssassin 3.4.4 (2020-01-24) on
	server2.sourceware.org
X-BeenThere:	cygwin AT cygwin DOT com
X-Mailman-Version:	2.1.29
List-Id:	General Cygwin discussions and problem reports <cygwin.cygwin.com>
List-Archive:	<https://cygwin.com/pipermail/cygwin/>
List-Post:	<mailto:cygwin AT cygwin DOT com>
List-Help:	<mailto:cygwin-request AT cygwin DOT com?subject=help>
List-Subscribe:	<https://cygwin.com/mailman/listinfo/cygwin>,
	<mailto:cygwin-request AT cygwin DOT com?subject=subscribe>
Sender:	"Cygwin" <cygwin-bounces+archive-cygwin=delorie DOT com AT cygwin DOT com>

webmaster	delorie software privacy
Copyright � 2019 by DJ Delorie	Updated Jul 2019