delorie.com/archives/browse.cgi   search  
Mail Archives: cygwin/2026/10/02/22:27:02

DMARC-Filter: OpenDMARC Filter v1.4.2 delorie.com 6932R1LG3458523
Authentication-Results: delorie.com; dmarc=pass (p=none dis=none) header.from=cygwin.com
Authentication-Results: delorie.com; spf=pass smtp.mailfrom=cygwin.com
DKIM-Filter: OpenDKIM Filter v2.11.0 delorie.com 6932R1LG3458523
Authentication-Results: delorie.com;
dkim=pass (1024-bit key, unprotected) header.d=cygwin.com header.i=@cygwin.com header.a=rsa-sha256 header.s=default header.b=gYH8X9Vs
X-Recipient: archive-cygwin AT delorie DOT com
DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org E25D64BAE7E9
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cygwin.com;
s=default; t=1790994403;
bh=B7IdPDBGRCiNj8NyCg8t3vN/TnsXMuRcDC/IL+uxRj0=;
h=To:References:In-Reply-To:Subject:Date:List-Id:List-Unsubscribe:
List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc:
From;
b=gYH8X9VsMEn6G5K+PWIHfkSdJh+OYUo3cnTrYOwdagRcV5VrgpMHlYHuHKQ2MjAFx
wGKsexVA0oY4oYVOgmmcGypTzYq4WQL6Ek3D0NSXGzXUn5NobIXyFQQLDnUoRmQhdu
8TUJx3KlAHxOD73/4BGG4PV3VozsJsTYrs7X4NG8=
X-Original-To: cygwin AT cygwin DOT com
Delivered-To: cygwin AT cygwin DOT com
DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 5F3174BA903C
ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 5F3174BA903C
ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1790994353; cv=none;
b=ZtREztDNpcRRBSoLopGNO7n/is7A6vOMLyXI5A6jDj3dYaYnY0Py+PJZ7HtntjzIh+su3NNZxWxB6njh1OSwhuDiLdShjCrSKOx1Z0wN3+xeY+KSTsVcOK8NKRaltsOa3pMFiBxT+6Sag8BSp7bUJB7JlOSbdId+W1FsKbCJADE=
ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key;
t=1790994353; c=relaxed/simple;
bh=B7XuYyxuO3DNI30BvjO6OiAe8w9Lp6BNuGY8GzpQVy0=;
h=From:To:Subject:Date:Message-ID:MIME-Version;
b=Jvn5eXgyenASdCikuRU1os0A47LpH7d34vCMdP7QAqgX5B70FkakE5IyQ8PhfwzIsEo3Wc/gqtPuvTifdqZTCwsM6ltjiS3HmDKicNHJ6kV1lrnbmCD4NOyX+ubG5f+OXUUvB0eeEr9Q9kOD0YxPgeVIQjkSfl0HL4SGlBG7qZQ=
ARC-Authentication-Results: i=1; sourceware.org
DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 5F3174BA903C
To: <Brian DOT Inglis AT SystematicSW DOT ab DOT ca>
References: <000001dd33b2$6a4f2570$3eed7050$@nickpopoff.net>
<20260824194051 DOT e61e716cbdbe3a1c246fa51d AT nifty DOT ne DOT jp>
In-Reply-To: <20260824194051.e61e716cbdbe3a1c246fa51d@nifty.ne.jp>
Subject: =?utf-8?Q?RE:_Regression_in_Cygwin_3.6.9:_?=
=?utf-8?Q?Deadlock_when_spawning_many_conc?=
=?utf-8?Q?urrent_Windows-native_processes_?=
=?utf-8?Q?=28nmake.exe=29_=E2=80=94_works_in_3.6.5_1?=
Date: Fri, 2 Oct 2026 22:25:51 -0400
Message-ID: <009401dd52de$82792e20$876b8a60$@nickpopoff.net>
MIME-Version: 1.0
X-Mailer: Microsoft Outlook 16.0
Thread-Index: AQISnvyw/+o0LnckBBDy9covbFvZ3gIB2zR+AlkR9pc=
X-BeenThere: cygwin AT cygwin DOT com
X-Mailman-Version: 2.1.30
List-Id: General Cygwin discussions and problem reports <cygwin.cygwin.com>
List-Archive: <https://cygwin.com/pipermail/cygwin/>
List-Post: <mailto:cygwin AT cygwin DOT com>
List-Help: <mailto:cygwin-request AT cygwin DOT com?subject=help>
List-Subscribe: <https://cygwin.com/mailman/listinfo/cygwin>,
<mailto:cygwin-request AT cygwin DOT com?subject=subscribe>
From: Nick Popoff via Cygwin <cygwin AT cygwin DOT com>
Reply-To: Nick Popoff <nick AT nickpopoff DOT net>
Cc: cygwin AT cygwin DOT com
Sender: "Cygwin" <cygwin-bounces~archive-cygwin=delorie DOT com AT cygwin DOT com>
X-MIME-Autoconverted: from base64 to 8bit by delorie.com id 6932R1LG3458523

Not sure what you mean by "massive graphic blob".  What blob?  I reported a comprehensive bug report in great detail.  Even, illustrated the problem with real bash code (see below).  Here is even a  more simple illustration:

Steps to Reproduce
1.	Install Cygwin 3.6.9
2.	Launch a Cygwin shell
3.	Start 15 parallel Windows-native builds, e.g.:
Code
for i in {1..15}; do
      (nmake.exe -f Makefile$i &)
done
wait

What can be a more simple illustration.  Each makefile above runs hundreds ASN.1 regression tests.  They started concurrently and after a certain number of nmake.exe processes running, everything stops, no movement.  Cannot be killed with Ctrl-C.  Only completely closing the Cygwin console kills the processes.  This problem was introduced recently.  There was no problem in Cygwin 3.6.5.

Nick

===============



Hi Nick,

Attaching a massive graphic blob with little data does not really highlight 
where the issue is.
Perhaps try a simple Cygwin command line (with procps-ng installed):

	$ procps -Aeflw --forest | tee issue.log

and feel free to trim the output options, and edit the content to highlight 
relevant processes where the issues exist.


FYI: This is the actual script sample that 3.6.9.-1 cannot handle properly:

.........
	    if [[ $case == all || ( -a "$tmp/$daemon$i.lst" && -s "$tmp/$daemon$i.lst" ) ]]; then
		if [[ "$1" == -test ]]; then
		    [[ -n $ra || -n $bc ]] && Wait
		    {
			time nmake -a -nologo -k -f $mutil/$daemon$i.mak \
			    makefile=$daemon$i update=no case=$case msdebug=$msdebug dir=. nrt=$nrt $dllver
		    } >_test$i.log 2>&1 &
		    found=yes
		elif [[ "$1" == -update ]]; then
		    [[ -n $ra || -n $bc ]] && Wait
		    {
			time nmake -a -nologo -k -f $mutil/$daemon$i.mak \
			    update=yes case=$case dir=$dir makefile=$daemon$i msdebug=$msdebug nrt=$nrt $dllver
		    } >_test$i.log 2>&1 &
		    found=yes
		fi
	    else
		rm -f $TMP/$daemon$i.lst
	    fi
	    i=$((1+$i))
	done

	if [[ $found == no && $difs == yes ]]; then
	    echo "$DAEMON: $(date "$format"): ERROR: Error in makefiles.  No tests found."
	    return 203
	elif [[ $difs == no ]]; then
	    echo "$DAEMON: $(date "$format"): WARNING: No tests found to update."
	    return 0
	fi

	wait
.........

Where Wait() is a throttling function that allows to run concurrently a given number (like 15 in this case) of 'nmake' processes, and the low case 'wait' is the bash wait.

Nick



> -----Original Message-----
> From: Takashi Yano <takashi DOT yano AT nifty DOT ne DOT jp>
> Sent: Monday, August 24, 2026 6:41 AM
> To: cygwin AT cygwin DOT com
> Cc: Nick Popoff <nick AT nickpopoff DOT net>
> Subject: Re: Regression in Cygwin 3.6.9: Deadlock when spawning many
> concurrent Windows-native processes (nmake.exe) — works in 3.6.5 1
> 
> Thanks for the report.
> 
> I guess the patch:
> https://cygwin.com/pipermail/cygwin-patches/2026q3/015286.html
> fixes the issue. Please wait for test version of cygwin, where the patch
> applied. The patch is now under review, so hopefully it will become soon.
> 
> On Mon, 24 Aug 2026 06:22:06 -0400
> Nick Popoff via Cygwin <cygwin AT cygwin DOT com> wrote:
> 
> > Hello,
> >
> >
> >
> > Regression in Cygwin 3.6.9: Deadlock when spawning many concurrent
> > Windows-native processes (nmake.exe) — works in 3.6.5‑1
> >
> > Version affected: Cygwin 3.6.9 (latest as of August 2026)
> >
> > Version NOT affected: Cygwin 3.6.5‑1 (stable, no issues)
> >
> >
> >
> > Summary
> >
> > Cygwin 3.6.9 introduces a regression in process management that causes
> complete build deadlock when spawning many concurrent Windows-native
> processes (specifically nmake.exe). The same workload has run flawlessly for
> 20+ years on all previous Cygwin versions, including 3.6.5‑1. Under 3.6.9, the
> build freezes once some parallel job limit is reached, leaving dozens of idle
> sh.exe processes and stalled nmake.exe processes. Ctrl‑C kills the Windows
> nmake.exe processes but Cygwin’s sh.exe instances remain stuck, and the
> terminal never returns control.  Reverting to 3.6.5‑1 immediately restores
> correct behavior.  This is a reproducible regression in Cygwin’s
> process/signal/pipe handling.
> >
> >
> >
> > Environment
> >
> > *	Windows Server 2025 (WS2025)
> > *	Heavy parallel build workload
> > *	15 concurrent nmake.exe processes
> > *	Each nmake.exe spawns many short-lived Windows-native tools
> > *	Cygwin shell (sh.exe) orchestrates the build
> > *	No Cygwin-native processes involved — all Windows-native
> >
> >
> >
> > Steps to Reproduce
> >
> > 1.	Install Cygwin 3.6.9
> > 2.	Launch a Cygwin shell
> > 3.	Start 15 parallel Windows-native builds, e.g.:
> >
> > Code
> >
> > for i in {1..15}; do
> >
> >     (nmake.exe -f Makefile$i &)
> >
> > done
> >
> > wait
> >
> > Each makefile repeatedly spawns short-lived Windows-native tools (ASN.1-
> compiler processes).
> >
> > 4.	Observe behavior once all 15 jobs are active.
> >
> >
> >
> > Actual Behavior (3.6.9)
> >
> > *	Build freezes completely once a certain parallel limit is reached
> > *	All nmake.exe processes become idle
> > *	Dozens of idle sh.exe accumulate
> > *	No CPU usage
> > *	No progress
> > *	Ctrl‑C kills nmake.exe but does not kill sh.exe
> > *	Cygwin console never returns
> > *	Terminal must be forcibly closed
> > *	No such behavior in 3.6.5‑1
> >
> > This is consistent with a signal-thread or pipe inheritance deadlock.
> >
> >
> >
> > Expected Behavior (3.6.5‑1 and earlier)
> >
> > *	All 15 parallel nmake.exe processes run normally
> > *	No idle sh.exe accumulation
> > *	No deadlock
> > *	Ctrl‑C cleanly terminates all processes
> > *	Console returns immediately
> > *	Build completes successfully
> >
> > This behavior has been stable for decades.
> >
> >
> >
> > Notes
> >
> > This workload is a strong stress test for:
> >
> > *	spawn.cc
> > *	sigproc.cc
> > *	pipe inheritance
> > *	process table updates
> > *	console control handling
> > *	synchronization primitives
> >
> > The regression appears only in 3.6.9.
> >
> >
> >
> > Conclusion
> >
> > Cygwin 3.6.9 introduces a reproducible deadlock when orchestrating many
> concurrent Windows-native processes via Cygwin’s shell. This is a regression
> from 3.6.5‑1. The issue affects real-world build systems and should be
> investigated.
> >
> > I can provide:
> >
> > *	process snapshots
> > *	strace logs
> > *	Windows process tree dumps
> > *	minimal reproduction scripts
> >
> > upon request.
> >
> >
> >
> > Nick
> >
> >
> >
> >
> >
> >
> >
> >
> > --
> > Problem reports:      https://cygwin.com/problems.html
> > FAQ:                  https://cygwin.com/faq/
> > Documentation:        https://cygwin.com/docs.html
> > Unsubscribe info:     https://cygwin.com/ml/#unsubscribe-simple
> 
> 
> --
> Takashi Yano <takashi DOT yano AT nifty DOT ne DOT jp>


-- 
Problem reports:      https://cygwin.com/problems.html
FAQ:                  https://cygwin.com/faq/
Documentation:        https://cygwin.com/docs.html
Unsubscribe info:     https://cygwin.com/ml/#unsubscribe-simple

- Raw text -


  webmaster     delorie software   privacy  
  Copyright © 2026   by DJ Delorie     Updated Sep 20 2026