X-Recipient:	archive-cygwin AT delorie DOT com
DKIM-Filter:	OpenDKIM Filter v2.11.0 sourceware.org 65CE5385841E
DKIM-Signature:	v=1; a=rsa-sha256; c=relaxed/relaxed; d=cygwin.com;
	s=default; t=1693246886;
	bh=LfYUGGxdxqUvqLyJ/57eorTcF/KL80DAPx0IHq0EZ58=;
	h=Date:Subject:To:References:In-Reply-To:List-Id:List-Unsubscribe:
	List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc:
	From;
	b=ezMwquxu2jcgjlQi65tw6oBNXRVuMLoQvxLqV73zwko/+3w49oM3h6oT8xpO5zWAE
	n5ahHaTdv/PAZlP2ZbG63QGS2V9zEy6yQu/hWXA3PoWCYaUYhYm6UDkvEetU5rF+hP
	mGT167FTt3kTvHU6yB1MYG7dI5SYw3lGurStVYuk=
X-Original-To:	cygwin AT cygwin DOT com
Delivered-To:	cygwin AT cygwin DOT com
DMARC-Filter:	OpenDMARC Filter v1.4.2 sourceware.org F305A3858D28
X-Authority-Analysis:	v=2.4 cv=VbHkgXl9 c=1 sm=1 tr=0 ts=64ece57c
	a=DxHlV3/gbUaP7LOF0QAmaA==:117 a=DxHlV3/gbUaP7LOF0QAmaA==:17
	a=IkcTkHD0fZMA:10 a=w_pzkKWiAAAA:8 a=30KeihfUasEDSKDPGhkA:9 a=QEXdDO2ut3YA:10
	a=sRI3_1zDfAgwuvI8zelB:22
Message-ID:	<50471900-708d-2e92-bd90-aaacfb64873a@Shaw.ca>
Date:	Mon, 28 Aug 2023 12:20:43 -0600
MIME-Version:	1.0
User-Agent:	Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101
	Thunderbird/102.14.0
Subject:	Re: gawk core dumped on too many input values
To:	cygwin AT cygwin DOT com
References:	<e0fa509d-890a-2db7-c91f-2b1d904a9a1e AT comcast DOT net>
	<0b55205e-fc7c-98cf-c186-783e5c0655b0 AT comcast DOT net>
	<CAOh4nm=yYKtBWhWOwW9rUxu3y8vFOgRA+rW-tTzq-=q63OL_0A AT mail DOT gmail DOT com>
	<CAEEMktOg5iA-6Q+14ZqgKf_DpjRkjmAyjsuJFDxWie_sjBJLQQ AT mail DOT gmail DOT com>
Organization:	Inglis
In-Reply-To:	<CAEEMktOg5iA-6Q+14ZqgKf_DpjRkjmAyjsuJFDxWie_sjBJLQQ@mail.gmail.com>
X-CMAE-Envelope:	MS4xfCl9dmoJh1JDz42VeQWHhEweVGUfybE4WcWy8pw4Zr477JcvtfHVGU4yd3tQOuaK2c4a92Gkw/JzBqT4Va86/AJyKowWpMrdx9G8zXmYVtzVijFu+Ky3
	ZbWEti2mdAgwvzj07F4MIxZb6rjRb5ql3Hv7d+db9TtDVz7GbyE0NdDQwHnPXZy1LDf6C4/9rlgIlTrz5yenwhtdEeQzC79IzkY=
X-Spam-Status:	No, score=-4.4 required=5.0 tests=BAYES_00, DKIM_SIGNED,
	DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, NICE_REPLY_A, RCVD_IN_DNSWL_LOW,
	RCVD_IN_MSPIKE_H5, RCVD_IN_MSPIKE_WL, SPF_HELO_NONE, SPF_PASS,
	TXREP autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version:	SpamAssassin 3.4.6 (2021-04-09) on
	server2.sourceware.org
X-BeenThere:	cygwin AT cygwin DOT com
X-Mailman-Version:	2.1.30
List-Id:	General Cygwin discussions and problem reports <cygwin.cygwin.com>
List-Unsubscribe:	<https://cygwin.com/mailman/options/cygwin>,
	<mailto:cygwin-request AT cygwin DOT com?subject=unsubscribe>
List-Archive:	<https://cygwin.com/pipermail/cygwin/>
List-Post:	<mailto:cygwin AT cygwin DOT com>
List-Help:	<mailto:cygwin-request AT cygwin DOT com?subject=help>
List-Subscribe:	<https://cygwin.com/mailman/listinfo/cygwin>,
	<mailto:cygwin-request AT cygwin DOT com?subject=subscribe>
From:	Brian Inglis via Cygwin <cygwin AT cygwin DOT com>
Reply-To:	cygwin AT cygwin DOT com
Cc:	Brian Inglis <Brian DOT Inglis AT Shaw DOT ca>
Errors-To:	cygwin-bounces+archive-cygwin=delorie DOT com AT cygwin DOT com
Sender:	"Cygwin" <cygwin-bounces+archive-cygwin=delorie DOT com AT cygwin DOT com>
X-MIME-Autoconverted:	from base64 to 8bit by delorie.com id 37SILR7q029757

On 2023-08-28 05:47, Joshuah Hurst via Cygwin wrote:
> On Mon, Aug 28, 2023 at 1:08 AM Jeremy Hetzler via Cygwin
> <cygwin AT cygwin DOT com> wrote:
>>
>> On Sun, Aug 27, 2023 at 2:25 PM Ed Morton via Cygwin <cygwin AT cygwin DOT com>
>> wrote:
>>>
>>> This (original email below) turned out to be a general cygwin issue, not
>>> a gawk issue:
>>>
>>> $ LC_ALL=C sed 's/x/y/' $(seq 1000000)
>>> Segmentation fault (core dumped)
>>>
>>> $ LC_ALL=C grep 'foo' $(seq 1000000)
>>> Segmentation fault (core dumped)
>>>
>>
>> Seems that all commands linked with cygwin1.dll will fault if you pass them
>> a long enough arglist.
>>
>> For me, /bin/true faults on {1..258231} but not {1..258230}.
>>
>>> $ /bin/true {1..258230}
>>>
>>
>>
>>> $ /bin/true {1..258231}
>>> Segmentation fault (core dumped)
>>
>>
>> strace, which is not linked with cygwin1.dll, exits cleanly.
>>
>>> $ /bin/strace {1..300000}
>>> -bash: /bin/strace: Argument list too long
>>
>>
>> See this page [1] on maximum argument lengths.
>>
>> It would be nice to document this limit, whatever it is.
> 
> Is this limit?
> 
> $ getconf -a | grep -E 'ARG_MAX'
> _POSIX_ARG_MAX                      4096
> ARG_MAX                             32000

On my system, /bin/true and /bin/echo get to between 250K and 256K before seg 
faulting, requiring between digits + separator + ptr (10+8)*(250-256)KB == 
3750-3840KB;

current Cygwin and `bash` `:` and `echo` internal commands get to between 128M 
and 256M before `bash` fork fails, and Windows gets unhappy with memory full, 
paging high, and page thrashing on both paging devices: calmed eventually with a 
few interrupts then `exec bash`;
requiring between digits + separator + ptr (10+8)*(128-256)MB == 2.25-4.5GB;

This seems to be extremely conservative:

$ xargs -r --show-limits <<< ' '
Your environment variables take up 9246 bytes
POSIX upper limit on argument length (this system): 20706
POSIX smallest allowable upper limit on argument length (all systems): 4096
Maximum length of command we could actually use: 11460
Size of command buffer we are actually using: 20706
Maximum parallelism (--max-procs must be no greater): 2147483647
$ seq $((256*1024)) | xargs | wc -lwcL
     102  262144 1723903   18094

Once a patch is in a test version of Cygwin, I can rebuild and release a test 
version of findutils, which includes xargs, with updated results in the test 
release announcement.

-- 
Take care. Thanks, Brian Inglis              Calgary, Alberta, Canada

La perfection est atteinte                   Perfection is achieved
non pas lorsqu'il n'y a plus rien à ajouter  not when there is no more to add
mais lorsqu'il n'y a plus rien à retirer     but when there is no more to cut
                                 -- Antoine de Saint-Exupéry

-- 
Problem reports:      https://cygwin.com/problems.html
FAQ:                  https://cygwin.com/faq/
Documentation:        https://cygwin.com/docs.html
Unsubscribe info:     https://cygwin.com/ml/#unsubscribe-simple

- Raw text -