X-Recipient: archive-cygwin AT delorie DOT com DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 65CE5385841E DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cygwin.com; s=default; t=1693246886; bh=LfYUGGxdxqUvqLyJ/57eorTcF/KL80DAPx0IHq0EZ58=; h=Date:Subject:To:References:In-Reply-To:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=ezMwquxu2jcgjlQi65tw6oBNXRVuMLoQvxLqV73zwko/+3w49oM3h6oT8xpO5zWAE n5ahHaTdv/PAZlP2ZbG63QGS2V9zEy6yQu/hWXA3PoWCYaUYhYm6UDkvEetU5rF+hP mGT167FTt3kTvHU6yB1MYG7dI5SYw3lGurStVYuk= X-Original-To: cygwin AT cygwin DOT com Delivered-To: cygwin AT cygwin DOT com DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org F305A3858D28 X-Authority-Analysis: v=2.4 cv=VbHkgXl9 c=1 sm=1 tr=0 ts=64ece57c a=DxHlV3/gbUaP7LOF0QAmaA==:117 a=DxHlV3/gbUaP7LOF0QAmaA==:17 a=IkcTkHD0fZMA:10 a=w_pzkKWiAAAA:8 a=30KeihfUasEDSKDPGhkA:9 a=QEXdDO2ut3YA:10 a=sRI3_1zDfAgwuvI8zelB:22 Message-ID: <50471900-708d-2e92-bd90-aaacfb64873a@Shaw.ca> Date: Mon, 28 Aug 2023 12:20:43 -0600 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101 Thunderbird/102.14.0 Subject: Re: gawk core dumped on too many input values Content-Language: en-CA To: cygwin AT cygwin DOT com References: <0b55205e-fc7c-98cf-c186-783e5c0655b0 AT comcast DOT net>

Organization: Inglis In-Reply-To: X-CMAE-Envelope: MS4xfCl9dmoJh1JDz42VeQWHhEweVGUfybE4WcWy8pw4Zr477JcvtfHVGU4yd3tQOuaK2c4a92Gkw/JzBqT4Va86/AJyKowWpMrdx9G8zXmYVtzVijFu+Ky3 ZbWEti2mdAgwvzj07F4MIxZb6rjRb5ql3Hv7d+db9TtDVz7GbyE0NdDQwHnPXZy1LDf6C4/9rlgIlTrz5yenwhtdEeQzC79IzkY= X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, NICE_REPLY_A, RCVD_IN_DNSWL_LOW, RCVD_IN_MSPIKE_H5, RCVD_IN_MSPIKE_WL, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: cygwin AT cygwin DOT com X-Mailman-Version: 2.1.30 Precedence: list List-Id: General Cygwin discussions and problem reports List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Brian Inglis via Cygwin Reply-To: cygwin AT cygwin DOT com Cc: Brian Inglis Content-Type: text/plain; charset="utf-8"; Format="flowed" Errors-To: cygwin-bounces+archive-cygwin=delorie DOT com AT cygwin DOT com Sender: "Cygwin" Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from base64 to 8bit by delorie.com id 37SILR7q029757 On 2023-08-28 05:47, Joshuah Hurst via Cygwin wrote: > On Mon, Aug 28, 2023 at 1:08 AM Jeremy Hetzler via Cygwin > wrote: >> >> On Sun, Aug 27, 2023 at 2:25 PM Ed Morton via Cygwin >> wrote: >>> >>> This (original email below) turned out to be a general cygwin issue, not >>> a gawk issue: >>> >>> $ LC_ALL=C sed 's/x/y/' $(seq 1000000) >>> Segmentation fault (core dumped) >>> >>> $ LC_ALL=C grep 'foo' $(seq 1000000) >>> Segmentation fault (core dumped) >>> >> >> Seems that all commands linked with cygwin1.dll will fault if you pass them >> a long enough arglist. >> >> For me, /bin/true faults on {1..258231} but not {1..258230}. >> >>> $ /bin/true {1..258230} >>> >> >> >>> $ /bin/true {1..258231} >>> Segmentation fault (core dumped) >> >> >> strace, which is not linked with cygwin1.dll, exits cleanly. >> >>> $ /bin/strace {1..300000} >>> -bash: /bin/strace: Argument list too long >> >> >> See this page [1] on maximum argument lengths. >> >> It would be nice to document this limit, whatever it is. > > Is this limit? > > $ getconf -a | grep -E 'ARG_MAX' > _POSIX_ARG_MAX 4096 > ARG_MAX 32000 On my system, /bin/true and /bin/echo get to between 250K and 256K before seg faulting, requiring between digits + separator + ptr (10+8)*(250-256)KB == 3750-3840KB; current Cygwin and `bash` `:` and `echo` internal commands get to between 128M and 256M before `bash` fork fails, and Windows gets unhappy with memory full, paging high, and page thrashing on both paging devices: calmed eventually with a few interrupts then `exec bash`; requiring between digits + separator + ptr (10+8)*(128-256)MB == 2.25-4.5GB; This seems to be extremely conservative: $ xargs -r --show-limits <<< ' ' Your environment variables take up 9246 bytes POSIX upper limit on argument length (this system): 20706 POSIX smallest allowable upper limit on argument length (all systems): 4096 Maximum length of command we could actually use: 11460 Size of command buffer we are actually using: 20706 Maximum parallelism (--max-procs must be no greater): 2147483647 $ seq $((256*1024)) | xargs | wc -lwcL 102 262144 1723903 18094 Once a patch is in a test version of Cygwin, I can rebuild and release a test version of findutils, which includes xargs, with updated results in the test release announcement. -- Take care. Thanks, Brian Inglis Calgary, Alberta, Canada La perfection est atteinte Perfection is achieved non pas lorsqu'il n'y a plus rien à ajouter not when there is no more to add mais lorsqu'il n'y a plus rien à retirer but when there is no more to cut -- Antoine de Saint-Exupéry -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation: https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple