DKIM-Filter: OpenDKIM Filter v2.11.0 delorie.com 45IMNJOJ2961994 Authentication-Results: delorie.com; dkim=pass (1024-bit key, unprotected) header.d=cygwin.com header.i=@cygwin.com header.a=rsa-sha256 header.s=default header.b=iwkBiCN4 X-Recipient: archive-cygwin AT delorie DOT com DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org D91743883038 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cygwin.com; s=default; t=1718749398; bh=LNTk9F61G4kmlzKzmP2IWVVm1uDnAfSAcpIs20lY4x8=; h=Subject:In-Reply-To:Date:Cc:References:To:List-Id: List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe: From:Reply-To:From; b=iwkBiCN4K7Ffy8cob5ByZQS6qouCYN05lXWTJfVGobXZw9GzuHCiZ09HA+zu9ArVE p6sCeEvAQxYYugmIfbLmgeojjK8ija7YuXyQAu71QuaGAnrtt+iE/dXiS5SYblF5hf a1YFcgYOkaubm1dta3hJFNr9MdBBqo2F0CYaX+70= X-Original-To: cygwin AT cygwin DOT com Delivered-To: cygwin AT cygwin DOT com DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 884223883010 ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 884223883010 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1718749345; cv=none; b=uP5UMqdKcAxO+fB/iluwSOZ5CEvEncNB+MifPwcvOrnjFtKr6eReKBtVo9kxuJtKLJSUaWrV2yUgl++vXxXTjHERo3GX7hL9dL7JiodGuVDeuCvxaIrqbMB7GayxbjwCqfMSnaqgaTI7AqVEHhmvwk6fHnMLOrYvLMM5wFMs21U= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1718749345; c=relaxed/simple; bh=PXVFsqsBLRp0fLBeVdmYQ20oXDvpCuiDMuqgiIhQskQ=; h=DKIM-Signature:Mime-Version:Subject:From:Date:Message-Id:To; b=TrOzxa/8xDkwHBH2JisI5fEYo6I8akOpZgr6KWAlT2PQ+qinPxup08JK7DwbfJA589AXAkrQRTl4VLK3nLaUp5wo4yE1IVSEnPXh/dAjvMas7GVv0BwWj2pIi8QntiKYmRBjrgbMDxHNpH7Wk0uNYrEY25By0SBKKapw3lW3QL8= ARC-Authentication-Results: i=1; server2.sourceware.org X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718749342; x=1719354142; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=taW5z/7GKqTtz2W8ZHEzLuSDgS5VFebjbXjGQhDB1ZI=; b=VCkHtHivCH0diZVVAXq1Ema5l0oknP8cdmUQhGeDuTlRqdsT/NxVorTKu+SP3BFJwQ rqrziOSW+sI6sPJZ/824a8oHEMECRm/0I7IjDlz7Od64qG54LmZTwdWy5S5WnbR1Z6h8 MsDJt0oDB2r+CD1WoEcBEmIO7/1zilD6F5hCnx2hWox8jegOztlTcFP1GE0Cm3mUHMjK 5HSjMhop6J4wiI6WngcbVwCjP1UXrNAU0fCSidm9XYe+5J1Oe98LEdFJXbTT9oT4G9Km 4+fhPt+Q4mk5IfmpdNAtvm6yZCBMdJvVw7cngPWGT/ywoNXh0+bZeG89uuNEX4FRXz5/ lCUw== X-Gm-Message-State: AOJu0YxA32dQgK5AcCdsS3T+4sEooop3ADfZhRlIS9KtTahiIXe2lCGJ dnd50oFO318s9+t42FWgqQBwcUTJ9h5LLqzz/pxKVrSM0BTq6/pbodeU38BD3bk4bFGdm7MydN2 Mxzk= X-Google-Smtp-Source: AGHT+IEZsPA+K1cVRJYuQG4509QZqc29kCJj2k5VEFzDXJ8T0xB7JnMp7TwF1jEoeQVVvNhnFpIlUA== X-Received: by 2002:a81:be11:0:b0:627:956b:dbca with SMTP id 00721157ae682-63a8f338556mr12399437b3.40.1718749341811; Tue, 18 Jun 2024 15:22:21 -0700 (PDT) Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.500.171.1.1\)) Subject: Re: Cygwin outputting message to stderr on dofork EAGAIN failure even when Python exception is caught and handled In-Reply-To: Date: Tue, 18 Jun 2024 17:22:10 -0500 Cc: "cygwin AT cygwin DOT com" Message-Id: <244BB875-5177-402C-B958-FCCB23FEAEEE@zaxiom.com> References: <1399464798 DOT 20240617105808 AT yandex DOT ru> To: Dan Shelton X-Mailer: Apple Mail (2.3774.500.171.1.1) X-Spam-Status: No, score=-1.7 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_NONE, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: cygwin AT cygwin DOT com X-Mailman-Version: 2.1.30 Precedence: list List-Id: General Cygwin discussions and problem reports List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Nicholas Williams via Cygwin Reply-To: Nicholas Williams Content-Type: text/plain; charset="utf-8" Errors-To: cygwin-bounces+archive-cygwin=delorie DOT com AT cygwin DOT com Sender: "Cygwin" Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from base64 to 8bit by delorie.com id 45IMNJOJ2961994 Dan, > On Jun 18, 2024, at 09:44, Dan Shelton via Cygwin wrote: > > On Mon, 17 Jun 2024 at 18:03, Dale Lobb (Sys Admin) via Cygwin > wrote: >> >> Greetings, Nicholas; >> >>> From: Cygwin On Behalf Of Andrey Repin via Cygwin >>> Sent: Monday, June 17, 2024 2:58 AM >>> To: Nicholas Williams ; cygwin AT cygwin DOT com >>> Cc: Andrey Repin >>> Subject: EXTERNAL SENDER: Re: Cygwin outputting message to stderr on dofork EAGAIN failure even when Python exception is caught and handled >>> >>> Greetings, Nicholas Williams! > We have a Python (installed and run through Cygwin) process running on > Windows Server 2022 that was very, very occasionally failing when subprocess. check_output was called: > 0 [main] python3 28481 >>> >>> Greetings, Nicholas Williams! >>> >>>> We have a Python (installed and run through Cygwin) process running on >>>> Windows Server 2022 that was very, very occasionally failing when subprocess.check_output was called: >>> >>>> 0 [main] python3 28481 dofork: child -1 - forked process 16856 died >>>> unexpectedly, retry 0, exit code 0xC0000142, errno 11 >>>> … >>>> subprocess.check_output(["cygpath", "-w", directory.name], encoding="utf-8").strip() >>>> File "/usr/lib/python3.9/subprocess.py", line 424, in check_output <> >>>> <>return run(*popenargs, stdout=PIPE, timeout=timeout, check=True, <> >>>> File "/usr/lib/python3.9/subprocess.py", line 505, in run <> >>>> <>with Popen(*popenargs, **kwargs) as process: <> >>>> File "/usr/lib/python3.9/subprocess.py", line 951, in __init__ <> >>>> <>self._execute_child(args, executable, preexec_fn, close_fds, <> >>>> File "/usr/lib/python3.9/subprocess.py", line 1754, in _execute_child >>>> self.pid = _posixsubprocess.fork_exec( >>>> BlockingIOError: [Errno 11] Resource temporarily unavailable >>> >>>> Setting aside for a minute the various reasons this might be happening >>>> occasionally, which we cannot solve for at this moment, the error number >>>> (EAGAIN) indicates that you should “try again.” So that’s exactly what we >>>> did. We added a try/catch to the Python code to catch the BlockingIOError >>>> and, if and only if the error number is EAGAIN, we try up to two more times. >>>> This fixed the problem and caused the application to stop quitting. We >>>> output a warning to our log so that we don’t forget about the problem, but >>>> the warning only ever appears once, so retrying a single time seems to help. >>> >>>> However … even though Python handles the dofork error, turns it into a >>>> Python exception, and our code catches the Python exception and handles it >>>> properly, Cygwin (not Python … Cygwin) still outputs a message to stderr >>>> right before our warning message. This Cygwin error message shows up as an error in our log tracking: >>> >>>> 0 [main] python3 15042 dofork: child -1 - forked process 6780 died >>>> unexpectedly, retry 0, exit code 0xC0000142, errno 11 >>>> 06/16 13:57:53. 87520: WARNING: Retrying command in 2 seconds due to EAGAIN: [the command we’re running] > > Does python require fork(), or can it use posix_spawn() in this case? I imagine this would be question for the CPython development team. While I have a basic understanding of C programming in Python (for the purposes of writing extensions), I do not know why subprocess.Popen is written in this way, and I do not know enough to question their choice here. It uses fork, and for now I have to live with that. Thanks, Nick -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation: https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple