X-Recipient:	archive-cygwin AT delorie DOT com
DKIM-Filter:	OpenDKIM Filter v2.11.0 sourceware.org 7081A388A009
DKIM-Signature:	v=1; a=rsa-sha256; c=relaxed/relaxed; d=cygwin.com;
	s=default; t=1585831632;
	bh=BfVpadlik202xC1OgJihBI9wC5Re8H6PL17gX6ZlPak=;
	h=Date:To:In-Reply-To:References:Subject:List-Id:List-Unsubscribe:
	List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc:
	From;
	b=kg1NVBnMlDQsjChqhfLQ0BI1MwPNErqu46aLZTprilHkjxny4PFzeoa+JAY/pUIV3
	g6OI+bBppPhQZTl4V1g3GhC9s8QgFm5q1JF/NdnbOUmTPCC0BsQ6sbwhwX4WYdJ1Ww
	Eigcs36TJYxCEGqShet9MiIkrzTLIShSzGh3eO+E=
X-Original-To:	cygwin AT cygwin DOT com
Delivered-To:	cygwin AT cygwin DOT com
DMARC-Filter:	OpenDMARC Filter v1.3.2 sourceware.org B659D385E02A
X-YMail-OSG:	Szd3kzkVM1lsQlVVpgAM3G2.l1K0_suU4h7Q0pKe7TvCOT1BxJoppWTd7TYTK6h
	_S_dpnHGjVBe4FAV0E04aQHkOoGWA5JUN78fc9E8v9w4gq5_ft8wdsB1rZxRHPQGkq_nj4ABWvUE
	NETxLUjfXuyDXXaswMFf_jWPoiHwFsbHTKIrp6LwhuWCjOlF9G1vd.i_3t1tCdxKgKeaux2tfJjI
	93gixioPx2sv5YGt1aZZdOY00qX_koYlLRNWGaMKMQ1wWdXCvdQb_5sG1MrlCU_IoHxDJIyMwf5d
	dUo3EC24QjzYH.qYKh_AjNjq39dY0mjva97k2BttcSlqHvC9pHyeeF8FS__GIhh_D9.QVthcLL2K
	BYeHWCnnmxELhjgd50_6I1V.4esgd.ySNy4h3KYAbAuaXPZAAw5Yjo4FY1FHnivNQdRn6XZDtxI0
	HTkWeUFLwygvM0Xo4LP1Kp8.BNpQ_ksnhpTsA_Vefe9ajCIReDfg286w3UQjuU0SX.qfLaNyhJmT
	XuohgbY_QlTf9B2qmPq5_aPs9bP5LK7sJ1zPPRPHBD_lW1tkmctpx20.iiOFNoDZfZZBtJic9OGo
	jz85qcgCVAp8.5iFCBS_w4LdytLGm770mgHH847zzAcsErgEkFNgSHFsORvjjV9doB24zwVS9Qhg
	.JWbslTxGt6bstpQfXFSrRhMcUN4rncXCvLMHyPm7TkZqW5B1lZMaBT3v9aBYvdevRV8WG97jU.G
	4svfWs1ULv4qAiLL1OrfgGz4cgNzvC7CB11e1gm4yR4TWboEZIOoZnAfY8C6qztIkrAhC9jQRPeq
	Ja16vulIwd7g8XqL.vS8VW3EOI3HKRAOB2GbQDqoyKRmyludEE01mi6P.igYdZIiudiSP1AtT0mq
	pLBViNRjsqUUpMYZ9VfzcDW.PjwnvyjiXuhuiLprNbCFp0IIaFpjRc7ugy9nGZ_isdLJRTAR3Iwc
	rf8X1vyKIZdLzsil3VgZbqPXRJ8Og.gunXA8daEfJQYBIxZ_cocdjs1sFHrCSbxzPTeQdKNLcKnG
	6IQxRWV_mTCxYf7CfS7jChbDjrXW7FAIb.2YcxzElxugl2NCf3AKtT0BEXsPCztps47J58AESEs9
	iThWivXjW56Q8p7_kzML0J0lP_JVR1KMN7ByRCOI2jVxiKfcw13a4gSz4kY0Wnem93JXzrL5QE3D
	1qm_.E9PHcUuJpQs5z9hKTPy6jZWPCKC6Ctjb0lSpvMU58tDkTQLFxtR9VLZhqpiKI5.7ENUd6bd
	5ESvcUywcx07jK9A5HcWUasUvcM.mGxmLLZuxHzCt8HGL2LgIddHuM8hmP8_z7RkAtrYbPJ_wXf8
	wM2Q_wT7EwFt4eUIne9sNwbVeqwhI4gt1.W8Ag01wVf1UfmaeCfv5JYg6g9zlIYJrejfT0fBDe6B
	smqVjPrWt6t6.5sBcV1a7FYBz1vTOGIGl
Date:	Thu, 2 Apr 2020 12:47:02 +0000 (UTC)
To:	"sten DOT kristian DOT ivarsson AT gmail DOT com" <sten DOT kristian DOT ivarsson AT gmail DOT com>,
	Kristian Ivarsson via Cygwin <cygwin AT cygwin DOT com>,
	"'Ken Brown'" <kbrown AT cornell DOT edu>
Message-ID:	<1644706448.958021.1585831622425@mail.yahoo.com>
In-Reply-To:	<000901d608c5$86361880$92a24980$@gmail.com>
References:	<1b1401d60296$2769e690$763db3b0$@gmail.com>
	<716e2076-f607-454e-2723-937c3959e2a3 AT cornell DOT edu>
	<18be01d602ab$0bbfca30$233f5e90$@gmail.com>
	<35b43b59-6410-f21f-710c-385e39cbae0b AT cornell DOT edu>
	<005201d603ba$2bc8ab20$835a0160$@gmail.com>
	<472d1df6-531a-ebd7-4ffa-583a06e270ff AT cornell DOT edu>
	<ce4a4877-df77-3400-e8ac-16655b313757 AT cornell DOT edu>
	<b311d907-7376-5bc6-3216-7d2b96728dbc AT cornell DOT edu>
	<00b901d60447$7ecb4c50$7c61e4f0$@gmail.com>
	<f8f5c9b4-9eb9-85b0-ab8f-44b3b2458e0b AT cornell DOT edu>
	<00e001d604f9$d0aa0720$71fe1560$@gmail.com>
	<8c6c5655-c162-8361-9f44-376bbd7cf114 AT cornell DOT edu>
	<d213c473-00c2-0308-b720-d8f274126681 AT cornell DOT edu>
	<3fe06192-7300-382a-8c98-f1bc2ff81e36 AT cornell DOT edu>
	<003701d607a0$c975f140$5c61d3c0$@gmail.com>
	<249be61e-da8a-7da1-ca67-0c4c6433a415 AT cornell DOT edu>
	<000a01d60802$d1525900$73f70b00$@gmail.com>
	<f7922d62-097c-e284-ded5-6d7a1c0f0d66 AT cornell DOT edu>
	<001601d60848$fcffd320$f6ff7960$@gmail.com>
	<7b5b058e-5047-4d49-8c31-5553056f3845 AT cornell DOT edu>
	<7897bc10-439d-64aa-c173-f0bf4ec8246 8 AT cornell DOT edu>
	<7897bc10-439d-64aa-c173-f0bf4ec82468 AT cornell DOT edu>
	<000901d608c5$86361880$92a24980$@gmail.com>
Subject:	Re: Sv: Sv: Sv: Sv: Sv: Sv: Sv: Sv: Named pipes and multiple wri
MIME-Version:	1.0
X-Mailer:	WebService/1.1.15585 YahooMailAndroidMobile YMobile/1.0
	(com.yahoo.mobile.client.android.mail/6.5.3; Android/10; QP1A.190711.020;
	beyond1; samsung; SM-G973F; 5.58; 2042x1080; )
X-Spam-Status:	No, score=0.6 required=5.0 tests=BAYES_00, DKIM_SIGNED,
	DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, FREEMAIL_ENVFROM_END_DIGIT,
	FREEMAIL_FROM, FREEMAIL_REPLYTO_END_DIGIT, HTML_MESSAGE, RCVD_IN_DNSWL_NONE,
	RCVD_IN_MSPIKE_H2, REPTO_QUOTE_YAHOO, SPF_HELO_NONE, SPF_PASS,
	TXREP autolearn=no autolearn_force=no version=3.4.2
X-Spam-Checker-Version:	SpamAssassin 3.4.2 (2018-09-13) on
	server2.sourceware.org
X-Content-Filtered-By:	Mailman/MimeDel 2.1.29
X-BeenThere:	cygwin AT cygwin DOT com
X-Mailman-Version:	2.1.29
List-Id:	General Cygwin discussions and problem reports <cygwin.cygwin.com>
List-Unsubscribe:	<http://cygwin.com/mailman/options/cygwin>,
	<mailto:cygwin-request AT cygwin DOT com?subject=unsubscribe>
List-Archive:	<https://cygwin.com/pipermail/cygwin/>
List-Post:	<mailto:cygwin AT cygwin DOT com>
List-Help:	<mailto:cygwin-request AT cygwin DOT com?subject=help>
List-Subscribe:	<http://cygwin.com/mailman/listinfo/cygwin>,
	<mailto:cygwin-request AT cygwin DOT com?subject=subscribe>
From:	Gregery Barton via Cygwin <cygwin AT cygwin DOT com>
Reply-To:	"gregery20 AT yahoo DOT com DOT au" <gregery20 AT yahoo DOT com DOT au>
Cc:	"'cygwin'" <cygwin AT cygwin DOT com>
Errors-To:	cygwin-bounces AT cygwin DOT com
Sender:	"Cygwin" <cygwin-bounces AT cygwin DOT com>
X-MIME-Autoconverted:	from base64 to 8bit by delorie.com id 032ClduC027709

How do I unsubscribed from cygwin mailing lists? I tried simple unsubscribe but it's not so simple. I don't have a password and it won't send me one.

Sent from Yahoo7 Mail on Android 
 
  On Thu, 2 Apr 2020 at 7:06 pm, Kristian Ivarsson via Cygwin<cygwin AT cygwin DOT com> wrote:   > On 4/1/2020 2:34 PM, Ken Brown via Cygwin wrote:
> > On 4/1/2020 1:14 PM, sten DOT kristian DOT ivarsson AT gmail DOT com wrote:
> >>> On 4/1/2020 4:52 AM, sten DOT kristian DOT ivarsson AT gmail DOT com wrote:
> >>>>> On 3/31/2020 5:10 PM, sten DOT kristian DOT ivarsson AT gmail DOT com wrote:
> >>>>>>> On 3/28/2020 10:19 PM, Ken Brown via Cygwin wrote:
> >>>>>>>> On 3/28/2020 11:43 AM, Ken Brown via Cygwin wrote:
> >>>>>>>>> On 3/28/2020 8:10 AM, sten DOT kristian DOT ivarsson AT gmail DOT com wrote:
> >>>>>>>>>>> On 3/27/2020 10:53 AM, sten DOT kristian DOT ivarsson AT gmail DOT com wrote:
> >>>>>>>>>>>>> On 3/26/2020 7:19 PM, Ken Brown via Cygwin wrote:
> >>>>>>>>>>>>>> On 3/26/2020 6:39 PM, Ken Brown via Cygwin wrote:
> >>>>>>>>>>>>>>> On 3/26/2020 6:01 PM, sten DOT kristian DOT ivarsson AT gmail DOT com
wrote:
> >>>>>>>>>>>>>>>> The ENIXIO occurs when parallel child-processes
> >>>>>>>>>>>>>>>> simultaneously using O_NONBLOCK opening the descriptor.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> This is consistent with my guess that the error is
> >>>>>>>>>>>>>>> generated by fhandler_fifo::wait.  I have a feeling that
> >>>>>>>>>>>>>>> read_ready should have been created as a manual-reset
> >>>>>>>>>>>>>>> event, and that more care is needed to make sure it's
> >>>>>>>>>>>>>>> set
> >> when it should be.
> >>
> >> [snip]
> >>
> >>>>>>>> Never mind.  I was able to reproduce the problem and find the
cause.
> >>>>>>>> What happens is that when the first subprocess exits,
> >>>>>>>> fhandler_fifo::close resets read_ready.  That causes the second
> >>>>>>>> and subsequent subprocesses to think that there's no reader
> >>>>>>>> open, so their attempts to open a writer with O_NONBLOCK fail
with ENXIO.
> >>
> >> [snip]
> >>
> >>>> I wrote in a previous mail in this topic that it seemed to work
> >>>> fine for me as well, but when I bumped up the numbers of writers
> >>>> and/or the number of messages (e.g. 25/25) it starts to fail again
> >>
> >> [snip]
> >>
> >>> Yes, it is a resource issue.  There is a limit on the number of
> >>> writers
> >> that can be open at one
> >>> time, currently 64.  I chose that number arbitrarily, with no idea
> >>> what
> >> might actually be
> >>> needed in practice, and it can easily be changed.
> >>
> >> Does it have to be a limit at all ? We would rather see that the
> >> application decide how much resources it would like to use. In our
> >> particular case there will be a process-manager with an incoming pipe
> >> that possible several thousands of processes will write to
> >
> > I agree.
> >
> >> Just for fiddling around (to figure out if this is the limit that
> >> make other things work a bit odd), where's this 64 limit defined now ?
> >
> > It's MAX_CLIENTS, defined in fhandler.h.  But there seem to be other
> > resource issues also; simply increasing MAX_CLIENTS doesn't solve the
> > problem.  I think there are also problems with the number of threads,
> > for example.  Each time your program forks, the subprocess inherits
> > the rfd file descriptor and its "fifo_reader_thread" starts up.  This
> > is unnecessary for your application, so I tried disabling it (in
> fhandler_fifo::fixup_after_fork), just as an experiment.
> >
> > But then I ran into some deadlocks, suggesting that one of the locks
> > I'm using isn't robust enough.  So I've got a lot of things to work on.
> >
> >>> In addition, a writer isn't recognized as closed until a reader
> >>> tries to
> >> read and gets an error.
> >>> In your example with 25/25, the list of writers quickly gets to 64
> >>> before
> >> the parent ever tries
> >>> to read.
> >>
> >> That explains the behaviour, but should there be some error returned
> >> from open/write (maybe it is but I'm missing it) ?
> >
> > The error is discovered in add_client_handler, called from
> > thread_func.  I think you'll only see it if you run the program under
> > strace.  I'll see if I can find a way to report it.  Currently,
> > there's a retry loop in fhandler_fifo::open when a writer tries to
> > open, and I think I need to limit the number of retries and then error
out.
> 
> I pushed a few improvements and bug fixes, and your 25/25 example now runs
without a
> problem.  I increased MAX_CLIENTS to 1024 just for the sake of this
example, but I'll
> work on letting the number of writers increase dynamically as needed.

I pulled it and tried it out and yes, the sample test program with 25/25
worked well and a whole bunch of our unit-tests passed with ok result now

We still do have some issues, but I cannot yet tell if they are related to
named pipes or not

It is great that you're looking into a totally dynamic solution

Kristian

> Ken

--
Problem reports:      https://cygwin.com/problems.html
FAQ:                  https://cygwin.com/faq/
Documentation:        https://cygwin.com/docs.html
Unsubscribe info:    https://cygwin.com/ml/#unsubscribe-simple  
--
Problem reports:      https://cygwin.com/problems.html
FAQ:                  https://cygwin.com/faq/
Documentation:        https://cygwin.com/docs.html
Unsubscribe info:     https://cygwin.com/ml/#unsubscribe-simple

- Raw text -