DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org 741E4385E02F
To: "'Ken Brown'" <kbrown@cornell.edu>
References: <1b1401d60296$2769e690$763db3b0$@gmail.com>
 <716e2076-f607-454e-2723-937c3959e2a3@cornell.edu>
 <18be01d602ab$0bbfca30$233f5e90$@gmail.com>
 <35b43b59-6410-f21f-710c-385e39cbae0b@cornell.edu>
 <005201d603ba$2bc8ab20$835a0160$@gmail.com>
 <472d1df6-531a-ebd7-4ffa-583a06e270ff@cornell.edu>
 <ce4a4877-df77-3400-e8ac-16655b313757@cornell.edu>
 <b311d907-7376-5bc6-3216-7d2b96728dbc@cornell.edu>
 <00b901d60447$7ecb4c50$7c61e4f0$@gmail.com>
 <f8f5c9b4-9eb9-85b0-ab8f-44b3b2458e0b@cornell.edu>
 <00e001d604f9$d0aa0720$71fe1560$@gmail.com>
 <8c6c5655-c162-8361-9f44-376bbd7cf114@cornell.edu>
 <d213c473-00c2-0308-b720-d8f274126681@cornell.edu>
 <3fe06192-7300-382a-8c98-f1bc2ff81e36@cornell.edu>
 <003701d607a0$c975f140$5c61d3c0$@gmail.com>
 <249be61e-da8a-7da1-ca67-0c4c6433a415@cornell.edu>
 <000a01d60802$d1525900$73f70b00$@gmail.com>
 <f7922d62-097c-e284-ded5-6d7a1c0f0d66@cornell.edu>
In-Reply-To: <f7922d62-097c-e284-ded5-6d7a1c0f0d66@cornell.edu>
Subject: Sv: Sv: Sv: Sv: Sv: Sv: Sv: Named pipes and multiple writers
Date: Wed, 1 Apr 2020 19:14:21 +0200
Message-ID: <001601d60848$fcffd320$f6ff7960$@gmail.com>
MIME-Version: 1.0
Content-Language: en-gb
Thread-Index: AQJE9fQg8TMZuJRGwTEKbo0ZGNgDeQHtMggPA01jw/MCH/KUAAKPevBvAf/qW+kC4eksHQHeBGSzAqmcYp4CB8F8lwIBR+2oARkkbuUCauCM3AJYP32sAmYz8EcC5hrYQwEcxqeZAcg8ToumW1EdAA==
From: Kristian Ivarsson via Cygwin <cygwin@cygwin.com>
Reply-To: sten.kristian.ivarsson@gmail.com
Cc: "'cygwin'" <cygwin@cygwin.com>
Content-Type: text/plain; charset="iso-8859-1"
Sender: "Cygwin" <cygwin-bounces@cygwin.com>
Content-Transfer-Encoding: 8bit

> On 4/1/2020 4:52 AM, sten.kristian.ivarsson@gmail.com wrote:
> >> On 3/31/2020 5:10 PM, sten.kristian.ivarsson@gmail.com wrote:
> >>>> On 3/28/2020 10:19 PM, Ken Brown via Cygwin wrote:
> >>>>> On 3/28/2020 11:43 AM, Ken Brown via Cygwin wrote:
> >>>>>> On 3/28/2020 8:10 AM, sten.kristian.ivarsson@gmail.com wrote:
> >>>>>>>> On 3/27/2020 10:53 AM, sten.kristian.ivarsson@gmail.com wrote:
> >>>>>>>>>> On 3/26/2020 7:19 PM, Ken Brown via Cygwin wrote:
> >>>>>>>>>>> On 3/26/2020 6:39 PM, Ken Brown via Cygwin wrote:
> >>>>>>>>>>>> On 3/26/2020 6:01 PM, sten.kristian.ivarsson@gmail.com wrote:
> >>>>>>>>>>>>> The ENIXIO occurs when parallel child-processes
> >>>>>>>>>>>>> simultaneously using O_NONBLOCK opening the descriptor.
> >>>>>>>>>>>>
> >>>>>>>>>>>> This is consistent with my guess that the error is
> >>>>>>>>>>>> generated by fhandler_fifo::wait.  I have a feeling that
> >>>>>>>>>>>> read_ready should have been created as a manual-reset
> >>>>>>>>>>>> event, and that more care is needed to make sure it's set
when it should be.

[snip] 

> >>>>> Never mind.  I was able to reproduce the problem and find the cause.
> >>>>> What happens is that when the first subprocess exits,
> >>>>> fhandler_fifo::close resets read_ready.  That causes the second
> >>>>> and subsequent subprocesses to think that there's no reader open,
> >>>>> so their attempts to open a writer with O_NONBLOCK fail with ENXIO.

[snip] 

> > I wrote in a previous mail in this topic that it seemed to work fine
> > for me as well, but when I bumped up the numbers of writers and/or the
> > number of messages (e.g. 25/25) it starts to fail again

[snip] 

> Yes, it is a resource issue.  There is a limit on the number of writers
that can be open at one
> time, currently 64.  I chose that number arbitrarily, with no idea what
might actually be
> needed in practice, and it can easily be changed.

Does it have to be a limit at all ? We would rather see that the application
decide how much resources it would like to use. In our particular case there
will be a process-manager with an incoming pipe that possible several
thousands of processes will write to

Just for fiddling around (to figure out if this is the limit that make other
things work a bit odd), where's this 64 limit defined now ?

> In addition, a writer isn't recognized as closed until a reader tries to
read and gets an error.
> In your example with 25/25, the list of writers quickly gets to 64 before
the parent ever tries
> to read.

That explains the behaviour, but should there be some error returned from
open/write (maybe it is but I'm missing it) ?

> I'll see if I can find a better way to manage this.
> 
> Ken

Kristian
Kristian 

--
Problem reports:      https://cygwin.com/problems.html
FAQ:                  https://cygwin.com/faq/
Documentation:        https://cygwin.com/docs.html
Unsubscribe info:     https://cygwin.com/ml/#unsubscribe-simple