Mail Archives: cygwin/2020/03/31/17:11:20

Kristian Ivarsson via Cygwin <cygwin AT cygwin DOT com> · Tue, 31 Mar 2020 23:10:19 +0200

This is a multipart message in MIME format.

------=_NextPart_000_0038_01D607B1.8D015950
Content-Type: text/plain;
	charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable

>On 3/28/2020 10:19 PM, Ken Brown via Cygwin wrote:
>> On 3/28/2020 11:43 AM, Ken Brown via Cygwin wrote:
>>> On 3/28/2020 8:10 AM, sten DOT kristian DOT ivarsson AT gmail DOT com wrote:
>>>>> On 3/27/2020 10:53 AM, sten DOT kristian DOT ivarsson AT gmail DOT com wrote:
>>>>>>> On 3/26/2020 7:19 PM, Ken Brown via Cygwin wrote:
>>>>>>>> On 3/26/2020 6:39 PM, Ken Brown via Cygwin wrote:
>>>>>>>>> On 3/26/2020 6:01 PM, sten DOT kristian DOT ivarsson AT gmail DOT com wrote:
>>>>>>>>>> The ENIXIO occurs when parallel child-processes =
simultaneously=20
>>>>>>>>>> using O_NONBLOCK opening the descriptor.
>>>>>>>>>
>>>>>>>>> This is consistent with my guess that the error is generated =
by=20
>>>>>>>>> fhandler_fifo::wait.=A0 I have a feeling that read_ready =
should=20
>>>>>>>>> have been created as a manual-reset event, and that more care=20
>>>>>>>>> is needed to make sure it's set when it should be.
>>>>>>>>>
>>>>>>>>>> I could provide a code-snippet to reproduce it if wanted ?
>>>>>>>>>
>>>>>>>>> Yes, please!
>>>>>>>>
>>>>>>>> That might not be necessary.=A0 If you're able to build the git =

>>>>>>>> repo master branch, please try the attached patch.
>>>>>>
>>>>>>> Here's a better patch.
>>>>>>
>>>>>>
>>>>>> I finally succeeded to build latest master (make is not my=20
>>>>>> favourite
>>>>>> tool) and added the patch, but still no success in my little=20
>>>>>> test-program (see
>>>>>> attachment) when creating a write-file-descriptor with O_NONBLOCK
>>>>
>>>>> Your test program fails for me on Linux too.=A0 Here's the output=20
>>>>> from one
>>>> run:
>>>>
>>>> You're right. That was extremely careless of me to not test this in =

>>>> Linux first :-)
>>>
>>> No problem.
>>>
>>>> I can assure that we have a use case that works on Linux but not in =

>>>> Cygwin, but it seems like I failed to narrow it down in the wrong=20
>>>> way
>>>>
>>>> I'll try to rearrange my code (that works in Linux) to mimic our=20
>>>> application but in a simple way (I'll be back)
>>>
>>> OK, I'll be waiting for you.=A0 BTW, if it's not too hard to write =
your=20
>>> test case in plain C, or at least less modern C++, that would=20
>>> simplify things for me.=A0 For example, your pipe.cpp failed to =
compile=20
>>> on one Linux machine I wanted to test it on, presumably because that
machine had an older C++ compiler.
>>=20
>> Never mind.=A0 I was able to reproduce the problem and find the =
cause.=A0=20
>> What happens is that when the first subprocess exits,=20
>> fhandler_fifo::close resets read_ready.=A0 That causes the second and =

>> subsequent subprocesses to think that there's no reader open, so =
their=20
>> attempts to open a writer with O_NONBLOCK fail with ENXIO.
>>=20
>> I should be able to fix this tomorrow.

>I've pushed what I think is a fix to the topic/fifo branch.  I tested =
it
with the attached program, which is a variant of the test case you sent =
last
week.=20
>Please test it in your use case.

>Note: If you've previously pulled the topic/fifo branch, then you will
probably get a lot of conflicts when you pull again, because I did a =
forced
push a few days ago.  If that happens, just do

>   git reset --hard origin/topic/fifo

>It turned out that the fix required some of the ideas that I've been
working on in connection with allowing multiple readers.  Even though =
the
code allows a FIFO to be *explicitly* opened for reading only once, =
there
can still be several open file descriptors for readers because of dup =
and
fork.  The existing code on git master doesn't handle those situations
properly.

>The code on topic/fifo doesn't completely fix that yet, but I think it
should work under the following assumptions:

>1. The FIFO is opened only once for reading.

>2. The file descriptor obtained from this is the only one on which a =
read
is attempted.

>I'm working on removing both of these restrictions.

>Ken=20

We finally took the time to make some kind of a simplified "hack" that =
works
on Ubuntu and BSD/OSX but with latest on master newlib-cygwin gave =
"ENXIO"
now and then but with your previous patch attached, there was no ENXIO =
but
::read returns EAGIN (until exhausted) (with cygwin) almost every run

I will try your newest things tomorrow

See latest attatched test-program (starts to get bloated but this time =
more
C-compatible though:-)

Kristian

------=_NextPart_000_0038_01D607B1.8D015950
Content-Type: text/plain;
	name="pipe.cpp"
Content-Transfer-Encoding: quoted-printable
Content-Disposition: attachment;
	filename="pipe.cpp"

#include <string.h>
#include <fcntl.h>
#include <sys/stat.h>
#include <sys/types.h>
#include <sys/wait.h>
#include <unistd.h>
#include <stdio.h>
#include <errno.h>

int print_error(const int error)
{
   printf("%s\n", strerror(error));
   return error;
}
struct Message
{
   int pid;
   int index;
};

int main()
{
   const char* name =3D "/tmp/my_pipe";
   const int result =3D mkfifo(name, 0666);
   if (result)
      return print_error(errno);
   const int writers{5};
   const int messages{5};
   printf("open parent pipe\n");
   const int rfd =3D open(name, O_RDONLY | O_NONBLOCK);
   const int wfd =3D open(name, O_WRONLY);
   if (rfd < 0)
      return print_error(errno);
   const int block_alternation[] =3D {0, O_NONBLOCK};
   int pids[writers];
   for (auto idx =3D 0; idx < writers; ++idx)
   {
      const auto pid =3D fork();
      if (pid < 0)
         return print_error(errno);
      if (pid =3D=3D 0)
      {
         printf("child %d\n", getpid());
         for (int index =3D 0; index < messages; ++index)
         {
            const int wfd =3D open(name, O_WRONLY | =
block_alternation[index % 2]);
            if (wfd < 0)
               return print_error(errno);
            Message message{getpid(), index};
            short attempts{10000};
            while (write(wfd, &message, sizeof(message)) =3D=3D -1 && =
errno =3D=3D EAGAIN && attempts > 0)
               --attempts;
            if(attempts =3D=3D 0)
               printf("child %d write - pipe busy\n", getpid());
            close(wfd);
         }
         return 0;
      }
      pids[idx] =3D pid;
   }
   {
      printf("parent read\n");
      for (int idx =3D 0; idx < writers * messages; ++idx)
      {
         short attempts{10000};
         Message message{};
         while (::read(rfd, &message, sizeof(message)) =3D=3D -1 && =
errno =3D=3D EAGAIN && attempts > 0)
            --attempts;

         if (attempts =3D=3D 0)
            printf("parent %d read - pipe empty\n", getpid());
         else
            printf("message %d %d\n", message.index, message.pid);
      }
      // wait for children.
      for (int idx =3D 0; idx < writers; ++idx)
      {
         int status =3D 0;
         waitpid(pids[idx], &status, 0);
         if (WIFEXITED(status))
            printf("child exited %d\n", pids[idx]);
      }
      close(wfd);
      close(rfd);
   }
   if (unlink(name) < 0)
      return print_error(errno);
   return 0;
}
------=_NextPart_000_0038_01D607B1.8D015950
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

--
Problem reports:      https://cygwin.com/problems.html
FAQ:                  https://cygwin.com/faq/
Documentation:        https://cygwin.com/docs.html
Unsubscribe info:     https://cygwin.com/ml/#unsubscribe-simple

------=_NextPart_000_0038_01D607B1.8D015950--

X-Recipient:	archive-cygwin AT delorie DOT com
DKIM-Filter:	OpenDKIM Filter v2.11.0 sourceware.org A3328385DC07
DKIM-Signature:	v=1; a=rsa-sha256; c=relaxed/relaxed; d=cygwin.com;
	s=default; t=1585689027;
	bh=hQwwcoUSHzcb/8pjwwg8orFsd2OdYTj4vB7e5i5OuYs=;
	h=To:References:In-Reply-To:Subject:Date:List-Id:List-Unsubscribe:
	List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc:
	From;
	b=hVrlVQJnMFRkk66qBV/pxj3iEn3o5c336Oue0ARXt7muHfAxPfxSdYfNS9sWhgamy
	OZ6WSbjEIZ6S4jVqt5k83/P0g52qww9D3bokydlMYy4TBvnl009/OgYMXHnEUck1IR
	IVvfP1jeFeQbdOZt1FVdBQ61PyRAoCYa9EtDHI7Y=
X-Original-To:	cygwin AT cygwin DOT com
Delivered-To:	cygwin AT cygwin DOT com
DMARC-Filter:	OpenDMARC Filter v1.3.2 sourceware.org E7D48385BF83
X-Google-DKIM-Signature:	v=1; a=rsa-sha256; c=relaxed/relaxed;
	d=1e100.net; s=20161025;
	h=x-gm-message-state:from:to:cc:references:in-reply-to:subject:date
	:message-id:mime-version:thread-index:content-language;
	bh=Xcl8Io+7n2xCdqs2QOds+h74wNzyZpIqT+Pm7PFAhPw=;
	b=UJYnvCQHDpQfdGcmKtSNa4ahGPHEfm+IJa1+IrQT7oiwFSGFOvNAm1M1kHtPx6PBkO
	eTXcLkeQPCU1V874Nt6U4AWGR6DocTD3DX2LYyxmqhj3f+rEjvsv4iFt9ICGvQCQKazi
	dpll0FBAoS7DpUOAfbg/W8OLF7YQqh6CEYCi+OUC1QjzDEHG0OFfKtvQfSJzcv7hS2sz
	iMyfgOmY9NH/eTBr3wXsqFhZSp1pOj64muNTWOdX7RAOR6CrWciMnj54wuD4RyVbS2Fl
	a1lUNV+rXxZiYP4bAJ1ngghs1LmKFi3N5zVMTno5oX2R+OW7grITJZu4twZ4xsGaD7Xa
	dRLg==
X-Gm-Message-State:	AGi0PuaFj4G+5co4N+YtCihRCeolZ2nTGjhSyrmf5bZGugYSLzEeWNoF
	myPTWvVv1lhRMRWPT45BsP61PEfx
X-Google-Smtp-Source:	APiQypJKXwTzSUWYNUqFzelvq19LbioXPu59wAS9Z3q14VbXKp3JpRup9myu2A14r0aYDx5aInK+7g==
X-Received:	by 2002:a19:c781:: with SMTP id
	x123mr12420404lff.140.1585689022231;
	Tue, 31 Mar 2020 14:10:22 -0700 (PDT)
To:	"'Ken Brown'" <kbrown AT cornell DOT edu>
References:	<1b1401d60296$2769e690$763db3b0$@gmail.com>
	<716e2076-f607-454e-2723-937c3959e2a3 AT cornell DOT edu>
	<18be01d602ab$0bbfca30$233f5e90$@gmail.com>
	<35b43b59-6410-f21f-710c-385e39cbae0b AT cornell DOT edu>
	<005201d603ba$2bc8ab20$835a0160$@gmail.com>
	<472d1df6-531a-ebd7-4ffa-583a06e270ff AT cornell DOT edu>
	<ce4a4877-df77-3400-e8ac-16655b313757 AT cornell DOT edu>
	<b311d907-7376-5bc6-3216-7d2b96728dbc AT cornell DOT edu>
	<00b901d60447$7ecb4c50$7c61e4f0$@gmail.com>
	<f8f5c9b4-9eb9-85b0-ab8f-44b3b2458e0b AT cornell DOT edu>
	<00e001d604f9$d0aa0720$71fe1560$@gmail.com>
	<8c6c5655-c162-8361-9f44-376bbd7cf114 AT cornell DOT edu>
	<d213c473-00c2-0308-b720-d8f274126681 AT cornell DOT edu>
	<3fe06192-7300-382a-8c98-f1bc2ff81e36 AT cornell DOT edu>
In-Reply-To:	<3fe06192-7300-382a-8c98-f1bc2ff81e36@cornell.edu>
Subject:	Sv: Sv: Sv: Sv: Sv: Named pipes and multiple writers
Date:	Tue, 31 Mar 2020 23:10:19 +0200
Message-ID:	<003701d607a0$c975f140$5c61d3c0$@gmail.com>
MIME-Version:	1.0
X-Mailer:	Microsoft Outlook 16.0
Thread-Index:	AQJE9fQg8TMZuJRGwTEKbo0ZGNgDeQHtMggPA01jw/MCH/KUAAKPevBvAf/qW+kC4eksHQHeBGSzAqmcYp4CB8F8lwIBR+2oARkkbuUCauCM3AJYP32sppuMheA=
X-Spam-Status:	No, score=-5.0 required=5.0 tests=BAYES_00, DKIM_SIGNED,
	DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, FREEMAIL_FROM, GIT_PATCH_1,
	RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS,
	TXREP autolearn=ham autolearn_force=no version=3.4.2
X-Spam-Checker-Version:	SpamAssassin 3.4.2 (2018-09-13) on
	server2.sourceware.org
X-BeenThere:	cygwin AT cygwin DOT com
X-Mailman-Version:	2.1.29
List-Id:	General Cygwin discussions and problem reports <cygwin.cygwin.com>
List-Archive:	<http://cygwin.com/pipermail/cygwin/>
List-Post:	<mailto:cygwin AT cygwin DOT com>
List-Help:	<mailto:cygwin-request AT cygwin DOT com?subject=help>
List-Subscribe:	<http://cygwin.com/mailman/listinfo/cygwin>,
	<mailto:cygwin-request AT cygwin DOT com?subject=subscribe>
From:	Kristian Ivarsson via Cygwin <cygwin AT cygwin DOT com>
Reply-To:	sten DOT kristian DOT ivarsson AT gmail DOT com
Cc:	"'cygwin'" <cygwin AT cygwin DOT com>
Sender:	"Cygwin" <cygwin-bounces AT cygwin DOT com>

webmaster	delorie software privacy
Copyright � 2019 by DJ Delorie	Updated Jul 2019