X-Recipient: archive-cygwin AT delorie DOT com
DomainKey-Signature: a=rsa-sha1; c=nofws; d=sourceware.org; h=list-id
	:list-unsubscribe:list-subscribe:list-archive:list-post
	:list-help:sender:date:from:to:subject:message-id:reply-to
	:references:mime-version:content-type:in-reply-to; q=dns; s=
	default; b=yUCMk6Xm7KKI4WTyuTt9RcPN6CWa5j26BBnHim5Zug+me1YJSH+qV
	WoCordgVcy976cm+9ygIUcnFJ6nZqbpIDIrH0svjWI5GMTXt4Qwiy2wV8fXJ7RG2
	EhOeJSH16hlLoG1GWGRnS/GJC6eteXQ2BgVxLTS+9m9WfYUK0YN2NI=
DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=sourceware.org; h=list-id
	:list-unsubscribe:list-subscribe:list-archive:list-post
	:list-help:sender:date:from:to:subject:message-id:reply-to
	:references:mime-version:content-type:in-reply-to; s=default;
	 bh=TQ9fe5zeGkr9o26w2Y0jxWtG9x0=; b=Tud2QpCKvamRnyk3loJqZVAJPNhn
	8rSRqLyJLCwVNEchIYJkzYhk0hT3pG5wReii53T4U+O+Khy/igE8EC6k4sj/Ge4c
	vjD7pnBnBRDRmmnu3qcBwgmYTV0hORmALdMmabL1axxZ9Hn5ipPUyiaVd6WpxTZZ
	1DIVNiaY5k+YEcw=
Mailing-List: contact cygwin-help AT cygwin DOT com; run by ezmlm
List-Id: <cygwin.cygwin.com>
List-Subscribe: <mailto:cygwin-subscribe AT cygwin DOT com>
List-Archive: <http://sourceware.org/ml/cygwin/>
List-Post: <mailto:cygwin AT cygwin DOT com>
List-Help: <mailto:cygwin-help AT cygwin DOT com>, <http://sourceware.org/ml/#faqs>
Sender: cygwin-owner AT cygwin DOT com
Mail-Followup-To: cygwin AT cygwin DOT com
Delivered-To: mailing list cygwin AT cygwin DOT com
Authentication-Results: sourceware.org; auth=none
X-Virus-Found: No
X-Spam-SWARE-Status: No, score=-5.9 required=5.0 tests=AWL,BAYES_00 autolearn=ham version=3.3.2
X-HELO: calimero.vinschen.de
Date: Fri, 20 Feb 2015 11:13:19 +0100
From: Corinna Vinschen <corinna-cygwin AT cygwin DOT com>
To: cygwin AT cygwin DOT com
Subject: Re: Clearing O_NONBLOCK from a pipe may lose data
Message-ID: <20150220101319.GQ26084@calimero.vinschen.de>
Reply-To: cygwin AT cygwin DOT com
Mail-Followup-To: cygwin AT cygwin DOT com
References: <20150218220859 DOT 1e8f8b19 AT tukaani DOT org> <20150219095147 DOT GC26084 AT calimero DOT vinschen DOT de> <54E660F1 DOT 3040509 AT towo DOT net> <145631367 DOT 20150220024700 AT yandex DOT ru> <54E6E8AF DOT 6000701 AT towo DOT net>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha1;	protocol="application/pgp-signature"; boundary="ZVOC9e0LfXEId8h2"
Content-Disposition: inline
In-Reply-To: <54E6E8AF.6000701@towo.net>
User-Agent: Mutt/1.5.23 (2014-03-12)

--ZVOC9e0LfXEId8h2
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On Feb 20 08:56, Thomas Wolff wrote:
>=20
> Am 20.02.2015 um 00:47 schrieb Andrey Repin:
> >Greetings, Thomas Wolff!
> >
> >>Am 19.02.2015 um 10:51 schrieb Corinna Vinschen:
> >>>On Feb 18 22:08, Lasse Collin wrote:
> >>>>(Please Cc me when replying, I'm not subscribed to the list.)
> >>>>
> >>>>Hi!
> >>>>
> >>>>I suspect that there is a bug in Cygwin:
> >>>>
> >>>>1. Create a pipe with both ends in blocking mode (O_NONBLOCK
> >>>>     is not set).
> >>>>2. The writer sets its end to non-blocking mode.
> >>>>3. The writer writes to the pipe.
> >>>>4. The writer restores its end of the pipe to blocking mode
> >>>>     before the reader has read anything from the pipe.
> >>>>5. The writer closes its end of the pipe.
> >>>>6. The reader reads from the pipe in blocking mode. The last
> >>>>     bytes written by the writer never appear at the reader,
> >>>>     thus data is silently lost.
> >>>>
> >>>>Omitting the step 4 above makes the problem go away.
> >>>I can imagine.  A few years back, when changing the pipe code to
> >>>using overlapped IO, we stumbled over a problem in Windows.  When
> >>>closing an overlapped pipe while I/O is still ongoing, Windows
> >>>simply destroys the pipe buffers without flushing the data to the
> >>>reader.  This is not much of a problem for blocking IO, but it
> >>>obviously is for non-blocking.
> >>>
> >>>The workaround for this behaviour is this:  If the pipe is closed, and
> >>>this is the writing side of a nonblocking pipe, a background thread ge=
ts
> >>>started which keeps the overlapped structure open and continues to wait
> >>>for IO completion (i.e. the data has been sent to the reader).
> >>>
> >>>However, if you switch back to blocking before closing the pipe, the
> >>>aforementioned mechanism does not kick in.
> >>Could not "switching back to blocking" simply be handled like closing as
> >>far as the waiting is concerned,
> >>thus effectively flushing the pipe buffer?
> >You can't "just flush" it, if the receiving end isn't reading from it.
> By flushing I meant actually waiting until it's been consumed at the
> other end in this case, if that's technically feasible.

You mean the actual act of changing the descriptor from non-blocking
to blocking, as in fcntl(fd, F_SETFL), shall perform the same action
of waiting as the close call on non-blocking descriptors does?

> I see no strict requirement that the fcntl call removing O_NONBLOCK from
> a file descriptor should itself still be handled as nonblocking (it can
> well be argued that the flag is changed first and then the call is
> allowed to block) - and even if this were not proper it is certainly
> more acceptable than losing data.

I'm not sure that works as desired, but it's probably worth a try.  An
fcntl method for pipes has to be added (there is none yet, it's all done
in fhandler_base::fcntl), then the F_SETFL command would have to be
augmented to create a thread calling FlushFileBuffers (which is
*supposed* to work on pipe handles but I never tried it myself), and the
fcntl call would have to wait for thread completion, allowing
interruption by signals (calling cygwait, that is).

The question with stuff like this is usually, how long are you willing
to wait?  You never know what the reader side of a pipe is doing.  It
might just be busy and intends to read from the pipe in a second, a
minute, or an hour.


Corinna

--=20
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

--ZVOC9e0LfXEId8h2
Content-Type: application/pgp-signature

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQIcBAEBAgAGBQJU5wi/AAoJEPU2Bp2uRE+gY9IP/RpDlh0Ly4vL4UO/GAi09VXl
xsVZR6cF2ZA25ROrSxgab+bdB9OF0SQ7TIw2HYzCCfpfguymm6sh16rSp3BlcZGf
dp9sKQflQeQgxn1xauhoZQt3hJagAYRQWy3Ng64Ox3HCOUG9iP0C6/gFF8LQlHAn
Q64Z/lchJmMZt/0INZfQ1Yxn8SXpZOKMsVw2NiUcDJGYLlNVD1piOX0VTV1Vumdj
BsvOaxG9SuPQLdgF1Fvpdoqj6GjrarTF8LGsEkzdOa1YsEjqNFIPE7EsmrcakQB/
H32S6EeCEoNSmV8s+8az2WURGwfCsc4vyxQ940RsKYVdLUgitfyp9t2IzP5nEyhU
rdtPXWxGYAfPtpdgUM9Z/b2BJEymrGxYcCWIU2PN11UwvqXDx/932n0g+nCS0EX4
O0LPZRf+T/U/fblawzj5LaShbMPcZpJjlBeF/AmJNGfTv9OWXJFu/wAKkfmUzZdv
768MHezkBMBwi2J1zmzvF8ztHYAQqJvtY/m8qtZwX0mBk/3Y7rZ71PdVxDfkz7Gp
OnXU2fFki8sg4nybPWROFeWpVPKaKQf6ppHx5EIewtw2FnagyNt1JMdOJpKFL44S
cTCK3MfN8ZU++4dYhiOJ3a1pIJ/6PzFiDjdLP8vIWG/sJSdKM6ZOYTO1jGBprLvs
nx4HkdKIHJW7P5EidWot
=ILqp
-----END PGP SIGNATURE-----

--ZVOC9e0LfXEId8h2--