DomainKey-Signature: a=rsa-sha1; c=nofws; d=sourceware.org; h=list-id
	:list-unsubscribe:list-subscribe:list-archive:list-post
	:list-help:sender:message-id:date:from:mime-version:to:subject
	:references:in-reply-to:content-type; q=dns; s=default; b=ul80hL
	5XNoNf/mw2X9TtcaQfqz/7+2OnPO9G5/FXupQyFyu7+ngwKY4yJTJAqDuc4Gdsjm
	3+szU8iJzTpuCF9ULwutz4RYghWgHMRVMykixfRRyzipxivoZU6Bf0uowOELbI8I
	c9H55ZIapZbCUoV+YQ78yMZPivVUXUKJ8XNpQ=
Mailing-List: contact cygwin-help AT cygwin DOT com; run by ezmlm
Sender: cygwin-owner AT cygwin DOT com
Mail-Followup-To: cygwin AT cygwin DOT com
Message-ID: <54E7C485.1010308@redhat.com>
Date: Fri, 20 Feb 2015 16:34:29 -0700
From: Eric Blake <eblake AT redhat DOT com>
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.4.0
MIME-Version: 1.0
To: cygwin AT cygwin DOT com
Subject: Re: Clearing O_NONBLOCK from a pipe may lose data
References: <20150218220859 DOT 1e8f8b19 AT tukaani DOT org>	<20150219095147 DOT GC26084 AT calimero DOT vinschen DOT de> <20150220222153 DOT 2be22811 AT tukaani DOT org>
In-Reply-To: <20150220222153.2be22811@tukaani.org>
OpenPGP: url=http://people.redhat.com/eblake/eblake.gpg
Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="HifrQThCsET4DPcuT66IvoiLcLBlATFBA"

--HifrQThCsET4DPcuT66IvoiLcLBlATFBA
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

On 02/20/2015 01:21 PM, Lasse Collin wrote:
>=20
> For example, if xz is modified to leave O_NONBLOCK enabled on stdout,
>=20
>     ( xz -dc smallfile.xz ; cat bigfile ) | ( sleep 1 ; wc -c )
>=20
> will make cat print
>=20
>     cat: write error: Resource temporarily unavailable
>=20
> on GNU/Linux and most of bigfile won't be seen by wc. However, on
> Cygwin the above works because the modifications to the file status
> flags in xz aren't seen by cat. That is, stdout in cat is in blocking
> mode even though xz left it in non-blocking mode.

That's a bug in cygwin, although fixing it may be difficult.  In POSIX
terminology, the blocking flag of an open file description is shared
among all the file descriptors visiting that file description, whether
the extra file descriptors were created by dup() or by fork(); and it is
one of the few bits of information where a child can affect the state
visible in the parent.  But if Windows doesn't share blocking status of
multiple handles visiting the same resource, then I'm not sure how we
can emulate it.

[various side notes: Another such bit of information shared between
processes is the file offset of a shared seekable file description, and
Windows DOES support that.  On the other hand, things like the cloexec
flag are associated with a file descriptor [fcntl F_GETFD] rather than a
file description [fcntl F_GETFL], and so are NOT shared across dup() or
fork().  Also, note that two consecutive calls to open() with the same
parameters produces two separate file descriptions (the two file
descriptors do not share state); it is only dup() or fork() that can
create two file descriptors sharing a file description.

Where it gets really weird is with flock() vs. fcntl(F_SETLK)/lockf()
locking: flock() is per-description, but fcntl() and lockf() are
per-inode, which means a second locker visiting a distinct open() can
wipe out the lock of the first description - making lockf() very painful
to use; but flock() lacks byte ranges, making it also unideal.  The
Linux kernel is pioneering a new lock, and POSIX is considering
standardizing it, called fcntl(F_OFD_SETLK) which has all the benefits
of per-description locking (the best of flock) and range locking (the
best of lockf) - eventually, Cygwin should probably implement that as
well.  We already have quite the hacks in place to get flock/lockf
coordination from child back to parent, which would be the sort of code
to borrow from if we have to figure out how to get nonblocking status
propagated in the same direction.

hmm - my side notes took more space than my real response - is that a
good thing?]

>=20
> The above Cygwin behavior would make it very easy to add a workaround
> to xz for the pipe-data-loss problem: simply don't clear O_NONBLOCK on
> stdout. However, I wonder if it is a bug in Cygwin that the changes to
> file status flags aren't seen via other file descriptors that refer to
> the same file description. If it is a bug, then the workaround idea will
> cause trouble in the future when the bug in Cygwin gets fixed.

Yeah, the fact that cygwin is buggy with respect to POSIX may break any
workaround you add if cygwin is later patched.

--=20
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


--HifrQThCsET4DPcuT66IvoiLcLBlATFBA
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
Comment: Public key at http://people.redhat.com/eblake/eblake.gpg
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQEcBAEBCAAGBQJU58SGAAoJEKeha0olJ0NqLCUH/AnCSKr9KqZBsFyYF2dU8m2n
5Ne7FnrOJS3neU5AjyGBQWTKlMFOA1g76K+aHeCOlir2n8vskhQzT6ZT+ZfPbA5R
EcRWIPf6EEU6EtHZYIk4I3f1DtCSyIKpSLAwMTSpXrBUUwbAUuO6TKpKcTZFuLqr
CEAwyYPq9jkH2xuQzzzHjWecAGeiLBpjGXjyy3wane5ZdHeZt8gU/33eoywZgT3L
e4ZCwyYuABYseWBO1ZckASudIgEoYCJ+hy5qCpmCvmhktXOz2JC8T5/wRBbcreh6
TljjA6X0vDEESL4Q+jyOyOIhWAkAO3j+1hTMTrlrQfONtt5wfiVUrvUB/A3UYPM=
=jlXa
-----END PGP SIGNATURE-----

--HifrQThCsET4DPcuT66IvoiLcLBlATFBA--