DomainKey-Signature: a=rsa-sha1; c=nofws; d=sourceware.org; h=list-id
	:list-unsubscribe:list-subscribe:list-archive:list-post
	:list-help:sender:date:from:to:subject:message-id:mime-version
	:content-type; q=dns; s=default; b=XdjoKM3qxrS0Gq3ot3NEmalK+sT8f
	3g0LPN/14pdpYjRVy431gNumbazrHrtsqj1lh9JvFlpOJNxw3sk+8Wf2cfLKOuIs
	K9UD5XvLUJrbbd3FSrcF1Yx8Ufuek74IyJlPMDWeUVQcG2CliuhhJ28DP9GBO8mA
	oO3ugei0Lzk0YQ=
Mailing-List: contact cygwin-help@cygwin.com; run by ezmlm
Sender: cygwin-owner@cygwin.com
Mail-Followup-To: cygwin@cygwin.com
Date: Wed, 18 Feb 2015 22:08:59 +0200
From: Lasse Collin <lasse.collin@tukaani.org>
To: cygwin@cygwin.com
Subject: Clearing O_NONBLOCK from a pipe may lose data
Message-ID: <20150218220859.1e8f8b19@tukaani.org>
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="MP_/CIXb11=_9CwfVfL8SY1E=8d"
Received-SPF: none

--MP_/CIXb11=_9CwfVfL8SY1E=8d
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

(Please Cc me when replying, I'm not subscribed to the list.)

Hi!

I suspect that there is a bug in Cygwin:

1. Create a pipe with both ends in blocking mode (O_NONBLOCK
   is not set).
2. The writer sets its end to non-blocking mode.
3. The writer writes to the pipe.
4. The writer restores its end of the pipe to blocking mode
   before the reader has read anything from the pipe.
5. The writer closes its end of the pipe.
6. The reader reads from the pipe in blocking mode. The last
   bytes written by the writer never appear at the reader,
   thus data is silently lost.

Omitting the step 4 above makes the problem go away.

A small test case is attached. I have tested it under Cygwin64 that was
installed a few days ago on Windows 7. Usage:

    gcc -std=gnu99 -Wall -Wextra writer.c -o writer
    gcc -std=gnu99 -Wall -Wextra reader.c -o reader
    ./writer | ./reader

The reader only gets the first 65536 bytes. The remaining 5555 bytes
are never seen by the reader even though all write() calls succeed at
the writer.

I found this problem because I had heard a rumor that there is something
wrong with xz 5.2.0 on Cygwin. Since the file I/O code had been changed
in 5.2.0, I investigated that first. There also seems to be something
wrong with xz 5.2.0 in threaded mode under Cygwin which needs more
investigation (random pthread_join() failures cause assertion failures
if xz was built with --enable-debug).

-- 
Lasse Collin  |  IRC: Larhzu @ IRCnet & Freenode

--MP_/CIXb11=_9CwfVfL8SY1E=8d
Content-Type: text/x-c++src
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment; filename=reader.c

// gcc -std=gnu99 -Wall -Wextra reader.c -o reader

#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <unistd.h>
#include <stdlib.h>
#include <stdio.h>

static void
fail(const char *str)
{
	fprintf(stderr, "R: %s\n", str);
	exit(EXIT_FAILURE);
}

int
main(void)
{
	// It is assumed that stdin is a pipe open for reading.
	//
	// Read 8 KiB at a time.
	static unsigned char buf[8 << 10];

	// Total amount read
	unsigned total = 0;

	for (unsigned i = 0; i < 10; ++i) {
		fprintf(stderr, "R: Sleeping\n");
		sleep(1);

		int amount = read(STDIN_FILENO, buf, sizeof(buf));
		fprintf(stderr, "R: read() returned %d\n", amount);

		if (amount >= 0) {
			total += amount;
			fprintf(stderr, "R: Total amount read is %u\n",
					total);
		}
	}

	if (close(STDIN_FILENO))
		fail("Error closing stdin");

	return EXIT_SUCCESS;
}

--MP_/CIXb11=_9CwfVfL8SY1E=8d
Content-Type: text/x-c++src
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment; filename=writer.c

// gcc -std=gnu99 -Wall -Wextra writer.c -o writer

#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <unistd.h>
#include <stdlib.h>
#include <stdio.h>

static void
fail(const char *str)
{
	fprintf(stderr, "W: %s\n", str);
	exit(EXIT_FAILURE);
}

int
main(void)
{
	// It is assumed that stdout is a pipe open for writing.
	//
	// Set stdout to non-blocking mode.
	int flags = fcntl(STDOUT_FILENO, F_GETFL);
	if (flags == -1)
		fail("Cannot get stdout file status flags");

	if (fcntl(STDOUT_FILENO, F_SETFL, flags | O_NONBLOCK) == -1)
		fail("Setting stdout to non-blocking mode failed");

	// Writing 64 KiB works as expected.
	static unsigned char buf[64 << 10];
	if (write(STDOUT_FILENO, buf, sizeof(buf)) != sizeof(buf))
		fail("The first write() failed");

	fprintf(stderr, "W: Wrote %zu bytes\n", sizeof(buf));

	// Writing more returns successfully but the reader won't see
	// these bytes. The reason appears to be the fcntl() below.
	const unsigned small_size = 5555;
	if (write(STDOUT_FILENO, buf, small_size) != (ssize_t)small_size)
		fail("The second write() failed");

	fprintf(stderr, "W: Wrote %u bytes\n", small_size);

	// Restore the original file status flags to stdout.
	// For some reason this makes the bytes from the last
	// write() disappear. If this is commented out, all
	// data is visible at the reader.
	if (fcntl(STDOUT_FILENO, F_SETFL, flags) == -1)
		fail("Restoring stdout file status flags failed");

	if (close(STDOUT_FILENO))
		fail("Error closing stdout");

	return EXIT_SUCCESS;
}


--MP_/CIXb11=_9CwfVfL8SY1E=8d
Content-Type: text/plain; charset=us-ascii

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
--MP_/CIXb11=_9CwfVfL8SY1E=8d--