delorie.com/archives/browse.cgi | search |
X-Recipient: | archive-cygwin AT delorie DOT com |
DomainKey-Signature: | a=rsa-sha1; c=nofws; d=sourceware.org; h=list-id |
:list-unsubscribe:list-subscribe:list-archive:list-post | |
:list-help:sender:date:from:to:subject:message-id:mime-version | |
:content-type; q=dns; s=default; b=XdjoKM3qxrS0Gq3ot3NEmalK+sT8f | |
3g0LPN/14pdpYjRVy431gNumbazrHrtsqj1lh9JvFlpOJNxw3sk+8Wf2cfLKOuIs | |
K9UD5XvLUJrbbd3FSrcF1Yx8Ufuek74IyJlPMDWeUVQcG2CliuhhJ28DP9GBO8mA | |
oO3ugei0Lzk0YQ= | |
DKIM-Signature: | v=1; a=rsa-sha1; c=relaxed; d=sourceware.org; h=list-id |
:list-unsubscribe:list-subscribe:list-archive:list-post | |
:list-help:sender:date:from:to:subject:message-id:mime-version | |
:content-type; s=default; bh=1dNDmkc+v3WWzuF1/Z9BKPURdcQ=; b=vXg | |
a/IXU5giF4iVwKT/GxQvXme19JCk4lVQSy91ajr1wd+DgqyK/T3AKRMsPTY11S6M | |
+Qu3JEm59RhY5OO6FGHdskjnmpZuevJa+GjgD8XUo2mUmXyvDzobo7bAv8bo0RHn | |
i1+GaH++0Lurt8gfAIhLwgPNQu91PhlO2dXYqn4A= | |
Mailing-List: | contact cygwin-help AT cygwin DOT com; run by ezmlm |
List-Id: | <cygwin.cygwin.com> |
List-Subscribe: | <mailto:cygwin-subscribe AT cygwin DOT com> |
List-Archive: | <http://sourceware.org/ml/cygwin/> |
List-Post: | <mailto:cygwin AT cygwin DOT com> |
List-Help: | <mailto:cygwin-help AT cygwin DOT com>, <http://sourceware.org/ml/#faqs> |
Sender: | cygwin-owner AT cygwin DOT com |
Mail-Followup-To: | cygwin AT cygwin DOT com |
Delivered-To: | mailing list cygwin AT cygwin DOT com |
Authentication-Results: | sourceware.org; auth=none |
X-Virus-Found: | No |
X-Spam-SWARE-Status: | No, score=-0.2 required=5.0 tests=AWL,BAYES_00,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.2 |
X-HELO: | mailscanner02.zoner.fi |
Date: | Wed, 18 Feb 2015 22:08:59 +0200 |
From: | Lasse Collin <lasse DOT collin AT tukaani DOT org> |
To: | cygwin AT cygwin DOT com |
Subject: | Clearing O_NONBLOCK from a pipe may lose data |
Message-ID: | <20150218220859.1e8f8b19@tukaani.org> |
MIME-Version: | 1.0 |
X-Antivirus-Scanner: | Clean mail though you should still use an Antivirus |
Received-SPF: | none |
--MP_/CIXb11=_9CwfVfL8SY1E=8d Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Content-Disposition: inline (Please Cc me when replying, I'm not subscribed to the list.) Hi! I suspect that there is a bug in Cygwin: 1. Create a pipe with both ends in blocking mode (O_NONBLOCK is not set). 2. The writer sets its end to non-blocking mode. 3. The writer writes to the pipe. 4. The writer restores its end of the pipe to blocking mode before the reader has read anything from the pipe. 5. The writer closes its end of the pipe. 6. The reader reads from the pipe in blocking mode. The last bytes written by the writer never appear at the reader, thus data is silently lost. Omitting the step 4 above makes the problem go away. A small test case is attached. I have tested it under Cygwin64 that was installed a few days ago on Windows 7. Usage: gcc -std=gnu99 -Wall -Wextra writer.c -o writer gcc -std=gnu99 -Wall -Wextra reader.c -o reader ./writer | ./reader The reader only gets the first 65536 bytes. The remaining 5555 bytes are never seen by the reader even though all write() calls succeed at the writer. I found this problem because I had heard a rumor that there is something wrong with xz 5.2.0 on Cygwin. Since the file I/O code had been changed in 5.2.0, I investigated that first. There also seems to be something wrong with xz 5.2.0 in threaded mode under Cygwin which needs more investigation (random pthread_join() failures cause assertion failures if xz was built with --enable-debug). -- Lasse Collin | IRC: Larhzu @ IRCnet & Freenode --MP_/CIXb11=_9CwfVfL8SY1E=8d Content-Type: text/x-c++src Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=reader.c // gcc -std=gnu99 -Wall -Wextra reader.c -o reader #include <sys/types.h> #include <sys/stat.h> #include <fcntl.h> #include <unistd.h> #include <stdlib.h> #include <stdio.h> static void fail(const char *str) { fprintf(stderr, "R: %s\n", str); exit(EXIT_FAILURE); } int main(void) { // It is assumed that stdin is a pipe open for reading. // // Read 8 KiB at a time. static unsigned char buf[8 << 10]; // Total amount read unsigned total = 0; for (unsigned i = 0; i < 10; ++i) { fprintf(stderr, "R: Sleeping\n"); sleep(1); int amount = read(STDIN_FILENO, buf, sizeof(buf)); fprintf(stderr, "R: read() returned %d\n", amount); if (amount >= 0) { total += amount; fprintf(stderr, "R: Total amount read is %u\n", total); } } if (close(STDIN_FILENO)) fail("Error closing stdin"); return EXIT_SUCCESS; } --MP_/CIXb11=_9CwfVfL8SY1E=8d Content-Type: text/x-c++src Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=writer.c // gcc -std=gnu99 -Wall -Wextra writer.c -o writer #include <sys/types.h> #include <sys/stat.h> #include <fcntl.h> #include <unistd.h> #include <stdlib.h> #include <stdio.h> static void fail(const char *str) { fprintf(stderr, "W: %s\n", str); exit(EXIT_FAILURE); } int main(void) { // It is assumed that stdout is a pipe open for writing. // // Set stdout to non-blocking mode. int flags = fcntl(STDOUT_FILENO, F_GETFL); if (flags == -1) fail("Cannot get stdout file status flags"); if (fcntl(STDOUT_FILENO, F_SETFL, flags | O_NONBLOCK) == -1) fail("Setting stdout to non-blocking mode failed"); // Writing 64 KiB works as expected. static unsigned char buf[64 << 10]; if (write(STDOUT_FILENO, buf, sizeof(buf)) != sizeof(buf)) fail("The first write() failed"); fprintf(stderr, "W: Wrote %zu bytes\n", sizeof(buf)); // Writing more returns successfully but the reader won't see // these bytes. The reason appears to be the fcntl() below. const unsigned small_size = 5555; if (write(STDOUT_FILENO, buf, small_size) != (ssize_t)small_size) fail("The second write() failed"); fprintf(stderr, "W: Wrote %u bytes\n", small_size); // Restore the original file status flags to stdout. // For some reason this makes the bytes from the last // write() disappear. If this is commented out, all // data is visible at the reader. if (fcntl(STDOUT_FILENO, F_SETFL, flags) == -1) fail("Restoring stdout file status flags failed"); if (close(STDOUT_FILENO)) fail("Error closing stdout"); return EXIT_SUCCESS; } --MP_/CIXb11=_9CwfVfL8SY1E=8d Content-Type: text/plain; charset=us-ascii -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple --MP_/CIXb11=_9CwfVfL8SY1E=8d--
webmaster | delorie software privacy |
Copyright © 2019 by DJ Delorie | Updated Jul 2019 |