X-Recipient:	archive-cygwin AT delorie DOT com
DomainKey-Signature:	a=rsa-sha1; c=nofws; d=sourceware.org; h=list-id
	:list-unsubscribe:list-subscribe:list-archive:list-post
	:list-help:sender:from:to:cc:subject:date:message-id
	:mime-version:content-type:content-transfer-encoding; q=dns; s=
	default; b=ACishpiH33A9CiBBQQX0Kwxp0k9j3kznJBJKOj7I18fLIgPDpWBRf
	h+fd8ycyKVHVILDxZpKMTID1SbDTcjcWuao7b+nm9pAhaN66rtbbAyFDknGQMgyZ
	HW+6pwj3GGWDKr+dW88/SgCVBTU8FMDNaubUmQlHUrGGBiMdMWPqwI=
DKIM-Signature:	v=1; a=rsa-sha1; c=relaxed; d=sourceware.org; h=list-id
	:list-unsubscribe:list-subscribe:list-archive:list-post
	:list-help:sender:from:to:cc:subject:date:message-id
	:mime-version:content-type:content-transfer-encoding; s=default;
	bh=zUn07d+q/ZhH7MAJhvnejPX7jR8=; b=f7yp+DCeHumWBN95tJbWbYAHr3KG
	9HQh/DCTMJWHcWQbV3xQM5q0HMrvdyJu6K4x5lLy/jsJwf4wDvLKNq9SfZoAZjNs
	xPv92LOrxDOFxMHZqXGkFyRo1cqKOtLHnNknbuz/7+OBTXOE3JlBoCCufpPByA/9
	CqZs+axgleFnBps=
Mailing-List:	contact cygwin-help AT cygwin DOT com; run by ezmlm
List-Id:	<cygwin.cygwin.com>
List-Subscribe:	<mailto:cygwin-subscribe AT cygwin DOT com>
List-Archive:	<http://sourceware.org/ml/cygwin/>
List-Post:	<mailto:cygwin AT cygwin DOT com>
List-Help:	<mailto:cygwin-help AT cygwin DOT com>, <http://sourceware.org/ml/#faqs>
Sender:	cygwin-owner AT cygwin DOT com
Mail-Followup-To:	cygwin AT cygwin DOT com
Delivered-To:	mailing list cygwin AT cygwin DOT com
Authentication-Results:	sourceware.org; auth=none
X-Spam-SWARE-Status:	No, score=0.1 required=5.0 tests=BAYES_00,HTML_MESSAGE,RCVD_IN_DNSWL_NONE,SPF_PASS,UNSUBSCRIBE_BODY autolearn=no version=3.3.2 spammy=GROUP, USE, ACCOUNTS, face
X-HELO:	smtprelay.synopsys.com
From:	Tom Honermann <thomas DOT honermann AT synopsys DOT com>
To:	"cygwin AT cygwin DOT com" <cygwin AT cygwin DOT com>
CC:	Cameron Gunnin <cameron DOT gunnin AT synopsys DOT com>,
	Marc-Andre Laverdiere <marc-andre DOT laverdiere AT synopsys DOT com>
Subject:	fcntl with O_APPEND fails to force append mode on stderr for native Windows programs
Date:	Fri, 8 Feb 2019 22:25:21 +0000
Message-ID:	<9E5DE55D2546134DACFE4D3C117B36FDAD0CEE1D@us01wembx1.internal.synopsys.com>
MIME-Version:	1.0
X-MIME-Autoconverted:	from quoted-printable to 8bit by delorie.com id x18MTrBK017713

This is a follow up to a bug originally reported as https://cygwin.com/ml/cygwin/2015-05/msg00140.html.  We continue to face this issue and recently spent some time trying to further isolate it.  We now have a better reproducer (below) that does not require GNU make.

From the original bug report, we bisected changes to GNU make and eventually identified that the reported problem was introduced by this change:

  *   http://git.savannah.gnu.org/cgit/make.git/commit/?id=9453a7451d66ee1ea5762a75161bf8a818c01a1f

The following program demonstrates the problem.

$ cat test.c
#include <fcntl.h>
#include <stdio.h>
#include <stdlib.h>

void set_append_mode(int fd) {
  int flags = fcntl(fd, F_GETFL, 0);
  if (flags >= 0)
    fcntl(fd, F_SETFL, flags | O_APPEND);
}

int main() {
  set_append_mode(fileno(stdout));
  set_append_mode(fileno(stderr));
  system("echo first > out");
  system("echo second >> out");
  system("net 2>> out");
}

The expected output of this program is that file 'out' have the content "first", followed by "second", followed by the error message emitted by the Windows 'net' utility (any native Windows program that writes to stderr should suffice to reproduce this issue).  And indeed, that is what is seen when the program is compiled and run with stdout and stderr set to the console.

$ gcc test.c -o test
$ ./test
$ cat out
first
second
The syntax of this command is:

NET
    [ ACCOUNTS | COMPUTER | CONFIG | CONTINUE | FILE | GROUP | HELP |
      HELPMSG | LOCALGROUP | PAUSE | SESSION | SHARE | START |
      STATISTICS | STOP | TIME | USE | USER | VIEW ]

Here is where things go bananas.  If the program is run with stdout initially redirected to a pipe, then the stderr output from the native Windows program ends up overwriting the stdout output from the two calls to echo:

$ ./test | cat
$ cat out
The syntax of this command is:

NET
    [ ACCOUNTS | COMPUTER | CONFIG | CONTINUE | FILE | GROUP | HELP |
      HELPMSG | LOCALGROUP | PAUSE | SESSION | SHARE | START |
      STATISTICS | STOP | TIME | USE | USER | VIEW ]

Note that the lines containing "first" and "second" are now absent in the file.

If the calls to set_append_mode are commented out, then the output is consistent in both cases.

The problem only occurs with native Windows programs that write to stderr.  If the above test is changed to append the stdout output from a native Windows program, then the output is consistent.

I've reproduced this problem on ancient versions of Cygwin such as 1.7.23 and recent versions such as 2.11.2.

Tom.

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

- Raw text -