X-Spam-Check-By: sourceware.org
Date: Tue, 24 Jul 2007 11:48:33 -0400
Message-Id: <200707241548.l6OFmXAE008571@linode.hodain.net>
To: cygwin AT cygwin DOT com
Subject: 1.5.24: incorrect default behavior of dd in popen context on text-mounted filesystem
From: Hugh Secker-Walker <hsw AT hodain DOT net>
MIME-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Content-Transfer-Encoding: 7bit
X-IsSubscribed: yes
Mailing-List: contact cygwin-help AT cygwin DOT com; run by ezmlm
List-Id: <cygwin.cygwin.com>
List-Subscribe: <mailto:cygwin-subscribe AT cygwin DOT com>
List-Archive: <http://sourceware.org/ml/cygwin/>
List-Post: <mailto:cygwin AT cygwin DOT com>
List-Help: <mailto:cygwin-help AT cygwin DOT com>, <http://sourceware.org/ml/#faqs>
Sender: cygwin-owner AT cygwin DOT com
Mail-Followup-To: cygwin AT cygwin DOT com
Delivered-To: mailing list cygwin AT cygwin DOT com

Firstly, a thank you to Eric Blake for his quick response to the dd
problem we uncovered yesterday.  As of coreutils 6.9-4, dd honors the
oflag=binary option in the popen() context in which I've been having
trouble.  This test case now works as expected:

$ cat popenbug.c
#include <stdio.h>
int main() {
  FILE * const out = popen("gzip | dd oflag=binary > popenbug.out.gz", "w");
  int i;
  for( i = 0; i < 25; ++i )
    fprintf(out, "line %d: * ANY EXPRESSED OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO  \n", i);
  fclose(out);
  return 0;
}
$
$ gcc popenbug.c -o popenbug.exe && rm -f popenbug.out.gz && ./popenbug.exe && sleep 1 && gzip -d -t popenbug.out.gz
0+1 records in
0+1 records out
172 bytes (172 B) copied, 0 s, Infinity B/s
$


However, in the discussion and the updated coreutils announcement
  http://cygwin.com/ml/cygwin/2007-07/msg00610.html
  http://cygwin.com/ml/cygwin/2007-07/msg00617.html
the behavior of dd is specified as defaulting to binary in the absence
of an [io]flag=text option.  This spec is in agreement with the Cygwin
documenation.  Bug, as of coreutils 6.9-4, this default behavior is
not honored in the popen() context:

$ cat popenbug.c
#include <stdio.h>
int main() {
  FILE * const out = popen("gzip | dd > popenbug.out.gz", "w");
  int i;
  for( i = 0; i < 25; ++i )
    fprintf(out, "line %d: * ANY EXPRESSED OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO  \n", i);
  fclose(out);
  return 0;
}
$ 
$ set|fgrep -i cygwin
BASH_VERSINFO=([0]="3" [1]="2" [2]="17" [3]="15" [4]="release" [5]="i686-pc-cygwin")
CYGWIN=binmode
MACHTYPE=i686-pc-cygwin
OSTYPE=cygwin
$ 
$ gcc popenbug.c -o popenbug.exe && rm -f popenbug.out.gz && ./popenbug.exe && sleep 1 && gzip -d -t popenbug.out.gz
0+1 records in
0+1 records out
172 bytes (172 B) copied, 0.015 s, 11.5 kB/s

gzip: popenbug.out.gz: invalid compressed data--crc error

gzip: popenbug.out.gz: invalid compressed data--length error
$

Notice that dd processes the same number of bytes in both cases, so
the problem is somewhere in the stdout handling when the oflag=binary
argument isn't provided....

Regards,
-Hugh

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/