X-Spam-Check-By: sourceware.org
To: cygwin AT cygwin DOT com
From: Eric Blake <ebb9 AT byu DOT net>
Subject:  Re: [Spam?]Re: problem concating (>>) to a large file
Date: Tue, 29 May 2007 14:42:40 +0000 (UTC)
Lines: 58
Message-ID:  <loom.20070529T161539-287@post.gmane.org>
References:  <001201c79ea3$4d82bce0$8500a8c0 AT RUSNAK> <4656D47E DOT 5050906 AT byu DOT net> <001201c79f49$b57b3c90$8500a8c0 AT RUSNAK> <46582F2B DOT 2000300 AT byu DOT net> <000001c7a1d8$c2178e40$8500a8c0 AT RUSNAK>
Mime-Version:  1.0
Content-Type:  text/plain; charset=us-ascii
Content-Transfer-Encoding:  7bit
User-Agent: Loom/3.14 (http://gmane.org/)
X-IsSubscribed: yes
Mailing-List: contact cygwin-help AT cygwin DOT com; run by ezmlm
List-Id: <cygwin.cygwin.com>
List-Subscribe: <mailto:cygwin-subscribe AT cygwin DOT com>
List-Archive: <http://sourceware.org/ml/cygwin/>
List-Post: <mailto:cygwin AT cygwin DOT com>
List-Help: <mailto:cygwin-help AT cygwin DOT com>, <http://sourceware.org/ml/#faqs>
Sender: cygwin-owner AT cygwin DOT com
Mail-Followup-To: cygwin AT cygwin DOT com
Delivered-To: mailing list cygwin AT cygwin DOT com

Peter Milne <peter.milne <at> netspeed.com.au> writes:

> 
> Hi Eric
> 
> Were you able to reproduce the problem I encountered with dd?

Yes - and it looks like there is indeed a cygwin bug, unrelated to my newlib 
patch to stdout.

$ cd /tmp
$ dd bs=1 seek=4540030013 if=/dev/zero of=huge count=1
$ ls -l huge blah
-rw-r--r-- 1 eblake Domain Users         15 May 29 08:27 blah
-rw-r--r-- 1 eblake Domain Users 4540030014 May 29 08:21 huge
$ cat foo.c
#include <stdio.h>
#include <stdlib.h>
#include <fcntl.h>
#include <unistd.h>

int main (int argc, char **argv)
{
   fprintf (stderr, "%lld\n", lseek (1, 0, SEEK_CUR));
   fprintf (stderr, "%x\n", fcntl (1, F_GETFL));
   return 0;
}
$ ./foo >> blah1
15
110209
$ ./foo >> huge
0
110209

Somehow, when the file size is huge, cygwin is not properly propogating that an 
O_APPEND bit (0x9 in the flags printed from F_GETFL) means that the initial 
offset of fd 1 is the end of the file.  (And it would be nice if strace would 
show the initial lseek offset of all inherited fd's when spawning a process).

You ALSO pointed out a bug in cat - it is calling freopen(NULL, "wb", stdout), 
which effectively wipes out whether stdout was previously in append mode; this 
is an upstream bug, but it only affects cygwin and other platforms where "wb" 
makes a difference.  I'll see about rolling out coreutils-6.9-3 that works 
around it, while posting a patch upstream to fix it.  For small files, you got 
lucky - even though the append mode was lost, the file position was already at 
the end of the file, and since cat doesn't seek, the output went to the right 
location.  But for large files, because of the cygwin bug where the file offset 
is wrong, the output is written to the wrong offset, even WITH the newlib 
stdout patch.

Three bugs in three different source repositories via one problem report (one 
in newlib, already fixed; one in cygwin, and one in coreutils)!  That's gotta 
be some sort of record.

-- 
Eric Blake
volunteer cygwin coreutils maintainer



--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/