X-Spam-Check-By: sourceware.org To: cygwin AT cygwin DOT com From: Eric Blake Subject: Re: [Spam?]Re: problem concating (>>) to a large file Date: Tue, 29 May 2007 14:42:40 +0000 (UTC) Lines: 58 Message-ID: References: <001201c79ea3$4d82bce0$8500a8c0 AT RUSNAK> <4656D47E DOT 5050906 AT byu DOT net> <001201c79f49$b57b3c90$8500a8c0 AT RUSNAK> <46582F2B DOT 2000300 AT byu DOT net> <000001c7a1d8$c2178e40$8500a8c0 AT RUSNAK> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit User-Agent: Loom/3.14 (http://gmane.org/) X-IsSubscribed: yes Mailing-List: contact cygwin-help AT cygwin DOT com; run by ezmlm List-Id: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: cygwin-owner AT cygwin DOT com Mail-Followup-To: cygwin AT cygwin DOT com Delivered-To: mailing list cygwin AT cygwin DOT com Peter Milne netspeed.com.au> writes: > > Hi Eric > > Were you able to reproduce the problem I encountered with dd? Yes - and it looks like there is indeed a cygwin bug, unrelated to my newlib patch to stdout. $ cd /tmp $ dd bs=1 seek=4540030013 if=/dev/zero of=huge count=1 $ ls -l huge blah -rw-r--r-- 1 eblake Domain Users 15 May 29 08:27 blah -rw-r--r-- 1 eblake Domain Users 4540030014 May 29 08:21 huge $ cat foo.c #include #include #include #include int main (int argc, char **argv) { fprintf (stderr, "%lld\n", lseek (1, 0, SEEK_CUR)); fprintf (stderr, "%x\n", fcntl (1, F_GETFL)); return 0; } $ ./foo >> blah1 15 110209 $ ./foo >> huge 0 110209 Somehow, when the file size is huge, cygwin is not properly propogating that an O_APPEND bit (0x9 in the flags printed from F_GETFL) means that the initial offset of fd 1 is the end of the file. (And it would be nice if strace would show the initial lseek offset of all inherited fd's when spawning a process). You ALSO pointed out a bug in cat - it is calling freopen(NULL, "wb", stdout), which effectively wipes out whether stdout was previously in append mode; this is an upstream bug, but it only affects cygwin and other platforms where "wb" makes a difference. I'll see about rolling out coreutils-6.9-3 that works around it, while posting a patch upstream to fix it. For small files, you got lucky - even though the append mode was lost, the file position was already at the end of the file, and since cat doesn't seek, the output went to the right location. But for large files, because of the cygwin bug where the file offset is wrong, the output is written to the wrong offset, even WITH the newlib stdout patch. Three bugs in three different source repositories via one problem report (one in newlib, already fixed; one in cygwin, and one in coreutils)! That's gotta be some sort of record. -- Eric Blake volunteer cygwin coreutils maintainer -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/