Date:	Wed, 8 Aug 2001 10:37:29 -0700 (PDT)
From:	Paul Eggert <eggert AT twinsun DOT com>
Message-Id:	<200108081737.f78HbTf00873@shade.twinsun.com>
To:	eliz AT is DOT elta DOT co DOT il
CC:	rich AT phekda DOT freeserve DOT co DOT uk, djgpp-workers AT delorie DOT com
In-reply-to:	<Pine.SUN.3.91.1010808174034.27069A-100000@is>
	(eliz AT is DOT elta DOT co DOT il)
Subject:	Re: GNU ls bug on DJGPP with large files
References:	<Pine DOT SUN DOT 3 DOT 91 DOT 1010808174034 DOT 27069A-100000 AT is>
Reply-To:	djgpp-workers AT delorie DOT com
Errors-To:	nobody AT delorie DOT com
X-Mailing-List:	djgpp-workers AT delorie DOT com
X-Unsubscribes-To:	listserv AT delorie DOT com

> Date: Wed, 8 Aug 2001 17:47:06 +0300 (IDT)
> From: Eli Zaretskii <eliz AT is DOT elta DOT co DOT il>
> 
> What can we do? DJGPP doesn't include the OS, it uses the system calls 
> available from the underlying OS.  There simply isn't a system call that 
> can support files larger than 4GB - 2 bytes.  So we cannot support larger 
> files.

OK, but in the short term if DJGPP cannot support large files, then it
should say so honestly.  For example, its emulation of the 'stat'
system call should return -1 on files larger than 2GB, and should set
errno to EOVERFLOW in that case.  That is what POSIX 1003.1-200x draft
7 requires, and it is what GNU applications expect.  Similarly for
'fstat', 'lseek', 'read', and any other system call that deals with or
modifies file offsets.

Another short-term possibility is to grow off_t to 64 bits, and to
return EOVERFLOW for file offsets larger than 4 GB.  That is better by
a factor of 2, since it supports files up to 4 GB in size.

In the longer term, though, DJGPP may well not survive if it can't
support files larger than 4 GB.  I realize that it will be a hassle to
fix the problem, and I also realize that I'm no expert in fixing it:
all I'm saying is that it is a real problem that will get worse with
time.

> > The patch doesn't fix the problem for
> > gzip, tar, find, or any of the dozens of other programs that have to
> > deal with large files in normal use.
> 
> It's possible that more programs are affected, but each case should be 
> examined individually.  The only programs which will have problems like 
> `ls' are those which print the value of st_size.

No, it's not just printing the value.  For example, the -size option
of 'find' looks for files larger than a given size.  tar needs to copy
the file into a tar archive, and the archive contains the size in the
archive header.  Lots more programs are affected, sometimes in
mysterious ways.

We cannot go through every application fixing it to support
nonstandard implementations like this.  (`ls' is an exception, for
reasons discussed earlier.)  The right way to fix this problem is at
the lower level, either by growing off_t to 64 bits, or by returning
EOVERFLOW, or both.

- Raw text -