Mailing-List: contact cygwin-help AT cygwin DOT com; run by ezmlm
List-Subscribe: <mailto:cygwin-subscribe AT cygwin DOT com>
List-Archive: <http://sources.redhat.com/ml/cygwin/>
List-Post: <mailto:cygwin AT cygwin DOT com>
List-Help: <mailto:cygwin-help AT cygwin DOT com>, <http://sources.redhat.com/ml/#faqs>
Sender: cygwin-owner AT cygwin DOT com
Mail-Followup-To: cygwin AT cygwin DOT com
Delivered-To: mailing list cygwin AT cygwin DOT com
X-Injected-Via-Gmane: http://gmane.org/
To: cygwin AT cygwin DOT com
From: "Markus Mauhart" <markus DOT mauhart AT chello DOT at>
Subject: Re: Sparse file criteria malfunction - binutils produces sparse .exe & .dll files
Date: Wed, 4 Jun 2003 17:33:46 +0200
Lines: 71
Message-ID: <bbl3cu$ph4$1@main.gmane.org>
References: <004201c32902$3e92b020$78d96f83 AT pomello> <20030602133202 DOT GC30498 AT redhat DOT com>
X-Complaints-To: usenet AT main DOT gmane DOT org
X-MSMail-Priority: Normal
X-Newsreader: Microsoft Outlook Express 6.00.2800.1158
X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1165

"Christopher Faylor" <cgf-rcm AT cygwin DOT com> wrote ...
>
> On Mon, Jun 02, 2003 at 01:26:50PM +0100, Max Bowsher wrote:
> >$ uname -svr
> >CYGWIN_NT-5.1 1.5.0(0.86/3/2) 2003-06-02 00:41
> >
> >The new sparse file heuristic is being triggered by the way binutils writes
> >.exe and .dll files.
> >
> >I'm unsure this could be worked around. Any ideas?
>
> Since you are the principal complainer about this particular feature,

IMHO Max is the inofficial spokesman of a suffering but silent majority.
We others are speachless when even thinking about people who confuse
the 'traditional unix feature' called "sparse files" with an advanced
special purpose filesystem feature like "NTFS sparse files", where the
people responsible for it deliberatly decided NOT to let it be the default
for any directory or even volume (unlike e.g. compression, which can be set
as default for dirs and volumes).

IIRC it was Vaclav Haisman who in the original discussion in cygwin-patches
gathered and presented enough facts to show the principle difference between
both concepts:
  "traditional unix sparse files" other than "NTFS sparse files" does
   not support unallocated holes inside files, and hence it is cheap.
Nevertheless he and the few (but prominent, well-deserved !-) other
proponents of the cygwin-sparse-default basically ignored this facts
but said "we have ONE [later TWO, IIRC] real case where it is an advantage,
now show us opposite examples".
Now it has been proven that each cygwin installation manipulating 1000s of
files only suffers from this "feature"; not to mention that it breaks
winfile.exe :-(((.

Btw, with XP, W2003 and later, probably there is now a windows feature
that could be the real equivalent of "traditional unix sparse files":
SetFileValidData(..).
Its description is a bit mysterious. My interpretation is the following:

1. Having any file with lenght L.
2. SetEndOfFile(L+1M) --> content of 1st L unchanged, file size is L+1M,
   file valid length is still unchanged ( <= L).
If you now write/read at L+x, you force the OS/FS to convert the range
[L,L+x] into valid data by allocating zeroed clusters for the hole.
If instead before the read/write-access you call SetFileValidData(L+1M),
then valid length extends to L+M and the gap is filled with allocated
but not necessary zeroed clusters.
No need to generate 1M zero clusters !-)))
But a potential security problem when reusing freed but not overwritten
clusters of files with other permissions - how do 'traditional unix' OSes
implement this ? From Vaclav's postings my conclusion is that they too
reuse foreign clusters, cause IIRC he said that traditional unixe OS/FS
(unlike NTFS sparse files) dont support unallocated holes inside files.
IMHO because of this theoretic security problem with XP/W2003 the docu says
     "The caller must have the SE_MANAGE_VOLUME_NAME privilege"

If this (my) interpretation of SetFileValidData() is correct, wouldnt
then SetFileValidData() do exactly what people can expect from
'traditional unix sparse files' (except XP beeing secure) ?

Otherwise, has anyone of the cygwin-is-sparse proponents the information
whether 'traditional unix sparse files' support unallocated holes inside
files ? If yes, since when ? If no, how is the security of reusing other
peoples not-overwritten clusters handled ?


Thanks,
Markus.





--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/