Mailing-List: contact cygwin-help AT cygwin DOT com; run by ezmlm
List-Subscribe: <mailto:cygwin-subscribe AT cygwin DOT com>
List-Archive: <http://sources.redhat.com/ml/cygwin/>
List-Post: <mailto:cygwin AT cygwin DOT com>
List-Help: <mailto:cygwin-help AT cygwin DOT com>, <http://sources.redhat.com/ml/#faqs>
Sender: cygwin-owner AT cygwin DOT com
Mail-Followup-To: cygwin AT cygwin DOT com
Delivered-To: mailing list cygwin AT cygwin DOT com
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Message-ID: <16072.9114.788880.677994@gargle.gargle.HOWL>
Date: Sun, 18 May 2003 17:21:46 -0700
From: Martin Buchholz <martin AT xemacs DOT org>
To: "Max Bowsher" <maxb AT ukf DOT net>
Cc: <cygwin AT cygwin DOT com>
Subject: Re: SPARSE files considered harmful - please revert
In-Reply-To: <003701c31d98$7a9649d0$78d96f83@pomello>
References: <16072 DOT 892 DOT 778395 DOT 24290 AT gargle DOT gargle DOT HOWL>
	<003901c31d8c$6ec495f0$78d96f83 AT pomello>
	<16072 DOT 6666 DOT 10124 DOT 338022 AT gargle DOT gargle DOT HOWL>
	<003701c31d98$7a9649d0$78d96f83 AT pomello>
Reply-To: martin AT xemacs DOT org

>>>>> "Max" == Max Bowsher <maxb AT ukf DOT net> writes:

Max> Martin Buchholz wrote:
>>>>>>> "Max" == Max Bowsher <maxb AT ukf DOT net> writes:
>> 
Max> May I suggest a middle road? Why not let sparse files be configurable
>> as a Max> $CYGWIN option? This would allow those users who actually want
>> them to Max> enable them with minimal effort, but keep them off for most
>> users.
>> 
>> I suspect that SPARSE files are genuinely useful, when storing large
>> files that have holes in them.  But I can't imagine one ever wanting
>> to use SPARSE for all files, because most files aren't like that.  So
>> I don't think sparseness is a good candidate for being put into
>> $CYGWIN.

Max> Agreed. I was just trying to find some simple compromise. Have you reviewed
Max> the long conversation that went on in cygwin-patches in February? Based on
Max> the ease with which this patch was accepted, I'm conjecturing that the core
Max> developers won't want a simple reversion.

Certainly using $CYGWIN is a solution that works.  When you want to
create your sparse file, you do

env CYGWIN="$CYGWIN sparse" make-hole

But it's not very elegant.

>> We could have a much cleverer implementation of sparseness, if we kept
>> statistics on the number and size of zero bytes in a file while it was
>> being written.  When we did the close(), we could automatically
>> transform it into a sparse file.  But I don't think even that should
>> be the default behavior, because it would make all IO slower.

Max> And it wouldn't achieve Vaclav Heisman's original goal, either - he wanted
Max> to avoid the delay caused by Windows zero-filling a file when it was
Max> initially writted to at a large offset.

I see.

Here's another idea...

Whenever a program does a seek on a file descriptor that has O_CREAT set,
we can check if the new position is more than, say 32kb from the current
position.  If so, we can call DeviceIoControl(FSCTL_SET_SPARSE).
This is cheap to implement and might just make everyone happy.
If it works.

Martin

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/