Mailing-List: contact cygwin-help@cygwin.com; run by ezmlm
Sender: cygwin-owner@cygwin.com
Mail-Followup-To: cygwin@cygwin.com
Message-ID: <015d01c508c3$9b384bd0$05010e0a@2mbit.com>
From: "Brian Bruns" <bruns@2mbit.com>
To: <cygwin@cygwin.com>
References: <1107286130.3063.65.camel@slb163188094105.sugar-land.nam.slb.com> <6.2.0.14.0.20050201144916.05a42210@pop.prospeed.net> <1107289362.3063.78.camel@slb163188094105.sugar-land.nam.slb.com> <4200247C.6F51F5E@dessent.net>
Subject: Re: slow handling of large sets of files?
Date: Tue, 1 Feb 2005 20:07:43 -0500
MIME-Version: 1.0
Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: 7bit

On Tuesday, February 01, 2005 7:53 PM [EST], Brian Dessent wrote:

> Ken Sheldon wrote:
>
>> Not only Cygwin apps incur this large performance penalty.
>> Something similar happens with the cmd.exe prompt command "DIR",
>> with the windows file explorer, or with IIS (FTP server).  This
>> only seems to happen in the directory structures created by my
>> CygWin scripts (using apps: tar, wget, cp)
>
> If I had to take a wild guess I'd say that's because when a normal
> win32 app creates a file it usually inherits its ACL from the parent
> directory, and presumably NTFS is tuned for this in some way so that
> checking thousands of such files that all inherit ACLs is fast.
> However, when cygwin creates a file or dir it usually sets the
> permissions explicitly to match what you would expect on a POSIX
> system (i.e. 644 or 755.)  Try creating your files/folders with
> "nontsec" set and see if the cygwin-created trees are as slow as
> native-created ones.
>
> Brian

NTFS in general just isn't very fast when you really start using some
of the advanced features.  Its just not tuned very well for large
amounts of files.  A perfect example of this is when you need to
delete alot of files and directories all at once - it takes an awfully
long time to do when compared to ext2/3 or reiserfs, and I've noticed
that sometimes the system doesn't properly clean up the directory
indexes, leaving stale ACL entries and such (run chkdsk, boot, do a
bunch of filesystem operations such as deleting files, then run chkdsk
again, and most likely, it will report even more errors).

MFT fragmentation seems to compound already existing problems.

-- 
Brian Bruns
The Summit Open Source Development Group
http://www.sosdg.org  /  http://www.ahbl.org


--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/