Mailing-List: contact cygwin-help AT cygwin DOT com; run by ezmlm
List-Subscribe: <mailto:cygwin-subscribe AT cygwin DOT com>
List-Archive: <http://sourceware.org/ml/cygwin/>
List-Post: <mailto:cygwin AT cygwin DOT com>
List-Help: <mailto:cygwin-help AT cygwin DOT com>, <http://sourceware.org/ml/#faqs>
Sender: cygwin-owner AT cygwin DOT com
Mail-Followup-To: cygwin AT cygwin DOT com
Delivered-To: mailing list cygwin AT cygwin DOT com
Message-ID: <015d01c508c3$9b384bd0$05010e0a@2mbit.com>
From: "Brian Bruns" <bruns AT 2mbit DOT com>
To: <cygwin AT cygwin DOT com>
References: <1107286130 DOT 3063 DOT 65 DOT camel AT slb163188094105 DOT sugar-land DOT nam DOT slb DOT com> <6 DOT 2 DOT 0 DOT 14 DOT 0 DOT 20050201144916 DOT 05a42210 AT pop DOT prospeed DOT net> <1107289362 DOT 3063 DOT 78 DOT camel AT slb163188094105 DOT sugar-land DOT nam DOT slb DOT com> <4200247C DOT 6F51F5E AT dessent DOT net>
Subject: Re: slow handling of large sets of files?
Date: Tue, 1 Feb 2005 20:07:43 -0500
MIME-Version: 1.0
Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: 7bit
X-Scan-Signature: 12165b451e859f13a791ef2235c53444
X-SA-Exim-Connect-IP: 138.89.133.88
X-SA-Exim-Mail-From: bruns AT 2mbit DOT com
X-IsSubscribed: yes

On Tuesday, February 01, 2005 7:53 PM [EST], Brian Dessent wrote:

> Ken Sheldon wrote:
>
>> Not only Cygwin apps incur this large performance penalty.
>> Something similar happens with the cmd.exe prompt command "DIR",
>> with the windows file explorer, or with IIS (FTP server).  This
>> only seems to happen in the directory structures created by my
>> CygWin scripts (using apps: tar, wget, cp)
>
> If I had to take a wild guess I'd say that's because when a normal
> win32 app creates a file it usually inherits its ACL from the parent
> directory, and presumably NTFS is tuned for this in some way so that
> checking thousands of such files that all inherit ACLs is fast.
> However, when cygwin creates a file or dir it usually sets the
> permissions explicitly to match what you would expect on a POSIX
> system (i.e. 644 or 755.)  Try creating your files/folders with
> "nontsec" set and see if the cygwin-created trees are as slow as
> native-created ones.
>
> Brian

NTFS in general just isn't very fast when you really start using some
of the advanced features.  Its just not tuned very well for large
amounts of files.  A perfect example of this is when you need to
delete alot of files and directories all at once - it takes an awfully
long time to do when compared to ext2/3 or reiserfs, and I've noticed
that sometimes the system doesn't properly clean up the directory
indexes, leaving stale ACL entries and such (run chkdsk, boot, do a
bunch of filesystem operations such as deleting files, then run chkdsk
again, and most likely, it will report even more errors).

MFT fragmentation seems to compound already existing problems.

-- 
Brian Bruns
The Summit Open Source Development Group
http://www.sosdg.org  /  http://www.ahbl.org


--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/