Mailing-List: contact cygwin-help AT cygwin DOT com; run by ezmlm
List-Subscribe: <mailto:cygwin-subscribe AT cygwin DOT com>
List-Archive: <http://sourceware.org/ml/cygwin/>
List-Post: <mailto:cygwin AT cygwin DOT com>
List-Help: <mailto:cygwin-help AT cygwin DOT com>, <http://sourceware.org/ml/#faqs>
Sender: cygwin-owner AT cygwin DOT com
Mail-Followup-To: cygwin AT cygwin DOT com
Delivered-To: mailing list cygwin AT cygwin DOT com
Reply-To: Cygwin List <cygwin AT cygwin DOT com>
Message-Id: <6.2.0.14.0.20050201144916.05a42210@pop.prospeed.net>
Date: Tue, 01 Feb 2005 14:51:18 -0500
To: Ken Sheldon <Ken_Sheldon AT ACM DOT org>, cygwin AT cygwin DOT com
From: Larry Hall <lh-no-personal-replies-please AT cygwin DOT com>
Subject: Re: slow handling of large sets of files?
In-Reply-To: <1107286130.3063.65.camel@slb163188094105.sugar-land.nam.sl b.com>
References: <1107286130 DOT 3063 DOT 65 DOT camel AT slb163188094105 DOT sugar-land DOT nam DOT slb DOT com>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"

At 02:28 PM 2/1/2005, you wrote:
>I too have been seeing a problem with very slow file access in large
>directories.
>
>Specifically,
>
>On a Cygwin/Win2k box, I have a mirror of an FTP site.  The site has 2.5
>million files spread between 100 directories.  
>(20,000 - 30,000 files per directory)  I have previously run this number
>of files in an NT4 NTFS filesystem without significant performance
>problems.
>
>In this site, operations like these are __VERY__ slow.
>ls ${some_dir}
>ls ${some_dir}/${some_path}
>cp ${some_file} ${some_path}
>cp -R ${some_path_with _only_a_few_files} ${some_path}
>
>
>If I look at the performance monitor, I can see a queue depth of 1-2 and
>300-500 disk reads per second.  (That's real.  It's a fast array)  The
>reads appear to be single-block reads, as the throughput during these
>events is 1.5 - 3MB/sec.
>
>I am beginning to think the disk activity relates to NTFS permission
>checking, which can be complex under Win2k.
>
>I don't know how to debug or tune this.
>
>Any ideas?


Have you looked at the '-x', '-X', and '-E' flags of mount?


--
Larry Hall                              http://www.rfk.com
RFK Partners, Inc.                      (508) 893-9779 - RFK Office
838 Washington Street                   (508) 893-9889 - FAX
Holliston, MA 01746                     


--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/