delorie.com/archives/browse.cgi   search  
Mail Archives: cygwin/2010/09/08/11:25:24

X-Recipient: archive-cygwin AT delorie DOT com
X-SWARE-Spam-Status: No, hits=-1.8 required=5.0 tests=AWL,BAYES_00,TW_YW
X-Spam-Check-By: sourceware.org
X-Yahoo-SMTP: Uu383n6swBCEN1G9up0WSnxbvN8fCPmk
Message-ID: <4C87AACB.6050805@cygwin.com>
Date: Wed, 08 Sep 2010 11:24:59 -0400
From: "Larry Hall \(Cygwin\)" <reply-to-list-only-lh AT cygwin DOT com>
Reply-To: cygwin AT cygwin DOT com
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.21) Gecko/20090320 Remi/2.0.0.21-1.fc8.remi Lightning/0.9 Thunderbird/2.0.0.21 Mnenhy/0.7.5.0
MIME-Version: 1.0
To: cygwin AT cygwin DOT com
Subject: Re: incredibly slow file listing script on windoze 7 pro 4 core 64 bit
References: <AANLkTi=ASzGNwU9TSKU2TZSbAksy6=Pu94=0S5Vt0Sc- AT mail DOT gmail DOT com> <4C86867D DOT 4050509 AT cygwin DOT com> <AANLkTi=rN9OA4Qp0eCbiWN0Kt_pvjDwDVfOERBzR1iCZ AT mail DOT gmail DOT com>
In-Reply-To: <AANLkTi=rN9OA4Qp0eCbiWN0Kt_pvjDwDVfOERBzR1iCZ@mail.gmail.com>
Mailing-List: contact cygwin-help AT cygwin DOT com; run by ezmlm
List-Id: <cygwin.cygwin.com>
List-Subscribe: <mailto:cygwin-subscribe AT cygwin DOT com>
List-Archive: <http://sourceware.org/ml/cygwin/>
List-Post: <mailto:cygwin AT cygwin DOT com>
List-Help: <mailto:cygwin-help AT cygwin DOT com>, <http://sourceware.org/ml/#faqs>
Sender: cygwin-owner AT cygwin DOT com
Mail-Followup-To: cygwin AT cygwin DOT com
Delivered-To: mailing list cygwin AT cygwin DOT com

On 9/8/2010 9:35 AM, mike marchywka wrote:
> On 9/7/10, Larry Hall (Cygwin)<reply-to-list-only-lh AT xxx DOT com>  wrote:
>> On 9/7/2010 12:05 PM, mike marchywka wrote:
>>> this takes a few minutes on old debian machine, taking much longer
>>> here on same data-
>>> about 19k file about 24Gb total size.
>>> Windoze finally has better perf stuff but still no help- one core at
>>> 25 pct all kernel
>>> time disk not exactly busy. All the time is in the "ls" loop not the
>>> find command.
>>> Now obbviously I expect the "ls" per file has to make a bunch of OS calls
>>> for
>>> each file but still even with cygwin layer seems a bit much.
>>> Thanks.
>>
>> The problem is that Cygwin has to open all the files to stat when you
>> use the '-l' flag.  Try without the flag to get the lower bound and see if
>> that's better for you.  If so, maybe '-s' will suit your purpose.  Or
>> even 'du'?
>
> Sure, I could find a better way to do it but I was just curious about
> performance diffs. I've got this expensive 'doze laptop with 4 cores and the
> latest OS and all and compared to this old emachines desktop with debian it
> seems a bit slow. Now maybe this is a bit of a contrived case, no one would
> really do this and afterall the new system may be optimized for real-life
> usages and of course cygwin can't speed up obligatory OS calls.

To somewhat sooth your curiousity, Windows (or perhaps it's more accurate
to say NTFS) ain't great with directories with a large number of files.
I expect you would be less than impressed with the performance of of 'dir'
in 'cmd.exe' in the same directory.  That said, you're asking for allot more
work than you may realize when doing the same thing in Cygwin.  In order to
fill in the information you ask for when you use the '-l' flag for 'ls',
Cygwin needs to open and close the files, which takes a good chunk of time
en masse.  The same does not need to happen in Linux/UNIX-land.  So what
you're looking at here, above and beyond the design decisions of Windows,
is some limitations of the emulated environment.  So you can now take some
joy in the knowledge that you have a spanking new system to run
Windows/Cygwin on rather than an old eMachines box. ;-)

> Anyway yes du seems much faster and is quite reasonable :)

Glad it helps.

-- 
Larry Hall                              http://www.rfk.com
RFK Partners, Inc.                      (508) 893-9779 - RFK Office
216 Dalton Rd.                          (508) 893-9889 - FAX
Holliston, MA 01746

_____________________________________________________________________

A: Yes.
> Q: Are you sure?
>> A: Because it reverses the logical flow of conversation.
>>> Q: Why is top posting annoying in email?

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

- Raw text -


  webmaster     delorie software   privacy  
  Copyright © 2019   by DJ Delorie     Updated Jul 2019