X-Recipient:	archive-cygwin AT delorie DOT com
DomainKey-Signature:	a=rsa-sha1; c=nofws; d=sourceware.org; h=list-id
	:list-unsubscribe:list-subscribe:list-archive:list-post
	:list-help:sender:date:from:to:subject:message-id:reply-to
	:references:mime-version:content-type:in-reply-to; q=dns; s=
	default; b=aprm/fgbuUdCfGoX1XOaR3+czHWOAAnzzet1d/CKBxcR1Fk7qCFw4
	p1V/zG7D4vC3r+vvOkOSGmSG8iSyGdFizkb2JiaKqrWTDSu34nXWpc0IvjMN4CFy
	8kWhIu64mTInkUtBNJeEqI09U1qX/dREmwzfOrdilctKDvAYGgxbMo=
DKIM-Signature:	v=1; a=rsa-sha1; c=relaxed; d=sourceware.org; h=list-id
	:list-unsubscribe:list-subscribe:list-archive:list-post
	:list-help:sender:date:from:to:subject:message-id:reply-to
	:references:mime-version:content-type:in-reply-to; s=default;
	bh=Yxb9whyPzIXrrljzmyvjKnQ8OAw=; b=spMsjjmU8JvmjaRH8Lpi/qMp4T+7
	SCcEShpjJ2EhCisj9ZiiJw2ZRN4nYPMnR0xeZhEiKSfxedYmXwF9mX90OhIuFQ+Q
	ygFPwI3bqX6O06jCn06dq6GEyxgxJoBIgA9gqqYSoZ7MTSMZsA8nvlaO42AzgXo4
	13HZV3zuXfCzM4c=
Mailing-List:	contact cygwin-help AT cygwin DOT com; run by ezmlm
List-Id:	<cygwin.cygwin.com>
List-Subscribe:	<mailto:cygwin-subscribe AT cygwin DOT com>
List-Archive:	<http://sourceware.org/ml/cygwin/>
List-Post:	<mailto:cygwin AT cygwin DOT com>
List-Help:	<mailto:cygwin-help AT cygwin DOT com>, <http://sourceware.org/ml/#faqs>
Sender:	cygwin-owner AT cygwin DOT com
Mail-Followup-To:	cygwin AT cygwin DOT com
Delivered-To:	mailing list cygwin AT cygwin DOT com
Authentication-Results:	sourceware.org; auth=none
X-Virus-Found:	No
X-Spam-SWARE-Status:	No, score=-92.8 required=5.0 tests=AWL,BAYES_50,KAM_LAZY_DOMAIN_SECURITY,KHOP_DYNAMIC,RCVD_IN_BRBL_LASTEXT,RCVD_IN_PBL,RDNS_DYNAMIC,USER_IN_WHITELIST autolearn=no version=3.3.2 spammy=1.4, prudent, 9.7, opens
X-HELO:	calimero.vinschen.de
Date:	Mon, 25 Jan 2016 21:08:41 +0100
From:	Corinna Vinschen <corinna-cygwin AT cygwin DOT com>
To:	cygwin AT cygwin DOT com
Subject:	Re: Performance of "ls -F"
Message-ID:	<20160125200841.GB26077@calimero.vinschen.de>
Reply-To:	cygwin AT cygwin DOT com
Mail-Followup-To:	cygwin AT cygwin DOT com
References:	<loom DOT 20160121T163405-489 AT post DOT gmane DOT org> <CAH5rLZVWiGz2n-3Ru-C0_5KU-Cc4NipJPn1NQVphN2UetUK4iA AT mail DOT gmail DOT com> <3a9ff6ec2d5e4e64a96a8f8e31d12f22 AT ntmaexbe04 DOT bedford DOT progress DOT com> <loom DOT 20160122T082744-501 AT post DOT gmane DOT org> <20160122201418 DOT GC3268 AT calimero DOT vinschen DOT de> <8737tpqqb6 DOT fsf AT Rainer DOT invalid> <20160123173552 DOT GE3268 AT calimero DOT vinschen DOT de> <loom DOT 20160125T111636-839 AT post DOT gmane DOT org>
MIME-Version:	1.0
In-Reply-To:	<loom.20160125T111636-839@post.gmane.org>
User-Agent:	Mutt/1.5.24 (2015-08-30)

--zYM0uCDKw75PZbzx
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On Jan 25 11:02, Achim Gratz wrote:
> Corinna Vinschen <corinna-cygwin <at> cygwin.com> writes:
> > In the noacl case, Cygwin tries to find out if files are scripts.  It
> > opens the file and checks the first two bytes in the file for a shebang
> > (and other stuff).
>=20
> acl > time ls --file-type > /dev/null
> 0.015u 0.015s 0:00.42 4.7%      0+0k 0+0io 2473pf+0w
> acl > time ls -F > /dev/null
> 0.078u 0.858s 1:10.69 1.3%      0+0k 0+0io 5243pf+0w
>=20
> noacl > time ls --file-type > /dev/null
> 0.015u 0.015s 0:00.37 5.4%      0+0k 0+0io 2391pf+0w
> noacl > time ls -F > /dev/null
> 0.093u 1.327s 1:38.90 1.4%      0+0k 0+0io 6309pf+0w
>=20
> >  This may take a lot of time, more so on network
> > drives.  Can you try adding the "notexec" mount option to the "noacl"
> > share and see if that helps?
>=20
> acl,notexec > time ls --file-type > /dev/null
> 0.015u 0.030s 0:00.41 9.7%      0+0k 0+0io 2471pf+0w
> acl,noexec > time ls -F > /dev/null
> 0.062u 0.811s 1:10.31 1.2%      0+0k 0+0io 5240pf+0w
>=20
> noacl,notexec > time ls --file-type > /dev/null
> 0.031u 0.030s 0:00.41 14.6%     0+0k 0+0io 2389pf+0w
> noacl,notexec > time ls -F > /dev/null
> 0.046u 0.718s 0:56.23 1.3%      0+0k 0+0io 4994pf+0w
>=20
> > This test is done for a looong time to accommodate FAT filesystems in
> > the first place.  It might be prudent to disable it by default these
> > days...
>=20
> Looks like that's not the main reason for the extra time spent.

Off the top of my head, the common denominator seems to be that on
directories returning d_type information, --file-type doesn't have to
call stat().  That makes it very fast since only a directory enumeration
is done.  Calling stat on the other hand is time consuming since on
Windows it requires to open the file and read meta information (including
the ACL) or data (noacl exe recognition) from it.

> Here's another NetApp share, but this time there are about half as many
> files with only two of them in each sub-directory.
>=20
> (1046)/mnt/upload/install > time ls --file-type x86*/patches/*/* > /dev/n=
ull
> 0.155u 1.358s 0:09.42 15.9%     0+0k 0+0io 10555pf+0w
> (1047)/mnt/upload/install > time ls -F x86*/patches/*/* > /dev/null
> 0.109u 1.046s 0:08.20 13.9%     0+0k 0+0io 9817pf+0w
>=20
> Somehow that takes a lot less time and there's no difference between the =
two
> invocations (or actually a bit less time for -F).  The getVolInfo helper
> sees these two shares with the same settings.  Not sure what to make of t=
hat...

Hmm, no, sorry, I have no idea.  Somebody will have to debug this.


Corinna

--=20
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

--zYM0uCDKw75PZbzx
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQIcBAEBCAAGBQJWpoDJAAoJEPU2Bp2uRE+gwhcQAIRfNRKHsYaQDhAA9XQOBM1D
h/BEgiK5PgiWc5CCht6l2ys1+mSCN4bF9zPXh7O2JQjTOc6gX51yLZT2cMh1jznc
wyqd3hoeF0ZQAhfy+nfiewrkbCKFIaXYNAtXadIkV94mXWg06LDQSb3EvaJuILUq
niP34EVydmF9SW++WbgVqjTnurJvBnIj9RPsbOsUjkzpwpZzAgwu2NT7VJT12mz/
QtLk7Mh+DK7onMf8UOo8+bBzEwtCvBSnIEIlah5KOKMVAogAIhCEhwO7yupKUoD3
8TBdhzrglw1Kv+C+fWh2sJpc95bRFC2XhVTBPKZ+9Gha0PAtr9Yw8I5CG/5F1aM4
BjEZyNt35JAB+hGlH2P7v1aRjocZRMfIBrpZvF9cyrn45WWoDZBu9mjhmkH7f5TG
HCVdykcRz4nYXHuWVcMZ/Je37vvOwSHFzWltEuu7zBFuQ+rALkkjxYXXUmi8Xr58
ItuoMaigc+jIlMb7mvJ8PNGNXGkUkT4gZnGYTu1M/5tDTQmtoE0GHFG45n0KEAh7
GprTXfj5RUaUkttxjf8+lwTOkPsHQGUktZ8080pMLPon5433+k9VooiVRbkutVO6
hIGJklmOrFwFR5mumBLE7S13MpMOvObmx3CikiB/Wg/WW9X+af8frkElGboyiciU
oB+sZPBCNbh8+P2t2MA8
=XYUq
-----END PGP SIGNATURE-----

--zYM0uCDKw75PZbzx--

- Raw text -