DomainKey-Signature: a=rsa-sha1; c=nofws; d=sourceware.org; h=list-id
	:list-unsubscribe:list-subscribe:list-archive:list-post
	:list-help:sender:date:from:to:cc:subject:message-id:reply-to
	:references:mime-version:content-type:in-reply-to; q=dns; s=
	default; b=urmX/yZLbB72zPRK7Vc3sBEwdxfvLNHCymJpR7PPkfUFgEg2+dQq6
	ewNWeq7BjlltG4hYXKDbzHLiUqF5J43aV/2Ka8Xk6oMcDQnOi+hMG+rjjIJK89n/
	WH46JpPAnQ6QSQhHcBa+PPYjyCPDF3C7XJo3m2ihU7Od0pLisrs0Mg=
Mailing-List: contact cygwin-help AT cygwin DOT com; run by ezmlm
Sender: cygwin-owner AT cygwin DOT com
Mail-Followup-To: cygwin AT cygwin DOT com
Date: Wed, 25 Feb 2015 13:03:41 +0100
From: Corinna Vinschen <corinna-cygwin AT cygwin DOT com>
To: cygwin AT cygwin DOT com
Cc: John Hein <3fbmqnhaz4 AT snkmail DOT com>
Subject: Re: TEST RELEASE: Cygwin 1.7.35-0.3
Message-ID: <20150225120341.GZ437@calimero.vinschen.de>
Reply-To: cygwin AT cygwin DOT com
Mail-Followup-To: cygwin AT cygwin DOT com, John Hein <3fbmqnhaz4 AT snkmail DOT com>
References: <20150220102927 DOT GR26084 AT calimero DOT vinschen DOT de> <6695-1424465180-282140 AT sneakemail DOT com> <20150223111721 DOT GH437 AT calimero DOT vinschen DOT de> <9116-1424712152-918733 AT sneakemail DOT com>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha1;	protocol="application/pgp-signature"; boundary="95xjieIIiUQImiTX"
Content-Disposition: inline
In-Reply-To: <9116-1424712152-918733@sneakemail.com>
User-Agent: Mutt/1.5.23 (2014-03-12)

--95xjieIIiUQImiTX
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

Hi John,

On Feb 23 10:22, John Hein wrote:
> Corinna Vinschen corinna-cygwin-at-cygwin.com |cygwin_ml_nodigest| wrote =
at 12:17 +0100 on Feb 23, 2015:
>  > 1.7.33:
>  >
>  >   Calls NetUserEnum/NetGroupEnum,NetLocalGroupEnum with maximum Buffer
>  >   size.
>  >
>  > 1.7.34+:
>  >
>  >   Calls an LDAP enumerator fetching 100 SIDs per call.
>  >   For each SID:
>  >     Call LookupAccountSid.
>  >     For each User:
>  >       Depending on nsswitch.conf, call LDAP to fetch the extended pass=
wd
>  >       info (pw_shell, pw_home, pw_gecos).
>  >
>  > I guess there's some room for improvement.
>  >
>  > OTOH, keep in mind that you're not suppsoed to call mkpasswd/mkgroup
>  > to enumerate your entire organization.  If you're using it at all, then
>  > only to create the required entries in /etc/passwd and /etc/group for
>  > your local acocunt to work, and then leave everything else to the "db"
>  > setting.
>=20
> Fair enough.  I'll stop stress testing mkpasswd and consider this
> closed unless there's something we want to try.
>=20
> But 1.7.33 seems much faster (if you can call 50 minutes fast) at it
> than 1.7.34-6 or 1.7.35-0.3 in this large-ish AD.  Maybe a knob to
> specify buffer size and/or some other knobs might help identifying the
> slowest parts (and/or some stats).  Just a thought.
>=20
> I'll add that the 1.7.34-6 'strace mkpasswd -d' that I had started
> above finished in 20+ hours and spewed ~3500 of ~8000 entries.

Can you retry this test with the latest developer snapshot (2015-02-25)
from https://cygwin.com/snapshots/, please?  As I already said in other
mails to this list, I rewrote the code doing the LDAP lookup for the
getpwent/getgrent calls, as well as for mkpasswd/mkgroup.  I reduced
the number of LDAP calls to 1 call per 100 users, with no intermediate
LDAP call within.  This should become as fast as it can get.  It may
still be slower than pre-1.7.34, but that is a bit unfair considering
the more complext setup using various different AD attributes depending
on the settings in /etc/nsswitch.conf.

OTOH, the number of accounts per LDAP call (100) is a fixed number right
now.  It might be prudent to use a different, higher number in calls to
mkpasswd/mkgroup, compared to a "real" getpwent/getgrent, let's say,
1000.  That might give us another performance gain, but needs testing in
a big environment like yours, of course.

As for stopping after 3500 of 8000 accounts, I'd be interested to
investigate this further, as I accidentally addressed to Roger Orr.

As I outlined in that mail, I stumbled over a bug in my code which also
resulted in enumerating less accounts as desired.  So I'm not entirely
sure your problem isn't related to a bug either.  So, first thing first,
does that problem still exist with the snapshot?


Thanks,
Corinna

--=20
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

--95xjieIIiUQImiTX
Content-Type: application/pgp-signature

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQIcBAEBAgAGBQJU7bodAAoJEPU2Bp2uRE+gjCAP+gM8+Yvxs2M73yrl+evZumoq
3DDIw/yC9KgQj3aKjp1hDIXCj9B9PiweEcWQ/IkNkE2aY9fNO5a88BYRolv2VyK5
PoniwzXdl5Q6uhKJS52GMRloqId+RCmV7A+0AQ0/ejB4Gr8t3YSTc4lkU1e4D/Q6
uyEYGxGch/dZO7z9i90S5Iofz0tYrI8CKrcxWJQu/hxAW/S0naCgJSHXJZeGQFyS
N9up3WX/KLabmEfJc6+CKTo+35XGerLP8MT/ffpI1XxgZmx1vPVMT+GHrqIbBLms
GpzvBPs9ACKSidfXS126O0K2mk6vC12oVvfRhc01HpMFEnS9DBAPr8knECAl9sP2
dPhF7lepr/VzhmH0Nhs3v/QH0l0C3Vh+lRc0NEXiNuSPabpyG24ikqiACuwHeZWQ
VSLAqJcI5G6Ho2UNoanRknFF+WHSsp2klAejbCxBGFLMIfKyd3mVkN+I3OXmPbTQ
M8kUiyuoJyBVzcQkWWNym75h7aURzqv1Mln4mIVELwrWoTrgi/Exus0fHndVLKf+
/9XW4PlQ36tO1VBvRIuIqBTvOGN6IjHUzg2g2s2Z93a13Z5Q7JP5sTwXqK0ZS15f
8cqQxQw7JR8HiOEP6r+kZALwMMjAbyLHkUN1QI3zAjPewTVeOx7U+LpkeTLIsjou
6Y+wXLIgRzCQbIeQTUgZ
=De/L
-----END PGP SIGNATURE-----

--95xjieIIiUQImiTX--