X-Recipient:	archive-cygwin AT delorie DOT com
DomainKey-Signature:	a=rsa-sha1; c=nofws; d=sourceware.org; h=list-id
	:list-unsubscribe:list-subscribe:list-archive:list-post
	:list-help:sender:date:from:to:subject:message-id:reply-to
	:references:mime-version:content-type:in-reply-to; q=dns; s=
	default; b=yRXoyg8SfXYJdP2MBIAVTXBTv/sXB28Pd/x3As+EjOkNJq58P5pHv
	ZMLtXx+FmBSC+35bl0W3XX0QgrDu/getXqZHb43JSH+4bO7WvClJ+XBP7RVOjs9X
	klsnEjbfJH2R+QY89KVa2JsfSaGUD/HQqC8892f7GNk0drwq7LFFVQ=
DKIM-Signature:	v=1; a=rsa-sha1; c=relaxed; d=sourceware.org; h=list-id
	:list-unsubscribe:list-subscribe:list-archive:list-post
	:list-help:sender:date:from:to:subject:message-id:reply-to
	:references:mime-version:content-type:in-reply-to; s=default;
	bh=aTtlxcWNEb4hHIYlKPmjJP6xMHw=; b=wrwVhAdrgv5LW5xbgdu+JNg7MrUj
	9huWbbRnzzQROTFIG5VA+8+L6TBa6oYWvz0n69H9l2NTwKaVpEeeJwce/cbkmDyY
	KmAYAMx/rqIYS53yeowfT4gdRoKugYFrIH+FX9JOJ1x+R5FWNBo/4ge/y7azPkzs
	kEP75LHMC/72HOg=
Mailing-List:	contact cygwin-help AT cygwin DOT com; run by ezmlm
List-Id:	<cygwin.cygwin.com>
List-Subscribe:	<mailto:cygwin-subscribe AT cygwin DOT com>
List-Archive:	<http://sourceware.org/ml/cygwin/>
List-Post:	<mailto:cygwin AT cygwin DOT com>
List-Help:	<mailto:cygwin-help AT cygwin DOT com>, <http://sourceware.org/ml/#faqs>
Sender:	cygwin-owner AT cygwin DOT com
Mail-Followup-To:	cygwin AT cygwin DOT com
Delivered-To:	mailing list cygwin AT cygwin DOT com
Authentication-Results:	sourceware.org; auth=none
X-Virus-Found:	No
X-Spam-SWARE-Status:	No, score=-6.0 required=5.0 tests=AWL,BAYES_00 autolearn=ham version=3.3.2
X-HELO:	calimero.vinschen.de
Date:	Sat, 1 Feb 2014 11:26:52 +0100
From:	Corinna Vinschen <corinna-cygwin AT cygwin DOT com>
To:	cygwin AT cygwin DOT com
Subject:	Re: How big are your /etc/passwd and /etc/group files?
Message-ID:	<20140201102652.GA26805@calimero.vinschen.de>
Reply-To:	cygwin AT cygwin DOT com
Mail-Followup-To:	cygwin AT cygwin DOT com
References:	<20140131203738 DOT GA8707 AT calimero DOT vinschen DOT de> <CAKf2h5R-FuQX9W=ojw4+ez8gqOHjwYshMh3FmmfNu3r12sRFeQ AT mail DOT gmail DOT com> <20140131220314 DOT GH2821 AT calimero DOT vinschen DOT de> <52EC4727 DOT 2000308 AT gmail DOT com>
MIME-Version:	1.0
In-Reply-To:	<52EC4727.2000308@gmail.com>
User-Agent:	Mutt/1.5.21 (2010-09-15)

--nFreZHaLTZJo0R7j
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On Jan 31 20:00, Max Polk wrote:
> On 1/31/2014 5:03 PM, Corinna Vinschen wrote:
> >On Jan 31 22:40, Frank Fesevur wrote:
> >>2014-01-31 Corinna Vinschen:
> >>>   Is anybody here who's using /etc/passwd and/or group files
> >>>   of more than 16K in size?
> >>The new way to store the stuff would make Cygwin definitely
> >>faster, but it would struggle with... uhm... 2.6 Megs file on
> >>the 32 bit version of Cygwin, Hmm. I'm wondering how to solve
> >>that elegantly. Corinna
>=20
> Every process needs to load only the current user's entry up front.
> Somewhere down the road it only *might* have to do things like
> translate from uid/gid into a string for directory listings, in some
> cases only a handful of these translations.  It's essentially a (old
> dbm style unix) database lookup.

Exactly, but that's just not how the existing code for reading
/etc/passwd and /etc/group works.  At the time of writing the original
code (1996 or so), it was considered faster to load the files once into
memory than having to read them every time a passwd or group entry is
requested.  Apparently, the idea that passwd and group files might
become rather big wasn't considered at the time, and later on we sped up
the code by various measures, but never changed the general idea.

> So defer the database lookups to a libgdbm that goes against a (old

The Cygwin DLL can't load other DLLs, except basic Windows DLLs.
Dependency chicken-egg.  But other than that...

> dbm style unix) database, and don't keep that in memory, unless you
> want a small LRU algorithm in there to keep a small fixed number.  I
> bet the tiny delay later on a ls or other unix translation won't be
> very noticeable.

This means, a rewrite of the /etc/passwd, /etc/group reading code from
"all or nothing" to "one at a time when needed".  Maybe that's in fact
the way to go.

In theory, we don't want to keep the files around anyway.  I have
implemented ways to configure user settings from /etc/passwd (like
home dir or login shell) in the local SAM or in AD.  So the admin
can switch to centralized maintainance even for Cygwin stuff.

On the other hand, we'd like to keep passwd and group for home users
which don't feel comfortable to make changes to the SAM, but then
we only need very tiny files with one or two entries anyway.

Reading the files on demand only might be the right thing to do.
Catching two birds with one stone...


Corinna

--=20
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

--nFreZHaLTZJo0R7j
Content-Type: application/pgp-signature

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1

iQIcBAEBAgAGBQJS7MvsAAoJEPU2Bp2uRE+gvJ4QAIbrXzIH2YzB/I/Rk7rkqTdm
sIDa3jNFFPZQg9LT42WSqX8xqqlAKZEWetJc4R0D/hRMO5T1MZTgzBpmQ5nrs28Y
RbEhe4+f/8Gfxw9vuI86GB6NHqd/hHuK2gOlQX2c28h94LIczMW22uaAv/U6edAF
/e4vugG1BD9BTGMNJdlxvv7Wd+rQ5X6i7kGZP+kQUow2VvAzn/TH8pryCaAy+aIj
QyT9MvKlafYj39MkYVTZZmF3KVXuuR6QfLvqFFZGKjRcftWVnOagKrWk68R9mDD9
rm2O8mMO8k042pCI9SgfCHN+UI2r5HjlGDDm54ZvFT3nv5zcq0B2ZmDsZIEAqKVs
VR0xjKbBLn0xvMhhCoRPMsMv7LsLhWBGHYxjMeCIwhbe0a+vH+v2AAI+hZIdMiL4
PQddQcFg9YsSYk6GPYIKlY3EkEfSE4yjkDIdx7Neu19uGb3Tso9/h6SDUxktyujS
Rj8g5w2vOLFMbCaqOtAvS3y7KSVjpnuF33PlmzQ9ZBpe1GysAI4QFLfQ9kCBwe5j
X4DkzMq2nyVZt203O0rI4CaG7mtfYZ1IBJuejrByHZGbXc/AAnt9wahhuWsEeOZp
K80wHl0aL2HFL/VsBbMkYX9MTyqpxXWfGlOCFoQI0TjiIwCk1mAUs6AZsve1UYO9
ssh1oHEEPco7mBM4Jh9g
=9oNq
-----END PGP SIGNATURE-----

--nFreZHaLTZJo0R7j--

- Raw text -