X-Recipient: archive-cygwin AT delorie DOT com
DomainKey-Signature: a=rsa-sha1; c=nofws; d=sourceware.org; h=list-id
	:list-unsubscribe:list-subscribe:list-archive:list-post
	:list-help:sender:date:from:to:subject:message-id:reply-to
	:references:mime-version:content-type:in-reply-to; q=dns; s=
	default; b=HnJWd1WDL8TCgAsuXd9S5U/tI1UUbcp0npE7MkpC5C+iWGcUHvl2g
	53aZmhzlSMrtCmj5Kd+E48SKvh2weh++OyuCIjOO8QBbQSsbHqxYFn+GzK9KoVkC
	DURX6cKTE3Q60iZzcCZEEdv2YB/Qi9AbbZ7NOE2zxpdAPVEksWyeyQ=
DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=sourceware.org; h=list-id
	:list-unsubscribe:list-subscribe:list-archive:list-post
	:list-help:sender:date:from:to:subject:message-id:reply-to
	:references:mime-version:content-type:in-reply-to; s=default;
	 bh=UDyozeJe3W7RWLHFlfD1+TxX3WM=; b=M/PmEBqUdvCEYaqRmXditVTncr5a
	0S/XBPYlbyjqMwyXSXXqNWAIxuFRMJLLu4vu8CkwkvOx5VGuj/6sjzB3RHOzBxSy
	fEmJ1qY+/KuUyaXB4b7qki8B4VJ2xEsSRNF85f89EgOltT46wkmDd5LLh3vldTx5
	u68j6POmWErXZN4=
Mailing-List: contact cygwin-help AT cygwin DOT com; run by ezmlm
List-Id: <cygwin.cygwin.com>
List-Subscribe: <mailto:cygwin-subscribe AT cygwin DOT com>
List-Archive: <http://sourceware.org/ml/cygwin/>
List-Post: <mailto:cygwin AT cygwin DOT com>
List-Help: <mailto:cygwin-help AT cygwin DOT com>, <http://sourceware.org/ml/#faqs>
Sender: cygwin-owner AT cygwin DOT com
Mail-Followup-To: cygwin AT cygwin DOT com
Delivered-To: mailing list cygwin AT cygwin DOT com
Authentication-Results: sourceware.org; auth=none
X-Virus-Found: No
X-Spam-SWARE-Status: No, score=-5.9 required=5.0 tests=AWL,BAYES_00 autolearn=ham version=3.3.2
X-HELO: calimero.vinschen.de
Date: Wed, 25 Jun 2014 23:13:55 +0200
From: Corinna Vinschen <corinna-cygwin AT cygwin DOT com>
To: cygwin AT cygwin DOT com
Subject: Re: timeout in LDAP access
Message-ID: <20140625211355.GA25116@calimero.vinschen.de>
Reply-To: cygwin AT cygwin DOT com
Mail-Followup-To: cygwin AT cygwin DOT com
References: <20140617100011 DOT GL23700 AT calimero DOT vinschen DOT de> <C462E4F3-1E51-46DC-BD27-BC4786A5E8BB AT Denis-Excoffier DOT org> <20140618083304 DOT GV23700 AT calimero DOT vinschen DOT de> <20140618180102 DOT GA27055 AT calimero DOT vinschen DOT de> <FEEBC1A4-B147-45C1-A5AC-F5B9108E998F AT Denis-Excoffier DOT org> <20140623090959 DOT GA1803 AT calimero DOT vinschen DOT de> <C2FB35D9-AE47-4461-8A94-20605D5EB996 AT Denis-Excoffier DOT org> <20140624155851 DOT GJ1803 AT calimero DOT vinschen DOT de> <20140625101526 DOT GO1803 AT calimero DOT vinschen DOT de> <E760D646-FFCB-434C-B990-7783DC011326 AT Denis-Excoffier DOT org>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha1;	protocol="application/pgp-signature"; boundary="ZGiS0Q5IWpPtfppv"
Content-Disposition: inline
In-Reply-To: <E760D646-FFCB-434C-B990-7783DC011326@Denis-Excoffier.org>
User-Agent: Mutt/1.5.23 (2014-03-12)

--ZGiS0Q5IWpPtfppv
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On Jun 25 22:44, Denis Excoffier wrote:
> On 2014-06-25 12:15, Corinna Vinschen wrote:
> >> Stay tuned.  I'm rewriting the LDAP access code to perform all critical
> >> LDAP calls in interruptible threads.  The Windows LDAP calls don't
> >> provide any kind of synchronization, only timeouts.  I hoped to get aw=
ay
> >> with short timeouts but it seems I hoped in vain.
> >>=20
> >> So the next iteration of this code will not use any timeout other than
> >> the default LDAP network timeout of 2 minutes, but the calls will be
> >> interruptible by signals.
> >>=20
> >=20
> > No more artificial timeouts, but the LDAP calls will be interruptible by
> > a signal now.
> >=20
> > Also, if an error occurs during ad enumeration, getpwent/getgrent will
> > return NULL with errno set accordingly.
> >=20
> > Please test,
> I did. Again, i instrumented ldap.cc by replacing all debug_printf() calls
> with system_printf() because my /usr/bin/strace does not work. Again, i
> tested with =E2=80=98getent passwd > result=E2=80=99 and 'db_enum: all=E2=
=80=99.
>=20
> I got the following message:
> [ldap_init] getent 6024 cyg_ldap::connect_non_ssl: ldap_bind(xxxxxx.zzz) =
0x51
> and getent stops after the 376000 users in my own domain. No timeout occu=
rred
> but the enumeration was stopped by LDAP_SERVER_DOWN (0x51) [the xxxxxx.zzz
> domain name has been edited here but it was completely new to me, never s=
een
> before].

You asked for errors being propagated up the chain to the
getpwent/getgrent calls and that's exactly what happens now.  There are
a lot of LDAP error codes.  How is Cygwin supposed to handle every one
of them?  Do we need a list of ignorable and non-ignorable error codes?

Alternatively this gets reverted and Cywin does *not* break the search
if an error occurs, but instead skips this domain and starts enumerating
the next domain, just as before?

> Also, there was a large delay (more than 2 min, say at least 8 minutes) b=
etween
> the end of output and the end of getent. I got one single system_printf
> message (see above).

I can't observe this.  It needs debugging in your environment so I know
which part of the source is responsible for this delay under what
circumstances.

(and I still think it's a crazy idea to enumerate 500K users)

> More than that, i added system_printf("starting open in domain %W", domai=
n)
> immediately at the beginning of cyg_ldap::open, and run =E2=80=98getent p=
asswd=E2=80=99 now during
> one minute (wait 60s, then Control-C). I got 1080 =E2=80=98starting open =
in domain (null)=E2=80=99
> messages on stderr and 1016 normal passwd entries on stdout. The discrepa=
ncy
> 1016 vs 1080 is ok because stdout was not properly flushed out.

60 seconds for 1016 user entries?  That sounds incredibly slow.

> It seems that
> - domain is printed as =E2=80=98(null)=E2=80=99? Strange

Not at all.  This indicates the primary domain.

> - there are as many open() calls as passwd entries in the output?

The open function is called for every account, but that doesn't mean it
really needs opening.  That's what the early return is for.  The code
starts like this:

int
cyg_ldap::open (PCWSTR domain)
{
  int ret =3D 0;

  /* Already open? */
  if (lh)
    return 0;

  if ((ret =3D connect (domain)) !=3D NO_ERROR)
    goto err;
  [...]

Did you add the system_printf before the "/* Already open? */" comment,
by any chance?

> Also strange
> - EIO (or equivalent) is produced for LDAP_SERVER_DOWN, it probably shoul=
d be
>   better if this were not the case?

See above.

> I suppose it will need more testing, but i=E2=80=99m currently unavailabl=
e for tests,
> by the way until Friday 08:00 UTC.

No worries.  Thanks for pulling this through.


Corinna
--=20
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

--ZGiS0Q5IWpPtfppv
Content-Type: application/pgp-signature

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1

iQIcBAEBAgAGBQJTqzuTAAoJEPU2Bp2uRE+gE68P/AhvxlmB5uKB3QjblwQe4n4v
HaPLFC5A0xec/WI8ctLZA7kDAEyU96HQrlhWifFUaaXIlG/V53tbuM3v+0UKFvQN
zJ7sUctHD/bzidd0GPReJYcLSb5KFZ+5Xkq8fZi9KY2g7SUwGpwNAimEFHUzY6RI
T1LIzgtlufuVmyGbjhYNLVeiWgMggwLA+T+S49WG+tbc1x+JfbzsQfvmOQvob0ar
1zHsLP9qI5CNiqYp3+hlpr0/g/hn07yd1I9wgvtf8iFEKcLAw2RrNHW815GpH9fx
LPp/l4L8yvbthSDgMRreNhcz14khxL1+Lqv0aqS9nAgiwlhuaHpFmCdtkGXJmyBJ
gaZ4PYV5+k5HxlkXTeMaxlPnG0g0HznkGwlN8uYs1WZ7ux/0OBnQLVDAL93BQ1G/
fzPJvUzbiYT4WP0h7ohYHecgE5Cx1BWK1NH6u1MWELSse18v4usEeMJnZGrYK4ZC
tg/kkADMrsXWb992MS37QmFBPcCNRIjCeZD5kMAN4kFqnSjDBceBpGHa198LsDfg
euVnDC6Nr2lPSvqxjxpudcZ8b/QbCpAZYtZHmiPj9uMLy+TaijkOqYT7Bmc+X+lz
Ni5K7LqnjQmTOTNZUfa/WYZWMZaT6f4VrY6owhZ/hOz3fs2x2PBYxvxQmOIrOAs2
VSk+3sWZrox5omT3q4bG
=HSym
-----END PGP SIGNATURE-----

--ZGiS0Q5IWpPtfppv--