DomainKey-Signature: a=rsa-sha1; c=nofws; d=sourceware.org; h=list-id
	:list-unsubscribe:list-subscribe:list-archive:list-post
	:list-help:sender:date:from:to:cc:subject:message-id:reply-to
	:references:mime-version:content-type:in-reply-to; q=dns; s=
	default; b=wbAYqxX/I8kComQwu3ia3jwnOfzN7BTWwJlVpCqKSsl/3lytdqDql
	rtRgb3xoo/+AE0a5TXOpCnpveIIQA3jE65yKqpADJiiUbxIrhmA6927191RzNvWi
	WxDmWt//BnhNKxBxVpFuomkSG5cKSozGdD/2w9aaFJPu4NYJ/ulUqs=
Mailing-List: contact cygwin-help AT cygwin DOT com; run by ezmlm
Sender: cygwin-owner AT cygwin DOT com
Mail-Followup-To: cygwin AT cygwin DOT com
Date: Fri, 24 Mar 2017 18:11:01 +0100
From: Corinna Vinschen <corinna-cygwin AT cygwin DOT com>
To: cygwin AT cygwin DOT com
Cc: Noah Misch <noah AT leadboat DOT com>
Subject: Re: cygserver - Postgres Multiple connection Load Testing - Inifinte Loop
Message-ID: <20170324171101.GI29995@calimero.vinschen.de>
Reply-To: cygwin AT cygwin DOT com
Mail-Followup-To: cygwin AT cygwin DOT com, Noah Misch <noah AT leadboat DOT com>
References: <200408030333 DOT i733XEXn023894 AT mx3 DOT redhat DOT com> <20040803100612 DOT GP31522 AT cygbert DOT vinschen DOT de> <20170321025614 DOT GA2100214 AT tornado DOT leadboat DOT com>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha256;	protocol="application/pgp-signature"; boundary="zhXaljGHf11kAtnf"
Content-Disposition: inline
In-Reply-To: <20170321025614.GA2100214@tornado.leadboat.com>
User-Agent: Mutt/1.8.0 (2017-02-23)

--zhXaljGHf11kAtnf
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

Hi Noah,

thanks for the report and especially the testcase.  It took me a while
to debug that, but I think I fixed it now.  At least your testcase is
working for me now.  It also got faster, albeit always slower than Linux
because of the communication overhead between processes and cygserver.

On Mar 21 02:56, Noah Misch wrote:
> On Tue, Aug 03, 2004 at 12:06:12PM +0200, Corinna Vinschen wrote:
> > On Aug  2 20:33, sarbx-cygwin6344 AT mailblocks DOT com wrote:
> > > This time around, cygserver does not eat CPU. But after 5 to 6=20
> > > concurrent
> > > connections nothing seem to work, looks kind of hung. There is no=20
> > > activity in the Postgres
> > > log file. Opening a new database connection also hangs. There is no=
=20
> > > activity on the machine.
>=20
> > Any chance to create a simple testcase which uncovers that behaviour
> > without involving a whole database system?
>=20
> Attached test program reproduces it on Cygwin 2.7.0, Cygwin 1.7.5, and a =
few
> intermediate versions.  The program creates sixteen processes that each
> perform a tight loop over the following:
>=20
> - select one of four semaphores
> - reduce semaphore's value from 1 to 0 ("lock" it)
> - raise semaphore's value from 0 to 1 ("unlock" it)
>=20
> On GNU/Linux, AIX, and Solaris, the processes keep busy and finish one mi=
llion
> lock/unlock cycles apiece in a few minutes.  On Cygwin, they hang within =
a few
> seconds and under one hundred cycles apiece.  At that point, cygserver is
> unresponsive to other clients; for example, "strace /bin/true", opening a=
 new
> Cygwin terminal, "cat /proc/sysvipc/sem" and "cygserver -S" all hang.  In=
 most
> tests, cygserver was not consuming CPU while unresponsive.

There are two problems here.

- cygserver is using a defined number of threads in a thread pool for
  application requests.  Every request is added to a request submission
  queue and handled by the next free thread in the pool.

  The default number of threads in the pool is 10.  Each wait for a
  semaphore is blocking one thread.  If more than the number of threads
  in the pool are supposed to wait on a semaphore the pool starves.

  Raising the pool size fixes this part, but the situation is still a bit
  unsatisfying.  You may not know the load and the number of competing
  processes in every scenario beforehand, but raising cygservers thread
  pool to some really big value doesn't always make sense either.

  So what I did now is to allow cygserver to raise the number of worker
  threads on demand.  That is, if a request is in the queue and all
  worker threads are busy, just create a new one.

  There's no way yet to drop threads again, but this should be a minor
  problem in scenarions which really have a lot of contention.

- The code emulating BSD msleep/wakeup wasn't quite up to speed.
  I rewrote a major part of the code to be more robust and faster.

I pushed a patchset now, and uploaded new developer snapshots for
testing to https://cygwin.com/snapshots/

I'm also going to create a 2.8.0-0.4 test release later today.

Please give it a try, and please note that *all* patches affect
cygserver itself, so you have to test the new cygserver in the
first place.  The Cygwin DLL is not affected by the changes.


Thanks,
Corinna

--=20
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

--zhXaljGHf11kAtnf
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQIcBAEBCAAGBQJY1VMlAAoJEPU2Bp2uRE+gZ7sP/A9hoFI5ZGWBUJXMo3q05FUL
yte2+/bWhpyXfXkKVZ8HMmUUQetEHsq3FSSaXdCKJIxrWhh3zb0X+HHU+OXXLszF
z9uhBgrIFlzQRXiomYL9MAtoK5iQSr8bX+1Mgnn7LViwLxFozziINaXZgaJjwcXV
dpQd8a0KzNq95OJiq4gSAhZTH+r4h+IZEW/r8xDMFRulfPYAToXs7X0gH9S3zoEm
xVnBwDXKOxaNo8zWtXBIQLOz19AI/Zcf/kjzrNeMuPkjkse2fvb76CvCYwRTahvE
XSZAMhF8wICu5gLzoMBdbKjRMQuouE4xhuYsBwiTAJKbnGI3jcbm7rWrCjmMEYoI
XljXigezGMAYDwLTI5XGTr+51jwHdQC3YmNxw2LKokr8NrNDnXRx6EcwISLzEaNo
21g3so7+ZJx7OyX7NgxZqCW4ju+AS60goLL3DGh2BkncW1yOXkIGG2YrQQYPcE/+
z8YqdX0fb8Sdk+/stLws6MY2PziUmlSzSEOVmZVv2IM1JHze+UUGrfilp8T6wy+n
1mD0w/C7ZkOL+R/5RF3e8YWHyGw/OoEzOCtv64CU5ZDspxDRYop+wwXtHC1UDHkV
b3kEuGy+IAZ410ozFSMElPGhMGYnspwiXDFmfZt2iVo4YClsSBVFr55WaWNHMbgV
KprsiHr8S04LoZhaHfIB
=O6TX
-----END PGP SIGNATURE-----

--zhXaljGHf11kAtnf--