Subject: PostgreSQL/Apache dying
From: John Sullivan <[email-masked]>
To: [email-masked]
Date: Fri, 7 Nov 2003 17:47:28 +0000 (GMT)

I'm running PostgreSQL with Apache/mod_php under cygwin and Windows
98SE. A perl script makes various HTTP requests, munges the results,
and uses DBI:Pg to insert the results into pgsql. Various PHP scripts
then use pg_* functions to extract the data for presentation.

I find that after a small number of transactions (usually <50) from either
source, the system-wide socket space fills up:

  * New connects to pgsql are refused with the error:

      pg_connect() unable to connect to PostgreSQL server: could not
      connect to server: No buffer space available Is the server running
      locally and accepting connections on Unix domain socket
      "/tmp/.s.PGSQL.5432"?

  * HTTP requests from the upload script fail with LWP-internal 500 Timeout
    errors.

  * Attempts to navigate in (non-cygwin) web browsers fail. Firebird reports
    "Done" immediately whilst remaining at the previous page, IE at
    least gives a "Couldn't contact remote site" error.

Killing and restarting postmaster fixes things (until the next
time). This problem happens even without starting apache - the upload
script can trigger it on its own if it finds enough interesting data,
and it does a single database open/close. However as shown below there
is clearly an apache related problem as well.

Sometimes postmaster can be killed by keyboard, sometimes it's wedged
so hard it requires Ctrl-Alt-Del to kill it from the task list.

Using sysinternals Process Explorer and TCPView may shed some light: the
top-level Postgres.exe process has the following handles open when wedged:

[...]
0x1EC	Process	POSTGRES.EXE(FFF930EB)
0x1F0	Process	POSTGRES.EXE(FFF930EB)
0x1F4	Thread	POSTGRES.EXE(FFFAEB57): FFCF79F3
0x1F8	Process	<Non-existant Process>(F4BDDA6B)
0x1FC	Event cygwin.local_socket.secret.7433.C0F190FA-90BA556A-6F9DB327-3272B533
0x200	Event cygwin.local_socket.secret.7433.C0F190FA-90BA556A-6F9DB327-3272B533
0x208	Process	<Non-existant Process>(FFF0BD3F)
0x210	Thread	POSTGRES.EXE(FFFAEB57): FFCF79F3
0x214	Event cygwin.local_socket.secret.7433.C0F190FA-90BA556A-6F9DB327-3272B533
0x218	Event cygwin.local_socket.secret.7433.C0F190FA-90BA556A-6F9DB327-3272B533
0x21C	Event cygwin.local_socket.secret.7433.C0F190FA-90BA556A-6F9DB327-3272B533
0x220	Event cygwin.local_socket.secret.7433.C0F190FA-90BA556A-6F9DB327-3272B533
0x224	Process	<Non-existant Process>(F4BC525F)
0x228	Process	<Non-existant Process>(FFCF1663)
0x22C	Event cygwin.local_socket.secret.7433.C0F190FA-90BA556A-6F9DB327-3272B533
0x230	Event cygwin.local_socket.secret.7433.C0F190FA-90BA556A-6F9DB327-3272B533
0x234	Event cygwin.local_socket.secret.7433.C0F190FA-90BA556A-6F9DB327-3272B533
0x238	Event cygwin.local_socket.secret.7433.C0F190FA-90BA556A-6F9DB327-3272B533
0x23C	Event cygwin.local_socket.secret.7433.C0F190FA-90BA556A-6F9DB327-3272B533
0x240	Event cygwin.local_socket.secret.7433.C0F190FA-90BA556A-6F9DB327-3272B533
0x244	Event cygwin.local_socket.secret.7433.C0F190FA-90BA556A-6F9DB327-3272B533
0x248	Event cygwin.local_socket.secret.7433.C0F190FA-90BA556A-6F9DB327-3272B533
0x24C	Event cygwin.local_socket.secret.7433.C0F190FA-90BA556A-6F9DB327-3272B533
0x250	Event cygwin.local_socket.secret.7433.C0F190FA-90BA556A-6F9DB327-3272B533
0x254	Event cygwin.local_socket.secret.7433.C0F190FA-90BA556A-6F9DB327-3272B533
0x258	Event cygwin.local_socket.secret.7433.C0F190FA-90BA556A-6F9DB327-3272B533
0x25C	Event cygwin.local_socket.secret.7433.C0F190FA-90BA556A-6F9DB327-3272B533
0x260	Event cygwin.local_socket.secret.7433.C0F190FA-90BA556A-6F9DB327-3272B533
0x264	Event cygwin.local_socket.secret.7433.C0F190FA-90BA556A-6F9DB327-3272B533
0x268	Event cygwin.local_socket.secret.7433.C0F190FA-90BA556A-6F9DB327-3272B533
0x26C	Event cygwin.local_socket.secret.7433.C0F190FA-90BA556A-6F9DB327-3272B533
0x270	Event cygwin.local_socket.secret.7433.C0F190FA-90BA556A-6F9DB327-3272B533
0x274	Event cygwin.local_socket.secret.7433.C0F190FA-90BA556A-6F9DB327-3272B533
0x278	Event cygwin.local_socket.secret.7433.C0F190FA-90BA556A-6F9DB327-3272B533
0x27C	Event cygwin.local_socket.secret.7433.C0F190FA-90BA556A-6F9DB327-3272B533
0x280	Event cygwin.local_socket.secret.7433.C0F190FA-90BA556A-6F9DB327-3272B533
0x284	Event cygwin.local_socket.secret.7433.C0F190FA-90BA556A-6F9DB327-3272B533
0x288	Event cygwin.local_socket.secret.7433.C0F190FA-90BA556A-6F9DB327-3272B533
0x28C	Event cygwin.local_socket.secret.7433.C0F190FA-90BA556A-6F9DB327-3272B533
0x290	Event cygwin.local_socket.secret.7433.C0F190FA-90BA556A-6F9DB327-3272B533
0x294	Event cygwin.local_socket.secret.7433.C0F190FA-90BA556A-6F9DB327-3272B533
0x298	Event cygwin.local_socket.secret.7433.C0F190FA-90BA556A-6F9DB327-3272B533
0x29C	Event cygwin.local_socket.secret.7433.C0F190FA-90BA556A-6F9DB327-3272B533
0x2A0	Event cygwin.local_socket.secret.7433.C0F190FA-90BA556A-6F9DB327-3272B533
0x2A4	Event cygwin.local_socket.secret.7433.C0F190FA-90BA556A-6F9DB327-3272B533

On initial startup Postgres has none of these duplicate handles open. For
each page load (which involves several queries within a single
pg_connect/pg_close) a new one appears.

Furthermore, at failure the following sockets are open:

UDP	127.0.0.1:2334	*:*		
UDP	127.0.0.1:3086	*:*		
UDP	192.168.1.40:137	*:*		
UDP	192.168.1.40:138	*:*		
TCP	127.0.0.1:1980	127.0.0.1:80	TIME_WAIT	
TCP	127.0.0.1:1028	127.0.0.1:1029	ESTABLISHED	
TCP	127.0.0.1:1029	127.0.0.1:1028	ESTABLISHED	
TCP	127.0.0.1:2333	127.0.0.1:1259	ESTABLISHED	
TCP	127.0.0.1:2333	127.0.0.1:1141	ESTABLISHED	
TCP	127.0.0.1:2895	127.0.0.1:2333	ESTABLISHED	
TCP	127.0.0.1:2947	127.0.0.1:2333	ESTABLISHED	
TCP	127.0.0.1:1141	127.0.0.1:2333	ESTABLISHED	
TCP	127.0.0.1:1259	127.0.0.1:2333	ESTABLISHED	
TCP	127.0.0.1:1377	127.0.0.1:2333	ESTABLISHED	
TCP	127.0.0.1:1426	127.0.0.1:2333	ESTABLISHED	
TCP	127.0.0.1:1487	127.0.0.1:2333	ESTABLISHED	
TCP	127.0.0.1:1549	127.0.0.1:2333	ESTABLISHED	
TCP	127.0.0.1:1607	127.0.0.1:2333	ESTABLISHED	
TCP	127.0.0.1:1665	127.0.0.1:2333	ESTABLISHED	
TCP	127.0.0.1:1755	127.0.0.1:2333	ESTABLISHED	
TCP	127.0.0.1:1813	127.0.0.1:2333	ESTABLISHED	
TCP	127.0.0.1:1868	127.0.0.1:2333	ESTABLISHED	
TCP	127.0.0.1:1923	127.0.0.1:2333	ESTABLISHED	
TCP	127.0.0.1:1983	127.0.0.1:2333	ESTABLISHED	
TCP	127.0.0.1:2333	127.0.0.1:1426	ESTABLISHED	
TCP	127.0.0.1:2333	127.0.0.1:1549	ESTABLISHED	
TCP	127.0.0.1:2333	127.0.0.1:1665	ESTABLISHED	
TCP	127.0.0.1:2333	127.0.0.1:1813	ESTABLISHED	
TCP	127.0.0.1:2333	127.0.0.1:1923	ESTABLISHED	
TCP	127.0.0.1:2333	127.0.0.1:2895	ESTABLISHED	
TCP	127.0.0.1:2333	127.0.0.1:3106	ESTABLISHED	
TCP	127.0.0.1:2333	127.0.0.1:3214	ESTABLISHED	
TCP	127.0.0.1:2333	127.0.0.1:3339	ESTABLISHED	
TCP	127.0.0.1:2333	127.0.0.1:3457	ESTABLISHED	
TCP	127.0.0.1:2333	127.0.0.1:3578	ESTABLISHED	
TCP	127.0.0.1:2333	127.0.0.1:3818	ESTABLISHED	
TCP	127.0.0.1:2333	127.0.0.1:4054	ESTABLISHED	
TCP	127.0.0.1:2333	127.0.0.1:4290	ESTABLISHED	
TCP	127.0.0.1:2333	127.0.0.1:4523	ESTABLISHED	
TCP	127.0.0.1:2333	127.0.0.1:4763	ESTABLISHED	
TCP	127.0.0.1:2333	127.0.0.1:4996	ESTABLISHED	
TCP	127.0.0.1:3106	127.0.0.1:2333	ESTABLISHED	
TCP	127.0.0.1:3162	127.0.0.1:2333	ESTABLISHED	
TCP	127.0.0.1:3214	127.0.0.1:2333	ESTABLISHED	
TCP	127.0.0.1:3275	127.0.0.1:2333	ESTABLISHED	
TCP	127.0.0.1:3339	127.0.0.1:2333	ESTABLISHED	
TCP	127.0.0.1:3388	127.0.0.1:2333	ESTABLISHED	
TCP	127.0.0.1:3457	127.0.0.1:2333	ESTABLISHED	
TCP	127.0.0.1:3529	127.0.0.1:2333	ESTABLISHED	
TCP	127.0.0.1:3578	127.0.0.1:2333	ESTABLISHED	
TCP	127.0.0.1:3696	127.0.0.1:2333	ESTABLISHED	
TCP	127.0.0.1:3818	127.0.0.1:2333	ESTABLISHED	
TCP	127.0.0.1:3936	127.0.0.1:2333	ESTABLISHED	
TCP	127.0.0.1:4054	127.0.0.1:2333	ESTABLISHED	
TCP	127.0.0.1:4172	127.0.0.1:2333	ESTABLISHED	
TCP	127.0.0.1:4290	127.0.0.1:2333	ESTABLISHED	
TCP	127.0.0.1:4408	127.0.0.1:2333	ESTABLISHED	
TCP	127.0.0.1:4523	127.0.0.1:2333	ESTABLISHED	
TCP	127.0.0.1:4641	127.0.0.1:2333	ESTABLISHED	
TCP	127.0.0.1:4763	127.0.0.1:2333	ESTABLISHED	
TCP	127.0.0.1:4881	127.0.0.1:2333	ESTABLISHED	
TCP	127.0.0.1:4996	127.0.0.1:2333	ESTABLISHED	
TCP	0.0.0.0:80	0.0.0.0:0	LISTENING	
TCP	0.0.0.0:1026	0.0.0.0:0	LISTENING	
TCP	0.0.0.0:1029	0.0.0.0:0	LISTENING	
TCP	0.0.0.0:2895	0.0.0.0:0	LISTENING	
TCP	0.0.0.0:2947	0.0.0.0:0	LISTENING	
TCP	127.0.0.1:1028	0.0.0.0:0	LISTENING	
TCP	127.0.0.1:2333	0.0.0.0:0	LISTENING	
TCP	127.0.0.1:2334	0.0.0.0:0	LISTENING	
TCP	127.0.0.1:2997	0.0.0.0:0	LISTENING	
TCP	127.0.0.1:2998	0.0.0.0:0	LISTENING	
TCP	127.0.0.1:3086	0.0.0.0:0	LISTENING	
TCP	127.0.0.1:3101	0.0.0.0:0	LISTENING	
TCP	192.168.1.40:137	0.0.0.0:0	LISTENING	
TCP	192.168.1.40:138	0.0.0.0:0	LISTENING	
TCP	192.168.1.40:139	0.0.0.0:0	LISTENING	
TCP	0.0.0.0:1141	0.0.0.0:0	LISTENING	
TCP	0.0.0.0:1259	0.0.0.0:0	LISTENING	
TCP	0.0.0.0:1377	0.0.0.0:0	LISTENING	
TCP	0.0.0.0:1426	0.0.0.0:0	LISTENING	
TCP	0.0.0.0:1487	0.0.0.0:0	LISTENING	
TCP	0.0.0.0:1549	0.0.0.0:0	LISTENING	
TCP	0.0.0.0:1607	0.0.0.0:0	LISTENING	
TCP	0.0.0.0:1665	0.0.0.0:0	LISTENING	
TCP	0.0.0.0:1755	0.0.0.0:0	LISTENING	
TCP	0.0.0.0:1813	0.0.0.0:0	LISTENING	
TCP	0.0.0.0:1868	0.0.0.0:0	LISTENING	
TCP	0.0.0.0:1923	0.0.0.0:0	LISTENING	
TCP	0.0.0.0:1983	0.0.0.0:0	LISTENING	
TCP	0.0.0.0:2037	0.0.0.0:0	LISTENING	
TCP	0.0.0.0:3106	0.0.0.0:0	LISTENING	
TCP	0.0.0.0:3162	0.0.0.0:0	LISTENING	
TCP	0.0.0.0:3214	0.0.0.0:0	LISTENING	
TCP	0.0.0.0:3275	0.0.0.0:0	LISTENING	
TCP	0.0.0.0:3339	0.0.0.0:0	LISTENING	
TCP	0.0.0.0:3388	0.0.0.0:0	LISTENING	
TCP	0.0.0.0:3457	0.0.0.0:0	LISTENING	
TCP	0.0.0.0:3529	0.0.0.0:0	LISTENING	
TCP	0.0.0.0:3578	0.0.0.0:0	LISTENING	
TCP	0.0.0.0:3696	0.0.0.0:0	LISTENING	
TCP	0.0.0.0:3818	0.0.0.0:0	LISTENING	
TCP	0.0.0.0:3936	0.0.0.0:0	LISTENING	
TCP	0.0.0.0:4054	0.0.0.0:0	LISTENING	
TCP	0.0.0.0:4172	0.0.0.0:0	LISTENING	
TCP	0.0.0.0:4290	0.0.0.0:0	LISTENING	
TCP	0.0.0.0:4408	0.0.0.0:0	LISTENING	
TCP	0.0.0.0:4523	0.0.0.0:0	LISTENING	
TCP	0.0.0.0:4641	0.0.0.0:0	LISTENING	
TCP	0.0.0.0:4763	0.0.0.0:0	LISTENING	
TCP	0.0.0.0:4881	0.0.0.0:0	LISTENING	
TCP	0.0.0.0:4996	0.0.0.0:0	LISTENING	
TCP	127.0.0.1:2034	0.0.0.0:0	LISTENING	
TCP	127.0.0.1:2036	0.0.0.0:0	LISTENING	
TCP	127.0.0.1:2040	0.0.0.0:0	LISTENING

That's a lot compared to the 16-odd open with no other apps
started. The extra ESTABLISHED sockets go when I kill postmaster. The
LISTENING ones go after I kill Apache.

Apache is started using "httpd -X" because in normal mode it generates
an Invalid Page Fault in CYGCRYPTO.DLL and exits. I've read that this
could be base-address and fork() related, have tried a rebase-all,
which sort of allows it to start but now generates tens of Invalid
Page Fault dialogs, all apparently harmless but extremely annoying to
have to dismiss. So I reinstalled the un-rebased cygwin dlls.

I haven't had any issue with the smaller cygwin utilities. Even gcc
appears to work fine. But there's clearly something seriously wrong
with these two.

Some version numbers:

apache               1.3.24-5           OK
base-files           2.6-1              OK
cygipc               2.02-1             OK
cygutils             1.2.2-1            OK
cygwin               1.5.5-1            OK
mod_php4             4.2.0-2            OK
openssl              0.9.7c-1           OK
openssl096           0.9.6j-1           OK
perl                 5.8.0-5            OK
postgresql           7.3.4-2            OK
rebase               2.2-3              OK

John
-- 
Dead stars still burn

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/