Mailing-List: contact cygwin-help@cygwin.com; run by ezmlm
Sender: cygwin-owner@cygwin.com
Mail-Followup-To: cygwin@cygwin.com
Message-ID: <41FD13A3.5060103@x-ray.at>
Date: Sun, 30 Jan 2005 18:04:35 +0100
From: Reini Urban <rurban@x-ray.at>
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; de-AT; rv:1.7) Gecko/20040616
MIME-Version: 1.0
To: cygwin@cygwin.com
Subject: Re: pthreads leaks handles and threads when threads use sockets
References: <41FBD281.2020108@x-ray.at> <003b01c50640$5a2c1840$173ca8c0@AlohaSunset.com>
In-Reply-To: <003b01c50640$5a2c1840$173ca8c0@AlohaSunset.com>
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: 7bit

Mark Pizzolato schrieb:
>> Reini Urban wrote:
>> Mark Pizzolato schrieb:
>> > I've been using clamav's clamd under cygwin and noticed that over time
>> > the handle count as viewed with TaskManager seems to grow to arbitrary
>> > values. I used clamd's option IdleTimeout set to 600 seconds which
>> > dramatically reduced the growth rate of the Handle Count. Of course
>> > clamd has many things going on that could contribute to handle leakags,
>> > so I tried to write a simple program to demonstrate the problem.
>>
>> Thanks a lot! Maybe we should restart the two daemons daily or weekly?
>>
>> I will change the default IdleTimeout to 600 secs with the upcoming
>> clamav-0.81 release. Which fixes the freshclam proxy problem and some
>> OLE issues.
> 
> Merely setting IdleTimeout to 600 is currently insufficient due to a bug 
> which only uses the IdleTimeout parameter for the Initial value used.  
> After the AV Database is reloaded, the idle timeout is hardset to 30 
> seconds.  The attached patch (to 0.80 or 0.81) fixes this issue.  This 
> patch has been submitted on the clamav-devel list.

Yes, I saw it and already patched my 0.81.
But I still have to find some time reproduce your problem.

> The right choice for the IdleTimeout is a value which is larger than the 
> largest gap between messages that arrive on your system.  This is 
> somewhat complicated by multiple connections arriving concurrently which 
> is handled by MaxThreads.  MaxThreads has a default value of 10.  This 
> would be fine for most systems, however libclamav uses tmpfile() 
> internally which is NOT threadsafe (using newlib's tmpfile()) for any 
> system which returns the same value for getpid() for each thread in a 
> process (i.e. it works fine on Linux since getpid() on Linux returns a 
> unique value for all threads on the system).  I've submitted changes 
> which address this to the clamav folks(avoiding tempfile()), but they 
> have not been accepted as yet.  To avoid this issue (and avoid clamd 
> producing "ERROR: ScanStream: Can't create temporary file." messages), 
> setting MaxThreads to 1 should work, but will probably affect the 
> behavior of client software that talks to it (possibly causing deadlocks).
> 
> Do you have any insight to help address the underlying socket issues in 
> threaded programs would clearly help with clamd and every other multi 
> threaded program which may not even know it has these issues.

Unfortunately not, but Corinna had.
clamav might not accept the tempfile() workaround when it's clearly 
cygwin's/newlib fault.

But they still have to fix (think about) their SESSION/SCAN socket 
protocol as you wrote there. This was really a great catch. CRLF would 
be easy to add.
-- 
Reini Urban
http://xarch.tu-graz.ac.at/home/rurban/

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/