DomainKey-Signature: a=rsa-sha1; c=nofws; d=sourceware.org; h=list-id
	:list-unsubscribe:list-subscribe:list-archive:list-post
	:list-help:sender:date:from:to:subject:message-id:reply-to
	:references:mime-version:content-type:in-reply-to; q=dns; s=
	default; b=SqbBq3aegodPObFYk+6lC7pNf4yuz0sYl+phIWBT18yiA6cCMv+sh
	wGohTWUyR5Qql8hi6yS8NyOo3FIvXhxAC6cDfUhzwPvwQ4b4a7mMPNz/+8rXsOpO
	VdCLlCgnLQX+SYKErcAmYyWRetOOJUPOwteJGG80KVNRjsu7nEcRcc=
Mailing-List: contact cygwin-help AT cygwin DOT com; run by ezmlm
Sender: cygwin-owner AT cygwin DOT com
Mail-Followup-To: cygwin AT cygwin DOT com
Date: Thu, 31 Oct 2013 22:59:11 -0400
From: Christopher Faylor <cgf-use-the-mailinglist-please AT cygwin DOT com>
To: cygwin AT cygwin DOT com
Subject: Re: Problem with multiprocessing module from Python
Message-ID: <20131101025911.GA2302@ednor.casa.cgf.cx>
Reply-To: cygwin AT cygwin DOT com
Mail-Followup-To: cygwin AT cygwin DOT com
References: <l4ei86$cos$2 AT ger DOT gmane DOT org> <l4p2uq$ps8$1 AT ger DOT gmane DOT org> <l4p6h3$801$1 AT ger DOT gmane DOT org> <20131029205935 DOT GC392 AT ednor DOT casa DOT cgf DOT cx> <l4p8oi$24u$1 AT ger DOT gmane DOT org> <l4p8vj$24u$3 AT ger DOT gmane DOT org> <20131030093313 DOT GA28558 AT calimero DOT vinschen DOT de> <l4u7er$2vj$1 AT ger DOT gmane DOT org> <20131031184114 DOT GD6599 AT ednor DOT casa DOT cgf DOT cx> <l4uflu$rrm$1 AT ger DOT gmane DOT org>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <l4uflu$rrm$1@ger.gmane.org>
User-Agent: Mutt/1.5.20 (2009-06-14)

On Thu, Oct 31, 2013 at 08:47:58PM +0000, Jean-Pierre Flori wrote:
>Le Thu, 31 Oct 2013 14:41:14 -0400, Christopher Faylor a ??crit??:
>
>> On Thu, Oct 31, 2013 at 06:27:39PM +0000, Jean-Pierre Flori wrote:
>>>Le Wed, 30 Oct 2013 10:33:13 +0100, Corinna Vinschen a ??crit??:
>>>
>>>> On Oct 29 21:22, Jean-Pierre Flori wrote:
>>>>> Le Tue, 29 Oct 2013 21:19:14 +0000, Jean-Pierre Flori a ??crit??:
>>>>> 
>>>>> > Le Tue, 29 Oct 2013 16:59:35 -0400, Christopher Faylor a ??crit??:
>>>>> >> If you want this fixed, the easiest way to get that to happen is
>>>>> >> to post a simple test case which reproduces the problem.  That is
>>>>> >> not the code snippet that you sent.  A real working example would
>>>>> >> be required.
>>>>> > Sorry about that.
>>>>> > 
>>>>> > Here you go:
>>>>> > """
>>>>> > from multiprocessing import Pool
>>>>> > 
>>>>> > def f(x): return x
>>>>> > 
>>>>> > p = pool(2)
>>>>> > 
>>>>> > p.map(f, [1, 2])
>>>>> > """
>>>>> And I managed to introduce a typo. The third line should read Pool,
>>>>> so it is:
>>>>> """
>>>>> from multiprocessing import Pool
>>>>> 
>>>>> def f(x): return x
>>>>> 
>>>>> p = Pool(2)
>>>>> 
>>>>> p.map(f, [1, 2])
>>>>> """
>>>> 
>>>> Works for me.  I guess.  At least, if I run the script, nothing
>>>> happens:
>>>> 
>>>>   $ python x.py $
>>>> 
>>>> Same on 32 and 64 bit Cygwin.
>>>> 
>>>> 
>>>> Corinna
>>>
>>>I think I got to the bottom of this.
>>>It seems the new implem of sem_getvalue in cgwin1.dll is the cause, see:
>>>http://cygwin.com/ml/cygwin-patches/2013-q3/msg00006.html It may also
>>>explain the random reproducibility if sval stays uninitialized or
>>>something like that (I did not check it is the case though).
>> 
>> I doubt that was the problem.  More likely it is something related to
>> the changes in thread.cc that followed that change.
>> 
>> cgf
>
>I must admit that was just a guess.
>The thread.cc code changed whereas python code in Modules/
>_multiprocessing/semaphore.c did not.
>The python multiprocessing module (file semaphore.c) contains:
>"""
>        if (sem_getvalue(self->handle, &sval) < 0) {
>            return PyErr_SetFromErrno(PyExc_OSError);
>        } else if (sval >= self->maxvalue) {
>            PyErr_SetString(PyExc_ValueError, "semaphore or lock "
>                            "released too many times");
>            return NULL;
>        }
>"""
>Changing this to
>"""
>        sval = sem_getvalue(self->handle, &sval);
>        if (sem_getvalue(self->handle, &sval) < 0) {
>            return PyErr_SetFromErrno(PyExc_OSError);
>        } else if (sval >= self->maxvalue) {
>            PyErr_SetString(PyExc_ValueError, "semaphore or lock "
>                            "released too many times");
>            return NULL;
>        }
>"""
>to emulate the previous behavior of sem_getvalue seems to solve my problem 
>(and was easier than rebuilding cygwin1.dll).

That doesn't emulate the previous behavior.  The return value of sem_getvalue
was changed to 0 or -1 as per POSIX.  If self->maxvalue is > 1 then
you won't see an error but it won't be correct either.

cgf

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple