DomainKey-Signature: a=rsa-sha1; c=nofws; d=sourceware.org; h=list-id
	:list-unsubscribe:list-subscribe:list-archive:list-post
	:list-help:sender:date:from:to:subject:message-id:reply-to
	:references:mime-version:content-type:in-reply-to; q=dns; s=
	default; b=ZxMh8u9VH2/u7LYHsWOBfhAw7medy4yTTC2kyyRsJI5exacuVLevk
	LNYORTEkR8wyKoV/44HRLoQ2t6aRItyX6+e0yUBTKynXCw2P3xetF32RmIMBNU4G
	8RYPw4/ZOoc6w9QgeZKdoDeccPr28CdN95B/SVD49PXVdcQo8Nh06A=
Mailing-List: contact cygwin-help AT cygwin DOT com; run by ezmlm
Sender: cygwin-owner AT cygwin DOT com
Mail-Followup-To: cygwin AT cygwin DOT com
Date: Thu, 31 Oct 2013 23:25:00 -0400
From: Christopher Faylor <cgf-use-the-mailinglist-please AT cygwin DOT com>
To: cygwin AT cygwin DOT com
Subject: Re: Problem with multiprocessing module from Python
Message-ID: <20131101032500.GB2302@ednor.casa.cgf.cx>
Reply-To: cygwin AT cygwin DOT com
Mail-Followup-To: cygwin AT cygwin DOT com
References: <l4p2uq$ps8$1 AT ger DOT gmane DOT org> <l4p6h3$801$1 AT ger DOT gmane DOT org> <20131029205935 DOT GC392 AT ednor DOT casa DOT cgf DOT cx> <l4p8oi$24u$1 AT ger DOT gmane DOT org> <l4p8vj$24u$3 AT ger DOT gmane DOT org> <20131030093313 DOT GA28558 AT calimero DOT vinschen DOT de> <l4u7er$2vj$1 AT ger DOT gmane DOT org> <20131031184114 DOT GD6599 AT ednor DOT casa DOT cgf DOT cx> <l4uflu$rrm$1 AT ger DOT gmane DOT org> <20131101025911 DOT GA2302 AT ednor DOT casa DOT cgf DOT cx>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20131101025911.GA2302@ednor.casa.cgf.cx>
User-Agent: Mutt/1.5.20 (2009-06-14)

On Thu, Oct 31, 2013 at 10:59:11PM -0400, Christopher Faylor wrote:
>On Thu, Oct 31, 2013 at 08:47:58PM +0000, Jean-Pierre Flori wrote:
>>Le Thu, 31 Oct 2013 14:41:14 -0400, Christopher Faylor a ??crit??:
>>
>>> On Thu, Oct 31, 2013 at 06:27:39PM +0000, Jean-Pierre Flori wrote:
>>>>Le Wed, 30 Oct 2013 10:33:13 +0100, Corinna Vinschen a ??crit??:
>>>>
>>>>> On Oct 29 21:22, Jean-Pierre Flori wrote:
>>>>>> Le Tue, 29 Oct 2013 21:19:14 +0000, Jean-Pierre Flori a ??crit??:
>>>>>> 
>>>>>> > Le Tue, 29 Oct 2013 16:59:35 -0400, Christopher Faylor a ??crit??:
>>>>>> >> If you want this fixed, the easiest way to get that to happen is
>>>>>> >> to post a simple test case which reproduces the problem.  That is
>>>>>> >> not the code snippet that you sent.  A real working example would
>>>>>> >> be required.
>>>>>> > Sorry about that.
>>>>>> > 
>>>>>> > Here you go:
>>>>>> > """
>>>>>> > from multiprocessing import Pool
>>>>>> > 
>>>>>> > def f(x): return x
>>>>>> > 
>>>>>> > p = pool(2)
>>>>>> > 
>>>>>> > p.map(f, [1, 2])
>>>>>> > """
>>>>>> And I managed to introduce a typo. The third line should read Pool,
>>>>>> so it is:
>>>>>> """
>>>>>> from multiprocessing import Pool
>>>>>> 
>>>>>> def f(x): return x
>>>>>> 
>>>>>> p = Pool(2)
>>>>>> 
>>>>>> p.map(f, [1, 2])
>>>>>> """
>>>>> 
>>>>> Works for me.  I guess.  At least, if I run the script, nothing
>>>>> happens:
>>>>> 
>>>>>   $ python x.py $
>>>>> 
>>>>> Same on 32 and 64 bit Cygwin.
>>>>> 
>>>>> 
>>>>> Corinna
>>>>
>>>>I think I got to the bottom of this.
>>>>It seems the new implem of sem_getvalue in cgwin1.dll is the cause, see:
>>>>http://cygwin.com/ml/cygwin-patches/2013-q3/msg00006.html It may also
>>>>explain the random reproducibility if sval stays uninitialized or
>>>>something like that (I did not check it is the case though).
>>> 
>>> I doubt that was the problem.  More likely it is something related to
>>> the changes in thread.cc that followed that change.
>>> 
>>> cgf
>>
>>I must admit that was just a guess.
>>The thread.cc code changed whereas python code in Modules/
>>_multiprocessing/semaphore.c did not.
>>The python multiprocessing module (file semaphore.c) contains:
>>"""
>>        if (sem_getvalue(self->handle, &sval) < 0) {
>>            return PyErr_SetFromErrno(PyExc_OSError);
>>        } else if (sval >= self->maxvalue) {
>>            PyErr_SetString(PyExc_ValueError, "semaphore or lock "
>>                            "released too many times");
>>            return NULL;
>>        }
>>"""
>>Changing this to
>>"""
>>        sval = sem_getvalue(self->handle, &sval);
>>        if (sem_getvalue(self->handle, &sval) < 0) {
>>            return PyErr_SetFromErrno(PyExc_OSError);
>>        } else if (sval >= self->maxvalue) {
>>            PyErr_SetString(PyExc_ValueError, "semaphore or lock "
>>                            "released too many times");
>>            return NULL;
>>        }
>>"""
>>to emulate the previous behavior of sem_getvalue seems to solve my problem 
>>(and was easier than rebuilding cygwin1.dll).
>
>That doesn't emulate the previous behavior.  The return value of sem_getvalue
>was changed to 0 or -1 as per POSIX.  If self->maxvalue is > 1 then
>you won't see an error but it won't be correct either.

Ok.  I was confused by the seemingly contradictory assertions that the
patch from http://cygwin.com/ml/cygwin-patches/2013-q3/msg00006.html was
both the cause and the solution for the problem.  And, I thought my change
to sem_getvalue had gotten into 1.7.25.  It hadn't.

So, it's my night to be wrong.  Larry, however, is not suffering
similarly: He was right.  Snapshots after 9/25 will be the only versions
of the DLL that have the fix for this problem.  It does seem like the
problem was introduced in changes to thread.cc months before the
abovementioned patch ever went in.

http://cygwin.com/snapshots/

cgf

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple