DomainKey-Signature: a=rsa-sha1; c=nofws; d=sourceware.org; h=list-id
	:list-unsubscribe:list-subscribe:list-archive:list-post
	:list-help:sender:to:from:subject:date:message-id:references
	:mime-version:content-type:content-transfer-encoding; q=dns; s=
	default; b=lWdZ4eC+I6UmXe2zI+HO7tyoxvoTScOO6GnRoG728gxVd2Y5jb3Yl
	KYlXvqXFu/Cumbmb2fu75TK6k2+t4UGyyYhuQCuKOGjTz5ksqnFbWZi6ZKEPlYiW
	dlpEGwo06g8yqA9Dn7oHxXOZPKUl6QQs0nIr0Mxtx5KKvUlrw7YHlY=
Mailing-List: contact cygwin-help@cygwin.com; run by ezmlm
Sender: cygwin-owner@cygwin.com
Mail-Followup-To: cygwin@cygwin.com
To: cygwin@cygwin.com
From: Jean-Pierre Flori <jpflori@gmail.com>
Subject: Re: Problem with multiprocessing module from Python
Date: Thu, 31 Oct 2013 20:47:58 +0000 (UTC)
Lines: 94
Message-ID: <l4uflu$rrm$1@ger.gmane.org>
References: 	<CAHhGz88t8H_xQ6h-a6-aQrMf+wTOVT6gMnx42tMpjs=3=qGVTQ@mail.gmail.com>	<l4ei86$cos$2@ger.gmane.org> <l4p2uq$ps8$1@ger.gmane.org>	<l4p6h3$801$1@ger.gmane.org> <20131029205935.GC392@ednor.casa.cgf.cx>	<l4p8oi$24u$1@ger.gmane.org> <l4p8vj$24u$3@ger.gmane.org>	<20131030093313.GA28558@calimero.vinschen.de> <l4u7er$2vj$1@ger.gmane.org>	<20131031184114.GD6599@ednor.casa.cgf.cx>
Mime-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
User-Agent: Pan/0.139 (Sexual Chocolate; GIT bf56508 git://git.gnome.org/pan2)

Le Thu, 31 Oct 2013 14:41:14 -0400, Christopher Faylor a écrit :

> On Thu, Oct 31, 2013 at 06:27:39PM +0000, Jean-Pierre Flori wrote:
>>Le Wed, 30 Oct 2013 10:33:13 +0100, Corinna Vinschen a ??crit??:
>>
>>> On Oct 29 21:22, Jean-Pierre Flori wrote:
>>>> Le Tue, 29 Oct 2013 21:19:14 +0000, Jean-Pierre Flori a ??crit??:
>>>> 
>>>> > Le Tue, 29 Oct 2013 16:59:35 -0400, Christopher Faylor a ??crit??:
>>>> >> If you want this fixed, the easiest way to get that to happen is
>>>> >> to post a simple test case which reproduces the problem.  That is
>>>> >> not the code snippet that you sent.  A real working example would
>>>> >> be required.
>>>> > Sorry about that.
>>>> > 
>>>> > Here you go:
>>>> > """
>>>> > from multiprocessing import Pool
>>>> > 
>>>> > def f(x): return x
>>>> > 
>>>> > p = pool(2)
>>>> > 
>>>> > p.map(f, [1, 2])
>>>> > """
>>>> And I managed to introduce a typo. The third line should read Pool,
>>>> so it is:
>>>> """
>>>> from multiprocessing import Pool
>>>> 
>>>> def f(x): return x
>>>> 
>>>> p = Pool(2)
>>>> 
>>>> p.map(f, [1, 2])
>>>> """
>>> 
>>> Works for me.  I guess.  At least, if I run the script, nothing
>>> happens:
>>> 
>>>   $ python x.py $
>>> 
>>> Same on 32 and 64 bit Cygwin.
>>> 
>>> 
>>> Corinna
>>
>>I think I got to the bottom of this.
>>It seems the new implem of sem_getvalue in cgwin1.dll is the cause, see:
>>http://cygwin.com/ml/cygwin-patches/2013-q3/msg00006.html It may also
>>explain the random reproducibility if sval stays uninitialized or
>>something like that (I did not check it is the case though).
> 
> I doubt that was the problem.  More likely it is something related to
> the changes in thread.cc that followed that change.
> 
> cgf

I must admit that was just a guess.
The thread.cc code changed whereas python code in Modules/
_multiprocessing/semaphore.c did not.
The python multiprocessing module (file semaphore.c) contains:
"""
        if (sem_getvalue(self->handle, &sval) < 0) {
            return PyErr_SetFromErrno(PyExc_OSError);
        } else if (sval >= self->maxvalue) {
            PyErr_SetString(PyExc_ValueError, "semaphore or lock "
                            "released too many times");
            return NULL;
        }
"""
Changing this to
"""
        sval = sem_getvalue(self->handle, &sval);
        if (sem_getvalue(self->handle, &sval) < 0) {
            return PyErr_SetFromErrno(PyExc_OSError);
        } else if (sval >= self->maxvalue) {
            PyErr_SetString(PyExc_ValueError, "semaphore or lock "
                            "released too many times");
            return NULL;
        }
"""
to emulate the previous behavior of sem_getvalue seems to solve my problem 
(and was easier than rebuilding cygwin1.dll).

Commit http://cygwin.com/cgi-bin/cvsweb.cgi/src/winsup/cygwin/thread.cc?
rev=1.286&content-type=text/x-cvsweb-markup&cvsroot=src
would be exactly what I need.
I'll just be waiting for 1.7.26.

Hopefully this will get sorted out.

Best,
JP


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple