X-Recipient: archive-cygwin AT delorie DOT com
X-SWARE-Spam-Status: No, hits=-2.6 required=5.0	tests=AWL,BAYES_00,RCVD_IN_DNSWL_LOW,TW_FD
X-Spam-Check-By: sourceware.org
Message-ID: <4D88F89B.6000308@dronecode.org.uk>
Date: Tue, 22 Mar 2011 19:29:31 +0000
From: Jon TURNEY <jon DOT turney AT dronecode DOT org DOT uk>
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.15) Gecko/20110303 Thunderbird/3.1.9
MIME-Version: 1.0
To: cygwin AT cygwin DOT com
Subject: python select() is limited to fds < 64
Content-Type: multipart/mixed; boundary="------------050204040409010304030401"
Mailing-List: contact cygwin-help AT cygwin DOT com; run by ezmlm
List-Id: <cygwin.cygwin.com>
List-Subscribe: <mailto:cygwin-subscribe AT cygwin DOT com>
List-Archive: <http://sourceware.org/ml/cygwin/>
List-Post: <mailto:cygwin AT cygwin DOT com>
List-Help: <mailto:cygwin-help AT cygwin DOT com>, <http://sourceware.org/ml/#faqs>
Sender: cygwin-owner AT cygwin DOT com
Mail-Followup-To: cygwin AT cygwin DOT com
Delivered-To: mailing list cygwin AT cygwin DOT com

--------------050204040409010304030401
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit


python seems to be built with the default value of FD_SETSIZE, which is only
64 on cygwin.

I noticed this as it causes numerous tests in the twisted test suite to fail
with "ValueError: filedescriptor out of range in select()" exceptions, but can
also be demonstrated with a simple test case:

$ cat select_test.py

from socket import *
from select import select

ins = []

for i in range(1024):
    s = socket(AF_INET, SOCK_STREAM)
    ins.append(s)
    print "socket opened with fd", s.fileno()
    select(ins, [], [], 0)

$ python select_test.py
[...]
socket opened with fd 64
Traceback (most recent call last):
  File "./select_test.py", line 11, in <module>
    select(ins, [], [], 0)
ValueError: filedescriptor out of range in select()


Looking at the source [1], note that steps are already taken to increase the
default value of FD_SETSIZE on Win32, and I'd suggest it's appropriate to do
the same on cygwin, patch attached.

Note some code motion is necessary as FD_SETSIZE must be defined before
sys/types.h is included if we are going to override the default value set there.

I don't believe this can cause any ABI issues as the interface is in terms of
python lists, rather than fd_set values.

[1] http://hg.python.org/cpython/file/3c0edb157ea2/Modules/selectmodule.c


--------------050204040409010304030401
Content-Type: text/plain;
 name="2.6.5-FD_SETSIZE.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename="2.6.5-FD_SETSIZE.patch"

--- origsrc/Python-2.6.5/Modules/selectmodule.c	2009-10-27 15:39:53.000000000 +0000
+++ src/Python-2.6.5/Modules/selectmodule.c	2011-03-14 18:25:48.859375000 +0000
@@ -6,6 +6,21 @@
    >= 0.
 */
 
+/* Windows #defines FD_SETSIZE to 64 if FD_SETSIZE isn't already defined.
+   64 is too small (too many people have bumped into that limit).
+   Here we boost it.
+
+   Cygwin also defines FD_SETSIZE to 64, so also increase the limit on
+   Cygwin.  We must do this before sys/types.h is included, which otherwise
+   sets FD_SETSIZE to the default.
+
+   Users who want even more than the boosted limit should #define
+   FD_SETSIZE higher before this; e.g., via compiler /D switch.
+*/
+#if (defined(MS_WINDOWS) || defined(__CYGWIN__)) && !defined(FD_SETSIZE)
+#define FD_SETSIZE 512
+#endif
+
 #include "Python.h"
 #include <structmember.h>
 
@@ -16,16 +31,6 @@
 #undef HAVE_BROKEN_POLL
 #endif
 
-/* Windows #defines FD_SETSIZE to 64 if FD_SETSIZE isn't already defined.
-   64 is too small (too many people have bumped into that limit).
-   Here we boost it.
-   Users who want even more than the boosted limit should #define
-   FD_SETSIZE higher before this; e.g., via compiler /D switch.
-*/
-#if defined(MS_WINDOWS) && !defined(FD_SETSIZE)
-#define FD_SETSIZE 512
-#endif 
-
 #if defined(HAVE_POLL_H)
 #include <poll.h>
 #elif defined(HAVE_SYS_POLL_H)



--------------050204040409010304030401
Content-Type: text/plain; charset=us-ascii

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
--------------050204040409010304030401--