From: boncz AT wins DOT uva DOT nl (Peter Boncz) Subject: proposal: join me for thread-safeness 28 Oct 1997 02:10:41 -0800 Message-ID: <199710280958.KAA05194.cygnus.gnu-win32@chart.wins.uva.nl> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii To: gnu-win32 AT cygnus DOT com Dear List, In an earlier message ("quest for thread-safeness?") I asked in dismay about the status of threads in cygwin32. I had then just discovered that: * the current library (b18) is not thread-safe * as far as I know of, Cygnus Solutions has no plans for this at least in b19. * as far as I know of, nobody from this list is working on thread-safeness. There is no posix emulation of threads, but programmers using the win32 CreateThread() etc. functions directly (like I did) must realize that they must somehow ensure that only one of their threads at any time calls any of the cygwin functions. If two of them do simultaneously, inconsistencies and crashes may occur. For me and my multi-threaded database system Monet, thread-safeness is a 'must-have' feature. Ipatches things by locking explicitly around the 'unsafe' cygwin functions, but this both wastes performance and probably is not exhaustive (I analyzed the source to see which functions cause problems, but..). And some things are un-amendable: you really do not want to set a lock around the tcpip accept() call, as it tends to block for long spells of time. But it creates a new file-descriptor, thereby modifying the static and unprotected file-descriptor table in the winsup/ part of the library. Maybe I should have switched to Cygwin's commercial competitor UWIN (originally bell labs). But this emulation library is not thread-safe either, though they announced it as a feature of the next release. It is not clear when that is coming. My problem with UWIN is that it dos not work with the GNU compilers and requires Micro$oft Visual C++ on NT. Yeah, for this reason I want to stay with gnuwin, and hereby propose to head an effort to produce a thread-safe version of the library. now! technical assessment ==================== I analyzed the cygwin source code and came to the conclusion that thread-safeness is probably quite simple to achieve :-) ! To my surprice, the newlib/ (other than has been mentioned on this list) has been programmed with thread-safeness in mind. It uses the _REENT structure wherever appropriate. Please correct me if I'm wrong. It's just that there is just one instance of _REENT in cygwin.dll (called impure_ptr, or something like it) that makes newlib unsafe at this moment. Producing a different _REENT for each thread will *not* be difficult to implement since the WIN32 interface provides explicit functions to manipulate thread-local storage (Tls*()). Thus, it is not necessary in my view to modify gcc or the linker for dealing with .tls sections (as Gunther mentioned); you just redefine _REENT to the WIN32 call getting the thread-local data. Maybe it is not hyper-efficient, but it will do. And, by letting the impure_ptr pointing to the first reent struct, we can even achieve binary backward compatibility with single-threaded cygwin programs. Memory allocation seems to be thread-safe if the _REENT structure is handled correctly. This is in newlib/libc/stdlib. It leads to a package that manipulates memory on a pure thread-basis, though blocks allocated in one thread *can* be freed by another. If one thread is out of memory, but all others have a lot of free space, this implementation will grow the sbrk() anyhow, since it has no way of stealing away memory from the other threads. The advantage is though, that no locks are necessary, so it will be fast. Some tweaks will be needed to do something sensible with the free-list when a thread dies (letting it hang out there will cause severe memory fragmentation on systems that spawn and kill a lot of threads). So, the thing will come down to * recompiling the entire library with the new definition of _REEENT * some modifications of crt*.cc to set up the impure_ptr and the first Tls*() win32 memory. * making winsup/ thread-safe. * adding a posix compatible thread-package, that uses win32 functions like CreateThread(), KillThread() (manipulating the Tls*() reent [de]allocation), and provides locks and semaphores. The last point I more or less already did, be it that I wrote a package in my database system compatibility layer. It will need to be posixified. Now to the point of making winsup/ thread-safe. This will actually be the biggest work of the whole undertaking. I was a bit annoyed to see that no provisions were made to deal with multi-threadedness in the C++ winsup/ code. The cygnus people could have thought about it, included null defines for [un]setting locks.. I mean, isn't multithreading a basic feature of embedded systems? First of all, all static datastructures maintained by the library, need to be protected with locks.Care needs to be taken in order to avoid dead-lock. Some static data might be converted to local data. Othe static data, like returned by localtime() might need extra space in the reent datastructure, as it is static memory passed into user-space. Note that changing the _REENT structure will destroy backward compatibility.. conclusion ========== * I am willing to do this, in my spare time. At home I have a Win95 system. I would prefer to keep it that way (going to NT has other drawbacks for me). Developing on Win95 has the additional advantage that what is produced will probably run on NT out-of-the-box, where the other way around is always doubtful. * If there are people that see obvious mistakes in the above assessment, please let me know. * If there are people that want to help out, or are undertaking the same project at this very moment, please let me know. I would like to team up with someone who has NT, to cover both platforms. * Since my platform is Win95 I would like to wait for b19. Cygnus b19 on Win95 I assume will some of the compiling problems of b18 (at least, Sergey's patches do this and I assume them to be included in b19). Starting out with b19 will make patching up later much easier, of course. * Big question: when will b19 be due? cheers, -- === P.A. Boncz ================= phone: +31-20-525 7580 === WCW room F106 === === (Ph.D. student) email: boncz AT fwi DOT uva DOT nl === === Magnum Database Project PGP: finger boncz AT chart DOT fwi DOT uva DOT nl === === University of Amsterdam ====== WWW: http://www.fwi.uva.nl/~boncz/ ======= - For help on using this list (especially unsubscribing), send a message to "gnu-win32-request AT cygnus DOT com" with one line of text: "help".