Mailing-List: contact cygwin-help AT sourceware DOT cygnus DOT com; run by ezmlm List-Subscribe: List-Archive: List-Post: List-Help: , Sender: cygwin-owner AT sources DOT redhat DOT com Delivered-To: mailing list cygwin AT sources DOT redhat DOT com Date: Tue, 1 Aug 2000 02:42:00 -0400 (EDT) From: "Ashif S. Harji" To: DJ Delorie cc: cygwin AT sourceware DOT cygnus DOT com, David Starks-Browning , Chris Faylor Subject: Re: user defined malloc In-Reply-To: <200007020245.WAA14181@envy.delorie.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII > > My question is do I have to also override the _malloc_r routine, and > > if so, could someone provide some information or a link on what is > > required? It appears to be a reentrant version of malloc. > > Yup, newlib is heavy into reentrancy. However, _malloc_r isn't > exported by cygwin1.dll and cygwin doesn't expect your program to > replace it. I'm not sure how much of a "bug" this is, but I'm open > for suggestions. Might want to ask on the newlib list also, I'm sure > they have really good reasons to call the reentrant malloc. > Note, if you wish to skip the preamble, the patches are included at the bottom of the message. As per the suggestion, I posted a message in the newlib mailing list. I received the following replies: http://sources.redhat.com/ml/newlib/2000/msg00143.html http://sources.redhat.com/ml/newlib/2000/msg00145.html Basically, it appears that because of reentrancy any reentrant library routines must call _malloc_r instead of malloc. Therefore, only overriding malloc can result in two separate allocation algorithms trying to manage the same memory. In fact, it does not matter if the same algorithm is used to manage the memory because the instances of the algorithms operate independently. Thus, the _r functions such as _malloc_r need to be overriden. Unfortunately, this is not currently possible. Note, it is probably sufficient to simply override the _r functions as malloc, for example, simply ends up calling _malloc_r anyways. Ideally, it would great if newlib included hooks similar to glibc that allowed the user to override the default memory allocation routines such as malloc, but this is currently not the case. Overriding _malloc_r is difficult because the cygwin1.dll appears to be statically linked. Therefore, all calls to _malloc_r are statically resolved and so all calls from inside the dll would ignore a user provided _malloc_r. The dll gets around this problem with malloc by storing the address of the user defined malloc (if one is provided) in the startup code of the program. As well, the dll provides its *own* version of malloc which either calls the user defined malloc or _malloc_r directly. This bypasses the version of malloc included in newlib because only one malloc can be included in the dll. Mimicking these semantics to override _malloc_r is complicated because only one version of _malloc_r can be included in the dll. If the dll provides its own version of _malloc_r, then, similar to malloc routine described above, this function can call the user defined _malloc_r if it exists (by using the startup code trick to store the address). However, in the default circumstance there is a problem as the actual _malloc_r defined in newlib is not available as a version is already defined by the dll. Note, malloc gets around this problem by calling _malloc_r in the default case and not the malloc provided by newlib. One possible solution is to try to change newlib so that _malloc_r calls another routine, for example, doMalloc which does the actual allocation. Then the version defined of _malloc_r in the dll can either call the user defined version of _malloc_r if it exists or call doMalloc directly. However, it is not clear that this solution provides any advantages to newlib. The patch given below tries to achieve these semantics without requiring any changes to newlib. It basically uses the --wrap parameter to the linker to rename calls to _malloc_r inside the dll. This change allows the dll to provide its own _malloc_r (called __wrap__malloc_r) and to include the _malloc_r inside of newlib (accessed via __real__malloc_r). So, any calls to _malloc_r in the dll get resolved to __wrap__malloc_r which can then call the user defined _malloc_r or the newlib version directly through __real__malloc_r. This patch is based on the cygwin 1.1.2 source. Let me know if I need to port it to the latest development version. Note, the check if ( user_data->_malloc_r != NULL ) { inside __wrap__malloc_r allows programs not compiled with these changes in the dll to continue to run. ************************* Patch Makefile.in *********************** --- back/Makefile.in Tue Jun 6 22:12:52 2000 +++ Makefile.in Sat Jul 22 01:39:16 2000 @@ -180,7 +180,8 @@ new-$(LIB_NAME): $(DEF_FILE) $(LIBCOS) # Rule to build cygwin.dll new-$(DLL_NAME): $(DLL_OFILES) $(DEF_FILE) $(DLL_IMPORTS) $(LIBC) $(LIBM) Makefile winver_stamp - $(LD) -shared -o $@ -e $(DLL_ENTRY) $(DEF_FILE) $(DLL_OFILES) version.o \ + $(LD) --wrap _malloc_r --wrap _free_r --wrap _calloc_r --wrap _realloc_r \ + -shared -o $@ -e $(DLL_ENTRY) $(DEF_FILE) $(DLL_OFILES) version.o \ winver.o $(DLL_IMPORTS) $(LIBM) $(LIBGCC) $(MALLOC_OBJ) $(LIBC) $(LIBGCC) dll_ofiles: $(DLL_OFILES) ************************* Patch libccrt0.cc *********************** --- back/libccrt0.cc Sun May 14 12:20:26 2000 +++ libccrt0.cc Sat Jul 22 01:35:42 2000 @@ -64,6 +64,11 @@ cygwin_crt0_common (MainFunc f) this_proc.realloc = &realloc; this_proc.calloc = &calloc; + this_proc._malloc_r = &_malloc_r; + this_proc._free_r = &_free_r; + this_proc._realloc_r = &_realloc_r; + this_proc._calloc_r = &_calloc_r; + /* Setup the module handle so fork can get the path name. */ this_proc.hmodule = GetModuleHandle (0); ************************* Patch malloc.cc *********************** --- back/malloc.cc Sun May 21 21:18:16 2000 +++ malloc.cc Sat Jul 22 01:35:10 2000 @@ -121,6 +121,73 @@ calloc (size_t nmemb, size_t size) res = user_data->calloc (nmemb, size); return res; } + +extern "C" +void * +__real__malloc_r (_reent * reent, size_t size); + +extern "C" +void +__real__free_r (_reent * reent, void *p); + +extern "C" +void * +__real__realloc_r (_reent * reent, void *p, size_t size); + +extern "C" +void * +__real__calloc_r (_reent * reent, size_t nmemb, size_t size); + +extern "C" +void * +__wrap__malloc_r (_reent * reent, size_t size) +{ + void *res; + if ( user_data->_malloc_r != NULL ) { + res = user_data->_malloc_r (reent, size); + } else { + res = __real__malloc_r (reent, size); + } + return res; +} + +extern "C" +void +__wrap__free_r (_reent * reent, void *p) +{ + if ( user_data->_free_r != NULL ) { + user_data->_free_r (reent, p); + } else { + __real__free_r (reent, p); + } +} + +extern "C" +void * +__wrap__realloc_r (_reent * reent, void *p, size_t size) +{ + void *res; + if ( user_data->_realloc_r != NULL ) { + res = user_data->_realloc_r (reent, p, size); + } else { + res = __real__realloc_r (reent, p, size); + } + return res; +} + +extern "C" +void * +__wrap__calloc_r (_reent * reent, size_t nmemb, size_t size) +{ + void *res; + if ( user_data->_calloc_r != NULL ) { + res = user_data->_calloc_r (reent, nmemb, size); + } else { + res = __real__calloc_r (reent, nmemb, size); + } + return res; +} + #endif /* These routines are used by the application if it @@ -173,6 +240,57 @@ export_calloc (size_t nmemb, size_t size res = _calloc_r (_impure_ptr, nmemb, size); else res = user_data->calloc (nmemb, size); + malloc_printf ("(%d, %d) = %x, called by %x", nmemb, size, res, ((int *)&nmemb)[-1]); + return res; +} + +extern "C" +void +export__free_r (_reent * reent, void *p) +{ + malloc_printf ("(%p), called by %x", p, ((int *)&p)[-1]); + if (use_internal_malloc) + __real__free_r (reent, p); + else + user_data->_free_r (reent, p); +} + +extern "C" +void * +export__malloc_r (_reent * reent, int size) +{ + void *res; + export_malloc_called = 1; + if (use_internal_malloc) + res = __real__malloc_r (reent, size); + else + res = user_data->_malloc_r (reent, size); + malloc_printf ("(%d) = %x, called by %x", size, res, ((int *)&size)[-1]); + return res; +} + +extern "C" +void * +export__realloc_r (_reent * reent, void *p, int size) +{ + void *res; + if (use_internal_malloc) + res = __real__realloc_r (reent, p, size); + else + res = user_data->_realloc_r (reent, p, size); + malloc_printf ("(%x, %d) = %x, called by %x", p, size, res, ((int *)&p)[-1]); + return res; +} + +extern "C" +void * +export__calloc_r (_reent * reent, size_t nmemb, size_t size) +{ + void *res; + if (use_internal_malloc) + res = __real__calloc_r (_impure_ptr, nmemb, size); + else + res = user_data->_calloc_r (reent, nmemb, size); malloc_printf ("(%d, %d) = %x, called by %x", nmemb, size, res, ((int *)&nmemb)[-1]); return res; } ************************* Patch winsup.h *********************** --- back/winsup.h Sun May 21 21:18:16 2000 +++ winsup.h Sat Jul 8 14:47:48 2000 @@ -142,8 +142,16 @@ class per_process void *bss_end; void *(*calloc)(size_t, size_t); + + /* Used to point to the memory machine we should use. Usually these + point back into the dll, but they can be overridden by the user. */ + void *(*_malloc_r)( _reent *, size_t); + void (*_free_r)( _reent *, void *); + void *(*_realloc_r)( _reent *, void *, size_t); + void *(*_calloc_r)( _reent *, size_t, size_t); + /* For future expansion of values set by the app. */ - void *public_reserved[4]; + //void *public_reserved[4]; /* The rest are *internal* to cygwin.dll. Those that are here because we want the child to inherit the value from ************************* Patch cygwin.din *********************** --- back/cygwin.din Sun May 14 12:20:24 2000 +++ cygwin.din Sat Jul 22 01:25:54 2000 @@ -75,6 +75,8 @@ cabsf _cabsf = cabsf calloc = export_calloc _calloc = export_calloc +_calloc_r = export__calloc_r +__calloc_r = export__calloc_r cbrt _cbrt = cbrt cbrtf @@ -269,6 +271,8 @@ fread _fread = fread free = export_free _free = export_free +_free_r = export__free_r +__free_r = export__free_r freopen _freopen = freopen frexp @@ -471,6 +475,8 @@ lstat _lstat = lstat malloc = export_malloc _malloc = export_malloc +_malloc_r = export__malloc_r +_malloc = export__malloc_r matherr _matherr = matherr mblen @@ -558,6 +564,8 @@ readv _readv = readv realloc = export_realloc _realloc = export_realloc +_realloc_r = export__realloc_r +__realloc_r = export__realloc_r regcomp _regcomp = regcomp regexec ******************************************************* Ashif Harji. -- Want to unsubscribe from this list? Send a message to cygwin-unsubscribe AT sourceware DOT cygnus DOT com