Message-ID: <3EB08219.3D55D32A@yahoo.com> Date: Wed, 30 Apr 2003 22:10:33 -0400 From: CBFalconer Organization: Ched Research X-Mailer: Mozilla 4.75 [en] (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: djgpp-workers AT delorie DOT com Subject: Re: nmalloc documentation revisions References: <3EAF5020 DOT 9E3C6C9A AT yahoo DOT com> <7458-Wed30Apr2003210444+0300-eliz AT elta DOT co DOT il> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Reply-To: djgpp-workers AT delorie DOT com Eli Zaretskii wrote: > ... snip ... > > Also, please produce the diffs like this: > > diff -c malloc.txh nmalloc.txh > > Note the "-c" switch--it's very important. Sigh. I guess I have been too used to being chief honcho :-) Try this. *** malloco.txh Sun Dec 8 05:57:28 2002 --- nmallocn.txh Wed Apr 30 21:32:22 2003 *************** *** 29,35 **** @subheading Example @example ! char *c = (char *)malloc(100); @end example @c ---------------------------------------------------------------------- --- 29,35 ---- @subheading Example @example ! char *c = malloc(100); @end example @c ---------------------------------------------------------------------- *************** *** 59,65 **** @subheading Example @example ! char *q = (char *)malloc(20); free(q); @end example --- 59,65 ---- @subheading Example @example ! char *q = malloc(20); free(q); @end example *************** *** 102,115 **** @subheading Example @example ! if (now+new > max) @{ char *old = p; ! max = now+new; p = realloc(p, max); ! if (p == NULL) p = old; /* retain the old pointer */ @} @end example --- 102,118 ---- @subheading Example @example ! if (now + new > max) @{ char *old = p; ! oldmax = max; ! max = now + new; p = realloc(p, max); ! if (p == NULL) @{ p = old; /* retain the old pointer */ + max = oldmax; + @} @} @end example *************** *** 120,125 **** --- 123,129 ---- @example #include + #include /* on non-DJGPP systems */ struct mallinfo mallinfo(void); @end example *************** *** 165,189 **** @item smblks The number of ``small blocks''. This is normally zero, unless ! @code{malloc} was compiled with the symbol @code{NUMSMALL} defined to a ! non-zero value. Doing so activates an optional algorithm which serves ! small allocations quickly from a special pool. If this option is ! activated, the @code{smblks} member returns the number of free small ! blocks (the allocated small blocks are included in the value of ! @code{ordblks}). @item hblks @itemx hblkhd ! Always zero, kept for compatibility with other systems. @item usmblks The space (in bytes) in ``small blocks'' that are in use. This is always zero in the DJGPP implementation. @item fsmblks ! The space in free ``small blocks''. Non-zero only of @code{malloc} was ! compiled with @code{NUMSMALL} defined to a non-zero value. In that ! case, gives the amount of space in bytes in free small blocks. @item uordblks The amount of space, in bytes, in the heap space currently used by the --- 169,192 ---- @item smblks The number of ``small blocks''. This is normally zero, unless ! a special version of @code{malloc} was installed. @item hblks + The count of freeblocks maintained in the malloc arena. This + is normally much smaller than the number of freed blocks, since + such blocks have been combined into larger blocks. + @itemx hblkhd ! The size of the currently available top portion of the memory ! assigned by sbrk. Sometimes known as 'slop'. @item usmblks The space (in bytes) in ``small blocks'' that are in use. This is always zero in the DJGPP implementation. @item fsmblks ! The space in free ``small blocks''. Normally zero in this ! implementation. @item uordblks The amount of space, in bytes, in the heap space currently used by the *************** *** 196,202 **** list. @item keepcost ! Always zero, kept for compatibility. @end table @subheading Return Value --- 199,206 ---- list. @item keepcost ! The total overhead used by @code(malloc) in maintaining the ! currently allocated blocks. @end table @subheading Return Value *************** *** 225,230 **** --- 229,235 ---- @example #include + #include /* on non-DJGPP systems */ int malloc_verify(void); @end example *************** *** 236,247 **** application, and also all the free blocks maintained by @code{malloc} and @code{free} in the internal free list. Each block is checked for consistency of the hidden bookkeeping information recorded in it by ! @code{malloc} and @code{free}. The blocks on the free list are ! additionally validated by chasing all the @code{next} pointers in the ! linked list and checking them against limits for valid pointers (between ! 0x1000 and the data segment limit), and the alignment. (Unaligned ! pointers are probably corrupted, since @code{malloc} always returns a ! properly aligned storage.) What happens when a bad block is found depends on the current @dfn{malloc diagnostics level}: for example, the block can be reported, --- 241,247 ---- application, and also all the free blocks maintained by @code{malloc} and @code{free} in the internal free list. Each block is checked for consistency of the hidden bookkeeping information recorded in it by ! @code{malloc} and @code{free}. What happens when a bad block is found depends on the current @dfn{malloc diagnostics level}: for example, the block can be reported, *************** *** 272,277 **** --- 272,278 ---- @example #include + #include /* on non-DJGPP systems */ int malloc_debug(int level); @end example *************** *** 283,338 **** functions which call them internally. The argument @var{level} is interpreted as follows: ! @table @asis @item Level 0 ! No checking; the memory allocation functions behave as they do if ! @code{malloc_debug} was never called. Memory in use by the application ! which was allocated while level 0 was in effect cannot be checked by ! @code{malloc_verify} unless it is @code{free}d first. @item Level 1 ! Each one of the allocated blocks is recorded in a special structure, ! where @code{malloc_verify} can test them for corruption, even if these ! blocks were not yet @code{free}d. If errors are detected by ! @code{malloc_verify}, it prints diagnostic messages to the standard ! error stream, with address and size of the offending block and other ! pertinent information. This level slows down memory allocation to some ! extent due to additional overhead of calling special functions which ! record extra debugging info. @item Level 2 ! Like level 1, but in addition the consistency of the entire heap is ! verified (by calling @code{malloc_verify}) on every call to the memory ! allocation functions. @emph{Warning: this may significantly slow down ! the application.} @item Level 3 Like level 2, except that the program is aborted whenever a heap corruption is detected. In addition, failed allocations (i.e.@: when @code{malloc} returns @code{NULL} because it cannot satisfy a request) ! are reported to standard error. Also, if the storage where allocated ! blocks are recorded is exhausted, a message to that effect is printed. @item Level 4 Like level 3, but calls to @code{free} with a @code{NULL} pointer as an argument are also reported. - @end table ! When @code{malloc_debug} is first called with a positive argument, it ! allocates storage for recording blocks in use. To avoid reentrancy ! problems, this storage is allocated via a direct call to @code{sbrk}, ! and its size is fixed. The size used to allocate this storage is by ! default 400KB, which is enough to record 100 thousand allocated blocks. ! You can tailor the size to your needs by setting the environment ! variable @code{MALLOC_DEBUG} to the maximum number of blocks you want to ! be able to track. (The value of @code{MALLOC_DEBUG} should only be as ! large as the maximum number of allocations which is expected to be in ! use at any given time, because when a buffer is freed, it is removed ! from this storage and its cell can be reused for another allocation.) ! Note that the larger this storage size, the more slow-down will your ! program experience when the diagnostic level is set to a non-zero value, ! since the debugging code needs to search the list of recorded blocks in ! use each time you call @code{malloc} or @code{free}. @subheading Return Value --- 284,329 ---- functions which call them internally. The argument @var{level} is interpreted as follows: ! @table @asis @item Level 0 ! The memory allocation functions behave as they do if ! @code{malloc_debug} was never called. Only passive checks are ! allowed, including malloc_verify, mallocmap, and malldbgdumpfile ! (to set the output file). In particular mallsethook() is ! disabled, to avoid accidents. The actual functioning of the ! malloc system remains unaffected. Any malloc hooks are reset. ! If errors are detected by @code{malloc_verify}, it prints ! diagnostic messages to the dumpfile (default standard error ! stream), with address and size of the offending block and other ! pertinent information. @item Level 1 ! All the passive checks of level 0 are allowed. In addition ! mallsethook() is enabled, thus allowing construction of custom ! debugging packages. @item Level 2 ! Similar to level 1, but the consistency of the entire heap is ! also verified (by calling @code{malloc_verify}) on every call to ! the memory allocation functions. @emph{Warning: this may ! significantly slow down the application.} Since the malloc ! hooks are in use to connect to this system, mallsethook is ! disabled. If there are hooks already in use the change to level ! 2 or above is rejected. @item Level 3 Like level 2, except that the program is aborted whenever a heap corruption is detected. In addition, failed allocations (i.e.@: when @code{malloc} returns @code{NULL} because it cannot satisfy a request) ! are reported to the dumpfile (default standard error). @item Level 4 Like level 3, but calls to @code{free} with a @code{NULL} pointer as an argument are also reported. ! @item Level -1 ! Does not alter the debug level, but returns the current value. ! @end table @subheading Return Value *************** *** 359,378 **** @example #include ! void mallocmap(int level); @end example @subheading Description ! This function prints a map of the heap storage to standard output. For ! each block, its address and size are printed, as well as an indication ! whether it is free or in use. If the @dfn{slop} (a special free block ! cached for performance reasons) and the small blocks are available, they ! are printed as well (these two are variants of free blocks). Blocks in ! use will only be printed if the diagnostic level was set to a non-zero ! value by a call to @code{malloc_debug} (@pxref{malloc_debug}), since ! otherwise the allocated blocks are not recorded by @code{malloc}. @subheading Return Value --- 350,366 ---- @example #include + #include /* on non-DJGPP systems */ ! void mallocmap(void); @end example @subheading Description ! This function prints a map of the heap storage to the selected ! dumpfile. (See malldbgdumpfile). For each block, its address and ! size are printed, as well as an indication whether it is free or ! in use, together with pointers to adjacent blocks. @subheading Return Value *************** *** 385,468 **** @c ---------------------------------------------------------------------- ! @node malloc hook functions, memory @subheading Syntax @example #include ! #include ! void (*__libc_malloc_hook)(size_t size, void *block); ! void (*__libc_malloc_fail_hook)(size_t size); ! void (*__libc_free_hook)(void *block); ! void (*__libc_free_null_hook)(void); ! void (*__libc_realloc_hook)(void *block, size_t size); @end example @subheading Description ! These hooks are provided for building custom @code{malloc} debugging ! packages. Such packages typically need to be notified when memory is ! allocated and freed by the application, in order to be able to find ! memory leaks, code that writes beyond the limits of allocated buffers or ! attempts to free buffers which were not allocated by @code{malloc}, etc. ! These hooks can be used to define callback functions which will be ! called by the library at strategic points. Each callback is only called ! if it is non- AT code{NULL}; by default, all of them are initialized to a @code{NULL} value. @table @code - @item __libc_malloc_hook - Called just before a chunk of memory is about to be returned to the - application in response to an allocation request. @var{size} is the - size requested by the application (@strong{not} the actual size of the - allocated buffer, which may be larger). @var{block} is a pointer to the - block that was allocated, which is 4 bytes before the pointer that - @code{malloc} will return to the application; these 4 bytes are used to - record the actual size of the buffer. An additional copy of the block's - size is recorded immediately after the buffer's end. Thus, - @w{@code{*(size_t *)((char *)block + 4 + (BLOCK *)block->size)}} gives - the second copy of the block's size. - - @item __libc_malloc_fail_hook - Called if @code{malloc} failed to find a free block large enough to - satisfy a request, and also failed to obtain additional memory from - @code{sbrk}. @var{size} is the requested allocation size. - - @item __libc_free_hook - Called when a buffer is about to be freed. @var{block} is a pointer 4 - bytes before the address passed to @code{free} by the application, - i.e.@: it is a pointer to the beginning of the size information - maintained before the user buffer. ! @item __libc_free_null_hook Called whenever a @code{NULL} pointer is passed to @code{free}. ! @acronym{ANSI} C specifically rules that this is allowed and should have ! no effect, but you might want to catch such cases if your program needs ! to be portable to old compilers whose libraries don't like @code{NULL} ! pointers in @code{free}. ! ! @item __libc_realloc_hook ! Called at entry to @code{realloc}, before the actual reallocation. ! @var{block} is a pointer 4 bytes before the address passed to ! @code{free} by the application, i.e.@: it is a pointer to the beginning ! of the size information maintained before the user buffer. @var{size} ! is the new size requested by the application. (This hook is called ! @emph{in addition} to the other hooks which will be called by ! @code{free} and @code{malloc} if and when @code{realloc} calls them.) @end table ! The @code{BLOCK} data type is used by @code{malloc} and @code{free} to ! maintain the heap. The only member which is always guaranteed to be ! valid is @code{size} (the additional copy of the size, recorded beyond ! the buffer's end, is also guaranteed to be valid). The @code{next} ! member is valid in all blocks that are part of the free list. This ! means that @code{__libc_malloc_hook} can use the @code{next} member, but ! @code{__libc_free_hook} cannot. @subheading Portability @portability !ansi, !posix ! These hooks are specific to DJGPP. --- 373,608 ---- @c ---------------------------------------------------------------------- ! @node mallsethook, memory @subheading Syntax @example #include ! #include /* on non-DJGPP systems */ ! M_HOOKFN mallsethook(enum m_hook_kind which, ! M_HOOKFN newhook); @end example @subheading Description ! These hooks are provided for building custom @code{malloc} ! debugging packages. Such packages typically need to be notified ! when memory is allocated and freed by the application, in order ! to be able to find memory leaks, code that writes beyond the ! limits of allocated buffers or attempts to free buffers which ! were not allocated by @code{malloc}, etc. These hooks can be ! used to define callback functions which will be called by the ! library at strategic points. Each callback is only called if it ! is non- AT code{NULL}; by default, all of them are initialized to a @code{NULL} value. + The actual hook to be set is described by the following + constants, enumerated in stdlib.h: + + malloc_HK = 0 on entry to malloc + malloc_fail_HK when malloc fails + free_HK on entry to free + free_null_HK when free is called with a NULL pointer + realloc_HK on entry to realloc + realloc_exit_HK when realloc exits. + HKCOUNT The first invalid value for a hook descriptor. + + and each ``callback'' function is described by: + + typedef void (*M_HOOKFN)(size_t sz, void *ptr); + + in stdlib.h. Note that in some cases the parameters supplied to + the ``callback'' function will be meaningless, in which case + they will be zero or NULL as appropriate. The callback function + must NOT call any of malloc, free, or realloc. + + mallsethook action is disabled unless the debug_level is set to + 1. This interlock avoids accidental use before enabling the + package, and avoids altering hooks that are already in use for + debug_level of 2 or higher. + + For access to the internal storage of the malloc arena, see the + _sysmalloc() call and the _sysinfo struct. + @table @code ! @item malloc_HK ! ! Called just before a chunk of memory is about to be returned to ! the application in response to an allocation request. ! @var{sz} is the size requested by the application ! (@strong{not} the actual size of the allocated buffer, which may ! be larger). @var{ptr} is a pointer to the block that was ! allocated. ! ! @item malloc_fail_HK ! ! Called if @code{malloc} failed to find a free block large enough ! to satisfy a request, and also failed to obtain additional ! memory from @code{sbrk}. @var{sz} is the requested allocation ! size. ! ! @item free_HK ! ! Called when a buffer is about to be freed. @var{ptr} is a ! pointer to the block to be freed. Its validity has not been ! checked. ! ! @item free_null_HK ! Called whenever a @code{NULL} pointer is passed to @code{free}. ! @acronym{ANSI} C specifically rules that this is allowed and ! should have no effect, but you might want to catch such cases if ! your program needs to be portable to old compilers whose ! libraries don't like @code{NULL} pointers in @code{free}. ! ! @item realloc_HK ! ! Called at entry to @code{realloc}, before the actual ! reallocation. @var{ptr} is the pointer whose contents are to be ! reallocated. @var{sz} is the new size requested by the ! application. (This hook is called @emph{in addition} to the ! other hooks which may be called by @code{malloc} if and when ! @code{realloc} calls them.) ! ! @item realloc_exit_HK ! ! Called at exit from realloc. @var(sz) is the requested size, ! and @var(ptr) is the (possibly) revised value of the user ! pointer which will be returned my realloc. This may be NULL if ! realloc was unable to allocate the space, in which case ONLY the ! original value of @var(ptr) (as reported by realloc_HK) remains ! valid. ! @end table ! @subheading Return Value ! ! @code{mallsethook} returns the previous hook pointer, (usually ! NULL). If mallsethook is disabled it will also return NULL ! ! @subheading Portability ! ! @portability !ansi, !posix ! ! These hooks and the function are specific to DJGPP. ! ! @subheading Example ! ! @example ! ! static void mallocfailalert(size_t sz, void *ptr) ! @{ ! printf("Failed to malloc %lu bytes\n", (unsigned long)sz); ! @} ! ! malloc_debug(1); ! mallsethook(malloc_fail_HK, mallocfailalert); ! ... ! /* any malloc failure will result in a message */ ! ! @end example ! ! @c ------------------------------------------------------------- ! ! @node malldbgdmpfile, memory ! @subheading Syntax ! ! @example ! #include ! #include (on non-DJGPP systems) ! ! FILE *malldbgdmpfile(FILE *fp); ! @end example ! ! @subheading Description ! ! This function allows diverting the output of the malldbg package ! to any suitable text file. By default the package uses stderr. ! ! @subheading Return Value ! ! The previous identity of the dumpfile. @subheading Portability @portability !ansi, !posix ! This function is specific to DJGPP. ! ! @subheading Example ! ! @example ! FILE *oldfile = malldbgdmpfile(stdout); ! @end example ! ! @c ------------------------------------------------------------- ! ! @node _sysmalloc, memory ! @subheading Syntax ! ! @example ! #include ! #include /* on non-DJGPP systems */ ! #include /* alternative on non-DJGPP systems */ ! ! struct _sysquery _sysmalloc(void); ! @end example ! ! @subheading Description ! ! This function exports the internal structure of the malloc arena ! and thus allows construction of such packages as malldbg. It ! avoids accidental exposure of system critical values, and also ! permits such values to be changed and reflected in external ! packages. ! ! @subheading Return Value ! ! A struct _sysquery, described as follows: ! ! @example ! struct _sysquery @{ ! unsigned char data, gdlo, sz, prvf, nxtf, nxt, prv, ohead; ! void *nilp; ! void *anchors; ! set_m_hook hookset; ! @}; ! @end example ! ! Any unsigned char field set to 255 signifies the value is ! meaningless. The ohead value is not useful in the present ! implementation. The remaining values are small offsets from the ! internal pointers to the appropriate fields. In particular ! data is the offset between the internal pointer and the user ! pointers. nxt and prv link all allocated memory blocks, while ! nxtf and prvf are used only to link free blocks. sz is the ! internal size of the block, not the user size, and is normally ! rounded upward. Thus the nxt field is redundant to the sz field ! (except when no nxt exists) and is used in validity checking. ! A null value in nxtf field signifies that the block is not free. ! The special value nilp is used in nxtf and prvf as a NULL value ! distinguishable from NULL. ! ! anchors points to a NULL terminated list of sets of chains of ! memory blocks. These were allocated by sbrk and were not ! contiguous to any previous allocations. ! ! hookset is a pointer to a function that can set malloc hooks, as ! described in mallsethook. It is only available through this ! link. ! ! @subheading Portability ! ! @portability !ansi, !posix ! ! This function is specific to DJGPP. ! ! @subheading Example ! ! @example ! struct _sysquery sysinfo = _sysmalloc(); ! @end example + @c ------------------------------------------------------------- -- Chuck F (cbfalconer AT yahoo DOT com) (cbfalconer AT worldnet DOT att DOT net) Available for consulting/temporary embedded and systems. USE worldnet address!