Mailing-List: contact cygwin-developers-help AT cygwin DOT com; run by ezmlm
List-Subscribe: <mailto:cygwin-developers-subscribe AT cygwin DOT com>
List-Archive: <http://sources.redhat.com/ml/cygwin-developers/>
List-Post: <mailto:cygwin-developers AT cygwin DOT com>
List-Help: <mailto:cygwin-developers-help AT cygwin DOT com>, <http://sources.redhat.com/ml/#faqs>
Sender: cygwin-developers-owner AT cygwin DOT com
Delivered-To: mailing list cygwin-developers AT cygwin DOT com
Date: Fri, 19 Jul 2002 13:32:04 -0400
From: Jason Tishler <jason AT tishler DOT net>
Subject: realloc overflow problem
To: Cygwin-Developers <cygwin-developers AT cygwin DOT com>
Mail-followup-to: Cygwin-Developers <cygwin-developers AT cygwin DOT com>
Message-id: <20020719173204.GD376@tishler.net>
MIME-version: 1.0
Content-type: multipart/mixed; boundary="Boundary_(ID_QPSO584em1rJoEN7fzqpCQ)"
User-Agent: Mutt/1.4i


--Boundary_(ID_QPSO584em1rJoEN7fzqpCQ)
Content-type: text/plain; charset=us-ascii
Content-transfer-encoding: 7BIT
Content-disposition: inline

[This may be more appropriate for the newlib list, but I will start
here.  Please feel free to redirect this post...]

While tracking down a Cygwin Python problem, I believe that I have
uncovered an overflow problem in realloc() (and possibly other malloc
routines).  The attached test program, realloc.cc, demonstrates the
problem:

    $ realloc 536870911  # realloc((maximum int value) / 4) fails
    realloc(536870911) failed
    $ realloc 2147483647 # realloc(maximum int value) "succeeds"
    $

The attached patch "fixes" the overflow problem by changing many (long)
casts into (unsigned long) ones:

    $ realloc 2147483647
    realloc(2147483647) failed

Is this the right approach?  Or, is there a better way?

Thanks,
Jason

--Boundary_(ID_QPSO584em1rJoEN7fzqpCQ)
Content-type: text/plain; charset=us-ascii; NAME=realloc.cc
Content-transfer-encoding: 7BIT
Content-disposition: attachment; filename=realloc.cc

#include <stdio.h>
#include <stdlib.h>

int
main(int argc, char* argv[])
{
	size_t size = atol(argv[1]);

	char* p = (char*) malloc(1);
	if (!p)
	{
		printf("malloc(1) failed\n");
		exit(1);
	}

	p = (char*) realloc(p, size);
	if (!p)
	{
		printf("realloc(%lu) failed\n", size);
		exit(2);
	}

	return 0;
}

--Boundary_(ID_QPSO584em1rJoEN7fzqpCQ)
Content-type: text/plain; charset=us-ascii; NAME=mallocr.c.diff
Content-transfer-encoding: 7BIT
Content-disposition: attachment; filename=mallocr.c.diff

Index: mallocr.c
===================================================================
RCS file: /cvs/src/src/newlib/libc/stdlib/mallocr.c,v
retrieving revision 1.6
diff -u -p -r1.6 mallocr.c
--- mallocr.c	13 Apr 2002 10:27:02 -0000	1.6
+++ mallocr.c	19 Jul 2002 17:16:00 -0000
@@ -1399,8 +1399,8 @@ nextchunk-> +-+-+-+-+-+-+-+-+-+-+-+-+-+-
 /* pad request bytes into a usable size */
 
 #define request2size(req) \
- (((long)((req) + (SIZE_SZ + MALLOC_ALIGN_MASK)) < \
-  (long)(MINSIZE + MALLOC_ALIGN_MASK)) ? ((MINSIZE + MALLOC_ALIGN_MASK) & ~(MALLOC_ALIGN_MASK)) : \
+ (((unsigned long)((req) + (SIZE_SZ + MALLOC_ALIGN_MASK)) < \
+  (unsigned long)(MINSIZE + MALLOC_ALIGN_MASK)) ? ((MINSIZE + MALLOC_ALIGN_MASK) & ~(MALLOC_ALIGN_MASK)) : \
    (((req) + (SIZE_SZ + MALLOC_ALIGN_MASK)) & ~(MALLOC_ALIGN_MASK)))
 
 /* Check if m has acceptable alignment */
@@ -2825,7 +2825,7 @@ Void_t* rEALLOc(RARG oldmem, bytes) RDEC
 
   check_inuse_chunk(oldp);
 
-  if ((long)(oldsize) < (long)(nb))  
+  if ((unsigned long)(oldsize) < (unsigned long)(nb))  
   {
 
     /* Try expanding forward */
@@ -2838,7 +2838,7 @@ Void_t* rEALLOc(RARG oldmem, bytes) RDEC
       /* Forward into top only if a remainder */
       if (next == top)
       {
-        if ((long)(nextsize + newsize) >= (long)(nb + MINSIZE))
+        if ((unsigned long)(nextsize + newsize) >= (unsigned long)(nb + MINSIZE))
         {
           newsize += nextsize;
           top = chunk_at_offset(oldp, nb);
@@ -2850,7 +2850,7 @@ Void_t* rEALLOc(RARG oldmem, bytes) RDEC
       }
 
       /* Forward into next chunk */
-      else if (((long)(nextsize + newsize) >= (long)(nb)))
+      else if (((unsigned long)(nextsize + newsize) >= (unsigned long)(nb)))
       { 
         unlink(next, bck, fwd);
         newsize  += nextsize;
@@ -2877,7 +2877,7 @@ Void_t* rEALLOc(RARG oldmem, bytes) RDEC
         /* into top */
         if (next == top)
         {
-          if ((long)(nextsize + prevsize + newsize) >= (long)(nb + MINSIZE))
+          if ((unsigned long)(nextsize + prevsize + newsize) >= (unsigned long)(nb + MINSIZE))
           {
             unlink(prev, bck, fwd);
             newp = prev;
@@ -2893,7 +2893,7 @@ Void_t* rEALLOc(RARG oldmem, bytes) RDEC
         }
 
         /* into next chunk */
-        else if (((long)(nextsize + prevsize + newsize) >= (long)(nb)))
+        else if (((unsigned long)(nextsize + prevsize + newsize) >= (unsigned long)(nb)))
         {
           unlink(next, bck, fwd);
           unlink(prev, bck, fwd);
@@ -2906,7 +2906,7 @@ Void_t* rEALLOc(RARG oldmem, bytes) RDEC
       }
       
       /* backward only */
-      if (prev != 0 && (long)(prevsize + newsize) >= (long)nb)  
+      if (prev != 0 && (unsigned long)(prevsize + newsize) >= (unsigned long)nb)  
       {
         unlink(prev, bck, fwd);
         newp = prev;
@@ -2949,7 +2949,7 @@ Void_t* rEALLOc(RARG oldmem, bytes) RDEC
 
   remainder_size = long_sub_size_t(newsize, nb);
 
-  if (remainder_size >= (long)MINSIZE) /* split off remainder */
+  if (remainder_size >= (unsigned long)MINSIZE) /* split off remainder */
   {
     remainder = chunk_at_offset(newp, nb);
     set_head_size(newp, nb);

--Boundary_(ID_QPSO584em1rJoEN7fzqpCQ)--