X-Authentication-Warning: delorie.com: mail set sender to djgpp-bounces using -f
From: Rod Pemberton <invalid AT lkntrgzxc DOT com>
Newsgroups: comp.os.msdos.djgpp
Subject: Re: malloc() returns pointer to already allocated memory
Date: Mon, 17 Jun 2019 02:14:55 -0400
Organization: Aioe.org NNTP Server
Lines: 108
Message-ID: <qe7avt$52r$2@gioia.aioe.org>
References: <158e5d20-0a90-4beb-de48-da328379d8fb AT gmail DOT com>
 <qe76u1$1kj8$1 AT gioia DOT aioe DOT org>
 <f0b68226-f6f4-244a-6dd5-a8ecbabb584b AT gmail DOT com>
 <qe79eb$1urs$1 AT gioia DOT aioe DOT org>
 <qe7ar9$52r$1 AT gioia DOT aioe DOT org>
NNTP-Posting-Host: +15yR2JuBIwiofOqK4kSZw.user.gioia.aioe.org
Mime-Version: 1.0
Content-Type: text/plain; charset=utf-8
X-Complaints-To: abuse AT aioe DOT org
X-Notice: Filtered by postfilter v. 0.9.2
Bytes: 6441
To: djgpp AT delorie DOT com
DJ-Gateway: from newsgroup comp.os.msdos.djgpp
Content-Transfer-Encoding: 8bit
X-MIME-Autoconverted: from quoted-printable to 8bit by delorie.com id x5H6UMXW023474
Reply-To: djgpp AT delorie DOT com

On Mon, 17 Jun 2019 02:12:26 -0400
Rod Pemberton <invalid AT lkntrgzxc DOT com> wrote:

> On Mon, 17 Jun 2019 01:48:19 -0400
> Rod Pemberton <invalid AT lkntrgzxc DOT com> wrote:
> 
> > On Mon, 17 Jun 2019 01:28:32 -0400
> > "Frank Sapone (emoaddict15 AT gmail DOT com) [via djgpp AT delorie DOT com]"
> > <djgpp AT delorie DOT com> wrote:
> >   
> > > On 6/17/2019 1:05 AM, Rod Pemberton wrote:    
> > > > On Mon, 17 Jun 2019 02:27:18 +0200
> > > > "J.W. Jagersma (jwjagersma AT gmail DOT com) [via djgpp AT delorie DOT com]"
> > > > <djgpp AT delorie DOT com> wrote:    
> >   
> > > >> I ran into this issue with malloc(). It seems that, given
> > > >> enough allocations, malloc() will eventually return a pointer
> > > >> into already allocated memory.
> > > >>
> > > >> The attached program is able to reproduce this rather
> > > >> consistently, but only under cwsdpmi. It also only happens if
> > > >> the memory has previously been written to (suggesting a paging
> > > >> issue?). However the code that first led me to investigate this
> > > >> also exhibits the same problem under hdpmi. As such, I'm still
> > > >> not entirely convinced that this initial issue wasn't caused by
> > > >> my own code. I also find it hard to believe that no one else
> > > >> noticed this rather obvious problem before me. Still, the
> > > >> attached program demonstrates this clobbering issue, and I
> > > >> think this would warrant further investigation.
> > > >>
> > > >> Any insight is much appreciated.
> > > >>      
> > > > First problem is trivial.  The code doesn't compile with older
> > > > DJGPP v1.3.  The declaration of 'i' within the for() loop
> > > > errors, but other C99 declarations only warn.
> > > >
> > > > Second problem is you don't call memset() prior to using memory,
> > > > nor free() after you're done using it.  Of course, calling
> > > > memset() would prevent your method of "clobber" detection from
> > > > working. But, not calling memset() means you don't know if the
> > > > magic clobber value is: a) from you setting it within your
> > > > program, or b) from some random garbage values in memory.
> > > >
> > > > Third problem is you apparently didn't test the program without
> > > > the "p[i] = magic;" line.  If you had, you would've noticed that
> > > > your program clobbers even without setting memory to magic
> > > > values. In other words, memory is filled with random values,
> > > > since it wasn't cleared by memset().  Also, some of those random
> > > > values happen to match your program's random magic value used to
> > > > detect clobbered memory.
> > > >
> > > > Fourth problem is that you can't actually confirm if memory is
> > > > being clobbered from within a C program for two reasons:
> > > > inability to distinguish a magic value from an identical random
> > > > value in memory which hasn't been cleared, and the inability in
> > > > C to allocate, clear, and free memory, prior to the re-use of
> > > > the exact same memory for a clobber test.  To test this issue
> > > > properly requires a modified version of the memory allocator,
> > > > i.e., CWSDPMI in this case.
> > > >
> > > > Finally, you didn't report which version of DJGPP, or CWSDPMI,
> > > > and whether or not your code is operating in a Windows
> > > > 98/SE/ME/XP etc console.  When operating in a Windows console,
> > > > CWSDPMI is not being used.  The Windows DPMI host is being used.
> > > >
> > > >    
> > > 
> > > Good points.  I was thinking some of the same things when I read
> > > this post earlier, but I am not as much of a C guru as others
> > > here.  My initial thoughts were that memset was not being called
> > > and that it was a random integer for the magic value that there is
> > > a possibility of grabbing the same value.
> > >     
> > 
> > One thing he could do is reduce the likelihood, i.e., probability,
> > that he'll get a false positive, i.e., random value.  He could check
> > for multiple magic values in a row.  The more values he checks for,
> > the lower the probability will be that it's random.  I'd probably
> > check for an entire block.  However, even long sequences can be
> > random or perhaps stale program data.  E.g., a roulette wheel could
> > in theory land one on the color red for millions of spins or more,
> > while the odds or likelihood that it lands on either black or red
> > hasn't changed.  I.e., there is no absolute certainty that a
> > detected "clobber" isn't random.
> >   
> 
> Well, I decided to see what happens when I check for two magic values
> in a row (without setting any magic values).  I'm no longer see any
> matches with random data (although there could still be).
> 
> So, I also decided to check for two magic values, while also setting
> the magic values, and then dump the eight prior values in a row.  I'm
> seeing eight magic values in a row ...  Not good.  Maybe, he found an
> issue with "clobber," but his posted test program was too simplified,
> i.e., matching random data.
> 
> At this point, it's up to someone else to verify, after making needed
> changes to his code.

Oops.  Never mind.  The program is setting the eight prior (more)
values.  I should've checked forwards. BRB.


Rod Pemberton
-- 
Once upon a time, many decades ago in a place far away, humble people
sought their freedom, and lost. "Ideas are bulletproof."