X-Spam-Check-By: sourceware.org
Date: Mon, 22 Jan 2007 21:21:38 +0100
From: Corinna Vinschen <corinna-cygwin AT cygwin DOT com>
To: cygwin AT cygwin DOT com
Subject: Re: Perl bug?
Message-ID: <20070122202137.GC20665@calimero.vinschen.de>
Reply-To: cygwin AT cygwin DOT com
Mail-Followup-To: cygwin AT cygwin DOT com
References: <20070122181727 DOT GC27843 AT calimero DOT vinschen DOT de> <1552 DOT 67 DOT 40 DOT 28 DOT 188 DOT 1169493973 DOT squirrel AT 67 DOT 40 DOT 28 DOT 188>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <1552.67.40.28.188.1169493973.squirrel@67.40.28.188>
User-Agent: Mutt/1.4.2.2i
Mailing-List: contact cygwin-help AT cygwin DOT com; run by ezmlm
Precedence: bulk
List-Id: <cygwin.cygwin.com>
List-Unsubscribe: <mailto:cygwin-unsubscribe-archive-cygwin=delorie DOT com AT cygwin DOT com>
List-Subscribe: <mailto:cygwin-subscribe AT cygwin DOT com>
List-Archive: <http://sourceware.org/ml/cygwin/>
List-Post: <mailto:cygwin AT cygwin DOT com>
List-Help: <mailto:cygwin-help AT cygwin DOT com>, <http://sourceware.org/ml/#faqs>
Sender: cygwin-owner AT cygwin DOT com
Mail-Followup-To: cygwin AT cygwin DOT com
Delivered-To: mailing list cygwin AT cygwin DOT com

On Jan 22 11:26, Yitzchak Scott-Thoennes wrote:
> > Hi,
> >
> > consider the following statement:
> >
> >  $a = "a" x (100 * 1024 * 1024)
> >
> > When you create a script which does this over and over again, you'll
> > observe a strange memory problem.
> 
> Can you show your script?

#!/usr/bin/perl
$a = "a" x (100 * 1024 * 1024);
sleep 5;
$b = "b" x (100 * 1024 * 1024);
sleep 5;
$c = "c" x (100 * 1024 * 1024);
sleep 5;
$d = "d" x (100 * 1024 * 1024);
sleep 5;
$e = "e" x (100 * 1024 * 1024);
sleep 5;
$f = "f" x (100 * 1024 * 1024);
sleep 5;
$g = "g" x (100 * 1024 * 1024);
sleep 5;
$h = "h" x (100 * 1024 * 1024);
sleep 5;
$h = "h" x (100 * 1024 * 1024);
sleep 5;
$i = "i" x (100 * 1024 * 1024);
sleep 5;
$j = "j" x (100 * 1024 * 1024);
sleep 5;
$k = "k" x (100 * 1024 * 1024);
sleep 5;
$l = "l" x (100 * 1024 * 1024);
sleep 100;

> > By stracing I found that for each of these statements the following
> > happens:
> >
> >  "a"         --> malloc (2 bytes)
> >  x 100 Megs  --> realloc (100 Megs) + malloc (100 Megs)
> >
> > So the result is that each string of 100 Megs requires 200 Megs of
> > memory.  Doing this once is no problem, but doing it over and over
> > again will hit the maximum memory available twice as early.
> 
> This is as I would expect.  Most operators have a target, a temporary
> lexical, allocated to store their results.  Like all lexicals, these
> hold on to any memory they have allocated in the hope of saving having
> to allocate it a second time.

The problem is that it's *not* reused.  If you use strace when running
this script you see the allocations as I described them.  When the
2 Gigs virtual memory size for the process are used up, mmap (which
is called by malloc for big memory chunks) is called and returns -1,
MAP_FAILED.  Then malloc tries to get the memory from the heap, when
that fails, it just prints "Out of memory!", munmaps half of the above
allocations and then exits.

Just call strace and you'll see.

> > I can only assume that either the garbage collector doesn't kick in when
> > it should, or the garbage collector doesn't even know about this wasted
> > memory, which would be a generic memory leak either way.  However, it's
> > not clear if this depends on the Perl version, or if it depends on the
> > 64 bit int setting when building Perl.
> 
> I'd more likely suspect differences in the usemymalloc setting (perl
> -V:usemymalloc).  I'm not clear on exactly what you are seeing; again,

All three perl's on all three systems use the system malloc, not the
builtin malloc implementation (usemymalloc='n').


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Project Co-Leader          cygwin AT cygwin DOT com
Red Hat

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/