X-Spam-Check-By: sourceware.org Date: Mon, 22 Jan 2007 21:21:38 +0100 From: Corinna Vinschen To: cygwin AT cygwin DOT com Subject: Re: Perl bug? Message-ID: <20070122202137.GC20665@calimero.vinschen.de> Reply-To: cygwin AT cygwin DOT com Mail-Followup-To: cygwin AT cygwin DOT com References: <20070122181727 DOT GC27843 AT calimero DOT vinschen DOT de> <1552 DOT 67 DOT 40 DOT 28 DOT 188 DOT 1169493973 DOT squirrel AT 67 DOT 40 DOT 28 DOT 188> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1552.67.40.28.188.1169493973.squirrel@67.40.28.188> User-Agent: Mutt/1.4.2.2i Mailing-List: contact cygwin-help AT cygwin DOT com; run by ezmlm Precedence: bulk List-Id: List-Unsubscribe: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: cygwin-owner AT cygwin DOT com Mail-Followup-To: cygwin AT cygwin DOT com Delivered-To: mailing list cygwin AT cygwin DOT com On Jan 22 11:26, Yitzchak Scott-Thoennes wrote: > > Hi, > > > > consider the following statement: > > > > $a = "a" x (100 * 1024 * 1024) > > > > When you create a script which does this over and over again, you'll > > observe a strange memory problem. > > Can you show your script? #!/usr/bin/perl $a = "a" x (100 * 1024 * 1024); sleep 5; $b = "b" x (100 * 1024 * 1024); sleep 5; $c = "c" x (100 * 1024 * 1024); sleep 5; $d = "d" x (100 * 1024 * 1024); sleep 5; $e = "e" x (100 * 1024 * 1024); sleep 5; $f = "f" x (100 * 1024 * 1024); sleep 5; $g = "g" x (100 * 1024 * 1024); sleep 5; $h = "h" x (100 * 1024 * 1024); sleep 5; $h = "h" x (100 * 1024 * 1024); sleep 5; $i = "i" x (100 * 1024 * 1024); sleep 5; $j = "j" x (100 * 1024 * 1024); sleep 5; $k = "k" x (100 * 1024 * 1024); sleep 5; $l = "l" x (100 * 1024 * 1024); sleep 100; > > By stracing I found that for each of these statements the following > > happens: > > > > "a" --> malloc (2 bytes) > > x 100 Megs --> realloc (100 Megs) + malloc (100 Megs) > > > > So the result is that each string of 100 Megs requires 200 Megs of > > memory. Doing this once is no problem, but doing it over and over > > again will hit the maximum memory available twice as early. > > This is as I would expect. Most operators have a target, a temporary > lexical, allocated to store their results. Like all lexicals, these > hold on to any memory they have allocated in the hope of saving having > to allocate it a second time. The problem is that it's *not* reused. If you use strace when running this script you see the allocations as I described them. When the 2 Gigs virtual memory size for the process are used up, mmap (which is called by malloc for big memory chunks) is called and returns -1, MAP_FAILED. Then malloc tries to get the memory from the heap, when that fails, it just prints "Out of memory!", munmaps half of the above allocations and then exits. Just call strace and you'll see. > > I can only assume that either the garbage collector doesn't kick in when > > it should, or the garbage collector doesn't even know about this wasted > > memory, which would be a generic memory leak either way. However, it's > > not clear if this depends on the Perl version, or if it depends on the > > 64 bit int setting when building Perl. > > I'd more likely suspect differences in the usemymalloc setting (perl > -V:usemymalloc). I'm not clear on exactly what you are seeing; again, All three perl's on all three systems use the system malloc, not the builtin malloc implementation (usemymalloc='n'). Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Project Co-Leader cygwin AT cygwin DOT com Red Hat -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/