X-Spam-Check-By: sourceware.org Date: Mon, 22 Jan 2007 19:17:27 +0100 From: Corinna Vinschen To: cygwin AT cygwin DOT com Cc: "Gerrit P. Haase" Subject: Perl bug? Message-ID: <20070122181727.GC27843@calimero.vinschen.de> Reply-To: cygwin AT cygwin DOT com Mail-Followup-To: cygwin AT cygwin DOT com, "Gerrit P. Haase" Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.4.2.2i Mailing-List: contact cygwin-help AT cygwin DOT com; run by ezmlm List-Id: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: cygwin-owner AT cygwin DOT com Mail-Followup-To: cygwin AT cygwin DOT com Delivered-To: mailing list cygwin AT cygwin DOT com Hi, consider the following statement: $a = "a" x (100 * 1024 * 1024) When you create a script which does this over and over again, you'll observe a strange memory problem. By stracing I found that for each of these statements the following happens: "a" --> malloc (2 bytes) x 100 Megs --> realloc (100 Megs) + malloc (100 Megs) So the result is that each string of 100 Megs requires 200 Megs of memory. Doing this once is no problem, but doing it over and over again will hit the maximum memory available twice as early. At this point I expected that unused memory is reused. After all, the Perl interpreter has a garbage collection mechanism which should be able to reuse unused space. To my surprise this didn't happen. Instead, it bailed out with the "Out of memory!" message as you already experienced. What's noteworthy here is the fact that, after printing the "Out of memory!" message, Perl free's only half of the memory allocations. The other half of the allocations are kept untouched until the perl process actually exits. this happens with Cygwin 1.5.18 as well as with 1.5.23 as well as with 1.7.0. I tested with the current Perl 5.8.7, cygwin-thread-multi-64int. As an additional test I ran the script on two Linux boxes, one 64 bit machine running Perl 5.8.8, x86_64-linux-thread-multi, one 32 bit machine running Perl 5.8.5, i386-linux-thread-multi. The behaviour of running this script on the x86_64 Linux box is the same as on Cygwin. Instead of reusing the overallocated areas, the Perl interpreter just fails to allocate space for another string when it bangs its head into the virtual address space available per process. The i386 Linux Perl 5.8.5 version does not show this behaviour. It takes only 100 Megs per statement and can so take as many 100 Meg strings as fit in the processes virtual memory space. I can only assume that either the garbage collector doesn't kick in when it should, or the garbage collector doesn't even know about this wasted memory, which would be a generic memory leak either way. However, it's not clear if this depends on the Perl version, or if it depends on the 64 bit int setting when building Perl. Gerrit? Any chance to get this fixed? Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Project Co-Leader cygwin AT cygwin DOT com Red Hat -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/