X-Spam-Check-By: sourceware.org
Date: Fri, 18 Aug 2006 16:00:08 +0200
From: Corinna Vinschen <corinna-cygwin AT cygwin DOT com>
To: cygwin AT cygwin DOT com
Cc: Eli Zaretskii <eliz AT gnu DOT org>
Subject: Re: reproducible cygwin memory problems
Message-ID: <20060818140008.GA18635@calimero.vinschen.de>
Reply-To: cygwin AT cygwin DOT com
Mail-Followup-To: cygwin AT cygwin DOT com, Eli Zaretskii <eliz AT gnu DOT org>
References: <BAY107-F22549D794C527187392FB2F8480 AT phx DOT gbl> <uoduqc8g9 DOT fsf AT gnu DOT org> <20060812165300 DOT GB19272 AT calimero DOT vinschen DOT de> <44DEEF84 DOT 803 AT x-ray DOT at> <ubqqofo1y DOT fsf AT gnu DOT org> <20060814065812 DOT GA29807 AT calimero DOT vinschen DOT de> <uy7tmi850 DOT fsf AT gnu DOT org>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <uy7tmi850.fsf@gnu.org>
User-Agent: Mutt/1.4.2i
Mailing-List: contact cygwin-help AT cygwin DOT com; run by ezmlm
Precedence: bulk
List-Unsubscribe: <mailto:cygwin-unsubscribe-archive-cygwin=delorie DOT com AT cygwin DOT com>
List-Subscribe: <mailto:cygwin-subscribe AT cygwin DOT com>
List-Archive: <http://sourceware.org/ml/cygwin/>
List-Post: <mailto:cygwin AT cygwin DOT com>
List-Help: <mailto:cygwin-help AT cygwin DOT com>, <http://sourceware.org/ml/#faqs>
Sender: cygwin-owner AT cygwin DOT com
Mail-Followup-To: cygwin AT cygwin DOT com
Delivered-To: mailing list cygwin AT cygwin DOT com

On Aug 18 14:40, Eli Zaretskii wrote:
> > From: Corinna Vinschen <XXXXXXX-XXXXXX AT XXXXXX DOT XXX>
> > There is no known issue.  The heap is an area of reserved memory, which
> > is on demand commited when sbrk is called.  If the heap is too small to
> > fit the new allocation, more memory is reserved/commited.  When sbrk is
> > called with negative value, memory is decommited, but it's not released.
> > Maybe that's why the OP doesn't see a decrease in the memory footprint.
> > However, this works as designed.
> 
> AFAIK, decommitting memory does decrease its memory footprint as shown
> by the task manager.  The emulation of sbrk used by the native Windows
> port of Emacs does the same as what you described: it calls
> VirtualFree with MEM_DECOMMIT flag (see w32heap.c:sbrk), and I do see
> decrease in memory footprint when a large buffer is killed.  So, at
> least in theory, the Cygwin build of Emacs should exhibit the same
> behavior.

Yes, right, sorry for jumping to a conclusion.

> > Sbrk resp. munmap are fairly basic operations.  I explained how they
> > work or are supposed to work.  If they don't work for emacs/lisp, they
> > don't work for hundreds of applications and it should be quite easy to
> > come up with a simple testcase in C.  So far I didn't see one, and my
> > local tests work as expected.
> 
> FWIW, I looked into the Emacs image-support code, and I don't see any
> memory allocation when it loads and displays an image, except the
> allocation that happens inside the external image libraries used for
> image support (libtiff.dll, jpeg62.dll, etc.).  So what's been said
> earlier in this thread -- that Emacs allocates memory for images, but
> doesn't release all of it -- sounds less and less likely, especially
> since Cygwin uses the same C code as any other Posix platform,
> including GNU/Linux.

Well, since a testcase is still missing, I wrote my own.

==== SNIP ====
#include <stdio.h>
#include <stdlib.h>
#include <malloc.h>

#define COUNT 320

int
main (int argc, char **argv)
{
  char *p[COUNT];

  /* Use sbrk.  Set argv[1] to a value > 256K to use mmap/munmap. */
  size_t size = 200000;
  int i;

  if (argc > 1)
    size = strtoul (argv[1], NULL, 0);

#if 1
  /* This early getchar is necessary because otherwise the next
     getchar, called after the malloc calls, would malloc another 1K
     chunk.  This chunk is then at the top of the sbrk'ed heap and
     would in turn disable the memory trimming in free.  In other
     words, sbrk with negative value could never be called.
     
     Consequentially, disable this getchar call and you'll see how the
     free calls never call sbrk.  However, the memory would still be
     free'd if argv[1] > 256K, since munmap works even if getchar
     allocates sbrk'ed memory at the top of the heap. */
  fprintf (stderr, "Press key...\n");
  getchar ();
#endif

  while (1)
    {
      for (i = 0; i < COUNT; ++i)
        {
	  p[i] = malloc (size);
	  if (!p[i])
	    fprintf(stderr, "Couldn't allocate %lu bytes\n", size);
	  else
	    fprintf(stderr, "allocate %lu bytes at %p\n", size, p[i]);
	}
      getchar ();

      for (i = 0; i < COUNT; ++i)
        {
	  free (p[COUNT - i - 1]);
	  fprintf(stderr, "freed %lu bytes at %p\n", size, p[COUNT - i - 1]);
	}
      getchar ();
    }

  return 0;
}
==== SNAP ====

Try this on Cygwin and open a task manager window in parallel.  After
the mallocs have been called, you'll see how the memory is free'd
again, which proves that sbrk with negative value is called and works
as expected.  Use argv[1] to set the chunk size to a value > 256K to
test the mmap/munmap implementation.  You'll see in task manager that
the result is again as expected, free'ing OS memory.

Now, disable the first call to getchar, and observe how an unlucky
placed call to malloc, resp. an unlucky placed library call which also
malloc's memory, can spoil all the efforts.

Now the testcase is available and still, I don't see anything unusual.


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Project Co-Leader          cygwin AT cygwin DOT com
Red Hat

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/