delorie.com/archives/browse.cgi   search  
Mail Archives: djgpp-workers/2002/05/16/01:31:37

X-Authentication-Warning: delorie.com: mailnull set sender to djgpp-workers-bounces using -f
Date: Thu, 16 May 2002 08:27:57 +0300 (IDT)
From: Eli Zaretskii <eliz AT is DOT elta DOT co DOT il>
X-Sender: eliz AT is
To: djgpp-workers AT delorie DOT com
Subject: Re: emacs under w2k
In-Reply-To: <3CE2B315.7576084C@yahoo.com>
Message-ID: <Pine.SUN.3.91.1020516075834.13145D-100000@is>
MIME-Version: 1.0
Reply-To: djgpp-workers AT delorie DOT com
Errors-To: nobody AT delorie DOT com
X-Mailing-List: djgpp-workers AT delorie DOT com
X-Unsubscribes-To: listserv AT delorie DOT com

On Wed, 15 May 2002, CBFalconer wrote:

> Which brings up, to me, "why does emacs replace malloc etc."? 

Because Emacs, being a Lisp interpreter, constantly allocates and 
deallocates memory in both very small chunks (for Lisp objects like cons 
cells), medium-size chunks (strings and arrays), and very large chunks 
(buffer text storage).  A typical malloc implementation, including the 
one in DJGPP's libc, doesn't handle this mix very well, especially in 
programs that run for many days and weeks.  (The normal usage of Emacs 
is to start a session once, then leave it running at all times.)  The 
result is memory fragmentation that inevitably leads to Emacs allocating 
more and more memory from the system, until you reboot.

> Possibly it has problems with free when many blocks have been
> allocated, or with multiple reallocs?

Actually, the main problem is to be able to join many small free'd chunks 
into larger chunks.  If you fail to do that, you'll hit the problem of 
being unable to allocate memory for reading a file because the free pool 
is fragmented.  Then you'll need to sbrk more from the OS.

This pattern happens a lot in Emacs because you typicall load a file, do 
something with it for some time, then load another file.  The ``do 
something'' part allocates many small Lisp objects (since many features in 
Emacs, even important parts of the display engine, are implemented in Lisp).
Then, all of a sudden, you need many KBytes for the next file.

> In both cases the problems have been handled in nmalloc

The Emacs relocating allocator does things that are unthinkable for any 
general-purpose malloc implementation.  For example, under certain 
conditions, it relocates allocated buffers from under your feet.  That's 
right: you call malloc or realloc, and when these return, some of the 
C pointers you've got from previous malloc calls are no longer valid,
they point to chunks of memory that doesn't hold what it held before, and 
text of some buffers has been relocated to other addresses(!).  (This, of 
course, requires some cooperation between malloc and the application, and 
some specific knowledge about other library functions that can call 
malloc.)

I doubt if any sane implementation of malloc can do such crazy things.  
And yet Emacs needs this to prevent indefinite growth of its memory 
footprint.

> which is just standing around
> waiting for critical testing in the DJGPP environment.

I agree that it's bad to leave contributions unreviewed.  Could someone 
please find time to do that?  (I'm sorry to ask for something I cannot do 
myself, but lately I have difficulty finding time for any substantial 
work on DJGPP; I even can hardly cope with putting out ports of new 
releases of GNU packages.)

> If the emacs module is independantly replaceable, Laurynas might
> want to try an immediate bodily replacement and see if the
> problems go away (or if new ones arise, which I doubt).

It's very easy to build Emacs with the library malloc: just add this 
line to src/config.h:

  #define SYSTEM_MALLOC 1

(You might also need to add _CRT0_FLAG_FILL_SBRK_MEMORY to 
_crt0_startup_flags, but I won't recommend doing that unless you see 
strange problems or aborts.)  To do this with nmalloc, one needs to 
replace malloc and friends in libc.a with the alternative implementation, 
or hack src/Makefile to add libnmalloc.a (or whatever it is called) to 
the link command line.

The ultimate test is to be able to have a live session, actively used for 
editing and other stuff which Emacs does, run for a significant amount of 
time, and watch its memory footprint with some tool.  The footprint 
should level out at some point, instead of constantly growing.

- Raw text -


  webmaster     delorie software   privacy  
  Copyright © 2019   by DJ Delorie     Updated Jul 2019