X-Authentication-Warning: delorie.com: mailnull set sender to djgpp-workers-bounces using -f Date: Thu, 16 May 2002 08:27:57 +0300 (IDT) From: Eli Zaretskii X-Sender: eliz AT is To: djgpp-workers AT delorie DOT com Subject: Re: emacs under w2k In-Reply-To: <3CE2B315.7576084C@yahoo.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Reply-To: djgpp-workers AT delorie DOT com Errors-To: nobody AT delorie DOT com X-Mailing-List: djgpp-workers AT delorie DOT com X-Unsubscribes-To: listserv AT delorie DOT com Precedence: bulk On Wed, 15 May 2002, CBFalconer wrote: > Which brings up, to me, "why does emacs replace malloc etc."? Because Emacs, being a Lisp interpreter, constantly allocates and deallocates memory in both very small chunks (for Lisp objects like cons cells), medium-size chunks (strings and arrays), and very large chunks (buffer text storage). A typical malloc implementation, including the one in DJGPP's libc, doesn't handle this mix very well, especially in programs that run for many days and weeks. (The normal usage of Emacs is to start a session once, then leave it running at all times.) The result is memory fragmentation that inevitably leads to Emacs allocating more and more memory from the system, until you reboot. > Possibly it has problems with free when many blocks have been > allocated, or with multiple reallocs? Actually, the main problem is to be able to join many small free'd chunks into larger chunks. If you fail to do that, you'll hit the problem of being unable to allocate memory for reading a file because the free pool is fragmented. Then you'll need to sbrk more from the OS. This pattern happens a lot in Emacs because you typicall load a file, do something with it for some time, then load another file. The ``do something'' part allocates many small Lisp objects (since many features in Emacs, even important parts of the display engine, are implemented in Lisp). Then, all of a sudden, you need many KBytes for the next file. > In both cases the problems have been handled in nmalloc The Emacs relocating allocator does things that are unthinkable for any general-purpose malloc implementation. For example, under certain conditions, it relocates allocated buffers from under your feet. That's right: you call malloc or realloc, and when these return, some of the C pointers you've got from previous malloc calls are no longer valid, they point to chunks of memory that doesn't hold what it held before, and text of some buffers has been relocated to other addresses(!). (This, of course, requires some cooperation between malloc and the application, and some specific knowledge about other library functions that can call malloc.) I doubt if any sane implementation of malloc can do such crazy things. And yet Emacs needs this to prevent indefinite growth of its memory footprint. > which is just standing around > waiting for critical testing in the DJGPP environment. I agree that it's bad to leave contributions unreviewed. Could someone please find time to do that? (I'm sorry to ask for something I cannot do myself, but lately I have difficulty finding time for any substantial work on DJGPP; I even can hardly cope with putting out ports of new releases of GNU packages.) > If the emacs module is independantly replaceable, Laurynas might > want to try an immediate bodily replacement and see if the > problems go away (or if new ones arise, which I doubt). It's very easy to build Emacs with the library malloc: just add this line to src/config.h: #define SYSTEM_MALLOC 1 (You might also need to add _CRT0_FLAG_FILL_SBRK_MEMORY to _crt0_startup_flags, but I won't recommend doing that unless you see strange problems or aborts.) To do this with nmalloc, one needs to replace malloc and friends in libc.a with the alternative implementation, or hack src/Makefile to add libnmalloc.a (or whatever it is called) to the link command line. The ultimate test is to be able to have a live session, actively used for editing and other stuff which Emacs does, run for a significant amount of time, and watch its memory footprint with some tool. The footprint should level out at some point, instead of constantly growing.