X-Recipient: archive-cygwin AT delorie DOT com
X-SWARE-Spam-Status: No, hits=-1.5 required=5.0	tests=AWL,BAYES_00,RP_MATCHES_RCVD,SPF_NEUTRAL
X-Spam-Check-By: sourceware.org
Message-ID: <4E417AB6.5070306@cornell.edu>
Date: Tue, 09 Aug 2011 14:21:42 -0400
From: Ken Brown <kbrown AT cornell DOT edu>
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2.18) Gecko/20110616 Thunderbird/3.1.11
MIME-Version: 1.0
To: cygwin AT cygwin DOT com
Subject: Re: emacs and large-address awareness under recent snapshots
References: <4E40093D DOT 10107 AT cornell DOT edu> <20110808162556 DOT GM11601 AT calimero DOT vinschen DOT de> <87sjpbhij7 DOT fsf AT Rainer DOT invalid> <4E404424 DOT 9000600 AT cornell DOT edu> <4E40526C DOT 9080605 AT cornell DOT edu> <4E406C21 DOT 7010007 AT cs DOT umass DOT edu> <20110809082652 DOT GA9492 AT calimero DOT vinschen DOT de> <4E4117AF DOT 3030305 AT cornell DOT edu> <4E414054 DOT 6040206 AT cornell DOT edu> <4E4142F7 DOT 8020708 AT cornell DOT edu> <20110809152155 DOT GB17030 AT calimero DOT vinschen DOT de>
In-Reply-To: <20110809152155.GB17030@calimero.vinschen.de>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
X-IsSubscribed: yes
Mailing-List: contact cygwin-help AT cygwin DOT com; run by ezmlm
List-Id: <cygwin.cygwin.com>
List-Subscribe: <mailto:cygwin-subscribe AT cygwin DOT com>
List-Archive: <http://sourceware.org/ml/cygwin/>
List-Post: <mailto:cygwin AT cygwin DOT com>
List-Help: <mailto:cygwin-help AT cygwin DOT com>, <http://sourceware.org/ml/#faqs>
Sender: cygwin-owner AT cygwin DOT com
Mail-Followup-To: cygwin AT cygwin DOT com
Delivered-To: mailing list cygwin AT cygwin DOT com

On 8/9/2011 11:21 AM, Corinna Vinschen wrote:
> On Aug  9 10:23, Ken Brown wrote:
>> On 8/9/2011 10:12 AM, Ken Brown wrote:
>>> On 8/9/2011 7:19 AM, Ken Brown wrote:
>>>> (gdb) thread 1
>>>> [Switching to thread 1 (Thread 19828.0x447c)]
>>>> #0  0x00622ee0 in morecore_nolock (size=1052672) at gmalloc.c:703
>>>> 703           while ((__malloc_size_t) BLOCK ((char *) result + size)>
>>>> newsize);
>>>> (gdb) p /x size
>>>> $1 = 0x101000
>>>> (gdb) p /x heapsize
>>>> $2 = 0x80000
>>>> (gdb) p result
>>>> $3 = (void *) 0x807d0000
>>>> (gdb) p newsize
>>>> $4 = 0
>>>> (gdb) p _heapbase
>>>> $5 = 0x816000 "\202"
>>>> (gdb) p _heapinfo
>>>> $6 = (malloc_info *) 0x80060000
>>>>
>>>> Is _heapbase the problem?  This is initialized to _heapinfo at the first
>>>> call of malloc and is never changed.  _heapinfo presumably points into
>>>> the static heap at that point.  (_heapinfo is later changed as a result
>>>> of realloc.)  This low value of _heapbase is used in the BLOCK macro.
>>>
>>> Here's what I think is happening.  When temacs.exe is running during the
>>> build process (see my explanation of this earlier in the thread),
>>> malloc_init is called and _heapbase is set.  At this point, temacs is
>>> using its own static buffer as the heap, and _heapbase gets the value
>>> 0x816000.  This gets dumped as initialized data into emacs.exe, as does
>>> the value __malloc_initialized = 1.  Now when emacs.exe is run, it sees
>>> that malloc has already been initialized, so _heapbase retains its
>>> value, which is no longer appropriate.  All code relying on the BLOCK
>>> macro is now invalid.
>>>
>>> AFAICS, this has always been wrong.  But the error didn't have dramatic
>>> consequences until the heap was put into high memory.
>>>
>>> I'm not sure what's the best way to fix this (assuming my analysis is
>>> right).  Would it be enough to set __malloc_initialized to 0 before
>>> dumping?  That would force emacs to reinitialize and get the correct
>>> value of _heapbase.
>>
>> No, that's too simple-minded.  I just tried it, and emacs aborted.
>> This seems like a mess.
>
> What happens if you remove the Cygwin-specific call to bss_sbrk in
> __default_morecore?

That will mess up dumping.  The point of using bss_sbrk and simulating 
the heap in a static buffer is that whatever has been stored in that 
buffer gets dumped into emacs.exe as initialized data.  See unexcw.c.

 > In theory that should also break, as long as
> temacs isn't also build large address aware.  The only difference,
> _heapbase = 0x20000000.  But if temacs gets build with large address
> awareness set, _heapbase should become 0x80000000.

That would make _heapbase (which is part of the dumped emacs.exe) depend 
on the build system.  Obviously, as you say below, _heapbase needs to be 
determined at run time.

> However, whatever you do, it will not really work.  Keep in mind that
> the large address awareness only makes sense (and has any effect!) on
> systems which provide a large address area.
>
> To me the bottom line here is, that emacs is doing the wrong thing.
> There are a couple of assumptions how a system maintains memory, which
> are just not valid on all systems.  The malloc initialization and the
> assignment of the heapbase (the first call to sbrk(0)) should happen
> in emacs every time it starts.

That makes sense to me.  I thought that was what I was accomplishing 
(for Cygwin) by setting __malloc_initialized to 0 before dumping.  I'm 
not sure why it didn't work.  In any case, the fix shouldn't be Cygwin 
specific.  It's probably time to report this as an emacs bug.

Ken

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple