X-Recipient: archive-cygwin AT delorie DOT com
X-SWARE-Spam-Status: No, hits=-0.9 required=5.0	tests=AWL,BAYES_00,SPF_NEUTRAL
X-Spam-Check-By: sourceware.org
Message-ID: <4E415E43.20407@cs.utoronto.ca>
Date: Tue, 09 Aug 2011 12:20:19 -0400
From: Ryan Johnson <ryan DOT johnson AT cs DOT utoronto DOT ca>
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:5.0) Gecko/20110624 Thunderbird/5.0
MIME-Version: 1.0
To: cygwin AT cygwin DOT com
Subject: Re: emacs and large-address awareness under recent snapshots
References: <4E40093D DOT 10107 AT cornell DOT edu> <20110808162556 DOT GM11601 AT calimero DOT vinschen DOT de> <87sjpbhij7 DOT fsf AT Rainer DOT invalid> <4E404424 DOT 9000600 AT cornell DOT edu> <4E40526C DOT 9080605 AT cornell DOT edu> <4E406C21 DOT 7010007 AT cs DOT umass DOT edu> <20110809082652 DOT GA9492 AT calimero DOT vinschen DOT de> <4E4117AF DOT 3030305 AT cornell DOT edu> <4E414054 DOT 6040206 AT cornell DOT edu> <4E4142F7 DOT 8020708 AT cornell DOT edu> <20110809152155 DOT GB17030 AT calimero DOT vinschen DOT de>
In-Reply-To: <20110809152155.GB17030@calimero.vinschen.de>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
X-IsSubscribed: yes
Mailing-List: contact cygwin-help AT cygwin DOT com; run by ezmlm
List-Id: <cygwin.cygwin.com>
List-Subscribe: <mailto:cygwin-subscribe AT cygwin DOT com>
List-Archive: <http://sourceware.org/ml/cygwin/>
List-Post: <mailto:cygwin AT cygwin DOT com>
List-Help: <mailto:cygwin-help AT cygwin DOT com>, <http://sourceware.org/ml/#faqs>
Sender: cygwin-owner AT cygwin DOT com
Mail-Followup-To: cygwin AT cygwin DOT com
Delivered-To: mailing list cygwin AT cygwin DOT com

On 09/08/2011 11:21 AM, Corinna Vinschen wrote:
> On Aug  9 10:23, Ken Brown wrote:
>> On 8/9/2011 10:12 AM, Ken Brown wrote:
>>> On 8/9/2011 7:19 AM, Ken Brown wrote:
>>>> (gdb) thread 1
>>>> [Switching to thread 1 (Thread 19828.0x447c)]
>>>> #0  0x00622ee0 in morecore_nolock (size=1052672) at gmalloc.c:703
>>>> 703           while ((__malloc_size_t) BLOCK ((char *) result + size)>
>>>> newsize);
>>>> (gdb) p /x size
>>>> $1 = 0x101000
>>>> (gdb) p /x heapsize
>>>> $2 = 0x80000
>>>> (gdb) p result
>>>> $3 = (void *) 0x807d0000
>>>> (gdb) p newsize
>>>> $4 = 0
>>>> (gdb) p _heapbase
>>>> $5 = 0x816000 "\202"
>>>> (gdb) p _heapinfo
>>>> $6 = (malloc_info *) 0x80060000
>>>>
>>>> Is _heapbase the problem?  This is initialized to _heapinfo at the first
>>>> call of malloc and is never changed.  _heapinfo presumably points into
>>>> the static heap at that point.  (_heapinfo is later changed as a result
>>>> of realloc.)  This low value of _heapbase is used in the BLOCK macro.
>>> Here's what I think is happening.  When temacs.exe is running during the
>>> build process (see my explanation of this earlier in the thread),
>>> malloc_init is called and _heapbase is set.  At this point, temacs is
>>> using its own static buffer as the heap, and _heapbase gets the value
>>> 0x816000.  This gets dumped as initialized data into emacs.exe, as does
>>> the value __malloc_initialized = 1.  Now when emacs.exe is run, it sees
>>> that malloc has already been initialized, so _heapbase retains its
>>> value, which is no longer appropriate.  All code relying on the BLOCK
>>> macro is now invalid.
>>>
>>> AFAICS, this has always been wrong.  But the error didn't have dramatic
>>> consequences until the heap was put into high memory.
>>>
>>> I'm not sure what's the best way to fix this (assuming my analysis is
>>> right).  Would it be enough to set __malloc_initialized to 0 before
>>> dumping?  That would force emacs to reinitialize and get the correct
>>> value of _heapbase.
>> No, that's too simple-minded.  I just tried it, and emacs aborted.
>> This seems like a mess.
> What happens if you remove the Cygwin-specific call to bss_sbrk in
> __default_morecore?  In theory that should also break, as long as
> temacs isn't also build large address aware.  The only difference,
> _heapbase = 0x20000000.  But if temacs gets build with large address
> awareness set, _heapbase should become 0x80000000.
>
> However, whatever you do, it will not really work.  Keep in mind that
> the large address awareness only makes sense (and has any effect!) on
> systems which provide a large address area.
>
> To me the bottom line here is, that emacs is doing the wrong thing.
> There are a couple of assumptions how a system maintains memory, which
> are just not valid on all systems.  The malloc initialization and the
> assignment of the heapbase (the first call to sbrk(0)) should happen
> in emacs every time it starts.
I'm pretty sure emacs [thinks it] doesn't even use the system heaps 
(sort of how cygwin doesn't use the windows heaps); from what I 
remember, the "heap" in [t]emacs is an .idata section of the image (12MB 
large on my version of emacs) which is supposed to have unused address 
space afterward, similar to how cygwin allocates its heap. There's even 
a comment there that says they got the idea from cygwin.

Does anybody know why emacs is accessing anything at 0x80000000 in the 
first place?

Ryan


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple