Mail Archives: djgpp-workers/2003/02/10/07:08:30

Date: Mon, 10 Feb 2003 14:08:25 +0200 (EET)
From: Esa A E Peuha <peuha AT cc DOT helsinki DOT fi>
Sender: peuha AT sirppi DOT helsinki DOT fi
To: djgpp-workers AT delorie DOT com
Subject: Re: Checking for stack overflow
In-Reply-To: <>
Message-ID: <>
References: <10302071430 DOT AA20844 AT clio DOT rice DOT edu>
MIME-Version: 1.0
Reply-To: djgpp-workers AT delorie DOT com
Errors-To: nobody AT delorie DOT com
X-Mailing-List: djgpp-workers AT delorie DOT com
X-Unsubscribes-To: listserv AT delorie DOT com

On Fri, 7 Feb 2003, Charles Sandmann wrote:

> > Won't that cause an exception even now?  But I'll keep that in mind.
> If near pointers are enabled, or under Windows where it can do the
> address wrap thing behind your back - no ...

OK.  Here's what I wrote during the weekend.  Assuming that esp has
already decremented to point to the newly allocated space, the possible
overflow is detected thus:

	cmpl	___djgpp_stack_limit, %esp
	jge	0f
	movl	$0f, ___djgpp_stack_overflow_eip
	jmp	___djgpp_stack_overflow_exit

This will fail only if the size of allocation is so ridiculously large
that esp wraps over to a huge positive value (when considered as signed
int), but gcc seems to guard against this already (at least the size of
a variable-sized automatic array is anded so that it's at most
0x20000000 IIRC, and built-in alloca should do something similar).

Then there's the overflow handler (which might be put in
src/libc/go32/overflow.S if there are no better suggestions):

	.balign	8
	.comm	exception_stack, 8000
	.balign	8
overflow_state_buf:		/* jmp_buf */
	.long	0, 0, 0, 0	/* eax, ebx, ecx, edx */
	.long	0, 0, 0, 0	/* esi, edi, ebp, esp */
	.globl	___djgpp_stack_overflow_eip
	.long	0, 0		/* eip, eflags */
	.word	0, 0, 0, 0, 0, 0/* cs, ds, es, fs, gs, ss */
	.long	0, 0, 0		/* sigmask, signum, exception_ptr */
	.ascii	"Out of stack. \0"
	.balign	16,,7
	.globl	___djgpp_stack_overflow_exit
	movl	%eax, overflow_state_buf
	movl	$overflow_state_buf, %eax

	/* change stack as soon as possible */
	movl	%esp, 28(%eax)
	movl	$exception_stack + 8000, %esp

	/* eax already stored */
	movl	%ebx, 4(%eax)
	movl	%ecx, 8(%eax)
	movl	%edx, 12(%eax)
	movl	%esi, 16(%eax)
	movl	%edi, 20(%eax)
	movl	%ebp, 24(%eax)
	/* esp already stored */
	/* eip stored before jumping here */
	popl	36(%eax)
	movw	%cs, 40(%eax)
	movw	%ds, 42(%eax)
	movw	%es, 44(%eax)
	movw	%fs, 46(%eax)
	movw	%gs, 48(%eax)
	movw	%ss, 50(%eax)

	/* fake exception number like in __djgpp_traceback_exit,
	   0x7e == 0x7a + 1 + SIGSEGV - SIGABRT */
	movl	$0x7e, 56(%eax)

	movl	$overflow_state_buf, ___djgpp_exception_state_ptr

	/* print error message */
	pushl	$14
	pushl	$errstring
	pushl	$2
	call	__write

	/* 291 == SIGSEGV */
	pushl	$291

	/* this does not return */
	call	___djgpp_traceback_exit

This works in the sense that it exits with a traceback, but I'm not
quite satisfied with the output.  For one thing, if the traceback has
more entries than will fit the screen (very likely in case of infinite
recursion), then the line with "Out of stack. Exiting due to signal
SIGSEGV" will be scrolled out of screen (this seems like a bug in
show_call_frame).  For another, do_faulting_finish_message might print
"Stack overflown" instead of "Exception 7e", but I didn't know what
value to pass in signum for that (clearly 0x7e won't do since it might
be used by __djgpp_traceback_exit in other cases with SIGSEGV).

Esa Peuha
student of mathematics at the University of Helsinki

- Raw text -

  webmaster     delorie software   privacy  
  Copyright 2019   by DJ Delorie     Updated Jul 2019