Date: Mon, 28 Apr 2003 21:20:12 +0300
From: "Eli Zaretskii" <eliz AT elta DOT co DOT il>
Sender: halo1 AT zahav DOT net DOT il
To: rich AT phekda DOT freeserve DOT co DOT uk
Message-Id: <9003-Mon28Apr2003212012+0300-eliz@elta.co.il>
X-Mailer: emacs 21.3.50 (via feedmail 8 I) and Blat ver 1.8.9
CC: djgpp-workers AT delorie DOT com
In-reply-to: <3EAC4195.67389371@phekda.freeserve.co.uk> (message from Richard
	Dawe on Sun, 27 Apr 2003 21:46:13 +0100)
Subject: Re: make 3.80 test failures
References: <3EA820C2 DOT DBBDD50B AT phekda DOT freeserve DOT co DOT uk> <1659-Thu24Apr2003232926+0300-eliz AT elta DOT co DOT il> <3EAC4195 DOT 67389371 AT phekda DOT freeserve DOT co DOT uk>
Reply-To: djgpp-workers AT delorie DOT com
Errors-To: nobody AT delorie DOT com
X-Mailing-List: djgpp-workers AT delorie DOT com
X-Unsubscribes-To: listserv AT delorie DOT com
Precedence: bulk

> Date: Sun, 27 Apr 2003 21:46:13 +0100
> From: Richard Dawe <rich AT phekda DOT freeserve DOT co DOT uk>
> > 
> > Please show the tests and the script that pinpoints the error.
> 
> I don't quite understand what you mean.

I meant to ask what Makefiles/scripts does each failing test run,
what results are expected by the suite, and what you actually get.
(Sorry, I don't have time to run the test suite and analyze the
results myself.)

> I didn't run any scripts.

I meant the session transcript, typically produced by a program such
as SCRIPT.

> The particular parts that fail (as identified by the .diff files from the work
> directory) with the 2.03 build are:
> 
> features/vpathplus: 2, 3
> targets/INTERMEDIATE: 2
> targets/SECONDARY: none
> 
> Hmmm, it's strange that SECONDARY doesn't fail, if I only run that test.
> 
> The particular parts that fail with 2.04 alpha 1 build are:
> 
> features/vpathplus: none
> targets/INTERMEDIATE: none
> targets/SECONDARY: 2, 3, 5
> 
> Hmmm, it's strange that vpathplus and INTERMEDIATE don't fail, if I only run a
> test.

This seems to be consistent with my hypothesis (see below).

> Is there some kind of weird interaction between tests here?

I have found some of the tests to be very sensitive to timing issues.
Since we are mostly using very fast machines, and file time stamps
returned by system calls we use have a 2-sec granularity, and because
Make depends on file time-stamps to decide which ones to rebuild, an
automated script can easily fool Make if it produces files whose time
stamps are not monotonically increasing.  Add to this the known
misfeature of all versions of Windows from 98 and later, whereby a
file's time stamp can be ahead of the system clock by as much as 3
seconds (!!), and you can easily understand, I think, what kind of
bloody mess this can create in the test suite.

You will see that quite a few of the tests sleep for several seconds
after they produce file--that's the result of my debugging such
problems in the past.  It's quite possible that faster machines and/or
newly added tests need similar changes.  As a rule of thumb, look for
files created by the scripts or commands run by the failing test, and
add "sleep(4)" after each file they create.  (If you find such changes
are necessary, please send them to the GNU Make maintainer.)