Date:	Wed, 12 Jan 2000 08:48:07 +0200 (IST)
From:	Eli Zaretskii <eliz AT is DOT elta DOT co DOT il>
X-Sender:	eliz AT is
To:	Martin Stromberg <Martin DOT Stromberg AT lu DOT erisoft DOT se>
cc:	djgpp-workers AT delorie DOT com
Subject:	Re: The test directory (djtst203.zip)
In-Reply-To:	<200001111402.PAA07929@spica-144.lu.erisoft.se>
Message-ID:	<Pine.SUN.3.91.1000112084728.28074C@is>
MIME-Version:	1.0
Reply-To:	djgpp-workers AT delorie DOT com
Errors-To:	dj-admin AT delorie DOT com
X-Mailing-List:	djgpp-workers AT delorie DOT com
X-Unsubscribes-To:	listserv AT delorie DOT com

On Tue, 11 Jan 100, Martin Stromberg wrote:

> > There's another problem with djtst: the test programs usually don't
> > tell you whether the results are okay or not.  You need to examine the
> > results and decide that by yorself.  Sometimes this decision requires
> > a fairly good knowledge of library internals.
> 
> Yeah, but the plan was to generate a log file of the output of make,
> then do my FAT32 changes and running it again and diff the log files.
> If the making starts to work perhaps a log file from DJ should be
> added (to djtst), as "this_worked_at_DJs_compilation.log"?

You are assuming that each test program prints something meaningful,
and that it prints it to stdout/stderr.  This is not true for quite a
few of the programs.

For example, even the Cygnus test suite included in djtst, which IMHO
comes closest to being a true regression test suite, reports its
results to a file.

> > We usually try very hard to avoid using anything but stock djdev
> > binaries (and Make).  Thus, `find', Bash, and `pwd' are out.  You can
> > instead use the special programs (like makemake.exe, misc.exe, etc.)
> > written specifically for building djlsr without any other GNU tools.
> 
> Yeah, but the test suite (djtst) isn't djlsr! 

Still, IMHO we should not require fancy tools.  The technology is
there already: using the same method as in djlsr is simple.

- Raw text -