From:	"Martin Ambuhl" <mambuhl AT tiac DOT net>
Newsgroups:	comp.os.msdos.djgpp
Subject:	Re: poor fp precision
Date:	Mon, 6 Apr 1998 13:56:35 -0400
Organization:	Honors Bridge Club
Lines:	19
Message-ID:	<6gb5gb$dmd@news-central.tiac.net>
References:	<35289C54 DOT 7FED AT hotmail DOT com DOT no DOT spam>
NNTP-Posting-Host:	209.61.81.120
To:	djgpp AT delorie DOT com
DJ-Gateway:	from newsgroup comp.os.msdos.djgpp

Spider wrote in message <35289C54 DOT 7FED AT hotmail DOT com DOT no DOT spam>...
2.390000        238.999987
                ^^^^^^^^^^
Isnīt this very poor????? I mean, if floats have a 6 decimals precision,
one would expect an error less than exp(-6), right??

Huh?
The error in 13/290000000 is 4.48276e-08=pow(10.,-7.348455)
The error in .5/100000 is 5e-06=pow(10.,-5.301030)
The error in .5/999999 is 5.00001e-07=pow(10.,-6.301030)

So 6 decimals precision requires at most that the error be less
than .5/999999.  Your example gives
7.348455 - 6.301030 =  1.047425 MORE digits of precision.

- Raw text -