Mail Archives: cygwin/2005/07/05/05:42:21
----Original Message----
>From: Peter J. Acklam
>Sent: 30 June 2005 13:13
> Dave Korn [dave DOT korn AT artimiOHHHHHNOOOOOOOOO!] wrote:
^^^^^^^^^^^^^^^^^^^^^^^^
>
>> haro AT kgtPLEASETAKEMORECARE! wrote:
^^^^^^^^^^^^^^^^^^^^^^
Peter, please do http://cygwin.com/acronyms#PCYMTNQREAIYR before you get
me loaded down with even more spam than I already suffer, thank you!
>>> How come "0.125" gets printed as "0.12", and not "1.3"?
>>
>> Absolutely, there's a rounding error of some sort.
>
> For what it's worth: My Sunblade 100 running Solaris 9 has
> Solaris' /bin/printf and GNU's printf as /usr/local/bin/printf
> and both give "0.12", not "0.13".
>
> I am quite sure it is the printf returning "0.13" that is buggy.
> Note that the value "0.125" can be represented exactly with IEEE
> double precision arithmetic, so inexact representation is not the
> matter here.
Progress report:
Stepping through the code at the weekend, I followed the 0.105 case as far
down as ldtoa_r, where I observed that although there was code that would
round 0.105 up to 0.11 if asked for only two sig.figs, the loop that
generates successive digits was coming up with the sequence
'0.10499999999.....', and because the rounding only looks at one digit
beyond the requested s.f., it 'correctly' rounds 0.104 down to 0.10.
I say 'correctly' in quotes, because it's the correct way to round 0.104,
but it's not in general correct to round a number by truncating it and then
rounding the truncated version. That's at least part of the problem. I
haven't checked yet whether 0.105 should be an exact representation in fp,
but it could also be the case that some accuracy has been lost by the time
it arrives there.
[ I notice that mingw has a maths library in a subdir called mingwex, and
it looks like this is a version of the same code (SLM's cephes lib, but the
ldtoa implementation appears to be something that's been added subsequently
AFAICT) and would show the same problem, but when you actually compile the
testcase with -mno-cygwin the generated exe calls the msvcrt version of
printf (which DTRT, and which is the version you'd expect it to call), so
I'm not sure why it's there or what it's relevant to. ]
cheers,
DaveK
--
Can't think of a witty .sigline today....
--
Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Problem reports: http://cygwin.com/problems.html
Documentation: http://cygwin.com/docs.html
FAQ: http://cygwin.com/faq/
- Raw text -