Floating point error (more info)

I just posted regarding a possible floating point error (not sure
where), and have since discovered that:

float f = 5.15002;
double d = 5.15002;
if (float(d) f)
puts("huh 1?");
float f2 = float(d);
if (f2 f)
puts("huh 2?");


Causes 'huh 1' to be printed, but NOT 'huh 2'.

I can't tell if this is a compiler program (MSVC 6), an FPU problem or
what!

The assembly code is:

6: float f = 5.15002;
00401028 C7 45 FC F7 CC A4 40 mov dword ptr [ebp-4],40A4CCF7h
7: double d = 5.15002;
0040102F C7 45 F4 D2 FB C6 D7 mov dword ptr
[ebp-0Ch],0D7C6FBD2h
00401036 C7 45 F8 9E 99 14 40 mov dword ptr [ebp-8],4014999Eh
8: if (float(d) f)
0040103D DD 45 F4 fld qword ptr [ebp-0Ch]
00401040 D9 55 EC fst dword ptr [ebp-14h]
00401043 D8 5D FC fcomp dword ptr [ebp-4]
00401046 DF E0 fnstsw ax
00401048 F6 C4 01 test ah,1
0040104B 74 0D je main+4Ah (0040105a)
9: puts("huh 1?");
0040104D 68 24 60 42 00 push offset string "huh 1?"
(00426024)
00401052 E8 39 00 00 00 call puts (00401090)
00401057 83 C4 04 add esp,4
10: float f2 = float(d);
0040105A DD 45 F4 fld qword ptr [ebp-0Ch]
0040105D D9 5D F0 fstp dword ptr [ebp-10h]
11: if (f2 f)
00401060 D9 45 F0 fld dword ptr [ebp-10h]
00401063 D8 5D FC fcomp dword ptr [ebp-4]
00401066 DF E0 fnstsw ax
00401068 F6 C4 01 test ah,1
0040106B 74 0D je main+6Ah (0040107a)
12: puts("huh 2?");
0040106D 68 1C 60 42 00 push offset string "huh 2?"
(0042601c)
00401072 E8 19 00 00 00 call puts (00401090)
00401077 83 C4 04 add esp,4

In general, floating point numbers cannot be represented precisely.

You can probably find various texts on this by searching the web for IEEE
Floating Pointing with your favorite search engine.
One such reference that may be helpful is:

http://msdn.microsoft.com/library/de...us/vccore/html
/_core_why_floating_point_numbers_may_lose_precisio n.asp

If you build your program for debug, step thru it, and look at the memory
contents
(the actual hex values, not the converted base 10 representation) being
compared this will probably be clearer.

- joel


"Dylan Nicholson" wrote in message
om...
I just posted regarding a possible floating point error (not sure
where), and have since discovered that:

float f = 5.15002;
double d = 5.15002;
if (float(d) f)
puts("huh 1?");
float f2 = float(d);
if (f2 f)
puts("huh 2?");


Causes 'huh 1' to be printed, but NOT 'huh 2'.

I can't tell if this is a compiler program (MSVC 6), an FPU problem or
what!

The assembly code is:

6: float f = 5.15002;
00401028 C7 45 FC F7 CC A4 40 mov dword ptr [ebp-4],40A4CCF7h
7: double d = 5.15002;
0040102F C7 45 F4 D2 FB C6 D7 mov dword ptr
[ebp-0Ch],0D7C6FBD2h
00401036 C7 45 F8 9E 99 14 40 mov dword ptr [ebp-8],4014999Eh
8: if (float(d) f)
0040103D DD 45 F4 fld qword ptr [ebp-0Ch]
00401040 D9 55 EC fst dword ptr [ebp-14h]
00401043 D8 5D FC fcomp dword ptr [ebp-4]
00401046 DF E0 fnstsw ax
00401048 F6 C4 01 test ah,1
0040104B 74 0D je main+4Ah (0040105a)
9: puts("huh 1?");
0040104D 68 24 60 42 00 push offset string "huh 1?"
(00426024)
00401052 E8 39 00 00 00 call puts (00401090)
00401057 83 C4 04 add esp,4
10: float f2 = float(d);
0040105A DD 45 F4 fld qword ptr [ebp-0Ch]
0040105D D9 5D F0 fstp dword ptr [ebp-10h]
11: if (f2 f)
00401060 D9 45 F0 fld dword ptr [ebp-10h]
00401063 D8 5D FC fcomp dword ptr [ebp-4]
00401066 DF E0 fnstsw ax
00401068 F6 C4 01 test ah,1
0040106B 74 0D je main+6Ah (0040107a)
12: puts("huh 2?");
0040106D 68 1C 60 42 00 push offset string "huh 2?"
(0042601c)
00401072 E8 19 00 00 00 call puts (00401090)
00401077 83 C4 04 add esp,4

On 15 Jul 2003 19:33:07 -0700, (Dylan Nicholson)
wrote:

I just posted regarding a possible floating point error (not sure
where), and have since discovered that:

float f = 5.15002;
double d = 5.15002;
if (float(d) f)
puts("huh 1?");
float f2 = float(d);
if (f2 f)
puts("huh 2?");


Causes 'huh 1' to be printed, but NOT 'huh 2'.


First, this appears to be c++, not c, although the concepts are
essentially the same in this case. VC6 supports C90, and not C99, so
there are several compiler errors in the above.

Second, set your compiler to conforming mode:
/Za Disable Language Extensions (check box selected) ANSI C
compatibility. Language constructs not compatible with ANSI C are
flagged as errors. Note that this also turns on the /Op (Improve Float
Consistency) flag, which is what is required here.

With /Za set, neither "huh 1" nor "huh 2" is output, and there are no
anomalies, and the behavior is conforming.

However, without /Za set, VC will default to a mode where floating-point
values remain in registers, and are not type-converted in all cases
dictated by the standard. I got both "huh 1" and "huh 2" output, which
is what I would expect, given VC's default behavior.

Relative to your first post,

float f = 5.15002;
double d = 5.15002;
if (d + FLT_EPSILON f)
puts("huh?");

it would appear that f is rounded up when converted from double to
float, more than FLT_EPSILON. Note that 5.15002 has type double.

Informally, what I see in the debugger is
5.1500201225281 = f (as double)
5.1500201192093 = d + FLT_EPSILON
5.1500200000000 = d
so the result we get is consistent with what we see here.

As another poster has suggested, you may want to look into the problems
associated with floating-point representations.

Regards,
Bruce Wheeler

"Dylan Nicholson" wrote in message
om...

(snip)

| float f = 5.15002;
| double d = 5.15002;
| if (float(d) f)
| puts("huh 1?");
| float f2 = float(d);
| if (f2 f)
| puts("huh 2?");
|
|
| Causes 'huh 1' to be printed, but NOT 'huh 2'.
|
| I can't tell if this is a compiler program (MSVC 6), an FPU problem or
| what!

snip
If you first cast the double to a float, both numbers are
truncated to 6 digits and compare equal.

Um, if you look at the code, that's exactly what I did, in both cases.
However only one of the comparisons behaves unexpectedly.
I can't see how the above can be anything other than a compiler error
or a FPU instruction set inconsistency (unfortunately I'm not at all
familiar with FPU instructions).

That is not how you cast in C. I don't know that a float() function exists,
though.

It should be if((float)df) or, to be sure about precedence
if(((float)d)f)

In any case, because of the way the internal registers in the floating point
processor it is common for values to be held with a higher precision than
specified. Good or bad, C allows this. Sometimes more precision is good,
sometimes bad.

-- glen
-- glen

In comp.lang.c Glen Herrmannsfeldt wrote:
[...]
It should be if((float)df) or, to be sure about precedence
if(((float)d)f)

There is a simple rule that applies here, which is worth memorising -
all unary operators (typecasts are unary operators) have higher
precedence than the binary operators.

- Kevin

(Bruce Wheeler) wrote in message ...
On 15 Jul 2003 19:33:07 -0700, (Dylan Nicholson)
wrote:

I just posted regarding a possible floating point error (not sure
where), and have since discovered that:

float f = 5.15002;
double d = 5.15002;
if (float(d) f)
puts("huh 1?");
float f2 = float(d);
if (f2 f)
puts("huh 2?");


Causes 'huh 1' to be printed, but NOT 'huh 2'.


First, this appears to be c++, not c, although the concepts are
essentially the same in this case. VC6 supports C90, and not C99, so
there are several compiler errors in the above.

Ah yes, sorry...I meant to convert to C before posting it...
snip

However, without /Za set, VC will default to a mode where floating-point
values remain in registers, and are not type-converted in all cases
dictated by the standard. I got both "huh 1" and "huh 2" output, which
is what I would expect, given VC's default behavior.

So you can't get it to just output huh 1 but not huh 2? You may need
to do it in debug mode (in fact in release mode the floating
comparisons are optimized out, as they are fixed at compile time).
I still don't understand how it could matter which way the casting is
done (explictly inline, or implicity via storing to a float variable).
And I just don't know the x87 FPU instruction set well enough to
understand the difference in the compiled output.

Relative to your first post,

float f = 5.15002;
double d = 5.15002;
if (d + FLT_EPSILON f)
puts("huh?");

it would appear that f is rounded up when converted from double to
float, more than FLT_EPSILON. Note that 5.15002 has type double.

Informally, what I see in the debugger is
5.1500201225281 = f (as double)
5.1500201192093 = d + FLT_EPSILON
5.1500200000000 = d
so the result we get is consistent with what we see here.

Still, even in this example, it's because I'm using a mixture of
doubles and floats that the problem occurs.

As another poster has suggested, you may want to look into the problems
associated with floating-point representations.

Well yes I'm well aware of those problems, but I don't see how they're
relevant here. I know I was using FLT_EPSILON incorrectly, but
likewise I can't see how that should matter. In general if you assign
the *same* decimal value to two floats, they should never compare as
anything but equal, even if they aren't exactly the value I assigned
to them.

Dylan

"Dylan Nicholson" wrote in message
om...
(Bruce Wheeler) wrote in message
...
On 15 Jul 2003 19:33:07 -0700, (Dylan Nicholson)
wrote:

I just posted regarding a possible floating point error (not sure
where), and have since discovered that:

float f = 5.15002;
double d = 5.15002;
if (float(d) f)
puts("huh 1?");
float f2 = float(d);
if (f2 f)
puts("huh 2?");


Causes 'huh 1' to be printed, but NOT 'huh 2'.


First, this appears to be c++, not c, although the concepts are
essentially the same in this case. VC6 supports C90, and not C99, so
there are several compiler errors in the above.

Ah yes, sorry...I meant to convert to C before posting it...
snip

However, without /Za set, VC will default to a mode where floating-point
values remain in registers, and are not type-converted in all cases
dictated by the standard. I got both "huh 1" and "huh 2" output, which
is what I would expect, given VC's default behavior.

So you can't get it to just output huh 1 but not huh 2? You may need
to do it in debug mode (in fact in release mode the floating
comparisons are optimized out, as they are fixed at compile time).
I still don't understand how it could matter which way the casting is
done (explictly inline, or implicity via storing to a float variable).
And I just don't know the x87 FPU instruction set well enough to
understand the difference in the compiled output.

Relative to your first post,

float f = 5.15002;
double d = 5.15002;
if (d + FLT_EPSILON f)
puts("huh?");

it would appear that f is rounded up when converted from double to
float, more than FLT_EPSILON. Note that 5.15002 has type double.

Informally, what I see in the debugger is
5.1500201225281 = f (as double)
5.1500201192093 = d + FLT_EPSILON
5.1500200000000 = d
so the result we get is consistent with what we see here.

Still, even in this example, it's because I'm using a mixture of
doubles and floats that the problem occurs.

As another poster has suggested, you may want to look into the problems
associated with floating-point representations.

Well yes I'm well aware of those problems, but I don't see how they're
relevant here. I know I was using FLT_EPSILON incorrectly, but
likewise I can't see how that should matter. In general if you assign
the *same* decimal value to two floats, they should never compare as
anything but equal, even if they aren't exactly the value I assigned
to them.


But you didn't have the "same" value, one was float, one was double.
The two approximations of 5.15002 are different. If you wanted them
to be the same, you might have said something like:

float f = (float)5.15002;
double d = (float)5.15002;

You chose instead to allow the compiler to decide what particular
approximation of 5.15002 was loaded into each variable.

In other languages you would have been more precise in your
description of the precision of each variable and each constant.

Just for grins, try it with a number that has an exact representation
in a float, and see what happens :-)

--

... Hank

Hank: http://horedson.home.att.net
W0RLI: http://w0rli.home.att.net

(Bruce Wheeler) wrote in message news:3f178ba5.2545420@news...

good explanations

Thanks for the replies, Bruce, I'm glad *someone* here understood my confusion!
Which group were you posting from, btw?

Dylan

On 18 Jul 2003 18:40:59 -0700, (Dylan
Nicholson) wrote:

(Bruce Wheeler) wrote in message news:3f178ba5.2545420@news...

good explanations

Thanks for the replies, Bruce, I'm glad *someone* here understood my confusion!
Which group were you posting from, btw?

Dylan


Glad to be of help.

I'm posting from comp.lang.c. However, clc is concerned with
portable c topics (topics which relate to all c implementations).
Generally, implementation-specific topics should not be discussed
there. I tried to answer your questions in the context of a
specific case of general floating-point topics.

I'm not sure what newsgroups would be appropriate for further VC
and Intel-specific questions. clc really isn't. Maybe you should
try comp.programming if you have more questions of this type
(although I am not active in that group).

Regards,
Bruce Wheeler