type range question

[Bora Sabrioglu]

Hi all,

if you have

long var = 1000000 * 1000000;
System.out.println(var);

the result is -727379968 aus... but Long.MAX_VALUE is 9223372036854775807 ... so 1000000000000 would fit in there quite comfortably....

another strange thing is:

if you have

double var = 0.1+0.1+0.1+0.1+0.1+0.1+0.1+0.1+0.1+0.1-1.0;
System.out.println(var);

then the result is: -1.1102230246251565E-16 and not 0.0 as expected...

Can someone explain this?

Thanks!

[Joanne Neal]

You are actually multiplying two ints together (which will produce an int result) and then putting that (truncated) value into a long.
You need to make one of the values a long.
Try
long var = 1000000 * 1000000L;
System.out.println(var);

[Bora Sabrioglu]

oh i see, yes... thanks alot...

what about this:

if you have

double var = 0.1+0.1+0.1+0.1+0.1+0.1+0.1+0.1+0.1+0.1-1.0;
System.out.println(var);

then the result is: -1.1102230246251565E-16 and not 0.0 as expected...

[Joanne Neal]

That's a common question

[Winston Gutkowski]

Joanne Neal wrote:That's a common question

And this is one of the best pages I know for explaining why it happens.
It's also the reason why you shouldn't use doubles or floats for calculations involving money.

Winston

[Campbell Ritchie]

Bora Sabrioglu wrote: . . . -1.1102230246251565E-16 and not 0.0 as expected...

I ahve seen that before and did not expect to see 0.0. If you try log(1.1102230246251565E-16) ÷ log(2), you get -53.000. Remember that a double has 53 bits’ precision, so that suggests you are 1 bit out in the last digit, what is here called a ulp.
By the way: is that an ulp or a ulp?

[Bora Sabrioglu]

thats right... if I delete one of the 0.1, then the result is -0.10000000000000009 ... thanks alot guys.

[Campbell Ritchie]

You’re welcome