• Post Reply Bookmark Topic Watch Topic
  • New Topic

loss of precision float and double  RSS feed

 
Sergej Smoljanov
Ranch Hand
Posts: 472
10
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

or

what is rule to determine max value for double and float that result System.out.println(big - (int)approx); will be 0?
and what is max value long and int?
 
Henry Wong
author
Sheriff
Posts: 23295
125
C++ Chrome Eclipse IDE Firefox Browser Java jQuery Linux VI Editor Windows
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Sergej Smoljanov wrote:
what is rule to determine max value for double and float that result System.out.println(big - (int)approx); will be 0?
and what is max value long and int?


Not sure what you are trying to test here... but ...

The max value of long and int are constants Long.MAX_VALUE and Integer.MAX_VALUE respectively.

The max value of double and float are ... well ... Infinity. However, if you want the max value that is just less than infinity, then they are constants Double.MAX_VALUE and Float.MAX_VALUE respectively.

Henry
 
Sergej Smoljanov
Ranch Hand
Posts: 472
10
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

result: -46, because loss of precision. if int big = 12345678; result is 0.
I trying understand, how JVM decides where is border where float variable will lost precision after assignment int value.
think answer in IEEE 754 and trying find it
 
Henry Wong
author
Sheriff
Posts: 23295
125
C++ Chrome Eclipse IDE Firefox Browser Java jQuery Linux VI Editor Windows
  • Likes 1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Sergej Smoljanov wrote:
result: -46, because loss of precision. if int big = 12345678; result is 0.
I trying understand, how JVM decides where is border where float variable will lost precision after assignment int value.
think answer in IEEE 754 and trying find it


Correct. It is not really related to Java, but related to the format that it follows. With Java, int and long follows the Twos Complement format, and float and double follows the IEEE 754 format.

As for figuring out "where is border" of the loss of precision. Good luck. The problem here is that you are trained in decimal math. You think in base 10. And even with scientific notation, you are trained in a decimal number times TEN that is raised to another decimal number. So, when there is loss of precision, you need to envision it as such.

The computer, and the IEEE 754 format, is defined in binary. It is a binary number times TWO that is raised to another binary number -- plus there are some special cases too. I am sure that the border is easy to envision if you think in binary -- but I know I can't do it...

Henry
 
Sergej Smoljanov
Ranch Hand
Posts: 472
10
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
for first example on my PC result is 16777216
used for loop for find
 
Winston Gutkowski
Bartender
Posts: 10575
66
Eclipse IDE Hibernate Ubuntu
  • Likes 1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Sergej Smoljanov wrote:I trying understand, how JVM decides where is border where float variable will lost precision after assignment int value.
think answer in IEEE 754 and trying find it

You're right. And the JVM doesn't "decide" anything; it's defined by the specification. Probably the best place to start is here, specifically the "Decimal digits" column.

The main thing to understand is that precision is not related directly to things like MAX_VALUE (although it's "sort of" related exponentially). A float can hold far bigger values than a long, despite the fact that it's half the size; they just won't be exact. Indeed, there are many values that a float or double can't hold exactly, the most famous of which is 0.1.

You might want to read this page, which goes into the business in more detail. And it's so good, it's simply known as "Goldberg" now.

HIH

Winston
 
Campbell Ritchie
Marshal
Posts: 56593
172
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
The Goldberg link is better than the other.
 
It is sorta covered in the JavaRanch Style Guide.
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
Boost this thread!