Win a copy of Functional Reactive Programming this week in the Other Languages forum!
  • Post Reply
  • Bookmark Topic Watch Topic
  • New Topic

Precision?

 
Mark Howard
Ranch Hand
Posts: 285
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
This program:

... produces this output: "Total = 99.99905"
Not 100 as expected. I believe this loss of precision is a well-documented Java fact.
All I want to know is ... WHY?
 
Junilu Lacar
Bartender
Pie
Posts: 7778
62
Android Eclipse IDE IntelliJ IDE Java Linux Mac Scala Spring Ubuntu
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I believe it has to do a lot with the limitations of representing decimal values in binary. If precision or loss thereof is a concern, maybe you could try using the java.lang.StrictMath class.
HTH
J. Lacar

... produces this output: "Total = 99.99905"
Not 100 as expected. I believe this loss of precision is a well-documented Java fact.
All I want to know is ... WHY? [/B]


[This message has been edited by JUNILU LACAR (edited February 28, 2001).]
 
Cindy Glass
"The Hood"
Sheriff
Posts: 8521
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
The loss of precision in both floats and doubles is a well documented PROGRAMMING fact. It impacts every language known to man. The issue is that a TRUE float or double would have an infinite number of decimal places. Obviously a computer can not do that. Therefore some rounding concession must be made. Each language handles the rounding issue differently, but for all of them you can only say the float or double is "ABOUT this value".
 
Mark Howard
Ranch Hand
Posts: 285
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Thanks for the response.
This loss of precision explains my depleting bank balance
[This message has been edited by mark howard (edited February 28, 2001).]
 
Cindy Glass
"The Hood"
Sheriff
Posts: 8521
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Yeah, I also have a float value in my account. I keep trying to cast it into a double but. . . . .
 
  • Post Reply
  • Bookmark Topic Watch Topic
  • New Topic