Win a copy of The Java Performance Companion this week in the Performance forum!
  • Post Reply
  • Bookmark Topic Watch Topic
  • New Topic

what accounts for precision problems when computing floating point numbers?

 
Fendel Coulptz
Greenhorn
Posts: 24
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
like, 0.1+0.1+0.1+0.1+0.1+0.1+0.1+0.1+0.1+0.1 is not equal to one, but 0.99999999999.

why is this so?
 
fred rosenberger
lowercase baba
Bartender
Posts: 12147
31
Chrome Java Linux
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Because computers can't REALLY represent .1

I don't remember exactly how to do it, but to accuratly store it, you'd need an infinite number of places after the decimal point... just like 1/3 is not really 0.3333333333

So, it represents is as close as it can. there are teeny tiny rounding errors, but they can (as you've seen) add up - or NOT add up as the case may be.

hope that helps!!!
 
Fendel Coulptz
Greenhorn
Posts: 24
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
i see... so... is there a maximum number of decimal places it can go? like, for eg, anything beyond 0.99999999 gets trucuated, so 0.999999998888245345 will be viewed as 0.99999999
 
Jeff Bosch
Ranch Hand
Posts: 805
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Exactly. Remember, computers do everything in powers of 2, not 10 like us.
 
fred rosenberger
lowercase baba
Bartender
Posts: 12147
31
Chrome Java Linux
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
you will have this accuracy problem for ALL floating point numbers, whether they are small like 0.000001 or large like 6.024 * 10^23.

the bigger the numbers, the larger the gaps between what can accuratly be represented. if you use this sites search feature, i'm sure you'll find several more threads on this topic. it's a fairly common question... i know i struggled with it when i started programing
 
Fendel Coulptz
Greenhorn
Posts: 24
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
thanks fred n jeff
 
  • Post Reply
  • Bookmark Topic Watch Topic
  • New Topic