This week's book giveaway is in the JavaScript forum.
We're giving away four copies of Cross-Platform Desktop Applications: Using Node, Electron, and NW.js and have Paul Jensen on-line!
See this thread for details.
Win a copy of Cross-Platform Desktop Applications: Using Node, Electron, and NW.js this week in the JavaScript forum!
  • Post Reply Bookmark Topic Watch Topic
  • New Topic

Mathematical Oddity in JSP?  RSS feed

 
Sahava Singh
Greenhorn
Posts: 2
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Why does
<%=(-(839540.58)+(-1111374.16)+(-1169778.73)+(-2050049.40))%>
equal 5170742.869999999 and not 5170742.87?
(I'm using win2k running oc4j 10g)
 
Billybob Marshall
Ranch Hand
Posts: 202
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
It's just simply because of the way IEEE floating-point numbers are represented and calculated. They are stored as complicated binary entities, not as strings like "123.45". Whenever floating-point math is performed, you have to account for minute precision loss. If you want to show the user a number with 2 decimal digits of precision, then just use the java.text.NumberFormat-related classes to format the result, and it will look fine. But you can't guarantee that the number in memory is exact - just very close.
 
Sahava Singh
Greenhorn
Posts: 2
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Thank you! :-)
 
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
Boost this thread!