• Post Reply Bookmark Topic Watch Topic
  • New Topic

precision !  RSS feed

 
Enamul Haque
Greenhorn
Posts: 21
Eclipse IDE Python Redhat
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
What is wrong with this float?
1230.00089F
ans:
There are nine decimal places of precision. Data type float can't handle that. (The compiler will round the number into a value that can fit in a float).
BUT WHY SO !!!
 
James Chegwidden
Author
Ranch Hand
Posts: 201
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
here is answer from a textbook:

Calculations involving floating-point numbers are approximated because these numbers are not stored with complete accuracy. For example,
System.out.println(1 - 0.1 - 0.1 - 0.1 - 0.1 - 0.1);
displays 0.5000000000000001, not 0.5, and
System.out.println(1.0 - 0.9);
displays 0.09999999999999998, not 0.1. Integers are stored precisely. Therefore, calculations with integers yield a precise integer result.
 
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
Boost this thread!