• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Liutauras Vilda
  • Devaka Cooray
  • Jeanne Boyarsky
  • Bear Bibeault
Sheriffs:
  • Junilu Lacar
  • Paul Clapham
  • Knute Snortum
Saloon Keepers:
  • Ron McLeod
  • Tim Moores
  • Stephan van Hulst
  • salvin francis
  • Carey Brown
Bartenders:
  • Tim Holloway
  • Frits Walraven
  • Ganesh Patekar

Rounding technique  RSS feed

 
Greenhorn
Posts: 26
2
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi,

1. What is the rounding technique used for Math.round()? I believe it is using HALF-UP rounding (The one we used to learn in elementary school).

2. I know float data type holds 7 significant digits and double holds 16 significant digits. However, when I tested it, the output looks weird.



output:
3.141592653589793 (why double type only outputs 15-significant digits? not 16 significant digits?)
3.1415923 (why is the 7th significant digit 3? What rounding technique does it use?)
3.141593 (why only shows 6-significant digits? not 7?)
 
All of the world's problems can be solved in a garden - Geoff Lawton. Tiny ad:
RavenDB is an Open Source NoSQL Database that’s fully transactional (ACID) across your database
https://coderanch.com/t/704633/RavenDB-Open-Source-NoSQL-Database
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
Boost this thread!