Win a copy of Event Streams in Action this week in the Java in General forum!
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Devaka Cooray
  • Liutauras Vilda
  • Jeanne Boyarsky
  • Bear Bibeault
Sheriffs:
  • Paul Clapham
  • Knute Snortum
  • Rob Spoor
Saloon Keepers:
  • Tim Moores
  • Ron McLeod
  • Piet Souris
  • Stephan van Hulst
  • Carey Brown
Bartenders:
  • Tim Holloway
  • Frits Walraven
  • Ganesh Patekar

What is the correct result of this computation?

 
Bartender
Posts: 1679
17
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Using BigDecimal for the simple calculation below, I get one result in Java (and Python). But using an online calculator and another program, I get another result.

Java Code:



Result:

0.000000000000045454545454545453631211255875255243966038341187705575727952898226001045920632102272727272727272727272727272727272727272727272727272727273

----

But, at this website: https://apfloat.appspot.com/

I get a different result:

.000000000001 / 22
0.00000000000004545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545

--

Which is correct and why?

Thanks,

-- mike
 
Ranch Foreman
Posts: 3298
22
  • Likes 1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
The second is correct - at least, it's what most of us would want.  To see what's wrong with the first, add this line:

I get

which is not what we expect for something that should be 0.000000000001.  Unfortunately, that's the sort of thing that can happen when we use the double primitive type.  The solution is to not use the constructor BigDecimal(double) - see the [url=https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/math/BigDecimal.html#%3Cinit%3E(double)]API for BigDecimal(double) for an explanation why.  Instead, try using BigDecimal(String):

That should give you what you want...
 
Mike London
Bartender
Posts: 1679
17
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

Mike Simmons wrote:The second is correct - at least, it's what most of us would want.  To see what's wrong with the first, add this line:

I get

which is not what we expect for something that should be 0.000000000001.  Unfortunately, that's the sort of thing that can happen when we use the double primitive type.  The solution is to not use the constructor BigDecimal(double) - see the [url=https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/math/BigDecimal.html#%3Cinit%3E(double)]API for BigDecimal(double) for an explanation why.  Instead, try using BigDecimal(String):

That should give you what you want...



Thanks very much!

- mike
 
Marshal
Posts: 65037
247
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Also look up the BigDecimal#valueOf(double) method and see whether that would help you.
 
Ranch Hand
Posts: 573
1
VI Editor Chrome Linux
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
You could also do 1/22, the only thing that changes is the decimal point.  This shows the second result is indeed correct (adjusting the decimal point, of course).

As others have said, you've just discovered the joys of big numbers and floating point math on computers.

 
Campbell Ritchie
Marshal
Posts: 65037
247
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

Jim Venolia wrote:You could also do 1/22 . . .

You mean 1.0 / 22; 1 / 22 evaluates to 0 
 
The only taste of success some people get is to take a bite out of you. Or this tiny ad:
Java Code Review and Psychology
https://coderanch.com/t/714798/java/Java-Code-Review-Psychology
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
Boost this thread!