posted 4 years ago

Hi,

I have this formula which divides an int by int and then multiplies it by 0.3, the I save this in variable "fre". according to my calculator variable fre should be 0.2, but my program returns 0.19999999999999998. I dont know what am I doing wrong?

variables values are:

freqValue:2

maxFreqInPage:3

and here is the code snippet:

I have converted the ints to doubles.

Thank you!

I have this formula which divides an int by int and then multiplies it by 0.3, the I save this in variable "fre". according to my calculator variable fre should be 0.2, but my program returns 0.19999999999999998. I dont know what am I doing wrong?

variables values are:

freqValue:2

maxFreqInPage:3

and here is the code snippet:

I have converted the ints to doubles.

Thank you!

posted 4 years ago

Your calculator probably comes up with the same value as Java, or one like it, but just rounds it before displaying.

The short version is this: It's impossible to exactly represent 0.2 in a double, for basically the same reason that it's impossible to represent 1/3 in a finite number of digits in base-10.

Longer version: What Every Computer Scientist Should Know About Floating-Point Arithmetic, by David Goldberg.

If you want it to display with less precision, you can use one of the following:

java.text.DecimalFormat String.format() System.out.printf() (which calls String.format())

The short version is this: It's impossible to exactly represent 0.2 in a double, for basically the same reason that it's impossible to represent 1/3 in a finite number of digits in base-10.

Longer version: What Every Computer Scientist Should Know About Floating-Point Arithmetic, by David Goldberg.

If you want it to display with less precision, you can use one of the following: