henry akoma

Greenhorn

Posts: 21

posted 11 years ago

hi guys,

some colleagues and i were writting a calculator program, but the result of some of our calculations were giving approximate answers and not the exact one.for example:

subtraction gave -0.100000024 instead of -0.1.

please can anyone help us with what to do to get the correct answer and may be decimal place

some colleagues and i were writting a calculator program, but the result of some of our calculations were giving approximate answers and not the exact one.for example:

subtraction gave -0.100000024 instead of -0.1.

please can anyone help us with what to do to get the correct answer and may be decimal place

Ulf Dittmer

Rancher

Posts: 42970

73

posted 11 years ago

floats are inherently limited in their precision, and some numbers just can't be expressed precisely using them. "0.1" is such a number, as is "0.9". Using doubles makes things a bit better (around 14 exact decimals instead of 7 or so), but the basic problem is the same. (If you're interested in the gory details, read up on What Every Computer Scientist Should Know About Floating-Point Arithmetic.)

If you require exact floating point arithmetic, you can use the java.math.BigDecimal class.

[ August 09, 2005: Message edited by: Ulf Dittmer ]

If you require exact floating point arithmetic, you can use the java.math.BigDecimal class.

[ August 09, 2005: Message edited by: Ulf Dittmer ]

Stuart Gray

Ranch Hand

Posts: 410

posted 11 years ago

There are well known problems with floating point arithmetic. You could try BigDecimal in the java.math package.