posted 14 years ago

Hi,

I'm getting a different value than expected using BigDecimal.setScale() method and I wanted to see if someone can tell me why.

Basically, (and this is not currency) a number rounded to 2 digits right of the decimal point. I also choose ROUND_HALF_DOWN so that <=5 is rounded down and >=5 is rounded up.

While JUniting, I set a test case that

0.195 should be equal to 0.19. I expected this to be correct because ROUND_HALF_DOWN is for <=5 and since 5=5 it should round down. But it's not. I'm getting 2.0.

Is this right or do I understand this wrong?

Thanks,

Mike

I'm getting a different value than expected using BigDecimal.setScale() method and I wanted to see if someone can tell me why.

Basically, (and this is not currency) a number rounded to 2 digits right of the decimal point. I also choose ROUND_HALF_DOWN so that <=5 is rounded down and >=5 is rounded up.

While JUniting, I set a test case that

0.195 should be equal to 0.19. I expected this to be correct because ROUND_HALF_DOWN is for <=5 and since 5=5 it should round down. But it's not. I'm getting 2.0.

Is this right or do I understand this wrong?

Thanks,

Mike

Java EE Evangelist — Author, EJB 3 in Action 2nd Edition — Java Community Process Member

posted 14 years ago

I assume you mean 0.20?

Here's what (is probably) going on, and this may be the case on your system as well (I am running JRE 1.4.1_02 on a Windows XP (sp1) box with a 2.4 Ghz Pentium 4 processor, in case it matters...)

Result?

aDecimal: 0.195000000000000006661338147750939242541790008544921875

another: 0.20

ROUND_HALF_DOWN states that it will "round towards 'nearest neighbor' unless both neighbors are equidistant, in which case round down."

However, we are not equidistant here! 0.20 is 000000000000000006661338147750939242541790008544921875 closer than .19 is, so it chooses 0.20

I've never come across a satisfactory solution to this problem, and in fact have my own methods for rounding as a work-around (they convert the BigDecimal to a string and round from there!) Anybody else have expierence with this/have a solution?

I'm getting 2.0.

I assume you mean 0.20?

Here's what (is probably) going on, and this may be the case on your system as well (I am running JRE 1.4.1_02 on a Windows XP (sp1) box with a 2.4 Ghz Pentium 4 processor, in case it matters...)

Result?

aDecimal: 0.195000000000000006661338147750939242541790008544921875

another: 0.20

ROUND_HALF_DOWN states that it will "round towards 'nearest neighbor' unless both neighbors are equidistant, in which case round down."

However, we are not equidistant here! 0.20 is 000000000000000006661338147750939242541790008544921875 closer than .19 is, so it chooses 0.20

I've never come across a satisfactory solution to this problem, and in fact have my own methods for rounding as a work-around (they convert the BigDecimal to a string and round from there!) Anybody else have expierence with this/have a solution?

Piscis Babelis est parvus, flavus, et hiridicus, et est probabiliter insolitissima raritas in toto mundo.

Jim Yingst

Wanderer

Sheriff

Sheriff

Posts: 18671

posted 14 years ago

Consult the API for BigDecimal. The comments for the BigDecimal(double) constructor explain your problem, and point to a solution: use BigDecimal(String) instead for an exact representation of a fractional value.

BigDecimal aDecimal = new BigDecimal("0.1950");

This gives the results you'd expect.

BigDecimal aDecimal = new BigDecimal("0.1950");

This gives the results you'd expect.

"I'm not back." - Bill Harding, *Twister*

posted 14 years ago

That did it. All my JUnit tests worked.

Java EE Evangelist — Author, EJB 3 in Action 2nd Edition — Java Community Process Member

Spencer J Lee

Ranch Hand

Posts: 30

posted 14 years ago

I have a similar problem, but the String solution doesn't quite solve it completely... Just wondering if anyone had any suggestions.

To summarize what I'm doing, the double I'm trying to round is the solution to some algebraic equation.

Here is an example of my current rounding...

Where someAlgEqution() is some algebraic equation that returns a double.

I have problems when someAlgEquation returns a double like 2.14634999999998. Since the 5th decimal is a 4, 2.1463. But really, its only a 4 because 2.14635 isn't represented exactly...

Any suggestions?

Thanks,

Spencer

To summarize what I'm doing, the double I'm trying to round is the solution to some algebraic equation.

Here is an example of my current rounding...

Where someAlgEqution() is some algebraic equation that returns a double.

I have problems when someAlgEquation returns a double like 2.14634999999998. Since the 5th decimal is a 4, 2.1463. But really, its only a 4 because 2.14635 isn't represented exactly...

Any suggestions?

Thanks,

Spencer

posted 14 years ago

I'm not sure about this one. Perhaps this is one case where you would want to use a double instead of a string. Or use a double first to do an initial rounding and then a string to finish it off?

Java EE Evangelist — Author, EJB 3 in Action 2nd Edition — Java Community Process Member

It is sorta covered in the JavaRanch Style Guide. |