We have a legacy Math utility class containing the following methods:
I realise it should be using BigDecimal everywhere but for the moment I can't change the code. What we are currently seeing is that in production on Websphere, the call
is returning 100.36999999, rather than 100.37. This does not happen in windows or in Tomcat. I have created a simple script and just called that exact line above (outside the application) and it works fine!
Following through the code, the result of the primitive subtraction is 100.36999999999989, which is seen in windows as well. When this passed to the roundDouble method, the factor is 10^8 = 100000000. This is multiplied by the input so its 100.36999999999989 * 100000000 = 10036999999. This is then rounded to the nearest int which is 10037000000. Finally we divide by the factor so its 10037000000 / 100000000 = 100.37.
I am getting 100.36999999 as the final result...so either the Math.round is truncating 10036999999, rather than rounding it or final divide 10037000000 / 100000000 is going wrong and returning 100.36999999.
Can anyone throw any ideas out why there would be a difference, especially between a running application calling a method and a simple script just calling the same method in the same jar on the same environment but outside the application, or any ideas on what could be going wrong?
Not sure what's going on, but you may want to look into using java.lang.StrictMath instead of java.lang.Math. Its API is identical, but it produces identical results across platforms (which Math is not required to do).
I've never checked what StrictMath does exactly. From what you say it sounds as if both should do the same thing in this case. Which means that any platform differences must originate elsewhere in the calculation. I'd do a step-by-step comparison of the intermediate values on both platforms to determine where that happens.
Yep I tried that....the code that seems to be going wrong is in a utility jar that we use, which I cannot change. So to do what you suggest, I copied the methods from the utility jar into my class and called them instead....should be the same thing. Unfortunately not! I couldn't get the error to happen calling the copied code, even though its identical! I can only think the utility jar is compiled "differently" than my copied version....or does this even even make sense?
If I remember correctly, the simple Math class methods (eg max, min, abs) use their own code,something like thisand the "complicated" methods simply call StrictMath, like thisAt least I think most of them do that.
pie. tiny ad:
Building a Better World in your Backyard by Paul Wheaton and Shawn Klassen-Koop