String type for income

[Prasanna Raman]

Hello,

I work at a financial firm and almost all of the income related fields in our application are declared as "String" instead of double or BigDecimal. Is this an acceptable thing to do? I'd have thought that we shouldn't be using strings for income.

Could someone please help me understand?

[Paul Clapham]

Your instinct is correct: many people advise against using Strings for money, and I'm sure you could go on to provide some reasons why that's the case. However, to understand why the system was designed in that way you would have to find the people who produced the design and ask them.

It's possible that there was a good reason for that decision, or maybe a reason which was a good reason back when the decision was made. You never know; for example it might have had something to do with interfacing with some ancient system which is no longer around. Or on the other hand it may be that the designers didn't know any better. But from here it's impossible to tell.

I can see how you might not want to ask within your organization, though. It's possible that the designers in question are now in senior positions and you might be seen as questioning their judgement. Which might cause trouble for you in some environments. On the other hand it might be that your senior staff inherited this application from people who are no longer around, in which case they might well agree with you about your instinct. Again it's impossible to tell.

[Zachary Griggs]

I guess it technically works, but it sounds like a bad use of String

[Prasanna Raman]

Thank you, Paul! Your reasons for me not wanting to ask them are spot on! I in fact tried to bring this up with someone and they said it was OK to do this. But I am in a position now where I am adding another kind of income field to the application but I don't feel good about making it a String also as they want me to. The only reason they gave me was because there's java script on the front end to add commas to the digits, a double datatype wouldn't be able to accept the value, whereas with a String, they're doing something like this before validating.

getIncome().replace(",", "") 

I am thinking about how I should do this now.

[Paul Clapham]

I would make the new income field look just like all of the other income fields. That's first of all because (I assume) there is already working code which handles the string data reliably. And second, if you make it different than all the others then future maintainers are always going to be asking "Why does this field work differently? What's different about it?" And then there's the possibility that you might have to write new validation and formatting code for your new income field.

[Prasanna Raman]

Thank you, Paul. Yes, there is reliable validation that's happening for Strings currently, although I am still thinking if I should do the right thing and make it a double. In your earlier post, you said many people advise not to use String for income fields, so is this something that does happen in some places? and is this ok?

Another question: if I do make it double and the field on the front end had commas, is the only way then is to remove it through javascript/jquery on the front end before it can be passed to the server?

[Tapas Chand]

Prasanna Raman wrote:..although I am still thinking if I should do the right thing and make it a double...

On a side note:IF you are not using String in your case, then BigDecimal would be a better choice than double when comes to money.

[Stephan van Hulst]

Using double or other floating point types is never correct for currency. For money, I would use a BigInteger to store money in cents. Barring the use of BigInteger or BigDecimal, String really is the best option.

Let's say the reason for using Strings is that they need an easy way to handle money at the front-end. There's nothing stopping a back-end developer to use BigInteger/BigDecimal wherever they can, and before sending data back to the client, do a conversion to String, and when you receive money as a String, convert it to BigInteger/BigDecimal.

I agree with Paul though that consistency is most important, and you may want to stick with what you have until you manage to convince your project lead that it's okay to replace strings with better types as you go.

[Prasanna Raman]

Stephan - Thank you. I was reading about BigDecimal and my understanding was that you use it when a high level of precision is required. I thought in my case a double would suffice. Is my understanding wrong?

Also, how would I convert from String to BigDecimal on the back end? In our application, they're setting it on the "String" object using Spring web flows. Wouldn't it fail if the data type was double or BigDecimala at the back end?

[Prasanna Raman]

Stephan van Hulst wrote:you may want to stick with what you have until you manage to convince your project lead that it's okay to replace strings with better types as you go.

By this you mean, until I manage to convince them to change it in all places consistently?

[Stephan van Hulst]

You can use BigDecimal to achieve arbitrary precision, regardless of whether you need a lot or a little precision. More importantly, BigDecimal is accurate for the chosen precision.

The floating point primitives have a fixed precision, and they are not accurate. That's okay if your calculations involve approximations of reality, but this is never the case with money. You don't want to lose money or for it to appear out of nowhere. Calculations need to be accurate.

You don't actually need precision beyond whole cents, unless you're performing division operations. If you don't perform division operations, BigInteger is fine.

As soon as you receive a string from the front end, you can convert it to a BigDecimal or BigInteger using the methods and constructors of those classes. I don't know how your application is set up, I imagine that there are frameworks out there that can implicitly convert strings to the intended data type upon deserializing the request. For instance, if you're using JAX-B, you can write an adapter that will perform this conversion for you whenever you try to deserialize XML/JSON.

[Stephan van Hulst]

Prasanna Raman wrote:

Stephan van Hulst wrote:you may want to stick with what you have until you manage to convince your project lead that it's okay to replace strings with better types as you go.

By this you mean, until I manage to convince them to change it in all places consistently?

No, that will never happen if your application is already big enough. Refactoring an application is easiest and will be least expensive when you change it as you go. Whenever you discover a String that's used to store money, refactor it so it uses a different type instead. Few project leads will ever approve committing a programmer to refactoring it all at once.

[Prasanna Raman]

Thank you. I am not sure if there's a way for Spring web flow to handle this conversion automatically. I think changing the setter method to receive a String and then converting it should work, but I am not sure they'd let me do it as that's going to be a big change.

But, is there a way that this conversion can be done on the front end before trying to set it on the object?

[Stephan van Hulst]

Probably not, because the front-end doesn't know what a BigDecimal is.

[Campbell Ritchie]

Stephan van Hulst wrote:. . . I would use a BigInteger to store money in cents. . . .

Or a long, which can store over $9×10¹⁷ if you use cents. There are new currency classes coming out with Java9, but I haven't managed to see how they work yet. So I don't know whether they will help you.

[Stephan van Hulst]

Currency represents... a currency!

You can model money using a combination of a Currency and a BigInteger (or a long!), but for most applications the currency is implicit. It's like in order to get an actual instance of time you need a time zone, but for most applications the time zone is implicit.

[Stephan van Hulst]

Ah, sorry, you said Java 9. Java already has a class called Currency, but then I suppose it's different from what you mentioned.

[Prasanna Raman]

Stephan van Hulst wrote:Probably not, because the front-end doesn't know what a BigDecimal is.

I am slightly confused here. Should the front end have to know what it is? If I pass in a valid number on form submit from the front end, wouldn't the value be set correctly on the BigDecimal income field that I have on my object?

[Prasanna Raman]

To give more context, I just found out that the income field that I will be dealing with is going to be a 11-digit whole number. And I don't think I'll have to deal with any arithmetic operations on it. Does this change things as far as data type selection goes?

[Campbell Ritchie]

It means you cannot use an int or an Integer because they don't accommodate 11 digits. You can use longs, Longs, BigInteger or BigDecimal, and I think the classes all have valueOf methods or constructors which accept such a String.

Once you have got those figures into your code, put them into a database as BIGINT or DECIMAL(12,2) or similar. I have probably got the numbers wrong for DECIMAL.

[Prasanna Raman]

Thank you, Campbell. I am confused about 1 thing though. Why does there need to be a conversion at the back end? If a number gets passed from the front end, can I not directly set it on my BigDecimal or BigInteger field without having to convert it?

If not, is everything always treated as a String then?

[Paul Clapham]

Of course, everything sent from the front end is a string. HTTP requests are just text -- at least, they are bytes, but you can't expect to get (say) 8 bytes which represent a Java long variable directly. You're going to get some kind of text and you're going to have to convert some of that text into a numeric type.

[Campbell Ritchie]

Not sure I quite understand the question; you won't want to keep your numbers in String format however. By getting them out of String format you can be sure they are stored as binary numbers (BIGINT) or BCD (=Binary Coded Decimal) numbers. That is the best way to guarantee that you will store numbers at all.

[Prasanna Raman]

Paul Clapham wrote:Of course, everything sent from the front end is a string. HTTP requests are just text -- at least, they are bytes, but you can't expect to get (say) 8 bytes which represent a Java long variable directly. You're going to get some kind of text and you're going to have to convert some of that text into a numeric type.

Paul, I am confused now :( If everything is a String, do you know how the following works?

@NumberFormat(style = Style.NUMBER)
private Integer employLength;


I see this Integer field declared in our application, but there is no explicit conversion done anywhere as far as I can tell.

[Paul Clapham]

I don't understand why you are asking me how that tiny code fragment works. All you have there is a declaration of a variable of Integer type, and an annotation which appears to describe how to format the value of that variable. No doubt there is code somewhere which assigns a value to that variable, and perhaps there is code which converts some String into a number before assigning it to that variable. But it's impossible to tell from that single line of code where that other code is located.

You should be asking somebody who knows something about your code base that question. And seriously, before you decide to start refactoring the system you should have a better knowledge of that system.

[Prasanna Raman]

Thank you, Paul. The reason I asked is because I don't see any code in the system that's doing the conversion from String to Integer, and I didn't know what this line was doing:

@NumberFormat(style = Style.NUMBER)

So, unless that line is taking care of the conversion, I don't understand how this works. That's why I asked.

Also, I am not trying to refactor the whole system. I am just trying to add a new income field, and I didn't want to use Strings like it's been used everywhere else in the application. Sorry for the confusion!

[Bear Bibeault]

Are you using the Spring framework?

[Prasanna Raman]

Yes.

[Bear Bibeault]

http://docs.spring.io/spring/docs/current/javadoc-api/org/springframework/format/annotation/NumberFormat.html

[Prasanna Raman]

Thank you, Bear. I read over the docs and it seems like using that annotation will take a number and convert it into currency or percent format.

I'm still trying to understand how I can use it for my needs. I need my program to receive a comma separated string from the UI, strip the numbers and store it in a long or BigInteger variable. Could someone kindly help?

[Knute Snortum]

If you have a string of number separated by commas, you can use split() to turn them into a String[].

String fromUI = "532.45, 55.43, 76.52, 124.23";
String[] numbers = fromUI.split("\\s*,\\s*");

The "\\s*" means "any amount of whitespace." You can loop through this array without having to assign it to a variable:

for (String number : fromUI.split("\\s*,\\s*")) {
    // do something with number
}

The problem would be if the UI might return something like this:

String fromUI = "532.45, \"4,555.43\", 76.52, 124.23";

Since a valid money amount might contain a comma, the value with a comma is quoted. If this is the case, I would use a third party CSV library.

[Prasanna Raman]

Thank you, Knute. I'm only dealing with one whole number here- maximum of 11 digits. To give this more context, we are an application that is a pass through - we receive a String from the UI and then pass it onto another application for storing in the database. Before passing the String, I could convert it into a double or long based on whatever they require.

So, is this OK to do? Can I receive the variable in a String in my application? I was trying to find ways for Spring to handle this conversion before even setting it to our variable (so that I can use a long type instead of String for our variable).

I might be confusing things here. Any comments are welcome!

[Paul Clapham]

In your original post you said

almost all of the income related fields in our application are declared as "String"

So I would expect that you already have Spring configuration, or code, which deals with setting all of those existing fields. You should find out how that works for a start. (Asking people on this forum isn't going to help you do that; you should ask people in your organization if you don't already know.) Then once you understand that, you can work on writing configuration or code to set your new field.

[Prasanna Raman]

Hello Paul,

Yes, all the fields are declared as String, and are converted to int or double using Integer.parseInt() before passing the fields on to the other application.

[Knute Snortum]

...we are an application that is a pass through - we receive a String from the UI and then pass it onto another application for storing in the database.

Are you asking how to pass the number to the DB application? This really depends on the API of the DB app.

[Paul Clapham]

Prasanna Raman wrote:Hello Paul,

Yes, all the fields are declared as String, and are converted to int or double using Integer.parseInt() before passing the fields on to the other application.

Then it should take you only a few minutes to copy the code and/or configuration of one of the other String fields and modify that to suit the new one. I don't see why this is a big problem.

[Winston Gutkowski]

Prasanna Raman wrote:Yes, all the fields are declared as String, and are converted to int or double using Integer.parseInt() before passing the fields on to the other application.

OK, well as has already been explained, that's not a good idea for currency - especially as it's already been established that you could overflow an int.

However, this is my take on it:
1. You have a String that contains a numeric amount with possible currency symbols involved.
2. Since you have no idea what they are, or whether they're leading or trailing, it would be nice to have a generic way of getting rid of them.

Unfortunately, you still have to make some assumptions. Mine would be:
1. Numbers are written using Arabic numerals, plus the characters '+', '-', ',' and '.'.
2. Those last four characters are never used for anything else (eg, a currency symbol).
3. Whitepsace is irrelevant.

And if those are reasonable assumptions, the operation to extract the "number" from a String boils down to a couple of regular expression operations, viz:

public static final String extractNumber(String input) {
   String condensed = input.replaceAll("\\s", "");
   return condensed.replaceFirst("^(?:.*?)([0-9.,+-]*)(?:.*)$", "\1");
}

and once you have your "number string", simply pass it to the relevant constructor for whatever type of number you want.

Note: That second bit (line 3) might need a bit of tweaking; and I'd probably add a third one to remove any leading or trailing '.'s or ','s - but hopefully you get the idea.

HIH

Winston

[Dave Tolls]

Winston Gutkowski wrote:
1. You have a String that contains a numeric amount with possible currency symbols involved.

I would question a UI that sent currency symbols to the backend as part of an amount string.

[Bear Bibeault]

Dave Tolls wrote:

Winston Gutkowski wrote:
1. You have a String that contains a numeric amount with possible currency symbols involved.

I would question a UI that sent currency symbols to the backend as part of an amount string.

Could be valid if it's because it allows the user to enter the amount in any currency and the backend will make conversions (though I'd likely send that info separately). But, generally, no.

[Winston Gutkowski]

Dave Tolls wrote:I would question a UI that sent currency symbols to the backend as part of an amount string.

Me too. I'm simply allowing for the fact that they might be included.

You also have the (ugh) possibility of "accounting-style" values - eg, "(500.00)" meaning -500, but that's just a refinement to the method.

And of course, the best scenario is if you don't have to do it at all.

Winston