isInteger(String) : Scanner vs Regex vs Throw

[Carey Brown]

There have been several threads dealing with ways to tell if a String will be able to be parsed by Integer.parseInt() without throwing an exception. Most of the suggestions involve the use of a Scanner object. Perhaps I'm just not comfortable with Scanner methods (or their documentation is lacking) but I found that I had to do some trial and error with Scanner to get it to function properly relative to parseInt(). The following code is my test harness for both correctness and performance. Based on this example I don't see the reason for advocating the use of Scanner. I'm open to suggestions.

The output is...

Assertion testing...
Performance testing...
Scanner:  113,561,398,928
Regex:      1,332,100,650
Throw:     10,460,757,740

import java.util.Scanner;
import java.util.regex.Pattern;

public class IsInteger
{
	private static final Pattern		INTEGER_PATTERN = Pattern.compile( "[-+]?\\d+" );
	private static final String[]		TRUE_TEST_CASES =
	{
		"123",
		"-123",
		"+123"
	};
	private static final String[]		FALSE_TEST_CASES =
	{
		"",
		"abc",
		" 123",
		"\t123",
		"123 ",
		"123\r",
		"123\n",
		"123\r\n",
		"123\n\r",
		"123\n123",
		"123.456",
		"123e10"
	};

	// Note: Specifically not trim()ing space as this would not
	// be acceptable to Integer.parseInt(). Scanner DOES permit
	// spaces so we have to explicitly check for the presence
	// of spaces ourselves.
	// Observation: Scanner takes 100 times longer than regex
	// and Scanner has too many idiosyncrasies for me to trust it.
	public static boolean isIntegerScan( String s )
	{
		int			n;
		if( (n = s.length()) == 0
			|| s.charAt(0)   == ' '  || s.charAt(0)   == '\t'
			|| s.charAt(0)   == '\r' || s.charAt(0)   == '\n'
			|| s.charAt(n-1) == ' '  || s.charAt(n-1) == '\t'
			|| s.charAt(n-1) == '\r' || s.charAt(n-1) == '\n'
			)
			return false;
		@SuppressWarnings("resource")
		Scanner sc = new Scanner( s );
		if( !sc.hasNextInt() )
			return false;
		sc.nextInt();
		return !sc.hasNext();
	}
	
	public static boolean isIntegerRegex( String s )
	{
		return INTEGER_PATTERN.matcher( s ).matches();
	}
	
	// Observation: Throw is 10 times slower than regex and 10
	// times faster than scanner. This is also considered bad use of
	// the exception mechanism due to using it for non-exceptional
	// conditions.
	public static boolean isIntegerThrow( String s )
	{
		try
		{
			Integer.parseInt( s );
		}
		catch( NumberFormatException ex )
		{
			return false;
		}
		return true;
	}
	
	public static void main( String... args )
	{
//		byte[]			buf = new byte[64];
//		int				n;
//		String			s;
//		
//		for(;;)
//		{
//			System.out.println( "Enter an integer to validate: " );
//			try
//			{
//				n = System.in.read( buf );
////				if( n > 0 && buf[n-1] == '\n' )
////					n--;
////				if( n > 0 && buf[n-1] == '\r' )
////					n--;
//				if( n <= 0 )
//					break;
//				s = new String( buf, 0, n );
//				System.out.println( "Valid scan:  " + isIntegerScan(s) );
//				System.out.println( "Valid regex: " + isIntegerRegex(s) );
//				System.out.println( "Valid throw: " + isIntegerThrow(s) );
//			}
//			catch( IOException e )
//			{
//				e.printStackTrace();
//			}
//		}
		
		System.out.println( "Assertion testing..." );

		for( String ss : TRUE_TEST_CASES )
		{
			myAssert( ss, true );
		}

		for( String ss : FALSE_TEST_CASES )
		{
			myAssert( ss, false );
		}
		
		System.out.println( "Performance testing..." );
		long			start, stop;
		
		start = System.nanoTime();
		for( int i = 0 ; i < 1_000_000 ; i++ )
		{
			for( String ss : TRUE_TEST_CASES )
			{
				isIntegerScan( ss );
			}
			for( String ss : FALSE_TEST_CASES )
			{
				isIntegerScan( ss );
			}
		}
		stop = System.nanoTime();
		System.out.format( "Scanner: %,16d%n", (stop - start) ); System.out.println();
		
		start = System.nanoTime();
		for( int i = 0 ; i < 1_000_000 ; i++ )
		{
			for( String ss : TRUE_TEST_CASES )
			{
				isIntegerRegex( ss );
			}
			for( String ss : FALSE_TEST_CASES )
			{
				isIntegerRegex( ss );
			}
		}
		stop = System.nanoTime();
		System.out.format( "Regex:   %,16d%n", (stop - start) ); System.out.println();
		
		start = System.nanoTime();
		for( int i = 0 ; i < 1_000_000 ; i++ )
		{
			for( String ss : TRUE_TEST_CASES )
			{
				isIntegerThrow( ss );
			}
			for( String ss : FALSE_TEST_CASES )
			{
				isIntegerThrow( ss );
			}
		}
		stop = System.nanoTime();
		System.out.format( "Throw:   %,16d%n", (stop - start) ); System.out.println();
	}

	private static void myAssert( String ss, boolean expected )
	{
		boolean			b;
		
		b = isIntegerScan( ss );
		if( b != expected )
			System.out.println( "Assertion failed - Scan: \"" + ss + "\"" );
		
		b = isIntegerRegex( ss );
		if( b != expected )
			System.out.println( "Assertion failed - Regex: \"" + ss + "\"" );
		
		b = isIntegerThrow( ss );
		if( b != expected )
			System.out.println( "Assertion failed - Throw: \"" + ss + "\"" );
	}
}

[Rodion Gork]

The following code is my test harness for both correctness and performance. Based on this example I don't see the reason for advocating the use of Scanner. I'm open to suggestions.

I suspect any of three approaches will work correctly if written properly.

As about performance - I bet you will quite rarely hit the case when you need to perform millions of such checks at once - i.e. when performance will be dramatically affected by the operation.

Moreover, to get superb performance you'd better use character comparison in the loop. I suspect this will outperform regexps significantly...

So I believe it is mainly matter of preference. I also suspect at least Scanner works via regexp inside.

However, consider that you maintain the code and you need to change your isInteger to isReal. With either approach except the regexp this will not take a lot of time from you. Probably few seconds to change "nextInt" to "nextDouble".

But if you try to extend your pattern, you may find it is more tricky business - and your team-lead will not allow you to merge your code without unit-test to check that your pattern is correct... And then someone should surely code-review both pattern and its unit-test... I believe this can take significant time from two persons...

Some nasty people will even ask you to write tests against this simple pattern for integer, I believe...

Though I confess that if I write code for myself (e.g. some side-project) I often prefer using regexps - so in this sense I'm on your side...

[Carey Brown]

Rodion Gork wrote:However, consider that you maintain the code and you need to change your isInteger to isReal. With either approach except the regexp this will not take a lot of time from you. Probably few seconds to change "nextInt" to "nextDouble".

Good point. That's probably the best pro Scanner argument I can see.

But if you try to extend your pattern, you may find it is more tricky business - and your team-lead will not allow you to merge your code without unit-test to check that your pattern is correct... And then someone should surely code-review both pattern and its unit-test... I believe this can take significant time from two persons...

Some nasty people will even ask you to write tests against this simple pattern for integer, I believe...)

As you can see I had to do some customization to the Scanner code to parallel the logic in parseInt(). Working this out required a unit test and was not obvious.

Thanks for your feedback.

[Campbell Ritchie]

You will find a mention of how Scanner determines whether the String is an int here, from two weeks ago. It seems to use some regex then a try. I suspect the regex is one of those shown here, and the Integer.parseInt call is for anything which passes the regex test, to ensure it is not >2147483647 or <-2147483648.

To avoid the problem with "123 Campbell" here, you would want to separate the String input into tokens.

String input = myScanner.next(); // Works if default delimiter used
Scanner numberScan = new Scanner(input);
System.out.println(
   numberScan.hasNextInt() ? String.valueOf(numberScan.nextInt()) : "No number"
);

You can use other delimiters as long as they don't divide numbers apart. If you enter a number like 123,456 then a comma as a delimiter is hardly going to give an accurate result.

[edit]Change the word here to a link[/edit]

[Carey Brown]

I took Rodion's comment about converting my code to isDouble() as a challenge. I found it needed just a tweak from nextInt() to nextDouble() because parseInt() does not permit leading/trailing white space while parseDouble() does (a curious difference). In creating the regex for type double I had to permit leading zeros whereas the regex's found on the internet did not. At this point I'm not even considering the overflow/underflow issue and haven't looked any closer to the Scanner code to see what it does with it.

import java.io.IOException;
import java.util.ArrayList;
import java.util.List;
import java.util.Scanner;
import java.util.regex.Pattern;

public class IsDouble
{
	private static final String			E = "([eE][+-]?\\d+)?";
	private static final Pattern		DOUBLE_PATTERN = Pattern.compile(
		"\\s*[-+]?("
			+ "(\\d*\\.\\d+" + E + ")|"
			+ "(\\d+\\.\\d*" + E + ")|"
			+ "(\\d+" + E + ")"
		+ ")\\s*" );
	
	private static final String[]		SPACE =
	{
		"",
		" ",
		"\t",
		"\r",
		"\n",
	};
	
	private static final String[]		SIGN =
	{
		"",
		"+",
		"-",
	};
	
	private static final String[]		INT =
	{
		"",			// special case handled in getPermutations()
		"0",
		"00",
		"123",
		"0123",
	};
	
	private static final String[]		DOT =
	{
		"",
		".",
	};
	
	private static final String[]		EXP =
	{
		"",
		"e10",
		"e0",
		"e01",
		"E10",
		"E0",
		"E01",
		"e+10",
		"e+0",
		"e+01",
		"E+10",
		"E+0",
		"E+01",
		"e-10",
		"e-0",
		"e-01",
		"E-10",
		"E-0",
		"E-01",
	};
	
	// TODO need more
	private static final String[]		FALSE_TEST_CASES =
	{
		"",
		".",
		"abc",
		"123 abc",
		"123\n123",
		"123.-1",
	};

	public static boolean isDoubleScan( String s )
	{
		// NOTE: Ripped out space handling that was present
		// in isIntegerScan()
		@SuppressWarnings("resource")
		Scanner sc = new Scanner( s );
		if( !sc.hasNextDouble() )
			return false;
		sc.nextDouble();
		return !sc.hasNext();
	}
	
	public static boolean isDoubleRegex( String s )
	{
		return DOUBLE_PATTERN.matcher( s ).matches();
	}
	
	// This is considered bad use of
	// the exception mechanism due to using it for non-exceptional
	// conditions.
	public static boolean isDoubleThrow( String s )
	{
		try
		{
			Double.parseDouble( s );
		}
		catch( NumberFormatException ex )
		{
			return false;
		}
		return true;
	}
	
	public static void main( String... args )
	{
//		interactive();
		
		System.out.println( "Assertion testing..." );
		
		List<String>	trueTestCases = getPermutations();
		
		System.out.println( "Number of true test cases: " + trueTestCases.size() );

		for( String ss : trueTestCases )
		{
			myAssert( ss, true );
		}

		for( String ss : FALSE_TEST_CASES )
		{
			myAssert( ss, false );
		}
				
		System.out.println( "Done." );
	}

	private static void myAssert( String ss, boolean expected )
	{
		boolean			b;
		
		b = isDoubleScan( ss );
		if( b != expected )
			System.out.println( "Assertion failed - Scan: \"" + ss + "\"  " + b );
		
		b = isDoubleRegex( ss );
		if( b != expected )
			System.out.println( "Assertion failed - Regex: \"" + ss + "\"  " + b );
		
		b = isDoubleThrow( ss );
		if( b != expected )
			System.out.println( "Assertion failed - Throw: \"" + ss + "\"  " + b );
	}
	
	private static void interactive()
	{
		byte[]			buf = new byte[64];
		int				n;
		String			s;

		for(;;)
		{
			System.out.println( "Enter an double to validate: " );
			try
			{
				n = System.in.read( buf );
				if( n <= 0 )
					break;
				s = new String( buf, 0, n );
				System.out.println( "Valid scan:  " + isDoubleScan(s) );
				System.out.println( "Valid regex: " + isDoubleRegex(s) );
				System.out.println( "Valid throw: " + isDoubleThrow(s) );
			}
			catch( IOException e )
			{
				e.printStackTrace();
			}
		}
	}
	
	// permutations of good examples of a double
	private static List<String> getPermutations()
	{
		ArrayList<String>	list = new ArrayList<>();
		
		for( int space = 0 ; space < SPACE.length ; space++ )
		{
			for( int sign = 0 ; sign < SIGN.length ; sign++ )
			{
				for( int integer1 = 0 ; integer1 < INT.length ; integer1++ )
				{
					for( int dot = 0 ; dot < DOT.length ; dot++ )
					{
						for( int integer2 = 0 ; integer2 < INT.length ; integer2++ )
						{
							for( int exp = 0 ; exp < EXP.length ; exp++ )
							{
								if( integer1 != 0 || integer2 != 0 ) // special case
								{
									StringBuilder		sb = new StringBuilder();
									
									sb.append( SPACE[space] );
									sb.append( SIGN[sign] );
									sb.append( INT[integer1] );
									sb.append( DOT[dot] );
									sb.append( INT[integer2] );
									sb.append( EXP[exp] );
									
									list.add( sb.toString() );
								}
							}
						}
					}
				}
			}
		}
		
		return list;
	}
}

[Carey Brown]

Campbell Ritchie wrote:You will find a mention of how Scanner determines whether the String is an int here, from two weeks ago. It seems to use some regex then a try.

I looked up the code and was surprised to see the try/catch around parseInt(), but it makes sense because somewhere the range checking needs to take place. My code does not address range checking except as provided by the Scanner and directly by the isIntegerThrow() approach. isIntegerRegex() does not address this at all.

If we already have a string that we think is an integer then calling hasNextInt() is somewhat redundant and we might as well use the isIntegerThrow() method in my code which only uses try/catch like hasNextInt() uses. Scanner does not have the luxury of throwing away non-matches because subsequent calls to Scanner methods may follow, whereas we are not constrained by that.

This whole topic turned out to be deeper than I thought.

[Winston Gutkowski]

Carey Brown wrote:In creating the regex for type double I had to permit leading zeros whereas the regex's found on the internet did not.

I'm not quite sure I agree with that statement. Your program may have to handle leading 0's, but there's no particular reason your regex has to, especially if you remove them before you pass the String to it.

It's actually a common failing of many regexes - they try to do too much in one pass, and in the process become overly complex.

My 2¢.

Winston

[Carey Brown]

Winston Gutkowski wrote:

Carey Brown wrote:In creating the regex for type double I had to permit leading zeros whereas the regex's found on the internet did not.

I'm not quite sure I agree with that statement. Your program may have to handle leading 0's, but there's no particular reason your regex has to, especially if you remove them before you pass the String to it.

It's actually a common failing of many regexes - they try to do too much in one pass, and in the process become overly complex.

My 2¢.

Winston

My goal is a regex that permits strings that would process properly by Double.parseDouble(), not counting range checking. parseDouble() not only permits leading zeros but it permits leading and trailing white space, which parseInt() does not.

A typical regex found on the internet starts out with: "[-+]?(0|([1-9][0-9]*))
To allow leading zeros all I need is: "[-+]?\\d+

My final regex looks like:

	private static final String			E = "([eE][+-]?\\d+)?";
	private static final Pattern		DOUBLE_PATTERN = Pattern.compile(
		"\\s*[-+]?("
			+ "(\\d*\\.\\d+" + E + ")|"
			+ "(\\d+\\.\\d*" + E + ")|"
			+ "(\\d+" + E + ")"
		+ ")\\s*" );

Works like a charm and I don't feel like it's overly complex when broken down this way.

[Campbell Ritchie]

There would be other ways to do it without using Exceptions:-

boolean integerFormat = inputText.matches("[1-9]\\d*");
BigInteger bi;
final BigInteger INT_MAX = new BigInteger(String.valueOf(Integer.MAX_VALUE)); // Might be field
final BigInteger INT_MIN = new BigInteger(String.valueOf(Integer.MIN_VALUE)); // Might be field
return inputText.equals("0") ||
       integerFormat && ((bi = new BigInteger(inputText)).compareTo(INT_MAX) <= 0
                     && bi.compareTo(INT_MIN) >= 0;

I am sorry for my mistake yesterday; I said there was a regex in the Scanner class. That should have read grammar.

[Winston Gutkowski]

Carey Brown wrote:My goal is a regex that permits strings that would process properly by Double.parseDouble(), not counting range checking. parseDouble() not only permits leading zeros but it permits leading and trailing white space, which parseInt() does not.

A typical regex found on the internet starts out with: "[-+]?(0|([1-9][0-9]*))
To allow leading zeros all I need is: "[-+]?\\d+
...

I understand, and I certainly agree that your solution is the simplest, but my point was that regexes are not panacaeas.

They do ONE thing - text-based pattern matching - very well, particularly when it doesn't involve any "smart stuff".
As soon as brackets (()), or forward or backward "looking" start to get involved, you're into the realms of a procedure, rather than a pattern, and my personal take is that I'd much prefer to see three simple regexes implemented procedurally, with notes, rather than a single one that takes someone an hour to decipher, but works in one pass.

But it is my personal opinion - albeit with a few grey hairs of experience. YMMV. :-)

Winston

[Junilu Lacar]

Winston Gutkowski wrote:
But it is my personal opinion - albeit with a few grey hairs of experience. YMMV. :-)

I agree with your opinion, and I don't have any hair ;)

[A.J. Côté]

Scanner is slow. It is just a level of abstraction above parse and throw.

To insure Java language consistency, We need to validate that an int is an int, a double is a double, etc. in a central point; that is the parse() and throw concept. All higher level abstraction layers use it under the cover.

In the end, it is always faster to use your own BufferedReader than a scanner. Let me know if you need more background. I will provide links to other threads on JavaRanch.

Then again, don't optimize too soon ;-))

[Carey Brown]

A.J. Côté wrote:Scanner is slow. It is just a level of abstraction above parse and throw.

To insure Java language consistency, We need to validate that an int is an int, a double is a double, etc. in a central point; that is the parse() and throw concept. All higher level abstraction layers use it under the cover.

In the end, it is always faster to use your own BufferedReader than a scanner. Let me know if you need more background. I will provide links to other threads on JavaRanch.

Then again, don't optimize too soon ;-))

I'm not optimizing too soon ;o) I'm about ten years too late for that. I was on board with Java before Scanner came into existence and I was just trying to get a sense of why people seem to promote it so heavily on this site. I'm not working on any projects right now so I thought I'd play with it. And I've come to the conclusion that Scanner is way SLOW, as you've said, about 100-200 times slower. This would be ok for user input but not for processing data streams.

In case you're interested, here's the final isDouble() method I came up with based on the performance over a very large data set of mixed good data and bad data. By the way, I did have to ditch regular expressions, while they worked, they were not as fast as I'd hoped. So this is a hybrid approach that was faster than just using the throw approach on its own.

Thanks for joining me on this ride.

public class IsDoubleFast
{
	private static final String			DOUBLE_CHARS = "0123456789+-eE";
	
	public static boolean isDouble( String s )
	{
		s = s.trim(); // because Double.parseDouble accepts spaces
		for( int i = 0 ; i < s.length() ; i++ )
			if( DOUBLE_CHARS.indexOf( s.charAt(i) ) == -1 )
				return false;
		
		try
		{
			Double.parseDouble( s );
		}
		catch( NumberFormatException ex )
		{
			return false;
		}
		return true;
	}
}

[Campbell Ritchie]

Scanner is useful because it is a level of abstraction higher. You might save a few μs but you might lose hours on readability and maintenance time.
Particularly when you are using keyboard inputs. When it takes a couple of seconds to write a number, who is going to care about μseconds?

[Winston Gutkowski]

Carey Brown wrote:I was on board with Java before Scanner came into existence and I was just trying to get a sense of why people seem to promote it so heavily on this site....

Not everyone. As Campbell will tell you, I'm not a big fan of Scanner - especially as an input stream handler - although I don't mind it so much as a String processor.

Another possibility for you:

public static final Double getDouble(String d) {
   try {
      return Double.valueOf(d);
   catch (NumberFormatException) [
      return null;
   }
}

public static final boolean isDouble(String d) {
   return getDouble(d) != null;
}

Some people might not like it, but to me it's an acceptable exception to the general rule that you don't use exceptions to control logic; since you'd otherwise have to duplicate logic - be it a regex or a procedure - that someone else already wrote in order to prevent it.

It's also probably worth mentioning that the docs for Double.valueOf() already contain an example regex for checking doubles.

HIH

Winston

[Paul Clapham]

Or if you aren't a fan of naked nulls, you could use this:

	public static final Optional<Double> getDouble(String d) {
		try {
			Double result = Double.valueOf(d);
			return Optional.of(result);
		} catch(NumberFormatException nfe) {
			return Optional.empty();
		}
	}
		 
	public static final boolean isDouble(String d) {
	   return getDouble(d).isPresent();
	}

[Winston Gutkowski]

Paul Clapham wrote:Or if you aren't a fan of naked nulls, you could use this:

Neat. I really must gen up on v8. :-)

Winston

[Campbell Ritchie]

I have a copy of Urma Fusco and Mycroft's Java8 book (Java8 in Practice: Manning) and they have a whole chapter starting about page 225 about the Optional<T> class. They suggest there are all sorts of uses for it to replace nulls. They appear to disagree with Optional's creator (Brian Goetz) who intended the class to be used as a return type from a Stream where no value is found.

If you can get a copy it is worth a read

[Winston Gutkowski]

Campbell Ritchie wrote:I have a copy of Urma Fusco and Mycroft's Java8 book...If you can get a copy it is worth a read

I'll keep an eye out for it. Cheers.

Funny that something as simple as that could be so useful. I've actually written something similar myself, but never thought of it as a placeholder for an "optional" value.

Winston

[Rodion Gork]

M

y final regex looks like:
...
Works like a charm and I don't feel like it's overly complex when broken down this way.

Yes, this looks correct, thank you

Now let us think. I live in such a strange country where people use comma instead of decimal dot (as a programmer I hate even thinking of this). Scanner respects this bewildering tradition if proper locale is chosen. Suppose, your code is added to some internationally-targeted web-application and your QA tells you about this mournful fact - will you insist on trying to add localization to regexp, or prefer not to waste your time (paid by customer) and switch to Scanner?

[Winston Gutkowski]

Rodion Gork wrote:Now let us think. I live in such a strange country where people use comma instead of decimal dot (as a programmer I hate even thinking of this).

I did too, for 11 years (Belgium).

Scanner respects this bewildering tradition if proper locale is chosen. Suppose, your code is added to some internationally-targeted web-application and your QA tells you about this mournful fact - will you insist on trying to add localization to regexp, or ... switch to Scanner?

Very good point. I'm not absolutely sure, but I think that Double.valueOf() takes Locale into account as well, but I wouldn't swear to it.

Winston

[Campbell Ritchie]

It is a bit complicated reading about Double#valueOf(String), but it says lexical structure as in the JLS. I think that means that it requires a . as the radix point and will not accept commas. It says to use NumberFormat for locales.