• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Jeanne Boyarsky
  • Ron McLeod
  • Paul Clapham
  • Liutauras Vilda
Sheriffs:
  • paul wheaton
  • Rob Spoor
  • Devaka Cooray
Saloon Keepers:
  • Stephan van Hulst
  • Tim Holloway
  • Carey Brown
  • Frits Walraven
  • Tim Moores
Bartenders:
  • Mikalai Zaikin

StringTokenizer Bug?

 
Ranch Hand
Posts: 134
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Ok simple short and sweet.
I'm writting an application that used the StringTokenizer to parse thru data. I am delimiting it by comma (,).

Example Data:
"a,b,c,,e"
When I iterate thru the String I would expect the StringTokenizer to process this result:
"a" "b" "c" "" "e"
Instead I'm getting:
"a" "b" "c" "e"

Any idea why the StringTokenizer isn't tokenizing empty Strings??? I looked in the javadoc and it says nothing about ignoring empty Strings. I guess I could write my own tokenizer or do something to the data before tokenizing to place N/A where data doesn't exist. But don't want to go to the trouble if it's handled somehow in the StringTokenizer class.

Regards,
Ryan
 
Ranch Hand
Posts: 1087
Oracle Spring Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Originally posted by Ryan Bailey:
Any idea why the StringTokenizer isn't tokenizing empty Strings???


Yes this a limitation of StringTokenizer or bug if you say.I have read somewhere that StringTokenizer is a legacy class and sun discourage use of it, but hasn't deprecated this so far

Originally posted by Ryan Bailey:
I guess I could write my own tokenizer or do something to the data before tokenizing



You need not to write your own tokenizer if you are using jdk1.4 as in JDK 1.4 String class has a split method.
also split method consider empty tokens in a string.

Shailesh
[ April 30, 2005: Message edited by: Shailesh Chandra ]
 
Ryan Bailey
Ranch Hand
Posts: 134
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
LOL... So much for my SuperStringTokenizer class

Thanks for the reply!
Ryan
 
Ranch Hand
Posts: 1646
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
By default, StringTokenizer groups one or more consecutive delimiter characters together to separate tokens. However, it has an alternate method that will return both the tokens and each single delimeter character, also as a token.

If the returnDelims flag is true, then the delimiter characters are also returned as tokens. Each delimiter is returned as a string of length one. If the flag is false, the delimiter characters are skipped and only serve as separators between tokens.

While not as simple to use as String.split(","), it would allow you to determine when to create empty tokens. If you have to run in pre-1.4 JVMs, it's a reasonable route. Of course, if comma is your only delimiter character, you could write your own split method in a few lines of code.
[ April 30, 2005: Message edited by: David Harkness ]
reply
    Bookmark Topic Watch Topic
  • New Topic