• Post Reply
  • Bookmark Topic Watch Topic
  • New Topic

StringTokenizer and null fields

 
Leslie Chaim
Ranch Hand
Posts: 336
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi,
I want to parse some CSV text using StringTokenizer, and I am having problem with the null fields. I searched around and found that the StringTokenizer indeed does skip over null fields.
Does anyone know of any API alternative, or must I roll my own little parser as suggested elsewhere.
Your replies are greatly appreciated.
 
Jim Yingst
Wanderer
Sheriff
Posts: 18671
  • Likes 1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
If using SDK 1.4, you can use split(String) method in String (and in the java.util.regex classes).

Otherwise rolling your own is best. Or you can use this, which I made some time ago:

[ July 09, 2002: Message edited by: Jim Yingst ]
 
William Brogden
Author and all-around good cowpoke
Rancher
Posts: 13071
6
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
If I understand your problem correctly, you can use the constructor that takes a returnDelims flag.
public StringTokenizer(String str,
String delim,
boolean returnDelims)Constructs a string tokenizer for the specified string. All characters in the delim argument are the delimiters for separating tokens.
If the returnDelims flag is true, then the delimiter characters are also returned as tokens. Each delimiter is returned as a string of length one. If the flag is false, the delimiter characters are skipped and only serve as separators between tokens.
Then count the delimiters to decide which field you are in.
Bill
 
Jim Yingst
Wanderer
Sheriff
Posts: 18671
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
But the problem with using the returnDelims parameter is that you have to put in extra logic to determine which tokens were delimiters and which weren't. It's not enough to just look at the length, since there might be a non-delimeter field with length 1. And then you still have to put in the logic for actually using the field count. I found it was easier to make my own tokenizer from scratch, rather than adapt the existing StringTokenizer. YMMV of course.
[ July 09, 2002: Message edited by: Jim Yingst ]
 
Leslie Chaim
Ranch Hand
Posts: 336
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
We use the 1.3 kit, I hope we upgrade to 1.4 for obvious reasons but the biggest reason would be the regex support.
I actually tried using the constructor with the return delim set to true but as Jim points out there is the extra logic where you almost wonder if the StringTokenizer class is a proper solution.
In my case, since I only have a single character as a delimiter I totally abandoned the StringTokenzier thingJ Instead I used methods of the string class as follows:
<code>
String data[] = new String[num_expected_tokens];
int begin_index = 0;
int end_index = line.indexOf('~', begin_index);
for ( int i = 0; i < num_expected_tokens; i++ )
{
data[i] = line.substring(begin_index, end_index);
begin_index = end_index + 1;
end_index = line.indexOf('~', begin_index);
}
</code>
Yes, in my case I figured out num_expected_tokens elsewhere but I think this solution is much cleaner then using the StringTokenizer class.
Thanks for your help.
 
Dilanga Thalpegama
Greenhorn
Posts: 1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
NICE..!
 
  • Post Reply
  • Bookmark Topic Watch Topic
  • New Topic