StreamTokenizer and hard returns

[Tom Griffith]

Hello. If somebody has a minute, I am trying to read the input from a semicolon delimited text file with the following format...

The Dog;The Cat;The Mouse
The Motorcycle;The Car;The Truck
The Square;The Circle;The Oval

I eventually want to convert each line to a row in a db table.

What I'm having trouble with is reading the hard return in the input stream. The crux of the code is as follows...

FileReader rd = new FileReader("relocated_input.txt");
StreamTokenizer st = new StreamTokenizer(rd);
st.quoteChar(';');
st.wordChars(0,' ');
st.eolIsSignificant(true);

// Parse the file
while (StreamTokenizer.TT_EOF != st.nextToken()) {
String word = st.sval;
System.out.println(word);
}
rd.close();

this is the output...

The Dog
The Cat
The Mouse
The Motorcycle
The Car
The Truck
The Square
The Circle
The Oval


Other than that, it seems I am able to get a null value for the hard returns if I have st.eolIsSIgnificant(true) and I ~don't~ designate the space (' ') as a word constituent. But I have to do that because the tokens might, or in this case, they all do, consist of two words and a space in between. So what I have is successful parsing but I can't get a handle on where the hard returns are located in the stream so I can seperate rows in the db. Thank you very much for reading. Any help is appreciated...I'll keep messing with it.
[ March 22, 2005: Message edited by: Tom Griffith ]

[Joe Ess]

I'd use a BufferedReader to read the file with readLine() so each record comes in as an individual string, then parse each line with a StringTokenizer (or better yet, String.split()).
[ March 22, 2005: Message edited by: Joe Ess ]

[Tom Griffith]

I was trying just that but it seems like readLine() appears to be giving me every other line or something...

BufferedReader rd = new BufferedReader(new FileReader("relocated_input.txt"));

while (rd.readLine() != null) {

String current_line = rd.readLine();

InputStream in = new StringBufferInputStream(current_line);

StreamTokenizer st = new StreamTokenizer(in);

st.quoteChar(';');
st.wordChars(0,' ');
st.eolIsSignificant(false);

// Parse the line

while (StreamTokenizer.TT_EOF != st.nextToken()) {

String word = st.sval;

System.out.println(word);

}

// END WHILE
}

yields...

The Motorcycle
The Car
The Truck

Yeah, I was jsut reading somewhere that StringTOkenizer is being phased out and the documentation suggested using Split on a StringBuffer. But even so, something is weird with my looping on the nextLine() method. Thank you very much for reading my question...
[ March 22, 2005: Message edited by: Tom Griffith ]

[Tom Griffith]

Making this change to the current_line declaration and assignment has the loop working ok...

BufferedReader rd = new BufferedReader(new FileReader("relocated_input.txt"));

String current_line;

while ( (current_line = rd.readLine()) != null) {


as opposed to this which was giving me unpredictable results (every other line, etc)...


while ( rd.readLine() != null) {

String current_line = rd.readLine();

Obviously creating a new String object within the loop as opposed to overwriting the reference to the String object was messing with it. Again, thank you very much for your help.

[Carol Enderlin]

Your while statement was throwing every other line away.

while ( rd.readLine() != null) { <<--- readLine, but don't capture it

String current_line = rd.readLine();