Win a copy of Kotlin in Action this week in the Kotlin forum!
  • Post Reply Bookmark Topic Watch Topic
  • New Topic

StreamTokenizer 'word' tokens  RSS feed

 
Daniel Walker
Greenhorn
Posts: 19
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi there. I am using a StreamTokenizer to parse a data file, containing multiple entries which look like this:

Things are working OK, until I get to any entry which has it's value spread over multiple lines. The StreamTokenizer will tokenize from a quote character (") to either another quote character or an end of line - and this is the problem. I have tried removing any newline characters from the source document in a preprocessing stage, but this did not work too well. Any ideas?
 
Dale DeMott
Ranch Hand
Posts: 515
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Are you reading in 1 line at a time? What are you removing from your string?? is it \n or \r??
-Dale
 
Jim Yingst
Wanderer
Sheriff
Posts: 18671
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Try setting eolIsSignificant(false).
 
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
Boost this thread!