• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other all forums
this forum made possible by our volunteer staff, including ...
  • Campbell Ritchie
  • Devaka Cooray
  • Knute Snortum
  • Paul Clapham
  • Tim Cooke
  • Liutauras Vilda
  • Jeanne Boyarsky
  • Bear Bibeault
Saloon Keepers:
  • Tim Moores
  • Stephan van Hulst
  • Ron McLeod
  • Piet Souris
  • Frits Walraven
  • Ganesh Patekar
  • Tim Holloway
  • salvin francis

StreamTokenizer help  RSS feed

Posts: 16
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
hi, does any1 know any good tutorial or examples of StreamTokenizer?? or maybe some1 can help me out a little?
i need to count words from a txt file..
i am really stupid and dont understand what it says in my txt book!
Ranch Hand
Posts: 1873
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
hi mike,
here is a sample code i wrote...

the input file i used is,]
243525 100 324200 2342 maulin
Ignoring WORD:maulin:
Ignoring WORD:ew22:
Ignoring WORD:fw:
Ignoring WORD:sdfsfsa:

i would try to explain the flow of events here...
- once we created the StreamTokenizerTest object and provided a file name to the argument of the method tokenizeFile() we are in the tokenizeFile() method...
- on Line-1 we create a Input stream to read the file
- on Line-2 we create a StreamTokenizer object that can tokenize the input file by reading it.
to do this we passed Reader type (which is FileReader) object to its constructor
- Line-3 says that -"run in the loop reading the file in tokens separated by white spaces untill you hit End Of File EOF". The StreamTokenizer.TT_EOF is a variable to indicate EOF value when the tokenizer reaches EOF while reading the file...
- Line-4 says that if the current token is "Number" then process it. "ttype" is a field in StreamToeknizer object to obtain the token type read by the tokenizer. the variable "nval" in the StreamToeknizer is to obtain the "number" value if the token was "number" indicated by "ttype".
- Line-5 is similary for WORD tokens (which are not numbers) and Line-6 is to identify the "new line" tokens in the file..
on Line-7 we close the FileReader as StreamTokenizer has reached the EOF and no more reading of the file is needed...
for more look at,
1. applet at this location
2. More explanation about the StreamTokenizer
3. Yet another tutorial
i bet u can find thousands on google...
hope this helps..
Maulin Vasavada
Ranch Hand
Posts: 1873
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
the second link i meant was,
this not the one i gave in above post...
this is better..
I miss the old days when I would think up a sinister scheme for world domination and you would show a little emotional support. So just look at this tiny ad:
how do I do my own kindle-like thing - without amazon
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
Boost this thread!