This week's giveaway is in the JDBC forum.
We're giving away four copies of Java Database Connections & Transactions (e-book only) and have Marco Behler on-line!
See this thread for details.
Win a copy of Java Database Connections & Transactions (e-book only) this week in the JDBC forum!
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Devaka Cooray
  • Knute Snortum
  • Paul Clapham
  • Tim Cooke
Sheriffs:
  • Liutauras Vilda
  • Jeanne Boyarsky
  • Bear Bibeault
Saloon Keepers:
  • Tim Moores
  • Stephan van Hulst
  • Ron McLeod
  • Piet Souris
  • Frits Walraven
Bartenders:
  • Ganesh Patekar
  • Tim Holloway
  • salvin francis

tokenizer  RSS feed

 
Greenhorn
Posts: 23
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi everyone,
I made a StreamTokenizer object to capture keyboard input from the user. I want to be able to input numbers like "001" and "002" but the in tokenizer.nval it only captures "1" and "2". Is there any way around this?
StreamTokenizer tokenizer = new StreamTokenizer(new InputStreamReader(System.in));
if (tokenizer.nextToken()==tokenizer.TT_NUMBER)
{ return (int)tokenizer.nval; }
 
Ranch Hand
Posts: 1865
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
It sounds like you want a string representation of the input numbers so why not just read them as a String using nextToken()?
 
henry wu
Greenhorn
Posts: 23
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I tried doing this:
if (tokenizer.nextToken()==tokenizer.TT_WORD) {
return tokenizer.sval;
}
else {
System.out.println("This is not a valid string\n");
}
When I entered in '001' It told me that it wasn't a valid string.. so it didn't pass the tokenizer.TT_WORD test. Any ideas?
 
Wanderer
Posts: 18671
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Evidently StreamTokenizer considers 001 to be a number, not a word, and it considers 001 to be exactly equivalent to 1. If it's important to you that 001 is different from 1, you'll need to tell the StreamTokenizer to treat numeric characters as parts of words:
 
henry wu
Greenhorn
Posts: 23
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
cool thanks.. I'll try this out!
 
henry wu
Greenhorn
Posts: 23
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Ok the new code looks like this:
tokenizer.wordChars('0','9');
if (tokenizer.nextToken()==tokenizer.TT_WORD) {
return tokenizer.sval;
}
else {
System.out.println("This is not a valid string\n");
}
But it is still not recognizing '001' as a word.. Am I placing the wordChars function call in the wrong place?
 
mister krabs
Posts: 13974
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
wordChars takes two ints as parameters. Try it without the single quotes.
 
henry wu
Greenhorn
Posts: 23
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Ok, changed to -
tokenizer.wordChars(0,9);
if (tokenizer.nextToken()==tokenizer.TT_WORD) {
return tokenizer.sval;
}
else {
System.out.println("This is not a valid string\n");
}
Still didn't work..
Do you guys have any recommendations for keyboard input for integers and strings other than StreamTokenizer and BufferedReader?
 
Jim Yingst
Wanderer
Posts: 18671
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
No, Thomas is on the wrong trail here. Passing int values of 0 and 9 just ends up referring to Unicode values of 0-9. You definitely want the char values, which are '0' and '9'. I think they only use int in the API so that people can use int literals without casting if they wish. Rather silly, and misleading.
Looking further in the API, it appears that characters can be both number chars and word chars simultaneously. Evidently the code checks for number chars before it checks for word chars, and so it ignores the fact that we told it that '0'-'9' are word chars. Instead, we must explicitly tell the tokenizer that '0'-'9' are not numbers:

Note that ordinaryChars() wipes out any special attributes associated with the given chars, so it's necessary to assert the word nature of the chars after ordinaryChars().
On inspection, it appears that StreamTokenizer has one of the more bizarre and confusing API's I've seen from Sun. Maybe even worse than Calendar. Ugh. Depending what you're doing, you may well find it's easier to write your own parsing classes.
 
henry wu
Greenhorn
Posts: 23
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
cool.. that worked! Thanks for all the help!
should i have used stringTokenizer from the start?
[ July 10, 2002: Message edited by: henry wu ]
 
To do a great right, do a little wrong - shakepeare. twisted little ad:
how do I do my own kindle-like thing - without amazon
https://coderanch.com/t/711421/engineering/kindle-amazon
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
Boost this thread!