word / sentence regex pattern ?

[jay vas]

Hi guys : I am trying to parse words and sentences in a tokenizer.

Im using a hand coded system :

       //for words 
	(letter==' ' || letter==',' || letter=='/' || letter=='\n')

        //for regexes
	(letter=='.' || letter=='?' || letter=='!')

Any suggestions on a regular expression which is more comprehensive ?
I assume this problem has been solved before .

[Amit ChaudhariC]

you can try out something like

        String result = "abc.def?ghi!jkl";  
        String delims = "[.?!]";
        String[] tokens = result.split(delims);

Regards,
Amit

[Campbell Ritchie]

Two of those characters are metacharacters, but they appear to work in this context.

[Rob Spoor]

Most meta characters loose their meaning inside character classes. Other meta characters change in meaning (^ is start of input outside, negating inside), others are introduced (- is nothing outside, inside it means range unless it's the first character).

[Campbell Ritchie]

. . . but I can never remember which is which.

You do realise you can use methods of the Character class like isWhitespace, jay vas?