JavaCC problem with duplicate Tokens

[Basti Schaf]

Hello,
I always get into the following situation:


Imagine I have the following file to parse

Foo Ball hall
Zoe Ball mall

Bar Ball hall
Zoe Ball anythingelse


Well, the file might be defined that way, that I know there will be group of lines (2 in this case)
One is alway starting with "Zoe" (the second one, as "Zoe Ball mall" and "Zoe Ball anythingelse")
The second token will always be "Ball" in every line.



parse()
{
String found = "";
}
{
(
// First line
(
<ALPHANUM>
<BALLTOKEN>
<ALPHANUM>
{ found = "Hey, I am in line one";}
)
//Second line, the one with zoe.
(
<ALPHANUMSTARTSWITHZ>
<BALLTOKEN>
<ALPHANUM>
{ found = "Hey, I am in line two!!! yaey.";}
)
)*
}

TOKEN : { <ALPHANUM ["a-z","A-Z","0-9"]> }
TOKEN : { <ALPHASTARTINGWITHZ "Z"(["a-z"])*> }
TOKEN : { <BALLTOKEN "Ball" >}

What is gonna happen?!

Foo will match <ALPHANUM>
Ball will match <BALLTOKEN>
hall will match <ALPHANUM>

Zoe is supposed to match <ALPHANUMSTARTINGWITHZ> but will match <ALPHANUM> and an Exception will be thrown, because Zoe also matches <ALPHANUM>
Ball
mall

How can I resolve that problem?

[Don Solomon]

you will need to consider LOOKAHEAD to resolve the ambiguity.

[Basti Schaf]

I solved the problems with states like

<IN_VARDEFINITION> TOKEN: { <TEMPVARNAME : ("-")(<CSIDENTIFIER> :wink: > : IN_VARASSIGNMENT }

Not depending on the example, defined in the first tag, but a general example.
I don't think that LOOKAHEAD would be the right sollution, because in fact the JavaCC just thought it found an TokenAB even if it was just a TokenA.