I believe you have hit on one of the instances where Object Databases are superior to relational databases,
polymorphism. In the relational world, Columns have a specific datatype, while in the OO world, a column can be a pointer (or ref or whatever) to another type. If you want to do this in a relational world, I can think of as few work arounds.
1) Maybe you can get by with a UDT or REF? This makes the programming more complicated, assuming your database (and
JDBC driver supports it).
2) Use an intermediate table, which has three columns, a primary key (which is the new foriegn key in your original table), and a column for group ID and a column for user ID. Then you just follow the ID which is not NULL.