SelectOneMenu, despite its name, is not a true menu. It renders as an HTML SELECT control, and HTML SELECT doesn't support separators. About the best you can do is add an item to the select list that displays as a row of dashes ("-----------"), but unlike true separators, this item will be selectable itself, so you'd have to allow for that.
There are
JSF extensions such as MyFaces Tomahawk and RichFaces that support GUI elements that look and feel more like true menus, but there's nothing like that in the core JSF tagset.