@Qualifier has indeed higher priority than @Primary. Works as expected for me with sample program according to your structure. Do you have all those classes in scope of Spring's component scan?
// Edit:
You should have class A in scope of component scan, otherwise bean would not be found, but check if class Y is in scope of component scan.