Potential problem causers:
Have you also configured web.xml with <servlet>...</servlet> & <servlet-mapping>...</servlet-mapping>?
Also (I'm sure this is the case) but I'm assuming your compiled servlets are in the classes directory?
Aside:
I'm a JBoss novice myself, and have found
Tomcat easier to configure for JSP/servlet deployment. If you're not using EJBs, have you considered using that instead?