A Java Application Server such as Glassfish, JBoss, Weblogic, etc is a complete implementation of all of the Java EE specifications, of which EJB is one. As such, app servers lay down a large number of jar files on your disk when you install them. Most java IDE's have a project wizard that asks you what technologies your project will include when you set it up. They use this to copy the appropriate jars to your class path.
Note that most (perhaps all) of the Java EE API's have standalone implementations. So for example, OpenEJB is a standalone implementation of EJB that allows one to develop against the EJB API w/o installing an application server.
IMO Ben, I would not invest in learning EJB. I would learn the Java EE 6 stack, with a focus on JPA,
JSF and CDI. EJB has a de-emphasized role in JEE5, and a further de-emphasized role in JEE6.