Per my understanding, you use the webserver to process requests from clients. An Application Server is an entirely different beast, which is used as the business logic tier. What this means is that the
Servlets and JSPs will be in the webserver and the EJBs(Session as well as Entity, which encapsulate the business logic) will be in the Application Server. The Application Server provides some primary services such as transaction management, persistence, security, object distribution etc. essential to mission-critical distributed applications and a webserver can not provide these services. Hence, the need for a webserver and an application server.
In a nutshell, your Servlets, JSPs and HTML pages should be in the webserver, while your EJBs should be in your application server.
Hope this helps!