I wanted to know how to design servlets/jsp based web applications to handle many users.
What I am looking for is resources where I can learn the basics and advanced know-how of multithreaded servlet apps and how to code and design them.
Dude seems like you are just starting with your Servlet/Jsp. my advice is don't directly jump to Design of your application. Try some "Hello world" application you will get good hold of basics.
Just to reiterate, the servlet spec was designed from day one with to be multi-user / multi-threaded so, as long as you avoid some of the common issues (primarily the use of instance or non-final static variables) your app -- at least the front end-- will automatically be multi-user. If you're app works with a shared resource such as a relational database, then you'll have to consider what could happen if two people try to access it at the same time.
Like Malatesh said, have fun with some simple stuff first.