100000 concurrent users

[kri shan]

My system has very high number of online users (For eg 100000 concurrent users). Which is the best way to acheive 100000 concurrent users.
Whether Thread Pooling is the beat way to handle the concurrent requests?

[Ilja Preuss]

First it's important that # of concurrent users is not the same as # of concurrent requests. The latter depends on how often you will get a request from a user and how long a request takes.

[steve souza]

Where I work they have a system with over 400,000 registered users. However, at any given time we only have a fraction of that doing anything. I think typically the maximum number of concurrent requests is about 30 in each of 4 JVMs. So say a max of 120 simultaneous requests for 400,000 users. If you truly have 100,000 simultaneous requests you have a tough job ahead of you I would expect...

[ak pillai]

100,000 users does not mean you are going to have as many concurrent requests. You need to some how evaluate the number of concurrent users during the peak hours first.

Your application server already has a thread pool and all you have to use is optimize it. Creating and destroying threads are expensive, so you need to have a thread-pool (say 20 � 50 threads, varies for different operating systems. read your server manual for recommended size).

Using a large number of threads adversely affects performance by consuming memory through thread stacks and CPU by context switching.

Say, you need to support 80 concurrent users, then you can have two ApplicationServers in a clustered environment, each with thread pool size configured at 40 threads per pool.

[kri shan]

We have our own application server(not any third party application server). Each application server will handle appx 1000 concurrent requests. Hence 100000/1000=100 application servers. Is it correct way to handle 100000 concurrent requests ?

[Ilja Preuss]

First, it's still not clear what you mean by "concurrent requests".

Second, it's unlikely that the number of requests you can handle will scale linearly with the number of application servers. There will be some single point where it gets decided which server to access, and there might be other places where the servers have to coordinate (for example, they might need to access a common database).

Therefore it's quite likely that 100 servers will be slower than hundred times the performance of one server.

[kri shan]

Ilja, our Load Balancer hardware decides which server to access.