I think Performance Tuning and Capacity planning are different things.
Capacity planning - decide the hardware and software configuration required for specific application load (specific TPS)
Performance Tuning - Tune the application/software/hardware to optimize the TPS for a specific hardware/software configuration.
Typically we will be doing the performance tuning as part of application development. Capacity planning before deploying the application for any customer. However we may need to do further performance tuning after capacity planning.
Performance tuning:
First pass:
1) Do a load
testing for 15 minutes.
2) Monitor the different parameters
a) CPU
b) Time spent in application vs time spend in Database (or any other external system)
c) Garbage collection (time spend in GC vs total time)
d) Take multiple
thread dump and look for threads waiting for monitor entry (you may use samurai thread dump analysis)
e) network i/o rate
f) if total commit charge exceeds the physical memory
Analysis:
if more time spend on DB tune DB
if lot of threads in waiting for monitor entry, tune code to remove synchronization blocs
if lot of time spend on GC, try increase/decrease heap, multiple JVMs
if total commit charge exceeds physical memory decrease heap size
if network i/o is close to i/o capacity, use compression to reduce i/o
Second pass:
Run the application with CPU profiler
Find the hotspots. Analyze the code for optimization.
Capacity planning:
Run load testing for 15 minutes with application on standard machine and database other dependencies in higher machines
measure the TPS
Find TPS per CPU
network i/o per transaction (typically this may not be a big issue)
Calculate the number of CPUs required
Decide number of machines (m number of n CPU machines)
Load testing:
1) Find the key transaction (end user input)
2) Find the ratio of key transactions (end user input)
3) Write load testing stubs (using some load testing tool)
4) Increase the number of threads until you reach CPU usage of application to 80+%
DB tuning (oracle 9.2)
take snaps before and after load testing
take statspack report
look at top 5 timed events.
Tune DB to reduce item which is taking more time. Typically increase db buffer size, increase pga aggregate etc
Find the Sqls taking more time.
Find the explain plan for these SQLs
Tune these SQLs to have better plan. Typically proper indexing will do.