Saturday, 22 September 2012

Thread analysis in weblogic on linux

This blog will take you through a few Linux commands to get the process id and thread id of any executing thread in a process and find the piece of code which is responsible of high usage of cpu.

This will be useful in cases where you have long running transactions and want to know which part of the code is consuming more amount of cpu.
Follow the below steps :

1. Get the pid of the java process of weblogic server running on a Linux machine :
ps -ef | grep java

Copy id of the java process

2. Get the list of all the threads belonging to the above process :
top -H -p <pid>
top commands sorts the output in descending order of cpu utilization. The first thread utilizes the higher amount of cpu.
Copy id of the first thread

3. Login to weblogic server administration console :
Go to Home > Summary of Servers > (serverName) > Monitoring > Threads > Dump Thread Stacks.

4. Search the copied tid in the thread dump.
Following stack shows thread with tid 32606 is blocked with a fat lock. This stack is generated by the weblogic socket thread and is for the explanatory purpose only :

"ExecuteThread: '0' for queue: 'weblogic.socket.Muxer'" id=24 idx=0x60 tid=32606 prio=5 alive, blocked, native_blocked, daemon
                -- Blocked trying to get lock: java/lang/String@0x86667f30[fat lock]
                at jrockit/vm/Threads.waitForUnblockSignal()V(Native Method)
                at jrockit/vm/Locks.fatLockBlockOrSpin([optimized]
                at jrockit/vm/Locks.lockFat([optimized]
                at jrockit/vm/Locks.monitorEnterSecondStageHard([optimized]
                at jrockit/vm/Locks.monitorEnterSecondStage([optimized]
                at jrockit/vm/Locks.monitorEnter([optimized]
                at weblogic/socket/EPollSocketMuxer.processSockets(
                at weblogic/socket/
                at weblogic/socket/SocketReaderRequest.execute(
                at weblogic/kernel/ExecuteThread.execute(
                at weblogic/kernel/
                at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
                -- end of trace
5. The stack corresponding to the tid is the piece of code where the cpu utilization is maximum.

This way you can find out the piece of code which is responsible for high cpu usage and replace it with an improved version.


Anonymous said...

Great Tip.. Thanks GSK!!

Anonymous said...

Great post dude. A useful post for performance monitoring.

Anonymous said...

Is there an alternative to get the top consuming CPU Thread Ids using JMX (via some MBean?)
I.e. can the data got from Step 2 be got from JMX Mbean?
If so can you share a sample

Ganesh Kamble said...

Have a look at the below links :