linux 性能分析常规逻辑和手段总结

浏览数：136 / 时间：2015年06月20日

一. 追查cpu占用较高的进程(线程)
1 . 如何查找出当前系统中占用cpu或者内存最高的进程?

ps aux |sort -rn -k 3 |head -n3  查找出当前系统中cpu资源占用前三名的进程, -k 3 表示用ps结果中的第三列排序,ps aux输出结果列顺序为:

USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND,因此要是按照mem排序使用 k 4

[root@b28-1330 ~]# ps aux |sort -rn -k 3 |head -n3
root      7855 21.1  8.1 8618980 2696340 ?     Sl   Apr14 595:14 /export/servers/jdk1.6.0_25/bin/java -Djava.util.logging.config.file=/export/home/tomcat/domains/item.soa.360buy.com/server1/conf/logging.properties -Djava.library.path=/usr/local/lib -server -Xms2048m -Xmx2048m -XX:MaxPermSize=256m -Djava.awt.headless=true -Dsun.net.client.defaultConnectTimeout=60000 -Dsun.net.client.defaultReadTimeout=60000 -Djmagick.systemclassloader=no -Dnetworkaddress.cache.ttl=300 -Dsun.net.inetaddr.ttl=300 -Dcom.sun.management.jmxremote.port=10004 -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=false -Djava.rmi.server.hostname=192.168.13.30 -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Djava.endorsed.dirs=/export/servers/tomcat6.0.33/endorsed -classpath /export/servers/tomcat6.0.33/bin/bootstrap.jar -Dcatalina.base=/export/home/tomcat/domains/item.soa.360buy.com/server1 -Dcatalina.home=/export/servers/tomcat6.0.33 -Djava.io.tmpdir=/export/home/tomcat/domains/item.soa.360buy.com/server1/temp org.apache.catalina.startup.Bootstrap -config /export/home/tomcat/domains/item.soa.360buy.com/server1/conf/server.xml start
root      8268 20.9  8.5 8613152 2801708 ?     Sl   Apr14 587:22 /export/servers/jdk1.6.0_25/bin/java -Djava.util.logging.config.file=/export/home/tomcat/domains/item.soa.360buy.com/server2/conf/logging.properties -Djava.library.path=/usr/local/lib -server -Xms2048m -Xmx2048m -XX:MaxPermSize=256m -Djava.awt.headless=true -Dsun.net.client.defaultConnectTimeout=60000 -Dsun.net.client.defaultReadTimeout=60000 -Djmagick.systemclassloader=no -Dnetworkaddress.cache.ttl=300 -Dsun.net.inetaddr.ttl=300 -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Djava.endorsed.dirs=/export/servers/tomcat6.0.33/endorsed -classpath /export/servers/tomcat6.0.33/bin/bootstrap.jar -Dcatalina.base=/export/home/tomcat/domains/item.soa.360buy.com/server2 -Dcatalina.home=/export/servers/tomcat6.0.33 -Djava.io.tmpdir=/export/home/tomcat/domains/item.soa.360buy.com/server2/temp org.apache.catalina.startup.Bootstrap -config /export/home/tomcat/domains/item.soa.360buy.com/server2/conf/server.xml start
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND

2. 通过第一步我们确定了占用cpu最高的进程的pid是7855,那么如何确定该进程的线程cpu占用情况呢?

top -Hp 7855    -H 显示线程 -p 指定pid   如果执行上述命令之后,发现还有一个进程,那么按下 shift+H, 即可显示threads

root@b28-1330 ~]# top -Hp 7855

top - 14:09:41 up 330 days, 22:33,  4 users,  load average: 0.73, 0.81, 0.71
Tasks: 5416 total,   0 running, 5416 sleeping,   0 stopped,   0 zombie
Cpu(s):  2.9%us,  0.6%sy,  0.0%ni, 96.4%id,  0.0%wa,  0.0%hi,  0.2%si,  0.0%st
Mem:  32933092k total, 30577956k used,  2355136k free,   438232k buffers
Swap: 16779884k total,        0k used, 16779884k free, 22525628k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND                                                                
 7906 root      15   0 8416m 2.6g  11m S  3.9  8.2  16:58.92 java                                                                   
21006 root      15   0 8416m 2.6g  11m S  3.2  8.2   9:59.56 java                                                                   
21019 root      15   0 8416m 2.6g  11m S  2.9  8.2   9:55.70 java

经过排查,我们发现 tid(Thread Id)为7906线程占用cpu最高

3. 上一步我们已经找到了占用cpu最高的线程,那么我们下面要确定一下这个线程的详细信息

我们使用 jstack来进行深度追踪:

将tid 7906转换为十六进制: printf "%x\n" --> 1ee2

使用命令 jstack 7855 |grep -i 1ee2 -A 20 来查询进程详细信息:

[root@b28-1330 ~]#  jstack 7855 |grep -i 1ee2 -A 20 
"New I/O client worker #1-1" daemon prio=10 tid=0x000000005ad02000 nid=0x1ee2 runnable [0x0000000043972000]
   java.lang.Thread.State: RUNNABLE
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:210)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
        - locked <0x00000007801074a0> (a sun.nio.ch.Util$2)
        - locked <0x0000000780107490> (a java.util.Collections$UnmodifiableSet)
        - locked <0x00000007800fbcc0> (a sun.nio.ch.EPollSelectorImpl)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
        at org.jboss.netty.channel.socket.nio.SelectorUtil.select(SelectorUtil.java:38)
        at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:163)
        at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
        at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:44)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        at java.lang.Thread.run(Thread.java:662)

ok这个线程的堆栈信息就出来了...

二. 如何查看占用JVM内存最大的前20个class实例对象
主要用到jmap 和sort命令

[root@b28-1330 ~]# jmap -histo 7855 |sort -rn -k3|head -n11  
Total       6221631      455243264
   1:        111761       95387048  [B
   2:        884396       63940104  [C
   3:         52573       52139904  [I
   4:        680476       21775232  java.lang.String
   5:        120038       17978192  <constMethodKlass>
   6:        120038       16335920  <methodKlass>
   7:         11558       12871496  <constantPoolKlass>
   8:        103599       12431880  java.net.SocksSocketImpl
   9:        350199       11206368  java.util.HashMap$Entry
  10:        167532        9294056  <symbolKlass>