JAVA服务CPU过高问题排查
- 使用top定位道占用cpu高的进程pid
top
top - 17:05:28 up 40 days, 6:22, 0 users, load average: 4.63, 3.96, 3.46
Tasks: 3 total, 1 running, 2 sleeping, 0 stopped, 0 zombie
%Cpu0 : 29.6 us, 2.7 sy, 0.0 ni, 66.0 id, 0.0 wa, 0.0 hi, 1.7 si, 0.0 st
%Cpu1 : 25.9 us, 2.4 sy, 0.0 ni, 70.4 id, 0.0 wa, 0.0 hi, 1.4 si, 0.0 st
%Cpu2 : 34.5 us, 4.1 sy, 0.0 ni, 60.1 id, 0.0 wa, 0.0 hi, 1.4 si, 0.0 st
%Cpu3 : 23.1 us, 2.7 sy, 0.0 ni, 73.6 id, 0.0 wa, 0.0 hi, 0.7 si, 0.0 st
%Cpu4 : 26.1 us, 4.4 sy, 0.0 ni, 67.8 id, 0.0 wa, 0.0 hi, 1.7 si, 0.0 st
%Cpu5 : 11.2 us, 3.1 sy, 0.0 ni, 84.4 id, 0.0 wa, 0.0 hi, 1.4 si, 0.0 st
%Cpu6 : 24.7 us, 5.1 sy, 0.0 ni, 68.5 id, 0.0 wa, 0.0 hi, 1.7 si, 0.0 st
%Cpu7 : 26.1 us, 3.1 sy, 0.0 ni, 70.2 id, 0.0 wa, 0.0 hi, 0.7 si, 0.0 st
%Cpu8 : 26.0 us, 3.4 sy, 0.0 ni, 66.9 id, 0.0 wa, 0.0 hi, 3.7 si, 0.0 st
%Cpu9 : 14.6 us, 2.4 sy, 0.0 ni, 82.0 id, 0.0 wa, 0.0 hi, 1.0 si, 0.0 st
%Cpu10 : 30.2 us, 7.5 sy, 0.0 ni, 59.7 id, 0.0 wa, 0.0 hi, 2.7 si, 0.0 st
%Cpu11 : 8.5 us, 2.4 sy, 0.0 ni, 87.8 id, 0.0 wa, 0.0 hi, 1.4 si, 0.0 st
%Cpu12 : 13.7 us, 3.8 sy, 0.0 ni, 79.2 id, 0.0 wa, 0.0 hi, 3.4 si, 0.0 st
%Cpu13 : 40.7 us, 2.4 sy, 0.0 ni, 56.2 id, 0.0 wa, 0.0 hi, 0.7 si, 0.0 st
%Cpu14 : 18.8 us, 3.4 sy, 0.0 ni, 74.7 id, 0.0 wa, 0.0 hi, 3.1 si, 0.0 st
%Cpu15 : 7.5 us, 3.7 sy, 0.0 ni, 87.8 id, 0.0 wa, 0.0 hi, 1.0 si, 0.0 st
KiB Mem : 65704540 total, 6177900 free, 35425588 used, 24101052 buff/cache
KiB Swap: 0 total, 0 free, 0 used. 29786196 avail Mem
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
1 root 20 0 12.5g 3.2g 20004 S 52.0 5.0 21323:28 java
43548 root 20 0 15872 4268 3056 S 0.0 0.0 0:00.03 bash
43564 root 20 0 59568 4420 3732 R 0.0 0.0 0:00.00 top
-
使用
top -H -p <pid>命令获取线程信息，并找到占用cpu高的线程，也可以通过命令ps -mp <pid> -o THREAD,tid,time | sort -rntop -H -p 1 top - 17:13:26 up 40 days, 6:30, 0 users, load average: 2.65, 3.87, 3.75 Threads: 180 total, 0 running, 180 sleeping, 0 stopped, 0 zombie %Cpu(s): 14.3 us, 3.7 sy, 0.0 ni, 80.0 id, 0.0 wa, 0.0 hi, 2.1 si, 0.0 st KiB Mem : 65704540 total, 5972536 free, 35453284 used, 24278720 buff/cache KiB Swap: 0 total, 0 free, 0 used. 29758584 avail Mem PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 51 root 20 0 12.5g 3.2g 20004 S 6.0 5.0 3479:41 java 52 root 20 0 12.5g 3.2g 20004 S 6.0 5.0 3485:09 java 53 root 20 0 12.5g 3.2g 20004 S 5.6 5.0 3488:47 java 54 root 20 0 12.5g 3.2g 20004 S 5.6 5.0 3477:56 java 43586 root 20 0 12.5g 3.2g 20004 S 3.0 5.0 0:00.29 java 67 root 20 0 12.5g 3.2g 20004 S 2.7 5.0 1137:14 java 43544 root 20 0 12.5g 3.2g 20004 S 2.3 5.0 0:11.16 java 43543 root 20 0 12.5g 3.2g 20004 S 2.0 5.0 0:19.46 java 50 root 20 0 12.5g 3.2g 20004 S 1.0 5.0 452:41.33 java 62 root 20 0 12.5g 3.2g 20004 S 0.7 5.0 446:12.64 java 12 root 20 0 12.5g 3.2g 20004 S 0.3 5.0 21:31.34 java 1 root 20 0 12.5g 3.2g 20004 S 0.0 5.0 0:00.04 java 10 root 20 0 12.5g 3.2g 20004 S 0.0 5.0 0:06.56 java -
将需要的线程id转换为16进制格式
# printf "%x\n" tid printf "%x\n" 43565 aa2d -
查找具体堆栈信息
jstack < pid >
可以查看进程下所有堆栈信息。然后根据十六进制的线程id定位到具体有问题的堆栈和业务代码。也可以直接定位具体线程id。
使用jstack查询线程的堆栈信息
语法：jstack < pid > | grep -a 线程id（十六进制）可以导入到文件，本地查看。例如
jstack < pid > | grep -A10 3d30>>/tmp/pid.log