update doc

This commit is contained in:
hengyunabc committed 2020-09-23 00:41:01 +08:00
1 parent c1dcc8ff9e
commit f599c74408
133 files changed
+793 -617

No files matched your search

+49 -37
View File
@@ -18,53 +18,65 @@ dashboard
```
$ dashboard
ID NAME GROUP PRIORITY STATE %CPU TIME INTERRUPTED DAEMON
889 RMI TCP Connection(15)-30.10.166. RMI Runtime 9 RUNNABLE 48 0:5 false true
1077 Timer-for-arthas-dashboard-0 system 9 RUNNABLE 24 0:0 false true
1074 as-selector-daemon system 9 RUNNABLE 12 0:0 false true
284 JMX server connection timeout 284 RMI Runtime 9 TIMED_WAITI 8 0:3 false true
16 Timer-1 main 5 TIMED_WAITI 5 0:9 false true
47 Pandora pandora-qos-reporter Pool main 5 TIMED_WAITI 0 0:0 false true
48 JmonitorClient-CheckThread Pool [ main 5 TIMED_WAITI 0 0:0 false true
49 JmonitorClient-HeartBeat Pool [Th main 5 TIMED_WAITI 0 0:0 false true
50 JmonitorClient-ReaderThread Pool main 5 TIMED_WAITI 0 0:0 false true
957 RMI TCP Connection(16)-30.10.166. RMI Runtime 9 RUNNABLE 0 0:2 false true
51 JmonitorClient-WriterThread Pool main 5 TIMED_WAITI 0 0:0 false true
52 ContainerBackgroundProcessor[Stan main 5 TIMED_WAITI 0 0:0 false true
53 http-bio-8080-Acceptor-0 main 5 RUNNABLE 0 0:2 false true
54 http-bio-8080-AsyncTimeout main 5 TIMED_WAITI 0 0:0 false true
11 GC Daemon system 2 TIMED_WAITI 0 0:0 false true
Memory used total max usage GC
heap 59M 223M 1820M 3.26% gc.ps_scavenge.count 118
ps_eden_space 14M 114M 668M 2.11% gc.ps_scavenge.time(ms) 1890
ps_survivor_space 6M 6M 6M 96.08% gc.ps_marksweep.count 5
ps_old_gen 39M 103M 1365M 2.86% gc.ps_marksweep.time(ms) 1140
nonheap 234M 240M 0M 97.46%
code_cache 46M 47M 240M 19.49%
metaspace 167M 172M 0M 97.36%
Runtime Tomcat
os.name Mac OS X connector http-bio-8080
os.version 10.10.5 QPS 0.00
java.version 1.8.0_60 RT(ms) 1.13
java.home error/s 0.00
received/s 0B
systemload.average 3.44 sent/s 0B
processors 4 threadpool http-bio-8080
uptime 16020s busy 0
ID NAME GROUP PRIORITY STATE %CPU DELTA_TIME TIME INTERRUPTE DAEMON
-1 C2 CompilerThread0 - -1 - 1.55 0.077 0:8.684 false true
53 Timer-for-arthas-dashboard-07b system 5 RUNNABLE 0.08 0.004 0:0.004 false true
22 scheduling-1 main 5 TIMED_WAI 0.06 0.003 0:0.287 false false
-1 C1 CompilerThread0 - -1 - 0.06 0.003 0:2.171 false true
-1 VM Periodic Task Thread - -1 - 0.03 0.001 0:0.092 false true
49 arthas-NettyHttpTelnetBootstra system 5 RUNNABLE 0.02 0.001 0:0.156 false true
16 Catalina-utility-1 main 1 TIMED_WAI 0.0 0.000 0:0.029 false false
-1 G1 Young RemSet Sampling - -1 - 0.0 0.000 0:0.019 false true
17 Catalina-utility-2 main 1 WAITING 0.0 0.000 0:0.025 false false
34 http-nio-8080-ClientPoller main 5 RUNNABLE 0.0 0.000 0:0.016 false true
23 http-nio-8080-BlockPoller main 5 RUNNABLE 0.0 0.000 0:0.011 false true
-1 VM Thread - -1 - 0.0 0.000 0:0.032 false true
-1 Service Thread - -1 - 0.0 0.000 0:0.006 false true
-1 GC Thread#5 - -1 - 0.0 0.000 0:0.043 false true
Memory used total max usage GC
heap 36M 70M 4096M 0.90% gc.g1_young_generation.count 12
g1_eden_space 6M 18M -1 33.33% 86
g1_old_gen 30M 50M 4096M 0.74% gc.g1_old_generation.count 0
g1_survivor_space 491K 2048K -1 24.01% gc.g1_old_generation.time(ms) 0
nonheap 66M 69M -1 96.56%
codeheap_'non-nmethods' 1M 2M 5M 22.39%
metaspace 46M 47M -1 98.01%
Runtime
os.name Mac OS X
os.version 10.15.4
java.version 15
java.home /Library/Java/JavaVirtualMachines/jdk-15.jdk/Contents/Home
systemload.average 10.68
processors 8
uptime 272s
```
### 数据说明
* ID: Java级别的线程ID,注意这个ID不能跟jstack中的nativeID一一对应
* ID: Java级别的线程ID,注意这个ID不能跟jstack中的nativeID一一对应。
* NAME: 线程名
* GROUP: 线程组名
* PRIORITY: 线程优先级, 1~10之间的数字,越大表示优先级越高
* STATE: 线程的状态
* CPU%: 线程消耗的cpu占比,采样100ms,将所有线程在这100ms内的cpu使用量求和,再算出每个线程的cpu使用占比。
* TIME: 线程运行总时间,数据格式为`分:秒`
* CPU%: 线程的cpu使用率。比如采样间隔1000ms,某个线程的增量cpu时间为100ms,则cpu使用率=100/1000=10%
* DELTA_TIME: 上次采样之后线程运行增量CPU时间,数据格式为`秒`
* TIME: 线程运行总CPU时间,数据格式为`分:秒`
* INTERRUPTED: 线程当前的中断位状态
* DAEMON: 是否是daemon线程
#### JVM内部线程
Java 8之后支持获取JVM内部线程CPU时间,这些线程只有名称和CPU时间,没有ID及状态等信息(显示ID为-1)。
通过内部线程可以观测到JVM活动,如GC、JIT编译等占用CPU情况,方便了解JVM整体运行状况。
* 当JVM 堆(heap)/元数据(metaspace)空间不足或OOM时,可以看到GC线程的CPU占用率明显高于其他的线程。
* 当执行`trace/watch/tt/redefine`等命令后,可以看到JIT线程活动变得更频繁。因为JVM热更新class字节码时清除了此class相关的JIT编译结果,需要重新编译。
JVM内部线程包括下面几种:
* JIT编译线程: 如 `C1 CompilerThread0`, `C2 CompilerThread0`
* GC线程: 如`GC Thread0`, `G1 Young RemSet Sampling`
* 其它内部线程: 如`VM Periodic Task Thread`, `VM Thread`, `Service Thread`
### 截图展示
![](_static/dashboard.png "dashboard")
![](_static/dashboard.png "dashboard")
+74 -55
View File
@@ -13,16 +13,23 @@ thread
|*id*|线程id|
|[n:]|指定最忙的前N个线程并打印堆栈|
|[b]|找出当前阻塞其他线程的线程|
|[i `<value>`]|指定cpu占比统计的采样间隔,单位为毫秒|
|[i `<value>`]|指定cpu使用率统计的采样间隔,单位为毫秒,默认值为200|
|[--all]|显示所有匹配的线程|
> cpu占比是如何统计出来的?
### cpu使用率是如何统计出来的?
> 这里的cpu统计的是,一段采样间隔内,当前JVM里各个线程所占用的cpu时间占总cpu时间的百分比。其计算方法为:
> 首先进行一次采样,获得所有线程的cpu的使用时间(调用的是`java.lang.management.ThreadMXBean#getThreadCpuTime`这个接口),然后睡眠一段时间,默认100ms,可以通过`-i`参数指定,然后再采样一次,最后得出这段时间内各个线程消耗的cpu时间情况,最后算出百分比。
这里的cpu使用率与linux 命令`top -H -p <pid>` 的线程`%CPU`类似,一段采样间隔时间内,当前JVM里各个线程的增量cpu时间与采样间隔时间的比例。
#### 工作原理说明:
* 首先第一次采样,获取所有线程的CPU时间(调用的是`java.lang.management.ThreadMXBean#getThreadCpuTime()`及`sun.management.HotspotThreadMBean.getInternalThreadCpuTimes()`接口)
* 然后睡眠等待一个间隔时间(默认为200ms,可以通过`-i`指定间隔时间)
* 再次第二次采样,获取所有线程的CPU时间,对比两次采样数据,计算出每个线程的增量CPU时间
* 线程CPU使用率 = 线程增量CPU时间 / 采样间隔时间 * 100%
> 注意: 这个统计也会产生一定的开销(JDK这个接口本身开销比较大),因此会看到as的线程占用一定的百分比,为了降低统计自身的开销带来的影响,可以把采样间隔拉长一些,比如5000毫秒。
> 如果想看从Java进程启动开始到现在的cpu占比情况:可以使用[show-busy-java-threads](https://github.com/oldratlee/useful-scripts/blob/master/docs/java.md#-show-busy-java-threads)这个脚本
> 另外一种查看Java进程的线程cpu使用率方法:可以使用[show-busy-java-threads](https://github.com/oldratlee/useful-scripts/blob/master/docs/java.md#-show-busy-java-threads)这个脚本
### 使用参考
@@ -30,60 +37,70 @@ thread
```shell
$ thread -n 3
"as-command-execute-daemon" Id=29 cpuUsage=75% RUNNABLE
at sun.management.ThreadImpl.dumpThreads0(Native Method)
at sun.management.ThreadImpl.getThreadInfo(ThreadImpl.java:440)
at com.taobao.arthas.core.command.monitor200.ThreadCommand$1.action(ThreadCommand.java:58)
at com.taobao.arthas.core.command.handler.AbstractCommandHandler.execute(AbstractCommandHandler.java:238)
at com.taobao.arthas.core.command.handler.DefaultCommandHandler.handleCommand(DefaultCommandHandler.java:67)
at com.taobao.arthas.core.server.ArthasServer$4.run(ArthasServer.java:276)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Number of locked synchronizers = 1
- java.util.concurrent.ThreadPoolExecutor$Worker@6cd0b6f8
"C1 CompilerThread0" [Internal] cpuUsage=1.63% deltaTime=3ms time=1170ms
"as-session-expire-daemon" Id=25 cpuUsage=24% TIMED_WAITING
at java.lang.Thread.sleep(Native Method)
at com.taobao.arthas.core.server.DefaultSessionManager$2.run(DefaultSessionManager.java:85)
"arthas-command-execute" Id=23 cpuUsage=0.11% deltaTime=0ms time=401ms RUNNABLE
at java.management@11.0.7/sun.management.ThreadImpl.dumpThreads0(Native Method)
at java.management@11.0.7/sun.management.ThreadImpl.getThreadInfo(ThreadImpl.java:466)
at com.taobao.arthas.core.command.monitor200.ThreadCommand.processTopBusyThreads(ThreadCommand.java:199)
at com.taobao.arthas.core.command.monitor200.ThreadCommand.process(ThreadCommand.java:122)
at com.taobao.arthas.core.shell.command.impl.AnnotatedCommandImpl.process(AnnotatedCommandImpl.java:82)
at com.taobao.arthas.core.shell.command.impl.AnnotatedCommandImpl.access$100(AnnotatedCommandImpl.java:18)
at com.taobao.arthas.core.shell.command.impl.AnnotatedCommandImpl$ProcessHandler.handle(AnnotatedCommandImpl.java:111)
at com.taobao.arthas.core.shell.command.impl.AnnotatedCommandImpl$ProcessHandler.handle(AnnotatedCommandImpl.java:108)
at com.taobao.arthas.core.shell.system.impl.ProcessImpl$CommandProcessTask.run(ProcessImpl.java:385)
at java.base@11.0.7/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base@11.0.7/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base@11.0.7/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
at java.base@11.0.7/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base@11.0.7/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base@11.0.7/java.lang.Thread.run(Thread.java:834)
"Reference Handler" Id=2 cpuUsage=0% WAITING on java.lang.ref.Reference$Lock@69ba0f27
at java.lang.Object.wait(Native Method)
- waiting on java.lang.ref.Reference$Lock@69ba0f27
at java.lang.Object.wait(Object.java:503)
at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:133)
"VM Periodic Task Thread" [Internal] cpuUsage=0.07% deltaTime=0ms time=584ms
```
#### 当没有参数时,显示所有线程的信息。
* 没有线程ID,包含`[Internal]`表示为JVM内部线程,参考[dashboard](dashboard.md)命令的介绍。
* `cpuUsage`为采样间隔时间内线程的CPU使用率,与[dashboard](dashboard.md)命令的数据一致。
* `deltaTime`为采样间隔时间内线程的增量CPU时间,小于1ms时被取整显示为0ms。
* `time` 线程运行总CPU时间。
注意:线程栈为第二采样结束时获取,不能表明采样间隔时间内该线程都是在处理相同的任务。建议间隔时间不要太长,可能间隔时间越大越不准确。
可以根据具体情况尝试指定不同的间隔时间,观察输出结果。
#### 当没有参数时,显示第一页线程的信息
默认按照CPU增量时间降序排列,只显示第一页数据。
```shell
$ thread
Threads Total: 16, NEW: 0, RUNNABLE: 7, BLOCKED: 0, WAITING: 5, TIMED_WAITING: 4, TERMINATED: 0
ID NAME GROUP PRIORITY STATE %CPU TIME INTERRUPTE DAEMON
30 as-command-execute-daemon system 9 RUNNABLE 72 0:0 false true
23 as-session-expire-daemon system 9 TIMED_WAIT 27 0:0 false true
22 Attach Listener system 9 RUNNABLE 0 0:0 false true
11 pool-2-thread-1 main 5 TIMED_WAIT 0 0:0 false false
12 Thread-2 main 5 RUNNABLE 0 0:0 false true
13 pool-3-thread-1 main 5 TIMED_WAIT 0 0:0 false false
25 as-selector-daemon system 9 RUNNABLE 0 0:0 false true
14 Thread-3 main 5 TIMED_WAIT 0 0:0 false false
26 pool-5-thread-1 system 5 WAITING 0 0:0 false false
15 Thread-4 main 5 RUNNABLE 0 0:0 false false
1 main main 5 WAITING 0 0:2 false false
2 Reference Handler system 10 WAITING 0 0:0 false true
3 Finalizer system 8 WAITING 0 0:0 false true
4 Signal Dispatcher system 9 RUNNABLE 0 0:0 false true
20 NonBlockingInputStreamThread main 5 WAITING 0 0:0 false true
21 Thread-8 main 5 RUNNABLE 0 0:0 false true
Threads Total: 33, NEW: 0, RUNNABLE: 9, BLOCKED: 0, WAITING: 3, TIMED_WAITING: 4, TERMINATED: 0, Internal threads: 17
ID NAME GROUP PRIORITY STATE %CPU DELTA_TIME TIME INTERRUPT DAEMON
-1 C2 CompilerThread0 - -1 - 5.06 0.010 0:0.973 false true
-1 C1 CompilerThread0 - -1 - 0.95 0.001 0:0.603 false true
23 arthas-command-execute system 5 RUNNABLE 0.17 0.000 0:0.226 false true
-1 VM Periodic Task Thread - -1 - 0.05 0.000 0:0.094 false true
-1 Sweeper thread - -1 - 0.04 0.000 0:0.011 false true
-1 G1 Young RemSet Sampling - -1 - 0.02 0.000 0:0.025 false true
12 Attach Listener system 9 RUNNABLE 0.0 0.000 0:0.022 false true
11 Common-Cleaner InnocuousThrea 8 TIMED_WAI 0.0 0.000 0:0.000 false true
3 Finalizer system 8 WAITING 0.0 0.000 0:0.000 false true
2 Reference Handler system 10 RUNNABLE 0.0 0.000 0:0.000 false true
4 Signal Dispatcher system 9 RUNNABLE 0.0 0.000 0:0.000 false true
15 arthas-NettyHttpTelnetBootstra system 5 RUNNABLE 0.0 0.000 0:0.029 false true
22 arthas-NettyHttpTelnetBootstra system 5 RUNNABLE 0.0 0.000 0:0.196 false true
24 arthas-NettyHttpTelnetBootstra system 5 RUNNABLE 0.0 0.000 0:0.038 false true
16 arthas-NettyWebsocketTtyBootst system 5 RUNNABLE 0.0 0.000 0:0.001 false true
17 arthas-NettyWebsocketTtyBootst system 5 RUNNABLE 0.0 0.000 0:0.001 false true
```
#### thread id, 显示指定线程的运行堆栈
#### thread --all, 显示所有匹配的线程
显示所有匹配线程信息,有时需要获取全部JVM的线程数据进行分析。
#### thread id, 显示指定线程的运行堆栈
```shell
$ thread 1
@@ -145,6 +162,10 @@ $ thread -b
#### thread -i, 指定采样时间间隔
* `thread -i 1000` : 统计最近1000ms内的线程CPU时间。
* `thread -n 3 -i 1000` : 列出1000ms内最忙的3个线程栈
```bash
$ thread -n 3 -i 1000
"as-command-execute-daemon" Id=4759 cpuUsage=23% RUNNABLE
@@ -170,11 +191,9 @@ $ thread -n 3 -i 1000
```bash
[arthas@28114]$ thread --state WAITING
Threads Total: 15, NEW: 0, RUNNABLE: 7, BLOCKED: 0, WAITING: 5, TIMED_WAITING: 3, TERMINATED: 0
ID NAME GROUP PRIORITY STATE %CPU TIME INTERRU DAEMON
198 AsyncAppender-Worker-arth system 9 WAITING 0 0:0 false true
3 Finalizer system 8 WAITING 0 0:0 false true
14 RMI Scheduler(0) system 9 WAITING 0 0:0 false true
2 Reference Handler system 10 WAITING 0 0:0 false true
204 pool-8-thread-1 system 5 WAITING 0 0:0 false false
Threads Total: 16, NEW: 0, RUNNABLE: 9, BLOCKED: 0, WAITING: 3, TIMED_WAITING: 4, TERMINATED: 0
ID NAME GROUP PRIORITY STATE %CPU DELTA_TIME TIME INTERRUPTE DAEMON
3 Finalizer system 8 WAITING 0.0 0.000 0:0.000 false true
20 arthas-UserStat system 9 WAITING 0.0 0.000 0:0.001 false true
14 arthas-timer system 9 WAITING 0.0 0.000 0:0.000 false true
```