android bugreport关键字

aosp关于read bugreport的文档：https://source.android.com/setup/contribute/read-bug-reports，建议详细阅读。

跟踪CPU info

while true; do dumpsys cpuinfo| grep TOTAL; sleep 1; done

焦点窗口

dumpsys window | grep -E mFocusedApp

获取堆栈

kill -3 然后到/data/anr下面找对应trace，因为art会监听SIGQUIT信号，然后调用runtime->DmpFOrSigQuit();

查看系统运行时间

 adb shell cat /proc/uptime| awk -F. '{run_days=$1 / 86400;run_hour=($1 % 86400)/3600;run_minute=($1 % 3600)/60;run_second=$1 % 60;printf("系统已运行：%d天%d时%d分%d 秒",run_days,run_hour,run_minute,run_second)}'

基本信息

开机运行时间和抓bugreport的时间

抓日志前手机运行的时间越短，包含有效信息的可能性越低。有些用户可能觉得很卡，但是重启手机后才反馈过来，这样就丢失了现场。看反馈之前首先要确认这点。

// 13:01 表示小时：分钟
------ UPTIME (uptime) ------
22:43:07 up 13:01, 0 users, load average: 0.59, 2.17, 2.24
// 抓取日志时间
== dumpstate: 2018-11-04 20:32:41

OTA升级时间

来自 dumpsys package 命令，可以结合开机时间推断本次开机是不是ota后的第一次开机，用来分析OTA后卡顿问题。

// 升级时间，原始版本和目标版本
2018/11/3 下午8:05: Upgrading from XXX/jason/jason:8.1.0/OPM1.171019.019/V10.0.4.0.OCHCNFH:user/release-keys to
XXX/jason/jason:8.1.0/OPM1.171019.019/V10.0.5.0.OCHCNFH:user/release-keys

手机是否被root

// 1就是被root了，是否root可能会对kernel config有影响
$ getprop ro.debuggable

版本信息

// 稳定版V10.0.5.0.OCHCNFH，android 8.1.0
Build: OPM1.171019.019
Build fingerprint:

// 内核版本
Kernel: Linux version 4.4.78-perf-gdf56d47 (builder@c4-miui-ota-bd09.bj) (gcc version 4.9.x 20150123 (prerelease) (GCC) ) #1 SMP PREEMPT Tue Oct 23 17:09:55 CST 2018

硬件信息

/proc/hwinfo
/sys/hwconf/hw_info

log分布

// 上次log
------ LAST KMSG (/sys/fs/pstore/console-ramoops-0) ------
------ LAST LOGCAT (logcat -L -b all -v threadtime -v printable -v uid -d *:v) ------
// logcat
------ KERNEL LOG (dmesg) ------
------ SYSTEM LOG (logcat -v threadtime -v printable -v uid -d *:v) ------
------ EVENT LOG (logcat -b events -v threadtime -v printable -v uid -d *:v) ------
------ LOG STATISTICS (logcat -b all -S) ------
// Property
SYSTEM PROPERTIES

查看处于聚焦状态的 Activity
关键字am_focused_activity

0-01 18:10:41.409 4600 14112 I am_focused_activity: [0,com.google.android.GoogleCamera/com.android.camera.CameraActivity]

tombstones

存在：
------ TOMBSTONE (/data/tombstones/tombstone_01: 2016-06-18 11:16:18) ------
不存在：
*** NO TOMBSTONES to dump in /data/tombstones

电源键按下

PowerManagerService: Waking up from Dozing (uid=1000, reason=WAKE_REASON_POWER_BUTTO

锁屏触屏

KeyguardIndication: updateIndication: mVisible true …上滑解锁

解锁：

KeyguardDisplayManager: hide
miui_face: face unlock success

内存问题

整体内存分布

关注总内存大小，2年前的旗舰机型可能配置是3G运行内存，放到现在已经算低内存配置了，不同内存大小看问题的标准也会不一样。

关注是否有异常占用，比如slab是否很大，这种一般root后有SLUB DEBUG会比较大，其他出问题的可能性比较小。

关注MemFree、Cached大小，从kernel角度看看可用内存的情况。

关注Mlocked（pinservice锁定），kernel vm是否耗尽。

关注swap大小以及使用情况，如果zram已经耗尽，那么系统就少了一种回收内存的手段，此时只能依靠杀进程去释放内存。另外，zram压缩/解压缩会对性能有一定影响，因此太多page在zram中虽然可以增加后台进程存活率，但是性能会有损失。
如果SwapTotal为0, 说明zram没有开. 一般来说设置zram大小: 3G内存 1G zram空间 ; 4G内存 2G zram空间

看当前内存剩余可使用量. 关键字 MemAvailable，经验值: MemAvailable <= 总内存的10% 证明内存比较紧张了.

------ MEMORY INFO (/proc/meminfo) ------
MemTotal: 2775056 kB // 伙伴系统管理的全部内存
…
Cached: 790832 kB
SwapCached: 6836 kB
Mlocked: 83072 kB
SwapTotal: 1048572 kB
SwapFree: 325600 kB
…
Slab: 241520 kB
SReclaimable: 90596 kB
SUnreclaim: 150924 kB
…
VmallocTotal: 258998208 kB
VmallocUsed: 327660 kB
VmallocChunk: 258581768 kB

内存碎片

------ PAGETYPEINFO (/proc/pagetypeinfo)------
Number of blocks type Unmovable Movable Reclaimable CMA HighAtomic Isolate
Node 0, zone DMA 167 239 8 43 0 0
Node 0, zone Normal 228 266 11 0 1 0
关键字: Unmovable 经验值:Unmovable 超过总数的20%,可能存在内存碎片.

进程内存占用

dumpsys meminfo会打印出系统所有进程的内存占用。
这里关注有没有进程内存占用太大，特别是adj比较高的进程是否占用比较大，可以通过 “MEMINFO in pid” 进一步找到这个进程的具体内存占用。
关注Lost RAM 有没有异常，是否大于500M，高优先级的进程数量太多。
Cached是不是基本被kill完，B service / cached 是不是太多，进程在swap中的内存的大小。
kernel占用是否过大（远大于500M）

Total RAM: 7,799,676K (status moderate)
Free RAM: 602,564K ( 0K cached pss + 277,692K cached kernel + 4K cached ion + 324,868K free)
ION: 172,304K ( 2,768K mapped + 169,536K unmapped + 0K pools)
Used RAM: 8,430,850K (2,685,326K used pss + 5,745,524K kernel)
Lost RAM: 637,683K
ZRAM: 506,708K physical used for 1,841,556K in swap (4,145,148K total swap)
Tuning: 256 (large 512), oom 1,451,520K, restore limit 107,520K (high-end-gfx)
--------- 9.236s was the duration of dumpsys meminfo, ending at: 2020-07-27 14:21:33
Shmem: 4232 kB
KReclaimable: 135196 kB
Slab: 5398236 kB // kernel数据结构的缓存大小，Slab = SReclaimable+SUnreclaim
SReclaimable: 135196 kB // 可回收的slab大小
SUnreclaim: 5263040 kB //不可回收的slab大小
KernelStack: 52544 kB

每个adj级别的进程占用内存信息:Total PSS by OOM adjustment:

Total PSS by OOM adjustment:
301,773K: Native ( 198,816K in swap)
…
364K: daemon (pid 11049) ( 240K in swap)
535,998K: Persistent ( 154,814K in swap)
228,965K: com.android.systemui (pid 2048 / activities) ( 83,156K in swap)
…
21,466K: Persistent Service ( 259K in swap)
21,466K: com.android.incallui (pid 7312) ( 259K in swap)
73,923K: Foreground ( 8,507K in swap)
34,254K: com.android.contacts (pid 7229 / activities) ( 229K in swap)
…
87,647K: Visible ( 43,326K in swap)
45,051K: com.sohu.inputmethod.sogou.xiaomi (pid 24060) ( 24,804K in swap)
…
424,717K: Perceptible ( 177,153K in swap)
132,781K: com.market2345 (pid 22132 / activities) ( 62,455K in swap)
…
109,014K: A Services ( 11,188K in swap)
26,796K: com.cleanmaster.security_cn:DefendService (pid 5581) ( 484K in swap)
…
98,657K: Home ( 58,125K in swap)
98,657K: com.xxx.home (pid 7125 / activities) ( 58,125K in swap)
39,406K: Previous ( 525K in swap)
39,406K: com.greenpoint.android.mc10086.activity:plugin (pid 7192) ( 525K in swap)
206,405K: B Services ( 40,929K in swap)
68,480K: com.greenpoint.android.mc10086.activity (pid 24919 / activities) ( 28,068K in swap)
…
3,730K: Cached ( 294K in swap)
3,730K: com.android.server.telecom:ui (pid 7299) ( 294K in swap)
…

各个进程内存占用情况

Total PSS by process:
306,260K: system (pid 1824) ( 44,784K in swap)
235,115K: com.wuba.town.client (pid 9986 / activities) ( 480K in swap)
183,849K: com.tencent.mm (pid 3067 / activities) ( 82,358K in swap)
168,236K: com.miui.home (pid 21665 / activities) ( 64,972K in swap)
153,955K: com.android.contacts (pid 11741 / activities) ( 302K in swap

ams根据cache进程数量判断内存状态，这个字段要辩证看待，其受到lmk水线的影响

Total RAM: 2,775,056K (status critical) ---------------手机总可用内存
Free RAM: 893,590K ( 3,730K cached pss + 747,316K cached kernel + 0K cached ion + 142,544K free)
Used RAM: 2,510,290K (1,899,006K used pss + 611,284K kernel) ---------主要使用内存
Lost RAM: -45,060K ----------丢失内存
ZRAM: 208,620K physical used for 785,324K in swap (1,048,572K total swap)
Tuning: 256 (large 512), oom 322,560K, restore limit 107,520K (high-end-gfx)

ION内存使用

系统很多进程，比如camera provider，会使用ion获取内存。此时如果ion内存泄漏，在pss中无法体现出来，需要在这里看下有没有ION占用异常的进程。

DUMP OF SERVICE perfshielder:
---- ION Memory Usage ----
total 54972416 // ION使用的内存大小，单位是字节。
pool total (uncached + cached + secure) = 233472 // ION driver从系统中申请的未被client使用的内存。

内存状态关键字

关注page allocation failure，表示内存分配失败了

[101039.767967] ui.cloudservice: page allocation failure: order:2, mode:0x3000d0

lmk杀进程的日志，当lmk开始杀进程的时候，表明内存分配走进了slow path，触发了lmk shrinker，且计算出来的剩余内存小于当前配置的minfree。
此时lmk就会根据配置的minfree和adj选择合适的进程去查杀。

lowmemorykiller: Kill’ com.google.process.gservices’ (16009), uid 10179, oom_adj 985 to free 67992kB
lmkd 594 594 I lowmemorykiller: Reclaimed 67992kB, cache(598776kB) and free(122512kB)-reserved(124800kB) below min(663552kB) for oom_adj 900
lmkd 594 594 I lowmemorykiller: Suppressed 5 failed kill reports

lmk的水线配置

实时水线sys.lmk.minfree_levels
[sys.lmk.minfree_levels]: [18432:0,23040:100,27648:200,85000:250,191250:900,241920:950]

kernel OOM

<3>[115608.725418] Out of memory: Kill process 7487 (id.AlipayGphone) score 946 or sacrifice child

Unmovalbe page block type是不是占比过大

------ PAGETYPEINFO (/proc/pagetypeinfo) ------
------ FRAGMENTATION INFO (/d/extfrag/unusable_index) ------

fork 耗时

ActivityManager: Slow operation: 2158ms so far, now at startProcess: returned from zygote!

lmk把cached都kill完才会打印这个，后面的88表示java进程的数量

11-05 22:12:25.270 2562 4121 I am_low_memory: 88

触发了GC，这种一般是app内部的。

11-03 21:12:19.037 10178 11894 11905 I zygote : Background concurrent copying GC freed 24079(2MB) AllocSpace objects, 21(356KB) LOS objects, 50% free, 2MB/5MB, paused 292us total 121.478ms
10-27 22:52:35.796 1500 1509 I art : Background partial concurrent mark sweep GC freed 73752(4MB) AllocSpace objects, 50(1760KB) LOS objects, 22% free, 55MB/71MB, paused 1.040ms total 115.179ms

// 这里是进程vma耗尽？
08-30 14:38:47.874 1040 896 7782 E IMemory : cannot map BpMemoryHeap (binder=0x725e61ac80), size=65536, fd=27 (Out of memory)

进程

进程启动和死亡相关log

进程创建
AMS调用ProcessList的startProcessLocked去真正启动进程，在该方法中

先去通过调用链添加当前的uid 到mActiveUids
调用链：
ProcessList.startProcessLocked → newProcessRecordLocked → addProcessNameLocked → EventLogTags.writeAmUidRunning
打印：
09-15 20:20:47.199 6135 6164 I am_uid_running: 10147

接着将进程启动post到异步线程去处理
调用链：
ProcessList.startProcessLocked → startProcessLocked→ startProcessLocked → startProcessLocked → mProcStartHandler.post
异步线程中先去通过系列调用到Zygote去创建进程
调用链：
mProcStartHandler.post → ProcessList.startProcess → Process.start → ZygoteProcess.start
进程创建成功后，接着通过系列调用先后打印am_proc_start 和 Start proc
调用链：
mProcStartHandler.post → ProcessList.handleProcessStartedLocked → handleProcessStartedLocked → ActivityManagerService.reportUidInfoMessageLocked
打印：
09-15 20:20:47.270 6135 6165 I am_proc_start: [0,3190,10147,XXX,service,{XXXXService}]
09-15 20:20:47.272 I/ActivityManager( 6135): Start proc 3190:XXX/u0a147 for service {XXXService} caller=XXX

Zygote创建进程并在ActivityThread执行attach方法中bind AMS 执行attachApplicationLocked方法
调用链：
Binder → ActivityManagerService.attachApplication → attachApplicationLocked
打印：
09-15 20:20:47.296 6135 6952 I am_proc_bound: [0,3190,XXX]
该方法中先打印am_proc_bound，随后binder App执行 bindApplication
然后通过调用链更新oomAdj 打印am_uid_active
调用链：
ActivityManagerService.attachApplication → updateOomAdjLocked → OomAdjuster.updateOomAdjLocked → EventLogTags.writeAmUidActive
打印：09-15 20:20:47.319 6135 6952 I am_uid_active: 10147

进程死亡回调
AMS注册了AppDeathRecipient，App死亡后会回调到binderDied方法走如下流程
调用链：
AppDeathRecipient.binderDied → ActivityManagerService.appDiedLocked → handleAppDiedLocked
打印：
在appDiedLocked会打印am_proc_died
09-16 11:25:32.794 6135 7070 I am_proc_died: [0,30634,XXX,905,19]

调用cleanUpApplicationRecordLocked
清除Service、Provider、Receiver相关
通过调用链从mActiveUids移除当前uid相关record
调用链：
AMS.cleanUpApplicationRecordLocked → ProcessList.removeProcessNameLocked → EventLogTags.writeAmUidStopped
打印：
09-16 11:25:32.851 6135 7070 I am_uid_stopped: 10069

通过removeLruProcessLocked 从Lru列表移除当前ProcessRecord

Activity相关处理
调用链：
ATMS.LocalService.handleAppDied → RootActivityContainer.handleAppDied RootActivityContainer -> ActivityStack.handleAppDiedLocked

常见的进程启动原因

hostingType=restart
在cleanUpApplicationRecordLocked过程中如果发现app有正在launching的provider则需要restart 进程，persistent进程也需要重启
打印：
system_process D/Boost: hostingType=restart, hostingName=com.ct.client, callerPackage=com.qixiao.qrxs, isSystem=false, isBoostNeeded=false.
system_process I/ActivityManager: Start proc 19129:com.ct.client/u0a217 for restart com.ct.client caller=com.qixiao.qrxs

策略FastRestart
打印：
system_process I/ActivityManager: Start proc 16611:com.android.camera/u0a92 for FastRestart com.android.camera caller=system

进程被kill后立刻被拉起

获取ContenProvider时
打印：
ActivityManager: ProcessRecord{fdd24a5 25575:XXX/u0a103} was killed by AM but isn’t really dead
.ActivityManagerService.getContentProviderImpl

设置新进程的precedence为老进程实例
打印：
ActivityManager: ProcessRecord{fdd24a5 25575:XXX/u0a103} is attached to a previous process

打印：
ActivityManager: ProcessRecord{fdd24a5 25575:XXX/u0a103} refused to die, but we need to launch ProcessRecord{3a8ae62 0:com.miui.gallery/u0a103}

进程被杀原因

先找am_kill
am_kill : [0,7442,com.android.browser,985,empty #25]
AMS执行的杀进程，最后一个参数代表kill reason，比如 stop 为调用forceStopPackageLocked而被杀

然后看am_proc_died
am_proc_died: [0,5782,com.google.android.gms.persistent,0,5]

通过killinfo查看详细信息
killinfo (Pid|1|5),(Uid|1|5),(OomAdj|1),(MinOomAdj|1),(TaskSize|1),(enum kill_reasons|1|5),(MemFree|1),(Cached|1),(SwapCached|1),(Buffers|1),(Shmem|1),(Unevictable|1),(SwapTotal|1),(SwapFree|1),(ActiveAnon|1),(InactiveAnon|1),(ActiveFile|1),(InactiveFile|1),(SReclaimable|1),(SUnreclaim|1),(KernelStack|1),(PageTables|1),(IonHeap|1),(IonHeapPool|1),(CmaFree|1)
为kmkd的杀进程
lmkd 594 594 I killinfo: [9967,1000,100,100,67456,-1,5396572,33728,221272,41660,396,15148,117864,2621436,1691212,708540,271688,23916,67180,79976,308952,55944,84416,0,0,3268]

查看进程被杀原因

确定被杀时间点
01-29 17:37:24.776 1000 1824 7769 I ActivityManager: Process com.wuba.town.client (pid 7469) has died: vis SVC
01-29 17:37:24.777 1000 1824 7769 I am_proc_died: [0,7469,com.wuba.town.client,100,10]
01-29 17:37:24.802 1000 1824 7769 I am_low_memory: 66
代表：am_low_memory 后面的数字代表当前系统中lru 中进程的数量，数字递减代表有进程被杀，has died 代表非AMS查杀（lmk查杀或信号查杀），hvy 代表进程的优先级，HVY(后台进程，但无法执行restore)，SVC(后台进程且正在运行Service)
详情查看：Android OomAdj

在system log（logcat -v 下面）或event log（logcat -b events下面）搜索时间点附近对应pid的log
或者看ApplicationExitInfo
ApplicationExitInfo #1:
timestamp=2021-01-29 17:37:24.799
pid=7469
realUid=10287
packageUid=10287
definingUid=10287
user=0
process=com.wuba.town.client
reason=2 (SIGNALED)
status=9
importance=300
pss=291MB
rss=411MB
description=null
state=empty
trace=null

正常查杀empty进程
正常现象
08-10 13:32:13.057 1000 1700 1812 I am_kill : [0,2409,com.xxxx,955,empty for 1800s]
empty进程数量达到阈值了，会查杀30min内没活跃的进程
08-10 16:00:54.244 1000 1700 1795 I ActivityManager: Killing 20081:com.android.providers.calendar/u0a72 (adj 985): empty #26
系统内empty进程数量达到阈值26（不同手机略有不同）会按时间顺序查杀进程

客户端调用startInstrumentation，会先forceStopPackage再拉起，可能会查杀到相关的其他进程
App的问题
08-03 15:10:02.933 1000 1303 12534 I ActivityManager: Force stopping com.eg.android.AlipayGphone appid=10198 user=0: start instr
08-03 15:10:02.933 1000 1303 12534 I ActivityManager: Killing 1441:com.kugou.android/u0a219 (adj 0): stop com.eg.android.AlipayGphone: start instr

进程自己正常退出
App的问题
8-13 10:17:04.053 root 689 689 I Zygote : Process 29263 exited cleanly (0)
08-13 10:17:04.056 1000 1692 1805 I libprocessgroup: Successfully killed process cgroup uid 10252 pid 29263 in 122ms
08-13 10:17:03.933 1000 1692 5189 I ActivityManager: Process com.tencent.tmgp.pubgmhd (pid 29263) has died: hvy HVY

idle maint 查杀
设备处于两次idle之间低内存状态持续时间超1/3,会查杀内存增长较高的进程，原生策略问题
08-14 09:43:55.617 1000 1477 1779 I am_wtf : [0,1477,system_server,-1,ActivityManager,Killcom.tencent.mm in idle maint: pss=365677, swapPss=174343, initialPss=173858, period=+1h17m4s218ms, lowRamPeriod=+1h8m19s250ms]
08-14 09:43:55.625 1000 1477 1477 I am_kill : [0,17579,com.byai.crm,100,idle maint (pss 165820 from 95133)]
08-14 09:43:55.772 1041 664 31875 I [30200] : 17579&10168&0&-1
08-14 09:43:55.724 1000 1477 1514 I am_proc_died: [0,17579,com.byai.crm,100,4]

一键清理或者相机boost等
08-11 15:14:37.873 1000 1737 3412 I ActivityManager: Killing 5831:com.android.camera/u0a74 (adj 905): OneKeyClean
01-29 17:37:21.321 1000 1824 4813 I ActivityManager: Killing 9291:com.miui.securitycore.remote/1000 (adj 925): camera boost

installPackageLI/ deletePackageX等
只要是forceStopPackage接口查杀就有可能查杀到关联的进程，如下拨号就是因为与gms关联被杀
08-10 15:59:30.827 1000 1700 1812 I ActivityManager: Force stopping com.google.android.gms appid=10190 user=-1: installPackageLI
08-10 15:59:30.984 1000 1700 1812 I ActivityManager: Killing 3876:com.google.android.dialer/u0a180 (adj -700): stop com.google.android.gms: installPackageLI

过度使用CPU
AMS的checkExcessivePowerUsageLocked，系统启动后，每隔5分钟调用一次，检测prcState大于PROCESS_STATE_HOME的应用，CPU使用5分钟超过25%,15分钟超过10%,大于15分钟2%,则杀死应用
09-06 18:07:44.445 1000 1651 1798 I am_kill : [0,22157,com.ximalaya.ting.android,900,excessive cpu 16720 during 300044 dur=2221553 limit=2]
09-06 18:07:44.445 1000 1651 1798 I ActivityManager: Killing 22157:com.ximalaya.ting.android/u0a239 (adj 900): excessive cpu 16720 during 300044 dur=2221553 limit=2
09-06 18:07:44.566 root 681 681 I Zygote : Process 22157 exited due to signal 9 (Killed)
上次adj降为Service以下到现在经历2221553 ms，cpu POWER_CHECK_MAX_CPU_4 = 2, 两次检测时间间隔为300044 ms，使用cpu时间为16720ms，超过2%

进程的某个线程发生异常，自己发Signal 9信号给Zygote 杀掉自己
09-07 16:53:03.027 1000 2566 2582 I Process : Sending signal. PID: 2566 SIG: 9
09-07 16:53:03.154 root 681 681 I Zygote : Process 2566 exited due to signal 9 (Killed)
09-07 16:53:03.167 1000 1734 4006 I ActivityManager: Process com.android.systemui (pid 2566) has died: pers PER

dumpsys ProcessManager里面的LO为加锁应用
LO:
userId=0
com.ximalaya.ting.lite
com.cisco.anyconnect.vpn.android.avf
com.tencent.mm
com.tencent.qqmusic
com.tencent.mobileqq
com.v2ray.ang
com.benjamin.intime
com.ss.android.lark.kami
com.youku.phone
userId=-100
com.jeejen.family.miui

am_kill

am_kill (User|1|5),(PID|1|5),(Process Name|3),(OomAdj|1|5),(Reason|3)

异常杀进程
anr
crash
start timeout

主动杀进程
stop user userId
kill background
Permission related app op changed

调度杀进程
empty 为调用trimApplications
remove task 为调用applyOomAdjLocked
cached #numCached 为调用updateOomAdjLocked
empty #numEmpty 为调用updateOomAdjLocked
empty for tableimes 为调用updateOomAdjLocked
isolated not needed 为调用updateOomAdjLocked

其他杀进程
remove task
error during init 为调用 attachApplicationLocked
system update done 为调用 systemReady

进程提升优先级的原因

首先搜索dumpsys activity processes找到Process LRU list找到对应进程名如下：
打印：
Proc # 6: hvy B/ /HVY trm: 0 16885:com.qiyi.video/u0a209 (cch-act)
解释：

hvy 和 HVY 分别代表adj和procState
B代表schedGroup，这里为SCHED_GROUP_BACKGROUND，
0代表trimMemoryLevel 为0
cch-act代表adjType

主要实现在ActivityManagerService的dumpProcessOomList中

最近任务加锁
打印：
Proc # 6: hvy B/ /HVY trm: 0 16885:com.qiyi.video/u0a209 (cch-act)
解释：
hvy即HEAVY_WEIGHT_APP_ADJ=400 ，这个adj的值比较特殊，MM一些策略以及最近任务加锁会设置进程的maxAdj为这个值
这个时候需要dumpsys ProcessManager 找到LO （加锁）查看对应列表是否有这个进程名
Process Policy:
LO:
userId=0
com.qiyi.video

进程adj > Service的由system拉起的client进程,且没有其他flag等，adj会为VISIBLE_APP_ADJ，adjType会为"service"
一般涉及到adjTarget基本跟Service和ContentProvider相关
打印：
Proc #12: vis F/S/FGS trm: 0 19817:com.xxx.wearable/u0a495 (service)
com.xx.wearable/.home.devices.ble.sync.NotifySyncService<=Proc{1705:system/1000}
解释：
vis = 100 代表可见进程，F = SCHED_GROUP_DEFAULT ，S代表该进程有前台Service，FGS = 4代表procState，adjtype为"service"
adjTarget为com.xxx.wearable/.home.devices.ble.sync.NotifySyncService ，adjSource为Proc{1705:system/1000}

MM设置进程maxAdj
具体逻辑ProcessPolicy的sAppProtectMap
打印：
Proc # 6: vis R/ /TOP trm: 0 20886:com.xxx.xxx/1000 (cch-started-ui-services)
解释：
vis=100 代表可见进程，R=SCHED_GROUP_RESTRICTED，TOP=2,代表当前进程拥有Top Activity，但是这些跟adjType="cch-started-ui-services"不匹配，所以被提升优先级了

UI进程与Service进程一定要分离，因为对于包含activity的service进程，一旦进入后台就成为”cch-started-ui-services”类型的cache进程(ADJ>=900)，随时可能会被系统回收；而分离后的Service进程服务属于SERVICE_ADJ(500)，被杀的可能性相对较小。尤其是系统允许自启动的服务进程必须做UI分离，避免消耗系统较大内存。

调试命令

直接使用dumpsys activity oom
dumpsys activity -p com.sina.weibo oom
查看所有进程提升优先级的原因
主要实现在ActivityManagerService的dumpOomLocked

调用dumpsys activity package [包名]
查看应用的adj及应用运行的activity、服务、providers及广播

命令行动态看adj变化
adb shell am watch-uids [ --oom < uid > ]
实现在ActivityManagerShellCommand.java中
是通过在OomAdjuster.reportOomAdjMessageLocked中记录内容输出的

dumpsys ProcessManager
查看加锁应用LO：
查看应用被杀的原因
ProcessConfig{mPolicy=7, mReason=‘null’, mKillingPackage=‘com.android.contacts’, mUserId=0, mTaskId=749, mUid=-1, mPriority=-10000, mWhiteList=null, mKillingPackageMaps=null, mRemoveTaskNeeded=true, mRemovingTaskIdList=null, mkillingClockTime=2021-03-04 15:52:18}

进程反复重启
进程反复启动与被杀，这种也可以从侧面说明内存状态比较差。同时进程反复重启也会造成比较严重的性能损失。

am_proc_start (User|1|5),(PID|1|5),(UID|1|5),(Process Name|3),(Type|3),(Component|3)
am_proc_died (User|1|5),(PID|1|5),(Process Name|3),(OomAdj|1|5),(ProcState|1|5)

10-01 18:07:06.494 4600 9696 I am_proc_died: [0,20074,com.android.musicfx]
10-01 18:07:06.555 4600 6606 I am_proc_died: [0,31166,com.concur.breeze]
10-01 18:07:06.566 4600 14112 I am_proc_died: [0,18812,com.google.android.apps.fitness]
10-01 18:07:07.018 4600 7513 I am_proc_start: [0,20361,10113,com.sony.playmemories.mobile,broadcast,com.sony.playmemories.mobile/.service.StartupReceiver]
10-01 18:07:07.357 4600 4614 I am_proc_start: [0,20381,10056,com.google.android.talk,service,com.google.android.talk/com.google.android.libraries.hangouts.video.CallService]
10-01 18:07:07.784 4600 4612 I am_proc_start: [0,20402,10190,com.andcreate.app.trafficmonitor:loopback_measure_serivce,service,com.andcreate.app.trafficmonitor/.loopback.LoopbackMeasureService]
10-01 18:07:10.753 4600 5997 I am_proc_start: [0,20450,10097,com.amazon.mShop.android.shopping,broadcast,com.amazon.mShop.android.shopping/com.amazon.identity.auth.device.storage.LambortishClock$ChangeTimestampsBroadcastReceiver]
10-01 18:07:15.267 4600 6605 I am_proc_start: [0,20539,10173,com.google.android.apps.fitness,service,com.google.android.apps.fitness/.api.services.ActivityUpsamplingService]
10-01 18:07:15.985 4600 4612 I am_proc_start: [0,20568,10022,com.android.musicfx,broadcast,com.android.musicfx/.ControlPanelReceiver]
10-01 18:07:16.315 4600 7512 I am_proc_died: [0,20096,com.google.android.GoogleCamera]

进程信息

里面有进程的优先级，VMA等

------ PROCESSES AND THREADS (ps -A -T -Z -O pri,nice,rtprio,sched,pcy) ------
LABEL USER PID TID PPID VSZ RSS WCHAN ADDR S PRI NI RTPRIO SCH PCY CMD
u:r:init:s0 root 1 1 0 20120 2060 0 0 S 19 0 - 0 fg init

进程运行时间，iowait时间

------ PROCESS TIMES (pid cmd user system iowait+percentage) ------

各个进程的trace

------ VM TRACES JUST NOW (/data/anr/dumptrace_CpBqkf: 2018-11-04 20:33:22) ------

最后一次ANR时各个进程的trace

------ VM TRACES AT LAST ANR (/data/anr/anr_2018-11-01-11-47-33-769: 2018-11-01 11:47:37) ------

CPU

负载

1, 5, and 15 minute load averages

------ UPTIME (uptime) ------
11:45:32 up 3:05, 0 users, load average: 8.19, 6.80, 8.16
------ DUMPSYS CPUINFO (/system/bin/dumpsys -t 10 cpuinfo -a) ------
Load: 9.15 / 10.17 / 10.47

进程的cpu占用率，留意后面的时间区间.CPU usage

CPU usage from 166158ms to 1082ms ago (2018-11-04 20:30:20.109 to 2018-11-04 20:33:05.185):
144% 1808/system_server: 129% user + 15% kernel / faults: 725714 minor 139 major

从cpu usage中看看，kswapd的占用率高，表示内存分配需求大，或者分配困难
kswapd0 占用过高是因为物理内存不足，频繁开始使用swap交换，导致CPU占用过高,证明当前内存不足。

0.1% 154/kswapd0: 0% user + 0.1% kernel
0% 90/kcompactd0: 0% user + 0% kernel

线程的cpu占用率(抓bugreport的时候)，里面还有 iow irq等信息

------ CPU INFO (top -b -n 1 -H -s 6 -o pid,tid,user,pr,ni,%cpu,s,virt,res,pcy,cmd,name) ------
Tasks: 4210 total, 8 running,4195 sleeping, 0 stopped, 0 zombie
Mem: 5849744k total, 5767096k used, 82648k free, 204112k buffers
Swap: 2621436k total, 293636k used, 2327800k free, 2065772k cached
800%cpu 181%user 90%nice 127%sys 362%idle 32%iow 6%irq 1%sirq 0%host
PID TID USER PR NI[%CPU]S VIRT RES PCY CMD NAME
1808 1808 system 18 -2 175 S 4.5G 435M ta system_server system_server

如果cpu占用率高，可以从 vm trace中看对应线程在做什么

------ VM TRACES JUST NOW (/data/anr/dumptrace_CpBqkf: 2018-11-04 20:33:22) ------
----- pid 1808 at 2018-11-04 20:33:08 -----

CPU降频及拔核引起卡顿
thermal engine 会打印cpu online/offline, cpu温度过高降频等信息

电池温度过高
ThermalEngine: [thermal-debug] battery temp is 427 高于40度算是比较高了
温控设置cpu频率
thermald: thermal_cpu_set_level: set cpu freq of cpu4 to 2112000
针对Camera的温控策略log:
thermald: [CAMERA

设备温度信息
thermal log以单独文件存放，位于bugreport压缩文件中，文件名为dumpstate_board.txt，其中VIRTUAL-SENSOR代表温度信息
[SS-CPU7][VIRTUAL-SENSOR0 48019][cpu7 844800]
[MONITOR-TEMP_STATE][VIRTUAL-SENSOR0 48019]
CPU7温度超过42度开始限频到1.6G左右

thermal控制的代码位置：device/XXX/机型i/thermal_config/thermal-normal.conf_modified
[SS-CPU7] //控制的设备
algo_type ss //使用的算法
sensor VIRTUAL-SENSOR0 //监听的sensor名称
device cpu7 //控制的设备
polling 1000 //轮寻周期
trig 39000 41000 43000 45000 48000 //触发调控的温度
clr 37000 39000 41000 43000 45000 //回复调控的温度
target 2496000 1785600 1420800 1075200 844800
Thermal策略经常用主板冷区的温度或者结合其他温度进行拔核降频的调节

电量过低时会进行CPU拔核
thermald: device hotplug_cpu4+hotplug_cpu5+hotplug_cpu6+hotplug_cpu7
thermald: hotplug_set_level: hotplug_cpu5 set cpu off
thermald: hotplug_set_level: hotplug_cpu6 set cpu off

低电量模式确定的log:
可以搜索关键字low power mode或lpm

充电状态
healthd : battery l=5 v=3611 t=31.0 h=2 st=3 c=676 fc=4502000 cc=3 tl=0 chg=
st为2是充电，为3是放电；l为当前电量5%；fc为电池容量4502000

查看是否正在充电

healthd : battery l=5 v=3611 t=31.0 h=2 st=3 c=676 fc=4502000 cc=3 tl=0 chg=
st为2是充电，为3是放电；l为当前电量5%；fc为电池容量4502000

IO相关

查看 data分区是否满

57% TOTAL: 43% user + 10% kernel + 1.7% iowait + 0.7% irq + 0.6% softirq
------ PROCESS TIMES (pid cmd user system iowait+percentage) ------
1 /init 157.38 67.03 0.00
------ FILESYSTEMS & FREE SPACE (df) ------
/dev/block/dm-2 113484780 50668592 62685116 45% /data

查看分区挂载选项，比如是否打开 discard 对IO性能有影响

------ VOLD DUMP (vdc dump) ------
0 13575 /dev/block/dm-2 /data f2fs rw,lazytime,seclabel,nosuid,nodev,noatime,background_gc=on,discard,no_heap,user_xattr,inline_xattr,acl,inline_data,inline_dentry,
flush_merge,nobarrier,extent_cache,mode=adaptive,active_logs=6,reserve_root=32768,resuid=0,resgid=1065,alloc_mode=default,fsync_mode=posix 0 0

ANR

VM TRACES AT LAST ANR
若存在该ANR则输出相应traces，否则输出
*** NO ANR VM TRACES FILE (/data/anr/traces.txt): No such file or directory

11-19 10:57:25.597 1000 1564 1582 E ActivityManager: ANR in com.facebook.katana
11-19 10:57:25.597 1000 1564 1582 E ActivityManager: PID: 28557
11-19 10:57:25.597 1000 1564 1582 E ActivityManager: Reason: executing service com.facebook.katana/com.facebook.fbservice.service.BlueService

已分配堆内存大小40MB，其中29M已用，总分配207772个对象 [见小节3.4]

Heap: 27% free, 29MB/40MB; 307772 objects

主线程调用栈
schedstat: CPU调度时间统计(Running、Runable、Switch),单位ns,Running=(utm + stm)10ms
utm/stm: 用户态/内核态的CPU时间,单位是jiffies，默认等于10ms,4210=420ms

“main” prio=5 tid=1 Native
| group=“main” sCount=1 dsCount=0 obj=0x75bd9fb0 self=0x5573d4f770
| sysTid=12078 nice=-2 cgrp=default sched=0/0 handle=0x7fa75fafe8
| state=S schedstat=( 5907843636 827600677 5112 ) utm=453 stm=137 core=0 HZ=100
| stack=0x7fd64ef000-0x7fd64f1000 stackSize=8MB

Framework相关

等jvm锁

dvm_lock_sample: [system_server,1,Binder:2015_18,4322,PackageManagerService.java,4407,
int com.android.server.pm.PackageManagerService.getPackageUid(java.lang.String, int, int),-,23335,void com.android
.server.pm.PackageManagerService.dump(java.io.FileDescriptor, java.io.PrintWriter, java.lang.String[]),0]

位置	内容	含义
1	system_server	出现锁竞争的线程名
2	1	是否为 sensitive 线程
3	Binder:2015_18	等锁线程名
4	4322	等锁时间
5	PackageManagerService.java	等锁线程的等锁位置: 源码文件名
6	4407	等锁线程的等锁位置: 源码文件行号
7	int com.android.server.pm.PackageManagerService.getPackageUid(java.lang.String, int, int)	等锁线程的等锁位置: 方法
8	-	持锁线程持锁位置: 源码文件名 (- 表示与等锁源码文件名相同)
9	23335	持锁线程持锁位置: 源码文件行号
10	void com.android.server.pm.PackageManagerService.dump(java.io.FileDescriptor, java.io.PrintWriter, java.lang.String[])	持锁线程持锁位置: 方法
11	0	锁等待的百分比

丢帧

11-01 22:14:09.956 10104 7584 7584 I [30089] : 106
Service：

Android Framework Services 这些service的dump信息

DUMP OF SERVICE SurfaceFlinger
DUMP OF SERVICE activity
Activity：

启动一个应用的关键log，可以通过搜索 am_on_resume_called（Q）或者wm_on_resume_called（Q之后）来判断界面的变化

wm_on_top_resumed_lost_called: [194023434,com.miui.home.launcher.Launcher,topStateChangedWhenResumed]
wm_on_paused_called: [0,XXXLauncher,performPause,0]
wm_on_create_called: [0,com.upin.app.mvp.start.activity.FlashActivity,performCreate,81]
wm_on_start_called: [0,com.upin.app.mvp.start.activity.StartActivity,handleStartActivity,1]
wm_on_resume_called: [0,com.upin.app.mvp.start.activity.StartActivity,RESUME_ACTIVITY,1]
wm_on_top_resumed_gained_called: [45091175,com.upin.app.mvp.start.activity.StartActivity,topStateChangedWhenResumed]
wm_on_stop_called: [0,XXXLauncher,STOP_ACTIVITY_ITEM,2]

卡顿

11-08 08:01:01.724 5931 5931 I Choreographer: Skipped 129 frames! The application may be doing too much work on its main thread.

后台dex2oat会加重cpu loading。后台编译一般发生在app自动更新，或者app运行了一些未经过编译的dex的时候。

09-14 07:29:20.433 15736 15736 I dex2oat : /system/bin/dex2oat -j4 --dex-file=/data/user/0/com.facebook.katana/app_secondary_program_dex/program-72cef82b591768306676e10161c886b58b34315a308602be.dex.jar --oat-file=/data/user/0/com.facebook.katana/app_secondary_program_dex_opt/program-72cef82b591768306676e10161c886b58b34315a308602be.dex.dex
…
09-14 07:29:25.102 15736 15736 I dex2oat : dex2oat took 4.669s (threads: 4) arena alloc=7MB java alloc=3MB native alloc=29MB free=4MB

反馈中含有“卡机”，“卡死”，“重启”等字眼，则可以先扫一眼看看有没有anr，system server crash或watchdog kill system

如果是system_server关键持锁超时被watchdog杀了，会有如下日志
11-01 17:37:09.018 1000 2026 2332 W Watchdog: *** WATCHDOG KILLING SYSTEM PROCESS: Blocked in monitor com.android.server.am.ActivityManagerService on foreground thread (android.fg), Blocked in handler on main thread (main), Blocked in handler on ui thread (android.ui), Blocked in handler on display thread (android.display), Blocked in handler on ActivityManager (ActivityManager), Blocked in handler on PowerManagerService (PowerManagerService)

如果system_server crash重启，会有日志：
AndroidRuntime: *** FATAL EXCEPTION IN SYSTEM PROCESS:

binder超时

A.功能说明: 监控每个进程的主线程的binder transaction的耗时情况, 当超过阈值时,则输出相应的目标调用信息，默认1000ms打开。

B.log格式: 52004 binder_sample (descriptor|3),(method_num|1|5),(time|1|3),(blocking_package|3),(sample_percent|1|6)

实例
binder_sample: [android.app.IActivityManager,35,2900,android.process.media,5]

分析

主线程2754;

执行android.app.IActivityManager接口

所对应方法code =35(即STOP_SERVICE_TRANSACTION),

所花费时间为2900ms.

该block所在package为 android.process.media，最后一个参数是sample比例

ActivityThread.H 处理 Message 是否超时, 阈值是 3000ms

am_lifecycle_sample (User|1|5),(Process Name|3),(MessageCode|1|5),(time|1|3)
am_lifecycle_sample: [0,com.facebook.katana,110,3206]

持锁超时dvm_lock_sample

A.功能说明: 当某个线程等待lock的时间blocked超过阈值,则输出当前的持锁状态 ;

B.log格式: 20003 dvm_lock_sample (process|3),(main|1|5),(thread|3),(time|1|3),(file|3),(line|1|5),(ownerfile|3),(ownerline|1|5),(sample_percent|1|6)

C.log实例:
dvm_lock_sample: [system_server,1,Binder_9,1500,ActivityManagerService.java,6403,-,1448,0]

分析
system_server: Binder_9,执行到ActivityManagerService.java的6403行代码,一直在等待AMS锁, 而该锁所同一文件的1448行代码所持有, 从而导致Binder_9线程被阻塞1500ms.

binder starved

A.功能说明: 当system_server等进程的线程池使用完, 无空闲线程时, 则binder通信都处于饥饿状态, 则饥饿状态超过一定阈值则输出信息;

B.云控参数: persist.sys.binder.starvation (默认值16ms)

C.log实例:
1232 1232 “binder thread pool (16 threads) starved for 100 ms”

分析
system_server进程的线程池已满的持续长达100ms

等锁超时日志

waiting to lock <0x01f0446d> 代表ActivityManagerService.getTopAppLocked等obj为0x01f0446d的锁，owner tid为158
locked <0x01f0446d> (a com.android.server.am.ActivityManagerService) 代表等锁挂起时，同时持有obj为0x01f0446d的ActivityManagerService锁这里会锁住AMS

08-16 14:44:09.332  1000  1823  5205 W system_server:   at com.android.server.wm.ActivityTaskManagerService$LocalService.getTopApp(ActivityTaskManagerService.java:7707)
08-16 14:44:09.332  1000  1823  5205 W system_server:   - waiting to lock <0x05c5120b> (a com.android.server.wm.WindowManagerGlobalLock) held by thread 158
08-16 14:44:09.332  1000  1823  5205 W system_server:   at com.android.server.am.ActivityManagerService.getTopAppLocked(ActivityManagerService.java:18784)
08-16 14:44:09.332  1000  1823  5205 W system_server:   at com.android.server.am.OomAdjuster.updateOomAdjLocked(OomAdjuster.java:408)
08-16 14:44:09.332  1000  1823  5205 W system_server:   at com.android.server.am.ActivityManagerService.updateOomAdjLocked(ActivityManagerService.java:19028)
08-16 14:44:09.332  1000  1823  5205 W system_server:   at com.android.server.am.ActivityManagerService.trimApplicationsLocked(ActivityManagerService.java:19284)
08-16 14:44:09.332  1000  1823  5205 W system_server:   at com.android.server.am.ActivityManagerService.finishReceiver(ActivityManagerService.java:17695)
08-16 14:44:09.332  1000  1823  5205 W system_server:   - locked <0x01f0446d> (a com.android.server.am.ActivityManagerService)
08-16 14:44:09.332  1000  1823  5205 W system_server:   at android.app.IActivityManager$Stub.onTransact(IActivityManager.java:2452)
08-16 14:44:09.332  1000  1823  5205 W system_server:   at com.android.server.am.ActivityManagerService.onTransact(ActivityManagerService.java:3150)
08-16 14:44:09.332  1000  1823  5205 W system_server:   at android.os.Binder.execTransactInternal(Binder.java:1165)
08-16 14:44:09.332  1000  1823  5205 W system_server:   at android.os.Binder.execTransact(Binder.java:1129)

亮屏流程日志

//手指按压开始检测,以这个为开始点
01-19 13:54:48.875 1612 1612 D FingerprintService: finger down, 6 22
01-19 13:54:48.875 1612 1612 V FingerprintService: Acquired: 6 22
01-19 13:54:49.002 1612 1612 V FingerprintService: Acquired: 0 0
01-19 13:54:49.004 1612 1612 I FingerprintService: intercept powerkey time:4879283
01-19 13:54:49.005 1612 1612 D FingerprintService: info:com.android.systemui,1##,20210119-13:54:49
01-19 13:54:49.005 4042 4042 I FingerprintController: fingerprint acquired, grabbing fp wakelock

//FingerprintService收到解锁成功的结果
01-19 13:54:49.006 1612 1612 V FingerprintService: onAuthenticated(true), ID:-588449547, Owner: com.android.systemui, isBP: false, listener: com.android.server.biometrics.fingerprint.FingerprintService$ServiceListenerImpl@62e56aa, requireConfirmation: false, user: 0
01-19 13:54:49.091 1612 1612 I FingerprintService: startExtCmd cmd: 12 param: 0 result:0
01-19 13:54:49.096 1612 1612 V FingerprintService: Done with client: com.android.systemui

// 侧边指纹灭屏的时候上层走的是快速解锁逻辑,这个时候先调用亮屏android.policy:FINGERPRINT
// 开始亮屏
45275:01-19 13:54:49.102 1000 1612 10213 I PowerManagerService: Waking up from Asleep (uid=1000, reason=WAKE_REASON_UNKNOWN, details=android.policy:FINGERPRINT)…
45276:01-19 13:54:49.110 1000 1612 1612 I WindowManager: Started waking up… (why=ON_BECAUSE_OF_UNKNOWN)
//等待窗口绘制
45277:01-19 13:54:49.113 1000 1612 2096 I DisplayPowerController: Blocking screen on until initial contents have been drawn.
45278:01-19 13:54:49.113 1000 1612 2096 I WindowManager: Screen turning on…
45282:01-19 13:54:49.129 1000 1612 2096 I AutomaticBrightnessControllerInjector: setLightSensorEnabled: false
45283:01-19 13:54:49.129 1000 1612 2096 I AutomaticBrightnessControllerInjector: getLatestData res: 20.856249
45290:01-19 13:54:49.147 1000 1612 2096 I AutomaticBrightnessControllerInjector: setAccSensorEnabled enable
45292:01-19 13:54:49.157 1000 1612 2096 I AutomaticBrightnessControllerInjector: setMotionSensorEnabled enable
45295:01-19 13:54:49.169 1000 1612 2096 I AutomaticBrightnessControllerInjector: getLatestData res: 0.0
45296:01-19 13:54:49.169 1000 1612 2096 D AutomaticBrightnessController: updateAutoBrightness: mScreenAutoBrightness=-1.0, newScreenAutoBrightness=0.14320625
45297:01-19 13:54:49.169 1000 1612 2096 I AutomaticBrightnessControllerInjector: skip debounce!
//更新亮度
45298:01-19 13:54:49.171 1000 1612 2096 V DisplayPowerController: Brightness [0.14320625] reason changing to: ‘automatic’, previous reason: ‘screen_off’.
45324:01-19 13:54:49.220 1000 1612 6002 I WindowManager: mKeyguardDelegate.ShowListener.onDrawn.
45326:01-19 13:54:49.223 1000 1612 2004 W WindowManager: Setting mKeyguardDrawComplete
45340:01-19 13:54:49.260 1000 1612 10415 D WindowManager: setKeyguardOrAodShowingOnDefaultDisplay showing = false
45341:01-19 13:54:49.287 1000 1612 2009 I WindowManager: All windows ready for display!
//绘制完成
45342:01-19 13:54:49.287 1000 1612 2004 W WindowManager: Setting mWindowManagerDrawComplete
45343:01-19 13:54:49.292 1000 1612 2009 I WindowManager: updateContrastOverlay, darkmode: true isContrastEnabled: false alpha: 0.10719135
45344:01-19 13:54:49.292 1000 1612 2096 I DisplayPowerController: Unblocked screen on after 178 ms
45345:01-19 13:54:49.293 1000 1612 2009 I WindowManager: updateTextureEyeCareLevel, texture eyecare enable : true, level : 13
//完成亮屏
45350:01-19 13:54:49.310 1000 1612 1612 I WindowManager: Finished waking up… (why=ON_BECAUSE_OF_UNKNOWN)