4.7.12. hrut_sched_log_parse

hrut_sched_log_parse 是一个用于解析调度日志的工具。它从指定的文件中读取调度日志数据,并将其格式化输出。这些日志通常存储在 pstore 文件系统中,用于记录系统调度事件。

4.7.12.1. 语法说明

以下是执行 hrut_sched_log_parse 的命令格式:

hrut_sched_log_parse [ 文件路径 ]

4.7.12.2. 参数选项说明

以下是 hrut_sched_log_parse 支持的选项及其说明:

  • 文件路径:指定包含调度日志的文件路径,通常是从 pstore 文件系统挂载点读取的文件,例如 /mnt/sched-ramoops-0 。

4.7.12.3. 各项命令选项及其功能说明

使用功能说明

hrut_sched_log_parse 是用于解析 Linux pstore 中调度日志的工具。该工具主要用于 异常分析、性能调优 及 系统故障排查,当系统异常重启时,日志文件保存在 /sys/fs/pstore 路径下。

  • console-ramoops-0 :该文件通常存储系统崩溃时控制台输出的日志(例如内核 panic 信息、错误消息、调试信息等)。

  • dmesg-ramoops-0 :该文件通常存储内核启动后的日志信息,包括硬件检测、驱动加载、系统启动期间发生的错误或警告等。

  • sched-ramoops-0 :该文件通常存储的是调度器( scheduler)相关的日志信息,记录内核调度器在崩溃时的状态。

可以输入以下命令触发内核 Panic,从而使系统发生异常重启:

echo c > /proc/sysrq-trigger

注意:

  • 上述方法会导致系统非正常重启,可能会丢失数据,请确保在测试环境使用,并做好数据备份。

解析调度日志文件

命令:

hrut_sched_log_parse /sys/fs/pstore/sched-ramoops-0 | sort

功能: 从 /mnt/sched-ramoops-0 文件中读取调度日志,然后通过 sort 命令对输出结果进行排序。

部分日志打印输出:

[  117.879128] [4] : rcK(1100:1100:120:S) -> swapper/4(0:0:120)
[  117.883224] [4] : swapper/4(0:0:120:R) -> rcK(1100:1100:120)
[  117.883816] [4] : rcK(1100:1100:120:S) -> swapper/4(0:0:120)
[  117.890071] [4] : swapper/4(0:0:120:R) -> phc2sys(558:558:120)
[  117.890601] [4] : phc2sys(558:558:120:Z) -> swapper/4(0:0:120)
[  117.895085] [4] : swapper/4(0:0:120:R) -> rcK(1100:1100:120)
[  117.895643] [4] : rcK(1100:1100:120:S) -> swapper/4(0:0:120)
[  117.907020] [4] : swapper/4(0:0:120:R) -> rcK(1100:1100:120)
[  117.907608] [4] : rcK(1100:1100:120:S) -> swapper/4(0:0:120)
[  117.912495] [4] : swapper/4(0:0:120:R) -> rcK(1100:1100:120)
[  117.913079] [4] : rcK(1100:1100:120:S) -> swapper/4(0:0:120)
[  117.917170] [4] : swapper/4(0:0:120:R) -> rcK(1100:1100:120)
[  117.917742] [4] : rcK(1100:1100:120:S) -> swapper/4(0:0:120)
[  117.925579] [4] : swapper/4(0:0:120:R) -> rcK(1100:1100:120)
[  117.926147] [4] : rcK(1100:1100:120:S) -> swapper/4(0:0:120)
[  117.944312] [4] : swapper/4(0:0:120:R) -> sshd(1083:1083:120)
[  117.945778] [4] : sshd(1083:1083:120:Z) -> swapper/4(0:0:120)
[  117.949755] [4] : swapper/4(0:0:120:R) -> rcK(1100:1100:120)
[  117.950353] [4] : rcK(1100:1100:120:S) -> swapper/4(0:0:120)
[  117.956562] [4] : swapper/4(0:0:120:R) -> ntpd(436:436:120)
[  117.957772] [4] : ntpd(436:436:120:R+) -> kworker/4:2(100:100:120)
[  117.957788] [4] : kworker/4:2(100:100:120:I) -> ntpd(436:436:120)
[  117.957815] [4] : ntpd(436:436:120:Z) -> swapper/4(0:0:120)
[  117.961505] [4] : swapper/4(0:0:120:R) -> rcK(1100:1100:120)
[  117.962072] [4] : rcK(1100:1100:120:S) -> swapper/4(0:0:120)
[  118.000704] [6] : run-parts(1192:1192:120:D) -> swapper/6(0:0:120)
[  118.003746] [6] : swapper/6(0:0:120:R) -> run-parts(1192:1192:120)
[  118.004010] [6] : run-parts(1192:1192:120:Z) -> swapper/6(0:0:120)
[  118.013362] [6] : swapper/6(0:0:120:R) -> ifdown(1195:1195:120)
[  118.018816] [6] : ip(1195:1195:120:Z) -> swapper/6(0:0:120)
[  118.020837] [6] : swapper/6(0:0:120:R) -> cam-service(580:611:120)
[  118.020895] [6] : cam-service(580:611:120:X) -> swapper/6(0:0:120)
[  118.021007] [4] : swapper/4(0:0:120:R) -> cam-service(580:612:120)
[  118.021072] [4] : cam-service(580:612:120:X) -> swapper/4(0:0:120)
[  118.025073] [6] : swapper/6(0:0:120:R) -> ifdown(1197:1197:120)
[  118.028848] [6] : sh(1197:1197:120:R+) -> kworker/6:0(44:44:120)
[  118.028880] [6] : kworker/6:0(44:44:120:I) -> ksoftirqd/6(43:43:120)
[  118.028947] [6] : ksoftirqd/6(43:43:120:S) -> kworker/6:0(44:44:120)
[  118.028960] [6] : kworker/6:0(44:44:120:I) -> sh(1197:1197:120)
[  118.030746] [6] : run-parts(1197:1197:120:Z) -> swapper/6(0:0:120)
[  118.032244] [4] : swapper/4(0:0:120:R) -> rcK(1100:1100:120)

结果解析:

  • 关键字段说明:

  • <timestamp>:调度事件发生的时间戳,格式为秒 . 微秒,例如 117.879128 表示 117 秒 879128 微秒。

  • [CPU ] : 进程名 (PID:PPID: 优先级 : 状态 ) -> 目标进程 (PID:PPID: 优先级 )

    • CPU 核心:日志开头的 [4]、[6] 代表该调度发生在 CPU 核 4 或 6 上。

    • 进程信息:

      • 进程名(如 swapper/4, rcK, phc2sys, sshd)

      • PID(进程 ID)

      • PPID(父进程 ID)

      • 优先级( 120 )

      • 状态:

        • R( Running):进程正在运行

        • S( Sleeping):进程在休眠

        • D( Uninterruptible Sleep):不可中断的休眠状态

        • Z( Zombie):僵尸进程

        • X( Dead):已终止的进程

        • I( Idle):空闲状态

解析控制台日志文件

命令:

hrut_sched_log_parse /sys/fs/pstore/console-ramoops-0 | sort

功能: 从 /sys/fs/pstore/console-ramoops-0 文件中读取控制台日志,然后通过 sort 命令对输出结果进行排序。

部分日志打印输出:

[    0.321146] Key ▒A(1702062444:1634301033:73:trusted keyrings
[    0.321146] Key ▒A) ->   0.321146] Key ▒A(544040308:1937339168:110)
[    0.625154] KERNEL  SRAM▒A(909586484:842411568:32:  [DONE]
[    0.625154] KERNEL  SRAM▒A) -> 54] KERNEL  SRAM▒A(1279610450:1162551389:32)
[    0.625▒A(1702521203:539766832:32)
[    0.625▒A(808464432:809007152:58:: 0x2000, alignment: 0x10
[    0.625▒A) ->  0x10
[    0.626524] random: cr▒A(1919164481:1128341536:76:t success!
[    0.626524] random: cr▒A) -> 6524] random: cr▒A(1768843552:1919252073:32)
[    0.697517] ***************▒A(542837363:1702125923:102:012]
[    0.697517] ***************▒A) ->  ***************▒A(909719086:807411744:105)
[    0.704▒A(707406378:707406378:42)
[    0.704▒A(707406378:707406378:42:*************************
[    0.704▒A) -> *****
[    0.719748] **  WRITEABLE clk D▒A) ->  WRITEABLE clk D▒A(706748448:538976288:32)
[    0.727157]▒A(1109414721:1210078290:80:LED IN THIS KERNEL **
[    0.727157]▒A(1111576133:542000453:79)
[    0.727157]▒A) -> *
[    0.734552] ** This means that▒A) ->  This means that▒A(538976288:538976288:32)
[    0.741957▒A(1818850658:544434464:101:ose clk operations  **
[    0.741957▒A(1886938400:1869881460:108)
[    0.741957▒A) -> **
[    0.764170] **▒A(538976288:538976288:32:                **
[    0.764170] **▒A) ->     0.764170] **▒A(538976288:538976288:32)
[    0.852621] dw-ap▒A(707406378:707406378:42:***************
[    0.852621] dw-ap▒A) ->  0.852621] dw-ap▒A(707406378:707406378:42)
[    1.501▒A(1651666790:540699745:48)
[    1.501▒A(1769104755:774910000:50:id DMA for kernel console
[    1.501▒A) -> nsole
[    1.518218] mtdblock: MTD devic▒A) -> block: MTD devic▒A(1869898092:1969710450:110)
[    1.546482] create /dev/nano2d d▒A) -> te /dev/nano2d d▒A(1702060386:1635017060:103)
[    1.556459]   irq line:60
[    1.556459]   irq line:60
[    1.562843]   regis▒A(1734701600:538991928:48:se:0x3c010000
[    1.562843]   regis▒A) -> .562843]   regis▒A(1633820786:1702130537:56)
[    1.568188]   iommu enabled
[    1.568188]   iommu enabled
[    1.574312]   contiguous▒A(1852793632:538991920:57:base:0x0
[    1.574312]   contiguous▒A) -> 12]   contiguous▒A(544437615:1969711476:57)
[    1.585705]

解析内核日志文件

命令:

hrut_sched_log_parse /sys/fs/pstore/dmesg-ramoops-0 | sort

功能: 从 /sys/fs/pstore/console-ramoops-0 文件中读取控制台日志,然后通过 sort 命令对输出结果进行排序。

部分日志打印输出:

<0>[  ▒A(1043741706:1681209907:58:05337] sysrq: Trigger a crash
<0>[  ▒A(841888053:538976347:102)
<0>[  ▒A) -> r a crash
<12>[  ▒A(1836412448:1953653104:101:2>[    4.094130] Normal boot
<12>[  ▒A(826018397:811278394:32)
<12>[  ▒A) -> mal boot
<12>[▒A(1936942435:1668641568:101: no need to mark it repeatedly
<12>[▒A(744845935:1717920288:100)
<12>[▒A) -> repeatedly
<14>[    2.635783] /dev/block/p▒A(761882989:1735536505:112:mode
<14>[    2.635783] /dev/block/p▒A) -> 83] /dev/block/p▒A(543911529:1814062185:104)
<14>[    2.823111] /dev/bl▒A(1701995112:795176289:45:result: 0
<14>[    2.823111] /dev/bl▒A) -> .823111] /dev/bl▒A(543908723:1714578720:110)
<14>[    2.911575] /de▒A(1701667182:762929711:114:sck result: 0
<14>[    2.911575] /de▒A) ->    2.911575] /de▒A(1714578720:1886413103:109)
<14>[    2.968035] Starti▒A(1869959456:778854766:117: disabled.
<14>[    2.968035] Starti▒A) -> 2.968035] Starti▒A(979723375:1830838644:114)
<14>[    3.152048] Starting▒A(1953064736:980379949:104: 42566cb
<14>[    3.152048] Starting▒A) -> 152048] Starting▒A(980707693:1836016416:119)
<14>[    3.549996] done
<14>[    3.549996] done
<14>[    3.783396] Start log pr▒A(775102496:538991422:49:done
<14>[    3.783396] Start log pr▒A) -> 96] Start log pr▒A(542979376:876033847:52)
<14>[    3.8070▒A(1919426653:959657268:91)
<14>[    3.8070▒A(959917619:538976288:62:ut_remoteproc_log ok
<14>[    3.8070▒A) ->
<14>[    3.975114] Config 2054000▒A) -> ] Config 2054000▒A(1868832816:540697972:97)
<14>[    3.984734] Config ▒A(1684104562:544436081:115:e: 0 done
<14>[    3.984734] Config ▒A) -> .984734] Config ▒A(1953067639:540024890:32)
<14>[    3.998717] C▒A(1869684851:1869700963:109:0 write: 0 done
<14>[    3.998717] C▒A) -> [    3.998717] C▒A(540697697:1701978227:97)
<14>[    4.0▒A(1869684851:1869700912:48)
<14>[    4.0▒A(842084724:1647194160:48:s read: 0 write: 0 done
<14>[    4.0▒A) -> one
<14>[▒A(1702125932:543516788:102)

显示帮助信息

命令:

root@buildroot:~# hrut_sched_log_parse -h

功能: 该命令会显示帮助信息,解释命令的使用方法。

打印输出:

This program parse dump sched log pstore sched-ramoops-0
Usage:
      hrut_sched_log_parse /mnt/sched-ramoops-0 | sort