projects/linux·blob master · refs

313 lines · 10.8 KiB

1.. SPDX-License-Identifier: GPL-2.0
2
3=================
4Boot-time tracing
5=================
6
7:Author: Masami Hiramatsu <[email protected]>
8
9Overview
10========
11
12Boot-time tracing allows users to trace boot-time process including
13device initialization with full features of ftrace including per-event
14filter and actions, histograms, kprobe-events and synthetic-events,
15and trace instances.
16Since kernel command line is not enough to control these complex features,
17this uses bootconfig file to describe tracing feature programming.
18
19Options in the Boot Config
20==========================
21
22Here is the list of available options for boot time tracing in
23boot config file [1]_. All options are under "ftrace." or "kernel."
24prefix. See kernel parameters for the options which starts
25with "kernel." prefix [2]_.
26
27.. [1] See :ref:`Documentation/admin-guide/bootconfig.rst <bootconfig>`
28.. [2] See :ref:`Documentation/admin-guide/kernel-parameters.rst <kernelparameters>`
29
30Ftrace Global Options
31---------------------
32
33Ftrace global options have "kernel." prefix in boot config, which means
34these options are passed as a part of kernel legacy command line.
35
36kernel.tp_printk
37 Output trace-event data on printk buffer too.
38
39kernel.dump_on_oops [= MODE]
40 Dump ftrace on Oops. If MODE = 1 or omitted, dump trace buffer
41 on all CPUs. If MODE = 2, dump a buffer on a CPU which kicks Oops.
42
43kernel.traceoff_on_warning
44 Stop tracing if WARN_ON() occurs.
45
46kernel.fgraph_max_depth = MAX_DEPTH
47 Set MAX_DEPTH to maximum depth of fgraph tracer.
48
49kernel.fgraph_filters = FILTER[, FILTER2...]
50 Add fgraph tracing function filters.
51
52kernel.fgraph_notraces = FILTER[, FILTER2...]
53 Add fgraph non-tracing function filters.
54
55
56Ftrace Per-instance Options
57---------------------------
58
59These options can be used for each instance including global ftrace node.
60
61ftrace.[instance.INSTANCE.]options = OPT1[, OPT2[...]]
62 Enable given ftrace options.
63
64ftrace.[instance.INSTANCE.]tracing_on = 0|1
65 Enable/Disable tracing on this instance when starting boot-time tracing.
66 (you can enable it by the "traceon" event trigger action)
67
68ftrace.[instance.INSTANCE.]trace_clock = CLOCK
69 Set given CLOCK to ftrace's trace_clock.
70
71ftrace.[instance.INSTANCE.]buffer_size = SIZE
72 Configure ftrace buffer size to SIZE. You can use "KB" or "MB"
73 for that SIZE.
74
75ftrace.[instance.INSTANCE.]alloc_snapshot
76 Allocate snapshot buffer.
77
78ftrace.[instance.INSTANCE.]cpumask = CPUMASK
79 Set CPUMASK as trace cpu-mask.
80
81ftrace.[instance.INSTANCE.]events = EVENT[, EVENT2[...]]
82 Enable given events on boot. You can use a wild card in EVENT.
83
84ftrace.[instance.INSTANCE.]tracer = TRACER
85 Set TRACER to current tracer on boot. (e.g. function)
86
87ftrace.[instance.INSTANCE.]ftrace.filters
88 This will take an array of tracing function filter rules.
89
90ftrace.[instance.INSTANCE.]ftrace.notraces
91 This will take an array of NON-tracing function filter rules.
92
93
94Ftrace Per-Event Options
95------------------------
96
97These options are setting per-event options.
98
99ftrace.[instance.INSTANCE.]event.GROUP.EVENT.enable
100 Enable GROUP:EVENT tracing.
101
102ftrace.[instance.INSTANCE.]event.GROUP.enable
103 Enable all event tracing within GROUP.
104
105ftrace.[instance.INSTANCE.]event.enable
106 Enable all event tracing.
107
108ftrace.[instance.INSTANCE.]event.GROUP.EVENT.filter = FILTER
109 Set FILTER rule to the GROUP:EVENT.
110
111ftrace.[instance.INSTANCE.]event.GROUP.EVENT.actions = ACTION[, ACTION2[...]]
112 Set ACTIONs to the GROUP:EVENT.
113
114ftrace.[instance.INSTANCE.]event.kprobes.EVENT.probes = PROBE[, PROBE2[...]]
115 Defines new kprobe event based on PROBEs. It is able to define
116 multiple probes on one event, but those must have same type of
117 arguments. This option is available only for the event which
118 group name is "kprobes".
119
120ftrace.[instance.INSTANCE.]event.synthetic.EVENT.fields = FIELD[, FIELD2[...]]
121 Defines new synthetic event with FIELDs. Each field should be
122 "type varname".
123
124ftrace.[instance.INSTANCE.]event.eprobes.EVENT.probes = PROBE[, PROBE2[...]]
125 Defines new event probe based on PROBEs. This option is available only
126 for the event which group name is "eprobes".
127
128ftrace.[instance.INSTANCE.]event.fprobes.EVENT.probes = PROBE[, PROBE2[...]]
129 Defines new fprobe event based on PROBEs. This option is available only
130 for the event which group name is "fprobes".
131
132ftrace.[instance.INSTANCE.]event.tracepoints.EVENT.probes = PROBE[, PROBE2[...]]
133 Defines new tracepoint probe based on PROBEs. This option is available only
134 for the event which group name is "tracepoints" or "tprobes".
135
136Note that dynamic event definitions can be written under instance node, but
137those are also visible from other instances. So please take care for event
138name conflict.
139
140Ftrace Histogram Options
141------------------------
142
143Since it is too long to write a histogram action as a string for per-event
144action option, there are tree-style options under per-event 'hist' subkey
145for the histogram actions. For the detail of the each parameter,
146please read the event histogram document (Documentation/trace/histogram.rst)
147
148ftrace.[instance.INSTANCE.]event.GROUP.EVENT.hist.[N.]keys = KEY1[, KEY2[...]]
149 Set histogram key parameters. (Mandatory)
150 The 'N' is a digit string for the multiple histogram. You can omit it
151 if there is one histogram on the event.
152
153ftrace.[instance.INSTANCE.]event.GROUP.EVENT.hist.[N.]values = VAL1[, VAL2[...]]
154 Set histogram value parameters.
155
156ftrace.[instance.INSTANCE.]event.GROUP.EVENT.hist.[N.]sort = SORT1[, SORT2[...]]
157 Set histogram sort parameter options.
158
159ftrace.[instance.INSTANCE.]event.GROUP.EVENT.hist.[N.]size = NR_ENTRIES
160 Set histogram size (number of entries).
161
162ftrace.[instance.INSTANCE.]event.GROUP.EVENT.hist.[N.]name = NAME
163 Set histogram name.
164
165ftrace.[instance.INSTANCE.]event.GROUP.EVENT.hist.[N.]var.VARIABLE = EXPR
166 Define a new VARIABLE by EXPR expression.
167
168ftrace.[instance.INSTANCE.]event.GROUP.EVENT.hist.[N.]<pause|continue|clear>
169 Set histogram control parameter. You can set one of them.
170
171ftrace.[instance.INSTANCE.]event.GROUP.EVENT.hist.[N.]onmatch.[M.]event = GROUP.EVENT
172 Set histogram 'onmatch' handler matching event parameter.
173 The 'M' is a digit string for the multiple 'onmatch' handler. You can omit it
174 if there is one 'onmatch' handler on this histogram.
175
176ftrace.[instance.INSTANCE.]event.GROUP.EVENT.hist.[N.]onmatch.[M.]trace = EVENT[, ARG1[...]]
177 Set histogram 'trace' action for 'onmatch'.
178 EVENT must be a synthetic event name, and ARG1... are parameters
179 for that event. Mandatory if 'onmatch.event' option is set.
180
181ftrace.[instance.INSTANCE.]event.GROUP.EVENT.hist.[N.]onmax.[M.]var = VAR
182 Set histogram 'onmax' handler variable parameter.
183
184ftrace.[instance.INSTANCE.]event.GROUP.EVENT.hist.[N.]onchange.[M.]var = VAR
185 Set histogram 'onchange' handler variable parameter.
186
187ftrace.[instance.INSTANCE.]event.GROUP.EVENT.hist.[N.]<onmax|onchange>.[M.]save = ARG1[, ARG2[...]]
188 Set histogram 'save' action parameters for 'onmax' or 'onchange' handler.
189 This option or below 'snapshot' option is mandatory if 'onmax.var' or
190 'onchange.var' option is set.
191
192ftrace.[instance.INSTANCE.]event.GROUP.EVENT.hist.[N.]<onmax|onchange>.[M.]snapshot
193 Set histogram 'snapshot' action for 'onmax' or 'onchange' handler.
194 This option or above 'save' option is mandatory if 'onmax.var' or
195 'onchange.var' option is set.
196
197ftrace.[instance.INSTANCE.]event.GROUP.EVENT.hist.filter = FILTER_EXPR
198 Set histogram filter expression. You don't need 'if' in the FILTER_EXPR.
199
200Note that this 'hist' option can conflict with the per-event 'actions'
201option if the 'actions' option has a histogram action.
202
203
204When to Start
205=============
206
207All boot-time tracing options starting with ``ftrace`` will be enabled at the
208end of core_initcall. This means you can trace the events from postcore_initcall.
209Most of the subsystems and architecture dependent drivers will be initialized
210after that (arch_initcall or subsys_initcall). Thus, you can trace those with
211boot-time tracing.
212If you want to trace events before core_initcall, you can use the options
213starting with ``kernel``. Some of them will be enabled earlier than the initcall
214processing (for example, ``kernel.ftrace=function`` and ``kernel.trace_event``
215will start before the initcall.)
216
217
218Examples
219========
220
221For example, to add filter and actions for each event, define kprobe
222events, and synthetic events with histogram, write a boot config like
223below::
224
225 ftrace.event {
226 task.task_newtask {
227 filter = "pid < 128"
228 enable
229 }
230 kprobes.vfs_read {
231 probes = "vfs_read $arg1 $arg2"
232 filter = "common_pid < 200"
233 enable
234 }
235 synthetic.initcall_latency {
236 fields = "unsigned long func", "u64 lat"
237 hist {
238 keys = func.sym, lat
239 values = lat
240 sort = lat
241 }
242 }
243 initcall.initcall_start.hist {
244 keys = func
245 var.ts0 = common_timestamp.usecs
246 }
247 initcall.initcall_finish.hist {
248 keys = func
249 var.lat = common_timestamp.usecs - $ts0
250 onmatch {
251 event = initcall.initcall_start
252 trace = initcall_latency, func, $lat
253 }
254 }
255 }
256
257Also, boot-time tracing supports "instance" node, which allows us to run
258several tracers for different purpose at once. For example, one tracer
259is for tracing functions starting with "user\_", and others tracing
260"kernel\_" functions, you can write boot config as below::
261
262 ftrace.instance {
263 foo {
264 tracer = "function"
265 ftrace.filters = "user_*"
266 }
267 bar {
268 tracer = "function"
269 ftrace.filters = "kernel_*"
270 }
271 }
272
273The instance node also accepts event nodes so that each instance
274can customize its event tracing.
275
276With the trigger action and kprobes, you can trace function-graph while
277a function is called. For example, this will trace all function calls in
278the pci_proc_init()::
279
280 ftrace {
281 tracing_on = 0
282 tracer = function_graph
283 event.kprobes {
284 start_event {
285 probes = "pci_proc_init"
286 actions = "traceon"
287 }
288 end_event {
289 probes = "pci_proc_init%return"
290 actions = "traceoff"
291 }
292 }
293 }
294
295
296This boot-time tracing also supports ftrace kernel parameters via boot
297config.
298For example, following kernel parameters::
299
300 trace_options=sym-addr trace_event=initcall:* tp_printk trace_buf_size=1M ftrace=function ftrace_filter="vfs*"
301
302This can be written in boot config like below::
303
304 kernel {
305 trace_options = sym-addr
306 trace_event = "initcall:*"
307 tp_printk
308 trace_buf_size = 1M
309 ftrace = function
310 ftrace_filter = "vfs*"
311 }
312
313Note that parameters start with "kernel" prefix instead of "ftrace".