projects/linux·blob master · refs

137 lines · 5.4 KiB

1.. SPDX-License-Identifier: GPL-2.0
2
3===========================
4MEMORY ALLOCATION PROFILING
5===========================
6
7Low overhead (suitable for production) accounting of all memory allocations,
8tracked by file and line number.
9
10Usage:
11kconfig options:
12- CONFIG_MEM_ALLOC_PROFILING
13
14- CONFIG_MEM_ALLOC_PROFILING_ENABLED_BY_DEFAULT
15
16- CONFIG_MEM_ALLOC_PROFILING_DEBUG
17 adds warnings for allocations that weren't accounted because of a
18 missing annotation
19
20Boot parameter:
21 sysctl.vm.mem_profiling={0|1|never}[,compressed]
22
23 When set to "never", memory allocation profiling overhead is minimized and it
24 cannot be enabled at runtime (sysctl becomes read-only).
25 When CONFIG_MEM_ALLOC_PROFILING_ENABLED_BY_DEFAULT=y, default value is "1".
26 When CONFIG_MEM_ALLOC_PROFILING_ENABLED_BY_DEFAULT=n, default value is "never".
27 "compressed" optional parameter will try to store page tag references in a
28 compact format, avoiding page extensions. This results in improved performance
29 and memory consumption, however it might fail depending on system configuration.
30 If compression fails, a warning is issued and memory allocation profiling gets
31 disabled.
32
33sysctl:
34 /proc/sys/vm/mem_profiling
35
36 1: Enable memory profiling.
37
38 0: Disable memory profiling.
39
40 The default value depends on CONFIG_MEM_ALLOC_PROFILING_ENABLED_BY_DEFAULT.
41
42 When CONFIG_MEM_ALLOC_PROFILING_DEBUG=y, this control is read-only to avoid
43 warnings produced by allocations made while profiling is disabled and freed
44 when it's enabled.
45
46 /proc/sys/vm/mem_profiling_compressed
47
48 1: Page alloc tag compression is enabled.
49
50 0: Page alloc tag compression is disabled.
51
52 This reflects a static boot-time configuration of how page allocation tags are
53 stored (in page flags when compression is enabled and in page_ext when disabled).
54 Toggling ``mem_profiling`` at runtime does not change the state of
55 ``mem_profiling_compressed``.
56
57Runtime info:
58 /proc/allocinfo
59
60 Profiling data can be retrieved either by reading `/proc/allocinfo` directly as
61 text or programmatically via `ioctl()` calls defined in `<uapi/linux/alloc_tag.h>`.
62 The ioctl interface supports structured binary data extraction as well as filtering
63 by module name, function, file, line number, accuracy, or allocation size limits.
64
65Example output::
66
67 root@moria-kvm:~# sort -g /proc/allocinfo|tail|numfmt --to=iec
68 2.8M 22648 fs/kernfs/dir.c:615 func:__kernfs_new_node
69 3.8M 953 mm/memory.c:4214 func:alloc_anon_folio
70 4.0M 1010 drivers/staging/ctagmod/ctagmod.c:20 [ctagmod] func:ctagmod_start
71 4.1M 4 net/netfilter/nf_conntrack_core.c:2567 func:nf_ct_alloc_hashtable
72 6.0M 1532 mm/filemap.c:1919 func:__filemap_get_folio
73 8.8M 2785 kernel/fork.c:307 func:alloc_thread_stack_node
74 13M 234 block/blk-mq.c:3421 func:blk_mq_alloc_rqs
75 14M 3520 mm/mm_init.c:2530 func:alloc_large_system_hash
76 15M 3656 mm/readahead.c:247 func:page_cache_ra_unbounded
77 55M 4887 mm/slub.c:2259 func:alloc_slab_page
78 122M 31168 mm/page_ext.c:270 func:alloc_page_ext
79
80Theory of operation
81===================
82
83Memory allocation profiling builds off of code tagging, which is a library for
84declaring static structs (that typically describe a file and line number in
85some way, hence code tagging) and then finding and operating on them at runtime,
86- i.e. iterating over them to print them in debugfs/procfs.
87
88To add accounting for an allocation call, we replace it with a macro
89invocation, alloc_hooks(), that
90- declares a code tag
91- stashes a pointer to it in task_struct
92- calls the real allocation function
93- and finally, restores the task_struct alloc tag pointer to its previous value.
94
95This allows for alloc_hooks() calls to be nested, with the most recent one
96taking effect. This is important for allocations internal to the mm/ code that
97do not properly belong to the outer allocation context and should be counted
98separately: for example, slab object extension vectors, or when the slab
99allocates pages from the page allocator.
100
101Thus, proper usage requires determining which function in an allocation call
102stack should be tagged. There are many helper functions that essentially wrap
103e.g. kmalloc() and do a little more work, then are called in multiple places;
104we'll generally want the accounting to happen in the callers of these helpers,
105not in the helpers themselves.
106
107To fix up a given helper, for example foo(), do the following:
108- switch its allocation call to the _noprof() version, e.g. kmalloc_noprof()
109
110- rename it to foo_noprof()
111
112- define a macro version of foo() like so:
113
114 #define foo(...) alloc_hooks(foo_noprof(__VA_ARGS__))
115
116It's also possible to stash a pointer to an alloc tag in your own data structures.
117
118Do this when you're implementing a generic data structure that does allocations
119"on behalf of" some other code - for example, the rhashtable code. This way,
120instead of seeing a large line in /proc/allocinfo for rhashtable.c, we can
121break it out by rhashtable type.
122
123To do so:
124- Hook your data structure's init function, like any other allocation function.
125
126- Within your init function, use the convenience macro alloc_tag_record() to
127 record alloc tag in your data structure.
128
129- Then, use the following form for your allocations:
130 alloc_hooks_tag(ht->your_saved_tag, kmalloc_noprof(...))
131
132Notes
133=====
134
135- When a slab object is allocated from KFENCE, its accounting is skipped.
136 KFENCE allocations are rare and limited to a small number, so this omission
137 is negligible.