| 1 | # SPDX-License-Identifier: GPL-2.0-only |
| 2 | # |
| 3 | # RCU-related configuration options |
| 4 | # |
| 5 | |
| 6 | menu "RCU Subsystem" |
| 7 | |
| 8 | config TREE_RCU |
| 9 | bool |
| 10 | default y if SMP |
| 11 | # Dynticks-idle tracking |
| 12 | select CONTEXT_TRACKING_IDLE |
| 13 | help |
| 14 | This option selects the RCU implementation that is |
| 15 | designed for very large SMP system with hundreds or |
| 16 | thousands of CPUs. It also scales down nicely to |
| 17 | smaller systems. |
| 18 | |
| 19 | config PREEMPT_RCU |
| 20 | bool |
| 21 | default y if (PREEMPT || PREEMPT_RT || PREEMPT_DYNAMIC) |
| 22 | select TREE_RCU |
| 23 | help |
| 24 | This option selects the RCU implementation that is |
| 25 | designed for very large SMP systems with hundreds or |
| 26 | thousands of CPUs, but for which real-time response |
| 27 | is also required. It also scales down nicely to |
| 28 | smaller systems. |
| 29 | |
| 30 | Select this option if you are unsure. |
| 31 | |
| 32 | config TINY_RCU |
| 33 | bool |
| 34 | default y if !PREEMPT_RCU && !SMP |
| 35 | help |
| 36 | This option selects the RCU implementation that is |
| 37 | designed for UP systems from which real-time response |
| 38 | is not required. This option greatly reduces the |
| 39 | memory footprint of RCU. |
| 40 | |
| 41 | config RCU_EXPERT |
| 42 | bool "Make expert-level adjustments to RCU configuration" |
| 43 | default n |
| 44 | help |
| 45 | This option needs to be enabled if you wish to make |
| 46 | expert-level adjustments to RCU configuration. By default, |
| 47 | no such adjustments can be made, which has the often-beneficial |
| 48 | side-effect of preventing "make oldconfig" from asking you all |
| 49 | sorts of detailed questions about how you would like numerous |
| 50 | obscure RCU options to be set up. |
| 51 | |
| 52 | Say Y if you need to make expert-level adjustments to RCU. |
| 53 | |
| 54 | Say N if you are unsure. |
| 55 | |
| 56 | config TINY_SRCU |
| 57 | bool |
| 58 | default y if TINY_RCU |
| 59 | help |
| 60 | This option selects the single-CPU non-preemptible version of SRCU. |
| 61 | |
| 62 | config TREE_SRCU |
| 63 | bool |
| 64 | default y if !TINY_RCU |
| 65 | help |
| 66 | This option selects the full-fledged version of SRCU. |
| 67 | |
| 68 | config FORCE_NEED_SRCU_NMI_SAFE |
| 69 | bool "Force selection of NEED_SRCU_NMI_SAFE" |
| 70 | depends on !TINY_SRCU |
| 71 | depends on RCU_EXPERT |
| 72 | depends on ARCH_HAS_NMI_SAFE_THIS_CPU_OPS |
| 73 | select NEED_SRCU_NMI_SAFE |
| 74 | default n |
| 75 | help |
| 76 | This option forces selection of the NEED_SRCU_NMI_SAFE |
| 77 | Kconfig option, allowing testing of srcu_read_lock_nmisafe() |
| 78 | and srcu_read_unlock_nmisafe() on architectures (like x86) |
| 79 | that select the ARCH_HAS_NMI_SAFE_THIS_CPU_OPS Kconfig option. |
| 80 | |
| 81 | config NEED_SRCU_NMI_SAFE |
| 82 | def_bool HAVE_NMI && !ARCH_HAS_NMI_SAFE_THIS_CPU_OPS && !TINY_SRCU |
| 83 | |
| 84 | config TASKS_RCU_GENERIC |
| 85 | def_bool TASKS_RCU || TASKS_RUDE_RCU |
| 86 | help |
| 87 | This option enables generic infrastructure code supporting |
| 88 | task-based RCU implementations. Not for manual selection. |
| 89 | |
| 90 | config FORCE_TASKS_RCU |
| 91 | bool "Force selection of TASKS_RCU" |
| 92 | depends on RCU_EXPERT |
| 93 | select TASKS_RCU |
| 94 | default n |
| 95 | help |
| 96 | This option force-enables a task-based RCU implementation |
| 97 | that uses only voluntary context switch (not preemption!), |
| 98 | idle, and user-mode execution as quiescent states. Not for |
| 99 | manual selection in most cases. |
| 100 | |
| 101 | config NEED_TASKS_RCU |
| 102 | bool |
| 103 | default n |
| 104 | |
| 105 | config TASKS_RCU |
| 106 | bool |
| 107 | default NEED_TASKS_RCU && PREEMPTION |
| 108 | select IRQ_WORK |
| 109 | |
| 110 | config FORCE_TASKS_RUDE_RCU |
| 111 | bool "Force selection of Tasks Rude RCU" |
| 112 | depends on RCU_EXPERT |
| 113 | select TASKS_RUDE_RCU |
| 114 | default n |
| 115 | help |
| 116 | This option force-enables a task-based RCU implementation |
| 117 | that uses only context switch (including preemption) and |
| 118 | user-mode execution as quiescent states. It forces IPIs and |
| 119 | context switches on all online CPUs, including idle ones, |
| 120 | so use with caution. Not for manual selection in most cases. |
| 121 | |
| 122 | config TASKS_RUDE_RCU |
| 123 | bool |
| 124 | default n |
| 125 | select IRQ_WORK |
| 126 | |
| 127 | config FORCE_TASKS_TRACE_RCU |
| 128 | bool "Force selection of Tasks Trace RCU" |
| 129 | depends on RCU_EXPERT |
| 130 | select TASKS_TRACE_RCU |
| 131 | default n |
| 132 | help |
| 133 | This option enables a task-based RCU implementation that uses |
| 134 | explicit rcu_read_lock_trace() read-side markers, and allows |
| 135 | these readers to appear in the idle loop as well as on the |
| 136 | CPU hotplug code paths. It can force IPIs on online CPUs, |
| 137 | including idle ones, so use with caution. Not for manual |
| 138 | selection in most cases. |
| 139 | |
| 140 | config TASKS_TRACE_RCU |
| 141 | bool |
| 142 | default n |
| 143 | |
| 144 | config TASKS_TRACE_RCU_NO_MB |
| 145 | bool "Override RCU Tasks Trace inclusion of read-side memory barriers" |
| 146 | depends on RCU_EXPERT && TASKS_TRACE_RCU |
| 147 | default ARCH_WANTS_NO_INSTR |
| 148 | help |
| 149 | This option prevents the use of read-side memory barriers in |
| 150 | rcu_read_lock_tasks_trace() and rcu_read_unlock_tasks_trace() |
| 151 | even in kernels built with CONFIG_ARCH_WANTS_NO_INSTR=n, that is, |
| 152 | in kernels that do not have noinstr set up in entry/exit code. |
| 153 | By setting this option, you are promising to carefully review |
| 154 | use of ftrace, BPF, and friends to ensure that no tracing |
| 155 | operation is attached to a function that runs in that portion |
| 156 | of the entry/exit code that RCU does not watch, that is, |
| 157 | where rcu_is_watching() returns false. Alternatively, you |
| 158 | might choose to never remove traces except by rebooting. |
| 159 | |
| 160 | Those wishing to disable read-side memory barriers for an entire |
| 161 | architecture can select this Kconfig option, hence the polarity. |
| 162 | |
| 163 | Say Y here if you need speed and will review use of tracing. |
| 164 | Say N here for certain esoteric testing of RCU itself. |
| 165 | Take the default if you are unsure. |
| 166 | |
| 167 | config RCU_STALL_COMMON |
| 168 | def_bool TREE_RCU |
| 169 | help |
| 170 | This option enables RCU CPU stall code that is common between |
| 171 | the TINY and TREE variants of RCU. The purpose is to allow |
| 172 | the tiny variants to disable RCU CPU stall warnings, while |
| 173 | making these warnings mandatory for the tree variants. |
| 174 | |
| 175 | config RCU_NEED_SEGCBLIST |
| 176 | def_bool ( TREE_RCU || TREE_SRCU || TASKS_RCU_GENERIC ) |
| 177 | |
| 178 | config RCU_FANOUT |
| 179 | int "Tree-based hierarchical RCU fanout value" |
| 180 | range 2 64 if 64BIT |
| 181 | range 2 32 if !64BIT |
| 182 | depends on TREE_RCU && RCU_EXPERT |
| 183 | default 64 if 64BIT |
| 184 | default 32 if !64BIT |
| 185 | help |
| 186 | This option controls the fanout of hierarchical implementations |
| 187 | of RCU, allowing RCU to work efficiently on machines with |
| 188 | large numbers of CPUs. This value must be at least the fourth |
| 189 | root of NR_CPUS, which allows NR_CPUS to be insanely large. |
| 190 | The default value of RCU_FANOUT should be used for production |
| 191 | systems, but if you are stress-testing the RCU implementation |
| 192 | itself, small RCU_FANOUT values allow you to test large-system |
| 193 | code paths on small(er) systems. |
| 194 | |
| 195 | Select a specific number if testing RCU itself. |
| 196 | Take the default if unsure. |
| 197 | |
| 198 | config RCU_FANOUT_LEAF |
| 199 | int "Tree-based hierarchical RCU leaf-level fanout value" |
| 200 | range 2 64 if 64BIT && !RCU_STRICT_GRACE_PERIOD |
| 201 | range 2 32 if !64BIT && !RCU_STRICT_GRACE_PERIOD |
| 202 | range 2 3 if RCU_STRICT_GRACE_PERIOD |
| 203 | depends on TREE_RCU && RCU_EXPERT |
| 204 | default 16 if !RCU_STRICT_GRACE_PERIOD |
| 205 | default 2 if RCU_STRICT_GRACE_PERIOD |
| 206 | help |
| 207 | This option controls the leaf-level fanout of hierarchical |
| 208 | implementations of RCU, and allows trading off cache misses |
| 209 | against lock contention. Systems that synchronize their |
| 210 | scheduling-clock interrupts for energy-efficiency reasons will |
| 211 | want the default because the smaller leaf-level fanout keeps |
| 212 | lock contention levels acceptably low. Very large systems |
| 213 | (hundreds or thousands of CPUs) will instead want to set this |
| 214 | value to the maximum value possible in order to reduce the |
| 215 | number of cache misses incurred during RCU's grace-period |
| 216 | initialization. These systems tend to run CPU-bound, and thus |
| 217 | are not helped by synchronized interrupts, and thus tend to |
| 218 | skew them, which reduces lock contention enough that large |
| 219 | leaf-level fanouts work well. That said, setting leaf-level |
| 220 | fanout to a large number will likely cause problematic |
| 221 | lock contention on the leaf-level rcu_node structures unless |
| 222 | you boot with the skew_tick kernel parameter. |
| 223 | |
| 224 | Select a specific number if testing RCU itself. |
| 225 | |
| 226 | Select the maximum permissible value for large systems, but |
| 227 | please understand that you may also need to set the skew_tick |
| 228 | kernel boot parameter to avoid contention on the rcu_node |
| 229 | structure's locks. |
| 230 | |
| 231 | Take the default if unsure. |
| 232 | |
| 233 | config RCU_BOOST |
| 234 | bool "Enable RCU priority boosting" |
| 235 | depends on (RT_MUTEXES && PREEMPT_RCU && RCU_EXPERT) || PREEMPT_RT |
| 236 | default y if PREEMPT_RT |
| 237 | help |
| 238 | This option boosts the priority of preempted RCU readers that |
| 239 | block the current preemptible RCU grace period for too long. |
| 240 | This option also prevents heavy loads from blocking RCU |
| 241 | callback invocation. |
| 242 | |
| 243 | Say Y here if you are working with real-time apps or heavy loads |
| 244 | Say N here if you are unsure. |
| 245 | |
| 246 | config RCU_BOOST_DELAY |
| 247 | int "Milliseconds to delay boosting after RCU grace-period start" |
| 248 | range 0 3000 |
| 249 | depends on RCU_BOOST |
| 250 | default 500 |
| 251 | help |
| 252 | This option specifies the time to wait after the beginning of |
| 253 | a given grace period before priority-boosting preempted RCU |
| 254 | readers blocking that grace period. Note that any RCU reader |
| 255 | blocking an expedited RCU grace period is boosted immediately. |
| 256 | |
| 257 | Accept the default if unsure. |
| 258 | |
| 259 | config RCU_EXP_KTHREAD |
| 260 | bool "Perform RCU expedited work in a real-time kthread" |
| 261 | depends on RCU_BOOST && RCU_EXPERT |
| 262 | default !PREEMPT_RT && NR_CPUS <= 32 |
| 263 | help |
| 264 | Use this option to further reduce the latencies of expedited |
| 265 | grace periods at the expense of being more disruptive. |
| 266 | |
| 267 | This option is disabled by default on PREEMPT_RT=y kernels which |
| 268 | disable expedited grace periods after boot by unconditionally |
| 269 | setting rcupdate.rcu_normal_after_boot=1. |
| 270 | |
| 271 | Accept the default if unsure. |
| 272 | |
| 273 | config RCU_NOCB_CPU |
| 274 | bool "Offload RCU callback processing from boot-selected CPUs" |
| 275 | depends on TREE_RCU |
| 276 | depends on RCU_EXPERT || NO_HZ_FULL |
| 277 | default n |
| 278 | help |
| 279 | Use this option to reduce OS jitter for aggressive HPC or |
| 280 | real-time workloads. It can also be used to offload RCU |
| 281 | callback invocation to energy-efficient CPUs in battery-powered |
| 282 | asymmetric multiprocessors. The price of this reduced jitter |
| 283 | is that the overhead of call_rcu() increases and that some |
| 284 | workloads will incur significant increases in context-switch |
| 285 | rates. |
| 286 | |
| 287 | This option offloads callback invocation from the set of |
| 288 | CPUs specified at boot time by the rcu_nocbs parameter. |
| 289 | For each such CPU, a kthread ("rcuox/N") will be created to |
| 290 | invoke callbacks, where the "N" is the CPU being offloaded, |
| 291 | and where the "x" is "p" for RCU-preempt (PREEMPTION kernels) |
| 292 | and "s" for RCU-sched (!PREEMPTION kernels). This option |
| 293 | also creates another kthread for each sqrt(nr_cpu_ids) CPUs |
| 294 | ("rcuog/N", where N is the first CPU in that group to come |
| 295 | online), which handles grace periods for its group. Nothing |
| 296 | prevents these kthreads from running on the specified CPUs, |
| 297 | but (1) the kthreads may be preempted between each callback, |
| 298 | and (2) affinity or cgroups can be used to force the kthreads |
| 299 | to run on whatever set of CPUs is desired. |
| 300 | |
| 301 | The sqrt(nr_cpu_ids) grouping may be overridden using the |
| 302 | rcutree.rcu_nocb_gp_stride kernel boot parameter. This can |
| 303 | be especially helpful for smaller numbers of CPUs, where |
| 304 | sqrt(nr_cpu_ids) can be a bit of a blunt instrument. |
| 305 | |
| 306 | Say Y here if you need reduced OS jitter, despite added overhead. |
| 307 | Say N here if you are unsure. |
| 308 | |
| 309 | config RCU_NOCB_CPU_DEFAULT_ALL |
| 310 | bool "Offload RCU callback processing from all CPUs by default" |
| 311 | depends on RCU_NOCB_CPU |
| 312 | default n |
| 313 | help |
| 314 | Use this option to offload callback processing from all CPUs |
| 315 | by default, in the absence of the rcu_nocbs or nohz_full boot |
| 316 | parameter. This also avoids the need to use any boot parameters |
| 317 | to achieve the effect of offloading all CPUs on boot. |
| 318 | |
| 319 | Say Y here if you want offload all CPUs by default on boot. |
| 320 | Say N here if you are unsure. |
| 321 | |
| 322 | config RCU_NOCB_CPU_CB_BOOST |
| 323 | bool "Offload RCU callback from real-time kthread" |
| 324 | depends on RCU_NOCB_CPU && RCU_BOOST |
| 325 | default y if PREEMPT_RT |
| 326 | help |
| 327 | Use this option to invoke offloaded callbacks as SCHED_FIFO |
| 328 | to avoid starvation by heavy SCHED_OTHER background load. |
| 329 | Of course, running as SCHED_FIFO during callback floods will |
| 330 | cause the rcuo[ps] kthreads to monopolize the CPU for hundreds |
| 331 | of milliseconds or more. Therefore, when enabling this option, |
| 332 | it is your responsibility to ensure that latency-sensitive |
| 333 | tasks either run with higher priority or run on some other CPU. |
| 334 | |
| 335 | Say Y here if you want to set RT priority for offloading kthreads. |
| 336 | Say N here if you are building a !PREEMPT_RT kernel and are unsure. |
| 337 | |
| 338 | config RCU_LAZY |
| 339 | bool "RCU callback lazy invocation functionality" |
| 340 | depends on RCU_NOCB_CPU |
| 341 | default n |
| 342 | help |
| 343 | To save power, batch RCU callbacks and delay starting the |
| 344 | corresponding grace period for multiple seconds. The grace |
| 345 | period will be started after this delay, in case of memory |
| 346 | pressure, or if the corresponding CPU's callback list grows |
| 347 | too large. |
| 348 | |
| 349 | These delays happen only on rcu_nocbs CPUs, that is, CPUs |
| 350 | whose callbacks have been offloaded. |
| 351 | |
| 352 | Use the rcutree.enable_rcu_lazy=0 kernel-boot parameter to |
| 353 | globally disable these delays. |
| 354 | |
| 355 | config RCU_LAZY_DEFAULT_OFF |
| 356 | bool "Turn RCU lazy invocation off by default" |
| 357 | depends on RCU_LAZY |
| 358 | default n |
| 359 | help |
| 360 | Build the kernel with CONFIG_RCU_LAZY=y, but cause the kernel |
| 361 | to boot with these energy-efficiency delays disabled. Use the |
| 362 | rcutree.enable_rcu_lazy=0 kernel-boot parameter to override |
| 363 | the this option at boot time, thus re-enabling these delays. |
| 364 | |
| 365 | config RCU_DOUBLE_CHECK_CB_TIME |
| 366 | bool "RCU callback-batch backup time check" |
| 367 | depends on RCU_EXPERT |
| 368 | default n |
| 369 | help |
| 370 | Use this option to provide more precise enforcement of the |
| 371 | rcutree.rcu_resched_ns module parameter in situations where |
| 372 | a single RCU callback might run for hundreds of microseconds, |
| 373 | thus defeating the 32-callback batching used to amortize the |
| 374 | cost of the fine-grained but expensive local_clock() function. |
| 375 | |
| 376 | This option rounds rcutree.rcu_resched_ns up to the next |
| 377 | jiffy, and overrides the 32-callback batching if this limit |
| 378 | is exceeded. |
| 379 | |
| 380 | Say Y here if you need tighter callback-limit enforcement. |
| 381 | Say N here if you are unsure. |
| 382 | |
| 383 | endmenu # "RCU Subsystem" |