Merge branch 'core-rcu-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
Merge commit. Changes below are compared with the first parent: 462bf234.
Changes
137 files changed · +2450 −1074
Only the first 600 diff lines are shown. Download the complete patch.
Documentation/RCU/RTFP.txt
View filediff --git a/Documentation/RCU/RTFP.txt b/Documentation/RCU/RTFP.txt
index 273e654..2f0fcb2 100644
--- a/Documentation/RCU/RTFP.txt
+++ b/Documentation/RCU/RTFP.txt
@@ -31,6 +31,14 @@ has lapsed, so this approach may be used in non-GPL software, if desired.
(In contrast, implementation of RCU is permitted only in software licensed
under either GPL or LGPL. Sorry!!!)
+In 1987, Rashid et al. described lazy TLB-flush [RichardRashid87a].
+At first glance, this has nothing to do with RCU, but nevertheless
+this paper helped inspire the update-side batching used in the later
+RCU implementation in DYNIX/ptx. In 1988, Barbara Liskov published
+a description of Argus that noted that use of out-of-date values can
+be tolerated in some situations. Thus, this paper provides some early
+theoretical justification for use of stale data.
+
In 1990, Pugh [Pugh90] noted that explicitly tracking which threads
were reading a given data structure permitted deferred free to operate
in the presence of non-terminating threads. However, this explicit
@@ -41,11 +49,11 @@ providing a fine-grained locking design, however, it would be interesting
to see how much of the performance advantage reported in 1990 remains
today.
-At about this same time, Adams [Adams91] described ``chaotic relaxation'',
-where the normal barriers between successive iterations of convergent
-numerical algorithms are relaxed, so that iteration $n$ might use
-data from iteration $n-1$ or even $n-2$. This introduces error,
-which typically slows convergence and thus increases the number of
+At about this same time, Andrews [Andrews91textbook] described ``chaotic
+relaxation'', where the normal barriers between successive iterations
+of convergent numerical algorithms are relaxed, so that iteration $n$
+might use data from iteration $n-1$ or even $n-2$. This introduces
+error, which typically slows convergence and thus increases the number of
iterations required. However, this increase is sometimes more than made
up for by a reduction in the number of expensive barrier operations,
which are otherwise required to synchronize the threads at the end
@@ -55,7 +63,8 @@ is thus inapplicable to most data structures in operating-system kernels.
In 1992, Henry (now Alexia) Massalin completed a dissertation advising
parallel programmers to defer processing when feasible to simplify
-synchronization. RCU makes extremely heavy use of this advice.
+synchronization [HMassalinPhD]. RCU makes extremely heavy use of
+this advice.
In 1993, Jacobson [Jacobson93] verbally described what is perhaps the
simplest deferred-free technique: simply waiting a fixed amount of time
@@ -90,27 +99,29 @@ mechanism, which is quite similar to RCU [Gamsa99]. These operating
systems made pervasive use of RCU in place of "existence locks", which
greatly simplifies locking hierarchies and helps avoid deadlocks.
-2001 saw the first RCU presentation involving Linux [McKenney01a]
-at OLS. The resulting abundance of RCU patches was presented the
-following year [McKenney02a], and use of RCU in dcache was first
-described that same year [Linder02a].
+The year 2000 saw an email exchange that would likely have
+led to yet another independent invention of something like RCU
+[RustyRussell2000a,RustyRussell2000b]. Instead, 2001 saw the first
+RCU presentation involving Linux [McKenney01a] at OLS. The resulting
+abundance of RCU patches was presented the following year [McKenney02a],
+and use of RCU in dcache was first described that same year [Linder02a].
Also in 2002, Michael [Michael02b,Michael02a] presented "hazard-pointer"
techniques that defer the destruction of data structures to simplify
non-blocking synchronization (wait-free synchronization, lock-free
synchronization, and obstruction-free synchronization are all examples of
-non-blocking synchronization). In particular, this technique eliminates
-locking, reduces contention, reduces memory latency for readers, and
-parallelizes pipeline stalls and memory latency for writers. However,
-these techniques still impose significant read-side overhead in the
-form of memory barriers. Researchers at Sun worked along similar lines
-in the same timeframe [HerlihyLM02]. These techniques can be thought
-of as inside-out reference counts, where the count is represented by the
-number of hazard pointers referencing a given data structure rather than
-the more conventional counter field within the data structure itself.
-The key advantage of inside-out reference counts is that they can be
-stored in immortal variables, thus allowing races between access and
-deletion to be avoided.
+non-blocking synchronization). The corresponding journal article appeared
+in 2004 [MagedMichael04a]. This technique eliminates locking, reduces
+contention, reduces memory latency for readers, and parallelizes pipeline
+stalls and memory latency for writers. However, these techniques still
+impose significant read-side overhead in the form of memory barriers.
+Researchers at Sun worked along similar lines in the same timeframe
+[HerlihyLM02]. These techniques can be thought of as inside-out reference
+counts, where the count is represented by the number of hazard pointers
+referencing a given data structure rather than the more conventional
+counter field within the data structure itself. The key advantage
+of inside-out reference counts is that they can be stored in immortal
+variables, thus allowing races between access and deletion to be avoided.
By the same token, RCU can be thought of as a "bulk reference count",
where some form of reference counter covers all reference by a given CPU
@@ -123,8 +134,10 @@ can be thought of in other terms as well.
In 2003, the K42 group described how RCU could be used to create
hot-pluggable implementations of operating-system functions [Appavoo03a].
-Later that year saw a paper describing an RCU implementation of System
-V IPC [Arcangeli03], and an introduction to RCU in Linux Journal
+Later that year saw a paper describing an RCU implementation
+of System V IPC [Arcangeli03] (following up on a suggestion by
+Hugh Dickins [Dickins02a] and an implementation by Mingming Cao
+[MingmingCao2002IPCRCU]), and an introduction to RCU in Linux Journal
[McKenney03a].
2004 has seen a Linux-Journal article on use of RCU in dcache
@@ -383,6 +396,21 @@ for Programming Languages and Operating Systems}"
}
}
+@phdthesis{HMassalinPhD
+,author="H. Massalin"
+,title="Synthesis: An Efficient Implementation of Fundamental Operating
+System Services"
+,school="Columbia University"
+,address="New York, NY"
+,year="1992"
+,annotation={
+ Mondo optimizing compiler.
+ Wait-free stuff.
+ Good advice: defer work to avoid synchronization. See page 90
+ (PDF page 106), Section 5.4, fourth bullet point.
+}
+}
+
@unpublished{Jacobson93
,author="Van Jacobson"
,title="Avoid Read-Side Locking Via Delayed Free"
@@ -671,6 +699,20 @@ Orran Krieger and Rusty Russell and Dipankar Sarma and Maneesh Soni"
[Viewed October 18, 2004]"
}
+@conference{Michael02b
+,author="Maged M. Michael"
+,title="High Performance Dynamic Lock-Free Hash Tables and List-Based Sets"
+,Year="2002"
+,Month="August"
+,booktitle="{Proceedings of the 14\textsuperscript{th} Annual ACM
+Symposium on Parallel
+Algorithms and Architecture}"
+,pages="73-82"
+,annotation={
+Like the title says...
+}
+}
+
@Conference{Linder02a
,Author="Hanna Linder and Dipankar Sarma and Maneesh Soni"
,Title="Scalability of the Directory Entry Cache"
@@ -727,6 +769,24 @@ Andrea Arcangeli and Andi Kleen and Orran Krieger and Rusty Russell"
}
}
+@conference{Michael02a
+,author="Maged M. Michael"
+,title="Safe Memory Reclamation for Dynamic Lock-Free Objects Using Atomic
+Reads and Writes"
+,Year="2002"
+,Month="August"
+,booktitle="{Proceedings of the 21\textsuperscript{st} Annual ACM
+Symposium on Principles of Distributed Computing}"
+,pages="21-30"
+,annotation={
+ Each thread keeps an array of pointers to items that it is
+ currently referencing. Sort of an inside-out garbage collection
+ mechanism, but one that requires the accessing code to explicitly
+ state its needs. Also requires read-side memory barriers on
+ most architectures.
+}
+}
+
@unpublished{Dickins02a
,author="Hugh Dickins"
,title="Use RCU for System-V IPC"
@@ -735,6 +795,17 @@ Andrea Arcangeli and Andi Kleen and Orran Krieger and Rusty Russell"
,note="private communication"
}
+@InProceedings{HerlihyLM02
+,author={Maurice Herlihy and Victor Luchangco and Mark Moir}
+,title="The Repeat Offender Problem: A Mechanism for Supporting Dynamic-Sized,
+Lock-Free Data Structures"
+,booktitle={Proceedings of 16\textsuperscript{th} International
+Symposium on Distributed Computing}
+,year=2002
+,month="October"
+,pages="339-353"
+}
+
@unpublished{Sarma02b
,Author="Dipankar Sarma"
,Title="Some dcache\_rcu benchmark numbers"
@@ -749,6 +820,19 @@ Andrea Arcangeli and Andi Kleen and Orran Krieger and Rusty Russell"
}
}
+@unpublished{MingmingCao2002IPCRCU
+,Author="Mingming Cao"
+,Title="[PATCH]updated ipc lock patch"
+,month="October"
+,year="2002"
+,note="Available:
+\url{https://lkml.org/lkml/2002/10/24/262}
+[Viewed February 15, 2014]"
+,annotation={
+ Mingming Cao's patch to introduce RCU to SysV IPC.
+}
+}
+
@unpublished{LinusTorvalds2003a
,Author="Linus Torvalds"
,Title="Re: {[PATCH]} small fixes in brlock.h"
@@ -982,6 +1066,23 @@ Realtime Applications"
}
}
+@article{MagedMichael04a
+,author="Maged M. Michael"
+,title="Hazard Pointers: Safe Memory Reclamation for Lock-Free Objects"
+,Year="2004"
+,Month="June"
+,journal="IEEE Transactions on Parallel and Distributed Systems"
+,volume="15"
+,number="6"
+,pages="491-504"
+,url="Available:
+\url{http://www.research.ibm.com/people/m/michael/ieeetpds-2004.pdf}
+[Viewed March 1, 2005]"
+,annotation={
+ New canonical hazard-pointer citation.
+}
+}
+
@phdthesis{PaulEdwardMcKenneyPhD
,author="Paul E. McKenney"
,title="Exploiting Deferred Destruction:
Documentation/RCU/checklist.txt
View filediff --git a/Documentation/RCU/checklist.txt b/Documentation/RCU/checklist.txt
index 9126619..9d10d1d 100644
--- a/Documentation/RCU/checklist.txt
+++ b/Documentation/RCU/checklist.txt
@@ -256,10 +256,10 @@ over a rather long period of time, but improvements are always welcome!
variations on this theme.
b. Limiting update rate. For example, if updates occur only
- once per hour, then no explicit rate limiting is required,
- unless your system is already badly broken. The dcache
- subsystem takes this approach -- updates are guarded
- by a global lock, limiting their rate.
+ once per hour, then no explicit rate limiting is
+ required, unless your system is already badly broken.
+ Older versions of the dcache subsystem take this approach,
+ guarding updates with a global lock, limiting their rate.
c. Trusted update -- if updates can only be done manually by
superuser or some other trusted user, then it might not
@@ -268,7 +268,8 @@ over a rather long period of time, but improvements are always welcome!
the machine.
d. Use call_rcu_bh() rather than call_rcu(), in order to take
- advantage of call_rcu_bh()'s faster grace periods.
+ advantage of call_rcu_bh()'s faster grace periods. (This
+ is only a partial solution, though.)
e. Periodically invoke synchronize_rcu(), permitting a limited
number of updates per grace period.
@@ -276,6 +277,13 @@ over a rather long period of time, but improvements are always welcome!
The same cautions apply to call_rcu_bh(), call_rcu_sched(),
call_srcu(), and kfree_rcu().
+ Note that although these primitives do take action to avoid memory
+ exhaustion when any given CPU has too many callbacks, a determined
+ user could still exhaust memory. This is especially the case
+ if a system with a large number of CPUs has been configured to
+ offload all of its RCU callbacks onto a single CPU, or if the
+ system has relatively little free memory.
+
9. All RCU list-traversal primitives, which include
rcu_dereference(), list_for_each_entry_rcu(), and
list_for_each_safe_rcu(), must be either within an RCU read-side
Documentation/kernel-per-CPU-kthreads.txt
View filediff --git a/Documentation/kernel-per-CPU-kthreads.txt b/Documentation/kernel-per-CPU-kthreads.txt
index 827104f..f3cd299 100644
--- a/Documentation/kernel-per-CPU-kthreads.txt
+++ b/Documentation/kernel-per-CPU-kthreads.txt
@@ -162,7 +162,18 @@ Purpose: Execute workqueue requests
To reduce its OS jitter, do any of the following:
1. Run your workload at a real-time priority, which will allow
preempting the kworker daemons.
-2. Do any of the following needed to avoid jitter that your
+2. A given workqueue can be made visible in the sysfs filesystem
+ by passing the WQ_SYSFS to that workqueue's alloc_workqueue().
+ Such a workqueue can be confined to a given subset of the
+ CPUs using the /sys/devices/virtual/workqueue/*/cpumask sysfs
+ files. The set of WQ_SYSFS workqueues can be displayed using
+ "ls sys/devices/virtual/workqueue". That said, the workqueues
+ maintainer would like to caution people against indiscriminately
+ sprinkling WQ_SYSFS across all the workqueues. The reason for
+ caution is that it is easy to add WQ_SYSFS, but because sysfs is
+ part of the formal user/kernel API, it can be nearly impossible
+ to remove it, even if its addition was a mistake.
+3. Do any of the following needed to avoid jitter that your
application cannot tolerate:
a. Build your kernel with CONFIG_SLUB=y rather than
CONFIG_SLAB=y, thus avoiding the slab allocator's periodic
Documentation/memory-barriers.txt
View filediff --git a/Documentation/memory-barriers.txt b/Documentation/memory-barriers.txt
index 102dc19..11c1d20 100644
--- a/Documentation/memory-barriers.txt
+++ b/Documentation/memory-barriers.txt
@@ -608,26 +608,30 @@ as follows:
b = p; /* BUG: Compiler can reorder!!! */
do_something();
-The solution is again ACCESS_ONCE(), which preserves the ordering between
-the load from variable 'a' and the store to variable 'b':
+The solution is again ACCESS_ONCE() and barrier(), which preserves the
+ordering between the load from variable 'a' and the store to variable 'b':
q = ACCESS_ONCE(a);
if (q) {
+ barrier();
ACCESS_ONCE(b) = p;
do_something();
} else {
+ barrier();
ACCESS_ONCE(b) = p;
do_something_else();
}
-You could also use barrier() to prevent the compiler from moving
-the stores to variable 'b', but barrier() would not prevent the
-compiler from proving to itself that a==1 always, so ACCESS_ONCE()
-is also needed.
+The initial ACCESS_ONCE() is required to prevent the compiler from
+proving the value of 'a', and the pair of barrier() invocations are
+required to prevent the compiler from pulling the two identical stores
+to 'b' out from the legs of the "if" statement.
It is important to note that control dependencies absolutely require a
a conditional. For example, the following "optimized" version of
-the above example breaks ordering:
+the above example breaks ordering, which is why the barrier() invocations
+are absolutely required if you have identical stores in both legs of
+the "if" statement:
q = ACCESS_ONCE(a);
ACCESS_ONCE(b) = p; /* BUG: No ordering vs. load from a!!! */
@@ -643,9 +647,11 @@ It is of course legal for the prior load to be part of the conditional,
for example, as follows:
if (ACCESS_ONCE(a) > 0) {
+ barrier();
ACCESS_ONCE(b) = q / 2;
do_something();
} else {
+ barrier();
ACCESS_ONCE(b) = q / 3;
do_something_else();
}
@@ -659,9 +665,11 @@ the needed conditional. For example:
q = ACCESS_ONCE(a);
if (q % MAX) {
+ barrier();
ACCESS_ONCE(b) = p;
do_something();
} else {
+ barrier();
ACCESS_ONCE(b) = p;
do_something_else();
}
@@ -723,8 +731,13 @@ In summary:
use smb_rmb(), smp_wmb(), or, in the case of prior stores and
later loads, smp_mb().
+ (*) If both legs of the "if" statement begin with identical stores
+ to the same variable, a barrier() statement is required at the
+ beginning of each leg of the "if" statement.
+
(*) Control dependencies require at least one run-time conditional
- between the prior load and the subsequent store. If the compiler
+ between the prior load and the subsequent store, and this
+ conditional must involve the prior load. If the compiler
is able to optimize the conditional away, it will have also
optimized away the ordering. Careful use of ACCESS_ONCE() can
help to preserve the needed conditional.
@@ -1249,6 +1262,23 @@ The ACCESS_ONCE() function can prevent any number of optimizations that,
while perfectly safe in single-threaded code, can be fatal in concurrent
code. Here are some examples of these sorts of optimizations:
+ (*) The compiler is within its rights to reorder loads and stores
+ to the same variable, and in some cases, the CPU is within its
+ rights to reorder loads to the same variable. This means that
+ the following code:
+
+ a[0] = x;
+ a[1] = x;
+
+ Might result in an older value of x stored in a[1] than in a[0].
+ Prevent both the compiler and the CPU from doing this as follows:
+
+ a[0] = ACCESS_ONCE(x);
+ a[1] = ACCESS_ONCE(x);
+
+ In short, ACCESS_ONCE() provides cache coherence for accesses from
+ multiple CPUs to a single variable.
+
(*) The compiler is within its rights to merge successive loads from
the same variable. Such merging can cause the compiler to "optimize"
the following code:
@@ -1644,12 +1674,12 @@ for each construct. These operations all imply certain barriers:
Memory operations issued after the ACQUIRE will be completed after the
ACQUIRE operation has completed.
- Memory operations issued before the ACQUIRE may be completed after the
- ACQUIRE operation has completed. An smp_mb__before_spinlock(), combined
- with a following ACQUIRE, orders prior loads against subsequent stores and
- stores and prior stores against subsequent stores. Note that this is
- weaker than smp_mb()! The smp_mb__before_spinlock() primitive is free on
- many architectures.
+ Memory operations issued before the ACQUIRE may be completed after
+ the ACQUIRE operation has completed. An smp_mb__before_spinlock(),
+ combined with a following ACQUIRE, orders prior loads against
+ subsequent loads and stores and also orders prior stores against
+ subsequent stores. Note that this is weaker than smp_mb()! The
+ smp_mb__before_spinlock() primitive is free on many architectures.
(2) RELEASE operation implication:
@@ -1694,24 +1724,21 @@ may occur as:
ACQUIRE M, STORE *B, STORE *A, RELEASE M
-This same reordering can of course occur if the lock's ACQUIRE and RELEASE are
-to the same lock variable, but only from the perspective of another CPU not
-holding that lock.
-
-In short, a RELEASE followed by an ACQUIRE may -not- be assumed to be a full
-memory barrier because it is possible for a preceding RELEASE to pass a
-later ACQUIRE from the viewpoint of the CPU, but not from the viewpoint
-of the compiler. Note that deadlocks cannot be introduced by this
-interchange because if such a deadlock threatened, the RELEASE would
-simply complete.
-
-If it is necessary for a RELEASE-ACQUIRE pair to produce a full barrier, the
-ACQUIRE can be followed by an smp_mb__after_unlock_lock() invocation. This
-will produce a full barrier if either (a) the RELEASE and the ACQUIRE are
-executed by the same CPU or task, or (b) the RELEASE and ACQUIRE act on the
-same variable. The smp_mb__after_unlock_lock() primitive is free on many
-architectures. Without smp_mb__after_unlock_lock(), the critical sections
-corresponding to the RELEASE and the ACQUIRE can cross:
+When the ACQUIRE and RELEASE are a lock acquisition and release,
+respectively, this same reordering can occur if the lock's ACQUIRE and
+RELEASE are to the same lock variable, but only from the perspective of
+another CPU not holding that lock. In short, a ACQUIRE followed by an
+RELEASE may -not- be assumed to be a full memory barrier.
+
+Similarly, the reverse case of a RELEASE followed by an ACQUIRE does not
+imply a full memory barrier. If it is necessary for a RELEASE-ACQUIRE
+pair to produce a full barrier, the ACQUIRE can be followed by an
+smp_mb__after_unlock_lock() invocation. This will produce a full barrier
+if either (a) the RELEASE and the ACQUIRE are executed by the same
+CPU or task, or (b) the RELEASE and ACQUIRE act on the same variable.
+The smp_mb__after_unlock_lock() primitive is free on many architectures.
+Without smp_mb__after_unlock_lock(), the CPU's execution of the critical
+sections corresponding to the RELEASE and the ACQUIRE can cross, so that:
*A = a;
RELEASE M
@@ -1722,7 +1749,36 @@ could occur as:
ACQUIRE N, STORE *B, STORE *A, RELEASE M
-With smp_mb__after_unlock_lock(), they cannot, so that:
+It might appear that this reordering could introduce a deadlock.
+However, this cannot happen because if such a deadlock threatened,
+the RELEASE would simply complete, thereby avoiding the deadlock.
+
+ Why does this work?
+
+ One key point is that we are only talking about the CPU doing
+ the reordering, not the compiler. If the compiler (or, for
+ that matter, the developer) switched the operations, deadlock
+ -could- occur.
+
+ But suppose the CPU reordered the operations. In this case,
+ the unlock precedes the lock in the assembly code. The CPU
+ simply elected to try executing the later lock operation first.
+ If there is a deadlock, this lock operation will simply spin (or
+ try to sleep, but more on that later). The CPU will eventually
+ execute the unlock operation (which preceded the lock operation
+ in the assembly code), which will unravel the potential deadlock,
+ allowing the lock operation to succeed.
+
+ But what if the lock is a sleeplock? In that case, the code will
+ try to enter the scheduler, where it will eventually encounter
+ a memory barrier, which will force the earlier unlock operation
+ to complete, again unraveling the deadlock. There might be
+ a sleep-unlock race, but the locking primitive needs to resolve
+ such races properly in any case.
+
+With smp_mb__after_unlock_lock(), the two critical sections cannot overlap.
+For example, with the following code, the store to *A will always be
+seen by other CPUs before the store to *B:
*A = a;
RELEASE M
@@ -1730,13 +1786,18 @@ With smp_mb__after_unlock_lock(), they cannot, so that:
smp_mb__after_unlock_lock();
*B = b;
-will always occur as either of the following:
+The operations will always occur in one of the following orders:
- STORE *A, RELEASE, ACQUIRE, STORE *B
- STORE *A, ACQUIRE, RELEASE, STORE *B
+ STORE *A, RELEASE, ACQUIRE, smp_mb__after_unlock_lock(), STORE *B
+ STORE *A, ACQUIRE, RELEASE, smp_mb__after_unlock_lock(), STORE *B
+ ACQUIRE, STORE *A, RELEASE, smp_mb__after_unlock_lock(), STORE *B
If the RELEASE and ACQUIRE were instead both operating on the same lock
-variable, only the first of these two alternatives can occur.
+variable, only the first of these alternatives can occur. In addition,
+the more strongly ordered systems may rule out some of the above orders.
+But in any case, as noted earlier, the smp_mb__after_unlock_lock()
+ensures that the store to *A will always be seen as happening before
+the store to *B.
Locks and semaphores may not provide any guarantee of ordering on UP compiled
systems, and so cannot be counted on in such a situation to actually achieve
@@ -2757,7 +2818,7 @@ in that order, but, without intervention, the sequence may have almost any
combination of elements combined or discarded, provided the program's view of
the world remains consistent. Note that ACCESS_ONCE() is -not- optional
in the above example, as there are architectures where a given CPU might
-interchange successive loads to the same location. On such architectures,
+reorder successive loads to the same location. On such architectures,
ACCESS_ONCE() does whatever is necessary to prevent this, for example, on
Itanium the volatile casts used by ACCESS_ONCE() cause GCC to emit the
special ld.acq and st.rel instructions that prevent such reordering.
fs/file.c
View filediff --git a/fs/file.c b/fs/file.c
index eb56a13..b61293b 100644
--- a/fs/file.c
+++ b/fs/file.c
@@ -497,7 +497,7 @@ repeat:
error = fd;
#if 1
/* Sanity check */
- if (rcu_dereference_raw(fdt->fd[fd]) != NULL) {
+ if (rcu_access_pointer(fdt->fd[fd]) != NULL) {
printk(KERN_WARNING "alloc_fd: slot %d not NULL!\n", fd);
rcu_assign_pointer(fdt->fd[fd], NULL);
}
include/linux/rculist.h
View filediff --git a/include/linux/rculist.h b/include/linux/rculist.h
index dbaf990..8183b46 100644
--- a/include/linux/rculist.h
+++ b/include/linux/rculist.h
@@ -247,9 +247,10 @@ static inline void list_splice_init_rcu(struct list_head *list,
* primitives such as list_add_rcu() as long as it's guarded by rcu_read_lock().
*/
#define list_entry_rcu(ptr, type, member) \
- ({typeof (*ptr) __rcu *__ptr = (typeof (*ptr) __rcu __force *)ptr; \
- container_of((typeof(ptr))rcu_dereference_raw(__ptr), type, member); \
- })
+({ \
+ typeof(*ptr) __rcu *__ptr = (typeof(*ptr) __rcu __force *)ptr; \
+ container_of((typeof(ptr))rcu_dereference_raw(__ptr), type, member); \
+})
/**
* Where are list_empty_rcu() and list_first_entry_rcu()?
@@ -285,11 +286,11 @@ static inline void list_splice_init_rcu(struct list_head *list,
* primitives such as list_add_rcu() as long as it's guarded by rcu_read_lock().
*/
#define list_first_or_null_rcu(ptr, type, member) \
- ({struct list_head *__ptr = (ptr); \
- struct list_head *__next = ACCESS_ONCE(__ptr->next); \
- likely(__ptr != __next) ? \
- list_entry_rcu(__next, type, member) : NULL; \
- })
+({ \
+ struct list_head *__ptr = (ptr); \
+ struct list_head *__next = ACCESS_ONCE(__ptr->next); \
+ likely(__ptr != __next) ? list_entry_rcu(__next, type, member) : NULL; \
+})
/**
* list_for_each_entry_rcu - iterate over rcu list of given type
include/linux/rcupdate.h
View filediff --git a/include/linux/rcupdate.h b/include/linux/rcupdate.h
index adff3c9..00a7fd6 100644
--- a/include/linux/rcupdate.h
+++ b/include/linux/rcupdate.h
@@ -12,8 +12,8 @@
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
+ * along with this program; if not, you can access it online at
+ * http://www.gnu.org/licenses/gpl-2.0.html.
*
* Copyright IBM Corporation, 2001
*
@@ -44,7 +44,9 @@
This diff is incomplete. Download the complete patch.
include/linux/rcutiny.h
View fileThis diff is incomplete. Download the complete patch.
include/linux/rcutree.h
View fileThis diff is incomplete. Download the complete patch.
include/linux/srcu.h
View fileThis diff is incomplete. Download the complete patch.
include/linux/torture.h
View fileThis diff is incomplete. Download the complete patch.
kernel/Makefile
View fileThis diff is incomplete. Download the complete patch.
kernel/ksysfs.c
View fileThis diff is incomplete. Download the complete patch.
kernel/locking/Makefile
View fileThis diff is incomplete. Download the complete patch.
kernel/locking/locktorture.c
View fileThis diff is incomplete. Download the complete patch.
kernel/notifier.c
View fileThis diff is incomplete. Download the complete patch.
kernel/rcu/Makefile
View fileThis diff is incomplete. Download the complete patch.
kernel/rcu/rcu.h
View fileThis diff is incomplete. Download the complete patch.
kernel/rcu/rcutorture.c
View fileThis diff is incomplete. Download the complete patch.
kernel/rcu/srcu.c
View fileThis diff is incomplete. Download the complete patch.
kernel/rcu/tiny.c
View fileThis diff is incomplete. Download the complete patch.
kernel/rcu/tiny_plugin.h
View fileThis diff is incomplete. Download the complete patch.
kernel/rcu/tree.c
View fileThis diff is incomplete. Download the complete patch.
kernel/rcu/tree.h
View fileThis diff is incomplete. Download the complete patch.
kernel/rcu/tree_plugin.h
View fileThis diff is incomplete. Download the complete patch.
kernel/rcu/tree_trace.c
View fileThis diff is incomplete. Download the complete patch.
kernel/rcu/update.c
View fileThis diff is incomplete. Download the complete patch.
kernel/torture.c
View fileThis diff is incomplete. Download the complete patch.
lib/Kconfig.debug
View fileThis diff is incomplete. Download the complete patch.
tools/testing/selftests/rcutorture/bin/functions.sh
View fileThis diff is incomplete. Download the complete patch.
tools/testing/selftests/rcutorture/bin/kvm-recheck-lock.sh
View fileThis diff is incomplete. Download the complete patch.
tools/testing/selftests/rcutorture/bin/kvm-recheck-rcu.sh
View fileThis diff is incomplete. Download the complete patch.
tools/testing/selftests/rcutorture/bin/kvm-recheck.sh
View fileThis diff is incomplete. Download the complete patch.
tools/testing/selftests/rcutorture/bin/kvm-test-1-run.sh
View fileThis diff is incomplete. Download the complete patch.
tools/testing/selftests/rcutorture/bin/kvm.sh
View fileThis diff is incomplete. Download the complete patch.
tools/testing/selftests/rcutorture/configs/lock/BUSTED
View fileThis diff is incomplete. Download the complete patch.
tools/testing/selftests/rcutorture/configs/lock/BUSTED.boot
View fileThis diff is incomplete. Download the complete patch.
tools/testing/selftests/rcutorture/configs/lock/CFLIST
View fileThis diff is incomplete. Download the complete patch.
tools/testing/selftests/rcutorture/configs/lock/CFcommon
View fileThis diff is incomplete. Download the complete patch.
tools/testing/selftests/rcutorture/configs/lock/LOCK01
View fileThis diff is incomplete. Download the complete patch.
tools/testing/selftests/rcutorture/configs/lock/ver_functions.sh
View fileThis diff is incomplete. Download the complete patch.
tools/testing/selftests/rcutorture/configs/rcu/BUSTED
View fileThis diff is incomplete. Download the complete patch.
tools/testing/selftests/rcutorture/configs/rcu/BUSTED.boot
View fileThis diff is incomplete. Download the complete patch.
tools/testing/selftests/rcutorture/configs/rcu/CFLIST
View fileThis diff is incomplete. Download the complete patch.
tools/testing/selftests/rcutorture/configs/rcu/CFcommon
View fileThis diff is incomplete. Download the complete patch.
tools/testing/selftests/rcutorture/configs/rcu/SRCU-N
View fileThis diff is incomplete. Download the complete patch.
tools/testing/selftests/rcutorture/configs/rcu/SRCU-N.boot
View fileThis diff is incomplete. Download the complete patch.
tools/testing/selftests/rcutorture/configs/rcu/SRCU-P
View fileThis diff is incomplete. Download the complete patch.
tools/testing/selftests/rcutorture/configs/rcu/SRCU-P.boot
View fileThis diff is incomplete. Download the complete patch.
tools/testing/selftests/rcutorture/configs/rcu/TINY01
View fileThis diff is incomplete. Download the complete patch.
tools/testing/selftests/rcutorture/configs/rcu/TINY02
View fileThis diff is incomplete. Download the complete patch.
tools/testing/selftests/rcutorture/configs/rcu/TREE01
View fileThis diff is incomplete. Download the complete patch.
tools/testing/selftests/rcutorture/configs/rcu/TREE01.boot
View fileThis diff is incomplete. Download the complete patch.
tools/testing/selftests/rcutorture/configs/rcu/TREE02
View fileThis diff is incomplete. Download the complete patch.
tools/testing/selftests/rcutorture/configs/rcu/TREE03
View fileThis diff is incomplete. Download the complete patch.
tools/testing/selftests/rcutorture/configs/rcu/TREE04
View fileThis diff is incomplete. Download the complete patch.
tools/testing/selftests/rcutorture/configs/rcu/TREE04.boot
View fileThis diff is incomplete. Download the complete patch.
tools/testing/selftests/rcutorture/configs/rcu/TREE05
View fileThis diff is incomplete. Download the complete patch.
tools/testing/selftests/rcutorture/configs/rcu/TREE05.boot
View fileThis diff is incomplete. Download the complete patch.
tools/testing/selftests/rcutorture/configs/rcu/TREE06
View fileThis diff is incomplete. Download the complete patch.
tools/testing/selftests/rcutorture/configs/rcu/TREE07
View fileThis diff is incomplete. Download the complete patch.
tools/testing/selftests/rcutorture/configs/rcu/TREE08
View fileThis diff is incomplete. Download the complete patch.
tools/testing/selftests/rcutorture/configs/rcu/TREE08-T
View fileThis diff is incomplete. Download the complete patch.
tools/testing/selftests/rcutorture/configs/rcu/TREE09
View fileThis diff is incomplete. Download the complete patch.
tools/testing/selftests/rcutorture/configs/rcu/v0.0/CFLIST
View fileThis diff is incomplete. Download the complete patch.
tools/testing/selftests/rcutorture/configs/rcu/v0.0/N1-S-T-NH-SD-SMP-HP
View fileThis diff is incomplete. Download the complete patch.
tools/testing/selftests/rcutorture/configs/rcu/v0.0/N2-2-t-nh-sd-SMP-hp
View fileThis diff is incomplete. Download the complete patch.
tools/testing/selftests/rcutorture/configs/rcu/v0.0/N3-3-T-nh-SD-SMP-hp
View fileThis diff is incomplete. Download the complete patch.
tools/testing/selftests/rcutorture/configs/rcu/v0.0/N4-A-t-NH-sd-SMP-HP
View fileThis diff is incomplete. Download the complete patch.
tools/testing/selftests/rcutorture/configs/rcu/v0.0/N5-U-T-NH-sd-SMP-hp
View fileThis diff is incomplete. Download the complete patch.
tools/testing/selftests/rcutorture/configs/rcu/v0.0/NT1-nh
View fileThis diff is incomplete. Download the complete patch.
tools/testing/selftests/rcutorture/configs/rcu/v0.0/NT3-NH
View fileThis diff is incomplete. Download the complete patch.
tools/testing/selftests/rcutorture/configs/rcu/v0.0/P1-S-T-NH-SD-SMP-HP
View fileThis diff is incomplete. Download the complete patch.
tools/testing/selftests/rcutorture/configs/rcu/v0.0/P2-2-t-nh-sd-SMP-hp
View fileThis diff is incomplete. Download the complete patch.
tools/testing/selftests/rcutorture/configs/rcu/v0.0/P3-3-T-nh-SD-SMP-hp
View fileThis diff is incomplete. Download the complete patch.
tools/testing/selftests/rcutorture/configs/rcu/v0.0/P4-A-t-NH-sd-SMP-HP
View fileThis diff is incomplete. Download the complete patch.
tools/testing/selftests/rcutorture/configs/rcu/v0.0/P5-U-T-NH-sd-SMP-hp
View fileThis diff is incomplete. Download the complete patch.
tools/testing/selftests/rcutorture/configs/rcu/v0.0/PT1-nh
View fileThis diff is incomplete. Download the complete patch.
tools/testing/selftests/rcutorture/configs/rcu/v0.0/PT2-NH
View fileThis diff is incomplete. Download the complete patch.
tools/testing/selftests/rcutorture/configs/rcu/v0.0/ver_functions.sh
View fileThis diff is incomplete. Download the complete patch.
tools/testing/selftests/rcutorture/configs/rcu/v3.12/CFLIST
View fileThis diff is incomplete. Download the complete patch.
tools/testing/selftests/rcutorture/configs/rcu/v3.12/N1-S-T-NH-SD-SMP-HP
View fileThis diff is incomplete. Download the complete patch.
tools/testing/selftests/rcutorture/configs/rcu/v3.12/N2-2-t-nh-sd-SMP-hp
View fileThis diff is incomplete. Download the complete patch.
tools/testing/selftests/rcutorture/configs/rcu/v3.12/N3-3-T-nh-SD-SMP-hp
View fileThis diff is incomplete. Download the complete patch.
tools/testing/selftests/rcutorture/configs/rcu/v3.12/N4-A-t-NH-sd-SMP-HP
View fileThis diff is incomplete. Download the complete patch.
tools/testing/selftests/rcutorture/configs/rcu/v3.12/N5-U-T-NH-sd-SMP-hp
View fileThis diff is incomplete. Download the complete patch.
tools/testing/selftests/rcutorture/configs/rcu/v3.12/N6---t-nh-SD-smp-hp
View fileThis diff is incomplete. Download the complete patch.
tools/testing/selftests/rcutorture/configs/rcu/v3.12/N7-4-T-NH-SD-SMP-HP
View fileThis diff is incomplete. Download the complete patch.
tools/testing/selftests/rcutorture/configs/rcu/v3.12/N8-2-T-NH-SD-SMP-HP
View fileThis diff is incomplete. Download the complete patch.
tools/testing/selftests/rcutorture/configs/rcu/v3.12/NT1-nh
View fileThis diff is incomplete. Download the complete patch.
tools/testing/selftests/rcutorture/configs/rcu/v3.12/NT3-NH
View fileThis diff is incomplete. Download the complete patch.
tools/testing/selftests/rcutorture/configs/rcu/v3.12/P1-S-T-NH-SD-SMP-HP
View fileThis diff is incomplete. Download the complete patch.
tools/testing/selftests/rcutorture/configs/rcu/v3.12/P2-2-t-nh-sd-SMP-hp
View fileThis diff is incomplete. Download the complete patch.
tools/testing/selftests/rcutorture/configs/rcu/v3.12/P3-3-T-nh-SD-SMP-hp
View fileThis diff is incomplete. Download the complete patch.
tools/testing/selftests/rcutorture/configs/rcu/v3.12/P4-A-t-NH-sd-SMP-HP
View fileThis diff is incomplete. Download the complete patch.
tools/testing/selftests/rcutorture/configs/rcu/v3.12/P5-U-T-NH-sd-SMP-hp
View fileThis diff is incomplete. Download the complete patch.
tools/testing/selftests/rcutorture/configs/rcu/v3.12/P6---t-nh-SD-smp-hp
View fileThis diff is incomplete. Download the complete patch.
tools/testing/selftests/rcutorture/configs/rcu/v3.12/P7-4-T-NH-SD-SMP-HP
View fileThis diff is incomplete. Download the complete patch.
tools/testing/selftests/rcutorture/configs/rcu/v3.12/P7-4-T-NH-SD-SMP-HP-all
View fileThis diff is incomplete. Download the complete patch.
tools/testing/selftests/rcutorture/configs/rcu/v3.12/P7-4-T-NH-SD-SMP-HP-none
View fileThis diff is incomplete. Download the complete patch.
tools/testing/selftests/rcutorture/configs/rcu/v3.12/P7-4-T-NH-SD-SMP-hp
View fileThis diff is incomplete. Download the complete patch.
tools/testing/selftests/rcutorture/configs/rcu/v3.12/PT1-nh
View fileThis diff is incomplete. Download the complete patch.
tools/testing/selftests/rcutorture/configs/rcu/v3.12/PT2-NH
View fileThis diff is incomplete. Download the complete patch.
tools/testing/selftests/rcutorture/configs/rcu/v3.3/CFLIST
View fileThis diff is incomplete. Download the complete patch.
tools/testing/selftests/rcutorture/configs/rcu/v3.3/N1-S-T-NH-SD-SMP-HP
View fileThis diff is incomplete. Download the complete patch.
tools/testing/selftests/rcutorture/configs/rcu/v3.3/N2-2-t-nh-sd-SMP-hp
View fileThis diff is incomplete. Download the complete patch.
tools/testing/selftests/rcutorture/configs/rcu/v3.3/N3-3-T-nh-SD-SMP-hp
View fileThis diff is incomplete. Download the complete patch.
tools/testing/selftests/rcutorture/configs/rcu/v3.3/N4-A-t-NH-sd-SMP-HP
View fileThis diff is incomplete. Download the complete patch.
tools/testing/selftests/rcutorture/configs/rcu/v3.3/N5-U-T-NH-sd-SMP-hp
View fileThis diff is incomplete. Download the complete patch.
tools/testing/selftests/rcutorture/configs/rcu/v3.3/NT1-nh
View fileThis diff is incomplete. Download the complete patch.
tools/testing/selftests/rcutorture/configs/rcu/v3.3/NT3-NH
View fileThis diff is incomplete. Download the complete patch.
tools/testing/selftests/rcutorture/configs/rcu/v3.3/P1-S-T-NH-SD-SMP-HP
View fileThis diff is incomplete. Download the complete patch.
tools/testing/selftests/rcutorture/configs/rcu/v3.3/P2-2-t-nh-sd-SMP-hp
View fileThis diff is incomplete. Download the complete patch.
tools/testing/selftests/rcutorture/configs/rcu/v3.3/P3-3-T-nh-SD-SMP-hp
View fileThis diff is incomplete. Download the complete patch.
tools/testing/selftests/rcutorture/configs/rcu/v3.3/P4-A-t-NH-sd-SMP-HP
View fileThis diff is incomplete. Download the complete patch.
tools/testing/selftests/rcutorture/configs/rcu/v3.3/P5-U-T-NH-sd-SMP-hp
View fileThis diff is incomplete. Download the complete patch.
tools/testing/selftests/rcutorture/configs/rcu/v3.3/PT1-nh
View fileThis diff is incomplete. Download the complete patch.
tools/testing/selftests/rcutorture/configs/rcu/v3.3/PT2-NH
View fileThis diff is incomplete. Download the complete patch.
tools/testing/selftests/rcutorture/configs/rcu/v3.3/ver_functions.sh
View fileThis diff is incomplete. Download the complete patch.
tools/testing/selftests/rcutorture/configs/rcu/v3.5/CFLIST
View fileThis diff is incomplete. Download the complete patch.
tools/testing/selftests/rcutorture/configs/rcu/v3.5/N1-S-T-NH-SD-SMP-HP
View fileThis diff is incomplete. Download the complete patch.
tools/testing/selftests/rcutorture/configs/rcu/v3.5/N2-2-t-nh-sd-SMP-hp
View fileThis diff is incomplete. Download the complete patch.
tools/testing/selftests/rcutorture/configs/rcu/v3.5/N3-3-T-nh-SD-SMP-hp
View fileThis diff is incomplete. Download the complete patch.
tools/testing/selftests/rcutorture/configs/rcu/v3.5/N4-A-t-NH-sd-SMP-HP
View fileThis diff is incomplete. Download the complete patch.
tools/testing/selftests/rcutorture/configs/rcu/v3.5/N5-U-T-NH-sd-SMP-hp
View fileThis diff is incomplete. Download the complete patch.
tools/testing/selftests/rcutorture/configs/rcu/v3.5/NT1-nh
View fileThis diff is incomplete. Download the complete patch.
tools/testing/selftests/rcutorture/configs/rcu/v3.5/NT3-NH
View fileThis diff is incomplete. Download the complete patch.
tools/testing/selftests/rcutorture/configs/rcu/v3.5/P1-S-T-NH-SD-SMP-HP
View fileThis diff is incomplete. Download the complete patch.
tools/testing/selftests/rcutorture/configs/rcu/v3.5/P2-2-t-nh-sd-SMP-hp
View fileThis diff is incomplete. Download the complete patch.
tools/testing/selftests/rcutorture/configs/rcu/v3.5/P3-3-T-nh-SD-SMP-hp
View fileThis diff is incomplete. Download the complete patch.
tools/testing/selftests/rcutorture/configs/rcu/v3.5/P4-A-t-NH-sd-SMP-HP
View fileThis diff is incomplete. Download the complete patch.
tools/testing/selftests/rcutorture/configs/rcu/v3.5/P5-U-T-NH-sd-SMP-hp
View fileThis diff is incomplete. Download the complete patch.
tools/testing/selftests/rcutorture/configs/rcu/v3.5/PT1-nh
View fileThis diff is incomplete. Download the complete patch.
tools/testing/selftests/rcutorture/configs/rcu/v3.5/PT2-NH
View fileThis diff is incomplete. Download the complete patch.
tools/testing/selftests/rcutorture/configs/rcu/v3.5/ver_functions.sh
View fileThis diff is incomplete. Download the complete patch.
tools/testing/selftests/rcutorture/configs/rcu/ver_functions.sh
View fileThis diff is incomplete. Download the complete patch.
tools/testing/selftests/rcutorture/doc/TREE_RCU-kconfig.txt
View fileThis diff is incomplete. Download the complete patch.