projects/linux·commit a4cbbf54

Merge branch 'perf-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip

a4cbbf54 · Linus Torvalds <[email protected]> · 2015-02-09 15:43:55 -08:00

Pull perf updates from Ingo Molnar:
 "Kernel side changes:

   - AMD range breakpoints support:

     Extend breakpoint tools and core to support address range through
     perf event with initial backend support for AMD extended
     breakpoints.

     The syntax is:

         perf record -e mem:addr/len:type

     For example set write breakpoint from 0x1000 to 0x1200 (0x1000 + 512)

         perf record -e mem:0x1000/512:w

   - event throttling/rotating fixes

   - various event group handling fixes, cleanups and general paranoia
     code to be more robust against bugs in the future.

    - kernel stack overhead fixes

  User-visible tooling side changes:

   - Show precise number of samples in at the end of a 'record' session,
     if processing build ids, since we will then traverse the whole
     perf.data file and see all the PERF_RECORD_SAMPLE records,
     otherwise stop showing the previous off-base heuristicly counted
     number of "samples" (Namhyung Kim).

   - Support to read compressed module from build-id cache (Namhyung
     Kim)

   - Enable sampling loads and stores simultaneously in 'perf mem'
     (Stephane Eranian)

   - 'perf diff' output improvements (Namhyung Kim)

   - Fix error reporting for evsel pgfault constructor (Arnaldo Carvalho
     de Melo)

  Tooling side infrastructure changes:

   - Cache eh/debug frame offset for dwarf unwind (Namhyung Kim)

   - Support parsing parameterized events (Cody P Schafer)

   - Add support for IP address formats in libtraceevent (David Ahern)

  Plus other misc fixes"

* 'perf-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip: (48 commits)
  perf: Decouple unthrottling and rotating
  perf: Drop module reference on event init failure
  perf: Use POLLIN instead of POLL_IN for perf poll data in flag
  perf: Fix put_event() ctx lock
  perf: Fix move_group() order
  perf: Fix event->ctx locking
  perf: Add a bit of paranoia
  perf symbols: Convert lseek + read to pread
  perf tools: Use perf_data_file__fd() consistently
  perf symbols: Support to read compressed module from build-id cache
  perf evsel: Set attr.task bit for a tracking event
  perf header: Set header version correctly
  perf record: Show precise number of samples
  perf tools: Do not use __perf_session__process_events() directly
  perf callchain: Cache eh/debug frame offset for dwarf unwind
  perf tools: Provide stub for missing pthread_attr_setaffinity_np
  perf evsel: Don't rely on malloc working for sz 0
  tools lib traceevent: Add support for IP address formats
  perf ui/tui: Show fatal error message only if exists
  perf tests: Fix typo in sample-parsing.c
  ...
Commit
a4cbbf549a9be10b7583c44249efccd64839533d
Parents
8308756f · 2fde4f94

Merge commit. Changes below are compared with the first parent: 8308756f.

Changes

76 files changed · +1611 −666

FileStatusAddDelContent
Documentation/ABI/testing/sysfs-bus-event_source-devices-eventsmodified+6−0View file
arch/x86/include/asm/cpufeature.hmodified+2−0View file
arch/x86/include/asm/debugreg.hmodified+5−0View file
arch/x86/include/asm/hw_breakpoint.hmodified+1−0View file
arch/x86/include/uapi/asm/msr-index.hmodified+4−0View file
arch/x86/kernel/cpu/amd.cmodified+19−0View file
arch/x86/kernel/hw_breakpoint.cmodified+17−28View file
include/linux/ftrace_event.hmodified+1−1View file
include/linux/perf_event.hmodified+22−8View file
include/trace/ftrace.hmodified+4−3View file
kernel/events/core.cmodified+344−120View file
kernel/events/ring_buffer.cmodified+2−1View file
kernel/sched/core.cmodified+1−1View file
kernel/trace/trace_event_perf.cmodified+3−1View file
kernel/trace/trace_kprobe.cmodified+2−2View file
kernel/trace/trace_syscalls.cmodified+2−2View file
kernel/trace/trace_uprobe.cmodified+1−1View file
tools/lib/api/fs/debugfs.cmodified+43−0View file
tools/lib/api/fs/debugfs.hmodified+3−0View file
tools/lib/traceevent/event-parse.cmodified+328−0View file
tools/perf/Documentation/perf-buildid-cache.txtmodified+1−1View file
tools/perf/Documentation/perf-list.txtmodified+13−0View file
tools/perf/Documentation/perf-mem.txtmodified+5−4View file
tools/perf/Documentation/perf-record.txtmodified+17−2View file
tools/perf/Documentation/perf-script.txtmodified+14−14View file
tools/perf/Documentation/perf-stat.txtmodified+16−4View file
tools/perf/bench/futex.hmodified+13−0View file
tools/perf/builtin-buildid-cache.cmodified+2−2View file
tools/perf/builtin-diff.cmodified+157−91View file
tools/perf/builtin-inject.cmodified+3−2View file
tools/perf/builtin-mem.cmodified+104−27View file
tools/perf/builtin-record.cmodified+48−22View file
tools/perf/builtin-report.cmodified+3−13View file
tools/perf/builtin-stat.cmodified+1−1View file
tools/perf/builtin-top.cmodified+1−1View file
tools/perf/builtin-trace.cmodified+57−49View file
tools/perf/config/Makefilemodified+6−0View file
tools/perf/config/feature-checks/Makefilemodified+4−0View file
tools/perf/config/feature-checks/test-all.cmodified+5−0View file
tools/perf/config/feature-checks/test-pthread-attr-setaffinity-np.cadded+14−0View file
tools/perf/tests/attr.pymodified+0−1View file
tools/perf/tests/hists_cumulate.cmodified+1−1View file
tools/perf/tests/hists_output.cmodified+1−1View file
tools/perf/tests/makemodified+0−1View file
tools/perf/tests/parse-events.cmodified+59−1View file
tools/perf/tests/sample-parsing.cmodified+1−1View file
tools/perf/ui/browsers/annotate.cmodified+1−2View file
tools/perf/ui/hist.cmodified+8−4View file
tools/perf/ui/progress.hmodified+2−2View file
tools/perf/ui/tui/helpline.cmodified+3−0View file
tools/perf/ui/tui/setup.cmodified+2−1View file
tools/perf/util/annotate.cmodified+1−1View file
tools/perf/util/color.cmodified+0−126View file
tools/perf/util/color.hmodified+0−2View file
tools/perf/util/dso.cmodified+1−5View file
tools/perf/util/dso.hmodified+1−0View file
tools/perf/util/evlist.cmodified+0−27View file
tools/perf/util/evlist.hmodified+0−1View file
tools/perf/util/evsel.cmodified+4−0View file
tools/perf/util/header.cmodified+1−1View file
tools/perf/util/hist.cmodified+23−25View file
tools/perf/util/hist.hmodified+7−4View file
tools/perf/util/parse-events.cmodified+14−13View file
tools/perf/util/parse-events.hmodified+2−1View file
tools/perf/util/parse-events.lmodified+1−0View file
tools/perf/util/parse-events.ymodified+24−2View file
tools/perf/util/parse-options.cmodified+1−1View file
tools/perf/util/pmu.cmodified+90−12View file
tools/perf/util/python.cmodified+1−1View file
tools/perf/util/scripting-engines/trace-event-python.cmodified+1−1View file
tools/perf/util/session.cmodified+4−4View file
tools/perf/util/session.hmodified+0−3View file
tools/perf/util/sort.cmodified+34−3View file
tools/perf/util/symbol-elf.cmodified+8−5View file
tools/perf/util/symbol.cmodified+1−1View file
tools/perf/util/unwind-libunwind.cmodified+20−11View file

Only the first 600 diff lines are shown. Download the complete patch.

Documentation/ABI/testing/sysfs-bus-event_source-devices-events

View file
diff --git a/Documentation/ABI/testing/sysfs-bus-event_source-devices-events b/Documentation/ABI/testing/sysfs-bus-event_source-devices-events
index 20979f8..505f080 100644
--- a/Documentation/ABI/testing/sysfs-bus-event_source-devices-events
+++ b/Documentation/ABI/testing/sysfs-bus-event_source-devices-events
@@ -52,12 +52,18 @@ Description: Per-pmu performance monitoring events specific to the running syste
event=0x2abc
event=0x423,inv,cmask=0x3
domain=0x1,offset=0x8,starting_index=0xffff
+ domain=0x1,offset=0x8,core=?
Each of the assignments indicates a value to be assigned to a
particular set of bits (as defined by the format file
corresponding to the <term>) in the perf_event structure passed
to the perf_open syscall.
+ In the case of the last example, a value replacing "?" would
+ need to be provided by the user selecting the particular event.
+ This is referred to as "event parameterization". Event
+ parameters have the format 'param=?'.
+
What: /sys/bus/event_source/devices/<pmu>/events/<event>.unit
Date: 2014/02/24
Contact: Linux kernel mailing list <[email protected]>

arch/x86/include/asm/cpufeature.h

View file
diff --git a/arch/x86/include/asm/cpufeature.h b/arch/x86/include/asm/cpufeature.h
index aede2c3..90a5485 100644
--- a/arch/x86/include/asm/cpufeature.h
+++ b/arch/x86/include/asm/cpufeature.h
@@ -174,6 +174,7 @@
#define X86_FEATURE_TOPOEXT ( 6*32+22) /* topology extensions CPUID leafs */
#define X86_FEATURE_PERFCTR_CORE ( 6*32+23) /* core performance counter extensions */
#define X86_FEATURE_PERFCTR_NB ( 6*32+24) /* NB performance counter extensions */
+#define X86_FEATURE_BPEXT (6*32+26) /* data breakpoint extension */
#define X86_FEATURE_PERFCTR_L2 ( 6*32+28) /* L2 performance counter extensions */
/*
@@ -388,6 +389,7 @@ extern const char * const x86_bug_flags[NBUGINTS*32];
#define cpu_has_cx16 boot_cpu_has(X86_FEATURE_CX16)
#define cpu_has_eager_fpu boot_cpu_has(X86_FEATURE_EAGER_FPU)
#define cpu_has_topoext boot_cpu_has(X86_FEATURE_TOPOEXT)
+#define cpu_has_bpext boot_cpu_has(X86_FEATURE_BPEXT)
#if __GNUC__ >= 4
extern void warn_pre_alternatives(void);

arch/x86/include/asm/debugreg.h

View file
diff --git a/arch/x86/include/asm/debugreg.h b/arch/x86/include/asm/debugreg.h
index 61fd18b..12cb66f 100644
--- a/arch/x86/include/asm/debugreg.h
+++ b/arch/x86/include/asm/debugreg.h
@@ -114,5 +114,10 @@ static inline void debug_stack_usage_inc(void) { }
static inline void debug_stack_usage_dec(void) { }
#endif /* X86_64 */
+#ifdef CONFIG_CPU_SUP_AMD
+extern void set_dr_addr_mask(unsigned long mask, int dr);
+#else
+static inline void set_dr_addr_mask(unsigned long mask, int dr) { }
+#endif
#endif /* _ASM_X86_DEBUGREG_H */

arch/x86/include/asm/hw_breakpoint.h

View file
diff --git a/arch/x86/include/asm/hw_breakpoint.h b/arch/x86/include/asm/hw_breakpoint.h
index ef1c4d2..6c98be8 100644
--- a/arch/x86/include/asm/hw_breakpoint.h
+++ b/arch/x86/include/asm/hw_breakpoint.h
@@ -12,6 +12,7 @@
*/
struct arch_hw_breakpoint {
unsigned long address;
+ unsigned long mask;
u8 len;
u8 type;
};

arch/x86/include/uapi/asm/msr-index.h

View file
diff --git a/arch/x86/include/uapi/asm/msr-index.h b/arch/x86/include/uapi/asm/msr-index.h
index c8aa65d..d979e5a 100644
--- a/arch/x86/include/uapi/asm/msr-index.h
+++ b/arch/x86/include/uapi/asm/msr-index.h
@@ -251,6 +251,10 @@
/* Fam 16h MSRs */
#define MSR_F16H_L2I_PERF_CTL 0xc0010230
#define MSR_F16H_L2I_PERF_CTR 0xc0010231
+#define MSR_F16H_DR1_ADDR_MASK 0xc0011019
+#define MSR_F16H_DR2_ADDR_MASK 0xc001101a
+#define MSR_F16H_DR3_ADDR_MASK 0xc001101b
+#define MSR_F16H_DR0_ADDR_MASK 0xc0011027
/* Fam 15h MSRs */
#define MSR_F15H_PERF_CTL 0xc0010200

arch/x86/kernel/cpu/amd.c

View file
diff --git a/arch/x86/kernel/cpu/amd.c b/arch/x86/kernel/cpu/amd.c
index 15c5df9..a220239 100644
--- a/arch/x86/kernel/cpu/amd.c
+++ b/arch/x86/kernel/cpu/amd.c
@@ -869,3 +869,22 @@ static bool cpu_has_amd_erratum(struct cpuinfo_x86 *cpu, const int *erratum)
return false;
}
+
+void set_dr_addr_mask(unsigned long mask, int dr)
+{
+ if (!cpu_has_bpext)
+ return;
+
+ switch (dr) {
+ case 0:
+ wrmsr(MSR_F16H_DR0_ADDR_MASK, mask, 0);
+ break;
+ case 1:
+ case 2:
+ case 3:
+ wrmsr(MSR_F16H_DR1_ADDR_MASK - 1 + dr, mask, 0);
+ break;
+ default:
+ break;
+ }
+}

arch/x86/kernel/hw_breakpoint.c

View file
diff --git a/arch/x86/kernel/hw_breakpoint.c b/arch/x86/kernel/hw_breakpoint.c
index 3d5fb50..7114ba2 100644
--- a/arch/x86/kernel/hw_breakpoint.c
+++ b/arch/x86/kernel/hw_breakpoint.c
@@ -126,6 +126,8 @@ int arch_install_hw_breakpoint(struct perf_event *bp)
*dr7 |= encode_dr7(i, info->len, info->type);
set_debugreg(*dr7, 7);
+ if (info->mask)
+ set_dr_addr_mask(info->mask, i);
return 0;
}
@@ -161,29 +163,8 @@ void arch_uninstall_hw_breakpoint(struct perf_event *bp)
*dr7 &= ~__encode_dr7(i, info->len, info->type);
set_debugreg(*dr7, 7);
-}
-
-static int get_hbp_len(u8 hbp_len)
-{
- unsigned int len_in_bytes = 0;
-
- switch (hbp_len) {
- case X86_BREAKPOINT_LEN_1:
- len_in_bytes = 1;
- break;
- case X86_BREAKPOINT_LEN_2:
- len_in_bytes = 2;
- break;
- case X86_BREAKPOINT_LEN_4:
- len_in_bytes = 4;
- break;
-#ifdef CONFIG_X86_64
- case X86_BREAKPOINT_LEN_8:
- len_in_bytes = 8;
- break;
-#endif
- }
- return len_in_bytes;
+ if (info->mask)
+ set_dr_addr_mask(0, i);
}
/*
@@ -196,7 +177,7 @@ int arch_check_bp_in_kernelspace(struct perf_event *bp)
struct arch_hw_breakpoint *info = counter_arch_bp(bp);
va = info->address;
- len = get_hbp_len(info->len);
+ len = bp->attr.bp_len;
return (va >= TASK_SIZE) && ((va + len - 1) >= TASK_SIZE);
}
@@ -277,6 +258,8 @@ static int arch_build_bp_info(struct perf_event *bp)
}
/* Len */
+ info->mask = 0;
+
switch (bp->attr.bp_len) {
case HW_BREAKPOINT_LEN_1:
info->len = X86_BREAKPOINT_LEN_1;
@@ -293,11 +276,17 @@ static int arch_build_bp_info(struct perf_event *bp)
break;
#endif
default:
- return -EINVAL;
+ if (!is_power_of_2(bp->attr.bp_len))
+ return -EINVAL;
+ if (!cpu_has_bpext)
+ return -EOPNOTSUPP;
+ info->mask = bp->attr.bp_len - 1;
+ info->len = X86_BREAKPOINT_LEN_1;
}
return 0;
}
+
/*
* Validate the arch-specific HW Breakpoint register settings
*/
@@ -312,11 +301,11 @@ int arch_validate_hwbkpt_settings(struct perf_event *bp)
if (ret)
return ret;
- ret = -EINVAL;
-
switch (info->len) {
case X86_BREAKPOINT_LEN_1:
align = 0;
+ if (info->mask)
+ align = info->mask;
break;
case X86_BREAKPOINT_LEN_2:
align = 1;
@@ -330,7 +319,7 @@ int arch_validate_hwbkpt_settings(struct perf_event *bp)
break;
#endif
default:
- return ret;
+ WARN_ON_ONCE(1);
}
/*

include/linux/ftrace_event.h

View file
diff --git a/include/linux/ftrace_event.h b/include/linux/ftrace_event.h
index 0bebb5c..d36f68b 100644
--- a/include/linux/ftrace_event.h
+++ b/include/linux/ftrace_event.h
@@ -595,7 +595,7 @@ extern int ftrace_profile_set_filter(struct perf_event *event, int event_id,
char *filter_str);
extern void ftrace_profile_free_filter(struct perf_event *event);
extern void *perf_trace_buf_prepare(int size, unsigned short type,
- struct pt_regs *regs, int *rctxp);
+ struct pt_regs **regs, int *rctxp);
static inline void
perf_trace_buf_submit(void *raw_data, int size, int rctx, u64 addr,

include/linux/perf_event.h

View file
diff --git a/include/linux/perf_event.h b/include/linux/perf_event.h
index 664de5a..5cad0e6 100644
--- a/include/linux/perf_event.h
+++ b/include/linux/perf_event.h
@@ -469,6 +469,7 @@ struct perf_event_context {
*/
struct mutex mutex;
+ struct list_head active_ctx_list;
struct list_head pinned_groups;
struct list_head flexible_groups;
struct list_head event_list;
@@ -519,7 +520,6 @@ struct perf_cpu_context {
int exclusive;
struct hrtimer hrtimer;
ktime_t hrtimer_interval;
- struct list_head rotation_list;
struct pmu *unique_pmu;
struct perf_cgroup *cgrp;
};
@@ -659,6 +659,7 @@ static inline int is_software_event(struct perf_event *event)
extern struct static_key perf_swevent_enabled[PERF_COUNT_SW_MAX];
+extern void ___perf_sw_event(u32, u64, struct pt_regs *, u64);
extern void __perf_sw_event(u32, u64, struct pt_regs *, u64);
#ifndef perf_arch_fetch_caller_regs
@@ -683,14 +684,25 @@ static inline void perf_fetch_caller_regs(struct pt_regs *regs)
static __always_inline void
perf_sw_event(u32 event_id, u64 nr, struct pt_regs *regs, u64 addr)
{
- struct pt_regs hot_regs;
+ if (static_key_false(&perf_swevent_enabled[event_id]))
+ __perf_sw_event(event_id, nr, regs, addr);
+}
+
+DECLARE_PER_CPU(struct pt_regs, __perf_regs[4]);
+/*
+ * 'Special' version for the scheduler, it hard assumes no recursion,
+ * which is guaranteed by us not actually scheduling inside other swevents
+ * because those disable preemption.
+ */
+static __always_inline void
+perf_sw_event_sched(u32 event_id, u64 nr, u64 addr)
+{
if (static_key_false(&perf_swevent_enabled[event_id])) {
- if (!regs) {
- perf_fetch_caller_regs(&hot_regs);
- regs = &hot_regs;
- }
- __perf_sw_event(event_id, nr, regs, addr);
+ struct pt_regs *regs = this_cpu_ptr(&__perf_regs[0]);
+
+ perf_fetch_caller_regs(regs);
+ ___perf_sw_event(event_id, nr, regs, addr);
}
}
@@ -706,7 +718,7 @@ static inline void perf_event_task_sched_in(struct task_struct *prev,
static inline void perf_event_task_sched_out(struct task_struct *prev,
struct task_struct *next)
{
- perf_sw_event(PERF_COUNT_SW_CONTEXT_SWITCHES, 1, NULL, 0);
+ perf_sw_event_sched(PERF_COUNT_SW_CONTEXT_SWITCHES, 1, 0);
if (static_key_false(&perf_sched_events.key))
__perf_event_task_sched_out(prev, next);
@@ -817,6 +829,8 @@ static inline int perf_event_refresh(struct perf_event *event, int refresh)
static inline void
perf_sw_event(u32 event_id, u64 nr, struct pt_regs *regs, u64 addr) { }
static inline void
+perf_sw_event_sched(u32 event_id, u64 nr, u64 addr) { }
+static inline void
perf_bp_event(struct perf_event *event, void *data) { }
static inline int perf_register_guest_info_callbacks

include/trace/ftrace.h

View file
diff --git a/include/trace/ftrace.h b/include/trace/ftrace.h
index 139b506..27609df 100644
--- a/include/trace/ftrace.h
+++ b/include/trace/ftrace.h
@@ -763,7 +763,7 @@ perf_trace_##call(void *__data, proto) \
struct ftrace_event_call *event_call = __data; \
struct ftrace_data_offsets_##call __maybe_unused __data_offsets;\
struct ftrace_raw_##call *entry; \
- struct pt_regs __regs; \
+ struct pt_regs *__regs; \
u64 __addr = 0, __count = 1; \
struct task_struct *__task = NULL; \
struct hlist_head *head; \
@@ -782,18 +782,19 @@ perf_trace_##call(void *__data, proto) \
sizeof(u64)); \
__entry_size -= sizeof(u32); \
\
- perf_fetch_caller_regs(&__regs); \
entry = perf_trace_buf_prepare(__entry_size, \
event_call->event.type, &__regs, &rctx); \
if (!entry) \
return; \
\
+ perf_fetch_caller_regs(__regs); \
+ \
tstruct \
\
{ assign; } \
\
perf_trace_buf_submit(entry, __entry_size, rctx, __addr, \
- __count, &__regs, head, __task); \
+ __count, __regs, head, __task); \
}
/*

kernel/events/core.c

View file
diff --git a/kernel/events/core.c b/kernel/events/core.c
index 19efcf1..7f2fbb8 100644
--- a/kernel/events/core.c
+++ b/kernel/events/core.c
@@ -872,22 +872,32 @@ void perf_pmu_enable(struct pmu *pmu)
pmu->pmu_enable(pmu);
}
-static DEFINE_PER_CPU(struct list_head, rotation_list);
+static DEFINE_PER_CPU(struct list_head, active_ctx_list);
/*
- * perf_pmu_rotate_start() and perf_rotate_context() are fully serialized
- * because they're strictly cpu affine and rotate_start is called with IRQs
- * disabled, while rotate_context is called from IRQ context.
+ * perf_event_ctx_activate(), perf_event_ctx_deactivate(), and
+ * perf_event_task_tick() are fully serialized because they're strictly cpu
+ * affine and perf_event_ctx{activate,deactivate} are called with IRQs
+ * disabled, while perf_event_task_tick is called from IRQ context.
*/
-static void perf_pmu_rotate_start(struct pmu *pmu)
+static void perf_event_ctx_activate(struct perf_event_context *ctx)
{
- struct perf_cpu_context *cpuctx = this_cpu_ptr(pmu->pmu_cpu_context);
- struct list_head *head = this_cpu_ptr(&rotation_list);
+ struct list_head *head = this_cpu_ptr(&active_ctx_list);
WARN_ON(!irqs_disabled());
- if (list_empty(&cpuctx->rotation_list))
- list_add(&cpuctx->rotation_list, head);
+ WARN_ON(!list_empty(&ctx->active_ctx_list));
+
+ list_add(&ctx->active_ctx_list, head);
+}
+
+static void perf_event_ctx_deactivate(struct perf_event_context *ctx)
+{
+ WARN_ON(!irqs_disabled());
+
+ WARN_ON(list_empty(&ctx->active_ctx_list));
+
+ list_del_init(&ctx->active_ctx_list);
}
static void get_ctx(struct perf_event_context *ctx)
@@ -907,6 +917,84 @@ static void put_ctx(struct perf_event_context *ctx)
}
/*
+ * Because of perf_event::ctx migration in sys_perf_event_open::move_group and
+ * perf_pmu_migrate_context() we need some magic.
+ *
+ * Those places that change perf_event::ctx will hold both
+ * perf_event_ctx::mutex of the 'old' and 'new' ctx value.
+ *
+ * Lock ordering is by mutex address. There is one other site where
+ * perf_event_context::mutex nests and that is put_event(). But remember that
+ * that is a parent<->child context relation, and migration does not affect
+ * children, therefore these two orderings should not interact.
+ *
+ * The change in perf_event::ctx does not affect children (as claimed above)
+ * because the sys_perf_event_open() case will install a new event and break
+ * the ctx parent<->child relation, and perf_pmu_migrate_context() is only
+ * concerned with cpuctx and that doesn't have children.
+ *
+ * The places that change perf_event::ctx will issue:
+ *
+ * perf_remove_from_context();
+ * synchronize_rcu();
+ * perf_install_in_context();
+ *
+ * to affect the change. The remove_from_context() + synchronize_rcu() should
+ * quiesce the event, after which we can install it in the new location. This
+ * means that only external vectors (perf_fops, prctl) can perturb the event
+ * while in transit. Therefore all such accessors should also acquire
+ * perf_event_context::mutex to serialize against this.
+ *
+ * However; because event->ctx can change while we're waiting to acquire
+ * ctx->mutex we must be careful and use the below perf_event_ctx_lock()
+ * function.
+ *
+ * Lock order:
+ * task_struct::perf_event_mutex
+ * perf_event_context::mutex
+ * perf_event_context::lock
+ * perf_event::child_mutex;
+ * perf_event::mmap_mutex
+ * mmap_sem
+ */
+static struct perf_event_context *
+perf_event_ctx_lock_nested(struct perf_event *event, int nesting)
+{
+ struct perf_event_context *ctx;
+
+again:
+ rcu_read_lock();
+ ctx = ACCESS_ONCE(event->ctx);
+ if (!atomic_inc_not_zero(&ctx->refcount)) {
+ rcu_read_unlock();
+ goto again;
+ }
+ rcu_read_unlock();
+
+ mutex_lock_nested(&ctx->mutex, nesting);
+ if (event->ctx != ctx) {
+ mutex_unlock(&ctx->mutex);
+ put_ctx(ctx);
+ goto again;
+ }
+
+ return ctx;
+}
+
+static inline struct perf_event_context *
+perf_event_ctx_lock(struct perf_event *event)
+{
+ return perf_event_ctx_lock_nested(event, 0);
+}
+
+static void perf_event_ctx_unlock(struct perf_event *event,
+ struct perf_event_context *ctx)
+{
+ mutex_unlock(&ctx->mutex);
+ put_ctx(ctx);
+}
+
+/*
* This must be done under the ctx->lock, such as to serialize against
* context_equiv(), therefore we cannot call put_ctx() since that might end up
* calling scheduler related locks and ctx->lock nests inside those.
@@ -1155,8 +1243,6 @@ list_add_event(struct perf_event *event, struct perf_event_context *ctx)
ctx->nr_branch_stack++;
list_add_rcu(&event->event_entry, &ctx->event_list);
- if (!ctx->nr_events)
- perf_pmu_rotate_start(ctx->pmu);
ctx->nr_events++;
if (event->attr.inherit_stat)
ctx->nr_stat++;
@@ -1275,6 +1361,8 @@ static void perf_group_attach(struct perf_event *event)
if (group_leader == event)
return;
+ WARN_ON_ONCE(group_leader->ctx != event->ctx);
+
if (group_leader->group_flags & PERF_GROUP_SOFTWARE &&
!is_software_event(event))
group_leader->group_flags &= ~PERF_GROUP_SOFTWARE;
@@ -1296,6 +1384,10 @@ static void
list_del_event(struct perf_event *event, struct perf_event_context *ctx)
{
struct perf_cpu_context *cpuctx;
+
+ WARN_ON_ONCE(event->ctx != ctx);
+ lockdep_assert_held(&ctx->lock);
+
/*
* We can have double detach due to exit/hot-unplug + close.
*/
@@ -1380,6 +1472,8 @@ static void perf_group_detach(struct perf_event *event)
/* Inherit group flags from the previous leader */
sibling->group_flags = event->group_flags;
+
+ WARN_ON_ONCE(sibling->ctx != event->ctx);
}
out:
@@ -1442,6 +1536,10 @@ event_sched_out(struct perf_event *event,
{
u64 tstamp = perf_event_time(event);
u64 delta;
+
+ WARN_ON_ONCE(event->ctx != ctx);
+ lockdep_assert_held(&ctx->lock);
+
/*
* An event which could not be activated because of
* filter mismatch still needs to have its timings
@@ -1471,7 +1569,8 @@ event_sched_out(struct perf_event *event,
if (!is_software_event(event))
cpuctx->active_oncpu--;
- ctx->nr_active--;
+ if (!--ctx->nr_active)
+ perf_event_ctx_deactivate(ctx);
if (event->attr.freq && event->attr.sample_freq)
ctx->nr_freq--;
if (event->attr.exclusive || !cpuctx->active_oncpu)
@@ -1654,7 +1753,7 @@ int __perf_event_disable(void *info)
* is the current context on this CPU and preemption is disabled,
* hence we can't get into perf_event_task_sched_out for this context.
*/
-void perf_event_disable(struct perf_event *event)
+static void _perf_event_disable(struct perf_event *event)
{
struct perf_event_context *ctx = event->ctx;
struct task_struct *task = ctx->task;
@@ -1695,6 +1794,19 @@ retry:
}
raw_spin_unlock_irq(&ctx->lock);
}
+
+/*
+ * Strictly speaking kernel users cannot create groups and therefore this
+ * interface does not need the perf_event_ctx_lock() magic.
+ */
+void perf_event_disable(struct perf_event *event)
+{
+ struct perf_event_context *ctx;
+
+ ctx = perf_event_ctx_lock(event);
+ _perf_event_disable(event);
+ perf_event_ctx_unlock(event, ctx);
+}
EXPORT_SYMBOL_GPL(perf_event_disable);
static void perf_set_shadow_time(struct perf_event *event,
@@ -1782,7 +1894,8 @@ event_sched_in(struct perf_event *event,
if (!is_software_event(event))
cpuctx->active_oncpu++;
- ctx->nr_active++;
+ if (!ctx->nr_active++)
+ perf_event_ctx_activate(ctx);
if (event->attr.freq && event->attr.sample_freq)
ctx->nr_freq++;
@@ -2158,7 +2271,7 @@ unlock:
* perf_event_for_each_child or perf_event_for_each as described
* for perf_event_disable.
*/
-void perf_event_enable(struct perf_event *event)
+static void _perf_event_enable(struct perf_event *event)
{
struct perf_event_context *ctx = event->ctx;
struct task_struct *task = ctx->task;
@@ -2214,9 +2327,21 @@ retry:
out:
raw_spin_unlock_irq(&ctx->lock);
}
+
+/*
+ * See perf_event_disable();
+ */
+void perf_event_enable(struct perf_event *event)
+{
+ struct perf_event_context *ctx;
+
+ ctx = perf_event_ctx_lock(event);
+ _perf_event_enable(event);
+ perf_event_ctx_unlock(event, ctx);
+}
EXPORT_SYMBOL_GPL(perf_event_enable);
-int perf_event_refresh(struct perf_event *event, int refresh)

This diff is incomplete. Download the complete patch.

kernel/events/ring_buffer.c

View file

This diff is incomplete. Download the complete patch.

kernel/sched/core.c

View file

This diff is incomplete. Download the complete patch.

kernel/trace/trace_event_perf.c

View file

This diff is incomplete. Download the complete patch.

kernel/trace/trace_kprobe.c

View file

This diff is incomplete. Download the complete patch.

kernel/trace/trace_syscalls.c

View file

This diff is incomplete. Download the complete patch.

kernel/trace/trace_uprobe.c

View file

This diff is incomplete. Download the complete patch.

tools/lib/api/fs/debugfs.c

View file

This diff is incomplete. Download the complete patch.

tools/lib/api/fs/debugfs.h

View file

This diff is incomplete. Download the complete patch.

tools/lib/traceevent/event-parse.c

View file

This diff is incomplete. Download the complete patch.

tools/perf/Documentation/perf-buildid-cache.txt

View file

This diff is incomplete. Download the complete patch.

tools/perf/Documentation/perf-list.txt

View file

This diff is incomplete. Download the complete patch.

tools/perf/Documentation/perf-mem.txt

View file

This diff is incomplete. Download the complete patch.

tools/perf/Documentation/perf-record.txt

View file

This diff is incomplete. Download the complete patch.

tools/perf/Documentation/perf-script.txt

View file

This diff is incomplete. Download the complete patch.

tools/perf/Documentation/perf-stat.txt

View file

This diff is incomplete. Download the complete patch.

tools/perf/bench/futex.h

View file

This diff is incomplete. Download the complete patch.

tools/perf/builtin-buildid-cache.c

View file

This diff is incomplete. Download the complete patch.

tools/perf/builtin-diff.c

View file

This diff is incomplete. Download the complete patch.

tools/perf/builtin-inject.c

View file

This diff is incomplete. Download the complete patch.

tools/perf/builtin-mem.c

View file

This diff is incomplete. Download the complete patch.

tools/perf/builtin-record.c

View file

This diff is incomplete. Download the complete patch.

tools/perf/builtin-report.c

View file

This diff is incomplete. Download the complete patch.

tools/perf/builtin-stat.c

View file

This diff is incomplete. Download the complete patch.

tools/perf/builtin-top.c

View file

This diff is incomplete. Download the complete patch.

tools/perf/builtin-trace.c

View file

This diff is incomplete. Download the complete patch.

tools/perf/config/Makefile

View file

This diff is incomplete. Download the complete patch.

tools/perf/config/feature-checks/Makefile

View file

This diff is incomplete. Download the complete patch.

tools/perf/config/feature-checks/test-all.c

View file

This diff is incomplete. Download the complete patch.

tools/perf/config/feature-checks/test-pthread-attr-setaffinity-np.c

View file

This diff is incomplete. Download the complete patch.

tools/perf/tests/attr.py

View file

This diff is incomplete. Download the complete patch.

tools/perf/tests/hists_cumulate.c

View file

This diff is incomplete. Download the complete patch.

tools/perf/tests/hists_output.c

View file

This diff is incomplete. Download the complete patch.

tools/perf/tests/make

View file

This diff is incomplete. Download the complete patch.

tools/perf/tests/parse-events.c

View file

This diff is incomplete. Download the complete patch.

tools/perf/tests/sample-parsing.c

View file

This diff is incomplete. Download the complete patch.

tools/perf/ui/browsers/annotate.c

View file

This diff is incomplete. Download the complete patch.

tools/perf/ui/hist.c

View file

This diff is incomplete. Download the complete patch.

tools/perf/ui/progress.h

View file

This diff is incomplete. Download the complete patch.

tools/perf/ui/tui/helpline.c

View file

This diff is incomplete. Download the complete patch.

tools/perf/ui/tui/setup.c

View file

This diff is incomplete. Download the complete patch.

tools/perf/util/annotate.c

View file

This diff is incomplete. Download the complete patch.

tools/perf/util/color.c

View file

This diff is incomplete. Download the complete patch.

tools/perf/util/color.h

View file

This diff is incomplete. Download the complete patch.

tools/perf/util/dso.c

View file

This diff is incomplete. Download the complete patch.

tools/perf/util/dso.h

View file

This diff is incomplete. Download the complete patch.

tools/perf/util/evlist.c

View file

This diff is incomplete. Download the complete patch.

tools/perf/util/evlist.h

View file

This diff is incomplete. Download the complete patch.

tools/perf/util/evsel.c

View file

This diff is incomplete. Download the complete patch.

tools/perf/util/header.c

View file

This diff is incomplete. Download the complete patch.

tools/perf/util/hist.c

View file

This diff is incomplete. Download the complete patch.

tools/perf/util/hist.h

View file

This diff is incomplete. Download the complete patch.

tools/perf/util/parse-events.c

View file

This diff is incomplete. Download the complete patch.

tools/perf/util/parse-events.h

View file

This diff is incomplete. Download the complete patch.

tools/perf/util/parse-events.l

View file

This diff is incomplete. Download the complete patch.

tools/perf/util/parse-events.y

View file

This diff is incomplete. Download the complete patch.

tools/perf/util/parse-options.c

View file

This diff is incomplete. Download the complete patch.

tools/perf/util/pmu.c

View file

This diff is incomplete. Download the complete patch.

tools/perf/util/python.c

View file

This diff is incomplete. Download the complete patch.

tools/perf/util/scripting-engines/trace-event-python.c

View file

This diff is incomplete. Download the complete patch.

tools/perf/util/session.c

View file

This diff is incomplete. Download the complete patch.

tools/perf/util/session.h

View file

This diff is incomplete. Download the complete patch.

tools/perf/util/sort.c

View file

This diff is incomplete. Download the complete patch.

tools/perf/util/symbol-elf.c

View file

This diff is incomplete. Download the complete patch.

tools/perf/util/symbol.c

View file

This diff is incomplete. Download the complete patch.

tools/perf/util/unwind-libunwind.c

View file

This diff is incomplete. Download the complete patch.