projects/linux·commit 4d846676

Merge tag 'perf-core-2026-02-09' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip

4d846676 · Linus Torvalds <[email protected]> · 2026-02-10 12:00:46 -08:00

Pull performance event updates from Ingo Molnar:
 "x86 PMU driver updates:

   - Add support for the core PMU for Intel Diamond Rapids (DMR) CPUs
     (Dapeng Mi)

     Compared to previous iterations of the Intel PMU code, there's been
     a lot of changes, which center around three main areas:

      - Introduce the OFF-MODULE RESPONSE (OMR) facility to replace the
        Off-Core Response (OCR) facility

      - New PEBS data source encoding layout

      - Support the new "RDPMC user disable" feature

   - Likewise, a large series adds uncore PMU support for Intel Diamond
     Rapids (DMR) CPUs (Zide Chen)

     This centers around these four main areas:

      - DMR may have two Integrated I/O and Memory Hub (IMH) dies,
        separate from the compute tile (CBB) dies. Each CBB and each IMH
        die has its own discovery domain.

      - Unlike prior CPUs that retrieve the global discovery table
        portal exclusively via PCI or MSR, DMR uses PCI for IMH PMON
        discovery and MSR for CBB PMON discovery.

      - DMR introduces several new PMON types: SCA, HAMVF, D2D_ULA, UBR,
        PCIE4, CRS, CPC, ITC, OTC, CMS, and PCIE6.

      - IIO free-running counters in DMR are MMIO-based, unlike SPR.

   - Also add support for Add missing PMON units for Intel Panther Lake,
     and support Nova Lake (NVL), which largely maps to Panther Lake.
     (Zide Chen)

   - KVM integration: Add support for mediated vPMUs (by Kan Liang and
     Sean Christopherson, with fixes and cleanups by Peter Zijlstra,
     Sandipan Das and Mingwei Zhang)

   - Add Intel cstate driver to support for Wildcat Lake (WCL) CPUs,
     which are a low-power variant of Panther Lake (Zide Chen)

   - Add core, cstate and MSR PMU support for the Airmont NP Intel CPU
     (aka MaxLinear Lightning Mountain), which maps to the existing
     Airmont code (Martin Schiller)

  Performance enhancements:

   - Speed up kexec shutdown by avoiding unnecessary cross CPU calls
     (Jan H. Schönherr)

   - Fix slow perf_event_task_exit() with LBR callstacks (Namhyung Kim)

  User-space stack unwinding support:

   - Various cleanups and refactorings in preparation to generalize the
     unwinding code for other architectures (Jens Remus)

  Uprobes updates:

   - Transition from kmap_atomic to kmap_local_page (Keke Ming)

   - Fix incorrect lockdep condition in filter_chain() (Breno Leitao)

   - Fix XOL allocation failure for 32-bit tasks (Oleg Nesterov)

  Misc fixes and cleanups:

   - s390: Remove kvm_types.h from Kbuild (Randy Dunlap)

   - x86/intel/uncore: Convert comma to semicolon (Chen Ni)

   - x86/uncore: Clean up const mismatch (Greg Kroah-Hartman)

   - x86/ibs: Fix typo in dc_l2tlb_miss comment (Xiang-Bin Shi)"

* tag 'perf-core-2026-02-09' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip: (58 commits)
  s390: remove kvm_types.h from Kbuild
  uprobes: Fix incorrect lockdep condition in filter_chain()
  x86/ibs: Fix typo in dc_l2tlb_miss comment
  x86/uprobes: Fix XOL allocation failure for 32-bit tasks
  perf/x86/intel/uncore: Convert comma to semicolon
  perf/x86/intel: Add support for rdpmc user disable feature
  perf/x86: Use macros to replace magic numbers in attr_rdpmc
  perf/x86/intel: Add core PMU support for Novalake
  perf/x86/intel: Add support for PEBS memory auxiliary info field in NVL
  perf/x86/intel: Add core PMU support for DMR
  perf/x86/intel: Add support for PEBS memory auxiliary info field in DMR
  perf/x86/intel: Support the 4 new OMR MSRs introduced in DMR and NVL
  perf/core: Fix slow perf_event_task_exit() with LBR callstacks
  perf/core: Speed up kexec shutdown by avoiding unnecessary cross CPU calls
  uprobes: use kmap_local_page() for temporary page mappings
  arm/uprobes: use kmap_local_page() in arch_uprobe_copy_ixol()
  mips/uprobes: use kmap_local_page() in arch_uprobe_copy_ixol()
  arm64/uprobes: use kmap_local_page() in arch_uprobe_copy_ixol()
  riscv/uprobes: use kmap_local_page() in arch_uprobe_copy_ixol()
  perf/x86/intel/uncore: Add Nova Lake support
  ...
Commit
4d84667627c4ff70826b349c449bbaf63b9af4e5
Parents
a9aabb3b · 7db06e32

Merge commit. Changes below are compared with the first parent: a9aabb3b.

Changes

45 files changed · +2154 −522

FileStatusAddDelContent
Documentation/ABI/testing/sysfs-bus-event_source-devices-rdpmcadded+44−0View file
arch/arm/probes/uprobes/core.cmodified+2−2View file
arch/arm64/kernel/probes/uprobes.cmodified+2−2View file
arch/mips/kernel/uprobes.cmodified+2−2View file
arch/riscv/kernel/probes/uprobes.cmodified+2−2View file
arch/s390/include/asm/Kbuildmodified+0−1View file
arch/x86/entry/entry_fred.cmodified+1−0View file
arch/x86/events/amd/core.cmodified+2−0View file
arch/x86/events/core.cmodified+61−5View file
arch/x86/events/intel/core.cmodified+352−18View file
arch/x86/events/intel/cstate.cmodified+26−7View file
arch/x86/events/intel/ds.cmodified+261−0View file
arch/x86/events/intel/p6.cmodified+1−1View file
arch/x86/events/intel/uncore.cmodified+90−49View file
arch/x86/events/intel/uncore.hmodified+26−0View file
arch/x86/events/intel/uncore_discovery.cmodified+40−44View file
arch/x86/events/intel/uncore_discovery.hmodified+7−1View file
arch/x86/events/intel/uncore_snb.cmodified+85−0View file
arch/x86/events/intel/uncore_snbep.cmodified+480−210View file
arch/x86/events/msr.cmodified+1−0View file
arch/x86/events/perf_event.hmodified+26−0View file
arch/x86/include/asm/amd/ibs.hmodified+1−1View file
arch/x86/include/asm/hardirq.hmodified+3−0View file
arch/x86/include/asm/idtentry.hmodified+6−0View file
arch/x86/include/asm/irq_vectors.hmodified+3−1View file
arch/x86/include/asm/msr-index.hmodified+5−0View file
arch/x86/include/asm/perf_event.hmodified+12−2View file
arch/x86/include/asm/unwind_user.hmodified+14−13View file
arch/x86/kernel/idt.cmodified+3−0View file
arch/x86/kernel/irq.cmodified+19−0View file
arch/x86/kernel/uprobes.cmodified+24−0View file
arch/x86/kvm/Kconfigmodified+1−0View file
include/asm-generic/Kbuildmodified+1−0View file
include/linux/perf_event.hmodified+29−6View file
include/linux/unwind_user.hmodified+16−2View file
include/linux/uprobes.hmodified+1−0View file
include/uapi/linux/perf_event.hmodified+24−3View file
init/Kconfigmodified+4−0View file
kernel/events/core.cmodified+422−122View file
kernel/events/uprobes.cmodified+14−10View file
kernel/unwind/user.cmodified+4−8View file
tools/include/uapi/linux/perf_event.hmodified+24−3View file
tools/perf/trace/beauty/arch/x86/include/asm/irq_vectors.hmodified+2−1View file
tools/perf/util/pmu.cmodified+8−6View file
virt/kvm/kvm_main.cmodified+3−0View file

Only the first 600 diff lines are shown. Download the complete patch.

Documentation/ABI/testing/sysfs-bus-event_source-devices-rdpmc

View file
diff --git a/Documentation/ABI/testing/sysfs-bus-event_source-devices-rdpmc b/Documentation/ABI/testing/sysfs-bus-event_source-devices-rdpmc
new file mode 100644
index 0000000..59ec18b
--- /dev/null
+++ b/Documentation/ABI/testing/sysfs-bus-event_source-devices-rdpmc
@@ -0,0 +1,44 @@
+What: /sys/bus/event_source/devices/cpu.../rdpmc
+Date: November 2011
+KernelVersion: 3.10
+Contact: Linux kernel mailing list [email protected]
+Description: The /sys/bus/event_source/devices/cpu.../rdpmc attribute
+ is used to show/manage if rdpmc instruction can be
+ executed in user space. This attribute supports 3 numbers.
+ - rdpmc = 0
+ user space rdpmc is globally disabled for all PMU
+ counters.
+ - rdpmc = 1
+ user space rdpmc is globally enabled only in event mmap
+ ioctl called time window. If the mmap region is unmapped,
+ user space rdpmc is disabled again.
+ - rdpmc = 2
+ user space rdpmc is globally enabled for all PMU
+ counters.
+
+ In the Intel platforms supporting counter level's user
+ space rdpmc disable feature (CPUID.23H.EBX[2] = 1), the
+ meaning of 3 numbers is extended to
+ - rdpmc = 0
+ global user space rdpmc and counter level's user space
+ rdpmc of all counters are both disabled.
+ - rdpmc = 1
+ No changes on behavior of global user space rdpmc.
+ counter level's rdpmc of system-wide events is disabled
+ but counter level's rdpmc of non-system-wide events is
+ enabled.
+ - rdpmc = 2
+ global user space rdpmc and counter level's user space
+ rdpmc of all counters are both enabled unconditionally.
+
+ The default value of rdpmc is 1.
+
+ Please notice:
+ - global user space rdpmc's behavior would change
+ immediately along with the rdpmc value's change,
+ but the behavior of counter level's user space rdpmc
+ won't take effect immediately until the event is
+ reactivated or recreated.
+ - The rdpmc attribute is global, even for x86 hybrid
+ platforms. For example, changing cpu_core/rdpmc will
+ also change cpu_atom/rdpmc.

arch/arm/probes/uprobes/core.c

View file
diff --git a/arch/arm/probes/uprobes/core.c b/arch/arm/probes/uprobes/core.c
index 3d96fb4..0e1c6b9 100644
--- a/arch/arm/probes/uprobes/core.c
+++ b/arch/arm/probes/uprobes/core.c
@@ -113,7 +113,7 @@ int arch_uprobe_analyze_insn(struct arch_uprobe *auprobe, struct mm_struct *mm,
void arch_uprobe_copy_ixol(struct page *page, unsigned long vaddr,
void *src, unsigned long len)
{
- void *xol_page_kaddr = kmap_atomic(page);
+ void *xol_page_kaddr = kmap_local_page(page);
void *dst = xol_page_kaddr + (vaddr & ~PAGE_MASK);
preempt_disable();
@@ -126,7 +126,7 @@ void arch_uprobe_copy_ixol(struct page *page, unsigned long vaddr,
preempt_enable();
- kunmap_atomic(xol_page_kaddr);
+ kunmap_local(xol_page_kaddr);
}

arch/arm64/kernel/probes/uprobes.c

View file
diff --git a/arch/arm64/kernel/probes/uprobes.c b/arch/arm64/kernel/probes/uprobes.c
index c451ca1..2a9b0bc 100644
--- a/arch/arm64/kernel/probes/uprobes.c
+++ b/arch/arm64/kernel/probes/uprobes.c
@@ -15,7 +15,7 @@
void arch_uprobe_copy_ixol(struct page *page, unsigned long vaddr,
void *src, unsigned long len)
{
- void *xol_page_kaddr = kmap_atomic(page);
+ void *xol_page_kaddr = kmap_local_page(page);
void *dst = xol_page_kaddr + (vaddr & ~PAGE_MASK);
/*
@@ -32,7 +32,7 @@ void arch_uprobe_copy_ixol(struct page *page, unsigned long vaddr,
sync_icache_aliases((unsigned long)dst, (unsigned long)dst + len);
done:
- kunmap_atomic(xol_page_kaddr);
+ kunmap_local(xol_page_kaddr);
}
unsigned long uprobe_get_swbp_addr(struct pt_regs *regs)

arch/mips/kernel/uprobes.c

View file
diff --git a/arch/mips/kernel/uprobes.c b/arch/mips/kernel/uprobes.c
index 401b148..05cfc32 100644
--- a/arch/mips/kernel/uprobes.c
+++ b/arch/mips/kernel/uprobes.c
@@ -214,11 +214,11 @@ void arch_uprobe_copy_ixol(struct page *page, unsigned long vaddr,
unsigned long kaddr, kstart;
/* Initialize the slot */
- kaddr = (unsigned long)kmap_atomic(page);
+ kaddr = (unsigned long)kmap_local_page(page);
kstart = kaddr + (vaddr & ~PAGE_MASK);
memcpy((void *)kstart, src, len);
flush_icache_range(kstart, kstart + len);
- kunmap_atomic((void *)kaddr);
+ kunmap_local((void *)kaddr);
}
/**

arch/riscv/kernel/probes/uprobes.c

View file
diff --git a/arch/riscv/kernel/probes/uprobes.c b/arch/riscv/kernel/probes/uprobes.c
index cc15f7c..f0d0691 100644
--- a/arch/riscv/kernel/probes/uprobes.c
+++ b/arch/riscv/kernel/probes/uprobes.c
@@ -165,7 +165,7 @@ void arch_uprobe_copy_ixol(struct page *page, unsigned long vaddr,
void *src, unsigned long len)
{
/* Initialize the slot */
- void *kaddr = kmap_atomic(page);
+ void *kaddr = kmap_local_page(page);
void *dst = kaddr + (vaddr & ~PAGE_MASK);
unsigned long start = (unsigned long)dst;
@@ -178,5 +178,5 @@ void arch_uprobe_copy_ixol(struct page *page, unsigned long vaddr,
}
flush_icache_range(start, start + len);
- kunmap_atomic(kaddr);
+ kunmap_local(kaddr);
}

arch/s390/include/asm/Kbuild

View file
diff --git a/arch/s390/include/asm/Kbuild b/arch/s390/include/asm/Kbuild
index 297bf71..80bad7d 100644
--- a/arch/s390/include/asm/Kbuild
+++ b/arch/s390/include/asm/Kbuild
@@ -5,6 +5,5 @@ generated-y += syscall_table.h
generated-y += unistd_nr.h
generic-y += asm-offsets.h
-generic-y += kvm_types.h
generic-y += mcs_spinlock.h
generic-y += mmzone.h

arch/x86/entry/entry_fred.c

View file
diff --git a/arch/x86/entry/entry_fred.c b/arch/x86/entry/entry_fred.c
index 94e626c..a9b7299 100644
--- a/arch/x86/entry/entry_fred.c
+++ b/arch/x86/entry/entry_fred.c
@@ -114,6 +114,7 @@ static idtentry_t sysvec_table[NR_SYSTEM_VECTORS] __ro_after_init = {
SYSVEC(IRQ_WORK_VECTOR, irq_work),
+ SYSVEC(PERF_GUEST_MEDIATED_PMI_VECTOR, perf_guest_mediated_pmi_handler),
SYSVEC(POSTED_INTR_VECTOR, kvm_posted_intr_ipi),
SYSVEC(POSTED_INTR_WAKEUP_VECTOR, kvm_posted_intr_wakeup_ipi),
SYSVEC(POSTED_INTR_NESTED_VECTOR, kvm_posted_intr_nested_ipi),

arch/x86/events/amd/core.c

View file
diff --git a/arch/x86/events/amd/core.c b/arch/x86/events/amd/core.c
index 44656d2..0c92ed5 100644
--- a/arch/x86/events/amd/core.c
+++ b/arch/x86/events/amd/core.c
@@ -1439,6 +1439,8 @@ static int __init amd_core_pmu_init(void)
amd_pmu_global_cntr_mask = x86_pmu.cntr_mask64;
+ x86_get_pmu(smp_processor_id())->capabilities |= PERF_PMU_CAP_MEDIATED_VPMU;
+
/* Update PMC handling functions */
x86_pmu.enable_all = amd_pmu_v2_enable_all;
x86_pmu.disable_all = amd_pmu_v2_disable_all;

arch/x86/events/core.c

View file
diff --git a/arch/x86/events/core.c b/arch/x86/events/core.c
index 576baa9..d4468ef 100644
--- a/arch/x86/events/core.c
+++ b/arch/x86/events/core.c
@@ -30,6 +30,7 @@
#include <linux/device.h>
#include <linux/nospec.h>
#include <linux/static_call.h>
+#include <linux/kvm_types.h>
#include <asm/apic.h>
#include <asm/stacktrace.h>
@@ -56,6 +57,8 @@ DEFINE_PER_CPU(struct cpu_hw_events, cpu_hw_events) = {
.pmu = &pmu,
};
+static DEFINE_PER_CPU(bool, guest_lvtpc_loaded);
+
DEFINE_STATIC_KEY_FALSE(rdpmc_never_available_key);
DEFINE_STATIC_KEY_FALSE(rdpmc_always_available_key);
DEFINE_STATIC_KEY_FALSE(perf_is_hybrid);
@@ -1760,6 +1763,25 @@ void perf_events_lapic_init(void)
apic_write(APIC_LVTPC, APIC_DM_NMI);
}
+#ifdef CONFIG_PERF_GUEST_MEDIATED_PMU
+void perf_load_guest_lvtpc(u32 guest_lvtpc)
+{
+ u32 masked = guest_lvtpc & APIC_LVT_MASKED;
+
+ apic_write(APIC_LVTPC,
+ APIC_DM_FIXED | PERF_GUEST_MEDIATED_PMI_VECTOR | masked);
+ this_cpu_write(guest_lvtpc_loaded, true);
+}
+EXPORT_SYMBOL_FOR_KVM(perf_load_guest_lvtpc);
+
+void perf_put_guest_lvtpc(void)
+{
+ this_cpu_write(guest_lvtpc_loaded, false);
+ apic_write(APIC_LVTPC, APIC_DM_NMI);
+}
+EXPORT_SYMBOL_FOR_KVM(perf_put_guest_lvtpc);
+#endif /* CONFIG_PERF_GUEST_MEDIATED_PMU */
+
static int
perf_event_nmi_handler(unsigned int cmd, struct pt_regs *regs)
{
@@ -1768,6 +1790,17 @@ perf_event_nmi_handler(unsigned int cmd, struct pt_regs *regs)
int ret;
/*
+ * Ignore all NMIs when the CPU's LVTPC is configured to route PMIs to
+ * PERF_GUEST_MEDIATED_PMI_VECTOR, i.e. when an NMI time can't be due
+ * to a PMI. Attempting to handle a PMI while the guest's context is
+ * loaded will generate false positives and clobber guest state. Note,
+ * the LVTPC is switched to/from the dedicated mediated PMI IRQ vector
+ * while host events are quiesced.
+ */
+ if (this_cpu_read(guest_lvtpc_loaded))
+ return NMI_DONE;
+
+ /*
* All PMUs/events that share this PMI handler should make sure to
* increment active_events for their events.
*/
@@ -2130,7 +2163,8 @@ static int __init init_hw_perf_events(void)
pr_cont("%s PMU driver.\n", x86_pmu.name);
- x86_pmu.attr_rdpmc = 1; /* enable userspace RDPMC usage by default */
+ /* enable userspace RDPMC usage by default */
+ x86_pmu.attr_rdpmc = X86_USER_RDPMC_CONDITIONAL_ENABLE;
for (quirk = x86_pmu.quirks; quirk; quirk = quirk->next)
quirk->func();
@@ -2582,6 +2616,27 @@ static ssize_t get_attr_rdpmc(struct device *cdev,
return snprintf(buf, 40, "%d\n", x86_pmu.attr_rdpmc);
}
+/*
+ * Behaviors of rdpmc value:
+ * - rdpmc = 0
+ * global user space rdpmc and counter level's user space rdpmc of all
+ * counters are both disabled.
+ * - rdpmc = 1
+ * global user space rdpmc is enabled in mmap enabled time window and
+ * counter level's user space rdpmc is enabled for only non system-wide
+ * events. Counter level's user space rdpmc of system-wide events is
+ * still disabled by default. This won't introduce counter data leak for
+ * non system-wide events since their count data would be cleared when
+ * context switches.
+ * - rdpmc = 2
+ * global user space rdpmc and counter level's user space rdpmc of all
+ * counters are enabled unconditionally.
+ *
+ * Suppose the rdpmc value won't be changed frequently, don't dynamically
+ * reschedule events to make the new rpdmc value take effect on active perf
+ * events immediately, the new rdpmc value would only impact the new
+ * activated perf events. This makes code simpler and cleaner.
+ */
static ssize_t set_attr_rdpmc(struct device *cdev,
struct device_attribute *attr,
const char *buf, size_t count)
@@ -2610,12 +2665,12 @@ static ssize_t set_attr_rdpmc(struct device *cdev,
*/
if (val == 0)
static_branch_inc(&rdpmc_never_available_key);
- else if (x86_pmu.attr_rdpmc == 0)
+ else if (x86_pmu.attr_rdpmc == X86_USER_RDPMC_NEVER_ENABLE)
static_branch_dec(&rdpmc_never_available_key);
if (val == 2)
static_branch_inc(&rdpmc_always_available_key);
- else if (x86_pmu.attr_rdpmc == 2)
+ else if (x86_pmu.attr_rdpmc == X86_USER_RDPMC_ALWAYS_ENABLE)
static_branch_dec(&rdpmc_always_available_key);
on_each_cpu(cr4_update_pce, NULL, 1);
@@ -3073,11 +3128,12 @@ void perf_get_x86_pmu_capability(struct x86_pmu_capability *cap)
cap->version = x86_pmu.version;
cap->num_counters_gp = x86_pmu_num_counters(NULL);
cap->num_counters_fixed = x86_pmu_num_counters_fixed(NULL);
- cap->bit_width_gp = x86_pmu.cntval_bits;
- cap->bit_width_fixed = x86_pmu.cntval_bits;
+ cap->bit_width_gp = cap->num_counters_gp ? x86_pmu.cntval_bits : 0;
+ cap->bit_width_fixed = cap->num_counters_fixed ? x86_pmu.cntval_bits : 0;
cap->events_mask = (unsigned int)x86_pmu.events_maskl;
cap->events_mask_len = x86_pmu.events_mask_len;
cap->pebs_ept = x86_pmu.pebs_ept;
+ cap->mediated = !!(pmu.capabilities & PERF_PMU_CAP_MEDIATED_VPMU);
}
EXPORT_SYMBOL_FOR_KVM(perf_get_x86_pmu_capability);

arch/x86/events/intel/core.c

View file
diff --git a/arch/x86/events/intel/core.c b/arch/x86/events/intel/core.c
index bdf3f0d..f3ae1f8 100644
--- a/arch/x86/events/intel/core.c
+++ b/arch/x86/events/intel/core.c
@@ -232,6 +232,29 @@ static struct event_constraint intel_skt_event_constraints[] __read_mostly = {
EVENT_CONSTRAINT_END
};
+static struct event_constraint intel_arw_event_constraints[] __read_mostly = {
+ FIXED_EVENT_CONSTRAINT(0x00c0, 0), /* INST_RETIRED.ANY */
+ FIXED_EVENT_CONSTRAINT(0x003c, 1), /* CPU_CLK_UNHALTED.CORE */
+ FIXED_EVENT_CONSTRAINT(0x0300, 2), /* pseudo CPU_CLK_UNHALTED.REF */
+ FIXED_EVENT_CONSTRAINT(0x013c, 2), /* CPU_CLK_UNHALTED.REF_TSC_P */
+ FIXED_EVENT_CONSTRAINT(0x0073, 4), /* TOPDOWN_BAD_SPECULATION.ALL */
+ FIXED_EVENT_CONSTRAINT(0x019c, 5), /* TOPDOWN_FE_BOUND.ALL */
+ FIXED_EVENT_CONSTRAINT(0x02c2, 6), /* TOPDOWN_RETIRING.ALL */
+ INTEL_UEVENT_CONSTRAINT(0x01b7, 0x1),
+ INTEL_UEVENT_CONSTRAINT(0x02b7, 0x2),
+ INTEL_UEVENT_CONSTRAINT(0x04b7, 0x4),
+ INTEL_UEVENT_CONSTRAINT(0x08b7, 0x8),
+ INTEL_UEVENT_CONSTRAINT(0x01d4, 0x1),
+ INTEL_UEVENT_CONSTRAINT(0x02d4, 0x2),
+ INTEL_UEVENT_CONSTRAINT(0x04d4, 0x4),
+ INTEL_UEVENT_CONSTRAINT(0x08d4, 0x8),
+ INTEL_UEVENT_CONSTRAINT(0x0175, 0x1),
+ INTEL_UEVENT_CONSTRAINT(0x0275, 0x2),
+ INTEL_UEVENT_CONSTRAINT(0x21d3, 0x1),
+ INTEL_UEVENT_CONSTRAINT(0x22d3, 0x1),
+ EVENT_CONSTRAINT_END
+};
+
static struct event_constraint intel_skl_event_constraints[] = {
FIXED_EVENT_CONSTRAINT(0x00c0, 0), /* INST_RETIRED.ANY */
FIXED_EVENT_CONSTRAINT(0x003c, 1), /* CPU_CLK_UNHALTED.CORE */
@@ -435,6 +458,62 @@ static struct extra_reg intel_lnc_extra_regs[] __read_mostly = {
EVENT_EXTRA_END
};
+static struct event_constraint intel_pnc_event_constraints[] = {
+ FIXED_EVENT_CONSTRAINT(0x00c0, 0), /* INST_RETIRED.ANY */
+ FIXED_EVENT_CONSTRAINT(0x0100, 0), /* INST_RETIRED.PREC_DIST */
+ FIXED_EVENT_CONSTRAINT(0x003c, 1), /* CPU_CLK_UNHALTED.CORE */
+ FIXED_EVENT_CONSTRAINT(0x0300, 2), /* CPU_CLK_UNHALTED.REF */
+ FIXED_EVENT_CONSTRAINT(0x013c, 2), /* CPU_CLK_UNHALTED.REF_TSC_P */
+ FIXED_EVENT_CONSTRAINT(0x0400, 3), /* SLOTS */
+ METRIC_EVENT_CONSTRAINT(INTEL_TD_METRIC_RETIRING, 0),
+ METRIC_EVENT_CONSTRAINT(INTEL_TD_METRIC_BAD_SPEC, 1),
+ METRIC_EVENT_CONSTRAINT(INTEL_TD_METRIC_FE_BOUND, 2),
+ METRIC_EVENT_CONSTRAINT(INTEL_TD_METRIC_BE_BOUND, 3),
+ METRIC_EVENT_CONSTRAINT(INTEL_TD_METRIC_HEAVY_OPS, 4),
+ METRIC_EVENT_CONSTRAINT(INTEL_TD_METRIC_BR_MISPREDICT, 5),
+ METRIC_EVENT_CONSTRAINT(INTEL_TD_METRIC_FETCH_LAT, 6),
+ METRIC_EVENT_CONSTRAINT(INTEL_TD_METRIC_MEM_BOUND, 7),
+
+ INTEL_EVENT_CONSTRAINT(0x20, 0xf),
+ INTEL_EVENT_CONSTRAINT(0x79, 0xf),
+
+ INTEL_UEVENT_CONSTRAINT(0x0275, 0xf),
+ INTEL_UEVENT_CONSTRAINT(0x0176, 0xf),
+ INTEL_UEVENT_CONSTRAINT(0x04a4, 0x1),
+ INTEL_UEVENT_CONSTRAINT(0x08a4, 0x1),
+ INTEL_UEVENT_CONSTRAINT(0x01cd, 0xfc),
+ INTEL_UEVENT_CONSTRAINT(0x02cd, 0x3),
+
+ INTEL_EVENT_CONSTRAINT(0xd0, 0xf),
+ INTEL_EVENT_CONSTRAINT(0xd1, 0xf),
+ INTEL_EVENT_CONSTRAINT(0xd4, 0xf),
+ INTEL_EVENT_CONSTRAINT(0xd6, 0xf),
+ INTEL_EVENT_CONSTRAINT(0xdf, 0xf),
+ INTEL_EVENT_CONSTRAINT(0xce, 0x1),
+
+ INTEL_UEVENT_CONSTRAINT(0x01b1, 0x8),
+ INTEL_UEVENT_CONSTRAINT(0x0847, 0xf),
+ INTEL_UEVENT_CONSTRAINT(0x0446, 0xf),
+ INTEL_UEVENT_CONSTRAINT(0x0846, 0xf),
+ INTEL_UEVENT_CONSTRAINT(0x0148, 0xf),
+
+ EVENT_CONSTRAINT_END
+};
+
+static struct extra_reg intel_pnc_extra_regs[] __read_mostly = {
+ /* must define OMR_X first, see intel_alt_er() */
+ INTEL_UEVENT_EXTRA_REG(0x012a, MSR_OMR_0, 0x40ffffff0000ffffull, OMR_0),
+ INTEL_UEVENT_EXTRA_REG(0x022a, MSR_OMR_1, 0x40ffffff0000ffffull, OMR_1),
+ INTEL_UEVENT_EXTRA_REG(0x042a, MSR_OMR_2, 0x40ffffff0000ffffull, OMR_2),
+ INTEL_UEVENT_EXTRA_REG(0x082a, MSR_OMR_3, 0x40ffffff0000ffffull, OMR_3),
+ INTEL_UEVENT_PEBS_LDLAT_EXTRA_REG(0x01cd),
+ INTEL_UEVENT_EXTRA_REG(0x02c6, MSR_PEBS_FRONTEND, 0x9, FE),
+ INTEL_UEVENT_EXTRA_REG(0x03c6, MSR_PEBS_FRONTEND, 0x7fff1f, FE),
+ INTEL_UEVENT_EXTRA_REG(0x40ad, MSR_PEBS_FRONTEND, 0xf, FE),
+ INTEL_UEVENT_EXTRA_REG(0x04c2, MSR_PEBS_FRONTEND, 0x8, FE),
+ EVENT_EXTRA_END
+};
+
EVENT_ATTR_STR(mem-loads, mem_ld_nhm, "event=0x0b,umask=0x10,ldlat=3");
EVENT_ATTR_STR(mem-loads, mem_ld_snb, "event=0xcd,umask=0x1,ldlat=3");
EVENT_ATTR_STR(mem-stores, mem_st_snb, "event=0xcd,umask=0x2");
@@ -650,6 +729,102 @@ static __initconst const u64 glc_hw_cache_extra_regs
},
};
+static __initconst const u64 pnc_hw_cache_event_ids
+ [PERF_COUNT_HW_CACHE_MAX]
+ [PERF_COUNT_HW_CACHE_OP_MAX]
+ [PERF_COUNT_HW_CACHE_RESULT_MAX] =
+{
+ [ C(L1D ) ] = {
+ [ C(OP_READ) ] = {
+ [ C(RESULT_ACCESS) ] = 0x81d0,
+ [ C(RESULT_MISS) ] = 0xe124,
+ },
+ [ C(OP_WRITE) ] = {
+ [ C(RESULT_ACCESS) ] = 0x82d0,
+ },
+ },
+ [ C(L1I ) ] = {
+ [ C(OP_READ) ] = {
+ [ C(RESULT_MISS) ] = 0xe424,
+ },
+ [ C(OP_WRITE) ] = {
+ [ C(RESULT_ACCESS) ] = -1,
+ [ C(RESULT_MISS) ] = -1,
+ },
+ },
+ [ C(LL ) ] = {
+ [ C(OP_READ) ] = {
+ [ C(RESULT_ACCESS) ] = 0x12a,
+ [ C(RESULT_MISS) ] = 0x12a,
+ },
+ [ C(OP_WRITE) ] = {
+ [ C(RESULT_ACCESS) ] = 0x12a,
+ [ C(RESULT_MISS) ] = 0x12a,
+ },
+ },
+ [ C(DTLB) ] = {
+ [ C(OP_READ) ] = {
+ [ C(RESULT_ACCESS) ] = 0x81d0,
+ [ C(RESULT_MISS) ] = 0xe12,
+ },
+ [ C(OP_WRITE) ] = {
+ [ C(RESULT_ACCESS) ] = 0x82d0,
+ [ C(RESULT_MISS) ] = 0xe13,
+ },
+ },
+ [ C(ITLB) ] = {
+ [ C(OP_READ) ] = {
+ [ C(RESULT_ACCESS) ] = -1,
+ [ C(RESULT_MISS) ] = 0xe11,
+ },
+ [ C(OP_WRITE) ] = {
+ [ C(RESULT_ACCESS) ] = -1,
+ [ C(RESULT_MISS) ] = -1,
+ },
+ [ C(OP_PREFETCH) ] = {
+ [ C(RESULT_ACCESS) ] = -1,
+ [ C(RESULT_MISS) ] = -1,
+ },
+ },
+ [ C(BPU ) ] = {
+ [ C(OP_READ) ] = {
+ [ C(RESULT_ACCESS) ] = 0x4c4,
+ [ C(RESULT_MISS) ] = 0x4c5,
+ },
+ [ C(OP_WRITE) ] = {
+ [ C(RESULT_ACCESS) ] = -1,
+ [ C(RESULT_MISS) ] = -1,
+ },
+ [ C(OP_PREFETCH) ] = {
+ [ C(RESULT_ACCESS) ] = -1,
+ [ C(RESULT_MISS) ] = -1,
+ },
+ },
+ [ C(NODE) ] = {
+ [ C(OP_READ) ] = {
+ [ C(RESULT_ACCESS) ] = -1,
+ [ C(RESULT_MISS) ] = -1,
+ },
+ },
+};
+
+static __initconst const u64 pnc_hw_cache_extra_regs
+ [PERF_COUNT_HW_CACHE_MAX]
+ [PERF_COUNT_HW_CACHE_OP_MAX]
+ [PERF_COUNT_HW_CACHE_RESULT_MAX] =
+{
+ [ C(LL ) ] = {
+ [ C(OP_READ) ] = {
+ [ C(RESULT_ACCESS) ] = 0x4000000000000001,
+ [ C(RESULT_MISS) ] = 0xFFFFF000000001,
+ },
+ [ C(OP_WRITE) ] = {
+ [ C(RESULT_ACCESS) ] = 0x4000000000000002,
+ [ C(RESULT_MISS) ] = 0xFFFFF000000002,
+ },
+ },
+};
+
/*
* Notes on the events:
* - data reads do not include code reads (comparable to earlier tables)
@@ -2167,6 +2342,26 @@ static __initconst const u64 tnt_hw_cache_extra_regs
},
};
+static __initconst const u64 arw_hw_cache_extra_regs
+ [PERF_COUNT_HW_CACHE_MAX]
+ [PERF_COUNT_HW_CACHE_OP_MAX]
+ [PERF_COUNT_HW_CACHE_RESULT_MAX] = {
+ [C(LL)] = {
+ [C(OP_READ)] = {
+ [C(RESULT_ACCESS)] = 0x4000000000000001,
+ [C(RESULT_MISS)] = 0xFFFFF000000001,
+ },
+ [C(OP_WRITE)] = {
+ [C(RESULT_ACCESS)] = 0x4000000000000002,
+ [C(RESULT_MISS)] = 0xFFFFF000000002,
+ },
+ [C(OP_PREFETCH)] = {
+ [C(RESULT_ACCESS)] = 0x0,
+ [C(RESULT_MISS)] = 0x0,
+ },
+ },
+};
+
EVENT_ATTR_STR(topdown-fe-bound, td_fe_bound_tnt, "event=0x71,umask=0x0");
EVENT_ATTR_STR(topdown-retiring, td_retiring_tnt, "event=0xc2,umask=0x0");
EVENT_ATTR_STR(topdown-bad-spec, td_bad_spec_tnt, "event=0x73,umask=0x6");
@@ -2225,6 +2420,22 @@ static struct extra_reg intel_cmt_extra_regs[] __read_mostly = {
EVENT_EXTRA_END
};
+static struct extra_reg intel_arw_extra_regs[] __read_mostly = {
+ /* must define OMR_X first, see intel_alt_er() */
+ INTEL_UEVENT_EXTRA_REG(0x01b7, MSR_OMR_0, 0xc0ffffffffffffffull, OMR_0),
+ INTEL_UEVENT_EXTRA_REG(0x02b7, MSR_OMR_1, 0xc0ffffffffffffffull, OMR_1),
+ INTEL_UEVENT_EXTRA_REG(0x04b7, MSR_OMR_2, 0xc0ffffffffffffffull, OMR_2),
+ INTEL_UEVENT_EXTRA_REG(0x08b7, MSR_OMR_3, 0xc0ffffffffffffffull, OMR_3),
+ INTEL_UEVENT_EXTRA_REG(0x01d4, MSR_OMR_0, 0xc0ffffffffffffffull, OMR_0),
+ INTEL_UEVENT_EXTRA_REG(0x02d4, MSR_OMR_1, 0xc0ffffffffffffffull, OMR_1),
+ INTEL_UEVENT_EXTRA_REG(0x04d4, MSR_OMR_2, 0xc0ffffffffffffffull, OMR_2),
+ INTEL_UEVENT_EXTRA_REG(0x08d4, MSR_OMR_3, 0xc0ffffffffffffffull, OMR_3),
+ INTEL_UEVENT_PEBS_LDLAT_EXTRA_REG(0x5d0),
+ INTEL_UEVENT_EXTRA_REG(0x0127, MSR_SNOOP_RSP_0, 0xffffffffffffffffull, SNOOP_0),
+ INTEL_UEVENT_EXTRA_REG(0x0227, MSR_SNOOP_RSP_1, 0xffffffffffffffffull, SNOOP_1),
+ EVENT_EXTRA_END
+};
+
EVENT_ATTR_STR(topdown-fe-bound, td_fe_bound_skt, "event=0x9c,umask=0x01");
EVENT_ATTR_STR(topdown-retiring, td_retiring_skt, "event=0xc2,umask=0x02");
EVENT_ATTR_STR(topdown-be-bound, td_be_bound_skt, "event=0xa4,umask=0x02");
@@ -2917,6 +3128,8 @@ static void intel_pmu_enable_fixed(struct perf_event *event)
bits |= INTEL_FIXED_0_USER;
if (hwc->config & ARCH_PERFMON_EVENTSEL_OS)
bits |= INTEL_FIXED_0_KERNEL;
+ if (hwc->config & ARCH_PERFMON_EVENTSEL_RDPMC_USER_DISABLE)
+ bits |= INTEL_FIXED_0_RDPMC_USER_DISABLE;
/*
* ANY bit is supported in v3 and up
@@ -3052,6 +3265,27 @@ static void intel_pmu_enable_event_ext(struct perf_event *event)
__intel_pmu_update_event_ext(hwc->idx, ext);
}
+static void intel_pmu_update_rdpmc_user_disable(struct perf_event *event)
+{
+ if (!x86_pmu_has_rdpmc_user_disable(event->pmu))
+ return;
+
+ /*
+ * Counter scope's user-space rdpmc is disabled by default
+ * except two cases.
+ * a. rdpmc = 2 (user space rdpmc enabled unconditionally)
+ * b. rdpmc = 1 and the event is not a system-wide event.
+ * The count of non-system-wide events would be cleared when
+ * context switches, so no count data is leaked.
+ */
+ if (x86_pmu.attr_rdpmc == X86_USER_RDPMC_ALWAYS_ENABLE ||
+ (x86_pmu.attr_rdpmc == X86_USER_RDPMC_CONDITIONAL_ENABLE &&
+ event->ctx->task))
+ event->hw.config &= ~ARCH_PERFMON_EVENTSEL_RDPMC_USER_DISABLE;
+ else
+ event->hw.config |= ARCH_PERFMON_EVENTSEL_RDPMC_USER_DISABLE;
+}
+
DEFINE_STATIC_CALL_NULL(intel_pmu_enable_event_ext, intel_pmu_enable_event_ext);
static void intel_pmu_enable_event(struct perf_event *event)
@@ -3060,6 +3294,8 @@ static void intel_pmu_enable_event(struct perf_event *event)
struct hw_perf_event *hwc = &event->hw;
int idx = hwc->idx;
+ intel_pmu_update_rdpmc_user_disable(event);
+
if (unlikely(event->attr.precise_ip))
static_call(x86_pmu_pebs_enable)(event);
@@ -3532,17 +3768,32 @@ static int intel_alt_er(struct cpu_hw_events *cpuc,
struct extra_reg *extra_regs = hybrid(cpuc->pmu, extra_regs);
int alt_idx = idx;

This diff is incomplete. Download the complete patch.

arch/x86/events/intel/cstate.c

View file

This diff is incomplete. Download the complete patch.

arch/x86/events/intel/ds.c

View file

This diff is incomplete. Download the complete patch.

arch/x86/events/intel/p6.c

View file

This diff is incomplete. Download the complete patch.

arch/x86/events/intel/uncore.c

View file

This diff is incomplete. Download the complete patch.

arch/x86/events/intel/uncore.h

View file

This diff is incomplete. Download the complete patch.

arch/x86/events/intel/uncore_discovery.c

View file

This diff is incomplete. Download the complete patch.

arch/x86/events/intel/uncore_discovery.h

View file

This diff is incomplete. Download the complete patch.

arch/x86/events/intel/uncore_snb.c

View file

This diff is incomplete. Download the complete patch.

arch/x86/events/intel/uncore_snbep.c

View file

This diff is incomplete. Download the complete patch.

arch/x86/events/msr.c

View file

This diff is incomplete. Download the complete patch.

arch/x86/events/perf_event.h

View file

This diff is incomplete. Download the complete patch.

arch/x86/include/asm/amd/ibs.h

View file

This diff is incomplete. Download the complete patch.

arch/x86/include/asm/hardirq.h

View file

This diff is incomplete. Download the complete patch.

arch/x86/include/asm/idtentry.h

View file

This diff is incomplete. Download the complete patch.

arch/x86/include/asm/irq_vectors.h

View file

This diff is incomplete. Download the complete patch.

arch/x86/include/asm/msr-index.h

View file

This diff is incomplete. Download the complete patch.

arch/x86/include/asm/perf_event.h

View file

This diff is incomplete. Download the complete patch.

arch/x86/include/asm/unwind_user.h

View file

This diff is incomplete. Download the complete patch.

arch/x86/kernel/idt.c

View file

This diff is incomplete. Download the complete patch.

arch/x86/kernel/irq.c

View file

This diff is incomplete. Download the complete patch.

arch/x86/kernel/uprobes.c

View file

This diff is incomplete. Download the complete patch.

arch/x86/kvm/Kconfig

View file

This diff is incomplete. Download the complete patch.

include/asm-generic/Kbuild

View file

This diff is incomplete. Download the complete patch.

include/linux/perf_event.h

View file

This diff is incomplete. Download the complete patch.

include/linux/unwind_user.h

View file

This diff is incomplete. Download the complete patch.

include/linux/uprobes.h

View file

This diff is incomplete. Download the complete patch.

include/uapi/linux/perf_event.h

View file

This diff is incomplete. Download the complete patch.

init/Kconfig

View file

This diff is incomplete. Download the complete patch.

kernel/events/core.c

View file

This diff is incomplete. Download the complete patch.

kernel/events/uprobes.c

View file

This diff is incomplete. Download the complete patch.

kernel/unwind/user.c

View file

This diff is incomplete. Download the complete patch.

tools/include/uapi/linux/perf_event.h

View file

This diff is incomplete. Download the complete patch.

tools/perf/trace/beauty/arch/x86/include/asm/irq_vectors.h

View file

This diff is incomplete. Download the complete patch.

tools/perf/util/pmu.c

View file

This diff is incomplete. Download the complete patch.

virt/kvm/kvm_main.c

View file

This diff is incomplete. Download the complete patch.