\page sapphirerapids Intel® SapphireRapids

This page is valid for SapphireRapids.

# Available performance monitors for the Intel® SapphireRapids microarchitecture

- \ref SPR_FIXED "Fixed-purpose counters"
- \ref SPR_METRICS "Performance metric counters"
- \ref SPR_PMC "General-purpose counters"
- \ref SPR_THERMAL "Thermal counters"
- \ref SPR_VOLTAGE "Core voltage counters"
- \ref SPR_POWER "Power measurement counters"
- \ref SPR_UBOX "Uncore global counters"
- \ref SPR_CBOX "Last level cache counters"
- \ref SPR_MBOX "Memory channel counters"
- \ref SPR_WBOX "Power control unit counters"
- \ref SPR_QBOX "UPI interface counters"
- \ref SPR_SBOX "Mesh-to-UPI counters (M3UPI)"
- \ref SPR_IBOX "IIO box counters"
- \ref SPR_BBOX "Mesh2Memory counters"
- \ref SPR_HBM "High bandwidth memory counters"

## Some units are currently not supported

- free-running IIO counters
- fixed-purpose counters in \ref SPR_WBOX
- free-running memory controller counters
- M2HBM unit

# Counters available for each hardware thread

\anchor SPR_FIXED

## Fixed-purpose counters

Since the Core2 microarchitecture, Intel® provides a set of fixed-purpose counters. Each can measure only one specific event. The Intel® SapphireRapids architecture adds a fourth fixed-purpose counter for the event TOPDOWN_SLOTS.

### Counter and events

Counter name | Event name
--- | ---
FIXC0 | INSTR_RETIRED_ANY
FIXC1 | CPU_CLK_UNHALTED_CORE
FIXC2 | CPU_CLK_UNHALTED_REF
FIXC3 | TOPDOWN_SLOTS

### Available Options

Option | Argument | Description | Comment
--- | --- | --- | ---
anythread | N | Set bit 2+(index*4) in config register |
kernel | N | Set bit (index*4) in config register |

\anchor SPR_METRICS

## Performance metric counters

With the Intel® Icelake SP microarchitecture a new class of core-local counters was introduced, the so-called perf-metrics. The reflect the first level of the Top-down Microarchitecture Analysis tree. The events return the fraction of slots used by the event. With Intel SapphireRapids, four additional events have been added that partly reflect the second level of the TMA tree. The metrics are only supported when built with `ACCESSMODE=accessdaemon` or `ACCESSMODE=direct`.

### Counter and events

Counter name | Event name
--- | ---
TMA0 | RETIRING
TMA1 | BAD_SPECULATION
TMA2 | FRONTEND_BOUND
TMA3 | BACKEND_BOUND
TMA4 | HEAVY_OPS
TMA5 | BR_MISPREDICT
TMA6 | FETCH_LATENCY
TMA7 | MEM_BOUND

\anchor SPR_PMC

## General-purpose counters

Commonly the Intel® SapphireRapids microarchitecture provides 4 general-purpose counters consisting of a config and a counter register.

### Counter and events

Counter name | Event name
--- | ---
PMC0 | *
PMC1 | *
PMC2 | *
PMC3 | *

### Available Options

Option | Argument | Description | Comment
--- | --- | --- | ---
edgedetect | N | Set bit 18 in config register |
kernel | N | Set bit 17 in config register |
anythread | N | Set bit 21 in config register | The anythread option is deprecated but still supported. Please check the official Intel® documentation how to use the option properly.
threshold | 8 bit hex value | Set bits 24-31 in config register |
invert | N | Set bit 23 in config register |
in_transaction | N | Set bit 32 in config register | Only available if Intel® Transactional Synchronization Extensions are available
in_transaction_aborted | N | Set bit 33 in config register | Only counter PMC2 and only if Intel® Transactional Synchronization Extensions are available

If HyperThreading is disabled, each core can use 8 general-purpose counters named <4-7>. \anchor SPR_THERMAL

## Thermal counters

The Intel® SapphireRapids microarchitecture provides one register for the current core temperature.

### Counter and events

Counter name | Event name
--- | ---
TMP0 | TEMP_CORE

\anchor SPR_VOLTAGE

## Core voltage counters

The Intel® SapphireRapids microarchitecture provides one register for the current core voltage.

### Counter and events

Counter name | Event name
--- | ---
VTG0 | VOLTAGE_CORE

# Counters available for one hardware thread per socket

\anchor SPR_POWER

## Power counter

The Intel® Icelake SP microarchitecture provides measurements of the current power consumption through the RAPL interface.

### Counter and events

Counter name | Event name
--- | ---
PWR0 | PWR_PKG_ENERGY
PWR1 | PWR_PP0_ENERGY
PWR2 | PWR_PP1_ENERGY
PWR3 | PWR_DRAM_ENERGY
PWR4 | PWR_PLATFORM_ENERGY

\anchor SPR_UBOX

## Uncore global counters

The Intel® SapphireRapids microarchitecture provides measurements for the global uncore.

### Counter and events

Counter name | Event name
--- | ---
UBOX0 | *
UBOX1 | *
UBOXFIX | UNCORE_CLOCK

### Available Options

Option | Argument | Description | Comment
--- | --- | --- | ---
edgedetect | N | Set bit 18 in config register |
threshold | 8 bit hex value | Set bits 24-31 in config register |
invert | N | Set bit 23 in config register |

\anchor SPR_CBOX

## Last level cache counters

The Intel® SapphireRapids microarchitecture provides measurements for the last level cache segments.

### Counter and events

Counter name | Event name
--- | ---
CBOX<0-55>C0 | *
CBOX<0-55>C1 | *
CBOX<0-55>C2 | *
CBOX<0-55>C3 | *

### Available Options

Option | Argument | Description | Comment
--- | --- | --- | ---
edgedetect | N | Set bit 18 in config register |
threshold | 8 bit hex value | Set bits 24-28 in config register |
invert | N | Set bit 23 in config register |

\anchor SPR_MBOX

## Memory channel counters

The Intel® SapphireRapids microarchitecture provides measurements of the memory channels of each integrated Memory Controllers (iMC) in the Uncore.
The integrated Memory Controllers performance counters are exposed to the operating system through MMIO interfaces. There may be 8 memory controllers in the system. Each controller provides four memory channels. Each channel has 4 different general-purpose counters and one fixed counter for the DRAM clock. The channels of the first memory controller are MBOX0-3, the two channels of the second memory controller are named MBOX3-7, and so on. The name MBOX originates from the Nehalem EX Uncore monitoring where those functional units are called MBOX.

### Counter and events

Counter name | Event name
--- | ---
MBOX<0-15>C0 | *
MBOX<0-15>C1 | *
MBOX<0-15>C2 | *
MBOX<0-15>C3 | *
MBOX<0-15>FIX | DRAM_CLOCKTICKS

### Available Options (Only for counter MBOX<0-15>C<0-3>)

Option | Argument | Operation | Comment
--- | --- | --- | ---
edgedetect | N | Set bit 18 in config register |
invert | N | Set bit 23 in config register |
threshold | 8 bit hex value | Set bits 24-31 in config register |

\anchor SPR_WBOX

## Power control unit counters

The Intel® SapphireRapids microarchitecture provides measurements of the power control unit (PCU) in the Uncore.
The PCU performance counters are exposed to the operating system through the MSR interface. The name WBOX originates from the Nehalem EX Uncore monitoring where those functional units are called WBOX.

### Counter and events

Counter name | Event name
--- | ---
PCU0 | *
PCU1 | *
PCU2 | *
PCU3 | *

### Available Options (Only for PCU<0-3> counters)

Option | Argument | Operation | Comment
--- | --- | --- | ---
edgedetect | N | Set bit 18 in config register |
invert | N | Set bit 23 in config register |
threshold | 5 bit hex value | Set bits 24-28 in config register |

\anchor SPR_QBOX

## UPI interface counters

The Intel® SapphireRapids microarchitecture provides measurements of the Ultra Path Interconnect Link layer (UPI LL) in the Uncore.
The UPI hardware performance counters are exposed to the operating system through PCI interfaces. There are four of those interfaces/units/slots for the UPI. The actual amount of UPI units depend on the CPU core count of one socket. If your system has not all interfaces but interface 0 does not work, try the other ones. The QBOX was introduced for the Skylake microarchitecture.

### Counter and events

Counter name | Event name
--- | ---
UPI<0-3>C0 | *
UPI<0-3>C1 | *
UPI<0-3>C2 | *
UPI<0-3>C3 | *

### Available Options

Option | Argument | Operation | Comment
--- | --- | --- | ---
edgedetect | N | Set bit 18 in config register |
invert | N | Set bit 23 in config register |
threshold | 8 bit hex value | Set bits 24-31 in config register |

\anchor SPR_SBOX

## Mesh-to-UPI counters (M3UPI)

The Intel® SapphireRapids microarchitecture provides measurements of the Mesh-to-UPI (M3UPI) interface in the Uncore.
The Mesh-to-UPI performance counters are exposed to the operating system through PCI interfaces. Since the RBOXes manage the traffic from the LLC-connecting mesh interface on the socket with the UPI interfaces (QBOXes), the amount is similar to the amount of QBOXes (4). See at UPI units how many are available for which system configuration.

### Counter and events

Counter name | Event name
--- | ---
RBOX<0-3>C0 | *
RBOX<0-3>C1 | *
RBOX<0-3>C2 | *
RBOX<0-3>C3 | *

### Available Options

Option | Argument | Operation | Comment
--- | --- | --- | ---
edgedetect | N | Set bit 18 in config register |
invert | N | Set bit 23 in config register |
threshold | 8 bit hex value | Set bits 24-31 in config register |

\anchor SPR_IBOX

## IIO box counters

The Intel® SapphireRapids microarchitecture provides measurements of the IIO box in the Uncore.
The IIO box counters are exposed to the operating system through the MSR interface.

### Counter and events

Counter name | Event name
--- | ---
IIO<0-12>C0 | *
IIO<0-12>C1 | *
IIO<0-12>C2 | *
IIO<0-12>C3 | *

### Available Options (Only for IIO<0-12> counters)

Option | Argument | Operation | Comment
--- | --- | --- | ---
edgedetect | N | Set bit 18 in config register |
invert | N | Set bit 23 in config register |
threshold | 8 bit hex value | Set bits 24-31 in config register |

\anchor SPR_BBOX

## Mesh2Memory counters

The Intel® SapphireRapids microarchitecture provides measurements of the mesh (M2M) which connects the cores with the Uncore devices.
The M2M devices is first introduced in the Intel® Skylake SP microarchitecture. There was no suitable unit name for this, so LIKWID calls them simply M2M.

### Counter and events

Counter name | Event name
--- | ---
M2M<0-3>C0 | *
M2M<0-3>C1 | *
M2M<0-3>C2 | *
M2M<0-3>C3 | *

### Available Options

Option | Argument | Operation | Comment
--- | --- | --- | ---
edgedetect | N | Set bit 18 in config register |
invert | N | Set bit 23 in config register |
threshold | 5 bit hex value | Set bits 24-28 in config register |

\anchor SPR_HBM

## High-Bandwidth Memory (HBM) counters

Some SapphireRapids systems provide on-chip HBM. If it is available, there are also corresponding perfmon units. The available events are almost similar to \ref SPR_MBOX.

### Counter and events

Counter name | Event name
--- | ---
HBM<0-32>C0 | *
HBM<0-32>C1 | *
HBM<0-32>C2 | *
HBM<0-32>C3 | *

### Available Options

Option | Argument | Operation | Comment
--- | --- | --- | ---
edgedetect | N | Set bit 18 in config register |
invert | N | Set bit 23 in config register |
threshold | 5 bit hex value | Set bits 24-28 in config register |