\page phi_knl Intel® Xeon Phi (Knights Landing)

# Available performance monitors for the Intel® Xeon Phi (Knights Landing) microarchitecture

- \ref KNL_FIXED "Fixed-purpose counters"
- \ref KNL_PMC "General-purpose counters"
- \ref KNL_THERMAL "Thermal counters"
- \ref KNL_POWER "Power measurement counters"
- \ref KNL_UBOX "Uncore management counters"
- \ref KNL_CBOX "Last level cache counters"
- \ref KNL_WBOX "Power control unit general-purpose counters"
- \ref KNL_MBOX "Memory controller (iMC) counters"
- \ref KNL_EBOX "Embedded DRAM controller (EDC) counters"
- \ref KNL_PBOX "Ring-to-PCIe counters"
- \ref KNL_IBOX "IRP box counters"

# Counters available for each hardware thread

\anchor KNL_FIXED

## Fixed-purpose counters

Since the Core2 microarchitecture, Intel® provides a set of fixed-purpose counters. Each can measure only one specific event.

### Counter and events

Counter name | Event name
--- | ---
FIXC0 | INSTR_RETIRED_ANY
FIXC1 | CPU_CLK_UNHALTED_CORE
FIXC2 | CPU_CLK_UNHALTED_REF

### Available Options

Option | Argument | Description | Comment
--- | --- | --- | ---
anythread | N | Set bit 2+(index*4) in config register |
kernel | N | Set bit (index*4) in config register |

\anchor KNL_PMC

## General-purpose counters

The Intel® Xeon Phi (Knights Landing) microarchitecture provides 2 general-purpose counters consisting of a config and a counter register.

### Counter and events

Counter name | Event name
--- | ---
PMC0 | *
PMC1 | *

### Available Options

Option | Argument | Description | Comment
--- | --- | --- | ---
edgedetect | N | Set bit 18 in config register |
kernel | N | Set bit 17 in config register |
threshold | 8 bit hex value | Set bits 24-31 in config register |
invert | N | Set bit 23 in config register |

### Special handling for events

The Intel® Xeon Phi (Knights Landing) microarchitecture provides measuring of offcore events in PMC counters. Therefore the stream of offcore events must be filtered using the OFFCORE_RESPONSE registers. The Intel® Xeon Phi (Knights Landing) microarchitecture has two of those registers. LIKWID defines some events that perform the filtering according to the event name. Although there are many bitmasks possible, LIKWID natively provides only the ones with response type ANY. Own filtering can be applied with the OFFCORE_RESPONSE_0_OPTIONS and OFFCORE_RESPONSE_1_OPTIONS events. Only OFFCORE_RESPONSE_0_OPTIONS can be used to measure average latencies. Only for those events two more counter options are available:

Option | Argument | Description | Comment
--- | --- | --- | ---
match0 | 16 bit hex value | Input value masked with 0xFFFF and written to bits 0-15 in the OFFCORE_RESPONSE register | Check the Intel® Software Developer System Programming Manual, Vol. 3, Chapter Performance Monitoring and https://download.01.org/perfmon/SLM.
match1 | 22 bit hex value | Input value is written to bits 16-38 in the OFFCORE_RESPONSE register | Check the Intel® Software Developer System Programming Manual, Vol. 3, Chapter Performance Monitoring and https://download.01.org/perfmon/SLM.

\anchor KNL_THERMAL

## Thermal counter

The Intel® Xeon Phi (Knights Landing) microarchitecture provides one register for the current core temperature.

### Counter and events

Counter name | Event name
--- | ---
TMP0 | TEMP_CORE

# Counters available for one hardware thread per socket

\anchor KNL_POWER

## Power counters

The Intel® Xeon Phi (Knights Landing) microarchitecture provides measurements of the current power consumption through the RAPL interface.

### Counter and events

Counter name | Event name
--- | ---
PWR0 | PWR_PKG_ENERGY
PWR1 | PWR_PP0_ENERGY
PWR3 | PWR_DRAM_ENERGY

\anchor KNL_UBOX

## Uncore management counters

The Intel® Xeon Phi (Knights Landing) microarchitecture provides measurements of the management box in the Uncore. The description from Intel®:
The Uncore management performance counters are exposed to the operating system through the MSR interface. The name UBOX originates from the Nehalem EX Uncore monitoring where those functional units are called UBOX.

### Counter and events

Counter name | Event name
--- | ---
UBOX0 | *
UBOX1 | *
UBOXFIX | UBOX_CLOCKTICKS

### Available Options (Only for UBOX<0,1> counters)

Option | Argument | Operation | Comment
--- | --- | --- | ---
edgedetect | N | Set bit 18 in config register |
threshold | 5 bit hex value | Set bits 24-28 in config register |
invert | N | Set bit 23 in config register |

\anchor KNL_CBOX

## Last level cache counters

The Intel® Xeon Phi (KNL) microarchitecture provides measurements for the last level cache segments.

### Counter and events

Counter name | Event name
--- | ---
CBOX<0-37>C0 | *
CBOX<0-37>C1 | *
CBOX<0-37>C2 | *
CBOX<0-37>C3 | *

### Available Options

Option | Argument | Description | Comment
--- | --- | --- | ---
edgedetect | N | Set bit 18 in config register |
invert | N | Set bit 23 in config register |
threshold | 8 bit hex value | Set bits 24-31 in config register |
opcode | 9 bit hex value | Set bits 9-28 in PERF_UNIT_CTL_1_CHA_<0-37> register | A list of valid opcodes can be found in the Intel® Xeon® Phi Processor Performance Monitoring Reference Manual.
state | 10 bit hex value | Set bits 17-26 in PERF_UNIT_CTL_CHA_<0-37> register | H: 0x08, E: 0x04, S: 0x02 All other bits reserved.
tid | 9 bit hex value | Set bits 0-8 in PERF_UNIT_CTL_CHA_<0-37> register and enables TID filtering with bit 19 in config register | 0-2 ThreadID, 3-8 CoreID
nid | 2 bit hex value | Set bits 0-1 in PERF_UNIT_CTL_1_CHA_<0-37> register | Remote: 0x1 Local: 0x2
match0 | 3 bit hex address | Set bits 29-31 in PERF_UNIT_CTL_1_CHA_<0-37> register | C6Opcode: 0x1 NonCohOpcode: 0x2 IsocOpcode: 0x3
match1 | 2 bit hex address | Set bits 4-5 in PERF_UNIT_CTL_1_CHA_<0-37> register | Count near memory cache events: 0x1 Count non-near memory cache events: 0x2

### Special handling for events

The Intel® Xeon Phi (KNL) microarchitecture provides an event LLC_LOOKUP which can be filtered with the 'state' option. If no 'state' is set, LIKWID sets the state to 0xE, the default value to measure all lookups.
If the match1 option is not used, bits 4 and 5 in PERF_UNIT_CTL_1_CHA_<0-37> are set.
If no opcode option is set, the bit 3 in PERF_UNIT_CTL_1_CHA_<0-37> is set.

\anchor KNL_WBOX

## Power control unit general-purpose counters

The Intel® Xeon Phi (KNL) microarchitecture provides measurements of the power control unit (PCU) in the uncore.

The PCU performance counters are exposed to the operating system through the MSR interface. The name WBOX originates from the Nehalem EX uncore monitoring.

### Counter and events

Counter name | Event name
--- | ---
WBOX0 | *
WBOX1 | *
WBOX2 | *
WBOX3 | *

### Available Options

Option | Argument | Operation | Comment
--- | --- | --- | ---
edgedetect | N | Set bit 18 in config register |
invert | N | Set bit 23 in config register |
threshold | 5 bit hex value | Set bits 24-28 in config register |
match0 | 32 bit hex value | Set bits 0-31 in MSR_UNC_PCU_PMON_BOX_FILTER register | Band0: bits 0-7, Band1: bits 8-15, Band2: bits 16-23, Band3: bits 24-31
occupancy | 2 bit hex value | Set bit 14-15 in config register | Cores in C0: 0x1, in C3: 0x2, in C6: 0x3
occ_edgedetect | N | Set bit 31 in config register |
occ_invert | N | Set bit 30 in config register |

\anchor KNL_MBOX

## Memory controller (iMC) counters

The Intel® Xeon Phi (KNL) microarchitecture provides measurements of the integrated Memory Controllers (iMC) in the uncore. The description from Intel®:
The processor implements two Memory Controllers on the processor die. Each memory controller is capable of controlling three DDR4 memory channels. The MC design is derived from the EDC (Near-Memory (MCDRAM) controller) and is a sub-set of EDC in functionality. The main difference from EDC is that the physical interface for MC will be DDR4 IOs. The processor MC will interface with the rest of the Untile via the mesh interface (R2Mem -> Ring-to-MC interface). Therefore, the MC agent is broken into three regions: The front-end ring/mesh interface called the "R2Mem", the core "EDC controller" logic, and three individual "DDR channel controllers/schedulers."

The integrated Memory Controllers performance counters are exposed to the operating system through PCI interfaces. There may be two memory controllers in the system. There are four different PCI devices per memory controller, three for each memory channel and one for the controller. Each device has four different general-purpose counters. The three channels of the first memory controller are MBOX0-2, the memory controller itself is MBOX3. The three channels of the second memory controller (if available) are named MBOX4-6 and the corresponding controller MBOX7. The name MBOX originates from the Nehalem EX uncore monitoring.

### Counter and events

Counter name | Event name
--- | ---
MBOX<0-2,4-6>C0 | MC_DCLK, MC_CAS*
MBOX<0-2,4-6>C1 | MC_DCLK, MC_CAS*
MBOX<0-2,4-6>C2 | MC_DCLK, MC_CAS*
MBOX<0-2,4-6>C3 | MC_DCLK, MC_CAS*
MBOX<3,7>C0 | MC_UCLK
MBOX<3,7>C1 | MC_UCLK
MBOX<3,7>C2 | MC_UCLK
MBOX<3,7>C3 | MC_UCLK
MBOX<0-7>FIX | DRAM_CLOCKTICKS

### Available Options

Option | Argument | Operation | Comment
--- | --- | --- | ---
edgedetect | N | Set bit 18 in config register |
invert | N | Set bit 23 in config register |
threshold | 8 bit hex value | Set bits 24-31 in config register |

\anchor KNL_EBOX

## Embedded DRAM controller (EDC) counters

The Intel® Xeon Phi (KNL) microarchitecture provides measurements of the Embedded DRAM Controllers (EDC) in the uncore, the interface to the MCDRAM. The description from Intel®:
The EDC is the high bandwidth near-memory controller for the processor. EDC refers to "Embedded DRAM Controller" (i.e. DRAM that is embedded in the processor package). The technology that is used to implement the embedded DRAM for the processor is MCDRAM (Multi-Chip (Stacked) DRAM). Eight channels of MCDRAM are supported by 8 MCDRAM Controllers (EDC). The EDC's are connected to the other components (clusters) within the processor by the internal mesh interconnect fabric.

The Embedded DRAM Controllers (EDC) performance counters are exposed to the operating system through PCI interfaces. There are eight embedded memory controllers in the system. There are two different PCI devices per memory controller, one for the mesh side (EUBOXC)and one on the DRAM side (EDBOXC). Each device has four different general-purpose counters.

The fixed-purpose counters are exposed to the operating system through PCI interfaces. There are eight embedded memory controllers in the system. There are two different PCI devices per memory controller, one for the mesh side (EUBOXFIX)and one on the DRAM side (EDBOXFIX).

### Counter and events

Counter name | Event name
--- | ---
EUBOX<0-7>C0 | EDC_UCLK, EDC_HIT_*, EDC_MISS_*
EUBOX<0-7>C1 | EDC_UCLK, EDC_HIT_*, EDC_MISS_*
EUBOX<0-7>C2 | EDC_UCLK, EDC_HIT_*, EDC_MISS_*
EUBOX<0-7>C3 | EDC_UCLK, EDC_HIT_*, EDC_MISS_*
EUBOX<0-7>FIX | EDC_CLOCKTICKS
EDBOX<0-7>C0 | EDC_ECLK, EDC_WPQ_INSERTS, EDC_RPQ_INSERTS
EDBOX<0-7>C1 | EDC_ECLK, EDC_WPQ_INSERTS, EDC_RPQ_INSERTS
EDBOX<0-7>C2 | EDC_ECLK, EDC_WPQ_INSERTS, EDC_RPQ_INSERTS
EDBOX<0-7>C3 | EDC_ECLK, EDC_WPQ_INSERTS, EDC_RPQ_INSERTS
EDBOX<0-7>FIX | MCDRAM_CLOCKTICKS

### Available Options

Option | Argument | Operation | Comment
--- | --- | --- | ---
edgedetect | N | Set bit 18 in config register |
invert | N | Set bit 23 in config register |
threshold | 8 bit hex value | Set bits 24-31 in config register |

\anchor KNL_PBOX

## Ring-to-PCIe counters

The Intel® Xeon Phi (KNL) microarchitecture provides measurements of the Ring-to-PCIe (R2PCIe) interface in the uncore. The description from Intel®:
The M2PCI is the logic which interfaces the IIO modules to the mesh and includes the mesh stop.

The Ring-to-PCIe performance counters are exposed to the operating system through a PCI interface. Independent of the system's configuration, there is only one Ring-to-PCIe interface per CPU socket.

### Counter and events

Counter name | Event name
--- | ---
PBOX0 | *
PBOX1 | *
PBOX2 | *
PBOX3 | *

### Available Options

Option | Argument | Operation | Comment
--- | --- | --- | ---
edgedetect | N | Set bit 18 in config register |
invert | N | Set bit 23 in config register |
threshold | 8 bit hex value | Set bits 24-31 in config register |

\anchor KNL_IBOX

## IRP box counters

The Intel® Xeon Phi (KNL) microarchitecture provides measurements of the IRP box in the uncore. The description from Intel®:
IRP is responsible for maintaining coherency for IIO traffic that needs to be coherent (e.g. cross-socket P2P).

The IRP box counters are exposed to the operating system through the PCI interface. The IBOX was introduced with the Intel® IvyBridge EP/EN/EX microarchitecture.

### Counter and events

Counter name | Event name
--- | ---
IBOX0 | *
IBOX1 | *

### Available Options

Option | Argument | Operation | Comment
--- | --- | --- | ---
edgedetect | N | Set bit 18 in config register |
invert | N | Set bit 23 in config register |
threshold | 8 bit hex value | Set bits 24-31 in config register |