Dell EMC NUMA Configuration for AMD EPYC (Naples) Processors Dell Engineering February 2018 A Dell EMC Deployment and Configuration Guide
Revisions Date Description February 2018 Initial release The information in this publication is provided “as is.” Dell Inc. makes no representations or warranties of any kind with respect to the information in this publication, and specifically disclaims implied warranties of merchantability or fitness for a particular purpose. Use, copying, and distribution of any software described in this publication requires an applicable software license. Copyright © 2018 Dell Inc. or its subsidiaries.
Table of contents Revisions.............................................................................................................................................................................2 Executive summary.............................................................................................................................................................4 1 AMD EPYC Architecture .............................................................................................................
Executive summary With the introduction of AMD’s EPYC (Naples) x86 Server CPUs featuring four Zeppelin dies per package there is a need to clarify how AMD’s new silicon design establishes Non-Uniform Memory Access (NUMA) domains across dies and sockets. The goal of this Dell EMC Deployment and Configuration Guide is demonstrate how Dell EMC Servers leverages AMD’s EPYC CPUs to configured NUMA domains for optimal performance by using Dell EMC BIOS Settings.
1 AMD EPYC Architecture 1.1 Zeppelin Die Layout AMD EPYC is a Multi-Chip Module (MCM) processor and per silicon package there are four Zeppelin SOCs/dies leveraged from AMD Ryzen. Each of the four dies have direct Infinity Fabric connections to each of the other dies as well as a possible socket-to-socket interconnect.
1.2 Memory Interleaving The Memory Interleave feature for AMD EPYC processors is what controls how may NUMA domains are generated. AMD EPYC processors support 4 memory interleaving options. Each option becomes available based on system configuration. 1.2.1 Socket Interleaving (2 processor configurations) Die Interleaving Channel Interleaving Memory Interleaving disabled Memory Interleaving Rules The following are the rules for each memory interleave option: 1.2.
NUMA Domain Count per Memory Interleave Option 7 Number of Processors Socket Interleave NUMA Domains Die Interleave NUMA Domains Channel Interleave NUMA Domains 2 1 2 8 1 NA 1 4 Dell EMC NUMA Configuration for AMD EPYC (Naples) Processors | version 1.
2 Performance Tuning For best performance from AMD EPYC processors, it is recommended that each die have one DIMM populated on each channel. This allows all IO behind each die to access memory, with optimal latency. 2.
D1 D0 D1 D0 D1 D0 D1 D0 CPU D0 D1 D0 D1 D0 D1 D0 D1 DIMM Layout 2.2 PCIe Configuration Guidelines 9 When PCIe cards are populated into particular slots with NUMA-unaware application/software, make sure to have memory DIMMs populated in the corresponding NUMA-node mapping as local memory. Mappings can be found in Section 4 Platform Specific NUMA/Die Domain Details Considering also pinning the interrupts to local CPUs to get maximum performance.
3 BIOS Setup The “Memory Interleaving” setting controls whether the system is configured for Socket, Die, Channel interleaving. In System Setup (F2 prompt during system boot), enter System BIOS > Memory Settings and navigate to “Memory Interleaving” to choose the memory interleave for desired configuration. This option is also available in system management consoles such as RACADM. 10 Dell EMC NUMA Configuration for AMD EPYC (Naples) Processors | version 1.
4 Platform Specific NUMA/Die Domain Details The following matrices shows how CPU die, memory and PCIe slots are physically grouped to each NUMA domain for Dell EMC EPYC based platforms, PowerEdge R6415, R7415, and R7425.
PowerEdge R7425 PCIe Riser 1A 1D 1E 2A 2C 2D 3A 3B 12 PCIe Slot / Device CPU SKT CPU Die Memory Slots NUMA Domain 1 1 2 A7, A8, A15, A16 2 3 1 3 A5, A6, A13, A14 3 1 1 2 A7, A8, A15, A16 2 2 1 3 A5, A6, A13, A14 3 3 1 3 A5, A6, A13, A14 3 1 1 2 A7, A8, A15, A16 2 2 1 3 A5, A6, A13, A14 3 4 2 2 B7, B8, B15, B16 6 6 2 3 B5, B6, B13, B14 7 4 2 2 B7, B8, B15, B16 6 4 1 1 A1, A2, A9, A10 1 5 2 2 B7, B8, B15, B16 6 6 2 3 B5, B6, B13, B14 7
5 Technical support and resources 5.1 Dell Dell.com/support is focused on meeting customer needs with proven services and support. Dell TechCenter is an online technical community where IT professionals have access to numerous resources for Dell EMC software, hardware and services. Storage Solutions Technical Documents on Dell TechCenter provide expertise that helps to ensure customer success on Dell EMC Storage platforms. 5.2 AMD https://community.amd.
