VxRail Networking Guide with Dell EMC S4148-ON Switches S4148-ON Deployment and Configuration Guide for VxRail Dell EMC Networking Infrastructure Solutions March 2018
Revisions Date Description Authors March 2018 Initial release Ed Blazek, Jordan Wilson THIS DOCUMENT IS FOR INFORMATIONAL PURPOSES ONLY, AND MAY CONTAIN TYPOGRAPHICAL ERRORS AND TECHNICAL INACCURACIES. THE CONTENT IS PROVIDED AS IS, WITHOUT EXPRESS OR IMPLIED WARRANTIES OF ANY KIND. Copyright © 2018 Dell Inc. All rights reserved. Dell and the Dell EMC logo are trademarks of Dell Inc. in the United States and/or other jurisdictions.
Table of Contents 1 Introduction ...................................................................................................................................................................5 1.1 Networking switches overview............................................................................................................................6 1.1.1 S4100-ON switch series (S4148U, S4148F, S4148FE, S4148T) ......................................................................6 1.1.
.3.2 ISL port channel configuration without VLT ......................................................................................................23 4.3.3 ISL port channel with VLT configuration ...........................................................................................................23 4.4 Network facing uplink configurations ................................................................................................................24 4.4.
1 Introduction VxRail™ sits at the forefront of a fundamental shift in IT infrastructure consumption away from applicationspecific, build-your-own infrastructure and toward virtualized, general-purpose, engineered systems. Dell EMC and VMware have embraced this shift with the VxRail hyper-converged appliance that delivers compelling economics, faster time-to-value, linear scaling over time, and operational simplicity.
1.1 Networking switches overview Dell EMC Networking has introduced a new networking OS, OS10 Enterprise Edition. OS10 is a disaggregated native Linux-based network operating system that has a wholly disaggregated software architecture. OS10 decouples the base software from the Layer 2 and Layer 3 protocol stack and services, which brings forth the ability for open programmability and portability.
1.1.2 S4128-ON switch (S4128F, S4128T) The Dell EMC Networking S4128-ON is a 1RU (Rack-Unit) 1/10/25/40/50/100GbE, top of rack switch running QS10. The S4128-ON switch operates with up to 28 dual-speed, 1/10GbE SFP+ (S4128T-ON with BaseT) ports, and 2 ports QSFP28 optical ports. This switch uses a non-blocking and cut-through switching architecture to provide ultra-low-latency performance for applications. S4128-ON front view 1.1.
1.2 VxRail nodes overview 1.2.1 VxRail P Series nodes The VxRail P Series is a single or dual socket 2RU node. Heavy workloads like databases use optimized VxRail P Series nodes. P Series nodes can be configured with up to four disk groups and offer high-power, high core count, and high clock speed processors. VxRail P Series node front view 1.2.2 VxRail V Series nodes The VxRail V Series is a dual socket 2RU node.
1.2.4 VxRail E Series nodes The VxRail E Series low-profile node is a single or dual-socket 1RU node. The VxRail E Series nodes are best suited for remote office, stretch cluster or entry-level workloads. E Series nodes have two disk groups, providing the capacity and performance to handle a wide variety of use cases. VxRail E Series node front view 1.2.5 The VxRail E, P, V and S Series supports SFP+ or BaseT network adapters.
1.2.6 Basic requirements The customer's IP gateway address and subnet mask. In-band or out-of-band switch management. The IP addresses for switch management. The number and type of Ethernet ports required for VxRail node connectivity in a single or dualswitch configuration. The number and type of Ethernet ports for the S4148-ON inter-switch links (ISLs) used to interconnect the switches.
2 Network topologies A VxRail deployment can be deployed using a single-switch or with a dual-switch configuration. The singleswitch provides the lowest cost option but creates a single point of failure. Dell EMC recommends two switches, as they provide redundancy for high availability. The following explains the difference between single-switch and dual-switch topologies and shows upstream switches to highlight solution examples. 2.1.
2.1.2 Dual-switch topology This topology establishes network redundancy at both the link and the switch level. Failure of a single-switch or network link does not cause an overall loss of communication. In the dual-switch topology, each node connects to both switches in an active/standby capability. Refer to Appendix C for detailed information. The two S4148-ON switches connect with either a VLT configuration or a port channel configuration.
As the customer’s environment grows with added VxRail nodes, a basic topology can quickly scale up to 32 server nodes while utilizing only 64 x 10GbE ToR ports (32 x 10GbE ToR ports per switch) for connecting to the server nodes. The 2 x 100GbE ports on the S4148-ON use ISL ports and upstream connectivity for a potential 2:1 oversubscription ratio 2.1.
2.1.4 Management network VxRail nodes located at a data center may not be easily accessible. Dell EMC recommends setting up an outof-band management switch to facilitate direct communication with each switch and node. One Dell EMC S3048-ON switch is configured in each rack to provide connectivity for access to the management network. Out-of-band (OOB) ports on switches and iDRAC ports on VxRail nodes are connected to the S3048-ON switch for the management network.
3 Switch initial setup This deployment guide assumes that the installer has a basic familiarity with the VxRail node requirements, including the number of nodes for installation, whether the VxRail deployment uses VLANs or if all network traffic resides in a single broadcast domain. The installer should also know how many ports are available on the upstream switches from the S4148-ON and whether they are 10GbE, 25GbE, 40GbE, 50GbE and 100GbE ports. 3.1 Installer laptop connection 1.
3.2.2 Factory default configuration Setting the switch to factory default will delete all configuration settings and restart the switch. Switch-1# delete startup-configuration Proceed to delete startup-configuration [confirm yes/no(default)]:yes Switch-1# reload System configuration has been modified.
3.2.4 Unified ports on the S4148U This section only applies to the S4148U switch. Skip this section if using other S4148 switches (S4148F, S4148FE, and S4148T). To configure unified ports on S4148U switches to Ethernet ports. Refer to the switch user’s guide for further details. Port-group 1/1/1-1/1/6 refers to the first 24 SFP+ ports on the switch. Use the following commands to convert these ports to Ethernet ports.
4 Switch configuration You will now configure the switch or switches using the parameters and values chosen and recorded in the Appendix A and Appendix B worksheets. Power-on the switch and connect the laptop's serial cable to the S4148-ON console port. No passwords or login accounts required for console access in factory default configuration. Table 1 provides overall values used to validate the environment and for example values. Top of Rack Switch Overview of values used as examples in this document.
4.1 Global switch settings and initial setup You will now configure the switch or switches using the parameters and values chosen and recorded in Appendix A and Appendix B worksheets. Power-on the switch and connect the laptop's serial cable to the S4148-ON console port. No passwords or login accounts are required for console access in factory default configuration. 1.
4.2 VxRail node facing interfaces Configure VxRail node-facing interfaces based on the number of nodes and switches selected. Table 2 defines the S4148-ON ports for single-switch and dual-switch topologies.
2. If the network is flat using a single broadcast domain, skip to step 3. If using VLANs, begin configuring VLAN interfaces using the following command: Command Example interface ethernet node/slot/port description desc_text switchport mode trunk switchport trunk allowed vlan vlan-id interface ethernet 1/1/1 description "Appliance 1 Port 1" switchport mode trunk switchport trunk allowed vlan 110,120,130 no ip address The commands above are common for all VLAN interfaces.
b. Switch-1# show running-configuration interface VLAN 120 interface vlan120 description VSAN no shutdown ip address 192.168.120.251/24 ip igmp snooping querier vrrp-group 120 priority 254 virtual-address 192.168.120.254 c.
4.3.2 ISL port channel configuration without VLT Static link aggregation groups (LAGs) establish ISL links between switches. These links between switches require the addition of the port channel created for the ISL to all interface VLANs defined for the network. This topology establishes network redundancy at both the link and the switch level. Failure of a single-switch or network link does not cause an overall loss of communication.
Configure the Inter-Switch Links (ISLs) for dual-switch topologies. a. Configure the Inter-Switch Links (ISLs) port channel with VLT for dual-switch topologies on Switch 1 using the following commands in the first column: (recommended port values shown) b. Configure Switch-2 using the commands in column 2 for Switch-2 ISL port channel with VLT configuration 4.
4.4.1 Network facing uplink configuration – 100GbE 1.
description "40Gb Uplink to Customer 2" channel-group 2 mode active no shutdown no switchport description "40Gb Uplink to Customer 2" channel-group 2 mode active no shutdown no switchport interface ethernet1/1/26 description "40Gb Uplink to Customer 2" channel-group 2 mode active no shutdown no switchport interface ethernet1/1/26 description "40Gb Uplink to Customer 2" channel-group 2 mode active no shutdown no switchport 2.
2. Configure Switch 2 using the commands in column 2 for Switch-2 4.4.4 Network facing uplink configuration - 4 x 10GbE breakout ports QSFP+ and QSFP28 ports on S4148-ON switches can be configured as breakouts with 10GbE uplink ports. To configure these ports as 4 x 10GbE ports, use the following commands. 1. Configure interface breakout node/slot/port map {10g-4x | 25g-4x | 40g-1x | 50g-2x | 100g-1x} Switch-1(config)# interface breakout 1/1/25 map 10g-4x Switch-1(config)# exit Switch-1# 2.
5 Configuration examples 5.1 Single-switch example The configuration below is the running configuration seen on the console from the show running-config command. Portions of the configuration have been removed that are repetitive or are not applicable to deployment of VxRail. Switch-1 (S4148-ON) Global switch settings hostname Switch-1 snmp-server community name ro ntp server 100.67.10.
VxRail node facing interfaces interface ethernet 1/1/1 description "Appliance 1 Port 1" switchport mode trunk switchport trunk allowed vlan 110,120,130 no ip address no shutdown spanning-tree port type edge ! interface ethernet 1/1/2 description "Appliance 1 Port 2" switchport mode trunk switchport trunk allowed vlan 110,120,130 no ip address no shutdown spanning-tree port type edge ! interface ethernet 1/1/3 description "Appliance 2 Port 1" switchport mode trunk switchport trunk allowed vlan 110,120,130 no
switchport mode trunk switchport trunk allowed vlan 110,120,130 no ip address no shutdown spanning-tree port type edge ! interface ethernet 1/1/8 description "Appliance 4 Port 2" switchport mode trunk switchport trunk allowed vlan 110,120,130 no ip address no shutdown spanning-tree port type edge Port channel 1 is used for 10 GbE upstream connections and requires the correct switchport profile interface port-channel 1 description "10Gb LAG to customer 1" no ip address switchport mode trunk switchport trunk
interface ethernet 1/1/26 description "40Gb Uplink to Customer 2" channel-group 2 mode active no ip address no shutdown Example for 100GbE uplink ports, this requires switch-port profile-1 interface ethernet 1/1/25 description "100Gb Uplink to Customer 2" channel-group 2 mode active no ip address no shutdown ! interface ethernet 1/1/26 description "100Gb Uplink to Customer 2" channel-group 2 mode active no ip address no shutdown 31 VxRail Networking Guide with Dell EMC S4148-ON Switches | version 1.
5.2 Dual-switch example The running configuration example below is seen on the console from the show running-config command. Portions of the configuration have been removed that are repetitive or are not applicable to deployment of VxRail. Switch-1 (S4148-ON) Switch-2 (S4148-ON) Global switch settings hostname Switch-1 snmp-server community name ro ntp server 100.67.10.20 spanning-tree mode rstp ip igmp snooping hostname Switch-2 snmp-server community name ro ntp server 100.67.10.
no shutdown ip igmp snooping querier ip address 192.168.120.251/24 ! vrrp-group 120 priority 254 virtual-address 192.168.120.254 ! interface vlan130 description VM_Network_A no shutdown no shutdown ip igmp snooping querier ip address 192.168.120.252/24 ! vrrp-group 120 priority 254 virtual-address 192.168.120.
no shutdown spanning-tree port type edge ! interface ethernet 1/1/6 description "Appliance 3 Reserved" switchport mode trunk switchport trunk allowed vlan 110,120,130 no ip address no shutdown spanning-tree port type edge ! interface ethernet 1/1/7 description "Appliance 4 Port 1" switchport mode trunk switchport trunk allowed vlan 110,120,130 no ip address no shutdown spanning-tree port type edge ! interface ethernet 1/1/8 description "Appliance 4 Reserved" switchport mode trunk switchport trunk allowed vl
VLT Configuration vlt-port-channel 2 no shutdown vlt-port-channel 2 no shutdown Example for 100GbE uplink ports, this requires switch-port profile-1 interface ethernet 1/1/25 description "100Gb Uplink to Customer 2" channel-group 2 mode active no ip address no shutdown ! interface ethernet 1/1/26 description "100Gb Uplink to Customer 2" channel-group 2 mode active no ip address no shutdown interface ethernet 1/1/25 description "100Gb Uplink to Customer 2" channel-group 2 mode active no ip address no shut
A Basic configuration worksheet Basic configuration worksheet. Workstation Setting Required Number of VxRail Nodes Yes (minimum 3) Number of S4148-ON switches Yes (minimum 1) Number of 10GbE network-facing uplink ports Yes (minimum 1) Number of 100GbE network-facing uplink ports 36 Number of 100GbE inter-switch links (ISLs) (only used in a dual-switch configuration) Yes (recommended 2) Switch 1 Out-of-band management port IP address and mask.
B VLAN worksheet The table below shows entries for up to five VM Network VLANs and example values used in the document. You can use VLAN ID values within the range of 2 to 4094. The table indicate no VLANs are required but best practices are to have a minimum of four VLANs (Management, vMotion, vSAN and Guest Networks).
C Network I/O control predefined shares Network I/O control allows users to allocate network bandwidth to the business-critical application and to resolve situations where several types of traffic compete for shared resources. Network teaming policies for both 10Gb and 1Gb network adapters along with predefined shares.
D Validated Components D.1 VxRail D.2 39 VxRail Product Version VxRail Manager 4.0.200-5234142 VMware vCenter Server Appliance 6.0.0 build-5183552 VMware ESXi 6.0.0 build-5224934 VxRail Manager 4.0.200-5234142 Networking switches Switch Switch S4148-ON – Leaf switches OS10 - 10.4.0E (R2) S6010-ON – Spine switches OS10 - 10.4.0E (R2) Z9100-ON – Spine switches OS10 - 10.4.0E (R2) S3048-ON – Management switch OS10 - 10.4.
E Technical support and resources Contacting Technical Support Dell.com/support - focused on meeting customer needs with proven services and support. Feedback for this deployment guide We encourage readers of this publication to provide feedback on the quality and usefulness of this best practices guide by sending an email to Dell_Networking_Solutions@Dell.
F Accessing the RS-232 console port Note: Before starting, be sure that your PC has a 9-pin serial port and that you have a terminal emulation program already installed and running on the PC. If your PC does not have a DB-9 serial port connection, use a USB-to-Serial adapter. S4148-ON RS-232 console ports 1. Install the RJ-45 connector side of the provided cable into the S4148-ON console port. 2.
