PowerEdge FX2 – FN I/O Module – VLT Deployment Guide Configuring, testing and validating VLT on a pair of FN410S I/O Modules in a PowerEdge FX2 chassis (v2.
Revisions Date Description Authors August 2016 V2.2 DNOS updates DNSE: EB, SJC July 2016 V2.1 EMC Integration changes DNSE: EB September 2015 V2.0 Added DNOS version 9.9 configuration information. Added FN IOM modes section DNSE: NB, KL, MMF, KL July 2015 V1.2 Added additional SMUX configuration. Removed redundant sections. DNSE: CB, KL, MMF, JCS May 2015 v1.1 Fixed Paragraph DNSE: KL December 2014 Initial Document Release DNSE: CB, KL MMF, LC Copyright © 2014-2016 Dell Inc.
Table of contents Revisions.............................................................................................................................................................................2 1 Introduction ...................................................................................................................................................................5 2 The Dell FX2 and FN I/O Modules .............................................................................................
Environment Four: Cisco Nexus 5000 Series vPC configuration .....................................................................76 9.3 Environment Four: Verify VLT connectivity ......................................................................................................83 9.4 Environment Four: Verify VPC connectivity .....................................................................................................84 A References ............................................................
1 Introduction Dell Networking provides customers with the most efficient use of current networking equipment at the lowest cost, while still providing today’s great new technologies focused around the explosive data growth in the industry. The various application demands have driven the need for increased bandwidth, lower latency infrastructures in today’s networks. Dell EMC’s portfolio covers all these key areas to provide the best in service and customer experience.
Dell Data Center Networking Storage Network Dell Servers V RT X WA N Remote/Branch Offi ce Intern et Remo te Data Center s Dell Network Controllers and Security Dell Storage D el l Campus N etworking D el l Data Center N etwork ing Storag e Netw or k Data Center Dell Servers C amp us Pu bli c Cl oud Dell Netw ork Con trol lers and Secur ity Dell Sto rage Data Center Zooming in to focus on the data center and storage networking This deployment guide will provide a step-by-step configuration in
2 The Dell FX2 and FN I/O Modules Introducing the Dell FX Architecture, a great way to bring optimized workloads, maximize efficiency and simplify complexity in today’s data center. Dell PowerEdge FX2 The Dell FX2 enclosure is unique in its modular capabilities that come in a compact 2U size. This innovative design combines the density and efficiency of blade servers with the simplicity and cost benefits of the smaller rack-based server systems.
FX2 compute and networking modules FX2 module descriptions 8 PowerEdge FX2 – FN I/O Module – VLT Deployment Guide | Version 2.
2.1 FN I/O Modules description The PowerEdge FX2 is a chassis capable of housing compute, network and storage modules. The network modules, referred to as one of three FN I/O Modules (FN IOMs), behave as traditional network switches that are housed in the rear of the FX2 chassis. All three FN IOMs offer network services for both the Ethernet and storage fabrics in a data center.
I/O Aggregation Offers Active-Active multi-path for servers Semi-customization FN410S Automated mode No Customization I/O Aggregation FN410T Standalone Mode (Default mode) FN2210S I/O Aggregation Creates a stack up to 6 units Semi-customization I/O Aggregation Offers custom configurations using CLI Moderate customization L2/L3 switching Everything needs to be configured using CLI Full customizations VLT Mode Stacking Mode PMUX Mode Full-switch Mode FN Series operati
3 Initial out-of-box connectivity check and default settings The environments documented within this guide cover configuring Virtual Link Trunking (VLT) in various topologies; however, many times there will be a need to simply bring up a single networking connection in the data center without VLT. This will be connecting a single link from the FN IOM to the ToR switch without configuration on the FN IOM side to show that basic network connections work.
4 VLT and the example environments The rest of this guide illustrates configuring two FN410S FN I/O Modules in a PowerEdge FX2 chassis to use the VLT protocol for redundant, load-balanced connections. The initial configuration from the default Standalone mode to VLT, Full Switch and Programmable Mux mode can be performed either via the Dell Blade I/O Manager GUI or via standard CLI. This configuration works with the FN410S, FN410T and FN2210S (applicable to ports in Ethernet mode only).
4.1 Benefits of VLT In a non-Virtual Link Trunking (VLT) environment utilizing spanning tree protocol, as in Figure 8, redundancy involves idle equipment and increased costs with no added value in the event of a device failure. On the other hand, in a VLT environment (Figure 9), all paths are active, utilizing all available links and switches to their fullest potential. This doubles the throughput, thus increasing performance and adding immediate value.
PowerEdge FX2 Chassis FN410S-A1 NIC Port 1 Te0/1 TEAM Internal Connections NIC Port 2 Te0/1 FN410S-A2 Blade Server 1 Internal interfaces participating in VLT VLTi / ICL peer link Physical Connectivity in a PowerEdge FX2 Chassis Using IOMs PowerEdge FX2 Chassis NIC1 TEAM Virtual Interface NIC2 Logical view of Teamed Virtual Interface connected to two IOMs as one logical switch Logical Connectivity in a PowerEdge FX2 Chassis Using IOMs 14 PowerEdge FX2 – FN I/O Module – VLT Deployment Guide | V
4.2 Typographical conventions and common abbreviations The following conventions are used in the command line examples in this document: Monospace Text CLI (Command Line Interface) examples Italic Monospace Text Variables in CLI examples Bold monospace Text Commands entered at the CLI prompt 5 FN IOM Dell Blade I/O manager and internal port mapping features 5.1 FN IOM Dell Blade I/O manager GUI In previous versions of the Dell Networking Operation System (DNOS) 9.
Module Overview section, but it can also be launched by typing the management IP address of the FN IOM into a browser with a port number of 8081, i.e. 172.25.189.209:8081 The FX2 CMC I/O Module Overview page (Figure 12) is the preferred method to launch the FN IOM Dell Blade I/O Manager GUI. When the Launch I/O Module GUI button is clicked the login page (Figure 13) will allow authorized users access to this feature-rich application.
FX2 FN IOM Dell Blade IO manager login screen When the correct account credentials (default is root/calvin) are entered into the Dell Blade I/O Manager Login page, the initial Dashboard view will be presented (Figure 14). Dell Blade I/O manager dashboard 17 PowerEdge FX2 – FN I/O Module – VLT Deployment Guide | Version 2.
The Dell Blade I/O Manager Dashboard page allows an overall configuration view of the FN IOM, which includes: FN IOM Mode Management IP address Recently logged activity Bandwidth utilization Detailed Port view Detailed mode view (Standalone, VLT, Stacked, Programmable Mux, Full Switch) The menu choices on the left include logging information, Port configuration (both internal and external), Switching, Security, and Settings.
1a 1b 1c 1d 3a 3b 3c 3d FN2210S IOA A1 (Top) Internal Ports 1 2 3 4 5 6 7 8 FN2210S IOA A2 (Bottom) Internal Ports 1 2 3 4 5 6 7 8 Quarter-height slots with dual-port CNAs Half-height slots with dual-port CNAs 19 Slot IOA A1 port numbers (top) IOA A2 port numbers (bottom) 1a 1 1 1b 2 2 1c 3 3 1d 4 4 3a 5 5 3b 6 6 3c 7 7 3d 8 8 PowerEdge FX2 – FN I/O Module – VLT Deployment Guide | Version 2.
Half-height slots – dual-port CNAs In half-height slots with dual-port CNAs installed in the server blades, the servers map to a single port on each of the two IOAs. Figure 16 and Table 3 present the port mapping for half-height slots with dual-port CNAs. Note: Ports 2, 4, 6 and 8 are not used when using half-height blades with dual-port adapters.
Half-height slots - quad-port CNAs In half-height slots with quad-port CNAs installed in the server blades, the slots map to two ports on each IOA. Figure 17 and Table 4 present the port mapping for half-height slots with quad-port CNAs.
6 Environment One: Basic VLT deployment with VLT mode In this section, a basic VLT deployment (Figure 18) is created using the low-touch VLT mode. The purpose of this topology is to present the concept of VLT and how to enable it. VLT Domain Te 0/1 FN IOM Te 0/10 Te 0/11 Te 0/9 Infrastructure VLTi FC630 Server Te 0/9 Te 0/10 Te 0/1 FN IOM Te 0/11 VLT deployment with FN 410S IOM In the diagram above, internal port Te0/1 and external ports Te0/10 and Te0/11 are VLT members on both IOMs.
6.1 Environment One: VLT mode configuration This configuration requires the IOMs to be changed from the default mode of Standalone to VLT mode. Figure 20 presents the steps necessary to first verify the current mode of the IOMs, then to change IOMs to VLT mode. When validating the configurations in this guide the following management IP addresses (Table 5) were used. Management IP address Hostname IP Address Subnet FN410S-A1 172.25.189.27 255.255.0.0 FN410S-A2 172.25.189.28 255.255.0.
FN410S-A1 Verify the initial mode for the IOM show system stack-unit 0 iom-mode FN410S-A2 Verify the initial mode for the IOM show system stack-unit 0 iom-mode If the previous command returns standalone, full switch, or programmable-mux then follow the next set of commands to set the IOM to VLT mode. If the previous command returns standalone, full switch, or programmablemux then follow the next set of commands to set the IOM to VLT mode.
While the command interface management 0/0 ip address 172.25.189.26/16 can set the ip address of the FN IOM module. Figure 21 below shows that from the FX2 CMC > IO Module Overview > Setup, an ip address can be either statically assigned or a DHCP IP address can be chosen to be assigned. FX2 IO module IP address configuration 25 PowerEdge FX2 – FN I/O Module – VLT Deployment Guide | Version 2.
After an IP address has been established via the FX2 CMC, it can be modified from the Dashboard view by clicking on the edit icon choosing DHCP. and then either assigning all the static IP information, or FN IOM Dell Blade I/O Manager IP address configuration 26 PowerEdge FX2 – FN I/O Module – VLT Deployment Guide | Version 2.
The next step in this process is to utilize the FN IOM Dell Blade I/O Manager GUI to change from Standalone Mode to VLT mode. In the Dashboard view click on the edit icon Mode; in the Mode Settings page choose Virtual Link Trunking (VLT). FN IOM Dell Blade I/O Manager standalone to VLT mode 27 PowerEdge FX2 – FN I/O Module – VLT Deployment Guide | Version 2.
Once the Apply button has been clicked, a final confirmation will appear. Click Yes to confirm and the FN IOM switches modes and reboots. FN IOM Dell Blade I/O Manager standalone to VLT mode final confirmation 28 PowerEdge FX2 – FN I/O Module – VLT Deployment Guide | Version 2.
After the FN IOM has rebooted, The Dell Blade I/O Manager Dashboard view shows the Active IOA Mode as VLT. This is equivalent to the command show system stack-unit 0 iom-mode. FN IOM Dell Blade I/O Manager VLT mode confirmation 6.2 Testing VLT functionality To test VLT functionality, a NIC team was created in the PowerEdge FC 630 series blade server. The LACP NIC team was created in Windows 2012 R2 using the native solution built into Windows.
Forcing traffic across VLT link Active Path Disabling links to force traffic across the VLT peer-link (ICL) While traffic is running, individual ports on the IOMs can be administratively disabled in order to channel traffic out certain ports. Figure 26 is an example showing how to force traffic through the VLTi peer link. The red “No” symbols represent the ports which should be administratively disabled. The show vlt statistics command can be run to show the VLTi peer link is carrying data traffic.
In VLT mode, the Dashboard view has a VLT tab (Figure 27) that can show most aspects of the VLT such as: Domain ID Role ICL, Heartbeat, and VLT Peer Status This view is equivalent to the command show vlt brief FN IOM Dell Blade I/O Manager show VLT brief 31 PowerEdge FX2 – FN I/O Module – VLT Deployment Guide | Version 2.
7 Environment Two: Dell Networking switches with mVLT and IOM in Full Switch mode Full Switch mode is a new DNOS 9.10 feature that allows a network administrator to incorporate layer 3 functionality into a FN IOM environment. This new switch mode mirrors the same functionality as Dell EMC’s award-winning MXL switching module, made for the M1000e chassis. The Dell Blade I/O Manager view (Figure 28) below shows the FN IOM in Full Switch mode.
VLT Domain VLT Domain 2 FN410S-A1 S4810-1 VLTi VLTi FC630 Server FN410S-A2 S4810-2 mVLT configuration with a PowerEdge FX2, IOMs and S4810 switches 7.1 Environment Two: IOM Full Switch mode configuration The configuration steps outlined below are repeated on both IOMs. When validating the configurations in this guide the following management IP addresses (Table 6) were used. Management IP address Hostname IP Address Subnet FN410S-A1 172.25.189.27 255.255.0.0 FN410S-A2 172.25.189.28 255.
7.1.1 Configuring the IOM for Full Switch mode In this section, the switch is set to Full Switch mode. In Figure 30, Global Configuration mode is used to set the switch to Full Switch mode. Then the configuration is saved and the switch is reloaded. Once the reload is complete, the IOM mode is verified using the show system stack-unit 0 iom-mode command. Then, the management IP address is assigned to allow VLT heartbeats as well as SSH/telnet access.
7.1.2 Configuring the VLTi peer link In this section, the VLTi peer link is configured (Figure 31). First, the port channel that will be assigned as the VLTi peer link is created and channel members are assigned. Next, in Full Switch mode, a VLT domain is created. The peer link is then assigned and the backup destination is configured to enable heartbeat communications over the management network.
FN410S- A1 Configure the VLTi port channel interface Add a description Add channel members Configure the VLTi port channel interface Add a description Add channel members interface port-channel 127 description VLTI link to VLT- Switch A2 channel-member te0/9 – 10 no switchport no shutdown exit interface port-channel 127 description VLTI link to VLT- Switch A1 channel-member te0/9 – 10 no switchport no shutdown exit Configure the VLT domain Set peer link port channel of Switch A2 Configure the backup de
Next, another port channel is created for the FC630 blade server to communicate. A corresponding VLT peer lag is assigned to the peer switch. Internal port Te0/1 is added to the port channel and LACP is enabled with the mode active command.
FN410S- A1 Configure the peer LAG, and then configure upstream links for the mVLT connection Create the port channel. interface port-channel 101 description “PO to CNA NIC 1” portmode hybrid switchport vlt-peer-lag po101 no shutdown exit Configure the internal port and set the port channel. interface te0/1 no switchport port-channel-protocol lacp port-channel 101 mode active exit no shutdown Configure the port channel for upstream links.
interface range te0/11-12 description “Link to Dell S4810” no switchport port-channel-protocol lacp port-channel 121 mode active exit no shutdown exit interface range te0/11-12 description “Link to Dell S4810” no switchport port-channel-protocol lacp port-channel 121 mode active exit no shutdown exit Connection to server NICs and connections to S4810 switches 7.2 Environment Two: Configuring the Dell Networking S4810 In this section, the S4810 switches are configured (Figure 35).
VLT Domain VLT Domain 2 FN410S-A1 S4810-1 Fo 0/56 Fo 0/60 VLTi (po 55) FC630 Server Fo 0/56 FN410S-A2 Fo 0/60 S4810-2 S4810 member and peer link (ICL) configuration The steps necessary to configure the VLT links are shown in Figure 36.
management IP (management IP of the other switch) vlt domain 2 peer-link port-channel 55 back-up destination 172.25.190.
7.3 Verifying VLT Connectivity VLT connectivity can be checked using the VLT commands listed in Table 8. show commands Command Purpose show vlt brief Displays brief information about the VLT configuration show vlt detail Displays running configuration for VLT members show macaddress-table Displays the MAC address table On each switch participating in VLT, perform a show vlt brief, show vlt detail and show mac-address-table. On the access switch, perform a show mac-address-table.
The following are examples of the output from these show commands. Example of show vlt brief on switch FN410S-A1.
FN410S-A1 FN410S-A1#show mac-address-table Codes: *N - VLT Peer Synced MAC VlanId Mac Address Type State 1 00:01:e8:8b:32:4a Dynamic (N) Active 1 00:01:e8:8b:34:db Dynamic (N) Active 1 00:18:8b:a2:cb:3b Dynamic (N) Active 1 18:a9:9b:d9:28:81 Dynamic Active 1 18:a9:9b:d9:28:84 Dynamic Active 1 18:a9:9b:d9:2f:27 Dynamic (N) Active 1 18:a9:9b:d9:2f:2a Dynamic (N) Active 44 PowerEdge FX2 – FN I/O Module – VLT Deployment Guide | Version 2.
Example of show vlt brief on switch FN410S-A2.
Example of show mac-address-table on switch FN410S-A2. FN410S-A2 FN410S-A2#show mac-address-table Codes: *N - VLT Peer Synced MAC VlanId Mac Address Type State 1 00:01:e8:8b:32:4a Dynamic Active 1 00:01:e8:8b:34:db Dynamic Active 1 00:18:8b:a2:cb:3b Dynamic Active 1 18:a9:9b:d9:28:81 Dynamic (N) Active 1 18:a9:9b:d9:28:84 Dynamic (N) Active 1 18:a9:9b:d9:2f:27 Dynamic Active 1 18:a9:9b:d9:2f:2a Dynamic Active 46 PowerEdge FX2 – FN I/O Module – VLT Deployment Guide | Version 2.
Example of show vlt brief on switch S4810-1.
Example of show mac-address-table on switch S4810-1. S4810-1 S4810-1#show mac-address-table Codes: *N - VLT Peer Synced MAC VlanId Mac Address Type State 1 00:01:e8:8b:36:10 Dynamic Active 1 00:18:8b:a2:cb:3b Dynamic Active 1 00:1e:c9:de:01:70 Dynamic Active 1 00:1e:c9:de:01:78 Dynamic Active 1 18:a9:9b:d9:28:81 Dynamic Active 1 18:a9:9b:d9:2f:27 Dynamic Active 48 PowerEdge FX2 – FN I/O Module – VLT Deployment Guide | Version 2.
Example of show vlt brief on switch S4810-2.
Example of show mac-address-table on switch S4810-2. S4810-2 S4810-2#show mac-address-table Codes: *N - VLT Peer Synced MAC VlanId Mac Address Type State 1 00:01:e8:8b:36:10 Dynamic (N) Active 1 00:18:8b:a2:cb:3b Dynamic (N) Active 1 00:1e:c9:de:01:70 Dynamic (N) Active 1 00:1e:c9:de:01:78 Dynamic (N) Active 1 18:a9:9b:d9:28:81 Dynamic (N) Active 1 18:a9:9b:d9:2f:27 Dynamic (N) Active 50 PowerEdge FX2 – FN I/O Module – VLT Deployment Guide | Version 2.
8 Environment Three: Dell Networking switches with mVLT and FN IOM in programmable MUX mode. This section presents an example of Dell Networking switches working together in an mVLT configuration. Figure 37 illustrates using Dell EMC’s VLT protocol on a pair of IOMs in a switch environment consisting of two Dell Networking S4810 switches.
8.1 Environment Three: FN IOM configuration The configuration steps outlined below are repeated on both IOMs. When validating the configurations in this guide the following management IP addresses (Table 9) were used. Management IP address Hostname IP Address Subnet FN410S-A1 172.25.189.27 255.255.0.0 FN410S-A2 172.25.189.28 255.255.0.0 Note: If the IOMs have been previously configured, existing configuration settings will be retained when changing IOM modes.
8.1.1 Configuring the FN IOM for programmable MUX mode In this section, the switch is set to Programmable MUX (PMUX) mode. In Figure 38, configuration mode is used to set the switch to PMUX IOM mode, then the configuration is saved and the switch is reloaded. Once the reload is complete, the IOM mode is verified using the show system stack-unit iom- mode command. Then, the management IP address is assigned to allow VLT heartbeats as well as SSH/telnet access.
8.1.2 Configuring VLT on the FN IOM In this section, the VLTi peer link is configured (Figure 39). First, the port channel that will be assigned as the VLTi peer link is created. Next, in PMUX mode, a VLT domain is created. The peer link is then assigned and the backup destination is configured to enable the heartbeats to go over the management network. Finally, a unit ID is specified to ensure that the two IOM peer switches are assigned primary or secondary roles.
Figure 40 contains the steps necessary to configure the VLT peer links.
Next, another port channel is created for the FC630 blade server to communicate. A corresponding VLT peer lag is assigned to the peer switch. Internal port te0/1 is added to the port channel and LACP is enabled with the active mode command (Figure 41).
Figure 42 contains the steps necessary to configure the peer LAG and the upstream links for the mVLT connection. FN410S-A1 57 FN410S-A2 Configure the peer LAG, and then configure upstream links for the mVLT connection. Create the port channel Configure the peer LAG, and then configure upstream links for the mVLT connection.
interface range te0/11-12 description “Link to Dell S4810” no switchport port-channel-protocol lacp port-channel 121 mode active exit no shutdown exit interface range te0/11-12 description “Link to Dell S4810” no switchport port-channelprotocol lacp port-channel 121 mode active exit no shutdown exit Connection to Server NICs and Connections to S4810 Switches 8.2 Environment Three: Configuring the Dell Networking S4810 In this section, the S4810 switches are configured (Figure 43).
VLT Domain FN410S-A1 VLT Domain 2 Te 0/1 Te 0/2 S4810-1 Fo 0/56 Fo 0/60 VLTi (po 55) FC630 Server Fo 0/56 FN410S-A2 Te 0/1 Te 0/2 S4810 member and peer link (ICL) configuration 59 PowerEdge FX2 – FN I/O Module – VLT Deployment Guide | Version 2.
The steps necessary to configure the VLT links are shown in Figure 44.
no shutdown exit interface range te0/1-2 description “Portchannel link to IOM” no switchport port-channel-protocol lacp port-channel 121 mode active exit no shutdown no shutdown exit interface range te0/12 description “Portchannel link to IOM” no switchport port-channel-protocol lacp port-channel 121 mode active exit no shutdown S4810 VLT configuration Note: When you create a VLT domain on a switch, the system automatically assigns a unique unit ID (0 or 1) to each peer switch.
show mac-address-table Each switch should be learning the upstream and downstream MAC addresses using all local port channels. The following are examples of the output from these show commands. Example of show vlt brief on switch FN410S-A1.
Example of show vlt detail on switch FN410S-A1. FN410S-A1 FN410S-A1#show vlt detail Local LAG Id Peer LAG Id Local Status Peer Status ---------------101 103 121 ----------- ------------ ----------- 101 104 122 UP UP UP UP UP UP Active VLANs -------1 1 1 Example of show mac-address-table on switch FN410S-A1.
Example of show vlt brief on switch FN410S-A2.
Example of show mac-address-table on switch FN410S-A2. FN410S-A2 FN410S-A2#show mac-address-table Codes: *N - VLT Peer Synced MAC VlanId Mac Address Type State 1 00:01:e8:8b:32:4a Dynamic Active 1 00:01:e8:8b:34:db Dynamic Active 1 00:18:8b:a2:cb:3b Dynamic Active 1 18:a9:9b:d9:28:81 Dynamic (N) Active 1 18:a9:9b:d9:28:84 Dynamic (N) Active 1 18:a9:9b:d9:2f:27 Dynamic Active 1 18:a9:9b:d9:2f:2a Dynamic Active 65 PowerEdge FX2 – FN I/O Module – VLT Deployment Guide | Version 2.
Example of show vlt brief on switch S4810-1.
Example of show mac-address-table on switch S4810-1. S4810-1 S4810-1#show mac-address-table Codes: *N - VLT Peer Synced MAC VlanId Mac Address Type State 1 00:01:e8:8b:36:10 Dynamic Active 1 00:18:8b:a2:cb:3b Dynamic Active 1 00:1e:c9:de:01:70 Dynamic Active 1 00:1e:c9:de:01:78 Dynamic Active 1 18:a9:9b:d9:28:81 Dynamic Active 1 18:a9:9b:d9:2f:27 Dynamic Active 67 PowerEdge FX2 – FN I/O Module – VLT Deployment Guide | Version 2.
Example of show vlt brief on switch S4810-2.
Example of show mac-address-table on switch S4810-2. S4810-2 S4810-2#show mac-address-table Codes: *N - VLT Peer Synced MAC VlanId Mac Address Type State 1 00:01:e8:8b:36:10 Dynamic (N) Active 1 00:18:8b:a2:cb:3b Dynamic (N) Active 1 00:1e:c9:de:01:70 Dynamic (N) Active 1 00:1e:c9:de:01:78 Dynamic (N) Active 1 18:a9:9b:d9:28:81 Dynamic (N) Active 1 18:a9:9b:d9:2f:27 Dynamic (N) Active 69 PowerEdge FX2 – FN I/O Module – VLT Deployment Guide | Version 2.
9 Environment Four: VLT interoperability with Cisco vPC As switching environments become more complex and non-vendor specific, fault tolerant designs have to interoperate with protocols which are similar, but not exactly the same as VLT. In this environment, VLT interacts with Cisco Nexus vPC (virtual PortChannel). The diagram in Figure 45 shows a typical small environment using the VLT protocol on a pair of FN IOMs interoperating with a pair of Cisco Nexus 5000 series switches using the vPC.
9.1 Environment Four: FN IOM configuration The configuration steps outlined below are repeated on both IOMs. When validating the configurations in this guide the following management IP addresses (Table 12) were used. Management IP Addresses 9.1.1 Hostname IP Address Subnet FN410S-A1 172.25.210.55 255.255.0.0 FN410S-A2 172.25.210.56 255.255.0.0 Configuring the FN IOM for programmable MUX mode In this section, the switch is set to Programmable MUX (PMUX) mode.
Note: When an FN IOM is connected to a Cisco Nexus in the default state, a number of DCBX (Data Center Bridging Exchange) error messages are logged and shown on the FN IOM serial console each time a port or port channel is shutdown or brought up. To suppress these messages, either configure DCBX on the FN IOM and Nexus switches (which is out of the scope of this document) or disable DCB on the FN IOM with the command: Dell(conf)#no dcb enable 9.1.
Figure 48 contains the steps necessary to configure the VLTi.
Next, the VLT port channels and member ports are configured (Figure 49). VLT Domain Te 0/1 vPC Domain Te 0/12 FN410S-A1 Te 0/11 Nx5672-1 VLTi FC630 Server Te 0/1 FN410S-A2 Te 0/11 Te 0/12 VLT port channels and member ports 74 PowerEdge FX2 – FN I/O Module – VLT Deployment Guide | Version 2.
Figure 50 shows the necessary steps to configure the VLT peer LAGs.
port-channel 128 mode active exit no shutdown exit port-channel 128 mode active exit no shutdown exit Configuring VLT member ports to downstream servers and uplinked Cisco Nexus 9.2 Environment Four: Cisco Nexus 5000 Series vPC configuration In this section, vPC is configured on the Cisco Nexus 5672 (Figure 51 thru Figure 57). Note: The following vPC configuration is based on recommendations from: Cisco NX-OS Software Virtual PortChannel: Fundamental Concepts 5.0 (Cisco Inc.).
Figure 51 shows that the Dell Networking architecture for mVLT is very similar to this brand-varied Multi-Chassis Link Aggregation Group (MC-LAG) architecture. VLT Domain vPC Domain Nexus 5000 FN IOM Peer Link VLTi FC630 Server FN IOM Nexus 5000 Cisco vPC – Dell Networking VLT – multi-vendor same port channels The configuration steps outlined below are repeated on both Nexus switches. When validating the configurations in this guide the following management IP addresses (Table 13) were used.
In this section, the vPC domain and vPC Peer Link ports are configured. VLT Domain FN410S-A1 vPC Domain Nx5672-1 E 1/19 VLTi FC630 Server E 1/19 FN410S-A2 vPC domain and vPC peer links 78 PowerEdge FX2 – FN I/O Module – VLT Deployment Guide | Version 2.
Figure 53 presents the steps to configure the vPC domain. Nx5672-1 Nx5672-2 Enable the features and management interface for vPC Enable LACP and vPC features Enable the features and management interface for vPC Enable LACP and vPC features configure feature lacp feature vpc configure feature lacp feature vpc Assign an IP address to management interface Assign an IP address to management interface interface mgmt 0 ip address 172.25.109.37/16 no shutdown exit interface mgmt 0 ip address 172.25.109.
In these next steps, the port channel and port channel members for the vPC peer link are configured. Interface ports Ethernet 1/19 and 1/20 are added to port channel 55 and set to use LACP unconditionally using the mode active option.
Once the vPC domain is created, the vPC Peer-Link is formed using port channel 55. The port channel connects both vPC peer switches and should be configured to carry all VLANs. Finally, the port channel is enabled and the vpc peer link command is issued to specify the role of this port channel in the vPC domain.
Nx5672-1 Configure the port channel and port channel member links for downstream connectivity to the FN IOMs Configure the port channel and port channel member links for downstream connectivity to the FN IOMs interface port-channel 100 description “vPC to FN410S” switchport mode trunk switchport mode trunk allowed vlan all vpc 100 no shutdown exit interface port-channel 100 description “vPC to FN410S” switchport mode trunk switchport mode trunk allowed vlan all vpc 100 no shutdown exit Assign interface
Port Channel 200 is created and connectivity to the external network is configured.
9.4 Environment Four: Verify VPC connectivity Use the following commands (Table 15) to verify that the vPC configuration is valid.
Nx5672-1 5548-1# show vpc brief Legend: (*) - local vPC is down, forwarding via vPC peer-link vPC domain id : 1 Peer status : peer adjacency formed ok vPC keep-alive status : peer is alive Configuration consistency status : success Per-vlan consistency status : success Type-2 consistency status : success vPC role : primary Number of vPCs configured : 2 Peer Gateway : Disabled Dual-active excluded VLANs : - Graceful Consistency Check : Enabled Auto-recovery status : Enabled (timeout =
100 Po100 up success success 1 200 Po200 up success success 1 The output of the show interface port-channel command is also updated with the vPC status and vPC number for port channels configured for vPC. Nx5672-1 Nx5672-1# show interface port-channel 100 port-channel100 is up vPC Status: Up, vPC number: 100 Hardware: Port-Channel, address: 002a.6adf.a0e8 (bia 002a.6adf.
input rate 1.60 Gbps, 143.70 Kpps; output rate 5.37 Gbps, 479.
The show running config vpc command can be used to filter the running configuration to show only vPC related settings. Nx5672-1 x5672-1# show running-config vpc !Command: show running-config vpc !Time: Sun Feb 17 21:06:03 2002 version 7.0(1)N1(1) feature vpc vpc domain 1 role priority 1 peer-keepalive destination 172.25.109.
Further information about the vPC peer switch keepalive can be extracted using show vpc peerkeepalive. The current status, IP address and the VRF associated with the keepalive are shown in the following output.
A References Alexis Dacquay. (2012). Virtual Link Trunking Overview. Retrieved from www.dell.com: http://i.dell.com/sites/content/business/largebusiness/merchandizing/en/Documents/Dell_Force10_S4810_VLT_Technical_Guide.pdf?forcedownlo ad=truehttp://www.dell.com/learn/us/en/04/shared-content~datasheets~en/documents~dell_force10_s4810_vlt_technical_guide.pdf Cisco Inc. (n.d.). Cisco NX-OS Software Virtual PortChannel: Fundamental Concepts 5.0. Retrieved from Cisco.com: http://www.cisco.
C Terminology CLI (Command Line Interface): Text-based telnet, secure shell (SSH), or serial type interface that is used for issuing commands to a device. DHCP (Dynamic Host Configuration Protocol): A network management protocol used to dynamically assign network settings to devices connected to a network. Fault Tolerance: Enables a network to continue operating properly in the event of a failure of some of its components. ICL (Inter-Chassis Links): Provides physical connectivity between two switches.
ToR (Top of Rack): A ToR switch is a switch that is positioned at the top of a server rack in a data center. SSH (Secure Shell): Permits a user to log in to another computer over a network, execute commands on a remote machine and move files from one machine to another. Secure Shell provides strong authentication and secure communications methods over insecure channels. Telnet: Terminal Emulation Protocol enables system users to log in and use resources on remote networks.
VLT Mode: An operational mode for an IOM that automatically configures VLT using external facing port 9 as the VLTi, leaving the three remaining ports available for upstream connectivity. VLT Peer Keepalive Link: A VLT peer keepalive link monitors the vitality of a VLT peer switch by sending periodic keepalive messages between VLT peer devices. No data or synchronization traffic is sent over this link, only keepalive messages. VLT Peer-Link: See VLTi.
D Reset FN IOM to Default Factory Configuration The FN IOM can be checked for factory default mode in two ways. With the new firmware revision 9.10, the Dell Blade I/O manager can show the currently configured mode from the Dashboard view. Figure 58 and Figure 59 show how to launch the Dell Blade I/O Manager and the Dashboard view with the Active IOA Mode showing Standalone.
Ensure the FN IOM is in Standalone Mode using the Dell Blade I/O Manager GUI Dashboard view (Figure 59) below. Dell Blade I/O Manager standalone mode dashboard view The CLI command show system stack-unit 0 iom-mode (Figure 60) can also be used to confirm this default state. Dell#show system stack-unit 0 iom-mode Unit Boot-Mode Next-Boot -----------------------------------------------0 standalone FN IOM Mode show command If it is not in Standalone Mode, reset the FN IOM to factory defaults.
The CLI command below will restore the FN IOM to factory defaults, put it into Standalone mode and reboot it. Dell#restore factory-defaults stack-unit 0 clear-all Proceed with factory settings? Confirm [yes/no]:y FN IOM restore factory defaults To reset the FN IOM to factory default Standalone Mode using the Blade I/O Manager GUI page below, from the initial Dashboard view (Figure 62) click on Settings, then click on the edit icon to the right of Restore IOA.
Confirm restore When the FN IOM Dell Blade I/O Manager is restarted after the factory reset process, an Initial Setup Wizard (Figure 65) automatically launches. This wizard allows a network administrator to set fundamental parameters such as new User Name and Password credentials other than the default root/calvin, as well as Management IP Address, Network Time Protocol, etc.
After answering all the setup questions, click Apply (Figure 66) FN IOM Initial Setup Wizard summary confirmation 98 PowerEdge FX2 – FN I/O Module – VLT Deployment Guide | Version 2.
After clicking the apply button, the FN IOM (Figure 67) requests a final confirmation to reboot and write the settings to the startup-config file. FN IOM Initial Setup Wizard final confirmation 99 PowerEdge FX2 – FN I/O Module – VLT Deployment Guide | Version 2.
E FN IOM initial out-of-box configuration and default settings This section applies to the Dell PowerEdge FN I/O Module (FN IOM) in Standalone mode. Modules in this section include the FN410S, FN410T, and FN2210S (applicable to ports in Ethernet mode only). This section provides configuration examples for common upstream switches to establish a functional uplink and bring up network ports on PowerEdge FC-Series servers (FC830, FC630, FC430, FC620 and FC420) in three easy steps: 1.
Setting Auto LACP LAG Default Value Enabled on all ports (Uplink created by default based on LACP PDU) All server and uplink ports are in all VLANs VLANS Standalone Mode default settings By default, network ports on the PowerEdge FC-Series servers installed in the FX2 chassis remain down until the uplink port channel is operational on the FN IOM. This is due to a feature called Uplink Failure Detection whereby the FN IOM disables the downstream links when upstream connectivity fails.
To accomplish this, start by connecting any combination of the FN IOM’s external Ethernet ports (ports Te 0/9-12) to the upstream switch. The port channel may have from one to four links. Configure a port channel on your upstream switch, as shown in the following examples. Upstream Switch Configuration Examples Note the following when configuring the upstream switch: No configuration is required on the FN IOM. All configuration is done on the upstream switch.
Cisco NX-OS Release 7 Nx5548(config)# interface Ethernet 1/1-4 Nx5548(config-if-range)# switchport mode trunk Nx5548(config-if-range)# channel-group 10 mode active Nx5548(config-if-range)# no shut Nx5548(config-if-range)# interface port-channel 10 Nx5548(config-if)# switchport mode trunk Nx5548(config-if)# no shut Cisco Nexus 5548 port channel configuration Brocade Network OS 4.
E.2 Verify the configuration When the port channel of the upstream switch has been properly configured, the port channel on the FN IOM will come up.
The Blade I/O Manager has a View LAG Membership feature (Figure 76) that is equivalent to the CLI show interfaces port-channel 128 verification command (Figure 74). Access this feature by going to Switching Layer-2 > View LAG Membership. View healthy LAGs from the main Switching Layer-2 page. Dell Blade I/O Manager LAG membership 105 PowerEdge FX2 – FN I/O Module – VLT Deployment Guide | Version 2.
F Support and feedback Contacting Technical Support Support Contact Information Web: http://Support.Dell.com/ Telephone: USA: 1-800-945-3355 Feedback for this document We encourage readers of this publication to provide feedback on the quality and usefulness of this deployment guide by sending an email to Dell_Networking_Solutions@Dell.com 106 PowerEdge FX2 – FN I/O Module – VLT Deployment Guide | Version 2.
