VMware Integration for VxRail Fabric Automation SmartFabric User Guide, Release 1.
Notes, cautions, and warnings NOTE: A NOTE indicates important information that helps you make better use of your product. CAUTION: A CAUTION indicates either potential damage to hardware or loss of data and tells you how to avoid the problem. WARNING: A WARNING indicates a potential for property damage, personal injury, or death. © 2018 - 2019 Dell Inc. or its subsidiaries. All rights reserved. Dell, EMC, and other trademarks are trademarks of Dell Inc. or its subsidiaries.
Contents 1 SmartFabric vCenter.................................................................................................................... 5 2 Getting started............................................................................................................................ 7 3 ON-Series switch installation........................................................................................................ 8 Download OS10 image and license..................................................
Index.......................................................................................................................................................................
1 SmartFabric vCenter The Dell EMC Open Manage Network Integration (OMNI) is an efficient REST API-based plug-in, which is integrated with VMware's vCenter. The plugin enables vCenter to easily deploy and manage a large virtual network of VMs and physical underlay on a VxRail HCI stack. Using OMNI, day two operations and management of the hyperconverged fabric is a breeze. OMNI provides visibility into the virtual environment and dynamic infrastructure configuration deployment.
VxRail domain deployed with SmartFabric With SmartFabric, the automation boundary is now extended to include Dell EMC top of rack (ToR) switches. The HCI administrator can deploy the VxRail domain, however, the ToR switches are automatically configured to provide the required networking services for the VxRail domain.
2 Getting started This information describes the component requirements. Dell EMC VxRail • Dell EMC VxRail release 4.7.410 or 4.7.211 with VMware vCenter 6.7 Update 2 (U2) or later. Dell EMC ToR switches • Dell EMC S4112F-ON, S4112T-ON, S4128F-ON, S4128T-ON, S4148F-ON, S4148T-ON, S5212F-ON, S5224F-ON, S5248F-ON, and S5296F-ON running Dell EMC SmartFabric OS10, release 10.4.3.4 or later. SmartFabric vCenter • • • Dell EMC SmartFabric for VMware vCenter VMware vCenter ESXi 6.
3 ON-Series switch installation This information describes how to install a Dell EMC PowerSwitch ON-Series switch in your network. If your switch came preinstalled with Dell EMC SmartFabric OS10 release 10.4.3.4 or later, see Log into OS10. NOTE: For detailed hardware installation steps, see the product-specific Installation Guide at www.dell.com/support/.
Download OS10 for a switch without OS installed You can purchase the operating system10 image with an after point-of-sale (APoperating system) order for a Dell EMC ONIE-enabled device that does not have a default operating system or license installed. After the order is complete, you receive an email message notification with a software entitlement ID, order number, and link to the DDL.
• Connect the Management port to the network if you prefer downloading an image over a network. To locate the Console port and the Management port, see the Getting Started Guide shipped with your device or the platform-specific Installation Guide at www.dell.com/support. Install OS10 During bootup, press ESC to display the ONIE boot menu. An ONIE-enabled device boots with preloaded diagnostics and ONIE software.
The OS10 installer image creates several partitions, including OS10-A (active and default) and OS10-B (standby). After installation completes, the switch automatically reboots and loads OS10. Install manually using USB drive You can manually install the OS10 software image using a USB device. Verify that the USB device supports a FAT or EXT2 file system.
Install OS10 license If OS10 is factory-loaded on your switch, you do not need to install an OS10 license. If you download OS10 on a trial basis, OS10 comes with a 120-day trial license. To continue with uninterrupted use, purchase and install a perpetual license to avoid the OS10 device rebooting every 72 hours. After you install OS10 and log in, install the license to run OS10 beyond the trial license period. For more information, see Download OS10 image and license.
File Size: Transfer Rate: 3795 bytes 8 kbps 3. Verify that the license is present in the home directory. OS10# dir home Directory contents for folder: home Date (modified) Size (bytes) Name --------------------- ------------ ----------------------2019-02-15T00:47:25Z 3795 7B900Q2-NOSEnterprise-License.XML 4. Run the license install command and specify the path to the home directory location where the license was downloaded in step 1.
4 Enable SmartFabric for VxRail This information explains how to enable SmartFabric for VxRail. 1. Log into the Linux shell. S4128F-1# system "sudo -i" We trust you have received the usual lecture from the local System Administrator. It usually boils down to these three things: #1) Respect the privacy of others. #2) Think before you type. #3) With great power comes great responsibility. [sudo] password for admin: root@S4128F-1:~# 2.
-m MANAGEMENT_VLAN, --management_vlan MANAGEMENT_VLAN Public Management VLAN (Untagged VLAN). By default management VLAN is 1. -j JUMPBOX_INTERFACE, --jumpbox_interface JUMPBOX_INTERFACE The jumpbox interface. -a ADDITIONAL_BREAKOUT, --additional_breakout ADDITIONAL_BREAKOUT The additional breakouts of phy-ports or port-groups. The format is -a '4X10GE{phy-port1/1/2,phyport1/1/3};1X40GE{phy-port1/1/4,phy-port1/1/5}'.
-d DOMAIN, --domain DOMAIN Domain-Id for the SmartFabric Services cluster formation. Accepted Values [1..254]. -u UPLINK, --uplink UPLINK Uplink Interfaces separated by a comma (,). If empty, it takes the default value for the platform. If there is no uplink, provide 'None/Null' -i ICL, --icl ICL VLT Inter Chassis Link(ICL) interfaces seperated by a comma (,). If empty, it takes the default value for the platform.
-j JUMPBOX_INTERFACE, --jumpbox_interface JUMPBOX_INTERFACE The jumpbox interface. -a ADDITIONAL_BREAKOUT, --additional_breakout ADDITIONAL_BREAKOUT The additional breakouts of phy-ports or port-groups. The format is -a '4X10GE{phy-port1/1/2,phyport1/1/3};1X40GE{phy-port1/1/4,phy-port1/1/5}'.
Server-facing port breakout # sfs_enable_vxrail_personality.py -a '4X10GE{port-group1/1/25}' 4. Verify the switch-port-profile. # show switch-port-profile | Node/Unit | Current | Next-boot | Default | |-----------+-----------+-----------+-----------| | 1/1 | profile-1 | profile-1 | profile-1 | S5200-ON Series breakout types S5200-ON Series breakout configuration is done using port-groups, unlike physical port configuration on the S4100-ON Series.
• Port-channel is static (without the -l option) S5212F-ON — Breakout interface eth1/1/1 (port-group1) for VxRail 10GE connection # sfs_enable_vxrail_personality.py -b '4x25GE' -u 1/1/14:2 -a '4X10GEFixedFormFactor' {portgroup1/1/1} -j ethernet1/1/1:1 • • Uplink is eth1/1/14:2, breaking out to 25GE Jump box is eth1/1/1 at 10GE S4100-ON Series — Breakout interface eth1/1/25 and eth 1/1/26for VxRail 10GE connection # sfs_enable_vxrail_personality.
Only the configuration changes that you make using the OS10 administrative commands — supported in both modes — are saved and retained. All other CLI configuration commands are disabled. • • • • • • • • • clock hostname ip name-server logging management route ntp snmp-server spanning-tree username NOTE: Although front-panel port interfaces operate in Layer 2 (L2) mode by default, L2 bridging is disabled.
reconfigured for manual deployment. Are you sure you want to continue with the changes by rebooting the OS10 node?.(y/n) 3. Enter Y to disable SmartFabric Services for VxRail configuration and reboot the switch automatically. Supported topologies This information captures only certified, tested, and supported topologies as it pertains to Dell EMC SmartFabric and VxRail HCI deployments.
• • • • • • 22 Redundancy at the customer device, allowing for the creation of redundant uplinks from the Dell EMC ToR to the customer device, and the opposite way VLT on the Dell EMC ToR; vPC, MLAG, VLT, or similar capability at the customer device Dual uplinks from each Dell EMC ToR, and the number of uplinks can be more than two if wanted. VxRail node connections use the rNDC connections only. Two separate "Fabrics" have been created.
5 OpenManage Network Integration OpenManage Network Integration (OMNI) is a component of SmartFabric Services that integrates with VMware vCenter for zero-touch fabric automation of the physical network infrastructure corresponding to the virtual network operations within vCenter. OMNI also serves as a front-end management application SmartFabric Service instances, enabling administrators to manage and operate one or more network fabrics that are deployed with SmartFabric Services.
NOTE: Even when OMNI is deployed in-band, it is recommended to set up connectivity with the out-of-band Management network of the switches in the network fabric to separate management traffic with user data traffic, and also to enable faster image downloads to the switches.
4. Select a name and folder for the VM, then click Next. 5. Select the destination compute resource, then click Next.
6. Review and verify the template details, then click Next. 7. Accept the end-user license agreement (EULA), then click Next.
8. Select the VSAN datastore for the configuration and disk files, then click Next. 9. Select a destination network for each network source, then click Next. The VxRail Management Network must be assigned to the VxRail internal Management network. The default VLAN ID for this network is 3939. The vCenter Server network must be connected to the port group where the vCenter is reachable for plug-in deployment of the VM.
10. Click Finish to start creation of the VM, then power on the VM. Log into VM console The SmartFabric VM console allows you to configure the software to best suit your needs. You can configure SmartFabric through the VM console after you complete the authentication step. The VM console automatically closes after 10 minutes (default), and can be customized to meet your needs.
• • • • • • • • • Self-signed SSH and TLS certificate regeneration Third-party SSH and TLS certificate installation SmartFabric password for the fabric Reboot and logout Export support bundle Display connectivity status with vCenter Server Display connectivity status with SmartFabric Supports up to 10 vCenter Server instances Supports up to 16 SmartFabric domains This information describes how to log into the VM console. 1. After the new VM powered on, open the VM console. 2.
3. Select Wired Connection 1, then click Edit Connection. 4. Verify Ethernet (ens160) is connected to the vCenter reachable network. Change the Profile name to vcenter server network, then click to the right of IPv4 configuration.
5. Select Manual/DHCP, set the DNS servers, set the Search Domains, then click . 6. Click to the right of IPv6 configuration, disable IPv6, click , then click OK.
7. Select Wired Connection 2, then click Edit Connection.
8. Verify Ethernet (ens192) is connected to the IPv6 VxRail management network. This is known as the VxRail internal Management network, and IPv6 is required for this network.
9. Change the name to vxrail management network, disable the IPv4 address on this interface, select IPv6 to be the link-local address, then click OK.
10. Your connection list is now updated with vCenter and VxRail profile names. Click Back to activate the connection profiles. Activate connection profiles NOTE: To populate DNS entries automatically, each profile must be deactivated, then activated. 1. Select Activate a Connection, then click OK. 2. Select the vcenter server network profile, then click Deactivate. Repeat for the vxrail management network.
3. Select the vcenter server network profile, then click Activate. Repeat for the vxrail management network.
4. Click Back, then OK to exit the network management UI. 5. Configure the valid NTP Server IP address or hostname, then click Enter. Go to the next section for the SSL certificate installation. NOTE: If the NTP Server is not configured, the SmartFabric appliance VM synchronizes with the ESXi server time zone. vCenter registration This information describes how to install an optional SSL certificate, register the vCenter plug-in, and register SmartFabric.
3. Enter the vCenter Server username and vCenter Server password. Repeat this step to register each vCenter instance (up to 10). 4. Enter the REST user password used during the VxRail cluster configuration through VxRail Manager. NOTE: The REST user password is only requested once. Use caution to avoid entering an incorrect password. The registration response displays before going to the full SmartFabric menu.
Map multiple fabric instances to a single OMNI plug-in This information explains how to map fabric instances to a single vCenter instance. If you do not want to create individual OMNI to vCenter mappings, you have the option of mapping multiple fabric instances to a single OMNI mapped to single or primary vCenter instance. Prepare the Management IP address 1. Configure a Management IP address on all Dell EMC ToRs enabled with SmartFabric Services. 2.
When you select SmartFabric, the home page displays information about the SmartFabric domains being managed. This page also allows you to update extensions if available. Information includes: • • • • Switch details that are part of the SmartFabric domain Operating system version Lifecycle status List of networks created Monitoring This information describes supported deployment, virtual standard switch (VSS), distributed virtual switch (DVS), and usage.
NOTE: Deleting port groups or the DVS itself clears all unused VLANs from the SmartFabric. Distributed virtual switch (DVS) provides an option to change the VLAN of uplink port-groups. SmartFabric ignores port-group configuration if the VLAN type port-group is set to VLAN trunking or private VLAN. Dell EMC recommends keeping the DVS uplink in Trunking mode and configures the virtual port groups with VLANs for each network. SmartFabric configures the respective VLANs on the ToRs and SmartFabric Uplinks.
6 System management This information explains common SmartFabric system management operations. • Appliance reboot Configurations are persistent. Any services running are enabled at startup and continue to run. If the services stop, the services must be brought up again to monitor the domain. • Go to SmartFabric Console Login and select Option 7. Reboot Appliance. Recommendation is to enable auto start for the VM after power on of the host or domain after the first setup.
Host network inventory This information describes how to view information about physical Dell EMC PowerSwitch infrastructure running SmartFabric OS10. Host network inventory page Select a host in vCenter, select the Monitor tab, then select OMNI vCenter in the monitor sidebar. Refresh button Click Refresh to update the host network inventory data and display updated contents. Physical adapter table Select a switch to view detailed information.
• Port groups tab — includes the name of port groups, and VLAN IDs for each port group • VMs tab — includes the name of VMs of that host that is connected to a single virtual switch 44 System management
Physical switch Displays information about the onboard interface. This information displays only when there is a physical connection between the VxRail domains and OMNI.
• • • Network Name — name of the VLAN network Network ID — unique identifier of the fabric network VLAN — tagged VLAN of the switch port Fabric summary This information describes the selected Fabric Summary. Select the fabric from the left-pane to view details on the fabric. Click Refresh at any time to update details. User uplinks This information describes how to create, edit, or delete fabric uplinks.
NOTE: If you delete an uplink, any unused networks and ports will be available for future use. 1. Select SmartFabric vCenter, then select SmartFabric. 2. Select the specific SmartFabric instance, select Configure, then select User Uplinks. Create user uplinks You can create an uplink by selecting the fabric with a unique user-given name, then select the interfaces and networks through the wizard interface to create a user uplink. 1. Select the Uplink Name, select the fabric, then click Next. 2.
3. The interfaces are now selected for the fabric uplink; click Next. 4. Select networks to add to the fabric uplink from the top list, then click Add. 5. The networks are now selected for the fabric uplink; click Finish. Edit user uplink NOTE: You can only edit user-created uplinks. Default system uplinks cannot be edited. 1. Select the uplink, then click Edit.
2. Add or delete the interfaces to change, then click Next. The new interface selection displays at the bottom. 3. Add or delete the networks to change; the new network selection displays at the bottom.
4. Click Finish. Delete user uplink NOTE: You can only delete user-created uplinks. Default system uplinks cannot be deleted. 1. Select the uplink to delete, then click Delete. 2. Click Delete. Port breakout This information describes how to display and configure breakout-capable ports. You can choose any breakout mode available on the configuration to move back and forth. 1. Select SmartFabric vCenter, then select SmartFabric. 2.
4. Select breakout mode, then click Submit. Jump port This information describes jump port operations. Only one port per switch can be configured as a jump port. You can select any available port that is not part of an uplink or intercluster link, or connected to an ESXi server which is part of SmartFabric deployment. If you would like to change an existing jump port, you must unconfigure it, then add the new jump port.
1. Select the switch that is configured with a jump port, then click Delete to delete the jump port. 2. Follow the steps in Add new jump port. Password change This information describes how to change the password for the SmartFabric domain. NOTE: If you change the password, the SmartFabric password must be updated on theVxRail Manager as well. 1. Select SmartFabric vCenter, then select SmartFabric. 2. Select the specific SmartFabric instance, select Configure, then select Password Change. 3.
4. Click Close. You can repeat these steps to upload more images. 5. Click Yes to confirm the upgrade. Delete image 1. Select the image to delete, then click Delete. 2. 3. Click Yes to delete the image, or No to cancel. Upgrade fabric 1. Select the image to upgrade, then click Upgrade.
2. Click Yes to confirm the upgrade, or No to cancel the upgrade. The successful image upgrade message displays. NOTE: One switch is upgraded at a time so that there is no loss of connectivity for the VxRail cluster. The vCenter Task Console displays including the status of the upgrade task. The percentage that is completed on vCenter shows 50% complete when the image has finished uploading the SmartFabric image, and 100% when SmartFabric completes the upgrade.
3. Click Add to create a Server Interfaces Profile. 4. Enter the name of the new profile, then click Next. 5. Click Select Networks, select the networks to assign to the profile, then click Add. 6. (Optional) Click Create Network, enter the name of a new network to assign to the profile, then click Create.
7. Click Finish. Edit profile 1. Select the Server Interface Profile from the display, then click Edit.
2. (Optional) Select the network to edit from the Server Interface Profile, then click Add. 3. Click Remove Network, then click Finish. Delete single profile 1. Select the Server Interface Profile from the display, then click Delete. 2. Click Delete to confirm removal of the Server Interface Profile. Delete multiple profiles 1. Select the Server Interface Profiles from the display, then click Delete.
2. Click Delete to confirm removal of the Server Interfaces Profile. Fabric topology This information explains how to display the fabric topology and switch information. Switch information includes destination interface, destination switch, interface ID, interface name, switch ID, and type. 1. Select SmartFabric vCenter, then select SmartFabric 2. Select the specific SmartFabric Instance, then click Topology. 3. Mouse over a fabric to display switch details. 4.
For example, if the uplink port is 1/1/4 and the interlink ports are 1/1/6,1/1/8, no VLAN tagging, LACP auto, management VLAN 1 as default. # sfs_enable_vxrail_personality.py -i 1/1/6,1/1/8 -u 1/1/4 -l 5. Reload the switch. 6. Connect the new switch to the existing switch running SmartFabric currently. Make all connections between the switches, uplinks, and VxRail nodes. 7.
7 Lifecycle management This information explains common lifecycle operations including upgrading of the SmartFabric solution including VxRail nodes, SmartFabric OS10, SmartFabric vCenter appliance, and switch replacement. These are the required steps to upgrade the SmartFabric solution: 1. Upgrade all VxRail nodes (see Upgrade VxRail nodes). 2. Upgrade OpenManage Network Integration (see Upgrade image). 3. Upgrade SmartFabric OS10 (see Upgrade OS10).
4. (Optional) Verify that the next boot partition has changed to standby in EXEC mode. show boot detail 5. Reload the new software image in EXEC mode. reload 6. Install the OS10 license. See Install OS10 license for complete instructions. Verify OS10 version OS10# show version Dell EMC SmartFabric OS10 Copyright (c) 1999-2019 by Dell Inc. All Rights Reserved. OS Version: 10.4.3.0 Build Version: 10.4.3.
After the upgrade completes, the session logs out. 4. Press Enter to continue. 5. Select 5. OMNI Web Client Plugin Menu to register the plug-in, then select Register/Update OMNI vSphere Client Plugin with vCenter. 6. Enter the FQDN to use for registration, then repeat the steps to update the plug-in with the vCenter Server. NOTE: Before you can use the plug-in, you must set up a Dell EMC SmartFabric vCenter Appliance (see SmartFabric network integration).
1. Log into the SmartFabric console. 2. Select option 4. SmartFabric vSphere Client Plugin Menu. 3. Select option 2. Unregister SmartFabric vSphere Client Plugin with vCenter, then press Enter to continue. 4. Provide the vCenter credentials to unregister the plug-in from the vCenter Server. NOTE: Once the unregistration completes, you must log out of the vCenter Server to delete the SmartFabric menu. Upgrade VxRail nodes This information describes upgrading of all VxRail nodes using VxRail Manager.
8 Troubleshooting This information troubleshoots the SmartFabric vCenter appliance connectivity, SmartFabric errors, and UI population errors. SmartFabric appliance connectivity Check the IP and DNS settings and connection status. 1. Select 2. Interface Configuration Menu. 2. Select 1. Show Interfaces (q to close view). 3. Select 2. Show Connection Status. 4. Issue a curl https request from the SmartFabric vCenter appliance console to the plug-in appliance IP configured on ens160.
All services must be active. If they are not all active, restart the service by selecting 4. Restart SmartFabric Management Service. Confirm that the error is resolved. 2. Confirm the password that is supplied during registration is correct. If not correct, reregister the plug-in to update the SmartFabric password on the appliance. Configure NTP server NOTE: If the NTP server is not configured, the SmartFabric vCenter appliance VM will not synchronize with the data center. 1. Select 2.
3. Enter the Validate NTP Server IP address/host name. UI is not populated NOTE: If any IP or SSL certificate changes on the SmartFabric VM, click Update Extension on the SmartFabric vCenter dashboard. Check the service status on the plug-in VM. 1. Select 4. Restart SmartFabric Management Service. 2. Select 2. Status SmartFabric Management Service to list the registered vCenter and SmartFabric the VM is managing. Confirm that all services are active. Create support bundle 1. Select 4.
NOTE: By default, the SmartFabric appliance comes with a log-level of ERROR. The appliance log can be swapped between ERROR to DEBUG, or the opposite way. 3. Stop if the log level is already on the wanted log level. Reset SmartFabric VM password 1. Reboot the VM from vCenter, then select Advanced Options for Debian GNU/Linux. 2. Press e to highlight Debian GNU/Linux, with Linux 4.9.0-7amd64. 3. Use the arrow keys to go to the line starting with linux and ending with ro quiet. 4.
5. Press Ctrl-X to boot into the shell with root access. 6. Remount the / directory. # mount / -re -o remount 7. Change the password for admin using passwd admin, then enter and confirm the new password. 8. Reset the VM from vCenter. You should be able to login through the new password for the SmartFabric VM.
Index A OS10 SmartFabric mode 19 add new jump port 51 P C port breakout 50 change password 52 configure extended options 15 create virtual machine 24 S D dashboard 39 disable SmartFabric 20 download OS10 image and license 8 E server interface profile 54 SmartFabric vCenter 5 SmartFabric vCenter setup 29 supported topologies 21 switch replacement 58 system management 42 T troubleshooting 64 enable SmartFabric for VxRail 14 F U fabric summary 46 fabric topology 58 upgrade OS10 60 upgrade Smart
