Broadcom® NetXtreme® BCM57XX User Guide Last revised: April 2019 2CS57XX-CDUM515
NetXtreme User Guide Broadcom, the pulse logo, Connecting everything, Avago, Avago Technologies, and the A logo are among the trademarks of Broadcom and/or its affiliates in the United States, certain other countries and/or the EU. Copyright © 2019 by Broadcom. All Rights Reserved. The term “Broadcom” refers to Broadcom Limited and/or its subsidiaries. For more information, please visit www.broadcom.com.
NetXtreme User Guide Table of Contents Section 1: Installing the Hardware .................................................................................. 10 Safety Precautions ..................................................................................................................................... 10 Preinstallation Checklist ............................................................................................................................ 11 Installing the Adapter ..................
NetXtreme User Guide Step 2: Grant DCOM Remote Launch and Activate Permission.................................................... 30 Special Configuration for WMI on Different Systems..................................................................... 32 Section 4: Linux Driver and Management Application Installation .............................. 33 Packaging ..................................................................................................................................................
NetXtreme User Guide Section 6: Using Broadcom Advanced Control Suite 4................................................. 50 Broadcom Advanced Control Suite Overview ......................................................................................... 50 Starting Broadcom Advanced Control Suite ........................................................................................... 51 BACS Interface.......................................................................................................
NetXtreme User Guide Configuring LiveLink for a Smart Load Balancing and Failover and SLB (Auto-Fallback Disable) Team...................................................................................................................................... 89 Saving and Restoring a Configuration .......................................................................................... 90 Viewing BASP Statistics ....................................................................................................
NetXtreme User Guide Architecture .........................................................................................................................................112 Outbound Traffic Flow ..................................................................................................................113 Inbound Traffic Flow (SLB Only) ..................................................................................................113 Protocol Support..............................................
NetXtreme User Guide Section 9: iSCSI Protocol and Broadcom Boot Agent Software ................................. 138 iSCSI Boot................................................................................................................................................. 138 Supported Operating Systems for iSCSI Boot .................................................................................... 138 iSCSI Boot Setup ....................................................................................
NetXtreme User Guide Section 12: Troubleshooting ......................................................................................... 160 Hardware Diagnostics ..............................................................................................................................160 BACS Diagnostic Tests Failures .........................................................................................................160 BACS Network Test Failures...................................................
NetXtreme User Guide Installing the Hardware Section 1: Installing the Hardware • Safety Precautions • Preinstallation Checklist • Installing the Adapter • Connecting the Network Cables Note: This section applies only to add-in NIC models of Broadcom NetXtreme Gigabit Ethernet adapters. Safety Precautions Caution! The adapter is being installed in a system that operates with voltages that can be lethal.
NetXtreme User Guide Installing the Hardware Preinstallation Checklist 1. Verify that your server is using the latest BIOS. 2. If your system is booted to an operating system, gracefully power down the OS. 3. When system shutdown is complete, turn off the power and unplug the power cord. 4. Holding the adapter card by the edges, remove it from its shipping package and place it on an antistatic surface. 5. Check the adapter for visible signs of damage, particularly on the card edge connector.
NetXtreme User Guide Installing the Hardware Connecting the Network Cables Copper The Broadcom NetXtreme Gigabit Ethernet adapter has one or more RJ-45 connectors used for attaching the system to an Ethernet copper-wire segment. Note: The Broadcom NetXtreme Gigabit Ethernet adapter supports Automatic MDI Crossover (MDIX), which eliminates the need for crossover cables when connecting machines back-to-back.
NetXtreme User Guide Functionality and Features Section 2: Functionality and Features • Functional Description • Features • Supported Operating Environments • Network Link and Activity Indication Functional Description Broadcom NetXtreme Gigabit Ethernet adapters connect a PCI Express™ compliant system to a Gigabit Ethernet network.
NetXtreme User Guide Functionality and Features • Virtual LAN (VLAN) support— IEEE 802.1q VLAN tagging • TCP, IP, UDP checksum offload • Large Send Offload (LSO), TCP Segmentation Offload (TSO) • Hardware assist for IEEE 1588 and IEEE 802.1AS time synchronization implementations • IEEE 802.3x flow control • SMBus 2.0 Interface • Statistics for SNMP MIB II, Ethernet-like MIB and Ethernet MIB (IEEE 802.
NetXtreme User Guide Functionality and Features ASIC with Embedded RISC Processor The core control for Broadcom NetXtreme Gigabit Ethernet adapters resides in a tightly integrated, highperformance ASIC. The ASIC includes a RISC processor. This functionality provides the flexibility to add new features to the card and adapts it to future network requirements through software downloads.
NetXtreme User Guide Windows Driver and Management Application Installation Section 3: Windows Driver and Management Application Installation • Installing the Driver Software • Modifying the Driver Software • Repairing or Reinstalling the Driver Software • Removing the Device Drivers • Viewing or Changing the Properties of the Adapter • Setting Power Management Options • Configuring the Communication Protocol To Use With BACS4 Broadcom® April 2019 • 2CS57XX-CDUM515 Page 16
NetXtreme User Guide Windows Driver and Management Application Installation Installing the Driver Software Note: These instructions are based on the assumption that your Broadcom NetXtreme adapter was not factory installed. If your controller was installed at the factory, the driver software has been installed for you.
NetXtreme User Guide Windows Driver and Management Application Installation Using Silent Installation Refer to the readme.txt file in the installation folder for command line instructions. Notes: • All commands are case sensitive. • For detailed instructions and information about unattended installs, refer to the Silent.txt file in the Driver_Management_Apps_Installer folder. Note: The REINSTALL switch should only be used if the same installer is already installed on the system.
NetXtreme User Guide Windows Driver and Management Application Installation Repairing or Reinstalling the Driver Software To repair or reinstall the driver software 1. In Control Panel, double-click Add or Remove Programs. 2. Click Broadcom Drivers and Management Applications, and then click Change. 3. Click Next to continue. 4. Click Repair or Reinstall to repair errors or install drivers for new adapters. 5. Click Next to continue. 6. Click Install. 7. Click Finish to close the wizard. 8.
NetXtreme User Guide Windows Driver and Management Application Installation Viewing or Changing the Properties of the Adapter To view or change the properties of the Broadcom network adapter 1. In Control Panel, click Broadcom Control Suite 4. 2. Click the Advanced section of the Configurations tab. Setting Power Management Options You can set power management options to allow the operating system to turn off the controller to save power or to allow the controller to wake up the computer.
NetXtreme User Guide Windows Driver and Management Application Installation Configuring the Communication Protocol To Use With BACS4 There are two main components of the BACS4 management application: the provider component and the client software. A provider is installed on a server, or “managed host”, that contains one or more NICs. The provider collects information on the NICs and makes it available for retrieval from a management PC on which the client software is installed.
NetXtreme User Guide Windows Driver and Management Application Installation Step 2: Perform Basic Configuration on the Server The Windows firewall must be enabled for WinRM to work properly. For detailed information about firewall configuration, see Step 7: Additional Server Configuration.
NetXtreme User Guide Windows Driver and Management Application Installation winrm qc or winrm quickconfig 13. When the tool displays “Make these changes[y/n]?“, enter “y”. 14. Enter one of the following commands to check whether an HTTP listener is created: winrm enumerate winrm/confg/listener or winrm e winrm/config/Listener 15. Enter the following command from the command prompt to test locally.
NetXtreme User Guide Windows Driver and Management Application Installation openssl req -new -key server.key -out server.csr If this command does not work, try the following: openssl req –new –key server.key –out server.csr –config openssl.cnf The openssl.cnf file should be placed in the same directory where openssl is placed. Openssl.cnf is located in the folder C:\Program Files (x86)\GnuWin32\share.
NetXtreme User Guide Windows Driver and Management Application Installation Enter the password and be sure to remember it. The password is required when importing the certificate on the Windows server and client. 8. Make a copy of the certificate file server.crt and place it on the server where BACS will be installed, so that it can be imported.
NetXtreme User Guide Windows Driver and Management Application Installation 16. Ensure that only Server Authentication is enabled, as shown in the figure. 17. Open Trusted Root Certification Authorities and then open Certificates. 18. Follow the instructions from Step 11 on page 25 to Step 17 on page 26 Note: See Perform HTTPS Configuration (if you plan to use HTTPS) for instructions on importing the self-signed certificate on a client. Step 6: Configure WinRM HTTPS/SSL on the Server 1.
NetXtreme User Guide Windows Driver and Management Application Installation Notes: • If the certificate was generated using the host name, enter the host name. If it was generated using the IP address, enter the IP address. For an IPv6 address, use brackets [ ] around the address. • If HTTPS is configured in your system, the listener must be deleted before creating a new HTTPS listener. Use the following command: winrm delete winrm/config/Listener?Address=*+Transport=HTTPS a.
NetXtreme User Guide Windows Driver and Management Application Installation 3. Click Add Port. 4. Enter a meaningful Name, for example “WinRM rule” and port number, for example, 5985 for HTTP or 5986 for HTTPS. 5. Click OK. Useful WinRM Commands Command Description winrm quickconfig or winrm qc Configures WinRM with default settings winrm enumerate winrm/config/Listener or winrm e winrm/config/Listener Helps to check which service listener are enabled and listening on which port and IP Address.
NetXtreme User Guide Windows Driver and Management Application Installation After you generate a self-signed certificate, as described in To generate a self-signed certificate for the Windows Server:, you can import the certificate on the client to facilitate a connection between server and client. Ensure that all steps mentioned in section To generate a self-signed certificate for the Windows Server: are completed, including copying hostname.
NetXtreme User Guide Windows Driver and Management Application Installation Using WMI No special configuration is required to use WMI on the Windows client. Perform the steps in the following sections to configure WMI on the Windows server. Step 1: Set up Namespace Security Using WMI Control The WMI Control provides one way to manage namespace security.
NetXtreme User Guide Windows Driver and Management Application Installation This configuration can be easily done using the following procedure. If you do not have sufficient privileges to configure security for WMI access, contact your Network Administrator. 1. Click Start, click Run, type DCOMCNFG, and then click OK. 2. The Component Services dialogue box displays. 3. Open Component Services and then open Computers. 4. Right-click My Computer and click Properties. 5.
NetXtreme User Guide Windows Driver and Management Application Installation Special Configuration for WMI on Different Systems In Windows Vista and Windows 7, in order to let all users in the administrator group connect using the WMI namespace, the user might need to change the LocalAccountTokenFilterPolicy as needed.
NetXtreme User Guide Linux Driver and Management Application Installation Section 4: Linux Driver and Management Application Installation • Packaging • Installing TG3 Driver Software • Network Installations • Unloading/Removing the TG3 Driver • Driver Messages • Teaming with Channel Bonding • Linux Management Application Installation Packaging The Linux TG3 driver is released in the following packaging formats (file names): • Source RPM (tg3-version.3dkms.src.
NetXtreme User Guide Linux Driver and Management Application Installation Installing TG3 Driver Software • Installing the Source RPM Package • Building the Driver from the Source TAR File Installing the Source RPM Package Prerequisites: • Linux kernel source • C compiler Procedure: 1. Install the source RPM package. rpm -ivh tg3-version.src.rpm 2. Change the directory to the RPM path and build the binary driver for your kernel (the RPM path is different for different Linux distributions).
NetXtreme User Guide Linux Driver and Management Application Installation Building the Driver from the Source TAR File 1. Create a directory (tg3-version) and extract the TAR files to the directory. tar xvzf tg3-version.tgz 2. Build the driver tg3.o as a loadable module for the running kernel. CD tg3-version make clean make; make install 3. Test the driver by loading it. rmmod tg3 modprobe tg3 No message should be returned if this command runs properly.
NetXtreme User Guide Linux Driver and Management Application Installation Driver Messages The following are the most common sample messages that may be logged in the /var/log/messages file. Use dmesg -nlevel to control the level at which messages appear on the console. Most systems are set to level 6 by default. Driver Sign on tg3.
NetXtreme User Guide Linux Driver and Management Application Installation Linux Management Application Installation • Overview • Installing WS-MAN or CIM-XML on Linux Server • Installing WS-MAN or CIM-XML on Linux Client • Installing the Broadcom Advanced Control Suite Overview The Broadcom Advanced Control Suite version 4 (BACS4) is a management application for configuring the NetXtreme I families of adapters.
NetXtreme User Guide If the client uses: Linux Driver and Management Application Installation And the managed host uses: BACS can use these communication protocols: • WMI = Windows Management Instrumentation. • WS-MAN = Web Service-Management. WinRM is a Windows-based implementation and OpenPegasus is an open-source implementation of the that operates on Linux. • CIM-XML = An XML-based version of OpenPegasus.
NetXtreme User Guide Linux Driver and Management Application Installation From Source (Red Hat and SuSE) The OpenPegasus source can be downloaded from www.openpegasus.org. Note: If not already installed, download and install the openssl and libopenssl-devel rpm. This step is optional and required only if you are planning to use HTTPS to connect the client to the managed host. Set the Environment Variables Set the environment variables for building OpenPegasus as follows.
NetXtreme User Guide Linux Driver and Management Application Installation Note: You can add these exports at the end of the .bash_profile. This file is located in the /root directory. • The environment variables will be set when a user logs in using PuTTY. • On the Linux system itself, for each terminal where the environment variables are not set, run the following command: source /root/.bash_profile • When you logout and login, the environment variables will be set.
NetXtreme User Guide Linux Driver and Management Application Installation Enable Authentication The following OpenPegasus properties have to be set as described in this section. Otherwise, the Broadcom CIM Provider will not work properly. Ensure the following are set before launching BACS and connecting to the provider. Start CIM server if it is not already started.
NetXtreme User Guide Linux Driver and Management Application Installation Ensure that OpenPegasus is installed properly before installing CMPI Provider. Install Enter following command to install Broadcom CMPI Provider. % rpm -i BRCM_CMPIProvider-{version}.{arch}.rpm Uninstall Enter following command to uninstall Broadcom CMPI Provider: % rpm -e BRCM_CMPIProvider Step 5: Perform Linux Firewall Configuration, If Required Follow these procedures to open the appropriate ports in the firewall: Red Hat 1.
NetXtreme User Guide Linux Driver and Management Application Installation Example: For CIM-XML, use the following values: • Source Network: 0/0 (means all) • Protocol: TCP • Destination Port: 5988:5989 • Source Port: Leave blank. Step 6: Install BACS and Related Management Applications See Installing the Broadcom Advanced Control Suite.
NetXtreme User Guide Linux Driver and Management Application Installation Installing WS-MAN or CIM-XML on Linux Client No special software components are required on the Linux client system to use the HTTP except installing the BACS management application. However, for WS-MAN installations, you can optionally configure the HTTPS protocol for use with BACS.
NetXtreme User Guide Linux Driver and Management Application Installation openssl rsa -in server.key.org -out server.key 5. Generate a self-signed certificate: To generate a self-signed certificate which is active for 365 days, enter the following command: openssl x509 -req -days 365 -in server.csr -signkey server.key -out server.crt The following output displays: Signature ok subject=/C=US/ST=California/L=Irvine/O=Broadcom Corporation/OU=Engineering/CN=MGMTAPPLAB3/emailAddress= Getting Private key 6.
NetXtreme User Guide Linux Driver and Management Application Installation Copy hostname.pem, which you created in Generate a Self-Signed Certificate for Windows/Linux Server, into the certificate directory of the Linux client. For example, if the certificate directory is /etc/ssl/certs, copy hostname.pem to /etc/ssl/certs. 1. Change directory to /etc/ssl/certs. 2. Create a hash value by running the following command. openssl x509 -noout -hash -in hostname.
NetXtreme User Guide VMware Driver Software Section 5: VMware Driver Software • Packaging • Drivers Packaging The VMware driver is released in the following packaging format. Table 3: VMware Driver Packaging Format Drivers VMware VIB vmware-esx-drivers-net-tg3-version.x86_64.vib Drivers Download, Install, and Update Drivers To download, install, or update the VMware ESX/ESXi driver for NetXtreme I GbE network adapters, see http:/ /www.vmware.com/support.
NetXtreme User Guide VMware Driver Software Note that the NIC # above does not correspond to the vmnic<#>. The NIC number is the system vmnic probe order number. Optimally, the number of NetQueues matches the number of CPUs on the machine. Driver Parameters Several optional parameters can be supplied as a command line argument to the vmkload_mod command. These parameters can also be set via the esxcfg-module command. See the man page for more information.
NetXtreme User Guide VMware Driver Software vmnic0#: Tigon3 [partno (BCM95xxx) rev 4202 PHY (57xx) (PCI Express) 10/100/1000BaseT Ethernet :00:xx:xx:xx:xx:xx vmnic0#: RXcsums [1] LinkChg REG [0] MIirq [0] ASF [0] Split [0] Wirespeed [1]TSOcap [1] vmnic0#: dma_rwctrl [76180000] ACPI : PCI interrupt 0000:02:02.0 [A] -> GSI 26 (level,low) -> IRQ 233 Link Up and Speed Indication tg3: vmnic0: Link is up at 1000 Mbps, full duplex. tg3: vmnic0: Flow control is on for TX and on for RX.
NetXtreme User Guide Using Broadcom Advanced Control Suite 4 Section 6: Using Broadcom Advanced Control Suite 4 • Broadcom Advanced Control Suite Overview • Managing the Network Adapter • Starting Broadcom Advanced Control Suite • Viewing Statistics • BACS Interface • Configuring Teaming • Configuring Preferences in Windows • Configuring With the Command Line Interface Utility • Connecting to a Host • Managing VLANs • Managing the Host • Troubleshooting BACS Broadcom Advanced Contr
NetXtreme User Guide Using Broadcom Advanced Control Suite 4 Starting Broadcom Advanced Control Suite In Control Panel, click Broadcom Control Suite 4, or click the BACS icon in the taskbar located at the bottom of the Windows or Windows Server desktop. On Linux systems, you can double-click the BACS4 desktop icon, or access the BACS program from the task bar under System Tools. (If you are having difficulty launching BACS on a Linux system, see the related topic in Troubleshooting BACS.
NetXtreme User Guide Using Broadcom Advanced Control Suite 4 Explorer View Pane You can dock and pin the Explorer View pane on the left side, right side, top, or bottom of the main window. The Explorer View pane lists the objects that can be viewed, analyzed, tested, or configured by BACS. When an item is selected in the Explorer View pane, the tabs showing the information and options that are available for the item appear in the Context View pane.
NetXtreme User Guide Using Broadcom Advanced Control Suite 4 Context View Selector The Context View selector appears below the menu bar and includes the filter and tab categories. Although you can expand and collapse the categories that appear on tabs in the Context View pane, you can alternatively display a category by selecting the box next to the category name.
NetXtreme User Guide Using Broadcom Advanced Control Suite 4 Teams (Windows only) • Create Teams: Creates new teams with either the Teaming Wizard or in Advanced mode. • Manage Teams: Manages existing teams with either the Teaming Wizard or in Advanced mode. Description Pane The Description pane provides information, configuration instructions, and options for the selected parameter in the Context View pane.
NetXtreme User Guide Using Broadcom Advanced Control Suite 4 Connecting to a Host You can add one or more Windows or Linux hosts to manage from BACS. To add a local host 1. From the Action menu, click Add Host. 2. For both Windows and Linux hosts, do not change the default settings. The User name and Password are not required while connecting to the local host. 3. Select Persist if you want BACS to save the information for this host. 4. Click Ok.
NetXtreme User Guide Using Broadcom Advanced Control Suite 4 Managing the Host At the host level, you can view host information and configure parameters from the following tabs: • Information • Configuration To view host information Select the host in the Explorer View pane, and then select the Information tab to view host-level information. Information Tab: Host Information Host Name. Displays the name of the host. OS Version Info. Displays the operating system, including the version. Platform.
NetXtreme User Guide Broadcom® April 2019 • 2CS57XX-CDUM515 Using Broadcom Advanced Control Suite 4 Page 57
NetXtreme User Guide Using Broadcom Advanced Control Suite 4 Managing the Network Adapter The installed network adapters appear one level below the host in the hierarchical tree in the Explorer View pane.
NetXtreme User Guide Using Broadcom Advanced Control Suite 4 Viewing Driver Information The Driver Information section of the Information tab displays data about the driver for the selected network adapter. To view Driver Information for any installed network adapter, click the name of the adapter listed in the Explorer View pane, then click the Information tab. Driver Status. The status of the adapter driver. • Loaded. Normal operating mode.
NetXtreme User Guide Using Broadcom Advanced Control Suite 4 Viewing Resource Information The Resources section of the Information tab displays information about connections and other essential functions for the selected network adapter. To view Resources for any installed network adapter, click the name of the adapter listed in the Explorer View pane, then click the Information tab. Note: Some information may not be available for all Broadcom network adapters. Bus Type.
NetXtreme User Guide Using Broadcom Advanced Control Suite 4 Device No. The number assigned to the adapter by the operating system. Function No. The port number of the adapter. For a single-port adapter, the function number is 0. For a two-port adapter, the function number for the first port is 0, and the function number for the second port is 1. Interrupt Request. The interrupt line number that is associated with the adapter. Valid numbers range from 2 to 25. Memory Address.
NetXtreme User Guide Using Broadcom Advanced Control Suite 4 ASIC Version. The chip version of the Broadcom adapter (this information is not available for adapters made by others). Firmware Version. The firmware version of the Broadcom adapter (this information is not available for adapters made by others). This information is only available for Broadcom NetXtreme adapters. Vendor ID. The vendor ID. Device ID. The adapter ID. Subsystem Vendor ID. The subsystem vendor ID. Subsystem ID. The subsystem ID.
NetXtreme User Guide Using Broadcom Advanced Control Suite 4 Running Diagnostic Tests The Diagnostic Tests option on the Diagnostics tab lets you check the state of the physical components on a Broadcom network adapter. You can trigger the tests manually, or choose to have BACS 3 continuously perform them. If the test are performed continuously, then the number of passes and fails in the Result field for each test increments every time the tests are performed.
NetXtreme User Guide Using Broadcom Advanced Control Suite 4 Interrupt. This test verifies that the Network Device Driver Interface Specification (NDIS) driver is able to receive interrupts from the adapter. LoopBack MAC. This test verifies that the NDIS driver is able to send packets to and receive packets from the adapter. LoopBack PHY. This test verifies that the NDIS driver is able to send packets to and receive packets from the adapter. Test LED.
NetXtreme User Guide • Using Broadcom Advanced Control Suite 4 N/A. Algorithm failed to reach a conclusion. Link. The link connection speed and duplex mode. Status. The status after the test is run, either completed or failed. There are several factors that could have an effect on the test results: • Link partner. Various switch and hub manufacturers implement different PHYs. Some PHYs are not IEEE compliant. • Cable quality. Category 3, 4, 5, and 6 may affect the test results.
NetXtreme User Guide Using Broadcom Advanced Control Suite 4 Speed & Duplex. The Speed & Duplex property sets the connection speed and mode to that of the network. Note that Full-Duplex mode allows the adapter to transmit and receive network data simultaneously. • 10 Mb Full. Sets the speed at 10 Mbit/s and the mode to Full-Duplex. • 10 Mb Half. Sets the speed at 10 Mbit/s and the mode to Half-Duplex. • 100 Mb Full. Sets the speed at 100 Mbit/s and the mode to Full-Duplex. • 100 Mb Half.
NetXtreme User Guide Using Broadcom Advanced Control Suite 4 Note: If an intermediate driver is managing the network adapter for VLAN tagging, the Priority & VLAN Disabled and Priority Enabled settings should not be used. Use the Priority & VLAN Enabled setting and change the VLAN ID to 0 (zero). VLAN ID. Enables VLAN tagging and configures the VLAN ID when Priority & VLAN Enabled is selected as the Priority & VLAN setting.
NetXtreme User Guide Using Broadcom Advanced Control Suite 4 General Statistics General Statistics show the transmitted and received statistics to and from the adapter. Frames Tx OK. A count of the frames that were successfully transmitted. This counter is incremented when the transmit status is reported as Transmit OK. Frames Rx OK. A count of the frames that were successfully received.
NetXtreme User Guide Using Broadcom Advanced Control Suite 4 Notes: • For further information regarding teaming protocols, see “Teaming” in your Broadcom network adapter user guide. • If you do not enable LiveLink™ when configuring teams, disabling Spanning Tree Protocol (STP) at the switch is recommended. This minimizes the downtime due to spanning tree loop determination when failing over. LiveLink mitigates such issues.
NetXtreme User Guide Using Broadcom Advanced Control Suite 4 Note: If you prefer to work without the wizard for now, click Expert Mode. If you want to always use Expert Mode to create a team, select Default to Expert Mode on next start. See Using Expert Mode. 2. To continue using the wizard, click Next. 3. Type the team name and then click Next. If you want to review or change any of your settings, click Back. Click Cancel to discard your settings and exit the wizard.
NetXtreme User Guide Using Broadcom Advanced Control Suite 4 4. Select the type of team you want to create. If the team type is an SLB type team, click Next. If the team type is not an SLB type team, then a dialog box appears. Verify that the network switch connected to the team members is configured correctly for the team type, click OK, and continue.
NetXtreme User Guide Using Broadcom Advanced Control Suite 4 5. From the Available Adapters list, click the adapter you want to add to the team and then click Add. Remove team members from the Team Members list by clicking the adapter and then clicking Remove. Click Next. Note: There must be at least one Broadcom network adapter assigned to the team. The Large Send Offload (LSO) and Checksum Offload (CO) columns indicate if the LSO, and/or the CO properties are supported for the adapter.
NetXtreme User Guide Using Broadcom Advanced Control Suite 4 6. If you want to designate one of the adapters as a standby member (optional), select Use the following member as a standby member, then choose the standby member from the list of adapters. 7. The Auto-Fallback Disable mode feature allows the team to continue using the standby member rather than switching back to the primary member if the primary member comes back online. To enable this feature, select Enable Auto-Fallback Disable mode.
NetXtreme User Guide Using Broadcom Advanced Control Suite 4 8. If you want to configure LiveLink, select Yes, otherwise select No, then click Next.
NetXtreme User Guide Using Broadcom Advanced Control Suite 4 9. Select the probe interval (the number of seconds between each retransmission of a link packet to the probe target) and the maximum number of probe retries (the number of consecutively missed responses from a probe target before a failover is triggered). 10. Set the Probe VLAN ID to allow for connectivity with probe targets residing on a tagged VLAN.
NetXtreme User Guide Using Broadcom Advanced Control Suite 4 13. If you want to create a VLAN on the team, select Add VLAN, or if you want to change the settings of an existing VLAN, select Edit VLAN, then click Next. If you do not want to create or edit a VLAN, select Skip Manage VLAN, then click Next, and continue with the wizard from the Finish screen (see Step 18 on page 79 of this procedure). VLANs enable you to add multiple virtual adapters that are on different subnets.
NetXtreme User Guide Using Broadcom Advanced Control Suite 4 15. To tag the VLAN, select Tagged and then click Next. Otherwise, click Untagged, click Next, and continue with the wizard to add additional VLANs (see Step 17 on page 79 of this procedure).
NetXtreme User Guide Using Broadcom Advanced Control Suite 4 16. Type the VLAN tag value and then click Next. The value must be between 1 and 4094.
NetXtreme User Guide Using Broadcom Advanced Control Suite 4 17. Select Yes to add or manage another VLAN and then click Next. Repeat until you do not want to add or manage any additional VLANs. Note: You can define up to 64 VLANs per team (63 VLANs that are tagged and 1 VLAN that is not tagged). Adding several VLANS may slow down the reaction time of the Windows interface due to memory and processor time usage for each VLAN.
NetXtreme User Guide Using Broadcom Advanced Control Suite 4 Note: At any point in the Broadcom Teaming Wizard procedure, click Preview to get a visual representation of what the team will look like before committing any changes.
NetXtreme User Guide Using Broadcom Advanced Control Suite 4 19. Click the team name in the Team Management pane to view the team's properties in the Information tab, transfer and receive data in the Statistics tab.
NetXtreme User Guide Using Broadcom Advanced Control Suite 4 Using Expert Mode Use Expert Mode to create a team, modify a team, add a VLAN, and configure LiveLink for a Smart Load Balance and Failover and SLB (Auto-Fallback Disable) team. To create a team using the wizard, see Using the Broadcom Teaming Wizard. To set the default Teaming Mode, select Options from the Tools menu, then select Expert Mode or Wizard Mode (the default is Wizard Mode).
NetXtreme User Guide Using Broadcom Advanced Control Suite 4 3. Click the Create Team tab. Note: The Create Team tab appears only if there are teamable adapters available. 4. Click the Team Name field to enter a team name. 5. Click the Team Type field to select a team type. 6. Assign any available adapter or adapters to the team by selecting the adapter from the Load Balance Members list. There must be at least one adapter selected in the Load Balance Members list. 7.
NetXtreme User Guide Using Broadcom Advanced Control Suite 4 Note: Adding a network adapter to a team where its driver is disabled may negatively affect the offloading capabilities of the team. This may have an impact on the team’s performance. Therefore, it is recommended that only driver-enabled network adapters be added as members to a team. 8. Type the value for Team MTU. 9. Click Create to save the team information. 10. Repeat steps Step 4 through Step 9 to define additional teams.
NetXtreme User Guide Using Broadcom Advanced Control Suite 4 Modifying a Team After you have created a team, you can modify the team in the following ways: • Change the type of team • Change the members assigned to the team • Add a VLAN • Modify a VLAN (using Expert Mode) • Remove a team or a VLAN (using Expert Mode) To modify a team 1. From the Team menu, click Edit Team, or right-click one of the teams in the list and select Edit Team.
NetXtreme User Guide Using Broadcom Advanced Control Suite 4 4. Make the desired changes, and then click Update. The changes have not yet been applied; click the Preview tab to view the updated team structure before applying the changes. 5. Click Apply/Exit to apply the updates and exit the Manage Teams window. 6. Click Yes when the message is displayed indicating that the network connection will be temporarily interrupted. Adding a VLAN You can add virtual LANs (VLANs) to a team.
NetXtreme User Guide Using Broadcom Advanced Control Suite 4 1. From the Teams menu, select Add VLAN. 2. The Welcome screen appears. 3. Click Expert Mode. 4. On the Create Team tab of the Manage Teams window, click Manage VLAN(s). 5. Type the VLAN name, then select the type and ID. 6. Click Create to save the VLAN information. As VLANs are defined, they can be selected from the Team Name list, but they have not yet been created. 7.
NetXtreme User Guide Using Broadcom Advanced Control Suite 4 Deleting a VLAN The procedure below applies when you are in Expert Mode. To delete a VLAN 1. Select the VLAN to delete. 2. From the Teams menu, select Remove VLAN. 3. Click Apply. 4. Click Yes when the message is displayed indicating that the network connection will be temporarily interrupted. Note: If you delete a team, any VLANs configured for that team are also deleted.
NetXtreme User Guide Using Broadcom Advanced Control Suite 4 Configuring LiveLink for a Smart Load Balancing and Failover and SLB (Auto-Fallback Disable) Team LiveLink is a feature of BASP that is available for the Smart Load Balancing (SLB) and SLB (Auto-Fallback Disable) type of teaming. The purpose of LiveLink is to detect link loss beyond the switch and to route traffic only through team members that have a live link. Read the following notes before you attempt to configure LiveLink.
NetXtreme User Guide Using Broadcom Advanced Control Suite 4 Note: All of the member IP addresses must be in the same subnet as the probe targets. 9. Click Update. Repeat these steps for each of the other listed team members. 10. Click Apply/Exit. Saving and Restoring a Configuration To save a configuration 1. From the File menu, select Team Save As. 2. Type the path and file name of the new configuration file, and then click Save (a .bcg extension is added).
NetXtreme User Guide Using Broadcom Advanced Control Suite 4 Viewing BASP Statistics The Statistics section shows performance information about the network adapters that are on a team. To view BASP Statistics information for any team member adapter or the team as a whole, click the name of the adapter or team listed in the Team Management pane, then click the Statistics tab. Click Refresh to get the most recent values for each statistic. Click Reset to change all values to zero. Tx. Packet.
NetXtreme User Guide Using Broadcom Advanced Control Suite 4 Configuring With the Command Line Interface Utility An alternate method to BACS for configuring Broadcom network adapters is with BACSCLI, which is a Broadcom utility that allows you to view information and configure network adapters using a console in either a non-interactive command line interface (CLI) mode or an interactive mode.
NetXtreme User Guide Using Broadcom Advanced Control Suite 4 Managing VLANs Overview Virtual LANs (VLANs) allow you to split your physical LAN into logical parts, to create logical segmentation of workgroups, and to enforce security policies for each logical segment. Each defined VLAN behaves as its own separate network with its traffic and broadcasts isolated from the others, increasing bandwidth efficiency within each logical group.
NetXtreme User Guide Using Broadcom Advanced Control Suite 4 Table 5: Example VLAN Network Topology (Cont.) Component Description VLAN #3 Includes the Main Server, the Accounting Server and PC #4. This VLAN is an accounting group. Main Server A high-use server that needs to be accessed from all VLANs and IP subnets. The Main Server has a Broadcom adapter installed. All three IP subnets are accessed via the single physical adapter interface.
NetXtreme User Guide Using Broadcom Advanced Control Suite 4 Troubleshooting BACS Problem: When attempting to open BACS on a Linux System, the following error message displays: “Another instance of the BACS client appears to be running on this system. Only one instance of the BACS client can be running at a time. If you are sure that no other BACS client is running, then a previous instance may have quit unexpectedly.” Solution: This message displays if you try to run a second instance of BACS.
NetXtreme User Guide Teaming Sect i on 7: Teaming • Overview • Load Balancing and Fault Tolerance Note: See Broadcom Gigabit Ethernet Teaming Services for detailed information on the following topics: • Glossary of Terms and Acronyms • Teaming Concepts • Software Components • Hardware Requirements • Supported Features by Team Type • Selecting a Team Type • Teaming Mechanisms • Architecture • Types of Teams • Driver Support by Operating System • Supported Teaming Speeds • Teaming
NetXtreme User Guide Teaming Overview Adapter teaming allows you to group network adapters together to function as a team. The benefits of teaming include allowing membership to VLANs, providing load balancing between adapters, and offering fault tolerance. These benefits can be combined such that you can couple the functionality of load balancing for the load balance members and the capability of employing a failover with having the team participate on different VLANs.
NetXtreme User Guide Teaming Table 6: Types of Teams Operating System Available Types of Teams Linux Team adapters using the bonding kernel module and a channel bonding interface. See your Linux documentation for more information. Smart Load Balancing™ and Failover Smart Load Balancing™ and Failover is the Broadcom implementation of load balancing based on IP flow. This feature supports balancing IP traffic across multiple adapters (team members) in a bidirectional manner.
NetXtreme User Guide Teaming SLB (Auto-Fallback Disable) The SLB (Auto-Fallback Disable) type of team is identical to the Smart Load Balancing and Failover type of team, with the following exception—when the standby member is active, if a primary member comes back on line, the team continues using the standby member, rather than switching back to the primary member.
NetXtreme User Guide Teaming The Smart Load Balancing type of team works with all Ethernet switches without having to configure the switch ports to any special trunking mode. Only IP traffic is load-balanced in both inbound and outbound directions. Other protocol packets are sent and received through one primary interface only. Failover for non-IP traffic is supported only for Broadcom network adapters.
NetXtreme User Guide Broadcom Gigabit Ethernet Teaming Services Section 8: Broadcom Gigabit Ethernet Teaming Services • Introduction • Teaming Mechanisms • Teaming and Other Advanced Networking Properties • General Network Considerations • Application Considerations • Troubleshooting Teaming Problems • Frequently Asked Questions • Event Log Messages Broadcom® April 2019 • 2CS57XX-CDUM515 Page 101
NetXtreme User Guide Broadcom Gigabit Ethernet Teaming Services Introduction • Teaming Glossary • Teaming Concepts • Software Components • Hardware Requirements • Supported Features by Team Type • Selecting a Team Type This section describes the technology and implementation considerations when working with the network teaming services offered by the Broadcom software shipped with systems.
NetXtreme User Guide Broadcom Gigabit Ethernet Teaming Services Table 8: Teaming Glossary Item Definition Smart Load Balancing (SLB) Switch-independent load balancing and failover type of team, in which the intermediate driver manages outgoing/incoming traffic.
NetXtreme User Guide Broadcom Gigabit Ethernet Teaming Services Teaming and Network Addresses A team of adapters function as a single virtual network interface and does not appear any different to other network devices than a non-teamed adapter. A virtual network adapter advertises a single Layer 2 and one or more Layer 3 addresses. When the teaming driver initializes, it selects one MAC address from one of the physical adapters that make up the team to be the Team MAC address.
NetXtreme User Guide Broadcom Gigabit Ethernet Teaming Services Table 9: Available Teaming Types Teaming Type Switch-Dependent (Switch must support specific type of team) Link Aggregation Control Protocol support is required on the switch Load Balancing Failover Generic Trunking (FEC/GEC)/802.
NetXtreme User Guide Broadcom Gigabit Ethernet Teaming Services When the clients and the system are on different subnets, and incoming traffic has to traverse a router, the received traffic destined for the system is not load balanced. The physical adapter that the intermediate driver has selected to carry the IP flow carries all of the traffic. When the router sends a frame to the team IP address, it broadcasts an ARP request (if not in the ARP cache).
NetXtreme User Guide Broadcom Gigabit Ethernet Teaming Services Link Aggregation (IEEE 802.3ad LACP) Link Aggregation is similar to Generic Trunking except that it uses the Link Aggregation Control Protocol to negotiate the ports that will make up the team. LACP must be enabled at both ends of the link for the team to be operational. If LACP is not available at both ends of the link, 802.3ad provides a manual aggregation that only requires both ends of the link to be in a link up state.
NetXtreme User Guide Broadcom Gigabit Ethernet Teaming Services SLB (Auto-Fallback Disable) This type of team is identical to the Smart Load Balance and Failover type of team, with the following exception—when the standby member is active, if a primary member comes back on line, the team continues using the standby member rather than switching back to the primary member. This type of team is supported only for situations in which the network cable is disconnected and reconnected to the network adapter.
NetXtreme User Guide Broadcom Gigabit Ethernet Teaming Services Hardware Requirements • Ethernet Switch • Router The various teaming modes described in this document place certain restrictions on the networking equipment used to connect clients to teamed systems. Each type of network interconnect technology has an effect on teaming as described in the following sections. Ethernet Switch Ethernet switches allow an Ethernet network to be broken into multiple broadcast domains.
NetXtreme User Guide Broadcom Gigabit Ethernet Teaming Services Table 11: Comparison of Team Types (Cont.) Type of Team Fault Tolerance Load Balancing SwitchIndependent Dynamic Link SwitchDependent Static Aggregation (IEEE 802.
NetXtreme User Guide Broadcom Gigabit Ethernet Teaming Services Selecting a Team Type The following flow chart provides the decision flow when planning for teaming. The primary rationale for teaming is the need for additional network bandwidth and fault tolerance. Teaming offers link aggregation and fault tolerance to meet both of these requirements.
NetXtreme User Guide Broadcom Gigabit Ethernet Teaming Services Teaming Mechanisms • Architecture • Driver Support by Operating System • Supported Teaming Speeds Architecture The Broadcom Advanced Server Program is implemented as an NDIS intermediate driver (see Figure 4). It operates below protocol stacks such as TCP/IP appears as a virtual adapter. This virtual adapter inherits the MAC Address of the first port initialized in the team.
NetXtreme User Guide Broadcom Gigabit Ethernet Teaming Services Outbound Traffic Flow The Broadcom Intermediate Driver manages the outbound traffic flow for all teaming modes. For outbound traffic, every packet is first classified into a flow, and then distributed to the selected physical adapter for transmission. The flow classification involves an efficient hash computation over known protocol fields. The resulting hash value is used to index into an Outbound Flow Hash Table.
NetXtreme User Guide Broadcom Gigabit Ethernet Teaming Services Protocol Support ARP and IP/TCP/UDP flows are load balanced. If the packet is an IP protocol only, such as ICMP or IGMP, then all data flowing to a particular IP address will go out through the same physical adapter. If the packet uses TCP or UDP for the L4 protocol, then the port number is added to the hashing algorithm, so two separate L4 flows can go out through two separate physical adapters to the same IP address.
NetXtreme User Guide Broadcom Gigabit Ethernet Teaming Services Driver Support by Operating System As previously noted, BASP is supported in Windows Server 2008 and 2012 operating system environments. The various teaming mode features are summarized in the table below.
NetXtreme User Guide Broadcom Gigabit Ethernet Teaming Services Table 12: Teaming Mode Features (Cont.) Features Windows Support User interface BACS Number of teams 16 Number of adapters per team 8 Hot replace Yes Hot add Yes Hot remove Yes Link speed support Different speeds Frame protocol All Incoming packet management Switch Outgoing packet management BASP Failover event Loss of link only Failover time <500 ms Fallback time 1.
NetXtreme User Guide Broadcom Gigabit Ethernet Teaming Services Teaming and Other Advanced Networking Properties • Checksum Offload • IEEE 802.1p QoS Tagging • Large Send Offload • Jumbo Frames • IEEE 802.1Q VLANs • Wake on LAN • Preboot Execution Environment (PXE) Before creating a team, adding or removing team members, or changing advanced settings of a team member, make sure each team member has been configured similarly.
NetXtreme User Guide Broadcom Gigabit Ethernet Teaming Services Checksum Offload Checksum Offload is a property of the Broadcom network adapters that allows the TCP/IP/UDP checksums for send and receive traffic to be calculated by the adapter hardware rather than by the host CPU. In high-traffic situations, this can allow a system to handle more connections more efficiently than if the host CPU were forced to calculate the checksums.
NetXtreme User Guide Broadcom Gigabit Ethernet Teaming Services Wake on LAN Wake on LAN (WOL) is a feature that allows a system to be awakened from a sleep state by the arrival of a specific packet over the Ethernet interface. Because a virtual adapter is implemented as a software only device, it lacks the hardware features to implement Wake on LAN and cannot be enabled to wake the system from a sleeping state via the virtual adapter.
NetXtreme User Guide Broadcom Gigabit Ethernet Teaming Services General Network Considerations • Teaming Across Switches • Spanning Tree Algorithm • Teaming with Microsoft NLB/WLBS Teaming Across Switches SLB teaming can be configured across switches. The switches, however, must be connected together. Generic Trunking and Link Aggregation do not work across switches because each of these implementations requires that all physical adapters in a team share the same Ethernet MAC address.
NetXtreme User Guide Broadcom Gigabit Ethernet Teaming Services Figure 5: Teaming Across Switches Without an Interswitch Link Blue SLB Team 192.168.1.101 ARP Table 100 = 49:C9 102 = 5E:CA Cannot ping 82:83 1 Top Switch CAM Table: Eth 1: 82:83 Eth2: 49:C9 Eth 4: 5E:C9 Gray SLB Team 192.168.1.102 82:82 (P) ARP Table 100 = 49:C9 101 = 82:82 5E:C9 5E:CA (P) 1 4 4 Bottom Switch CAM Table: Eth 1: 82:83 Eth2: 49:C8 Eth 4: 5E:CA 2 2 49:C9(P) Red SLB Team 192.168.1.
NetXtreme User Guide Broadcom Gigabit Ethernet Teaming Services Figure 7 represents a failover event in which the cable is unplugged on the Top Switch port 4. This is a successful failover with all stations pinging each other without loss of connectivity. Figure 7: Failover Event Blue SLB Team 192.168.1.101 ARP Table 100 = 49:C9 102 = 5E:CA Gray SLB Team 192.168.1.
NetXtreme User Guide Broadcom Gigabit Ethernet Teaming Services After a stable network topology has been established, all bridges listen for hello BPDUs (Bridge Protocol Data Units) transmitted from the root bridge. If a bridge does not get a hello BPDU after a predefined interval (Max Age), the bridge assumes that the link to the root bridge is down. This bridge then initiates negotiations with other bridges to reconfigure the network to re-establish a valid network topology.
NetXtreme User Guide Broadcom Gigabit Ethernet Teaming Services Teaming with Microsoft NLB/WLBS The SLB mode of teaming does not work in Microsoft’s Network Load Balancing (NLB) unicast mode, only in multicast mode. Due to the mechanism used by the NLB service, the recommended teaming configuration in this environment is Failover (SLB with a standby NIC) as load balancing is managed by NLB.
NetXtreme User Guide Broadcom Gigabit Ethernet Teaming Services Application Considerations • Teaming and Clustering—Microsoft Cluster Software • Teaming and Network Backup Teaming and Clustering—Microsoft Cluster Software In each cluster node, it is strongly recommended that customers install at least two network adapters (on-board adapters are acceptable). These interfaces serve two purposes. One adapter is used exclusively for intra-cluster heartbeat communications.
NetXtreme User Guide Broadcom Gigabit Ethernet Teaming Services Load Balancing and Failover As the number of backup streams increases, the overall throughput increases. Each data stream, however, may not be able to maintain the same performance as a single backup stream of 25 MB/s. In other words, even though a backup server can stream data from a single client at 25 MB/s, it is not expected that four simultaneously running backup jobs will stream at 100 MB/s (25 MB/s x 4 streams).
NetXtreme User Guide Broadcom Gigabit Ethernet Teaming Services The same algorithm applies if a third and fourth backup operation is initiated from the backup server. The teamed interface on the backup server transmits a unicast G-ARP to backup clients to inform them to update their ARP cache. Each client then transmits backup data along a route to the target MAC address on the backup server.
NetXtreme User Guide Broadcom Gigabit Ethernet Teaming Services Figure 8: Network Backup With SLB Teaming Across Two Switches Tape Backup SAN Backup Server #1 Client Server - Blue NIC Team NIC Team A B A B GE Switch #1 GE Switch #2 LACP or GEC A B A NIC Team NIC Team Client Server - RED B NIC Team Client Server - Green NIC Team A B Client Server - Yellow Broadcom® April 2019 • 2CS57XX-CDUM515 Page 128
NetXtreme User Guide Broadcom Gigabit Ethernet Teaming Services Troubleshooting Teaming Problems • Teaming Configuration Tips • Troubleshooting Guidelines When running a protocol analyzer over a virtual adapter teamed interface, the MAC address shown in the transmitted frames may not be correct. The analyzer does not show the frames as constructed by BASP and shows the MAC address of the team and not the MAC address of the interface transmitting the frame.
NetXtreme User Guide Broadcom Gigabit Ethernet Teaming Services 11. It is not recommended to connect any team to a hub, as a hub only support half duplex. Hubs should be connected to a team for troubleshooting purposes only. Disabling the device driver of a network adapter participating in an LACP or GEC/FEC team may have adverse affects with network connectivity.
NetXtreme User Guide Broadcom Gigabit Ethernet Teaming Services Frequently Asked Questions Question: Under what circumstances is traffic not load balanced? Why is all traffic not load balanced evenly across the team members? Answer: The bulk of traffic does not use IP/TCP/UDP or the bulk of the clients are in a different network. The receive load balancing is not a function of traffic load, but a function of the number of clients that are connected to the system.
NetXtreme User Guide Broadcom Gigabit Ethernet Teaming Services Question: Can WOL work over a virtual adapter (team)? Answer: Wake-on-LAN functionality operates in an environment before the operating system is loaded. WOL occurs when the system is off or in standby, so no team is configured. Question: What is the maximum number of ports that can be teamed together? Answer: Up to 8 ports can be assigned to a team.
NetXtreme User Guide Broadcom Gigabit Ethernet Teaming Services Question: Is there any special configuration required in the tape backup software or hardware to work with adapter teaming? Answer: No special configuration is required in the tape software to work with teaming. Teaming is transparent to tape backup applications.
NetXtreme User Guide Broadcom Gigabit Ethernet Teaming Services Event Log Messages • Windows System Event Log Messages • Base Driver (Physical Adapter/Miniport) • Intermediate Driver (Virtual Adapter/Team) Windows System Event Log Messages The known base driver and intermediate driver Windows System Event Log status messages for the Broadcom NetXtreme Gigabit Ethernet adapters are listed in the following section.
NetXtreme User Guide Broadcom Gigabit Ethernet Teaming Services Table 15: Base Driver Event Log Messages (Cont.) Message Number Message Cause 7 Network controller configured for 10Mb full-duplex link. Informational message only. The adapter has been manually configured for the No action is required. selected line speed and duplex settings. 8 Network controller configured for 100Mb half-duplex link. Informational message only. The adapter has been manually configured for the No action is required.
NetXtreme User Guide Broadcom Gigabit Ethernet Teaming Services Table 15: Base Driver Event Log Messages (Cont.) Message Number Message Cause 20 Driver initialization failed. Unspecified failure during Reinstall the driver, update driver initialization. to a newer driver, run Broadcom Advanced Control Suite diagnostics, or replace the adapter. 21 Ethernet@WireSpeed is enabled Potential faulty cable or and could not negotiate maximum connection. link speed. Reconnect the cable or change the cable.
NetXtreme User Guide Broadcom Gigabit Ethernet Teaming Services Table 16: Intermediate Driver Event Log Messages (Cont.) System Event Message Number Message Cause Corrective Action 9 Network adapter is disconnected. 10 Network adapter is connected. The physical adapter is Informational message only. No connected to the network action is required. (it has established link). 11 Broadcom Advanced Program Features Driver is not designed to run on this version of Operating System.
NetXtreme User Guide iSCSI Protocol and Broadcom Boot Agent Software Section 9: iSCSI Protocol and Broadcom Boot Agent Software • iSCSI Boot • Broadcom Boot Agent Driver Software iSCSI Boot Broadcom NetXtreme Gigabit Ethernet adapters support iSCSI boot to enable network boot of operating systems to diskless systems. The iSCSI boot allows a Windows or Linux operating system boot from an iSCSI target machine located remotely over a standard IP network.
NetXtreme User Guide iSCSI Protocol and Broadcom Boot Agent Software 3. Map the virtual disk to the iSCSI target created in step 1. 4. Associate an iSCSI initiator with the iSCSI target. 5. Record the iSCSI target name, TCP port number, iSCSI Logical Unit Number (LUN), initiator Internet Qualified Name (IQN), and CHAP authentication details. 6.
NetXtreme User Guide iSCSI Protocol and Broadcom Boot Agent Software Table 17: Configuration Options (Cont.) Option Description Link Up Delay Time Controls how long the iSCSI boot host software waits, in seconds, after an Ethernet link is established before sending any data over the network. The valid values are 0 to 255. As an example, a user may need to set a value for this option if a network protocol, such as Spanning Tree, is enabled on the switch interface to the client system.
NetXtreme User Guide iSCSI Protocol and Broadcom Boot Agent Software • iSCSI parameters via DHCP: Disabled • CHAP Authentication: Disabled • Boot to iSCSI target: Disabled • DHCP Vendor ID: BRCM ISAN • Link Up Delay Time: 0 • Use TCP Timestamp: Enabled (for some targets such as the Dell/EMC AX100i, it is necessary to enable Use TCP Timestamp) • Target as First HDD: Disabled • LUN Busy Retry Count: 0 • IP Version: IPv6. (For IPv6) 2. Select ESC to return to the Main menu. 3.
NetXtreme User Guide iSCSI Protocol and Broadcom Boot Agent Software In a dynamic configuration, you only need to specify that the system’s IP address and target/initiator information are provided by a DHCP server (see IPv4 and IPv6 configurations in Configuring the DHCP Server to Support iSCSI Boot). For IPv4, with the exception of the initiator iSCSI name, any settings on the Initiator Parameters, 1st Target Parameters, or 2nd Target Parameters screens are ignored and do not need to be cleared.
NetXtreme User Guide iSCSI Protocol and Broadcom Boot Agent Software Note: Information on the Initiator Parameters, and 1st Target Parameters screens are ignored and do not need to be cleared. 3. Select Exit and Save Configurations.
NetXtreme User Guide iSCSI Protocol and Broadcom Boot Agent Software Enabling CHAP Authentication Ensure that CHAP authentication is enabled on the target. To enable CHAP authentication 1. From the General Parameters screen, set CHAP Authentication to Enabled. 2. From the Initiator Parameters screen, type values for the following: • CHAP ID (up to 128 bytes) • CHAP Secret (if authentication is required, and must be 12 characters in length or longer) 3. Select ESC to return to the Main menu. 4.
NetXtreme User Guide iSCSI Protocol and Broadcom Boot Agent Software Table 18: DHCP Option 17 Parameter Definition Parameter Definition "iscsi:" A literal string The IP address or FQDN of the iSCSI target ":" Separator The IP protocol used to access the iSCSI target. Currently, only TCP is supported so the protocol is 6. The port number associated with the protocol. The standard port number for iSCSI is 3260.
NetXtreme User Guide iSCSI Protocol and Broadcom Boot Agent Software Note: If using Option 43, you also need to configure Option 60. The value of Option 60 should match the DHCP Vendor ID value. The DHCP Vendor ID value is BRCM ISAN, as shown in General Parameters of the iSCSI Boot Configuration menu.
NetXtreme User Guide iSCSI Protocol and Broadcom Boot Agent Software DHCP iSCSI Boot Configuration for IPv6 The DHCPv6 server can provide a number of options, including stateless or stateful IP configuration, as well s information to the DHCPv6 client. For iSCSI boot, Broadcom adapters support the following DHCP configurations: • DHCPv6 Option 16, Vendor Class Option • DHCPv6 Option 17, Vendor-Specific Information Note: The DHCPv6 standard Root Path option is not yet available.
NetXtreme User Guide iSCSI Protocol and Broadcom Boot Agent Software Preparing the iSCSI Boot Image • Windows Server 2008 R2 and SP2 iSCSI Boot Setup • Windows Server 2012 iSCSI Boot Setup • Linux iSCSI Boot Setup Windows Server 2008 R2 and SP2 iSCSI Boot Setup Windows Server 2008 R2 and Windows Server 2008 SP2 support iSCSI booting. The following procedure references Windows Server 2008 R2 but is common to both the Windows Server 2008 R2 and SP2.
NetXtreme User Guide iSCSI Protocol and Broadcom Boot Agent Software A few minutes after the Windows Server 2008 R2 DVD installation process starts, a system reboot will follow. After the reboot, the Windows Server 2008 R2 installation routine should resume and complete the installation. 13. Following another system restart, check and verify that the remote system is able to boot to the desktop. 14. After Windows Server 2008 R2 is booted up, load the driver and run Bindview.exe. a. Select All Services. b.
NetXtreme User Guide iSCSI Protocol and Broadcom Boot Agent Software 14. After Windows 2012 boots to the OS, Broadcom recommends running the driver installer to complete the Broadcom driver and application installation. Linux iSCSI Boot Setup Linux iSCSI boot is supported on Red Hat Enterprise Linux 6.9, 7.1 and later and SUSE Linux Enterprise Server (SLES) 11 SP4, or SLES 12 SP2 and later. 1. For driver update, obtain the latest Broadcom Linux driver CD. 2.
NetXtreme User Guide iSCSI Protocol and Broadcom Boot Agent Software 20. Install bibt package. 21. Remove ifcfg-eth*. 22. Reboot. 23. For SUSE 11.1, follow the remote DVD installation workaround shown below. 24. After the system reboots, log in, change to the /opt/bcm/bibt folder, and run iscsi_setup.sh script to create the initrd image. 25. Copy the initrd image to the /boot folder. 26. Change the grub menu to point to the new initrd image. 27. To enable CHAP, you need to modify iscsid.
NetXtreme User Guide iSCSI Protocol and Broadcom Boot Agent Software Virtual LANs Virtual LAN (VLAN) tagging is not supported for iSCSI boot with the Microsoft iSCSI Software Initiator.
NetXtreme User Guide iSCSI Protocol and Broadcom Boot Agent Software Broadcom Boot Agent Driver Software • Overview • Setting Up MBA in a Client Environment Overview Broadcom NetXtreme Gigabit Ethernet adapters support Preboot Execution Environment (PXE), Remote Program Load (RPL), iSCSI boot, and Bootstrap Protocol (BootP). Multi-Boot Agent (MBA) is a software module that allows your networked system to boot with the images provided by remote systems across the network.
NetXtreme User Guide iSCSI Protocol and Broadcom Boot Agent Software 3. Use the UP ARROW and DOWN ARROW keys to move to the Boot Protocol menu item. Then use the RIGHT ARROW or LEFT ARROW key to select the boot protocol of choice if other boot protocols besides Preboot Execution Environment (PXE) are available. If available, other boot protocols include Remote Program Load (RPL) and Bootstrap Protocol (BOOTP). Note: For some, but not all, iSCSI boot-capable LOMs, the boot protocol is set via the BIOS.
NetXtreme User Guide iSCSI Protocol and Broadcom Boot Agent Software 5. Use the drop-down menu to select the boot protocol of choice, if boot protocols other than Preboot Execution Environment (PXE) are available. If available, other boot protocols include iSCSI and Bootstrap Protocol (BOOTP). Note: For iSCSI boot-capable LOMs, the boot protocol is set via the BIOS. See your system documentation for more information. 6.
NetXtreme User Guide Manageability Section 10: Manageability • CIM • SNMP CIM The Common Information Model (CIM) is an industry standard defined by the Distributed Management Task Force (DMTF). Microsoft implements CIM on Windows platforms such as Windows Server 2008. Broadcom will support CIM on Windows Server 2008, 2012, and 2012 R2 platforms. Broadcom's implementation of CIM will provide various classes to provide information to users through CIM client applications.
NetXtreme User Guide Manageability SNMP BASP Subagent The BASP subagent, baspmgnt.dll, is designed for Windows Server 2008 SNMP service. It is required to install the SNMP service before installing the BASP subagent. The BASP subagent allows an SNMP manager software to actively monitor the configurations and performance of the Broadcom Advanced Server features. The subagent also provides an alarm trap to an SNMP manager to inform the manager of any changes to the conditions of the BASP component.
NetXtreme User Guide • Duplex mode • Memory range • Interrupt setting • Bus number • Device number • Function number Manageability To monitor this information, a SNMP manager needs to load the Broadcom Extended information MIB file to allow monitoring of the information described above. This file, bcmif.mib, is included on the Broadcom NetXtreme adapter installation CD. The monitored workstation requires the installation of the Broadcom Extended Information SNMP extensibleagent, bcmif.
NetXtreme User Guide Specifications Section 11: Specifications • 10/100/1000BASE-T Cable Specifications • Performance Specifications 10/100/1000BASE-T Cable Specifications Table 21: 10/100/1000BASE-T Cable Specifications Port Type Connector Media Maximum Distance 10BASE-T RJ-45 Category 3, 4, or 5 unshielded twisted pairs (UTP) 100m (328 ft) 100/1000BASE-T1 RJ-45 Category 52 UTP 100m (328 ft) 1 1000BASE-T signaling requires 4 twisted pairs of Category 5 balanced cabling, as specified in I
NetXtreme User Guide Troubleshooting Section 12: Troubleshooting • Hardware Diagnostics • Troubleshooting Checklist • Checking for Network Link and Activity • Checking if Current Drivers Are Loaded • Running a Cable Length Test • Testing Network Connectivity • Broadcom Boot Agent • Broadcom Advanced Server Program (BASP) • Kernel Debugging over Ethernet • Miscellaneous Hardware Diagnostics Loopback diagnostic tests are available for testing the adapter hardware.
NetXtreme User Guide Troubleshooting 4. Power down the machine, remove AC power from the machine, and then reboot the system. 5. Remove and re-install the diagnostic software. 6. Contact your hardware vendor. BACS Network Test Failures Typically, the BACS Testing the Network failures are the result of a configuration problem on the network or with the IP addresses. Below are common steps when troubleshooting the network. 1. Verify that the cable is attached and you have proper link. 2.
NetXtreme User Guide Troubleshooting Troubleshooting Checklist Caution! Before you open the case of your system, review Safety Precautions. The following checklist provides recommended actions to take to resolve problems installing the Broadcom NetXtreme Gigabit Ethernet adapter or running it in your system. • Inspect all cables and connections. Verify that the cable connections at the network adapter and the switch are attached properly.
NetXtreme User Guide Troubleshooting Checking if Current Drivers Are Loaded Windows See Viewing Adapter Information to view useful information about the adapter, its link status, and network connectivity. Linux To verify that the TG3 Linux driver is loaded properly, run: lsmod | grep tg3 If the driver is loaded, a line similar to the one below is displayed, where size is the size of the driver in bytes, and n is the number of adapters configured.
NetXtreme User Guide Troubleshooting Testing Network Connectivity Note: When using forced link speeds, verify that both the adapter and the switch are forced to the same speed, or that both sides are configured for auto-negotiation. Windows Use the ping command to determine if the network connection is working. Note: Network connectivity can also be tested using the Testing the Network feature in Broadcom Advanced Control Suite 2. 1. Verify that the drivers are loaded and enabled. 2.
NetXtreme User Guide Troubleshooting Broadcom Advanced Server Program (BASP) Problem: After physically removing a NIC that was part of a team and then rebooting, the team did not perform as expected. Solution: To physically remove a teamed NIC from a system, you must first delete the NIC from the team. Not doing this before shutting down could result in breaking the team on a subsequent reboot, which may result in unexpected team behavior.
NetXtreme User Guide Regulatory Information Section 13: Regulatory Information • FCC Class B Notice • VCCI Class B Notice • CE Notice • Canadian Regulatory Information (Canada Only) • MIC Notice (Republic of Korea Only) • BSMI FCC Class B Notice Broadcom NetXtreme Gigabit Ethernet Controller BCM95721A211 BCM95722A2202 The equipment complies with Part 15 of the FCC Rules.
NetXtreme User Guide Regulatory Information VCCI Class B Notice The equipment is a Class B product based on the standard of the Voluntary Control Council for Interference from Information Technology Equipment (VCCI). If used near a radio or television receiver in a domestic environment, it may cause radio interference. Install and use the equipment according to the instruction manual.
NetXtreme User Guide Broadcom® April 2019 • 2CS57XX-CDUM515 Regulatory Information Page 168
NetXtreme User Guide Broadcom® April 2019 • 2CS57XX-CDUM515 Regulatory Information Page 169
NetXtreme User Guide Broadcom® April 2019 • 2CS57XX-CDUM515 Regulatory Information Page 170
NetXtreme User Guide Regulatory Information Canadian Regulatory Information (Canada Only) Industry Canada, Class B This Class B digital apparatus complies with Canadian ICES-003. Notice: The Industry Canada regulations provide that changes or modifications not expressly approved by Broadcom could void your authority to operate this equipment. Industry Canada, classe B Cet appareil numérique de la classe B est conforme à la norme NMB-003 du Canada.
NetXtreme User Guide Regulatory Information MIC Notice (Republic of Korea Only) B CLASS Device Broadcom NetXtreme Gigabit Ethernet Controller BCM95721A211 BCM95722A2202 Note that this device has been approved for non-business purposes and may be used in any environment, including residential areas.
NetXtreme User Guide Regulatory Information BSMI Broadcom® April 2019 • 2CS57XX-CDUM515 Page 173
