Mellanox Firmware Tools (MFT) User’s Manual Rev 1.50 www.mellanox.
Rev 1.50 NOTE: THIS HARDWARE, SOFTWARE OR TEST SUITE PRODUCT (“PRODUCT(S)”) AND ITS RELATED DOCUMENTATION ARE PROVIDED BY MELLANOX TECHNOLOGIES “AS-IS” WITH ALL FAULTS OF ANY KIND AND SOLELY FOR THE PURPOSE OF AIDING THE CUSTOMER IN TESTING APPLICATIONS THAT USE THE PRODUCTS IN DESIGNATED SOLUTIONS. THE CUSTOMER'S MANUFACTURING TEST ENVIRONMENT HAS NOT MET THE STANDARDS SET BY MELLANOX TECHNOLOGIES TO FULLY QUALIFY THE PRODUCTO(S) AND/OR THE SYSTEM USING IT.
Rev 1.50 Table of Contents Chapter 1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 1.1 1.2 1.3 1.4 1.5 Chapter 2 mlxburn - FW Image Generator & Burner. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 2.1 2.2 2.3 2.4 2.
Rev 1.50 Appendix A MTUSB-1 USB to I2C Adapter A.1 Overview A.2 Hardware Installation A.
Rev 1.50 Revision History Printed on December 29, 2011. Table 1 - Revision History Table Date Revision Description December 2011 1.50 Added the following note “This step in not required in Windows.” to the following sections: • Section 1.3, “MFT Access to Hardware Devices,” on page 8 • Section C.1.2, “Operation,” on page 43 • Section C.2.1, “Operation,” on page 45 • Section C.3.1, “Operation,” on page 46 • Section C.4.1, “Operation,” on page 47 • Section E.
Rev 1.50 Table 1 - Revision History Table Date Revision Description August 2008 1.0 • • • • Added support for Mellanox InfiniScale IV switch device Expanded In-Band support with the command ‘mst ib add’ Modified the Windows MFT installation (now it is a standalone installation) Added the -qq flag to flint April 2008 0.65 • • • • Added MFT installation instruction in Section 1.
Rev 1.50 1 Introduction The Mellanox Firmware Tools (MFT) package is a set of firmware management tools for a single InfiniBand node. MFT can be used for: y Generating a standard or customized Mellanox firmware image y Querying for firmware information y Burning a firmware image to a single Mellanox device The following is a list of the available tools in the package, together with a brief description of what each tool performs.
Rev 1.50 Introduction Figure 1: Mellanox Firmware Tools – A Scheme of Operation 1.1 Supported Operating Systems MFT distributions are available for the following operating systems: Linux, Windows, and VMware® ESX Server 3.5. Please refer to the release notes of your version for supported platforms and kernels. Unless explicitly specified, the usage of the tools is identical for all operating systems.
Rev 1.50 1.2 Software Prerequisites 1.2.1 On Linux Table 2 - MFT Software Dependencies on Linux Software Package Required Version Kernel sources Machine’s kernel version OFED / MLNX_OFED1, 2 1.1 or later Perl 5.6 or later zlib 1.1.4 or later 1. OFED can be downloaded from http://www.openfabrics.org. Note that installing OFED is not required if you wish to install MFT without In-Band capabilities. 2.
Rev 1.50 1.3 Introduction MFT Access to Hardware Devices Table 5 lists the Mellanox devices supported by MFT, the supporting tools, and the access methods to these devices.
Rev 1.50 To list the available mst device names on the local machine, run ‘mst status’ (on Linux and Windows only). Notes for Windows: y The mst service, which provides PCI access to the target adapter, is run automatically upon boot. y To access an adapter or switch device via the USB bus: A USB to I2C Adapter should be used to connect the host USB port and the I2C port of the target device.
Rev 1.50 Introduction See Appendix D,“In-Band Access to Multiple IB Subnets,” on page 52 for details on how to access devices in multiple IB subnets. y Via PCI user level: The mst device name obtained in this method can only be used with the mlxburn tool with the ‘-ul’ flag. For example, if you run lspci -d 15b3: Mellanox devices and PCI Device IDs will be displayed. > /sbin/lspci -d 15b3: 02:00.0 Ethernet controller: Mellanox Technologies Unknown device 6368 (rev a0) 1.
Rev 1.50 1.4.4 Uninstall MFT (Windows) To uninstall WinMFT, perform one of the following: 1. Activate the ‘Add or Remove Programs’ utility of Windows and click the ‘Remove’ button of the MFT tool entry. 2. Click start > Programs > Mellanox > WinMFT > uninstall. 1.4.5 Install MFT On VMware ESX Server 3.5 1. Download the mstflint for VMware ESX Server 3.5 RPM from the Mellanox Management Tools webpage: http://www.mellanox.com/products/management_tools.php 2. Download the MFT for VMware ESX Server 3.
Rev 1.50 mlxburn - FW Image Generator & Burner 2 mlxburn - FW Image Generator & Burner 2.1 Overview mlxburn is a tool for firmware (FW) image generation and/or for burning a firmware image to the Flash/EEPROM attached to a Mellanox device. Both functions or a single function of mlxburn can be activated by means of command line options (see Section 2.3, “mlxburn Synopsis”). It can also query for firmware attributes (e.g., firmware version, GUIDs, etc.) and VPD info of adapter cards and switch systems.
Rev 1.50 mlxburn runs both stages by default, but it may perform only one by means of command options. If the ‘-wrimage’ is specified (see Section 2.3, “mlxburn Synopsis”), only image generation is performed. Specifying the ‘-image’ option skips the image generation stage and loads the provided image (generated in a previous run of mlxburn using the ‘-wrimage’ option). When generating an image file for a Mellanox InfiniScale III switch device, the produced image file name must end with a .
Rev 1.50 mlxburn - FW Image Generator & Burner -wrimage - Write the generated binary image to the provided file name -fw - Specify the Mellanox firmware file to use (file extension is .mlx) -format - Specify which image format to use. Can be used only with the -wrimage flag. Default is BINARY. -image - Use the given firmware image to burn (file extension is .bin or .img) -conf - Firmware configuration file (.
Rev 1.50 V2:BIN = 1234abcd White spaces before and after VALUE are trimmed. -vpd_set_keyword 1 - (on Linux only): Add or change a keyword value in the VPD-W tag (the writable section of the VPD) with the data given in the keywordassignment string. The string format is identical to a line in the rw-keywords-file described above.
Rev 1.50 mlxburn - FW Image Generator & Burner 48437 - For MT48436 InfiniScale IV based switch systems (IB @ DDR) 48438 - For MT48436 InfiniScale IV based switch systems (IB @ QDR) 64102, 64122 - For MT64102 BridgeX based gateway systems 1016 - For MT1016 PhyX devices 51000 - For MT58100 SwitchX based switch systems 4099 - For MT27500 ConnectX-3 2.3.1 Additional mlxburn Options The following is a list of additional options.
Rev 1.50 y Generate an MT48436 image and perform an In-Band update of the device with LID 0x18: mlxburn -dev lid-0x000E -fw ./fw-sx.mlx y Generate and burn a new MSX6025 via I2C: Set the I2C network to access the InfiniScaleIV switch. mlxi2c -d /dev/mst/mtusb-1 p /SX y Burn the new image (the flash is still blank ) specifying the Node GUID, system GUID, base MAC and Switch MAC. Note that 4 guids (in quotes) should be specified as an argument to the -guids flag.
Rev 1.50 mlxburn - FW Image Generator & Burner y Generate and Burn a new MT3600 switch system via I2C in 2 steps: a. Generate the image: mlxburn -fw ./fw-IS4.mlx -conf ./MTS3600Q-1UNC_A1.ini -wrimage ./fw-is4.bin b. Burn using flint tool: flint -d /dev/mst/mtusb-1 -i /tmp/fw-is4.bin -nofs -guids 0002c9000100d060 0 0 0002c9000100d060 b y To generate an MT48436 image and perform an In-Band update of the device with LID 0x18, enter: mlxburn -fw ./fw-IS4.mlx -dev lid-0x18 2.4.
Rev 1.50 y To update firmware on an MT43132 InfiniScale device in a switch system such as Flextronics’ F-X430066 Stallion 8 4X IB port switch, enter: mlxburn -image Stallion_5_5_0.eeprom -dev /dev/mst/mtusb-1 -dev_type 43132 This firmware update cannot be performed before initializing the MST device (mtusb1) to connect to the I2C-compatible bus of the InfiniScale and its EEPROM. y To generate an MT47396 image and perform an In-Band update of the device with LID 0x11, enter: mlxburn -dev lid-0x11 -fw .
Rev 1.50 flint – Firmware Burning Tools 3 flint – Firmware Burning Tools 3.1 Overview The flint (Flash interface) utility performs the following functions: y Burns a binary firmware image to the Flash device attached to an adapter, bridge or switch device y Burns an Expansion ROM image to the Flash device attached to a ConnectX / Con- nectX-2 adapter device y Queries for firmware attributes (version, GUIDs, UIDs, MACs, PSID, etc.
Rev 1.50 -mac 1 - MAC base address value. Each of the two ConnectX EN ports is assigned a MAC address as follows: MAC is assigned to Port 1 MAC+1 is assigned to Port 2 Affected commands: burn sg -macs 1 - 2 MAC addresses must be specified here—one for each ConnectX EN port. The first MAC address is assigned to Port 1 and the second MAC address to Port 2. Affected commands: burn sg -uid - BridgeX only. Derive and set the device UIDs (GUIDs, MACs, WWNs).
Rev 1.50 flint – Firmware Burning Tools -striped_image - Use this flag to indicate that the given image file is in a "striped image" format. See “mlxburn Synopsis” -striped_image flag for details. Affected commands: query verify -h[elp] - Print this message and exit. -hh - Print extended command help. -nofs - Do not burn image in failsafe mode. -allow_psid_change - Allow burning a FW image with a different PSID (Parameter Set ID)than the one currently on flash.
Rev 1.50 num_of_flashes: The number of the flashes connected to the device -v - Print version information 3.2.2 Command Descriptions The flint utility commands are: Common FW Update and Query: b[urn] - Burn flash q[uery] [full] - Query misc. flash/firmware characteristics, use "full" to get more information. v[erify] - Verify entire flash swreset - SW reset the target un-managed switch device. This command is supported only in the In-Band access method.
Rev 1.50 flint – Firmware Burning Tools y Section 3.2.2.1,“Burning a FW Image,” on page 24 y Section 3.2.2.2,“Querying the FW Image,” on page 26 y Section 3.2.2.3,“Verifying the FW Image,” on page 26 y Section 3.2.2.4,“Managing an Expansion ROM Image,” on page 26 3.2.2.1 Burning a FW Image The FLINT utility enables you to burn the Flash from a binary image.
Rev 1.50 Examples: 1. Update the FW on the device, keeping the current GUIDs and VSD. (Note: This is the common way to use flint.) > flint -d /dev/mst/mt25418_pci_cr0 -i fw-25408-2_1_000-MHGH28-XSC_A1.bin burn 2. Update the FW on the device, specifying the GUIDs to burn. > flint -d /dev/mst/mt25418_pci_cr0 -i 25408-2_1_000-MHGH28-XSC_A1.bin-guid 12345678deadbeef burn 3. Update the FW on the device, specifying the MACs to burn. > flint -d /dev/mst/mt25448_pci_cr0 -i fw-25448-6_0_111-MNEH28-XTC_A1.
Rev 1.50 3.2.2.2 flint – Firmware Burning Tools Querying the FW Image y To query the FW image on a device, use the following command line: flint -d q y To query the FW image in a file, use the following command line: flint -i q where: device – Device on which the query is run. image file – Image file on which the query is run. Examples: a. Query the FW on the device. > flint -d /dev/mst/mt25418_pciconf0 query b. Query the FW image file. > flint -i 25408-2_1_000-MHGH28-XSC_A1.
Rev 1.50 Example: # flint -d /dev/mst/mt25418_pci_cr0 brom example.rom Current ROM info on flash: N/A New ROM info: type=GPXE version=2.0.100 devid=25418 proto=IB Burning ROM image - OK Restoring signature - OK # 2. To read an Expansion ROM image to a file, use the following command: flint -d rrom .rom Example: # flint -d /dev/mst/mt25418_pci_cr0 rrom example.rom # flint -d /dev/mst/mt25418_pci_cr0 q Image type: ConnectX FW Version: 2.6.1410 Rom Info: type=GPXE version=2.0.
Rev 1.50 flint – Firmware Burning Tools On pre-ConnectX Devices On pre-ConnectX devices, the ‘sg’ command is applicable only for images with blank (0xff) GUID values and a blank CRC. In other words, the firmware image was generated using the ‘-blank_guids’ flag. The following is an example of a complete flow, where a blank-GUIDs image is first generated and burnt to the Flash. Then the GUIDs are set using the ‘sg’ command. 1. Generate the image with blank guids. > mlxburn -fw ./fw-25408-rel.mlx -c .
Rev 1.50 as GUIDS,VSD) are taken from current image on flash. Burn process will not be failsafe. No checks will be performed. ALL flash, including the Invariant Sector will be overwritten. If this process fails, computer may remain in an inoperable state. Do you want to continue ? (y/n) [n] : y 100% 4. Set the GUIDs and MACs using the ‘flint sg’ command. > flint -d /dev/mst/mt25418_pci_cr0 -guid 0x0002c90001777050 -mac 0x0002c9777051 sg 5.
Rev 1.50 flint – Firmware Burning Tools Device ID: 25418 Chip Revision: A0 Description: Node GUIDs: 0002c9000120d050 0002c9000120d051 0002c9000120d052 0002c9000120d053 Port1 MACs: 02c90120d050 Board ID: VSD (MT_04C0110002) VSD: VSD PSID: MT_04C0110002 Port2 Sys image 02c90120d051 > flint -d /dev/mst/mt25418_pci_cr0 -guid 0002c9000abcdef0 -mac 02c90abcdef0 sg -W- GUIDs are already set, re-burining image with the new GUIDs ...
Rev 1.50 -W- Running quick query - Skipping full image integrity checks. Image type: ConnectX FW Version: 2.7.9450 Rom Info: type=PXE version=3.0.0 devid=25418 proto=VPI Device ID: 25418 Chip Revision: A0 Description: Node GUIDs: 0002c9000120d050 0002c9000120d051 0002c9000120d052 0002c9000120d053 Port1 MACs: 02c90120d050 Board ID: VSD (MT_04C0110002) VSD: VSD PSID: MT_04C0110002 Port2 Sys image 02c90120d051 > flint -i /tmp/image.
Rev 1.50 flint – Firmware Burning Tools PSID: 3.2.3.2 MT_04C0110002 Preparing a Binary Firmware Image for Pre-assembly Burning In some cases, OEMs may prefer to pre-burn the flash before it is assembled on board. To generate an image for pre-burning for 4th generation devices (ConnectX® and newer), use the mlxburn "striped_image" flag. The "striped image" file layout is identical to the image layout on the flash, hence making it suitable for burning verbatim.
Rev 1.50 MACs: ffffffffffffffff Board ID: n/a (MT_0D80110009) VSD: n/a PSID: MT_0D80110009 ffffffffffffffff -W- GUIDs/MACs values and their CRC are not set. 4. Set the correct GUIDs. Since the image is with blank GUIDs, this operation takes less than 1 second > flint -d /dev/mst/mt26428_pci_cr0 -guid 0x0002c9030abcdef0 -mac 0x0002c9bcdef1 sg 5. Query the image on flash to verify that the GUIDs are set correctly. > flint -d /dev/mst/mt26428_pci_cr0 q Image type: ConnectX FW Version: 2.8.
Rev 1.50 flint – Firmware Burning Tools 3. (Optional) Query the image to verify the GUIDs are set. The “-striped_image” flag must be specified when querying a striped image. > flint -i ./fw-ConnectX2-rel.bin -striped_image q Image type: ConnectX FW Version: 2.8.
Rev 1.50 3.2.3.4 VSD: VSD PSID: MT_04C0110002 Disabling/enabling Access to the Hardware The secure host feature enables ConnectX® family devices to block access to its internal hardware registers. The hardware access in this mode is allowed only if a correct 64 bits key is provided Examples: 1. Set the key: flint -d /dev/mst/mt26428_pci_cr0 set_key 22062011 Setting the HW Key - OK Restoring signature - OK A driver restart is required to activate the new key 2.
Rev 1.50 3.2.3.6 flint – Firmware Burning Tools Writing a Dword to Flash To write one dword to Flash memory, use the following command line: flint -d ww addr data where: device is the device the dword is written to. addr is the address of the word to write. data is the value of the word. Example: flint -d /dev/mst/mt23108_pci_conf01 ww 0x10008 0x5a445a44 3.2.3.
Rev 1.50 4 spark - InfiniScale® III Firmware Burning Tool 4.1 Overview The spark tool enables burning a binary firmware image to the EEPROM device attached to an InfiniScale (MT43132) or InfiniScale III (MT47396) switch device via a direct I2C connection or via vendor-specific MADs over the InfiniBand fabric. This tool is described in Section 4.2. 4.2 spark 4.2.1 spark Synopsis spark [switches...] [parameters...
Rev 1.50 spark - InfiniScale® III Firmware Burning Tool -se_i2c - Provides the I2C address of the secondary EEPROM. By default, this address is read from the Mellanox switch device. Use this flag only if the switch device is not accessible. -h[elp] - Prints this help message and exits. -hh - Prints an extended command help -nofs - Do not burn the firmware image in failsafe mode. -s[ilent] - Print errors only.
Rev 1.50 Example: spark -d /dev/mst/mtusb-1 rb 0x56 0x0 0x1000 out.img swreset - SW-resets the target switch device. In response to this command, the target switch device: continues to transport packets to their destinations ignores management packets that are destined to the target switch device itself resets itself after about 15 seconds from receiving the command. The 15second delay is intended to allow for spark to reset other switches in the fabric.
Rev 1.50 spark - InfiniScale® III Firmware Burning Tool Appendix A: PSID Assignment In some cases, OEMs or board manufacturers may wish to use a specific FW configuration not supplied by Mellanox. After setting the new FW parameters in an INI file, the user should assign a unique PSID (Parameter Set ID) to this new configuration. The PSID is kept as part of the FW image on the device NVMEM. The firmware burning tools use this field to retain FW settings while updating FW versions.
Rev 1.50 Appendix B: Flow Examples - mlxburn To update an MT47396 InfiniScale® III and MT48436 InfiniScale® IV switch devices having a specific GUID (for example, 0x00000006660abcd0) or LID, the following are the recommended steps to update the device firmware. For Linux device names should be listed with the /dev/mst prefix. For Windows, no prefix is required. 1. Make sure all subnet ports are in the active state. One way to check this is to run opensm, the Subnet Manager. [root@mymach]> /etc/init.
Rev 1.50 spark - InfiniScale® III Firmware Burning Tool /dev/mst/SW_MT48438_lid-0x0003 [root@mymach]> b. Using the ibnetdiscover tool, run: [root@mymach]# ibnetdiscover | grep 00000006660abcd0 | grep -w Switch Switch 24 "S-00000006660abcd0" 17 lmc 0 "MT47396 Infiniscale-III Mellanox Technologies" base port 0 lid The resulting LID is given as a decimal number. 1. Run mlxburn with the LID retrieved in step #3 above to perform the In-Band burning operation.
Rev 1.50 Appendix C: Debug Utilities C.1 itrace Utility C.1.1 Overview The itrace utility extracts and prints trace messages generated by the embedded iRISC processors of Mellanox Technologies’ InfiniBand or Ethernet adapter devices. These trace messages inform developers of software drivers about internal status, events, critical errors, etc., for each iRISC. Trace messages generated by iRISCs are stored in the trace buffer. The trace buffer is located in host memory for MemFree adapter cards (i.e.
Rev 1.50 spark - InfiniScale® III Firmware Burning Tool To enable generating trace messages for an iRISC, the trace_mask register must be set according to the specifications in the device’s Programmer’s Reference Manual. Setting or clearing bits of the trace_mask register enables or disables, respectively, the generation of specific types of trace messages. The TRACE_MASK parameter must be a hexadecimal or decimal number and its value will be written into the trace_mask register.
Rev 1.50 (0000002d ddaef0d5) SCHD: exes_scheduler: try to insert (0000002e ddaef2d9) SCHD: SQP:0x000000 exes_scheduler chosen (0000002f ddaef6aa) SCHD: EXES_GO(0x0) . . . -w, --wait - Runs itrace in wait mode. itrace will exit only if you press . This is not the default behavior of itrace. Without the -w option, itrace will exit if there have been no new traces in the last 0.5 seconds. -d, --d=DEVICE - Specifies the name of the MST device driver for accessing the cr-space.
Rev 1.50 spark - InfiniScale® III Firmware Burning Tool 2. Enter an mstdump command that complies with the following command syntax: mstdump [-full] > where the -full flag dumps all internal registers. On BridgeX devices, using the -full flag may have undesired side-effects and require resetting the device. Example: [root@mymach]# mstdump /dev/mst/mt25408_pci_cr0 > mt25408.dmp This dumps the internal configuration data of the device into the file mt25408.dmp. C.
Rev 1.50 set_gpio - Set the output value of the given GPIO pin. set_fan - Set the power percent of the given fan. set_power - Power on/off the specified module. Supported modules: MTS3610_LEAF, MTS3610_SPINE. set_led - Change the color of the specify led to the given color.
Rev 1.50 spark - InfiniScale® III Firmware Burning Tool 2. Run i2c with the following command line syntax: i2c [OPTIONS] [] where [OPTIONS] can be the following: -h – Prints this message. -a – Sets address width (in bytes) to the specified value. May be 0, 1, 2 or 4. Default: 1. -d – Sets data width (in bytes) to the specified value. May be 1, 2 or 4. Default is 1.
Rev 1.50 C.4.2 Exit Return Values The following exit values are returned: y 0 - successful completion y >0 - an error occurred C.5 mget_temp Utility The mget_temp utility reads the hardware temperature from Mellanox Technologies devices with temperature sensors (ConnectX®, ConnectX®-2, ConnectX®-3, BridgeX devices, and 4th generation switches), and prints the reading in Celsius degrees. C.5.1 Operation The MST driver must be started prior to running mget_temp. To start mget_temp: 1.
Rev 1.50 spark - InfiniScale® III Firmware Burning Tool Appendix D: In-Band Access to Multiple IB Subnets In most cases, an adapter is connected to a single InfiniBand subnet. The LIDs (Infiniband Local IDs) on this subnet are unique. In this state, the device access MADs are sent (to the target LID) from the first active port on the first adapter on the machine. In case that the different IB ports are connected to different IB subnets, source IB port on the local host should be specified explicitly.
Rev 1.50 Running "ibv_devinfo" command on H1 would list the 2 adapter names. For ConnectX adapters, the names would be mlx4_0 and mlx4_1. Running "mst ib add" would add ib devices from the default port (first active port on the first adapter) - only Subnet1 nodes would be listed. To add the nodes of the second subnet, the source adapter and port should be specified to the "mst ib add" command in the following format: mst ib add Example: 1.
Rev 1.50 spark - InfiniScale® III Firmware Burning Tool Appendix E: MTUSB-1 USB to I2C Adapter E.1 Overview The MTUSB-1 is a USB to I2C bus adapter. This chapter provides the user with hardware and software installation instructions on machines running Linux or Windows operating systems. Figure 3: MTUSB-1 Device MTUSB-1 Device I2C Cable USB Cable E.1.1 Package Contents Please make sure that your package contains the items listed in Table 7 and that they are in good condition.
Rev 1.50 E.2 Hardware Installation To install the MTUSB-1 hardware, please execute the following steps in the exact order: 1. Connect one end of the USB cable to the MTUSB-1 and the other end to the PC. 2. Connect one end of the I2C cable to the MTUSB-1 and the other end to the system/board you wish to control via the I2C interface.
