TROUBLESHOOTING GUIDE Brocade Fabric OS Troubleshooting and Diagnostics Guide Supporting Fabric OS 8.0.
© 2016, Brocade Communications Systems, Inc. All Rights Reserved. Brocade, Brocade Assurance, the B-wing symbol, ClearLink, DCX, Fabric OS, HyperEdge, ICX, MLX, MyBrocade, OpenScript, VCS, VDX, Vplane, and Vyatta are registered trademarks, and Fabric Vision is a trademark of Brocade Communications Systems, Inc., in the United States and/or in other countries. Other brands, products, or service names mentioned may be trademarks of others.
Contents Preface...........................................................................................................................................................................................................................................................................................7 Document conventions...............................................................................................................................................................................................
Port initialization and FCP auto-discovery process...............................................................................................................................................................................35 Link issues.....................................................................................................................................................................................................................................................................
FIPS ................................................................................................................................................................................................................................................................................. 66 Virtual Fabrics.......................................................................................................................................................................................................................
Checking the status of the fans............................................................................................................................................................................................................... 94 Checking the status of a power supply............................................................................................................................................................................................... 94 Checking temperature status...........
Preface ∙ ∙ ∙ ∙ Document conventions..................................................................................................................................................................................................... 7 Brocade resources...............................................................................................................................................................................................................8 Contacting Brocade Technical Support.............
Preface Convention Description ... Repeat the previous element, for example, member[member...]. \ Indicates a “soft” line break in command examples. If a backslash separates two lines of a command input, enter the entire command at the prompt without the backslash. Notes, cautions, and warnings Notes, cautions, and warning statements may be used in this document. They are listed in the order of increasing severity of potential hazards.
Preface If you have purchased Brocade product support directly from Brocade, use one of the following methods to contact the Brocade Technical Assistance Center 24x7. Online Telephone E-mail Preferred method of contact for non-urgent issues: Required for Sev 1-Critical and Sev 2-High issues: support@brocade.
Preface 10 Brocade Fabric OS Troubleshooting and Diagnostics Guide 53-1004126-01
About This Document ∙ ∙ Supported hardware and software..............................................................................................................................................................................11 What's new in this document........................................................................................................................................................................................
About This Document Brocade Gen 6 Directors ∙ Brocade X6-4 Director ∙ Brocade X6-8 Director Fabric OS support for the Brocade Analytics Monitoring Platform (AMP) device depends on the specific version of the software running on that platform. Refer to the AMP Release Notes and documentation for more information. What's new in this document The following changes are made for the 8.0.1 release: 12 ∙ The document is updated for 32 Gbps port speed, Gen 6 devices, and Gen 6 blades support.
Introduction ∙ ∙ ∙ ∙ ∙ Troubleshooting overview.............................................................................................................................................................................................. 13 Most common problem areas.....................................................................................................................................................................................13 Questions for common symptoms................................
Introduction TABLE 1 Common troubleshooting problems and tools (continued) Problem area Hosts Storage Management Applications Investigate ∙ Physical issues between switch and devices ∙ Lower or incompatible HBA firmware ∙ ∙ Tools ∙ Device LEDs ∙ Host operating system diagnostic tools Incorrect device driver installation ∙ Device driver diagnostic tools Incorrect device driver configuration ∙ ∙ Incorrect installation and configuration of the storage devices that the software references.
Introduction TABLE 2 Common symptoms (continued) Symptom Areas to check Chapter or Document Fabric segments Licensing General Troubleshooting on page 23 Zoning Connectivity on page 35 Virtual Fabrics Virtual Fabrics on page 67 Fabric parameters Introduction on page 13 Introduction on page 13 Fabric merge failed FCIP tunnel bounces FCIP tunnel, including the network between FCIP tunnel endpoints Fabric OS FCIP Administrator's Guide FCIP tunnel does not come online FCIP tunnel, including the
Introduction TABLE 2 Common symptoms (continued) Symptom Areas to check Chapter or Document Zoning, if applicable Incompatible firmware versions No light on LEDs Links Connectivity on page 35 Performance problems Links Connectivity on page 35 FCR LSAN tags General Troubleshooting on page 23 FCIP tunnels Fabric OS FCIP Administrator's Guide Port cannot be moved Virtual Fabrics Virtual Fabrics on page 67 SCSI retry errors Buffer credits Fabric OS FCIP Administrator's Guide FCIP tunnel band
Introduction TABLE 2 Common symptoms (continued) Symptom Areas to check Chapter or Document Zone content mismatch Effective configuration Zoning on page 75 Zone type mismatch Effective configuration Zoning on page 75 Gathering information for your switch support provider If you are troubleshooting a production system, you must gather data quickly. As soon as a problem is observed, perform the following tasks.
Introduction NOTE The -c option works only if you used the supportftp -s command to set up the FTP server. When invoked without operands, this command goes into interactive mode. The following operands are optional: -n —Does not prompt for confirmation. This operand is optional; if omitted, you are prompted for confirmation. -c —Uses the FTP parameters saved by the supportFtp command. This operand is optional; if omitted, specify the FTP parameters through command line options or interactively.
Introduction 4. Enter the command or start the process to capture the required data on the console. Capturing command output 1. Connect to the switch through a Telnet or SSH utility. 2. Log in using an account with admin permissions. 3. Set the Telnet or SSH utility to capture output from the screen. Some Telnet or SSH utilities require this step to be performed prior to opening up a session. Check with your Telnet or SSH utility vendor for instructions. 4.
Introduction Use the portShow iproute geX command to determine the answer. ∙ Are you using NPIV? Yes or no. Use the switchShow command to determine the answer. ∙ Are there security policies turned on in the fabric? If so, what are they? Gather the output from the following commands: – – – – – – secPolicyShow fddCfg --showall ipFilter --show authUtil --show secAuthSecret --show fipsCfg --showall 5. Is the fabric redundant? If yes, what is the MPIO software? (List vendor and version.) 6.
Introduction – – ∙ Storage information: – – – – – – ∙ HBA driver version Configuration settings Disk/tape type Disk/tape firmware level Controller type Controller firmware level Configuration settings Storage software (such as EMC Control Center, Veritas SPC, and so on.) If this is a Brocade DCX 8510 or X6 family enterprise-class platform, are the CPs in-sync? Yes or no. Use the haShow command to determine the answer.
Introduction 22 Brocade Fabric OS Troubleshooting and Diagnostics Guide 53-1004126-01
General Troubleshooting ∙ ∙ ∙ ∙ ∙ ∙ Licenses..................................................................................................................................................................................................................................23 Time .........................................................................................................................................................................................................................................
General Troubleshooting Symptom Frames are being dropped. Probable cause and recommended action Frames are timing out. Viewing frames ∙ Connect to the switch and log in using an account with admin permissions. ∙ Enter the framelog --show command. Switch message logs Switch message logs (RAS logs) contain information on events that happen on the switch or in the fabric. This is an effective tool in understanding what is going on in your fabric or on your switch.
General Troubleshooting Symptom I2C bus errors are appearing in the switch log. Probable cause and recommended action I2C bus errors generally indicate defective hardware or poorly seated devices or blades; the specific item is listed in the error message. Refer to the Fabric OS Message Reference for information specific to the error that was received. Some Chip-Port (CPT) and Environmental Monitor (EM) messages contain I2C-related information.
General Troubleshooting ∙ haFailover ∙ fastBoot ∙ firmwareDownload The RRD feature is activated and halts rebooting when an unexpected reboot reason is shown continuously in the reboot history within a certain period of time. For example, the switch re-boots five times in 300 seconds. The period of time depends on the switch.
General Troubleshooting Collecting limited supportSave output on the Rolling Reboot Detection 1. Log in to the switch on the admin account. A user account with admin privileges is not able to collect limited supportSave output. 2. After you see the message in the following example, press Enter . 3. Enter the supportSave command to go into interactive mode. Alternatively, if using a USB device, enter usbstorage -e . 4. Respond to the prompts. 5.
General Troubleshooting ∙ Generates an ELS ECHO request to the destination port specified and validates the response.
General Troubleshooting Superping Superping refers to the fcPing --allpaths command, which is a diagnostic tool used to test all least-cost ISLs between a source and destination switch. When you run the command, you are provided with a list of all available least-cost paths from a source domain to a destination device. Superping isolates links with potential failures so that you can investigate these ISLs to determine the exact links.
General Troubleshooting FIGURE 1 Superping and fcPing paths In the following example, Superping is invoked using the fcPing --allpaths command to destination domain 165. The following example displays each hop in (Domain1/Index1-> Domain2/Index2) format. To reach destination domain 165 from source domain 3 there are two unique end-to-end paths. In the first path, the frame traverses from egress port index 205 on source domain 3 to ingress port index 25 on domain 207.
General Troubleshooting NOTE Superping provides an indication if all ISLs are covered. If all the ISLs are not covered, you can increase the coverage count and maximum retries to transmit, so that complete coverage of all ISLs is achieved. Consider the following example in which a few errors are recorded on ISLs 3/205-->2/25, 3/204-->2/27, 2/42-->101/3, and 2/1->101/8. The potential faulty link is internal port 0/284 on domain 2 with the maximum of 100 percent failure.
General Troubleshooting Routing and statistical information The pathInfo command displays routing and statistical information from a source port index on the local switch to a destination port index on another switch. This routing information describes the full path that a data stream travels between these ports, including all intermediate switches. ATTENTION Using the pathInfo command when exchange-based routing is turned on can provide different paths with each attempt.
General Troubleshooting 8 24 9 3 10 8 11 6 (output truncated) 4 3 2 1 (sw0) (sw0) (sw0) (sw0) 2 4 23 4G 4G 4G 500 10000 2 500 - - For details about the pathInfo command, refer to the Fabric OS Command Reference. Performance issues Symptom General slow-down in FCR performance and scalability. Probable cause and recommended action As LSAN zone databases get bigger, it takes more switch resources to process them.
General Troubleshooting 34 Brocade Fabric OS Troubleshooting and Diagnostics Guide 53-1004126-01
Connectivity ∙ ∙ ∙ ∙ ∙ ∙ ∙ ∙ ∙ Port initialization and FCP auto-discovery process...................................................................................................................................... 35 Link issues.............................................................................................................................................................................................................................36 Connection problems......................................
Connectivity FIGURE 2 Simple port initialization process The FCP auto-discovery process enables private storage devices that accept the process login (PRLI) to communicate in a fabric. If device probing is enabled, the embedded port commences with port login (PLOGI) and attempts a PRLI into the device to retrieve information to enter into the Name Server.
Connectivity Symptom Port LEDs are steady. Probable cause and recommended action The color of the port LED is important in this instance. To determine what is happening on either your port status LED or power status LED, refer to that switch’s model hardware reference manual. There is a table that describes the purpose of the LEDs and explains the current behavior as well as suggested resolutions. Symptom No light from the port LEDs.
Connectivity ∙ If the missing device is not logically connected, check the device and everything on that side of the data path. Also refer to Link failures on page 39 for additional information. Checking the path includes verifying the following for the host: – – – – – – – – – The host OS is configured correctly. The third-party vendor multi-pathing input/output (MPIO) software, if it is being used, is configured correctly.
Connectivity 2. Look for the device in the Name Server (NS) list, which lists the nodes connected to that switch. This allows you to determine if a particular node is accessible on the network. ∙ If the device is not present in the NS list, the problem is between the device and the switch. There may be a time-out communication problem between edge devices and the name server, or there may be a login issue.
Connectivity If these fields do not appear, proceed to Step 6. 6. If the fields in Step 5 do not appear, correct the negotiation by entering the portCfgSpeed command with the speed_level option. switch:admin> portcfgspeed Usage: portCfgSpeed PortNumber Speed_Level Speed_Level: 0 - Auto Negotiate 4 - 4Gbps 8 - 8Gbps 10 - 10bps 16 - 16Gbps 32 - 32Gbps ax - Auto Negotiate + enhanced retries Checking for a loop initialization failure 1. Verify the port is an L_Port. a) Enter the switchShow command.
Connectivity 3. Verify the event area for the port state entry is pstate. The command entry AC indicates that the port has completed point-topoint initialization. switch:admin> portlogdumpport 4 time task event port cmd args ------------------------------------------------11:38:21.726 INTR pstate 4 AC 4. Skip over the loop initialization phase. After becoming an active port, the port becomes an F_Port or an E_Port depending on the device on the opposite side.
Connectivity The portLoopbackTest command is used to verify the functional operation of a path on a switch. This test sends frames from a given port’s transmitter and loops them back into the same port’s receiver. The loopback is done at the parallel loopback path. The path traversed in this test does not include the media or the fiber cable. Only one frame is transmitted and received at any given time. An external cable is not required to run this test.
Connectivity 0: 665k 7.0k 0 1: 0 0 0 2: 0 0 0 3: 0 0 0 4: 0 0 0 5: 0 0 0 6: 0 0 0 7: 0 0 0 8: 78 60 0 9: 12 4 0 10: 0 0 0 11: 0 0 0 12: 0 0 0 13: 0 0 0 14: 0 0 0 15: 0 0 0 16: 665k 7.4k 0 (output truncated) 4.
Connectivity FC Router: OFF FC Router BB Fabric ID: 1 Area Port Media Speed State Proto ===================================== 0 0 -N32 No_Module 1 1 -N16 Online 2 2 -N32 No_Module 3 3 -N32 No_Module (output truncated) 61 61 -N32 No_Module 62 62 -N32 No_Module 63 63 -N32 No_Module 64 64 id N32 Online 65 65 -N32 No_Module 66 66 -N32 No_Module 67 67 id AN No_Sync 68 68 id N16 Online 69 69 -N32 No_Module 70 70 -N32 No_Module 71 71 id N16 Online 72 72 -N32 No_Module 73 73 -N32 No_Module 74 74 -N32 No_Module 75
Connectivity 5.
Connectivity ∙ More than 126 logins attempted via an F_Port configured for default number of logins (126). ∙ portTrunkArea configured on edge switch but trunking license is missing on Access Gateway. ∙ Link level issues between edge switch and access gateway ∙ Other access gateway policy specific features like port grouping, advance device security To resolve this issue, follow these steps: 1. Identify all the affected F_Ports with duplicate ALPA entries and do the following for each port. 2.
Connectivity Testing components to and from the HBA 1. Connect to the switch and log in as admin. 2. Enter the portTest command (refer to the Fabric OS Command Reference for information on the command options). Refer to Table 7 for a list of additional tests that can be used to determine the switch components that are not functioning properly. Refer to the Fabric OS Command Reference for additional command information.
Connectivity TABLE 8 Segmented fabric scenarios (continued) Fabric Parameter LISL XISL ISL (Fabric OS 7.3.0 or later) ISL (Fabric OS 7.1.
Connectivity Repeat the following procedure until all domain ID conflicts are resolved. 1. Enter the fabricShow command on a switch from one of the fabrics. 2. In a separate Telnet window, enter the fabricShow command on a switch from the second fabric. 3. Compare the fabricShow output from the two fabrics. Note the number of domain ID conflicts; there may be several duplicate domain IDs that must be changed.
Connectivity Reconciling incompatible software features Earlier releases of software may not be supported in new versions of Fabric OS code. This may be due to a software feature changing or new services being supported. If you suspect that you are trying to introduce a switch into a fabric that has an older version of code, check the release notes to verify that any features on that switch are supported in the fabric with the newer code.
Configuration ∙ ∙ Configuration upload and download issues.........................................................................................................................................................51 Brocade configuration form........................................................................................................................................................................................
Configuration Symptom The configuration upload fails. If your ping is successful from your computer, but you cannot reach it from inside your data center, there could be a block on the firewall to not allow FTP connections from inside the data center. Contact your network administrator to determine if this is the cause and to resolve it by opening the port up on both inbound and outbound UDP and TCP traffic. Example of a failed ping C:\> ping 192.168.163.50 Pinging 192.168.163.
Configuration Symptom The configuration download fails NOTE Root access is required to see the mentioned path. ∙ The FTP or SCP server’s IP address cannot be contacted. Verify that you can connect to the FTP server. Use your local PC to connect to the FTP server or ping the FTP server. ∙ There was a reason to disable the switch. Note, however, that you must disable the switch for some configuration downloads.
Configuration Send this and all logs to your switch support provider. Messages captured in the logs Configuration download generates both RASlog and Audit log messages resulting from execution of the configDownload command. The following messages are written to the logs: ∙ configDownload completed successfully ... (RASlog and Audit log) ∙ configUpload completed successfully ... (RASlog) ∙ configDownload not permitted ... (Audit log) ∙ configUpload not permitted ...
Firmware Download Errors ∙ ∙ ∙ ∙ ∙ Blade troubleshooting tips........................................................................................................................................................................................... 55 Firmware download issues..........................................................................................................................................................................................
Firmware Download Errors Symptom The AP blade is stuck in the "LOADING" state (issue slotShow to confirm). Probable cause and recommended action If the blade remains in the loading state for a significant period of time, the firmware download times out. Remove the blade and re-insert it. When it boots up, autoleveling is triggered and the firmware download is attempted again.
Firmware Download Errors Symptom Server is inaccessible or firmware path is invalid. ∙ The FTP or SCP server’s host name is not known to the switch. Verify with your network administrator that the switch has access to the FTP server. Verify the path to the FTP or SCP server is accessible from the switch. For more information on checking your FTP or SCP server, refer to Configuration on page 51. ∙ The USB path is not correct.
Firmware Download Errors Symptom Cannot download on a switch with Interop mode turned on. Probable cause and recommended action On single CP system, the Interop fabric does not support Coordinated HotCode Load. Perform a firmwareDownload -o command. The operand bypasses the checking of Coordinated HotCode Load (HCL). On single CP systems in interop fabrics, the HCL protocol is used to ensure data traffic is not disrupted during firmware upgrades.
Firmware Download Errors Then reissue the firmwareDownload command to download the desired firmware version to both CPs. For example, if CP0 is running Fabric OS v7.3.0 on the primary and secondary partitions, and CP1 is running Fabric OS v7.4.0 on the primary and secondary partitions, then synchronize them by issuing the firmwareDownload command. NOTE Some of the messages include error codes (as shown in the following example). These error codes are for internal use only and you can disregard them.
Firmware Download Errors Preinstallation messages The system messages in this section are displayed if an exception is encountered during firmware download. The following examples show feature-related messages that you may see if you were upgrading from Fabric OS v7.2.0 to v7.4.0: NOTE The system messages in this section are for illustration purposes only. They do not represent the entire range of possible error messages appropriate to a wide variety of installation scenarios. Upgrade to 7.
Firmware Download Errors Downgrade is not allowed because Monitoring and Alerting Policy Suite (MAPS) is enabled. Please disable MAPS using "mapsconfig --disable" before proceeding. Downgrade to pre-7.4 Fabric OS versions is not allowed. Please configure TACACS+ keys of length in the range (8-40) characters using aaaconfig CLI {Usage: aaaconfig --change -conf tacacs+ [-s ], before performing a firmware downgrade. Downgrade is not allowed because ICL ports are configured as EX-ports.
Firmware Download Errors Message Cannot upgrade directly to v7.4.0. Upgrade your switch to v7.3.0 first before upgrading to the requested version. Probable cause and recommended action If the switch is running v7.2.0 or earlier, you are not allowed to upgrade directly to v7.4.0 because of the "two-version" rule. Upgrade your switch to Fabric OS version v7.3.0 before upgrading to v7.4.0 Message Non-disruptive firmwaredownload is not supported when downgrading to 7.2.
Security ∙ ∙ ∙ ∙ ∙ Passwords............................................................................................................................................................................................................................. 63 Device authentication ....................................................................................................................................................................................................
Security TABLE 11 Password recovery options (continued) Topic Solution If a user has only the root password, what is the password recovery mechanism? Use the passwd command to set other passwords. How to recover boot PROM password? Contact your switch service provider and provide the recovery string. How to recover a user, or admin password? Refer to Passwords on page 63 for more information on recovering these passwords. Symptom User is unable to modify switch settings.
Security . TABLE 12 SSL messages and actions Message Action The page cannot be displayed The SSL certificate is not installed correctly or HTTPS is not enabled correctly. Make sure that the certificate has not expired, that HTTPS is enabled, and that certificate file names are configured correctly. The security certificate was issued by a company you have not chosen to trust. The certificate is not installed in the browser. Install it as described in the Fabric OS Administrator's Guide.
Security Symptom SNMP management station server is unable to receive traps from fabric. If you continue to have problems, collect the data in the next section and contact your switch support provider. Gathering additional information In addition to supportSave , gather the MIB browser snapshot with the problem (like Adventnet screen snapshot) for an MIB variable. FIPS This section describes symptoms with associated causes and recommended actions for problems related to FIPS.
Virtual Fabrics ∙ ∙ ∙ ∙ ∙ ∙ ∙ General Virtual Fabrics troubleshooting................................................................................................................................................................67 Fabric identification issues........................................................................................................................................................................................... 68 Logical Fabric issues.......................................
Virtual Fabrics Fabric identification issues Symptom E_Ports directly connecting two logical switches do not form, or are disabled. Probable cause and recommended action The FIDs on each of the logical switches must be the same. Use the lsCfg --show command to view the current FIDs on the chassis and then the lsCfg --change command with the -newfid option to change the FID. Symptom Invalid FID.
Virtual Fabrics Symptom EX_Port is disabled with reason "Export in non base switch". Probable cause and recommended action An EX_Port must be in the base switch. Use the lsCfg --create command with the -b option to create a base switch. Then use the lsCfg --config command with the -slot and -port options to move the port to the base switch. If the port is not intended to be used as an EX_Port, use the portCfgDefault command to reset the port to its default configuration.
Virtual Fabrics Symptom Given slot number is not valid on this platform. Probable cause and recommended action You are specifying a slot number that is not valid on the platform., for example, slot 0 on a Brocade X6-8 or slot 12 on a Brocade X6-4. Symptom Slot must be enabled to configure ports. Probable cause and recommended action You may only attempt to configure ports on enabled blades (blades may be faulted). Symptom Unable to determine slot type.
Virtual Fabrics Symptom A port or ports cannot be moved to the requested switch because it would exceed the 256 area limit for this switch. Probable cause and recommended action The area limit would be exceeded if the lsCfg command were allowed. Symptom A port or ports cannot be moved to the requested switch because it may only exist in a base or default switch. Probable cause and recommended action You are attempting to move ports on a core blade into a non-default or non-base switch.
Virtual Fabrics 72 Brocade Fabric OS Troubleshooting and Diagnostics Guide 53-1004126-01
ISL Trunking ∙ ∙ Link issues............................................................................................................................................................................................................................. 73 Buffer credit issues...........................................................................................................................................................................................................
ISL Trunking Symptom A link that is part of an ISL trunk failed. To correct this issue, connect additional ISLs to the switch with which you want to communicate. ∙ The ports are not the same speed or they are not set to an invalid speed. Manually set port speeds to a speed supported on both sides of the trunk. ∙ The ports are not set to the same long distance mode. Set the long distance mode to the same setting on all ports on both sides of the trunk.
Zoning ∙ ∙ ∙ ∙ Overview of corrective action..................................................................................................................................................................................... 75 Segmented fabrics............................................................................................................................................................................................................ 76 Zone conflicts...........................................
Zoning To display information about the TI zone purplezone, use the following command: switch:admin> zone --show purplezone Defined TI zone configuration: TI Zone Name: redzone: Port List: 1,2; 1,3; 3,3; 4,5 Configured Status: Activated / Failover-Enabled Enabled Status: Activated / Failover-Enabled To display information about all TI zones in the defined configuration, use the following command: switch:admin> zone --show Defined TI zone configuration: TI Zone Name: greenzone: Port List: 2,2; 3,3; 5,3; 4,
Zoning Symptom Fabric segmentation is caused by a "type mismatch". Probable cause and recommended action Occurs when the name of a zone object in one fabric is also used for a different type of zone object in the other fabric. A zone object is any device in a zone. Symptom Fabric segmentation is caused by a "content mismatch". Probable cause and recommended action Occurs when the definition in one fabric is different from the definition of a zone object with the same name in the other fabric.
Zoning TABLE 13 Commands for debugging zoning (continued) Command Function licenseShow Displays current license keys and associated (licensed) products. switchShow Displays currently enabled configuration and any E_Port segmentations resulting from zone conflicts. zoneAdd Use to add a member to an existing zone. zoneCreate Use to create a zone. Before a zone becomes active, the cfgSave and cfgEnable commands must be used. zoneDelete Use to delete zones.
Zoning ATTENTION The cfgClear command clears the zoning database on the switch where the command is run. 6. Enter the switchEnable command on each switch once the zoning configuration has been cleared. This forces the zones to merge and populates the switches with the correct zoning database. The fabrics then merge. Changing the default zone access A switch is not allowed to merge with the another switch that does not have an active effective configuration and if the default zone modes mismatch.
Zoning 3. Rearrange zone members so the configuration for both switches is the same. Arrange zone members in alphabetical order, if possible. Checking for Fibre Channel connectivity problems Enter the fcPing command to: ∙ Generate an Extended Link Service (ELS) frame ECHO request to the source port specified and validate the response. ∙ Generate an ELS ECHO request to the destination port specified and validate the response.
Zoning Request timed out Request timed out Request timed out Request timed out Request timed out 5 frames sent, 0 frames received, 0 frames rejected, 5 frames timeout Round-trip min/avg/max = 0/0/0 usec Checking for zoning problems 1. 2. Enter the cfgActvShow command to determine if zoning is enabled. ∙ If zoning is enabled, it is possible that the problem is being caused by zoning enforcement (for example, two devices in different zones cannot detect each other).
Zoning 82 Brocade Fabric OS Troubleshooting and Diagnostics Guide 53-1004126-01
Diagnostic Features ∙ ∙ ∙ ∙ ∙ ∙ ∙ ∙ ∙ ∙ ∙ ∙ Fabric OS diagnostics.................................................................................................................................................................................................... 83 Diagnostic information...................................................................................................................................................................................................83 Power-on self-test........
Diagnostic Features Power-on self-test By default, when you power on the system, the boot loader automatically performs power-on self-tests and loads a Fabric OS kernel image. Likewise, if you issue the slotPowerOn command, or insert a new blade, a power-on self test is run on that blade. The POST tests provide a quick indication of hardware readiness when hardware is powered up. These tests do not require user input to function.
Diagnostic Features POST1: Started running Thu Mar 31 20:12:51 GMT 2005 POST1: Test #1 - Running turboramtest POST1: Test #2 - Running portregtest POST1: Script PASSED with exit status of 0 Thu Mar 31 20:12:54 GMT 2005 took (0:0:3) POST2: Started running Thu Mar 31 20:12:55 GMT 2005 POST2: Test #1 - Running portloopbacktest (SERDES) POST2: Test #2 - Running minicycle (SERDES) POST2: Running diagshow POST2: Script PASSED with exit status of 0 Thu Mar 31 20:13:12 GMT 2005 took (0:0:17) 2005/03/31-20:13:13, [
Diagnostic Features Viewing the overall status of the switch 1. Connect to the switch and log in as admin. 2. Enter the mapsdb --show command.
Diagnostic Features TABLE 14 Switch summary information Variable Definition switchType Switch model and revision numbers switchName Switch name switchState Switch state: Online, Offline, Testing, or Faulty switchMode Switch operation mode: Native, Interop, or Access Gateway switchRole Principal, Subordinate, or Disabled switchDomain ID: 0-31 or 1-23 switchId Switch embedded port D_ID switchWwn Switch World Wide Name (WWN) switchBeacon Switch beaconing state: On or Off zoning When Acces
Diagnostic Features 2. Enter the uptime command. ecp:admin> uptime 10:50:19 up 11 days, 6:28, 1 user, load average: 0.49, 0.53, 0.54 The uptime command displays the length of time the system has been in operation, the total cumulative amount of uptime since the system was first powered on, and the load average over the past one minute. The reason for the last switch reboot is also recorded in the system message log.
Diagnostic Features Link errors Once the frame is sent out of the port, the spinFab command monitors the link errors in the ASIC. If any of the error counters are nonzero, spinFab reports an error and the test fails on the port.
Diagnostic Features If you receive any of the Tx/Rx errors, follow the suggest debugging procedures: ∙ Check whether the same port is reporting link errors as discussed in Link errors on page 89. If yes, follow the same set of debugging procedures as discussed in Link errors on page 89. ∙ Check whether the local port or the remote port is beyond port 255. If yes, try connecting to the lower number of ports. This behavior is found in Fabric OS v6.2.0 and earlier versions only.
Diagnostic Features 2. Enter the portShow command, specifying the number that corresponds to the port you are troubleshooting. In this example, the status of port 10 is shown: switch:admin> portshow 10 portName: portHealth: HEALTHY Authentication: None portDisableReason: None portCFlags: 0x1 portFlags: 0x20b03 PRESENT ACTIVE F_PORT G_PORT U_PORT LOGICAL_ONLINE LOGIN NOELP ACCEPT FLOGI portType: 18.
Diagnostic Features tim_rdy_pri tim_txcrd_z time_txcrd_z_vc 0- 3: time_txcrd_z_vc 4- 7: time_txcrd_z_vc 8-11: time_txcrd_z_vc 12-15: er_enc_in er_crc er_trunc er_toolong er_bad_eof er_enc_out er_bad_os er_c3_timeout er_c3_dest_unreach er_other_discard er_type1_miss er_type2_miss er_type6_miss er_zone_miss er_lun_zone_miss er_crc_good_eof er_inv_arb open transfer opened starve_stop fl_tenancy nl_tenancy zero_tenancy 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 810 0 409856 0 1715 331135 4 Time R_RDY high
Diagnostic Features 28: 0 0 0 29: 0 0 0 30: 664k 6.7k 0 31: 12 4 0 (output truncated) 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 6 3 0 0 0 0 0 0 0 0 0 0 1 1 1 1 2 2 0 0 0 0 0 0 0 0 The portErrShow command output provides one output line per port. Refer to Table 18 for a description of the error types.
Diagnostic Features Checking the status of the fans 1. Connect to the switch and log in as admin. 2. Enter the fanShow command. switch:admin> fanshow Fan 1 is Absent Fan 2 is Ok, speed is 6553 RPM Fan 3 is Ok, speed is 6367 RPM The possible status values are: ∙ OK—Fan is functioning correctly. ∙ Absent—Fan is not present. ∙ Below minimum—Fan is present but rotating too slowly or stopped. ∙ Above minimum—Fan is rotating too quickly. ∙ Unknown—Unknown fan unit installed.
Diagnostic Features ================================================= 1 Ok 28 82 2 Ok 16 60 3 Ok 18 64 Information is displayed for each temperature sensor in the switch. The possible temperature status values are: ∙ OK: Temperature is within acceptable range. ∙ FAIL: Temperature is outside of acceptable range. Damage might occur. Refer to the hardware reference manual for your switch to determine the normal temperature range.
Diagnostic Features Clearing the system message log 1. Connect to the switch and log in as admin. 2. Enter the errClear command. 3. Repeat step 2 on the standby CP for a complete erasure of the message log. All switch and chassis events are removed from both CPs. Port log Fabric OS maintains an internal log of all port activity. The port log stores entries for each port as a circular buffer. For all other switches, the number of lines range from 8192 to 16384.
Diagnostic Features The portLogDump command output (trace) is a powerful tool that is used to troubleshoot fabric issues. The portLogDump output provides detailed information about the actions and communications within a fabric. By understanding the processes that are taking place in the fabric, issues can be identified and located. The portLogDump command displays the port log, showing a portion of the Fibre Channel payload and header (FC-PH).
Diagnostic Features TABLE 20 Fabric OS to UNIX message severities Fabric OS message severity UNIX message severity Critical (1) Emergency (0) Error (2) Error (3) Warning (3) Warning (4) Info (4) Info (6) In this example, Fabric OS messages map to local7 facility level 7 in the /etc/syslog.conf file: local7.emerg local7.alert local7.crit local7.err local7.warning local7.notice local7.info local7.
Diagnostic Features Removing a syslogd host from the list 1. Connect to the switch and log in as admin. 2. Enter the syslogdIpRemove command followed by the IP address of the host that you want to remove. switch:admin> syslogdipremove 10.1.2.1 3. Enter the syslogdIpShow command to verify if the IP address was deleted. Automatic trace dump transfers You can set up a switch so that diagnostic information is transferred automatically to a remote server.
Diagnostic Features Example of setting the interval in hours switch:admin> supportftp -t 4 supportftp: ftp check period changed The minimum interval is 1 hour. Specify 0 to disable the checking feature. Saving comprehensive diagnostic files to the server 1. Connect to the switch and log in as admin. 2. Enter the supportSave -c command and respond to the prompts.
Diagnostic Features Trace dump support You can run the traceDump command to display the trace dump details. The following is an example from a Gen 6 device. For Gen 5 devices, refer to Fabric OS Command Reference.
Diagnostic Features 102 Brocade Fabric OS Troubleshooting and Diagnostics Guide 53-1004126-01
Switch Type and Blade ID The switchType is a displayed field listed when you run the switchShow command. When you are gathering information to give to your switch support provider, you may be asked the switch model. If you do not know the model, refer to Table 21 to convert the switchType to a B-Series model number. switch:admin> switchshow switchName: wedge-g620 switchType: 145.1 <=== convert this number using the following table.
Switch Type and Blade ID 8 9 10 11 12 CORE BLADE SW BLADE SW BLADE SW BLADE AP BLADE 177 153 96 97 186 CR32-8 FC16-64 FC16-48 FC16-32 SX6 ENABLED ENABLED ENABLED ENABLED ENABLED TABLE 22 B-series blade model descriptions Blade ID B-series blade model Description 50 CP8 Gen 5 DCX 8510 Director control processor blade 75 FX8-24 24-FC port with 10 1-GbE and two 10-GbE ports Fibre Channel routing and FCIP blade 96 FC16-48 16 Gb 48-FC ports blade 97 FC16-32 16 Gb 32-FC ports blade 98 CR16-8
Hexadecimal Conversion ∙ ∙ ∙ Hexadecimal overview..................................................................................................................................................................................................105 Example conversion of the hexadecimal triplet Ox616000................................................................................................................... 105 Decimal-to-hexadecimal conversion table...................................................
Hexadecimal Conversion Decimal-to-hexadecimal conversion table Decimal 01 02 03 04 05 06 07 08 09 10 Hex 01 02 03 04 05 06 07 08 09 0a Decimal 11 12 13 14 15 16 17 18 19 20 Hex 0b 0c 0d 0e 0f 10 11 12 13 14 Decimal 21 22 23 24 25 26 27 28 29 30 Hex 15 16 17 18 19 1a 1b 1c 1d 1e Decimal 31 32 33 34 35 36 37 38 39 40 Hex 1f 20 21 22 23 24 25 26 27 28 Decimal 41 42 43 44 45 46 47 48 49 50 Hex 29 2a 2b 2c 2d
Hexadecimal Conversion Decimal 211 212 213 214 215 216 217 218 219 220 Hex d3 d4 d5 d6 d7 d8 d9 da db dc Decimal 221 222 223 224 225 226 227 228 229 230 Hex dd de df e0 e1 e2 e3 e4 e5 e6 Decimal 231 232 233 234 235 236 237 238 239 240 Hex e7 e8 e9 ea eb ec ed ef ee f0 Decimal 241 242 243 244 245 246 247 248 249 250 Hex f1 f2 f3 f4 f5 f6 f7 f8 f9 fa Decimal 251 252 253 254 255 Hex fb fc fd fe ff Brocade Fabri
