SGI® InfiniteStorage™ 4000 RAID User’s Guide 007-4834-001
CONTRIBUTORS Writing contributions by Mark Schwenden Illustration contributions by Chrystie Danzer Production by Mark Schwenden Engineering contributions by Marty Castilla, Henry Ortiz, Afaq Syed, and Sammy Wilborn COPYRIGHT © 2006, Silicon Graphics, Inc. All rights reserved; provided portions may be copyright in third parties, as indicated elsewhere herein.
Record of Revision Version Description 001 April 2006 First publication 007-4834-001 iii
Contents 1. Record of Revision . . . . . . . . . . . . . . . . . . . . . . iii Figures . . . . . . . . . . . . . . . . . . . . . . . . . . ix Tables . . . . . . . . . . . . . . . . . . . . . . . . . . xi About This Guide. . . . . . . . . . . . . . . . . . . . . . . xiii Important Information . . . . . . . . . . . . . . . . . . . . . xiv Chapter Descriptions . . . . . . . . . .
Contents RAID Controllers . 3. . . . . . . . . . . . . . . . . . . . . . . 17 Environmental Status Modules (ESMs) . . . . . . . . . . . . . . . . . 19 Power Supply and Fan Modules . . . . . . . . . . . . . . . . . 20 . . Tray ID Selection . . . . . . . . . . . . . . . . . . . . . . . 21 Link Rate Switch . . . . . . . . . . . . . . . . . . . . . . . 22 SFP Transceivers . . . . . . . . . . .
Contents Upgrading Drives . 5. . . . . . . . . . . . . . . . . . . . . 57 Adding a Drive to an Empty Slot . . . . . . . . . . . . . . . . . 57 Adding Drives of Larger Capacity . . . . . . . . . . . . . . . . . 59 B. 007-4834-001 . Adding Expansion Enclosures . . . . . . . . . . . . . . . . . . . 61 Cabling . . . . . . . . . . . . . . . . . . . . 63 RAID and Expansion Enclosure Cabling Tips . . . . . .
Contents FCC Warning viii . . . . . . . . . . . . . . . . . . . . . . . 101 VDE 0871/6.78 . . . . . . . . . . . . . . . . . . . . . . . 102 European Union Statement . . . . . . . . . . . . . . . . . . . 102 International Special Committee on Radio Interference (CISPR) . . . . . . . . . 103 Canadian Department of Communications Statement . . . . . . . . . . . . 103 Attention . . . . . . . . . . . . .
Figures 007-4834-001 Figure 1-1 InfiniteStorage 4000 Rackmount Storage Enclosures . . . . . 7 Figure 1-2 InfiniteStorage 4000 Controller and Expansion Enclosures . . . 8 Figure 1-3 InfiniteStorage 4000 Rack Dimensions . . . . . . . . . 10 Figure 1-4 Rack PDU Connection Example . . . . . . . . . 11 Figure 2-1 Front and Back View of the Storage Enclosure . . . . . . . 14 Figure 2-2 Storage Enclosure and Bezel (Front View) . . . . . . . .
Figures x Figure 4-6 Replacing an SFP Transceiver . . . . . . . . . . . . 53 Figure 4-7 SFPs and Cable Used . . . . . . . . . . . 54 Figure 4-8 Removing and Installing an Environmental Status Module . . . 56 Figure 4-9 Adding a New Expansion Enclosure to an Existing RAID Enclosure . 62 Figure 5-1 RAID Controller Enclosure Channel Port Locations . . . . . 64 Figure 5-2 Drive Expansion ESM Ports . . . . .
Tables 007-4834-001 Table 3-1 Enclosure Front Panel Status LEDs . . . . . . . . 33 Table 3-2 Storage Enclosure Rear LED Status Information . . . . . . 34 Table 5-1 Cabling a RAID Enclosure With One Drive Expansion . . . . 66 Table 5-2 Cabling a RAID Enclosure to Two Disk Expansion Enclosures . . 67 Table 5-3 One RAID Enclosure With 4 Drive Expansion Modules . . . . 68 Table 5-4 One RAID Enclosure With Five Drive Expansion Modules . . .
About This Guide About This Guide This guide provides an overview of the SGI InfiniteStorage 4000 RAID system. It covers routine operation and replacement procedures and provides troubleshooting and reference information for all customer-replaceable components. The SGI InfiniteStorage 4000 is a highly scalable RAID storage system designed for continuous availability. Some of the features of the system include hot-swappable components, easy expansion, and redundant power and cooling systems.
About This Guide Important Information Danger: Never look into the end of a fiber-optic cable to confirm that light is being emitted (or for any other reason). Most fiber-optic laser wavelengths (1300 nmand 1550nm) are invisible to the eye and cause permanent eye damage. Shorter wavelength lasers (for example, 780 nm) are visible and can cause significant eye damage. Use only an optical power meter to verify light output.
About This Guide Related Publications This guide is part of a document set that fully supports the installation, operation, and service of the SGI InfiniteStorage 4000 RAID system. See the following documents for more information about your system. If a document number ends in “X,” use the latest available version of that document. • SGI InfiniteStorage TPSSM Software Concepts Guide (007-4749-00x) This guide gives complete instructions on how to install the TPSSM software for host and/or client operation.
About This Guide Conventions The following conventions are used throughout this document: Convention Meaning Command This fixed-space font denotes literal items such as commands, files, routines, path names, signals, messages, and programming language structures. variable The italic typeface denotes variable entries and words or concepts being defined. Italic typeface also is used for book titles. user input This fixed-space font denotes literal items that the user enters in interactive sessions.
About This Guide Reader Comments If you have comments about the technical accuracy, content, or organization of this document, contact SGI. Be sure to include the title and document number of the manual with your comments. (Online, the document number is located in the front matter of the manual. In printed manuals, the document number is located at the bottom of each page.) You can contact SGI in any of the following ways: • Send e-mail to the following address: techpubs@sgi.
Chapter 1 1. Introduction to the SGI InfiniteStorage 4000 The SGI InfiniteStorage 4000 is a high-bandwidth RAID Fibre Channel storage system that uses Fibre Channel (SCSI-protocol) disks. All drive arrays are fibre channel attached.
1: Introduction to the SGI InfiniteStorage 4000 InfiniteStorage 4000 RAID Features The InfiniteStorage 4000 RAID system is a mass storage system using Fibre Channel (SCSI-protocol) disks. All drive arrays are fibre channel attached.
System Features Performance Features The SGI InfiniteStorage 4000 RAID storage system has the following basic features: • Outstanding performance, built on multi-channel end-to-end Fibre Channel technology. • Support for fibre channel attached (SCSI-protocol) drive array technology. • Continuous availability, with constant monitoring and optional redundancy of all active components. • Dynamic scalability, making it easy to grow all subsystem resources without disruption.
1: Introduction to the SGI InfiniteStorage 4000 RAID Controller Features The RAID controllers in the InfiniteStorage 4000 have the following features: 4 • 112-drive maximum configuration • 4 RAID levels (0, 1, 3, and 5) • 4 Gbit/s front end (FE) and back end (BE) • Fibre Channel switched connection to disk (SBOD) • Immediate LUN availability (ILA) • Transparent disk drive rebuilds • Variable stripe size per controller (16K, 32K, 64K, 128K, 256K and 512K) • Mirrored cache • Drive roaming
System Features Availability Features The InfiniteStorage 4000 has the following availability features: • Dual power feeds with dual power supplies • Redundant cooling • Battery back-up (BBU) maintains cache in case of power failure • Dynamic hot-sparing • Non-disruptive component replacement • Enclosure services interface (ESI) for SCSI enclosure services (SES) Supported Platforms The InfiniteStorage 4000 supports the following hardware and software platforms: 007-4834-001 • Hardware: SGI A
1: Introduction to the SGI InfiniteStorage 4000 Adapter and Switch Compatibility The InfiniteStorage 4000 supports the following host bus adapters (HBAs) and switches: Note: The InfiniteStorage 4000 does not support copper Fibre Channel HBAs.
System Components Storage Enclosures The InfiniteStorage 4000 is comprised of storage enclosures. These enclosures provide all of the logic, power, and I/O functions of the system. Each enclosure can be operated as an independent storage system, or multiple enclosures can be cabled together to create much larger disk arrays. There are two types of storage enclosures: controller enclosures and expansion enclosures.
1: Introduction to the SGI InfiniteStorage 4000 Figure 1-2 shows a rear view of both a controller enclosure and an expansion enclosure. Disk controller enclosure rear panel Disk expansion enclosure rear panel Figure 1-2 InfiniteStorage 4000 Controller and Expansion Enclosures See Chapter 2, “Storage Enclosures” for more information about the controller and expansion enclosures and their components.
System Components SGI InfiniteStorage 4000 Rack The SGI InfiniteStorage 4000 rack has the following features: • 78-in. high x 24-in. wide x 40-in. deep (198 cm x 60.9 cm x 101.6 cm), see Figure 1-3. • 40 rack units internal vertical capacity: 1 rack unit = 1.75 inches (4.44 cm). • Removable rear panel. • Heavy-duty casters for ease of movement. • Dual power distribution units (PDUs), see the example in Figure 1-4. • Optional stabilizer foot to reduce tipping.
1: Introduction to the SGI InfiniteStorage 4000 Figure 1-3 10 InfiniteStorage 4000 Rack Dimensions 007-4834-001
System Components Figure 1-4 shows an example of the power distrubution connection features of the rack. Figure 1-4 Rack PDU Connection Example For more detailed information on the rack, see “Rack Specifications” on page 83.
Chapter 2 2. Storage Enclosures The storage enclosure is the basic building block of the SGI InfiniteStorage 4000 mass storage RAID system. Each storage enclosure provides redundant power and cooling to its individual components. The storage enclosures also house the I/O components of the storage system. Chapter 4, “Replacing Enclosure Components” provides information on removing and replacing customer replaceable units (CRUs).
2: Storage Enclosures A Quick Glance at the Storage Enclosure Hardware • The top-left ESM is inverted from the bottom-right ESM. • The top-right power-fan unit is inverted from the bottom-left power-fan unit. • The drive tray is in the correct (top) orientation when the lights of the drives are at the bottom. • Four fans pull air through the enclosure from front to back across the drives. 1. Drive Unit 2. Alarm Mute Button 3. Link (Data) Rate Switch (4/2 Gb/s) 4. ESM CRU 5. Power-Fan CRU 6.
Overview Overview Important: Several references are made in this chapter to the storage management software (TPSSM). For complete information on the operation and use of this software, see the SGI InfiniteStorage TPSSM Software Concepts Guide (007-4749-00x). Figure 2-2 shows a front view of the storage enclosure. The front of the enclosure contains the following components: • Front bezel: The bezel is a removable decorative cover with holes for viewing the status LEDs and openings for the drive bays.
2: Storage Enclosures Callout 1 in Figure 2-2 shows the location of the first drive unit, callout 2 shows the location of the mute button and callout 3 the location of the data rate select switch (2Gb/4Gb per second). Figure 2-3 shows a rear view of an expansion enclosure. Callout 4 shows the location of an environmental status module (ESM) unit and callout 5 the location of a power supply/fan unit. Both are customer replaceable units (CRUs).
The Disk Drives The Disk Drives Each storage enclosure can contain up to 16 low-profile Fibre Channel disk drives. Controller enclosures can be cabled to as many as six expansion enclosures for a total capacity of 112 disks in one storage system. Each disk is mounted in a sled for ease of installation and removal. These drive sleds have a lever and latch mechanism that allows you to cam the drive in and out of the enclosure.
2: Storage Enclosures Each controller also has two labels on its rear panel: one shows the MAC address of the controller and one shows the battery service date (see Figure 2-5).
Environmental Status Modules (ESMs) Front View 1. Drive Canister 2. Alarm Mute Button 3. Link Rate Switch Back View 4. Battery Canister 5. Controller A (Inverted) 6. Power-Fan Canister 7. Power Connector 8. Power Switch 9. Host Channels (two or four) 10. Controller B 11. Ethernet Ports 12. Serial Port 13. Dual-ported Drive Channel 14.
2: Storage Enclosures Disk expansion enclosure rear panel ESMs Figure 2-6 Environmental Status Modules Power Supply and Fan Modules The storage enclosures each have two combined function power/fan units. These units are mounted in the rear of the enclosure (see Figure 2-7). They pull air through the front bezel to cool the internal components of the enclosure. The air is exhausted out the rear of the fan units. For more information on the power/fan units, see “Wiring and Power” on page 95.
Tray ID Selection light that illuminates if there is a problem with the units. Callout 1 shows the latching mechanism used to remove or secure the power/fan unit in the disk enclosure. Figure 2-7 Power/Fan Units Tray ID Selection The tray ID number is set through the graphical user interface (GUI) or command line interface (CLI) that comes with the InfiniteStorage system. During initial power-up of the RAID system, the controller and ESM enclosures are given automatically assigned tray IDs.
Note: Each enclosure in a storage system must have a unique ID. If you are installing multiple systems in one rack, there may be enclosures with the same ID numbers because the systems are independent. Link Rate Switch The link rate switch is located on the same panel as the enclosure ID switch, between the power supplies on the rear of the enclosure (see Figure 2-8). The link rate switch controls the speed of the Fibre Channel connections to the enclosure. This switch can be set to 2 Gbit/s or 4 Gbit/s.
SFP Transceivers transceiver. As shown in Figure 2-9, the SFP assembly consists of an active SFP transceiver unit (callout 1), and the fibre optic cable assembly (callout 2) that connects with it. Or, the SFP may have a passive SFP transceiver (callout 3) that is integrated with a copper cable assembly (callout 4). Figure 2-9 SFP Transceiver You can hot-swap a failed SFP transceiver, which means you can replace it while the controller enclosure is in operation.
Chapter 3 3.
3: Operating Storage Enclosures To turn on the enclosure after a normal shutdown (as described in “Turning the Power Off” on page 30), turn on both power switches on the rear of the enclosure (see Figure 3-1) or the main circuit breaker, whichever is applicable. You must turn on both power supply switches to take advantage of the redundant power supplies. Note: Always wait at least 30 seconds between the time you turn a power switch off and the time you turn it back on again.
Turning the Power On after an Unexpected Shutdown Turning the Power On after an Unexpected Shutdown The enclosure (and the entire storage array) may shut down unexpectedly under the following conditions: • If the internal temperature of the enclosure exceeds the maximum operating temperature (an overtemp condition). See the “Troubleshooting an Overtemp Condition” section for more information on overtemp conditions. • If there is a general power failure or a loss of power to the storage array.
3: Operating Storage Enclosures In the expansion enclosures, the overtemp indicator on the environmental status module illuminates if the temperature reaches 40 ºC (104 ºF). If both power supplies shut down, the overtemp indicator cannot come on. See Figure 3-3 on page 34 for the location of the overtemp indicators on expansion enclosures. The controller enclosure does not have an overtemp indicator. Check the storage management software for temperature information about the controller enclosures.
Turning the Power On after an Unexpected Shutdown Turning the Power On after an Emergency Shutdown To restart the enclosure after a power failure or emergency shutdown, follow these steps: 1. After the emergency situation is over or power is restored to the site, turn off all power switches (see Figure 3-1 on page 26). Danger: Severe electrical shock can occur. Never turn on the power to any equipment when there is evidence of fire, water, or structural damage.
3: Operating Storage Enclosures Turning the Power Off The enclosure is designed to run continuously, 24 hours a day. However, you may need to turn the power off for maintenance, such as upgrading the drives or replacing certain modules.To turn the power off, follow these steps: 1. Use TPSSM to determine the status of your system components and any special instructions before proceeding. The operating system software may require you to perform other procedures before turning off the power. 2.
Monitoring Status with Software Monitoring Status with Software Use storage management software (TPSSM) to monitor enclosure status. You should run the software constantly and check it frequently. TPSSM provides the best method to diagnose and repair failures. This software helps you do the following: • Determine the nature of the failure. • Locate the failed component. • Provide recovery procedures to repair the failure.
3: Operating Storage Enclosures Checking the Indicator Lights The enclosure’s indicator lights display the status of the enclosure and its components. Green indicators mean a normal operating status; amber indicators mean a possible failure. It is important that you check all the indicators on the front and rear of the enclosure when you turn on the power.
Checking the Indicator Lights Table 3-1 Enclosure Front Panel Status LEDs LED location # Component LED Normal status Problem status 1 Drive service action allowed (OK to remove) Off On 2 Drive fault Off On (not blinking) drive failure If blinking – drive, volume, or storage array locate function 3 Drive active Blinking – data Off - no power to drive being processed or drive not seated On (steady) – no data processing 4 Global locate On 5 Global summary Off fault On - enclosure compone
3: Operating Storage Enclosures Rear Indicator Lights This section describes the rear indicator lights for the expansion enclosure. Figure 3-3 shows the rear indicators on the expansion enclosure. Controller enclosures have RAID controller LEDs, but not ESM LEDs. Figure 3-3 Storage Enclosure Rear Indicator Lights Table 3-2 describes the rear indicators on the enclosure.
Checking the Indicator Lights Table 3-2 (continued) Storage Enclosure Rear LED Status Information Callout location Component Light Normal Status Problem Status 8 ESM Port 1 In Bypass Off - (Also off if no SFP connected) On – No valid device detected and port is bypassed 9 ESM Ports 1 and 2 Data Rate On with light 10 on if 4 Gb 10 ESM Ports 1 and 2 Data Rate On and light 9 off if 2 Gb Not applicable On with light 9 on if 4 Gb 11 ESM Port 2 In Bypass Off - (Also off if no SFP connected) On –
3: Operating Storage Enclosures 36 Figure 3-4 Link Rate Switch Location (Front of Enclosure) Figure 3-5 Link Rate Rear Indicator Lights (Rear of Enclosure) 007-4834-001
Moving the Enclosure Service Action Allowed Light Each drive CRU, power-fan CRU, and ESM CRU has a Service Action Allowed light. The Service Action Allowed light lets you know when you can remove a customer replaceable unit (CRU) with another component safely. Caution: Potential loss of data access – Never remove a drive CRU, a power-fan CRU, or an ESM CRU unless the Service Action Allowed light is turned on.
3: Operating Storage Enclosures Removing the Enclosure To remove the enclosure from the rack, follow these steps: 1. Unsnap the side brackets from the front of the enclosure (see Figure 3-6). 2. Unscrew the front bezel from the mounting pins. 3. Using a flat-blade screwdriver, carefully remove the bezel from the front of the drive enclosure. You must remove the bezel before you can remove the enclosure from the rack. Figure 3-6 Removing the Front Bezel and Front Retention Screws from the Enclosure 4.
Moving the Enclosure 5. Remove the front mounting screws (see Figure 3-6 on page 38). There are also two mounting screws at the rear, one for each support rail. 6. Slide the enclosure out of the front of the rack. 7. Repeat steps 1 through 6 for any other enclosures to be moved.
3: Operating Storage Enclosures Reinstalling the Enclosure If you are installing the enclosure in another location in the rack, (or in another rack) use the following mounting rail tips as applicable: • Attach the support rails to the new location in a rack. For more information, refer to the installation instructions included with your support rails. • If you are installing the rails above an existing module, position the rails directly above the module.
Front Bezel Air Filter Front Bezel Air Filter Use the following procedures to check, replace, or clean an air filter installed in the bezel of your InfiniteStorage 4000 RAID controller or disk expansion enclosure. Under certain conditions you may need to clean or replace the air filter in your RAID enclosure (callout 3 in Figure 3-7) as often as every three months. Removing and Replacing the Filter Use the following steps to replace the bezel filter: 1.
3: Operating Storage Enclosures 4. Disengage the air filter (callout 3 in Figure 3-8) from the screen frame’s retaining clips (callout 2). Figure 3-8 Remove or Replace an Air Filter in the Bezel’s Screen Frame 5. Insert the new air filter (or clean filter) into the screen frame and secure it within the screen frame by pressing it underneath the retaining clips on the sides of the frame. 6. Replace the screen frame in the back of the front bezel. 7. Reinstall the front bezel onto the drive module.
Chapter 4 4.
4: Replacing Enclosure Components 1. ! Check TPSSM for instructions on drive failure recovery procedures. Follow any instructions provided by the software before proceeding. Caution: Removing the wrong drive can cause data loss. Ensure that you remove only the failed drive. The drive fault indicator of the failed drive module illuminates amber. Removing a drive module while its green activity indicator is blinking can cause data loss and may cause the host controller to mark the drive as failed.
Replacing a Failed Disk Drive 7. Review all documentation shipped with the new drive module for any updated replacement procedures and other information. If necessary, modify the remaining steps to meet the system requirements. Kits often contain the most current servicing information. If the kit instructions conflict with those in this procedure, follow the kit instructions.
4: Replacing Enclosure Components 14. Check the status of each enclosure in the storage array. 15. Does any component have a Needs Attention status? • Yes – Click the Recovery Guru toolbar button in the Array Management Window, and complete the recovery procedure. If a problem is still indicated, contact your Technical Support organization. • No – Go to the next step. 16. Create, save, and print a new storage array profile.
Replacing a Failed Power/Fan Module 4. Review all documentation shipped with the new power/fan module for any updated replacement procedures and other information. If necessary, modify the remaining steps to meet the system requirements. Kits often contain the most current servicing information. If the kit instructions conflict with those in this procedure, follow the kit instructions. 5. If TPSSM instructs you to do so, turn off the power to the enclosure. Otherwise, leave the power on.
4: Replacing Enclosure Components 9. With the module fully unlatched pull the unit gently outward to remove the power/fan module from the enclosure. 10. Determine the correct orientation for the new power/fan module. If replacing the module on the left, orient the unit so the latch is at the top. If replacing the right-side module, orient the unit so the latch is at the bottom. 11. Slide the new module into the slot and push firmly until the latch snaps into place. 12.
Replacing a Failed RAID Controller Module Replacing a Failed RAID Controller Module When TPSSM instructs you to replace a failed RAID controller module, read “RAID Controllers” on page 17, and then follow these steps: ! Caution: Electrostatic discharge can damage sensitive components. Use a grounded wrist strap or other antistatic precautions before handling enclosure components. 1. Check TPSSM for instructions on RAID controller module failure recovery procedures.
4: Replacing Enclosure Components Figure 4-4 Removing the Controller’s Fibre Channel Cable and Transceiver 6. Wearing antistatic protection, pull up and outward on the latch handle near the top of the RAID controller module. The module will pop out of the locked position (see Figure 4-5). 7. Grasp the latch handle and pull to remove the failed canister.
Replacing an SFP Transceiver 8. Slide the new controller into the empty slot, and close the latch. 9. Reconnect the SFP transceivers and the host interface cables to the controller. 10. Turn on the power, and wait for the drives to spin up. 11. Check the power and fault indicators on the new controller as follows (see Figure 4-4 on page 50): • If the power indicator is off, the controller module may not have been inserted correctly. Remove the module and reinsert it.
4: Replacing Enclosure Components Caution: Electrostatic discharge can damage sensitive components – Touching the tray or its components without using a proper ground might damage the equipment. To avoid damage, use proper antistatic protection. 1. If needed, use the storage management software to create, save, and print a new storage array profile. 2. Did the Recovery Guru direct you to replace a failed SFP transceiver? • Yes – Go to the next step.
Replacing an SFP Transceiver 9. Remove the failed SFP transceiver from the controller. 10. Install the new SFP transceiver into the controller. 11. Reconnect the fiber-optic cable. 12. Check the bypass indicator next to the new SFP transceiver as follows: – If the bypass indicator is on, the SFP transceiver might be installed incorrectly. Remove the SFP transceiver and reinsert it.
4: Replacing Enclosure Components 17. Does any component have a Needs Attention status? – Yes – Click the Recovery Guru toolbar button in the Array Management Window, and complete the recovery procedure. If a problem is still indicated, contact Customer and Technical Support. – No – Go to the next step. 18. Create, save, and print a new storage array profile.
Replacing a Failed Environmental Status Module Replacing a Failed Environmental Status Module When TPSSM instructs you to replace a failed environmental status module, read “Environmental Status Modules (ESMs)” on page 19, and then follow the steps in this section. ! Caution: Electrostatic discharge can damage sensitive components. Use a grounded wrist strap or other antistatic precautions before handling enclosure components. 1.
4: Replacing Enclosure Components 9. Close both levers until the latch locks into place. 10. Install the SFP transceivers and Fibre Channel interface cables in their original locations. 11. Check the power and fault indicators on the new module. • If the power indicator is off, the module may not have been inserted correctly. Remove the module and reinsert it. • If the fault indicator is on, the power indicator stays off, or any other fault indicator is on, use TPSSM to determine the problem. 12.
Upgrading Drives Upgrading Drives You can upgrade drives in the following two ways: • Add drives to empty slots in the enclosure (see “Adding a Drive to an Empty Slot” on page 57). • Replace existing drives with drives of larger capacity (see “Adding Drives of Larger Capacity” on page 59). Adding a Drive to an Empty Slot Follow these instructions to install an additional drive into an empty slot in the enclosure. You can install additional drives while the enclosure is powered on and running.
4: Replacing Enclosure Components 7. Review all documentation shipped with the new drive module for updated replacement procedures and other information. ! Caution: If the temperature of the shipping or storage environment differs significantly from the temperature of the environment in which the drive module will be installed, acclimate the drive module before you install it. 8. Slide the new drive module fully into the slot. 9. Lower (close) the lever. 10.
Upgrading Drives Adding Drives of Larger Capacity This section provides guidelines for upgrading the disk drives in a storage array. Carefully read your software documentation and this entire section to determine if you should follow this procedure, a modified version of this procedure, or use a different procedure provided by your operating system. Instructions provided with your software should supersede the instructions in the section. Warning: Electrostatic discharge can damage sensitive components.
4: Replacing Enclosure Components 4. Shut down all I/O activity on the enclosure. Make sure the drive activity indicators on the front are not blinking. 5. Turn off both power switches on the rear of the enclosure. 6. Wearing antistatic protection, lift (open) the lever of a drive module and pull out the drive module (see Figure 4-1 on page 44). 7. Repeat step 6 for all of the drive modules in the enclosure. 8. Unpack a new drive.
Adding Expansion Enclosures Adding Expansion Enclosures You can add new expansion enclosures to an existing storage array without turning off the array or interrupting data flow. “RAID and Expansion Enclosure Cabling Tips” in Chapter 5 provides additional information on connecting multiple enclosures together into a larger RAID array. ! Caution: You can add only new expansion enclosures to the storage array.
4: Replacing Enclosure Components Important: Depending on how long it takes you to complete step 2, TPSSM may report a loss of redundancy error on the expansion enclosures in the loop. This error will not interfere with data flow, and will disappear after you finish installing the expansion enclosure(s). 4. Use TPSSM to check the status of the new drives and correct any errors found. 5. Use TPSSM to configure the new drives.
Chapter 5 5. Cabling This chapter describes the cabling between the RAID controller enclosure, the front-end host, and the expansion enclosures.
5: Cabling RAID and Expansion Enclosure Cabling Tips Keep the following in mind when cabling the RAID controller enclosure to a drive expansion enclosure: • Controller A contains drive channel 1. • Controller B contains drive channel 2. • Each controller has one dual-port drive channel for a total of two drive ports per controller. • If a failure occurred in drive channel 1, channel 2 allows communication with the drives and vice versa.
Connecting the Drive Loop Cables In Figure 5-2, callout 1 shows the location of ESM A (inverted), callout 2 shows ESM B. Callout 3 shows ports 1A (In) and 1B (Out), while callout 4 shows ports 2A/2B (which are reserved). Figure 5-2 Drive Expansion ESM Ports Note that drive enclosures connected to the same drive channel must operate at the same speed. Connecting the Drive Loop Cables The system is designed to support redundant drive loops.
5: Cabling To complete these procedures, you will need at least two interface cables (depending on the number of drive expansion enclosures in the array). 1. Connect the drive loop in the first two enclosures (see Figure 5-3). Starting with the controller enclosure, connect an interface cable from the drive loop connector on the left RAID controller module to the drive channel in connector on the left ESM. Figure 5-3 RAID Controller With Single Connection Drive Cabling Example 2.
Connecting the Drive Loop Cables 3. To cable two drive expansion enclosures, use Table 5-2 and see Figure 5-4.
5: Cabling 4. For additional drive enclosures with a dual controller RAID system, use the information in Table 5-3 that lists the connection points and (see Figure 5-5).
Connecting the Drive Loop Cables Drive expansion 4 Drive expansion 2 RAID enclosure Drive expansion 1 Drive expansion 3 Figure 5-5 007-4834-001 Drive Loop Cabling Example (Four Expansion Enclosures) 69
5: Cabling 5. Connect a RAID controller enclosure to five drive expansion enclosures using a loop (see Figure 5-6). Table 5-4 lists the connection points illustrated in the figure.
Connecting the Drive Loop Cables Drive expansion 4 Drive expansion 2 RAID enclosure Drive expansion 1 Drive expansion 3 Drive expansion 5 Figure 5-6 007-4834-001 Drive Loop Cabling for Five Expansion Enclosures 71
5: Cabling 6. Connect a RAID controller enclosure to six drive expansion enclosures using a loop (see Figure 5-7). Table 5-5 lists the connection points illustrated in the figure.
Connecting the Drive Loop Cables Drive expansion 6 Drive expansion 4 Drive expansion 2 RAID enclosure Drive expansion 1 Drive expansion 3 Drive expansion 5 Figure 5-7 007-4834-001 Drive Loop Cabling for Six Drive Expansion Enclosures 73
5: Cabling Connecting the System to the Host(s) You can connect your system to a host computer (or multiple host computers) in many ways. The following figures show typical examples of connecting either by direct connect or fabric connect. All examples assume dual-controller high-availability (HA) configurations.
Connecting the System to the Host(s) Figure 5-8 shows a possible configuration for direct dual-HBA connection to a host system with HBA failover. As shown, RAID controller A (callout 1) is connected to HBA 1 in the host, and RAID controller B (callout 2) is connected to HBA 2 in the same host.
5: Cabling Switched-Host Connection With HBA Failover Figure 5-9 shows a possible configuration for switch-connected systems with HBA failover capability. Callout 1 shows the location of RAID controller A, callout 2 shows the location of RAID controller B.
Connecting the System to the Host(s) Example Fabric/Switch Host Connection Cabling Figure 5-10 shows an example of connecting the InfiniteStorage 4000 to multiple hosts and HBAs using two Fibre Channel switches. For more information, see the documentation shipped with the Fibre Channel switch. Figure 5-10 shows a dual-switch configuration. RAID controller A (callout 1) is directly connected to one host’s HBA 1 and switch connected to two additional host’s HBA 1 boards.
5: Cabling Out-of-band management requires that Ethernet or serial connections are cabled to each controller. The controllers are managed directly from a management station through the connections on each controller. In-band management requires that only the Fibre Channel host interface is present and functional. The controllers are managed from a management station through a host using the I/O connections between the host and the controllers. See Figure 5-12 on page 80 for an example cabling diagram.
Connecting the System to the Host(s) 1. To cable the system for out-of-band management via Ethernet, connect a pair of Ethernet interface cables from the storage management station to the Ethernet connectors for controller A (left) and controller B (right) on the rear of the controller enclosure (see Figure 5-11). 2.
5: Cabling Figure 5-12 80 In-Band Management Cabling Example 007-4834-001
Powering On the System Powering On the System To connect the controller and the expansion enclosures to the AC power source and power on the system, follow these steps: Warning: Electrostatic discharge can damage sensitive components. Use appropriate antistatic precautions before handling any components. 1. Make sure the switches on the power distribution units (on the rear of the rack inside the access panel) are in the OFF position. 2.
5: Cabling Figure 5-14 Redundant Power Cabling Example Note: To speed drive spin-up, it is recommended that you start the expansion enclosures before or at the same time as the controller enclosure. This procedure powers up all components in the system at the same time by turning on the power distribution units. 6. Turn on the power switches on the rear of each expansion enclosure. Then turn on both power switches on the rear of the RAID controller enclosure (see Figure 5-13). 7.
Appendix A A. Specifications and Requirements This appendix describes the technical specifications and requirements of the SGI InfiniteStorage 4000 system rack and components in the following sections: • “Rack Specifications” on page 83 • “Enclosure Specifications” on page 93 Rack Specifications The system is housed in a 22-inch wide rack that contains two AC distribution boxes. Figure A-1 shows an example of the rack’s power connection boxes at the base of the unit.
A: Specifications and Requirements Figure A-1 Rack Power Connection Example Rack Dimensions Figure A-2 shows the system rack and its dimensions. Note: Make sure the cabinet is installed where it meets the minimum clearance requirements of 76-cm (30-in) in front of the cabinet and 61-cm (24-in) in back of the cabinet.
Rack Specifications Figure A-2 007-4834-001 Dimensions of the InfiniteStorage 4000 RAID System Rack 85
A: Specifications and Requirements Weight The total weight of the system depends on the quantity of enclosures installed. Table A-1 lists the overall weight of the rack, the maximum weight of an enclosure and the approximate weight of a fully loaded rack. You can use these weights to estimate the total weight of your system, based on the number of enclosures installed in the rack. Figure A-3 provides an example of calculating the weight of a rack.
Rack Specifications Empty rack weight 139.2 kg (307 lbs) 7 enclosures (7 x 38.5 kg)= 269.5 kg (592.9 lbs) Total weight uncrated 408.7 kg (899.
A: Specifications and Requirements Area Requirements The floor area at the installation site must provide the following: • Enough stability to support the weight of the system and installed devices (see Table A-1). • Sufficient space to install and service the rack and components (see Figure A-4). Required service area 61 cm (24 in.) Computer floor grid Rear Cable access 229 cm (90 in.) Caster Stability foot Front Required service area 76 cm (30 in.) 48220 56 cm (22 in.
Rack Specifications Wiring and Power The rack’s AC distribution boxes use common industrial wiring. Consider the following site wiring and power source requirements: • AC power source: The AC power source must provide the correct voltage, current, and frequency specified on the manufacturer’s name plate. • Earth ground: You must have an earth grounding conductor to the rack’s power receptacles.
A: Specifications and Requirements Power Requirements Table A-2 lists the power requirements for the rack. Table A-2 AC Power Requirements (Domestic and International) Unit of Measure Requirement AC distribution, 250 VAC, 16 A Approved IEC 320-C19 connector Domestic, 250 VAC, 30 A NEMA L6-30P locking plug, 6-30R receptacle (2) International, 230 VAC, 32 A IEC 309 locking plug; IEC 309 receptacle (2) Voltage range 180 to 257 VAC Frequency 49 to 50.5 Hz or 59 to 60.
Rack Specifications (USA and Canada) or worldwide (excluding USA and Canada) components. Each AC power distribution box includes the following parts: • Two cords per side, NEMA L6-30P or IEC309 • Four circuit breakers that are 24 A • Twenty IEC320 power outlets per side, plus an additional outlet for the optional fan tray Figure A-5 shows the connectors and receptacles for these cords.
A: Specifications and Requirements Environmental Table A-3 list the environmental requirements for the InfiniteStorage 4000 RAID system. ! Caution: If you receive the rack and enclosures in cold weather (below 32 ºF (0 ºC)), leave them crated for at least 24 hours to prevent condensation. This 24-hour stabilization period can be modified either up or down, depending on the outside temperature at arrival.
Enclosure Specifications Enclosure Specifications This section lists the dimensions and weights, as well as the wiring, power, environmental, and airflow requirements for the enclosures. Dimensions Make sure that the installation site provides a minimum of two feet of space around each side of the rack to install and service the controller enclosure, and to allow adequate ventilation during operation. Figure A-6 shows the dimensions for the RAID enclosure.
A: Specifications and Requirements Weights Ensure that the floor space at the installation site has sufficient stability to support the maximum weight of the controller enclosure and associated equipment. The controller can hold two to 16 drives and the enclosure’s total weight depends on the number of drives and other modules in the enclosure. Table A-4 and Table A-5 lists the unit and shipping weight of the controller enclosure and the weight of the individual modules.
Enclosure Specifications Wiring and Power The enclosure is a 120/220 VAC, 50/60 Hz unit that meets standard voltage requirements for both domestic (USA) and international operation. It uses standard industrial wiring with a line-to-neutral power connection (see Table A-6). Review the following specifications when preparing the controller enclosure installation site: • Earth ground: The unit must be properly grounded, including an earth ground conductor on the AC power source.
A: Specifications and Requirements Environmental Table A-7 and Table A-8 lists the environmental requirements for the system enclosure. Table A-7 Altitude Requirements for System Enclosures Condition Range Requirement below Sea Level Requirement above Sea Level Altitudea Operating Storage Transit 30.5 m (100 ft.) 30.5 m (100 ft.) 30.5 m (100 ft.) 3,048 m (10,000 ft.) 3,048 m (10,000 ft.) 12,000 m (40,000 ft.) a. If planning to operate the enclosure at altitudes between 1,000 m (3,280 ft.
Enclosure Specifications System Enclosure Air Flow Figure A-7 shows the air flow in the system enclosure. Make sure your installation site will allow adequate ventilation during operation. Important: Allow a minimum of two feet of clearance in front of and behind the controller enclosure for proper ventilation and servicing.
Appendix B B. SGI Field Engineering Compliance Statements Electromagnetic Emissions The equipment described in this guide complies with the Class A limits of Part 15 of the FCC Rules. Operation is subject to the following two conditions: 1. This device may not cause harmful interference. 2. This device must accept any interference received, including interference that may cause undesired operation.
B: SGI Field Engineering Compliance Statements are designed to provide reasonable protection against such interference in an industrial or office installation. However, there is no guarantee that the interference will not occur in a particular installation. This system is not certified for home use. You can determine whether your system is causing interference by turning it off. If the interference stops, it was probably caused by the system.
Shielded Cables It is important that while you are operating this equipment you keep all the covers and doors, including the plastics, in place. The shielded cables that came with the system and its peripherals should be installed correctly, with all thumbscrews fastened securely. An ESD wrist strap is included with some products, such as memory and graphics upgrades.
B: SGI Field Engineering Compliance Statements If this equipment does cause harmful interference to radio or television reception, which can be determined by turning the equipment off and on, try to correct the interference with one or more of the following measures: ! • Reorient or relocate the receiving antenna • Increase the separation between the equipment and the receiver • Connect the equipment into an outlet on a circuit different from that to which the receiver is connected • Consult the d
International Special Committee on Radio Interference (CISPR) International Special Committee on Radio Interference (CISPR) The equipment described in this guide has been tested to and is in compliance with the Class A limits per CISPR publication 22. Canadian Department of Communications Statement This digital apparatus does not exceed the Class A limits for radio noise emissions from digital apparatus as set out in the Radio Interference Regulations of the Canadian Department of Communications.
