HP P6000 Continuous Access Implementation Guide Abstract This guide explains the major factors in designing a successful disaster recovery solution using HP P6000 Continuous Access. In addition to explaining how distance and bandwidth affect performance and cost, this guide describes optional configurations and key planning for your operating systems, applications, and arrays.
© Copyright 2008, 2012 Hewlett–Packard Development Company, L.P. Confidential computer software. Valid license from HP required for possession, use or copying. Consistent with FAR 12.211 and 12.212, Commercial Computer Software, Computer Software Documentation, and Technical Data for Commercial Items are licensed to the U.S. Government under vendor's standard commercial license. Warranty The information contained herein is subject to change without notice.
Contents 1 HP P6000 Continuous Access .....................................................................7 Features..................................................................................................................................7 Remote replication concepts.......................................................................................................8 Write modes ......................................................................................................................
HP Cluster Extension ..........................................................................................................26 HP Metrocluster Continuous Access......................................................................................27 HP Continentalcluster..........................................................................................................27 Reduced-availability configurations...........................................................................................
H-series switch configuration...........................................................................................54 M-series switch configuration..........................................................................................54 Verifying cabling...............................................................................................................54 Changing host port data replication settings..........................................................................
Protecting data from a site failure............................................................................................116 8 Operating system procedures...................................................................118 Resuming host I/O after failover.............................................................................................118 HP OpenVMS.................................................................................................................118 HP Tru64 UNIX...........
1 HP P6000 Continuous Access HP P6000 Continuous Access is the remote replication component of HP controller software. When this component is licensed and configured, the controller copies data online, in real time, to a remote array over a SAN. Properly configured, HP P6000 Continuous Access provides a disaster-tolerant storage solution that ensures data integrity and, optionally, data currency RPO if an array or site fails.
• Support for array-to-array fan-in and fan-out • HP SCSI FC Compliant Data Replication Protocol (HP SCSI-FCP), a full SCSI protocol implementation that takes advantage of the exchange-based routing available in fabric switches. For more information, see “Planning the data replication protocol” (page 38). See the HP P6000 Enterprise Virtual Array Compatibility Reference for more information on remote replication support by controller software version.
Figure 2 DR group replication 1. Host server 6. Destination array 2. Fibre Channel switch 7. Source virtual disk 3. Host I/O 8. Destination virtual disk 4. Replication writes 9. DR group 5. Source array DR group write history log The DR group write history log is a virtual disk that stores a DR group's host write data. The log is created when you create the DR group. Once the log is created, it cannot be moved. For more information, see “Planning for DR group write history logs” (page 42).
NOTE: Managed sets are simply a feature that enables you to manage multiple resources easily. They do not contribute to the data consistency of a DR group. Write order consistency is maintained at the DR group level. In managed sets: • All resources, or members, in a single managed set must be of the same type (for example, all virtual disks). • You can add a specific resource to more than one managed set. • You can add resources on more than one array to a managed set.
NOTE: Failsafe mode is available only in synchronous write mode. Host I/O can be recovered by changing affected DR groups from failsafe-enabled mode to failsafe-disabled mode. This action will begin logging of all incoming writes to the source member of the Data Replication group.
Table 1 HP P6000 Continuous Access implementation checklist (continued) Appropriate tool for gathering bandwidth data. See “Tools for gathering SAN data” (page 14). __ Evaluate all factors impacting selection of the ISL. __ Define the RPO. __ Select a write mode that supports the RPO. __ Verify that the ISL meets bandwidth and Quality of Service (QoS) requirements for HP P6000 Continuous Access.
Table 1 HP P6000 Continuous Access implementation checklist (continued) __ Select a failover and recovery procedure. __ Observe all operating system-specific failover and recovery procedures. __ Test failover and recovery.
2 Designing a remote application solution This chapter describes important factors involved in designing a successful remote replication solution using HP P6000 Continuous Access: • “Choosing the remote site” (page 14) • “Choosing the intersite link” (page 17) • “Choosing a write mode” (page 20) Tools for gathering SAN data A critical task in designing your HP P6000 Continuous Access implementation is gathering and analyzing the characteristics of your current SAN operating environment.
High availability High availability reduces the risk of downtime through redundant systems, software, and IT processes with no SPOF. HP P6000 Continuous Access contributes to high availability by providing redundant data. Products such as HP Metrocluster and HP Cluster Extensions provide highly available applications. For more information about these and other high-availability products, see the HP website: http://docs.hp.com/en/ha.
Figure 3 Threat categories based on radius 1. Regional threat (radius between 10 and 100 kilometers, affecting up to 314,000 square kilometers) 2. Metropolitan threat (radius between 1 and 10 kilometers, affecting up to 314 square kilometers) 3. Local threat (radius less than 1 kilometer, affecting 3 square kilometers) When determining the threat radius, identify the threats to both sites and the specific threat range.
Table 2 (page 17) lists the intersite one-way latency inherent to other distances. Table 2 Sample one-way latencies One-way latency (ms) Point-to-point cable distance in km (miles) 1 200 (125) 3 600 (375) 9 1,800 (1,125) 18 3,600 (2,250) 36 7,200 (4,500) 60 12,000 (7,500) 100 20,000 (12,500) current maximum limit Determining intersite latency To determine intersite latency on an existing network, use network utilities such as the ping command. Obtain a 24-hour average.
• Fiber with WDM supports up to 500 kilometers. • FC-to-IP gateways support the longest distances. For detailed descriptions of supported link technologies, see “Planning the remote replication fabric” (page 23). NOTE: Regardless of the transmission technology, HP P6000 Continuous Access does not take into account the type of media used for the intersite network connection.
rate based on the RPO (or over the RPO interval). Insufficient replication bandwidth impacts user response time, RPO, or both. Determining the critical sample period Working with large measurement samples can be tedious and problematic. A two-stage approach to data collection generally helps to reduce the effort. In the first stage, the historical write byte rate trends are analyzed to determine peak periods that can occur during monthly or yearly business cycles and daily usage cycles.
-millisecond average response time budget for latency sensitive applications such as Microsoft Exchange. Sizing bandwidth for asynchronous replication HP P6000 Continuous Access includes an enhanced buffering technique for asynchronous replication solutions that utilizes the DR group write history log. This disk-based journaling insulates application users from latency delays caused by propagation delays and intermittent congestion on the link.
NOTE: • A request to convert from synchronous mode to asynchronous mode is executed immediately. A request to convert from asynchronous mode to synchronous mode is executed after the data in the DR group write history log or the I/Os in the write pending queue are merged to the destination array. • During the conversion from enhanced asynchronous mode to synchronous mode, an I/O throttling mechanism allows one new write to a DR group for every two DR group write history log entry merges.
5. When the source array receives the acknowledgment from the target array, it removes the data from the write history log. Synchronous mode In synchronous write mode, the source array acknowledges I/O completion after replicating the data on the destination array. Synchronous replication prioritizes data currency over response time. 1. A source array controller receives data from a host and stores it in cache. 2. The source array controller replicates the data to the destination array controller. 3.
3 Planning the remote replication fabric This chapter describes the supported HP P6000 Continuous Access configurations. Basic dual-fabric configuration Figure 5 (page 23) shows the basic HP P6000 Continuous Access configuration with Fibre Channel links. Hosts (5) have two HBAs, one connected to the blue fabric (6) and the other connected to a redundant gold fabric (8). Arrays have two controllers, each connected to both fabrics for a total of four connections.
You can use switches to create zones and work around some fabric limitations. For example: • Unsupported hosts and incompatible arrays can be on a fabric with HP P6000 Continuous Access if they are in independent zones. For compatibility with operating systems and other software, see the HP P6000 Enterprise Virtual Array Compatibility Reference. • A fabric can include multiple HP P6000 Continuous Access solutions. For more information about zoning, see “Creating fabrics and zones” (page 58).
When using WDM, consider the following: • WDM installation must conform to vendor specifications. • Performance is affected by extreme distance and/or limited buffer-to-buffer credits on the Fibre Channel switch. • Some switch vendors may limit the maximum distance between sites. Additional configuration rules apply to WDM configurations: • Connecting the switch to the WDM unit typically requires one switch-to-WDM interface cable per wavelength of multimode fiber.
Contact a third-party vendor to acquire and install all SMF optic cables, any MMF optic cables longer than 50 meters, and the FC-to-IP interface boxes. Configurations with application failover The configurations in this section include host software that works with HP P6000 Continuous Access to provide application and data failover capability.
4. Management server 10. Dual-controller arrays 5. Hosts 11. Cluster site 6. Host I/O and replication fabric—blue HP Metrocluster Continuous Access HP P6000 Continuous Access supports a ServiceGuard cluster running on HP-UX 11i v1 or HP-UX 11i v2 Update 2. Also known as HP-UX Metrocluster Continuous Access, this configuration has half the cluster in each of two data centers and uses HP P6000 Continuous Access to replicate data between the data centers.
NOTE: When creating an intersite FCIP link using B-series or C-series routers, the respective LSAN and IVR functionality can provide SAN traffic routing over the FCIP connection while preventing the merging of the two sites' fabrics in to a single fabric. LSANs and IVR enable logical fabric separation of the two sites, ensuring that a change on one site's fabric does not affect the other site. The HP FCIP Distance Gateways (MPX110) will allow the fabrics on both sites to merge into a single large fabric.
Figure 8 Single-switch configuration 1. LAN connection 4. Hosts 2. Switch 5. Dual-controller arrays 3. Management server In this example, two hosts might be clustered together using a supported cluster technology for the operating system. The third host would be a single server running the same operating system as the clustered hosts, and therefore available as a backup to the cluster.
NOTE: • For more information about the connections used to implement this configuration, see “A single physical fabric” (page 66). The five fabric configurations can be physically separate fabrics (see Figure 24 (page 62)) or a single physical fabric zoned into five logical fabrics using switch zoning (see Figure 27 (page 66)).
Figure 10 Five-fabric configuration with FC-to-IP 1. Data center 1 7. Host I/O fabric—gold 2. Data center 2 8. Dedicated replication fabric 3. LAN connection 9. Intersite link 4. Management server 10. Dual-controller arrays 5. Hosts 11. FC-to-IP 6.
Six-fabric configuration The six-fabric configuration shown in Figure 11 (page 33) consists of two fabrics that are dedicated to replication and four fabrics that are dedicated to I/O between the hosts and arrays. Figure 12 (page 34) shows the same configuration using FC-to-IP for the replication fabrics. NOTE: • For more information about the connections used to implement this configuration, see “Dual physical fabric with six zones” (page 67).
Figure 11 Six-fabric configuration 1. Data center 1 7. Host I/O fabric—gold 2. Data center 2 8. Dedicated replication fabric—gold 3. LAN connection 9. Intersite link—gold fabric 4. Management server 10. Dedicated replication fabric—blue 5. Hosts 11. Intersite link—blue fabric 6. Host I/O fabric—blue 12.
Figure 12 Six-fabric configuration with FC-to-IP 34 1. Data center 1 8. Dedicated replication fabric—gold 2. Data center 2 9. Intersite link—gold fabric 3. LAN connection 10. Dedicated replication fabric—blue 4. Management server 11. Intersite link—blue fabric 5. Hosts 12. Dual-controller arrays 6. Host I/O fabric—blue 13. FC-to-IP—gold replication fabric 7. Host I/O fabric—gold 14.
4 Planning the array configuration This chapter provides an overview of factors to consider when planning an array configuration for remote replication. Many remote replication features depend on the array controller software. For more information about planning, see the HP Enterprise Virtual Array Configuration Best Practices White Paper for your array model. Planning disk groups Planning the necessary disk groups to meet your I/O requirements should be done when configuring the array.
Planning DR groups Virtual disks that contain the data for one application must be in one DR group. For optimum failover performance, limit the virtual disks in a DR group to as few as possible. DR group guidelines The following guidelines apply for DR groups: 36 • Source and destination arrays must have remote replication licenses. • The maximum number of virtual disks in a DR group and the maximum number of DR groups per array vary with controller software versions.
To be added to a DR group, a virtual disk: • Cannot be a member of another DR group • Cannot be a snapshot • Cannot be a mirrorclone • Must be in a normal operational state • Must use mirrored cache CAUTION: Before replicating a Vraid0 source Vdisk or creating a Vraid0 remote copy, consider the limitations of Vraid0. Although Vraid0 offers the most compact storage of your data, it carries no data redundancy.
Increasing the size of the write history log file in enhanced or basic asynchronous mode You can expand the size of a DR group member (virtual disk) whether the DR group is in synchronous or asynchronous mode. However, when you expand the size of a virtual disk in a DR group operating in enhanced asynchronous or basic asynchronous mode, the write history log file size does not increase. See the HP P6000 Enterprise Virtual Array Compatibility Reference.
NOTE: With HP B-series switches the SID/DID protocol is known as Port-based routing; with HP C-series switches the SID/DID protocol is known as Flow based load balancing. Both B-series and C-series switches use exchange-based routing terminology for SID/DID and originator exchange ID (OXID) routing. The original replication protocol used with HP P6000 Continuous Access is HP FC Data Replication Protocol (HP-FC). This protocol uses multiple exchanges for each data replication transfer.
NOTE: An array running XCS version 09534000 or later with the protocol selection set for Either may have DR groups created with arrays configured for HP-FC or HP SCSI-FCP. However, the fabric must be set to SID/DID routing protocol only. The three available options are shown in “Data replication protocol selection” (page 40). The window indicates the protocol currently selected.
also assumed that the packet loss and network latency jitter on the networks falls within HP’s defined acceptable range for HP P6000 Continuous Access replication. • HP currently supports three FCIP router families: B-series, C-series, and HP IP Distance Gateway. The FCIP configuration can have either single or dual long-distance network links between the local and remote gateways.
If tunnel thrash occurs, perform the following tasks to resolve the situation and return to normal operation: • Check all switches and routers to determine if there are high volumes of packet loss. • Ensure that all switches and routers are configured correctly. • Contact your service provider to determine if the circuit routing has been changed. • Determine if tunnel thrash only occurs during periods of peak activity.
to reduce the log size. When the log content has been drained, you can return the DR group to enhanced asynchronous mode. Until the DR group is returned to enhanced asynchronous mode, the DR group operates in synchronous mode, which may impact performance. Allocated log file space is not decreased when DR group members are removed.
Write history log size in synchronous or basic asynchronous mode When using synchronous mode or basic asynchronous mode, if logging occurs because replication has been suspended or the replication links have failed, the size of the log file expands in proportion to the amount of writes. The size of the log file can increase only up to the user-specified maximum value or to the controller's software default maximum value. The size of the log can't be changed while in basic asynchronous mode.
log group assignment and specify the location of the log when you create the DR group. For version-specific features, see the HP P6000 Enterprise Virtual Array Compatibility Reference. IMPORTANT: When using XCS 6.000 or later, create the DR group write history log using online Fibre Channel disks, not near-online disks. Constant writes to the DR group write history log in enhanced asynchronous mode significantly shorten the expected lifetime of near-online disks.
Table 4 Default DR group write history log placement Array status VCS, XCS 6.000 or earlier XCS 6.000 or later XCS 09003000 or later The array contains one defined disk group. Use the defined disk group. Use the defined disk group. The array contains one near-online disk group and one online disk group. Use the near-online disk group. Use the online disk group. The array contains only multiple near-online disk groups.
System fan-out replication In the system fan-out replication shown in Figure 14 (page 47), one DR group is replicated from array A to array B, and another DR group is replicated from array A to array C. CAUTION: In a mixed array environment that includes an EVA3000/5000 array, the host ports of the fan-in target or the fan-out source should be isolated. This is necessary because an EVA3000/5000 has fewer resources to handle inter-array replication traffic.
Fan-in replication In the fan-in replication shown in Figure 15 (page 48), one DR group is replicated from array A to array C, and another DR group is replicated from array B to array C. Figure 15 Fan-in replication 1. Array A 2. Array B 3.
Cascaded replication In cascaded replication, one DR group is replicated from array A to array B, and another DR group is replicated from array B to array C. In this configuration, the source disk for replication from the array B to array C is a snapclone of the destination disk in the replication from array A to array B. See Figure 16 (page 49). Snapclone normalization must complete on array B before the new snapclone can be put in a new DR group. Figure 16 Cascaded replication 1. Array A 2. Array B 3.
5 Planning the solution This chapter describes general design considerations for the different operating systems, applications, and storage management components that you can use when planning a remote replication solution. Operating system considerations This section describes the operating systems supported in remote replication solutions. It also describes the operating system capabilities that are available in an HP P6000 Continuous Access environment.
6 Implementing remote replication This chapter describes the basic steps for setting up HP P6000 Continuous Access. Remote replication configurations There are a number of options for configuring your solution to support remote replication. For detailed information on remote replication configurations, see “Planning the remote replication fabric” (page 23).
B-series switch configuration The following routing policies are available on B-series switches: • Exchange-based routing—The routing path is based on the SID, DID, and OXID optimizing path utilization for the best performance. Each SCSI exchange can take a different path through the fabric. Exchange-based routing requires using the Dynamic Load Sharing (DLS) feature. When this policy is in effect, you cannot disable DLS.
C-series switch settings for the HP-FC protocol NOTE: The following configuration must be used in an HP P6000 Continuous Access environment for all Fibre Channel switches in the path from the source array to the destination array. This includes FCIP routers that are connected to the fabric. Execute the following commands to establish the required switch settings: 1. config terminal 2. (config)# in-order-guarantee (enable in-order delivery) 3. (config)# vsan database 4.
12. show fcip summary (review your settings) • When using FCIP, the TCP send buffers must be set to 4,096 K on all FCIP profiles on both the source and destination sites. One exception is a solution using a link less than OC3 (155 Mb/s), in which case buffers must be set to 8,192 K (tcp send-buffer-size 8192). H-series switch configuration H-series switches only support either the HP IP Distance Gateway or the MPX200 Multifunction Router.
Figure 17 Cabling for arrays with two-port controllers Figure 18 Cabling for EVA4400 with two-port controllers Figure 19 (page 55) shows the standard cabling scheme for remote replication on arrays with four-port controllers. Each controller has redundant connections to both fabrics. Even-numbered ports are connected to one fabric and odd-numbered ports are connected to the other fabric.
Changing host port data replication settings NOTE: • Manually setting host port preferences should be done carefully and only when absolutely necessary. Using this feature requires advanced knowledge of array operation to ensure the expected results are achieved. • Host port replication settings can be managed using HP P6000 Command View, HP P6000 Replication Solutions Manager, or the HP Storage System Scripting Utility.
Figure 20 Host port data replication settings Verifying path status The current remote connection path status can be checked from HP P6000 Command View. In the navigation pane, select the Data Replication folder for the storage system. The Data Replication Folder Properties page opens. Click View remote connections to view the Remote System Connections Status page. To view additional information, click the path details icon.
Installing replication licenses When you purchase HP P6000 Continuous Access, you receive a replication license for each array (local and remote) in a remote replication relationship. Replication licenses are based on the amount (in TB) of replicated data on each array. License kits include instructions for retrieving electronic license keys and for installing the keys in HP P6000 Command View.
Two-fabric configuration Figure 21 (page 59) through Figure 23 (page 61) show two-fabric connection configurations. In these configurations each fabric is used for both host I/O and replication. • Figure 21 (page 59) shows the basic two-fabric connections. • Figure 22 (page 60) shows the two-fabric connection used between an EVA3000/5000 (four–port controller pair) and an EVA8x00 (eight–port controller pair). These connections are also used for the EVA4400 controller without the embedded FC switch.
Figure 22 Two-fabric configuration (EVA3000/5000 to EVA8x00) 60 1. Data center 1 6. Fibre Channel switches 2. Data center 2 7. Host I/O and replication fabric 3. Array controller pair 8. Intersite link 4. Management server 9. Host I/O and replication fabric 5. Hosts 10.
Figure 23 Two-fabric configuration (EVA4400 with embedded Fibre Channel switch) 1. Data center 1 6. Replication fabric 2. Data center 2 7. Intersite link 3. Array controller pair with embedded Fibre Channel switches 8. Replication fabric 4. Management server 9. Intersite link 5. Hosts NOTE: • On an EVA4400 with the embedded switch option, the connections between controller host ports FP1 and FP2 and controller switch ports 0 and 11 are internal to the controller and cannot be changed.
Figure 24 Five-fabric configuration 62 1. Data center 1 6. Fibre Channel switches 2. Data center 2 7. Host I/O fabric 3. Array controller pair 8. Host I/O fabric 4. Management server 9. Replication fabric 5. Hosts 10.
Figure 25 Six-fabric configuration (eight–port controller pairs) 1. Data center 1 7. Host I/O fabric 2. Data center 2 8. Replication fabric 3. Array controller pair 9. Intersite link 4. Management server 10. Host I/O fabric 5. Hosts 11. Replication fabric 6. Fibre Channel switches 12.
Figure 26 Six-fabric configuration (four–port controller pair to eight–port controller pair) 1. Data center 1 7. Host I/O fabric 2. Data center 2 8. Replication fabric 3. Array controller pair 9. Intersite link 4. Management server 10. Host I/O fabric 5. Hosts 11. Replication fabric 6. Fibre Channel switches 12. Intersite link The new configuration connections are shown in Figure 27 (page 66), Figure 28 (page 68), and Figure 29 (page 69).
HBA ports are now zoned to use the same ports on the array controller as other server traffic, and are also zoned to see the remote array across the replication fabric. Using switch zoning, you can separate host I/O traffic from HP P6000 Continuous Access traffic in each data center.
A single physical fabric Figure 27 (page 66) shows a single physical fabric zoned into five logical fabrics. Four zones are dedicated to host I/O, and a single zone is used for replication. NOTE: To compensate for the SPOF in a five-zone solution (the intersite link [10]), HP recommends that availability be a QoS metric on the intersite link. Figure 27 Single physical fabric with five zones 1. Data center 1 6. Fibre Channel switches 2. Data center 2 7. Host I/O zone1 3. Array controller pair 8.
Dual physical fabric with six zones Figure 28 (page 68) shows a dual physical fabric with six zones used between eight–port controller pairs. Four zones are dedicated to host I/O and two zones are dedicated to replication. Figure 29 (page 69) shows the connections used when one four-port controller pair is used. NOTE: • In a dual physical fabric with a six-zone configuration, each HP P6000 Continuous Access relationship must include at least one EVA8x00 or one EVA6400.
Figure 28 Dual physical fabric with six zones (eight-port controller pairs) 68 1. Data center 1 7. Host I/O zone1 2. Data center 2 8. Replication zone 3. Array controller pair 9. Intersite link 4. Management server 10.
5. Hosts 11. Replication zone 6. Fibre Channel switches 12. Intersite link 1 The zones depicted in the figure above as yellow or green should not be zoned across the sites. Each site should have a unique yellow and green zone. Figure 29 Dual physical fabric with six zones (four-port controller pair to eight-port controller pair) 1. Data center 1 7. Host I/O zone1 2. Data center 2 8. Replication zone 3. Array controller pair 9. Intersite link 4. Management server 10. Host I/O zone1 5.
• When adding array controllers to a zone, use the controller port World Wide Port Name (WWPN), not the node WWN. For example, use the WWPN 5000-1FE1-001A-012A, not the WWN 5000-1FE1-001A-0120. • When creating a replication zone, include only arrays that have an HP P6000 Continuous Access relationship with each other. Avoid including other arrays in the zone. • Zoning allows you to enforce supported maximum visible connections.
are placed into the configuration, separate replication zones should be created for those pairs of arrays. If array system fan-out is required, then the additional array should be added to a new replication zone. Only arrays that will participate in data replication relationships should be zoned together. Additional arrays (up to supported limits) can be added to either site. See the HP P6000 Enterprise Virtual Array Compatibility Reference for the number of arrays supported for system fan-out.
IMPORTANT: This reduced availability configuration is not recommended for production environments requiring high availability. The single switch on each site represents an SPOF for the host I/O and the HP P6000 Continuous Access replication traffic. 72 • Single-fabric components—Figure 30 (page 73) shows the various components that comprise the single-fabric topology. • Host I/O zones—Figure 31 (page 74) and Figure 32 (page 75) show the ports used to create the host I/O zones.
Figure 30 Single-fabric zoning components 1. Site 1 Array controller pair 6. Site 2 Fibre Channel switch 2. Site 1 Fibre Channel switch 7. Site 2 management serve 3. Site 1 host 8. Site 2 host 4. Site 1 management server 9. Intersite link 5.
Figure 31 Single-fabric host I/O zones: sheet 1 74 Zone 1 ports Zone 2 ports Site 1 host port HBA1 Site 2 host port HBA1 Site 1 Array controller A port FP1 Site 2 Array controller A port FP1 Site 1 Array controller B port FP1 Site 2 Array controller B port FP1 Implementing remote replication
Figure 32 Single-fabric host I/O zones: sheet 2 Zone 3 ports Zone 4 ports Site 1 host port HBA2 Site 2 host port HBA2 Site 1 Array controller A port FP1 Site 2 Array controller A port FP1 Site 1 Array controller B port FP1 Site 2 Array controller B port FP1 Creating fabrics and zones 75
Figure 33 Single-fabric HP P6000 Command View local management zones 76 Zone 5 ports Zone 6 ports Site 1 management server port HBA1 Site 2 management server port HBA1 Site 1 management server port HBA2 Site 2 management server port HBA2 Site 1 Array controller A port FP1 Site 2 Array controller A port FP1 Site 1 Array controller B port FP1 Site 2 Array controller B port FP1 Implementing remote replication
Figure 34 Single-fabric HP P6000 Command View remote management zones Zone 7 ports Zone 8 ports Site 1 management server port HBA1 Site 2 management server port HBA1 Site 1 management server port HBA2 Site 2 management server port HBA2 Site 2 Array controller A port FP1 Site 1 Array controller A port FP1 Site 2 Array controller B port FP1 Site 1 Array controller B port FP1 Creating fabrics and zones 77
Figure 35 Single-fabric replication zone Zone 9 ports Site 1 Array controller A port FP2 Site 1 Array controller B port FP2 Site 2 Array controller A port FP2 Site 2 Array controller B port FP2 78 Implementing remote replication
Figure 36 Single-fabric replication zone fan-out: sheet 1 Zone 10 ports Site 1 Array (1) controller A port FP2 Site 1 Array (1) controller B port FP2 Site 1 Array (10) controller A port FP2 Site 1 Array (10) controller B port FP2 Creating fabrics and zones 79
Figure 37 Single-fabric replication zone fan-out: sheet 2 Zone 11 ports Site 1 Array (10) controller A port FP2 Site 1 Array (10) controller B port FP2 Site 2 Array (5) controller A port FP2 Site 2 Array (5) controller B port FP2 Recommended dual-fabric zoning configurations This is a high availability configuration consisting of two switches per site, which creates two distinct extended fabrics that span the two sites. This configuration eliminates the SPOF for host I/O traffic.
Figure 38 Dual-fabric zoning components 1. Site 1 Array controller pair 7. Site 2 host 2. Site 1 host 8. Site 2 management server 3. Site 1 management server 9. Site 2 Fibre Channel switch 4. Site 1 Fibre Channel switch 10. Site 2 Fibre Channel switch 5. Site 1 Fibre Channel switch 11. Intersite link 6. Site 2 Array controller pair 12.
Figure 39 Dual-fabric host I/O zones: sheet 1 82 Zone 1 ports Zone 2 ports Site 1 host port HBA1 Site 2 host port HBA1 Site 1 Array controller A port FP1 Site 2 Array controller A port FP1 Site 1 Array controller B port FP1 Site 2 Array controller B port FP1 Implementing remote replication
Figure 40 Dual-fabric host I/O zones: sheet 2 Zone 3 ports Zone 4 ports Site 1 host port HBA2 Site 2 host port HBA2 Site 1 Array controller A port FP2 Site 2 Array controller A port FP2 Site 1 Array controller B port FP2 Site 2 Array controller B port FP2 Creating fabrics and zones 83
Figure 41 Dual-fabric HP P6000 Command View local management zones: sheet 1 84 Zone 5 ports Zone 6 ports Site 1 management server port HBA1 Site 2 management server port HBA1 Site 1 Array controller A port FP1 Site 2 Array controller A port FP1 Site 1 Array controller B port FP1 Site 2 Array controller B port FP1 Implementing remote replication
Figure 42 Dual-fabric HP P6000 Command View local management zones: sheet 2 Zone 7 ports Zone 8 ports Site 1 management server port HBA2 Site 2 management server port HBA2 Site 1 Array controller A port FP2 Site 2 Array controller A port FP2 Site 1 Array controller B port FP2 Site 2 Array controller B port FP2 Creating fabrics and zones 85
Figure 43 Dual-fabric HP P6000 Command View remote management zones: sheet 1 86 Zone 9 ports Zone 10 ports Site 2 management server port HBA1 Site 1 management server port HBA1 Site 1 Array controller A port FP1 Site 2 Array controller A port FP1 Site 1 Array controller B port FP1 Site 2 Array controller B port FP1 Implementing remote replication
Figure 44 Dual-fabric HP P6000 Command View remote management zones: sheet 2 Zone 11 ports Zone 12 ports Site 2 management server port HBA2 Site 1 management server port HBA2 Site 1 Array controller A port FP2 Site 2 Array controller A port FP2 Site 1 Array controller B port FP2 Site 2 Array controller B port FP2 Creating fabrics and zones 87
Figure 45 Dual-fabric replication zone 88 Zone 13 ports Zone 14 ports Site 1 Array controller A port FP3 Site 1 Array controller A port FP4 Site 1 Array controller B port FP3 Site 1 Array controller B port FP4 Site 2 Array controller A port FP3 Site 2 Array controller A port FP4 Site 2 Array controller B port FP3 Site 2 Array controller B port FP4 Implementing remote replication
Figure 46 Dual-fabric replication zone fan-out: sheet 1 Zone 15 ports Zone 16 ports Site 1 Array (1) controller A port FP3 Site 1 Array controller A port FP4 Site 1 Array (1) controller B port FP3 Site 1 Array (1) controller B port FP4 Site 1 Array (13) controller A port FP3 Site 1 Array (13) controller A port FP4 Site 1 Array (13) controller B port FP3 Site 1 Array (13) controller B port FP4 Creating fabrics and zones 89
Figure 47 Dual-fabric replication zone fan-out: sheet 2 90 Zone 17 ports Zone 18 ports Site 1 Array (13) controller A port FP3 Site 1 Array (13) controller A port FP4 Site 1 Array (13) controller B port FP3 Site 1 Array (13) controller B port FP4 Site 2 Array (6) controller A port FP3 Site 2 Array (6) controller A port FP4 Site 2 Array (6) controller B port FP3 Site 2 Array (6) controller B port FP4 Implementing remote replication
NOTE: “Dual-fabric replication zone fan-in/fan-out for port isolation: sheet 1” (page 91) and “Dual-fabric replication zone fan-in/fan-out for port isolation: sheet 2” (page 92) show a mixed array environment that includes an EVA3000/5000 (item 1). In this type of configuration, changes should be made to isolate host ports to eliminate the possibility of reduced replication performance caused by the reduction in resource utilization.
Figure 49 Dual-fabric replication zone fan-in/fan-out for port isolation: sheet 2 Zone 20 ports Site 1 Array (13) controller A port FP4 Site 1 Array (13) controller B port FP4 Site 2 Array (6) controller A port FP4 Site 2 Array (6) controller B port FP4 92 Implementing remote replication
Figure 50 Dual-fabric replication zone 4-port to 8-port controllers (straight-cabled) Zone 21 ports Zone 22 ports Site 1 Array controller A port FP1 Site 1 Array controller B port FP2 Site 2 Array controller A port FP3 Site 2 Array controller A port FP4 Site 2 Array controller B port FP3 Site 2 Array controller B port FP4 Creating fabrics and zones 93
Figure 51 Dual-fabric replication zone 4-port to 8-port controllers (cross-cabled) Zone 23 ports Zone 24 ports Site 1 Array controller A port FP2 Site 1 Array controller B port FP2 Site 2 Array controller A port FP3 Site 2 Array controller A port FP4 Site 2 Array controller B port FP3 Site 2 Array controller B port FP4 NOTE: This zoning is used when the 4-port controller pair is configured with FP1 cross-cabled as shown.
FCIP gateway zoning configurations When using FCIP gateways as the interconnect between sites, consider the following: • HP B-series and HP C-series switch products can be used to create the configuration shown in Figure 38 (page 81). The zoning options shown in “Recommended dual-fabric zoning configurations” (page 80) can be used with B-series and C-series switch products. These products allow independent fabrics on each site to access objects on the other site.
Figure 52 FCIP gateway zoning configurations using the mpx110 1. Site 1 Array controller pair 8. Site 2 management server 2. Site 1 host 9. Site 2 Fibre Channel switch 3. Site 1 management server 10. Site 2 Fibre Channel switch 4. Site 1 Fibre Channel switch 11. Intersite link 5. Site 1 Fibre Channel switch 12. Intersite link 6. Site 2 Array controller pair 13. mpx110 gateways 7.
Selecting a preferred controller The following are important points to remember when selecting a preferred controller for DR group members: • The same preferred controller is used for all members of a DR group. The preferred controller value used for each virtual disk is set automatically based on the value assigned to the first DR group member. Ensure that the first member you add to the DR group is set to the desired preferred controller. • On XCS 6.
Using the failover only setting The failover only setting should be used for virtual disks that are presented to a host using HP Secure Path. This setting automatically redirects I/O to the alternate controller if the preferred controller becomes unavailable, but enables the host server to use HP Secure Path settings or commands to control the return to the preferred controller when it becomes available.
• Cannot be a snapshot • Cannot be a mirrorclone • Must be in normal operational state • Must use mirrored write cache (the default) The maximum number of virtual disks in a DR group and the maximum number of DR groups per array vary with controller software versions. For current supported limits, see the HP P6000 Enterprise Virtual Array Compatibility Reference.
Presenting destination virtual disks After creating the DR groups, you can present the destination virtual disks to the remote hosts in any of the following modes. You can use HP P6000 Command View, HP P6000 Replication Solutions Manager, or HP Storage System Scripting Utility to present destination virtual disks to remote hosts. • None—The virtual disk cannot be presented to any hosts. (HP Storage System Scripting Utility uses the term “disable” for this mode.
7 Failover and recovery This chapter provides information about failing over and resuming operations after a planned or unplanned loss of operation. The several scenarios describe situations you may encounter, with procedures for handling each scenario. Failover example Figure 53 (page 102) shows data replication among DR groups at three locations. Arrays 1 and 4 are at the local site, and arrays 2 and 5 are at a remote site.
Figure 53 DR groups before and after failover 1. Source array before failover 6. Local site 2. Destination array before failover 7. Remote site 3. Replication 8. Failover 4. Destination array 9. Logging 5.
Planning for a disaster Planning helps to minimize downtime caused by a disaster. When planning for disaster recovery, include the following: • Ensure that you have a supported disaster-tolerant solution. NOTE: Not all supported cable configurations will provide for dual fabrics and ISLs. • Have at least one management server available at every site in case of a hardware or communication failure. • Verify that each destination virtual disk within a DR group has been presented to a host.
Performing failover and recovery Failover and recovery procedures include such actions as failover, suspend, resume, disable failsafe, mounting, and unmounting. You can perform these actions using the following interfaces and tools: • HP P6000 Replication Solutions Manager GUI • HP P6000 Replication Solutions Manager CLI • HP P6000 Replication Solutions Manager jobs • HP Storage System Scripting Utility • HP P6000 Command View For specific procedures, see the interface documentation.
Table 5 When to fail over a DR group, managed set, or array Failure situation Recommended HP P6000 Continuous Access action DR group in normal mode DR group in failsafe mode Maintenance preventing access to source array Perform a planned failover on the destination array. See “Planned failover” (page 105). Total loss of source array Loss of both source controllers Manually intervene to fail over data on destination array, and then restart processing at the destination array.
Figure 54 Planned and unplanned failover TIP: With HP P6000 Replication Solutions Manager, you can designate a Home DR group to identify the preferred source. By default, the source at the time the DR group is created is Home. As the role of the array changes after multiple failover and failback events, the Home designation persists.
Planned Failover Procedure To execute a planned failover: 1. Optionally, move storage management to another management server. For instructions, see the HP P6000 Command View User Guide. 2. Ensure that all DR groups have resumed and are fully normalized. Check the DR group status using HP P6000 Command View or HP P6000 Replication Solutions Manager. If a DR group is merging or a normalization is in progress, wait for the process to complete. 3.
8. If you plan to operate for an extended time at the remote site and need to enable failsafe mode on a DR group. Make sure the new destination (previous source) and Fibre Channel links are functioning, and then perform the following steps: a. If the DR group is suspended, resume it and wait for the log disk to finish merging. b. Once the log has finished merging, change the DR group to failsafe mode. NOTE: You can enable failsafe mode at the destination array during a merge or normalization.
Action summary: Change from failsafe-enabled mode to normal mode and resume host I/O until the connection to the remote array is re-established. When the connection is stable, change back to failsafe-enabled mode. Figure 55 (page 109) illustrates the steps required to resume operations if you cannot access the destination while in a failsafe-locked state.
Procedure: To resume operation if you are unable to access the destination in failsafe mode: 1. Change affected DR groups from failsafe-enabled mode to normal mode. 2. If necessary, issue operating system commands to the local hosts to restart I/O on the virtual disks that were failsafe-locked. See “Resuming host I/O after failover” (page 118). 3. When connections to the destination are re-established, control the merging of data at the destination by suspending replication on the less important DR groups.
Figure 56 Returning operations to replaced hardware Procedure: This procedure does not include steps for rebuilding servers (this should be part of your disaster plan). For more information about the steps in this procedure, see the online help for HP P6000 Command View or HP P6000 Replication Solutions Manager.
Table 6 Sample array log Array with failed or new hardware Current source array Array name Array name Array name Array name Array name Array name Array name Array name 1. 2. 3. 4. 5. 6. Record the name of the array with failed or new hardware (current destination) and the name of the current source array in a log such as the one shown in Table 6 (page 112). For example, the array with new hardware is named HSV01 and the current source array is named HSV02.
You also have to take into account that the DR groups may have changed membership (or new ones have been created) since the time the original source array was destroyed. • If you are not using a utility script for recovery, re-create all DR groups on the source array using the information recorded in “Configuration form ” (page 126). Specify the replaced array for the destination. 12.
Table 7 HP P6000 Replication Solutions Manager display icons Resource Icon Description Array Indicates that the array is in an abnormal state and requires attention. Virtual disks Indicates a catastrophic failure and requires immediate action. DR groups Red indicates a failure; yellow indicates that the DR group is in a degraded state. Either condition requires immediate attention.
Figure 57 Disk Group Hardware Failure window 4. Click Start Resolution Process. After a prompt for confirmation, a list of failed DR groups is displayed. 5. One at a time, select the affected DR groups and click Delete, then OK to confirm the deletion. Deleting a DR group removes the relationship between the virtual disk members; it does not delete data from the virtual disks as they remain intact on source and destination systems. 6.
Your first indication that a disk group has become inoperative may be icons like those shown in “HP P6000 Replication Solutions Manager display icons” (page 114), except that the destination disk group status is Unknown. NOTE: When you delete DR groups to recover from a disk group hardware failure, you do not have disaster tolerance until you re-create the DR groups and they have completed normalization.
In HP P6000 Replication Solutions Manager: • On the General tab of the Virtual Disk Properties window, BLOCKED appears in the Operational State box. • On the General tab of the selected DR group window, BLOCKED appears in the Operational State box. In HP P6000 Command View: • On the Vdisk Properties window, None (DR members protected) appears in the host access box. • On the DR Group Properties window, Blocked appears in the Group host access box.
8 Operating system procedures This chapter describes operating system procedures that accompany remote replication procedures, especially for failover and recovery. Resuming host I/O after failover Procedures for detecting disk devices and restarting I/O operations after DR group failover differ among operating systems. HP OpenVMS • If the remote hosts are shut down, boot them now. Booting the hosts enables OpenVMS to recognize the drives.
# mknod /dev/vg09/group c 64 0x090000 b. After the folders are created, the disks appear, and the special files are present, use the vgimport command to create the virtual group: # vgimport Example: # vgimport /dev/vg09 /dev/dsk/c18t0d /dev/dsk/c18t1d0 /dev/dsk/c25t0d0 c. To display this virtual group using # vgdisplay -v /dev/vg09.
1. 2. Reboot the servers at the remote site. Remount the file system. For more information, see “Bootless failover using LVM with Linux ” (page 122). Novell NetWare • If the remote hosts are shut down, boot them now. If you are using traditional NetWare volumes, booting the hosts allows NetWare to recognize the drives and automatically mount the volumes. If you are using NSS logical volumes, booting the hosts allows NetWare to recognize the NSS pools and activate them.
If you are using MPIO, and it has not been configured for these devices: 1. Issue the cfgadm command. # cfgadm –al –o show_FCP_dev Output: c3 fc-private connected unconfigured unknown c3::210000e08b0a5b65 unknown connected unconfigured unknown c3::500060e802eb2b0b,0 disk connected unconfigured unknown c4 fc-private connected unconfigured unknown c4::210100e08b2a5b65 unknown connected unconfigured unknown c4::500060e802eb2b14,0 disk connected unconfigured unknown 2.
1. 2. 3. 4. On each host, log on to an account that has administrative privileges. Select Programs > Administrative Tools > Computer Management. Click Disk Management. After Disk Management has initialized, select Actions > Rescan Disks. If the units fail to appear, press F5 (Refresh). All of the failed over units are displayed in the right pane.
3. 4. Perform a failover of the DR group. Depending on the number of LUNs, do one of the following to prevent HP Secure Path from detecting a failed disk: • For individual LUNs, run spmgr quiesce –p path for each path visible to the LUNs. • For all LUNs at once, run spmgr set –p off to turn off path verification for all LUNs visible to the system. Destination host procedure If this is the first time that LUNs are being presented to the destination host, reboot the host to detect the new LUNs.
9 Managing remote replication This chapter describes routine and advanced procedures for managing remote replication. Using remote replication in a mixed array environment The remote replication features that are available depend on the controller software version.
snapshot, mirrorclone, or snapclone feature to create a copy of the destination virtual disk before a normalization starts. If a major failure occurs at the local site during a normalization, HP P6000 Business Copy can provide a clean copy of data as it existed before normalization writes started on the remote array. (Note that any new writes that occurred on the source between the time the copy was created and the major failure occurred are lost.
Backing up replication configuration HP recommends regular backups of replication configurations and jobs. Current backups are essential to successful disaster recovery. You can also use backups to duplicate the local configuration and jobs on remote and standby management servers and keep those servers current. You can use HP P6000 Replication Solutions Manager and HP Storage System Scripting Utility to make backups, or you can back up your configuration manually.
Table 8 Configuration form (continued) Folder name: (default = \Hosts\) Host name: Operating system: For each HBA port: WWN: Virtual disk information Folder name: (default = \Virtual Disks\) Virtual disk name: Disk group: Size: Redundancy level: (Vraid0, Vraid1, Vraid5) Write cache policy: (Mirrored write-back, unmirrored write-back) Read cache policy: (On, off) Read/write permissions: (Read/write, read-only) OS unit ID: (default = 0) Preferred controller (None, Path A FO, Path A FO/FB, Pat
10 Support and other resources Contacting HP HP technical support Telephone numbers for worldwide technical support are listed on the HP support website: http://www.hp.com/support/ Collect the following information before calling: • Technical support registration number (if applicable) • Product serial numbers • Product model names and numbers • Error messages • Operating system type and revision level • Detailed questions For continuous quality improvement, calls be recorded or monitored.
http://h18006.www1.hp.com/storage/arraywhitepapers.html The following SAN-related documents are located on the HP Storage Networking website: http://h18006.www1.hp.com/storage/networking/index.html • HP SAN Design Reference Guide • HP Essentials Data Replication Designer 1.0 User Guide The following may be useful when planning your array configuration: • HP 4400 Enterprise Virtual Array (EVA4400) Performance White Paper: http://h71028.www7.hp.com/ERC/downloads/4AA1-8473ENW.
Table 9 Document conventions (continued) Convention Element Monospace text • File and directory names • System output • Code • Commands, their arguments, and argument values Monospace, bold text Emphasized monospace text WARNING! An alert that calls attention to important information that if not understood or followed can result in personal injury.
Glossary A ABM array-based management array Synonymous with storage array, storage system, and virtal array. A group of disks in one or more disk enclosures connected to two controllers running controller software that presents disk storage capacity as one or more virtual disks. See also virtual array, storage system. array-based management A management structure in which HP P6000 Command View is installed on the management module within the EVA controller enclosure.
DR group Data Replication group. A logical group of virtual disks in a remote replication relationship with a corresponding group on another array. DR group member states Normal. Remote replication involving the virtual disk is normal. There is no logging or merging involving the virtual disk. Full Copy. The DR group log threshold has been reached and the virtual disk is involved in a full copy. Async Merging (Adding).
general-purpose server A server that runs customer applications such as file and print services. HP P6000 Command View and HP P6000 Replication Solutions Manager can be used on a general purpose server in limited configurations. H home The DR group that is the preferred source in a replication relationship. By default, home is the original source, but it can be set to the destination DR group. host A computer that runs user applications and uses the information stored on an array.
back to the state they were in as of no longer than 6 hours ago. To achieve this objective, you need to be making backups or other data copies at least every 6 hours. RTO Recovery Time Objective. The time needed to recover from a disaster—usually determined by how long you can afford to be without your systems. RWP Replication Workload Profiler S SAN Storage area network.
X XCS See controller software.
Index A AIX procedure to resume I/O, 119 application and bandwidth, 20 and disk groups, 35 and DR groups, 24 determining peak load, 18 failover, 26 recovery, 15 asynchronous mode description, 21 asynchronous write mode description, 8 specifying, 99 B B-series switch configuration, 52 backup configuration, 100 manual, 126 procedure, 126 using HP P6000 Replication Solutions Manager, 126 using HP Storage System Scripting Utility, 126 bandwidth and performance, 18 and RPO, 18 evaluating, 20 in bidirectional re
conventions, 129 related documentation, 128 documentation HP website, 128 providing feedback, 128 DR group bidirectional, 46 bootless failover (Linux), 122 cascaded, 49 creating, 98 description, 8, 36 fan-in, 48 guidelines, 36 log disks, 9, 42 log size, 43 planning, 36 rules, 24 system fan-out, 47 virtual disk member, 98 DR group write history log planning for, 42 specifying location, 44 specifying size, 43 E enhanced asynchronous mode description, 21 examples one-way latencies, 16 F failback procedure, 1
license, 58 HP P6000 Replication Solutions Manager backing up the configuration, 126 installing, 58 HP SCSI-FCP data replication protocol, 38 HP-FC data replication protocol, 38 HP-UX application recovery, 15 procedure to detect devices, 118 configuration, 24 supported distance, 17 M I/O consistency, 22 implementing remote replication, 51 implicit LUN transition, 37 initializing arrays, 96 installing HP P6000 Replication Solutions Manager, 58 license, 58 interface options for failover, 104 intersite link
R recovery after destination loss, 108 choosing a procedure, 104 from destination disk group failure, 115 from disk group failover, 113 Red Hat Linux LifeKeeper clusters, 122 related documentation, 128 remote copy, 8 remote replication concept, 8 implementing, 51 in a mixed array environment, 124 remote site choosing the location, 14 replication bidirectional, 46 cascaded, 49 fan-in, 48 system fan-out, 47 Replication Solutions Manager interface managed sets, 9 Replication Workload Profiler (RWP), 14 resume
HP support, 128 Windows procedure to resume I/O, 121 write history log, 9 planning for, 42 specifying location, 44 specifying size, 43 write mode asynchronous, 21 choosing, 20 enhanced asynchronous, 21 selecting, 99 synchronous, 22 write modes asynchronous, 8 defined, 8 synchronous, 8 write order, maintaining correct, 22 Z zones creating, 58 zoning, 24 in single-switch configuration, 28 single-fabric configuration, 27 140 Index
