Specifications

ManualsBrandsDell ManualsConsumer electronicsPowerConnect MPS 1U Shelf

MCS Documentation for LWA-1 CDR

Steve Ellingson on behalf of the Virginia Tech MCS Development Team

Nov 10, 2009

Summary of Status

• For MCS, “hardware design” consists primarily of selection of computers, computer components,

and networking equipment; and verifying performance. Hardware design for MCS (excluding data

recorders (MCS-DR); see below) is complete.

• MCS/Scheduler software is available in a functional pre-alpha release status. Subsystems SHL and

ASP are fully supported. DP is partially supported.

• The most demanding speed requirement for MCS is the ability to re-point a “calibration beam”

within 5 ms, repeating at every 60 ms (LWA Memo 146). We have confirmed that MCS will be at

least 2 orders of magnitude faster than necessary, and is limited primarily by network throughput.

• MCS-DR has been designed and demonstrated. Up to 5 TB can be acquired at 115 MB/s from each

of the 5 outputs of the DP subsystem. This corresponds to about 10 hours of continuous acquisition

per beam, plus TBN/TBW, and exceeds the requirement of 112 MB/s (corresponding to TBN at its

highest rate). It should be noted that these tests were done using a second computer emulating DP,

since DP is not yet available.

• Two of the five MCS-DR computers have been procured. Unfortunately, the COTS computer used

in the MCS-DR design was discontinued before the remaining three were ordered. A replacement

has been identified, but not yet tested.

• MCS-DR software is available in a functional pre-alpha release status. The primary difference

between this software and the software used to validate the MCS-DR design is the implementation

of the MCS Common ICD, which allows MCS-DRs to be controlled by MCS as individual level-1

subsystems.

• Software development for MCS/Executive and MCS/Task Processor remains in the planning stage

and no releasable software is available. We consider this to be on schedule. The risk involved in

these software components is very low relative to the risk associated with MCS/Scheduler and MCS-

DR, which have been our focus. Furthermore, it should be noted that the station can be controlled

via a bash-scriptable command line interface using only MCS/Scheduler.

MCS Specification, Requirements, and Design Documentation

The documents cited below are included as an attachments. Note that the most recent documentation is

always available at the “Virginia Tech LWA Contributions” web site, http://www.ece.vt.edu/swe/lwavt/.

MCS is described in the following documents:

• S. Ellingson, “MCS Subsystem Definition,” Ver. 2, LWA Engineering Memo MCS0004,

Feb. 23, 2009. [online] http://www.ece.vt.edu/swe/lwavt/. Note: Paragraph 4 of Section 3 (“Data

Recording”) is no longer applicable due to the adoption of DRSUs as the standard medium for

data storage/transfer; see MCS0007.

Summary of content (129 pages)

PAGE 1
MCS Documentation for LWA-1 CDR Steve Ellingson on behalf of the Virginia Tech MCS Development Team Nov 10, 2009 Summary of Status • For MCS, “hardware design” consists primarily of selection of computers, computer components, and networking equipment; and verifying performance. Hardware design for MCS (excluding data recorders (MCS-DR); see below) is complete. • MCS/Scheduler software is available in a functional pre-alpha release status. Subsystems SHL and ASP are fully supported.
PAGE 2
• S. Ellingson, “MCS Architecture,” Ver. 4, LWA Engineering Memo MCS0007, Nov 7, 2009. [online] http://www.ece.vt.edu/swe/lwavt/. The MCS Common ICD is the basis for passing of command and status information between MCS and other LWA-1 Level-1 subsystems: • S. Ellingson, “MCS Common ICD,” Ver. 1.0, Long Wavelength Array Engineering Memo MCS0005, April 4, 2009. [online] http://www.ece.vt.edu/swe/lwavt/.
PAGE 3
Summary of Component Cost, Size, and Power MCS not including Data Recording (MCS-DR): Item Description Power 1,2 Scheduler computer 750W Executive computer 1,2 750W 1,2 Task Processor computer 750W Gateway managed switch 3 50W Command Hub managed switch 3 50W 4 Other (misc) TOTAL 2350W Cost $1916 $1916 $1916 $1528 $1528 $1000 $9804 2U 2U 2U 1U 1U 2U 10U Status (Nov 10, 2009) Purchased; at VT Not yet ordered Not yet ordered Purchased; at VT Not yet ordered 1 Dell R5400; see attached quote and technical
PAGE 4
backup to the 10GbE NIC, is not supported, and should not normally be needed. ATI FireMV 2250 256MB DDR2 PCIe (x1) video card, Model No. 100-505179. This is needed even if video is not desired, because the 10GbE NIC displaces the existing video card and the motherboard's BIOS requires that a video card is present. 10 Comment above will likely also apply to the T1500. The video card needed for the T1500 has not yet been determined. Using the ATI card cited above as worst case for cost.
PAGE 5
List of Attachments 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. MCS0004 (MCS Definition) MCS0007 (MCS Architecture) MCS0005 (MCS Common ICD) “readme” file from MCS0021 (MCS/Scheduler Software Version 0.4) MCS0018 (MCS-DR Design & Verification) MCS0019 (MCS-DR Storage Unit) MCS0020 (MCS-DR ICD) “readme” file from MCS0022 (MCS-DR OS Version 0.
PAGE 6
MCS Subsystem Definition Ver. 2 Steve Ellingson∗ February 23, 2009 Contents 1 Scope of Work 2 2 MCS Functions 2 3 Data Recording 3 4 User Interface 3 5 Application Software 4 6 User Observing Paradigm 5 7 Document History 6 ∗ Bradley Dept. of Electrical & Computer Engineering, 302 Whittemore Hall, Virginia Polytechnic Institute & State University, Blacksburg VA 24061 USA. E-mail: ellingson@vt.
PAGE 7
1 Scope of Work MCS stands for “monitoring and control system”. With the current allocation of ONR FY07 funds plus the currently-planned allocation of FY08 funds, Virginia Tech will develop MCS for the first LWA station (“LWA-1”) to a sufficient level to facilitate LWA-1 “initial operational capability” (IOC). Here are some caveats: 1.
PAGE 8
3 Data Recording According to the station architecture document [1], LWA stations are to have a data aggregation and communication (DAC) subsystem that is responsible for the routing of data from digital processing, and further says “The DAC includes separate output to facilitate local recording of output directly to disk. For example, this would allow continued operation of the station should data path to the LWA central processing facility be interrupted, or not yet implemented.
PAGE 9
For security and safety reasons, certain functions may be restricted for users which are not physically present. 2. The primary logical interface with users will be via selected standard internet protocols. Command line operation via the secure shell (ssh) protocol will be provided for both monitoring and control. To use this facility, users will login to a Linux-based operating system using personal accounts set up for them by the MCS system administrator.
PAGE 10
3. Frequency-domain analyzer (max-hold, median, mean). By antenna, or by beam. Variable spectral resolution and integration time. The command line version will provide the same information, but in numerical form only and without continuous updating. 4. Time-domain analyzer. By antenna, or by beam. Variable bandwidth and integration time. The command line version provides same information, but in numerical form only and without continuous updating. 5. Time-frequency analyzer (i.e., spectrograms).
PAGE 11
7 Document History • Version 2 (Feb 23, 2009): – Indicating explicitly that MCS-DR only records data from DP, and does not reformat it. – More specific comments on MCS-DR transfer rates and storage, based on project-level decisions to use 10 Gb/s ethernet.
PAGE 12
References [1] S. Ellingson, “Long Wavelength Array Station Architecture Ver. 1.0,” Long Wavelength Array Memo Series No. 119, November 19, 2007. [online] http://www.phys.unm.edu/∼lwa/memos.
PAGE 13
MCS Architecture Ver. 4 Steve Ellingson∗ November 7, 2009 Contents 1 Architecture 2 2 Document History 5 ∗ Bradley Dept. of Electrical & Computer Engineering, 302 Whittemore Hall, Virginia Polytechnic Institute & State University, Blacksburg VA 24061 USA. E-mail: ellingson@vt.
PAGE 14
1 Architecture MCS stands for “monitoring and control system”. MCS is defined in [1]. The purpose of this document is to describe the architecture of MCS (not discussed in [1]) and to provide the next lower level of design detail. Figure 1 shows the MCS architecture, as well as interfaces to adjacent subsystems. First, note that here we use the term “MCS” to refer to both MCS (the upper shaded region) as well as the MCS Data Recorder (“MCS-DR”; the lower shaded region).
PAGE 15
Figure 1: MCS architecture and interfaces to adjacent subsystems. “DAC” refers to “Data Aggregation and Communications” (not currently implemented in LWA-1). “ABE” refers to “alternative back ends” (not currently defined, but taken into account to facilitate future expansion). “DRSU” refers to “data recorder storage unit” (see text). “Maint.
PAGE 16
computer is connected to a separate, removable data recorder storage unit (“DRSU”). A DRSU is a hard drive array having total storage of 5TB, connected to an MCS-DR computer via an eSATA cable. All data acquired by an MCS-DR computer is streamed directly to its associated DRSU. For additional information, see [3, 4]. When not being used for data recording, MCS-DR computers are also available for general purpose computing by MCS, or by users (under MCS control).
PAGE 17
2 Document History • Version 4 (Nov 7, 2009): – Revised architecture diagram to remove PPS and 10 MHz interfaces into MCS/Scheduler (determined that NTP will suffice). – Revised architecture diagram to replace LTO tape drives with DRSUs. • Version 3 (Feb 25, 2009): – Revised architecture diagram to indicate an “MCS Common ICD” connection to SHL. J. Craig indicates that SHL will have a computer which in turn will control SHL-PCD and SHL-ECS.
PAGE 18
References [1] S. Ellingson, “MCS Subsystem Definition,” Ver. 2, Long Wavelength Array Engineering Memo MCS0004, Feb. 23, 2009. [online] http://www.ece.vt.edu/swe/lwavt/. [2] S. Ellingson, “MCS Common ICD,” Ver. 1.0, Long Wavelength Array Engineering Memo MCS0005, Apr 04, 2009. [online] http://www.ece.vt.edu/swe/lwavt/. [3] C. Wolfe, S. Ellingson & C. Patterson, “Interface Control Document for Monitor and Control System Data Recorder,” MCS0020, Oct 10, 2009. [online] http://www.ece.vt.edu/swe/lwavt/. [4] C.
PAGE 19
MCS Common ICD Ver. 1.0 Steve Ellingson∗ April 4, 2009 Contents 1 Introduction and Scope 2 2 Summary 2 3 MIB 2 4 Message Structure 4 5 Message Types 5 6 Command/Response Examples 6.1 PNG Command/Response . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.2 RPT Command/Response . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 6 7 7 Document History 8 ∗ Bradley Dept.
PAGE 20
1 Introduction and Scope MCS stands for “monitoring and control system”. As described in the LWA station architecture document [1], MCS monitors and controls ASP (“analog signal processing”), DP (“digital processing”), and other subsystems. The purpose of this interface control document (ICD) is to define a common interface between MCS and connected subsytems. Compliance with this ICD is necessary, but not sufficient for integration with MCS.
PAGE 21
index 2 2.1 2.2 2.2.1 2.2.2 label A2 B21 C22 D221 E222 data remarks 3.4 5 bytes, ASCII, base-10, decimal point allowed PRR 7 3 bytes, ASCII, alphanumeric 2 bytes, ASCII, base-10 integer Figure 1: A MIB fragment, provided as an example only. MIB labels must consist only of letters (case is significant), integer numbers, and the underscore character. Spaces are not allowed. The length must be less than or equal to 40 characters. Data referenced by MIB entries need not be ASCII, and can be raw binary.
PAGE 22
1.6. VERSION [maximum 256 bytes, ASCII]. Version number of locally-installed software. May include additional information or elaboration; if so, the “principal” version number must appear first and be followed by a single space. Any unused bytes at the end of this string should be spaces. 1.x Additional MIB entries beginning “1.” TBD x.
PAGE 23
field. MJD/MPM should reflect the “best available estimate” of station time as known to the sender. A satisfactory “best available estimate” can be obtained simply by calling an appropriate time function immediately prior to assembling the message and sending it, and it is expected that this time will represent the time at which the message was actually transmitted to within a few milliseconds. See Section 6 for examples of command and response messages. 5 Message Types Messages from MCS are commands.
PAGE 24
3. R-COMMENT [variable length, ASCII]. The definition of this field depends on R-RESPONSE and the message TYPE. If R-RESPONSE is “R”, then this field shall be used to send error codes or log messages, as specified by the subsystem ICD or other subsystem design documents. 6 Command/Response Examples For clarity in the following examples, single quotes (’) are used in lieu of spaces and “@” is used to represent a byte of raw binary data. 6.
PAGE 25
6.2 RPT Command/Response The following is an example of an RPT command sent from MCS to DP. This example assumes the MIB fragment shown in Figure 1. MCS sends the message DP’MCSRPT’’’’’1391’’’3’54828’12345678’B21 which is interpreted as a request for the data value associated with MIB index 2.1. In response, DP sends the message MCSDP’RPT’’’’’1391’’’5’54828’12345698’A’NORMAL’’3.4 in which DP is indicating that B21 = 3.4. Note that all 5 bytes of data value (per the specification of Figure 1) are sent.
PAGE 26
7 Document History • Version 0.2 (Feb 23, 2009): – Julian day (JDAY) changed to mean Julian day (MJD). – Fixed error in specification of left- vs. right-justification. – Pointed out in the definition of MPM that a UT day can have have a leap second. – More specific guidance for MIB labels (allowable characters, lengths, etc). • Version 0.3 (Mar 20, 2009): – Fixed error in representation of MJD. – Fixed error in which MCS-RESERVED MIB labels “SUBSYSTEM” and “SERIALNO” were both index 1.4.
PAGE 27
References [1] J. Craig, “Long Wavelength Array Station Architecture,” Ver. 2.0, Long Wavelength Array Memo 161, February 26, 2009. [on-line] http://www.phys.unm.edu/∼lwa/memos. [2] S. Ellingson, “MCS Subsystem Definition,” Ver. 2, Long Wavelength Array Engineering Memo MCS0004, Feb. 23, 2009. [online] http://www.ece.vt.edu/swe/lwavt/.
PAGE 28
��
PAGE 29
��
PAGE 30
��
PAGE 31
��
PAGE 32
��
PAGE 33
��
PAGE 34
MCS - Data Recorder Preliminary Design & Veriﬁcation Christopher Wolfe∗, Steve Ellingson, Cameron Patterson August 26, 2009 Contents 1 Introduction 2 1.1 Purpose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 1.2 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 1.3 Document Conventions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 1.4 Summary of Current Status of MCS-DR Development .
PAGE 35
1 1.1 Introduction Purpose The MCS-DR, or “monitoring and control system - data recorder” is a data capture and storage system for the LWA radio telescope project implemented using general-purpose, commercially available oﬀ-the-shelf (COTS) components. A preliminary design for the MCS-DR has been completed and validated. Speciﬁcally, we have demonstrated the ability to reliably record up to 10 hours of data at 115 MiB/s.
PAGE 36
of 115 MiB/s. This exceeds the highest rate required, which is 112 MiB/s (corresponding to TBN mode at it’s largest speciﬁed bandwidth). Tests have veriﬁed the ability to record data streams at 115 MiB/s for a period of at least ten hours. The absolute ceiling on recording speed has not been established, but may be in the neighborhood of 150 MiB/s on average, or 200 MiB/s with optimal circumstances such as short recordings at the very beginning of the drive.
PAGE 37
Figure 1: MCS-DR PC System Overview 4
PAGE 38
R Image⃝ PCI-E (“PCI express”) 1x external SATA-II adapter based on the SteelVineTM series of storage controllers is added to the stock system. The eSATA adapter came packaged with the VenusTM T5 RAID enclosure. The enclosure contains ﬁve hot-swappable Seagate 1 TB 7200 RPM SATA- II hard disk drives, and provides a total storage capacity of 5 TB less ﬁle system and formatting overhead. This is the conﬁguration that has been implemented and veriﬁed.
PAGE 39
Figure 2: MCS-DR PC System Software Overview Initial tests with the ext2 ﬁle system yielded inconsistent results. Consequently, we implemented a custom ﬁle system based on raw access to the drive array. The ﬁle system used by the software is a ﬂat ﬁle system with a simple bitmap structure written to the beginning of the drive, and the remaining space available for ﬁle storage. The current (validated) ﬁle system supports as many as 1023 recordings, all of which combined may be up to 4.9 TiB in size.
PAGE 40
2.3 Hardware Considerations The MCS-DR PC must be capable of recording data streams at a sustained rate of 112 MiB/s with data payload sizes depending on the data source and operational mode. The hardware for the MCS-DR PC was selected such that the speed of all components of the internal data path exceeded this requirement. The only deviation from this is with the hard drives, where the speed requirement is met by having 5 drives in RAID 0 instead of one drive capable of the desired rate.
PAGE 41
Myricom oﬀers 10 Gb/s transfer rates, large receive oﬀ-loading, automatic checksum generation, and an open-source API for software interfacing (as well as open-source, Linux-friendly drivers). 2.4 Software Considerations As a low-cost alternative to available commercial data capture options, it was desired to avoid proprietary technologies and their consequent licensing royalties.
PAGE 42
drives in a RAID 0 conﬁguration, the limiting factor should have been the SATA-II bus. However this was not the case, and sustained transfer rates for the RAID as a whole were limited to about 120 MiB/s. Most of this loss was due to the way Linux caches writes to the drive, but part was due to the non-optimal factory tuning of the drives’ ﬁrmware. When testing the SV35.5 series with the same options, the performance was approximately the same.
PAGE 43
3.3 Memory and CPU Testing The ﬁrst tests which incorporated socket-based communication aimed at verifying that the CPU and memory could maintain suﬃcient transfer rates. By creating a socket connection to “localhost”, the test was able to send UDP datagrams from one part of the data path and have them received in another.
PAGE 44
3.5 Complete System / Duration Testing To perform the complete system and duration test, the hardware from a second MCS-DR PC was used to emulate the DP subsystem and provide a stream of data for the MCS-DR PC under test. The duration tests involved streaming data from the emulated DP and having it recorded on an MCS-DR PC. Tests conﬁrmed that the hardware is capable of the required rates.
PAGE 45
– Evaluate/test 1U rack-mount enclosure to replace Venus T5 – Re-testing of 7200.11 series drives but with cache disabled. (time permitting) – Establish maximum rate sustainable for 2, 4, 6, 8, and 10 hour observations. (time permitting) ∙ Candidate Future (Post-CDR or Post-IOC) Development Tasks – Linux VFS-compliant ﬁle system extension to allow mounting of MCS-DR PC’s custom, raw ﬁle system.
PAGE 46
5 Document History ∙ Version 0.4 (Aug 26, 2009): – Fourth draft of document. – Updated Title, Figures, Ongoing and Future Eﬀorts, Software Brief – removed typos ∙ Version 0.3 (Aug 26, 2009): – Third draft of document. – Included discussion of ﬁle system changes – extraneous content removed ∙ Version 0.2 (Aug 23, 2009): – Second draft of document with changes in RE: hardware, software, and testing. ∙ Version 0.1 (Jul 14, 2009): – Initial draft of document.
PAGE 47
References [1] S. Ellingson, “MCS Subsystem Deﬁnition,” Ver. 2, Long Wavelength Array Engineering Memo MCS0004, Feb. 23, 2009. [online] http://www.ece.vt.edu/swe/lwavt/. [2] International Electrotechnical Commission, “Letter symbols to be used in electrical technology Part 2: Telecommunications and electronics,” Third Ed., 2005. [online] http://www.iec.
PAGE 48
MCS-DR STORAGE UNIT Christopher Wolfe*, Steve Ellingson, Cameron Patterson September 23, 2009 Bradley Dept. of Electrical & Computer Engineering, 302 Whittemore Hall, Virginia Polytechnic Institute & State University, Blacksburg VA 24061 USA. Email: chwolfe2@vt.
PAGE 49
1 PURPOSE This document will describe the parts and required assembly of an MCS-DR RAID array. The RAID array must neither be formatted nor partitioned as the MCS-DR uses a custom file system. Accessing and manipulating data currently requires a custom application, though plans for a future linux file system driver are being developed. 2 HARDWARE SPECIFICATION The MCS-DR's RAID storage unit consists of a 1U rack-mount RAID enclosure, and 5 streamingtuned hard disk drives.
PAGE 50
3 ASSEMBLY C B A B C Figure 1. The RAID enclosure. To open the enclosure, remove 1 black screw (A), 2 silver screws (B), and slide the top cover back. Note that the rack mount tabs for the front sides of the enclosure are included in the kit, but not installed in the unit shown (C).
PAGE 51
Figure 2. Mounting the first four drives. The first four drives mount on a raised aluminum tray at the front of the unit. The tray is mounted to standoffs with 6 flush-seated machine screws. remove these screws, and flip the tray back towards the end of the unit. Each of the four drives is mounted to the tray with four coarse thread machine screws.
PAGE 52
A Figure 3. Mounting the first four drives. Here is a close-up shot showing the mounting screws for one of the four drives which mount on the tray. Be sure to thread all four screws before tightening as the drives may not align very well otherwise. Once all four drives have been mounted, connect the SATA cables and power cables.
PAGE 53
Figure 4. Mounting the fifth drive. The fifth drive mounts on the smaller aluminum tray near the power supply. Remove the four pancake-head machine screws and flip the tray towards the back of the unit. The clearance between the cables for the 1st and 2nd drives and the small aluminum tray is insufficient, and folding one of the four tabs to be parallel with the drive's side wall was necessary. In the figure above, this is the circled area showing the tab already bent.
PAGE 54
Figure 5. Cable arrangement. The cables for the first and second drives must be routed to go under the rear aluminum tray as shown. Be careful not to bend the cable sharply or allow the metal edges to rest against the cables with too much pressure.
PAGE 55
Figure 5. Cable arrangement with fifth drive in place. This image shows the rear tray mounted and position of the cabling. The green circled area highlights the tab on the rear tray that had to be folded, and how the SATA and power cables must be routed underneath the rear tray.
PAGE 56
5 4 3 2 1 a.) b.) Figure 6. Cable connection order and notes. It is advisable but not necessary to attach the drives in a consistent ordering in the event that data needs to be recovered or a drive replaced. In figure 6.a, the circled SATA cable bundle shows the desirable drive numbering used.
PAGE 57
4 3 2 1 5 Figure 7. The enclosure with 5 mounted drives. All that remains now is to replace the lid with its 3 retaining screws. The preferred drive numbering is indicated in green.
PAGE 58
4 APPENDIX A: ENCLOSURE VENDOR WEBSITE http://www.cwol.com/serial-ata/sata-port-multiplier-encl-rack-mount.htm SATA Rack Mount Enclosure, 5 Drive Retail Value: $499.00 As Low As: $359.50 In Qty. IN STOCK Enter quantities of items you wish to purchase then "Add" to cart Not a do-it yourselfer? Buy this enclosure with up to 10TB of installed and tested storage! 1 5 Drive SATA Rackmount Port Multiplier Encl. Retail 1 ea 2 ea 3-4 5+ $499.00 $374.50 $367.50 $364.50 $359.
PAGE 59
2 2TB 64 MB Cache WD RE4 RAID SATA II HDD Retail 2-4 5-9 10+ $699.00 $310.50 $308.50 $307.50 5-DRIVE SATA PORT MULTIPLIER RACK MOUNT ENCLOSURE Super High Density Near Line Storage! Up to 10 Terabytes in a single 1U rack space! This sleek 1U rack mount enclosure features the latest Silicon Image SiI-3726 SATA Port Multiplier technology and houses five (5) high capacity SATA I or SATA II disks in a single enclosure while connecting to the host system via a single external eSATA cable.
PAGE 60
 Platform independent—works with any Mac, Windows or Linux system with compatible SATA Host controller Specifications:          Connectors: One eSATA connector Cooling: DirectAir technology Performance: Up to 3.0Gb/s (300 MB/s) Compatibility: Compatible with all 3.5" SATA I (Serial ATA 1.5Gb/s) and SATA II (Serial ATA 3Gb/s) Hard Drives Dimensions: 1U rackmount (W x D x H):19.0"x 20.8"x 1.75" Weight: 9 lbs / 4.
PAGE 61
4 APPENDIX A: HARD DRIVE VENDOR WEBSITE http://www.pcconnection.com/IPA/Shop/Product/Detail.
PAGE 62
Interface Control Document for Monitor and Control System Data Recorder (MCS-DR) Christopher Wolfe∗, Steve Ellingson, Cameron Patterson October 10, 2009 ∗ Bradley Dept. of Electrical & Computer Engineering, 302 Whittemore Hall, Virginia Polytechnic Institute & State University, Blacksburg VA 24061 USA. Email: chwolfe2@vt.
PAGE 63
Contents 1 Description 1.1 Purpose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.2 Scope . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.3 Related Documents and Drawings . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 5 5 5 2 Document Conventions 2.1 Abbreviations and Acronyms . . . . 2.2 Command Parameter Types . . . . . 2.3 Mark-up Conventions . . . . . . . . 2.4 Numeric Representation Convention . . . .
PAGE 64
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
PAGE 65
List of Tables 1 2 3 MCS-DR MIB structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . MCS-DR MIB structure (Continued) . . . . . . . . . . . . . . . . . . . . . . . . . . . MCS-DR Commands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 10 22 List of Figures 1 2 3 4 5 6 7 8 9 10 An MCS-DR PC and storage unit mounted on a 19” rack . . . . . Diagram of electrical connections . . . . . . . . . . . . . . . . . . . Example of checking system status . . . . .
PAGE 66
1 1.1 Description Purpose The purpose of this document is to deﬁne the interface between Monitor and Control System Data Recorder (MCS-DR) and other Long Wavelength Array (LWA) station subsystems. The MCS-DR subsystem records output of the Digital Processing (DP) subsystem and is controlled by the Monitor and Control System (MCS). Whereas station architecture and subsystem ICDs may refer to the MCS-DR as a whole, this ICD applies to a single MCS-DR PC. 1.
PAGE 67
2.2 Command Parameter Types uint8 ASCII-XXX-# ASCII-XXX-A 2.3 unsigned integer, 8 bits An ASCII string exactly XXX characters in length which is interpreted as a number. Valid characters are numbers and right-padding spaces only. An ASCII string exactly XXX characters in length which is interpreted as a text string. Unless otherwise noted, valid characters are letters, numbers, the underscore character, and periods. Mark-up Conventions Symbol/Mark-up italics Bold Fixed-width <...> 2.
PAGE 68
3 3.1 Physical System Interfaces Mechanical Interface Figure 1 shows the MCS-DR mounted in one possible conﬁguration. The MCS-DR consists of a PC and a RAID storage unit. The PC is mounted in a 6U EIA 19” shelf rack, and the storage unit requires 1U of rack space. The PC and storage unit may be mounted anywhere within the shelter so long as they are within cable’s reach of each other (approx. 1 m). For more details on the 1U storage unit, see “MCS-DR Storage Unit” ([7]).
PAGE 69
3.3 Electronic Interface Figure 2 illustrates the electrical and electronic connections between the MCS-DR PC, MCS-DR storage unit, station power, and station subsystems. The insets of Figure 2 show expanded rear views of the MCS-DR PC and storage unit. Figure 2: Diagram of electrical connections 4 4.1 Monitor and Control Interface Overview Control and monitoring of the MCS-DR is performed by the exchange of two diﬀerent classes of messages.
PAGE 70
4.2 Timing Restrictions The MCS-DR supports up to 100 commands per second. Commands which schedule recording must allow at least 5 seconds between the receipt of the command, and the start of recording. Additionally, recordings may not be scheduled to begin within 5 seconds of the termination of a prior recording session. 4.3 MIB Index 2 2.1 Label CURRENT-OPERATION OP-TYPE 2.2 2.2.1 2.2.2 2.3 OP-SCHEDULE OP-START OP-STOP OP-REFERENCE 2.4 OP-ERRORS 2.5 2.5.1 OP-FILEINFO-INTERNAL OP-TAG 2.5.
PAGE 71
Index 6 6.1 Label REMOVABLE-DEVICES DEVICE-COUNT 6.2 6.2.X DEVICE-IDS DEVICE-ID-X 6.3 6.3.X DEVICE-STORAGES DEVICE-STORAGE-X 7 7.1 7.2 7.2.X CPU-INFO CPU-COUNT CPUTEMPS CPU-TEMP-X 8 8.1 HDD-INFO HDD-COUNT 8.2 8.2.X 9 9.1 HDD-TEMPS HDD-TEMP-X DATA-FORMATS FORMATS-COUNT 9.2 9.2.X 9.3 9.3.X FORMAT-NAMES FORMAT-NAME-X FORMAT-PAYLOADS FORMAT-PAYLOAD-X 9.4 9.4.X 9.5 9.5.X FORMAT-RATES FORMAT-RATE-X FORMAT-SPECS FORMAT-SPEC-X 10 10.1 10.2 10.2.
PAGE 72
4.4 MIB Entries in Detail 4.4.1 OP-TYPE MIB Entry: Index: Label: Description: Response Format: Operation Type 2.1 OP-TYPE OP-TYPE reports the current operation type. If no operation is in progress, it indicates the idle state. Response Element Type and Size Description Operation Type (ASCII-11-A) One of “Idle”, “Initialize”, “Record”, “Dump”, “Down”, “Synchronize”. Operation Type “Idle” “Initialize” “Record” “Copy” “Dump” “Down” “Synchronize” 4.4.
PAGE 73
4.4.3 OP-STOP MIB Entry: Index: Label: Description: Response Format: Current Operation Expected Stop-time 2.2.2 OP-STOP OP-STOP reports the scheduled or expected end-time of the current operation. This MIB entry is not valid if the current operation (as reported by MIB 2.1 Operation Type) is “Idle”, or “Down”. If Operation Type is “Copy”, or “Dump”, this entry will represent an estimation of completion time, and as such may not be accurate until average transfer rates have been determined.
PAGE 74
4.4.6 OP-TAG MIB Entry: Index: Label: Description: Response Format: Current Operation File Tag 2.5.1 OP-TAG OP-TAG reports the tag value used to identify the ﬁle in use by current operation. The ﬁle may be in read or write mode, depending on whether the current operation (as reported by MIB 2.1 Operation Type) is “Record”, “Copy”, or “Dump”. This MIB entry is not valid if the current operation is “Idle”, “Initialize”, “Down”, or “Synchronize”.
PAGE 75
4.4.8 OP-FILEPOSITION MIB Entry: Index: Label: Description: Response Format: Current Operation File Position Information 2.5.3 OP-FILEPOSITION OP-FILEPOSITION reports the start position, length, and current position of reading or writing with respect to the ﬁle in use by current operation (as reported by MIB 2.1 Operation Type). The Current Position value is always an oﬀset relative to Start Position. This MIB entry is only valid if the current operation (as reported by MIB 2.
PAGE 76
4.4.10 OP-FILEINDEX MIB Entry: Index: Label: Description: Response Format: Current Operation 2.6.2 OP-FILEINDEX OP-FILEINDEX reports the which ﬁle of the ﬁle series is being written to. This MIB entry is only valid if the current operation (as reported by MIB 2.1 Operation Type) is “Dump”. ’ Response Element Type and Size Description (ASCII-9-#) Indicates which ﬁle of the series is being dumped to. File index 4.4.
PAGE 77
4.4.13 DIRECTORY-COUNT MIB Entry: Index: Label: Description: Response Format: Directory File Count 4.1 DIRECTORY-COUNT DIRECTORY-COUNT reports the number of recordings contained on internal storage. Response Element Type and Size Description (ASCII-6-#) The number of recordings. Count 4.4.14 DIRECTORY-ENTRY-X MIB Entry: Index: Label: Description: Response Format: Directory Entry X 4.2.
PAGE 78
4.4.15 TOTAL-STORAGE MIB Entry: Index: Label: Description: Response Format: Total Storage 5.1 TOTAL-STORAGE TOTAL-STORAGE reports the total storage capacity of internal storage in bytes. Response Element Type and Size Description (ASCII-15#) Total size of internal storage in bytes. This number does not reﬂect the number of bytes unavailable due to formatting and ﬁle system usage.
PAGE 79
4.4.18 DEVICE-ID-X MIB Entry: Index: Label: Description: Response Format: Removable Device ID X 6.2.X DEVICE-ID-X DEVICE-ID-X reports the device id of the Xth external storage device. Response Element Type and Size Description Storage ID (ASCII-64-A) Linux partition (e.g. /dev/sdf1) of detected storage device. The device/partition must be formatted with the ext2 ﬁle system to be properly recognized and usable. 4.4.
PAGE 80
4.4.21 CPU-TEMP-X MIB Entry: Index: Label: Description: Response Format: CPU Temperatures 7.2.X CPU-TEMP-X CPU-TEMP-X reports temperature of the of core X. Response Element Type and Size Description (ASCII-3-#) Temperature in degrees Celsius of core X. Core X Temp 4.4.22 HDD-COUNT MIB Entry: Index: Label: Description: Response Format: HDD Count 8.1 HDD-COUNT HDD-COUNT reports the number of hard drives comprising internal storage.
PAGE 81
4.4.25 FORMAT-NAME-X MIB Entry: Index: Label: Description: Data Format X Name 9.2.X FORMAT-NAME-X FORMAT-NAME-X returns the name of the Xth recording format. Response Element Type and Size Description Format Name (ASCII-32-A) The name assigned to the format. Must include only numbers, letters, and the underscore character. Data formats should be named appropriately. e.g.: TBN 1024 112 for a TBN packet of 1024 bytes at a rate of 112 MiB/s. 4.4.
PAGE 82
4.4.28 FORMAT-SPEC-X MIB Entry: Index: Label: Description: Data Format X speciﬁcation 9.5.X FORMAT-SPEC-X FORMAT-SPEC-X returns the speciﬁcation of the Xth recording format. This speciﬁcation is an ordered list of Keep or Drop operations to be performed on portions of the received data packet. This feature’s primary use is in conserving storage space by discarding portions of a packet that may not be needed before the packet is written to disk.
PAGE 83
4.5 Control Commands Command Name INI REC DEL STP GET CPY DMP FMT DWN UP EJT SYN TST Description Initialize or restore the MCS-DR to its initial boot-up state. Schedule a recording operation with the start-time, duration, and data format speciﬁed. Delete existing recording speciﬁed by a supplied tag-value Stop the recording speciﬁed by a supplied tag-value, halting if inprogress, and canceling if not yet begun.
PAGE 84
4.6.1 INI Command: Description: Argument Format: Initialize Initialize restores the MCS-DR to the initial boot-up state in all regards except for the system log and the contents of internal storage. Argument Type and Size Description Flags (ASCII-256-A) To force re-initialization of the system log, specify the ﬂag “--flush-log” or “-L”. To force reinitialization of internal RAID storage, specify the ﬂag “--flush-data” or “-D”.
PAGE 85
Response Element Type and Size Description Tag (ASCII-16-A) A ﬁle name of the form _, where MJD is the MJD on which recording is scheduled to begin, and Reference number is the reference number of the command message which scheduled the recording. 4.6.3 DEL Command: Description: Argument Format: Delete This command deletes a recording from internal storage. Argument Type and Size Description Tag (ASCII-16-A) A ﬁle name of the form _.
PAGE 86
Response Element Type and Size Description Data (uint8)xLength On success, this ﬁeld will contain Length bytes of data from the speciﬁed position in the ﬁle. 4.6.6 CPY Command: Description: Argument Format: Copy The Copy command copies portions of a recording to a ﬁle an external storage device. If the ﬁle already exists, it will be overwritten without warning or notiﬁcation. The Copy and Dump commands are not available if there are any recordings scheduled.
PAGE 87
4.6.7 DMP Command: Description: Argument Format: Dump The Dump command copies blocks of data from a recording to a series of ﬁles on an external storage device. If any of the ﬁles already exist, they will be overwritten without warning or notiﬁcation. The Copy and Dump commands are not available if there are any recordings scheduled.
PAGE 88
4.6.8 FMT Command: Description: Argument Format: Format This command formats either internal storage, or an attached external storage device. When formatting an external device, the operation can require a substantial amount of time, proportional to the size of the device. To determine whether a format command completed, poll the OP-INFO MIB entry until it no longer indicates that an operation is in progress, and then request the REMOVABLE-DEVICES MIB entry.
PAGE 89
4.6.11 EJT Command: Description: Argument Format: Eject Eject un-mounts an external storage device in preparation for removal. Argument Type and Size Description Storage ID (ASCII-64-A) Linux partition (e.g. /dev/sdf1) of an attached external storage device. The device/partition must be formatted with the ext2 ﬁle system to be properly recognized and usable. 4.6.12 SYN Command: Description: 4.6.
PAGE 90
4.7 Error Messages Error Message Error Description Operation not permitted Operations scheduled or in progress prevent the execution of this command. The speciﬁed Format Name contains illegal characters. The speciﬁed Format Name is already in use. Speciﬁed UDP Packet Payload Size exceeds the maximum allowable size–determined by the Ethernet Jumbo Frames MTU less MAC, IP, and UDP header data. Speciﬁed limit is 8192 bytes.
PAGE 91
5 Control and Monitoring Session Examples The following examples walk through a usage scenario and demonstrate the types of command and monitor messages needed to operate the MCS-DR as well as the responses and error messages that might be generated. It should be noted that the error conditions in the scenario are atypical, and are included for the sake of demonstrating the interface. In the following examples, a single quote is used to denote spaces appearing in arguments and responses.
PAGE 92
5.
PAGE 93
5.
PAGE 94
5.
PAGE 95
5.
PAGE 96
5.
PAGE 97
5.
PAGE 98
5.
PAGE 99
5.
PAGE 100
6 Change Record Version 0.4 Date 2009-10-10 Aﬀected Section(s) All 0.3 2009-10-04 All 0.2 2009-09-24 All 0.
PAGE 101
References [1] J. Craig, “Long Wavelength Array Station Architecture,” Ver. 2.0, Long Wavelength Array Memo 161, Feb. 26, 2009. [online] http://www.phys.unm.edu/∼lwa/memos. [2] S. Ellingson, “MCS Architecture,” Ver. 3, Long Wavelength Array Memo MCS0007, Feb. 25, 2009. [online] http://www.ece.vt.edu/swe/lwavt/. [3] S. Ellingson, “MCS Subsystem Deﬁnition,” Ver. 2, Long Wavelength Array Engineering Memo MCS0004, Feb. 23, 2009. [online] http://www.ece.vt.edu/swe/lwavt/. [4] S. Ellingson, “MCS Common ICD,” Ver.
PAGE 102
File: /home/steve/Desktop/readme.txt Page 1 of 4 MCS-DR Operating Software Version 0.8 C.N. Wolfe Nov. 9, 2009 Introduction ============ This archive contains the MCS-DR Operating Software (MCS-DROS or DROS). DROS is designed to accept commands from MCS to record and later retrieve data provided by DP. In this tarball exists all of the souce for the executable, as well as make files needed to build it.
PAGE 103
File: /home/steve/Desktop/readme.txt Page 2 of 4 crash the system. Safeguards against such possibilities are developed for future releases, but not included with this release. Additional Files required ========================= DROS expects to be installed to a specific path, and expects to find additional support files at specified loactions. /LWA/scripts /LWA/config /LWA/bin /LWA/database Contains the launch.sh host script. see sec. "Host Script" Conatins defaults.cfg, and formats.
PAGE 104
File: /home/steve/Desktop/readme.txt Page 3 of 4 | FORMAT-NAME-2 TEST_TBN | | FORMAT-RATE-2 117440512 | | FORMAT-SPEC-2 K100D100K100D724 | | FORMAT-PAYLOAD-2 1024 | | | \----------------------------------------------------------------------------------------/ Host Script =========== The DROS executable is launched from within a simple bash shell script which facilitates the SHT and INI commands. Without this script, those commands will not function as expected.
PAGE 105
File: /home/steve/Desktop/readme.txt Page 4 of 4 Main.c Defines.h Globals.h performs initialization, main loop, message rx/tx loop, shutdown defines several constants and macros used throughout defines several globals which are frequently required Message.c MessageQueue.c Socket.c RingQueue.c Message.h MessageQueue.h RingQueue.h Socket.h Wrapper functions and data structure definitions for POSIX features and data buffering mechanisms Persistence.c Persistence.
PAGE 106
� ��
PAGE 107
� ��
PAGE 108
THE DELL PRECISION™ R5400 The Dell Precision™ R5400 is a high performance dual socket 2U rack workstation that provides an industry-standard alternative to blade workstations providing fully scalable high performance graphics options. A RACK WORKSTATION THAT IS BIG ON PERFORMANCE AND FLEXIBILITY Looking for a high-end workstation that delivers world class performance and exceptional processing and graphics power but is engineered for a high density rack environment? Then look no further.
PAGE 109
DELL PRECISION™ R5400 RACK WORKSTATION System Processors Dual-Core (6MB L2 cache) & Quad-Core (2X 6MB L2 cache) Intel® Xeon® Processors. Operating Systems Genuine Windows Vista® Ultimate 32Bit; Genuine Windows Vista® Ultimate 64Bit; Genuine Windows Vista® Business 32Bit; Genuine Windows Vista® Business 64Bit; Genuine Windows Vista® Ultimate Downgrade, XP Professional; Genuine Windows Vista® Ultimate Downgrade, XP Professional x64 Edition; Red Hat® Enterprise Linux WS v.5 EM64T.
PAGE 110
��
PAGE 111
DELL™ ™ POWERCONNECT 6200 SERIES SWITCHES The PowerConnect 6200 series is Dell’s most advanced switching product line, offering advanced switching capabilities including high-density, high-performance stacking and 10 Gigabit Ethernet capabilities scalable from the small business to the Enterprise Edge. High Density The PowerConnect 6200 series offers versions with 24 and 48 ports of 10/100/1000BASE-T Gigabit Ethernet.
PAGE 112
Advanced Security Support for L2-L4 Access Control Lists (ACLs) on the switch allows the user to perform deep packet inspection. 802.1x port authentication offers both single and multiple host access.
PAGE 113
FEATURES Dell™ PowerConnect™ 6248 Dell™ PowerConnect™ 6224 Dell™ PowerConnect™ 6224F High Performance all-Fiber Gigabit Ethernet Switch.
PAGE 114
� ��
PAGE 115
� ��
PAGE 116
� ��
PAGE 117
� ��
PAGE 118
� ��
PAGE 119
� ��
PAGE 120
� ��
PAGE 121
� ��
PAGE 122
� ��
PAGE 123
� ��
PAGE 124
� ��
PAGE 125
� ��
PAGE 126
The Dell Online Store: Build Your System http://configure.us.dell.com/Dellstore/print_summary_details_popu... ® TM Windows . Life without Walls . Dell recommends Windows 7. Precision T1500 64 bit $1,989 Starting Price Quickly create your own professionallooking custom forms, such as customer estimates, invoices and reports by using any of the over 100 included templates. Upgrade to Quickbooks Software Now! 1 Lease from $53/mo.
PAGE 127
The Dell Online Store: Build Your System http://configure.us.dell.com/Dellstore/print_summary_details_popu...
PAGE 128
Dell Precision™ T1500 Workstation Performance, Desktop Value Looking for a cost-effective system to drive your Computer Aided Design productivity? Look no further. The Dell Precision T1500 is an excellent system providing advanced performance for workstation applications like AutoCAD®. It boasts workstation-class professional graphics card options, yet offers a cost-effective combination of technologies designed to help make the most of ever tightening budgets.
PAGE 129
FEATURES Dell Precision™ T1500 Quad-core Intel® Core® i7 and i5 processors Processors All processors are 64-bit, select CPUs support Turbo Mode and HyperThreading technology Microsoft® Windows 7® Business and Ultimate Microsoft® Windows Vista® Business and Vista® Ultimate, Operating Systems Microsoft® Windows Vista® Business and Vista® Ultimate 64-bit Microsoft® Windows® XP Professional via Vista downgrade Ubuntu 9.