Nomad User’s Manual Chapter 8 – Analyzing the Results 8 Analyzing the Results Once a suitable amount of data has been collected, use the Stop button to terminate data collection. If the current test has been set to end after a specific number of cycles or calls, data collection will stop automatically at the conclusion of the test sequence. When a Voice Quality test is stopped, the End Active Calls dialog will appear.
Nomad User’s Manual Chapter 8 – Analyzing the Results 8.1 Generating a Report In most cases, Nomad will automatically merge the downlink data collected at the test PC with the uplink data collected at the Audio or Call Server provided that an internet connection is available at merge time. The merged file will be listed on the Data → Complete tab and will be available for report generation.
Nomad User’s Manual Chapter 8 – Analyzing the Results To generate a report using merged session files: Select one or more files to be included in the output report. Click the Generate Report From Selected button. If multiple session files have been selected, choose whether to Merge multiple logging sessions into one report or to Produce one report for each logging session. Figure 8-3 - Output File Type Selection The Review Merged Session Report Header Data dialog will appear.
Nomad User’s Manual Chapter 8 – Analyzing the Results Figure 8-4 - Review Merged Session Report Header Data Dialog The source files used to generate the formatted output report and KML file for geographic visualization of MOS data will be listed on the Data → Reports tab. A report file created from a single test file will take the name of that file. A report file created from multiple test files will take the name assigned by the user during the report generation process.
Nomad User’s Manual Chapter 8 – Analyzing the Results Note: If the View Standard link is not present, reports must be generated by a team member with access to this functionality. Contact Spirent Support if you believe this capability has been omitted in error. In the Save As dialog, provide a name for the formatted Excel report. Note: Formatted output reports are displayed in .XLSX format. Microsoft Excel 2007 or newer is required to open files in this format.
Nomad User’s Manual Chapter 8 – Analyzing the Results Figure 8-8 - Device Overview Report Landing Page Formatted Voice Quality data can be found on the Voice Quality Summary tab of the output and includes statistical and distribution information for Voice Quality session results. Figure 8-9 - MOS Statistical Summary Copyright © Spirent Communications, Inc.
Nomad User’s Manual Chapter 8 – Analyzing the Results Figure 8-10 - Downlink MOS Distribution Figure 8-11 - Uplink MOS Distribution Formatted Delay Performance data can be found on the Delay Performance Summary tab. Copyright © Spirent Communications, Inc.
Nomad User’s Manual Chapter 8 – Analyzing the Results Figure 8-12 - Delay Performance Summary Report Formatted Call Performance data can be found on these tabs: Call Performance Summary Call Initiation Call Retention Audio Verification Device Performance Copyright © Spirent Communications, Inc.
Nomad User’s Manual Chapter 8 – Analyzing the Results Figure 8-13 - Call Performance Summary Report Figure 8-14 - Call Initiation Performance Report Copyright © Spirent Communications, Inc.
Nomad User’s Manual Chapter 8 – Analyzing the Results Figure 8-15 - Call Retention Performance Report Figure 8-16 - Audio Verification Performance Report Copyright © Spirent Communications, Inc.
Nomad User’s Manual Chapter 8 – Analyzing the Results Figure 8-17 - Device Performance Report 8.3 Report Template Customization Nomad supports report template customization. To customize a report template: Find the default Nomad report template in the Templates directory of the Nomad installation location (typically C:\Program Files\Spirent Communications\Nomad). The name of the file is Nomad-Template.xlsx. Make a copy of the Nomad-Template.xlsx template file in the same directory.
Nomad User’s Manual Chapter 8 – Analyzing the Results 8.4 Interactive Viewer To display the Interactive Viewer: On the Data → Reports tab, click the Interactive button for the data of interest. Figure 8-19 – Interactive button on Reports tab This will display the Interactive Viewer, which will represent cycle level data from each channel in the session report. For Voice Quality data, a graph will be shown displaying the MOS scores over time.
Nomad User’s Manual Chapter 8 – Analyzing the Results Figure 8-20 – Interactive Viewer Voice Quality data For Call Performance data, a table will be shown displaying the results of each call: Copyright © Spirent Communications, Inc.
Nomad User’s Manual Chapter 8 – Analyzing the Results Figure 8-21 – Interactive Viewer Call Performance data 8.5 Visualizing Data Geographically The Nomad KML output capability provides a simple method of visualizing MOS and Call Performance data geographically. Google EarthTM must be installed in order to take advantage of geographic visualization. Google EarthTM may be obtained from: http://earth.google.com An internet connection is required for map access while running the program.
Nomad User’s Manual Chapter 8 – Analyzing the Results Figure 8-22 - Naming the KML Map File The KML Generation Complete dialog will appear. Choose whether to view the data now. The data can be viewed at any time by opening the .KML file using the File → Open command from the main menu in Google EarthTM. Figure 8-23 - KML Generation Complete Dialog Google EarthTM will open and zoom to the map location.
Nomad User’s Manual Chapter 8 – Analyzing the Results Figure 8-24 - Geographic Display of MOS Data in Google EarthTM In the Places window in the left-hand panel, use the selection boxes to isolate the data by channel and link. For example, this map has been customized to include only Downlink data for Channel 1. Figure 8-25 - Isolating the Data of Interest The map image may be saved as a .JPG file by selecting File → Save → Save Image. 8.
Nomad User’s Manual Chapter 8 – Analyzing the Results On the Data → Complete tab, select one or more files with which you wish to associate data in a pcap file. Click the Import IP Analytics File button. Navigate to a pcap file that was collected on the device or server side during the session(s) selected. The data from the pcap file will be analyzed and correlated with the time period of the selected session(s).
Nomad User’s Manual Chapter 8 – Analyzing the Results Figure 8-27 – Selecting Displayed data. The IP Analytics data can also be viewed in the Interactive Viewer. To view the data in the Interactive Viewer, click the Interactive button for the corresponding report on the Data → Reports tab. For Call Performance tests, a table will be displayed for each call result. To display the associated stream data, click on a call result within the table.
Nomad User’s Manual Chapter 8 – Analyzing the Results These charts include RTP Throughput, RTP Jitter, and RTP Relative Delay over time, as well as charts of the distribution of RTP Relative Delay and Distribution of RTP Jitter. For Voice Quality tasks, a graph will be displayed displaying the MOS scores for the task. Beneath the graph will be displayed the above mentioned charts for the duration of the task for all associated streams (Uplink and Downlink).
Nomad User’s Manual Chapter 8 – Analyzing the Results Figure 8-29 – Interactive Viewer – Voice Quality data Copyright © Spirent Communications, Inc.
Nomad User’s Manual Chapter 8 – Analyzing the Results 8.7 Collecting Random Audio Sample Files (Voice Quality Testing) Nomad has the ability to set aside a random sample of audio files from a Voice Quality test within each MOS performance range. This allows users to audibly verify the performance of Great, Good and Bad audio samples. Random audio file sampling must be enabled prior to data collection on the Settings → Voice Quality tab in order to capture this data (see Section 4.4.1).
Nomad User’s Manual Chapter 8 – Analyzing the Results 8.8 Viewing the Logs The log file containing the messaging for each test is stored in the Log File Storage Location specified in the Start Logging Session Wizard (see Section 6.4). The log files are maintained in an open-format, comma-delimited file for direct access to the information. Figure 8-31 - Nomad Test Log 8.
Nomad User’s Manual Chapter 8 – Analyzing the Results Figure 8-32 - PESQ Tools GUI The PESQ algorithm calculates MOS by comparing audio samples degraded by the communication channel to the original source sample. Therefore, the PESQ Tools GUI requires both the original and the degraded samples as inputs. To analyze a degraded sample: 1. Open the Reference File From the Ref File tab of the control panel, click on the Open button in the Reference file path area.
Nomad User’s Manual Chapter 8 – Analyzing the Results 2. Open the Degraded File From the Deg File tab of the control panel, click on the Open button in the Degraded file path area. Browse to the storage location for the degraded log file and open the file. Degraded session log files are stored in the Log File Storage Location specified in the Start Logging Session Wizard (see Section 6.4). Figure 8-34 - Deg File Tab 3.
Nomad User’s Manual Chapter 8 – Analyzing the Results 8.10 Troubleshooting Incomplete Data On occasion, Nomad may be unsuccessful in merging the uplink and downlink data from a test session into a unified log file for report generation. Such files will appear on the Data → Incomplete tab at the conclusion of a test (successfully merged files appear on the Data → Complete tab).
Nomad User’s Manual Chapter 8 – Analyzing the Results Nomad provides several options to modify an incomplete logging session to render it suitable for report generation. Click on the Session dialog.
Nomad User’s Manual Chapter 8 – Analyzing the Results Figure 8-38 - Edit Channel Configuration Dialog To troubleshoot other incomplete data issues: Data collected on a Remote Unit must be copied to the local machine for merging and report generation. Once these files are available locally: o Use the Import Logging Session button to display the data on the Incomplete tab. o Select both the local and remote data and choose Merge Selected.
Nomad User’s Manual Chapter 8 – Analyzing the Results 8.11 Offline Scoring (Voice Quality Testing) The ability to generate MOS outputs for .WAV files outside of an active test session is called Offline Scoring. Use Offline Scoring to retrieve .WAV files stored on the Audio Server when uplink results are not available during data collection and data must be retrieved after test completion. Uplink .WAV files are stored on the Audio Server for three weeks after collection.
Nomad User’s Manual Chapter 8 – Analyzing the Results Figure 8-39 - Uplink Scoring Utility Settings Screen 8.11.1 Creating a New Logging Session Using Offline Scoring On the Session Selection Screen of the Uplink Scoring Utility: Select the start and end date/time values in UTC. These can be found by opening the corresponding downlink log file and noting the timestamps of the first and last entries. Enable the channels for which to retrieve data.
Nomad User’s Manual Chapter 8 – Analyzing the Results 8.11.2 Retrieving Complementary Uplink Data for an Existing Logging Session On the Session Selection Screen of the Uplink Scoring Utility: Enable the channels for which to retrieve data. The details of the downlink voice quality task will appear for each channel, including start and end time of data collection. Specify the session(s) for which to retrieve the audio files.
Nomad User’s Manual Chapter 8 – Analyzing the Results 8.12 Scoring PESQ and POLQA It may be desirable to score Voice Quality test results using both the PESQ and POLQA scoring models. On the first pass, the data will always be scored using the Scoring Model specified in the test definition. These options exist to score the same data using the other Scoring Model: Re-Score Files Batch Scoring Uplink Offline Scoring 8.12.
Nomad User’s Manual Chapter 8 – Analyzing the Results Figure 8-43 - Start Re-scoring Figure 8-44 - Re-scoring in Progress Find the re-scored file on the Data → Incomplete tab. The file name will contain the word “rescored” for identification purposes. Use the Retry Auto-Complete button to move the re-scored file to the Complete tab. Generate a report from the re-scored file. Copyright © Spirent Communications, Inc.
Nomad User’s Manual Chapter 8 – Analyzing the Results Nomad automatically detects and uses the Scoring Model not used to score the original file. For example, if POLQA was used to score the original file, PESQ will be used when re-scoring. The Scoring Algorithm in use can be verified on the Voice Quality Summary tab of the output report.
Nomad User’s Manual Chapter 8 – Analyzing the Results Click Start Scoring to accept the settings and proceed. Figure 8-46 - Batch Scoring in Progress When scoring is complete, processing and MOS statistics will be presented for the selected scoring algorithm(s). Figure 8-47 - Batch Scoring Complete Copyright © Spirent Communications, Inc.
Nomad User’s Manual Chapter 8 – Analyzing the Results Use the Open File button to view the processed data. This delimited text file can be analyzed in its raw form, opened in Excel or used as the basis of a custom processing script. Figure 8-48 - Batch Scoring Output 8.12.3 Offline Scoring The Nomad Offline Scoring utility provides a method of retrieving and scoring uplink .WAV files from the Audio Server outside of an active test session.
Nomad User’s Manual Chapter 9 – Voice Quality Configuration Options 9 Voice Quality Configuration Options Nomad provides numerous configuration options for voice quality testing.
Nomad User’s Manual Chapter 9 – Voice Quality Configuration Options 9.1 Voice Quality Test Calibration Spirent Communications recommends calibrating the Spirent ME or Spirent HD ME hardware used with the Nomad system prior to testing for the first time to verify proper operation.
Nomad User’s Manual Chapter 9 – Voice Quality Configuration Options 9.2 Audio Server Testing Spirent Communications provides access to a centralized call server that sources downlink data and records uplink data in the audio server testing configuration. The mobile handset will source speech while the audio server will record and score the uplink speech sample for ten seconds.
Nomad User’s Manual Chapter 9 – Voice Quality Configuration Options 9.3 IMS Client Testing Nomad provides the option for Utilizing a local IMS Client as the Uplink Device. The mobile handset will source speech while the IMS Client will record the uplink speech sample for ten seconds, acting in the traditional role as the voice server. In the next ten second cycle, the IMS client will source speech while the mobile handset records the downlink speech sample.
Nomad User’s Manual Chapter 9 – Voice Quality Configuration Options Figure 9-3 - Mobile-to-Mobile Hardware Configuration On the Nomad Test Setup screen, configure Channel 1 as a Mobile task with the following settings: o Session → Uplink Device: Base task on channel 2 o Channel Settings → Audio interface for this channel: Analog Interface(ME), Narrowband (ME HD) or High Definition (ME HD) On the Nomad Test Setup screen, configure Channel 2 as a Base task with the following settings:
Nomad User’s Manual Chapter 9 – Voice Quality Configuration Options Figure 9-4 - Mobile-to-Mobile Testing Assuming all four channels are available for testing, a second Mobile-to-Mobile test may be performed simultaneously on Channel 3 and Channel 4. Configure this test following the same instructions as above. In the Nomad output report generated with Mobile-to-Mobile test data, find results for audio received at Channel 1 in the Channel 1 – DL area.
Nomad User’s Manual Chapter 9 – Voice Quality Configuration Options 9.5 Remote Unit Testing Nomad provides the option for Remote Unit testing. In this configuration, a handset connected to the local Spirent ME hardware unit sources audio to and receives audio from a handset connected to a second hardware unit. In many cases, the second hardware unit is located remotely (i.e. in another office or city). One handset will source speech while the other will record speech for ten seconds.
Nomad User’s Manual Chapter 9 – Voice Quality Configuration Options End the test when desired. Assuming all four channels are available for testing, up to four Remote Unit test calls may be placed simultaneously. Spirent Communications recommends aligning calling handsets on the local unit with the receiving handsets on the remote unit. For example, Handset 1 on the local unit should call Handset 1 on the remote unit; Handset 2 on the local unit should call Handset 2 on the remote unit, etc.
Nomad User’s Manual Chapter 9 – Voice Quality Configuration Options 9.6 Landline Module Testing The Nomad Landline Module provides an additional option for voice quality testing without needing the Audio Server to handset uplink data collection and downlink audio sourcing. In the Landline Module configuration, the mobile end consists of the standard test handset and Spirent ME unit connected to a laptop which handles downlink data collection and uplink audio sourcing.
Nomad User’s Manual Chapter 9 – Voice Quality Configuration Options 9.7 Base Station Simulator Testing The Rohde & Schwarz CMU200 and the Agilent 8690 are two commercial base station simulators commonly used to simulate radio conditions in a test lab. An optional Nomad upgrade provides the cables required to interface to these pieces of equipment for degraded channel, noise cancellation and other test scenarios.
Nomad User’s Manual Chapter 9 – Voice Quality Configuration Options If testing is performed using Spirent HD ME hardware, make sure the proper Audio interface (High definition or Narrowband) is selected. Establish a call between the handset and the base station simulator. Start a new test. The test sequence starts with the handset sourcing data to the base station simulator. Data collected at the simulator will be replayed in the Channel 2 area of the Nomad interface.
Nomad User’s Manual Chapter 9 – Voice Quality Configuration Options 9.8 Head and Torso Simulator A HATS system may be used in conjunction with the test mobile and base station simulator in Nomad testing to realistically simulate the effect of an adult head and torso on voice quality. The HATS system may be configured with the base station simulator in two different configurations. In each configuration, MOS results at the handset will be displayed in the Nomad Channel 1 area during testing.
Nomad User’s Manual Chapter 9 – Voice Quality Configuration Options o Channel Settings → Output: 100 Channel 2 – Connects to base station simulator – “Uplink” o Task Type: Base o Session → Downlink Device: Mobile task on channel 1 o Channel Settings → Audio interface for this channel: Analog Interface(ME), Narrowband (ME HD) or High Definition (ME HD) o Channel Settings → Input: 180 o Channel Settings → Output: 200 Channel 3 – Connects to AMP and HATS mouth speaker – “mouth” o Task Type: Mobile o
Nomad User’s Manual Chapter 9 – Voice Quality Configuration Options 9.8.
Nomad User’s Manual Chapter 9 – Voice Quality Configuration Options 9.9 Wideband Testing Nomad contains a wideband speech stimulus designed to exercise various wideband AMR codecs enabled on a wideband AMR device under test. Wideband testing works as follows: During the audio quality testing process, Nomad injects a wideband speech stimulus into the communication test path. The degraded speech at the receiving end is compared to the original wideband reference speech.
Nomad User’s Manual Chapter 9 – Voice Quality Configuration Options Set the Output level to 195. End the test when desired. 9.10 Multi-RAB Testing The objective of Multi-RAB testing with Nomad is to compare the voice quality or call performance of a mobile engaged in data services to a device not transferring data. An Email Campaign may be launched during a Voice Quality or Call Performance task to test Multi-RAB performance.
Nomad User’s Manual Chapter 9 – Voice Quality Configuration Options o If desired, Generate Test Email to be sent to the mobile device. Note: In the event that Nomad becomes inaccessible during an Email Campaign (i.e. due to PC crash, etc.), it is possible to stop e-mail messages from being sent to the handset(s). Simply reply to any message generated by the Email Campaign to stop unwanted messages from being sent to the phone. Figure 9-12 - Email Campaign Dialog Start a new test.
Nomad User’s Manual Chapter 9 – Voice Quality Configuration Options 9.11 Voice Delay Testing Nomad allows the precise measurement of speech delay between two mobile devices connected to the same Spirent ME unit. The measurement includes delay introduced by both handsets and the time it takes to traverse the network. High levels of delay (generally over 250 milliseconds round-trip) may impact typical conversation.
Nomad User’s Manual Chapter 9 – Voice Quality Configuration Options Figure 9-14 - Voice Delay Task Session Dialog Spirent recommends leaving the default values for the remaining items in the Session dialog. Maintaining the default parameters ensures consistency across tests for benchmarking purposes.
Nomad User’s Manual Chapter 9 – Voice Quality Configuration Options The test sequence starts with Handset 1 sourcing audio to Handset 2. Assuming Alternate channels after each cycle has been selected, in the next cycle Handset 2 will source audio to Handset 1. The Average Delay and Median Delay will be plotted for each direction on the Delay Times (ms) / Cycle chart. Detailed statistics for each test cycle are available in the Cycle History area.
Nomad User’s Manual Chapter 9 – Voice Quality Configuration Options Figure 9-16 - Delay Performance Summary Report Copyright © Spirent Communications, Inc.
Nomad User’s Manual Chapter 10 – Call Performance Configuration Options 10 Call Performance Configuration Options Nomad provides various configuration options for call performance testing. These options include: Mobile Originated Testing Mobile Terminated Testing Mobile-to-Mobile Testing This section contains detailed instructions for each call performance configuration option. 10.
Nomad User’s Manual Chapter 10 – Call Performance Configuration Options Figure 10-1 - Call Campaign Dialog for a Mobile Originated Task Establish the Bluetooth connection for the test device. Click the Start Logging Session button and proceed through the Start Logging Session Wizard. Calls will automatically be placed from the test handset to the Call Server. During testing, the call status is displayed in the Call Performance Task Status Window.
Nomad User’s Manual Chapter 10 – Call Performance Configuration Options The test will stop after the configured number of Attempts. Alternatively, use the Stop button to end the test at any time as desired. Test results can be found on the Call Performance Summary, Call Initiation, Call Retention, Audio Verification and Device Performance tabs of the output report. Copyright © Spirent Communications, Inc.
Nomad User’s Manual Chapter 10 – Call Performance Configuration Options 10.2 Mobile Terminated Testing In a Mobile Terminated test, the Call Server makes calls to the mobile test device for call control testing. To perform Mobile Terminated call performance testing: Set the volume of the test handset to the maximum volume for optimal Audio Verification results. Determine which channel the test will be conducted on. All call performance testing operates via Bluetooth connection.
Nomad User’s Manual Chapter 10 – Call Performance Configuration Options Figure 10-3 - Call Campaign Dialog for a Mobile Terminated Task Establish the Bluetooth connection for the test device. Click the Start Logging Session button and proceed through the Start Logging Session Wizard. Calls will automatically be placed from the Call Server to the test handset. During testing, the call status is displayed in the Call Performance Task Status Window.
Nomad User’s Manual Chapter 10 – Call Performance Configuration Options Figure 10-4 - Mobile Terminated Call Performance Testing The test will stop after the configured number of Attempts. Alternatively, use the Stop button to end the test at any time as desired. Note: Losing internet connectivity during a Mobile Terminated Call Performance campaign may cause the Stop button to cease working. In this case, wait for a call to come to the phone, answer the call and dial 9999.
Nomad User’s Manual Chapter 10 – Call Performance Configuration Options 10.3 Mobile-to-Mobile Testing In a Mobile-to-Mobile test, one handset makes calls that are received by a second handset attached to a different channel on the same Spirent ME unit. To perform Mobile-to-Mobile call performance testing: Determine which handset will be the originating device and which handset will be the receiving device. All call performance testing operates via Bluetooth connection.
Nomad User’s Manual Chapter 10 – Call Performance Configuration Options Figure 10-6 - Mobile-to-Mobile Terminated Call Campaign Dialog Establish the Bluetooth connection for both test devices. Click the Start Logging Session button and proceed through the Start Logging Session Wizard. Calls will automatically be placed from the originating handset to the receiving handset. During testing, the call status is displayed in the Call Performance Task Status Window for each device.
Nomad User’s Manual Chapter 10 – Call Performance Configuration Options When testing is complete, the matching Mobile Originated and Mobile Terminated results are merged together to create a single session file. Generate a report in the standard manner for this merged session file. Test results can be found on the Call Performance Summary, Call Initiation, Call Retention, Audio Verification and Device Performance tabs of the output report.
Nomad User’s Manual Appendix A – Glossary Appendix A – Glossary General Testing Terms Term Call Performance Task Definition A task designed to measure how well the device performs with regard to call initiation, call retention, Bluetooth performance, signal strength and battery life. Configured Task A Configured Task saves all of the settings for the currently selected task. Session Profile A Session Profile defines the tasks and settings for all available test channels.
Nomad User’s Manual Appendix A – Glossary Term Mobile Task Definition The test device on a Mobile channel serves as the downlink device in the mobile communication path. This option is used for: Standard bi-directional testing using the Audio Server. A mobile acting as the downlink device in a test using the Landline Module. One of the test handsets in a bi-directional Mobile-toMobile test scenario (the other handset will be set as a Base).
Nomad User’s Manual Appendix B – Call Performance Events Appendix B – Call Performance Events Call Outcome Events Call Outcome No Service Definition There was no service available when the call was attempted. Failed Attempt A Mobile Originated Call was placed and an outgoing call was established followed by the phone’s return to the call placement state. Voicemail A Mobile Terminated call was placed and the nonstandard voicemail response was received from the mobile.
Nomad User’s Manual Appendix B – Call Performance Events Device State Events Device State Event Device State Value Definition The device may change to any of the following: o Initializing o Ready o Discovering o Connecting o Connected o Outgoing Call Established o Incoming Call Established o Active Call Signal Strength The phone signal strength corresponding to the handset bar display (0 – 5) Battery Life The phone battery life corresponding to the handset bar display (0 – 5) RSSI The RS
Nomad User’s Manual Appendix C – Confidence Interval Calculation Appendix C – Confidence Interval Calculation The Nomad Call Performance output reports employ a 90% Confidence Interval for call initiation failure and dropped call events. The Confidence Interval allows us to report, with 90% confidence, that the event rate for a given device will fall within the calculated range above and below the measured value.
Nomad User’s Manual Appendix D – PESQ Tools GUI Case Study Appendix D – PESQ Tools GUI Case Study The PESQ Tools GUI is available for Nomad users wishing to perform advanced analysis of any waveform captured during Voice Quality testing and scored using PESQ. Basic operation of the PESQ Tools GUI is described in Section 8.7. The PESQ Tools GUI can assist in the identification of issues that contribute to low MOS such as background noise and speech clipping.
Nomad User’s Manual Appendix D – PESQ Tools GUI Case Study The issue of speech clipping is visible when comparing the Reference Surface to the Degraded Surface: Figure D-3 - Reference Surface Figure D-4 - Degraded Surface Speech clipping is also apparent on the Error Surface graph. The Error Surface represents the Reference Surface minus the Degraded Surface. Therefore, errors that represent missing signal (i.e. clipping) will have positive values on this chart. Errors that add to the signal (i.e.
Nomad User’s Manual Appendix D – PESQ Tools GUI Case Study The Reference and Degraded Signal spectrums available in the PESQ Tools GUI provide another means of analysis. These figures can be used to visually compare the reference to the degraded frequency response. Note that although this example exhibits frequency fade above 3.5 kHz, this is a normal response for mobiles calling the Audio Server and is not expected to affect the MOS outcome.
Nomad User’s Manual Appendix D – PESQ Tools GUI Case Study Figure D-8 - Degraded Spectrogram In conclusion, this example has illustrated how the PESQ Tools GUI may be used to identify specific areas of a speech sample contributing to poor MOS results. In this case, multiple factors of background buzzing and speech clipping were identified. Using this information, the source of each issue may now be identified and rectified in order to achieve the desired MOS results.
Nomad User’s Manual Appendix E – Introduction to POLQA Appendix E – Introduction to POLQA The worldwide prevailing standard for mobile voice quality analysis has been ITU-T P.862, known as Perceptual Evaluation of Speech Quality (PESQ). PESQ implements automated testing of telecommunications using actual speech samples, comparison of the reference signal (transmitting side) to the degraded channel (listening side), and generation of mean opinion scores (MOS) to model subjective listening patterns.
Nomad User’s Manual Appendix E – Introduction to POLQA The table below compares PESQ and POLQA at a glance: Codecs PESQ AMR EFR POLQA AMR AMR-WB EFR EVRC EVRC-WB iLBC AMB+ AAC Skype / SILK G.711 G.729 Reference Speech Material 8 kHz 8 kHz 48 kHz Applications POTS VoIP 3G HD Voice Voice Enhancement Devices Skype Calls Benchmarking CDMA and GSM POLQA scoring is available as an optional upgrade to Nomad.
Nomad User’s Manual Appendix E – Introduction to POLQA Figure E-0-1 - Voice Quality Task Status Window for a POLQA Test When generating the output report, Nomad always scores the data using the Scoring Model specified in the test definition. For example, a test defined with the POLQA scoring model will be scored using POLQA.
Nomad User’s Manual Appendix E – Introduction to POLQA Appendix F – Nomad HD Hardware Figure F-0-1 – Spirent Communications HD ME Hardware Unit The Nomad HD hardware allows for connection to a mobile device via one of three interfaces: Narrowband, High Definition, or Bluetooth. Important Safety Note Any usage of the equipment in a manner not specified by the manufacturer may impair features related to safety and user protection.
Nomad User’s Manual Pwr Appendix E – Introduction to POLQA progress Red - Overdrive When lit, indicates that power is applied to the unit. Base Unit Physical Interfaces Interface HD1, HD2, HD3, HD4 NB1, NB2, NB3, NB4 PC GPS +12V Description Interface for High Definition Audio Interface for Narrowband Audio USB interface for communication with the PC USB interface for communication with a GPS device. The interface functions as a normal USB port for the PC. Power supply interface.
