Measuring Voice Quality
Subjective Testing
Subjective testing is the most "authentic" method of measuring voice quality however is a specialized and costly process. This approach is typically used by CODEC designers and equipment manufacturers to validate VoIP technology before deployment. A MOS score ranges from 1 for an unacceptable call to 5 for an excellent call. A typical range for Voice over IP would be from 3.5 to 4.2.
P.861 (PSQM)/ P.862 Testing
PSQM and the newer P.862 are used to analyse the distortion that has occurred on test voice signals that have been transmitted through a VoIP network, and to produce an estimated MOS score. These algorithms are implemented in test equipment available from a number of companies. The advantage of this approach is that it is measuring the effects of many different impairments and their interactions - the disadvantage is that it requires a call to be set up through the network for each test.
Non-Intrusive Monitoring
Non-Intrusive or Passive Monitoring examines a stream of voice traffic and produces a transmission quality metric that can be used to estimate a MOS score. This has the advantage that all calls in a network can be monitored without any additional network overhead but the disadvantage that the effects of some impairments are not incorporated.
VQmon provides passive monitoring through observation of the RTP stream and incorporates effects such as packet loss burstiness. This produces an R Factor which can be used to estimate a MOS score. VQmon can be embedded into VoIP Gateways and other end systems with virtually no impact on equipment cost or network traffic. Reports can be sent to VoIP management applications (e.g. SQmediator) through the RFC 6035 SIP based performance reporting protocol.
The E Model
The E Model was originally developed within ETSI as a transmission planning tool, described in ETSI technical report ETR 250, and then standardized by the ITU as G.107. The objective of the model was to determine a quality rating that incorporated the "mouth to ear" characteristics of a speech path. The range of the R factor is nominally from 0-100 however values of below 50 are generally unacceptable and typical telephone connections don't get above 94 giving a typical range of 50-94. For wideband CODECs the R factor may increase above 100, typically 110 for an unimpaired connection.
The basic model is: R = Ro - Is - Id - Ie + A + W
Where Ro is a base factor determined from noise levels, loudness etc. Is represents impairments occurring simultaneously with speech, Id represents impairments that are delayed with respect to speech, Ie represents the so-called "equipment impairment factor" and A the "advantage factor" and W is a wideband correction factor.
The Equipment Impairment factor, Ie, is typically used to represent the effects of Voice over IP. For example, assuming default values for everything else then the R Factor for an "ideal" G.729A connection with no loss, jitter or delay would be R = Ro - Ie = 94 - 11 = 83
The Advantage factor, A, is used to represent the convenience to the user of being able to make the phone call, i.e. a cellphone is convenient to use therefore people are more forgiving on quality.
Acceptable MOS and R Scores for Narrowband CODECs
| User Opinion | R Factor | MOS Score |
| Maximum obtainable for G.711 | 93 | 4.4 |
| Very satisfied | 90-100 | 4.3-5.0 |
| Satisfied | 80-90 | 4.0-4.3 |
| Some users satisfied | 70-80 | 3.6-4.0 |
| Many users dissatisfied | 60-70 | 3.1-3.6 |
| Nearly all users dissatisfied | 50-60 | 2.6-3.1 |
| Not recommended | 0 - 50 | 1.0-2.6 |
Note: MOS scores, although widely used for comparison, do vary from test to test. Note also that typical subjective test for G.711 may result in a MOS score of around 4.2 whereas the assumed MOS score for G.711 that results from the use of the E Model or P.862 is 4.4.
For Wideband CODECs the MOS score range is still 1-5 even though the R factor range is higher. This means that a Narrowband CODEC may have a MOS score of 4.3 and a Wideband CODEC may have a MOS score of 3.9, even though the Wideband CODEC sounds much better.
Conditions of use: The material on this site is copyright VoIP Troubleshooter LLC and may be freely used but not copied or downloaded. In making use of this site the user acknowledges that VoIP Troubleshooter LLC or Contributor has no liability for any issues or problems that may arise directly or indirectly as a result of such use. VoIP Troubleshooter LLC and Contributor are providing this material as-is with no warranty as to correctness or completeness and do not accept any responsibility for any issues or problems of any nature whatsoever that may arise from the use of the material on this site.