Differentiated services testing key to customer satisfaction
Quality of service (QoS) and maximum throughput are essential for delivering data, voice and video on Ethernet networks, but traditional testing methods fall short in their ability to monitor, analyze and identify critical problems. For turning up VLAN, MPLS-, and IP-based services, and to keep service quality and revenues up, a multi-stream, multi-port, multi-service testing model is required.
Proper network pre-qualification and monitoring is imperative to ensure that service quality is maintained from the headend, through the network backbone, and to the customer premises as more customers are added and more services are offered.
|Figure 1: To verify differentiated services, the test set must generate traffic streams in the same format or formats (VLAN, MPLS, IP) used by the network architecture for each of the service types.|
Evolution of Ethernet services
Ethernet services have typically run point-to-point, from one business campus to another, for example, or from one data center to another. These early Ethernet services were relatively straightforward to turn-up, monitor and maintain, as the majority of testing was isolated to a single dark fiber, a single wavelength on a DWDM network, or a channel in an Ethernet over SONET/SDH network. Performance was verified by completing hard loop back tests running at data rates of 10 Mbps, 100 Mbps, or even 1 Gbps, based on the bandwidth sold.
Under those circumstances, performance verification of Ethernet services used to be a simple task. Many services were sold as a fixed bandwidth, such as 100 Mbps, without any Class of Service (CoS) designation, only a promise of bandwidth, which in most instances was not guaranteed.
Service providers did not provide traffic grooming or policing to ensure the QoS; they simply installed the network, verified end-to-end throughput, made a note of the roundtrip delay, and performed troubleshooting when customers complained. However, as the demand for Ethernet services and bandwidth continues to increase, network providers are under pressure to conform to standards established by the ITU and Metro Ethernet Forum (MEF) and to add more capacity without impacting existing services.
When selling commercial Ethernet services, today’s providers have the option to offer multiple tiers of service based on the level of CoS. By grooming networks to handle these differentiated services, providers have significantly more flexibility than they had when Ethernet service was sold like traditional point-to-point TDM. In addition, Ethernet over copper solutions provide greater granularity in bandwidth control and cost savings when using their existing T1 and T3 infrastructure.
The commercial services offered range from LAN extension and Internet access, to voice, video and data storage. Because these services all reside on the same network, each consuming a portion of the shared bandwidth, they all must be qualified at the same time. Each of these types of traffic have very different requirements, and stress the network in a unique manner.
Table 1: VLAN user priority.
Voice, for instance, requires extremely low latency, typically 50 milliseconds from end-to-end. Any delay or gap in transmission is instantaneously noticeable by users. In a generation of packet-based and mobile telephony, latency is perhaps even more important than the sound quality of voice, meaning a few dropped packets are more acceptable than any form of packet delay.
Video presents a completely different set of challenges. With video, extremely large amounts of data are transported from a central video server or headend to the customer premise, and ultimately, to the television set. Home video delivery is not a real-time application like video conferencing (which has the same latency requirements as voice calls). Small changes in latency go unnoticed and are typically smoothed over by constant buffering on the television set. The biggest challenge with sending large amounts of data over a long period of time is ensuring every packet is delivered. If packets are dropped, the picture can pixelate, freeze or disappear altogether.
Internet traffic and non-critical data services, such as LAN extensions, are the least stringent in terms of service level parameters when compared to voice and video. If a few frames are lost, traditional TCP/IP mechanisms manage data restoration adequately, provided the frame loss stays under 1 percent or so. Latency is also an issue, but in many cases, the primary source of delay is not with the Ethernet service, but with the ISP and Internet servers themselves. For mission-critical data services, such as storage area networks (SAN), latency and packet loss must be minimized.
Storage over Ethernet is a low-cost alternative to Fibre Channel and other SAN technologies that integrate with existing LAN architectures. However, the critical nature of storage data and the protocols used place tougher restrictions on frame loss and latency. Fortunately, the transfer of storage data often can be scheduled to avoid the network loads generated by other services, such as performing backup functions in the middle of the night when voice and LAN traffic are at a minimum.
In addition to customer services, it is important to verify network management traffic, which signals devices such as routers and switches and directs communications between them. This traffic may account only for about 1 percent of the total traffic on the network, yet it is critical that it is transmitted reliably, and typically has the highest priority of all traffic.
Services are differentiated in the network in a number of ways, depending on the architecture. With VLAN-based services, including those using stacked VLAN tagging or Q-in-Q, each service is assigned a VLAN tag and a user priority. The priority is a number from 0 to 7 based on the type of traffic (see Table 1). MPLS-based services can utilize the 3-bit experimental field for Class of Service designation, using a scheme similar to VLAN. IP-based services utilize the Type of Service (TOS) field in the IP header. Of course, some networks combine one or more of these technologies. Table 1 shows the typical VLAN user priorities and the associated traffic types.
Evolution of testing methodology
Traditional test sets have focused on testing one piece of this puzzle at a time, testing each type of service separately. With new triple play services and the need to generate multiple types of traffic simultaneously, these test sets no longer provide adequate test coverage. Because differentiated services now exist side-by-side on the network, rather than on their own dedicated fibers, wavelengths, or TDM tributaries, they must be tested simultaneously.
To verify differentiated services, the test set must generate traffic streams in the same format or formats (VLAN, MPLS, IP) used by the network architecture for each of the service types. The priority and/or TOS for each traffic stream must be specified, based on the appropriate class of service.
At the far end, the test set must separate out each traffic stream and perform QoS measurements on each. (See Figure 1.) While each service tested individually may meet QoS standards, the increase in traffic load caused by multiple services will negatively impact performance. Therefore, services should be tested at their maximum subscribed rate, and also at higher rates to verify network policing and load management.
To verify that a network element can maintain QoS, it must be loaded to full capacity. Rather than test through a single ingress and egress port, testing a single customer connection, the device must be tested through all ports. Even if the full port capacity of the device is not being utilized, the performance must be characterized upon installation so that the future capacity can be gauged.
The three key metrics to characterizing differentiated services are:
- frame loss,
- latency, and
- packet jitter or packet delay variation.
When one or more packets of data traveling across a network fails to reach its destination, the effect upon the service may be minor, but may lead to further service degradation as packets are re-sent. Service testing must verify that the ratio of lost frames falls within the acceptable limits defined by the class of service and that higher-priority services have a lower loss ratio than lower-priority ones.
Table 2: M.2301 IP QoS class definitions and network performance objectives for an end-to-end IP flow.
Latency is the delay between the time a frame is transmitted and when it is received. Low latency, as described, is critical for voice, as well as for SAN over Ethernet, where increased latency requires larger buffer-to-buffer credits. It also negatively impacts TCP sessions, where increases in latency have a profound effect on throughput.
Packet jitter or packet delay variation is the difference in the time of arrival of the packets. For classic data applications, jitter is easily managed and not a key parameter. But for voice and video, jitter becomes a critical parameter that must be tested and verified to ensure quality of service.
Table 2 summarizes some of the ITU-T QoS standards for various classes of service.
In some cases, bit error ratio (BER) is used as a QoS metric due to its traditional importance to TDM networks. BER is calculated by taking the ratio of errored data bits received to the number of bits transmitted. While an interesting measurement, it can be misleading, as it is possible, for example, to measure a BER of zero on all received frames and still have a data loss of 97 percent. For this reason, Ethernet service metrics do not rely on BER testing.
With the increasing importance of CoS standards, Carrier Class Ethernet certification, and real-time applications, assuring QoS is a critical step when offering revenue-generating commercial Ethernet services. This assurance comes from properly testing all of the differentiated services using multi-stream traffic generation and prioritization techniques that had not played a large role in traditional point-to-point services. Furthermore, special attention must be paid to the key QoS metrics and matched against the Service Level Agreement. By adopting test procedures focused on differentiated services testing, network providers can be confident that they are satisfying the needs of their customers today and building a reliable source of revenue for the future.