Doing more with less
At this point in time, the argument has been presented many times regarding the value of return path monitoring to provide quality service. In this traditional framework, it has proven its worth as an effective tool. However, operators have the unique opportunity to create a new paradigm for return path monitoring that will impact critical areas for the deployment and management of advanced services. Return path monitoring can and should be used not only for monitoring, but also for efficiently managing the workforce to achieve gains in advanced service revenue goals.
It is inevitable that the burden placed on cable operators for installation, service and monitoring of advanced services will continue to increase in the future. As operators deploy digital services, they are asking their workforce to do more and more. With the aggressive growth rates set forth by operators, the gauntlet has been thrown in the race not only to acquire but also maintain customers. Questions arise regarding the tools available to assist the workforce with installing and servicing advanced services as well as the qualifications of the workforce. The skilled labor pool is not nearly large enough to support these growth targets.
Financial drivers for the workforce include experienced and knowledgeable labor, multiple truck rolls, effective communication with the network operations center (NOC) and mean time to repair (MTTR). Each of these factors' costs can quickly skyrocket as an operator's network for advanced services expands—and the impact to customer satisfaction can be catastrophic.
We all must accept that the migration from reactive service calls and reliance on the greatest network alarm known in most industries—customer complaints—to a proactive environment with five-nine (99.999 percent up time) network reliability is expanding rapidly. Return path monitoring is a requirement that will help replace a repair-driven environment with a maintenance-driven environment. Most individuals agree the old model will not breed success in the current competitive landscape, and that new tools from vendors will eventually proliferate in the marketplace, but what action can be taken today to combat this mindset?
Relatively speaking, the answer is fairly simple: successful implementation of the new return path monitoring paradigm for management of one of the scarcest resources of all—trained personnel. This new paradigm is more feasible, scaleable and financially attractive compared with tripling or quadrupling the technology staff. By no means will return path monitoring eliminate the need for trained personnel. However, it will manage them more efficiently and allow for the expansion of advanced services at an exponential rate to the expansion of labor, resulting in the revenue gain we all seek.
Simply speaking, all operators would like to achieve the following value proposition: Maximize the profitability of advanced services by growing the subscriber base at a greater rate than the growth of labor required to install, manage, and maintain services. With the right return path monitoring philosophy and use of a system to its fullest potential, any operator can achieve this goal.
Now comes the hard part: Actual implementation and use of return path monitoring in this framework in order to achieve the aforementioned value proposition. To simplify this process, an operator should focus on achieving success in five general areas and utilize equipment that will facilitate each area.
Utilization of individual node performance history for planning, management, and problem assessment.
Operators must ensure that technicians' time is spent in the right area resolving or avoiding problems (maintenance). Intelligent performance history for nodes over time allows operators to determine what the problem looks like and assess the severity of the problem before dispatch (problem assessment), and properly prioritize problems (planning). The end result is that operators are able to dispatch appropriate personnel accordingly and accurately (management). Success in these three areas avoids the use of personnel on a search mission and allows operators to achieve the following:
- Instantaneous node isolation
- A drastic decrease in driving time and/or truck rolls
- Reduction in MTTR, resulting in a decrease in the number of customer service complaints.
How familiar is this scenario? A technician is paged to respond to a service call only to discover that the problem he thought he was looking for is erroneous. When he gets to the right node the original problem is gone. In the end this reconnaissance mission's costs include a truck roll, hourly wages, lost time that could have been spent on another call, and the difficult one to quantify—customer dissatisfaction.
Performance history of nodes over time will improve the deployment of service personnel with the right information to find and fix solutions proactively and more efficiently.
Intelligent alarming that uses statistical representation of trends rather than isolated events decreasing the probability of alarm storms.
With ever-more advanced services offered on cable networks, alarms are generated from a variety of network elements in the NOC. Management of true alarms and minimization of false alarms becomes more important on the return path.
The best way to avoid alarm storms is to filter out single bursts of ingress and use statistical results to capture actual ingress events. In addition, flexible measurement parameters and user programmable thresholds will minimize false alarms. For example, user programmable limits can track a rise in average noise over an extended time period. With alarms, the focus is often on the actual event that caused the alarm, but it is also important to look for developing problems. The reward is an increase in customer retention that translates to a loyal customer base to offer higher priced advanced services and increase average revenue per subscriber. Remember that increasing the availability of advanced services is only one key to increase average revenue per subscriber— the facilities and technology must be in place to support the reliable platform.
Advanced information, diagnosis and analysis of problems for field technicians before truck rolls.
Too many times, searches for "ghost" problems occur. Advanced diagnosis and information for technicians before a truck rolls represents tremendous gains in efficiency as well as a cost reduction. By the time an alarm occurs, the trouble ticket is generated, the technician is dispatched and he arrives at the site, the problem probably vanished (the nature of the nemesis known as ingress). Return path monitoring should be used to diagnose the problem before dispatch to determine what time of day the problem occurs. By using performance history of the node over time, NOC technicians can arm field technicians with the information necessary to know exactly what they are looking for in the field and at what time of day.
The 3D-performance history graph in Figure 2 indicates common path distortion. However, it only occurs between 6 p.m. and 6:30 p.m. and 10:15 p.m. to 11 p.m. From one quick glance at the graph, it is easy to determine when a technician should be sent to fix a specific problem.
"Applying consistent anomalies over time helps us manage data and limited resources more efficiently," notes Jim Haag, director HFC Networks at MediaOne. "Implementation of this model is necessary for success in full deployment of advanced services." With automated analysis performed by a return path monitoring system, the same number of technicians can resolve more trouble tickets in the same time frame.
Real-time communication with field technician while troubleshooting.
Once a problem is correctly identified, only half the battle is won. A technician, or in some cases multiple technicians, must be dispatched to fix a problem. Troubleshooting problems quickly and efficiently requires effective communication with individuals at the NOC. There are multiple examples, regardless of operator size, of enormous amounts of time spent via cell phones, e-mail, and radios trying to rectify a headend technician's view and a field technician's view. Confusion and frustration are often the result. Today, some operators may be able to operate in this environment, but what happens in the future when operators achieve subscriber goals for cable modems and telephony? The most probable result is inefficiency and a sacrifice in network performance/reliability.
After a technician knows exactly what to look for and at what time of day, the MTTR is further reduced by real-time communication between the headend and the field technician while fixing the problem.
Collaborative troubleshooting from different physical locations is the key to decrease the MTTR and an important factor in workforce management. This is achieved through communication of headend spectrum information to a single field unit as displayed in Figure 3. Dynamic field spectral information in comparison to the headend and vice versa will help a single technician isolate a problem more quickly, provide immediate feedback, and conclusive evidence that their direct actions resolved the problem.
"This model has decreased our field troubleshooting time by 65 percent, resulting in tremendous gains in efficiency," says Mark Button, technical operations manager, AT&T Cable Services. "In addition, we reduced the entire troubleshooting process from a two- to one-man job, leaving more time and resources for other issues."
The bottom line for an operator that implements this approach is a reduction of labor costs per incidence, reduction of field test equipment, and an improvement in customer satisfaction levels.
Applying these concepts to achieve gains in efficiency is transferable to dollar amounts specific for each operator. The new return path monitoring paradigm results in the same number of technicians completing more service calls per day. By following the steps in the worksheet in Figure 4, a total efficiency gain dollar amount is produced that assists in justifying the purchase of a return path monitoring system that facilitates these areas.
This financial worksheet is a conservative calculation of the total efficiency gain resulting from the numerous combinations possible and additional resources used or omitted on a service call. The total cost-per-call components are similar across operators, but each system maintains its own mix, depending on the troubleshooting model. The worksheet provides a framework to follow—it is each operator's responsibility to determine the combination to calculate the total efficiency gain. The critical factor that drives the total efficiency gain is that the average time per call can be decreased in the new paradigm.
The worksheet does not consider two additional factors representing major costs: truck rolls and customer churn. In this paradigm, some truck rolls are avoidable with NOC analysis, resulting in a cost reduction for truck rolls and technician labor. The goal is to empower a single technician with the information required to solve a problem quickly and efficiently hence eliminating or decreasing cost components in the total cost-per-call calculation. Additionally, customer churn is decreased with gains in efficiency, which directly affects the bottom line. Based on the total subscriber base, a single percentage decrease in customer churn rates multiplied by average revenue per customer represents a significant dollar amount. Do not forget the cost to re-acquire a customer once he switches providers. This cost can skyrocket to two or three times the average revenue per customer, quickly pushing the return on investment for new customers beyond an acceptable time frame.
Decrease customer service complaints through proactive maintenance.
It is true that more service calls will be generated when migrating from a repair environment to a maintenance environment. It is also true that more service calls will be generated with the mass deployment of advanced services. Remember the original hypothesis: the labor pool cannot grow as fast as the subscriber advanced service rate.
First, it would lead to a smaller profit margin and/or loss as displayed in Figure 5, which is the converse of the profitable scenario demonstrated in Figure 1. Second, the qualified workforce does not exist. To keep profit margins high and still support the reliability expected by advanced service subscribers, return path monitoring is the answer. When the day is done and the same numbers of technicians are resolving, not researching, more problems, the net result will be a higher profit margin.
Operators can enjoy a model comprised of a proactive means to improve customer satisfaction, improvements in QoS, and minimization of revenue losses from system downtime and customer satisfaction credits. As the cable industry begins to solidify a position in the world of high-speed data and telephony, it is important to establish and maintain a quality service reputation. This reputation helps reduce customer churn (increase retention) and improve customer acquisition. Growth can result from this two-pronged approach—acquire rapidly and aggressively, then retain over time. This paradigm proved to be highly profitable in the business telecommunications industry by such giants as AT&T and MCI. There is no logical reason not to apply this concept to the cable industry when competing for the same customers and services.
By taking return path monitoring out of the traditional framework of simply being a monitoring tool and realizing its full potential as a maintenance tool, operators are able to enjoy the benefits of the deployment of high revenue advanced services with existing resources. How important will return path monitoring be to the deployment of advanced services? Without it, it may just depend on who pays the most for trained personnel, and who can find them.
|About the author: Craig Morrall is product marketing manager for Wavetek Wandel Goltermann's PathTrak product line.|