Case Study: SIP Business Voice

Small to medium-size business (SMB) phone service is a major focus for cable operators this year. You can’t turn too many pages of any industry publication without seeing mention of the great opportunity this represents to cable operators, or a vendor advertisement for product that can help operators go after it. Virtually every major multiple system operator (MSO) has put forth efforts to go after the SMB phone market, some putting impressive points on the board. For example, Cox Business Services announced its 200,000 customer milestone this year – including customers above and beyond the SMB.

But while the opportunity is clear, the right approach is not quite as straightforward. In particular, the small to medium-size cable operator has its own set of challenges.

With vendors working hard to scale their solutions up to meet the performance and scalability needs of large cable operators, less focus has been put on scaling solutions down to meet a reasonable minimum cost of entry for the smaller operator. Challenges Small-to-medium size cable operators must provide the same product to their customers as larger cable operators. They also have the same service quality and reliability requirements for those products, but do not have the same potential customer base. The challenge for these operators lies in right-sizing technology solutions. Oftentimes, the cost of third-party licensing alone can make some parts of an overall solution difficult to justify. Licensing a third-party database, for example, can move the needle enough to make a solution cost prohibitive. This challenges small-to-medium size operators to search for the right mix of technology solutions in order to provide the services their customers need.

Quincy, MA-based Atlantic Broadband recently faced and overcame these challenges. With a strong existing base of business high-speed Internet customers and a few successful years of residential phone service experience, Atlantic Broadband was well-positioned to go after the SMB phone opportunity. Atlantic Broadband found itself moving toward a combination of hosted solutions, off-the-shelf products, and custom development in order to meets its market needs and deployment schedules. SIP Given the evolution of industry standards, Atlantic Broadband decided that a session initiation protocol (SIP)-based platform would be the foundation of its design. Realizing we were out in front of the industry standards, Atlantic Broadband found the maturity of the feature set in a SIP-based offering compelling enough to move forward.

There are a number of functional areas that comprise a complete solution that provides the right set of product features in a reliable, easily repeatable manner for service fulfillment.

• Softswitch and voice application platform
• Billing and online portal
• Order processing and customer relationship management (CRM)
• Customer premises equipment (CPE)
• Provisioning and network services
• Capacity management

Atlantic Broadband preferred to own and manage all aspects of providing service to its customers. This would provide the control needed to quickly respond to its customers’ needs for features and enhancements.

In each functional area making up the infrastructure for offering service, the decision to select a hosted vs. internally managed solution was considered. Furthermore, within the internally managed components, build vs. buy needed to be considered. Softswitch To provide the reliability its customers have come to expect, Atlantic Broadband considered a softswitch and voice application solution that was internally redundant as a must to provide the service with the high availability desired. It was important that if something failed, the system or application would continue working uninterrupted. In the event of a larger failure, a solution that was geographically redundant would allow for continued operation.

Creating a solution with an off-the-shelf product in a manner consistent with redundancy needs quickly modeled out to be an expensive proposition. Some operators have chosen the path of custom-developed platforms based upon Asterisk, an OpenSource private branch exchange (PBX) and telephony platform. While this approach was interesting, it represented some support risk and a longer development phase than Atlantic Broadband’s plans allowed. In either case, building the expertise on the new infrastructure added to the lead time before a solution would be in place.

The decision was to partner with a hosted solutions provider that had a proven platform and a track record of performance with it. Figure 1 shows the high-level network design of the hosted softswitch and voice application implementation showing a sample call-flow. A hybrid approach was taken with the access session border controller (SBC). Atlantic Broadband wanted this system onsite with full ownership. Since the SBC should be managed in concert with the entire core telephony switching infrastructure, we decided to share its management with its hosted softswitch partner. Billing and customer portal Atlantic Broadband had a proven billing platform for its residential services and business data customers. An SMB phone service brings with it more advanced billing requirements such as departmental coding and online rendering.

While Atlantic Broadband’s existing billing platform had options to support such advanced requirements, the process of creating a relationship between the hosted softswitch and voice application partner and the existing billing environment represented a schedule impact that did not fit with Atlantic Broadband’s time-to-market needs. This schedule impact would be further impacted by the desire to support both feature control and billing through a single online Web portal with a single sign-on.

The decision was to use the existing billing platform for monthly recurring as well as aggregate usage charges and augment its capabilities with a hosted billing application with an existing relationship with the new hosted softswitch and voice application partner. Advanced billing features are handled by this hosted application, and aggregate billing records based upon usage are fed back into Atlantic Broadband’s existing billing platform for a unified, single billing statement across product lines. The existing relationship between both hosted solutions providers enabled the online portal billing and feature control integration to be done quickly with a single sign-on for the customer. Furthermore, this hosted billing application had underlying workflow capabilities that could be further leveraged for business process management if desired. Order processing The order process for SMB phone service is more complex than that of residential services and commercial high-speed Internet. The order process for SMB phone consists of multiple trading partner orders (porting, directory listing, etc.), manual feature configuration steps (hunt groups, auto-attendant, etc.), as well as managing documents such as network diagrams and site survey results. With this combination of synchronous and asynchronous programmatic order processing tasks, along with manual tasks, a workflow management system is important to creating a reliable and easily repeatable fulfillment process. A business process management system with defined workflows that perform order status, manual task coordination, exception handling, and updates to network-layer provisioning systems was desired.

A number of off-the-shelf software applications were available that could elegantly manage process across the business and network layer. Many of these solutions had a full suite of supported integrations with the who’s who of operations support system/business support system (OSS/BSS) and network applications. Many also had a strong set of customer Web portal, product catalog architectures, and extensibility options through software development kits (SDKs).

Atlantic Broadband needed only a small segment of these off-the-shelf solutions’ overall value propositions, and segmenting out these specific capabilities conflicted with the pricing model for most vendors.

Atlantic Broadband started off with a hybrid approach, which evolved to a single hosted application managing the order process. It was divided into two phases.

Phase 1 employed a level of manual process management, as well as some scripting, to create basic asset inventory management tools and Web-based wizards that walked the order processing staff through the process. This combination of manual processes and scripting helped Atlantic Broadband really learn what was involved in the end-to-end order process, where it can break, and eventually helped model a desired automated solution.

Phase 2 built upon the hosted billing software application used in the billing and online portal components; an automated solution was created using the business process management features available in this application. This software, while initially used only for customer online portal and to generate aggregate billing records, had strong workflow capabilities. In phase 2, this software was fully leveraged, becoming the single point of entry for automating the order process designed from lessons learned in phase 1.

This complicated SMB phone order process needed to be an easy to follow on-screen dialog with as few points of entry as possible to reduce the swivel chair effect. Figure 2 is a linear representation of the five major order processing steps. This representation is rendered along the top of the order processing screens as the order moves through each step showing progress along the way. Atlantic Broadband implemented a system and a set of system integrations in phase 2 that leveraged its new hosted billing applications workflow and order entry capabilities to manage the entire customer setup process. Figure 3 shows the initial customer setup screen. Downstream workflow is driven as information is entered along each step of the process. For example, ordering a hunt group drives the downstream step of document collection and manual configuration for that service. Figure 4 shows the documents required for services ordered, all driven by workflow. Collecting customer information, submitting orders for phone lines, provisioning phone lines on the network, coordinating the manual tasks such as configuring auto-attendant, and tracking of required documents is all handled at a single order processing system. CPE In Atlantic Broadband’s residential network-based control signaling (NCS) phone service, DOCSIS embedded multimedia terminal adapters (EMTAs) are installed at the customer premises. The desire was to keep on this path of embedded devices in its SIP phone service launch; however, the schedule for firmware loads supporting SIP for these devices did not with accommodate planned timelines. Since Atlantic Broadband also expected subsequent phases of commercial phone to require any number of different CPE types as it moved beyond the SMB phone line replacement, it decided to pursue both an embedded and non-embedded strategy.

Out of the gate, the first SIP service offering used an analog telephone adapter (ATA) behind a DOCSIS 1.1 enabled cable modem, leveraging an independent DOCSIS 1.1 service flow as it evolved its quality of service (QOS) strategy. The maturity of the SIP support for the various ATAs made the case as a starting point, as the EMTAs enhanced their support.

Beyond ATAs and EMTAs, Atlantic Broadband anticipates the need to support other CPE such as Internet protocol (IP) phones, IP PBX, and integrated access devices (IADs) as it moved further into and beyond the SMB market. Provisioning and network Atlantic Broadband already had a solid and reliable IP and DOCSIS network services and provisioning platform in place to support existing services. Supporting its NCS residential phone service, this platform was built with support for a provisioning flow for EMTAs. The desire was to leverage this platform to support its entrance into SIP and SMB phone service.

Since Atlantic Broadband was starting with non-embedded CPE, the provisioning and network services platform would need to be augmented to serve up the necessary configuration properties required to enable services. To augment the existing provisioning and network services platform, strong high-level provisioning and network services criteria were defined.

• Business to network abstraction layer: Any solution needed to allow the business layer to communicate service entitlements without knowledge of the underlying network layer, topology, or CPE specific configurations.
• Centralized administration: Any solution needed to allow central administration and provide a single point of integration for the business layer and system management.
• Extensibility: Standards are still evolving because there is no current standard for configuration of non-embedded CPE devices. They all reach for configurations in their own way and their own formats. This system needed to allow for custom provisioning templates that define how and what is provided for configuration based upon the given CPE make and model.
• High availability: Each application needs awareness of availability and redundancy at the hardware and software level.
• Geographic redundancy: In the event of a major failure, this system needs to have a geographically redundant set of servers and applications that are replicated from the original source and can take over when necessary.

A number of off-the-shelf software applications can support the provisioning of non-embedded SIP ATAs and other SIP devices. Many of the vendors evaluated were able to meet most, if not all, of these criteria. They all had elegant solutions.

Some had third-party software requirements that made them cost prohibitive. Others that did not require third-party software only supported high availability and redundancy with their higher-end configurations – also cost-prohibitive.

Atlantic Broadband decided to create a custom-built solution based upon the selection criteria it created when evaluating off-the-shelf products. These criteria evolved into a strong set of requirements that was the foundation for its design.

Deployed in a geographically redundant configuration across two data centers, the custom SIP provisioning application provides high-availability SIP network services. The SIP administration application is divided into two logical and physical components that support both administration and configuration services.

The SIP administration server (Figure 5) provides management and configuration to network administrators, as well as a northbound Web services application programming interface (API) for service entitlements from the business layer. Through the SIP administration server, network engineering can control system processes and configure changes to add new network deployments as well as new CPE device types. The northbound API receives entitlement messages from order processing based on services ordered and workflow for all changes in the lifecycle of a customer that require network configuration. It provides a layer of abstraction for the business layer, translating service entitlements into the device-specific configurations required to enable service. The SIP configuration server (shown in Figure 5) provides SIP device configuration based on the service ordered as well as the configuration format and method required for a given CPE make and model. These CPE configurations are offered to the device using trivial file transfer protocol (TFTP) and hypertext transfer protocol over secure socket layer (HTTPS). Each customer configuration is stored normalized in the database and provided in the device configuration dynamically at the time of the CPE request. The SIP configuration server also provides CPE logging services that enable Atlantic Broadband network engineers to quickly identify and troubleshoot unexpected device or network conditions.

Replication of critical configuration data is performed between the two SIP administration servers in two separate geographic locations. SIP configuration servers are then replicated from their corresponding local SIP administration servers. This replication setup provides for continued operation under outage conditions. Capacity management With voice applications traversing the same network as the existing data network, Atlantic Broadband found it prudent to change its capacity management practices. Capacity was broken down into three conceptual segments, Internet drains, backbone, and HFC access.

The first step was to begin monthly reviews of each segment. Internet drains and backbone were combined into one report showing actual usage for the month, broken down to an average usage per cable modem or EMTA. This information combined with budgeted growth information is used to create predictions of capacity growth for the remaining year. Predictions of capacity growth are not only calculated based on current averages per device, but also on the assumption that the average user will increase consumption by 50 percent over the next 12 months. Atlantic Broadband found it important to show baseline growth and to define worst-case scenario growth.

For the HFC access network, Atlantic Broadband instituted monthly meetings with among network engineering, technical operations and system HFC plant managers. With voice, video and data services all being delivered across the same HFC network, Atlantic Broadband found that, more than ever, capacity management on the HFC plant is a way of life. Monthly meetings with all groups involved in capacity planning, engineering and re-fencing of the HFC plant has proven valuable in keeping good communications among all groups.

Old rules of thumb for capacity on the network were no longer valuable as a technique for engineering capacity for the HFC. Atlantic Broadband is evolving away from planning HFC capacity solely on homes passed or cable modem counts. Instead, three basic sets of data combine to form a key performance indicator: peak average utilization per interface of the cable modem termination system (CMTS); max calls during peak busy hour per interface of the CMTS; and CPE counts per interface of the CMTS, the last carrying the least weight.

Using defined ranges for each metric, the interface of the CMTS is flagged green, yellow or red. During the monthly meetings, yellow interfaces are reviewed and assigned a budgeted cost for resolution. Red interfaces typically have already been assigned a cost from a previous meeting. A project is created for them and is scheduled.

Current metric thresholds are as follows.

Utilization for HFC, backbone and Internet drains:
    • Green ≤40 percent peak average
    • Yellow 41 percent-59 percent peak average
    • Red ≥60 percent peak average
Max calls during peak busy hour
    Upstream interfaces at QPSK
        • ≤24 Green
        • 25-30 Yellow
        • ≥31 Red
    Upstream interfaces at QAM
        • ≤44 Green
        • 45-55 Yellow
        • ≥55 Red
CPE counts for CMTS interfaces as a preference
    • Downstream interfaces 600-800
    • Upstream interfaces under 200

With convergence finally taking place across the entire network, Atlantic Broadband found a need to have a scalable QoS strategy, with its different data services broken into logical layer 3 networks that included also breaking down the routing protocols into separate routing domains. CPE devices are used to tag packets with the correct markings for the service; these markings are used throughout the backbone to place the packets into the correct queuing mechanism as they travel across the various switches and routers of the backbone. In most cases, low latency queuing techniques are used across the backbone.

SBCs were deployed into Atlantic’s network, allowing for a defined anchor point of voice traffic. This allowed QoS policies not otherwise feasible in a fully hosted configuration, including the ability to have inbound packets to the customer marked correctly for QoS inside the network. Bringing it all together Atlantic Broadband’s approach pulled together a mix of off-the-shelf, hosted, and custom built solutions. (See Figure 6.) While Atlantic started off with a strong preference to own and manage all components of a service offering, a combination of existing infrastructure, hosted solutions and custom-built applications emerged as the appropriate path for a quick entrance into the SMB phone market. Leveraging existing platforms, taking advantage of existing technology partnerships, getting creative with custom software, and realigning its approach to capacity management, Atlantic Broadband quickly put in place a solid platform to support its SMB phone opportunity. Walter Hubley is vice president, information technology and program management, for Atlantic Broadband. Reach him at [email protected]. Shane Schilling is director of Network Operations for Atlantic Broadband. Reach him at [email protected]. Sidebar: ABB at aGlance Atlantic Broadband, headquartered in Quincy, MA, is the nation’s 15th largest cable operator. It provides video, voice and data services to 286,000 customers across seven states on the east coast of the United States. It has residential and business customers in service areas ranging from metropolitan markets like Miami Beach, FL, to the more rural and mountainous areas of western Pennsylvania and West Virginia. Organized in four separate operating regions, Atlantic Broadband is a mix of larger interconnected markets and smaller non-contiguous standalone sites; this market composition adds to its challenge of right-sizing technology solutions.