Constructing Cable IP Backbone
Operators face a range of legacy and new transport options in fashioning an IP backbone that can carry more traffic and deliver greater returns.
Cable operators considering the future of their backbone networks may feel like homeowners watching a home improvement television show.
The host invariably has the right tool for each job, works with pristine sheet rock and knows exactly what to do. Real-world homeowners have less ideal tools and material; they know how things should look, but are unsure of exactly how to get there.
Cable operators are like these homeowners. They are working with legacy protocols in systems that have taken on unique characteristics. Engineers and planners are still learning about new applications and, therefore, are unsure of how best to proceed.
But operators can’t chuck it all and move into a rental. They have to deal with the great divergence in the type of backbone infrastructure that exists where they live, even in single, newly consolidated systems.
Only one certainty
At this point, one thing is certain and another thing is probable. The certainty is that the future—based on pressure from the Data Over Cable Service Interface Specification (DOCSIS) distribution plant on one side and the long-haul network on the other—will be an Internet protocol (IP) one. What’s probable is that some operators will opt for an emerging standard, resilient packet ring (RPR). "The debate about the RPR piece is when, and at what cost," says Stephen Garrison, senior director of marketing at Riverstone Networks.
What is much more uncertain is how the industry will get from here to there. While the unprecedented scale of the new Comcast makes it a unique case, the choices the company faces suggest the deep differences in what the industry’s backbone looks like today and the difficulty in deploying the right architecture.
Comcast is running three different networks, says CTO David Fellows. These are the Road Runner holdings that Comcast got when it acquired Media One, AT&T’s former Excite@Home systems and the Excite@Home systems that Comcast already held.
Fellows says that the Excite@Home systems from both MSOs are now running point-to-point gigabit Ethernet (GigE) aggregation networks, but that they are not in rings. The former Road Runner systems are using a pre-standard proprietary Cisco version of RPR called dynamic packet transport (DPT). Fellows says that Cisco is doing a satisfactory job of morphing DPT into RPR.
This year, Fellows says, the elements of the company running GigE will be physically rearranged into rings. This will be a precursor to a switch next year to RPR. Likewise, the element of the network running DPT will move up to RPR when the standard is set and Cisco’s work is done.
The other piece of the puzzle is video. Comcast is deploying GigE links to support its burgeoning video-on-demand (VOD) business. VOD bandwidth demand dwarfs high-speed data. "My challenge next year is to start taking video gigabit Ethernet and merging it with high high-speed data gigabit Ethernet," Fellows says.
Comcast endorses RPR
RPR is a perfect solution for the marriage of video and data, which currently run side-by-side on the same transport technology, Fellows says. To offer reliable VOD on a point-to-point link, Comcast must deploy top-of-the-line servers, and any failure takes the service down totally. However, if a topology such as RPR is implemented, the pressure is taken off each individual server. If one fails, the service can be redistributed.
Comcast expects to undertake the merger of VOD and data on RPR over Ethernet next year. "That means the video takes on carrier-class elements of the data network, because the video service becomes much more reliable," he says.
Though other operators don’t have as great a task as Fellows and his staff, the variables collectively are as great across the industry. Key issues include the extent of an operator’s physical rings; the location and upgrade status of cable modem termination systems (CMTSs) and VOD servers; the operator’s service mix going forward; and the type of underlying transport protocols currently in use.
Add to those variables one that has little to do with technology but, in the final analysis, may be the most important of all. Are operators willing to invest more today for what may in the long term be a simpler and less expensive system to run? Or are capital expenditure issues so critical that investments today must be kept to a minimum, though it may end up costing more to run these systems and add new services in the future?
"The question is do they pay low CapEx now, which means they will pay more ongoing OpEx, or do they invest more now to reduce ongoing operational expenses," says Chris Bridge, Juniper Networks’ product marketing manager for cable solutions.
Transport modes
The cable industry has adopted many transport protocols over the past decade. Originally, networks were disparate islands of connectivity. The arrival of high-speed data and the advancing state of digital technology encouraged operators to interconnect systems, the main idea being to link a master headend with smaller facilities in the field.
The rings linking the master and secondary headends today are primarily synchronous optical network (SONET) and asynchronous transfer mode (ATM). The widespread deployment of the C-COR.net’s proprietary DV6000 family is another factor, as is the bevy of protocols used within these networks for distribution and collection of high-speed data and video, each with its advantages and disadvantages.
All of these protocols are represented in the cable industry today. How they will be used in the future is beginning to come into focus. It is now thought that many operators will move toward a hybrid protocol—RPR—which combines the advantages of SONET and Ethernet.
Indeed, RPR—an offshoot of resilient packet transport (RPT), which is geared toward bringing time division multiplexing (TDM) traffic under its umbrella—in a way goes beyond SONET. While SONET guarantees restoration times by keeping half its capacity in reserve, RPR enables full use of all bandwidth. Each packet is assigned a primary and secondary router. If the network has a problem (euphemistically called "protection mode" by engineers) packets report to either their primary or secondary router for instructions on getting around the problem. Thus, keeping half the capacity in reserve for an emergency isn’t necessary.
While the technology is elegant, its implementation will no doubt depend on how the network in place can be adopted to its use and whether the operator feels that the emerging business model strongly suggests that the extra expense will be justified.
Whether or not RPR and RPT are the end game, the concern now is how operators ought to evolve their infrastructures in anticipation of future services. The near-term goal is to get video and data to share the same physical infrastructure.
An operator, for instance, could first consolidate VOD servers and CMTS gear in the same place. That would enable running the two sets of data separately on the same network. Eventually—using RPR—all applications could intermingle in the same bit stream. "The next step is the potential migration of the two edges: MPEG and the DOCSIS boxes," says Enzo Signore, senior director of marketing for Cisco’s CMTS line.
The fast evolution of the CMTS, coupled with the merging of the transport network may point to a device that handles all applications. "No doubt, video is the third piece in the CMTS," says Tim Doiron, the director of marketing for Arris Interactive. "It will be voice, video and data—all over IP, all over DOCSIS, all over fiber."
QoS options
That’s farther in the future. In the nearer term, an element that operators need to think through is how they will maintain quality of service (QoS) once the transport issues are settled. Four choices appear possible: the resource reservation protocol (RSVP), differentiated services (DiffServe), type of service (ToS) and multiprotocol label switching- traffic engineering (MPLS-TE). "I think the first step is understanding exactly what services you want to deploy," says Gregory Goetz, a senior manager of business development for ADC.
DOCSIS 1.x creates different classes of service between the end user and CMTS. ToS, DiffServ and MPLS-TE maintain a desired QoS into the core of the network.
In general, operators that feel that they will need many levels of QoS likely will opt for DiffServ or MPLS-TE. The latter is essentially a way to manage bandwidth efficiently and with a degree of planning. The second approach, setting ToS bits, is a more limited way of indicating what level of service the arriving packet should receive. ToS bits are actually a part of DiffServ.
"Basically ToS and DiffServ aim at the fundamental problems, which are latency and management of congestion in the network," says Benoit Legault, the director of the business development group at ADC.
None of these issues will be settled anytime soon. Operators, however, need to begin considering their options. "This is coming to a head now because of the extensive opportunity for (traditional) data overlaid with VoIP and other services and the video realm," says Andy Paff, CTO of Broadband Services. All are new revenue generators.
As always, it will be up to the engineers to ensure that the network can exploit these looming opportunities.
Carl Weinschenk is a contributing editor to Communications Technology. Email him at [email protected].
Weighing Transport Options
* GigE. Advantages:It is optimized for carriage of packets. As a ubiquitous enterprise protocol, it is very cheap—not a trivial point these days. It is also enabling an arguably more efficient (and centralized) VOD distribution infrastructure.
Disadvantages:It has nowhere near the resiliency necessary for use in the metropolitan area network (MAN). There is little quality of service, which is necessary for many of the services operators want to add. "It doesn’t have robust QoS built into it," says Irv Duling, the chief scientist for Optinel. "[You] get around QoS by over-provisioning. But it doesn’t scale particularly well."
* DWDM. Advantage: It provides a terrific amount of bandwidth.
Disadvantages:Each application requires its own wavelength. This makes it impossible to combine applications and perform other internetworking tasks. Because many applications require far less than a full wavelength, DWDM is often wasteful. "DWDM is in essence the poor man’s convergence," says Jay Rolls, the vice president of data engineering for Cox. "The true utopian convergence is when you are hauling all the bits around all mixed together. The problem [with DWDM] is that it’s pretty expensive. If it were cheap, we’d be done."
* SONET. Advantage: As a telecom protocol, SONET reroutes traffic in 50 milliseconds or less from the time the network goes down. It is widely deployed, and is relatively cheap.
Disadvantages: It is optimized for time division multiplexing (TDM) traffic. Extra work—which means expense and complexity—must be undertaken to make it appropriate for carrying IP packets. Because SONET works in the electrical domain, conversion to and from optical are necessary.
* ATM. Advantage:It is a widely deployed standard that promised to intermix voice, video and data traffic.
Disadvantages: ATM gear is expensive. The protocol is based on fixed 53-byte cells. This means that everything—even IP packets that can run upwards of 1,500 bytes—have to be sliced and diced to fit in. That adds complexity and expense.
BOTTOMLINE
Backbone Building
A single Internet protocol (IP) backbone may be what cable operators need to carry increasingly heavy traffic loads across their plants. The emerging resilient packet ring (RPR) standard could be the enabling technology. While considering new and legacy technologies, operators must also consider the following questions:
* How extensive are your physical rings?
* Is your CMTS gear deployed in headends or hubs?
* Are they upgraded?
* Where are your VOD servers?
* Have you defined your service-mix going forward?
* What underlying transport protocols do you use?
