Broadband: One for the X-Files

I recently heard from a regular reader about an interesting and somewhat unusual problem. Here’s the background.

A defective downstream laser transmitter that serves three nodes was found to be causing degraded bit error rate (BER) in the downstream digitally modulated signals. Two different quadrature amplitude modulation (QAM) analyzers measured post-forward error correction (FEC) BER around 5.9E-05 on the QAM channel used for voice service. Ideally, there should be no downstream bit errors, but for reference, the DOCSIS spec for downstream post-FEC BER is 1.0E-08 or less at the cable modem input. Signal levels were fine, and equalized modulation error ratio (MER) was ~35 dB. The two QAM analyzers’ constellation displays had no visible impairments. Here’s where things get interesting. The degraded downstream post-FEC BER was causing upstream voice quality problems. That is, voice quality from the hub site to the embedded multimedia terminal adapter (EMTA, a cable modem with a built-in MTA) sounded fine, but voice quality from the EMTAs back to the hub was affected. The upstream RF spectrum from the three nodes checked out OK. When the defective downstream laser was replaced, the upstream voice quality problem cleared up. This was later replicated on the test bench, using the defective downstream laser in the test lashup. How it’s supposed to work So, what might be happening here? To get an idea, we need to start with a look at the communications between a cable modem termination system (CMTS) and modems in the outside plant. We’ll assume that the modems (or EMTAs) have gone through the ranging and registration process and are already online. When a cable modem has data to transmit to the CMTS, the two devices follow a media access control (MAC) protocol. DOCSIS 1.0 and 1.1 cable modem upstream transmission uses time division multiple access (TDMA) to allow multiple modems to share the same channel. Each modem is assigned a specific time during which it can transmit. The upstream channel is divided into 6.25 microsecond (µsec) units of time called ticks. There are 160,000 ticks per second, since 1 second divided by 6.25 µsec equals 160,000. Ticks are the building blocks of larger units of time called mini-slots. The number of ticks per mini-slot is configured at the CMTS. Mini-slot size and burst profiles are part of the upstream channel descriptor (UCD) message that is periodically broadcast to cable modems. The CMTS transmits MAC management messages called allocation MAPs—think roadmap—to the modems. MAPs describe the usage of the upstream mini-slots. In a nutshell, a cable modem transmits a bandwidth request to the CMTS, asking for a certain number of mini-slots in which to transmit its payload. The CMTS sends a MAP that tells the modem which mini-slots to use for its payload, and the modem sends its data in the assigned mini-slots. This is known as a request/grant cycle. The modem requests mini-slots for upstream transmission, and the CMTS grants them. The request/grant cycle goes something like this. Modem: "Yo, CMTS, I have a 160-byte packet plus some overhead that’s ready to go, and I need 13 mini-slots." CMTS: "Yo, modem. Transmit your data in the 13 mini-slots available starting at 3:15 p.m." The modem transmits its data at 3:15 p.m. Modem: "Yo, CMTS. I have another 160-byte packet plus overhead that’s ready to go, and I need 13 mini-slots for it." CMTS: "Yo, modem. Transmit your data in the 13 mini-slots available starting at 3:17 p.m." The modem transmits its payload at 3:17 p.m. Modem: "Yo, CMTS. I need another 13 mini-slots for still another packet and its overhead." CMTS: "Yo, modem. Transmit your data in the 13 mini-slots available starting at 3:19 p.m." And so on. I’ve taken some editorial liberties describing the request/grant cycle, but you get the idea. If you’re interested in a more detailed description of the MAC protocol, see Section 7 in the DOCSIS 1.1 Radio Frequency Interface Specification (ANSI/SCTE 23-1 2005, available at www.scte.org/documents/standards/approved/ANSI_SCTE%2023-1%202005.pdf). In the case of voice over Internet protocol (VoIP), DOCSIS 1.1 and later technology include something called unsolicited grant service (UGS). UGS supports real-time service flows that generate fixed-size data packets on a periodic basis—for instance, upstream voice packets. UGS minimizes the overhead and latency associated with the normal request/grant cycle by assigning a bunch of fixed-size mini-slots for a given modem’s recurring voice packets, so the modem doesn’t have to ask for mini-slots for each upstream transmission. UGS grants are included in the CMTS’s MAP messages. Where problems begin If the downstream MAP message is lost, the EMTA won’t see its UGS grant opportunity. If the EMTA is unable to get additional upstream mini-slots, and assuming enough missing grant opportunities, this could lead to its queue filling up. Depending on the rate of missing grants, the queue may overflow, and packets would be dropped. This would be the scenario if what is called the queue indicator (QI) bit for upstream transmission policy is disabled. If UGS flow policy has the QI bit enabled, it essentially allows the EMTA to request extra grants in addition to the normal UGS grants at periodic intervals. In this case, the EMTA may be able to slowly catch up, but the result is voice packet jitter. Excessive jitter also could lead to dropped packets. While there may be something else going on here, missing MAPs caused by the crummy downstream BER is more than likely what caused the upstream voice quality problems. Common assumptions When upstream voice quality isn’t up to par, we tend to think the problem must be something in the upstream: ingress, impulse noise, common path distortion (CPD), linear impairments, maybe even laser clipping. The example discussed here shows that it’s possible for some upstream voice quality issues to be caused by a downstream problem! As I was penning this month’s column, I heard from my contact at the system again. He said they found a second problematic downstream laser that was causing the same type of upstream voice quality issues as before. New downstream laser, upstream problem fixed. He also noted that the availability of a QAM analyzer made troubleshooting the problem a whole lot easier. Other possibilities This also suggests that other downstream gremlins such as laser clipping caused by improper headend RF levels, or maybe something as simple as sweep transmitter interference—basically anything that can degrade downstream post-FEC BER at the cable modem or EMTA input—may be the source of some of those mysterious upstream dropped packet problems you’ve been pulling your hair out trying to fix. Ron Hranac is a technical leader, Broadband Network Engineering, for Cisco Systems and senior technology editor for Communications Technology. Reach him at [email protected].