Bringing AI to Closed Captioning

In the heat of the national anthem kneeling controversy, ESPN closed captioning displayed the words “Oer the Land of the Knee,” a mistake chalked up to human error. In 2014, WeatherNation was called out at the FCC by rival Weather Channel over some nonsense captioning that included phrases such as “Some orange flames as we head to the weekend. Some nymph restaurants the sun may make for some freaking travel conditions. Nitrates found in a match.”

Could AI help improve the accuracy of closed captioning?

IBM is hoping to have an impact, introducing its Watson Captioning service this week. The idea is to improve accuracy over time through machine learning capabilities. No one is promising that Watson is a babel fish with instant translations, but the goal is to get companies closer to compliance. “It may make mistakes at time because it’s machine learning. There are always new words it has to learn, but this gives you the tools and a system to make sure it’s seamless and automated,” said David Kulczar, senior product manager of Watson Video Analytics.  With AI, there is an increased accuracy and efficiency over time.

The Twenty First Century Communications and Video Accessibility Act has steadily increased closed captioning requirements, including for videos online, over the past few years. Compliance standards are often changing and vary in other countries. Watson’s aim is to provide flexible tools to help hit compliance quicker.

The service had its first outing at the US Open last year. While the IBM Watson Media suite was focused primarily on video highlighting and clipping, there was some closed captioning involved. “That was done almost as a prototype. What we’re doing here is productizing a lot of the technology that successfully did captioning for the US Open into a formal, scalable product that can be rolled out much more quickly without a lot heavy services,” Kulczar said.

While Watson Captioning is focused on media and entertainment to start, IBM is hoping to expand into other industries, including enterprise and retail. So many brands are creating video content these days, externally and internally. Kulczar sees the potential in everything from CEO webcasts to talks to investor groups. The next piece down the road could be using AI to take content and translate it into multiple languages.