YouTube videos will soon get captions

By Hiawatha Bray
Globe Staff / November 20, 2009

E-mail this article

Invalid E-mail address
Invalid E-mail address

Sending your article

Your article has been sent.

  • E-mail|
  • Print|
  • Reprints|
  • |
Text size +

Soon you’ll be able to understand thousands of Internet videos even if the sound is turned down. With an assist from Boston public TV station WGBH, Google Inc. has developed technology that will automatically add on-screen captions to video streams on its popular YouTube website.

Ken Harrenstien, Google’s lead software engineer on the project, said the new technology will make many YouTube videos accessible to hearing-impaired people for the first time. “Many people have wanted this to happen for many years,’’ said Harrenstien, who is deaf.

Harrenstien said the captioning service would be made available “sometime this week.’’ It uses speech recognition software that translates spoken words into written text. Google already uses this technology in Google Voice, the company’s new telephone service. Google Voice subscribers can read their voice mails, which are translated into text and displayed on a Web page. The YouTube captioning system uses the same technology to “listen’’ to the soundtrack of a video, translate the voices to text, and display the text as subtitles as the video plays. In addition, the text can be fed through Google’s language translation software, so the subtitles can be read in 51 languages.

Some YouTube video makers already create their own transcripts, but they must include special timing codes to synchronize the subtitles with the person on screen. Google will now automate this process. A video maker simply uploads a written transcript, without timing codes, and Google’s software will display each portion of text at the right time.

WGBH, the first station in the United States to add captions for deaf viewers, assisted in the Google transcription project, producing manual transcripts for 50 hours of video produced by Google for the company’s YouTube “channel,’’ a site where its videos are collected. Google used the WGBH-produced transcripts to train its software and improve its speech recognition capabilities.

“We’ve got the reputation for really understanding the technology and having really high quality,’’ said Larry Goldberg, director of WGBH’s media accessibility program.

The caption system will be introduced as a beta test, an in-progress version that represents Google’s admission that all the bugs haven’t yet been worked out. “Speech recognition is a long way from perfect,’’ said Harrenstien.

At first, the captions will only be available on videos produced by a number of major publications, including National Geographic, and major universities including the Massachusetts Institute of Technology.

MIT’s OpenCourseWare program publishes materials from all MIT courses on the Internet, including its own YouTube channel with complete videos of 30 courses. OpenCourseWare spokesman Steve Carson said MIT already publishes manually-typed transcripts of these videos. He’s eager to see how well the new Google system performs, but he predicted that it won’t be good enough to replace traditional transcription. “At this point, I just don’t think the technology is there yet,’’ Carson said.

Google hopes to add instant captions to virtually all YouTube video content as the technology improves. Google vice president Vint Cerf said the company must also find out whether it needs the creator’s permission to generate a transcript of a video. “We have to be a little careful about doing all this automatically, because we don’t own the rights,’’ Cerf said.

Automatic transcription could make video information more accessible to everybody, not just the hearing-impaired. The text file generated by the system could be added to the Google search engine, making it easier to look up videos on any topic. “If there is specific oral content of interest,’’ said Cerf, “we can find it.’’

Hiawatha Bray can be reached at

Correction: Because of incorrect information supplied to the Globe, a story in yesterday’s Business section about Google Inc.’s plan to add text captions to YouTube videos mischaracterized the company’s legal right to do so. Google said it has the right to add captions.