Use-cases

Open captions driven by REST API calls

On Linux operating systems Wowza installation ships with OpenJDK that does not include fonts. On Linux you need to manually install fontmanager and at least one font openttf-dejavu to make open captions work.

To get the REST API up and running, you need to add the below HTTP provider to your VHost.xml file:

<HTTPProvider>
    <BaseClass>io.ztream.scribe.HttpProviderCaptionApi</BaseClass>
    <RequestFilters>captions*</RequestFilters>
    <AuthenticationMethod>none</AuthenticationMethod>
</HTTPProvider>

You need to add ModuleLiveOpenCaptions module to your Application.xml

<Modules>
    ...
    <Module>
        <Name>ModuleLiveOpenCaptions</Name>
        <Description>ModuleLiveOpenCaptions</Description>
        <Class>io.ztream.scribe.ModuleLiveOpenCaptions</Class>
    </Module>

</Modules>

You have to enable transcoder in the application via the Engine Manager UI or directly in the xml:

<Transcoder>
    <!-- To turn on transcoder set to: transcoder -->
    <LiveStreamTranscoder>transcoder</LiveStreamTranscoder>

You also need to set the raskenlund.804.license application property:

<Property>
    <Name>raskenlund.804.license</Name>
    <Value>W804-1038-xxxx-xxxx-xxxx</Value>
    <Type>String</Type>
</Property>

Assuming your application is called live and your incoming stream is called myStream, you can trigger a test caption using curl:

curl --location --request POST "http://localhost:1935/captions/v1/applications/live/streams/myStream/caption"
\ --header "Content-Type: application/json" --data-raw "{
\"captions\": [
    {
        \"textData\": \"Hello World, this is the captioning module\",
        \"language\": \"en\"
    }
],
\"delay\": 0
\}"

CEA608 captions driven by REST API calls

To get the REST API up and running, you need to add the below HTTP provider to your VHost.xml file:

<HTTPProvider>
    <BaseClass>io.ztream.scribe.HttpProviderCaptionApi</BaseClass>
    <RequestFilters>captions*</RequestFilters>
    <AuthenticationMethod>none</AuthenticationMethod>
</HTTPProvider>

Wowza Streaming Engine provides this functionality natively. You need to add Wowza’s built-in ModuleOnTextDataToCEA608 module to your Application.xml

<Module>
    <Name>ModuleOnTextDataToCEA608</Name>
    <Description>ModuleOnTextDataToCEA608</Description>
    <Class>com.wowza.wms.timedtext.live.module.ModuleOnTextDataToCEA608</Class>
</Module>

You need to configure the below timed text settings in the application:

<TimedText>
    <VODTimedTextProviders></VODTimedTextProviders>
    <!-- Properties for TimedText -->
    <Properties>
        <Property>
            <Name>captionLiveIngestType</Name>
            <Value>onTextData</Value>
            <Type>String</Type>
        </Property>
        <Property>
            <Name>captionLiveIngestLanguages</Name>
            <Value>eng,nld,nor,hun</Value>
            <Type>String</Type>
        </Property>
    </Properties>
</TimedText>

WebVTT captions driven by REST API calls

Please follow the setup for the CEA608 captions, except that you should not add ModuleOnTextDataToCEA608 to the application, but configure the below timed text property instead:

<TimedText>
    <VODTimedTextProviders></VODTimedTextProviders>
    <!-- Properties for TimedText -->
    <Properties>
        <Property>
            <Name>cupertinoLiveCaptionsUseWebVTT</Name>
            <Value>true</Value>
            <Type>Boolean</Type>
        </Property>

Speech To Text with a transcribe module

There are two transcribe modules, depending on which transcription service you want to use. You should choose only one.

To use AWS Transcribe, you need to add ModuleLiveTranscribeAWS module to your Application.xml. (In earlier versions, this module was called ModuleLiveTranscribe)

<Module>
    <Name>ModuleLiveTranscribeAWS</Name>
    <Description>ModuleLiveTranscribeAWS</Description>
    <Class>io.ztream.scribe.ModuleLiveTranscribeAWS</Class>
</Module>

Or to use Speechmatics, add ModuleLiveTranscribeSpeechmatics module to your Application.xml.

<Module>
    <Name>ModuleLiveTranscribeSpeechmatics</Name>
    <Description>ModuleLiveTranscribeSpeechmatics</Description>
    <Class>io.ztream.scribe.ModuleLiveTranscribeSpeechmatics</Class>
</Module>

And you must enable transcoder in the application via the Engine Manager UI or directly in the xml. If you do not need video transcoding, you can use the “audioonly” template so that video transcoding is disabled:

<Transcoder>
    <!-- To turn on transcoder set to: transcoder -->
    <LiveStreamTranscoder>transcoder</LiveStreamTranscoder>
    <Templates>audioonly.xml</Templates>

Please make sure that at least one Encode in your active transcoder profiles have the audio codec set to something other than PassThru. If no audio is transcoded, the plugin is not able to capture audio frames and send them to AWS, which will time out the transcoding session after 15 seconds without audio data.

Translating the transcript with Speechmatics

In addition to transcribing the source audio, ModuleLiveTranscribeSpeechmatics can have Speechmatics translate the transcript into one or more additional languages. This requires two properties to be set:

smTranslationLanguages must be set in the Application’s top-level Properties (the same place as smApiKey and transcribeLanguageCode, not inside <TimedText><Properties>), using the language codes Speechmatics itself expects (e.g. fi for Finnish):

<Properties>
    <Property>
        <Name>transcribeLanguageCode</Name>
        <Value>en-US</Value>
        <Type>String</Type>
    </Property>
    <Property>
        <Name>smTranslationLanguages</Name>
        <Value>fi</Value>
        <Type>String</Type>
    </Property>
</Properties>

The TimedText property captionLiveIngestLanguages must then list the corresponding Wowza 3 character language codes, in the same order: the source transcription language first, followed by each translation language in the order they appear in smTranslationLanguages:

<TimedText>
    <VODTimedTextProviders></VODTimedTextProviders>
    <!-- Properties for TimedText -->
    <Properties>
        <Property>
            <Name>captionLiveIngestLanguages</Name>
            <Value>eng,fin</Value>
            <Type>String</Type>
        </Property>
    </Properties>
</TimedText>

WebVTT tracks in Adaptive Live Streaming HLS playlists

When adding WebVTT captions to a live stream, by default Wowza publishes the WebVTT track in each rendition’s own playlist.m3u8 but the adaptive playlist that starts with ngrp: does not contain the text track.

To fix this problem the Scribe module contains a custom Module that can be configured in the Application.xml file and make it responsible for returning the patched playlists.

<Root version="1">
    <Application>
        <Modules>
            <Module>
                <Name>ModuleAMLSTWebVTT</Name>
                <Description>ModuleAMLSTWebVTT</Description>
                <Class>io.ztream.scribe.ModuleAMLSTWebVTT</Class>
            </Module>

With this module enabled, replace the ngrp prefix with amlst in the playlist URL, e.g. https://domain.tld/application/amlst:streamname_all/playlist.m3u8 to ensure that each rendition has a reference to the WebVTT caption track.

Note

This Module assumes that the StreamName parameter of the Stream Name Group in the transcoder template uses the underscore character to identify the group, e.g. ${SourceStreamName}_all or ${SourceStreamName}_test. Stream Name Groups that do not follow this naming convention will be ignored when retrieving the subtitles list for the group.

Here is an example StreamNameGroup section of a transcoder template

<StreamNameGroup>
  <Name>all</Name>
  <StreamName>${SourceStreamName}_all</StreamName>
  ...
</StreamNameGroup>