Receiving Captionate Embedded Data February 27, 2006, Last updated July 18, 2006
 
Important update: On October 24, 2006, FLVPlayback support skins by Michael A. Jordan was made available for free by Adobe. After installing the MXP (download from this page), it becomes very easy to work with Captionate captions.
Introduction
Captionate lets you easily place data to correct time for Flash Video (FLV) files. After that, you can use the exported XML file or use the FLV with embedded data. Since a component that handles caption data is currently not provided with Captionate, in this tutorial we will look into how to receive the embedded data.
 
Note that cue points data type is supported by Flash 8 FLVPlayback component. You can convert your caption data to cue points with Captionate.
 
Main reason we haven't provided a sample component yet is that Captionate is a generic tool in a sense. For example, you may choose to have HTML formatted captions, or you may choose to have plain text captions, you can make use of the additional fields provided for speakers or language tracks in any way you choose. Even there are options when you are converting captions to cue points. So, while Captionate gives you freedom, any sample component will have to force you to work in a compatible way - or will have to have many options.

 
What are FLV Data and Events?
In a FLV file, along with video and audio, metadata (which we will simply call data) can be present. When you play your FLV, video is displayed, you see it, audio is played, you hear it. For data, Flash Player generates events, which you will need to handle to get the actual data. Name of the event is present in the data block, so Flash Player knows which event to generate. Captionate supports and embeds many different event types like onCaption, onCaptionInfo, onMarker etc.
 
This part is rather fundamental, so in summary, while the FLV is being played, when Flash Player encounters a data packet, it calls the appropriate event handler (that is, a function).
 
To be able to handle the events, you need to have the handler functions in place. Events (event names) are not pre-defined. So in order to handle a specific event, you'll need to have a handler function for that event.

 
onMetaData is an event you may have heard of. It contains information like video dimensions, frame rate, etc. Receiving all FLV data events are similar.
 
Receiving Events
All external FLV files are played using NetConnection and NetStream objects. If you have played video using a component, you may have not seen these objects, but internally all components also use these objects to play external FLV files.
 
FLV data events are called for the current NetStream object playing the FLV. For example, when the Flash Player encounters an onCaption event, it calls the .onCaption function of the NetStream object.
 
First, lets have a look at how external video file can be played using Flash and actionscript:
 
For Flash 8:
  1. Create a new Flash document.
  2. In the Library, choose new video command from the context menu. In the Video Properties dialog, make sure video (actionscript controlled) is selected.
  3. Drag your newly created video symbol to stage.
  4. Click on the video symbol on stage if necessary first and in the Property inspector, assign an instance name. For the following code to work, use the instance name: video.
  5. Click on frame 1 on the timeline and enter the following actionscript (in Actions panel F9):
    	nc = new NetConnection();
    	nc.connect(null);
    	nets = new NetStream(nc);
    	video.attachVideo(nets);
    	nets.play("test.flv");
    
  6. Save the FLA file to a folder where you have a sample FLV file named test.flv.
  7. When you Test Movie (Ctrl+Enter), your FLV will play.
For Flash MX 2004:
  1. Create a new Flash document.
  2. In the Library panel (Window > Library) select New Video from the Library options menu.
  3. Drag your newly created video symbol to stage.
  4. Click on the video symbol on stage if necessary first and in the Property inspector, assign an instance name. For the following code to work, use the instance name: video.
  5. Click on frame 1 on the timeline and enter the following actionscript (in Actions panel F9):
    	nc = new NetConnection();
    	nc.connect(null);
    	nets = new NetStream(nc);
    	video.attachVideo(nets);
    	nets.play("test.flv");
    
  6. Save the FLA file to a folder where you have a sample FLV file named test.flv.
  7. When you Test Movie (Ctrl+Enter), your FLV will play.

 
As you see what we do is this: first we create a video symbol with an instance name. Then, in the 5 lines of actionscript, we create a NetConnection object, then a NetStream object, then we attach the video symbol instance to the NetStream object. Then we supply the FLV name to the play function of the NetStream object and the FLV plays.
 
This sample is probably the shortest code to play a FLV file. In a real application you may want to do more, including providing a controller.
 
In order to receive any data there might be in the FLV file, we need to add event handlers to the NetStream object instance (nets in the above example) which has the video instance attached.
 
Adding the event handlers are quite simple, but you need to make sure you add the handler functions to the correct NetStream instance. For example, in a class code, while seems obvious, you need to make sure the NetStream object is created before adding the events.
 
Now lets add an onCaption event handler to our sample:
	nc = new NetConnection();
	nc.connect(null);
	nets = new NetStream(nc);
	video.attachVideo(nets);
	
	nets.onCaption = function(captions,speaker){
		trace('onCaption event at '+nets.time);	
	}
	
	nets.play("test.flv");
Also note that when using AS2, you may need to use the [""] syntax, as follows, which is exactly the same thing, written in a different way:
	nets["onCaption"] = function(captions,speaker){
		trace('onCaption event at '+nets.time);	
	}
Lets copy onCaption and onCaptionInfo handler samples from Captionate's help file, events section:
	nc = new NetConnection();
	nc.connect(null);
	nets = new NetStream(nc);
	video.attachVideo(nets);
	
	nets.onCaption = function(captions,speaker){
		trace('onCaption event at '+nets.time);
		trace(' Speaker index: '+speaker);
		for (n=0;n<captions.length;n++){
			str=captions[n];
			trace(' Caption for track '+n+' > "'+str+'"');
	}

	nets.onCaptionInfo = function(info){
		trace('onCaptionInfo event at '+nets.time);
		speakers=info.speakers;
		for (n=0;n<speakers.length;n++){
			obj=speakers[n];
			trace(' Speaker '+n+' > '+
			' name: "'+obj.name+'"'+
			' data: "'+obj.data+'"' );
		}
		tracks=info.tracks;
		for (n=0;n<tracks.length;n++){
			obj=tracks[n];
			trace(' Track '+n+' > '+
			' displayname: "'+obj.displayname+'"'+
			' type: "'+obj.type+'"'+
			' languagecode: "'+obj.languagecode+'"'+
			' targetwpm: '+obj.targetwpm+
			' data: "'+obj.data+'"' );
			}
		}

	}

	
	nets.play("test.flv");
Now our simple sample can trace all the data from onCaption and onCaptionInfo events. You can add other event handlers from Captionate's help and trace the data.
 
As you see every event has its own parameters and data structure. For example, onCaptionInfo has speakers and tracks arrays. onCaption has two parameters, one holds caption text for all language tracks and the other is index to the speakers array. The type of the parameter for onMarker is a string... So, you need to add the correct event handler to the NetStream instance and in the handler function, you need to 'parse' the data received...

 
Receiving Events when Using Standard Components
'That's fine', you say, 'but I use the MediaPlayback component in Flash MX Professional 2004, now what?'
 
All components play the FLV internally similar to our sample above, using the NetStream object. You'll need to add the events to the NetStream instance in the components instance on stage, indeed very similar to our simple sample.
 
Flash MX Professional 2004 MediaDisplay Component
Lets say the instance name of the MediaDisplay component is md. The Netstream object instance can be reached as md._playerImpl._ns:.
	var nets = md._playerImpl._ns;
Using the above line, you can get the NetStream object instance to the nets variable and use Captionate help samples for adding events.
 
Flash MX Professional 2004 MediaPlayback Component
Lets say the instance name of the MediaPlayback component is mp. The Netstream object instance can be reached as mp._display._playerImpl._ns:.
	var nets = mp._display._playerImpl._ns;
Using the above line, again you can get the NetStream object instance to the nets variable and use Captionate help samples for adding events.
 
(If this is not clear: Create a new Flash document, drag a MediaPlayback component on stage, assign component instance name as 'mp'. In the component inspector, enter the path to your FLV into the URL field. Add the above code to the first frame, then append handler functions just as the first sample , i.e. like: nets.onCaption = function(captions,speaker){....).
 
Flash 8 Professional FLVPlayback Component
FLVPlayback unfortunately does not expose the internal NetStream object like other components, as it supports multiple videos. Still you can add events by replacing the original _createStream function.
 
Create a new FLA and drag a FLVPlayback component to stage. Then enter the following code to the first frame:
_global.mx.video.VideoPlayer.prototype._createStream  =
function () {
           //original _createStream code
           this._ns = new NetStream (this._ncMgr.getNetConnection());
           this._ns.mc = this;
           if (this._ncMgr.isRTMP()) {
               this._ns.onStatus = function (info) {
                   this.mc.rtmpOnStatus(info);
               };
           } else {
               this._ns.onStatus = function (info) {
                   this.mc.httpOnStatus(info);
               };
            }
           this._ns.onMetaData = function (info) {
               this.mc.onMetaData(info);
           };
           this._ns.onCuePoint = function (info) {
               this.mc.onCuePoint(info);
           };
           this._ns.setBufferTime(this._bufferTime);

           //new code
           this._ns.onCaption = function(captions,speaker){
             trace("onCaption");
          }
          // add other events here
       };

 
The above is really a hack-ish workaround. Converting captions to cue points will suit your needs most of the time. (Also please see the contribution by Peter Edwards below).

 
Handling the Data
Handling the received data will be covered in another article.
 
Still, if you are not aware, the easiest way to display text on stage is using a dynamic text with a variable. Create a text field on stage using the text tool. In the property inspector, set the type to 'dynamic text' and input a variable name into the 'var' field. Now, anytime you assign to that variable, it will be displayed in the text field.
 
Lets see this in action!
  1. Create a new FLA in Flash 8 and drag a FLVPlayback component on stage. Set the content path to your FLV.
  2. In a new layer, draw a rectangle, convert it to symbol, change color to cyan (or white), alpha to 50% and put it over the FLVPlayback component.
  3. In a new layer, create a text field using text tool, change type to 'dynamic', enter 'caption' as the 'var' name. Put it over the rectangle. Select 'Render as HTML' if your captions have HTML formatting, set the font.
  4. Enter the code above for FLVPlayback component to the first frame. Add the following line to onCaption function:
    	caption = captions[0];
    
    This will display the caption text for the first language track, it will do for now.
  5. Here's the result:
  6. Hint: You may add a button someplace in your interface that toggles the visibility of the captions.

 

Simpler, Less hacky code for Flash 8 Professional FLVPlayback Component
by Peter Edwards of derivco.com (July 17, 2006)
 
I thought you might like to know that your example for Flash 8 Professional FLVPlayback Component can be replaced with some much simpler, less hacky code:
// save the old function
var tmp_createStream : Function = _global.mx.video.VideoPlayer.prototype._createStream;

_global.mx.video.VideoPlayer.prototype._createStream = function() : Void
{
  // apply the old function in the instance context
  tmp_createStream.apply(this, null);

  // add our specifics
  this._ns.onCaption = function(captions,speaker) : Void
  {
    trace("onCaption");
  }
};

In fact, if you write a function like:
  private function fixVideoPlayer(func : Function) : Void
  {
    var tmp : Function = _global.mx.video.VideoPlayer.prototype._createStream;

    _global.mx.video.VideoPlayer.prototype._createStream = function() : Void
    {
      tmp.apply(this, null);
      this._ns.onCaption = func;
    };
  }
Then just before the instance of the FLVPlayback component is created, you can make a call like:
fixVideoPlayer(Delegate.create(this, this.onCaption));
and define the onCaption function in your class appropriately. This could even work per instance / per VideoPlayer with a little tweaking.
 
An even Better Version:
by Peter Edwards of derivco.com (July 18, 2006)
function fixVideoPlayerInst(player : mx.video.VideoPlayer, nsFuncName:
String, func : Function) : Void
{
  var tmp : Function = player["_createStream"];
  player["_createStream"] = function() : Void
  {
    tmp.apply(this, null);
    this._ns[nsFuncName] = func;
  };
}
With this one, only the instance of the VideoPlayer you are interested in will be patched, the function to add to the NetStream is dynamic and so is the actual function to call:
var flvp : FLVPlayback = FLVPlayback(mc.flvp);

fixVideoPlayerInst(flvp.getVideoPlayer(0), "onCaption",
Delegate.create(this, this.onCaption));
Which also allows for a metadata function per video stream (if you have multiple VideoPlayers in the FLVPlayback instance).
 
Which is probably a little cleaner.


 
Please send any feedback about this article to support@captionate.com
 
Copyright © 2005-2006 Manitu Group. All rights reserved. All trademarks acknowledged.