Creating Custom Packetizers and Depacketizers

Note: The RTP 1.0 API supported custom packetizers and depacketizers through RTP-specific APIs. These APIs have been replaced by the generic JMF plug-in API and any custom packetizers or depacketizers created for RTP 1.0 will need to be ported to the new architecture.

RTP packetizers are responsible for taking entire video frames or multiple audio samples and distributing them into packets of a particular size that can be streamed over the underlying network. Video frames are divided into smaller chunks, while audio samples are typically grouped together. RTP depacketizers reverse the process and reconstruct complete video frames or extract individual audio samples from a stream of RTP packets.

The RTP session manager itself does not perform any packetization or depacketization. These operations are performed by the Processor using specialized Codec plug-ins.

Figure 12-1: JMF RTP architecture.

To determine what RTP packetizer and depacketizer plug-ins are available, you can query the PlugInManager by calling getPlugInList(CODEC). The input and output formats of a particular plug-in can be determined through the getSupportedInputFormats and getSupportedOutputFormats methods.

To receive or transmit any format not supported by one of the standard plug-ins, you need to implement a custom plug-in to perform the necessary conversions. The formats of the data streamed by the DataSource created by the session manager are well-defined to facilitate packetization and depacketization of the formatted data.

For a custom plug-in to work, there must be either a standard or custom plug-in available that can handle the output format it generates. In some cases, if the necessary encoder or decoder is available, you might only need to write a packetizer or depacketizer. In other cases, you might need to provide both the encoder/decoder and packetizer/depacketizer.

Custom packetizers and depacketizers can be combined with custom encoders and decoders, or you can implement independent packetizer and depacketizer plug-ins. For example, a depacketizer-only plug-in might advertise DVI_RTP as its input format and DVI as its output format.

Figure 12-2: Data flow with a custom depacketizer plug-in.

A combined depacketizer-decoder plug-in that decompressed DVI to linear audio would advertise DVI_RTP as its input format and AUDIO_LINEAR as its output format.

Figure 12-3: Data flow with combined depacketizer/decoder plug-in.

RTP Data Handling1

Data is transferred between the session manager and a Player or Processor using the Buffer object. Therefore, all DataSources created by the Processor with an RTP-specific format are buffer DataSources. Similarly, all DataSources created by the session manager and handed over to the Manager for Player creation are buffer DataSources.

All RTP-specific data uses an RTP-specific format encoding as defined in the AudioFormat and VideoFormat classes. For example, gsm RTP encapsulated packets have the encoding set to AudioFormat.GSM_RTP, while jpeg-encoded video formats have the encoding set to VideoFormat.JPEG_RTP.

AudioFormat defines four standard RTP-specific encoding strings:

 public static final String ULAW_RTP = "JAUDIO_G711_ULAW/rtp";
public static final String DVI_RTP = "dvi/rtp";
public static final String G723_RTP = "g723/rtp";
public static final String GSM_RTP = "gsm/rtp";

VideoFormat defines three standard RTP-specific encoding strings:

 public static final String JPEG_RTP = "jpeg/rtp";
public static final String H261_RTP = "h261/rtp";
public static final String H263_RTP = "h263/rtp";

Buffers that have an RTP-specific encoding might have a non-null header defined in Payload-specific headers are not part of the RTPHeader. Instead, payload headers are part of the data object in the Buffers transferred between the Player or Processor and the session manager. The packet's actual RTP header is also included as part of the Buffer object's data. The Buffer object's offset points to the end of this header.

For packets received from the network by the SessionManager, all available fields from the RTP Header (as defined in RFC 1890) are translated to appropriate fields in the Buffer object: timestamp and sequence number. The marker bit from the RTP header is sent over as flags on the Buffer object, which you can access by calling the Buffer getFlags method. The flag used to indicate the marker bit is Buffer.FLAG_RTP_MARKER. If there is an extension header, it is sent over in the header of be Buffer, which is a RTPHeader object. The format of the Buffer is set to AudioFormat.GSM_RTP.

All source streams streamed out on RTP DataSources have their content descriptor set to an empty content descriptor of "" and their format set to the appropriate RTP-specific format and encoding. To be able to intercept or depacketize this data, plug-in codecs must advertise this format as one of their input formats.

For packets being sent over the network, the Processor's format must be set to one of the RTP-specific formats (encodings). The plug-in codec must advertise this format as one of its supported output formats. All Buffer objects passed to the SessionManager through the DataSource sent to createSendStream must have an RTP-specific format. The header of the Buffer is as described in

Dynamic RTP Payloads

The SessionManager has a provision for entering information on dynamic RTP payloads. For more information about how dynamic payloads are used in RTP, refer to IETF RFC 1890, the RTP Audio-Video profile2 that accompanies the RTP specification.

The dynamic RTP-payload information typically contains a mapping from a predetermined RTP payload ID to a specific encoding. In the JMF RTP API, this information is passed via the Format object. To enable playback or transmission of dynamic RTP payloads, you must associate a specific Format with an RTP payload number. This information can be sent to the session manager in two ways:

Registering Custom Packetizers and Depacketizers

Whenever custom packetizers or depacketizers are used, a new payload number must be associated with the RTP format in the session manager's registry. For RTP transmission, you need to call addFormat on the SessionManager to register new formats. For RTP reception, you can either:

1 See the IETF RTP payload specifications for more information about how particular payloads are to be carried in RTP.
2 This document is being revised in preparation for advancement from Proposed Standard to Draft standard. At the time of publication, the most recent draft was


Copyright © 1998-1999 Sun Microsystems, Inc. All Rights Reserved.