CONTENTS | PREV | NEXT | INDEX JMF 2.0 API Guide



9

Receiving and Presenting RTP Media Streams

JMF Players and Processors provide the presentation, capture, and data conversion mechanisms for RTP streams.


Figure 9-1: RTP reception data flow.

A separate player is used for each stream received by the session manager. You construct a Player for an RTP stream through the standard Manager createPlayer mechanism. You can either:

If you use a MediaLocator to construct a Player, you can only present the first RTP stream that's detected in the session. If you want to play back multiple RTP streams in a session, you need to use the SessionManager directly and construct a Player for each ReceiveStream.

Creating a Player for an RTP Session

When you use a MediaLocator to construct a Player for an RTP session, the Manager creates a Player for the first stream detected in the session. This Player posts a RealizeCompleteEvent once data has been detected in the session.

By listening for the RealizeCompleteEvent, you can determine whether or not any data has arrived and if the Player is capable of presenting any data. Once the Player posts this event, you can retrieve its visual and control components.

Note: Because a Player for an RTP media stream doesn't finish realizing until data is detected in the session, you shouldn't try to use Manager.createRealizedPlayer to construct a Player for an RTP media stream. No Player would be returned until data arrives and if no data is detected, attempting to create a Realized Player would block indefinitely.

A Player can export one RTP-specific control, RTPControl, which provides overall session statistics and can be used for registering dynamic payloads with the SessionManager.

Example 9-1: Creating a Player for an RTP session (1 of 2)
        String url= "rtp://224.144.251.104:49150/audio/1";
 
         MediaLocator mrl= new MediaLocator(url);
         
         if (mrl == null) {
             System.err.println("Can't build MRL for RTP");
             return false;
         }
         
         // Create a player for this rtp session
         try {
             player = Manager.createPlayer(mrl);
         } catch (NoPlayerException e) {
             System.err.println("Error:" + e);
             return false;
         } catch (MalformedURLException e) {
             System.err.println("Error:" + e);
             return false;
         } catch (IOException e) {
             System.err.println("Error:" + e);
             return false;
         }
         
         if (player != null) {
             if (this.player == null) {
                 this.player = player;
                 player.addControllerListener(this);
                 player.realize();
             }
         }

Listening for Format Changes

When a Player posts a FormatChangeEvent, it might indicate that a payload change has occurred. Players constructed with a MediaLocator automatically process payload changes. In most cases, this processing involves constructing a new Player to handle the new format. Applications that present RTP media streams need to listen for FormatChangeEvents so that they can respond if a new Player is created.

When a FormatChangeEvent is posted, check whether or not the Player object's control and visual components have changed. If they have, a new Player has been constructed and you need to remove references to the old Player object's components and get the new Player object's components.

Example 9-2: Listening for RTP format changes (1 of 2)
     public synchronized void controllerUpdate(ControllerEvent ce) {
         if (ce instanceof FormatChangeEvent) {
             Dimension vSize = new Dimension(320,0);
             Component oldVisualComp = visualComp;
             
             if ((visualComp = player.getVisualComponent()) != null) {
                 if (oldVisualComp != visualComp) {
                     if (oldVisualComp != null) {
                         oldVisualComp.remove(zoomMenu);
                     }
                     
                     framePanel.remove(oldVisualComp);
                     
                     vSize = visualComp.getPreferredSize();
                     vSize.width = (int)(vSize.width * defaultScale);
                     vSize.height = (int)(vSize.height * defaultScale);
                     
                     framePanel.add(visualComp);
                     
                     visualComp.setBounds(0, 
                                          0, 
                                          vSize.width, 
                                          vSize.height);
                     addPopupMenu(visualComp);
                 }
             }
 
             Component oldComp = controlComp;
             
             controlComp = player.getControlPanelComponent();
 
             if (controlComp != null) 
             {
                 if (oldComp != controlComp)
                 {
                     framePanel.remove(oldComp);
                     framePanel.add(controlComp);
                                          
                     if (controlComp != null) {
                         int prefHeight = controlComp
                                          .getPreferredSize()
                                          .height;
                         
                         controlComp.setBounds(0, 
                                               vSize.height,
                                               vSize.width,
                                               prefHeight);
                     }
                 }
             }
         }
     }

Creating an RTP Player for Each New Receive Stream

To play all of the ReceiveStreams in a session, you need to create a separate Player for each stream. When a new stream is created, the session manager posts a NewReceiveStreamEvent. Generally, you register as a ReceiveStreamListener and construct a Player for each new ReceiveStream. To construct the Player, you retrieve the DataSource from the ReceiveStream and pass it to Manager.createPlayer.

To create a Player for each new receive stream in a session:

  1. Set up the RTP session:
    1. Create a SessionManager. For example, construct an instance of com.sun.media.rtp.RTPSessionMgr. (RTPSessionMgr is an implementation of SessionManager provided with the JMF reference implementation.)
    2. Call RTPSessionMgr addReceiveStreamListener to register as a listener.
    3. Initialize the RTP session by calling RTPSessionMgr initSession.
    4. Start the RTP session by calling RTPSessionMgr startSession.
Example 9-3: Setting up an RTP session (1 of 2)
     public SessionManager createManager(String address,
                                         int port,
                                         int ttl,
                                         boolean listener,
                                         boolean sendlistener)
     {
         mgr = (SessionManager)new com.sun.media.rtp.RTPSessionMgr();
         
         if (mgr == null) return null;
 
         mgr.addFormat(new AudioFormat(AudioFormat.DVI_RTP,
                                       44100,
                                       4,
                                       1),
                       18);
 
         if (listener) mgr.addReceiveStreamListener(this);
         if (sendlistener) new RTPSendStreamWindow(mgr);
         
         // ask session mgr to generate the local participant's CNAME
         String cname = mgr.generateCNAME();
         String username = null;
 
         try {
             username = System.getProperty("user.name");
         } catch (SecurityException e){
             username = "jmf-user";
         }
         
         // create our local Session Address
         SessionAddress localaddr = new SessionAddress();
         
         try{
             InetAddress destaddr = InetAddress.getByName(address);
 
             SessionAddress sessaddr = new SessionAddress(destaddr,
                                                          port,
                                                          destaddr,
                                                          port + 1);
             SourceDescription[] userdesclist= new SourceDescription[]
             {
                 new SourceDescription(SourceDescription
                                       .SOURCE_DESC_EMAIL,
                                       "jmf-user@sun.com",
                                       1,
                                       false),
 
                 new SourceDescription(SourceDescription
                                       .SOURCE_DESC_CNAME,
                                       cname,
                                       1,
                                       false),
 
                 new SourceDescription(SourceDescription
                                       .SOURCE_DESC_TOOL,
                                       "JMF RTP Player v2.0",
                                       1,
                                       false)
             };
 
             mgr.initSession(localaddr,
                             userdesclist,
                             0.05,
                             0.25);
             
             mgr.startSession(sessaddr,ttl,null);
         } catch (Exception e) {
             System.err.println(e.getMessage());
             return null;
         }
         
         return mgr;
     }

  1. In your ReceiveStreamListener update method, watch for NewReceiveStreamEvent, which indicates that a new data stream has been detected.
  2. When a NewReceiveStreamEvent is detected, retrieve the ReceiveStream from the NewReceiveStreamEvent by calling getReceiveStream.
  3. Retrieve the RTP DataSource from the ReceiveStream by calling getDataSource. This is a PushBufferDataSource with an RTP-specific Format. For example, the encoding for a DVI audio player will be DVI_RTP.
  4. Pass the DataSource to Manager.createPlayer to construct a Player. For the Player to be successfully constructed, the necessary plug-ins for decoding and depacketizing the RTP-formatted data must be available. (For more information, see Creating Custom Packetizers and Depacketizers).
Example 9-4: Listening for NewReceiveStreamEvents  
     public void update( ReceiveStreamEvent event)
     {
         Player newplayer = null;
         RTPPlayerWindow playerWindow = null;
 
         // find the sourceRTPSM for this event
         SessionManager source = (SessionManager)event.getSource();
 
         // create a new player if a new recvstream is detected
         if (event instanceof NewReceiveStreamEvent)
         {
             String cname = "Java Media Player";
             ReceiveStream stream = null;
             
             try
             {
                 // get a handle over the ReceiveStream
                 stream =((NewReceiveStreamEvent)event)
                         .getReceiveStream();
 
                 Participant part = stream.getParticipant();
 
                 if (part != null) cname = part.getCNAME();
 
                 // get a handle over the ReceiveStream datasource
                 DataSource dsource = stream.getDataSource();
                 
                 // create a player by passing datasource to the 
                 // Media Manager
                 newplayer = Manager.createPlayer(dsource);
                 System.out.println("created player " + newplayer);
             } catch (Exception e) {
                 System.err.println("NewReceiveStreamEvent exception " 
                                    + e.getMessage());
                 return;
             }
 
             if (newplayer == null) return;
 
             playerlist.addElement(newplayer);
             newplayer.addControllerListener(this);
            
             // send this player to player GUI
             playerWindow = new RTPPlayerWindow( newplayer, cname);
         }
     }

See RTPUtil in RTPUtil for a complete example.

Handling RTP Payload Changes

If the payload of a stream in the RTP session changes, the ReceiveStream posts a RemotePayloadChangeEvent. Generally, when the payload changes, the existing Player will not be able to handle the new format and JMF will throw an error if you attempt to present the new payload. To avoid this, your ReceiveStreamListener needs to watch for RemotePayloadChangeEvents. When a RemotePayloadChangeEvent is detected, you need to:

  1. Close the existing Player.
  2. Remove all listeners for the removed Player.
  3. Create a new Player with the same RTP DataSource.
  4. Get the visual and control Components for the new Player.
  5. Add the necessary listeners to the new Player.
Example 9-5: Handling RTP payload changes (1 of 2)
     public void update(ReceiveStreamEvent event) {
         if (event instanceof RemotePayloadChangeEvent) {
             // payload has changed. we need to close the old player
             // and create a new player  
             
             if (newplayer != null) {
                 // stop player and wait for stop event
                 newplayer.stop();
                
                 // block until StopEvent received...
        
                 // remove controllerlistener
                 newplayer.removeControllerListener(listener);
                
                 // remove any visual and control components 
                 // attached to this application
                 // close the player and wait for close event    
                 newplayer.close();
                
                 // block until ControllerClosedEvent received...
        
                 try {
                     // when the player was closed, its datasource was
                     // disconnected. Now we must reconnect the data-
                     // source before a player can be created for it.
                     // This is the same datasource received from 
                     // NewReceiveStreamEvent and used to create the 
                     // initial rtp player
                     
                     rtpsource.connect();
                     newplayer = Manager.createPlayer(rtpsource);
                
                     if (newplayer == null) {
                         System.err.println("Could not create player");
                         return;
                     }
                     
                     newplayer.addControllerListener(listener);
                     newplayer.realize();
        
                     // when the new player is realized, retrieve its
                     // visual and control components
                 } catch (Exception e) {
                     System.err.println("could not create player");
                 }
             }
         }
     }

Controlling Buffering of Incoming RTP Streams

You can control the RTP receiver buffer through the BufferControl exported by the SessionManager. This control enables you to set two parameters, buffer length and threshold.

The buffer length is the size of the buffer maintained by the receiver. The threshold is the minimum amount of data that is to be buffered by the control before pushing data out or allowing data to be pulled out (jitter buffer). Data will only be available from this object when this minimum threshold has been reached. If the amount of data buffered falls below this threshold, data will again be buffered until the threshold is reached.

The buffer length and threshold values are specified in milliseconds. The number of audio packets or video frames buffered depends on the format of the incoming stream. Each receive stream maintains its own default and maximum values for both the buffer length and minimum threshold. (The default and maximum buffer lengths are implementation dependent.)

To get the BufferControl for a session, you call getControl on the SessionManager. You can retrieve a GUI Component for the BufferControl by calling getControlComponent.

Presenting RTP Streams with RTPSocket

RTP is transport-protocol independent. By using RTPSocket, you can stream RTP from any underlying network. The format of the RTP socket is designed to have both a data and a control channel. Each channel has an input and output stream to stream data into and out of the underlying network.

SessionManager expects to receive individual RTP packets from the RTPSocket. Users are responsible for streaming individual RTP packets to the RTPSocket.

To play an RTP stream from the RTPSocket, you pass the socket to Manager.createPlayer to construct the Player. Alternatively, you could construct a Player by calling createPlayer(MediaLocator) and passing in a MediaLocator with a new protocol that is a variant of RTP, "rtpraw". For example:

 Manager.createPlayer(new MediaLocator("rtpraw://"));

According to the JMF Player creation mechanism, Manager will attempt to construct the DataSource defined in:

 <protocol package-prefix>.media.protocol.rtpraw.DataSource

This must be the RTPSocket. The content of the RTPsocket should be set to rtpraw. Manager will then attempt to create a player of type <content-prefix>.media.content.rptraw.Handler and set the RTPSocket on it.

Note: The RTPSocket created at <protocol package-prefix>.media.protocol.rtpraw.DataSource is your own implementation of RTPSocket. The JMF API does not define a default implementation of RTPSocket. The implementation of RTPSocket is dependent on the underlying transport protocol that you are using. Your RTPSocket class must be located at <protocol package-prefix>.media.protocol.rtpraw.DataSource and its control and data channel streams must be set as shown in the following example.

RTPControl interfaces for the RTPSocket can be used to add dynamic payload information to the RTP session manager.

The following example implements an RTP over UDP player that can receive RTP UDP packets and stream them to the Player or session manager, which is not aware of the underlying network/transport protocol. This sample uses the interfaces defined in javax.media.rtp.RTPSocket and its related classes.

Example 9-6: RTPSocketPlayer (1 of 6)
 import java.io.*;
 import java.net.*;
 import java.util.*;
 
 import javax.media.*;
 import javax.media.format.*;
 import javax.media.protocol.*;
 import javax.media.rtp.*;
 import javax.media.rtp.event.*;
 import javax.media.rtp.rtcp.*;
 
 public class RTPSocketPlayer implements ControllerListener {
     // ENTER THE FOLLOWING SESSION PARAMETERS FOR YOUR RTP SESSION 
     
     // RTP Session address, multicast, unicast or broadcast address
     String address = "224.144.251.245";
     
     // RTP Session port
     int port = 49150;
     
     // Media Type i.e. one of audio or video
     String media = "audio";
     
     // DO NOT MODIFY ANYTHING BELOW THIS LINE  
          
     // The main rtpsocket abstraction which we will create and send
     // to the Manager for appropriate handler creation
     RTPSocket rtpsocket = null;
        
     // The control RTPPushDataSource of the above RTPSocket 
     RTPPushDataSource rtcpsource = null;
     
     // The GUI to handle the player
     // PlayerWindow playerWindow;
     
     // The handler created for the RTP session, 
     // as returned by the Manager
     Player player;
 
     // maximum size of buffer for UDP receive from the sockets
     private  int maxsize = 2000;
 
     UDPHandler rtp = null;
     UDPHandler rtcp = null;
       
     public RTPSocketPlayer() {
         // create the RTPSocket
         rtpsocket = new RTPSocket();
         // set its content type : 
         // rtpraw/video for a video session 
         // rtpraw/audio for an audio session
         String content = "rtpraw/" + media;
         rtpsocket.setContentType(content);
         
         // set the RTP Session address and port of the RTP data
         rtp = new UDPHandler(address, port);
         
         // set the above UDP Handler to be the 
         // sourcestream of the rtpsocket
         rtpsocket.setOutputStream(rtp);
         
         // set the RTP Session address and port of the RTCP data
         rtcp = new UDPHandler(address, port +1);
         
         // get a handle over the RTCP Datasource so that we can 
         // set the sourcestream and deststream of this source 
         // to the rtcp udp handler we created above.
         rtcpsource = rtpsocket.getControlChannel();
         
         // Since we intend to send RTCP packets from the         
         // network to the session manager and vice-versa, we need
         // to set the RTCP UDP handler as both the input and output 
         // stream of the rtcpsource.
         rtcpsource.setOutputStream(rtcp);
         rtcpsource.setInputStream(rtcp);
         
         // connect the RTP socket data source before 
         // creating the player
         try {
             rtpsocket.connect();
             player = Manager.createPlayer(rtpsocket);
             rtpsocket.start();
         } catch (NoPlayerException e) {
             System.err.println(e.getMessage());
             e.printStackTrace();
             return;
         }
         catch (IOException e) {
             System.err.println(e.getMessage());
             e.printStackTrace();
             return;
         }
 
         if (player != null) {
             player.addControllerListener(this);
             // send this player to out playerwindow
             // playerWindow = new PlayerWindow(player);
         }
     }
 
     public synchronized void controllerUpdate(ControllerEvent ce) {
         if ((ce instanceof DeallocateEvent) ||
             (ce instanceof ControllerErrorEvent)) {
         
             // stop udp handlers
             if (rtp != null) rtp.close();
             
             if (rtcp != null) rtcp.close();
         }
     }
     
     // method used by inner class UDPHandler to open a datagram or
     // multicast socket as the case maybe
     
     private DatagramSocket InitSocket(String sockaddress, 
                                      int     sockport) 
     {
         InetAddress addr = null;
         DatagramSocket sock = null;
 
         try {
             addr = InetAddress.getByName(sockaddress);
             
             if (addr.isMulticastAddress()) {
                 MulticastSocket msock;
                 
                 msock = new MulticastSocket(sockport);
 
                 msock.joinGroup(addr);
 
                 sock = (DatagramSocket)msock;           
             } 
             else {              
                 sock = new DatagramSocket(sockport,addr);
             }
             
             return sock;
         }
         catch (SocketException e) {
             e.printStackTrace();
             return null;
         }
         catch (UnknownHostException e) {
             e.printStackTrace();
             return null;
         }
         catch (IOException e) {
             e.printStackTrace();
             return null;
         }
     }
 
     // INNER CLASS UDP Handler which will receive UDP RTP Packets and
     // stream them to the handler of the sources stream. IN case of
     // RTCP, it will also accept RTCP packets and send them on the
     // underlying network.
 
     public class UDPHandler extends Thread implements PushSourceStream, 
                                                       OutputDataStream
     {
         DatagramSocket        mysock;
         DatagramPacket        dp;
         SourceTransferHandler outputHandler;
         String                myAddress;
         int                   myport;
         boolean               closed = false;
 
 
         // in the constructor we open the socket and create the main
         // UDPHandler thread.
         
         public UDPHandler(String haddress, int hport) {
             myAddress = haddress;
             myport = hport;
             mysock = InitSocket(myAddress,myport);                  
             setDaemon(true);
             start();
         }
 
         // the main thread receives RTP data packets from the
         // network and transfer's this data to the output handler of
         // this stream.
         
         public void run() {
             int len;
 
             while(true) {
                 if (closed) {
                     cleanup();
                     return;
                 }
                 try {
                     do {
                         dp = new DatagramPacket( new byte[maxsize],
                                                  maxsize);
                         
                         mysock.receive(dp);
 
                         if (closed){
                             cleanup();
                             return;
                         }
                         
                         len = dp.getLength();
                         if (len > (maxsize >> 1)) maxsize = len << 1;
                     }
                     while (len >= dp.getData().length);
                 }catch (Exception e){
                     cleanup();
                     return;
                 }
                 
                 if (outputHandler != null) {
                     outputHandler.transferData(this);
                 }
             }
         }
 
         public void close() {
             closed = true;
         }
 
         private void cleanup() {
             mysock.close();
             stop();
         }
         
         // methods of PushSourceStream
         public Object[] getControls() {
             return new Object[0];
         }
         
         public Object getControl(String controlName) {
             return null;
         }
 
         public ContentDescriptor getContentDescriptor() {
             return null;
         }
 
         public long getContentLength() {
             return SourceStream.LENGTH_UNKNOWN;
         }
 
         public boolean endOfStream() {
             return false;
         }
 
         // method by which data is transferred from the underlying
         // network to the session manager.
         
         public int read(byte buffer[],
                         int offset,
                         int length) 
         {
             System.arraycopy(dp.getData(),
                              0,
                              buffer,
                              offset,
                              dp.getLength());
             
             return dp.getData().length;
         }                
         
         public int getMinimumTransferSize(){
             return dp.getLength();
         }
         
         public void setTransferHandler(SourceTransferHandler
                                        transferHandler)
         {
             this.outputHandler = transferHandler;
         }
         
         // methods of OutputDataStream used by the session manager to 
         // transfer data to the underlying network.
         
         public int write(byte[] buffer,
                          int offset,
                          int length)
         {
             InetAddress addr = null;
         
             try {
                 addr = InetAddress.getByName(myAddress);
             } catch (UnknownHostException e) {
                 e.printStackTrace();
             }
 
             DatagramPacket dp = new DatagramPacket( buffer, 
                                                     length,
                                                     addr,
                                                     myport);
             try {
                 mysock.send(dp);
             } catch (IOException e){
                 e.printStackTrace();
             }
             
             return dp.getLength();
         }
     }
 
     public static void main(String[] args) {
         new RTPSocketPlayer();
     }
 }



CONTENTS | PREV | NEXT | INDEX

Copyright © 1998-1999 Sun Microsystems, Inc. All Rights Reserved.