Internet Engineering Task Force Audio Video Transport WG Internet Draft J.Rosenberg,H.Schulzrinne draft-ietf-avt-fec-05.txt Bell Laboratories,Columbia U. February 26, 1999 Expires: August 1999 An RTP Payload Format for Generic Forward Error Correction STATUS OF THIS MEMO This document is an Internet-Draft and is in full conformance with all provisions of Section 10 of RFC2026. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet- Drafts as reference material or to cite them other than as work in progress. The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. 1 Abstract This document specifies a payload format for generic forward error correction of media encapsulated in RTP. It is engineered for FEC algorithms based on the exclusive-or (parity) operation. The payload format allows end systems to transmit using arbitrary block lengths and parity schemes. It also allows for the recovery of both the pay- load and critical RTP header fields. Since FEC is sent as a separate stream, it is backwards compatible with non-FEC capable hosts, so that receivers which do not wish to implement FEC can just ignore the extensions. 2 Introduction The quality of packet voice on the Internet has been mediocre due, in part, to high packet loss rates. This is especially true on wide-area connections. Unfortunately, the strict delay requirements of real- time multimedia usually eliminate the possibility of retransmissions. J.Rosenberg,H.Schulzrinne [Page 1]
Internet Draft Generic FEC February 26, 1999 It is for this reason that forward error correction (FEC) has been proposed to compensate for packet loss in the Internet [1] [2]. In particular, the use of traditional error correcting codes, such as parity, Reed-Solomon, and Hamming codes, has attracted attention. To support these mechanisms, protocol support is required. This document defines a payload format for RTP [3] which allows for generic forward error correction of real time media. In this context, generic means that the FEC protocol is (1) independent of the nature of the media being protected, be it audio, video, or otherwise, (2) flexible enough to support a wide variety of FEC mechanisms, (3) designed for adaptivity so that the FEC technique can be modified easily without out of band signaling, and (4) supportive of a number of different mechanisms for transporting the FEC packets. 3 Terminology The following terms are used throughout this document: 1. Media Payload: is a piece of raw, un-protected user data which is to be transmitted from the sender. The media pay- load is placed inside of an RTP packet. 2. Media Header: is the RTP header for the packet containing the media payload. 3. Media Packet: The combination of a media payload and media header is called a media packet. 4. FEC Packet: The forward error correction algorithms at the transmitter take the media packets as an input. They output both the media packets that they are passed, and new pack- ets called FEC packets. The FEC packets are formatted according to the rules specified in this document. 5. FEC Header: The FEC header is the header information con- tained in an FEC packet. 6. FEC Payload: The FEC payload is the payload in an FEC packet. 7. Associated: An FEC packet is said to be "associated" with one or more media packets when those media packets are used to generate the FEC packet (by use of the exclusive or operation). The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this J.Rosenberg,H.Schulzrinne [Page 2]
Internet Draft Generic FEC February 26, 1999 document are to be interpreted as described in RFC 2119 [4]. 4 Basic Operation The FEC packets are sent as a separate stream from the media packets. This implies that the FEC packets have their own sequence number space. Although the timestamps for the FEC packets are derived from the media packets, they increment monotonically. FEC packet streams thus work well with any header compression mechanism which requires fixed deltas between fields in the packet header. The media packet stream is essentially unaffected by the use of FEC. This allows the two to be sent on a separate multicast group, so that non-FEC receivers can ignore the FEC and just receive the original media. The separation also allows for coherent values for the sequence numbers and timestamps. This document does not prescribe the definition of "separate streams", but leaves this to applications and higher level protocols to define. For multicast, the separate stream may be implemented by separate multicast groups, different ports in the same group, or by a different SSRC within the same group/port. For unicast, different ports or different SSRC may be used. Each of these approaches has drawbacks and benefits which depend on the application. At the receiver, arriving FEC and media packets are used to generate a stream of media packets for direct use by the application. This results in a clean separation of error protection from the applica- tion. RTP packets which contain data formatted according to this specifica- tion (i.e., FEC packets) are signaled using dynamic RTP payload types. 5 Parity Codes For brevity, we define the function f(x,y,..) to be the XOR (parity) operator applied to the data blocks x,y,... Each data block is simply a set of bits of length L. The function is only well defined when the lengths of the data blocks it operates on are equal. The output of this function is a single data block equal in length to the inputs, called the parity block. The parity block is the bitwise XOR of the input blocks. Note that f(x) = x. Recovery of data blocks using parity codes is accomplished by gen- erating one or more parity blocks over a group of k packets. To be effective, the parity blocks must be generated by linearly indepen- dent combinations of data blocks. The particular combination is called a parity code. After the parity operation, there will be a J.Rosenberg,H.Schulzrinne [Page 3]
Internet Draft Generic FEC February 26, 1999 total of n data plus parity blocks (i.e., n-k parity blocks). There are a large number of possible parity codes for a given n,k. Reason- able codes exist for large ranges of n and k. The payload format does not mandate a particular code. For example, consider a parity code which generates a single parity block over two data blocks. The stream of blocks generated by the code is thus: a, b, f(a,b), c, d, f(c,d) In this example, the error correction scheme (we use the terms scheme and code interchangeably) introduces a 50% overhead. But if b is lost, a and f(a,b) can be used to recover b. Some additional codes are listed below. In each, the letters on the left represent the stream of input data blocks, and the right represents the stream of data and parity blocks. Scheme 0 -------- This scheme is null, and has no error correction. The scheme is formally defined as: a,b,c,d, ... -> a, b, c, d, .... Scheme 1 -------- This scheme is the similar to the one in the example above. The switching of the positions of f(b) and f(a,b) allow some bursts of two consecutive packet losses to be recovered. It is defined as: a,b,c,d,e,f -> a, f(a,b), b, f(b,c), c, f(c,d), d, ... Scheme 2 -------- This scheme allows for recovery of all single packet losses and some consecutive packet losses, but with less overhead than scheme 1: a,b,c,d,e,f,g -> f(a,b),f(a,c),f(a,b,c),f(c,d),f(c,e),f(c,d,e)... J.Rosenberg,H.Schulzrinne [Page 4]
Internet Draft Generic FEC February 26, 1999 Scheme 3 -------- This scheme requires 4 packet delays to recover the original media payloads, but it can recover from 1,2, or 3 consecutive packet losses: a,b,c,d,e,f -> f(a),f(b),f(a,b,c),f(c),f(a,c,d),f(a,b,d),f(d), ... The FEC protocol takes the media payloads as data blocks, and uses the XOR operator on them to generate parity blocks which are the FEC payloads. The xor operator is also applied to portions of the media headers to assist in their recovery. In order to decode the FEC payloads to media payloads, all that is necessary is for the receiver to know the set of media payloads in each FEC payload to which it is applied. This is exactly the informa- tion provided by the payload format. To determine which packets are associated with the FEC packet, a field is present in the FEC header, called the offset mask. Assume this mask is M bits. If bit i in the mask is set to 1, then the media packet with sequence number N + i is associated with this FEC packet, where N is the sequence number base in the FEC packet header. This effectively allows an FEC packet to be associated with any of the M packets before it. The offset mask and payload type are sufficient to signal arbitrary parity based forward error correction schemes with little overhead. 6 RTP Media Packet Structure The media packets and FEC packets are sent as separate streams. The media packets are unaffected by FEC, and are sent in the same fashion they would be sent if there were no FEC. This lends to a very efficient encoding. When little (or no) FEC is used, there are mostly media packets being sent. This means that the overhead (present in FEC packets only) tracks the amount of FEC in use. 7 FEC Packet Structure An FEC packet is constructed by placing an FEC header and FEC payload in the RTP payload, as shown in Figure 1: J.Rosenberg,H.Schulzrinne [Page 5]
Internet Draft Generic FEC February 26, 1999 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | RTP Header | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | FEC Header | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | FEC Payload | | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Figure 1: FEC Packet Structure 7.1 RTP Header The version field is set to 2. The padding bit is computed via the protection operation, defined below. The extension bit is also com- puted via the protection operation. The SSRC value will generally be the same as the SSRC value of the media stream it protects. It MAY be different if the FEC stream is being demultiplexed via the SSRC value. The CC value is computed via the protection operation. The CSRC list is never present, independent of the value of the CC field. The extension is never present, independent of the value of the X bit. The marker bit is computed via the protection operation. The sequence number has the standard definition: it MUST be one higher than the sequence number in the previously transmitted FEC packet. The timestamp is set in the following fashion. When the FEC packet is sent, the value of the media RTP timestamp is examined. This value SHOULD be used as the timestamp of the FEC packet. This results in the TS value in FEC packets to be monotonically increas- ing, independent of the FEC scheme. The payload type for the FEC packet is determined through dynamic, out of band means. According to RFC1889 [3], RTP participants which cannot recognize a payload type must discard it. This provides back- wards compatibility. The FEC mechanisms can then be used in a multi- cast group with mixed FEC-capable and FEC-incapable receivers. 7.2 Parity Header This header is 96 bits. The format of the header is shown in Figure 2. Figure 2: Parity Header Format J.Rosenberg,H.Schulzrinne [Page 6]
Internet Draft Generic FEC February 26, 1999 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | SN Base | length recovery | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |E| PT Recovery | Mask | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | TS Recovery | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ The length recovery field is used to determine the length of any recovered packets. It is computed via the protection operation applied to the 16 bit natural binary representation of the lengths (in bytes) of the media payload, CSRC list, extension and padding of media packets associated with this FEC packet (in other words, the CSRC list, extension, and padding, if present, are "counted" as part of the payload). This allows for the FEC procedure to be applied even when the lengths of the media packets are not identical. For example, assume an FEC packet is being generated by xor'ing two media packets together. The length of the two media packets are 3 (0b011) and 5 (0b101) bytes, respectively. The length recovery field is then encoded as 0b011 xor 0b101 = 0b110. The E bit indicates a header extension. Implementations conforming to this version of the specification MUST set this bit to zero. The PT recovery field is obtained via the protection operation applied to the payload type values of the media packets associated with the FEC packet. The mask field is 24 bits. If bit i in the mask is set to 1, then the media packet with sequence number N + i is associated with this FEC packet, where N is the SN Base field in the FEC packet header. The least significant bit corresponds to i=0, and the most significant to i=23. The SN base field SHOULD be set to the minimum sequence number of those media packets protected by FEC. This allows for the FEC opera- tion to extend over any string of at most 24 packets. The TS recovery field is computed via the protection operation applied to the timestamps of the media packets associated with this FEC packet. This allows the timestamp to be completely recovered. The payload of the FEC packet is the protection operation applied to the CSRC List plus payload plus padding plus extension of the media packets associated with the FEC packet. J.Rosenberg,H.Schulzrinne [Page 7]
Internet Draft Generic FEC February 26, 1999 8 Protection Operation The protection operation involves taking a variety of fields from the various headers, appending the payloads, appending the whole thing together, padding zeroes, and then computing the FEC across the resulting binary block. The result is then placed into the FEC packet. The following procedure MUST be followed for the protection opera- tion. For each media packet to be protected, a binary array is gen- erated by appending the following fields together: o Padding Bit (1 bit) o Extension Bit (1 bit) o CC bits (3 bits) o Marker bit (1 bit) o Payload Type (7 bits) o Timestamp (32 bits) o Natural binary representation of the length of the CSRC List plus padding plus payload plus extension of the media packet (16 bits) o CSRC List (if CC is 1), else null (variable) o Header Extension (if X is 1), else null (variable) o the payload (variable) o Padding, if present (variable) Note that the padding bit forms the most significant bit of the binary array. If the lengths of the binary arrays are not equal, the arrays MUST be padded to be of equal length. Padding to the length of the longest array is RECOMMENDED, but longer lengths MAY be used if additional padding is desired. Any value for the pad MAY be used. The parity operation is then applied across the binary arrays. The result is the binary array of the FEC packet. The first (most significant) bit in the FEC packet binary array is J.Rosenberg,H.Schulzrinne [Page 8]
Internet Draft Generic FEC February 26, 1999 written into the Padding Bit of the FEC packet. The second bit in the FEC packet binary array is written into the Extension bit of the FEC packet. The next three bits of the FEC packet binary array are writ- ten into the CC field of the FEC packet. The next bit of the FEC packet binary array is written into the marker bit of the FEC packet. The next 7 bits of the FEC packet binary array are written into the PT recovery field in the FEC packet header. The next 32 bits of the FEC packet binary array are written into the TS recovery field in the packet header. The next 16 bits are written into the Length Recovery field in the FEC packet header. The remaining bits are set to be the payload of the FEC packet. 9 Recovery Procedures The FEC packets allow end systems to recover from the loss of media packets. All of the header fields of the missing packets, including CSRC lists, extensions, padding bits, marker and payload type, are recoverable. This section describes the procedure for performing this recovery. Recovery requires two distinct operations. The first determines which packets (media and FEC) must be combined in order to recover a miss- ing packet. Once this is done, the second step is to actually recon- struct the data. The second step MUST be performed as described below. The first step MAY be based on any algorithm chosen by the implementor. Different algorithms result in a tradeoff between com- plexity and the ability to recover missing packets if at all possi- ble. 9.1 Reconstruction Let T be the list of packets (FEC and media) which can be combined to recover some media packet xi. For parity, this is an FEC packet plus all but one of the media packets associated with the FEC packet. The procedure is as follows: 1. For the media packets in T, compute the binary array as described in the protection operation of the previous sec- tion. 2. For the FEC packet in T, compute the binary array in the same fashion, except always set the CSRC list, extension, and padding to null. 3. If the resulting binary arrays are not of equal length, pad them with zeroes to be the length of the longest binary array. J.Rosenberg,H.Schulzrinne [Page 9]
Internet Draft Generic FEC February 26, 1999 4. Perform the exclusive or (parity) operation across the binary arrays, resulting in a recovery array. 5. Create a new packet with the standard 12 byte RTP header and no payload. 6. Set the version of the new packet to 2. 7. Set the Padding bit in the new packet to the first bit in the recovery array. 8. Set the Extension bit in the new packet to the second bit in the recovery array. 9. Set the CC field to the next three bits in the recovery array. 10. Set the marker bit in the new packet to the next bit in the recovery array. 11. Set the payload type in the new packet to the next 7 bits in the recovery array. 12. Set the SN field in the new packet to xi. 13. Set the TS field in the new packet to the next 32 bits in the recovery array. 14. Take the next 16 bits of the recovery array. Whatever the natural binary number this corresponds to, take that many bytes from the recovery array and append them to the new packet. This represents the CSRC list, extension, payload, and padding. 15. Set the SSRC of the new packet to the SSRC of the media stream its protecting. This procedure will completely recover both the header and payload of an RTP packet. 9.2 Determination of when to recover The previous section discussed how to recover a media packet with sequence number xi when all of the packets needed to recover it were available. The decision about whether to attempt recovery of some media packet xi, and how to determine if sufficient data is available to recover it, is left to the implementor. However, this section pro- vides a simple algorithm which MAY be used for this purpose. J.Rosenberg,H.Schulzrinne [Page 10]
Internet Draft Generic FEC February 26, 1999 The algorithm is described below in C code. The code assumes that several functions exist. recover_packet() takes the sequence number of a packet, and an FEC packet. Using the FEC packet and data packets received previously, the data packet with the given sequence number is recovered. add_fec_to_pending_list() adds the given FEC packet to a linked list of FEC packets which have not yet been used for recovery. wait_for_packet() waits for a packet, FEC or data, from the network. remove_from_pending_list() removes the FEC packet from the pending list. The structure packet contains a boolean variable fec which is true when the packet is FEC, false if its media. When its an FEC packet, the mask and snbase field contain those values from the FEC packet header. When its a media packet, the sn variable contains the sequence number of the packet. The global array A indicates which media packets have been received, and which have not. It is indexed by the sequence number of the packet. The function fec_recovery implements the algorithm. It waits for packets, and when it receives an FEC packet, calls recover_with_fec() to attempt to use it to recover. If no recovery is possible, the FEC packet is stored for later attempts. If the received packet was a media packet, its presence is noted, and any old FEC packets are checked to see if recovery is now possible. Recovered packets are treated as if they were received, triggering further attempts at recovery. A real implementation will need to use a circular buffer instead of the simple array (A in the code) in order to avoid running off the end of the buffer. typedef struct packet_s { BOOLEAN fec; /* FEC or media */ int sn; /* SN of the packet, for media only */ BOOLEAN mask[24]; /* Mask, FEC only */ int snbase; /* SN Base, FEC only */ struct packet_s *next; } packet; BOOLEAN A[65535]; packet *pending_list; J.Rosenberg,H.Schulzrinne [Page 11]
Internet Draft Generic FEC February 26, 1999 packet *recover_with_fec(packet *fec_pkt) { packet *data_pkt; int pkts_present, /* number of packets from the mask that are present */ pkts_needed, /* number of packets needed is the number of ones in the mask minus 1 */ pkt_to_recover, /* sn of the packet we are recovering */ i; pkts_present = 0; /* The number of packets needed is the number of ones in the mask minus 1. The code below increments pkts_needed by the number of ones in the mask, so we initialize this to -1 so that the final count is correct */ pkts_needed = -1; /* Go through all 24 bits in the mask, and check if we have all but one of the media packets */ for(i = 0; i < 24; i++) { /* If the packet is here and in the mask, increment counter */ if(A[i+fec_pkt->snbase] && fec_pkt->mask[i]) pkts_present++; /* Count the number of packets needed as well */ if(fec_pkt->mask[i]) pkts_needed++; /* The packet to recover is the one with a bit in the mask thats not here yet */ if(!A[i+fec_pkt->snbase] && fec_pkt->mask[i]) pkt_to_recover = i+fec_pkt->snbase; } /* If we can recover, do so. Otherwise, return NULL */ if(pkts_present == pkts_needed) { data_pkt = recover_packet(pkt_to_recover, fec_pkt); free(fec_pkt); } else { data_pkt = NULL; } return(data_pkt); } J.Rosenberg,H.Schulzrinne [Page 12]
Internet Draft Generic FEC February 26, 1999 void fec_recovery() { packet *p, /* packet received or regenerated */ *fecp, /* fec packet from pending list */ *pnew; /* new packets recovered */ while(1) { p = wait_for_packet(); /* get packet from network */ while(p) { /* if its an FEC packet, try to recover with it. If we can't, store it for later potential use. If we can recover, act as if the recovered packet is received and try to recover some more. Otherwise, if its a data packet, mark it as received, and check if we can now recover a data packet with the list of pending FEC packets */ if(p->fec == TRUE) { pnew = recover_with_fec(p); if(pnew) A[pnew->sn] = TRUE; else add_fec_to_pending_list(p); /* We assign pnew to p since the while loop will continue to recover based on p not being NULL */ p = pnew; } else { /* Mark this data packet as here */ A[p->sn] = TRUE; free(p); p = NULL; /* Go through pending list. Try and recover a packet using each FEC. If we are successful, add the data packet to the list of received packets, remove the FEC packet from the pending list, since we've used it, and then try to recover some more */ for(fecp = pending_list; fecp != NULL; fecp = fecp->next) { pnew = recover_with_fec(fecp); if(pnew) { J.Rosenberg,H.Schulzrinne [Page 13]
Internet Draft Generic FEC February 26, 1999 /* The packet is now here, as we've recovered it */ A[pnew->sn] = TRUE; /* One FEC packet can only be used once to recover, so remove it from the pending list */ remove_fec_from_pending_list(fecp); p = pnew; break; } } /*for*/ } /*p->fec was false */ } /* while p*/ } /* while 1 */ } 10 Example Consider 2 media packets to be sent, x and y, from SSRC 2. Their sequence numbers are 8 and 9, respectively, with timestamps of 3 and 5, respectively. Packet x uses payload type 11, and packet x uses payload type 18. Packet x is has 10 bytes of payload, and packet y 11. Packet y has its marker bit set. The RTP headers for packets x and y are shown in Figures 3 and 4 respectively. An FEC packet is generated from these two. We assume that payload type 127 is used to indicate an FEC packet. The resulting RTP header is shown in Figure 5 J.Rosenberg,H.Schulzrinne [Page 14]
Internet Draft Generic FEC February 26, 1999 Figure 3: Packet X Media Packet x 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |1 0|0|0|0 0 0 0|0|0 0 0 1 0 1 1|0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Version: 2 Padding: 0 Extension: 0 Marker: 0 PTI: 11 SN: 8 TS: 3 SSRC: 2 Figure 4: Packet Y Media Packet y 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |1 0|0|0|0 0 0 0|1|0 0 1 0 0 1 0|0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 1| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 1| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Version: 2 Padding: 0 Extension: 0 Marker: 1 PTI: 18 SN: 9 TS: 5 SSRC: 2 Figure 5: RTP Header of Result J.Rosenberg,H.Schulzrinne [Page 15]
Internet Draft Generic FEC February 26, 1999 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |1 0|0|0|0 0 0 0|1|1 1 1 1 1 1 1|0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 1| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Version: 2 Padding: 0 Extension: 0 Marker: 1 PTI: 127 SN: 1 TS: 5 SSRC: 2 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0|0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |0|0 0 1 1 0 0 1|0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 0| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ SN base: 8 [min(8,9)] len. rec.: 1 [8 xor 9] E: 0 PTI rec.: 24 [11 xor 18] mask: 3 TS rec.: 6 [3 xor 5] The payload length is 11 bytes. Figure 6: FEC Header of Result 11 Use with Redundant Encodings J.Rosenberg,H.Schulzrinne [Page 16]
Internet Draft Generic FEC February 26, 1999 One can consider an FEC packet as a "redundant coding" of the media. Because of this, the payload format for encoding of redundant audio data [5] can be used to carry the FEC data along with the media. The procedure for this is as follows. The FEC operation defined above acts on a stream of RTP media pack- ets. The stream which is operated on is the stream before the encap- sulation defined in rfc2198. In other words, the media stream to be protected is encapsulated in standard RTP media packets. The FEC operation above is performed, generating a stream of FEC packets. Then, the media payload is extracted from the media packets. This payload is used as the primary encoding as defined in rfc2198. Then, the FEC header and payload of the FEC packets is extracted, and treated as a redundant encoding. Additional redundant encodings, besides FEC, MAY be added to the packet as well. These encodings will not be protected by FEC, however. The redundant encodings header for the primary codec is set as defined in rfc2198. The redundant encodings header for the FEC data is set as follows. The block PT is set to the dynamic PT associated with the FEC format. The block length is set to the length of the FEC header and payload. The timestamp offset SHOULD be set to zero. The secondary coder payload includes the FEC header and FEC payload. At the receiver, the primary codec and all secondary codecs are extracted as separate RTP packets. This is done by copying the sequence number, SSRC, marker bit, CC field, RTP version, and exten- sion bit from the RTP header of the redundant encodings packet to the RTP header of each extracted packet. The payload type identifier in the extracted packet is copied from the block PT of the redundant encodings header. The timestamp of the extracted packet is the difference between the timestamp in the RTP header and the offset in the block header. The payload of the extracted packet is the data block. This will result in the FEC stream being extracted. Then, using the recovery procedures defined above, lost primary-encoding packets can be recovered. Using the redundant encodings payload format implies that the marker bit will not be reconstructed correctly. Applications SHOULD set the marker bit to zero in packets reconstructed with FEC encapsulated in rfc2198 redundancy. An advantage of this approach is a reduction in the overhead for sending FEC packets. 12 Signaling FEC in SDP FEC packets contain RTP packets with dynamic payload type values. In J.Rosenberg,H.Schulzrinne [Page 17]
Internet Draft Generic FEC February 26, 1999 addition, the FEC packets can be sent on separate multicast groups or separate ports from the media. The FEC can even be carried in packets containing media, using the redundant encodings payload format [5]. These configuration options must be signaled out of band. This sec- tion describes how this is accomplished using the Session Description Protocol (SDP) [6]. 12.1 FEC as a separate stream In the first case, the FEC packets are sent as a separate stream. This can mean they are sent on a different port or multicast group from the media. When this is done, several pieces of information must be conveyed: o The address and port where the FEC is being sent to o The payload type number for the FEC o Which media stream the FEC is protecting Since the FEC is a separate stream, it is indicated by means of a separate m= line in the SDP. The media type for this stream MUST be the same as the media type of the stream it protects. The m line is otherwise formatted as defined in rfc2327. For example, if the FEC stream is sent to port 3443 with payload type identifier 3483, and is protecting an audio stream, the m line would be: m=audio 3443 RTP/AVP 3483 The payload type number 3483 is bound to the payload type format defined here by means of an rtpmap attribute. The encoding name of the parity FEC format defined here is "parityfec". The rtpmap attri- bute MUST be included in the encoding media stream description. Encoding parameters MUST NOT be specified. The rtpmap attribute that would be used in the above example is: a=rtpmap:3483 parityfec/8000 As with any other media description - media titles, connection infor- mation, bandwidth information, and media keys MAY be specified within J.Rosenberg,H.Schulzrinne [Page 18]
Internet Draft Generic FEC February 26, 1999 it. The rules for their inclusion, and their syntax and semantics, are the same as those defined in RFC2327. In order to determine which media stream the encoding stream is pro- tecting, a new attribute is defined here, called the tag. The tag MUST be an integer between 0 and 65535. It MUST be included within the media desription of a stream that is protected by FEC, and MAY be included in other media descriptions. The tag is used so that the FEC stream can indicate which media stream it protects. An example tag attribute is: m=audio 3456 RTP/AVP 0 1 2 a=tag:88 To associate a tagged media stream with the FEC stream, the fmtp attribute is used. For FEC, the format of the fmtp attribute is: a=fmtp:<format> <tag> The format is the payload type associated with the encoding stream. The tag is the numeric value of the tag of the media stream which is being protected by FEC. Only one media stream can be protected by a single FEC stream. This attribute MUST be included in the media description of the FEC stream. An example SDP description using these attributes is: v=0 o=mhandley 2890844526 2890842807 IN IP4 126.16.64.4 s=SDP Seminar c=IN IP4 224.2.17.12/127 t=2873397496 2873404696 m=audio 49170 RTP/AVP 0 a:tag=0 m=audio 49171 RTP/AVP 0 a:tag=1 m=video 51372 RTP/AVP 31 m=audio 49172 RTP/AVP 298 c=IN IP4 224.2.17.13/127 a=rtpmap:298 parityfec/8000 a=fmtp:298 1 J.Rosenberg,H.Schulzrinne [Page 19]
Internet Draft Generic FEC February 26, 1999 This specifies an mbone session where the media (two audio streams and a video stream) are sent to 224.2.17.12 with a TTL of 127. An FEC stream is defined, sent to a different multicast group - 224.2.17.13, also with a TTL of 17. The FEC stream is formatted according to this specification, and assigned a dynamic payload type of 298. The fmtp attribute indicates that the FEC is protecting the second media stream (the one sent to port 49171). 12.2 Use with redundant encodings When the FEC stream is being sent as a secondary codec in the redun- dant encodings format, this must be signaled through SDP. To do this, the procedures defined in RFC2198 are used to signal the use of redundant encodings. The FEC payload type is indicated in the same fashion as any other secondary codec. An rtpmap attribute MUST be used to indicate a dynamic payload type for the FEC packets. The FEC MUST protect only the main codec. For example: m=audio 12345 RTP/AVP 121 0 5 1092 a=rtpmap:121 red/8000/1 a=rtpmap:1092 parityfec/8000 a=fmtp:121 0/5/1092 This indicates that there is a single audio stream, which can consist of PCM, DVI, the redudant encodings, or FEC. Although the FEC format is specified as a possible coding for this stream, the FEC MUST NOT be sent by itself for this stream. Its presence in the m line is required only because non-primary codecs must be listed here accord- ing to RFC2198. The fmtp attribute indicates the the redundant encod- ings format can be used, with DVI as a secondary coding and FEC as a tertiary encoding. 13 Security Considerations The use of FEC has implications on the usage and changing of keys for encryption. As the FEC packets do consist of a separate stream, there are a number of permutations on the usage of encryption. In particu- lar: o The FEC stream may be encrypted, while the media stream is not. o The media stream may be encrypted, while the FEC stream is J.Rosenberg,H.Schulzrinne [Page 20]
Internet Draft Generic FEC February 26, 1999 not. o The media stream and FEC stream are both encrypted, but using different keys. o The media stream and FEC stream are both encrypted, but using the same key. The first three of these would require any application level signal- ing protocols to be aware of the usage of FEC, and to thus exchange keys for it and negotiate its usage on the media and FEC streams separately. In the final case, no such additional mechanisms are needed. Applications utilizing encryption SHOULD encrypt both streams, however. Encrypting just one may make certain known- plaintext attacks possible. However, the changing of keys becomes problematic. For example, if two packets a and b are sent, and FEC packet f(a,b) is sent, and the keys used for a and b are different, which key should be used to decode f(a,b)? In general, old keys will likely need to be cached, so that when the keys change for the media stream, the old key is kept, and used, until it is determined that the key has changed on the FEC packets as well. Another issue with the use of FEC is its impact on network conges- tion. Adding FEC in the face of increasing network losses is a bad idea, as it can lead to increased congestion and eventual congestion collapse if done on a widespread basis. As a result, implementors MUST NOT substantially increase the amount of FEC in use as network losses increase. 14 Acknowledgments The authors would like to thank Budge and Mackenzie, who submitted the initial draft on FEC, and upon which this work is based. We would also like to thank Steve Casner, Orion Hodson and Colin Perkins for their comments. 15 Author's Addresses Jonathan Rosenberg Lucent Technologies, Bell Laboratories 101 Crawfords Corner Rd. Holmdel, NJ 07733 Rm. 4C-526 email: jdrosen@bell-labs.com J.Rosenberg,H.Schulzrinne [Page 21]
Internet Draft Generic FEC February 26, 1999 Henning Schulzrinne Columbia University M/S 0401 1214 Amsterdam Ave. New York, NY 10027-7003 email: schulzrinne@cs.columbia.edu 16 Bibliography [1] J.-C. Bolot and A. Garcia, "The case for fec-based error control for packet audio in the internet," Multimedia Systems , 1997. [2] C. Perkins and O. Hodson, "Options for repair of streaming media," Request for Comments (Informational) 2354, Internet Engineer- ing Task Force, June 1998. [3] H. Schulzrinne, S. Casner, R. Frederick, and V. Jacobson, "RTP: a transport protocol for real-time applications," Request for Comments (Proposed Standard) 1889, Internet Engineering Task Force, Jan. 1996. [4] S. Bradner, "Key words for use in RFCs to indicate requirement levels," BC 2119, Internet Engineering Task Force, Mar. 1997. [5] C. Perkins, I. Kouvelas, V. Hardman, M. Handley, and J. Bolot, "RTP payload for redundant audio data," Request for Comments (Pro- posed Standard) 2198, Internet Engineering Task Force, Sept. 1997. [6] M. Handley and V. Jacobson, "SDP: session description protocol," Request for Comments (Proposed Standard) 2327, Internet Engineering Task Force, Apr. 1998. J.Rosenberg,H.Schulzrinne [Page 22]
