Internet-Draft M. Pullen draft-pullen-srmp-06.txt B. Shanmugam Expires: December 2005 F. Zhao George Mason Univ D. Cohen Sun Microsystems June 2005 Selectively Reliable Multicast Protocol (SRMP) Status of this Memo This document is an Internet-Draft and is in full conformance with all provisions of Section 10 of RFC2026. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. By submitting this Internet-Draft, each author represents that any applicable patent or other IPR claims of which he or she is aware have been or will be disclosed, and any of which he or she becomes aware will be disclosed, in accordance with Section 6 of BCP 79. Copyright Notice Copyright (C) The Internet Society (2005). All Rights Reserved. Abstract The Selectively Reliable Multicast Protocol (SRMP) is a transport protocol, intended to deliver a mix of reliable and best-effort messages in an any-to-any multicast environment, where the best- effort traffic occurs in significantly greater volume than the reliable traffic and therefore can be used to carry sequence numbers of reliable messages for loss detection. SRMP is intended Pullen Expires: December 2005 [Page 1]
Internet Draft draft-pullen-srmp-06.txt November 2005 for use in a distributed simulation application environment, where only the latest value of reliable transmission for any particular data identifier requires delivery. SRMP has two sublayers: a bundling sublayer that combines short messages, peforms NAK suppression, and incorporates the TCP-Friendly Multicast Congestion Control of Widmer and Handley, dropping best- effort traffic in order to achieve congestion control; and a selectively reliable transport (SRT) sublayer that formats best- effort and reliable transmissions and also creates negative acknowledgements when loss of reliable messages is detected. In SRMP, selection between reliable and best-effort messages is performed by the application. The protocol bundles messages within a configured time interval, attempting to achieve a configured bundle size which typically is set to the expected smallest MTU in the delivery path. The bundle header carries the latest sequence number for each reliable transmission. Rate reduction is achieved when necessary by dropping best-effort messages at random. Table of Contents 1. Introduction 3 1.1 Terminology 4 2. Protocol description 4 3. Message formats 6 3.1 Bundle packet format 6 3.2 bundle header format 7 3.3 feedback message format 9 3.4 SRT Mode 0 header format 10 3.5 SRT Mode 1 header format 10 3.6 SRT Mode 2 header format 11 3.7 SRT NACK format 11 3.8 User-configurable parameters 12 4. TFMCC Operation 12 4.1 TCP rate prediction equation 12 4.2 Bundling 13 4.3 Congestion Control 13 4.4 Any-Source Multicast 13 4.5 Multiple Sources Issue 14 4.6 Bundle Size 14 4.7 Data rate control 14 4.8 Mode 1 loss detection 14 4.8.1 Sending a Negative Acknowledgement 15 4.9 Unbundling 15 4.10 Heartbeat bundle 16 5. SRT Operation 16 5.1 Mode 0 operation 16 5.1.1 Sending Mode 0 messages 17 5.1.2 Receiving Mode 0 Messages 17 Pullen Expires: December 2005 [Page 2]
Internet Draft draft-pullen-srmp-06.txt June 2005 5.2 Mode 1 operation 17 5.2.1 Sending Mode 1 Data messages 17 5.2.2 Receiving Mode 1 Data messages 18 5.2.3 Scheduling a Negative Acknowledgement 18 5.2.4 Receiving a Negative Acknowledgement 19 5.3 Mode 2 Operation 20 5.3.1 Sending Mode 2 Data messages 20 5.3.2 Receiving Mode 2 Data messages 21 5.3.3 Sending a Positive Acknowledgement 21 5.3.4 Receiving a Positive Acknowledgement 22 6. RFC 2357 Analysis 22 6.1 Scalability 22 6.2 Congestion 22 7. IANA considerations 23 8. Security considerations 23 9. List of Acronyms 25 10. Authors' addresses 25 11. References 26 12. Full copyright statement 27 1. Introduction There is no viable generic approach to achieving reliable transport over multicast networks. Existing successful approaches require that the transport protocol take advantage of special properties of the traffic in a way originally proposed by Cohen. The protocol described here is applicable to real-time traffic containing a mix of two categories of messages: a small fraction requiring reliable delivery, mixed with a predominating flow of best-effort messages. This sort of traffic is associated with distributed virtual simulation (RFC 2502 [9]) and also with some forms of distributed multimedia conferencing. These applications typically have some data that changes rarely, or not at all, so the best efficiency will be achieved by transmitting that data reliably (the external appearance of a simulated vehicle is an excellent example). They also require real-time transmission of a best-effort stream (for example the position and orientation of the vehicle). There is no value to reliable transmission of this stream because typically new updates arrive faster than loss identification and retransmission could take place. By piggy-backing the sequence number (SN) of the latest reliable transmission on each bundle of traffic, the reliable and best-effort traffic can co-exist synergistically. This approach is implemented in the Selectively Reliable Multicast transport Protocol (SRMP). The IETF has conducted a successful working group on Reliable Multicast Transport (RMT) that has produced RFCs 2357, 2887, and 3450 through 3453 which define building block protocols for reliable multicast. Selectively reliable multicast is similar in spirit to these protocols and in fact uses one of them, TCP Friendly Multicast Congestion Control (TFMCC). This document provides the basis for Pullen Expires: December 2005 [Page 3]
Internet Draft draft-pullen-srmp-06.txt June 2005 specifying SRMP with TFMCC for use on an experimental basis. Key requirements of the RMT process that is carried forward here are specified in RFC 2357 [3]. These generally relate to scalability and congestion control, and are addressed in section 6 of this document. 1.1 Terminology In this document, the key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" are to be interpreted as described in RFC 2119 [7] and indicate requirement levels for compliant implementations. 2. Protocol Description The Selectively Reliable Multicast Protocol (SRMP) has two major components: Selectively Reliable Transport (SRT) and a "bundling sublayer" that implements TCP-Friendly Multicast Congestion Control (TFMCC), as proposed by Widmer and Handley [8], in order to meet the requirements of RFC3269 [3] for congestion avoidance. SRMP is capable of reliable message delivery over multicast networks, when the messages to be delivered reliably represent a fraction of a larger, associated best-effort flow and only the latest reliable message must be delivered. The basic strategy of SRMP is trade as little network capacity as possible for reliability by buffering the latest sent reliable message at each sender and piggybacking its sequence number on associated best- effort messages. For this purpose, three modes of sending are defined: o Mode 0 messages. These will be delivered best-effort; if lost, no retransmission will be done. o Mode 1 messages. When a Mode 1 message loss is detected, the receiver will send back a NACK to the sender, where SRMP will retransmit the latest reliable message from that sender. Senders define data identifiers (DataIDs), allowing multiple reliable message streams to be supported. Mode 1 messages may be up to 131,071 bytes long; SRMP provides for segmentation and reassembly, but only for the latest Mode 1 message for any given <sourceAddress, multicastAddress, DataID>. o Mode 2 messages. Through Mode 2 messages, SRMP provides for a lightweight, reliable, connectionless peer-to-peer unicast transaction exchange between any two members of the multicast group. This is a unicast message requiring positive acknowledgement (ACK). Pullen Expires: December 2005 [Page 4]
Internet Draft draft-pullen-srmp-06.txt June 2005 | Application | ----------------- ---------- | SRT | ----------------- -> SRMP |Bundling(TFMCC)| ----------------- ---------- | UDP | The Bundling sublayer is transparent to the Selectively Reliable Transport (SRT) sublayer. It implements congestion control both by dropping Mode 0 messages at the source when needed, and also by bundling multiple short messages that are presented by applications within a short time window. It also performs NACK suppression. A bundling sublayer data unit is called a bundle. A bundle is made up of a bundle header and any of Mode 0 and 1 SRMP messages. Retransmission of Mode 1 messages does not imply retransmission of the original bundle; the retransmitted message becomes part of a new bundle. The TFMCC layer's behavior follows the mechanism described by Widmer and Handley. This is an equation-based multicast congestion control mechanism: in a multicast group, each receiver determines its loss rate with regard to the sender, and calculates a desired source sending rate based on an equation that provides that models the steady-state sending rate of TCP. A distributed feedback suppression mechanism restricts feedback to those receivers likely to report the lowest desired rates. Congestion control is achieved by dropping best-effort (Mode 0) messages at random. For example, in distributed simulation Mode 0 messages are part of a stream of state update for dynamic data such as geographic location; therefore the application can continue to function (with lower fidelity) when they are dropped. As described by its authors, TFMCC's congestion control mechanism works as follows: o Each receiver measures the loss event rate and its RTT to the sender. o Each receiver then uses this information, together with an equation for TCP throughput, to derive a TCP-friendly sending rate. o Through a distributed feedback suppression mechanism, only a subset of the receivers are allowed to give feedback to prevent a feedback implosion at the sender. The feedback mechanism ensures that receivers reporting a low desired transmission rate have a high probability of sending feedback. o Receivers whose feedback is not suppressed report the calculated transmission rate back to the sender in so-called receiver reports. The receiver reports serve two purposes: they inform the sender about the appropriate transmit rate, and they allow the receivers to measure their RTT. Pullen Expires: December 2005 [Page 5]
Internet Draft draft-pullen-srmp-06.txt June 2005 o The sender selects the receiver that reports the lowest rate as current limiting receiver (CLR). Whenever feedback with an even lower rate reaches the sender, the corresponding receiver becomes CLR and the sending rate is reduced to match that receiver's calculated rate. The sending rate increases when the CLR reports a calculated rate higher than the current sending rate. TFMCC was intended for fixed-size packets with variable rate. SRMP applies it to variable-size SRMP messages that are mostly the same size because the best-effort updates typically all represent the same sort of simulation information and are grouped into bundles of size just under one MTU during periods of heavy netwrok activity. Future developments in TFMCC for variable-size messages will be of high value for inclusion in SRMP if, as expected, they prove to be appropriate for the types of traffic SRMP is intended to support. SRMP is intended for general use under applications that needs its services and may exist in parallel instances on the same host. The UDP port is therefore established ad hoc from available application ports, thus it would not be appropriate to have a well-known port for SRMP. 3. Message formats 3.1 Bundle message format: -------------------------------------------------------------------- | bundle header | SRT Message 0 | SRT message 1 | SRT message 2 |... -------------------------------------------------------------------- A bundle is an aggregation of multiple SRMP messages fo the same multicast address. A bundle can contain only Mode 0 and Mode 1 messages; Mode 2 messages are exchanged using unicast addresses. SRMP identifies sender and receiver using their 32-bit Sender_ID, which may be an IPv4 address. For use with IPv6, a user group will need to establish a unique identifier per host. There is no requirement for this identifier to be unique in the Internet; it need only be unique in the communicating group. Pullen Expires: December 2005 [Page 6]
Internet Draft draft-pullen-srmp-06.txt June 2005 3.2 bundle header format: 0 8 16 24 32 +--------------+--------------+--------------+--------------+ |Version| 000 |fb_nr | flag | bundle_SN | +--------------+--------------+--------------+--------------+ | Sender_ID | +--------------+--------------+--------------+--------------+ | Receiver_ID | +--------------+--------------+--------------+--------------+ | Sender_Timestamp | Receiver_Timestamp | +--------------+--------------+--------------+--------------+ | x_supp | R_max | +--------------+--------------+--------------+--------------+ | DSN_count | padding | Length | +--------------+--------------+--------------+--------------+ | 0 to 255 DSN: <DataID, SN, NoSegs> of this sender | +-----------------------------------------------------------+ Version: 4 bits currently 0010 Type: 4 bits 0000 - indicates bundle fb_nr: 4 bits feedback round, range 0 - 15 flag: 4 bits 0001 Is_CLR other bits reserved bundle_SN: 16 bits range 0-65535 Sender_Timestamp: 16 bits Representing the time that the bundle was sent out (in milliseconds) based on the sender's local clock. Receiver_Timestamp: 16 bits Echo of the Receiver_Time_Stamp field (in milliseconds) of the receiver feedback message. If the sender has time delay between receiving the feedback and echoing the timestamp, it MUST adjust the Receiver_Timestamp value to compensate. Receiver_ID 32 bits Unique identifier for the receiver within the multicast group. IPv4 address may be used. Pullen Expires: December 2005 [Page 7]
Internet Draft draft-pullen-srmp-06.txt June 2005 Sender_ID: 32 bits Unique identifier for the sender within the multicast group. IPv4 address may be used. X_supp: 16 bits The suppression rate corresponding with the sender, in bits/s. Only those receivers whose desired rate is less than the suppression rate, or RTT larger than R_max, may send feedback information to the sender. The suppression rate is represented as a 16 bit floating point value with 8 bits for the unsigned exponent and 8 bits for the unsigned mantissa. R_max: 16 bits The maximum of the RTTs of all receivers, in milliseconds. The Maximum RTT should be represented as a 16-bit floating point value with 8 bits for the unsigned exponent and 8 bits for the unsigned mantissa. DSN_count: 8 bits The count of DSN blocks following the header. Length: 16 bits Range from 0~65535. The total length of the bundle (including header). DSN: 32 bits There can be up to 256 of these in a header. An SRMP implementation MUST support a minimum of 1. Each DSN consists of three fields: DataID: 16 bits A unique number associated with a particular data element on the sending host, used to identify a Mode 1 message SN: 9 bits Sequence Number associated with a particular Mode 1 transmission of a particular DataID. NoSegs: 7 bits Number of segments, if the DataID was long enough to require segmentation; otherwise 0x0. Pullen Expires: December 2005 [Page 8]
Internet Draft draft-pullen-srmp-06.txt June 2005 3.3 feedback message format 0 8 16 24 32 +--------------+--------------+--------------+--------------+ |Version| 001 | fb_nr| flag | X_r | +--------------+--------------+--------------+--------------+ | Sender_Timestamp | Receiver_Timestamp | +--------------+--------------+--------------+--------------+ | Sender_ID | +--------------+--------------+--------------+--------------+ | Receiver_ID | +--------------+--------------+--------------+--------------+ Version: 4 bits currently 0010 Type: 4 bits value 0001 fb_nr: 4 bits current feedback round of the sender. flag: 4 bits 0001 - have_RTT 0010 - have_loss 0100 - receiver_leave other values reserved X_r: 16 bits desired sending rate X_r in bits/s, calculated by the receiver to be TCP-friendly Sender_Timestamp: 16 bits echo of the Sender_Timestamp in bundle header. If the receiver has time delay between receiving the bundle and echoing the timestamp, it MUST adjust the Sender_Timestamp value correspondently. Receiver_Timestamp: 16 bits the time when the feedback message was sent out from the receiver. Receiver_ID: 32 bits Unique identifier for the receiver within the multicast group. IPv4 address may be used. (identifies the receiver that sends the feedback message) Sender_ID: 32 bits Unique identifier for the sender within the multicast group. IPv4 address may be used. (identifies the sender that is the destination of the current feedback message) Pullen Expires: December 2005 [Page 9]
Internet Draft draft-pullen-srmp-06.txt June 2005 X_r: 12 bits the receiver's desired rate. The desired rate should be represented as a 12 bits floating point value with 5 bits for the unsigned exponent and 7 bits for the unsigned mantissa. 3.4 SRT Mode 0 header format 0 8 16 24 32 +--------------+--------------+--------------+--------------+ |Version| 0000 | 000 | 00000000 | Length | +--------------+--------------+--------------+--------------+ Version: 4 bits currently 0010 Type: 4 bits 0000 Mode: 3 bits 000 Padding 8 bits 00000000 Length: 11 bits Length of the payload data (does not include header) 3.5 SRT Mode 1 header format 0 8 16 24 32 +--------------+--------------+--------------+--------------+ |Version| 0010 | 001 | SegNo | Length | +--------------+--------------+--------------+--------------+ | DSN | +--------------+--------------+--------------+--------------+ Version: 4 bits currently 0010 Type: 4 bits 0000 Mode: 3 bits 001 SegNo: 7 bits The index number of this segment Length: 14 bits Length of the payload data (does not include header) Pullen Expires: December 2005 [Page 10]
Internet Draft draft-pullen-srmp-06.txt June 2005 DSN: 32 bits Same as in bundle header. Note that this contains NoSegs, whereas SegNo is a separate element. 3.6 SRT Mode 2 header format 0 8 16 24 32 +--------------+--------------+--------------+--------------+ |Version| 0010 |010 | 00000 | Length | +--------------+--------------+--------------+--------------+ | SN | +--------------+--------------+--------------+--------------+ Version: 4 bits currently 0010 Type: 4 bits 0010 Mode: 3 bits 010 padding: 5 bits 00000 Length: 16 bits Length of the payload data (does not include header) SN: 32 bits Same as in bundle header. 3.7 SRT NACK format 0 8 16 24 32 +--------------+--------------+--------------+--------------+ |Version| 0010 |111 | 00000 | reserved | +--------------+--------------+--------------+--------------+ | DSN | +--------------+--------------+--------------+--------------+ | Sender Address | +--------------+--------------+--------------+--------------+ Version: 4 bits currently 0010 Type: 4 bits 0010 Mode: 3 bits 111 Pullen Expires: December 2005 [Page 11]
Internet Draft draft-pullen-srmp-06.txt June 2005 padding: 5 bits 00000 reserved 16 bits DSN: 32 bits sequence number Sender Address: The IP address of the sender of message being NACKed 3.8 User-configurable parameters Name Minimum Value Recommended Value Units DSN_Max 1 32 messages DataID_Timeout none none ms Segment_Timeout 50 250 ms Bundle_Timeout 1 10 ms Heartbeat_Interval 1 none s Mode2_Max 1 none messages ACK_Threshold none worst RTT in group ms 4 TFMCC Operation 4.1 TCP rate prediction equation for TFMCC The RECOMMENDED throughput equation for SRMP is a slightly simplified version of the throughput equation for Reno TCP from [2]: 8 s X = ------------------------------------------------------ (1) R * (sqrt(2*p/3) + (3*sqrt(6*p) * p * (1+32*p^2))) (the formula may be simplified for implementation), where X is the transmit rate in bits/second. s is the message size in bytes. R is the round-trip time in seconds. p is the loss event rate, between 0.0 and 1.0, of the number of loss events as a fraction of the number of messages transmitted. In the future, different TCP formulas may be substituted for this equation. The requirement is that the throughput equation be a reasonable approximation of the sending rate of TCP for conformant TCP congestion control. Pullen Expires: December 2005 [Page 12]
Internet Draft draft-pullen-srmp-06.txt June 2005 4.2 Bundling Multiple SRMP messages will be encapsulated into a bundle. When a new SRMP message (mode 0 or mode 1) arrives, the SRMP daemon will try to add the new message into the current bundle. The SRMP daemon should keep a timer, which will be reset when the first SRMP message is added into the bundle. After Bundle_Timeout, the timer will time out, and the current bundle should be transmitted immediately. A new bundle will then be initialized to hold new SRMP messages. Bundle_Timeout SHALL not be less than 1 ms. Recommended value is 10 ms. Also, the bundle length MUST not exceed LENGTH_MAX. If adding a new SRMP message will produce a greater length, the SRMP daemon MUST initialize a new bundle for the new SRMP messages, and the current bundle should be transmitted immediately. Recommended value for LENGTH_MAX is 1454 bytes (Ethernet MTU minus IP and UDP header lengths). In a bundle, there may exist multiple SRMP messages with the same DataID. In this case, only the latest version of that DataID is useful. SRMP may check for duplicate DataIDs in the same bundle and delete all but the latest one. If a mode 1 message appears in the outgoing bundle, then the corresponding DSN should not appear in the bundle header. The bundle header contains the DSN <DataID,SN,NoSegs> for Mode 1 messages from this sender. The absolute maximum number of DSN is 255, however an implementation may apply a user-specified DSN_Max, no smaller than 1. An implementation may support a user-defined DataID_Timeout, after which a given DataID will not be announced in the bundle header unless a new Mode 1 message has been sent. If the sender has more DataIDs sent (and not timed out) than will fit in the bundle header, the DSNs MUST be announced on a round-robin basis, with the exception that no bundle header will announce a DSN for a Mode 1 message contained within that bundle. 4.3 Congestion Control The congestion control mechanism operates as described in [4]. 4.4 Any-Source Multicast SRMP uses Any-Source Multicast Mode. Each sender will determine its Maximum RTT, Suppression data rate and Sending rate with respect to each sender. Each receiver will measure its RTT and Desired Rate to each sender in the group, and send feedback to every sender by sending to the multicast group. Pullen Expires: December 2005 [Page 13]
Internet Draft draft-pullen-srmp-06.txt June 2005 4.5 Multiple Sources Under SRMP, each group member in a multicast group is a sender as well as receiver. Each receiver may need to participate in TFMCC information exchange with all senders. Thus, when a receiver sends a feedback message, it must identify to which source the message should be sent using the "Sender ID" field in the header. The feedback is multicast to the group. Depending on the network situation, senders may select different receivers to provide feedback Feedback messages from receivers that are not among those selected by the local TFMCC to provide feedback should be silently discarded. 4.6. Bundle Size TFMCC is designed for traffic with fixed message size. The maximum bundle size (including header) for SRMP is set to a configurable maximum, typically 1454 bytes (Ethernet MTU less IP and UDP header length). The bundle size will be used in a TCP throughput equation, to get a desired source rate. However, in SRMP the message size is variable because: 1. After bundle time out, the current bundle will not wait for new SRMP messages. This happens with sources sending at a slow rate. 2. Long messages; there is no further space in the current bundle for new SRMP messages. This will happen with sources sending at a high rate or sending messages with length over half of the bundle payload size. The case 1 bundle size is likely to be much smaller than that of case 2. Therefore in SRMP the mean value of the 10 most recent bundles' sizes will be used as the bundle size in the TCP throughput equation. This mean value is independent from the network condition and reflects current activity of the source. 4.7 Data rate control Each host will have a single instance of SRMP supporting all of its applications. Thus the sender's source rate is the sum of the rates all clients of the same multicast group. If the source rate is larger than the senders desired transmission rate, it is the sender's responsibility to do traffic shaping. Any method that conforms to the target sending rate may be used. The RECOMMENDED method is to randomly discard enough Mode 0 messages to meet the target rate. Pullen Expires: December 2005 [Page 14]
Internet Draft draft-pullen-srmp-06.txt June 2005 4.8. Mode 1 loss detection Bundle header processing includes checking each DSN in the bundle header and scheduling a NACK for each DSN bearing a DataID for which some application has indicated interest, if the SN/SegNo in that DSN indicates a NACK is needed. NACKs are sent in bundles, and may be bundled with data messages. A NACK is required if: o the SN is one or more greater (mod 512) than the latest received Mode 1 message for that DataID, or o the SegNo has not been received, some segment of the <DataID,SN> has been received, and a user-defined Segment_Timeout, which SHALL not be less than 50 ms, has expired since receipt of the first SegNo for the <DataID,SN>. The bundling sublayer will pass the DSN list in any received bundle header to the SRT sublayer. It also will suppress NACKs in outgoing bundles, as described in the next section. 4.8.1 Sending a Negative Acknowledgement Negative acknowledgements are used by SRMP for multicast messages in order to avoid the congestion of an "ACK implosion" at the original sender that would likely occur if positive acknowledgements were used instead. However, with a large multicast group spread out over a congested wide-area network, there is the potential for enough members of the multicast group to fail to receive the message and generate NACKs to cause considerable congestion at the original sender despite the use of negative acknowledgements instead of positive acknowledgements. For this reason, SRMP uses a NACK suppression mechanism to reduce the number of NACKs generated in response to any single lost message. The NACK suppression mechanism uses the Bundle_Timeout to distribute NACKs over an appropriate time window. This assumes that the user has Selected a bundle timeout appropriate to the needs of the application for real-time responsiveness. When the bundling sublayer is ready to send a bundle, it removes from the bundle any NACKs for which a response has been sent by another member of the multicast group within the NACK_Repeat_Timeout window. If the original Bundle_Timeout has not expired, transmission of the bundle may then be delayed until the original Bundle_Timeout expires or the bundle is full, whichever happens first. 4.9 Unbundling After a receiver completes congestion control processing on a bundle, it parses the bundle into SRT messages and sends these to SRT sublayer. Pullen Expires: December 2005 [Page 15]
Internet Draft draft-pullen-srmp-06.txt June 2005 4.10 Heartbeat bundle SRMP implementations may support a user-defined Heartbeat_Interval, Which SHALL not be less than one second. At the end of each heartbeat interval, if the sender has not sent any bundle, an empty bundle will be sent in order to trigger Mode 1 loss detection. 5. SRT Operation SRMP operates in three distinct transmission modes in order to deliver varying levels of reliability: Mode 0 for multicast data that does not require reliable transmission, Mode 1 for data that must be received reliably by all members of a multicast group, and Mode 2 for data that must be received reliably by a single dynamically-determined member of a multicast group. Mode 0 operates as a pure best-effort service. Mode 1 operates with negative acknowledgements only, triggered by bundle arrivals that indicate loss of a Mode 1 message. Mode 2 uses a positive acknowledgement for each message to provide reliability and low latency. Mode 2 is used where a transaction between two members of a multicast group is needed. Because there can be many members in such a group, use of a transaction protocol, with reliability achieved by SRMP retransmission, avoids the potentially large amount of connection setup and associated state that would be required if each pair of hosts in the group established a separate TCP connection. Use of SRMP anticipates that only a small fraction of messages will require reliable multicast, and a comparably small fraction will require reliable unicast. This is due to a property of distributed virtual simulation: the preponderance of messages consist of state update streams for object attributes such as position and orientation. SRMP is unlikely to provide effective reliable multicast if the traffic does not have this property. In SRMP, "DataID" is used to associate related messages with each other. Typically, all messages with the same DataID are associated with the same application entity. All the messages with the same "DataID" must be transmitted in the same Mode. Among all the messages with the same Dataid, the latest version will obsolete all older messages. 5.1 Mode 0 operation Mode 0 is for multicast messages that do not require reliable transmission because they are part of a real-time stream of data that is periodically updated with high frequency. Any such message is very likely to have been superceded by a more recent update before retransmission could be completed. Pullen Expires: December 2005 [Page 16]
Internet Draft draft-pullen-srmp-06.txt June 2005 5.1.1 Sending Mode 0 messages When an application requests transmission of Mode 0 data, a destination multicast group must be provided to SRMP along with the data to be sent. After verifying the data length and multicast group, the following steps MUST be performed by the SRT sublayer: 1. An SRT message MUST be generated with the following characteristics: the version is set to the current version, the message type is set to 0x0, the Mode is set to 0x0. User data is included after the message header. If the message cannot be generated as described above, the user data is discarded and the error MUST be reported to the application. 2. If step 1 was completed without error, the newly generated message MUST be sent to the bundling sublayer The implementation MUST report to the application whether or not the message was ultimately accepted by UDP. 5.1.2. Receiving Mode 0 Messages When a mode 0 message is received by SRMP it MUST be processed as follows, after verifying the version, message type, and destination multicast address fields, the user data MUST be delivered to all applications that are associated with the multicast group in the message. If the SRMP receiver has never received any Mode 1 messages before the mode 0 message received, the mode 0 message should be silently discarded. It is RECOMMENDED that the following information should be provided to the receiving applications: message body, multicast address. 5.2. Mode 1 operation Mode 1 is for multicast data that requires reliable transmission. A Mode 1 message can be either a data message or a NACK. Mode 1 data messages are expected to be part of a data stream. This data stream is likely to contain Mode 0 messages as well (see section 3.1.1), but it is possible for a data stream to be comprised solely of Mode 1 messages. 5.2.1. Sending Mode 1 Data messages After the data length, dataID, and destination multicast group are verified, SRT MUST take the following steps: Pullen Expires: December 2005 [Page 17]
Internet Draft draft-pullen-srmp-06.txt June 2005 1. If the message will not fit in an empty bundle with DSN_Max DSN in the header, the message MUST be segmented. The remaining steps pertain to each segment of the message. Each segment receives a unique SegNo, starting with 0 and ending with (NoSegs-1). 2. An SRT message is generated with the following characteristics: the Version is set to 0x02, the message type is set to 0x0, the transmission Mode is set to 0x111, The SN is set equal to the SN of the most recently sent Mode 1 complete message of the same DataID, incremented by 1 modulo 512. If no such mode 1 message exists, the SN is set to 0x0. 3. The newly generated message (all segments) must then be buffered, replacing any formerly buffered Mode 1 message of the same DataID, destination multicast address. If the message cannot be buffered, the user data is discarded and the error is reported to the application. 4. If step 2 was completed without error, the newly generated message is sent to the TFMCC sublayer. 5.2.2. Receiving Mode 1 Data messages When a Mode 1 data message is received by SRT it will be processed as follows (assuming that the version field has already been verified to be 0x02): 1. The destination address MUST be verified to be a valid IP multicast address on which this instance of SRMP is a member. If this is not the case, the message should be silently discarded. 2. The destination address MUST be verified to be one for which some application has indicated interest. Otherwise, the message should be discarded silently. 3. The SN, SegNo, source_ip_address, and the body of the received message MUST be buffered, and the user data MUST then be delivered to all applications that have indicated interest in the multicast group of the received message. 4. When a new DSN value is received with NoSegs greater than zero, a timer should be set for Segment_Timeout, after which a NACK should be sent to the bundling sublayer and the timer should be restarted for Segment_Timeout. 5. If NoSegs in the received message is not 0, a reassembly process MUST be started. Each segment MUST be buffered. If receipt of the current message completes the segment, the reassembled message MUST be released to the application and the Pullen Expires: December 2005 [Page 18]
Internet Draft draft-pullen-srmp-06.txt June 2005 Segment_Timeout timer cancelled. 6. If a new DSN is received before all segments of the previous DSN are received, the segments that have been received should be dropped silently. 7. It is RECOMMENDED that the following information should be provided to the receiving applications: message body, dataid, source_ip_address, multicast_group address. 8. When a client signs on to a new multicast group, all locally buffered mode 1 messages related to that multicast group should be delivered to the client immediately. 5.2.3 Scheduling a Negative Acknowledgement Whenever a bundle is received, the bundling sublayer will forward the DSN list from the bundle header to the SRT sublayer. The SRT sublayer will examine buffered values of <SenderID,DataID,SN,Segno> to determine whether a NACK is required. If so, it will generate a NACK message and send it to the bundling sublayer. The NACK message will have version set to 0x2, message type set to 0x1, transmission mode set to 0x1. DataID, SN, and destination address are set to that of the Mode 1 message for which the NACK is being sent. If a NACK has been received from any member of the destination multicast group for the Mode 1 message in question within the NACK threshold, no NACK is generated. For segmented messages, there are two possible types of NACKs: o Based on the DSN list in the bundle header, the SRT implementation may determine that an entire segmented Mode 1 message was lost. In this case the NACK MUST carry Segno=0x127 (all one in field). o Based on Segment Timeout, the SRT implementation may determine that one or more segments of a message have not been delivered. In this case, a NACK will be sent for each missing segment. 5.2.4 Receiving a Negative Acknowledgement When a NACK is received by SRT, it MUST be processed as follows, after verifying the multicast address, DataID, source IP address, and transmission mode: 1. If this instance of SRT's most recent Mode 1 message of the DataID indicated in the NACK has a SN newer than SN in the NACK, that message (which is buffered) should be immediately retransmitted to the multicast address indicated in the received NACK. If the most recent Mode 1 message has a SN equal to the SN indicated in the NACK, and if the SegNo field Pullen Expires: December 2005 [Page 19]
Internet Draft draft-pullen-srmp-06.txt June 2005 in the NACK contains 0x127, all segments of the buffered Mode 1 message MUST be retransmitted; if the SegNo has some other value, only the indicated segment should be retransmitted. 2. Whether or not step 1 results in the retransmission of a message, the event of receiving the NACK and the (local machine) time at which the NACK was received should be buffered. Each instance of SRT MUST buffer the number of NACKs that have been received for each DataID - multicast address pair, since the most recent Mode 1 message of the same pair was received and the time at which the most recent of these NACKs was received. 5.3. Mode 2 Operation Mode 2 is for infrequent reliable transaction-oriented communication between two dynamically determined members of a multicast group. TCP could be used for such communication, but there would be unnecessary overhead and delay in establishing a stream-oriented connection for a single exchange of data, whereas there is already an ongoing stream of best-effort data between the hosts that require Mode 2 transmission. An example is a Distributed Interactive Simulation (DIS) collision PDU. 5.3.1 Sending Mode 2 Data messages When an application requests transmission of Mode 2 data, a DataID and a destination unicast IP address MUST be provided to SRT along with the data to be sent. After verifying the data length, DataID and destination address, SRT MUST perform the following steps: 1. An SRT message is generated with the following characteristics: the version is set to 0x02, the message type is set to 0x0, the transmission mode is set to 0x2, the DataID is set to the application-provided value, and the destination address is set to the application-provided IP address. The SN is set equal to the SN of the most recently sent Mode 2 message of the same DataID incremented by 1 modulo 65536. If no such Mode 1 message exists, it is set to 0x0. 2. The newly generated message is buffered. This new message does not replace any formerly buffered Mode 2 messages. An implementation MUST provide a Mode 2 message buffer that can hold one or more Mode 2 messages. Mode 2 messages are expected to be infrequent (less than 1% percent of total traffic), but it is still strongly RECOMMENDED that an implementation provide a buffer of user-configurable size Mode2_Max that can hold more than a single Mode 2 message. If the message cannot be buffered, the user data is discarded and the error MUST be reported to the application. Pullen Expires: December 2005 [Page 20]
Internet Draft draft-pullen-srmp-06.txt June 2005 3. If step 2 was completed without error, the newly generated message MUST be sent to the IP address contained in its destination address field, encapsulated within a UDP datagram. If the UDP interface on the sending system reports an error to SRT when the attempt to send the SRT message is made, an implementation may attempt to resend the message any finite number of times. However, every implementation MUST provide a mode in which no retries are attempted. Implementations should default to this latter mode of operation. The implementation MUST report to the application whether or not the message was ultimately accepted by UDP. 4. If some user-configurable "ACK_Threshold" (which should be greater than the worst-case round-trip time for the multicast group) elapses without receipt of an ACK for the Mode 2 message, it is retransmitted. An implementation may define a maximum number of retransmissions to be attempted before the Mode 2 message is removed from the buffer. 5.3.2 Receiving Mode 2 Data messages When a Mode 2 data message is received by SRT it should be processed as follows after verifying Version, DataID, sender address, and SN: 1. For Mode 2 messages, the sequence number field is used to associate the required positive acknowledgement with a specific Mode 2 message. If the message passes verification, the encapsulated user data is delivered to all applications that have indicated interest in the DataID and multicast address of the received message, regardless of the value of the SN field. 2. Additionally, an ACK MUST be sent to the host from which the Mode 2 data message originated. See section 3.3.3. below for details. 5.3.3 Sending a Positive Acknowledgement A positive acknowledgement (ACK) is triggered by the receipt of a Mode 2 data message. To send an ACK, a new SRT message is generated with version set to 0x02, message type set to 0x2, and transmission mode set to 0x2. DataID and SN are those of the Mode 2 data message being acknowledged. The destination address field is set to the source IP address from which the data message was received. Since Mode 2 data messages are unicast there is little concern about an ACK implosion causing excessive congestion at the original sender, so no suppression mechanism is necessary. Pullen Expires: December 2005 [Page 21]
Internet Draft draft-pullen-srmp-06.txt June 2005 5.3.4 Receiving a Positive Acknowledgement When an ACK is received by SRT, after verifying the transmission mode, DataID, and source IP address against outstanding Mode 2 transmission SRT MUST remove the pending transmission from its buffer. 6. RFC 2357 Analysis This section provides answers to the questions posed by RFC 2357 for reliable multicast protocols, which are quoted. 6.1 Scalability "How scalable is the protocol to the number of senders or receivers in a group, the number of groups, and wide dispersion of group members?" SRMP is intended to scale at least to hundreds of group members. It has been designed not to impose limitations on the scalability of the underlying multicast network. No problems have been identified in its mechanisms that would preclude this on uncongested networks. "Identify the mechanisms which limit scalability and estimate those limits." There is a practical concern with use of TFMCC, in that the receiver with the most congested path constraints delivery to the entire group. Distributed virtual simulation requires data delivery at rates perceived as continuous by humans. Therefore it may prove necessary to assign such receivers to different, lower- fidelity groups as a practical means of sustaining performance to the majority of participating hosts. SRMP does not have a mechanism to support such pruning at this time. 6.2 Congestion "How does the protocol protect the Internet from congestion? How well does it perform? When does it fail? Under what circumstances will the protocol fail to perform the functions needed by the applications it serves? Is there a congestion control mechanism? How well does it perform? When does it fail?" Both simulations and tests indicate that SRMP with TFMCC displays backoff comparable to that of TCP under conditions of significant packet loss. The mechanism fails in a network-friendly way, in that under severe congestion it reduces sending of the best-effort traffic to a very small rate that typically is unsatisfactory to support a virtual simulation. This is possible because the reliable traffic typically is a few percent of the overall traffic and SRMP is NACK-oriented, with NACK suppression, so that reliable traffic loss adds little traffic to the total. If the traffic mix assumption is not met, the reliable traffic (which does not back off under increased RTT) could produce a higher level of traffic than a comparable TCP connection. However, levels of reliable traffic this Pullen Expires: December 2005 [Page 22]
Internet Draft draft-pullen-srmp-06.txt June 2005 large are not in the intended application domain of SRMP. "Include a description of trials and/or simulations which support the development of the protocol and the answers to the above questions." SRMP has been simulated using a discrete event simulator developed for academic use [5]. The design assumptions were validated by the results. It also has been emulated in a LAN-based cluster and application-tested in a wide-area testbed under its intended traffic mix (distributed virtual simulation) and using a traffic generator with losses emulated losses by random dropping of packets [6]. "Include an analysis of whether the protocol has congestion avoidance mechanisms strong enough to cope with deployment in the Global Internet, and if not, clearly document the circumstances in which congestion harm can occur. How are these circumstances to be prevented?" Because it provides sending backoff comparable to TCP, SRMP is able to function as well as TCP for congestion avoidance, even in the Global Internet. The only way an SRMP sender can generate congestion is to use the protocol for unintended purposes, for example reliable transmission of a large fraction of the traffic. Doing this would produce unsatisfactory results for the application, as SRMP's mechanism for providing reliability will not function well if the best-effort traffic is does not constitute the majority of the total traffic. "Include a description of any mechanisms which contain the traffic within limited network environments." SRMP has no such mechanisms, as it is intended for use over the open Internet. "Reliable multicast protocols must include an analysis of how they address a number of security and privacy concerns." See section 9 below. 7. IANA Considerations There are no IANA actions required for this document. 8. Security Considerations As a transport protocol, SRMP is subject to denial of service by hostile third parties sending conflicting values of its parameters on the multicast address. SRMP could attempt to protect itself from this sort of behavior, however it can be shielded from such attacks by traffic authentication at the network layer, as decribed below. A comparable level of authentication also could be obtained by message using MD5 or a similar message hash in each bundle and using the SRMP bundle header to detect duplicate transmissions from a given host, however this would duplicate the function of existing network layer authentication protocols. Pullen Expires: December 2005 [Page 23]
Internet Draft draft-pullen-srmp-06.txt June 2005 Specific threats that can be eliminated by packet-level authentication are: a. Amplification Attack: SRMP receivers could be manipulated into sending large amounts of NACK traffic which could cause network congestion or overwhelm the processing capabilities of a sender. This could be done by sending them faked traffic indicating that a reliable transmission has been lost. SRMP's NACK suppression limits the effect of such manipulation, however true protection requires authentication of each bundle. b. Denial of service attack: if an SRMP sender accepts a large number of forged NACKS, it will flood the multicast group with repair messages. This attack also is stopped by per-bundle authentication. c. Replay Attack: the attacker could copy a valid, authenticated bundle containing a NACK and send it repeatedly to the original sender of the NACKed data. Protection against this attack requires a sequence number per transmission per source host. The SRMP bundle header sequence number would satisfy this need, however the SN also can be applied at a lower layer. d. Reverse Path Forwarding Attack (spoofing): if checks are not enabled in all network routers and switches along the path from each sender to all receivers, forged packets can be injected into the multicast tree data path to manipulate the protocol into sending large volume of repairs. Packet-level authentication can eliminate this possibility. e. Inadvertent errors: A receiver with an incorrect or corrupted implementation of TFMCC could respond with values of RTT that might stimulate a TFMCC sender to create or increase congestion in the path to that sender. It is therefore RECOMMENDED that receivers be required to identify themselves as legitimate before they receive the Session Description needed to join the session. How receivers identify themselves as legitimate is outside the scope of this document. The required authentication could become part of SRMP or could be accomplished by a lower layer protocol. In any case it needs to be (1) scalable and (2) not very computationally demanding so it can be performed with minimal delay on a real-time virtual simulation stream. Public-key encryption meets the first requirement but not the second. Using the IPSEC Authentication Header (AH) (RFC 2402 [9]) meets the second requirement using symmetric-key cryptography. However, RFC 2402 guidance is that a separate Security Association (SA) should be established for each sender-receiver pair so that no host in the group can spoof traffic from another source. This approach is not scalable over a large number of multicast group members. Moving the authentication function to SRMP would introduce Pullen Expires: December 2005 [Page 24]
Internet Draft draft-pullen-srmp-06.txt June 2005 the same problem. If the policy is to trust all hosts that are legitimate senders, IPSEC AH with one SA per multicast group meets both requirements. If the guidance in RFC 2402 is treated as policy, no scalable solution is known to exist. In practice, users of distributed simulation are likely to work over a (possibly virtual) private network and thus not to need special authentication for SRMP. For those who do not, it is RECOMMENDED that users of SRMP over the open Internet should use the IPSEC AH but in so doing must accept the risk of the unlikely case where a member of the trusted group of hosts undertakes an attack on the group by masquerading as another group member. 9. List of acronyms used ACK - positive acknowledgement AH - Authentication Header CLR - current limiting receiver IPSEC - Internet Protocol Security MTU - maximum transmission unit NACK - negative acknowledgement RTT - round-trip time SA - security association SRMP - Selectively Reliable Multicast Procotol SRT - Selectively Reliable Transport TFMCC - TCP Friendly Multicast Congestion Control 10. Authors' Addresses J. Mark Pullen C3I Center George Mason University Fairfax, VA 22030 USA mpullen@gmu.edu Babu Shanmugam C3I Center George Mason University Fairfax, VA 22030 USA bshanmug@netlab.gmu.edu Fei Zhao C3I Center George Mason University Fairfax, VA 22030 USA fzhao@netlab.gmu.edu Pullen Expires: December 2005 [Page 25]
Internet Draft draft-pullen-srmp-06.txt June 2005 Danny Cohen Sun Microsystems 6601 Center Drive West Los Angeles, CA 90045 USA danny.cohen@sun.com 11. References 11.1 Informative references [1] J. M. Pullen, J.M., M. Myjak, and C. Bouwens, "Limitations of Internet Protocol Suite for Distributed Simulation in the Large Multicast Environment," Internet Engineering Task Force Informational RFC 2502, Internet Society, 1999 [2] J. Padhye, V. Firoiu, D. Towsley and J. Kurose, "Modeling TCP Throughput: A Simple Model and its Empirical Validation", Proceedings of ACM SIGCOMM 1998 [3] A. Mankin, A. Romanow, S. Bradner, and V. Paxson, "IETF Criteria for Evaluating Reliable Multicast Transport and Application Protocols," Internet Engineering Task Force Informational RFC 2357, Internet Society, 1998 [4] S. Floyd, "Congestion Control Principles", Internet Engineering Task Force Best Current Practice RFC 2914, Internet Society, 1999 [5] J. M. Pullen, "The Network Workbench: Network Simulation Software for Academic Investigation of Internet Concepts," Computer Networks Vol 32 No 3 pp 365-378, March 2000 [6] J. M. Pullen, R. Simon, F. Zhao and W. Chang, "NGI-FOM over RTI-NG and SRMP: Lessons Learned," Proceedings of the IEEE Fall Simulation Interoperability Workshop, paper 03F-SIW-111, Orlando, FL, September 2003 11.2 Normative references [7] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels, Internet Engineering Task Force RFC 2119, Internet Society, 1997 [8] J. Widmer and M. Handley, "TCP-Friendly Multicast Congestion Control (TFMCC): Protocol Specification", work in progress, IETF Reliable Multicast Transport Working Group, October 2004 [9] S. Kent and R. Atkinson, IP Authentication Header, Internet Engineering Task Force Standards Track RFC 2402, November 1998 Pullen Expires: December 2005 [Page 26]
Internet Draft draft-pullen-srmp-06.txt June 2005 12. Full Copyright Statement Copyright (C) The Internet Society (2005). This document is subject to the rights, licenses and restrictions contained in BCP 78, and except as set forth therein, the authors retain all their rights. This document and the information contained herein are provided on an "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Pullen Expires: December 2005 [Page 27]
