Multipath sequence maintenance
draft-amend-iccrg-multipath-reordering-00
The information below is for an old version of the document.
| Document | Type |
This is an older version of an Internet-Draft whose latest revision state is "Expired".
|
|
|---|---|---|---|
| Authors | Markus Amend , Dirk Von Hugo | ||
| Last updated | 2020-07-08 | ||
| RFC stream | (None) | ||
| Formats | |||
| Additional resources | |||
| Stream | Stream state | (No stream defined) | |
| Consensus boilerplate | Unknown | ||
| RFC Editor Note | (None) | ||
| IESG | IESG state | I-D Exists | |
| Telechat date | (None) | ||
| Responsible AD | (None) | ||
| Send notices to | (None) |
draft-amend-iccrg-multipath-reordering-00
ICCRG Working Group M. Amend
Internet-Draft D. von Hugo
Intended status: Experimental DT
Expires: 8 January 2021 7 July 2020
Multipath sequence maintenance
draft-amend-iccrg-multipath-reordering-00
Abstract
This document discusses the issue of packet re-ordering which occurs
as a specific problem in multi-path connections without reliable
transport protocols such as TCP. The topic is relevant for devices
connected via multiple accesses technologies towards the network as
is foreseen e.g. within Access Traffic Selection, Switching, and
Splitting (ATSSS) service of 3rd Generation Partnership Project
(3GPP) enabling fixed mobile converged (FMC) scenario.
Status of This Memo
This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet-
Drafts is at https://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
This Internet-Draft will expire on 8 January 2021.
Copyright Notice
Copyright (c) 2020 IETF Trust and the persons identified as the
document authors. All rights reserved.
Amend & von Hugo Expires 8 January 2021 [Page 1]
Internet-Draft Multipath sequence maintenance July 2020
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents (https://trustee.ietf.org/
license-info) in effect on the date of publication of this document.
Please review these documents carefully, as they describe your rights
and restrictions with respect to this document. Code Components
extracted from this document must include Simplified BSD License text
as described in Section 4.e of the Trust Legal Provisions and are
provided without warranty as described in the Simplified BSD License.
Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2
2. State of the Art . . . . . . . . . . . . . . . . . . . . . . 4
3. Problem Statement . . . . . . . . . . . . . . . . . . . . . . 5
4. Scheduling mechanisms . . . . . . . . . . . . . . . . . . . . 6
5. Resequencing mechanisms . . . . . . . . . . . . . . . . . . . 6
5.1. Passive . . . . . . . . . . . . . . . . . . . . . . . . . 7
5.2. Exact . . . . . . . . . . . . . . . . . . . . . . . . . . 7
5.3. Static Expiration . . . . . . . . . . . . . . . . . . . . 7
5.4. Adaptive Expiration . . . . . . . . . . . . . . . . . . . 7
5.5. Delay Equalization . . . . . . . . . . . . . . . . . . . 7
5.6. Fast Packet Loss Detection . . . . . . . . . . . . . . . 7
5.6.1. Using overall sequencing . . . . . . . . . . . . . . 7
5.6.2. Using per-path sequencing . . . . . . . . . . . . . . 7
6. Recovery mechanisms . . . . . . . . . . . . . . . . . . . . . 7
6.1. FEC . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
6.2. Network Coding . . . . . . . . . . . . . . . . . . . . . 8
7. Retransmission mechanisms . . . . . . . . . . . . . . . . . . 8
7.1. Fast re-transmission . . . . . . . . . . . . . . . . . . 8
8. Informative References . . . . . . . . . . . . . . . . . . . 8
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 10
1. Introduction
Mobile end user devices nowadays are mostly equipped with multiple
network interfaces allowing to connect to more than one network at a
time and thus increase data throughput, reliability, coverage and so
on. Ideally the user data stream originating from the application at
the device is split between the available (here: N) paths at the
sender side and re-assembled at an intermediate aggregation node
before transmitted to the corresponding host in the network as
depicted in Figure 1.
Amend & von Hugo Expires 8 January 2021 [Page 2]
Internet-Draft Multipath sequence maintenance July 2020
------------
/ \
+---------| Access Net 1 |--+
| \ / |
| ------------- |
| ------------ |
| / \ |
| +------| Access Net 2 |-+ |
| | \ / | |
| | ------------ | |
| | | |
| | | |
| | +-------+-+---+
+--+-+-+ | | +-----+ +------+
|End | | Aggregation +---|Gate-|-/.../--| Host |
|User | | Node | |way | +------+
|Device| | | +-----+
+--+-+-+ +-------+-+---+
| | | |
| | -------------- | |
| | / \ | |
| +---| Access Net N-1 |--+ |
| \ / |
| -------------- |
| |
| ------------ |
| / \ |
+--------| Access Net N |---+
\ /
------------
Figure 1: Reference Architecture for multi-path re-ordering
However, when several paths are utilized concurrently to transmit
user data between the sender and the receiver, different
characteristics of the paths in terms of bandwidth, delay, or error
proneness can impact the overall performance due to delayed packet
arrival and need for re-transmit in case of lost packets. Without
further arrangements the original order of packets at the sending UE
side is no longer maintained at the receiving host and a re-ordering
or re-arrangement has to occur before delivery to the application at
the far end site. This can be performed at earliest at the
aggregation node with a minimum additional delay due to re-
transmission requests or at latest either by the application on the
host itself or the transmission protocol.
Amend & von Hugo Expires 8 January 2021 [Page 3]
Internet-Draft Multipath sequence maintenance July 2020
It is a goal of the present document to collect and describe
mechanisms to maintain the sequence of split traffic over multiple
paths. These mechanisms are generic and not dedicated to a specific
multipath network protocol, but give clear guidance on requirements
and benefits to maintainers of multipath network protocols.
2. State of the Art
Regular TCP protocol [RFC0793] offers such mechanism with queues for
in-order and out-of order (including damaged, lost, duplicated)
arrival of packets.
This is also provided by MPTCP [RFC6824] as the first and successful
Multipath protocol which however also requires new methods as
sequence numbers both on (whole) data (stream) and subflow level to
ensure in-order delivery to the application layer on the receiver
side [RFC8684]. Moreover, careful design of buffer sizes and
interpretation of sequence numbers to distinguish between (delayed)
out-of-order packets and completely lost ones has to be considered.
[I-D.bonaventure-iccrg-schedulers] already reflects on proper packet
scheduling schemes (at the sender side) to reduce the effort for re-
assembly or even make such (time consuming) treatment unnecessary.
MP-QUIC [I-D.deconinck-quic-multipath] introduces the concept of
uniflows with own IDs claiming to get rid of additional sequence
numbers for re-ordering as required in Multipath TCP [RFC6824].
Although [] admits that statistical performance information should
help a host in deciding on optimum packet scheduling and flow control
a dedicated packet scheduling policy is out of scope of that
document. A further improvement versus MPTCP can be achieved by
decoupling paths used for data transmission from those for sending
acknowledgments (ACKs) or claiming for re-transmission by NACKs to
not introduce further latency.
[I-D.ietf-quic-recovery] specifies algorithms for QUIC Loss Detection
and Congestion Control by using measurement of Round Trip Time (RTT)
to determine when packets should be retransmitted. Draft
[I-D.huitema-quic-ts] proposes to enable one way delay (1WD)
measurements in QUIC by defining a TIME_STAMP frame to carry the time
at which a packet is sent and combine the ACKs sent with a timestamp
field and thus allow for more precise estimation of the (one-way)
delay of each uniflow, assisting proper scheduling decisions.
Also other protocols as Multi-Access Management Services (MAMS)
[RFC8743] consider the need for re-ordering on User Plane level which
may be done at network and client level by introducing a new Multi-
Access (MX) Convergence Layer. [I-D.zhu-intarea-mams-user-protocol]
Amend & von Hugo Expires 8 January 2021 [Page 4]
Internet-Draft Multipath sequence maintenance July 2020
introduces accordingly Traffic Splitting Update (TSU) messages and
Packet Loss Report (PLR) messages including beside others Traffic
Splitting Parameters and an expected next (in-order) sequence number,
respectively.
[I-D.zhu-intarea-gma] on Generic Multi-Access (GMA) Convergence
Encapsulation Protocols introduces a trailer-based encapsulation
which carries one or multiple IP packets or fragments thereof in a
Protocol Data Unit (PDU). At receiver side PDUs with identical
Sequence Numbers (in the trailer) are to be placed in the relative
order indicated by a so-called Fragment Offset.
3. Problem Statement
Assuming for simplicity the minimum multipath scenario with two
separate paths for transmission of a flow of packets with sequence
numbers (SN) SN1 ... SM. In case the scheduling of packets is done
equally to both paths and path 2 exhibits a delay of the duration of
transmission time required for e.g. two packets (assuming fixed
packet size and same constant data for both paths) for an exemplary
App-originated sequence of packets as SN1 SN2 SN3 SN4 SN5 SN6 SN7 SN8
... the resulting sequence of packets could look as depicted in
Figure 2 which of course depends on the queue processing and
buffering at the Aggregation Proxy.
APP UE Aggregation Proxy Host
| SN1 ... SN8 | | |
|-------------->| path 1 SN1 SN3 SN5 SN7...| |
| |------------------------->| |
| | path 2 SN2 SN4 SN6 SN8...| |
| |------------------------->| |
| | |SN1 SN3 SN2 SN5 SN4 SN7|
| | |======================>|
| | | |
Figure 2: Exemplary data transmission for a dual-path scenario
In such a case re-ordering at the Aggregation Proxy would be simple
and straight forward. It even could be avoided if the scheduling
would already take the expected different delays into account (e.g.
by pre-delaying the traffic on path 1 thus of course not leveraging
the lower delay). Different from this simplistic scenario in general
the data rate on both paths will vary in time and be not equal, also
different and variable latency (jitter) per path will be introduced
and in addition loss of packets as well as potential duplication may
occur making the situation much more complicated. In case of loss
detection after a threshold waiting time a retransmission could be
initiated by the Host or if possible by the Proxy. Alternatively the
Amend & von Hugo Expires 8 January 2021 [Page 5]
Internet-Draft Multipath sequence maintenance July 2020
UE could send redundant packets in advance coded in such a way that
it allows for derivation of e.g. one lost packet per M correctly
received ones or by a (real-time) application able to survive single
lost packets.
Holding multiple queues and a large enough buffer both at UE and at
the Aggregation Proxy would be required to apply proper scheduling at
UE and reordering during re-assembly at Aggregation Proxy to mitigate
the sketched impact of multiple paths variable characteristics in
terms of transmission performance.
...
4. Scheduling mechanisms
Scheduling mechanisms decides on sender side how traffic is
distributed over the paths of a multipath-setup.
[I-D.bonaventure-iccrg-schedulers] gives an overview of possible
distribution schemes. For this document it is assumed, that
schedulers are used, which simultaneously distributes traffic over
more than one path and latency difference(s) exists between those
multiple paths.
5. Resequencing mechanisms
Resequencing mechanism are responsible to modify the sequence of
received data split over multiple paths according to a sequencing
scheme. The degree of resequencing can reach from now measure up to
re-generating the exact order.
Typically at least one sequencing scheme describing the order of how
data was generated on sender side and is referred to as "overall
sequencing". Under certain circumstances an additional sequencing
scheme per path of the multi-path setup can be leveraged to optimize
packet loss detection. For most multipath protocols both sequencing
schemes are already available. Packet loss detection becomes
important when multipath protocols are applied which does not
guarantee successful transmission. For example
[I-D.amend-tsvwg-multipath-dccp] or the combination of
[I-D.deconinck-quic-multipath] and [I-D.ietf-quic-datagram] are
unreliable in that sense.
For simplicity all the mechanism described in the following are
explained based on two paths but work with an unlimited number
though.
Amend & von Hugo Expires 8 January 2021 [Page 6]
Internet-Draft Multipath sequence maintenance July 2020
5.1. Passive
Nothing is done.
5.2. Exact
Exact order is re-generated. Only useful for network protocols which
re-transmits. Risk: Head-of-Line blocking
5.3. Static Expiration
Packet gap is assumed as packet loss after certain time threshold
5.4. Adaptive Expiration
Packet gap is assumed as packet loss after dynamic time threshold
derived from the latency differences between paths.
5.5. Delay Equalization
Delay data on the faster path by the latency difference to the slower
path. Strictly spoken no resequencing based on sequencing
information.
5.6. Fast Packet Loss Detection
5.6.1. Using overall sequencing
Compare the overall sequence number arriving. When on all path a
higher number is received than the one which is waited for, packet
loss can is given.
5.6.2. Using per-path sequencing
For environments where no per-path scrambling is given. Compare
distance between path sequencing and overall sequencing. When
mismatch then packet loss. Requires at least three packets in-order
on a path to work to identify loss of the middle packet.
6. Recovery mechanisms
Recovering packets, in particular lost packets or assumed lost
packets on receiver side avoids re-transmission and potentially
mitigates the resequencing process in respect to detecting packet
loss. Shorter latencies will be an expected outcome. Discussing the
complexity, computation overhead and reachable benefit is subject of
this section.
Amend & von Hugo Expires 8 January 2021 [Page 7]
Internet-Draft Multipath sequence maintenance July 2020
6.1. FEC
Introduce redundancy to reconstruct data. Are there two
possibilities: Per-path FEC & overall FEC?
6.2. Network Coding
Linear network coding
7. Retransmission mechanisms
Re-transmission becomes interesting when it can help to reduce the
time spent on waiting for outstanding packets for re-sequencing. In
particular scenarios when the RTT lets expect a shorter time to re-
transmit than wait for packet loss detection, a likely scenario in
e.g. Figure 1. It could also avoid a potential late triggering of
re-transmission by the end-to-end service.
7.1. Fast re-transmission
Signal to the sender to re-transmit a missing packet.
8. Informative References
[I-D.amend-tsvwg-multipath-dccp]
Amend, M., Bogenfeld, E., Brunstrom, A., Kassler, A., and
V. Rakocevic, "DCCP Extensions for Multipath Operation
with Multiple Addresses", Work in Progress, Internet-
Draft, draft-amend-tsvwg-multipath-dccp-03, 4 November
2019, <http://www.ietf.org/internet-drafts/draft-amend-
tsvwg-multipath-dccp-03.txt>.
[I-D.bonaventure-iccrg-schedulers]
Bonaventure, O., Piraux, M., Coninck, Q., Baerts, M.,
Paasch, C., and M. Amend, "Multipath schedulers", Work in
Progress, Internet-Draft, draft-bonaventure-iccrg-
schedulers-00, 9 March 2020, <http://www.ietf.org/
internet-drafts/draft-bonaventure-iccrg-schedulers-
00.txt>.
[I-D.deconinck-quic-multipath]
Coninck, Q. and O. Bonaventure, "Multipath Extensions for
QUIC (MP-QUIC)", Work in Progress, Internet-Draft, draft-
deconinck-quic-multipath-04, 5 March 2020,
<http://www.ietf.org/internet-drafts/draft-deconinck-quic-
multipath-04.txt>.
Amend & von Hugo Expires 8 January 2021 [Page 8]
Internet-Draft Multipath sequence maintenance July 2020
[I-D.huitema-quic-ts]
Huitema, C., "Quic Timestamps For Measuring One-Way
Delays", Work in Progress, Internet-Draft, draft-huitema-
quic-ts-02, 1 March 2020, <http://www.ietf.org/internet-
drafts/draft-huitema-quic-ts-02.txt>.
[I-D.ietf-quic-datagram]
Pauly, T., Kinnear, E., and D. Schinazi, "An Unreliable
Datagram Extension to QUIC", Work in Progress, Internet-
Draft, draft-ietf-quic-datagram-00, 26 February 2020,
<http://www.ietf.org/internet-drafts/draft-ietf-quic-
datagram-00.txt>.
[I-D.ietf-quic-recovery]
Iyengar, J. and I. Swett, "QUIC Loss Detection and
Congestion Control", Work in Progress, Internet-Draft,
draft-ietf-quic-recovery-29, 9 June 2020,
<http://www.ietf.org/internet-drafts/draft-ietf-quic-
recovery-29.txt>.
[I-D.zhu-intarea-gma]
Zhu, J. and S. Kanugovi, "Generic Multi-Access (GMA)
Encapsulation Protocol", Work in Progress, Internet-Draft,
draft-zhu-intarea-gma-07, 14 May 2020,
<http://www.ietf.org/internet-drafts/draft-zhu-intarea-
gma-07.txt>.
[I-D.zhu-intarea-mams-user-protocol]
Zhu, J., Seo, S., Kanugovi, S., and S. Peng, "User-Plane
Protocols for Multiple Access Management Service", Work in
Progress, Internet-Draft, draft-zhu-intarea-mams-user-
protocol-09, 4 March 2020, <http://www.ietf.org/internet-
drafts/draft-zhu-intarea-mams-user-protocol-09.txt>.
[RFC0793] Postel, J., "Transmission Control Protocol", STD 7,
RFC 793, DOI 10.17487/RFC0793, September 1981,
<https://www.rfc-editor.org/info/rfc793>.
[RFC6824] Ford, A., Raiciu, C., Handley, M., and O. Bonaventure,
"TCP Extensions for Multipath Operation with Multiple
Addresses", RFC 6824, DOI 10.17487/RFC6824, January 2013,
<https://www.rfc-editor.org/info/rfc6824>.
[RFC8684] Ford, A., Raiciu, C., Handley, M., Bonaventure, O., and C.
Paasch, "TCP Extensions for Multipath Operation with
Multiple Addresses", RFC 8684, DOI 10.17487/RFC8684, March
2020, <https://www.rfc-editor.org/info/rfc8684>.
Amend & von Hugo Expires 8 January 2021 [Page 9]
Internet-Draft Multipath sequence maintenance July 2020
[RFC8743] Kanugovi, S., Baboescu, F., Zhu, J., and S. Seo, "Multiple
Access Management Services Multi-Access Management
Services (MAMS)", RFC 8743, DOI 10.17487/RFC8743, March
2020, <https://www.rfc-editor.org/info/rfc8743>.
Authors' Addresses
Markus Amend
Deutsche Telekom
Deutsche-Telekom-Allee 9
64295 Darmstadt
Germany
Email: Markus.Amend@telekom.de
Dirk von Hugo
Deutsche Telekom
Deutsche-Telekom-Allee 9
64295 Darmstadt
Germany
Email: dirk.von-Hugo@telekom.de
Amend & von Hugo Expires 8 January 2021 [Page 10]