The Subnetwork Encapsulation and Adaptation Layer (SEAL)
draft-templin-intarea-seal-49
The information below is for an old version of the document.
| Document | Type |
This is an older version of an Internet-Draft whose latest revision state is "Expired".
|
|
|---|---|---|---|
| Author | Fred Templin | ||
| Last updated | 2012-07-16 | ||
| RFC stream | Internet Engineering Task Force (IETF) | ||
| Formats | |||
| Reviews | |||
| Additional resources | |||
| Stream | WG state | (None) | |
| Document shepherd | (None) | ||
| IESG | IESG state | AD Evaluation | |
| Consensus boilerplate | Unknown | ||
| Telechat date | (None) | ||
| Responsible AD | Ralph Droms | ||
| Send notices to | fltemplin@acm.org, draft-templin-intarea-seal@tools.ietf.org | ||
| RFC Editor | RFC Editor state | ISR | |
| Details |
draft-templin-intarea-seal-49
Transport Area working group (tsvwg) K. De Schepper
Internet-Draft Nokia Bell Labs
Intended status: Experimental B. Briscoe, Ed.
Expires: November 22, 2021 Independent
G. White
CableLabs
May 21, 2021
DualQ Coupled AQMs for Low Latency, Low Loss and Scalable Throughput
(L4S)
draft-ietf-tsvwg-aqm-dualq-coupled-15
Abstract
The Low Latency Low Loss Scalable Throughput (L4S) architecture
allows data flows over the public Internet to achieve consistent low
queuing latency, generally zero congestion loss and scaling of per-
flow throughput without the scaling problems of standard TCP Reno-
friendly congestion controls. To achieve this, L4S data flows have
to use one of the family of 'Scalable' congestion controls (TCP
Prague and Data Center TCP are examples) and a form of Explicit
Congestion Notification (ECN) with modified behaviour. However,
until now, Scalable congestion controls did not co-exist with
existing Reno/Cubic traffic --- Scalable controls are so aggressive
that 'Classic' (e.g. Reno-friendly) algorithms sharing an ECN-capable
queue would drive themselves to a small capacity share. Therefore,
until now, L4S controls could only be deployed where a clean-slate
environment could be arranged, such as in private data centres (hence
the name DCTCP). This specification defines `DualQ Coupled Active
Queue Management (AQM)', which enables Scalable congestion controls
that comply with the Prague L4S requirements to co-exist safely with
Classic Internet traffic.
Analytical study and implementation testing of the Coupled AQM have
shown that Scalable and Classic flows competing under similar
conditions run at roughly the same rate. It achieves this
indirectly, without having to inspect transport layer flow
identifiers. When tested in a residential broadband setting, DCTCP
also achieves sub-millisecond average queuing delay and zero
congestion loss under a wide range of mixes of DCTCP and `Classic'
broadband Internet traffic, without compromising the performance of
the Classic traffic. The solution has low complexity and requires no
configuration for the public Internet.
De Schepper, et al. Expires November 22, 2021 [Page 1]
Internet-Draft DualQ Coupled AQMs May 2021
Status of This Memo
This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet-
Drafts is at https://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
This Internet-Draft will expire on November 22, 2021.
Copyright Notice
Copyright (c) 2021 IETF Trust and the persons identified as the
document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents
(https://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must
include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as
described in the Simplified BSD License.
Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3
1.1. Outline of the Problem . . . . . . . . . . . . . . . . . 3
1.2. Scope . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.3. Terminology . . . . . . . . . . . . . . . . . . . . . . . 7
1.4. Features . . . . . . . . . . . . . . . . . . . . . . . . 9
2. DualQ Coupled AQM . . . . . . . . . . . . . . . . . . . . . . 10
2.1. Coupled AQM . . . . . . . . . . . . . . . . . . . . . . . 10
2.2. Dual Queue . . . . . . . . . . . . . . . . . . . . . . . 12
2.3. Traffic Classification . . . . . . . . . . . . . . . . . 12
2.4. Overall DualQ Coupled AQM Structure . . . . . . . . . . . 13
2.5. Normative Requirements for a DualQ Coupled AQM . . . . . 16
2.5.1. Functional Requirements . . . . . . . . . . . . . . . 16
2.5.1.1. Requirements in Unexpected Cases . . . . . . . . 17
2.5.2. Management Requirements . . . . . . . . . . . . . . . 18
De Schepper, et al. Expires November 22, 2021 [Page 2]
Internet-Draft DualQ Coupled AQMs May 2021
2.5.2.1. Configuration . . . . . . . . . . . . . . . . . . 18
2.5.2.2. Monitoring . . . . . . . . . . . . . . . . . . . 20
2.5.2.3. Anomaly Detection . . . . . . . . . . . . . . . . 20
2.5.2.4. Deployment, Coexistence and Scaling . . . . . . . 21
3. IANA Considerations (to be removed by RFC Editor) . . . . . . 21
4. Security Considerations . . . . . . . . . . . . . . . . . . . 21
4.1. Overload Handling . . . . . . . . . . . . . . . . . . . . 21
4.1.1. Avoiding Classic Starvation: Sacrifice L4S Throughput
or Delay? . . . . . . . . . . . . . . . . . . . . . . 22
4.1.2. Congestion Signal Saturation: Introduce L4S Drop or
Delay? . . . . . . . . . . . . . . . . . . . . . . . 23
4.1.3. Protecting against Unresponsive ECN-Capable Traffic . 24
5. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 24
6. Contributors . . . . . . . . . . . . . . . . . . . . . . . . 25
7. References . . . . . . . . . . . . . . . . . . . . . . . . . 25
7.1. Normative References . . . . . . . . . . . . . . . . . . 25
7.2. Informative References . . . . . . . . . . . . . . . . . 26
Appendix A. Example DualQ Coupled PI2 Algorithm . . . . . . . . 30
A.1. Pass #1: Core Concepts . . . . . . . . . . . . . . . . . 31
A.2. Pass #2: Overload Details . . . . . . . . . . . . . . . . 40
Appendix B. Example DualQ Coupled Curvy RED Algorithm . . . . . 44
B.1. Curvy RED in Pseudocode . . . . . . . . . . . . . . . . . 44
B.2. Efficient Implementation of Curvy RED . . . . . . . . . . 50
Appendix C. Choice of Coupling Factor, k . . . . . . . . . . . . 52
C.1. RTT-Dependence . . . . . . . . . . . . . . . . . . . . . 52
C.2. Guidance on Controlling Throughput Equivalence . . . . . 53
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 54
1. Introduction
This document specifies a framework for DualQ Coupled AQMs, which is
the network part of the L4S architecture [I-D.ietf-tsvwg-l4s-arch].
L4S enables both very low queuing latency (sub-millisecond on
average) and high throughput at the same time, for ad hoc numbers of
capacity-seeking applications all sharing the same capacity.
1.1. Outline of the Problem
Latency is becoming the critical performance factor for many (most?)
applications on the public Internet, e.g. interactive Web, Web
services, voice, conversational video, interactive video, interactive
remote presence, instant messaging, online gaming, remote desktop,
cloud-based applications, and video-assisted remote control of
machinery and industrial processes. In the developed world, further
increases in access network bit-rate offer diminishing returns,
whereas latency is still a multi-faceted problem. In the last decade
or so, much has been done to reduce propagation time by placing
De Schepper, et al. Expires November 22, 2021 [Page 3]
Internet-Draft DualQ Coupled AQMs May 2021
quot; for this ETE link path to the MTU value in
the PTB message. If PMTU==0, the ITE consults a plateau table (e.g.,
as described in [RFC1191]) to determine PMTU based on the length
field in the outer IP header of the packet-in-error. For example, if
the ITE receives a PTB message with MTU==0 and length 4KB, it can set
PMTU=2KB. If the ITE subsequently receives a PTB message with MTU==0
and length 2KB, it can set PMTU=1792, etc. to a minimum value of
PMTU=(1500+HLEN). If the ITE is performing stateful MTU
determination for this ETE link path (see Section 4.4.9), the ITE
next sets PATH_MTU=MAX((PMTU-HLEN), 1500).
If the ICMP message was not discarded, the ITE then transcribes it
into a message to return to the previous hop. If the inner packet
was a SEAL data packet, the ITE transcribes the ICMP message into an
SCMP message. Otherwise, the ITE transcribes the ICMP message into a
message appropriate for the inner protocol version.
To transcribe the message, the ITE extracts the inner packet from
within the ICMP message packet-in-error field and uses it to generate
a new message corresponding to the type of the received ICMP message.
For SCMP messages, the ITE generates the message the same as
described for ETE generation of SCMP messages in Section 4.6.1. For
(S)PTB messages, the ITE writes (PMTU-HLEN) in the MTU field.
The ITE finally forwards the transcribed message to the previous hop
toward the inner source address.
4.4.8. IPv4 Middlebox Reassembly Testing
The ITE can perform a qualification exchange to ensure that the
subnetwork correctly delivers fragments to the ETE. This procedure
can be used, e.g., to determine whether there are middleboxes on the
path that violate the [RFC1812], Section 5.2.6 requirement that: "A
router MUST NOT reassemble any datagram before forwarding it".
The ITE should use knowledge of its topological arrangement as an aid
in determining when middlebox reassembly testing is necessary. For
example, if the ITE is aware that the ETE is located somewhere in the
public Internet, middlebox reassembly testing should not be
Templin Expires January 17, 2013 [Page 20]
Internet-Draft SEAL July 2012
necessary. If the ITE is aware that the ETE is located behind a NAT
or a firewall, however, then middlebox reassembly testing is
recommended.
The ITE can perform a middlebox reassembly test by selecting a data
packet to be used as a probe. While performing the test with real
data packets, the ITE should select only inner packets that are no
larger than (1500-HLEN) bytes for testing purposes. The ITE can also
construct a dummy probe packet instead of using ordinary SEAL data
packets.
To generate a dummy probe packet, the ITE creates a packet buffer
beginning with the same outer headers, SEAL header and inner network
layer header that would appear in an ordinary data packet, then pads
the packet with random data to a length that is at least 128 bytes
but no longer than (1500-HLEN) bytes. The ITE then writes the value
'0' in the inner network layer TTL (for IPv4) or Hop Limit (for IPv6)
field.
The ITE then sets (C=0; R=0) in the SEAL header of the probe packet
and sets the NEXTHDR field to the inner network layer protocol type.
(The ITE may also set A=1 if it requires a positive acknowledgement;
otherwise, it sets A=0.) Next, the ITE sets LINK_ID and LEVEL to the
appropriate values for this ETE link path, sets Identification and
I=1 (when USE_ID is TRUE), then finally calculates the ICV and sets
V=1(when USE_ICV is TRUE).
The ITE then encapsulates the probe packet in the appropriate outer
headers, splits it into two outer IPv4 fragments, then sends both
fragments over the same ETE link path.
The ITE should send a series of probe packets (e.g., 3-5 probes with
1sec intervals between tests) instead of a single isolated probe in
case of packet loss. If the ETE returns an SCMP PTB message with MTU
!= 0, then the ETE link path correctly supports fragmentation;
otherwise, the ITE enables stateful MTU determination for this ETE
link path as specified in Section 4.4.9.
(Examples of middleboxes that may perform reassembly include stateful
NATs and firewalls. Such devices could still allow for stateless MTU
determination if they gather the fragments of a fragmented IPv4 SEAL
data packet for packet analysis purposes but then forward the
fragments on to the final destination rather than forwarding the
reassembled packet.)
Templin Expires January 17, 2013 [Page 21]
Internet-Draft SEAL July 2012
4.4.9. Stateful MTU Determination
SEAL supports a stateless MTU determination capability, however the
ITE may in some instances wish to impose a stateful MTU limit on a
particular ETE link path. For example, when the ETE is situated
behind a middlebox that performs IPv4 reassembly (see: Section 4.4.8)
it is imperative that fragmentation be avoided. In other instances
(e.g., when the ETE link path includes performance-constrained
links), the ITE may deem it necessary to cache a conservative static
MTU in order to avoid sending large packets that would only be
dropped due to an MTU restriction somewhere on the path.
To determine a static MTU value, the ITE can send a series of dummy
probe packets of various sizes to the ETE with A=1 in the SEAL header
and DF=1 in the outer IP header. The ITE can then cache the size 'S'
of the largest packet for which it receives a probe reply from the
ETE by setting PATH_MTU=MAX((S-HLEN), 1500) for this ETE link path.
For example, the ITE could send probe packets of 4KB, followed by
2KB, followed by 1792 bytes, etc. While probing, the ITE processes
any ICMP PTB message it receives as a potential indication of probe
failure then discards the message.
4.4.10. Detecting Path MTU Changes
When stateful MTU determination is used, the ITE can periodically
reset PATH_MTU and/or re-probe the path to determine whether PATH_MTU
has increased. If the path still has a too-small MTU, the ITE will
receive a PTB message that reports a smaller size.
For IPv4 ETE link paths, when the path correctly implements
fragmentation and RATE_LIMIT is TRUE, the ITE can periodically reset
RATE_LIMIT=FALSE to determine whether the path still requires rate
limiting. If the ITE receives an SPTB message it should again set
RATE_LIMIT=TRUE.
4.5. ETE Specification
4.5.1. Minimum Reassembly Buffer Requirements
For IPv6, the ETE must configure a minimum reassembly buffer size of
1500 bytes for the reassembly of outer IPv6 packets (see: [RFC2460].
For IPv4, the ETE must also configure a minimum reassembly buffer
size of 1500 bytes for the reassembly of outer IPv4 packets, i.e.,
even though the true minimum reassembly size for IPv4 is only 576
bytes [RFC1122].
In addition to this outer reassembly buffer requirement, the ETE must
Templin Expires January 17, 2013 [Page 22]
Internet-Draft SEAL July 2012
further configure a minimum SEAL reassembly buffer size of (1500 +
HLEN) bytes for the reassembly of segmented SEAL packets (see:
Section 4.5.4).
4.5.2. Tunnel Neighbor Soft State
When data origin authentication and integrity checking is required,
the ETE maintains a per-ITE ICV calculation algorithm and a symmetric
secret key to verify the ICV. When per-packet identification is
required, the ETE also maintains a window of Identification values
for the packets it has recently received from this ITE.
When the tunnel neighbor relationship is bidirectional, the ETE
further maintains a per ETE link path mapping of outer IP and
transport layer addresses to the LINK_ID that appears in packets
received from the ITE.
4.5.3. IP-Layer Reassembly
The ETE should maintain conservative reassembly cache high- and low-
water marks. When the size of the reassembly cache exceeds this
high-water mark, the ETE should actively discard stale incomplete
reassemblies (e.g., using an Active Queue Management (AQM) strategy)
until the size falls below the low-water mark. The ETE should also
actively discard any pending reassemblies that clearly have no
opportunity for completion, e.g., when a considerable number of new
fragments have arrived before a fragment that completes a pending
reassembly arrives.
The ETE processes non-SEAL IP packets as specified in the normative
references, i.e., it performs any necessary IP reassembly then
discards the packet if it is larger than the reassembly buffer size
or delivers the (fully-reassembled) packet to the appropriate upper
layer protocol module.
For SEAL packets, the ITE performs any necessary IP reassembly then
submits the packet for SEAL decapsulation as specified in Section
4.5.4. (Note that if the packet is larger than the reassembly buffer
size, the ITE still returns the leading portion of the (partially)
reassembled packet.)
4.5.4. Decapsulation and Re-Encapsulation
For each SEAL packet accepted for decapsulation, when I==1 the ETE
first examines the Identification field. If the Identification is
not within the window of acceptable values for this ITE, the ETE
silently discards the packet.
Templin Expires January 17, 2013 [Page 23]
Internet-Draft SEAL July 2012
Next, if V==1 the ETE verifies the ICV value (with the ICV field
itself reset to 0) and silently discards the packet if the value is
incorrect.
Next, if the packet arrived as multiple IPv4 fragments and L ==0, the
ETE sends an SPTB message back to the ITE with MTU set to the size of
the largest fragment received minus HLEN (see: Section 4.6.1.1).
Next, if the packet arrived as multiple IP fragments and the inner
packet is larger than 1500 bytes, the ETE silently discards the
packet; otherwise, it continues to process the packet.
Next, if there is an incorrect value in a SEAL header field (e.g., an
incorrect "VER" field value), the ETE discards the packet. If the
SEAL header has C==0, the ETE also returns an SCMP "Parameter
Problem" (SPP) message (see Section 4.6.1.2).
Next, if the SEAL header has C==1, the ETE processes the packet as an
SCMP packet as specified in Section 4.6.2. Otherwise, the ETE
continues to process the packet as a SEAL data packet.
Next, if the SEAL header has (M==1 || Offset!==0) the ETE checks to
see if the other segments of this already-segmented SEAL packet have
arrived, i.e., by looking for additional segments that have the same
outer IP source address, destination address, source transport port
number (if present) and SEAL Identification value. If the other
segments have already arrived, the ETE discards the SEAL header and
other outer headers from the non-initial segments and appends them
onto the end of the first segment. Otherwise, the ETE caches the
segment for at most 60 seconds while awaiting the arrival of its
partners.
Next, if the SEAL header in the (reassembled) packet has A==1, the
ETE sends an SPTB message back to the ITE with MTU=0 (see: Section
4.6.1.1).
Finally, the ETE discards the outer headers and processes the inner
packet according to the header type indicated in the SEAL NEXTHDR
field. If the inner (TTL / Hop Limit) field encodes the value 0, the
ETE silently discards the packet. Otherwise, if the next hop toward
the inner destination address is via a different interface than the
SEAL packet arrived on, the ETE discards the SEAL header and delivers
the inner packet either to the local host or to the next hop
interface if the packet is not destined to the local host.
If the next hop is on the same interface the SEAL packet arrived on,
however, the ETE submits the packet for SEAL re-encapsulation
beginning with the specification in Section 4.4.3 above and without
Templin Expires January 17, 2013 [Page 24]
Internet-Draft SEAL July 2012
decrementing the value in the inner (TTL / Hop Limit) field. In this
process, the packet remains within the tunnel (i.e., it does not exit
and then re-enter the tunnel); hence, the packet is not discarded if
the LEVEL field in the SEAL header contains the value 0.
4.6. The SEAL Control Message Protocol (SCMP)
SEAL provides a companion SEAL Control Message Protocol (SCMP) that
uses the same message types and formats as for the Internet Control
Message Protocol for IPv6 (ICMPv6) [RFC4443]. As for ICMPv6, each
SCMP message includes a 32-bit header and a variable-length body.
The ITE encapsulates the SCMP message in a SEAL header and outer
headers as shown in Figure 3:
+--------------------+
~ outer IP header ~
+--------------------+
~ other outer hdrs ~
+--------------------+
~ SEAL Header ~
+--------------------+ +--------------------+
| SCMP message header| --> | SCMP message header|
+--------------------+ +--------------------+
| | --> | |
~ SCMP message body ~ --> ~ SCMP message body ~
| | --> | |
+--------------------+ +--------------------+
SCMP Message SCMP Packet
before encapsulation after encapsulation
Figure 3: SCMP Message Encapsulation
The following sections specify the generation, processing and
relaying of SCMP messages.
4.6.1. Generating SCMP Error Messages
ETEs generate SCMP error messages in response to receiving certain
SEAL data packets using the format shown in Figure 4:
Templin Expires January 17, 2013 [Page 25]
Internet-Draft SEAL July 2012
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Type | Code | Checksum |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Type-Specific Data |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| As much of the invoking SEAL data packet as possible |
~ (beginning with the SEAL header) without the SCMP ~
| packet exceeding MINMTU bytes (*) |
(*) also known as the "packet-in-error"
Figure 4: SCMP Error Message Format
The error message includes the 32-bit SCMP message header, followed
by a 32-bit Type-Specific Data field, followed by the leading portion
of the invoking SEAL data packet beginning with the SEAL header as
the "packet-in-error". The packet-in-error includes as much of the
invoking packet as possible extending to a length that would not
cause the entire SCMP packet following outer encapsulation to exceed
MINMTU bytes.
When the ETE processes a SEAL data packet for which the
Identification and ICV values are correct but an error must be
returned, it prepares an SCMP error message as shown in Figure 4.
The ETE sets the Type and Code fields to the same values that would
appear in the corresponding ICMPv6 message [RFC4443], but calculates
the Checksum beginning with the SCMP message header using the
algorithm specified for ICMPv4 in [RFC0792].
The ETE next encapsulates the SCMP message in the requisite SEAL and
outer headers as shown in Figure 3. During encapsulation, the ETE
sets the outer destination address/port numbers of the SCMP packet to
the values associated with the ITE and sets the outer source address/
port numbers to its own outer address/port numbers.
The ETE then sets (C=1; A=0; R=0; L=0; X=0; M=0; Offset=0) in the
SEAL header, then sets I, V, NEXTHDR and LEVEL to the same values
that appeared in the SEAL header of the data packet. If the neighbor
relationship between the ITE and ETE is unidirectional, the ETE next
sets the LINK_ID field to the same value that appeared in the SEAL
header of the data packet. Otherwise, the ETE sets the LINK_ID field
to the value it would use in sending a SEAL packet to this ITE.
When I==1, the ETE next sets the Identification field to an
appropriate value for the ITE. If the neighbor relationship between
the ITE and ETE is unidirectional, the ETE sets the Identification
Templin Expires January 17, 2013 [Page 26]
Internet-Draft SEAL July 2012
field to the same value that appeared in the SEAL header of the data
packet. Otherwise, the ETE sets the Identification field to the
value it would use in sending the next SEAL packet to this ITE.
When V==1, the ETE then calculates and sets the ICV field the same as
specified for SEAL data packet encapsulation in Section 4.4.4.
Finally, the ETE sends the resulting SCMP packet to the ITE the same
as specified for SEAL data packets in Section 4.4.5.
The following sections describe additional considerations for various
SCMP error messages:
4.6.1.1. Generating SCMP Packet Too Big (SPTB) Messages
An ETE generates an SCMP "Packet Too Big" (SPTB) message when it
receives a SEAL data packet that arrived as multiple outer IPv4
fragments and for which L==0. The ETE prepares the SPTB message the
same as for the corresponding ICMPv6 PTB message, and writes the
length of the largest outer IP fragment received minus HLEN in the
MTU field of the message.
The ETE also generates an SPTB message when it accepts a SEAL
protocol data packet with A==1 in the SEAL header. The ETE prepares
the SPTB message the same as above, except that it writes the value 0
in the MTU field.
4.6.1.2. Generating Other SCMP Error Messages
An ETE generates an SCMP "Destination Unreachable" (SDU) message
under the same circumstances that an IPv6 system would generate an
ICMPv6 Destination Unreachable message.
An ETE generates an SCMP "Parameter Problem" (SPP) message when it
receives a SEAL packet with an incorrect value in the SEAL header.
TEs generate other SCMP message types using methods and procedures
specified in other documents. For example, SCMP message types used
for tunnel neighbor coordinations are specified in VET
[I-D.templin-intarea-vet].
4.6.2. Processing SCMP Error Messages
An ITE may receive SCMP messages with C==1 in the SEAL header after
sending packets to an ETE. The ITE first verifies that the outer
addresses of the SCMP packet are correct, and (when I==1) that the
Identification field contains an acceptable value. The ITE next
verifies that the SEAL header fields are set correctly as specified
Templin Expires January 17, 2013 [Page 27]
caches or servers closer to users. However, queuing remains a major
intermittent component of latency.
Traditionally very low latency has only been available for a few
selected low rate applications, that confine their sending rate
within a specially carved-off portion of capacity, which is
prioritized over other traffic, e.g. Diffserv EF [RFC3246]. Up to
now it has not been possible to allow any number of low latency, high
throughput applications to seek to fully utilize available capacity,
because the capacity-seeking process itself causes too much queuing
delay.
To reduce this queuing delay caused by the capacity seeking process,
changes either to the network alone or to end-systems alone are in
progress. L4S involves a recognition that both approaches are
yielding diminishing returns:
o Recent state-of-the-art active queue management (AQM) in the
network, e.g. FQ-CoDel [RFC8290], PIE [RFC8033], Adaptive
RED [ARED01] ) has reduced queuing delay for all traffic, not just
a select few applications. However, no matter how good the AQM,
the capacity-seeking (sawtoothing) rate of TCP-like congestion
controls represents a lower limit that will either cause queuing
delay to vary or cause the link to be under-utilized. These AQMs
are tuned to allow a typical capacity-seeking Reno-friendly flow
to induce an average queue that roughly doubles the base RTT,
adding 5-15 ms of queuing on average (cf. 500 microseconds with
L4S for the same mix of long-running and web traffic). However,
for many applications low delay is not useful unless it is
consistently low. With these AQMs, 99th percentile queuing delay
is 20-30 ms (cf. 2 ms with the same traffic over L4S).
o Similarly, recent research into using e2e congestion control
without needing an AQM in the network (e.g.BBR [BBRv1],
[I-D.cardwell-iccrg-bbr-congestion-control]) seems to have hit a
similar lower limit to queuing delay of about 20ms on average (and
any additional BBRv1 flow adds another 20ms of queuing) but there
are also regular 25ms delay spikes due to bandwidth probes and
60ms spikes due to flow-starts.
L4S learns from the experience of Data Center TCP [RFC8257], which
shows the power of complementary changes both in the network and on
end-systems. DCTCP teaches us that two small but radical changes to
congestion control are needed to cut the two major outstanding causes
of queuing delay variability:
1. Far smaller rate variations (sawteeth) than Reno-friendly
congestion controls;
De Schepper, et al. Expires November 22, 2021 [Page 4]
Internet-Draft DualQ Coupled AQMs May 2021
2. A shift of smoothing and hence smoothing delay from network to
sender.
Without the former, a 'Classic' (e.g. Reno-friendly) flow's round
trip time (RTT) varies between roughly 1 and 2 times the base RTT
between the machines in question. Without the latter a 'Classic'
flow's response to changing events is delayed by a worst-case
(transcontinental) RTT, which could be hundreds of times the actual
smoothing delay needed for the RTT of typical traffic from localized
CDNs.
These changes are the two main features of the family of so-called
'Scalable' congestion controls (which includes DCTCP). Both these
changes only reduce delay in combination with a complementary change
in the network and they are both only feasible with ECN, not drop,
for the signalling:
1. The smaller sawteeth allow an extremely shallow ECN packet-
marking threshold in the queue.
2. And no smoothing in the network means that every fluctuation of
the queue is signalled immediately.
Without ECN, either of these would lead to very high loss levels.
But, with ECN, the resulting high marking levels are just signals,
not impairments.
However, until now, Scalable congestion controls (like DCTCP) did not
co-exist well in a shared ECN-capable queue with existing ECN-capable
TCP Reno [RFC5681] or Cubic [RFC8312] congestion controls ---
Scalable controls are so aggressive that these 'Classic' algorithms
would drive themselves to a small capacity share. Therefore, until
now, L4S controls could only be deployed where a clean-slate
environment could be arranged, such as in private data centres (hence
the name DCTCP).
This document specifies a `DualQ Coupled AQM' extension that solves
the problem of coexistence between Scalable and Classic flows,
without having to inspect flow identifiers. It is not like flow-
queuing approaches [RFC8290] that classify packets by flow identifier
into separate queues in order to isolate sparse flows from the higher
latency in the queues assigned to heavier flows. If a flow needs
both low delay and high throughput, having a queue to itself does not
isolate it from the harm it causes to itself. In contrast, L4S
addresses the root cause of the latency problem --- it is an enabler
for the smooth low latency scalable behaviour of Scalable congestion
controls, so that every packet in every flow can enjoy very low
De Schepper, et al. Expires November 22, 2021 [Page 5]
Internet-Draft DualQ Coupled AQMs May 2021
latency, then there is no need to isolate each flow into a separate
queue.
1.2. Scope
L4S involves complementary changes in the network and on end-systems:
Network: A DualQ Coupled AQM (defined in the present document);
End-system: A Scalable congestion control (defined in Section 2.1).
Packet identifier: The network and end-system parts of L4S can be
deployed incrementally, because they both identify L4S packets
using the experimentally assigned explicit congestion notification
(ECN) codepoints in the IP header: ECT(1) and CE [RFC8311]
[I-D.ietf-tsvwg-ecn-l4s-id].
Data Center TCP (DCTCP [RFC8257]) is an example of a Scalable
congestion control that has been deployed for some time in Linux,
Windows and FreeBSD operating systems and Relentless TCP [Mathis09]
is another example. During the progress of this document through the
IETF a number of other Scalable congestion controls were implemented,
e.g. TCP Prague [PragueLinux], QUIC Prague and the L4S variant of
SCREAM for real-time media [RFC8298]. (Note: after the v3.19 Linux
kernel, bugs were introduced into DCTCP's scalable behaviour and not
all the patches applied for L4S evaluation had been applied to the
mainline Linux kernel, which was at v5.5 at the time of writing. TCP
Prague includes these patches and is available for all these Linux
kernels).
The focus of this specification is to enable deployment of the
network part of the L4S service. Then, without any management
intervention, applications can exploit this new network capability as
their operating systems migrate to Scalable congestion controls,
which can then evolve _while_ their benefits are being enjoyed by
everyone on the Internet.
The DualQ Coupled AQM framework can incorporate any AQM designed for
a single queue that generates a statistical or deterministic mark/
drop probability driven by the queue dynamics. Pseudocode examples
of two different DualQ Coupled AQMs are given in the appendices. In
many cases the framework simplifies the basic control algorithm, and
requires little extra processing. Therefore it is believed the
Coupled AQM would be applicable and easy to deploy in all types of
buffers; buffers in cost-reduced mass-market residential equipment;
buffers in end-system stacks; buffers in carrier-scale equipment
including remote access servers, routers, firewalls and Ethernet
De Schepper, et al. Expires November 22, 2021 [Page 6]
Internet-Draft DualQ Coupled AQMs May 2021
switches; buffers in network interface cards, buffers in virtualized
network appliances, hypervisors, and so on.
For the public Internet, nearly all the benefit will typically be
achieved by deploying the Coupled AQM into either end of the access
link between a 'site' and the Internet, which is invariably the
bottleneck. Here, the term 'site' is used loosely to mean a home, an
office, a campus or mobile user equipment.
Latency is not the only concern of L4S:
o The 'Low Loss" part of the name denotes that L4S generally
achieves zero congestion loss (which would otherwise cause
retransmission delays), due to its use of ECN.
o The "Scalable throughput" part of the name denotes that the per-
flow throughput of Scalable congestion controls should scale
indefinitely, avoiding the imminent scaling problems with 'TCP-
Friendly' congestion control algorithms [RFC3649].
The former is clearly in scope of this AQM document. However, the
latter is an outcome of the end-system behaviour, and therefore
outside the scope of this AQM document, even though the AQM is an
enabler.
The overall L4S architecture [I-D.ietf-tsvwg-l4s-arch] gives more
detail, including on wider deployment aspects such as backwards
compatibility of Scalable congestion controls in bottlenecks where a
DualQ Coupled AQM has not been deployed. The supporting papers [PI2]
and [DCttH15] give the full rationale for the AQM's design, both
discursively and in more precise mathematical form.
1.3. Terminology
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in [RFC2119] when, and
only when, they appear in all capitals, as shown here.
The DualQ Coupled AQM uses two queues for two services. Each of the
following terms identifies both the service and the queue that
provides the service:
Classic service/queue: The Classic service is intended for all the
congestion control behaviours that co-exist with Reno [RFC5681]
(e.g. Reno itself, Cubic [RFC8312], TFRC [RFC5348]).
De Schepper, et al. Expires November 22, 2021 [Page 7]
Internet-Draft DualQ Coupled AQMs May 2021
Low-Latency, Low-Loss Scalable throughput (L4S) service/queue: The
'L4S' service is intended for traffic from scalable congestion
control algorithms, such as Data Center TCP [RFC8257]. The L4S
service is for more general traffic than just DCTCP--it allows the
set of congestion controls with similar scaling properties to
DCTCP to evolve (e.g. Relentless TCP [Mathis09], TCP
Prague [PragueLinux] and the L4S variant of SCREAM for real-time
media [RFC8298]).
Classic Congestion Control: A congestion control behaviour that can
co-exist with standard TCP Reno [RFC5681] without causing
significantly negative impact on its flow rate [RFC5033]. With
Classic congestion controls, as flow rate scales, the number of
round trips between congestion signals (losses or ECN marks) rises
with the flow rate. So it takes longer and longer to recover
after each congestion event. Therefore control of queuing and
utilization becomes very slack, and the slightest disturbance
prevents a high rate from being attained [RFC3649].
Scalable Congestion Control: A congestion control where the average
time from one congestion signal to the next (the recovery time)
remains invariant as the flow rate scales, all other factors being
equal. This maintains the same degree of control over queueing
and utilization whatever the flow rate, as well as ensuring that
high throughput is robust to disturbances. For instance, DCTCP
averages 2 congestion signals per round-trip whatever the flow
rate. For the public Internet a Scalable transport has to comply
with the requirements in Section 4 of [I-D.ietf-tsvwg-ecn-l4s-id]
(aka. the 'Prague L4S requirements').
C: Abbreviation for Classic, e.g. when used as a subscript.
L: Abbreviation for L4S, e.g. when used as a subscript.
The terms Classic or L4S can also qualify other nouns, such as
'codepoint', 'identifier', 'classification', 'packet', 'flow'.
For example: an L4S packet means a packet with an L4S identifier
sent from an L4S congestion control.
Both Classic and L4S queues can cope with a proportion of
unresponsive or less-responsive traffic as well (e.g. DNS, VoIP,
game sync datagrams), just as a single queue AQM can if this
traffic makes minimal contribution to queuing. The DualQ Coupled
AQM behaviour is defined to be similar to a single FIFO queue with
respect to unresponsive and overload traffic.
Reno-friendly: The subset of Classic traffic that excludes
unresponsive traffic and excludes experimental congestion controls
De Schepper, et al. Expires November 22, 2021 [Page 8]
Internet-Draft DualQ Coupled AQMs May 2021
intended to coexist with Reno but without always being strictly
friendly to it (as allowed by [RFC5033]). Reno-friendly is used
in place of 'TCP-friendly', given that friendliness is a property
of the congestion controller (Reno), not the wire protocol (TCP),
which is used with many different congestion control behaviours.
Classic ECN: The original Explicit Congestion Notification (ECN)
protocol [RFC3168], which requires ECN signals to be treated the
same as drops, both when generated in the network and when
responded to by the sender.
The names used for the four codepoints of the 2-bit IP-ECN field
are as defined in [RFC3168]: Not ECT, ECT(0), ECT(1) and CE, where
ECT stands for ECN-Capable Transport and CE stands for Congestion
Experienced.
1.4. Features
The AQM couples marking and/or dropping from the Classic queue to the
L4S queue in such a way that a flow will get roughly the same
throughput whichever it uses. Therefore both queues can feed into
the full capacity of a link and no rates need to be configured for
the queues. The L4S queue enables Scalable congestion controls like
DCTCP or TCP Prague to give very low and predictably low latency,
without compromising the performance of competing 'Classic' Internet
traffic.
Thousands of tests have been conducted in a typical fixed residential
broadband setting. Experiments used a range of base round trip
delays up to 100ms and link rates up to 200 Mb/s between the data
centre and home network, with varying amounts of background traffic
in both queues. For every L4S packet, the AQM kept the average
queuing delay below 1ms (or 2 packets where serialization delay
exceeded 1ms on slower links), with 99th percentile no worse than
2ms. No losses at all were introduced by the L4S AQM. Details of
the extensive experiments are available [PI2] [DCttH15].
Subjective testing was also conducted by multiple people all
simultaneously using very demanding high bandwidth low latency
applications over a single shared access link [L4Sdemo16]. In one
application, each user could use finger gestures to pan or zoom their
own high definition (HD) sub-window of a larger video scene generated
on the fly in 'the cloud' from a football match. Another user
wearing VR goggles was remotely receiving a feed from a 360-degree
camera in a racing car, again with the sub-window in their field of
vision generated on the fly in 'the cloud' dependent on their head
movements. Even though other users were also downloading large
amounts of L4S and Classic data, playing a gaming benchmark and
De Schepper, et al. Expires November 22, 2021 [Page 9]
Internet-Draft DualQ Coupled AQMs May 2021
watchings videos over the same 40Mb/s downstream broadband link,
latency was so low that the football picture appeared to stick to the
user's finger on the touch pad and the experience fed from the remote
camera did not noticeably lag head movements. All the L4S data (even
including the downloads) achieved the same very low latency. With an
alternative AQM, the video noticeably lagged behind the finger
gestures and head movements.
Unlike Diffserv Expedited Forwarding, the L4S queue does not have to
be limited to a small proportion of the link capacity in order to
achieve low delay. The L4S queue can be filled with a heavy load of
capacity-seeking flows (TCP Prague etc.) and still achieve low delay.
The L4S queue does not rely on the presence of other traffic in the
Classic queue that can be 'overtaken'. It gives low latency to L4S
traffic whether or not there is Classic traffic, and the latency of
Classic traffic does not suffer when a proportion of the traffic is
L4S.
The two queues are only necessary because:
o the large variations (sawteeth) of Classic flows need roughly a
base RTT of queuing delay to ensure full utilization
o Scalable flows do not need a queue to keep utilization high, but
they cannot keep latency predictably low if they are mixed with
Classic traffic,
The L4S queue has latency priority, but the coupling from the Classic
to the L4S AQM (explained below) ensures that it does not have
bandwidth priority over the Classic queue.
2. DualQ Coupled AQM
There are two main aspects to the approach:
o The Coupled AQM that addresses throughput equivalence between
Classic (e.g. Reno, Cubic) flows and L4S flows (that satisfy the
Prague L4S requirements).
o The Dual Queue structure that provides latency separation for L4S
flows to isolate them from the typically large Classic queue.
2.1. Coupled AQM
In the 1990s, the `TCP formula' was derived for the relationship
between the steady-state congestion window, cwnd, and the drop
probability, p of standard Reno congestion control [RFC5681] . To a
De Schepper, et al. Expires November 22, 2021 [Page 10]
Internet-Draft DualQ Coupled AQMs May 2021
first order approximation, the steady-state cwnd of Reno is inversely
proportional to the square root of p.
The design focuses on Reno as the worst case, because if it does no
harm to Reno, it will not harm Cubic or any traffic designed to be
friendly to Reno. TCP Cubic implements a Reno-compatibility mode,
which is relevant for typical RTTs under 20ms as long as the
throughput of a single flow is less than about 700Mb/s. In such
cases it can be assumed that Cubic traffic behaves similarly to Reno
(but with a slightly different constant of proportionality). The
term 'Classic' will be used for the collection of Reno-friendly
traffic including Cubic and potentially other experimental congestion
controls intended not to significantly impact the flow rate of Reno.
A supporting paper [PI2] includes the derivation of the equivalent
rate equation for DCTCP, for which cwnd is inversely proportional to
p (not the square root), where in this case p is the ECN marking
probability. DCTCP is not the only congestion control that behaves
like this, so the term 'Scalable' will be used for all similar
congestion control behaviours (see examples in Section 1.2). The
term 'L4S' is also used for traffic driven by a Scalable congestion
control that also complies with the additional 'Prague L4S&Internet-Draft SEAL July 2012
in Section 4.6.1. When V==1, the ITE then verifies the ICV value.
The ITE next verifies the Checksum value in the SCMP message header.
If any of these values are incorrect, the ITE silently discards the
message; otherwise, it processes the message as follows:
4.6.2.1. Processing SCMP PTB Messages
After an ITE sends a SEAL data packet to an ETE, it may receive an
SPTB message with a packet-in-error containing the leading portion of
the packet (see: Section 4.6.1.1). For IP SPTB messages with MTU==0,
the ITE processes the message as confirmation that the ETE received a
SEAL data packet with A==1 in the SEAL header. The ITE then discards
the message.
For SPTB messages with MTU != 0, the ITE processes the message as an
indication of a packet size limitation as follows. If the inner
packet is itself a SEAL packet, and the inner packet length is less
than 1500, the ITE reduces its MINMTU value for this ITE. If the
inner packet is a non-SEAL IPv4 packet and the inner packet length is
less than 1500, the ITE instead sets RATE_LIMIT=1. For all other
cases, if the inner packet length is larger than 1500 and the MTU
value is not substantially less than 1500 bytes, the value is likely
to reflect the true MTU of the restricting link on the path to the
ETE; otherwise, a router on the path may be generating runt
fragments.
In that case, the ITE can consult a plateau table (e.g., as described
in [RFC1191]) to rewrite the MTU value to a reduced size. For
example, if the ITE receives an IPv4 SPTB message with MTU==256 and
inner packet length 4KB, it can rewrite the MTU to 2KB. If the ITE
subsequently receives an IPv4 SPTB message with MTU==256 and inner
packet length 2KB, it can rewrite the MTU to 1792, etc., to a minimum
of 1500 bytes. If the ITE is performing stateful MTU determination
for this ETE link path, it then writes the new MTU value minus HLEN
in PATH_MTU.
The ITE then checks its forwarding tables to discover the previous
hop toward the source address of the inner packet. If the previous
hop is reached via the same tunnel interface the SPTB message arrived
on, the ITE relays the message to the previous hop. In order to
relay the message, the first writes zero in the Identification and
ICV fields of the SEAL header within the packet-in-error. The ITE
next rewrites the outer SEAL header fields with values corresponding
to the previous hop and recalculates the ICV using the ICV
calculation parameters associated with the previous hop. Next, the
ITE replaces the SPTB's outer headers with headers of the appropriate
protocol version and fills in the header fields as specified in
Sections 5.5.4-5.5.6 of [I-D.templin-intarea-vet], where the
Templin Expires January 17, 2013 [Page 28]
Internet-Draft SEAL July 2012
destination address/port correspond to the previous hop and the
source address/port correspond to the ITE. The ITE then sends the
message to the previous hop the same as if it were issuing a new SPTB
message. (Note that, in this process, the values within the SEAL
header of the packet-in-error are meaningless to the previous hop and
therefore cannot be used by the previous hop for authentication
purposes.)
If the previous hop is not reached via the same tunnel interface, the
ITE instead transcribes the message into a format appropriate for the
inner packet (i.e., the same as described for transcribing ICMP
messages in Section 4.4.7) and sends the resulting transcribed
message to the original source. (NB: if the inner packet within the
SPTB message is an IPv4 SEAL packet with DF==0, the ITE should set
DF=1 and re-calculate the IPv4 header checksum while transcribing the
message in order to avoid bogon filters.) The ITE then discards the
SPTB message.
Note that the ITE may receive an SPTB message from another ITE that
is at the head end of a nested level of encapsulation. The ITE has
no security associations with this nested ITE, hence it should
consider this SPTB message the same as if it had received an ICMP PTB
message from an ordinary router on the path to the ETE. That is, the
ITE should examine the packet-in-error field of the SPTB message and
only process the message if it is able to recognize the packet as one
it had previously sent.
4.6.2.2. Processing Other SCMP Error Messages
An ITE may receive an SDU message with an appropriate code under the
same circumstances that an IPv6 node would receive an ICMPv6
Destination Unreachable message. The ITE either transcribes or
relays the message toward the source address of the inner packet
within the packet-in-error the same as specified for SPTB messages in
Section 4.6.2.1.
An ITE may receive an SPP message when the ETE receives a SEAL packet
with an incorrect value in the SEAL header. The ITE should examine
the SEAL header within the packet-in-error to determine whether a
different setting should be used in subsequent packets, but does not
relay the message further.
TEs process other SCMP message types using methods and procedures
specified in other documents. For example, SCMP message types used
for tunnel neighbor coordinations are specified in VET
[I-D.templin-intarea-vet].
Templin Expires January 17, 2013 [Page 29]
Internet-Draft SEAL July 2012
5. Link Requirements
Subnetwork designers are expected to follow the recommendations in
Section 2 of [RFC3819] when configuring link MTUs.
6. End System Requirements
End systems are encouraged to implement end-to-end MTU assurance
(e.g., using Packetization Layer PMTUD per [RFC4821]) even if the
subnetwork is using SEAL.
7. Router Requirements
Routers within the subnetwork are expected to observe the router
requirements found in the normative references, including the
implementation of IP fragmentation and reassembly [RFC1812][RFC2460]
as well as the generation of ICMP messages [RFC0792][RFC4443].
8. Nested Encapsulation Considerations
SEAL supports nested tunneling for up to 8 layers of encapsulation.
In this model, the SEAL ITE has a tunnel neighbor relationship only
with ETEs at its own nesting level, i.e., it does not have a tunnel
neighbor relationship with other ITEs, nor with ETEs at other nesting
levels.
Therefore, when an ITE 'A' within an inner nesting level needs to
return an error message to an ITE 'B' within an outer nesting level,
it generates an ordinary ICMP error message the same as if it were an
ordinary router within the subnetwork. 'B' can then perform message
validation as specified in Section 4.4.7, but full message origin
authentication is not possible.
Since ordinary ICMP messages are used for coordinations between ITEs
at different nesting levels, nested SEAL encapsulations should only
be used when the ITEs are within a common administrative domain
and/or when there is no ICMP filtering middlebox such as a firewall
or NAT between them. An example would be a recursive nesting of
mobile networks, where the first network receives service from an
ISP, the second network receives service from the first network, the
third network receives service from the second network, etc.
NB: As an alternative, the SCMP protocol could be extended to allow
ITE 'A' to return an SCMP message to ITE 'B' rather than return an
ICMP message. This would conceptually allow the control messages to
Templin Expires January 17, 2013 [Page 30]
Internet-Draft SEAL July 2012
pass through firewalls and NATs, however it would give no more
message origin authentication assurance than for ordinary ICMP
messages. It was therefore determined that the complexity of
extending the SCMP protocol was of little value within the context of
the anticipated use cases for nested encapsulations.
9. IANA Considerations
The IANA is instructed to allocate a User Port number for "SEAL" in
the 'port-numbers' registry for the TCP and UDP protocols.
The IANA is further instructed to allocate an IP protocol number for
"SEAL" in the "protocol-numbers" registry.
Considerations for port and protocol number assignments appear in
[RFC2780][RFC5226][RFC6335].
10. Security Considerations
SEAL provides a segment-by-segment data origin authentication and
anti-replay service across the (potentially) multiple segments of a
re-encapsulating tunnel. It further provides a segment-by-segment
integrity check of the headers of encapsulated packets, but does not
verify the integrity of the rest of the packet beyond the headers
unless fragmentation is unavoidable. SEAL therefore considers full
message integrity checking, authentication and confidentiality as
end-to-end considerations in a manner that is compatible with
securing mechanisms such as TLS/SSL [RFC5246].
An amplification/reflection/buffer overflow attack is possible when
an attacker sends IP fragments with spoofed source addresses to an
ETE in an attempt to clog the ETE's reassembly buffer and/or cause
the ETE to generate a stream of SCMP messages returned to a victim
ITE. The SCMP message ICV, Identification, as well as the inner
headers of the packet-in-error, provide mitigation for the ETE to
detect and discard SEAL segments with spoofed source addresses.
The SEAL header is sent in-the-clear the same as for the outer IP and
other outer headers. In this respect, the threat model is no
different than for IPv6 extension headers. Unlike IPv6 extension
headers, however, the SEAL header can be protected by an integrity
check that also covers the inner packet headers.
Security issues that apply to tunneling in general are discussed in
[RFC6169].
Templin Expires January 17, 2013 [Page 31]
Internet-Draft SEAL July 2012
#x27;
requirements [I-D.ietf-tsvwg-ecn-l4s-id].
For safe co-existence, under stationary conditions, a Scalable flow
has to run at roughly the same rate as a Reno TCP flow (all other
factors being equal). So the drop or marking probability for Classic
traffic, p_C has to be distinct from the marking probability for L4S
traffic, p_L. The original ECN specification [RFC3168] required
these probabilities to be the same, but [RFC8311] updates RFC 3168 to
enable experiments in which these probabilities are different.
Also, to remain stable, Classic sources need the network to smooth
p_C so it changes relatively slowly. It is hard for a network node
to know the RTTs of all the flows, so a Classic AQM adds a _worst-
case_ RTT of smoothing delay (about 100-200 ms). In contrast, L4S
shifts responsibility for smoothing ECN feedback to the sender, which
only delays its response by its _own_ RTT, as well as allowing a more
immediate response if necessary.
The Coupled AQM achieves safe coexistence by making the Classic drop
probability p_C proportional to the square of the coupled L4S
probability p_CL. p_CL is an input to the instantaneous L4S marking
probability p_L but it changes as slowly as p_C. This makes the Reno
flow rate roughly equal the DCTCP flow rate, because the squaring of
p_CL counterbalances the square root of p_C in the 'TCP formula' of
Classic Reno congestion control.
De Schepper, et al. Expires November 22, 2021 [Page 11]
Internet-Draft DualQ Coupled AQMs May 2021
Stating this as a formula, the relation between Classic drop
probability, p_C, and the coupled L4S probability p_CL needs to take
the form:
p_C = ( p_CL / k )^2 (1)
where k is the constant of proportionality, which is termed the
coupling factor.
2.2. Dual Queue
Classic traffic needs to build a large queue to prevent under-
utilization. Therefore a separate queue is provided for L4S traffic,
and it is scheduled with priority over the Classic queue. Priority
is conditional to prevent starvation of Classic traffic.
Nonetheless, coupled marking ensures that giving priority to L4S
traffic still leaves the right amount of spare scheduling time for
Classic flows to each get equivalent throughput to DCTCP flows (all
other factors such as RTT being equal).
2.3. Traffic Classification
Both the Coupled AQM and DualQ mechanisms need an identifier to
distinguish L4S (L) and Classic (C) packets. Then the coupling
algorithm can achieve coexistence without having to inspect flow
identifiers, because it can apply the appropriate marking or dropping
probability to all flows of each type. A separate
specification [I-D.ietf-tsvwg-ecn-l4s-id] requires the network to
treat the ECT(1) and CE codepoints of the ECN field as this
identifier. An additional process document has proved necessary to
make the ECT(1) codepoint available for experimentation [RFC8311].
For policy reasons, an operator might choose to steer certain packets
(e.g. from certain flows or with certain addresses) out of the L
queue, even though they identify themselves as L4S by their ECN
codepoints. In such cases, [I-D.ietf-tsvwg-ecn-l4s-id] says that the
device "MUST NOT alter the end-to-end L4S ECN identifier", so that it
is preserved end-to-end. The aim is that each operator can choose
how it treats L4S traffic locally, but an individual operator does
not alter the identification of L4S packets, which would prevent
other operators downstream from making their own choices on how to
treat L4S traffic.
In addition, an operator could use other identifiers to classify
certain additional packet types into the L queue that it deems will
not risk harm to the L4S service. For instance addresses of specific
applications or hosts (see [I-D.ietf-tsvwg-ecn-l4s-id]), specific
De Schepper, et al. Expires November 22, 2021 [Page 12]
Internet-Draft DualQ Coupled AQMs May 2021
Diffserv codepoints such as EF (Expedited Forwarding) and Voice-Admit
service classes (see [I-D.briscoe-tsvwg-l4s-diffserv]), the Non-
Queue-Building (NQB) per-hop behaviour [I-D.ietf-tsvwg-nqb] or
certain protocols (e.g. ARP, DNS). Note that the mechanism only
reads these identifiers. [I-D.ietf-tsvwg-ecn-l4s-id] says it "MUST
NOT alter these non-ECN identifiers". Thus, the L queue is not
solely an L4S queue, it can be consider more generally as a low
latency queue.
2.4. Overall DualQ Coupled AQM Structure
Figure 1 shows the overall structure that any DualQ Coupled AQM is
likely to have. This schematic is intended to aid understanding of
the current designs of DualQ Coupled AQMs. However, it is not
intended to preclude other innovative ways of satisfying the
normative requirements in Section 2.5 that minimally define a DualQ
Coupled AQM.
The classifier on the left separates incoming traffic between the two
queues (L and C). Each queue has its own AQM that determines the
likelihood of marking or dropping (p_L and p_C). It has been
proved [PI2] that it is preferable to control load with a linear
controller, then square the output before applying it as a drop
probability to Reno-friendly traffic (because Reno congestion control
decreases its load proportional to the square-root of the increase in
drop). So, the AQM for Classic traffic needs to be implemented in
two stages: i) a base stage that outputs an internal probability p'
(pronounced p-prime); and ii) a squaring stage that outputs p_C,
where
p_C = (p')^2. (2)
Substituting for p_C in Eqn (1) gives:
p' = p_CL / k
So the slow-moving input to ECN marking in the L queue (the coupled
L4S probability) is:
p_CL = k*p'. (3)
The actual ECN marking probability p_L that is applied to the L queue
needs to track the immediate L queue delay under L-only congestion
conditions, as well as track p_CL under coupled congestion
conditions. So the L queue uses a native AQM that calculates a
probability p'_L as a function of the instantaneous L queue delay.
And, given the L queue has conditional priority over the C queue,
whenever the L queue grows, the AQM ought to apply marking
De Schepper, et al. Expires November 22, 2021 [Page 13]
Internet-Draft DualQ Coupled AQMs May 2021
probability p'_L, but p_L ought not to fall below p_CL. This
suggests:
p_L = max(p'_L, p_CL), (4)
which has also been found to work very well in practice.
The two transformations of p' in equations (2) and (3) implement the
required coupling given in equation (1) earlier.
The constant of proportionality or coupling factor, k, in equation
(1) determines the ratio between the congestion probabilities (loss
or marking) experienced by L4S and Classic traffic. Thus k
indirectly determines the ratio between L4S and Classic flow rates,
because flows (assuming they are responsive) adjust their rate in
response to congestion probability. Appendix C.2 gives guidance on
the choice of k and its effect on relative flow rates.
_________
| | ,------.
L4S queue | |===>| ECN |
,'| _______|_| |marker|\
<' | | `------'\\
//`' v ^ p_L \\
// ,-------. | \\
// |Native |p'_L | \\,.
// | L4S |--->(MAX) < | ___
,----------.// | AQM | ^ p_CL `\|.'Cond-`.
| IP-ECN |/ `-------' | / itional \
==>|Classifier| ,-------. (k*p') [ priority]==>
| |\ | Base | | \scheduler/
`----------'\\ | AQM |---->: ,'|`-.___.-'
\\ | |p' | <' |
\\ `-------' (p'^2) //`'
\\ ^ | //
\\,. | v p_C //
< | _________ .------.//
`\| | | | Drop |/
Classic |queue |===>|/mark |
__|______| `------'
Legend: ===> traffic flow; ---> control dependency.
Figure 1: DualQ Coupled AQM Schematic
After the AQMs have applied their dropping or marking, the scheduler
forwards their packets to the link. Even though the scheduler gives
De Schepper, et al. Expires November 22, 2021 [Page 14]
Internet-Draft DualQ Coupled AQMs May 2021
priority to the L queue, it is not as strong as the coupling from the
C queue. This is because, as the C queue grows, the base AQM applies
more congestion signals to L traffic (as well as C). As L flows
reduce their rate in response, they use less than the scheduling
share for L traffic. So, because the scheduler is work preserving,
it schedules any C traffic in the gaps.
Giving priority to the L queue has the benefit of very low L queue
delay, because the L queue is kept empty whenever L traffic is
controlled by the coupling. Also there only has to be a coupling in
one direction - from Classic to L4S. Priority has to be conditional
in some way to prevent the C queue starving under overload conditions
(see Section 4.1). With normal responsive traffic simple strict
priority would work, but it would make new Classic traffic wait until
its queue activated the coupling and L4S flows had in turn reduced
their rate enough to drain the L queue so that Classic traffic could
be scheduled. Giving a small weight or limited waiting time for C
traffic improves response times for short Classic messages, such as
DNS requests and improves Classic flow startup because immediate
capacity is available.
Example DualQ Coupled AQM algorithms called DualPI2 and Curvy RED are
given in Appendix A and Appendix B. Either example AQM can be used
to couple packet marking and dropping across a dual Q.
DualPI2 uses a Proportional-Integral (PI) controller as the Base AQM.
Indeed, this Base AQM with just the squared output and no L4S queue
can be used as a drop-in replacement for PIE [RFC8033], in which case
it is just called PI2 [PI2]. PI2 is a principled simplification of
PIE that is both more responsive and more stable in the face of
dynamically varying load.
Curvy RED is derived from RED [RFC2309], but its configuration
parameters are insensitive to link rate and it requires less
operations per packet. However, DualPI2 is more responsive and
stable over a wider range of RTTs than Curvy RED. As a consequence,
at the time of writing, DualPI2 has attracted more development and
evaluation attention than Curvy RED, leaving the Curvy RED design
incomplete and not so fully evaluated.
Both AQMs regulate their queue in units of time rather than bytes.
As already explained, this ensures configuration can be invariant for
different drain rates. With AQMs in a dualQ structure this is
particularly important because the drain rate of each queue can vary
rapidly as flows for the two queues arrive and depart, even if the
combined link rate is constant.
De Schepper, et al. Expires November 22, 2021 [Page 15]
Internet-Draft DualQ Coupled AQMs May 2021
It would be possible to control the queues with other alternative
AQMs, as long as the normative requirements (those expressed in
capitals) in Section 2.5 are observed.
2.5. Normative Requirements for a DualQ Coupled AQM
The following requirements are intended to capture only the essential
aspects of a DualQ Coupled AQM. They are intended to be independent
of the particular AQMs used for each queue.
2.5.1. Functional Requirements
A Dual Queue Coupled AQM implementation MUST comply with the
prerequisite L4S behaviours for any L4S network node (not just a
DualQ) as specified in section 5 of [I-D.ietf-tsvwg-ecn-l4s-id].
These primarily concern classification and remarking as briefly
summarized in Section 2.3 earlier. But there is also a subsection
(5.5) giving guidance on reducing the burstiness of the link
technology underlying any L4S AQM.
A Dual Queue Coupled AQM implementation MUST utilize two queues, each
with an AQM algorithm. The two queues can be part of a larger
queuing hierarchy [I-D.briscoe-tsvwg-l4s-diffserv].
The AQM algorithm for the low latency (L) queue MUST be able to apply
ECN marking to ECN-capable packets.
The scheduler draining the two queues MUST give L4S packets priority
over Classic, although priority MUST be bounded in order not to
starve Classic traffic. The scheduler SHOULD be work-conserving.
[I-D.ietf-tsvwg-ecn-l4s-id] defines the meaning of an ECN marking on
L4S traffic, relative to drop of Classic traffic. In order to ensure
coexistence of Classic and Scalable L4S traffic, it says, "The
likelihood that an AQM drops a Not-ECT Classic packet (p_C) MUST be
roughly proportional to the square of the likelihood that it would
have marked it if it had been an L4S packet (p_L)." The term
'likelihood' is used to allow for marking and dropping to be either
probabilistic or deterministic.
For the current specification, this translates into the following
requirement. A DualQ Coupled AQM MUST apply ECN marking to traffic
in the L queue that is no lower than that derived from the likelihood
of drop (or ECN marking) in the Classic queue using Eqn. (1).
The constant of proportionality, k, in Eqn (1) determines the
relative flow rates of Classic and L4S flows when the AQM concerned
is the bottleneck (all other factors being equal).
De Schepper, et al. Expires November 22, 2021 [Page 16]
Internet-Draft DualQ Coupled AQMs May 202111. Related Work
Section 3.1.7 of [RFC2764] provides a high-level sketch for
supporting large tunnel MTUs via a tunnel-level segmentation and
reassembly capability to avoid IP level fragmentation.
Section 3 of [RFC4459] describes inner and outer fragmentation at the
tunnel endpoints as alternatives for accommodating the tunnel MTU.
Section 4 of [RFC2460] specifies a method for inserting and
processing extension headers between the base IPv6 header and
transport layer protocol data. The SEAL header is inserted and
processed in exactly the same manner.
IPsec/AH is [RFC4301][RFC4301] is used for full message integrity
verification between tunnel endpoints, whereas SEAL only ensures
integrity for the inner packet headers. The AYIYA proposal
[I-D.massar-v6ops-ayiya] uses similar means for providing message
authentication and integrity.
The concepts of path MTU determination through the report of
fragmentation and extending the IPv4 Identification field were first
proposed in deliberations of the TCP-IP mailing list and the Path MTU
Discovery Working Group (MTUDWG) during the late 1980's and early
1990's. An historical analysis of the evolution of these concepts,
as well as the development of the eventual PMTUD mechanism, appears
in Appendix D of this document.
12. Implementation Status
An early implementation of the first revision of SEAL [RFC5320] is
available at: http://isatap.com/seal/pre-rfc5320.txt
13. Acknowledgments
The following individuals are acknowledged for helpful comments and
suggestions: Jari Arkko, Fred Baker, Iljitsch van Beijnum, Oliver
Bonaventure, Teco Boot, Bob Braden, Brian Carpenter, Steve Casner,
Ian Chakeres, Noel Chiappa, Remi Denis-Courmont, Remi Despres, Ralph
Droms, Aurnaud Ebalard, Gorry Fairhurst, Washam Fan, Dino Farinacci,
Joel Halpern, Sam Hartman, John Heffner, Thomas Henderson, Bob
Hinden, Christian Huitema, Eliot Lear, Darrel Lewis, Joe Macker, Matt
Mathis, Erik Nordmark, Dan Romascanu, Dave Thaler, Joe Touch, Mark
Townsley, Ole Troan, Margaret Wasserman, Magnus Westerlund, Robin
Whittle, James Woodyatt, and members of the Boeing Research &
Technology NST DC&NT group.
Templin Expires January 17, 2013 [Page 32]
Internet-Draft SEAL July 2012
Discussions with colleagues following the publication of [RFC5320]
have provided useful insights that have resulted in significant
improvements to this, the Second Edition of SEAL.
Path MTU determination through the report of fragmentation was first
proposed by Charles Lynn on the TCP-IP mailing list in 1987.
Extending the IP identification field was first proposed by Steve
Deering on the MTUDWG mailing list in 1989.
14. References
14.1. Normative References
[RFC0791] Postel, J., "Internet Protocol", STD 5, RFC 791,
September 1981.
[RFC0792] Postel, J., "Internet Control Message Protocol", STD 5,
RFC 792, September 1981.
[RFC1122] Braden, R., "Requirements for Internet Hosts -
Communication Layers", STD 3, RFC 1122, October 1989.
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, March 1997.
[RFC2460] Deering, S. and R. Hinden, "Internet Protocol, Version 6
(IPv6) Specification", RFC 2460, December 1998.
[RFC3971] Arkko, J., Kempf, J., Zill, B., and P. Nikander, "SEcure
Neighbor Discovery (SEND)", RFC 3971, March 2005.
[RFC4443] Conta, A., Deering, S., and M. Gupta, "Internet Control
Message Protocol (ICMPv6) for the Internet Protocol
Version 6 (IPv6) Specification", RFC 4443, March 2006.
[RFC4861] Narten, T., Nordmark, E., Simpson, W., and H. Soliman,
"Neighbor Discovery for IP version 6 (IPv6)", RFC 4861,
September 2007.
14.2. Informative References
[FOLK] Shannon, C., Moore, D., and k. claffy, "Beyond Folklore:
Observations on Fragmented Traffic", December 2002.
[FRAG] Kent, C. and J. Mogul, "Fragmentation Considered Harmful",
October 1987.
Templin Expires January 17, 2013 [Page 33]
Internet-Draft SEAL July 2012
[I-D.generic-6man-tunfrag]
Templin, F., "IPv6 Path MTU Updates",
draft-generic-6man-tunfrag-05 (work in progress),
July 2012.
[I-D.ietf-intarea-ipv4-id-update]
Touch, J., "Updated Specification of the IPv4 ID Field",
draft-ietf-intarea-ipv4-id-update-05 (work in progress),
May 2012.
[I-D.ietf-savi-framework]
Wu, J., Bi, J., Bagnulo, M., Baker, F., and C. Vogt,
"Source Address Validation Improvement Framework",
draft-ietf-savi-framework-06 (work in progress),
January 2012.
[I-D.massar-v6ops-ayiya]
Massar, J., "AYIYA: Anything In Anything",
draft-massar-v6ops-ayiya-02 (work in progress), July 2004.
[I-D.templin-aero]
Templin, F., "Asymmetric Extended Route Optimization
(AERO)", draft-templin-aero-08 (work in progress),
February 2012.
[I-D.templin-intarea-vet]
Templin, F., "Virtual Enterprise Traversal (VET)",
draft-templin-intarea-vet-33 (work in progress),
December 2011.
[I-D.templin-ironbis]
Templin, F., "The Internet Routing Overlay Network
(IRON)", draft-templin-ironbis-10 (work in progress),
December 2011.
[MTUDWG] "IETF MTU Discovery Working Group mailing list,
gatekeeper.dec.com/pub/DEC/WRL/mogul/mtudwg-log, November
1989 - February 1995.".
[RFC0994] International Organization for Standardization (ISO) and
American National Standards Institute (ANSI), "Final text
of DIS 8473, Protocol for Providing the Connectionless-
mode Network Service", RFC 994, March 1986.
[RFC1063] Mogul, J., Kent, C., Partridge, C., and K. McCloghrie, "IP
MTU discovery options", RFC 1063, July 1988.
[RFC1070] Hagens, R., Hall, N., and M. Rose, "Use of the Internet as
Templin Expires January 17, 2013 [Page 34]
Internet-Draft SEAL July 2012
a subnetwork for experimentation with the OSI network
layer", RFC 1070, February 1989.
[RFC1146] Zweig, J. and C. Partridge, "TCP alternate checksum
options", RFC 1146, March 1990.
[RFC1191] Mogul, J. and S. Deering, "Path MTU discovery", RFC 1191,
November 1990.
[RFC1701] Hanks, S., Li, T., Farinacci, D., and P. Traina, "Generic
Routing Encapsulation (GRE)", RFC 1701, October 1994.
[RFC1812] Baker, F., "Requirements for IP Version 4 Routers",
RFC 1812, June 1995.
[RFC1981] McCann, J., Deering, S., and J. Mogul, "Path MTU Discovery
for IP version 6", RFC 1981, August 1996.
[RFC2003] Perkins, C., "IP Encapsulation within IP", RFC 2003,
October 1996.
[RFC2473] Conta, A. and S. Deering, "Generic Packet Tunneling in
IPv6 Specification", RFC 2473, December 1998.
[RFC2675] Borman, D., Deering, S., and R. Hinden, "IPv6 Jumbograms",
RFC 2675, August 1999.
[RFC2764] Gleeson, B., Heinanen, J., Lin, A., Armitage, G., and A.
Malis, "A Framework for IP Based Virtual Private
Networks", RFC 2764, February 2000.
[RFC2780] Bradner, S. and V. Paxson, "IANA Allocation Guidelines For
Values In the Internet Protocol and Related Headers",
BCP 37, RFC 2780, March 2000.
[RFC2827] Ferguson, P. and D. Senie, "Network Ingress Filtering:
Defeating Denial of Service Attacks which employ IP Source
Address Spoofing", BCP 38, RFC 2827, May 2000.
[RFC2923] Lahey, K., "TCP Problems with Path MTU Discovery",
RFC 2923, September 2000.
[RFC3232] Reynolds, J., "Assigned Numbers: RFC 1700 is Replaced by
an On-line Database", RFC 3232, January 2002.
[RFC3366] Fairhurst, G. and L. Wood, "Advice to link designers on
link Automatic Repeat reQuest (ARQ)", BCP 62, RFC 3366,
August 2002.
Templin Expires January 17, 2013 [Page 35]
Internet-Draft SEAL July 2012
[RFC3819] Karn, P., Bormann, C., Fairhurst, G., Grossman, D.,
Ludwig, R., Mahdavi, J., Montenegro, G., Touch, J., and L.
Wood, "Advice for Internet Subnetwork Designers", BCP 89,
RFC 3819, July 2004.
[RFC4191] Draves, R. and D. Thaler, "Default Router Preferences and
More-Specific Routes", RFC 4191, November 2005.
[RFC4213] Nordmark, E. and R. Gilligan, "Basic Transition Mechanisms
for IPv6 Hosts and Routers", RFC 4213, October 2005.
[RFC4301] Kent, S. and K. Seo, "Security Architecture for the
Internet Protocol", RFC 4301, December 2005.
[RFC4302] Kent, S., "IP Authentication Header", RFC 4302,
December 2005.
[RFC4459] Savola, P., "MTU and Fragmentation Issues with In-the-
Network Tunneling", RFC 4459, April 2006.
[RFC4821] Mathis, M. and J. Heffner, "Packetization Layer Path MTU
Discovery", RFC 4821, March 2007.
[RFC4963] Heffner, J., Mathis, M., and B. Chandler, "IPv4 Reassembly
Errors at High Data Rates", RFC 4963, July 2007.
[RFC4987] Eddy, W., "TCP SYN Flooding Attacks and Common
Mitigations", RFC 4987, August 2007.
[RFC5226] Narten, T. and H. Alvestrand, "Guidelines for Writing an
IANA Considerations Section in RFCs", BCP 26, RFC 5226,
May 2008.
[RFC5246] Dierks, T. and E. Rescorla, "The Transport Layer Security
(TLS) Protocol Version 1.2", RFC 5246, August 2008.
[RFC5320] Templin, F., "The Subnetwork Encapsulation and Adaptation
Layer (SEAL)", RFC 5320, February 2010.
[RFC5445] Watson, M., "Basic Forward Error Correction (FEC)
Schemes", RFC 5445, March 2009.
[RFC5720] Templin, F., &
[I-D.ietf-tsvwg-ecn-l4s-id] says, "The constant of proportionality
(k) does not have to be standardised for interoperability, but a
value of 2 is RECOMMENDED."
Assuming Scalable congestion controls for the Internet will be as
aggressive as DCTCP, this will ensure their congestion window will be
roughly the same as that of a standards track TCP Reno congestion
control (Reno) [RFC5681] and other Reno-friendly controls, such as
TCP Cubic in its Reno-compatibility mode.
The choice of k is a matter of operator policy, and operators MAY
choose a different value using Table 1 and the guidelines in
Appendix C.2.
If multiple customers or users share capacity at a bottleneck
(e.g. in the Internet access link of a campus network), the
operator's choice of k will determine capacity sharing between the
flows of different customers. However, on the public Internet,
access network operators typically isolate customers from each other
with some form of layer-2 multiplexing (OFDM(A) in DOCSIS3.1, CDMA in
3G, SC-FDMA in LTE) or L3 scheduling (WRR in DSL), rather than
relying on host congestion controls to share capacity between
customers [RFC0970]. In such cases, the choice of k will solely
affect relative flow rates within each customer's access capacity,
not between customers. Also, k will not affect relative flow rates
at any times when all flows are Classic or all flows are L4S, and it
will not affect the relative throughput of small flows.
2.5.1.1. Requirements in Unexpected Cases
The flexibility to allow operator-specific classifiers (Section 2.3)
leads to the need to specify what the AQM in each queue ought to do
with packets that do not carry the ECN field expected for that queue.
It is expected that the AQM in each queue will inspect the ECN field
to determine what sort of congestion notification to signal, then it
will decide whether to apply congestion notification to this
particular packet, as follows:
o If a packet that does not carry an ECT(1) or CE codepoint is
classified into the L queue:
* if the packet is ECT(0), the L AQM SHOULD apply CE-marking
using a probability appropriate to Classic congestion control
and appropriate to the target delay in the L queue
* if the packet is Not-ECT, the appropriate action depends on
whether some other function is protecting the L queue from
misbehaving flows (e.g. per-flow queue
De Schepper, et al. Expires November 22, 2021 [Page 17]
Internet-Draft DualQ Coupled AQMs May 2021
protection [I-D.briscoe-docsis-q-protection] or latency
policing):
+ If separate queue protection is provided, the L AQM SHOULD
ignore the packet and forward it unchanged, meaning it
should not calculate whether to apply congestion
notification and it should neither drop nor CE-mark the
packet (for instance, the operator might classify EF traffic
that is unresponsive to drop into the L queue, alongside
responsive L4S-ECN traffic)
+ if separate queue protection is not provided, the L AQM
SHOULD apply drop using a drop probability appropriate to
Classic congestion control and appropriate to the target
delay in the L queue
o If a packet that carries an ECT(1) codepoint is classified into
the C queue:
* the C AQM SHOULD apply CE-marking using the coupled AQM
probability p_CL (= k*p').
The above requirements are worded as "SHOULDs", because operator-
specific classifiers are for flexibility, by definition. Therefore,
alternative actions might be appropriate in the operator's specific
circumstances. An example would be where the operator knows that
certain legacy traffic marked with one codepoint actually has a
congestion response associated with another codepoint.
If the DualQ Coupled AQM has detected overload, it MUST begin using
Classic drop, and continue until the overload episode has subsided.
Switching to drop if ECN marking is persistently high is required by
Section 7 of [RFC3168] and Section 4.2.1 of [RFC7567].
2.5.2. Management Requirements
2.5.2.1. Configuration
By default, a DualQ Coupled AQM SHOULD NOT need any configuration for
use at a bottleneck on the public Internet [RFC7567]. The following
parameters MAY be operator-configurable, e.g. to tune for non-
Internet settings:
o Optional packet classifier(s) to use in addition to the ECN field
(see Section 2.3);
o Expected typical RTT, which can be used to determine the queuing
delay of the Classic AQM at its operating point, in order to
De Schepper, et al. Expires November 22, 2021 [Page 18]
Internet-Draft DualQ Coupled AQMs May 2021
prevent typical lone flows from under-utilizing capacity. For
example:
* for the PI2 algorithm (Appendix A) the queuing delay target is
set to the typical RTT;
* for the Curvy RED algorithm (Appendix B) the queuing delay at
the desired operating point of the curvy ramp is configured to
encompass a typical RTT;
* if another Classic AQM was used, it would be likely to need an
operating point for the queue based on the typical RTT, and if
so it SHOULD be expressed in units of time.
An operating point that is manually calculated might be directly
configurable instead, e.g. for links with large numbers of flows
where under-utilization by a single flow would be unlikely.
o Expected maximum RTT, which can be used to set the stability
parameter(s) of the Classic AQM. For example:
* for the PI2 algorithm (Appendix A), the gain parameters of the
PI algorithm depend on the maximum RTT.
* for the Curvy RED algorithm (Appendix B) the smoothing
parameter is chosen to filter out transients in the queue
within a maximum RTT.
Stability parameter(s) that are manually calculated assuming a
maximum RTT might be directly configurable instead.
o Coupling factor, k (see Appendix C.2);
o A limit to the conditional priority of L4S. This is scheduler-
dependent, but it SHOULD be expressed as a relation between the
max delay of a C packet and an L packet. For example:
* for a WRR scheduler a weight ratio between L and C of w:1 means
that the maximum delay to a C packet is w times that of an L
packet.
* for a time-shifted FIFO (TS-FIFO) scheduler (see Section 4.1.1)
a time-shift of tshift means that the maximum delay to a C
packet is tshift greater than that of an L packet. tshift could
be expressed as a multiple of the typical RTT rather than as an
absolute delay.
De Schepper, et al. Expires November 22, 2021 [Page 19]
Internet-Draft DualQ Coupled AQMs May 2021
o The maximum Classic ECN marking probability, p_Cmax, before
switching over to drop.
2.5.2.2. Monitoring
An experimental DualQ Coupled AQM SHOULD allow the operator to
monitor each of the following operational statistics on demand, per
queue and per configurable sample interval, for performance
monitoring and perhaps also for accounting in some cases:
o Bits forwarded, from which utilization can be calculated;
o Total packets in the three categories: arrived, presented to the
AQM, and forwarded. The difference between the first two will
measure any non-AQM tail discard. The difference between the last
two will measure proactive AQM discard;
o ECN packets marked, non-ECN packets dropped, ECN packets dropped,
which can be combined with the three total packet counts above to
calculate marking and dropping probabilities;
o Queue delay (not including serialization delay of the head packet
or medium acquisition delay) - see further notes below.
Unlike the other statistics, queue delay cannot be captured in a
simple accumulating counter. Therefore the type of queue delay
statistics produced (mean, percentiles, etc.) will depend on
implementation constraints. To facilitate comparative evaluation
of different implementations and approaches, an implementation
SHOULD allow mean and 99th percentile queue delay to be derived
(per queue per sample interval). A relatively simple way to do
this would be to store a coarse-grained histogram of queue delay.
This could be done with a small number of bins with configurable
edges that represent contiguous ranges of queue delay. Then, over
a sample interval, each bin would accumulate a count of the number
of packets that had fallen within each range. The maximum queue
delay per queue per interval MAY also be recorded.
2.5.2.3. Anomaly Detection
An experimental DualQ Coupled AQM SHOULD asynchronously report the
following data about anomalous conditions:
o Start-time and duration of overload state.
A hysteresis mechanism SHOULD be used to prevent flapping in and
out of overload causing an event storm. For instance, exit from
overload state could trigger one report, but also latch a timer.
De Schepper, et al. Expires November 22, 2021 [Page 20]
Internet-Draft DualQ Coupled AQMs May 2021
Then, during that time, if the AQM enters and exits overload state
any number of times, the duration in overload state is accumulated
but no new report is generated until the first time the AQM is out
of overload once the timer has expired.
2.5.2.4. Deployment, Coexistence and Scaling
[RFC5706] suggests that deployment, coexistence and scaling should
also be covered as management requirements. The raison d'etre of the
DualQ Coupled AQM is to enable deployment and coexistence of Scalable
congestion controls - as incremental replacements for today's Reno-
friendly controls that do not scale with bandwidth-delay product.
Therefore there is no need to repeat these motivating issues here
given they are already explained in the Introduction and detailed in
the L4S architecture [I-D.ietf-tsvwg-l4s-arch].
The descriptions of specific DualQ Coupled AQM algorithms in the
appendices cover scaling of their configuration parameters, e.g. with
respect to RTT and sampling frequency.
3. IANA Considerations (to be removed by RFC Editor)
This specification contains no IANA considerations.
4. Security Considerations
4.1. Overload Handling
Where the interests of users or flows might conflict, it could be
necessary to police traffic to isolate any harm to the performance of
individual flows. However it is hard to avoid unintended side-
effects with policing, and in a trusted environment policing is not
necessary. Therefore per-flow policing
(e.g. [I-D.briscoe-docsis-q-protection]) needs to be separable from a
basic AQM, as an option under policy control.
However, a basic DualQ AQM does at least need to handle overload. A
useful objective would be for the overload behaviour of the DualQ AQM
to be at least no worse than a single queue AQM. However, a trade-
off needs to be made between complexity and the risk of either
traffic class harming the other. In each of the following three
subsections, an overload issue specific to the DualQ is described,
followed by proposed solution(s).
Under overload the higher priority L4S service will have to sacrifice
some aspect of its performance. Alternative solutions are provided
below that each relax a different factor: e.g. throughput, delay,
De Schepper, et al. Expires November 22, 2021 [Page 21]
Internet-Draft DualQ Coupled AQMs May 2021
drop. These choices need to be made either by the developer or by
operator policy, rather than by the IETF.
4.1.1. Avoiding Classic Starvation: Sacrifice L4S Throughput or Delay?
Priority of L4S is required to be conditional to avoid total
starvation of Classic by heavy L4S traffic. This raises the question
of whether to sacrifice L4S throughput or L4S delay (or some other
policy) to mitigate starvation of Classic:
Sacrifice L4S throughput: By using weighted round robin as the
conditional priority scheduler, the L4S service can sacrifice some
throughput during overload. This can either be thought of as
guaranteeing a minimum throughput service for Classic traffic, or
as guaranteeing a maximum delay for a packet at the head of the
Classic queue.
The scheduling weight of the Classic queue should be small
(e.g. 1/16). Then, in most traffic scenarios the scheduler will
not interfere and it will not need to - the coupling mechanism and
the end-systems will share out the capacity across both queues as
if it were a single pool. However, because the congestion
coupling only applies in one direction (from C to L), if L4S
traffic is over-aggressive or unresponsive, the scheduler weight
for Classic traffic will at least be large enough to ensure it
does not starve.
In cases where the ratio of L4S to Classic flows (e.g. 19:1) is
greater than the ratio of their scheduler weights (e.g. 15:1), the
L4S flows will get less than an equal share of the capacity, but
only slightly. For instance, with the example numbers given, each
L4S flow will get (15/16)/19 = 4.9% when ideally each would get
1/20=5%. In the rather specific case of an unresponsive flow
taking up just less than the capacity set aside for L4S
(e.g. 14/16 in the above example), using WRR could significantly
reduce the capacity left for any responsive L4S flows.
The scheduling weight of the Classic queue should not be too
small, otherwise a C packet at the head of the queue could be
excessively delayed by a continually busy L queue. For instance
if the Classic weight is 1/16, the maximum that a Classic packet
at the head of the queue can be delayed by L traffic is the
serialization delay of 15 MTU-sized packets.
Sacrifice L4S Delay: To control milder overload of responsive
traffic, particularly when close to the maximum congestion signal,
the operator could choose to control overload of the Classic queue
by allowing some delay to 'leak' across to the L4S queue. The
De Schepper, et al. Expires November 22, 2021 [Page 22]
Internet-Draft DualQ Coupled AQMs May 2021
scheduler can be made to behave like a single First-In First-Out
(FIFO) queue with different service times by implementing a very
simple conditional priority scheduler that could be called a
"time-shifted FIFO" (see the Modifier Earliest Deadline First
(MEDF) scheduler of [MEDF]). This scheduler adds tshift to the
queue delay of the next L4S packet, before comparing it with the
queue delay of the next Classic packet, then it selects the packet
with the greater adjusted queue delay. Under regular conditions,
this time-shifted FIFO scheduler behaves just like a strict
priority scheduler. But under moderate or high overload it
prevents starvation of the Classic queue, because the time-shift
(tshift) defines the maximum extra queuing delay of Classic
packets relative to L4S.
The example implementations in Appendix A and Appendix B could both
be implemented with either policy.
4.1.2. Congestion Signal Saturation: Introduce L4S Drop or Delay?
To keep the throughput of both L4S and Classic flows roughly equal
over the full load range, a different control strategy needs to be
defined above the point where one AQM first saturates to a
probability of 100% leaving no room to push back the load any harder.
If k>1, L4S will saturate first, even though saturation could be
caused by unresponsive traffic in either queue.
The term 'unresponsive"Routing and Addressing in Networks with
Global Enterprise Recursion (RANGER)", RFC 5720,
February 2010.
[RFC5927] Gont, F., "ICMP Attacks against TCP", RFC 5927, July 2010.
Templin Expires January 17, 2013 [Page 36]
Internet-Draft SEAL July 2012
[RFC6139] Russert, S., Fleischman, E., and F. Templin, "Routing and
Addressing in Networks with Global Enterprise Recursion
(RANGER) Scenarios", RFC 6139, February 2011.
[RFC6169] Krishnan, S., Thaler, D., and J. Hoagland, "Security
Concerns with IP Tunneling", RFC 6169, April 2011.
[RFC6335] Cotton, M., Eggert, L., Touch, J., Westerlund, M., and S.
Cheshire, "Internet Assigned Numbers Authority (IANA)
Procedures for the Management of the Service Name and
Transport Protocol Port Number Registry", BCP 165,
RFC 6335, August 2011.
[SIGCOMM] Luckie, M. and B. Stasiewicz, "Measuring Path MTU
Discovery Behavior", November 2010.
[TBIT] Medina, A., Allman, M., and S. Floyd, "Measuring
Interactions Between Transport Protocols and Middleboxes",
October 2004.
[TCP-IP] "Archive/Hypermail of Early TCP-IP Mail List,
http://www-mice.cs.ucl.ac.uk/multimedia/misc/tcp_ip/, May
1987 - May 1990.".
[WAND] Luckie, M., Cho, K., and B. Owens, "Inferring and
Debugging Path MTU Discovery Failures", October 2005.
Appendix A. Reliability
Although a SEAL tunnel may span an arbitrarily-large subnetwork
expanse, the IP layer sees the tunnel as a simple link that supports
the IP service model. Links with high bit error rates (BERs) (e.g.,
IEEE 802.11) use Automatic Repeat-ReQuest (ARQ) mechanisms [RFC3366]
to increase packet delivery ratios, while links with much lower BERs
typically omit such mechanisms. Since SEAL tunnels may traverse
arbitrarily-long paths over links of various types that are already
either performing or omitting ARQ as appropriate, it would therefore
be inefficient to require the tunnel endpoints to also perform ARQ.
Appendix B. Integrity
The SEAL header includes an integrity check field that covers the
SEAL header and at least the inner packet headers. This provides for
header integrity verification on a segment-by-segment basis for a
segmented re-encapsulating tunnel path.
Templin Expires January 17, 2013 [Page 37]
Internet-Draft SEAL July 2012
Fragmentation and reassembly schemes must also consider packet-
splicing errors, e.g., when two fragments from the same packet are
concatenated incorrectly, when a fragment from packet X is
reassembled with fragments from packet Y, etc. The primary sources
of such errors include implementation bugs and wrapping IPv4 ID
fields.
In particular, the IPv4 16-bit ID field can wrap with only 64K
packets with the same (src, dst, protocol)-tuple alive in the system
at a given time [RFC4963]. When the IPv4 ID field is re-written by a
middlebox such as a NAT or Firewall, ID field wrapping can occur with
even fewer packets alive in the system.
When outer IPv4 fragmentation is unavoidable, SEAL institutes rate
limiting so that the number of packets admitted into the tunnel by
the ITE does not exceed the number of unique packets that may be
alive within the Internet.
Appendix C. Transport Mode
SEAL can also be used in "transport-mode", e.g., when the inner layer
comprises upper-layer protocol data rather than an encapsulated IP
packet. For instance, TCP peers can negotiate the use of SEAL (e.g.,
by inserting an unspecified 'SEAL_OPTION' TCP option during
connection establishment) for the carriage of protocol data
encapsulated as IP/SEAL/TCP. In this sense, the "subnetwork" becomes
the entire end-to-end path between the TCP peers and may potentially
span the entire Internet.
If both TCPs agree on the use of SEAL, their protocol messages will
be carried as IP/SEAL/TCP and the connection will be serviced by the
SEAL protocol using TCP (instead of an encapsulating tunnel endpoint)
as the transport layer protocol. The SEAL protocol for transport
mode otherwise observes the same specifications as for Section 4.
Appendix D. Historic Evolution of PMTUD
The topic of Path MTU discovery (PMTUD) saw a flurry of discussion
and numerous proposals in the late 1980's through early 1990. The
initial problem was posed by Art Berggreen on May 22, 1987 in a
message to the TCP-IP discussion group [TCP-IP]. The discussion that
followed provided significant reference material for [FRAG]. An IETF
Path MTU Discovery Working Group [MTUDWG] was formed in late 1989
with charter to produce an RFC. Several variations on a very few
basic proposals were entertained, including:
Templin Expires January 17, 2013 [Page 38]
Internet-Draft SEAL July 2012
1. Routers record the PMTUD estimate in ICMP-like path probe
messages (proposed in [FRAG] and later [RFC1063])
2. The destination reports any fragmentation that occurs for packets
received with the "RF" (Report Fragmentation) bit set (Steve
Deering's 1989 adaptation of Charles Lynn's Nov. 1987 proposal)
3. A hybrid combination of 1) and Charles Lynn's Nov. 1987 (straw
RFC draft by McCloughrie, Fox and Mogul on Jan 12, 1990)
4. Combination of the Lynn proposal with TCP (Fred Bohle, Jan 30,
1990)
5. Fragmentation avoidance by setting "IP_DF" flag on all packets
and retransmitting if ICMPv4 "fragmentation needed" messages
occur (Geof Cooper's 1987 proposal; later adapted into [RFC1191]
by Mogul and Deering).
Option 1) seemed attractive to the group at the time, since it was
believed that routers would migrate more quickly than hosts. Option
2) was a strong contender, but repeated attempts to secure an "RF"
bit in the IPv4 header from the IESG failed and the proponents became
discouraged. 3) was abandoned because it was perceived as too
complicated, and 4) never received any apparent serious
consideration. Proposal 5) was a late entry into the discussion from
Steve Deering on Feb. 24th, 1990. The discussion group soon
thereafter seemingly lost track of all other proposals and adopted
5), which eventually evolved into [RFC1191] and later [RFC1981].
In retrospect, the "RF" bit postulated in 2) is not needed if a
"contract" is first established between the peers, as in proposal 4)
and a message to the MTUDWG mailing list from jrd@PTT.LCS.MIT.EDU on
Feb 19. 1990. These proposals saw little discussion or rebuttal, and
were dismissed based on the following the assertions:
o routers upgrade their software faster than hosts
o PCs could not reassemble fragmented packets
o Proteon and Wellfleet routers did not reproduce the "RF" bit
properly in fragmented packets
o Ethernet-FDDI bridges would need to perform fragmentation (i.e.,
"translucent" not "transparent" bridging)
o the 16-bit IP_ID field could wrap around and disrupt reassembly at
high packet arrival rates
Templin Expires January 17, 2013 [Page 39]
Internet-Draft SEAL July 2012
The first four assertions, although perhaps valid at the time, have
been overcome by historical events. The final assertion is addressed
by the mechanisms specified in SEAL.
Author's Address
Fred L. Templin (editor)
Boeing Research & Technology
P.O. Box 3707
Seattle, WA 98124
USA
Email: fltemplin@acm.org
Templin Expires January 17, 2013 [Page 40]