INTAREA Working Group S. Bryant
Internet-Draft U. Chunduri
Intended status: Informational T. Eckert
Expires: September 10, 2020 A. Clemm
Futurewei Technologies Inc.
March 09, 2020
Forwarding Layer Problem Statement
draft-bryant-arch-fwd-layer-ps-00
Abstract
This document considers the new use cases for IP together with the
network capabilities and services that will be needed to address
those use cases. It then looks at the underlying packet requirements
and considers the changing deployment models and the issues with
existing packet designs that need to be addressed. It concludes by
looking at some parameters of a solution.
Status of This Memo
This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet-
Drafts is at https://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
This Internet-Draft will expire on September 10, 2020.
Copyright Notice
Copyright (c) 2020 IETF Trust and the persons identified as the
document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents
(https://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must
Bryant, et al. Expires September 10, 2020 [Page 1]
Internet-Draft FWD PS March 2020
include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as
described in the Simplified BSD License.
Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3
1.1. Forwarding Layer . . . . . . . . . . . . . . . . . . . . 4
2. New Use Cases for packet networks . . . . . . . . . . . . . . 5
2.1. Video and AR/VR . . . . . . . . . . . . . . . . . . . . . 5
2.2. Role of Fixed Networks in 5G and Beyond 5G . . . . . . . 6
2.3. ITU-T Focus Group Network-2030 . . . . . . . . . . . . . 6
3. New Network Capabilities and Services . . . . . . . . . . . . 8
3.1. New Services . . . . . . . . . . . . . . . . . . . . . . 8
3.2. New Capabilities . . . . . . . . . . . . . . . . . . . . 9
4. Underlying New Requirements . . . . . . . . . . . . . . . . . 10
4.1. Better than Best Effort . . . . . . . . . . . . . . . . . 10
4.2. Efficient Packet Design . . . . . . . . . . . . . . . . . 10
4.3. Forwarding Identifiers . . . . . . . . . . . . . . . . . 11
4.4. Operational visibility . . . . . . . . . . . . . . . . . 12
4.5. Holistic solution . . . . . . . . . . . . . . . . . . . . 12
5. Deployment Models . . . . . . . . . . . . . . . . . . . . . . 13
5.1. Edge-2-Edge Model . . . . . . . . . . . . . . . . . . . . 13
5.2. End-2-End Model Single Provider . . . . . . . . . . . . . 13
5.3. End-2-End Model with multiple Providers . . . . . . . . . 14
5.4. Embedded Service . . . . . . . . . . . . . . . . . . . . 15
5.5. Embedded Global Service . . . . . . . . . . . . . . . . . 16
6. Existing Protocol and Layering Challenges and Gaps . . . . . 17
6.1. Challenges with IPv6 . . . . . . . . . . . . . . . . . . 17
6.1.1. The End-to-End Model . . . . . . . . . . . . . . . . 17
6.1.2. Fixed Address Length . . . . . . . . . . . . . . . . 21
6.2. Better Than Best Effort E2E Network Services . . . . . . 22
6.3. Adaptive Bit-rate Video streaming . . . . . . . . . . . . 23
6.4. DetNet and Higher Precision Networking Service . . . . . 24
6.5. Forwarding Plane vs. Control Plane . . . . . . . . . . . 24
6.6. User-Network/Network-User Interface Signaling . . . . . . 26
7. Candidate Solution Directions . . . . . . . . . . . . . . . . 26
7.1. Variable Length Addresses . . . . . . . . . . . . . . . . 27
7.2. Address Semantics . . . . . . . . . . . . . . . . . . . . 27
7.3. Multiple Instructions . . . . . . . . . . . . . . . . . . 28
7.4. Node and Path Specific Processing Instructions . . . . . 28
7.5. Integrated Assurance and Verification . . . . . . . . . . 28
8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 29
9. Security Considerations . . . . . . . . . . . . . . . . . . . 29
10. Appendix 1: Expanded Summary of Sub-G1 Use Cases . . . . . . 29
10.1. Holographic-type communications . . . . . . . . . . . . 29
10.2. Tactile Internet for Remote Operations . . . . . . . . . 30
10.3. Space-Terrestrial Integrated Networks . . . . . . . . . 30
Bryant, et al. Expires September 10, 2020 [Page 2]
Internet-Draft FWD PS March 2020
10.4. ManyNets . . . . . . . . . . . . . . . . . . . . . . . . 31
11. Appendix 2: Expanded Summary of Sub-G2 New Network
Capabilities and Services . . . . . . . . . . . . . . . . . . 32
11.1. New Services . . . . . . . . . . . . . . . . . . . . . . 32
11.1.1. High-Precision Communications Services . . . . . . . 32
11.1.2. In-time Services . . . . . . . . . . . . . . . . . . 33
11.1.3. On-time Services . . . . . . . . . . . . . . . . . . 33
11.1.4. Coordinated Services . . . . . . . . . . . . . . . . 34
11.1.5. Qualitative Communication Services . . . . . . . . . 34
11.2. New Capabilities . . . . . . . . . . . . . . . . . . . . 34
11.2.1. Manageability . . . . . . . . . . . . . . . . . . . 34
11.2.2. High Programmability and Agile Lifecycle . . . . . . 35
11.2.3. Security . . . . . . . . . . . . . . . . . . . . . . 36
11.2.4. Trustworthiness . . . . . . . . . . . . . . . . . . 37
11.2.5. Resilience . . . . . . . . . . . . . . . . . . . . . 38
11.2.6. Privacy-Sensitive . . . . . . . . . . . . . . . . . 38
11.2.7. Accountability and Verifiability . . . . . . . . . . 39
12. References . . . . . . . . . . . . . . . . . . . . . . . . . 40
12.1. Normative References . . . . . . . . . . . . . . . . . . 40
12.2. Informative References . . . . . . . . . . . . . . . . . 40
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 45
1. Introduction
There is an emerging set of new requirements that exceed the network
and transport services of the current Internet, which only delivers
"best effort" service. While many controlled or private networks
include further services, such as other DiffServ QoS in addition to
best effort and traffic engineering with bandwidth guarantees, the
solutions used today only support walled gardens and are thus not
available to application service providers and consumers across the
Internet.
The purpose of this document is to look at current, evolving and
future use cases and to examine the shortcomings that the existing
network and transport layer protocols a well as their associated
control plane need to overcome to meet these needs.
The IETF is the body responsible for the long term evolution of the
IP protocol suit, but is missing a work track to discuss the long-
term Internet network architecture evolution. In particular it lacks
a programme for the long term evolution of IP itself.
Approximately 30 years ago, the IETF started a process to
revolutionize the IPv4 [RFC0791] Internet Protocol. In this process,
researchers, industry, and service providers got together, and
brought up a number of new proposals, and worked toward a successor
to IPv4, which became IPv6 [RFC2460] and later [RFC8200].
Bryant, et al. Expires September 10, 2020 [Page 3]
Internet-Draft FWD PS March 2020
30 years later, there is heavy resistance to anything more than minor
incremental evolutions to IPv6. There are a number of reasons for
this ranging from opinions that all future IP needs can be met
through minor incremental evolutions to fears that major proposals
for innovation at the IP would be an unwelcome disrupter to the
current business of the vendors or the service providers.
The authors take no position on the scale of the problem or the
difficulties of deploying any solutions at scale in the Internet.
What we seek to do is to establish the scope and nature of the
problem. A decision on which aspects of the problem are economically
tractable is out of scope of this text, but technologies to support
monetization are not.
As a problem statement, this documents goal is to not propose or
promote specific solutions to the problems raised. Instead it uses
references to not Internet adopted, but proposed or existing
solutions only as example evidence that the described problem can
actually be solved.
Because the document does not propose specific solutions, it also
does not attempt to structure the problem description in a way that
would identify sub-set of problems to be resolved by specific
solution components.
The purpose of this text is thus to stimulate discussion on the
emerging needs of the forwarding layer and to start the process of
determining how they are best satisfied within the IETF protocol
suite.
1.1. Forwarding Layer
The term "forwarding layer" is used in this document because none of
the standard terms encompass the parts of the network stack that need
attention to address the needs of the applications that are foreseen.
It is possible that development work will need to reach down to layer
2.5 in order to ensure that packets are handled correctly down to the
physical layer. The MAC layer is quite sophisticated and includes
its own switching function so we need to be sure that the good work
done in the network layer is not undone lower down the stack.
Equally it is possible that development work will need to reach into
the transport layer to address new approaches to congestion and to
ensure that the network layer understands the requirements placed on
it by the application. An open mind is needed on the boundaries of
the layers as they exist today when analyzing the consequential
network changes needed to support the evolving application space.
Bryant, et al. Expires September 10, 2020 [Page 4]
Internet-Draft FWD PS March 2020
In the network layer itself, this document is only concerned with the
forwarding component, not path selection or the other components of
routing.
Thus we use the term forwarding layer to describe the scope of the
stack that this document addresses.
2. New Use Cases for packet networks
This section summarizes the use case areas that have been observed by
the authors and are considered relevant to the following analysis of
gaps.
This section is structured into sub-sections discussing either group
of use cases directly or the work of specific groups that are
identifying use cases and that may also work on identifying issues
and or proposed architectures or solutions for them.
Subsections are ordered from what might be considered to be the most
near-term use cases to the potentially most far reaching ones.
2.1. Video and AR/VR
Audio/Video streaming for production, entertainment, surveillance and
other purposes, and interactive audio/video are the most ubiquitous
applications on the Internet and private IP networks after web-
services. They have grown primarily through an evolution of the
applications to work with the constraints of todays Internet and
adopting pre-existing infrastructure such as content caches: best-
effort streaming with adaptive video, no service guaranteees for most
services, and co-location of caches with large user communities. In
environments where more than best-effort services for these
applications are required and deployment of current technologies to
support them is feasible, it is done. Examples include DiffServ or
even on or offpath bandwidth reservations in controlled networks.
Networked AR/VR is a very near term set of use cases, where solution
models are very much attempting to use and expand existing solution
approaches for video network streaming but where the limits of above
current best practices are also amplified by the larger bandwidth
requirements and stricter latency and jitter requirements of AR/VR.
To ensure a good user experience, for live Virtual Reality (VR), a
much higher resolution than 8K video is required. In addition to the
high bandwidth requirements of VR, there needs to be a supporting
transmission network to provide a communications path with bounded
low latency as well. This stringent VR latency requirement is a
challenge to existing networks.
Bryant, et al. Expires September 10, 2020 [Page 5]
Internet-Draft FWD PS March 2020
In cellular networks, even though the the air interface link latency
needed is significantly reduced e.g. with New Radio (5GNR), the end-
to-end (E2E) requirements for live VR is harder to meet. This is
because of the fixed L2/IP/MPLS networks in front/mid/backhaul
components, and because of the best effort nature of the packet
delivery systems in these networks.
2.2. Role of Fixed Networks in 5G and Beyond 5G
The 5G and beyond 5G (B5G) services are not meant to be limited to
the 5G-NR (new-radio). In fact for those services relating to uRLLC,
mMTC and eMBB packet networks have evolve along with the radio
technologies. While 5G-NR protocol stack has evolved to provide per-
frame reliability and latency guarantees, the IP/MPLS transport
network by and large remains best-effort. It is no longer possible
to solve network problems simply by increasing the capacity. The
expectations 5G devices have of 5G networks, can not be met without
improving IP/MPLS based backhaul networks. For example, the 5G based
systems involve machine to machine communications, generally using
command-based smaller payloads. In this case the overheads of packet
headers and overlays become apparent when computing latency budget of
such packets.
The IETF has produced a large body of work on the deterministic needs
of network applications [RFC8578]. These range from refinements and
expansions of above summarized Audio/Video and AR/VR use cases over
gaming into many more "industrial" use cases. Industrial use cases
generally involve industrial controllers for high-precision machinery
and equipment, such as robotic arms, centrifuges, or manufacturing
equipment for the assembly of electronic components.
These use cases have in common that they require delivery of packets
with very precise and "deterministic" performance characteristics, as
the controlled equipment and the control loops involved have very
exact timing requirements and are not tolerant of any latency
variations, as otherwise control loop issues and other undesired
effects may occur.
Specifically, the use cases involve curtailing maximum latency that
could be incurred. However, deterministic networking, by itself,
does not appear to be sufficient to meet all of the emerging needs.
2.3. ITU-T Focus Group Network-2030
The ITU-T has been running a Focus Group (FG) Network-2030
[FGNETWORK2030] to analyze the needs of networks in the period post
2030. This work started in July 2018 and has been an open process
with contribution by a cross-section of the networking industry.
Because this is non-IETF work, this section summarizes the currently
finalied key findings of the ITU-T Focus Group Network-2030 to make
Bryant, et al. Expires September 10, 2020 [Page 6]
Internet-Draft FWD PS March 2020
it easier for the reader to better undersstand the work. Note that
this work is still ongoing and additional findings may be published.
The Focus Group Network 2030 considered a number of use cases that it
was postulated would need to be addressed in the 2030 time-frame and
the technology gaps that need to be bridged in order to address these
needs. It then considered a number of new network services that
would be needed to support these services.
An ongoing piece of work on the architecture of the network post 2030
has not yet been completed at the time of writing and is only
partially discussed in this document.
The reader is referred to [WP], [NET2030SubG2], [UC] for information
beyond that provided in this summary.
ITU-T FG NET2030 Sub-group Sub-G1 (Sub-G1) considered a number of use
cases that it considered to be representative of the network needs
post 2030. These needs are legitimate needs in their own right, but
as is always the case act as poster-children for new applications
that will inevitable conceived in the light of the new network
capabilities that we postulate to be necessary.
o Holographic-type communications (HCT)
o Tactile Internet for Remote Operations (TIRO)
o Network and Computing Convergence (NCC)
o Digital Twin (DT)
o Space-Terrestrial Integrated Networks (STIN)
o ManyNets
o Industrial IoT (IIoT) with cloudification.
Further information on these use cases is provided in Section 10, and
in the ITU documents [UC] and [WP].
Note to the reader: Unlike ITU-T Study Groups which are restricted to
members, ITU-T Focus Groups are open to anyone without payment. At
the time of writing, ITU-T Focus Group Network-2030 material that is
not available for anonymous download, is accessible for free by
joining the Study Group.
Bryant, et al. Expires September 10, 2020 [Page 7]
Internet-Draft FWD PS March 2020
3. New Network Capabilities and Services
In order to support the use cases presented in Section 2, a number of
new network services will be needed. Likewise, a number of
additional more general network capabilities will becoming
increasingly important. Neither services nor capabilities are
sufficiently supported to the degree that will be required by
Internet technology in use today.
This section describes these services and capabilities at a high
level. It builds on a corresponding analysis that was conducted at
ITU-T FG-NET2030; readers are referred Section 11 for further detail
and, of course, to output produced by that group [NET2030SubG2] for a
more complete explanation of their considerations.
3.1. New Services
[NET2030SubG2] identifies a number of network services that will be
needed to support many of the new use cases. These network services
are divided into two categories:
o Foundational Services (FS) require which dedicated support on some
or all network system nodes which are delivering the service
between two or more application system nodes.
o Compound Services (CS) are composed of one or more foundational
services, and are used to make network services easier to consume
by certain applications or categories of use cases. An example of
a CS would be a Tactile Internet Service which consisted of
tactile control channel and a haptic feedback channel.
The following are a set of Foundational Services :
o High-Precision Communications Services: services with precisely
defined service level objectives related to end-to-end latency.
Three high-precision communications services that have so far been
proposed:
* In-time Services: services that require end-to-end latency
within a quantifiable limit. This service is similar to the
service provided by DetNet [RFC8655] but with more demanding
applications which need to be satisfied over IP.
* On-time Services: services require end-to-end-latency to be of
an exact duration.
* Coordinated Services: Coordinated services require multiple
interdependent flows to be delivered with the same end-to-end
Bryant, et al. Expires September 10, 2020 [Page 8]
Internet-Draft FWD PS March 2020
latency, regardless of any (potential additional) service level
objective.
o Qualitative Communication Services: services that are able to
suppress retransmission of less relevant portions of the payload
in order to meet requirements on latency by applications that are
tolerant to this.
These are described in more detain in Section 11.1.
3.2. New Capabilities
[NET2030SubG2] identifies also a number of network capabilities that
will become increasingly important going forward, in addition to the
support for any particular services.
A number of those need to be taken into consideration from the very
beginning when thinking about how future data-planes need to evolve.
These capabilities are described in more detail in Section 11.2.
o Manageability: Many of the services that need to be supported in
the future will require advances in measurements and telemetry
will be required in order to monitor and validate that promised
service levels are indeed being delivered. These will requires
advanced instrumentation that is ideally built.
o High Programmability and Agile Life-cycle: Methods to provide
operators need to be able to rapidly nd easily introduce new
network services and adapt to new contexts and application needs.
o Security and Trustworthiness: New mechanisms are needed to
authorize packets to enter the network from a host or from another
network, and for them to then receive the required premium service
that can operate. This must operate without impacting the latency
and MTU requirements. This security mechanism has to protect both
the network, the user data and the user privacy, but still expose
sufficient information to the network that the correct premium
service can be delivered.
o Resilience: Ultra-low-latency requirements and the huge increase
of bandwidth demands of new services such as holographic type
communication services make retransmission as a mechanism to
recover data that was lost in transit increasingly less feasible.
Therefore, network resilience and avoidance of loss becomes more
importance that it is for best effort networks.
o Privacy-Sensitive: There is a growing awareness of the lack of
privacy in the Internet and its implications.
Bryant, et al. Expires September 10, 2020 [Page 9]
Internet-Draft FWD PS March 2020
New network services have to be sensitive to and comply with
heightened user privacy expectations.
At the same time, the need for privacy needs to be balanced with
legitimate needs of network providers to operate and maintain
their networks, which requires some visibility into what is
happening on the network and how it is being used. There are a
variety of privacy-related requirements that ensue, such as:
* Anonymization
* Opaque User data
* Secured Storage
* Flow anonymization
o Accountability and Verifiability: Provision of the methods to
account for an verify delivery of premium services.
4. Underlying New Requirements
4.1. Better than Best Effort
The current Internet is essentially of best-effort system, but future
applications require high-precision KPIs on throughput, latency and
packet loss for industrial manufacturing, control, automation, and
machine-to-machine communications.
With upcoming Cellular technologies (5G/B5G) there is a need for
Service Providers to expand the type of customers for metropolitan
size networks to address their better than best-effort traffic needs.
DetNet has been proposed to support this, however:
o Only some aspects of DetNet currently only run on top of current
IP/IPv6.
o DetNet service is too constrained: It only supports constant bit
rate (CBR), reserved bandwidth. It does not support flexible
bandwidth. The notion of contracts in a future development of the
forwarding layer will support more flexible managed bandwidth and
managed latency contracts for traffic.
4.2. Efficient Packet Design
The ratio of useful data in the payload to overhead has a direct
financial impact on communication links; these links are of finite
capacity and hence have a finite cost-per-unit-data that can be
Bryant, et al. Expires September 10, 2020 [Page 10]
Internet-Draft FWD PS March 2020
calculated. The capacity used to transport information as compared
to the overhead which is unavailable for use by a customer, but
required to transmit is often expresses as a good-put efficiency and
can be related to cost to transmit payload data.
o There is a need to support large number of low power user
equipment (UE) devices (low-power IoTs) connecting through various
radio networks (LTE/5G/B5G) where spectral efficiency is needed.
This needs to be achieved without header compression techniques
like as [RFC6282] since, compression can result in additional
processing and energy consumption overhead.
o The handling network protocol headers, requires that portions of
each packet be held in memory or buffer structures; the more
levels of information which need to be held for processing by
network nodes, the more memory space will be required, and this
directly effects the cost of operation and cost of manufacture/
provision of such equipment.
On the other hand, in various non-constrained environments where
various network layer functionalities are desired, there are
different set of requirements. For example:
o Segment Routing over IPv6 (SRv6) parameter encoding
[I-D.filsfils-spring-srv6-network-programming] in the SRv6 SID
[I-D.ietf-6man-segment-routing-header] is limited by the prefix
portion of the IPv6 address.
o In Identifier Locator Addressing (ILA), the identifier (ID)
portion of the address length is limited because of 128 bits
limit.
4.3. Forwarding Identifiers
Developments in IPv6 [I-D.filsfils-spring-srv6-network-programming]
formalize a trend that has been happening for a long time: the
morphing of network layer addresses into forwarding identifiers (FI).
However, constraining FIs to a fixed size ill serves the development
of the forwarding layer. There are clear cases as illustrated above
where it would be useful to have shorter network layer addresses.
Equally we can see that there will be future cases where 128 bits may
be insufficient to specify a forwarding operation. The requirement
is thus to formally introduce the concept of forwarding identifiers
in place of network layer addresses, and use a forwarding identifier
construct that supports multiple semantics and multiple, possibly
fully variable, lengths.
Bryant, et al. Expires September 10, 2020 [Page 11]
Internet-Draft FWD PS March 2020
4.4. Operational visibility
Network operators crave facilities that let them better understand
and fine tune detailed network behavior, which are hard to retrofit
with current IP/IPv6.
The rise of machine learning has led to the expectation of being able
to better optimize networks This in turn leads to the increase of
network telemetry as a source of data to base these systems on. In-
Situ OAM (IOAM) [I-D.ietf-ippm-ioam-data] represents one of the
latest developments in that space, allowing the data plane to piggy-
back telemetry data onto individual packets in order to diagnose and
fine-tune service levels such as latency or jitter. However, there
are several issues with this approach:
o MTU issues limit amount of data that can be obtained. With IOAM
packet size increases with number of data items and number of
hops.
o The data that can be obtained is very limited.
o The OAM data volume can easily exceeds that of production traffic
which is wasteful
o There is no ability to aggregate OAM data, or make context
dependent OAM collection.
o Integration with other solutions such as DetNet is unclear.
While useful, IOAM exposes the limits of what add-on solutions can
provide. Solutions that provide visibility at the level of flows or
that provide automatic verification of Service Level Objectives are
missing entirely.
4.5. Holistic solution
It needs to be also recognized that it will not be sufficient for
solutions to support new services and capabilities one at a time and
independently from one another. Instead, solutions need to be
holistic and be able to support new services and capabilities in
integrated fashion and simultaneously.
For example, better-than-best-effort, operational visibility, and
efficient packet design should go together, without leading to
additional integration problems ore requiring users to to make a
choice.
This is in contrast to the current piecemeal approach, in which
solutions for any one particular problem may well be developed but
Bryant, et al. Expires September 10, 2020 [Page 12]
Internet-Draft FWD PS March 2020
emerge one at a time, resulting in fragmented solutions that are may
be hard to integrate.
5. Deployment Models
Service requirements from networks and security implications vastly
differ in various deployment models as categorized below.
5.1. Edge-2-Edge Model
This is the traditional service provider deployment where various
network services (VPN, security, Bandwidth..) are offered to the
endpoints of the communication and other providers. Such
capabilities are purchased through contract with the service provider
and are typically expensive.
These networks predominantly use MPLS technology though native IP
(IPv4/IPv6) with GRE and IPv6 with routing extension headers with
SRv6 are being deployed recently.
..................................
+---+ . +---+ Single +---+ . +---+
|CE1|---|PE1|---.. Provider ..---|PE2|---|CE2|
+---+ . +---+ Network +---+ . +---+
..................................
Figure 1: An Edge-2-Edge Network
5.2. End-2-End Model Single Provider
In this case there is a single provider network in which E2E
offerings and host session are initiated and terminated with in the
single provider network.
1. OTT Provider Networks: Endpoints of the communication (virtual or
physical hosts) consuming services through with in the OTT
provider network servers (Cloud and Data Center (DC) networks);
where the other endpoint can be in the same server form or on the
DC Gateway or on the other end of the DC Server Farm connected
through Data Center Interconnect (DCI).
2. Wireless and Wire-line Networks: Endpoints (UE's) connecting to
the provider wireless or wired networks, where service is
terminated inside the provider network end points. Based on the
service offerings connection termination can happen close to the
Radio/access nodes with multi-access edge computing (MEC) clouds
or in the provider core network (core-cloud) before going to the
Bryant, et al. Expires September 10, 2020 [Page 13]
Internet-Draft FWD PS March 2020
Internet eventually. Example of these deployments include BNG,
4G and 5G wireless access/RAN/backhaul networks.
There are two sub cases:
a) Where the host is physically (wired/wireless) connected to the
Provider Edge (PE)
.............................................
+---+ .+----+ +----+. +---+
| H +--+ PE |--- 1 Provider Network ---| PE +--+ H |
+---+ .+----+ (Single/Multiple domains) +----+. +---+
.............................................
Figure 2: An Edge-2-Edge Network Direct
In this case the provider controls the whole path and can certify the
correct operation of the service according to contract.
b) Where the host is connected via its own network to the PE
+---+ +---+
| H | | H |
+-+-+ +-+-+
| |
| ............................................. |
+-+--+ .+----+ +----+. +-+-+
| CE +--+ PE |--- 1 Provider Network ---| PE +--+ H |
+----+ .+----+ (Single/Multiple domains) +----+. +---+
..............................................
Figure 3: An Edge-2-Edge Network Indirect
In this case the provider controls only the path to the CE and can
certify the correct operation of the service according to contract
from that point but the user is responsible for providing the
required service characteristics into their own network.
5.3. End-2-End Model with multiple Providers
There are two cases to consider:
1. Multiple provider with Transit Networks: These are traditional
E2E deployments where communication endpoints of the data traffic
on different provider networks with regional, transit network
providers through Internet Exchange Providers (IXPs) providing
the global inter connection.
Bryant, et al. Expires September 10, 2020 [Page 14]
Internet-Draft FWD PS March 2020
2. Two Providers with no Internet Transit Network: Another variant
of the E2E connectivity can be seen as evolving comprises only
endpoints provider (access) network and receiver access provider
network with global transit provided by one ISP.
The first case is very difficult to support since it is unlikely that
the whole path is know to support extended capabilities in the
forwarding plane. It is not infeasible, and it would be possible to
set up such paths in principle given suitable enhancements to the
routing system. However such a scenario must be considered
infeasible for the foreseeable future.
The second case is more tractable provided there is co-operation
between the providers.
5.4. Embedded Service
The industry move is towards content and application service
providers embedding themselves within the edge network. This is
currently done to save bandwidth and improve response time. As the
need for high precision low latency networking develops the need for
edge computing rises since the closer the client and the server the
less the scope for network induced performance degradation.
+---+
| H |
+-+-+
|
| .....................................
+-+--+ .+----+ +---+ .
| CE +--+ PE |--------+ S | .
+----+ .+----+ +---+ Provider 1 .
.....................................
Figure 4: An Edge-2-Provider
In this network the server S (owned by the content and applications
provider) has a contractual relationship with provider 1 and is thus
able to negotiate the network characteristics needed to meet its
service requirement. This model in which the server brokers the user
to network interface (UNI) requirements removes many of the
objections to the classical UNI model in which the client requests
the service requirements. In this model the host authenticates
itself with the server, having formed a previous business
relationship (for example by purchasing a holographic conferencing
service). The server has a relationship with Provider1, and thus is
a trusted party able to request that the service be set up between
itself and and its client, paying as necessary. As this is a
Bryant, et al. Expires September 10, 2020 [Page 15]
Internet-Draft FWD PS March 2020
requested paid service traversing a limited distance over a defined
network, a bespoke packet protocol can, if necessary, be used with in
a contained and constrained way.
How the server communicates with any other part of the application
domain is out of scope for this document and possibly out of scope
for Provider 1.
This takes us to consider the embedded global service described in
{#EGS}.
5.5. Embedded Global Service
+---+
| H1|
+-+-+
|
| ......................................
+-+--+ . +----+ +---+ .
| CE +---+ PE |--------+ S1| .
+----+ . +----+ +-+-+ Provider 1 .
..................|...................
|
|
..................|...................
+----+ . +----+ +-+-+ .
| CE +---+ PE |--------+ S2| .
+----+ . +----+ +---+ Provider 2 .
| ......................................
|
+-+-+
| H2|
+-+-+
Figure 5: Edge-2-Edge via Provider
In this network model, the server S1 (owned by the content and
applications provider) has a contractual relationship with provider 1
and is thus able to negotiate the network characteristics needed to
meet its service requirement. It is servicing the needs of host H1.
Similarly that same provider has a contractual relationship with
provider 2 where it is servicing the needs of host H2.
By a method outside the scope of this document and outside the scope
of the global Internet the contents and applications provider has a
private path between S1 and S2.
Bryant, et al. Expires September 10, 2020 [Page 16]
Internet-Draft FWD PS March 2020
This scenario shown in Figure 5 is important because it removes the
overwhelming issues associated with providing enhanced service across
the global Internet. Furthermore it describes a model where there is
commercial incentive, at scale, for the edge providers (Provider 1
and 2 above) to invest in providing and enhanced access service.
6. Existing Protocol and Layering Challenges and Gaps
Despite IPv4 still having a large user base, and having a number of
useful properties the IETF has abandoned future development of IPv4
as a way to force the deployment of IPv6. For example, in terms of
traffic steering the segment routing could have usefully been applied
to IPv4 to support network operators that wished to retain IPv4 as
their preferred internal protocol.
Given the gaps in each of the existing network layer protocols the
IETF may wish to look at the design of a protocol that both fills the
gaps and unifies its three existing network layer protocols.
Additionally there is a clear need for a more sophisticated approach
to indicating the required quality of service that a packet, or flow,
needs in an IP network.
6.1. Challenges with IPv6
6.1.1. The End-to-End Model
IPv6 and specifically [RFC8200] was designed to fit within an
Internet architecture centered around the end-to-end model with
"Internet Paths" potentially passing through one or more networks
without any relationship to the endpoints of a communication such as
most so-called transit-AS. As history already from IPv4 had shown,
anything more than the most simple per-hop processing options can
cause interoperability issues. In result, [RFC8200] has drastically
limited such per-hop processing options.
Two core restrictions of RFC8200 are the following:
o Restrictions on extension headers (EH): EHs must never be deleted
or changed in size by any node on the path the packet takes.
Intermediate nodes are only expected to examine these headers (if
they are configured to do so). Implementations cannot expect
intermediate nodes to examine, or act on, except for hop-by-hop
header (section 4.8 of [RFC8200]).
At the time of writing this is an area of considerable active
discussion in the IETF 6MAN and SPRING WGs. The issues that
Bryant, et al. Expires September 10, 2020 [Page 17]
Internet-Draft FWD PS March 2020
arrise from allowing unrestricted insertion, deletion or
modification of EHs are for example:
* Breakage of path MTU discovery
* Impact on the Authetication Header protocol
* Inability to return ICMP error messages to the correct node.
See Section 6.1.1.1 for further discussion.
o No new hop-by-hop headers (HBH) in IPV6: No new EHs that require
hop-by-hop behavior should be defined (section 4 of [RFC8200]) -
the only EH that has hop-by-hop behavior is the Hop-by-Hop Options
header. The only alternative available to the designer is instead
to use destination headers (section 6.8 of [RFC8200]).
6.1.1.1. IPv6 For Controlled Networks
While [RFC8200] is a conservative set of requirements to enable
proliferation of the target use case of "Internet Paths", the same
set of requirements limit the flexibility of IPv6 unnecessarily when
it is used in controlled networks where the constraints and
interoperability issues for "Internet Paths" do not equally apply,
for example the deployment scenarios shown in Section 5.4 and
Section 5.5.
One typical type of controlled networks are service providers (SP)
where SRv6 is used as the architecture within the SP network.
o IPv6 extension headers can not be added on a midpoint. Any
addition/change requires an encapsulation where another IPv6
header with optional SRH extension header is prepended to the
carried IPv6 packet. This is expensive in terms of packet MTU,
and in terms of packet buffer requirements at the ends of the
provider path which can be an economic issue in cost sensitive
network segments.
o The requirement to encapsulate instead of being allowed to add an
EH along the path stems from the desire to isolate any header
changes from Path MTU Discovery (PMTUD). This is a necessary
complexity when traversing uncontrolled hops across the Internet,
but it is unnecessary overhead when only passing through
controlled hops. In MPLS and SR-MPLS, the MPLS header size is not
included in the MTU available to the MPLS payload, instead the
network is managed such that the maximum MPLS header size plus the
available payload MTU is always smaller that the encapsulating L2
frame MTU. In IPv6 instead, the encapsulating and decapsulating
Bryant, et al. Expires September 10, 2020 [Page 18]
Internet-Draft FWD PS March 2020
would logically have to perform signaling for PMTUD
(unnecessarily).
o Because of the authorization header (AH) [RFC4302] and OAM
concerns, [RFC8200] likewise prohibits removing extension headers
or fields thereof on hops along the path, requiring for example
more complex packet parsers. In SR-MPLS it is possible to simply
remove the top SID on a node that has processed it, in SRv6 it is
instead necessary to look up an offset field in the SRH and, read
the appropriate SID (which may be deep in the packet), and then
increment the offset field.
o Even though the number of identifiers required within a controlled
network is often less than 16 bit, and almost always 32 bits,
carrying the overhead of 128 bits per SID in SRv6 can be seen as a
significant unnecessary overhead, and workarounds such a proposed
micro programs [I-D.bonica-6man-comp-rtg-hdr],
[I-D.bonica-spring-srv6-plus],
[I-D.filsfils-spring-net-pgm-extension-srv6-usid] require complex
forwarding plane processing and SRv6 programmability in the lower
64 bit is not required in the majority of use-cases for SIDs on
midpoints.
For use-cases like this, it would be a lot easier to innovate IPv6 by
clone & modify: E.g.: defining (say) IPv7 to be similar to IPv6, but
without the constraints that are not useful for the controlled
network use-case. A better alternative would be to create different
profiles of IPv6 with [RFC8200] being one. However, there is, as
yet, no concept of "profiles" in IPv6.
The issue of IP protocol operation in limited domains is discussed in
[I-D.carpenter-limited-domains].
Some possible solutions are described in
[I-D.herbert-6man-eh-attrib]. This will be considered further in a
future version of this text.
6.1.1.2. IPv6 for Edge-Compute
Today, the majority of end-to-end connections already do not pass via
the traditional "Internet-Path" but instead toward a server in data
center co-located with the access service provider Figure 4[DOT]. In
this case, there is no transit service provider, but there is a well-
established commercial relationship between either end of the
communications and the access service provider.
Bryant, et al. Expires September 10, 2020 [Page 19]
Internet-Draft FWD PS March 2020
Today, the majority of traffic consists of video-streaming/TV
services, but in the future, Edge-Compute will enable ever more
applications to operate in such a controlled environment.
The difference between the aforementioned use-case of IPv6 within an
service provider, and this use-case is that enhanced services in this
would naturally operate end-to-end between a Data Center application
server and the subscriber endpoints.
In the case of SRv6, it is not necessary to incur the overhead of an
IPv6 in IPv6 encapsulation, the SRH can be inserted by the endpoint
and removed by the endpoint on the other side. Nevertheless, the
[RFC8200] limitations of not being able to add/remove or freely
change the content of the SRH payload or any other EH on a midpoint
router still exists. This seriously limits the usage and evolution
of of IPv6 to the edge-to-edge model.
6.1.1.3. Hop-by-Hop Extension Header processing
Hop-by-hop IPv6 extension headers caused interoperability and
performance issues and as a result caused resistance to further
leverage and extend them except for SRv6-SRH RPL-SRH [RFC6554]. In
the authors opinions, this regression on hop-by-hop extension headers
is because of a combination of insufficient specifications and
resulting implementation issues. Both could be solved in future work
with new hop-by-hop processing specifications.
For example, router alert (RA) was (and still maybe) implemented in
routers so that all router alert packets are punted from the fast-
path to the slow-path even when the "value" field identifies a
protocol that the router can not process. As a result, protocols
that rely on RA such as RSVP [RFC2205] or even more so Pragmatic
General Multicast (PGM) [RFC3208] where filtered in networks because
they caused high control plane load on routers that did not support
either protocols but still unnecessarily punted their packets with
RA.
There are no normative statements about the need that fast-path
forwarding planes "MUST" be able to ignore unsupported/not-enabled EH
features at a speed such that such a packet can be forward at the
same speed as the same packet without the EH. For example, for RA,
there is only a "SHOULD" requirement to do this in [RFC6398], a BCP
published a decade after IPv6 router alert [RFC2711]. With such a
gap in time between the specification and the BCP, it is impossible
to rely on the existing RA and expect safe deployment across the
Internet without still running into performance issues.
Bryant, et al. Expires September 10, 2020 [Page 20]
Internet-Draft FWD PS March 2020
6.1.1.4. Segment Routing Header Constraints
The same design paradigm could have been used for the Segment Routing
Header (SRH) [I-D.ietf-6man-segment-routing-header], but there is no
distinction possible for IPv6 instances running in such a controlled
network or running as an Internetwork instance to form the Internet.
This is particularly unfortunate as we are evolving to a model where,
as noted earlier in this document, in most cases the packet will only
travel through two well-known networks: the hosts network and the
service provider network hosting the server to which the client is
interacting.
6.1.2. Fixed Address Length
When IPv6 was designed, the key focus was on solving the problem of
growth of the Internet and resulting growth of global Internet
address space. Variable length and a hetrogenious address approach
were proposed [RFC1347] however, these were rejected partially for
political reasons and partially out of a concern over the difficulty
of parsing the packet and doing a fast address lookup.
There was seemingly no focus on better supporting the now millions of
often network-layer isolated TCP/IP networks in industrial, defense,
research, embedded, industrial or other commercial environments.
One key problems with with 128 bit addresses is the overhead on low-
speed radio/IoT-wire networks. This is especially the case when
using source-routing, where multiple of these addresses have to be
included in the header. Current solutions are only able to resolve
these issues with CPU expensive IETF standardized header compression
techniques [RFC2507], [RFC3095], [RFC5795]. Even though these
approaches are feasible in many of todays IoT networks, there is a
strong desire to reduce power consumption in such devices. This is
particularly the case where they are powered by a single-for-life-
battery, or are self-powering through automatic replenished energy
sources. As a result of this CPU performance in future IoT network
should not be expected to increase but whenever feasible is more
likely to decrease.
Another, often overlooked, problem of the 128 bit IPv6 addresses is
that global address prefix allocation is a a big up-front burden on
many IoT networks, but also isolated networks (industrial, defense,
research, industrial). Often, this leads to the use of Unique Local
Addresses (ULA) [RFC4193], which have the risk of conflicts when
those previously isolated networks need to interconnect with other
networks.
Bryant, et al. Expires September 10, 2020 [Page 21]
Internet-Draft FWD PS March 2020
While solutions to these problems may look easier enough, it should
be noted that in the time when IPv6 was designed, variable length
addresses in the fastest forwarding planes were not seen as feasible,
and there was also a lack of experience with the impact of
interconnecting heterogeneous address spaces other than as ships-in-
the-night parallel operation of protocols. A lot of that experience
came later through 14++ IPv4/IPv6 transition solutions designed in
the past 20 years and respective work on address discovery in IETF
frameworks such as SIP/STUN/ICE.
Another issue with the fixed length homogeneous address approach is
the constraints this places on the current practice of overloading
addresses with other functionality for example
[I-D.filsfils-spring-srv6-network-programming].
6.2. Better Than Best Effort E2E Network Services
Some of the fastest growing network segments where new services are
being introduced in an End-2-End manner belong to deployment models
as described in Section 5.2. The requirements here for service
delivery involves stringent E2E latency with no retransmission and no
packet loss. Not all scenarios need "lower" latency but bounded to a
particular value/range. Example use cases involving an user
equipment (UE) consuming service from the provider cloud network or
another UE (e.g. Vehicular device, IIoT) in the same network. Here
the service endpoints could be connected over wire or wireless (LTE/
NR) and the service termination happens in the provider network
either close to the access network or provider core network as
illustrated in Figure 2, Figure 3. The existing network layer and
best-effort model simply cannot guarantee needed service level
objectives in these scenarios.
Some specific needs and requirements from cellular fixed transport
networks are:
o Need for determinism on E2E throughput and latency. The current
TCP/IP is hence not-suitable for Mission-critical and real-time
E2E applications.
o Need for E2E QoS for ultra-reliable-low-latency communications
(uRLLC).
o Efficient use of protocols in the network by minimizing tunnels
over tunnels and duplicate header fields.
o Efficient deployment of network slicing
Bryant, et al. Expires September 10, 2020 [Page 22]
Internet-Draft FWD PS March 2020
6.3. Adaptive Bit-rate Video streaming
Even without going to future application requirements as described
elsewhere in this document, even the majority of existing Internet
traffic is lacking competitively usable and standardized service to
support quality of service.
The majority of traffic today is Adaptive Bit-rate (ABR) based audio/
video streaming. The primary benefit of this approach is that it can
adjust itself to much lower bandwidth than the bandwidth to offer the
ideal/target experience quality to the user. It therefore enabled
Over The Top (OTT) services to offer streaming media. Nevertheless,
ABR itself does not provide any actual quality guarantees.
Service providers that use ABR streaming to their subscribers do
therefore combine ABR with IP developments, some non-published, which
are often out-of-band bandwidth reservation schemes. These allow ABR
video streams to have their ideal/target experience bandwidth within
the SP's network and only need to degrade if there was bandwidth
contention in the subscribers (home) network.
If a subscriber, or a content provider which is not the access
service provider wanted to get the same type of bandwidth guarantees
for other content across the access providers network, they could do
so with existing IETF standards via RSVP [RFC2205] which is widely
implemented, or NSIS [RFC4080], which was to the knowledge of the
authors never implemented in widely used router products (because it
does not offer sufficient benefits over RSVP). In either case, the
per-flow control-plane based signaling architecture including the
aforementioned router-alert issues make these protocols a difficult,
likely not future-proof solution.
Even more fundamentally, ABR has shown that media streaming can
easily support elastic adjustment between a range of bandwidth limits
in which the quality is between acceptable and ideal, but there is as
of today no standardized mechanisms by which to express relative
bandwidth allocations when streams compete against each other that
goes beyond the very loosely defined "internet fairness". For
example, more intelligent congestion management could defend
bandwidth the more the bandwidth approaches the minimum acceptable
bandwidth, or admission control of bandwidth could be elastic. Some
work in these direction exists in [RFC8698] with its ability for
weighted congestion control or
[I-D.ietf-tsvwg-intserv-multiple-tspec] for (limited) elastic
admission control management.
Bryant, et al. Expires September 10, 2020 [Page 23]
Internet-Draft FWD PS March 2020
6.4. DetNet and Higher Precision Networking Service
Time Critical (TC), Ultra-Reliable, Low Latency (URLLC), Internet-of-
Things is another important use case scenario-set that highlights
requirements that are difficult to satisfy with existing Internet
connectivity paths where a part of that path includes a radio access
link. These kind of close-loop control systems borne over
heterogeneous communications networks have very precision and bounded
latency requirements for the E2E network connecting the sensor and
actuator.
Deterministic networking within the IETF is focused on only one
dimension of the URLLC problem.
DetNet is also far from attempting to identify currently if/how the
services it plans to introduce could be made to operate over the
Internet in general, instead, it focusses mostly on the shorter term
goal to enable them in controlled networks within a limited domains.
Currently, the requirements for a DetNet forwarding plane have been
reasonably mapped out for an MPLS based forwarding layer.
Nevertheless, in addressing these needs within an IP network
[I-D.ietf-detnet-ip] the solution has of necessity been limited to
the capabilities of the IP as it exists today. It has not, for
example, been possible to add the packet replication elimination and
reordering function (PREOF)which allows multiple concurrent packet
delivery attempts in an MPLS network [I-D.ietf-detnet-mpls]. The
DETNET body of requirements needs to be revisited in the light of any
development to network forwarding capabilities.
6.5. Forwarding Plane vs. Control Plane
High-end hardware with accelerated forwarding plane devices, can
support a significant number of forwarding states including
destination entries (IP destination/mask, MPLS label, SR SID) as well
as 2, 3 or 5 tuple IP/IPv6 "flow" entries. Nevertheless, the control
plane that builds and changes these entries often limits their
usability because the control plane does not even scale to the number
of hardware accelerated forwarding entries possible, or because the
supported rate of changes is slow.
The root of this problem is that with the increase of speed and scale
of hardware accelerated forwarding hardware, control plane had
challenges to keep up in performance. The performance of
appropriately priced control plane CPUs (relative to the cost of the
forwarding plane) has not grown at the same speed as that of hardware
accelerated forwarding plane chips.
Bryant, et al. Expires September 10, 2020 [Page 24]
Internet-Draft FWD PS March 2020
One of the directions to overcome these challenges is invisible
outside these forwarder devices and it is to optimize the control-
plane to forwarding plane interactions, such as programming the
building of forwarding state directly on the accelerated forwarding
infrastructure (e.g. NPU), but using otherwise existing control
plane protocols.
A more fundamental approach is to redesign control plane protocols
such that they are lighter weight in their signaling and state
machinery, and can therefore be completely implemented in the
hardware accelerated forwarding plane. Effectively turning a control
plane protocol into an advanced forwarding plane protocol function.
This approach is logically most easily applicable to on-path per-flow
signaling mechanisms such as RSVP or RSVP-TE, both of which are quite
complex with their signaling messaging and state keeping and
therefore directly infeasible to become hardware accelerated
forwarding implementations. An example approach to provide similar
functionality to RSVP with signaling light-weight enough to allow
hardware accelerated implementation are the in-band signaling
mechanisms (e.g. for TCP or UDP) described in [DIP1]
[I-D.han-tsvwg-ip-transport-qos] [I-D.han-tsvwg-enhanced-diffserv].
Signaling that is feasible to become part of a complete in-
forwarding-plane signaling solution is not limited to in-band on-path
flow signaling, but would likely also be applied to other signaling
options. Of the aforementioned existing signaling protocols, IGPs
are likely the ones whose signaling could most easily be processed in
an NPU compute elements except that the SPF calculation itself
introduces a complexity that would make this very complex. One
example of a solution that solves this problem by signaling the
actual per-hop adjacencies in IGP and therefore eases NPU
implementation can be found in
[I-D.chunduri-isis-preferred-path-routing].
In summary: The scope of what should be considered forwarding plane
today is defined by decade historic architectures, but should for the
future be scoped by the realities of the new, different "layers" of
hardware and their capabilities. Hence also the use of the term
forwarding plane, because it can span not only across classical
bridging (L2), label/tag/SIG switching (L2.5), network/internetwork
(L3) and transport (L4) layers, but also across the classical "data
plane" and "control plane" components of each such layer.
Bryant, et al. Expires September 10, 2020 [Page 25]
Internet-Draft FWD PS March 2020
6.6. User-Network/Network-User Interface Signaling
Some of the deployment models as described in Section 5.2, needs
specific signaling mechanism from user/applications. These are
needed for E2E service offering for better than best effort
Section 6.2 or high-precision networking Section 6.4. These may
involve new transport mechanisms at hosts, middle-boxes and routers
to meet the E2E service requirements in these limited domain
deployments.
Here one of the functional requirements is to signal the service
level objectives (SLOs) dynamically for a particular service from the
network. This signaling includes the service description, the
service negotiation with the network, the service setup or
modification, or the need to execute some functions at network device
and send the results back to the sender. However, the current IP was
not designed for this. For example, the result of SLO negotiation at
any hop needs to be updated in the IP packet at the router and
returned back to the sender (originating host or gateway device for a
Service Provider).
There are some attempts to achieve the above as described in
[I-D.han-tsvwg-ip-transport-qos], which describes general in-band
signaling for QoS control with IPv6 protocol and
[I-D.han-tsvwg-enhanced-diffserv], which proposes a backward
compatible class-based queuing and scheduling schema for hybrid
service to support guaranteed service from the network (e.g. for
latency and bandwidth).
In summary, it is difficult to do better than best effort or High
Precision Services described in Section Section 6.4, in closed
domains with current IP given the best effort congestion control
(TCP/QUIC) and explicit congestion notification (ECN) framework. A
comprehensive mechanism needs to be explored as the limitations in
silicon technologies or deployment models 30 years ago are not
relevant with respect to security, scalability, packet size change,
MSS or FCS recalculation, etc.
7. Candidate Solution Directions
This section describes a number of solution considerations, but is
not prescriptive about any specific approach or technical solution,
and is provided to stimulate thought on the subject.
Bryant, et al. Expires September 10, 2020 [Page 26]
Internet-Draft FWD PS March 2020
7.1. Variable Length Addresses
When private networks are set up, they only need to use an address
length that allows the construction of networks sufficiently large to
meet the expected service requirements. If a future network layer
protocol could support address length of e.g.: 16, 24, 32, 48, 64 and
128 bits (or maybe more), it would be easy for such networks to pick
a right size. This would allow them to have as efficient packets
without compression as possible, and it would also avoid for them to
have to think about allocation procedures for "global" addresses.
Whenever networks with a smaller address size would later on have to
interconnect to other networks, the shorter length address would have
to be interpreted as the suffix of a sufficiently larger address
space through which those connecting networks could achieve unique,
non-overlapping addresses. At the border between these networks,
high speed forwarding planes could easily perform per-packet
stateless prefix addition/deletion transformations of addresses in
the packet header when the interconnection should be free of further
policy. When such an interconnection is desired to employ specific
traffic control policies, mapping of addresses in a stateful manner
is a convenient way to enforce and support such policies through the
forwarding plane.
7.2. Address Semantics
Classically IP unicast addresses identify an interface. There is the
special case of a loop-back address, but this is normally modeled as
an internal interface. Addresses are often silently mapped to
include other semantics and this is most developed in the IP network
programming concept [I-D.filsfils-spring-srv6-network-programming].
MPLS is more general. It defines the concept of a Forwarding
Equivalence Class in which a Label which can be visualized as an
offset into a specific table with up to 2^20 entries, with the table
containing the instruction to be executed. Thus a single identifier
is able to specify: forward towards an egress, forward along a
specific path, decapsulate and sent to an interface, decapsulate and
forward via an IP lookup in a label specific address table etc.
The semantics of the MPLS label and the size of the label are such
that it is not possible to include any instruction parameters in the
label and very inefficient to include those parameters in one or more
further labels. The only example of doing this is the Entropy Label
indicator [RFC6790] which uses two Label Stack Entries (LSEs). Any
future development along these lines will need at least three LSEs.
Bryant, et al. Expires September 10, 2020 [Page 27]
Internet-Draft FWD PS March 2020
Whilst an IPv6 is larger there is still limited space to add
parameters within the address. In the current work on this the size
is limited to 16 bits, and there is a fundamental limit of 64 bits.
It is clear that move is towards a multiplicity of semantic for the
network layer address, and indeed a formal recognition that the
address is in reality an instruction with a specific scope.
7.3. Multiple Instructions
What we have learned from MPLS and then from SRv6 is that it is often
desirable for a node (be that the originating host or a router) to
impose on a packet a set of instructions to be executed in sequence
by one or more entities in the network. An development of IP or any
successor needs to recognize this and provide a simple and efficient
way to incorporate a list (or stack) of instructions within the
packet header.
7.4. Node and Path Specific Processing Instructions
There is an established need to do node specific instructions as is
indicated by the design of MPLS and Segment Routing (SR). Any
development of the forwarding system needs to retain this feature and
ideally develop a method that is simultaneously both general and
efficient.
References to efficiency include efficiency in packet size and
efficiency decoding and and executing the instruction. The
efficiency of encoding is not simply a matter of on the wire
bandwidth, but is also a matter of the size of the forwarder packet
header cache. This cache has to operate at wire speed can be an
expensive silicon element.
There is also a need to do path specific operations as are done in
RSVP-TE. However RSVP has a significant path set-up and path
maintenance cost. Clearly a per path instruction can be specified as
a set of N per node instructions where N is the number of hops along
the path, for example by using SR, but that is not an efficient
encoding where N is large. It is thus a useful optimization to
include the ability to include per path instructions, and this is the
subject of further study.
7.5. Integrated Assurance and Verification
Being best effort in nature, assurance for services provided using IP
is left to add-on solutions built after the fact. How to perform
tasks such as verifying of service levels is left as an exercise for
network providers, often approached using statistical approaches that
Bryant, et al. Expires September 10, 2020 [Page 28]
Internet-Draft FWD PS March 2020
are themselves "best effort" in nature. This will be no longer
sufficient for mission-critical services such as tele-driving or
tele-operations that demand guarantees, where failure to meet those
guarantees may expose providers and users exposed to liability
demands and call the feasibility of applications relying on those
services into question.
Moving forward, network protocols suitable to deliver high-precision
services for mission critical applications need to address assurance
as an intrinsic property, not left to afterthoughts.
8. IANA Considerations
This document does not request any allocations from IANA.
9. Security Considerations
Security is likely to be more significant with the applications being
considered in this work. With interest in tightly controlled access
and latency, and contractual terms of business it is going to be
necessary to have provable right of access to network resources.
However heavyweight security is a contra-requirement to the light-
weight process needed for power efficiency, fast forwarding and low
latency. Addressing this will require new insights into network
security.
Further information on the issue of providing security in latency
sensitive environments can be found in [I-D.ietf-detnet-security]
which are a sub-set of the considerations applicable to the new use
cases considered in this text.
10. Appendix 1: Expanded Summary of Sub-G1 Use Cases
10.1. Holographic-type communications
This work projects that we will move towards a holographic society
where users remotely interact with the physical world over the
network. In industry the digital twin model will enable the control
of real objects through digital replicas. Telepresence will move to
a new level with multi-site collaborations becoming much closer to
physical meetings that can take place without the time and
environmental cost of physical travel. 3D medical scans will become
full 3D views rather than the body/ organ slices that too many of us
are regrettably familiar with. It is easy to imagine that this
technology will take message delivery to a completely new level.
Analysis of these concepts results in the conclusion that the
following key network requirements are necessary:
Bryant, et al. Expires September 10, 2020 [Page 29]
Internet-Draft FWD PS March 2020
o Ultra-high bandwidth (Tbps class)
o Ultra-low latency (sub-ms)
o Multi-stream synchronization
o Enhanced network security
o Enhanced network reliability
o Edge computation
10.2. Tactile Internet for Remote Operations
Two cases were proposed as examples of this class of application.
The first is remote industrial management which involves the real-
time monitoring and control of industrial infrastructure operations.
The second involves remote robotic surgery. Remote robotic surgery
within an operating suite complex is a standard practice today,
however there are cases where it would be desirable to extend the
range of this facility.
Analysis of these concepts results in the conclusion that the
following key network requirements are necessary:
o Ultra-high bandwidth (Tbps class)
o Ultra-low latency (sub-ms)
o Sensory input synchronization
o Enhanced network security
o Enhanced network reliability
o Differentiated prioritization levels
10.3. Space-Terrestrial Integrated Networks
The game-changer in the area of space-terrestrial networking is the
active deployment of huge clusters of cheap Low Earth Orbit (LEO)
satellite constellations. These LEOs have a number of properties
that make them attractive, but arguably the most important is that
they combine global coverage with low latency. Studies [Handley]
show that for distances over 3000Km latency via a LEO cluster is
lower than the latency of terrestrial networks. The up-link to a LEO
cluster has to constantly change the point of attachment to the
cluster as the satellites that form the cluster rapidly move across
Bryant, et al. Expires September 10, 2020 [Page 30]
Internet-Draft FWD PS March 2020
the sky relative to both the ground and relative to the satellites in
other orbits. In this scenario a number of access and connection
models need to be considered.
Analysis of these concepts results in the conclusion that the
following key network requirements are necessary:
o A suitable addressing and routing mechanism to deal with a network
that is constantly in flux.
o Sufficient bandwidth capacity on the satellite side to support the
new application needs
o A suitable satellite admission system
o Edge computation and storage
10.4. ManyNets
There is evidence that there is a change in direction from the
Internet as a single hetrogenious network back to a true Internet,
that is an interconnection of a number of networks each optimized for
its local use but capable of inter-working.
For example, satellite and the terrestrial networks adopt different
protocol architecture, which causes the difficulty to internetwork
between them, yet the common goal is to provide access to the
Internet. Secondly, there will be a massive number of IoT-type
devices connecting to the networks but the current interconnection
schemes are too complex for these services. There are further trends
in 5G/B5G back-haul infrastructure, requiring diverse set of resource
guarantees in networks to support different industry verticals. The
collection of such special purpose networks, existing together and
requiring interconnection among themselves are called ManyNets.
Much closer the traditional Internet model is the move to edge
computing services in which the client traffic is terminated at a
compute node very close to access edge. [DOT] Any resultant
application traffic is a private matter between the application on
the edge server and the servers it communicates with in the
fulfillment of those needs. Furthermore the application on the
client may be using a tunnel to the edge compute server. In such a
network the protocol used inside the tunnel and the protocol used
between the servers executing the service is a private matter.
The ManyNets concept aims to support flexible methods to support the
communication among such heterogeneous devices and their networks.
Bryant, et al. Expires September 10, 2020 [Page 31]
Internet-Draft FWD PS March 2020
11. Appendix 2: Expanded Summary of Sub-G2 New Network Capabilities and
Services
This appendix expands on the ITU-T Sub-G2 new network capabilities
and services introcuced in Section 3 It builds upon the analysis that
was conducted at ITU-T FG-NET2030; readers are also referred to
output produced by that group [NET2030SubG2] for more detail.
11.1. New Services
[NET2030SubG2] identifies a number of network services that will be
needed to support many of the new use cases. These network services
are divided into two categories:
o Foundational Services (FS) require which dedicated support on some
or all network system nodes which are delivering the service
between two or more application system nodes. FS cannot be
decomposed into other services. For example, IP packet routing
and forwarding are is a (pre-existing) foundational network
services.
o Compound Services (CS) are composed of one or more foundational
services. CS are "convenience services" that make network
services easier to consume by certain applications or categories
of use cases, but do not by themselves introduce new network
services or requirements into network system nodes. One example
would be a Tactile Internet Service which consists of two
communications channels, one for tactile control and the other for
haptic feedback.
The following sections focus on Foundational Services only, as these
are the ones that provide the basic building blocks with which the
needs of all other services can be addressed, and which are the ones
that potentially introduce new foundational requirements on network
system nodes.
11.1.1. High-Precision Communications Services
High-Precision Communications Services refers to services that have
precisely defined service level objectives related to end-to-end
latency, in many cases coupled with stringent requirements regarding
to packet loss and to bandwidth needs. These requirements are in
stark contrast to the best effort nature with related to existing
network services.
Of course, existing services often go to great lengths in order to
optimize service levels and minimize latency, and QoS techniques aim
to mitigate adverse effects of e.g. congestion by applying various
forms of prioritization and admission control. However,
Bryant, et al. Expires September 10, 2020 [Page 32]
Internet-Draft FWD PS March 2020
fundamentally all of these techniques still constitute patches that,
while alleviating the symptoms of the underlying best-effort nature,
do not address the underlying cause and fall short of providing
service level guarantees that will not be just of a statistical
nature but that will be met by design.
The high-precision communications services that have been identified
are described in the following three sub-sections.
11.1.2. In-time Services
In-time services require end-to-end latency within a quantifiable
limit. They specific a service level objective that is not to be
exceeded, such as a maximum acceptable latency (putting a hard
boundary on the worst case). In-time services are required by
applications and use cases that have clear bounds on acceptable
latency, beyond which the Quality of Experience would deteriorate
rapidly, rendering the application unusable. An example concerns use
cases that involve providing tactile feedback to users. Creating an
illusion of touch requires a control loop with a hard-bounded round-
trip time that is determined by human / biological factors, beyond
which the sense of touch is lost and with it the ability to e.g.
operate a piece of machinery from remote. Because many such use
cases are mission-critical (such as tele-driving or remote surgery),
in addition any loss or need for retransmission is unacceptable.
This service is similar to the service provided by DetNet [RFC8655]
but with more demanding applications which need to be satisfied over
IP.
11.1.3. On-time Services
On-time services require end-to-end-latency to be of an exact
duration, with the possibility of a small quantifiable variance as
can be specified either by an acceptable window around the targeted
latency or by a lower bound in addition to an upper bound. Examples
of use cases include applications that require synchronization
between multiple flows that have the same in-time latency target, or
applications requiring fairness between multiple participants
regardless of path lengths, such as gaming or market exchanges when
required by regulatory authorities. The concept of a lowest
acceptable latency imposes new requirements on networks to
potentially slow down packets by buffering or other means, which
introduces challenges due to high data rates and the cost e.g. of
associated memory.
Bryant, et al. Expires September 10, 2020 [Page 33]
Internet-Draft FWD PS March 2020
11.1.4. Coordinated Services
Coordinated services require multiple interdependent flows to be
delivered with the same end-to-end latency, regardless of any
(potential additional) service level objective. Use cases and
applications include applications that require synchronization
between multiple flows, such as use cases involving data streams from
multiple cameras and telemetry sources. In the special case where an
on-time service is required, no additional service is needed (as
synchronization occurs by virtue of the fact that each flow adheres
to the same SLO), but coordination may also be required in cases
where no specific end-to-end latency is required, as long as all
flows are serviced with service levels that are identical.
11.1.5. Qualitative Communication Services
Qualitative communication services (QCS) are able to suppress
retransmission of portions of the payload that are deemed less
relevant when necessary in order to meet requirements on latency by
applications that are tolerant of certain quality degradation. They
may involve the application of network coding schemes.
QCS is a new service type that is needed to support AR/VR,
holographic-type communications Industrial Internet and services such
as autonomous driving. This needs the support of a new network
capability that is as yet to be developed.
11.2. New Capabilities
[NET2030SubG2] identifies also a number of network capabilities that
will become increasingly important going forward, in addition to the
support for any particular services. These were introduced in
Section 3.2. A number of these capabilities need to be taken into
consideration from the very beginning when thinking about how future
dataplanes need to evolve.
While many of those capabilities are well known, the past has shown
that retrofitting dataplanes with such capabilities after the fact
and in a way that is adequate to the problem at hand is very hard.
11.2.1. Manageability
Many of the services that need to be supported in the future have in
common that they place very high demands on latency and precision
that need to be supported at very high scales, coupled with
expectations of zero packet loss and much higher availability than
today.
Bryant, et al. Expires September 10, 2020 [Page 34]
Internet-Draft FWD PS March 2020
In order to assure in-time and on-time services with high levels of
accuracy, advances in measurements and telemetry will be required in
order to monitor and validate that promised service levels are indeed
being delivered. This requires advanced instrumentation that is
ideally built-in all the way to the protocol level.
For example, the ability to identify and automatically eliminate
potential sources of service-level degradations and fluctuations will
become of increasing importance. This requires the ability to
generate corresponding telemetry data and the ability to observe the
performance of packets as they traverse the network. Some of the
challenges that need to be addressed include the very high volume of
data that gets generated and needs to be assessed, and the effects of
the collection itself on performance. In general, greater emphasis
will need to be placed on the ability to monitor, observe, and
validate packet performance and behavior than is the case today. For
seamless support, these capabilities will be inherently integrated
with the forwarding function itself, for example delivered together
with the packets. Today's solution approach, IOAM, is a promising
technology currently that points in the right direction, and that
also highlights some of the challenges - from MTU considerations due
to extending packet sizes to the ability to customize and obtain the
"right" data. It will therefore be not sufficient by itself. Data
to be generated from the network will need to be "smarter", i.e. more
insightful and action-able. This will require additional abilities
to process data "on-device". In additional, the need for new
management functions may arise, such as functions that allow to
validate adherence with agreed-upon service levels for a flow as a
whole, and to prevent data or privacy leakage as well as provide
evidence for the possibility or absence of such leakage.
11.2.2. High Programmability and Agile Lifecycle
Operators need to be able to rapidly introduce new network services
and adapt to new contexts and application needs. This will require
advances in network programmability. Today's model of vendor-defined
(supporting service features via new firmware or hardware-based
networking features) or operator-defined (supporting service features
via programmable software-defined networking (SDN) controllers,
virtualized network functions (VNF) and Network Function
Virtualization (NFV), and service function chaining (SFC) will no
longer be sufficient.
Software Defined Networking and Network Function Virtualization (NFV)
have opened up the possibility to accelerate development life-cycles
and enable network providers to develop new networking features on
their own if needed. Segment Routing is being evolved for that
purpose as well. Furthermore, network slicing promises more agility
Bryant, et al. Expires September 10, 2020 [Page 35]
Internet-Draft FWD PS March 2020
in the introduction of new network services. However, the complexity
of the associated controller software results in its own challenges
with software development cycles that, while more agile than life-
cycles before, are still prohibitive and that can only be undertaken
by network providers, not by their customers. Rapid customization of
networking services for specific needs or adaptation to unique
deployments are out of reach for network provider customers. What is
lacking is the ability for applications to rapidly introduce and
customize novel behavior at the network flow level, without need to
introduce application-level over-the-top (OTT) overlays. Such a
capability would be analogous to server-less computing that is
revolutionizing cloud services today. In addition, it should be
noted that softwarized networks are built on relatively stable (and
slowly evolving) underlying physical commodity hardware network
infrastructure. This is insufficient to deliver on new high-
precision network services, which require hardware advances at many
levels to provide programmable flow and QoS behavior at line rate,
affecting everything from queuing and scheduling to packet processing
pipelines.
The evolution of forwarding planes should allow development life-
cycles that are much more agile than today and move from "Dev Ops" to
"Flow Ops" (i.e. dynamic programmability of networks at the flow
level).
This requires support of novel network and data-plane programming
models which can possibly be delivered and effected via the
forwarding plane itself.
11.2.3. Security
The possibility of security threats increases with complexity of
networks, the potential ramifications of attacks are growing more
serious with increasing mission-criticality of networking services
and applications.
The forwarding plane plays a large role in the ability to thwart
attacks.
For example, the fact that source addresses are not authenticated in
existing IP is at the root of a wide range security problems from
phishing and fraudulent impersonation designed to compromise and
steal user assets to amplification attacks designed to bring down
services.
Going forward, it is absolutely critical, then, to minimize the
attack surface of the forwarding plane as it evolves.
A key security aspects needed from the network point of view includes
to verify if the packet is authorized to enter into the network and
if it is sufficiently integrity protected. However, when packets are
emitted from the host for these new communication services, the
Bryant, et al. Expires September 10, 2020 [Page 36]
Internet-Draft FWD PS March 2020
network portion of the packet (e.g., an extension header or an
overlay header) should not be encrypted because network nodes may
need to interpret the header and provide the desired service.
Lack of encryption and integrity validation, of course, would at the
same time increase the threat surface and open up the possibility for
attacks.
Mechanisms for authorization and integrity protection must be
developed to meet the line rate performance as services delivered can
be time sensitive. At the same time, the size of packets should not
be significantly increased to avoid negative impact on utilization
and overhead tax.
This limits the options for additional security collateral that can
be included with packets.
Homomorphic forms of encryption may need to be devised in which
network operations can be performed in privacy-preserving manner on
encrypted packet headers and tunneled packets without exposing any of
their contents.
Another dimension to security arises when the end to end service that
needs to be delivered crosses the administrative boundary of the
originating host. For those cases, additional mechanisms need to be
specified to sufficiently ensure the privacy and confidentiality of
the network layer information. While there are lot of avenues to
tackle these issues and some aspects are being looked into by various
Standards Development Organizations, e.g. IRTF PANRG on Path-Aware
Networking, comprehensive solutions are yet to be worked out.
Any mechanisms specified for authorization, integrity protection, and
network header confidentiality should be orthogonal to the transport
layer and above transport layer security mechanisms set in place by
the end host/user. Regardless of whether or not the latest security
advances in transport and layers above (e.g. TLS1.3, QUIC or HTTPSx)
are applied on the payload, network nodes should not have to act on
this information to deliver new services to avoid layer violations.
11.2.4. Trustworthiness
As future network services are deployed, deployment scenarios will
include cases in which packets need to traverse trust boundaries
which are under different administrative domains. As the forwarding
plane evolves, it should do so in such a way that trustworthiness of
packets is maintained - i.e. integrity of data is protected,
tampering with packet meta-data (such as source authentication or
service level telemetry) would be evident, and privacy of users is
guarded.
Bryant, et al. Expires September 10, 2020 [Page 37]
Internet-Draft FWD PS March 2020
11.2.5. Resilience
Ultra-low-latency requirements and the huge increase of bandwidth
demands of new services such as holographic type communication
services make retransmission as a mechanism to recover data that was
lost in transit increasingly less feasible. Therefore, network
resilience and avoidance of loss becomes of paramount importance.
There are many methods for providing network resilience. This
includes providing redundancy and diversity of both physical (e.g.
ports, routers, line cards) and logical (e.g. shapers, policers,
classifiers) entities. It also includes the use of protocols that
provide quick re-convergence and maintain high availability of
existing connections after a failure event occurs in the network.
Other techniques include packet replication or network coding and
error correction techniques to overcome packet loss.
As the forwarding plane evolves, mechanisms to provide network
resilience should be inherently supported.
11.2.6. Privacy-Sensitive
Today, there is a growing awareness of the lack of privacy in the
Internet and its implications.
New network services have to be sensitive to and comply with
heightened user privacy expectations.
At the same time, the need for privacy needs to be balanced with
legitimate needs of network providers to operate and maintain their
networks, which requires some visibility into what is happening on
the network and how it is being used.
Likewise, mechanisms to provide privacy must be provided in such a
way to not compromise security, such as allowing anonymous attackers
to prey on other users.
An evolved forwarding plane must provide mechanisms that ensure users
privacy by design and prevent illegitimate exposing of personally-
identifiable information (PII), while preventing abuse of those
mechanisms by attack exploits and while affording network providers
with legitimate visibility into use of their network and services.
There are a variety of privacy-related requirements that ensue, such
as:
o Anonymization: To prevent tracking by eavesdropper by packet
capture, visible information in packets such as source and
destination addresses should be difficult (ideally: impossible) to
directly correlate to PII.
Bryant, et al. Expires September 10, 2020 [Page 38]
Internet-Draft FWD PS March 2020
o Opaque User data: Networks must not rely on the user data to
provide or improve the service. However, network providers may
use specific service-visible data in packets.
o Secured Storage: Some services may require the network to slow
down the delivery of the packets, implying the possibility that
packets are temporarily buffered on the router. The storage of
those packets must be secured and prevented from extraction for
deep inspection or analysis.
o Flow anonymization: Flows of information should be randomized in a
dynamic manner so that it is difficult through traffic analysis to
deduce patterns and identify the type of traffic.
Potential mechanisms to consider include (but are not limited to)
avoiding the need for long-lived addresses (to prevent trackablity)
and the use of homomorphic encryption for packet headers and tunneled
packets (in addition to traditional payload encryption) that allow to
perform network operations in privacy-preserving manner without
exposing meta-data carried in headers.
11.2.7. Accountability and Verifiability
Many new services demand guarantees instead of being accepting of
"best effort".
As a result, today's "best effort" accounting may no longer be
sufficient.
Today's accounting technology largely relies on interface statistics
and flow records.
Those statistics and records may not be entirely accurate.
For example, in many cases their generation involves sampling and is
thus subject to sampling inaccuracies.
In addition, this data largely accounts for volume but not so much
for actual service levels (e.g. latencies, let alone coordination
across flows) that are delivered.
Service level measurements can be used to complement other statistics
but come with significant overhead and also have various limitations,
from sampling to the consumption of network and edge node processing
bandwidth.
Techniques that rely on passive measurements are infeasible in many
network deployments and hampered by encryption as well as issues
relating to privacy.
Guarantees demand their price. This makes it increasingly important
both for providers and users of services to be able to validate that
promised service levels were delivered on.
Bryant, et al. Expires September 10, 2020 [Page 39]
Internet-Draft FWD PS March 2020
For example, proof of service delivery (including proof of service
level delivery) may need to be provided to account and charge for
network services.
This will require advances in accounting technology that should be
considered as forwarding technology evolves, possibly providing
accounting as a function that is intrinsically coupled with
forwarding itself.
12. References
12.1. Normative References
[RFC0791] Postel, J., "Internet Protocol", STD 5, RFC 791,
DOI 10.17487/RFC0791, September 1981,
<https://www.rfc-editor.org/info/rfc791>.
[RFC8200] Deering, S. and R. Hinden, "Internet Protocol, Version 6
(IPv6) Specification", STD 86, RFC 8200,
DOI 10.17487/RFC8200, July 2017,
<https://www.rfc-editor.org/info/rfc8200>.
12.2. Informative References
[DIP1] ETSI, "Recommendation for New Transport Technologies, GR
NGP 010", September 2018,
<https://www.etsi.org/deliver/etsi_gr/
NGP/001_099/010/01.01.01_60/gr_NGP010v010101p.pdf>.
[DOT] Huston, G., "The Death of Transit and Beyond", n.d.,
<https://hknog.net/wp-content/uploads/2018/03/01_GeoffHust
on_TheDeath_of_Transit_and_Beyond.pdf>.
[FGNETWORK2030]
"Focus Group on Technologies for Network 2030", n.d.,
<https://www.itu.int/en/ITU-T/focusgroups/net2030/Pages/
default.aspx>.
[Handley] Handley, M., "Delay is Not an Option: Low Latency Routing
in Space", n.d.,
<http://nrg.cs.ucl.ac.uk/mjh/starlink-draft.pdf>.
[I-D.bonica-6man-comp-rtg-hdr]
Bonica, R., Kamite, Y., Niwa, T., Alston, A., and L.
Jalil, "The IPv6 Compressed Routing Header (CRH)", draft-
bonica-6man-comp-rtg-hdr-13 (work in progress), March
2020.
Bryant, et al. Expires September 10, 2020 [Page 40]
Internet-Draft FWD PS March 2020
[I-D.bonica-spring-srv6-plus]
Bonica, R., Hegde, S., Kamite, Y., Alston, A., Henriques,
D., Jalil, L., Halpern, J., Linkova, J., and G. Chen,
"Segment Routing Mapped To IPv6 (SRm6)", draft-bonica-
spring-srv6-plus-06 (work in progress), October 2019.
[I-D.carpenter-limited-domains]
Carpenter, B. and B. Liu, "Limited Domains and Internet
Protocols", draft-carpenter-limited-domains-13 (work in
progress), February 2020.
[I-D.chunduri-isis-preferred-path-routing]
Chunduri, U., Li, R., White, R., Tantsura, J., Contreras,
L., and Y. Qu, "Preferred Path Routing (PPR) in IS-IS",
draft-chunduri-isis-preferred-path-routing-00 (work in
progress), June 2018.
[I-D.filsfils-spring-net-pgm-extension-srv6-usid]
Filsfils, C., Camarillo, P., Cai, D., Voyer, D., Meilik,
I., Patel, K., Henderickx, W., Jonnalagadda, P., and D.
Melman, "Network Programming extension: SRv6 uSID
instruction", draft-filsfils-spring-net-pgm-extension-
srv6-usid-04 (work in progress), February 2020.
[I-D.filsfils-spring-srv6-network-programming]
Filsfils, C., Camarillo, P., Leddy, J.,
daniel.voyer@bell.ca, d., Matsushima, S., and Z. Li, "SRv6
Network Programming", draft-filsfils-spring-srv6-network-
programming-07 (work in progress), February 2019.
[I-D.han-tsvwg-enhanced-diffserv]
Han, L., Qu, Y., and R. Li, "Enhanced DiffServ by In-band
Signaling", draft-han-tsvwg-enhanced-diffserv-00 (work in
progress), November 2019.
[I-D.han-tsvwg-ip-transport-qos]
Han, L., Qu, Y., Dong, L., Li, R., Nadeau, T., Smith, K.,
and J. Tantsura, "Resource Reservation Protocol for IP
Transport QoS", draft-han-tsvwg-ip-transport-qos-03 (work
in progress), October 2019.
[I-D.herbert-6man-eh-attrib]
Herbert, T., "Attribution Option for Extension Header
Insertion", draft-herbert-6man-eh-attrib-00 (work in
progress), December 2019.
Bryant, et al. Expires September 10, 2020 [Page 41]
Internet-Draft FWD PS March 2020
[]
Filsfils, C., Dukes, D., Previdi, S., Leddy, J.,
Matsushima, S., and D. Voyer, "IPv6 Segment Routing Header
(SRH)", draft-ietf-6man-segment-routing-header-26 (work in
progress), October 2019.
[I-D.ietf-detnet-ip]
Varga, B., Farkas, J., Berger, L., Fedyk, D., Malis, A.,
and S. Bryant, "DetNet Data Plane: IP", draft-ietf-detnet-
ip-05 (work in progress), February 2020.
[I-D.ietf-detnet-mpls]
Varga, B., Farkas, J., Berger, L., Fedyk, D., Malis, A.,
Bryant, S., and J. Korhonen, "DetNet Data Plane: MPLS",
draft-ietf-detnet-mpls-05 (work in progress), February
2020.
[I-D.ietf-detnet-security]
Mizrahi, T., Grossman, E., Hacker, A., Das, S., Dowdell,
J., Austad, H., and N. Finn, "Deterministic Networking
(DetNet) Security Considerations", draft-ietf-detnet-
security-08 (work in progress), February 2020.
[I-D.ietf-ippm-ioam-data]
Brockners, F., Bhandari, S., Pignataro, C., Gredler, H.,
Leddy, J., Youell, S., Mizrahi, T., Mozes, D., Lapukhov,
P., remy@barefootnetworks.com, r., daniel.bernier@bell.ca,
d., and J. Lemon, "Data Fields for In-situ OAM", draft-
ietf-ippm-ioam-data-08 (work in progress), October 2019.
[I-D.ietf-tsvwg-intserv-multiple-tspec]
Polk, J. and S. Dhesikan, "Integrated Services (IntServ)
Extension to Allow Signaling of Multiple Traffic
Specifications and Multiple Flow Specifications in
RSVPv1", draft-ietf-tsvwg-intserv-multiple-tspec-02 (work
in progress), February 2013.
[NET2030SubG2]
ITU-T FGNET2030, "New Services and Capabilities for
Network 2030: Description, Technical Gap and Performance
Target Analysis", October 2019, <https://www.itu.int/en/
ITU-T/focusgroups/net2030/Documents/
Deliverable_NET2030.pdf>.
[RFC1347] Callon, R., "TCP and UDP with Bigger Addresses (TUBA), A
Simple Proposal for Internet Addressing and Routing",
RFC 1347, DOI 10.17487/RFC1347, June 1992,
<https://www.rfc-editor.org/info/rfc1347>.
Bryant, et al. Expires September 10, 2020 [Page 42]
Internet-Draft FWD PS March 2020
[RFC2205] Braden, R., Ed., Zhang, L., Berson, S., Herzog, S., and S.
Jamin, "Resource ReSerVation Protocol (RSVP) -- Version 1
Functional Specification", RFC 2205, DOI 10.17487/RFC2205,
September 1997, <https://www.rfc-editor.org/info/rfc2205>.
[RFC2460] Deering, S. and R. Hinden, "Internet Protocol, Version 6
(IPv6) Specification", RFC 2460, DOI 10.17487/RFC2460,
December 1998, <https://www.rfc-editor.org/info/rfc2460>.
[RFC2507] Degermark, M., Nordgren, B., and S. Pink, "IP Header
Compression", RFC 2507, DOI 10.17487/RFC2507, February
1999, <https://www.rfc-editor.org/info/rfc2507>.
[RFC2711] Partridge, C. and A. Jackson, "IPv6 Router Alert Option",
RFC 2711, DOI 10.17487/RFC2711, October 1999,
<https://www.rfc-editor.org/info/rfc2711>.
[RFC3095] Bormann, C., Burmeister, C., Degermark, M., Fukushima, H.,
Hannu, H., Jonsson, L-E., Hakenberg, R., Koren, T., Le,
K., Liu, Z., Martensson, A., Miyazaki, A., Svanbro, K.,
Wiebke, T., Yoshimura, T., and H. Zheng, "RObust Header
Compression (ROHC): Framework and four profiles: RTP, UDP,
ESP, and uncompressed", RFC 3095, DOI 10.17487/RFC3095,
July 2001, <https://www.rfc-editor.org/info/rfc3095>.
[RFC3208] Speakman, T., Crowcroft, J., Gemmell, J., Farinacci, D.,
Lin, S., Leshchiner, D., Luby, M., Montgomery, T., Rizzo,
L., Tweedly, A., Bhaskar, N., Edmonstone, R.,
Sumanasekera, R., and L. Vicisano, "PGM Reliable Transport
Protocol Specification", RFC 3208, DOI 10.17487/RFC3208,
December 2001, <https://www.rfc-editor.org/info/rfc3208>.
[RFC4080] Hancock, R., Karagiannis, G., Loughney, J., and S. Van den
Bosch, "Next Steps in Signaling (NSIS): Framework",
RFC 4080, DOI 10.17487/RFC4080, June 2005,
<https://www.rfc-editor.org/info/rfc4080>.
[RFC4193] Hinden, R. and B. Haberman, "Unique Local IPv6 Unicast
Addresses", RFC 4193, DOI 10.17487/RFC4193, October 2005,
<https://www.rfc-editor.org/info/rfc4193>.
[RFC4302] Kent, S., "IP Authentication Header", RFC 4302,
DOI 10.17487/RFC4302, December 2005,
<https://www.rfc-editor.org/info/rfc4302>.
Bryant, et al. Expires September 10, 2020 [Page 43]
Internet-Draft FWD PS March 2020
[RFC5795] Sandlund, K., Pelletier, G., and L-E. Jonsson, "The RObust
Header Compression (ROHC) Framework", RFC 5795,
DOI 10.17487/RFC5795, March 2010,
<https://www.rfc-editor.org/info/rfc5795>.
[RFC6282] Hui, J., Ed. and P. Thubert, "Compression Format for IPv6
Datagrams over IEEE 802.15.4-Based Networks", RFC 6282,
DOI 10.17487/RFC6282, September 2011,
<https://www.rfc-editor.org/info/rfc6282>.
[RFC6398] Le Faucheur, F., Ed., "IP Router Alert Considerations and
Usage", BCP 168, RFC 6398, DOI 10.17487/RFC6398, October
2011, <https://www.rfc-editor.org/info/rfc6398>.
[RFC6554] Hui, J., Vasseur, JP., Culler, D., and V. Manral, "An IPv6
Routing Header for Source Routes with the Routing Protocol
for Low-Power and Lossy Networks (RPL)", RFC 6554,
DOI 10.17487/RFC6554, March 2012,
<https://www.rfc-editor.org/info/rfc6554>.
[RFC6790] Kompella, K., Drake, J., Amante, S., Henderickx, W., and
L. Yong, "The Use of Entropy Labels in MPLS Forwarding",
RFC 6790, DOI 10.17487/RFC6790, November 2012,
<https://www.rfc-editor.org/info/rfc6790>.
[RFC8578] Grossman, E., Ed., "Deterministic Networking Use Cases",
RFC 8578, DOI 10.17487/RFC8578, May 2019,
<https://www.rfc-editor.org/info/rfc8578>.
[RFC8655] Finn, N., Thubert, P., Varga, B., and J. Farkas,
"Deterministic Networking Architecture", RFC 8655,
DOI 10.17487/RFC8655, October 2019,
<https://www.rfc-editor.org/info/rfc8655>.
[RFC8698] Zhu, X., Pan, R., Ramalho, M., and S. Mena, "Network-
Assisted Dynamic Adaptation (NADA): A Unified Congestion
Control Scheme for Real-Time Media", RFC 8698,
DOI 10.17487/RFC8698, February 2020,
<https://www.rfc-editor.org/info/rfc8698>.
[UC] ITU-T FGNET2030, "Use Cases and Requirements for Network
2030 Summary report "Representative use cases and key
network requirements for Network 2030"", January 2020,
<https://www.itu.int/en/ITU-
T/focusgroups/net2030/Documents/Technical_Report.pdf>.
Bryant, et al. Expires September 10, 2020 [Page 44]
Internet-Draft FWD PS March 2020
[WP] "Network 2030 - A Blueprint of Technology, Applications,
and Market Drivers towards the Year 2030 and Beyond, a
White Paper on Network 2030, ITU-T", May 2019,
<https://www.itu.int/en/ITU-
T/focusgroups/net2030/Documents/White_Paper.pdf>.
Authors' Addresses
Stewart Bryant
Futurewei Technologies Inc.
Email: sb@stewartbryant.com
Uma Chunduri
Futurewei Technologies Inc.
Email: uma.chunduri@futurewei.com
Toerless Eckert
Futurewei Technologies Inc.
Email: tte@cs.fau.de
Alexander Clemm
Futurewei Technologies Inc.
Email: ludwig@clemm.org
Bryant, et al. Expires September 10, 2020 [Page 45]