Skip to main content

DetNet Data Plane solution
draft-dt-detnet-dp-sol-00

The information below is for an old version of the document.
Document Type
This is an older version of an Internet-Draft whose latest revision state is "Replaced".
Authors Jouni Korhonen , Loa Andersson , Yuanlong Jiang , Balazs Varga , János Farkas , Carlos J. Bernardos , Tal Mizrahi
Last updated 2017-03-13
Replaced by draft-ietf-detnet-dp-sol
RFC stream (None)
Formats
Additional resources
Stream Stream state (No stream defined)
Consensus boilerplate Unknown
RFC Editor Note (None)
IESG IESG state I-D Exists
Telechat date (None)
Responsible AD (None)
Send notices to (None)
draft-dt-detnet-dp-sol-00
DetNet                                                  J. Korhonen, Ed.
Internet-Draft                                                  Broadcom
Intended status: Informational                              L. Andersson
Expires: September 14, 2017                                     Y. Jiang
                                                                  Huawei
                                                                B. Varga
                                                               J. Farkas
                                                                Ericsson
                                                           CJ. Bernardos
                                                                    UC3M
                                                              T. Mizrahi
                                                                 Marvell
                                                          March 13, 2017

                       DetNet Data Plane solution
                       draft-dt-detnet-dp-sol-00

Abstract

   This document specifies a PseudoWire-based Deterministic Networking
   data plane solution.  The data plane solution can be applied over
   either IP or MPLS Packet Switched Networks.

Status of This Memo

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF).  Note that other groups may also distribute
   working documents as Internet-Drafts.  The list of current Internet-
   Drafts is at http://datatracker.ietf.org/drafts/current/.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   This Internet-Draft will expire on September 14, 2017.

Copyright Notice

   Copyright (c) 2017 IETF Trust and the persons identified as the
   document authors.  All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents

Korhonen, et al.       Expires September 14, 2017               [Page 1]
Internet-Draft         DetNet data plane solution             March 2017

   (http://trustee.ietf.org/license-info) in effect on the date of
   publication of this document.  Please review these documents
   carefully, as they describe your rights and restrictions with respect
   to this document.  Code Components extracted from this document must
   include Simplified BSD License text as described in Section 4.e of
   the Trust Legal Provisions and are provided without warranty as
   described in the Simplified BSD License.

Table of Contents

   1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   2
   2.  Terminology . . . . . . . . . . . . . . . . . . . . . . . . .   3
   3.  Requirements language . . . . . . . . . . . . . . . . . . . .   4
   4.  DetNet data plane Overview  . . . . . . . . . . . . . . . . .   4
     4.1.  DetNet data plane solution requirements . . . . . . . . .   6
   5.  DetNet data plane solution  . . . . . . . . . . . . . . . . .   6
     5.1.  DetNet Control Word . . . . . . . . . . . . . . . . . . .   7
     5.2.  DetNet flow identity word . . . . . . . . . . . . . . . .   7
     5.3.  DetNet encapsulation  . . . . . . . . . . . . . . . . . .   8
   6.  PE reference model considerations . . . . . . . . . . . . . .  11
     6.1.  Forwarded clarifications  . . . . . . . . . . . . . . . .  11
     6.2.  DA-T-PE processing clarifications . . . . . . . . . . . .  12
     6.3.  DA-S-PE processing clarifications . . . . . . . . . . . .  14
   7.  Other DetNet considerations . . . . . . . . . . . . . . . . .  15
     7.1.  Class of Service  . . . . . . . . . . . . . . . . . . . .  15
     7.2.  Quality of Service  . . . . . . . . . . . . . . . . . . .  15
     7.3.  Time synchronization  . . . . . . . . . . . . . . . . . .  15
     7.4.  Bidirectional traffic . . . . . . . . . . . . . . . . . .  16
   8.  Control plane considerations  . . . . . . . . . . . . . . . .  17
     8.1.  PW Label assignment and distribution  . . . . . . . . . .  17
   9.  Security considerations . . . . . . . . . . . . . . . . . . .  17
   10. IANA Considerations . . . . . . . . . . . . . . . . . . . . .  17
   11. Acknowledgements  . . . . . . . . . . . . . . . . . . . . . .  17
   12. References  . . . . . . . . . . . . . . . . . . . . . . . . .  18
     12.1.  Normative References . . . . . . . . . . . . . . . . . .  18
     12.2.  Informative References . . . . . . . . . . . . . . . . .  18
   Appendix A.  Example of DetNet data plane operation . . . . . . .  19
   Appendix B.  Example of pinned paths using IP PSN . . . . . . . .  21
   Authors' Addresses  . . . . . . . . . . . . . . . . . . . . . . .  21

1.  Introduction

   This document specifies a Deterministic Networking (DetNet) data
   plane solution.  The solution is based on PseudoWires (PW) [RFC3985]
   and makes use of the multi-segment (MS-PW) [RFC6073] to map DetNet
   Relay and Edge Nodes
   [I-D.ietf-detnet-architecture][I-D.ietf-detnet-dp-alt] to PW

Korhonen, et al.       Expires September 14, 2017               [Page 2]
Internet-Draft         DetNet data plane solution             March 2017

   architecture.  The PW-based data plane can be run over either an IP
   or MPLS [RFC4448][RFC6658] Packet Switched Network (PSN).

   For the purpose of DetNet data plane, this document specifically
   specifies the PW encapsulation for DetNet flows, a DetNet Control
   Word (CW), a DetNet label, how MS-PW derived DetNet Relay and Edge
   nodes work, and as a specific new PW feature how the Packet
   Replication and Elimination function (PREF) is implemented using PWs.
   This document does not define the associated control plane functions,
   or operations and management (OAM).

2.  Terminology

   This document uses the terminology established in the DetNet
   architecture [I-D.ietf-detnet-architecture] and the DetNet Data Plane
   Solution Alternatives [I-D.ietf-detnet-dp-alt].

   The following terms are also used in this document:

   DA-T-PE       A DetNet aware PseudoWire Terminating Provider Edge
                 (T-PE).

   DA-S-PE       A DetNet aware PseudoWire Switching Provider Edge
                 (S-PE).

   T-Label       A hop-by-hop tunnel label layer between label switching
                 routers (LSR).

   L-Label       A DetNet topology overlay label that is used between
                 DA-*-PE devices.

   flow-ID       A DetNet flow identity that uniquely identifies a
                 DetNet flow in a DetNet network.  The flow-ID is part
                 of the PseudoWire Encapsulation header.

   local-ID      A DA-T-PE and DA-S-PE node internal construct that
                 uniquely identifies a DetNet flow.  The local-ID can be
                 equal to flow-ID or be derived using other means, e.g.,
                 programming required label to local-ID mappings
                 directly into the label information base (LFIB).

   PW Label      A PseudoWire label that is used to identify DetNet flow
                 related PW Instances within a PE node.

   PREF          A Packet Replication and Elimination Function (PREF)
                 does the replication and elimination processig of
                 DetNet flow packets in DA-T-PE or DA-S-PE nodes.  The
                 replication function is essentially the existing 1+1

Korhonen, et al.       Expires September 14, 2017               [Page 3]
Internet-Draft         DetNet data plane solution             March 2017

                 protection mechanism.  The elimination function reuses
                 and extends the existing [RFC3985] PseudoWire
                 sequencing provided duplicate detection mechanism to
                 operate over multiple (separate) PseudoWires that are
                 sub-flows of a compound DetNet flow.

3.  Requirements language

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL" "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
   document are to be interpreted as described in [RFC2119].

4.  DetNet data plane Overview

   [Ed. to be written.. describe the scope here fot this document: this
   document only addresses the inter-connect case i.e., 802.1 over
   routed network (enlarge the layer-2 domain - EVPAN', and the native
   DetNet case.]

  TSN              Edge          Transit        Relay        DetNet
  End System       Node            Node         Node         End System

  +---------+    +.........+                                 +---------+
  |  Appl.  |<---:Svc Proxy:-- End to End Service ---------->|  Appl.  |
  +---------+    +---------+                   +---------+   +---------+
  |   TSN   |    |TSN| |Svc|<-- DetNet flow ---: Service :-->| Service |
  +---------+    +---+ +---+    +---------+    +---------+   +---------+
  |Transport|    |Trp| |Trp|    |Transport|    |Trp| |Trp|   |Transport|
  +-------.-+    +-.-+ +-.-+    +--.----.-+    +-.-+ +-.-+   +---.-----+
          :  Link  :    /  ,-----.  \   :  Link  :    /  ,-----.  \
          +........+    +-[  Sub  ]-+   +........+    +-[  Sub  ]-+
                          [Network]                     [Network]
                           `-----'                       `-----'

          Figure 1: A simple DetNet enabled network architecture

   Figure 2 illustrates how DetNet can provide services for IEEE
   802.1TSN end systems over a DetNet enabled network.  The edge nodes
   insert and remove required DetNet data plane encapsulation.  The 'X'
   in the edge and relay nodes represents a potential DetNet flow packet
   replication and elimination point.  This conceptually parallels L2VPN
   services, and could leverage existing related solutions as discussed
   below.

Korhonen, et al.       Expires September 14, 2017               [Page 4]
Internet-Draft         DetNet data plane solution             March 2017

      TSN    |<---------- End to End DetNet Service ------>|  TSN
     Service |           Transit           Transit         | Service
 TSN  (AC)   |        |<-Tunnel->|        |<-Tnl->|        |  (AC)  TSN
 End    |    V        V     1    V        V   2   V        V   |    End
 System |    +--------+          +--------+       +--------+   |  System
 +---+  |    |DA-T-PE1|==========|DA-S-PE1|=======|DA-T-PE2|   |   +---+
 |   |--|----|._X_....|..DetNet..|.._ _...|..DF3..|...._X_.|---|---|   |
 |CE1|  |    |    \   |  Flow 1  |   X    |       |   /    |   |   |CE2|
 |   |       |     \_.|...DF2....|._/ \_..|..DF4..|._/     |       |   |
 +---+       |        |==========|        |=======|        |       +---+
     ^       +--------+          +--------+       +--------+       ^
     |        Edge Node          Relay Node       Edge Node        |
     |                                                             |
     |<----- Emulated Time Sensitive Networking (TSN) Service ---->|

                    Figure 2: IEEE 802.1TSN over DetNet

   Figure 3 illustrates how end to end DetNet service can be provided.
   In this case, the end systems are able to send and receive DetNet
   flows.  For example, put application data in PseudoWire (PW) and
   encapsulated in IP.  Like earlier the 'X' in the end systems, edge
   and relay nodes represents potential DetNet flow packet replication
   and elimination points.  Here the relay nodes may change the
   underlying transport, for example replacing IP with MPLS or tunneling
   IP over MPLS, or simply interconnect network segments.

         DetNet                                             DetNet
         Service          Transit          Transit          Service
   DetNet  |             |<-Tnl->|        |<-Tnl->|            | DetNet
   End     |             V   1   V        V   2   V            | End
   System  |    +--------+       +--------+       +--------+   | System
   +---+   |    |DA-S-PE1|=======|DA-S-PE2|=======|DA-S-PE3|   |  +---+
   |  X...DFa...|._X_....|..DF1..|.__ ___.|..DF3..|...._X_.|.DFa..|.X |
   |CE1|========|    \   |       |   X    |       |   /    |======|CE2|
   |   |   |    |     \_.|..DF2..|._/ \__.|..DF4..|._/     |   |  |   |
   +---+        |        |=======|        |=======|        |      +---+
       ^        +--------+       +--------+       +--------+      ^
       |       Relay Node       Relay Node       Relay Node       |
       |                                                          |
       |<-- End to End Time Sensitive Networking (TSN) Service -->|

                          Figure 3: Native DetNet

Korhonen, et al.       Expires September 14, 2017               [Page 5]
Internet-Draft         DetNet data plane solution             March 2017

4.1.  DetNet data plane solution requirements

   Two major groups of scenarios can be distinguished which require flow
   identification during transport:

   1.  DetNet function related scenarios:

       *  Congestion protection: usage of allocated resources (queuing,
          policing, shaping).

       *  Explicit routes: select/apply the flow specific path.

       *  Service protection: recognize compound / member flows for
          replication an elimination.

   2.  OAM function related scenarios:

       *  troubleshooting (e.g., identify misbehaving flows, etc.)

       *  recognize flow(s) for analytics (e.g, increase counters, etc.)

       *  correlate events with flows (e.g., volume above threshold,
          etc.)

       *  etc.

   Each node (DA-T-PE, DA-S-PE and P) use a local-ID of the DetNet-
   (compound)-flow in order to accomplish its role during transport.
   Recognizing the flow-ID is more relaxed for DA-T-PE and DA-S-PE
   nodes, as they are fully aware of both the DetNet service and
   transport layers.  The DetNet role of intermediate "P" nodes is
   limited to ensure congestion protection from the above listed DetNet
   functions.  However, P nodes can usually recognize only "T-label" and
   cannot consider the whole label stack for flow recognition.
   Therefore, identifying each individual DetNet flow on a P node may
   not be achieved in some network scenarios.

   On each node dealing with DetNet flows, a local-ID is assumed to
   determine what local operation a packet goes through.  Therefore,
   local-IDs MUST be unique on each DA-T-PE and DA-S-PE nodes.  Local-ID
   MUST be unambiguously bound to the Flow-ID encoded in the DetNet
   packet.

5.  DetNet data plane solution

Korhonen, et al.       Expires September 14, 2017               [Page 6]
Internet-Draft         DetNet data plane solution             March 2017

5.1.  DetNet Control Word

   The DetNet control word (d-CW) is identical to the control word
   defined for Ethernet over MPLS networks in [RFC4448].  The DetNet
   control word is illustrated in Figure 4.

      0                   1                   2                   3
      0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |0 0 0 0|  reserved - set to 0  |   16 bit Sequence Number      |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

                       Figure 4: DetNet Control Word

   [Editor's note: Shoudl we care about high speed links, here 16 bits
   of sequence number wraps fast?  For example, in a case of 100Gb/s
   link, 16 bits of sequence number will wrap in ~6.6ms assuming 1250
   octets of packets and ~3.3ms for 625 octets packets.  Both numbers
   mean quite long fiber distances, though.]

5.2.  DetNet flow identity word

   The DetNet flow identity word (flow-ID) identifies a DetNet flow
   uniquely within a DetNet network.  The flow-ID is also associated
   with the sequence number carried in the DetNet control word and used
   also for PREF purposes.

      0                   1                   2                   3
      0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |   reserved  |D|               24 bit flow identity            |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

                    Figure 5: DetNet flow identity word

   The management and assignment of the flow-IDs is outside of this
   specification.  It is assumed that each DA-T-PE node have either a
   preconfigured flow-ID number space or some dynamic control plane
   protocol is able to coordinate the allocation of the flow-IDs across
   the DA-T-PE nodes, or a central entity, e.g., a Software Defined
   Networking controller.

   The D bit defines the direction of the flow.  The value 0 means
   'east' and the value 1 means 'west'.  The D bit can be used in ring
   topologies to allow DetNet flows with the same flow-ID cross with or
   without PREF processing taking place.  Whether a DA-*-PE checks for

Korhonen, et al.       Expires September 14, 2017               [Page 7]
Internet-Draft         DetNet data plane solution             March 2017

   the D bit is based on a local policy setting.  The support for D bit
   is optional.  If a DA-*-PE does not support the D bit it MUST be
   treated as 0.

   The reserved field in the flow-ID MUST be set to zero and ignored
   when received.

   [Editor's note: we need some configuration knob defined for this
   feature.]

5.3.  DetNet encapsulation

   The DetNet data plane follows PW encapsulation.  This document
   specifies a single encapsulation that can be used over both MPLS and
   IP packet switched Networks (PSN).  The DetNet data plane
   encapsulation consists of a

   o  DetNet control word (d-CW): contains sequencing information for
      packet replication and duplicate elimination purposes.  There is a
      separate sequence number space per each DetNet label.

   o  DetNet flow-ID (f-ID): uniquely identifies a DetNet flow within a
      DetNet network.  Multiple DetNet PWs with different PW labels may
      have the same f-ID, which then implies the PWs are actually
      subflows of one compound flow.

   o  PseudoWire Label (PW Label;): a standard PW label that identifies
      a PW Instance within a (DA-)T-PE or (DA-)S-PE device.

   o  DetNet topology overlay label (L-label): an optional label used
      between (DA-)T-PE or (DA-)S-PE nodes.  The main use of L-labels is
      to tunnel PWs through a PE node and therefore effectively making a
      PE node to behave like a P node.

   In a case of MPLS-based PSN, the tunnel labels between LSRs are
   referred as T-labels.

   The DetNet CW and the Detnet flow-ID together constitute the DetNet
   PseudoWire encapsulation header.

      [Editor's note: The current design has the DetNet flow-ID as part
      of the every DetNet flow packet.  The flow-ID identifies the flow
      uniquely within the DetNet network and together with the sequence
      number information from the DetNet control word is used for PREF
      purposes.  The flow-ID makes is easy for the DA-*-PE node to
      associate different PWs into one compound flow and perform the
      elimination of duplicate packets.  The flow-ID would point at the
      node internal construct that holds the received packet history for

Korhonen, et al.       Expires September 14, 2017               [Page 8]
Internet-Draft         DetNet data plane solution             March 2017

      each DetNet flow of interest.  However, it could also be possible
      to associate multiple PWs into one DetNet flow just using the
      control plane provided information.  In this case different PWs
      (using any PW label) would be mapped internally within a node to a
      local-ID (or similar construct), which again points at the
      internal per DetNet flow received packets history construct.  The
      explicit in-band flow-ID is easy from the processing and control
      plane point of view.  The local-ID approach does not need the in-
      band information (thus has less overhead) but requires more from
      the control plane and the mapping information has to be stored
      into the LFIB.  Current design decision is the in-band flow-ID but
      may be changed to local-ID if there is a strong reason to do the
      change.]

   Figure 6 illustrates a DetNet PseudoWire encapsulation using an MPLS
   PSN.  Similarly, Figure 7 illustrates the DetNet PseudoWire
   encapsulation when IP PSN is used.  The encapsulation is uniform
   above the PSN.

   Depending on the network topology the "overlay label" (L-label) may
   be part of the label stack.  The L-label tunnels guarantee PW labels
   remain unchanged between DA-*-PE nodes.  Furthermore, L-labels
   tunnels allow selectively exposing the PW label to DA-*-PE nodes,
   which means some overlay topologies may just pass through specific
   DA-S-PEs without any DetNet specific processing.

Korhonen, et al.       Expires September 14, 2017               [Page 9]
Internet-Draft         DetNet data plane solution             March 2017

    RFC3985 Encapsulation                  DetNet PW Encapsulation

                                    +---------------------------------+
   +---------------------+          |                                 |
   |      Payload        |          |           DetNet Flow           |
   /=====================\          |         Payload  Packet         |
   H Payload Convergence H--.       |                                 |
   H---------------------H  |       /=================================\
   H       Timing        H  +-\     H         DetNet Flow ID          H
   H---------------------H  |  \--->H---------------------------------H
   H     Sequencing      H--'       H       DetNet Control Word       H
   \=====================/          \=================================/
   |  PW Demultiplexer   |--------->|            PW Label             |
   +---------------------+          +---------------------------------+
   |  PSN Convergence    |     .--->| Optional Topology overlay Label |
   +---------------------+     |    +---------------------------------+
   |         PSN         |-----+--->|         MPLS T-Label(s)         |
   +---------------------+          +---------------------------------+
   |      Data-Link      |
   +---------------------+
   |       Physical      |
   +---------------------+

    Figure 6: Encapsulation of a DetNet flow in a PW with MPLS(-TP) PSN

   When IP PSN is used, the label stack it transports is only inspected
   when the IP packet destination address equals to the IP address of a
   DA-*-PE or a P node.  Essentially there are one more IP tunnels
   between a number of DA-*-PE and/or P nodes.  The LFIB and the
   forwarding information base (FIB) combination determines whether a PW
   gets terminated at the node or forwarded to another node within a new
   IP tunnel.

Korhonen, et al.       Expires September 14, 2017              [Page 10]
Internet-Draft         DetNet data plane solution             March 2017

    RRC3985 Encapsulation                  DetNet PW Encapsulation

                                    +---------------------------------+
   +---------------------+          |                                 |
   |      Payload        |          |           DetNet Flow           |
   /=====================\          |         Payload  Packet         |
   H Payload Convergence H--.       |                                 |
   H---------------------H  |       /=================================\
   H       Timing        H  +-\     H         DetNet Flow ID          H
   H---------------------H  |  \--->H---------------------------------H
   H     Sequencing      H--'       H       DetNet Control Word       H
   \=====================/          \=================================/
   |  PW Demultiplexer   |--------->|             PW Label            |
   +---------------------+          +---------------------------------+
   |  PSN Convergence    |     .--->| Optional Topology overlay Label |
   +---------------------+     |    +---------------------------------+
   |        PSN          |-----+    |       Optional UDP Header       |
   +---------------------+     |    +---------------------------------+
   |      Data-Link      |     `--->|       IPv4 or IPv6 header       |
   +---------------------+          +---------------------------------+
   |       Physical      |
   +---------------------+

       Figure 7: Encapsulation of a DetNet flow in a PW with IP PSN

6.  PE reference model considerations

6.1.  Forwarded clarifications

   [Editor's note: The Detnet-aware "extended forwarder" does the heavy
   lifting on maintaining the sequence numbers associated with the
   DetNet labels.  Extended forwarder is also responsible for packet
   replication and duplicate elimination.  See the excerpt from RFC3985
   Section 4.2.1. about forwarder's functions.  We extend that to PREF:

      Some applications have to forward payload elements selectively
      from one or more ACs to one or more PWs.  In such cases, there
      will also be a need to perform the inverse function on PWE3-PDUs
      received by a PE from the PSN.  This is the function of the
      forwarder.

   ]

   The DetNet specific new functionality in a DA-*-PE PW processing is
   the packet replication and duplication elimination function (PREF).
   This functional is a part of the "extended" forwarder.  The PREF
   processing is triggered by the LFIB actions i.e., not all PWs receive

Korhonen, et al.       Expires September 14, 2017              [Page 11]
Internet-Draft         DetNet data plane solution             March 2017

   DetNet specific processing.  Basically the LFIB has to be extended
   with a "PREF enabled" boolean configuration switch that is associated
   with the normal label actions (e.g., swap, push, pop, ..).  The
   output of the PREF elimination function is always a single packet.
   The output of the PREF replication function is always one or more
   packet (i.e., 1:M replication).  The replicated packets MUST share
   the same DetNet PW control word sequence number and flow identity
   word flow-id.

   The complex part of the DetNet PREF processing is tracking the
   history of received packets for multiple PWs.  These PWs do not have
   the same PW label value while they still share the same PW sequence
   number counter and the history information.  That is where the DetNet
   encapsulation header flow-ID plays an important role and binds the
   control word sequence number to the flow specific shared counter and
   history information within the PREF function.

   The DetNet flow word contains a D flag bit (see Section 5.2), which
   makes the DA-*-PE node aware of the direction the flow-ID arrived
   from.  If the node, based on the local policy, checks for the D bit
   setting that effectively means the sequence number history has to
   contain also the D bit information.

   [Editor's note: draw here an example of LFIB with the elimination
   action.]

6.2.  DA-T-PE processing clarifications

   The PW-based DetNet data plane solution overloads the T-PE with a
   DetNet Edge Node function.  Such T-PE is referred as DA-T-PE and
   implies the T-PE is also aware of DetNet flows and may need to
   operate upon those.  Figure 8 illustrates the overall DA-T-PE device
   functions.  The figure shows both physical attachment circuit (AC)
   (e.g., Ethernet [RFC4448]) connecting to the PE, and a packet service
   connecting to the PE via an embedded label switching router (LSR)
   function [RFC6658].  Whether traffic flow from from a client AC and
   PSN LSP receives DetNet specific treatment is up to a local
   configuration and policy.  A DA-T-PE can also serve as a normal T-PE.

Korhonen, et al.       Expires September 14, 2017              [Page 12]
Internet-Draft         DetNet data plane solution             March 2017

                 +---------------------------------------+
                 |             DA-T-PE Device            |
                 +---------------------------------------+   Egress/
                 |             | Forwarder |             |   Ingress
                 |  LSR        |           |    Single   | PW Instance
     Client PSN  |  ("Packet   o <-X-----> o PW Instance o<---------->
     LSPs        |    NSP")    |   | Repl. |             |
     <---------->o             |   | Elim. +-------------+ Duplicate
                 |             |   :       |             |   Egress
                 |             |   .       |    Single   | PW Instance
                 |             |       +-> o PW Instance o<---------->
                 |             |       |   |             |
                 +-------------+       |   +-------------+   Egress/
                 |             |       |   |             |   Ingress
     Client AC   |    NSP      | Repl. |   |    Single   | PW Instance
     <---------->o             o <-----X-> o PW Instance o<---------->
                 |             | Elim.     |             |
                 +-------------+           +-------------+   Egress/
                 |             |           |             |   Ingress
     Client AC   |    NSP      |           |    Single   | PW Instance
     <---------->o             o <-------> o PW Instance o<---------->
                 |             |           |             |
                 +---------------------------------------+

                  Figure 8: DetNet Edge Node as a DA-T-PE

   A DA-T-PE participates to the packet replication and duplication
   elimination.  Required processing is done within an extended
   forwarder function.  In the case the native service processing (NSP)
   is IEEE 802.1CB [IEEE8021CB] capable, the packet replication and
   duplicate elimination MAY entirely be done in the NSP and bypassing
   the DetNet PW encapsulation and logic entirely, and thus is able to
   operate over unmodified PW implementation and deployment.  The NSP
   approach works only between DA-T-PEs and cannot make use of DA-S-PEs
   (see Section 6.3).

   The DetNet-aware extended forwarder selects the egress segment PW
   based on the rules described in [RFC4448] and [RFC6658].  In both
   "normal AC" and "Packet AC" cases there is no DetNet encapsulation
   header available yet as it is the case with DA-S-PEs (see
   Section 6.3).  It is the responsibility of the extended forwarder
   within the DA-T-PE to push the DetNet encapsulation header (i.e., the
   DetNet CW and the DetNet flow-ID) to the packet before forwarding it
   to the appropriate egress PW instance(s).  The extended forwarder MAY
   copy the sequencing information from the native packet into the
   DetNet CW and vice versa.  If there is no existing sequencing
   information available in the native packet or the forwarder chose not

Korhonen, et al.       Expires September 14, 2017              [Page 13]
Internet-Draft         DetNet data plane solution             March 2017

   to copy it from the native packet, then the extended forwarder MUST
   maintain a sequence number counter for each DetNet flow (indexed by
   the flow-ID).

6.3.  DA-S-PE processing clarifications

   The PW-based DetNet data plane solution overloads a S-PE with a
   DetNet Relay function.  Such S-PE device is referred as DA-S-PE and
   implies the S-PE is also aware of DetNet flows and may operate upon
   those.  Figure 9 illustrates the overall DA-S-PE device functions.

   A DA-S-PE participates to the packet replication and duplication
   elimination.  This processing is done within an extended forwarder
   function.  Whether an ingress PW receives DetNet specific processing
   depends on how the LFIB is programmed.  For some PWs the DA-S-PE can
   act as a normal S-PE and for some apply the DetNet specific
   processing.  It is also possible to treat the DA-S-PE as a P router
   using the L-label tunnels.  Again, this is entirely up to how the
   LFIB has been programmed.

   The DetNet-aware forwarder selects the egress segment PW based on the
   PW label.  The mapping of ingress PW label to egress PW label may be
   statically or dynamically configured.  Additionally the DetNet-aware
   forwarder does duplicate frame elimination based on the DetNet flow-
   ID and DetNet Control Word sequence number combination.  The packet
   replication is also done within the DetNet-aware forwarder.  During
   elimination and the replication process both DetNet CW sequence
   number and DetNet flow-ID MUST be preserved and copied to the egress
   PW.

Korhonen, et al.       Expires September 14, 2017              [Page 14]
Internet-Draft         DetNet data plane solution             March 2017

                 +---------------------------------------+
                 |             DA-S-PE Device            |
                 +---------------------------------------+
       Ingress   |             | Forwarder |             |   Egress
     PW instance |   Single    |           |    Single   | PW Instance
     ----------->o PW Instance o --X-----> o PW Instance o----------->
                 |             |   | Elim. |             |
                 +-------------+   |       +-------------+ Duplicate
       Ingress   |             |   |       |             |   Egress
     PW instance |   Single    |   |       |    Single   | PW Instance
     ----------->o PW Instance o --+   +-> o PW Instance o----------->
                 |             |       |   |             |
                 +-------------+       |   +-------------+
       Ingress   |             |       |   |             |   Egress
     PW instance |   Single    | Repl. |   |    Single   | PW Instance
     ----------->o PW Instance o ------X-> o PW Instance o----------->
                 |             |           |             |
                 +-------------+           +-------------+
       Ingress   |             |           |             |   Egress
     PW instance |   Single    |           |    Single   | PW Instance
     ----------->o PW Instance o --------> o PW Instance o----------->
                 |             |           |             |
                 +---------------------------------------+

                 Figure 9: DetNet Relay Node as a DA-S-PE

7.  Other DetNet considerations

7.1.  Class of Service

   [Editor's note: Discuss the CoS.. and how that is archived when using
   MPLS or IP PSN.]

7.2.  Quality of Service

   [Editor's note: Elaborate the QoS issues here..]

7.3.  Time synchronization

   [Editor's note: describe a bit of issues and deployment
   considerations related to time-synchronization within DetNet.  Refer
   to DT discussion and the slides that summarize different approaches
   and rough synchronization performance numbers.  Finally, scope time-
   synchronization solution outside data plane.]

   When DetNet is used, there is an underlying assumption that a clock
   synchronization method is used, such as the Precision Time Protocol

Korhonen, et al.       Expires September 14, 2017              [Page 15]
Internet-Draft         DetNet data plane solution             March 2017

   (PTP) [IEEE1588].  In this case, there are a few possible approaches
   of how synchronization protocol packets are forwarded and handled by
   the network:

   o  PTP packets are sent as a normal DetNet flow: in this approach PTP
      traffic is forwarded as a DetNet flow, and as such it is forwarded
      in a way that allows a low delay variation.  However, since
      intermediate nodes do not take part in the synchronization
      protocol, this approach provides a relatively low degree of
      accuracy.

   o  PTP with on-path support: in this approach PTP packets are sent as
      DetNet flows, and intermediate nodes take part in the protocol as
      Transparent Clocks or Boundary Clocks [IEEE1588].  The on-path PTP
      support by intermediate nodes provides a higher degree of accuracy
      than the previous approach.  The actual accuracy depends on
      whether all intermediate nodes are PTP-capable, or only a subset
      of them.

   o  Time-as-a-service: in this approach accurate time is provided as-
      a-service to the DetNet source and destination, as well as the
      intermediate nodes.  Since traffic between the source and
      destination is sent over a provider network, if the provider
      supports time-as-a-service, then accurate time can be provided to
      both the source and the destination of DetNet traffic.  This
      approach can potentially provide the highest degree of accuracy.

   It is expected that the latter approach will be the most common one,
   as it provides the highest degree of accuracy, and creates a layer
   separation between the DetNet data and the synchronization service.

   It should be noted that in all three approaches it is not recommended
   to use replication and elimination for synchronization packets; the
   replication/elimination approach may in some cases reduce the
   synchronization accuracy, since the observed path delay will be
   bivalent.

7.4.  Bidirectional traffic

   Some DetNet applications generate bidirectional traffic and may
   require symmetric flows.  There are already mechanisms that can be
   used to create bidirectional tunnels at the transport network level,
   such as MPLS-TP.  The data plane solution SHOULD allow establishing
   bidirectional symmetric flows.  Control plane mechanisms would need
   to also support this, though this is out of scope of this document.
   [Summary of existing mechanisms to create bidirectional tunnels that
   can be used.]

Korhonen, et al.       Expires September 14, 2017              [Page 16]
Internet-Draft         DetNet data plane solution             March 2017

8.  Control plane considerations

   [Editor's note: discuss here what kind of enhancements are needed for
   DetNet and specifically for PREF.]

8.1.  PW Label assignment and distribution

   The PW label distribution follows the same mechanisms specified for
   MS-PW [RFC6073].

9.  Security considerations

   The security considerations of DetNet in general are discussed in
   [I-D.ietf-detnet-architecture] and [I-D.sdt-detnet-security].  Other
   security considerations will be added in a future version of this
   draft.

10.  IANA Considerations

   TBD.

11.  Acknowledgements

   The author(s) ACK and NACK.

   The following people were part of the DetNet Data Plane Solution
   Design Team:

      Jouni Korhonen

      Janos Farkas

      Norman Finn

      Balazs Varga

      Loa Andersson

      Tal Mizrahi

      David Mozes

      Yuanlong Jiang

      Carlos J.  Bernardos

   The DetNet chairs serving during the DetNet Data Plane Solution
   Design Team:

Korhonen, et al.       Expires September 14, 2017              [Page 17]
Internet-Draft         DetNet data plane solution             March 2017

      Lou Berger

      Pat Thaler

12.  References

12.1.  Normative References

   [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
              Requirement Levels", BCP 14, RFC 2119,
              DOI 10.17487/RFC2119, March 1997,
              <http://www.rfc-editor.org/info/rfc2119>.

   [RFC3985]  Bryant, S., Ed. and P. Pate, Ed., "Pseudo Wire Emulation
              Edge-to-Edge (PWE3) Architecture", RFC 3985,
              DOI 10.17487/RFC3985, March 2005,
              <http://www.rfc-editor.org/info/rfc3985>.

   [RFC4448]  Martini, L., Ed., Rosen, E., El-Aawar, N., and G. Heron,
              "Encapsulation Methods for Transport of Ethernet over MPLS
              Networks", RFC 4448, DOI 10.17487/RFC4448, April 2006,
              <http://www.rfc-editor.org/info/rfc4448>.

   [RFC6073]  Martini, L., Metz, C., Nadeau, T., Bocci, M., and M.
              Aissaoui, "Segmented Pseudowire", RFC 6073,
              DOI 10.17487/RFC6073, January 2011,
              <http://www.rfc-editor.org/info/rfc6073>.

   [RFC6658]  Bryant, S., Ed., Martini, L., Swallow, G., and A. Malis,
              "Packet Pseudowire Encapsulation over an MPLS PSN",
              RFC 6658, DOI 10.17487/RFC6658, July 2012,
              <http://www.rfc-editor.org/info/rfc6658>.

12.2.  Informative References

   [I-D.ietf-detnet-architecture]
              Finn, N. and P. Thubert, "Deterministic Networking
              Architecture", draft-ietf-detnet-architecture-00 (work in
              progress), September 2016.

   [I-D.ietf-detnet-dp-alt]
              Korhonen, J., Farkas, J., Mirsky, G., Thubert, P.,
              Zhuangyan, Z., and L. Berger, "DetNet Data Plane Protocol
              and Solution Alternatives", draft-ietf-detnet-dp-alt-00
              (work in progress), October 2016.

Korhonen, et al.       Expires September 14, 2017              [Page 18]
Internet-Draft         DetNet data plane solution             March 2017

   [I-D.sdt-detnet-security]
              Mizrahi, T., Grossman, E., Hacker, A., Das, S.,
              "Deterministic Networking (DetNet) Security
              Considerations, draft-sdt-detnet-security, work in
              progress", 2017.

   [IEEE1588]
              IEEE, "IEEE 1588 Standard for a Precision Clock
              Synchronization Protocol for Networked Measurement and
              Control Systems Version 2", 2008.

   [IEEE8021CB]
              Finn, N., "Draft Standard for Local and metropolitan area
              networks - Seamless Redundancy", IEEE P802.1CB
              /D2.1 P802.1CB, December 2015,
              <http://www.ieee802.org/1/files/private/cb-drafts/
              d2/802-1CB-d2-1.pdf>.

Appendix A.  Example of DetNet data plane operation

   [Editor's note: Simplified example of DetNet data plane and how
   labels etc work in the case of MPLS-based PSN and utilizing PREF.
   The figure is subject to change depending on the further DT decisions
   on the label handling..]

Korhonen, et al.       Expires September 14, 2017              [Page 19]
Internet-Draft         DetNet data plane solution             March 2017

        T1      ->R1----->DA-S-PE1---->R2---->DA-S-PE2--->R3
        L1     /     L1            T2     L2           T3   \    L2
        PW1   /      PW1           L2     PW1          L2    \   PW1
       #seq  /       #seq          PW1    #seq         PW1    \  #seq
       FID1 /        FID1          #seq   FID1         #seq    \ FID1
           /                                           FID1     v
  E1-->DA-T-PE1                                            DA-T-PE2-->E3
           \                                T5     T6           ^
         T4 \        L2     L3              L4     L5          /
         L2  \       PW1    PW2             PW1    PW2        /
         PW1  \      #seq   #seq            #seq   #seq      /
         #seq  \     FID1   FID2            FID1   FID2     /
         FID1   =>R4------------->DA-S-PE2--------------->R5
      T11      /                    ^                       \    L5
      L3      /                      \    L9                 \   PW2
      PW2    /                        \   PW2                 \  #seq
      #seq  /                          \  #seq                 \ FID2
      FID2 /                            \ FID2                  v
  E2-->DA-T-PE3                         R9<-    T10        DA-T-PE4-->E4
           \                     T8         \   L9    T9        ^
         T7 \      L6            L7     L7   \  PW2   L8       / L8
         L6  \     PW2           PW2    PW2   \ #seq  PW2     /  PW2
         PW2  \    #seq          #seq   #seq   \FID2  #seq   /   #seq
         #seq  \   FID2          FID2   FID2    \     FID2  /    FID2
         FID2   ->R6---->DA-S-PE2---->R7---->DA-S-PE6---->R8

              Figure 10: Replication and elimination example

   [Editor's note: the LFIB Figure 11 content to be updated.]

Korhonen, et al.       Expires September 14, 2017              [Page 20]
Internet-Draft         DetNet data plane solution             March 2017

     +========+================+=====================================+
     |        |                |     PREF     | Forwarding Semantics |
     | Device |       In-Label |--------------|----------------------|
     |        |                |  flow-ID | D | Out-Label | Out-Link |
     +========+================+==========+===+===========+==========+
     | T-PE1  | N/A (from AC)  |          |   |           |          |
     |        |                |          |   |           |          |
     +========+================+==========+===+===========+==========+
     | T-PE2  |                |          |   |           |          |
     |        |                |          |   |           |          |
     +========+================+==========+===+===========+==========+
     | T-PE3  | N/A (from AC)  |          |   |           |          |
     |        |                |          |   |           |          |
     +========+================+==========+===+===========+==========+
     | T-PE4  |                |          |   |           |          |
     |        |                |          |   |           |          |
     +========+================+==========+===+===========+==========+
     | S-PE1  |                |          |   |           |          |
     +========+================+==========+===+===========+==========+
     | S-PE2  |                |          |   |           |          |
     +========+================+==========+===+===========+==========+
     | S-PE3  |                |          |   |           |          |
     +========+================+==========+===+===========+==========+
     | S-PE4  |                |          |   |           |          |
     +========+================+==========+===+===========+==========+
     | S-PE5  |                |          |   |           |          |
     +========+================+==========+===+===========+==========+
     | S-PE6  |                |          |   |           |          |
     +========+================+==========+===+===========+==========+
     | R1     |                |     N/A  |   |           |          |
     +========+================+==========+===+===========+==========+
     | R2     |                |     N/A  |   |           |          |
     +========+================+==========+===+===========+==========+

                         Figure 11: LFIB contents

Appendix B.  Example of pinned paths using IP PSN

Authors' Addresses

   Jouni Korhonen (editor)
   Broadcom
   3151 Zanker Road
   San Jose, CA  95134
   USA

   Email: jouni.nospam@gmail.com

Korhonen, et al.       Expires September 14, 2017              [Page 21]
Internet-Draft         DetNet data plane solution             March 2017

   Loa Andersson
   Huawei

   Email: loa@pi.nu

   Yuanlong Jiang
   Huawei

   Email: jiangyuanlong@huawei.com

   Balazs Varga
   Ericsson
   Konyves Kalman krt. 11/B
   Budapest  1097
   Hungary

   Email: balazs.a.varga@ericsson.com

   Janos Farkas
   Ericsson
   Konyves Kalman krt. 11/B
   Budapest  1097
   Hungary

   Email: janos.farkas@ericsson.com

   Carlos J. Bernardos
   Universidad Carlos III de Madrid
   Av. Universidad, 30
   Leganes, Madrid  28911
   Spain

   Phone: +34 91624 6236
   Email: cjbc@it.uc3m.es
   URI:   http://www.it.uc3m.es/cjbc/

   Tal Mizrahi
   Marvell
   6 Hamada st.
   Yokneam
   Israel

   Email: talmi@marvell.com

Korhonen, et al.       Expires September 14, 2017              [Page 22]