Keep inter-domain forwarding conformance to routing
draft-cheng-idr-conformance-forwarding-00

IDR                                                             W. Cheng
Internet-Draft                                                    S. Yue
Intended status: Standards Track                            China Mobile
Expires: 24 August 2024                                           M. Liu
                                                                M. Huang
                                                     Huawei Technologies
                                                        21 February 2024

          Keep inter-domain forwarding conformance to routing
               draft-cheng-idr-conformance-forwarding-00

Abstract

   This document introduces what the conformance forwarding is and why
   nonconformance forwarding is prevalent in the Internet, describes the
   risks of nonconformance forwarding and defines the requiremenets for
   conformance forwarding.

Status of This Memo

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF).  Note that other groups may also distribute
   working documents as Internet-Drafts.  The list of current Internet-
   Drafts is at https://datatracker.ietf.org/drafts/current/.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   This Internet-Draft will expire on 24 August 2024.

Copyright Notice

   Copyright (c) 2024 IETF Trust and the persons identified as the
   document authors.  All rights reserved.

Cheng, et al.            Expires 24 August 2024                 [Page 1]
Internet-Draft           Conformance forwarding            February 2024

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents (https://trustee.ietf.org/
   license-info) in effect on the date of publication of this document.
   Please review these documents carefully, as they describe your rights
   and restrictions with respect to this document.  Code Components
   extracted from this document must include Revised BSD License text as
   described in Section 4.e of the Trust Legal Provisions and are
   provided without warranty as described in the Revised BSD License.

Table of Contents

   1.  The conformance of inter-domain routing and forwarding  . . .   2
   2.  The risk of nonconformance inter-domain forwarding  . . . . .   4
   3.  Reasons for nonconformance inter-domain forwarding  . . . . .   4
     3.1.  Traffic Redirection . . . . . . . . . . . . . . . . . . .   5
     3.2.  Traffic engineering protocols . . . . . . . . . . . . . .   5
     3.3.  Ambiguous routing specifications  . . . . . . . . . . . .   5
   4.  Potential Use case  . . . . . . . . . . . . . . . . . . . . .   6
   5.  Requirements for conformance inter-domain forwarding  . . . .   7
     5.1.  Requirement 1: Obtaining deviation AS paths . . . . . . .   7
       5.1.1.  Obtaining redirection deviation AS paths  . . . . . .   7
       5.1.2.  Obtaining deviation AS path from other protocols  . .   8
     5.2.  Requirement 2: Advertising the deviation path . . . . . .   8
   6.  Benefits  . . . . . . . . . . . . . . . . . . . . . . . . . .   9
   7.  Management Considerations . . . . . . . . . . . . . . . . . .   9
   8.  IANA Considerations . . . . . . . . . . . . . . . . . . . . .   9
   9.  Security Considerations . . . . . . . . . . . . . . . . . . .   9
   10. Acknowledgements  . . . . . . . . . . . . . . . . . . . . . .   9
   11. Informative References  . . . . . . . . . . . . . . . . . . .  10
   Authors' Addresses  . . . . . . . . . . . . . . . . . . . . . . .  10

1.  The conformance of inter-domain routing and forwarding

   This document defines conformance forwarding as the forwarding of
   traffic exactly along the path planned by routing.

   Nonconformance forwarding means that the actual forwarding path is
   different from the path selected by other routers through routing.
   Routers route traffic depending on the routing information provided
   by routing protocols.  Many factors, such as policy-based routing or
   traffic engineering, may result in that the forwarding path may be
   altered on the data plane by any routers and not conform to the
   routing path.  In other words, the actual forwarding path is not
   verified by any consistent policies or routing algorithms.  Therefore
   nonconformance forwarding may not match the intent of the network
   user and the route, leading to low quality, security risks or even
   unreachability.

Cheng, et al.            Expires 24 August 2024                 [Page 2]
Internet-Draft           Conformance forwarding            February 2024

   Nonconformance forwarding is especially common in inter-domain
   scenarios.  The AS_PATH attribute of BGP UPDATE records the AS number
   that it has passed through.  Ideally, the traffic to the destination
   address should be forwarded along the AS number recorded in the
   AS_PATH.  For purposes of load balancing, traffic analysis, and so
   on, the actual forwarding AS path can be affected by multiple
   entities and usually different from the propagation path of BGP
   UPDATE.  In addition, the AS_PATH attribute is easy to be modified
   and may not reveal the actual propagation path of BGP UPDATE, which
   also contributes to nonconformance inter-domain forwarding.

   Nonconformance inter-domain forwarding causes the forwarding path of
   traffic in the Internet to be a black box for any AS.  Worsely, the
   inter-domain forwarding path is not validated by any consistent
   policies or routing algorithms, which may incurs many forwarding
   risks such as traffic black hole, traffic loop, traffic detour and
   crossing the malicious AS, etc.  Operators can only engineer inter-
   domain traffic based on simple consensus and best practices,
   burdening network maintenance and making it difficult to prevent
   these forwarding risks.

    +-----+       +-----+       +-----+       +-----+
    | AS1 |-------| ASx |-------| ASy |-------| AS2 |
    +-----+       +-----+       +-----+       +-----+
            BGP Path    Non-BGP Path   BGP Path

                 Figure 1: The complete forwarding AS path

   Figure 1 shows a simple case for nonconformance inter-domain
   forwarding.  The comlpete forwarding AS path from AS1 to AS2 consists
   fo BGP paths and non-BGP paths.  The non-BGP path from ASx to ASy,
   decided by non-BGP protocols, is invisible to AS1 and AS2.
   Similarly, the BGP path from ASy to AS2 is invisible to ASx and the
   BGP path from AS1 to ASx is invisible to ASy.  If the two BGP paths
   contains the same AS, a loop occurs between AS1 and AS2.  This loop
   cannot be detected or broken by any AS on the path.

   Conformance forwarding aims to empower BGP to open the forwarding
   black box in the Internet, enabling the actual forwarding path to be
   visible to the origin AS.  It is unlikely to force all traffic to be
   forwarded according to the routing decision.  However, it is
   essential to extend BGP to reveal the actual forwarding AS path or
   the non-BGP fators that affect the forwarding path.  In fact, the
   community has already proposed lots of work along these lines, such
   as BPG Flowspec [RFC8955], Add-Path [RFC7911].  The document aims to
   provide a risk analysis of nonconformance forwarding, summarize the
   reasons for nonconformance, introduce definition and potential
   scenarios of conformance forwarding.

Cheng, et al.            Expires 24 August 2024                 [Page 3]
Internet-Draft           Conformance forwarding            February 2024

2.  The risk of nonconformance inter-domain forwarding

   The nonconformance forwarding result in the origin AS, possibly all
   ASes, not seeing the complete actual AS forwarding path to the
   destination AS.  The reachability of BGP relay on simple consensus,
   e.g., valley-free principle, and best practices.  After more than 50
   years of practice, this principle is seems pretty solid.  The
   complete AS path, which is not verified by routing principles and
   filters of the control plane of any AS, still has many problems.

   Specifically, the nonconformance inter-domain forwarding has the
   following risks:

   *  Traffic blackhole: there is no corresponding route for the next-
      hop AS, resulting in a traffic black hole.  For example, an AS
      that violates the valley-free principle redirects traffic that
      should be forwarded to the provider AS to another unrelated
      customer AS.  Both BGP Flowspec and PBR support redirection of
      traffic on the data plane, which is invisible to the control
      plane.

   *  Loop: the complete AS path that has not been checked by secure
      routing principles of any AS may lead to loops, e.g., the AS path
      before redirection and the AS path after redirection may contain
      the same AS.

   *  Detour: the complete forwarding AS path is composed of multiple AS
      paths from different protocol, which may lead to unnecessary
      lengthening of AS path.

   *  Malicious AS: the actual forwarding path is not visible to the
      origin AS, which may cause its traffic to pass through some
      insecure ASes it does not expect.

   *  Non-optimal route: the AS may select the optimal route based on
      some preferred ASes.  However, the actual forwarding AS path may
      not contain the AS it expectes.

3.  Reasons for nonconformance inter-domain forwarding

   This document focuses on nonconformance forwarding resulting from
   control-plane and data-plane disagreement.  The inter-domain
   forwarding that does conform to routing may be caused by traffic
   redirection, traffic engineering protocols, and ambiguous routing
   specifications, etc.

Cheng, et al.            Expires 24 August 2024                 [Page 4]
Internet-Draft           Conformance forwarding            February 2024

3.1.  Traffic Redirection

   Due to load-balancing, anti-DDoS, etc., policy-based routing (PBR) or
   BGP Flowspec may be configured to redirect traffic on the data plane
   to a new next-hop AS which is different with the next-hop AS
   determined by AS_PATH in BGP UPDATE.  In addition, ECMP optionally
   ignores comparing AS_PATH of different routes, resulting in load-
   sharing of traffic across multiple different next-hop ASes.  However,
   BGP usually only advertises the optimal route outwardly.

   The forwarding AS path of the traffic after the above redirection is
   determined by the next-hop AS that not in the AS_PATH attribute,
   which is unknown to the origin AS and the upstream AS.  The complete
   forwarding AS path consists of two parts: the AS path before
   redirection and the AS path after redirection.  This path is not
   verified by any AS and may violate the valley-free principle or even
   contain duplicate ASes, causing traffic blackholes or loops.

3.2.  Traffic engineering protocols

   Due to load-balancing, network optimization, etc., operators may
   utilize other protocols such as MPLS，SR to locally steer traffic to
   the specified AS path which is different from the AS_PATH in BGP
   UPDATE.

   The complete forwarding AS path is determined by BGP and TE
   protocols.  It may include multiple segments, for example, the first
   segment is an AS path determined by BGP that starts with the origin
   AS, the second segment is a TE path and the last segment is still the
   AS path determined by BGP that ends with the destination AS.

   The complete forwarding path is actually transparent to the origin AS
   and is not verified by its filters, which may incur similar risks as
   redirection.

3.3.  Ambiguous routing specifications

   Ambiguous routing specifications, such as route aggregation,
   convergence and redistribution, may also contribute to the
   nonconformance between inter-domain routing and forwarding.

   To minimize routing tables, route aggregation is widely used in IP
   networks.  BGP route aggregation causes the ordered AS_SEQUENCE to be
   converted to the unordered AS_SET.  AS_SEQUENCE and AS_SET which are
   different types of AS_PATH.  AS_SET does not indicate the forwarding
   AS path of traffic, which can lead to the nonconformance between
   inter-domain routing and forwarding.

Cheng, et al.            Expires 24 August 2024                 [Page 5]
Internet-Draft           Conformance forwarding            February 2024

   Redistributing BGP routes into IGP can result in the loss of AS_PATH.
   Before advertised to the other AS, the route should be redistributed
   from IGP into BGP.  The other AS that receives the route by BGP
   considers the destination AS of the route to be the redistribution
   AS.  In fact, the complete AS path that the origin AS wishes to
   obtain is actually the AS path from it to the destination AS.
   Fortunately, this reditribution is too dangerous to be practiced.

4.  Potential Use case

   The purpose of comformance forwarding is to enhance the ability of
   routing protocols to keep data-plane and control-plane alignment by
   ensuring that all forwarding behavior is controlled by the routing
   protocol or reflected in routing announcements.

   The primary goal of the routing protocol is to provide connectivity.
   With the emergence of hyperscale networks and diverse applications,
   routing protocols are given the ability to advertise more routing or
   forwarding information to reduce the burden on network management and
   provide better network quality.  Existing routing protocols,
   expecially BGP, already support the advertisemnet of almost all
   routing and forwarding information.

   There is a lack of frameworks and requirements to gudie BGP to
   integrate all the information and open the black box of forwarding on
   the control plane.  There are some scenarios where conformance inter-
   domain forwarding may be required:

   *  Network Visualization

   Visualization has been an enduring topic since the birth of the
   Internet.  The forwarding behavior defined by the data plane makes
   visualization and predictability of the forwarding path even more
   challenging.  Conformance forwarding allows the actual forwarding
   path of packets to be predicted based on route announcements.

   Current developments in BGP, such as BGP flowspec and extensions for
   TE, have demonstrated that route announcements can carry forwarding
   information that directly affects the path of packets on the data
   plane.

   *  Intent-based routing

Cheng, et al.            Expires 24 August 2024                 [Page 6]
Internet-Draft           Conformance forwarding            February 2024

   Intent-based routing provides flexible, scalable, and reliable end-
   to-end connectivity for multiple services accross the network domains
   by carrying multiple supported intent in routing protocols.  Intent
   reflects the demand of application for transmission quality.  Intent
   routing presupposes conformance forwarding.  If inter-domain
   forwarding does not follow the routing, the intent may be not
   satisfied.

   *  source address validation

   Source address validation based on RIB or FIB suffers from
   unavoidable improper pass or improper filter problems.  Accurate
   source address validation relies on the actual forwarding path of the
   packet.  However, nonconformance forwarding makes the high overhead
   of the discovery of the path unacceptable.  Conformance forwarding
   can reduce the overhead, which contributes to TE and route security.

5.  Requirements for conformance inter-domain forwarding

   All use cases require the inter-domain routing protocol, i.e., BGP,
   to learn deviating paths that are determined by non-BGP factors.
   Therefore, to get the complete actual forwarding AS path by BGP,
   there are two requirements:

   *  Obtaining deviation AS paths that results from local non-BGP
      factors.

   *  Advertising the deviation AS path and the steered flow by BGP

5.1.  Requirement 1: Obtaining deviation AS paths

   Rotuers need to understand how non-BGP factors affect forwarding AS
   paths to learn deviating paths, which is vital to network
   visualization.

5.1.1.  Obtaining redirection deviation AS paths

   A router redirecting traffic to a new next-hop AS generates a
   deviation AS path.  In order to get the direction deviation AS path,
   the router first acquires the next-hop AS and the destination prefix
   of the redirection rule.  The router then looks up the AS_PATH in
   Adj_RIBs_In from the next-hop AS by the destination prefix.  The
   AS_PATH is the forwarding AS path after redirection, i.e., the
   redirection deviation AS path.

Cheng, et al.            Expires 24 August 2024                 [Page 7]
Internet-Draft           Conformance forwarding            February 2024

5.1.2.  Obtaining deviation AS path from other protocols

   Some TE protocols that may direct the specific flow into pre-planned
   AS paths.  Their destination prefix of the steered traffic is
   obtained in a similar way as in Section 5.1.1.

   In the egress router of the TE path, the Adj-RIBs-In is then looked
   up by the destination prefix to obtain the subsequent AS path.  The
   complete forwarding path, therefore, is stitched together from the TE
   Path and other AS Paths controlled by BGP.

5.2.  Requirement 2: Advertising the deviation path

   Advertising the devation path benefits other ASes not only in terms
   of network visualization but also in terms of optimal routing.

   The AS that generates the deviation AS path is obliged to advertise
   it to other ASes.  For example, the AS that is configured with the
   redirection rule advertises the redirection deviation path to the
   origin AS.  The origin AS then combines the AS path to the
   redirection AS and the redirection deviation AS path to get the
   complete forwarding AS path which is verified to be optimal or secure
   using the local AS_PATH filter.  The path that passes through
   malicious ASes will not be selected by the origin AS.

   However, only the flow that matches the redirection rule will go
   through the deviation path, and the other missed flow is still
   forwarded along the original path planned by BGP.  Therefore, the
   redirection AS should advertise the deviation path and the flow
   specification to other ASes.

                                                      +-----+
                                                    / | AS3 | \
                                                   /  +-----+  \
                                                  /             \
                                                 /               \
    +-----+               +-----+            +-----+            +-----+
    | ASx |---------------| AS1 |------------| AS2 |------------| AS4 |
    +-----+ <-----------  +-----+ <--------  +-----+ <--------  +-----+
         BGP UPDATE:               BGP UPDATE:           BGP UPDATE:
          AS_PATH:[AS1,AS2,AS4]     AS_PATH:[AS2,AS4]     AS_PATH:[AS4]
          D_PATH:[AS1,AS2,AS3,AS4]  D_PATH:[AS2,AS3,AS4]

                  Figure 2: Advertising the deviation path

   Figure 2 shows an example of advertising the devation path.  As shown
   in the figure, AS4 advertises a BGP UPDATE to ASx along the AS path
   ([AS1,AS2,AS4]).  AS2 is a redirection AS which redirect the packet

Cheng, et al.            Expires 24 August 2024                 [Page 8]
Internet-Draft           Conformance forwarding            February 2024

   routing to AS4 to AS3.  As a result, the actual forwarding path of
   packets from AS1 to AS2 is [AS1, AS2, AS3, AS4].  So AS2 should add
   D_PATH to BGP UPDATE.  D_PATH refers to the actual forwarding AS path
   through which the packet sent to AS4.

   The router generating the deviation AS path needs to advertise the
   non-BGP factors or the deviation AS path to other routers by the
   inter-domain routing protocol, i.e., BGP.  In other words,
   conformance inter-domain forwarding requires that BGP announcements
   contain all possible forwarding behaviors.  The routing algorithm
   integrates all possible forwarding behaviors and paths to select the
   optimal forwarding path.  In short, routing announcements are subject
   to forwarding, but routing dictates the ultimate forwarding path.

6.  Benefits

   Since the birth of the Internet, inter-AS TE has been a very complex
   and difficult task.  One of the major reasons is that inter-domain
   routing and forwarding are nonconformance The AS_PATH attrtibute in
   BGP UPDATE only represents the propagation path of the route, which
   may not correspond to the actual forwarding path.

   The community has designed many complex protocols to plan the
   forwarding path and guarantee SLA, while routing protocols are
   usually utilized to get the basic reachability.  If inter-domain
   forwarding conforms to inter-domain routing announcements, TE and
   routing security will be significantly simplified.  Moreover, the
   ability of the origin AS to plan the forwarding AS path of its own
   path opens the black box of the Internet.  Problems that have plagued
   the Internet for decades, such as visualization and troubleshooting,
   may be solved.

7.  Management Considerations

   TBD

8.  IANA Considerations

   TBD.

9.  Security Considerations

   This document does not introduce any new security considerations.

10.  Acknowledgements

   TBD

Cheng, et al.            Expires 24 August 2024                 [Page 9]
Internet-Draft           Conformance forwarding            February 2024

11.  Informative References

   [RFC8955]  Loibl, C., Hares, S., Raszuk, R., McPherson, D., and M.
              Bacher, "Dissemination of Flow Specification Rules",
              RFC 8955, DOI 10.17487/RFC8955, December 2020,
              <https://www.rfc-editor.org/info/rfc8955>.

   [RFC7911]  Walton, D., Retana, A., Chen, E., and J. Scudder,
              "Advertisement of Multiple Paths in BGP", RFC 7911,
              DOI 10.17487/RFC7911, July 2016,
              <https://www.rfc-editor.org/info/rfc7911>.

Authors' Addresses

   Weiqiang Cheng
   China Mobile
   Beijing
   China
   Email: chengweiqiang@chinamobile.com

   Shengnan Yue
   China Mobile
   Beijing
   China
   Email: yueshengnan@chinamobile.com

   Mingxing Liu
   Huawei Technologies
   Beijing
   China
   Email: liumingxing7@huawei.com

   Mingqing Huang
   Huawei Technologies
   Beijing
   China
   Email: huangmingqing@huawei.com

Cheng, et al.            Expires 24 August 2024                [Page 10]