L2VPNs                                                          K. Patel
Internet-Draft                                                    Arrcus
Intended status: Standards Track                              A. Sajassi
Expires: September 12, 2017                                Cisco Systems
                                                                J. Drake
                                                                Z. Zhang
                                                  Juniper Networks, Inc.
                                                           W. Henderickx
                                                                   Nokia
                                                          March 12, 2017


                   Virtual Hub-and-Spoke in BGP EVPNs
                   draft-keyupate-bess-evpn-virtual-hub-00

Abstract

   Ethernet Virtual Private Network (EVPN) solution is becoming
   pervasive for Network Virtualization Overlay (NVO) services in data
   center (DC) applications and as the next generation virtual private
   LAN services in service provider (SP) applications.

   The use of host IP default route and host unknown MAC route within a
   DC is well understood in order to ensure that leaf nodes within a DC
   only learn and store host MAC and IP addresses for that DC.  All
   other host MAC and IP addresses from remote DCs are learned and
   stored in DC GW nodes thus alleviating leaf nodes from learning host
   MAC and IP addresses from the remote DCs.

   This draft further optimizes the MAC and IP address learning at the
   leaf nodes such that a leaf node within a DC only needs to learn and
   store MAC and IP addresses associated with the sites directly
   connected to it.  A leaf node does not need to learn and store MAC
   and IP addresses from any other leaf nodes thus reducing the number
   of learned MACs and IP addresses per EVI substantially.

   The modifications provided by this draft updates and extends RFC7024
   for BGP EVPN Address Family.

Status of This Memo

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF).  Note that other groups may also distribute
   working documents as Internet-Drafts.  The list of current Internet-
   Drafts is at http://datatracker.ietf.org/drafts/current/.



Patel, et al.           Expires September 8, 2017               [Page 1]


Internet-Draft                                                March 2017


   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   This Internet-Draft will expire on September 8, 2017.

Copyright Notice

   Copyright (c) 2017 IETF Trust and the persons identified as the
   document authors.  All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents
   (http://trustee.ietf.org/license-info) in effect on the date of
   publication of this document.  Please review these documents
   carefully, as they describe your rights and restrictions with respect
   to this document.  Code Components extracted from this document must
   include Simplified BSD License text as described in Section 4.e of
   the Trust Legal Provisions and are provided without warranty as
   described in the Simplified BSD License.

Table of Contents

   1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   3
   2.  Requirements Language . . . . . . . . . . . . . . . . . . . .   4
   3.  Terminology . . . . . . . . . . . . . . . . . . . . . . . . .   4
   4.  Routing Information Exchange for EVPN routes  . . . . . . . .   5
   5.  EVPN unknown MAC Route  . . . . . . . . . . . . . . . . . . .   5
     5.1.  Originating EVPN Unknown MAC Route by a V-Hub . . . . . .   5
     5.2.  Processing VPN-MAC EVPN unknown Route by a V-SPOKE  . . .   5
     5.3.  Aliasing  . . . . . . . . . . . . . . . . . . . . . . . .   6
     5.4.  Split-Horizon And Mass Withdraw . . . . . . . . . . . . .   7
   6.  Forwarding Considerations . . . . . . . . . . . . . . . . . .   7
     6.1.  IP-only Forwarding  . . . . . . . . . . . . . . . . . . .   7
     6.2.  MAC-only Forwarding - Bridging  . . . . . . . . . . . . .   7
     6.3.  MAC and IP Forwarding - IRB . . . . . . . . . . . . . . .   8
   7.  Handling of the Broadcast and Multicast traffic . . . . . . .   8
     7.1.  Split Horizon . . . . . . . . . . . . . . . . . . . . . .   9
     7.2.  Route Advertisement . . . . . . . . . . . . . . . . . . .   9
     7.3.  Designated Forwarder in a Cluster . . . . . . . . . . . .  10
     7.4.  Traffic Forwarding Rules  . . . . . . . . . . . . . . . .  10
       7.4.1.  Traffic from Local ACs  . . . . . . . . . . . . . . .  10
       7.4.2.  Traffic Received by a V-hub from Another PE . . . . .  11
       7.4.3.  Traffic received by a V-spoke from a V-hub  . . . . .  11
     7.5.  Multi-homing support  . . . . . . . . . . . . . . . . . .  11
     7.6.  Direct V-spoke to V-spoke traffic . . . . . . . . . . . .  12
   8.  ARP/ND Suppression  . . . . . . . . . . . . . . . . . . . . .  12



Patel, et al.           Expires September 8, 2017               [Page 2]


Internet-Draft                                                March 2017


   9.  IANA Considerations . . . . . . . . . . . . . . . . . . . . .  13
   10. Security Considerations . . . . . . . . . . . . . . . . . . .  13
   11. Acknowledgements  . . . . . . . . . . . . . . . . . . . . . .  13
   12. Change Log  . . . . . . . . . . . . . . . . . . . . . . . . .  13
   13. References  . . . . . . . . . . . . . . . . . . . . . . . . .  13
     13.1.  Normative References . . . . . . . . . . . . . . . . . .  13
     13.2.  Informative References . . . . . . . . . . . . . . . . .  15
   Authors' Addresses  . . . . . . . . . . . . . . . . . . . . . . .  15

1.  Introduction

   Ethernet Virtual Private Network (EVPN) solution is becoming
   pervasive for Network Virtualization Overlay (NVO) services in data
   center (DC) applications and as the next generation virtual private
   LAN services in service provider (SP) applications.

   With EVPN, providing any-to-any connectivity among sites of a given
   EVPN Instance (EVI) would require each Provider Edge (PE) router
   connected to one or more of these sites to hold all the host MAC and
   IP addresses for that EVI.  The use of host IP default route and host
   unknown MAC route within a DC is well understood in order to
   alleviate the learning of host MAC and IP addresses to only leaf
   nodes (PEs) within that DC.  All other host MAC and IP addresses from
   remote DCs are learned and stored in DC GW nodes thus alleviating
   leaf nodes from learning host MAC and IP addresses from the remote
   DCs.

   This draft further optimizes the MAC and IP address learning at the
   leaf nodes such that a leaf node within a DC only needs to learn and
   store MAC and IP addresses associated with the sites directly
   connected to it.  A leaf node does not need to learn and store MAC
   and IP addresses from any other leaf nodes thus reducing the number
   of learned MACs and IP addresses per EVI substantially.

   [RFC7024] provides rules for Hub and Spoke VPNs for BGP L3VPNs.  This
   draft updates and extends [RFC7024] for BGP EVPN Address Family.
   This draft provides rules for Originating and Processing of the EVPN
   host unknown MAC route and host default IP route by EVPN Virtual Hub
   (V-HUB).  This draft also provides rules for the handling of the BUM
   traffic in Hub and Spoke EVPNs and handling of ARP suppression.

   The leaf nodes and DC GW nodes in a data center are referred to as
   Virtual Spokes (V-spokes) and Virtual Hubs (V-hubs) respectively.  A
   set of V-spoke can be associated with one or more V-hubs.  If a V-
   spokes is associated with more than one V-hubs, then it can load
   balanced traffic among these V-hubs.  Different V-spokes can be
   associated with different sets of V-hubs such that at one extreme
   each V-spoke can have a different V-hub set although this may not be



Patel, et al.           Expires September 8, 2017               [Page 3]


Internet-Draft                                                March 2017


   desirable and a more typical scenario may be to associate a set of V-
   spokes to a set of V-hubs - e.g., topology for a DC POD where a set
   of V-spokes are associated with a set of spine nodes or DC GW nodes.

   In order to avoid repeating many of the materials covered in
   [RFC7024], this draft is written as a delta document with its
   sections organized to follow those of that RFC with only delta
   description pertinent to EVPN operation in each section.  Therefore,
   it is assumed that the readers are very familiar with [RFC7024] and
   EVPN.

2.  Requirements Language

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" are to
   be interpreted as described in [RFC2119] only when they appear in all
   upper case.  They may also appear in lower or mixed case as English
   words, without any normative meaning.

3.  Terminology

         ARP: Address Resolution Protocol
         BEB: Backbone Edge Bridge
         B-MAC: Backbone MAC Address
         CE: Customer Edge
         C-MAC: Customer/Client MAC Address
         ES: Ethernet Segment
         ESI: Ethernet Segment Identifier
         IRB: Integrated Routing and Bridging
         LSP: Label Switched Path
         MP2MP: Multipoint to Multipoint
         MP2P: Multipoint to Point
         ND: Neighbor Discovery
         NA: Neighbor Advertisement
         P2MP: Point to Multipoint
         P2P: Point to Point
         PE: Provider Edge
         EVPN: Ethernet VPN
         EVI: EVPN Instance
         RT: Route Target

   Single-Active Redundancy Mode: When only a single PE, among a group
   of PEs attached to an Ethernet segment, is allowed to forward traffic
   to/from that Ethernet Segment, then the Ethernet segment is defined
   to be operating in Single-Active redundancy mode.

   All-Active Redundancy Mode: When all PEs attached to an Ethernet
   segment are allowed to forward traffic to/from that Ethernet Segment,



Patel, et al.           Expires September 8, 2017               [Page 4]


Internet-Draft                                                March 2017


   then the Ethernet segment is defined to be operating in All-Active
   redundancy mode.

4.  Routing Information Exchange for EVPN routes

   [RFC7024] defines multiple Route Types NLRI along with procedures for
   advertisements and processing of these routes.  Some of these
   procedures are impacted as the result of hub-and-spoke architecture.
   The routing information exchange among the hub, spoke, and vanilla
   PEs are subject to the same rules as described in section 3 of
   [RFC7024].  Furthermore, if there are any changes to the EVPN route
   advisements and processing from advertisements and processing from
   [RFC7024], they are described below.

5.  EVPN unknown MAC Route

   Section 3 of [RFC7024] talks about how a V-hub of a given VPN must
   export a VPN-IP default route for that VPN and this route must be
   exported to only the V-spokes of that VPN associated with that V-hub.
   [I-D.EVPN-overlay] defines the notion of the unknown MAC route for an
   EVI which is analogous to a VPN-IP default route for a VPN.  This
   unknown MAC route is exported by a V-hub to its associated V-spokes.
   If multiple V-hubs are associated with a set of V-spokes, then each
   V- hub advertises it with a distinct RD when originating this route.
   If a V-spoke imports several of these unknown MAC routes and they all
   have the same preference, then traffic from the V-spoke to other
   sites of that EVI would be load balanced among the V-hubs.

5.1.  Originating EVPN Unknown MAC Route by a V-Hub

   Section 7.3 of the [RFC7024] defines procedures for originating a
   VPN-IP default route for a VPN.  The same procuedures apply when a
   V-hub wants to originate EVPN unknown MAC route for a given EVI.  The
   V-hub MUST announce unknown MAC route using the MAC/IP advertisement
   route along with the Default Gateway extended community as defined in
   section 10.1 of the [RFC7432].

5.2.  Processing VPN-MAC EVPN unknown Route by a V-SPOKE

   Within a given EVPN, a V-spoke MUST import all the unknown MAC routes
   unless the route-target mismatch happens.  The processing of the
   received VPN-MAC EVPN default route follows the rules explained in
   the section 3 of the [RFC7024].  The unknown MAC route MUST be
   installed according to the rules of MAC/IP Advertisement route
   installation rules in section 9.2.2 of [RFC7024].

   In absense of any more specific VPN-MAC EVPN routes, V-spokes
   installing the unknown MAC route MUST use the route when performing



Patel, et al.           Expires September 8, 2017               [Page 5]


Internet-Draft                                                March 2017


   ARP proxy.  This behavior would allow V-Spokes to forward the traffic
   towards V-Hub.

5.3.  Aliasing

   [RFC7432] describes the concept and procedures for Aliasing where a
   station is multi-homed to multiple PEs operating in an All-Active
   redundancy mode, it is possible that only a single PE learns a set of
   MAC addresses associated with traffic transmitted by the station.
   [RFC7432] describes the concepts and procedures for Aliasing, which
   occurs when a CE is multi-homed to multiple PE nodes, operating in
   all-active redundancy mode, but not all of the PEs learn the CE's set
   of MAC addresses.  This leads to a situation where remote PEs receive
   MAC advertisement routes, for these addresses, from a single NVE even
   though multiple NVEs are connected to the multi-homed station.  As a
   result, the remote NVEs are not able to effectively load-balance
   traffic among the NVEs connected to the multi-homed Ethernet segment.

   To alleviate this issue, EVPN introduces the concept of Aliasing.
   This refers to the ability of a PE to signal that it has reachability
   to a given locally attached Ethernet segment, even when it has learnt
   no MAC addresses from that segment.  The Ethernet A-D per-EVI route
   is used to that end.  Remote PEs which receive MAC advertisement
   routes with non-zero ESI SHOULD consider the MAC address as reachable
   via all NVEs that advertise reachability to the relevant Segment
   using Ethernet A-D routes with the same ESI and with the Single-
   Active flag reset.

   This procedure is impacted for virtual hub-and-spoke topology because
   a given V-spoke does not receive any MAC/IP advertisements from
   remote V-spokes; therefore, there is no point in propagating Ethernet
   A-D per-EVI route to the remote V-spokes.  In this solution, the V-
   hubs terminate the Ethernet A-D per-EVI route (used for Aliasing) and
   follows the procedures described in [RFC7432] for handling this
   route.

   There are scenarios for which it is desirable to establish direct
   communication path between a pair of V-spokes for a given host MAC
   address.  In such scenario, the advertising V-spoke advertises both
   the MAC/IP route and Ethernet A-D per-EVI route with the RT of V-hub
   (RT-VH) per section 3 of [RFC7024].  The use of RT-VH, ensures that
   these routes are received by the V-spokes associated with that V-hub
   set and thus enables the V-spokes to perform the Aliasing procedure.

   In summary, PE devices (V-hubs in general and V-spokes occasionally)
   that receive EVPN MAC/IP route advertisements (associated with a
   multi-homed site) need to also receive the associated Ethernet A-D




Patel, et al.           Expires September 8, 2017               [Page 6]


Internet-Draft                                                March 2017


   per-EVI route advertisement(s) in order for them to perform Aliasing
   procedure.

5.4.  Split-Horizon And Mass Withdraw

   [RFC7432] uses Ethernet A-D per-ES route to a) signal to remote PEs
   the multi-homing redundancy type (Single-Active versus All-Active),
   b) advertise ESI label for split-horizon filtering when MPLS
   encapsulation is used, and c) advertise mass-withdraw when a failure
   of an access interface impacts many MAC addresses.  This route does
   not need to be advertise from a V-spoke to any remote V-spoke unless
   a direct communication path between a pair of spoke is needed for a
   given flow.

   Even if communication between a pair of V-spoke is needed for just a
   single flow, the Ethernet A-D per ES route needs to be advertised
   from the originating V-spoke for that ES which may handle tens or
   hundreds of thousands of flows.  This is because in order to perform
   Aliasing function for a given flow, the Ethernet A-D per-EVI route is
   needed and this route itself is dependent on the Ethernet A-D per-ES
   route.  In such scenario, the advertising V-spoke advertises the
   Ethernet A-D per-ES route with the RT of V-hub (RT-VH) per section 3
   of [RFC7024].

   In summary, PE devices (V-hubs in general and V-spokes occasionally)
   that receive EVPN MAC/IP route advertisements (associated with a
   multi-homed site) need to also receive the associated Ethernet A-D
   per-ES route advertisement(s).

6.  Forwarding Considerations

6.1.  IP-only Forwarding

   When EVPN operates in IP-only forwarding mode using EVPN Route Type
   5, then all forwarding considerations in section 4 of [RFC7024] are
   directly applicable here.

6.2.  MAC-only Forwarding - Bridging

   When EVPN operates in MAC-only forwarding mode (i.e., bridging mode),
   then for a given EVI, the MPLS label that a V-hub advertises with an
   Unknown MAC address MUST be the label that identifies the MAC-VRF of
   the V-hub in absense of a more specific MAC route.  When the V-hub
   receives a packet with such label, the V- hub pops the label and
   determines further disposition of the packet based on the lookup in
   the MAC-VRF.  Otherwise, the MPLS label of the matching more specific
   route is used and packet is is forwarded towards the associated
   NEXTHOP of the more specific route.



Patel, et al.           Expires September 8, 2017               [Page 7]


Internet-Draft                                                March 2017


6.3.  MAC and IP Forwarding - IRB

   When a EVPN speaker operates in IRB mode, it implements both the
   a€œIP and MAC forwarding Modesa€ (aka Integrated
   Routing and Bridging - IRB).  On a packet by packet basis, the
   V-spoke decides whether to do forwarding based on a MAC address
   lookup (bridge) or based on a IP address lookup (route).  If the host
   destination MAC address is that of the IRB interface (i.e., if the
   traffic is inter-subnet), then the V-spoke performs an additional IP
   lookup in the IP-VRF.  However, if the host destination MAC address
   is that of an actual host MAC address (i.e., the traffic is intra-
   subnet) , then the V-spoke only performs a MAC lookup in the MAC-VRF.
   The procedure specified in Section 6.1 and Section 6.2 are applicable
   to inter-subnet and intra-subnet forwarding respectively.  For intra-
   subnet traffic, if the MAC address is not found in the MAC-VRF, then
   the V-spoke forwards the traffic to the V-hub with the MPLS label
   received from the V-hub for the unknown MAC address.  For the Inter-
   subnet traffic, if the IP prefix is not found in the IP-VRF, then the
   V-spoke forwards the traffic to the V-hub with the MPLS label
   received from the V-hub for the default IP address.

7.  Handling of the Broadcast and Multicast traffic

   Just like that V-spoke to V-spoke known unicast traffic is relayed by
   V-hubs, V-spoke to V-spoke BUM traffic can also relayed by V-hubs.
   This is especially desired if Ingress Replication (IR) would be used
   otherwise for V-spokes to send traffic to other V-spokes.  This way,
   a V-spoke can unicast BUM traffic to a single V-hub, who will then
   relay the traffic.  This achieves Assisted Replication, and reduces
   multicast state in the core.  Note that a V-hub may relay traffic
   using MPLS P2MP tunnels or BIER as well as IR.  While a V-spoke may
   use P2MP tunnels or BIER to send traffic to V-hubs, this
   specification focuses on using IR by V-spokes.

   In this particular section, all traffic refers to BUM traffic unless
   explicitly stated otherwise.  The term PE refers to a V-hub or
   V-spoke when there is no need to distinguish the two.

   Consider the following topology, where V-spokes VS1/2/3 are
   associated with V-hubs VH1/2 in one cluster, and V-spokes VS4/5/6 are
   associated with V-hubs VH3/4 in another cluster.  Note that the
   lines/dots in the diagram indcate association, not connection.

                  VH1 ...VH2          VH3 ...VH4
                 / |.\ . .           / |.\ . .
                / .| .\  .          / .| .\  .
               /.  |.  \ .         /.  |.  \ .
             VS1  VS2  VS3       VS4  VS5  VS6



Patel, et al.           Expires September 8, 2017               [Page 8]


Internet-Draft                                                March 2017


7.1.  Split Horizon

   When VH1 relays traffic that it receives from VS1, in case of IR it
   MUST not send traffic back to VS1, and in case of P2MP tunnel it must
   indicate that traffic is sourced from VS1 so that VS1 will discard
   the traffic.  In case of IR with IP unicsat tunnels, the outer source
   IP address identifies the sending PE.  In case of IR with MPLS
   unicast tunnels, VH1 must advertise different labels to different
   PEs, so that it can identify the sending PE based on the label in the
   traffic from a V-spoke.

   If MPLS P2MP/multicast tunnels (including VXLAN-GPE and MPLS-over-
   GRE/UDP) are used by a V-hub to relay traffic, an upstream allocated
   (by the V-hub) label MUST be imposed in the label stack to identify
   the source of the V-spoke.  The label is advertised as part of the PE
   Distinguisher (PED) Label Attribute of the Inclusive Multicast
   Ethernet Tag (IMET) route from the V-hub, as specified in Section 8
   of [RFC 6514].

   Notice that an "upstream-assigned" label used by a V-hub to send
   traffic with on a P2MP tunnel to identify the source V-spoke is the
   same "downstream-assigned" label used by the V-hub to receive traffic
   on the IR tunnel from the V-spoke.  Therefore, the same PED Label
   attribute serves two purposes.  With [RFC 6514], a PED label may only
   identify a PE but not a particular VPN.  Here the PED label
   identifies both the PE and a particular EVI/BD.  A V-spoke programs
   its context MPLS forwarding table for the V-hub to discard any
   traffic with the PED label that the V-hub advertised for this
   V-spoke, or pop other PED labels and direct traffic into a
   corresponding EVI for L2 forwarding.

   Note that a V-hub cannot use VXLAN/NVGRE multicast tunnels to relay
   traffic because if the V-hub uses the source V-spoke's IP address in
   the outer IP header (for the purpose of identifying the source
   V-spoke), multicast RPF would fail and the packets will be discarded.

7.2.  Route Advertisement

   As with other route types, IMET routes from V-hubs are advertised
   with RT-VH and RT-EVI so they are imported by associated V-spokes and
   all V-hubs.  They carry the PED Label attribute as described above.

   IMET routes from V-spokes are advertised with RT-EVI so they are
   imported by all V-hubs.  They also carry PED Label attribute for
   multi-homing split horizon purpose if and only if V-hubs uses IR to
   relay traffic.





Patel, et al.           Expires September 8, 2017               [Page 9]


Internet-Draft                                                March 2017


   If a V-hub uses RSVP-TE P2MP tunnel, IR, or BIER to send or relay
   traffic, all other PEs (V-hubs or V-spokes) will receive traffic
   directly because the V-hub sees all PEs.  If a V-hub uses mLDP P2MP
   tunnel to send or relay traffic, only its associated V-spokes and all
   V-hubs will see the V-hub's IMET route and join the tunnel announced
   in the route.  Another V-hub need to relay traffic to its associated
   V-spokes that are not associated with this V-hub.

   For that V-hub to announce the mLDP relay tunnel in its cluster, it
   needs to advertise a (*,*) S-PMSI AD route, as specified in [draft-
   zzhang-bess-evpn-bum-procedure-updates].  The route is advertised
   with the RT-VH for that cluster, and associated V-spokes will join
   the tunnel announced in the S-SPMI AD route.

7.3.  Designated Forwarder in a Cluster

   When there are multiple V-hubs in a cluster, a V-spoke in that
   cluster decides by itself to which V-hub to send traffic.  If the
   receiving V-hub uses mLDP tunnel to relay traffic, V-hubs in other
   clusters need to further relay traffic, but only one V-hub in each
   cluster can do so.  As a result, a DF must be elected among the
   V-hubs for each cluster.

   The election is similar to DF election in RFC 7432, with the
   folllowing differences.

   o  Instead of using Ethernet Segment route to discover the PEs on a
      multi-homing ES, the IMET route are used to determine the V-hubs
      in the same cluster - they all carry the same pair of RT-EVI and
      RT-VH, and advertises the unknown mac route.

   o  Instead of using VLAN to do per-VLAN DF election, the Local
      Administration Field of the RT-EVI is used to do per-EVI DF
      election.

7.4.  Traffic Forwarding Rules

   When a PE needs to forward received traffic from local Attachment
   Circuits (ACs) or remote PEs to local ACs, it follows the rules in
   RFC 7432, except that traffic sourced from this local PE but relayed
   back on a p2mp tunnel is discarded.  It may also need to forward to
   other PEs, subject to rules in the following sections.

7.4.1.  Traffic from Local ACs

   Traffic from a V-hub's local ACs is forwarded using the tunnel
   announced in its IMET route, as specified in RFC 7432.  In case of an
   mLDP tunnel, the traffic need to be relayed by V-hubs of other



Patel, et al.           Expires September 8, 2017              [Page 10]


Internet-Draft                                                March 2017


   clusters to their associated V-spokes.  For other tunnel types, no
   relay is needed.

   Traffic from a V-spoke's local ACs is forwarded to an associated
   V-hub of its choice.  In case of MPLS IR, the label in the V-hub's
   IMET route's PED attribute corresponding to this V-spoke is used.

7.4.2.  Traffic Received by a V-hub from Another PE

   When a V-hub receives traffic from an associated V-spoke, it needs to
   relay to other PEs, using the tunnel announced in its IMET route.  In
   case of IR or BIER, the source V-spoke, which is determined from the
   incoming label or source IP address, is excluded from the replication
   list.  In case of a P2MP tunnel, the popped incoming label is imposed
   again to identify the source PE, before the tunnel label is imposed.

   When a V-hub receives traffic from another V-hub on a P2MP tunnel,
   and the tunnel is announced in an IMET route carrying the same RT-VH
   as this V-hub is configured with, it does not need to relay the
   traffic.  Otherwise, the traffic is from a V-hub in a different
   cluster, and this V-hub needs to relay to its associated V-spokes, if
   and only if it is the DF for this cluster, using the tunnel announced
   in its (*,*) S-PMSI route carrying its RT-VH.

   When a V-hub recevies traffic from another V-hub via IR or BIER, it
   does not further relay the traffic as that V-hub can reach all PEs.

7.4.3.  Traffic received by a V-spoke from a V-hub

   In case of P2MP tunnel, the V-spoke discards the traffic if the label
   following the tunnel label identifies the V-spoke itself.

7.5.  Multi-homing support

   Consider that an ES spans across two V-spokes in the same cluster and
   the V-hub uses MPLS IR to relay traffic.  With ESI Label split
   horizon method, a source V-spoke uses the ESI label advertised by the
   V-hub for the ES, and the V-hub must change that to the ESI label
   advertised by receiving v-spokes when it relays traffic.  That means
   V-hubs must advertise ESI labels for all multi-homing segments, even
   when they're not on those segments.  They must also do double label
   swap (EVI/BD label and ESI label) or mac lookup when relaying
   traffic.

   To avoid that complexity, Local Bias is the preferred method for
   split horizon.  The PED label following the mpls transport tunnel
   label or BIER header identifies the PE that originated the traffic in
   addition to identifying the EVI/BD.



Patel, et al.           Expires September 8, 2017              [Page 11]


Internet-Draft                                                March 2017


   If a V-hub uses P2MP or BIER to relay traffic, the PED label is one
   of the labels in the PE Distinguiser Label attribute in the V-hub's
   IMET route, allocated by the V-hub for the source V-spoke.

   If a V-hub uses IR to relay traffic, for each V-spoke that it relays
   to, the PED label advertised by that receiving V-spoke for the source
   V-spoke needs to be imposed by the V-hub.  For that purpose, each
   V-spoke must include the PED Label attribute in its IMET route, to
   advertise different labels for different PEs.  It discovers the PEs
   that it needs to advertise labels for via the PED label Attribute in
   the V-hub's IMET route.

7.6.  Direct V-spoke to V-spoke traffic

   It may be desired for allow direct V-spoke to V-spoke traffic in a
   cluster, without the relay by a V-hub.

   To do that, V-spokes advertise their IMET routes with both RT-VH and
   RT-EVI.

   Forwarding rules will be specified in future revisions.

8.  ARP/ND Suppression

   [RFC7432] defines the procedures for ARP/ND suppression where a PE
   can terminate gratuitous ARP/ND request message from directly
   connected site and advertises the associated MAC and IP addresses in
   an EVPN MAC/IP advertisement route to all other remote PEs.  The
   remote PEs that receive this EVPN route advertisement, install the
   MAC/IP pair in their ARP/ND cache table thus enabling them to
   terminate ARP/ND requests and generate ARP/ND responses locally thus
   suppressing the flooding of ARP/ND requests over the EVPN network.

   In this hub-and-spoke approach, the ARP suppression needs to be
   performed by both the EVPN V-hubs as well V-spokes as follow.  When a
   V-Spoke receives a gratuitous ARP/ND request, it terminates it and
   stores the source MAC/IP pair in its ARP/ND cache table.  Then, it
   advertises the source MAC/IP pair to its associated V-Hubs using EVPN
   MAC/IP advertisement route.  The V-Hubs upon receiving this EVPN
   route advertisement, create an entry in their ARP/ND cache table for
   this MAC/IP pair.

   Now when a V-Spoke receives an ARP/ND request, it first looks up its
   ARP cache table, if an entry for that MAC/IP pair is found, then an
   ARP/ND response is generated locally and sent to the CE.  However, if
   an entry is not found, then the ARP/ND request is unicasted to one of
   the V-hub associated with this V-spoke.  Since, the associated V-hub
   keeps all the MAC/IP ARP entries in its cache table, it can formulate



Patel, et al.           Expires September 8, 2017              [Page 12]


Internet-Draft                                                March 2017


   and ARP/ND response and forward it to that CE via the corresponding
   V-spoke.

9.  IANA Considerations

   This document does NOT make any new requests for IANA allocations.

10.  Security Considerations

   All the security considerations in [RFC7432] apply directly to this
   document because this document leverages [RFC7432] control plane and
   their associated procedures - although not the complete set but
   rather a subset.

   This draft does not introduce any new security considerations beyond
   that of [RFC7432] and [RFC4761] because advertisements and processing
   of B-MAC addresses follow that of [RFC7432] and processing of C-MAC
   addresses follow that of [RFC4761] - i.e, B-MAC addresses are learned
   in control plane and C-MAC addresses are learned in data plane.

11.  Acknowledgements

   The authors would like to thank Yakov Rekhter for initial idea
   discussions.

12.  Change Log

   Initial Version:  Sep 21 2014

13.  References

13.1.  Normative References

   [I-D.zzhang-bess-evpn-bum-procedure-updates]
              Zhang, J., Lin, W., Rabadan, J., and K. Patel, "Updates on
              EVPN BUM Procedures", draft-zzhang-bess-evpn-bum-
              procedure-updates-01 (work in progress), December 2015.

   [RFC1771]  Rekhter, Y. and T. Li, "A Border Gateway Protocol 4 (BGP-
              4)", RFC 1771, DOI 10.17487/RFC1771, March 1995,
              <http://www.rfc-editor.org/info/rfc1771>.

   [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
              Requirement Levels", BCP 14, RFC 2119,
              DOI 10.17487/RFC2119, March 1997,
              <http://www.rfc-editor.org/info/rfc2119>.





Patel, et al.           Expires September 8, 2017              [Page 13]


Internet-Draft                                                March 2017


   [RFC2784]  Farinacci, D., Li, T., Hanks, S., Meyer, D., and P.
              Traina, "Generic Routing Encapsulation (GRE)", RFC 2784,
              DOI 10.17487/RFC2784, March 2000,
              <http://www.rfc-editor.org/info/rfc2784>.

   [RFC3484]  Draves, R., "Default Address Selection for Internet
              Protocol version 6 (IPv6)", RFC 3484,
              DOI 10.17487/RFC3484, February 2003,
              <http://www.rfc-editor.org/info/rfc3484>.

   [RFC3931]  Lau, J., Ed., Townsley, M., Ed., and I. Goyret, Ed.,
              "Layer Two Tunneling Protocol - Version 3 (L2TPv3)",
              RFC 3931, DOI 10.17487/RFC3931, March 2005,
              <http://www.rfc-editor.org/info/rfc3931>.

   [RFC4213]  Nordmark, E. and R. Gilligan, "Basic Transition Mechanisms
              for IPv6 Hosts and Routers", RFC 4213,
              DOI 10.17487/RFC4213, October 2005,
              <http://www.rfc-editor.org/info/rfc4213>.

   [RFC4271]  Rekhter, Y., Ed., Li, T., Ed., and S. Hares, Ed., "A
              Border Gateway Protocol 4 (BGP-4)", RFC 4271,
              DOI 10.17487/RFC4271, January 2006,
              <http://www.rfc-editor.org/info/rfc4271>.

   [RFC4374]  McCobb, G., "The application/xv+xml Media Type", RFC 4374,
              DOI 10.17487/RFC4374, January 2006,
              <http://www.rfc-editor.org/info/rfc4374>.

   [RFC6459]  Korhonen, J., Ed., Soininen, J., Patil, B., Savolainen,
              T., Bajko, G., and K. Iisakkila, "IPv6 in 3rd Generation
              Partnership Project (3GPP) Evolved Packet System (EPS)",
              RFC 6459, DOI 10.17487/RFC6459, January 2012,
              <http://www.rfc-editor.org/info/rfc6459>.

   [RFC7024]  Jeng, H., Uttaro, J., Jalil, L., Decraene, B., Rekhter,
              Y., and R. Aggarwal, "Virtual Hub-and-Spoke in BGP/MPLS
              VPNs", RFC 7024, DOI 10.17487/RFC7024, October 2013,
              <http://www.rfc-editor.org/info/rfc7024>.

   [RFC7432]  Sajassi, A., Ed., Aggarwal, R., Bitar, N., Isaac, A.,
              Uttaro, J., Drake, J., and W. Henderickx, "BGP MPLS-Based
              Ethernet VPN", RFC 7432, DOI 10.17487/RFC7432, February
              2015, <http://www.rfc-editor.org/info/rfc7432>.







Patel, et al.           Expires September 8, 2017              [Page 14]


Internet-Draft                                                March 2017


13.2.  Informative References

   [I-D.drao-bgp-l3vpn-virtual-network-overlays]
              Rao, D., Mullooly, J., and R. Fernando, "Layer-3 virtual
              network overlays based on BGP Layer-3 VPNs", draft-drao-
              bgp-l3vpn-virtual-network-overlays-03 (work in progress),
              July 2014.

   [I-D.ietf-bess-evpn-overlay]
              Sajassi, A., Drake, J., Bitar, N., Isaac, A., Uttaro, J.,
              and W. Henderickx, "A Network Virtualization Overlay
              Solution using EVPN", draft-ietf-bess-evpn-overlay-01
              (work in progress), February 2015.

   [RFC4389]  Thaler, D., Talwar, M., and C. Patel, "Neighbor Discovery
              Proxies (ND Proxy)", RFC 4389, DOI 10.17487/RFC4389, April
              2006, <http://www.rfc-editor.org/info/rfc4389>.

   [RFC4761]  Kompella, K., Ed. and Y. Rekhter, Ed., "Virtual Private
              LAN Service (VPLS) Using BGP for Auto-Discovery and
              Signaling", RFC 4761, DOI 10.17487/RFC4761, January 2007,
              <http://www.rfc-editor.org/info/rfc4761>.

   [RFC7080]  Sajassi, A., Salam, S., Bitar, N., and F. Balus, "Virtual
              Private LAN Service (VPLS) Interoperability with Provider
              Backbone Bridges", RFC 7080, DOI 10.17487/RFC7080,
              December 2013, <http://www.rfc-editor.org/info/rfc7080>.

   [RFC7209]  Sajassi, A., Aggarwal, R., Uttaro, J., Bitar, N.,
              Henderickx, W., and A. Isaac, "Requirements for Ethernet
              VPN (EVPN)", RFC 7209, DOI 10.17487/RFC7209, May 2014,
              <http://www.rfc-editor.org/info/rfc7209>.

Authors' Addresses

   Keyur Patel
   Arrcus

   Email: keyur@arrcus.com


   Ali Sajassi
   Cisco Systems
   170 W. Tasman Drive
   San Jose, CA 95124  95134
   USA

   Email: sajassi@cisco.com



Patel, et al.           Expires September 8, 2017              [Page 15]


Internet-Draft                                                March 2017


   John E. Drake
   Juniper Networks, Inc.

   Email: jdrake@juniper.net


   Zhaohui Zhang
   Juniper Networks, Inc.

   Email: zzhang@juniper.net


   Wim Henderickx
   Nokia

   Email: wim.henderickx@nokia.com



































Patel, et al.           Expires September 8, 2017              [Page 16]