Skip to main content

Signaling control/forward plane information between network virtualization edges (NVEs)
draft-wu-nvo3-nve2nve-02

The information below is for an old version of the document.
Document Type
This is an older version of an Internet-Draft whose latest revision state is "Expired".
Author Qin Wu
Last updated 2013-04-02
RFC stream (None)
Formats
Additional resources
Stream Stream state (No stream defined)
Consensus boilerplate Unknown
RFC Editor Note (None)
IESG IESG state I-D Exists
Telechat date (None)
Responsible AD (None)
Send notices to (None)
draft-wu-nvo3-nve2nve-02
Network Virtualization Overlays Working                            Q. Wu
Group                                                             Huawei
Internet-Draft                                             April 3, 2013
Intended status: Standards Track
Expires: October 5, 2013

      Signaling control/forward plane information between network
                      virtualization edges (NVEs)
                        draft-wu-nvo3-nve2nve-02

Abstract

   This document focuses on control plane aspect related to both tenant
   system to NVE control interface and NVE to Oracle control interface
   NVE use to enable communication between tenant systems, which is
   complementary to [draft-kreeger-nvo3-hypervisor-nve-cp] that
   describes the high level control plane requirements related to the
   interaction between tenant system and NVE when the two entities are
   not co-located on the same physical device.

Status of this Memo

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF).  Note that other groups may also distribute
   working documents as Internet-Drafts.  The list of current Internet-
   Drafts is at http://datatracker.ietf.org/drafts/current/.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   This Internet-Draft will expire on October 5, 2013.

Copyright Notice

   Copyright (c) 2013 IETF Trust and the persons identified as the
   document authors.  All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents
   (http://trustee.ietf.org/license-info) in effect on the date of
   publication of this document.  Please review these documents
   carefully, as they describe your rights and restrictions with respect

Wu                       Expires October 5, 2013                [Page 1]
Internet-Draft                   NVE2NVE                      April 2013

   to this document.  Code Components extracted from this document must
   include Simplified BSD License text as described in Section 4.e of
   the Trust Legal Provisions and are provided without warranty as
   described in the Simplified BSD License.

Table of Contents

   1.  Introduction . . . . . . . . . . . . . . . . . . . . . . . . .  3
   2.  Conventions used in this document  . . . . . . . . . . . . . .  4
   3.  NVO3 Control plane Overview  . . . . . . . . . . . . . . . . .  5
   4.  Mapping table entry at the NVE and Information mapping
       authority  . . . . . . . . . . . . . . . . . . . . . . . . . .  7
     4.1.  Mapping table Entry Fields relationship  . . . . . . . . .  8
     4.2.  Multihoming support using one vNIC . . . . . . . . . . . . 10
     4.3.  Interconnection functionality  . . . . . . . . . . . . . . 10
   5.  NVE to Oracle Control plane protocol functionality . . . . . . 13
     5.1.  NVE connect/disconnect notification  . . . . . . . . . . . 13
     5.2.  VN membership Registration and Query . . . . . . . . . . . 13
     5.3.  Address Mapping information reflection/distribution  . . . 14
   6.  Hypervisor-to-NVE Control Plane Protocol Functionality . . . . 15
     6.1.  Multiple vNIC addresses association  . . . . . . . . . . . 15
   7.  Key functions aspect for signaling control/forwarding info
       to NVEs  . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
     7.1.  Create and Update tenant Virtual Network (VN)  . . . . . . 16
     7.2.  Associate the NVE and tenant system with VN context  . . . 16
     7.3.  Populate mapping tables information at the local NVE . . . 17
     7.4.  Distribute the mapping table information to remote
           NVEs in the VN . . . . . . . . . . . . . . . . . . . . . . 17
     7.5.  The mapping table information update at the NVE when
           VM moves or connection fails . . . . . . . . . . . . . . . 17
     7.6.  The VN context re-association at the NVE when VM moves . . 18
   8.  IANA Considerations  . . . . . . . . . . . . . . . . . . . . . 19
   9.  Security Considerations  . . . . . . . . . . . . . . . . . . . 20
   10. References . . . . . . . . . . . . . . . . . . . . . . . . . . 21
     10.1. Normative References . . . . . . . . . . . . . . . . . . . 21
     10.2. Informative References . . . . . . . . . . . . . . . . . . 21
   Author's Address . . . . . . . . . . . . . . . . . . . . . . . . . 22

Wu                       Expires October 5, 2013                [Page 2]
Internet-Draft                   NVE2NVE                      April 2013

1.  Introduction

   In [I.D-ietf-nvo3-overlay-problem-statement], two control planes are
   identified to realize an overlay solution:

   o  NVE to Oracle control plane.

   o  Tenant system to NVE control plane.

   Where NVE to Oracle Control plane is used to deal with address
   mapping dissemination and Tenant System to NVE control plane is used
   to deal with VM attachment and detachment.

   In [I.D-ietf-nvo3-framework], three control plane components are
   defined to build these two control planes and provide the following
   capabilities:

   o  Auto-provision/service discovery

   o  Address advertisement

   o  Tunnel Managment

   In [I.D-fw-nvo3-server2vcenter],the control interface between NVE and
   Oracle backend system or Information Mapping Authority is defined to
   provide the capability:

   o  Enforce the network policy for each VM in the path from the NVE
      Edge associated with VM to the Tenant End System.

   o  Populate forwarding table in the path from the NVE Edge associated
      with VM to the Tenant End System in the data center.

   o  Populate mapping table in each NVE Edge that is in the virtual
      network across data centers under the control of the Director.

   This document focuses on control plane aspect related to both tenant
   system to NVE control interface and NVE to Oracle control interface
   NVE use to enable communication between tenant systems, which is
   complementary to [draft-kreeger-nvo3-hypervisor-nve-cp] that
   describes the high level control plane requirements related to the
   interaction between tenant system and NVE when the two entities are
   not co-located on the same physical device.

Wu                       Expires October 5, 2013                [Page 3]
Internet-Draft                   NVE2NVE                      April 2013

2.  Conventions used in this document

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
   document are to be interpreted as described in RFC2119 [RFC2119].

   Site :

      If multiple tenant systems connect to the VN through one NVE, the
      collection of these tenant systems and the NVE associated with
      these tenant systems are referred to as a site or virtualization
      network subnet.

   Tenant System:

      A physical or virtual system that can play the role of a host, or
      a forwarding element such as a router, switch, firewall, etc.  It
      belongs to a single tenant and connects to one or more VNs of that
      tenant.

   vNIC:

      A vNIC is similar to a physical NIC.  Each virtual machine has one
      or more vNIC adapters that it uses to communicate with both the
      virtual and physical networks.  Each vNIC has its own MAC address
      and can be assigned one or more IP address just like a NIC found
      in a non virtualized machine.

Wu                       Expires October 5, 2013                [Page 4]
Internet-Draft                   NVE2NVE                      April 2013

3.  NVO3 Control plane Overview

   Figure 1 shows the example NVO3 Networking architecture to give an
   overview of NVO3 control plane for interconnection between VNs or
   between VN and Non-VN.  There are 4 basic network components that
   make up networking architecture: Tenant system, local NVE, remote NVE
   and Information mapping authority.  This example NVO3 networking
   architecture assumes that:

   o  One tenant system, can host multiple virtual machines.  Each
      virtual machine has one or more vNIC adapters that it uses to
      communicate with both the virtual and physical networks.

   o  One tenant system or a NVE may belong to one tenant VN or several
      tenant VNs, e.g., TS4 and NVE Edge4 belong to both VN2 and VN3.

   o  If one tenant system belongs to multiple tenant VNs, it may have
      one VM with multiple vNICs and connect to each tenant VN by using
      each of vNICs being attached to one or multiple NVEs,e.g., VM1
      connect to VN1 by being attached to NVE Edge 1.

   o  If one tenant system belongs to multiple tenant VNs, it may have
      multiple VM with each having one vNIC and connect to each tenant
      VN by using each VM vNIC being attached to one or multiple NVEs.

   o  One NVE only belongs to one site.  One site may belong to one
      tenant VN or several tenant VN, e.g., Site 2 belong to both VN2
      and VN3.

   o  If one tenant system in one VN want to communicate with one tenant
      system in another VN, the interconnection functionality or
      information mapping authority or NVE controller may get involved
      to help establish interconnection between VN or setup tunnel
      between NVEs associated with the tenant system.

Wu                       Expires October 5, 2013                [Page 5]
Internet-Draft                   NVE2NVE                      April 2013

      +---------------+-------------+--------------+
      | VN1           | +--------+  |   +--------+ |
      |               | |VM1VM2VM3  |   |VM4VM5VM6 |
      |               | +--------+  |   +--------+ |
      |               | |        |  |   |        | |
      |               | |   TS1  |  |   |  TS2   | |
      |               | |        |  |   |        | |
      |               | +--------+  |   +--------+ |
      |               | +---------+ |   +---------+|
      |              -+-|NVE Edge1+-+---+NVE Edge2++-
      |            // | +---------+ |   +---------+| \\
      |           |   |Site1        |              |   |
      |           |   +-------------+  +VN3--------+---+------------+
      | |--+---+ +---+                 |           |+---+ +---|--+  |
      | |VMd   | |   |                 |           ||   | |   |VMh  |
      | |  |   | |NVE|                 |           ||NVE| |   |  |  |
      | |  |   | |   |                 |           ||   | |   |  |  |
      | |VMe T | | E |   ,---------.   |           || E | | T |VMi  |
      | |  |   | | d | Interconnection |           || d | |   |  |  |
      | |  | S | | g |(               )|           || g | | S |  |  |
      | |VMf   | | e | `Functionality' |           || e | |   |VMj  |
      | |  | 5 | | 5 |   `---------'   |           || 6 | | 6 |  |  |
      | |VMg   | |   |                 |           ||   | |   |VMk  |
      | |--|---+ ++--+                 |           |++--+ |---+--+  |
      +-----------+--------------------+-----------+   |            |
                  |                    |               |            |
                  |  +VN2--------------+------------+  |            |
                 .   |... .............|Site2..... .|  |            |
                 .|  | +---------+ ..  |+---------+ |//             |
                 . \\| |NVE Edge3+-----++NVE Edge4+-+               |
                 .   | +---------+     |+---------+ |               |
                 .   | +--------+  ..  | +--------+ | .             |
                 .   | |        |  ..  | |        | | .             |
                 .   | |   TS3  |  ..  | |   TS4  | | .             |
                 .   | |        |  ..  | |        | | .             |
                 .   | +--------+  ..  | +--------+ | .             |
                 .   | |VM7VM8VM9  ..  | |VMaVMbVMc | .             |
                 ....| ---------+......|.---------+ |..             |
                     +-----------------+------------+               |
                                       +----------------------------+

               Figure 1: Example NVO3 control plane Overview

Wu                       Expires October 5, 2013                [Page 6]
Internet-Draft                   NVE2NVE                      April 2013

4.  Mapping table entry at the NVE and Information mapping authority

   Tenant system VNICs should be able to directly connect to multiple
   VNs without needing to traverse a NVE or gateway.  Therefore every
   NVE pair( local NVE and remote NVE ) associated with the tenant
   system vNIC MUST maintain one or multiple mapping table entry for
   each currently attached VNIC of tenant system.  Each mapping table
   entry should include a IP address of vNIC and corresponds to the
   connection to each VN.  Each mapping table entry conceptually may
   contain all or a sub set of the following fields:

   o  The tunnel interface identifier (tunnel-if-id) of the tunnel
      between the remote NVE and the local NVE where the tenant system
      is currently attached.  The tunnel interface identifier is
      acquired during the tunnel creation.

   o  The MAC address of the attached TS vNIC.  This MAC address is
      obtained from auto-discovery protocol between Tenant System and
      its local NVE.

   o  The IP address of the attached TS vNIC.  This IP address is
      obtained from auto-discovery protocol between Tenant System and
      its local NVE.

   o  The logical interface identifier (e.g., VLAN ID, internal vSwitch
      Interface ID connected to a VM) of the access link between the
      tenant system and the local NVE.  This field is required to
      associate with Tenant System vNIC if local NVE is an external NVE
      to Tenant system and is internal to the local NVE and is used to
      associate the tunnel to the access link where the tenant system is
      attached.

   o  The MAC address of the local NVE associated with the tenant
      system.

   o  The IP address of the local NVE associated with the tenant system.

   o  The Identifier of VN context-VNID.  This Identifier is obtained
      from auto-discovery protocol between Tenant System and its local
      NVE.

   In addition, Oracle backend system or information mapping authority
   may also maintain a mapping table entry for each currently attached
   tenant system vNIC or each newly joined NVE.  Each mapping table
   entry conceptually may contain all or a sub set of the following
   fields:

Wu                       Expires October 5, 2013                [Page 7]
Internet-Draft                   NVE2NVE                      April 2013

   o  The MAC address of the attached TS vNIC.  This MAC address is
      obtained from auto-discovery protocol between Tenant System and
      its local NVE.(Optional)

   o  The IP address of the attached TS vNIC.  This IP address is
      obtained from auto-discovery protocol between Tenant System and
      its local NVE.  (Optional)

   o  The logical interface identifier (e.g., VLAN ID, internal vSwitch
      Interface ID connected to a VM) of the access link between the
      tenant system and the local NVE.  This field is required to
      associate with Tenant System vNIC if local NVE is an external NVE
      to Tenant system and is internal to the local NVE and is used to
      associate the tunnel to the access link where the tenant system is
      attached.

   o  The MAC address of the local NVE associated with the tenant
      system.

   o  The IP address of the local NVE associated with the tenant system.

   o  The Identifier of VN context-VNID.  This Identifier is obtained
      from auto-discovery protocol between Tenant System and its local
      NVE.

4.1.  Mapping table Entry Fields relationship

   One Tenant system can host multiple VMs.  Each VM has one or more
   vNIC adapters that it uses to communicate with both the virtual and
   physical networks.  Each vNIC must have one and only one unique MAC
   address.  In addition, each vNIC have at least one IP address.  When
   a VM using one vNIC connects to multiple VNs,the vNIC should be
   assigned with multiple IP addresses with each connecting to different
   VN.  In this case, VM may create multiple binding cache entries with
   each associating one of multiple IP addresses to the same unique vNIC
   MAC address. vNIC MAC address may be modified or assigned with a new
   MAC address at any time.  However vNIC should not use more than one
   MAC addresses to connect to multiple VNs at the same time.  When
   multiple vNICs hosted in the same VM connect to multiple VNs, it is
   allowed that some of these vNICs may connect to different VNs through
   the same NVE.

                            Tenant System (TS)
                                    |
                     +--------------+--------------+
                     |              |              |
                                    |              |
                    VM1             VM2     .....  VMx

Wu                       Expires October 5, 2013                [Page 8]
Internet-Draft                   NVE2NVE                      April 2013

                     |
           |---------+--------+
           |         |        |
           |         |        |
          vNIC1     vNIC2 ... vNICx
           |
           |
    +------+------+
    |      |      |
   VN1    VN2 ...VNx

           Figure 2. VM information Hierarchy

                      VM1
                       |
             |---------+--------+
             |         |        |
             |         |        |
            vNIC1     vNIC2 ... vNICx
             |
             |
             |
            NVE
             |
     +-------+------+
     |       |      |
    VN1     VN2    VN3

    Binding Cache Database
    VM1 vNIC1's Binding
    binding[VNIC1 MAC addr, IP1 addr, BID1]
    binding[vNIC1 MAC addr, IP2 addr, BID2]
    binding[vNIC1 MAC addr, IP3 addr, BID3]

   Figure 3 Simultaneous multiple connections
   for Layer 3 Virtual Network Service

                     VM1
                      |
            |---------+--------+
            |         |        |
            |         |        |
           vNIC1     vNIC2 ... vNICx
            |
            |

Wu                       Expires October 5, 2013                [Page 9]
Internet-Draft                   NVE2NVE                      April 2013

            |
           NVE
            |
    +-------+------+
    |       |      |
   VN1     VN2    VN3

   Binding Cache Database
   VM1 vNIC1's Binding
   binding[VNIC1 MAC addr, VLAN ID1, BID1]
   binding[vNIC1 MAC addr, VLAN ID2, BID2]
   binding[vNIC1 MAC addr, VLAN ID3, BID3]

   Figure 4. Simultaneous multiple connections
    for Layer 2 Virtual Network Service

4.2.  Multihoming support using one vNIC

   Tenant System vNIC may use several IP addresses to connect to
   multiple VNs.  Each IP address corresponding to each connection to
   VN.  In order to support connecting to multiple VNs using a single
   vNIC, Tenant system may create a Binding Identifier (BID) to each IP
   address that is used to connect to VN.  The BID should be unique for
   a given Tenant System vNIC MAC address.If the tenant system vNIC has
   only one IP address, the assignment of a BID is not needed until it
   has multiple IP addresses for a single vNIC, at which time all of the
   IP addresses of vNIC MUST be mapped to BIDs.  BID is suggested to be
   stored in the mapping table entry at the NVE and information mapping
   authority.

4.3.  Interconnection functionality

   Suppose the tenant system A hosts only one VM that have only one vNIC
   adapter, when the tenant system A plays the role of interconnection
   functionality to connect two VN, three cases should be considered.

      (a) Both two VNs support Layer 3 forwarding;

      (b) Both two VNs support layer 2 forwarding;

      (c) One VN support Layer3 forwarding and the other VN support
      layer 2 forwarding;

   For (a), tenant system A vNIC or external system that is close to
   tenant system A should support layer 3 forwarding functionality.
   When source tenant system in one VN communicates with destination
   tenant system in another VN through the tenant system A, if tenant
   system A vNIC support layer 3 forwarding, the tenant system A should

Wu                       Expires October 5, 2013               [Page 10]
Internet-Draft                   NVE2NVE                      April 2013

   forward IP packets on the behalf of source Tenant System and
   destination tenant system irrespective of data plane encapsulation
   format(e.g., VXLAN or NVGREW, MPLS over GRE).  If two VNs use
   different data plane encapsulation format, the tenant system A should
   also support converting one data plane encapsulation format into
   another.  If tenant system A vNIC doesn't support layer 3
   forwarding,the external system that is close to tenant system A
   should associate vNIC to local NVE using TS vNIC MAC address and VLAN
   tag information and forward IP packets on the behalf of source tenant
   system and destination tenant system.

   For (b), tenant system A vNIC or external system that is close to
   tenant system A should support layer 2 forwarding functionality.
   When source tenant system in one VN communicates with destination
   tenant system in another VN through the tenant system A, if tenant
   system A support layer 2 forwarding, the tenant system A should know
   which tenant systems connecting to itself are allowed for layer 2
   forwarding and then forward layer 2 frames on the behalf of source
   Tenant System and destination tenant system based on forwarding
   allowed list.  If two layer 2 VNs support different data plane
   encapsulation format, the tenant system A should also support
   converting one data plane encapsulation format to another.  If tenant
   system A vNIC doesn't support layer 2 forwarding,the external system
   that is close to tenant system A should associate vNIC to local NVE
   using TS vNIC MAC address and VLAN tag information and forward layer
   2 frames on the behalf of source tenant system and destination tenant
   system.

   For (c), tenant system A or external system that is close to tenant
   system A should support both layer 2 forwarding and layer 3
   forwarding.  When source tenant systems in layer 2 VN communicates
   with destination tenant system in layer 3 VN through the tenant
   system A, if tenant system A support both layer 2 and layer 3
   forwarding the tenant system A should support translating layer 2
   frame into layer 3 packet and forward traffic between layer 2 VN and
   layer 3 VN.  If two VNs support different data plane encapsulation
   format, the tenant system A should also support converting one data
   plane encapsulation format to another.  If tenant system A vNIC
   doesn't support layer 2 forwarding or layer3 forwarding,the external
   system that is close to tenant system A should associate vNIC to
   local NVE using TS vNIC MAC address and VLAN tag information and
   forward traffic on the behalf of source tenant system and destination
   tenant system.

   When the tenant system A plays the role of interconnection
   functionality to connect between VN and Non-VN, suppose source tenant
   system in one VN communicates with destination end device in Non-VN
   environment through the tenant system A, the tenant system A acts as

Wu                       Expires October 5, 2013               [Page 11]
Internet-Draft                   NVE2NVE                      April 2013

   NVO3 GW between VN and Non-VN in this case and should be explicitly
   configured with a list of destination MAC addresses that allows
   passed to the Non-VN environment.  For outgoing frames on VN
   connected interface, the tenant system A decapsulates NVO3 outer
   header and forward the inner frame to Non-VN environment based on
   configured allowed list.  For incoming frames on non-VN connected
   interface, the tenant system A should map the incoming frames from
   end device to specific VN based on inner Ethernet frame information
   (e.g., VLAN ID).

Wu                       Expires October 5, 2013               [Page 12]
Internet-Draft                   NVE2NVE                      April 2013

5.  NVE to Oracle Control plane protocol functionality

   The core functional entities for NVE to Oracle Control plane
   infrastructure are the NVE and Oracle backend system.  The Oracle
   backend system is responsible for maintaining the tenant system
   VNIC's reachability state and is the topological anchor point for the
   Tenant system vNIC MAC address or vNIC Name.  The NVE is the entity
   that performs the address mapping management on behalf of tenant
   system vNIC, and it resides on the access link where the tenant
   system is anchored.  The NVE is responsible for detecting the VM's
   movements to and from the access link and for initiating location
   binding registrations to the tenant system's Oracle backend system.
   There can be multiple Oracle backend system in a VN each serving a
   different group of tenant system.

5.1.  NVE connect/disconnect notification

   When a tenant system connects to one VN by attaching to a local NVE,
   the local NVE should also be added into VN context together with
   tenant system information.  This helps Oracle Backend System know
   with which NVE a group of the tenant systems are attached or current
   location of these tenant systems.  When the last tenant system is
   disconnected to one VN through one local NVE, this local NVE should
   also be removed from VN context.  This should also be updated to
   Oracle Backend system and let Oracle backend system know that there
   are no tenant system associated with that NVE.

5.2.  VN membership Registration and Query

   In order to enable tenant system A to communicate with any tenant
   system that is not under the same local NVE, the mapping table should
   be distributed to all the remote NVEs that belong to the same VN even
   through there is no tenant system which communicates with tenant
   system A behind that remote NVE.  However how should local NVE know
   the list of remote NVEs that belong to the same VN as local NVE?  In
   order to tackle this, when a tenant system connects to one VN by
   attaching to a local NVE, VN membership (e.g., VNID, VN Name, a list
   of NVE that belong to VN) should be registered to the Oracle Backend
   system.  When local NVE needs to know which remote NVEs it should
   forward a data packet, Oracle backend system can be queried by the
   local NVE.  The Oracle backend system can redirect the query from
   local NVE to the remote NVEs based on VN membership registration and
   obtain answer from the right remote NVE.  In addition, VM membership
   may contain detailed mapping between tenant system,NVE and VN in the
   form of TESID=<VNID, NVE_ID,TS_ID>.  In this case, Oracle backend
   system can directly supply answer for the request from the local NVE.

Wu                       Expires October 5, 2013               [Page 13]
Internet-Draft                   NVE2NVE                      April 2013

5.3.  Address Mapping information reflection/distribution

   Data plane learning can be used to build mapping table without need
   for control plane protocol.  However it requires each data packet to
   be flooded to the whole VN.  In order to eliminate flooding
   introduced by data plane learning, a control protocol is needed to
   provide both MAC address and IP address in the form of mapping
   information.  When VN membership registration complete, the NVE can
   forward such address mapping information directly to all the remote
   NVEs based on VN membership, alternatively, the NVE also can forward
   such address mapping information indirectly to the Oracle backend
   system and let Oracle backend system to reflect the address mapping
   information to all the relevant remote NVEs based on VN membership.

Wu                       Expires October 5, 2013               [Page 14]
Internet-Draft                   NVE2NVE                      April 2013

6.  Hypervisor-to-NVE Control Plane Protocol Functionality

6.1.  Multiple vNIC addresses association

   Typically, a VNIC is assigned a single MAC address and a single IP
   address.  However, a vNIC may be assigned multiple IP addresses with
   each to connect to one VN.  In such case, BID may be assigned for
   each IP address of vNIC when tenant system vNIC wants to register
   multiple IP address with its MAC address simultaneously to the local
   NVE.  If a tenant system vNIC has only one IP address, the assignment
   of a BID is not needed until it has multiple IP addresses to register
   with, at which time all of the IP addresses of vNIC MUST be mapped to
   BIDs.  When a tenant system vNIC registers a given BID for the first
   time it MUST send BID together with the IP address.  For any
   subsequent registrations that either re-register or de-register the
   same BID, the TS only need send BID and does not need to send IP
   address associated with BID.

Wu                       Expires October 5, 2013               [Page 15]
Internet-Draft                   NVE2NVE                      April 2013

7.  Key functions aspect for signaling control/forwarding info to NVEs

7.1.  Create and Update tenant Virtual Network (VN)

   The tenant virtualization network(VN) is a collection of tenant
   systems, Network Virtualization Edges (NVE)(s) and end systems that
   are interconnected with each other.  The tenant VN also consists of a
   set of sites where each can send traffic directly to the other.

   In order to create or update a tenant VN, when a Tenant System is
   attached to a local NVE, the tenant system should inform the attached
   local NVE which VN the tenant system belong to.

   o  If the tenant system are the first participant in the VN through
      the local NVE, the tenant system and associated local NVE should
      be firstly added to the VN and the mapping table should be setup
      at the local NVE for each attached tenant system.

   o  If both the tenant system and the local NVE are not on the VN, the
      tenant system and associated local should be firstly added to the
      VN and then the mapping table associated with this tenant system
      should be setup at the local NVE and distributed to the other
      remote NVEs that belong to the same VN.

   o  If the local NVE is on the same tenant VN as the tenant system
      associated with the local NVE, only the tenant system needs to be
      added into the VN, i.e., the local NVE only needs to distribute
      mapping table at the local NVE to the other remote NVEs that
      belong to the same tenant VN.

   o  If the local NVE is not on the same tenant VN as the tenant system
      associated with that local NVE, the local NVE should firstly be
      added into the VN and then distributes the new mapping table at
      the local NVE to the other remote NVEs that belong to the same
      tenant VN.

   o  If one tenant system is the last participant connecting to the VN
      through local NVE, when this tenant system leave the VN, the local
      NVE associated with this tenant system should be removed from the
      VN.The mapping table associated with this tenant system should be
      removed from the local NVE associated with this tenant system.

7.2.  Associate the NVE and tenant system with VN context

   The VN context includes a set of configuration attributes defining
   access and tunnel policies and (L2 and/or L3) forwarding functions.
   When a Tenant System is attached to a local NVE, a VN network
   instance should be allocated to the local NVE.  The tenant system

Wu                       Expires October 5, 2013               [Page 16]
Internet-Draft                   NVE2NVE                      April 2013

   should be associated with the specific VN context using virtual
   Network Instance(VNI).  The tenant system should also inform the
   attached local NVE which VN context the tenant system belong to.
   Therefore the VN context can be bound with the data path from the
   tenant system to the local NVE and the tunnel from local NVE
   associated with the tenant system and all the remote NVEs that belong
   to the same VN as the local NVE.  For the data path from the tenant
   system and the local NVE, the network policy can be installed on the
   underlying switched network and forwarding tables also can be
   populated to each network elements in the underlying network based on
   the specific VNI associated with the tenant system.  For the tunnel
   from local NVE to the remote NVEs, the traffic engineering
   information can be applied to each tunnel based on VNI associated
   with the tenant system.

7.3.  Populate mapping tables information at the local NVE

   In some cases, two tenant systems may be attached to the same local
   NVE.  In order to allow the NVE to locally route traffic between two
   tenant systems that are attached to the same NVE, the mapping table
   that maps a final destination address to the proper tunnel should be
   populated at the local NVE.

   In some cases, two tenant systems may connect to the different VNs
   through the same interconnection functionality, in order to allow two
   tenant systems communication between two VNs, the mapping table that
   maps a final destination address to the proper tunnel should be
   populated in both NVE associated with two communicated tenant system
   and the interconnection functionality associated corresponding NVE.

7.4.  Distribute the mapping table information to remote NVEs in the VN

   When the packet sent from one tenant system arrives at the ingress
   NVE associated with that tenant system, in order to determine which
   tunnel the packet needs to be sent to, the mapping table that maps a
   final destination address to the proper tunnel should also be
   distributed to all the remote NVEs in the VN using a control plane
   protocol or dynamic data plane learning.  The mapping table may be
   advertised directly to other remote NVEs that belong to the same VN
   or firstly advertised to the centralized controller that maintain
   global view of NVEs that belong to the same VN and then let the
   centralized controller distribute the mapping tables to all the
   relevant remote NVEs that belong to the same VN.

7.5.  The mapping table information update at the NVE when VM moves or
      connection fails

   In some cases, one tenant system may be detached from one NVE and

Wu                       Expires October 5, 2013               [Page 17]
Internet-Draft                   NVE2NVE                      April 2013

   move to another NVE.  In such cases, the mapping table should be
   removed from the NVE to which the tenant system was previously
   attached and the new mapping table should be created at the new NVE
   to which the tenant system is currently attached.  Such mapping table
   should be updated at each remote NVE associated with the tenant
   system and the new NVE.

   In some cases, one tenant system may fail to connect to the VN
   through the NVE.  In such cases, the mapping table should be removed
   from the NVE to which the tenant system is currently attached.  In
   addition, the mapping table should be updated at each remote NVE in
   the same VN through which the tenant system is communicating with the
   destination tenant system.

7.6.  The VN context re-association at the NVE when VM moves

   In some cases, one tenant system may be detached from one NVE and
   move to another NVE.  In such cases, the VN context should be moved
   from the NVE to which the tenant system was previously attached to
   the new NVE to which the tenant system is currently attached.  In
   order to achieve this, the per tenant system VN context can be
   maintained at the centralized database and be retrieved at the new
   place based on the VN Identifier (VNID).

Wu                       Expires October 5, 2013               [Page 18]
Internet-Draft                   NVE2NVE                      April 2013

8.  IANA Considerations

   This document has no actions for IANA.

Wu                       Expires October 5, 2013               [Page 19]
Internet-Draft                   NVE2NVE                      April 2013

9.  Security Considerations

   TBC.

Wu                       Expires October 5, 2013               [Page 20]
Internet-Draft                   NVE2NVE                      April 2013

10.  References

10.1.  Normative References

   [I.D-ietf-nvo3-framework]
              Lasserre, M., "Framework for DC Network Virtualization",
              ID draft-ietf-nvo3-framework-00, September 2012.

   [I.D-ietf-nvo3-overlay-problem-statement]
              Narten, T., "Problem Statement: Overlays for Network
              Virtualization",
              ID draft-ietf-nvo3-overlay-problem-statement-02,
              Feburary 2013.

   [I.D-kreeger-nvo3-hypervisor-nve-cp]
              Kreeger, L., "Network Virtualization Hypervisor-to-NVE
              Overlay Control Protocol Requirements",
              ID draft-kreeger-nvo3-hypervisor-nve-cp-01, Feburary 2013.

   [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
              Requirement Levels", March 1997.

10.2.  Informative References

   [I.D-fw-nvo3-server2vcenter]
              Wu, Q. and R. Scott, "Network Virtualization
              Architecture", ID draft-fw-nvo3-server2vcenter-01,
              January 2013.

Wu                       Expires October 5, 2013               [Page 21]
Internet-Draft                   NVE2NVE                      April 2013

Author's Address

   Qin Wu
   Huawei
   101 Software Avenue, Yuhua District
   Nanjing, Jiangsu  210012
   China

   Email: bill.wu@huawei.com

Wu                       Expires October 5, 2013               [Page 22]