Skip to main content

Stateless user-plane architecture for virtualized EPC (vEPC)
draft-matsushima-stateless-uplane-vepc-00

The information below is for an old version of the document.
Document Type
This is an older version of an Internet-Draft whose latest revision state is "Expired".
Authors Satoru Matsushima , Ryuji Wakikawa
Last updated 2013-07-10
RFC stream (None)
Formats
Additional resources
Stream Stream state (No stream defined)
Consensus boilerplate Unknown
RFC Editor Note (None)
IESG IESG state I-D Exists
Telechat date (None)
Responsible AD (None)
Send notices to (None)
draft-matsushima-stateless-uplane-vepc-00
Network Group                                              S. Matsushima
Internet-Draft                                          Softbank Telecom
Intended status: Informational                               R. Wakikawa
Expires: January 11, 2014                                Softbank Mobile
                                                           July 10, 2013

      Stateless user-plane architecture for virtualized EPC (vEPC)
               draft-matsushima-stateless-uplane-vepc-00

Abstract

   We envision a new mobile architecture for the future Evolved Packet
   Core (EPC).  The new architecture is designed to support the
   virtualization scheme called NFV (Network Function Virtualization).
   In our architecture, the user plane of EPC is decoupled from the
   control-plane and uses routing information to forward packets of
   mobile nodes.  Although the EPC control plane will run on hypervisor,
   our proposal does not modify the signaling of the EPC control plane.
   The benefits of our architecture are 1) scalability, 2) flexibility
   and 3) Manageability.  How to run the EPC control plane on NFV is out
   of our focus in this document.

Status of This Memo

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF).  Note that other groups may also distribute
   working documents as Internet-Drafts.  The list of current Internet-
   Drafts is at http://datatracker.ietf.org/drafts/current/.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   This Internet-Draft will expire on January 11, 2014.

Copyright Notice

   Copyright (c) 2013 IETF Trust and the persons identified as the
   document authors.  All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents
   (http://trustee.ietf.org/license-info) in effect on the date of

Matsushima & Wakikawa   Expires January 11, 2014                [Page 1]
Internet-Draft             Draft Specification                 July 2013

   publication of this document.  Please review these documents
   carefully, as they describe your rights and restrictions with respect
   to this document.  Code Components extracted from this document must
   include Simplified BSD License text as described in Section 4.e of
   the Trust Legal Provisions and are provided without warranty as
   described in the Simplified BSD License.

Table of Contents

   1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   2
     1.1.  The Benefits of NFV . . . . . . . . . . . . . . . . . . .   3
   2.  Motivations and Requirements, - Why IETF? - . . . . . . . . .   4
     2.1.  Motivations . . . . . . . . . . . . . . . . . . . . . . .   4
     2.2.  Requirements  . . . . . . . . . . . . . . . . . . . . . .   5
   3.  Stateless user-plane architecture for virtualized  EPC  . . .   7
     3.1.  Architecture Overview . . . . . . . . . . . . . . . . . .   7
     3.2.  Protocol Overview . . . . . . . . . . . . . . . . . . . .   9
     3.3.  Control-plane awareness of stateless user-plane . . . . .  13
     3.4.  Routing mechanism . . . . . . . . . . . . . . . . . . . .  13
     3.5.  IPv4 Support  . . . . . . . . . . . . . . . . . . . . . .  16
   4.  IANA Considerations . . . . . . . . . . . . . . . . . . . . .  17
   5.  Security Considerations . . . . . . . . . . . . . . . . . . .  17
   6.  References  . . . . . . . . . . . . . . . . . . . . . . . . .  17
     6.1.  Normative References  . . . . . . . . . . . . . . . . . .  17
     6.2.  Informative References  . . . . . . . . . . . . . . . . .  17
   Authors' Addresses  . . . . . . . . . . . . . . . . . . . . . . .  19

1.  Introduction

   3GPP introduces Evolved Packet Core (EPC) that is fully IP based
   mobile system for LTE and -advanced in their Release-8 specification
   and beyond.  Operators are now deploying EPC for LTE services and
   encounter rapid LTE traffic growth.  There are various activities to
   offload mobile traffic in 3GPP and IETF such as LIPA, SIPTO and DMM.
   The concept is similar that traffic of OTT (Over The Top) application
   is offloaded at entity that is closer to the mobile node (ex. eNodeB
   or closer anchor).

   Likewise, overload of signaling (control plane) is also increasing
   day by day.  Network operators expect recent innovation and trends of
   NFV (Network Function Virtualization) to solve this overloaded
   control plane.  NFV is discussed at the ETSI NFV ISG and is
   introduced in [NFV-WHITEPAPER].  Mobile operator's network is built
   with variety of proprietary hardware appliances today.  If we can get
   rid of these physical appliances and could shift to a cloud-based
   service, we will have a lot of benefits explained in the next
   section.  This document assumes that NFV will push networking
   functions currently run on dedicated hardwares onto a cloud network.

Matsushima & Wakikawa   Expires January 11, 2014                [Page 2]
Internet-Draft             Draft Specification                 July 2013

   Expected network functions are Mobility Management Entity (MME),
   Serving Gateway (SGW) PDN Gateway(PGW), etc.  With NFV, EPC can be
   operated onto servers/hyper-visors.  We name it virtualized-EPC
   (vEPC) in this document.

   This document uses a lot of 3GPP specific terms.  These terms can be
   found mostly at [RFC6459].

1.1.  The Benefits of NFV

   This section briefly explains the benefits of NFV.  The detailed
   benefits can be found in [NFV-WHITEPAPER].  Although today's eco-
   system of EPC appliances might be affected, we believe there are
   various approaches to enhance current eco-system and migrate to new
   NFV approaches.  For example, operators could pay monthly recurring
   charges for the NFV services and operations to vendors, instead of
   one-time purchase and a little maintenance cost.

   o  [Flexible Network Operations]: The control functions of EPC are no
      longer in appliances deployed widely in operator's network and can
      be run at hypervisor (cloud).  It is easier to add and/ or delete
      functions from the services, because no physical construction is
      needed.  Network operations will be much simpler and easier
      because complications of today's network are pushed to NFV (i.e.
      hypervisor).

   o  [Flexible Resource Managements]: The EPC functions can be run on
      hypervisor and are now less dependent on proprietary hardwares.
      Adding additional resources is easier in hypervisor, while adding
      or replacing physical appliances require installation,
      construction, configuration, and even migration plan without
      service cutoff.  A hypervisor can be also shared across various
      functions such as PGW, SGW and MME.  NFV also brings multi-tenancy
      and allows a single platform for different services and users.
      The operator can optimize resources and costs to share a NFV
      platform for multiple customers (ex.  MVNO customers) and services
      (ex.  multiple APNs).

   o  [Faster Speed of Time to Market]: When an operator wants a new
      function to its network and services, the operator needs to
      negotiate appliance vendors to implement the new functions or to
      find alternative equipment supporting the new function.  It takes
      a longer time to convince the vendors, or to replace existing
      hardwares.  However, if functions can be implemented as a
      software, it is much faster to implement the functions on NFV.
      Even the operator may implement them and try the new functions by
      themselves.  Field trial is also getting easier because of no
      physical installation or replacement.  You may turn on a new

Matsushima & Wakikawa   Expires January 11, 2014                [Page 3]
Internet-Draft             Draft Specification                 July 2013

      function in NFV and observe how the new function behaves in your
      network.  NFV can save preparation time and tuning time of the new
      function.

   o  [Cost Optimization]: Last but not least, Cost is the most
      important motivation for operators to realize NFV.  Operators can
      remove many of proprietary appliances from its network and replace
      them with industry standard servers, switches and routers.  In
      addition, it is easy to scale up and down operator's services so
      that resources can be always tuned to the size of services.  In
      addition, operational costs led by any physical hardware such as
      power supply, maintenance, installation, construction and
      replacement can be minimized or even removed.  The network design
      can be simpler, because complicated functions could be handled by
      NFV.  That simple operation may enable automatic configurations
      and prevent unnecessary trouble-shooting.  As a result, CAPEX and
      OPEX can be always optimized and lowered.

2.  Motivations and Requirements, - Why IETF? -

2.1.  Motivations

   What is a role of IETF to realize vEPC in the future?  IETF is not
   the right place to discuss, for instance, how to run MME on
   hypervisor.  An important IETF activity must be to decouple the
   control- and user- planes of mobility protocols used in EPC.  In
   doing so, NFV-enabled solutions can be easily designed and
   implemented with interoperability across multiple vendors and
   platforms.  Otherwise, NFV solutions can be easily fragmented due to
   many proprietary solutions for the protocol separations.  As stated
   in [NFV-WHITEPAPER], interoperability is highly important.

   In the past, IETF has developed tunnel based mechanisms for mobile
   nodes such as Mobile IPv6 [RFC6275][RFC5555], Proxy Mobile IPv6
   [RFC5213][RFC5844] and NEMO [RFC3963].  Similarly, 3GPP has developed
   tunnel protocols called GPRS Tunneling Protocol (GTP).  These tunnel-
   based protocols establish a data path for a mobile node between the
   mobile node and an anchor point (s).  There is a case where an access
   router terminates a tunnel instead of a mobile node (ex.  Proxy
   Mobile IP).  In 3GPP, a tunnel is established between SGW and PGW per
   a mobile node by either Proxy Mobile IPv6 or GTP.  The control and
   the user planes of these mobility protocols are tightly related and
   cannot be decoupled.  The signaling like Binding Update and user's
   packets are routed along a same path in EPC.  It might be necessary
   to extend these mobility protocols for the user- and control- planes
   separation.  The protocol separation of Mobile IP is discussed in
   [I-D.yokota-dmm-scenario].

Matsushima & Wakikawa   Expires January 11, 2014                [Page 4]
Internet-Draft             Draft Specification                 July 2013

   Alternatively, if vEPC was realized, we should have an opportunity to
   re-visit the basic architecture of mobility system.  Instead of
   tunneling packets on today's EPC, why can't we just route packets to
   a mobile node?  Since a role of the user plane is "routing", BGP and
   other routing protocols could be used to forward UE's traffic.  This
   document introduces a BGP-based solution.  Software Defined
   Networking (SDN) can be an alternative solution.  Open Flow and other
   relevant protocols can setup the forward path dynamically according
   to UE's states available in the control plane.

   We have to remember that there is a good reason of adapting tunneling
   in Mobile IP based solutions, that is global mobility.  A mobile node
   should be able to move anywhere on the Internet and be reachable from
   anyone on the Internet.  There were routing based global mobility
   solutions like Boeing global mobility [Boeing-BGP] and WINMO
   [RFC6301].  In these proposals, BGP was used to propagate forwarding
   information of mobile nodes to the Internet.  Whenever a mobile node
   changes its point of attachment, the route must be updated.  Due to
   scalability and stability issues of the Internet, this solution was
   not recommended by IETF [Boeing-BGP].  However, as Boeing showed, it
   is doable to support global mobility by using BGP routing update.  If
   scalability is not your concern, a routing based approach becomes a
   candidate of the mobility solution.

   While global mobility is important, the "reality" is that your cell
   phones (i.e. UE/mobile node) are moving just within an operator's
   network and fully controlled in your local EPC.  If mobility is
   limited within an operator, we believe a routing based approach is
   feasible and practical for today's mobile system.  Instead of
   dedicated proprietary equipment like SGW and PGW to manage a tunnel
   path for a mobile node, multiple industry standard routers and
   switches are configured in the user plane.  These switches and
   routers receive mobile nodes' forwarding information from the control
   plane of vEPC by routing update.

2.2.  Requirements

   Requirements of our stateless user plane for vEPC are followings.

   NFV Support
           The future EPC architecture must support NFV capability.  The
           control plane of EPC operated on NFV framework is named
           "virtualized EPC (vEPC)" in this document.  The control plane
           of vEPC should keep backward compatibility with the today's
           EPC's control plane.  It means this document doesn't modify
           the control plane at all.  It only assumes software-based
           MME, SGW, and PGW run on hypervisor.

Matsushima & Wakikawa   Expires January 11, 2014                [Page 5]
Internet-Draft             Draft Specification                 July 2013

   Separation of Control- and User-     Planes
           Due to tight relationship of the control- and user- planes in
           today's EPC, resource increase is always provisioned to both
           planes at once.  It prevents flexible resource arrangement
           and introduces high capital investment and over-provisioned
           resources to one of planes.  If NFV is deployed, it is
           expected that computing resources can be independently
           allocated to the control planes of the vEPC in a flexible
           manner.

                                      _+---+_ _ _+---+_ _ _
      EPC                            / | S |     | P |    /
      Control-plane                 /  | G |     | G |   /
                                   /_ _| W |_ _ _| W |__/
                  +---+   +---+        |   |     |   |
                  |   |   | e |        |   |     |   |
                  | U |   | N |       _|   |_ _ _|   |_ _ _
      EPC         | E |   | B |      / |   |     |   |    /
      User-plane  +---+   +---+     /  +---+     +---+   /
                                   /_ _ _Existing EPC_ _/

                                      _+---+_ _ _+---+_ _ _
      vEPC                           / | S |     | P |    /
      Control-plane                 /  | G |     | G |   /
                                   /_ _| W |_ _ _| W |__/
                                       |   |     |   |
                  +---+   +---+        +---+     +---+
                  |   |   | e |          +----------+
                  | U |   | N |         _|IP Routing|_ _ _
      Stateless   | E |   | B |       /  | Network  |    /
      User-plane  +---+   +---+      /   +----------+   /
                                    /_ _ _ _ _ _ _ _ _ /

   NFV enabled EPC architecture

                                 Figure 1

           Figure 1 shows a possibility that the entities of EPC
           Control- plane are virtualized in generic cloud environment,
           however user packets won't go through those virtualized EPC
           nodes.  Decoupling User-plane from the Control-plane entities
           will be made virtualized Control-plane nodes relax hyper-
           visor data- path capacity requirements.  On the other hand,

Matsushima & Wakikawa   Expires January 11, 2014                [Page 6]
Internet-Draft             Draft Specification                 July 2013

           decoupled User-plane into IP routing network will be agnostic
           from sessions and bearers states, of which are generated and
           maintained in the Control-plane.  In terms of IP routing,
           forwarding packets through the networks is based on the
           destination address of the packets evaluated with network
           reachable information in the routing table that accommodated
           in the routing nodes.  To forward EPC User-plane packets
           correctly, those states must be indicated by network
           reachable information.

   Flat Design for Distributed      Operation
           Today's 3GPP architecture introduces PDN gateway (PGW) as a
           gateway to external networks like the Internet.  PGW manages
           all traffic from and to UEs and could be a bottleneck and
           single point of failure of network connectivity.  In
           addition, due to recent rapid traffic increase, it is
           important to perform traffic engineering and to offload
           traffic to multiple locations (ex.  SGW, PGW, eNodeB).  For
           enhancements of traffic engineering capability, more flat
           design with multiple gateways is expected so that traffic can
           be distributed to all these gateways.  There were proposals
           how to enable flat design to (Proxy) Mobile IP such as
           [I-D.wakikawa-mext-haha-interop2008] in IETF.  Distributed
           Mobility Management (DMM) Working Group has also discussed
           how to extend Mobile IP-based solutions to support traffic
           distribution in an optimal way by removing centrally deployed
           anchors that is like a Home Agent.

   Stateless in User Plane
           Ultimate goal of vEPC is to remove all mobility specific
           states from the forwarding nodes in the user-plane of vEPC.
           If we succeed in this, industry standard routers and switches
           can be used to forward mobile nodes traffic in the user plane
           of vEPC.  A mobile node's specific states are kept in both an
           IP header of the mobile node's packets and a routing entry of
           the mobile node.  The detail is described in Section 3.2

3.  Stateless user-plane architecture for virtualized EPC

   This section explains our solution that is the stateless user-plane
   architecture for vEPC.  This solution is basically a combination of
   existing protocols defined in IETF.  A minor extension might be
   needed but it should be easily addressed in IETF.  We first introduce
   our architecture and then protocol overview.

3.1.  Architecture Overview

Matsushima & Wakikawa   Expires January 11, 2014                [Page 7]
Internet-Draft             Draft Specification                 July 2013

   Figure 2 shows the user plane of the current EPC architecture.  A
   tunnel is established between SGW and PGW by either Proxy Mobile IP
   or GTP.  PGW is an anchor point of UE for incoming packets.  All the
   packet destined to UE is routed first to PGW.  The UE's packets are
   intercepted by PGW and tunneled to SGW.  SGW then forwards the packet
   to UE via access points (i.e. eNodeB) over Radio Area Network (RAN).

                                   .--.
                                 _(    `)_
                               _(         `)
              +---+    +---+  (   EPC-NW    )  +---+
      [UE]~~~~|RAN|====|SGW|===================|PGW| -> the Internet
              +---+    +---+  `--(______)---'  +---+
                         <--------Tunneling------>

      ~~~ Wireless Link
      === GTP Tunnel (PMIP tunnel)
      --- IP link

   User plane of the current EPC

                                 Figure 2

   Figure 3 is our proposed user plane of vEPC.  The control plane is
   not shown in this figure.

              [EPC-E anycast address]
                    |             .--.
                    |            _(    `)_
                    |  +---+   _(         `)
              +---+  \ |EPC|  (  IPv6 Core  )  +---+
      [UE]~~~~|RAN|====| -E|-( `    NW       )-|RTR| --> the Internet
              +---+    +---+  ( `         ) )  +---+
            (eNodeB)           `--(_____)--'
                         <--------Routing-------->

   User plane of vEPC

                                 Figure 3

   We introduce two new entities such as

Matsushima & Wakikawa   Expires January 11, 2014                [Page 8]
Internet-Draft             Draft Specification                 July 2013

   EPC Edge Router (EPC-E)
      EPC-E is located at the same place of today's SGW and terminates
      GTP tunnel established with eNodeB (RAN).  EPC-E supports the user
      plane functions of SGW and PGW.  EPC-E is configured an anycast
      address to the network interface facing to eNodeB.  The eNodeB
      establishes a GTP tunnel per UE with this anycast address.  Thanks
      for anycast address, UE's traffic forwarded by eNodeB is always
      routed to the closest EPC-E of UE.  EPC-E is a router and
      maintains routing information of every UE that is notified by the
      control plane.  Detail of routing mechanism can be found in
      Section 3.4.

   Router (RTR)
      It is a regular IP router.  The control plane of vEPC distributes
      routing information of every UE by a routing protocol like BGP.
      Therefore any additional protocols other than routing protocols
      are not needed for RTR.  Multiple RTRs can be configured anywhere
      in the user plane of vEPC.  RTRs announce UE's routing information
      to the external network (ex.  The Internet).

   As you see in Figure 3, we omit a tunneling mechanism originally
   established between SGW and PGW for routing UE's packets.  By
   removing this tunnel, UE's packets are forwarded to and from the
   Internet according to routing tables on routers in the core network.
   Note that, although we remove the tunnel for UE's traffic, the
   control-plane signaling stays same.  If Proxy Mobile IP is used for
   this tunnel, Proxy Binding Update and Acknowledgment are exchanged
   between PGW and SGW that are managed by NFV on servers/hyper-visor.
   Instead of a tunnel setup, states created by Proxy Mobile IP are
   distributed to all routing entities (EPC-E and RTR) by a routing
   protocol.  From the user plane point of view, these states are just
   seen as routing entries.  EPC-E and RTR are not involved in any
   signaling of the control plane.  The control plane just injects
   routing information to EPC-E and RTR to setup routing paths to and
   from UEs.

   Although this architecture just uses IPv6 core network, it supports
   both IPv4 and IPv6 packets.  The detailed operation of IPv4 support
   will be discussed in Section 3.5.

   A mobile node can still use a global mobility solution like Mobile IP
   when it moves out an operator's network.  We should keep backward
   compatibility for the control plane of EPC and should be able to fall
   back to the tunneling based packet forwarding for non-operator's
   network and roaming scenarios.

3.2.  Protocol Overview

Matsushima & Wakikawa   Expires January 11, 2014                [Page 9]
Internet-Draft             Draft Specification                 July 2013

   This section gives an example of protocols used for vEPC.  Figure 5
   is the procedure of the PDN connection setup in vEPC.  This figure is
   copied from the section 3 of [RFC6459].  All the steps from (1) to
   (13) are same as the original except for NFV-based MME, SGW, PGW,
   HSS, and AAA.

   The vEPC introduces two new steps, (14) and (15), to setup paths in
   the user-plane after finishing all the signaling on the control-
   plane. (16) and (17) are the steps to assign IP address to the mobile
   node.

   In (14), vEPC advertises a routing information of UE whose next hop
   is set to GTP tunnel between eNodeB and UE in RAN.  To do this, we
   will introduce two existing solutions as the key to build vEPC and
   also show potential modification required for vEPC.

   o  "Stateless IPv6 Prefix Delegation for IPv6 enabled networks"
      [I-D.savolainen-stateless-pd] can be used as the algorithm to
      generate UE's prefix.

   In the case of an UE's prefix length is equal, or shorter than /64,
   the generated prefix is consisted as follow:

           |          n bits           | o bits (=< 16)|64-n-o bits|
           +---------------------------+---------------+-----------+
           |        PDN Prefix         |      TEID     | subnet ID |
           +---------------------------+---------------+-----------+
           <---------------------------64bits---------------------->

   Stateless-pd Prefix

                                 Figure 4

   Each PDN is assumed to have single or several prefixes (called PDN
   prefix) used to generate UE's address.  Followed by the PDN prefix in
   Figure 4, there is 16-bit TEID assigned for a UE's session at SGW on
   the control plane.  TEID is 16 bits identifier in GTP header to
   distinguish each bearer.  The remaining bits are filled by subnet ID.
   The prefix is allocated per UE and used for address assignment by
   SLAAC or DHCPv6.

   o  [I-D.vandevelde-idr-remote-next-hop] can be used to carry end
      points information of GTP tunnel over BGP.

   In order to distribute a route entry of the UE's generated prefix of
   UE from vEPC to all EPC-E, BGP is one of suitable routing protocol.

Matsushima & Wakikawa   Expires January 11, 2014               [Page 10]
Internet-Draft             Draft Specification                 July 2013

   [I-D.vandevelde-idr-remote-next-hop] enables BGP to carry tunnel
   information as Network Layer Reachability Information (NLRI) within
   BGP route.  In our document, we use GTP tunnel endpoint information
   as a next-hop of the route entry of UE's prefix.  The GTP information
   is a combination of SI-U address and TEID of UE's attaching eNodeB.

   BGP has already been extended for BGP speakers to advertise tunneling
   information to its peers [RFC5512], but [RFC5512] does not support
   GTP as a tunnel type.  Moreover, it assumes only a single tunnel
   between a pair of BGP speakers although there are multiple tunnels
   between RAN and vEPC in cellular system.  To keep compatibility to
   RAN architecture, it is not possible for eNodeB to act as a BGP
   speaker.  Some mechanism will thus be required to advertise and to
   reflect tunnel endpoint routes to EPC-E instead of eNodeB.
   [I-D.vandevelde-idr-remote-next-hop] will be dealt with these issues.

   In step (15), EPC-E can aggregate multiple UE's prefixes into less
   BGP routes as a part of normal routing operation within operator's
   network.

   When tunnel endpoint is updated by an UE hand-over between eNode,
   vEPC must refresh the route of UE with updated tunnel endpoint as a
   remote next-hop.  The updated route should be immediately advertised
   to EPC-Es.  In the case of UE detachment, vEPC simply removes the
   detached UE route.

Matsushima & Wakikawa   Expires January 11, 2014               [Page 11]
Internet-Draft             Draft Specification                 July 2013

                         vEPC(MME/SGW
      UE        eNodeB   PGW/HSS/AAA)   EPC-E        RTR
      |           |           |           |           |
      |---------->|(1)        |           |           |
      |           |---------->|(1) MME    |           |
      |/---------------------\|           |           |
      |  Authentication       |(2) AAA    |           |
      |\---------------------/|           |           |
      |           |           |--+SGW/PGW |           |
      |           |           |  |(3)(4)  |(4)SGW's TEID is configured
      |           |           |<-+        |   and notified to MME
      |           |<----------|(5) MME    |           |
      |/---------\|           |           |           |
      | RB setup  |(6)        |           |           |
      |\---------/|           |           |           |
      |           |---------->|(7) MME    |           |
      |---------->|(8)        |           |           |
      |           |---------->|(9) MME    |(9)eNodeB TEID is configured
      |           |           |           |   and sent to MME
      |= = = = = = = Uplink Data = = = = =>= = = = ==>|(10)
      |       The uplink is not yet built here        |
      |           |           |           |           |
      |           |           |--+SGW/MME |           |
      |           |           |  |(11,12) |           |
      |           |           |<-+        |           |
      |<= = = = = = Downlink Data = = = = <= = = = = =|(13)
      |       The downlink is not yet built here      |
      |           |           |           |           |
      |           |           |---------->|(14) Route Update
      |           |           |  [Dst: UE-prefix,
      |           |           |   NxtHop: S1-U addr and TEID of enodeB
      |           |           |           |           |
      |           |           |           |---------->|(15)Route update
      |           |           |           |[Dst: UE-prefix,
      |           |           |           | NxtHop: EPC-E address]
      |           |           |           |           |
      |---------RS or DHCP Request------->|(16)       |
      |<--------RA or DHCP Reply----------|(17)       |

   Extended PDN Connection Setup Procedure (copied Figure 8 of RFC6459)

                                 Figure 5

   UE requests an IPv6 prefix for its address assignment in the step
   (16).  In our architecture, instead of PDN-GW, EPC-E is responsible
   to allocate an IPv6 prefix to UE.

Matsushima & Wakikawa   Expires January 11, 2014               [Page 12]
Internet-Draft             Draft Specification                 July 2013

   When EPC-E receives a packet of either Router Solicitation or DHCPv6
   request message, it looks up the remote next-hop field of its routing
   information base (RIB) with the source IP address and the TEID of the
   received packet.  A route entry matched for this search is the route
   entry created for the requesting UE.  Therefore, EPC-E simply uses
   the destination prefix of the route entry as an assigned UE's prefix.
   In (17), EPC-E simply returns the prefix to UE by either Router
   Advertisement or DHCPv6 reply message.  UE now creates an configures
   an address(es) from the received prefix.  It is important to
   highlight that UE can obtain the same prefix information from any
   EPC-E all the time because the same UE's route information is
   available on all the EPC-E.

3.3.  Control-plane awareness of stateless user-plane

   Control-plane nodes in vEPC must be aware that anycast address
   assigned to EPC-E is the S1-U address of SGW.  They must use the
   anycast address for any control-plane signaling packets on vEPC.  By
   doing this, packets from RAN are correctly forwarded to an
   appropriate EPC-E. Due to anycast nature, every SGW have the same
   address as the S1-U address.  It means there is no hand-off procedure
   between SGWs because all eNodeB in the RAN send packets to the
   anycast address are covered by all EPC-E of which sharing same routes
   set.

3.4.  Routing mechanism

   Figure 6 shows a packet forwarding mechanism of our stateless user
   plane.  As an example, there are four eNodeB (illustrated as eNB-x) ,
   three EPC-Edge routers(shown as EPC-Ex) and two routers (RTRx) in
   Figure 6.  UE is first connected to eNB-C and then moves to eNB-D.
   The UE at the new location is illustrated as UE'.  Routing entry for
   UE is also illustrated at the right side in Figure 6.

   EPC-E has two interfaces facing either RAN or CORE networks.  An
   anycast address (shown as X) is configured to the interface facing
   RAN of all EPC-E.  EPC-E assigns an individual IPv6 address to
   another interface (illustrated "a" to "d" in the figure).  It is
   important to mention that the anycast address X can be treated as the
   SGW's S1-U address.

   Since RTRs are a gateway to the Internet, they advertise routes of an
   operator's prefix to the Internet.  After one of RTR receives a
   packet of UE from the Internet, it needs to routing it to UE in the
   user plane.  RTR has a simple routing entry for PDN prefix whose next
   hop points to the EPC-E.  One of RTR (let's say RTR2 in this case)
   looks up a routing table with UE's address and matched it with a
   routing entry of PDN prefix.  Since multiple EPC-Es advertise a route

Matsushima & Wakikawa   Expires January 11, 2014               [Page 13]
Internet-Draft             Draft Specification                 July 2013

   for the same PDN prefix, RTR2 should forward the packet to one of
   EPC-E according to the routing entry.  This routing is known as hot-
   potato routing.  In this example, the RTR2 uses EPC-E2-b as a nexthop
   of PDN prefix.

   When the UE's packet is arrived at EPC-E2, EPC-E2 needs to forwards
   them to UE via eNodeB to which UE is connecting by using GTP tunnel.
   For this operation, EPC-E2 has a routing entry that destination is
   UE's prefix and that next hop points to GTP tunnel between eNB-C and
   the EPC-Es.  In order to identify the GTP tunnel for UE, EPC-E needs
   S1-U address and Tunnel Endpoint ID (TEID) of eNB-C that is eNB-C-3
   in Figure 6.  The eNB-C TEID for UE is illustrated as TEID[eNB-C].
   The SGW assigned TEID is utilized to generate the UE's prefix as we
   explained in Section 3.2.  These TEID are assigned per UE.  The TEID
   and S1-U address of eNodeB are retrieved from the next hop field of
   the routing entry of the mobile node.  By using the GTP information,
   every EPC-E can now forward the UE's packets to right eNodeB.

   Routing outgoing packets from UE is much simpler.  The packets from
   UE are always routed to the closest EPC-E to UE because of anycast
   routing.  In Figure 6, when UE sends a packet to a destination, the
   packet is reached to eNB-C and tunneled to EPC-E's anycast address.
   The GTP-tunneled packet is routed to the closest EPC-E that is EPC-E2
   in this case.  The packet is decapsulated by EPC-E2 and then
   forwarded to one of RTR according to the routing table.  Since the
   decapsulated packet is regular IPv6 packet, no extra control other
   than routing is necessary.

   When UE moves to a new location (UE'), it updates its location on the
   control plane.  After signaling completion for location update, vEPC
   needs to update the UE's routing entry of all EPC-E so that vEPC
   advertises updated route with new location to all EPC-Es by a routing
   protocol.  The routing entry should be updated with the new eNodeB's
   address that is eNB-D-4.  During handover, there might be some
   traffic arriving to the older eNodeB (eNB-C).  These packets can be
   re-routed to the new eNodeB (eNB-D) via X2-U interface in RAN.

   The UE's address isn't changed when UE changes its attachment.  In
   our scenario, SGW run on hypervisor and is independent from network
   topology.  Therefore, logically we don't have handover across
   different SGWs.  UE can stay connected with the same SGW all the time
   and can keep using the same TEID after handover.  Thus, UE's address
   is unchanged even after handover.

Matsushima & Wakikawa   Expires January 11, 2014               [Page 14]
Internet-Draft             Draft Specification                 July 2013

             +--------------------+
             | Correspondent Node |
             +--------------------+
                       |
                      .--.        [Route on the Internet]
                    _(.   `)      Destination: Operator's Prefix
                  _(        `)_   NextHop: RTR-1/2
                 (  Internet   `)
                ( `  .        )  )
                 `--(_______)---'
                   /        \     [Route at RTR]
                +----+    +----+  Destination: PDN Prefix
                |RTR1|    |RTR2|  NextHop: EPC-E2-b
                +----+    +----+
                    \ .--. /
                    _( IP `.
                   ( CORE NW )
                ___( `  .  )  )__
               /   `--(___.--'   \
              /         |         \
             |a         |b         |c   [Route at EPC-E]
         +---+--+   +---+--+   +---+--+ Destination: UE's Prefix *1
         |EPC-E1|   |EPC-E2|   |EPC-E3| NextHop: GTP tunnel (eNB-C) *2
         +---+--+   +---+--+   +---+--+
             |X         |X         |X
              \       .--.        /
               \_____( RAN `.____/
                   (  ACCESS )
              ____( `  NW  )  )_______
            /     `--(___.--'         \
           /        |         |        \
          |1        |2        |3        |4
       +--+--+   +--+--+   +--+--+   +--+--+
       |eNB-A|   |eNB-B|   |eNB-C|   |eNB-D|
       +--+--+   +--+--+   +--+--+   +--+--+
          :         :         :         :
                              UE  -->   UE'
    *1 TEID used at EPC-E for the UE is included in this UE's prefix.
       see Figure 4.
    *2 GTP tunnel state is stored in the next hop field. The state
       information is the combination of eNB-C S1 address that is
       eNB-C-3 and TEID(eNB-C) assigned for the UE.

                        Routing Mechanism Overview

                                 Figure 6

Matsushima & Wakikawa   Expires January 11, 2014               [Page 15]
Internet-Draft             Draft Specification                 July 2013

3.5.  IPv4 Support

   Recent IPv6 transition mechanisms enable IPv6-only network to forward
   IPv4 packet with encapsulation or translation techniques.  By using
   one of mechanisms, we can use IPv6 for our stateless user-plane
   network for transporting both IPv4 and IPv6 packets.  Figure 7 shows
   available solutions of IPv4 support for each bearer type to deal with
   that requirement.

                         Bearer     UE         EPC-E     Gateway
                          type    function    function   function
                     --------------------------------------------
                          IPv4       -          B4        AFTR
                          IPv4       -          CLAT      PLAT
                          IPv6     MAP-CE        -       MAP-BR
                          IPv6       B4          -        AFTR
                          IPv6      CLAT         -        PLAT

                 Solutions and functions for IPv4 support

                                 Figure 7

   In the case of a UE only support IPv4 bearer, B4 function of DS-Lite
   [RFC6333] or CLAT function of 464XLAT [RFC6877] may be implemented in
   a EPC-E. Both functions are stateless therefore EPC-E isn't required
   to maintain any tunneling or translation state.

   Figure 8 shows how to support IPv4 on IPv6 core network in our vEPC.
   Instead of using RqTR as a gateway to the Internet, DS-LITE AFTR or
   464XLAT PLAT is installed as a gateway to the IPv4 Internet.

                                       .--.
                          EPC-E      _(    `)_     Gateway
                          +----+   _(         `)   +------+
                          | B4 |  (  IPv6 Core  )  | AFTR |
         [UE]==IPv4-only==| or |-( `    NW       )-|  or  | --> Internet
                 bearer   |CLAT|  ( `         ) )  | PLAT |     (IPv4)
                          +----+   `--(_____)--'   +------+
                            <-----IPv4 over IPv6----->
                                        or
                                 v4v6 translation

Matsushima & Wakikawa   Expires January 11, 2014               [Page 16]
Internet-Draft             Draft Specification                 July 2013

   IPv4 User plane of vEPC

                                 Figure 8

   If UE supports IPv6 capable bearer, IPv6 transition function may be
   implemented in the UE such as MAP-CE [I-D.ietf-softwire-map], B4 or
   CLAT.  That means an EPC-E receives IPv6 packets from UE in this case
   so that the EPC-E does not need to be involved in the part of IPv4
   support functions.

4.  IANA Considerations

   This memo includes no request to IANA.

5.  Security Considerations

   There are no security considerations specific to this document at
   this moment.

6.  References

6.1.  Normative References

   [I-D.savolainen-stateless-pd]
              Savolainen, T. and J. Korhonen, "Stateless IPv6 Prefix
              Delegation for IPv6 enabled networks", draft-savolainen-
              stateless-pd-01 (work in progress), February 2010.

   [I-D.vandevelde-idr-remote-next-hop]
              Velde, G., Patel, K., Rao, D., Raszuk, R., and R. Bush,
              "BGP Remote-Next-Hop", draft-vandevelde-idr-remote-next-
              hop-03 (work in progress), October 2012.

   [RFC5512]  Mohapatra, P. and E. Rosen, "The BGP Encapsulation
              Subsequent Address Family Identifier (SAFI) and the BGP
              Tunnel Encapsulation Attribute", RFC 5512, April 2009.

6.2.  Informative References

   [Boeing-BGP]
              Andrew, ., "Global IP Network Mobility using Border
              Gateway Protocol (BGP)", IAB Plenary IAB Plenary of IETF
              62nd, March 2005.

   [I-D.ietf-softwire-map]

Matsushima & Wakikawa   Expires January 11, 2014               [Page 17]
Internet-Draft             Draft Specification                 July 2013

              Troan, O., Dec, W., Li, X., Bao, C., Matsushima, S.,
              Murakami, T., and T. Taylor, "Mapping of Address and Port
              with Encapsulation (MAP)", draft-ietf-softwire-map-07
              (work in progress), May 2013.

   [I-D.wakikawa-mext-haha-interop2008]
              Wakikawa, R., Shima, K., and N. Shigechika, "The Global
              HAHA Operation at the Interop Tokyo 2008", draft-wakikawa-
              mext-haha-interop2008-00 (work in progress), July 2008.

   [I-D.yokota-dmm-scenario]
              Yokota, H., Seite, P., Demaria, E., and Z. Cao, "Use case
              scenarios for Distributed Mobility Management", draft-
              yokota-dmm-scenario-00 (work in progress), October 2010.

   [NFV-WHITEPAPER]
              Network Operators, ., "Network Functions Virtualization,
              An Introduction, Benefits, Enablers, Challenges and Call
              for Action", SDN and OpenFlow SDN and OpenFlow World
              Congress, October 2012.

   [RFC3963]  Devarapalli, V., Wakikawa, R., Petrescu, A., and P.
              Thubert, "Network Mobility (NEMO) Basic Support Protocol",
              RFC 3963, January 2005.

   [RFC5213]  Gundavelli, S., Leung, K., Devarapalli, V., Chowdhury, K.,
              and B. Patil, "Proxy Mobile IPv6", RFC 5213, August 2008.

   [RFC5555]  Soliman, H., "Mobile IPv6 Support for Dual Stack Hosts and
              Routers", RFC 5555, June 2009.

   [RFC5844]  Wakikawa, R. and S. Gundavelli, "IPv4 Support for Proxy
              Mobile IPv6", RFC 5844, May 2010.

   [RFC6275]  Perkins, C., Johnson, D., and J. Arkko, "Mobility Support
              in IPv6", RFC 6275, July 2011.

   [RFC6301]  Zhu, Z., Wakikawa, R., and L. Zhang, "A Survey of Mobility
              Support in the Internet", RFC 6301, July 2011.

   [RFC6333]  Durand, A., Droms, R., Woodyatt, J., and Y. Lee, "Dual-
              Stack Lite Broadband Deployments Following IPv4
              Exhaustion", RFC 6333, August 2011.

   [RFC6459]  Korhonen, J., Soininen, J., Patil, B., Savolainen, T.,
              Bajko, G., and K. Iisakkila, "IPv6 in 3rd Generation
              Partnership Project (3GPP) Evolved Packet System (EPS)",
              RFC 6459, January 2012.

Matsushima & Wakikawa   Expires January 11, 2014               [Page 18]
Internet-Draft             Draft Specification                 July 2013

   [RFC6877]  Mawatari, M., Kawashima, M., and C. Byrne, "464XLAT:
              Combination of Stateful and Stateless Translation", RFC
              6877, April 2013.

Authors' Addresses

   Satoru Matsushima
   Softbank Telecom
   1-9-1,Higashi-Shimbashi,Minato-Ku
   Tokyo  105-7322
   Japan

   Email: satoru.matsushima@g.softbank.co.jp

   Ryuji Wakikawa
   Softbank Mobile
   1-9-1,Higashi-Shimbashi,Minato-Ku
   Tokyo  105-7322
   Japan

   Email: ryuji.wakikawa@gmail.com

Matsushima & Wakikawa   Expires January 11, 2014               [Page 19]