ALTO Extension: Path Vector Cost Mode
draft-ietf-alto-path-vector-01
The information below is for an old version of the document.
| Document | Type |
This is an older version of an Internet-Draft that was ultimately published as RFC 9275.
|
|
|---|---|---|---|
| Authors | Greg M. Bernstein , Shiwei Dawn Chen , Kai Gao , Young Lee , Wendy Roome , Michael Scharf , Y. Richard Yang , Jingxuan Zhang | ||
| Last updated | 2017-07-03 | ||
| Replaces | draft-yang-alto-path-vector | ||
| RFC stream | Internet Engineering Task Force (IETF) | ||
| Formats | |||
| Reviews |
GENART Last Call review
(of
-17)
by Suresh Krishnan
Ready w/nits
ARTART Last Call review
(of
-16)
by Paul Kyzivat
Ready w/issues
|
||
| Additional resources | Mailing list discussion | ||
| Stream | WG state | WG Document | |
| Document shepherd | (None) | ||
| IESG | IESG state | Became RFC 9275 (Experimental) | |
| Consensus boilerplate | Unknown | ||
| Telechat date | (None) | ||
| Responsible AD | (None) | ||
| Send notices to | (None) |
draft-ietf-alto-path-vector-01
ALTO WG G. Bernstein
Internet-Draft Grotto Networking
Intended status: Standards Track S. Chen
Expires: January 4, 2018 Tongji University
K. Gao
Tsinghua University
Y. Lee
Huawei
W. Roome
M. Scharf
Nokia
Y. Yang
Yale University
J. Zhang
Tongji University
July 3, 2017
ALTO Extension: Path Vector Cost Mode
draft-ietf-alto-path-vector-01.txt
Abstract
The Application-Layer Traffic Optimization (ALTO) protocol [RFC7285]
has defined several resources and services to provide clients with
basic network information. However, the base ALTO protocol and
latest extensions only provide end-to-end metrics, which are
insufficient to satisfy the demands of solving more complex network
optimization problems. This document introduces an extension to the
base ALTO protocol, namely the path-vector extension, which allows
ALTO clients to query information such as capacity regions for a
given set of flows. A non-normative example called multi-flow
scheduling is presented to illustrate the limitations of existing
ALTO (endpoint) cost maps. After that, details of the extension are
defined.
Requirements Language
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in RFC 2119 [RFC2119].
Status of This Memo
This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79.
Bernstein, et al. Expires January 4, 2018 [Page 1]
Internet-Draft ALTO Extension: Path Vector July 2017
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet-
Drafts is at http://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
This Internet-Draft will expire on January 4, 2018.
Copyright Notice
Copyright (c) 2017 IETF Trust and the persons identified as the
document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must
include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as
described in the Simplified BSD License.
Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 4
2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 5
3. Use Case: Capacity Region for Multi-Flow Scheduling . . . . . 5
4. Overview . . . . . . . . . . . . . . . . . . . . . . . . . . 7
4.1. Path Vector . . . . . . . . . . . . . . . . . . . . . . . 7
4.2. Cost Type Extension . . . . . . . . . . . . . . . . . . . 8
4.3. Abstract Network Element Property Map . . . . . . . . . . 8
4.4. New Media Type: multipart/related . . . . . . . . . . . . 8
5. Path-Vector Extension: Basic Data Types . . . . . . . . . . . 9
5.1. Cost Type . . . . . . . . . . . . . . . . . . . . . . . . 9
5.1.1. Cost Metric . . . . . . . . . . . . . . . . . . . . . 9
5.1.2. Cost Mode . . . . . . . . . . . . . . . . . . . . . . 10
5.2. ANE Domain . . . . . . . . . . . . . . . . . . . . . . . 10
5.2.1. Domain Name . . . . . . . . . . . . . . . . . . . . . 10
5.2.2. Domain-Specific Entity Addresses . . . . . . . . . . 10
5.3. Abstract Network Element Name . . . . . . . . . . . . . . 11
5.4. Version Tag . . . . . . . . . . . . . . . . . . . . . . . 11
6. Path-Vector Extension: Services . . . . . . . . . . . . . . . 11
6.1. IRD Extensions . . . . . . . . . . . . . . . . . . . . . 11
Bernstein, et al. Expires January 4, 2018 [Page 2]
Internet-Draft ALTO Extension: Path Vector July 2017
6.2. Cost Map Extensions . . . . . . . . . . . . . . . . . . . 12
6.2.1. Media Type . . . . . . . . . . . . . . . . . . . . . 12
6.2.2. Capabilities . . . . . . . . . . . . . . . . . . . . 12
6.2.3. Property-map . . . . . . . . . . . . . . . . . . . . 12
6.2.4. Response . . . . . . . . . . . . . . . . . . . . . . 13
6.3. Filtered Cost Map Extensions . . . . . . . . . . . . . . 13
6.3.1. Media Type . . . . . . . . . . . . . . . . . . . . . 13
6.3.2. Capabilities . . . . . . . . . . . . . . . . . . . . 13
6.3.3. Property-map . . . . . . . . . . . . . . . . . . . . 14
6.3.4. Accept Input Parameters . . . . . . . . . . . . . . . 14
6.3.5. Response . . . . . . . . . . . . . . . . . . . . . . 14
6.4. Endpoint Cost Service Extensions . . . . . . . . . . . . 14
6.4.1. Media Type . . . . . . . . . . . . . . . . . . . . . 15
6.4.2. Capabilities . . . . . . . . . . . . . . . . . . . . 15
6.4.3. Property-map . . . . . . . . . . . . . . . . . . . . 15
6.4.4. Accept Input Parameters . . . . . . . . . . . . . . . 15
6.4.5. Response . . . . . . . . . . . . . . . . . . . . . . 15
7. Examples . . . . . . . . . . . . . . . . . . . . . . . . . . 16
7.1. Workflow . . . . . . . . . . . . . . . . . . . . . . . . 16
7.2. Information Resource Directory Example . . . . . . . . . 17
7.3. Single Query Example # 1 . . . . . . . . . . . . . . . . 18
7.4. Single Query Example # 2 . . . . . . . . . . . . . . . . 20
7.5. Multiple Queries Example . . . . . . . . . . . . . . . . 21
7.5.1. Endpoint Cost Service Example . . . . . . . . . . . . 21
7.5.2. Abstract Network Element Property Map Example . . . . 23
8. Compatibility . . . . . . . . . . . . . . . . . . . . . . . . 23
8.1. Compatibility with Legacy ALTO Clients/Servers . . . . . 23
8.2. Compatibility with Multi-Cost Extensions . . . . . . . . 23
8.3. Compatibility with Incremental Update . . . . . . . . . . 24
9. Design Decisions and Discussions . . . . . . . . . . . . . . 24
9.1. Provide More General Calendar Extension . . . . . . . . . 24
10. Security Considerations . . . . . . . . . . . . . . . . . . . 24
10.1. Privacy Concerns . . . . . . . . . . . . . . . . . . . . 24
10.2. Resource Consumption on ALTO Servers . . . . . . . . . . 25
11. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 25
11.1. ALTO Cost Mode Registry . . . . . . . . . . . . . . . . 25
11.2. ALTO Cost Metric Registry . . . . . . . . . . . . . . . 25
11.3. ALTO Entity Domain Registry . . . . . . . . . . . . . . 26
11.4. ALTO Network Element Property Type Registry . . . . . . 26
12. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 26
13. References . . . . . . . . . . . . . . . . . . . . . . . . . 26
13.1. Normative References . . . . . . . . . . . . . . . . . . 27
13.2. Informative References . . . . . . . . . . . . . . . . . 27
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 28
Bernstein, et al. Expires January 4, 2018 [Page 3]
Internet-Draft ALTO Extension: Path Vector July 2017
1. Introduction
The ALTO base protocol [RFC7285] is designed for exposing network
information through services such as the Network Map service and the
Cost Map service. These services use an extreme "single-node"
abstraction, which represents the whole network with a single node
and hosts with "endpoint groups" directly connected to the node.
Although the "single-node" abstraction works well in many settings,
it lacks the ability to support new emerging use cases, such as
inter-datacenter flow scheduling and scientific high-performance
computing data transfers. Specifically, the base ALTO protocol MUST
provide the following two functionalities:
o Providing information on shared bottlenecks: In the aforementioned
use cases, the volume of a single flow can reach 10s - 100s Gbps,
so that the network cannot treat the flows as independent like in
the base ALTO protocol. In this case, ALTO servers MUST be able
to provide information on shared bottlenecks to help applications
avoid congestion.
o Encapsulating multiple cost values in a single session: Some flow
scheduling problems take multiple metrics into consideration.
Making multiple queries introduces larger communication overhead,
and more importantly, out-of-sync data for different cost types.
Encapsulating multiple cost values in a single query and response
session reduces communication overhead and simplifies the
synchronization in use cases involving multiple cost types.
This draft aims to extend the base ALTO protocol to support these new
functionalities, with the path-vector extension. The path-vector
extension specifies how to encode the shared bottlenecks in a network
for a given set of flows with many design details driven by
effectiveness, performance and backward compatibility considerations.
The second functionality for simple cost types, such as those
introduced in the base protocol, is already addressed in a recent
extension, e.g. [I-D.ietf-alto-multi-cost]. However, the path-
vector extension in this document has introduced a new cost type
which complicates the situation. Thus, the multiple cost
encapsulation must still be taken into consideration.
The document is organized as follows. Section 3 gives an example of
flow scheduling and illustrates the limitations of the base ALTO
protocol in such a use case. Section 4 gives an overview of the
path-vector extension, before specifying the details of the extension
in Section 5 and Section 6. Section 7 presents several examples, and
Section 9 explains some design decisions. Section 8 discusses
Bernstein, et al. Expires January 4, 2018 [Page 4]
Internet-Draft ALTO Extension: Path Vector July 2017
compatibility issues with some other ALTO extensions. Section 10 and
Section 11 discusses about security and IANA considerations.
2. Terminology
This document uses the same terms as defined in [RFC7285],
[I-D.ietf-alto-multi-cost] and [I-D.roome-alto-unified-props] with
the following additional terms: Abstract Network Element, Abstract
Network Element Name, Abstract Network Element Property, Abstract
Network Element Property Map and Path Vector.
o Abstract Network Element (ANE): An abstract network element is an
abstraction of network components, it can be an aggregation of
links, middle boxes, Virtualized Network Function (VNF), or even a
sub-network. An abstract network element has two attributes:
abstract network element name and abstract network element
property, which are defined below.
o Abstract Network Element Name (ANEN): An abstract network element
name is an identifier which uniquely identifies an abstract
network element, as defined in Section 5.3.
o Abstract Network Element Property (ANEP): An abstract network
element property is a specific metric associated with a given
abstract network element, as introduced in Section 4.3. An
abstract network element CAN have several network element
properties.
o Abstract Network Element Property Map (ANEP Map): An abstract
network element property map is a Filtered Property Map defined in
[I-D.roome-alto-unified-props] which supports the "ane" domain in
its "domain-types" capability.
o Path Vector (PV): A path vector is an array of abstract network
elements, representing an abstract path between entities (PIDs or
endpoints).
3. Use Case: Capacity Region for Multi-Flow Scheduling
Consider the case that routing is given. Then what application-layer
traffic optimization will focus on is traffic scheduling among
application-layer paths. Specifically, assume that an application
has control over a set of flows F = {f_1, f_2, ..., f_|F|}. If
routing is given, what the application can control is x_1, x_2, ...,
x_|F|, where x_i is the amount of traffic for flow i. Let x = [x_1,
..., x_|F|] be the vector of the flow traffic amounts. Due to shared
links, feasible values of x where link capacities are not exceeded
can be a complex polytype.
Bernstein, et al. Expires January 4, 2018 [Page 5]
Internet-Draft ALTO Extension: Path Vector July 2017
Specifically, consider a network as shown in Figure 1. The network
has 7 switches (sw1 to sw7) forming a dumb-bell topology. Switches
sw1/sw3 provide access on one side, sw2/sw4 provide access on the
other side, and sw5-sw7 form the backbone. End hosts eh1 to eh4 are
connected to access switches sw1 to sw4 respectively. Assume that
the bandwidth of link eh1 -> sw1 and link sw1 -> sw5 are 150 Mbps,
and the bandwidth of the rest links are 100 Mbps.
+------+
| |
--+ sw6 +--
/ | | \
PID1 +-----+ / +------+ \ +-----+ PID2
eh1__| |_ / \ ____| |__eh2
| sw1 | \ +--|---+ +---|--+ / | sw2 |
+-----+ \ | | | |/ +-----+
\_| sw5 +---------+ sw7 |
PID3 +-----+ / | | | |\ +-----+ PID4
eh3__| |__/ +------+ +------+ \____| |__eh4
| sw3 | | sw4 |
+-----+ +-----+
Figure 1: Raw Network Topology.
The single-node ALTO topology abstraction of the network is shown in
Figure 2.
+----------------------+
{eh1} | | {eh2}
PID1 | | PID2
+------+ +------+
| |
| |
{eh3} | | {eh4}
PID3 | | PID4
+------+ +------+
| |
+----------------------+
Figure 2: Base Single-Node Topology Abstraction.
Consider an application overlay (e.g., a large data analysis system)
which needs to schedule the traffic among a set of end host source-
destination pairs, say eh1 -> eh2 and eh1 -> eh4. The application
can request a cost map providing end-to-end available bandwidth,
using 'availbw' as cost-metric and 'numerical' as cost-mode.
Bernstein, et al. Expires January 4, 2018 [Page 6]
Internet-Draft ALTO Extension: Path Vector July 2017
The application will receive from ALTO server that the bandwidth of
eh1 -> eh2 and eh1 -> eh4 are both 100 Mbps. But this information is
not enough. Consider the following two cases:
o Case 1: If eh1 -> eh2 uses the path eh1 -> sw1 -> sw5 -> sw6 ->
sw7 -> sw2 -> eh2 and eh1 -> eh4 uses path eh1 -> sw1 -> sw5 ->
sw7 -> sw4 -> eh4, then the application will obtain 150 Mbps.
o Case 2: If eh1 -> eh2 uses the path eh1 -> sw1 -> sw5 -> sw7 ->
sw2 -> eh2 and eh1 -> eh4 uses the path eh1 -> sw1 -> sw5 -> sw7
-> sw4 -> eh4, then the application will obtain only 100 Mbps.
To allow applications to distinguish the two aforementioned cases,
the network needs to provide more details. In particular, it needs
to provide the following new capabilities:
o The network needs to expose more detailed routing information to
show the shared bottlenecks.
o The network needs to provide the necessary abstraction to hide the
real topology information as possible.
The path-vector extension defined in this document will satisfy all
the requirements.
See [I-D.bernstein-alto-topo] for a survey of use-cases where
extended network topology information is needed.
4. Overview
This section presents a non-normative overview of the path-vector
extension. It assumes the readers are familiar with (Filtered) Cost
Map and Endpoint Cost Service defined in [RFC7285], their extensions
defined in [I-D.ietf-alto-multi-cost] and Filtered Property Map
defined in [I-D.roome-alto-unified-props].
4.1. Path Vector
A path vector is an array of abstract network elements, representing
an abstract path between entities (PIDs or endpoints). Each abstract
network element has two attributes: name and property. The abstract
network element names are encoded in cost maps and the abstract
network element properties are encoded in abstract network element
property maps.
Bernstein, et al. Expires January 4, 2018 [Page 7]
Internet-Draft ALTO Extension: Path Vector July 2017
4.2. Cost Type Extension
To provide abstract network element names of a path in cost maps,
each cost value is a list of abstract network element names.
However, as defined in Section 6.1.2 of [RFC7285], a cost mode is
either "numerical" or "ordinal", none of which can be used to present
a list.
This document specifies a new cost mode "array" and a new cost metric
"ane-path". The new cost mode "array" means each cost value in the
cost maps is a list. The new cost metric "ane-path" means each cost
value represents an abstract path consisting of abstract network
element names between two entities (PIDs or endpoints).
The new cost type follows the convention of the cost types in the
base protocol. For example:
+------------+--------------+---------------------------------------+
| cost mode | cost metric | meaning |
+------------+--------------+---------------------------------------+
| numerical | routingcost | a number representing the routing |
| | | cost |
| ordinal | hopcount | a ranking representing the hop count |
| array | ane-path | a list representing the ane path |
+------------+--------------+---------------------------------------+
Table 1: Cost Types and Their Meanings
4.3. Abstract Network Element Property Map
Given that Cost Map and Endpoint Cost service now provide the
abstract network element names along a flow path, ALTO clients can
learn that there exist bottlenecks between different flows. However,
only providing the abstract network element names without abstract
network element properties is not enough, because ALTO clients often
require the information on specific metric values like the link
capacity. This document adopts the property map defined in a recent
draft [I-D.roome-alto-unified-props] to encode the properties of
abstract network elements. A new domain "ane" is registered in the
property map. Each entity in the "ane" domain is an abstract network
element. The property map which supports "ane" domain is an Abstract
Network Element Property Map.
4.4. New Media Type: multipart/related
In the base ALTO protocol, ALTO servers use media types in the HTTP
header to indicate the type of the response. Typically one response
only contains a single media type, such as "application/alto-
Bernstein, et al. Expires January 4, 2018 [Page 8]
Internet-Draft ALTO Extension: Path Vector July 2017
costmap+json" or "application/alto-propmap+json". This has limited
the capability of ALTO servers to return multiple services in a
single response.
Thus, an ALTO client MUST make multiple queries to get the
information from services of different types. This has led to the
data synchronization problem between dependent ALTO services because
when making the second query, the result for the first query may have
already changed. The very same problem can happen to Network Map and
Cost Map resources. However, unlike Network Map and Cost Map which
are considered more stable, path vectors and the dependent abstract
network element property maps might change more frequently.
Instead of introducing a new media type to encapsulate multiple types
in a single response, this documents adopts the "multipart" media
type defined in [RFC2387]. Thus, a response can contain both the
path vector as a Cost Map (or Endpoint Cost Map) and the
corresponding abstract network element property map as a Property
Map. The media types of the path vector and the abstract network
element property map can still be retrieved from the response,
achieving consistency with the base ALTO protocol.
For backward compatibility, this extension also allows ALTO clients
to make multiple queries instead of encapsulating abstract network
element property map along with the path vector. Thus, each Cost Map
or Endpoint Cost Service with this extension MUST include a "prop-
map" in their capabilities to indicate where to retrieve the network
element properties. An additional field "query-id" MUST also be
added to the "vtag" field to uniquely identify a path vector query
session.
5. Path-Vector Extension: Basic Data Types
This section formally specifies the path-vector extension of some
basic data types.
5.1. Cost Type
This document extends the cost types defined in Section 6.1 of
[RFC7285] by introducing a new cost mode "array" and a new cost
metric "ane-path".
5.1.1. Cost Metric
This document specifies a new cost metric: "ane-path". It is of type
CostMetric as defined in Section 10.6 of [RFC7285]. The cost metric
"ane-path" MUST NOT be used when the cost mode is not "array" unless
it is explicitly specified by a future extension. Meanwhile, an ALTO
Bernstein, et al. Expires January 4, 2018 [Page 9]
Internet-Draft ALTO Extension: Path Vector July 2017
server with path-vector extension MUST support the cost metric "ane-
path".
Cost metric "ane-path": This cost metric MUST be encoded as the
JSONString "ane-path".
5.1.2. Cost Mode
This document extends the CostMode defined in Section 10.5 of
[RFC7285] with a new cost mode: "array". The extended CostMode is
encoded as a string and MUST have a value of either "numerical",
"ordinal" or "array" unless it is explicitly specified by a future
extension. In particular, this extension has specified that when the
cost metric is "ane-path", the cost value MUST be interpreted as a
JSONArray of Abstract Network Element Names (defined in Section 5.3).
An ALTO cost service MUST return a JSONArray of JSONValue when the
cost mode is "array" unless the interpretation is explicitly
specified by an ALTO extension.
Cost mode "array": This cost mode MUST be encoded as the JSONString
"array".
5.2. ANE Domain
This document specifies a new domain in addition to the ones in [I-
D.roome-alto-unified-props].
5.2.1. Domain Name
ane
5.2.2. Domain-Specific Entity Addresses
The entity address of ane domain is encoded as a JSON string. The
string MUST be no more than 64 characters, and it MUST NOT contain
characters other than US-ASCII alphanumeric characters
(U+0030-U+0039, U+0041-U+005A, and U+0061-U+007A), the hyphen ('-',
U+002D), the colon (':', U+003A), the at sign ('@', code point
U+0040), the low line ('_', U+005F), or the '.' separator (U+002E).
The '.' separator is reserved for future use and MUST NOT be used
unless specifically indicated in this document, or an extension
document.
Bernstein, et al. Expires January 4, 2018 [Page 10]
Internet-Draft ALTO Extension: Path Vector July 2017
5.3. Abstract Network Element Name
An Abstract Network Element Name MUST be encoded as an EntityAddr as
defined in Section 5.2.2. It MUST belong to the "ane" domain.
5.4. Version Tag
This document extends the VersionTag, previously defined in
Section 10.3 of [RFC7285] with an optional field "query-id". If an
ALTO cost service supports the path-vector extension, this field MUST
be included in the "vtag" field, and the "vtag" field MUST be
included in the "meta" field in the response in order to provide the
"query-id" information.
object {
ResourceID resource-id;
JSONString tag;
[JSONString query-id;]
} VersionTag;
resource-id, tag: As defined in Section 10.3 of [RFC7285].
query-id: A string used to uniquely identify the abstract network
element names in the response and correlate abstract network
element names with abstract network element properties. A "query-
id" MUST be encoded in the same format as defined in Section 10.1
of [RFC7285].
6. Path-Vector Extension: Services
This section extends IRDResourceEntry, Cost Map Service and Endpoint
Cost Service.
6.1. IRD Extensions
This document extends IRDResourceEntry defined in Section 9.2.2 of
[RFC7285] by introducing a new entry named "property-map", which
indicates where the specific properties of the abstract network
elements can be retrieved. The IRDResourceEntry object is extended
as follows:
Bernstein, et al. Expires January 4, 2018 [Page 11]
Internet-Draft ALTO Extension: Path Vector July 2017
object {
JSONString uri;
JSONString media-type;
[JSONString accepts;]
[Capabilities capabilities;]
[ResourceID uses<0..*>;]
[ResourceID property-map;]
} IRDResourceEntry;
uri, media-type, accepts, capabilities, uses: The same as defined in
Section 9.2.2 of [RFC7285].
property-map: A resource ID defined in the same IRD pointing to an
abstract network element property map as defined in Section 2.
6.2. Cost Map Extensions
This document extends the Cost Map defined in Section 11.2.3 of
[RFC7285].
The specifications for "HTTP method", "accept input parameters" and
"uses" are the same as defined in Section 11.2.3 of [RFC7285].
6.2.1. Media Type
The path vector extension now enables ALTO clients to receive
multiple services in a cost map response.
Specifically, if an ALTO client accepts "multipart/related",
"application/alto-costmap+json" and "application/alto-propmap+json"
at the same time, the ALTO server MUST use "multipart/related" as the
media type in the HTTP header.
6.2.2. Capabilities
If a service supports the path-vector extension, the "cost-type-
names" field MUST include a single cost type with "ane-path" as cost
metric and "array" as cost mode.
6.2.3. Property-map
If a service supports the path-vector extension, the "property-map"
field MUST be specified. This field is a resource ID of an abstract
network element property map where the abstract network element
properties are provided.
Bernstein, et al. Expires January 4, 2018 [Page 12]
Internet-Draft ALTO Extension: Path Vector July 2017
6.2.4. Response
If an ALTO client accepts "multipart/related" as defined in
Section 6.3.1, HTTP body of the response MUST consists of two parts
with the media types "application/alto-costmap+json" and
"application/alto-propmap+json" accordingly. Specifically, the part
with media type "application/alto-costmap+json" MUST be the first
part.
The content of the "application/alto-costmap+json" part uses the
format in Section 11.2.3.6 of [RFC7285] with the following
constraints:
o The cost value for a path vector query, e.g. the cost mode is
"array" and the cost metric is "ane-path", MUST be encoded as a
JSONArray of AbstractNetworkElementName.
o If the query sent by the client includes cost type path vector,
the "vtag" field defined in Section 5.4 has to be included in the
response. And the "query-id" information in "vtag" MUST be
provided to ALTO clients.
6.3. Filtered Cost Map Extensions
This document extends the Filtered Cost Map defined in Section 4.1 of
[I-D.ietf-alto-multi-cost].
The specifications for "HTTP method" and "uses" are the same as
defined in Section 4.1 of [I-D.ietf-alto-multi-cost].
6.3.1. Media Type
The same as Section 6.2.1.
6.3.2. Capabilities
The FilteredCostMapCapabilities object has the same format as defined
in Section 4.1.1 of [I-D.ietf-alto-multi-cost] with the following
constraint:
testable-cost-type-names: The path vector cost type with "ane-path"
as the cost metric and "array" as the cost mode MUST NOT be
included in "testable-cost-type-names".
Bernstein, et al. Expires January 4, 2018 [Page 13]
Internet-Draft ALTO Extension: Path Vector July 2017
6.3.3. Property-map
The same as Section 6.2.3.
6.3.4. Accept Input Parameters
The ReqFilteredCostMap uses the same format as defined in
Section 4.1.2 of [I-D.ietf-alto-multi-cost], with the following
constraints:
constraints, or-constraints: If the path vector cost type is
included in either "cost-type" or "multi-cost-types", ALTO clients
MUST NOT use it in "constraints" or "or-constraints". Otherwise,
the ALTO server MUST return an error with error code
"E_INVALID_FIELD_VALUE".
testable-cost-types: The path vector cost type MUST NOT be included
in the "testable-cost-types" field. Otherwise, the ALTO server
MUST return an error with error code "E_INVALID_FIELD_VALUE".
6.3.5. Response
If an ALTO client accepts "multipart/related" as defined in
Section 6.3.1, HTTP body of the response MUST consist of two parts
with the media types "application/alto-costmap+json" and
"application/alto-propmap+json" accordingly. Specifically, the part
with media type "application/alto-costmap+json" MUST be the first
part.
The content of the "application/alto-costmap+json" part has the same
format as defined in Section 4.1.3 of [I-D.ietf-alto-multi-cost] with
the following constraints:
o When the path vector cost type is included in "cost type" or
"multi-cost-type", the corresponding cost value MUST be encoded as
a JSONArray of AbstractNetworkElementName.
o If the query sent by the client includes cost type path vector,
the "vtag" field defined in Section 5.4 has to be included in the
response. And the "query-id" information in "vtag" MUST be
provided to ALTO clients.
6.4. Endpoint Cost Service Extensions
This document extends the Endpoint Cost Service defined in
Section 4.2 in [I-D.ietf-alto-multi-cost].
Bernstein, et al. Expires January 4, 2018 [Page 14]
Internet-Draft ALTO Extension: Path Vector July 2017
The specifications for "HTTP method" and "uses" are the same as
defined in Section 4.2 in [I-D.ietf-alto-multi-cost].
6.4.1. Media Type
The path vector extension now enables ALTO clients to receive
multiple objects from the endpoint cost service response.
Specifically, if an ALTO client accepts "multipart/related",
"application/alto-endpointcost+json" and "application/alto-
propmap+json" at the same time, the ALTO server MUST use "multipart/
related" as the media type in the HTTP header.
6.4.2. Capabilities
The same as defined in Section 6.3.2.
6.4.3. Property-map
The same as Section 6.2.3.
6.4.4. Accept Input Parameters
The ReqEndpointCostMap uses the same format as defined in
Section 4.2.2 of [I-D.ietf-alto-multi-cost], with the following
constraints:
cost-type, multi-cost-types: ALTO clients MUST include the path
vector cost type, e.g. the one with "ane-path" as cost metric and
"array" as cost mode, in either "cost-type" or "multi-cost-types"
to activate the path vector extension.
constraints, or-constraints: If the path vector cost type is
included in either "cost-type" or "multi-cost-types", ALTO clients
MUST NOT use it in "constraints" or "or-constraints". Otherwise,
the ALTO server MUST return an error with error code
"E_INVALID_FIELD_VALUE".
testable-cost-types: The path vector cost type MUST NOT be included
in the "testable-cost-types" field. Otherwise, the ALTO server
MUST return an error with error code "E_INVALID_FIELD_VALUE".
6.4.5. Response
If an ALTO client accepts "multipart/related" as defined in
Section 6.4.1, HTTP body of the response MUST consist of two parts
with the media types "application/alto-endpointcost+json" and
"application/alto-propmap+json" accordingly. Specifically, the part
Bernstein, et al. Expires January 4, 2018 [Page 15]
Internet-Draft ALTO Extension: Path Vector July 2017
with media type "application/alto-endpointcost+json" MUST be the
first part.
The content of the "application/alto-endpointcost+json" part has the
same format as defined in Section 4.2.3 of [I-D.ietf-alto-multi-cost]
with the following constraints:
o When the path vector cost type is included in "cost type" or
"multi-cost-type", the corresponding cost value MUST be encoded as
a JSONArray of AbstractNetworkElementName.
o If the query sent by the client includes cost type path vector,
the "vtag" field defined in Section 5.4 has to be included in the
response. And the "query-id" information in "vtag" MUST be
provided to ALTO clients.
7. Examples
This section lists a series of examples to proceed the flow
scheduling use case in Section 3.
7.1. Workflow
This section gives a typical workflow of an ALTO client using the
path-vector extension.
1. Send a GET request for the whole Information Resource Directory.
2. Look for the resource of the (Filtered) Cost Map/Endpoint Cost
Service which contains the path vector cost type and get the
resource ID of the dependent abstract network element property
map.
3. Check whether the capabilities of the property map includes the
desired "prop-types".
4. Send a path-vector request which accepts "multipart/related"
media type following Section 6.2.1, Section 6.3.1 or
Section 6.4.1.
Alternatively, one can replace step 4 with the following:
1. Send a path-vector request which accepts "application/alto-
costmap+json" or "application/alto-endpointcost+json".
2. Find the "query-id" in "vtag" in the response.
Bernstein, et al. Expires January 4, 2018 [Page 16]
Internet-Draft ALTO Extension: Path Vector July 2017
3. Query the dependent abstract network element property map with
the query ID and abstract network element names to retrieve the
associated properties.
7.2. Information Resource Directory Example
Here is an example of an Information Resource Directory. In this
example, filtered cost map "cost-map-pv" doesn't support the multi-
cost extension but support the path-vector extension, "endpoint-
multicost-map" supports both multi-cost extension and path-vector
extension. Filtered Property Map "propmap-delay-availbw" supports
properties "availbw" and "delay", and "propmap-location" supports
property "location".
{
"meta": {
"cost-types": {
"pv": {
"cost-mode": "array",
"cost-metric": "ane-path"
},
"num-routingcost": {
"cost-mode": "numerical",
"cost-metric": "routingcost"
},
"num-hopcount": {
"cost-mode": "numerical",
"cost-metric": "hopcount"
}
}
},
"resources": {
"my-default-networkmap": {
"uri" : "http://alto.example.com/networkmap",
"media-type" : "application/alto-networkmap+json"
}
"cost-map-pv" : {
"uri": "http://alto.example.com/costmap/pv",
"media-type": "application/alto-costmap+json",
"accepts": "application/alto-costmapfilter+json",
"capabilities": {
"cost-type-names": [ "pv", "num-hopcount" ]
},
"property-map": "propmap-delay",
"uses": [ "my-default-networkmap" ]
},
"endpoint-multicost-map" : {
"uri": "http://alto.exmaple.com/endpointcostmap/multicost",
Bernstein, et al. Expires January 4, 2018 [Page 17]
Internet-Draft ALTO Extension: Path Vector July 2017
"media-type": "application/alto-endpointcost+json",
"accepts": "application/alto-endpointcostparams+json",
"capabilities": {
"cost-constraints": true,
"cost-type-names": [ "pv", "num-routingcost" ],
"max-cost-types": 2
},
"property-map": "propmap-availbw"
},
"propmap-availbw" : {
"uri": "http://alto.exmaple.com/propmap/availbw",
"media-type": "application/alto-propmap+json",
"accepts": "application/alto-propmapparams+json",
"capabilities": {
"domain-types": [ "ane" ],
"prop-types": [ "delay", "availbw" ]
}
},
"propmap-delay" : {
"uri": "http://alto.exmaple.com/propmap/delay",
"media-type": "application/alto-propmap+json",
"accepts": "application/alto-propmapparams+json",
"capabilities": {
"domain-types": [ "ane" ],
"prop-types": [ "delay" ]
}
}
}
}
7.3. Single Query Example # 1
POST /costmap/pv HTTP/1.1
Host: alto.example.com
Accept: multipart/related, application/alto-costmap+json,
application/alto-propmap+json, application/alto-error+json
Content-Length: [TBD]
Content-Type: application/alto-costmapfilter+json
{
"cost-type": {
"cost-mode": "array",
"cost-metric": "ane-path"
},
"pids": {
"srcs": [ "PID1" ],
"dsts": [ "PID2", "PID3" ]
}
Bernstein, et al. Expires January 4, 2018 [Page 18]
Internet-Draft ALTO Extension: Path Vector July 2017
}
HTTP/1.1 200 OK
Content-Length: [TBD]
Content-Type: multipart/related; boundary=42
--42
Content-Type: application/alto-costmap+json
{
"meta": {
"dependent-vtags": [
{
"resource-id": "default-network-map",
"tag": "75ed013b3cb58f896e839582504f622838ce670f"
}
],
"cost-type": { "cost-mode": "array", "cost-metric": "ane-path" },
"vtag": {
"resource-id": "cost-map-pv",
"tag": "27612897acf278ffu3287c284dd28841da78213",
"query-id": "query1"
}
},
"cost-map": {
"PID1": {
"PID2": [ "ane:L001", "ane:L003" ],
"PID3": [ "ane:L001", "ane:L004" ]
}
}
}
--42
Content-Type: application/alto-propmap+json
{
"property-map": {
"ane:L001": { "delay": 46},
"ane:L003": { "delay": 50},
"ane:L004": { "delay": 70}
}
}
--42--
Bernstein, et al. Expires January 4, 2018 [Page 19]
Internet-Draft ALTO Extension: Path Vector July 2017
7.4. Single Query Example # 2
POST /endpointcostmap/multicost HTTP/1.1
Host: alto.example.com
Accept: multipart/related, application/alto-costmap+json,
application/alto-propmap+json, application/alto-error+json
Content-Length: [TBD]
Content-Type: application/alto-costmapfilter+json
{
"multi-cost-types": [
{
"cost-mode": "array",
"cost-metric": "ane-path"
},
{
"cost-mode": "numerical",
"cost-metric": "routingcost"
}
],
"endpoints": {
"srcs": [ "ipv4:192.0.2.2" ],
"dsts": [ "ipv4:192.0.2.89",
"ipv4:203.0.113.45",
"ipv6:2001:db8::10" ]
}
}
HTTP/1.1 200 OK
Content-Length: [TBD]
Content-Type: multipart/related; boundary=example-2
--example-2
Content-Type: application/alto-endpointcost+json
{
"meta": {
"multi-cost-types": [
{"cost-mode": "array", "cost-metric": "ane-path"},
{"cost-mode": "numerical", "cost-metric": "routingcost"}
]
"vtag": {
"resource-id": "endpoint-multicost-map",
"tag": "47612897acf278ffa3287cb84dd28841da78213",
"query-id": "query2"
}
},
Bernstein, et al. Expires January 4, 2018 [Page 20]
Internet-Draft ALTO Extension: Path Vector July 2017
"endpoint-cost-map": {
"ipv4:192.0.2.2": {
"ipv4:192.0.2.89": [[ "ane:L001", "ane:L003", "ane:L004" ], 77],
"ipv4:203.0.113.45": [[ "ane:L001", "ane:L004", "ane:L005" ], 68],
"ipv6:2001:db8::10": [[ "ane:L001", "ane:L005", "ane:L007" ], 98]
}
}
}
--example-2
Content-Type: application/alto-propmap+json
{
"property-map": {
"ane:L001": { "availbw": 50 },
"ane:L003": { "availbw": 48 },
"ane:L004": { "availbw": 55 },
"ane:L005": { "availbw": 60 },
"ane:L007": { "availbw": 35 }
}
}
--example-2--
7.5. Multiple Queries Example
7.5.1. Endpoint Cost Service Example
Bernstein, et al. Expires January 4, 2018 [Page 21]
Internet-Draft ALTO Extension: Path Vector July 2017
POST /endpointcostmap/multicost HTTP/1.1
Host: alto.example.com
Accept: application/alto-costmap+json, application/alto-error+json
Content-Length: [TBD]
Content-Type: application/alto-costmapfilter+json
{
"multi-cost-types": [
{
"cost-mode": "array",
"cost-metric": "ane-path"
},
{
"cost-mode": "numerical",
"cost-metric": "routingcost"
}
],
"endpoints": {
"srcs": [ "ipv6:2001:db8::10" ],
"dsts": [ "ipv4:192.0.2.3",
"ipv4:203.0.113.56" ]
}
}
HTTP/1.1 200 OK
Content-Length: [TBD]
Content-Type: application/alto-endpointcost+json
{
"meta": {
"vtag": {
"resource-id": "endpoint-multicost-map",
"tag": "f7622897bcf278ffu3287c284dd23841da78213",
"query-id": "query3"
},
"multi-cost-types": [
{ "cost-mode": "array", "cost-metric": "ane-path" },
{ "cost-mode": "numerical", "cost-metric": "routingcost"}
]
},
"endpoint-cost-map": {
"ipv6:2001:db8::10": {
"ipv4:192.0.2.3": [ "ane:L001", "ane:L006" ],
"ipv4:203.0.113.56": [ "ane:L001", "ane:L007" ]
}
}
}
Bernstein, et al. Expires January 4, 2018 [Page 22]
Internet-Draft ALTO Extension: Path Vector July 2017
7.5.2. Abstract Network Element Property Map Example
POST /propmap/availbw HTTP/1.1
Host: alto.example.com
Accept: application/alto-propmap+json,application/alto-error+json
Content-Length: [TBD]
Content-Type: application/alto-propmapparams+json
{
"query-id": "query3",
"entities" : [ "ane:L001",
"ane:L006" ],
"properties" : [ "availbw" ]
}
HTTP/1.1 200 OK
Content-Length: [TBD]
Content-Type: application/alto-propmap+json
{
"property-map": {
"ane:L001": { "availbw": 25 },
"ane:L006": { "availbw": 40 }
}
}
8. Compatibility
8.1. Compatibility with Legacy ALTO Clients/Servers
Legacy ALTO clients SHOULD NOT send queries with the path-vector
extension and ALTO servers with this extension SHOULD NOT have any
compatibility issue. Legacy ALTO servers do not support cost types
with cost mode being "array" and cost metric being "ane-path", so
they MUST NOT announce the extended cost types in IRD. Thus, ALTO
clients MUST NOT send queries specified in this extension to legacy
ALTO servers according to Section 11.3.2.3 [RFC7285].
8.2. Compatibility with Multi-Cost Extensions
Path Vector is not a testable cost type. Any format of constraints
SHOULD NOT be applied to cost type path-vector in order for multi-
cost to support the path-vector extension. Specifically,
o Cost type path-vector MUST NOT be included in "testable-cost-
types-names" or "testable-cost-types".
Bernstein, et al. Expires January 4, 2018 [Page 23]
Internet-Draft ALTO Extension: Path Vector July 2017
o When "testable-cost-types-names" is omitted in the "capabilities"
and "testable-cost-types" is omitted in the input parameters,
"constraints" or "or-constraints" SHOULD NOT add any format of
constraints on cost type path-vector.
8.3. Compatibility with Incremental Update
Without considering the incremental update of multipart/related
information, there is no compatibility issue with incremental update
extension. Compatibility issue with the incremental update of
multipart/related information will be discussed and addressed in the
next version.
9. Design Decisions and Discussions
9.1. Provide More General Calendar Extension
Cost Calendar is proposed as a useful ALTO extension to provide the
historical cost values for Filtered Cost Map Service and Endpoint
Cost Service. Since path vector is an extension to these services,
it SHOULD be compatible with Cost Calendar extension.
However, the calendar of a path-vector (Endpoint) Cost Map is
insufficient for the application which requires the historical data
of routing state information. The (Endpoint) Cost Map can only
provide the changes of the paths. But more useful information is the
history of network element properties which are recorded in the
dependent Network Element Property Map.
Before the Unified Property Map is introduced as an ALTO extension,
Filtered Cost Map Service and Endpoint Cost Service are the only
resources which require the calendar supported. Because other
resources don't have to be updated frequently. But Network Element
Property Map as a use case of Unified Property Map will collect the
real-time information of the network. It SHOULD be updated as soon
as possible once the metrics of network elements change.
So the requirement is to provide a general calendar extension which
not only meets the Filtered Cost Map and Endpoint Cost Service but
also applies to the Property Map Service.
10. Security Considerations
10.1. Privacy Concerns
We can identify multiple potential security issues. A main security
issue is network privacy, as the path-vector information may reveal
more network internal structures than the more abstract single-node
Bernstein, et al. Expires January 4, 2018 [Page 24]
Internet-Draft ALTO Extension: Path Vector July 2017
abstraction. The network should consider protection mechanisms to
reduce information exposure, in particular, in settings where the
network and the application do not belong to the same trust domain.
On the other hand, in a setting of the same trust domain, a key
benefit of the path-vector abstraction is reduced information
transfer from the network to the application.
The path-vector query may also reveal more information about the
application. In particular, the application may reveal all potential
transfers sites (e.g., where the data source is replicated, and where
the potential replication sites are). The application should
evaluate the potential privacy concerns.
Beyond the privacy issues, the computation of the path-vector is
unlikely to be cachable, in that the results will depend on the
particular requests (e.g., where the flows are distributed). Hence,
this service may become an entry point for denial of service attacks
on the availability of an ALTO server. Hence, authenticity and
authorization of this ALTO service may need to be better protected.
10.2. Resource Consumption on ALTO Servers
The Abstract Network Element Property Map is dynamically enriched
when the (Filtered) Cost Map/Endpoint Cost Service is queried of the
path-vector information. The properties of the abstract network
elements can consume a large amount of resources when cached. So, a
time-to-live is needed to remove outdated entries in the Network
Element Property Map.
11. IANA Considerations
11.1. ALTO Cost Mode Registry
This document specifies a new cost mode "array". However, the base
ALTO protocol does not have a Cost Mode Registry where new cost mode
can be registered. This new cost mode will be registered once the
registry is defined either in a revised version of [RFC7285] or in
another future extension.
11.2. ALTO Cost Metric Registry
A new cost metric needs to be registered in the "ALTO Cost Metric
Registry", listed in Table 2.
Bernstein, et al. Expires January 4, 2018 [Page 25]
Internet-Draft ALTO Extension: Path Vector July 2017
+-------------+---------------------+
| Identifier | Intended Semantics |
+-------------+---------------------+
| ane-path | See Section 5.1.1 |
+-------------+---------------------+
Table 2: ALTO Cost Metrics
11.3. ALTO Entity Domain Registry
As proposed in Section 9.2 of [I-D.roome-alto-unified-props], "ALTO
Entity Domain Registry" is requested. Besides, a new domain is to be
registered, listed in Table 3.
+-------------+--------------------------+--------------------------+
| Identifier | Entity Address Encoding | Hierarchy & Inheritance |
+-------------+--------------------------+--------------------------+
| ane | See Section 5.2.2 | None |
+-------------+--------------------------+--------------------------+
Table 3: ALTO Entity Domain
11.4. ALTO Network Element Property Type Registry
The "ALTO Abstract Network Element Property Type Registry" is
required by the ALTO Entity Domain "ane", listed in Table 4.
+-------------+--------------------------+
| Identifier | Intended Semantics |
+-------------+--------------------------+
| availbw | The available bandwidth |
| delay | The transmission delay |
+-------------+--------------------------+
Table 4: ALTO Abstract Network Element Property Types
12. Acknowledgments
The authors would like to thank discussions with Andreas Voellmy,
Erran Li, Haibin Son, Haizhou Du, Jiayuan Hu, Qiao Xiang, Tianyuan
Liu, Xiao Shi, Xin Wang, and Yan Luo.
13. References
Bernstein, et al. Expires January 4, 2018 [Page 26]
Internet-Draft ALTO Extension: Path Vector July 2017
13.1. Normative References
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119,
DOI 10.17487/RFC2119, March 1997,
<http://www.rfc-editor.org/info/rfc2119>.
13.2. Informative References
[I-D.amante-i2rs-topology-use-cases]
Medved, J., Previdi, S., Lopez, V., and S. Amante,
"Topology API Use Cases", draft-amante-i2rs-topology-use-
cases-01 (work in progress), October 2013.
[I-D.bernstein-alto-topo]
Bernstein, G., Yang, Y., and Y. Lee, "ALTO Topology
Service: Uses Cases, Requirements, and Framework", draft-
bernstein-alto-topo-00 (work in progress), October 2013.
[I-D.clemm-i2rs-yang-network-topo]
Clemm, A., Medved, J., Tkacik, T., Varga, R., Bahadur, N.,
and H. Ananthakrishnan, "A YANG Data Model for Network
Topologies", draft-clemm-i2rs-yang-network-topo-01 (work
in progress), October 2014.
[I-D.gao-alto-fcs]
Gao, K., Zhang, J., Wang, J., Xiang, Q., and Y. Yang,
"ALTO Extension: Flow-based Cost Query", draft-gao-alto-
fcs-01 (work in progress), March 2017.
[I-D.ietf-alto-cost-calendar]
Randriamasy, S., Yang, Y., Wu, Q., Lingli, D., and N.
Schwan, "ALTO Cost Calendar", draft-ietf-alto-cost-
calendar-01 (work in progress), February 2017.
[I-D.ietf-alto-incr-update-sse]
Roome, W. and Y. Yang, "ALTO Incremental Updates Using
Server-Sent Events (SSE)", draft-ietf-alto-incr-update-
sse-03 (work in progress), September 2016.
[I-D.ietf-alto-multi-cost]
Randriamasy, S., Roome, W., and N. Schwan, "Multi-Cost
ALTO", draft-ietf-alto-multi-cost-07 (work in progress),
March 2017.
Bernstein, et al. Expires January 4, 2018 [Page 27]
Internet-Draft ALTO Extension: Path Vector July 2017
[I-D.lee-alto-app-net-info-exchange]
Lee, Y., Bernstein, G., Choi, T., and D. Dhody, "ALTO
Extensions to Support Application and Network Resource
Information Exchange for High Bandwidth Applications",
draft-lee-alto-app-net-info-exchange-02 (work in
progress), July 2013.
[I-D.roome-alto-unified-props]
Roome, W., "Extensible Property Maps for the ALTO
Protocol", draft-roome-alto-unified-props-01 (work in
progress), July 2016.
[RFC2387] Levinson, E., "The MIME Multipart/Related Content-type",
RFC 2387, DOI 10.17487/RFC2387, August 1998,
<http://www.rfc-editor.org/info/rfc2387>.
[RFC7285] Alimi, R., Ed., Penno, R., Ed., Yang, Y., Ed., Kiesel, S.,
Previdi, S., Roome, W., Shalunov, S., and R. Woundy,
"Application-Layer Traffic Optimization (ALTO) Protocol",
RFC 7285, DOI 10.17487/RFC7285, September 2014,
<http://www.rfc-editor.org/info/rfc7285>.
Authors' Addresses
Greg Bernstein
Grotto Networking
Fremont, CA
USA
Email: gregb@grotto-networking.com
Shiwei Dawn Chen
Tongji University
4800 Caoan Road
Shanghai 201804
China
Email: dawn_chen_f@hotmail.com
Kai Gao
Tsinghua University
Beijing Beijing
China
Email: gaok12@mails.tsinghua.edu.cn
Bernstein, et al. Expires January 4, 2018 [Page 28]
Internet-Draft ALTO Extension: Path Vector July 2017
Young Lee
Huawei
TX
USA
Email: leeyoung@huawei.com
Wendy Roome
Nokia/Bell Labs
600 Mountain Ave, Rm 3B-324
Murray Hill, NJ 07974
USA
Phone: +1-908-582-7974
Email: wendy.roome@nokia.com
Michael Scharf
Nokia
Germany
Email: michael.scharf@nokia.com
Y. Richard Yang
Yale University
51 Prospect St
New Haven CT
USA
Email: yry@cs.yale.edu
Jingxuan Jensen Zhang
Tongji University
4800 Caoan Road
Shanghai 201804
China
Email: jingxuan.n.zhang@gmail.com
Bernstein, et al. Expires January 4, 2018 [Page 29]