INTERNET-DRAFT Roger Lapuh
Intended Status: Informational Paul Unbehagen
Expires: <September 3, 2012> Avaya
Peter Ashwood-Smith
Huawei
March 5, 2012
SPB Deployment Considerations
draft-lapuh-spb-deployment-00
Abstract
Based on life deployments and three interoperability events, this
document provides advice to network operators about best practices
when implementing IEEE 802.1aq Shortest Path Bridging (SPB) networks.
It is principally addressed to system integrators and solution
providers, including those that do not yet support SPB. Some advice
to implementers is also included. The intention of the advice is to
facilitate multi vendor network deployments as well as provide
guidance for new installations.
Status of this Memo
This Internet-Draft is submitted to IETF in full conformance with the
provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that
other groups may also distribute working documents as
Internet-Drafts.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
The list of current Internet-Drafts can be accessed at
http://www.ietf.org/1id-abstracts.html
The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html
Lapuh, R. Expires September 3, 2012 [Page 1]
INTERNET DRAFT SPB Deployment Recommendations March 5th
Copyright and License Notice
Copyright (c) 2012 IETF Trust and the persons identified as the
document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must
include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as
described in the Simplified BSD License.
Table of Contents
1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.1 Terminology . . . . . . . . . . . . . . . . . . . . . . . . 3
1.2 MOTIVATION AND BACKGROUND . . . . . . . . . . . . . . . . . 3
2. General Deployment Recommendations . . . . . . . . . . . . . . 3
3. INFRASTRUCTURE CONFIGURATION RECOMMENDATIONS . . . . . . . . . 4
3.1 IS-IS SYSTEM ID AND SPB NICKNAME CONFIGURATION
RECOMMENDATIONS . . . . . . . . . . . . . . . . . . . . . . 4
3.2 SPB FABRIC INTERFACE TYPES . . . . . . . . . . . . . . . . 5
3.3 SPB FABRIC ACCESS . . . . . . . . . . . . . . . . . . . . . 6
3.4 SPB Fabric configuration . . . . . . . . . . . . . . . . . 6
3.5 SPB SERVICES MAPPING . . . . . . . . . . . . . . . . . . . 6
3.6 SPB AND IP ROUTING . . . . . . . . . . . . . . . . . . . . 7
4. STANDARD IMPROVEMENT RECOMMENDATIONS . . . . . . . . . . . . . 7
5. OA&M . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
6. Tenant Separation Considerations . . . . . . . . . . . . . . . 7
7 Deployment Experiences . . . . . . . . . . . . . . . . . . . . 8
7.1 DEPLOYMENT SCENARIO A . . . . . . . . . . . . . . . . . . . 8
7.2 DEPLOYMENT SCENARIO B . . . . . . . . . . . . . . . . . . . 9
7.3 DEPLOYMENT SCENARIO C . . . . . . . . . . . . . . . . . . . 9
7.4 DEPLOYMENT SCENARIO D . . . . . . . . . . . . . . . . . . . 10
8 Security Considerations . . . . . . . . . . . . . . . . . . . . 10
9 IANA Considerations . . . . . . . . . . . . . . . . . . . . . . 11
5 References . . . . . . . . . . . . . . . . . . . . . . . . . . 11
5.1 Normative References . . . . . . . . . . . . . . . . . . . 11
5.2 Informative References . . . . . . . . . . . . . . . . . . 11
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 11
Lapuh, R. Expires September 3, 2012 [Page 2]
INTERNET DRAFT SPB Deployment Recommendations March 5th
1 Introduction
This document provides a set of recommendations and reference points
for the deployment of IEEE 802.1aq based Shortest Path Bridging (SPB)
networks based on MAC in MAC encapsulation. It focuses on the key
network design items and does not go into describing the protocol
details.
The IEEE 802.1aq standard has been technically frozen since early
2011 before several multi vendor interoperability events had taken
place, thus the recommendations described here are valid despite the
minor editorial work that has caused non technical change in the IEEE
base standard since.
1.1 Terminology
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in RFC 2119 [RFC2119].
1.2 MOTIVATION AND BACKGROUND
This document provides a checklist of recommendations which are based
on multiple documented multi vendor Interoperability tests [SPBWIKI]
and more than 12 months of production deployment experiences. It
summarizes the learning's and experience acquired during those
activities. New SPB installations can benefit from following the
recommendations below.
2. General Deployment Recommendations
All the following described deployments have shown sub second
convergence times in case of link or nodal failures within the SPB
fabric.
Recommendation 1:
To achieve this, strictly connection oriented point-to-point
interfaces are used, and shared segments between fabric nodes have
been avoided. Ethernet based mechanisms are used to detect link
faults quickly and to trigger a shortest path calculations in case of
a link and nodal failure.
The end-point-only provisioning for network virtualization with SPB
Lapuh, R. Expires September 3, 2012 [Page 3]
INTERNET DRAFT SPB Deployment Recommendations March 5th
has proven very effective in many of the installations described
below.
Recommendation 2:
SPB's service ID (I-SID) with its (24 bit) addressing space has
helped to keep the VLAN spaces (1 to 4095) local to the respective
data centers, avoiding the complexity of managing a global VLAN space
out of a range of only 4096. It is recommended to define a global
virtualization schema based on I-SIDs, and not tie VLAN ids directly
to ISIDs ids in a 1 to 1 relationship throughout the network.
Recommendation 3:
It has been seen that using SPB to keep Spanning Tree regions (and
therefore any impacts in case of network changes) local to access
networks, has significantly improved the end user experience.
Besides the need for L2 traffic virtualization for hypervisor
migrations, all the deployments also were required to route the
virtualized traffic between IP subnets/broadcast domains which are
provided with an SPB service.
Recommendation 4:
Routing between services can be done with dedicated routers external
to the SPB fabric, but it would be an advantage if SPB nodes could
route traffic between services similar to traditional routing
switches that are able to perform routing between VLANs/IP subnets
without having to leave the Ethernet fabric.
3. INFRASTRUCTURE CONFIGURATION RECOMMENDATIONS
3.1 IS-IS SYSTEM ID AND SPB NICKNAME CONFIGURATION RECOMMENDATIONS
As of this writing the IEEE SPB standard defines a single ISIS area
for an SPB region. Even though large SPB regions can be defined and
operated, in the future this will likely be extended to multi-area
support.
Recommendation 5:
The interoperability events have shown it is a good practice to
manually configure System IDs and SPB Nicknames with a simple
identification scheme, coordinating the system ID numerically with
the SPB Nickname for ease of troubleshooting. It is also a good
practice to define an area per SPB region. We recommend using the
Lapuh, R. Expires September 3, 2012 [Page 4]
INTERNET DRAFT SPB Deployment Recommendations March 5th
upper 24 bits to indicate the area ID. For example, System IDs start
with 4900.bb00.1000 for the first node, 4900.bb00.2000 for the second
node and so on. In these System IDs, 49 is indicating a private
address, the "00bb" indicates area "00bb", and 1000, 2000, etc.,
indicate the node number (1 through n). These System IDs correspond
to SPB Nicknames of 1.bb.10, 1.bb,20, 1.bb.30 for nodes 1, 2 and 3
respectively, and so on. Using manually configured BMAC addresses and
also coordinating the BMAC with the System ID and SPB Nickname
enhances-, ease of identification for management and troubleshooting
as well as making possible future PBB-EVPN.
As an additional option, with the goal to reduce configuration tasks,
System-IDs could be automatically inherited from the systems chassis
MAC addresses. Also the SPB Nickname could be derived from the lower
bits of the chassis MAC. This approach could be targeted for SPB
access switches where a simplified deployment model would be of most
interest.
3.2 SPB FABRIC INTERFACE TYPES
Details on Recommendation 1:
SPB Fabric inter connections in the preceding SPB deployments are all
based on point-to-point Ethernet links, optical CWDM/DWDM connection
or some sort of transparent E-LINE service. By avoiding connecting
SPB over a shared segment (or E-LAN) failure detection and network
convergence times have been kept very low. Failure detection and
recovery is thus not dependent on IS-IS hello-multiplier intervals
but triggered by lower layer protocols.
Such E-LINE services (to interconnect SPB nodes) can be based on any
type of transparent Ethernet service (MPLS- or PBB based), as long as
they are loop free and the service Maximum Transmission Unit (MTU)
size is allowing for a minimum of [MTU] 1544 bytes.
Tagged IP: = 1522 = 1500(IP MTU)+ 2(Ethertype)+ 12(MAC SA/DA) +
4(TAG) + 4 (CRC) and MacInMac Header = 22 bytes.
On dark-fiber based Ethernet connections, link failures can be
detected by the Ethernet remote fault detection mechanisms; however,
on service provider based links, there are multiple active components
between two SPB nodes, and thus not all failures can be detected
easily. To ensure quick failover times across an E-LINE service, an
end-to-end connectivity check mechanism such as 802.1ag based
Connectivity Check Mechanism (CCM), or similar, is recommended.
Lapuh, R. Expires September 3, 2012 [Page 5]
INTERNET DRAFT SPB Deployment Recommendations March 5th
3.3 SPB FABRIC ACCESS
Details on Recommendation 3:
Many networks today still operate with some sort of Spanning Tree
(MSTP/RSTP or proprietary versions). SPB can be leveraged to separate
Spanning Tree regions into smaller independent domains. Thus a
Spanning Tree root bridge change impacts smaller regions only and is
not spread across the whole network. Keeping root bridge elections
and the effect of Topology Change Notifications local has proven a
significant improvement of network availability in larger Spanning
Tree deployments.
3.4 SPB Fabric configuration
Recommendation 6:
In an SPB network the Backbone VLAN IDs (BVIDs) are used to separate
and load-spread SPB traffic across multiple paths. The 802.1aq
standard defines up to 16 BVIDs. Those BVIDs need to be consistently
configured across the SPB region. Those BVIDs can be selected out of
the available VLAN range [1-4095]. Using a pre-defined set of VLANs
is recommended.
Usually the lowest 4000 IDs are used by customers for network access
VLAN configurations; thus it has been seen as a good practice to use
BVLAN numbering that is in the highest upper addressable range, e.g,
start from 4050 for the primary BVLAN and all switches and 4051 to
4065 for the subsequent ones. It is recommended to use at least two
BVIDs for load-spreading reasons.
3.5 SPB SERVICES MAPPING
Details on Recommendation 2:
When network virtualization needs to be extended between regions, for
example for Virtual Machine movements, it is very important to use a
unique virtualization index to achieve this. SPB, with its 802.1ah
based I-SID provides an inherent virtualization technique which
allows localizing the VLAN significance and using the service ID (I-
SID) as a global virtualization index. Especially in VMWare
deployments where the Portgroup IDs of VCenter instances need to
correspond with the VM VLAN memberships this is an advantage. With
SPB, it is thus easily possible to run a hosted environment with
multiple VCenter instances in parallel on the same infrastructure
without having any VLAN space interference. In the preceding virtual
Data Center deployment where multiple domains are interconnected, the
Lapuh, R. Expires September 3, 2012 [Page 6]
INTERNET DRAFT SPB Deployment Recommendations March 5th
VLAN spaces can be kept independent of each other, and the
virtualization is achieved by the usage of the I-SIDs.
3.6 SPB AND IP ROUTING
Details on Recommendation 4:
In an SPB network the typical size of the user and server subnets are
not being changed from what one is used to with traditional
technologies. This means that there is always a need for routing
functionality. The best case is if a SPB node can directly route
individual IP subnets which consist of I-SIDs, similar to how those
nodes can route VLAN based IP subnets. Optimally this routing should
be available within the SPB Ethernet fabric between I-SID based
services.
4. STANDARD IMPROVEMENT RECOMMENDATIONS
Recommendation 7:
In an SPB network, failure propagation is achieved by SPB nodes re-
laying topology change notifications through IS-IS on a hop by hop
basis in the control plane. With an assumption of 10ms relay latency
per node, a ring of 20 SPB nodes could see up to 100ms of propagation
latency to reach all nodes in the ring. As an optimization of SPB,
the default L2 service instance described in IEEE 802.1aq could be
used to flood all propagation changes into this default service,
reducing the propagation delay in this example from 100ms to 10ms by
leveraging fast path forwarding. The SPB IEEE standard could be
enhanced to also include this default-flooding behavior.
5. OA&M
In all deployment experiences, the use of L2 based OAM capabilities
have been invaluable in managing the network.
Recommendation 8:
It is recommended that IEEE 802.1ag based connectivity check
mechanisms: Layer 2 Ping, Layer 2 Traceroute and Layer 2 Tracetree
are being implemented.
6. Tenant Separation Considerations
Lapuh, R. Expires September 3, 2012 [Page 7]
INTERNET DRAFT SPB Deployment Recommendations March 5th
SPB separates any type of traffic at the edge access of the SPB
region into its own service instance (I-SID). Classification into the
I-SID can be done based on port, vlan or a combination of port/vlan.
Once a customer-or application traffic is classified into an I-SID,
it is kept separate until it exits the SPB region, very similar to
MPLS with its tunnel and service labels. In SPB the "tunnel label" is
comprised of the BMAC pair and the "service label" is the I-SID. Thus
SPB is as secure as any other packet switched solution. Today there
are many service provider based networks in production using the same
802.1ah (PBB) encapsulation methods as SPB is using.
7 Deployment Experiences
7.1 DEPLOYMENT SCENARIO A
SPB AS INTER-DATACENTER-FABRIC FOR DC REDUNDANCY OR DC MIGRATIONS
Typically in a large enterprise core, it is not viewed as good
practice to extend L2 broadcast domains across the backbone network.
However, with the advent of server virtualization, it has become a
common requirement to extend server VLAN segments between geo-
redundant Data Centers to dynamically, efficiently and cost
effectively leverage the ability to perform Virtual Machine
migrations and run load balancing techniques across multiple Data
Centers.
With the deployment of SPB in a data center interconnect, the
following challenges have been addressed: In many cases SPB can be
deployed on the existing network architecture with IS-IS running side
by side and independently from other routing protocols such as OSPF.
OSPF is being used to populate the IP routing table and provide L3
routed connectivity. IS-IS is being used for SPB, bringing the
ability to extend server VLANs across the backbone. Typically the
server VLANs to be extended across the network are locally configured
within the Data Centers, on the server aggregation Top of Rack
switch(es) as well as on the distribution layer nodes for the Data
Center which aggregate the Top of Rack switches. On the distribution
nodes, the server VLANs are assigned to a service ID (I-SID) and thus
extended across the SPB network. Access redundancy is provided with
an active-active model which ties the SPB core region to the VLAN
based access region. The same distribution nodes can act as a routing
gateway for the server VLANs. VRRP is also being used to create a
single default gateway IP address for the server VMs. The VRRP
instance per Server IP-subnet thus exists on all distribution nodes
and provides redundant and distributed default gateway functionality.
Core failure recovery times in the SPB region can be kept well below
1 second and L3 recovery times, depending on the configured VRRP
Lapuh, R. Expires September 3, 2012 [Page 8]
INTERNET DRAFT SPB Deployment Recommendations March 5th
timers.
7.2 DEPLOYMENT SCENARIO B
SPB TO RE-ARCHITECT SECURITY ZONES
There are other valid reasons why it might be necessary to extend L2
segments across the enterprise core. A good example is a major
manufacturing plant which has a very rigorous design based on a pure
IP routed architecture with a strong focus on firewalling different
parts of the network. This is achieved by physically wedging
firewalls within the physical topology in such a way as to deny any
unwanted interaction between different network zones. The security
provided by this model has to be offset by the rigidity it imposes in
terms of where devices are allowed to be connected to the network. In
this particular example, connecting devices in locations where they
were not initially intended to be located was addressed by laying
additional cabling, with the costs and delays that this involves.
Once deployed, SPB brought to this model the ability to decouple the
physical infrastructure from the logical connectivity running above
it. This means that it is no longer necessary to wedge firewalls into
the physical topology to intercept traffic, but rather let SPB force
L2 VLANs to reach the desired firewalls, wherever those firewalls
might be located on the network. It is now possible to connect
devices anywhere on the physical network infrastructure and simply
connect these devices to the VLAN segment to which they need to
belong.
7.3 DEPLOYMENT SCENARIO C
SPB FOR CAMPUS VIRTUALIZATION
Another example of where it is useful to extend L2 segments can be
found in the health care vertical. An operational challenge, typical
of most hospitals, is to be able to support network connectivity for
mobile medical equipment which typically needs to connect to a server
application hosted in the Data Center. The real challenge with this
equipment is often the fact that it is supplied and maintained by
separate, often external, technicians with little or no IP skills. As
such this equipment is usually not able, or not configured, to use
DHCP and instead uses a single flat IP subnet which encompasses the
mobile units as well as the server application in the Data Center.
The hospital's network team essentially has limited control over the
IP configuration of these devices and hence a desire to segregate
such applications within a constrained L2 service. By deploying SPB
Lapuh, R. Expires September 3, 2012 [Page 9]
INTERNET DRAFT SPB Deployment Recommendations March 5th
L2 instances, it is now possible to much more easily manage such
applications.
7.4 DEPLOYMENT SCENARIO D
SPB AS MULTI-TENANT FABRIC SOLUTION
In a multitenant deployment SPB was leveraged to provide secured and
separated services for several tenants. In this implementation SPB
leverages 10 Gigabit Ethernet heavily. In the two geo-disbursed data
centers LAN and IP connectivity is utilized in a way that makes both
appear as one virtual data center. A common 3-tier design is utilized
for the entire network. There may be multiple tenants per edge which
are then segregated into their own private broadcast domain.
Over 500 L2 services are spread across the network providing IP
subnet connectivity to any of the tenants. At the data center those
IP subnets are assigned to over a dozen of Virtual Router Forwarding
(VRF)instances corresponding to their security requirements. VRRP is
used to provide router redundancy.
Layer 3 routing between VRF instances, hence between tenants, to
external organizations and to the Internet is performed by stateful
firewalls.
This simplified model utilizing Layer 2 I-SIDs, including routing
between service instances, across a common SPB backbone allows this
solution provider to quickly and effectively extend either Layer 2
services or Layer 3 services to any location in the network for any
application.
For Voice over IP a Quality of Service (QoS) framework for traffic
prioritization has been employed. IP Differentiated Services
(DiffServ) EF DSCP and several specific AF DSCP groups are mapped
into the appropriate 802.1p priority classes at the SPB BEB nodes to
provide the necessary traffic prioritization within the SPB backbone.
8 Security Considerations
Security implications of SPB deployments are to be discussed in
separate documents.
Lapuh, R. Expires September 3, 2012 [Page 10]
INTERNET DRAFT SPB Deployment Recommendations March 5th
9 IANA Considerations
This document makes no requests to IANA
5 References
5.1 Normative References
5.2 Informative References
[RFC6329] Fedyk, D., "IS-IS Extensions Supporting IEEE 802.1aq
Shortest Path Bridging" July 2011
[SPBWIKI] http://en.wikipedia.org/wiki/Shortest_Path_Bridging
Authors' Addresses
Roger Lapuh (editor)
Avaya
Wallisellen, 8304
Switzerland
EMail: rogerlapuh@avaya.com
Paul Unbehagen
Avaya
1300 W. 120th Avenue
Westminster, CO 80234 USA
Email: unbehagen@avaya.com
Steven Emert
Avaya
225 South Sixth Street, Suite 4350
Minneapolis, Minnesota 55402-4619
US
Email: semert@avaya.com
Ludovico Stevens (editor)
Avaya
25 Allee Pierre Ziller
06560 Valbonne
Lapuh, R. Expires September 3, 2012 [Page 11]
INTERNET DRAFT SPB Deployment Recommendations March 5th
France
EMail: ludovicostev@avaya.com
Peter Ashwood-Smith (editor)
Huawei Technologies Canada Ltd.
303 Terry Fox Drive, Suite 400
Kanata, Ontario, K2K 3J1
CANADA
EMail: Peter.AshwoodSmith@huawei.com
Srikanth Keesara
Avaya
600 Technology Park
Billerica MA 01821
US
EMmail: skeesara@avaya.com
Lapuh, R. Expires September 3, 2012 [Page 12]