IP over InfiniBand (IPoIB) Architecture
RFC 4392

• Status
• IESG evaluation record
• IESG writeups
• Email expansions
• History

• Versions
• 04

Network Working Group                                         V. Kashyap
Request for Comments: 4392                                           IBM
Category: Informational                                       April 2006

                IP over InfiniBand (IPoIB) Architecture

Status of This Memo

   This memo provides information for the Internet community.  It does
   not specify an Internet standard of any kind.  Distribution of this
   memo is unlimited.

Copyright Notice

   Copyright (C) The Internet Society (2006).

Abstract

   InfiniBand is a high-speed, channel-based interconnect between
   systems and devices.

   This document presents an overview of the InfiniBand architecture.
   It further describes the requirements and guidelines for the
   transmission of IP over InfiniBand.  Discussions in this document are
   applicable to both IPv4 and IPv6 unless explicitly specified.  The
   encapsulation of IP over InfiniBand and the mechanism for IP address
   resolution on IB fabrics are covered in other documents.

Table of Contents

   1. Introduction to InfiniBand ......................................2
      1.1. InfiniBand Architecture Specification ......................2
      1.2. Overview of InfiniBand Architecture ........................2
           1.2.1. InfiniBand Addresses ................................6
                  1.2.1.1. Unicast GIDs ...............................7
                  1.2.1.2. Multicast GIDs .............................7
      1.3. InfiniBand Multicast Group Management ......................9
           1.3.1. Multicast Member Record ............................10
                  1.3.1.1. JoinState .................................10
           1.3.2. Join and Leave Operations ..........................11
                  1.3.2.1. Creating a Multicast Group ................11
                  1.3.2.2. Deleting a Multicast Group ................11
                  1.3.2.3. Multicast Group Create/Delete Traps .......12
   2. Management of InfiniBand Subnet ................................12
   3. IP over IB .....................................................12
      3.1. InfiniBand as Datalink ....................................13

Kashyap                      Informational                      [Page 1]
RFC 4392                   IPoIB Architecture                 April 2006

      3.2. Multicast Support .........................................13
           3.2.1. Mapping IP Multicast to IB Multicast ...............14
           3.2.2. Transient Flag in IB MGIDs .........................14
      3.3. IP Subnets Across IB Subnets ..............................14
   4. IP Subnets in InfiniBand Fabrics ...............................14
      4.1. IPoIB VLANs ...............................................16
      4.2. Multicast in IPoIB subnets ................................16
           4.2.1. Sending IP Multicast Datagrams .....................17
           4.2.2. Receiving Multicast Packets ........................18
           4.2.3. Router Considerations for IPoIB ....................18
           4.2.4. Impact of InfiniBand Architecture Limits ...........19
           4.2.5. Leaving/Deleting a Multicast Group .................19
      4.3. Transmission of IPoIB Packets .............................20
      4.4. Reverse Address Resolution Protocol (RARP) and
           Static ARP Entries ........................................20
      4.5. DHCPv4 and IPoIB ..........................................21
   5. QoS and Related Issues .........................................21
   6. Security Considerations ........................................21
   7. Acknowledgements ...............................................21
   8. References .....................................................21
      8.1. Normative References ......................................21
      8.2. Informative References ....................................22

1.  Introduction to InfiniBand

   The InfiniBand Trade Association (IBTA) was formed to develop an I/O
   specification to deliver a channel based, switched fabric technology.
   The InfiniBand standard is aimed at meeting the requirements of
   scalability, reliability, availability, and performance of servers in
   data centers.

1.1.  InfiniBand Architecture Specification

   The InfiniBand Trade Association specification is available for
   download from http://www.infinibandta.org.

1.2.  Overview of InfiniBand Architecture

   For a more complete overview, the reader is referred to chapter 3 of
   the InfiniBand specification.

   InfiniBand Architecture (IBA) defines a System Area Network (SAN) for
   connecting multiple independent processor platforms, I/O platforms,
   and I/O devices.  The IBA SAN is a communications and management
   infrastructure supporting both I/O and inter-processor communications
   for one or more computer systems.

Kashyap                      Informational                      [Page 2]