Transmission of IP over InfiniBand (IPoIB)
draft-ietf-ipoib-ip-over-infiniband-09

[Ballot comment]
Since my discuss is clearing with no text changes, I'm entering the author's comments to me.
I'm okay with these answers.

>Ted Hardie:
>Discuss:
>[2004-08-30] Section 10 and 11 confused me on a couple of points.
>First, the emulation of
>promiscuous mode seemed a bit odd; I can understand why you would use it
>if you're medium supports it, but why emulate rather than send the packets
>only to the nodes listening?

The emulation is done on the RECEIVE end since we don't have control on
all the senders. An earlier solution requiring all the senders to also
send a copy of all multicast packets to the routers was rejected in
favor of the emulation.

Also, the emulation is not done for local multicast - the packets are sent
out to the corresponding IB multicast address. When there are no local
listeners the packet needs to be received by the router. It was decided by
the WG to keep the senders simple and let the router do the emulation since
the triggers are already present.

>
>I was also confused by this:
>
>    Note that for an IPoIB link that spans more than one IB subnet
>    connected by IB routers, an adequate multicast forwarding support at
>    the IB level is required for multicast packets to reach listeners on
>    a remote IB subnet. The specific mechanism for this is beyond the
>    scope of IPoIB.
>
>I realize that the specifics are meant to be elsewhere, but this left me
>unsure about whether it was possible to have IP multicast without using
>the IB multicast at all.  The term "remote IB subnet" also seemed like it
>could be ambiguous if there are multicast nodes on multiple links.

Not sure about the question. IP multicast is building on top of IB
multicast, and hence won't work without IB mulitcast. This paragraph
simply reiterates this fact, especially for those IPoIB links that span
multicast IB links.

>

[Ballot comment]
>    attributes, and how to setup basic broadcast and multicast services

s/setup/set up/

>    of its parameters are upto the implementation and/or the

s/upto/up to/

References not split.

[Ballot discuss]
>    In IPv6 subnets the MTU may be reduced by a Router Advertisement
>    [DISC] containing an MTU option which specifies a smaller MTU, or by
>    manual configuration of each node. If a Router Advertisement
>    received on an IPoIB interface has an MTU option specifying an MTU
>    larger than the link MTU or larger than a manually configured value,
>    that MTU option may be logged to system management but must be
>    otherwise ignored.

This could be worded better. I think that the intent of the IPv6 RA
MTU is to allow the sys admin to also specify a higher (not just
lower) value if it knows something about the implementations. But the
above wording suggests that they can be ignored.

What I think you want to say is that the MTU value in an RA MUST be
used, if the implementation supports it (and then maybe say what
happens otherwise). I don't think the above words say that.


>        a) Reserved Flags
>
>            These 8 bits are reserved for future use. These bits MUST be
>            set to zero on send and ignored on receive unless specified
>            differently in a future document.

IANA considerations?

> 9.4 Cautionary Note on QPN Caching
>
>    The link-layer address for IPoIB includes the QPN which might not be
>    constant across reboots or even across network interface resets.
>    Cached QPN entries, such as in static ARP entries or in RARP servers
>    will only work if the implementation(s) using these options ensure
>    that the QPN associated with an interface is invariant across
>    reboots/network resets.

ARP has no caching lifetimes... Seems like this spec should  include a
recommendation that ARP caches be revalidated periodically, e.g.,
every few minutes or so. IPv6 does this by default, so there is no
issue here.

[Ballot discuss]
I agree with most other issues already raised by the other DISCUSSes.
Below are (additional) discussion points as received from the ops directorare by Pekka Savola:

substantial
-----------
4.0 Multicast Mapping
[...]
    A special case exists for the IPv4 limited broadcast address
    "255.255.255.255" [HOSTS]. The address SHALL be mapped to the
    "broadcast-GID", which is defined as follows:

==> should the same mapping be done to the all-nodes IPv6 link-local
multicast address ff02::1 ?

9.3 Address Resolution in IPv6 Subnets

[...]
    The source/target address option is specified as follows:

        Length: 3

        Link-layer address:

            The link-layer address is as specified in section 9.1.1 and
            depicted in figure 5.

==> section 9.1.1 specifies that the length of the link-layer address is 20
bytes (note, it never says this, you have to calculate it yourself from the
figure, please fix this!).

The SLLA/TLLA option length in RFC2461, however, is defined as the
amount in the number of 8 octets, i.e., the length here would be 24
bytes, of which 2 bytes is consumed by the length/type fields.  That
leaves two unspecified bytes, which would implicitly be at the end.
Where do you want to put them (does 32 bit alignment matter?) ?  THis
requires more text.


editorial
---------

==> fails a number of nits, like the old boilerplate, too long lines,
reference split, etc.

3.0 InfiniBand Datalink
[...]
The packet
    transmission and routing within the IB fabric is also affected by
    additional parameters such as the traffic class (TClass), hop limit
    (HopLimit), service level (SL) and the flow label (FlowLabel).

==> what is the "service level" ?  The rest have a clear definition at least
in IPv6 land, though not necessarily with IPv4..


11.0 IP Multicast Routing

    IP multicast routing requires multicast routers to receive a copy of
    every link multicast packet on a locally connected link [IPMULT,
    IP6MLD]. For Ethernet this is usually achieved by turning on the
    promiscuous multicast mode on a locally connected Ethernet
    interface.

==> IP6MLD is an inappropriate reference here, because MLD/IGMP protocols do
not require for the routers to get the copies of all multicast packets.  On
the other hand, if a node wishes to *send* a multicast packet, it obviously
needs to be received by the router (in case it needs to be forwarded off-link).

Further, the reference to promiscuous mode may be slightly wrong in a
technical sense.  At least some Ethernet driver implementations I've seen
support either multicast filters (up to some number of groups), or passing
all the multicast traffic (which is a subset of promiscuous mode) upwards.

13.0 Security Considerations

    Controlled Q_Keys SHOULD be used in all transmissions. This is to
    prevent non-privileged software from fabricating IP datagrams.

==> (the same in section 4.1)  I'm concerned about the security model
provided by that approach.  Is this something similar to how some
applications could treat e.g., packets sourced from a "privileged TCP/UDP
port" (<1024) as being "secure"?  This was considered broken already 10
years ago.  This could work (to a degree) if you had an implicilit
assumption that only trustworthy nodes would be granted access to an IB
link.  Note that there is no security whatsoever on who is able to join to
an IB link.

In other words, this paragraph probably needs to be expanded to not to give
the reader a false sense of security.

[Ballot discuss]
Section 4 says:
  >
  > For IPv4, the group ID is only 28-bit long and the rest of the
  > bits are filled with 0.
  >
  How?  Is the 28-bit value left justified or right justified?
  because of the structure used in Figure 2, I suspect the zeros are
  on the left and the 28-bit value is on the right.

  Section 5 says:
  >
  > It is advisable to ensure that the creation and deletion of
  > the broadcast group are under administrative control.
  >
  I agree.  Why isn't there a MUST statement that requires this to
  be supported?  The RFC 2119 term "RECOMMENDED" is not used here
  either.  This is the minimum change to resolve this comment.

  Section 6 specifies a reserved field, but it does not describe any
  processing rules for this field.  What value should the transmitter
  use?  I assume that the receiver will ignore any value that is
  present, but this is not stated.  I am looking for words similar to
  those used in section 9.1.1.

[Ballot discuss]
Section 10 and 11 confused me on a couple of points. First, the emulation of
promiscuous mode seemed a bit odd; I can understand why you would use it
if you're medium supports it, but why emulate rather than send the packets
only to the nodes listening?

I was also confused by this:

    Note that for an IPoIB link that spans more than one IB subnet
    connected by IB routers, an adequate multicast forwarding support at
    the IB level is required for multicast packets to reach listeners on
    a remote IB subnet. The specific mechanism for this is beyond the
    scope of IPoIB.

I realize that the specifics are meant to be elsewhere, but this left me
unsure about whether it was possible to have IP multicast without using
the IB multicast at all.  The term "remote IB subnet" also seemed like it
could be ambiguous if there are multicast nodes on multiple links.

[Ballot discuss]
DISCUSS:
Section 4 says:
    For IPv4, the group ID is only 28-bit long and the rest of the
    bits are filled with 0.


State explicitly where the 0 bits are inserted, i.e., which end of the
80-bit field.  The exmaple seems to imply that it's 52 bits of 0s
followed by the group ID, but that isn't clear.


How can a node do an expanding ring search for the scope bits?  What
results tell you if you've gone too far or not far enough?


5.0:  It is not clear that this:


    The method creation of the broadcast group and the assignment/choice
    of its parameters are upto the implementation and/or the
    administrator of the IPoIB subnet.


will yield interoperable implementations.  What if one implementation
uses administrative control while another has some
implementation-specific scheme?  I suggest that administrative control
be mandatory, while other schemes are optional and
implementation-dependent.


6.0: It says that "implementation MUST be able to handle packets
received with or without the use of GRH."  What about sending?  It
seems that implementations MUST be able to send with GRH. 


7.0 says " Larger and smaller MTUs MAY be supported".  Smaller than
1280 violates the IPv6 spec, I believe.  This should be clarified here.


10.0:
    Implementations should cache the information about the existence of
    an IB multicast group, its MLID and other attributes.


What are typical cache lifetimes?  What would suggest longer or shorter
lifetimes?  What other events should cause cache flushes?

[Ballot discuss]
References need to be split normative/informative.  The reference to the InfiniBand Architecture Specification should then be listed as a normative reference, and a better locator than www.infinibandta.org should be provided -- there is no copy of the specification on that page, just a link to "Specifications".  It would also be helpful if people didn't have to register with the web site to download a copy of the specification.

RFC 2119 should be cited as a reference in section 1.0.  2119 is listed, but not cited (yes, this is a nit).

In section 4 it says "For IPv4, the group ID is only 28-bit long and the rest of the bits are filled with 0."  Please explain if the leading or trailing bits are to be padded with 0.

[Note]: 'Update needed to clarify status of the "u" bit in IB GUIDs, so that we know whether or not to toggle it when making IPv6 IIDs.  The IBTA needs to clarify their draft first, and work is currently underway in the IBTA link WG to do so.' added by Margaret Wasserman

[Note]: 'Update needed to clarify status of the "u" bit in IB GUIDs, so that we know whether or not to toggle it when making IPv6 IIDs.' added by Margaret Wasserman

[Note]: 'Comments posted to the mailing list on 2002-10-21 regarding all three IPoIB documents. Followup discussion has been ongoing since then. Documents considered to be in WG hands.' has been cleared by Margaret Wasserman

Comments posted to the mailing list on 2002-10-21 regarding all three IPoIB documents. Followup discussion has been ongoing since then. Documents considered to be in WG hands.