Remote Direct Memory Access Transport for Remote Procedure Call, Version One
draft-ietf-nfsv4-rfc5666bis-05

Internet-Draft          RPC-Over-RDMA Version One             April 2016

9.2.2.3.  RPC-Over-RDMA With RPCSEC_GSS Integrity Or Privacy

   The RPCSEC_GSS integrity service enables endpoints to detect
   modification of RPC messages in flight.  The RPCSEC_GSS privacy
   service prevents all but the intended recipient from viewing the
   cleartext content of RPC arguments and results.  RPCSEC_GSS integrity
   and privacy are end-to-end.  They protect RPC arguments and results
   from application to server endpoint, and back.

   The RPCSEC_GSS integrity and encryption services operate on whole RPC
   messages after they have been XDR encoded for transmit, and before
   they have been XDR decoded after receipt.  Both sender and receiver
   endpoints use intermediate buffers to prevent exposure of encrypted
   data or unverified cleartext data to RPC consumers.  After
   verification, encryption, and message wrapping has been performed,
   the transport layer MAY use RDMA data transfer between these
   intermediate buffers.

   The process of reducing a DDP-eligible data item removes the data
   item and its XDR padding from the encoded XDR stream.  XDR padding of
   a reduced data item is not transferred in an RPC-over-RDMA message.
   After reduction, the Payload stream contains fewer octets then the
   whole XDR stream did beforehand.  XDR padding octets are often zero
   bytes, but they don't have to be.  Thus reducing DDP-eligible items
   affects the result of message integrity verification or encryption.

   Therefore a sender MUST NOT reduce a Payload stream when RPCSEC_GSS
   integrity or encryption services are in use.  Effectively, no data
   item is DDP-eligible in this situation, and Chunked Messages cannot
   be used.  In this mode, an RPC-over-RDMA transport operates in the
   same manner as a transport that does not support direct data
   placement.

   When RPCSEC_GSS integrity or privacy is in use, a requester provides
   both a Read list and a Reply chunk in the same RPC-over-RDMA header
   to convey a Long call and provision a receptacle for a Long reply.

9.2.2.4.  Protecting RPC-Over-RDMA Transport Headers

   Like the base fields in an ONC RPC message (XID, call direction, and
   so on), the contents of an RPC-over-RDMA message's Transport stream
   are not protected by RPCSEC_GSS.  This exposes XIDs, connection
   credit limits, and chunk lists (but not the content of the data items
   they refer to) to malicious behavior, which could redirect data that
   is transferred by the RPC-over-RDMA message, result in spurious
   retransmits, or trigger connection loss.

Lever, et al.           Expires October 10, 2016               [Page 47]
Internet-Draft          RPC-Over-RDMA Version One             April 2016

   In particular, if an attacker alters the information contained in the
   chunk lists of an RPC-over-RDMA header, data contained in those
   chunks can be redirected to other registered memory segments on
   requesters.  An attacker might alter the arguments of RDMA Read and
   RDMA Write operations on the wire to similar effect.  The use of
   RPCSEC_GSS integrity or privacy services enable the requester to
   detect if such tampering has been done and reject the RPC message.

   Encryption at lower layers, as described in Section 9.2.1, protects
   the content of the Transport stream.  To address attacks on RDMA
   protocols themselves, RDMA transport implementations should conform
   to [RFC5042].

10.  IANA Considerations

   Three assignments are specified by this document.  These are
   unchanged from [RFC5666]:

   o  A set of RPC "netids" for resolving RPC-over-RDMA services

   o  Optional service port assignments for Upper Layer Bindings

   o  An RPC program number assignment for the configuration protocol

   These assignments have been established, as below.

   The new RPC transport has been assigned an RPC "netid", which is an
   rpcbind [RFC1833] string used to describe the underlying protocol in
   order for RPC to select the appropriate transport framing, as well as
   the format of the service addresses and ports.

   The following "Netid" registry strings are defined for this purpose:

      NC_RDMA "rdma"
      NC_RDMA6 "rdma6"

   These netids MAY be used for any RDMA network satisfying the
   requirements of Section 3.2.2, and able to identify service endpoints
   using IP port addressing, possibly through use of a translation
   service as described above in Section 6.  The "rdma" netid is to be
   used when IPv4 addressing is employed by the underlying transport,
   and "rdma6" for IPv6 addressing.

   The netid assignment policy and registry are defined in [RFC5665].

Lever, et al.           Expires October 10, 2016               [Page 48]
Internet-Draft          RPC-Over-RDMA Version One             April 2016

   As a new RPC transport, this protocol has no effect on RPC Program
   numbers or existing registered port numbers.  However, new port
   numbers MAY be registered for use by RPC-over-RDMA-enabled services,
   as appropriate to the new networks over which the services will
   operate.

   For example, the NFS/RDMA service defined in [RFC5667] has been
   assigned the port 20049, in the IANA registry:

      nfsrdma 20049/tcp Network File System (NFS) over RDMA
      nfsrdma 20049/udp Network File System (NFS) over RDMA
      nfsrdma 20049/sctp Network File System (NFS) over RDMA

   The RPC program number assignment policy and registry are defined in
   [RFC5531].

11.  Acknowledgments

   The editor gratefully acknowledges the work of Brent Callaghan and
   Tom Talpey on the original RPC-over-RDMA Version One specification
   [RFC5666].

   Dave Noveck provided excellent review, constructive suggestions, and
   consistent navigational guidance throughout the process of drafting
   this document.  Dave also contributed much of the organization and
   content of Section 8 and helped the authors understand the
   complexities of XDR extensibility.

   The comments and contributions of Karen Deitke, Dai Ngo, Chunli
   Zhang, Dominique Martinet, and Mahesh Siddheshwar are accepted with
   great thanks.  The editor also wishes to thank Bill Baker, Greg
   Marsden, and Matt Benjamin for their support of this work.

   The extract.sh shell script and formatting conventions were first
   described by the authors of the NFSv4.1 XDR specification [RFC5662].

   Special thanks go to nfsv4 Working Group Chair Spencer Shepler and
   nfsv4 Working Group Secretary Thomas Haynes for their support.

12.  References

12.1.  Normative References

Lever, et al.           Expires October 10, 2016               [Page 49]
Internet-Draft          RPC-Over-RDMA Version One             April 2016

   [I-D.ietf-nfsv4-rpcrdma-bidirection]
              Lever, C., "Size-Limited Bi-directional Remote Procedure
              Call On Remote Direct Memory Access Transports", draft-
              ietf-nfsv4-rpcrdma-bidirection-01 (work in progress),
              September 2015.

   [I-D.ietf-nfsv4-rpcsec-gssv3]
              Adamson, A. and N. Williams, "Remote Procedure Call (RPC)
              Security Version 3", draft-ietf-nfsv4-rpcsec-gssv3-17
              (work in progress), January 2016.

   [RFC1833]  Srinivasan, R., "Binding Protocols for ONC RPC Version 2",
              RFC 1833, DOI 10.17487/RFC1833, August 1995,
              <http://www.rfc-editor.org/info/rfc1833>.

   [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
              Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/
              RFC2119, March 1997,
              <http://www.rfc-editor.org/info/rfc2119>.

   [RFC4506]  Eisler, M., Ed., "XDR: External Data Representation
              Standard", STD 67, RFC 4506, DOI 10.17487/RFC4506, May
              2006, <http://www.rfc-editor.org/info/rfc4506>.

   [RFC5042]  Pinkerton, J. and E. Deleganes, "Direct Data Placement
              Protocol (DDP) / Remote Direct Memory Access Protocol
              (RDMAP) Security", RFC 5042, DOI 10.17487/RFC5042, October
              2007, <http://www.rfc-editor.org/info/rfc5042>.

   [RFC5056]  Williams, N., "On the Use of Channel Bindings to Secure
              Channels", RFC 5056, DOI 10.17487/RFC5056, November 2007,
              <http://www.rfc-editor.org/info/rfc5056>.

   [RFC5531]  Thurlow, R., "RPC: Remote Procedure Call Protocol
              Specification Version 2", RFC 5531, DOI 10.17487/RFC5531,
              May 2009, <http://www.rfc-editor.org/info/rfc5531>.

   [RFC5660]  Williams, N., "IPsec Channels: Connection Latching", RFC
              5660, DOI 10.17487/RFC5660, October 2009,
              <http://www.rfc-editor.org/info/rfc5660>.

   [RFC5665]  Eisler, M., "IANA Considerations for Remote Procedure Call
              (RPC) Network Identifiers and Universal Address Formats",
              RFC 5665, DOI 10.17487/RFC5665, January 2010,
              <http://www.rfc-editor.org/info/rfc5665>.

Lever, et al.           Expires October 10, 2016               [Page 50]
Internet-Draft          RPC-Over-RDMA Version One             April 2016

12.2.  Informative References

   [IB]       InfiniBand Trade Association, "InfiniBand Architecture
              Specifications", <http://www.infinibandta.org>.

   [IBPORT]   InfiniBand Trade Association, "IP Addressing Annex",
              <http://www.infinibandta.org>.

   [RFC0768]  Postel, J., "User Datagram Protocol", STD 6, RFC 768, DOI
              10.17487/RFC0768, August 1980,
              <http://www.rfc-editor.org/info/rfc768>.

   [RFC0793]  Postel, J., "Transmission Control Protocol", STD 7, RFC
              793, DOI 10.17487/RFC0793, September 1981,
              <http://www.rfc-editor.org/info/rfc793>.

   [RFC1094]  Nowicki, B., "NFS: Network File System Protocol
              specification", RFC 1094, DOI 10.17487/RFC1094, March
              1989, <http://www.rfc-editor.org/info/rfc1094>.

   [RFC1813]  Callaghan, B., Pawlowski, B., and P. Staubach, "NFS
              Version 3 Protocol Specification", RFC 1813, DOI 10.17487/
              RFC1813, June 1995,
              <http://www.rfc-editor.org/info/rfc1813>.

   [RFC5040]  Recio, R., Metzler, B., Culley, P., Hilland, J., and D.
              Garcia, "A Remote Direct Memory Access Protocol
              Specification", RFC 5040, DOI 10.17487/RFC5040, October
              2007, <http://www.rfc-editor.org/info/rfc5040>.

   [RFC5041]  Shah, H., Pinkerton, J., Recio, R., and P. Culley, "Direct
              Data Placement over Reliable Transports", RFC 5041, DOI
              10.17487/RFC5041, October 2007,
              <http://www.rfc-editor.org/info/rfc5041>.

   [RFC5532]  Talpey, T. and C. Juszczak, "Network File System (NFS)
              Remote Direct Memory Access (RDMA) Problem Statement", RFC
              5532, DOI 10.17487/RFC5532, May 2009,
              <http://www.rfc-editor.org/info/rfc5532>.

   [RFC5661]  Shepler, S., Ed., Eisler, M., Ed., and D. Noveck, Ed.,
              "Network File System (NFS) Version 4 Minor Version 1
              Protocol", RFC 5661, DOI 10.17487/RFC5661, January 2010,
              <http://www.rfc-editor.org/info/rfc5661>.

Lever, et al.           Expires October 10, 2016               [Page 51]
Internet-Draft          RPC-Over-RDMA Version One             April 2016

   [RFC5662]  Shepler, S., Ed., Eisler, M., Ed., and D. Noveck, Ed.,
              "Network File System (NFS) Version 4 Minor Version 1
              External Data Representation Standard (XDR) Description",
              RFC 5662, DOI 10.17487/RFC5662, January 2010,
              <http://www.rfc-editor.org/info/rfc5662>.

   [RFC5666]  Talpey, T. and B. Callaghan, "Remote Direct Memory Access
              Transport for Remote Procedure Call", RFC 5666, DOI
              10.17487/RFC5666, January 2010,
              <http://www.rfc-editor.org/info/rfc5666>.

   [RFC5667]  Talpey, T. and B. Callaghan, "Network File System (NFS)
              Direct Data Placement", RFC 5667, DOI 10.17487/RFC5667,
              January 2010, <http://www.rfc-editor.org/info/rfc5667>.

   [RFC7530]  Haynes, T., Ed. and D. Noveck, Ed., "Network File System
              (NFS) Version 4 Protocol", RFC 7530, DOI 10.17487/RFC7530,
              March 2015, <http://www.rfc-editor.org/info/rfc7530>.

Authors' Addresses

   Charles Lever (editor)
   Oracle Corporation
   1015 Granger Avenue
   Ann Arbor, MI  48104
   USA

   Phone: +1 734 274 2396
   Email: chuck.lever@oracle.com

   William Allen Simpson
   DayDreamer
   1384 Fontaine
   Madison Heights, MI  48071
   USA

   Email: william.allen.simpson@gmail.com

   Tom Talpey
   Microsoft Corp.
   One Microsoft Way
   Redmond, WA  98052
   USA

   Phone: +1 425 704-9945
   Email: ttalpey@microsoft.com

Lever, et al.           Expires October 10, 2016               [Page 52]