Multipath TCP (MPTCP) Application Interface Considerations
draft-ietf-mptcp-api-07

[Ballot comment]

Thanks for adding the text about TLS.

(This comment is the same as before, I can't recall if
we/you decided it was worth thinking about or not;-)

- Would it be useful to add a security consideration to the
effect that applications may need to revisit some
assumptions about the sockets API that could impact on
security? Not sure what text precisely but I'd expect new
buffer problems might be likely here in practice if the
size of the return from getpeername() etc changes and
not all calling code is aware that MPTCP is in use.

[Ballot discuss]

Suggest text below updated based on discussion on the TLS wg
list. [1]

  [1] http://www.ietf.org/mail-archive/web/tls/current/msg09069.html

My suggestion for this is to add words like the following to the
security considerations section:

  "Implementations of TLS [RFC5246] making use of the basic
  API that compare any addresses used by MPTCP against names
  or addresses present in X.509 certificates [RFC5280,RFC6125]
  MUST only consider the address used to start the initial subflow
  as presented to TLS. That is, the addresses used for subflows
  need not be compared against any TLS certificate information.
  A need for finer-grained controls would imply a
  need to use an advanced API."

[Ballot comment]

(This comment is the same as before, I can't recall if
we/you decided it was worth thinking about or not;-)

- Would it be useful to add a security consideration to the
effect that applications may need to revisit some
assumptions about the sockets API that could impact on
security? Not sure what text precisely but I'd expect new
buffer problems might be likely here in practice if the
size of the return from getpeername() etc changes and
not all calling code is aware that MPTCP is in use.

[Ballot discuss]

Updated, based on mail so far.

(1) cleared - the basic API doesn't have a set socket option
for new subflows to the close_notify thing isn't a problem.
(Put another way - I mis-read the draft;-)

(2) In a similar vein, if TLS with an SNI with an IP address is
being used (rfc6066 says you can't, but who knows what's done
in the wild) and/or if a TLS certificate has IP addresses in
the SAN, then should something special be done to handle that
or could it cause an unexpected failure?  Again, if text is
needed it should probably be checked with the TLS wg. This may
also affect TLS/MPTCP via a legacy API (not sure).

My suggestion for this is to add:

  "Implementations of TLS [RFC5246] making use of the basic
  API that compare the addresses used by MPTCP against names
  or addresses present in X.509 certificates [RFC5280,RFC6125]
  MUST only consider the addresses used in the initial subflow
  since MPTCP itself handles the security of subsequent
  subflows. A need for finer-grained controls would imply a
  need to use an advanced API."

But that should be checked with the TLS WG once the MPTCP
folks are ok with it, just in case.

[Ballot comment]
- Would it be useful to add a security consideration to the
effect that applications may need to revisit some
assumptions about the sockets API that could impact on
security? Not sure what text precisely but I'd expect new
buffer problems might be likely here in practice if the
size of the return from getpeername() etc changes and
not all calling code is aware that MPTCP is in use.

[Ballot discuss]
1) s3.2.*: I'm keying here on the multiple IP bit, so if I'm misunderstanding something just let me know:

- There's got to be a logging/storage impact based on mptcp?  If servers were tracking source IP addresses and now instead of one coming from a client there's two that means at least a double - right?

- There's protocols that include things like IP addresses in the received from header will it now need to include one for each subflow? 

- If certificates are used by an application and the certificates include an IP address, how many need to be included in the certificate?  Will the application know how many subflows are going to get started?  It's going to have know which subflow to give which cert?

2) I echo Stephen's concerns wrt TLS.  TLS explicitly discusses a DoS attack based on an attacker who initiates a large number of TCP connections (F.5 of RFC 5246).  Are we now making lots of people potential attackers?  Should some guidance on the # of connections to open be given?  Also, isn't this worth a mention in the security considerations.

[Ballot comment]
1) s3.1.1: The throughput assertion isn't always true -right? If you have one of the two links splitting what would have gone down one link go down then don't you have less than half the throughput of the original link - at least until the now combined subflow ramps back up?

2) s3.1.1: Should you also point out that if all the subflows collapse in to one that the overhead actually lowers the throughput compared to regular TCP?

3) s3.1.2: Wouldn't there also be a delay with the collapse of a subflow?

[Ballot comment]
I am not an expert on mptcp, but will note that from a routing point of view using a different IP address is only one way of hinting to us that you want a packet to go on a different path to the destination. If the bottle neck is not at the first hop and you are singly attached, routers can use other fields to make ECMP choices if that is of any help to you.

[Ballot comment]
1. Minor clarifications in section 5.3.1:

TCP_MULTIPATH_ADD: "add a new local address to an existing MPTCP
connection," is it allowed to add multiple addresses to an existing
connection?

Similarly: "Remove a local address from an MPTCP connection," is it
allowed to remove multiple addresses from an MPCTCP connection?

In both cases, Table 1 indicates "list of addresses"

2. Minor clarification in section 5.3.2: Table 1 defines
TCP_MULTIPATH_ENABLE as boolean, while section 5.3.2 has text:
"setting TCP_MULTIPATH_ENABLE to a value larger than 0."  For
consistency, is TCP_MULTIPATH_ENABLE a boolean or integer type?  (I
see that this comment is the same as a Comment issue raised by Martin
Stiemerling.)

[Ballot comment]
[Note: These are things I've found in my review with a couple of comments from friends in Apps. I've asked -- no, begged -- the apps-discuss list for some additional opinions as we all seem to have missed this during Last Call. Mea culpa; bad AD. I will update with additional comments should they come in. At this point, I see no showstoppers, so I'm leaving this as No Objection.]


4.2.2

This section gives me the most pause. I am worried (though not enough to hold up the document over) about how the legacy getsockname and getpeername APIs will perform. In particular:

  This problem is addressed as follows: If used by a legacy
  application, the MPTCP stack MUST always return the addresses of the
  first subflow of an MPTCP connection, in all circumstances, even if
  that particular subflow is no longer in use.

Is it possible that an IP Address/Port pair can be reused while a connection is still in progress? So, I start on 1.2.3.4:23456. MPTCP makes some subflows, and then 1.2.3.4:23456 goes away. Is it possible for another process on the machine to pick up 1.2.3.4:23456 at this point?

On a different note:

  As this address may not be valid any more if the first subflow is
  closed, the MPTCP stack MAY close the whole MPTCP connection if the
  first subflow is closed (i.e. fate sharing between the initial
  subflow and the MPTCP connection as a whole).

I don't understand why this is in there. Of course, any implementation can close a connection for any reason it chooses, but that MAY makes it sound like fate sharing the initial subflow is a good idea. AFAICT, it's a terrible idea. Why is this paragraph in there?

If fate-sharing the initial subflow remains, I assume this will be configurable in the Advanced API, yes? It is not mentioned in Appendix A.

5.3.1: Why is the connection identifier only 32-bit?

[Ballot comment]
Apologies if I missed it, but is there a way using this Basic API for an application to learn that the mptcp stack it is using has accepted a new substream without polling? If not, should there be? Or was this explicitly deferred to the advanced API?

[Ballot discuss]

These ought be easy to resolve. If they're real issues, I'm
fine with making them into comments that can be resolved by the
chairs/AD or holding the discuss if that's better. If not,
then I'll be glad I checked anyway:-)

(1) Sorry if this is the wrong document for this comment, but
it only occurred to me now to check, so maybe you can say if it
applies and we can see what, if anything to do then. RFC 5246
section 7.2.1 says that once you've gotten a TLS close_notify
then you close the TLS session, and also that each side is
required to send a close_notify before closing the write side
of a TCP connection. Does that mean we missed a security
consideration in MPTCP that should have highlighted that there
may be work to be done in a TLS implementation for it to run
over MPTCP gracefully if one side wants to close some of the
subflows? Or, is that something that should/could go in here
where e.g. we say if using a basic or advanced API to implement
TLS then don't do that? If something ought go in this draft
then it'd probably be a good plan to check the text with the
TLS wg before you're done.

(2) In a similar vein, if TLS with an SNI with an IP address is
being used (rfc6066 says you can't, but who knows what's done
in the wild) and/or if a TLS certificate has IP addresses in
the SAN, then should something special be done to handle that
or could it cause an unexpected failure?  Again, if text is
needed it should probably be checked with the TLS wg. This may
also affect TLS/MPTCP via a legacy API (not sure).

[Ballot comment]

- Would it be useful to add a security consideration to the
effect that applications may need to revisit some
assumptions about the sockets API that could impact on
security? Not sure what text precisely but I'd expect new
buffer problems might be likely here in practice if the
size of the return from getpeername() etc changes and
not all calling code is aware that MPTCP is in use.

[Ballot comment]
A good document and I almost hit the 'YES' button instead of 'No Objection', but I have some comments below.

My main point is whether there is the need to define the data types for the parameters of the socket API. The implementers are free to do whatever they like. However, I can see that the data types are for information only. But this isn't so clear from the document.

Here are the comments and I would like to see a response to them, though it is mandatory to respond:

Section 5.3.1., paragraph 7:

>    Table 1 shows a list of the abstract socket operations for the basic
>    configuration of MPTCP.  The first column gives the symbolic name of
>    the operation.  The second and third columns indicate whether the
>    operation provides values to be read ("Get") or takes values to
>    configure ("Set").  The fourth column lists the type of data
>    associated with this operation.  In addition to IP addresses, an
>    application MAY also indicate TCP port numbers, as further detailed
>    below.

  I would add that the data types are listed for information only.


Section 5.3.1., paragraph 10:

>    o  TCP_MULTIPATH_ENABLE: This value SHOULD only be set before the
>      establishment of a TCP connection.  Its value SHOULD only be read
>      after the establishment of a connection.

  The usage of 'SHOULD' isn't right here, or asking differently: Can
  a MPTCP isocket return to regular TCP socket once the connection
  is established? I would propose to simply say 'should only be set'.


Section 5.3.1., paragraph 13:

>    o  TCP_MULTIPATH_SUBFLOWS: This value is read-only and SHOULD only be
>      used after connection setup.

  I am not sure that the 'SHOULD' is correct here. It is more than it
  doesn't make sense to use this before a connection has been setup,
  but there is nothing that prevents apps in doing so, but they need
  to cope with the resulting error.


Section 5.3.1., paragraph 14:

>    o  TCP_MULTIPATH_CONNID: This value is read-only and SHOULD only be
>      used after connection setup.

  Same comment as for TCP_MULTIPATH_SUBFLOWS.


Section 5.3.2., paragraph 1:

>    An application can explicitly indicate multipath capability by
>    setting TCP_MULTIPATH_ENABLE to a value larger than 0.  In this case,

  TCP_MULTIPATH_ENABLE is a boolean (at least in your above table)
  which can either take true or false, which depends on the programming
  language used, but 'by larger than 0 looks' odd, as 0 can be also true, if
  negative logic is used


Section 5.3.2., paragraph 4:

>    An application can disable MPTCP setting TCP_MULTIPATH_ENABLE to a
>    value of 0.  In that case, MPTCP MUST NOT be used on that connection.

  Again a zero, but it should read false.


Section 5.3.2., paragraph 5:

>    After connection establishment, an application can get the value of
>    TCP_MULTIPATH_ENABLE.  A value of 0 then means lack of MPTCP support.
>    Any value equal to or larger than 1 means that MPTCP is supported.

  Again integer vs. boolean values.


Section 5.3.5., paragraph 2:

>    This SHOULD be a 32-bit number, and SHOULD be the locally unique (e.
>    g., the MPTCP token).

  Is there really the need to nail down what type the connection ID
  is? And can we mandate this anyhow, as it is implementation specific
  to a node? Stating that is should locally unique seems to me enough.


Section 6.1., paragraph 2:

>    API developers MAY wish to integrate SCTP and MPTCP calls to provide
>    a consistent interface to the application.  Yet, it must be

  This 'MAY' is misplaced, as it is more a hint to implementers that
  they might consider an integrated API. This is sort of also enfored
  by the last sentence which says that this integration is anyhow
  out of scope.


Section 11.2., paragraph 1:

>    [8]  Sarolahti, P., "Multi-address Interface in the Socket API",
>          March 2010.

  Any information what the type of document this is? Draft, paper, etc?

[Ballot comment]
Nice document -- well written and interesting.  Just a few minor, non-blocking comments (happy to chat about them if you like):

-- Section 3.1 --
  3.1.  Performance Impact

  One of the key goals of adding multipath capability to TCP is to
  improve the performance of a transport connection by load
  distribution over separate subflows across potentially disjoint
  paths.  Furthermore, it is an explicit goal of MPTCP that it should
  not provide a worse performing connection than would have existed
  through the use of single-path TCP.  A corresponding congestion
  control algorithm is described in [7].  The following sections
  summarize the performance impact of MPTCP as seen by an application.

The word "impact" is a dicey one: historically, it refers to a collision.  Because of that, it has a negative connotation: something that has "impact on our schedule" is assumed to hurt the schedule, not help it.  "Performance impact" carries an implication of poorer, not better, performance.  I strongly suggest you change the section title to "Performance Improvements" (or the neutral "Effect on Performance").

I would also like to see "impact" changed to "effect" throughout the document, but perhaps the editors and WG will disagree.  Remember that these are non-blocking comments, and you don't have to change this just because I said so.

Also, "should not provide a worse..." is pretty awkward.  May I suggest, "Furthermore, it is an explicit goal of MPTCP that it provide a connection than performs at least as well as one using single-path TCP."

-- Section 4.6 --
"Socket buffet dimensioning" should, I think, refer to buffers, not buffets, tasty affairs though the latter be.

-- Section 5.3.3 --
  If no port number is
  provided by the application, the port number is automatically
  selected by the MPTCP implementation, and will usually be the same
  across all subflows.

Is it worth saying any more about this?  If not, what's the value of knowing that the port number will "usually" be the same, if it's not behaviour that applications can rely on?

  It should be noted that this signal is only a hint, and an MPTCP
  implementation MAY only use a subset of the addresses.

This is fine, but might be mis-read: someone might take "MAY only use" to mean "is only allowed to use" (as though it were "MUST only use").  You might consider re-wording this to say, "MAY use only a subset", or, maybe better, "MAY select only a subset".

-- Section 10 --
  The views expressed
  here are those of the author(s) only.

I normally wouldn't quibble about anything written in an Acknowledgments section, and I'm sure this is boilerplate text that comes from elsewhere.  But it's patently false (and I'm on the fence about whether this should be a DISCUSS): what's expressed in this document comes from the MPTCP working group, and, indeed, the IETF as a whole, not from the authors only.  Is there a way you can fix this text that would be acceptable to the BMBF, the Trilogy Project, etc?  Maybe, "The views expressed here are those of the IETF," or, "The views expressed here are those of the Internet community." ?

IANA has reviewed ietf-mptcp-api-05, which is currently in Last Call, and has the following comments:

IANA understands that, upon approval of this document, there are no IANA Actions that need completion.

The following Last Call announcement was sent out:

From: The IESG
To: IETF-Announce
CC:
Reply-To: ietf@ietf.org
Subject: Last Call:  (MPTCP Application Interface Considerations) to Informational RFC


The IESG has received a request from the Multipath TCP WG (mptcp) to
consider the following document:
- 'MPTCP Application Interface Considerations'
  as Informational RFC

The IESG plans to make a decision in the next few weeks, and solicits
final comments on this action. Please send substantive comments to the
ietf@ietf.org mailing lists by 2012-08-14. Exceptionally, comments may be
sent to iesg@ietf.org instead. In either case, please retain the
beginning of the Subject line to allow automated sorting.

Abstract


  Multipath TCP (MPTCP) adds the capability of using multiple paths to
  a regular TCP session.  Even though it is designed to be totally
  backward compatible to applications, the data transport differs
  compared to regular TCP, and there are several additional degrees of
  freedom that applications may wish to exploit.  This document
  summarizes the impact that MPTCP may have on applications, such as
  changes in performance.  Furthermore, it discusses compatibility
  issues of MPTCP in combination with non-MPTCP-aware applications.
  Finally, the document describes a basic application interface which
  is a simple extension of TCP's interface for MPTCP-aware
  applications.




The file can be obtained via
http://datatracker.ietf.org/doc/draft-ietf-mptcp-api/

IESG discussion can be tracked via
http://datatracker.ietf.org/doc/draft-ietf-mptcp-api/ballot/


No IPR declarations have been submitted directly on this I-D.

(1) What type of RFC is being requested (BCP, Proposed Standard,
Internet Standard, Informational, Experimental, or Historic)?  Why
is this the proper type of RFC?  Is this type of RFC indicated in the
title page header?


Informational, as indicated in the header.
The document has two main parts. Firstly it discusses the potential impact of MPTCP on application performance; this is clearly Informational.
It also describes a basic application interface for MPTCP-aware applications. Almost all previous API documents are Informational rather than Experimental, so we continue the general custom (eg RFC6458).



(2) The IESG approval announcement includes a Document Announcement
Write-Up. Please provide such a Document Announcement Write-Up. Recent
examples can be found in the "Action" announcements for approved
documents. The approval announcement contains the following sections:


Technical Summary
  Multipath TCP (MPTCP) adds the capability of using multiple paths to
  a regular TCP session.  Even though it is designed to be totally
  backward compatible to applications, the data transport differs
  compared to regular TCP, and there are several additional degrees of
  freedom that applications may wish to exploit.  This document
  summarizes the impact that MPTCP may have on applications, such as
  changes in performance.  Furthermore, it discusses compatibility
  issues of MPTCP in combination with non-MPTCP-aware applications.
  Finally, the document describes a basic application interface which
  is a simple extension of TCP's interface for MPTCP-aware
  applications.

Working Group Summary

The document was not particularly controversial. There was very good consensus in the WG about the document. The document has been stable for quite a while, since it was decided to advance it in parallel with the MPTCP protocol.

Document Quality

The dcoument has been reviewed within the WG. Michael Tüxen (SCTP author) provided an expert review. There was also a WG last call, which resulted in some clarifications.
An implementation of the protocol is publicly available, and other implementations are in progress. The API for MPTCP-aware applications has not been implemented afaik.

Personnel

The document shepherd is Philip Eardley. The Responsible Area Director is Wes Eddy.




(3) Briefly describe the review of this document that was performed by
the Document Shepherd.  If this version of the document is not ready
for publication, please explain why the document is being forwarded to
the IESG.

I've reviewed the document, resulting in a few points being clarified. We also discussed with the AD whether the document should be Informational or Experimental, with the former agreed as more suitable.
I believe the document is ready for publication.



(4) Does the document Shepherd have any concerns about the depth or
breadth of the reviews that have been performed? 


I have no concerns about the reviews.


(5) Do portions of the document need review from a particular or from
broader perspective, e.g., security, operational complexity, AAA, DNS,
DHCP, XML, or internationalization? If so, describe the review that
took place.


I don't believe that any portions need particular reviews.



(6) Describe any specific concerns or issues that the Document Shepherd
has with this document that the Responsible Area Director and/or the
IESG should be aware of? For example, perhaps he or she is uncomfortable
with certain parts of the document, or has concerns whether there really
is a need for it. In any event, if the WG has discussed those issues and
has indicated that it still wishes to advance the document, detail those
concerns here.


I have no concerns with the document.


(7) Has each author confirmed that any and all appropriate IPR
disclosures required for full conformance with the provisions of BCP 78
and BCP 79 have already been filed. If not, explain why.


Yes.



(8) Has an IPR disclosure been filed that references this document?
If so, summarize any WG discussion and conclusion regarding the IPR
disclosures.


I don't know of any IPR disclosure.


(9) How solid is the WG consensus behind this document? Does it
represent the strong concurrence of a few individuals, with others
being silent, or does the WG as a whole understand and agree with it? 


There is strong WG consensus behind the document, It has been presented several times at WG meetings and has been stable (apart from clarifications) for quite some time. I believe it is widely understood amongst the WG and agreed with.



(10) Has anyone threatened an appeal or otherwise indicated extreme
discontent? If so, please summarise the areas of conflict in separate
email messages to the Responsible Area Director. (It should be in a
separate email because this questionnaire is publicly available.)


No-one has indicated discontent.


(11) Identify any ID nits the Document Shepherd has found in this
document. (See http://www.ietf.org/tools/idnits/ and the Internet-Drafts
Checklist). Boilerplate checks are not enough; this check needs to be
thorough.


The document has no nits, except warnings that the document date is >50days old, and a reference to an older version of draft-ietf-mptcp-multiaddressed. In the latter case, note that we propose that draft-ietf-mptcp-multiaddressed and this document (draft-ietf-mptcp-api) advance at the same time to the IESG and to RFC.


(12) Describe how the document meets any required formal review
criteria, such as the MIB Doctor, media type, and URI type reviews.


I believe that no formal review is needed.



(13) Have all references within this document been identified as
either normative or informative?

Yes.



(14) Are there normative references to documents that are not ready for
advancement or are otherwise in an unclear state? If such normative
references exist, what is the plan for their completion?


Yes, normative reference to  draft-ietf-mptcp-multiaddressed. They advance to the IESG at the same time.


(15) Are there downward normative references references (see RFC 3967)?
If so, list these downward references to support the Area Director in the
Last Call procedure.


No.



(16) Will publication of this document change the status of any
existing RFCs? Are those RFCs listed on the title page header, listed
in the abstract, and discussed in the introduction? If the RFCs are not
listed in the Abstract and Introduction, explain why, and point to the
part of the document where the relationship of this document to the
other RFCs is discussed. If this information is not in the document,
explain why the WG considers it unnecessary.


No - won't change the status of existing RFCs.



(17) Describe the Document Shepherd's review of the IANA considerations
section, especially with regard to its consistency with the body of the
document. Confirm that all protocol extensions that the document makes
are associated with the appropriate reservations in IANA registries.
Confirm that any referenced IANA registries have been clearly
identified. Confirm that newly created IANA registries include a
detailed specification of the initial contents for the registry, that
allocations procedures for future registrations are defined, and a
reasonable name for the new registry has been suggested (see RFC 5226).


This document has no IANA actions. I believe this is consistent with the body of the document.



(18) List any new IANA registries that require Expert Review for future
allocations. Provide any public guidance that the IESG would find
useful in selecting the IANA Experts for these new registries.


None.


(19) Describe reviews and automated checks performed by the Document
Shepherd to validate sections of the document written in a formal
language, such as XML code, BNF rules, MIB definitions, etc.


None; not applicable.