TCP Extensions for High Performance
draft-ietf-tcpm-1323bis-21

[Ballot comment]
This was a very helpful key document for the TCP protocol. Thanks for producing it.

I had some questions while reading the document. Please consider them along with any other comments you receive.

In this text:

1.  Introduction

  For brevity, the full discussions of the merits and history behind
  the TCP options defined within this document have been omitted.
  [RFC1323] should be consulted for reference.  It is recommended that
  a modern TCP stack implements and make use of the extensions
    ^^^^^^^^^^^^^^^^
  described in this document.

I don't know what "a modern TCP stack" will mean in a decade or so (RFC 1323 was published more than two decades ago, so ...).

Is there another way to say what you want to say?

In this text:

1.3.  Using TCP options

  Research has shown that the use of the Timestamps option to take
  additional RTT samples within each RTT has little effect on the
  ultimate retransmission timeout value [Allman99].  However, there are
  other uses of the Timestamps option, such as the Eifel mechanism
  [RFC3522], [RFC4015], and PAWS (see Section 5) which improve overall
  TCP security and performance.  The extra header bandwidth used by
  this option should be evaluated for the gains in performance and
  security in an actual deployment.

Does the evaluation suggested in the last sentence happen in practice? I'm also looking a bit further down at this text:

  A TCP vendor concerned about optimal performance
  over low-speed paths might consider turning these extensions off for
  low- speed paths, or allow a user or installation manager to disable
  them.

I guess there are TCP vendors who care _only_ about optimal performance over low-speed paths, but does that happen in practice? Maybe no reason to change anything, but ... mumble.

In this text:

3.2.  Timestamps option

  TCP is a symmetric protocol, allowing data to be sent at any time in
  either direction, and therefore timestamp echoing may occur in either
  direction.  For simplicity and symmetry, we specify that timestamps
  always be sent and echoed in both directions.  For efficiency, we
  combine the timestamp and timestamp reply fields into a single TCP
  Timestamps option.

I'm pretty sure I know what you mean, but I struggled a bit between "may occur" and "always be sent and echoed". Is it correct to say "we specify that when timestamps are used, they can be sent and echoed in both directions using a single TCP Timestamps option that combines both timestamp and reply fields for efficiency"?

In this text:

5.5.  Outdated Timestamps

  However, the TCP specification has never included keep-alives, so the
  solution based upon invalidation was chosen.)

I'm reading http://tools.ietf.org/html/rfc1122#section-4.2.3.6, and I'm confused. Is the point that the TCP specification has never mandated the use of keep-alives? Or that RFC 1122 doesn't count as part of "the TCP specification" (but it's in http://tools.ietf.org/html/rfc4614)? Could you help me understand what's going on here?

In this text:

7.  Security Considerations

  The TCP sequence space is a fixed size, and as the window becomes
  larger it becomes easier for an attacker to generate forged packets
  that can fall within the TCP window, and be accepted as valid
  segments.  While use of timestamps and PAWS can help to mitigate
  this, when using PAWS, if an attacker is able to forge a packet that
  is acceptable to the TCP connection, a timestamp that is in the
  future would cause valid segments to be dropped due to PAWS checks.
  Hence, implementers should take care to not open the TCP window
  drastically beyond the requirements of the connection.

Is "the requirements of the connection" clear to implementers? If I'm using TCP for bulk transfers and always have additional segments to send, is it obvious what implementers should be looking at?

[Ballot comment]
Below is Fred Baker's OPS DIR review. It's so nicely done that I wanted to copy it here.
Mainly FYI, except maybe one final check related to some pre-2008 text, as Fred mentions the document is ready.

I have reviewed this document as part of the Operational directorate's ongoing effort to review all IETF documents being processed by the IESG. These comments were written primarily for the benefit of the operational area directors. Document editors and WG chairs should treat these comments just like any other last call comments.

As background, I'll note that RFC 1323, which draft-ietf-tcpm-1323bis replaces, has quite of bit of history in the Internet; when I look at traces, the question isn't whether window scaling and timestamps are used, but the scaling factor in use. The authors are recognized experts in TCP, having among them early contributors to the specification and a more recent worker with significant accomplishments. The bibliography of the document cites significant research behind the recommendations, and an rfcdiff shows changes, minor or major, throughout the document. As near as I can tell, the specification documents current practice in deployed TCP implementations and/or makes language more accessible (such as changing the variable in which the window scale value is stored from "scale" to "shift"; the operation is to shift the advertised window left the stated number of bits).

idnits notes the possibility of pre-2008 text and recommends ensuring that the authors of any such text sign off on its publication. The first version of the draft is indeed dated January 2008, and it replaces draft-borman-1323bis, which is a 2007 document. David is a principle author on this document, so I think the concern is likely unwarranted, but it would be worthwhile for the authors to ask themselves that question and deal with it as it deserves.

Important information is included in the document regarding the recommendations of RFC 1323 and other documents, and how they have worked out in practice over time. For example, it was originally thought that including a timestamp in all or many TCP segments would improve estimates of the retransmission timeout. It turns out that, while they help, they don't help as much as expected. Hence, a timestamp once or twice per RTT probably produces the same benefit as a timestamp option on every packet. These observations contribute enormously, in my opinion, to the value of the document.

Security Considerations and Privacy Considerations are included; RFC 1323 didn't consider security. The sections appear well thought through and reach essentially the same conclusions that I would were I writing them. They include not only end to end considerations, but considerations related to middle boxes, which are an operational aspect of the Internet that the IETF likes to overlook.

Regarding the questions from RFC 5706:

The document is grounded in significant operational experience with well-understood and widely-used code. It has not had scaling issues in deployment; it is in fact a response to scaling issues. It does not change the options or their interpretation, so co-existence should not be an issue.

Configuration is not discussed in the document per se. In implementations I am aware of, configuration varies. In some, there is simply a standard value used for all peers; in others, different settings are used on a per-address or per-prefix basis derived either from history or manual settings. There is really only three values that *can* be set: whether or not scaling is in use, a recommended shift value (which may or may not differ based on the peer), and whether or not timestamps are in use. I believe that this is adequate for known purposes.

With respect to the other questions in the list, in my opinion the answer is "it is satisfactory".

I consider the document to be "Ready" for IESG consideration.

[Ballot comment]
In 2.3:
  o  The Window Scale option MUST be sent with shift.cnt = R, where R
      is the value that the TCP would like to use for its receive
      window.

Don't you mean "use for scaling its receive window?"  It's unlikely the reader will get this wrong, but the text doesn't really seem to say what is clearly intended.

It might make sense to divide section 7.1 into more sections.  For instance, why isn't the bit on middleboxes a subsection of section 7?

Also, I didn't see mention of a middlebox failure mode I've heard reported: some middleboxes set the window scale to zero in the SYN but leave the option in the header, so that the other end sees it as a willingness to negotiate, but records the wrong value for the sender window size as a consequence.  I don't know if this is worth mentioning, but I was a bit surprised not to see it there.  Perhaps this is so damaging that middleboxes that do this have all been fixed?

All in all this is a very clear and informative document.  Thanks for doing it!

[Ballot comment]
In 2.3:
  o  The Window Scale option MUST be sent with shift.cnt = R, where R
      is the value that the TCP would like to use for its receive
      window.

Don't you mean "use for scaling its receive window?"  It's unlikely the reader will get this wrong, but the text doesn't really seem to say what is clearly intended.

It might make sense to divide section 7.1 into more sections.  For instance, why isn't the bit on middleboxes a subsection of section 7?

Also, I didn't see mention of a middlebox failure mode I've heard reported: some middleboxes set the window scale to zero i the SYN but leave the option in the header, so that the other end sees it as a willingness to negotiate, but records the wrong value for the sender window size as a consequence.  I don't know if this is worth mentioning, but I was a bit surprised not to see it there.  Perhaps this is so damaging that middleboxes that do this have all been fixed?

All in all this is a very clear and informative document.  Thanks for doing it!

[Ballot comment]
1) In Sec 5.6, it says " However, this probabilistic argument is not universally accepted, and
  the consensus at present is that the performance gain does not
  justify the hazard in the general case.  It is therefore recommended
  that H2 follow H1."

IMHO, a problem with the probabilistic argument is that it ignores the possibility of an attacker deliberately
sending the problem packet.  Has that been considered or is it just not a concern?

[Ballot discuss]
Section 7:

"It is therefore RECOMMENDED to generate a random, per-connection offset to be used
  with the clock source when generating the Timestamps option value
  (see Section 5.4)."
 
Why not make this MUST, if it's better for both security and privacy? What would the exceptional case be where adding a random offset shouldn't be done?

[Ballot comment]
Thanks for addressing the SecDir comments.

I just found a nit to be addressed:

Nit: Section 4.1 - The RTT acronym should be spelled out in the previous sentence instead of the one it is currently.  Here are the two sentences:

  Values of this clock MUST be at least approximately
  proportional to real time, in order to measure actual RTT.

  TCP measures the round trip time (RTT), primarily for the purpose of
  arriving at a reasonable value for the Retransmission Timeout (RTO)
  timer interval.

IESG/Authors/WG Chairs:

IANA has reviewed draft-ietf-tcpm-1323bis-19.  Authors should review the comments and/or questions below.  Please report any inaccuracies and respond to any questions as soon as possible.

We received the following comments/questions from the IANA's reviewer:

IANA understands that, upon approval of this document, there are two
actions which IANA must complete.

Action 1:
In the TCP Option Kind Numbers subregistry of the Transmission Control Protocol (TCP) Parameters located at:

http://www.iana.org/assignments/tcp-parameters/

The reference for TCP Option Kind Numbers 3 and 8 will be made to
point to [ RFC-to-be ].

Action 2: This document updates the "Meanings" for TCP Option Kind
Numbers 3 and 8 in the TCP Option Kind Numbers subregistry as follows:

OLD:
3 3 WSOPT - Window Scale
8 10 TSOPT - Time Stamp Option

NEW:
3 3 WSOPT - Window Scale Option
8 10 TSOPT - Timestamps Option

IANA understands that this is the only action required to be completed upon approval of this document.

Note:  The actions requested in this document will not be completed
until the document has been approved for publication as an RFC.
This message is only to confirm what actions will be performed.

The following Last Call announcement was sent out:

From: The IESG
To: IETF-Announce
CC:
Reply-To: ietf@ietf.org
Sender:
Subject: Last Call:  (TCP Extensions for High Performance) to Proposed Standard


The IESG has received a request from the TCP Maintenance and Minor
Extensions WG (tcpm) to consider the following document:
- 'TCP Extensions for High Performance'
  as Proposed Standard

The IESG plans to make a decision in the next few weeks, and solicits
final comments on this action. Please send substantive comments to the
ietf@ietf.org mailing lists by 2014-02-18. Exceptionally, comments may be
sent to iesg@ietf.org instead. In either case, please retain the
beginning of the Subject line to allow automated sorting.

Abstract


  This document specifies a set of TCP extensions to improve
  performance over paths with a large bandwidth * delay product and to
  provide reliable operation over very high-speed paths.  It defines
  the TCP Window Scale (WS) option and the TCP Timestamps (TS) option
  and their semantics.  The Window Scale option is used to support
  larger receive windows, while the Timestamps option can be used for
  at least two distinct mechanisms, PAWS (Protection Against Wrapped
  Sequences) and RTTM (Round Trip Time Measurement), that are also
  described herein.

  This document obsoletes RFC1323 and describes changes from it.




The file can be obtained via
http://datatracker.ietf.org/doc/draft-ietf-tcpm-1323bis/

IESG discussion can be tracked via
http://datatracker.ietf.org/doc/draft-ietf-tcpm-1323bis/ballot/


No IPR declarations have been submitted directly on this I-D.

(1) What type of RFC is being requested (BCP, Proposed Standard,
Internet Standard, Informational, Experimental, or Historic)?  Why
is this the proper type of RFC?  Is this type of RFC indicated in the
title page header?

Proposed Standard. This draft replaces earlier RFC 1323 that
also was a Proposed Standard. The title page header currently
says "Standards Track".


(2) The IESG approval announcement includes a Document Announcement
Write-Up. Please provide such a Document Announcement Write-Up. Recent
examples can be found in the "Action" announcements for approved
documents. The approval announcement contains the following sections:

Technical Summary

This document replaces RFC 1323, making minor fixes and
clarifications to the original document. The document
specifies three performance enhancing extensions to TCP, using
two TCP options. The first is window scale option, that allows
representation of receive window sizes larger than 2^16 bytes
originally allowed by the 16-bit window field. The second
option is TCP timestamps option that allows round-trip time
measurement on each TCP segment, and third is an algorithm to
detect and reject old duplicate segments that happen to match
the current TCP window (Protect Against Wrapped Sequence
numbers).

Working Group Summary

This document has been a chartered TCPM working group item
since year 2008. It has not been under significant
controversy, but the progress has been slow because of earlier
lack of WG (and authoring) cycles. Since adding a new
co-editor, the progress on the draft became faster in the
working group.

Document Quality

The predecessor of this document, RFC 1323, was published in
1992, and is deployed in most TCP implementations. This
document includes fixes and clarifications based on the gained
deployment experience. The recent versions of the document
have been reviewed and discussed by multiple working group
participants.

Personnel

Document Shepherd is Pasi Sarolahti.
Reponsible Area Director is Martin Stiemerling

(3) Briefly describe the review of this document that was performed by
the Document Shepherd.  If this version of the document is not ready
for publication, please explain why the document is being forwarded to
the IESG.

The document shepherd has read the latest version of the
document and the recent mailing list discussion, and thinks
the document is ready for publication.

(4) Does the document Shepherd have any concerns about the depth or
breadth of the reviews that have been performed?

No. 

(5) Do portions of the document need review from a particular or from
broader perspective, e.g., security, operational complexity, AAA, DNS,
DHCP, XML, or internationalization? If so, describe the review that
took place.

No.

(6) Describe any specific concerns or issues that the Document Shepherd
has with this document that the Responsible Area Director and/or the
IESG should be aware of? For example, perhaps he or she is uncomfortable
with certain parts of the document, or has concerns whether there really
is a need for it. In any event, if the WG has discussed those issues and
has indicated that it still wishes to advance the document, detail those
concerns here.

Document shepherd does not have concerns about the document

(7) Has each author confirmed that any and all appropriate IPR
disclosures required for full conformance with the provisions of BCP 78
and BCP 79 have already been filed. If not, explain why.

Not all authors responded when queried about this (most of
them are not actively participating IETF these days). The
editor of the draft confirms this on his part, however.

It should be noted the original RFC 1323 was dated in 1992,
and the documented mechanism has been included in various
implementations for a long time. No IPR disclosures on RFC
1323 are known.

(8) Has an IPR disclosure been filed that references this document?
If so, summarize any WG discussion and conclusion regarding the IPR
disclosures.

No IPR disclosures have been filed on this document

(9) How solid is the WG consensus behind this document? Does it
represent the strong concurrence of a few individuals, with others
being silent, or does the WG as a whole understand and agree with it? 

The recent versions of the document were extensively discussed
on the TCPM mailing list, by several members of the community,
for example regarding the normative language about how to
handle packets without TCP timestamps option after use of
timestamps has been negotiated (which is considered a corner
case in today's implementations).  The current approach is not
unanimously supported by all working group participants, but
the WG chairs think it represents the rough consensus of the
WG.

There were also some opinions suggesting that some of the
benefits could be achieved with less TCP option overhead. The
WG chairs concluded that considering the original scope of
the document (fixes and clarifications to RFC 1323), and its
large existing deployment base, it is important to publish the
current document, but keep the discussion open for future
enhancements.

(10) Has anyone threatened an appeal or otherwise indicated extreme
discontent? If so, please summarise the areas of conflict in separate
email messages to the Responsible Area Director. (It should be in a
separate email because this questionnaire is publicly available.)

No.

(11) Identify any ID nits the Document Shepherd has found in this
document. (See http://www.ietf.org/tools/idnits/ and the Internet-Drafts
Checklist). Boilerplate checks are not enough; this check needs to be
thorough.

There are informational references to obsolete RFCs 1072 and
RFC 1185. These are left in for documenting the history behind
this document, and are included intentionally.

(12) Describe how the document meets any required formal review
criteria, such as the MIB Doctor, media type, and URI type reviews.

No formal review required.

(13) Have all references within this document been identified as
either normative or informative?

Yes.

(14) Are there normative references to documents that are not ready for
advancement or are otherwise in an unclear state? If such normative
references exist, what is the plan for their completion?

No.

(15) Are there downward normative references references (see RFC 3967)?
If so, list these downward references to support the Area Director in the
Last Call procedure.

No.

(16) Will publication of this document change the status of any
existing RFCs? Are those RFCs listed on the title page header, listed
in the abstract, and discussed in the introduction? If the RFCs are not
listed in the Abstract and Introduction, explain why, and point to the
part of the document where the relationship of this document to the
other RFCs is discussed. If this information is not in the document,
explain why the WG considers it unnecessary.

This document revises, and therefore obsoletes RFC 1323. This
is stated in the header, and introduction explains the history
rather clearly.

(17) Describe the Document Shepherd's review of the IANA considerations
section, especially with regard to its consistency with the body of the
document. Confirm that all protocol extensions that the document makes
are associated with the appropriate reservations in IANA registries.
Confirm that any referenced IANA registries have been clearly
identified. Confirm that newly created IANA registries include a
detailed specification of the initial contents for the registry, that
allocations procedures for future registrations are defined, and a
reasonable name for the new registry has been suggested (see RFC 5226).

There are no new IANA considerations, even though the document
specifies two TCP options, because the needed numbers have
already been allocated when publishing RFC 1323.

(18) List any new IANA registries that require Expert Review for future
allocations. Provide any public guidance that the IESG would find
useful in selecting the IANA Experts for these new registries.

No new IANA registeries needed.

(19) Describe reviews and automated checks performed by the Document
Shepherd to validate sections of the document written in a formal
language, such as XML code, BNF rules, MIB definitions, etc.

No automated checks needed.

(1) What type of RFC is being requested (BCP, Proposed Standard,
Internet Standard, Informational, Experimental, or Historic)?  Why
is this the proper type of RFC?  Is this type of RFC indicated in the
title page header?

Proposed Standard. This draft replaces earlier RFC 1323 that
also was a Proposed Standard. The title page header currently
says "Standards Track".


(2) The IESG approval announcement includes a Document Announcement
Write-Up. Please provide such a Document Announcement Write-Up. Recent
examples can be found in the "Action" announcements for approved
documents. The approval announcement contains the following sections:

Technical Summary

This document replaces RFC 1323, making minor fixes and
clarifications to the original document. The document
specifies three performance enhancing extensions to TCP, using
two TCP options. The first is window scale option, that allows
representation of receive window sizes larger than 2^16 bytes
originally allowed by the 16-bit window field. The second
option is TCP timestamps option that allows round-trip time
measurement on each TCP segment, and third is an algorithm to
detect and reject old duplicate segments that happen to match
the current TCP window (Protect Against Wrapped Sequence
numbers).

Working Group Summary

This document has been a chartered TCPM working group item
since year 2008. It has not been under significant
controversy, but the progress has been slow because of earlier
lack of WG (and authoring) cycles. Since adding a new
co-editor, the progress on the draft became faster in the
working group.

Document Quality

The predecessor of this document, RFC 1323, was published in
1992, and is deployed in most TCP implementations. This
document includes fixes and clarifications based on the gained
deployment experience. The recent versions of the document
have been reviewed and discussed by multiple working group
participants.

Personnel

Document Shepherd is Pasi Sarolahti.
Reponsible Area Director is Martin Stiemerling

(3) Briefly describe the review of this document that was performed by
the Document Shepherd.  If this version of the document is not ready
for publication, please explain why the document is being forwarded to
the IESG.

The document shepherd has read the latest version of the
document and the recent mailing list discussion, and thinks
the document is ready for publication.

(4) Does the document Shepherd have any concerns about the depth or
breadth of the reviews that have been performed?

No. 

(5) Do portions of the document need review from a particular or from
broader perspective, e.g., security, operational complexity, AAA, DNS,
DHCP, XML, or internationalization? If so, describe the review that
took place.

No.

(6) Describe any specific concerns or issues that the Document Shepherd
has with this document that the Responsible Area Director and/or the
IESG should be aware of? For example, perhaps he or she is uncomfortable
with certain parts of the document, or has concerns whether there really
is a need for it. In any event, if the WG has discussed those issues and
has indicated that it still wishes to advance the document, detail those
concerns here.

Document shepherd does not have concerns about the document

(7) Has each author confirmed that any and all appropriate IPR
disclosures required for full conformance with the provisions of BCP 78
and BCP 79 have already been filed. If not, explain why.

Not all authors responded when queried about this (most of
them are not actively participating IETF these days). The
editor of the draft confirms this on his part, however.

It should be noted the original RFC 1323 was dated in 1992,
and the documented mechanism has been included in various
implementations for a long time. No IPR disclosures on RFC
1323 are known.

(8) Has an IPR disclosure been filed that references this document?
If so, summarize any WG discussion and conclusion regarding the IPR
disclosures.

No IPR disclosures have been filed on this document

(9) How solid is the WG consensus behind this document? Does it
represent the strong concurrence of a few individuals, with others
being silent, or does the WG as a whole understand and agree with it? 

The recent versions of the document were extensively discussed
on the TCPM mailing list, by several members of the community,
for example regarding the normative language about how to
handle packets without TCP timestamps option after use of
timestamps has been negotiated (which is considered a corner
case in today's implementations).  The current approach is not
unanimously supported by all working group participants, but
the WG chairs think it represents the rough consensus of the
WG.

There were also some opinions suggesting that some of the
benefits could be achieved with less TCP option overhead. The
WG chairs concluded that considering the original scope of
the document (fixes and clarifications to RFC 1323), and its
large existing deployment base, it is important to publish the
current document, but keep the discussion open for future
enhancements.

(10) Has anyone threatened an appeal or otherwise indicated extreme
discontent? If so, please summarise the areas of conflict in separate
email messages to the Responsible Area Director. (It should be in a
separate email because this questionnaire is publicly available.)

No.

(11) Identify any ID nits the Document Shepherd has found in this
document. (See http://www.ietf.org/tools/idnits/ and the Internet-Drafts
Checklist). Boilerplate checks are not enough; this check needs to be
thorough.

There are informational references to obsolete RFCs 1072 and
RFC 1185. These are left in for documenting the history behind
this document, and are included intentionally.

(12) Describe how the document meets any required formal review
criteria, such as the MIB Doctor, media type, and URI type reviews.

No formal review required.

(13) Have all references within this document been identified as
either normative or informative?

Yes.

(14) Are there normative references to documents that are not ready for
advancement or are otherwise in an unclear state? If such normative
references exist, what is the plan for their completion?

No.

(15) Are there downward normative references references (see RFC 3967)?
If so, list these downward references to support the Area Director in the
Last Call procedure.

No.

(16) Will publication of this document change the status of any
existing RFCs? Are those RFCs listed on the title page header, listed
in the abstract, and discussed in the introduction? If the RFCs are not
listed in the Abstract and Introduction, explain why, and point to the
part of the document where the relationship of this document to the
other RFCs is discussed. If this information is not in the document,
explain why the WG considers it unnecessary.

This document revises, and therefore obsoletes RFC 1323. This
is stated in the header, and introduction explains the history
rather clearly.

(17) Describe the Document Shepherd's review of the IANA considerations
section, especially with regard to its consistency with the body of the
document. Confirm that all protocol extensions that the document makes
are associated with the appropriate reservations in IANA registries.
Confirm that any referenced IANA registries have been clearly
identified. Confirm that newly created IANA registries include a
detailed specification of the initial contents for the registry, that
allocations procedures for future registrations are defined, and a
reasonable name for the new registry has been suggested (see RFC 5226).

There are no new IANA considerations, even though the document
specifies two TCP options, because the needed numbers have
already been allocated when publishing RFC 1323.

(18) List any new IANA registries that require Expert Review for future
allocations. Provide any public guidance that the IESG would find
useful in selecting the IANA Experts for these new registries.

No new IANA registeries needed.

(19) Describe reviews and automated checks performed by the Document
Shepherd to validate sections of the document written in a formal
language, such as XML code, BNF rules, MIB definitions, etc.

No automated checks needed.