Test Cases for Evaluating Congestion Control for Interactive Real-Time Media
draft-ietf-rmcat-eval-test-10

[Ballot comment]
We don't really explain the usage of the "variation pattern index" that
I can see.

Section 3

        +  Bottleneck-link capacity: defines minimum capacity of the
            end-to-end path

I'm not sure I'd describe this as the "minimum capacity of the
end-to-end path", since attempts to use anything larger than this value
will pile up at the bottleneck.

Section 5.6, 5.7

Do we want anyone to consider testing with alternative TCP congestion
control algorithms?

Section 6.1

If you're not going to reference what scheme is used for implementing
priority, say how these priority values are interpreted.

Section 6.2

nit: "delay-based" (hyphenated)

Section 6.3

  Expected behavior: The candidate algorithm is expected to achieve
  full utilization at both bottleneck links without starving any of the

I'm not sure how we can get "full utilization" at both bottlenecks, when
the links in question have different capacity ratios.

Section 11.2

Some of these may need to be normative references; e.g., if we require
default TCP congestion control (RFC5681) for the competing TCP flows,
then it's mandatory.

[Ballot comment]
Thanks for the work on this document. I have only one minor nit to call out.

ID Nits reports:

  ** The document seems to lack a both a reference to RFC 2119 and the
    recommended RFC 2119 boilerplate, even if it appears to use RFC 2119
    keywords.

    RFC 2119 keyword, line 458: '...didate proposals MAY show the effectiv...'

[Ballot comment]
I share the Gen-ART reviewer's discomfort about the weak language in the security considerations section, but will leave it to the responsible AD to do the right thing.

(Via drafts-lastcall@iana.org): IESG/Authors/WG Chairs:

The IANA Functions Operator has reviewed draft-ietf-rmcat-eval-test-08, which is currently in Last Call, and has the following comments:

We understand that this document doesn't require any registry actions.

While it's often helpful for a document's IANA Considerations section to remain in place upon publication even if there are no actions, if the authors strongly prefer to remove it, we do not object.

If this assessment is not accurate, please respond as soon as possible.

Thank you,

Sabrina Tanamal
Senior IANA Services Specialist

The following Last Call announcement was sent out (ends 2019-02-11):

From: The IESG
To: IETF-Announce
CC: rmcat-chairs@ietf.org, draft-ietf-rmcat-eval-test@ietf.org, Colin Perkins , rmcat@ietf.org, ietf@kuehlewind.net, csp@csperkins.org
Reply-To: ietf@ietf.org
Sender:
Subject: Last Call:  (Test Cases for Evaluating RMCAT Proposals) to Informational RFC


The IESG has received a request from the RTP Media Congestion Avoidance
Techniques WG (rmcat) to consider the following document: - 'Test Cases for
Evaluating RMCAT Proposals'
  as Informational RFC

The IESG plans to make a decision in the next few weeks, and solicits final
comments on this action. Please send substantive comments to the
ietf@ietf.org mailing lists by 2019-02-11. Exceptionally, comments may be
sent to iesg@ietf.org instead. In either case, please retain the beginning of
the Subject line to allow automated sorting.

Abstract


  The Real-time Transport Protocol (RTP) is used to transmit media in
  multimedia telephony applications.  These applications are typically
  required to implement congestion control.  This document describes
  the test cases to be used in the performance evaluation of such
  congestion control algorithms in a controlled environment.




The file can be obtained via
https://datatracker.ietf.org/doc/draft-ietf-rmcat-eval-test/

IESG discussion can be tracked via
https://datatracker.ietf.org/doc/draft-ietf-rmcat-eval-test/ballot/


No IPR declarations have been submitted directly on this I-D.

> As required by RFC 4858, this is the current template for the Document
> Shepherd Write-Up.
>
> Changes are expected over time. This version is dated 24 February 2012.
>
> (1) What type of RFC is being requested (BCP, Proposed Standard,
> Internet Standard, Informational, Experimental, or Historic)?  Why
> is this the proper type of RFC?  Is this type of RFC indicated in the
> title page header?

  Informational.

  This was chosen because the draft describes test cases for evaluation of
  congestion control algorithm candidates but doesn't specify an algorithm
  or protocol in itself. As such, there is no requirement that it be
  standards track or BCP.

  This is indicated in the title page header.

> (2) The IESG approval announcement includes a Document Announcement
> Write-Up. Please provide such a Document Announcement Write-Up. Recent
> examples can be found in the "Action" announcements for approved
> documents. The approval announcement contains the following sections:
>
> Technical Summary
>
>  Relevant content can frequently be found in the abstract
>  and/or introduction of the document. If not, this may be
>  an indication that there are deficiencies in the abstract
>  or introduction.

  This document describes test cases that can be used for performance
  evaluation of the congestion control algorithms being developed by the
  RMCAT working group. These are suitable for laboratory experiments, in
  a controlled testbed environment, and are intended to demonstrate the
  basic suitability of the algorithms. They will complement, but do not
  replace, real-world testing.

> Working Group Summary
>
>  Was there anything in WG process that is worth noting? For
>  example, was there controversy about particular points or
>  were there decisions where the consensus was particularly
>  rough?

  WG determined that draft-ietf-rmcat-eval-criteria was getting unwieldy
  and over-long, and decided to split the test cases out into a separate
  draft. This was done at IETF 89, with the result being adopted as WG
  item at IETF 90. Steady discussion since then, gradually accumulating
  test cases as the candidate congestion control algorithm, and the WGs
  experiences with them, have developed. No particular controversies.

> Document Quality
>
>  Are there existing implementations of the protocol? Have a
>  significant number of vendors indicated their plan to
>  implement the specification? Are there any reviewers that
>  merit special mention as having done a thorough review,
>  e.g., one that resulted in important changes or a
>  conclusion that the document had no substantive issues? If
>  there was a MIB Doctor, Media Type or other expert review,
>  what was its course (briefly)? In the case of a Media Type
>  review, on what date was the request posted?

  The draft has been carefully reviewed by the WG, including by the
  implementers of the candidate congestion control algorithms, and
  has received extensive input based on evaluation experience. The
  test cases presented represent a good consensus from the WG about
  what is needed for basic evaluation of the algorithms. No MIB Doctor,
  Media Type, or other expert review needed or undertaken.

> Personnel
>
>  Who is the Document Shepherd? Who is the Responsible Area
>  Director?

  The document shepherd is Colin Perkins.
  The responsible AD is Mirja Kuehlewind.

> (3) Briefly describe the review of this document that was performed by
> the Document Shepherd.  If this version of the document is not ready
> for publication, please explain why the document is being forwarded to
> the IESG.

  The shepherd carefully reviewed the draft for WG last call in July 2018,
  and has ensured that the minor issues raised, and those raised by others
  in the WG, have been addressed. Draft was discussed at IETF 103, and it
  was agreed that issues were resolved. A short WG last on the list to
  confirm has just concluded.

> (4) Does the document Shepherd have any concerns about the depth or
> breadth of the reviews that have been performed?

  No concerns.

> (5) Do portions of the document need review from a particular or from
> broader perspective, e.g., security, operational complexity, AAA, DNS,
> DHCP, XML, or internationalization? If so, describe the review that
> took place.

  No such review needed.

> (6) Describe any specific concerns or issues that the Document Shepherd
> has with this document that the Responsible Area Director and/or the
> IESG should be aware of? For example, perhaps he or she is uncomfortable
> with certain parts of the document, or has concerns whether there really
> is a need for it. In any event, if the WG has discussed those issues and
> has indicated that it still wishes to advance the document, detail those
> concerns here.

  No concerns.

> (7) Has each author confirmed that any and all appropriate IPR
> disclosures required for full conformance with the provisions of BCP 78
> and BCP 79 have already been filed. If not, explain why.

  Each author has confirmed this.

> (8) Has an IPR disclosure been filed that references this document?
> If so, summarize any WG discussion and conclusion regarding the IPR
> disclosures.

  No IPR disclosure has been filed.

> (9) How solid is the WG consensus behind this document? Does it
> represent the strong concurrence of a few individuals, with others
> being silent, or does the WG as a whole understand and agree with it? 

  There is solid consensus. No objections have no raised.

> (10) Has anyone threatened an appeal or otherwise indicated extreme
> discontent? If so, please summarise the areas of conflict in separate
> email messages to the Responsible Area Director. (It should be in a
> separate email because this questionnaire is publicly available.)

  There is no such discontent.

> (11) Identify any ID nits the Document Shepherd has found in this
> document. (See https://www.ietf.org/tools/idnits/ and the Internet-Drafts
> Checklist). Boilerplate checks are not enough; this check needs to be
> thorough.

  The idnits tools reports a few instances of overly long lines. They are
  due to figures that are indented too far from the left margin, and will
  be fixed when the RFC Editor processes the document.

  The idnits tool also reports missing references, but inspection shows
  this is due to a limitation of the tool that's interpreting the range
  [300ms, 1000ms] as two document references.

> (12) Describe how the document meets any required formal review
> criteria, such as the MIB Doctor, media type, and URI type reviews.

  No such review needed.

> (13) Have all references within this document been identified as
> either normative or informative?

  Yes.

> (14) Are there normative references to documents that are not ready for
> advancement or are otherwise in an unclear state? If such normative
> references exist, what is the plan for their completion?

  There are normative references to the following:

  - draft-ietf-rmcat-eval-criteria has completed WG last call, and is
    waiting on the authors to resolve minor issues. I expect it to be
    ready to advance in the next few weeks.
  - draft-ietf-rmcat-video-traffic-model is with the IESG
  - draft-ietf-rmcat-wireless-tests is complete, and scheduled to go
    to WG last call once this draft, draft-ietf-rmcat-eval-criteria,
    and draft-ietf-rmcat-video-traffic-model have completed WG last
    call.

> (15) Are there downward normative references references (see RFC 3967)?
> If so, list these downward references to support the Area Director in
> the Last Call procedure.

  No downward normative references.

> (16) Will publication of this document change the status of any
> existing RFCs? Are those RFCs listed on the title page header, listed
> in the abstract, and discussed in the introduction? If the RFCs are not
> listed in the Abstract and Introduction, explain why, and point to the
> part of the document where the relationship of this document to the
> other RFCs is discussed. If this information is not in the document,
> explain why the WG considers it unnecessary.

  No such status changes are made.

> (17) Describe the Document Shepherd's review of the IANA considerations
> section, especially with regard to its consistency with the body of the
> document. Confirm that all protocol extensions that the document makes
> are associated with the appropriate reservations in IANA registries.
> Confirm that any referenced IANA registries have been clearly
> identified. Confirm that newly created IANA registries include a
> detailed specification of the initial contents for the registry, that
> allocations procedures for future registrations are defined, and a
> reasonable name for the new registry has been suggested (see RFC 5226).

  No IANA actions are needed.

> (18) List any new IANA registries that require Expert Review for future
> allocations. Provide any public guidance that the IESG would find
> useful in selecting the IANA Experts for these new registries.

  No IANA actions are needed.

> (19) Describe reviews and automated checks performed by the Document
> Shepherd to validate sections of the document written in a formal
> language, such as XML code, BNF rules, MIB definitions, etc.

  No use of such formal languages. The WG has seen implementations
  of the various test cases included in the draft, however, and we
  have confidence that they are reasonable.