Skip to main content

Evaluation Test Cases for Interactive Real-Time Media over Wireless Networks
draft-ietf-rmcat-wireless-tests-11

Revision differences

Document history

Date Rev. By Action
2020-07-27
11 (System) RFC Editor state changed to AUTH48-DONE from AUTH48
2020-07-20
11 (System) RFC Editor state changed to AUTH48 from RFC-EDITOR
2020-06-05
11 (System) RFC Editor state changed to RFC-EDITOR from EDIT
2020-03-25
11 (System) RFC Editor state changed to EDIT from MISSREF
2020-03-16
11 (System) RFC Editor state changed to MISSREF
2020-03-16
11 (System) IESG state changed to RFC Ed Queue from Approved-announcement sent
2020-03-16
11 (System) Announcement was received by RFC Editor
2020-03-13
11 (System) IANA Action state changed to No IANA Actions from In Progress
2020-03-13
11 (System) IANA Action state changed to In Progress
2020-03-13
11 Amy Vezza IESG state changed to Approved-announcement sent from Approved-announcement to be sent
2020-03-13
11 Amy Vezza IESG has approved the document
2020-03-13
11 Amy Vezza Closed "Approve" ballot
2020-03-13
11 Amy Vezza Ballot approval text was generated
2020-03-13
11 Mirja Kühlewind IESG state changed to Approved-announcement to be sent from IESG Evaluation::AD Followup
2020-03-13
11 Xiaoqing Zhu New version available: draft-ietf-rmcat-wireless-tests-11.txt
2020-03-13
11 (System) New version approved
2020-03-13
11 (System) Request for posting confirmation emailed to previous authors: Jian Fu , Michael Ramalho , rmcat-chairs@ietf.org, Zaheduzzaman Sarker , Wei-Tian Tan , Xiaoqing Zhu
2020-03-13
11 Xiaoqing Zhu Uploaded new revision
2020-03-10
10 (System) Sub state has been changed to AD Followup from Revised ID Needed
2020-03-10
10 (System) IANA Review state changed to Version Changed - Review Needed from IANA OK - No Actions Needed
2020-03-10
10 Xiaoqing Zhu New version available: draft-ietf-rmcat-wireless-tests-10.txt
2020-03-10
10 (System) New version approved
2020-03-10
10 (System)
Request for posting confirmation emailed to previous authors: Zaheduzzaman Sarker , rmcat-chairs@ietf.org, Michael Ramalho , Xiaoqing Zhu , Jian Fu , Wei-Tian Tan , …
Request for posting confirmation emailed to previous authors: Zaheduzzaman Sarker , rmcat-chairs@ietf.org, Michael Ramalho , Xiaoqing Zhu , Jian Fu , Wei-Tian Tan , Ingemar Johansson
2020-03-10
10 Xiaoqing Zhu Uploaded new revision
2020-03-05
09 Roman Danyliw
[Ballot comment]
==[ Old Discuss ]==
Please let me know if I've misunderstood the test execution protocol incorrectly:

Section 6.  Per the paragraph “The evaluation …
[Ballot comment]
==[ Old Discuss ]==
Please let me know if I've misunderstood the test execution protocol incorrectly:

Section 6.  Per the paragraph “The evaluation of the test cases are intended to carry out in a controlled lab environment … It is important to take appropriate caution to avoid leaking non-responsive traffic from unproven congestion avoidance techniques onto the open Internet”, this is good guidance in general case.  However, in the case of this document how applicable is it?  Didn’t Section 3 (“We, therefore, recommend that a cellular network simulator is used for the test cases defined in this document …” and practically establish it can’t be done without simulation with the scenario of the underground mine) and Section 4 (“We recommend to carry out the test cases as defined in this document using a simulator, such as [NS-2] or [NS-3]).  If all the testing is supposed to be in a simulator how is it leaking out onto the internet?  As far as I can tell, this helpful text is common in RMCAT document, but in this case could it please be tailored for the proposed testing regime. 

Perhaps something on the order adding text on the order of “Given the difficulty of deterministic wireless testing, it is RECOMMENDED and expected that the tests described in this document would be done via simulation.  However,  ”

[Telechat discussion was that that using a simulator is only recommended, so there is the possibility of live testing which might leak onto live networks]

==[ end ]==

** Section 3.1.  Per “In this test case, each of the user/UE in the media session is an RMCAT compliant endpoint”, what is a “RMCAT compliant endpoint”?  Could this be cited please.

** Section 3.1.  Per “At the beginning of the simulation, there should be enough time to warm-up the network”, intuitively the notion of “warm[ing]-up the network” makes sense.  However, is more precision required to side-by-side analysis of test runs of when the network is “warm enough”?

** Section 4.  Per “Statistics collected from enterprise Wi-Fi networks show that the two dominant physical modes are 802.11n and 802.11ac, accounting for 41% and 58% of connected devices”, it is would be valuable to cite this value and provide a timestamp.  This distribution will certainly change as this document ages.

** Section 4.  Per “Unless otherwise mentioned, test cases in this section are described using the underlying PHY- and MAC-layer parameters based on the IEEE 802.11n Standard.”, this focus on only 802.11n is surprising, since the next sentence establishes that 802.11.n is already less than half (41%) of the Wi-Fi traffic (and likely will continue to shrink).  Why not ac?

** There appear to be some differences between the description of the Cellular (Section 3) and Wifi (Section 4) test.
-- Section 4.1.2 and 4.2.2 are called “Test setup”, but Section 3.1.2 and 3.2.2. are called “Simulation setup”.  Was this intentional?

-- Section 4.*.4 discusses the expected test behavior, but Section 3.* does not?  Was that explicit?
2020-03-05
09 Roman Danyliw [Ballot Position Update] Position for Roman Danyliw has been changed to No Objection from Discuss
2020-03-05
09 Cindy Morgan IESG state changed to IESG Evaluation::Revised I-D Needed from Waiting for Writeup
2020-03-05
09 Éric Vyncke
[Ballot comment]
Thank you for the work put into this document. It is really easy to read.

Please find below some non-blocking COMMENTs and NITs. …
[Ballot comment]
Thank you for the work put into this document. It is really easy to read.

Please find below some non-blocking COMMENTs and NITs. An answer will be appreciated.

I hope that this helps to improve the document,

Regards,

-éric

== COMMENTS ==

-- Section 1 --
Does the following assertion always stand ? "This contrasts with the wired network setting where traffic flows from all users share the same queue."

-- Section 3.1.2 --
As a frequent high speed train passenger, I would have appreciated a simulation with mobility of 300 km/h ;-)

== NITS ==

-- Section 3.1.2 --
s/10Mhz/10MHz/ ?
2020-03-05
09 Éric Vyncke [Ballot Position Update] New position, No Objection, has been recorded for Éric Vyncke
2020-03-05
09 Mirja Kühlewind [Ballot comment]
GENART and OPSDIR review have been addressed in -09.
2020-03-05
09 Mirja Kühlewind Ballot comment text updated for Mirja Kühlewind
2020-03-05
09 Benjamin Kaduk
[Ballot comment]
Thanks to Adam for noting the author-count already.

Section 3

  Different mobile operators deploy their own cellular networks with
  their own …
[Ballot comment]
Thanks to Adam for noting the author-count already.

Section 3

  Different mobile operators deploy their own cellular networks with
  their own set of network functionalities and policies.  Usually, a
  mobile operator network includes 2G, EDGE, 3G and 4G radio access

Is 2G still "typical"?  I was given to understand it was getting phased
out.

Section 3.1, 3.2

How do I tell how much time is "enough time to warm-up the network"?

Section 4.2.3

  b.  Multiple RTP-based media flows sharing the wireless uplink:N = 16
      (all downlink); M = 0.  When multiple clients attempt to transmit

I'm not sure how to interpret the parenthetical; is it a copy/paste
issue from (a) that should read "all uplink"?
2020-03-05
09 Benjamin Kaduk [Ballot Position Update] New position, No Objection, has been recorded for Benjamin Kaduk
2020-03-05
09 Éric Vyncke Request closed, assignment withdrawn: Ron Bonica Telechat INTDIR review
2020-03-05
09 Éric Vyncke
Closed request for Telechat review by INTDIR with state 'Withdrawn': The document is on the IESG telechat today. No need anymore for a review (still …
Closed request for Telechat review by INTDIR with state 'Withdrawn': The document is on the IESG telechat today. No need anymore for a review (still welcome by the authors probably).
2020-03-05
09 Mirja Kühlewind This document now replaces draft-fu-rmcat-wifi-test-case, draft-sarker-rmcat-cellular-eval-test-cases instead of None
2020-03-04
09 Barry Leiba
[Ballot comment]
You include the BCP 14 boilerplate and references, but the only BCP 14 key word in the document is a MUST in Section …
[Ballot comment]
You include the BCP 14 boilerplate and references, but the only BCP 14 key word in the document is a MUST in Section 6 that seems like it doesnt need to be BCP 14 in this document (it seems to refer to a tenet, not a requirement specified in this document).  I suggest removing the boilerplate and the references and making the “must” lower case.
2020-03-04
09 Barry Leiba [Ballot Position Update] New position, No Objection, has been recorded for Barry Leiba
2020-03-04
09 Adam Roach
[Ballot comment]
Thanks for the work that went into creating this document.

ID Nits correctly reports:

  == Unused Reference: 'I-D.ietf-rmcat-cc-requirements' is defined on line …
[Ballot comment]
Thanks for the work that went into creating this document.

ID Nits correctly reports:

  == Unused Reference: 'I-D.ietf-rmcat-cc-requirements' is defined on line
    998, but no explicit reference was found in the text


As a note for the AD: I can’t find a justification in the ballot or shepherd’s writeup for why this document has six authors instead of a smaller number of editors and a list of contributors. It would be nice to capture the exception rationale in one of those two places for posterity.
2020-03-04
09 Adam Roach Ballot comment text updated for Adam Roach
2020-03-04
09 Adam Roach
[Ballot comment]
Thanks for the work that went into creating this document.
ID Nits correctly reports:
  == Unused Reference: 'I-D.ietf-rmcat-cc-requirements' is defined on line …
[Ballot comment]
Thanks for the work that went into creating this document.
ID Nits correctly reports:
  == Unused Reference: 'I-D.ietf-rmcat-cc-requirements' is defined on line
    998, but no explicit reference was found in the text

As a note for the AD: I can’t find a justification in the ballot or shepherd’s writeup for why this document has six authors instead of a smaller number of editors and a list of contributors. It would be nice to capture the exception rationale in one of those two places for posterity.
2020-03-04
09 Adam Roach [Ballot Position Update] New position, No Objection, has been recorded for Adam Roach
2020-03-04
09 Deborah Brungard [Ballot Position Update] New position, No Objection, has been recorded for Deborah Brungard
2020-03-04
09 Amanda Baber IANA Review state changed to IANA OK - No Actions Needed from Version Changed - Review Needed
2020-03-04
09 Roman Danyliw
[Ballot discuss]
Please let me know if I've misunderstood the test execution protocol incorrectly:

Section 6.  Per the paragraph “The evaluation of the test cases …
[Ballot discuss]
Please let me know if I've misunderstood the test execution protocol incorrectly:

Section 6.  Per the paragraph “The evaluation of the test cases are intended to carry out in a controlled lab environment … It is important to take appropriate caution to avoid leaking non-responsive traffic from unproven congestion avoidance techniques onto the open Internet”, this is good guidance in general case.  However, in the case of this document how applicable is it?  Didn’t Section 3 (“We, therefore, recommend that a cellular network simulator is used for the test cases defined in this document …” and practically establish it can’t be done without simulation with the scenario of the underground mine) and Section 4 (“We recommend to carry out the test cases as defined in this document using a simulator, such as [NS-2] or [NS-3]).  If all the testing is supposed to be in a simulator how is it leaking out onto the internet?  As far as I can tell, this helpful text is common in RMCAT document, but in this case could it please be tailored for the proposed testing regime. 

Perhaps something on the order adding text on the order of “Given the difficulty of deterministic wireless testing, it is RECOMMENDED and expected that the tests described in this document would be done via simulation.  However,  ”
2020-03-04
09 Roman Danyliw
[Ballot comment]
** Section 3.1.  Per “In this test case, each of the user/UE in the media session is an RMCAT compliant endpoint”, what is …
[Ballot comment]
** Section 3.1.  Per “In this test case, each of the user/UE in the media session is an RMCAT compliant endpoint”, what is a “RMCAT compliant endpoint”?  Could this be cited please.

** Section 3.1.  Per “At the beginning of the simulation, there should be enough time to warm-up the network”, intuitively the notion of “warm[ing]-up the network” makes sense.  However, is more precision required to side-by-side analysis of test runs of when the network is “warm enough”?

** Section 4.  Per “Statistics collected from enterprise Wi-Fi networks show that the two dominant physical modes are 802.11n and 802.11ac, accounting for 41% and 58% of connected devices”, it is would be valuable to cite this value and provide a timestamp.  This distribution will certainly change as this document ages.

** Section 4.  Per “Unless otherwise mentioned, test cases in this section are described using the underlying PHY- and MAC-layer parameters based on the IEEE 802.11n Standard.”, this focus on only 802.11n is surprising, since the next sentence establishes that 802.11.n is already less than half (41%) of the Wi-Fi traffic (and likely will continue to shrink).  Why not ac?

** There appear to be some differences between the description of the Cellular (Section 3) and Wifi (Section 4) test.
-- Section 4.1.2 and 4.2.2 are called “Test setup”, but Section 3.1.2 and 3.2.2. are called “Simulation setup”.  Was this intentional?

-- Section 4.*.4 discusses the expected test behavior, but Section 3.* does not?  Was that explicit?
2020-03-04
09 Roman Danyliw [Ballot Position Update] New position, Discuss, has been recorded for Roman Danyliw
2020-03-04
09 Suresh Krishnan [Ballot Position Update] New position, No Objection, has been recorded for Suresh Krishnan
2020-03-03
09 Alvaro Retana
[Ballot comment]
If my reading of the mailing list is correct, it looks like this document was the result of merging draft-sarker-rmcat-cellular-eval-test-cases and draft-fu-rmcat-wifi-test-case.  It …
[Ballot comment]
If my reading of the mailing list is correct, it looks like this document was the result of merging draft-sarker-rmcat-cellular-eval-test-cases and draft-fu-rmcat-wifi-test-case.  It would be nice if this document was tagged as replacing them.
2020-03-03
09 Alvaro Retana [Ballot Position Update] New position, No Objection, has been recorded for Alvaro Retana
2020-03-03
09 Alissa Cooper [Ballot Position Update] New position, No Objection, has been recorded for Alissa Cooper
2020-02-27
09 (System) IANA Review state changed to Version Changed - Review Needed from IANA OK - No Actions Needed
2020-02-27
09 Xiaoqing Zhu New version available: draft-ietf-rmcat-wireless-tests-09.txt
2020-02-27
09 (System) New version approved
2020-02-27
09 (System)
Request for posting confirmation emailed to previous authors: Jian Fu , Ingemar Johansson , Xiaoqing Zhu , Zaheduzzaman Sarker , rmcat-chairs@ietf.org, Michael Ramalho , …
Request for posting confirmation emailed to previous authors: Jian Fu , Ingemar Johansson , Xiaoqing Zhu , Zaheduzzaman Sarker , rmcat-chairs@ietf.org, Michael Ramalho , Wei-Tian Tan
2020-02-27
09 Xiaoqing Zhu Uploaded new revision
2020-02-26
08 Carlos Jesús Bernardos Request for Telechat review by INTDIR is assigned to Ron Bonica
2020-02-26
08 Carlos Jesús Bernardos Request for Telechat review by INTDIR is assigned to Ron Bonica
2020-02-22
08 Éric Vyncke Requested Telechat review by INTDIR
2020-02-21
08 Amy Vezza Placed on agenda for telechat - 2020-03-05
2020-02-21
08 Mirja Kühlewind Changed consensus to Yes from Unknown
2020-02-21
08 Mirja Kühlewind [Ballot comment]
GENART and OPSDIR review will be address in the next days.
2020-02-21
08 Mirja Kühlewind Ballot comment text updated for Mirja Kühlewind
2020-02-21
08 Mirja Kühlewind Ballot has been issued
2020-02-21
08 Mirja Kühlewind [Ballot Position Update] New position, Yes, has been recorded for Mirja Kühlewind
2020-02-21
08 Mirja Kühlewind Created "Approve" ballot
2020-02-21
08 Mirja Kühlewind Ballot writeup was changed
2020-01-23
08 Tero Kivinen Request for Last Call review by SECDIR Completed: Has Issues. Reviewer: Hilarie Orman. Submission of review completed at an earlier date.
2020-01-21
08 Gyan Mishra Request for Last Call review by GENART Completed: Ready with Issues. Reviewer: Gyan Mishra. Sent review to list.
2020-01-21
08 (System) IESG state changed to Waiting for Writeup from In Last Call
2020-01-20
08 Tero Kivinen Request for Last Call review by SECDIR Completed: Has Issues. Reviewer: Hilarie Orman.
2020-01-16
08 (System) IANA Review state changed to IANA OK - No Actions Needed from IANA - Review Needed
2020-01-16
08 Sabrina Tanamal
(Via drafts-lastcall@iana.org): IESG/Authors/WG Chairs:

The IANA Functions Operator has reviewed draft-ietf-rmcat-wireless-tests-08, which is currently in Last Call, and has the following comments:

We …
(Via drafts-lastcall@iana.org): IESG/Authors/WG Chairs:

The IANA Functions Operator has reviewed draft-ietf-rmcat-wireless-tests-08, which is currently in Last Call, and has the following comments:

We understand that this document doesn't require any registry actions.

While it's often helpful for a document's IANA Considerations section to remain in place upon publication even if there are no actions, if the authors strongly prefer to remove it, we do not object.

If this assessment is not accurate, please respond as soon as possible.

Thank you,

Sabrina Tanamal
Senior IANA Services Specialist
2020-01-13
08 Fred Baker Request for Last Call review by OPSDIR Completed: Has Nits. Reviewer: Fred Baker. Sent review to list.
2020-01-12
08 Gunter Van de Velde Request for Last Call review by OPSDIR is assigned to Fred Baker
2020-01-12
08 Gunter Van de Velde Request for Last Call review by OPSDIR is assigned to Fred Baker
2020-01-09
08 Jean Mahoney Request for Last Call review by GENART is assigned to Gyan Mishra
2020-01-09
08 Jean Mahoney Request for Last Call review by GENART is assigned to Gyan Mishra
2020-01-09
08 Tero Kivinen Request for Last Call review by SECDIR is assigned to Hilarie Orman
2020-01-09
08 Tero Kivinen Request for Last Call review by SECDIR is assigned to Hilarie Orman
2020-01-07
08 Cindy Morgan IANA Review state changed to IANA - Review Needed
2020-01-07
08 Cindy Morgan
The following Last Call announcement was sent out (ends 2020-01-21):

From: The IESG
To: IETF-Announce
CC: rmcat-chairs@ietf.org, draft-ietf-rmcat-wireless-tests@ietf.org, Colin Perkins , rmcat@ietf.org, …
The following Last Call announcement was sent out (ends 2020-01-21):

From: The IESG
To: IETF-Announce
CC: rmcat-chairs@ietf.org, draft-ietf-rmcat-wireless-tests@ietf.org, Colin Perkins , rmcat@ietf.org, ietf@kuehlewind.net, csp@csperkins.org
Reply-To: last-call@ietf.org
Sender:
Subject: Last Call:  (Evaluation Test Cases for Interactive Real-Time Media over Wireless Networks) to Informational RFC


The IESG has received a request from the RTP Media Congestion Avoidance
Techniques WG (rmcat) to consider the following document: - 'Evaluation Test
Cases for Interactive Real-Time Media over Wireless
  Networks'
  as Informational RFC

The IESG plans to make a decision in the next few weeks, and solicits final
comments on this action. Please send substantive comments to the
last-call@ietf.org mailing lists by 2020-01-21. Exceptionally, comments may
be sent to iesg@ietf.org instead. In either case, please retain the beginning
of the Subject line to allow automated sorting.

Abstract


  The Real-time Transport Protocol (RTP) is a common transport choice
  for interactive multimedia communication applications.  The
  performance of such applications typically depends on a well-
  functioning congestion control algorithm.  To ensure seamless and
  robust user experience, a well-designed RTP-based congestion control
  algorithm should work well across all access network types.  This
  document describes test cases for evaluating performances of such
  congestion control algorithms over LTE and Wi-Fi networks.




The file can be obtained via
https://datatracker.ietf.org/doc/draft-ietf-rmcat-wireless-tests/

IESG discussion can be tracked via
https://datatracker.ietf.org/doc/draft-ietf-rmcat-wireless-tests/ballot/


No IPR declarations have been submitted directly on this I-D.




2020-01-07
08 Cindy Morgan IESG state changed to In Last Call from Last Call Requested
2020-01-07
08 Mirja Kühlewind Last call was requested
2020-01-07
08 Mirja Kühlewind Ballot approval text was generated
2020-01-07
08 Mirja Kühlewind Ballot writeup was generated
2020-01-07
08 Mirja Kühlewind IESG state changed to Last Call Requested from Publication Requested
2020-01-07
08 Mirja Kühlewind Last call announcement was generated
2019-09-29
08 Colin Perkins
> As required by RFC 4858, this is the current template for the Document
> Shepherd Write-Up.
>
> Changes are expected over time. …
> As required by RFC 4858, this is the current template for the Document
> Shepherd Write-Up.
>
> Changes are expected over time. This version is dated 24 February 2012.
>
> (1) What type of RFC is being requested (BCP, Proposed Standard,
> Internet Standard, Informational, Experimental, or Historic)?  Why
> is this the proper type of RFC?  Is this type of RFC indicated in the
> title page header?

  An Informational RFC is being requested. This is indicated on the title
  page of the draft. An informational RFC is appropriate, since the draft
  provides guidelines for evaluating congestion control algorithms. While
  intended to be useful for implementers and designers of new algorithms,
  such evaluation is not needed for interoperability and this doesn't need
  to be standards track.

> (2) The IESG approval announcement includes a Document Announcement
> Write-Up. Please provide such a Document Announcement Write-Up. Recent
> examples can be found in the "Action" announcements for approved
> documents. The approval announcement contains the following sections:
>
> Technical Summary

  This document describes test cases for evaluating the performance of RTP
  congestion control algorithms over LTE and Wi-Fi networks. It's part of
  a set of drafts describing evaluation criteria for congestion control
  algorithms developed in the RMCAT working group.

> Working Group Summary
>
>  Was there anything in WG process that is worth noting? For
>  example, was there controversy about particular points or
>  were there decisions where the consensus was particularly
>  rough?

  Some discussion early in the process, but the draft has been stable for
  some years. The test cases presented are not controversial, and have been
  used in the evaluation of algorithms developed by the WG.


> Document Quality
>
>  Are there existing implementations of the protocol? Have a
>  significant number of vendors indicated their plan to
>  implement the specification? Are there any reviewers that
>  merit special mention as having done a thorough review,
>  e.g., one that resulted in important changes or a
>  conclusion that the document had no substantive issues? If
>  there was a MIB Doctor, Media Type or other expert review,
>  what was its course (briefly)? In the case of a Media Type
>  review, on what date was the request posted?

  The test cases described were used in the evaluation of the SCReAM and
  NADA algorithms, demonstrating their utility. Sergio Mena provided a
  helpful and detailed WG last call review.

> Personnel
>
>  Who is the Document Shepherd? Who is the Responsible Area
>  Director?

  The shepherd is Colin Perkins. The responsible area directory is Mirja
  Kühlewind.

> (3) Briefly describe the review of this document that was performed by
> the Document Shepherd.  If this version of the document is not ready
> for publication, please explain why the document is being forwarded to
> the IESG.

  Colin Perkins reviewed the draft prior to WG last call in December 2018,
  and noted that the security considerations were missing, but otherwise
  found no major concerns. Sergio Mena provided a detailed review during
  WG last call, in Feb 2019, catching some issues. The draft was updated
  to address these in July 2019, just prior to IETF 105.

> (4) Does the document Shepherd have any concerns about the depth or
> breadth of the reviews that have been performed?

  No concerns. The draft has received extensive review over the years,
  and has been used in the evaluation of congestion control algorithms
  by the working group.

> (5) Do portions of the document need review from a particular or from
> broader perspective, e.g., security, operational complexity, AAA, DNS,
> DHCP, XML, or internationalization? If so, describe the review that
> took place.

  The draft focusses on evaluation of congestion control algorithms. The
  RMCAT WG has appropriate experience to evaluate this. There is nothing
  needing specialist review from other areas.

> (6) Describe any specific concerns or issues that the Document Shepherd
> has with this document that the Responsible Area Director and/or the
> IESG should be aware of? For example, perhaps he or she is uncomfortable
> with certain parts of the document, or has concerns whether there really
> is a need for it. In any event, if the WG has discussed those issues and
> has indicated that it still wishes to advance the document, detail those
> concerns here.

  No concerns.

> (7) Has each author confirmed that any and all appropriate IPR
> disclosures required for full conformance with the provisions of BCP 78
> and BCP 79 have already been filed. If not, explain why.

  All authors have confirmed that no IPR disclosures are required.

> (8) Has an IPR disclosure been filed that references this document?
> If so, summarize any WG discussion and conclusion regarding the IPR
> disclosures.

  No IPR disclosures filed.

> (9) How solid is the WG consensus behind this document? Does it
> represent the strong concurrence of a few individuals, with others
> being silent, or does the WG as a whole understand and agree with it? 

  There is strong consensus.

> (10) Has anyone threatened an appeal or otherwise indicated extreme
> discontent? If so, please summarise the areas of conflict in separate
> email messages to the Responsible Area Director. (It should be in a
> separate email because this questionnaire is publicly available.)

  No such discontent.

> (11) Identify any ID nits the Document Shepherd has found in this
> document. (See https://www.ietf.org/tools/idnits/ and the Internet-Drafts
> Checklist). Boilerplate checks are not enough; this check needs to be
> thorough.

  The idnits checker incorrectly reports issues with the references
  (the draft uses [...] to indicate ranges and lists, confusing the
  checker).

> (12) Describe how the document meets any required formal review
> criteria, such as the MIB Doctor, media type, and URI type reviews.

  No such review needed.

> (13) Have all references within this document been identified as
> either normative or informative?

  Yes.

> (14) Are there normative references to documents that are not ready for
> advancement or are otherwise in an unclear state? If such normative
> references exist, what is the plan for their completion?

  The normative references are either to RFCs or 3GPP references, except
  for draft-ietf-rmcat-eval-criteria which is being send for publication
  in parallel to this, and to the NS3 network simulator documentation.

> (15) Are there downward normative references references (see RFC 3967)?
> If so, list these downward references to support the Area Director in
> the Last Call procedure.

  No such references.

> (16) Will publication of this document change the status of any
> existing RFCs? Are those RFCs listed on the title page header, listed
> in the abstract, and discussed in the introduction? If the RFCs are not
> listed in the Abstract and Introduction, explain why, and point to the
> part of the document where the relationship of this document to the
> other RFCs is discussed. If this information is not in the document,
> explain why the WG considers it unnecessary.

  No changes made.

> (17) Describe the Document Shepherd's review of the IANA considerations
> section, especially with regard to its consistency with the body of the
> document. Confirm that all protocol extensions that the document makes
> are associated with the appropriate reservations in IANA registries.
> Confirm that any referenced IANA registries have been clearly
> identified. Confirm that newly created IANA registries include a
> detailed specification of the initial contents for the registry, that
> allocations procedures for future registrations are defined, and a
> reasonable name for the new registry has been suggested (see RFC 8126).

  No IANA actions.

> (18) List any new IANA registries that require Expert Review for future
> allocations. Provide any public guidance that the IESG would find
> useful in selecting the IANA Experts for these new registries.

  No IANA actions.

> (19) Describe reviews and automated checks performed by the Document
> Shepherd to validate sections of the document written in a formal
> language, such as XML code, BNF rules, MIB definitions, etc.

  None needed.

2019-09-29
08 Colin Perkins Responsible AD changed to Mirja Kühlewind
2019-09-29
08 Colin Perkins IETF WG state changed to Submitted to IESG for Publication from Waiting for WG Chair Go-Ahead
2019-09-29
08 Colin Perkins IESG state changed to Publication Requested from I-D Exists
2019-09-29
08 Colin Perkins IESG process started in state Publication Requested
2019-09-29
08 Colin Perkins
> As required by RFC 4858, this is the current template for the Document
> Shepherd Write-Up.
>
> Changes are expected over time. …
> As required by RFC 4858, this is the current template for the Document
> Shepherd Write-Up.
>
> Changes are expected over time. This version is dated 24 February 2012.
>
> (1) What type of RFC is being requested (BCP, Proposed Standard,
> Internet Standard, Informational, Experimental, or Historic)?  Why
> is this the proper type of RFC?  Is this type of RFC indicated in the
> title page header?

  An Informational RFC is being requested. This is indicated on the title
  page of the draft. An informational RFC is appropriate, since the draft
  provides guidelines for evaluating congestion control algorithms. While
  intended to be useful for implementers and designers of new algorithms,
  such evaluation is not needed for interoperability and this doesn't need
  to be standards track.

> (2) The IESG approval announcement includes a Document Announcement
> Write-Up. Please provide such a Document Announcement Write-Up. Recent
> examples can be found in the "Action" announcements for approved
> documents. The approval announcement contains the following sections:
>
> Technical Summary

  This document describes test cases for evaluating the performance of RTP
  congestion control algorithms over LTE and Wi-Fi networks. It's part of
  a set of drafts describing evaluation criteria for congestion control
  algorithms developed in the RMCAT working group.

> Working Group Summary
>
>  Was there anything in WG process that is worth noting? For
>  example, was there controversy about particular points or
>  were there decisions where the consensus was particularly
>  rough?

  Some discussion early in the process, but the draft has been stable for
  some years. The test cases presented are not controversial, and have been
  used in the evaluation of algorithms developed by the WG.


> Document Quality
>
>  Are there existing implementations of the protocol? Have a
>  significant number of vendors indicated their plan to
>  implement the specification? Are there any reviewers that
>  merit special mention as having done a thorough review,
>  e.g., one that resulted in important changes or a
>  conclusion that the document had no substantive issues? If
>  there was a MIB Doctor, Media Type or other expert review,
>  what was its course (briefly)? In the case of a Media Type
>  review, on what date was the request posted?

  The test cases described were used in the evaluation of the SCReAM and
  NADA algorithms, demonstrating their utility. Sergio Mena provided a
  helpful and detailed WG last call review.

> Personnel
>
>  Who is the Document Shepherd? Who is the Responsible Area
>  Director?

  The shepherd is Colin Perkins. The responsible area directory is Mirja
  Kühlewind.

> (3) Briefly describe the review of this document that was performed by
> the Document Shepherd.  If this version of the document is not ready
> for publication, please explain why the document is being forwarded to
> the IESG.

  Colin Perkins reviewed the draft prior to WG last call in December 2018,
  and noted that the security considerations were missing, but otherwise
  found no major concerns. Sergio Mena provided a detailed review during
  WG last call, in Feb 2019, catching some issues. The draft was updated
  to address these in July 2019, just prior to IETF 105.

> (4) Does the document Shepherd have any concerns about the depth or
> breadth of the reviews that have been performed?

  No concerns. The draft has received extensive review over the years,
  and has been used in the evaluation of congestion control algorithms
  by the working group.

> (5) Do portions of the document need review from a particular or from
> broader perspective, e.g., security, operational complexity, AAA, DNS,
> DHCP, XML, or internationalization? If so, describe the review that
> took place.

  The draft focusses on evaluation of congestion control algorithms. The
  RMCAT WG has appropriate experience to evaluate this. There is nothing
  needing specialist review from other areas.

> (6) Describe any specific concerns or issues that the Document Shepherd
> has with this document that the Responsible Area Director and/or the
> IESG should be aware of? For example, perhaps he or she is uncomfortable
> with certain parts of the document, or has concerns whether there really
> is a need for it. In any event, if the WG has discussed those issues and
> has indicated that it still wishes to advance the document, detail those
> concerns here.

  No concerns.

> (7) Has each author confirmed that any and all appropriate IPR
> disclosures required for full conformance with the provisions of BCP 78
> and BCP 79 have already been filed. If not, explain why.

  All authors have confirmed that no IPR disclosures are required.

> (8) Has an IPR disclosure been filed that references this document?
> If so, summarize any WG discussion and conclusion regarding the IPR
> disclosures.

  No IPR disclosures filed.

> (9) How solid is the WG consensus behind this document? Does it
> represent the strong concurrence of a few individuals, with others
> being silent, or does the WG as a whole understand and agree with it? 

  There is strong consensus.

> (10) Has anyone threatened an appeal or otherwise indicated extreme
> discontent? If so, please summarise the areas of conflict in separate
> email messages to the Responsible Area Director. (It should be in a
> separate email because this questionnaire is publicly available.)

  No such discontent.

> (11) Identify any ID nits the Document Shepherd has found in this
> document. (See https://www.ietf.org/tools/idnits/ and the Internet-Drafts
> Checklist). Boilerplate checks are not enough; this check needs to be
> thorough.

  The idnits checker incorrectly reports issues with the references
  (the draft uses [...] to indicate ranges and lists, confusing the
  checker).

> (12) Describe how the document meets any required formal review
> criteria, such as the MIB Doctor, media type, and URI type reviews.

  No such review needed.

> (13) Have all references within this document been identified as
> either normative or informative?

  Yes.

> (14) Are there normative references to documents that are not ready for
> advancement or are otherwise in an unclear state? If such normative
> references exist, what is the plan for their completion?

  The normative references are either to RFCs or 3GPP references, except
  for draft-ietf-rmcat-eval-criteria which is being send for publication
  in parallel to this, and to the NS3 network simulator documentation.

> (15) Are there downward normative references references (see RFC 3967)?
> If so, list these downward references to support the Area Director in
> the Last Call procedure.

  No such references.

> (16) Will publication of this document change the status of any
> existing RFCs? Are those RFCs listed on the title page header, listed
> in the abstract, and discussed in the introduction? If the RFCs are not
> listed in the Abstract and Introduction, explain why, and point to the
> part of the document where the relationship of this document to the
> other RFCs is discussed. If this information is not in the document,
> explain why the WG considers it unnecessary.

  No changes made.

> (17) Describe the Document Shepherd's review of the IANA considerations
> section, especially with regard to its consistency with the body of the
> document. Confirm that all protocol extensions that the document makes
> are associated with the appropriate reservations in IANA registries.
> Confirm that any referenced IANA registries have been clearly
> identified. Confirm that newly created IANA registries include a
> detailed specification of the initial contents for the registry, that
> allocations procedures for future registrations are defined, and a
> reasonable name for the new registry has been suggested (see RFC 8126).

  No IANA actions.

> (18) List any new IANA registries that require Expert Review for future
> allocations. Provide any public guidance that the IESG would find
> useful in selecting the IANA Experts for these new registries.

  No IANA actions.

> (19) Describe reviews and automated checks performed by the Document
> Shepherd to validate sections of the document written in a formal
> language, such as XML code, BNF rules, MIB definitions, etc.

  None needed.

2019-09-24
08 Colin Perkins Document was updated in July 2019 to address the last WGLC comments. Waiting on shepherd write-up.
2019-09-24
08 Colin Perkins Tag Revised I-D Needed - Issue raised by WG cleared.
2019-09-24
08 Colin Perkins IETF WG state changed to Waiting for WG Chair Go-Ahead from In WG Last Call
2019-07-05
08 Xiaoqing Zhu New version available: draft-ietf-rmcat-wireless-tests-08.txt
2019-07-05
08 (System) New version approved
2019-07-05
08 (System)
Request for posting confirmation emailed to previous authors: rmcat-chairs@ietf.org, Xiaoqing Zhu , Zaheduzzaman Sarker , Ingemar Johansson , Wei-Tian Tan , Jian Fu , …
Request for posting confirmation emailed to previous authors: rmcat-chairs@ietf.org, Xiaoqing Zhu , Zaheduzzaman Sarker , Ingemar Johansson , Wei-Tian Tan , Jian Fu , Michael Ramalho
2019-07-05
08 Xiaoqing Zhu Uploaded new revision
2019-07-01
07 Xiaoqing Zhu New version available: draft-ietf-rmcat-wireless-tests-07.txt
2019-07-01
07 (System) New version approved
2019-07-01
07 (System)
Request for posting confirmation emailed to previous authors: rmcat-chairs@ietf.org, Xiaoqing Zhu , Zaheduzzaman Sarker , Ingemar Johansson , Wei-Tian Tan , Jian Fu , …
Request for posting confirmation emailed to previous authors: rmcat-chairs@ietf.org, Xiaoqing Zhu , Zaheduzzaman Sarker , Ingemar Johansson , Wei-Tian Tan , Jian Fu , Michael Ramalho
2019-07-01
07 Xiaoqing Zhu Uploaded new revision
2019-06-25
06 (System) Document has expired
2019-05-07
06 Colin Perkins Revised I-D needed to address review comments from Sergio Mena
2019-05-07
06 Colin Perkins Tag Revised I-D Needed - Issue raised by WG set.
2019-04-18
06 Colin Perkins Notification list changed to Colin Perkins <csp@csperkins.org>
2019-04-18
06 Colin Perkins Document shepherd changed to Colin Perkins
2019-01-14
06 Colin Perkins WG last call until 1 Feb 2019
2019-01-14
06 Colin Perkins IETF WG state changed to In WG Last Call from WG Document
2018-12-22
06 Xiaoqing Zhu New version available: draft-ietf-rmcat-wireless-tests-06.txt
2018-12-22
06 (System) New version approved
2018-12-22
06 (System)
Request for posting confirmation emailed to previous authors: rmcat-chairs@ietf.org, Xiaoqing Zhu , Zaheduzzaman Sarker , Ingemar Johansson , Wei-Tian Tan , Jian Fu , …
Request for posting confirmation emailed to previous authors: rmcat-chairs@ietf.org, Xiaoqing Zhu , Zaheduzzaman Sarker , Ingemar Johansson , Wei-Tian Tan , Jian Fu , Michael Ramalho
2018-12-22
06 Xiaoqing Zhu Uploaded new revision
2018-06-22
05 Xiaoqing Zhu New version available: draft-ietf-rmcat-wireless-tests-05.txt
2018-06-22
05 (System) New version approved
2018-06-22
05 (System)
Request for posting confirmation emailed to previous authors: rmcat-chairs@ietf.org, Xiaoqing Zhu , Zaheduzzaman Sarker , Ingemar Johansson , Wei-Tian Tan , Jian Fu , …
Request for posting confirmation emailed to previous authors: rmcat-chairs@ietf.org, Xiaoqing Zhu , Zaheduzzaman Sarker , Ingemar Johansson , Wei-Tian Tan , Jian Fu , Michael Ramalho
2018-06-22
05 Xiaoqing Zhu Uploaded new revision
2017-11-17
04 (System) Document has expired
2017-05-16
04 Xiaoqing Zhu New version available: draft-ietf-rmcat-wireless-tests-04.txt
2017-05-16
04 (System) New version approved
2017-05-16
04 (System)
Request for posting confirmation emailed to previous authors: rmcat-chairs@ietf.org, Xiaoqing Zhu , Zaheduzzaman Sarker , Ingemar Johansson , Wei-Tian Tan , Jian Fu , …
Request for posting confirmation emailed to previous authors: rmcat-chairs@ietf.org, Xiaoqing Zhu , Zaheduzzaman Sarker , Ingemar Johansson , Wei-Tian Tan , Jian Fu , Michael Ramalho
2017-05-16
04 Xiaoqing Zhu Uploaded new revision
2016-11-24
03 Colin Perkins Intended Status changed to Informational from None
2016-11-14
03 Xiaoqing Zhu New version available: draft-ietf-rmcat-wireless-tests-03.txt
2016-11-14
03 (System) New version approved
2016-11-14
03 (System)
Request for posting confirmation emailed to previous authors: rmcat-chairs@ietf.org, "Xiaoqing Zhu" , "Jian Fu" , "Michael Ramalho" , "Zaheduzzaman Sarker" , "Wei-Tian Tan" , …
Request for posting confirmation emailed to previous authors: rmcat-chairs@ietf.org, "Xiaoqing Zhu" , "Jian Fu" , "Michael Ramalho" , "Zaheduzzaman Sarker" , "Wei-Tian Tan" , "Ingemar Johansson"
2016-11-14
03 Xiaoqing Zhu Uploaded new revision
2016-11-14
02 (System) Document has expired
2016-05-07
02 Xiaoqing Zhu New version available: draft-ietf-rmcat-wireless-tests-02.txt
2015-11-05
01 Zaheduzzaman Sarker New version available: draft-ietf-rmcat-wireless-tests-01.txt
2015-06-12
00 Zaheduzzaman Sarker New version available: draft-ietf-rmcat-wireless-tests-00.txt