Characterization Guidelines for Active Queue Management (AQM)
draft-ietf-aqm-eval-guidelines-13
Revision differences
Document history
Date | Rev. | By | Action |
---|---|---|---|
2016-07-19
|
13 | (System) | RFC Editor state changed to AUTH48-DONE from AUTH48 |
2016-06-30
|
13 | (System) | RFC Editor state changed to AUTH48 from RFC-EDITOR |
2016-06-23
|
13 | (System) | RFC Editor state changed to RFC-EDITOR from EDIT |
2016-06-14
|
13 | (System) | RFC Editor state changed to EDIT |
2016-06-14
|
13 | (System) | IESG state changed to RFC Ed Queue from Approved-announcement sent |
2016-06-14
|
13 | (System) | Announcement was received by RFC Editor |
2016-06-14
|
13 | (System) | IANA Action state changed to No IC from In Progress |
2016-06-14
|
13 | (System) | IANA Action state changed to In Progress |
2016-06-14
|
13 | Cindy Morgan | IESG state changed to Approved-announcement sent from IESG Evaluation::AD Followup |
2016-06-14
|
13 | Cindy Morgan | IESG has approved the document |
2016-06-14
|
13 | Cindy Morgan | Closed "Approve" ballot |
2016-06-14
|
13 | Cindy Morgan | Ballot approval text was generated |
2016-06-14
|
13 | Cindy Morgan | Ballot writeup was changed |
2016-06-14
|
13 | Nicolas Kuhn | New version available: draft-ietf-aqm-eval-guidelines-13.txt |
2016-06-13
|
12 | Benoît Claise | [Ballot comment] - Random Early Detection (RED), BLUE, and Proportional Integral controller (PI) Would you have references? |
2016-06-13
|
12 | Benoît Claise | [Ballot Position Update] Position for Benoit Claise has been changed to No Objection from Discuss |
2016-06-10
|
12 | (System) | Sub state has been changed to AD Followup from Revised ID Needed |
2016-06-10
|
12 | Nicolas Kuhn | IANA Review state changed to Version Changed - Review Needed from IANA OK - No Actions Needed |
2016-06-10
|
12 | Nicolas Kuhn | New version available: draft-ietf-aqm-eval-guidelines-12.txt |
2016-06-09
|
11 | Ralph Droms | Request for Telechat review by GENART Completed: Ready. Reviewer: Ralph Droms. |
2016-05-19
|
11 | Amy Vezza | IESG state changed to IESG Evaluation::Revised I-D Needed from IESG Evaluation |
2016-05-19
|
11 | Joel Jaeggli | [Ballot comment] On 5/19/16 6:50 AM, João Taveira wrote: > Hi, > > Overall the document looks good, if somewhat strenuous to implement > fully … [Ballot comment] On 5/19/16 6:50 AM, João Taveira wrote: > Hi, > > Overall the document looks good, if somewhat strenuous to implement > fully in practice. Nitpicks inline. > > # Section 2.4 > > s/in the bottleneck available/at the bottleneck are available > > # Section 2.5 > > s/allows the tester evaluate/allows the tester to evaluate > s/This metric should be also/This metric should also be > > # Section 2.7 > > The initial paragraph of this section is confusing since it asks the > reader to consider the metrics "as explained in the rest of this > document", which obviously the reader will have not been exposed to. > > Some of the implementation details were ambiguous to me. For example: > "It is suggested to take measurements at least every K x minRTT (to > smooth out the fluctuations), with K=10. Higher values for K are > encouraged whenever it is more appropriate for the presentation of the > results.". There seems to be some contradiction between _suggesting_ > K=10, and then _encouraging_ a higher K, which would reduce the sampling > frequency in practice. Finally the paragraph suggests "It is generally > RECOMMENDED to provide at least 10 samples per RTT.", which would > suggest K = 1/10. I believe my misinterpretation stems from the fact a > measurement period is introduced (K x minRTT), but all subsequent > context refers to a measurement frequency. > > At the end of the section, section references could be enclosed in > parentheses since they are not part of the text: i.e. "This graph > provides part of a better understanding of (1) the delay/goodput > trade-off for a given congestion control mechanism (Section 5) ..." > > # Section 3.1 > > It is not obvious how the asymmetrical topology characterizes the AQM > system in any useful manner. If the objective is to see how a particular > AQM algorithm performs under TCP ACKs, it would be more useful to study > how the drop rate varies for streams of small packets. My main concern > here is that testing reverse path performance will be tightly coupled > with transport protocol behaviour, since you are reliant on a feedback loop. > > If you do decide that an asymmetrical test case is useful, it would be > helpful to provide more guidance on how asymmetrical a topology should > be. For TCP, is setting the capacity for the reverse path to 50% of the > forwarding path any different from the symmetrical case in practice? > > # Section 3.3 > > There is no baseline congestion control provided for LBE. Given LEDBAT > is mentioned in section 5.4, it would probably be wise to just define it > as a baseline in order to minimize implementation cost. > > # Section 4.4 > > s/impact the application performance/impact application performance > s/smaller probability/lower probability > > I'm somewhat surprised the topic of packet sizes is not covered more. > This section highlights that attackers can gain undue advantage by using > smaller packets, but if an AQM scheme does not take packet size into > account, the converse could be true. In datacenter environments in > particular, jumbo frames are relatively common, and so it would be > valuable to understand whether an AQM scheme is biased towards a > particular end of the size spectrum. > > # Section 5.4 > > s/any of those listed/such as any of those listed > > # Section 6.1 > > The first sentence of the second paragraph spans 6 lines, making it hard > to read. I'd suggest breaking it at "... can be discussed. In this > scenario ..." > > # Section 7.2 > > It is unclear what the intent of describing what a web page download > "could" be is. The document suggests that certain combinations of > patterns MUST be tested, but then leaves most of the details up to the > implementer, which makes comparisons between results trickier. > > This section would like benefit from being more agnostic to what > applications it is trying to mimic, and instead describe traffic mixes > in more general terms. As it stands it is unclear whether the intent of > the section is to approximate existing application patterns (which will > be out of date by the time the document is published), or underlying > traffic characteristics (bursty, inelastic, parallel, etc). > > # Section 13 > > This table would have been helpful sooner in the document as a summary > of implementation requirements. Independently of where it ends up > "summary" may be a better header than "conclusion", since it can be > picked up by a hasty implementer as a reference to the rest of the document. > > One nitpick in this table is the "Operator Control" row, which suggests > "Tuning SHOULD NOT be required.". This is more implementation guidance > than characterization guideline, and should therefore be out of scope > for this document (and indeed the recommendation is already part of > RFC7567). In terms of characterization it would be more appropriate for > this document to suggest that any tunable parameters MUST be explicitly > declared and their impact discussed, in the form of a YANG definition or > otherwise. > > Cheers, |
2016-05-19
|
11 | Joel Jaeggli | [Ballot Position Update] New position, No Objection, has been recorded for Joel Jaeggli |
2016-05-19
|
11 | Benoît Claise | [Ballot discuss] Has a RFC6390 performance directorate review done for the 2.X metrics? It should. See http://www.ietf.org/iesg/directorate/performance-metrics.html I guess that the metrics will be recorded … [Ballot discuss] Has a RFC6390 performance directorate review done for the 2.X metrics? It should. See http://www.ietf.org/iesg/directorate/performance-metrics.html I guess that the metrics will be recorded in the future (See draft-ietf-ippm-metric-registry-06 ), right? For example, Flow Completion Time and Packet Loss Synchronization are new, I believe. And some other metrics are already documented in RFC6390 compliant documents. Pointers should be provided. See https://tools.ietf.org/html/draft-ietf-xrblock-independent-burst-gap-discard-01#appendix-A for an example |
2016-05-19
|
11 | Benoît Claise | [Ballot comment] - Random Early Detection (RED), BLUE, and Proportional Integral controller (PI) Would you have references? - BDP is mentioned a few times. Please … [Ballot comment] - Random Early Detection (RED), BLUE, and Proportional Integral controller (PI) Would you have references? - BDP is mentioned a few times. Please expand. - Glossary section = terminology section, right? If we want to be consistent across documents - section 12.2. Why not a MUST below? In order to understand an AQM's deployment considerations and performance under a specific environment, AQM proposals SHOULD describe the parameters that control the macroscopic AQM behavior, and identify any parameters that require tuning to operational conditions. |
2016-05-19
|
11 | Benoît Claise | [Ballot Position Update] New position, Discuss, has been recorded for Benoit Claise |
2016-05-18
|
11 | Jari Arkko | [Ballot comment] Several issues from the Gen-ART review by Ralph Droms need to be addressed, in my opinion. In particular, Figure 1 or text under … [Ballot comment] Several issues from the Gen-ART review by Ralph Droms need to be addressed, in my opinion. In particular, Figure 1 or text under needs clarification, the advise on ECN seems superfluous, and section 10 example needs to be labeled as an example. |
2016-05-18
|
11 | Jari Arkko | [Ballot Position Update] New position, No Objection, has been recorded for Jari Arkko |
2016-05-18
|
11 | Alia Atlas | [Ballot Position Update] New position, No Objection, has been recorded for Alia Atlas |
2016-05-18
|
11 | Suresh Krishnan | [Ballot Position Update] New position, No Objection, has been recorded for Suresh Krishnan |
2016-05-18
|
11 | Ben Campbell | [Ballot Position Update] New position, No Objection, has been recorded for Ben Campbell |
2016-05-18
|
11 | Kathleen Moriarty | [Ballot Position Update] New position, No Objection, has been recorded for Kathleen Moriarty |
2016-05-18
|
11 | Spencer Dawkins | [Ballot comment] I'm pretty sure I know what "steady state" means in this text The transmission of the non application-limited flow must start … [Ballot comment] I'm pretty sure I know what "steady state" means in this text The transmission of the non application-limited flow must start before the transmission of the application-limited flow and only after the steady state has been reached by non application-limited flow. but I'm not sure how someone using this specification knows what it means, and it's asking the user to do something specific during evaluation. Is there a reference or definition you could provide? (There are other uses of the phrase "steady state" in the document, and they would also benefit, but this is the use that needs the precision) |
2016-05-18
|
11 | Spencer Dawkins | [Ballot Position Update] New position, Yes, has been recorded for Spencer Dawkins |
2016-05-18
|
11 | Alissa Cooper | [Ballot comment] (1) In the abstract, I'm not sure "precautionary" is really the right adjective. (2) Sec. 4.2 says: "However, the evaluation MUST be primarily … [Ballot comment] (1) In the abstract, I'm not sure "precautionary" is really the right adjective. (2) Sec. 4.2 says: "However, the evaluation MUST be primarily based on externally observed end-to-end metrics." This seems like a bit of a weird use of normative language. It makes sense to me that you would normatively recommend how to setup an evaluation environment to be able to compare AQMs, but if people want to weigh their evaluations in the wrong the direction, there is not much to be done about it. It seems like this would be more useful if it focused on explaining why primarily focusing on e2e measurements is preferable. (3) Overall I would recommend a pass through the whole document to check for the consistency with which normative language is used -- it seems like in some cases it isn't really necessary but it's there anyway, or its use is uneven (e.g., 4.3.2, 5.2, 6.1, 6.2, 6.3, 7.2 to name a few). There also seems to be a mix of normative statements about what proposals should say and how evaluation environments should be setup. It would help if each time a normative statement was made it clearly distinguished who the recommendation is directed towards. This became especially confusing in Sec. 13, where it is unclear whether "Scenario MUST be considered" means that an AQM spec needs to discuss this, or an AQM evaluation environment needs to test for this. (4) Since this document provides guidelines for how to characterize AQM proposals, it seems inappropriate for it to be making normative recommendations about how AQM proposals should work, even if they are just re-statements of things from RFC 7567. In particular, these statements seem out of place: - Sec. 4.4: "An AQM scheme SHOULD adhere to the recommendations outlined in [RFC7141], and SHOULD NOT provide undue advantage to flows with smaller packets [RFC7567]. - Sec. 4.5: "Deployed AQM algorithms SHOULD implement Explicit Congestion Notification (ECN) as well as loss to signal congestion to endpoints [RFC7567]." - Sec. 13: "Tuning SHOULD NOT be required" (5) Sec. 4.6 says: "This discussion as an instance, MAY explain whether the dropping policy is applied when packets are being enqueued or dequeued." I don't understand what "this discussion as an instance" means. (6) In Sec. 7.2, is there some justification that could be provided for choosing the particular parameters listed to simulate web traffic? E.g., are these parameters considered typical or is there a reference to some standard way of simulating web traffic that could be referenced? (7) Also in Sec. 7.2, why is the detail for the web flow given but the detail for bursty video is not? Surely there is more than one way to generate or simulate bursty video -- does it not matter what the bursts look like or how they are timed for this measurement purpose? (8) In 9.1, I would suggest that bi-directional video is prevalent and important enough to be listed in the traffic mix analysis as well. (9) I think the statement in Sec. 17 assumes that the tests envisioned by this document will be based on active measurements of simulated traffic. If that is a real scope limitation, and evaluation of AQMs based on passive measurement of traffic is out of scope, I think that should be stated somewhere in the document. (10) Just curious: has anyone thought about using LMAP, or the LMAP information model at least, to configure the kinds of tests envisioned in this document? |
2016-05-18
|
11 | Alissa Cooper | [Ballot Position Update] New position, No Objection, has been recorded for Alissa Cooper |
2016-05-18
|
11 | Alvaro Retana | [Ballot comment] Too bad there are no implementations... :-( |
2016-05-18
|
11 | Alvaro Retana | [Ballot Position Update] New position, No Objection, has been recorded for Alvaro Retana |
2016-05-18
|
11 | Stephen Farrell | [Ballot Position Update] New position, No Objection, has been recorded for Stephen Farrell |
2016-05-17
|
11 | Terry Manderson | [Ballot Position Update] New position, No Objection, has been recorded for Terry Manderson |
2016-05-16
|
11 | Deborah Brungard | [Ballot Position Update] New position, No Objection, has been recorded for Deborah Brungard |
2016-05-13
|
11 | Mirja Kühlewind | Notification list changed to wes@mti-systems.com |
2016-05-13
|
11 | Mirja Kühlewind | IESG state changed to IESG Evaluation from Waiting for Writeup |
2016-05-13
|
11 | Mirja Kühlewind | Ballot has been issued |
2016-05-13
|
11 | Mirja Kühlewind | [Ballot Position Update] New position, Yes, has been recorded for Mirja Kühlewind |
2016-05-13
|
11 | Mirja Kühlewind | Created "Approve" ballot |
2016-05-13
|
11 | Mirja Kühlewind | Ballot writeup was changed |
2016-05-13
|
11 | Mirja Kühlewind | Changed consensus to Yes from Unknown |
2016-05-13
|
11 | Peter Yee | Request for Telechat review by GENART is assigned to Ralph Droms |
2016-05-13
|
11 | Peter Yee | Request for Telechat review by GENART is assigned to Ralph Droms |
2016-05-09
|
11 | Gunter Van de Velde | Request for Last Call review by OPSDIR Completed: Has Issues. Reviewer: Linda Dunbar. |
2016-05-04
|
11 | (System) | IESG state changed to Waiting for Writeup from In Last Call |
2016-04-28
|
11 | Ralph Droms | Request for Last Call review by GENART Completed: On the Right Track. Reviewer: Ralph Droms. |
2016-04-28
|
11 | Tero Kivinen | Request for Last Call review by SECDIR Completed: Has Nits. Reviewer: Tero Kivinen. |
2016-04-27
|
11 | (System) | IANA Review state changed to IANA OK - No Actions Needed from IANA - Review Needed |
2016-04-27
|
11 | Sabrina Tanamal | (Via drafts-lastcall-comment@iana.org): IESG/Authors/WG Chairs: IANA has reviewed draft-ietf-aqm-eval-guidelines-11.txt, which is currently in Last Call, and has the following comments: We understand that this … (Via drafts-lastcall-comment@iana.org): IESG/Authors/WG Chairs: IANA has reviewed draft-ietf-aqm-eval-guidelines-11.txt, which is currently in Last Call, and has the following comments: We understand that this document doesn't require any IANA actions. While it's often helpful for a document's IANA Considerations section to remain in place upon publication even if there are no actions, if the authors strongly prefer to remove it, IANA does not object. If this assessment is not accurate, please respond as soon as possible. Thank you, Sabrina Tanamal IANA Specialist ICANN |
2016-04-23
|
11 | Gunter Van de Velde | Request for Last Call review by OPSDIR is assigned to Linda Dunbar |
2016-04-23
|
11 | Gunter Van de Velde | Request for Last Call review by OPSDIR is assigned to Linda Dunbar |
2016-04-22
|
11 | Mirja Kühlewind | Placed on agenda for telechat - 2016-05-19 |
2016-04-21
|
11 | Jean Mahoney | Request for Last Call review by GENART is assigned to Ralph Droms |
2016-04-21
|
11 | Jean Mahoney | Request for Last Call review by GENART is assigned to Ralph Droms |
2016-04-21
|
11 | Tero Kivinen | Request for Last Call review by SECDIR is assigned to Tero Kivinen |
2016-04-21
|
11 | Tero Kivinen | Request for Last Call review by SECDIR is assigned to Tero Kivinen |
2016-04-21
|
11 | Mirja Kühlewind | As required by RFC 4858, this is the current template for the Document Shepherd Write-Up. Changes are expected over time. This version is dated … As required by RFC 4858, this is the current template for the Document Shepherd Write-Up. Changes are expected over time. This version is dated 24 February 2012. (1) What type of RFC is being requested (BCP, Proposed Standard, Internet Standard, Informational, Experimental, or Historic)? Why is this the proper type of RFC? Is this type of RFC indicated in the title page header? Informational; this is appropriate. (2) The IESG approval announcement includes a Document Announcement Write-Up. Please provide such a Document Announcement Write-Up. Recent examples can be found in the "Action" announcements for approved documents. The approval announcement contains the following sections: Technical Summary Relevant content can frequently be found in the abstract and/or introduction of the document. If not, this may be an indication that there are deficiencies in the abstract or introduction. Abstract: Unmanaged large buffers in today's networks have given rise to a slew of performance issues. These performance issues can be addressed by some form of Active Queue Management (AQM) mechanism, optionally in combination with a packet scheduling scheme such as fair queuing. This document describes various criteria for performing precautionary characterizations of AQM schemes. Working Group Summary Was there anything in WG process that is worth noting? For example, was there controversy about particular points or were there decisions where the consensus was particularly rough? Over it's lifetime, this document changed scope a couple of times. It currently contains guidance for characterizing AQM algorithm performance, though it had started out to prescribe evaluation methods for AQM algorithms being considered in the working group. As time went on, the focus shifted, and the WG did not use the guidelines in a prescriptive way. The current guidelines are not fully parameterized, and it is assumed that someone performing a characterization would consider how to parameterize configurations relevant to their own networks or scenarios of interest. Ideally there would be running code (as mentioned by Dave, among others) for performing the characterizations, e.g. implemented using some tool such as ns-3, mininet, etc, however, this has not come about yet. Where it sits now, the document is suitable as Informational guidance, but it is not a prescriptive and fully defined spec, like for example, an IPPM metric would be. It simply collects guidance and description of the set of things to look at and the measurements of interest in common AQM performance testing situations. Document Quality Are there existing implementations of the protocol? Have a significant number of vendors indicated their plan to implement the specification? Are there any reviewers that merit special mention as having done a thorough review, e.g., one that resulted in important changes or a conclusion that the document had no substantive issues? If there was a MIB Doctor, Media Type or other expert review, what was its course (briefly)? In the case of a Media Type review, on what date was the request posted? The document is of sufficient quality that someone (e.g. a graduate student) could read it and implement proper tests. Personnel Who is the Document Shepherd? Who is the Responsible Area Director? Wesley Eddy (wes@mti-systems.com) is the shepherd, and Mirja Kühlewind (ietf@kuehlewind.net) is the AD. (3) Briefly describe the review of this document that was performed by the Document Shepherd. If this version of the document is not ready for publication, please explain why the document is being forwarded to the IESG. I have reviewed it multiple times; it is ready for publication. (4) Does the document Shepherd have any concerns about the depth or breadth of the reviews that have been performed? No concerns. (5) Do portions of the document need review from a particular or from broader perspective, e.g., security, operational complexity, AAA, DNS, DHCP, XML, or internationalization? If so, describe the review that took place. N/A (6) Describe any specific concerns or issues that the Document Shepherd has with this document that the Responsible Area Director and/or the IESG should be aware of? For example, perhaps he or she is uncomfortable with certain parts of the document, or has concerns whether there really is a need for it. In any event, if the WG has discussed those issues and has indicated that it still wishes to advance the document, detail those concerns here. During last call in the WG there were concerns raised about the test descriptions not being fully specified, but the document at that point was inconsistent about its goals, and it seems that fixing that remedied the issue. There was not any threat of appeal or other action though, and after announcing the intention to continue with the revised document following WGLC, there was no protest. (7) Has each author confirmed that any and all appropriate IPR disclosures required for full conformance with the provisions of BCP 78 and BCP 79 have already been filed. If not, explain why. Yes. (8) Has an IPR disclosure been filed that references this document? If so, summarize any WG discussion and conclusion regarding the IPR disclosures. No. (9) How solid is the WG consensus behind this document? Does it represent the strong concurrence of a few individuals, with others being silent, or does the WG as a whole understand and agree with it? Reasonable consensus. (10) Has anyone threatened an appeal or otherwise indicated extreme discontent? If so, please summarise the areas of conflict in separate email messages to the Responsible Area Director. (It should be in a separate email because this questionnaire is publicly available.) No. (11) Identify any ID nits the Document Shepherd has found in this document. (See https://www.ietf.org/tools/idnits/ and the Internet-Drafts Checklist). Boilerplate checks are not enough; this check needs to be thorough. There are a couple of warnings about obsolete references; this can be fixed by the RFC Editor, but does not impact the tests described in the document. (12) Describe how the document meets any required formal review criteria, such as the MIB Doctor, media type, and URI type reviews. N/A. (13) Have all references within this document been identified as either normative or informative? Yes. (14) Are there normative references to documents that are not ready for advancement or are otherwise in an unclear state? If such normative references exist, what is the plan for their completion? No. (15) Are there downward normative references references (see RFC 3967)? If so, list these downward references to support the Area Director in the Last Call procedure. No. (16) Will publication of this document change the status of any existing RFCs? Are those RFCs listed on the title page header, listed in the abstract, and discussed in the introduction? If the RFCs are not listed in the Abstract and Introduction, explain why, and point to the part of the document where the relationship of this document to the other RFCs is discussed. If this information is not in the document, explain why the WG considers it unnecessary. N/A. (17) Describe the Document Shepherd's review of the IANA considerations section, especially with regard to its consistency with the body of the document. Confirm that all protocol extensions that the document makes are associated with the appropriate reservations in IANA registries. Confirm that any referenced IANA registries have been clearly identified. Confirm that newly created IANA registries include a detailed specification of the initial contents for the registry, that allocations procedures for future registrations are defined, and a reasonable name for the new registry has been suggested (see RFC 5226). There are no IANA considerations. (18) List any new IANA registries that require Expert Review for future allocations. Provide any public guidance that the IESG would find useful in selecting the IANA Experts for these new registries. N/A (19) Describe reviews and automated checks performed by the Document Shepherd to validate sections of the document written in a formal language, such as XML code, BNF rules, MIB definitions, etc. N/A |
2016-04-20
|
11 | Amy Vezza | IANA Review state changed to IANA - Review Needed |
2016-04-20
|
11 | Amy Vezza | The following Last Call announcement was sent out: From: The IESG To: "IETF-Announce" CC: wes@mti-systems.com, ietf@kuehlewind.net, aqm-chairs@ietf.org, draft-ietf-aqm-eval-guidelines@ietf.org, aqm@ietf.org Reply-To: ietf@ietf.org … The following Last Call announcement was sent out: From: The IESG To: "IETF-Announce" CC: wes@mti-systems.com, ietf@kuehlewind.net, aqm-chairs@ietf.org, draft-ietf-aqm-eval-guidelines@ietf.org, aqm@ietf.org Reply-To: ietf@ietf.org Sender: Subject: Last Call: (AQM Characterization Guidelines) to Informational RFC The IESG has received a request from the Active Queue Management and Packet Scheduling WG (aqm) to consider the following document: - 'AQM Characterization Guidelines' as Informational RFC The IESG plans to make a decision in the next few weeks, and solicits final comments on this action. Please send substantive comments to the ietf@ietf.org mailing lists by 2016-05-04. Exceptionally, comments may be sent to iesg@ietf.org instead. In either case, please retain the beginning of the Subject line to allow automated sorting. Abstract Unmanaged large buffers in today's networks have given rise to a slew of performance issues. These performance issues can be addressed by some form of Active Queue Management (AQM) mechanism, optionally in combination with a packet scheduling scheme such as fair queuing. This document describes various criteria for performing precautionary characterizations of AQM schemes. The file can be obtained via https://datatracker.ietf.org/doc/draft-ietf-aqm-eval-guidelines/ IESG discussion can be tracked via https://datatracker.ietf.org/doc/draft-ietf-aqm-eval-guidelines/ballot/ No IPR declarations have been submitted directly on this I-D. |
2016-04-20
|
11 | Amy Vezza | IESG state changed to In Last Call from Last Call Requested |
2016-04-20
|
11 | Mirja Kühlewind | Last call was requested |
2016-04-20
|
11 | Mirja Kühlewind | Ballot approval text was generated |
2016-04-20
|
11 | Mirja Kühlewind | IESG state changed to Last Call Requested from AD Evaluation |
2016-04-20
|
11 | Mirja Kühlewind | Last call announcement was changed |
2016-04-18
|
11 | Mirja Kühlewind | Ballot writeup was generated |
2016-04-18
|
11 | Mirja Kühlewind | Last call announcement was generated |
2016-04-06
|
11 | Cindy Morgan | Shepherding AD changed to Mirja Kühlewind |
2016-03-21
|
11 | Martin Stiemerling | IESG state changed to AD Evaluation from Publication Requested |
2016-03-14
|
11 | Wesley Eddy | As required by RFC 4858, this is the current template for the Document Shepherd Write-Up. Changes are expected over time. This version is dated … As required by RFC 4858, this is the current template for the Document Shepherd Write-Up. Changes are expected over time. This version is dated 24 February 2012. (1) What type of RFC is being requested (BCP, Proposed Standard, Internet Standard, Informational, Experimental, or Historic)? Why is this the proper type of RFC? Is this type of RFC indicated in the title page header? Informational; this is appropriate. (2) The IESG approval announcement includes a Document Announcement Write-Up. Please provide such a Document Announcement Write-Up. Recent examples can be found in the "Action" announcements for approved documents. The approval announcement contains the following sections: Technical Summary Relevant content can frequently be found in the abstract and/or introduction of the document. If not, this may be an indication that there are deficiencies in the abstract or introduction. Abstract: Unmanaged large buffers in today's networks have given rise to a slew of performance issues. These performance issues can be addressed by some form of Active Queue Management (AQM) mechanism, optionally in combination with a packet scheduling scheme such as fair queuing. This document describes various criteria for performing precautionary characterizations of AQM schemes. Working Group Summary Was there anything in WG process that is worth noting? For example, was there controversy about particular points or were there decisions where the consensus was particularly rough? Over it's lifetime, this document changed scope a couple of times. It currently contains guidance for characterizing AQM algorithm performance, though it had started out to prescribe evaluation methods for AQM algorithms being considered in the working group. As time went on, the focus shifted, and the WG did not use the guidelines in a prescriptive way. The current guidelines are not fully parameterized, and it is assumed that someone performing a characterization would consider how to parameterize configurations relevant to their own networks or scenarios of interest. Ideally there would be running code (as mentioned by Dave, among others) for performing the characterizations, e.g. implemented using some tool such as ns-3, mininet, etc, however, this has not come about yet. Where it sits now, the document is suitable as Informational guidance, but it is not a prescriptive and fully defined spec, like for example, an IPPM metric would be. It simply collects guidance and description of the set of things to look at and the measurements of interest in common AQM performance testing situations. Document Quality Are there existing implementations of the protocol? Have a significant number of vendors indicated their plan to implement the specification? Are there any reviewers that merit special mention as having done a thorough review, e.g., one that resulted in important changes or a conclusion that the document had no substantive issues? If there was a MIB Doctor, Media Type or other expert review, what was its course (briefly)? In the case of a Media Type review, on what date was the request posted? The document is of sufficient quality that someone (e.g. a graduate student) could read it and implement proper tests. Personnel Who is the Document Shepherd? Who is the Responsible Area Director? Wesley Eddy (wes@mti-systems.com) is the shepherd, and Martin Stiemerling (mls.ietf@gmail.com) is the AD. (3) Briefly describe the review of this document that was performed by the Document Shepherd. If this version of the document is not ready for publication, please explain why the document is being forwarded to the IESG. I have reviewed it multiple times; it is ready for publication. (4) Does the document Shepherd have any concerns about the depth or breadth of the reviews that have been performed? No concerns. (5) Do portions of the document need review from a particular or from broader perspective, e.g., security, operational complexity, AAA, DNS, DHCP, XML, or internationalization? If so, describe the review that took place. N/A (6) Describe any specific concerns or issues that the Document Shepherd has with this document that the Responsible Area Director and/or the IESG should be aware of? For example, perhaps he or she is uncomfortable with certain parts of the document, or has concerns whether there really is a need for it. In any event, if the WG has discussed those issues and has indicated that it still wishes to advance the document, detail those concerns here. During last call in the WG there were concerns raised about the test descriptions not being fully specified, but the document at that point was inconsistent about its goals, and it seems that fixing that remedied the issue. There was not any threat of appeal or other action though, and after announcing the intention to continue with the revised document following WGLC, there was no protest. (7) Has each author confirmed that any and all appropriate IPR disclosures required for full conformance with the provisions of BCP 78 and BCP 79 have already been filed. If not, explain why. Yes. (8) Has an IPR disclosure been filed that references this document? If so, summarize any WG discussion and conclusion regarding the IPR disclosures. No. (9) How solid is the WG consensus behind this document? Does it represent the strong concurrence of a few individuals, with others being silent, or does the WG as a whole understand and agree with it? Reasonable consensus. (10) Has anyone threatened an appeal or otherwise indicated extreme discontent? If so, please summarise the areas of conflict in separate email messages to the Responsible Area Director. (It should be in a separate email because this questionnaire is publicly available.) No. (11) Identify any ID nits the Document Shepherd has found in this document. (See https://www.ietf.org/tools/idnits/ and the Internet-Drafts Checklist). Boilerplate checks are not enough; this check needs to be thorough. There are a couple of warnings about obsolete references; this can be fixed by the RFC Editor, but does not impact the tests described in the document. (12) Describe how the document meets any required formal review criteria, such as the MIB Doctor, media type, and URI type reviews. N/A. (13) Have all references within this document been identified as either normative or informative? Yes. (14) Are there normative references to documents that are not ready for advancement or are otherwise in an unclear state? If such normative references exist, what is the plan for their completion? No. (15) Are there downward normative references references (see RFC 3967)? If so, list these downward references to support the Area Director in the Last Call procedure. No. (16) Will publication of this document change the status of any existing RFCs? Are those RFCs listed on the title page header, listed in the abstract, and discussed in the introduction? If the RFCs are not listed in the Abstract and Introduction, explain why, and point to the part of the document where the relationship of this document to the other RFCs is discussed. If this information is not in the document, explain why the WG considers it unnecessary. N/A. (17) Describe the Document Shepherd's review of the IANA considerations section, especially with regard to its consistency with the body of the document. Confirm that all protocol extensions that the document makes are associated with the appropriate reservations in IANA registries. Confirm that any referenced IANA registries have been clearly identified. Confirm that newly created IANA registries include a detailed specification of the initial contents for the registry, that allocations procedures for future registrations are defined, and a reasonable name for the new registry has been suggested (see RFC 5226). There are no IANA considerations. (18) List any new IANA registries that require Expert Review for future allocations. Provide any public guidance that the IESG would find useful in selecting the IANA Experts for these new registries. N/A (19) Describe reviews and automated checks performed by the Document Shepherd to validate sections of the document written in a formal language, such as XML code, BNF rules, MIB definitions, etc. N/A |
2016-03-14
|
11 | Wesley Eddy | Responsible AD changed to Martin Stiemerling |
2016-03-14
|
11 | Wesley Eddy | IETF WG state changed to Submitted to IESG for Publication from WG Consensus: Waiting for Write-Up |
2016-03-14
|
11 | Wesley Eddy | IESG state changed to Publication Requested |
2016-03-14
|
11 | Wesley Eddy | IESG process started in state Publication Requested |
2016-03-14
|
11 | Wesley Eddy | Tag Doc Shepherd Follow-up Underway cleared. |
2016-03-14
|
11 | Wesley Eddy | Changed document writeup |
2016-03-07
|
11 | Wesley Eddy | Tag Doc Shepherd Follow-up Underway set. |
2016-03-07
|
11 | Wesley Eddy | IETF WG state changed to WG Consensus: Waiting for Write-Up from In WG Last Call |
2016-02-15
|
11 | Nicolas Kuhn | New version available: draft-ietf-aqm-eval-guidelines-11.txt |
2016-02-02
|
10 | Nicolas Kuhn | New version available: draft-ietf-aqm-eval-guidelines-10.txt |
2015-11-27
|
09 | Nicolas Kuhn | New version available: draft-ietf-aqm-eval-guidelines-09.txt |
2015-10-14
|
08 | (System) | Notify list changed from "Wesley Eddy" to (None) |
2015-09-21
|
08 | Nicolas Kuhn | New version available: draft-ietf-aqm-eval-guidelines-08.txt |
2015-08-10
|
07 | Wesley Eddy | Notification list changed to "Wesley Eddy" <wes@mti-systems.com> |
2015-08-10
|
07 | Wesley Eddy | Document shepherd changed to Wesley Eddy |
2015-08-10
|
07 | Wesley Eddy | IETF WG state changed to In WG Last Call from WG Document |
2015-07-23
|
07 | Richard Scheffenegger | Intended Status changed to Informational from None |
2015-07-06
|
07 | Nicolas Kuhn | New version available: draft-ietf-aqm-eval-guidelines-07.txt |
2015-06-30
|
06 | Nicolas Kuhn | New version available: draft-ietf-aqm-eval-guidelines-06.txt |
2015-06-29
|
05 | Nicolas Kuhn | New version available: draft-ietf-aqm-eval-guidelines-05.txt |
2015-06-19
|
04 | Nicolas Kuhn | New version available: draft-ietf-aqm-eval-guidelines-04.txt |
2015-05-21
|
03 | Nicolas Kuhn | New version available: draft-ietf-aqm-eval-guidelines-03.txt |
2015-03-23
|
02 | Naeem Khademi | New version available: draft-ietf-aqm-eval-guidelines-02.txt |
2015-03-05
|
01 | Naeem Khademi | New version available: draft-ietf-aqm-eval-guidelines-01.txt |
2015-03-05
|
01 | Naeem Khademi | New version available: draft-ietf-aqm-eval-guidelines-01.txt |
2014-09-18
|
00 | Wesley Eddy | This document now replaces draft-kuhn-aqm-eval-guidelines instead of None |
2014-09-18
|
00 | Nicolas Kuhn | New version available: draft-ietf-aqm-eval-guidelines-00.txt |