Applicability Statement for RFC 2544: Use on Production Networks Considered Harmful
draft-ietf-bmwg-2544-as-08
Revision differences
Document history
Date | Rev. | By | Action |
---|---|---|---|
2012-10-24
|
08 | Cindy Morgan | State changed to RFC Ed Queue from Approved-announcement sent |
2012-10-23
|
08 | (System) | IANA Action state changed to No IC |
2012-10-23
|
08 | Cindy Morgan | State changed to Approved-announcement sent from Approved-announcement sent |
2012-10-23
|
08 | Cindy Morgan | IESG has approved the document |
2012-10-23
|
08 | Cindy Morgan | Closed "Approve" ballot |
2012-10-23
|
08 | Cindy Morgan | Ballot approval text was generated |
2012-10-23
|
08 | Cindy Morgan | Ballot writeup was changed |
2012-10-23
|
08 | Ron Bonica | State changed to Approved-announcement sent from Approved-announcement sent::Point Raised - writeup needed |
2012-10-23
|
08 | Ron Bonica | State changed to Approved-announcement sent::Point Raised - writeup needed from IESG Evaluation::AD Followup |
2012-10-22
|
08 | Stewart Bryant | [Ballot comment] Thank you for your work you have done to address my concerns. |
2012-10-22
|
08 | Stewart Bryant | [Ballot Position Update] Position for Stewart Bryant has been changed to No Objection from Discuss |
2012-10-22
|
08 | Al Morton | New version available: draft-ietf-bmwg-2544-as-08.txt |
2012-09-20
|
07 | Tero Kivinen | Closed request for Last Call review by SECDIR with state 'No Response' |
2012-09-19
|
07 | Al Morton | New version available: draft-ietf-bmwg-2544-as-07.txt |
2012-09-13
|
06 | Gonzalo Camarillo | [Ballot Position Update] New position, No Objection, has been recorded for Gonzalo Camarillo |
2012-09-13
|
06 | Benoît Claise | [Ballot comment] I agree with the intend of the draft. Hence no objection. Text from Al Morton We're saying that almost any test using RFC2544 … [Ballot comment] I agree with the intend of the draft. Hence no objection. Text from Al Morton We're saying that almost any test using RFC2544 methods on a production network will produce a form of harm from wrong and misleading results with inherent wasted time and resources, and that's just one of the ways we defined harm in version 05 clarifications. In the test and measurement community (the main audience of this draft), frequent flawed results are the end of the world with too few transport ships ready to go. I agree with Al: "a form of harm from wrong and misleading results" While I understand Stewart's point (a customer could use RFC2544 to test his CIR), the Device Under Test (DUT) is a black box, and BMWG does black box testing. Per the "black box" definition, we can't control or even monitor what's in it. By testing live network, the DUT becomes the network (or a circuit). And we have no way to know what we influence, if anything This is the way I understand harmful. That being said. Harmful might be understood in a different way. The title "Use on Production Networks Considered Harmful" might be a little bit alarming... While the text in the draft is more realistic - abstract "Overload of shared resources would likely be harmful to user traffic performance on a production network" - section 5 "Overload of shared resources would likely be harmful to user traffic performance on a production network" Text from Scott Bradner We feel that use of the 2544 in production networks is harmful in the sense that IETF has been using that term for a very long time. Some of the tests would harm other users of the network and some of the tests would get bad (misleading) results. Any wording that would soften the word "harmful" would be a step in the right direction IMHO. Alternatively, your own definition of harmful would also help. For example, "By harmful, we mean that the consequences are unknown/unpredictable/etc ..." or "By harmful, we mean: a form of harm from wrong and misleading results" |
2012-09-13
|
06 | Benoît Claise | [Ballot Position Update] New position, No Objection, has been recorded for Benoit Claise |
2012-09-11
|
06 | Adrian Farrel | [Ballot comment] I am retaining my Yes ballot, and I thank the authors for addressing my Comments entered against revision 05. It continues to be … [Ballot comment] I am retaining my Yes ballot, and I thank the authors for addressing my Comments entered against revision 05. It continues to be my opinion that, in stating an important point about the use of these tests on live, and dynamically routed, shared-resource IP/MPLS networks, the document overstates its case an implies that the use on network paths in dedicated-resource networks (such as MPLS-TE or MPLS-TP) is harmful when it is neither obvious nor proven that this is the case. In this, I agree with Stewart's Discuss and wish that the authors would make a clearer and more upfront statement about the scope of the network in which this may be harmful. The solution to this may be as simple as a change to the title to something like: Use of RFC 2544 Benchmarking Test Methodologies on Production Networks with Shared Resources Considered Harmful In making this change you would not be commenting about TE or transport networks, but would be making the clear statement about where you know the harm to be. --- New typo introduced... Section 4.2 In other words, devices operating on the Internet may be configured to discard any traffic they observe in this address range, as it is intended for laboratory ITE use only. Thus, testers using the assigned testing address ranges are connected to the Internet and test packets are forwarded across the Internet, it is likely that the packets will be discarded and the test will not work. s/Thus, testers/Thus, if testers/ |
2012-09-11
|
06 | Adrian Farrel | Ballot comment text updated for Adrian Farrel |
2012-09-09
|
06 | Ron Bonica | Telechat date has been changed to 2012-09-13 from 2012-08-30 |
2012-09-05
|
06 | Stewart Bryant | [Ballot discuss] The steps that I agreed with the sponsoring Area Director to resolve my Discuss were that the comments posted by Adrian would be … [Ballot discuss] The steps that I agreed with the sponsoring Area Director to resolve my Discuss were that the comments posted by Adrian would be included and I would then re-review the draft to see it it addressed my concerns. I note that a number of Adrian's comments have not been included in version 6 of the draft. I would additionally like to propose two changes to the draft that I think would address the fundamental basis of the my concerns, and if these are acceptable to the authors I will review the updated text with a view to clearing my discuss. I think one of the following document titles would be more appropriate: a) RFC 2544 Applicability Statement: Use on Production Networks Employing Contended Resources Considered Harmful or b) RFC 2544 Applicability Statement: Describing When Used on Production Networks Can be Considered Harmful My preference is for (a), but I can accept (b). Then in the Abstract and Introduction there needs to be some scoping text that describes the network situation considered by the authors. This document is scoped to the case where the production network provides a service using contended resources. Production networks using un-contended resources are out of scope. |
2012-09-05
|
06 | Stewart Bryant | Ballot discuss text updated for Stewart Bryant |
2012-09-04
|
06 | (System) | Sub state has been changed to AD Followup from Revised ID Needed |
2012-09-04
|
06 | Al Morton | New version available: draft-ietf-bmwg-2544-as-06.txt |
2012-08-30
|
05 | Cindy Morgan | State changed to IESG Evaluation::Revised ID Needed from IESG Evaluation |
2012-08-29
|
05 | Ralph Droms | [Ballot Position Update] New position, No Objection, has been recorded for Ralph Droms |
2012-08-29
|
05 | Russ Housley | [Ballot Position Update] New position, No Objection, has been recorded for Russ Housley |
2012-08-29
|
05 | Robert Sparks | [Ballot Position Update] New position, No Objection, has been recorded for Robert Sparks |
2012-08-29
|
05 | Barry Leiba | [Ballot comment] To directly address this situation, the past and present Chairs of the IETF Benchmarking Methodology Working Group (BMWG) have prepared … [Ballot comment] To directly address this situation, the past and present Chairs of the IETF Benchmarking Methodology Working Group (BMWG) have prepared this Applicability Statement for RFC 2544. I'm going to push this also, where Pete gave it up... though it's still non-blocking. I very strongly ask you to change this. (1) Working group documents are the products of working groups, not of individuals. (2) Statements like this imply that people in positions of leadership rammed something through, forcing "consensus" with a bulldozer. (3) At best, statements like this appear to be appeal-to-authority arguments. We should avoid that when it's not necessary, and I don't see that it's necessary here. If BMWG has strong consensus on what this document says, then say *that*. That statement means more to me than one that says that four chairs thought it was important. (I'll also note that when you couple that statement with the Acknowledgments section, which recognizes five people and doesn't mention the working group, one *could* take away from this that there were nine people who read and agreed with this. Saying instead -- explicitly, rather than the usual implicit message we give -- that this represents *strong* consensus of the BMWG makes it very clear.) Apart from that, I'll let Stewart and Adrian sort out whether or not you're overstating anything. |
2012-08-29
|
05 | Barry Leiba | [Ballot Position Update] New position, No Objection, has been recorded for Barry Leiba |
2012-08-29
|
05 | Adrian Farrel | [Ballot comment] I have reviewed this document on a number of occasions before it came for IESG review. The text is much improved and I … [Ballot comment] I have reviewed this document on a number of occasions before it came for IESG review. The text is much improved and I thank the authors for their work. Isuport the publication of this document, but I agree with Stewart that the case appears to be overstated. I believe that much of the problem is stylistic, but it leads to some statements that are stronger than they need to be. It is simply not the case that all use of 2544 mechanisms on production networks are perforce harmful. What we need to do is strike a balance between pointing out the risks and indicating that use of 2544 is inadvisable because of those risks on the one hand, and trying to use an Informational RFC to ban people from doing stupid things. Here are some suggested edits to improve matters... Abstract OLD Recent application of the methods beyond their intended scope is cause for concern. NEW NOTE I don't think this statement is expanded upon in the document (and if it were, I would be suggesting that pointing fingers is not helpful) and I don't think it matters. Your advice presumably holds true whether or not people have started to do silly things. END Abstract OLD The methods described in RFC 2544, where overload is a possible outcome, would no doubt be harmful to user traffic performance on a production network. NEW The methods described in RFC 2544 generate traffic that load the network. In heavily utilized networks, the combination of user traffic and benchmarking traffic could cause network overload that would potentially harm user traffic performance. an NOTE The phrase "would no doubt be harmful" is not correct when applied to "a possible outcome". In fact, it is the overload itself that is the harmful effect. END Abstract OLD This memo clarifies the scope of RFC 2544 and other benchmarking work for the IETF community. NEW This memo clarifies the scope of RFC 2544 and other benchmarking work. NOTE I think that, although you are working within the IETF community, your actual intention is wider visiblity. END Section 1 OLD This memo clarifies the scope of RFC 2544 [RFC2544], which discusses and defines several tests that may be used to characterize the performance of a network interconnecting device, and other benchmarking work for the IETF community. NEW This memo clarifies the scope and use of benchmarking tests including RFC 2544 [RFC2544], which discusses and defines several tests that may be used to characterize the performance of a network interconnecting device. END Section 1 OLD Benchmarking Methodologies (beginning with [RFC2544]) have always relied on test conditions that can only be produced and replicated reliably in the laboratory. Thus it was unfortunate to find that this foundation methodology was being cited in several unintended specifications and products performing applications such as: 1. Validation of telecommunication service configuration, such as the Committed Information Rate (CIR). 2. Validation of performance metrics in a telecommunication Service Level Agreement (SLA), such as frame loss and latency. 3. Telecommunication service activation testing, where traffic that shares network resources with the test might be adversely affected. NEW Benchmarking Methodologies (beginning with [RFC2544]) rely on test conditions that can only be produced and replicatedreliably in the laboratory. These methodologies are not appropriate for inclusion in wider specifications such as: 1. Validation of telecommunication service configuration, such as the Committed Information Rate (CIR). 2. Validation of performance metrics in a telecommunication Service Level Agreement (SLA), such as frame loss and latency. 3. Telecommunication service activation testing, where traffic that shares network resources with the test might be adversely affected. NOTE I think that the commentary is itself "unfortunate." It is cleaner to state what is not approriate, and move on. END Section 1 OLD Although RFC 2544 is held up as the standard reference for such testing, we believe that the actual methods used vary from RFC 2544 in significant ways. Since the only citation is to RFC 2544, the modifications are opaque to the standards community and to users in general (an undesirable situation). There is risk of harm to user traffic from applying the test traffic and methods described in [RFC2544] on a production network, because overload in shared resources is a possible outcome. To directly address this situation, the past and present Chairs of the IETF Benchmarking Methodology Working Group (BMWG) have prepared this Applicability Statement for RFC 2544. NEW RFC 2544 is used as a key reference for such testing implying that the mechanisms are directly derived from RFC 2544. However, the actual methods often vary from the mechanisms described in RFC 2544, and the modifications are opaque to the standards community and to users in general. Since applying the test traffic and methods described in RFC 2544 on a production network risks causing overload in shared resources, there is a direct risk ofharming user traffic if these mechanisms are misued. To directly address this situation, the past and present Chairs of the IETF Benchmarking Methodology Working Group (BMWG) have prepared this Applicability Statement for RFC 2544. NOTE Again, the commentary is unnecessarily perjorative when direct report would be cleaner. END Section 4 OLD 4. Why RFC 2544 Methods are intended for ITE NEW 4. Why RFC 2544 Methods are intended only for ITE END Section 4 OLD The following sections discuss some of the reasons why RFC 2544 [RFC2544] methods were intended only for isolated laboratory use, and the difficulties of applying these methods outside the lab environment. NEW The following sections discuss some of the reasons why RFC 2544 [RFC2544] methods are applicable only for isolated laboratory use, and the consequences of applying these methods outside the lab environment. NOTE I don't think that the intention when 2544 was written has as much bearing as you imply. What is important is the applicability now. That is, you could have intended use only in ITE but discovered wide applicability later. I also think that "difficulties" implies "it is hard, but you can do it." That is not the message you intend to send. END Section 4.1 OLD All of the tests described in RFC 2544 require that the tester and device under test are the only devices on the networks that are transmitting data. The presence of other unwanted traffic on the network would mean that the specified test conditions have not been achieved. If any unwanted traffic appears and the amount varies over time, the repeatability of any test result will likely depend to some degree on the unwanted traffic. The presence of unwanted or unknown traffic makes accurate, repeatable, and consistent measurements of the performance of the device under test very unlikely, since the complete details of test conditions will not be reported. NEW All of the tests described in RFC 2544 require that the tester and device under test are the only devices on the networks that are transmitting data. The presence of other traffic on the network would mean that the specified test conditions have not been achieved and the results could not be guaranteed to be valid. If any other traffic appears and the amount varies over time, the repeatability of any test result will likely depend to some degree on the amount and variation of the other traffic. The presence of other traffic makes accurate, repeatable, and consistent measurements of the performance of the device under test very unlikely, since the complete details of test conditions will not be reported. NOTE Two things... The traffic *is* wanted. It is just not wanted by you. Need to say why it matters if the test conditions are not achieved. END Section 4.2 OLD In other words, devices operating on the Internet may be configured to discard any traffic they observe in this address range, as it is intended for laboratory ITE use only. Thus, testers using the assigned testing address ranges MUST NOT be connected to the Internet. NEW In other words, devices operating on the Internet may be configured to discard any traffic they observe in this address range, as it is intended for laboratory ITE use only. Thus, if testers using the assigned testing address ranges are connected to the Internet, and test packets are forwarded across the Internet it is likely that the packets will be discarded and the testing will not work correctly. NOTE I think that there is no logical progression here. It would appear to be perfectly safe to attach the DUT to the Intrnet precisely because the test packets will be discarded and not propagated further. OTOH, the test itself will not work if any packets are discarded and this might happen if those packets are routed over the Internet. END Section 4.2 OLD We note that a range of IPv6 addresses has been assigned to BMWG for laboratory test purposes, in [RFC5180] (as amended by errata). Also, the strong statements in the Security Considerations Section of this memo make the scope even more clear; this is now a standard fixture of all BMWG memos. NOTE I couldn't decide whether it was the Security Considerations Section of this memo that you intended or of 5180. Seems like, given the juxtoposition, that you meant 5180. But surely this is not the right section of this document to mention that material. I suggest moving "Also..." to Section 7 of this document, and rewording it a little. END Section 5 OLD There will be unanticipated difficulties when applying these methods outside the lab environment. NOTE I think you are anticipating the diffuclties in this document. Try s/unanticipated/undesriable/ END Section 5 NOTE s/resource exhaust/resource exhaustion/ END Section 5 OLD Operating test equipment on production networks according to the methods described in [RFC2544], where overload is a possible outcome, would no doubt be harmful to user traffic performance. These tests MUST NOT be used on production networks and as discussed above, the tests will never produce a reliable or accurate benchmarking result on a production network. NOTE As per this paragraph in the Abstract. END Section 6 OLD 6. What to do without RFC 2544? NEW 6. Considering Performance Testing in Production Networks NOTE The scope of "life without 2544" may be a little larger than the purpose of this Section. END Section 6 OLD The world will not spin off axis while waiting for appropriate and standardized methods to emerge from the consensus process. NOTE I sense a certain frustration amongst the authors. I don't think this statement is helpful or respectful to the readers for whom production testing *is* the whole world. Would it not be more constructive to suggest that individuals who find the need for methods and mechanisms to carry out performance testing on produciton networks to be extremely pressing should participate actively in the work of the IPPM working group with a view to helping develop the necessary specifications? END |
2012-08-29
|
05 | Adrian Farrel | [Ballot Position Update] New position, Yes, has been recorded for Adrian Farrel |
2012-08-28
|
05 | Pete McCann | Request for Telechat review by GENART No Response. Reviewer: Pete McCann. |
2012-08-28
|
05 | Stewart Bryant | [Ballot discuss] I have read various version of this text, and I think that the authors are still overstating their case. There are some types … [Ballot discuss] I have read various version of this text, and I think that the authors are still overstating their case. There are some types of production network that use IETF technology and are designed to provide ingress rate policing, guaranteed bandwidth and path isolation. It is not clear why it should be harmful to the network to run these tests over such a network. I would support an RFC that advised operators of the issues, but I am not convinced that the use of the technology is necessarily always harmful to their networks. I think it is important to distinguish between the case where the network is broken for other users, and the case where a test protocol does not work. In the former case the term harmful is applicable, but using the term "harmful" in the essentially "private" case of a test protocol that does not work as expected dilutes the meaning of the IETF keyword "harmful". |
2012-08-28
|
05 | Stewart Bryant | [Ballot Position Update] New position, Discuss, has been recorded for Stewart Bryant |
2012-08-28
|
05 | Wesley Eddy | [Ballot Position Update] New position, No Objection, has been recorded for Wesley Eddy |
2012-08-28
|
05 | Pete Resnick | [Ballot comment] If anything needs an "Updates:" header, it seems to me this document does. Is there a reason you have not put in an … [Ballot comment] If anything needs an "Updates:" header, it seems to me this document does. Is there a reason you have not put in an "Updates: 2544"? (I wish 2544 were standards track. I wish this one were as well. Sailed ships I expect. No action needed; I'll just go weep in my beer.) Section 1: To directly address this situation, the past and present Chairs of the IETF Benchmarking Methodology Working Group (BMWG) have prepared this Applicability Statement for RFC 2544. The mention of the past and present Chairs seems a bit self-aggrandizing. Could you simply say: "This Applicability Statement for RFC 2544 (or even just "This document") directly addresses this situation." |
2012-08-28
|
05 | Pete Resnick | [Ballot Position Update] New position, No Objection, has been recorded for Pete Resnick |
2012-08-27
|
05 | Sean Turner | [Ballot Position Update] New position, No Objection, has been recorded for Sean Turner |
2012-08-27
|
05 | Stephen Farrell | [Ballot comment] - section 1: "unintended specifications" sound quite odd, I think you could usefully re-phrase that. |
2012-08-27
|
05 | Stephen Farrell | [Ballot Position Update] New position, No Objection, has been recorded for Stephen Farrell |
2012-08-23
|
05 | Jean Mahoney | Request for Telechat review by GENART is assigned to Pete McCann |
2012-08-23
|
05 | Jean Mahoney | Request for Telechat review by GENART is assigned to Pete McCann |
2012-08-23
|
05 | Brian Haberman | [Ballot Position Update] New position, No Objection, has been recorded for Brian Haberman |
2012-08-17
|
05 | Martin Stiemerling | [Ballot Position Update] New position, No Objection, has been recorded for Martin Stiemerling |
2012-08-14
|
05 | Ron Bonica | Placed on agenda for telechat - 2012-08-30 |
2012-08-14
|
05 | Ron Bonica | Ballot has been issued |
2012-08-14
|
05 | Ron Bonica | [Ballot Position Update] New position, Yes, has been recorded for Ronald Bonica |
2012-08-14
|
05 | Ron Bonica | Created "Approve" ballot |
2012-08-14
|
05 | Ron Bonica | State changed to IESG Evaluation from Waiting for AD Go-Ahead |
2012-08-11
|
05 | Al Morton | New version available: draft-ietf-bmwg-2544-as-05.txt |
2012-06-26
|
04 | (System) | State changed to Waiting for AD Go-Ahead from In Last Call |
2012-06-22
|
04 | Jean Mahoney | Request for Last Call review by GENART is assigned to Pete McCann |
2012-06-22
|
04 | Jean Mahoney | Request for Last Call review by GENART is assigned to Pete McCann |
2012-06-19
|
04 | Samuel Weiler | Request for Last Call review by SECDIR is assigned to Rob Austein |
2012-06-19
|
04 | Samuel Weiler | Request for Last Call review by SECDIR is assigned to Rob Austein |
2012-06-18
|
04 | Pearl Liang | IANA has reviewed draft-ietf-bmwg-2544-as-04, which is currently in Last Call, and has the following comments: IANA notes that the authors suggest that there are … IANA has reviewed draft-ietf-bmwg-2544-as-04, which is currently in Last Call, and has the following comments: IANA notes that the authors suggest that there are no IANA actions required by this document. However, IANA notes the following text in the document in Section 4.2" "We note that a range of IPv6 addresses has been assigned to BMWG for laboratory test purposes, in [RFC5180]. Also, the strong statements in the Security Considerations Section of this memo make the scope even more clear; this is now a standard fixture of all BMWG memos." and strongly suggests that it should be updated to refer to the erratum noting the actual IPv6 assignment: http://www.rfc-editor.org/errata_search.php?rfc=5180 Other than that request, IANA understands that, upon approval of this document, there are no IANA Actions that need completion. |
2012-06-12
|
04 | Cindy Morgan | The following Last Call announcement was sent out: From: The IESG To: IETF-Announce CC: Reply-To: ietf@ietf.org Subject: Last Call: (RFC 2544 Applicability Statement: Use … The following Last Call announcement was sent out: From: The IESG To: IETF-Announce CC: Reply-To: ietf@ietf.org Subject: Last Call: (RFC 2544 Applicability Statement: Use on Production Networks Considered Harmful) to Informational RFC The IESG has received a request from the Benchmarking Methodology WG (bmwg) to consider the following document: - 'RFC 2544 Applicability Statement: Use on Production Networks Considered Harmful' as Informational RFC The IESG plans to make a decision in the next few weeks, and solicits final comments on this action. Please send substantive comments to the ietf@ietf.org mailing lists by 2012-06-26. Exceptionally, comments may be sent to iesg@ietf.org instead. In either case, please retain the beginning of the Subject line to allow automated sorting. Abstract Benchmarking Methodology Working Group (BMWG) has been developing key performance metrics and laboratory test methods since 1990, and continues this work at present. Recent application of the methods beyond their intended scope is cause for concern. The methods described in RFC 2544, where overload is a possible outcome, would no doubt be harmful to user traffic performance on a production network. This memo clarifies the scope of RFC 2544 and other benchmarking work for the IETF community. The file can be obtained via http://datatracker.ietf.org/doc/draft-ietf-bmwg-2544-as/ IESG discussion can be tracked via http://datatracker.ietf.org/doc/draft-ietf-bmwg-2544-as/ballot/ No IPR declarations have been submitted directly on this I-D. |
2012-06-12
|
04 | Cindy Morgan | State changed to In Last Call from Last Call Requested |
2012-06-12
|
04 | Ron Bonica | Last call was requested |
2012-06-12
|
04 | Ron Bonica | Last call announcement was generated |
2012-06-12
|
04 | Ron Bonica | Ballot approval text was generated |
2012-06-12
|
04 | Ron Bonica | State changed to Last Call Requested from AD Evaluation |
2012-06-12
|
04 | Ron Bonica | State changed to AD Evaluation from Publication Requested |
2012-06-12
|
04 | Al Morton | New version available: draft-ietf-bmwg-2544-as-04.txt |
2012-05-07
|
03 | Al Morton | Changed shepherd to Bill Cerveny |
2012-04-26
|
03 | Ron Bonica | Ballot writeup was changed |
2012-04-26
|
03 | Al Morton | New version available: draft-ietf-bmwg-2544-as-03.txt |
2012-04-25
|
02 | Ron Bonica | Ballot writeup was generated |
2012-04-25
|
02 | Amy Vezza | State Change Notice email list changed to bmwg-chairs@tools.ietf.org, draft-ietf-bmwg-2544-as@tools.ietf.org, wcerveny@wjcerveny.com from bmwg-chairs@tools.ietf.org, draft-ietf-bmwg-2544-as@tools.ietf.org |
2012-04-25
|
02 | Amy Vezza | (1) What type of RFC is being requested (BCP, Proposed Standard, Internet Standard, Informational, Experimental, or Historic)? Why is this the proper type of RFC? … (1) What type of RFC is being requested (BCP, Proposed Standard, Internet Standard, Informational, Experimental, or Historic)? Why is this the proper type of RFC? Is this type of RFC indicated in the title page header? Informational, as indicated on the title page. All BMWG RFCs are traditionally Informational, in part because they do not define protocols and the traditional conditions for standards track advancement did not apply. However, they are specifications and the RFC 2119 terms are applicable to identify the level of requirements. (2) The IESG approval announcement includes a Document Announcement Write-Up. Please provide such a Document Announcement Write-Up. Recent examples can be found in the "Action" announcements for approved documents. The approval announcement contains the following sections: Technical Summary Benchmarking Methodology Working Group (BMWG) has been developing key performance metrics and laboratory test methods since 1990, and continues this work at present. Recent application of the methods beyond their intended scope is cause for concern. This memo clarifies the scope of RFC 2544 and other benchmarking work for the IETF community. Working Group Summary Working group consensus was achieved in a single WGLC, after allowing a year for review and many useful comments and exchanges. Document Quality The single-sentence message of this memo was clear from first publication. Most comments helped to make the wording more concise. Personnel Document Shepherd: Bill Cerveny Area Director: Ronald Bonica (3) Briefly describe the review of this document that was performed by the Document Shepherd. If this version of the document is not ready for publication, please explain why the document is being forwarded to the IESG. The document shepherd has reviewed this document three times. There are no outstanding issues, as far as the document shepherd is concerned. (4) Does the document Shepherd have any concerns about the depth or breadth of the reviews that have been performed? No (5) Do portions of the document need review from a particular or from broader perspective, e.g., security, operational complexity, AAA, DNS, DHCP, XML, or internationalization? If so, describe the review that took place. No (6) Describe any specific concerns or issues that the Document Shepherd has with this document that the Responsible Area Director and/or the IESG should be aware of? For example, perhaps he or she is uncomfortable with certain parts of the document, or has concerns whether there really is a need for it. In any event, if the WG has discussed those issues and has indicated that it still wishes to advance the document, detail those concerns here. There are no specific concerns or issues that the document shepherd has with this document. (7) Has each author confirmed that any and all appropriate IPR disclosures required for full conformance with the provisions of BCP 78 and BCP 79 have already been filed. If not, explain why. Each author has confirmed that there are no known IPR concerns, nor are any IPR disclosures required. (8) Has an IPR disclosure been filed that references this document? If so, summarize any WG discussion and conclusion regarding the IPR disclosures. No (9) How solid is the WG consensus behind this document? Does it represent the strong concurrence of a few individuals, with others being silent, or does the WG as a whole understand and agree with it? Many WG members have reviewed this draft, including long-time participants, new participants, and (most importantly) participants who work at companies most affected by the statements in this memo. WG consensus was called by Area Director Ron Bonica, since the current chair and *all past chairs* are co-authors. (10) Has anyone threatened an appeal or otherwise indicated extreme discontent? If so, please summarise the areas of conflict in separate email messages to the Responsible Area Director. (It should be in a separate email because this questionnaire is publicly available.) No appeals have been threatened. (11) Identify any ID nits the Document Shepherd has found in this document. (See http://www.ietf.org/tools/idnits/ and the Internet-Drafts Checklist). Boilerplate checks are not enough; this check needs to be thorough. ID nits have been checked. There was only one nit identified and this appears to be an error. This one id nit complained about addresses falling within "non-RFC5735-compliant IPv4 addresses" As far as I can tell, the only addresses are in section 4.2 and these are exactly within the range described in RFC5735. The tools team has been notified as this appears to be a bug or error in the nits checking program. (12) Describe how the document meets any required formal review criteria, such as the MIB Doctor, media type, and URI type reviews. Not applicable (13) Have all references within this document been identified as either normative or informative? Yes (14) Are there normative references to documents that are not ready for advancement or are otherwise in an unclear state? If such normative references exist, what is the plan for their completion? No (15) Are there downward normative references references (see RFC 3967)? If so, list these downward references to support the Area Director in the Last Call procedure. No (16) Will publication of this document change the status of any existing RFCs? Are those RFCs listed on the title page header, listed in the abstract, and discussed in the introduction? If the RFCs are not listed in the Abstract and Introduction, explain why, and point to the part of the document where the relationship of this document to the other RFCs is discussed. If this information is not in the document, explain why the WG considers it unnecessary. No (17) Describe the Document Shepherd's review of the IANA considerations section, especially with regard to its consistency with the body of the document. Confirm that all protocol extensions that the document makes are associated with the appropriate reservations in IANA registries. Confirm that any referenced IANA registries have been clearly identified. Confirm that newly created IANA registries include a detailed specification of the initial contents for the registry, that allocations procedures for future registrations are defined, and a reasonable name for the new registry has been suggested (see RFC 5226). There are no IANA considerations. (18) List any new IANA registries that require Expert Review for future allocations. Provide any public guidance that the IESG would find useful in selecting the IANA Experts for these new registries. None (19) Describe reviews and automated checks performed by the Document Shepherd to validate sections of the document written in a formal language, such as XML code, BNF rules, MIB definitions, etc. Not applicable. |
2012-04-25
|
02 | Amy Vezza | Note added 'Document Shepherd: Bill Cerveny ' |
2012-04-25
|
02 | Amy Vezza | Intended Status changed to Informational |
2012-04-25
|
02 | Amy Vezza | IESG process started in state Publication Requested |
2012-04-25
|
02 | (System) | Earlier history may be found in the Comment Log for draft-chairs-bmwg-2544-as |
2012-03-12
|
02 | Al Morton | New version available: draft-ietf-bmwg-2544-as-02.txt |
2011-10-20
|
01 | (System) | New version available: draft-ietf-bmwg-2544-as-01.txt |
2011-08-06
|
00 | (System) | New version available: draft-ietf-bmwg-2544-as-00.txt |