Evaluation of a Sample of RFCs Produced in 2018
RFC 8963

Document Type RFC - Informational (January 2021; Errata)
Author Christian Huitema 
Last updated 2021-01-24
Stream ISE
Formats plain text html xml pdf htmlized with errata bibtex
IETF conflict review conflict-review-huitema-rfc-eval-project
Stream ISE state Published RFC
Consensus Boilerplate Unknown
Document shepherd Adrian Farrel
Shepherd write-up Show (last changed 2020-10-25)
IESG IESG state RFC 8963 (Informational)
Telechat date
Responsible AD (None)
Send notices to Adrian Farrel <rfc-ise@rfc-editor.org>
IANA IANA review state IANA OK - No Actions Needed
IANA action state No IANA Actions

Independent Submission                                        C. Huitema
Request for Comments: 8963                          Private Octopus Inc.
Category: Informational                                     January 2021
ISSN: 2070-1721

            Evaluation of a Sample of RFCs Produced in 2018


   This document presents the author's effort to understand the delays
   involved in publishing an idea in the IETF or through the Independent
   Stream, from the first individual draft to the publication of the
   RFC.  We analyze a set of randomly chosen RFCs approved in 2018,
   looking for history and delays.  We also use two randomly chosen sets
   of RFCs published in 2008 and 1998 for comparing delays seen in 2018
   to those observed 10 or 20 years ago.  The average RFC in the 2018
   sample was produced in 3 years and 4 months, of which 2 years and 10
   months were spent in the working group, 3 to 4 months for IETF
   consensus and IESG review, and 3 to 4 months in RFC production.  The
   main variation in RFC production delays comes from the AUTH48 phase.

   We also measure the number of citations of the chosen RFC using
   Semantic Scholar, and compare citation counts with what we know about
   deployment.  We show that citation counts indicate academic interest,
   but correlate only loosely with deployment or usage of the
   specifications.  Counting web references could complement that.

Status of This Memo

   This document is not an Internet Standards Track specification; it is
   published for informational purposes.

   This is a contribution to the RFC Series, independently of any other
   RFC stream.  The RFC Editor has chosen to publish this document at
   its discretion and makes no statement about its value for
   implementation or deployment.  Documents approved for publication by
   the RFC Editor are not candidates for any level of Internet Standard;
   see Section 2 of RFC 7841.

   Information about the current status of this document, any errata,
   and how to provide feedback on it may be obtained at

Copyright Notice

   Copyright (c) 2021 IETF Trust and the persons identified as the
   document authors.  All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents
   (https://trustee.ietf.org/license-info) in effect on the date of
   publication of this document.  Please review these documents
   carefully, as they describe your rights and restrictions with respect
   to this document.

Table of Contents

   1.  Introduction
   2.  Methodology
     2.1.  Defining the Important Milestones
     2.2.  Selecting a Random Sample of RFCs
     2.3.  Conventions Used in This Document
   3.  Analysis of 20 Selected RFCs
     3.1.  RFC 8411
     3.2.  RFC 8456
     3.3.  RFC 8446
     3.4.  RFC 8355
     3.5.  RFC 8441
     3.6.  RFC 8324
     3.7.  RFC 8377
     3.8.  RFC 8498
     3.9.  RFC 8479
     3.10. RFC 8453
     3.11. RFC 8429
     3.12. RFC 8312
     3.13. RFC 8492
     3.14. RFC 8378
     3.15. RFC 8361
     3.16. RFC 8472
     3.17. RFC 8471
     3.18. RFC 8466
     3.19. RFC 8362
     3.20. RFC 8468
   4.  Analysis of Process and Delays
     4.1.  Delays from First Draft to RFC
     4.2.  Working Group Processing Time
     4.3.  Preparation and Publication Delays
     4.4.  Copy Editing
     4.5.  Independent Stream
   5.  Citation Counts
     5.1.  Citation Numbers
     5.2.  Comparison to 1998 and 2008
     5.3.  Citations versus Deployments
     5.4.  Citations versus Web References
   6.  Observations and Next Steps
   7.  Security Considerations
   8.  IANA Considerations
   9.  Informative References
   Author's Address

1.  Introduction

   As stated on the organization's web site, "The IETF is a large open
   international community of network designers, operators, vendors, and
   researchers concerned with the evolution of the Internet architecture
   and the smooth operation of the Internet."  The specifications
   produced by the IETF are published in the RFC series, along with
   documents from the IAB, IRTF, and Independent streams (as per RFC
   8729).  In this memo, the author attempts to understand the delays
   involved in publishing an idea in the IETF or through the Independent
   Stream, from the first individual draft to the publication of the
   RFC.  This is an individual effort, and the author's conclusions
   presented here are personal.  There was no attempt to seek IETF

   The IETF keeps records of documents and process actions in the IETF
   Datatracker [TRKR].  The IETF Datatracker provides information about
   RFCs and drafts, from which we can infer statistics about the
   production system.  We can measure how long it takes to drive a
   proposition from initial draft to final publication, and how these
   delays can be split between working group discussions, IETF reviews,
   IESG assessment, RFC Editor delays and final reviews by the authors
Show full document text