IETF conflict review for draft-clausen-lln-rpl-experiences
conflict-review-clausen-lln-rpl-experiences-00

The following approval message was sent
From: The IESG
To: draft-clausen-lln-rpl-experiences@ietf.org,
    Nevil Brownlee ,
    Nevil Brownlee ,
    Adrian Farrel ,
    aretana.ietf@gmail.com,
    rfc-ise@rfc-editor.org
Cc: IETF-Announce ,
    The IESG ,
    iana@iana.org
Subject: Results of IETF-conflict review for draft-clausen-lln-rpl-experiences-11

The IESG has completed a review of draft-clausen-lln-rpl-experiences-11
consistent with RFC5742.

The IESG recommends that 'Observations on RPL: IPv6 Routing Protocol for Low
power and Lossy Networks'  NOT be
published as an Informational RFC.

The IESG has concluded that publication could potentially disrupt the IETF
work done in the roll WG and recommends not publishing the document at this
time.

The IESG would also like the Independent Submissions Editor to review the
comments in the datatracker related to this document and determine whether or
not they merit incorporation into the document. Comments may exist in both
the ballot and the history log.

The IESG review is documented at:
https://datatracker.ietf.org/doc/conflict-review-clausen-lln-rpl-experiences/

A URL of the reviewed Internet Draft is:
https://datatracker.ietf.org/doc/draft-clausen-lln-rpl-experiences/

The process for such documents is described at
https://www.rfc-editor.org/indsubs.html

Thank you,

The IESG Secretary

[Ballot comment]
I'm actually not sure about the conclusion here. I agree to all points that Alvaro brought up and I think these are good reasons for the ISE to not publish this document, however, I am not sure if it warrens an IESG reply that concludes "that publication could potentially disrupt the IETF work", especially as the work is already published elsewhere. However, I only had a brief look at the draft itself and can also see that there is a relation to IETF work which could be disruptive!

[Ballot comment]
Agree with Alvaro - without the implementation and deployment details, this document is harmful.

It only mentions in the abstract that it is based on prototype implementations, the rest of the document
references only the RFC, the reader can not distinguish. More than 26 pages of statements on negatives
and limitations without details. The implementation of a protocol and how it is deployed are critical in
understanding an evaluation - either positive or negative.

The independent stream does allow for documents outside the official IETF process that are relevant to
the Internet community and achieve reasonable levels of technical and editorial quality. It is questionable
if this document is outside the official IETF process based on the refusal of the authors to improve the
document after the working group reviewed and questioned the technical quality. As Alvaro noted,
justification to publish based on a prior non-IETF conference publication, should not be the basis for IETF's
independent stream need to publish.

[Ballot discuss]
I'm actually not sure about the conclusion here. I agree to all points that Alvaro brought up and I think these are good reasons for the ISE to not publish this document, however, I am not sure if it warrens an IESG reply that concludes "that publication could potentially disrupt the IETF work", especially as the work is already published elsewhere. However, I only had a brief look at the draft itself and happy to discuss this as I might be wrong!

[Ballot comment]
This document, which "aims at providing a better understanding of possible limits of RPL, notably the possible directions that further protocol developments should explore" was initially published in 2012.  The Abstract further says that it "presents a selection of observations on the protocol characteristics, exposes experiences acquired when producing various prototype implementations of RPL, and presents results obtained from testing this protocol - by way of network simulations, in network testbeds and in deployments."  While discussing it in the roll WG [1], the authors were repeatedly asked to provide details of the tests used (implementations, simulations, deployments).  However, the answer was always along the lines of not needing to do so because "the experiments only served to help us see "what to look for" in the RPL RFC", and, "the issues raised in this I-D are based on what can be observed from the RPL RFC".  As a result the WG lost interest.

Sharing experiences gained through deployment and/or experimentation is very valuable in any protocol development setting -- specially if it leads to improvements.  RPL is a routing protocol that is used in many different scenarios, from IoT-oriented applications (see RFC5826, RFC5548, RFC5867, RFC5673, RFC7733, RFC8036) to autonomic networking [2].  A clear understanding of the details of the implementation, the types of devices used, and the deployment scenarios is important.  Details do matter!

Besides not providing appropriate details, the document has failed to even include pointers to work that may address some of the observations; RFC6997 and draft-ietf-roll-dao-projection are two examples.  In fact, the document has not significantly changed since its initial publication.

Furthermore, the similarities between this document and [rpl-eval] (used as a reference) are evident.  The rpl-eval document has already been published as part of the Proceedings of the 5th IEEE International Conference on Wireless & Mobile Computing, Networking & Communication (WiMob) in 2011.  Re-publishing that document with minor changes and no updates to reflect the current work doesn't seem valuable.

The reasons above have led to the conclusion that publishing this document as an RFC could prove disruptive to the work of the roll WG.  At best, the document lacks complete and up-to-date information.

Ideally, an up-to-date version of this document, with proper details, could serve as a discussion driver in the roll WG.


[1] https://mailarchive.ietf.org/arch/msg/roll/94pPESUq_4beceta0RKx0GREGpQ/
[2] https://tools.ietf.org/html/draft-ietf-anima-autonomic-control-plane
[rpl-eval] https://hal.inria.fr/inria-00597036/document

=== Review of draft-clausen-lln-rpl-experiences-11 ===

From: Alvaro Retana
To: draft-clausen-lln-rpl-experiences@ietf.org
cc: Nevil Brownlee, RFC ISE (Adrian Farrel)
Date: May 25, 2018 at 1:52 PM

Dear authors:

As you know, the IESG has been asked to do a Conflict Review [rfc5742] for this document -- I am the Shepherding AD, and just finished reading it.  I have some comments on the text (see below), mostly about clarity and accuracy.  I will post the result of the Conflict Review (most likely) next week for IESG Evaluation.

I also have some general comments/questions:

[I am aware of the 2012 discussion about this document in the roll WG [1] -- I know that some of the points below where discussed then.  Because time has passed, and I'm freshly looking at the this document, I'm asking again.  You may still have the same position -- if so, then I don't think there's a need to rehash the discussion here.]

(1) This document was first published as a draft in 2012.  While there have been some changes/updates since then, it seems to me that most of them were made 4 or more years ago.  Are there any plans to update the document before publication?  I ask in part because there is active work related to the observations in the roll WG.

(2) The other reason behind my question above is that the results presented correspond to work done at least 7 years ago (as the work cited was published in 2011).  I wonder whether the implementations, tests and/or understanding of the deployments may have changed to impact the observations (one way or another).

For example, one of the references ([bidir]) resulted in the observation that "considerable control traffic overhead" exists in RPL (§5.1).  However, another study [2] published in 2014 concluded the opposite: "Unlike previous studies, our results show that RPL provides shorter delays, less control overhead, and requires less memory...", and also says (specifically about the bidir-reference work):

    "The  paper shows a significantly larger control overhead of RPL caused by the maintenance of downward routes. ... As the RPL RFC does not specify the period or the mechanism to use for maintaining downward routes, the study assumed an interval of 15 seconds. This choice is questionable, because it is the main cause of the high control overhead of RPL."

Note that I am not claiming that any one if these studies is better/correct, just showing how different tests may result in different observations, while they both mention the non-specificity of the RPL RFC.  To this point, one of my comments below is precisely about the lack of specific information in the document itself related to the implementations, test setups, etc.

(3) I find the similarities between this document and one of the published references ([rpl-eval]) striking, and wonder whether there is value in republishing that document (with some minor changes).  I leave to the ISE the exercise of comparing the two documents.


Note that the roll WG Chairs have also recently asked for feedback on this document [3].

Thanks!

Alvaro.

[1] https://mailarchive.ietf.org/arch/msg/roll/94pPESUq_4beceta0RKx0GREGpQ/
[2] https://arxiv.org/pdf/1401.0997.pdf
[3] https://mailarchive.ietf.org/arch/msg/roll/tj1VkWdl1cwIVrBC9fyf8UFE3Pw/





[I used the output of id-nits, with line numbers, for my comments.]


...
18 Abstract

20  With RPL - the "IPv6 Routing Protocol for Low-power Lossy Networks" -
21  published as a Proposed Standard after a ~2-year development cycle,
22  this document presents an observation of the resulting protocol, of
23  its applicability, and of its limits.  The documents presents a
24  selection of observations on the protocol characteristics, exposes
25  experiences acquired when producing various prototype implementations
26  of RPL, and presents results obtained from testing this protocol - by
27  way of network simulations, in network testbeds and in deployments.
28  The document aims at providing a better understanding of possible
29  limits of RPL, notably the possible directions that further protocol
30  developments should explore, in order to address these.

"various prototype implementations...network simulations, in network testbeds and in deployment": this document would benefit from an Implementation Status Section (rfc7942), or a description of the simulations/testbeds/deployments.


...
99 1.  Introduction

101  RPL - the "Routing Protocol for Low Power and Lossy Networks"
102  [RFC6550] - is a proposal for an IPv6 routing protocol for Low-power
103  Lossy Networks (LLNs), by the ROLL Working Group in the Internet
104  Engineering Task Force (IETF).  This routing protocol is intended to
105  be the IPv6 routing protocol for LLNs and sensor networks, applicable
106  in all kinds of deployments and applications of LLNs.

108  The objective of RPL and ROLL at the time of development and
109  publication of [RFC6550] in 2012 is to provide routing in networks
110  which "comprise up to thousands of nodes" [roll-charter], where the
111  majority of the nodes have very constrained resources [RFC7102], and
112  where handling mobility is not an explicit design criteria [RFC5867],
113  [RFC5826], [RFC5673], [RFC5548].

The actual quote from the -03 version of the charter is: "...potentially comprising a very large number (several thousands) of nodes."

Along the same lines of being accurate in the descriptions...  I found the reference to rfc7102 interesting in that it is illustrating the "objective of RPL and ROLL at the time of development and publication of [RFC6550]"...but it was published 2 years after rfc6550.  That difference in publication time shouldn't matter much as rfc7102 is titled "Terms Used in Routing for Low-Power and Lossy Networks".  However, rfc7102 doesn't define anything similar to "very constrained resources" -- the word "constrained", which I think is the important word here, appears only twice, both within the definition of "field device": "A field device is usually (but not always) a device with constrained CPU, memory footprint, storage capacity, bandwidth, and sometimes power (battery operated).  ... Although continuous improvement of hardware and software technologies is expected, such devices will likely continue to be seen as resource-constrained devices compared to computers and routers used in the rest of the Internet."  This definition is in fact in line with what the charter at the time said: "Low power and Lossy networks (LLNs) are made up of many embedded devices with limited power, memory, and processing resources."

The comment above is simply intended at pointing at the fact that not all the quotes and references are accurate, and that may confuse some readers.

RFC5867, RFC5826 and RFC5673 do have explicit sections on mobility.


115  [roll-charter] in 2012 states that "Typical traffic patterns are not
116  simply unicast flows (e.g. in some cases most if not all traffic can
117  be point to multipoint)".  [RFC7102] further categorizes the
118  supported traffic types into "upward" traffic from sensors to an
119  actuator or LBR (LLN Border Router) (denoted multipoint-to-point),
120  "downward" traffic from the actuator or LBR to the sensors (denoted
121  point-to-multipoint) and traffic from "sensor to sensor" (denoted
122  point-to-point traffic), and establishes this terminology for these
123  traffic types.  Thus, while the target for RPL and ROLL is to support
124  all of these traffic types, the emphasis among these, according to
125  [roll-charter], appears to be to optimize for multipoint-to-point
126  traffic, while also supporting point-to-multipoint and point-to-point
127  traffic.

True.  However, the charter also says that the "application-specific routing requirement documents will be used for protocol design".  Those documents provide some (not a lot) additional information.


...
430 5.1.  Observations

432  RPL is well suited for networks in which the sink for data traffic is
433  co-located with, (or is outside the LLN and reachable via), the DODAG
434  root.  However, these data traffic characteristics does not represent
435  a universal distribution of traffic types in LLNs.  There are
436  scenarios where the sink is not co-located with (or is outside the
437  LLN and reachable via) the DODAG.  These include:

...the DODAG root.

439  o  Command/control networks in which point-to-point traffic is a more
440      common occurrence, documented, e.g., in [RFC5867] ("Building
441      Automation Routing Requirements in Low Power and Lossy Networks").

In §4 (Traffic Pattern), rfc5867 reads: "Thirty percent of each node's packets are destined for other nodes within the LLN.  Seventy percent of each node's packets are destined for an aggregation device (multipoint to point (MP2P)) and routed off the LLN."


...
482  For scenarios with bi-directional traffic, as there is no provision
483  for on-demand generation of routing information from the DODAG Root
484  to a proper subset of all routers, each router (besides the Root) is
485  required to generate DAOs.  In particular in non-storing mode, each
486  router will unicast a DAO to the DODAG Root (whereas in storing mode,
487  the DAOs propagate upwards towards the Root).  The effects of the
488  requirement to establish downward routes to all routers are:

Take a look at draft-ietf-roll-dao-projection (Root initiated routing state in RPL).


...
642 7.  The DAO Mechanism: Downward and Point-to-Point Routes

644  RPL specifies two distinct and incompatible "modes of operation" for
645  downward traffic: 1) storing mode: each router is assumed to maintain
646  routes to all destinations in its sub-DODAG, i.e., routers that are
647  "deeper down" in the DODAG, 2) non-storing mode: only the DODAG Root
648  stores routes to destinations inside the LLN.  The DODAG Root employs
649  strict source routing in order to route data traffic to the
650  destination router.

draft-ietf-roll-dao-projection also enables the ability to use loose source routing.


...
755                                  .---.
756                                  |  |
757                                  '---'
758                                    |
759                                    |
760                                    (a)
761                                    |
762                                    |1.x.x.x/8
763                                    |
764                                    (b)
765                                    / \
766                        1.1.x.x/16/  \ 1.2.x.x/16
767                                  /    \
768                              .---.    .---.
769                              | c |    | d |
770                              '---'    '---'

772                      Figure 2: Address Hierarchies

It would have been nice for the figure to use IPv6 addresses...


...
803  A slightly less strict hierarchy can be envisioned, where a router
804  can change its preferred parent without necessarily changing
805  addresses of itself and of its sub-DODAG, provided that its former
806  and new preferred parents both have the same preferred parent, and
807  have addresses hierarchically assigned from that - from the
808  "preferred grandparent".  With reference to Figure 1, this could be e
809  changing its preferred parent from d to c, provided that both d and c
810  have b as preferred parent.  Doing so would impose a restriction on

...that seems like a bad example since, in Figure 1, d and c don't both have b as a parent.  [rpl-eval] does have the correct Figure 1 (with b as the common parent for d and c)...


...
842 9.1.  Observations

844  Unidirectional links are no rare occurrence, such as is known from
845  wireless multi-hop networks.  Preliminary results from a test-bed of
846  AMI (Automated Metering Infrastructure) devices using 950MHz radio
847  interfaces, and with a total of 22 links, show that 36% of these
848  links are unidirectional.  If a router receives a DIO on such a

No reference or more details?

849  unidirectional link, and selects the originator of the DIO as parent,
850  which would be a bad choice: unicast traffic in the upward direction
851  would be lost.  If the router had verified the bidirectionality of
852  links, it might have selected a better parent, to which it has a
853  bidirectional link.

855  [RFC6550] discusses some mechanisms which can (if deemed needed) be
856  used to verify that a link is bidirectional before choosing a router
857  as a parent.  While requiring one mechanism for bidirectional
858  verification to be used, the document does not specify which method
859  to be used, and how to be used.  The mechanisms discussed include NUD

From rfc6550:

  A node SHOULD verify that bidirectional connectivity and adequate
  link quality is available with a candidate neighbor before it
  considers that candidate as a DODAG parent.

"SHOULD" is a lot stronger than "can (if deemed needed)".

  RPL operations require bidirectional links.  In some LLN scenarios,
  those links may exhibit asymmetric properties.  It is required that
  the reachability of a router be verified before the router can be
  used as a parent.  RPL expects an external mechanism to be triggered
  during the parent selection phase in order to verify link properties
  and neighbor reachability.  Neighbor Unreachability Detection (NUD)
  is such a mechanism, but alternates are possible, including...

Yes, rfc2119 language is not used, but I think the intent of using NUD is clear.

860  [RFC4861], BFD [RFC5881] and [RFC5184].  BFD is explicitly called out
861  as "often not desirable" as it uses a proactive approach (exchange of
862  periodic HELLO messages), which would "lead to excessive control
863  traffic".  Furthermore, not all L2 protocols provide L2

True...however, that text comes from §13 (Maintenance of Routing Adjacency), and not from the discussion about bidirectional verification when selecting parents (§8.4 DODAG Selection).  To be fair, §1.1 (Design Principles) does say that "a detection mechanism that is reactive to traffic is favored in order to minimize the cost of monitoring links that are not being used."


...
951  Finally, upon having been notified by NUD that the "next hop" is
952  unreachable, a router must discard the preferred parent and select
953  another - hoping that this time, the preferred parent is actually
954  reachable.  Also, if NUD indicates "no forward progress" based on an
955  upper-layer protocol, there is no guarantee that the problem stems
956  exclusively from the preferred parent being unreachable.  Indeed, it
957  may be a problem further ahead, possibly outside the LLN, thus
958  changing preferred parent will not alleviate the situation.
959  Moreover, using information from an upper-layer protocol, e.g., to
960  return TCP ACKs back to the source, requires established downward
961  routes in the DODAG (i.e., each router needs to send DAO messages to
962  the DODAG Root, as described in Section 7).

I don't think that rfc7048 necessarily mitigates your observations, but it might be worth looking at.


...
990 11.1.  Observations

992  Since RPL is intended as the routing protocol for LLNs, which covers
993  all the diverse applications requirements listed in [RFC5867],
994  [RFC5673], [RFC5826], [RFC5548], it is likely that (i) due to limited
995  memory capacity of the routers, and (ii) due to expensive development
996  cost of the routing protocol implementation, RPL implementations will
997  only support a partial set of features from the specification,
998  leading to non-interoperable implementations.

I believe that rfc6550 attempts to address the point of partial implementations by providing some guidance in §16 (Summary of Requirements for Interoperable Implementations).


...
1039  2.  RPL does not specify to use jitter [RFC5148] (i.e., small random
1040      delay for message transmissions).  If DAOs are sent periodically,
1041      adjacent routers may transmit DAO messages at the same time,
1042      leading to link layer collisions.

The reference to rfc5148 looks gratuitous to me since it seems to only help in defining jitter, which is defined in the text as well.


...
1127 14.  Loops

1129  [RFC6550] states that it "guarantees neither loop free route
1130  selection nor tight delay convergence times, but can detect and
1131  repair a loop as soon as it is used.  RPL uses this loop detection to
1132  ensure that packets make forward progress [...] and trigger repairs
1133  when necessary".  This implies that a loop may only then be detected
1134  and fixed when data traffic is sent through the network.

The exact quote from rfc6550 (§3.7) is: "guarantees neither loop-free path selection nor tight delay convergence times, but it can detect and repair a loop as soon as it is used.  RPL uses this loop detection to ensure that packets make forward progress within the DODAG Version and trigger repairs when necessary."

...
1213 15.  Security Considerations

1215  This document does currently not specify any security considerations.
1216  This document also does not provide any evaluation of the security
1217  mechanisms of RPL.

Given recent IoT-related security incidents, it would have been nice/useful for this document to address security in RPL...

...
1229  Moreover, the authors would like to express their gratitude to Ralph
1230  Droms (Cisco) for his careful review of various versions of this
1231  document, for many long discussions, and for his considerable
1232  contributions to both the content and the quality of this document.

AFAIK, Ralph is no longer at cisco.

1234 18.  Informative References

I think several of the references should be treated as Normative ("documents that must be read to understand or implement the technology in the new RFC" - that's from an IESG Statement, which I know doesn't apply to the ISE, but that you may still want to consider)...at least rfc6550.