Advanced Unidirectional Route Assessment (AURA)
draft-ietf-ippm-route-10

[Ballot comment]
Thank you for addressing my DISCUSS.


Questions / Comments:
1: "Although primarily intended for hosts communicating on the Internet
  with IP, the definitions and metrics are constructed to be applicable
  to other network domains, if desired."
This above says "with IP", but then the rest implies it works with other things too - I suggest dropping the "with IP" part, unless this does work with other protocols?

2: "  Also, to specify a framework for active methods of measurement which
  use the techniques described in [PT] at a minimum, and a framework
  for hybrid active-passive methods of measurement, such as the Hybrid
  Type I method [RFC7799] described in
  [I-D.ietf-ippm-ioam-data](intended only for single administrative
  domains), which do not rely on ICMP and provide a protocol for
  explicit interrogation of nodes on a path."
This is a really long / hard to read sentence - can you split it into multiple? I had to read it multiple time and am still having a hard time with the nested clauses. Perhaps:
"This also specifies a framework for active methods of measurement which uses the techniques described in [PT], as well as a framework for hybrid active-passive methods of measurement(such as the Hybrid Type I method [RFC7799])."

3: " Node Identity  The unique address for Nodes communicating within the
      network domain.  For Nodes communicating on the Internet with IP,
      it is the globally routable IP address(es) which the Node uses
      when communicating with other Nodes under normal or error
      conditions. "
I'm not sure I understand -- this says the "unique address" and also "IP address(es)" - it cannot be both singular/unique and multiple. This definition needs clarification or a pointer to later in the document.

4: "Cooperating Node  Nodes that MUST respond to direct queries for their
      Node identity as part of a previously agreed and established
      interrogation protocol. "
I don't think that the "MUST" works here -- I think s/MUST/do/ (or s/MUST//).

5: "  Hop  A Hop MUST contain a Node Identity, and MAY contain arrival and/
      or departure interface identification, round trip delay, and an
      arrival timestamp." - this definition also doesn't really seem to define something, and I think needs to be rewritten or dropped.

6: "Routing Class  A route that treats equally a class C of different  types of packets (unrelated to address classes of the past)".
I think that it is really unfortunate that the term "class C" was chosen for this - I understand that there is a clarification, and don't really expect you to change it, but it needlessly makes the document harder to read for people who naturally associate Class A, B, C, D with classful addresses. I recognize that this terminology comes from RFC2330, but it is till confusing. An additional clarification that this term is being used because of history would probably help. 

7: "Next define a *singleton* definition for a Hop on the path, with sufficient indexes to identify all Hops identified in a measurement interval."
I'm not sure that "singleton" is a well understood term outside of software engineering, and that you need to better explain it (esp because you have felt it important enough to emphasize). Perhaps "data structure to uniquely identify a hop" or similar?

8: "  o  If a pair of discovered Nodes identify two different addresses,
      then they will appear to be different Nodes.

  o  If a pair of discovered Nodes identify two different IP addresses,
      and the IP addresses resolve to the same Node name (in the DNS),
      then they will appear to be the same Nodes."
Please either include "IP" in both or neither of the above, otherwise it sounds like it is a differentiating factor.

9: " UDP and TCP are used when a particular characteristic needs to
  be verified, such as filtering or traffic shaping on specific ports
  (i.e., services). "
TCP traceroute is also used when ICMP responses are not received - while "such as filtering" *could* cover this, it would require squinting...

10: "Furthermore, either Paris-traceroute or
  Scamper is capable of unveiling the many available paths between a
  source and destination (which are visible to this method)."
Visible to *which* method? The one described in this doc? The methods implemented in Paris-traceroute/Scamper?




Nits:

1: "1.1.  Issues with Earlier Work to define Route"
It seems that this is missing a word after "Route" - perhaps metric? Also, I understand the Title Case, but please either drop it, or uppercase D in define.

2: "Paris-traceroute allows its users to measure RTD in every hop of the
  path for a particular flow."
s/measure RTD/measure the RTD/

3: You seem to be using multiple ways of adding emphasis (*something*, --from the Observation Point --, etc). This is confusing / doesn't follow existing style.

[Ballot comment]
Hi,

Thank you for this document, I found it mostly straight forward to read, but I find the definition in section 3.2 slightly confusing.

3.2.  Parameters

  This section lists the REQUIRED input factors to specify a Route
  metric.

...

Possibly I'm slightly confused by what a Route metric is (it is outside of domain of knowledge) but I was surprised by some of the input parameters (e.g. "i", "T", "Ta").  This is probably because I initially read this sentence as defining what parameters would need to be specified to enable an program to perform a route metric calculation across the network (e.g., like the set of options that might be provided to traceroute).  However, I suspect a different meaning is intended, perhaps that these are the mandatory parameters that must be considered in a route metric calculation?  Perhaps some tweaking of this text might help clarify the intent?

Regards,
Rob

[Ballot comment]
Thank you for the work put into this document.

I support many of the previous COMMENT / DISCUSS points raised by Warren and Erik. I will avoid repeating other ADs' points.

Please find below a couple of non-blocking COMMENTs (and I would appreciate a reply to each of my COMMENTs).

I hope that this helps to improve the document,

Regards,

-éric

== COMMENTS ==

-- Section 1.1 --
"path detection methods like [PT] rely on TTL limits", please add the name rather than a reference to [PT] (I assume it is Paris Traceroute). Also, I suggest to s/TTL/hop limit/ (or use the same verbiage as for "cloud subpath".

Please fix "that decrement TTL or Hop Count" into "hop limit" ;-)

-- Section 3.5 --
About bullet 3,
  "If a discovered Node always replies using the same network
      address, regardless of the interface a packet arrives on, then
      multiple parallel links cannot be detected in that network domain."
If RFC 5837 is implemented, then in this case parallel links can still be detected based on the discovery mechanism or did I miss anything ?

-- Section 4.1 --
"[SCAMPER] supports IPv6 traceroute measurements,
  keeping the FlowLabel constant in all packets."
All packets of the same flow or among all flows ? (I guess from the same measurement but probably worth adding more description). The way the sentence is worded sounds like 'only scamper can do IPv6 traceroutes.

As noted in other IESG reviews, I would appreciate clarification / fixes in
"it is sufficient to be routed
  identically if the IP source and destination addresses and the
  FlowLabel are constant" and the following list
 
About IPv6, I fear that the presence of some extension headers may prevent the hashing in ECMP/UCMP. Should there be some text around this issue ?

Other reviews have raise a DISCUSS about "The IP identification field." that does not exist in IPv6, so, I won't raise it again.

-- Section 6 --
This section is written by using only IPv4 vocabulary while other sections are also mentioning IPv6 concepts.

== NITS ==

https://tools.ietf.org/idnits?url=https://tools.ietf.org/id/draft-ietf-ippm-route-09.txt points to a couple of downrefs.

ECMP and UCMP are expanded multiple times.

[Ballot comment]
Section 3's title made me expect a glossary, but it also lays out actual normative requirements.  This seems odd.  Personal preference, but I suggest separating them out.  I note that Warren also tripped over this.

Also in Section 3, I tripped over "class C" as Warren did.  But I think what you're doing is trying to introduce a label "C" to define the class of packet types in this context, so maybe:

OLD:

    A route that treats equally a class C of different ...

NEW:

    A route that treats equally a class, referred to here as "C", of different ...

Then you can just refer to "C" later and drop the confusing references to "class C".

Section 3.1 starts with something that isn't a sentence.  I suggest making it one.

Section 4.1 uses SHOULD in a number of places that leave me wondering, "Or else what?"  You're presenting the implementer with a choice here, but it doesn't feel like these choices are fully developed.  The same occurs in 4.3.  Are nodes not conforming to these SHOULDs malfunctioning, misconfigured, or simply opting out?

[Ballot discuss]
I have 2 DISCUSS points. I think that both should be easy to address...

1: "While most of the routers perform load balancing on flows using Equal Cost Multiple Path (ECMP), a few still divide the workload through packet-based techniques.  The former scenario is defined according to [RFC2991], while the latter generates a round-robin scheme to deliver every new outgoing packet.  ECMP uses a hashing function to ensure that every packet of a flow is delivered by the same path, and this avoids increasing the packet delay variation and possibly producing overwhelming packet reordering in TCP flows."

Round-robin / "per packet" is a form ECMP, and RFC2991 doesn't "define" just the former, it explains both, and recommends the flow based /  hashed approach. RFC 2991 describes a number of approaches (e.g Modulo-N Hash, Hash-Threshold, Highest Random Weight (HRW)) and makes some good suggestions, but the text as written isn't correct.

2: "In IPv6, it is sufficient to be routed identically if the IP source and destination addresses and the FlowLabel are constant, see [RFC6437]."
Many routers currently ignore the FlowLabel and use other header info, like port-numbers. The sentence might(?) be true in an idealized network, but in the real world isn't.
Some additional text should solve this...

[Ballot comment]
Questions / Comments:
1: "Although primarily intended for hosts communicating on the Internet
  with IP, the definitions and metrics are constructed to be applicable
  to other network domains, if desired."
This above says "with IP", but then the rest implies it works with other things too - I suggest dropping the "with IP" part, unless this does work with other protocols?

2: "  Also, to specify a framework for active methods of measurement which
  use the techniques described in [PT] at a minimum, and a framework
  for hybrid active-passive methods of measurement, such as the Hybrid
  Type I method [RFC7799] described in
  [I-D.ietf-ippm-ioam-data](intended only for single administrative
  domains), which do not rely on ICMP and provide a protocol for
  explicit interrogation of nodes on a path."
This is a really long / hard to read sentence - can you split it into multiple? I had to read it multiple time and am still having a hard time with the nested clauses. Perhaps:
"This also specifies a framework for active methods of measurement which uses the techniques described in [PT], as well as a framework for hybrid active-passive methods of measurement(such as the Hybrid Type I method [RFC7799])."

3: " Node Identity  The unique address for Nodes communicating within the
      network domain.  For Nodes communicating on the Internet with IP,
      it is the globally routable IP address(es) which the Node uses
      when communicating with other Nodes under normal or error
      conditions. "
I'm not sure I understand -- this says the "unique address" and also "IP address(es)" - it cannot be both singular/unique and multiple. This definition needs clarification or a pointer to later in the document.

4: "Cooperating Node  Nodes that MUST respond to direct queries for their
      Node identity as part of a previously agreed and established
      interrogation protocol. "
I don't think that the "MUST" works here -- I think s/MUST/do/ (or s/MUST//).

5: "  Hop  A Hop MUST contain a Node Identity, and MAY contain arrival and/
      or departure interface identification, round trip delay, and an
      arrival timestamp." - this definition also doesn't really seem to define something, and I think needs to be rewritten or dropped.

6: "Routing Class  A route that treats equally a class C of different  types of packets (unrelated to address classes of the past)".
I think that it is really unfortunate that the term "class C" was chosen for this - I understand that there is a clarification, and don't really expect you to change it, but it needlessly makes the document harder to read for people who naturally associate Class A, B, C, D with classful addresses. I recognize that this terminology comes from RFC2330, but it is till confusing. An additional clarification that this term is being used because of history would probably help. 

7: "Next define a *singleton* definition for a Hop on the path, with sufficient indexes to identify all Hops identified in a measurement interval."
I'm not sure that "singleton" is a well understood term outside of software engineering, and that you need to better explain it (esp because you have felt it important enough to emphasize). Perhaps "data structure to uniquely identify a hop" or similar?

8: "  o  If a pair of discovered Nodes identify two different addresses,
      then they will appear to be different Nodes.

  o  If a pair of discovered Nodes identify two different IP addresses,
      and the IP addresses resolve to the same Node name (in the DNS),
      then they will appear to be the same Nodes."
Please either include "IP" in both or neither of the above, otherwise it sounds like it is a differentiating factor.

9: " UDP and TCP are used when a particular characteristic needs to
  be verified, such as filtering or traffic shaping on specific ports
  (i.e., services). "
TCP traceroute is also used when ICMP responses are not received - while "such as filtering" *could* cover this, it would require squinting...

10: "Furthermore, either Paris-traceroute or
  Scamper is capable of unveiling the many available paths between a
  source and destination (which are visible to this method)."
Visible to *which* method? The one described in this doc? The methods implemented in Paris-traceroute/Scamper?




Nits:

1: "1.1.  Issues with Earlier Work to define Route"
It seems that this is missing a word after "Route" - perhaps metric? Also, I understand the Title Case, but please either drop it, or uppercase D in define.

2: "Paris-traceroute allows its users to measure RTD in every hop of the
  path for a particular flow."
s/measure RTD/measure the RTD/

3: You seem to be using multiple ways of adding emphasis (*something*, --from the Observation Point --, etc). This is confusing / doesn't follow existing style.

[Ballot discuss]
[ section 4.1 ]

* I don't quite understand this apparent reliance on IPv6 flows being routed or
  load-balanced by src/dst and flow label alone.  I know of implementations
  where that is not the case at all, and in fact RFC 6437 specifically advises
  that the flow label alone not be relied upon.  Quoting section 2:

  o  Forwarding nodes such as routers and load distributors MUST NOT
    depend only on Flow Label values being uniformly distributed.  In
    any usage such as a hash key for load distribution, the Flow Label
    bits MUST be combined at least with bits from other sources within
    the packet, so as to produce a constant hash value for each flow
    and a suitable distribution of hash values across flows.
    Typically, the other fields used will be some or all components of
    the usual 5-tuple.  In this way, load distribution will still
    occur even if the Flow Label values are poorly distributed.

  Perhaps I'm seriously misunderstanding something though...

* In the bulleted list following the final paragraph, the IP identification
  field is an IPv4-only header field.  What is the recommended mechanism for
  IPv6?

[Ballot comment]
[[ comments ]]

Overall, a very well written document, I thought; thank you.

[ section 3.3 ]

* Are N and Nmax actually determined at dst or are they determined by
  src when replies are received?

[ section 3.5 ]

* s/can't be detected at IP layer/can't be detected at the IP layer/

[ section 6 ]

* It looks like "parameter E" appears first in paragraph 5?  It might be good
  to provide some "E means exhaustive..." explanatory sentence.

[Ballot comment]
Section 1

  test.  However, metrics for assessment of path components and related
  performance aspects had not been attempted in IPPM when the [RFC2330]
  framework was written.

nit(?): this seems to be the only place in the document that uses the
phrase "path component", and it was a bit confusing to me.  That is, I'm
used to seeing "path component" being (e.g.) one hop or link in a
multi-link path from source to destination, but the context here
suggests that it might be referring to something more like a single
route within a route ensemble.  I don't have a specific proposed change;
I just wanted to mention that I puzzled over this wording for a bit.

Section 2

  Also, to specify a framework for active methods of measurement which
  use the techniques described in [PT] at a minimum, and a framework
  for hybrid active-passive methods of measurement, such as the Hybrid
  Type I method [RFC7799] described in
  [I-D.ietf-ippm-ioam-data](intended only for single administrative
  domains), which do not rely on ICMP and provide a protocol for
  explicit interrogation of nodes on a path.  [...]

nit: could you remind us that the "Also" refers to the scope of this
work (the previous paragraph started with "The scope is" but there was a
fair bit of intervening stuff between there and here).

  Hop  A Hop MUST contain a Node Identity, and MAY contain arrival and/
      or departure interface identification, round trip delay, and an
      arrival timestamp.

In my head a "hop" is the operation of going from node A to node B,
i.e., the journey and not the destination.  So if we're associating a
node identity with the hop, would we need to say if that's the source or
destination node of the "hop" (or am I just misunderstanding things)?

Section 3.3

  Nmax, the largest value of i needed for a packet to be received at
  Dst (sent between T0 and Tf).  Nmax may be equal to N.

I'm not really sure I understand how Nmax works.  That is, for any given
packet, it has a value for 'i', and that packet may or may not be
delivered.  But with 'i' defined as the limit on the number of hops for
"a specific packet may visit", I don't see how we get to do multiple
tries for the same packet with different 'i', as we would need to do in
order to determine the "largest value of i *needed*" (emphasis mine).
That is, we can set 'i' unreasonably large and that would result in
delivery, but that does not (as I understand it) meet the definition for
Nmax -- Nmax is supposed to be a boundary value for some condition.  I'm
just not entirely clear on exactly what that boundary is, since we're
talking about a specific packet, and my understanding is that we only
get one crack at a specific packet.

  o  t(i,j) Arrival Timestamp (where t(i,j) is ideally supplied by the
      Hop, or approximated from the sending time of the packet that
      revealed the Hop)

I'm getting a bit of deja vu as I write this, but it seems risky to
propose using the term "arrival timestamp" both for something generated
at the Hop and something generated by the sender of the packet, since
that loses information about the accuracy of the value for a given use
case.  Perhaps it's best to reserve the term for the ideal quantity and
leave approximations for the processing pipeline (as opposed to the
measurement pipeline).

Section 3.4

  RTL(i,j,T) is defined as a singleton of the [RFC6673] Round-trip Loss
  between the Node with address = Src and the Node at Hop h(i,j) at
  time T.

Just to check my understanding: does the "singleton" and "at time T"
force us into a logical (boolean) loss metric for this case, as opposed
to a sample-ratio-type loss metric?

Section 3.6

  identities at a minimum.  The models need to be expanded to include
  these features, as well as Arrival Interface ID, Departure Interface
  ID, and Arrival Timestamp, when available.  [...]

Is this expected to occur as future work?  (Is there an I-D worth
referencing?)

Section 4.1

  Internet routing is complex because it depends on the policies of
  thousands of Autonomous Systems (AS).  While most of the routers
  perform load balancing on flows using Equal Cost Multiple Path
  (ECMP), a few still divide the workload through packet-based
  techniques.  The former scenario is defined according to [RFC2991],
  while the latter generates a round-robin scheme to deliver every new
  outgoing packet.  [...]

I was surprised to see "packet-based techniques" refer to a round-robin
scheme for queuing outgoing packets across interfaces, but assume this
just means I need to do more background reading.  Any tips for where to
start?

  Formally, to maintain the same flow in the measurements to a
  particular hop, the Type-P-Route-Ensemble-Method-Variant packets
  should be[PT]:

It's not entirely clear that we need a reference for these attributes;
they seem to be fairly easy to verify independently, IIUC.

  o  TCP case: For IPv4, the fields Src, Dst, port-Src, port_Dst,
      sequence number, and Diffserv Field SHOULD be the same.  For IPv6,
      the field FlowLabel, Src and Dst SHOULD be the same.

Are we happy to pair "to maintain the same flow" (previous paragraph)
with "SHOULD be the same" (here, and subsequent bullet points)?

Section 4.1

  1.  SHOULD occasionally check path portions which have exhibited
      stable results over time, particularly ingress and egress
      portions of the path.

Do we want to give any (e.g., order of magnitude) guidance for
"occasionally"?

Section 4.2

  In addition to node identity, nodes may also identify the ingress and
  egress interfaces utilized by the tracing packet, the time of day
  when the packet was processed, and other generic data (as described

nit: I'd consider "absolute time" over "time of day", since the latter
may have connotations of implying that there is 24-hour-periodic
behavior.  (It may also be subject to local time zone on the node
inserting the value.)

Section 4.3

  1.  Nodes at the ingress to the domain SHOULD be both Discoverable
      and Cooperating, and SHOULD reveal the same Node Identity in
      response to both active and hybrid methods.

The second clause here seems redundant with point (2) that follows.
(Similarly for point (3)'s similar clause.)  That is, the recommendation
to use the same identity for both methods can only ever apply when the
node in question is both Discoverable and Cooperating -- if it's only
one, then there will only be one identity available, and no need for
consistency across methods.

  When Nodes follow these requirements, it becomes a simple matter to

Perhaps a philosophical question, but are "SHOULD"-level directives more
of "requirements" or "guidelines"?

Section 5

  traffic load is not stationary.  Nonetheless, as the first approach,
  a link seems to be congested if, after sending several traceroute
  probes, it is possible to detect congestion observing different
  statistics parameters (e.g., see [IDCong]).  Finally, to recognize

nit: please check the wording of this sentence; I think there may be a
missing word or similar.

Section 6

  every hop.  This procedure could be done for a single Member Route
  flow, with parameter E set as False, or for every detected Route
  Ensemble flow (E=True).

nit: I don't think we define 'E' until the full procedure, a few
paragraphs below.

  made.  Line 8 and 13 set a timer for each cycle of measurements.  A
  cycle comprises the traceroutes packets, considering every possible
  Hop and the alternatives paths in the Alt variable (ensured in lines
  9-12).  [...]

I recognize that this is pseudocode, but "what if the
advanced-traceroute() operation takes longer than the i_t time?"

Section 7

We could consider some bland statement about the measurement
technologies (e.g., ICMP) not providing cryptographic protection for the
requested/returned data, and the risks of processing untrusted data in
general.  This is particularly true for reverse DNS data, used in
determining whether node identities represent the same node or not,
since DNSSEC deployment remains inconsistent (especially for the reverse
zone).

  The security considerations that apply to any active measurement of
  live paths are relevant here as well.  See [RFC4656] and [RFC5357].

(The security considerations of 5357 don't seem to be adding much
discussion, here, though I don't object to referencing it.)

Section 9

Should we intuitively know who the "original 3 authors" are?

Section 11.1

I was surprised to see draft-ietf-ippm-ioam-data in the normative
references section, as all the previous discussion was consistent with
merely an informative reference.

Similarly, RFCs 5388, 5835, 6437, and 7312 do not seem to be referenced
in a fashion that requires a normative reference.

[Ballot comment]
Section 1

  test.  However, metrics for assessment of path components and related
  performance aspects had not been attempted in IPPM when the [RFC2330]
  framework was written.

nit(?): this seems to be the only place in the document that uses the
phrase "path component", and it was a bit confusing to me.  That is, I'm
used to seeing "path component" being (e.g.) one hop or link in a
multi-link path from source to destination, but the context here
suggests that it might be referring to something more like a single
route within a route ensemble.  I don't have a specific proposed change;
I just wanted to mention that I puzzled over this wording for a bit.

Section 2

  Also, to specify a framework for active methods of measurement which
  use the techniques described in [PT] at a minimum, and a framework
  for hybrid active-passive methods of measurement, such as the Hybrid
  Type I method [RFC7799] described in
  [I-D.ietf-ippm-ioam-data](intended only for single administrative
  domains), which do not rely on ICMP and provide a protocol for
  explicit interrogation of nodes on a path.  [...]

nit: could you remind us that the "Also" refers to the scope of this
work (the previous paragraph started with "The scope is" but there was a
fair bit of intervening stuff between there and here).

  Hop  A Hop MUST contain a Node Identity, and MAY contain arrival and/
      or departure interface identification, round trip delay, and an
      arrival timestamp.

In my head a "hop" is the operation of going from node A to node B,
i.e., the journey and not the destination.  So if we're associating a
node identity with the hop, would we need to say if that's the source or
destination node of the "hop" (or am I just misunderstanding things)?

Section 3.3

  Nmax, the largest value of i needed for a packet to be received at
  Dst (sent between T0 and Tf).  Nmax may be equal to N.

I'm not really sure I understand how Nmax works.  That is, for any given
packet, it has a value for 'i', and that packet may or may not be
delivered.  But with 'i' defined as the limit on the number of hops for
"a specific packet may visit", I don't see how we get to do multiple
tries for the same packet with different 'i', as we would need to do in
order to determine the "largest value of i *needed*" (emphasis mine).
That is, we can set 'i' unreasonably large and that would result in
delivery, but that does not (as I understand it) meet the definition for
Nmax -- Nmax is supposed to be a boundary value for some condition.  I'm
just not entirely clear on exactly what that boundary is, since we're
talking about a specific packet, and my understanding is that we only
get one crack at a specific packet.

  o  t(i,j) Arrival Timestamp (where t(i,j) is ideally supplied by the
      Hop, or approximated from the sending time of the packet that
      revealed the Hop)

I'm getting a bit of deja vu as I write this, but it seems risky to
propose using the term "arrival timestamp" both for something generated
at the Hop and something generated by the sender of the packet, since
that loses information about the accuracy of the value for a given use
case.  Perhaps it's best to reserve the term for the ideal quantity and
leave approximations for the processing pipeline (as opposed to the
measurement pipeline).

Section 3.4

  RTL(i,j,T) is defined as a singleton of the [RFC6673] Round-trip Loss
  between the Node with address = Src and the Node at Hop h(i,j) at
  time T.

Just to check my understanding: does the "singleton" and "at time T"
force us into a logical (boolean) loss metric for this case, as opposed
to a sample-ratio-type loss metric?

Section 3.6

  identities at a minimum.  The models need to be expanded to include
  these features, as well as Arrival Interface ID, Departure Interface
  ID, and Arrival Timestamp, when available.  [...]

Is this expected to occur as future work?  (Is there an I-D worth
referencing?)

Section 4.1

  Internet routing is complex because it depends on the policies of
  thousands of Autonomous Systems (AS).  While most of the routers
  perform load balancing on flows using Equal Cost Multiple Path
  (ECMP), a few still divide the workload through packet-based
  techniques.  The former scenario is defined according to [RFC2991],
  while the latter generates a round-robin scheme to deliver every new
  outgoing packet.  [...]

I was surprised to see "packet-based techniques" refer to a round-robin
scheme for queuing outgoing packets across interfaces, but assume this
just means I need to do more background reading.  Any tips for where to
start?

  Formally, to maintain the same flow in the measurements to a
  particular hop, the Type-P-Route-Ensemble-Method-Variant packets
  should be[PT]:

It's not entirely clear that we need a reference for these attributes;
they seem to be fairly easy to verify independently, IIUC.

  o  TCP case: For IPv4, the fields Src, Dst, port-Src, port_Dst,
      sequence number, and Diffserv Field SHOULD be the same.  For IPv6,
      the field FlowLabel, Src and Dst SHOULD be the same.

Are we happy to pair "to maintain the same flow" (previous paragraph)
with "SHOULD be the same" (here, and subsequent bullet points)?

Section 4.1

  1.  SHOULD occasionally check path portions which have exhibited
      stable results over time, particularly ingress and egress
      portions of the path.

Do we want to give any (e.g., order of magnitude) guidance for
"occasionally"?

Section 4.2

  In addition to node identity, nodes may also identify the ingress and
  egress interfaces utilized by the tracing packet, the time of day
  when the packet was processed, and other generic data (as described

nit: I'd consider "absolute time" over "time of day", since the latter
may have connotations of implying that there is 24-hour-periodic
behavior.  (It may also be subject to local time zone on the node
inserting the value.)

Section 4.3

  1.  Nodes at the ingress to the domain SHOULD be both Discoverable
      and Cooperating, and SHOULD reveal the same Node Identity in
      response to both active and hybrid methods.

The second clause here seems redundant with point (2) that follows.
(Similarly for point (3)'s similar clause.)  That is, the recommendation
to use the same identity for both methods can only ever apply when the
node in question is both Discoverable and Cooperating -- if it's only
one, then there will only be one identity available, and no need for
consistency across methods.

  When Nodes follow these requirements, it becomes a simple matter to

Perhaps a philosophical question, but are "SHOULD"-level directives more
of "requirements" or "guidelines"?

Section 5

  traffic load is not stationary.  Nonetheless, as the first approach,
  a link seems to be congested if, after sending several traceroute
  probes, it is possible to detect congestion observing different
  statistics parameters (e.g., see [IDCong]).  Finally, to recognize

nit: please check the wording of this sentence; I think there may be a
missing word or similar.

Section 6

  every hop.  This procedure could be done for a single Member Route
  flow, with parameter E set as False, or for every detected Route
  Ensemble flow (E=True).

nit: I don't think we define 'E' until the full procedure, a few
paragraphs below.

  made.  Line 8 and 13 set a timer for each cycle of measurements.  A
  cycle comprises the traceroutes packets, considering every possible
  Hop and the alternatives paths in the Alt variable (ensured in lines
  9-12).  [...]

I recognize that this is pseudocode, but "what if the
advanced-traceroute() operation takes longer than the i_t time?"

Section 7

We could consider some bland statement about the measurement
technologies (e.g., ICMP) not providing cryptographic protection for the
requested/returned data, and the risks of processing untrusted data in
general.  This is particularly true for reverse DNS data, used in
determining whether node identities represent the same node or not,
since DNSSEC deployment remains inconsistent (especially for the reverse
zone).

  The security considerations that apply to any active measurement of
  live paths are relevant here as well.  See [RFC4656] and [RFC5357].

(The security considerations of 5357 don't seem to be adding much
discussion, here, though I don't object to referencing it.)

Section 9

Should we intuitively know who the "original 3 authors" are?

Section 11.1

I was surprised to see draft-ietf-ippm-ioam-data in the normative
references section, as all the previous discussion was consistent with
merely an informative reference.

Similarly, RFCs 5388, 5835, 6437, and 7312 do not seem to be referenced
in a fashion that requires a normative reference.

(Via drafts-lastcall@iana.org): IESG/Authors/WG Chairs:

The IANA Functions Operator has reviewed draft-ietf-ippm-route-08, which is currently in Last Call, and has the following comments:

We understand that this document doesn't require any registry actions.

While it's often helpful for a document's IANA Considerations section to remain in place upon publication even if there are no actions, if the authors strongly prefer to remove it, we do not object.

If this assessment is not accurate, please respond as soon as possible.

Thank you,

Sabrina Tanamal
Senior IANA Services Specialist

The following Last Call announcement was sent out (ends 2020-07-03):

From: The IESG
To: IETF-Announce
CC: ietf@trammell.ch, Brian Trammell , martin.h.duke@gmail.com, draft-ietf-ippm-route@ietf.org, ippm@ietf.org, ippm-chairs@ietf.org
Reply-To: last-call@ietf.org
Sender:
Subject: Last Call:  (Advanced Unidirectional Route Assessment (AURA)) to Proposed Standard


The IESG has received a request from the IP Performance Measurement WG (ippm)
to consider the following document: - 'Advanced Unidirectional Route
Assessment (AURA)'
  as Proposed Standard

The IESG plans to make a decision in the next few weeks, and solicits final
comments on this action. Please send substantive comments to the
last-call@ietf.org mailing lists by 2020-07-03. Exceptionally, comments may
be sent to iesg@ietf.org instead. In either case, please retain the beginning
of the Subject line to allow automated sorting.

Abstract


  This memo introduces an advanced unidirectional route assessment
  (AURA) metric and associated measurement methodology, based on the IP
  Performance Metrics (IPPM) Framework RFC 2330.  This memo updates RFC
  2330 in the areas of path-related terminology and path description,
  primarily to include the possibility of parallel subpaths between a
  given Source and Destination pair, owing to the presence of multi-
  path technologies.





The file can be obtained via
https://datatracker.ietf.org/doc/draft-ietf-ippm-route/



No IPR declarations have been submitted directly on this I-D.


The document contains these normative downward references.
See RFC 3967 for additional information:
    draft-ietf-ippm-ioam-data: Data Fields for In-situ OAM (None - IETF stream)
    rfc2991: Multipath Issues in Unicast and Multicast Next-Hop Selection (Informational - Legacy stream)
    rfc5835: Framework for Metric Composition (Informational - IETF stream)
    rfc7312: Advanced Stream and Sampling Framework for IP Performance Metrics (IPPM) (Informational - IETF stream)
    rfc7799: Active and Passive Metrics and Methods (with Hybrid Types In-Between) (Informational - IETF stream)
    rfc8468: IPv4, IPv6, and IPv4-IPv6 Coexistence: Updates for the IP Performance Metrics (IPPM) Framework (Informational - IETF stream)

As required by RFC 4858, this is the current template for the Document
Shepherd Write-Up. Changes are expected over time.

This version is dated 1 November 2019.

(1) What type of RFC is being requested (BCP, Proposed Standard, Internet Standard, Informational, Experimental, or Historic)? Why is this the proper type of RFC? Is this type of RFC indicated in the title page header?

Proposed Standard, appropriate as the document proposes terminology and methodology for the measurement of comparable route-awareness in metrics, updating the standards-track IPPM framework (RFC 2330).

(2) The IESG approval announcement includes a Document Announcement Write-Up. Please provide such a Document Announcement Write-Up. Recent examples can be found in the "Action" announcements for approved documents. The approval announcement contains the following sections:

Technical Summary:

  This memo introduces an advanced unidirectional route assessment
  (AURA) metric and associated measurement methodology, based on the IP
  Performance Metrics (IPPM) Framework RFC 2330.  This memo updates RFC
  2330 in the areas of path-related terminology and path description,
  primarily to include the possibility of parallel subpaths between a
  given Source and Destination pair, owing to the presence of multi-
  path technologies.

Working Group Summary:

The document has working group consensus to publish; there was no particular controversy during the WG process.

Document Quality:

The document has been discussed within the IPPM working group for about two years, as a generalization of earlier work on adding partial route awareness to TWAMP. It has been reviewed by the community within the IPPM WG that maintains metrics within the 2330 framework. No formal or machine readable language reviews were made, as the document does not use any.

Personnel:

Brian Trammell is the document shepherd. Martin Duke is the responsible AD.

(3) Briefly describe the review of this document that was performed by the Document Shepherd. If this version of the document is not ready for publication, please explain why the document is being forwarded to the IESG.

The document shepherd has followed the development of the document (as WG co-chair and as a document reviewer) throughout its development, and has performed one further review while preparing the shepherd writeup; in the shepherd's opinion after that review, the document is ready for review by the IESG for publication, boilerplate nit aside (see below).

(4) Does the document Shepherd have any concerns about the depth or breadth of the reviews that have been performed?

No; the document has received the typical amount of review and feedback for documents published by IPPM.

(5) Do portions of the document need review from a particular or from broader perspective, e.g., security, operational complexity, AAA, DNS, DHCP, XML, or internationalization? If so, describe the review that took place.

No; the shepherd is confident the right people to review this document have seen it.

(6) Describe any specific concerns or issues that the Document Shepherd has with this document that the Responsible Area Director and/or the IESG should be aware of? For example, perhaps he or she is uncomfortable with certain parts of the document, or has concerns whether there really is a need for it. In any event, if the WG has discussed those issues and has indicated that it still wishes to advance the document, detail those concerns here.

No concerns.

(7) Has each author confirmed that any and all appropriate IPR disclosures required for full conformance with the provisions of BCP 78 and BCP 79 have already been filed. If not, explain why?

Confirmed.

(8) Has an IPR disclosure been filed that references this document? If so, summarize any WG discussion and conclusion regarding the IPR disclosures.

No IPR disclosures.

(9) How solid is the WG consensus behind this document? Does it represent the strong concurrence of a few individuals, with others being silent, or does the WG as a whole understand and agree with it?

IPPM as a WG comprises multiple communities; each community generally focuses on its own work and reviews the work of the others; this document received extensive review (and benefited from the feedback) of several working on IOAM; I judge consensus therefore to be broad and solid.

(10) Has anyone threatened an appeal or otherwise indicated extreme discontent? If so, please summarise the areas of conflict in separate email messages to the Responsible Area Director. (It should be in a separate email because this questionnaire is publicly available.)

No.

(11) Identify any ID nits the Document Shepherd has found in this document. (See http://www.ietf.org/tools/idnits/ and the Internet-Drafts Checklist). Boilerplate checks are not enough; this check needs to be thorough.

The document seems to lack the recommended RFC 2119 boilerplate, even if it appears to use RFC 2119 keywords -- however, there's a paragraph with a matching beginning. Boilerplate error?
(The document does seem to have the reference to RFC 2119 which the ID-Checklist requires).

The references also require cleanup.

The shepherd has followed up with the authors to ensure these changes are made in a subsequent version; however, these changes will not have a substantive effect on the document's technical content.

(12) Describe how the document meets any required formal review criteria, such as the MIB Doctor, YANG Doctor, media type, and URI type reviews.

None required.

(13) Have all references within this document been identified as either normative or informative?

Yes, though some appear stale and will be updated.

(14) Are there normative references to documents that are not ready for advancement or are otherwise in an unclear state? If such normative references exist, what is the plan for their completion?

Normative reference to draft-ippm-ioam-data (for security considerations) which also has WG consensus and is waiting for writeup.

(15) Are there downward normative references references (see RFC 3967)? If so, list these downward references to support the Area Director in the Last Call procedure.

No downrefs.

(16) Will publication of this document change the status of any existing RFCs? Are those RFCs listed on the title page header, listed in the abstract, and discussed in the introduction? If the RFCs are not listed in the Abstract and Introduction, explain why, and point to the part of the document where the relationship of this document to the other RFCs is discussed. If this information is not in the document, explain why the WG considers it unnecessary.

This document updates 2330 but does not change its status. 2330 is mentioned in the abstract.

(17) Describe the Document Shepherd's review of the IANA considerations section, especially with regard to its consistency with the body of the document. Confirm that all protocol extensions that the document makes are associated with the appropriate reservations in IANA registries. Confirm that any referenced IANA registries have been clearly identified. Confirm that newly created IANA registries include a detailed specification of the initial contents for the registry, that allocations procedures for future registrations are defined, and a reasonable name for the new registry has been suggested (see RFC 8126).

The document shepherd's review of the IANA considerations section is thorough: I read it three times in a row in its entirety. (The document has no actions for IANA).

(18) List any new IANA registries that require Expert Review for future allocations. Provide any public guidance that the IESG would find useful in selecting the IANA Experts for these new registries.

None.

(19) Describe reviews and automated checks performed by the Document Shepherd to validate sections of the document written in a formal language, such as XML code, BNF rules, MIB definitions, YANG modules, etc.

None.

(20) If the document contains a YANG module, has the module been checked with any of the recommended validation tools (https://trac.ietf.org/trac/ops/wiki/yang-review-tools) for syntax and formatting validation? If there are any resulting errors or warnings, what is the justification for not fixing them at this time? Does the YANG module comply with the Network Management Datastore Architecture (NMDA) as specified in RFC8342?

No YANG.

As required by RFC 4858, this is the current template for the Document
Shepherd Write-Up. Changes are expected over time.

This version is dated 1 November 2019.

(1) What type of RFC is being requested (BCP, Proposed Standard, Internet Standard, Informational, Experimental, or Historic)? Why is this the proper type of RFC? Is this type of RFC indicated in the title page header?

Proposed Standard, appropriate as the document proposes terminology and methodology for the measurement of comparable route-awareness in metrics, updating the standards-track IPPM framework (RFC 2330).

(2) The IESG approval announcement includes a Document Announcement Write-Up. Please provide such a Document Announcement Write-Up. Recent examples can be found in the "Action" announcements for approved documents. The approval announcement contains the following sections:

Technical Summary:

  This memo introduces an advanced unidirectional route assessment
  (AURA) metric and associated measurement methodology, based on the IP
  Performance Metrics (IPPM) Framework RFC 2330.  This memo updates RFC
  2330 in the areas of path-related terminology and path description,
  primarily to include the possibility of parallel subpaths between a
  given Source and Destination pair, owing to the presence of multi-
  path technologies.

Working Group Summary:

The document has working group consensus to publish; there was no particular controversy during the WG process.

Document Quality:

The document has been discussed within the IPPM working group for about two years, as a generalization of earlier work on adding partial route awareness to TWAMP. It has been reviewed by the community within the IPPM WG that maintains metrics within the 2330 framework. No formal or machine readable language reviews were made, as the document does not use any.

Personnel:

Brian Trammell is the document shepherd. Martin Duke is the responsible AD.

(3) Briefly describe the review of this document that was performed by the Document Shepherd. If this version of the document is not ready for publication, please explain why the document is being forwarded to the IESG.

The document shepherd has followed the development of the document (as WG co-chair and as a document reviewer) throughout its development, and has performed one further review while preparing the shepherd writeup; in the shepherd's opinion after that review, the document is ready for review by the IESG for publication, boilerplate nit aside (see below).

(4) Does the document Shepherd have any concerns about the depth or breadth of the reviews that have been performed?

No; the document has received the typical amount of review and feedback for documents published by IPPM.

(5) Do portions of the document need review from a particular or from broader perspective, e.g., security, operational complexity, AAA, DNS, DHCP, XML, or internationalization? If so, describe the review that took place.

No; the shepherd is confident the right people to review this document have seen it.

(6) Describe any specific concerns or issues that the Document Shepherd has with this document that the Responsible Area Director and/or the IESG should be aware of? For example, perhaps he or she is uncomfortable with certain parts of the document, or has concerns whether there really is a need for it. In any event, if the WG has discussed those issues and has indicated that it still wishes to advance the document, detail those concerns here.

No concerns.

(7) Has each author confirmed that any and all appropriate IPR disclosures required for full conformance with the provisions of BCP 78 and BCP 79 have already been filed. If not, explain why?

Confirmed.

(8) Has an IPR disclosure been filed that references this document? If so, summarize any WG discussion and conclusion regarding the IPR disclosures.

No IPR disclosures.

(9) How solid is the WG consensus behind this document? Does it represent the strong concurrence of a few individuals, with others being silent, or does the WG as a whole understand and agree with it?

IPPM as a WG comprises multiple communities; each community generally focuses on its own work and reviews the work of the others; this document received extensive review (and benefited from the feedback) of several working on IOAM; I judge consensus therefore to be broad and solid.

(10) Has anyone threatened an appeal or otherwise indicated extreme discontent? If so, please summarise the areas of conflict in separate email messages to the Responsible Area Director. (It should be in a separate email because this questionnaire is publicly available.)

No.

(11) Identify any ID nits the Document Shepherd has found in this document. (See http://www.ietf.org/tools/idnits/ and the Internet-Drafts Checklist). Boilerplate checks are not enough; this check needs to be thorough.

The document seems to lack the recommended RFC 2119 boilerplate, even if it appears to use RFC 2119 keywords -- however, there's a paragraph with a matching beginning. Boilerplate error?
(The document does seem to have the reference to RFC 2119 which the ID-Checklist requires).

The references also require cleanup.

The shepherd has followed up with the authors to ensure these changes are made in a subsequent version; however, these changes will not have a substantive effect on the document's technical content.

(12) Describe how the document meets any required formal review criteria, such as the MIB Doctor, YANG Doctor, media type, and URI type reviews.

None required.

(13) Have all references within this document been identified as either normative or informative?

Yes, though some appear stale and will be updated.

(14) Are there normative references to documents that are not ready for advancement or are otherwise in an unclear state? If such normative references exist, what is the plan for their completion?

Normative reference to draft-ippm-ioam-data (for security considerations) which also has WG consensus and is waiting for writeup.

(15) Are there downward normative references references (see RFC 3967)? If so, list these downward references to support the Area Director in the Last Call procedure.

No downrefs.

(16) Will publication of this document change the status of any existing RFCs? Are those RFCs listed on the title page header, listed in the abstract, and discussed in the introduction? If the RFCs are not listed in the Abstract and Introduction, explain why, and point to the part of the document where the relationship of this document to the other RFCs is discussed. If this information is not in the document, explain why the WG considers it unnecessary.

This document updates 2330 but does not change its status. 2330 is mentioned in the abstract.

(17) Describe the Document Shepherd's review of the IANA considerations section, especially with regard to its consistency with the body of the document. Confirm that all protocol extensions that the document makes are associated with the appropriate reservations in IANA registries. Confirm that any referenced IANA registries have been clearly identified. Confirm that newly created IANA registries include a detailed specification of the initial contents for the registry, that allocations procedures for future registrations are defined, and a reasonable name for the new registry has been suggested (see RFC 8126).

The document shepherd's review of the IANA considerations section is thorough: I read it three times in a row in its entirety. (The document has no actions for IANA).

(18) List any new IANA registries that require Expert Review for future allocations. Provide any public guidance that the IESG would find useful in selecting the IANA Experts for these new registries.

None.

(19) Describe reviews and automated checks performed by the Document Shepherd to validate sections of the document written in a formal language, such as XML code, BNF rules, MIB definitions, YANG modules, etc.

None.

(20) If the document contains a YANG module, has the module been checked with any of the recommended validation tools (https://trac.ietf.org/trac/ops/wiki/yang-review-tools) for syntax and formatting validation? If there are any resulting errors or warnings, what is the justification for not fixing them at this time? Does the YANG module comply with the Network Management Datastore Architecture (NMDA) as specified in RFC8342?

No YANG.