IPv6 Application of the Alternate-Marking Method
draft-ietf-6man-ipv6-alt-mark-17

Hi Giuseppe,

Thanks for the updates in version 16. I've cleared my DISCUSS. I do have some new comments which I'm providing below.

1. In §1,

   Besides, [RFC8799] also discusses the trend towards network behaviors
   that can be applied only within limited domains.

I suggest slightly re-wording like

   [RFC8799] provides further discussion of network behaviors
   that can be applied only within limited domains.

Apart from just stylistic preference, this change reduces the risk that the reader may think RFC 8799 is being pointed to as a source of authority rather than supplemental information.


2. In §2.1.1,

      the CPE (Customer Premises Equipment) and the PE (Provider Edge)
      router are most likely to be the starting or ending node since

Shouldn't this be CPE *or* PE?

      they can border a controlled domain.  For instance, the CPE, which

"They can border" doesn't seem quite right since it implies the CPE or PE would be on the exterior of, but adjacent to, the domain. Do you mean, "they can be border routers of the controlled domain"?


3. In §4,

                                  If segment list compression is in use,
   as described in [I-D.ietf-spring-srv6-srh-compression], Destination
   Options or Hop-by-Hop Options can always be applicable.
   
Please forgive me but the meaning of this sentence isn't clear to me. When I work through what I *think* you're trying to say, it's something like "when compression is in use there might not be a RH and in that case DOs if present, are processed at every SR node along the compressed segment list". Is that right? In looking at draft-ietf-spring-srv6-srh-compression-01 and other related specs, I actually can't figure out WHAT is supposed to happen with DOs in the NEXT-C-SID case :-(. That leaves us with a few options I guess:

- Educate me to understand why draft-ietf-spring-srv6-srh-compression-01 is actually not ambiguous in this regard (I am already working on this but have not reached a happy conclusion).
- Correct draft-ietf-spring-srv6-srh-compression-01 to be unambiguous, and update the text in your draft to match (but then this gives you a dependency on that work).
- Find a way to remove the dependency on SRv6 from your draft.

To me, the final option seems like the most attractive since it introduces no dependencies. A sketch of text that might do the job:

   IPv6 [RFC8200] provides both Hop-by-Hop Options and Destination
   Options headers, furthermore depending on how the Destination Options
   are situated in the header chain (before, or after, the Routing
   Header if any), they may be processed by either intermediate routers
   specified in the Routing Header, or the destination node. Using these
   tools, it is possible to control on which nodes measurement occurs:

   o  Destination Option not preceding a Routing Header => measurement
      only by node in Destination Address.

   o  Hop-by-Hop Option => every router on the path with feature
      enabled.

   o  Destination Option preceding a Routing Header => every destination
      node in the route list.

Then leave it to the SRv6 document set to specify whatever's necessary with respect to DO processing, and if any variance with regard to placement before/after RH is required, it's up to the SRv6 document set to specify it.


4. In §5.3, This is OK,

   The FlowMonID allocation procedure can be stateful or stateless.  In
   case of a stateful approach, it is required that each source node
   stores and keeps track of the FlowMonID historic information in order
   to assign unique values within the domain.  This may imply a complex
   procedure and it is considered out of scope for this document.
   
But, isn't most of the complexity required in this case for the purposes of tracking and retrieving what ID you associated with which flow? And therefore, that complexity would also be required in the case of a centralized controller, mentioned just a few paragraphs down in bullet 2?

I support the DISCUSS positions of Lars, Warren, Martin, and Roman.  I'm particularly interested in Martin's DISCUSS point about the two Experimental references and their statuses.

That's a lot right there, so in addition from me, just some minor things:

The answer to question 1 of the shepherd document is not complete.

Section 1:

* "... an on-path telemetry technique and consists in synchronizing ..." -- s/in/of/

* "... and therefore divide the packet ..." -- s/divide/dividing/ (must match "synchronizing" and "switching" earlier in the sentence)

OLD:
   The threat model for the application of the Alternate Marking Method
   in an IPv6 domain is reported in Section 6.  As for all the on-path
   telemetry technique, the only definitive solution is that this
   methodology MUST be applied in a controlled domain and therefore the
   application to untrusted domain is NOT RECOMMENDED.
NEW:
   The threat model for the application of the Alternate Marking Method
   in an IPv6 domain is reported in Section 6.  As with all on-path
   telemetry techniques, the only definitive solution is that this
   methodology MUST be applied in a controlled domain and therefore
   application to untrusted domains is NOT RECOMMENDED.

Sections 2, 4, and 6 variably use "behavior" and "behaviour".  I think you need to pick one and use it consistently.  Same for "analyze" and "analyse" (though in different sections).

Thank you to Chris Wood for the SECDIR review and to the authors for responding to it.

Thank you for the document revisions to address my DISCUSS and COMMENT feedback.

Giuseppe,

Thank you for the -10 revision, which indeed addresses all my previous DISCUSS. Hence, I am now balloting NO OBJECTION to the document. Please ignore the DISCUSS points below that I kept only for documentation.

Regards

-éric

-- everything below is for archiving purpose only --

Thank you for the work put into this document.

As Brian Carpenter kindly reminded me that an Area Director may not put back a previous argument when the WG consensus was to ignore it, I am updating this ballot by moving the generic DISCUSS as a COMMENT.

The other two DISCUSS points remains blocking ones but Giuseppe have already addressed them over email, I am clearing the DISCUSS position as soon as the revised I-D is posted.

Special thanks to Ole Trøan for his shepherd's write-up, which contains a good summary of the WG consensus and the SW (?) implementation status. Thanks as well to Bob Halley for his INT directorate review.

Please find below some blocking DISCUSS points, some non-blocking COMMENT points (but replies would be appreciated).

I also agree with DISCUSS points by other ADs:
- Lars's one about the size and usefulness of FlowMonID (see my very similar point)
- Roman's one about the inconsistencies about limited domain (from my point of view, this should not be limited to a single domain)

All in all, I strongly suggest to change the intended status of this document to 'experimental' as this document builds on experimental RFCs.

I hope that this helps to improve the document,

Regards,

-éric


== PREVIOUS DISCUSS (only for archives) ==


-- Section 2 --
  "In the end, [I-D.fioccola-v6ops-ipv6-alt-mark] demonstrated that a
   new Hop-by-Hop or a new Destination Option was the best approach."

The above draft has not been published and is even expired for 2 years now... so please remove the 'demonstrated' as there is no proof at all ;-)

-- Section 4 --
Please remove "the destination node removes it" as there is no reason to remove it. Extension headers are usually not presented / used by upper layers anyway.

-- generic --
This discussion already happened in the 6MAN WG but I still have to be convinced that hop-by-hop is a good idea at all. The fate of packets with HbH is unknown at best and, IMHO, this is a wrong use of HbH because the forwarding/processing of packets is NOT influenced at all by the AltMark option. What is required is a simple 'marking/coloring', which could be at any place in a packet (including destination option). The only benefit of HbH is its location just after the IPv6 header.

After all, existing devices can measure and generate IPFIX records in hardware/fastpath by inspecting at the bare minimum the 5-tuple and many devices can also parse extension headers in the fast path (of course not processing HbH in this case). I.e., finding the marking in the Dest Option is probably much easier and doable in current hardware rather than using HbH as HbH forces a software processing.

--- PREVIOUS COMMENT (for archive) ---

-- Section 2.1 --
While I agree with my fellow AD about the inconsistencies of the text around 'limited domain' (and being inconsistent deserves a blocking DISCUSS), I do not agree with this section as I do not see any harm to extend this marking outside of a single domain, marked packets with this specification can traverse the whole Internet without causing any harm and not opening for any attack on the transit nodes. (See also my point about Hop-by-hop)

E.g., packets from organisation A can traverse networks of organizations B, C, ..., Z to reach another site of organization A and still provide useful measurements for A without causing harm to B, C, ..., Z. Of course the survivability of those marked packets is not guaranteed.

-- Section 3.1 --
Is there any reason why the FlowMonID is 20 bit long ? Exactly as the 20-bit Flow Label of the IPv6 header... See my comments for section 5.3

s/Unrecognized Types MUST be ignored on receipt./Unrecognized Types MUST be ignored on processing./ as the option should be processed not only by the destination node.

-- Section 5 --
This section (in a standard track I-D) repeats many things from experimental RFCs... This is really confusing and, while I am usually not dogmatic, I support my fellow AD on this point.

Moreover, repeating contents from other docs can only introduce inconsistencies among docs => suggest to remove section 5.1 and 5.2.

-- Section 5.3 --
The general reasons of using a 20-bit FlowMonID appear quite weak to me with the risk of collision being very high, e.g., in a DC with thousands of VM and millions of containers there will be collisions rendering this field useless. Strongly suggest to make its use optional

(1) I support the DISCUSS positions from Lars, Roman, and Martin.  

(1a) Of special concern to me is Martin's point about the relationship between this document and rfc8321/8889, and the potential ability to reference this work without proper review by the ippm WG.  Note that neither RFC is explicit about the criteria to complete the respective experiments; the Shepherd writeup for rfc8321 [a] states that "the measurement utility of this extension still is to be demonstrated at a variety of scales in a plurality of network conditions." Furthermore, I am not aware of discussions about the maturity of rfc8321 in the ippm WG.  

[a] https://datatracker.ietf.org/doc/draft-ietf-ippm-alt-mark/shepherdwriteup/


(2) There are several references to I-D.fioccola-v6ops-ipv6-alt-mark, which was replaced by draft-fz-6man-ipv6-alt-mark and ultimately by this document.  IOW, it looks like this document refers to an old version of itself.  Since the references are mostly about analysis made in the early drafts, it may be better to include some of that in an appendix instead.

My colleagues already made some good reviews, so I don't have any more
comments to add at the Discuss level (though I will follow the
resolution of the other reviews).  I do have a number of COMMENT-level
comments (plus some NITS-level remarks at the end that probably don't
need individual responses).

Section 1

   application in an IPv6 domain.  While the case of Segment Routing
   Header (SRH), defined in [RFC8754], is also discussed, it is valid
   for all the types of Routing Header (RH).

Given that it's not required to use a routing header with this option,
and the discussion in section 4 explicitly covers the ordering between
destination option and (an arbitrary) routing header, I think it might
be okay to drop this sentence.

Section 2

   The Alternate Marking Method requires a marking field.  As mentioned,
   several alternatives have been analysed in
   [I-D.fioccola-v6ops-ipv6-alt-mark] such as IPv6 Extension Headers,
   IPv6 Address and Flow Label.

If one follows the "replaced by" chain, it seems that
draft-fioccola-v6ops-ipv6-alt-mark is listed as a direct predecessor of
this document.  So it seems a little odd to refer back to a previous
version of this draft (without even acknowledging that it is a previous
version of this work) for additional details on a topic that is also
treated here in a summary form.  Why not just eliminate the reference
and include the needed information here (perhaps in an appendix if
needed)?  For what it's worth, I think that the summary in this section
can stand on its own without need for further supporting references.

   Furthermore the Flow Label may be changed en route and this may also
   violate the measurement task.  [...]

My (hasty) reading of RFC 6437 is that such changes are strongly
discouraged and only allowed in exceptional circumstances.  Perhaps the
description here should account for that?

   monitored flow.  Flow Label and FlowMonID within the same packet are
   totally disjoint, have different scope, identify different flows, and
   are intended for different use cases.

Can you explain a bit more (not necessarily in the document) what
"identify different flows" means?  It seems like if two identifiers are
carried in the same packet they must in some sense identify the same
thing, so I'm not sure what the different "flows" would be in this
scenario.

   The rationale for the FlowMonID is further discussed in Section 5.3.
   This 20 bit field allows easy and flexible identification of the
   monitored flow and enables a finer granularity and improved
   measurement correlation. [...]

I'm a bit confused at how "finer granularity" comes in -- if the
FlowMonID is 20 bits, isn't that the same granularity as the 20-bit Flow
Label?

Section 2.1

   [RFC8799] introduces the concept of specific limited domain solutions
   and, in this regard, it is reported the IPv6 Application of the
   Alternate Marking Method as an example.

I think Roman already touched on this point as well, but RFC 8799 only
introduces the concept; it's not the final word on the concept, and
8799 seems to expect that documents using the concept of a controlled
domain will add on much more specifics about what is actually meant.

   Some scenarios may imply that the Alternate Marking Method is applied
   outside a controlled domain, either intentionally or unintentionally,
   but in these cases, IPsec authentication and encryption MUST be used.

I appreciate the sentiment, but a hard MUST requirement for IPsec
specifically may be needlessly limiting.  Why would some other VPN
technology that provides full packet confidentiality+integrity
protection while traversing the external domain be insufficient?

Section 4

   Options Header to a slow processing path.  In case of the AltMark
   data fields described in this document, the motivation to standardize
   a new Hop-by-Hop Option is that it is needed for OAM (Operations,
   Administration, and Maintenance).  [...]

It seems to also be worth mentioning here that the limitation of use to
a controlled domain helps mitigate the risk of arbitrary nodes dropping
packets with HbH options.

Section 5

   This section describes how the method operates.  [RFC8321] introduces
   several alternatives but in this section the most applicable methods
   are reported and a new field is introduced to facilitate the
   deployment and improve the scalability.

If only the "most applicable methods" are reported, and both RFC 8321
and this document say to prefer a batching strategy based on a fixed
time interval, doesn't that make the batching strategy based on fixed
packet count "not most applicable" and thus undeserving of mention at
all in this document?

Section 5.2

   The same principle used to measure packet loss can be applied also to
   one-way delay measurement.  Delay metrics MAY be calculated using the
   two possibilities:

Again, how are there two possibilities that are both "most applicable
methods"?

   In summary the approach with double marking is better than the
   approach with single marking.  Moreover the two approaches can also
   be combined to have even more information and statistics on delay.

It might be worth saying a little more about how the approaches can be
"combined"; my current understanding seems more like they are both
executed in parallel on the same input data, but they produce
independent result values and any combination of those results is left
to the system (or human) processing the results.

Section 5.3

   o  First, it helps to reduce the per node configuration.  Otherwise,
      each node needs to configure an access-control list (ACL) for each
      of the monitored flows.  Moreover, using a flow identifier allows
      a flexible granularity for the flow definition.

This seems to only be relevant in the case where there is definitely a
central controller involved, if I'm understanding correctly -- if the
source is assigning the FlowMonIDs the consumers will still need to look
for those flows by ACL.  So maybe this could be rephrased slightly to
account for that.

Section 5.3.1

   of collision for 1206 flows.  So, for more entropy, FlowMonID can
   either be combined with other identifying flow information in a
   packet (e.g. it is possible to consider the hashed 3-tuple Flow
   Label, Source and Destination addresses) or the FlowMonID size could
   be increased.

This document is about to become a final, standards-track, RFC.  I
strongly suggest removing discussion of things that are not possible
once the protocol becomes immortalized in an RFC (that is, increasing
the FlowMonID size).

Section 6

Thank you for putting so much effort into the security considerations
section already!  I have a few thoughts on potential additional topics
to consider.

Not necessarily something that needs to be discussed in the security
considerations section itself, but the security considerations in
practice will depend on whether the marking is always enabled for all
flows, always enabled for a subset of flows, only enabled for specific
flows on-demand, etc..  Some kind of scope-setting for the expected
usage could be helpful to see, since the expected usage shapes what
analysis has been done of the security properties.

Separately, it is perhaps too banal to mention, but attacks on the
reporting/consolidation of the statistics between the monitoring devices
and the analysis platform can interfere with the proper functioning of
the system.  The channels used to report back flow statistics should be
secured.

There are also some considerations to note when the FlowMonIDs are
centrally allocated by a controller -- the controller knows a lot about
what traffic is flowing!  That might make it the target of an attack,
depending on the attacker goals.

Another factor that comes into play when the controller (or anyone,
really) is non-randomly assigning IDs, is that the actual structure of
the ID allocation itself can cause issues -- for example, predictable
sequential assignment can give an attacker insight into the rate of flow
creation, what fraction of flows they have visibility into from their
current vantage point, and makes attacks that are contingent on guessing
the "next" identifier very feasible.  The considerations in
draft-gont-numeric-ids-sec-considerations seem to apply.

   Harm caused by the measurement: Alternate Marking implies

I think Roman already touched on this to some extent, but the FlowMonID
in the measurement option adds some privacy considerations that can in
some cases mean that the measurement causes harm.

      single monitoring point.  The only way for an attacker can be to
      eavesdrop on multiple monitoring points at the same time, because
      they have to do the same kind of calculation and aggregation as
      Alternate Marking requires, and, after that, it can finally gain
      information about the network performance, but this is not
      immediate.

Just my own opinion, but I think this would be more useful if it stopped
after "have to do the same kind of calculation and aggregation as
Alternate Marking requires."  That is, the attacker doesn't have an
advantage over the rightful admin, but can get the same data; we don't
need to say anything more, and the mention of "not immediate" is
arguably misleading.

   Furthermore, in a pervasive surveillance attack, the information that
   can be derived over time is more.  But the application of the
   Alternate Marking to a controlled domain helps to mitigate all the
   above aspects of privacy concerns.

I suggest adding another clause or sentence about why/how the controlled
domain mitigates the privacy concerns (perhaps just as a forward
reference to the text currently two paragraphs later than this one,
which might be easier to achieve if the security considerations were
split into subsections).

   this document.  As an example, the AltMark Option can be used as Hop-
   by-Hop or Destination Option and, in case of Destination Option,
   multiple domains may be traversed by the AltMark Option that is not
   confined to a single domain.  In this case, the user, aware of the

(This statement might be added to the list that Roman enumerated of
places where we do/don't talk about being confined to a limited domain.)

   confined to a single domain.  In this case, the user, aware of the
   kind of risks, may still want to use Alternate Marking for telemetry
   and test purposes but the inter-domain links need to be secured
   (e.g., by IPsec) in order to avoid external threats.  For these

(Interesting that here IPsec is only an "e.g." and not a MUST...even
though the "MUST" is repeated in the following sentence!  C.f. my remark
on Section 2.1.)

   measurement.  A detailed discussion about the threats against time
   protocols and how to mitigate them is presented in [RFC7384].  Also,

NTS was recently published as RFC 8915; it might be worth mentioning
here.

Section 9.2

It's not really clear to me that RFCs 8321 and 8889 can be classified as
informative.

NITS

Section 1

   The Alternate Marking is an on-path telemetry technique and consists
   in synchronizing the measurements in different points of a network by
   switching the value of a marking bit and therefore divide the packet
   flow into batches.  Each batch represents a measurable entity

s/divide/dividing/, to match up with the conjugation of "synchronizing"
and "switching".

   unambiguously recognizable by all network nodes along the path.  By

I think that "unambiguously" can fail in the face of extreme reording
(relative to the marking window).

   in an IPv6 domain is reported in Section 6.  As for all the on-path
   telemetry technique, the only definitive solution is that this

s/all the on-path telemetry technique/all on-path telemetry techniques/

Section 2

   Hop-by-Hop Option Header is also useful to signal to routers on the
   path to process the Alternate Marking.  However, as said, routers
   will examine this option if properly configured.

I think that "only" should be added, either for "will only examine" or
"only if properly configured" (but not both).

   Furthermore the Flow Label may be changed en route and this may also
   violate the measurement task.  [...]

"violate the measurement task" is an unusual phrasing, and I'm not
actually sure what meaning it's intended to convey.  Perhaps it's
something about invalidating the integrity of the measurement result?

Section 4

   In general, it is needed to perform both end to end and hop by hop
   measurements, and the Alternate Marking methodology allows, by
   definition, both performance measurements.  But, in many cases the
   end-to-end measurement is not enough and it is required also the hop-
   by-hop measurement, so the most complete choice is the Hop-by-Hop
   Options Header.

The paragraph structure here doesn't make much sense: we say that in
general, both (end-to-end and hop-by-hop) measurements are possible, but
then follow up with a note that "in many cases end-to-end is not
enough".  There's no previous situation described where only end-to-end
is possible that merits this comparison.  It seems that some specific
mention of Destination Options would be needed in order to be able to
make such a comparison.

Section 5.1

   single batch between any two nodes.  Each batch represents a
   measurable entity unambiguously recognizable by all network nodes
   along the path.

(Same remark about "unambiguous" as above)

   It is worth mentioning that the length of the batches is considered
   stable over time in the previous figure.  In theory, it is possible

I'd consider "duration" rather than "length", here (indeed, the figure
shows batches of both 10 and 11 packets, and possibly 9 as well if
"batch 1" is not considered to be truncated

Section 6

      The FlowMonID field is used in the AltMark Option as identifier of
      the monitored flow.  It represents a more sensitive information

"identifier" needs an article; I'm not sure whether "a" or "the" would
better match the intended meaning.

Section 5.1. , paragraph 4, comment:
>    issues.  Specifically, if the effects of network delay are ignored,
>    the condition to implement the methodology is that the clocks in
>    different nodes MUST be synchronized to the same clock reference with
>    an accuracy of +/- B/2 time units, where B is the fixed time duration
>    of the block.

What is a block? Do you mean a batch? I see that RFC8321 uses "block", should
this document adopt that term?

Also, how can a block (of packets) have a "fixed time duration"? Is this
supposed to mean transmission time of the block at line-rate? What about pacing?

-------------------------------------------------------------------------------
All comments below are about very minor potential issues that you may choose to
address in some way - or ignore - as you see fit. Some were flagged by
automated tools (via https://github.com/larseggert/ietf-reviewtool), so there
will likely be some false positives. There is no need to let me know what you
did with these suggestions.

Section 2.1. , paragraph 4, nit:
-    an implementation can be able to reject packets that carry Alternate
-                      ---------------
+    an implementation rejects packets that carry Alternate
+                            +

Section 3.1. , paragraph 5, nit:
-    o  Option Type: 8 bit identifier of the type of Option that needs to
-                     ^
+    o  Option Type: 8-bit identifier of the type of Option that needs to
+                     ^

Section 3.1. , paragraph 7, nit:
-    o  FlowMonID: 20 bits unsigned integer.  The FlowMon identifier is
-                    ^   -
+    o  FlowMonID: 20-bit unsigned integer.  The FlowMon identifier is
+                    ^

Section 3.1. , paragraph 7, nit:
-       picked 20 bit since it is a reasonable value and a good compromise
+       picked as 20 bit since it is a reasonable value and a good compromise
+             +++

Section 4. , paragraph 6, nit:
-    are useable when SRv6 header is present.  Because SRv6 is implemented
-          -
+    are usable when SRv6 header is present.  Because SRv6 is implemented

Section 1. , paragraph 3, nit:
> sely measure the packet loss. In a similar way the alternation of the values
>                               ^^^^^^^^^^^^^^^^
Consider replacing this phrase with the adverb "similarly" to avoid wordiness.

Section 1.1. , paragraph 1, nit:
> I-D.fioccola-v6ops-ipv6-alt-mark] analyzed and discussed all the available po
>                                   ^^^^^^^^
Do not mix variants of the same word ("analyze" and "analyse") within a single
text.

Section 2. , paragraph 8, nit:
> and on the nodes that support it. However it is important to mention that th
>                                   ^^^^^^^
A comma may be missing after the conjunctive/linking adverb "However".

Section 2. , paragraph 11, nit:
> intent and forwarding behaviour. Furthermore the Flow Label may be changed en
>                                  ^^^^^^^^^^^
A comma may be missing after the conjunctive/linking adverb "Furthermore".

Section 2.1. , paragraph 2, nit:
> f IPsec for an unintentional use outside of a controlled domain? If the head
>                                  ^^^^^^^^^^
This phrase is redundant. Consider using "outside".

Section 3.1. , paragraph 1, nit:
> rd-highest-order bit specifies whether or not the Option Data can change en r
>                                ^^^^^^^^^^^^^^
Consider shortening this phrase to just "whether". It is correct though if you
mean "regardless of whether".

Section 3.1. , paragraph 4, nit:
> ssed below, it has been picked as 20 bit since it is a reasonable value and a
>                                      ^^^
Possible agreement error. The noun "bit" seems to be countable.

Section 4. , paragraph 6, nit:
> intermediate node can read it or not but this does not affect the packet beh
>                                     ^^^^
Use a comma before "but" if it connects two independent clauses (unless they
are closely connected and short).

Section 4. , paragraph 8, nit:
> an-hbh-processing], this means "skip if do not recognize and data do not cha
>                                      ^^
It seems that a pronoun is missing.

Section 4. , paragraph 12, nit:
> most applicable methods are reported and a new field is introduced to facili
>                                     ^^^^
Use a comma before "and" if it connects two independent clauses (unless they
are closely connected and short).

Section 5.1. , paragraph 4, nit:
> ge the length of batches over time and and among different flows for more fle
>                                    ^^^^^^^
Possible typo: you repeated a word.

Section 5.1. , paragraph 9, nit:
> rage timestamp for each batch. In addition the information is limited to a me
>                                   ^^^^^^^^
A comma may be missing after the conjunctive/linking adverb "addition".

Section 5.2. , paragraph 3, nit:
> the approach with single marking. Moreover the two approaches can also be com
>                                   ^^^^^^^^
A comma may be missing after the conjunctive/linking adverb "Moreover".

Section 5.2. , paragraph 7, nit:
> table in most of the applications. Indeed with 20 bits the number of combinat
>                                    ^^^^^^
A comma may be missing after the conjunctive/linking adverb "Indeed".

Section 5.3.1. , paragraph 3, nit:
> r of IPv6 packets by the source node but this must be performed in a way tha
>                                     ^^^^
Use a comma before "but" if it connects two independent clauses (unless they
are closely connected and short).

Section 6. , paragraph 2, nit:
> privacy concerns also need to be analyzed even if the method only relies on
>                                  ^^^^^^^^
Do not mix variants of the same word ("analyze" and "analyse") within a single
text.

Section 6. , paragraph 4, nit:
>  paths. AltMark Option contains two kind of metadata: the marking bits (L and
>                                     ^^^^
Possible agreement error. The noun "kind" seems to be countable.

Section 6. , paragraph 5, nit:
> n data-plane traffic is difficult. Indeed an attacker cannot gain information
>                                    ^^^^^^
A comma may be missing after the conjunctive/linking adverb "Indeed".

Section 6. , paragraph 6, nit:
> hin the network domain. [RFC8799] analyzes and discusses the trend towards ne
>                                   ^^^^^^^^
Do not mix variants of the same word ("analyze" and "analyse") within a single
text.

Section 6. , paragraph 7, nit:
> tMark Option leaving the domain. Therefore the trusted domain is unlikely su
>                                  ^^^^^^^^^
A comma may be missing after the conjunctive/linking adverb "Therefore".

Section 6. , paragraph 7, nit:
> ome packets might exceed the MTU. However the relative small size (48 bit in
>                                   ^^^^^^^
A comma may be missing after the conjunctive/linking adverb "However".

Section 6. , paragraph 9, nit:
>  described in this document relies on an time synchronization protocol. Thus,
>                                       ^^
Use "a" instead of "an" if the following word doesn't start with a vowel sound,
e.g. "a sentence", "a university".

Document references draft-peng-v6ops-hbh-04, but -05 is the latest available
revision.

Document references draft-fioccola-v6ops-ipv6-alt-mark-01, but -08 is the
latest available revision.

Thanks for addressing my DISCUSS.

Thanks to Yoshi for the TSVART review.

very late review and as such not much to add to the existing ones, with which I largely agree