Multicast Using Bit Index Explicit Replication (BIER)
draft-ietf-bier-architecture-08

[Ballot comment]
Not wishing to re-litigate any of the previous discussions (for which I was not available), I have reviewed only the deltas between -07 and -08. I have no objection to the changed and new material.

[Ballot comment]
Update:
While it is now clearer why this doc is not informational, I agree with the gen-art review (Thanks Dan!) that is could be made even clearer what the experiment is/why this is experimental and not PS.

Old ballot text:
I also initially expected that this document should be informational because it specifies an architecture, however, while reading I realized that is rather specifies abstract mechanisms and a data model. So it's basically an abstract protocol spec without defining the on the wire format. Maybe it would be helpful to describe the content of this document this way, rather than speaking of an architecture. However, I guess the word architecture is rather board though.

Thanks for a well-written doc; especially the ECMP part is very clear!

[Ballot comment]
I also initially expected that this document should be informational because it specifies an architecture, however, while reading I realized that is rather specifies abstract mechanisms and a data model. So it's basically an abstract protocol spec without defining the on the wire format. Maybe it would be helpful to describe the content of this document this way, rather than speaking of an architecture. However, I guess the word architecture is rather board though.

Thanks for a well-written doc; especially the ECMP part is very clear!

[Ballot comment]
Just asking for my own understanding ...

In this text,

  If, at some time, the routing underlay is not providing a loop free
  path between BFIR-A and BFER-B, then BIER encapsulated packets may
  loop while traveling from BFIR-A to BFER-B.  However, such loops will
  never result in delivery of duplicate packets to BFER-B.

I'm assuming that these loops could result in out of order delivery of packets to BFER-B?

No document change requested, of course. And either answer is fine.

[Ballot comment]
Not asking except for my own understanding ...

In this text,

  If, at some time, the routing underlay is not providing a loop free
  path between BFIR-A and BFER-B, then BIER encapsulated packets may
  loop while traveling from BFIR-A to BFER-B.  However, such loops will
  never result in delivery of duplicate packets to BFER-B.

I'm assuming that these loops could result in out of order delivery of packets to BFER-B?

No document change requested, of course. And either answer is fine.

(Via drafts-lastcall@iana.org): IESG/Authors/WG Chairs:

The IANA Services Operator has reviewed draft-ietf-bier-architecture-08, which is currently in Last Call, and has the following comments:

We understand that this document doesn't require any registry actions.

While it's often helpful for a document's IANA Considerations section to remain in place upon publication even if there are no actions, if the authors strongly prefer to remove it, we do not object.

If this assessment is not accurate, please respond as soon as possible.

Thank you,

Sabrina Tanamal
IANA Services Specialist

The following Last Call announcement was sent out (ends 2017-09-20):

From: The IESG
To: IETF-Announce
CC: Greg Shepherd , gjshep@gmail.com, bier@ietf.org, draft-ietf-bier-architecture@ietf.org, akatlas@gmail.com, bier-chairs@ietf.org
Reply-To: ietf@ietf.org
Sender:
Subject: Last Call:  (Multicast using Bit Index Explicit Replication) to Experimental RFC


The IESG has received a request from the Bit Indexed Explicit Replication WG
(bier) to consider the following document: - 'Multicast using Bit Index
Explicit Replication'
  as Experimental RFC

This Last Call is on the technical changes between draft-ietf-bier-architecture-07
and draft-ietf-bier-architecture-08.  Specifically, this is specifying that each
encapsulation should specify its own default bitStringLength.

The IESG plans to make a decision in the next few weeks, and solicits final
comments on this action. Please send substantive comments to the
ietf@ietf.org mailing lists by 2017-09-20. Exceptionally, comments may be
sent to iesg@ietf.org instead. In either case, please retain the beginning of
the Subject line to allow automated sorting.

Abstract


  This document specifies a new architecture for the forwarding of
  multicast data packets.  It provides optimal forwarding of multicast
  packets through a "multicast domain".  However, it does not require a
  protocol for explicitly building multicast distribution trees, nor
  does it require intermediate nodes to maintain any per-flow state.
  This architecture is known as "Bit Index Explicit Replication"
  (BIER).  When a multicast data packet enters the domain, the ingress
  router determines the set of egress routers to which the packet needs
  to be sent.  The ingress router then encapsulates the packet in a
  BIER header.  The BIER header contains a bitstring in which each bit
  represents exactly one egress router in the domain; to forward the
  packet to a given set of egress routers, the bits corresponding to
  those routers are set in the BIER header.  The procedures for
  forwarding a packet based on its BIER header are specified in this
  document.  Elimination of the per-flow state and the explicit tree-
  building protocols results in a considerable simplification.




The file can be obtained via
https://datatracker.ietf.org/doc/draft-ietf-bier-architecture/

IESG discussion can be tracked via
https://datatracker.ietf.org/doc/draft-ietf-bier-architecture/ballot/

The following IPR Declarations may be related to this I-D:

  https://datatracker.ietf.org/ipr/2624/
  https://datatracker.ietf.org/ipr/2439/
  https://datatracker.ietf.org/ipr/2986/
  https://datatracker.ietf.org/ipr/2987/
  https://datatracker.ietf.org/ipr/2447/
  https://datatracker.ietf.org/ipr/2512/

[Ballot comment]
Next point moved from DISCUSS to COMMENT after the IESG telechat:
1. I want to come back to the experimental status.

I have seen Alia's reply:

    In the BIER Charter, work item 9 describes a document based on deployment
    experience that can justify moving the BIER work from Experimental to Standards
    Track.  When I chartered the WG, it wasn't clear whether it was merely a good idea
    or a good idea with enough motivations towards deployment that it is worth complicating
    the architecture at the narrow point of the IP hourglass.

From the writeup, it seems that the experiment is successful already:

    The vendors are being quite tight lipped about current implementations.
    Operator feedback indicates there are at least two implementations currently, with others in the works.
    There are currently five vendors collaborating on the work in the IETF.

However, I've not been following the specifications development and implementation to provide a definitive answer.

At the minimum, the document needs to describe what the criteria are for a successful experiment.
    Implementations (RFC7942?)?
    two interop implementations?
    hardware implementation?
    "deployment experience" (to cite the charter text)?
    impact analysis?
    something else?

The ideal BIER document for this explanation is this architecture doc.
A reference to the charter Item 9 would be a good start.

Examples:
https://tools.ietf.org/html/rfc7360#section-1.3
https://tools.ietf.org/html/rfc7499#section-2



-  Each BFR MUST be assigned a single "BFR-Prefix" for each sub-domain
  to which it belongs.  A BFR's BFR-Prefix MUST be an IP address
  (either IPv4 or IPv6) of the BFR.  It is RECOMMENDED that the
  BFR-prefix be a loopback address of the BFR.

Just one observation. Was thinking, so it's like an (OSPF) routerId, except that it's called a prefix.
Then I understood, reading further. It's not called a routerId because this address/prefix must be routable

- Was wondering. Is this mechanism only for multicast packets? What about anycast/unicast?
Some multicast packets would take BIER encap, while others packets would take a different encap
From a troubleshooting point of view, does it make sense (is it even allowed) to send non multicast traffic over BIER.

[Ballot discuss]
1. operations and management section
I'm sure there are such considerations:
    - configuration
    - sub-domain management
    - BFR-id management
    - adding new BFIR/BFER device to the domain
    - logging
    - troubleshooting
    - OAM
    - etc.

There are also two other management documents that should be referenced: chartered item 6 and 7.
There are some management sentences, from time to time, in the doc.
Ex: avoiding a second copy is an important from an operational point of view

  It is generally advantageous to assign the BFR-ids of a given sub-
  domain so that as many BFERs as possible can be represented in a
  single bit string.

  ...

  In order to minimize the number of copies that must be made of a
  given multicast packet, it is RECOMMENDED that the BFR-ids be
  assigned "densely" (see Section 2) from the numbering space...

    Btw, I guess you meant: assigned densely per subdomain, right?

What this "operations and management section" should contain is the points that operators, who will deploy this technology, have to pay attention to.
An architecture document is typically the first document people will read. RFC 5706 appendix A provide typical questions from an OPS point of view.
I understand that this is an experiment and you might not have all the answers at this point in time.

[Ballot comment]
Next point moved from DISCUSS to COMMENT after the IESG telechat:
1. I want to come back to the experimental status.

I have seen Alia's reply:

    In the BIER Charter, work item 9 describes a document based on deployment
    experience that can justify moving the BIER work from Experimental to Standards
    Track.  When I chartered the WG, it wasn't clear whether it was merely a good idea
    or a good idea with enough motivations towards deployment that it is worth complicating
    the architecture at the narrow point of the IP hourglass.

From the writeup, it seems that the experiment is successful already:

    The vendors are being quite tight lipped about current implementations.
    Operator feedback indicates there are at least two implementations currently, with others in the works.
    There are currently five vendors collaborating on the work in the IETF.

However, I've not been following the specifications development and implementation to provide a definitive answer.

At the minimum, the document needs to describe what the criteria are for a successful experiment.
    Implementations (RFC7942?)?
    two interop implementations?
    hardware implementation?
    "deployment experience" (to cite the charter text)?
    impact analysis?
    something else?

The ideal BIER document for this explanation is this architecture doc.
A reference to the charter Item 9 would be a good start.

Examples:
https://tools.ietf.org/html/rfc7360#section-1.3
https://tools.ietf.org/html/rfc7499#section-2



-  Each BFR MUST be assigned a single "BFR-Prefix" for each sub-domain
  to which it belongs.  A BFR's BFR-Prefix MUST be an IP address
  (either IPv4 or IPv6) of the BFR.  It is RECOMMENDED that the
  BFR-prefix be a loopback address of the BFR.

Just one observation. Was thinking, so it's like an (OSPF) routerId, except that it's called a prefix.
Then I understood, reading further. It's not called a routerId because this address/prefix must be routable

- Was wondering. Is this mechanism only for multicast packets? What about anycast/unicast?
End of section 4 you have an example about MVPN over ingress/egress PEs.
Some multicast packets would take BIER encap, while others packets would take a different encap
From a troubleshooting point of view, does it make sense (is it even allowed) to send non multicast traffic over BIER.

Victor Kuarsing performed the OPS-DIR review, copied below. Discussion is ongoing.

First, there were no textual changes recommended as part of this
review.  The document seems to be well written and had undergone
previous review/updates.  Given the nature of this document, there are
limited operational specific points made as part of this review
however three points are noted for follow-up with the actors.


(1). Security Considerations


It was not 100% clear to the reviewer (me) if the use case of how the
BIER architecture (or guidance for future implementation) may avoid
certain DoS like activity from illegitimate neighbors (BFR-NBRs).  I
may have missed specific text which addresses the point made herein
(if so, please disregard my first point and please make reference
where it is addressed).  The concern is if a packet is introduced into
the system by a host (compromised perhaps) which fabricates packets
(may or may not have valid payload inside the BIER encapsulation) that
attempts to starve resources in the system.  One way of doing this
(which may require control plane access - maybe) would be to
set/create packets which forces additional replication at BFRs (i.e.
set a single  - or all bits -  for all SIs).  This would then create
the maximum amount of replication within the system.  Perhaps this use
case is addressed as noted.


I was no able to find specific operations/functions which scrutinized
incoming packets and/or guard for bad NBRs.  Could this be an issue
from the authors standpoint?  In more traditional networks, there are
methods operators use to ensure that illegitimate traffic is not
introduced into the system.  I wanted to make sure there was thoughts
on how to do this with BIER.


(2). SPF assumed.


I agree most places where the BIER architecture would likely live (say
in Enterprises and SP networks) the network would operate a standard
IGP.  However, in some use cases, like modern DCs, some are designing
fabrics which don’t use a traditional IGP.  These networks use all BGP
(eBGP) with full sets of neighbor relationships.  This helps reduce
the amount of running protocols within the underlay, but may (or may
not) cause issues with BIER in relation distribution of system
information.  Do the authors consider this use case one that can be
address with BIER as it’s currently outlined, or would additional
considerations be needed?


I reviewed the “draft-ietf-bier-use-cases” document and did not see
text that specifically help or hinder the operational mode I described
above.


(3) Broadcast Video Operation. (more of a question then a point of note)


I did noticed in the use case document (as noted above in point 2),
that considers more traditional broadcast video networks was
described.  So my question is, would an operational model which
required two simultaneous M/C flows from separate sources (normally
two packagers/encryptors etc) be supported by the BIER architecture as
described?  My best estimate would be that the operator would use two
sub-domains that would allow each BFER to potentially get two of each
packet (which are sourced by two different injection points / BIFRs).
This mode of operation is common in some places to allow the M/C
broadcast feed to be “pinned” to the egress router/device to allow for
fast switchover in case of network failure (where normal network level
detection and re-routing is not sufficient).

[Ballot discuss]
1. I want to come back to the experimental status.

I have seen Alia's reply:

    In the BIER Charter, work item 9 describes a document based on deployment
    experience that can justify moving the BIER work from Experimental to Standards
    Track.  When I chartered the WG, it wasn't clear whether it was merely a good idea
    or a good idea with enough motivations towards deployment that it is worth complicating
    the architecture at the narrow point of the IP hourglass.

From the writeup, it seems that the experiment is successful already:

    The vendors are being quite tight lipped about current implementations.
    Operator feedback indicates there are at least two implementations currently, with others in the works.
    There are currently five vendors collaborating on the work in the IETF.

However, I've not been following the specifications development and implementation to provide a definitive answer.

At the minimum, the document needs to describe what the criteria are for a successful experiment.
    Implementations (RFC7942?)?
    two interop implementations?
    hardware implementation?
    "deployment experience" (to cite the charter text)?
    impact analysis?
    something else?

The ideal BIER document for this explanation is this architecture doc.
A reference to the charter Item 9 would be a good start.

Examples:
https://tools.ietf.org/html/rfc7360#section-1.3
https://tools.ietf.org/html/rfc7499#section-2

2. operations and management section
I'm sure there are such considerations:
    - configuration
    - sub-domain management
    - BFR-id management
    - adding new BFIR/BFER device to the domain
    - logging
    - troubleshooting
    - OAM
    - etc.

There are also two other management documents that should be referenced: chartered item 6 and 7.
There are some management sentences, from time to time, in the doc.
Ex: avoiding a second copy is an important from an operational point of view

  It is generally advantageous to assign the BFR-ids of a given sub-
  domain so that as many BFERs as possible can be represented in a
  single bit string.

  ...

  In order to minimize the number of copies that must be made of a
  given multicast packet, it is RECOMMENDED that the BFR-ids be
  assigned "densely" (see Section 2) from the numbering space...

    Btw, I guess you meant: assigned densely per subdomain, right?

What this "operations and management section" should contain is the points that operators, who will deploy this technology, have to pay attention to.
An architecture document is typically the first document people will read. RFC 5706 appendix A provide typical questions from an OPS point of view.
I understand that this is an experiment and you might not have all the answers at this point in time.

[Ballot comment]
-  Each BFR MUST be assigned a single "BFR-Prefix" for each sub-domain
  to which it belongs.  A BFR's BFR-Prefix MUST be an IP address
  (either IPv4 or IPv6) of the BFR.  It is RECOMMENDED that the
  BFR-prefix be a loopback address of the BFR.

Just one observation. Was thinking, so it's like an (OSPF) routerId, except that it's called a prefix.
Then I understood, reading further. It's not called a routerId because this address/prefix must be routable

- Was wondering. Is this mechanism only for multicast packets? What about anycast/unicast?
End of section 4 you have an example about MVPN over ingress/egress PEs.
Some multicast packets would take BIER encap, while others packets would take a different encap
From a troubleshooting point of view, does it make sense (is it even allowed) to send non multicast traffic over BIER.

Victor Kuarsing performed the OPS-DIR review, copied below. Discussion is ongoing.

First, there were no textual changes recommended as part of this
review.  The document seems to be well written and had undergone
previous review/updates.  Given the nature of this document, there are
limited operational specific points made as part of this review
however three points are noted for follow-up with the actors.


(1). Security Considerations


It was not 100% clear to the reviewer (me) if the use case of how the
BIER architecture (or guidance for future implementation) may avoid
certain DoS like activity from illegitimate neighbors (BFR-NBRs).  I
may have missed specific text which addresses the point made herein
(if so, please disregard my first point and please make reference
where it is addressed).  The concern is if a packet is introduced into
the system by a host (compromised perhaps) which fabricates packets
(may or may not have valid payload inside the BIER encapsulation) that
attempts to starve resources in the system.  One way of doing this
(which may require control plane access - maybe) would be to
set/create packets which forces additional replication at BFRs (i.e.
set a single  - or all bits -  for all SIs).  This would then create
the maximum amount of replication within the system.  Perhaps this use
case is addressed as noted.


I was no able to find specific operations/functions which scrutinized
incoming packets and/or guard for bad NBRs.  Could this be an issue
from the authors standpoint?  In more traditional networks, there are
methods operators use to ensure that illegitimate traffic is not
introduced into the system.  I wanted to make sure there was thoughts
on how to do this with BIER.


(2). SPF assumed.


I agree most places where the BIER architecture would likely live (say
in Enterprises and SP networks) the network would operate a standard
IGP.  However, in some use cases, like modern DCs, some are designing
fabrics which don’t use a traditional IGP.  These networks use all BGP
(eBGP) with full sets of neighbor relationships.  This helps reduce
the amount of running protocols within the underlay, but may (or may
not) cause issues with BIER in relation distribution of system
information.  Do the authors consider this use case one that can be
address with BIER as it’s currently outlined, or would additional
considerations be needed?


I reviewed the “draft-ietf-bier-use-cases” document and did not see
text that specifically help or hinder the operational mode I described
above.


(3) Broadcast Video Operation. (more of a question then a point of note)


I did noticed in the use case document (as noted above in point 2),
that considers more traditional broadcast video networks was
described.  So my question is, would an operational model which
required two simultaneous M/C flows from separate sources (normally
two packagers/encryptors etc) be supported by the BIER architecture as
described?  My best estimate would be that the operator would use two
sub-domains that would allow each BFER to potentially get two of each
packet (which are sourced by two different injection points / BIFRs).
This mode of operation is common in some places to allow the M/C
broadcast feed to be “pinned” to the egress router/device to allow for
fast switchover in case of network failure (where normal network level
detection and re-routing is not sufficient).

[Ballot comment]
I also initially expected that this document should be informational because it specifies an architecture, however, will reading I realized that is rather specifies abstract mechanisms and a data model. So it's basically an abstract protocol spec without defining the on the wire format. Maybe it would be helpful to describe the content of this document this way, rather than speaking of an architecture. However, I guess the word architecture is rather board though.

Thanks for a well-written doc; especially the ECMP part is very clear!

[Ballot comment]
[ After response from Alia I'm clearing my discuss. I still somewhat feel that an Architecture document which discusses an experimental protocol / idea doesn't need to be Experimental itself, but a: I don't feel it very strongly / could be wrong, and b: her "I did discuss with the BIER WG that it is possible to do a status update to move a
document from Experimental to Proposed Standard without needing to update the RFC." makes me happy.  I'm copying my original discuss comment below for hysterical raisins. ] 


Notes:
Section 1:
"If the packet also needs to be sent to a BFER whose BFR-id is 257, the BFIR would have to create a second copy of the packet, and the BIER encapsulation would specify an SI of 1, and a BitString with bit 1 set and all the other bits clear." - while reading this it seemed to me that it would be much simpler to do away with the SI completely and just make the BitString N bits longer (instead of having e.g a one octet SI and 2 octet BitString, concatenate this and have a 3 octet BitString). It was only when I got down to Section 3 that I discovered that this is (kinda) discussed, and that each SI can have a different BitString length. I'd suggest adding a refernce (like: "and a BitString with bit 1 set and all the other bits clear (see Section 3 for more details)."

Section 4.1.  The Routing Underlay:
"Alternatively, one could deploy a routing underlay that creates a multicast-specific tree of some sort, perhaps a Steiner tree."
A reference for Steiner trees would be nice -- best I could find was probably the WA page at: Weisstein, Eric W. "Steiner Tree." From MathWorld--A Wolfram Web Resource. http://mathworld.wolfram.com/SteinerTree.html
Please note: I'd pay good money for a router doing Steiner optimizations using the soap and surface tension trick.

Nits and notes:
1: " A router that supports BIER is known as a "Bit-Forwarding Router" (BFR)." -- this makes me sad; the BFR acronym was already in use, and had interesting history behind it (Big er... Fast Router) - http://photos.kumari.net/Technology/Networking/i-k4vCdwv/A

2: "MVPN signaling also has several components that depend on the
  type of "tunneling technology" used to carry multicast data though
  the network." -- though the network what?! I suspect you meant "through" :-)
3: Section 6.1.  Overview
"If a BFR receives a BIER-encapsulated packet whose sub-domain, SI and BitString identify that BFR itself, ..." -- identifies.

O: "When BIER on a BFER is to pass a packet to the multicast flow overlay, it of course decapsulates..."
P: "When BIER on a BFER is to pass a packet to the multicast flow overlay, it, of course decapsulates"
C: Missing a comma. I suspect: "When BIER on a BFER is to pass a packet to the multicast flow overlay, it decapsulates..." would be even better.

Section 6.9.  When Some Nodes do not Support BIER
"In this section, we assume that the routing underlay is an SPF-based IGP that computes a shortest path tree from each node to all other nodes in the domain."
C: I *think* that this should actually be "the shortest path", but I'm not sure; everything talks about e.g Dijkstra's algorithm finding the shortest path, but what if there are multiple equal cost paths? But, this is a singular tree, but may not be unique, so any SP tree will... erk... Good this this is a nit and I can move on :-)


Section 6.10.3.  Transitioning from One BitStringLength to Another
Typo:  Dispostion ->  Disposition



------ ORIGINAL DISCUSS TEXT ----
I agree with Ben, but feel more strongly than him - why is this Experimental? Where is the experiment / how do you determine if it is successful? The shepherd writeup says: "(1) Experimental, as per charter. The document title page header indicates experimental. ", but this doesn't really answer the question; the charter says:
"BIER is initially chartered to do experimental work on this new
multicast forwarding mechanism as follows:

1) BIER architecture: The WG will publish an architecture, based
upon draft-wijnands-bier-architecture-04...."

I really think that this document should be Informational or PS, or something.
I can understand the *work* to be experimental in nature, but the document *itself* seems like it shouldn't be - it doesn't (really) explain how to implement.

I'm making this a DISCUSS, but am more than happy to clear if the chairs / AD says that the've considered this and Experimental really is best.

[Ballot discuss]
I agree with Ben, but feel more strongly than him - why is this Experimental? Where is the experiment / how do you determine if it is successful? The shepherd writeup says: "(1) Experimental, as per charter. The document title page header indicates experimental. ", but this doesn't really answer the question; the charter says:
"BIER is initially chartered to do experimental work on this new
multicast forwarding mechanism as follows:

1) BIER architecture: The WG will publish an architecture, based
upon draft-wijnands-bier-architecture-04...."

I really think that this document should be Informational or PS, or something.
I can understand the *work* to be experimental in nature, but the document *itself* seems like it shouldn't be - it doesn't (really) explain how to implement.

I'm making this a DISCUSS, but am more than happy to clear if the chairs / AD says that the've considered this and Experimental really is best.

[Ballot comment]
Notes:
Section 1:
"If the packet also needs to be sent to a BFER whose BFR-id is 257, the BFIR would have to create a second copy of the packet, and the BIER encapsulation would specify an SI of 1, and a BitString with bit 1 set and all the other bits clear." - while reading this it seemed to me that it would be much simpler to do away with the SI completely and just make the BitString N bits longer (instead of having e.g a one octet SI and 2 octet BitString, concatenate this and have a 3 octet BitString). It was only when I got down to Section 3 that I discovered that this is (kinda) discussed, and that each SI can have a different BitString length. I'd suggest adding a refernce (like: "and a BitString with bit 1 set and all the other bits clear (see Section 3 for more details)."

Section 4.1.  The Routing Underlay:
"Alternatively, one could deploy a routing underlay that creates a multicast-specific tree of some sort, perhaps a Steiner tree."
A reference for Steiner trees would be nice -- best I could find was probably the WA page at: Weisstein, Eric W. "Steiner Tree." From MathWorld--A Wolfram Web Resource. http://mathworld.wolfram.com/SteinerTree.html
Please note: I'd pay good money for a router doing Steiner optimizations using the soap and surface tension trick.

Nits and notes:
1: " A router that supports BIER is known as a "Bit-Forwarding Router" (BFR)." -- this makes me sad; the BFR acronym was already in use, and had interesting history behind it (Big er... Fast Router) - http://photos.kumari.net/Technology/Networking/i-k4vCdwv/A

2: "MVPN signaling also has several components that depend on the
  type of "tunneling technology" used to carry multicast data though
  the network." -- though the network what?! I suspect you meant "through" :-)
3: Section 6.1.  Overview
"If a BFR receives a BIER-encapsulated packet whose sub-domain, SI and BitString identify that BFR itself, ..." -- identifies.

O: "When BIER on a BFER is to pass a packet to the multicast flow overlay, it of course decapsulates..."
P: "When BIER on a BFER is to pass a packet to the multicast flow overlay, it, of course decapsulates"
C: Missing a comma. I suspect: "When BIER on a BFER is to pass a packet to the multicast flow overlay, it decapsulates..." would be even better.

Section 6.9.  When Some Nodes do not Support BIER
"In this section, we assume that the routing underlay is an SPF-based IGP that computes a shortest path tree from each node to all other nodes in the domain."
C: I *think* that this should actually be "the shortest path", but I'm not sure; everything talks about e.g Dijkstra's algorithm finding the shortest path, but what if there are multiple equal cost paths? But, this is a singular tree, but may not be unique, so any SP tree will... erk... Good this this is a nit and I can move on :-)


Section 6.10.3.  Transitioning from One BitStringLength to Another
Typo:  Dispostion ->  Disposition

[Ballot comment]
I'm a bit confused about the experimental status of this document as well, given that the other WG drafts that define protocol extensions to support BIER seem to be headed for the standards track. If what is documented here really is the only experimental part, this seems like the place to document the experiment.

[Ballot comment]
It's not clear why this is experimental. What is the purpose or nature of the experiment? What does one hope to learn from the results?

The document seems to go well beyond what I think of as architecture. Please consider clarifying that in the title and intro.

(Via drafts-lastcall@iana.org): IESG/Authors/WG Chairs:

The IANA Services Operator has reviewed draft-ietf-bier-architecture-06.txt, which is currently in Last Call, and has the following comments:

We understand that this document doesn't require any registry actions.

While it's often helpful for a document's IANA Considerations section to remain in place upon publication even if there are no actions, if the authors strongly prefer to remove it, we do not object.

If this assessment is not accurate, please respond as soon as possible.

Thank you,

Sabrina Tanamal
IANA Services Specialist
PTI

The following Last Call announcement was sent out:

From: The IESG
To: IETF-Announce
CC: Greg Shepherd , gjshep@gmail.com, bier@ietf.org, draft-ietf-bier-architecture@ietf.org, akatlas@gmail.com, bier-chairs@ietf.org
Reply-To: ietf@ietf.org
Sender:
Subject: Last Call:  (Multicast using Bit Index Explicit Replication) to Experimental RFC


The IESG has received a request from the Bit Indexed Explicit Replication WG
(bier) to consider the following document: - 'Multicast using Bit Index
Explicit Replication'
  as Experimental RFC

The IESG plans to make a decision in the next few weeks, and solicits final
comments on this action. Please send substantive comments to the
ietf@ietf.org mailing lists by 2017-06-29. Exceptionally, comments may be
sent to iesg@ietf.org instead. In either case, please retain the beginning of
the Subject line to allow automated sorting.

Abstract


  This document specifies a new architecture for the forwarding of
  multicast data packets.  It provides optimal forwarding of multicast
  packets through a "multicast domain".  However, it does not require a
  protocol for explicitly building multicast distribution trees, nor
  does it require intermediate nodes to maintain any per-flow state.
  This architecture is known as "Bit Index Explicit Replication"
  (BIER).  When a multicast data packet enters the domain, the ingress
  router determines the set of egress routers to which the packet needs
  to be sent.  The ingress router then encapsulates the packet in a
  BIER header.  The BIER header contains a bitstring in which each bit
  represents exactly one egress router in the domain; to forward the
  packet to a given set of egress routers, the bits corresponding to
  those routers are set in the BIER header.  Elimination of the per-
  flow state and the explicit tree-building protocols results in a
  considerable simplification.




The file can be obtained via
https://datatracker.ietf.org/doc/draft-ietf-bier-architecture/

IESG discussion can be tracked via
https://datatracker.ietf.org/doc/draft-ietf-bier-architecture/ballot/

The following IPR Declarations may be related to this I-D:

  https://datatracker.ietf.org/ipr/2624/
  https://datatracker.ietf.org/ipr/2439/
  https://datatracker.ietf.org/ipr/2986/
  https://datatracker.ietf.org/ipr/2987/
  https://datatracker.ietf.org/ipr/2447/
  https://datatracker.ietf.org/ipr/2512/

First, I would like to thank the authors - Ice, Eric, Andrew, Tony, and Sam - for an excellently written and very clear document which forms the foundation for this technology. I wish I saw many more drafts with this level of clarity, thought, and details.  I would also like to thank the BIER WG for the reviews and discussions that have led to this quality.

As is customary, I have done my AD review.  I do have a few minor issues that need to be resolved, but I am comfortable requesting that the IETF Last Call start immediately, with the assumption that the excellent & responsive authors will resolve these issues and issue an updated draft quite rapidly.

My goal is to have this document on the IESG telechat of July 7 so that it will either be in the RFC Editor queue or the WG will be aware of any issues before IETF 99.

Minor:

1) Sec 4.1: "its BFR prefix"  Is it possible to have multiple BFR prefixes for the same BFR?  I am thinking about renumbering cases (for acquisitions) as well as migrations from IPv4 to IPv4 + IPv6 and then to IPv6.  Conceptually, I see no reason that there couldn't be multiple BFR prefixes for the same BFR as long as they really are that BFR's prefixes. 

2) Sec 4.2: This is missing the procedures for removing the BIER header for the payload - which would presumably include potential details such as TTL inheritance and DSCP/EXP handling....

3) Sec 6.5:  This has a block of "pseudocode", but it isn't surrounded by the  and  indications.  Those are useful because there are different copyright terms for code in RFCs versus the text of RFCs.  It would be worth taking a look at the difference and deciding whether using  and  would be useful.

4) Sec 6.6.1:"At BFER-D, the BIFT entry (not pictured) for BFR-id 1 will specify an
  F-BM of 0000 and a BFR-NBR of BFR-D itself."
Sec 6.6.2: "At BFER-D, the BIFT entry (not pictured) for BFR-id 1 will specify an
  F-BM of 0000 and a BFR-NBR of BFR-D itself."  and "At BFER-E, the BIFT entry (not pictured) for BFR-id 3 will specify an F-BM of 0000 and a BFR-NBR of BFR-E itself."
I don't understand why the F-BM would be 0000 and not 0001 nor how the packet's BitString would end up cleared if the F-BM were 0000.  I suspect a typo, but it is repeated thrice...  I will note that in Sec 6.5 after the pseudocode, it says " It also assumes that the corresponding F-BM has only one bit set, the bit representing the BFER itself."  which appears to contradict the two examples.

5) Sec 6.9:"This may be as simple as finding the IGP unicast next hop to the child node, and pushing on (above the BIER-MPLS label advertised by the child) the MPLS label that the IGP next hop has bound to an address of the child node.  (If
  for some reason the unicast tunnel cannot be an MPLS tunnel, any
  other kind of tunnel can be used, as long as it is possible to
  encapsulate MPLS within that kind of tunnel.)"
Given the generalization of the BIER encapsulation & possibility of IPv6 encapsulations & that a BIER-MPLS label hasn't been introduced,  could this be rephrased a bit more generally?  The concept, obviously, is fine.
The same applies to "That way, the TTL of the BIER-MPLS label constrains only the number of BFRs that the packet may traverse, not the total number of hops."
which could easily be rewritten as "...the TTL of the BIER header constrains..."

6) Sec 10.2.1: "This fallback procedure will work best if the value of F is
  established by the architecture, rather than by provisioning."  Since this is the
architecture and it does specify that all BFRs must support a BitStringLength of 256 - surely this is exactly the point to establish the value of F and there isn't a better time. :-)

Nit:

a) End of Section 3:"An encapsulation for use in MPLS networks is described in
  [MPLS_BIER_ENCAPS]"
Given that encap is also now a generic encapsulation that will work for Ethernet, this statement could be expanded to be more complete.

b) Sec 4.3: "Since BIER is, in effect, a new type of "tunneling
  technology", some extensions to the MVPN signaling are needed in
  order to properly interface the multicast flow overlay with the BIER
  layer.  These will be specified in a companion document."
I think that this is draft-ietf-bier-mvpn, so it'd be helpful to have an informative reference to it.

(1) Experimental, as per charter. The document title page header indicates experimental.
(2)
Technical Summary:
This document specifies a new architecture for the forwarding of multicast data packets.  It provides optimal forwarding of multicast data packets through a "multicast domain".  However, it does not require the use of a protocol for explicitly building multicast distribution trees, and it does not require intermediate nodes to maintain any per-flow state.  This architecture is known as "Bit Index Explicit Replication" (BIER).
Working Group Summary:
Working group showed solid consensus for this document. The relevant point of contention was only around the charter?s current direction for this work to be submitted on the Experimental track. Feedback from the AD indicated that there is a path to transition from Experimental to Standards track if there in consensus to do so in the future. When a multicast data packet enters the domain, the ingress router determines the set of egress routers to which the packet needs to be sent.  The ingress router then encapsulates the packet in a BIER header.  The BIER header contains a bitstring in which each bit represents exactly one egress router in the domain; to forward the packet to a given set of egress routers, the bits corresponding to those routers are set in the BIER header.  Elimination of the per-flow state and the explicit tree-building protocols results in a considerable simplification.
Document Quality:
The vendors are being quite tight lipped about current implementations. Operator feedback indicates there are at least two implementations currently, with others in the works. There are currently five vendors collaborating on the work in the IETF. There is no MIB doctor or other expert review outside of the active working group members
Personnel:
Document Shepherd: Greg Shepherd gjshep@gmail.com
Area Director: Alia Atlas akatlas@gmail.com
(3) This document reached consensus in Working Group Last Call to progress to the IESG.
(4) No concerns
(5) This work is viewed as being considerably valuable. It does however describe a new forwarding plane, and therefore has drawn the necessary attention to vet the proposed solution as part of the Internet Architecture.
(6) No concerns
(7) All authors have confirmed IPR disclosures.
(8) All authors confirmed on the email list that any relevant IPR disclosures have been filed in reference to this docmument.
(9) The WG as a whole understands and agrees.
(10) No appeals.
(11) No nits.
(12) No formal review required.
(13) All references correctly documented.
(14) All normative references are existing RFCs.
(15) No downward normative references.
(16) This document does not change the status of any existing RFCs
(17) This document contains no actions for IANA.
(18) This document contains no actions for IANA.
(19) XML nit tool returns with no errors.