Enterprise Multihoming using Provider-Assigned IPv6 Addresses without Network Prefix Translation: Requirements and Solutions
Note: This ballot was opened for revision 08 and is now closed.
(Suresh Krishnan) Yes
Comment (2019-06-27 for -08)
* Section 3.3 Paragraph 2 The destination address of H41 seems to be wrong. Shouldn't it be "D=2001:db8:0:b020::41" instead of "D=2001:db8:0:b010::41"? * Section 4 Page 22 I think this text needs to be rephrased as a requirement rather than a two statements. "Any traffic that needs to exit the site will eventually hit a SADR- capable router. Once that traffic enters the SADR-capable domain, then it will not leave that domain until it exits the site." * Section 5.2.1. Not sure what the reference to RFC8415 accomplishes in this contact. Is it just a pointer to DHCPv6? If so it needs to be earlier in the document. If there is a more relevant reason, I think the pointer needs to be more specific (e.g. a section in RFC8415). * Section 7.3 I think a reference to something like RFC6824 might be useful here
Martin Vigoureux Yes
Deborah Brungard No Objection
Comment (2019-06-25 for -08)
Much thanks for a very comprehensive document! Similar to Alvaro's (F), I find a couple of sentences confusing. I think it would be very helpful to clarify the scope of this document (in this document), especially as Alvaro notes, the same working group is progressing a PS document with another solution. Examples: - Abstract: I find the sentence "attempts..complete" solution a bit in conflict. "Attempts" - either it does or doesn't. "complete" is questionable as it is focused on a set of use cases. Suggest: This document attempts to define a complete solution to this problem /s/ This document examines currently available mechanisms for providing a solution to this problem for a broad range of enterprise topologies. - Section 4: "The method described in the current document is functionally equivalent, but it is intended to be easier to understand for enterprise network operators." I don't find the justification "easier to understand for enterprise network operators" to be convincing. Especially if there is already a PS document being progressed in the same working group. Hopefully the PS document will also be easy to understand for both operators and vendors. Suggest a better qualifier, even simply: but it is intended to be easier to understand for enterprise network operators /s/ but it is based on application of existing mechanisms for the described scenarios
Alissa Cooper (was Discuss) No Objection
Comment (2019-08-01 for -11)
Thanks for addressing my DISCUSS. I think the disclaimer in Section 6 about default address selection is sufficient to address remaining concerns about the use of the term "host" throughout Section 6. I think Section 6 might be slightly improved if the section headings that talk about "source address selection" instead said "default source address selection." I agree with the Gen-ART reviewer that a forward reference to Section 8.3 in the last sentence of Section 6.7.1 would be useful.
Roman Danyliw No Objection
Comment (2019-06-26 for -08)
A few questions and comments: (1) Section 1. Per “That is, some routers must be capable of some form of Source Address Dependent Routing (SADR), if only as described in [RFC3704]”, I’m not sure what minimal SADR technique is being described in RFC3704. What section? (2) Section 3.5. Per “However, when evaluating scalable implementations of the solution, it would be reasonable to assume that the maximum number of ISPs that a site would connect to is five”, what is the basis of the up to _five_ ISPs? (3) Section 5. Per “If all of the ISP uplinks are working, the choice of source address by the host may be driven by the desire to load share across ISP uplinks …”, how does an individual host have enough information to make that kind of decision? (4) Section 5. Per “An external host initiating communication with a host internal to a PA multihomed site will need to know multiple addresses for that host in order to communicate with it using different ISPs to the multihomed site.”, what is mean by “in order to communicate with it using different ISPs”? Why would being multi-homed this matter? Wouldn’t one IP address be sufficient? Section 9. Per “It recommends that a given host verify the original packet header included into ICMPv6 error message was actually sent by the host itself”, is this guidance to the host or to network stack implementers? (5) Section 9. Given the current (helpful) text about the threat of a spoofed ICMPv6 message, it would be equally useful to discuss the threat to the other approach in Section 5 (i.e., DHCPv6) – a rogue DHCPv6 server. A few editorial nits: ** Section 4. s/As as/As/ ** Section 5.1. Typo. s/such as way/such a way/ ** Section 5.2.3 Multiple Typos. s/signalling/signaling/g ** Section 5.2.3. Typo. s/An site/A site/ ** Section 5.6. Per the use of the term “wall garden”, do you mean “walled garden”? ** Section 5.6 and 5.7.1. Multiple Typos. s/envinronments /environments/g ** Section 6. Typo. s/mutiple/multiple ** Section 6. Missing space. /[I-D.ietf-intarea-provisioning-domains]takes/[I-D.ietf-intarea-provisioning-domains] takes/ ** Section 9. Typo. /mechanim/mechanism/
Benjamin Kaduk No Objection
Comment (2019-06-26 for -08)
I mostly only have editorial comments, but please note the potential additional security considerations for ICMPv6 "use this source address" messages, and the question about leaving a SADR domain being equivalent to leaving the site. Abstract This document attempts to define a complete solution to this problem. It covers the behavior of routers to forward traffic taking into account source address, and it covers the behavior of host to select appropriate source addresses. [...] nit: singular/plural mismatch routers/host Section 1 The return packet will be routed over the Internet to ISP-A, but it will not be delivered to the multihomed site because its link with ISP-A has failed. [...] nit: I think formally the subject that "it" refers to in "its link" is the packet, not the site, so we'd want to disambiguate here. Note that the same may be true with a provider that does not implement BCP 38, if his upstream provider does, or has no corresponding route. The issue is not that the immediate provider implements ingress filtering; it is that someone upstream does, or lacks a route. I'm sure this is just my lack of background, but I didn't see much introduction of what a "corresponding route" means. That is, some routers must be capable of some form of Source Address Dependent Routing (SADR), if only as described in [RFC3704]. [...] I do not see reference to either "source address dependent routing" or "SADR" in RFC 3704. Section 3.2 In Figure 2, we modify the topology slightly by inserting R7, so that SERa and SERb are no longer directly connected. With this topology, it is not enough to just enable SADR routing on SERa and SERb to support PA multi-homing. There are two solutions to ways to enable PA multihoming in this topology. nit: "solutions to ways" seems redundant Section 4 3. Augment each less specific source-prefix-scoped forwarding table with all more specific source-prefix-scoped forwarding tables entries based on the following rule. If the destination prefix of the less specific source-prefix-scoped forwarding entry exactly matches the destination prefix of an existing more specific source-prefix-scoped forwarding entry (including destination prefix length), then do not add the less specific source-prefix-scoped forwarding entry. [...] I think this is just editorial, but we start by saying ~"augment less-specific routes" and thenwe say ~"do not add the less-specific routes", which doesn't match up -- we can't add X to the baseline when X is the baseline, and would have to remove X and replace it with the more-specific thing. The forward tables produced by this process are used in the following way to forward packets. nit: "forwarding tables" Any traffic that needs to exit the site will eventually hit a SADR- capable router. Once that traffic enters the SADR-capable domain, then it will not leave that domain until it exits the site. [...] Er, what enforces/provides this property? (I assume it's not a new requirement being introduced here...) An interesting side-effect of deploying SADR is if all routers in a given network support SADR and have a scoped forwarding table, then the unscoped forwarding table can be eliminated which ensures that packets with legitimate source addresses only can leave the network nit: s/packets with legitimate source addresses only/only packets with legitimate source addresses/ It would prevent accidental leaks of ULA/reserved/link- local sources to the Internet as well as ensures that no spoofing is possible from the SADR-enabled network. nit: s/ensures/ensuring/ Section 5 If all of the ISP uplinks are working, the choice of source address by the host may be driven by the desire to load share across ISP uplinks, or it may be driven by the desire to take advantage of certain properties of a particular uplink or ISP. If any of the ISP uplinks is not working, then the choice of source address by the host can determine if packets get dropped. nit: maybe s/determine if packets get dropped/cause packets to be dropped/ ? It seems unlikely that a host would specifically choose a source address in order to provide the "will be dropped" behavior, since it could just not send the packet in the first place instead. For a session originated from an external host to an internal host, the choice of source address used by the internal host is simple. The internal host has no choice but to use the destination address in the received packet as the source address of the transmitted packet. (side note) I guess there may be cases where the internal host has a prearranged agreement with the external host to triangle-route packets, but (quibbles about "no choice" aside) that doesn't seem pedagogically relevant to mention here. Section 5.2 Again we return to the topology in Figure 3. Suppose that the site administrator wants to implement a policy by which all hosts need to use ISP-A to reach H01 at D=2001:db8:0:1234::101. [...] nit: I think this wants s/H01/H101/ Section 5.2.3 We can also use this source-prefix-scoped route originated by SERa to communicate the desired routing policy to SERb1. We can define an EXCLUSIVE flag to be advertised together with the IGP route for (S=2001:db8:0:a000::/52, D=2001:db8:0:1234::/64). [...] Just to check my understanding, is this "we can define" a statement of future possibilities (viz. https://tools.ietf.org/html/draft-pioxfolks-6man-pio-exclusive-bit-02) or something being undertaken in this current document? However using ICMPv6 for signalling source address information back to hosts introduces new challenges. [...] New security risks, too! In addition, the same source prefix SHOULD be used for other destinations in the same /64 as the original destination address. The source prefix SHOULD have a specific lifetime. Expiration of the lifetime SHOULD trigger the source address selection algorithm again. nit: I assume this lifetime is for the cached mapping of src/dst prefixes, and not for using the source prefix at all. Section 5.2.4 As all those options have been standardized by IETF and are supported by various operating systems, no changes are required on hosts. [...] nit: this is a comma splice. The policy distribution can be automated by defining an EXCLUSIVE flag for the source-prefix-scoped route which can be set on the SER that originates the route. [...] nit: "can" is present tense and implies the capability already exists today; my understanding from the rest of the document is that this statement refers to potential future work. Section 5.3.3 Potential issue with using ICMPv6 for signalling source address issues back to hosts is that uplink to an ISP-B failure immediately invalidates source addresses from 2001:db8:0:b000::/52 for all hosts which triggers a large number of ICMPv6 being sent back to hosts - the same scalability/rate limiting issues discussed in Section 5.2.3 would apply. nit: the grammar here is not great. Also, is the invalidation "for all hosts" just for use with external destinations? Section 5.5.2 In the absence of (S=ULA_Prefix; D=::/0) first-hop routers will send dedicated RAs from a unique link-local source LLA_ULA with PIO from ULA address space, RIO for the ULA prefix and Router Lifetime set to zero. [...] (This is still scoped to the "no external connectivity" case, right?) particularly useful if all ISPs assign prefixes via DHCP-PD. In the absence of ULAs uplinks failure hosts would lost all their GUAs upon prefix lifetime expiration which again makes intra-site communication impossible. nit: I think this is supposed to be "In the absence of ULAs, on uplink failure hosts would lose [...]" Section 5.6 [I stopped noting most grammar nits here] 1. no new (non-standard) functionality needs to be implemented on hosts (except for [RFC4191] support); RFC 4191 is from 2005; does it really still count as "new"? ;) To fully benefit from the RA-based solution, first-hop routers need to implement SADR and be able to send dedicated RAs per scoped It's not just the first-hop routers, though -- won't all the first-hops need to be part of the same connected SADR domain? Section 5.7.1 [RFC8106] defines IPv6 Router Advertisement options to allow IPv6 routers to advertise a list of DNS recursive server addresses and a DNS Search List to IPv6 hosts. Using RDNSS together with 'scoped' RAs as described above would allow a first-hop router (R3 in the Figure 3) to send DNS server addresses and search lists provided by each ISP (or the corporate DNS servers addresses if the enterprise is running its own DNS servers). I only skimmed RFC 8106, but it seems like this suffers from the same issue described above for linking PIO and RIO information (that inspired draft-pfister-6man-sadr-ra) -- we aren't guaranteed an information link between (source) address to use and DNS recursive to use. I do see a note in 8106 that requires this linkage when link-local addresses are used as DNS recursives, but not in the general case. While one might counter that this doesn't matter, since the DNS is a globally consistent database, in practice that proves to not be the case, with "walled gardens" being available only within a given ISP, etc., so it does seem like we could at least mention the potential for issues. And in fact we do have such discussion a couple paragraphs later, so maybe all we want is a hint that there's more to come. It should be noted that [RFC8106] explicitly prohibits using DNS information if the RA router Lifetime expired: "An RDNSS address or a DNSSL domain name MUST be used only as long as both the RA router Lifetime (advertised by a Router Advertisement message) and the corresponding option Lifetime have not expired.". Therefore hosts might ignore RDNSS information provided in ULA-scoped RAs as those RAs would have router lifetime set to 0. However the updated version of RFC6106 ([RFC8106]) has that requirement removed. It seems that the first reference here needs to be the old one, 6106, not 8106 as presently indicated. Section 9 Section 5.2.3 discusses a mechanism for controlling source address selection on hosts using ICMPv6 messages. It describes how an attacker could exploit this mechansim by sending spoofed ICMPv6 messages. It recommends that a given host verify the original packet header included into ICMPv6 error message was actually sent by the host itself. Section 5.2.3 also talks about a potential extension to ICMPv6 that would indicate what source address to use, in addition to noting that the selected source address does not work. Such an extension would also have some new security considerations, in that it would provide an attacker some measure of control over where the resulting traffic ended up, as (e.g.) might be useful in steering a DDoS.
(Mirja Kühlewind) (was Discuss) No Objection
Comment (2019-07-01 for -09)
Thanks for addressing my discuss as well as the TSV-ART review (thanks Michael for the review!) and adding section 6.7!
Barry Leiba No Objection
Alvaro Retana No Objection
Comment (2019-06-25 for -08)
Thank you for the work on this document!! I have several comments below -- some of them are substantive and I would like to see them addressed before publication. Substantive (A) It would be nice to add a short terminology section to at least include important terms every reader may not be familiar with: PA, PI, provider-aggregatable. Other terms that are used but not clearly described, which would benefit from a definition include "SADR domain" and source-prefix-scoped routing/destination/forwarding table...or more-specific-source-prefix-scoped forwarding table. Note that some of the examples help, but going through them is not the best way to find out the meaning. (B) §4: "if all routers in a given network support SADR and have a scoped forwarding table, then the unscoped forwarding table can be eliminated which ensures that packets with legitimate source addresses only can leave the network" This statement is true for traffic existing the network, but not in the general case where the unscoped table has to be used to deliver, for example, packets originated in the Internet to H32 (or any internal host). If the unscoped forwarding table is eliminated, then how are those packets routed? Am I missing something? (C) §5.2.2: "[RFC8028] recommends that a host SHOULD select default routers for each prefix in which it is assigned an address. It also recommends that hosts SHOULD implement Rule 5.5. of [RFC6724]." These SHOULDs are not Normative in this document, but come from rfc8028. I think there should either be a direct quotation or Normative language shouldn't be used. §5.6/§6 also mention Normative language from rfc8028 without properly quoting it. (D) The documents talks in several places about SADR support and how it is not necessary for all routers in the enterprise to support it. There are some mentions of this as examples are described... However, there is no clear guidance about the considerations for deploying a "SADR domain" in the network, and should be covered in the Deployment Considerations section. (E) I think that rfc4443, rfc4861, rfc4862, rfc6724 and rfc7078 should all be Normative references. (F) This point is for the WG Chairs/AD. I don't think changes to this document are needed, but will leave that up to the Chairs/AD. §4 says: Note that the mechanism described here for converting source-prefix- scoped destination prefix routing advertisements into forwarding state is somewhat different from that proposed in [I-D.ietf-rtgwg-dst-src-routing]. The method described in the current document is functionally equivalent, but it is intended to be easier to understand for enterprise network operators. This text makes me wonder about the relationship to I-D.ietf-rtgwg-dst-src-routing, which is currently marked as on the Standards Track (at least on the document itself), and makes no reference to this document (but it does point to rfc8043). Should the two be more closely related? This document "attempts to define a complete solution" for the multihoming problem, which is one of the use cases in I-D.ietf-rtgwg-dst-src-routing -- it seems like the answer could be Yes. The question might be more appropriate in the context of I-D.ietf-rtgwg-dst-src-routing, but I'm asking it now because of the explicit mention (above), and discussion in the WG around the compatibility of the two documents (for example in ). This makes me think that it is an important point to consider before publication of this document.  https://mailarchive.ietf.org/arch/msg/rtgwg/n2K1ZDD_Fco1CO7Oy4ZQ6MYJebg Editorial/Nits: (1) I think a title with "Solutions" (not "Solution") at the end would better reflect the contents. (2) §1: "Multihoming with provider-assigned addresses is typically less expensive for the enterprise relative to using provider-independent addresses." I assume you mean "less expensive" from the point of view of acquiring the addresses, right? However, operationally it may be more expensive because of the need to manage more moving parts, either NATs, or the mechanisms described in this document. It might be good to clarify. (3) s/Section 7 discussed other solutions/Section 7 discusses other solutions (4) s/router(SER)/router (SER) (5) §3.2: "using GRE for example" A reference would be nice. (6) §1: "...some routers must be capable of some form of Source Address Dependent Routing (SADR), if only as described in [RFC3704]." rfc3704 doesn't talk about SADR, at least not with that name. Maybe pointing to §4.3 could help. (7) §4: "...then add the entry to the more source-prefix-scoped forwarding table." The more what?? (8) s/As as an example/As an example (9) s/while for `2001:db8:0:b101::31/while for 2001:db8:0:b101::31 (10) s/H01/H101 (11) The first reference to rfc8415 is in §5.2.1. It would be nice to make it earlier, maybe when DHCPv6 is first mentioned. (12) "DHCPv6 support is not a mandatory requirement for IPv6 hosts, so this method might not work for all devices." A reference to rfc8504 might be nice. (13) Given that I-D.pfister-6man-sadr-ra was last updated in 2015, and that it "might need tweaking", I think this document shouldn't even mention it. (14) s/this is traffic is not following/this traffic is not following (15) s/An site administrator/A site administrator (16) The first reference to rfc4443 is in §5.2.3. It would be nice to make it earlier, maybe when ICMPv6 is first mentioned. (17) s/reach the a host on the Internet/reach a host on the Internet (18) [style nit/personal preference] Much of the text is written in first person ("In this document we assume that..."). I find the use of the third person ("In this document it is assumed that..." or "This document assumes...") more appropriate in IETF documents. Maybe just a manner of taste...
(Adam Roach) No Objection
Éric Vyncke No Objection
Comment (2019-06-27 for -08)
Thank you all for the work put into this document; this is an important network deployment use case. I second Alvaro's COMMENTs and I sincerely believe that a revised ID could fix a lot of small details in the text in order to improve the quality of the text (see my COMMENTs and NITs) Also, this is mostly a tutorial / guide. I also wonder why it is in RTGWG rather than V6OPS as part of the document is not about SADR but source address selection. But, this is really a detail. == COMMENTS == -- Abstract -- " a complete solution" while the document is not about a single solution but rather multiple of them. -- Section 1 -- Unsure whether "one of the goals of IPv6 is to eliminate the need for and the use of NAT or NAPT" is true. Even, if I would hate to use NAT with IPv6, but, this was probably not a design goal for IPv6. -- Section 1 -- Is there a reason why the issue of stateful device (firewall, ...) requiring to inspect all ingress/egress traffic is not mentioned in the list of issues? -- Section 3.5 -- While I agree that the scalability of the SADR solution puts some limit on the number of ISP, why does this document select the value 5? More generally, this section could probably be removed. -- Section 4 -- The "way to forward packets" does not read easily. Esp point 2), is it "selected forwarding table" or "selected forwarding table entries" (from point 1). Or should point 1 select a specific source-scoped forwarding table rather than forwarding table entries. -- Section 5.2.2 -- AFAIK, pfister-6man-sadr-ra is 'dead' since 2015. Is it still worth mentioning here ? -- Section 5.2.4 -- Why this section have "SHOULD" in uppercase while in the other section it is in lower case ? -- Section 5.6 -- Using RA for influencing source-address selection is probably not "the most reliable" as RA being multicast can be lost. -- Section 5.7.1 -- The DNS issue should also probably refer to RFC 7556 (PvD). -- Section 6 -- Mostly at the end of the document, there is a mention of PvD and of draft-ietf-intarea-provisioning-domains, possibly a little late in the flow. -- Section 7.3 -- There should be a reference to MPTCP or ICE. == NITS == -- Abstract -- I would suggest to add the word "IPv6" in the abstract as well for clarity. -- Section 1 -- Sorry but I cannot parse "...without stopping to provide ..." -- Section 3 --- The section title should be changed into "use cases" rather than "requirements" as it already describes part of the solution. -- Section 3.6 -- s/the aim of this draft/the aim of this document/ also in a couple of places. -- Section 4 -- Please expand ULA before first use (+ reference) -- Section 5 -- s/host internal to the multi-homed site/host inside the multi-homed site/ -- Section 5.1 -- Please put all rules explanation in a separate paragraphs (esp rules 1 & 2). -- Section 5.5.2 -- Please expand GUA before first use.
Magnus Westerlund No Objection
Comment (2019-06-27 for -08)
Regarding Section 7.3 and Section 1 paragraph: It may be desirable for an enterprise site to connect to multiple ISPs using provider-assigned (PA) addresses, instead of PI addresses. Multihoming with provider-assigned addresses is typically less expensive for the enterprise relative to using provider-independent addresses. PA multihoming is also a practice that should be facilitated and encouraged because it does not add to the size of the Internet routing table, whereas PI multihoming does. Note that PA is also used to mean "provider-aggregatable". In this document we assume that provider-assigned addresses are always provider- aggregatable. A possible addition here either in the above paragraph or at least in Section 7.3 that deploying enterprise PA based multi-homing solution actually benefits the usage of multi-path protocols as this ensures that the MP capable transport protocol get a well defined handle to something that likely lead to path diversity. So from my perspective, a working well enough PA based multi-homing solution benefits the deployment of multi-path protocols which in its turn makes the PA based multi-homing work even better than NATed or PI based ones.