IMAP Support for UTF-8
draft-ietf-eai-5738bis-12

[Ballot comment]
Please double-check that this document doesn't need the "This document may contain material from IETF Documents or IETF Contributions published or made publicly available before November 10, 2008." boilerplate. RFC5738 did have that. Did the text that caused that document to need it get removed?

[Ballot comment]
I took tripped over the same sentence Stephen did.  Please consider the following changes proposed by Barry:

OLD
  Because such messages are really variations on the original ones, not
  really "downgraded ones" (although that terminology is often used for
  convenience), they inevitably have relationships to the original ones
  that the IMAP specification [RFC3501] did not anticipate.

NEW
[no change; just including that sentence here for context]

OLD
  In particular, digital signatures computed over the original message
  will often not be applicable to the variant version and servers that
  may be accessed by the same user with different clients or methods
  (e.g., POP or webmail systems in addition to IMAP or IMAP clients
  with different capabilities) will need to exert extreme care to be
  sure that UIDVALIDITY behaves as the user would expect.

NEW
  This brings up two concerns in particular:

  First, digital signatures computed over and intended for the original
  message will often not be applicable to the variant message, and
  will often fail signature verification.  (It will be possible for some
  digital signatures to be verified, if they cover only parts of the
  original message that are not affected in the creation of the variant.)

  [no change to the next; I've just split it out as its own sentence.]
  Second, servers that
  may be accessed by the same user with different clients or methods
  (e.g., POP or webmail systems in addition to IMAP or IMAP clients
  with different capabilities) will need to exert extreme care to be
  sure that UIDVALIDITY behaves as the user would expect.

[Ballot comment]
Barry tells me the working group considered and rejected marking this
document as "updates RFC 3501".  I'd like the working group to
reconsider that decision, considering this text from the IANA
considerations:

  This document redefines two capabilities ("UTF8=ACCEPT" and
  "UTF8=ONLY") in the IMAP 4 Capabilities registry [RFC3501].  Three
  other capabilities that were described in the experimental
  predecessor to this document (UTF8=ALL, UTF8=APPEND, UTF8=USER) are
  now made OBSOLETE.

It seems to me anyone implementing IMAP from scratch or updating an
existing implementation should read this document as part of that
implementation.

[Ballot comment]
Section 4

  IMAP servers that advertise support for "UTF8=ACCEPT" or "UTF8=ONLY"
  MUST reject an APPEND command that includes any 8-bit in the message
  headers with a "NO" response, when IMAP clients do not issue "ENABLE
  UTF8=ACCEPT" or "ENABLE UTF8=ONLY".

This looks like errata 2075 has not been applied.
http://www.rfc-editor.org/errata_search.php?rfc=5738

---

When a new document obsoletes an old one, it is really nice to include
a short section stating the changes. I think it is unfortunate that this
document doesn't include such a section.

[Ballot comment]


I checked the diff with 5738 but its not that useful since there
are a lot of changes. Apologies in advance if I comment on
something that's unchanged from there and feel free to ignore
such comments.

- section 1: "Most of this specification" is an odd phrase. It'd
be nicer if you could be more precise, or maybe just better to
s/Most of this/This/

- p4, 3rd last para says that the "server MUST NOT send UTF-8 in
quoted strings to the client unless the client has indicated
support using the "ENABLE UTF8=ACCEPT" command." Earlier you
said that the "UTF8=ONLY" capability implies this, so I guess
that this MUST NOT also doesn't apply in that case. But is that
clear enough?  Maybe s/using the "ENABLE UTF8=ACCEPT"
command/using the "ENABLE UTF8=ACCEPT" command or "UTF8=ONLY"
capability/ would be better?

- section 7 says that signatures might "not be applicable" to
the variant version, which reads oddly to me. I think it'd be
better to say that signatures may not longer be verifiable
with the variant message.

(The next two questions maybe apply to all four documents in
this set, but I'll ask 'em here anyway.)

- I have a security question, the answer to which isn't clear to
me, but maybe you can help. Is there any situation where a
mailbox name or a user id might contain non-ascii characters and
where the IMAP server will treat those as equivalent to some
mapping to ascii? For example if joerg can use an umlaut or not,
but has only set up one of the two, is there ever a threat that
I could get access to joerg's mail by setting up an account with
the one he's not using? I guess I'm wondering if it might be
worth adding a warning about that kind of mapping. I don't know
if that belongs here or not, or is already somewhere else, since
its really like the general problem of "cousin" domains in
phishing (paypa1 etc), but I guess some discussion somewhere
would be good.

- Can EAI make it more likely that a sender would encrypt a
message for the wrong recipient? If so, is that worth noting?
Not sure where'd be right for that, but I wondered.

[Ballot comment]
No objection to the publication of this document.
However, please fix the following issue.
http://tools.ietf.org/html/draft-ietf-eai-5738bis-09#section-9. "The reference making them obsolete would be good to include", quoting my discussion with Michelle Cotton

[Ballot discuss]

  5721bis is clear that support for downgrade is optional.  As I read
  5721bis, it says that one can downgrade, and if one does so, it must
  downgrade according to Section 7 of this document.  However, there
  is no normative language in Section 7 of this document.

  This document informatively references the downgrade drafts as
  examples, and the downgrade drafts are targeting the standards track.

  What specification MUST be followed to implement downgrade?

[Ballot comment]
I really didn't work on this version of the document, but they seemed determined to leave my name on it, so I shall Recuse.

[Ballot comment]
I really didn't work on this version of the document, but they seemed determined to leave my name on it, so I shall Abstain.

[Ballot comment]
For the IESG, I'll note the following from the ballot text, in case anyone misses it.  All four documents will be on the 27 Sept telechat.

[Please note: This document is one a set of four interdependent
documents:

draft-ietf-eai-5738bis
draft-ietf-eai-popimap-downgrade
draft-ietf-eai-rfc5721bis
draft-ietf-eai-simpledowngrade

These documents should be reviewed, evaluated, and understood
together.]

IANA has reviewed draft-ietf-eai-5738bis-09 and has the following comments:

IANA understands that, upon approval of this document, there is a single
IANA action which must be completed.

In the IMAP 4 Capabilities registry located at:

http://www.iana.org/assignments/imap4-capabilities/imap4-capabilities.xml

There are five registrations which begin with the characters UTF8. 
They are:

+--------------+---------------------------+
| UTF8=ACCEPT  |  [RFC5738]                |
| UTF8=ALL    |  [RFC5738]                |
| UTF8=APPEND  |  [RFC5738]                |
| UTF8=ONLY    |  [RFC5738]                |
| UTF8=USER    |  [RFC5738]                |
+--------------+---------------------------+

these five registration should be changed to the following:

+--------------+---------------------------+
| UTF8=ACCEPT  |  [[this RFC]]            |
| UTF8=ALL    |  OBSOLETE (was [RFC5738]) |
| UTF8=APPEND  |  OBSOLETE (was [RFC5738]) |
| UTF8=ONLY    |  [[this RFC]]            |
| UTF8=USER    |  OBSOLETE (was [RFC5738]) |
+--------------+---------------------------+

IANA understands that this is the only action which needs to be completed
upon approval of this document.

Note:  The actions requested in this document will not be completed
until the document has been approved for publication as an RFC.

The following Last Call announcement was sent out:

From: The IESG
To: IETF-Announce
CC:
Reply-To: ietf@ietf.org
Subject: Last Call:  (IMAP Support for UTF-8) to Proposed Standard


The IESG has received a request from the Email Address
Internationalization WG (eai) to consider the following document:
- 'IMAP Support for UTF-8'
  as Proposed Standard

Please note: This document is one a set of four interdependent
documents:

draft-ietf-eai-5738bis
draft-ietf-eai-popimap-downgrade
draft-ietf-eai-rfc5721bis
draft-ietf-eai-simpledowngrade

These documents should be reviewed, evaluated, and understood
together.

The IESG plans to make a decision in the next few weeks, and solicits
final comments on this action. Please send substantive comments to the
ietf@ietf.org mailing lists by 2012-09-20. Exceptionally, comments may be
sent to iesg@ietf.org instead. In either case, please retain the
beginning of the Subject line to allow automated sorting.

Abstract


  This specification extends the Internet Message Access Protocol
  version 4rev1 (IMAP4rev1) to support UTF-8 encoded international
  characters in user names, mail addresses and message headers.  This
  specification replaces RFC 5738.




The file can be obtained via
http://datatracker.ietf.org/doc/draft-ietf-eai-5738bis/

IESG discussion can be tracked via
http://datatracker.ietf.org/doc/draft-ietf-eai-5738bis/ballot/


No IPR declarations have been submitted directly on this I-D.

    Document Shepherd Writeup - draft-ietf-eai-5738bis-07 (EAI-IMAP)
      Date: 2012-08-22
      Shepherd: John C Klensin, john-ietf@jck.com

1. What type of RFC is being requested (BCP, Proposed Standard, Internet
  Standard, Informational, Experimental, or Historic)?  Why is this the
  proper type of RFC?  Is this type of RFC indicated in the title page
  header?

  Proposed Standard.  This set of documents are all protocol
  specifications, following up earlier Experimental treatment of POP3
  and IMAP access to messages with internationalized envelopes and/or
  header fields.


2. The IESG approval announcement includes a Document Announcement
  Write-Up.  Please provide such a Document Announcement Write-Up.
  Recent examples can be found in the "Action" announcements for
  approved documents.  The approval announcement contains the following
  sections:

  Technical Summary
      Relevant content can frequently be found in the abstract and/or
      introduction of the document.  If not, this may be an indication
      that there are deficiencies in the abstract or introduction.

      These documents make up a set of four that are interdependent and
      should be reviewed, evaluated, and understood together.  Their
      abstracts have been examined and verified to sufficiency to
      describe the individual documents.

      The abstract for this particular document reads:

        This specification extends the Internet Message Access Protocol
        version 4rev1 (IMAP4rev1) to support UTF-8 encoded
        international characters in user names, mail addresses and
        message headers.  This specification replaces RFC 5738.

  Working Group Summary
      Was there anything in WG process that is worth noting?  For
      example, was there controversy about particular points or were
      there decisions where the consensus was particularly rough?

      Short answer: No.  Longer answer: The WG had extensive and
      constructive discussions about the role of "downgrading" (e.g.,
      converting a message stored on the server that contains non-ASCII
      header or envelope information) in the transition to an all-i18n
      environment.  Some of those issues and tradeoffs are discussed in
      draft-ietf-eai-popimap-downgrade and
      draft-ietf-eai-simpledowngrade.  In some cases, the best strategy
      may be to "hide" those messages that cannot be delivered without
      change to legacy clients either with or without some attempt at an
      error message.  A complete treatment of those options is
      impossible because the optimal strategies will depend considerably
      on local circumstances.  Consequently the base IMAP and POP3
      documents are no longer dependent on particular downgrading
      choices and that two methods presented are, to a considerable
      extent, just examples.  They are recommended as alternative
      Standards Track documents because they are protocol specifications
      and their sometimes-subtle details have have been carefully worked
      out, even though the WG has no general recommendation to make
      between them (or other strategies).

      While opinions differ in the WG about which downgrading mechanisms
      are likely to see the most use, if any, consensus is strong that
      these four documents represent the correct output.

  Document Quality
      Are there existing implementations of the protocol?  Have a
      significant number of vendors indicated their plan to implement
      the specification?  Are there any reviewers that merit special
      mention as having done a thorough review, e.g., one that resulted
      in important changes or a conclusion that the document had no
      substantive issues?  If there was a MIB Doctor, Media Type or
      other expert review, what was its course (briefly)?  In the case
      of a Media Type review, on what date was the request posted?

      Some development and interoperability testing has occurred and is
      progressing.  There are strong commitments in various countries to
      implement and deploy the EAI (more properly, SMTPUTF8) messages
      and functions specified in RFCs 6530 through 6533.  Those messages
      will be inaccessible to many users without POP3 and IMAP support,
      so these specifications are quite likely to be implemented and
      deployed in a timely fashion.

      Reviewers who made particular contributions prior to IETF Last
      Call are acknowledged in the documents.  See Section 3 for
      additional information.

  Personnel
      Who is the Document Shepherd?  Who is the Responsible Area
      Director?

      Document Shepherd:  John C Klensin
      Responsible Area Director:  Pete Resnick

        Note that Pete Resnick is listed as a co-author on this
        document as a result of contributions well before he became AD
        (and primarily to its the Experimental predecessor).  He has not
        been actively involved in an author or editor role since
        joining the IESG.

3. Briefly describe the review of this document that was performed by
  the Document Shepherd.  If this version of the document is not ready
  for publication, please explain why the document is being forwarded
  to the IESG.

  The document shepherd, with assistance from the other co-chair, did
  extensive, line by line and paragraph by paragraph reviews during the
  WG LC window with the intention of identifying and eliminating as
  many issues that might otherwise be spotted during IETF review as
  possible.  Those reviews were posted to the WG mailing list; the
  documents being submitted include changes made on that basis.


4. Does the document Shepherd have any concerns about the depth or
  breadth of the reviews that have been performed?

  This particular document shepherd has almost always been concerned
  about breadth and quality of reviews in the IETF.  HOwever, the co-
  chairs have identified areas of expertise and perspective needed for
  reviews of the specifications in these documents and their
  relationship to the widely-deployed and well-tested IMAP and POP3
  specifications and are confident that the reviews are adequate.

  Although considerable improvements have been made in readability and
  editorial and technical quality, the base IMAP
  (draft-ietf-eai-5738bis) and POP (draft-ietf-eai-rfc5721bis)
  documents represent an orderly and uncontroversial evolution from
  their Experimental predecessors.

  It is probably worth pointing out, as draft-ietf-eai-5738bis does,
  that the transition to general adoption of SMTPUTF8 mail will not be
  an easy one in many environments.  In the case of the transport and
  mail header specifications of RFCs 6530ff, the model that permits a
  sender to test whether the potential receiver can handle the message
  is clear, as is an orderly response if it is not (even if that
  response may not be completely satisfactory from the user's
  standpoint.  That same relationship does not apply to these
  specifications because, for many environments, a POP3 or IMAP server
  must be prepared to deal with clients who do not have the needed
  capabilities and there is no completely satisfactory way with either
  protocol to either tell a client that it cannot access a message that
  is known to be waiting nor to deliver an intact version of the
  message (where "intact" includes, e.g., being able to pass signature
  verification on body parts and/or headers).


5. Do portions of the document need review from a particular or from
  broader perspective, e.g., security, operational complexity, AAA,
  DNS, DHCP, XML, or internationalization?  If so, describe the review
  that took place.

  It is our strong belief that all such issues and perspectives have
  been addressed by the WG and reviews already obtained.


6. Describe any specific concerns or issues that the Document Shepherd
  has with this document that the Responsible Area Director and/or the
  IESG should be aware of?  For example, perhaps he or she is
  uncomfortable with certain parts of the document, or has concerns
  whether there really is a need for it.  In any event, if the WG has
  discussed those issues and has indicated that it still wishes to
  advance the document, detail those concerns here.

  All such substantive issues have been identified and resolved within
  the WG and incorporated into the documents.  I do expect that IETF
  Last Call will turn up demands to solve problems that the WG has
  concluded are impossible (some discussed above) but it is unlikely
  that any will be a significant surprise.

  The WG has strong consensus that some interoperability difficulties
  can be anticipated during the period in which these protocols are
  deployed.  Those difficulties are inevitable in the absence of an
  effective flag day (possible for POP and IMAP in some installations
  but not generally).  The alternative is to not deploy changes of this
  sort at all, but the adoption of RFCs 6530-6533 eliminated that
  possibility.  To the extent possible, the transitional issues have
  been removed from this document, discussed briefly in this document
  and treated at more length in the two "downgrade" documents.  That
  strategy should most easily permit the transitional issues and
  protocols to be put aside once the base IMAP and POP two protocols
  (and the internationalization of email envelope and header field
  protocols more generally) are widely deployed.

  The WG has an outstanding question about the status of this document
  (and draft-ietf-eai-rfc5721bis) relative to whether or not they
  update the base IMAP and POP specifications.  The WG's conclusion is
  that they do not -- these specifications are extensions, not required
  changes to the base specifications -- and the documents for IETF Last
  Call were produced on that basis.  The WG also notes that SMTP
  extensions and mail header field additions have generally not been
  identified as updating the base email specifications.  However, the
  topic raises a more general issue that we believe the IESG should
  address.  If the IESG concludes that all such extension documents
  should be listed as updating the corresponding base specifications,
  the WG has no objection to these documents being modified
  accordingly.


7. Has each author confirmed that any and all appropriate IPR
  disclosures required for full conformance with the provisions of BCP
  78 and BCP 79 have already been filed.  If not, explain why.

  Authors of all four documents have been queried to verify that they
  have examined BCP 78 and 79 and are in compliance with them.  The
  three authors who have not yet replied are expected to be in
  Vancouver and acknowledgments will be extracted from them there.


8. Has an IPR disclosure been filed that references this document?  If
  so, summarize any WG discussion and conclusion regarding the IPR
  disclosures.

  No IPR disclosures have been filed on any of the four documents.


9. How solid is the WG consensus behind this document?  Does it
  represent the strong concurrence of a few individuals, with others
  being silent, or does the WG as a whole understand and agree with it?

  The WG's meeting participants and mailing list have included a rather
  large proportion of people who are anxious to see well-defined
  standards in this area agreed upon and deployed, but who behave as if
  they have little interest or expertise in the details of the
  technology (some of them are probably correct about the latter).  Of
  those who have participated technically and more actively, the
  consensus that these documents are ready to go seems rather solid.
  In particular, multiple inquiries and WG Last Calls have not turned
  up any significant controversy or unresolved issues.


10. Has anyone threatened an appeal or otherwise indicated extreme
  discontent?  If so, please summarise the areas of conflict in
  separate email messages to the Responsible Area Director.  (It should
  be in a separate email because this questionnaire is publicly
  available.)

  No


11. Identify any ID nits the Document Shepherd has found in this
  document.  (See http://www.ietf.org/tools/idnits/ and the Internet-
  Drafts Checklist).  Boilerplate checks are not enough; this check
  needs to be thorough.

  o  The "Obsoletes" line in the header contains the string "RFC", not
      just the numbers.  This should be corrected when a version is
      produced subsequent to IETF Last Call or is easily fixed by the
      RFC Editor.  This document shepherd fails to understand why issues
      of this sort should take up the time of the WG, document shepherd,
      or IESG.

  o  The nits checker complains that the abstract indicates that this
      document obsoletes RFC 5738 but the document header does not
      indicate this.  In fact, the relationship is noted in the headers,
      but obscured for the nits checker by the presence of "RFC".  This
      is believed to be a bug in the nits checker.

  o  For consistency with the other documents in the set (see
      explanation in the Shepherd's report for
      draft-ietf-eai-rfc5721bis), the sentence and the principle that
      abstracts should contain only important information, the sentence
      "This specification replaces RFC 5738." should be removed from the
      Abstract when the document is updated after IETF Last Call.

  o  The document contains an abbreviated version of the RFC 2119 text
      that lists only terms actually used.  If the IESG or RFC Editor
      prefer, that text can be changed to the more complete (but more
      misleading) version when the document is updated after IETF Last
      Call.

  o  The document references draft-ietf-eai-simpledowngrade-05 but the
      current correct version is -07.  Because
      draft-ietf-eai-simpledowngrade is being put through IETF Last Call
      concurrent with this document and the reference will be replaced
      by the RFC Editor with an RFC number, the discrepancy is
      considered trivial.


12. Describe how the document meets any required formal review criteria,
  such as the MIB Doctor, media type, and URI type reviews.

  No such review criterial apply to any of these four documents.


13. Have all references within this document been identified as either
  normative or informative?

  Yes


14. Are there normative references to documents that are not ready for
  advancement or are otherwise in an unclear state?  If such normative
  references exist, what is the plan for their completion?

  There are no normative references to unpublished documents.


15. Are there downward normative references references (see RFC 3967)?

  These documents are intended for Proposed Standard.  There are no
  normative references to Experimental or Informational documents.


16. Will publication of this document change the status of any existing
  RFCs?  Are those RFCs listed on the title page header, listed in the
  abstract, and discussed in the introduction?  If the RFCs are not
  listed in the Abstract and Introduction, explain why, and point to
  the part of the document where the relationship of this document to
  the other RFCs is discussed.  If this information is not in the
  document, explain why the WG considers it unnecessary.

  See discussion of nits checking above.  RFC 5738 is obsoleted by this
  document.  That status is shown in the header (albeit with the error
  mentioned above) and mentioned in the Introduction.


17. Describe the Document Shepherd's review of the IANA considerations
  section, especially with regard to its consistency with the body of
  the document.  Confirm that all protocol extensions that the document
  makes are associated with the appropriate reservations in IANA
  registries.

  Confirm that any referenced IANA registries have been clearly
  identified.  Confirm that newly created IANA registries include a
  detailed specification of the initial contents for the registry, that
  allocations procedures for future registrations are defined, and a
  reasonable name for the new registry has been suggested (see RFC
  5226).

  ???


18. List any new IANA registries that require Expert Review for future
  allocations.  Provide any public guidance that the IESG would find
  useful in selecting the IANA Experts for these new registries.

  No new IANA registries are created by any of this set of documents.


19. Describe reviews and automated checks performed by the Document
  Shepherd to validate sections of the document written in a formal
  language, such as XML code, BNF rules, MIB definitions, etc.

  The very small amount of ABNF in this document has been hand-checked.
  Automated checks did not seem to be either necessary or appropriate.