On Consensus and Humming in the IETF
draft-resnick-on-consensus-07

I went through a private discussion between Dave Crocker and Pete Resnick, looked at the draft and the last call discussion. I advised Pete to take into account a number of specific aspects of the comments from Dave. Waiting for further action from Pete.

[Ballot comment]
This may be the most inconsistent ballot I've stated a position for since May, but ...

I'm a Yes. I think this doc is the right thing to do, and Pete has been doing the right things on revisions.

Having said that, I think Adrian's voluminous comments are helpful in many cases, and I think if they were all taken on board the diff would be bigger than some of the draft revisions Pete has made so far.

Do we need to talk about them? Especially since Adrian has already pointed out "Pete's view of consensus" in his ballot, so sending Pete away to address the comments after approving the document just seems odd ...

[Ballot comment]
This draft has already been very useful in several occasions.  Whether people follow the advice in the document or not, it should be strongly recommended reading for every IETF participant.

[Ballot comment]
  Now, a conclusion of having only rough consensus relies heavily on
  the good judgement of the consensus caller.  The group must truly
  consider and weigh an issue before the objection can be dismissed as
  being "in the rough".  ("In the rough" is terminology from golf.  The
  phrase, which gets used quite a bit in the IETF, is just a play on
  words.)

As a non native speaker, I had someone explained to me the  "to be in the rough" meaning.
And believe me, I searched the web. So, just telling it's a play on words without being precise on what it means is misleading, at the very minimum.
One might get it from "participant is not in the "rough" part of a rough consensus", but this document needs an explanation/definition within the IETF context.

-

  The chair in this case needs to understand what the responses mean;
  only sufficiently well-informed responses that justify the position
  taken can really "count".

Not sure I understand "well-informed".
I'm well-informed (because I'm the marketing guy for the feature I prefer), and my view is "I prefer A" is not what you mean.
You're after responses with
- well-argumented responses
or even better:
- well-argumented responses with new objections that have not addressed before.

"well-argumented" is better but I'm not totally happy with it ...

[Ballot comment]
I'm setting aside the issue of whether AD-sponsored is appropriate for
this document. It is somewhat ironic that this document about consensus
has debatable consensus on its content because it is basically Pete's
views.

But we need to have the wider debate about this issue and not pin it to
any one I-D in the pipe.

So here is my "No Objection".

---

Thanks to Pete for some long conversations about this document. It
leaves me with a list of issues that I think are basically editorial
and can be worked through. I trust Pete as author to make the necessary
changes, and I believe he and I understand the changes necessary.

---

It may help to draw out (further) the disctinction between issues and
people. That is, consensus is not just the agreement of people to 
continue on a specific path (e.g., publish this document whatever its
contents) but also requires that issues have been recognised and
handled.

As an example, if tourist who really doesn't care about the fate of the
work raises an issue. It is not enough for the tourist to say "I don't
care" - that might give consensus to move ahead, but the issue also
needs to be addressed.

---

I do feel that the document mixes (confuses?) the decision-making
process with the decision-evaluation mechanism. While the two are hard
to separate, I think that it would be very valuable to do so.

This is really a question of how we reach/drive consensus, how we
track and address issues, and how we listen to opinions.

I'm afraid I can't come up with a specific change to help with this.
but maybe it would help to talk about what chairs should be doing to
help the WG reach consensus.

---

The statement of intent for this document in the Abstract is altogether
too wishy-washy. Contrast with the final paragraph of the Introduction
that is much more clear (although see separate comments about it).

My main concern with the Abstract is that you have nothing here we can
build consensus around. Does "it is a collection of thoughts" mean that
the IETF has consensus that it is a collection of thoughts? I don't
think that is what you want - I think that, per your note (to be
removed) is much more helpful about describing the purpose which is
that "this document contains a collection of principles about rough
consensus and how it can and does operate within the IETF."

I also think that "this Infomational document does not change any IETF
processes" is important to retain in the Abstract.

---

Section 1

  Almost every IETF participant knows the aphorism from Dave Clark's
  1992 plenary presentation [Clark] regarding how we make decisions in
  the IETF:

I actually doubt this, and I bet you have no evidence for it. Maybe just
a simple re-wording such as "A well-known aphorism..."


---

Section 1

s/(kings and presidents)/(king or president)

---

Section 1

I would replace

  nor should decisions be made by a
  simple vote of the majority

with

  nor should decisions be made by a
  vote

since "simple vote" implies decisions can be made by complex vote,
and "of the majority" implies decisions can be made by some other
portion of the whole.

---

Section 1

  This document attempts

Maybe a bit more positive to strike "attempts"

  to explain some features of rough consensus,
  explain what is not rough consensus, discuss some new ways to think
  about rough consensus, and suggest ways that we might achieve rough
  consensus and judge it in the IETF.  Though this document describes
  some behaviors of working groups and chairs, it does so in broad
  brushstrokes and it does not proscribe specific procedures.  Rather,
  this document is intended to foster understanding of the underlying
  principles of IETF consensus processes.  While it may be of general
  interest to anyone interested in the IETF consensus processes, the
  primary audience for this document is expected to be those who have

Strike "expected to be"

  experience working in the IETF, and it is particularly aimed at
  generating thought and discussion among those who might lead a
  consensus discussion.  Although most of the examples in this document
  talk about working group chairs, these principles are meant to apply

Strike "are meant to"

  to any person who is trying to lead a group to rough consensus,
  whether a chair, a design team leader, a document editor, an IESG
  area director, or any community member who is facilitating a
  discussion or trying to assess consensus.


Just a specific quesiton on this: While you might be happy for anyone in
the IETF to read this document, it is not your intent to be specifically
providing guidance to IETF participants trying to participate in
consensus-building processes nor to understand how consensus might have
been judged?

---

In section 2 I found that

  We might have to compromise processor speed for lower
  power consumption, or compromise throughput for congestion
  resistance.  Those sorts of compromises are among engineering
  choices, and they are expected and essential.  We always want to be
  weighing tradeoffs and collectively choosing the best set.

Did not sit well.
I think it was "best set" that caused me trouble. "Best" has become the
enemy of "compromise" and I would recommend dodging that bullet by
cutting the last sentence after "tradeoffs".

---

In section 2 you say...
  the word "compromise" gets used in two
  different ways
...and then you list three ways

---

Section 3 final para

A technical error is always a valid basis for an appeal.

s/error/objection/

---

I don't think your use of "appease" is right in Section 2. And I don't
think capitulation is necessarily involved either. Both of those words
seem more in line with the "horse-trading" than the walking away.

How about saying

"walked away for whatever reason"

---

Section 3

  If the chair finds, in
  their technical judgement, that the issue has truly been considered,
  and that the vast majority of the working group has come to the
  conclusion that the tradeoff is worth making, even in the face of
  continued objection from the person(s) who raised the issue, the
  chair can declare that the group has come to rough consensus.

That'll be a vote, then :-)

Let me pretend to be an insufferable pedant for a moment. Is "vast
majority" more than 65%? More than 75%?

It isn't really a matter of the vastness of the majority. Rephrase?

---

I don't think that saying that "in the rough" comes from golf and is a
play on words helps people understand the meaning. How about...

  ("In the rough" is terminology from golf.  The rough is the longer
  grass at the side of the fairway, and if your ball has landed in the
  rough you are off course and away from the normal direction of play.
  The phrase gets used quite a bit in the IETF as a play on words to
  complement "rough consensus" meaning that you are in the rough if you
  find yourself not agreeing with the rough consensus.)

---

Section 3

  In order to do this, the chair will need to have a good idea
  of the purpose and architecture of the work being done, perhaps
  referring to the charter of the working group or a previously
  published requirements document, and use their own technical
  judgement to make sure that the solution meets those requirements.
  What can't happen is that the chair bases their decision solely on
  hearing a large number of voices simply saying, "The objection isn't
  valid."  That would simply be to take a vote.  A valid justification
  needs to me made.

Pete and I debated "technical judegment" quite a bit. We agreed (I
think) that sometimes this judgement extends to delegated judgement. We
also agreed that the appealability of that judgement is important to
highlight.

---

I snarf when I see "traditional".

  It is important to recognize that this view of rough consensus is a
  change from the way it has been traditionally characterized in the
  IETF.  Rough consensus in the IETF is often talked about as the
  "dominant view" of the group, as RFC 1603 [RFC1603] described:

Note that RFC 1603 and Dave Clark's "we reject voting" are almost
contemporaneous. Yes one says "ballot good" and the other appears to
say "ballot bad".

Can I suggest cutting the first sentence.

Furthermore, in...
  The above does say that "volume" is not the basis for a determination
  of consensus, but over the years many IETF folks have thought of
  rough consensus as being primarily about the percentage of people who
  agree with a decision.
...I think you have misinterpretted how 1603 meant "volume". I believe
it was talking about loudness. That is why "volume" is grouped with
"persistence". It is saying that rough consensus is not about how loudly
Adrian shouts, but about how the bulk of the WG feels. In other words,
it is closer to the percentage argument than you make out.

[Not that I am saying that a straight percentage is all that counts, and
neither are you in this section (notwithstanding the fact that you
talked about the "vast majority" in earlier text). But I am saying that
it is a factor.]

---

  One of the strengths of a consensus model is that
  minority views are addressed

I do struggle with "addressed". I think it has more of an implication
of accommodation than is intended. I think "properly considered" is a
better forumlation.

---

  The chair (or whoever is responsible to hear an appeal)...

It counds like you are saying that the chair hears the appeal, but you
haven't said that (and you would have been wrong). Why not...

  The chair in making the consensus call (or whoever is responsible to
  hear an appeal)...

---

I like section 5. It stands against some of the other text, especially
the first sentence of section 6 which jars quite badly coming
immediately after section 5.

The fix would be to try to reflect the message of section 5 into the
rest of the text.

---

The whole concept of humming applies, AFAICS, only to physical meetings.

The document could use consideration of humming on mailing lists and
how that kind of temperature guage can be used so that we don't have to
wait four months to move a discussion forward.

Curiously, Stewart just raised this exact question and a couple of ADs
made suggestions about how they have handled this.

---

Section 10 : something broken in

>  [Clark]    Clark, D., "A Cloudy Crystal Ball - Visions of the
>            Future", July 1992.
>
>            In

---

The RFC Editor will object to you including references without citing
them from your text. [IETF24] seems to be missing.

---

I appreciate you referencing Sheeran, but I note that there is no over-
riding intent in the IETF to reach consensus. Maybe that is a serious
lack in the IETF. But, of course, we typically care more about our jobs
and welfare than about perfection of consensus and the welfare of the
Internet :-)

Maybe you also need to discuss timeliness of decisions in this context.
Should the IETF strive for consensus (rough or otherwise) in the face of
immediate need for a solution. Some might argue that our process is
already slow enough (partly because of consensus). Others might argue
that quick and good are not usually compatible.

In discussing this point, Pete and I concluded that it may be helpful to
note that an acceptable way of moving forward with unresolved issues is
to note them in the RFC and have a plan to revist and resolve.

[Ballot comment]

- I think the note below the abstract should be kept,
somewhere.

- s/factious/fractious/g ?

- s/proscribe/prescribe/

- section 2 title: s/disagreement/valid technical
disagreement/ would be better, if longer.

- s2: "Can anyone not live with..." That can also
be problematic since its hard to do multi-valued
(cf httpbis at IETF-88). I think it only really
works well for binary choices and then
quickly gets counterprodutive. Bit wary of saying
its just ok.

- s2: saying horse-trading "has no place" in
consensus decision making seems a bit idealistic

- s3: "file an appeal" maybe add a reference there
to whatever says "start with a maili to wg chair, 
then..." in case this encourages folks to go
straight to the IETF chair or something.

- s4: s/nearly impossible/impossible/ ? in para 1,
ignoring IESG etc. might be better

[Ballot comment]
-- Section 1 --

  Though this document describes
  some behaviors of working groups and chairs, it does so in broad
  brushstrokes and it does not proscribe specific procedures.

While it may be true that this document "does not [prohibit] specific procedures," I don't think that's what you mean here.  I think "prescribe" is the word you meant.

IESG/Authors/WG Chairs:

IANA has reviewed draft-resnick-on-consensus-05, which is currently in Last Call, and has the following comments:

We note that this document does not contain a standard IANA Considerations section. After examining the draft, IANA understands that, upon approval of this document, there are no IANA Actions that need completion.

If this assessment is not accurate, please respond as soon as possible.

The following Last Call announcement was sent out:

From: The IESG
To: IETF-Announce
Reply-To: ietf@ietf.org
Sender:
Subject: Last Call:  (On Consensus and Humming in the IETF) to Informational RFC


The IESG has received a request from an individual submitter to consider
the following document:
- 'On Consensus and Humming in the IETF'
  as Informational RFC

The IESG plans to make a decision in the next few weeks, and solicits
final comments on this action. Please send substantive comments to the
ietf@ietf.org mailing lists by 2013-11-04. Exceptionally, comments may be
sent to iesg@ietf.org instead. In either case, please retain the
beginning of the Subject line to allow automated sorting.

Abstract


  The IETF has had a long tradition of doing its technical work through
  a consensus process, taking into account the different views among
  IETF participants and coming to (at least rough) consensus on
  technical matters.  In particular, the IETF is supposed not to be run
  by a "majority rule" philosophy.  This is why we engage in rituals
  like "humming" instead of voting.  However, more and more of our
  actions are now indistinguishable from voting, and quite often we are
  letting the majority win the day, without consideration of minority
  concerns.  This document is a collection of thoughts on what rough
  consensus is, how we have gotten away from it, and the things we can
  do in order to really achieve rough consensus.

      Note (to be removed before publication): This document is quite
      consciously being put forward as Informational.  It does not
      propose to change any IETF processes and is therefore not a BCP.
      It is simply a collection of principles, hopefully around which
      the IETF can come to (at least rough) consensus.




The file can be obtained via
http://datatracker.ietf.org/doc/draft-resnick-on-consensus/

IESG discussion can be tracked via
http://datatracker.ietf.org/doc/draft-resnick-on-consensus/ballot/


No IPR declarations have been submitted directly on this I-D.

Here are my AD review comments:

This is an excellent document and I'd love to take it forward to become a more permanent record. With this in mind, I did an AD review on it.

If we take this forward, the note under the abstract needs to go away. Maybe also tone down the "also in the leadership" comment in Section 1, third paragraph.

In Section 2, the A vs. B example is great. However, from reading the second paragraph, I get a feeling that the discussion starts from the point of view wanting to make A the final solution. In many WG situations there are some alternatives, and at the beginning, there is no clear idea that we want to go with a particular choice. Hence the chairs probe the group on what direction seems most useful, and at some point, of course, they are convinced enough that they start determining whether the primary choice (A in this case) is acceptable to everyone. But I think the draft would benefit from pointing out that there are earlier phases of the discussion, and usually, some of the tradeoffs between the choices already get understood at that stage, even from a consensus point of view.

In the last paragraph of Section 3, it says "A good outcome is for the objector to be satisfied that, although their issue is not being accommodated in the final product, they understand and accept the outcome." Right. The rest of the section deals with appeals, but I think it would be fair to mention the (rather common, unfortunately) situation where the person's issue has been understood and recognised as something that we should be able to live with, but the person himself or herself continues to believe it is of utmost importance ("really, we need to use host byte order"). Perhaps the paragraph could end with a note that says something about appeals determining if the issue was satisfactorily addressed/understood, and that this, not the belief of the complainer, determines whether the appeal has weight.

The document should somewhere mention the need to ensure that a sufficient portion of the community (including implementors and other important stakeholders) have been included in the discussion, and not just tuned out.

  A smaller bunch of loud hums for choice A and a larger
  number of non-committal hums for choice B might indicate that some
  people believe that there are serious problems with choice B, albeit
  the more popular by sheer number of people.

(This is my only real complaint with your document.)

I'm not so sure you can draw these inferences. This is so culturally and personally affected issue, that it is difficult to draw conclusions. Not quite sure what to do about it, however. Further discussion inline:

The chair could always be
  surprised because the hum turns out to be unanimous.

This is true

Otherwise, the
  hum begins the discussion, it doesn't end it.

Right.

Of course, a chair
  could get the temperature of the room by doing a show of hands too,
  and knowing who specifically feels one way or another can help a good
  chair guide the subsequent conversation.  However, a show of hands
  will often leave the impression that the number of people matters in
  some formal way.  It takes a chair and a working group with a solid
  understanding of how consensus works to do a show of hands and not
  end up reinforcing the mistaken notion that a vote is taking place.

Agreed. I wonder if we could balance this by saying something about the difficulties in measuring hums due to cultural and personal variation.