Qualitative consensus

This article describes a process tentatively called qualitative consensus, as opposed to numerical or quantitative consensus. It is meant to be suitable for yes-or-no motions or change proposals on wikis such as Wikipedia. The process is called qualitative since not only quantity but also the quality of the votes cast matters. One inspiration for the name is the distinction between qualitative identity and numerical identity, although it does not bear a perfect analogy. The main process does not describe seeking consensus via content editing with edit summaries, a vital part of any wiki; that is briefly treated in a dedicated section only.

Process description

Entity types involved include the following: yes-or-no motion or change proposal, arguments in favor of the proposal, arguments against the proposal, human participants, votes cast by the participants, textual comments placed by the participants next to their votes, and responses on different nesting level.

There are the following requirements on the process:

R1: Votes that fail to identify any argument in their support shall not count. One may either state the argument or point to the place where the argument is located.
Two competing requirements:
- R2a: The number of votes shall matter, not only the quality of the arguments identified. Thus, it should make a difference whether an argument is supported by 3 people or 20 people, although from the argument analysis perspective, it is still a single argument.
- R2b: The number of votes supporting an argument shall not matter; each argument counts as one as long as it is strong.
R3: Votes that identify low-quality arguments only can be dismissed. However, this can be difficult to determine objectively, and is therefore difficult to administer well. In case of doubt, this requirement may need to be dropped.
Two competing requirements:
- R4a: The strength of the argument shall be assessed in reference to established policies; therefore, policy override is not granted.
- R4b: The strength of the argument shall be assessed in reference to established policies but with an override granted for relatively rare cases. The allowance of override does not mean the policies should be ignored as if they did not exist. If policies get overriden very often, something is wrong.
- R4c: The strength of the argument shall be assessed in general, and thus, the process can take on the character of original policy making to some extent.
R5: A single vote/request for comments can host multiple alternative yes-or-no proposals, each treated as a separate vote for evaluation purposes. If multiple competing proposals pass, the determination which is the ultimate winner is left unspecified; in case of doubt, a follow-up vote can be set up to unequivocally pick the estimated winner. Examples include Wiktionary:Wiktionary:Votes/pl-2017-03/Desysopping for inactivity and Wiktionary:Wiktionary:Votes/pl-2022-09/Meaning of consensus for discussions other than formal votes created at Wiktionary:Votes.

We propose that R2b is unworkable since it de facto disenfranchises individual people as decision makers whose will carries weight, and leaves the determination of the strength of the arguments to the single process closer. Leaving it to process closer creates a vulnerability, a single point of failure. Wikipedia does not seem to adopt R2b in its actual practice, but it seems to preach it by stating that "The quality of an argument is more important than whether it represents a minority or a majority view."

We further propose the default threshold for consensus to be 2/3 (66.6%) of supports to supports plus opposes (abstains do not count), among those votes that met their argumentation duty and whose votes were not dismissed. The basis for the threshold is unclear; it appears in the rules for constitution change of multiple countries, while some countries have 60% and the U.K. seems to have de facto 50% for whatever semi-formal constitution it may have. What seems clear is that 50%, the plain majority, would create too low a protection against error in judgment and random effects depending on who came to the discussion, but some may differ. Too high a threshold for a pass would give too much power to dissenting voices to block decisions. In Wikipedia, "anyone" can edit, and furthermore, people hardly ever agree unanimously on anything.

The threshold that is for matters of taste can be 60% or perhaps even 55%. Matters of taste cannot be reduced to facts only.

A discussion seeking to find consensus, which may be called request for comments or even a vote, shall be closed by someone as passed or not passed. There shall be a possibility of other people to challenge the closure.

Principled policy override

When policy override is allowed, one may require the following:

R6: The overriding vote has to put forward a putative principle guiding the override that one could wish to be made part of the policy. The principle does not need to be perfect and have all the details ironed out, but it should not be grossly inadequate either.

Discussion before the vote

Whether the main vote-combined-with-discussion process is called a vote or a request for comments, it is advisable to allow some time, say 7 days, before the start of the process for a preliminary discussion of the text of the proposal and argument and counterargument discovery. During that time, no one should be taking a boldfaced position in favor of the motion, or if they do, it should not count into a tally. The idea is that an initial position taken before the preliminary argument discovery cannot be well enough informed since it disregards what those who have not spoken yet have to say. This prelimiary argument discovery can be referred to as 1st phase, a preliminary one.

In a pure process separating the discussion from the vote, the vote would not allow extended discussion; instead, each vote cast would be accompanied by a short summary of reasoning, and there would be no to-and-fro interactions. However, in the wiki environment, that is impractical. Discussion should be able to develop also during the 2nd phase, the vote or request for comments itself. It has the downside that those who came early in the 2nd phase may disregard what those who come later in that phase have to say. That is a disadvantage that probably has to be tolerated.

What follows is a look at how these things work in the English Wiktionary and Wikipedia.

In the English Wiktionary, there is an institution called vote. It is codified as pure numerical consensus since 2019, at 2/3 supports to supports plus opposes threshold for pass. It is required that there be enough time for discussion before the vote start, usually at least 7 days. The discussion can take place in the public discussion place Beer parlour, an analogue of Wikipedia's Village pump, or on the vote talk page. The discussion makes it possible to amend the proposal, thereby making it better or easier to pass. It may help explore the alternatives as well.

A similar principle could be applied to Wikipedia's requests for comments. The two-stage process of discussion + vote/request for comments has some merits: first, arguments are hashed out and alternatives explored, and people only start to vote when the argument collection process has reached some level of maturity. However, experience shows that people often start to participate in a serious way only after the vote starts. It is therefore not reasonable to expect that the argument collection processes finishes before the vote starts.

Since participants in Wikipedia's requests for comments start taking discrete positions from the beginning, the requirement to allow some discussion before the first quasi-votes are cast seems to make sense for Wikipedia as well.

Discussion venue

In the context of wikis, at least three discussion venues come to mind:

General policy discussion forum such as Wikipedia's Village pump or Wiktionary Beer parlour.
The talk page of the vote/request for comments.
The vote/request for comments page itself.

The talk page of the vote/request for comment has the advantage that it has copious space for subsections, and can host the phase 1 discussion.

Closure process

A vote or a request for comments is closed by a single person based on their evaluation of the votes cast. The closure shall be a closure proposal, not a final closure. After the closure proposal, there shall be some time for other editors to post objections, reservations or questions. It is only after a period of objections and answers to these objections that the consensus can be considered to be finally determined. In particular, the editor posting the closure proposal shall not seal the vote, that is, shall not prevent objections, reservations and questions from being posted.

Vote eligibility

One can consider competing vote eligibility requirements:

R7a: Any user is eligible to vote, including anonymous IP users.
R7b: Only registered user accounts are eligible; anonymous IP users are not. Anonymous IP users are welcome to post productive comments, but these shall carry no weight in tallying.
R7c: Only registered user accounts are eligible; the account must have been created a certain number of months before the start of the vote and must have made a certain number of content edits before the start of the vote. Anonymous IP users are welcome to post productive comments, but these shall carry no weight in tallying.

We propose R7c as most meaningful, assuming R2a.

Futher reading:

Wikt: Wiktionary:Voting policy

Process exit/escape

There needs to be a way to apply exceptions to the process for extraordinary circumstances. One can imagine an attack on a wiki by a host of pseudonymous editors each making enough edits to meet the voting eligibility requirements, each providing something like non-empty rationale for their votes, trying to cause damage to the project by having damaging proposals passed. Long-term established editors and admins need to be able to dismiss vote results based on extraordinaty circumstances; this should happen extremely rarely, but needs to be a possibility.

Sealing the vote

In a wiki with its revision histories, there is no vital need to seal the vote/request for comments in a box and prevent it from being edited any longer. For instance, a link to a discussion may need to be updated when the discussion is moved to an archive. Moreover, especially if the outcome was indecisive, it does no harm when people add their votes after the time of the closure as long as their being late is clearly marked; their votes are part of the larger process of collection of positions and arguments, and can serve as input into a follow-up vote/request for comments.

Consensus via edits and their summaries

Aspects of qualitative consensus can be brought into dispute resolution via edit summaries, with no separate discussion.

Principles:

A debate via edit summaries can develop on a page where an edit is reverted, which is then counter-reverted.
Each revert should state a specific reason for reverting. When the revert does not specify a reason, we have numerical consensus seeking, not qualitative one.
Each repeated revert should try to bring some new material into the revert edit summary.
This process should probably stop at 3 revert limit per day and page.
Too many to-and-fro reverts suggest the discussion should continue elsewhere, such as on the article talk page or in the general project discussion forum.
If only part of an edit can be revert it, one should ideally try to do so, salvaging parts the questioned edit. However, it may be too much work compared to a simple wholesale revert.

Definitions of consensus in literature

This section is here for reference.

Some definitions of consensus from dictionaries:

Webster 1913 public domain definition is "Agreement; accord; consent."
M-W sense 1 a is unanimity and this is Macmillan sense 1, the only sense Macmillan has and the sole meaning Oxford Learner's Dictionaries have and the sole meaning in Britannica Dictionary.
M-W sense 1 b is weaker, speaking of agreement by "most concerned".
AHD has two definitions, not wholly clear whether refering to M-W 1 a or 1 b; Collins has a single definition, not clear whether M-W 1 a or 1 b; ditto for Cambridge Advanced Learner's Dictionary.

Some definitions of consensus found online:

What is Consensus Decision Making?, lucidmeetings.com
"Consensus is a decision-making approach that seeks to secure the support of the whole group for the decision at hand. Many people believe that consensus is the same thing as unanimous agreement, but this is not necessarily the case. Unanimity is when everyone agrees. Consensus is when no one disagrees."

Note: There is no way this could work in a wiki; there is no chance we will argue so long until everyone agrees or at least abstains.

The distinction between "everyone agrees" and "no one disagrees" is not very clear.
What is Consensus?, creativemanitoba.ca
"Consensus is a decision-making process that works creatively to include all persons making the decision. Instead of simply voting for an item, and having the majority of the group getting their way, the group is committed to finding solutions that everyone can live with. This ensures that everyone’s opinions, ideas, and reservations are taken into account. There is an agreement not to move forward in a direction or take action unless all parties agree."

Note: No chance we could make all opposers abstain. Not realistic.
What does it mean when a decision is taken “by consensus”?, ask.un.org
"When a vote is taken and all Member States vote the same way, the decision is unanimous. When a decision is taken by consensus, no formal vote is taken. A 2005 Legal Opinion distinguishes consensus as follows: consensus “is understood as the absence of objection rather than a particular majority” (UN Juridical Yearbook 2005, page 457). Resolutions and decisions adopted by consensus are considered as “adopted without a vote”, although they are distinct from decisions made under the without-a-vote procedure."

Note: No chance we could make all opposers abstain. Not realistic.
What is consensus?, quora.com
"An outcome that all members of a group can live with." --Flemming Funch

Note: No chance we could make all opposers abstain. Not realistic.
What is consensus?, webopedia.com
"In blockchain technology, the consensus mechanism formalizes an agreement in which the majority, or at least 51%, of network nodes agree that a particular digital transaction was executed and completed in the blockchain network."

Note: Very unconventional, but here it is, a plain majority.
What Is Consensus?, consensusclassroom.org
'For the purposes of this site, we have agreed on this definition: consensus is conscious agreement by everyone. Under this definition, a decision is not final until everyone in the group agrees with it. We call the interaction that leads to consensus--or at least attempts to find consensus--the “consensus process.”'

Note: No chance we could make all opposers abstain. Not realistic.