Pairwise voting allows you to rank ideas by asking a defined set of participants to complete one-to-one comparisons. It is a great way to include a broader set of participants in the evaluation process.
Configuring the pairwise stage
When setting up or editing a Challenge, there is a new stage type called ‘Pairwise’. It is only available when creating a Phased Challenge.
The settings for this stage are:
- Closing date
- Option for idea creators to edit their ideas during the pairwise stage
- Who can see the challenge ideas: everyone or Admins, Managers and participants only
- Participants: everyone, specific groups or individuals.
Progression options: manual, automatically progress all ideas, or automatically progress ideas ranking the highest when closing the stage.
When the Pairwise stage starts, eligible participants receive a task reminding them to compare an allocated number of pairs. The allocated number of pairs is generated automatically by the platform, based on the number of ideas and participants.
When accessing the Challenge page, participants are automatically prompted to complete the comparisons. For each pair, there are 3 possible decisions: a preference for the idea on the left, the idea on the right or no preference.
The task is considered complete when all assigned pairs have been compared. If there are more available pairs, the participants can continue to compare ideas until all pairs have been compared.
Pairwise tab for Admins and Challenge Managers
A pairwise tab is available for Challenges that include a pairwise stage.
Metrics: engaging a broad audience is key to a successful pairwise stage. To allow you to measure the participation level, the pairwise tab includes metrics on the number of comparisons, the number of participants and their level of participation.
Results table: ranks the ideas based on the results of the pairwise comparisons. Once generated, click ‘refresh’ to update the results. Because the results might vary depending on the captured data, the 'confidence level' indicates how sure the ranking is. We recommend reaching a 'high' level of confidence before making decisions to progress or reject ideas.
Progressing ideas: From the pairwise tab, Admins and Challenge Managers can select the ideas and make progression decisions.
Automatic progression options
When creating or editing the Challenge, Admins and Challenge Managers have the option to turn on automatic progression. The stage can indeed be configured so that only the ideas ranking the highest progress when closing the stage.
If a tie falls within the limit defined for automatic progression, none of these ideas will progress and a manual decision is required.