Use case: How to set up Roles in the Evaluation process

For effective prioritization, it is necessary to consider criteria from the business and technology perspectives. For this purpose, each participant evaluates tasks from their role.

You can learn more about assigning criteria to users with their role-specific vision.

But there is more than just evaluating issues with your criteria. It’s important to support this process consistently.

In this article, you will learn about the responsibility of each specific user role in the evaluation process.

Users and responsibilities

For a successful and transparent process, it’s better to assign responsibilities among users.

Board Team leader

The user who oversees the entire board prioritization process.

• Approves the final board configurations (criteria, estimators, backlog issues, prioritization deadlines).
• Can edit everything (criteria, estimators, others’ scores, final scores). Everyone involved in the backlog evaluation process trusts this person’s final word. Everyone on the team can see the modifications to the evaluation criteria or final scores. 

Tip: This user can have a special criterion like “Objective Urgency” to prioritize tasks above all others.

Examples of actions taken by a team leader:

• They analyze the proposed criteria and, if necessary, add or remove them.

Find information on how to create criteria here.

• Assign estimators evaluation criteria according to their expertise and experience.

Find information about how to assign estimators evaluation criteria.

• Determines deadlines for which the team should have finished evaluation and task prioritization.  

Facilitator

A user at the intersection of business and technology. They can understand the problem from various angles. They may not be part of the team – a scrum master. There can be two facilitators, one for business criteria and the other for technical criteria.

• They are responsible for identifying missing information or alternative solutions to tasks.
• Assists in resolving the team’s misalignment in the evaluation and finding consensus among estimators. They can overwrite the final score of any colleague’s evaluation.

Example of actions taken by a facilitator:

• Moderates the discussion of areas of misalignment in evaluation.
• Asks questions to estimators to better understand their perspectives and arguments. 
• Propose alternative approaches to solving a task if it can help achieve consensus.

Read how to set up the Facilitator role.

Criteria manager

This user is responsible for managing the organization’s criteria. Constantly checks the relevance of criteria to make sure they match the company’s goal.

• Can refine and rewrite the rating scale, description, and criterion’s weight, as needed.
• In the Global Criteria Management section, they can “orchestrate” criteria across all boards. Modify descriptions, rating scales, and weights.

Example of actions:

• Monitors the relevance of the criterion description across teams.

Read how to edit criteria. 

• Initiates discussions with estimators to clarify their opinions based on the evaluation criteria.
• Adjusts criteria’s rating scales when necessary for a more accurate and relevant evaluation.

Read more about mass criteria management.

Estimator

A user is assigned as a criterion estimator based on the principle: “Who would you ask this question about the task in a live discussion?”

• Evaluates tasks based on their criterion, relying on the rating scale and their own experience.

• If necessary, they can request additional information or clarifications about a task they are evaluating.

You can read more about asking questions during evaluation here.

Example of actions taken by an Estimator:

• Studies the task description and evaluates it based on the criteria they are assigned, using the provided rating scale. 

 Read about the evaluation process here.

• If there is insufficient information about the task being evaluated, they can request clarifications from the author or other teammates.

You can read more about asking questions during evaluation here.

• Participates in discussions about tasks where there are team misalignments and provides arguments for their position.

You can read about the Alignment report.

Evaluation Process With Users’ Responsibilities

Let’s examine the step-by-step evaluation process, taking into account the users’ roles and responsibilities mentioned above.

1. The Board team leader assigns estimators evaluation criteria and decides the prioritization deadlines.
2. Estimators assess backlog for prioritization according to the criteria they are assigned, using the provided rating scales.
3. In case of conflicts among estimators or the need for clarification of task information, the Facilitator assists in finding consensus and clarifying all issues.
4. The criteria manager monitor the relevance and timeliness of the criteria, making adjustments to rating scales if necessary.
5. After completing the evaluation of all tasks, the Board team leader reviews the results and approves the final prioritization.

Useful Tips

1. Start by evaluating 10–20 tasks using the proposed model to gather feedback and adjust criteria and processes if necessary.
2. Discuss the results of the first iteration and the mechanics of further work on the next call.
3. Make changes to the prioritization process after discussion: change criteria (description, weight, scale, estimators).
4. Resolve the Questions across the tasks and remove/add/edit tasks from the backlog to prioritize.

Go through the process several times until your team finds the formula that suits the company’s goal.