Evaluation Criteria

Evaluation Criteria are the set of rules Qualiteam uses to assess conversation quality. They describe what to check in a dialogue and how to score it. You can create your own criteria to match your internal standards, or start with the built-in templates and customize them.
Qualiteam includes starter evaluation criteria by default. You can use them as-is, edit them to fit your workflow, or delete them at any time.

Add a new criterion

• Go to Evaluation Criteria and click New Criteria

• In Criteria Name, enter a clear name for the criterion (e.g., “Problem Resolution”)

• Set the Maximum Score (the highest score an agent can receive for this criterion)

• In Detailed Instructions, describe exactly how the criterion should be evaluated — include scoring guidance (e.g., what earns full score, partial score, or zero)

• Click Save Changes

Delete a criterion

• Go to Evaluation Criteria — you’ll see the full list of criteria

• Click the trash icon next to the criterion you want to remove

💡Useful tips: For the most consistent results, start simple with 5–6 criteria and expand only once the evaluations look stable. A clear 0–2 scale is usually the easiest to calibrate (0 = poor, 1 = average, 2 = good); if you choose a higher maximum score, make sure you define what “partial credit” means. When writing criteria, focus on observable behaviors (what the agent says or does) rather than abstract goals, and don’t be afraid to be detailed — the more specific and thorough your instructions are, the more reliable the evaluation will be across different cases. Add 1–2 examples of what earns a high vs low score, avoid vague wording, and iterate by testing your criteria in Conversation Playground before applying them at scale.

      **Flows (step-by-step rules with branching)**

Evaluation Flows let you define process rules for specific request types — as a step-by-step decision tree that depends on what the customer says and what the agent should do next. This is especially useful when you have standardised workflows (e.g., payments, refunds, verification, and account access) and want to ensure agents follow the correct path in each scenario. Flows complement Evaluation Criteria: criteria cover overall communication quality, while flows validate that the right steps were taken in the right order.

How to create a Flow

1. Open Evaluation Flows and click Create New Flow. Give the flow a clear name and short description (e.g., “Payment issue”).

2. Build the flow using nodes. Start from the Start node and add the next steps from Add Nodes:

   • Classification — define how to categorize the request (e.g., “Deposit issue” vs “Withdrawal issue”).

   • Action Guidance — describe what the agent should do or ask at this step (specific instructions, required questions, checks, or responses).

   • Validation — define what must be true for the step to be considered correctly handled (what the agent should have done/said).

3. Add branching options in Classification nodes and connect them to the next steps, so the flow reflects real customer paths (different answers → different actions).

4. Use Preview to test the flow logic and make sure the branches and instructions behave as expected.

5. Mark the flow as Active when it’s ready to be used for evaluation.

💡Useful tips: Start with one high-impact scenario (one flow, 5–10 nodes), test it on real conversations, and only then expand to additional branches and edge cases.