Split your CSAT by who handled the conversation

Time to set up

One day

Cost

£0

Driver

Insights

The problem

One CSAT score can rise while the conversations AI handles score lower. Routine questions get faster answers, which lifts the average, and the blended number hides the rest. The middle group, where AI hands over to a human, is often where satisfaction drops, because customers have to start again. Zendesk’s 2026 research found 74% of consumers are frustrated when they have to repeat information.

The fix

1. Tag every conversation by who handled it: AI only, AI to human, or human only. Most helpdesks can do this with a rule on whether an agent replied.
2. Survey all three groups the same way. Same question, same timing, same scale.
3. Split by contact reason too (see Ten tags that tell you why customers call), so you compare like with like.
4. Report the three scores side by side every month.
5. Read the middle group’s low scores first. Check whether the agent had the full conversation history when they picked it up.
6. Set a floor. Agree a CSAT level for AI-only conversations below which that contact reason goes back to people, and write it down.

Tools to try

• Your helpdesk’s built-in survey (Zendesk, Intercom, Freshdesk), sent after every conversation type.
• A tagging or routing rule that marks each conversation AI only, AI to human, or human only.
• Looker Studio or Power BI to show the three scores side by side by contact reason.

How to measure it

Track the CSAT gap between AI-only and human-only conversations, by contact reason, and CSAT for the AI-to-human group over time. If the middle group improves after you fix the handover, that’s the clearest sign the change worked.

Why this matters: The CSAT drop nobody puts in the business case