CSAT, AHT, NPS. Ignore them all for the first 90 days. Deflection rate is the only number that tells you whether the chat agent is paying rent.
Every chat agent vendor will show you a dashboard with eight metrics. CSAT, AHT, NPS, sentiment, escalation rate, response time, abandonment, resolution time. Ignore all of them for the first 90 days. The only number that tells you whether the chat agent is paying rent is deflection rate, and most vendors define it wrong. The failure modes we have learned to engineer around Five failure modes show up in this category over and over.
A prompt that worked in week one starts producing off-brand replies by week four because the model behind it silently versioned. Fix: pin the model version, run a weekly voice-drift eval against 50 canonical scenarios, alert on a 3-point deviation. The KB updated, the embeddings did not. The agent confidently quotes last quarter's pricing. Fix: a freshness-check job that compares KB modified timestamps against embedding job runs hourly, and a hard ceiling that prevents serving any answer grounded in a document older than the freshness window.
CSAT, AHT, NPS. Ignore them all for the first 90 days. Deflection rate is the only number that tells you whether the chat agent is paying rent. It is filed under AI Strategy because that is where operators looking for this problem actually start, and it is written from production work rather than from a content calendar.