Regression eval, adversarial eval, and shadow-traffic eval. The three eval categories every shipped agent should run on every prompt change.
Every production agent should pass three eval types on every prompt change: regression, adversarial, and shadow-traffic. The teams that ship reliably have all three. The stack we actually use Below is the production stack we ship for this category. This one ships in 72 hours and survives a real-world Monday morning. Pipecat for voice, a thin Express service for chat, deployed on Fly.io or Render.
This one ships in 72 hours and survives a real-world Monday morning. Pipecat for voice, a thin Express service for chat, deployed on Fly.io or Render. We avoid Vercel for anything stateful. Default to Claude 3.7 Haiku for the chat layer and Claude 3.7 Sonnet for the planning and tool-use steps. Fall back to GPT-4.1-mini when Anthropic has capacity pressure.
Regression eval, adversarial eval, and shadow-traffic eval. The three eval categories every shipped agent should run on every prompt change. It is filed under AI Infrastructure because that is where operators looking for this problem actually start, and it is written from production work rather than from a content calendar.