LangGraph, CrewAI, Mastra, plain function-calling. The decision tree we use to pick orchestration, and the cost of picking wrong.
Orchestration frameworks are a fashion industry. LangGraph this quarter, CrewAI last quarter, Mastra next quarter. Most teams ship the framework before they understand whether they needed one. Below: the decision tree we use, and the cost of picking wrong. What we would do differently next time Looking back at the last six deployments in this category, three things we would do differently: Start the eval harness on day zero.
We have always said this and we have always slipped it. The first time we shipped without a regression eval, we caught a prompt change that silently degraded conversion by 11 percent for two weeks before anyone noticed. Now we treat the eval harness as the first deliverable, before the first prompt. Get the executive sponsor in the first user-acceptance session. Not the project manager, not the ops lead.
LangGraph, CrewAI, Mastra, plain function-calling. The decision tree we use to pick orchestration, and the cost of picking wrong. It is filed under AI Infrastructure because that is where operators looking for this problem actually start, and it is written from production work rather than from a content calendar.