AI framework vs custom stack: the layer you own

6 min readSimon BudziakBy Simon Budziak

On this page6 sections
Title card reading Own the layer, not the side. A navy background with a stylized stack of four horizontal slabs outlined in thin gold, three of them filled with a soft cream glow and one slab in the middle lifted clear of the stack and held slightly apart, showing that a single layer can be pulled out and replaced without disturbing the others.

Teams still frame this as a fork in the road: adopt an AI agent framework and move fast, or build custom orchestration and own everything. That framing is what produces the regret, because the real question is which layer you keep the right to replace, and the answer is almost never all of them or none of them. A framework you can swap a piece of costs far less to be wrong about than one you have wired through every module. This post covers what the failure numbers do and do not say, why the binary softened during 2026, and the four questions we actually ask before picking.

What the failure numbers actually say

Two figures get quoted in every version of this argument, and both are usually stretched.

The current one is narrower than its headlines suggest. Gartner’s April 2026 release states that “Only 28% of AI use cases in infrastructure and operations (I&O) fully succeed and meet ROI expectations, while 20% fail outright”, from a survey of 782 I&O leaders conducted in November and December 2025. That is infrastructure and operations use cases, not all AI projects, and the secondary coverage routinely drops the scope.

The older one is real but aging. Gartner’s May 2024 release found that “on average, only 48% of AI projects make it into production, and it takes 8 months to go from AI prototype to production”, from a survey conducted in the fourth quarter of 2023. Useful, and now describing a world two and a half years back.

The “more than 80% of AI projects fail” line is the one to handle carefully. It traces to RAND’s 2024 report, which says “By some estimates, more than 80 percent of AI projects fail”. By some estimates is RAND hedging someone else’s number, not RAND measuring one. What RAND did measure is more useful anyway: they interviewed 65 experienced data scientists and engineers and found five root causes of failure, the first being that stakeholders misunderstand or miscommunicate what problem needs solving. Not one of the five is a framework choice.

That is the honest starting point. Picking the wrong library is not what is killing these projects.

The binary got softer during 2026

When this post first ran in January 2026, “framework or custom” was a reasonable way to describe the fork. It reads differently now, because the frameworks stopped being all-or-nothing.

The clearest example is the one we wrote up separately: adding a human approval gate used to mean hand-writing the graph around it, and LangChain now ships that gate as configurable middleware, so you declare a policy instead of building a control-plane. The same shape shows up across the ecosystem. You increasingly adopt a framework and replace one layer inside it, rather than adopting or rejecting the whole thing.

A four-row diagram of a typical AI application stack on a navy card, ordered by how expensive each layer is to replace. Orchestration and control flow sits at the top, outlined in gold and marked most expensive. Data and retrieval and then interface follow, both marked moderate. Model and provider sits at the bottom, marked cheapest if it is bounded behind one adapter you own.
The question is not which framework. It is which of these rows you would be willing to rewrite in a hurry, and which you would not.

The regret tax is a migration cost, so price it before you commit

The expensive mistake is not choosing a framework. It is choosing one and then spreading it through the codebase so thoroughly that changing it means a rewrite. We have paid this: a chatbot build where the framework picked at week one turned out to be the wrong fit, and by the time that was obvious, rewiring the chains, callbacks and memory cost more than living with the choice.

Surprised Pikachu meme. Top caption: called the framework from every module. Bottom caption, over Pikachu's shocked face: replacing it means a rewrite.
The regret tax is not picking the wrong framework. It is picking it everywhere.

The useful discipline is to ask the migration question at the start, when it is cheap:

  • If we replaced this in six months, what would we touch? If the answer is “every module”, the dependency is not a library, it is the architecture.
  • What is behind a boundary we own? A model provider behind one adapter is a config change. A model provider called directly from forty files is a project.
  • Which layer is most likely to change? Model providers change constantly, orchestration rarely, data and retrieval somewhere between. Spend the abstraction budget where the churn is.

Frameworks do not remove production complexity. They postpone it, and they decide who is holding it when it arrives.

What actually forces a custom stack

In regulated work the decision is usually made for you, and it is made by the data rather than the architecture. Strict data boundaries, self-hosting requirements, tight latency budgets and compliance-driven audit trails all push toward owning the control-plane, because the constraint is where the data may go, not how the agent loops.

That is a genuinely different reason from “the framework felt heavy”. If your constraint is a data-residency rule, no amount of framework ergonomics resolves it. If your constraint is that the agent loop got hard to debug, a framework with explicit control flow probably does.

Worth naming a small correction while we are here: LangGraph is often described as bringing DAGs to agents, including in the January version of this post. It is not a DAG. Its graph API supports sequences, branches and loops, and the cycles are the point, since that is what an agent doing more than one pass actually needs.

The four questions we ask

Not the download counts. We publish a ranking of the most used Python agent frameworks and it is a popularity signal, not a decision rule. What we actually ask:

  1. What is the hard constraint? Compliance, residency, latency, or none of those. This answers build versus buy faster than any feature comparison.
  2. Which layer will churn? Put the abstraction there and nowhere else.
  3. Can the team debug it at 3am? A framework nobody on the team understands is a custom stack with worse documentation.
  4. What do we still own regardless? The evals, the human gates and the cost. We covered that in deep agents in production: the framework never takes those off you.

Start with the highest-level tool that does not compromise a hard constraint, keep one boundary around whatever is most likely to move, and spend the saved effort on the things the RAND interviews actually flagged: knowing which problem you are solving, and being able to tell whether it is working.

Sources

Last updated:

How we build on LangChain in production