What a functioning AI team actually looks like

There is a specific failure mode we see often enough to have a name for it: the pilot plateau. A company runs six AI pilots in a year. Four produce a demo that impresses the leadership team. One reaches a small group of real users. None of them are still running eighteen months later.

The pilots are rarely the problem. The demos usually worked. What is missing is an operating model — a standing answer to who owns this, who is on call for it, and how it gets better next quarter. Below is what that looks like at the companies that get past this stage.

Four roles that have to exist

These are roles, not headcount. At a small company one person may hold two of them. What does not work is when a role is held by nobody.

• A product owner who can say no. Someone accountable for whether this use case is worth continuing, with the authority to kill it. Without this you accumulate zombie pilots that nobody will shut down because nobody owns them.
• An engineer who owns the system in production. Not a data scientist who hands off a notebook — someone who carries the pager, owns the deploy pipeline, and treats the AI feature like any other service.
• A domain expert with real hours committed. The person who actually does the work being automated, budgeted for several hours a week, not borrowed occasionally. They write the eval cases and they arbitrate what “correct” means. This is the role most often left implicit and it is the one that most reliably sinks projects.
• Someone accountable for evaluation. Owns the eval set, the graders, and the release gate. Frequently the same person as the engineer, but the responsibility has to be named or it quietly becomes nobody's.

Three rituals worth the calendar time

Teams that keep improving share a small set of recurring habits. They are unglamorous and they are the whole game.

1. 01A weekly failure review. Thirty minutes, the domain expert in the room, looking at ten real outputs the system got wrong. Not metrics — actual traces. Every serious team we have worked with does some version of this, and none of them started doing it early enough.
2. 02A release gate tied to evals. A change ships when it clears the eval suite, with the slice-level scores visible. This turns launches from a debate into a check.
3. 03A quarterly cost and latency review. Token spend and p95 latency drift upward silently as prompts grow and context expands. Reviewed quarterly, it is an adjustment; discovered by a finance escalation, it is a crisis.

Draw the ownership boundary early

The most consequential decision is often organisational rather than technical: does the AI system belong to a central platform team or to the product team that uses it?

The pattern that holds up is a split. A central team owns the shared substrate — gateway and model routing, evaluation tooling, tracing and observability, guardrails, cost controls, the golden-path template for a new use case. Product teams own their own use cases end to end: the prompts, the eval set, the domain logic, and the pager.

The failure mode on either extreme is predictable. A fully central team becomes a queue that everyone routes around. Fully devolved teams rebuild the same tracing and eval plumbing five times and none of it is good.

What to do on Monday

If you are on the plateau and want off it, the first move is not another pilot. Take the one use case with the clearest business value and give it the full apparatus: a named owner, an engineer on the pager, a domain expert with committed hours, an eval set, and a weekly failure review. Run it for a quarter.

One use case operated properly teaches your organisation more than six pilots run casually — and it leaves behind the tooling and the habits that make the second one substantially cheaper.